- Published on
vLLM의 PagedAttention 아키텍처와 Ollama의 로컬 LLM 실행 환경을 심층 비교한다. 설치부터 서버 구동, API 호출, 주요 CLI 인자, 샘플링 파라미터, 환경변수, 양자화(AWQ/GPTQ/GGUF), 멀티 GPU 구성, Docker 배포, 성능 튜닝까지 — LLM 서빙에 필요한 모든 설정을 실전 예제와 함께 총정리한다.
Quantization
All Posts
- deep-learning (21)
- hadoop (19)
- kubernetes (16)
- hbase (15)
- ai-papers (13)
- linux (13)
- llm (12)
- devops (12)
- gpu (11)
- java (8)
- transformer (6)
- cuda (6)
- mlops (5)
- kerberos (5)
- docker (5)
- nvidia (5)
- cloud-native (5)
- retrospect (4)
- terminal (4)
- security (4)
- spring (4)
- ai (4)
- productivity (4)
- cli (4)
- diffusion (4)
- generative-ai (4)
- agent (4)
- image-generation (3)
- optimization (3)
- hive (3)
- command (3)
- zookeeper (3)
- yarn (3)
- observability (3)
- monitoring (3)
- ai-coding (3)
- open-source (3)
- vllm (3)
- nlp (2)
- pre-training (2)
- fine-tuning (2)
- google (2)
- generative-model (2)
- stable-diffusion (2)
- openai (2)
- rlhf (2)
- langchain (2)
- computer-vision (2)
- image-classification (2)
- ml-pipeline (2)
- database (2)
- ansible (2)
- python (2)
- sql (2)
- pytorch (2)
- parallel-computing (2)
- podman (2)
- container (2)
- performance (2)
- infrastructure (2)
- automation (2)
- distributed-training (2)
- hdfs (2)
- cncf (2)
- gpu-operator (2)
- networking (2)
- gateway-api (2)
- langgraph (2)
- tmux (2)
- vim (2)
- neovim (2)
- text-to-video (2)
- image-to-video (2)
- video-generation (2)
- inference (2)
- production (2)
- model-serving (2)
- mlflow (2)
- attention (1)
- bert (1)
- masked-language-model (1)
- ddpm (1)
- diffusion-model (1)
- score-matching (1)
- latent-diffusion (1)
- flash-attention (1)
- gan (1)
- adversarial-training (1)
- dcgan (1)
- stylegan (1)
- wasserstein (1)
- gpt (1)
- language-model (1)
- instruction-tuning (1)
- scaling-law (1)
- lora (1)
- peft (1)
- rag (1)
- vector-database (1)
- reinforcement-learning (1)
- dqn (1)
- ppo (1)
- a3c (1)
- q-learning (1)
- policy-gradient (1)
- alphago (1)
- resnet (1)
- residual-learning (1)
- cnn (1)
- skip-connection (1)
- batch-normalization (1)
- vit (1)
- vision-transformer (1)
- patch-embedding (1)
- self-attention (1)
- airflow (1)
- orchestration (1)
- mongodb (1)
- kdc (1)
- kafka (1)
- pheonix (1)
- phoenix (1)
- profiler (1)
- crontab (1)
- mapreduce (1)
- api (1)
- grpc (1)
- scala (1)
- javascript (1)
- raspberry (1)
- mysql (1)
- spark (1)
- ubuntu (1)
- jvm (1)
- ycsb (1)
- machine-learning (1)
- hpc (1)
- oci (1)
- dockerfile (1)
- containerfile (1)
- buildah (1)
- skopeo (1)
- fluent-bit (1)
- logging (1)
- fluentd (1)
- elasticsearch (1)
- prometheus (1)
- keycloak (1)
- iam (1)
- sso (1)
- oauth2 (1)
- oidc (1)
- authentication (1)
- authorization (1)
- nginx (1)
- reverse-proxy (1)
- load-balancing (1)
- ssl (1)
- tls (1)
- web-server (1)
- caching (1)
- rate-limiting (1)
- compression (1)
- opentelemetry (1)
- microservices (1)
- distributed-tracing (1)
- otlp (1)
- sampling (1)
- terraform (1)
- iac (1)
- infrastructure-as-code (1)
- configuration-management (1)
- cloud (1)
- aws (1)
- provisioning (1)
- english (1)
- business-english (1)
- it-company (1)
- communication (1)
- career (1)
- email (1)
- meeting (1)
- slack (1)
- code-review (1)
- interview (1)
- incident-response (1)
- agile (1)
- mixed-precision (1)
- japanese (1)
- jlpt (1)
- n5 (1)
- n4 (1)
- n3 (1)
- business-japanese (1)
- vocabulary (1)
- kanji (1)
- hiragana (1)
- katakana (1)
- joby-aviation (1)
- evtol (1)
- 항공택시 (1)
- 주식 (1)
- 투자 (1)
- 사우디-투자 (1)
- 기술 (1)
- volcano (1)
- kubeflow (1)
- cnpe (1)
- platform-engineering (1)
- gitops (1)
- argocd (1)
- crossplane (1)
- backstage (1)
- kubevirt (1)
- virtualization (1)
- vgpu (1)
- device-plugin (1)
- mig (1)
- ingress-nginx (1)
- http3 (1)
- quic (1)
- migration (1)
- nftables (1)
- ipvs (1)
- istio (1)
- ambient-mesh (1)
- cilium (1)
- ebpf (1)
- cni (1)
- service-mesh (1)
- kube-proxy (1)
- kubestronaut (1)
- cka (1)
- ckad (1)
- cks (1)
- kcna (1)
- kcsa (1)
- certification (1)
- rootless (1)
- daemonless (1)
- pasta-networking (1)
- lcel (1)
- nvidia-driver (1)
- multiplexer (1)
- screen (1)
- editor (1)
- ide (1)
- dotfiles (1)
- developer-experience (1)
- workflow (1)
- ace-step (1)
- music-generation (1)
- ai-music (1)
- audio-generation (1)
- text-to-music (1)
- antigravity (1)
- agentic-ide (1)
- gemini-3 (1)
- claude-code (1)
- anthropic (1)
- developer-tools (1)
- vscode (1)
- cursor (1)
- deepseek (1)
- moe (1)
- mla (1)
- deepseek-v3 (1)
- deepseek-r1 (1)
- reasoning (1)
- hunyuan-video (1)
- ltx-video (1)
- ltx2 (1)
- diffusion-transformer (1)
- dit (1)
- tencent (1)
- lightricks (1)
- tool-use (1)
- react (1)
- tensorrt-llm (1)
- multimodal (1)
- omni-model (1)
- any-to-any (1)
- gpt-4o (1)
- gemini (1)
- codex (1)
- code-generation (1)
- github-copilot (1)
- openclaw (1)
- ai-agent (1)
- mcp (1)
- self-hosted (1)
- architecture (1)
- guardrails (1)
- sglang (1)
- llm-inference (1)
- radix-attention (1)
- serving (1)
- structured-generation (1)
- disaggregation (1)
- text-to-image (1)
- training (1)
- flow-matching (1)
- ollama (1)
- llm-serving (1)
quantization (1)
- openai-api (1)
- wan (1)
- z-image-turbo (1)
- alibaba (1)
- kserve (1)
- triton (1)
- experiment-tracking (1)
- model-registry (1)
- redis (1)
- hbm (1)
- semiconductor (1)
- memory (1)
- ai-hardware (1)
- sk-hynix (1)
- samsung (1)
- micron (1)
- dram (1)
- hbm4 (1)
- data-center (1)