https://www.youngju.dev/blog 천천히 올바르게. AI Researcher & DevOps Engineer Youngju's tech blog. GPU/CUDA, LLM, MLOps, Kubernetes AI workloads, distributed training, and data engineering. ko fjvbn2003@gmail.com (Youngju Kim) fjvbn2003@gmail.com (Youngju Kim) Sat, 16 May 2026 00:00:00 GMT https://www.youngju.dev/blog/culture/2026-05-16-ai-inference-engines-2026-vllm-sglang-llama-cpp-tgi-tensorrt-llm-mlx-mistralrs-deepspeed-aphrodite-deep-dive.en https://www.youngju.dev/blog/culture/2026-05-16-ai-inference-engines-2026-vllm-sglang-llama-cpp-tgi-tensorrt-llm-mlx-mistralrs-deepspeed-aphrodite-deep-dive.en In 2026, LLM engineering is no longer about which model — it is about which inference engine. We dissect vLLM V1, SGLang 0.4, TensorRT-LLM, TGI 3.x, llama.cpp, MLX-LM, mistral.rs, DeepSpeed-MII, Aphrodite, CTranslate2, ExLlamaV3, OpenVINO, AWS Neuron, Triton — 10+ engines through the lens of PagedAttention, Continuous Batching, Speculative Decoding, Disaggregated Inference, KV quantization, NIM, and Groq LPU. Plus self-hosting ROI math and Korean/Japanese inference infrastructure. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llm-inferencevllmsglangllama-cpptgitensorrt-llmmlxmistral-rsdeepspeedaphroditeinferenceenglish https://www.youngju.dev/blog/culture/2026-05-16-ai-inference-engines-2026-vllm-sglang-llama-cpp-tgi-tensorrt-llm-mlx-mistralrs-deepspeed-aphrodite-deep-dive.ja https://www.youngju.dev/blog/culture/2026-05-16-ai-inference-engines-2026-vllm-sglang-llama-cpp-tgi-tensorrt-llm-mlx-mistralrs-deepspeed-aphrodite-deep-dive.ja 2026 年の LLM エンジニアリングはもうモデル選定の問題ではなく、エンジン選定の問題になった。vLLM V1、SGLang 0.4、TensorRT-LLM、TGI 3.x、llama.cpp、MLX-LM、mistral.rs、DeepSpeed-MII、Aphrodite、CTranslate2、ExLlamaV3、OpenVINO、AWS Neuron、Triton — 10 以上のエンジンを PagedAttention・Continuous Batching・Speculative Decoding・Disaggregated Inference・KV 量子化・NIM・Groq LPU の観点から横並びで比較する。セルフホスト ROI 計算と日韓の推論インフラまで。 Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llm-inferencevllmsglangllama-cpptgitensorrt-llmmlxmistral-rsdeepspeedaphroditeinference日本語 https://www.youngju.dev/blog/culture/2026-05-16-ai-inference-engines-2026-vllm-sglang-llama-cpp-tgi-tensorrt-llm-mlx-mistralrs-deepspeed-aphrodite-deep-dive https://www.youngju.dev/blog/culture/2026-05-16-ai-inference-engines-2026-vllm-sglang-llama-cpp-tgi-tensorrt-llm-mlx-mistralrs-deepspeed-aphrodite-deep-dive 2026년 LLM 추론은 더 이상 모델 선택의 문제가 아니라 엔진 선택의 문제가 됐다. vLLM V1, SGLang 0.4, TensorRT-LLM, TGI 3.x, llama.cpp, MLX-LM, mistral.rs, DeepSpeed-MII, Aphrodite, CTranslate2, ExLlamaV3, OpenVINO, AWS Neuron, Triton — 10개+ 엔진을 PagedAttention·Continuous Batching·Speculative Decoding·Disaggregated Inference·KV 양자화·NIM·Groq LPU 관점에서 한 줄로 비교한다. 자가 호스팅 ROI 계산과 한국·일본 추론 인프라까지. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llm-inferencevllmsglangllama-cpptgitensorrt-llmmlxmistral-rsdeepspeedaphroditeinference https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.en https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.en A map of the 2026 LLM serving and inference landscape. Datacenter camp (vLLM, SGLang, TGI, Triton, TensorRT-LLM), local camp (llama.cpp, MLX, llamafile, Ollama, LM Studio, GPT4All), emerging camp (KTransformers, MLC LLM, Modular MAX), and cloud serving SaaS (Together, Fireworks, Groq, Cerebras, SambaNova, Lepton acquired by NVIDIA). Quantization formats (GGUF Q4_K_M, AWQ, GPTQ, FP8), plus Korean and Japanese model ecosystems (Upstage Solar, KT Mi:dm, Sakana, NTT Tsuzumi, ELYZA) — who should pick what. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmservinginferencevllmllama-cppmlxllamafileollamalm-studiogpt4allsglangtgiktransformerstensorrt-llmmodular-maxquantization2026deep-diveenglish https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.ja https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.ja 2026 年の LLM サービング・推論フレームワークの地図を描く。データセンター陣営(vLLM・SGLang・TGI・Triton・TensorRT-LLM)、ローカル陣営(llama.cpp・MLX・llamafile・Ollama・LM Studio・GPT4All)、新興陣営(KTransformers・MLC LLM・Modular MAX)、そしてクラウドサービング SaaS(Together・Fireworks・Groq・Cerebras・SambaNova・NVIDIA が 2025 年に買収した Lepton)。量子化フォーマット(GGUF Q4_K_M・AWQ・GPTQ・FP8)、韓国・日本のモデル生態系(Upstage Solar・KT Mi:dm・Sakana・NTT つづみ・ELYZA)まで — 誰が何を選ぶべきか。 Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmservinginferencevllmllama-cppmlxllamafileollamalm-studiogpt4allsglangtgiktransformerstensorrt-llmmodular-maxquantization2026deep-dive日本語 https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive 2026년 LLM 서빙·추론 프레임워크의 지도를 그린다. 데이터센터 진영(vLLM·SGLang·TGI·Triton·TensorRT-LLM), 로컬 진영(llama.cpp·MLX·llamafile·Ollama·LM Studio·GPT4All), 신생 진영(KTransformers·MLC LLM·Modular MAX), 그리고 클라우드 서빙 SaaS(Together·Fireworks·Groq·Cerebras·SambaNova·Lepton-NVIDIA 인수). 양자화 포맷(GGUF Q4_K_M·AWQ·GPTQ·FP8), 한국·일본 모델 생태계(Upstage Solar·KT Mi:dm·Sakana·NTT Tsuzumi·ELYZA)까지 — 누가 무엇을 골라야 하나. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmservinginferencevllmllama-cppmlxllamafileollamalm-studiogpt4allsglangtgiktransformerstensorrt-llmmodular-maxquantization2026deep-dive