https://www.youngju.dev/blog 천천히 올바르게. AI Researcher & DevOps Engineer Youngju's tech blog. GPU/CUDA, LLM, MLOps, Kubernetes AI workloads, distributed training, and data engineering. ko fjvbn2003@gmail.com (Youngju Kim) fjvbn2003@gmail.com (Youngju Kim) Sat, 16 May 2026 00:00:00 GMT https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.en https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.en A map of the 2026 LLM serving and inference landscape. Datacenter camp (vLLM, SGLang, TGI, Triton, TensorRT-LLM), local camp (llama.cpp, MLX, llamafile, Ollama, LM Studio, GPT4All), emerging camp (KTransformers, MLC LLM, Modular MAX), and cloud serving SaaS (Together, Fireworks, Groq, Cerebras, SambaNova, Lepton acquired by NVIDIA). Quantization formats (GGUF Q4_K_M, AWQ, GPTQ, FP8), plus Korean and Japanese model ecosystems (Upstage Solar, KT Mi:dm, Sakana, NTT Tsuzumi, ELYZA) — who should pick what. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmservinginferencevllmllama-cppmlxllamafileollamalm-studiogpt4allsglangtgiktransformerstensorrt-llmmodular-maxquantization2026deep-diveenglish https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.ja https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive.ja 2026 年の LLM サービング・推論フレームワークの地図を描く。データセンター陣営(vLLM・SGLang・TGI・Triton・TensorRT-LLM)、ローカル陣営(llama.cpp・MLX・llamafile・Ollama・LM Studio・GPT4All)、新興陣営(KTransformers・MLC LLM・Modular MAX)、そしてクラウドサービング SaaS(Together・Fireworks・Groq・Cerebras・SambaNova・NVIDIA が 2025 年に買収した Lepton)。量子化フォーマット(GGUF Q4_K_M・AWQ・GPTQ・FP8)、韓国・日本のモデル生態系(Upstage Solar・KT Mi:dm・Sakana・NTT つづみ・ELYZA)まで — 誰が何を選ぶべきか。 Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmservinginferencevllmllama-cppmlxllamafileollamalm-studiogpt4allsglangtgiktransformerstensorrt-llmmodular-maxquantization2026deep-dive日本語 https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive https://www.youngju.dev/blog/culture/2026-05-16-llm-serving-local-inference-2026-vllm-llama-cpp-mlx-ollama-lm-studio-sglang-tgi-deep-dive 2026년 LLM 서빙·추론 프레임워크의 지도를 그린다. 데이터센터 진영(vLLM·SGLang·TGI·Triton·TensorRT-LLM), 로컬 진영(llama.cpp·MLX·llamafile·Ollama·LM Studio·GPT4All), 신생 진영(KTransformers·MLC LLM·Modular MAX), 그리고 클라우드 서빙 SaaS(Together·Fireworks·Groq·Cerebras·SambaNova·Lepton-NVIDIA 인수). 양자화 포맷(GGUF Q4_K_M·AWQ·GPTQ·FP8), 한국·일본 모델 생태계(Upstage Solar·KT Mi:dm·Sakana·NTT Tsuzumi·ELYZA)까지 — 누가 무엇을 골라야 하나. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmservinginferencevllmllama-cppmlxllamafileollamalm-studiogpt4allsglangtgiktransformerstensorrt-llmmodular-maxquantization2026deep-dive https://www.youngju.dev/blog/culture/2026-05-16-local-ai-on-device-llms-2026-ollama-lm-studio-jan-msty-open-webui-gpt4all-anythingllm-faraday-deep-dive.en https://www.youngju.dev/blog/culture/2026-05-16-local-ai-on-device-llms-2026-ollama-lm-studio-jan-msty-open-webui-gpt4all-anythingllm-faraday-deep-dive.en By May 2026, local AI is no longer a hobby. An M4 Max MacBook Pro runs Llama 4 Scout 109B MoE at 24 tokens per second. Desktop runtimes like Ollama, LM Studio, Jan, and Msty unify GUI and CLI, while Open WebUI, AnythingLLM, and LibreChat deliver ChatGPT-class interfaces. The backends — llama.cpp, MLX-LM, vLLM — hold up the stack, and quantization formats (GGUF, AWQ, GPTQ, EXL3, MXFP4, BitNet) let an 8GB laptop infer 13B models. Llama 3.3 70B, DeepSeek R1 Distill, Qwen 3, Phi-4, Gemma 3, MiniCPM 3, DeepSeek Coder V2 — open models competitive with the cloud ship every week. Meanwhile Apple Intelligence (iOS 18), Phi Silica (Windows Copilot+), and Gemini Nano (Android / Chrome) have standardized OS-level on-device AI. This piece maps the entire local AI landscape as of May 2026 — runtimes, GUIs, backends, quantization, recommended models, and Korean / Japanese local stacks. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) local-aion-device-llmollamalm-studiojanmstyopen-webuigpt4allanythingllmfaradayllama-cppmlxggufquantizationragprivategptlibrechatkhojreorpinokiochatboxapple-intelligencephi-silicagemini-nano2026deep-diveenglish https://www.youngju.dev/blog/culture/2026-05-16-local-ai-on-device-llms-2026-ollama-lm-studio-jan-msty-open-webui-gpt4all-anythingllm-faraday-deep-dive.ja https://www.youngju.dev/blog/culture/2026-05-16-local-ai-on-device-llms-2026-ollama-lm-studio-jan-msty-open-webui-gpt4all-anythingllm-faraday-deep-dive.ja 2026年5月、ローカルAIはもはや「趣味」ではない。M4 Max MacBook ProがLlama 4 Scout 109B MoEを毎秒24トークンで動かす時代だ。Ollama、LM Studio、Jan、MstyのようなデスクトップランタイムがGUI/CLIを統一し、Open WebUI、AnythingLLM、LibreChatがChatGPT級のインターフェースを提供する。バックエンドはllama.cpp、MLX-LM、vLLMが支え、GGUF / AWQ / GPTQ / EXL3 / MXFP4 / BitNet という量子化フォーマットが、8GB VRAMのノートでも13Bモデルの推論を可能にする。Llama 3.3 70B、DeepSeek R1 Distill、Qwen 3、Phi-4、Gemma 3、MiniCPM 3、DeepSeek Coder V2 — クラウドに匹敵するオープンモデルが毎週リリースされる。一方でApple Intelligence(iOS 18)、Phi Silica(Windows Copilot+)、Gemini Nano(Android / Chrome)はOSレベルのオンデバイスAIを標準化した。本稿はランタイム → GUI → バックエンド → 量子化 → 推奨モデル → 韓国・日本のローカル事例まで、2026年5月時点のローカルAI全景を一気に整理する。 Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) local-aion-device-llmollamalm-studiojanmstyopen-webuigpt4allanythingllmfaradayllama-cppmlxggufquantizationragprivategptlibrechatkhojreorpinokiochatboxapple-intelligencephi-silicagemini-nano2026deep-dive日本語 https://www.youngju.dev/blog/culture/2026-05-16-local-ai-on-device-llms-2026-ollama-lm-studio-jan-msty-open-webui-gpt4all-anythingllm-faraday-deep-dive https://www.youngju.dev/blog/culture/2026-05-16-local-ai-on-device-llms-2026-ollama-lm-studio-jan-msty-open-webui-gpt4all-anythingllm-faraday-deep-dive 2026년, 로컬 AI는 더 이상 "취미"가 아니다. M4 Max MacBook Pro에서 Llama 4 Scout 109B MoE가 24토큰/초로 도는 시대다. Ollama, LM Studio, Jan, Msty 같은 데스크탑 런타임이 GUI/CLI를 통일하고, Open WebUI, AnythingLLM, LibreChat이 ChatGPT급 인터페이스를 제공한다. 백엔드는 llama.cpp, MLX-LM, vLLM이 떠받치고, GGUF / AWQ / GPTQ / EXL3 / MXFP4 / BitNet 양자화 포맷이 8GB VRAM 노트북도 13B 모델 추론을 가능케 한다. Llama 3.3 70B, DeepSeek R1 Distill, Qwen 3, Phi-4, Gemma 3, MiniCPM 3, DeepSeek Coder V2 — 클라우드와 견줄 만한 오픈 모델이 매주 쏟아진다. 한편 Apple Intelligence(iOS 18), Phi Silica(Windows Copilot+), Gemini Nano(Android/Chrome)는 OS 레벨 온디바이스 AI를 표준화했다. 이 글은 런타임 → GUI → 백엔드 → 양자화 → 추천 모델 → 한국·일본 로컬 사례까지, 2026년 5월 기준 로컬 AI 풍경을 한 번에 정리한다. Sat, 16 May 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) local-aion-device-llmollamalm-studiojanmstyopen-webuigpt4allanythingllmfaradayllama-cppmlxggufquantizationragprivategptlibrechatkhojreorpinokiochatboxapple-intelligencephi-silicagemini-nano2026deep-dive