필사 모드: Open Source Worth Watching Right Now (1) AI Agents and LLM Tooling

• Introduction — a stack that was one lump has split into layers
• Snapshot
• Layer 1 — where the model actually runs
• Layer 2 — the gateway that abstracts providers
• Layer 3 — orchestration and RAG
• Layer 4 — agents
• Interface — please look at the license before you use it
• Before you adopt
• Links

Introduction — a stack that was one lump has split into layers

In 2023, saying you were building an LLM application meant picking a single framework, plugging in an API key, and wiring up a chain.

Now, where you run the model, how you put several providers behind one interface, whether to attach retrieval, and how many tools to hand an agent are all separate decisions. A different project has claimed each one.

The list below is not a ranking. It is a map grouped by role, and only projects inside the same layer are actually in competition with each other.

Snapshot

All figures as of 2026-08-12.

Layer 1 — where the model actually runs

ggml-org/llama.cpp is a C++ inference engine that runs GGUF quantized models on CPUs and consumer GPUs. A large share of the layers above stand on this project or on something derived from it. The repository was moved from a personal account to an organization account, so do not trust the paths in older documentation as they are. It is not meant for taking large volumes of concurrent requests across several GPUs.

ollama/ollama puts model distribution and lifecycle management on top of that, shrinking local execution down to one line. It has an OpenAI-compatible endpoint, so existing client code attaches almost unchanged. It suits development environments and personal workstations, but it is not a thing aimed at multi-tenant production serving.

# Example: start it locally and call it over the OpenAI-compatible path
ollama serve &
ollama pull <model-name>
curl http://localhost:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"<model-name>","messages":[{"role":"user","content":"ping"}]}'

vllm-project/vllm sits at the opposite end. It is a serving engine that pushes GPU throughput up with PagedAttention and continuous batching, and it is the closest thing there is to a de facto default for self-hosted inference. In exchange it is sensitive to the combination of GPU and driver and it needs memory tuning, so it is overkill at a scale of a few hundred requests a day.

sgl-project/sglang focuses on prefix cache reuse (RadixAttention), which brings a large benefit on workloads where the front of the prompt repeats. It is a young repository created in January 2024, so operational experience is thicker on the vLLM side.

Layer 2 — the gateway that abstracts providers

BerriAI/litellm unifies a great many model providers into the single OpenAI format, and in proxy mode it takes charge of key management, budget limits, fallbacks, and usage logging. It removes the work of fixing application code every time you change providers.

You need to look at the license precisely. The LICENSE file in the repository declares that the enterprise/ directory follows a separate license defined inside it, and that everything outside that is MIT. It is not uniformly MIT throughout.

Layer 3 — orchestration and RAG

langchain-ai/langchain is the framework with the widest integration surface. Its structure has been reorganized several times since the early criticism of excessive abstraction, so always check when your learning material was written. Laying it over a task that is only one or two calls deep loses you more than it gains.

run-llama/llama_index concentrates more on the RAG axis of document loading, chunking, indexing, and retrieval. If reworking retrieval quality is most of the job, this side is a good fit.

infiniflow/ragflow is not a library but a RAG engine you deploy and use, and it puts its weight on parsing PDFs with complex tables and layouts. You have to operate one more piece of infrastructure, so if your team is small, calculate that burden first.

Layer 4 — agents

crewAIInc/crewAI turned multi-agent collaboration with divided roles into a standard pattern. It fits well when you can split the problem the way a human organization would. Conversely, putting multiple agents into a pipeline that needs deterministic results is usually a loss.

browser-use/browser-use lets an LLM operate a real browser, opening a path to automating screens that have no API. That said, the repository was created in October 2024, and the interface kept changing while the stars climbed to 108,900. In terms of reliability and exposure to prompt injection, it still has to be handled carefully.

Aider-AI/aider is a pair programming tool that edits a repository from the terminal and even creates commits. The activity signal is carried over here as it is. The last push at the time of checking was 2026-05-22, which contrasts with other projects that were pushed the same day. That does not mean it is dead, but check the recent commits and the responsiveness to issues yourself before adopting it.

Interface — please look at the license before you use it

open-webui/open-webui is widely used as a chat UI for local models. But the LICENSE file matters. It takes the form of a 3-clause BSD with one more clause attached, and that clause prohibits changing or removing Open WebUI branding (the name, the logo, and so on). Because a usage restriction is attached, it is not open source as OSI defines it but a source-available license.

If your plan is to put it on an internal portal as a white label, this is where you get stuck. There are exception conditions alongside it, so read the full text yourself.

Before you adopt

Check the full licence text yourself, and route commercial adoption through legal review. This post is not legal advice.

There are also cautions that apply only to this area. LLM tooling changes its interfaces fast, so pin your versions and put upgrades into a regular budget. When you give an agent permission to execute tools, you have to deal at the design stage with the fact that this opens a path where untrusted input becomes a command directly.

Repository details (stars, licence, recent activity) were checked directly on GitHub on 2026-08-12 and are point-in-time values. The numbers and the status change.

Links

Next in the series: Open Source Worth Watching Right Now (2) Developer Tooling

Related posts on this blog:

• Taking RAG Pipelines and Vector Databases to Production
• RAG Pipeline Optimization — Chunking, Reranking, Hybrid Search
• Open Source License Shifts in 2026

Tools: curl Command Builder · JSON Formatter