Chaos and Order

💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction — A Vector RAG Recap, and Where It Stalls
Two Questions Vanilla RAG Can't Answer
What GraphRAG Actually Is — Building the Graph Index
Local Search vs Global Search
A Concrete Example — An Incident Corpus
The Honest Tradeoffs — It Isn't Cheap
So, Should You Use It?
Closing
References

Introduction — A Vector RAG Recap, and Where It Stalls

The standard RAG (retrieval-augmented generation) recipe is already laid out in Production RAG Patterns. In short: chunk your documents, embed them into a vector database, pull the top-k chunks closest in meaning to the question, and paste them into the prompt. Add hybrid search and reranking, and most "where does this document say X?" questions are handled well.

But this pipeline has a structural blind spot. Vector search is, in the end, the act of finding "the chunk closest in meaning," and it works best when the answer sits whole inside a single chunk. The trouble starts when the answer is scattered across many chunks, or is stated in no single chunk at all.

Two Questions Vanilla RAG Can't Answer

First, multi-hop questions. "Which other services owned by the team involved in the biggest outages depend on Postgres?" That answer only emerges by following a chain of links — outage → team → service → dependency. No chunk holds the whole chain. However well you tune top-k, stitching together links scattered across different documents is not retrieval; it is reasoning.

Second, global "sensemaking" questions. "What are the recurring themes across all of our postmortems?" This is the exact example the GraphRAG paper gives — "What are the main themes in the dataset?". This is not a retrieval problem but a summarization problem. In the paper's terms it is query-focused summarization, and the answer is spread across the whole corpus rather than any particular chunk. The moment you take a top-k, you have already lost — k chunks are not the whole. And "just stuff everything into context" is impossible for a one-million-token corpus, and collapses into lost-in-the-middle even when it fits.

What GraphRAG Actually Is — Building the Graph Index

Microsoft's GraphRAG (Edge et al., arXiv:2404.16130) takes direct aim at those two blind spots. The core idea is: "before you retrieve, use an LLM to extract a knowledge graph from the corpus ahead of time." In the paper's words, an LLM builds a graph index in two stages — first deriving an entity knowledge graph from the source documents, then pregenerating community summaries for all groups of closely related entities.

The indexing pipeline flows like this:

Chunking. Slice the corpus into analyzable units called TextUnits. (Same as ordinary RAG up to here.)
Extraction. From each TextUnit, an LLM extracts entities, relationships, and key claims. Triples pile up — "payments-service —(owned-by)→ Alice's team," "payments-service —(depends-on)→ Postgres."
Graph construction. Wire the extracted entities as nodes and relationships as edges into one knowledge graph over the whole corpus.
Community detection. Cluster the graph hierarchically using the Leiden technique. Densely connected nodes fall into one community, and communities in turn form higher-level communities. (Leiden is a community-detection algorithm that improves on Louvain and guarantees well-connected communities — Traag et al., 2019.)
Community summaries. For each community, an LLM generates a summary bottom-up. Summaries like "this community is about payment infrastructure and its incident history" are produced at every level of the hierarchy.

The key shift here is that the expensive work of understanding the corpus is done at index time, not query time. You pay the cost of comprehension up front.

Local Search vs Global Search

The two structures you have built (the graph plus the community summaries) are used differently depending on the kind of question.

Global Search is for the sensemaking questions above. The mechanism is map-reduce — in the map step, each community summary produces its own partial answer to the question (each with a helpfulness score), and in the reduce step those partial answers are combined into a final response. In the paper's own words: each community summary is used to generate a partial response, before all partial responses are summarized again into a final response. For a question that has to sweep the whole corpus, you sweep the pregenerated summaries rather than the raw chunks.

Local Search is for questions about a specific entity. It finds the entities in the question and fans out to their neighbors in the graph, gathering the connected relationships, text units, and community summaries into context. This is where multi-hop questions get solved — walking the graph reconnects the scattered links. (There is also a variant, DRIFT search, that adds community context on top of this.)

One-line summary: Global sees the "forest," Local sees "a tree and its neighbors."

A Concrete Example — An Incident Corpus

Say you have indexed several hundred outage postmortems from the past year.

Question A (local / multi-hop): "What other services are owned by the team involved in the payments outage?"
  Vector RAG:        returns a few payments-service chunks -> misses the team -> other-service hop
  GraphRAG (local):  payments-service node -(involved)-> team node -(owns)-> service node
                     walk the graph a few hops, assemble the answer

Question B (global / sensemaking): "What are the recurring themes across last year's postmortems?"
  Vector RAG:        returns top-k postmortems -> "overall themes" never emerge by construction
  GraphRAG (global): map-reduce over community summaries
                     community 1: DB failover / community 2: deploy rollback / community 3: auth expiry
                     -> reduce to "3 recurring themes"

Question C (simple local lookup): "What does ERR_2043 mean?"
  Vector RAG:        returns the one relevant chunk -> sufficient (GraphRAG is overkill)

Question C matters. Not every question needs a graph — most real-world questions still look like C.

The Honest Tradeoffs — It Isn't Cheap

Now for the honest part. GraphRAG is not free.

Indexing cost. The extraction step runs an LLM over every chunk in the corpus (and GraphRAG does several gleaning passes to catch missed entities). Then the community-summary step runs an LLM again for every community at every level of the hierarchy. So indexing cost scales with corpus size, and most of it is LLM calls. Microsoft's published material does not state a specific dollar figure, so I will not invent one here — but it is plainly an operation of a different order of magnitude than vector RAG's "just embed the documents" indexing.

Latency and updates. Indexing is a batch job, taking minutes to hours on a large corpus. The sharper pain is updates — every time a document changes you have to rebuild the graph and the summaries (incremental indexing exists, but it adds complexity), which makes GraphRAG a poor fit for a corpus that churns in real time.

Query cost. Global Search itself is not cheap either — the map-reduce fires the LLM many times across many community summaries. Local Search is far cheaper.

Dependence on extraction quality. The graph is only as good as what extraction pulled out. Extract sloppy entities and you get a sloppy graph, which is why extraction prompts have to be tuned per domain. What counts as an entity, and which relationships are meaningful, is ultimately a domain-modeling problem — a topic covered separately in Modeling Domain Knowledge as an Ontology and Actually Building a Knowledge Graph.

So, Should You Use It?

The decision boils down to this.

When it earns its cost

The questions are global and thematic — "the overall themes," "the general trend," "how X and Y relate across the corpus."
You need multi-hop reasoning that crosses many documents.
The corpus is relatively stable and is queried often enough, and long enough, to amortize the indexing cost.

When it's overkill

The questions are mostly local lookups — "where does it say X," "what does this error mean."
The corpus churns constantly.
Budget and latency are tight, and hybrid search plus reranking already clears your eval bar.

Real production systems usually do not pick one; they mix. Local lookups go to cheap vector or hybrid search, and the graph is reserved for the genuine sensemaking questions. The fact that GraphRAG itself ships a Local Search mode already tells you its hybrid nature — it is not a graph-versus-vector dichotomy but a spectrum where you pick the tool that fits the question. A comparison of the actual implementations — from the microsoft/graphrag library to Neo4j and LlamaIndex's graph indices — is covered separately in The Tools for Running Graph RAG.

And the principle from Production RAG Patterns holds here too — evaluate on your own queries, not on benchmarks. The improvements the paper reports (gains in the comprehensiveness and diversity of answers on global sensemaking questions) are author-reported figures graded by an LLM-as-judge, on datasets in the one-million-token range. Whether the same gains appear on your corpus and your questions is something you have to confirm yourself with a small eval set.

Closing

To sum up: vector RAG is strong on questions whose answer sits in a single chunk, and structurally stuck on multi-hop and global sensemaking. GraphRAG targets those two blind spots by moving the expensive work of understanding to index time — it extracts a knowledge graph with an LLM, finds communities with Leiden, pregenerates community summaries, then answers global questions with a map-reduce over those summaries and local questions with entity-centric graph traversal.

In exchange, indexing eats a lot of LLM budget and updates are a chore. So GraphRAG is not "better RAG" — it is "RAG that solves a different question." Start with vanilla or hybrid, and only when global sensemaking turns out to be your actual failure mode — that is when you reach for the graph. The question comes first, not the tool.