Skip to content
Published on

The Context Budget — Design Is What You Leave Out, Not What You Put In

🇰🇷KO🇺🇸EN🇯🇵JA🇨🇳ZH
Split View필사 모드
Share
Authors

The window has room, but accuracy is falling

Strange things happen as an agent session gets long. The context window has not hit its limit, yet the agent forgets instructions given early on, rereads files it already checked, and writes code that contradicts rules it agreed to. This description is a constructed situation, not a specific measurement, but the direction matches published observations. The Anthropic post on context engineering calls the phenomenon context rot — retrieval accuracy degrades as tokens pile up — and points at the pairwise structure of attention and a training distribution centered on shorter sequences as reasons.

The conclusion is simple. The context window is not storage to be filled; it is an attention budget to be allocated. And a budget needs line items and cut criteria.

Context engineering is the extension of prompt engineering

The same post frames the relationship between the two jobs like this. If prompt engineering is writing instructions well, context engineering is curating and maintaining the optimal set of tokens that enter the window at each inference step. Not just the system prompt: tool definitions, external material, and message history are all in scope. The difference is that it is not a document you write well once — it is an editing decision that repeats every turn.

From the harness point of view, this definition matters for a clear reason. The context policy is one of the six knobs of the harness, a decision deployed as code. The answer to "what goes in and what comes out each turn" should exist as a function somewhere in your repository.

Tool schemas spend the budget too

The line item most often forgotten is tool schemas. Every tool you expose occupies part of the window every turn — name, description, parameter schema. Expose 21 tools and a slice of the budget is gone before the model calls anything. And since choice gets shakier as the tool count grows, schema cost is not only about tokens. That trade-off gets its own treatment in part 3 on tool surface design.

The savings lever in the opposite direction is just-in-time retrieval. Instead of preloading whole documents into the window, keep lightweight references — file paths, queries, links — and fetch at the moment of need with a tool. It is the same structure as a person remembering where the document lives instead of memorizing it, and it is one of the default strategies the Anthropic post recommends.

From prompt accumulation to a playbook

The most common context policy is having none. Newly learned rules keep getting appended to the end of the system prompt, and the document only grows. A month later, contradictory instructions coexist and nobody knows which items still hold. In this state, deletion is impossible — there are no grounds to delete on.

Lilian Weng's harness post introduces an agentic context engineering approach that solves this with structure. Treat context as an evolving playbook, and split the roles that generate entries, reflect on them, and curate them. A single entry looks roughly like this.

- id: pb-041
  rule: 'Run integration tests with -j1. Parallel runs collide on ports.'
  scope: 'repo:payments-api / task:test'
  added: '2026-07-18'
  last_used: '2026-08-07' # long-unused entries become curation candidates
  helped: 23 # how many times this entry actually helped

The point is not the format but the metadata. Once each entry records when it was added, where it applies, and whether it recently helped, deletion becomes possible. The hard half of context management is not the adding but the removing, and removal needs grounds attached to each item.

Drop policies and compaction: what, and when

When the budget overflows, something has to go. What goes is decided by policy. Pushing out the oldest first is free to implement but loses the early core rules first. Cutting the biggest chunks first saves tokens but knows nothing about importance. Only a policy that drops the least applicable items first — the playbook approach — uses "less needed for the task at hand" as its criterion. Either way, the drop policy must be an explicit harness decision. Leaving it to defaults is the same as choosing oldest-first.

Summarization — compaction — is the complement of dropping. Near the limit, compress the progress so far into a summary and restart from it. The Anthropic post lists what to preserve: architectural decisions, unresolved bugs, implementation details. Put the other way around, a compaction that does not specify what to preserve fails precisely by losing those three. A companion technique is structured note-taking: write progress to a file outside the window and reread it when needed, so even a failed summary leaves a recovery point.

Subagent delegation: buying a clean window, paying in tokens

The last resort of the context budget is splitting the window. Hand a wide-exploration subtask to a subagent, and the subagent works in its own clean window and returns only a condensed summary. The main agent's budget pays only for the conclusion.

It is not free. Anthropic's multi-agent system post reports, for their own system, that agents use roughly 4 times the tokens of ordinary chat, and multi-agent setups roughly 15 times. Delegation becomes justified when the value of the task clears that cost — and when you do delegate, the same post's lesson is to state the objective, the output format, tool guidance, and task boundaries, or you get duplicated exploration and runaway searches. Subagents are not an escape hatch from the context problem; they are a knob that reallocates the budget at a larger granularity.

Practice it yourself

Tier 4 of the harness engineering RPG, "Context budget", is the exercise for this post. Plug the four policies — append, trim, playbook, curation — into the same scenario and you can watch the drop policy change the outcome. You can also see tool schemas eating the budget as an in-game number.

References

Discuss on Twitter • View on GitHub