필사 모드: Coding Agent Spend Is Controlled by Friction, Not by Caps

• The bill arrived and nobody knows who spent what
• Why a hard budget is the last resort
• The two-tier structure that separates runaway protection from governance
• The size of one increment is nearly the whole design
• Route to the cheapest model that can do the job, not to the cheapest model
• What actually dominates cost is context, not model pricing
• The cache is not free
• A minimum configuration for bringing this into your own organization
• References

The bill arrived and nobody knows who spent what

About two months after coding agents are turned loose on a team, the same conversation usually starts. The bill has tripled against last month, and nobody can explain where it came from. There are three kinds of tools, each calling different models under different accounts, and usage is scattered across vendor consoles.

The most common response in this state is a per-person cap. And it mostly fails. The person who hits the cap is blocked mid-task, opens a ticket, and waits for someone to approve it. The interrupted working time costs more than the money saved.

Why a hard budget is the last resort

Managing AI Coding Costs at Scale, published by Databricks on 7 August 2026, interviews a number of companies and then writes this: hard budgets were used only as a last resort at every company they talked to.

The reason lies in the cost structure. Coding agent spend is not a normal distribution, it is a long tail. Most engineers use an ordinary amount, and a small number run large jobs. And those few large jobs are generally the most valuable work — big refactors, migrations of aging services, tracing the cause of an outage, the kind of thing that takes a human days. Draw the cap at the average and you cut exactly those jobs.

The second problem with caps is that they produce no learning. The person who gets blocked does not come to understand why they spent so much; they only learn that they were blocked. Next month they work the same way and get blocked in the same place.

So the direction changes. Instead of blocking use, make the amount being spent visible, and require one confirmation to keep going.

The two-tier structure that separates runaway protection from governance

How Databricks manages its own coding agent spend, from the same company on 28 July 2026, lays out that design concretely. The budget is separated into two layers.

The daily budget is a low runaway-protection line. Its purpose is not saving but accident detection. It catches things like an agent stuck in an infinite loop or a misconfigured batch job within a day. When usage gets near ninety percent, a Slack notification goes out, and with a single button the person can raise their own limit by one step. There is no cap on how many times this self-approval can be used.

The monthly budget has a different character. It is set high enough that most engineers will never reach it in their working lives, raising it requires manager approval, and the increase carries an expiry — one month, three months, six months — after which it reverts automatically. The steps are not fine-grained either: roughly two times, five times, effectively unlimited. The coarseness is intentional, because that is what turns the approval conversation into a real discussion rather than a formality.

The size of one increment is nearly the whole design

In this structure, the value that needs the most care is not the limit but the increment. As the original puts it, the increment size was chosen so that an engineer spending evenly against the monthly budget would never see a notification at all.

And when a manager raises the monthly budget, the daily limit and the increment grow proportionally with it. Skip that, and someone approved for a big project spends the whole day pressing the self-approval button, and the runaway-protection line loses its meaning.

"""Gateway budget decision: two limits, one effective limit."""
from dataclasses import dataclass


@dataclass
class BudgetPolicy:
    monthly_max: float        # ceiling that only manager approval can raise
    runaway_increment: float  # the width one self-approval opens up

    def effective_limit(self, month_to_date: float) -> float:
        """Month to date plus one increment, but never above the monthly ceiling."""
        return min(month_to_date + self.runaway_increment, self.monthly_max)

    def decide(self, month_to_date: float, today: float, today_limit: float):
        limit = self.effective_limit(month_to_date)
        if month_to_date + today >= self.monthly_max:
            return "block", "monthly ceiling reached - manager approval required"
        if today >= today_limit:
            return "self_ack", "one confirmation button and you may continue"
        if today >= today_limit * 0.9:
            return "notify", f"remaining today: {round(today_limit - today, 2)} units"
        return "allow", ""


# a generous monthly ceiling combined with a small daily increment
policy = BudgetPolicy(monthly_max=1000.0, runaway_increment=40.0)
for mtd, today, day_limit in [(120, 10, 40), (120, 37, 40), (120, 41, 40), (990, 15, 40)]:
    print(mtd, today, policy.decide(mtd, today, day_limit))

What this code does is simple, but the whole of the organizational policy lives in it. What counts as a block, what counts as a confirmation, and what counts as silence are settled in four lines. Instead of scattered dashboards, you review this one function.

Route to the cheapest model that can do the job, not to the cheapest model

The second axis is routing. Databricks states that the smart router in its internal gateway cuts the average cost per task by more than thirty percent while keeping results broadly comparable to the highest-quality model.

The thing to watch is that this is not the same as "let us use cheap models." It is a decision, made per request, about the cheapest model that can actually do that job, and a bad decision brings retries that make it more expensive instead. So a router must always be evaluated with quality metrics and retry rate alongside cost, before and after adoption. Look at cost alone and it always looks like an improvement.

The same post also describes a decision not to open a top-tier model internally, on the grounds that a quality improvement over the previous version had not been confirmed. What matters here is that not using the newest model is also an option.

What actually dominates cost is context, not model pricing

The third axis is the biggest. To carry over the original wording: by the time the expensive inference runs, the sentence the user first typed is a negligible fraction of the data entering the system, and cost is dominated by context.

What rides along on a single agent call is the system prompt, the tool definitions, file snippets, retrieval results, and the entire history of previous turns. The one line the user typed is a rounding error next to that. So halving the context generally produces a larger effect than moving to a model that costs half as much per token.

Databricks writes that harness tuning and cache configuration changes cut the number of generated tokens by nearly fifty percent. The key point is that this number was obtained without changing the model.

The cache is not free

One sentence about caching is exactly right: writing to the cache costs money, and reading from the cache greatly lowers the cost per inference.

The fact that these two live together is frequently forgotten in practice. Turn on the cache in a configuration whose prefix changes often, and you keep paying the write cost while the hits never come. So the thing to check before enabling a cache is not the hit rate but the stability of the prefix. If the system prompt and the tool definitions come out in a different order every request, or a timestamp gets mixed in, the hit rate structurally cannot rise.

Checking is not hard. Pull a hundred requests from real traffic and print a hash of the prefix that would be cached. If ninety distinct hashes come out, then in that configuration the cache only adds cost. It is common for the number of distinct hashes to drop into single digits merely by fixing the sort order of the tool list and moving the dynamically injected current time and session identifier behind the prefix.

A minimum configuration for bringing this into your own organization

You do not need to clone this design exactly, but the order is worth keeping.

The Databricks post states that before the budget structure was changed internally, somewhere between 500 and 1,000 engineers were hitting limits every month. That is hundreds of tickets and the same number of interrupted working sessions. Which means the success or failure of cost management should be judged not only by how much the bill fell, but also by how many times a human hand was required in the process of lowering it.

References

• Managing AI Coding Costs at Scale — Databricks, 2026-08-07 — the four techniques, routing savings above 30 percent, the 50 percent reduction in generated tokens, and the mention of cache write cost all come from this post.
• How Databricks manages its own coding agent spend with Unity AI Gateway Budgets — Databricks, 2026-07-28 — the two-tier budget, self-approval, proportional scaling of the increment, and the situation of 500 to 1,000 people a month hitting limits appear here.
• Some media summaries circulate a claim that these techniques delivered savings of up to 90 percent, but the figures I could confirm in the Databricks originals are the ones written above, and the original itself states that the savings ranges collected in its table are directional numbers based on an informal survey. The 90 percent figure is not confirmed in the original.
• The budget decision code in this post is my own implementation of the rules described in the original, and the numbers are examples.