Skip to content
←

필사 모드: How AI Agents Fail in Production — 14 Failure Modes, and Why Retrying Is Not Safe

English
0%
정확도 0%
💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction — The Three Real Questions About Production Agents

An agent that runs fine in a demo changes the questions the moment it goes to production. From "does this work?" to "why didn't this work, how much did it burn, and is it safe to try again?" This post answers those three questions in order.

The conclusion first. Failures come mostly from system design, not the model. When UC Berkeley researchers classified 1642 multi-agent-system execution traces, 44.2% of failures were system-design issues, 32.3% were inter-agent misalignment, and 23.5% were task-verification failures. And the two most common failure modes are step repetition (15.7%) and unaware of termination conditions (12.4%) — together 28.1% of the whole. These two are not just "failure." They mean the agent repeats the same work and doesn't know when to stop, and that is both a token bill and a side effect happening twice.

That is where the third question comes in. When a nondeterministic caller (the LLM) drives real side effects (a payment, an email, a ticket), is a retry safe? The answer is "not unless you make it safe yourself." MCP, the standard protocol by which agents call tools, has a hint that declares idempotency but no mechanism that guarantees it. And the closest thing to an industry standard for this problem — IETF's Idempotency-Key header draft — began in 2020 and still isn't an RFC; as of April 2026 it is expired.

Where Agents Actually Fail — MAST's 14 Modes

The most citable source is MAST (Multi-Agent System Failure Taxonomy). It is a paper by researchers at UC Berkeley and Intesa Sanpaolo (Cemri, Pan, Yang et al., arXiv:2503.13657), presented in the NeurIPS 2025 Datasets & Benchmarks track — the first attempt to empirically classify multi-agent-system failures.

Start with the methodology. The researchers analyzed some 150 traces from 5 open-source frameworks (each averaging more than 15,000 lines) with Grounded Theory to build the taxonomy, with 6 expert annotators involved. Three annotators independently labeled 15 traces to reach an inter-annotator agreement of kappa 0.88, and then, using those definitions, built an OpenAI o1-based LLM-as-a-judge pipeline to label all 1642 traces.

Here is the classification. The figures are each mode's occurrence rate out of all 1642 traces.

[Category 1] System Design Issues                          44.2%
  1.1 Disobey Task Specification                           11.8%
  1.2 Disobey Role Specification                            1.50%
  1.3 Step Repetition                                      15.7%   <- most common
  1.4 Loss of Conversation History                          2.80%
  1.5 Unaware of Termination Conditions                    12.4%

[Category 2] Inter-Agent Misalignment                      32.3%
  2.1 Conversation Reset                                    2.20%
  2.2 Fail to Ask for Clarification                         6.80%
  2.3 Task Derailment                                       7.40%
  2.4 Information Withholding                                0.80%
  2.5 Ignored Other Agent's Input                           1.90%
  2.6 Reasoning-Action Mismatch                            13.2%

[Category 3] Task Verification                             23.5%
  3.1 Premature Termination                                 6.20%
  3.2 No or Incomplete Verification                         8.20%
  3.3 Incorrect Verification                                9.10%

It is worth checking that the numbers add up. 11.8 + 1.5 + 15.7 + 2.8 + 12.4 = 44.2, 2.2 + 6.8 + 7.4 + 0.8 + 1.9 + 13.2 = 32.3, 6.2 + 8.2 + 9.1 = 23.5. The three categories sum to exactly 100%.

One warning here. Some secondary summaries of this paper circulate figures like "42% specification problems, 37% coordination breakdown, 21% weak verification" — none of those three numbers are in the paper. The category names differ too: the paper's first category is not "specification issues" but "system design issues" (it looks like the name changed from an earlier version). If you are going to cite it, go straight to Figure 1 of the paper.

The baseline failure rates are worth noting too. The failure rates of the 7 SOTA open-source systems the paper evaluated ranged from 41% to 86.7%. The worst was AppWorld's Test-C (86.7% failure), the best AG2's OlympiadBench (41% failure). That is the baseline for the kind of thing you put in production.

What Each Failure Category Bills You For

One honest thing up front. There are no public figures that say "this failure costs X" per mode. MAST counted occurrence frequency, not cost, and no primary source I found publishes a dollar amount per failure mode. So instead of inventing numbers here, I will distinguish the kind of cost each category bills. In practice that distinction is more useful than an amount — a different kind of cost calls for a different response.

Category 1 (system design, 44.2%) is billed in tokens. Step repetition (15.7%) and unaware of termination conditions (12.4%) make up more than half of this category, and both have the same symptom: "the agent keeps going." Each turn around the loop feeds the whole context back into the model, so cost piles up not linearly but steeper. The good thing about this category is that it is the cheapest to catch — a turn cap, a budget cap, and repetition detection can all go in without touching the model.

Category 2 (inter-agent misalignment, 32.3%) is billed in wrong side effects. The most common mode here is reasoning-action mismatch (13.2%), which by the paper's definition is when an agent's reasoning and its actual action diverge. This is not a token problem. When an agent says "I'll do A" and executes B, the bill lands on whatever system B touched. Task derailment (7.40%) is the same in character.

Category 3 (task verification, 23.5%) is billed in silently wrong output. No or incomplete verification (8.20%) and incorrect verification (9.10%) sum to 17.3%, and what they share is that the failure doesn't look like a failure. The paper's example is striking — a chess program ChatDev built passed superficial tests. This category is found latest, and so is usually the most expensive.

To sum up:

Category 1 (44.2%)  ->  token bill            ->  cheapest to catch (caps / detection)
Category 2 (32.3%)  ->  wrong side effects    ->  defend with idempotency / permissions
Category 3 (23.5%)  ->  silently wrong output ->  found latest, costs the most

Why Retrying Isn't Safe

Now the crux. Overlay the two most common modes of category 1 (repetition, non-termination) with the most common mode of category 2 (reasoning-action mismatch) and one sentence falls out: a nondeterministic caller repeatedly calls a tool that causes real side effects.

In traditional distributed systems this is not a new problem. When the network drops, the client doesn't know whether its request was processed, and retrying creates duplicates. So we use idempotency keys. But agents have one more problem — in a traditional client, code decides whether to retry; in an agent, the model decides. When the model judges that a tool failed (or misjudges that it did), it simply calls again. The retry policy lives inside the prompt, and that is not a policy but a probability.

There is one crucial distinction here. A retry is dangerous not only when the tool failed. It is more dangerous when the tool succeeded but the response was lost. What the model sees then is "no result," and the model's rational move is to retry. And the side effect has already happened.

Does MCP Guarantee Idempotency? — A Hint, No Mechanism

So how does MCP — the standard protocol by which agents call tools — handle this? Open the schema of the current latest spec revision (2025-11-25) and the answer is clear.

MCP has tool annotations, and one of them is the idempotency hint. Quoting the schema's definition directly: if true, it means that "calling the tool repeatedly with the same arguments has no additional effect on its environment." The four hints' defaults are these:

readOnlyHint      default false   (assume the tool modifies its environment)
destructiveHint   default true    (assume the modification is destructive)
idempotentHint    default false   (assume repeated calls have additional effects)
openWorldHint     default true    (assume interaction with an external world)

It is worth noting that every default leans pessimistic. It means without annotations, every tool should be treated as a destructive, non-idempotent write. That is good design.

But the spec nails these hints down as follows (the schema's own comment):

Every property in ToolAnnotations is a hint. They are not guaranteed to provide a faithful description of tool behavior (including descriptive properties like title). Clients should never make tool use decisions based on ToolAnnotations received from untrusted servers.

Here the real problem shows itself. In the entire 2573-line schema, idempotentHint appears only in that one place where it is declared. And the word "retry" never appears in the schema at all. No idempotency key, no request-deduplication mechanism, no retry semantics.

In other words, on idempotency MCP gives you vocabulary but not tools. A server can claim "I am idempotent," but that claim (a) is not enforced, (b) comes with the spec's own warning not to trust it from an untrusted server, and (c) offers no way in the protocol to make a tool that is not idempotent safe to retry.

You might ask whether the transport layer has redelivery. The Streamable HTTP spec does have resumability and redelivery — the server attaches an id to each SSE event, and after a disconnect the client sends a GET with a Last-Event-ID header, letting the server replay messages after that point. But this replays server-to-client messages, and only on that broken stream (the spec states it MUST NOT replay messages from a different stream); it is just a cursor within a stream. It is not a mechanism that deduplicates the tool calls a client sent. The direction of the MCP spec itself is covered separately in MCP Sheds Its Sessions — Reading the Stateless Core.

So How Far Has the Industry Standard Come? — A Draft for Five and a Half Years

"Just use an idempotency key" is the natural reaction. Right. But surprisingly, there is no standard for idempotency keys.

Checking the history of the IETF HTTPAPI working group's Idempotency-Key HTTP header draft directly in the Datatracker, here it is:

draft-idempotency-header-00                  2020-11-17  (individual draft)
draft-ietf-httpapi-idempotency-key-header-00 2021-07-01  (WG adoption)
                                        -01  2022-05-08
                                        -02  2022-11-09
                                        -03  2023-07-04
                                        -04  2023-11-16
                                        -05  2024-05-27
                                        -06  2025-02-24
                                        -07  2025-10-15  (latest)

Current status: Expired (expired 2026-04-18)
IESG state: "I-D Exists" — the IESG has never begun processing it

It began in November 2020, was revised 8 times, and the latest revision, posted in October 2025, expired on 18 April 2026 under the six-month lifetime rule for Internet-Drafts. Its IESG state is "I-D Exists" — in the Datatracker's own words, "the IESG has not started processing this document, or has stopped processing it without publication." For more than five and a half years it has failed to become an RFC.

So in practice an "idempotency key" is not a standard but a convention. The place that documents that convention best is Stripe, and what the IETF draft above was trying to codify is largely this convention. The behavior documented by Stripe is this:

  • For a given idempotency key, it stores the first request's status code and response body, whether that was a success or a failure.
  • A subsequent request with the same key returns the same result — it replays even a 500 error verbatim.
  • The idempotency layer compares the incoming parameters against the original request and errors out if they differ.
  • A key may be pruned after at least 24 hours, and reusing the same key after it is pruned creates a new request.
  • A result is only stored once the endpoint has started executing. Parameter-validation failures and concurrent-execution conflicts are not stored, so they can be retried.

From an agent's point of view, two lines especially sting. One is that it replays even a 500 — an idempotent retry is not "try again" but "show the original result again." If the model sees a failure and retries, it sees the failure again. This is design, not a bug, and if the model doesn't understand it, it falls into an infinite loop (and that is exactly failure mode 1.3, step repetition). The other is 24 hours. When a long-running agent that churns over several days retries with yesterday's key, that is not a retry but a new side effect.

How Do You Measure Agent Cost? — OTel Has No Cost Metric

Now observability. If you want to follow a standard you end up looking at OpenTelemetry's GenAI semantic conventions, and it is best to start knowing exactly what state they are in.

First, the GenAI conventions have moved. They used to live in docs/gen-ai in the open-telemetry/semantic-conventions repo, but the files at that path now carry only a note: "this page has moved and is no longer maintained in this repository." Their new home is open-telemetry/semantic-conventions-genai.

Second, that new repo was created on 5 May 2026 and has no release yet at all. (The latest release of the main semantic conventions is v1.43.0, dated 3 July 2026.) And both the agent-spans doc and the MCP doc carry the document status "Development." Development hasn't stalled — the last push was two days ago. It just means that the thing you call a "standard" and build dashboards on is a never-released, in-development spec.

Third, and this matters most in practice — there is no cost metric. Search the GenAI semantic conventions' metrics doc and the entire attribute registry for cost, price, usd, dollar and you get zero hits. The defined metrics are all of these:

gen_ai.client.token.usage
gen_ai.client.operation.duration
gen_ai.client.operation.time_to_first_chunk
gen_ai.client.operation.time_per_output_chunk
gen_ai.server.request.duration
gen_ai.server.time_to_first_token
gen_ai.server.time_per_output_token

The standard gives you tokens. The money is on you. To turn tokens into an amount you need a per-model price table, and vendors change that table. This is why vendors' cost dashboards each have their own proprietary format and their numbers don't agree.

Why Naively Summing Tokens Is Wrong

If you have decided to compute cost yourself, there is one trap you must know. The attribute registry defines five token attributes:

gen_ai.usage.input_tokens
gen_ai.usage.output_tokens
gen_ai.usage.cache_creation.input_tokens
gen_ai.usage.cache_read.input_tokens
gen_ai.usage.reasoning.output_tokens

The problem is that these are not flat but nested. The registry's comment is explicit — cache-creation tokens and cache-read tokens SHOULD be included in gen_ai.usage.input_tokens, and input tokens should include every kind of input token, cached tokens included. Reasoning tokens likewise should be included in gen_ai.usage.output_tokens.

So this calculation is wrong:

Wrong calculation:
  cost = input_tokens x input_price + output_tokens x output_price

Why it's wrong:
  input_tokens already contains cache_read.input_tokens
  cache reads are billed cheaper than new input (that's the point of caching)
  -> the better your cache hits, the more your math overstates the real cost

Right direction:
  new_input = input_tokens - cache_read - cache_creation
  cost = new_input     x input_price
       + cache_read    x cache_read_price
       + cache_creation x cache_write_price
       + output_tokens x output_price
  (prices vary by vendor and model, so no numbers here)

In an agent this error is not small. Because an agent loop resends the same system prompt and tool definitions every turn, the cache hit rate is structurally high, so cache reads make up a large share of input_tokens. Sum them naively and the better your caching works, the more your books overstate cost. Trying to see caching's cost benefit on a dashboard, you can reach the exact opposite conclusion.

There is one more layer. Those five attributes are span attributes, and the well-known values of gen_ai.token.type — the dimension of the metric gen_ai.client.token.usage — are only input and output. That is, from the aggregated metric alone you cannot separate cached tokens from new ones. To get an accurate cost you have to drop down to the span level. (You can use a custom value, but the moment you do it is no longer standard.)

What to Instrument — and the Opt-In Trap

Tool-call instrumentation uses the execute_tool span. The span name is execute_tool followed by the tool name, and the attributes are these:

gen_ai.operation.name        Required            (Development)
gen_ai.tool.name             Required            (Development)
error.type                   Recommended         (Stable)
gen_ai.agent.name            Conditionally Req.  (Development)
gen_ai.tool.call.id          Recommended         (Development)
gen_ai.tool.description      Recommended         (Development)
gen_ai.tool.type             Recommended         (Development)
gen_ai.tool.call.arguments   Opt-In              (Development)   <- not collected by default
gen_ai.tool.call.result      Opt-In              (Development)   <- not collected by default

The last two lines are the trap. What arguments the agent called the tool with and what it got back are "Opt-In" — that is, not collected by default. There is a good reason (privacy and storage cost). But the upshot is that exactly the two fields you need to debug why the agent did something wrong are off by default. To track category 2 (reasoning-action mismatch) you have to turn them on, and the moment you do, sensitive data can enter your traces. It is not a free lunch — it is a tradeoff you decide deliberately, together with sampling and masking.

If you use MCP there is one more thing to know. The MCP semantic-conventions doc points out that HTTP trace-context propagation covers only the HTTP request and not the individual messages exchanged inside the request/response stream. So instrumentation has to inject context into the params._meta pocket of the MCP request (SEP-414 defines how the traceparent, tracestate, and baggage keys are handled), and the receiver uses it as the remote parent. Without this, the agent trace breaks at the MCP boundary. For logging and tracing in general, see LLM Logs and Tracing.

Does Fixing It Actually Help? — Honest Intervention Results

This is the part of the post that has to be most honest. The MAST paper did not stop at classifying; it tried actual interventions based on the taxonomy. The results (paper Table 5) are these:

AG2 / GSM-Plus (GPT-4)      baseline 84.75 +- 1.94 | prompt fix 89.75 +- 1.44 | topology change 85.50 +- 1.18
AG2 / GSM-Plus (GPT-4o)     baseline 84.25 +- 1.86 | prompt fix 89.00 +- 1.38 | topology change 88.83 +- 1.51
ChatDev / ProgramDev-v0     baseline 25.0          | prompt fix 34.4          | topology change 40.6
ChatDev / HumanEval         baseline 89.6          | prompt fix 90.3          | topology change 91.5

The widely cited "+9.4%" and "+15.6%" come from here. On ChatDev's ProgramDev-v0, 25.0 → 34.4 is +9.4 and 25.0 → 40.6 is +15.6. These are not percentages but percentage points.

But if you are going to cite these numbers, there are things you must say alongside them.

First, ProgramDev-v0 is 32 tasks. Out of 32, 25.0% is 8, 34.4% is 11, and 40.6% is 13 (the arithmetic lands exactly). So the celebrated "+9.4%p" means 3 more tasks passed and "+15.6%p" means 5 more tasks. It is a small sample.

Second, even after the best intervention the success rate is 40.6%. In other words, about 59% still fail. In the paper's own words: "while these initial interventions improve performance, they do not resolve all failure modes and task completion rates remain low, suggesting that more substantial improvements are needed."

Third, the effect is not universal. On AG2, the topology change was not statistically significant on GPT-4 (Wilcoxon test p-value 0.4), while the same change was significant on GPT-4o (p-value 0.03). The paper's conclusion: "these strategies are not universal, and their effectiveness depends on factors such as the underlying LLM."

Fourth, there is a direction. The paper reports that on both systems, topology-based changes were more effective than prompt-based ones. And the authors insist that robust reliability needs "more than isolated fixes" and should aim at a "fundamental MAS redesign."

Finally, the limits of the source itself. Most of the 1642 traces were labeled by the authors' own o1-based LLM annotator. Only a much smaller subset was human-verified (the kappa 0.88 is the agreement of 3 annotators on 15 traces). The authors themselves write that they do not claim MAST covers every potential failure pattern. This is not a definitive map, but the most empirical one produced so far.

So What Should You Do? — Decision Rules

Rules that follow directly from the sources above.

1. Classify tools by side effect first. The MCP defaults already tell you the answer — without annotations, treat a tool as destructive and non-idempotent. Don't mix read-only tools and write tools under one retry policy.

2. Implement idempotency in your own layer. MCP's idempotentHint is a hint, not a guarantee, and the spec itself says not to trust an untrusted server's annotations. There is no IETF standard either. So for tools that cause side effects, take a key on your server side, store the first result, and replay it. Stripe's convention is a proven reference implementation — down to storing the result (errors included), comparing the parameter fingerprint, and stating a retention period.

3. Don't let the model make the key. Because the thing deciding on a retry is nondeterministic, the idempotency key has to be derived from somewhere deterministic — the orchestrator's step ID or task ID. If the model invents a new key every time, the idempotency key is decoration.

4. Set caps as a budget. If 28.1% of failures are repetition and non-termination, then a turn cap and a token budget are not an optimization but a circuit breaker. It is the cheapest way to block the most common failure category.

5. Compute cost from spans. Metrics alone can't separate cache tokens. And don't just sum tokens — cache reads are already inside input_tokens.

6. Decide argument and result collection deliberately. The default is off. Leave it off and you can't debug category 2; turn it on and sensitive data enters your traces. Design sampling and masking alongside it.

When Not to Do Any of This

The honest other side. Most of the above earns its keep only when the agent drives real side effects.

  • For a read-only agent, idempotency infrastructure is overkill. In an agent that only searches, summarizes, and analyzes, a duplicate call just costs more money without corrupting state. Just set caps and move on.
  • If a human approves every write, most of the retry risk goes away. An approval gate is cheaper to build than an idempotency layer, and in early stages it is usually better.
  • If a workflow is enough, don't use an agent. Most MAST failures are system-design issues, and those arise "in proportion to how much you handed the decision to the model." If the path is fixed, write it in code — the workflow-versus-agent distinction covered in Building Effective AI Agents applies directly here.
  • If a single agent works, don't go multi-agent. 32.3% of MAST failures happened between agents. The moment you add one more agent, you buy that 32.3% of surface area anew.

Closing

To sum up. Agent failure is not the model's failure but mostly a failure of system design (44.2% by MAST). The two most common modes are repetition and non-termination (28.1% together), and these two burn tokens while duplicating side effects. And there is no protocol-level device to prevent that duplication — MCP gives only an idempotency hint, and IETF's idempotency-key standard has been a draft for five and a half years and is expired right now. Observability is similar — the OTel GenAI conventions are a never-released, in-development spec, have no cost metric at all, and leave the arguments and results you need for debugging off by default.

This list is not a reason to be pessimistic but a map of where you have to fill the gaps yourself. The box you left empty expecting a standard to fill it is exactly the box the bill comes from in production.

And finally, look at the intervention results again. Even after the best structural intervention, code-generation success ran from 25.0% to 40.6% — more than half still fail. Running a production agent is less about eliminating failure than about knowing the cost of failure — which failures only burn tokens, which corrupt state, and which quietly emit a wrong answer. Operations begins the moment you tell those three apart. The question of what to measure an agent by, and how, continues in The Simulated Customer Never Leaves, and the problem of running agents in isolation in Lambda microVM Agent Sandboxes.

References

현재 단락 (1/155)

An agent that runs fine in a demo changes the questions the moment it goes to production. From "does...

작성 글자: 0자원문 글자: 22,751자작성 단락: 0/155