LLM API Failure Taxonomy: Classify Errors by Origin and Retry Safety
A call times out, you send it again, and the next response comes back with a body your parser rejects. Two failures, one screen, no way to tell whether the retry button helps. An LLM API failure taxonomy settles that. It is a fixed, hierarchical classification structure: every observed failure maps to exactly one origin class, and each class carries four assignments, a retry-safety judgment, a customer-impact level, an owner, and an evidence source. The mapping ends in a decision, not a label, and once a failure sits in its class you answer two questions in order. Should this call be retried, and which owner acts first?
The origin classes come from published catalogues of failures that occur before or at the model call: template-loading exceptions, failed initialisation, malformed responses. Retry safety and customer impact come from no published source, so you define both and apply them under a methodology you publish. No latency, error-rate, failover-timing, or savings figures are asserted here, because the published sources carry no frequency, severity, or outcome data. The one validation figure below comes from the arXiv study “Characterizing Faults in Agentic AI” (arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31).
What published taxonomies already organize LLM failure modes?
Two published taxonomies supply the structure: ErrorAtlas and AgentErrorTaxonomy. ErrorAtlas is model-agnostic and defines 17 top-level error categories. AgentErrorTaxonomy is modular and spans five domains. Both give the classification that follows its structural anchor.
ErrorAtlas: 17 categories, model-agnostic
ErrorAtlas organizes LLM failure modes into 17 top-level error categories. The published set is 17 categories, and the authors state they plan to periodically update it as new benchmarks and models emerge (arXiv 2601.15812v2, checked 2026-08-31). Model-agnostic means the same 17 categories apply across models, so a trace from one provider maps onto the same category set as a trace from another provider. If you are pulling traces from two providers, they land in the same 17 buckets. The useful property is that a bounded category count is enough to structure the whole field, and that is what the taxonomy here borrows from it.
AgentErrorTaxonomy: five domains, modular
AgentErrorTaxonomy takes a modular shape and covers five domains: memory, reflection, planning, action, and system-level operations. Modular means each domain extends or updates separately from the others. The agentic-fault study, arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31, groups runtime exceptions during prompt or template loading, invalid or malformed model responses, and unsuccessful LLM initialisation under a dimension it calls Agent Cognition and Orchestration, listing all three as consequences of LLM misconfiguration. That placement is what drives the origin axis in the next section.
How should failures be classified by origin?
Origin separates failures by where they happened in the call path, whatever the output looks like. Runtime exceptions during prompt or template loading, invalid or malformed model responses, and unsuccessful LLM initialisation are catalogued under the dimension Agent Cognition and Orchestration in v1 of that study, so they originate before or at the model-call boundary, not in the correctness of a completed response. A loading exception means the model call never ran. An output-content failure can only be judged after a complete response exists.
Four origin classes, one decision per row
Treat orchestration and pre-inference failures as their own origin class, separate from output-content failures. The table below maps four origin classes to retry safety, customer impact, owner, and evidence source: one row per origin class, one decision per row.
| Origin class | Example failures | Retry safety | Customer impact | Owner | Evidence source |
|---|---|---|---|---|---|
| Orchestration or pre-inference | Runtime exceptions during prompt or template loading; unsuccessful LLM initialisation | Retry-safe when transient and free of side effects, judged under operator methodology | Wide when a shared template or model configuration is broken | Gateway or client integration | Application and gateway request logs |
| Model-call boundary | Invalid or malformed model responses; protocol and transport errors at the API edge | Transient timeout and transport classes are retry candidates under idempotency; malformed-response classes route to the owner | Affects every dependent call waiting on the response | Model provider or gateway path | Gateway response logs and provider status records |
| Output content | Completed responses that violate the prompt contract | Not a transport retry; regenerate under operator policy or escalate | Affects downstream response correctness | Application or prompt layer | Output review and evaluator labels |
| Client side | Request construction faults; key and quota configuration errors in the client | Retry only after the request fault is corrected | Limited to the affected client | Client integration | Client and gateway request logs |
The table assigns one owner per origin class and one evidence source per owner, which is what makes a retry and escalation decision executable from a single row. The published quotes list these modes without frequency, severity, retry-safety, or customer-impact data, so nothing about retry outcome or impact follows from them.
Which LLM API failures are retry-safe?
Retry safety hangs on two properties: whether the failure is transient, and whether repeating the call is free of side effects. The published taxonomies report no frequency, severity, or retry-outcome data, so the retry-safety column in the table is a framework you apply under your own methodology, not a measurement.
A transient timeout at the model-call boundary is a retry candidate when the call is idempotent. A malformed model response produced by a prompt defect would reproduce the same defect on retry, so the framework routes it to the owner instead. An output-content failure is not a transport failure at all, because the request reached the model and produced a response, so the correction is regeneration under your policy.
Retry decision rules
Three checks, in order:
- Is the failure transient? Transient means the failure does not persist when the call is repeated under the same inputs.
- Is a retry free of side effects? Free of side effects means a retry does not double-charge, double-write, or double-notify.
- Does the retry have a bounded number of attempts and a backoff policy?
If any check fails, the trace routes to the owner named in the table. Rate limiting is the most common transient class at this boundary, and 429 rate limit errors when using AI APIs have their own diagnosis path. Your evidence block should state how both conditions were tested, because the published taxonomies do not supply that testing.
How do origin, retry safety, and impact map to an owner?
Read one row and you have the whole decision. Orchestration and pre-inference failures go to the gateway or client integration, model-call boundary failures go to the provider or the gateway path, and output-content failures go to the application or prompt layer. Customer impact then prioritizes which owner acts first. The owner column decides who receives the ticket, the retry-safety column decides whether the ticket carries a retry instruction, and the impact column decides the order of work. Once the owner is named, handling rate limits, outages, and failover covers the response each class needs.
Routing and escalation from one row
At the model-call boundary, the provider holds the primary path and the gateway holds the observation path, because the gateway sees the request and the response and can attach that evidence to the provider ticket. At the application or prompt layer, a high customer-impact score moves the fix ahead of lower-impact work. Client-side request construction faults stay with client integration until the fault is corrected.
Customer impact is a dimension you define. The published taxonomies report no impact data, so the impact assignments in the table come from your own published methodology, not from MixRoute telemetry. If your queue is backing up, that impact column is what tells you which fix goes first. Routing and escalation collapse into a single lookup: read the origin class, read the retry-safety cell, send the ticket to the owner named in the owner cell. The evidence-source column assumes the instrumentation in LLM API observability metrics, traces, and alerts.
What makes a failure taxonomy testable?
Testability is what makes the origin table decidable in practice. Write failure-mode definitions independently and self-contained, split modes that are too broad, merge duplicates, and every new trace maps to exactly one mode.
Split and merge, one criterion
Apply split-when-too-broad and merge-when-duplicate together as a single testability criterion. A mode that absorbs several distinct behaviors cannot decide a trace, so it splits. Two labels that describe the same gap collapse into one, the way the recipe-domain pair Missing Serving Size and Missing Portion Information merge into a single mode. Those examples come from the recipe domain, so transferring the criterion to LLM API failure modes is a judgment you state, not a fact the published sources assert. A self-contained definition names its triggers and its exclusions, so a new trace either matches it or it does not.
Symptom classes and root causes
The agentic-fault study reports agentic system failures across 5 fault dimensions, 13 symptom classes, and 12 root cause categories. Keep the two apart when you map a trace. One root cause can produce several symptoms, and one symptom class can have several root causes, so record the symptom class first and search for the root cause second. The origin table classifies at the trace level, and that separation keeps an origin assignment from drifting into diagnosis.
How well-validated is a practical failure taxonomy?
One figure, with a boundary attached to it. The survey in Characterizing Faults in Agentic AI (arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31) asked practitioners whether its agentic-AI fault taxonomy covered the faults they had met in practice. In the results section, 88 of 105 surveyed practitioners, about 83.8%, said it covered those faults, and 17 respondents, about 16.2%, named fault types the taxonomy missed.
The 83.8% coverage figure and its boundary
That survey validated the paper’s own agentic-AI fault taxonomy. It does not validate an operator-defined origin table. The 83.8% is self-reported survey data and reads as a survey snapshot, never as MixRoute first-party evidence, and it means the starting taxonomy is broadly validated without being exhaustive. If you adopt a taxonomy into your own stack, treat the gap rate as a design constraint: expect a residual unclassified tail and name where that tail lies in your tests. One loose thread in the paper: v1 reports 145 practitioners for the developer study in its abstract, its method summary, and its results section, while the coverage question in section 3.3.2 is reported over 105 responses, and the paper does not reconcile the two. 83.8% is the percentage the paper prints, and it matches 88 of 105.
Which adjacent failure classes stay outside this taxonomy?
Three adjacent areas stay outside the origin taxonomy: prompt injection, multi-agent collaboration failures, and the MAD dataset reference path. Each one sits outside for its own reason, and the line is worth drawing rather than leaving implicit.
Prompt injection is adjacent, not a class
Prompt injection is ranked LLM01, the top risk in the OWASP GenAI LLM Top 10 published on 4 August 2026, with direct variants that inject instructions into the prompt and indirect variants that hide instructions in content the model later reads. It is a distinct security vulnerability, not a runtime failure mode. An origin-based taxonomy classifies failures that happen during a legitimate call, so prompt injection stays adjacent, and detection belongs to a security control rather than to the retry decision.
Multi-agent failures and the MAD dataset
Multi-agent coordination failures get a named theme in v1 section 3.3.2, covering misinterpreted inter-agent messages, cascading prompt injection across agents, and circular task dependencies, and v2 carries the same theme into its abstract. Leaving them out is a scope decision, not an absence of material, because agent-to-agent coordination is a different unit of failure from one client calling one provider, and the retry decision here is made per model call. The MAD dataset is referenced for its download location, not for classification. The scope is single model-call API failures from one client to one provider.
When a new trace arrives, the lookup does not change: origin class, retry-safety cell, owner cell. Apply the four origin classes to your next failure: decide which failures to retry and which owner acts first.
FAQ
What is an LLM API failure taxonomy?
It is a fixed, hierarchical structure of error categories that maps each observed failure to exactly one actionable bucket. The structure stays stable as models change, and each bucket carries four assignments: a retry-safety judgment, a customer-impact level, an owner, and an evidence source. The output is a decision, not a label. That is the part that separates it from an error list: an error list tells you what happened, while the taxonomy tells you whether to retry and whose queue the ticket lands in.
What are the main origins of LLM API failures?
Four, and the sort starts with one question: does a response body exist yet? If the call died during prompt or template loading, or during model initialization, the model never ran, so the origin is pre-inference or orchestration. If a completed response violates your prompt contract, the origin is output content. Invalid or malformed responses and protocol errors at the API edge belong to the model-call boundary, and request construction faults, key errors, and quota configuration errors sit on the client side.
Which LLM API failures are retry-safe?
Only the classes that are both transient and free of side effects. A transient timeout at the model-call boundary is a retry candidate when the call is idempotent, and a bounded attempt count with backoff keeps a repeat from compounding. A malformed response caused by a prompt defect reproduces itself on the next attempt, so it goes to the owner instead. An output-content failure is corrected by regeneration, not by a transport retry. Because the published taxonomies carry no frequency, severity, retry-outcome, or impact data, the retry call itself comes from a methodology you publish.
How do you detect failure modes automatically?
With an LLM-as-Judge pass. You write a judge prompt that defines one failure mode, then have the judge label each trace as exhibiting that mode or not. The prompt works like a failure-mode definition: self-contained, non-overlapping, and decidable, so a trace either matches it or it does not. The evaluator labels then feed the evidence-source column of the classification table.
How many categories do published taxonomies use?
ErrorAtlas defines 17 top-level error categories and is model-agnostic, so the same 17 cover a trace from any provider. AgentErrorTaxonomy takes the other shape: a modular set spanning memory, reflection, planning, action, and system-level operations, where each domain can be extended or updated on its own. A fixed count is one list to apply to every trace, and a modular set lets one domain move without touching the rest.
How do you write a good failure-mode definition?
Write each definition independently and self-contained, split modes that are too broad, and merge duplicates, such as Missing Serving Size and Missing Portion Information. Split and merge act as one testability criterion, and every new trace should map to exactly one mode. A definition that names its triggers and its exclusions is the test: if you need an “or” to hold two behaviors, you are looking at two modes.
How well-validated are these taxonomies?
One practical taxonomy covers 88 of 105 surveyed practitioners (83.8%), with about 16% (17 respondents) reporting gaps, so it is broadly validated without being exhaustive. The figure is a survey snapshot from arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31, and it applies to that paper’s agentic-AI fault taxonomy, not to an operator-defined origin table. Self-reported coverage like that sets an expectation for your own rollout: keep a place for traces that match nothing, so the residual tail shows up in your counts instead of disappearing into the nearest category.
Is prompt injection part of an LLM API failure taxonomy?
No. It is a distinct security vulnerability, ranked LLM01, the top risk in the OWASP GenAI LLM Top 10 published on 4 August 2026, and it sits outside an origin-based runtime failure taxonomy. Direct variants put instructions into the prompt, and indirect variants hide instructions in content the model later reads, which means a trace that looks like a strange output can be adversarial input. Those traces belong with whoever owns your security controls, not with the retry decision, because the origin taxonomy covers failures during legitimate calls.
