LLM API Failure Taxonomy: Classify Errors by Origin and Retry Safety
An LLM API failure taxonomy is a fixed, hierarchical classification structure. It maps each observed LLM API failure to exactly one origin class, then attaches four assignments to that class: a retry-safety judgment, a customer-impact level, an owner, and an evidence source. The output of the mapping is a decision, not a label. Once a failure sits in its origin class, an operator answers two questions in order: should this call be retried, and which owner acts first?
The origin classes come from published catalogues of failures that occur before or at the model call, including template-loading exceptions, failed initialisation, and malformed responses. Retry safety and customer impact are not taken from any published source. The operator defines both dimensions and applies them under a published methodology. The single validation figure used later comes from the arXiv study “Characterizing Faults in Agentic AI” (arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31). No latency, error-rate, failover-timing, or savings figures are asserted here, because the published sources carry no frequency, severity, or outcome data.
What published taxonomies already organize LLM failure modes?
The field already has structure in the form of two published taxonomies, ErrorAtlas and AgentErrorTaxonomy. ErrorAtlas is a model-agnostic taxonomy with 17 top-level error categories that the authors plan to periodically update as new benchmarks and models emerge, and AgentErrorTaxonomy is a modular classification spanning memory, reflection, planning, action, and system-level operations. Both provide the structural anchor for the taxonomy on this page.
ErrorAtlas: the 17-category anchor
ErrorAtlas organizes LLM failure modes into 17 top-level error categories. The published set is 17 categories and the authors state they plan to periodically update it as new benchmarks and models emerge (arXiv 2601.15812v2, checked 2026-08-31). Model-agnostic means the same 17 categories apply across models, so a trace from one provider maps onto the same category set as a trace from another provider. ErrorAtlas demonstrates that a bounded category count is enough to structure the whole field, which is the property this page adopts from it.
AgentErrorTaxonomy: the modular five-domain classification
AgentErrorTaxonomy takes a modular shape and covers five domains: memory, reflection, planning, action, and system-level operations. Modular means each domain extends or updates separately from the others. A different study, arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31, groups runtime exceptions during prompt or template loading, invalid or malformed model responses, and unsuccessful LLM initialisation under a dimension it calls Agent Cognition and Orchestration, listing all three as consequences of LLM misconfiguration. That placement drives the origin axis in the next section.
When surveying current taxonomies, the condition this page follows is to adopt the category structure from published taxonomies as the anchor. The published categories provide the skeleton, and the origin classes provide the decision axis.
How should failures be classified by origin?
Under this taxonomy, origin is the axis that separates failures by where they happened in the call path. Runtime exceptions during prompt or template loading, invalid or malformed model responses, and unsuccessful LLM initialisation are catalogued under the dimension Agent Cognition and Orchestration in v1 of that study, so they originate before or at the model-call boundary rather than in the correctness of the model’s completed output. A loading exception, by definition, means the model call never ran. An output-content failure, by definition, can only be judged after a complete response exists.
Four origin classes and the decision table
When classifying by origin, the decision is to treat orchestration or pre-inference failures as a distinct origin class, separate from output-content failures. The table below maps four origin classes to retry safety, customer impact, owner, and evidence source: one row per origin class, and one decision per row.
| Origin class | Example failures | Retry safety | Customer impact | Owner | Evidence source |
|---|---|---|---|---|---|
| Orchestration or pre-inference | Runtime exceptions during prompt or template loading; unsuccessful LLM initialisation | Retry-safe when transient and free of side effects, judged under operator methodology | Wide when a shared template or model configuration is broken | Gateway or client integration | Application and gateway request logs |
| Model-call boundary | Invalid or malformed model responses; protocol and transport errors at the API edge | Transient timeout and transport classes are retry candidates under idempotency; malformed-response classes route to the owner | Affects every dependent call waiting on the response | Model provider or gateway path | Gateway response logs and provider status records |
| Output content | Completed responses that violate the prompt contract | Not a transport retry; regenerate under operator policy or escalate | Affects downstream response correctness | Application or prompt layer | Output review and evaluator labels |
| Client side | Request construction faults; key and quota configuration errors in the client | Retry only after the request fault is corrected | Limited to the affected client | Client integration | Client and gateway request logs |
The table assigns one owner per origin class and one evidence source per owner, which is what makes a retry and escalation decision executable from a single row. The published quotes list these modes without frequency, severity, retry-safety, or customer-impact data, so no claim about retry outcome or impact is derivable from them.
Which LLM API failures are retry-safe?
Retry safety depends on whether the failure is transient and free of side effects. Because the published taxonomies report no frequency, severity, or retry-outcome data, operators must apply their own methodology and publish an evidence block for any telemetry claims. The retry-safety column in the table is therefore a framework, not a measurement.
Under that framework, a transient timeout at the model-call boundary is a retry candidate when the call is idempotent. A malformed model response produced by a prompt defect would reproduce the same defect on retry, so the framework routes it to the owner instead. An output-content failure is not a transport failure at all, because the request reached the model and produced a response; regeneration under operator policy is the correction, not a retry of the same call.
Retry decision rules
Operators apply three checks in order. First, is the failure transient? Second, is a retry free of side effects, such as duplicate charges or duplicate downstream writes? Third, does the retry have a bounded number of attempts and a backoff policy? Transient, in this framework, means the failure does not persist when the call is repeated under the same inputs. Free of side effects means a retry does not double-charge, double-write, or double-notify. Rate limiting is the most common transient class at this boundary, and 429 rate limit errors when using AI APIs have their own diagnosis path. When any check fails, the trace routes to the owner named in the table. The operator’s evidence block should state how both conditions were tested, because the published taxonomies do not supply that testing. This page asserts no latency, 429-rate, failover-duration, or savings figures, and a published methodology block must accompany any telemetry claim before that claim may be asserted.
How do origin, retry safety, and impact map to an owner?
Classifying by owner assigns orchestration and pre-inference failures to the gateway or client integration, model-call boundary failures to the provider or gateway path, and output-content failures to the application or prompt layer. Customer impact is then used to prioritize which owner acts first. The owner column decides who receives the ticket, the retry-safety column decides whether the ticket carries a retry instruction, and the impact column decides the order of work. Once the owner is named, handling rate limits, outages, and failover covers the response each class needs.
Routing and escalation from one row
At the model-call boundary, the provider holds the primary path and the gateway holds the observation path, because the gateway sees the request and the response and can attach the evidence to the provider ticket. At the application or prompt layer, a high customer-impact score moves the fix ahead of lower-impact work. Client-side request construction faults stay with client integration until the fault is corrected.
Customer impact is an operator-defined dimension under this taxonomy. The published taxonomies report no impact data, so the impact assignments in the table are made under the operator’s own published methodology, not from MixRoute telemetry. Routing and escalation become a single decision: look up the origin class, read the retry-safety cell, and send the ticket to the owner named in the owner cell. The evidence-source column assumes the instrumentation described in LLM API observability metrics, traces, and alerts.
What makes a failure taxonomy testable?
Failure-mode definitions must be written independently and self-contained, and a taxonomy should split modes that are too broad and merge duplicates, so that applying a definition to a new trace is unambiguous and every new trace maps to exactly one mode. Testability is the property that makes the origin table decidable in practice.
Split and merge as one criterion
When structuring any failure taxonomy, apply split-when-too-broad and merge-when-duplicate together as a single testability criterion. A mode that absorbs several distinct behaviors cannot decide a trace, so it splits. Two labels that describe the same gap collapse, as with the recipe-domain pair Missing Serving Size and Missing Portion Information, which merge into one mode. The split or merge examples come from the recipe domain, so transferring the criterion to LLM API failure modes is a judgment the operator states, not a fact the published sources assert. An independent, self-contained definition names its triggers and its exclusions, so a new trace either matches it or does not.
Symptom classes versus root causes
The agentic-fault study reports agentic systems failures across 5 fault dimensions, 13 symptom classes, and 12 root cause categories. The practical consequence is to distinguish symptom classes from root causes when mapping traces. One root cause can produce several symptoms, and one symptom class can have several root causes, so the operator records the symptom class first and searches for the root cause second. The origin table classifies the trace level, and the symptom-to-root-cause distinction keeps that origin assignment from drifting into diagnosis.
How well-validated is a practical failure taxonomy?
The survey in Characterizing Faults in Agentic AI (arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31) provides the only coverage figure this page reports. The paper surveyed practitioners about whether its agentic-AI fault taxonomy covered the faults they had met in practice. In the results section, 88 of 105 surveyed practitioners, about 83.8%, said it covered those faults, and 17 respondents, about 16.2%, named fault types the taxonomy missed.
The 83.8% coverage figure and its boundary
That survey validated the paper’s agentic-AI fault taxonomy, not the taxonomy on this page. When reporting validation of the starting taxonomy, this page publishes the 83.8% coverage figure with explicit bounded-validation framing: the coverage is self-reported survey data and must be read as a survey snapshot, never as MixRoute first-party evidence. The figure means the taxonomy is broadly validated but not exhaustive. v1 reports 145 practitioners for the developer study in its abstract, its method summary and its results section, while the coverage question in section 3.3.2 is reported over 105 responses, and the paper does not reconcile the two. 83.8% is the percentage the paper prints and it matches 88 of 105. For the taxonomy on this page, the gap rate operates as a design constraint: any adopted failure taxonomy should expect a residual unclassified tail, and testing should name where that tail lies.
Which adjacent failure classes stay outside this taxonomy?
Three adjacent areas stay outside the origin taxonomy: prompt injection, multi-agent collaboration failures, and the MAD dataset reference path. Each is documented here so the boundary is explicit rather than implicit.
Prompt injection is adjacent, not a class
Prompt injection is ranked LLM01, the top risk in the OWASP GenAI LLM Top 10 published on 4 August 2026, with direct variants that inject instructions into the prompt and indirect variants that hide instructions in content the model later reads. The class is a distinct security vulnerability, not a runtime failure mode. An origin-based taxonomy classifies failures that happen during a legitimate call, so prompt injection stays adjacent, and detection belongs to a security control rather than to the retry decision.
Multi-agent failures and the MAD dataset are out of scope
The sources discussed multi-agent coordination failures substantively: v1 section 3.3.2 has a named theme Multi-agent coordination failures covering misinterpreted inter-agent messages, cascading prompt injection across agents and circular task dependencies, and v2 carries the same theme into its abstract. The exclusion here is a scope decision, not an absence of material: agent-to-agent coordination is a different unit of failure from one client calling one provider, and the retry decision on this page is made per model call. The MAD dataset is referenced for its download location rather than for classification. The scope is single model-call API failures from one client to one provider.
When a new trace arrives, the decision procedure on this page is fixed. Find the origin class, read the retry-safety cell, confirm the customer impact, and route to the owner named in the table. One trace, one row, two decisions. Apply the four origin classes to your next failure: decide which failures to retry and which owner acts first.
FAQ
What is an LLM API failure taxonomy?
An LLM API failure taxonomy is a fixed, hierarchical structure of error categories that lets you map each observed failure to one actionable bucket. The structure stays stable as models change, and each bucket carries a retry-safety judgment, a customer-impact level, an owner, and an evidence source. The purpose is a decision: retry this failure, or escalate it to its owner.
What are the main origins of LLM API failures?
Failures split into pre-inference or orchestration-layer origins, such as runtime exceptions during prompt loading or failed model initialization, and output-content failures inside the model’s completed response. Two more classes complete the boundary: the model-call boundary catches invalid or malformed model responses, and the client side catches request construction faults.
Which LLM API failures are retry-safe?
Retry safety depends on whether the failure class is transient and free of side effects, and the published taxonomies report no frequency, severity, retry-outcome, or impact data, so retry decisions must come from an operator’s own published methodology. A transient boundary timeout is a retry candidate; an output-content failure is corrected by regeneration, not by transport retry.
How do you detect failure modes automatically?
Use LLM-as-Judge with a judge prompt that defines the failure mode and then labels each trace as exhibiting it or not. The judge prompt works like a failure-mode definition: self-contained, non-overlapping, and decidable. The evaluator labels then feed the evidence-source column of the classification table.
How many categories do published taxonomies use?
ErrorAtlas defines 17 top-level error categories, while AgentErrorTaxonomy spans memory, reflection, planning, action, and system-level operations. The shapes differ, one fixed count and one modular set of domains, but both show that a bounded category list is enough to structure the field.
How do you write a good failure-mode definition?
Write each failure-mode definition independently and self-contained, split modes that are too broad, and merge duplicates such as Missing Serving Size and Missing Portion Information. Split and merge act as one testability criterion, and every new trace should map to exactly one mode.
How well-validated are these taxonomies?
One practical taxonomy covers 88 of 105 surveyed practitioners (83.8%), with about 16% (17 respondents) reporting gaps, so it is broadly validated but not exhaustive. The figure is a survey snapshot from arXiv 2603.06847v1, the 6 March 2026 version, checked 2026-08-31, and it applies to that paper’s agentic-AI fault taxonomy, not to the taxonomy on this page. Self-reported coverage of that kind sets an expectation: any failure taxonomy should leave room for a residual tail that practitioners will name.
Is prompt injection part of an LLM API failure taxonomy?
Prompt injection is a distinct security vulnerability, ranked LLM01, the top risk in the OWASP GenAI LLM Top 10 published on 4 August 2026, and sits outside an origin-based runtime failure taxonomy. Direct variants inject instructions into the prompt, and indirect variants hide instructions in content the model later reads. The origin taxonomy handles failures during legitimate calls, not adversarial input.
