Why AI Agents Cost So Much More Than You Budgeted

An AI agent does not cost the number of steps it takes multiplied by the price of one call. It costs far more, because every step re-sends everything that came before it. Twenty steps is not twenty times one call, it is closer to sixty six times, and the agent’s own output is a rounding error next to the context it re-reads. Research from Stanford’s Digital Economy Lab found agentic coding tasks consuming up to a thousand times more tokens than equivalent chat, driven almost entirely by input tokens rather than generated ones.

This guide explains the mechanism, shows the arithmetic, and covers what actually controls the number.

What an agent actually does

An agent is a loop. It receives a goal, decides on an action, calls a tool, reads the result, and decides again. It repeats until it believes it is finished or you stop it.

Each pass through that loop is a complete API call. And because a language model has no memory between calls, every single pass must carry the full state with it: the original goal, the system prompt, every tool definition, every action taken so far, and every result those actions returned.

That last sentence is the entire cost story. The model does not remember step three when it runs step four. You pay to remind it.

The context snowball

Here is the mechanism made concrete. Take an agent with a 3,000 token system prompt including tool schemas, generating roughly 300 tokens of reasoning per step, and receiving roughly 500 tokens back from each tool call. Illustrative figures, but the shape is what matters.

Read the last column. Doubling the steps from ten to twenty more than triples the cost. Going from twenty to fifty multiplies it by more than five.

The reason is that history accumulates and gets re-read. Step one carries the base prompt. Step twenty carries the base prompt plus nineteen steps of accumulated reasoning and tool output. Cost grows with the square of the step count, not linearly with it.

The practical consequence is a budgeting error most teams make once. Estimating an agent run as steps multiplied by the cost of a single call underestimates a twenty step run by roughly three times. Your forecast is not slightly optimistic. It is structurally the wrong shape.

Input dominates, which inverts everything you know

Break down where the tokens actually go in that twenty step run:

The agent’s output, the part that feels like the work, is almost nothing. Over ninety percent of what you pay for is the model re-reading material it already produced.

This inverts the usual cost intuition, and it matters because most cost advice is built for chat. In a normal request, output is the expensive side of the meter, billed at several times the input rate, so the standard guidance is to cap your output and keep responses short. In an agent loop the ratio flips so hard that the rate difference stops mattering: at thirty five input tokens per output token, the cheap side of the meter is where nearly all your money goes.

Stanford’s research on agentic coding tasks found exactly this, that the high cost sits in input rather than output, because the agent re-reads the original prompt and every prior response before each new action. Anthropic’s own engineering measurements put single agents at roughly four times the token consumption of chat interactions, and multi-agent systems at roughly fifteen times.

The three multipliers

History accumulation. The main one, described above. Unavoidable in principle, manageable in practice.

Tool schema overhead. Every tool an agent can access carries its schema in the prompt, on every step, whether the agent uses it or not. An agent with access to a dozen APIs pays for a dozen tool descriptions on every single pass. A tool you added months ago and the agent has never selected is still billing you on every step of every run.

Retries and course correction. When a tool returns unexpected data or a step fails, the agent tries again. Each retry is a full context retransmission at the current, already inflated size. Failures late in a run cost dramatically more than failures early in one.

You cannot forecast it the way you forecast chat

There is a second problem that is arguably worse than the absolute cost, and it is the reason finance teams struggle with agent budgets.

Chat is predictable. Users times conversations times average length gives you a workable forecast. Agent runs are not, because trajectories are stochastic. The same task, run twice, can take a different number of steps, call different tools, and hit different retries. Research measuring token consumption in agentic tasks found variance of up to thirty times between runs on the identical task.

That is not a forecasting inconvenience. It means an agent product priced on a flat fee has an unbounded cost per unit, and it explains why providers have been restructuring how agent workloads are billed, moving them onto explicit usage meters rather than bundling them into flat allowances. The economics of a loop with a variable step count do not fit a subscription.

What actually controls the number

Cache the stable prefix, and treat this as the top lever. Agents are close to the perfect caching workload and almost nobody exploits it fully. The system prompt and tool schemas are byte identical on every step, and in the run above that is twenty eight percent of all input tokens, re-sent twenty times. On top of that, the conversation history is append only, meaning each step’s prefix is the previous step’s prefix plus new content, which is exactly the shape a prefix cache is built for. Caching is a cost lever on almost any workload. On an agent loop it is the difference between viable and not.

Cut the tool list. Give each agent only the tools it plausibly needs. Two thousand tokens of unused tool schemas across a twenty step run is forty thousand tokens paid for capability that was never exercised. If you have one agent with twenty tools, consider several narrower agents instead.

Compact the history. Do not carry raw tool output forever. Summarize completed sub tasks, drop verbose intermediate results once their conclusion is extracted, and keep only what later steps actually need. This directly attacks the seventy two percent.

Cap the steps. A hard iteration limit is a cost control, not just a safety measure. Because cost grows quadratically, the tail of a long run is disproportionately expensive: the last five steps of a thirty step run cost far more than the first five.

Fail fast. Since retries retransmit the whole accumulated context, an agent that detects a dead end early saves much more than the same detection late.

Route by step, not by agent

Look at what an agent loop actually contains. Planning and decomposition, which need real reasoning. Tool selection, which is close to classification. Result parsing, which is extraction. Final synthesis, which needs quality because a human reads it.

Those are wildly different tasks, and most agent implementations run all of them on one model chosen for the hardest of them. That means the frontier model that was necessary for planning is also handling the tool selection step, thirty times per run, at frontier rates.

Routing per step type is the structural fix. Reasoning steps earn a strong model. Tool selection and result parsing are exactly the kind of high volume, low judgment work where quality saturates and a cheap model performs identically. Given that the same loop can run dozens of steps, moving the mechanical majority to a cheaper model compounds across every iteration.

The reason more teams do not do this is friction: reaching several models means several integrations, keys, and failure modes inside a loop that is already complex. That is the friction a gateway removes.

MixRoute puts every major model behind one OpenAI compatible endpoint with automatic failover and zero markup, so choosing a different model for a different step in your loop is a string change rather than another integration. Start building on MixRoute

FAQ

Why do AI agents cost so much more than chatbots? Because every step of an agent loop re-sends the entire history that came before it. The model has no memory between calls, so the goal, system prompt, tool schemas, prior reasoning, and every tool result must be retransmitted on each pass. Cost grows with the square of the step count rather than linearly, and measurements put agents at roughly four times chat token consumption, multi-agent systems at roughly fifteen times, and agentic coding tasks at up to a thousand times.

How do I estimate the cost of an agent run? Not by multiplying steps by the cost of one call, which underestimates a twenty step run by roughly three times. Model it as accumulating context: each step carries the fixed prompt plus everything generated so far. Then expect wide variance, since research has found the same task can consume up to thirty times more tokens on one run than another.

Why are input tokens the main cost in agent workloads? Because the agent re-reads far more than it writes. In a typical twenty step loop the generated output is around three percent of total tokens, while accumulated history that is re-read each step accounts for the large majority. This is the opposite of chat, where output usually dominates the bill, and it means output focused cost advice does not transfer to agents.

Does prompt caching help with AI agents? More than with almost any other workload. The system prompt and tool schemas are identical on every step, and conversation history is append only, so each step’s prefix extends the previous one. That is exactly the pattern prefix caching is designed for, and on a long agent loop it is often the single largest available saving.

How do I reduce AI agent costs? Cache the stable prefix first, since it applies to every step. Give each agent only the tools it needs, because unused tool schemas bill on every pass. Compact or summarize history rather than carrying raw tool output forever. Set a hard step cap, since late steps cost disproportionately more. And route different step types to different models rather than running tool selection on the model you chose for planning.

Why is agent cost so hard to predict? Because agent trajectories are stochastic. The number of steps, which tools get called, and how many retries occur all vary between runs of the identical task. This makes per unit cost effectively unbounded, which is why flat fee pricing struggles with agent products and why usage based metering has become the norm for them.

The bottom line

An agent is a loop that pays to re-read itself. Cost grows with the square of the step count, input tokens dominate to a degree that inverts normal cost advice, and the agent’s own output is a rounding error in the bill.

So budget it as accumulation rather than repetition, cache the prefix that repeats on every step, strip tools the agent does not use, compact the history, cap the iterations, and stop running mechanical steps on the model you picked for the hard ones.

MixRoute gives you every major model behind one OpenAI compatible endpoint with zero markup, so routing each step of a loop to the cheapest model that clears its bar is a string change instead of another integration. Start building on MixRoute