What Is a Token? The Unit Behind Every AI API Bill

A token is a chunk of text, usually a common word or a piece of a longer one, and it is the unit every LLM reads, writes, bills, and limits in. In English, one token averages about four characters or three quarters of a word, so roughly 750 words comes to about 1,000 tokens. But that average hides the thing that costs people money: the same text produces different token counts on different providers, and the same meaning in a different language can cost many times more.

This guide covers what a token actually is, why counts vary, and every part of your bill and your architecture that tokens quietly control.

Why models do not use words

The obvious design would be one number per word. It fails for a simple reason: English has well over a hundred thousand words in current use, plus names, jargon, slang, and typos, and a model with a fixed word list has no way to represent anything outside it. Type a new product name or a misspelling and the system has nothing to point at.

The opposite extreme, one number per character, handles anything you throw at it but makes every sequence enormously long, and the model has to learn meaning from scratch out of individual letters.

Subword tokenization is the compromise everything uses now. Break text into pieces bigger than a character and usually smaller than a word. Common words stay whole. Rare words split into recognizable fragments. The result is a manageable vocabulary, typically somewhere between thirty thousand and a couple hundred thousand entries, that can still represent any text at all, including words the model has never seen.

The dominant method is byte pair encoding. It starts with individual characters and repeatedly merges the most frequently occurring adjacent pair into a single new unit, thousands of times over, until it reaches a target vocabulary size. The merges are learned from a training corpus and then applied deterministically to any new text.

That last detail matters more than it sounds. The splits reflect what was statistically common in the training data, not grammar. A word gets its own token because it appeared often, not because it is a word. This is why tokenization sometimes looks arbitrary: a common compound may be a single token while a rarer one built from the same parts splits into several.

The rules of thumb, and what breaks them

For ordinary English prose, these hold up well enough for planning:

Now the caveats, which is where estimates go wrong in production.

Technical and specialized vocabulary costs more. Everyday words are single tokens because they were frequent in training. Domain jargon, product names, and unusual terms fragment into several pieces each.

Code tokenizes worse than prose. Identifiers in camelCase or snake_case split at boundaries, punctuation is dense, and indentation consumes tokens. The same character count of code is reliably more tokens than the same character count of English.

Structured data is expensive. JSON is largely punctuation and repeated key names. Every brace, quote, and comma is billable, and the keys repeat on every record.

Numbers are unpredictable. Long numbers do not map cleanly to single tokens and often split into several pieces in ways that are hard to guess.

Whitespace counts. Leading spaces are typically part of the token that follows, and repeated indentation adds up in long documents.

The practical rule: estimates are fine for a rough budget and useless for anything precise. If a number matters, count it rather than approximating it.

The same text, different counts

Here is the part with real financial consequences, and it is the one most people never think about.

Every model family uses its own tokenizer, trained on its own corpus, with its own vocabulary. Send identical text to two different providers and you get two different token counts. Not wildly different for ordinary English, but different, and different enough to matter at volume.

The consequence is arithmetic. Your bill is the rate multiplied by the token count. The pricing page shows you the rate. It says nothing about the count. So two models advertising the same price per million tokens can charge you meaningfully different amounts for the exact same job, and the cheaper sticker can be the more expensive model.

This also applies within a single provider over time. Tokenizers change between model generations, and vocabularies have grown considerably, from around fifty thousand entries in earlier models to a couple hundred thousand in more recent ones. A larger vocabulary generally means fewer tokens for the same text, but the direction is not guaranteed for every kind of content.

The habit that protects you: never carry a token count across a model change. Re-measure. Your old number described a tokenizer you are no longer using.

The language tax

This is the least known consequence and the largest in magnitude.

Researchers at Oxford analyzed seventeen tokenizers against parallel translations of the same text and found that identical meaning, expressed in different languages, produced token counts differing by up to fifteen times. The disparity persisted even in tokenizers explicitly trained for multilingual support, and even character and byte level models still showed more than fourfold differences between some language pairs.

The mechanism is straightforward once you see it. Tokenizer vocabularies are learned from training corpora dominated by English, so English words earn their own tokens through sheer frequency. Text in languages with different scripts or less representation in that corpus gets shattered into many small fragments instead.

Because API pricing and context limits are uniform per token, that fragmentation lands directly on the user in three separate ways:

Cost. The same request costs more in one language than another, for identical meaning.

Effective context. A context window is measured in tokens, so a fragmented language fits materially less actual content into the same window.

Latency. More tokens means more to process and more to generate, so responses take longer.

If you are building for a multilingual audience, this belongs in your cost model rather than arriving as a surprise. Budget per language, not per request. Newer tokenizers with larger vocabularies narrow the gap, but no current approach eliminates it.

Everything tokens control

Tokens are not just a billing unit. Most of the constraints you work within are counted in them.

Your bill. Input and output are billed separately, and output is billed at a higher rate on essentially every provider, because generating a token genuinely costs more compute than reading one.

Your context window. A stated limit is tokens, not words or characters. A document that looks comfortably small in characters can overrun a window once tokenized, especially if it is code, structured data, or non-English text.

Your rate limits. Many providers limit tokens per minute alongside or instead of requests per minute, so your throughput ceiling is denominated in tokens too.

Your caching eligibility. Prompt caching has minimum prefix lengths measured in tokens, and they are model-specific. A prompt below the threshold silently does not cache, with no error.

Your latency. Because responses are generated one token at a time while prompts are processed in parallel, output token count is the dominant driver of how long a request takes.

That is five separate systems keyed to one unit, which is why understanding it pays for itself.

How to actually count them

Estimating is fine for a rough plan. For anything that touches a budget or a limit, measure.

Read the usage field on your responses. Every provider returns the exact input and output token counts for the request it just processed. This is ground truth, it costs nothing extra, and it is the single most useful thing to log. Log it per task type rather than globally, because an average hides the specific workload that is expensive.

Use the tokenizer that matches the model. Counting libraries are tokenizer-specific. A count from one provider’s tokenizer is an approximation for another’s, not an answer.

Measure your real inputs, not samples. Your production traffic includes long documents, unusual characters, and pasted structured data. A clean test sentence will underestimate.

Re-measure after any model change. This is worth repeating because it is the most common expensive mistake. A new model can change your token count without changing your prompt or the advertised rate, which means your bill moves while every number you were watching stays the same.

Where this leaves you

The single sentence worth taking away: your bill is a rate multiplied by a count, and only the rate is published.

That is why cost per task is a more useful metric than cost per token. Cost per task captures the count, the tokenizer, the language, and the content type all at once, which means it survives a model change, a tokenizer change, and a price change. Cost per token survives none of them.

It is also why comparing providers on their pricing pages alone is unreliable. The only comparison that means anything is running your own representative workload through each option and measuring what it actually costs to complete.

MixRoute puts every major model behind one OpenAI compatible endpoint with zero markup on provider pricing, which makes that comparison a loop over model strings rather than a set of separate integrations built just to run the test. Start building on MixRoute

FAQ

What is a token in AI? A token is the basic unit of text a language model processes, produced by splitting text into subword pieces. Common words are usually one token, while rare or long words split into several. In English a token averages about four characters or three quarters of a word, so roughly 750 words comes to about 1,000 tokens.

How many tokens is 1,000 words? Roughly 1,300 to 1,500 tokens for ordinary English prose. Technical writing, code, structured data like JSON, and non-English text all produce more tokens for the same word count, so treat this as a planning estimate rather than a precise figure.

Why do different models count tokens differently? Each model family uses its own tokenizer with its own learned vocabulary, so identical text produces different counts on different models. Tokenizers also change between generations of the same provider’s models. Since your bill is the rate multiplied by the count, two models at the same advertised price can cost different amounts for the same work.

Do non-English languages cost more to process? Yes, and often substantially. Research across seventeen tokenizers found the same meaning translated into different languages can produce token counts differing by up to fifteen times, because tokenizer vocabularies are learned from largely English training data. This raises cost, reduces effective context, and increases latency for the same request.

How do I count tokens accurately? Read the usage field returned with every API response, which reports the exact counts for the request just processed. For counting before you send, use a tokenizer library matching the specific model. Avoid character-based estimates for anything that touches a budget or a hard limit.

Is a token the same as a word? No. Common words are often a single token, but longer or unusual words split into multiple tokens, and punctuation and whitespace consume tokens too. On average one token is about three quarters of a word in English, but the ratio varies considerably by content type and language.

The bottom line

A token is a subword chunk of text, and it is the unit behind your bill, your context window, your rate limits, your caching eligibility, and your latency. About four characters in English, less efficient for code and structured data, and much less efficient for many non-English languages.

The consequence to hold onto: the rate is published and the count is not. Estimate for planning, measure for decisions, log the usage field per task, and re-measure any time the model changes, because the count can move while everything you were watching stays still.

MixRoute gives you every major model behind one OpenAI compatible endpoint with zero markup, so measuring what a task truly costs across models is a string change rather than a project. Start building on MixRoute