Self-Hosted vs Cloud AI for Job Search Apps: The Real Math (2026)

Self-hosting an AI model is a data control decision, not a cost decision. For a job search product parsing resumes, a single rented H100 has to process roughly 6 million resumes a month before it beats calling an API. Almost nobody is at that volume. The teams that self-host and are right about it did it for GDPR, not for the bill.

This guide runs the actual arithmetic, then covers the case where self-hosting genuinely wins, because it exists and it matters more in this vertical than most.

What each option actually means

Cloud or API. You call a hosted model over HTTP. You are billed per token. Someone else owns the GPU, the scaling, the uptime, and the model updates. Your candidate data leaves your infrastructure.

Self-hosted. You run open weights (Llama, Qwen, DeepSeek, Mistral) on GPUs you rent or own. You are billed per hour, not per token. You own the serving stack, the batching, the scaling, and the 3am page. Your candidate data never leaves.

That billing difference is the whole article. Per token versus per hour. Everything below follows from it.

The breakeven nobody runs

Take the workload from the companion guide on model selection: resume parsing at roughly 2,000 input tokens and 400 output tokens per resume.

On DeepSeek V4 Flash at $0.14/$0.28 per MTok, that is $0.000392 per resume.

Now price one H100. Rental rates land between roughly $2 and $4 per GPU-hour on specialised GPU clouds, with hyperscalers running considerably higher. A tracker covering 46 providers puts the current average at $3.39 per hour. Run one 24/7 for a month and here is where the lines cross:

Read the middle row again. You need to parse 208,000 resumes a day, every day, before the GPU wins.

At 10,000 resumes a month, a volume most job search startups never reach, the API costs $3.92 and the GPU costs $2,441. That is a 620x gap in the API’s favour.

Methodology: API rate from DeepSeek’s published pricing. GPU rates from public 2026 provider rate cards. Breakeven is monthly GPU cost divided by per resume API cost. Excludes engineering time, storage, egress, and the serving stack, all of which push the breakeven higher still.

The part that makes it worse: you pay for the gaps

There is a second problem hiding in that table, and it is the one that actually kills self-hosting for this vertical.

An API bills you for tokens. A GPU bills you for time. Idle time is still time.

Job search traffic is bursty and diurnal. People update resumes at night and on weekends. Recruiters run searches during business hours. Your traffic has enormous gaps, and your GPU bills through every one of them at full rate.

At 25% utilisation, a realistic number for a young product, your $3.39 GPU is really a $13.56 GPU. The breakeven in the table above quadruples.

An API at 25% utilisation costs exactly the same per token as an API at 100% utilisation. That is not a small advantage. For bursty workloads it is the entire ballgame.

And one more turn of the screw. To hit the 6.2M breakeven you would need to sustain roughly 5,765 tokens per second, around the clock, with zero idle. A single H100 will not hold that for a model worth using, so you add a second GPU. The cost doubles and the breakeven moves again. The volume that justifies the hardware is the same volume that outgrows it.

When self-hosting genuinely wins

Everything above is an argument for the API. Here is the honest other side, and for a job search product it is stronger than in most verticals.

Data residency and GDPR. This is the real reason, and it is a good one. A resume is dense personal data: name, address, employment history, sometimes age and nationality. If you are processing EU candidate data, sending it to a US API is a decision with legal weight, not just a technical one. Self-hosting inside your own perimeter removes that question entirely. If your buyer is an EU enterprise with a procurement team, this may not be negotiable regardless of what the arithmetic says.

Worth checking before you rack servers over it: several providers now offer zero data retention terms and regional processing, which addresses part of this without the GPU. [verify: current ZDR and EU residency terms per provider]

Fine-tuning on proprietary data. If you have a large labelled corpus of resumes and outcomes and you have proven a tuned small model beats a general one on your task, self-hosting is how you ship it.

No rate limits. At real scale, provider rate limits become an architecture constraint. Your own GPU has exactly one limit, which is the GPU.

Predictable cost at high steady volume. Above the breakeven, with flat traffic, the GPU stops being a liability and becomes a fixed cost you control.

Notice what is not on that list: saving money at startup volume. That is the belief this article exists to correct.

The hybrid nearly everyone actually lands on

The framing is not either or. Different tasks have different answers, and the sane architecture mixes them.

Most teams self-host exactly one thing, the task touching raw PII, and call APIs for everything else. That is a defensible architecture. “We self-host to save money” is not.

Running both without rewriting your app

The hybrid above has a practical cost: two integrations, two auth models, two failure modes, two sets of retry logic.

It does not have to. Most self-hosted serving stacks (vLLM, TGI, Ollama) expose an OpenAI compatible endpoint. So does an AI API gateway. Point both at the same client interface and the decision of where a task runs collapses into a base URL and a model string.

from openai import OpenAI

# PII heavy work stays inside your perimeter
local = OpenAI(api_key="none", base_url="http://vllm.internal:8000/v1")

# Everything else goes out through the gateway
cloud = OpenAI(
    api_key="YOUR_MIXROUTE_KEY",
    base_url="https://api.mixroute.ai/v1",  # [verify: exact base URL]
)

parsed = local.chat.completions.create(model="qwen3-8b", messages=[...])
letter = cloud.chat.completions.create(model="claude-sonnet-5", messages=[...])

Same SDK. Same call shape. The only thing that changes is which client you reach for, and that becomes a routing decision you can revisit later instead of a rewrite.

MixRoute is the cloud half of that setup: one OpenAI compatible API across [verify: model count] models, automatic failover when a provider drops, and zero markup on provider pricing. It does not solve data residency, and no gateway does. If your compliance team says the resume cannot leave the building, self-host that one task and route the rest.

FAQ

Is self-hosting AI cheaper than using an API? Almost never at startup volume. A rented H100 runs roughly $1,440 to $4,954 a month depending on provider. Against DeepSeek V4 Flash at $0.14/$0.28 per MTok, you would need to parse over 6 million resumes a month at average GPU rates before the hardware wins. Self-hosting is a data control decision, not a cost decision.

How many requests before self-hosting an LLM breaks even? For resume parsing at roughly 2,400 tokens per document, the breakeven against a cheap API model is between 3.7 million and 12.6 million documents a month depending on your GPU rate. Bursty traffic pushes it higher, because a GPU bills for idle hours and an API does not.

Should I self-host AI for GDPR compliance? It is the strongest argument for self-hosting a job search product, since resumes are dense personal data. Before committing, check whether your provider offers zero data retention terms and regional processing, which addresses part of the concern without running GPUs.

What is the cheapest way to run resume parsing at scale? An API on a small model, until you are past roughly 200,000 resumes a day. At 10,000 resumes a month the API costs about $3.92 against $2,441 for a dedicated H100.

Can I use self-hosted and cloud models together? Yes, and most production teams do. Self-hosted stacks like vLLM and TGI expose OpenAI compatible endpoints, as do API gateways. Keep PII heavy tasks local and route everything else out, using the same SDK for both.

Do I need an H100 to self-host a model for resume parsing? Not necessarily, since smaller models run on cheaper hardware. But the structural problem does not change with the GPU: you are still paying per hour instead of per token, and your traffic still has gaps.

The bottom line

The instinct is that self-hosting saves money. The arithmetic says the opposite by two or three orders of magnitude at the volume most job search products actually run.

Self-host when the data cannot leave. Call an API when it can. Do not self-host to save money unless you are moving 200,000 resumes a day, and if you are, you already knew that.

MixRoute is the cloud half: every major model behind one OpenAI compatible endpoint, automatic failover, zero markup. Start building on MixRoute