Self-Hosted vs Cloud AI for Job Search Apps: The Real Math (2026)

Self-hosting an AI model is a data control decision, not a cost decision. For a job search product parsing resumes, one rented H100 has to process roughly 1.9 million resumes a month before it costs less than calling DeepSeek V4.1 Flash through an API, and that assumes the GPU never sits idle. Almost nobody is at that volume. The teams that self-host and are right about it did it for GDPR, not for the bill. Prices and rates are as of October 19, 2026.

What each option actually means

Cloud or API. You call a hosted model over HTTP. You are billed per token. Someone else owns the GPU, the scaling, the uptime, and the model updates. Your candidate data leaves your infrastructure.

Self-hosted. You run open weights (Llama, Qwen, DeepSeek, Mistral) on GPUs you rent or own. You are billed per hour, not per token. You own the serving stack, the batching, the scaling, and the 3am page. Your candidate data never leaves.

That billing difference is the whole article. Per token versus per hour. Everything below follows from it.

The breakeven nobody runs

One H100 beats the API only above roughly 1.8 to 4.6 million resumes a month, depending on the GPU rate. Take the workload from the companion guide on model selection: resume parsing at roughly 2,000 input tokens and 400 output tokens per resume.

On DeepSeek V4.1 Flash at $0.30 input and $1.20 output per MTok, that is $0.00108 per resume.

Now price one H100. Silicon Data’s neo-cloud index put the H100 at $2.83 per GPU-hour on October 18, 2026. A GPU cloud listed it at $2.64 on demand the next day. AWS charges $55.04 an hour for a p5.48xlarge with eight H100s, which is $6.88 per GPU. Run one GPU 24/7 (720 hours) for a month and here is where the lines cross:

Read the middle row again. You need to parse about 63,000 resumes a day, every day, before the GPU wins.

At 10,000 resumes a month, a volume most job search startups never reach, the API costs $10.80 and the GPU costs $2,038. That is a gap of about 189x in the API’s favor.

Methodology: the API rate is the flat $0.30 / $1.20 per MTok listed on the MixRoute page for DeepSeek V4.1 Flash. DeepSeek’s own API charges half of its peak rate off peak (peak is 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays, excluding Chinese public holidays), and at that rate every breakeven above doubles. Breakeven is monthly GPU cost, rate times 720 hours, divided by the per resume API cost. The middle row also needs about 1,750 tokens per second of combined input and output, around the clock, so measure your model’s throughput on your own prompts before you size a GPU. Engineering time, storage, egress, and the serving stack are excluded, and all of them push the breakeven higher still.

The part that makes it worse: you pay for the gaps

There is a second problem hiding in that table, and it is the one that actually kills self-hosting for this vertical.

An API bills you for tokens. A GPU bills you for time. Idle time is still time.

Job search traffic is bursty and diurnal. People update resumes at night and on weekends. Recruiters run searches during business hours. Your traffic has enormous gaps, and your GPU bills through every one of them at full rate.

At 25% utilization, a realistic number for a young product, your $2.83 GPU is really an $11.32 GPU. The breakeven in the table above quadruples to about 7.5 million resumes a month.

An API at 25% utilization costs exactly the same per token as an API at 100% utilization. That is not a small advantage. For bursty workloads it is the entire ballgame.

The 1,750 tokens per second from the methodology note is an average. Traffic with gaps peaks well above its average, so the card has to be sized for the busy hour while you pay for all 720 hours.

When self-hosting genuinely wins

Self-hosting wins in four cases, and for a job search product the first is stronger than in most verticals.

Data residency and GDPR. This is the real reason, and it is a good one. A resume is dense personal data: name, address, employment history, sometimes age and nationality. If you are processing EU candidate data, sending it to a US API is a decision with legal weight, not just a technical one. Self-hosting inside your own perimeter removes that question entirely. If your buyer is an EU enterprise with a procurement team, this may not be negotiable regardless of what the arithmetic says.

Before you rack servers over it, compare what the API route already offers. OpenAI offers EU data residency and Zero Data Retention, both subject to its approval, and non-US residency also requires a Modified Retention amendment. Anthropic’s inference_geo setting lists global and us, so it does not cover a requirement that Claude inference stay in the EU. MixRoute’s documentation says it does not store request or response content and keeps system logs (timestamps, token counts, model name, response status) for 7 days. That describes the gateway, not the model provider behind it.

Fine-tuning on proprietary data. If you have a large labeled corpus of resumes and outcomes and you have proven a tuned small model beats a general one on your task, self-hosting is how you ship it.

No rate limits. At real scale, provider rate limits become an architecture constraint. Your own GPU has exactly one limit, which is the GPU.

Predictable cost at high steady volume. Above the breakeven, with flat traffic, the GPU stops being a liability and becomes a fixed cost you control.

Saving money at startup volume is not on that list.

The hybrid nearly everyone actually lands on

The framing is not either or. Different tasks have different answers, and the sane architecture mixes them.

A common split is to self-host exactly one thing, the task touching raw PII, and call APIs for everything else. That is a defensible architecture. “We self-host to save money” is not.

Running both without rewriting your app

The hybrid above has a practical cost: two integrations, two auth models, two failure modes, two sets of retry logic.

It does not have to. vLLM and Ollama both serve an OpenAI compatible API, and so does MixRoute. Point both at the same client interface and the decision of where a task runs collapses into a base URL and a model string.

from openai import OpenAI

# PII heavy work stays inside your perimeter
local = OpenAI(api_key="none", base_url="http://vllm.internal:8000/v1")

# Everything else goes out through MixRoute
cloud = OpenAI(
    api_key="YOUR_MIXROUTE_KEY",
    base_url="https://api.mixroute.ai/v1",
)

parsed = local.chat.completions.create(
    model="Qwen/Qwen3.8-27B",
    messages=[...],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
letter = cloud.chat.completions.create(model="claude-sonnet-5-5", messages=[...])

Same SDK. Same call shape. The only thing that changes is which client you reach for, and that becomes a routing decision you can revisit later instead of a rewrite.

Qwen3.8-27B is the local model in that example. Its weights are on Hugging Face under the Apache 2.0 license, and the model has 27.8B parameters, about 56 GB in BF16 at two bytes each. An H100 has 80 GB, which leaves roughly 24 GB for the KV cache and runtime. Qwen3.8 thinks before it answers by default, so the call above turns thinking off, the setting the model card shows for direct responses.

MixRoute is the cloud half of that setup: one OpenAI compatible API across 250+ models and no markup on provider pricing. claude-sonnet-5-5 is the Model ID of Claude Sonnet 5.5, listed at $2 input and $10 output per MTok, and the MixRoute model catalog shows the ID and price of every other model. MixRoute does not solve data residency. If your compliance team says the resume cannot leave, self-host that one task and route the rest.

FAQ

Is self-hosting AI cheaper than using an API? Almost never at startup volume. A rented H100 runs roughly $1,901 to $4,954 a month depending on provider. Against DeepSeek V4.1 Flash at $0.30 input and $1.20 output per MTok, you would need to parse between 1.8 million and 4.6 million resumes a month before the hardware wins. Self-hosting is a data control decision, not a cost decision.

How many requests before self-hosting an LLM breaks even? For resume parsing at roughly 2,400 tokens per document, the breakeven against DeepSeek V4.1 Flash is between 1.8 million and 4.6 million documents a month depending on your GPU rate. Bursty traffic pushes it higher, because a GPU bills for idle hours and an API does not.

Should I self-host AI for GDPR compliance? It is the strongest argument for self-hosting a job search product, since resumes are dense personal data. Before committing, compare what the API route offers: OpenAI’s EU data residency and Zero Data Retention, both subject to approval, and MixRoute’s policy of storing no request content and keeping system logs for 7 days. Your legal team decides whether that is enough.

What is the cheapest way to run resume parsing at scale? An API on a small model, until you are past roughly 63,000 resumes a day. At 10,000 resumes a month the API costs about $10.80 against $2,038 for a dedicated H100.

Can I use self-hosted and cloud models together? Yes, and most production teams do. Self-hosted stacks like vLLM and Ollama expose OpenAI compatible endpoints, as does MixRoute. Keep PII heavy tasks local and route everything else out, using the same SDK for both.

Do I need an H100 to self-host a model for resume parsing? No. Qwen3.8-27B needs about 56 GB in BF16, so the weights fit on an 80 GB card, and smaller models fit cheaper cards. A cheaper card lowers the bar without removing it: an A100 80GB listed at $1.43 an hour is about $1,030 a month, which still takes roughly 950,000 resumes a month to beat the API. You are still paying per hour instead of per token, and your traffic still has gaps.

The bottom line

The instinct is that self-hosting saves money. The arithmetic says the opposite, by about 189x at 10,000 resumes a month, the volume most job search products actually run.

Self-host when the data cannot leave. Call an API when it can. Do not self-host to save money unless you are moving more than roughly 60,000 resumes a day, and if you are, you already knew that.

MixRoute is the cloud half: 250+ models behind one OpenAI compatible endpoint with no markup on provider pricing. Start building on MixRoute