Oct 23, 2026
Stop overpaying for tokens: find your API vs self-host break-even
If your team pays per token for a hosted model, or is weighing a private GPU deployment of an open one, LLM inference economics comes down to how many tokens each rented GPU-hour produces. DeepSeek has published unusually detailed figures for its own serving stack and a partial release of its engine. This article works out the break-even math and the daily volume at which running it yourself matches the API bill.
Key takeaways
What: DeepSeek reports one V4.1 Flash cluster serving 155 trillion tokens a day at $38 of revenue per GPU-hour (off-peak), 2.9x its 2025 rate, according to its deployment Q&A.
Why it matters: your token cost is set by tokens per GPU-hour; at $8.99 per B200-hour an 8-GPU server needs about 3,000 output tokens/s just to break even on V4 Pro list prices.
How: split prefill from decode, run wide expert parallelism, and cache prompts on SSD, where cache-hit input costs 50x less than a miss.
What to do: start on the API with cache-friendly prompts; a private deployment pays on cost above roughly 150 billion tokens a day, or earlier when data control demands it.
Background you need
Three concepts carry the rest of the article:
Prefill and decode: prefill reads the whole prompt in parallel and is compute-bound; decode emits one token at a time and is limited by memory bandwidth.
KV cache: the stored attention state of earlier tokens, which can be reused when a new request shares the same prefix.
Expert parallelism (EP): a mixture-of-experts model spreads its experts across GPUs, and every token is routed to whichever GPUs hold its experts.
The bottleneck: LLM inference economics start with idle GPUs
An inference provider, in bycloud’s words, will “make money from the gap between what it costs them to run the model and the value that they added in serving it to users”. The cost side is nearly fixed per hour, so the revenue side is a throughput question. Start with how much a million output tokens of workload earns once you count the input that came with it:
Eq. 1: Revenue per million output tokens.
where:
R is revenue per million output tokens of workload, in dollars
pₒ, pₘ and pₕ are the per-million prices for output, cache-miss input and cache-hit input
k is input tokens per output token and h is the cache-hit fraction of input
At Together AI’s listed V4 Pro 0813 price of $1.32 input and $3.96 output, with two input tokens per output token and no cache hits, Eq. 1 gives R = 3.96 + 2 × 1.32 = $6.60. The break-even output rate for a server then follows from its hourly cost:
Eq. 2: Break-even output throughput.
where:
T* is the output tokens per second the server must sustain
C is the server’s cost per hour in dollars, and R comes from Eq. 1
For eight B200s at Together’s $8.99 per GPU-hour, C = $71.92, so Eq. 2 gives about 3,027 tokens/s. That assumes constant demand, ignores prefill compute, and leaves out CPUs, networking and staff, so it is a floor. Table 1 shows how sensitive it is to the rental price.
Table 1: Break-even output throughput for one 8-GPU server at $6.60 per million output tokens.
GPU price basis | $/GPU-hour | Server $/hour | Break-even output tok/s |
|---|---|---|---|
Together AI on-demand HGX B200 | 8.99 | 71.92 | 3,027 |
Fireworks AI on-demand B200 | 13.00 | 104.00 | 4,377 |
A host that sells only half its hours has to double those rates. DeepSeek’s tactics all aim at the same two quantities: tokens per GPU-hour and revenue per token.
How does batching turn demand into cheaper tokens?
Batching raises tokens per GPU-hour because a decode step re-reads the model weights once and applies them to every sequence in the batch. Demand therefore lowers unit cost as well as raising revenue: a busy provider builds larger batches than a quiet one. The limit is latency, since a larger batch makes each user wait longer for each token.
DeepSeek’s 2025 numbers show how that trade plays out. Each user received 20 to 22 tokens/s, and each node produced about 14.8 thousand output tokens/s. A slow per-user speed is what let each node run large batches, so per-user speed works as a price tier: a provider that sells fast streaming gets fewer users per pass over the weights and needs wide expert-parallel groups to keep GPUs busy.
Intuition: a GPU is cheapest per token when many users share each pass over the weights, and a faster promised speed means fewer users per pass.
Wide groups are the answer, and building them raises a scheduling problem of its own.
Why does DeepSeek run prefill and decode on different GPUs?
Prefill and decode want opposite hardware, so DeepSeek deploys them as separate pools sized independently. Its V4.1 deployment Q&A recommends prefill at EP2 to EP8 (“use the smallest EP/DP that HBM allows”) and decode at EP64 to EP128, with 64 prefill plus 64 decode GPUs as the smallest efficient unit. The pipeline appears in Fig 1. The 2025 system used EP32 for prefill and EP144 for decode, so the split is old and the numbers have shifted.
Fig 1: Disaggregated serving pipeline. Prefill GPUs read cached prefixes from a shared SSD store, write new ones back, and pass the finished KV cache to decode GPUs; load balancers keep both pools even.
Wide decode groups pay off because each GPU holds fewer experts and reads fewer weights per step. The cost is imbalance: if one GPU draws popular experts, all the others wait for it. DeepSeek runs an expert-parallel load balancer with redundant experts, and its Q&A says that balancing gets it to roughly 80% of a perfectly balanced offline benchmark. An offline benchmark is therefore a ceiling, not a forecast.
Builds on: the 2025 V3/R1 system. The model-side changes that make V4.1 Flash cheap to cache, such as an 890-byte-per-token global KV footprint, are described in DeepSeek’s V4.1 Flash report.
Intuition: keep the compute-hungry stage and the bandwidth-hungry stage on separate machines so neither starves the other.
Disaggregation fixes utilization, but it does not remove work. Coding agents resend the same long prefix on every step, and recomputing it is waste.
How does context caching change the price list?
A cache hit lets the prefill pool skip recomputation, and DeepSeek prices that skip directly. On the current price page, Flash cache-hit input is $0.003 per million tokens against $0.15 for a miss at off-peak rates, a 50x gap; for V4 Pro it is $0.022 against $0.66, 30x. DeepSeek introduced disk-based context caching on August 2, 2024, advertising savings of up to 90%.
The 2025 traffic log (a 24-hour window) shows which tokens bring in the money. Of 776 billion tokens in one day, 342 billion were cache-hit input (44%), 266 billion cache-miss input (34%) and 168 billion output (22%). Valued at R1 list prices, output tokens produce about 65% of the theoretical revenue (Fig 2).
Fig 2: Tokens versus revenue. Output is 21.6% of token volume but 65.5% of the valued revenue, while cache hits are 44.1% of tokens and 8.5% of revenue.
Cache-hit input is cheap to serve and cheap to buy, so output tokens supply most of the valued revenue. Sustaining the hit rate takes storage. DeepSeek’s Q&A describes a 3FS file system with roughly 100 PiB of SSD for KV cache, about 14.4 TB of logical cache per GPU, and retention of 3 days for agent traffic and 7 for chatbots. The trade is explicit: “Empirically, a TTL of around 3 days costs chatbot traffic 0.5%–1% of KV cache hit rate.”
Intuition: the cheapest token is one the GPU never computes, and an SSD is a cheap place to remember it.
A fixed fleet still faces daily traffic swings, which is the next lever.
Why does off-peak pricing cut the bill in half?
DeepSeek halves its rates outside business hours so that idle capacity earns something. Peak hours are 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays, excluding Chinese public holidays, which is 09:00 to 12:00 and 14:00 to 18:00 in Beijing. The scheme took effect at 16:00 UTC on August 16, 2026, according to the change log.
The 2025 system already worked this way: nodes served inference by day and were handed to research and training at night, with automatic night discounts. A fleet sized for the daytime peak leaves GPUs idle after dark, and Eq. 2 says idle hours raise the break-even rate. The headline $38 per GPU-hour is quoted at the off-peak price, so it would understate revenue per GPU-hour if traffic were similar in both periods.
Intuition: a discount that moves demand into empty hours is cheaper than buying GPUs for the busiest hour.
Can a third-party host match DeepSeek’s cost?
Not on price alone, and the listings show it. Together lists V4 Pro 0813 at $1.32 input and $3.96 output, the same as DeepSeek’s own peak rate, and Fireworks lists V4.1 Flash at $0.30 input and $1.20 output, again DeepSeek’s peak rate. Hosts match the peak list while DeepSeek offers half that off-peak.
bycloud attributes the first-party edge to kernels tuned to one model’s shapes, early access to weights and visibility into real traffic. DeepInfer narrows part of that gap: DeepSeek released it under Apache-2.0 with its kernels, expert-parallel load balancing, DSpark speculative decoding and a bounded-replay mode that cuts decoder prefill work by replaying only a bounded suffix. The release supports one model, V4.1 Flash, on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, and provides offline inference. The README lists the KV cache system, prefill/decode disaggregation and load balancing as required for a full service and not included.
By the numbers
Table 2 sets the two DeepSeek disclosures side by side. They use different models, hardware and price lists, so the ratios are directional.
Table 2: DeepSeek serving disclosures, 2025 versus 2026.
Metric | V3/R1 (Feb 2025) | V4.1 Flash (Oct 2026) |
|---|---|---|
GPU | H800 | SM100 or Ascend 950 (not stated for the $38) |
Tokens per day | 776B (24 h window) | 155T (one cluster) |
Revenue per GPU-hour ($) | 12.91 (computed at list prices) | 38 (off-peak) |
Output tokens/s per GPU | about 1,850 | about 14,000 (per decode rank) |
The 2025 revenue per GPU-hour is $562,027 divided by 43,536 GPU-hours (226.75 average nodes × 8 GPUs × 24 hours). The 2026 decode rate is the Q&A’s estimate at 60 concurrent sequences per rank: 13,730 tokens/s with fixed-length speculative verification and 14,466 with dynamic-length verification.
On margin, $38 of revenue against a rental price gives 76% at Together’s $8.99 per GPU-hour and 66% at Fireworks’ $13. DeepSeek owns its hardware, and these figures exclude staff, networking and training.
What’s the catch?
The first catch is that none of the disclosed figures is profit. DeepSeek itself warned in 2025 that “our actual revenue is substantially lower for the following reasons:” V3 is priced below R1, web and app access are free, and night discounts apply. The headline 545% is a ratio of profit to cost, which is about an 84.5% gross margin on revenue ($475,028 divided by $562,027).
The second catch is that cache hits are cheap, not free. Storage, transfer and the new tokens after the prefix still cost compute, and the 3-day versus 7-day retention is a hit-rate trade.
Limitations and open questions
Several things remain open:
DeepSeek discloses no cost, margin or GPU count for V4.1 Flash, so every margin figure here is an estimate.
The hardware behind the $38 figure is not stated, and the cluster is a single one.
Independent reproduction is impossible until the cache system and load balancers are released.
What this means for your organization
Teams that buy tokens from a hosted API, and platform teams asked whether to host an open model themselves, are the ones that should act on these numbers. The decision turns on token volume, cache-hit share and how much control over data the workload needs.
Using DeepSeek through the hosted API
Use deepseek-flash or deepseek-v4-pro, both with 1M-token context, and put the stable prefix (system prompt, tool definitions, reference documents) first in every request so later requests can hit the cache. Table 3 lists current rates. At a 90% hit rate, Flash off-peak input costs 0.1 × $0.15 + 0.9 × $0.003 = $0.0177 per million tokens, 8.5x below the miss price. Peak hours are 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays and cost 2x, so batch jobs belong outside those windows.
Table 3: DeepSeek API list prices per million tokens (USD), November 2026.
Model | Input, cache hit | Input, cache miss | Output | Peak multiplier |
|---|---|---|---|---|
| 0.003 | 0.15 | 0.60 | 1x |
| 0.006 | 0.30 | 1.20 | 2x |
| 0.022 | 0.66 | 1.98 | 1x |
| 0.044 | 1.32 | 3.96 | 2x |
Prices change, so check the pricing page before budgeting.
Running it privately: forward-deployed inference
The V4.1 Flash weights are MIT-licensed on the model card, and DeepSeek released DeepInfer under Apache-2.0. DeepInfer supports V4.1 Flash only, on NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs, and provides offline inference. Its README says it was originally built primarily on vLLM.
A deployment team has to build the rest. The README lists a KV cache system, prefill/decode disaggregation and load balancing across data-parallel replicas as required for a complete inference service and not included. Those three pieces are the forward-deployment work: the cache store behind the hit rates above, the split pools in Fig 1, and the routing that keeps them even.
The deployment Q&A gives the minimum efficient footprint:
64 prefill GPUs plus 64 decode GPUs, the smallest efficient deployment unit
20 × 30.73 TB SSDs per KV cache storage server
roughly 400 Gbps of interconnect per 8 GPUs for cross-data-center deployments
a 3FS file system with roughly 100 PiB of SSD for the KV cache at DeepSeek’s scale
On hardware fit, DeepInfer lists SM100 and Ascend 950 and does not list Hopper, so check your GPU generation before planning around it. The V4.1 Flash model card states no hardware requirements. Its usage panel shows vLLM and SGLang serving commands, and its inference folder holds code that its README calls “a readable reference implementation rather than a production serving engine.”
Cost is where the decision gets concrete. A 128-GPU unit rented at Together’s $8.99 per GPU-hour costs 128 × $8.99 = $1,150.72 an hour, or $27,617 a day. To compare that with the API, assume your traffic looks like DeepSeek’s 2025 day: 44.1% cache-hit input, 34.3% cache-miss input and 21.6% output. Table 4 prices that mix at Flash off-peak rates.
Table 4: Blended Flash off-peak API price for the assumed token mix.
Token type | Share of tokens | Price, $/M tokens | Contribution, $/M tokens |
|---|---|---|---|
Cache-hit input | 44.1% | 0.003 | 0.0013 |
Cache-miss input | 34.3% | 0.15 | 0.0515 |
Output | 21.6% | 0.60 | 0.1296 |
The contributions sum to a blended price of about $0.18 per million tokens, so $27,617 a day buys about 27,617 ÷ 0.182 ≈ 151,000 million tokens, or roughly 150 billion tokens a day. Below that volume the API is cheaper on compute alone, and the figure leaves out SSD servers, networking and staff. A different mix moves it: a lower cache-hit share raises the API price and lowers the break-even volume. Organizations below it go private for data control, sovereignty or latency, not for cost.
Next steps for your team
Measure your own traffic against the DeepSeek API pricing page: cache-hit share, input-to-output ratio and the share that can run off-peak, then compute your blended price as in Table 4.
Pilot self-hosting on the DeepSeek-V4.1-Flash model card, which lists vLLM and SGLang serving commands for the MIT-licensed weights.
Read the DeepInfer README and its deployment Q&A to confirm SM100 or Ascend 950 availability and to scope the KV cache, disaggregation and load-balancing work.
Price a managed private alternative: Together AI lists dedicated inference as single-tenant GPU instances, and Fireworks lists Fireworks Sovereign, inference on GPUs in your own cloud account or data center, in private preview for Enterprise.
My take: DeepSeek’s advantage seems to come mainly from compute it avoids, and the avoidance depends on a cache that was not released. Open kernels help hosts, but the cache hit rate is a property of storage and traffic, so the price gap may persist. That is speculation until DeepSeek publishes costs.
What to watch next
Three open items decide whether these economics spread.
Whether Together and Fireworks cut V4.1 Flash below DeepSeek’s peak list price after DeepInfer’s October 2026 release.
Release of the KV cache system and load balancers, without which DeepInfer stays offline-only.
Any DeepSeek disclosure of GPU count or cost, which would settle the margin estimates.
FAQ
The answers below reuse the break-even model and the disclosures above.
How much does LLM inference cost per token?
It is the GPU’s hourly cost divided by the tokens it produces in that hour. For eight B200s at $71.92 an hour, serving 3,027 output tokens/s costs about $6.60 per million output tokens, which equals list-price revenue at a 2:1 input-to-output mix, so it is break-even (Eq. 2).
Is DeepSeek’s API profitable?
DeepSeek has not published an audited profit figure. Its 2025 write-up reported a theoretical 545% cost-profit ratio for one day and said actual revenue was lower. For V4.1 Flash it reports revenue of $38 per GPU-hour and no costs.
Why do output tokens cost more than input tokens?
Decode produces one token per step and is limited by memory bandwidth, while prefill processes many tokens at once. Flash charges 4x more for output than for cache-miss input at off-peak rates.
Can you run DeepSeek V4.1 Flash on your own infrastructure?
Yes for the weights, with work for a full service: they are MIT-licensed, and DeepInfer offers offline inference on SM100 or Ascend 950 hardware. A production service also needs a KV cache system, prefill/decode disaggregation and load balancing, which DeepSeek did not release, and DeepSeek’s smallest efficient unit is 64 prefill plus 64 decode GPUs.
What is DeepInfer?
DeepInfer is DeepSeek’s Apache-2.0 inference framework for V4.1 Flash, released as an offline-inference version without the cache system or load balancers.