On-premises or API: how many users before local LLM pays off?

Pay-per-token compares poorly with an amortised installation, and here is where the break-even point lies and why the KV cache changes everything.

By QDNA · Published on 5 July 2026 · Updated on 6 September 2026 · 10 min read

Updated on 2 September 2026: every cost is recomputed with the formula shown and named, dated assumptions (DGX Spark €4,875 excl. VAT on idealo.fr, 240 W from the NVIDIA sheet, EDF Tarif Bleu as of 1 August 2026, DeepSeek V4 Flash price list of 2 September 2026, rate €1 = $1.159, ECB of 1 September 2026). The "15 to 100 users" thresholds, the quantified "field reports" (€2,400, €800, €300, €2,000) and the lever gains (30 to 100%, factor of two to three, half) had no source and are removed or replaced by the calculation.

Two opposing cost structures

A pay-as-you-go interface has almost no fixed cost, but a cost that grows with every token, while an on-premises installation carries a high upfront investment and then a marginal cost close to zero. The two curves cross at a break-even point that depends on usage intensity, and that is not guessed: it is computed from five sourced numbers.

The assumptions of the calculation

Monthly API cost per user = 21 × (uncached input × €0.38 + cached input × €0.012 + output × €1.14) per million tokens. Occasional profile: 21 × (0.05 × 0.38 + 0.05 × 0.012 + 0.005 × 1.14) = €0.53.

• Hardware: DGX Spark Founders Edition, €4,875 excl. VAT (€5,850 incl. VAT, best price on idealo.fr read on 2 September 2026), depreciated linearly over 36 months, i.e. €135 excl. VAT per month. The model served is MIT-licensed, with no fee.
• Electricity: 240 W, the rated power supply on the NVIDIA DGX Spark sheet taken as an upper-bound assumption (ServeTheHome measures 60 to 200 W under load), 24 hours a day, i.e. 175 kWh per month, at EDF's regulated Tarif Bleu for professionals, Base option, up to 36 kVA, grid of 1 August 2026 (€0.1624 per kWh excl. VAT, excise and CTA included; CRE deliberation 2026-147, value read on Selectra Entreprises on 2 September 2026): €28 excl. VAT per month. On-premises total: ≈ €164 excl. VAT per month, excluding integration and operations, which are quoted separately and add to both columns alike when equivalent offers are compared.
• API price: DeepSeek V4 Flash, public price list read on 2 September 2026, peak hours (01:00–04:00 and 06:00–10:00 UTC on weekdays, a window that a French office day overlaps): $0.44 per million uncached input tokens, $0.014 cached, $1.32 output; off-peak half price. At €1 = $1.159 (ECB, 1 September 2026): €0.38, €0.012 and €1.14 per million.
• Occasional profile: 100,000 input tokens per working day, half of them served from cache, and 5,000 output tokens; 21 working days per month.
• Agentic profile: 1,000,000 input tokens per working day, 80% of them repeated prefixes served from cache, and 50,000 output tokens.

Agentic profile: 21 × (0.2 × 0.38 + 0.8 × 0.012 + 0.05 × 1.14) = €2.99. Threshold = €164 ÷ cost per user.

At how many users does on-premises pay off?

On-premises pays off beyond roughly 55 agentic users, or 309 occasional ones, at DeepSeek V4 Flash's peak rate. At the off-peak rate the threshold doubles, to 110 agentic or 620 occasional users. Below those counts, a pay-per-token interface stays cheaper.

Two caveats bound this result. The first is capacity: 55 agentic users produce 58 million output tokens per month, i.e. 95 tokens per second on average over office hours, above the 18 to 20 tokens per second single-stream and the 59 aggregate tokens per second at twelve requests read by Entrpi on a DGX Spark for DeepSeek V4 Flash (DSpark engine, NVIDIA forum, 15 July 2026); two Sparks under vLLM FP8 reach 210.8 aggregate tokens/s with six users (Classmethod, 10 August 2026), and a single Spark under heavy concurrency remains [TO BE MEASURED], and if the machine is not enough, the price of the next one enters the formula. The second is the choice of reference API: a price list dearer than DeepSeek's lowers the threshold in the same proportion. Finally, the presence of sensitive data mandates on-premises regardless of headcount, for compliance and sovereignty reasons.

Why is the KV cache on-premises' trump card?

Agentic workloads are dominated by input tokens, twenty to one in the profile above, more on tool loops. A ten-thousand-token system prompt sent a thousand times a day is ten million tokens: €3.80 a day at DeepSeek's peak rate if it is not served from cache, €0.12 if it is, and almost nothing locally once the prefix is cached.

The vLLM inference engine reuses the prefix cache for free, so time to first token drops sharply when the start of a conversation repeats, and repeated loops become nearly free, where a remote interface bills any token not held in cache at a markedly higher rate. The combination of document cache and prompt cache sharply reduces the cost of agentic scenarios.

Sovereign hosting in France

Two paths exist: on-premises combines hardware investment with French electricity, largely nuclear in origin, hence stable and low-carbon, while managed colocation in a sovereign French data center avoids buying premises and preserves legal control, unlike operators subject to extraterritorial law.

Rationalising without exposing data

A shared hardware base serves all agents with per-team quotas, and innovation on public data can overflow to a remote service, while sensitive data stays local, which rationalises cost without exposing information. The general framework is presented in our definition of the sovereign AI platform.

Fake local and its hidden costs

A team sometimes believes it runs locally while paying a remote interface bill every month: a sentiment analysis or a classification buried inside a conversational agent keeps calling an external provider, and the matching invoice line is attached to no project.

The calculation is the same as above, applied to the gateway's actual log: as long as that monthly spend exceeds the €164 excl. VAT of a depreciated machine, bringing it back in-house pays for itself, and the rule fits in one sentence: a partial local setup remains a billed dependency.

Which levers reduce the cost per token?

On-premises cost drops further once the inference engine is tuned, with several levers adding up and no extra hardware required.

• Continuous batching serves several requests on each compute pass; the gain is measured on the target load, it is not a universal percentage.
• Prefix and KV caching turn repeated input tokens into nearly free tokens.
• Quantisation fits more model onto the same graphics card, raising tokens per second and per euro.
• Speculative decoding has a small model propose several tokens that the large one verifies in a single pass, which lowers the time between tokens when the proposals are accepted; the factor depends on the model pair and is measured.
• A compact exchange format between agents cuts the tokens consumed at every turn; the saving is counted on the gateway log, not in advance.

The LiteLLM gateway unifies more than a hundred providers and models behind a single interface ("Call 100+ LLMs", official documentation), so switching from one provider to another takes one line of configuration. Virtual keys set a budget and limits per team, while spend stays stored in your own database, on your own infrastructure in Europe, so the bill stops being an unknown.

References

• DeepSeek API price list, read on 2 September 2026
• DGX Spark Founders Edition on idealo.fr, best price read on 2 September 2026
• NVIDIA DGX Spark, technical sheet (240 W power supply)
• EDF Tarif Bleu price grid, applicable from 1 August 2026
• LiteLLM, official documentation
• Artificial Analysis, performance and cost comparisons
• vLLM, official documentation
• On-Premise vs Cloud: Generative AI Total Cost of Ownership