TokelyTHE SHORT ANSWER
Renting a dedicated cluster to self-host DeepSeek V4.1 Flash costs more than the public API for almost every team. You cross the 76% line, where hardware rental beats DeepSeek's off-peak API price of $0.60 per 1M output tokens, only if you maintain more than 76% sustained GPU utilization day and night. Below that threshold, or with monthly spend under $880,000, paying per token is cheaper.
What did DeepSeek open-source on October 20?
DeepSeek published DeepInfer on October 20, 2026, under the Apache 2.0 license. DeepSeek describes the software as the high-performance inference framework that powers DeepSeek's large-scale production services. By October 25, the GitHub repository reached about 22,300 stars, and the Hacker News discussion passed 2,300 points.
The framework supports one model: DeepSeek-V4.1-Flash. Its default checkpoint is DeepSeek-V4.1-Flash-1017, released on October 17, 2026, with 763B parameters, a 1M-token context window, and MIT-licensed weights. The official API serves this checkpoint under the model ID deepseek-flash.
DeepInfer was originally built mainly on vLLM, incorporating ideas from SGLang and TensorRT-LLM. DeepSeek also released deepseek-recipe, which converts Chat Completions, Responses, and Messages API requests into model tokens matching the behavior of the official DeepSeek API.
The engine includes specialized kernels such as FlashMLA, DeepGEMM, and DeepEP. It provides attention data parallelism alongside MoE expert parallelism, expert load balancing, speculative decoding with draft tokens, confidence-scheduled verification, and asynchronous scheduling. It also supports structured outputs using regex, JSON schema, or EBNF grammars. Batch-invariant inference and deterministic sampling work on NVIDIA hardware only.
What hardware does DeepInfer need?
DeepInfer requires NVIDIA SM100 architecture GPUs or Huawei Ascend 950 NPUs. NVIDIA compute capability 10.0 corresponds to B200 and GB200 accelerators. The host environment requires Linux, GCC 13 or newer, Python 3.12, and CUDA 13 on NVIDIA or CANN 9.2 on Ascend.
Inference splits prefill and decode across separate hardware configurations. DeepSeek runs prefill on standard 8-GPU servers, while decode requires 64 to 128 GPU supernodes. The published decode benchmark ran across 64 GPUs within a single scale-up domain, which in NVIDIA's lineup means a rack-scale system such as the GB200 NVL72, which links 72 Blackwell GPUs in one NVLink domain.
DeepSeek states that one prefill node and one decode node form a deployable setup. However, DeepSeek recommends running decode at an expert parallelism of 64 or higher with a 1:1 prefill to decode GPU ratio. That makes 64 prefill GPUs plus 64 decode GPUs the smallest efficient deployment unit.
DeepSeek's production setup also leans on a large KV cache store. DeepSeek provisions 14.4 TB of logical KV cache storage per GPU on roughly 100 PiB of SSDs, retaining cached prefixes for 3 days on agent traffic and 7 days on chatbot traffic. DeepSeek specifies storage servers containing 20 SSDs of 30.73 TB each. Without this storage layer, the system recomputes repeated prompt prefixes instead of loading them from cache.
What is not in the release?
DeepInfer covers offline inference, but it does not supply a complete production service. DeepSeek lists three missing architectural elements needed for full production serving.
First, the release omits the distributed KV cache management system. Second, while most of the engine side is implemented, it lacks the full disaggregation pipeline across vision encoders, prefill nodes, and decode nodes. Third, it omits production load balancing across data-parallel replicas.
DeepSeek states that its KV cache management and load balancing code remains tied to internal cluster infrastructure and is not fully open. Engineering teams that want to replicate the official API service must build routing and caching layers independently.
How much does the smallest cluster cost to rent?
The smallest efficient deployment unit of 128 Blackwell GPUs costs $1,222.40 per hour on CoreWeave on-demand pricing. That equals $29,337.60 per day and $880,128 across a 30-day month.
This setup pairs 64 B200 GPUs across 8 HGX B200 instances for prefill at $8.60 per GPU-hour with 64 GB200 NVL72 GPUs for decode at $10.50 per GPU-hour. HGX B200 instances cost $68.80 per hour for 8 GPUs, totaling $550.40 per hour. GB200 NVL72 instances cost $42.00 per hour for 4 GPUs, totaling $672.00 per hour.
Renting an entire 72-GPU NVL72 rack rather than a 64-GPU slice raises the cluster price to $1,306.40 per hour, or $940,608 per month. For comparison, Lambda lists HGX B200 clusters between $8.87 and $9.86 per GPU-hour under commitments ranging from 2 weeks to 1 year.
The absolute minimum deployable unit of 1 prefill node and 1 decode node with 16 B200 GPUs costs $137.60 per hour, or $99,072 per month. None of these figures include KV cache storage servers, inter-rack networking, data egress, or engineering staff.
When does self-hosting beat the API price?
A 128-GPU cluster running DeepInfer delivers about 740,700 output tokens per second under DeepSeek's documented production settings. That figure multiplies 14,466 tokens per second per GPU across 64 decode GPUs, discounted to the 80 percent of the offline benchmark that DeepSeek says its load balancing reaches. Running continuously for 24 hours yields roughly 64 billion output tokens per day.
Dividing the cluster lease across output volume gives a unit cost of $0.46 per 1M output tokens at 100 percent utilization. At 50 percent utilization, the cost climbs to $0.92 per 1M output tokens. At 20 percent utilization, the cost reaches $2.29 per 1M output tokens.
Comparing these costs to DeepSeek's off-peak output rate of $0.60 per 1M output tokens establishes the 76% line. A rented 128-GPU cluster must maintain at least 76 percent sustained utilization around the clock to undercut the off-peak API. Against an output price of $0.80 per 1M output tokens, break-even sits at 57 percent utilization, dropping to 38 percent against the peak API rate of $1.20 per 1M output tokens.
This comparison assigns the entire cluster lease to output generation and assumes inputs are free, which favors the API. Factoring in a workload with 4M input tokens per 1M output tokens, with half of the input tokens served from cache, raises the off-peak API bill to $0.906 per 1M output tokens. Under this adjusted input mix, the break-even utilization threshold drops to about 51 percent.
The absolute budget sets a clear ceiling on whether self-hosting makes sense. A 128-GPU cluster costs $29,337.60 every 24 hours whether processing requests or idling, which buys 48.9 billion output tokens on the off-peak API. If your monthly DeepSeek V4.1 Flash API bill sits below $880,000, paying per token costs less than renting this cluster regardless of utilization.
Downsizing to the 16-GPU setup of 2 nodes does not lower the break-even threshold. DeepSeek's benchmark data shows 8-GPU expert parallelism produces about 12,600 tokens per second per GPU at 64 concurrent requests. Factoring in 80 percent load balancing yields 7 billion output tokens per day, translating to $0.47 per 1M output tokens at 100 percent load and a 79 percent break-even threshold against off-peak API pricing.
How does DeepSeek make money at $0.60 per million tokens?
In its deployment documentation, DeepSeek reports that a single production cluster serves 155 trillion tokens per day, generating US$38 of revenue per GPU-hour at off-peak valley pricing. This figure is DeepSeek's own unaudited claim. DeepSeek does not disclose how many total GPUs make up that cluster, and US$38 reflects gross revenue rather than profit.
CoreWeave lists comparable B200 and GB200 hardware between $8.60 and $10.50 per GPU-hour on demand. That spread between raw hardware rental costs and DeepSeek's $38 per GPU-hour revenue explains how DeepSeek charges $0.60 per 1M output tokens. DeepSeek pools demand from its whole user base and keeps its accelerators busy at a scale a single team rarely reaches.
DeepSeek's benchmark results show how concurrency drives that revenue density. At 256 requests per GPU, a 64-GPU decode cluster achieves about 26,000 output tokens per second per GPU, with individual streams receiving 100 tokens per second. Dropping concurrency to 8 requests per GPU raises individual user speeds to 325 tokens per second but reduces cluster efficiency to 2,600 tokens per second per GPU. The benchmark fixes draft-token acceptance at 75 percent.
Prefill clusters show similar scaling under a 131,072 context length. Running prefill across 4 GPUs yields between 60,000 and 89,000 uncached input tokens per second per GPU depending on batch size. Without enough traffic to keep batches full, a private cluster runs small batches, and each GPU produces fewer tokens per dollar.
Who should self-host DeepSeek V4.1 Flash?
Self-hosting DeepSeek V4.1 Flash makes sense for four specific groups of developers. First, teams that must keep prompts and outputs inside their own private infrastructure for compliance reasons require dedicated hardware. For these organizations, data containment justifies the engineering overhead.
Second, engineering teams that require bit-identical outputs need to deploy DeepInfer directly. DeepInfer delivers batch-invariant inference and deterministic sampling on NVIDIA SM100 systems, ensuring that variations in batch size or concurrent load do not alter output tokens.
Third, companies with steady traffic large enough to keep at least 128 Blackwell GPUs busy around the clock can achieve lower costs. If your application sends continuous traffic that stays above the 76% line every hour of the week, private hosting beats per-token rates.
Fourth, commercial inference providers can build competitive hosting businesses. Combining MIT-licensed weights with an Apache 2.0 inference engine permits any host to sell V4.1 Flash endpoints. For everyone else, per-token APIs remain cheaper, especially since DeepSeek's off-peak window covers about 79 percent of the week.
How do you use DeepSeek V4.1 Flash without a cluster?
Developers who want to avoid provisioning 128 Blackwell GPUs can access the model directly through Tokely. Tokely lists DeepSeek V4.1 Flash on Tokely at $0.20 input, $0.004 cached input, and $0.80 output per 1M tokens.
These prices remain the same at any hour, removing the need to schedule jobs around DeepSeek's peak windows. Tokely's $0.80 output price beats DeepSeek's peak rate of $1.20 per 1M output tokens while providing consistent per-token billing without cluster management. Tokely prices can change, and the Tokely models catalog shows current rates.
The platform uses an OpenAI-compatible interface. A single API key and shared balance route requests to Claude, GPT, Gemini, Kimi, and over 130 other models, letting teams change models by editing the model parameter.
from openai import OpenAI
client = OpenAI(base_url="https://api.tokely.me/v1", api_key="YOUR_TOKELY_API_KEY")
reply = client.chat.completions.create(
model="deepseek-v4.1-flash",
messages=[{"role": "user", "content": "Summarize this incident report in five lines."}],
max_tokens=2048,
)
print(reply.choices[0].message.content)Methodology and limitations
This analysis prices hardware at CoreWeave's on-demand list prices checked on October 25, 2026, with Lambda's term prices shown for comparison. Reserved capacity costs less per hour, which would lower the break-even utilization.
Throughput figures come from DeepSeek's published offline benchmarks running on a 131,072-token context length with speculative decoding draft-token acceptance fixed at 75 percent. In live production environments with varying query patterns, draft acceptance and output throughput will fluctuate. The figure of 12,600 tokens per second per GPU for the 16-GPU setup was read directly from DeepSeek's benchmark chart.
The cost models exclude KV cache storage, inter-rack networking, cloud data egress, and engineering personnel. Real production workloads experience uneven demand curves, meaning continuous 24/7 utilization serves as a theoretical upper bound rather than everyday reality.
Sources
Data in this analysis comes from DeepInfer on GitHub, the DeepSeek V4.1 Flash inference deployment Q&A, model weights on DeepSeek-V4.1-Flash-1017 on Hugging Face, the DeepSeek API pricing schedule, CoreWeave GPU pricing, and Lambda GPU pricing. Hardware details come from NVIDIA GB200 NVL72 specifications and the NVIDIA CUDA GPU compute capability table. Reception: the Hacker News discussion. For the U.S. policy side, see our earlier post DeepSeek Banned From U.S. Federal Agencies.