DeepSeek Open-Sources Its AI Engine a Day After US Ban
DeepSeek just open-sourced the inference engine powering its production AI, and it runs on both Nvidia and Huawei chips.
Tools · Source: MarkTechPost
What happened
DeepSeek just open-sourced DeepInfer. This is the exact inference framework that runs DeepSeek-V4.1-Flash in their own production environment. They released the code under the permissive Apache 2.0 license. The timing is aggressive. The release dropped just one day after the US Department of Homeland Security banned the company from federal systems.
The framework is built for massive scale. DeepSeek claims a single cluster serves 155 trillion tokens per day. It ships with custom kernels like FlashMLA, DeepGEMM, and DeepEP. It also includes DSpark. This is a speculative decoding system that uses confidence estimates to cut verified draft tokens by about thirty percent.
But this release is strictly for offline inference right now. It leaves out the KV cache system, the load balancer, and prefill-decode disaggregation. DeepSeek says those pieces are too tightly coupled to their internal infrastructure to share. The system also requires heavy hardware. The smallest efficient production unit needs 64 prefill GPUs and 64 decode GPUs.
Key facts
- 155 trillion — tokens served per day by a single DeepSeek cluster
- 25,000 — output tokens per second per GPU during decode on 64 GPUs
- 96% — input cache hit rate for serving context from memory
- 128 — tokens of sliding-window attention state recomputed on a cache hit
Why it matters
This release is a direct threat to managed inference providers like Fireworks, Together, and Groq. DeepSeek is handing developers the exact blueprint that makes their own API pricing so cheap. You can now self-host production-grade V4.1-Flash without paying middleman markups. This exposes hosted providers to massive cost arbitrage.
The hardware support is the real geopolitical flex. DeepInfer treats Huawei Ascend 950 NPUs as first-class citizens alongside Nvidia SM100 GPUs. This is the first time a frontier AI lab has shipped a production open-source stack optimized for Chinese silicon. US export controls were meant to lock China out of the AI race. Instead, those bans are accelerating the adoption of foreign hardware. DeepSeek is proving they can build world-class infrastructure without relying exclusively on American chips.
For builders
Self-host production AI cheaper
You can run DeepSeek-V4.1-Flash using the exact optimizations the creators use. Managed inference providers lose out as startups move workloads in-house to save money.
Prepare for offline workloads only
DeepInfer currently lacks the KV cache and load balancing needed for real-time user chat. Builders must build these missing pieces or restrict usage to offline batch processing.
Leverage speculative decoding
DeepInfer includes DSpark for confidence-scheduled speculative decoding. Founders building high-throughput applications can use this to cut draft token verification by thirty percent.
My take
Banning DeepSeek did not slow them down. It just made them weaponize open source. By optimizing for Huawei chips and giving the code away for free, they are proving that the AI infrastructure war will be won with code, not government sanctions. Middlemen charging a premium for inference are officially on notice.
Original reporting: MarkTechPost. This is my rewrite and opinion.