Breaking
DeepSeek open-sources DeepInfer, the inference engine behind its own service, with Huawei Ascend support
First seen on X 2 hours ago@deepseek_ai ♥ 3,4749 views
Chinese AI lab DeepSeek released DeepInfer, the large-scale inference framework that runs its production service, as open source.
Per DeepSeek's official X account on Oct 20, DeepInfer serves DeepSeek-V4.1-Flash at scale. The company said it is sharing its production stack 'to help the community lower compute costs, reduce energy use, and make AI more affordable,' and that inference code for future models will land in the same repository.
The GitHub README lists two supported targets, NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. Features include speculative decoding with draft tokens, batch-invariant inference, deterministic sampling and structured outputs constrained by JSON schema or regex. It ships under Apache 2.0. A KV cache system, prefill/decode disaggregation and load balancing across replicas are not part of this release. DeepSeek said it was originally built on vLLM.
DeepSeek also open-sourced deepseek-recipe, its production token encoding and decoding component that matches the official API. The announcement passed 3,400 likes within an hour and the repo drew 3,600 GitHub stars. A popular reply said OpenAI, 'supposed to be open for the benefit of humanity, ended up staying closed.'
The release came a day after the US government ordered federal agencies to remove DeepSeek models from their systems.
“🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency.”
