SGLang matches DeepSeek's DeepInfer on its own benchmarks across B300 and GB300

SGLang reaches parity with DeepSeek's own engine

The SGLang team has measured its engine head-to-head against DeepInfer, the inference framework DeepSeek built for its own service and later open-sourced alongside the GPU kernels it runs on. According to a dev.to write-up of the SGLang results, the team ran DeepInfer's own benchmark workload on the same hardware and reports prefill throughput of up to 1.04x on both 4x GB300 and 4x B300 nodes, plus decode steps that come in 1.1% to 7.4% shorter on a 64x GB300 deployment. At the remaining measurement points the two engines sit within roughly 1% of each other.

What SGLang integrated

DeepSeek's release gave the community the vendor's kernels, and SGLang has adopted them. The engine now runs FlashMLA, DeepGEMM, DeepEP, DeepSelect and the TileLang kernels, along with the Mega series: MegaAttention, MegaMoE, MegaGate and Mega-mHC. MegaMoE fuses token dispatch, the expert matrix multiplications and the combine step into one kernel, and MegaAttention is the default attention path on B300 and GB300 hardware for both prefill and decode.

Above the kernels sit three mechanisms. DSpark speculative decoding uses the checkpoint's draft model to propose a block of tokens that the target model verifies together, cutting the number of target-model forward passes. SWA bounded replay runs the late decoder layers over only the last 128 prompt tokens while earlier source layers still process the full prompt, and on a prefix-cache hit an encoder replay rebuilds missing sliding-window KV entries from the last 128 cached tokens. Engram table placement ships in two modes: one copy per node in host memory, which leaves more HBM for the KV cache, or tables sharded across GPUs with direct peer reads over NVLink and no collective communication around the lookup.

How the comparison was run

The SGLang post, published October 24, describes the setup. Prefill ran on a 4-GPU node at DP4/EP4 with 128K-token contexts and 4K new tokens per request, sweeping batches from 4K to 64K new tokens per GPU, with balanced expert routing, EPLB off and 32 redundant experts. Decode ran on 64 GB300 GPUs across 16 nodes at DP64/EP64, again at 128K-token contexts with a shared cached prefix, three draft tokens per step under static verification, and 384 routed plus 64 redundant experts.

The results vary by workload. On 4x GB300, SGLang leads at 4K, 32K and 64K new tokens per GPU and is level at 8K and 16K. On 4x B300 it leads at 4K, 8K and 32K and is level at 16K and 64K. On the decode side, its step is 1.1% to 7.4% shorter from 16 to 256 requests per GPU and level at 8.

Differences the post discloses

The appendix lists four divergences between the measurement methods. DeepInfer times roughly nine prefill batches on GPU 0, whereas SGLang averages every complete batch on every GPU between the points where 20% and 90% of requests have finished. DeepInfer repeats the same prompt across requests on each GPU, while SGLang varies new tokens per request. Configuration differs too: SGLang keeps the Engram tables on device while DeepInfer places them in host memory, and DeepInfer's draft model carries 64 redundant experts where SGLang's carries none. Each choice changes memory pressure and verification cost, which is why the post states both configurations explicitly.

Where the performance comes from

The post attributes the numbers to plumbing as much as kernels: Mega-mHC feeds MegaMoE directly, attention produces MXFP8 output for the output projection, and the prefill compressor and request-window KV copies use fused kernels, reducing intermediate memory traffic and kernel launches. Breakable CUDA graphs cover prefill steps up to 8K tokens per GPU, including long contexts and data-parallel attention, and decoder replay uses separate graphs sized to the bounded suffix.

There is also a lineage note. Since v0.4 in December 2024, SGLang has prepared step N+1 on the CPU while the GPU executes step N, and the post observes that DeepInfer's documentation credits this overlapping-scheduler design to SGLang, which makes the comparison partly a test of an older design against a newer engine that inherited it. DeepSeek's chat format arrived too: prompts render and outputs parse through the deepseek-recipe package, and grammar-constrained decoding covers JSON schema, regex and EBNF.

What throughput alone does not settle

As the dev.to summary points out, the charts leave the capacity question underneath the throughput number open: where the KV cache tier lives, whether the draft model carries redundant experts, and what a gateway must do when replicas hold different cached prefixes. Notably, DeepInfer's own README names a KV cache system, prefill/decode disaggregation and load balancing across data-parallel replicas as requirements for a complete inference service, and SGLang ships all three.

Why it matters

DeepInfer was the vendor measuring its own model on its own stack, which made it the reference and the performance ceiling for anyone deploying DeepSeek weights. These results indicate that an open, community-maintained engine now matches that ceiling while also providing the operational components the vendor's own README demands. Just as important, the comparison models how parity claims should be published: methodology and configuration differences disclosed next to the curves rather than buried. For serving teams the practical caveat stands, since throughput is one axis and cache placement, draft redundancy and gateway routing will decide real capacity and cost.