A few days after DeepSeek open-sourced DeepInfer, its own engine for DeepSeek-V4.1-Flash, both big general-purpose engines answered. The vLLM team and Inferact published Serving DeepSeek-V4.1-Flash at Scale on October 23, 2026. The SGLang team followed with More Than DeepInfer: What SGLang Delivers on October 24.

Both say the same thing in the title: we match the model-specific engine. I wrote about DeepInfer's deployment Q&A when it came out, and the obvious question was whether you'd need to run DeepSeek's engine to get DeepSeek's numbers. The short answer from these two posts is no. The longer answer is that the two comparisons measure different things, and a few of the settings that make them fair also make them unlike production. Every number below comes from the two posts unless I say otherwise.

1. What each team actually compared

vLLM, decode. Under DeepSeek's decode configuration (131K context, three draft tokens, a fixed 3.25 accepted tokens per step), vLLM on 32 GB200 GPUs (DEP32, 384 routed plus 32 redundant experts) lands within 6% of DeepInfer's published curve (EP64DP64, read off its README chart) across batch sizes 8 to 256.

vLLM, prefill. The post is careful here. DeepInfer's published prefill used EP4DP4, 32 redundant experts, a 140-token bounded replay, and hardware the chart doesn't name. vLLM ran DEP2 on GB300 with no redundant experts and a 128-token replay. The team says outright that the gap between the curves should not be read as an engine-to-engine speedup, only that vLLM tracks DeepSeek's curve.

SGLang, both phases. SGLang ran DeepInfer itself, at commit 774ce24, next to SGLang on the same machines with DeepInfer's own benchmark scripts. Prefill ran on one 4x GB300 node and one 4x B300 node. Decode ran on 64 GB300 GPUs across 16 nodes at DP64/EP64. Results:

  • Prefill, 4x GB300: up to 1.04x DeepInfer's throughput, ahead at 4K, 32K and 64K new tokens per GPU per step, on par at 8K and 16K.
  • Prefill, 4x B300: up to 1.04x, ahead at 4K, 8K and 32K, on par at 16K and 64K.
  • Decode, 64x GB300: steps 1.1% to 7.4% shorter from 16 to 256 requests per GPU, on par at 8.

That's the stronger head-to-head of the two, because both engines ran on the same hardware. The launch commands for both sides are in a public gist, along with SGLang commits 7d996300b9 (prefill) and 3b95120f00 (decode).

2. The fine print that matters

Read the SGLang appendix before quoting the 7.4%. A few details:

  • Both prefill runs used balanced expert routing with EPLB off and 32 redundant experts. The post says settings like balanced routing "exist for the comparison only."
  • The two engines measured prefill differently. DeepInfer times about 9 batches on GPU 0. SGLang averages every complete batch on every GPU between the points where 20% and 90% of requests have finished. DeepInfer also repeats the same prompt across requests on each GPU; SGLang uses different new tokens per request.
  • In decode, SGLang kept the Engram tables on device while DeepInfer kept them in host memory, and DeepInfer's draft model carried 64 redundant experts while SGLang's had none.
  • Decode uses static verification: every drafted token is checked for every request, and step time is converted to tokens at a fixed 3.25 accepted per step. That's the same convention vLLM used. Real acceptance depends on your traffic.

It's all written down, but "1.1% to 7.4% faster" is a small margin measured under synthetic routing and fixed acceptance. What I take from it is parity, not a winner.

The balanced-routing point matters more than it looks, and the vLLM post shows why. On real routing with EPLB off, vLLM measured MoE balancedness (mean rank load over max rank load) of 0.35 for the target model and 0.21 for the DSpark draft. With EPLB on, those rose to 0.94 and 0.89. A benchmark that routes tokens evenly by construction skips the whole problem of expert imbalance, which on a real wide-EP deployment is one of the biggest things you tune.

3. What the vLLM post teaches about this model

The vLLM post is the more useful of the two if you're planning a deployment, because most of it is about the model's shape, not the comparison.

The KV cache is tiny. Global KV is 890 bytes per token, from cross-layer KV sharing and FP4 compression. KV capacity rarely limits batch size, so the team pushes each operation toward its compute or bandwidth limit instead.

Decode is MoE-bound, even at long context. At DEP4, 128K context and 64 requests per rank, MoE takes 62% of the target step; attention and the indexer together take 13%. At DEP32 with 16 requests per rank, target forward time goes from 8.93 ms at 128K context to 9.53 ms at 855K. That's 6.7x the context for 0.6 ms more. A practical consequence the post spells out: decode DP load balancing can just be least-load by request count, because context length barely moves step time.

Prefill wants small instances. In an AgentX prefill-only run at concurrency 256, DEP4 got only 83% of DEP2's per-GPU throughput (53.3K vs 64.4K tokens/s). Replaying identical requests with ranks out of step cost DEP2 5.9% of its throughput and DEP4 13.6%. That's DP imbalance, and smaller instances suffer less of it.

Three prefill fixes, about 40% together. On DEP2 after a 128K cache hit:

  • Running layer 20 on only the 128-token replay batch, while it still writes KV for every token (#59894): +19.9% at 32K new tokens.
  • Moving the shared Engram host tables from /dev/shm (64 KB GPU pages, lots of TLB misses) to host-NUMA cuMem allocations with 2 MB pages (#62359): another +2.8%.
  • Using DeepSelect top-k in the prefill indexer (#62240): another +15.6%.

Combined, new-token prefill throughput rises 35.6% to 43.9% across 4K to 64K new tokens. At 64K it goes from 71.2K to 102.5K tokens/s per GPU. The TLB fix is the one I'd remember for other models: host-side tables read randomly by the GPU should sit on large pages.

4. Wide-EP has a knee, and the CPU becomes the problem

On GB200 with 64 requests per GPU at 128K context, output throughput per GPU goes 9,385 tokens/s at DEP4, 11,445 at DEP8 (+22%), 12,915 at DEP16 (+13%), and 13,274 at DEP32 (+3%). Target MegaMoE time falls from 13.4 to 6.0 ms as each rank holds fewer experts, but the rest of the step doesn't shrink. Past DEP16 you're mostly buying flexibility, not per-GPU throughput.

The more interesting finding is that an unoptimized DEP32 ran slower than DEP16 (17.57 vs 13.84 ms per step) even though its GPU work was 1.1 ms faster. Wide-EP syncs every rank at every layer, so one late rank stalls everyone, and shorter GPU steps leave less room to hide CPU work. The fixes they list:

  • KV reads posted synchronously on the decode worker thread, replaced by a NIXL push connector with UCX scatter-gather: 141K to 440K tokens/s per GPU on an 8P1D GB300 setup at concurrency 3,072.
  • Gloo all-reduce latency growing linearly with rank count, replaced by a recursive-doubling CPU all-reduce (#61767): DP sync from 10.8 to 1.5 ms per step.
  • Admission bursts of up to 140 requests in one step, capped with an opt-in limit of 4 new KV loads per step (#61725): 460K to 516K tokens/s per GPU, with P90 per-user speed from 109 to 153 tokens/s.

5. The headline numbers, read carefully

vLLM's top-line result is 954K tokens/s per GPU at a P90 of 196 tokens/s per user on one GB300 NVL72 rack, 7.9x its earlier single-node result, priced at $38.96 per GPU-hour at DeepSeek's off-peak rates. Two cautions. That throughput counts cached input, uncached input and output tokens, so on an agentic trace with heavy prefix reuse it's dominated by cache hits. And the deployment uses several DEP2 prefill instances plus one DEP16 or DEP32 decode instance, 34 to 64 GPUs, not necessarily the full 72.

6. What I'd do with this

If you were waiting to see whether DeepInfer was the only fast way to serve V4.1-Flash, it isn't. DeepInfer is an offline engine, and its README lists the KV cache system, prefill/decode disaggregation and cross-replica load balancing as not included. SGLang ships all three (radix cache with Linker to 3FS or Mooncake, P/D over Mooncake, NIXL or MORI, and a prefix-aware gateway), and vLLM's numbers are already from an online P/D deployment.

So pick on operations, not on these margins. My order would be:

  1. Use whichever engine your team already runs. A 1% to 7% difference under balanced routing won't survive your real traffic either way.
  2. Turn on EPLB for both the target and the draft. The 0.35 balancedness figure is what you get without it.
  3. Start prefill at DEP2 and measure DEP4 against it on your own trace before going wider.
  4. Watch CPU time per step and admission bursts before you scale decode past DEP16. That's where the vLLM team found its losses.
  5. Report acceptance from your own traffic, not the fixed 3.25, when you compare anything.

Both teams published commits, PR numbers and launch commands, so you can rerun all of it on your own routing before trusting any of these deltas, including the ones I've repeated here.

Sources

Anpoo Sivanadi, Staff Software Engineer