SGLang v0.5.22 was published at 02:55 UTC on October 19, 2026, which was 7:55 PM PT on October 18. The notes count 950 PRs from 245 contributors. The headline list is mostly model speedups: DeepSeek-V4.1 on H200, Qwen3.8-Flash-Next on B200, MiMo with DFlash on B300, and a new weight loader.

Those are worth reading, and I'll get to the ones that hold up. But the part of this release I'd spend time on before rolling a new image is shorter and less exciting. A few launch flags now mean something different, one flag's encoding was inverted, a cache class you may have depended on is gone, and the install can quietly give you an old version. None of these show up as a crash in every configuration. Some of them show up as a server that starts fine and behaves differently.

1. --dp-size now means replicas, and nothing else

Before this release, --dp-size had two meanings. With --enable-dp-attention it counted attention data-parallel groups inside the tensor-parallel group. Without it, it counted full model replicas. You had to read both flags to know which one you had.

PR #41818 splits them. --attn-dp-size N (alias --attention-data-parallel-size) now sets attention DP, and --dp-size counts replicas only. --enable-dp-attention is deprecated. The old spelling --dp-size N --enable-dp-attention still resolves to --attn-dp-size N --dp-size 1 and prints a warning, so existing launch scripts keep working. Two things are now rejected at startup: combining replicas with attention DP, and passing both the old and new spellings.

The PR checked that the rename is a no-op on outputs. On H200, DeepSeek-V2-Lite at --tp 4 --attn-dp-size 4 --ep-size 4 scored 0.386 on a 300-question GSM8K slice, the same as the old spelling. Output throughput was 5,883 tok/s against 6,069 on the base, inside the run-to-run range of 5,626 to 6,456.

Two details matter for anything that reads server state. /server_info now reports attn_dp_size alongside dp_size, and the gateway counts DP ranks as dp_size * attn_dp_size. If you have dashboards or autoscalers keyed on dp_size from /server_info, a server launched with the new flag will report dp_size=1. Wire formats such as KV-event descriptors and PD prefill server info keep the dp_size key with the old meaning, the total DP rank count. Under DeepSeek context parallelism, --dp-size N still means attention DP, warns, and will be rejected after December 31, 2026 (PR #42348).

2. The decode host-receive threshold flipped

--disaggregation-decode-host-receive-threshold is a fraction of device token usage. Once decode reaches it, incoming KV goes into the host retraction pool instead of GPU memory. The old encoding used 0 both as the default and as "off", which left no way to say "always receive on host" and read backwards for a threshold.

PR #43407 makes it a plain threshold. 1 is the new default and means off. 0 means always receive on host. Values in between kick in at that usage fraction. The default behavior hasn't changed, but if your PD config passes 0 explicitly to turn the feature off, after the upgrade it turns it fully on. That's the one change in this release I'd grep every deployment manifest for.

3. ChunkCache is gone, and so is a prefill graph backend

With --disable-radix-cache, full-attention, hybrid-SWA and Mamba models now run UnifiedRadixCache in disabled mode, and pure-SWA models run PureSWARadixCache (PR #43198 and siblings). Streaming sessions now start only on UnifiedRadixCache. Pure-SWA, FlexKV and plugin tree caches fail at startup with a "not verified" error (PR #42295). If you run streaming sessions on one of those, this upgrade stops the server from starting, which is at least loud.

Separately, --cuda-graph-backend-prefill tc_piecewise and any tc_compiler setting now raise an error (PR #42287). CUDA keeps breakable as the default prefill graph backend.

4. Python 3.10 doesn't fail, it just stops upgrading

PR #43021 sets requires-python = ">=3.11". The release note is careful about the failure mode: on Python 3.10, pip install sglang does not error. It resolves to the last release that supported 3.10. A CI job that builds on the Ubuntu 22.04 system Python will keep producing images, and they'll keep running 0.5.21. The official CUDA image already uses uv-managed Python 3.12, so this mostly bites custom images. I'd add a version assertion after install rather than trust the pin.

The same release moves CUDA builds from PyTorch 2.13.0 to 2.14.1, FlashInfer to 0.7.1, and Transformers to 5.19.0. If you build your own image with pinned wheels, those all move together.

5. Small API and security changes

Three OpenAI-compatible response changes: tool_choice: "none" no longer shows tool schemas to DeepSeek models (#43006), an absent tool_calls field is omitted instead of returned as null (#43027), and a stream keeps one created timestamp for its whole life (#43026). The null change is the one that breaks strict client parsers.

On security, PR #41322 adds admin auth to /load_lora_adapter_from_tensors and /set_trace_level. On a server started with --admin-api-key and no --api-key, both routes previously accepted unauthenticated requests. The PR notes that the LoRA route fix had landed once before, in August, and was lost when that squash commit diverged from main. If you run that key combination on a build without this fix, you have an unauthenticated endpoint that loads tensors. That alone is a reason to upgrade.

There's also a correctness fix worth knowing about: hybrid linear models could resume from the wrong Mamba state when speculative decoding and structured output were used together (#43029). The release credits Baseten for finding it. If you've seen odd outputs from a hybrid model under JSON mode with speculation on, this is a candidate.

The performance items, with their fine print

Prefill-complete decode allocation. PR #40703 adds an opt-in --disaggregation-decode-allocation-policy prefill_complete. In PD serving, decode normally reserves a request's KV slots while its prefill is still running, so a long prefill holds decode capacity it can't use yet. The new policy waits until prefill finishes. The PR's test used GLM-5.2-NVFP4 with MTP on 12 Rubin GPUs, decode KV deliberately capped at 163,840 tokens per rank, and a mix of 131,072-token and 4,096-token prompts.

Mixed workload, mean of two runs early prefill_complete Change
Output tokens/s 458.88 / 435.98 616.39 / 518.39 +26.81%
TTFT p95 (s) 71.22 / 79.97 36.03 / 41.27 -48.87%
Mean TPOT (ms) 4.108 / 3.869 4.204 / 3.943 +2.13%
ITL p99 (ms) 34.40 / 12.03 46.40 / 43.10 +92.77%

The release note's "27% more throughput and half the p95 TTFT" is accurate, but the ITL p99 row is the cost, and a Qwen3-8B comparison showed no clear gain. It also only supports Mooncake, attention TP1/CP1/PP1 per rank, and must be enabled on both roles after every worker is upgraded. Treat it as a tool for capacity-pressured long-prompt mixes, not a default.

InstantTensor loader. --load-format instanttensor (PR #40453) is where the "10x faster loading" line comes from. On 50 GB/s storage, DeepSeek-R1 on 8x H200 loaded in 15.3 s against 160 s for plain Safetensors, 10.5x. Qwen3-30B-A3B on one H200 went from 57.4 s to 1.77 s. From tmpfs, the DeepSeek-R1 gain drops to 1.87x, because the baseline is already fast. So the win is real when your weights come off network or NVMe storage, which is the usual cold-start case. The numbers measure weight loading only, not time to ready.

DeepSeek-V4.1 Flash on Hopper. PR #41251 tunes group32 GEMMs, the paged indexer and Marlin MoE paths. On 8x H200 at 8,192 in / 1,024 out, output throughput per GPU went up 2.06x to 2.35x across concurrency 1 to 64, with TTFT 1.34x to 1.57x faster. GSM8K and AIME24 counts stayed within a few problems of main. The Blackwell part of the same PR is a 1.7% to 2.8% prefill gain, and the author says plainly it isn't a proven speedup.

GLM-5.3 decode context parallelism with MTP. PR #43735 is the "up to 2.1x" item. On 8x B300 with FP8, DCP8 with MTP generated 1,784 tok/s on AIME26 against 838 for DCP8 alone, which is the 2.1x. Against plain TP8 at 1,140 tok/s, the gain is about 1.56x, and DCP2 with MTP was faster still at 1,876. The point of DCP is fitting longer contexts, so compare against the layout you'd otherwise run. The DSA override also rejects HiSparse, HiCache, PD disaggregation and tree drafting.

My upgrade order

  1. Assert the installed SGLang version in your image build, and move off Python 3.10.
  2. Grep manifests for --disaggregation-decode-host-receive-threshold 0 and change it to 1.
  3. Move --dp-size N --enable-dp-attention to --attn-dp-size N, and fix anything that reads dp_size from /server_info.
  4. Check for streaming sessions on non-unified caches and for tc_piecewise.
  5. Test strict OpenAI clients against the missing tool_calls field.
  6. Try instanttensor and prefill_complete as separate changes, each with its own benchmark.

Sources

Anpoo Sivanadi, Staff Software Engineer