vLLM v0.32.0 was published at 06:50 UTC on October 18, 2026, which was 11:50 PM PT on October 17. The release notes count 814 commits from 363 contributors, 121 of them new. Most of the headline space goes to a new structured-decision endpoint and per-model kernel work for GLM-5.3, DeepSeek-V4.1, Kimi-K3 and Qwen3.8-Flash-Next.

I'm not going to cover those. When I upgrade a serving fleet, the risk is rarely the new feature I opt into. It's the default that changed and the request that used to fail and now succeeds, or the reverse. This post goes through the 0.32 changes I'd check before rolling the new image, with the numbers taken from the PRs themselves. Any estimate that is mine is labeled as mine.

1. max_tokens is no longer reserved from the context window

This is the change most likely to show up in your dashboards. Before 0.32, vLLM treated the client's max_tokens as a hard reservation: the largest prompt it would accept was the context length minus max_tokens. PR #57035 changes that so max_tokens is only an upper bound on output, which is how the OpenAI API spec describes it.

The PR's own example: a 128,000-token context, a 62,466-token prompt, and max_tokens: 65535. Before, the request was rejected, because 62,466 is one token more than 128,000 minus 65,535. Now the prompt is accepted and the sampling limit is clamped to 65,534. Coding agents that always send a large max_tokens were hitting exactly this.

What it means operationally, in my reading: long-prompt requests that used to bounce with a 400 will now be admitted. If your error rate drops after the upgrade, that's probably why. Those requests now take KV cache and prefill time they never got before. If a client was effectively being throttled by those rejections, expect more long prefills and possibly more preemption under load. Genuine overflow, where the prompt alone exceeds the context, is still rejected.

2. CORS no longer allows every origin

PR #61009 changes the default allowed origins from ["*"] to an empty list in both the Python and Rust frontends. A browser script on another origin no longer gets an Access-Control-Allow-Origin header back, and JSON preflight requests fail.

Server-to-server clients don't care about CORS, so most API traffic is unaffected. What breaks is any web UI that calls the vLLM endpoint directly from the browser. The fix is --allowed-origins with your real origins. The PR also points out that CORS is a browser control, not authentication, so a reachable server without an API key is still a reachable server.

3. NVFP4 KV cache now runs on Ampere, Hopper and consumer Blackwell, with an accuracy cost on small models

--kv-cache-dtype nvfp4 used to work only on SM100. PR #46963 adds SM8x and SM12x through FlashInfer's fa2 path, with no new CUDA kernel, and PR #60620 adds SM90 the same way. SM90 has no FP4 conversion instructions, so FlashInfer dequantizes in software there, as it does on SM8x.

The capacity gain is real. On the PR author's sm_80 host, KV capacity went from 1,446,932 tokens to 4,805,405, which is 3.32 times as many. My own back-of-envelope figure for the ceiling: NVFP4 stores 4 bits per element plus one FP8 scale per 16 elements, about 4.5 bits against 16 for BF16, or roughly 3.56 times the density. The measured 3.32 times is close to that.

The accuracy numbers in the same PR are the part I'd read twice:

Model (GSM8K, FlashInfer backend) BF16 KV NVFP4 KV
Qwen3-0.6B, all 1,319 questions 0.384 0.174
Qwen3-4B, 100 questions, two runs 0.920, 0.910 0.880, 0.880

On a long-context retrieval test (MRCR on Qwen3.5-4B, prompts of 16,626 to 33,231 tokens, 30 samples), the overall match ratio barely moved, 0.8523 to 0.8452, but the two-needle score fell from 0.9949 to 0.8951. Those MRCR numbers came from an earlier revision of the PR. The 0.6B model losing more than half its GSM8K score is a strong hint that small models with few KV heads have little room for 4-bit KV.

Two combinations are refused at backend selection on these GPUs, before the weights load: head size 64, where fa2 prefill misreads NVFP4 by 10 to 15 percent, and attention sinks. The PR says openai/gpt-oss-20b with an NVFP4 cache stops in 45 seconds with the head-size reason. The SM90 path was checked against FlashInfer source, not run on Hopper hardware, according to that PR. My takeaway: NVFP4 KV on an H100 or an RTX card is now worth an experiment, but I'd run my own evals on my own model before trusting it, and I wouldn't use it on anything around 1B parameters.

4. GLM-5.x quietly switches to FP8 KV on SM100

If you serve GLM-5.3 on B200-class GPUs with --kv-cache-dtype auto, you now get an FP8 KV cache by default (PR #60140). In the PR's benchmark (TP4, MTP with five speculative tokens, 8,000 input and 1,000 output tokens), throughput rose 2.3 to 5.5 percent and KV memory at a 458,880-token budget dropped from 40.13 GiB to 20.69 GiB per rank. TTFT got worse by 6.9 to 11.1 percent. With FP8 KV, the PR reports 0.9212 on GSM8K (flexible-extract).

That trade is reasonable for throughput-bound batch work. For an interactive service with a TTFT target, it's a regression you didn't ask for. Set the KV dtype explicitly if you need the old behavior.

5. API workers finally share the load

With --api-server-count above 1, connections used to land on whichever worker accepted first. PR #60723 sends each new connection to the worker with the fewest open ones. In one run on main with two data-parallel engines and eight API servers, connections per worker were 48, 15, 1, 0, 0, 0, 0, 0.

The PR's results on Qwen3-0.6B with 32 input tokens and 1 output token:

Config Concurrency Median req/s change Mean / p99 TTFT, before to after
DP2, 8 API servers 64 +85% 57/81 to 30/39 ms
DP2, 8 API servers 1024 +43% 685/5014 to 427/673 ms
DP1, 4 API servers 256 +48% 171/308 to 110/200 ms

A one-output-token workload is about as frontend-bound as serving gets, so I'd expect much smaller gains for long generations. The p99 TTFT improvement is the number that matters more to me, because it comes from the busiest worker no longer setting the tail. The same PR measured an RL abort cycle on GB200 going from 40.0 seconds to 21.5 seconds. A single API server is unchanged, and there are no new flags.

6. Security fixes with new limits

Three advisories are fixed. The one with operational knobs is GHSA-frvg-qw88-hww9 (PR #51504). A client could put unique, non-listening peer addresses in kv_transfer_params, and each one created three ZMQ sockets that were never cleaned up. About 340 such peers used up the ZMQ context's limit of 1,023 sockets and crashed EngineCore. The fix caps sessions with VLLM_P2P_MAX_PEERS (default 128), adds VLLM_P2P_HANDSHAKE_TIMEOUT_S (default 30) and evicts idle sessions after VLLM_P2P_IDLE_TIMEOUT_S (default 300). If you run P2P KV offload with more than 128 peers per engine, raise the cap before you upgrade, or new sessions will be refused.

The other two cover chat-template resource exhaustion, now rendered under VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT, and an off-by-one in the tokenizer's max_token_id. Untrusted images are now limited to raster formats, and JSON schemas nested more than 64 levels deep return a 400.

7. The breaking changes list

These are the ones I'd grep my deployment configs and client code for:

  • Python 3.10 is dropped. The minimum is 3.11.
  • transformers>=5.16.1,<5.20.0 and tokenizers>=0.23.2 are required. Pinned base images will need a bump.
  • Cascade attention is removed (PR #61126). It was off by default, so nothing changes for most setups, and the scheduler no longer scans each running request's blocks every step. --disable-cascade-attn still parses but only warns, and it goes away in v0.33.0.
  • --mamba-cache-mode all now fails validation.
  • The torch.compile sequence-parallelism and AsyncTP passes are gone, along with PassConfig.enable_sp and fuse_gemm_comms.
  • Responses API streaming renames the reasoning events to response.content_part.added and response.content_part.done with part.type="reasoning_text". Clients that parse the old event names will silently miss reasoning output.

8. Startup: run vllm download-kernels

On Hopper and newer, the worker now logs a warning when flashinfer-cubin or flashinfer-jit-cache is missing or doesn't match flashinfer-python, and names the fix: vllm download-kernels (PR #58765). Unlike FlashInfer's own CLI, it handles the stale-kernel state that every FlashInfer bump leaves behind. The official CUDA Docker image already installs them. If you build your own image, add the command to the build stage so the first pod doesn't spend its startup on JIT compilation. It pairs with the vllm preload daemon I wrote about in vLLM 0.31 Fast Restart: vllm preload --preload-autotune now keeps the FlashInfer autotune table on the daemon, so the tuning pass runs once per GPU instead of once per restart (PR #60085).

My upgrade order

  1. Check Python, transformers and tokenizers versions in the base image.
  2. Search clients for the old Responses reasoning event names, and browser UIs that hit vLLM directly. Set --allowed-origins where needed.
  3. Pin --kv-cache-dtype explicitly on GLM-5.x if TTFT matters.
  4. Raise VLLM_P2P_MAX_PEERS if you use P2P KV offload at scale.
  5. Add vllm download-kernels to custom images.
  6. After rollout, watch the 400 rate, long-prefill volume and preemptions. The max_tokens change moves those numbers in ways that look like a traffic shift.

If you want to try NVFP4 KV on Hopper, do it as a separate change with its own eval run, not as part of the upgrade.

Sources

Anpoo Sivanadi, Staff Software Engineer