The llm-d maintainers published llm-d v1.0 on October 20, 2026. It's the Kubernetes-native distributed inference stack that sits in front of vLLM and SGLang and handles prefix-aware routing, KV cache offload, prefill/decode disaggregation and wide expert parallelism. It is a CNCF sandbox project, and the announcement counts more than 500 contributors from over 25 organizations.

The announcement is about "Day 2": multi-tenant quality of service, router high availability, coordinated upgrades of disaggregated deployments, and agent-aware serving. But a 1.0 tag doesn't change what I check before rolling a serving stack: which versions got pinned, which prerequisites moved, and which names changed under my dashboards. The v1.0.0 release notes answer most of that, and they're refreshingly honest. The first line links the release goal issue, #2619, and adds "although not all of that was accomplished."

Here's what I'd check, with numbers from the release notes and the announcement. Anything that is my reading rather than theirs is labeled.

1. Every component is now v1.0.0, so read the previous-version column

The project used the 1.0 cut to align versions across the organization, and says it intends to keep major and minor versions aligned from now on. That's welcome, but it hides how far some pieces jumped. The Endpoint Picker comes from v0.11.0, the router coordinator from v0.1.0, and the batch gateway from v0.6.0. A matching major number says nothing about how much changed underneath.

Two structural moves sit in that table. The llm-d-kv-cache library has moved into llm-d-router and is only kept around while users migrate. And ghcr.io/llm-d/llm-d-rocm image builds are deprecated, which follows v0.10 deprecating the project's own CUDA and AWS images. The project is getting out of the business of building inference images and pointing at upstream vLLM, SGLang and TensorRT-LLM images instead.

2. The engine versions differ per accelerator

The core component table lists the vLLM base as v0.31.0, "varies per accelerator." The model server image table spells that out:

  • vllm/vllm-openai (the CUDA image): v0.30.0, up from v0.26.0
  • vLLM TPU, XPU, ROCm and CPU images: v0.31.0
  • vLLM Torch TPU, new in this release: v0.29.0
  • ROCm with P/D support: a temporary ghcr.io/llm-d/vllm-openai-rocm:v0.32.0-rc.2, cherry-picked into llm-d's registry to protect it from image garbage collection
  • SGLang: v0.5.22, up from v0.5.20
  • TensorRT-LLM: 1.3.0rc29

My reading: a single llm-d 1.0 deployment can run four different vLLM minor versions depending on the hardware. On NVIDIA GPUs the stock guides are on 0.30, so none of the changed defaults I covered in vLLM 0.32 Upgrade Notes apply, and neither does anything new in 0.31. On AMD with disaggregation you're on a 0.32 release candidate. If you validated client behavior against one engine version, check which tag your guide actually pins before assuming it carries over. The SGLang line is the same 0.5.22 I wrote about in SGLang 0.5.22 Upgrade Notes.

For supply-chain policy: quay.io holds unsigned PR builds, and ghcr.io holds release images signed with cosign. If your admission policy verifies signatures, make sure no overlay still points at quay.io.

3. Disaggregated serving now needs a newer LeaderWorkerSet

This is the change most likely to stop a rollout. DisaggregatedSet is the API that manages prefill and decode replicas as one unit. It gates readiness on a snapshot of the underlying pods and routes by revision, so a request only reaches a prefill/decode pair when both sides are rolled out and compatible. It lives in LeaderWorkerSet, not in llm-d.

The release notes say the P/D disaggregation guide now requires LeaderWorkerSet v0.11.1 or newer with the DisaggregatedSet API turned on (--set enableDisaggregatedSet=true in Helm). They recommend LWS v1.0.0, and they warn that "almost all guides will break without v0.11.1+." The changelog shows why: most P/D paths, NVIDIA and TPU 7x included, were converted to DisaggregatedSets (#2626, #2709, #2784).

My reading: upgrade LWS and enable the API as a separate step, before you touch llm-d. Then read the new safe rollout guide for DisaggregatedSet P/D before the first in-place upgrade. Lockstep upgrades are the headline feature, and also where a mistake takes out both halves at once.

Also on disaggregation: a docs change (#2680) states that SGLang P/D disaggregates unconditionally. If you assumed short prompts could skip the prefill pool, as some selective P/D setups allow, that assumption doesn't hold on the SGLang path.

4. Flow control is opt-in, but its config moved

v1.0 puts priority- and fairness-aware flow control in the router. Each request gets a flow key made of a fairness ID (the tenant or workload) and a priority band. Each flow gets its own queue. Dispatch goes by priority first, then round-robins across tenants within a band. Under saturation, the router holds requests instead of pushing them into backend queues that are already overloaded, and a probabilistic admission controller sheds load instead of failing hard.

The part that matters for an upgrade is that all of this now sits under one flowControl configuration block, and the defaults still behave like first-come-first-served. So an existing deployment shouldn't see different scheduling just from upgrading. Any flow-control settings you set by hand on 0.x need to move into the consolidated block. The router configuration API also graduates to v1, which is the stability promise you want before standardizing a platform on it.

My reading: turn on multi-tenant policy as its own change, after the upgrade has soaked, and measure before and after. "Holds requests in the router" is the right behavior under overload, but it moves where queueing time shows up. Your backend queue-depth alerts will look calmer while client-side latency stays the same. Watch router-side queue metrics too.

5. Router HA has trade-offs worth reading first

The Endpoint Picker can now run with a hot standby that takes over if the active instance fails or is replaced, and there's a new guide on HA switchover trade-offs (#2715). Active-active routers and in-flight request failover are listed under "what's next," so plan for a switchover window, not seamless failover.

6. Your dashboards may break quietly

Two changelog lines look like housekeeping but can silently break observability:

  • Grafana dashboards are now prefixed and tagged with llm-d (#2789). Links, folder rules and provisioning that match on the old titles can miss them.
  • The docs replaced vLLM metric names that current vLLM no longer exports (#2835). If your alerts were copied from older llm-d docs, some may be querying series that no longer exist. An alert on a missing series doesn't fire. It just goes quiet.

I'd grep alert rules and dashboard JSON for every vLLM metric name you use and confirm each one returns data on the new images before you cut over.

7. Agent-aware serving changes cache behavior

Session-aware routing pins a multi-turn conversation to the replica that already holds its KV cache, "sticky until saturated," and then rebalances. Turn-priority scheduling keeps an active agent loop responsive. Native Responses API state, including previous_response_id, means an agent doesn't have to resend its full history.

These change your cache-hit and load-balance numbers. My reading: expect less even load across replicas and higher prefix-cache hit rates on agent-heavy pools, and re-baseline autoscaling thresholds instead of carrying 0.x numbers forward.

8. New tools worth trying first

llm-d-pd-utils provides preflight diagnostics, GPU topology discovery and network benchmarks for disaggregated deployments, and it's what I'd run first on any cluster where KV transfer between pools has been flaky. ModelExpress v0.7.0 adds peer-to-peer weight transfer for faster cold starts; it's the same version Dynamo 1.6 ships, which I covered in Dynamo 1.6 Upgrade Notes. A DRA/DRANET guide covers topology-aware placement, using the NVIDIA DRA ComputeDomain on GB200 NVL72 to keep tightly coupled parallelism inside one NVLink domain.

On the production numbers

The announcement leads with adopter results. IBM Research reports up to 3,000 concurrent coding agents on 544 H100 GPUs, a roughly 98% prefix-cache hit rate, and about $0.11 per million tokens, which it puts at around 11.5x cheaper than a proprietary stack. On TPU 7x, prefix-aware routing plus host-memory KV offload cut median TTFT by more than 85% and raised throughput 2.4x compared with a plain Kubernetes Service. These are self-reported, workload-specific results against weak baselines; a plain Service has no cache awareness at all. They show what prefix-aware routing is worth on high-reuse agent traffic, not what your fleet will see.

Pre-upgrade checklist

  1. Write down which vLLM or SGLang tag your guide pins for each accelerator, and diff engine behavior against what you validated.
  2. Upgrade LeaderWorkerSet to v0.11.1 or later (ideally v1.0.0) and set enableDisaggregatedSet=true before upgrading llm-d.
  3. Move any hand-set flow-control settings into the flowControl block, and leave multi-tenant policy off for the first rollout.
  4. Point image references at signed ghcr.io images, not quay.io.
  5. Check every alert and dashboard against the renamed dashboards and current vLLM metric names.
  6. Read the HA switchover and DisaggregatedSet safe-rollout guides before the first in-place upgrade.
  7. Run llm-d-pd-utils preflight checks on the target cluster.
  8. Re-baseline autoscaling and cache-hit expectations once session-aware routing is live.

Vera Rubin support is still listed as in progress.

Sources

Anpoo Sivanadi, Staff Software Engineer