Astera Leo X: Fabric-Attached DDR for KV Cache — 62% Faster Warm TTFT on H200 Math

This morning’s NewPhotonics NPG102 note was who ships the laser-integrated engines for Open CPX. Last night’s Cornelis + Delos note was who samples open fabric and I/O chiplets when scale-up is not NVLink. This evening is the memory path under both: where KV cache goes when HBM fills, and what changes if that tier attaches to the accelerator fabric instead of fighting CPU DDR5 or falling through to NVMe.

On September 15, 2026, Astera Labs announced three additions to its Leo Smart Memory Controller family: Leo X-Series (fabric-attached memory for accelerator-side KV-cache and agent-context offload) plus Leo 2 E-Series and Leo 2 P-Series (CPU-attached and pooled CXL memory on PCIe 6 / CXL 3.2). Primary sources: the Astera Labs press release, Astera’s KV-cache offload engineering blog, Converge Digest’s Sep 15 table, and ServeTheHome’s controller deep dive. Specs and TTFT/TPS deltas below are vendor claims from Astera’s internal test config unless a third-party pod result is cited.

Spec sheet (named numbers)

Leo X is the architectural sentence. Leo 2 E/P are the CXL 3.2 / PCIe 6 refresh that keep CPU-side expansion and rack pooling on the same family — useful on a BOM, secondary to the fabric-attach story for this note.

What “fabric-attached” changes on the KV path

The usual KV-offload ladder after GPU HBM fills: spill to host DDR5, then NVMe, or chase remote pools over RDMA/TCP. Astera’s blog walks the failure modes:

1. Host DDR5 is fine until multi-GPU nodes plus tokenization/RAG/orchestration contend for the same channels — restores that should be memory-bound become arbitration-bound.
2. NVMe scales capacity and kills TTFT / decode TPS when restores are slower than recomputing tokens.
3. Network pools move gigabytes of KV across NICs and add congestion you did not budget in a single-node decode model.

Leo X’s claim is a dedicated memory tier on the accelerator (or host PCIe) fabric, reached through Scorpio peer-to-peer paths that bypass the CPU root complex. That is a different purchase than “add another CXL expander behind the host” — KV capacity as a scale-up fabric citizen, the way Cornelis/Delos treat collectives and I/O dies.

ServeTheHome’s color: Scorpio is primarily a PCIe switch family; high-end Scorpio X-Series can also speak proprietary hyperscaler fabrics. Astera does not name which fabrics are in the Leo X win set. Read “platform-specific protocols” as co-design with a specific accelerator or CSP, not drop-in UALink/NVLink silicon.

The published H200 / Qwen2.5-32B table (and what it is not)

Astera’s engineering blog puts numbers on one topology. Hardware: 1S server, 64 physical cores / 128 logical CPUs, 4× H200 143 GB, 1 TB DDR5 on NUMA0 (~200 GB/s peak), one Leo X with 256 GB, Scorpio P-Series 320L. Software: vLLM v1 TP=4, LMCache v0.4.2 + patch registering Leo as I/O memory. Workload: Qwen2.5-32B, 10 sessions with growing 32k-token contexts, warm turns restoring KV from the offload pool; optional CPU stressors (threads with 2× 4 GB buffers) to simulate agentic host traffic.

Reported warm TTFT:

• CPU-DRAM baseline: 246 ms average warm turn → Leo path: 127 ms
• With 32 stressor threads: CPU-DRAM 413 ms → Leo 158 ms → the cited 62% faster TTFT
• Throughput: up to 22% more tokens/sec on the 10-session / cold+warm-turn mix (Astera says the 22% held for deeper 20–50 warm-turn sessions in their runs)

Three caveats belong on any capacity ticket before this becomes a CapEx assumption:

1. Gen-5 foil, not Rubin/Blackwell. The published accelerator is H200. ServeTheHome notes the same. Do not transpose 62% / 22% onto a Vera Rubin NVL72 or MI355X pod without re-measuring.
2. Vendor internal, LMCache-patched. The PR footnote and blog both attribute the deltas to Astera’s own lab. The LMCache change is “allocator-level” (register Leo as I/O memory); still not a stock vLLM config you can paste from docs today.
3. 256 GB demo ≠ 2 TB claim. The measured config is 256 GB per Leo X (one per GPU in their narrative). The 2 TB per device figure is a product capacity claim — keep them on separate rows.

How Leo 2 E/P sit next to Leo X

Do not collapse the family into one “CXL memory” slide.

• Leo 2 E is classic CPU-attached expansion: CXL 3.2 on PCIe 6 x16, four memory controllers, DDR4/DDR5 — add capacity without another CPU socket, reuse pulled DDR4 with Astera’s RAS / test / soft-ECC story.
• Leo 2 P is pooled/shared memory across hosts: dual PCIe 6 x8, dynamic capacity management — the stranded-DIMM utilization argument under DDR5 shortage.
• Leo X is the accelerator-fabric attach for KV/agent state.

ServeTheHome’s capacity table — 768 GB DDR4 or 4 TB DDR5 per expander, including aggressive 3DPC DDR4 boards — is the BOM detail if you are chasing reused DDR4 economics rather than fabric TTFT.

How this sits next to NVHBM, Cornelis/Delos, and NewPhotonics

• NVHBM — custom HBM / package memory inside the accelerator envelope.
• Leo X — DDR4/DDR5 fabric tier for KV that should not burn HBM.
• Cornelis + Delos — open scale-up fabric compute and >30 Tbps I/O dies; Scorpio/Leo can ride PCIe-class fabrics without being those products.
• NewPhotonics NPG102 / Lightmatter Passage L20 CPX — optical engine and BiDi plant once scale-up goes NPO/CPO.

Four purchase domains — do not write “memory wall solved” on one slide.

Caveats I would write into any fabric-KV ticket

1. Sampling ≠ fleet. Hyperscaler sampling and AI Infra Summit demos are not a published CSP SKU with FITs and firmware rev.
2. Protocol opacity. “Platform-specific” fabric support is a design-win sentence. Ask which Scorpio mode (PCIe P2P vs proprietary) the win uses before modeling radix and hop count.
3. DDR4 reuse RAS is half the product. Astera leans hard on test engines, soft-ECC, scrubbing, and COSMOS telemetry for aged DIMMs. Budget qualification time; do not assume pulled DDR4 is free capacity.
4. No pJ/bit or fabric RTT table in the PR. Compare Leo X restores to your measured HBM and DDR5 paths on your model and concurrent session count.
5. Do not mix Leo X TTFT with OceanStor / Ascend SuperPoD KV numbers. Different vendors and media — and the Ascend 960 SuperPoD lane stays closed on this desk.

How I would use this tomorrow morning

Put a Leo X (fabric KV) / Leo 2 E (CXL expand) / Leo 2 P (pool) row next to your HBM and host-DDR5 model so offload topology is an explicit choice, not an LMCache default. If you are already sizing Cornelis CN6000 or Scorpio-class PCIe fabrics, ask whether a fabric-attached DDR tier collapses TTFT variance under multi-tenant host stress the way Astera’s 413 → 158 ms case suggests — then re-run on your accelerator generation. On NVLink Fusion or UALoE, read Leo as memory companion silicon, not a fabric change.

Create-only. Specs and dates from Astera’s Sep 15 release, the KV-offload blog’s H200 / Qwen2.5-32B config, Converge Digest’s portfolio table, and ServeTheHome’s Leo 2 capacity notes. Update when a CSP publishes a production Leo X topology, or when Astera posts Blackwell/Rubin-class foils with the same TTFT methodology.

Sources: Astera Labs PR · Astera KV-cache blog · Converge Digest · ServeTheHome.

Anpoo Sivanadi, Staff Software Engineer