OpenAI walked into Hot Chips 2026 on August 25 with the first measured results for Jalapeño, its Broadcom-built, TSMC-fabbed inference ASIC. The pitch is not “we invented a new FLOP.” It is tokens per megawatt at a latency an interactive agent can live with — and a claim that a first-generation captive part already beats shipping Blackwell racks on that axis.
I am reading the OpenAI scorecard, the SemiAnalysis lab write-up, and The Register’s rack math. This is create-only. Jalapeño is not for sale. Treat every “beats Rubin” line as analyst estimate until matched AgentX / MTP runs land.
Spec sheet (package + rack)
| Role | Inference-only ASIC (training stays on GPUs) |
| Partners | Design: OpenAI + Broadcom; fab: TSMC N3P compute + N3E I/O chiplet; systems: Celestica |
| Package TDP | 700 W rated; measured sustained ≤550 W on tested workloads |
| Memory | 216 GiB HBM4, six 12-high stacks, 15.4 TB/s (~10 Gbps pin) |
| Compute (B0) | ~13.4 PFLOPs MXFP4 per reticle-sized compute die (SemiAnalysis) |
| Scale-up / rack | 128 XPUs per accelerator rack; 1.7 EFLOPS MXFP4, ~27.5 TB HBM4, ~2 PB/s aggregate BW (Register) |
| Domain | Local: 128 XPUs over copper backplane; global: up to 2,048 XPUs across 16 racks (Tomahawk 6 + optics) |
| Host | Sidecar Katsu trays: 2× Turin-class EPYC, 1.5 TB DRAM, 400G frontend; ASIC trays nicknamed Vindaloo |
| Software | Gluon (Triton-family) + internal Teacup serving; Codex/GPT-Astra for kernels |
| Availability | Engineering samples now; deploy late 2026; volume ramp through 2027 |
| Buy / rent | No — captive OpenAI silicon |
Primary sources: OpenAI, SemiAnalysis, The Register, Locsic audit.
InferenceX table (STP, 8k in / 1k out)
OpenAI normalized by published package TDP (700 W vs 1,200 W GB200 / 1,400 W GB300). Mode: single-token prediction — no speculative decoding on the Jalapeño side of the published table.
| Model | Baseline | Peak mixed TPS/kW | E2E latency | Min TBT | Throughput @ baseline’s best TBT |
|---|---|---|---|---|---|
| GPT-OSS 120B | GB200 | 85,448 vs 44,960 (~1.9×) | 1.03 s vs 1.80 s (~1.7× lower) | 0.69 vs 1.87 ms (~2.7×) | ~53.7× (22,935 vs 427) |
| DeepSeek R1 670B MXFP4 | GB300 | 19,641 vs 11,781 (~1.7×) | 1.65 s vs 5.99 s (~3.6×) | 1.43 vs 5.90 ms (~4.1×) | ~104.3× (12,258 vs 118) |
| Kimi K2.5 1T MXFP4 | GB300 | 18,195 vs 11,862 (~1.5×) | 1.56 s vs 5.31 s (~3.4×) | 1.44 vs 5.48 ms (~3.8×) | ~56.1× (6,744 vs 120) |
SemiAnalysis verified InferenceX runs in OpenAI’s lab. They did not run the full suite or publish AgentX (long-context, multi-turn) yet. That matters: 8k/1k is a cleaner serving microbenchmark than agent traffic with prefix-cache and router stress.
The “100×” column is easy to misuse. It is not “100× faster chip.” It is “how many mixed tokens per kW you still get when you lock time-between-tokens to the baseline’s previous-best interactive point.” GB300 collapses on throughput when you force that latency; Jalapeño does not. Cite the conditional clause or do not cite the multiple.
Architecture: data movement, not peak density
SemiAnalysis’s B0 note is the useful correction: Jalapeño’s paper MXFP4 density is ~24% below a similar-size Rubin die on the same node (13.4 vs 17.5 PFLOPs), yet measured throughput per watt lands ahead of the published Blackwell tables. The gap is idle time and KV traffic, not a magic MAC.
Design thesis from OpenAI:
- Slice cores with matching HBM slices so each core keeps a low-latency local view of its memory
- Keep KV cache explicitly placed and local instead of bouncing it across a disaggregated prefill/decode fleet
- Homogeneous fungible XPUs rather than fixed prefill vs decode pools (traffic mix moves; stranded pools do not)
- Weight-stationary MXFP systolic with support for smaller matmul shapes (fewer cliffs than giant TPU-style arrays)
- Out-of-order cores + prefetch instead of only software scratchpads — SemiAnalysis flags the tradeoff: harder to reason about, friendlier to Codex-written kernels
Scale-up is rack-first: 128 XPUs local (4.8 Tb/s uni-dir per XPU on the local fabric in SemiAnalysis’s recount), then 2,048 across 16 racks. That is a different product shape from NVL72’s 72-GPU coherent domain. If your mental model is “one NVLink island,” update it — Jalapeño is betting that a larger, Ethernet/Tomahawk-class domain plus local KV beats moving state between specialized pools.
What the caveats actually are
- Blackwell, not Rubin, in the headline table. GB200/GB300 are HBM3E-era. Rubin ships HBM4. SemiAnalysis calls the Blackwell comparison “somewhat incomplete and unfair,” then says Jalapeño’s STP tokens/MW still look ahead of Rubin’s published July MTP numbers — preliminary, immature-vs-immature. Hold “beats Rubin” loosely until matched MTP + AgentX exists.
- Vendor-run InferenceX. Public harness, OpenAI fingers on the keyboard. SemiAnalysis watched; they did not own the full battery.
- STP vs production MTP. Real Nvidia fleets lean on speculative / multi-token tricks. The published table strips that from Jalapeño for cleanliness; production Blackwell may look better than the STP baseline.
- Engineering samples → 2027 volume. Late-2026 deploy is “very small” in Richard Ho’s framing; SemiAnalysis’s shipping model puts the main wave in Q4’27. Yield, dock-to-rack, and multi-rack ops are still ahead.
- Captive. You will feel this as OpenAI API latency and cost, not as a cloud SKU named Jalapeño.
- Training unchanged. OpenAI still needs Nvidia (and others) for training and for overflow inference. This is a denominator play on power-limited serving, not an exit from merchant GPUs.
How I would evaluate the claim if I ran clusters
- Demand tokens/s/MW at a fixed TBT, not peak FLOPs. Same model, same quant, same 8k/1k and a long-context AgentX-class trace.
- Insist on Rubin NVL72 as the peer, with MTP enabled on both sides or disabled on both. Blackwell-only tables are marketing scaffolding now.
- Separate prefill and decode curves. OpenAI’s “no PDD” bet only wins if the homogenous pool stays busy when the input/output mix swings.
- Track measured wall power, not just 700 W nameplate. Their ≤550 W sustained claim is the interesting number; verify it under your concurrency.
- Watch API economics after late-2026 deploy. Captive silicon shows up as rate limits and $/successful-agent-task before it shows up as a datasheet you can PO.
Where it sits next to what I already wrote about
| Rung | Product | Memory / power story |
|---|---|---|
| Deskside discrete | Threadripper Halo Station | 288–576 GB HBM3E on PCIe MI350P |
| Merchant rack | GB300 / Rubin NVL72 | Coherent GPU island, HBM3E→HBM4 |
| Captive inference | Jalapeño | 216 GiB HBM4 @ 15.4 TB/s, 700 W, 128→2,048 XPU domain |
Halo Station is the deskside HBM math problem. Jalapeño is the datacenter power-budget problem. Same decade, different purchase order.
Bottom line
Jalapeño’s first public scorecard is the strongest first-gen captive inference ASIC result I have seen with a third-party harness and named package numbers. The honest read is 1.5–1.9× peak mixed TPS/kW and materially lower latency vs Blackwell on InferenceX STP, with architecture that prioritizes KV locality and fungible XPUs over peak density. The dishonest read is “100× faster than Nvidia” or “CUDA is dead tomorrow.” Volume is 2027, Rubin is the real peer, and AgentX is still owed. Until those land, I treat Jalapeño as a verified tokens-per-megawatt existence proof — and a reminder that when the grid is the bottleneck, the winning datasheet is the one that keeps KV local and the rack busy.
Anpoo Sivanadi, Staff Software Engineer