Staff Software Engineer · Tesla Design Technology · Bay Area

Anpoo Sivanadi

I build reliable software platforms and the engineering systems that help teams ship with confidence.

github linkedin cv

Selected work

Talk

Real Work, Real State: How TESLA Leverages 3DEXPERIENCE R2025x on Linux

3DEXPERIENCE & ENOVIA User Conference 2026 · Session 207 · 11 March 2026 · with Jay Vyas
Writing

Humming Is Already in Your vLLM Install. Here's What the Quantized Kernel Numbers Actually Say.

On October 21, 2026, Ant Group and the vLLM community published Announcing Humming: Fast, Flexible Kernels for Quantized Inference.
Writing

vLLM and SGLang Both Caught DeepInfer on DeepSeek-V4.1-Flash. Read the Setups Before the Headlines.

A few days after DeepSeek open-sourced DeepInfer, its own engine for DeepSeek-V4.1-Flash, both big general-purpose engines answered.
Writing

Million-Token GLM-5.3 on 16 H200s: What PCP, DCP and HiSparse Actually Buy You

The vLLM team published GLM-5.3 Optimizations, Part 2 on October 23, 2026.
Writing

llm-d 1.0 Upgrade Notes: Pinned Engines, a DisaggregatedSet Prerequisite, and Quiet Dashboard Breaks

The llm-d maintainers published llm-d v1.0 on October 20, 2026.
Writing

MiniMax-M3.1-Flash on vLLM: What an NVFP4 KV Cache Changes for Capacity Planning

MiniMax put MiniMax-M3.1-Flash on Hugging Face on October 23, and the vLLM team published a day-0 support post the same day.
Writing

Dynamo 1.6 Upgrade Notes: Renamed Workers, a Mandatory Queue, and the Known Issues That Matter

NVIDIA's distributed inference platform Dynamo v1.6.0 was published at 22:31 UTC on October 22, 2026, which was 3:31 PM PT.
Writing

Raon-1 mini Preview: 8 KiB of KV per Token, and a Commit Log Worth Reading

KRAFTON released Raon-1 mini preview on October 21 under Apache 2.0.
Writing

DeepSeek Open-Sourced DeepInfer. The Deployment Q&A Is the Part to Read.

DeepSeek put DeepInfer on GitHub this week under Apache 2.0.
Writing

Reading Flint's Serving Kit Before the Benchmarks: A 40B MoE That Wants One Big GPU

Recursive released Flint today, an Apache 2.0 mixture-of-experts coding model with 39.8B total parameters and about 2.5B active per token.
Writing

The Layer Your Autotuner Never Touches: Helion Now Searches ptxas Controls

Every kernel autotuner I've used searches the same kind of thing.
Writing

TorchTPU Is Open Source: What Running PyTorch Natively on TPUs Actually Involves

This morning I wrote about TorchNeuron and noted that TorchTPU was only a conference session with a repo on the way.
Writing

TorchNeuron Public Beta: What Changes When Trainium Becomes a Normal PyTorch Device

AWS opened up TorchNeuron today at PyTorch Conference North America in San Jose.
Writing

Meta's Prometheus Numbers: What 64-of-72 Sparing and a 54K GPU-Hour MTBI Mean for NVL72 Operators

Meta published Inside Prometheus on October 19, 2026.
Writing

vLLM 0.32 Upgrade Notes: The Defaults That Changed Under You

vLLM v0.32.0 was published at 06:50 UTC on October 18, 2026, which was 11:50 PM PT on October 17.
Writing

SGLang 0.5.22 Upgrade Notes: Flags That Changed Meaning

SGLang v0.5.22 was published at 02:55 UTC on October 19, 2026, which was 7:55 PM PT on October 18.
Writing

Dory1.0-4B: What a Looped 4B Model Actually Costs Per Token

NVIDIA put Nemotron-Labs Dory1.0-4B on Hugging Face on October 16 under the OpenMDW-1.1 license.
Writing

Granite Switch 4.1: Three Models' Worth of Jobs on One KV Cache

IBM shipped Granite Switch 4.1 this week in 8B and 30B sizes, Apache 2.0.
Writing

vLLM's Two New KV Transfer Modes, and Why You Pick One

Two posts on the vLLM blog this week change how KV cache moves between prefill and decode workers in a disaggregated deployment.
Writing

Ling-3.1-flash: What 63 Linear-Attention Layers Do to the KV Budget

inclusionAI, the Ant Group team behind the Ling, Ring and Ming model families, put the weights for Ling-3.1-flash on Hugging Face this week under the MIT license.
Writing

Nemotron 3.5 Super VL and Bolt: Hybrid MoE Served From Rack to 24 GB

NVIDIA put Nemotron 3.5 Super VL and Bolt on Hugging Face on October 15, the same window RTX Spark PCs started shipping.
Writing

Muse Spark Open: Latent MoE, Hybrid Attention, and a 1M Context That Actually Fits

Meta put the Muse Spark Open weights on Hugging Face today, October 15, the same day vLLM shipped day-0 support and NVIDIA published GB300 NVL72 serving numbers.
Writing

Step 5 Preview Weights Are Out: 600B, 92 Layers, and a 74 GB Million-Token KV Cache

StepFun put the Step 5 Preview weights on Hugging Face last night, the same day SGLang published its day-0 support post.
Writing

Intel GPU D270: 480 GB on a PCIe Card, and What That Memory Is Actually For

Intel used the OCP Global Summit this week to put final numbers and a name on Crescent Island.
Writing

ReplaySSM: Hybrid Models Were Paying an HBM Tax on Every Token

A lot of the models people serve this year are hybrids.
Writing

Gemini 4 Argon: One 1M-Token Budget, a Continuation Protocol, and a Scorecard That Doesn't Win Everywhere

Google shipped Gemini 4 Argon today, October 13, as the first model in the Gemini 4 family.
Writing

Maia 300: Microsoft’s Inference ASIC Puts 288 GB of HBM4 and a 12,288-Chip Ethernet Superpod on the Table

Microsoft used the same stage to put numbers on Azure Maia 300, the third generation of its in-house inference accelerator and the first Maia built as a multi-chiplet part.
Writing

OSPA: Meta, Microsoft and Alibaba Want Every AI Fabric Switch to Boot the Same Way

On October 8, OCP posted Open Switch Platform Architecture, a draft under OCP review initiated by Meta, Microsoft, and Alibaba: a vendor-neutral switch platform layer so AI fabric SKUs can boot, inventory, and flash the same way.
Writing

Mellum2.1: A 12B MoE Coding Agent Model, and the GGUF Builds That Make It Runnable Tonight

On October 8, JetBrains released Mellum2.1, the post-trained successor to the Mellum2 Thinking checkpoint they open-sourced in June.
Writing

Liquid Open d1: Zero Output Tokens, Named Jetson Latencies, and What a Decision Model Actually Is

On October 7, Liquid AI open-weighted two models in its d1 family: d1-3B and experimental d1-omni-600M.
Writing

Vera Rubin Day Zero: What SGLang Changed in the Kernels, and What vLLM Measured on AgentX

On October 9, two open serving stacks published the first useful Vera Rubin numbers that are not a keynote slide.
Writing

RTX Spark N1X: The Windows Numbers That Matter Are TDP, Memory Ceiling, and Prefill

On October 7, NVIDIA and Microsoft opened Windows pre-orders for RTX Spark laptops (availability October 16; compact desktops in November) and previewed DGX Station for Windows for Q4.
Writing

Scality AI Inference Factory: Object Storage Over RDMA as the Shared KV Cache

Scality launched AI Inference Factory on October 8: an open-code stack for serving open-weight models on infrastructure you own, with Scality ADI as a shared key-value (KV) cache behind the GPUs.
Writing

Upscale Token Fabric: SkyFabriX at 115.2 Tbps Meets Spectrum-X Scale-Out Under One OS

A day after Arista’s Etherlink SU-144, Upscale published Token Fabric: SkyFabriX at 115.2 Tbps, Spectrum-X scale-out, and one OS for both, with general availability aimed at early 2027.
Writing

Claude Haiku 5.5: The Price Cut Is Real, but Read the 100K Tier, the Tokenizer, and the Output Bill

Anthropic released Claude Haiku 5.5 on October 7: the small model next to Sonnet 5.5 and Opus 5.5 for high-volume work, with a real price cut that still needs the 100K tier, the tokenizer, and the output bill read first.
Writing

Arista Etherlink SU-144: Ethernet Moves Inside the Scale-Up Domain, and the Rack Becomes the Unit

Arista’s Etherlink SU-144 puts 144 accelerators in a single-hop Ethernet scale-up domain inside one rack, and up to 1,024 across racks, on OCP ESUN 1.0 — a partner-built reference design, not a SKU.
Writing

openTPU: An Agent-Built LLM Accelerator on a Kintex-7 PCIe Card, and What Its Numbers Teach

openTPU is a complete Apache 2.0 LLM inference stack on a Kintex-7 PCIe card — SystemVerilog RTL, ISA, bit-exact simulator, and host tools — with self-reported decode numbers that put memory-bound inference on one line.
Writing

AICR v1.0: NVIDIA Freezes the Contract for GPU Kubernetes Recipes

NVIDIA’s AICR v1.0.0 freezes the aicr CLI, aicrd REST API, Go SDK, and bundle schemas — a cluster configuration generator for GPU Kubernetes recipes, not a distribution or provisioner.
Writing

Mistral Large 4: 1.05T Parameters, 49B Active, and What It Takes to Serve It

Mistral’s October 6 Large 4 is a 1.05T / 49B-active open-weight multimodal MoE with a 1M-token context — public-preview API today, weights and a custom license still October 27.
Writing

Reflection’s Beam: A 501B Open-Weight MoE, and the GB300 Cluster Numbers Behind It

Reflection’s October 5 Beam is a 501B / 23B-active sparse MoE for coding, reasoning and agentic work — self-reported GB300 cluster numbers, with weights and the tech report still later this month.
Writing

vLLM 0.31 Fast Restart: A GPU Weight-Cache Daemon and CRIU Engine Snapshots

vLLM v0.31.0 Fast restart: a GPU weight-cache daemon and experimental CRIU engine snapshots — project claims of minutes-to-seconds restarts, with an open fingerprint bug and Kubernetes caveats.
Writing

TLX JFA: Meta’s Blackwell Attention Kernel — ~13% Fwd / ~50% Bwd vs FA4 on Jagged

Meta’s Oct 1 TLX Jagged Flash Attention is a Blackwell B200 kernel for packed jagged / broadcast-Q — Meta’s bf16 benches claim ~+13% forward / ~+50% backward vs May 2026 FA4 on their shapes, not an independent rematch.
Writing

Dist-MoE: Meta’s Blackwell SM100 MoE Kernels — BF16 / MXFP8 Train, NVFP4 Infer

Meta’s Oct 3 dist-moe v0.1.0 is CuTe DSL distributed MoE kernels for Blackwell SM100 — BF16 / MXFP8 E4M3 train+infer, NVFP4 infer-only — Meta’s docs, not a published throughput rematch.
Writing

Kolibri-1: 78.1B / 3.46B Open MoE — Hybrid SWA, 1M Context Claim, ~78 GB FP8

Aleph Alpha’s Oct 3 Kolibri-1 is a 78.1B / 3.46B open MoE with hybrid 4:1 SWA(512):full, native 256k validated to 1M, and ~78 GB FP8 — vendor benches and a named B200 train topology, not an independent rematch.
Writing

Credo Weaver: HBM4-Class Edge Bandwidth at ~25× Capacity — OmniConnect VSR, No CoWoS

Credo’s Oct 2 OmniConnect / Weaver post claims HBM4-class die-edge bandwidth (6.4 vs 6.1 TB/s) at ~25× capacity (4.8 TB vs 192 GB) on organic, no CoWoS — vendor architecture claims, not an independent bring-up.
Writing

Lightning V2 on Tenstorrent: 4× TTS Cost Cut vs L40S — BFP8, NoC, Perceptual Fragility

Tenstorrent and Smallest.ai put Lightning V2 on Blackhole — vendor L40S bake-off with ~3–4× modeled TTS CapEx, BFP8/LoFi co-design, and a paper on perceptual fragility — not a blanket NVIDIA cost claim.
Writing

Olmo-core 3: Open MoE Training Stack Hits 1.2T on B300 — DDP, Rowwise EP, MXFP8

Ai2 released Olmo-core 3 — open DDP+EP+PP MoE trainer on NVIDIA B300 through 1.20T / 58.4B active, 19.4K → 52K tok/s/GPU at 47B vs the old FSDP path — systems measurements, not next-Olmo weights.
Writing

Everspin Hooks PERSYST MRAM to CXL — 1GB Persistent Demo Tier Between DRAM and NAND

Everspin demonstrated CXL-connected PERSYST 1GB MRAM at SNIA SDC 2026 — memory-semantic persistence between DRAM and NAND, vendor 100× writes vs NAND, working demo not a shipping SKU.
Writing

CoreWeave Puts Vera Rubin NVL72 Into Production With Cognition — 4.8× SWE-2 Tokens, Not Another Shipping BOM

CoreWeave puts NVIDIA Vera Rubin NVL72 into limited availability with Cognition as first production customer — 4.8× SWE-2 and 3.8× RL token throughput vs GB200 (customer claims) — neocloud production, not another shipping BOM.
Writing

Claude Sonnet 5.5: Everyday Coding Ladder Beside Opus 5.5 — Cost Per Task, Not Another Flagship Rematch

Claude Sonnet 5.5: same $2/$10 list as Sonnet 5, vendor fewer-tokens plus 30%+ faster generation, near-Opus on several Max benches — everyday coding ladder beside live Opus 5.5, not another flagship rematch.
Writing

Naive-N0.5-Flash: 309B / 15.5B MoE with Zero Full-Attention Layers — Native 1M via SWA+DSA

Naive-N0.5-Flash: open-weight 309B / 15.5B MoE with zero full-attention layers, native 1M via SWA+DSA, MIT, announced $0.10/$0.40/$0.01, NaiveRT peak you should re-time — decode-cost architecture, not an AA rematch.
Writing

NVIDIA Open Agent Safety: OpenShell 0.1.0 + BlueField-4 Sentry — Runtime Policy, Milliseconds Quarantine

NVIDIA Open Agent Safety: Apache-2.0 OpenShell 0.1.0 plus optional BlueField-4 Sentry — YAML → OPA/Rego, OCSF, milliseconds quarantine as a vendor claim, no DPU required — runtime policy for agent fleets, not an AA rematch.
Writing

JEV-27B: Open System 1 Decisions on a Frozen Qwen3.8-27B — KL 0.017, One B200

AutoTrust JEV-27B: open System 1 decisions on a frozen Qwen3.8-27B — KL ≈0.017, six-bench mean 84.07, 137 ms on one B200 — vendor self-comparison vs hosted TypeSafe Jev 1.13, not a third-party bake-off.
Writing

Holo4: Computer-Use Agents With Named $/Task — 27B Dense, 35B-A3B MoE, Open Trajectories

H Company Holo4: 27B dense + 35B-A3B MoE computer-use VLMs with named OSWorld / AutomationBench $/task and open trajectories — no AA Index row, and a CC BY-NC vs Apache-2.0 license split you should write into the routing ticket.
Writing

AliceAI-Foundation-80B: From-Scratch 3B Activate, KDA:GA 3:1 — and Two Open Russian Factual Benches

Yandex AliceAI-Foundation-80B-A3B-Base: from-scratch 80B/3B hybrid MoE, KDA:GA 3:1, Apache 2.0, WikiWebFacts / HardMultiQA — no AA Index row yet, and a base you still owe SFT/RL before you compare it to Flash/Pro chat endpoints.
Writing

DeepSeek-V4.1-Flash: 890 Bytes of KV per Token — and Why V4-Pro Is Being Retired

DeepSeek-V4.1-Flash: 890 bytes KV/token, 8B prefill / 16B decode MoE, MIT weights, V4-Pro traffic steered to Flash rates — AA Index 39, with a Terminal-Bench 4.0 gap vs Opus/Sol on DeepSeek’s own table.
Writing

MiMo-V2.6-Pro: Open Weights at AA 46 — and the Terminal-Bench 4.0 Asterisk

Xiaomi MiMo-V2.6-Pro / Flash: MIT open weights, AA Intelligence Index 46, $0.435/$0.87 list (Flash cheaper), same-day foil to Grok 4.7 — with a Terminal-Bench 4.0 gap vs Opus/Astra on Xiaomi’s own table.
Writing

Lessengers’ 3.2T Glass NPO: TGVs, Lens-Free DOW, and the Millimeters-from-ASIC Bet

Lessengers’ 3.2T glass-substrate NPO: TGVs, lens-free DOW polymer waveguides, millimeters-from-ASIC placement, immersion-native claim, aimed at 51.2T/102.4T switch generations — vendor claims until your BER and yield tickets land.
Writing

PicoJool’s 200G GaAs VCSEL: >37 GHz, Three Paths to 1.6T, and the Unconstrained-Supply Bet

PicoJool’s 200G GaAs VCSEL / microVCSEL bet: >37 GHz device claims, three 1.6T recipes, WIN foundry, Series A funding qualification → early-2027 volume — vendor claims until your BER tickets land.
Writing

Supermicro Is Shipping Vera Rubin NVL72: 20.7 TB HBM4 and a Cold-Plate-to-Tower BOM You Can Order

Supermicro says Vera Rubin NVL72 racks are shipping now — 72 Rubin GPUs, 20.7 TB HBM4, and a cold-plate-to-tower DLC BOM you can order as a 16-rack Scalable Unit.
Writing

EdgeCortix RAIDEN: 3.36 PFLOPS FP4 on Four Dies — Thick-Edge Physical AI as One Chiplet Stack

EdgeCortix RAIDEN claims up to 3.36 PFLOPS FP4, 256 GB, and 1.54 TB/s die-to-die on a four-die Physical AI chiplet stack shipping samples in early 2027.
Writing

WhiteFiber Continuum: 136 Tbps Across 83 km — Scale-Across When One Campus Hits the Power Wall

WhiteFiber Continuum claims 136 Tbps / 0.9 ms RTT across 83 km of metro fiber so two campuses can act as one GPU supercluster when one site hits the power wall.
Writing

Claude Opus 5.5: Fable-Class Work at Opus Prices — and the Safeguard Asterisk

Claude Opus 5.5 — Fable-class work at $4/$20, and the fallback-safeguard asterisk.
Writing

GPT-6 Sol and Luna: Half the Price, Not Always a Higher Ceiling

GPT-6 Sol and Luna — half the GPT-5.6 promo price, cache-steerable effort, not always a higher ceiling.
Writing

T-Head Zhenwu V900: 216 GB, 1,200 GB/s Chip-to-Chip, and a 500,000-Card Cluster Story

Zhenwu V900 — 216 GB, 1,200 GB/s chip-to-chip, and a 500,000-card cluster story.
Writing

Axelera Europa: 629 TOPS at 45 W — Shipping Edge 232p Math With a Dell/Supermicro Asterisk

Europa 629 TOPS at 45 W — shipping Edge 232p math with a Dell/Supermicro asterisk.
Writing

Lightbits Inferra: KV Cache Prefetch Beyond HBM — Named TTFT Tables on L40S Math

Inferra KV-cache prefetch beyond HBM — named TTFT tables on L40S math vs recompute.
Writing

Astera Leo X: Fabric-Attached DDR for KV Cache — 62% Faster Warm TTFT on H200 Math

Leo X fabric-attached DDR for KV cache — 62% faster warm TTFT on H200 math vs CPU-DRAM offload.
Writing

NewPhotonics NPG102 + Tower PH18DA: Laser-Integrated PICs Shipping for 800G–1.6T — and the CPX 6.4T On-Ramp

NPG102 + Tower PH18DA — laser-integrated PICs shipping for 800G–1.6T, plus the CPX 6.4T on-ramp.
Writing

Cornelis Active Compute Fabric + Delos Nonstop AI: Open Scale-Up Math Beyond NVLink

Cornelis ACF + Delos Nonstop AI — 800G fabric sampling and >30 Tbps I/O dies beyond NVLink.
Writing

AMD EPYC Venice vs Nvidia Vera: 2.24× SPECrate, 1.6 TB/s Stream — and the Compiler Asterisk

EPYC Venice vs Vera — 2.24× SPECrate and 1.6 TB/s Stream, plus the compiler asterisk.
Writing

Meta MTIA 450 / Arke: 18.4 TB/s HBM, FlashAttention in Silicon — and the Sep 1 Bring-Up Asterisk

MTIA 450 / Arke — 18.4 TB/s HBM and FlashAttention in silicon, plus the Sep 1 bring-up asterisk.
Writing

Lightmatter Passage L20 CPX: BiDi NPO That Halves the Fiber Plant — and the Open CPX On-Ramp

Passage L20 CPX — BiDi NPO that halves the 512-GPU fiber plant, plus the Open CPX MSA on-ramp.
Writing

AMD’s 512-GPU MI355X MLPerf: 5.75M tok/s on RoCE — and the +38% ROCm Asterisk

512-GPU MI355X on RoCE — 5.75M tok/s, plus the +38% same-silicon ROCm asterisk.
Writing

Vera Rubin NVL72’s First MLPerf: 3.7× GB300 on Qwen3-VL — and the AgentX Tokens-per-Megawatt Asterisk

First MLPerf Inference v6.1 preview — 3.7× GB300 on Qwen3-VL, plus the AgentX tok/MW asterisk.
Writing

d-Matrix Raptor Joins NVLink Fusion: 100 TB/s Cards, NVL144 Math, and the Decode-Silicon Bet

Raptor on NVLink Fusion / MGX — 100 TB/s card math, NVL144 aggregates, and the decode bet next to LPX.
Writing

Qualcomm × Amazon: Custom Inference Silicon, 1.6T Optics, and a Second ASIC Line Next to Trainium

Custom inference silicon and 1.6T optics for AWS — a second ASIC line next to Trainium.
Writing

Release Workflow Policy Tests: Catch Signing Drift Before Release Day

Mutation-tested release-workflow policy — sole signer, OIDC scope, SHA pins, and asset-list drift.
Writing

NVIDIA Groq 3 LPX: 3,431 tok/s at 100K — and the Batch-1 Caveat

3,431 tok/s at 100K on Gemma 4 31B — and the batch-1 concurrency caveat for agent fleets.
Writing

AMD Helios UALoE: Open Scale-Up on Ethernet, With the Bridge Caveats

72× MI455X, UALink-over-Ethernet, and the open scale-up caveats next to NVLink Fusion.
Writing

NVIDIA NVHBM: Moving the Memory Controller Into the Stack

Custom HBM base die for NVLink Fusion — bandwidth, power, and die-area targets, with the caveats.
Writing

OpenAI Jalapeño: Spec Sheet, InferenceX Numbers, and the Tokens-per-Megawatt Bet

InferenceX tokens-per-megawatt numbers, HBM4 package math, and the captive ASIC caveats.
Writing

AMD’s Threadripper Halo Station: Spec Sheet, HBM Math, and the DGX Station Bet It Is Making

MI350P HBM math, the path to four cards, and AMD’s deskside bet against coherent DGX Station.
Writing

K2 Horizon Ships Six Open Models — and Publishes the Reward-Hack Audit on Its Own Terminal-Bench Score

Six Apache-2.0 models, MoVA, and a published Terminal-Bench reward-hack audit from IFM.
Writing

GPT-6 Astra Is Live: Spec Sheet, Vendor Benches, and Where It Beats Fable on Cost-per-Task

Spec sheet, vendor benches, and where GPT-6 Astra beats Fable on cost-per-task.
Writing

Claude Fable 5.1 Is the New Default for Agentic Coding Fleets: Here’s the Spec Sheet

Scores, cache pricing, and how I would route coding-agent traffic after Anthropic’s Fable 5.1 / Mythos 5.1 GA.
Writing

A first look at NVIDIA Cluster Readiness Engine

What NVCRE v0.8.0 is, how certifications compose, and how to run your first category from the official docs.
Repo

ipfs-k8s-auto-backup

Experience

My background spans automotive, identity, security, communications, test infrastructure, and developer tooling. The thread is reliability: making complex systems easier to test, understand, and operate. On Design Technology I work on platforms that deliver advanced tooling to mechanical and electrical engineers designing Tesla products.

2026—now

Staff Software Engineer

Tesla · Design Technology
2023—26

Senior Software Engineer

Tesla
2021—23

Staff Software Engineer

Okta
2019—21

Senior Software Engineer in Test

Okta
2017—19

Staff Engineer → Senior Staff Engineer

Palo Alto Networks
2014—17

Intern → QA Engineer → Software Development Engineer in Test

8x8

full career chronology →

Writing

25 Oct 2026

Humming Is Already in Your vLLM Install. Here's What the Quantized Kernel Numbers Actually Say.

On October 21, 2026, Ant Group and the vLLM community published Announcing Humming: Fast, Flexible Kernels for Quantized Inference.
25 Oct 2026

vLLM and SGLang Both Caught DeepInfer on DeepSeek-V4.1-Flash. Read the Setups Before the Headlines.

A few days after DeepSeek open-sourced DeepInfer, its own engine for DeepSeek-V4.1-Flash, both big general-purpose engines answered.
24 Oct 2026

Million-Token GLM-5.3 on 16 H200s: What PCP, DCP and HiSparse Actually Buy You

The vLLM team published GLM-5.3 Optimizations, Part 2 on October 23, 2026.
24 Oct 2026

llm-d 1.0 Upgrade Notes: Pinned Engines, a DisaggregatedSet Prerequisite, and Quiet Dashboard Breaks

The llm-d maintainers published llm-d v1.0 on October 20, 2026.
23 Oct 2026

MiniMax-M3.1-Flash on vLLM: What an NVFP4 KV Cache Changes for Capacity Planning

MiniMax put MiniMax-M3.1-Flash on Hugging Face on October 23, and the vLLM team published a day-0 support post the same day.
23 Oct 2026

Dynamo 1.6 Upgrade Notes: Renamed Workers, a Mandatory Queue, and the Known Issues That Matter

NVIDIA's distributed inference platform Dynamo v1.6.0 was published at 22:31 UTC on October 22, 2026, which was 3:31 PM PT.
22 Oct 2026

Raon-1 mini Preview: 8 KiB of KV per Token, and a Commit Log Worth Reading

KRAFTON released Raon-1 mini preview on October 21 under Apache 2.0.
22 Oct 2026

DeepSeek Open-Sourced DeepInfer. The Deployment Q&A Is the Part to Read.

DeepSeek put DeepInfer on GitHub this week under Apache 2.0.
21 Oct 2026

Reading Flint's Serving Kit Before the Benchmarks: A 40B MoE That Wants One Big GPU

Recursive released Flint today, an Apache 2.0 mixture-of-experts coding model with 39.8B total parameters and about 2.5B active per token.
21 Oct 2026

The Layer Your Autotuner Never Touches: Helion Now Searches ptxas Controls

Every kernel autotuner I've used searches the same kind of thing.
20 Oct 2026

TorchTPU Is Open Source: What Running PyTorch Natively on TPUs Actually Involves

This morning I wrote about TorchNeuron and noted that TorchTPU was only a conference session with a repo on the way.
20 Oct 2026

TorchNeuron Public Beta: What Changes When Trainium Becomes a Normal PyTorch Device

AWS opened up TorchNeuron today at PyTorch Conference North America in San Jose.
19 Oct 2026

Meta's Prometheus Numbers: What 64-of-72 Sparing and a 54K GPU-Hour MTBI Mean for NVL72 Operators

Meta published Inside Prometheus on October 19, 2026.
19 Oct 2026

vLLM 0.32 Upgrade Notes: The Defaults That Changed Under You

vLLM v0.32.0 was published at 06:50 UTC on October 18, 2026, which was 11:50 PM PT on October 17.
19 Oct 2026

SGLang 0.5.22 Upgrade Notes: Flags That Changed Meaning

SGLang v0.5.22 was published at 02:55 UTC on October 19, 2026, which was 7:55 PM PT on October 18.
18 Oct 2026

Dory1.0-4B: What a Looped 4B Model Actually Costs Per Token

NVIDIA put Nemotron-Labs Dory1.0-4B on Hugging Face on October 16 under the OpenMDW-1.1 license.
17 Oct 2026

Granite Switch 4.1: Three Models' Worth of Jobs on One KV Cache

IBM shipped Granite Switch 4.1 this week in 8B and 30B sizes, Apache 2.0.
17 Oct 2026

vLLM's Two New KV Transfer Modes, and Why You Pick One

Two posts on the vLLM blog this week change how KV cache moves between prefill and decode workers in a disaggregated deployment.
16 Oct 2026

Ling-3.1-flash: What 63 Linear-Attention Layers Do to the KV Budget

inclusionAI, the Ant Group team behind the Ling, Ring and Ming model families, put the weights for Ling-3.1-flash on Hugging Face this week under the MIT license.
16 Oct 2026

Nemotron 3.5 Super VL and Bolt: Hybrid MoE Served From Rack to 24 GB

NVIDIA put Nemotron 3.5 Super VL and Bolt on Hugging Face on October 15, the same window RTX Spark PCs started shipping.
15 Oct 2026

Muse Spark Open: Latent MoE, Hybrid Attention, and a 1M Context That Actually Fits

Meta put the Muse Spark Open weights on Hugging Face today, October 15, the same day vLLM shipped day-0 support and NVIDIA published GB300 NVL72 serving numbers.
15 Oct 2026

Step 5 Preview Weights Are Out: 600B, 92 Layers, and a 74 GB Million-Token KV Cache

StepFun put the Step 5 Preview weights on Hugging Face last night, the same day SGLang published its day-0 support post.
14 Oct 2026

Intel GPU D270: 480 GB on a PCIe Card, and What That Memory Is Actually For

Intel used the OCP Global Summit this week to put final numbers and a name on Crescent Island.
14 Oct 2026

ReplaySSM: Hybrid Models Were Paying an HBM Tax on Every Token

A lot of the models people serve this year are hybrids.
13 Oct 2026

Gemini 4 Argon: One 1M-Token Budget, a Continuation Protocol, and a Scorecard That Doesn't Win Everywhere

Google shipped Gemini 4 Argon today, October 13, as the first model in the Gemini 4 family.
13 Oct 2026

Maia 300: Microsoft’s Inference ASIC Puts 288 GB of HBM4 and a 12,288-Chip Ethernet Superpod on the Table

Microsoft used the same stage to put numbers on Azure Maia 300, the third generation of its in-house inference accelerator and the first Maia built as a multi-chiplet part.
12 Oct 2026

OSPA: Meta, Microsoft and Alibaba Want Every AI Fabric Switch to Boot the Same Way

On October 8, OCP posted Open Switch Platform Architecture, a draft under OCP review initiated by Meta, Microsoft, and Alibaba: a vendor-neutral switch platform layer so AI fabric SKUs can boot, inventory, and flash the same way.
11 Oct 2026

Mellum2.1: A 12B MoE Coding Agent Model, and the GGUF Builds That Make It Runnable Tonight

On October 8, JetBrains released Mellum2.1, the post-trained successor to the Mellum2 Thinking checkpoint they open-sourced in June.
11 Oct 2026

Liquid Open d1: Zero Output Tokens, Named Jetson Latencies, and What a Decision Model Actually Is

On October 7, Liquid AI open-weighted two models in its d1 family: d1-3B and experimental d1-omni-600M.
10 Oct 2026

Vera Rubin Day Zero: What SGLang Changed in the Kernels, and What vLLM Measured on AgentX

On October 9, two open serving stacks published the first useful Vera Rubin numbers that are not a keynote slide.
10 Oct 2026

RTX Spark N1X: The Windows Numbers That Matter Are TDP, Memory Ceiling, and Prefill

On October 7, NVIDIA and Microsoft opened Windows pre-orders for RTX Spark laptops (availability October 16; compact desktops in November) and previewed DGX Station for Windows for Q4.
9 Oct 2026

Scality AI Inference Factory: Object Storage Over RDMA as the Shared KV Cache

Scality launched AI Inference Factory on October 8: an open-code stack for serving open-weight models on infrastructure you own, with Scality ADI as a shared key-value (KV) cache behind the GPUs.
9 Oct 2026

Upscale Token Fabric: SkyFabriX at 115.2 Tbps Meets Spectrum-X Scale-Out Under One OS

A day after Arista’s Etherlink SU-144, Upscale published Token Fabric: SkyFabriX at 115.2 Tbps, Spectrum-X scale-out, and one OS for both, with general availability aimed at early 2027.
8 Oct 2026

Claude Haiku 5.5: The Price Cut Is Real, but Read the 100K Tier, the Tokenizer, and the Output Bill

Anthropic released Claude Haiku 5.5 on October 7: the small model next to Sonnet 5.5 and Opus 5.5 for high-volume work, with a real price cut that still needs the 100K tier, the tokenizer, and the output bill read first.
7 Oct 2026

Arista Etherlink SU-144: Ethernet Moves Inside the Scale-Up Domain, and the Rack Becomes the Unit

Arista’s Etherlink SU-144 puts 144 accelerators in a single-hop Ethernet scale-up domain inside one rack, and up to 1,024 across racks, on OCP ESUN 1.0 — a partner-built reference design, not a SKU.
7 Oct 2026

openTPU: An Agent-Built LLM Accelerator on a Kintex-7 PCIe Card, and What Its Numbers Teach

openTPU is a complete Apache 2.0 LLM inference stack on a Kintex-7 PCIe card — SystemVerilog RTL, ISA, bit-exact simulator, and host tools — with self-reported decode numbers that put memory-bound inference on one line.
6 Oct 2026

AICR v1.0: NVIDIA Freezes the Contract for GPU Kubernetes Recipes

NVIDIA’s AICR v1.0.0 freezes the aicr CLI, aicrd REST API, Go SDK, and bundle schemas — a cluster configuration generator for GPU Kubernetes recipes, not a distribution or provisioner.
6 Oct 2026

Mistral Large 4: 1.05T Parameters, 49B Active, and What It Takes to Serve It

Mistral’s October 6 Large 4 is a 1.05T / 49B-active open-weight multimodal MoE with a 1M-token context — public-preview API today, weights and a custom license still October 27.
5 Oct 2026

Reflection’s Beam: A 501B Open-Weight MoE, and the GB300 Cluster Numbers Behind It

Reflection’s October 5 Beam is a 501B / 23B-active sparse MoE for coding, reasoning and agentic work — self-reported GB300 cluster numbers, with weights and the tech report still later this month.
5 Oct 2026

vLLM 0.31 Fast Restart: A GPU Weight-Cache Daemon and CRIU Engine Snapshots

vLLM v0.31.0 Fast restart: a GPU weight-cache daemon and experimental CRIU engine snapshots — project claims of minutes-to-seconds restarts, with an open fingerprint bug and Kubernetes caveats.
4 Oct 2026

TLX JFA: Meta’s Blackwell Attention Kernel — ~13% Fwd / ~50% Bwd vs FA4 on Jagged

Meta’s Oct 1 TLX Jagged Flash Attention is a Blackwell B200 kernel for packed jagged / broadcast-Q — Meta’s bf16 benches claim ~+13% forward / ~+50% backward vs May 2026 FA4 on their shapes, not an independent rematch.
3 Oct 2026

Dist-MoE: Meta’s Blackwell SM100 MoE Kernels — BF16 / MXFP8 Train, NVFP4 Infer

Meta’s Oct 3 dist-moe v0.1.0 is CuTe DSL distributed MoE kernels for Blackwell SM100 — BF16 / MXFP8 E4M3 train+infer, NVFP4 infer-only — Meta’s docs, not a published throughput rematch.
3 Oct 2026

Kolibri-1: 78.1B / 3.46B Open MoE — Hybrid SWA, 1M Context Claim, ~78 GB FP8

Aleph Alpha’s Oct 3 Kolibri-1 is a 78.1B / 3.46B open MoE with hybrid 4:1 SWA(512):full, native 256k validated to 1M, and ~78 GB FP8 — vendor benches and a named B200 train topology, not an independent rematch.
2 Oct 2026

Credo Weaver: HBM4-Class Edge Bandwidth at ~25× Capacity — OmniConnect VSR, No CoWoS

Credo’s Oct 2 OmniConnect / Weaver post claims HBM4-class die-edge bandwidth (6.4 vs 6.1 TB/s) at ~25× capacity (4.8 TB vs 192 GB) on organic, no CoWoS — vendor architecture claims, not an independent bring-up.
2 Oct 2026

Lightning V2 on Tenstorrent: 4× TTS Cost Cut vs L40S — BFP8, NoC, Perceptual Fragility

Tenstorrent and Smallest.ai put Lightning V2 on Blackhole — vendor L40S bake-off with ~3–4× modeled TTS CapEx, BFP8/LoFi co-design, and a paper on perceptual fragility — not a blanket NVIDIA cost claim.
1 Oct 2026

Olmo-core 3: Open MoE Training Stack Hits 1.2T on B300 — DDP, Rowwise EP, MXFP8

Ai2 released Olmo-core 3 — open DDP+EP+PP MoE trainer on NVIDIA B300 through 1.20T / 58.4B active, 19.4K → 52K tok/s/GPU at 47B vs the old FSDP path — systems measurements, not next-Olmo weights.
1 Oct 2026

Everspin Hooks PERSYST MRAM to CXL — 1GB Persistent Demo Tier Between DRAM and NAND

Everspin demonstrated CXL-connected PERSYST 1GB MRAM at SNIA SDC 2026 — memory-semantic persistence between DRAM and NAND, vendor 100× writes vs NAND, working demo not a shipping SKU.
30 Sep 2026

CoreWeave Puts Vera Rubin NVL72 Into Production With Cognition — 4.8× SWE-2 Tokens, Not Another Shipping BOM

CoreWeave puts NVIDIA Vera Rubin NVL72 into limited availability with Cognition as first production customer — 4.8× SWE-2 and 3.8× RL token throughput vs GB200 (customer claims) — neocloud production, not another shipping BOM.
30 Sep 2026

Claude Sonnet 5.5: Everyday Coding Ladder Beside Opus 5.5 — Cost Per Task, Not Another Flagship Rematch

Claude Sonnet 5.5: same $2/$10 list as Sonnet 5, vendor fewer-tokens plus 30%+ faster generation, near-Opus on several Max benches — everyday coding ladder beside live Opus 5.5, not another flagship rematch.
29 Sep 2026

Naive-N0.5-Flash: 309B / 15.5B MoE with Zero Full-Attention Layers — Native 1M via SWA+DSA

Naive-N0.5-Flash: open-weight 309B / 15.5B MoE with zero full-attention layers, native 1M via SWA+DSA, MIT, announced $0.10/$0.40/$0.01, NaiveRT peak you should re-time — decode-cost architecture, not an AA rematch.
29 Sep 2026

NVIDIA Open Agent Safety: OpenShell 0.1.0 + BlueField-4 Sentry — Runtime Policy, Milliseconds Quarantine

NVIDIA Open Agent Safety: Apache-2.0 OpenShell 0.1.0 plus optional BlueField-4 Sentry — YAML → OPA/Rego, OCSF, milliseconds quarantine as a vendor claim, no DPU required — runtime policy for agent fleets, not an AA rematch.
28 Sep 2026

JEV-27B: Open System 1 Decisions on a Frozen Qwen3.8-27B — KL 0.017, One B200

AutoTrust JEV-27B: open System 1 decisions on a frozen Qwen3.8-27B — KL ≈0.017, six-bench mean 84.07, 137 ms on one B200 — vendor self-comparison vs hosted TypeSafe Jev 1.13, not a third-party bake-off.
28 Sep 2026

Holo4: Computer-Use Agents With Named $/Task — 27B Dense, 35B-A3B MoE, Open Trajectories

H Company Holo4: 27B dense + 35B-A3B MoE computer-use VLMs with named OSWorld / AutomationBench $/task and open trajectories — no AA Index row, and a CC BY-NC vs Apache-2.0 license split you should write into the routing ticket.
27 Sep 2026

AliceAI-Foundation-80B: From-Scratch 3B Activate, KDA:GA 3:1 — and Two Open Russian Factual Benches

Yandex AliceAI-Foundation-80B-A3B-Base: from-scratch 80B/3B hybrid MoE, KDA:GA 3:1, Apache 2.0, WikiWebFacts / HardMultiQA — no AA Index row yet, and a base you still owe SFT/RL before you compare it to Flash/Pro chat endpoints.
27 Sep 2026

DeepSeek-V4.1-Flash: 890 Bytes of KV per Token — and Why V4-Pro Is Being Retired

DeepSeek-V4.1-Flash: 890 bytes KV/token, 8B prefill / 16B decode MoE, MIT weights, V4-Pro traffic steered to Flash rates — AA Index 39, with a Terminal-Bench 4.0 gap vs Opus/Sol on DeepSeek’s own table.
26 Sep 2026

MiMo-V2.6-Pro: Open Weights at AA 46 — and the Terminal-Bench 4.0 Asterisk

Xiaomi MiMo-V2.6-Pro / Flash: MIT open weights, AA Intelligence Index 46, $0.435/$0.87 list (Flash cheaper), same-day foil to Grok 4.7 — with a Terminal-Bench 4.0 gap vs Opus/Astra on Xiaomi’s own table.
26 Sep 2026

Lessengers’ 3.2T Glass NPO: TGVs, Lens-Free DOW, and the Millimeters-from-ASIC Bet

Lessengers’ 3.2T glass-substrate NPO: TGVs, lens-free DOW polymer waveguides, millimeters-from-ASIC placement, immersion-native claim, aimed at 51.2T/102.4T switch generations — vendor claims until your BER and yield tickets land.
25 Sep 2026

PicoJool’s 200G GaAs VCSEL: >37 GHz, Three Paths to 1.6T, and the Unconstrained-Supply Bet

PicoJool’s 200G GaAs VCSEL / microVCSEL bet: >37 GHz device claims, three 1.6T recipes, WIN foundry, Series A funding qualification → early-2027 volume — vendor claims until your BER tickets land.
25 Sep 2026

Supermicro Is Shipping Vera Rubin NVL72: 20.7 TB HBM4 and a Cold-Plate-to-Tower BOM You Can Order

Supermicro says Vera Rubin NVL72 racks are shipping now — 72 Rubin GPUs, 20.7 TB HBM4, and a cold-plate-to-tower DLC BOM you can order as a 16-rack Scalable Unit.
24 Sep 2026

EdgeCortix RAIDEN: 3.36 PFLOPS FP4 on Four Dies — Thick-Edge Physical AI as One Chiplet Stack

EdgeCortix RAIDEN claims up to 3.36 PFLOPS FP4, 256 GB, and 1.54 TB/s die-to-die on a four-die Physical AI chiplet stack shipping samples in early 2027.
24 Sep 2026

WhiteFiber Continuum: 136 Tbps Across 83 km — Scale-Across When One Campus Hits the Power Wall

WhiteFiber Continuum claims 136 Tbps / 0.9 ms RTT across 83 km of metro fiber so two campuses can act as one GPU supercluster when one site hits the power wall.
23 Sep 2026

Claude Opus 5.5: Fable-Class Work at Opus Prices — and the Safeguard Asterisk

Claude Opus 5.5 — Fable-class work at $4/$20, and the fallback-safeguard asterisk.
22 Sep 2026

GPT-6 Sol and Luna: Half the Price, Not Always a Higher Ceiling

GPT-6 Sol and Luna — half the GPT-5.6 promo price, cache-steerable effort, not always a higher ceiling.
22 Sep 2026

T-Head Zhenwu V900: 216 GB, 1,200 GB/s Chip-to-Chip, and a 500,000-Card Cluster Story

Zhenwu V900 — 216 GB, 1,200 GB/s chip-to-chip, and a 500,000-card cluster story.
21 Sep 2026

Axelera Europa: 629 TOPS at 45 W — Shipping Edge 232p Math With a Dell/Supermicro Asterisk

Europa 629 TOPS at 45 W — shipping Edge 232p math with a Dell/Supermicro asterisk.
21 Sep 2026

Lightbits Inferra: KV Cache Prefetch Beyond HBM — Named TTFT Tables on L40S Math

Inferra KV-cache prefetch beyond HBM — named TTFT tables on L40S math vs recompute.
20 Sep 2026

Astera Leo X: Fabric-Attached DDR for KV Cache — 62% Faster Warm TTFT on H200 Math

Leo X fabric-attached DDR for KV cache — 62% faster warm TTFT on H200 math vs CPU-DRAM offload.
20 Sep 2026

NewPhotonics NPG102 + Tower PH18DA: Laser-Integrated PICs Shipping for 800G–1.6T — and the CPX 6.4T On-Ramp

NPG102 + Tower PH18DA — laser-integrated PICs shipping for 800G–1.6T, plus the CPX 6.4T on-ramp.
19 Sep 2026

Cornelis Active Compute Fabric + Delos Nonstop AI: Open Scale-Up Math Beyond NVLink

Cornelis ACF + Delos Nonstop AI — 800G fabric sampling and >30 Tbps I/O dies beyond NVLink.
19 Sep 2026

AMD EPYC Venice vs Nvidia Vera: 2.24× SPECrate, 1.6 TB/s Stream — and the Compiler Asterisk

EPYC Venice vs Vera — 2.24× SPECrate and 1.6 TB/s Stream, plus the compiler asterisk.
18 Sep 2026

Meta MTIA 450 / Arke: 18.4 TB/s HBM, FlashAttention in Silicon — and the Sep 1 Bring-Up Asterisk

MTIA 450 / Arke — 18.4 TB/s HBM and FlashAttention in silicon, plus the Sep 1 bring-up asterisk.
18 Sep 2026

Lightmatter Passage L20 CPX: BiDi NPO That Halves the Fiber Plant — and the Open CPX On-Ramp

Passage L20 CPX — BiDi NPO that halves the 512-GPU fiber plant, plus the Open CPX MSA on-ramp.
17 Sep 2026

AMD’s 512-GPU MI355X MLPerf: 5.75M tok/s on RoCE — and the +38% ROCm Asterisk

512-GPU MI355X on RoCE — 5.75M tok/s, plus the +38% same-silicon ROCm asterisk.
16 Sep 2026

Vera Rubin NVL72’s First MLPerf: 3.7× GB300 on Qwen3-VL — and the AgentX Tokens-per-Megawatt Asterisk

Vera Rubin NVL72’s first MLPerf numbers, and why AgentX tok/MW is a separate column.
10 Sep 2026

d-Matrix Raptor Joins NVLink Fusion: 100 TB/s Cards, NVL144 Math, and the Decode-Silicon Bet

d-Matrix Raptor joins NVLink Fusion: decode silicon on the same MGX rack family as Vera Rubin.
8 Sep 2026

Qualcomm × Amazon: Custom Inference Silicon, 1.6T Optics, and a Second ASIC Line Next to Trainium

Qualcomm × Amazon: inference ASIC + 1.6T optics, with the merchant-roadmap caveats.
8 Sep 2026

Release Workflow Policy Tests: Catch Signing Drift Before Release Day

Catch signing drift in CI before release day — policy tests for attested binary pipelines.
6 Sep 2026

NVIDIA Groq 3 LPX: 3,431 tok/s at 100K — and the Batch-1 Caveat

NVIDIA Groq 3 LPX: long-context interactivity numbers, with the concurrency ceiling.
6 Sep 2026

AMD Helios UALoE: Open Scale-Up on Ethernet, With the Bridge Caveats

AMD Helios UALoE: open scale-up on Ethernet, with the bridge caveats.
5 Sep 2026

NVIDIA NVHBM: Moving the Memory Controller Into the Stack

NVIDIA NVHBM: moving the memory controller into the HBM stack for Fusion partners.
5 Sep 2026

OpenAI Jalapeño: Spec Sheet, InferenceX Numbers, and the Tokens-per-Megawatt Bet

OpenAI Jalapeño at Hot Chips: InferenceX STP vs Blackwell, and the tokens-per-megawatt bet.
4 Sep 2026

AMD’s Threadripper Halo Station: Spec Sheet, HBM Math, and the DGX Station Bet It Is Making

AMD Threadripper Halo Station: HBM capacity, discrete PCIe vs coherent DGX Station.
4 Sep 2026

K2 Horizon Ships Six Open Models — and Publishes the Reward-Hack Audit on Its Own Terminal-Bench Score

IFM’s K2 Horizon open fleet: sizes, MoVA, and the Terminal-Bench contamination delta.
3 Sep 2026

GPT-6 Astra Is Live: Spec Sheet, Vendor Benches, and Where It Beats Fable on Cost-per-Task

OpenAI’s GPT-6 Astra launch: benches, cache math, and how I would route it against Fable 5.1.
3 Sep 2026

Claude Fable 5.1 Is the New Default for Agentic Coding Fleets: Here’s the Spec Sheet

A practitioner reading of Fable 5.1: Terminal-Bench numbers, cache-read pricing, and a migration plan.
2 Sep 2026

A first look at NVIDIA Cluster Readiness Engine

A practitioner reading of NVCRE v0.8.0 from the official docs: catalog, APIs, and first certification.

all writing →

Connect

Find me on LinkedIn and GitHub.

Anpoo Sivanadi

Selected work

Experience

Staff Software Engineer

Senior Software Engineer

Staff Software Engineer

Senior Software Engineer in Test

Staff Engineer → Senior Staff Engineer

Intern → QA Engineer → Software Development Engineer in Test

Writing