Meta put the Muse Spark Open weights on Hugging Face today, October 15, the same day vLLM shipped day-0 support and NVIDIA published GB300 NVL72 serving numbers. The Meta research post calls it the most capable US open-weight model to date: 816B total parameters, 42B active per token, natively multimodal (text, image, video, audio in; text out), and a 1,048,576-token context. The useful story for anyone who runs clusters is not the headline. It is how Meta compressed the experts and cut the KV bill so a million-token agent session is not a fantasy on paper.

This morning I wrote about Step 5 Preview, which also advertises 1M context and pays for it with sparse GQA storage. Muse Spark Open takes a different bet: shrink the experts with a latent pooler, and keep full-sequence KV on only a quarter of the layers. Same problem, opposite levers.

What the card and the blogs agree on

The Hugging Face model card and the Hugging Face architecture post line up with vLLM's day-0 write-up:

Item Muse Spark Open
Parameters 816B total, ~42B active
Layers 62, every one MoE
Hidden size 8,192
Experts 256 per layer, top-8 routed
Expert width latent: pooler 8192→4096, unpooler back (shared per layer)
Attention 128 Q heads / 16 KV heads, head dim 64, QK-norm, output gate + output norm
Attention pattern 3 sliding-window (2,048 tokens) + 1 full, cycle anchored so the last layer is full → 46 SWA, 16 full
Full-attention position NoPE on the 16 full layers (layer_rope_theta: 0)
Context 1,048,576 tokens
Vision Perception Encoder ~1.8–1.9B (50 layers, 14×14 patches)
Audio ~118M params; 24 kHz waveform → one token per 80 ms
Released precision BF16 (~1.5–1.6 TB safetensors) and NVFP4 (~540 GB weight footprint per Baseten)

vLLM's weight breakdown puts about 799B in the 256 experts per layer, 13.5B in attention, 4.2B in the latent projections, and 3.3B in the untied embedding and output matrices, with the vision tower separate. That is why every serving recipe is really an expert-parallelism recipe.

Latent MoE: why the experts are half-width

A normal MoE at hidden size 8192 would run each expert at full width. Muse Spark Open inserts a shared pooler and unpooler around the expert block. The router still sees the full 8192-dimensional token. The experts only ever see 4096. Baseten's launch post spells out the point: that halves each expert's parameters, cuts roughly 800B parameters versus an uncompressed design of the same shape, frees HBM for KV, and shrinks the all-to-all traffic when experts live on different GPUs.

For a sparse model that activates eight of 256 experts every token, expert traffic is the interconnect tax. Cutting the expert activation width in half is the kind of change you feel on a multi-node NVLink domain before you feel it on a single-GPU microbenchmark.

Hybrid attention and the KV arithmetic

Grouped-query attention already keeps the cache smaller than multi-head. My arithmetic from the published layout, matching the HF blog's all-full figure: each token stores K and V for 16 heads × 64 dimensions × 62 layers, which is 126,976 values. In BF16 that is about 248 KB per token, or about 266 GB for a full 1M-token session if every layer kept the whole history.

They do not. Forty-six layers only keep a 2,048-token sliding window. Sixteen layers keep the full sequence, and those full layers drop rotary position encoding. At 1M tokens the hybrid cache is just over a quarter of the all-full number: about 68 GB for the sixteen full layers, plus a fixed ~386 MB for the forty-six windows. That is my arithmetic; vLLM says the same "just over ¼" ratio. Baseten measures the working-set case directly: at 128K context, hybrid cuts KV from 31 GiB to 8.4 GiB, about 3.7×.

Compare that to this morning's Step 5 Preview number. Step 5's sparse full-attention layers still store about 70.7 KB per token across 23 layers, so a 1M session is about 74 GB even though sparse attention only reads a fraction of it each step. Muse Spark Open's full-length bill lands in the same ballpark, but most layers never grow with context at all. You are not paying an indexer scan that scales with every historical block.

What it takes to run it

The model card is blunt on VRAM: at least 700 GB for NVFP4 and 2 TB for BF16. The HF blog's transformers path runs BF16 across four nodes of eight H200s with tensor parallelism 32, and NVFP4 on a single B200-class node. The SGLang cookbook in the same post starts NVFP4 on four GB300 GPUs with FlashInfer TRT-LLM MoE and matching TP/EP sizes. Baseten says the NVFP4 checkpoint brings weights from roughly 1.6 TB down to about 540 GB, which is what lets a single 4×GB300 node hold the model without spilling expert traffic across a second chassis.

NVIDIA's GB300 NVL72 post is the first rack-scale number that is useful. On agentic workloads with speculative decode at acceptance length 6, they report up to 50K tokens per second per GPU and up to 600 tokens per second per user on the 72-GPU NVLink domain (130 TB/s). Those are vendor measurements on their own stack; treat them as an upper envelope, not a purchase order. Meta also ships custom Helion kernels in SGLang and claims 1.03–3.95× kernel-level speedups on NVIDIA and AMD, with a 3% end-to-end gain on MI350.

If you are not buying a rack, the desktop path is concrete: Meta says two Apple M5 Ultra Mac Studios with MLX can hold the model, and on a single M3 Ultra (512 GB) llama.cpp's Q2_K quant uses about 304 GB at around 18 tokens/s while MLX's oQ3e uses about 344 GB at roughly 21 tokens/s. Those are not production agent rates. They prove the checkpoint is not locked to one vendor's fabric. vLLM also trained a DSpark draft model that proposes up to seven tokens per pass on the same sliding-window pattern; AMD shipped MXFP4 for Instinct, and TPU megakernels with DSpark are claimed above 650 tokens/s on four TPU v7 chips.

License, before you put it on an endpoint

Weights ship under the Muse Model License. Products that use it or derivatives must show "Built with Muse Spark Open." If you offer it as model-as-a-service and you and your affiliates clear US$20 million in aggregate gross revenue over any consecutive twelve months, you have to notify Meta and sign a separate agreement before including it in that MaaS business. Internal use and embedding it as a feature of your own non-MaaS product are carved out. Same shape as other large open releases this month: free to download, not free to run as a public API at scale without talking to the lab.

What I'd measure first

  • Decode tokens/s per sequence at 32K, 128K, and 1M on the same NVFP4 node, with and without DSpark, to see where hybrid attention stops hiding the full-layer KV.
  • Expert-parallel all-to-all bytes per token on 4×GB300 versus a two-node BF16 H200 mesh, to price the latent-MoE interconnect win.
  • Concurrent 128K agent sessions per node after NVFP4, using Baseten's 8.4 GiB KV figure as the planning floor and adding activation and graph overhead yourself.

Where I land

Muse Spark Open is the open-weight release I would actually capacity-plan around today. Latent MoE makes the 816B parameter count less scary on the wire, hybrid attention makes 1M context a memory schedule instead of a wish, and the day-0 SGLang, vLLM, TensorRT-LLM, MLX, and llama.cpp paths mean you can verify both claims on hardware you already have. It is a different answer than Step 5 Preview's sparse GQA: fewer layers that grow with context, smaller experts, still a real KV bill at a million tokens. Plan for the sixteen full layers, not the marketing total.

Sources: Meta research blog, Hugging Face model card, Hugging Face Muse Spark Open blog, vLLM day-0 blog, NVIDIA GB300 NVL72 blog, Baseten launch post. Model, benchmark, serving, and architecture figures are from Meta, Hugging Face, vLLM, NVIDIA, and Baseten. Per-token KV sizes, the 1M hybrid cache total, the fixed SWA overhead, and the 128K cross-check against Baseten's GiB figures are my own arithmetic from the published layout.

Related reading on this site: Step 5 Preview open weights, ReplaySSM hybrid decode, Intel GPU D270.

Anpoo Sivanadi, Staff Software Engineer