StepFun put the Step 5 Preview weights on Hugging Face last night, the same day SGLang published its day-0 support post. StepFun announced the model in September with API access and promised open weights for October 15. They kept the date, so you can now download it, read its config.json, and work out what it costs to run. There are three checkpoints: FP8 (about 650 GB of safetensors), BF16 (about 1.21 TB), and NVFP4 (about 394 GB), going by the file listings on Hugging Face.
The headline numbers are 600B total parameters, 27B active per token, and a 1M-token context. Those tell you less than the attention layout does. StepFun chose plain grouped-query attention over MLA, then made it sparse. That choice decides how much memory a long agent session takes and how many of them a node can hold, so most of this post is about it.
What the config says
StepFun's architecture write-up and the published config agree on the shape:
| Item | Step 5 Preview |
|---|---|
| Parameters | 600B total, 27B active |
| Layers | 92 (plus 3 MTP layers for speculative decoding) |
| Hidden size | 4,096 |
| Experts | 8 of 352 routed, plus 1 shared, in layers 3–90 |
| Attention pattern | 3 gated sliding-window layers (512 tokens) per 1 sparse full-attention layer: 69 SWA, 23 sparse |
| Heads | 64 query heads, 4 KV groups, head dim 192 |
| Sparse indexer | one 256-dim key per 8-token block; top 512 blocks (k = 4,096 tokens) |
| Context | 1,048,576 tokens |
| Released precision | FP8 E4M3 routed experts with 128×128 block scales; last three MoE layers in BF16 |
The model is narrow and deep on purpose. StepFun argues that agents spend most of their tokens in prefill (tool output, files, logs) and get little room for explicit chain-of-thought, so the multi-hop reasoning has to happen across layers. They put the depth on par with much larger open models while keeping total and active parameters well below them. Depth wasn't free: the blog describes keeping hidden states and RMSNorm scales in FP32 because BF16 let a few massive activations blow up, plus a per-layer gradient clamp that clips only 0.00004% of gradient values. That's why the config carries fp32_residual_connection: true. For inference it means one more thing to check in any custom kernel path.
Why GQA, and what it costs
The blog is direct about the trade. StepFun picked GQA over MQA and MLA for long-context accuracy, which it says it validated in experiments, and because GQA parallelizes well across GPUs for fast inference. The price is a much larger KV cache than the latent-compressed designs.
Here's my own arithmetic from the config. Each full-attention layer stores K and V for 4 groups at 192 dimensions, which is 3,072 bytes per token in BF16. Across 23 sparse layers that's about 70.7 KB per token. A 128K-token session holds about 9.3 GB; a full 1M-token session holds about 74 GB. The 69 sliding-window layers add a fixed ~109 MB per sequence no matter how long it runs, and the indexer's compressed keys add roughly 2% on top if they're stored at two bytes per value (my assumption; the config gives the dimension, not the storage type).
For comparison, DeepSeek reported about 890 bytes of KV per token for V4.1 Flash. Step 5 Preview's full-attention cache is roughly 80 times that. Both models advertise 1M tokens. They are not equally cheap to hold at 1M tokens.
What sparse attention does and doesn't save
Sparse GQA cuts the reads, not the storage. In each sparse layer, a small indexer scores 8-token blocks and each KV group picks up to 512 historical blocks plus the current one. Every token still has to stay in memory because any block might be picked on the next step.
The read savings are large. My estimate for a decode step at 1M context: the selected blocks come to about 290 MB across the 23 sparse layers, and the sliding window layers about 109 MB. Dense attention over the same cache would read about 74 GB. The piece that grows with context is the indexer itself, which has to score every block. At 1M tokens that's about 131K blocks per layer, or roughly 0.8 to 1.5 GB per step depending on whether the index keys sit at one or two bytes. At that length the indexer scan is the largest attention read in the step, bigger than the attention it's selecting for.
That matches what StepFun says it found: in some DSA-style implementations the indexer can cost more than the sparse attention kernel at very long contexts. Their fix was to merge 8 tokens into one index entry, which cuts indexer and top-k cost by 8×, and to raise k to 4,096 tokens to recover coverage. Their reported token coverage is 78.06% against 79.09% for a per-token, k = 2,048 reference. Without the block merge, my numbers say the indexer read at 1M would be eight times larger.
Fitting it on a node
The model card lists FP8 on one 4-GPU B300 node or one 8-GPU H200 node, and BF16 on 8 B300s or two H200 nodes. The SGLang cookbook notes that FP8 does not fit on four H200s, which tracks: 650 GB of weights against 564 GB of HBM.
The vLLM recipe lists B300 at 268 GB per GPU, so four of them give 1,072 GB. At the recipe's --gpu-memory-utilization 0.85 and 650 GB of FP8 weights, my estimate leaves about 260 GB for KV, activations, and graphs. At 70.7 KB per token in BF16, that's roughly 3.7 million cached tokens across the node before you count runtime overhead. Three full 1M-token sessions, or a couple dozen at 128K. The same recipe serves the NVFP4 checkpoint on four B300s with --max-model-len 320000, and doesn't explain the cap.
This is why the cookbook spends so much space on KV placement. SGLang supports FP8 KV for the full-attention layers (it warns this is lossy and should be re-verified), HiCache to push evicted prefixes into host memory or an L3 store like Mooncake, and prefill/decode disaggregation including the MTP draft cache. StepFun's own RL serving stack went further: the tech blog says expert parallelism, CPU KV offload, MXFP4 weight storage, and FP8 KV quantization-aware training together gave 8 to 12 times more effective KV capacity. The public recipes default to BF16 KV, so you don't get that out of the box.
The serving numbers so far
SGLang's measurements on four GB300 GPUs, all with FP8 experts, BF16 KV, and MTP at an acceptance length of 2.85:
- About 299 tokens/s for a single request with the FlashInfer TRT-LLM MoE backend.
- About 740 output tokens/s per GPU at 64 concurrent requests with MegaMoE. That's about 2,960 tokens/s for the node, or around 46 per request (my arithmetic).
- Prefill of one 128K prompt at about 11,440 tokens/s per GPU, which puts time to first token near 2.9 seconds on four GPUs (also my arithmetic).
The decode runs used about 8K input tokens and no prefix-cache hits. That's a fair short-context benchmark, but it says nothing about the case this model is built for, where each decode step pays the indexer scan over hundreds of thousands of tokens. Nobody has published decode throughput at 256K or 1M yet. Practical setup notes: the first launch compiles FlashInfer modules for about 15 minutes on B300, the TRT-LLM MoE path needs FlashInfer patches that come baked into the preview Docker image (custom builds have to apply them), SGLang says Step 5 support isn't in a tagged release yet, and vLLM's recipe also runs from a dedicated preview image.
Quality, briefly
StepFun's launch page puts it at 44 on the Artificial Analysis Intelligence Index. The scorecard is mixed, and it compares Step 5 at its High setting against other models at Max. The long-context result is the strongest: 88.3% on AA-LCR v1.1, level with Kimi K3 at 88.7% and ahead of every closed model in the table. Coding is closer to the pack. DeepSWE v1.1 is 67.7% against 74.1% for GPT-6 Astra, and Terminal-Bench v4 is 33.3% against 41.9% for GLM-5.3. If you're choosing an open model for long-document or long-trajectory work, the AA-LCR number is the one that justifies the KV bill.
License, for anyone serving it
The weights are under the StepFun Model License 1.0, which is permissive with conditions that matter to operators. Offering it as a model-as-a-service requires a separate commercial license once you and your affiliates exceed US$20 million in revenue over a twelve-month period. Device makers shipping it as a feature of their hardware need one too. Internal use that doesn't expose the model or its outputs to third parties is exempt. Read it before you put it behind a public endpoint.
What I'd check before deploying it
- Decode tokens/s per sequence at 128K, 512K, and 1M, with and without FP8 KV, to see where the indexer scan starts to dominate.
- Accuracy on long-context tasks with FP8 KV versus BF16, since SGLang itself flags it as lossy.
- Prefix-cache hit rates through the cache-aware router on real agent traffic, which SGLang says is being prepared for a formal release.
- Why NVFP4 is capped at 320K in the vLLM recipe, and whether that's a kernel limit or a memory choice.
Where I land
Step 5 Preview is the most transparent large open release in a while: config, kernels upstreamed into SGLang, serving recipes for two frameworks, and an architecture write-up that explains its trade-offs. The trade-off that matters most for a cluster is that GQA plus 1M context means about 74 GB of KV per full-length session in BF16. Sparse attention keeps the per-step reads manageable, but it doesn't shrink what you have to hold. Plan capacity around sessions resident, not tokens per second.
Sources: Step 5 Preview model card, config.json, StepFun: Inside Step 5 Preview, StepFun launch page and benchmarks, SGLang day-0 blog, SGLang cookbook, vLLM recipe, StepFun Model License 1.0, Fresh Weights on DeepSeek V4.1 Flash. Model, benchmark, and serving figures are from StepFun, SGLang, and vLLM. KV sizes, read volumes, memory headroom, and the per-request and TTFT figures are my own arithmetic from the published config and measurements.
Related reading on this site: DeepSeek-V4.1-Flash, ReplaySSM hybrid decode, Intel GPU D270.
Anpoo Sivanadi, Staff Software Engineer