Naive-N0.5-Flash: 309B / 15.5B MoE with Zero Full-Attention Layers — Native 1M via SWA+DSA

This morning’s OpenShell note was a runtime policy ticket. Last night’s JEV-27B was System 1; yesterday morning’s Holo4 was computer-use $/task. Agents that already have policy, micro-decisions, and a GUI path still need a different board — what decode costs at a million tokens when the network has no full-attention layers. On September 27, 2026, NaiveAI published Naive-N0.5-Flash: open-weight 309B / 15.5B MoE under MIT, native 1M via hybrid SWA + DSA, plus companion runtime NaiveRT.

Primary sources: NaiveAI technical post (Sep 27), Hugging Face model card, Aivio (Sep 28). Numbers below are NaiveAI’s (or Aivio’s restatement). Treat NaiveRT peaks, indexer wall-time, CPT throughput, and chart benches as vendor claims. Do not invent percentages from the research-post figures — point readers there.

Spec sheet (named numbers)

Lab context (short): founded Feb 2026; Dai Jifeng; Tencent-backed; ~$400M raised / ~$1.42B valuation before first model (Aivio). Infra claim: ~10M sandboxes/week, 100k concurrent peak (vendor).

What actually changed for long-context decode

The durable change is not “another 300B MoE.” It is a full-attention-free stack at native 1M, built as a MiMo-V2.5 retrofit.

Most transformers keep a few global layers so long-range information survives; at a million tokens those layers dominate decode. Naive removes them. Every layer is local (SWA, 128) or sparse (DSA, top 2,048). The indexer still walks history and the full KV cache still sits in memory — same storage sentence as a dense stack — but backbone attention compute and memory traffic shrink to the selected set. Decode-cost board: not “we deleted the KV,” yes “we stopped paying full attention on every layer.”

GQA4 instead of MLA, a 16-head indexer, and the vendor −44% indexer wall-time claim are the co-design details that matter if you already read DeepSeek-V4.1-Flash. DeepSeek Flash sells 890 bytes/token KV and activate asymmetry. Naive sells zero full-attn layers + hybrid SWA/DSA + a named NaiveRT peak. Different tickets; both care about long loops.

CPT adapts the base: freeze everything but the indexer for 50B under KL to a full-attention reference, then 3T sparse CPT, then 200B decay/SFT — all at native 1M. Hybrid sequence parallelism matches the split: Ulysses for DSA’s full-history need, Halo for SWA’s 128-token ghost region.

NaiveRT and vendor benches

NaiveRT: mega-kernel fusion, PDL, speculative decoding with fused DFlash draft, bitwise-deterministic sampling (same seed → identical tokens, logits, KV). Peak 2,122 tok/s on 8 GPUs is a best one-second window with thinking off on 41 HTML/SVG requests — not a multi-user SLA. Speculative-round cut (3.4 ms vs SGLang 12.3 ms) is same-system. Source “by Oct 12”; until then re-time Ultrafast yourself.

Coding/agentic and AI R&D figures are NaiveAI’s own on Claude Code 2.1.207 (1M ctx, temp 1.0, top-p 0.95, basic file I/O + Bash only). Competitors are quoted from other vendors’ blogs/leaderboards, not re-run. Named families: SWE-Bench Pro, DeepSWE v1.1, Terminal-Bench 2.1, ALE-CLI, ProgramBench, PostTrainBench, MLE-bench-30, PaperBench. No invented chart percentages — open the research post figures and re-run your suite before retiring a Sol / Opus / MiMo / DeepSeek Flash route.

Optional RSI aside (not the story): 400 hours / 15 rounds → AutoWM WorldArena-1 Track 1 77.43 vs prior published 73.64 (vendor). Process anecdote only.

Where it sits next to OpenShell, JEV, Holo4, Flash, MiMo, and closed agents

OpenShell answers what the process may touch. JEV answers micro-decisions. Holo4 answers desktop computer-use. Naive answers decode cost when context hits 1M and every layer is SWA or DSA. Inferra / Leo X remain memory CapEx. MiMo-V2.6-Pro remains the open price-floor night — do not collapse this V2.5 retrofit into that SKU.

Caveats I would write into any Naive-N0.5-Flash ticket

1. Vendor harness ≠ your fleet. Re-run SWE / DeepSWE / Terminal-Bench / your repo before moving traffic.
2. Do not invent chart percentages. Point at the research-post figures.
3. Full KV cache retained. Sparse cuts compute/traffic, not storage — pair with Inferra / Leo X if CapEx is the constraint.
4. 2,122 tok/s is a peak window, not a multi-tenant Ultrafast SLA.
5. NaiveRT “by Oct 12.” Near-term path is transformers FP8 (NaiveAI/Naive-N0.5-Flash-FP8, ~315 GB).
6. API priced, not necessarily live. Confirm when the endpoint answers.
7. MiMo-V2.5 ≠ V2.6-Pro. Cross-link MiMo-V2.6-Pro as lineage foil only.
8. Not OpenShell / JEV / Holo4 / DeepSeek Flash KV-bytes. This is decode-cost architecture.

How I would evaluate if I ran million-token decode loops

1. Self-host smoke — load FP8 checkpoint; confirm 1M paths and temp 1.0 / top-p 0.95.
2. Decode cost at long context — tok/s and attention traffic at 128K → 1M vs a full-attn or MLA baseline.
3. Harness parity — your SWE / DeepSWE / Terminal-Bench / internal suite; no pasted NaiveAI chart rows.
4. NaiveRT after Oct 12 — re-time speculative round and Ultrafast; keep SGLang as control.
5. Keep boards separate — OpenShell for policy, JEV for System 1, Holo4 for computer-use, MiMo-V2.6-Pro for open price, DeepSeek Flash for cacheable API loops, Inferra / Leo X for memory fabric, Sol / Opus 5.5 for hard unattended coding. Naive is the zero-full-attn decode board.

This is an open decode-architecture release — 309B/15.5B, 39 SWA + 9 DSA, native 1M, MIT, announced $0.10/$0.40/$0.01, NaiveRT peak you should re-time — not a chat flagship and not an OpenShell rematch. Put Naive where million-token loops make full-attention layers the bill. Keep the other boards where they belong. Measure, then move traffic.

Create-only. Specs from NaiveAI’s Sep 27 research post, the HF card, and Aivio’s Sep 28 coverage. Update when NaiveRT GitHub lands, when a named third party reproduces the coding/agentic figures under a common harness, or when API availability and live rates are confirmed.

Sources: NaiveAI technical post (Sep 27) · Hugging Face model card · Aivio (Sep 28).

Anpoo Sivanadi, Staff Software Engineer