Raon-1 mini Preview: 8 KiB of KV per Token, and a Commit Log Worth Reading

KRAFTON released Raon-1 mini preview on October 21 under Apache 2.0. It's a 28.8B-parameter mixture-of-experts model with 4.3B active per token, trained from scratch, and Yonhap's report frames it as a Korean-first model built for low inference cost. The benchmark table is fine. What caught my attention is the memory profile: only 7 of its 25 layers keep a KV cache, and the model card puts that cache at about 8 KiB per token in BF16.

That number changes what you can do with a long-context model on one GPU. The release also has a short, very honest commit history on Hugging Face that tells you where the preview tooling breaks. Here's my read as someone who has to size hardware for these things.

The architecture, in operator terms

From the model card and the GitHub README:

• 25 layers, hidden size 2,560. 18 layers are Raon Delta Attention (RDA), a linear-attention variant that KRAFTON says decouples the erase and write gates of Kimi Delta Attention. RDA layers hold a fixed-size state, so their memory doesn't grow with context.
• 7 layers are Gated MLA, with KV LoRA rank 512, query LoRA rank 576, and 40 heads (192-dim QK, 128-dim V). These are the only layers with a per-token cache.
• No positional encoding (NoPE) in either attention type.
• MoE: 128 routed experts, 6 routed plus 1 shared active per token, expert intermediate size 1,088. The first layer is a dense FFN at 12,032.
• Raon-mHC residuals with 4 streams, a cheaper take on DeepSeek's manifold-constrained hyper-connections.
• Vocabulary 259,840, untied LM head, 262,144-token native context.

The ratio is 3 RDA layers to 1 MLA layer. Combine a mostly-linear stack with MLA's compressed latent cache, and you get a very small KV footprint.

What 8 KiB per token buys you

Some arithmetic, using the card's figure:

• A full 262,144-token sequence needs about 2 GiB of MLA cache in BF16.
• For comparison, when I read the Recursive Flint serving kit earlier this week, its KV came to 28 KiB per token. That's roughly 3.5x more per token for a model with 39.8B total parameters, not wildly larger.

Both quickstart profiles run the cache in fp8_e4m3. If FP8 halves the BF16 figure, which is my assumption since the card only gives the BF16 number, a 128-sequence batch at full context on the B300 profile needs on the order of 128 GiB of MLA cache. That's a lot, but it's within reach of a single B300. A dense-attention model of this size couldn't come close.

Keep the caveat in mind. The RDA layers still carry per-sequence state, and the card doesn't say how big it is. When you plan concurrency, measure the real per-sequence footprint under load. Don't size from the 8 KiB figure alone.

The consumer and desk-side profiles

KRAFTON published two small-box configurations, and the flags are the interesting part.

RTX 5090 (32 GB), NVFP4 checkpoint:

--max-model-len 32768
--max-num-batched-tokens 2048
--max-num-seqs 8
--gpu-memory-utilization 0.85
--kv-cache-dtype fp8_e4m3
--kv-cache-memory 3221225472
--spec-method dflash --spec-tokens 7

That's a hard-capped 3 GiB KV budget, 32K context, and eight sequences. Using my FP8 assumption, eight full 32K sequences need about 1 GiB of MLA cache, so I'd guess the rest of the budget goes to RDA state and the draft model's needs. The card doesn't break this down. It also requires CUDA Toolkit 13 with nvcc on PATH, and it sets VLLM_WSL2_ENABLE_PIN_MEMORY=1 for WSL2, so somebody clearly tested this on a Windows gaming box.

DGX Spark (GB10): a public ARM64 container, ghcr.io/krafton-ai/raon-vllm-spark:sm121-4e09534, with SM 12.1 compatibility fixes. It's tested on driver 580.159.03 and runs 128K context, eight sequences, a 2,048-token scheduler batch, and a 3 GiB FP8 KV cache. The instructions pin both the target and draft snapshots by commit hash and run with HF_HUB_OFFLINE=1. I wish more model releases did this. It's the only way to know the weights you benchmark are the weights you deploy.

The MXFP8 base model on a 32 GB card is a different story. The Transformers path tells you to use device_map="auto" with CPU offload, which is fine for a smoke test and nothing more.

DFlash2: read the concurrency columns

KRAFTON ships a DFlash2 draft model and reports up to 2.18x output throughput on a single B300. That number holds at concurrency 1 on GSM8K. The full table, measured with vLLM 0.28.0, an MXFP8 target, 7 draft tokens, and thinking enabled, is more useful:

Two things stand out. First, the speedup drops by roughly a quarter to a third once you batch, which is what you'd expect: at higher concurrency the GPU has less idle compute for speculation to use. Second, open-ended chat (MT-Bench) has the shortest acceptance length and the smallest gain. If your traffic is agentic chat rather than math, plan around 1.4–1.6x, not 2x. At C=32 on MATH-500, KRAFTON measured 5,166.9 output tok/s with DFlash2 against 3,082.5 without.

The commit log is the real release note

The Hugging Face commit history from launch day is short, and it's the most practical document in the release. Things I'd flag before anyone on my team tried this:

• Kernel mismatch on the vLLM fork. One commit explains that pip install git+... checks out a detached HEAD, so the fork's setup.py couldn't resolve its upstream base. It then pulled nightly precompiled kernels whose op signatures didn't match, and "NVFP4 fails in gptq_marlin_repack." The fix: the fork's setup.py now defaults to the matching commit (krafton-ai/vllm a9d4a5c85). If you built from an early copy of the README, rebuild.
• The draft repo was renamed. KRAFTON/Raon-1-mini-preview-assistant is now KRAFTON/Raon-1-mini-preview-dflash2. Old scripts will fail to find it.
• MLX truncates thinking by default. mlx_lm defaults to 512 output tokens. Clients that don't send max_tokens, Hermes Agent among them, got thinking output cut off. The README now passes --max-tokens 32768. Also, mlx_lm server loads whatever model the request names, so the MLX path needs KRAFTON/Raon-1-mini-preview-mlx as the model ID or you get a 404.
• Tokenizer warning is a false positive. Without transformers_version in config.json, Transformers 5 warns about an "incorrect regex pattern" and suggests fix_mistral_regex=True. The commit says the flag would change tokenization. Don't turn it on.
• First vLLM start compiles kernels for about 10 minutes. Keep the compile cache on a persistent volume, or every pod restart costs you that time.

None of these is unusual for a launch day. They all mean the same thing, though: this is a preview on forked runtimes. vLLM support is "beta" from krafton-ai/vllm, and llama.cpp support comes from krafton-ai/llama.cpp with "upstream support pending." The repos are also gated, with automatic approval, so a CI job needs an HF token.

Benchmarks, with the footnotes attached

The card compares against Qwen3.6-35B-A3B and Gemma 4 26B-A4B as MoE peers. Raon-1 mini leads on IFBench (84.2 vs 66.6 and 80.9), BrowseComp Plus (55.7 vs 34.8 and 32.2), and SWE-Verified (70.8 vs 63.2 and 61.6). It trails on MMLU Pro (76.5 vs 85.9 and 82.9), Terminal-Bench Lite (47.0 vs 63.3 and 52.0), and BFCL v4 (59.8 vs 64.7 and 65.8).

Read the footnotes. Every evaluation used gpt-5.6-luna as judge and user simulator. SWE-Verified ran on the mini-SWE-agent harness. AutomationBench used the public split, not the official evaluation set. The Korean chatbot score comes from an internal benchmark that hasn't been released. These are KRAFTON's numbers on KRAFTON's harnesses. The weak terminal and function-calling scores are the ones I'd check first if I wanted this model for agents.

Where I'd use it

If you need long context on a single 32 GB card or a GB10 box, this is one of the better-documented options right now, mainly because of that 7-of-25 cache layout. For a shared serving cluster, I'd wait. The preview label, two forked runtimes, and a judge-scored benchmark table add up to something to benchmark, not something to put behind an SLO. Pull the pinned revisions, measure real per-sequence memory with RDA state included, run your own traffic mix through DFlash2 at your actual concurrency, and check again when the final release lands with upstream vLLM support.

Anpoo Sivanadi, Staff Software Engineer