Mellum2.1: A 12B MoE Coding Agent Model, and the GGUF Builds That Make It Runnable Tonight

On October 8, JetBrains released Mellum2.1, the post-trained successor to the Mellum2 Thinking checkpoint they open-sourced in June. The architecture did not move. What moved is everything after pre-training: a summer of reinforcement learning inside real repositories, with shell and file tools, across millions of sandboxed runs. By October 9 the official GGUF repository was live with named quantizations — the launch post still said “coming soon,” so the artifact trail is what I am reading, not the press tense.

I care about this class of model the same way I care about vLLM 0.31 and the Vera Rubin day-zero serving notes: small, fast workers you can put next to a bigger planner, under your own keys, on hardware you already own. Mellum2.1 is not a frontier chat model. It is a compact MoE built to explore a repo, edit files, and check its own changes.

What did not change: the MoE shape

From the Hugging Face card:

That is the same skeleton as Mellum2. Sliding-window attention on three-quarters of the layers keeps decode cost bounded while every fourth layer still sees full context — the same efficiency bet Aleph Alpha made on Kolibri-1, just at a smaller total parameter count. JetBrains is explicit that the Thinking variant emits chain-of-thought before the final answer; treat that as part of the serving contract when you size max_tokens.

What did change: RL as the main stage

The launch post and the card agree on the post-training story. RL stopped being a short final polish and became the main stage. JetBrains added filtered open RL datasets plus in-house tasks across math, competitive programming, science, tool use, and software engineering, and they built infrastructure to run thousands of environments with millions of sandboxes over the training run. For software engineering, the model trains inside real repositories with a shell and file-editing tools and is rewarded when tests pass.

That is the difference between “fast autocomplete that cannot navigate a tree” and “a worker you can put under an agent harness.” Mellum2 was fast; Mellum2.1 is the same speed class with agent behavior that actually moves on repository tasks.

The self-reported table you should read carefully

JetBrains evaluated Mellum2.1 Thinking, Mellum2 Thinking (re-run on this pipeline), Gemma 4 E4B, and Qwen3.5-9B with the same harness. Non-agentic benches use greedy decoding. Agentic benches use Pi v0.73.1 with shell and file tools, each model’s default sampling (temperature 1.0 for Mellum2.1), 114K context, and up to 16K tokens per turn. All numbers below are self-reported.

The agentic rows are the headline. SWE-bench Verified jumps from 2.0 on Mellum2 to 47.0 on Mellum2.1 — still a few points behind Qwen3.5-9B at 50.0, but in the same band after a summer of RL. Terminal-Bench 2.1 goes from 0.6 to 17.4. SWE-bench Pro from 0.0 to 28.0. LiveCodeBench v6 is where Mellum2.1 leads the group at 82.0.

I would not treat these as independent leaderboard submissions. They are useful because the harness is held fixed across four models. If you rematch SWE-bench Verified in your own agent, expect your absolute numbers to move with the harness, not with a blog table.

Speed claims, and what is still a chart

The architecture freeze means Mellum2.1 should decode like Mellum2. JetBrains says multi-token prediction (MTP) makes a single request about 1.6× faster, and that under heavy load Mellum2.1 serves almost twice as many tokens as Qwen3.5-9B in their group. Those are relative vendor claims from charts I cannot paste here. Absolute tokens/s on a named GPU are not in the text of the launch post or the model card I fetched. Plan capacity from your own vllm or llama.cpp smoke test, not from “almost 2×.”

The MTP head for speculative decoding in vLLM is still listed as coming soon on the BF16 card. The GGUF path is what landed first for local runtimes.

Artifacts you can actually pull

BF16 / safetensors — JetBrains/Mellum2.1-12B-A2.5B-Thinking. Documented vLLM serve:

vllm serve JetBrains/Mellum2.1-12B-A2.5B-Thinking \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

Drop the tool flags if you only need reasoning. Sampling defaults on the card for a quick chat call: temperature=0.6, top_p=0.95, top_k=20, with a high max_tokens ceiling for Thinking traces.

GGUF — JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF, one file per quantization. JetBrains publishes KL divergence and top-token match vs BF16 on Wikitext-2 (n_ctx=512):

llama-server -hf JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M \
  --ctx-size 131072 \
  --temp 0.6 --top-p 0.95 --top-k 20

Ollama one-liner from the same card: ollama run hf.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M. The OpenAI-compatible endpoint on llama.cpp is http://localhost:8080/v1.

How I would wire it

For a cluster that already runs generative serving, I would treat Mellum2.1 as a sub-agent, not as the planner: root-cause a failing test, draft a patch, run the suite, report back. Put BF16 under vLLM when you want tool calling and long Thinking traces with Hermes parsing. Put Q4_K_M under llama.cpp on a workstation when the constraint is privacy and laptop RAM, and accept the KLD trade. Keep Liquid open d1 for the zero-token typed decisions at the edge of the same agent loop — routing and triage — and reserve Mellum for the repository work that needs shell and files.

Do not confuse Apache 2.0 here with every other open card on the site. Mellum2.1 really is Apache. That is rarer than the HF default implies, and it matters if your legal stack is tired of “open until $10M ARR” footnotes.

The honest limit: agentic scores are JetBrains’ harness, MTP absolute throughput is still a chart, and the vLLM MTP head is not in the GGUF drop. What you have tonight is a named MoE, a fixed evaluation table, and an 8.1 GB file that serves on your machine. That is enough to decide whether a 2.5B-active coding worker belongs next to your planner — by running it, not by waiting for another keynote.

Related: Liquid Open d1: Zero Output Tokens, Named Jetson Latencies, and What a Decision Model Actually Is · Claude Haiku 5.5: The Price Cut Is Real, but Read the 100K Tier, the Tokenizer, and the Output Bill · Reflection’s Beam: A 501B Open-Weight MoE, and the GB300 Cluster Numbers Behind It.

Anpoo Sivanadi, Staff Software Engineer