JEV-27B: Open System 1 Decisions on a Frozen Qwen3.8-27B — KL 0.017, One B200

This morning’s Holo4 note was a computer-use cost-per-task ticket: GUI + code + MCP, open trajectories, OSWorld 2.0 at published $/task. Last night’s AliceAI-Foundation-80B was a from-scratch hybrid base. Agents still spend most of their wall-clock on a different class of work — small typed decisions that do not need a long chain of thought: is this phishing, which queue owns the ticket, does this diff break a code rule, how urgent is this on a 0–5 scale. On September 28, 2026 (Hugging Face blog September 27), AutoTrust AI shipped JEV-27B.

Primary sources: autotrust/JEV-27B model card, AutoTrust HF blog, PR Newswire release, the public distillation corpus SargeDev/jev-distill-corpus-v3, and the JEV-27B demo space. Benchmark rows below are AutoTrust’s harness unless marked independent. Treat the head-to-head vs hosted TypeSafe Jev 1.13 as vendor self-comparison, not third-party validation — AutoTrust says so explicitly.

Spec sheet (named numbers)

Sibling: JEV-9B (same recipe on Qwen3.5-9B; faster; weaker OOD / System 2) — use for short-option triage; promote to 27B for long lists, unfamiliar tasks, or System 2 quality.

What actually changed for people who run agent fleets

The durable product change is a detachable System 1 head that does not tax System 2.

Most stacks either call a closed decision API on every hop, or fine-tune a whole LLM into a classifier and lose the generator. AutoTrust’s BoE keeps the pretrained model frozen and trains only the decision block. On the 9B generation, merging the LoRA into the backbone dropped HumanEval 70.7% → 61.6%; on JEV-27B with blocks separate, HumanEval stays 78.0% and all 164 completions match the base byte-for-byte.

A decision is one prefill pass — no decode, no JSON scrape — returning a calibrated distribution you can threshold. The same engine answers ordinary chat / thinking through the untouched lm_head. Purchase: private, local System 1 next to an intact System 2.

Vendor benches vs what you should not invent

Six public text-decision groups (AutoTrust ran JEV-27B and hosted TypeSafe Jev 1.13 on 26 Sep 2026; NeoHorse / Open-Jev / Kev / Laya rows are published baselines AutoTrust did not re-run):

JEV-27B’s six-group mean is +0.22 pts vs their Jev run; higher on 4 of 6. Do not paste this as an independent bake-off.

Distribution fidelity (25,376 held-out Jev-labelled rows): mean KL(Jev ‖ JEV-27B) ≈0.017 — roughly 60 sampled decisions to gather one nat of evidence about which model produced them. ECE 0.0009; fitted temperatures ≈1.00. Among open Jev reproductions AutoTrust surveyed, only the JEV models publish this KL measure.

Independent transfer (gazelle93/decision-models-under-pressure, human gold; none of the data in training): at 16 options, JEV-27B reaches 96% of Jev’s accuracy (0.740 vs 0.769); option-order flip rate 7.4% vs teacher 7.0%. Faithful copy includes the teacher’s blind spots — poker shove 0.63 vs teacher 0.62 on a spot a solver always checks.

Speed (self-reported local vs third-party hosted; not like-for-like — hosted includes network):

Card also places full bf16 on one H100 80 GB at ~100 decisions/s and ~64 concurrent System 2 chats. Latency tables elsewhere were measured on B200.

Where it sits next to Holo4, AliceAI, Flash, MiMo, and closed agents

Holo4 answers what the agent does on the desktop. JEV-27B answers how cheaply and privately you gate the micro-decisions inside that loop. Different SKUs; same fleet.

Caveats I would write into any JEV-27B ticket

1. Self-comparison ≠ third-party. The +0.22 pts six-bench lead is AutoTrust measuring both models. Wait for Decision Index / JevBench v1.4.2 submissions before claiming leaderboard wins — AutoTrust has not submitted yet.
2. Distillation copies mistakes. Multi-hop, arithmetic, counting, adversarial inputs — if Jev is wrong, assume JEV-27B is wrong. Confidence-gate; do not run high-stakes without a human or stronger System 2 path.
3. Systems share weights, not knowledge. System 2 is unmodified Qwen3.8-27B; it was not trained to agree with System 1. Escalation can disagree.
4. KL claim is in-distribution. The 25,376 Jev-labelled rows sit inside the 53 training domains. OOD evidence is programmatic KL 0.104 / the independent 16-option 96%-of-teacher row — not another KL-to-Jev number.
5. Speed vs hosted is not A/B. Local median excludes network; hosted includes it and rate limits.
6. Not AutoJev-27B. denis-pplx/autojev-27b is an unrelated full-weight SFT decision model. Do not conflate cards.
7. Not a rematch of Holo4 / Inferra / Leo X. This is a decision adapter + frozen backbone, not computer-use VLMs or memory/optics CapEx.

How I would evaluate if I ran agent fleets with a decision hop

1. Local canary on one H100 or B200 — reproduce KL / six-bench subsets and your own routing / triage / code-rule set before retiring a hosted Jev path.
2. Confidence-gated escalate — System 1 above a threshold you fit on your validation data; System 2 (same engine) or Sol / Opus 5.5 below it.
3. Keep JEV-9B for high-QPS short-option triage if 27B latency budget is wrong; promote to 27B when option lists exceed ~8 or OOD / System 2 quality matters.
4. A/B against Holo4-style computer-use only where the decision is which GUI action — that is Holo4’s lane. Use JEV-27B where the state is already text/JSON and you need a typed distribution.
5. Keep AliceAI / MiMo / Flash on base / AA floor / cacheable API volume — JEV-27B is the System 1 board.

This is an open System 1 + intact System 2 release with named KL, six-bench means, and B200 latency — not a new closed flagship and not an AA rematch. Put JEV-27B where every agent hop is a calibrated yes/no, choice, or 0–5 score you want private and fast. Keep Holo4 on the computer-use board, MiMo on the open ceiling/price row, DeepSeek-V4.1-Flash on cacheable API loops, AliceAI on from-scratch bases, and Sol / Opus 5.5 on hard unattended work until your own thresholds say otherwise. Measure, then move the decision hop.

Create-only. Specs from AutoTrust’s Sep 27 HF blog, the Sep 28 PR Newswire note, the JEV-27B card, and the public distill corpus. Update when Decision Index / JevBench v1.4.2 rows land, when a Jev-labelled OOD set exists, or when a named third party reproduces the six-bench head-to-head.

Sources: autotrust/JEV-27B model card · AutoTrust HF blog · PR Newswire release · jev-distill-corpus-v3 · JEV-27B demo space.

Anpoo Sivanadi, Staff Software Engineer