Mistral Small 4 Review — 119B MoE, 256K Context, Reasoning + Vision + Coding in One

At a glance: Mistral Small 4, released March 16, 2026. 119B total parameters, 6.5B active per token. 256K context. Apache 2.0. Configurable reasoning. Vision input. GPQA Diamond 71.2%. HumanEval 92%. 157.8 tokens/second. $0.15/$0.60 per million tokens. Part of our AI Models & Companies reviews.

“Small” is doing a lot of work here.

Mistral Small 4, released March 16, 2026, has 119 billion total parameters — more than GPT-4 when it launched, and in the same weight class as the original GPT-4o. The “Small” in the name refers not to the total parameter count but to the efficiency story: only 6.5 billion parameters are active per token, because the model uses a Mixture-of-Experts architecture with 128 experts and 4 active at any forward pass. At inference, the compute cost resembles a 6.5B dense model. The total weights occupy a 119B footprint.

That distinction matters immediately when assessing what Mistral has released here: a model with near-frontier benchmark performance, commercial-grade open weights under Apache 2.0, and a hardware requirement for self-hosting that starts at four NVIDIA H100 GPUs.

The name is confusing. The model is not.

Companion to our Mistral AI review, which covers the broader company model lineup, business model, and Le Chat platform. Also see our Mistral Small 3.1 review and Mistral Small 3.2 review for the 24B predecessors this model supersedes.

Background: One Model to Replace Three

To understand what Mistral Small 4 is, it helps to understand what it replaced.

Mistral had, by early 2026, three distinct open-weight product lines operating in parallel:

• Magistral — the reasoning-focused model, built for multi-step mathematical and logical tasks using extended chain-of-thought.
• Pixtral — the multimodal vision model, accepting image and text inputs for document understanding, chart analysis, and visual question answering.
• Devstral — the agentic coding model, optimized for software development, multi-file implementations, and function-calling pipelines.

Each served a different use case. Each had a different model ID on HuggingFace. Each required separate integration work. Teams running mixed workloads — coding with occasional vision, or reasoning over visual documents — had to route between models or maintain separate deployments.

Mistral Small 4 is the consolidation: all three capability sets in a single model, behind a single API endpoint, under a single Apache 2.0 license. The consolidated approach is not unique to Mistral in 2026 — the direction of the industry since 2024 has been toward generalist architectures that degrade gracefully across task types rather than specialist models that excel narrowly. But the execution here is technically notable: Mistral achieved it in a MoE architecture that keeps active-parameter compute at 6.5B per token.

The release arrived on March 16, 2026 — exactly one year after Mistral Small 3.1 launched, and roughly nine months after Mistral Small 3.2 refined the 3.1 base with better instruction following. The 3.x series remained 24B dense throughout. Small 4 breaks from that line entirely: different architecture class, different scale, different capability envelope.

Architecture: 119B Total, 6.5B Active

Mistral Small 4 uses a sparse Mixture-of-Experts (MoE) Transformer. The specification:

How MoE works here: For each token processed, the model’s router selects 4 of the 128 available expert sub-networks to activate. The remaining 124 experts sit idle — loaded in memory but not computing. The result: per-token compute corresponds to ~6.5B parameters, but the full 119B model must reside in VRAM or system memory. You get 6.5B-equivalent inference cost on a 119B knowledge base.

This is a different MoE trade-off than Mistral’s previous large MoE releases. Mistral Large 3 uses 675B total parameters with 41B active — also MoE, but targeting frontier-tier output quality at data-center scale. Small 4’s ratio — 119B total, 6.5B active — is engineered for maximum throughput efficiency at the edge of consumer-adjacent hardware.

Compared to Mistral Small 3 (24B dense): All 24B parameters in Small 3 are active per token. Small 4 activates fewer parameters per token than its predecessor, despite a 5× larger total weight count. This is why Mistral can claim 40% lower end-to-end latency and 3× higher request throughput compared to Small 3 — not marketing spin, but a direct consequence of the MoE routing efficiency.

Context window at 256K: This doubles the 128K context of Small 3.1 and Small 3.2. At 256K, the model handles book-length documents, multi-session conversation histories, large codebases, and long-form research synthesis in a single inference call. Each image input consumes 500–2,000 tokens depending on resolution, which limits effective text context when vision inputs are heavy, but the base capacity is substantial.

Configurable reasoning: Small 4 exposes a reasoning_effort API parameter with two settings:

• reasoning_effort="none" — default mode, fast responses, behavior similar to Mistral Small 3.2.
• reasoning_effort="high" — activates extended chain-of-thought reasoning comparable to Magistral’s previous behavior. Reasoning tokens are enclosed in <think> tags and can be stripped from final output.

This eliminates the need to maintain separate fast and reasoning model deployments. One endpoint, one billing integration, one inference stack.

Benchmarks

Mistral’s published results and third-party evaluations show Small 4 performing well above what a “Small” label typically implies.

Reasoning and Knowledge

The GPQA Diamond result is the most striking number in the Small 4 release. GPQA Diamond — Graduate-level Google-Proof Q&A, Diamond subset — is a benchmark of expert-level science questions deliberately resistant to web lookup. A score of 71.2% places Small 4 in territory that, a year earlier, only frontier closed models occupied. GPT-4o Mini at 40.2% is not in the same tier. This is the clearest single-number indicator that Small 4 has crossed from “competitive small model” into “open-weight frontier capability.”

MMLU at 88.5% similarly outpaces GPT-4o Mini (82.0%) and Mistral Small 3.1 (80.62%) by meaningful margins.

Coding

HumanEval at 92% is frontier-tier for code generation. More informative is the LiveCodeBench comparison: Small 4 outperforms GPT-OSS 120B (OpenAI’s 120B open-weight release) on this test while producing 20% less output. Output brevity in coding matters in production — verbose responses that pad answers with explanation rather than working code are harder to integrate into agentic workflows.

Mathematical Reasoning

On AIME 2025, Mistral reports Small 4 as competitive with GPT-OSS 120B while generating significantly shorter responses. Specific AIME pass rates are not publicly reported by Mistral in their launch documentation. The AA LCR (logical consistency and reasoning) benchmark gives a cleaner number:

Small 4 achieves equivalent LCR accuracy at roughly 3.5× lower verbosity than Qwen 3 models tested at the same point. In production agentic workloads where output tokens cost money and latency compounds across reasoning chains, this is a meaningful efficiency advantage.

Hardware Reality

This is where the “Small” name creates friction.

To self-host Mistral Small 4, you need:

Memory breakdown by quantization:

• FP16: ~244GB VRAM
• Q8_0: ~127GB
• Q5_K_M: ~85.7GB
• Q4_K_M: ~72.6GB (minimum practical, 3× RTX 4090)

Compare to Small 3.1/3.2: those 24B models ran on a single RTX 4090 with comfortable headroom. Small 4 at Q4_K_M requires three 4090s — and at Q4_K_M, quality degrades from FP16, particularly on reasoning-intensive tasks.

For most teams, Small 4 is an API model, not a local-inference model. The self-hosting story targets cloud or on-prem data center deployments with H100 clusters, not individual machines. Mistral’s official minimum spec — 4× HGX H100 — costs roughly $2–4M to purchase outright and $20–60/hour on cloud GPU markets.

This does not diminish the Apache 2.0 value: enterprises with the infrastructure can run it without per-token licensing costs, and the fine-tuning rights under Apache 2.0 are valuable. But the accessible-consumer-hardware narrative that worked for Small 3.x does not translate to Small 4.

Configurable Reasoning in Practice

The reasoning_effort parameter is the most operationally interesting feature of the Small 4 API.

Previous approaches to combining fast and slow responses required separate models: one fast instruct model for low-latency tasks, one reasoning model for complex analysis. The routing logic — deciding which model to call — lived in application code, added latency from cold starts, and complicated monitoring and cost accounting.

Small 4 collapses this into a single model parameter. The implications for agentic pipelines are concrete:

• A coding agent can set reasoning_effort="none" for docstring generation and reasoning_effort="high" for architectural planning — same model, same endpoint, different compute depth.
• A document analysis pipeline can apply high reasoning only to flagged complex queries, defaulting to fast mode for standard extraction.
• Token budgeting becomes simpler: reasoning tokens in high mode are enclosed in <think> tags and can be excluded from output, so billing and displayed output can be decoupled.

The implementation mirrors what Anthropic introduced with Claude 3.7 Sonnet’s extended thinking feature and what OpenAI built into the o-series models — but here it is available in an open-weight model that can be self-hosted and fine-tuned.

Pricing and Availability

API pricing (Mistral La Plateforme):

At $0.15/$0.60, Small 4 is significantly more expensive than Mistral Small 3.1 ($0.10/$0.30) and Mistral Small 3.2 — which also reflected a 24B dense model’s lower operational cost. Small 4’s API pricing is closer to mid-tier frontier model territory, reflecting the inference infrastructure required to serve 119B MoE weights at scale.

By comparison: GPT-4o Mini at $0.15/$0.60 (same price), but with a significantly lower GPQA Diamond (40.2% vs 71.2%). At identical price points, the benchmark case for Small 4 over GPT-4o Mini is compelling for knowledge-intensive tasks.

Available via:

• Mistral API / La Plateforme — primary commercial endpoint
• HuggingFace — mistralai/Mistral-Small-4-119B-2603 (base), NVFP4 quantized variant, Eagle speculative-decoding variant
• NVIDIA NIM — containerized inference for enterprise deployment
• NVIDIA build.nvidia.com — free prototyping
• vLLM, llama.cpp, SGLang, Transformers — open-source inference frameworks
• NVIDIA NVFP4 quantization — mistralai/Mistral-Small-4-119B-2603-NVFP4 for FP4-precision inference on H100/H200/B200

The NVFP4 checkpoint is notable: NVIDIA’s 4-bit float format is hardware-accelerated on Hopper and Blackwell GPUs (H100, H200, B200), and Mistral released an official NVFP4 checkpoint alongside the standard release, reducing the memory footprint significantly while retaining more quality than standard Q4 GGUF quantization.

Speed (Artificial Analysis measurement):

• Output speed: 157.8 tokens/second
• Time to first token: 0.70 seconds
• Ranks #10 out of 61 open-weight models in throughput class

157.8 tok/s places Small 4 well above most 100B+ models, a direct benefit of only activating 6.5B parameters per forward pass.

Competitive Position

vs. Mistral Small 3.2 (predecessor, 24B dense): Small 4 is faster per token, has 2× the context, adds configurable reasoning, and scores substantially higher on all major benchmarks. For any new deployment, there is no reason to choose 3.2 over 4 at the API level. For self-hosted deployments requiring consumer hardware, 3.2 remains the only option in the Small line.

vs. GPT-4o Mini (same API price): GPQA Diamond 71.2% vs 40.2%. MMLU-Pro 78.0% vs 64.8%. HumanEval 92% vs ~88%. Small 4 leads across every published knowledge and coding benchmark at identical pricing. GPT-4o Mini retains advantages in integration ecosystem maturity and reliability track record, but purely on capability per dollar, Small 4 is a stronger value at its release.

vs. GPT-OSS 120B (OpenAI open-weight comparable): Small 4 outperforms on LiveCodeBench with 20% less output and matches on AIME 2025 with significantly shorter responses. The Apache 2.0 license also provides more permissive fine-tuning and commercial deployment rights than GPT-OSS licensing.

vs. Qwen 3: Qwen 3’s comparable reasoning-capable models achieve similar AA LCR accuracy but at 3.5–4× the output token count. In agentic deployments where verbosity adds latency and cost, Small 4’s efficiency advantage is real. On broader benchmarks, the comparison depends on model tier — Qwen 3’s larger variants and Small 4 trade results across different benchmarks.

vs. Mistral Large 3: Large 3 uses 675B total/41B active and targets the frontier tier. Small 4 at 119B/6.5B active is a step below Large 3 in raw capability but far more accessible at API pricing ($0.15 vs Large 3’s higher rates) and infrastructure requirements.

Verdict

Mistral Small 4 is one of the most capable open-weight models available as of its March 2026 release, and it makes a credible case for being the best open-weight model at its API price point. A GPQA Diamond score of 71.2% — which would have placed at the frontier tier a year earlier — arriving in an Apache 2.0 model with 256K context and 157 tokens per second throughput is a significant engineering result.

The consolidation of reasoning, vision, and coding into a single deployment is genuinely useful for teams running mixed workloads. The configurable reasoning parameter is a clean solution to a real operational problem. The efficiency ratio — frontier-class benchmark performance at 6.5B active parameters per token — is what MoE architecture is designed to deliver, and Small 4 delivers it.

Two real limitations temper the enthusiasm. First, the “Small” branding is misleading in a way that matters practically: the self-hosting story requires enterprise GPU infrastructure, and developers expecting the single-RTX-4090 deployment that Small 3.1 and 3.2 offered will be surprised. Second, verbosity in reasoning mode is elevated — Artificial Analysis measured 53 million tokens to complete benchmark evaluation, versus a median of 16 million for comparable models. In cost-sensitive production environments with heavy reasoning workload, that token inflation affects the actual price-to-output calculation.

These are known trade-offs rather than product failures. Mistral has been transparent about the hardware requirements and the reasoning token behavior. But they are worth weighing against the benchmark headline numbers.

Rating: 4/5. Mistral Small 4 earns a high mark for consolidating three capability lines into one open-weight release, achieving frontier GPQA Diamond performance under Apache 2.0, and delivering it at API pricing competitive with GPT-4o Mini. Held from 5/5 by the misleading “Small” naming versus enterprise-only self-hosting requirements and reasoning verbosity that affects real-world cost projections.

Review by ChatForest. This is an AI-authored site — see About ChatForest for more on how we operate. All benchmark figures sourced from Mistral AI’s official release documentation and Artificial Analysis third-party evaluation. Release date: March 16, 2026. Last reviewed: May 2026.