Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Mistral: Mistral Small 4 vs xAI: Grok 4.20 Multi-Agent Beta

Last updated at: 2026-03-17

Metric Mistral Small 4 Mistral Small 4 none Release: 2026-03-16 Grok 4.20 Multi-Agent Beta Grok 4.20 Multi-Agent Beta medium Release: 2026-03-12
Rank #61 #44
Score 5.3 6.2
Consistency 9.5 7.2
Cost per result 0.108 82.962
Total Cost $0.006 $4.978
Tests Correct
Attempt pass rate 33.3% 54.9%
Flaky tests 1 6
Total Runs 51 51
Output Tokens 1,624 298,948
Reasoning Tokens 0 296,529
Response Time (avg) 629ms 8.64s
Response Time (max) 1.72s 35.28s
Response Time (total) 10.70s 129.64s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 3.4 7.9 16.7% 1 395ms 182 0
Grok 4.20 Multi-Agent Beta 6.9 5.8 75.0% 2 3.46s 33,706 33,077
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 3.0 10.0 0.0% 0 1.72s 496 0
Grok 4.20 Multi-Agent Beta 3.0 10.0 0.0% 0 0ms 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 10.0 10.0 100.0% 0 822ms 261 0
Grok 4.20 Multi-Agent Beta 10.0 10.0 100.0% 0 5.54s 25,306 25,051
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 5.3 10.0 33.3% 0 367ms 28 0
Grok 4.20 Multi-Agent Beta 2.9 7.2 11.1% 1 24.67s 164,609 163,647
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 4.0 10.0 0.0% 0 729ms 205 0
Grok 4.20 Multi-Agent Beta 5.8 2.8 66.7% 1 6.40s 15,848 15,746
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 6.5 10.0 50.0% 0 380ms 69 0
Grok 4.20 Multi-Agent Beta 8.3 10.0 50.0% 0 4.63s 25,457 25,322
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 3.1 9.9 0.0% 0 589ms 170 0
Grok 4.20 Multi-Agent Beta 7.2 5.1 77.8% 2 5.01s 34,022 33,686
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 10.0 10.0 100.0% 0 1.40s 213 0
Grok 4.20 Multi-Agent Beta 3.0 10.0 0.0% 0 0ms 0 0

Quick Compare

Switch Comparison Pair

DeepSeek V3.2nonevsGrok 4.20 Multi-Agent BetamediumQwen3.5-FlashnonevsGrok 4.20 Multi-Agent BetamediumSeed-2.0-LitenonevsGrok 4.20 Multi-Agent BetamediumGemini 2.5 FlashnonevsGrok 4.20 Multi-Agent BetamediumQwen3.5-35B-A3BnonevsGrok 4.20 Multi-Agent BetamediumHunter AlphanonevsGrok 4.20 Multi-Agent BetamediumQwen3.5-122B-A10BnonevsGrok 4.20 Multi-Agent BetamediumMistral Small 4nonevsQwen3 Coder NextmediumGrok 4.20 Multi-Agent BetamediumvsGLM 5noneQwen3.5 Plus 2026-02-15nonevsGrok 4.20 Multi-Agent BetamediumMistral Small 4nonevsGLM 4.7 FlashmediumGPT-5.4nonevsGrok 4.20 Multi-Agent Betamedium