AI BENCHY Compare

Google: Gemma 4 26B A4B vs Mistral: Mistral Small 4

Last updated at: 2026-04-04

Metric	Gemma 4 26B A4B Gemma 4 26B A4B none Release: 2026-04-03	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16

Metric	Gemma 4 26B A4B Gemma 4 26B A4B none Release: 2026-04-03	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16
Score	6.3	5.6
Rank	#53	#68
Consistency	9.1	7.0
Tests Correct
Attempt pass rate	47.1%	49.0%
Flaky tests	2	6
Total Runs	51	51
Cost per result	0.068	0.502
Total Cost	$0.005	$0.026
Input Price	$0.130 / 1M	$0.150 / 1M
Output Price	$0.400 / 1M	$0.600 / 1M
Output Tokens	1,335	12,288
Reasoning Tokens	0	28,112
Response Time (avg)	6.56s	4.18s
Response Time (max)	57.10s	25.25s
Response Time (total)	111.54s	71.03s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	8.3	10.0	75.0%	0		1.28s	230	0
Mistral Small 4	5.6	3.8	66.7%	3		2.67s	4,055	4,778

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	3.0	10.0	0.0%	0		30.53s	309	0
Mistral Small 4	3.0	10.0	0.0%	0		25.25s	2,612	10,700

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		1.70s	285	0
Mistral Small 4	7.3	5.9	83.3%	1		1.23s	335	723

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	3.6	7.2	22.2%	1		2.49s	27	0
Mistral Small 4	5.3	7.2	44.4%	1		6.11s	2,621	6,904

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	4.0	10.0	0.0%	0		3.54s	85	0
Mistral Small 4	4.8	10.0	0.0%	0		2.05s	821	828

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	4.4	6.9	16.7%	1		1.08s	75	0
Mistral Small 4	7.3	5.8	83.3%	1		1.38s	540	1,031

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	5.7	10.0	33.3%	0		739ms	114	0
Mistral Small 4	3.4	9.7	0.0%	0		2.00s	983	2,338

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		57.10s	210	0
Mistral Small 4	10.0	10.0	100.0%	0		3.50s	321	810

Quick Compare

Switch Comparison Pair

Gemma 4 26B A4BnonevsGPT-5 Nanomedium Mistral Small 4mediumvsGLM 5 Turbonone Mistral Small 4mediumvsGLM 4.7 Flashnone Gemma 4 26B A4BnonevsGrok 4.20 Multi Agent Betamedium Mistral Small 4mediumvsQwen3.5-27Bnone Gemma 4 26B A4BnonevsMercury 2medium Mistral Small 4mediumvsGPT-5.4none Mistral Small 4mediumvsMiMo-V2-Pronone Mistral Small 4mediumvsQwen3.5-122B-A10Bnone Mistral Small 4mediumvsGrok 4.20none Mistral Small 4mediumvsGrok 4.20 Betanone Mistral Small 4mediumvsKimi K2.5none