AI BENCHY Compare

Google: Gemma 4 26B A4B vs Google: Gemma 4 31B

Last updated at: 2026-04-04

Metric	Gemma 4 26B A4B Gemma 4 26B A4B medium Release: 2026-04-03	Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02

Metric	Gemma 4 26B A4B Gemma 4 26B A4B medium Release: 2026-04-03	Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02
Score	8.3	8.6
Rank	#13	#5
Consistency	8.9	9.6
Tests Correct
Attempt pass rate	80.4%	80.4%
Flaky tests	2	1
Total Runs	51	51
Cost per result	0.189	0.109
Total Cost	$0.025	$0.015
Input Price	$0.130 / 1M	$0.140 / 1M
Output Price	$0.400 / 1M	$0.400 / 1M
Output Tokens	12,412	9,568
Reasoning Tokens	39,955	22,501
Response Time (avg)	17.38s	21.81s
Response Time (max)	73.55s	68.92s
Response Time (total)	278.01s	327.16s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		6.20s	1,142	3,045
Gemma 4 31B	10.0	10.0	100.0%	0		12.89s	962	2,046

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	9.6	10.0	100.0%	0		73.55s	5,415	13,112
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		16.51s	1,567	2,827
Gemma 4 31B	10.0	10.0	100.0%	0		21.11s	1,822	2,951

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	2.9	4.4	22.2%	2		23.62s	2,469	7,105
Gemma 4 31B	7.7	10.0	66.7%	0		38.48s	4,349	8,985

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		29.76s	25	5,075
Gemma 4 31B	10.0	10.0	100.0%	0		9.57s	105	888

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		17.54s	887	4,470
Gemma 4 31B	10.0	10.0	100.0%	0		12.76s	533	2,035

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	7.9	9.6	66.7%	0		8.52s	457	3,065
Gemma 4 31B	8.8	7.9	88.9%	1		27.63s	1,797	5,596

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		9.01s	450	1,256
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0

Quick Compare

Switch Comparison Pair

Gemma 4 26B A4BmediumvsGPT-5.2 Chatnone Gemma 4 31BmediumvsGPT-5.2 Chatnone Gemma 4 26B A4BmediumvsGPT-5.3 Chatnone Gemma 4 31BmediumvsGPT-5.3 Chatnone Claude Sonnet 4.6nonevsGemma 4 26B A4Bmedium Claude Sonnet 4.6nonevsGemma 4 31Bmedium Gemma 4 26B A4BmediumvsQwen3.5 Plus 2026-02-15none Gemma 4 26B A4BmediumvsGLM 5none Gemma 4 31BmediumvsQwen3.5 Plus 2026-02-15none Gemma 4 31BmediumvsGLM 5none Gemma 4 26B A4BmediumvsMiMo-V2-Omninone DeepSeek V3.2nonevsGemma 4 26B A4Bmedium