AI BENCHY Compare

Google: Gemma 4 31B vs Qwen: Qwen3.5-35B-A3B

Last updated at: 2026-04-02

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24
Score	6.7	7.3
Rank	#47	#37
Consistency	10.0	6.6
Tests Correct
Attempt pass rate	52.9%	78.4%
Flaky tests	0	7
Total Runs	51	51
Cost per result	0.023	3.877
Total Cost	$0.002	$0.349
Input Price	$0.140 / 1M	$0.163 / 1M
Output Price	$0.400 / 1M	$1.300 / 1M
Output Tokens	660	5,864
Reasoning Tokens	0	175,683
Response Time (avg)	2.55s	42.48s
Response Time (max)	4.68s	106.00s
Response Time (total)	38.20s	722.12s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		1.85s	45	0
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		21.13s	798	42,652

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
Qwen3.5-35B-A3B	4.7	1.6	66.7%	1		75.34s	775	12,485

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.25s	285	0
Qwen3.5-35B-A3B	7.3	5.9	83.3%	1		59.33s	235	19,493

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	7.7	10.0	66.7%	0		3.22s	27	0
Qwen3.5-35B-A3B	4.1	4.4	44.5%	2		88.34s	41	46,368

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.09s	117	0
Qwen3.5-35B-A3B	2.8	1.6	33.3%	1		30.30s	20	3,753

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		2.84s	78	0
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		24.45s	97	17,361

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	5.5	10.0	33.3%	0		2.95s	108	0
Qwen3.5-35B-A3B	6.4	4.4	77.8%	2		31.58s	3,589	32,206

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		4.65s	309	1,365

Quick Compare

Switch Comparison Pair

Claude Sonnet 4.6nonevsQwen3.5-35B-A3Bmedium Gemma 4 31BnonevsGPT-5 Minimedium Gemma 4 31BnonevsNemotron 3 SupermediumFree Available Gemma 4 31BnonevsGrok 4.1 Fastmedium Gemma 4 31BnonevsHunter Alphamedium GPT-5.3 ChatnonevsQwen3.5-35B-A3Bmedium Gemma 4 31BnonevsGPT-5.4 Minimedium Gemma 4 31BnonevsGrok 4.20medium Gemma 4 31BnonevsMercury 2medium Gemma 4 31BnonevsGPT-5 Nanomedium Gemma 4 31BnonevsKimi K2.5medium Gemma 4 31BnonevsGrok 4.20 Multi Agent Betamedium