AI BENCHY Compare

Google: Gemma 4 31B vs Qwen: Qwen3.6 35B A3B

Last updated at: 2026-05-22

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	Qwen3.6 35B A3B Qwen3.6 35B A3B medium Release: 2026-04-20

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	Qwen3.6 35B A3B Qwen3.6 35B A3B medium Release: 2026-04-20
Score	6.7	7.8
Rank	#76	#30
Reliability	10.0	9.4
Consistency	10.0	9.1
Tests Correct
Attempt pass rate	50.0%	70.4%
Flaky tests	0	2
Total Runs	60	60
Cost per result	0.030	1.050
Total Cost	$0.003	$0.126
Input Price	$0.120 / 1M	$0.150 / 1M
Output Price	$0.370 / 1M	$1.000 / 1M
Output Tokens	1,398	18,325
Reasoning Tokens	0	115,750
Response Time (avg)	3.84s	17.26s
Response Time (max)	26.13s	86.11s
Response Time (total)	69.13s	310.77s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		1.85s	45	0
Qwen3.6 35B A3B	10.0	10.0	100.0%	0		6.02s	1,154	12,385

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.8	10.0	50.0%	0		14.84s	726	0
Qwen3.6 35B A3B	6.6	10.0	50.0%	0		59.35s	6,601	22,535

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
Qwen3.6 35B A3B	0.0	0.0	0.0%	0		0ms	0	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.25s	285	0
Qwen3.6 35B A3B	10.0	10.0	100.0%	0		12.99s	2,591	9,968

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	7.7	10.0	66.7%	0		3.22s	27	0
Qwen3.6 35B A3B	5.3	7.2	44.4%	1		22.50s	6,193	39,116

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.09s	117	0
Qwen3.6 35B A3B	4.4	9.9	0.0%	0		8.66s	129	4,569

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		2.84s	78	0
Qwen3.6 35B A3B	10.0	10.0	100.0%	0		7.50s	219	7,404

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	33.3%	0		2.95s	108	0
Qwen3.6 35B A3B	7.9	7.5	77.8%	1		5.98s	676	9,447

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
Qwen3.6 35B A3B	0.0	0.0	0.0%	0		0ms	0	0

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		1.25s	12	0
Qwen3.6 35B A3B	3.0	10.0	0.0%	0		32.90s	762	10,326

Quick Compare

Switch Comparison Pair

Gemma 4 31BnoneFree AvailablevsGrok 4.20medium Gemma 4 31BnoneFree AvailablevsKimi K2.5medium DeepSeek V4 ProhighvsGemma 4 31BnoneFree Available Gemma 4 31BnoneFree AvailablevsQwen3.6 27Bmedium Gemini 3 Flash PreviewnonevsQwen3.6 35B A3Bmedium Gemini 3.5 FlashminimalvsQwen3.6 35B A3Bmedium Gemini 3.1 Flash Lite PreviewlowvsQwen3.6 35B A3Bmedium GPT-5.2 ChatnonevsQwen3.6 35B A3Bmedium Gemma 4 31BnoneFree AvailablevsGPT-5 Minimedium Gemma 4 31BnoneFree AvailablevsMiMo-V2-Omnimedium DeepSeek V3.2mediumvsGemma 4 31BnoneFree Available Gemma 4 31BnoneFree AvailablevsQwen3.5-35B-A3Bmedium