AI BENCHY Compare

Google: Gemma 4 31B vs Qwen: Qwen3.5-27B

Last updated at: 2026-04-02

Metric	Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02	Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24

Metric	Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02	Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24
Score	8.6	8.3
Rank	#5	#13
Consistency	9.6	8.7
Tests Correct
Attempt pass rate	80.4%	80.4%
Flaky tests	1	3
Total Runs	51	51
Cost per result	0.109	3.885
Total Cost	$0.015	$0.467
Input Price	$0.140 / 1M	$0.195 / 1M
Output Price	$0.400 / 1M	$1.560 / 1M
Output Tokens	9,568	2,125
Reasoning Tokens	22,501	223,335
Response Time (avg)	21.81s	52.01s
Response Time (max)	68.92s	163.96s
Response Time (total)	327.16s	884.10s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		12.89s	962	2,046
Qwen3.5-27B	8.7	7.9	91.7%	1		19.75s	569	31,505

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
Qwen3.5-27B	10.0	10.0	100.0%	0		163.96s	483	9,991

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		21.11s	1,822	2,951
Qwen3.5-27B	10.0	10.0	100.0%	0		30.26s	270	16,150

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	7.7	10.0	66.7%	0		38.48s	4,349	8,985
Qwen3.5-27B	5.3	10.0	33.3%	0		79.53s	43	52,368

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		9.57s	105	888
Qwen3.5-27B	6.1	3.1	66.7%	1		101.41s	70	23,147

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		12.76s	533	2,035
Qwen3.5-27B	10.0	10.0	100.0%	0		19.66s	97	11,638

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	8.8	7.9	88.9%	1		27.63s	1,797	5,596
Qwen3.5-27B	8.2	7.7	77.8%	1		64.61s	245	77,213

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
Qwen3.5-27B	10.0	10.0	100.0%	0		7.45s	348	1,323

Quick Compare

Switch Comparison Pair

Gemini 3.1 Flash Lite PreviewlowvsQwen3.5-27Bmedium Gemini 3 Flash PreviewnonevsQwen3.5-27Bmedium Gemini 3 Flash PreviewlowvsQwen3.5-27Bmedium Gemini 3.1 Flash Lite PreviewnonevsQwen3.5-27Bmedium GPT-5.2 ChatnonevsQwen3.5-27Bmedium Gemma 4 31BmediumvsGPT-5.2 Chatnone GPT-5.3 ChatnonevsQwen3.5-27Bmedium Gemma 4 31BmediumvsGPT-5.3 Chatnone Claude Sonnet 4.6nonevsQwen3.5-27Bmedium Claude Sonnet 4.6nonevsGemma 4 31Bmedium Gemma 4 31BnonevsQwen3.5-27Bmedium Qwen3.5-27BmediumvsGLM 5none