AI BENCHY Compare

Google: Gemma 4 31B vs Z.ai: GLM 5.1

Last updated at: 2026-04-07

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	GLM 5.1 GLM 5.1 medium Release: 2026-04-07

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	GLM 5.1 GLM 5.1 medium Release: 2026-04-07
Score	6.7	8.0
Rank	#49	#23
Consistency	10.0	9.0
Tests Correct
Attempt pass rate	52.9%	76.5%
Flaky tests	0	2
Total Runs	51	51
Cost per result	0.023	1.270
Total Cost	$0.002	$0.153
Input Price	$0.140 / 1M	$1.000 / 1M
Output Price	$0.400 / 1M	$3.200 / 1M
Output Tokens	660	6,666
Reasoning Tokens	0	35,313
Response Time (avg)	2.55s	18.23s
Response Time (max)	4.68s	43.11s
Response Time (total)	38.20s	291.73s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		1.85s	45	0
GLM 5.1	10.0	10.0	100.0%	0		8.31s	401	5,122

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
GLM 5.1	9.5	10.0	100.0%	0		43.11s	327	4,206

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.25s	285	0
GLM 5.1	10.0	10.0	100.0%	0		9.33s	991	4,552

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	7.7	10.0	66.7%	0		3.22s	27	0
GLM 5.1	5.3	10.0	33.3%	0		29.77s	969	11,314

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.09s	117	0
GLM 5.1	10.0	10.0	100.0%	0		20.95s	2,875	2,875

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		2.84s	78	0
GLM 5.1	6.4	5.8	66.7%	1		7.47s	204	1,617

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	5.5	10.0	33.3%	0		2.95s	108	0
GLM 5.1	8.2	7.2	88.9%	1		23.85s	899	5,627

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
GLM 5.1	3.0	10.0	0.0%	0		0ms	0	0

Quick Compare

Switch Comparison Pair

Gemini 3 Flash PreviewnonevsGLM 5.1medium Gemini 3.1 Flash Lite PreviewlowvsGLM 5.1medium Gemma 4 31BnoneFree AvailablevsGPT-5 Minimedium Gemini 3.1 Flash Lite PreviewnonevsGLM 5.1medium Gemma 4 31BnoneFree AvailablevsNemotron 3 SupermediumFree Available GPT-5.2 ChatnonevsGLM 5.1medium Gemma 4 31BnoneFree AvailablevsGrok 4.1 Fastmedium Gemma 4 31BnoneFree AvailablevsHunter Alphamedium Gemma 4 31BnoneFree AvailablevsGPT-5.4 Minimedium Gemma 4 31BnoneFree AvailablevsGrok 4.20medium Gemma 4 31BnoneFree AvailablevsMercury 2medium GPT-5.3 ChatnonevsGLM 5.1medium