AI BENCHY Compare

Google: Gemma 4 26B A4B vs Z.ai: GLM 5.1

Last updated at: 2026-04-07

Metric	Gemma 4 26B A4B Gemma 4 26B A4B none Release: 2026-04-03 Free Available	GLM 5.1 GLM 5.1 medium Release: 2026-04-07

Metric	Gemma 4 26B A4B Gemma 4 26B A4B none Release: 2026-04-03 Free Available	GLM 5.1 GLM 5.1 medium Release: 2026-04-07
Score	6.3	8.0
Rank	#54	#23
Consistency	9.1	9.0
Tests Correct
Attempt pass rate	47.1%	76.5%
Flaky tests	2	2
Total Runs	51	51
Cost per result	0.068	1.270
Total Cost	$0.005	$0.153
Input Price	$0.130 / 1M	$1.000 / 1M
Output Price	$0.400 / 1M	$3.200 / 1M
Output Tokens	1,335	6,666
Reasoning Tokens	0	35,313
Response Time (avg)	6.56s	18.23s
Response Time (max)	57.10s	43.11s
Response Time (total)	111.54s	291.73s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	8.3	10.0	75.0%	0		1.28s	230	0
GLM 5.1	10.0	10.0	100.0%	0		8.31s	401	5,122

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	3.0	10.0	0.0%	0		30.53s	309	0
GLM 5.1	9.5	10.0	100.0%	0		43.11s	327	4,206

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		1.70s	285	0
GLM 5.1	10.0	10.0	100.0%	0		9.33s	991	4,552

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	3.6	7.2	22.2%	1		2.49s	27	0
GLM 5.1	5.3	10.0	33.3%	0		29.77s	969	11,314

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	4.0	10.0	0.0%	0		3.54s	85	0
GLM 5.1	10.0	10.0	100.0%	0		20.95s	2,875	2,875

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	4.4	6.9	16.7%	1		1.08s	75	0
GLM 5.1	6.4	5.8	66.7%	1		7.47s	204	1,617

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	5.7	10.0	33.3%	0		739ms	114	0
GLM 5.1	8.2	7.2	88.9%	1		23.85s	899	5,627

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 26B A4B	10.0	10.0	100.0%	0		57.10s	210	0
GLM 5.1	3.0	10.0	0.0%	0		0ms	0	0

Quick Compare

Switch Comparison Pair

Gemini 3 Flash PreviewnonevsGLM 5.1medium Gemini 3.1 Flash Lite PreviewlowvsGLM 5.1medium Gemma 4 26B A4BnoneFree AvailablevsGPT-5 Nanomedium Gemma 4 26B A4BnoneFree AvailablevsGrok 4.20 Multi Agent Betamedium Gemma 4 26B A4BnoneFree AvailablevsMercury 2medium Gemini 3.1 Flash Lite PreviewnonevsGLM 5.1medium GPT-5.2 ChatnonevsGLM 5.1medium Gemma 4 26B A4BnoneFree Availablevsgpt-oss-120bmediumFree Available Gemma 4 26B A4BnoneFree AvailablevsMiniMax M2.5mediumFree Available GPT-5.3 ChatnonevsGLM 5.1medium Gemma 4 26B A4BnoneFree AvailablevsGPT-5 Minimedium Gemma 4 26B A4BnoneFree AvailablevsNemotron 3 SupermediumFree Available