AI BENCHY Compare

Google: Gemma 4 31B vs Z.ai: GLM 5

Last updated at: 2026-05-19

Metric	Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02 Free Available	GLM 5 GLM 5 medium Release: 2026-02-12

Metric	Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02 Free Available	GLM 5 GLM 5 medium Release: 2026-02-12
Score	8.2	8.1
Rank	#18	#24
Reliability	6.7	10.0
Consistency	9.6	8.3
Tests Correct
Attempt pass rate	77.2%	80.7%
Flaky tests	1	4
Total Runs	57	57
Cost per result	0.158	1.379
Total Cost	$0.023	$0.180
Input Price	$0.120 / 1M	$0.600 / 1M
Output Price	$0.370 / 1M	$1.920 / 1M
Output Tokens	14,426	20,564
Reasoning Tokens	37,964	70,787
Response Time (avg)	28.72s	27.34s
Response Time (max)	90.14s	79.09s
Response Time (total)	488.27s	300.78s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		12.89s	962	2,046
GLM 5	10.0	10.0	100.0%	0		23.66s	480	7,056

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	4.7	1.6	66.7%	1		70.97s	3,166	5,449
GLM 5	10.0	10.0	100.0%	0		79.09s	330	19,814

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
GLM 5	10.0	10.0	100.0%	0		28.96s	662	3,242

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		21.11s	1,822	2,951
GLM 5	7.1	5.6	83.3%	1		8.90s	567	3,734

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	7.7	10.0	66.7%	0		38.48s	4,349	8,985
GLM 5	3.5	4.4	33.3%	2		0ms	13,176	14,137

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		9.57s	105	888
GLM 5	6.1	3.1	66.7%	1		14.69s	2,020	2,248

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		12.76s	533	2,035
GLM 5	10.0	10.0	100.0%	0		7.25s	1,001	2,129

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	9.9	10.0	100.0%	0		27.63s	1,797	5,596
GLM 5	10.0	10.0	100.0%	0		15.64s	1,694	4,983

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0
GLM 5	10.0	10.0	100.0%	0		15.93s	233	994

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		90.14s	1,692	10,014
GLM 5	3.0	10.0	0.0%	0		67.37s	401	12,450

Quick Compare

Switch Comparison Pair

Gemini 3.5 FlashminimalvsGLM 5medium Gemini 3 Flash PreviewnonevsGLM 5medium Gemini 3.1 Flash Lite PreviewlowvsGLM 5medium Gemini 3.1 Flash Lite PreviewnonevsGLM 5medium GPT-5.2 ChatnonevsGLM 5medium Gemini 3.1 Flash LitelowvsGLM 5medium GPT-5.3 ChatnonevsGLM 5medium DeepSeek V4 FlashhighFree AvailablevsGLM 5medium Gemma 4 31BmediumFree AvailablevsGPT-5.2 Chatnone Gemma 4 31BmediumFree AvailablevsGPT-5.5low Gemma 4 31BmediumFree AvailablevsGPT-5.3 Chatnone DeepSeek V4 FlashhighFree AvailablevsGemma 4 31BmediumFree Available