AI BENCHY Compare

Mistral: Mistral Small 4 vs Z.ai: GLM 5.1

Last updated at: 2026-05-22

Metric	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16	GLM 5.1 GLM 5.1 none Release: 2026-04-07

Metric	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16	GLM 5.1 GLM 5.1 none Release: 2026-04-07
Score	5.4	5.6
Rank	#121	#113
Reliability	10.0	10.0
Consistency	7.1	8.1
Tests Correct
Attempt pass rate	45.0%	40.0%
Flaky tests	7	5
Total Runs	60	60
Cost per result	1.109	0.941
Total Cost	$0.056	$0.057
Input Price	$0.150 / 1M	$0.980 / 1M
Output Price	$0.600 / 1M	$3.080 / 1M
Output Tokens	21,628	3,749
Reasoning Tokens	68,055	0
Response Time (avg)	8.33s	4.16s
Response Time (max)	59.15s	32.57s
Response Time (total)	166.59s	83.23s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	5.6	3.8	66.7%	3		2.67s	4,055	4,778
GLM 5.1	4.0	6.3	25.0%	2		2.11s	305	0

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	5.1	6.8	33.3%	1		44.82s	9,322	38,386
GLM 5.1	4.3	9.5	0.0%	0		6.33s	519	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.0	10.0	0.0%	0		25.25s	2,612	10,700
GLM 5.1	2.8	2.1	33.3%	1		32.57s	2,129	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	7.3	5.9	83.3%	1		1.23s	335	723
GLM 5.1	10.0	10.0	100.0%	0		1.08s	204	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	5.3	7.2	44.4%	1		6.11s	2,621	6,904
GLM 5.1	2.9	7.2	11.1%	1		1.99s	24	0

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	4.8	10.0	0.0%	0		2.05s	821	828
GLM 5.1	5.0	10.0	0.0%	0		790ms	39	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	7.3	5.8	83.3%	1		1.38s	540	1,031
GLM 5.1	9.8	10.0	100.0%	0		1.58s	66	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.4	9.7	0.0%	0		2.00s	983	2,338
GLM 5.1	6.1	7.8	44.4%	1		1.48s	152	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	10.0	10.0	100.0%	0		3.50s	321	810
GLM 5.1	10.0	10.0	100.0%	0		10.68s	300	0

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.0	10.0	0.0%	0		5.92s	18	1,557
GLM 5.1	3.0	10.0	0.0%	0		2.34s	11	0

Quick Compare

Switch Comparison Pair

Mistral Small 4mediumvsGrok 4.20none gpt-oss-120bmediumFree AvailablevsGLM 5.1none Mistral Small 4mediumvsQwen3.5-122B-A10Bnone Mistral Small 4mediumvsGLM 5 Turbonone Mistral Small 4mediumvsKimi K2.5none Ling-2.6-flashnonevsMistral Small 4medium Mistral Small 4mediumvsQwen3.6 Flashnone MiniMax M2.5mediumFree AvailablevsGLM 5.1none Mistral Small 4mediumvsMiMo-V2.5-Pronone DeepSeek V3.2nonevsMistral Small 4medium CobuddymediumFree AvailablevsGLM 5.1none Mistral Small 4mediumvsGPT-5.4none