Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Mistral: Mistral Small 4 vs Z.ai: GLM 5.1

Last updated at: 2026-04-29

Metric Mistral Small 4 Mistral Small 4 none Release: 2026-03-16 GLM 5.1 GLM 5.1 none Release: 2026-04-07
Score 5.2 5.6
Rank #109 #98
Reliability N/A N/A
Consistency 9.5 8.2
Tests Correct
Attempt pass rate 31.5% 37.0%
Flaky tests 1 4
Total Runs 54 54
Cost per result 0.118 1.046
Total Cost $0.006 $0.053
Input Price $0.150 / 1M $1.050 / 1M
Output Price $0.600 / 1M $3.500 / 1M
Output Tokens 2,207 3,720
Reasoning Tokens 0 0
Response Time (avg) 665ms 4.33s
Response Time (max) 1.72s 32.57s
Response Time (total) 11.97s 78.02s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 3.4 7.9 16.7% 1 395ms 182 0
GLM 5.1 4.0 6.3 25.0% 2 2.11s 305 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 4.5 9.0 0.0% 0 1.28s 583 0
GLM 5.1 5.1 9.1 0.0% 0 9.79s 501 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 3.0 10.0 0.0% 0 1.72s 496 0
GLM 5.1 2.8 2.1 33.3% 1 32.57s 2,129 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 10.0 10.0 100.0% 0 822ms 261 0
GLM 5.1 10.0 10.0 100.0% 0 1.08s 204 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 5.3 10.0 33.3% 0 367ms 28 0
GLM 5.1 2.9 7.2 11.1% 1 1.99s 24 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 4.0 10.0 0.0% 0 729ms 205 0
GLM 5.1 5.0 10.0 0.0% 0 790ms 39 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 6.5 10.0 50.0% 0 380ms 69 0
GLM 5.1 8.3 10.0 50.0% 0 1.58s 66 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 3.1 9.9 0.0% 0 589ms 170 0
GLM 5.1 5.7 10.0 33.3% 0 1.48s 152 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mistral Small 4 10.0 10.0 100.0% 0 1.40s 213 0
GLM 5.1 10.0 10.0 100.0% 0 10.68s 300 0

Quick Compare

Switch Comparison Pair

Mistral Small 4nonevsNemotron 3 Nano Omni 30b A3b ReasoningmediumFree AvailableMistral Small 4nonevsElephant AlphamediumMiniMax M2.7mediumvsMistral Small 4noneMistral Small 4mediumvsGLM 5.1noneMiniMax M2.5mediumFree AvailablevsGLM 5.1nonegpt-oss-120bmediumFree AvailablevsGLM 5.1noneMiniMax M2.7mediumvsGLM 5.1noneElephant AlphamediumvsGLM 5.1noneNemotron 3 Nano Omni 30b A3b ReasoningmediumFree AvailablevsGLM 5.1noneMiniMax M2.5mediumFree AvailablevsMistral Small 4noneMistral Small 4nonevsQwen3 Coder NextmediumLaguna Xs.2mediumFree AvailablevsGLM 5.1none