Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Compared models

Last updated at: 2026-04-20

Metric GLM 5 GLM 5 medium Release: 2026-02-12 GLM 5.1 GLM 5.1 medium Release: 2026-04-07 GLM 5 GLM 5 none Release: 2026-02-12 GLM 5.1 GLM 5.1 none Release: 2026-04-07
Score 8.4 7.8 6.6 5.6
Rank #13 #33 #53 #75
Consistency 8.2 8.6 9.6 8.2
Tests Correct
Attempt pass rate 85.2% 75.9% 51.9% 37.0%
Flaky tests 4 3 1 4
Total Runs 54 54 54 54
Cost per result 1.188 1.674 0.217 1.046
Total Cost $0.155 $0.201 $0.020 $0.053
Input Price $0.700 / 1M $1.050 / 1M $0.700 / 1M $1.050 / 1M
Output Price $0.140 / 1M $3.500 / 1M $0.140 / 1M $3.500 / 1M
Output Tokens 20,163 8,005 1,959 3,720
Reasoning Tokens 58,337 49,090 0 0
Response Time (avg) 23.34s 24.13s 4.23s 4.33s
Response Time (max) 79.09s 118.52s 11.07s 32.57s
Response Time (total) 233.40s 410.25s 46.51s 78.02s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 23.66s 480 7,056
GLM 5.1 10.0 10.0 100.0% 0 8.31s 401 5,122
GLM 5 4.8 10.0 25.0% 0 2.37s 275 0
GLM 5.1 4.0 6.3 25.0% 2 2.11s 305 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 79.09s 330 19,814
GLM 5.1 4.7 1.6 66.7% 1 118.52s 1,339 13,777
GLM 5 5.6 3.5 33.3% 1 8.84s 408 0
GLM 5.1 5.1 9.1 0.0% 0 9.79s 501 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 28.96s 662 3,242
GLM 5.1 9.5 10.0 100.0% 0 43.11s 327 4,206
GLM 5 3.0 10.0 0.0% 0 4.98s 406 0
GLM 5.1 2.8 2.1 33.3% 1 32.57s 2,129 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 7.1 5.6 83.3% 1 8.90s 567 3,734
GLM 5.1 10.0 10.0 100.0% 0 9.33s 991 4,552
GLM 5 10.0 10.0 100.0% 0 5.78s 203 0
GLM 5.1 10.0 10.0 100.0% 0 1.08s 204 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 3.5 4.4 33.3% 2 0ms 13,176 14,137
GLM 5.1 5.3 10.0 33.3% 0 29.77s 969 11,314
GLM 5 3.0 10.0 0.0% 0 2.24s 19 0
GLM 5.1 2.9 7.2 11.1% 1 1.99s 24 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 6.1 3.1 66.7% 1 14.69s 2,020 2,248
GLM 5.1 10.0 10.0 100.0% 0 20.95s 2,875 2,875
GLM 5 10.0 10.0 100.0% 0 3.27s 103 0
GLM 5.1 5.0 10.0 0.0% 0 790ms 39 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 7.25s 1,001 2,129
GLM 5.1 6.4 5.8 66.7% 1 7.47s 204 1,617
GLM 5 10.0 10.0 100.0% 0 1.48s 61 0
GLM 5.1 8.3 10.0 50.0% 0 1.58s 66 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 15.64s 1,694 4,983
GLM 5.1 8.2 7.2 88.9% 1 23.85s 899 5,627
GLM 5 7.7 10.0 66.7% 0 2.05s 264 0
GLM 5.1 5.7 10.0 33.3% 0 1.48s 152 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 15.93s 233 994
GLM 5.1 3.0 10.0 0.0% 0 0ms 0 0
GLM 5 10.0 10.0 100.0% 0 11.07s 220 0
GLM 5.1 10.0 10.0 100.0% 0 10.68s 300 0

Quick Compare

Switch Comparison Pair

Grok 4.1 FastmediumvsGLM 5noneNemotron 3 SupermediumFree AvailablevsGLM 5noneMistral Small 4mediumvsGLM 5.1noneMiniMax M2.5mediumFree AvailablevsGLM 5.1noneGemini 3.1 Flash Lite PreviewnonevsGLM 5.1mediumGPT-5.2 ChatnonevsGLM 5.1mediumGPT-5.3 ChatnonevsGLM 5.1mediumMercury 2mediumvsGLM 5noneGemini 3 Flash PreviewnonevsGLM 5mediumgpt-oss-120bmediumFree AvailablevsGLM 5.1noneGemini 3.1 Flash Lite PreviewlowvsGLM 5mediumMiniMax M2.7mediumvsGLM 5.1none