Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Compared models

Last updated at: 2026-05-26

Metric GLM 5 GLM 5 medium Release: 2026-02-12 GLM 5.1 GLM 5.1 medium Release: 2026-04-07 Kimi K2.5 Kimi K2.5 medium Release: 2026-01-27 Qwen3.6 Plus Preview Qwen3.6 Plus Preview medium Release: 2026-04-20 Free Available
Score 8.2 7.4 6.7 8.2
Rank #18 #54 #79 #17
Reliability 10.0 3.3 10.0 N/A
Consistency 8.4 8.3 6.8 10.0
Tests Correct
Attempt pass rate 81.7% 71.7% 66.7% 75.0%
Flaky tests 4 4 8 0
Total Runs 98 98 98 57
Cost per result 1.818 2.890 4.008 0.000
Total Cost $0.255 $0.347 $0.361 $0.000
Input Price $0.600 / 1M $0.980 / 1M $0.400 / 1M $0.000 / 1M
Output Price $1.920 / 1M $3.080 / 1M $1.900 / 1M $0.000 / 1M
Output Tokens 21,596 13,107 70,680 1,153
Reasoning Tokens 105,424 88,661 150,779 62,197
Response Time (avg) 32.67s 33.45s 89.02s 15.25s
Response Time (max) 99.85s 172.60s 281.00s 43.55s
Response Time (total) 392.01s 635.63s 1157.32s 182.96s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 23.66s 480 7,056
GLM 5.1 10.0 10.0 100.0% 0 8.31s 401 5,122
Kimi K2.5 7.3 5.8 83.3% 2 51.38s 2,789 8,880
Qwen3.6 Plus Preview 10.0 10.0 100.0% 0 11.69s 61 5,812
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 89.47s 2,985 45,706
GLM 5.1 4.7 1.6 66.7% 2 145.56s 4,727 34,384
Kimi K2.5 4.1 1.9 50.0% 2 215.89s 5,700 45,419
Qwen3.6 Plus Preview 0.0 0.0 0.0% 0 0ms 0 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 28.96s 662 3,242
GLM 5.1 9.5 10.0 100.0% 0 43.11s 327 4,206
Kimi K2.5 10.0 10.0 100.0% 0 71.37s 703 3,713
Qwen3.6 Plus Preview 10.0 10.0 100.0% 0 34.95s 452 13,073
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 7.1 5.6 83.3% 1 8.90s 567 3,734
GLM 5.1 10.0 10.0 100.0% 0 9.33s 991 4,552
Kimi K2.5 10.0 10.0 100.0% 0 49.78s 563 7,940
Qwen3.6 Plus Preview 10.0 10.0 100.0% 0 14.95s 270 10,706
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 3.5 4.4 33.3% 2 0ms 13,176 14,137
GLM 5.1 5.3 10.0 33.3% 0 29.77s 969 11,314
Kimi K2.5 3.5 4.4 33.3% 2 137.29s 20,753 30,564
Qwen3.6 Plus Preview 3.0 10.0 0.0% 0 22.08s 49 26,895
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 6.1 3.1 66.7% 1 14.69s 2,020 2,248
GLM 5.1 10.0 10.0 100.0% 0 20.95s 2,875 2,875
Kimi K2.5 6.5 3.4 66.7% 1 69.73s 3,815 4,262
Qwen3.6 Plus Preview 0.0 0.0 0.0% 0 0ms 0 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 7.25s 1,001 2,129
GLM 5.1 6.4 5.8 66.7% 1 7.47s 204 1,617
Kimi K2.5 10.0 10.0 100.0% 0 92.47s 5,371 6,547
Qwen3.6 Plus Preview 10.0 10.0 100.0% 0 3.40s 27 1,383
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 11.33s 71 13,728
GLM 5.1 8.2 7.2 88.9% 1 31.64s 2,531 22,412
Kimi K2.5 5.3 7.3 44.4% 1 43.23s 30,732 34,998
Qwen3.6 Plus Preview 10.0 10.0 100.0% 0 7.52s 27 2,998
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 10.0 10.0 100.0% 0 15.93s 233 994
GLM 5.1 3.0 10.0 0.0% 0 0ms 0 0
Kimi K2.5 10.0 10.0 100.0% 0 31.74s 242 812
Qwen3.6 Plus Preview 10.0 10.0 100.0% 0 5.87s 267 1,330
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GLM 5 3.0 10.0 0.0% 0 67.37s 401 12,450
GLM 5.1 3.0 10.0 0.0% 0 29.40s 82 2,179
Kimi K2.5 3.0 10.0 0.0% 0 83.95s 12 7,644
Qwen3.6 Plus Preview 0.0 0.0 0.0% 0 0ms 0 0

Quick Compare

Switch Comparison Pair

Gemini 3.1 Flash LiteminimalvsKimi K2.5mediumGemini 3.1 Flash LitelowvsGLM 5.1mediumGPT-5.3 ChatnonevsGLM 5.1mediumGemma 4 31BnoneFree AvailablevsKimi K2.5mediumGemini 3.1 Flash Lite PreviewnonevsGLM 5.1mediumKimi K2.5mediumvsGrok Build 0.1noneDeepSeek V4 ProhighvsKimi K2.5mediumGemini 3.1 Flash LitenonevsKimi K2.5mediumKimi K2.5mediumvsGPT-5.5noneRing-2.6-1TnonevsGLM 5.1mediumGemini 3.1 Flash Lite PreviewlowvsGLM 5.1mediumDeepSeek V4 FlashhighFree AvailablevsGLM 5.1medium