Navigate
AI BENCHY
AD
Track all your projects in one dashboard. Get 📊stats, 🔥heatmaps and 👀recordings in one self-hosted dashboard.
uxwizz.com

AI BENCHY Compare

Inception: Mercury 2 vs Z.ai: GLM 5.1

Last updated at: 2026-04-07

Metric Mercury 2 Mercury 2 medium Release: 2026-02-24 GLM 5.1 GLM 5.1 none Release: 2026-04-07
Score 6.3 5.6
Rank #53 #67
Consistency 8.5 8.2
Tests Correct
Attempt pass rate 51.0% 39.2%
Flaky tests 3 4
Total Runs 51 51
Cost per result 0.634 1.000
Total Cost $0.045 $0.050
Input Price $0.250 / 1M $1.000 / 1M
Output Price $0.750 / 1M $3.200 / 1M
Output Tokens 3,723 3,219
Reasoning Tokens 46,120 0
Response Time (avg) 2.25s 4.01s
Response Time (max) 14.63s 32.57s
Response Time (total) 35.99s 68.23s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 6.9 9.9 50.0% 0 1.12s 2,546 2,609
GLM 5.1 4.0 6.3 25.0% 2 2.11s 305 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 3.28s 268 4,887
GLM 5.1 2.8 2.1 33.3% 1 32.57s 2,129 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 7.3 5.9 83.3% 1 1.11s 183 1,656
GLM 5.1 10.0 10.0 100.0% 0 1.08s 204 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 2.9 7.2 11.1% 1 6.48s 41 30,754
GLM 5.1 2.9 7.2 11.1% 1 1.99s 24 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 4.8 10.0 0.0% 0 821ms 137 542
GLM 5.1 5.0 10.0 0.0% 0 790ms 39 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.07s 14 958
GLM 5.1 8.3 10.0 50.0% 0 1.58s 66 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 3.9 7.5 22.2% 1 934ms 354 2,758
GLM 5.1 5.7 10.0 33.3% 0 1.48s 152 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.89s 180 1,956
GLM 5.1 10.0 10.0 100.0% 0 10.68s 300 0

Quick Compare

Switch Comparison Pair

Gemma 4 26B A4BnoneFree AvailablevsMercury 2mediumDeepSeek V3.2nonevsMercury 2mediumMistral Small 4mediumvsGLM 5.1noneMercury 2mediumvsMiMo-V2-OmninoneMiniMax M2.5mediumFree AvailablevsGLM 5.1noneMercury 2mediumvsQwen3.5-FlashnoneMercury 2mediumvsGLM 5V TurbononeSeed-2.0-LitenonevsMercury 2mediumgpt-oss-120bmediumFree AvailablevsGLM 5.1noneGemini 2.5 FlashnonevsMercury 2mediumMercury 2mediumvsQwen3.5-35B-A3BnoneMercury 2mediumvsGLM 5none