Inception: Mercury 2 vs Z.ai: GLM 5.1

GLM 5.1 (medium) leads on average score with 7.1 vs 7.0. Mercury 2 (medium) has the lower benchmark cost at $0.093 vs $0.535. Mercury 2 (medium) is faster at 2.72s vs 46.77s, with pass rates of 51.5% vs 69.7%.

Recommended modelMercury 2 (medium)Its score stays close to the best score here (7.0 vs 7.1), while costing about 5.8x less than GLM 5.1 (medium).

Last updated at: 2026-07-18

Metric	Mercury 2 Mercury 2 medium Release: 2026-02-24	GLM 5.1 GLM 5.1 medium Release: 2026-04-07

Metric	Mercury 2 Mercury 2 medium Release: 2026-02-24	GLM 5.1 GLM 5.1 medium Release: 2026-04-07
Score	7.0	7.1
Rank	#77	#73
Reliability	10.0	8.3
Consistency	8.8	8.4
Tests Correct
Attempt pass rate	51.5%	69.7%
Flaky tests	3	4
Total Runs	66	66
Cost per result	0.928	4.202
Total Cost	$0.093	$0.535
Input Price	$0.250 / 1M	$0.966 / 1M
Output Price	$0.750 / 1M	$3.036 / 1M
Total Input Tokens	109,572	82,623
Output Tokens	10,313	16,089
Reasoning Tokens	76,806	136,463
Response Time (avg)	2.72s	46.77s
Response Time (max)	14.63s	308.75s
Response Time (total)	57.12s	982.16s

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#77 Mercury 2

medium

Cost: $0.002
Time: 2.1s
Tokens: 1,702 tok

#73 GLM 5.1

medium

Invalid SVG

Cost: $0.000
Time: 300.0s
Tokens: 0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	6.9	9.9	50.0%	0		1.12s	554	2,546	2,609
GLM 5.1	10.0	10.0	100.0%	0		8.31s	555	401	5,122

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	8.2	7.7	77.8%	1		2.04s	7,065	296	11,328
GLM 5.1	4.6	3.7	44.5%	2		109.63s	5,702	4,871	37,826

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	6.7	9.1	50.0%	0		7.84s	87,365	6,533	20,474
GLM 5.1	9.8	10.0	100.0%	0		175.93s	66,926	4,761	65,248

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	7.3	5.9	83.3%	1		1.11s	6,234	183	1,656
GLM 5.1	10.0	10.0	100.0%	0		9.33s	7,107	991	4,552

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	2.9	7.2	11.1%	1		6.48s	695	41	30,754
GLM 5.1	5.3	10.0	33.3%	0		29.77s	489	969	11,314

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	4.8	10.0	0.0%	0		821ms	456	137	542
GLM 5.1	10.0	10.0	100.0%	0		20.95s	477	2,875	2,875

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	10.0	10.0	100.0%	0		1.07s	340	14	958
GLM 5.1	6.4	5.8	66.7%	1		7.47s	634	204	1,617

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	5.4	10.0	33.3%	0		949ms	601	361	2,781
GLM 5.1	8.2	7.2	88.9%	1		31.64s	609	935	5,730

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	10.0	10.0	100.0%	0		1.89s	6,080	180	1,956
GLM 5.1	3.0	10.0	0.0%	0		0ms	0	0	0

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		2.58s	182	22	3,748
GLM 5.1	3.0	10.0	0.0%	0		29.40s	124	82	2,179

Quick Compare

Switch Comparison Pair

Gemini 3.5 FlashnonevsMercury 2medium Qwen3.7 PlusnonevsGLM 5.1medium DeepSeek V4 PrononevsMercury 2medium Mercury 2mediumvsGPT-5.6 Solnone KAT-Coder-Pro V2.5highvsGLM 5.1medium Step 3.7 FlashlowvsGLM 5.1medium Mercury 2mediumvsStep 3.7 Flashhigh Mercury 2mediumvsGPT-5.5none Gemini 3.5 FlashnonevsGLM 5.1medium Claude Opus 4.8nonevsGLM 5.1medium Mercury 2mediumvsQwen3.7 Plusnone Gemini 3.5 FlashminimalvsMercury 2medium