Inception: Mercury 2 vs xAI: Grok 4.20

Grok 4.20 (medium) leads on average score with 7.1 vs 7.0. Mercury 2 (medium) has the lower benchmark cost at $0.093 vs $0.777. Mercury 2 (medium) is faster at 2.72s vs 29.47s, with pass rates of 51.5% vs 63.6%.

Recommended modelMercury 2 (medium)Its score stays close to the best score here (7.0 vs 7.1), while costing about 8.4x less than Grok 4.20 (medium).

Last updated at: 2026-07-18

Metric	Mercury 2 Mercury 2 medium Release: 2026-02-24	Grok 4.20 Grok 4.20 medium Release: 2026-03-31

Metric	Mercury 2 Mercury 2 medium Release: 2026-02-24	Grok 4.20 Grok 4.20 medium Release: 2026-03-31
Score	7.0	7.1
Rank	#77	#74
Reliability	10.0	10.0
Consistency	8.8	8.5
Tests Correct
Attempt pass rate	51.5%	63.6%
Flaky tests	3	4
Total Runs	66	66
Cost per result	0.928	9.709
Total Cost	$0.093	$0.777
Input Price	$0.250 / 1M	$1.250 / 1M
Output Price	$0.750 / 1M	$2.500 / 1M
Total Input Tokens	109,572	102,791
Output Tokens	10,313	5,363
Reasoning Tokens	76,806	253,977
Response Time (avg)	2.72s	29.47s
Response Time (max)	14.63s	199.66s
Response Time (total)	57.12s	648.35s

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#77 Mercury 2

medium

Cost: $0.002
Time: 2.1s
Tokens: 1,702 tok

#74 xAI: Grok 4.20

medium

Cost: $0.041
Time: 110.3s
Tokens: 16,336 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	6.9	9.9	50.0%	0		1.12s	554	2,546	2,609
Grok 4.20	8.2	7.9	83.3%	1		3.95s	2,010	287	8,312

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	8.2	7.7	77.8%	1		2.04s	7,065	296	11,328
Grok 4.20	6.3	6.6	55.6%	1		109.93s	8,307	268	103,150

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	6.7	9.1	50.0%	0		7.84s	87,365	6,533	20,474
Grok 4.20	8.7	6.9	83.3%	1		42.25s	71,267	3,776	44,009

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	7.3	5.9	83.3%	1		1.11s	6,234	183	1,656
Grok 4.20	10.0	10.0	100.0%	0		4.17s	7,761	180	5,333

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	2.9	7.2	11.1%	1		6.48s	695	41	30,754
Grok 4.20	5.3	10.0	33.3%	0		27.03s	1,764	375	49,339

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	4.8	10.0	0.0%	0		821ms	456	137	542
Grok 4.20	3.9	2.6	33.3%	1		24.48s	825	65	6,440

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	10.0	10.0	100.0%	0		1.07s	340	14	958
Grok 4.20	9.8	10.0	100.0%	0		4.26s	1,362	57	6,419

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	5.4	10.0	33.3%	0		949ms	601	361	2,781
Grok 4.20	7.7	10.0	66.7%	0		6.22s	1,689	149	7,913

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	10.0	10.0	100.0%	0		1.89s	6,080	180	1,956
Grok 4.20	3.0	10.0	0.0%	0		13.68s	7,275	197	6,620

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		2.58s	182	22	3,748
Grok 4.20	3.0	10.0	0.0%	0		63.48s	531	9	16,442

Quick Compare

Switch Comparison Pair

Gemini 3.5 FlashnonevsMercury 2medium Qwen3.7 PlusnonevsGrok 4.20medium DeepSeek V4 PrononevsMercury 2medium Mercury 2mediumvsGPT-5.6 Solnone KAT-Coder-Pro V2.5highvsGrok 4.20medium Gemini 3.5 FlashnonevsGrok 4.20medium Mercury 2mediumvsStep 3.7 Flashhigh Step 3.7 FlashlowvsGrok 4.20medium Mercury 2mediumvsGPT-5.5none Claude Opus 4.8nonevsGrok 4.20medium Mercury 2mediumvsQwen3.7 Plusnone Gemini 3.5 FlashminimalvsMercury 2medium