Navigate
AI BENCHY
Advertise here

Mercury 2 vs Grok 4.20

Mercury 2 leads on average score with 4.7 vs 4.1. Mercury 2 has the lower benchmark cost at $0.030 vs $0.057. Mercury 2 is faster at 825ms vs 1.11s, with pass rates of 24.2% vs 27.3%.

Last updated at: 2026-08-15

Rank
#259
Total Output Tokens
9,584
Response Time (avg)
825ms
Total Cost
$0.030
Rank
#269
Total Output Tokens
1,923
Response Time (avg)
1.11s
Total Cost
$0.057
Recommended model Mercury 2

It has the best score here (4.7), while costing about 1.9x less than Grok 4.20.

Detailed comparison

Metric Mercury 2 Mercury 2 none Release: 2026-02-24 Grok 4.20 Grok 4.20 none Release: 2026-03-31
Score 4.7 4.1
Rank #259 #269
Reliability 10.0 N/A
Consistency 9.2 8.1
Attempts 66/66 54/66
Tests Correct
Attempt pass rate 24.2% 27.3%
Flaky tests 2 0
Total Runs 66 54
Cost per result 0.740 1.570
Total Cost $0.030 $0.057
Input Price $0.250 / 1M $1.250 / 1M
Output Price $0.750 / 1M $2.500 / 1M
Total Input Tokens 89,637 41,313
Output Tokens 9,584 1,923
Reasoning Tokens 0 0
Response Time (avg) 825ms 1.11s
Response Time (max) 4.52s 6.04s
Response Time (total) 18.14s 19.96s
Parameters ~100B ~1T total (~100B active)
Availability Closed Closed

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#259 Mercury 2

none
Cost
$0.002
Time
1.8s
Tokens
1,514 tok

#269 xAI: Grok 4.20

none
Cost
$0.004
Time
6.5s
Tokens
1,367 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 3.4 9.6 0.0% 0 1.03s 7,229 3,088 0
Grok 4.20 1.1 3.1 0.0% 0 1.22s 1,074 312 0

Quick Compare

Switch Comparison Pair

CobuddymediumvsMercury 2noneMercury 2nonevsQwen3 Coder NextmediumMercury 2nonevsMiniMax M2.5mediumTrinity Large ThinkinghighvsMercury 2noneMercury 2nonevsNemotron 3.5 LightninglowFree AvailableGrok 4.20nonevsGLM 4.7 FlashmediumLing 3.0 TinyhighvsGrok 4.20noneLing 3.0 TinylowvsGrok 4.20noneQwen3.5-9BmediumvsGrok 4.20noneLing 3.0 TinymediumvsGrok 4.20noneMercury 2nonevsLaguna S 2.1lowFree AvailableMercury 2nonevsGLM 4.7 Flashmedium