Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Inception: Mercury 2 vs xAI: Grok 4.20

Last updated at: 2026-04-16

Metric Mercury 2 Mercury 2 none Release: 2026-02-24 Grok 4.20 Grok 4.20 none Release: 2026-03-31
Score 4.8 5.2
Rank #89 #80
Consistency 9.0 9.5
Tests Correct
Attempt pass rate 27.8% 29.6%
Flaky tests 2 1
Total Runs 54 54
Cost per result 0.165 1.889
Total Cost $0.007 $0.095
Input Price $0.250 / 1M $2.000 / 1M
Output Price $0.750 / 1M $6.000 / 1M
Output Tokens 1,625 1,967
Reasoning Tokens 0 0
Response Time (avg) 613ms 1.11s
Response Time (max) 1.27s 6.04s
Response Time (total) 11.04s 20.02s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 3.0 10.0 0.0% 0 483ms 286 0
Grok 4.20 4.8 10.0 25.0% 0 501ms 267 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 3.6 8.9 0.0% 0 969ms 310 0
Grok 4.20 3.4 9.3 0.0% 0 1.22s 312 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 3.0 10.0 0.0% 0 606ms 131 0
Grok 4.20 3.0 10.0 0.0% 0 6.04s 282 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 7.3 5.9 83.3% 1 667ms 180 0
Grok 4.20 10.0 10.0 100.0% 0 522ms 207 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 5.3 7.2 44.4% 1 534ms 46 0
Grok 4.20 3.0 10.0 0.0% 0 687ms 325 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 4.8 10.0 0.0% 0 628ms 159 0
Grok 4.20 4.8 10.0 0.0% 0 659ms 83 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 6.5 10.0 50.0% 0 551ms 82 0
Grok 4.20 4.8 10.0 0.0% 0 455ms 60 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 3.1 10.0 0.0% 0 533ms 234 0
Grok 4.20 5.3 7.4 44.4% 1 487ms 242 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.27s 197 0
Grok 4.20 10.0 10.0 100.0% 0 4.63s 189 0

Quick Compare

Switch Comparison Pair

ElephantmediumvsGrok 4.20noneMiniMax M2.7mediumvsGrok 4.20noneMercury 2nonevsQwen3 Coder NextmediumMercury 2nonevsGLM 4.7 FlashmediumMercury 2nonevsQwen3.5-9BmediumMistral Small 4mediumvsGrok 4.20noneMercury 2nonevsElephantmediumMiniMax M2.5mediumFree AvailablevsGrok 4.20noneMercury 2nonevsMiniMax M2.7mediumQwen3 Coder NextmediumvsGrok 4.20noneGrok 4.20nonevsGLM 4.7 Flashmediumgpt-oss-120bmediumFree AvailablevsGrok 4.20none