Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Inception: Mercury 2 vs xAI: Grok 4.20

Last updated at: 2026-04-02

Metric Mercury 2 Mercury 2 medium Release: 2026-02-24 Grok 4.20 Grok 4.20 none Release: 2026-03-31
Score 6.3 5.4
Rank #51 #69
Consistency 8.5 9.5
Tests Correct
Attempt pass rate 51.0% 31.4%
Flaky tests 3 1
Total Runs 51 51
Cost per result 0.634 1.809
Total Cost $0.045 $0.091
Input Price $0.250 / 1M $2.000 / 1M
Output Price $0.750 / 1M $6.000 / 1M
Output Tokens 3,723 1,655
Reasoning Tokens 46,120 0
Response Time (avg) 2.25s 1.11s
Response Time (max) 14.63s 6.04s
Response Time (total) 35.99s 18.80s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 6.9 9.9 50.0% 0 1.12s 2,546 2,609
Grok 4.20 4.8 10.0 25.0% 0 501ms 267 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 3.28s 268 4,887
Grok 4.20 3.0 10.0 0.0% 0 6.04s 282 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 7.3 5.9 83.3% 1 1.11s 183 1,656
Grok 4.20 10.0 10.0 100.0% 0 522ms 207 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 2.9 7.2 11.1% 1 6.48s 41 30,754
Grok 4.20 3.0 10.0 0.0% 0 687ms 325 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 4.8 10.0 0.0% 0 821ms 137 542
Grok 4.20 4.8 10.0 0.0% 0 659ms 83 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.07s 14 958
Grok 4.20 4.8 10.0 0.0% 0 455ms 60 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 3.9 7.5 22.2% 1 934ms 354 2,758
Grok 4.20 5.3 7.4 44.4% 1 487ms 242 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.89s 180 1,956
Grok 4.20 10.0 10.0 100.0% 0 4.63s 189 0

Quick Compare

Switch Comparison Pair

DeepSeek V3.2nonevsMercury 2mediumMercury 2mediumvsMiMo-V2-OmninoneMistral Small 4mediumvsGrok 4.20noneMercury 2mediumvsQwen3.5-FlashnoneMercury 2mediumvsGLM 5V TurbononeSeed-2.0-LitenonevsMercury 2mediumMiniMax M2.7mediumvsGrok 4.20noneGemini 2.5 FlashnonevsMercury 2mediumMercury 2mediumvsQwen3.5-35B-A3BnoneMercury 2mediumvsGLM 5noneGemma 4 31BnonevsMercury 2mediumMercury 2mediumvsHunter Alphanone