Navigate
AI BENCHY
AD
Track all your projects in one dashboard. Get 📊stats, 🔥heatmaps and 👀recordings in one self-hosted dashboard.
uxwizz.com

AI BENCHY Compare

Inception: Mercury 2 vs OpenAI: GPT-5.4

Summary

Mercury 2 vs GPT-5.4 benchmark comparison: GPT-5.4 leads on average score with 5.8 vs 4.6. Mercury 2 has the lower benchmark cost at $0.011 vs $0.122. Mercury 2 is faster at 653ms vs 1.42s, with pass rates of 23.8% vs 36.5%.

Recommended model: Mercury 2 - It offers the best overall trade-off: a competitive score (4.6), lower cost than GPT-5.4, and balanced response time.

Last updated at: 2026-06-18

Metric Mercury 2 Mercury 2 none Release: 2026-02-24 GPT-5.4 GPT-5.4 none Release: 2026-03-05
Score 4.6 5.8
Rank #151 #112
Reliability 10.0 10.0
Consistency 9.2 9.2
Tests Correct
Attempt pass rate 23.8% 36.5%
Flaky tests 2 2
Total Runs 63 63
Cost per result 0.259 1.740
Total Cost $0.011 $0.122
Input Price $0.250 / 1M $2.500 / 1M
Output Price $0.750 / 1M $15.000 / 1M
Total Input Tokens 28,113 34,212
Output Tokens 4,439 2,417
Reasoning Tokens 0 0
Response Time (avg) 653ms 1.42s
Response Time (max) 1.43s 2.95s
Response Time (total) 13.72s 29.87s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#151 Mercury 2

none
Cost
$0.002
Time
1.8s
Tokens
1,514 tok

#112 GPT-5.4

none
Cost
$0.026
Time
18.1s
Tokens
1,792 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 3.0 10.0 0.0% 0 483ms 631 286 0
GPT-5.4 3.2 8.0 8.3% 1 1.21s 606 406 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 3.4 9.6 0.0% 0 1.03s 7,229 3,088 0
GPT-5.4 5.5 10.0 33.3% 0 1.62s 7,305 516 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 3.0 10.0 0.0% 0 606ms 4,821 131 0
GPT-5.4 3.0 10.0 0.0% 0 2.89s 11,019 291 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 7.3 5.9 83.3% 1 667ms 6,362 180 0
GPT-5.4 10.0 10.0 100.0% 0 1.04s 7,140 222 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 5.3 7.2 44.4% 1 534ms 784 46 0
GPT-5.4 5.3 7.2 44.4% 1 1.07s 723 50 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 4.8 10.0 0.0% 0 628ms 495 159 0
GPT-5.4 4.4 9.9 0.0% 0 1.78s 477 184 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 6.5 10.0 50.0% 0 551ms 691 82 0
GPT-5.4 6.5 10.0 50.0% 0 1.07s 660 81 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 3.1 10.0 0.0% 0 535ms 694 251 0
GPT-5.4 5.6 9.8 33.3% 0 1.44s 642 381 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.27s 6,193 197 0
GPT-5.4 10.0 10.0 100.0% 0 2.75s 5,445 246 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 3.0 10.0 0.0% 0 548ms 213 19 0
GPT-5.4 3.0 10.0 0.0% 0 990ms 195 40 0

Quick Compare

Switch Comparison Pair

Mercury 2nonevsQwen3 Coder NextmediumNorth Mini CodemediumFree AvailablevsGPT-5.4noneMercury 2nonevsMiniMax M2.5mediumCobuddymediumvsMercury 2noneMercury 2nonevsGLM 4.7 FlashmediumGemini 3.1 Flash LiteminimalvsGPT-5.4noneMercury 2nonevsMistral Small 4mediumGPT-5.4nonevsQwen3.5-35B-A3BmediumGemma 4 31BmediumFree AvailablevsGPT-5.4noneNemotron 3 SupermediumFree AvailablevsGPT-5.4noneMercury 2nonevsMiniMax M2.7mediumMiniMax M2.7mediumvsGPT-5.4none