Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Inception: Mercury 2 vs OpenAI: GPT-5.4

Last updated at: 2026-04-16

Metric Mercury 2 Mercury 2 medium Release: 2026-02-24 GPT-5.4 GPT-5.4 none Release: 2026-03-05
Score 6.5 5.9
Rank #53 #65
Consistency 8.6 9.1
Tests Correct
Attempt pass rate 53.7% 42.6%
Flaky tests 3 2
Total Runs 54 54
Cost per result 0.580 1.477
Total Cost $0.047 $0.104
Input Price $0.250 / 1M $2.500 / 1M
Output Price $0.750 / 1M $15.000 / 1M
Output Tokens 3,972 2,317
Reasoning Tokens 48,333 0
Response Time (avg) 2.21s 1.51s
Response Time (max) 14.63s 2.95s
Response Time (total) 37.51s 27.21s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 6.9 9.9 50.0% 0 1.12s 2,546 2,609
GPT-5.4 3.2 8.0 8.3% 1 1.21s 406 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.53s 249 2,213
GPT-5.4 10.0 10.0 100.0% 0 2.95s 480 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 3.28s 268 4,887
GPT-5.4 3.0 10.0 0.0% 0 2.89s 291 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 7.3 5.9 83.3% 1 1.11s 183 1,656
GPT-5.4 10.0 10.0 100.0% 0 1.04s 222 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 2.9 7.2 11.1% 1 6.48s 41 30,754
GPT-5.4 5.3 7.2 44.4% 1 1.07s 50 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 4.8 10.0 0.0% 0 821ms 137 542
GPT-5.4 4.4 9.9 0.0% 0 1.78s 184 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.07s 14 958
GPT-5.4 6.5 10.0 50.0% 0 1.07s 81 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 3.9 7.5 22.2% 1 934ms 354 2,758
GPT-5.4 5.6 9.8 33.3% 0 1.52s 357 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Mercury 2 10.0 10.0 100.0% 0 1.89s 180 1,956
GPT-5.4 10.0 10.0 100.0% 0 2.75s 246 0

Quick Compare

Switch Comparison Pair

Mercury 2mediumvsMiMo-V2-OmninoneMercury 2mediumvsGLM 5noneMiniMax M2.5mediumFree AvailablevsGPT-5.4noneMistral Small 4mediumvsGPT-5.4noneMercury 2mediumvsQwen3.5 Plus 2026-02-15noneMercury 2mediumvsGLM 5V TurbononeMercury 2mediumvsQwen3.5-FlashnoneGemma 4 26B A4BnoneFree AvailablevsMercury 2mediumSeed-2.0-LitenonevsMercury 2mediumGemini 2.5 FlashnonevsMercury 2mediumMercury 2mediumvsQwen3.5-35B-A3BnoneDeepSeek V3.2nonevsMercury 2medium