AI BENCHY Compare

Inception: Mercury 2 vs MiniMax: MiniMax M2.5

Last updated at: 2026-06-03

Metric	Mercury 2 Mercury 2 none Release: 2026-02-24	MiniMax M2.5 MiniMax M2.5 medium Release: 2026-02-12

Metric	Mercury 2 Mercury 2 none Release: 2026-02-24	MiniMax M2.5 MiniMax M2.5 medium Release: 2026-02-12
Score	4.6	5.5
Rank	#153	#125
Reliability	10.0	10.0
Consistency	9.1	6.4
Tests Correct
Attempt pass rate	25.0%	48.3%
Flaky tests	2	9
Total Runs	60	60
Cost per result	0.216	6.075
Total Cost	$0.009	$0.294
Input Price	$0.250 / 1M	$0.150 / 1M
Output Price	$0.750 / 1M	$1.150 / 1M
Total Input Tokens	25,515	41,948
Output Tokens	3,001	109,492
Reasoning Tokens	0	251,674
Response Time (avg)	614ms	49.87s
Response Time (max)	1.27s	237.27s
Response Time (total)	12.28s	598.39s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		483ms	631	286	0
MiniMax M2.5	7.9	6.3	83.3%	2		20.82s	612	286	45,344

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.5	9.4	0.0%	0		831ms	4,631	1,650	0
MiniMax M2.5	3.5	9.8	0.0%	0		125.80s	4,318	354	27,037

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		606ms	4,821	131	0
MiniMax M2.5	4.5	2.1	66.7%	1		60.39s	21,104	740	9,713

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	7.3	5.9	83.3%	1		667ms	6,362	180	0
MiniMax M2.5	4.6	1.7	66.7%	2		7.48s	6,584	266	3,835

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	5.3	7.2	44.4%	1		534ms	784	46	0
MiniMax M2.5	2.9	4.4	22.2%	2		237.27s	308	105,047	133,487

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	4.8	10.0	0.0%	0		628ms	495	159	0
MiniMax M2.5	3.8	2.5	33.3%	1		6.63s	492	25	1,686

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	6.5	10.0	50.0%	0		551ms	691	82	0
MiniMax M2.5	7.5	10.0	50.0%	0		621ms	699	156	1,495

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.1	10.0	0.0%	0		535ms	694	251	0
MiniMax M2.5	5.3	7.2	44.4%	1		11.21s	495	1,069	9,605

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	10.0	10.0	100.0%	0		1.27s	6,193	197	0
MiniMax M2.5	10.0	10.0	100.0%	0		15.35s	7,123	269	937

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		548ms	213	19	0
MiniMax M2.5	3.0	10.0	0.0%	0		80.79s	213	1,280	18,535

Quick Compare

Switch Comparison Pair

MiniMax M2.5mediumvsgpt-oss-120bnoneFree Available MiniMax M2.5mediumvsGrok 4.20none MiniMax M2.5mediumvsQwen3.6 Flashnone DeepSeek V3.2nonevsMiniMax M2.5medium MiniMax M2.5mediumvsKimi K2.6noneFree Available MiniMax M2.5mediumvsQwen3.5-122B-A10Bnone MiniMax M2.5mediumvsMiMo-V2.5-Pronone MiniMax M2.5mediumvsGPT-5.4none Mercury 2nonevsQwen3 Coder Nextmedium Mercury 2nonevsGLM 4.7 Flashmedium MiniMax M2.5mediumvsGLM 5 Turbonone MiniMax M2.5mediumvsKimi K2.5none