Inception: Mercury 2 vs Mistral: Mistral Small 4

Mistral Small 4 (medium) leads on average score with 5.1 vs 4.6. Mercury 2 has the lower benchmark cost at $0.030 vs $0.096. Mercury 2 is faster at 829ms vs 10.77s, with pass rates of 22.7% vs 42.4%.

Recommended modelMercury 2Its score stays close to the best score here (4.6 vs 5.1), while costing about 3.3x less than Mistral Small 4 (medium).

Last updated at: 2026-07-18

Metric	Mercury 2 Mercury 2 none Release: 2026-02-24	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16

Metric	Mercury 2 Mercury 2 none Release: 2026-02-24	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16
Score	4.6	5.1
Rank	#185	#163
Reliability	10.0	10.0
Consistency	9.2	7.0
Tests Correct
Attempt pass rate	22.7%	42.4%
Flaky tests	2	8
Total Runs	66	66
Cost per result	0.734	1.913
Total Cost	$0.030	$0.096
Input Price	$0.250 / 1M	$0.150 / 1M
Output Price	$0.750 / 1M	$0.600 / 1M
Total Input Tokens	88,704	140,494
Output Tokens	9,564	39,462
Reasoning Tokens	0	92,362
Response Time (avg)	829ms	10.77s
Response Time (max)	4.52s	59.15s
Response Time (total)	18.24s	236.94s

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#185 Mercury 2

none

Cost: $0.002
Time: 1.8s
Tokens: 1,514 tok

#163 Mistral Small 4

medium

Cost: $0.006
Time: 47.9s
Tokens: 9,857 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		483ms	631	286	0
Mistral Small 4	5.6	3.8	66.7%	3		2.67s	708	4,055	4,778

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.4	9.6	0.0%	0		1.03s	7,229	3,088	0
Mistral Small 4	4.4	5.1	33.3%	2		39.98s	7,636	11,635	54,715

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		2.56s	65,412	5,256	0
Mistral Small 4	3.0	10.0	0.0%	0		32.40s	116,624	17,890	18,384

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	7.3	5.9	83.3%	1		667ms	6,362	180	0
Mistral Small 4	7.3	5.9	83.3%	1		1.23s	6,171	335	723

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	5.3	7.2	44.4%	1		534ms	784	46	0
Mistral Small 4	5.3	7.2	44.4%	1		6.11s	742	2,621	6,904

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	4.8	10.0	0.0%	0		628ms	495	159	0
Mistral Small 4	4.8	10.0	0.0%	0		2.05s	519	821	828

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	6.5	10.0	50.0%	0		551ms	691	82	0
Mistral Small 4	7.3	5.8	83.3%	1		1.38s	729	540	1,031

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.1	10.0	0.0%	0		535ms	694	251	0
Mistral Small 4	3.4	9.7	0.0%	0		2.17s	735	1,226	2,632

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	10.0	10.0	100.0%	0		1.27s	6,193	197	0
Mistral Small 4	10.0	10.0	100.0%	0		3.50s	6,420	321	810

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Mercury 2	3.0	10.0	0.0%	0		548ms	213	19	0
Mistral Small 4	3.0	10.0	0.0%	0		5.92s	210	18	1,557

Quick Compare

Switch Comparison Pair

Mistral Small 4mediumvsMiMo-V2.5none Mistral Small 4mediumvsQwen3.5-9Bnone Mistral Small 4mediumvsQwen3 Coder Nextnone North Mini CodenoneFree AvailablevsMistral Small 4medium CobuddymediumvsMercury 2none Mercury 2nonevsQwen3 Coder Nextmedium Mercury 2nonevsMiniMax M2.5medium Mistral Small 4mediumvsInklingnone DeepSeek V3.2nonevsMistral Small 4medium Mistral Small 4mediumvsGPT-4o-mininone Ling-2.6-1TnonevsMistral Small 4medium Mistral Small 4mediumvsQwen3.6 35B A3Bnone