#40

Mercury 2

Inception · Release: 2026-02-24 · inception/mercury-2::medium

Avg Score

48

Cost per result

0.726

Consistency

83

Total Cost

$0.044

Tests Correct

6

A test is fully passed only if every run passed for that test.

Wrong Tests

9

Attempt pass rate: 51.1%

Flaky tests

3

Response time: avg 2.47s · total 34.56s · max 14.63s

Wrong answer: 5 Did not follow instructions: 3 API error: 1

Top Models by Score

Choose the first model, then click a second model to open a side-by-side page.

#34 GPT-5 Nano 54

#35 gpt-oss-120b 51

#36 Gemini 2.5 Flash 51

#37 Qwen3.5-27B 49

#38 DeepSeek V3.2 49

#39 Qwen3.5-122B-A10B 49

#40 Mercury 2 48

#41 Qwen3.5-Flash 47

#42 MiniMax M2.5 46

#43 Qwen3.5-35B-A3B 45

#44 GPT-5.4 45

#45 Trinity Large Preview (free) 41

#46 GPT-4o-mini 41

Category	Avg Score	Consistency	Tests Correct
Anti-AI Tricks	73	98	2/3
Combined	100	100	1/1
Data parsing and extraction	55	59	1/2
Domain specific	100	72	0/3
Instructions following	55	100	1/2
Puzzle Solving	17	75	0/3
Tool Calling	100	100	1/1