#50

Mercury 2

Inception · Release: 2026-02-24 · inception/mercury-2::none

Avg Score

31

Cost per result

0.196

Consistency

89

Total Cost

$0.006

Tests Correct

3

A test is fully passed only if every run passed for that test.

Wrong Tests

12

Attempt pass rate: 26.7%

Flaky tests

2

Response time: avg 594ms · total 8.91s · max 1.27s

Wrong answer: 11 Did not follow instructions: 1

Top Models by Score

Choose the first model, then click a second model to open a side-by-side page.

#43 Qwen3.5-35B-A3B 45

#44 GPT-5.4 45

#45 Trinity Large Preview (free) 41

#46 GPT-4o-mini 41

#47 GLM 4.7 Flash 38

#48 Kimi K2.5 37

#49 Qwen3 Coder Next 34

#50 Mercury 2 31

#51 Qwen3 Coder Next 31

#52 Grok 4.1 Fast 29

#53 GLM 4.7 Flash 29

#54 MiMo-V2-Flash 27

#55 LFM2-24B-A2B 23

Category	Avg Score	Consistency	Tests Correct
Anti-AI Tricks	100	100	0/3
Combined	100	100	0/1
Data parsing and extraction	55	59	1/2
Domain specific	40	72	1/3
Instructions following	35	100	0/2
Puzzle Solving	100	100	0/3
Tool Calling	100	100	1/1