Navigate
Advertise here

Mercury 2 vs Qwen3.5-9B (medium)

Mercury 2 leads on average score with 4.4 vs 4.0. Mercury 2 has the lower benchmark cost at $0.033 vs $0.106. Mercury 2 is faster at 1.18s vs 110.18s, with pass rates of 23.2% vs 26.1%.

Last updated at: 2026-11-22

Compared models

Rank
#471
Total Output Tokens
9,709
Response Time (avg)
1.18s
Total Cost
$0.033
Rank
#480
Total Output Tokens
447,247
Response Time (avg)
110.18s
Total Cost
$0.106
Recommended model Mercury 2

It has the best score here (4.4), while costing about 3.3x less than Qwen3.5-9B (medium).

Detailed comparison

Metric Mercury 2 Mercury 2 none Release: 2026-02-24 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Score 4.4 4.0
Rank #471 #480
Reliability 10.0 8.7
Consistency 9.2 7.9
Attempts 69/69 69/69
Tests Correct
Attempt pass rate 23.2% 26.1%
Flaky tests 2 6
Total Runs 69 69
Cost per result 0.812 3.523
Total Cost $0.033 $0.106
Input Price $0.250 / 1M $0.100 / 1M
Output Price $0.750 / 1M $0.150 / 1M
Cache Read Price $0.025 / 1M N/A
Cache Write Price N/A N/A
Total Input Tokens 100,634 387,323
Output Tokens 9,609 48,562
Reasoning Tokens 100 410,431
Response Time (avg) 1.18s 110.18s
Response Time (max) 9.00s 569.97s
Response Time (total) 27.14s 1873.00s
Parameters ~100B 9B
Availability Closed Open source

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#471 Mercury 2

none
Cost
$0.002
Time
1.8s
Tokens
1,514 tok

#480 Qwen3.5-9B

medium
Cost
$0.001
Time
35.9s
Tokens
3,030 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2 3.4 9.6 0.0% 0 1.03s 7,229 3,088 0
Qwen3.5-9B 2.9 10.0 0.0% 0 100.88s 2,396 7,890 41,129

Switch Comparison Pair