Navigate
AD
Track all your projects in one dashboard. Get 📊stats, 🔥heatmaps and 👀recordings in one self-hosted dashboard.
uxwizz.com

Mercury 2.5 Preview (medium) vs Qwen4 Max

Qwen4 Max leads on average score with 7.0 vs 6.8. Mercury 2.5 Preview (medium) has the lower benchmark cost at $0.039 vs $0.465. Qwen4 Max is faster at 6.38s vs 6.66s, with pass rates of 66.7% vs 63.8%.

Last updated at: 2026-11-16

Compared models

Rank
#248
Total Output Tokens
150,107
Response Time (avg)
6.66s
Total Cost
$0.039
Rank
#230
Total Output Tokens
9,342
Response Time (avg)
6.38s
Total Cost
$0.465
Recommended model Mercury 2.5 Preview (medium)

Its score stays close to the best score here (6.8 vs 7.0), while costing about 12.1x less than Qwen4 Max.

Detailed comparison

Metric Mercury 2.5 Preview Mercury 2.5 Preview medium Release: 2026-09-02 Qwen4 Max Qwen4 Max none Release: 2026-11-16
Score 6.8 7.0
Rank #248 #230
Reliability 10.0 10.0
Consistency 9.6 9.6
Attempts 69/69 69/69
Tests Correct
Attempt pass rate 66.7% 63.8%
Flaky tests 1 1
Total Runs 69 69
Cost per result 0.244 3.321
Total Cost $0.039 $0.465
Input Price $0.040 / 1M $1.500 / 1M
Output Price $0.150 / 1M $4.500 / 1M
Cache Read Price N/A $0.075 / 1M
Cache Write Price N/A $1.875 / 1M
Total Input Tokens 397,291 281,902
Output Tokens 6,355 9,342
Reasoning Tokens 143,752 0
Response Time (avg) 6.66s 6.38s
Response Time (max) 74.63s 67.27s
Response Time (total) 153.29s 146.64s
Parameters ~100B ~2.4T total (~95B active)
Availability Closed Closed

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#248 Mercury 2.5 Preview

medium
Cost
$0.001
Time
2.7s
Tokens
2,241 tok

#230 Qwen4 Max

none
Cost
$0.027
Time
123.5s
Tokens
6,091 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2.5 Preview 7.8 10.0 66.7% 0 3.86s 7,839 552 22,126
Qwen4 Max 5.8 10.0 33.3% 0 1.97s 8,010 384 0

Switch Comparison Pair