Navigate
AI BENCHY
Advertise here

Mercury 2.5 Preview (high) vs Qwen3.5 Plus 2026-02-15

The average score is effectively tied at 6.6 vs 6.6. Mercury 2.5 Preview (high) has the lower benchmark cost at $0.030 vs $0.073. Mercury 2.5 Preview (high) is faster at 4.01s vs 9.86s, with pass rates of 65.2% vs 53.0%.

Last updated at: 2026-09-02

Rank
#157
Total Output Tokens
163,505
Response Time (avg)
4.01s
Total Cost
$0.030
Rank
#158
Total Output Tokens
29,370
Response Time (avg)
9.86s
Total Cost
$0.073
Recommended model Mercury 2.5 Preview (high)

It has the best score here (6.6), while costing about 2.5x less than Qwen3.5 Plus 2026-02-15.

Detailed comparison

Metric Mercury 2.5 Preview Mercury 2.5 Preview high Release: 2026-09-02 Qwen3.5 Plus 2026-02-15 Qwen3.5 Plus 2026-02-15 none Release: 2026-02-15
Score 6.6 6.6
Rank #157 #158
Reliability 10.0 10.0
Consistency 8.2 9.4
Attempts 66/66 66/66
Tests Correct
Attempt pass rate 65.2% 53.0%
Flaky tests 5 2
Total Runs 66 66
Cost per result 0.243 0.682
Total Cost $0.030 $0.073
Input Price $0.040 / 1M $0.260 / 1M
Output Price $0.150 / 1M $1.560 / 1M
Total Input Tokens 115,759 102,655
Output Tokens 2,562 29,370
Reasoning Tokens 160,943 0
Response Time (avg) 4.01s 9.86s
Response Time (max) 17.71s 123.00s
Response Time (total) 88.25s 157.77s
Parameters ~100B ~397B total (~17B active)
Availability Closed Closed

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#157 Mercury 2.5 Preview

high
Cost
$0.001
Time
4.5s
Tokens
3,851 tok

#158 Qwen3.5 Plus 2026-02-15

none
Cost
$0.012
Time
153.2s
Tokens
7,787 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mercury 2.5 Preview 6.7 7.8 55.6% 1 6.13s 7,751 485 42,569
Qwen3.5 Plus 2026-02-15 4.3 7.9 11.1% 1 2.05s 7,913 473 0

Quick Compare

Switch Comparison Pair

Mercury 2.5 PreviewhighvsQwen3.6 35B A3BmediumGemini 3.1 Flash LitelowvsMercury 2.5 PreviewhighMercury 2.5 PreviewhighvsInkling SmallmediumLaguna XS 2.1mediumFree AvailablevsQwen3.5 Plus 2026-02-15noneGemini 3.1 Flash Lite PreviewlowvsMercury 2.5 PreviewhighGemini 3.1 Flash LitelowvsQwen3.5 Plus 2026-02-15noneMercury 2.5 PreviewhighvsQwen3.5-27BnoneGemini 3.5 Flash LitelowvsMercury 2.5 PreviewhighQwen3.5 Plus 2026-02-15nonevsInkling SmallmediumMercury 2.5 PreviewhighvsQwen3.6 27BmediumMercury 2.5 PreviewhighvsLaguna XS 2.1mediumFree AvailableGemini 3.1 Flash Lite PreviewlowvsQwen3.5 Plus 2026-02-15none