Navigate
Advertise here

GPT-4o-mini (default) vs Qwen3.5-9B

GPT-4o-mini (default) leads on average score with 5.0 vs 4.9. GPT-4o-mini (default) has the lower benchmark cost at $0.024 vs $0.041. GPT-4o-mini (default) is faster at 4.16s vs 37.19s, with pass rates of 21.7% vs 18.8%.

Last updated at: 2026-10-12

Compared models

Rank
#322
Total Output Tokens
3,627
Response Time (avg)
4.16s
Total Cost
$0.024
Rank
#331
Total Output Tokens
70,427
Response Time (avg)
37.19s
Total Cost
$0.041
Recommended model GPT-4o-mini (default)

It has the best score here (5.0), while costing about 1.7x less than Qwen3.5-9B.

Detailed comparison

Metric GPT-4o-mini GPT-4o-mini default Release: 2024-07-18 Qwen3.5-9B Qwen3.5-9B none Release: 2026-03-02
Score 5.0 4.9
Rank #322 #331
Reliability 10.0 9.8
Consistency 9.9 9.7
Attempts 69/69 69/69
Tests Correct
Attempt pass rate 21.7% 18.8%
Flaky tests 0 1
Total Runs 69 69
Cost per result 0.480 1.022
Total Cost $0.024 $0.041
Input Price $0.150 / 1M $0.100 / 1M
Output Price $0.600 / 1M $0.150 / 1M
Cache Read Price $0.075 / 1M N/A
Cache Write Price N/A N/A
Total Input Tokens 145,180 307,670
Output Tokens 3,627 70,427
Reasoning Tokens 0 0
Response Time (avg) 4.16s 37.19s
Response Time (max) 39.99s 433.02s
Response Time (total) 70.70s 855.27s
Parameters ~8B 9B
Availability Closed Open source

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#322 GPT-4o-mini

default
Cost
$0.001
Time
6.6s
Tokens
742 tok

#331 Qwen3.5-9B

none
Reached the allocated time limit (300 seconds) without receiving showcase output.
Cost
$0.000
Time
300.0s
Tokens
0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-4o-mini 3.2 9.6 0.0% 0 1.63s 7,314 367 0
Qwen3.5-9B 3.9 7.8 11.1% 1 5.60s 7,913 1,042 0

Quick Compare

Switch Comparison Pair

KAT Coder AIR V2.5lowvsGPT-4o-minidefaultLing 2.6 1tdefaultvsQwen3.5-9BnoneMiniMax M2.7mediumvsGPT-4o-minidefaultLaguna S 2.1highFree AvailablevsQwen3.5-9BnoneGPT-4o-minidefaultvsInkling SmallnoneFree AvailableMercury 2.5 PreviewlowvsGPT-4o-minidefaultGPT-4o-minidefaultvsLaguna S 2.1highFree AvailableGPT-4o-minidefaultvsLaguna S 2.1mediumFree AvailableGPT-4o-minidefaultvsMiMo-V2.5noneKAT Coder AIR V2.5lowvsQwen3.5-9BnoneGPT-4o-minidefaultvsHy4 previewnoneDots 3 Note PreviewdefaultFree AvailablevsQwen3.5-9Bnone