Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.4 Nano vs Qwen: Qwen3.5-9B

Summary

GPT-5.4 Nano vs Qwen3.5-9B benchmark comparison: GPT-5.4 Nano leads on average score with 4.8 vs 3.8. GPT-5.4 Nano has the lower benchmark cost at $0.011 vs $0.036. GPT-5.4 Nano is faster at 1.48s vs 82.24s, with pass rates of 30.2% vs 27.0%.

Recommended model: GPT-5.4 Nano - It has the best score here (4.8), while costing about 3.5x less than Qwen3.5-9B.

Last updated at: 2026-06-12

Metric GPT-5.4 Nano GPT-5.4 Nano none Release: 2026-03-17 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Score 4.8 3.8
Rank #149 #164
Reliability 10.0 6.7
Consistency 8.2 8.0
Tests Correct
Attempt pass rate 30.2% 27.0%
Flaky tests 5 5
Total Runs 63 63
Cost per result 0.259 1.187
Total Cost $0.011 $0.036
Input Price $0.200 / 1M $0.100 / 1M
Output Price $1.250 / 1M $0.150 / 1M
Total Input Tokens 34,212 17,070
Output Tokens 2,784 29,045
Reasoning Tokens 0 209,516
Response Time (avg) 1.48s 82.24s
Response Time (max) 4.47s 226.38s
Response Time (total) 31.01s 1315.88s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#149 GPT-5.4 Nano

none
Cost
$0.008
Time
46.1s
Tokens
5,735 tok

#164 Qwen3.5-9B

medium
Cost
$0.001
Time
35.9s
Tokens
3,030 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.5 8.0 16.7% 1 1.18s 606 800 0
Qwen3.5-9B 5.1 5.8 50.0% 2 34.44s 369 2,621 12,411
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 4.6 7.9 22.2% 1 2.22s 7,305 613 0
Qwen3.5-9B 2.9 10.0 0.0% 0 100.88s 2,396 7,890 41,129
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.0 10.0 0.0% 0 3.84s 11,019 280 0
Qwen3.5-9B 3.0 10.0 0.0% 0 0ms 0 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 6.5 10.0 50.0% 0 1.11s 7,140 219 0
Qwen3.5-9B 3.6 5.6 33.3% 1 87.31s 4,722 1,383 32,113
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 2.9 4.4 22.2% 2 926ms 723 52 0
Qwen3.5-9B 3.6 7.2 22.2% 1 137.75s 295 11,549 48,475
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.8 2.5 33.3% 1 1.31s 477 180 0
Qwen3.5-9B 2.8 1.6 33.3% 1 226.38s 180 0 30,695
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 6.3 10.0 50.0% 0 784ms 660 89 0
Qwen3.5-9B 6.5 10.0 50.0% 0 5.75s 381 491 1,824
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 5.4 10.0 33.3% 0 1.25s 642 308 0
Qwen3.5-9B 3.0 10.0 0.0% 0 32.27s 376 1,593 12,026
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 10.0 10.0 100.0% 0 3.40s 5,445 222 0
Qwen3.5-9B 10.0 10.0 100.0% 0 4.31s 8,283 444 1,149
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.0 10.0 0.0% 0 773ms 195 21 0
Qwen3.5-9B 3.0 10.0 0.0% 0 177.02s 68 3,074 29,694

Quick Compare

Switch Comparison Pair

MiniMax M2.5mediumvsGPT-5.4 NanononeGPT-5.4 NanononevsQwen3 Coder NextmediumCobuddymediumvsGPT-5.4 NanononeGranite 4.1 8BnonevsQwen3.5-9BmediumMistral Small 4mediumvsGPT-5.4 NanononeMiniMax M2.7mediumvsGPT-5.4 NanononeQwen3.5-9BmediumvsMiMo-V2-FlashnoneGPT-5.4 NanononevsGLM 4.7 FlashmediumMercury 2nonevsQwen3.5-9BmediumLing-2.6-1TnonevsQwen3.5-9Bmediumgpt-oss-120bnoneFree AvailablevsQwen3.5-9BmediumRing-2.6-1TnonevsQwen3.5-9Bmedium