Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

OpenAI: GPT-5.4 Nano vs Qwen: Qwen3.5-9B

Last updated at: 2026-03-17

Metric GPT-5.4 Nano GPT-5.4 Nano none Release: 2026-03-17 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Rank #73 #70
Score 4.3 4.5
Consistency 7.3 7.0
Cost per result 0.404 0.787
Total Cost $0.009 $0.024
Tests Correct
Attempt pass rate 29.4% 35.3%
Flaky tests 6 6
Total Runs 51 51
Output Tokens 2,185 18,141
Reasoning Tokens 0 141,204
Response Time (avg) 1.39s 69.21s
Response Time (max) 3.84s 226.38s
Response Time (total) 23.70s 968.99s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.5 8.0 16.7% 1 1.18s 800 0
Qwen3.5-9B 5.1 5.8 50.0% 2 34.44s 2,621 12,411
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.0 10.0 0.0% 0 3.84s 280 0
Qwen3.5-9B 3.0 10.0 0.0% 0 0ms 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 6.5 10.0 50.0% 0 1.11s 219 0
Qwen3.5-9B 3.6 5.6 33.3% 1 87.31s 1,383 32,113
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 2.9 4.4 22.2% 2 926ms 52 0
Qwen3.5-9B 3.6 7.2 22.2% 1 137.75s 11,549 48,475
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.8 2.5 33.3% 1 1.31s 180 0
Qwen3.5-9B 2.8 1.6 33.3% 1 226.38s 0 30,695
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 5.0 6.8 33.3% 1 787ms 84 0
Qwen3.5-9B 6.4 5.8 66.7% 1 17.15s 599 4,517
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 3.7 7.3 22.2% 1 1.29s 348 0
Qwen3.5-9B 3.1 10.0 0.0% 0 33.38s 1,545 11,844
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 Nano 10.0 10.0 100.0% 0 3.40s 222 0
Qwen3.5-9B 10.0 10.0 100.0% 0 4.31s 444 1,149

Quick Compare

Switch Comparison Pair

Qwen3.5-9BmediumvsGrok 4.1 FastnoneQwen3.5-9BmediumvsMiMo-V2-FlashnoneGPT-5.4 MininonevsQwen3.5-9BmediumMercury 2nonevsQwen3.5-9BmediumGPT-5.4 NanononevsGLM 4.7 FlashmediumGPT-5.4 NanononevsQwen3 Coder NextmediumGPT-4o-mininonevsQwen3.5-9BmediumNemotron 3 Super 120b A12bnoneFree AvailablevsQwen3.5-9BmediumMistral Small 4nonevsQwen3.5-9BmediumTrinity Large PreviewnoneFree AvailablevsQwen3.5-9BmediumKimi K2.5nonevsQwen3.5-9BmediumQwen3.5-9BmediumvsGrok 4.20 Betanone