Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

OpenAI: GPT-5.4 vs Qwen: Qwen3.5-9B

Last updated at: 2026-03-12

Metric GPT-5.4 GPT-5.4 none Release: 2026-03-05 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Rank #51 #66
Avg Score 4.5 2.6
Consistency 8.9 7.4
Cost per result 1.562 0.779
Total Cost $0.094 $0.024
Tests Correct
Attempt pass rate 41.7% 35.4%
Flaky tests 2 5
Total Runs 48 48
Output Tokens 1,819 17,930
Reasoning Tokens 0 139,706
Response Time (avg) 1.48s 71.44s
Response Time (max) 2.89s 226.38s
Response Time (total) 23.64s 928.77s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Avg Score vs Response Time (avg)

Total Output Tokens

Avg Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 7.3 11.1% 1 1.41s 388 0
Qwen3.5-9B 4.0 7.2 55.6% 1 31.54s 2,410 10,913
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 0.0% 0 2.89s 291 0
Qwen3.5-9B 10.0 10.0 0.0% 0 0ms 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 9.9 10.0 100.0% 0 1.04s 222 0
Qwen3.5-9B 5.0 5.6 33.3% 1 87.31s 1,383 32,113
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 4.0 7.2 44.4% 1 1.07s 50 0
Qwen3.5-9B 10.0 7.2 22.2% 1 137.75s 11,549 48,475
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 3.0 9.9 0.0% 0 1.78s 184 0
Qwen3.5-9B 10.0 1.6 33.3% 1 226.38s 0 30,695
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 5.5 10.0 50.0% 0 1.07s 81 0
Qwen3.5-9B 5.5 5.8 66.7% 1 17.15s 599 4,517
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 4.0 9.8 33.3% 0 1.52s 357 0
Qwen3.5-9B 10.0 10.0 0.0% 0 33.38s 1,545 11,844
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 2.75s 246 0
Qwen3.5-9B 10.0 10.0 100.0% 0 4.31s 444 1,149

Quick Compare

Switch Comparison Pair

MiniMax M2.5mediumvsGPT-5.4noneQwen3.5-9BmediumvsMiMo-V2-FlashnoneQwen3.5-9BmediumvsGrok 4.1 FastnoneGPT-5.4nonevsGrok 4.20 Multi-Agent BetamediumMercury 2nonevsQwen3.5-9BmediumNemotron 3 Super 120b A12bnoneFree AvailablevsQwen3.5-9BmediumMercury 2mediumvsGPT-5.4noneGPT-5.4nonevsQwen3 Coder NextmediumGPT-5.4nonevsQwen3.5-35B-A3BmediumQwen3.5-9BmediumvsGLM 4.7 FlashnoneNemotron 3 Super 120b A12bmediumFree AvailablevsGPT-5.4noneGPT-5.4nonevsGLM 4.7 Flashmedium