Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

OpenAI: GPT-4o-mini vs Qwen: Qwen3.5-9B

Last updated at: 2026-04-26

Metric GPT-4o-mini GPT-4o-mini none Release: 2024-07-18 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Score 4.9 4.4
Rank #104 #114
Reliability N/A N/A
Consistency 9.9 7.2
Tests Correct
Attempt pass rate 22.2% 33.3%
Flaky tests 0 6
Total Runs 54 54
Cost per result 0.122 0.976
Total Cost $0.005 $0.030
Input Price $0.150 / 1M $0.100 / 1M
Output Price $0.600 / 1M $0.150 / 1M
Output Tokens 1,947 24,291
Reasoning Tokens 0 172,597
Response Time (avg) 2.00s 73.64s
Response Time (max) 7.58s 226.38s
Response Time (total) 21.99s 1104.60s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 4.8 10.0 25.0% 0 1.34s 186 0
Qwen3.5-9B 5.1 5.8 50.0% 2 34.44s 2,621 12,411
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 3.0 8.7 0.0% 0 2.55s 347 0
Qwen3.5-9B 2.6 10.0 0.0% 0 135.61s 6,150 31,393
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 3.0 10.0 0.0% 0 7.58s 568 0
Qwen3.5-9B 3.0 10.0 0.0% 0 0ms 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 10.0 10.0 100.0% 0 1.27s 183 0
Qwen3.5-9B 3.6 5.6 33.3% 1 87.31s 1,383 32,113
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 3.0 10.0 0.0% 0 637ms 15 0
Qwen3.5-9B 3.6 7.2 22.2% 1 137.75s 11,549 48,475
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 4.0 10.0 0.0% 0 909ms 66 0
Qwen3.5-9B 2.8 1.6 33.3% 1 226.38s 0 30,695
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 4.8 10.0 0.0% 0 1.27s 69 0
Qwen3.5-9B 6.4 5.8 66.7% 1 17.15s 599 4,517
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 3.7 10.0 0.0% 0 1.30s 308 0
Qwen3.5-9B 3.1 10.0 0.0% 0 33.38s 1,545 11,844
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-4o-mini 10.0 10.0 100.0% 0 2.51s 205 0
Qwen3.5-9B 10.0 10.0 100.0% 0 4.31s 444 1,149

Quick Compare

Switch Comparison Pair

GPT-5.4 NanononevsQwen3.5-9BmediumLing 2.6 1tnoneFree AvailablevsQwen3.5-9BmediumQwen3.5-9BmediumvsGrok 4.1 FastnoneGPT-4o-mininonevsQwen3 Coder NextmediumQwen3.5-9BmediumvsMiMo-V2-FlashnoneGPT-4o-mininonevsGLM 4.7 FlashmediumQwen3.5-9BmediumvsHY3 PreviewnoneFree AvailableGPT-4o-mininonevsElephant AlphamediumMiniMax M2.7mediumvsGPT-4o-mininoneMercury 2nonevsQwen3.5-9BmediumNemotron 3 SupernoneFree AvailablevsQwen3.5-9BmediumQwen3.5-9BmediumvsMiMo-V2.5none