Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.4 vs Qwen: Qwen3.6 35B A3B

Last updated at: 2026-05-22

Metric GPT-5.4 GPT-5.4 medium Release: 2026-03-05 Qwen3.6 35B A3B Qwen3.6 35B A3B medium Release: 2026-04-20
Score 7.9 7.8
Rank #27 #30
Reliability 10.0 9.4
Consistency 8.5 9.1
Tests Correct
Attempt pass rate 75.0% 70.4%
Flaky tests 4 2
Total Runs 60 60
Cost per result 8.767 1.050
Total Cost $1.140 $0.126
Input Price $2.500 / 1M $0.150 / 1M
Output Price $15.000 / 1M $1.000 / 1M
Output Tokens 2,222 18,325
Reasoning Tokens 68,503 115,750
Response Time (avg) 22.31s 17.26s
Response Time (max) 100.41s 86.11s
Response Time (total) 446.15s 310.77s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 8.3 10.0 75.0% 0 4.11s 240 1,511
Qwen3.6 35B A3B 10.0 10.0 100.0% 0 6.02s 1,154 12,385
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 8.2 6.7 83.3% 1 54.98s 412 19,995
Qwen3.6 35B A3B 6.6 10.0 50.0% 0 59.35s 6,601 22,535
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 20.57s 301 3,543
Qwen3.6 35B A3B 0.0 0.0 0.0% 0 0ms 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 5.32s 234 804
Qwen3.6 35B A3B 10.0 10.0 100.0% 0 12.99s 2,591 9,968
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 5.3 7.2 44.4% 1 74.27s 61 34,748
Qwen3.6 35B A3B 5.3 7.2 44.4% 1 22.50s 6,193 39,116
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 4.7 3.1 33.3% 1 4.92s 145 321
Qwen3.6 35B A3B 4.4 9.9 0.0% 0 8.66s 129 4,569
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 3.11s 93 897
Qwen3.6 35B A3B 10.0 10.0 100.0% 0 7.50s 219 7,404
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 8.2 7.2 88.9% 1 9.13s 442 3,832
Qwen3.6 35B A3B 7.9 7.5 77.8% 1 5.98s 676 9,447
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 13.28s 264 1,031
Qwen3.6 35B A3B 0.0 0.0 0.0% 0 0ms 0 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 3.0 10.0 0.0% 0 13.95s 30 1,821
Qwen3.6 35B A3B 3.0 10.0 0.0% 0 32.90s 762 10,326

Quick Compare

Switch Comparison Pair

GPT-5.4mediumvsQwen3.7 MaxnoneGemini 3.5 FlashminimalvsGPT-5.4mediumGemini 3 Flash PreviewnonevsQwen3.6 35B A3BmediumGemini 3.5 FlashminimalvsQwen3.6 35B A3BmediumGemini 3.1 Flash Lite PreviewlowvsQwen3.6 35B A3BmediumGPT-5.2 ChatnonevsQwen3.6 35B A3BmediumGemini 3 Flash PreviewnonevsGPT-5.4mediumGemini 3.1 Flash Lite PreviewlowvsGPT-5.4mediumGemini 3.1 Flash Lite PreviewnonevsQwen3.6 35B A3BmediumGemini 3.1 Flash LitelowvsQwen3.6 35B A3BmediumGPT-5.3 ChatnonevsQwen3.6 35B A3BmediumDeepSeek V4 FlashhighFree AvailablevsQwen3.6 35B A3Bmedium