Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.4 vs Qwen: Qwen3.7 Plus

Last updated at: 2026-06-04

Metric GPT-5.4 GPT-5.4 medium Release: 2026-03-05 Qwen3.7 Plus Qwen3.7 Plus medium Release: 2026-06-03
Score 8.0 8.2
Rank #21 #18
Reliability 10.0 10.0
Consistency 8.6 9.1
Tests Correct
Attempt pass rate 76.2% 77.8%
Flaky tests 4 2
Total Runs 63 63
Cost per result 8.640 1.474
Total Cost $1.210 $0.221
Input Price $2.500 / 1M $0.400 / 1M
Output Price $15.000 / 1M $1.600 / 1M
Total Input Tokens 34,108 40,939
Output Tokens 2,242 2,125
Reasoning Tokens 72,707 125,754
Response Time (avg) 22.35s 38.95s
Response Time (max) 100.41s 178.04s
Response Time (total) 469.29s 817.85s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 8.3 10.0 75.0% 0 4.11s 606 240 1,511
Qwen3.7 Plus 10.0 10.0 100.0% 0 8.58s 672 195 5,065
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 8.8 7.8 88.9% 1 44.36s 7,305 433 24,216
Qwen3.7 Plus 6.1 6.6 55.6% 1 108.60s 6,472 414 43,576
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 20.57s 11,019 301 3,543
Qwen3.7 Plus 10.0 10.0 100.0% 0 65.24s 14,934 366 10,132
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 5.32s 7,140 234 804
Qwen3.7 Plus 10.0 10.0 100.0% 0 21.75s 7,782 270 6,713
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 5.3 7.2 44.4% 1 74.27s 619 61 34,748
Qwen3.7 Plus 3.6 7.2 22.2% 1 45.35s 771 57 27,073
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 4.7 3.1 33.3% 1 4.92s 477 145 321
Qwen3.7 Plus 10.0 10.0 100.0% 0 25.48s 516 123 3,998
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 3.11s 660 93 897
Qwen3.7 Plus 10.0 10.0 100.0% 0 16.13s 699 102 5,013
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 8.2 7.2 88.9% 1 9.14s 642 441 3,815
Qwen3.7 Plus 10.0 10.0 100.0% 0 16.38s 696 280 7,312
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 13.28s 5,445 264 1,031
Qwen3.7 Plus 10.0 10.0 100.0% 0 15.02s 8,193 292 1,831
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 3.0 10.0 0.0% 0 13.95s 195 30 1,821
Qwen3.7 Plus 3.0 10.0 0.0% 0 91.07s 204 26 15,041

Quick Compare

Switch Comparison Pair

Gemini 3.5 FlashnonevsQwen3.7 PlusmediumGemini 3.5 FlashnonevsGPT-5.4mediumGemini 3 Flash PreviewlowvsQwen3.7 PlusmediumGPT-5.2 ChatnonevsQwen3.7 PlusmediumDeepSeek V4 FlashhighvsGPT-5.4mediumGemini 3.5 FlashminimalvsGPT-5.4mediumGPT-5.4mediumvsQwen3.7 MaxnoneGemini 3 Flash PreviewlowvsGPT-5.4mediumDeepSeek V4 FlashhighvsQwen3.7 PlusmediumGemini 3.5 FlashminimalvsQwen3.7 PlusmediumGemini 3 Flash PreviewnonevsGPT-5.4mediumGemini 3.1 Flash Lite PreviewlowvsGPT-5.4medium