Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.6 Sol vs Qwen: Qwen3.7 Max

Summary

GPT-5.6 Sol (medium) vs Qwen3.7 Max (medium) benchmark comparison: The average score is effectively tied at 9.4 vs 9.4. Qwen3.7 Max (medium) has the lower benchmark cost at $0.523 vs $0.966. GPT-5.6 Sol (medium) is faster at 10.79s vs 16.02s, with pass rates of 90.5% vs 88.9%.

Recommended model: Qwen3.7 Max (medium) - It has the best score here (9.4), while costing about 1.9x less than GPT-5.6 Sol (medium).

Last updated at: 2026-07-09

Metric GPT-5.6 Sol GPT-5.6 Sol medium Release: 2026-07-09 Qwen3.7 Max Qwen3.7 Max medium Release: 2026-05-22
Score 9.4 9.4
Rank #5 #4
Reliability 10.0 10.0
Consistency 8.8 9.6
Tests Correct
Attempt pass rate 90.5% 88.9%
Flaky tests 3 1
Total Runs 63 63
Cost per result 5.682 5.517
Total Cost $0.966 $0.523
Input Price $5.000 / 1M $1.250 / 1M
Output Price $30.000 / 1M $3.750 / 1M
Total Input Tokens 34,218 42,360
Output Tokens 1,895 2,129
Reasoning Tokens 24,600 122,959
Response Time (avg) 10.79s 16.02s
Response Time (max) 79.40s 59.98s
Response Time (total) 226.57s 336.51s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#5 GPT-5.6 Sol

medium
Cost
$0.124
Time
57.2s
Tokens
4,199 tok

#4 Qwen3.7 Max

medium
Cost
$0.017
Time
68.8s
Tokens
4,526 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 2.81s 606 173 572
Qwen3.7 Max 10.0 10.0 100.0% 0 6.36s 672 222 8,742
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 9.40s 7,302 394 4,706
Qwen3.7 Max 10.0 10.0 100.0% 0 35.31s 7,893 423 34,808
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 12.13s 11,019 318 1,214
Qwen3.7 Max 10.0 10.0 100.0% 0 19.60s 14,934 366 8,405
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 1.38s 7,140 222 0
Qwen3.7 Max 10.0 10.0 100.0% 0 8.80s 7,782 270 6,254
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 5.9 7.2 55.6% 1 47.86s 723 71 16,201
Qwen3.7 Max 5.9 7.2 55.6% 1 24.94s 771 61 31,793
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 3.48s 477 99 214
Qwen3.7 Max 10.0 10.0 100.0% 0 11.70s 516 135 4,457
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 2.50s 660 96 249
Qwen3.7 Max 10.0 10.0 100.0% 0 7.46s 699 102 5,452
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 8.2 7.2 88.9% 1 2.98s 642 228 566
Qwen3.7 Max 10.0 10.0 100.0% 0 8.84s 696 259 8,908
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 6.30s 5,454 264 361
Qwen3.7 Max 10.0 10.0 100.0% 0 6.63s 8,193 267 1,220
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 4.7 1.6 66.7% 1 4.94s 195 30 517
Qwen3.7 Max 3.0 10.0 0.0% 0 33.37s 204 24 12,920

Quick Compare

Switch Comparison Pair

GPT-5.6 SolhighvsQwen3.7 MaxmediumGPT-5.5lowvsQwen3.7 MaxmediumGemini 3.5 FlashlowvsGPT-5.6 SolmediumGemini 3.5 FlashlowvsQwen3.7 MaxmediumGPT-5.6 SollowvsQwen3.7 MaxmediumGemini 3.5 FlashhighvsQwen3.7 MaxmediumGemini 3.5 FlashhighvsGPT-5.6 SolmediumGPT-5.2 ChatnonevsQwen3.7 MaxmediumDeepSeek V4 FlashhighvsGPT-5.6 SolmediumDeepSeek V4 FlashhighvsQwen3.7 MaxmediumGPT-5.6 TerrahighvsQwen3.7 MaxmediumGPT-5.6 SolmediumvsGrok 4.5low