Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.6 Sol vs Qwen: Qwen3.6 27B

Summary

GPT-5.6 Sol vs Qwen3.6 27B (medium) benchmark comparison: Qwen3.6 27B (medium) leads on average score with 6.6 vs 6.3. GPT-5.6 Sol has the lower benchmark cost at $0.225 vs $0.336. GPT-5.6 Sol is faster at 1.66s vs 59.71s, with pass rates of 57.1% vs 60.3%.

Recommended model: GPT-5.6 Sol - Its score stays close to the best score here (6.3 vs 6.6), while responding about 36.0x faster than Qwen3.6 27B (medium).

Last updated at: 2026-07-09

Metric GPT-5.6 Sol GPT-5.6 Sol none Release: 2026-07-09 Qwen3.6 27B Qwen3.6 27B medium Release: 2026-04-20
Score 6.3 6.6
Rank #101 #94
Reliability 10.0 10.0
Consistency 9.0 8.2
Tests Correct
Attempt pass rate 57.1% 60.3%
Flaky tests 3 5
Total Runs 63 63
Cost per result 2.244 3.361
Total Cost $0.225 $0.336
Input Price $5.000 / 1M $0.286 / 1M
Output Price $30.000 / 1M $2.400 / 1M
Total Input Tokens 34,218 39,376
Output Tokens 1,776 16,189
Reasoning Tokens 0 122,521
Response Time (avg) 1.66s 59.71s
Response Time (max) 3.93s 168.22s
Response Time (total) 34.81s 1254.01s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#101 GPT-5.6 Sol

none
Cost
$0.116
Time
78.7s
Tokens
3,944 tok

#94 Qwen3.6 27B

medium
Cost
$0.009
Time
39.6s
Tokens
3,090 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 8.3 10.0 75.0% 0 1.27s 606 154 0
Qwen3.6 27B 8.3 10.0 75.0% 0 12.62s 453 582 4,311
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 5.5 10.0 33.3% 0 1.39s 7,302 390 0
Qwen3.6 27B 7.7 10.0 66.7% 0 142.99s 5,051 7,968 43,367
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 3.0 10.0 0.0% 0 3.93s 11,019 300 0
Qwen3.6 27B 7.0 3.7 66.7% 1 83.07s 15,104 2,088 14,689
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 1.53s 7,140 222 0
Qwen3.6 27B 3.5 1.4 50.0% 2 37.30s 7,778 568 9,404
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 3.6 7.2 22.2% 1 1.43s 723 54 0
Qwen3.6 27B 2.9 7.2 11.1% 1 73.38s 662 3,510 20,352
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 6.5 3.4 66.7% 1 1.52s 477 91 0
Qwen3.6 27B 6.5 3.4 66.7% 1 39.53s 516 81 3,045
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 8.5 6.8 83.3% 1 1.33s 660 84 0
Qwen3.6 27B 10.0 10.0 100.0% 0 37.96s 699 346 6,548
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 7.7 10.0 66.7% 0 1.49s 642 211 0
Qwen3.6 27B 7.7 10.0 66.7% 0 61.14s 696 255 12,044
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 3.70s 5,454 246 0
Qwen3.6 27B 10.0 10.0 100.0% 0 16.88s 8,213 390 2,954
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 3.0 10.0 0.0% 0 1.95s 195 24 0
Qwen3.6 27B 3.0 10.0 0.0% 0 80.99s 204 401 5,807

Quick Compare

Switch Comparison Pair

Nemotron 3 SupermediumFree AvailablevsGPT-5.6 SolnoneGemma 4 31BmediumFree AvailablevsGPT-5.6 SolnoneGPT-5.6 SolnonevsQwen3.5-35B-A3BmediumGemini 3.1 Flash Lite PreviewlowvsQwen3.6 27BmediumGemini 3.1 Flash LitelowvsGPT-5.6 SolnoneGemini 3.1 Flash Lite PreviewnonevsQwen3.6 27BmediumGemini 3.1 Flash LitelowvsQwen3.6 27BmediumGemini 3.1 Flash Lite PreviewlowvsGPT-5.6 SolnoneGemini 3.1 Flash LiteminimalvsGPT-5.6 SolnoneGemini 3.5 FlashminimalvsQwen3.6 27BmediumGPT-5.6 SolnonevsStep 3.5 FlashmediumGPT-5.5nonevsQwen3.6 27Bmedium