Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.6 Sol vs Qwen: Qwen3.6 35B A3B

Summary

GPT-5.6 Sol vs Qwen3.6 35B A3B (medium) benchmark comparison: Qwen3.6 35B A3B (medium) leads on average score with 6.7 vs 6.3. Qwen3.6 35B A3B (medium) has the lower benchmark cost at $0.146 vs $0.225. GPT-5.6 Sol is faster at 1.66s vs 18.08s, with pass rates of 57.1% vs 63.5%.

Recommended model: GPT-5.6 Sol - Its score stays close to the best score here (6.3 vs 6.7), while responding about 10.9x faster than Qwen3.6 35B A3B (medium).

Last updated at: 2026-07-09

Metric GPT-5.6 Sol GPT-5.6 Sol none Release: 2026-07-09 Qwen3.6 35B A3B Qwen3.6 35B A3B medium Release: 2026-04-20
Score 6.3 6.7
Rank #101 #88
Reliability 10.0 10.0
Consistency 9.0 9.6
Tests Correct
Attempt pass rate 57.1% 63.5%
Flaky tests 3 1
Total Runs 63 63
Cost per result 2.244 1.094
Total Cost $0.225 $0.146
Input Price $5.000 / 1M $0.140 / 1M
Output Price $30.000 / 1M $1.000 / 1M
Total Input Tokens 34,218 16,385
Output Tokens 1,776 19,632
Reasoning Tokens 0 130,219
Response Time (avg) 1.66s 18.08s
Response Time (max) 3.93s 86.11s
Response Time (total) 34.81s 343.61s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#101 GPT-5.6 Sol

none
Cost
$0.116
Time
78.7s
Tokens
3,944 tok

#88 Qwen3.6 35B A3B

medium
Invalid SVG
Cost
$0.000
Time
300.0s
Tokens
0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 8.3 10.0 75.0% 0 1.27s 606 154 0
Qwen3.6 35B A3B 10.0 10.0 100.0% 0 6.02s 672 1,154 12,385
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 5.5 10.0 33.3% 0 1.39s 7,302 390 0
Qwen3.6 35B A3B 7.7 10.0 66.7% 0 50.55s 5,051 7,929 37,223
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 3.0 10.0 0.0% 0 3.93s 11,019 300 0
Qwen3.6 35B A3B 3.0 10.0 0.0% 0 0ms 0 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 1.53s 7,140 222 0
Qwen3.6 35B A3B 10.0 10.0 100.0% 0 12.99s 7,776 2,591 9,968
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 3.6 7.2 22.2% 1 1.43s 723 54 0
Qwen3.6 35B A3B 5.3 7.2 44.4% 1 22.50s 771 6,193 39,116
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 6.5 3.4 66.7% 1 1.52s 477 91 0
Qwen3.6 35B A3B 4.4 9.9 0.0% 0 8.66s 516 129 4,569
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 8.5 6.8 83.3% 1 1.33s 660 84 0
Qwen3.6 35B A3B 10.0 10.0 100.0% 0 7.50s 699 219 7,404
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 7.7 10.0 66.7% 0 1.49s 642 211 0
Qwen3.6 35B A3B 8.0 10.0 66.7% 0 5.95s 696 655 9,228
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 3.70s 5,454 246 0
Qwen3.6 35B A3B 3.0 10.0 0.0% 0 0ms 0 0 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 3.0 10.0 0.0% 0 1.95s 195 24 0
Qwen3.6 35B A3B 3.0 10.0 0.0% 0 32.90s 204 762 10,326

Quick Compare

Switch Comparison Pair

Nemotron 3 SupermediumFree AvailablevsGPT-5.6 SolnoneGemma 4 31BmediumFree AvailablevsGPT-5.6 SolnoneGPT-5.6 SolnonevsQwen3.5-35B-A3BmediumGemini 3.5 FlashminimalvsQwen3.6 35B A3BmediumGemini 3.1 Flash LitelowvsGPT-5.6 SolnoneGemini 3.1 Flash Lite PreviewlowvsGPT-5.6 SolnoneGemini 3 Flash PreviewnonevsQwen3.6 35B A3BmediumGemini 3.1 Flash LiteminimalvsGPT-5.6 SolnoneGemini 3.1 Flash Lite PreviewlowvsQwen3.6 35B A3BmediumGemini 3.5 FlashnonevsQwen3.6 35B A3BmediumGPT-5.6 SolnonevsQwen3.6 27BmediumGemini 3.1 Flash Lite PreviewnonevsQwen3.6 35B A3Bmedium