Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.6 Sol vs OpenAI: GPT-5.6 Sol

Summary

GPT-5.6 Sol (low) vs GPT-5.6 Sol (medium) benchmark comparison: GPT-5.6 Sol (low) leads on average score with 9.5 vs 9.4. GPT-5.6 Sol (low) has the lower benchmark cost at $0.649 vs $0.966. GPT-5.6 Sol (low) is faster at 8.29s vs 10.79s, with pass rates of 85.7% vs 90.5%.

Recommended model: GPT-5.6 Sol (low) - It has the strongest score in this comparison (9.5) and the best overall balance of cost and response time across all 2 models.

Last updated at: 2026-07-09

Metric GPT-5.6 Sol GPT-5.6 Sol low Release: 2026-07-09 GPT-5.6 Sol GPT-5.6 Sol medium Release: 2026-07-09
Score 9.5 9.4
Rank #3 #5
Reliability 10.0 10.0
Consistency 9.2 8.8
Tests Correct
Attempt pass rate 85.7% 90.5%
Flaky tests 2 3
Total Runs 63 63
Cost per result 3.816 5.682
Total Cost $0.649 $0.966
Input Price $5.000 / 1M $5.000 / 1M
Output Price $30.000 / 1M $30.000 / 1M
Total Input Tokens 34,218 34,218
Output Tokens 1,898 1,895
Reasoning Tokens 14,023 24,600
Response Time (avg) 8.29s 10.79s
Response Time (max) 53.91s 79.40s
Response Time (total) 174.04s 226.57s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#3 GPT-5.6 Sol

low
Cost
$0.062
Time
26.7s
Tokens
2,150 tok

#5 GPT-5.6 Sol

medium
Cost
$0.124
Time
57.2s
Tokens
4,199 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 8.3 10.0 75.0% 0 2.60s 606 174 492
GPT-5.6 Sol 10.0 10.0 100.0% 0 2.81s 606 173 572
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 11.25s 7,302 412 3,482
GPT-5.6 Sol 10.0 10.0 100.0% 0 9.40s 7,302 394 4,706
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 9.87s 11,019 318 792
GPT-5.6 Sol 10.0 10.0 100.0% 0 12.13s 11,019 318 1,214
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 1.82s 7,140 222 0
GPT-5.6 Sol 10.0 10.0 100.0% 0 1.38s 7,140 222 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 5.3 7.2 44.4% 1 29.07s 723 71 7,822
GPT-5.6 Sol 5.9 7.2 55.6% 1 47.86s 723 71 16,201
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 3.07s 477 99 135
GPT-5.6 Sol 10.0 10.0 100.0% 0 3.48s 477 99 214
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 2.27s 660 90 146
GPT-5.6 Sol 10.0 10.0 100.0% 0 2.50s 660 96 249
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 8.2 7.2 88.9% 1 3.44s 642 224 534
GPT-5.6 Sol 8.2 7.2 88.9% 1 2.98s 642 228 566
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 7.56s 5,454 258 294
GPT-5.6 Sol 10.0 10.0 100.0% 0 6.30s 5,454 264 361
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.6 Sol 10.0 10.0 100.0% 0 3.67s 195 30 326
GPT-5.6 Sol 4.7 1.6 66.7% 1 4.94s 195 30 517

Quick Compare

Switch Comparison Pair

Gemini 3 Flash PreviewmediumvsGPT-5.6 SollowGemini 3.5 FlashlowvsGPT-5.6 SolmediumGPT-5.6 SollowvsQwen3.7 MaxmediumGemini 3.5 FlashhighvsGPT-5.6 SollowClaude Fable 5mediumvsGPT-5.6 SollowGemini 3.1 Pro PreviewmediumvsGPT-5.6 SollowGemini 3.5 FlashmediumvsGPT-5.6 SollowGemini 3.5 FlashhighvsGPT-5.6 SolmediumGPT-5.6 SollowvsQwen3.6 Max PreviewmediumClaude Opus 4.8mediumvsGPT-5.6 SollowClaude Opus 4.7mediumvsGPT-5.6 SollowGPT-5.6 SollowvsGLM 5.2medium