Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Compared models

Summary

Claude Opus 4.6 vs Claude Sonnet 4.6 vs GPT-5.3-Codex vs Gemini 3.1 Pro Preview benchmark comparisonGemini 3.1 Pro Preview leads on Score with 9.2. Claude Opus 4.6 leads on Reliability with 10.0. GPT-5.3-Codex has the lowest Total Cost at $0.740. GPT-5.3-Codex is fastest at 16.22s.

Recommended model: GPT-5.3-Codex - Its score stays close to the best score here (8.9 vs 9.2), while costing about 2.0x less than the other models in this comparison.

Last updated at: 2026-06-18

Metric Claude Opus 4.6 Claude Opus 4.6 medium Release: 2026-02-05 Claude Sonnet 4.6 Claude Sonnet 4.6 medium Release: 2026-02-17 GPT-5.3-Codex GPT-5.3-Codex medium Release: 2026-02-05 Gemini 3.1 Pro Preview Gemini 3.1 Pro Preview medium Release: 2026-02-19
Score 7.7 7.8 8.9 9.2
Rank #38 #31 #10 #7
Reliability 10.0 10.0 10.0 10.0
Consistency 8.8 9.1 8.5 10.0
Tests Correct
Attempt pass rate 61.9% 65.1% 82.5% 90.5%
Flaky tests 3 2 4 0
Total Runs 63 63 63 63
Cost per result 17.103 10.904 4.932 5.546
Total Cost $2.053 $1.418 $0.740 $1.054
Input Price $5.000 / 1M $3.000 / 1M $1.750 / 1M $2.000 / 1M
Output Price $25.000 / 1M $15.000 / 1M $14.000 / 1M $12.000 / 1M
Total Input Tokens 53,227 49,112 34,299 41,617
Output Tokens 47,446 54,703 2,357 1,977
Reasoning Tokens 24,000 29,970 46,189 78,896
Response Time (avg) 25.89s 17.06s 16.22s 20.14s
Response Time (max) 83.40s 46.35s 100.93s 88.68s
Response Time (total) 362.49s 221.83s 340.67s 281.92s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#38 Claude Opus 4.6

medium
Invalid SVG
Cost
$0.000
Time
300.0s
Tokens
0 tok

#31 Claude Sonnet 4.6

medium
Invalid SVG
Cost
$0.000
Time
300.0s
Tokens
0 tok

#10 GPT-5.3-Codex

medium
Cost
$0.049
Time
54.9s
Tokens
3,580 tok

#7 Gemini 3.1 Pro Preview

medium
Cost
$0.115
Time
87.2s
Tokens
9,629 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 6.4 5.8 66.7% 2 7.45s 840 986 1,071
Claude Sonnet 4.6 6.5 10.0 50.0% 0 2.98s 789 1,046 1,093
GPT-5.3-Codex 8.7 7.9 91.7% 1 4.16s 606 240 1,722
Gemini 3.1 Pro Preview 10.0 10.0 100.0% 0 7.90s 498 112 3,218
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 5.7 7.1 44.4% 1 30.10s 8,522 13,057 4,121
Claude Sonnet 4.6 5.7 6.6 44.4% 1 33.29s 6,995 16,089 3,686
GPT-5.3-Codex 10.0 10.0 100.0% 0 19.50s 7,302 535 10,890
Gemini 3.1 Pro Preview 7.9 9.9 66.7% 0 40.17s 8,124 435 41,247
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 10.0 10.0 100.0% 0 76.66s 20,685 8,178 5,194
Claude Sonnet 4.6 10.0 10.0 100.0% 0 46.35s 18,351 5,871 3,962
GPT-5.3-Codex 10.0 10.0 100.0% 0 19.56s 11,019 364 2,731
Gemini 3.1 Pro Preview 9.5 10.0 100.0% 0 40.61s 17,240 432 9,281
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 10.0 10.0 100.0% 0 7.37s 8,676 691 757
Claude Sonnet 4.6 10.0 10.0 100.0% 0 13.90s 8,676 649 742
GPT-5.3-Codex 10.0 10.0 100.0% 0 3.07s 7,140 234 728
Gemini 3.1 Pro Preview 10.0 10.0 100.0% 0 7.72s 7,265 279 3,904
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 3.0 10.0 0.0% 0 83.40s 674 14,642 8,687
Claude Sonnet 4.6 2.9 7.2 11.1% 1 0ms 471 25,790 16,919
GPT-5.3-Codex 5.9 7.2 55.6% 1 64.31s 813 64 25,308
Gemini 3.1 Pro Preview 7.7 10.0 66.7% 0 32.73s 635 18 12,424
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 10.0 10.0 100.0% 0 5.04s 564 188 292
Claude Sonnet 4.6 10.0 10.0 100.0% 0 4.94s 564 256 433
GPT-5.3-Codex 4.6 10.0 0.0% 0 4.87s 477 187 331
Gemini 3.1 Pro Preview 10.0 10.0 100.0% 0 11.77s 490 108 1,179
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 10.0 10.0 100.0% 0 2.43s 792 266 467
Claude Sonnet 4.6 10.0 10.0 100.0% 0 2.61s 792 318 552
GPT-5.3-Codex 10.0 10.0 100.0% 0 3.04s 660 93 693
Gemini 3.1 Pro Preview 10.0 10.0 100.0% 0 9.56s 621 72 2,236
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 7.7 10.0 66.7% 0 4.71s 816 532 630
Claude Sonnet 4.6 10.0 10.0 100.0% 0 5.31s 816 592 646
GPT-5.3-Codex 9.0 7.9 88.9% 1 5.05s 642 356 1,593
Gemini 3.1 Pro Preview 10.0 10.0 100.0% 0 6.90s 570 235 3,128
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 10.0 10.0 100.0% 0 9.73s 11,454 861 329
Claude Sonnet 4.6 10.0 10.0 100.0% 0 7.48s 11,454 655 351
GPT-5.3-Codex 10.0 10.0 100.0% 0 6.37s 5,445 254 492
Gemini 3.1 Pro Preview 10.0 10.0 100.0% 0 23.15s 6,018 274 982
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Claude Opus 4.6 3.0 10.0 0.0% 0 63.24s 204 8,045 2,452
Claude Sonnet 4.6 3.0 10.0 0.0% 0 30.09s 204 3,437 1,586
GPT-5.3-Codex 2.8 1.6 33.3% 1 14.43s 195 30 1,701
Gemini 3.1 Pro Preview 10.0 10.0 100.0% 0 6.27s 156 12 1,297

Quick Compare

Switch Comparison Pair

Claude Opus 4.6mediumvsStep 3.7 FlashlowClaude Opus 4.6mediumvsDeepSeek V4 ProhighClaude Sonnet 4.6mediumvsStep 3.7 FlashlowGemini 3.1 Pro PreviewmediumvsGPT-5.5lowClaude Opus 4.6mediumvsGPT-5.3 ChatnoneClaude Sonnet 4.6mediumvsDeepSeek V4 ProhighGemini 3.5 FlashlowvsGPT-5.3-CodexmediumClaude Sonnet 4.6mediumvsGPT-5.3 ChatnoneClaude Opus 4.6mediumvsGemini 3 Flash PreviewlowClaude Sonnet 4.6mediumvsDeepSeek V4 FlashhighClaude Sonnet 4.6mediumvsGemini 3 Flash PreviewlowClaude Opus 4.6mediumvsDeepSeek V4 Pronone