Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Compared models

Last updated at: 2026-05-01

Metric GPT-5.4 GPT-5.4 medium Release: 2026-03-05 GPT-5.3-Codex GPT-5.3-Codex medium Release: 2026-02-05 GPT-5.2 GPT-5.2 medium Release: 2025-12-11
Score 8.2 8.6 7.5
Rank #23 #13 #54
Reliability N/A N/A N/A
Consistency 8.7 8.7 8.1
Tests Correct
Attempt pass rate 79.6% 83.3% 72.2%
Flaky tests 3 3 4
Total Runs 54 54 54
Cost per result 6.399 4.405 3.193
Total Cost $0.832 $0.573 $0.352
Input Price $2.500 / 1M $1.750 / 1M $1.750 / 1M
Output Price $15.000 / 1M $14.000 / 1M $14.000 / 1M
Output Tokens 2,169 2,279 2,705
Reasoning Tokens 48,732 35,179 18,977
Response Time (avg) 18.63s 15.38s 14.04s
Response Time (max) 100.41s 100.93s 77.80s
Response Time (total) 335.26s 276.91s 154.41s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 8.3 10.0 75.0% 0 4.11s 240 1,511
GPT-5.3-Codex 8.7 7.9 91.7% 1 4.16s 240 1,722
GPT-5.2 6.5 8.0 58.3% 1 7.81s 567 2,002
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 13.03s 389 2,045
GPT-5.3-Codex 10.0 10.0 100.0% 0 8.95s 491 1,530
GPT-5.2 10.0 10.0 100.0% 0 15.12s 467 2,166
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 20.57s 301 3,543
GPT-5.3-Codex 10.0 10.0 100.0% 0 19.56s 364 2,731
GPT-5.2 10.0 10.0 100.0% 0 14.06s 291 1,757
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 5.32s 234 804
GPT-5.3-Codex 10.0 10.0 100.0% 0 3.07s 234 728
GPT-5.2 10.0 10.0 100.0% 0 3.15s 234 420
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 5.3 7.2 44.4% 1 74.27s 61 34,748
GPT-5.3-Codex 5.9 7.2 55.6% 1 64.31s 64 25,308
GPT-5.2 5.9 7.2 55.6% 1 77.80s 42 10,342
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 4.7 3.1 33.3% 1 4.92s 145 321
GPT-5.3-Codex 4.6 10.0 0.0% 0 4.87s 187 331
GPT-5.2 3.7 9.7 0.0% 0 4.32s 162 269
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 3.11s 93 897
GPT-5.3-Codex 10.0 10.0 100.0% 0 3.04s 93 693
GPT-5.2 9.9 10.0 100.0% 0 3.12s 94 614
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 8.2 7.2 88.9% 1 9.13s 442 3,832
GPT-5.3-Codex 9.0 7.9 88.9% 1 5.12s 352 1,644
GPT-5.2 7.7 7.3 77.8% 1 5.47s 609 938
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 13.28s 264 1,031
GPT-5.3-Codex 10.0 10.0 100.0% 0 6.37s 254 492
GPT-5.2 4.7 1.6 66.7% 1 10.30s 239 469

Quick Compare

Switch Comparison Pair

GPT-5.2mediumvsQwen3.6 Max PreviewnoneClaude Sonnet 4.6nonevsGPT-5.2mediumGPT-5.3-CodexmediumvsHY3 PreviewhighFree AvailableGPT-5.4mediumvsHY3 PreviewlowFree AvailableGemini 3 Flash PreviewnonevsGPT-5.4mediumGemini 3.1 Flash Lite PreviewlowvsGPT-5.4mediumGemini 3 Flash PreviewlowvsGPT-5.3-CodexmediumGemini 3.1 Flash Lite PreviewnonevsGPT-5.4mediumDeepSeek V4 FlashhighvsGPT-5.2mediumDeepSeek V4 FlashhighvsGPT-5.4mediumGPT-5.4mediumvsHY3 PreviewhighFree AvailableGemini 3.1 Flash Lite PreviewnonevsGPT-5.2medium