Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

OpenAI: GPT-5.4 vs StepFun: Step 3.7 Flash

Last updated at: 2026-06-03

Metric GPT-5.4 GPT-5.4 medium Release: 2026-03-05 Step 3.7 Flash Step 3.7 Flash high Release: 2026-05-29
Score 7.9 7.2
Rank #27 #66
Reliability 10.0 10.0
Consistency 8.5 8.6
Tests Correct
Attempt pass rate 75.0% 63.3%
Flaky tests 4 3
Total Runs 60 60
Cost per result 8.765 8.723
Total Cost $1.140 $0.960
Input Price $2.500 / 1M $0.200 / 1M
Output Price $15.000 / 1M $1.150 / 1M
Total Input Tokens 31,489 35,763
Output Tokens 2,221 828,084
Reasoning Tokens 68,486 0
Response Time (avg) 22.31s 49.43s
Response Time (max) 100.41s 192.75s
Response Time (total) 446.17s 988.58s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 8.3 10.0 75.0% 0 4.11s 606 240 1,511
Step 3.7 Flash 10.0 10.0 100.0% 0 13.40s 696 42,656 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 8.2 6.7 83.3% 1 54.98s 4,686 412 19,995
Step 3.7 Flash 3.7 8.3 0.0% 0 126.82s 3,429 164,069 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 20.57s 11,019 301 3,543
Step 3.7 Flash 10.0 10.0 100.0% 0 13.01s 13,638 8,802 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 5.32s 7,140 234 804
Step 3.7 Flash 10.0 10.0 100.0% 0 14.72s 7,368 23,113 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 5.3 7.2 44.4% 1 74.27s 619 61 34,748
Step 3.7 Flash 4.1 4.4 44.5% 2 149.64s 783 410,502 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 4.7 3.1 33.3% 1 4.92s 477 145 321
Step 3.7 Flash 5.5 10.0 0.0% 0 4.17s 510 2,862 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 3.11s 660 93 897
Step 3.7 Flash 9.8 10.0 100.0% 0 1.52s 705 2,010 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 8.2 7.2 88.9% 1 9.14s 642 441 3,815
Step 3.7 Flash 5.3 7.2 44.4% 1 10.22s 711 25,422 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 10.0 10.0 100.0% 0 13.28s 5,445 264 1,031
Step 3.7 Flash 10.0 10.0 100.0% 0 2.79s 7,701 1,172 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
GPT-5.4 3.0 10.0 0.0% 0 13.95s 195 30 1,821
Step 3.7 Flash 3.0 10.0 0.0% 0 149.34s 222 147,476 0

Quick Compare

Switch Comparison Pair

GPT-5.4mediumvsQwen3.7 MaxnoneGPT-5.4 NanomediumvsStep 3.7 FlashhighMiniMax M3mediumvsStep 3.7 FlashhighGPT-5 MinimediumvsStep 3.7 FlashhighClaude Opus 4.6mediumvsStep 3.7 FlashhighGemini 3.5 FlashminimalvsGPT-5.4mediumQwen3.5-35B-A3BmediumvsStep 3.7 FlashhighQwen3.6 Max PreviewnonevsStep 3.7 FlashhighClaude Opus 4.8nonevsStep 3.7 FlashhighGemini 3.5 FlashnonevsGPT-5.4mediumStep 3.7 FlashhighvsMiMo-V2-FlashmediumSeed-2.0-MinimediumvsStep 3.7 Flashhigh