AI BENCHY Compare

OpenAI: GPT-5.4 vs xAI: Grok 4.5

Summary

GPT-5.4 (medium) leads on average score with 8.5 vs 7.8. Grok 4.5 (low) has the lower benchmark cost at $0.760 vs $1.210. Grok 4.5 (low) is faster at 15.48s vs 22.35s, with pass rates of 76.2% vs 74.6%.

Recommended modelGPT-5.4 (medium)It has the strongest score in this comparison (8.5) and the best overall balance of cost and response time across all 2 models.

Last updated at: 2026-07-14

Metric	GPT-5.4 GPT-5.4 medium Release: 2026-03-05	Grok 4.5 Grok 4.5 low Release: 2026-07-08

Metric	GPT-5.4 GPT-5.4 medium Release: 2026-03-05	Grok 4.5 Grok 4.5 low Release: 2026-07-08
Score	8.5	7.8
Rank	#20	#41
Reliability	10.0	10.0
Consistency	8.6	9.7
Tests Correct
Attempt pass rate	76.2%	74.6%
Flaky tests	4	1
Total Runs	63	63
Cost per result	8.640	5.061
Total Cost	$1.210	$0.760
Input Price	$2.500 / 1M	$2.000 / 1M
Output Price	$15.000 / 1M	$6.000 / 1M
Total Input Tokens	34,108	58,652
Output Tokens	2,242	1,735
Reasoning Tokens	72,707	105,233
Response Time (avg)	22.35s	15.48s
Response Time (max)	100.41s	205.28s
Response Time (total)	469.29s	324.98s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#20 GPT-5.4

medium

Cost: $0.214
Time: 199.6s
Tokens: 14,349 tok

#41 xAI: Grok 4.5

low

Cost: $0.011
Time: 12.4s
Tokens: 2,018 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	8.3	10.0	75.0%	0		4.11s	606	240	1,511
Grok 4.5	10.0	10.0	100.0%	0		2.75s	2,991	261	3,000

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	8.8	7.8	88.9%	1		44.36s	7,305	433	24,216
Grok 4.5	10.0	10.0	100.0%	0		13.72s	9,579	303	15,641

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	10.0	10.0	100.0%	0		20.57s	11,019	301	3,543
Grok 4.5	3.0	10.0	0.0%	0		8.16s	15,084	261	2,370

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	10.0	10.0	100.0%	0		5.32s	7,140	234	804
Grok 4.5	10.0	10.0	100.0%	0		3.44s	8,937	225	2,549

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	5.3	7.2	44.4%	1		74.27s	619	61	34,748
Grok 4.5	3.0	10.0	0.0%	0		72.64s	2,514	14	70,275

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	4.7	3.1	33.3%	1		4.92s	477	145	321
Grok 4.5	6.1	3.1	66.7%	1		4.88s	1,077	98	1,253

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	10.0	10.0	100.0%	0		3.11s	660	93	897
Grok 4.5	9.8	10.0	100.0%	0		2.80s	1,857	57	1,878

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	8.2	7.2	88.9%	1		9.14s	642	441	3,815
Grok 4.5	10.0	10.0	100.0%	0		3.20s	2,430	217	2,590

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	10.0	10.0	100.0%	0		13.28s	5,445	264	1,031
Grok 4.5	10.0	10.0	100.0%	0		5.83s	13,394	284	871

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.4	3.0	10.0	0.0%	0		13.95s	195	30	1,821
Grok 4.5	3.0	10.0	0.0%	0		13.98s	789	15	4,806

Quick Compare

Switch Comparison Pair

Claude Sonnet 4.6mediumvsGrok 4.5low GPT-5.6 TerramediumvsGrok 4.5low Qwen3.6 PlusmediumvsGrok 4.5low Gemini 3.1 Flash Lite PreviewmediumvsGrok 4.5low Qwen3.5 Plus 2026-04-20mediumvsGrok 4.5low Claude Sonnet 5mediumvsGrok 4.5low Gemini 3.1 Flash LitemediumvsGrok 4.5low Kimi K2.6mediumFree AvailablevsGrok 4.5low Qwen3.5-27BmediumvsGrok 4.5low Qwen3.5-122B-A10BmediumvsGrok 4.5low Claude Opus 4.6mediumvsGrok 4.5low GPT-5.6 LunahighvsGrok 4.5low