AI BENCHY Compare

OpenAI: GPT-5.2 Chat vs xAI: Grok Build 0.1

Last updated at: 2026-05-21

Metric	GPT-5.2 Chat GPT-5.2 Chat none Release: 2025-12-11	Grok Build 0.1 Grok Build 0.1 medium Release: 2026-05-21

Metric	GPT-5.2 Chat GPT-5.2 Chat none Release: 2025-12-11	Grok Build 0.1 Grok Build 0.1 medium Release: 2026-05-21
Score	7.6	7.8
Rank	#47	#41
Reliability	10.0	10.0
Consistency	8.8	8.9
Tests Correct
Attempt pass rate	71.9%	71.9%
Flaky tests	3	3
Total Runs	57	57
Cost per result	2.572	4.064
Total Cost	$0.309	$0.488
Input Price	$1.750 / 1M	$1.000 / 1M
Output Price	$14.000 / 1M	$2.000 / 1M
Output Tokens	18,585	1,947
Reasoning Tokens	0	223,372
Response Time (avg)	6.85s	22.28s
Response Time (max)	38.52s	88.28s
Response Time (total)	130.06s	423.30s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	8.7	7.9	91.7%	1		3.40s	1,807	0
Grok Build 0.1	10.0	10.0	100.0%	0		5.46s	195	9,825

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	10.0	10.0	100.0%	0		8.97s	1,345	0
Grok Build 0.1	7.3	3.7	66.7%	1		30.98s	354	17,734

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	10.0	10.0	100.0%	0		9.12s	1,243	0
Grok Build 0.1	10.0	10.0	100.0%	0		30.81s	231	18,779

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	10.0	10.0	100.0%	0		3.05s	980	0
Grok Build 0.1	10.0	10.0	100.0%	0		7.76s	180	10,343

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	5.3	10.0	33.3%	0		17.78s	7,810	0
Grok Build 0.1	5.3	10.0	33.3%	0		77.75s	501	111,807

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	4.4	3.0	33.3%	1		3.20s	335	0
Grok Build 0.1	3.8	2.5	33.3%	1		10.14s	78	5,386

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	7.3	5.9	83.3%	1		5.46s	1,528	0
Grok Build 0.1	9.8	10.0	100.0%	0		9.62s	57	12,436

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	7.7	10.0	66.7%	0		4.42s	1,743	0
Grok Build 0.1	6.2	7.5	55.6%	1		8.67s	161	15,476

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	10.0	10.0	100.0%	0		4.68s	555	0
Grok Build 0.1	10.0	10.0	100.0%	0		9.40s	180	5,319

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	3.0	10.0	0.0%	0		6.89s	1,239	0
Grok Build 0.1	3.0	10.0	0.0%	0		26.07s	10	16,267

Quick Compare

Switch Comparison Pair

Kimi K2.6mediumvsGPT-5.2 Chatnone GPT-5.2 ChatnonevsStep 3.5 Flashmedium Gemini 3.1 Flash LitelowvsGPT-5.2 Chatnone GPT-5.2 ChatnonevsQwen3.5-Flashmedium Gemma 4 26B A4BmediumFree AvailablevsGPT-5.2 Chatnone GPT-5.2 ChatnonevsGLM 5.1medium Gemini 3.1 Flash Lite PreviewnonevsGrok Build 0.1medium GPT-5.2 ChatnonevsMiMo-V2-Promedium DeepSeek V4 FlashhighFree AvailablevsGPT-5.2 Chatnone Gemini 3.1 Flash Lite PreviewlowvsGrok Build 0.1medium Gemini 3 Flash PreviewnonevsGrok Build 0.1medium GPT-5.2 ChatnonevsGLM 5V Turbomedium