AI BENCHY Compare

OpenAI: GPT-5.2 vs Qwen: Qwen3.6 27B

Last updated at: 2026-05-22

Metric	GPT-5.2 GPT-5.2 medium Release: 2025-12-11	Qwen3.6 27B Qwen3.6 27B medium Release: 2026-04-20

Metric	GPT-5.2 GPT-5.2 medium Release: 2025-12-11	Qwen3.6 27B Qwen3.6 27B medium Release: 2026-04-20
Score	7.4	6.6
Rank	#57	#83
Reliability	10.0	9.9
Consistency	8.3	8.1
Tests Correct
Attempt pass rate	70.0%	58.3%
Flaky tests	4	5
Total Runs	60	60
Cost per result	4.081	3.015
Total Cost	$0.490	$0.272
Input Price	$1.750 / 1M	$0.317 / 1M
Output Price	$14.000 / 1M	$3.200 / 1M
Output Tokens	2,754	13,007
Reasoning Tokens	28,303	105,697
Response Time (avg)	16.44s	57.65s
Response Time (max)	77.80s	168.22s
Response Time (total)	213.77s	1153.04s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	6.5	8.0	58.3%	1		7.81s	567	2,002
Qwen3.6 27B	8.3	10.0	75.0%	0		12.62s	582	4,311

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	10.0	10.0	100.0%	0		23.15s	490	8,269
Qwen3.6 27B	6.6	10.0	50.0%	0		165.39s	4,760	26,668

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	10.0	10.0	100.0%	0		14.06s	291	1,757
Qwen3.6 27B	7.0	3.7	66.7%	1		83.07s	2,088	14,689

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	10.0	10.0	100.0%	0		3.15s	234	420
Qwen3.6 27B	3.5	1.4	50.0%	2		37.30s	568	9,404

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	5.9	7.2	55.6%	1		77.80s	42	10,342
Qwen3.6 27B	2.9	7.2	11.1%	1		73.38s	3,510	20,352

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	3.7	9.7	0.0%	0		4.32s	162	269
Qwen3.6 27B	6.5	3.4	66.7%	1		39.53s	81	3,045

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	9.9	10.0	100.0%	0		3.12s	94	614
Qwen3.6 27B	10.0	10.0	100.0%	0		37.96s	346	6,548

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	7.6	7.3	77.8%	1		5.47s	609	938
Qwen3.6 27B	7.7	10.0	66.7%	0		60.21s	281	11,919

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	4.7	1.6	66.7%	1		10.30s	239	469
Qwen3.6 27B	10.0	10.0	100.0%	0		16.88s	390	2,954

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
GPT-5.2	3.0	10.0	0.0%	0		28.18s	26	3,223
Qwen3.6 27B	3.0	10.0	0.0%	0		80.99s	401	5,807

Quick Compare

Switch Comparison Pair

DeepSeek V4 FlashhighFree AvailablevsGPT-5.2medium Qwen3.6 27BmediumvsGrok Build 0.1none Gemini 3.1 Flash LitenonevsQwen3.6 27Bmedium GPT-5.5nonevsQwen3.6 27Bmedium Gemini 3.1 Flash LitelowvsGPT-5.2medium DeepSeek V4 ProhighvsQwen3.6 27Bmedium Gemini 3.1 Flash Lite PreviewnonevsGPT-5.2medium Gemini 3.1 Flash LiteminimalvsQwen3.6 27Bmedium Gemma 4 31BnoneFree AvailablevsQwen3.6 27Bmedium Ring-2.6-1TnonevsGPT-5.2medium Gemini 3.1 Flash Lite PreviewlowvsGPT-5.2medium Gemini 3 Flash PreviewnonevsGPT-5.2medium