AI BENCHY Compare

OpenAI: GPT-5.2 Chat vs Qwen: Qwen3.5-27B

Last updated at: 2026-06-03

Metric	GPT-5.2 Chat GPT-5.2 Chat none Release: 2025-12-11	Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24

Metric	GPT-5.2 Chat GPT-5.2 Chat none Release: 2025-12-11	Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24
Score	7.9	7.9
Rank	#30	#24
Reliability	10.0	10.0
Consistency	8.9	8.9
Tests Correct
Attempt pass rate	73.3%	73.3%
Flaky tests	3	3
Total Runs	60	60
Cost per result	2.703	4.532
Total Cost	$0.352	$0.488
Input Price	$1.750 / 1M	$0.195 / 1M
Output Price	$14.000 / 1M	$1.560 / 1M
Total Input Tokens	31,593	39,329
Output Tokens	21,144	2,569
Reasoning Tokens	0	304,894
Response Time (avg)	6.82s	60.09s
Response Time (max)	38.52s	177.36s
Response Time (total)	136.34s	1201.89s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	8.7	7.9	91.7%	1		3.40s	606	1,807	0
Qwen3.5-27B	8.7	7.9	91.7%	1		19.75s	672	569	31,505

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	8.2	6.7	83.3%	1		8.05s	4,686	4,131	0
Qwen3.5-27B	7.0	9.8	50.0%	0		123.86s	5,060	416	64,993

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	10.0	10.0	100.0%	0		9.12s	11,019	1,243	0
Qwen3.5-27B	10.0	10.0	100.0%	0		163.96s	14,946	483	9,991

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	10.0	10.0	100.0%	0		3.05s	7,140	980	0
Qwen3.5-27B	10.0	10.0	100.0%	0		30.26s	7,782	270	16,150

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	5.3	10.0	33.3%	0		17.78s	723	7,810	0
Qwen3.5-27B	5.3	10.0	33.3%	0		79.53s	553	43	52,368

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	4.4	3.0	33.3%	1		3.20s	477	335	0
Qwen3.5-27B	6.1	3.1	66.7%	1		101.41s	524	70	23,147

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	9.8	10.0	100.0%	0		5.51s	660	1,441	0
Qwen3.5-27B	10.0	10.0	100.0%	0		19.66s	699	97	11,638

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	7.7	10.0	66.7%	0		4.10s	642	1,603	0
Qwen3.5-27B	8.2	7.7	77.8%	1		59.60s	696	242	70,096

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	10.0	10.0	100.0%	0		4.68s	5,445	555	0
Qwen3.5-27B	10.0	10.0	100.0%	0		7.45s	8,193	348	1,323

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
GPT-5.2 Chat	3.0	10.0	0.0%	0		6.89s	195	1,239	0
Qwen3.5-27B	3.0	10.0	0.0%	0		85.11s	204	31	23,683

Quick Compare

Switch Comparison Pair

Gemini 3 PRO PreviewmediumvsGPT-5.2 Chatnone Gemma 4 26B A4BmediumFree AvailablevsGPT-5.2 Chatnone GPT-5.2 ChatnonevsGLM 5 Turbomedium Gemini 3.5 FlashminimalvsQwen3.5-27Bmedium GPT-5.2 ChatnonevsStep 3.7 Flashmedium GPT-5.2 ChatnonevsGrok 4.3medium Gemini 3.5 FlashminimalvsGPT-5.2 Chatnone GPT-5.2 ChatnonevsQwen3.6 Plusmedium Gemini 3.5 FlashnonevsQwen3.5-27Bmedium Gemini 3.1 Flash Lite PreviewmediumvsGPT-5.2 Chatnone Gemini 3.1 Flash LitemediumvsGPT-5.2 Chatnone Gemini 2.5 FlashmediumvsGPT-5.2 Chatnone