AI BENCHY Compare

Kwaipilot: KAT-Coder-Pro V2.5 vs OpenAI: GPT-5.5

Summary

GPT-5.5 (low) leads on average score with 9.3 vs 6.8. KAT-Coder-Pro V2.5 has the lower benchmark cost at $0.220 vs $0.907. GPT-5.5 (low) is faster at 9.76s vs 10.81s, with pass rates of 71.4% vs 85.7%.

Recommended modelGPT-5.5 (low)It has the strongest score in this comparison (9.3) and the best overall balance of cost and response time across all 2 models.

Last updated at: 2026-07-14

Metric	KAT-Coder-Pro V2.5 KAT-Coder-Pro V2.5 none Release: 2026-07-14	GPT-5.5 GPT-5.5 low Release: 2026-04-24

Metric	KAT-Coder-Pro V2.5 KAT-Coder-Pro V2.5 none Release: 2026-07-14	GPT-5.5 GPT-5.5 low Release: 2026-04-24
Score	6.8	9.3
Rank	#89	#7
Reliability	10.0	10.0
Consistency	7.3	10.0
Tests Correct
Attempt pass rate	71.4%	85.7%
Flaky tests	7	0
Total Runs	63	63
Cost per result	1.992	5.035
Total Cost	$0.220	$0.907
Input Price	$0.740 / 1M	$5.000 / 1M
Output Price	$2.960 / 1M	$30.000 / 1M
Total Input Tokens	42,749	34,209
Output Tokens	63,338	2,046
Reasoning Tokens	0	22,460
Response Time (avg)	10.81s	9.76s
Response Time (max)	53.57s	56.19s
Response Time (total)	227.02s	204.92s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#89 KAT-Coder-Pro V2.5

none

Cost: $0.010
Time: 29.0s
Tokens: 3,439 tok

#7 GPT-5.5

low

Cost: $0.068
Time: 37.0s
Tokens: 2,339 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	8.7	7.9	91.7%	1		4.58s	672	3,998	0
GPT-5.5	10.0	10.0	100.0%	0		4.41s	606	238	1,020

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	6.1	4.7	66.7%	2		22.52s	7,893	22,440	0
GPT-5.5	10.0	10.0	100.0%	0		15.04s	7,302	423	6,402

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	4.7	1.6	66.7%	1		30.76s	15,131	5,366	0
GPT-5.5	10.0	10.0	100.0%	0		9.56s	11,019	303	717

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	10.0	10.0	100.0%	0		5.07s	7,776	2,361	0
GPT-5.5	10.0	10.0	100.0%	0		3.28s	7,140	228	157

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	3.6	7.2	22.2%	1		21.64s	771	20,972	0
GPT-5.5	5.3	10.0	33.3%	0		28.05s	723	69	11,609

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	4.8	3.2	33.3%	1		5.16s	516	1,067	0
GPT-5.5	10.0	10.0	100.0%	0		5.17s	477	133	245

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	9.8	10.0	100.0%	0		2.61s	699	1,084	0
GPT-5.5	9.9	10.0	100.0%	0		3.74s	660	93	415

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	8.2	7.2	88.9%	1		3.03s	696	1,929	0
GPT-5.5	10.0	10.0	100.0%	0		4.74s	642	279	954

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	10.0	10.0	100.0%	0		5.93s	8,391	1,103	0
GPT-5.5	10.0	10.0	100.0%	0		4.96s	5,445	250	101

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Pro V2.5	3.0	10.0	0.0%	0		9.91s	204	3,018	0
GPT-5.5	3.0	10.0	0.0%	0		10.06s	195	30	840

Quick Compare

Switch Comparison Pair