AI BENCHY Compare

Kwaipilot: KAT-Coder-Air V2.5 vs Qwen: Qwen3.5-35B-A3B

Summary

Qwen3.5-35B-A3B (medium) leads on average score with 6.3 vs 6.0. KAT-Coder-Air V2.5 (low) has the lower benchmark cost at $0.030 vs $0.401. KAT-Coder-Air V2.5 (low) is faster at 6.47s vs 72.57s, with pass rates of 46.0% vs 69.8%.

Recommended modelKAT-Coder-Air V2.5 (low)Its score stays close to the best score here (6.0 vs 6.3), while costing about 13.4x less than Qwen3.5-35B-A3B (medium).

Last updated at: 2026-07-14

Metric	KAT-Coder-Air V2.5 KAT-Coder-Air V2.5 low Release: 2026-07-14	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24

Metric	KAT-Coder-Air V2.5 KAT-Coder-Air V2.5 low Release: 2026-07-14	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24
Score	6.0	6.3
Rank	#124	#108
Reliability	9.7	10.0
Consistency	7.6	7.5
Tests Correct
Attempt pass rate	46.0%	69.8%
Flaky tests	6	6
Total Runs	63	63
Cost per result	0.427	5.162
Total Cost	$0.030	$0.401
Input Price	$0.150 / 1M	$0.140 / 1M
Output Price	$0.600 / 1M	$1.000 / 1M
Total Input Tokens	42,464	42,196
Output Tokens	4,227	40,630
Reasoning Tokens	34,864	353,577
Response Time (avg)	6.47s	72.57s
Response Time (max)	25.49s	409.98s
Response Time (total)	135.80s	1524.04s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#124 KAT-Coder-Air V2.5

low

Cost: $0.004
Time: 29.9s
Tokens: 6,008 tok

#108 Qwen3.5-35B-A3B

medium

Cost: $0.009
Time: 71.4s
Tokens: 8,631 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	7.3	5.8	83.3%	2		3.50s	672	422	1,548
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		21.13s	672	798	42,652

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	3.5	7.3	11.1%	1		11.06s	7,893	2,001	13,623
Qwen3.5-35B-A3B	5.9	9.3	33.3%	0		206.65s	4,106	23,844	111,462

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	10.0	10.0	100.0%	0		25.49s	15,132	368	5,707
Qwen3.5-35B-A3B	4.7	1.6	66.7%	1		75.34s	20,992	775	12,485

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	6.5	10.0	50.0%	0		2.82s	7,776	261	1,069
Qwen3.5-35B-A3B	7.3	5.9	83.3%	1		59.33s	6,061	235	19,493

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	2.9	7.2	11.1%	1		4.99s	553	310	3,332
Qwen3.5-35B-A3B	4.1	4.4	44.5%	2		88.34s	500	41	46,368

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	5.0	10.0	0.0%	0		10.11s	516	142	596
Qwen3.5-35B-A3B	2.8	1.6	33.3%	1		30.30s	172	20	3,753

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	9.8	10.0	100.0%	0		1.64s	699	89	640
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		24.45s	699	97	17,361

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	3.1	4.6	22.2%	2		1.57s	696	294	796
Qwen3.5-35B-A3B	8.2	7.2	88.9%	1		33.13s	597	3,592	26,585

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	10.0	10.0	100.0%	0		4.47s	8,391	330	1,065
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		4.65s	8,193	309	1,365

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	3.0	10.0	0.0%	0		19.98s	136	10	6,488
Qwen3.5-35B-A3B	3.0	10.0	0.0%	0		177.35s	204	10,919	72,053

Quick Compare

Switch Comparison Pair