AI BENCHY Compare

Kwaipilot: KAT-Coder-Air V2.5 vs Qwen: Qwen3.5-35B-A3B

Summary

Qwen3.5-35B-A3B (medium) leads on average score with 6.3 vs 6.2. KAT-Coder-Air V2.5 (high) has the lower benchmark cost at $0.064 vs $0.401. KAT-Coder-Air V2.5 (high) is faster at 11.02s vs 72.57s, with pass rates of 46.0% vs 69.8%.

Recommended modelKAT-Coder-Air V2.5 (high)Its score stays close to the best score here (6.2 vs 6.3), while costing about 6.3x less than Qwen3.5-35B-A3B (medium).

Last updated at: 2026-07-14

Metric	KAT-Coder-Air V2.5 KAT-Coder-Air V2.5 high Release: 2026-07-14	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24

Metric	KAT-Coder-Air V2.5 KAT-Coder-Air V2.5 high Release: 2026-07-14	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24
Score	6.2	6.3
Rank	#110	#108
Reliability	9.7	10.0
Consistency	7.6	7.5
Tests Correct
Attempt pass rate	46.0%	69.8%
Flaky tests	6	6
Total Runs	63	63
Cost per result	0.910	5.162
Total Cost	$0.064	$0.401
Input Price	$0.150 / 1M	$0.140 / 1M
Output Price	$0.600 / 1M	$1.000 / 1M
Total Input Tokens	41,748	42,196
Output Tokens	3,894	40,630
Reasoning Tokens	91,773	353,577
Response Time (avg)	11.02s	72.57s
Response Time (max)	104.98s	409.98s
Response Time (total)	231.37s	1524.04s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#110 KAT-Coder-Air V2.5

high

Invalid SVG

Cost: $0.000
Time: 300.0s
Tokens: 0 tok

#108 Qwen3.5-35B-A3B

medium

Cost: $0.009
Time: 71.4s
Tokens: 8,631 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	6.9	5.8	75.0%	2		2.49s	615	204	1,290
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		21.13s	672	798	42,652

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	4.3	7.3	11.1%	1		39.07s	6,948	1,166	55,883
Qwen3.5-35B-A3B	5.9	9.3	33.3%	0		206.65s	4,106	23,844	111,462

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	10.0	10.0	100.0%	0		30.61s	15,132	422	8,958
Qwen3.5-35B-A3B	4.7	1.6	66.7%	1		75.34s	20,992	775	12,485

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	6.5	10.0	50.0%	0		3.59s	7,776	284	2,140
Qwen3.5-35B-A3B	7.3	5.9	83.3%	1		59.33s	6,061	235	19,493

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	2.9	7.2	11.1%	1		7.47s	771	914	8,602
Qwen3.5-35B-A3B	4.1	4.4	44.5%	2		88.34s	500	41	46,368

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	5.1	10.0	0.0%	0		7.10s	516	179	539
Qwen3.5-35B-A3B	2.8	1.6	33.3%	1		30.30s	172	20	3,753

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	9.8	10.0	100.0%	0		1.51s	699	90	675
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		24.45s	699	97	17,361

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	3.5	4.4	33.3%	2		2.47s	696	285	1,576
Qwen3.5-35B-A3B	8.2	7.2	88.9%	1		33.13s	597	3,592	26,585

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	10.0	10.0	100.0%	0		4.77s	8,391	330	1,082
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		4.65s	8,193	309	1,365

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT-Coder-Air V2.5	3.0	10.0	0.0%	0		21.72s	204	20	11,028
Qwen3.5-35B-A3B	3.0	10.0	0.0%	0		177.35s	204	10,919	72,053

Quick Compare

Switch Comparison Pair

GPT-5.5nonevsQwen3.5-35B-A3Bmedium GPT-5.6 SolnonevsQwen3.5-35B-A3Bmedium Seed-2.0-LitenonevsKAT-Coder-Air V2.5high KAT-Coder-Air V2.5highvsGPT-5.6 Lunalow KAT-Coder-Air V2.5highvsGPT-5.5none Gemini 2.5 FlashnonevsKAT-Coder-Air V2.5high Gemma 4 31BmediumFree AvailablevsKAT-Coder-Air V2.5high KAT-Coder-Air V2.5highvsGPT-5.6 Solnone Gemini 3.1 Flash LiteminimalvsKAT-Coder-Air V2.5high KAT-Coder-Air V2.5highvsNemotron 3 SupermediumFree Available Seed-2.0-LitenonevsQwen3.5-35B-A3Bmedium GPT-5.6 LunalowvsQwen3.5-35B-A3Bmedium