AI BENCHY Compare

ByteDance Seed: Seed-2.0-Lite vs Kwaipilot: KAT-Coder-Air V2.5

Summary

The average score is effectively tied at 6.2 vs 6.2. Seed-2.0-Lite has the lower benchmark cost at $0.019 vs $0.064. Seed-2.0-Lite is faster at 2.49s vs 11.02s, with pass rates of 46.0% vs 46.0%.

Recommended modelSeed-2.0-LiteIt has the best score here (6.2), while costing about 3.5x less than KAT-Coder-Air V2.5 (high).

Last updated at: 2026-07-14

Metric	Seed-2.0-Lite Seed-2.0-Lite none Release: 2026-02-14	KAT-Coder-Air V2.5 KAT-Coder-Air V2.5 high Release: 2026-07-14

Metric	Seed-2.0-Lite Seed-2.0-Lite none Release: 2026-02-14	KAT-Coder-Air V2.5 KAT-Coder-Air V2.5 high Release: 2026-07-14
Score	6.2	6.2
Rank	#112	#110
Reliability	10.0	9.7
Consistency	8.4	7.6
Tests Correct
Attempt pass rate	46.0%	46.0%
Flaky tests	4	6
Total Runs	63	63
Cost per result	0.228	0.910
Total Cost	$0.019	$0.064
Input Price	$0.250 / 1M	$0.150 / 1M
Output Price	$2.000 / 1M	$0.600 / 1M
Total Input Tokens	46,573	41,748
Output Tokens	3,259	3,894
Reasoning Tokens	0	91,773
Response Time (avg)	2.49s	11.02s
Response Time (max)	6.70s	104.98s
Response Time (total)	52.26s	231.37s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#112 Seed-2.0-Lite

none

Cost: $0.005
Time: 83.8s
Tokens: 2,311 tok

#110 KAT-Coder-Air V2.5

high

Invalid SVG

Cost: $0.000
Time: 300.0s
Tokens: 0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	3.0	5.9	16.7%	2		2.43s	894	709	0
KAT-Coder-Air V2.5	6.9	5.8	75.0%	2		2.49s	615	204	1,290

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	5.6	10.0	33.3%	0		2.83s	8,215	410	0
KAT-Coder-Air V2.5	4.3	7.3	11.1%	1		39.07s	6,948	1,166	55,883

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	3.0	10.0	0.0%	0		6.59s	16,215	498	0
KAT-Coder-Air V2.5	10.0	10.0	100.0%	0		30.61s	15,132	422	8,958

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	10.0	10.0	100.0%	0		1.82s	8,538	246	0
KAT-Coder-Air V2.5	6.5	10.0	50.0%	0		3.59s	7,776	284	2,140

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	3.6	7.2	22.2%	1		1.33s	939	17	0
KAT-Coder-Air V2.5	2.9	7.2	11.1%	1		7.47s	771	914	8,602

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	10.0	10.0	100.0%	0		3.45s	570	294	0
KAT-Coder-Air V2.5	5.1	10.0	0.0%	0		7.10s	516	179	539

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	10.0	10.0	100.0%	0		1.06s	810	73	0
KAT-Coder-Air V2.5	9.8	10.0	100.0%	0		1.51s	699	90	675

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	5.3	7.2	44.4%	1		2.78s	858	709	0
KAT-Coder-Air V2.5	3.5	4.4	33.3%	2		2.47s	696	285	1,576

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	10.0	10.0	100.0%	0		3.94s	9,270	292	0
KAT-Coder-Air V2.5	10.0	10.0	100.0%	0		4.77s	8,391	330	1,082

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Seed-2.0-Lite	3.0	10.0	0.0%	0		1.96s	264	11	0
KAT-Coder-Air V2.5	3.0	10.0	0.0%	0		21.72s	204	20	11,028

Quick Compare

Switch Comparison Pair