KAT Coder AIR V2.5 (high) vs Mistral Small 4 (medium)

Compared models

Recommended model KAT Coder AIR V2.5 (high)

It has the best score here (5.2), while costing about 1.6x less than Mistral Small 4 (medium).

Metric	KAT Coder AIR V2.5 KAT Coder AIR V2.5 high Release: 2026-07-14	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16

Metric	KAT Coder AIR V2.5 KAT Coder AIR V2.5 high Release: 2026-07-14	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16
Score	5.2	5.1
Rank	#288	#291
Reliability	9.8	9.7
Consistency	8.2	7.1
Attempts	69/69	69/69
Tests Correct
Attempt pass rate	40.6%	42.0%
Flaky tests	5	8
Total Runs	69	69
Cost per result	1.091	2.502
Total Cost	$0.077	$0.126
Input Price	$0.000 / 1M	$0.150 / 1M
Output Price	$0.000 / 1M	$0.600 / 1M
Total Input Tokens	50,423	321,955
Output Tokens	4,144	41,555
Reasoning Tokens	110,510	94,946
Response Time (avg)	15.23s	12.68s
Response Time (max)	118.35s	59.15s
Response Time (total)	350.26s	291.66s
Parameters	~35B total (~3B active)	119B total (6B active)
Availability	Closed	Open source

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

high

Reached the allocated time limit (300 seconds) without receiving showcase output.

medium

Category:

Agentic	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	3.0	10.0	0.0%	0		1.10s	0	0	0
Mistral Small 4	5.0	10.0	0.0%	0		54.05s	181,396	1,287	1,617

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	6.9	5.8	75.0%	2		2.49s	615	204	1,290
Mistral Small 4	5.6	3.8	66.7%	3		2.67s	708	4,055	4,778

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	4.3	7.3	11.1%	1		39.07s	6,948	1,166	55,883
Mistral Small 4	4.4	5.1	33.3%	2		39.98s	7,636	11,635	54,715

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	6.5	10.0	50.0%	0		74.48s	23,854	485	28,559
Mistral Small 4	3.0	10.0	0.0%	0		32.40s	116,624	17,890	18,384

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	6.5	10.0	50.0%	0		3.59s	7,776	284	2,140
Mistral Small 4	7.3	5.9	83.3%	1		1.23s	6,171	335	723

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	3.0	10.0	0.0%	0		7.28s	724	1,101	7,738
Mistral Small 4	5.9	7.2	55.6%	1		6.33s	807	3,427	7,871

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	5.1	10.0	0.0%	0		7.10s	516	179	539
Mistral Small 4	4.8	10.0	0.0%	0		2.05s	519	821	828

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	9.8	10.0	100.0%	0		1.51s	699	90	675
Mistral Small 4	7.3	5.8	83.3%	1		1.38s	729	540	1,031

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	3.5	4.4	33.3%	2		2.47s	696	285	1,576
Mistral Small 4	3.4	9.7	0.0%	0		2.17s	735	1,226	2,632

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	10.0	10.0	100.0%	0		4.77s	8,391	330	1,082
Mistral Small 4	10.0	10.0	100.0%	0		3.50s	6,420	321	810

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
KAT Coder AIR V2.5	3.0	10.0	0.0%	0		21.72s	204	20	11,028
Mistral Small 4	3.0	10.0	0.0%	0		5.92s	210	18	1,557

Switch Comparison Pair