Navigate
Advertise here

KAT Coder AIR V2.5 (low) vs GPT-4o-mini (default)

The average score is effectively tied at 5.0 vs 5.0. GPT-4o-mini (default) has the lower benchmark cost at $0.024 vs $0.046. GPT-4o-mini (default) is faster at 4.16s vs 10.21s, with pass rates of 42.0% vs 21.7%.

Last updated at: 2026-11-13

Compared models

Rank
#400
Total Output Tokens
60,414
Response Time (avg)
10.21s
Total Cost
$0.046
Rank
#399
Total Output Tokens
3,627
Response Time (avg)
4.16s
Total Cost
$0.024
Recommended model GPT-4o-mini (default)

It has the best score here (5.0), while costing about 1.9x less than KAT Coder AIR V2.5 (low).

Detailed comparison

Metric KAT Coder AIR V2.5 KAT Coder AIR V2.5 low Release: 2026-07-14 GPT-4o-mini GPT-4o-mini default Release: 2024-07-18
Score 5.0 5.0
Rank #400 #399
Reliability 9.9 10.0
Consistency 7.8 9.9
Attempts 69/69 69/69
Tests Correct
Attempt pass rate 42.0% 21.7%
Flaky tests 6 0
Total Runs 69 69
Cost per result 0.649 0.480
Total Cost $0.046 $0.024
Input Price $0.150 / 1M $0.150 / 1M
Output Price $0.600 / 1M $0.600 / 1M
Cache Read Price N/A $0.075 / 1M
Cache Write Price N/A N/A
Total Input Tokens 61,094 145,180
Output Tokens 6,532 3,627
Reasoning Tokens 53,882 0
Response Time (avg) 10.21s 4.16s
Response Time (max) 86.23s 39.99s
Response Time (total) 234.86s 70.70s
Parameters ~35B total (~3B active) ~8B
Availability Closed Closed

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#400 KAT Coder AIR V2.5

low
Cost
$0.004
Time
29.9s
Tokens
6,008 tok

#399 GPT-4o-mini

default
Cost
$0.001
Time
6.6s
Tokens
742 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
KAT Coder AIR V2.5 3.5 7.3 11.1% 1 11.06s 7,893 2,001 13,623
GPT-4o-mini 3.2 9.6 0.0% 0 1.63s 7,314 367 0

Switch Comparison Pair