Navigate
Advertise here

KAT Coder AIR V2.5 (default) vs Nemotron 3.5 Lightning (low)

Nemotron 3.5 Lightning (low) leads on average score with 4.6 vs 4.4. KAT Coder AIR V2.5 (default) has the lower benchmark cost at $0.070 vs $0.175. KAT Coder AIR V2.5 (default) is faster at 11.96s vs 83.19s, with pass rates of 34.8% vs 40.6%.

Last updated at: 2026-11-22

Compared models

Rank
#465
Total Output Tokens
97,752
Response Time (avg)
11.96s
Total Cost
$0.070
Rank
#456
Total Output Tokens
647,336
Response Time (avg)
83.19s
Total Cost
$0.175
Recommended model KAT Coder AIR V2.5 (default)

Its score stays close to the best score here (4.4 vs 4.6), while costing about 2.5x less than Nemotron 3.5 Lightning (low).

Detailed comparison

Metric KAT Coder AIR V2.5 KAT Coder AIR V2.5 default Release: 2026-07-14 Nemotron 3.5 Lightning Nemotron 3.5 Lightning low Release: 2026-08-11 Free Available
Score 4.4 4.6
Rank #465 #456
Reliability 10.0 9.5
Consistency 7.4 6.8
Attempts 69/69 69/69
Tests Correct
Attempt pass rate 34.8% 40.6%
Flaky tests 7 9
Total Runs 69 69
Cost per result 1.382 0.000
Total Cost $0.070 $0.175
Input Price $0.150 / 1M $0.060 / 1M
Output Price $0.600 / 1M $0.160 / 1M
Cache Read Price N/A $0.030 / 1M
Cache Write Price N/A N/A
Total Input Tokens 69,376 207,883
Output Tokens 7,792 144,718
Reasoning Tokens 89,960 525,897
Response Time (avg) 11.96s 83.19s
Response Time (max) 121.05s 597.61s
Response Time (total) 275.18s 1913.41s
Parameters ~35B total (~3B active) 30B total (3B active)
Availability Closed Weights available

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#465 KAT Coder AIR V2.5

default
Cost
$0.002
Time
14.5s
Tokens
2,619 tok

#456 Nemotron 3.5 Lightning

low
Cost
$0.000
Time
32.2s
Tokens
6,089 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
KAT Coder AIR V2.5 3.3 9.6 0.0% 0 14.31s 6,472 2,032 14,886
Nemotron 3.5 Lightning 4.0 7.8 11.1% 1 224.42s 7,522 72,614 262,558

Switch Comparison Pair