- Rank
- #288
- Total Output Tokens
- 114,654
- Response Time (avg)
- 15.23s
- Total Cost
- $0.077
KAT Coder AIR V2.5 (high) vs Mistral Small 4
The average score is effectively tied at 5.2 vs 5.1. Mistral Small 4 has the lower benchmark cost at $0.047 vs $0.077. Mistral Small 4 is faster at 3.12s vs 15.23s, with pass rates of 40.6% vs 24.6%.
Compared models
- Rank
- #292
- Total Output Tokens
- 10,913
- Response Time (avg)
- 3.12s
- Total Cost
- $0.047
Recommended model
Mistral Small 4
It has the best score here (5.1), while costing about 1.6x less than KAT Coder AIR V2.5 (high).
Detailed comparison
| Metric | KAT Coder AIR V2.5 KAT Coder AIR V2.5 high | Mistral Small 4 Mistral Small 4 none |
|---|---|---|
| Score | 5.2 | 5.1 |
| Rank | #288 | #292 |
| Reliability | 9.8 | 10.0 |
| Consistency | 8.2 | 9.6 |
| Attempts | 69/69 | 69/69 |
| Tests Correct | ||
| Attempt pass rate | 40.6% | 24.6% |
| Flaky tests | 5 | 1 |
| Total Runs | 69 | 69 |
| Cost per result | 1.091 | 0.928 |
| Total Cost | $0.077 | $0.047 |
| Input Price | $0.000 / 1M | $0.150 / 1M |
| Output Price | $0.000 / 1M | $0.600 / 1M |
| Total Input Tokens | 50,423 | 265,602 |
| Output Tokens | 4,144 | 10,913 |
| Reasoning Tokens | 110,510 | 0 |
| Response Time (avg) | 15.23s | 3.12s |
| Response Time (max) | 118.35s | 45.46s |
| Response Time (total) | 350.26s | 71.87s |
| Parameters | ~35B total (~3B active) | 119B total (6B active) |
| Availability | Closed | Open source |
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#288 KAT Coder AIR V2.5
high
Reached the allocated time limit (300 seconds) without receiving showcase output.
- Cost
- $0.000
- Time
- 300.0s
- Tokens
- 0 tok
#292 Mistral Small 4
none- Cost
- $0.002
- Time
- 10.4s
- Tokens
- 2,370 tok
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
| Agentic | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 3.0 | 10.0 | 0.0% | 0 | 1.10s | 0 | 0 | 0 | |
| Mistral Small 4 | 5.0 | 10.0 | 0.0% | 0 | 45.46s | 160,885 | 1,101 | 0 |
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 6.9 | 5.8 | 75.0% | 2 | 2.49s | 615 | 204 | 1,290 | |
| Mistral Small 4 | 3.4 | 7.9 | 16.7% | 1 | 395ms | 708 | 182 | 0 |
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 4.3 | 7.3 | 11.1% | 1 | 39.07s | 6,948 | 1,166 | 55,883 | |
| Mistral Small 4 | 3.7 | 9.7 | 0.0% | 0 | 901ms | 7,636 | 619 | 0 |
| Combined | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 6.5 | 10.0 | 50.0% | 0 | 74.48s | 23,854 | 485 | 28,559 | |
| Mistral Small 4 | 3.0 | 10.0 | 0.0% | 0 | 7.44s | 79,039 | 8,107 | 0 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 6.5 | 10.0 | 50.0% | 0 | 3.59s | 7,776 | 284 | 2,140 | |
| Mistral Small 4 | 10.0 | 10.0 | 100.0% | 0 | 822ms | 7,914 | 261 | 0 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 3.0 | 10.0 | 0.0% | 0 | 7.28s | 724 | 1,101 | 7,738 | |
| Mistral Small 4 | 5.3 | 10.0 | 33.3% | 0 | 377ms | 807 | 28 | 0 |
| General Intelligence | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 5.1 | 10.0 | 0.0% | 0 | 7.10s | 516 | 179 | 539 | |
| Mistral Small 4 | 4.0 | 10.0 | 0.0% | 0 | 729ms | 519 | 205 | 0 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 9.8 | 10.0 | 100.0% | 0 | 1.51s | 699 | 90 | 675 | |
| Mistral Small 4 | 6.5 | 10.0 | 50.0% | 0 | 380ms | 729 | 69 | 0 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 3.5 | 4.4 | 33.3% | 2 | 2.47s | 696 | 285 | 1,576 | |
| Mistral Small 4 | 3.1 | 9.9 | 0.0% | 0 | 399ms | 735 | 111 | 0 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 10.0 | 10.0 | 100.0% | 0 | 4.77s | 8,391 | 330 | 1,082 | |
| Mistral Small 4 | 10.0 | 10.0 | 100.0% | 0 | 1.40s | 6,420 | 213 | 0 |
| Trivia | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| KAT Coder AIR V2.5 | 3.0 | 10.0 | 0.0% | 0 | 21.72s | 204 | 20 | 11,028 | |
| Mistral Small 4 | 3.0 | 10.0 | 0.0% | 0 | 397ms | 210 | 17 | 0 |
Quick Compare
Switch Comparison Pair
Mercury 2.5lowvsMistral Small 4noneKAT Coder AIR V2.5mediumvsMistral Small 4noneLing 3.0 FlashnonevsKAT Coder AIR V2.5highKAT Coder AIR V2.5highvsInkling SmalllowFree AvailableKAT Coder AIR V2.5highvsMistral Small 4mediumMistral Small 4nonevsNemotron 3.5 LightninghighFree AvailableMercury 2.5lowvsKAT Coder AIR V2.5highKAT Coder AIR V2.5highvsMiMo-V2.5-PrononeMistral Small 4nonevsLaguna S 2.1mediumFree AvailableMistral Small 4nonevsInkling SmalllowFree AvailableKAT Coder AIR V2.5highvsQwen3.6 35B A3BnoneKAT Coder AIR V2.5highvsLaguna XS 2.1noneFree Available