- Rank
- #64
- Total Output Tokens
- 61,650
- Response Time (avg)
- 7.51s
- Total Cost
- $0.061
Claude Haiku 5.5 (low) vs KAT Coder AIR V2.5 (high)
Claude Haiku 5.5 (low) leads on average score with 8.6 vs 5.2. Claude Haiku 5.5 (low) has the lower benchmark cost at $0.061 vs $0.077. Claude Haiku 5.5 (low) is faster at 7.51s vs 15.23s, with pass rates of 76.8% vs 40.6%.
Compared models
- Rank
- #309
- Total Output Tokens
- 114,654
- Response Time (avg)
- 15.23s
- Total Cost
- $0.077
Recommended model
Claude Haiku 5.5 (low)
It has the best score here (8.6), while responding about 2.0x faster than KAT Coder AIR V2.5 (high).
Detailed comparison
| Metric | Claude Haiku 5.5 Claude Haiku 5.5 low | KAT Coder AIR V2.5 KAT Coder AIR V2.5 high |
|---|---|---|
| Score | 8.6 | 5.2 |
| Rank | #64 | #309 |
| Reliability | 10.0 | 9.8 |
| Consistency | 8.9 | 8.2 |
| Attempts | 69/69 | 69/69 |
| Tests Correct | ||
| Attempt pass rate | 76.8% | 40.6% |
| Flaky tests | 3 | 5 |
| Total Runs | 69 | 69 |
| Cost per result | 0.378 | 1.091 |
| Total Cost | $0.061 | $0.077 |
| Input Price | $0.100 / 1M | $0.150 / 1M |
| Output Price | $0.500 / 1M | $0.600 / 1M |
| Cache Read Price | $0.010 / 1M | N/A |
| Cache Write Price | $0.125 / 1M | N/A |
| Total Input Tokens | 296,301 | 50,423 |
| Output Tokens | 10,672 | 4,144 |
| Reasoning Tokens | 50,978 | 110,510 |
| Response Time (avg) | 7.51s | 15.23s |
| Response Time (max) | 44.43s | 118.35s |
| Response Time (total) | 172.74s | 350.26s |
| Parameters | ~50B | ~35B total (~3B active) |
| Availability | Closed | Closed |
Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#64 Claude Haiku 5.5
low- Cost
- $0.001
- Time
- 7.4s
- Tokens
- 1,851 tok
#309 KAT Coder AIR V2.5
high
Reached the allocated time limit (300 seconds) without receiving showcase output.
- Cost
- $0.000
- Time
- 300.0s
- Tokens
- 0 tok
Category Breakdown
| Agentic | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 10.0 | 10.0 | 100.0% | 0 | 44.43s | 153,966 | 2,991 | 3,811 | |
| KAT Coder AIR V2.5 | 3.0 | 10.0 | 0.0% | 0 | 1.10s | 0 | 0 | 0 |
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 8.7 | 7.9 | 91.7% | 1 | 2.16s | 858 | 483 | 1,480 | |
| KAT Coder AIR V2.5 | 6.9 | 5.8 | 75.0% | 2 | 2.49s | 615 | 204 | 1,290 |
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 7.9 | 7.5 | 77.8% | 1 | 7.30s | 10,608 | 531 | 11,618 | |
| KAT Coder AIR V2.5 | 4.3 | 7.3 | 11.1% | 1 | 39.07s | 6,948 | 1,166 | 55,883 |
| Combined | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 10.0 | 10.0 | 100.0% | 0 | 23.94s | 104,796 | 4,741 | 19,543 | |
| KAT Coder AIR V2.5 | 6.5 | 10.0 | 50.0% | 0 | 74.48s | 23,854 | 485 | 28,559 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 10.0 | 10.0 | 100.0% | 0 | 3.45s | 10,515 | 312 | 1,609 | |
| KAT Coder AIR V2.5 | 6.5 | 10.0 | 50.0% | 0 | 3.59s | 7,776 | 284 | 2,140 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 5.9 | 7.2 | 55.6% | 1 | 7.35s | 990 | 64 | 9,207 | |
| KAT Coder AIR V2.5 | 3.0 | 10.0 | 0.0% | 0 | 7.28s | 724 | 1,101 | 7,738 |
| General Intelligence | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 4.6 | 9.8 | 0.0% | 0 | 1.38s | 714 | 319 | 0 | |
| KAT Coder AIR V2.5 | 5.1 | 10.0 | 0.0% | 0 | 7.10s | 516 | 179 | 539 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 9.8 | 10.0 | 100.0% | 0 | 2.20s | 921 | 94 | 515 | |
| KAT Coder AIR V2.5 | 9.8 | 10.0 | 100.0% | 0 | 1.51s | 699 | 90 | 675 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 7.7 | 10.0 | 66.7% | 0 | 2.76s | 912 | 611 | 1,783 | |
| KAT Coder AIR V2.5 | 3.5 | 4.4 | 33.3% | 2 | 2.47s | 696 | 285 | 1,576 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 10.0 | 10.0 | 100.0% | 0 | 4.19s | 11,757 | 354 | 459 | |
| KAT Coder AIR V2.5 | 10.0 | 10.0 | 100.0% | 0 | 4.77s | 8,391 | 330 | 1,082 |
| Trivia | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Haiku 5.5 | 3.0 | 10.0 | 0.0% | 0 | 2.70s | 264 | 172 | 953 | |
| KAT Coder AIR V2.5 | 3.0 | 10.0 | 0.0% | 0 | 21.72s | 204 | 20 | 11,028 |
Quick Compare
Switch Comparison Pair
Ling 3.0 FlashnonevsKAT Coder AIR V2.5highClaude Haiku 5.5lowvsGrok 4.6highKAT Coder AIR V2.5highvsInkling SmalllowFree AvailableKAT Coder AIR V2.5highvsMistral Small 4mediumKAT Coder AIR V2.5highvsMistral Small 4noneClaude Haiku 5.5lowvsDeepSeek V4.1 FlashmaxClaude Haiku 5.5lowvsQwen3.7 MaxmediumMercury 2.5lowvsKAT Coder AIR V2.5highKAT Coder AIR V2.5highvsMiMo-V2.5-PrononeClaude Haiku 5.5lowvsSeed 2.1 TurbomediumClaude Haiku 5.5lowvsMiMo-V2.6-PromediumKAT Coder AIR V2.5highvsQwen3.6 35B A3Bnone