#74 DeepSeek V4 Pro
none
Invalid SVG
- Cost
- $0.000
- Time
- 300.0s
- Tokens
- 0 tok
AI BENCHY Compare
DeepSeek: DeepSeek V4 Pro vs Kwaipilot: KAT-Coder-Pro V2.5
Summary
KAT-Coder-Pro V2.5 (medium) leads on average score with 7.5 vs 7.2. DeepSeek V4 Pro has the lower benchmark cost at $0.034 vs $0.275. DeepSeek V4 Pro is faster at 6.41s vs 12.95s, with pass rates of 52.4% vs 65.1%.
Recommended modelDeepSeek V4 ProIts score stays close to the best score here (7.2 vs 7.5), while costing about 8.3x less than KAT-Coder-Pro V2.5 (medium).
Last updated at: 2026-07-14
| Metric | DeepSeek V4 Pro DeepSeek V4 Pro none | KAT-Coder-Pro V2.5 KAT-Coder-Pro V2.5 medium |
|---|---|---|
| Score | 7.2 | 7.5 |
| Rank | #74 | #60 |
| Reliability | 9.9 | 10.0 |
| Consistency | 8.8 | 7.7 |
| Tests Correct | ||
| Attempt pass rate | 52.4% | 65.1% |
| Flaky tests | 3 | 6 |
| Total Runs | 63 | 63 |
| Cost per result | 0.333 | 2.498 |
| Total Cost | $0.034 | $0.275 |
| Input Price | $0.435 / 1M | $0.740 / 1M |
| Output Price | $0.870 / 1M | $2.960 / 1M |
| Total Input Tokens | 53,558 | 42,750 |
| Output Tokens | 11,424 | 2,098 |
| Reasoning Tokens | 0 | 80,027 |
| Response Time (avg) | 6.41s | 12.95s |
| Response Time (max) | 30.09s | 81.45s |
| Response Time (total) | 134.66s | 271.92s |
Generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#60 KAT-Coder-Pro V2.5
medium- Cost
- $0.009
- Time
- 25.6s
- Tokens
- 2,939 tok
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| DeepSeek V4 Pro | 5.6 | 10.0 | 33.3% | 0 | 13.38s | 7,275 | 5,500 | 0 | |
| KAT-Coder-Pro V2.5 | 7.8 | 10.0 | 66.7% | 0 | 33.10s | 7,893 | 416 | 33,658 |
Quick Compare
Switch Comparison Pair
DeepSeek V4 PrononevsGemma 4 26B A4BmediumFree AvailableKAT-Coder-Pro V2.5mediumvsGPT-5.3 ChatnoneDeepSeek V4 ProhighvsKAT-Coder-Pro V2.5mediumDeepSeek V4 PrononevsStep 3.7 FlashhighDeepSeek V4 PrononevsGLM 5.1mediumDeepSeek V4 PrononevsLaguna XS 2.1mediumFree AvailableGemini 3 Flash PreviewlowvsKAT-Coder-Pro V2.5mediumDeepSeek V4 PrononevsKimi K2.7 CodemediumKAT-Coder-Pro V2.5mediumvsGPT-5.6 LunahighKAT-Coder-Pro V2.5mediumvsStep 3.7 FlashlowDeepSeek V4 PrononevsGrok 4.20mediumKAT-Coder-Pro V2.5mediumvsGPT-5.6 Terralow