- Rank
- #47
- Total Output Tokens
- 63,204
- Response Time (avg)
- 12.54s
- Total Cost
- $0.922
Claude Sonnet 5 (medium) vs DeepSeek V4 Flash 0731 (high)
Claude Sonnet 5 (medium) leads on average score with 8.2 vs 8.0. DeepSeek V4 Flash 0731 (high) has the lower benchmark cost at $0.134 vs $0.922. Claude Sonnet 5 (medium) is faster at 12.54s vs 114.94s, with pass rates of 75.8% vs 75.8%.
- Rank
- #54
- Total Output Tokens
- 513,307
- Response Time (avg)
- 114.94s
- Total Cost
- $0.134
Recommended model
DeepSeek V4 Flash 0731 (high)
Its score stays close to the best score here (8.0 vs 8.2), while costing about 6.9x less than Claude Sonnet 5 (medium).
Detailed comparison
| Metric | Claude Sonnet 5 Claude Sonnet 5 medium | DeepSeek V4 Flash 0731 DeepSeek V4 Flash 0731 high |
|---|---|---|
| Score | 8.2 | 8.0 |
| Rank | #47 | #54 |
| Reliability | 10.0 | 10.0 |
| Consistency | 9.0 | 8.0 |
| Attempts | 66/66 | 66/66 |
| Tests Correct | ||
| Attempt pass rate | 75.8% | 75.8% |
| Flaky tests | 3 | 5 |
| Total Runs | 66 | 66 |
| Cost per result | 6.144 | 0.956 |
| Total Cost | $0.922 | $0.134 |
| Input Price | $2.000 / 1M | $0.140 / 1M |
| Output Price | $10.000 / 1M | $0.280 / 1M |
| Total Input Tokens | 145,953 | 96,702 |
| Output Tokens | 52,330 | 16,966 |
| Reasoning Tokens | 10,874 | 496,341 |
| Response Time (avg) | 12.54s | 114.94s |
| Response Time (max) | 66.71s | 600.61s |
| Response Time (total) | 275.90s | 2528.65s |
| Parameters | ~1T total (~100B active) | 284B total (13B active) |
| Availability | Closed | Closed |
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#47 Claude Sonnet 5
medium- Cost
- $0.007
- Time
- 6.4s
- Tokens
- 832 tok
#54 DeepSeek V4 Flash 0731
high
Invalid SVG
- Cost
- $0.000
- Time
- 187.3s
- Tokens
- 19,048 tok
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 10.0 | 10.0 | 100.0% | 0 | 3.80s | 834 | 1,220 | 446 | |
| DeepSeek V4 Flash 0731 | 8.7 | 7.9 | 91.7% | 1 | 13.62s | 1,488 | 542 | 10,576 |
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 9.0 | 7.9 | 88.9% | 1 | 17.28s | 10,590 | 13,153 | 2,379 | |
| DeepSeek V4 Flash 0731 | 6.3 | 6.3 | 55.6% | 1 | 252.67s | 7,044 | 2,160 | 191,210 |
| Combined | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 7.3 | 6.0 | 83.3% | 1 | 51.86s | 107,934 | 23,169 | 5,371 | |
| DeepSeek V4 Flash 0731 | 10.0 | 10.0 | 100.0% | 0 | 87.10s | 67,283 | 11,903 | 41,397 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 10.0 | 10.0 | 100.0% | 0 | 3.16s | 10,503 | 312 | 0 | |
| DeepSeek V4 Flash 0731 | 10.0 | 10.0 | 100.0% | 0 | 3.65s | 7,764 | 336 | 1,259 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 5.3 | 10.0 | 33.3% | 0 | 20.54s | 972 | 12,137 | 1,994 | |
| DeepSeek V4 Flash 0731 | 3.5 | 4.4 | 33.3% | 2 | 458.65s | 700 | 104 | 215,629 |
| General Intelligence | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 4.8 | 3.2 | 33.3% | 1 | 4.32s | 708 | 264 | 0 | |
| DeepSeek V4 Flash 0731 | 10.0 | 10.0 | 100.0% | 0 | 5.64s | 708 | 72 | 533 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 9.9 | 10.0 | 100.0% | 0 | 3.10s | 909 | 318 | 269 | |
| DeepSeek V4 Flash 0731 | 9.8 | 10.0 | 100.0% | 0 | 4.46s | 1,101 | 119 | 939 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 7.7 | 10.0 | 66.7% | 0 | 2.98s | 894 | 407 | 121 | |
| DeepSeek V4 Flash 0731 | 8.2 | 7.7 | 77.8% | 1 | 9.36s | 1,305 | 397 | 6,801 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 10.0 | 10.0 | 100.0% | 0 | 10.70s | 12,351 | 433 | 90 | |
| DeepSeek V4 Flash 0731 | 10.0 | 10.0 | 100.0% | 0 | 5.42s | 8,889 | 357 | 451 |
| Trivia | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5 | 3.0 | 10.0 | 0.0% | 0 | 7.06s | 258 | 917 | 204 | |
| DeepSeek V4 Flash 0731 | 3.0 | 10.0 | 0.0% | 0 | 110.63s | 420 | 976 | 27,546 |
Quick Compare
Switch Comparison Pair
DeepSeek V4 Flash 0731highvsInklingmediumDeepSeek V4 Flash 0731highvsMuse Glimmer 30BxhighDeepSeek V4 Flash 0731highvsGPT-5 MinimediumDeepSeek V4 Flash 0731highvsQwen3.8 2.4T A95BlowClaude Sonnet 5mediumvsQwen3.8 2.4T A95BlowDeepSeek V4 Flash 0731highvsKimi K3maxClaude Sonnet 5mediumvsMuse Glimmer 30BxhighDeepSeek V4 Flash 0731highvsGPT-5.2 ChatnoneDeepSeek V4 Flash 0731highvsStep 3.7 FlashmediumClaude Sonnet 5mediumvsMuse Spark 1.1lowDeepSeek V4 Flash 0731highvsQwen3.8 27BlowClaude Sonnet 5mediumvsInklinghigh