- Rank
- #291
- Total Output Tokens
- 136,501
- Response Time (avg)
- 12.68s
- Total Cost
- $0.126
Mistral Small 4 (medium) vs Inkling Small (low)
Inkling Small (low) leads on average score with 5.2 vs 5.1. Inkling Small (low) has the lower benchmark cost at $0.059 vs $0.126. Inkling Small (low) is faster at 1.99s vs 12.68s, with pass rates of 42.0% vs 47.8%.
Compared models
- Rank
- #285
- Total Output Tokens
- 14,686
- Response Time (avg)
- 1.99s
- Total Cost
- $0.059
Recommended model
Inkling Small (low)
It has the best score here (5.2), while costing about 2.1x less than Mistral Small 4 (medium).
Detailed comparison
| Metric | Mistral Small 4 Mistral Small 4 medium | Inkling Small Inkling Small low Free Available |
|---|---|---|
| Score | 5.1 | 5.2 |
| Rank | #291 | #285 |
| Reliability | 9.7 | 10.0 |
| Consistency | 7.1 | 8.7 |
| Attempts | 69/69 | 69/69 |
| Tests Correct | ||
| Attempt pass rate | 42.0% | 47.8% |
| Flaky tests | 8 | 4 |
| Total Runs | 69 | 69 |
| Cost per result | 2.502 | 0.649 |
| Total Cost | $0.126 | $0.059 |
| Input Price | $0.150 / 1M | $0.450 / 1M |
| Output Price | $0.600 / 1M | $1.200 / 1M |
| Total Input Tokens | 321,955 | 81,447 |
| Output Tokens | 41,555 | 2,715 |
| Reasoning Tokens | 94,946 | 11,971 |
| Response Time (avg) | 12.68s | 1.99s |
| Response Time (max) | 59.15s | 5.82s |
| Response Time (total) | 291.66s | 45.67s |
| Parameters | 119B total (6B active) | 276B total (12B active) |
| Availability | Open source | Closed |
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#291 Mistral Small 4
medium- Cost
- $0.006
- Time
- 47.9s
- Tokens
- 9,857 tok
#285 Thinking Machines: Inkling Small
low- Cost
- $0.001
- Time
- 5.9s
- Tokens
- 759 tok
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
| Agentic | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 5.0 | 10.0 | 0.0% | 0 | 54.05s | 181,396 | 1,287 | 1,617 | |
| Inkling Small | 3.0 | 10.0 | 0.0% | 0 | 24ms | 0 | 0 | 0 |
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 5.6 | 3.8 | 66.7% | 3 | 2.67s | 708 | 4,055 | 4,778 | |
| Inkling Small | 9.2 | 8.4 | 91.7% | 1 | 1.81s | 702 | 240 | 1,831 |
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 4.4 | 5.1 | 33.3% | 2 | 39.98s | 7,636 | 11,635 | 54,715 | |
| Inkling Small | 4.6 | 5.0 | 33.3% | 2 | 3.70s | 7,374 | 375 | 4,332 |
| Combined | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 3.0 | 10.0 | 0.0% | 0 | 32.40s | 116,624 | 17,890 | 18,384 | |
| Inkling Small | 3.0 | 10.0 | 0.0% | 0 | 4.08s | 55,293 | 893 | 1,420 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 7.3 | 5.9 | 83.3% | 1 | 1.23s | 6,171 | 335 | 723 | |
| Inkling Small | 10.0 | 10.0 | 100.0% | 0 | 1.25s | 7,068 | 216 | 372 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 5.9 | 7.2 | 55.6% | 1 | 6.33s | 807 | 3,427 | 7,871 | |
| Inkling Small | 5.3 | 10.0 | 33.3% | 0 | 852ms | 804 | 68 | 602 |
| General Intelligence | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 4.8 | 10.0 | 0.0% | 0 | 2.05s | 519 | 821 | 828 | |
| Inkling Small | 5.3 | 10.0 | 0.0% | 0 | 1.42s | 501 | 141 | 389 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 7.3 | 5.8 | 83.3% | 1 | 1.38s | 729 | 540 | 1,031 | |
| Inkling Small | 9.9 | 10.0 | 100.0% | 0 | 1.89s | 708 | 93 | 902 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 3.4 | 9.7 | 0.0% | 0 | 2.17s | 735 | 1,226 | 2,632 | |
| Inkling Small | 5.3 | 7.2 | 44.4% | 1 | 1.56s | 714 | 315 | 1,225 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 10.0 | 10.0 | 100.0% | 0 | 3.50s | 6,420 | 321 | 810 | |
| Inkling Small | 3.0 | 10.0 | 0.0% | 0 | 2.21s | 8,064 | 339 | 84 |
| Trivia | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | 3.0 | 10.0 | 0.0% | 0 | 5.92s | 210 | 18 | 1,557 | |
| Inkling Small | 3.0 | 10.0 | 0.0% | 0 | 2.00s | 219 | 35 | 814 |
Quick Compare
Switch Comparison Pair
Ling 3.0 FlashnonevsInkling SmalllowFree AvailableMercury 2.5lowvsMistral Small 4mediumMistral Small 4mediumvsMiMo-V2.5-PrononeNemotron 3.5 LightninghighFree AvailablevsInkling SmalllowFree AvailableKAT Coder AIR V2.5highvsInkling SmalllowFree AvailableKAT Coder AIR V2.5highvsMistral Small 4mediumQwen3.6 35B A3BnonevsInkling SmalllowFree AvailableLaguna XS 2.1noneFree AvailablevsInkling SmalllowFree AvailableMistral Small 4mediumvsNemotron 3.5 LightninghighFree AvailableLing 3.0 FlashnonevsMistral Small 4mediumKimi K2.5nonevsInkling SmalllowFree AvailableMistral Small 4nonevsInkling SmalllowFree Available