- Rank
- #65
- Total Output Tokens
- 241,365
- Response Time (avg)
- 21.14s
- Total Cost
- $0.644
Gemini 2.5 Flash (medium) vs Inkling Small (high)
Inkling Small (high) leads on average score with 8.4 vs 8.2. Inkling Small (high) has the lower benchmark cost at $0.398 vs $0.644. Gemini 2.5 Flash (medium) is faster at 21.14s vs 30.43s, with pass rates of 71.2% vs 69.7%.
Compared models
- Rank
- #61
- Total Output Tokens
- 298,893
- Response Time (avg)
- 30.43s
- Total Cost
- $0.398
Recommended model
Inkling Small (high)
It has the best score here (8.4), while costing about 1.6x less than Gemini 2.5 Flash (medium).
Detailed comparison
| Metric | Gemini 2.5 Flash Gemini 2.5 Flash medium | Inkling Small Inkling Small high |
|---|---|---|
| Score | 8.2 | 8.4 |
| Rank | #65 | #61 |
| Reliability | 10.0 | 10.0 |
| Consistency | 9.6 | 9.2 |
| Attempts | 66/66 | 66/66 |
| Tests Correct | ||
| Attempt pass rate | 71.2% | 69.7% |
| Flaky tests | 1 | 2 |
| Total Runs | 66 | 66 |
| Cost per result | 4.288 | 2.868 |
| Total Cost | $0.644 | $0.398 |
| Input Price | $0.300 / 1M | $0.450 / 1M |
| Output Price | $2.500 / 1M | $1.200 / 1M |
| Total Input Tokens | 132,507 | 85,669 |
| Output Tokens | 12,739 | 6,092 |
| Reasoning Tokens | 228,626 | 292,801 |
| Response Time (avg) | 21.14s | 30.43s |
| Response Time (max) | 140.50s | 122.60s |
| Response Time (total) | 465.10s | 669.38s |
| Parameters | ~350B total (~20B active) | 276B total (12B active) |
| Availability | Closed | Closed |
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#65 Gemini 2.5 Flash
medium
No output was saved. The original provider response or failure reason is unavailable.
- Cost
- $0.000
- Time
- 274.0s
- Tokens
- 0 tok
#61 Thinking Machines: Inkling Small
high- Cost
- $0.006
- Time
- 34.7s
- Tokens
- 4,735 tok
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 8.4 | 10.0 | 75.0% | 0 | 6.30s | 492 | 255 | 10,233 | |
| Inkling Small | 10.0 | 10.0 | 100.0% | 0 | 6.30s | 702 | 335 | 9,678 |
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 7.8 | 10.0 | 66.7% | 0 | 41.01s | 6,669 | 543 | 32,303 | |
| Inkling Small | 10.0 | 10.0 | 100.0% | 0 | 93.23s | 7,374 | 463 | 120,474 |
| Combined | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 84.47s | 110,544 | 11,112 | 95,241 | |
| Inkling Small | 9.9 | 10.0 | 100.0% | 0 | 40.41s | 61,630 | 4,001 | 28,812 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 4.06s | 7,257 | 279 | 2,325 | |
| Inkling Small | 10.0 | 10.0 | 100.0% | 0 | 3.99s | 7,068 | 216 | 2,763 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 5.9 | 7.2 | 55.6% | 1 | 37.07s | 642 | 18 | 80,864 | |
| Inkling Small | 5.3 | 10.0 | 33.3% | 0 | 56.28s | 804 | 69 | 84,578 |
| General Intelligence | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 4.8 | 10.0 | 0.0% | 0 | 4.86s | 486 | 92 | 1,899 | |
| Inkling Small | 5.3 | 10.0 | 0.0% | 0 | 3.62s | 501 | 153 | 1,395 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 9.8 | 10.0 | 100.0% | 0 | 2.62s | 615 | 69 | 1,203 | |
| Inkling Small | 9.8 | 10.0 | 100.0% | 0 | 3.94s | 708 | 94 | 2,742 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 7.7 | 10.0 | 66.7% | 0 | 3.18s | 558 | 126 | 2,499 | |
| Inkling Small | 4.8 | 4.4 | 44.5% | 2 | 14.39s | 714 | 482 | 18,286 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 6.20s | 5,088 | 234 | 1,140 | |
| Inkling Small | 3.0 | 10.0 | 0.0% | 0 | 2.91s | 5,949 | 249 | 577 |
| Trivia | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash | 3.0 | 10.0 | 0.0% | 0 | 2.76s | 156 | 11 | 919 | |
| Inkling Small | 3.0 | 10.0 | 0.0% | 0 | 49.27s | 219 | 30 | 23,496 |
Quick Compare
Switch Comparison Pair
Seed 2.1 TurbomediumvsInkling SmallhighGPT-5.2mediumvsInkling SmallhighInkling SmallhighvsGrok 4.5lowDeepSeek V4.1 FlashmaxvsInkling SmallhighGemini 2.5 FlashmediumvsMuse Spark 1.3lowInkling SmallhighvsGrok 4.6mediumMuse Spark 1.1lowvsInkling SmallhighGemini 2.5 FlashmediumvsMuse Spark 1.1lowMuse Spark 1.2lowvsInkling SmallhighDeepSeek V4.1 FlashmaxvsGemini 2.5 FlashmediumInkling SmallhighvsGrok 4.5mediumDeepSeek V4.1 FlashmediumvsInkling Smallhigh