The average score is effectively tied at 5.4 vs 5.3. Inkling Small has the lower benchmark cost at $0.054 vs $0.114. Inkling Small is faster at 1.51s vs 111.60s, with pass rates of 27.3% vs 24.2%.
Last updated at:
2026-09-10
Compared models
Rank
#255
Total Output Tokens
584,864
Response Time (avg)
111.60s
Total Cost
$0.114
Rank
#256
Total Output Tokens
8,770
Response Time (avg)
1.51s
Total Cost
$0.054
Recommended modelInkling Small
It has the best score here (5.3), while costing about 2.1x less than Laguna S 2.1 (high).
A test is fully passed only if every run passed for that test.Wrong answer: 15Did not follow instructions: 1Invalid tool call: 1Response Time (avg)1.51sResponse Time (max)12.35sResponse Time (total)33.21sA test is fully passed only if every run passed for that test.…
Attempt pass rate
27.3%Attempt pass rate = passed attempts / total attempts across runs.…
24.2%Attempt pass rate = passed attempts / total attempts across runs.…
Flaky tests
5Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
Total Runs
66Total Runs…
66Total Runs…
Cost per result
3.153
1.158
Total Cost
$0.114
$0.054
Input Price
$0.090 / 1MInput Price…
$0.450 / 1MInput Price…
Output Price
$0.180 / 1MOutput Price…
$1.200 / 1MOutput Price…
Total Input Tokens
208,937Total Input Tokens…
94,733Total Input Tokens…
Output Tokens
175,588Output Tokens…
8,770Output Tokens…
Reasoning Tokens
409,276Reasoning Tokens…
0Reasoning Tokens…
Response Time (avg)
111.60sResponse Time (avg)…
1.51sResponse Time (avg)…
Response Time (max)
1190.11sResponse Time (max)…
12.35sResponse Time (max)…
Response Time (total)
2455.24sResponse Time (total)…
33.21sResponse Time (total)…
Parameters
118B total (8B active)
276B total (12B active)
Availability
Weights available
Closed
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
3.4Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
5.8Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
25.0%Attempt pass rate = passed attempts / total attempts across runs.…
2Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 3No answer: 1Response Time (avg)52.20sResponse Time (max)207.32sResponse Time (total)208.79sA test is fully passed only if every run passed for that test.…
3.1Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
8.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
8.3%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)526msResponse Time (max)1.03sResponse Time (total)2.10sA test is fully passed only if every run passed for that test.…
5.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
44.4%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.API error: 1No answer: 1Response Time (avg)271.42sResponse Time (max)423.72sResponse Time (total)814.25sA test is fully passed only if every run passed for that test.…
3.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)759msResponse Time (max)1.52sResponse Time (total)2.28sA test is fully passed only if every run passed for that test.…
2.9Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
6.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
16.7%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Invalid tool call: 2Response Time (avg)702.26sResponse Time (max)1190.11sResponse Time (total)1404.52sA test is fully passed only if every run passed for that test.…
6.5Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
50.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)9.33sResponse Time (max)12.35sResponse Time (total)18.66sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.96sResponse Time (max)3.41sResponse Time (total)3.91sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)782msResponse Time (max)961msResponse Time (total)1.56sA test is fully passed only if every run passed for that test.…
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.51sResponse Time (max)6.82sResponse Time (total)7.52sA test is fully passed only if every run passed for that test.…
5.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
33.3%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)442msResponse Time (max)517msResponse Time (total)1.33sA test is fully passed only if every run passed for that test.…
4.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
9.8Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)855msResponse Time (max)855msResponse Time (total)855msA test is fully passed only if every run passed for that test.…
5.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)964msResponse Time (max)964msResponse Time (total)964msA test is fully passed only if every run passed for that test.…
4.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Did not follow instructions: 1Wrong answer: 1Response Time (avg)453msResponse Time (max)492msResponse Time (total)905msA test is fully passed only if every run passed for that test.…
6.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
50.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)767msResponse Time (max)1.02sResponse Time (total)1.53sA test is fully passed only if every run passed for that test.…
2.9Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
11.1%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 2Extra formatting: 1Response Time (avg)1.62sResponse Time (max)3.81sResponse Time (total)4.87sA test is fully passed only if every run passed for that test.…
3.2Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
9.7Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)665msResponse Time (max)734msResponse Time (total)2.00sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.69sResponse Time (max)2.69sResponse Time (total)2.69sA test is fully passed only if every run passed for that test.…
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)2.31sResponse Time (max)2.31sResponse Time (total)2.31sA test is fully passed only if every run passed for that test.…
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)6.94sResponse Time (max)6.94sResponse Time (total)6.94sA test is fully passed only if every run passed for that test.…
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)481msResponse Time (max)481msResponse Time (total)481msA test is fully passed only if every run passed for that test.…