Muse Ember (low) vs Inkling 2 Mini (local Q4) (minimal)
The average score is effectively tied at 8.6 vs 8.7. Muse Ember (low) has the lower benchmark cost at $0.114 vs ~$0.206. Muse Ember (low) is faster at 8.22s vs 108.28s, with pass rates of 72.5% vs 73.9%.
Last updated at:
2026-11-22
Compared models
Rank
#111
Total Output Tokens
169,226
Response Time (avg)
8.22s
Total Cost
$0.114
Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
Rank
#108
Total Output Tokens
219,084
Response Time (avg)
108.28s
Total Cost
~$0.206Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 2.0753 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.205185.
Recommended modelMuse Ember (low)
It has the best score here (8.6), while costing about 1.8x less than Inkling 2 Mini (local Q4) (minimal).
Inkling 2 Mini (local Q4)Inkling 2 Mini (local Q4)minimalCandidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.Release: 2026-11-20
Inkling 2 Mini (local Q4)Inkling 2 Mini (local Q4)minimalCandidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.Release: 2026-11-20
Score
8.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
8.7Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
Rank
#111
#108
Reliability
10.0First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
9.8First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
Consistency
8.9Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
Attempts
69/69
69/69
Tests Correct
A test is fully passed only if every run passed for that test.Wrong answer: 6Did not follow instructions: 1No answer: 1Response Time (avg)8.22sResponse Time (max)49.79sResponse Time (total)189.12sA test is fully passed only if every run passed for that test.…
A test is fully passed only if every run passed for that test.Wrong answer: 4Did not follow instructions: 1Invalid tool call: 1Response Time (avg)108.28sResponse Time (max)1322.88sResponse Time (total)2490.37sA test is fully passed only if every run passed for that test.…
Attempt pass rate
72.5%Attempt pass rate = passed attempts / total attempts across runs.…
73.9%Attempt pass rate = passed attempts / total attempts across runs.…
Flaky tests
3Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
Total Runs
69Total Runs…
69Total Runs…
Cost per result
0.757
~1.207Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 2.0753 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.205185.
Total Cost
$0.114
~$0.206Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 2.0753 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.205185.
Input Price
$0.100 / 1MInput Price…
N/AInput Price…
Output Price
$0.500 / 1MOutput Price…
N/AOutput Price…
Cache Read Price
$0.004 / 1MCache Read Price…
N/ACache Read Price…
Cache Write Price
$0.125 / 1MCache Write Price…
N/ACache Write Price…
Total Input Tokens
288,136Total Input Tokens…
309,808Total Input Tokens…
Output Tokens
13,507Output Tokens…
1,579Output Tokens…
Reasoning Tokens
155,719Reasoning Tokens…
217,505Reasoning Tokens…
Response Time (avg)
8.22sResponse Time (avg)…
108.28sResponse Time (avg)…
Response Time (max)
49.79sResponse Time (max)…
1322.88sResponse Time (max)…
Response Time (total)
189.12sResponse Time (total)…
2490.37sResponse Time (total)…
Parameters
~150B total (~10B active)
26.4B total (3.4B active)
Availability
Closed
Open source
Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#111 Muse Ember
low
Cost
$0.003
Time
11.3s
Tokens
4,679 tok
#108 Thinking Machines: Inkling 2 Mini (local Q4)
minimalCandidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
Cost
~$0.002Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.0188 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.001862.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)49.79sResponse Time (max)49.79sResponse Time (total)49.79sA test is fully passed only if every run passed for that test.…
49.79sResponse Time (avg)…
149,643Total Input Tokens…
2,638Output Tokens…
1,880Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)59.04sResponse Time (max)59.04sResponse Time (total)59.04sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.86sResponse Time (max)4.29sResponse Time (total)11.43sA test is fully passed only if every run passed for that test.…
2.86sResponse Time (avg)…
798Total Input Tokens…
299Output Tokens…
3,786Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.20sResponse Time (max)10.50sResponse Time (total)24.78sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.21sResponse Time (max)7.13sResponse Time (total)15.62sA test is fully passed only if every run passed for that test.…
5.21sResponse Time (avg)…
7,410Total Input Tokens…
765Output Tokens…
15,944Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)225.36sResponse Time (max)477.83sResponse Time (total)676.09sA test is fully passed only if every run passed for that test.…
7.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
5.8Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
83.3%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)15.23sResponse Time (max)25.12sResponse Time (total)30.45sA test is fully passed only if every run passed for that test.…
15.23sResponse Time (avg)…
108,813Total Input Tokens…
7,610Output Tokens…
18,518Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
6.4Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
50.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)671.08sResponse Time (max)1322.88sResponse Time (total)1342.15sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.15sResponse Time (max)2.51sResponse Time (total)4.30sA test is fully passed only if every run passed for that test.…
2.15sResponse Time (avg)…
7,215Total Input Tokens…
279Output Tokens…
2,719Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)28.12sResponse Time (max)35.26sResponse Time (total)56.25sA test is fully passed only if every run passed for that test.…
3.5Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
4.4Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
33.3%Attempt pass rate = passed attempts / total attempts across runs.…
2Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)8.96sResponse Time (max)10.79sResponse Time (total)26.89sA test is fully passed only if every run passed for that test.…
8.96sResponse Time (avg)…
873Total Input Tokens…
599Output Tokens…
26,649Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
7.7Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
66.7%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)88.16sResponse Time (max)147.25sResponse Time (total)264.48sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.68sResponse Time (max)2.68sResponse Time (total)2.68sA test is fully passed only if every run passed for that test.…
2.68sResponse Time (avg)…
522Total Input Tokens…
147Output Tokens…
1,018Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
5.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)6.44sResponse Time (max)6.44sResponse Time (total)6.44sA test is fully passed only if every run passed for that test.…
6.5Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
50.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)2.87sResponse Time (max)4.54sResponse Time (total)5.75sA test is fully passed only if every run passed for that test.…
2.87sResponse Time (avg)…
759Total Input Tokens…
165Output Tokens…
2,751Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
9.8Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.25sResponse Time (max)5.70sResponse Time (total)10.49sA test is fully passed only if every run passed for that test.…
5.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
33.3%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)4.80sResponse Time (max)9.28sResponse Time (total)14.41sA test is fully passed only if every run passed for that test.…
4.80sResponse Time (avg)…
786Total Input Tokens…
552Output Tokens…
11,600Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
6.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
33.3%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Did not follow instructions: 1Wrong answer: 1Response Time (avg)7.32sResponse Time (max)7.68sResponse Time (total)21.96sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.29sResponse Time (max)3.29sResponse Time (total)3.29sA test is fully passed only if every run passed for that test.…
3.29sResponse Time (avg)…
11,089Total Input Tokens…
288Output Tokens…
764Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.52sResponse Time (max)7.52sResponse Time (total)7.52sA test is fully passed only if every run passed for that test.…
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)24.54sResponse Time (max)24.54sResponse Time (total)24.54sA test is fully passed only if every run passed for that test.…
24.54sResponse Time (avg)…
228Total Input Tokens…
165Output Tokens…
70,090Reasoning Tokens…
Inkling 2 Mini (local Q4)Candidate responses were generated on local hardware. OpenRouter was used only for judging. inkling-2-mini-ud-q4_k_xl · UD-Q4_K_XL · llama.cpp b12631-e300919cc · NVIDIA GeForce RTX 3090.
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)21.18sResponse Time (max)21.18sResponse Time (total)21.18sA test is fully passed only if every run passed for that test.…