Navigate
AD
Track all your projects in one dashboard. Get 📊stats, 🔥heatmaps and 👀recordings in one self-hosted dashboard.
uxwizz.com

Ember-1 vs Mistral Small 4 (medium)

The average score is effectively tied at 5.2 vs 5.1. Mistral Small 4 (medium) has the lower benchmark cost at $0.097 vs $0.123. Mistral Small 4 (medium) is faster at 10.80s vs 19.46s, with pass rates of 42.4% vs 43.9%.

Last updated at: 2026-09-28

Compared models

Rank
#297
Total Output Tokens
2,664
Response Time (avg)
19.46s
Total Cost
$0.123
Rank
#301
Total Output Tokens
133,597
Response Time (avg)
10.80s
Total Cost
$0.097
Recommended model Mistral Small 4 (medium)

It has the best score here (5.1), while responding about 1.8x faster than Ember-1.

Detailed comparison

Metric Ember-1 Ember-1 none Release: 2026-09-28 Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16
Score 5.2 5.1
Rank #297 #301
Reliability 8.1 10.0
Consistency 8.9 7.0
Attempts 66/66 66/66
Tests Correct
Attempt pass rate 42.4% 43.9%
Flaky tests 3 8
Total Runs 66 66
Cost per result 1.528 1.923
Total Cost $0.123 $0.097
Input Price $3.000 / 1M $0.150 / 1M
Output Price $15.000 / 1M $0.600 / 1M
Total Input Tokens 27,426 140,559
Output Tokens 2,664 40,268
Reasoning Tokens 0 93,329
Response Time (avg) 19.46s 10.80s
Response Time (max) 93.12s 59.15s
Response Time (total) 428.04s 237.60s
Parameters ~2.8T total (~104B active) 119B total (6B active)
Availability Closed Open source

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#297 Ember-1

none
Cost
$0.156
Time
224.6s
Tokens
10,555 tok

#301 Mistral Small 4

medium
Cost
$0.006
Time
47.9s
Tokens
9,857 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Ember-1 5.5 10.0 33.3% 0 1.94s 7,485 393 0
Mistral Small 4 4.4 5.1 33.3% 2 39.98s 7,636 11,635 54,715

Quick Compare

Switch Comparison Pair