Ember-1 (max) vs MiniMax M3 (medium)

Compared models

Recommended model MiniMax M3 (medium)

It has the best score here (6.7), while costing about 8.8x less than Ember-1 (max).

Metric	Ember-1 Ember-1 max Release: 2026-09-28	MiniMax M3 MiniMax M3 medium Release: 2026-06-01

Metric	Ember-1 Ember-1 max Release: 2026-09-28	MiniMax M3 MiniMax M3 medium Release: 2026-06-01
Score	6.7	6.7
Rank	#181	#180
Reliability	8.3	9.7
Consistency	7.8	8.1
Attempts	69/69	69/69
Tests Correct
Attempt pass rate	59.4%	59.4%
Flaky tests	6	5
Total Runs	69	69
Cost per result	29.671	3.381
Total Cost	$3.264	$0.372
Input Price	$3.000 / 1M	$0.300 / 1M
Output Price	$15.000 / 1M	$1.200 / 1M
Total Input Tokens	153,399	166,517
Output Tokens	7,803	107,386
Reasoning Tokens	179,098	217,548
Response Time (avg)	97.34s	77.03s
Response Time (max)	587.08s	431.03s
Response Time (total)	2238.73s	1694.74s
Parameters	~2.8T total (~104B active)	428B total (23B active)
Availability	Closed	Weights available

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

max

medium

Category:

Agentic	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		62.53s	126,950	3,620	2,475
MiniMax M3	3.0	10.0	0.0%	0		116.50s	44,521	17,335	31,817

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	8.7	7.9	91.7%	1		23.74s	1,533	613	1,394
MiniMax M3	5.5	3.7	66.7%	3		14.95s	2,526	874	3,414

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	5.2	4.4	55.6%	2		105.09s	3,362	1,149	27,957
MiniMax M3	6.1	6.5	55.6%	1		144.74s	5,804	6,223	32,667

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.5	10.0	50.0%	0		64.37s	12,819	775	4,538
MiniMax M3	10.0	10.0	100.0%	0		138.24s	90,204	30,186	87,263

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.4	5.9	66.7%	1		39.79s	4,285	185	1,182
MiniMax M3	10.0	10.0	100.0%	0		14.92s	8,088	514	3,164

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.6	7.2	22.2%	1		471.21s	1,015	84	136,143
MiniMax M3	3.2	9.3	0.0%	0		234.97s	875	28,389	31,248

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		7.04s	741	193	716
MiniMax M3	5.1	3.4	33.3%	1		33.25s	954	2,487	2,523

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.2	5.8	66.7%	1		32.97s	777	132	574
MiniMax M3	9.8	10.0	100.0%	0		6.14s	1,623	103	920

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	7.7	10.0	66.7%	0		6.57s	1,443	681	1,368
MiniMax M3	7.9	9.9	66.7%	0		49.91s	2,079	11,946	13,761

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		26.36s	0	0	0
MiniMax M3	10.0	10.0	100.0%	0		11.91s	9,168	281	555

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		24.95s	474	371	2,751
MiniMax M3	3.0	10.0	0.0%	0		100.80s	675	9,048	10,216

Switch Comparison Pair