Ember-1 (low) vs GLM 5.3 FlashX (medium)

Compared models

Recommended model GLM 5.3 FlashX (medium)

It has the best score here (7.1), while costing about 13.1x less than Ember-1 (low).

Metric	Ember-1 Ember-1 low Release: 2026-09-28	GLM 5.3 FlashX GLM 5.3 FlashX medium Release: 2026-09-21

Metric	Ember-1 Ember-1 low Release: 2026-09-28	GLM 5.3 FlashX GLM 5.3 FlashX medium Release: 2026-09-21
Score	7.0	7.1
Rank	#157	#150
Reliability	8.8	10.0
Consistency	8.2	7.1
Attempts	69/69	69/69
Tests Correct
Attempt pass rate	65.2%	69.6%
Flaky tests	5	8
Total Runs	69	69
Cost per result	11.441	0.946
Total Cost	$1.488	$0.114
Input Price	$3.000 / 1M	$0.370 / 1M
Output Price	$15.000 / 1M	$1.250 / 1M
Total Input Tokens	211,908	205,100
Output Tokens	14,158	8,639
Reasoning Tokens	42,608	21,409
Response Time (avg)	34.90s	7.37s
Response Time (max)	121.29s	74.39s
Response Time (total)	802.79s	169.54s
Parameters	~2.8T total (~104B active)	320B total (18B active)
Availability	Closed	Closed

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

low

medium

Category:

Agentic	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	4.7	3.1	33.3%	1		121.29s	178,897	9,378	9,921
GLM 5.3 FlashX	10.0	10.0	100.0%	0		74.39s	120,771	922	9,693

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	8.2	7.9	83.3%	1		22.18s	1,402	571	613
GLM 5.3 FlashX	7.3	5.8	83.3%	2		1.71s	639	343	368

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		37.91s	8,097	2,518	13,142
GLM 5.3 FlashX	6.0	4.6	66.7%	2		6.37s	7,317	384	5,562

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.5	10.0	50.0%	0		51.45s	12,828	381	689
GLM 5.3 FlashX	2.8	1.6	33.3%	2		15.97s	59,549	5,920	2,786

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	7.3	5.9	83.3%	1		34.82s	6,076	223	778
GLM 5.3 FlashX	10.0	10.0	100.0%	0		1.74s	7,149	210	134

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	2.9	7.2	11.1%	1		56.17s	1,154	234	12,476
GLM 5.3 FlashX	3.6	7.2	22.2%	1		3.52s	759	38	2,005

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		5.67s	744	139	462
GLM 5.3 FlashX	3.4	2.2	33.3%	1		1.79s	498	135	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.2	5.8	66.7%	1		32.73s	781	121	466
GLM 5.3 FlashX	10.0	10.0	100.0%	0		1.51s	678	72	70

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		3.27s	1,452	393	595
GLM 5.3 FlashX	10.0	10.0	100.0%	0		1.84s	672	306	371

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		26.43s	0	0	0
GLM 5.3 FlashX	10.0	10.0	100.0%	0		10.30s	6,861	231	20

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		30.60s	477	200	3,466
GLM 5.3 FlashX	3.0	10.0	0.0%	0		2.60s	207	78	400

Switch Comparison Pair