Ember-1 (high) vs Grok 4.6 (low)

Compared models

Rank: #163
Total Output Tokens: 121,223
Response Time (avg): 58.14s
Total Cost: $2.405

Rank: #159
Total Output Tokens: 42,315
Response Time (avg): 16.44s
Total Cost: $0.781

Recommended model Grok 4.6 (low)

It has the best score here (6.9), while costing about 3.1x less than Ember-1 (high).

Detailed comparison

Metric	Ember-1 Ember-1 high Release: 2026-09-28	Grok 4.6 Grok 4.6 low Release: 2026-08-12

Metric	Ember-1 Ember-1 high Release: 2026-09-28	Grok 4.6 Grok 4.6 low Release: 2026-08-12
Score	6.9	6.9
Rank	#163	#159
Reliability	8.6	10.0
Consistency	7.2	9.6
Attempts	69/69	69/69
Tests Correct
Attempt pass rate	71.0%	66.7%
Flaky tests	8	1
Total Runs	69	69
Cost per result	20.041	5.205
Total Cost	$2.405	$0.781
Input Price	$3.000 / 1M	$2.000 / 1M
Output Price	$15.000 / 1M	$6.000 / 1M
Total Input Tokens	195,489	263,427
Output Tokens	12,475	6,006
Reasoning Tokens	108,748	36,309
Response Time (avg)	58.14s	16.44s
Response Time (max)	255.06s	57.97s
Response Time (total)	1337.19s	378.14s
Parameters	~2.8T total (~104B active)	~1.7T total (~170B active)
Availability	Closed	Closed

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#163 Ember-1

high

Cost: $0.038
Time: 55.1s
Tokens: 2,671 tok

#159 SpaceXAI: Grok 4.6

low

Cost: $0.011
Time: 26.0s
Tokens: 1,941 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Agentic	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.1	3.1	66.7%	1		88.31s	146,912	5,130	5,968
Grok 4.6	5.0	10.0	0.0%	0		57.97s	153,467	882	3,385

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	8.7	7.9	91.7%	1		24.05s	1,544	886	992
Grok 4.6	10.0	10.0	100.0%	0		9.00s	2,991	104	3,343

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	8.2	7.2	88.9%	1		72.23s	7,144	1,281	23,093
Grok 4.6	5.5	7.1	44.4%	1		21.69s	9,579	349	7,740

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.4	5.8	66.7%	1		79.89s	28,994	1,667	6,614
Grok 4.6	6.5	10.0	50.0%	0		23.33s	71,233	3,899	2,805

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	7.3	5.9	83.3%	1		36.10s	6,076	230	1,228
Grok 4.6	10.0	10.0	100.0%	0		11.63s	8,937	225	1,423

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	2.9	4.4	22.2%	2		175.49s	1,154	1,023	61,582
Grok 4.6	3.0	10.0	0.0%	0		18.19s	2,523	16	7,921

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		4.49s	744	194	304
Grok 4.6	10.0	10.0	100.0%	0		16.29s	1,077	55	1,009

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	7.1	5.8	83.3%	1		33.38s	992	162	784
Grok 4.6	9.8	10.0	100.0%	0		6.64s	1,857	57	2,310

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		4.84s	1,452	390	848
Grok 4.6	10.0	10.0	100.0%	0		8.16s	2,430	154	2,506

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		26.45s	0	0	0
Grok 4.6	10.0	10.0	100.0%	0		14.42s	8,544	255	359

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		65.34s	477	1,512	7,335
Grok 4.6	3.0	10.0	0.0%	0		26.11s	789	10	3,508

Quick Compare

Switch Comparison Pair

Qwen3.5-122B-A10BmediumvsGrok 4.6low Ember-1highvsGemini 3.1 Flash Litemedium Ember-1highvsGemma 4 31BmediumFree Available Ember-1highvsGPT-5.6 Lunalow Claude Sonnet 5.5lowvsEmber-1high Claude Opus 4.8nonevsEmber-1high Ember-1highvsKimi K2.6medium Kimi K2.7 CodemediumvsGrok 4.6low Gemini 3.1 Flash Lite PreviewmediumvsGrok 4.6low Ember-1highvsQwen3.5-Flashmedium Gemini 3.1 Flash LitemediumvsGrok 4.6low Ember-1highvsInklinglowFree Available