Ember-1 vs Qwen3.8 27B

Compared models

Rank: #329
Total Output Tokens: 11,832
Response Time (avg): 22.65s
Total Cost: $0.786

Rank: #324
Total Output Tokens: 13,791
Response Time (avg): 5.05s
Total Cost: ~$0.010

Recommended model Qwen3.8 27B

It has the best score here (5.4), while costing about 82.1x less than Ember-1.

Detailed comparison

Metric	Ember-1 Ember-1 none Release: 2026-09-28	Qwen3.8 27B Qwen3.8 27B none Release: 2026-08-14 Free Available

Metric	Ember-1 Ember-1 none Release: 2026-09-28	Qwen3.8 27B Qwen3.8 27B none Release: 2026-08-14 Free Available
Score	5.4	5.4
Rank	#329	#324
Reliability	8.8	10.0
Consistency	8.6	10.0
Attempts	69/69	69/69
Tests Correct
Attempt pass rate	43.5%	39.1%
Flaky tests	4	0
Total Runs	69	69
Cost per result	9.817	~0.107
Total Cost	$0.786	~$0.010
Input Price	$3.000 / 1M	N/A
Output Price	$15.000 / 1M	N/A
Cache Read Price	$0.300 / 1M	N/A
Cache Write Price	N/A	N/A
Total Input Tokens	202,617	281,460
Output Tokens	11,832	13,791
Reasoning Tokens	0	0
Response Time (avg)	22.65s	5.05s
Response Time (max)	93.12s	47.42s
Response Time (total)	520.91s	116.11s
Parameters	~2.8T total (~104B active)	27.3B
Availability	Closed	Weights available

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#329 Ember-1

none

Cost: $0.156
Time: 224.6s
Tokens: 10,555 tok

#324 Qwen3.8 27B

none

Cost: ~$0.001
Time: 23.9s
Tokens: 1,922 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Agentic	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.1	3.1	66.7%	1		92.87s	175,191	9,168	0
Qwen3.8 27B	5.0	10.0	0.0%	0		47.42s	158,997	2,346	0

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	5.2	7.9	41.7%	1		22.37s	796	867	0
Qwen3.8 27B	6.5	10.0	50.0%	0		828ms	696	267	0

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	5.5	10.0	33.3%	0		1.94s	7,485	393	0
Qwen3.8 27B	5.5	10.0	33.3%	0		1.19s	7,911	408	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		49.01s	12,162	327	0
Qwen3.8 27B	3.0	10.0	0.0%	0		24.10s	95,181	9,588	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.4	5.9	66.7%	1		35.25s	4,017	161	0
Qwen3.8 27B	6.5	10.0	50.0%	0		1.14s	7,788	246	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	5.3	10.0	33.3%	0		21.61s	804	68	0
Qwen3.8 27B	7.7	10.0	66.7%	0		552ms	798	27	0

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	5.0	10.0	0.0%	0		2.28s	540	160	0
Qwen3.8 27B	4.2	9.9	0.0%	0		1.31s	522	126	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.4	5.8	66.7%	1		31.00s	509	81	0
Qwen3.8 27B	6.3	10.0	50.0%	0		582ms	711	72	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		2.54s	840	569	0
Qwen3.8 27B	6.0	10.0	33.3%	0		1.25s	714	411	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		26.44s	0	0	0
Qwen3.8 27B	10.0	10.0	100.0%	0		2.75s	7,932	273	0

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		1.05s	273	38	0
Qwen3.8 27B	3.0	10.0	0.0%	0		703ms	210	27	0

Switch Comparison Pair

Ember-1nonevsStep 3.5 Flashmedium North Mini CodemediumFree AvailablevsQwen3.8 27BnoneFree Available Ember-1nonevsHy4 previewlow Qwen3.8 27BnoneFree AvailablevsHy4 previewlow North Mini CodemediumFree AvailablevsEmber-1none Ember-1nonevsGranite 4.2 8Blow Qwen3.8 27BnoneFree AvailablevsStep 3.5 Flashmedium Nemotron 3 SupermediumFree AvailablevsQwen3.8 27BnoneFree Available Granite 4.2 8BlowvsQwen3.8 27BnoneFree Available Ember-1nonevsInkling SmalllowFree Available Ember-1nonevsNemotron 3.5 LightninghighFree Available Ember-1nonevsNemotron 3 SupermediumFree Available