Ember-1 (low) vs Qwen3.5-27B (medium)

Compared models

Rank: #120
Total Output Tokens: 37,467
Response Time (avg): 30.98s
Total Cost: $0.662

Rank: #124
Total Output Tokens: 614,909
Response Time (avg): 113.28s
Total Cost: $0.982

Recommended model Ember-1 (low)

It has the best score here (7.5), while responding about 3.7x faster than Qwen3.5-27B (medium).

Detailed comparison

Metric	Ember-1 Ember-1 low Release: 2026-09-28	Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24

Metric	Ember-1 Ember-1 low Release: 2026-09-28	Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24
Score	7.5	7.5
Rank	#120	#124
Reliability	8.1	10.0
Consistency	8.5	7.8
Attempts	66/66	66/66
Tests Correct
Attempt pass rate	66.7%	75.8%
Flaky tests	4	6
Total Runs	66	66
Cost per result	5.085	8.217
Total Cost	$0.662	$0.982
Input Price	$3.000 / 1M	$0.195 / 1M
Output Price	$15.000 / 1M	$1.560 / 1M
Total Input Tokens	33,011	111,644
Output Tokens	4,780	16,113
Reasoning Tokens	32,687	598,796
Response Time (avg)	30.98s	113.28s
Response Time (max)	93.10s	1026.43s
Response Time (total)	681.50s	2492.26s
Parameters	~2.8T total (~104B active)	27B
Availability	Closed	Open source

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#120 Ember-1

low

Cost: $0.016
Time: 24.2s
Tokens: 1,165 tok

#124 Qwen3.5-27B

medium

Cost: $0.008
Time: 62.0s
Tokens: 3,099 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	8.2	7.9	83.3%	1		22.18s	1,402	571	613
Qwen3.5-27B	8.7	7.9	91.7%	1		19.75s	672	569	31,505

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		37.91s	8,097	2,518	13,142
Qwen3.5-27B	6.2	7.1	55.6%	1		160.69s	7,895	6,381	89,388

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.5	10.0	50.0%	0		51.45s	12,828	381	689
Qwen3.5-27B	7.3	5.8	83.3%	1		595.19s	84,417	7,948	279,132

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	7.3	5.9	83.3%	1		34.82s	6,076	223	778
Qwen3.5-27B	10.0	10.0	100.0%	0		30.26s	7,782	270	16,150

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	2.9	7.2	11.1%	1		56.17s	1,154	234	12,476
Qwen3.5-27B	5.9	7.2	55.6%	1		89.39s	562	157	52,734

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		5.67s	744	139	462
Qwen3.5-27B	6.1	3.1	66.7%	1		101.41s	524	70	23,147

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	6.2	5.8	66.7%	1		32.73s	781	121	466
Qwen3.5-27B	10.0	10.0	100.0%	0		19.66s	699	97	11,638

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	10.0	10.0	100.0%	0		3.27s	1,452	393	595
Qwen3.5-27B	8.2	7.7	77.8%	1		59.60s	696	242	70,096

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		26.43s	0	0	0
Qwen3.5-27B	10.0	10.0	100.0%	0		7.45s	8,193	348	1,323

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Ember-1	3.0	10.0	0.0%	0		30.60s	477	200	3,466
Qwen3.5-27B	3.0	10.0	0.0%	0		85.11s	204	31	23,683

Quick Compare

Switch Comparison Pair

Ember-1lowvsMiniMax M3medium Ember-1lowvsGrok 4.7high GPT 5.3 ChatnonevsQwen3.5-27Bmedium Ember-1lowvsGPT 5.3 Chatnone Ember-1lowvsQwen3.5 Plus 2026-02-15medium Qwen3.5-27BmediumvsGrok 4.7high Ember-1lowvsMercury 2.5 Previewmedium Gemini 3.8 FlashlowvsQwen3.5-27Bmedium GPT-5.6 TerralowvsQwen3.5-27Bmedium Ember-1lowvsGPT-5.4 Minimedium Ember-1lowvsGrok Build 0.1medium Ember-1lowvsGPT-5.4 Nanomedium