Navigate
Advertise here

Ember-1 (high) vs Qwen3.5-Flash (medium)

Ember-1 (high) leads on average score with 6.9 vs 6.8. Qwen3.5-Flash (medium) has the lower benchmark cost at $0.184 vs $2.405. Ember-1 (high) is faster at 58.14s vs 83.68s, with pass rates of 71.0% vs 66.7%.

Last updated at: 2026-11-13

Compared models

Rank
#235
Total Output Tokens
121,223
Response Time (avg)
58.14s
Total Cost
$2.405
Rank
#241
Total Output Tokens
531,668
Response Time (avg)
83.68s
Total Cost
$0.184
Recommended model Qwen3.5-Flash (medium)

Its score stays close to the best score here (6.8 vs 6.9), while costing about 13.1x less than Ember-1 (high).

Detailed comparison

Metric Ember-1 Ember-1 high Release: 2026-09-28 Qwen3.5-Flash Qwen3.5-Flash medium Release: 2026-02-24
Score 6.9 6.8
Rank #235 #241
Reliability 8.6 10.0
Consistency 7.2 7.9
Attempts 69/69 69/69
Tests Correct
Attempt pass rate 71.0% 66.7%
Flaky tests 8 6
Total Runs 69 69
Cost per result 20.041 1.533
Total Cost $2.405 $0.184
Input Price $3.000 / 1M $0.065 / 1M
Output Price $15.000 / 1M $0.260 / 1M
Cache Read Price $0.300 / 1M N/A
Cache Write Price N/A N/A
Total Input Tokens 195,489 349,177
Output Tokens 12,475 54,279
Reasoning Tokens 108,748 477,389
Response Time (avg) 58.14s 83.68s
Response Time (max) 255.06s 515.38s
Response Time (total) 1337.19s 1840.92s
Parameters ~2.8T total (~104B active) ~35B total (~3B active)
Availability Closed Closed

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#235 Ember-1

high
Cost
$0.038
Time
55.1s
Tokens
2,671 tok

#241 Qwen3.5-Flash

medium
Cost
$0.002
Time
25.8s
Tokens
4,294 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Ember-1 8.2 7.2 88.9% 1 72.23s 7,144 1,281 23,093
Qwen3.5-Flash 3.7 7.2 22.2% 1 58.87s 6,685 302 90,081

Switch Comparison Pair