Navigate
Advertise here

Compared models

Ember-1 (low) vs Gemini 3 Flash Preview (medium) vs Qwen3.8 Max (0902) (low) benchmark comparison: Gemini 3 Flash Preview (medium) leads on Score with 9.6. Gemini 3 Flash Preview (medium) leads on Reliability with 10.0. Gemini 3 Flash Preview (medium) has the lowest Total Cost at $0.876. Gemini 3 Flash Preview (medium) is fastest at 21.57s.

Last updated at: 2026-11-18

Compared models

Rank
#247
Total Output Tokens
56,766
Response Time (avg)
34.90s
Total Cost
$1.488
Rank
#27
Total Output Tokens
252,536
Response Time (avg)
21.57s
Total Cost
$0.876
Rank
#120
Total Output Tokens
88,923
Response Time (avg)
29.07s
Total Cost
$1.131
Recommended model Gemini 3 Flash Preview (medium)

It has the strongest score in this comparison (9.6) and the best overall balance of cost and response time across 3 models.

Detailed comparison

Metric Ember-1 Ember-1 low Release: 2026-09-28 Gemini 3 Flash Preview Gemini 3 Flash Preview medium Release: 2025-12-17 Qwen3.8 Max (0902) Qwen3.8 Max (0902) low Release: 2026-09-07
Score 7.0 9.6 8.6
Rank #247 #27 #120
Reliability 8.8 10.0 10.0
Consistency 8.2 9.7 9.0
Attempts 69/69 69/69 69/69
Tests Correct
Attempt pass rate 65.2% 94.2% 85.5%
Flaky tests 5 1 3
Total Runs 69 69 69
Cost per result 11.441 4.169 6.283
Total Cost $1.488 $0.876 $1.131
Input Price $3.000 / 1M $0.500 / 1M $2.000 / 1M
Output Price $15.000 / 1M $3.000 / 1M $6.000 / 1M
Cache Read Price $0.300 / 1M $0.050 / 1M $0.250 / 1M
Cache Write Price N/A $0.084 / 1M $2.500 / 1M
Total Input Tokens 211,908 235,726 298,682
Output Tokens 14,158 6,669 8,374
Reasoning Tokens 42,608 245,867 80,549
Response Time (avg) 34.90s 21.57s 29.07s
Response Time (max) 121.29s 117.26s 207.79s
Response Time (total) 802.79s 496.12s 668.66s
Parameters ~2.8T total (~104B active) ~500B total (~20B active) 2.4T total (~100B active)
Availability Closed Closed Closed

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#247 Ember-1

low
Cost
$0.016
Time
24.2s
Tokens
1,165 tok

#27 Gemini 3 Flash Preview

medium
Cost
$0.010
Time
18.4s
Tokens
3,351 tok

#120 Qwen3.8 Max (0902)

low
Cost
$0.014
Time
41.2s
Tokens
2,421 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Ember-1 10.0 10.0 100.0% 0 37.91s 8,097 2,518 13,142
Gemini 3 Flash Preview 8.6 7.6 88.9% 1 84.40s 8,122 462 161,084
Qwen3.8 Max (0902) 10.0 10.0 100.0% 0 37.42s 8,127 485 17,404

Switch Comparison Pair