The average score is effectively tied at 9.9 vs 9.9. max has the lower benchmark cost at $5.044 vs $6.213. xhigh is faster at 29.67s vs 61.63s, with pass rates of 95.7% vs 97.1%.
Last updated at:
2026-10-19
Compared models
Rank
#5
Total Output Tokens
64,616
Response Time (avg)
61.63s
Total Cost
$5.044
Rank
#8
Total Output Tokens
90,095
Response Time (avg)
29.67s
Total Cost
$6.213
Recommended modelxhigh
It has the best score here (9.9), while responding about 2.1x faster than GPT-6 Astra (max).
GPT-6 AstraGPT-6 AstraxhighArchived model: this model is no longer updated or tested on new tests.Release: 2026-09-04
Score
9.9Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
9.9Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
Rank
#5
#8
Reliability
9.6First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
9.9First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
Consistency
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
9.6Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
Attempts
69/69
69/69
Tests Correct
A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)61.63sResponse Time (max)960.01sResponse Time (total)1417.51sA test is fully passed only if every run passed for that test.…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)29.67sResponse Time (max)381.11sResponse Time (total)682.42sA test is fully passed only if every run passed for that test.…
Attempt pass rate
95.7%Attempt pass rate = passed attempts / total attempts across runs.…
97.1%Attempt pass rate = passed attempts / total attempts across runs.…
Flaky tests
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
Total Runs
69Total Runs…
69Total Runs…
Cost per result
22.923
28.237
Total Cost
$5.044
$6.213
Input Price
$10.000 / 1MInput Price…
$10.000 / 1MInput Price…
Output Price
$50.000 / 1MOutput Price…
$50.000 / 1MOutput Price…
Cache Read Price
$1.000 / 1MCache Read Price…
$1.000 / 1MCache Read Price…
Cache Write Price
$12.500 / 1MCache Write Price…
$12.500 / 1MCache Write Price…
Total Input Tokens
181,223Total Input Tokens…
170,731Total Input Tokens…
Output Tokens
5,707Output Tokens…
5,865Output Tokens…
Reasoning Tokens
58,909Reasoning Tokens…
84,230Reasoning Tokens…
Response Time (avg)
61.63sResponse Time (avg)…
29.67sResponse Time (avg)…
Response Time (max)
960.01sResponse Time (max)…
381.11sResponse Time (max)…
Response Time (total)
1417.51sResponse Time (total)…
682.42sResponse Time (total)…
Parameters
~2T total (~150B active)
~2T total (~150B active)
Availability
Closed
Closed
Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#5 GPT-6 Astra
max
Reached the allocated time limit (600 seconds) without receiving showcase output.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)69.56sResponse Time (max)69.56sResponse Time (total)69.56sA test is fully passed only if every run passed for that test.…
69.56sResponse Time (avg)…
102,165Total Input Tokens…
854Output Tokens…
3,532Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)56.73sResponse Time (max)56.73sResponse Time (total)56.73sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.87sResponse Time (max)8.14sResponse Time (total)19.46sA test is fully passed only if every run passed for that test.…
4.87sResponse Time (avg)…
606Total Input Tokens…
171Output Tokens…
1,982Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.71sResponse Time (max)5.18sResponse Time (total)14.85sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)14.64sResponse Time (max)23.54sResponse Time (total)43.92sA test is fully passed only if every run passed for that test.…
14.64sResponse Time (avg)…
7,302Total Input Tokens…
395Output Tokens…
8,298Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.27sResponse Time (max)11.08sResponse Time (total)24.81sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)64.02sResponse Time (max)96.57sResponse Time (total)128.04sA test is fully passed only if every run passed for that test.…
64.02sResponse Time (avg)…
56,162Total Input Tokens…
3,313Output Tokens…
18,967Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)41.20sResponse Time (max)63.24sResponse Time (total)82.41sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.78sResponse Time (max)4.82sResponse Time (total)9.56sA test is fully passed only if every run passed for that test.…
4.78sResponse Time (avg)…
7,140Total Input Tokens…
234Output Tokens…
944Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.34sResponse Time (max)3.64sResponse Time (total)6.69sA test is fully passed only if every run passed for that test.…
7.7Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
66.7%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)356.37sResponse Time (max)960.01sResponse Time (total)1069.12sA test is fully passed only if every run passed for that test.…
356.37sResponse Time (avg)…
420Total Input Tokens…
42Output Tokens…
14,986Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
7.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
77.8%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)146.97sResponse Time (max)381.11sResponse Time (total)440.90sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)13.69sResponse Time (max)13.69sResponse Time (total)13.69sA test is fully passed only if every run passed for that test.…
13.69sResponse Time (avg)…
477Total Input Tokens…
110Output Tokens…
1,823Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.68sResponse Time (max)7.68sResponse Time (total)7.68sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.34sResponse Time (max)8.87sResponse Time (total)12.67sA test is fully passed only if every run passed for that test.…
6.34sResponse Time (avg)…
660Total Input Tokens…
93Output Tokens…
1,669Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.91sResponse Time (max)5.08sResponse Time (total)9.82sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.32sResponse Time (max)10.41sResponse Time (total)21.95sA test is fully passed only if every run passed for that test.…
7.32sResponse Time (avg)…
642Total Input Tokens…
201Output Tokens…
2,765Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.48sResponse Time (max)5.60sResponse Time (total)13.44sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)20.65sResponse Time (max)20.65sResponse Time (total)20.65sA test is fully passed only if every run passed for that test.…
20.65sResponse Time (avg)…
5,454Total Input Tokens…
264Output Tokens…
2,786Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)11.04sResponse Time (max)11.04sResponse Time (total)11.04sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.91sResponse Time (max)8.91sResponse Time (total)8.91sA test is fully passed only if every run passed for that test.…
8.91sResponse Time (avg)…
195Total Input Tokens…
30Output Tokens…
1,157Reasoning Tokens…
GPT-6 AstraArchived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)14.06sResponse Time (max)14.06sResponse Time (total)14.06sA test is fully passed only if every run passed for that test.…