GPT-5.6 Luna (high) leads on average score with 8.0 vs 6.3. GPT-5.6 Luna (high) has the lower benchmark cost at $0.118 vs $2.961. GPT-5.6 Luna (high) is faster at 18.39s vs 32.23s, with pass rates of 60.9% vs 71.0%.
Last updated at:
2026-10-19
Compared models
Rank
#261
Total Output Tokens
96,722
Response Time (avg)
32.23s
Total Cost
$2.961
Rank
#110
Total Output Tokens
139,395
Response Time (avg)
18.39s
Total Cost
$0.118
Recommended modelGPT-5.6 Luna (high)
It has the best score here (8.0), while costing about 25.2x less than Claude Opus 4.6 (medium).
Detailed comparison
Metric
Claude Opus 4.6Claude Opus 4.6mediumArchived model: this model is no longer updated or tested on new tests.Release: 2026-02-05
6.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
8.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
Rank
#261
#110
Reliability
10.0First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
10.0First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
Consistency
8.5Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
8.6Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
Attempts
66/69
69/69
Tests Correct
A test is fully passed only if every run passed for that test.Extra formatting: 4Wrong answer: 4Did not follow instructions: 1Response Time (avg)32.23sResponse Time (max)151.51sResponse Time (total)515.65sA test is fully passed only if every run passed for that test.…
A test is fully passed only if every run passed for that test.Wrong answer: 8Response Time (avg)18.39sResponse Time (max)111.09sResponse Time (total)422.93sA test is fully passed only if every run passed for that test.…
Attempt pass rate
60.9%Attempt pass rate = passed attempts / total attempts across runs.…
71.0%Attempt pass rate = passed attempts / total attempts across runs.…
Flaky tests
3Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
4Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
Total Runs
66Total Runs…
69Total Runs…
Cost per result
22.777
5.882
Total Cost
$2.961
$0.118
Input Price
$5.000 / 1MInput Price…
$0.200 / 1MInput Price…
Output Price
$25.000 / 1MOutput Price…
$1.200 / 1MOutput Price…
Cache Read Price
$0.500 / 1MCache Read Price…
$0.020 / 1MCache Read Price…
Cache Write Price
$6.250 / 1MCache Write Price…
$0.250 / 1MCache Write Price…
Total Input Tokens
108,573Total Input Tokens…
197,896Total Input Tokens…
Output Tokens
69,994Output Tokens…
6,211Output Tokens…
Reasoning Tokens
26,728Reasoning Tokens…
133,184Reasoning Tokens…
Response Time (avg)
32.23sResponse Time (avg)…
18.39sResponse Time (avg)…
Response Time (max)
151.51sResponse Time (max)…
111.09sResponse Time (max)…
Response Time (total)
515.65sResponse Time (total)…
422.93sResponse Time (total)…
Parameters
~5T total (~500B active)
~400B total (~17B active)
Availability
Closed
Closed
Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#261 Claude Opus 4.6
medium
Reached the allocated time limit (300 seconds) without receiving showcase output.
Cost
$0.000
Time
300.0s
Tokens
0 tok
#110 GPT-5.6 Luna
high
Cost
$0.033
Time
34.2s
Tokens
5,484 tok
Rank
-
Cost
-
Time
-
Tokens
-
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
Agentic
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
0.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
0.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0msA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)60.15sResponse Time (max)60.15sResponse Time (total)60.15sA test is fully passed only if every run passed for that test.…
60.15sResponse Time (avg)…
116,969Total Input Tokens…
1,123Output Tokens…
2,957Reasoning Tokens…
Anti-AI Tricks
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
6.4Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
5.8Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
66.7%Attempt pass rate = passed attempts / total attempts across runs.…
2Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Extra formatting: 2Response Time (avg)7.45sResponse Time (max)11.88sResponse Time (total)14.90sA test is fully passed only if every run passed for that test.…
8.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
75.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.99sResponse Time (max)4.94sResponse Time (total)11.95sA test is fully passed only if every run passed for that test.…
2.99sResponse Time (avg)…
606Total Input Tokens…
162Output Tokens…
1,265Reasoning Tokens…
Coding
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
5.7Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.1Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
44.4%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Extra formatting: 1Did not follow instructions: 1Response Time (avg)30.10sResponse Time (max)35.63sResponse Time (total)90.31sA test is fully passed only if every run passed for that test.…
5.5Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
4.7Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
55.6%Attempt pass rate = passed attempts / total attempts across runs.…
2Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)15.63sResponse Time (max)32.41sResponse Time (total)46.88sA test is fully passed only if every run passed for that test.…
15.63sResponse Time (avg)…
7,302Total Input Tokens…
510Output Tokens…
17,746Reasoning Tokens…
Combined
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)114.08sResponse Time (max)151.51sResponse Time (total)228.16sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)19.05sResponse Time (max)24.81sResponse Time (total)38.10sA test is fully passed only if every run passed for that test.…
19.05sResponse Time (avg)…
57,719Total Input Tokens…
3,366Output Tokens…
8,005Reasoning Tokens…
Data parsing and extraction
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.37sResponse Time (max)7.37sResponse Time (total)7.37sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.18sResponse Time (max)2.64sResponse Time (total)4.36sA test is fully passed only if every run passed for that test.…
2.18sResponse Time (avg)…
7,140Total Input Tokens…
234Output Tokens…
470Reasoning Tokens…
Domain specific
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 2Extra formatting: 1Response Time (avg)42.53sResponse Time (max)83.40sResponse Time (total)85.06sA test is fully passed only if every run passed for that test.…
5.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
44.4%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)62.87sResponse Time (max)111.09sResponse Time (total)188.61sA test is fully passed only if every run passed for that test.…
62.87sResponse Time (avg)…
732Total Input Tokens…
69Output Tokens…
75,712Reasoning Tokens…
General Intelligence
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.04sResponse Time (max)5.04sResponse Time (total)5.04sA test is fully passed only if every run passed for that test.…
5.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.65sResponse Time (max)3.65sResponse Time (total)3.65sA test is fully passed only if every run passed for that test.…
3.65sResponse Time (avg)…
477Total Input Tokens…
117Output Tokens…
563Reasoning Tokens…
Instructions following
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.43sResponse Time (max)2.43sResponse Time (total)2.43sA test is fully passed only if every run passed for that test.…
9.9Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.79sResponse Time (max)1.96sResponse Time (total)3.58sA test is fully passed only if every run passed for that test.…
1.79sResponse Time (avg)…
660Total Input Tokens…
97Output Tokens…
481Reasoning Tokens…
Puzzle Solving
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
7.7Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
66.7%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.71sResponse Time (max)4.75sResponse Time (total)9.41sA test is fully passed only if every run passed for that test.…
7.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
77.8%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)14.55sResponse Time (max)38.14sResponse Time (total)43.66sA test is fully passed only if every run passed for that test.…
14.55sResponse Time (avg)…
642Total Input Tokens…
242Output Tokens…
18,281Reasoning Tokens…
Tool Calling
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.73sResponse Time (max)9.73sResponse Time (total)9.73sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.98sResponse Time (max)4.98sResponse Time (total)4.98sA test is fully passed only if every run passed for that test.…
4.98sResponse Time (avg)…
5,454Total Input Tokens…
260Output Tokens…
324Reasoning Tokens…
Trivia
Score
Consistency
Attempt pass rate
Flaky tests
Tests Correct
Response Time (avg)
Input Tokens
Output Tokens
Reasoning Tokens
Claude Opus 4.6Archived model: this model is no longer updated or tested on new tests.
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)63.24sResponse Time (max)63.24sResponse Time (total)63.24sA test is fully passed only if every run passed for that test.…
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)17.00sResponse Time (max)17.00sResponse Time (total)17.00sA test is fully passed only if every run passed for that test.…