Mistral Large 4 (low) vs Qwen3.8 Flash Next (medium)
The average score is effectively tied at 7.6 vs 7.6. Qwen3.8 Flash Next (medium) has the lower benchmark cost at ~$0.047 vs $1.648. Qwen3.8 Flash Next (medium) is faster at 24.79s vs 300.15s, with pass rates of 65.2% vs 75.4%.
Last updated at:
2026-10-06
Compared models
Rank
#119
Total Output Tokens
714,287
Response Time (avg)
300.15s
Total Cost
$1.648
Candidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
Rank
#120
Total Output Tokens
143,965
Response Time (avg)
24.79s
Total Cost
~$0.047Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.4752 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.046984.
Recommended modelQwen3.8 Flash Next (medium)
It has the best score here (7.6), while costing about 35.1x less than Mistral Large 4 (low).
Qwen3.8 Flash NextQwen3.8 Flash NextmediumCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.Release: 2026-10-04
Qwen3.8 Flash NextQwen3.8 Flash NextmediumCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.Release: 2026-10-04
Score
7.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.6Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
Rank
#119
#120
Reliability
9.0First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
10.0First-attempt success score: 10.0 means no retryable target API or rate-limit failures before successful calls; tracked failures lower the score.…
Consistency
8.7Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
7.9Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
Attempts
69/69
69/69
Tests Correct
A test is fully passed only if every run passed for that test.Wrong answer: 6Did not follow instructions: 2No answer: 2Response Time (avg)300.15sResponse Time (max)1798.01sResponse Time (total)6903.40sA test is fully passed only if every run passed for that test.…
A test is fully passed only if every run passed for that test.Wrong answer: 6Did not follow instructions: 1No answer: 1Response Time (avg)24.79sResponse Time (max)159.62sResponse Time (total)570.26sA test is fully passed only if every run passed for that test.…
Attempt pass rate
65.2%Attempt pass rate = passed attempts / total attempts across runs.…
75.4%Attempt pass rate = passed attempts / total attempts across runs.…
Flaky tests
4Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
6Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
Total Runs
69Total Runs…
69Total Runs…
Cost per result
12.670
~0.314Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.4752 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.046984.
Total Cost
$1.648
~$0.047Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.4752 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.046984.
Input Price
$0.680 / 1MInput Price…
N/AInput Price…
Output Price
$2.090 / 1MOutput Price…
N/AOutput Price…
Cache Read Price
$0.070 / 1MCache Read Price…
N/ACache Read Price…
Cache Write Price
N/ACache Write Price…
N/ACache Write Price…
Total Input Tokens
226,734Total Input Tokens…
278,792Total Input Tokens…
Output Tokens
278,013Output Tokens…
1,695Output Tokens…
Reasoning Tokens
638,296Reasoning Tokens…
142,270Reasoning Tokens…
Response Time (avg)
300.15sResponse Time (avg)…
24.79sResponse Time (avg)…
Response Time (max)
1798.01sResponse Time (max)…
159.62sResponse Time (max)…
Response Time (total)
6903.40sResponse Time (total)…
570.26sResponse Time (total)…
Parameters
1.05T total (49B active)
180B total (6B active)
Availability
Closed
Weights available
Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#119 Mistral Large 4
low
Cost
$0.015
Time
115.1s
Tokens
6,915 tok
#120 Qwen3.8 Flash Next
mediumCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
Cost
~$0.002Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.0113 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.001119.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)69.63sResponse Time (max)69.63sResponse Time (total)69.63sA test is fully passed only if every run passed for that test.…
69.63sResponse Time (avg)…
143,243Total Input Tokens…
3,788Output Tokens…
3,053Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
4.7Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
3.1Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
33.3%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)159.62sResponse Time (max)159.62sResponse Time (total)159.62sA test is fully passed only if every run passed for that test.…
8.3Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
75.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)19.75sResponse Time (max)61.41sResponse Time (total)79.00sA test is fully passed only if every run passed for that test.…
19.75sResponse Time (avg)…
705Total Input Tokens…
999Output Tokens…
9,354Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.81sResponse Time (max)2.74sResponse Time (total)7.24sA test is fully passed only if every run passed for that test.…
3.4Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
22.2%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No answer: 2Did not follow instructions: 1Response Time (avg)1379.72sResponse Time (max)1798.01sResponse Time (total)4139.16sA test is fully passed only if every run passed for that test.…
1379.72sResponse Time (avg)…
5,896Total Input Tokens…
129,426Output Tokens…
386,126Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)45.53sResponse Time (max)98.91sResponse Time (total)136.59sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)556.27sResponse Time (max)1036.66sResponse Time (total)1112.54sA test is fully passed only if every run passed for that test.…
556.27sResponse Time (avg)…
60,360Total Input Tokens…
5,603Output Tokens…
60,531Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
6.4Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
5.8Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
66.7%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)76.72sResponse Time (max)116.08sResponse Time (total)153.43sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)27.46sResponse Time (max)29.74sResponse Time (total)54.92sA test is fully passed only if every run passed for that test.…
27.46sResponse Time (avg)…
7,314Total Input Tokens…
1,014Output Tokens…
7,623Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.65sResponse Time (max)7.50sResponse Time (total)13.31sA test is fully passed only if every run passed for that test.…
2.9Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
11.1%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)291.62sResponse Time (max)350.20sResponse Time (total)874.85sA test is fully passed only if every run passed for that test.…
291.62sResponse Time (avg)…
798Total Input Tokens…
82,605Output Tokens…
111,098Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
2.9Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
4.4Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
22.2%Attempt pass rate = passed attempts / total attempts across runs.…
2Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)17.29sResponse Time (max)18.55sResponse Time (total)51.86sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)39.29sResponse Time (max)39.29sResponse Time (total)39.29sA test is fully passed only if every run passed for that test.…
39.29sResponse Time (avg)…
516Total Input Tokens…
5,529Output Tokens…
5,865Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
4.8Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
3.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
33.3%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)7.29sResponse Time (max)7.29sResponse Time (total)7.29sA test is fully passed only if every run passed for that test.…
8.4Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
6.9Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
83.3%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)23.21sResponse Time (max)39.23sResponse Time (total)46.42sA test is fully passed only if every run passed for that test.…
23.21sResponse Time (avg)…
708Total Input Tokens…
5,408Output Tokens…
6,402Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
9.8Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.04sResponse Time (max)2.85sResponse Time (total)4.09sA test is fully passed only if every run passed for that test.…
7.7Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.3Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
77.8%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)35.82sResponse Time (max)63.21sResponse Time (total)107.45sA test is fully passed only if every run passed for that test.…
35.82sResponse Time (avg)…
717Total Input Tokens…
8,012Output Tokens…
9,030Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
8.2Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
7.2Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
88.9%Attempt pass rate = passed attempts / total attempts across runs.…
1Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.23sResponse Time (max)2.98sResponse Time (total)6.69sA test is fully passed only if every run passed for that test.…
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)21.53sResponse Time (max)21.53sResponse Time (total)21.53sA test is fully passed only if every run passed for that test.…
21.53sResponse Time (avg)…
6,267Total Input Tokens…
307Output Tokens…
1,015Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
10.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
100.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.75sResponse Time (max)5.75sResponse Time (total)5.75sA test is fully passed only if every run passed for that test.…
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)358.62sResponse Time (max)358.62sResponse Time (total)358.62sA test is fully passed only if every run passed for that test.…
358.62sResponse Time (avg)…
210Total Input Tokens…
35,322Output Tokens…
38,199Reasoning Tokens…
Qwen3.8 Flash NextCandidate responses were generated on local hardware. OpenRouter was used only for judging. qwen3.8-flash-next-iq3_s · IQ3_S · Strata 0.1.39 · NVIDIA GeForce RTX 3090.
3.0Summarizes broad quality across our full private benchmark suite, so ranking reflects consistent performance.…
10.0Consistency score reflects run-to-run stability (10 = very consistent, even if consistently wrong).…
0.0%Attempt pass rate = passed attempts / total attempts across runs.…
0Flaky tests had mixed outcomes across runs (at least one pass and one fail).…
A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)24.39sResponse Time (max)24.39sResponse Time (total)24.39sA test is fully passed only if every run passed for that test.…