#274 Mistral Large 4
default- Cost
- $0.004
- Time
- 15.2s
- Tokens
- 1,547 tok
Summary
Mistral Large 4 scores 5.5 on AI BENCHY and ranks #274. It has 9.9 reliability, a 30.4% pass rate, $0.353 total cost, and 28.46s average response time.
Researched on 2026-10-06
Mistral reports 1.05T total and 49B active parameters, plus a 1.6B vision encoder. The vendor describes this public preview as open-weight, but its checkpoint page currently supplies no weight download or license, and no Large 4 checkpoint was found in the official Hugging Face registry. Classified as API-only until the exact weights and license can be verified. The OpenRouter route exposes 524,288 context tokens and 262,144 completion tokens, with tools and structured outputs but no configurable reasoning parameter.
5.5
Consistency
8.8
9.9
$0.353
Total Output Tokens
84,218
Total Input Tokens
259,048
Input Price
$0.680 / 1M
Output Price
$2.090 / 1M
Cache Read Price
$0.070 / 1M
Cache Write Price
N/A
Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.
Flaky tests
4
Flaky tests had mixed outcomes across runs (at least one pass and one fail).
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
Run history
| Tested on | Score | Reliability | Tests Correct | Total Cost | Compare |
|---|---|---|---|---|---|
| 2026-10-06 18:09 Re-test | 5.7 | 9.7 | $0.290 | Compare | |
| 2026-10-06 15:23 Initial run | 5.5 | 9.9 | $0.353 | Current run |
Run comparison
| Run | Benchmark coverage | Score | Consistency | Reliability | Tests Correct | Flaky tests | Total Output Tokens | Total Input Tokens | Total Cost | Response Time (avg) |
|---|---|---|---|---|---|---|---|---|---|---|
| 2026-10-06 15:23 · Initial run | 69/69 attempts | 5.5 | 8.8 | 9.9 | 5/23 | 4 | 84,218 | 259,048 | $0.353 | 28.46s |
| 2026-10-06 18:09 · Re-test | 69/69 attempts | 5.7 | 9.3 | 9.7 | 6/23 | 2 | 54,388 | 257,872 | $0.290 | 20.43s |
| Difference | — | -0.2 | -0.5 | +0.2 | -1 | +2 | +29830 | +1176 | +$0.064 | +8026ms |
Choose the first model, then click a second model to open a side-by-side page.
| Category | Score | Consistency | Tests Correct |
|---|---|---|---|
| Agentic | 4.7 | 3.1 | |
| Anti-AI Tricks | 3.7 | 8.4 | |
| Coding | 5.4 | 7.8 | |
| Combined | 3.0 | 10.0 | |
| Data parsing and extraction | 10.0 | 10.0 | |
| Domain specific | 3.0 | 10.0 | |
| General Intelligence | 5.1 | 10.0 | |
| Instructions following | 6.5 | 10.0 | |
| Puzzle Solving | 6.1 | 7.1 | |
| Tool Calling | 10.0 | 10.0 | |
| Trivia | 3.0 | 10.0 |