- Rank
- #80
- Total Output Tokens
- 67,842
- Response Time (avg)
- 16.72s
- Total Cost
- $3.490
Claude Opus 4.8 (medium) vs Gemini 3.5 Flash (high)
The average score is effectively tied at 8.3 vs 8.3. Gemini 3.5 Flash (high) has the lower benchmark cost at $2.737 vs $3.490. Claude Opus 4.8 (medium) is faster at 16.72s vs 18.59s, with pass rates of 82.6% vs 85.5%.
Compared models
- Rank
- #77
- Total Output Tokens
- 246,012
- Response Time (avg)
- 18.59s
- Total Cost
- $2.737
Recommended model
Gemini 3.5 Flash (high)
It has the strongest score in this comparison (8.3) and the best overall balance of cost and response time across all 2 models.
Detailed comparison
| Metric | Claude Opus 4.8 Claude Opus 4.8 medium | Gemini 3.5 Flash Gemini 3.5 Flash high |
|---|---|---|
| Score | 8.3 | 8.3 |
| Rank | #80 | #77 |
| Reliability | 10.0 | 10.0 |
| Consistency | 9.0 | 9.3 |
| Attempts | 69/69 | 69/69 |
| Tests Correct | ||
| Attempt pass rate | 82.6% | 85.5% |
| Flaky tests | 3 | 2 |
| Total Runs | 69 | 69 |
| Cost per result | 20.527 | 14.402 |
| Total Cost | $3.490 | $2.737 |
| Input Price | $5.000 / 1M | $1.500 / 1M |
| Output Price | $25.000 / 1M | $9.000 / 1M |
| Total Input Tokens | 360,294 | 348,103 |
| Output Tokens | 43,441 | 11,045 |
| Reasoning Tokens | 24,401 | 234,967 |
| Response Time (avg) | 16.72s | 18.59s |
| Response Time (max) | 99.37s | 145.92s |
| Response Time (total) | 384.66s | 427.52s |
| Parameters | ~5T total (~500B active) | ~500B total (~20B active) |
| Availability | Closed | Closed |
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#80 Claude Opus 4.8
medium- Cost
- $0.057
- Time
- 23.1s
- Tokens
- 2,412 tok
#77 Gemini 3.5 Flash
high- Cost
- $0.208
- Time
- 118.2s
- Tokens
- 23,158 tok
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
| Agentic | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 6.1 | 3.1 | 66.7% | 1 | 99.37s | 221,846 | 2,676 | 13,239 | |
| Gemini 3.5 Flash | 3.4 | 9.6 | 0.0% | 0 | 87.32s | 240,957 | 2,268 | 38,262 |
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 10.0 | 10.0 | 100.0% | 0 | 3.95s | 834 | 1,179 | 478 | |
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 2.57s | 492 | 174 | 4,997 |
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 10.0 | 10.0 | 100.0% | 0 | 15.33s | 10,590 | 9,945 | 1,381 | |
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 22.96s | 8,118 | 456 | 47,129 |
| Combined | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 9.9 | 10.0 | 100.0% | 0 | 54.29s | 101,005 | 19,531 | 4,762 | |
| Gemini 3.5 Flash | 8.2 | 6.9 | 66.7% | 1 | 84.14s | 82,416 | 7,153 | 93,585 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 7.1 | 5.6 | 83.3% | 1 | 12.29s | 10,503 | 481 | 312 | |
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 6.43s | 7,548 | 279 | 8,466 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 3.6 | 7.2 | 22.2% | 1 | 18.11s | 972 | 7,476 | 2,987 | |
| Gemini 3.5 Flash | 5.3 | 7.2 | 44.4% | 1 | 16.99s | 642 | 12 | 28,526 |
| General Intelligence | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 10.0 | 10.0 | 100.0% | 0 | 2.46s | 708 | 237 | 0 | |
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 3.63s | 486 | 115 | 1,650 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 10.0 | 10.0 | 100.0% | 0 | 3.32s | 909 | 373 | 320 | |
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 3.35s | 615 | 70 | 3,799 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 10.0 | 10.0 | 100.0% | 0 | 3.95s | 894 | 791 | 483 | |
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 3.23s | 558 | 241 | 4,940 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 10.0 | 10.0 | 100.0% | 0 | 8.96s | 11,775 | 301 | 225 | |
| Gemini 3.5 Flash | 9.8 | 10.0 | 100.0% | 0 | 4.96s | 6,115 | 265 | 1,608 |
| Trivia | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 3.0 | 10.0 | 0.0% | 0 | 6.14s | 258 | 451 | 214 | |
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 3.94s | 156 | 12 | 2,005 |
Quick Compare
Switch Comparison Pair
Gemini 3.5 FlashhighvsKimi K3maxClaude Opus 4.8mediumvsGrok 4.5highClaude Opus 4.8mediumvsKimi K3maxGemini 3.5 FlashhighvsGPT-6 LunamediumGemini 3.5 FlashhighvsQwen3.7 PlusmediumClaude Opus 4.8mediumvsGPT-5.6 TerrahighGemini 3.5 FlashhighvsGLM 5.2mediumGemini 3.5 FlashhighvsGPT-5.6 TerramediumClaude Opus 4.8mediumvsGLM 5.2highClaude Opus 4.8mediumvsDeepSeek V4 Flash 0731highGemini 3.5 FlashhighvsMuse Glimmer 30BxhighGemini 3.5 FlashhighvsGPT-5 Minimedium