- Rank
- #225
- Total Output Tokens
- 156,181
- Response Time (avg)
- 14.35s
- Total Cost
- $1.778
Gemini 3.5 Flash vs Solar Pro 4 (high)
Solar Pro 4 (high) leads on average score with 6.3 vs 6.3. Solar Pro 4 (high) has the lower benchmark cost at $0.164 vs $1.778. Gemini 3.5 Flash is faster at 14.35s vs 128.91s, with pass rates of 66.7% vs 60.9%.
Compared models
- Rank
- #219
- Total Output Tokens
- 402,203
- Response Time (avg)
- 128.91s
- Total Cost
- $0.164
Recommended model
Solar Pro 4 (high)
It has the best score here (6.3), while costing about 10.9x less than Gemini 3.5 Flash.
Detailed comparison
| Metric | Gemini 3.5 Flash Gemini 3.5 Flash none | Solar Pro 4 Solar Pro 4 high |
|---|---|---|
| Score | 6.3 | 6.3 |
| Rank | #225 | #219 |
| Reliability | 10.0 | 10.0 |
| Consistency | 8.9 | 8.0 |
| Attempts | 69/69 | 69/69 |
| Tests Correct | ||
| Attempt pass rate | 66.7% | 60.9% |
| Flaky tests | 3 | 6 |
| Total Runs | 69 | 69 |
| Cost per result | 12.695 | 0.652 |
| Total Cost | $1.778 | $0.164 |
| Input Price | $1.500 / 1M | $0.090 / 1M |
| Output Price | $9.000 / 1M | $0.360 / 1M |
| Total Input Tokens | 247,729 | 209,002 |
| Output Tokens | 156,181 | 24,675 |
| Reasoning Tokens | 0 | 377,528 |
| Response Time (avg) | 14.35s | 128.91s |
| Response Time (max) | 89.30s | 491.07s |
| Response Time (total) | 272.60s | 2965.04s |
| Parameters | ~500B total (~20B active) | ~100B |
| Availability | Closed | Closed |
Model generation showcase
Hamster playing table tennis
Prompt: Create a detailed SVG illustration of a hamster playing table tennis.
#225 Gemini 3.5 Flash
none- Cost
- $0.225
- Time
- 125.5s
- Tokens
- 25,004 tok
#219 Solar Pro 4
high
No showcase result has been generated for this model yet.
- Cost
- N/A
- Time
- -
- Tokens
- 0 tok
Top Models by Score
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Category Breakdown
| Agentic | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 3.5 | 8.3 | 0.0% | 0 | 89.30s | 233,877 | 37,081 | 0 | |
| Solar Pro 4 | 2.8 | 1.6 | 33.3% | 1 | 312.27s | 129,262 | 1,318 | 37,691 |
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 2.53s | 492 | 5,101 | 0 | |
| Solar Pro 4 | 8.7 | 7.9 | 91.7% | 1 | 58.83s | 1,104 | 313 | 40,629 |
| Coding | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 8.8 | 7.8 | 88.9% | 1 | 34.69s | 8,122 | 75,927 | 0 | |
| Solar Pro 4 | 5.7 | 9.9 | 33.3% | 0 | 244.73s | 7,887 | 18,577 | 101,928 |
| Combined | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 3.0 | 10.0 | 0.0% | 0 | 0ms | 0 | 0 | 0 | |
| Solar Pro 4 | 7.3 | 5.8 | 83.3% | 1 | 167.40s | 51,594 | 3,394 | 37,597 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 6.5 | 10.0 | 50.0% | 0 | 8.10s | 2,781 | 5,895 | 0 | |
| Solar Pro 4 | 10.0 | 10.0 | 100.0% | 0 | 62.46s | 7,575 | 252 | 22,209 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 5.3 | 7.2 | 44.4% | 1 | 12.18s | 642 | 19,492 | 0 | |
| Solar Pro 4 | 3.6 | 7.2 | 22.2% | 1 | 280.84s | 1,098 | 37 | 79,201 |
| General Intelligence | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 3.46s | 486 | 1,620 | 0 | |
| Solar Pro 4 | 5.5 | 10.0 | 0.0% | 0 | 42.22s | 609 | 123 | 7,474 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 9.8 | 10.0 | 100.0% | 0 | 3.38s | 615 | 3,928 | 0 | |
| Solar Pro 4 | 7.9 | 6.9 | 66.7% | 1 | 41.51s | 909 | 75 | 13,182 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 10.0 | 10.0 | 100.0% | 0 | 3.13s | 558 | 4,640 | 0 | |
| Solar Pro 4 | 8.7 | 7.9 | 77.8% | 1 | 55.22s | 1,017 | 345 | 23,424 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 3.0 | 10.0 | 0.0% | 0 | 0ms | 0 | 0 | 0 | |
| Solar Pro 4 | 10.0 | 10.0 | 100.0% | 0 | 8.96s | 7,620 | 225 | 770 |
| Trivia | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Response Time (avg) | Input Tokens | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.5 Flash | 2.8 | 1.6 | 33.3% | 1 | 4.87s | 156 | 2,497 | 0 | |
| Solar Pro 4 | 3.0 | 10.0 | 0.0% | 0 | 81.13s | 327 | 16 | 13,423 |
Quick Compare
Switch Comparison Pair
Gemini 3.1 Flash LitelowvsSolar Pro 4highQwen3.5 Plus 2026-04-20nonevsSolar Pro 4highGemini 3.5 FlashnonevsGrok 4.20mediumGemini 3.1 Flash Lite PreviewlowvsSolar Pro 4highSpace Bunny AlphamediumvsSolar Pro 4highGemini 3.5 Flash LitelowvsSolar Pro 4highGemini 3.5 FlashnonevsSpace Bunny AlphaxhighDeepSeek V3.2mediumvsSolar Pro 4highDeepSeek V3.2mediumvsGemini 3.5 FlashnoneQwen3.6 27BnonevsSolar Pro 4highGemini 3.5 FlashnonevsKAT-Coder-Pro V2.5mediumQwen3.5-35B-A3BmediumvsSolar Pro 4high