AI BENCHY Compare
Anthropic: Claude Sonnet 4.6 vs Google: Gemini 3.1 Flash Lite Preview
Compare:
Last updated at: 2026-03-03
| Metric | Anthropic: Claude Sonnet 4.6 medium Release: 2026-02-17 | Google: Gemini 3.1 Flash Lite Preview none Release: 2026-03-03 |
|---|---|---|
| Rank | #11 | #10 |
| Avg Score | 7.43 | 7.70 |
| Consistency | 9.40 | 9.54 |
| Cost per result | 8.105 | 0.116 |
| Total Cost | $0.811 | $0.011 |
| Tests Correct | ||
| Attempt pass rate | 73.8% | 69.1% |
| Flaky tests | 1 | 1 |
| Output Tokens | 29,098 | 4,307 |
| Reasoning Tokens | 20,435 | 0 |
Score vs Total Cost
Category Breakdown
| Anti-AI Tricks | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|
| Anthropic: Claude Sonnet 4.6 | 7.00 | 10.00 | 66.7% | 0 | 1,031 | 1,093 | |
| Google: Gemini 3.1 Flash Lite Preview | 6.00 | 7.85 | 55.6% | 1 | 1,086 | 0 |
| Data parsing and extraction | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|
| Anthropic: Claude Sonnet 4.6 | 10.00 | 10.00 | 100.0% | 0 | 727 | 907 | |
| Google: Gemini 3.1 Flash Lite Preview | 9.88 | 10.00 | 100.0% | 0 | 399 | 0 |
| Domain specific | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|
| Anthropic: Claude Sonnet 4.6 | 1.00 | 7.21 | 11.1% | 1 | 25,790 | 16,919 | |
| Google: Gemini 3.1 Flash Lite Preview | 4.00 | 10.00 | 33.3% | 0 | 568 | 0 |
| Instructions following | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|
| Anthropic: Claude Sonnet 4.6 | 10.00 | 10.00 | 100.0% | 0 | 316 | 523 | |
| Google: Gemini 3.1 Flash Lite Preview | 9.00 | 10.00 | 50.0% | 0 | 574 | 0 |
| Puzzle Solving | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|
| Anthropic: Claude Sonnet 4.6 | 10.00 | 10.00 | 100.0% | 0 | 579 | 642 | |
| Google: Gemini 3.1 Flash Lite Preview | 10.00 | 10.00 | 100.0% | 0 | 898 | 0 |
| Tool Calling | Score | Consistency | Attempt pass rate | Flaky tests | Tests Correct | Output Tokens | Reasoning Tokens |
|---|---|---|---|---|---|---|---|
| Anthropic: Claude Sonnet 4.6 | 10.00 | 10.00 | 100.0% | 0 | 655 | 351 | |
| Google: Gemini 3.1 Flash Lite Preview | 10.00 | 10.00 | 100.0% | 0 | 782 | 0 |
Quick Compare
Switch Comparison Pair
Claude Sonnet 4.6mediumvsGPT-5.2 ChatnoneClaude Sonnet 4.6mediumvsGPT-5.3 ChatnoneClaude Sonnet 4.6mediumvsGemini 3 Flash PreviewnoneGemini 3.1 Flash Lite PreviewnonevsGPT-5.3-CodexmediumClaude Sonnet 4.6mediumvsGemini 3.1 Flash Lite PreviewhighClaude Sonnet 4.6mediumvsGemini 3.1 Flash Lite PreviewlowGemini 3.1 Flash Lite PreviewnonevsGLM 5mediumGemini 3.1 Flash Lite PreviewnonevsStep 3.5 FlashmediumFree AvailableDeepSeek V3.2mediumvsGemini 3.1 Flash Lite PreviewnoneGemini 3.1 Flash Lite PreviewnonevsGPT-5.2mediumGemini 3.1 Flash Lite PreviewnonevsQwen3.5-27BmediumGemini 3.1 Flash Lite PreviewnonevsQwen3.5-122B-A10Bmedium