Navigate
AI BENCHY
Advertise here

Gemini 3.1 Flash Lite (low) vs Mercury 2.5 Preview (high)

The average score is effectively tied at 6.6 vs 6.6. Mercury 2.5 Preview (high) has the lower benchmark cost at $0.030 vs $0.621. Mercury 2.5 Preview (high) is faster at 4.01s vs 16.25s, with pass rates of 63.6% vs 65.2%.

Last updated at: 2026-09-02

Rank
#155
Total Output Tokens
397,891
Response Time (avg)
16.25s
Total Cost
$0.621
Rank
#157
Total Output Tokens
163,505
Response Time (avg)
4.01s
Total Cost
$0.030
Recommended model Mercury 2.5 Preview (high)

It has the best score here (6.6), while costing about 21.3x less than Gemini 3.1 Flash Lite (low).

Detailed comparison

Metric Gemini 3.1 Flash Lite Gemini 3.1 Flash Lite low Release: 2026-05-08 Mercury 2.5 Preview Mercury 2.5 Preview high Release: 2026-09-02
Score 6.6 6.6
Rank #155 #157
Reliability 10.0 10.0
Consistency 9.2 8.2
Attempts 66/66 66/66
Tests Correct
Attempt pass rate 63.6% 65.2%
Flaky tests 2 5
Total Runs 66 66
Cost per result 4.773 0.243
Total Cost $0.621 $0.030
Input Price $0.250 / 1M $0.040 / 1M
Output Price $1.500 / 1M $0.150 / 1M
Total Input Tokens 94,233 115,759
Output Tokens 7,759 2,562
Reasoning Tokens 390,132 160,943
Response Time (avg) 16.25s 4.01s
Response Time (max) 318.02s 17.71s
Response Time (total) 357.39s 88.25s
Parameters ~150B total (~10B active) ~100B
Availability Closed Closed

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#155 Gemini 3.1 Flash Lite

low
Cost
$0.003
Time
4.0s
Tokens
1,479 tok

#157 Mercury 2.5 Preview

high
Cost
$0.001
Time
4.5s
Tokens
3,851 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemini 3.1 Flash Lite 5.5 10.0 33.3% 0 1.53s 8,132 471 1,072
Mercury 2.5 Preview 6.7 7.8 55.6% 1 6.13s 7,751 485 42,569

Quick Compare

Switch Comparison Pair

Gemini 3.1 Flash LitelowvsInkling SmallmediumGemini 3.1 Flash LitelowvsQwen3.6 35B A3BmediumMercury 2.5 PreviewhighvsQwen3.6 35B A3BmediumMercury 2.5 PreviewhighvsQwen3.5 Plus 2026-02-15noneGemini 3.1 Flash LitelowvsQwen3.5-27BnoneMercury 2.5 PreviewhighvsInkling SmallmediumGemini 3.1 Flash Lite PreviewlowvsMercury 2.5 PreviewhighGemini 3.1 Flash LitelowvsQwen3.5 Plus 2026-02-15noneMercury 2.5 PreviewhighvsQwen3.5-27BnoneGemini 3.5 Flash LitelowvsMercury 2.5 PreviewhighMercury 2.5 PreviewhighvsQwen3.6 27BmediumMercury 2.5 PreviewhighvsLaguna XS 2.1mediumFree Available