Navigate
Advertise here

Gemma 4 31B (medium) vs GPT-6 Sol

Gemma 4 31B (medium) leads on average score with 6.9 vs 6.8. Gemma 4 31B (medium) has the lower benchmark cost at $0.144 vs $0.551. GPT-6 Sol is faster at 9.02s vs 78.76s, with pass rates of 66.7% vs 63.8%.

Last updated at: 2026-11-13

Compared models

Rank
#236
Total Output Tokens
277,315
Response Time (avg)
78.76s
Total Cost
$0.144
Rank
#244
Total Output Tokens
7,989
Response Time (avg)
9.02s
Total Cost
$0.551
Recommended model Gemma 4 31B (medium)

It has the best score here (6.9), while costing about 3.8x less than GPT-6 Sol.

Detailed comparison

Metric Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02 Free Available GPT-6 Sol GPT-6 Sol none Release: 2026-09-23
Score 6.9 6.8
Rank #236 #244
Reliability 10.0 10.0
Consistency 8.7 8.5
Attempts 69/69 69/69
Tests Correct
Attempt pass rate 66.7% 63.8%
Flaky tests 3 4
Total Runs 69 69
Cost per result 1.195 4.232
Total Cost $0.144 $0.551
Input Price $0.090 / 1M $2.000 / 1M
Output Price $0.340 / 1M $10.000 / 1M
Cache Read Price $0.050 / 1M $0.200 / 1M
Cache Write Price N/A $2.500 / 1M
Total Input Tokens 251,164 235,086
Output Tokens 36,112 7,989
Reasoning Tokens 241,203 0
Response Time (avg) 78.76s 9.02s
Response Time (max) 437.40s 57.02s
Response Time (total) 1653.96s 207.45s
Parameters 30.7B ~2T total (~150B active)
Availability Open source Closed

Cache prices apply to input tokens. Reads reuse cached prompts; writes store them and can cost extra. Output tokens use the output price.

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#236 Gemma 4 31B

medium
Cost
$0.002
Time
45.7s
Tokens
2,696 tok

#244 GPT-6 Sol

none
Cost
$0.030
Time
31.9s
Tokens
3,055 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 4.3 5.8 22.2% 1 219.76s 5,568 11,098 33,212
GPT-6 Sol 5.5 10.0 33.3% 0 12.23s 7,302 389 0

Switch Comparison Pair