Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Google: Gemma 4 31B vs StepFun: Step 3.5 Flash

Last updated at: 2026-04-16

Metric Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available Step 3.5 Flash Step 3.5 Flash medium Release: 2026-02-01
Score 6.9 7.9
Rank #47 #30
Consistency 10.0 9.1
Tests Correct
Attempt pass rate 55.6% 70.6%
Flaky tests 0 2
Total Runs 54 49
Cost per result 0.024 0.000
Total Cost $0.003 $0.000
Input Price $0.130 / 1M $0.100 / 1M
Output Price $0.380 / 1M $0.300 / 1M
Output Tokens 1,359 71,904
Reasoning Tokens 0 155,607
Response Time (avg) 4.02s 26.78s
Response Time (max) 26.13s 170.45s
Response Time (total) 64.33s 294.58s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 6.5 10.0 50.0% 0 1.85s 45 0
Step 3.5 Flash 10.0 10.0 100.0% 0 13.56s 14,376 17,668
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 26.13s 699 0
Step 3.5 Flash - - - - - - - -
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0
Step 3.5 Flash 10.0 10.0 100.0% 0 29.57s 1,176 12,984
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 2.25s 285 0
Step 3.5 Flash 10.0 10.0 100.0% 0 15.01s 600 13,886
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 7.7 10.0 66.7% 0 3.22s 27 0
Step 3.5 Flash 5.3 7.2 44.4% 1 170.45s 45,350 90,436
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 2.09s 117 0
Step 3.5 Flash 5.5 10.0 0.0% 0 6.54s 2,214 2,584
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 6.5 10.0 50.0% 0 2.84s 78 0
Step 3.5 Flash 8.5 6.8 83.3% 1 4.98s 2,284 3,412
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 5.5 10.0 33.3% 0 2.95s 108 0
Step 3.5 Flash 5.3 10.0 33.3% 0 7.72s 5,629 10,835
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0
Step 3.5 Flash 10.0 10.0 100.0% 0 11.91s 275 3,802

Quick Compare

Switch Comparison Pair

Gemini 3.1 Flash Lite PreviewnonevsStep 3.5 FlashmediumGPT-5.2 ChatnonevsStep 3.5 FlashmediumGemma 4 31BnoneFree AvailablevsGrok 4.20mediumGemma 4 31BnoneFree AvailablevsKimi K2.5mediumGemma 4 31BnoneFree AvailablevsGPT-5 MinimediumGemma 4 31BnoneFree AvailablevsNemotron 3 SupermediumFree AvailableGemini 3.1 Flash Lite PreviewlowvsStep 3.5 FlashmediumGemma 4 31BnoneFree AvailablevsGrok 4.1 FastmediumGPT-5.3 ChatnonevsStep 3.5 FlashmediumGemini 3 Flash PreviewnonevsStep 3.5 FlashmediumGemma 4 31BnoneFree AvailablevsGPT-5.4 MinimediumGemma 4 31BnoneFree AvailablevsMercury 2medium