Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Google: Gemma 4 26B A4B vs StepFun: Step 3.7 Flash

Last updated at: 2026-05-29

Metric Gemma 4 26B A4B Gemma 4 26B A4B medium Release: 2026-04-03 Free Available Step 3.7 Flash Step 3.7 Flash low Release: 2026-05-29
Score 7.8 7.4
Rank #34 #60
Reliability 10.0 10.0
Consistency 9.2 8.7
Tests Correct
Attempt pass rate 73.3% 68.3%
Flaky tests 2 3
Total Runs 60 60
Cost per result 0.317 2.796
Total Cost $0.045 $0.336
Input Price $0.060 / 1M $0.200 / 1M
Output Price $0.330 / 1M $1.150 / 1M
Output Tokens 28,000 285,209
Reasoning Tokens 82,045 0
Response Time (avg) 50.92s 16.06s
Response Time (max) 369.32s 124.75s
Response Time (total) 967.47s 321.11s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 6.20s 1,142 3,045
Step 3.7 Flash 8.7 7.9 91.7% 1 4.02s 10,896 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 2.9 10.0 0.0% 0 258.40s 14,838 26,122
Step 3.7 Flash 10.0 10.0 100.0% 0 9.43s 14,569 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 9.6 10.0 100.0% 0 73.55s 5,415 13,112
Step 3.7 Flash 10.0 10.0 100.0% 0 7.98s 6,426 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 16.51s 1,567 2,827
Step 3.7 Flash 7.3 5.8 83.3% 1 2.29s 2,667 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 2.9 4.4 22.2% 2 23.62s 2,469 7,105
Step 3.7 Flash 5.3 7.2 44.4% 1 43.31s 104,487 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 29.76s 25 5,075
Step 3.7 Flash 3.4 9.3 0.0% 0 7.00s 4,604 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 17.54s 887 4,470
Step 3.7 Flash 9.8 10.0 100.0% 0 1.58s 1,857 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 5.79s 410 2,128
Step 3.7 Flash 5.5 9.9 33.3% 0 1.84s 3,564 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 9.01s 450 1,256
Step 3.7 Flash 10.0 10.0 100.0% 0 3.25s 1,360 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 3.0 10.0 0.0% 0 180.87s 797 16,905
Step 3.7 Flash 3.0 10.0 0.0% 0 124.75s 134,779 0

Quick Compare

Switch Comparison Pair

Kimi K2.6mediumFree AvailablevsStep 3.7 FlashlowStep 3.7 FlashlowvsMiMo-V2.5mediumGemma 4 26B A4BmediumFree AvailablevsGPT-5.2 ChatnoneGPT-5.3 ChatnonevsStep 3.7 FlashlowStep 3.7 FlashlowvsGLM 5V TurbomediumStep 3.7 FlashlowvsGLM 5.1mediumGPT-5.2mediumvsStep 3.7 FlashlowGPT-5.4 MinimediumvsStep 3.7 FlashlowGemma 4 26B A4BmediumFree AvailablevsQwen3.7 MaxnoneGemini 3.1 Flash Lite PreviewnonevsStep 3.7 FlashlowClaude Opus 4.8nonevsStep 3.7 FlashlowDeepSeek V3.2mediumvsStep 3.7 Flashlow