Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Google: Gemma 4 31B vs Grok 4.20 Multi Agent Beta

Last updated at: 2026-04-02

Metric Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Grok 4.20 Multi Agent Beta Grok 4.20 Multi Agent Beta medium Release: 2026-03-12
Score 6.7 6.2
Rank #47 #53
Consistency 10.0 7.2
Tests Correct
Attempt pass rate 52.9% 54.9%
Flaky tests 0 6
Total Runs 51 51
Cost per result 0.023 82.962
Total Cost $0.002 $4.978
Input Price $0.140 / 1M $0.000 / 1M
Output Price $0.400 / 1M $0.000 / 1M
Output Tokens 660 298,948
Reasoning Tokens 0 296,529
Response Time (avg) 2.55s 8.64s
Response Time (max) 4.68s 35.28s
Response Time (total) 38.20s 129.64s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 6.5 10.0 50.0% 0 1.85s 45 0
Grok 4.20 Multi Agent Beta 6.9 5.8 75.0% 2 3.46s 33,706 33,077
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0
Grok 4.20 Multi Agent Beta 3.0 10.0 0.0% 0 0ms 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 2.25s 285 0
Grok 4.20 Multi Agent Beta 10.0 10.0 100.0% 0 5.54s 25,306 25,051
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 7.7 10.0 66.7% 0 3.22s 27 0
Grok 4.20 Multi Agent Beta 2.9 7.2 11.1% 1 24.67s 164,609 163,647
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 2.09s 117 0
Grok 4.20 Multi Agent Beta 5.8 2.8 66.7% 1 6.40s 15,848 15,746
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 6.5 10.0 50.0% 0 2.84s 78 0
Grok 4.20 Multi Agent Beta 8.3 10.0 50.0% 0 4.63s 25,457 25,322
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 5.5 10.0 33.3% 0 2.95s 108 0
Grok 4.20 Multi Agent Beta 7.2 5.1 77.8% 2 5.01s 34,022 33,686
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0
Grok 4.20 Multi Agent Beta 3.0 10.0 0.0% 0 0ms 0 0

Quick Compare

Switch Comparison Pair

Gemma 4 31BnonevsGPT-5 MinimediumDeepSeek V3.2nonevsGrok 4.20 Multi Agent BetamediumQwen3.5-FlashnonevsGrok 4.20 Multi Agent BetamediumGemma 4 31BnonevsNemotron 3 SupermediumFree AvailableGrok 4.20 Multi Agent BetamediumvsMiMo-V2-OmninoneGemma 4 31BnonevsGrok 4.1 FastmediumGrok 4.20 Multi Agent BetamediumvsGLM 5V TurbononeGemma 4 31BnonevsHunter AlphamediumSeed-2.0-LitenonevsGrok 4.20 Multi Agent BetamediumGemini 2.5 FlashnonevsGrok 4.20 Multi Agent BetamediumQwen3.5-35B-A3BnonevsGrok 4.20 Multi Agent BetamediumHunter AlphanonevsGrok 4.20 Multi Agent Betamedium