Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Google: Gemma 4 31B vs OpenAI: GPT-5.4 Nano

Last updated at: 2026-04-04

Metric Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 GPT-5.4 Nano GPT-5.4 Nano medium Release: 2026-03-17
Score 6.7 7.4
Rank #48 #35
Consistency 10.0 9.0
Tests Correct
Attempt pass rate 52.9% 66.7%
Flaky tests 0 2
Total Runs 51 51
Cost per result 0.023 0.769
Total Cost $0.002 $0.077
Input Price $0.140 / 1M $0.200 / 1M
Output Price $0.400 / 1M $1.250 / 1M
Output Tokens 660 2,474
Reasoning Tokens 0 54,516
Response Time (avg) 2.55s 11.08s
Response Time (max) 4.68s 94.06s
Response Time (total) 38.20s 188.39s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 6.5 10.0 50.0% 0 1.85s 45 0
GPT-5.4 Nano 8.3 10.0 75.0% 0 4.52s 683 2,254
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0
GPT-5.4 Nano 9.8 10.0 100.0% 0 24.13s 349 5,719
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 2.25s 285 0
GPT-5.4 Nano 10.0 10.0 100.0% 0 2.54s 234 516
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 7.7 10.0 66.7% 0 3.22s 27 0
GPT-5.4 Nano 5.9 7.2 55.6% 1 38.18s 60 43,325
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 2.09s 117 0
GPT-5.4 Nano 4.5 10.0 0.0% 0 4.15s 179 443
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 6.5 10.0 50.0% 0 2.84s 78 0
GPT-5.4 Nano 9.8 10.0 100.0% 0 1.88s 95 521
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 5.5 10.0 33.3% 0 2.95s 108 0
GPT-5.4 Nano 4.0 7.1 22.2% 1 3.65s 640 1,356
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0
GPT-5.4 Nano 10.0 10.0 100.0% 0 7.71s 234 382

Quick Compare

Switch Comparison Pair

Gemma 4 31BnonevsGPT-5 MinimediumClaude Sonnet 4.6nonevsGPT-5.4 NanomediumGemma 4 31BnonevsNemotron 3 SupermediumFree AvailableGemma 4 31BnonevsGrok 4.1 FastmediumGemma 4 31BnonevsHunter AlphamediumGemma 4 31BnonevsGPT-5.4 MinimediumGemma 4 31BnonevsGrok 4.20mediumGemini 3.1 Flash Lite PreviewnonevsGPT-5.4 NanomediumGemma 4 31BnonevsMercury 2mediumGemma 4 31BnonevsGPT-5 NanomediumGemma 4 31BnonevsKimi K2.5mediumGemma 4 31BnonevsGrok 4.20 Multi Agent Betamedium