Navigate
AI BENCHY
AD
Track all your projects in one dashboard. Get 📊stats, 🔥heatmaps and 👀recordings in one self-hosted dashboard.
uxwizz.com

AI BENCHY Compare

Google: Gemma 4 31B vs OpenAI: GPT-5.4

Last updated at: 2026-06-03

Metric Gemma 4 31B Gemma 4 31B medium Release: 2026-04-02 Free Available GPT-5.4 GPT-5.4 medium Release: 2026-03-05
Score 8.0 7.9
Rank #22 #27
Reliability 10.0 10.0
Consistency 9.5 8.5
Tests Correct
Attempt pass rate 73.3% 75.0%
Flaky tests 1 4
Total Runs 60 60
Cost per result 0.205 8.765
Total Cost $0.028 $1.140
Input Price $0.120 / 1M $2.500 / 1M
Output Price $0.370 / 1M $15.000 / 1M
Total Input Tokens 15,899 31,489
Output Tokens 19,602 2,221
Reasoning Tokens 49,686 68,486
Response Time (avg) 35.39s 22.31s
Response Time (max) 150.90s 100.41s
Response Time (total) 637.01s 446.17s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 12.89s 816 962 2,046
GPT-5.4 8.3 10.0 75.0% 0 4.11s 606 240 1,511
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 4.1 4.7 33.3% 1 110.94s 3,510 8,344 17,172
GPT-5.4 8.2 6.7 83.3% 1 54.98s 4,686 412 19,995
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0 0
GPT-5.4 10.0 10.0 100.0% 0 20.57s 11,019 301 3,543
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 21.11s 8,334 1,822 2,951
GPT-5.4 10.0 10.0 100.0% 0 5.32s 7,140 234 804
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 7.7 10.0 66.7% 0 38.48s 876 4,349 8,985
GPT-5.4 5.3 7.2 44.4% 1 74.27s 619 61 34,748
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 9.57s 567 105 888
GPT-5.4 4.7 3.1 33.3% 1 4.92s 477 145 321
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 10.0 10.0 100.0% 0 12.76s 777 533 2,035
GPT-5.4 10.0 10.0 100.0% 0 3.11s 660 93 897
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 9.9 10.0 100.0% 0 26.91s 801 1,795 5,595
GPT-5.4 8.2 7.2 88.9% 1 9.14s 642 441 3,815
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 0ms 0 0 0
GPT-5.4 10.0 10.0 100.0% 0 13.28s 5,445 264 1,031
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 31B 3.0 10.0 0.0% 0 90.14s 218 1,692 10,014
GPT-5.4 3.0 10.0 0.0% 0 13.95s 195 30 1,821

Quick Compare

Switch Comparison Pair

GPT-5.4mediumvsQwen3.7 MaxnoneGemini 3.5 FlashminimalvsGPT-5.4mediumGemma 4 31BmediumFree AvailablevsQwen3.7 MaxnoneGemini 3.5 FlashnonevsGPT-5.4mediumGemma 4 31BmediumFree AvailablevsGPT-5.2 ChatnoneGemini 3 Flash PreviewnonevsGPT-5.4mediumDeepSeek V4 FlashhighvsGPT-5.4mediumGemini 3.1 Flash Lite PreviewlowvsGPT-5.4mediumDeepSeek V4 FlashhighvsGemma 4 31BmediumFree AvailableGemini 3.1 Flash Lite PreviewnonevsGPT-5.4mediumGemini 3.1 Flash LitelowvsGPT-5.4mediumGPT-5.4mediumvsStep 3.7 Flashlow