AI BENCHY Compare

Google: Gemma 4 31B vs OpenAI: GPT-5.5

Summary

Gemma 4 31B vs GPT-5.5 benchmark comparison: GPT-5.5 leads on average score with 8.8 vs 6.5. Gemma 4 31B has the lower benchmark cost at $0.004 vs $3.679. Gemma 4 31B is faster at 4.05s vs 37.98s, with pass rates of 47.6% vs 87.3%.

Recommended model: GPT-5.5 - It has the strongest score in this comparison (8.8) and the best overall balance of cost and response time across all 2 models.

Last updated at: 2026-06-04

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	GPT-5.5 GPT-5.5 medium Release: 2026-04-24

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	GPT-5.5 GPT-5.5 medium Release: 2026-04-24
Score	6.5	8.8
Rank	#85	#9
Reliability	10.0	10.0
Consistency	10.0	8.9
Tests Correct
Attempt pass rate	47.6%	87.3%
Flaky tests	0	3
Total Runs	63	63
Cost per result	0.034	21.638
Total Cost	$0.004	$3.679
Input Price	$0.120 / 1M	$5.000 / 1M
Output Price	$0.360 / 1M	$30.000 / 1M
Total Input Tokens	20,911	34,212
Output Tokens	1,407	1,985
Reasoning Tokens	0	114,925
Response Time (avg)	4.05s	37.98s
Response Time (max)	26.13s	332.10s
Response Time (total)	76.87s	797.60s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#85 Gemma 4 31B

none

Cost: $0.001
Time: 12.8s
Tokens: 795 tok

#9 GPT-5.5

medium

Cost: $0.112
Time: 71.9s
Tokens: 3,807 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		1.85s	852	45	0
GPT-5.5	10.0	10.0	100.0%	0		4.66s	606	250	1,335

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	5.5	10.0	33.3%	0		11.19s	8,381	735	0
GPT-5.5	8.8	7.8	88.9%	1		59.77s	7,305	362	24,959

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0	0
GPT-5.5	10.0	10.0	100.0%	0		19.29s	11,019	312	2,841

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.25s	8,352	285	0
GPT-5.5	10.0	10.0	100.0%	0		4.18s	7,140	234	593

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	7.7	10.0	66.7%	0		3.22s	903	27	0
GPT-5.5	5.3	7.2	44.4%	1		164.14s	723	67	79,625

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.09s	576	117	0
GPT-5.5	10.0	10.0	100.0%	0		4.16s	477	138	223

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		2.84s	795	78	0
GPT-5.5	10.0	10.0	100.0%	0		3.36s	660	93	538

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	33.3%	0		4.23s	828	108	0
GPT-5.5	10.0	10.0	100.0%	0		6.76s	642	241	2,225

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0	0
GPT-5.5	10.0	10.0	100.0%	0		10.57s	5,445	258	832

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		1.25s	224	12	0
GPT-5.5	2.8	1.6	33.3%	1		37.86s	195	30	1,754

Quick Compare

Switch Comparison Pair