Gemma 4 31B vs GPT-5 Nano (medium)

Compared models

Recommended model Gemma 4 31B

Its score stays close to the best score here (5.7 vs 5.7), while costing about 4.7x less than GPT-5 Nano (medium).

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	GPT-5 Nano GPT-5 Nano medium Release: 2025-08-07

Metric	Gemma 4 31B Gemma 4 31B none Release: 2026-04-02 Free Available	GPT-5 Nano GPT-5 Nano medium Release: 2025-08-07
Score	5.7	5.7
Rank	#258	#254
Reliability	10.0	10.0
Consistency	9.6	6.5
Attempts	69/69	69/69
Tests Correct
Attempt pass rate	42.0%	53.6%
Flaky tests	1	10
Total Runs	69	69
Cost per result	0.520	1.737
Total Cost	$0.030	$0.139
Input Price	$0.090 / 1M	$0.050 / 1M
Output Price	$0.340 / 1M	$0.400 / 1M
Total Input Tokens	270,970	314,436
Output Tokens	15,499	13,578
Reasoning Tokens	0	294,343
Response Time (avg)	7.28s	56.55s
Response Time (max)	44.73s	227.89s
Response Time (total)	152.84s	961.32s
Parameters	30.7B	~120B total (~5B active)
Availability	Open source	Closed

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

none

medium

Category:

Agentic	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.9	9.6	0.0%	0		44.73s	145,267	2,182	0
GPT-5 Nano	4.7	3.1	33.3%	1		94.09s	219,492	1,541	23,943

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		1.85s	852	45	0
GPT-5 Nano	6.5	7.9	58.3%	1		25.50s	606	1,221	21,184

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	5.5	10.0	33.3%	0		11.19s	8,381	735	0
GPT-5 Nano	7.0	7.7	55.6%	1		41.62s	7,305	740	41,152

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.8	5.8	33.3%	1		29.95s	104,817	11,910	0
GPT-5 Nano	6.4	5.8	66.7%	1		146.93s	71,846	7,156	86,976

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.25s	8,352	285	0
GPT-5 Nano	3.7	1.7	50.0%	2		21.42s	7,140	453	10,560

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	5.3	10.0	33.3%	0		3.65s	878	27	0
GPT-5 Nano	3.4	1.6	44.4%	3		124.13s	628	232	73,792

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	10.0	10.0	100.0%	0		2.09s	576	117	0
GPT-5 Nano	4.1	10.0	0.0%	0		17.51s	477	202	4,608

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	50.0%	0		2.84s	795	78	0
GPT-5 Nano	9.8	10.0	100.0%	0		15.64s	660	312	4,736

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	6.5	10.0	33.3%	0		4.23s	828	108	0
GPT-5 Nano	5.3	7.2	44.4%	1		20.63s	642	929	14,272

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		0ms	0	0	0
GPT-5 Nano	10.0	10.0	100.0%	0		33.30s	5,445	558	6,976

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Gemma 4 31B	3.0	10.0	0.0%	0		1.25s	224	12	0
GPT-5 Nano	3.0	10.0	0.0%	0		20.13s	195	234	6,144

Switch Comparison Pair