AI BENCHY Compare

IBM: Granite 4.1 8B vs MiniMax: MiniMax M2.5

Summary

Granite 4.1 8B vs MiniMax M2.5 benchmark comparison: MiniMax M2.5 leads on average score with 4.7 vs 4.0. Granite 4.1 8B has the lower benchmark cost at $0.003 vs $0.303. Granite 4.1 8B is faster at 728ms vs 65.37s, with pass rates of 9.5% vs 46.0%.

Recommended model: Granite 4.1 8B - Its score stays close to the best score here (4.0 vs 4.7), while costing about 116.1x less than MiniMax M2.5.

Last updated at: 2026-06-12

Metric	Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01	MiniMax M2.5 MiniMax M2.5 medium Release: 2026-02-12

Metric	Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01	MiniMax M2.5 MiniMax M2.5 medium Release: 2026-02-12
Score	4.0	4.7
Rank	#163	#151
Reliability	10.0	10.0
Consistency	10.0	6.5
Tests Correct
Attempt pass rate	9.5%	46.0%
Flaky tests	0	9
Total Runs	63	63
Cost per result	0.131	7.900
Total Cost	$0.003	$0.303
Input Price	$0.050 / 1M	$0.150 / 1M
Output Price	$0.100 / 1M	$0.900 / 1M
Total Input Tokens	46,285	43,706
Output Tokens	2,911	109,495
Reasoning Tokens	0	330,814
Response Time (avg)	728ms	65.37s
Response Time (max)	2.17s	251.36s
Response Time (total)	15.29s	849.76s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#163 IBM: Granite 4.1 8B

none

Cost: $0.001
Time: 3.2s
Tokens: 491 tok

#151 MiniMax M2.5

medium

Invalid SVG

Cost: $0.000
Time: 300.0s
Tokens: 0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	4.9	10.0	25.0%	0		844ms	645	903	0
MiniMax M2.5	7.9	6.3	83.3%	2		20.82s	612	286	45,344

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	4.5	10.0	0.0%	0		775ms	8,344	525	0
MiniMax M2.5	3.4	9.1	0.0%	0		188.58s	6,076	357	106,177

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	3.0	10.0	0.0%	0		1.88s	19,089	396	0
MiniMax M2.5	4.5	2.1	66.7%	1		60.39s	21,104	740	9,713

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	3.0	10.0	0.0%	0		575ms	7,617	195	0
MiniMax M2.5	4.6	1.7	66.7%	2		7.48s	6,584	266	3,835

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	3.0	10.0	0.0%	0		357ms	768	24	0
MiniMax M2.5	2.9	4.4	22.2%	2		237.27s	308	105,047	133,487

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	4.0	10.0	0.0%	0		499ms	528	115	0
MiniMax M2.5	3.8	2.5	33.3%	1		6.63s	492	25	1,686

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	3.6	9.9	0.0%	0		344ms	687	66	0
MiniMax M2.5	7.5	10.0	50.0%	0		621ms	699	156	1,495

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	3.2	10.0	0.0%	0		608ms	672	432	0
MiniMax M2.5	5.3	7.2	44.4%	1		11.21s	495	1,069	9,605

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	10.0	10.0	100.0%	0		2.17s	7,719	243	0
MiniMax M2.5	10.0	10.0	100.0%	0		15.35s	7,123	269	937

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.1 8B	3.0	10.0	0.0%	0		306ms	216	12	0
MiniMax M2.5	3.0	10.0	0.0%	0		80.79s	213	1,280	18,535

Quick Compare

Switch Comparison Pair

Ling-2.6-1TnonevsMiniMax M2.5medium MiniMax M2.5mediumvsgpt-oss-120bnoneFree Available MiniMax M2.5mediumvsGPT-5.4 Nanonone Mercury 2nonevsMiniMax M2.5medium Ring-2.6-1TnonevsMiniMax M2.5medium Ling-2.6-flashnonevsMiniMax M2.5medium MiniMax M2.5mediumvsNemotron 3 SupernoneFree Available MiniMax M2.5mediumvsGLM 4.7 Flashnone Granite 4.1 8BnonevsQwen3.5-9Bmedium MiniMax M2.5mediumvsGPT-4o-mininone Trinity Large PreviewnonevsMiniMax M2.5medium Granite 4.1 8BnonevsGLM 4.7 Flashmedium