Granite 4.2 8B (low) vs Step 3.5 Flash (medium)

Compared models

Recommended model Granite 4.2 8B (low)

It has the best score here (5.3), while costing about 3.0x less than Step 3.5 Flash (medium).

Metric	Granite 4.2 8B Granite 4.2 8B low Release: 2026-09-02	Step 3.5 Flash Step 3.5 Flash medium Release: 2026-02-01

Metric	Granite 4.2 8B Granite 4.2 8B low Release: 2026-09-02	Step 3.5 Flash Step 3.5 Flash medium Release: 2026-02-01
Score	5.3	5.4
Rank	#280	#276
Reliability	9.2	10.0
Consistency	9.7	8.3
Attempts	69/69	66/69
Tests Correct
Attempt pass rate	33.3%	49.3%
Flaky tests	1	3
Total Runs	69	66
Cost per result	0.508	1.047
Total Cost	$0.036	$0.105
Input Price	$0.060 / 1M	$0.100 / 1M
Output Price	$0.250 / 1M	$0.300 / 1M
Total Input Tokens	332,917	139,086
Output Tokens	18,989	193,584
Reasoning Tokens	42,362	413,546
Response Time (avg)	58.20s	190.48s
Response Time (max)	566.80s	1597.85s
Response Time (total)	1338.65s	3238.23s
Parameters	8B	196B total (11B active)
Availability	Open source	Open source

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

low

Provider returned error

medium

Category:

Agentic	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	6.1	3.1	66.7%	1		142.68s	258,196	9,784	23,639
Step 3.5 Flash	2.8	1.6	33.3%	1		172.16s	73,370	4,160	34,434

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	8.3	10.0	75.0%	0		11.68s	777	1,463	898
Step 3.5 Flash	10.0	10.0	100.0%	0		40.57s	694	20,391	24,176

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	5.5	10.0	33.3%	0		13.01s	8,439	2,469	2,984
Step 3.5 Flash	2.4	5.2	0.0%	0		258.38s	2,211	13,207	22,429

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	3.0	10.0	0.0%	0		298.21s	46,805	4,085	13,637
Step 3.5 Flash	6.5	10.0	50.0%	0		813.71s	44,914	18,150	111,004

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	6.5	10.0	50.0%	0		1.39s	7,683	270	75
Step 3.5 Flash	10.0	10.0	100.0%	0		15.01s	7,368	600	13,886

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	5.3	10.0	33.3%	0		26.16s	876	154	290
Step 3.5 Flash	3.5	4.4	33.3%	2		311.60s	682	126,213	175,555

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	4.0	10.0	0.0%	0		659ms	561	108	24
Step 3.5 Flash	5.5	10.0	0.0%	0		22.39s	509	240	3,506

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	3.0	10.0	0.0%	0		75.80s	402	84	378
Step 3.5 Flash	8.3	10.0	50.0%	0		4.78s	705	2,364	3,521

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	3.0	10.0	0.0%	0		90.84s	517	181	82
Step 3.5 Flash	5.3	10.0	33.3%	0		7.22s	711	5,630	10,861

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	10.0	10.0	100.0%	0		2.23s	8,412	358	265
Step 3.5 Flash	10.0	10.0	100.0%	0		11.91s	7,701	275	3,802

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Granite 4.2 8B	3.0	10.0	0.0%	0		5.53s	249	33	90
Step 3.5 Flash	3.0	10.0	0.0%	0		108.45s	221	2,354	10,372

Switch Comparison Pair