Navigate
AI BENCHY
Advertise here

Granite 4.2 8B (low) vs Nemotron 3.5 Lightning (medium)

The average score is effectively tied at 5.1 vs 5.1. Granite 4.2 8B (low) has the lower benchmark cost at $0.012 vs $0.156. Granite 4.2 8B (low) is faster at 54.36s vs 70.11s, with pass rates of 31.8% vs 48.5%.

Last updated at: 2026-09-02

Rank
#247
Total Output Tokens
27,928
Response Time (avg)
54.36s
Total Cost
$0.012
Rank
#244
Total Output Tokens
760,441
Response Time (avg)
70.11s
Total Cost
$0.156
Recommended model Granite 4.2 8B (low)

It has the best score here (5.1), while costing about 13.4x less than Nemotron 3.5 Lightning (medium).

Detailed comparison

Metric Granite 4.2 8B Granite 4.2 8B low Release: 2026-09-02 Nemotron 3.5 Lightning Nemotron 3.5 Lightning medium Release: 2026-08-11 Free Available
Score 5.1 5.1
Rank #247 #244
Reliability 9.0 9.9
Consistency 10.0 6.0
Attempts 66/66 66/66
Tests Correct
Attempt pass rate 31.8% 48.5%
Flaky tests 0 11
Total Runs 66 66
Cost per result 0.167 0.000
Total Cost $0.012 $0.156
Input Price $0.100 / 1M $0.080 / 1M
Output Price $0.150 / 1M $0.200 / 1M
Total Input Tokens 74,721 126,330
Output Tokens 9,205 165,702
Reasoning Tokens 18,723 594,739
Response Time (avg) 54.36s 70.11s
Response Time (max) 566.80s 437.11s
Response Time (total) 1195.97s 1542.52s
Parameters 8B 30B total (3B active)
Availability Open source Weights available

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#247 IBM: Granite 4.2 8B

low
Provider returned error
Cost
$0.000
Time
0.2s
Tokens
0 tok

#244 Nemotron 3.5 Lightning

medium
Cost
$0.000
Time
130.5s
Tokens
6,750 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.2 8B 5.5 10.0 33.3% 0 13.01s 8,439 2,469 2,984
Nemotron 3.5 Lightning 4.4 5.1 33.3% 2 285.47s 7,623 78,065 297,359

Quick Compare

Switch Comparison Pair

Mistral Small 4nonevsNemotron 3.5 LightningmediumFree AvailableGranite 4.2 8BlowvsMistral Small 4mediumGranite 4.2 8BlowvsMistral Small 4noneGranite 4.2 8BlowvsQwen3 Coder NextnoneNemotron 3.5 LightningmediumFree AvailablevsQwen3 Coder NextnoneGranite 4.2 8BlowvsMiMo-V2.5noneNemotron 3.5 LightningmediumFree AvailablevsMiMo-V2.5noneGranite 4.2 8BlowvsQwen3.5-9BnoneNemotron 3.5 LightningmediumFree AvailablevsQwen3.5-9BnoneNemotron 3.5 LightningmediumFree AvailablevsInklingnoneDots 3 Note PreviewnoneFree AvailablevsNemotron 3.5 LightningmediumFree AvailableNorth Mini CodenoneFree AvailablevsGranite 4.2 8Blow