Navigate
AI BENCHY
Advertise here

Granite 4.2 8B (medium) vs Nemotron 3.5 Lightning (medium)

The average score is effectively tied at 5.2 vs 5.1. Granite 4.2 8B (medium) has the lower benchmark cost at $0.009 vs $0.156. Granite 4.2 8B (medium) is faster at 45.97s vs 70.11s, with pass rates of 34.9% vs 48.5%.

Last updated at: 2026-09-02

Rank
#243
Total Output Tokens
19,330
Response Time (avg)
45.97s
Total Cost
$0.009
Rank
#244
Total Output Tokens
760,441
Response Time (avg)
70.11s
Total Cost
$0.156
Recommended model Granite 4.2 8B (medium)

It has the best score here (5.2), while costing about 19.0x less than Nemotron 3.5 Lightning (medium).

Detailed comparison

Metric Granite 4.2 8B Granite 4.2 8B medium Release: 2026-09-02 Nemotron 3.5 Lightning Nemotron 3.5 Lightning medium Release: 2026-08-11 Free Available
Score 5.2 5.1
Rank #243 #244
Reliability 8.7 9.9
Consistency 9.6 6.0
Attempts 66/66 66/66
Tests Correct
Attempt pass rate 34.9% 48.5%
Flaky tests 1 11
Total Runs 66 66
Cost per result 0.117 0.000
Total Cost $0.009 $0.156
Input Price $0.100 / 1M $0.080 / 1M
Output Price $0.150 / 1M $0.200 / 1M
Total Input Tokens 52,909 126,330
Output Tokens 7,058 165,702
Reasoning Tokens 12,272 594,739
Response Time (avg) 45.97s 70.11s
Response Time (max) 557.96s 437.11s
Response Time (total) 1011.42s 1542.52s
Parameters 8B 30B total (3B active)
Availability Open source Weights available

Model generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#243 IBM: Granite 4.2 8B

medium
Provider returned error
Cost
$0.000
Time
0.2s
Tokens
0 tok

#244 Nemotron 3.5 Lightning

medium
Cost
$0.000
Time
130.5s
Tokens
6,750 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.2 8B 5.5 10.0 33.3% 0 12.09s 8,439 2,445 3,900
Nemotron 3.5 Lightning 4.4 5.1 33.3% 2 285.47s 7,623 78,065 297,359

Quick Compare

Switch Comparison Pair

Mistral Small 4nonevsNemotron 3.5 LightningmediumFree AvailableGranite 4.2 8BlowvsNemotron 3.5 LightningmediumFree AvailableNemotron 3.5 LightningmediumFree AvailablevsQwen3 Coder NextnoneGranite 4.2 8BmediumvsInklingnoneDots 3 Note PreviewnoneFree AvailablevsGranite 4.2 8BmediumGranite 4.2 8BmediumvsMistral Small 4noneNemotron 3.5 LightningmediumFree AvailablevsMiMo-V2.5noneNemotron 3.5 LightningmediumFree AvailablevsQwen3.5-9BnoneGranite 4.2 8BmediumvsQwen3 Coder NextnoneNemotron 3.5 LightningmediumFree AvailablevsInklingnoneDots 3 Note PreviewnoneFree AvailablevsNemotron 3.5 LightningmediumFree AvailableNorth Mini CodenoneFree AvailablevsNemotron 3.5 LightningmediumFree Available