Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Google: Gemma 4 26B A4B vs Nemotron 3 Ultra 550b A55b

Summary

Gemma 4 26B A4B vs Nemotron 3 Ultra 550b A55b benchmark comparison: Nemotron 3 Ultra 550b A55b leads on average score with 8.1 vs 7.2. Gemma 4 26B A4B has the lower benchmark cost at $0.045 vs $0.158. Nemotron 3 Ultra 550b A55b is faster at 15.05s vs 63.41s, with pass rates of 69.8% vs 69.8%.

Recommended model: Nemotron 3 Ultra 550b A55b - It has the best score here (8.1), while responding about 4.2x faster than Gemma 4 26B A4B.

Last updated at: 2026-07-02

Metric Gemma 4 26B A4B Gemma 4 26B A4B medium Release: 2026-04-03 Free Available Nemotron 3 Ultra 550b A55b Nemotron 3 Ultra 550b A55b medium Release: 2026-06-04 Free Available
Score 7.2 8.1
Rank #61 #26
Reliability 10.0 9.7
Consistency 9.2 8.8
Tests Correct
Attempt pass rate 69.8% 69.8%
Flaky tests 2 3
Total Runs 63 63
Cost per result 0.361 0.000
Total Cost $0.045 $0.158
Input Price $0.060 / 1M $0.500 / 1M
Output Price $0.330 / 1M $2.200 / 1M
Total Input Tokens 40,252 46,813
Output Tokens 28,000 18,002
Reasoning Tokens 100,490 53,091
Response Time (avg) 63.41s 15.05s
Response Time (max) 369.32s 43.93s
Response Time (total) 1268.28s 316.09s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#61 Gemma 4 26B A4B

medium
Invalid SVG
Cost
$0.000
Time
300.0s
Tokens
0 tok

#26 Nemotron 3 Ultra 550b A55b

medium
Invalid SVG
Cost
$0.000
Time
300.0s
Tokens
0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 6.20s 816 1,142 3,045
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 8.62s 780 835 1,485
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 2.9 10.0 0.0% 0 272.54s 5,062 14,838 44,567
Nemotron 3 Ultra 550b A55b 8.4 7.4 88.9% 1 26.53s 7,686 2,854 17,725
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 9.6 10.0 100.0% 0 73.55s 17,092 5,415 13,112
Nemotron 3 Ultra 550b A55b 9.8 10.0 100.0% 0 43.93s 17,574 1,040 3,590
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 16.51s 8,334 1,567 2,827
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 5.68s 7,989 473 1,285
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 2.9 4.4 22.2% 2 23.62s 516 2,469 7,105
Nemotron 3 Ultra 550b A55b 3.5 4.4 33.3% 2 24.90s 858 11,169 16,249
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 29.76s 567 25 5,075
Nemotron 3 Ultra 550b A55b 3.7 9.5 0.0% 0 2.52s 360 70 235
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 17.54s 777 887 4,470
Nemotron 3 Ultra 550b A55b 9.8 10.0 100.0% 0 6.35s 765 182 1,243
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 5.79s 801 410 2,128
Nemotron 3 Ultra 550b A55b 5.5 9.9 33.3% 0 3.54s 792 771 2,055
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 9.01s 6,096 450 1,256
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 7.72s 9,781 304 984
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Gemma 4 26B A4B 3.0 10.0 0.0% 0 180.87s 191 797 16,905
Nemotron 3 Ultra 550b A55b 3.0 10.0 0.0% 0 38.47s 228 304 8,240

Quick Compare

Switch Comparison Pair

Gemma 4 26B A4BmediumFree AvailablevsQwen3.7 PlusnoneDeepSeek V4 PrononevsGemma 4 26B A4BmediumFree AvailableGemma 4 26B A4BmediumFree AvailablevsGLM 5.2noneClaude Opus 4.8nonevsGemma 4 26B A4BmediumFree AvailableGemma 4 26B A4BmediumFree AvailablevsStep 3.7 FlashhighDeepSeek V4 FlashhighvsNemotron 3 Ultra 550b A55bmediumFree AvailableClaude Sonnet 4.6nonevsGemma 4 26B A4BmediumFree AvailableGemma 4 26B A4BmediumFree AvailablevsQwen3.7 MaxnoneGemma 4 26B A4BmediumFree AvailablevsGPT-5.3 ChatnoneNemotron 3 Ultra 550b A55bmediumFree AvailablevsGPT-5.2 ChatnoneClaude Opus 4.8lowvsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsStep 3.7 Flashlow