Navigate
AI BENCHY
AD
Track all your projects in one dashboard. Get 📊stats, 🔥heatmaps and 👀recordings in one self-hosted dashboard.
uxwizz.com

AI BENCHY Compare

Nemotron 3 Ultra 550b A55b vs Grok 4.20 Beta

Summary

Nemotron 3 Ultra 550b A55b vs Grok 4.20 Beta benchmark comparison: Nemotron 3 Ultra 550b A55b leads on average score with 7.5 vs 5.8. Grok 4.20 Beta has the lower benchmark cost at $0.087 vs $0.177. Grok 4.20 Beta is faster at 1.19s vs 15.05s, with pass rates of 69.8% vs 37.0%.

Recommended model: Grok 4.20 Beta - It offers the best overall trade-off: a competitive score (5.8), lower cost than Nemotron 3 Ultra 550b A55b, and balanced response time.

Last updated at: 2026-06-10

Metric Nemotron 3 Ultra 550b A55b Nemotron 3 Ultra 550b A55b medium Release: 2026-06-04 Free Available Grok 4.20 Beta Grok 4.20 Beta none Release: 2026-03-12
Score 7.5 5.8
Rank #42 #107
Reliability 9.7 N/A
Consistency 8.8 9.6
Tests Correct
Attempt pass rate 69.8% 37.0%
Flaky tests 3 1
Total Runs 63 52
Cost per result 0.000 1.510
Total Cost $0.177 $0.087
Input Price $0.500 / 1M $2.041 / 1M
Output Price $2.500 / 1M $2.041 / 1M
Total Input Tokens 46,813 40,597
Output Tokens 18,002 1,657
Reasoning Tokens 53,091 0
Response Time (avg) 15.05s 1.19s
Response Time (max) 43.93s 6.48s
Response Time (total) 316.09s 21.43s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#42 Nemotron 3 Ultra 550b A55b

medium
No showcase result has been generated for this model yet.
Cost
$0.000
Time
-
Tokens
0 tok

#107 Grok 4.20 Beta

none
Invalid SVG
Cost
$0.004
Time
8.5s
Tokens
1,672 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 8.62s 780 835 1,485
Grok 4.20 Beta 4.0 8.4 16.7% 1 597ms 1,986 251 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 8.4 7.4 88.9% 1 26.53s 7,686 2,854 17,725
Grok 4.20 Beta 5.5 10.0 0.0% 0 1.14s 358 74 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 9.8 10.0 100.0% 0 43.93s 17,574 1,040 3,590
Grok 4.20 Beta 3.0 10.0 0.0% 0 6.48s 17,673 282 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 5.68s 7,989 473 1,285
Grok 4.20 Beta 10.0 10.0 100.0% 0 601ms 7,749 197 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 3.5 4.4 33.3% 2 24.90s 858 11,169 16,249
Grok 4.20 Beta 3.0 10.0 0.0% 0 611ms 1,746 160 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 3.7 9.5 0.0% 0 2.52s 360 70 235
Grok 4.20 Beta 5.0 10.0 0.0% 0 541ms 819 87 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 9.8 10.0 100.0% 0 6.35s 765 182 1,243
Grok 4.20 Beta 6.3 10.0 50.0% 0 649ms 1,350 60 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 5.5 9.9 33.3% 0 3.54s 792 771 2,055
Grok 4.20 Beta 7.7 10.0 66.7% 0 586ms 1,671 357 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 7.72s 9,781 304 984
Grok 4.20 Beta 10.0 10.0 100.0% 0 4.79s 7,245 189 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 3.0 10.0 0.0% 0 38.47s 228 304 8,240
Grok 4.20 Beta - - - - - - - - -

Quick Compare

Switch Comparison Pair

Gemini 3 Flash PreviewnonevsNemotron 3 Ultra 550b A55bmediumFree AvailableGemini 3.1 Flash Lite PreviewlowvsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsQwen3.7 MaxnoneGemini 3.5 FlashminimalvsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsStep 3.7 FlashlowDeepSeek V4 FlashhighvsNemotron 3 Ultra 550b A55bmediumFree AvailableGemini 3.1 Flash Lite PreviewnonevsNemotron 3 Ultra 550b A55bmediumFree AvailableGemini 3.1 Flash LitelowvsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsGPT-5.3 ChatnoneNemotron 3 Ultra 550b A55bmediumFree AvailablevsGPT-5.2 ChatnoneClaude Opus 4.8nonevsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsStep 3.7 Flashhigh