Navigate
AI BENCHY
AD
Track all your projects in one dashboard. Get 📊stats, 🔥heatmaps and 👀recordings in one self-hosted dashboard.
uxwizz.com

AI BENCHY Compare

Nemotron 3 Ultra 550b A55b vs Grok 4.1 Fast

Summary

Nemotron 3 Ultra 550b A55b vs Grok 4.1 Fast benchmark comparison: Nemotron 3 Ultra 550b A55b leads on average score with 8.1 vs 5.6. Grok 4.1 Fast has the lower benchmark cost at $0.069 vs $0.158. Nemotron 3 Ultra 550b A55b is faster at 15.05s vs 23.85s, with pass rates of 69.8% vs 55.6%.

Recommended model: Nemotron 3 Ultra 550b A55b - It has the best score here (8.1), while responding about 1.6x faster than Grok 4.1 Fast.

Last updated at: 2026-07-02

Metric Nemotron 3 Ultra 550b A55b Nemotron 3 Ultra 550b A55b medium Release: 2026-06-04 Free Available Grok 4.1 Fast Grok 4.1 Fast medium Release: 2025-11-19
Score 8.1 5.6
Rank #26 #119
Reliability 9.7 10.0
Consistency 8.8 6.6
Tests Correct
Attempt pass rate 69.8% 55.6%
Flaky tests 3 6
Total Runs 63 57
Cost per result 0.000 0.642
Total Cost $0.158 $0.069
Input Price $0.500 / 1M $0.484 / 1M
Output Price $2.200 / 1M $0.484 / 1M
Total Input Tokens 46,813 42,845
Output Tokens 18,002 2,006
Reasoning Tokens 53,091 96,334
Response Time (avg) 15.05s 23.85s
Response Time (max) 43.93s 121.79s
Response Time (total) 316.09s 286.16s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#26 Nemotron 3 Ultra 550b A55b

medium
Invalid SVG
Cost
$0.000
Time
300.0s
Tokens
0 tok

#119 Grok 4.1 Fast

medium
Grok 4.1 Fast is deprecated. xAI recommends switching to Grok 4.3 (https://openrouter.ai/x-ai/grok-4.3)
Cost
$0.000
Time
0.1s
Tokens
0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 8.62s 780 835 1,485
Grok 4.1 Fast 8.7 7.9 91.7% 1 3.81s 2,358 108 4,741
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 8.4 7.4 88.9% 1 26.53s 7,686 2,854 17,725
Grok 4.1 Fast 7.8 4.0 11.1% 1 23.58s 1,167 821 6,703
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 9.8 10.0 100.0% 0 43.93s 17,574 1,040 3,590
Grok 4.1 Fast 10.0 10.0 100.0% 0 37.64s 13,899 261 12,272
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 5.68s 7,989 473 1,285
Grok 4.1 Fast 10.0 10.0 100.0% 0 6.63s 8,001 180 5,409
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 3.5 4.4 33.3% 2 24.90s 858 11,169 16,249
Grok 4.1 Fast 5.8 4.4 66.7% 2 121.79s 1,777 11 37,657
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 3.7 9.5 0.0% 0 2.52s 360 70 235
Grok 4.1 Fast 4.2 9.9 0.0% 0 16.25s 912 127 3,456
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 9.8 10.0 100.0% 0 6.35s 765 182 1,243
Grok 4.1 Fast 6.5 10.0 50.0% 0 4.63s 1,536 54 3,326
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 5.5 9.9 33.3% 0 3.54s 792 771 2,055
Grok 4.1 Fast 5.3 7.2 44.4% 1 7.40s 1,950 169 5,904
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 10.0 10.0 100.0% 0 7.72s 9,781 304 984
Grok 4.1 Fast 2.8 1.6 33.3% 1 27.71s 10,627 260 11,485
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Nemotron 3 Ultra 550b A55b 3.0 10.0 0.0% 0 38.47s 228 304 8,240
Grok 4.1 Fast 3.0 10.0 0.0% 0 25.52s 618 15 5,381

Quick Compare

Switch Comparison Pair

DeepSeek V4 FlashhighvsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsGPT-5.2 ChatnoneClaude Opus 4.8lowvsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsStep 3.7 FlashlowDeepSeek V4 ProhighvsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsGPT-5.3 ChatnoneGemini 3 Flash PreviewlowvsNemotron 3 Ultra 550b A55bmediumFree AvailableClaude Sonnet 4.6nonevsNemotron 3 Ultra 550b A55bmediumFree AvailableClaude Opus 4.8nonevsNemotron 3 Ultra 550b A55bmediumFree AvailableDeepSeek V4 PrononevsNemotron 3 Ultra 550b A55bmediumFree AvailableNemotron 3 Ultra 550b A55bmediumFree AvailablevsQwen3.7 PlusnoneNemotron 3 Ultra 550b A55bmediumFree AvailablevsGLM 5.2none