#51

Nemotron 3 Ultra

NVIDIA Release: 2026-06-04 Tested on: 2026-07-16 23:07 nvidia/nemotron-3-ultra-550b-a55b::medium

(medium) (none)

Summary

Nemotron 3 Ultra scores 7.5 on AI BENCHY and ranks #51. It has 9.8 reliability, a 68.2% pass rate, $0.774 total cost, and 32.21s average response time.

What makes Nemotron 3 Ultra unique: It stands out most in Anti-AI Tricks, where it ranks #1, while Combined is its weakest area at #16.

Score

7.5

Consistency

8.5

Reliability

9.8

Total Cost (Current Price)

$0.774 ↕ New

Tested at: $0.000

Total Output Tokens

185,978

Total Input Tokens

233,488

Input Price

$0.600 / 1M

Output Price

$3.600 / 1M

Tests Correct

Wrong Tests: 9

Attempt pass rate: 68.2%

Flaky tests

Flaky tests had mixed outcomes across runs (at least one pass and one fail).

Response Time (avg)

32.21s

Response Time (max): 392.56s

Response Time (total): 708.65s

Wrong answer: 7 API error: 1 Invalid tool call: 1

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#51 Nemotron 3 Ultra

medium

Invalid SVG

Cost: $0.000
Time: 300.0s
Tokens: 0 tok

Run history

Tested on	Score	Reliability	Tests Correct	Total Cost	Compare
2026-07-16 23:07 New test added	7.5	9.8		$0.774