AI BENCHY Compare

Nemotron 3 Ultra 550b A55b vs Elephant Alpha

Summary

Nemotron 3 Ultra 550b A55b (medium) leads on average score with 8.1 vs 4.5. Elephant Alpha (medium) has the lower benchmark cost at $0.000 vs $0.158. Elephant Alpha (medium) is faster at 1.27s vs 15.05s, with pass rates of 69.8% vs 30.2%.

Recommended modelNemotron 3 Ultra 550b A55b (medium)It has the strongest score in this comparison (8.1) and the best overall balance of cost and response time across all 2 models.

Last updated at: 2026-07-14

Metric	Nemotron 3 Ultra 550b A55b Nemotron 3 Ultra 550b A55b medium Release: 2026-06-04 Free Available	Elephant Alpha Elephant Alpha medium Release: 2026-04-14

Metric	Nemotron 3 Ultra 550b A55b Nemotron 3 Ultra 550b A55b medium Release: 2026-06-04 Free Available	Elephant Alpha Elephant Alpha medium Release: 2026-04-14
Score	8.1	4.5
Rank	#31	#181
Reliability	9.7	N/A
Consistency	8.8	9.7
Tests Correct
Attempt pass rate	69.8%	30.2%
Flaky tests	3	1
Total Runs	63	63
Cost per result	0.000	0.000
Total Cost	$0.158	$0.000
Input Price	$0.500 / 1M	$0.000 / 1M
Output Price	$2.200 / 1M	$0.000 / 1M
Total Input Tokens	46,813	33,744
Output Tokens	18,002	2,596
Reasoning Tokens	53,091	0
Response Time (avg)	15.05s	1.27s
Response Time (max)	43.93s	3.70s
Response Time (total)	316.09s	22.82s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#31 Nemotron 3 Ultra 550b A55b

medium

Invalid SVG

Cost: $0.000
Time: 300.0s
Tokens: 0 tok

#181 Elephant Alpha

medium

Elephant Alpha was a stealth model revealed on April 21st as Ling-2.6-flash. Find it here: https://openrouter.ai/inclusionai/ling-2.6-flash:free

Cost: $0.000
Time: 0.1s
Tokens: 0 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	10.0	10.0	100.0%	0		8.62s	780	835	1,485
Elephant Alpha	6.6	10.0	50.0%	0		1.19s	726	815	0

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	8.4	7.4	88.9%	1		26.53s	7,686	2,854	17,725
Elephant Alpha	3.7	7.8	11.1%	1		1.30s	813	365	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	9.8	10.0	100.0%	0		43.93s	17,574	1,040	3,590
Elephant Alpha	3.0	10.0	0.0%	0		3.70s	14,046	562	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	10.0	10.0	100.0%	0		5.68s	7,989	473	1,285
Elephant Alpha	6.5	10.0	50.0%	0		979ms	8,004	246	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	3.5	4.4	33.3%	2		24.90s	858	11,169	16,249
Elephant Alpha	3.0	10.0	0.0%	0		925ms	810	24	0

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	3.7	9.5	0.0%	0		2.52s	360	70	235
Elephant Alpha	4.3	10.0	0.0%	0		920ms	540	105	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	9.8	10.0	100.0%	0		6.35s	765	182	1,243
Elephant Alpha	9.8	10.0	100.0%	0		987ms	732	82	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	5.5	9.9	33.3%	0		3.54s	792	771	2,055
Elephant Alpha	5.3	10.0	33.3%	0		868ms	729	166	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	10.0	10.0	100.0%	0		7.72s	9,781	304	984
Elephant Alpha	3.0	10.0	0.0%	0		2.83s	7,344	231	0

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Nemotron 3 Ultra 550b A55b	3.0	10.0	0.0%	0		38.47s	228	304	8,240
Elephant Alpha	3.0	10.0	0.0%	0		0ms	0	0	0

Quick Compare

Switch Comparison Pair