NVIDIA
Release: 2026-08-11
Tested on: 2026-08-11 18:28
nvidia/nemotron-3.5-lightning::medium
Summary
Nemotron 3.5 Lightning scores 5.2 on AI BENCHY and ranks #208. It has 9.9 reliability, a 47.0% pass rate, $0.193 total cost, and 69.64s average response time.
What makes Nemotron 3.5 Lightning unique: It stands out most in Instructions following, where it ranks #1, while Data parsing and extraction is its weakest area at #15. It uses unusually many reasoning tokens, which can help explain its slower or more expensive runs.
5.2
Consistency
6.4
9.9
Total Output Tokens
741,084
Total Input Tokens
126,321
Input Price
$0.100 / 1M
Output Price
$0.250 / 1M
Flaky tests
10
Flaky tests had mixed outcomes across runs (at least one pass and one fail).
Price History
Historical pricing data for this model from OpenRouter.
| Date | Input Price | Output Price |
|---|---|---|
| 2026-08-11 17:51 | $0.000 / 1M | $0.000 / 1M |
| 2026-08-11 18:26 | $0.100 / 1M | $0.250 / 1M |
Charts
Choose the first model, then click a second model to open a side-by-side page.
Score vs Total Cost
Response Time (avg)
Score vs Response Time (avg)
Total Output Tokens
Score vs Total Output Tokens
Quick Compare
Nemotron 3.5 LightningmediumFree AvailablevsInklingnoneNemotron 3.5 LightningmediumFree AvailablevsMistral Small 4noneNemotron 3.5 LightningmediumFree AvailablevsLing-2.6-1TnoneNemotron 3.5 LightningmediumFree AvailablevsQwen3 Coder NextnoneNemotron 3.5 LightningmediumFree AvailablevsQwen3.6 35B A3BnoneNemotron 3.5 LightningmediumFree AvailablevsMistral Small 4mediumNemotron 3.5 LightningmediumFree AvailablevsLaguna XS 2.1noneFree AvailableNemotron 3.5 LightningmediumFree AvailablevsMiMo-V2.5noneNemotron 3.5 LightningmediumFree AvailablevsInkling SmallnoneNemotron 3.5 LightningmediumFree AvailablevsQwen3.5-9Bnone
Category Breakdown
| Category | Score | Consistency | Tests Correct |
|---|---|---|---|
| Anti-AI Tricks | 6.9 | 7.9 | |
| Coding | 4.4 | 5.1 | |
| Combined | 6.2 | 5.8 | |
| Data parsing and extraction | 3.7 | 1.7 | |
| Domain specific | 2.9 | 7.2 | |
| General Intelligence | 5.5 | 10.0 | |
| Instructions following | 8.5 | 6.8 | |
| Puzzle Solving | 4.2 | 4.4 | |
| Tool Calling | 10.0 | 10.0 | |
| Trivia | 3.0 | 10.0 |