AI BENCHY
Advertise here
#20

DeepSeek V4 Pro

DeepSeek Release: 2026-04-24 Tested on: 2026-04-25 21:53 deepseek/deepseek-v4-pro::high
(high) (none)

Summary

DeepSeek V4 Pro scores 8.2 on AI BENCHY and ranks #20. It has N/A reliability, a 77.8% pass rate, $0.329 total cost, and 72.92s average response time.

What makes DeepSeek V4 Pro unique: Its total benchmark cost is unusually low for its score range.

Score

8.2

Consistency

9.2

Reliability

N/A

Total Cost (Current Price)

$0.329

Total Output Tokens

84,562

Total Input Tokens

0

Input Price

$1.740 / 1M

Output Price

$3.480 / 1M

Tests Correct

Wrong Tests: 5

Attempt pass rate: 77.8%

Flaky tests

2

Flaky tests had mixed outcomes across runs (at least one pass and one fail).

Response Time (avg)

72.92s

Response Time (max): 414.38s

Response Time (total): 1312.59s

Wrong answer: 3 Did not follow instructions: 2

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#20 DeepSeek V4 Pro

high
Cost
$0.023
Time
257.6s
Tokens
14,870 tok

Run history

Tested on Score Reliability Tests Correct Total Cost Compare
2026-07-16 23:19 New test added 7.7 10.0 $0.200 Compare
2026-06-16 15:31 Re-test 7.6 9.3 $0.157 Compare
2026-06-16 14:47 Suite changed 8.1 9.6 $0.098 Compare
2026-06-04 14:38 New test added 6.0 8.9 $0.079 ↓ Compare
2026-05-22 00:54 Suite changed 6.6 9.0 $0.212 Compare
2026-04-29 14:47 Re-test 7.5 9.3 $0.209 Compare
2026-04-26 10:50 Re-test 7.5 8.4 $0.201 Compare
2026-04-25 21:53 Initial run 8.2 N/A $0.329 Current run

Run comparison

RunScoreConsistencyReliabilityTests CorrectFlaky testsTotal Output TokensTotal Input TokensTotal CostResponse Time (avg)
2026-04-25 21:53 · Initial run8.29.2N/A13/18284,5620$0.32972.92s
2026-05-22 00:54 · Suite changed6.68.39.011/20465,9850$0.21258.93s
Difference+1.6+0.9+2-2+185770+$0.117+13988ms

These two runs used different benchmark suites, so the deltas reflect both model changes and suite changes.

Charts

Choose the first model, then click a second model to open a side-by-side page.

Total Output Tokens

Score vs Total Output Tokens

Quick Compare

DeepSeek V4 ProhighvsDeepSeek V4 FlashhighDeepSeek V4 ProhighvsMiniMax M3mediumDeepSeek V4 ProhighvsGPT-5.6 LunahighDeepSeek V4 ProhighvsGrok Build 0.1mediumDeepSeek V4 ProhighvsClaude Opus 4.6mediumDeepSeek V4 ProhighvsGPT-5.6 LunamediumDeepSeek V4 ProhighvsGLM 5mediumDeepSeek V4 ProhighvsNemotron 3 UltramediumFree AvailableDeepSeek V4 ProhighvsClaude Opus 4.8lowDeepSeek V4 ProhighvsKimi K2.7 Codemedium

Category Breakdown

Category Score Consistency Tests Correct
Anti-AI Tricks 8.3 10.0
Coding 10.0 10.0
Combined 10.0 10.0
Data parsing and extraction 10.0 10.0
Domain specific 5.3 10.0
General Intelligence 6.1 3.1
Instructions following 10.0 10.0
Puzzle Solving 7.6 7.4
Tool Calling 10.0 10.0

Compared models