AI BENCHY
Advertise here
#47

DeepSeek V4 Pro

DeepSeek Release: 2026-04-24 Tested on: 2026-04-26 10:50 deepseek/deepseek-v4-pro::high
(high) (none)

Summary

DeepSeek V4 Pro scores 7.5 on AI BENCHY and ranks #47. It has 8.4 reliability, a 73.2% pass rate, $0.201 total cost, and 60.02s average response time.

Score

7.5

Consistency

8.0

Reliability

8.4

Total Cost (Current Price)

$0.201

Total Output Tokens

57,875

Total Input Tokens

0

Input Price

$1.740 / 1M

Output Price

$3.480 / 1M

Tests Correct

Wrong Tests: 7

Attempt pass rate: 73.2%

Flaky tests

4

Flaky tests had mixed outcomes across runs (at least one pass and one fail).

Response Time (avg)

60.02s

Response Time (max): 358.35s

Response Time (total): 1080.31s

Wrong answer: 4 Did not follow instructions: 3

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#47 DeepSeek V4 Pro

high
Cost
$0.023
Time
257.6s
Tokens
14,870 tok

Run history

Tested on Score Reliability Tests Correct Total Cost Compare
2026-07-16 23:19 New test added 7.7 10.0 $0.200 Compare
2026-06-16 15:31 Re-test 7.6 9.3 $0.157 Compare
2026-06-16 14:47 Suite changed 8.1 9.6 $0.098 Compare
2026-06-04 14:38 New test added 6.0 8.9 $0.079 ↓ Compare
2026-05-22 00:54 Suite changed 6.6 9.0 $0.212 Compare
2026-04-29 14:47 Re-test 7.5 9.3 $0.209 Compare
2026-04-26 10:50 Re-test 7.5 8.4 $0.201 Current run
2026-04-25 21:53 Initial run 8.2 N/A $0.329 Compare

Run comparison

RunScoreConsistencyReliabilityTests CorrectFlaky testsTotal Output TokensTotal Input TokensTotal CostResponse Time (avg)
2026-04-26 10:50 · Re-test7.58.08.411/18457,8750$0.20160.02s
2026-06-04 14:38 · New test added6.07.38.98/21784,50732,240$0.07965.21s
Difference+1.5+0.7-0.5+3-3-26632-32240+$0.123-5192ms

These two runs used different benchmark suites, so the deltas reflect both model changes and suite changes.

Charts

Choose the first model, then click a second model to open a side-by-side page.

Total Output Tokens

Score vs Total Output Tokens

Quick Compare

DeepSeek V4 ProhighvsDeepSeek V4 FlashhighDeepSeek V4 ProhighvsMiniMax M3mediumDeepSeek V4 ProhighvsGPT-5.6 LunahighDeepSeek V4 ProhighvsGrok Build 0.1mediumDeepSeek V4 ProhighvsClaude Opus 4.6mediumDeepSeek V4 ProhighvsGPT-5.6 LunamediumDeepSeek V4 ProhighvsGLM 5mediumDeepSeek V4 ProhighvsNemotron 3 UltramediumFree AvailableDeepSeek V4 ProhighvsClaude Opus 4.8lowDeepSeek V4 ProhighvsKimi K2.7 Codemedium

Category Breakdown

Category Score Consistency Tests Correct
Anti-AI Tricks 8.3 10.0
Coding 3.0 0.0
Combined 10.0 10.0
Data parsing and extraction 10.0 10.0
Domain specific 3.2 6.9
General Intelligence 6.1 3.1
Instructions following 10.0 10.0
Puzzle Solving 8.0 7.0
Tool Calling 10.0 10.0

Compared models