AI BENCHY
Your ad here

#73

DeepSeek V4 Pro

DeepSeek · Release: 2026-04-24 · Tested on: 2026-04-24 09:45 · deepseek/deepseek-v4-pro::none

(high) (none)

Score

5.9

Consistency

8.6

Total Cost

$0.029

Total Output Tokens

1,615

Input Price

$1.740 / 1M

Output Price

$3.480 / 1M

Tests Correct

Wrong Tests: 11

Attempt pass rate: 44.4%

Flaky tests

3

Flaky tests had mixed outcomes across runs (at least one pass and one fail).

Response Time (avg)

27.94s

Response Time (max): 109.46s

Response Time (total): 502.88s

API error: 5 Wrong answer: 4 Extra formatting: 1 Did not follow instructions: 1

Run history

Charts

Choose the first model, then click a second model to open a side-by-side page.

Total Output Tokens

Score vs Total Output Tokens

Quick Compare

DeepSeek V4 PrononevsSeed-2.0-LitenoneDeepSeek V4 PrononevsQwen3.5-35B-A3BnoneDeepSeek V4 PrononevsGemma 4 26B A4BnoneFree AvailableDeepSeek V4 PrononevsDeepSeek V3.2noneDeepSeek V4 PrononevsQwen3.5-FlashnoneDeepSeek V4 PrononevsMiMo-V2-PrononeDeepSeek V4 PrononevsGemini 3 Flash PreviewmediumDeepSeek V4 PrononevsGemini 3.1 Pro PreviewmediumDeepSeek V4 PrononevsHY3 PreviewhighFree Available

Category Breakdown

Category Score Consistency Tests Correct
Anti-AI Tricks 4.8 10.0
Coding 10.0 10.0
Combined 9.5 10.0
Data parsing and extraction 6.4 5.8
Domain specific 5.3 10.0
General Intelligence 4.5 10.0
Instructions following 3.3 6.1
Puzzle Solving 6.0 7.1
Tool Calling 10.0 10.0

Compared models