Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Compared models

Summary

Grok 4.20 vs Grok 4.20 Beta vs Grok 4.3 benchmark comparisonGrok 4.3 leads on Score with 7.7. Grok 4.20 leads on Reliability with 10.0. Grok 4.20 has the lowest Total Cost at $0.609. Grok 4.20 Beta is fastest at 9.75s.

Recommended model: Grok 4.20 Beta - It offers the best overall trade-off: a competitive score (6.8), faster response than the other models in this comparison, and balanced cost.

Last updated at: 2026-06-17

Metric Grok 4.20 Grok 4.20 medium Release: 2026-03-31 Grok 4.20 Beta Grok 4.20 Beta medium Release: 2026-03-12 Grok 4.3 Grok 4.3 medium Release: 2026-05-01
Score 7.3 6.8 7.7
Rank #53 #69 #37
Reliability 10.0 N/A 10.0
Consistency 8.8 8.2 8.5
Tests Correct
Attempt pass rate 63.5% 69.8% 71.4%
Flaky tests 3 1 4
Total Runs 63 52 63
Cost per result 8.309 4.505 4.724
Total Cost $0.609 $0.750 $0.614
Input Price $1.250 / 1M $5.805 / 1M $1.250 / 1M
Output Price $2.500 / 1M $5.805 / 1M $2.500 / 1M
Total Input Tokens 44,433 35,955 44,472
Output Tokens 1,819 1,647 1,981
Reasoning Tokens 219,524 91,565 221,382
Response Time (avg) 27.68s 9.75s 47.51s
Response Time (max) 199.66s 31.36s 216.69s
Response Time (total) 581.26s 175.48s 997.68s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#53 xAI: Grok 4.20

medium
Cost
$0.041
Time
110.3s
Tokens
16,336 tok

#69 Grok 4.20 Beta

medium
Cost
$0.034
Time
91.0s
Tokens
13,523 tok

#37 xAI: Grok 4.3

medium
Cost
$0.009
Time
19.0s
Tokens
3,661 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 8.2 7.9 83.3% 1 3.95s 2,010 287 8,312
Grok 4.20 Beta 8.7 7.9 91.7% 1 3.16s 2,010 268 7,583
Grok 4.3 10.0 10.0 100.0% 0 8.83s 2,010 88 8,207
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 6.3 6.6 55.6% 1 109.93s 8,307 268 103,150
Grok 4.20 Beta 3.3 3.3 33.3% 0 31.36s 360 81 3,987
Grok 4.3 5.9 7.7 44.4% 1 41.23s 8,340 1,028 31,226
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 10.0 10.0 100.0% 0 17.40s 12,909 232 9,556
Grok 4.20 Beta 10.0 10.0 100.0% 0 20.93s 12,909 227 12,212
Grok 4.3 10.0 10.0 100.0% 0 63.99s 12,909 234 15,301
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 10.0 10.0 100.0% 0 4.17s 7,761 180 5,333
Grok 4.20 Beta 10.0 10.0 100.0% 0 4.01s 7,761 180 5,281
Grok 4.3 10.0 10.0 100.0% 0 18.97s 7,761 180 9,546
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 5.3 10.0 33.3% 0 27.03s 1,764 375 49,339
Grok 4.20 Beta 5.3 10.0 33.3% 0 21.33s 1,764 251 40,255
Grok 4.3 5.3 7.2 44.4% 1 181.74s 1,764 14 111,300
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 3.9 2.6 33.3% 1 24.48s 825 65 6,440
Grok 4.20 Beta 10.0 10.0 100.0% 0 5.78s 825 72 3,440
Grok 4.3 5.4 2.5 66.7% 1 24.70s 825 70 5,020
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 9.8 10.0 100.0% 0 4.26s 1,362 57 6,419
Grok 4.20 Beta 9.8 10.0 100.0% 0 4.89s 1,362 57 7,123
Grok 4.3 9.8 10.0 100.0% 0 18.58s 1,362 57 8,713
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 7.7 10.0 66.7% 0 6.22s 1,689 149 7,913
Grok 4.20 Beta 10.0 10.0 100.0% 0 3.52s 1,689 328 6,300
Grok 4.3 5.9 7.2 55.6% 1 22.52s 1,689 128 14,468
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 3.0 10.0 0.0% 0 13.68s 7,275 197 6,620
Grok 4.20 Beta 3.0 10.0 0.0% 0 12.39s 7,275 183 5,384
Grok 4.3 10.0 10.0 100.0% 0 17.66s 7,263 168 4,615
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Grok 4.20 3.0 10.0 0.0% 0 63.48s 531 9 16,442
Grok 4.20 Beta 0.0 0.0 0.0% 0 0ms 0 0 0
Grok 4.3 3.0 10.0 0.0% 0 44.47s 549 14 12,986

Quick Compare

Switch Comparison Pair

Step 3.7 FlashlowvsGrok 4.3mediumGemini 3 Flash PreviewlowvsGrok 4.20mediumClaude Sonnet 4.6nonevsGrok 4.20mediumDeepSeek V4 ProhighvsGrok 4.3mediumClaude Opus 4.8nonevsGrok 4.20mediumGPT-5.3 ChatnonevsGrok 4.20mediumDeepSeek V4 PrononevsGrok 4.20mediumQwen3.7 PlusnonevsGrok 4.20mediumGPT-5.3 ChatnonevsGrok 4.3mediumGrok 4.20mediumvsGLM 5.2noneStep 3.7 FlashhighvsGrok 4.20mediumDeepSeek V4 ProhighvsGrok 4.20medium