Wrong answer Failure Ranking

AI BENCHY Failures

See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Tests Correct ↑.

Models Shown

Total Failures

572

Most Affected Model

Categories

In category Domain specific182 In category Anti-AI Tricks165 In category Puzzle Solving85 In category Instructions following44 In category Combined37 In category Coding28 In category Data parsing and extraction19 In category General Intelligence10 In category Tool Calling2

Rank	Model	Company	Wrong answer Count	Score	Tests Correct	Response Time (avg)
#9	Qwen3.6 Plus Preview medium	Qwen	3	8.5	13/17	13.9s
#8	Qwen3.5 Plus 2026-02-15 medium	Qwen	2	8.5	14/18	46.6s
#12	Gemini 3 PRO Preview medium	Google	3	8.4	14/18	9.06s
#5	Gemini 3 Flash Preview low	Google	3	8.8	15/18	6.01s
#3	Claude Opus 4.7 medium	Anthropic	1	9.2	16/18	3.53s
#4	Claude Opus 4.7 none	Anthropic	2	9.2	16/18	3.13s
#2	Gemini 3.1 Pro Preview medium	Google	1	9.6	17/18	16.0s

←

1 2 3 4 5 6 7

Wrong answer Failures