Wrong answer Failure Ranking

AI BENCHY Failures

See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↑.

Models Shown

Total Failures

1243

Most Affected Model

Categories

In category Domain specific325 In category Anti-AI Tricks250 In category Coding201 In category Puzzle Solving154 In category Trivia133 In category Instructions following54 In category Combined53 In category General Intelligence36 In category Data parsing and extraction35 In category Tool Calling2

169/169

Rank	Model	Company	Wrong answer Count	Score	Total Cost	Tests Correct	Response Time (avg)
#95	Gemini 3.1 Flash Lite Preview high	Google	2	6.1	$2.310	13/16	68.1s
Total Tests 16 Wrong Tests 3 Total Cost $2.310 Response Time (avg) 68.1s
#6	Claude Fable 5 medium	Anthropic	2	9.2	$3.165	17/21	17.0s
Total Tests 21 Wrong Tests 4 Total Cost $3.165 Response Time (avg) 17.0s
#9	GPT-5.5 medium	OpenAI	4	9.0	$3.679	17/21	38.0s
Total Tests 21 Wrong Tests 4 Total Cost $3.679 Response Time (avg) 38.0s
#136	Grok 4.20 Multi Agent Beta medium	X AI	4	5.0	$5.599	8/18	9.69s
Total Tests 18 Wrong Tests 10 Total Cost $5.599 Response Time (avg) 9.69s

←

1 11 12

Wrong answer Failures