Wrong answer Failure Ranking

AI BENCHY Failures

See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one.

Models Shown

Total Failures

1204

Most Affected Model

Categories

In category Domain specific314 In category Anti-AI Tricks245 In category Coding194 In category Puzzle Solving147 In category Trivia130 In category Instructions following53 In category Combined52 In category Data parsing and extraction35 In category General Intelligence32 In category Tool Calling2

Rank	Model	Company	Wrong answer Count	Score	Tests Correct	Response Time (avg)
#155	Mercury 2 none	Inception	16	4.5	4/21	653ms
#135	Kimi K2.5 none	Moonshot AI	15	5.2	6/21	13.2s
#142	Mistral Small 4 none	Mistral	15	4.9	5/21	630ms
#147	GPT-4o-mini none	OpenAI	15	4.8	5/21	1.77s
#148	GPT-5.4 Nano none	OpenAI	15	4.7	4/21	1.48s
#140	Qwen3 Coder Next none	Qwen	14	4.9	5/21	8.62s
#141	Nemotron 3 Super none	NVIDIA	14	4.9	5/21	5.30s
#143	MiMo-V2.5 none	Xiaomi	14	4.9	5/21	2.20s
#154	Qwen3.5-9B none	Qwen	14	4.6	4/21	1.89s
#108	Qwen3.5-Flash none	Qwen	13	5.8	8/21	3.58s
#110	Seed-2.0-Lite none	Bytedance Seed	13	5.8	8/21	2.49s
#112	GLM 5.1 none	Z.ai	13	5.7	7/21	4.10s
#122	GLM 4.7 Flash none	Z.ai	13	5.5	6/21	2.86s
#125	GPT-5.4 none	OpenAI	13	5.5	7/21	1.42s
#131	Qwen3.5-122B-A10B none	Qwen	13	5.3	6/21	3.41s

1 2 11

→

Wrong answer Failures