Wrong answer Failure Ranking

AI BENCHY Failures

See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one.

Models Shown

Total Failures

1204

Most Affected Model

Categories

In category Domain specific314 In category Anti-AI Tricks245 In category Coding194 In category Puzzle Solving147 In category Trivia130 In category Instructions following53 In category Combined52 In category Data parsing and extraction35 In category General Intelligence32 In category Tool Calling2

Rank	Model	Company	Wrong answer Count	Score	Tests Correct	Response Time (avg)
#160	LFM2-24B-A2B none	Liquid	9	4.2	2/16	782ms
#162	Nemotron 3 Nano Omni 30b A3b Reasoning none	NVIDIA	9	4.1	2/19	728ms
#36	Qwen3.5 Plus 2026-04-20 medium	Qwen	8	7.6	13/21	46.4s
#39	Qwen3.6 Flash medium	Qwen	8	7.5	12/21	19.2s
#48	Gemini 3 Flash Preview none	Google	8	7.4	13/21	1.65s
#57	Step 3.7 Flash low	Stepfun	8	7.3	12/21	15.7s
#70	GPT-5.4 Nano medium	OpenAI	8	7.0	11/21	12.0s
#81	Mercury 2 medium	Inception	8	6.6	10/21	2.24s
#85	Gemma 4 31B none	Google	8	6.5	10/21	4.05s
#87	Gemini 3.1 Flash Lite minimal	Google	8	6.4	10/21	1.33s
#126	gpt-oss-120b none	OpenAI	8	5.4	6/19	21.6s
#146	Laguna Xs.2 none	Poolside	8	4.8	5/19	806ms
#156	Hy3 preview none	Tencent	8	4.4	4/21	12.9s
#34	Qwen3.7 Max none	Qwen	7	7.7	14/21	1.30s
#40	Gemini 3.1 Flash Lite Preview medium	Google	7	7.5	13/21	3.96s

Wrong answer Failures