Did not follow instructions Failure Ranking

AI BENCHY Failures

See which AI models run into Did not follow instructions most often, so you can spot reliability risks before choosing one. Sort by: Score ↓.

Models Shown

Total Failures

180

Most Affected Model

Categories

In category Puzzle Solving75 In category General Intelligence58 In category Instructions following22 In category Anti-AI Tricks17 In category Tool Calling6 In category Coding2

Rank	Model	Company	Did not follow instructions Count	Score	Tests Correct	Response Time (avg)
#6	Seed-2.0-Lite medium	Bytedance Seed	2	8.6	13/18	30.4s
#7	GPT-5.3-Codex medium	OpenAI	2	8.6	13/18	15.4s
#9	Qwen3.6 Plus Preview medium	Qwen	1	8.5	13/17	13.9s
#10	Qwen3.5-27B medium	Qwen	2	8.4	13/18	53.0s
#11	Gemini 3.1 Flash Lite Preview high	Google	1	8.4	12/16	68.8s
#13	GLM 5 medium	Z.ai	1	8.4	13/18	23.3s
#14	Gemma 4 31B medium	Google	1	8.3	13/18	24.9s
#15	Gemini 2.5 Flash medium	Google	1	8.2	13/18	12.1s
#16	GPT-5.4 medium	OpenAI	2	8.2	13/18	18.6s
#17	Gemini 3.1 Flash Lite Preview medium	Google	1	8.2	13/18	3.74s
#18	GLM 5 Turbo medium	Z.ai	2	8.1	12/18	17.7s
#20	Qwen3.6 Plus medium	Qwen	1	8.1	13/18	15.3s
#22	Gemini 3.1 Flash Lite Preview low	Google	1	8.1	13/18	3.22s
#23	MiMo-V2-Pro medium	Xiaomi	1	8.1	12/18	12.3s
#24	Gemma 4 26B A4B medium	Google	1	8.0	13/18	25.0s

1 2 3 4 5 6

→

Did not follow instructions Failures