Did not follow instructions Failure Ranking

See which AI models run into Did not follow instructions most often, so you can spot reliability risks before choosing one.

Models Shown

Total Failures

522

Most Affected Model

Claude Sonnet 5.5 5

Categories

In category Puzzle Solving166 In category General Intelligence142 In category Instructions following77 In category Anti-AI Tricks52 In category Coding50 In category Tool Calling13 In category Agentic10 In category Data parsing and extraction8 In category Domain specific3 In category Combined1

293/293

Rank	Model	Company	Did not follow instructions Count	Score	Total Cost	Tests Correct	Response Time (avg)
#67	Qwen3.8 27B high	Qwen	2	0.9	~$0.298	17/23	166.3s
Total Tests 23 Wrong Tests 6 Total Cost ~$0.298 Response Time (avg) 166.3s
#75	Claude Haiku 5.5 low	Anthropic	2	0.9	$0.061	16/23	7.51s
Total Tests 23 Wrong Tests 7 Total Cost $0.061 Response Time (avg) 7.51s
#82	Nemotron 3.5 Super VL high	NVIDIA	2	0.8	$0.349	15/23	49.9s
Total Tests 23 Wrong Tests 8 Total Cost $0.349 Response Time (avg) 49.9s
#88	Muse Spark 1.1 high	Meta	2	0.8	$2.570	13/23	37.4s
Total Tests 23 Wrong Tests 10 Total Cost $2.570 Response Time (avg) 37.4s
#89	DeepSeek V4 Flash 0731 high	DeepSeek	2	0.8	$0.576	15/23	118.8s
Total Tests 23 Wrong Tests 8 Total Cost $0.576 Response Time (avg) 118.8s
#100	DeepSeek V4.1 Flash 1017 max	DeepSeek	2	0.8	$0.757	15/23	78.2s
Total Tests 23 Wrong Tests 8 Total Cost $0.757 Response Time (avg) 78.2s
#104	Seed-2.0-Lite medium	Bytedance Seed	2	0.8	$0.271	14/23	48.9s
Total Tests 23 Wrong Tests 9 Total Cost $0.271 Response Time (avg) 48.9s
#106	DeepSeek V4 Pro 0423 high	DeepSeek	2	0.8	$0.149	11/23	91.3s
Total Tests 23 Wrong Tests 12 Total Cost $0.149 Response Time (avg) 91.3s
#108	DeepSeek V4 Flash 0423 high	DeepSeek	2	0.8	$0.261	14/23	56.0s
Total Tests 23 Wrong Tests 9 Total Cost $0.261 Response Time (avg) 56.0s
#112	GPT-5.4 Nano medium	OpenAI	2	0.8	$0.235	13/23	16.0s
Total Tests 23 Wrong Tests 10 Total Cost $0.235 Response Time (avg) 16.0s
#116	Qwen3.5-27B medium	Qwen	2	0.8	$1.135	14/23	114.7s
Total Tests 23 Wrong Tests 9 Total Cost $1.135 Response Time (avg) 114.7s
#122	Qwen3.8 2.4T A95B high	Qwen	2	0.8	$2.949	15/23	120.9s
Total Tests 23 Wrong Tests 8 Total Cost $2.949 Response Time (avg) 120.9s
#126	DeepSeek V4.1 Flash 1017 high	DeepSeek	2	0.8	$0.589	13/23	58.6s
Total Tests 23 Wrong Tests 10 Total Cost $0.589 Response Time (avg) 58.6s
#129	Step 3.7 Flash medium	Stepfun	2	0.8	$0.571	13/23	26.6s
Total Tests 23 Wrong Tests 10 Total Cost $0.571 Response Time (avg) 26.6s
#138	Claude Haiku 5.5 medium	Anthropic	2	0.8	$0.063	13/23	7.71s
Total Tests 23 Wrong Tests 10 Total Cost $0.063 Response Time (avg) 7.71s

Did not follow instructions Failures

Filter models

Top Models by Did not follow instructions Count

Did not follow instructions Count vs Score

Top Models by Response Time (avg)