Coding x Did not follow instructions Ranking

See which AI models are most likely to hit Did not follow instructions on Coding, so you can spot weak points faster. Sort by: Total Cost ↑.

Models Shown

Total Failures

Most Affected Model

Failure Reasons

Wrong answer456 API error64 Did not follow instructions41 No answer41 Extra formatting36 Timed out24

Categories

Puzzle Solving147 General Intelligence134 Instructions following65 Anti-AI Tricks49 Coding41 Tool Calling12 Agentic10 Data parsing and extraction8 Domain specific3 Combined1

38/38

Rank	Model	Company	Did not follow instructions Count	Category Score	Total Cost	Tests Correct	Response Time (avg)
#204	Claude Sonnet 5.5 medium	Anthropic	2	0.3	$1.100	0/3	6.07s
Total Tests 3 Wrong Tests 3 Total Cost $1.100 Response Time (avg) 6.07s
#107	Gemini 3.5 Flash medium	Google	1	0.8	$1.316	2/3	12.6s
Total Tests 3 Wrong Tests 1 Total Cost $1.316 Response Time (avg) 12.6s
#130	Mistral Large 4 high	Mistral	1	0.3	$1.391	0/3	1277.8s
Total Tests 3 Wrong Tests 3 Total Cost $1.391 Response Time (avg) 1277.8s
#131	Seed-2.0-Code high	Bytedance Seed	1	0.6	$1.448	1/3	266.7s
Total Tests 3 Wrong Tests 2 Total Cost $1.448 Response Time (avg) 266.7s
#178	Claude Opus 4.8 none	Anthropic	1	0.5	$2.334	1/3	3.29s
Total Tests 3 Wrong Tests 2 Total Cost $2.334 Response Time (avg) 3.29s
#108	Qwen3.8 2.4T A95B high	Qwen	1	0.6	$2.949	1/3	160.5s
Total Tests 3 Wrong Tests 2 Total Cost $2.949 Response Time (avg) 160.5s
#232	Claude Opus 4.6 medium	Anthropic	1	0.6	$2.961	1/3	30.1s
Total Tests 3 Wrong Tests 2 Total Cost $2.961 Response Time (avg) 30.1s
#243	Claude Fable 5.1 low	Anthropic	2	0.3	$3.395	0/3	6.75s
Total Tests 3 Wrong Tests 3 Total Cost $3.395 Response Time (avg) 6.75s

Filter models