AI BENCHY Category Failures
Instructions following: Did not follow instructions
Instructions following
Did not follow instructions
See which AI models are most likely to hit Did not follow instructions on Instructions following, so you can spot weak points faster. Sort by: Response Time (avg) ↓.
Failure Reasons
| Rank | Model | Company | Did not follow instructions Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #11 | Gemini 3.1 Flash Lite Preview high | 1 | 7.9 | 1/2 | 70.1s | |
| #45 | GPT-5 Mini medium | OpenAI | 1 | 8.0 | 1/2 | 15.7s |
| #80 | MiniMax M2.7 medium | Minimax | 1 | 3.7 | 0/2 | 12.6s |
| #57 | GPT-5 Nano medium | OpenAI | 1 | 8.5 | 1/2 | 11.9s |
| #51 | Nemotron 3 Super medium | NVIDIA | 1 | 7.2 | 1/2 | 7.72s |
| #92 | Qwen3 Coder Next medium | Qwen | 1 | 4.8 | 0/2 | 7.34s |
| #52 | Grok 4.1 Fast medium | X AI | 1 | 6.6 | 1/2 | 5.30s |
| #84 | gpt-oss-120b none | OpenAI | 1 | 8.4 | 1/2 | 5.10s |
| #30 | Step 3.5 Flash medium | Stepfun | 1 | 8.5 | 1/2 | 4.98s |
| #25 | Grok 4.20 Beta medium | X AI | 1 | 8.3 | 1/2 | 4.97s |
| #35 | MiMo-V2-Omni medium | Xiaomi | 1 | 8.3 | 1/2 | 4.92s |
| #71 | MiniMax M2.5 medium | Minimax | 1 | 8.1 | 1/2 | 4.64s |
| #56 | Grok 4.20 Multi Agent Beta medium | X AI | 1 | 8.3 | 1/2 | 4.63s |
| #44 | GPT-5.4 Mini medium | OpenAI | 1 | 7.4 | 1/2 | 2.50s |
| #75 | GLM 5.1 none | Z.ai | 1 | 8.3 | 1/2 | 1.58s |