AI BENCHY Category Failures
Instructions following
Wrong answer
Instructions following
Wrong answer
See which AI models are most likely to hit Wrong answer on Instructions following, so you can spot weak points faster. Sort by: Response Time (avg) ↑.
Related Failure Reasons
| Rank | Model | Company | Wrong answer Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #51 | Mercury 2 none | Inception | 1 | 5.5 | 1/2 | 551ms |
| #40 | Qwen3.5-122B-A10B none | Qwen | 2 | 4.5 | 0/2 | 585ms |
| #38 | Gemini 2.5 Flash none | 1 | 9.0 | 1/2 | 672ms | |
| #42 | Qwen3.5-35B-A3B none | Qwen | 1 | 5.0 | 1/2 | 809ms |
| #41 | Qwen3.5-27B none | Qwen | 2 | 4.5 | 0/2 | 815ms |
| #54 | MiMo-V2-Flash none | Xiaomi | 1 | 5.5 | 1/2 | 857ms |
| #49 | GLM 4.7 Flash none | Z.ai | 1 | 5.5 | 1/2 | 888ms |
| #53 | Grok 4.1 Fast none | X AI | 2 | 10.0 | 0/2 | 923ms |
| #44 | GPT-5.4 none | OpenAI | 1 | 5.5 | 1/2 | 1.07s |
| #55 | LFM2-24B-A2B none | Liquid | 2 | 4.5 | 0/2 | 1.09s |
| #45 | Trinity Large Preview none | Arcee AI | 1 | 3.5 | 0/2 | 1.09s |
| #47 | GPT-4o-mini none | OpenAI | 1 | 4.5 | 0/2 | 1.27s |
| #20 | Gemini 3 Flash Preview none | 1 | 5.5 | 1/2 | 1.58s | |
| #25 | Claude Sonnet 4.6 none | Anthropic | 1 | 5.5 | 1/2 | 1.96s |
| #46 | Kimi K2.5 none | Moonshot AI | 1 | 5.5 | 1/2 | 2.67s |
| #52 | GLM 4.7 Flash medium | Z.ai | 1 | 5.0 | 1/2 | 2.97s |
| #19 | GPT-5.3 Chat none | OpenAI | 1 | 9.0 | 1/2 | 3.29s |
| #15 | GPT-5.2 Chat none | OpenAI | 1 | 6.0 | 1/2 | 5.46s |
| #50 | Qwen3 Coder Next medium | Qwen | 1 | 4.5 | 0/2 | 7.34s |
| #48 | Qwen3 Coder Next none | Qwen | 2 | 4.5 | 0/2 | 7.71s |
| #37 | Qwen3.5-Flash none | Qwen | 1 | 5.0 | 1/2 | 8.81s |