AI BENCHY Category Failures
Coding: Wrong answer
Coding
Wrong answer
See which AI models are most likely to hit Wrong answer on Coding, so you can spot weak points faster. Sort by: Response Time (avg) ↑.
Failure Reasons
| Rank | Model | Company | Wrong answer Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #153 | Granite 4.1 8B none | IBM Granite | 1 | 5.2 | 0/2 | 706ms |
| #92 | Gemini 2.5 Flash none | 1 | 6.8 | 1/2 | 810ms | |
| #143 | Mercury 2 none | Inception | 2 | 3.5 | 0/2 | 831ms |
| #78 | Gemini 3.1 Flash Lite minimal | 1 | 6.8 | 1/2 | 951ms | |
| #97 | Qwen3.5-Flash none | Qwen | 1 | 6.8 | 1/2 | 993ms |
| #136 | GPT-5.4 Mini none | OpenAI | 1 | 6.8 | 1/2 | 1.01s |
| #135 | Mistral Small 4 none | Mistral | 2 | 4.0 | 0/2 | 1.03s |
| #49 | Gemini 3.1 Flash Lite Preview none | 1 | 6.8 | 1/2 | 1.06s | |
| #147 | GPT-5.4 Nano none | OpenAI | 2 | 5.4 | 0/2 | 1.09s |
| #85 | Gemini 3.1 Flash Lite none | 1 | 6.8 | 1/2 | 1.13s | |
| #117 | Grok 4.20 Beta none | X AI | 1 | 5.5 | 0/1 | 1.14s |
| #141 | Qwen3 Coder Next medium | Qwen | 1 | 4.1 | 0/2 | 1.17s |
| #120 | Grok 4.20 none | X AI | 1 | 3.4 | 0/1 | 1.22s |
| #122 | Elephant Alpha medium | Openrouter | 1 | 4.0 | 0/2 | 1.30s |
| #26 | Qwen3.7 Max none | Qwen | 1 | 6.8 | 1/2 | 1.39s |