AI BENCHY Failures
Wrong answer Failures
See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↑.
Categories
169/169
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #117 | DeepSeek V4 Flash none | DeepSeek | 12 | 5.5 | $0.007 | 5/21 | 26.8s |
| #161 | Grok 4.1 Fast none | X AI | 13 | 4.0 | $0.008 | 3/19 | 1.62s |
| #137 | Trinity Large Preview none | Arcee AI | 12 | 5.0 | $0.008 | 4/21 | 2.98s |
| #148 | Qwen3 Coder Next medium | Qwen | 13 | 4.7 | $0.008 | 4/21 | 8.58s |
| #130 | Qwen3 Coder Next none | Qwen | 14 | 5.1 | $0.009 | 5/21 | 8.62s |
| #164 | gpt-oss-120b none | OpenAI | 8 | 4.0 | $0.010 | 6/19 | 21.6s |
| #145 | GPT-5.4 Nano none | OpenAI | 15 | 4.8 | $0.011 | 4/21 | 1.48s |
| #151 | Mercury 2 none | Inception | 16 | 4.6 | $0.011 | 4/21 | 653ms |
| #103 | Qwen3.5-35B-A3B none | Qwen | 12 | 5.9 | $0.012 | 7/21 | 3.37s |
| #78 | gpt-oss-120b medium | OpenAI | 9 | 6.7 | $0.013 | 9/21 | 22.3s |
| #96 | Gemini 3.1 Flash Lite none | 11 | 6.1 | $0.013 | 9/21 | 1.06s | |
| #94 | Gemini 3.1 Flash Lite minimal | 8 | 6.1 | $0.013 | 10/21 | 1.33s | |
| #102 | Qwen3.6 Flash none | Qwen | 12 | 6.0 | $0.015 | 7/21 | 1.60s |
| #104 | Qwen3.5-27B none | Qwen | 12 | 5.9 | $0.015 | 7/21 | 1.68s |
| #93 | Gemini 2.5 Flash none | 12 | 6.2 | $0.016 | 9/21 | 875ms |