AI BENCHY Failures
Wrong answer Failures
See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↓.
Categories
169/169
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #84 | Gemini 3.1 Flash Lite Preview none | 7 | 6.4 | $0.018 | 12/21 | 1.21s | |
| #126 | DeepSeek V3.2 none | DeepSeek | 7 | 5.3 | $0.017 | 6/21 | 13.8s |
| #119 | MiMo-V2.5-Pro none | Xiaomi | 11 | 5.5 | $0.017 | 6/21 | 1.78s |
| #106 | Qwen3.5 Plus 2026-02-15 none | Qwen | 12 | 5.8 | $0.016 | 9/21 | 2.31s |
| #93 | Gemini 2.5 Flash none | 12 | 6.2 | $0.016 | 9/21 | 875ms | |
| #104 | Qwen3.5-27B none | Qwen | 12 | 5.9 | $0.015 | 7/21 | 1.68s |
| #102 | Qwen3.6 Flash none | Qwen | 12 | 6.0 | $0.015 | 7/21 | 1.60s |
| #94 | Gemini 3.1 Flash Lite minimal | 8 | 6.1 | $0.013 | 10/21 | 1.33s | |
| #96 | Gemini 3.1 Flash Lite none | 11 | 6.1 | $0.013 | 9/21 | 1.06s | |
| #78 | gpt-oss-120b medium | OpenAI | 9 | 6.7 | $0.013 | 9/21 | 22.3s |
| #103 | Qwen3.5-35B-A3B none | Qwen | 12 | 5.9 | $0.012 | 7/21 | 3.37s |
| #151 | Mercury 2 none | Inception | 16 | 4.6 | $0.011 | 4/21 | 653ms |
| #145 | GPT-5.4 Nano none | OpenAI | 15 | 4.8 | $0.011 | 4/21 | 1.48s |
| #164 | gpt-oss-120b none | OpenAI | 8 | 4.0 | $0.010 | 6/19 | 21.6s |
| #130 | Qwen3 Coder Next none | Qwen | 14 | 5.1 | $0.009 | 5/21 | 8.62s |