AI BENCHY Failures
Wrong answer Failures
See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↓.
Categories
169/169
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #29 | Qwen3.5-27B medium | Qwen | 4 | 7.9 | $0.536 | 13/21 | 68.4s |
| #27 | GPT-5.4 Mini medium | OpenAI | 6 | 8.0 | $0.526 | 12/21 | 22.3s |
| #3 | Qwen3.7 Max medium | Qwen | 3 | 9.4 | $0.523 | 18/21 | 16.0s |
| #49 | Claude Opus 4.7 none | Anthropic | 3 | 7.4 | $0.505 | 16/19 | 3.02s |
| #56 | GLM 5V Turbo medium | Z.ai | 7 | 7.3 | $0.457 | 11/21 | 23.1s |
| #81 | Qwen3.6 27B medium | Qwen | 6 | 6.6 | $0.440 | 10/21 | 59.7s |
| #45 | GPT-5.3 Chat none | OpenAI | 7 | 7.5 | $0.433 | 12/21 | 6.34s |
| #89 | Qwen3.5-35B-A3B medium | Qwen | 2 | 6.3 | $0.401 | 11/21 | 72.6s |
| #19 | GPT-5.2 Chat none | OpenAI | 6 | 8.5 | $0.393 | 14/21 | 7.13s |
| #91 | Gemini 3 PRO Preview medium | 3 | 6.2 | $0.385 | 14/21 | 9.05s | |
| #24 | Gemini 2.5 Flash medium | 6 | 8.2 | $0.379 | 14/21 | 15.5s | |
| #20 | Step 3.7 Flash medium | Stepfun | 5 | 8.5 | $0.376 | 14/21 | 20.4s |
| #5 | Gemini 3.5 Flash low | 2 | 9.2 | $0.349 | 19/21 | 3.27s | |
| #43 | Kimi K2.5 medium | Moonshot AI | 5 | 7.5 | $0.348 | 10/21 | 98.4s |
| #39 | Step 3.7 Flash low | Stepfun | 8 | 7.7 | $0.341 | 12/21 | 15.7s |