AI BENCHY Failures
Wrong answer Failures
See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↓.
Categories
169/169
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #136 | Grok 4.20 Multi Agent Beta medium | X AI | 4 | 5.0 | $5.599 | 8/18 | 9.69s |
| #9 | GPT-5.5 medium | OpenAI | 4 | 9.0 | $3.679 | 17/21 | 38.0s |
| #6 | Claude Fable 5 medium | Anthropic | 2 | 9.2 | $3.165 | 17/21 | 17.0s |
| #95 | Gemini 3.1 Flash Lite Preview high | 2 | 6.1 | $2.310 | 13/16 | 68.1s | |
| #38 | Claude Opus 4.6 medium | Anthropic | 3 | 7.7 | $2.053 | 12/21 | 25.9s |
| #83 | Gemini 3.1 Flash Lite high | 4 | 6.5 | $2.044 | 10/18 | 62.0s | |
| #31 | Claude Sonnet 4.6 medium | Anthropic | 4 | 7.8 | $1.418 | 13/21 | 17.1s |
| #17 | GPT-5.4 medium | OpenAI | 5 | 8.5 | $1.210 | 14/21 | 22.3s |
| #63 | Step 3.7 Flash high | Stepfun | 6 | 7.1 | $1.148 | 11/21 | 64.5s |
| #1 | Gemini 3.5 Flash high | 1 | 9.8 | $1.115 | 20/21 | 8.84s | |
| #12 | Claude Opus 4.8 medium | Anthropic | 3 | 8.8 | $1.107 | 17/21 | 9.66s |
| #66 | Gemini 3.5 Flash none | 3 | 7.0 | $1.079 | 15/21 | 9.93s | |
| #7 | Gemini 3.1 Pro Preview medium | 2 | 9.2 | $1.054 | 19/21 | 20.1s | |
| #11 | Qwen3.6 Max Preview medium | Qwen | 5 | 8.9 | $0.960 | 16/21 | 59.6s |
| #42 | Grok Build 0.1 medium | X AI | 5 | 7.6 | $0.927 | 13/21 | 49.9s |