AI BENCHY Failures
Wrong answer Failures
See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↑.
Categories
169/169
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #10 | GPT-5.3-Codex medium | OpenAI | 4 | 8.9 | $0.740 | 15/21 | 16.2s |
| #69 | Grok 4.20 Beta medium | X AI | 3 | 6.8 | $0.750 | 14/18 | 9.75s |
| #35 | Kimi K2.6 medium | Moonshot AI | 3 | 7.8 | $0.889 | 12/21 | 71.7s |
| #4 | GPT-5.5 low | OpenAI | 3 | 9.3 | $0.907 | 18/21 | 9.76s |
| #42 | Grok Build 0.1 medium | X AI | 5 | 7.6 | $0.927 | 13/21 | 49.9s |
| #11 | Qwen3.6 Max Preview medium | Qwen | 5 | 8.9 | $0.960 | 16/21 | 59.6s |
| #7 | Gemini 3.1 Pro Preview medium | 2 | 9.2 | $1.054 | 19/21 | 20.1s | |
| #66 | Gemini 3.5 Flash none | 3 | 7.0 | $1.079 | 15/21 | 9.93s | |
| #12 | Claude Opus 4.8 medium | Anthropic | 3 | 8.8 | $1.107 | 17/21 | 9.66s |
| #1 | Gemini 3.5 Flash high | 1 | 9.8 | $1.115 | 20/21 | 8.84s | |
| #63 | Step 3.7 Flash high | Stepfun | 6 | 7.1 | $1.148 | 11/21 | 64.5s |
| #17 | GPT-5.4 medium | OpenAI | 5 | 8.5 | $1.210 | 14/21 | 22.3s |
| #31 | Claude Sonnet 4.6 medium | Anthropic | 4 | 7.8 | $1.418 | 13/21 | 17.1s |
| #83 | Gemini 3.1 Flash Lite high | 4 | 6.5 | $2.044 | 10/18 | 62.0s | |
| #38 | Claude Opus 4.6 medium | Anthropic | 3 | 7.7 | $2.053 | 12/21 | 25.9s |