AI BENCHY Failures
Wrong answer Failures
See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↓.
Categories
169/169
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #4 | GPT-5.5 low | OpenAI | 3 | 9.3 | $0.907 | 18/21 | 9.76s |
| #35 | Kimi K2.6 medium | Moonshot AI | 3 | 7.8 | $0.889 | 12/21 | 71.7s |
| #69 | Grok 4.20 Beta medium | X AI | 3 | 6.8 | $0.750 | 14/18 | 9.75s |
| #10 | GPT-5.3-Codex medium | OpenAI | 4 | 8.9 | $0.740 | 15/21 | 16.2s |
| #73 | Mimo V2 Omni medium | Xiaomi | 5 | 6.8 | $0.683 | 10/21 | 41.2s |
| #13 | Claude Opus 4.7 medium | Anthropic | 3 | 8.7 | $0.679 | 17/21 | 4.73s |
| #2 | Gemini 3 Flash Preview medium | 1 | 9.6 | $0.667 | 20/21 | 18.6s | |
| #37 | Grok 4.3 medium | X AI | 5 | 7.7 | $0.614 | 13/21 | 47.5s |
| #53 | Grok 4.20 medium | X AI | 6 | 7.3 | $0.609 | 12/21 | 27.7s |
| #36 | Qwen3.5-122B-A10B medium | Qwen | 5 | 7.7 | $0.588 | 14/21 | 42.5s |
| #65 | Kimi K2.7 Code medium | Moonshot AI | 5 | 7.0 | $0.583 | 11/21 | 83.6s |
| #8 | Gemini 3.5 Flash medium | 2 | 9.1 | $0.582 | 18/21 | 4.94s | |
| #22 | GPT-5.2 medium | OpenAI | 3 | 8.4 | $0.548 | 13/21 | 16.9s |
| #160 | Grok Build 0.1 none | X AI | 7 | 4.2 | $0.547 | 7/19 | 28.7s |
| #57 | Claude Opus 4.8 none | Anthropic | 4 | 7.2 | $0.539 | 12/21 | 3.47s |