AI BENCHY Category Failures
Coding: Wrong answer
Coding
Wrong answer
See which AI models are most likely to hit Wrong answer on Coding, so you can spot weak points faster. Sort by: Tests Correct ↑.
Failure Reasons
134/134
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Category Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #47 | Grok 4.3 medium | X AI | 1 | 5.9 | $0.614 | 1/3 | 41.2s |
| #48 | GPT-5.6 Terra low | OpenAI | 2 | 6.6 | $0.343 | 1/3 | 9.56s |
| #51 | GPT-5.6 Luna high | OpenAI | 2 | 5.5 | $0.924 | 1/3 | 15.6s |
| #54 | Grok Build 0.1 medium | X AI | 1 | 5.7 | $0.927 | 1/3 | 108.5s |
| #55 | GPT-5.6 Luna medium | OpenAI | 2 | 5.4 | $0.258 | 1/3 | 10.4s |
| #58 | GPT-5.3 Chat none | OpenAI | 2 | 5.6 | $0.433 | 1/3 | 10.5s |
| #59 | GPT-5.4 Nano medium | OpenAI | 2 | 6.1 | $0.107 | 1/3 | 19.1s |
| #61 | DeepSeek V3.2 medium | DeepSeek | 1 | 6.0 | $0.042 | 1/3 | 248.7s |
| #63 | Seed-2.0-Mini medium | Bytedance Seed | 1 | 5.5 | $0.044 | 1/3 | 220.5s |
| #65 | Gemini 3 Flash Preview low | 2 | 5.8 | $0.111 | 1/3 | 6.00s | |
| #66 | Grok 4.20 medium | X AI | 2 | 6.3 | $0.609 | 1/3 | 109.9s |
| #68 | Claude Sonnet 4.6 none | Anthropic | 1 | 5.5 | $0.316 | 1/3 | 5.19s |
| #69 | GLM 5V Turbo medium | Z.ai | 2 | 6.0 | $0.457 | 1/3 | 63.4s |
| #70 | Claude Opus 4.8 none | Anthropic | 1 | 5.5 | $0.539 | 1/3 | 3.29s |
| #71 | DeepSeek V4 Pro none | DeepSeek | 1 | 5.6 | $0.034 | 1/3 | 13.4s |