AI BENCHY Category Failures
Coding: Wrong answer
Coding
Wrong answer
See which AI models are most likely to hit Wrong answer on Coding, so you can spot weak points faster. Sort by: Total Cost ↓.
Failure Reasons
134/134
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Category Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #47 | Grok 4.3 medium | X AI | 1 | 5.9 | $0.614 | 1/3 | 41.2s |
| #66 | Grok 4.20 medium | X AI | 2 | 6.3 | $0.609 | 1/3 | 109.9s |
| #45 | Qwen3.5-122B-A10B medium | Qwen | 1 | 6.0 | $0.588 | 1/3 | 114.5s |
| #36 | Claude Sonnet 5 medium | Anthropic | 1 | 9.0 | $0.550 | 2/3 | 17.3s |
| #70 | Claude Opus 4.8 none | Anthropic | 1 | 5.5 | $0.539 | 1/3 | 3.29s |
| #35 | Qwen3.5-27B medium | Qwen | 2 | 6.2 | $0.536 | 1/3 | 160.7s |
| #33 | GPT-5.4 Mini medium | OpenAI | 1 | 8.4 | $0.526 | 2/3 | 57.9s |
| #38 | GPT-5.6 Terra medium | OpenAI | 2 | 6.1 | $0.496 | 1/3 | 7.19s |
| #69 | GLM 5V Turbo medium | Z.ai | 2 | 6.0 | $0.457 | 1/3 | 63.4s |
| #58 | GPT-5.3 Chat none | OpenAI | 2 | 5.6 | $0.433 | 1/3 | 10.5s |
| #22 | GPT-5.2 Chat none | OpenAI | 1 | 8.8 | $0.393 | 2/3 | 9.82s |
| #28 | Gemini 2.5 Flash medium | 1 | 7.8 | $0.379 | 2/3 | 41.0s | |
| #23 | Step 3.7 Flash medium | Stepfun | 1 | 8.8 | $0.376 | 2/3 | 27.4s |
| #8 | Gemini 3.5 Flash low | 1 | 7.8 | $0.349 | 2/3 | 6.71s | |
| #48 | GPT-5.6 Terra low | OpenAI | 2 | 6.6 | $0.343 | 1/3 | 9.56s |