AI BENCHY Category Failures
Coding: Wrong answer
Coding
Wrong answer
See which AI models are most likely to hit Wrong answer on Coding, so you can spot weak points faster. Sort by: Tests Correct ↑.
Failure Reasons
134/134
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Category Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #182 | Laguna Xs.2 none | Poolside | 1 | 8.3 | $0.004 | 0/1 | 1.96s |
| #183 | Granite 4.1 8B none | IBM Granite | 1 | 4.5 | $0.003 | 0/3 | 775ms |
| #184 | gpt-oss-120b none | OpenAI | 1 | 1.5 | $0.010 | 0/1 | 9.57s |
| #186 | Nemotron 3 Nano Omni 30b A3b Reasoning medium | NVIDIA | 1 | 1.1 | $0.000 | 0/1 | 38.1s |
| #30 | Qwen3.7 Plus medium | Qwen | 1 | 6.1 | $0.177 | 1/3 | 108.6s |
| #34 | Qwen3.5 Plus 2026-02-15 medium | Qwen | 1 | 6.6 | $0.310 | 1/3 | 180.7s |
| #35 | Qwen3.5-27B medium | Qwen | 2 | 6.2 | $0.536 | 1/3 | 160.7s |
| #37 | Qwen3.6 Plus medium | Qwen | 1 | 6.1 | $0.294 | 1/3 | 153.1s |
| #38 | GPT-5.6 Terra medium | OpenAI | 2 | 6.1 | $0.496 | 1/3 | 7.19s |
| #39 | Claude Sonnet 4.6 medium | Anthropic | 1 | 5.7 | $1.418 | 1/3 | 33.3s |
| #41 | Gemini 3.1 Flash Lite Preview medium | 2 | 5.5 | $0.068 | 1/3 | 4.09s | |
| #42 | Qwen3.5 Plus 2026-04-20 medium | Qwen | 2 | 6.2 | $0.317 | 1/3 | 125.3s |
| #43 | Gemini 3.1 Flash Lite medium | 2 | 5.5 | $0.071 | 1/3 | 3.81s | |
| #45 | Qwen3.5-122B-A10B medium | Qwen | 1 | 6.0 | $0.588 | 1/3 | 114.5s |
| #46 | Claude Opus 4.8 low | Anthropic | 1 | 6.6 | $1.270 | 1/3 | 7.58s |