AI BENCHY Category Failures
Coding: Wrong answer
Coding
Wrong answer
See which AI models are most likely to hit Wrong answer on Coding, so you can spot weak points faster. Sort by: Total Cost ↓.
Failure Reasons
134/134
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Category Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #107 | Seed-2.0-Lite none | Bytedance Seed | 2 | 5.6 | $0.019 | 1/3 | 2.83s |
| #98 | Gemini 3.1 Flash Lite Preview none | 2 | 5.5 | $0.018 | 1/3 | 967ms | |
| #137 | MiMo-V2.5-Pro none | Xiaomi | 2 | 4.3 | $0.017 | 0/3 | 1.41s |
| #123 | Qwen3.5 Plus 2026-02-15 none | Qwen | 3 | 4.3 | $0.016 | 0/3 | 2.05s |
| #146 | DeepSeek V3.2 none | DeepSeek | 2 | 3.1 | $0.016 | 0/3 | 14.5s |
| #109 | Gemini 2.5 Flash none | 2 | 5.5 | $0.016 | 1/3 | 736ms | |
| #176 | Laguna Xs.2 medium | Poolside | 1 | 2.1 | $0.015 | 0/1 | 14.4s |
| #121 | Qwen3.5-27B none | Qwen | 2 | 5.8 | $0.015 | 1/3 | 1.80s |
| #119 | Qwen3.6 Flash none | Qwen | 2 | 5.4 | $0.015 | 1/3 | 1.79s |
| #110 | Gemini 3.1 Flash Lite minimal | 2 | 5.5 | $0.013 | 1/3 | 831ms | |
| #112 | Gemini 3.1 Flash Lite none | 2 | 5.5 | $0.013 | 1/3 | 938ms | |
| #92 | gpt-oss-120b medium | OpenAI | 2 | 5.9 | $0.013 | 1/3 | 38.4s |
| #120 | Qwen3.5-35B-A3B none | Qwen | 2 | 5.5 | $0.012 | 1/3 | 1.39s |
| #171 | Mercury 2 none | Inception | 3 | 3.4 | $0.011 | 0/3 | 1.03s |
| #165 | GPT-5.4 Nano none | OpenAI | 3 | 4.6 | $0.011 | 0/3 | 2.22s |