AI BENCHY Category Failures
Coding: Wrong answer
Coding
Wrong answer
See which AI models are most likely to hit Wrong answer on Coding, so you can spot weak points faster. Sort by: Total Cost ↑.
Failure Reasons
134/134
Filter models
No models match the current search and filters.
| Rank | Model | Company | Wrong answer Count | Category Score | Total Cost | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|---|
| #131 | Mimo V2 Omni none | Xiaomi | 1 | 4.4 | $0.021 | 0/3 | 2.75s |
| #73 | Qwen3.7 Plus none | Qwen | 2 | 5.5 | $0.023 | 1/3 | 2.15s |
| #81 | Gemini 3 Flash Preview none | 2 | 5.5 | $0.025 | 1/3 | 1.80s | |
| #138 | Qwen3.6 27B none | Qwen | 2 | 5.5 | $0.025 | 1/3 | 4.16s |
| #179 | MiMo-V2-Flash none | Xiaomi | 2 | 4.3 | $0.025 | 0/3 | 2.64s |
| #96 | Gemini 3.1 Flash Lite Preview low | 2 | 5.5 | $0.026 | 1/3 | 1.39s | |
| #118 | GLM 5 none | Z.ai | 3 | 4.0 | $0.027 | 0/3 | 5.12s |
| #27 | DeepSeek V4 Flash high | DeepSeek | 1 | 7.8 | $0.027 | 2/3 | 50.6s |
| #115 | Nemotron 3 Ultra 550b A55b none | NVIDIA | 2 | 5.5 | $0.027 | 1/3 | 1.02s |
| #136 | Kimi K2.5 none | Moonshot AI | 2 | 5.5 | $0.027 | 1/3 | 24.6s |
| #99 | Gemini 3.1 Flash Lite low | 2 | 5.5 | $0.028 | 1/3 | 1.53s | |
| #148 | Qwen3.6 35B A3B none | Qwen | 2 | 5.5 | $0.031 | 1/3 | 8.77s |
| #140 | Qwen3.5 Plus 2026-04-20 none | Qwen | 2 | 3.9 | $0.032 | 0/3 | 1.69s |
| #71 | DeepSeek V4 Pro none | DeepSeek | 1 | 5.6 | $0.034 | 1/3 | 13.4s |
| #78 | Laguna XS 2.1 medium | Poolside | 2 | 5.5 | $0.036 | 1/3 | 70.3s |