AI BENCHY Category Failures
Puzzle Solving: Wrong answer
Puzzle Solving
Wrong answer
See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster.
Failure Reasons
| Rank | Model | Company | Wrong answer Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #108 | Qwen3.5-Flash none | Qwen | 3 | 3.1 | 0/3 | 10.9s |
| #135 | Kimi K2.5 none | Moonshot AI | 3 | 3.0 | 0/3 | 4.04s |
| #140 | Qwen3 Coder Next none | Qwen | 3 | 3.0 | 0/3 | 24.3s |
| #155 | Mercury 2 none | Inception | 3 | 3.1 | 0/3 | 535ms |
| #157 | Grok 4.1 Fast none | X AI | 3 | 3.0 | 0/3 | 1.10s |
| #22 | Step 3.7 Flash medium | Stepfun | 2 | 5.7 | 1/3 | 6.19s |
| #41 | Nemotron 3 Ultra 550b A55b medium | NVIDIA | 2 | 5.5 | 1/3 | 3.54s |
| #57 | Step 3.7 Flash low | Stepfun | 2 | 5.5 | 1/3 | 1.84s |
| #70 | GPT-5.4 Nano medium | OpenAI | 2 | 4.1 | 0/3 | 3.79s |
| #71 | Step 3.7 Flash high | Stepfun | 2 | 5.3 | 1/3 | 10.2s |
| #105 | Nemotron 3 Super medium | NVIDIA | 2 | 3.0 | 0/3 | 3.15s |
| #110 | Seed-2.0-Lite none | Bytedance Seed | 2 | 5.3 | 1/3 | 2.78s |
| #114 | Qwen3.5 Plus 2026-04-20 none | Qwen | 2 | 6.7 | 1/3 | 1.97s |
| #117 | Qwen3.5-35B-A3B none | Qwen | 2 | 3.7 | 0/3 | 1.35s |
| #119 | Cobuddy medium | Baidu | 2 | 3.6 | 0/3 | 12.8s |