AI BENCHY Category Failures
Puzzle Solving
Wrong answer
Puzzle Solving
Wrong answer
See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster. Sort by: Failure Count ↑.
Related Failure Reasons
| Rank | Model | Company | Wrong answer Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #8 | Gemini 3.1 Flash Lite Preview high | 1 | 7.0 | 2/3 | 46.3s | |
| #12 | Gemini 3.1 Flash Lite Preview medium | 1 | 7.0 | 2/3 | 3.58s | |
| #13 | Step 3.5 Flash medium | Stepfun | 1 | 4.0 | 1/3 | 7.72s |
| #15 | GPT-5.2 Chat none | OpenAI | 1 | 7.0 | 2/3 | 4.42s |
| #16 | Gemini 2.5 Flash medium | 1 | 7.0 | 2/3 | 3.94s | |
| #18 | DeepSeek V3.2 medium | DeepSeek | 1 | 7.0 | 2/3 | 36.9s |
| #20 | Gemini 3 Flash Preview none | 1 | 7.0 | 2/3 | 1.06s | |
| #21 | MiMo-V2-Flash medium | Xiaomi | 1 | 7.0 | 2/3 | 3.77s |
| #23 | Seed-2.0-Mini medium | Bytedance Seed | 1 | 7.0 | 2/3 | 25.9s |
| #26 | Claude Opus 4.6 medium | Anthropic | 1 | 7.0 | 2/3 | 4.60s |
| #28 | Kimi K2.5 medium | Moonshot AI | 1 | 4.0 | 1/3 | 45.4s |
| #29 | Qwen3.5 Plus 2026-02-15 none | Qwen | 1 | 7.0 | 2/3 | 2.82s |
| #30 | Grok 4.1 Fast medium | X AI | 1 | 4.0 | 1/3 | 8.08s |
| #31 | GLM 5 none | Z.ai | 1 | 7.0 | 2/3 | 2.05s |
| #32 | GPT-5 Mini medium | OpenAI | 1 | 4.3 | 1/3 | 14.1s |
| #33 | DeepSeek V3.2 none | DeepSeek | 1 | 7.7 | 2/3 | 7.37s |
| #34 | GPT-5 Nano medium | OpenAI | 1 | 4.0 | 1/3 | 19.8s |
| #35 | Qwen3.5-35B-A3B medium | Qwen | 1 | 4.0 | 1/3 | 31.6s |
| #36 | Mercury 2 medium | Inception | 1 | 1.7 | 0/3 | 934ms |
| #38 | Gemini 2.5 Flash none | 1 | 4.7 | 1/3 | 576ms | |
| #39 | gpt-oss-120b medium | OpenAI | 1 | 1.7 | 0/3 | 11.8s |
| #41 | Qwen3.5-27B none | Qwen | 1 | 6.3 | 1/3 | 1.37s |
| #43 | MiniMax M2.5 medium | Minimax | 1 | 4.0 | 1/3 | 11.5s |
| #44 | GPT-5.4 none | OpenAI | 1 | 4.0 | 1/3 | 1.52s |
| #49 | GLM 4.7 Flash none | Z.ai | 1 | 3.7 | 0/3 | 1.00s |
| #50 | Qwen3 Coder Next medium | Qwen | 1 | 10.0 | 0/3 | 2.30s |
| #55 | LFM2-24B-A2B none | Liquid | 1 | 3.3 | 0/3 | 1.69s |
| #37 | Qwen3.5-Flash none | Qwen | 2 | 1.3 | 0/3 | 5.90s |
| #40 | Qwen3.5-122B-A10B none | Qwen | 2 | 4.0 | 1/3 | 982ms |
| #42 | Qwen3.5-35B-A3B none | Qwen | 2 | 1.7 | 0/3 | 1.34s |
| #45 | Trinity Large Preview none | Arcee AI | 2 | 4.0 | 1/3 | 3.30s |
| #52 | GLM 4.7 Flash medium | Z.ai | 2 | 10.0 | 0/3 | 12.9s |
| #46 | Kimi K2.5 none | Moonshot AI | 3 | 10.0 | 0/3 | 4.73s |
| #47 | GPT-4o-mini none | OpenAI | 3 | 2.3 | 0/3 | 1.30s |
| #48 | Qwen3 Coder Next none | Qwen | 3 | 1.3 | 0/3 | 22.9s |
| #51 | Mercury 2 none | Inception | 3 | 10.0 | 0/3 | 533ms |
| #53 | Grok 4.1 Fast none | X AI | 3 | 1.3 | 0/3 | 1.28s |
| #54 | MiMo-V2-Flash none | Xiaomi | 3 | 10.0 | 0/3 | 1.38s |