AI BENCHY Category Failures
Puzzle Solving
Wrong answer
Puzzle Solving
Wrong answer
See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster. Sort by: Response Time (avg) ↑.
Related Failure Reasons
| Rank | Model | Company | Wrong answer Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #51 | Mercury 2 none | Inception | 3 | 10.0 | 0/3 | 533ms |
| #38 | Gemini 2.5 Flash none | 1 | 4.7 | 1/3 | 576ms | |
| #36 | Mercury 2 medium | Inception | 1 | 1.7 | 0/3 | 934ms |
| #40 | Qwen3.5-122B-A10B none | Qwen | 2 | 4.0 | 1/3 | 982ms |
| #49 | GLM 4.7 Flash none | Z.ai | 1 | 3.7 | 0/3 | 1.00s |
| #20 | Gemini 3 Flash Preview none | 1 | 7.0 | 2/3 | 1.06s | |
| #53 | Grok 4.1 Fast none | X AI | 3 | 1.3 | 0/3 | 1.28s |
| #47 | GPT-4o-mini none | OpenAI | 3 | 2.3 | 0/3 | 1.30s |
| #42 | Qwen3.5-35B-A3B none | Qwen | 2 | 1.7 | 0/3 | 1.34s |
| #41 | Qwen3.5-27B none | Qwen | 1 | 6.3 | 1/3 | 1.37s |
| #54 | MiMo-V2-Flash none | Xiaomi | 3 | 10.0 | 0/3 | 1.38s |
| #44 | GPT-5.4 none | OpenAI | 1 | 4.0 | 1/3 | 1.52s |
| #55 | LFM2-24B-A2B none | Liquid | 1 | 3.3 | 0/3 | 1.69s |
| #31 | GLM 5 none | Z.ai | 1 | 7.0 | 2/3 | 2.05s |
| #50 | Qwen3 Coder Next medium | Qwen | 1 | 10.0 | 0/3 | 2.30s |
| #29 | Qwen3.5 Plus 2026-02-15 none | Qwen | 1 | 7.0 | 2/3 | 2.82s |
| #45 | Trinity Large Preview none | Arcee AI | 2 | 4.0 | 1/3 | 3.30s |
| #12 | Gemini 3.1 Flash Lite Preview medium | 1 | 7.0 | 2/3 | 3.58s | |
| #21 | MiMo-V2-Flash medium | Xiaomi | 1 | 7.0 | 2/3 | 3.77s |
| #16 | Gemini 2.5 Flash medium | 1 | 7.0 | 2/3 | 3.94s | |
| #15 | GPT-5.2 Chat none | OpenAI | 1 | 7.0 | 2/3 | 4.42s |
| #26 | Claude Opus 4.6 medium | Anthropic | 1 | 7.0 | 2/3 | 4.60s |
| #46 | Kimi K2.5 none | Moonshot AI | 3 | 10.0 | 0/3 | 4.73s |
| #37 | Qwen3.5-Flash none | Qwen | 2 | 1.3 | 0/3 | 5.90s |
| #33 | DeepSeek V3.2 none | DeepSeek | 1 | 7.7 | 2/3 | 7.37s |
| #13 | Step 3.5 Flash medium | Stepfun | 1 | 4.0 | 1/3 | 7.72s |
| #30 | Grok 4.1 Fast medium | X AI | 1 | 4.0 | 1/3 | 8.08s |
| #43 | MiniMax M2.5 medium | Minimax | 1 | 4.0 | 1/3 | 11.5s |
| #39 | gpt-oss-120b medium | OpenAI | 1 | 1.7 | 0/3 | 11.8s |
| #52 | GLM 4.7 Flash medium | Z.ai | 2 | 10.0 | 0/3 | 12.9s |
| #32 | GPT-5 Mini medium | OpenAI | 1 | 4.3 | 1/3 | 14.1s |
| #34 | GPT-5 Nano medium | OpenAI | 1 | 4.0 | 1/3 | 19.8s |
| #48 | Qwen3 Coder Next none | Qwen | 3 | 1.3 | 0/3 | 22.9s |
| #23 | Seed-2.0-Mini medium | Bytedance Seed | 1 | 7.0 | 2/3 | 25.9s |
| #35 | Qwen3.5-35B-A3B medium | Qwen | 1 | 4.0 | 1/3 | 31.6s |
| #18 | DeepSeek V3.2 medium | DeepSeek | 1 | 7.0 | 2/3 | 36.9s |
| #28 | Kimi K2.5 medium | Moonshot AI | 1 | 4.0 | 1/3 | 45.4s |
| #8 | Gemini 3.1 Flash Lite Preview high | 1 | 7.0 | 2/3 | 46.3s |