AI BENCHY Category Failures
Puzzle Solving
Wrong answer
Puzzle Solving
Wrong answer
See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster. Sort by: Response Time (avg) ↓.
Related Failure Reasons
| Rank | Model | Company | Wrong answer Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #8 | Gemini 3.1 Flash Lite Preview high | 1 | 7.0 | 2/3 | 46.3s | |
| #28 | Kimi K2.5 medium | Moonshot AI | 1 | 4.0 | 1/3 | 45.4s |
| #18 | DeepSeek V3.2 medium | DeepSeek | 1 | 7.0 | 2/3 | 36.9s |
| #35 | Qwen3.5-35B-A3B medium | Qwen | 1 | 4.0 | 1/3 | 31.6s |
| #23 | Seed-2.0-Mini medium | Bytedance Seed | 1 | 7.0 | 2/3 | 25.9s |
| #48 | Qwen3 Coder Next none | Qwen | 3 | 1.3 | 0/3 | 22.9s |
| #34 | GPT-5 Nano medium | OpenAI | 1 | 4.0 | 1/3 | 19.8s |
| #32 | GPT-5 Mini medium | OpenAI | 1 | 4.3 | 1/3 | 14.1s |
| #52 | GLM 4.7 Flash medium | Z.ai | 2 | 10.0 | 0/3 | 12.9s |
| #39 | gpt-oss-120b medium | OpenAI | 1 | 1.7 | 0/3 | 11.8s |
| #43 | MiniMax M2.5 medium | Minimax | 1 | 4.0 | 1/3 | 11.5s |
| #30 | Grok 4.1 Fast medium | X AI | 1 | 4.0 | 1/3 | 8.08s |
| #13 | Step 3.5 Flash medium | Stepfun | 1 | 4.0 | 1/3 | 7.72s |
| #33 | DeepSeek V3.2 none | DeepSeek | 1 | 7.7 | 2/3 | 7.37s |
| #37 | Qwen3.5-Flash none | Qwen | 2 | 1.3 | 0/3 | 5.90s |
| #46 | Kimi K2.5 none | Moonshot AI | 3 | 10.0 | 0/3 | 4.73s |
| #26 | Claude Opus 4.6 medium | Anthropic | 1 | 7.0 | 2/3 | 4.60s |
| #15 | GPT-5.2 Chat none | OpenAI | 1 | 7.0 | 2/3 | 4.42s |
| #16 | Gemini 2.5 Flash medium | 1 | 7.0 | 2/3 | 3.94s | |
| #21 | MiMo-V2-Flash medium | Xiaomi | 1 | 7.0 | 2/3 | 3.77s |
| #12 | Gemini 3.1 Flash Lite Preview medium | 1 | 7.0 | 2/3 | 3.58s | |
| #45 | Trinity Large Preview none | Arcee AI | 2 | 4.0 | 1/3 | 3.30s |
| #29 | Qwen3.5 Plus 2026-02-15 none | Qwen | 1 | 7.0 | 2/3 | 2.82s |
| #50 | Qwen3 Coder Next medium | Qwen | 1 | 10.0 | 0/3 | 2.30s |
| #31 | GLM 5 none | Z.ai | 1 | 7.0 | 2/3 | 2.05s |
| #55 | LFM2-24B-A2B none | Liquid | 1 | 3.3 | 0/3 | 1.69s |
| #44 | GPT-5.4 none | OpenAI | 1 | 4.0 | 1/3 | 1.52s |
| #54 | MiMo-V2-Flash none | Xiaomi | 3 | 10.0 | 0/3 | 1.38s |
| #41 | Qwen3.5-27B none | Qwen | 1 | 6.3 | 1/3 | 1.37s |
| #42 | Qwen3.5-35B-A3B none | Qwen | 2 | 1.7 | 0/3 | 1.34s |
| #47 | GPT-4o-mini none | OpenAI | 3 | 2.3 | 0/3 | 1.30s |
| #53 | Grok 4.1 Fast none | X AI | 3 | 1.3 | 0/3 | 1.28s |
| #20 | Gemini 3 Flash Preview none | 1 | 7.0 | 2/3 | 1.06s | |
| #49 | GLM 4.7 Flash none | Z.ai | 1 | 3.7 | 0/3 | 1.00s |
| #40 | Qwen3.5-122B-A10B none | Qwen | 2 | 4.0 | 1/3 | 982ms |
| #36 | Mercury 2 medium | Inception | 1 | 1.7 | 0/3 | 934ms |
| #38 | Gemini 2.5 Flash none | 1 | 4.7 | 1/3 | 576ms | |
| #51 | Mercury 2 none | Inception | 3 | 10.0 | 0/3 | 533ms |