AI BENCHY Category Failures
Puzzle Solving
Wrong answer
Puzzle Solving
Wrong answer
See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster. Sort by: Tests Correct ↓.
Related Failure Reasons
| Rank | Model | Company | Wrong answer Count | Category Score | Tests Correct | Response Time (avg) |
|---|---|---|---|---|---|---|
| #8 | Gemini 3.1 Flash Lite Preview high | 1 | 7.0 | 2/3 | 46.3s | |
| #12 | Gemini 3.1 Flash Lite Preview medium | 1 | 7.0 | 2/3 | 3.58s | |
| #15 | GPT-5.2 Chat none | OpenAI | 1 | 7.0 | 2/3 | 4.42s |
| #16 | Gemini 2.5 Flash medium | 1 | 7.0 | 2/3 | 3.94s | |
| #18 | DeepSeek V3.2 medium | DeepSeek | 1 | 7.0 | 2/3 | 36.9s |
| #20 | Gemini 3 Flash Preview none | 1 | 7.0 | 2/3 | 1.06s | |
| #21 | MiMo-V2-Flash medium | Xiaomi | 1 | 7.0 | 2/3 | 3.77s |
| #23 | Seed-2.0-Mini medium | Bytedance Seed | 1 | 7.0 | 2/3 | 25.9s |
| #26 | Claude Opus 4.6 medium | Anthropic | 1 | 7.0 | 2/3 | 4.60s |
| #29 | Qwen3.5 Plus 2026-02-15 none | Qwen | 1 | 7.0 | 2/3 | 2.82s |
| #31 | GLM 5 none | Z.ai | 1 | 7.0 | 2/3 | 2.05s |
| #33 | DeepSeek V3.2 none | DeepSeek | 1 | 7.7 | 2/3 | 7.37s |
| #13 | Step 3.5 Flash medium | Stepfun | 1 | 4.0 | 1/3 | 7.72s |
| #28 | Kimi K2.5 medium | Moonshot AI | 1 | 4.0 | 1/3 | 45.4s |
| #30 | Grok 4.1 Fast medium | X AI | 1 | 4.0 | 1/3 | 8.08s |
| #32 | GPT-5 Mini medium | OpenAI | 1 | 4.3 | 1/3 | 14.1s |
| #34 | GPT-5 Nano medium | OpenAI | 1 | 4.0 | 1/3 | 19.8s |
| #35 | Qwen3.5-35B-A3B medium | Qwen | 1 | 4.0 | 1/3 | 31.6s |
| #38 | Gemini 2.5 Flash none | 1 | 4.7 | 1/3 | 576ms | |
| #40 | Qwen3.5-122B-A10B none | Qwen | 2 | 4.0 | 1/3 | 982ms |
| #41 | Qwen3.5-27B none | Qwen | 1 | 6.3 | 1/3 | 1.37s |
| #43 | MiniMax M2.5 medium | Minimax | 1 | 4.0 | 1/3 | 11.5s |
| #44 | GPT-5.4 none | OpenAI | 1 | 4.0 | 1/3 | 1.52s |
| #45 | Trinity Large Preview none | Arcee AI | 2 | 4.0 | 1/3 | 3.30s |
| #36 | Mercury 2 medium | Inception | 1 | 1.7 | 0/3 | 934ms |
| #37 | Qwen3.5-Flash none | Qwen | 2 | 1.3 | 0/3 | 5.90s |
| #39 | gpt-oss-120b medium | OpenAI | 1 | 1.7 | 0/3 | 11.8s |
| #42 | Qwen3.5-35B-A3B none | Qwen | 2 | 1.7 | 0/3 | 1.34s |
| #46 | Kimi K2.5 none | Moonshot AI | 3 | 10.0 | 0/3 | 4.73s |
| #47 | GPT-4o-mini none | OpenAI | 3 | 2.3 | 0/3 | 1.30s |
| #48 | Qwen3 Coder Next none | Qwen | 3 | 1.3 | 0/3 | 22.9s |
| #49 | GLM 4.7 Flash none | Z.ai | 1 | 3.7 | 0/3 | 1.00s |
| #50 | Qwen3 Coder Next medium | Qwen | 1 | 10.0 | 0/3 | 2.30s |
| #51 | Mercury 2 none | Inception | 3 | 10.0 | 0/3 | 533ms |
| #52 | GLM 4.7 Flash medium | Z.ai | 2 | 10.0 | 0/3 | 12.9s |
| #53 | Grok 4.1 Fast none | X AI | 3 | 1.3 | 0/3 | 1.28s |
| #54 | MiMo-V2-Flash none | Xiaomi | 3 | 10.0 | 0/3 | 1.38s |
| #55 | LFM2-24B-A2B none | Liquid | 1 | 3.3 | 0/3 | 1.69s |