Kushindwa kwa kategoria za AI BENCHY
Utatuzi wa mafumbo: Hakufuata maelekezo
Utatuzi wa mafumbo
Hakufuata maelekezo
Ona ni modeli gani za AI zina uwezekano mkubwa wa kupata Hakufuata maelekezo katika Utatuzi wa mafumbo, ili uone udhaifu haraka.
Sababu za kushindwa
| Nafasi | Modeli | Kampuni | Idadi ya Hakufuata maelekezo | Alama ya kategoria | Majaribio sahihi | Muda wa majibu (wastani) |
|---|---|---|---|---|---|---|
| #139 | DeepSeek V4 Flash none | DeepSeek | 1 | 3.1 | 0/3 | 23.7s |
| #141 | Nemotron 3 Super none | NVIDIA | 1 | 5.5 | 1/3 | 2.36s |
| #142 | Mistral Small 4 none | Mistral | 1 | 3.1 | 0/3 | 399ms |
| #143 | MiMo-V2.5 none | Xiaomi | 1 | 5.4 | 1/3 | 2.13s |
| #144 | GPT-5.4 Mini none | OpenAI | 1 | 5.4 | 1/3 | 836ms |
| #147 | GPT-4o-mini none | OpenAI | 1 | 3.5 | 0/3 | 1.21s |
| #148 | GPT-5.4 Nano none | OpenAI | 1 | 5.4 | 1/3 | 1.25s |
| #150 | Qwen3 Coder Next medium | Qwen | 1 | 3.0 | 0/3 | 1.25s |
| #151 | Trinity Large Preview none | Arcee AI | 1 | 3.6 | 0/3 | 1.97s |
| #154 | Qwen3.5-9B none | Qwen | 1 | 3.2 | 0/3 | 621ms |
| #156 | Hy3 preview none | Tencent | 1 | 3.1 | 0/3 | 4.56s |
| #158 | GLM 4.7 Flash medium | Z.ai | 1 | 2.9 | 0/3 | 12.9s |
| #159 | Ling-2.6-1T none | Inclusionai | 1 | 3.1 | 0/3 | 5.36s |
| #162 | Nemotron 3 Nano Omni 30b A3b Reasoning none | NVIDIA | 1 | 3.0 | 0/3 | 532ms |
| #163 | Granite 4.1 8B none | IBM Granite | 1 | 3.2 | 0/3 | 608ms |