Kushindwa kwa kategoria za AI BENCHY
Utatuzi wa mafumbo: Hakufuata maelekezo
Utatuzi wa mafumbo
Hakufuata maelekezo
Ona ni modeli gani za AI zina uwezekano mkubwa wa kupata Hakufuata maelekezo katika Utatuzi wa mafumbo, ili uone udhaifu haraka.
Sababu za kushindwa
| Nafasi | Modeli | Kampuni | Idadi ya Hakufuata maelekezo | Alama ya kategoria | Majaribio sahihi | Muda wa majibu (wastani) |
|---|---|---|---|---|---|---|
| #90 | Gemini 3.1 Flash Lite none | 1 | 6.3 | 1/3 | 720ms | |
| #94 | GPT-5 Nano medium | OpenAI | 1 | 5.3 | 1/3 | 20.6s |
| #96 | Ring-2.6-1T none | Inclusionai | 1 | 7.7 | 2/3 | 31.5s |
| #99 | gpt-oss-120b medium | OpenAI | 1 | 5.3 | 1/3 | 21.7s |
| #100 | Grok Build 0.1 none | X AI | 1 | 6.4 | 1/3 | 9.55s |
| #102 | Gemma 4 26B A4B none | 1 | 6.2 | 1/3 | 744ms | |
| #104 | Nemotron 3 Ultra 550b A55b none | NVIDIA | 1 | 5.9 | 1/3 | 1.06s |
| #105 | Nemotron 3 Super medium | NVIDIA | 1 | 3.0 | 0/3 | 3.15s |
| #109 | GLM 5V Turbo none | Z.ai | 1 | 5.3 | 1/3 | 2.40s |
| #111 | Owl Alpha medium | Openrouter | 1 | 5.3 | 1/3 | 3.40s |
| #115 | Qwen3.5-27B none | Qwen | 1 | 6.7 | 1/3 | 1.38s |
| #116 | Hunter Alpha none | OpenRouter | 1 | 5.8 | 1/3 | 3.71s |
| #117 | Qwen3.5-35B-A3B none | Qwen | 1 | 3.7 | 0/3 | 1.35s |
| #118 | Qwen3.6 27B none | Qwen | 1 | 5.3 | 1/3 | 5.15s |
| #119 | Cobuddy medium | Baidu | 1 | 3.6 | 0/3 | 12.8s |