Anti-AI Tricks x Wrong answer Ranking

AI BENCHY Category Failures

See which AI models are most likely to hit Wrong answer on Anti-AI Tricks, so you can spot weak points faster. Sort by: Response Time (avg) ↓.

Models Shown

Total Failures

245

Most Affected Model

Failure Reasons

Wrong answer245 Did not follow instructions30 Extra formatting14 API error13 Timed out4 No answer2

Categories

Domain specific314 Anti-AI Tricks245 Coding194 Puzzle Solving147 Trivia130 Instructions following53 Combined52 Data parsing and extraction35 General Intelligence32 Tool Calling2

Rank	Model	Company	Wrong answer Count	Category Score	Tests Correct	Response Time (avg)
#81	Mercury 2 medium	Inception	1	6.9	2/4	1.12s
#87	Gemini 3.1 Flash Lite minimal	Google	1	8.3	3/4	1.10s
#34	Qwen3.7 Max none	Qwen	2	6.5	2/4	1.08s
#157	Grok 4.1 Fast none	X AI	3	3.2	0/4	1.07s
#90	Gemini 3.1 Flash Lite none	Google	2	7.5	2/4	1.07s
#58	Gemini 3.1 Flash Lite Preview none	Google	1	7.5	2/4	1.04s
#137	Elephant Alpha none	Openrouter	1	6.6	2/4	963ms
#144	GPT-5.4 Mini none	OpenAI	4	3.1	0/4	929ms
#32	Gemini 3.5 Flash minimal	Google	2	6.5	2/4	892ms
#163	Granite 4.1 8B none	IBM Granite	2	4.9	1/4	844ms
#115	Qwen3.5-27B none	Qwen	3	4.8	1/4	788ms
#145	Laguna M.1 none	Poolside	3	3.4	0/4	705ms
#106	Grok 4.20 Beta none	X AI	4	4.0	0/4	597ms
#162	Nemotron 3 Nano Omni 30b A3b Reasoning none	NVIDIA	2	4.8	1/4	584ms
#97	Gemini 2.5 Flash none	Google	4	3.0	0/4	582ms

←

1 6 7 8

→

Top Models by Wrong answer Count