Puzzle Solving x Wrong answer Ranking

See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster.

Models Shown

Total Failures

275

Most Affected Model

Failure Reasons

Wrong answer275 Did not follow instructions128 API error15 Extra formatting14 No answer11 Timed out11

Categories

Domain specific654 Anti-AI Tricks386 Coding361 Puzzle Solving275 Trivia244 Combined95 General Intelligence90 Instructions following82 Data parsing and extraction66 Tool Calling5

186/186

Rank	Model	Company	Wrong answer Count	Category Score	Total Cost	Tests Correct	Response Time (avg)
#294	Laguna Xs.2 medium	Poolside	1	5.3	$0.015	1/3	1.93s
Total Tests 3 Wrong Tests 2 Total Cost $0.015 Response Time (avg) 1.93s
#300	Grok Build 0.1 none	X AI	1	6.4	$0.547	1/3	9.55s
Total Tests 3 Wrong Tests 2 Total Cost $0.547 Response Time (avg) 9.55s
#304	Qwen3.5-9B medium	Qwen	1	3.0	$0.062	0/3	32.3s
Total Tests 3 Wrong Tests 3 Total Cost $0.062 Response Time (avg) 32.3s
#306	Laguna Xs.2 none	Poolside	1	5.3	$0.004	1/3	650ms
Total Tests 3 Wrong Tests 2 Total Cost $0.004 Response Time (avg) 650ms
#307	gpt-oss-120b none	OpenAI	1	6.0	$0.010	1/3	8.21s
Total Tests 3 Wrong Tests 2 Total Cost $0.010 Response Time (avg) 8.21s
#309	Nemotron 3 Nano Omni 30b A3b Reasoning none	NVIDIA	1	3.0	$0.000	0/3	532ms
Total Tests 3 Wrong Tests 3 Total Cost $0.000 Response Time (avg) 532ms

Filter models