Puzzle Solving x Wrong answer Ranking

See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster. Sort by: Tests Correct ↓.

Models Shown

Total Failures

352

Most Affected Model

Failure Reasons

Wrong answer352 Did not follow instructions166 API error35 Extra formatting16 No answer14 Timed out11

Categories

Domain specific893 Anti-AI Tricks519 Coding484 Puzzle Solving352 Trivia315 General Intelligence134 Agentic128 Combined128 Instructions following117 Data parsing and extraction93

246/246

Rank	Model	Company	Wrong answer Count	Category Score	Total Cost	Tests Correct	Response Time (avg)
#443	D1 default	Liquid	1	0.2	$0.003	0/1	570ms
Total Tests 1 Wrong Tests 1 Total Cost $0.003 Response Time (avg) 570ms
#445	LFM2-24B-A2B default	Liquid	2	0.4	$0.001	0/3	1.78s
Total Tests 3 Wrong Tests 3 Total Cost $0.001 Response Time (avg) 1.78s
#446	Solar Decide Flash default	Upstage	1	1.0	$0.003	0/1	1.66s
Total Tests 1 Wrong Tests 1 Total Cost $0.003 Response Time (avg) 1.66s
#447	Microsoft-Decision-1 default	Microsoft	1	0.2	$0.002	0/1	669ms
Total Tests 1 Wrong Tests 1 Total Cost $0.002 Response Time (avg) 669ms
#448	Solar Decide default	Upstage	1	1.0	$0.009	0/1	3.47s
Total Tests 1 Wrong Tests 1 Total Cost $0.009 Response Time (avg) 3.47s
#449	Tev1 4B Experimental default	Togethercomputer	1	0.2	$0.002	0/1	432ms
Total Tests 1 Wrong Tests 1 Total Cost $0.002 Response Time (avg) 432ms

Filter models