Puzzle Solving x Wrong answer Ranking

See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster.

Models Shown

Total Failures

352

Most Affected Model

Failure Reasons

Wrong answer352 Did not follow instructions166 API error35 Extra formatting16 No answer14 Timed out11

Categories

Domain specific893 Anti-AI Tricks519 Coding484 Puzzle Solving352 Trivia315 General Intelligence134 Agentic128 Combined128 Instructions following117 Data parsing and extraction93

246/246

Rank	Model	Company	Wrong answer Count	Category Score	Total Cost	Tests Correct	Response Time (avg)
#441	Decider Flash V1.2 default	Perplexity	1	1.0	$0.001	0/1	424ms
Total Tests 1 Wrong Tests 1 Total Cost $0.001 Response Time (avg) 424ms
#443	D1 default	Liquid	1	0.2	$0.003	0/1	570ms
Total Tests 1 Wrong Tests 1 Total Cost $0.003 Response Time (avg) 570ms
#446	Solar Decide Flash default	Upstage	1	1.0	$0.003	0/1	1.66s
Total Tests 1 Wrong Tests 1 Total Cost $0.003 Response Time (avg) 1.66s
#447	Microsoft-Decision-1 default	Microsoft	1	0.2	$0.002	0/1	669ms
Total Tests 1 Wrong Tests 1 Total Cost $0.002 Response Time (avg) 669ms
#448	Solar Decide default	Upstage	1	1.0	$0.009	0/1	3.47s
Total Tests 1 Wrong Tests 1 Total Cost $0.009 Response Time (avg) 3.47s
#449	Tev1 4B Experimental default	Togethercomputer	1	0.2	$0.002	0/1	432ms
Total Tests 1 Wrong Tests 1 Total Cost $0.002 Response Time (avg) 432ms

Filter models