Puzzle Solving x Wrong answer Ranking

See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster. Sort by: Response Time (avg) ↓.

Models Shown

Total Failures

352

Most Affected Model

Failure Reasons

Wrong answer352 Did not follow instructions166 API error35 Extra formatting16 No answer14 Timed out11

Categories

Domain specific893 Anti-AI Tricks519 Coding484 Puzzle Solving352 Trivia315 General Intelligence134 Agentic128 Combined128 Instructions following117 Data parsing and extraction93

246/246

Rank	Model	Company	Wrong answer Count	Category Score	Total Cost	Tests Correct	Response Time (avg)
#397	Mercury 2.5 none	Inception	3	0.4	$0.026	0/3	515ms
Total Tests 3 Wrong Tests 3 Total Cost $0.026 Response Time (avg) 515ms
#416	Grok 4.20 none	X AI	2	0.5	$0.057	1/3	473ms
Total Tests 3 Wrong Tests 2 Total Cost $0.057 Response Time (avg) 473ms
#449	Tev1 4B Experimental default	Togethercomputer	1	0.2	$0.002	0/1	432ms
Total Tests 1 Wrong Tests 1 Total Cost $0.002 Response Time (avg) 432ms
#441	Decider Flash V1.2 default	Perplexity	1	1.0	$0.001	0/1	424ms
Total Tests 1 Wrong Tests 1 Total Cost $0.001 Response Time (avg) 424ms
#434	Decider V1.1 27B default	Perplexity	1	0.2	$0.002	0/1	422ms
Total Tests 1 Wrong Tests 1 Total Cost $0.002 Response Time (avg) 422ms
#344	Mistral Small 4 none	Mistral	2	0.3	$0.047	0/3	399ms
Total Tests 3 Wrong Tests 3 Total Cost $0.047 Response Time (avg) 399ms

Filter models