Puzzle Solving x Wrong answer Ranking

See which AI models are most likely to hit Wrong answer on Puzzle Solving, so you can spot weak points faster. Sort by: Response Time (avg) ↓.

Models Shown

Total Failures

352

Most Affected Model

Xing4.0 29b A4b 1

Failure Reasons

Wrong answer352 Did not follow instructions166 API error35 Extra formatting16 No answer14 Timed out11

Categories

Domain specific893 Anti-AI Tricks519 Coding484 Puzzle Solving352 Trivia315 General Intelligence134 Agentic128 Combined128 Instructions following117 Data parsing and extraction93

246/246

Rank	Model	Company	Wrong answer Count	Category Score	Total Cost	Tests Correct	Response Time (avg)
#298	DeepSeek V4.1 Flash none	DeepSeek	2	0.5	$0.391	1/3	1.49s
Total Tests 3 Wrong Tests 2 Total Cost $0.391 Response Time (avg) 1.49s
#234	GLM 5.1 none	Z.ai	1	0.8	$0.451	2/3	1.45s
Total Tests 3 Wrong Tests 1 Total Cost $0.451 Response Time (avg) 1.45s
#309	GPT-5.4 none	OpenAI	1	0.6	$0.690	1/3	1.44s
Total Tests 3 Wrong Tests 2 Total Cost $0.690 Response Time (avg) 1.44s
#205	GPT-6 Sol none	OpenAI	1	0.8	$0.551	2/3	1.40s
Total Tests 3 Wrong Tests 1 Total Cost $0.551 Response Time (avg) 1.40s
#316	Kimi K2.6 none	Moonshot AI	2	0.3	$0.486	0/3	1.40s
Total Tests 3 Wrong Tests 3 Total Cost $0.486 Response Time (avg) 1.40s
#429	Nemotron 3 Nano Omni 30b A3b Reasoning medium	NVIDIA	2	0.3	$0.000	0/3	1.40s
Total Tests 3 Wrong Tests 3 Total Cost $0.000 Response Time (avg) 1.40s
#294	Qwen3.7 Flash none	Qwen	2	0.4	$0.028	0/3	1.39s
Total Tests 3 Wrong Tests 3 Total Cost $0.028 Response Time (avg) 1.39s
#231	Mercury 2.5 high	Inception	1	0.9	$0.044	2/3	1.39s
Total Tests 3 Wrong Tests 1 Total Cost $0.044 Response Time (avg) 1.39s
#168	Qwen3.5-27B none	Qwen	1	0.7	$0.116	1/3	1.38s
Total Tests 3 Wrong Tests 2 Total Cost $0.116 Response Time (avg) 1.38s
#439	Classify-1 default	Scaledown	1	1.0	$0.008	0/1	1.37s
Total Tests 1 Wrong Tests 1 Total Cost $0.008 Response Time (avg) 1.37s
#313	Qwen3.5-35B-A3B none	Qwen	2	0.4	$0.120	0/3	1.35s
Total Tests 3 Wrong Tests 3 Total Cost $0.120 Response Time (avg) 1.35s
#347	MiMo-V2.5-Pro none	Xiaomi	1	0.7	$0.135	1/3	1.30s
Total Tests 3 Wrong Tests 2 Total Cost $0.135 Response Time (avg) 1.30s
#295	Qwen3.8 Flash Next none	Qwen	2	0.6	~$0.011	1/3	1.30s
Total Tests 3 Wrong Tests 2 Total Cost ~$0.011 Response Time (avg) 1.30s
#219	GPT-5.5 none	OpenAI	1	0.8	$1.212	2/3	1.29s
Total Tests 3 Wrong Tests 1 Total Cost $1.212 Response Time (avg) 1.29s
#324	Qwen3.8 27B none	Qwen	1	0.6	~$0.010	1/3	1.25s
Total Tests 3 Wrong Tests 2 Total Cost ~$0.010 Response Time (avg) 1.25s

Filter models

Top Models by Wrong answer Count

Wrong answer Count vs Score

Top Models by Response Time (avg)

Top Models by Estimated Wasted Cost

Puzzle Solving: Wrong answer

Filter models

Top Models by Wrong answer Count

Wrong answer Count vs Score

Top Models by Response Time (avg)

Top Models by Estimated Wasted CostEstimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.1095 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.010823.

Top Models by Estimated Wasted Cost