Wrong answer Failure Ranking

See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↓.

Models Shown

Total Failures

3190

Most Affected Model

GPT-6 Astra 1

Categories

In category Domain specific900 In category Anti-AI Tricks521 In category Coding485 In category Puzzle Solving355 In category Trivia317 In category General Intelligence134 In category Combined130 In category Agentic128 In category Instructions following118 In category Data parsing and extraction93

447/447

Rank	Model	Company	Wrong answer Count	Score	Total Cost	Tests Correct	Response Time (avg)
#376	DeepSeek V3.2 none	DeepSeek	9	0.5	$0.119	6/23	22.4s
Total Tests 23 Wrong Tests 17 Total Cost $0.119 Response Time (avg) 22.4s
#113	GPT-5.6 Luna high	OpenAI	8	0.8	$0.118	15/23	18.4s
Total Tests 23 Wrong Tests 8 Total Cost $0.118 Response Time (avg) 18.4s
#171	Qwen3.5-27B none	Qwen	11	0.7	$0.116	10/23	9.98s
Total Tests 23 Wrong Tests 13 Total Cost $0.116 Response Time (avg) 9.98s
#290	Seed-2.0-Lite none	Bytedance Seed	14	0.6	$0.114	8/23	7.49s
Total Tests 23 Wrong Tests 15 Total Cost $0.114 Response Time (avg) 7.49s
#183	GLM 5.3 FlashX medium	Z.ai	8	0.7	$0.114	12/23	7.37s
Total Tests 23 Wrong Tests 11 Total Cost $0.114 Response Time (avg) 7.37s
#289	Inkling Small medium	Thinkingmachines	6	0.6	$0.113	10/23	6.03s
Total Tests 23 Wrong Tests 13 Total Cost $0.113 Response Time (avg) 6.03s
#374	Granite 4.2 8B high	IBM Granite	7	0.5	$0.111	4/23	235.4s
Total Tests 23 Wrong Tests 19 Total Cost $0.111 Response Time (avg) 235.4s
#380	Laguna S 2.1 low	Poolside	16	0.5	$0.108	3/23	88.2s
Total Tests 23 Wrong Tests 20 Total Cost $0.108 Response Time (avg) 88.2s
#402	Qwen3.5-9B medium	Qwen	2	0.4	$0.106	3/23	110.2s
Total Tests 23 Wrong Tests 20 Total Cost $0.106 Response Time (avg) 110.2s
#331	Step 3.5 Flash medium	Stepfun	6	0.5	$0.105	10/22	190.5s
Total Tests 22 Wrong Tests 12 Total Cost $0.105 Response Time (avg) 190.5s
#339	Qwen3.6 35B A3B none	Qwen	13	0.5	$0.105	4/23	8.43s
Total Tests 23 Wrong Tests 19 Total Cost $0.105 Response Time (avg) 8.43s
#254	LongCat 2.0 none	Meituan	13	0.6	$0.104	8/23	8.50s
Total Tests 23 Wrong Tests 15 Total Cost $0.104 Response Time (avg) 8.50s
#303	Ring 2.6 1t medium	Inclusionai	6	0.6	$0.102	11/23	65.5s
Total Tests 23 Wrong Tests 12 Total Cost $0.102 Response Time (avg) 65.5s
#163	Gemma 4 26B A4B medium	Google	2	0.7	$0.101	16/23	105.0s
Total Tests 23 Wrong Tests 7 Total Cost $0.101 Response Time (avg) 105.0s
#214	Solar Pro 4 low	Upstage	7	0.7	$0.101	11/23	145.2s
Total Tests 23 Wrong Tests 12 Total Cost $0.101 Response Time (avg) 145.2s

Wrong answer Failures

Filter models

Top Models by Wrong answer Count

Wrong answer Count vs Score

Top Models by Response Time (avg)