Wrong answer Failure Ranking

See which AI models run into Wrong answer most often, so you can spot reliability risks before choosing one. Sort by: Total Cost ↓.

Models Shown

Total Failures

3190

Most Affected Model

GPT-6 Astra 1

Categories

In category Domain specific900 In category Anti-AI Tricks521 In category Coding485 In category Puzzle Solving355 In category Trivia317 In category General Intelligence134 In category Combined130 In category Agentic128 In category Instructions following118 In category Data parsing and extraction93

447/447

Rank	Model	Company	Wrong answer Count	Score	Total Cost	Tests Correct	Response Time (avg)
#150	GLM 5.1 medium	Z.ai	5	0.8	$1.546	13/23	66.4s
Total Tests 23 Wrong Tests 10 Total Cost $1.546 Response Time (avg) 66.4s
#42	Pareto default	Unbiased	3	0.9	$1.520	19/23	33.5s
Total Tests 23 Wrong Tests 4 Total Cost $1.520 Response Time (avg) 33.5s
#75	Claude Sonnet 5.5 xhigh	Anthropic	3	0.9	$1.516	16/23	15.0s
Total Tests 23 Wrong Tests 7 Total Cost $1.516 Response Time (avg) 15.0s
#191	Ember-1 low	Fireworks	3	0.7	$1.488	13/23	34.9s
Total Tests 23 Wrong Tests 10 Total Cost $1.488 Response Time (avg) 34.9s
#227	Step 3.7 Flash high	Stepfun	8	0.7	$1.478	11/23	72.7s
Total Tests 23 Wrong Tests 12 Total Cost $1.478 Response Time (avg) 72.7s
#153	Seed-2.0-Code high	Bytedance Seed	6	0.7	$1.448	12/23	124.9s
Total Tests 23 Wrong Tests 11 Total Cost $1.448 Response Time (avg) 124.9s
#81	Claude Sonnet 5 medium	Anthropic	5	0.9	$1.438	16/23	15.0s
Total Tests 23 Wrong Tests 7 Total Cost $1.438 Response Time (avg) 15.0s
#212	Grok Build 0.1 medium	X AI	6	0.7	$1.419	13/23	55.1s
Total Tests 23 Wrong Tests 10 Total Cost $1.419 Response Time (avg) 55.1s
#152	Mistral Large 4 high	Mistral	6	0.8	$1.391	12/23	263.7s
Total Tests 23 Wrong Tests 11 Total Cost $1.391 Response Time (avg) 263.7s
#6	GPT-6.1 Sol xhigh	OpenAI	1	1.0	$1.377	22/23	33.1s
Total Tests 23 Wrong Tests 1 Total Cost $1.377 Response Time (avg) 33.1s
#135	Grok 4.5 low	X AI	7	0.8	$1.345	16/23	18.1s
Total Tests 23 Wrong Tests 7 Total Cost $1.345 Response Time (avg) 18.1s
#38	GPT-5.3-Codex medium	OpenAI	4	0.9	$1.318	17/23	18.9s
Total Tests 23 Wrong Tests 6 Total Cost $1.318 Response Time (avg) 18.9s
#124	Gemini 3.5 Flash medium	Google	3	0.8	$1.316	18/23	11.8s
Total Tests 23 Wrong Tests 5 Total Cost $1.316 Response Time (avg) 11.8s
#157	Seed-2.0-Code medium	Bytedance Seed	6	0.7	$1.275	11/23	100.6s
Total Tests 23 Wrong Tests 12 Total Cost $1.275 Response Time (avg) 100.6s
#63	Muse Spark Open medium	Meta	3	0.9	$1.260	15/23	77.4s
Total Tests 23 Wrong Tests 8 Total Cost $1.260 Response Time (avg) 77.4s

Wrong answer Failures

Filter models

Top Models by Wrong answer Count

Wrong answer Count vs Score

Top Models by Response Time (avg)