Coding x Wrong answer Ranking

See which AI models are most likely to hit Wrong answer on Coding, so you can spot weak points faster. Sort by: Response Time (avg) ↓.

Models Shown

Total Failures

456

Most Affected Model

Failure Reasons

Wrong answer456 API error64 Did not follow instructions41 No answer41 Extra formatting36 Timed out24

Categories

Domain specific811 Anti-AI Tricks470 Coding456 Puzzle Solving323 Trivia289 Combined121 General Intelligence121 Agentic120 Instructions following101 Data parsing and extraction77

265/265

Rank	Model	Company	Wrong answer Count	Category Score	Total Cost	Tests Correct	Response Time (avg)
#322	Inkling Small none	Thinkingmachines	3	0.4	$0.054	0/3	759ms
Total Tests 3 Wrong Tests 3 Total Cost $0.054 Response Time (avg) 759ms
#293	MiMo-V2.6-Flash none	Xiaomi	2	0.4	$0.060	0/3	744ms
Total Tests 3 Wrong Tests 3 Total Cost $0.060 Response Time (avg) 744ms
#284	Gemini 2.5 Flash none	Google	2	0.5	$0.017	1/3	736ms
Total Tests 3 Wrong Tests 2 Total Cost $0.017 Response Time (avg) 736ms
#301	Laguna XS 2.1 none	Poolside	3	0.4	$0.020	0/3	623ms
Total Tests 3 Wrong Tests 3 Total Cost $0.020 Response Time (avg) 623ms
#394	Jev 1.13 default	Typesafe	2	0.2	$0.003	0/2	598ms
Total Tests 2 Wrong Tests 2 Total Cost $0.003 Response Time (avg) 598ms
#336	Laguna S 2.1 none	Poolside	3	0.3	$0.044	0/3	550ms
Total Tests 3 Wrong Tests 3 Total Cost $0.044 Response Time (avg) 550ms
#333	Mercury 2.5 Preview default	Inception	3	0.5	$0.023	0/3	459ms
Total Tests 3 Wrong Tests 3 Total Cost $0.023 Response Time (avg) 459ms
#399	D1 default	Liquid	2	0.2	$0.003	0/2	456ms
Total Tests 2 Wrong Tests 2 Total Cost $0.003 Response Time (avg) 456ms
#396	GPT-6 Luna Decisions default	OpenAI	2	0.3	$0.005	0/2	447ms
Total Tests 2 Wrong Tests 2 Total Cost $0.005 Response Time (avg) 447ms
#395	Decider V1.1 27B default	Perplexity	1	0.5	$0.002	1/2	305ms
Total Tests 2 Wrong Tests 1 Total Cost $0.002 Response Time (avg) 305ms

Filter models