AI Benchy Leaderboard

Name: AI BENCHY Model Benchmark Results
Creator: AI BENCHY
License: https://aibenchy.com/methodology/

Last updated at: 2026-10-17 Models Evaluated: 442

Recommended models

GPT-6.1 Sol xhigh Intelligent Attempt pass rate 97.1% Tests Correct 22 / 23 Score 9.9/10 Gemini 3.7 Flash low Fast Response Time (avg) 6.24s Total Output Tokens 38,453 Score 9.4/10 GPT-6 Luna high Affordable Input Price $0.100 / 1M Output Price $0.500 / 1M Cache Read Price $0.010 / 1M Cache Write Price $0.125 / 1M Score 7.7/10

442/442

Rank	Model	Score	Company	Total Cost	Response Time (avg)
#398#398	Xing4.0 29b A4bmedium	3.7	Xingchen AGI	$0.000	371.69s
View model card Total Tests 23 Wrong Tests 22 Reliability 7.3 Attempt pass rate 26.1% Flaky tests 11 Input Tokens 98,467 Output Tokens 3,385 Reasoning Tokens 104,595 Response Time (avg) 371.69s Response Time (total) 8548.88s Response Time (max) 971.85s Wrong answer: 9 Timed out: 6 API error: 4 Did not follow instructions: 3 Agentic : 3.0 Anti-AI Tricks : 3.8 Coding : 4.2 Combined : 3.0 Data parsing and extraction : 2.9 Domain specific : 3.6 General Intelligence : 3.2 Instructions following : 4.3 Puzzle Solving : 2.9 Tool Calling : 10.0 Trivia : 3.0
#146#146	Mistral Large 4high	7.5	Mistral	$1.391	263.69s
View model card Total Tests 23 Wrong Tests 11 Reliability 9.0 Attempt pass rate 65.2% Flaky tests 6 Input Tokens 230,895 Output Tokens 161,281 Reasoning Tokens 510,655 Response Time (avg) 263.69s Response Time (total) 6064.89s Response Time (max) 1727.29s Wrong answer: 6 Did not follow instructions: 2 No answer: 2 Timed out: 1 Agentic : 10.0 Anti-AI Tricks : 8.3 Coding : 3.3 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 3.5 General Intelligence : 10.0 Instructions following : 8.4 Puzzle Solving : 5.4 Tool Calling : 10.0 Trivia : 3.0
#362#362	Granite 4.2 8Bhigh	4.7	IBM Granite	$0.111	235.35s
View model card Total Tests 23 Wrong Tests 19 Reliability 8.2 Attempt pass rate 37.7% Flaky tests 10 Input Tokens 282,658 Output Tokens 198,563 Reasoning Tokens 462,804 Response Time (avg) 235.35s Response Time (total) 5413.09s Response Time (max) 685.87s Timed out: 10 Wrong answer: 7 Invalid tool call: 1 No answer: 1 Agentic : 5.0 Anti-AI Tricks : 6.9 Coding : 3.8 Combined : 5.0 Data parsing and extraction : 3.8 Domain specific : 2.9 General Intelligence : 5.5 Instructions following : 6.4 Puzzle Solving : 2.9 Tool Calling : 10.0 Trivia : 3.0
#317#317	Hy4 previewlow	5.4	Tencent	$1.844	207.21s
View model card Total Tests 23 Wrong Tests 20 Reliability 6.5 Attempt pass rate 21.7% Flaky tests 4 Input Tokens 162,180 Output Tokens 5,030 Reasoning Tokens 682,447 Response Time (avg) 207.21s Response Time (total) 4765.91s Response Time (max) 597.45s Wrong answer: 13 API error: 5 Did not follow instructions: 1 No answer: 1 Agentic : 4.7 Anti-AI Tricks : 6.0 Coding : 5.2 Combined : 2.9 Data parsing and extraction : 8.7 Domain specific : 4.5 General Intelligence : 7.5 Instructions following : 9.8 Puzzle Solving : 5.3 Tool Calling : 3.0 Trivia : 3.0
#319#319	Step 3.5 Flashmedium	5.4	Stepfun	$0.105	190.48s
View model card Total Tests 22 Wrong Tests 12 Reliability 10.0 Attempt pass rate 49.3% Flaky tests 3 Input Tokens 139,086 Output Tokens 193,584 Reasoning Tokens 413,546 Response Time (avg) 190.48s Response Time (total) 3238.23s Response Time (max) 1597.85s Wrong answer: 6 Did not follow instructions: 3 API error: 1 No answer: 1 Timed out: 1 Agentic : 2.8 Anti-AI Tricks : 10.0 Coding : 2.4 Combined : 6.5 Data parsing and extraction : 10.0 Domain specific : 3.5 General Intelligence : 5.5 Instructions following : 8.3 Puzzle Solving : 5.3 Tool Calling : 10.0 Trivia : 3.0
#68#68	Seed 2.1 Turbomedium	8.7	Bytedance Seed	$2.042	189.13s
View model card Total Tests 23 Wrong Tests 7 Reliability 10.0 Attempt pass rate 79.7% Flaky tests 5 Input Tokens 234,572 Output Tokens 8,019 Reasoning Tokens 761,563 Response Time (avg) 189.13s Response Time (total) 4349.92s Response Time (max) 885.33s Wrong answer: 5 Did not follow instructions: 1 No answer: 1 Agentic : 10.0 Anti-AI Tricks : 8.2 Coding : 8.2 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 6.1 Instructions following : 9.8 Puzzle Solving : 9.9 Tool Calling : 10.0 Trivia : 3.0
#36#36	Qwen3.8 Max (0902)high	9.1	Qwen	$3.306	186.06s
View model card Total Tests 23 Wrong Tests 5 Reliability 9.8 Attempt pass rate 87.0% Flaky tests 3 Input Tokens 328,637 Output Tokens 8,619 Reasoning Tokens 432,733 Response Time (avg) 186.06s Response Time (total) 4279.49s Response Time (max) 912.78s Wrong answer: 2 Did not follow instructions: 1 Invalid tool call: 1 Timed out: 1 Agentic : 10.0 Anti-AI Tricks : 10.0 Coding : 10.0 Combined : 8.7 Data parsing and extraction : 10.0 Domain specific : 5.9 General Intelligence : 6.5 Instructions following : 9.8 Puzzle Solving : 10.0 Tool Calling : 10.0 Trivia : 3.0
#42#42	Seed 2.1 Turbohigh	9.1	Bytedance Seed	$1.892	170.52s
View model card Total Tests 23 Wrong Tests 6 Reliability 10.0 Attempt pass rate 79.7% Flaky tests 3 Input Tokens 246,954 Output Tokens 8,378 Reasoning Tokens 698,942 Response Time (avg) 170.52s Response Time (total) 3922.04s Response Time (max) 584.71s Wrong answer: 4 Did not follow instructions: 1 No answer: 1 Agentic : 10.0 Anti-AI Tricks : 8.2 Coding : 10.0 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 6.5 Instructions following : 9.8 Puzzle Solving : 10.0 Tool Calling : 10.0 Trivia : 3.0
#66#66	Qwen3.8 27Bhigh	8.7	Qwen	~$0.298	166.34s
View model card Total Tests 23 Wrong Tests 6 Reliability 9.7 Attempt pass rate 78.3% Flaky tests 2 Input Tokens 348,967 Output Tokens 1,005 Reasoning Tokens 676,075 Response Time (avg) 166.34s Response Time (total) 3825.81s Response Time (max) 640.85s Did not follow instructions: 2 No answer: 2 Wrong answer: 2 Agentic : 10.0 Anti-AI Tricks : 10.0 Coding : 8.1 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 5.3 General Intelligence : 5.0 Instructions following : 9.8 Puzzle Solving : 7.6 Tool Calling : 10.0 Trivia : 3.0
#29#29	Seed 2.1 Turbolow	9.3	Bytedance Seed	$1.645	161.25s
View model card Total Tests 23 Wrong Tests 5 Reliability 10.0 Attempt pass rate 81.2% Flaky tests 2 Input Tokens 241,855 Output Tokens 8,387 Reasoning Tokens 600,946 Response Time (avg) 161.25s Response Time (total) 3708.70s Response Time (max) 739.98s Wrong answer: 3 No answer: 2 Agentic : 10.0 Anti-AI Tricks : 8.2 Coding : 10.0 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 10.0 Instructions following : 9.8 Puzzle Solving : 10.0 Tool Calling : 10.0 Trivia : 3.0
#185#185	Step 5 Previewhigh	7.0	Stepfun	$4.513	154.41s
View model card Total Tests 23 Wrong Tests 13 Reliability 9.5 Attempt pass rate 52.2% Flaky tests 4 Input Tokens 243,381 Output Tokens 1,581,325 Reasoning Tokens 0 Response Time (avg) 154.41s Response Time (total) 3551.40s Response Time (max) 455.72s No answer: 7 Wrong answer: 3 API error: 2 Did not follow instructions: 1 Agentic : 10.0 Anti-AI Tricks : 6.5 Coding : 2.9 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 3.0 General Intelligence : 3.4 Instructions following : 7.3 Puzzle Solving : 5.8 Tool Calling : 10.0 Trivia : 3.0
#30#30	Muse Spark Openhigh	9.2	Meta	$1.782	151.96s
View model card Total Tests 23 Wrong Tests 6 Reliability 9.4 Attempt pass rate 76.8% Flaky tests 1 Input Tokens 270,889 Output Tokens 10,688 Reasoning Tokens 328,857 Response Time (avg) 151.96s Response Time (total) 3495.01s Response Time (max) 960.01s Timed out: 2 Wrong answer: 2 API error: 1 Did not follow instructions: 1 Agentic : 10.0 Anti-AI Tricks : 8.3 Coding : 10.0 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 3.6 General Intelligence : 10.0 Instructions following : 9.8 Puzzle Solving : 7.9 Tool Calling : 10.0 Trivia : 3.0
#162#162	LongCat 2.0high	7.3	Meituan	$0.778 ↑	151.67s
View model card Total Tests 23 Wrong Tests 13 Reliability 10.0 Attempt pass rate 53.6% Flaky tests 4 Input Tokens 275,080 Output Tokens 32,447 Reasoning Tokens 372,188 Response Time (avg) 151.67s Response Time (total) 3488.43s Response Time (max) 941.66s Wrong answer: 7 Did not follow instructions: 2 No answer: 2 Timed out: 2 Agentic : 10.0 Anti-AI Tricks : 8.9 Coding : 7.8 Combined : 10.0 Data parsing and extraction : 3.6 Domain specific : 3.6 General Intelligence : 5.1 Instructions following : 6.5 Puzzle Solving : 3.1 Tool Calling : 10.0 Trivia : 3.0
#385#385	GLM 4.7 Flashmedium	4.2	Z.ai	$0.193	150.90s
View model card Total Tests 23 Wrong Tests 19 Reliability 9.4 Attempt pass rate 30.4% Flaky tests 8 Input Tokens 228,711 Output Tokens 42,254 Reasoning Tokens 422,121 Response Time (avg) 150.90s Response Time (total) 2414.33s Response Time (max) 1539.97s Wrong answer: 10 No answer: 3 Did not follow instructions: 2 Invalid tool call: 2 Timed out: 2 Agentic : 3.9 Anti-AI Tricks : 4.7 Coding : 3.2 Combined : 2.9 Data parsing and extraction : 6.3 Domain specific : 3.5 General Intelligence : 3.6 Instructions following : 6.2 Puzzle Solving : 2.9 Tool Calling : 10.0 Trivia : 3.0
#67#67	MiMo-V2.6-Promedium	8.7	Xiaomi	$0.586	148.11s
View model card Total Tests 23 Wrong Tests 9 Reliability 9.5 Attempt pass rate 78.3% Flaky tests 7 Input Tokens 298,476 Output Tokens 7,762 Reasoning Tokens 515,918 Response Time (avg) 148.11s Response Time (total) 3406.53s Response Time (max) 904.91s Extra formatting: 3 Wrong answer: 3 Did not follow instructions: 1 No answer: 1 Timed out: 1 Agentic : 10.0 Anti-AI Tricks : 7.3 Coding : 8.2 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 10.0 Instructions following : 8.4 Puzzle Solving : 8.4 Tool Calling : 10.0 Trivia : 2.8
#206#206	Solar Pro 4low	6.7	Upstage	$0.101 ↑	145.21s
View model card Total Tests 23 Wrong Tests 12 Reliability 10.0 Attempt pass rate 60.9% Flaky tests 7 Input Tokens 384,974 Output Tokens 13,946 Reasoning Tokens 371,916 Response Time (avg) 145.21s Response Time (total) 3339.87s Response Time (max) 640.89s Wrong answer: 7 Did not follow instructions: 2 API error: 1 No answer: 1 Timed out: 1 Agentic : 6.1 Anti-AI Tricks : 8.2 Coding : 5.5 Combined : 6.3 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 5.5 Instructions following : 8.0 Puzzle Solving : 9.0 Tool Calling : 10.0 Trivia : 3.0
#171#171	LongCat 2.0medium	7.2	Meituan	$0.767 ↑	144.88s
View model card Total Tests 23 Wrong Tests 11 Reliability 10.0 Attempt pass rate 59.4% Flaky tests 3 Input Tokens 287,669 Output Tokens 47,204 Reasoning Tokens 375,030 Response Time (avg) 144.88s Response Time (total) 3332.19s Response Time (max) 939.52s Wrong answer: 8 API error: 1 No answer: 1 Timed out: 1 Agentic : 6.1 Anti-AI Tricks : 10.0 Coding : 10.0 Combined : 7.3 Data parsing and extraction : 6.3 Domain specific : 3.0 General Intelligence : 4.8 Instructions following : 6.5 Puzzle Solving : 5.4 Tool Calling : 10.0 Trivia : 3.0
#202#202	Solar Pro 4xhigh	6.7	Upstage	$0.067	142.76s
View model card Total Tests 23 Wrong Tests 11 Reliability 9.9 Attempt pass rate 58.0% Flaky tests 2 Input Tokens 150,033 Output Tokens 22,347 Reasoning Tokens 402,143 Response Time (avg) 142.76s Response Time (total) 3283.43s Response Time (max) 491.42s Wrong answer: 5 Did not follow instructions: 3 No answer: 2 Extra formatting: 1 Agentic : 3.0 Anti-AI Tricks : 8.3 Coding : 5.8 Combined : 10.0 Data parsing and extraction : 10.0 Domain specific : 3.0 General Intelligence : 5.5 Instructions following : 8.4 Puzzle Solving : 8.9 Tool Calling : 10.0 Trivia : 3.0
#316#316	North Mini Codemedium	5.4	Cohere	$0.000	140.39s
View model card Total Tests 23 Wrong Tests 15 Reliability 8.9 Attempt pass rate 42.0% Flaky tests 4 Input Tokens 166,347 Output Tokens 423,884 Reasoning Tokens 1,394,387 Response Time (avg) 140.39s Response Time (total) 3229.07s Response Time (max) 786.72s Wrong answer: 10 Extra formatting: 2 API error: 1 Invalid tool call: 1 Timed out: 1 Agentic : 3.9 Anti-AI Tricks : 8.4 Coding : 4.5 Combined : 2.9 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 5.1 Instructions following : 9.8 Puzzle Solving : 3.3 Tool Calling : 10.0 Trivia : 3.0
#93#93	Kimi K3max	8.3	Moonshot AI	$3.409 ↓	139.41s
View model card Total Tests 23 Wrong Tests 6 Reliability 9.3 Attempt pass rate 78.3% Flaky tests 2 Input Tokens 170,700 Output Tokens 7,719 Reasoning Tokens 225,608 Response Time (avg) 139.41s Response Time (total) 2788.26s Response Time (max) 766.58s API error: 2 No answer: 2 Extra formatting: 1 Timed out: 1 Agentic : 10.0 Anti-AI Tricks : 10.0 Coding : 10.0 Combined : 6.5 Data parsing and extraction : 7.3 Domain specific : 5.3 General Intelligence : 10.0 Instructions following : 10.0 Puzzle Solving : 10.0 Tool Calling : 3.0 Trivia : 3.0
#302#302	Step 5 Previewlow	5.6	Stepfun	$4.110	138.27s
View model card Total Tests 23 Wrong Tests 13 Reliability 9.7 Attempt pass rate 58.0% Flaky tests 7 Input Tokens 261,256 Output Tokens 1,425,268 Reasoning Tokens 0 Response Time (avg) 138.27s Response Time (total) 3180.10s Response Time (max) 436.56s No answer: 7 API error: 2 Wrong answer: 2 Did not follow instructions: 1 Invalid tool call: 1 Agentic : 3.4 Anti-AI Tricks : 6.9 Coding : 2.7 Combined : 7.3 Data parsing and extraction : 10.0 Domain specific : 3.6 General Intelligence : 3.5 Instructions following : 10.0 Puzzle Solving : 7.6 Tool Calling : 10.0 Trivia : 3.0
#271#271	Step 5 Previewmedium	6.1	Stepfun	$4.136	136.10s
View model card Total Tests 23 Wrong Tests 13 Reliability 9.6 Attempt pass rate 53.6% Flaky tests 5 Input Tokens 261,842 Output Tokens 1,434,621 Reasoning Tokens 0 Response Time (avg) 136.10s Response Time (total) 3130.27s Response Time (max) 438.09s No answer: 6 Wrong answer: 3 Did not follow instructions: 2 API error: 1 Invalid tool call: 1 Agentic : 6.1 Anti-AI Tricks : 6.4 Coding : 2.8 Combined : 7.3 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 3.4 Instructions following : 10.0 Puzzle Solving : 7.8 Tool Calling : 10.0 Trivia : 3.0
#235#235	Solar Pro 4medium	6.4	Upstage	$0.098 ↑	133.93s
View model card Total Tests 23 Wrong Tests 11 Reliability 9.9 Attempt pass rate 60.9% Flaky tests 5 Input Tokens 299,558 Output Tokens 33,248 Reasoning Tokens 374,819 Response Time (avg) 133.93s Response Time (total) 3080.36s Response Time (max) 519.91s Wrong answer: 6 Did not follow instructions: 2 API error: 1 Extra formatting: 1 No answer: 1 Agentic : 4.7 Anti-AI Tricks : 8.2 Coding : 5.5 Combined : 6.4 Data parsing and extraction : 10.0 Domain specific : 3.0 General Intelligence : 3.0 Instructions following : 9.8 Puzzle Solving : 9.0 Tool Calling : 10.0 Trivia : 3.0
#250#250	Solar Pro 4high	6.3	Upstage	$0.077 ↑	128.91s
View model card Total Tests 23 Wrong Tests 12 Reliability 10.0 Attempt pass rate 60.9% Flaky tests 6 Input Tokens 209,002 Output Tokens 24,675 Reasoning Tokens 377,528 Response Time (avg) 128.91s Response Time (total) 2965.04s Response Time (max) 491.07s Wrong answer: 6 Did not follow instructions: 3 API error: 1 Extra formatting: 1 No answer: 1 Agentic : 2.8 Anti-AI Tricks : 8.7 Coding : 5.7 Combined : 7.3 Data parsing and extraction : 10.0 Domain specific : 3.6 General Intelligence : 5.5 Instructions following : 7.9 Puzzle Solving : 8.7 Tool Calling : 10.0 Trivia : 3.0
#237#237	Kimi K2.5medium	6.4	Moonshot AI	$0.841 ↑	125.76s
View model card Total Tests 23 Wrong Tests 13 Reliability 9.6 Attempt pass rate 60.9% Flaky tests 8 Input Tokens 292,271 Output Tokens 55,025 Reasoning Tokens 186,071 Response Time (avg) 125.76s Response Time (total) 2137.84s Response Time (max) 566.79s Wrong answer: 5 Timed out: 3 Did not follow instructions: 2 No answer: 2 Invalid tool call: 1 Agentic : 3.9 Anti-AI Tricks : 7.3 Coding : 6.1 Combined : 6.7 Data parsing and extraction : 10.0 Domain specific : 2.9 General Intelligence : 6.5 Instructions following : 10.0 Puzzle Solving : 5.3 Tool Calling : 10.0 Trivia : 3.0

1 2 18

→

Global benchmark trends

See how current models trade score for cost, speed, output length, and category strength.

Charts →

Intelligence vs Cost Pareto Frontier

Leading Company Score Frontier Over Time

Intelligence vs Response Speed Pareto Frontier

Score vs Output Tokens Pareto Frontier

Category Difficulty Across Models

Quick Compare

Gemini 3.6 FlashhighvsGemini 3.8 Flashhigh Gemini 3.8 FlashhighvsGPT-6 Solhigh GPT-6 SolhighvsGPT-6.1 Solmax GPT-6.1 SolmaxvsGPT-6 Astramax GPT-6 AstramaxvsGPT-6.1 Solxhigh GPT-6.1 SolxhighvsGPT-6 Astrahigh GPT-6 AstrahighvsGemini 3.7 Flashhigh Gemini 3.7 FlashhighvsGemini 3.6 Flashmedium Gemini 3.6 FlashmediumvsGPT-6.1 Solmedium GPT-6.1 SolmediumvsGPT-6.1 Solhigh GPT-6.1 SolhighvsGPT-5.6 Sollow GPT-5.6 SollowvsGPT-6 Astralow