Anti-AI Tricks
: 3.2 A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)1.19sResponse Time (max)2.73sResponse Time (total)4.76s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.87sResponse Time (max)2.87sResponse Time (total)2.87s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)564msResponse Time (max)564msResponse Time (total)564ms
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)857msResponse Time (max)955msResponse Time (total)1.71s
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.86sResponse Time (max)2.70sResponse Time (total)3.71s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.28sResponse Time (max)2.28sResponse Time (total)2.28s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.82sResponse Time (max)1.82sResponse Time (total)1.82s
Anti-AI Tricks
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 2Extra formatting: 1Response Time (avg)5.14sResponse Time (max)14.88sResponse Time (total)20.55s
Coding
: 4.2 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)1.19sResponse Time (max)1.84sResponse Time (total)3.56s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)124.21sResponse Time (max)240.61sResponse Time (total)248.43s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)880msResponse Time (max)917msResponse Time (total)1.76s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.64sResponse Time (max)5.23sResponse Time (total)7.91s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)3.05sResponse Time (max)3.05sResponse Time (total)3.05s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)988msResponse Time (max)988msResponse Time (total)988ms
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 2Response Time (avg)9.28sResponse Time (max)16.67sResponse Time (total)18.55s
Data parsing and extraction
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)575msResponse Time (max)583msResponse Time (total)1.15s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)287msResponse Time (max)348msResponse Time (total)860ms
General Intelligence
: 4.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)499msResponse Time (max)499msResponse Time (total)499ms
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.17sResponse Time (max)2.17sResponse Time (total)2.17s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)306msResponse Time (max)306msResponse Time (total)306ms
Anti-AI Tricks
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.12sResponse Time (max)6.30sResponse Time (total)8.49s
Coding
: 3.1 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)1.55sResponse Time (max)1.76sResponse Time (total)4.64s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)2.02sResponse Time (max)2.27sResponse Time (total)4.05s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)683msResponse Time (max)1.30sResponse Time (total)2.05s
General Intelligence
: 4.4 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)586msResponse Time (max)586msResponse Time (total)586ms
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)511msResponse Time (max)524msResponse Time (total)1.02s
Puzzle Solving
: 3.1 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)750msResponse Time (max)1.07sResponse Time (total)2.25s
Tool Calling
: 2.8 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)1.70sResponse Time (max)1.70sResponse Time (total)1.70s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)575msResponse Time (max)575msResponse Time (total)575ms
Anti-AI Tricks
: 8.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)6.30sResponse Time (max)9.80sResponse Time (total)25.20s
Coding
: 3.3 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)21.41sResponse Time (max)21.41sResponse Time (total)21.41s
Combined
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
General Intelligence
: 4.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)12.47sResponse Time (max)12.47sResponse Time (total)12.47s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.36sResponse Time (max)11.05sResponse Time (total)14.73s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)36.09sResponse Time (max)36.09sResponse Time (total)36.09s
Anti-AI Tricks
: 6.4 A test is fully passed only if every run passed for that test.No answer: 1Wrong answer: 1Response Time (avg)24.93sResponse Time (max)83.87sResponse Time (total)99.73s
Coding
: 2.9 A test is fully passed only if every run passed for that test.No answer: 2Did not follow instructions: 1Response Time (avg)177.65sResponse Time (max)213.57sResponse Time (total)532.96s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)179.85sResponse Time (max)345.15sResponse Time (total)359.71s
Data parsing and extraction
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)4.16sResponse Time (max)5.89sResponse Time (total)8.31s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.No answer: 2API error: 1Response Time (avg)70.12sResponse Time (max)81.88sResponse Time (total)140.23s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2No answer: 1Response Time (avg)54.15sResponse Time (max)148.72sResponse Time (total)162.46s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.76sResponse Time (max)5.76sResponse Time (total)5.76s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)213.35sResponse Time (max)213.35sResponse Time (total)213.35s
Anti-AI Tricks
: 6.5 A test is fully passed only if every run passed for that test.No answer: 1Wrong answer: 1Response Time (avg)25.30sResponse Time (max)84.32sResponse Time (total)101.18s
Coding
: 2.9 A test is fully passed only if every run passed for that test.No answer: 2Did not follow instructions: 1Response Time (avg)170.73sResponse Time (max)214.91sResponse Time (total)512.19s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)134.79sResponse Time (max)252.08sResponse Time (total)269.57s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.No answer: 2API error: 1Response Time (avg)72.15sResponse Time (max)81.30sResponse Time (total)144.29s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2No answer: 1Response Time (avg)32.49sResponse Time (max)91.44sResponse Time (total)97.46s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.94sResponse Time (max)5.94sResponse Time (total)5.94s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)213.54sResponse Time (max)213.54sResponse Time (total)213.54s
A test is fully passed only if every run passed for that test.Wrong answer: 13Did not follow instructions: 3Response Time (avg)1.62sResponse Time (max)5.51sResponse Time (total)19.48s…
Coding
: 1.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.79sResponse Time (max)1.79sResponse Time (total)1.79s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.33sResponse Time (max)3.33sResponse Time (total)3.33s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)943msResponse Time (max)943msResponse Time (total)943ms
Domain specific
: 5.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.06sResponse Time (max)1.06sResponse Time (total)1.06s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)1.10sResponse Time (max)1.36sResponse Time (total)2.21s
Tool Calling
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)5.51sResponse Time (max)5.51sResponse Time (total)5.51s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)731msResponse Time (max)731msResponse Time (total)731ms
Anti-AI Tricks
: 5.1 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)34.44sResponse Time (max)57.86sResponse Time (total)103.31s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 2Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Timed out: 2No answer: 1Response Time (avg)272.50sResponse Time (max)569.97sResponse Time (total)817.51s
General Intelligence
: 2.8 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)226.38sResponse Time (max)226.38sResponse Time (total)226.38s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)5.75sResponse Time (max)5.75sResponse Time (total)5.75s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)32.27sResponse Time (max)47.31sResponse Time (total)96.80s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.31sResponse Time (max)4.31sResponse Time (total)4.31s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)177.02sResponse Time (max)177.02sResponse Time (total)177.02s
Anti-AI Tricks
: 6.5 A test is fully passed only if every run passed for that test.No answer: 1Wrong answer: 1Response Time (avg)23.73sResponse Time (max)81.06sResponse Time (total)94.91s
Coding
: 2.9 A test is fully passed only if every run passed for that test.No answer: 2Did not follow instructions: 1Response Time (avg)211.99sResponse Time (max)219.75sResponse Time (total)635.97s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)139.78sResponse Time (max)262.20sResponse Time (total)279.57s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.No answer: 2API error: 1Response Time (avg)81.40sResponse Time (max)82.11sResponse Time (total)162.80s
Puzzle Solving
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 2No answer: 1Response Time (avg)34.65sResponse Time (max)88.47sResponse Time (total)103.95s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.91sResponse Time (max)4.91sResponse Time (total)4.91s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)100.96sResponse Time (max)100.96sResponse Time (total)100.96s
Coding
: 8.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.96sResponse Time (max)1.96sResponse Time (total)1.96s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.01sResponse Time (max)2.01sResponse Time (total)2.01s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)646msResponse Time (max)658msResponse Time (total)1.29s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)371msResponse Time (max)419msResponse Time (total)1.11s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)439msResponse Time (max)448msResponse Time (total)878ms
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)650msResponse Time (max)843msResponse Time (total)1.30s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)1.93sResponse Time (max)1.93sResponse Time (total)1.93s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Coding
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)9.57sResponse Time (max)9.57sResponse Time (total)9.57s
Combined
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)7.12sResponse Time (max)7.12sResponse Time (total)7.12s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)34.98sResponse Time (max)68.97sResponse Time (total)104.94s
General Intelligence
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)10.79sResponse Time (max)10.79sResponse Time (total)10.79s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.06sResponse Time (max)5.85sResponse Time (total)10.12s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)47.29sResponse Time (max)47.29sResponse Time (total)47.29s
Anti-AI Tricks
: 6.4 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)1.20sResponse Time (max)1.48sResponse Time (total)3.59s
Coding
: 1.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)38.09sResponse Time (max)38.09sResponse Time (total)38.09s
Combined
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.72sResponse Time (max)3.88sResponse Time (total)5.43s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2No answer: 1Response Time (avg)56.67sResponse Time (max)147.45sResponse Time (total)170.02s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Puzzle Solving
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)1.40sResponse Time (max)1.57sResponse Time (total)2.79s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)584msResponse Time (max)772msResponse Time (total)1.75s
Coding
: 3.3 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.27sResponse Time (max)1.27sResponse Time (total)1.27s
Combined
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 3.8 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.42sResponse Time (max)2.21sResponse Time (total)2.84s
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)489msResponse Time (max)513msResponse Time (total)1.47s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)34.99sResponse Time (max)109.60sResponse Time (total)139.95s
Coding
: 9.8 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Domain specific
: 3.3 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)34.54sResponse Time (max)34.54sResponse Time (total)34.54s
Instructions following
: 5.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.30sResponse Time (max)9.30sResponse Time (total)9.30s
Puzzle Solving
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)114.12sResponse Time (max)114.12sResponse Time (total)114.12s
Anti-AI Tricks
: 2.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)471msResponse Time (max)872msResponse Time (total)1.41s
Coding
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)714msResponse Time (max)987msResponse Time (total)1.43s
Domain specific
: 5.9 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)287msResponse Time (max)334msResponse Time (total)860ms
Instructions following
: 6.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)752msResponse Time (max)1.22sResponse Time (total)1.50s
Puzzle Solving
: 3.8 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)1.78sResponse Time (max)3.15sResponse Time (total)5.34s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Global benchmark trends
See how current models trade score for cost, speed, output length, and category strength.