Agentic
: 5.4 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)79.61sResponse Time (max)79.61sResponse Time (total)79.61s
Anti-AI Tricks
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)4.33sResponse Time (max)9.08sResponse Time (total)17.30s
Coding
: 5.9 A test is fully passed only if every run passed for that test.Extra formatting: 1Wrong answer: 1Response Time (avg)5.54sResponse Time (max)8.42sResponse Time (total)16.63s
Combined
: 8.3 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)30.54sResponse Time (max)47.72sResponse Time (total)61.09s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.88sResponse Time (max)4.06sResponse Time (total)7.75s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)11.10sResponse Time (max)26.91sResponse Time (total)33.31s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.70sResponse Time (max)3.70sResponse Time (total)3.70s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.81sResponse Time (max)2.98sResponse Time (total)5.62s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)3.66sResponse Time (max)4.75sResponse Time (total)10.98s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.89sResponse Time (max)7.89sResponse Time (total)7.89s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)4.08sResponse Time (max)4.08sResponse Time (total)4.08s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)74.39sResponse Time (max)74.39sResponse Time (total)74.39s
Anti-AI Tricks
: 7.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.71sResponse Time (max)2.32sResponse Time (total)6.83s
Coding
: 6.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)6.37sResponse Time (max)10.11sResponse Time (total)19.10s
Combined
: 2.8 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)15.97sResponse Time (max)22.89sResponse Time (total)31.94s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.74sResponse Time (max)1.87sResponse Time (total)3.48s
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)3.52sResponse Time (max)5.14sResponse Time (total)10.57s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.51sResponse Time (max)1.66sResponse Time (total)3.03s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.84sResponse Time (max)2.15sResponse Time (total)5.53s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)10.30sResponse Time (max)10.30sResponse Time (total)10.30s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)2.60sResponse Time (max)2.60sResponse Time (total)2.60s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)46.61sResponse Time (max)46.61sResponse Time (total)46.61s
Combined
: 7.9 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)22.50sResponse Time (max)22.74sResponse Time (total)45.00s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.31sResponse Time (max)2.95sResponse Time (total)4.63s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)2.50sResponse Time (max)3.14sResponse Time (total)7.51s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.67sResponse Time (max)4.67sResponse Time (total)4.67s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)4.97sResponse Time (max)4.97sResponse Time (total)4.97s
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)65.07sResponse Time (max)65.07sResponse Time (total)65.07s
Anti-AI Tricks
: 6.5 A test is fully passed only if every run passed for that test.Extra formatting: 2Response Time (avg)3.40sResponse Time (max)6.36sResponse Time (total)13.58s
Combined
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)26.38sResponse Time (max)35.03sResponse Time (total)52.76s
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.77sResponse Time (max)1.93sResponse Time (total)3.53s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.76sResponse Time (max)2.16sResponse Time (total)5.29s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.48sResponse Time (max)3.48sResponse Time (total)3.48s
Instructions following
: 9.9 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.37sResponse Time (max)1.40sResponse Time (total)2.73s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)2.74sResponse Time (max)3.46sResponse Time (total)8.22s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.35sResponse Time (max)5.35sResponse Time (total)5.35s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)3.41sResponse Time (max)3.41sResponse Time (total)3.41s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)59.70sResponse Time (max)59.70sResponse Time (total)59.70s
Anti-AI Tricks
: 8.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.14sResponse Time (max)2.97sResponse Time (total)8.57s
Coding
: 5.4 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)10.38sResponse Time (max)16.53sResponse Time (total)31.15s
Combined
: 3.8 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)54.70sResponse Time (max)90.45sResponse Time (total)109.40s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)2.39sResponse Time (max)2.80sResponse Time (total)4.78s
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)7.93sResponse Time (max)13.69sResponse Time (total)23.78s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.65sResponse Time (max)1.82sResponse Time (total)3.30s
Puzzle Solving
: 7.8 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)2.67sResponse Time (max)3.68sResponse Time (total)8.00s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)17.00sResponse Time (max)17.00sResponse Time (total)17.00s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.54sResponse Time (max)4.54sResponse Time (total)4.54s
Agentic
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)46.94sResponse Time (max)46.94sResponse Time (total)46.94s
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.70sResponse Time (max)5.66sResponse Time (total)14.80s
Combined
: 6.9 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)50.17sResponse Time (max)79.65sResponse Time (total)100.33s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.17sResponse Time (max)11.71sResponse Time (total)14.35s
Domain specific
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)6.35sResponse Time (max)7.33sResponse Time (total)19.05s
General Intelligence
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.42sResponse Time (max)4.42sResponse Time (total)4.42s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.84sResponse Time (max)4.88sResponse Time (total)7.68s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.31sResponse Time (max)3.63sResponse Time (total)9.92s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)15.76sResponse Time (max)15.76sResponse Time (total)15.76s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.41sResponse Time (max)3.41sResponse Time (total)3.41s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)51.18sResponse Time (max)51.18sResponse Time (total)51.18s
Anti-AI Tricks
: 6.5 A test is fully passed only if every run passed for that test.Extra formatting: 2Response Time (avg)2.54sResponse Time (max)4.30sResponse Time (total)10.16s
Combined
: 8.5 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)23.58sResponse Time (max)26.16sResponse Time (total)47.16s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)2.56sResponse Time (max)2.88sResponse Time (total)5.13s
Instructions following
: 6.3 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)1.88sResponse Time (max)2.03sResponse Time (total)3.76s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)2.43sResponse Time (max)2.70sResponse Time (total)7.29s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.75sResponse Time (max)4.75sResponse Time (total)4.75s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)8.57sResponse Time (max)8.57sResponse Time (total)8.57s
Agentic
: 5.2 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)48.84sResponse Time (max)48.84sResponse Time (total)48.84s
Anti-AI Tricks
: 6.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)4.16sResponse Time (max)6.41sResponse Time (total)16.65s
Coding
: 6.2 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)21.02sResponse Time (max)44.29sResponse Time (total)63.06s
Combined
: 3.8 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)53.02sResponse Time (max)83.60sResponse Time (total)106.04s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.42sResponse Time (max)4.01sResponse Time (total)6.84s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)19.23sResponse Time (max)30.42sResponse Time (total)57.69s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.40sResponse Time (max)3.57sResponse Time (total)6.79s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.20sResponse Time (max)4.53sResponse Time (total)12.59s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)18.45sResponse Time (max)18.45sResponse Time (total)18.45s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)7.42sResponse Time (max)7.42sResponse Time (total)7.42s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)107.19sResponse Time (max)107.19sResponse Time (total)107.19s
Anti-AI Tricks
: 4.0 A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)2.11sResponse Time (max)3.94sResponse Time (total)8.46s
Coding
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)4.96sResponse Time (max)9.79sResponse Time (total)14.89s
Combined
: 2.8 A test is fully passed only if every run passed for that test.Invalid tool call: 1No answer: 1Response Time (avg)46.88sResponse Time (max)61.20sResponse Time (total)93.77s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.08sResponse Time (max)1.62sResponse Time (total)2.15s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)2.27sResponse Time (max)3.99sResponse Time (total)6.80s
General Intelligence
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)790msResponse Time (max)790msResponse Time (total)790ms
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.98sResponse Time (max)2.28sResponse Time (total)3.97s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.45sResponse Time (max)2.09sResponse Time (total)4.36s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)10.68sResponse Time (max)10.68sResponse Time (total)10.68s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.34sResponse Time (max)2.34sResponse Time (total)2.34s
A test is fully passed only if every run passed for that test.Wrong answer: 4Extra formatting: 3Timed out: 1Response Time (avg)28.08sResponse Time (max)140.96sResponse Time (total)421.15s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Coding
: 5.7 A test is fully passed only if every run passed for that test.Extra formatting: 1Wrong answer: 1Response Time (avg)33.29sResponse Time (max)35.76sResponse Time (total)99.86s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)93.65sResponse Time (max)140.96sResponse Time (total)187.31s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)13.90sResponse Time (max)13.90sResponse Time (total)13.90s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.94sResponse Time (max)4.94sResponse Time (total)4.94s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.61sResponse Time (max)2.61sResponse Time (total)2.61s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.31sResponse Time (max)6.24sResponse Time (total)10.62s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.48sResponse Time (max)7.48sResponse Time (total)7.48s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)30.09sResponse Time (max)30.09sResponse Time (total)30.09s
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 6.4 A test is fully passed only if every run passed for that test.Extra formatting: 2Response Time (avg)7.45sResponse Time (max)11.88sResponse Time (total)14.90s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)114.08sResponse Time (max)151.51sResponse Time (total)228.16s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.37sResponse Time (max)7.37sResponse Time (total)7.37s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.04sResponse Time (max)5.04sResponse Time (total)5.04s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.43sResponse Time (max)2.43sResponse Time (total)2.43s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.71sResponse Time (max)4.75sResponse Time (total)9.41s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.73sResponse Time (max)9.73sResponse Time (total)9.73s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)63.24sResponse Time (max)63.24sResponse Time (total)63.24s
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.82sResponse Time (max)7.69sResponse Time (total)19.26s
Coding
: 8.2 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)45.90sResponse Time (max)95.57sResponse Time (total)137.71s
Combined
: 5.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)13.88sResponse Time (max)13.88sResponse Time (total)13.88s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.19sResponse Time (max)6.42sResponse Time (total)12.38s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Timed out: 1Response Time (avg)71.07sResponse Time (max)194.23sResponse Time (total)213.22s
General Intelligence
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)10.05sResponse Time (max)10.05sResponse Time (total)10.05s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.38sResponse Time (max)5.70sResponse Time (total)10.77s
Puzzle Solving
: 8.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)5.23sResponse Time (max)7.26sResponse Time (total)15.69s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.84sResponse Time (max)9.84sResponse Time (total)9.84s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)40.17sResponse Time (max)40.17sResponse Time (total)40.17s
Agentic
: 2.8 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)34.77sResponse Time (max)34.77sResponse Time (total)34.77s
Anti-AI Tricks
: 6.5 A test is fully passed only if every run passed for that test.Extra formatting: 2Response Time (avg)4.54sResponse Time (max)4.91sResponse Time (total)18.16s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)29.40sResponse Time (max)33.10sResponse Time (total)58.80s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.71sResponse Time (max)4.86sResponse Time (total)9.43s
Domain specific
: 4.1 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)19.30sResponse Time (max)19.72sResponse Time (total)57.90s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.62sResponse Time (max)5.62sResponse Time (total)5.62s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)5.24sResponse Time (max)5.70sResponse Time (total)10.49s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)4.78sResponse Time (max)5.07sResponse Time (total)14.33s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)13.22sResponse Time (max)13.22sResponse Time (total)13.22s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)19.18sResponse Time (max)19.18sResponse Time (total)19.18s
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Coding
: 5.5 A test is fully passed only if every run passed for that test.Extra formatting: 1Wrong answer: 1Response Time (avg)5.19sResponse Time (max)9.79sResponse Time (total)15.56s
Combined
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)37.51sResponse Time (max)51.18sResponse Time (total)75.03s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.43sResponse Time (max)3.43sResponse Time (total)3.43s
Domain specific
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.27sResponse Time (max)3.54sResponse Time (total)4.54s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.96sResponse Time (max)1.96sResponse Time (total)1.96s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)2.53sResponse Time (max)2.54sResponse Time (total)5.06s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.11sResponse Time (max)4.11sResponse Time (total)4.11s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.67sResponse Time (max)4.67sResponse Time (total)4.67s
Agentic
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)77.99sResponse Time (max)77.99sResponse Time (total)77.99s
Anti-AI Tricks
: 8.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.56sResponse Time (max)2.05sResponse Time (total)6.26s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)5.45sResponse Time (max)6.31sResponse Time (total)16.36s
Combined
: 2.8 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)42.40sResponse Time (max)73.01sResponse Time (total)84.79s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.59sResponse Time (max)1.63sResponse Time (total)3.17s
Domain specific
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)3.87sResponse Time (max)4.85sResponse Time (total)11.62s
General Intelligence
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.07sResponse Time (max)2.07sResponse Time (total)2.07s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.34sResponse Time (max)1.42sResponse Time (total)2.69s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.89sResponse Time (max)2.38sResponse Time (total)5.68s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.52sResponse Time (max)9.52sResponse Time (total)9.52s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)2.69sResponse Time (max)2.69sResponse Time (total)2.69s
Agentic
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)64.73sResponse Time (max)64.73sResponse Time (total)64.73s
Anti-AI Tricks
: 5.3 A test is fully passed only if every run passed for that test.Extra formatting: 2Wrong answer: 1Response Time (avg)3.60sResponse Time (max)7.10sResponse Time (total)14.39s
Coding
: 4.6 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)3.67sResponse Time (max)5.83sResponse Time (total)11.02s
Combined
: 6.5 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)31.43sResponse Time (max)33.39sResponse Time (total)62.86s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.01sResponse Time (max)3.29sResponse Time (total)6.02s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)3.32sResponse Time (max)4.75sResponse Time (total)9.96s
Instructions following
: 6.4 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)2.58sResponse Time (max)2.89sResponse Time (total)5.17s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.80sResponse Time (max)6.80sResponse Time (total)6.80s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)4.31sResponse Time (max)4.31sResponse Time (total)4.31s
A test is fully passed only if every run passed for that test.Wrong answer: 13Response Time (avg)8.74sResponse Time (max)74.80sResponse Time (total)131.17s…
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)74.80sResponse Time (max)74.80sResponse Time (total)74.80s
Anti-AI Tricks
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.37sResponse Time (max)3.39sResponse Time (total)4.75s
Coding
: 4.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)5.12sResponse Time (max)8.84sResponse Time (total)15.36s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.98sResponse Time (max)4.98sResponse Time (total)4.98s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.78sResponse Time (max)5.78sResponse Time (total)5.78s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.24sResponse Time (max)2.24sResponse Time (total)2.24s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.27sResponse Time (max)3.27sResponse Time (total)3.27s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.48sResponse Time (max)1.48sResponse Time (total)1.48s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.91sResponse Time (max)2.08sResponse Time (total)3.82s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)11.07sResponse Time (max)11.07sResponse Time (total)11.07s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.62sResponse Time (max)3.62sResponse Time (total)3.62s
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Coding
: 6.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)63.38sResponse Time (max)95.88sResponse Time (total)190.15s
Combined
: 3.4 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)15.06sResponse Time (max)15.06sResponse Time (total)15.06s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.60sResponse Time (max)9.92sResponse Time (total)19.19s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)38.15sResponse Time (max)67.08sResponse Time (total)114.45s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)11.09sResponse Time (max)11.09sResponse Time (total)11.09s
Instructions following
: 9.9 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.74sResponse Time (max)5.23sResponse Time (total)7.47s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)10.24sResponse Time (max)16.95sResponse Time (total)30.72s
Tool Calling
: 7.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)12.53sResponse Time (max)12.53sResponse Time (total)12.53s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)40.96sResponse Time (max)40.96sResponse Time (total)40.96s
A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)3.02sResponse Time (max)18.27sResponse Time (total)57.44s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.12sResponse Time (max)3.75sResponse Time (total)8.50s
Coding
: 3.3 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.84sResponse Time (max)2.84sResponse Time (total)2.84s
Combined
: 4.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)18.27sResponse Time (max)18.27sResponse Time (total)18.27s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.15sResponse Time (max)2.33sResponse Time (total)4.29s
Domain specific
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.19sResponse Time (max)1.40sResponse Time (total)3.58s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.47sResponse Time (max)3.47sResponse Time (total)3.47s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.46sResponse Time (max)1.68sResponse Time (total)2.91s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.46sResponse Time (max)3.72sResponse Time (total)7.38s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.74sResponse Time (max)4.74sResponse Time (total)4.74s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.46sResponse Time (max)1.46sResponse Time (total)1.46s
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)80.73sResponse Time (max)80.73sResponse Time (total)80.73s
Anti-AI Tricks
: 5.2 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)5.51sResponse Time (max)6.59sResponse Time (total)11.02s
Coding
: 4.3 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.54sResponse Time (max)5.57sResponse Time (total)7.62s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 2Response Time (avg)50.18sResponse Time (max)97.15sResponse Time (total)100.36s
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.82sResponse Time (max)4.82sResponse Time (total)4.82s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)2.22sResponse Time (max)3.70sResponse Time (total)4.44s
General Intelligence
: 4.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.59sResponse Time (max)1.59sResponse Time (total)1.59s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)888msResponse Time (max)888msResponse Time (total)888ms
Tool Calling
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)7.05sResponse Time (max)7.05sResponse Time (total)7.05s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)692msResponse Time (max)692msResponse Time (total)692ms
A test is fully passed only if every run passed for that test.Wrong answer: 11Did not follow instructions: 2Response Time (avg)2.99sResponse Time (max)6.51sResponse Time (total)62.74s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)3.13sResponse Time (max)5.90sResponse Time (total)12.50s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)3.13sResponse Time (max)5.30sResponse Time (total)9.40s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)6.51sResponse Time (max)6.51sResponse Time (total)6.51s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.81sResponse Time (max)5.69sResponse Time (total)7.62s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)2.09sResponse Time (max)2.39sResponse Time (total)6.26s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.97sResponse Time (max)2.43sResponse Time (total)3.93s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.86sResponse Time (max)4.86sResponse Time (total)4.86s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.23sResponse Time (max)2.23sResponse Time (total)2.23s
Agentic
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)399.20sResponse Time (max)399.20sResponse Time (total)399.20s
Coding
: 3.2 A test is fully passed only if every run passed for that test.Timed out: 2No answer: 1Response Time (avg)55.33sResponse Time (max)89.40sResponse Time (total)110.66s
Combined
: 2.9 A test is fully passed only if every run passed for that test.Invalid tool call: 2Response Time (avg)802.77sResponse Time (max)1539.97sResponse Time (total)1605.54s
Data parsing and extraction
: 6.3 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)1.51sResponse Time (max)1.51sResponse Time (total)1.51s
Domain specific
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 2No answer: 1Response Time (avg)96.74sResponse Time (max)174.55sResponse Time (total)193.47s
General Intelligence
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)18.14sResponse Time (max)18.14sResponse Time (total)18.14s
Instructions following
: 6.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.97sResponse Time (max)2.97sResponse Time (total)2.97s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)15.95sResponse Time (max)15.95sResponse Time (total)15.95s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)11.13sResponse Time (max)11.13sResponse Time (total)11.13s
A test is fully passed only if every run passed for that test.Wrong answer: 13Did not follow instructions: 2Response Time (avg)2.82sResponse Time (max)8.21sResponse Time (total)59.29s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)2.84sResponse Time (max)4.15sResponse Time (total)11.35s
Coding
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.41sResponse Time (max)3.93sResponse Time (total)7.22s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.89sResponse Time (max)4.89sResponse Time (total)4.89s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.47sResponse Time (max)2.48sResponse Time (total)4.95s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.97sResponse Time (max)2.65sResponse Time (total)5.92s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.13sResponse Time (max)2.53sResponse Time (total)4.27s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.21sResponse Time (max)8.21sResponse Time (total)8.21s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.37sResponse Time (max)2.37sResponse Time (total)2.37s
A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)13.58sResponse Time (max)13.58sResponse Time (total)13.58s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Coding
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Domain specific
: 3.3 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)13.58sResponse Time (max)13.58sResponse Time (total)13.58s
General Intelligence
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Instructions following
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Puzzle Solving
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Tool Calling
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Global benchmark trends
See how current models trade score for cost, speed, output length, and category strength.