A test is fully passed only if every run passed for that test.Wrong answer: 16Did not follow instructions: 2Response Time (avg)7.11sResponse Time (max)30.51sResponse Time (total)163.46s…
Agentic
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)30.51sResponse Time (max)30.51sResponse Time (total)30.51s
Anti-AI Tricks
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)4.46sResponse Time (max)9.94sResponse Time (total)17.83s
Coding
: 3.3 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)2.64sResponse Time (max)3.05sResponse Time (total)7.92s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)18.22sResponse Time (max)20.00sResponse Time (total)36.45s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.92sResponse Time (max)13.23sResponse Time (total)15.84s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)6.78sResponse Time (max)14.38sResponse Time (total)20.34s
General Intelligence
: 4.6 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)950msResponse Time (max)950msResponse Time (total)950ms
Instructions following
: 6.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)804msResponse Time (max)921msResponse Time (total)1.61s
Tool Calling
: 4.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)16.00sResponse Time (max)16.00sResponse Time (total)16.00s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)8.94sResponse Time (max)8.94sResponse Time (total)8.94s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)50.14sResponse Time (max)50.14sResponse Time (total)50.14s
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)252.99sResponse Time (max)960.01sResponse Time (total)1011.96s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)33.49sResponse Time (max)79.41sResponse Time (total)100.47s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)111.75sResponse Time (max)191.35sResponse Time (total)223.49s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)14.72sResponse Time (max)27.84sResponse Time (total)29.44s
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 2Timed out: 1Response Time (avg)412.11sResponse Time (max)730.94sResponse Time (total)1236.34s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.35sResponse Time (max)8.35sResponse Time (total)8.35s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)28.50sResponse Time (max)38.63sResponse Time (total)57.00s
Puzzle Solving
: 7.9 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)145.68sResponse Time (max)407.90sResponse Time (total)437.03s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)10.65sResponse Time (max)10.65sResponse Time (total)10.65s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)330.14sResponse Time (max)330.14sResponse Time (total)330.14s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)66.63sResponse Time (max)66.63sResponse Time (total)66.63s
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.35sResponse Time (max)11.86sResponse Time (total)17.41s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)62.50sResponse Time (max)80.58sResponse Time (total)187.50s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)160.31sResponse Time (max)164.09sResponse Time (total)320.63s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)4.56sResponse Time (max)8.91sResponse Time (total)9.12s
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)198.41sResponse Time (max)335.08sResponse Time (total)595.23s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.00sResponse Time (max)4.00sResponse Time (total)4.00s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.64sResponse Time (max)11.30sResponse Time (total)15.29s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.49sResponse Time (max)9.49sResponse Time (total)9.49s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)98.48sResponse Time (max)98.48sResponse Time (total)98.48s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)64.52sResponse Time (max)64.52sResponse Time (total)64.52s
Anti-AI Tricks
: 6.6 A test is fully passed only if every run passed for that test.Timed out: 1Wrong answer: 1Response Time (avg)74.75sResponse Time (max)182.10sResponse Time (total)298.98s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Timed out: 1Wrong answer: 1Response Time (avg)220.48sResponse Time (max)243.66sResponse Time (total)440.97s
Combined
: 7.3 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)282.30sResponse Time (max)301.78sResponse Time (total)564.61s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)24.27sResponse Time (max)27.52sResponse Time (total)48.54s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)103.41sResponse Time (max)103.41sResponse Time (total)103.41s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)17.47sResponse Time (max)19.46sResponse Time (total)34.93s
Puzzle Solving
: 8.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)31.79sResponse Time (max)50.78sResponse Time (total)95.38s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)88.68sResponse Time (max)88.68sResponse Time (total)88.68s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)56.76sResponse Time (max)56.76sResponse Time (total)56.76s
Agentic
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)399.20sResponse Time (max)399.20sResponse Time (total)399.20s
Coding
: 3.2 A test is fully passed only if every run passed for that test.Timed out: 2No answer: 1Response Time (avg)55.33sResponse Time (max)89.40sResponse Time (total)110.66s
Combined
: 2.9 A test is fully passed only if every run passed for that test.Invalid tool call: 2Response Time (avg)802.77sResponse Time (max)1539.97sResponse Time (total)1605.54s
Data parsing and extraction
: 6.3 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)1.51sResponse Time (max)1.51sResponse Time (total)1.51s
Domain specific
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 2No answer: 1Response Time (avg)96.74sResponse Time (max)174.55sResponse Time (total)193.47s
General Intelligence
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)18.14sResponse Time (max)18.14sResponse Time (total)18.14s
Instructions following
: 6.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.97sResponse Time (max)2.97sResponse Time (total)2.97s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)15.95sResponse Time (max)15.95sResponse Time (total)15.95s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)11.13sResponse Time (max)11.13sResponse Time (total)11.13s
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 5.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)491msResponse Time (max)496msResponse Time (total)1.47s
Coding
: 4.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)572msResponse Time (max)575msResponse Time (total)1.14s
Combined
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)482msResponse Time (max)521msResponse Time (total)964ms
Domain specific
: 5.9 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)541msResponse Time (max)776msResponse Time (total)1.62s
General Intelligence
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Instructions following
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)508msResponse Time (max)508msResponse Time (total)508ms
Puzzle Solving
: 1.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)490msResponse Time (max)490msResponse Time (total)490ms
Tool Calling
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
A test is fully passed only if every run passed for that test.Wrong answer: 8API error: 1Response Time (avg)4.58sResponse Time (max)13.35sResponse Time (total)55.01s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 4.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)3.69sResponse Time (max)4.83sResponse Time (total)11.08s
Coding
: 2.7 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)12.73sResponse Time (max)13.35sResponse Time (total)25.46s
Combined
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.10sResponse Time (max)6.24sResponse Time (total)8.21s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.54sResponse Time (max)1.84sResponse Time (total)4.63s
General Intelligence
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Instructions following
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.26sResponse Time (max)4.26sResponse Time (total)4.26s
Puzzle Solving
: 1.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.37sResponse Time (max)1.37sResponse Time (total)1.37s
Tool Calling
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)74.24sResponse Time (max)74.24sResponse Time (total)74.24s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)10.24sResponse Time (max)15.21sResponse Time (total)40.97s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)325.79sResponse Time (max)589.43sResponse Time (total)977.36s
Combined
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)223.03sResponse Time (max)223.03sResponse Time (total)223.03s
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)16.72sResponse Time (max)18.73sResponse Time (total)33.44s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.No answer: 1Timed out: 1Response Time (avg)683.62sResponse Time (max)766.58sResponse Time (total)1367.25s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)14.91sResponse Time (max)14.91sResponse Time (total)14.91s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.66sResponse Time (max)7.99sResponse Time (total)15.33s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.36sResponse Time (max)12.30sResponse Time (total)22.07s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)19.67sResponse Time (max)19.67sResponse Time (total)19.67s
A test is fully passed only if every run passed for that test.API error: 6Wrong answer: 2Response Time (avg)22.96sResponse Time (max)143.21sResponse Time (total)528.08s…
Agentic
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)461msResponse Time (max)461msResponse Time (total)461ms
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)5.93sResponse Time (max)11.89sResponse Time (total)23.71s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)36.94sResponse Time (max)49.30sResponse Time (total)110.81s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)34.90sResponse Time (max)45.19sResponse Time (total)69.81s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.09sResponse Time (max)8.39sResponse Time (total)14.18s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)78.62sResponse Time (max)143.21sResponse Time (total)235.86s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.52sResponse Time (max)5.52sResponse Time (total)5.52s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)1.97sResponse Time (max)3.43sResponse Time (total)3.93s
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)3.78sResponse Time (max)10.03sResponse Time (total)11.34s
Tool Calling
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)23.70sResponse Time (max)23.70sResponse Time (total)23.70s
Trivia
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)28.77sResponse Time (max)28.77sResponse Time (total)28.77s
Agentic
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)600.81sResponse Time (max)600.81sResponse Time (total)600.81s
Coding
: 3.4 A test is fully passed only if every run passed for that test.No answer: 1Timed out: 1Wrong answer: 1Response Time (avg)188.58sResponse Time (max)251.36sResponse Time (total)377.16s
Combined
: 3.7 A test is fully passed only if every run passed for that test.Invalid tool call: 1No answer: 1Response Time (avg)83.19sResponse Time (max)105.99sResponse Time (total)166.38s
Data parsing and extraction
: 4.6 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)7.48sResponse Time (max)7.48sResponse Time (total)7.48s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Timed out: 1Response Time (avg)158.88sResponse Time (max)237.27sResponse Time (total)317.77s
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.Timed out: 1Wrong answer: 1Response Time (avg)11.21sResponse Time (max)17.37sResponse Time (total)22.43s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)15.35sResponse Time (max)15.35sResponse Time (total)15.35s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)80.79sResponse Time (max)80.79sResponse Time (total)80.79s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)225.59sResponse Time (max)225.59sResponse Time (total)225.59s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.31sResponse Time (max)14.20sResponse Time (total)33.24s
Combined
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)175.93sResponse Time (max)308.75sResponse Time (total)351.86s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.33sResponse Time (max)9.40sResponse Time (total)18.66s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)20.95sResponse Time (max)20.95sResponse Time (total)20.95s
Instructions following
: 6.4 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)7.47sResponse Time (max)10.16sResponse Time (total)14.94s
Puzzle Solving
: 8.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)31.64sResponse Time (max)46.04sResponse Time (total)94.91s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)29.40sResponse Time (max)29.40sResponse Time (total)29.40s
Agentic
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)142.68sResponse Time (max)142.68sResponse Time (total)142.68s
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)11.68sResponse Time (max)40.72sResponse Time (total)46.73s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)13.01sResponse Time (max)27.71sResponse Time (total)39.04s
Combined
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Timed out: 1Response Time (avg)298.21sResponse Time (max)566.80sResponse Time (total)596.42s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.39sResponse Time (max)2.27sResponse Time (total)2.78s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)26.16sResponse Time (max)77.41sResponse Time (total)78.47s
Instructions following
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 1Wrong answer: 1Response Time (avg)75.80sResponse Time (max)150.25sResponse Time (total)151.60s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Timed out: 1Response Time (avg)90.84sResponse Time (max)169.53sResponse Time (total)272.52s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.23sResponse Time (max)2.23sResponse Time (total)2.23s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)5.53sResponse Time (max)5.53sResponse Time (total)5.53s
A test is fully passed only if every run passed for that test.API error: 4Wrong answer: 2Response Time (avg)21.58sResponse Time (max)162.73sResponse Time (total)496.40s…
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)46.05sResponse Time (max)46.05sResponse Time (total)46.05s
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)4.84sResponse Time (max)9.81sResponse Time (total)19.36s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)26.87sResponse Time (max)35.98sResponse Time (total)80.62s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)28.23sResponse Time (max)35.12sResponse Time (total)56.46s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.12sResponse Time (max)7.58sResponse Time (total)12.23s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)70.72sResponse Time (max)162.73sResponse Time (total)212.15s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.32sResponse Time (max)7.32sResponse Time (total)7.32s
Instructions following
: 6.6 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)3.88sResponse Time (max)4.42sResponse Time (total)7.77s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)9.63sResponse Time (max)16.54sResponse Time (total)28.88s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.26sResponse Time (max)9.26sResponse Time (total)9.26s
Trivia
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)16.33sResponse Time (max)16.33sResponse Time (total)16.33s
A test is fully passed only if every run passed for that test.API error: 6Wrong answer: 3Response Time (avg)34.40sResponse Time (max)236.64sResponse Time (total)791.16s…
Agentic
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)525msResponse Time (max)525msResponse Time (total)525ms
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)6.04sResponse Time (max)13.58sResponse Time (total)24.14s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)40.90sResponse Time (max)52.61sResponse Time (total)122.71s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)51.45sResponse Time (max)55.44sResponse Time (total)102.90s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.90sResponse Time (max)9.31sResponse Time (total)13.80s
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)150.71sResponse Time (max)236.64sResponse Time (total)452.14s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.77sResponse Time (max)5.77sResponse Time (total)5.77s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)2.64sResponse Time (max)4.79sResponse Time (total)5.27s
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)3.53sResponse Time (max)9.25sResponse Time (total)10.58s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)27.97sResponse Time (max)27.97sResponse Time (total)27.97s
Trivia
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)25.37sResponse Time (max)25.37sResponse Time (total)25.37s
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)49.21sResponse Time (max)49.21sResponse Time (total)49.21s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)105.60sResponse Time (max)314.09sResponse Time (total)316.81s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 1Wrong answer: 1Response Time (avg)424.28sResponse Time (max)747.17sResponse Time (total)848.56s
Data parsing and extraction
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.05sResponse Time (max)1.58sResponse Time (total)2.09s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)53.62sResponse Time (max)102.40sResponse Time (total)160.87s
General Intelligence
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)770msResponse Time (max)770msResponse Time (total)770ms
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.39sResponse Time (max)1.39sResponse Time (total)1.39s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)14.60sResponse Time (max)14.60sResponse Time (total)14.60s
Agentic
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)381msResponse Time (max)381msResponse Time (total)381ms
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.00sResponse Time (max)11.34sResponse Time (total)27.99s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)58.38sResponse Time (max)68.45sResponse Time (total)175.15s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)62.60sResponse Time (max)78.52sResponse Time (total)125.21s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.40sResponse Time (max)9.53sResponse Time (total)14.80s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.API error: 2Wrong answer: 1Response Time (avg)189.55sResponse Time (max)287.79sResponse Time (total)568.65s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.06sResponse Time (max)6.06sResponse Time (total)6.06s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)8.97sResponse Time (max)14.45sResponse Time (total)26.92s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.08sResponse Time (max)8.08sResponse Time (total)8.08s
Trivia
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)41.30sResponse Time (max)41.30sResponse Time (total)41.30s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)85.76sResponse Time (max)85.76sResponse Time (total)85.76s
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)14.97sResponse Time (max)39.34sResponse Time (total)59.87s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)391.89sResponse Time (max)707.01sResponse Time (total)783.78s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)34.99sResponse Time (max)36.31sResponse Time (total)69.98s
Domain specific
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)289.43sResponse Time (max)392.05sResponse Time (total)868.29s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)41.23sResponse Time (max)41.23sResponse Time (total)41.23s
Puzzle Solving
: 5.4 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)55.18sResponse Time (max)76.48sResponse Time (total)165.54s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)24.21sResponse Time (max)24.21sResponse Time (total)24.21s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)88.77sResponse Time (max)88.77sResponse Time (total)88.77s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)139.94sResponse Time (max)139.94sResponse Time (total)139.94s
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)35.07sResponse Time (max)82.43sResponse Time (total)140.28s
Combined
: 7.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)144.87sResponse Time (max)236.85sResponse Time (total)289.75s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)24.28sResponse Time (max)34.05sResponse Time (total)48.55s
Domain specific
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 2No answer: 1Response Time (avg)419.73sResponse Time (max)675.30sResponse Time (total)1259.19s
General Intelligence
: 4.6 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)36.45sResponse Time (max)36.45sResponse Time (total)36.45s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)11.26sResponse Time (max)19.26sResponse Time (total)22.53s
Puzzle Solving
: 7.8 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)15.91sResponse Time (max)17.11sResponse Time (total)47.73s
Tool Calling
: 4.7 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)35.09sResponse Time (max)35.09sResponse Time (total)35.09s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)52.23sResponse Time (max)52.23sResponse Time (total)52.23s
Agentic
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)79.95sResponse Time (max)79.95sResponse Time (total)79.95s
Coding
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)13.42sResponse Time (max)22.57sResponse Time (total)40.25s
Combined
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Wrong answer: 1Response Time (avg)78.38sResponse Time (max)132.84sResponse Time (total)156.76s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.53sResponse Time (max)6.63sResponse Time (total)9.05s
Domain specific
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)1.91sResponse Time (max)4.51sResponse Time (total)5.72s
General Intelligence
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.82sResponse Time (max)2.82sResponse Time (total)2.82s
Instructions following
: 6.2 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)1.80sResponse Time (max)2.04sResponse Time (total)3.60s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)10.04sResponse Time (max)15.52sResponse Time (total)30.11s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)48.81sResponse Time (max)48.81sResponse Time (total)48.81s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)5.36sResponse Time (max)5.36sResponse Time (total)5.36s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)90.18sResponse Time (max)90.18sResponse Time (total)90.18s
Anti-AI Tricks
: 8.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)39.56sResponse Time (max)120.09sResponse Time (total)158.26s
Coding
: 7.0 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)109.70sResponse Time (max)141.71sResponse Time (total)329.11s
Combined
: 7.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)99.12sResponse Time (max)173.03sResponse Time (total)198.23s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)24.88sResponse Time (max)30.28sResponse Time (total)49.75s
Domain specific
: 4.1 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)238.45sResponse Time (max)485.92sResponse Time (total)715.34s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)12.20sResponse Time (max)21.22sResponse Time (total)24.41s
Puzzle Solving
: 9.9 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)12.50sResponse Time (max)20.68sResponse Time (total)37.50s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)55.56sResponse Time (max)55.56sResponse Time (total)55.56s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)10.02sResponse Time (max)10.02sResponse Time (total)10.02s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)149.50sResponse Time (max)149.50sResponse Time (total)149.50s
Anti-AI Tricks
: 8.4 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)10.34sResponse Time (max)35.62sResponse Time (total)41.36s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)12.09sResponse Time (max)21.31sResponse Time (total)36.27s
Combined
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Timed out: 1Response Time (avg)291.12sResponse Time (max)557.96sResponse Time (total)582.25s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.43sResponse Time (max)2.31sResponse Time (total)2.87s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)13.96sResponse Time (max)40.61sResponse Time (total)41.88s
Puzzle Solving
: 3.1 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)75.99sResponse Time (max)116.19sResponse Time (total)227.96s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.28sResponse Time (max)2.28sResponse Time (total)2.28s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.24sResponse Time (max)2.24sResponse Time (total)2.24s
Agentic
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)105.20sResponse Time (max)105.20sResponse Time (total)105.20s
Anti-AI Tricks
: 8.4 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)64.79sResponse Time (max)230.24sResponse Time (total)259.15s
Coding
: 4.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)320.43sResponse Time (max)357.05sResponse Time (total)961.28s
Combined
: 2.9 A test is fully passed only if every run passed for that test.Extra formatting: 1Invalid tool call: 1Response Time (avg)554.89sResponse Time (max)786.72sResponse Time (total)1109.78s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)24.06sResponse Time (max)26.90sResponse Time (total)48.13s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Timed out: 1Response Time (avg)107.19sResponse Time (max)195.94sResponse Time (total)321.57s
General Intelligence
: 5.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)25.08sResponse Time (max)25.08sResponse Time (total)25.08s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)15.43sResponse Time (max)28.25sResponse Time (total)30.85s
Puzzle Solving
: 3.3 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)19.70sResponse Time (max)36.03sResponse Time (total)59.10s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.93sResponse Time (max)3.93sResponse Time (total)3.93s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)305.02sResponse Time (max)305.02sResponse Time (total)305.02s
Agentic
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)18.87sResponse Time (max)18.87sResponse Time (total)18.87s
Anti-AI Tricks
: 3.0 A test is fully passed only if every run passed for that test.Extra formatting: 2Wrong answer: 2Response Time (avg)22.48sResponse Time (max)51.34sResponse Time (total)89.90s
Coding
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)21.96sResponse Time (max)44.81sResponse Time (total)65.89s
Combined
: 3.2 A test is fully passed only if every run passed for that test.Invalid tool call: 2Response Time (avg)96.20sResponse Time (max)159.85sResponse Time (total)192.40s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)28.00sResponse Time (max)42.87sResponse Time (total)55.99s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)14.73sResponse Time (max)32.97sResponse Time (total)44.20s
General Intelligence
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)34.77sResponse Time (max)34.77sResponse Time (total)34.77s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)30.68sResponse Time (max)37.37sResponse Time (total)61.35s
Tool Calling
: 9.5 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.64sResponse Time (max)3.64sResponse Time (total)3.64s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)37.37sResponse Time (max)37.37sResponse Time (total)37.37s
A test is fully passed only if every run passed for that test.API error: 6Wrong answer: 3Invalid tool call: 1Response Time (avg)34.90sResponse Time (max)121.29sResponse Time (total)802.79s…
Agentic
: 4.7 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)121.29sResponse Time (max)121.29sResponse Time (total)121.29s
Anti-AI Tricks
: 8.2 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)22.18sResponse Time (max)60.05sResponse Time (total)88.72s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)37.91sResponse Time (max)60.34sResponse Time (total)113.74s
Combined
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)51.45sResponse Time (max)93.10sResponse Time (total)102.91s
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)34.82sResponse Time (max)62.94sResponse Time (total)69.64s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)56.17sResponse Time (max)77.05sResponse Time (total)168.51s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.67sResponse Time (max)5.67sResponse Time (total)5.67s
Instructions following
: 6.2 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)32.73sResponse Time (max)60.73sResponse Time (total)65.46s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.27sResponse Time (max)4.30sResponse Time (total)9.82s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)26.43sResponse Time (max)26.43sResponse Time (total)26.43s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)30.60sResponse Time (max)30.60sResponse Time (total)30.60s
A test is fully passed only if every run passed for that test.Wrong answer: 10API error: 5Response Time (avg)22.65sResponse Time (max)93.12sResponse Time (total)520.91s…
Agentic
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)92.87sResponse Time (max)92.87sResponse Time (total)92.87s
Anti-AI Tricks
: 5.2 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)22.37sResponse Time (max)65.56sResponse Time (total)89.48s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.94sResponse Time (max)3.25sResponse Time (total)5.81s
Combined
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)49.01sResponse Time (max)93.12sResponse Time (total)98.01s
Data parsing and extraction
: 6.4 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)35.25sResponse Time (max)68.56sResponse Time (total)70.50s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)21.61sResponse Time (max)60.47sResponse Time (total)64.83s
General Intelligence
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.28sResponse Time (max)2.28sResponse Time (total)2.28s
Instructions following
: 6.4 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)31.00sResponse Time (max)59.73sResponse Time (total)62.01s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.54sResponse Time (max)4.51sResponse Time (total)7.63s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)26.44sResponse Time (max)26.44sResponse Time (total)26.44s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.05sResponse Time (max)1.05sResponse Time (total)1.05s
Global benchmark trends
See how current models trade score for cost, speed, output length, and category strength.