Agentic
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)21.61sResponse Time (max)21.61sResponse Time (total)21.61s
Coding
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)231.73sResponse Time (max)634.11sResponse Time (total)695.20s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1No answer: 1Response Time (avg)32.43sResponse Time (max)32.51sResponse Time (total)64.86s
Data parsing and extraction
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)12.34sResponse Time (max)14.35sResponse Time (total)24.68s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)5.73sResponse Time (max)7.07sResponse Time (total)17.19s
Tool Calling
: 4.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)24.08sResponse Time (max)24.08sResponse Time (total)24.08s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)12.12sResponse Time (max)12.12sResponse Time (total)12.12s
Agentic
: 5.2 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)152.85sResponse Time (max)152.85sResponse Time (total)152.85s
Anti-AI Tricks
: 5.1 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)34.44sResponse Time (max)57.86sResponse Time (total)103.31s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 2Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Timed out: 2No answer: 1Response Time (avg)272.50sResponse Time (max)569.97sResponse Time (total)817.51s
General Intelligence
: 2.8 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)226.38sResponse Time (max)226.38sResponse Time (total)226.38s
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)5.75sResponse Time (max)5.75sResponse Time (total)5.75s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)32.27sResponse Time (max)47.31sResponse Time (total)96.80s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.31sResponse Time (max)4.31sResponse Time (total)4.31s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)177.02sResponse Time (max)177.02sResponse Time (total)177.02s
Agentic
: 6.1 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)88.31sResponse Time (max)88.31sResponse Time (total)88.31s
Anti-AI Tricks
: 8.7 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)24.05sResponse Time (max)63.76sResponse Time (total)96.19s
Coding
: 8.2 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)72.23sResponse Time (max)102.69sResponse Time (total)216.69s
Combined
: 6.4 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)79.89sResponse Time (max)134.76sResponse Time (total)159.78s
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)36.10sResponse Time (max)64.17sResponse Time (total)72.19s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)175.49sResponse Time (max)255.06sResponse Time (total)526.47s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.49sResponse Time (max)4.49sResponse Time (total)4.49s
Instructions following
: 7.1 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)33.38sResponse Time (max)62.32sResponse Time (total)66.76s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.84sResponse Time (max)6.35sResponse Time (total)14.53s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)26.45sResponse Time (max)26.45sResponse Time (total)26.45s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)65.34sResponse Time (max)65.34sResponse Time (total)65.34s
A test is fully passed only if every run passed for that test.API error: 8Wrong answer: 3No answer: 1Response Time (avg)97.34sResponse Time (max)587.08sResponse Time (total)2238.73s…
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)62.53sResponse Time (max)62.53sResponse Time (total)62.53s
Anti-AI Tricks
: 8.7 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)23.74sResponse Time (max)63.86sResponse Time (total)94.97s
Coding
: 5.2 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)105.09sResponse Time (max)165.67sResponse Time (total)315.27s
Combined
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)64.37sResponse Time (max)93.11sResponse Time (total)128.74s
Data parsing and extraction
: 6.4 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)39.79sResponse Time (max)70.07sResponse Time (total)79.59s
Domain specific
: 3.6 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)471.21sResponse Time (max)587.08sResponse Time (total)1413.62s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.04sResponse Time (max)7.04sResponse Time (total)7.04s
Instructions following
: 6.2 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)32.97sResponse Time (max)60.73sResponse Time (total)65.94s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)6.57sResponse Time (max)7.54sResponse Time (total)19.72s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)26.36sResponse Time (max)26.36sResponse Time (total)26.36s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)24.95sResponse Time (max)24.95sResponse Time (total)24.95s
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)223.46sResponse Time (max)223.46sResponse Time (total)223.46s
Anti-AI Tricks
: 6.9 A test is fully passed only if every run passed for that test.Timed out: 2Response Time (avg)205.31sResponse Time (max)413.57sResponse Time (total)821.25s
Coding
: 3.8 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)515.34sResponse Time (max)567.74sResponse Time (total)1546.02s
Combined
: 5.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Timed out: 1Response Time (avg)570.43sResponse Time (max)685.87sResponse Time (total)1140.86s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Timed out: 1Response Time (avg)167.15sResponse Time (max)183.96sResponse Time (total)501.46s
General Intelligence
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)16.37sResponse Time (max)16.37sResponse Time (total)16.37s
Instructions following
: 6.4 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)87.47sResponse Time (max)150.79sResponse Time (total)174.94s
Puzzle Solving
: 2.9 A test is fully passed only if every run passed for that test.Timed out: 2No answer: 1Response Time (avg)202.69sResponse Time (max)252.93sResponse Time (total)608.07s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)15.08sResponse Time (max)15.08sResponse Time (total)15.08s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)252.49sResponse Time (max)252.49sResponse Time (total)252.49s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)91.69sResponse Time (max)91.69sResponse Time (total)91.69s
Anti-AI Tricks
: 6.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)45.98sResponse Time (max)130.73sResponse Time (total)183.93s
Coding
: 8.9 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)89.94sResponse Time (max)105.48sResponse Time (total)269.82s
Combined
: 4.7 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)102.33sResponse Time (max)177.88sResponse Time (total)204.66s
Data parsing and extraction
: 7.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)18.78sResponse Time (max)27.01sResponse Time (total)37.56s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)429.77sResponse Time (max)626.58sResponse Time (total)1289.31s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)10.34sResponse Time (max)15.43sResponse Time (total)20.67s
Puzzle Solving
: 8.2 A test is fully passed only if every run passed for that test.Extra formatting: 1Response Time (avg)34.30sResponse Time (max)46.61sResponse Time (total)102.91s
Tool Calling
: 4.7 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)36.36sResponse Time (max)36.36sResponse Time (total)36.36s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)66.56sResponse Time (max)66.56sResponse Time (total)66.56s
Agentic
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)579.54sResponse Time (max)579.54sResponse Time (total)579.54s
Anti-AI Tricks
: 3.8 A test is fully passed only if every run passed for that test.Wrong answer: 3API error: 1Response Time (avg)145.43sResponse Time (max)435.60sResponse Time (total)581.70s
Coding
: 4.2 A test is fully passed only if every run passed for that test.Timed out: 2Wrong answer: 1Response Time (avg)798.00sResponse Time (max)960.02sResponse Time (total)2394.01s
Combined
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Timed out: 1Response Time (avg)898.87sResponse Time (max)971.85sResponse Time (total)1797.74s
Data parsing and extraction
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)147.51sResponse Time (max)252.91sResponse Time (total)295.02s
General Intelligence
: 3.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)121.15sResponse Time (max)121.15sResponse Time (total)121.15s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)56.09sResponse Time (max)56.09sResponse Time (total)56.09s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)755.91sResponse Time (max)755.91sResponse Time (total)755.91s
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 3.2 A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)1.19sResponse Time (max)2.73sResponse Time (total)4.76s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.87sResponse Time (max)2.87sResponse Time (total)2.87s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)564msResponse Time (max)564msResponse Time (total)564ms
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)857msResponse Time (max)955msResponse Time (total)1.71s
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.86sResponse Time (max)2.70sResponse Time (total)3.71s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.28sResponse Time (max)2.28sResponse Time (total)2.28s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.82sResponse Time (max)1.82sResponse Time (total)1.82s
Agentic
: 4.7 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)52.36sResponse Time (max)52.36sResponse Time (total)52.36s
Anti-AI Tricks
: 6.0 A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)129.60sResponse Time (max)237.97sResponse Time (total)518.41s
Coding
: 5.2 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)219.15sResponse Time (max)259.22sResponse Time (total)657.45s
Combined
: 2.9 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)139.43sResponse Time (max)171.85sResponse Time (total)278.85s
Data parsing and extraction
: 8.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)88.53sResponse Time (max)120.56sResponse Time (total)177.06s
Domain specific
: 4.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)470.24sResponse Time (max)547.52sResponse Time (total)1410.71s
General Intelligence
: 7.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)110.43sResponse Time (max)110.43sResponse Time (total)110.43s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)77.72sResponse Time (max)91.04sResponse Time (total)155.43s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)93.67sResponse Time (max)93.67sResponse Time (total)93.67s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)597.45sResponse Time (max)597.45sResponse Time (total)597.45s
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)65.46sResponse Time (max)65.46sResponse Time (total)65.46s
Anti-AI Tricks
: 3.3 A test is fully passed only if every run passed for that test.Wrong answer: 3Extra formatting: 1Response Time (avg)31.54sResponse Time (max)41.19sResponse Time (total)126.18s
Combined
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Wrong answer: 1Response Time (avg)75.12sResponse Time (max)95.32sResponse Time (total)150.23s
Data parsing and extraction
: 8.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)22.81sResponse Time (max)29.63sResponse Time (total)45.63s
Domain specific
: 6.8 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)25.61sResponse Time (max)45.05sResponse Time (total)76.83s
General Intelligence
: 5.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)58.28sResponse Time (max)58.28sResponse Time (total)58.28s
Instructions following
: 7.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)47.90sResponse Time (max)50.25sResponse Time (total)95.81s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)81.53sResponse Time (max)81.53sResponse Time (total)81.53s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)19.87sResponse Time (max)19.87sResponse Time (total)19.87s
A test is fully passed only if every run passed for that test.API error: 22Wrong answer: 1Response Time (avg)91.72sResponse Time (max)112.76sResponse Time (total)2109.54s…
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)112.76sResponse Time (max)112.76sResponse Time (total)112.76s
Anti-AI Tricks
: 3.0 A test is fully passed only if every run passed for that test.API error: 4Response Time (avg)94.10sResponse Time (max)102.33sResponse Time (total)376.41s
Coding
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)93.13sResponse Time (max)94.47sResponse Time (total)279.38s
Combined
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)89.35sResponse Time (max)95.40sResponse Time (total)178.70s
Data parsing and extraction
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)92.87sResponse Time (max)97.01sResponse Time (total)185.73s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)77.87sResponse Time (max)91.53sResponse Time (total)233.61s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)80.14sResponse Time (max)80.14sResponse Time (total)80.14s
Instructions following
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)103.03sResponse Time (max)105.69sResponse Time (total)206.06s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)92.07sResponse Time (max)99.99sResponse Time (total)276.20s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)78.42sResponse Time (max)78.42sResponse Time (total)78.42s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)102.12sResponse Time (max)102.12sResponse Time (total)102.12s
A test is fully passed only if every run passed for that test.API error: 22Wrong answer: 1Response Time (avg)89.96sResponse Time (max)156.00sResponse Time (total)2069.01s…
Agentic
: 3.9 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)156.00sResponse Time (max)156.00sResponse Time (total)156.00s
Anti-AI Tricks
: 3.0 A test is fully passed only if every run passed for that test.API error: 4Response Time (avg)85.94sResponse Time (max)91.77sResponse Time (total)343.74s
Coding
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)92.74sResponse Time (max)101.39sResponse Time (total)278.22s
Combined
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)80.65sResponse Time (max)82.20sResponse Time (total)161.31s
Data parsing and extraction
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)86.20sResponse Time (max)89.21sResponse Time (total)172.39s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)67.89sResponse Time (max)70.95sResponse Time (total)203.68s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)99.30sResponse Time (max)99.30sResponse Time (total)99.30s
Instructions following
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)110.33sResponse Time (max)114.87sResponse Time (total)220.65s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)86.20sResponse Time (max)92.42sResponse Time (total)258.61s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)84.61sResponse Time (max)84.61sResponse Time (total)84.61s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)90.50sResponse Time (max)90.50sResponse Time (total)90.50s
A test is fully passed only if every run passed for that test.API error: 22Wrong answer: 1Response Time (avg)92.50sResponse Time (max)128.54sResponse Time (total)2127.40s…
Agentic
: 2.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)128.54sResponse Time (max)128.54sResponse Time (total)128.54s
Anti-AI Tricks
: 3.0 A test is fully passed only if every run passed for that test.API error: 4Response Time (avg)87.46sResponse Time (max)103.34sResponse Time (total)349.85s
Coding
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)86.43sResponse Time (max)92.24sResponse Time (total)259.28s
Combined
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)83.53sResponse Time (max)90.11sResponse Time (total)167.05s
Data parsing and extraction
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)85.39sResponse Time (max)99.05sResponse Time (total)170.77s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)82.74sResponse Time (max)103.01sResponse Time (total)248.23s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)101.81sResponse Time (max)101.81sResponse Time (total)101.81s
Instructions following
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)105.69sResponse Time (max)116.28sResponse Time (total)211.38s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)95.76sResponse Time (max)102.09sResponse Time (total)287.28s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)99.68sResponse Time (max)99.68sResponse Time (total)99.68s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)103.52sResponse Time (max)103.52sResponse Time (total)103.52s
A test is fully passed only if every run passed for that test.API error: 23Response Time (avg)77.87sResponse Time (max)106.37sResponse Time (total)1791.03s…
Agentic
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)1.05sResponse Time (max)1.05sResponse Time (total)1.05s
Anti-AI Tricks
: 3.0 A test is fully passed only if every run passed for that test.API error: 4Response Time (avg)90.54sResponse Time (max)106.21sResponse Time (total)362.18s
Coding
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)80.82sResponse Time (max)92.50sResponse Time (total)242.45s
Combined
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)46.61sResponse Time (max)51.89sResponse Time (total)93.22s
Data parsing and extraction
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)62.59sResponse Time (max)88.50sResponse Time (total)125.18s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)90.26sResponse Time (max)106.37sResponse Time (total)270.78s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)84.05sResponse Time (max)84.05sResponse Time (total)84.05s
Instructions following
: 3.0 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)95.73sResponse Time (max)97.43sResponse Time (total)191.46s
Puzzle Solving
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)88.10sResponse Time (max)94.19sResponse Time (total)264.30s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)81.86sResponse Time (max)81.86sResponse Time (total)81.86s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)74.52sResponse Time (max)74.52sResponse Time (total)74.52s
A test is fully passed only if every run passed for that test.Wrong answer: 3Did not follow instructions: 1Response Time (avg)9.75sResponse Time (max)31.36sResponse Time (total)175.48s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 8.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.16sResponse Time (max)3.44sResponse Time (total)12.65s
Coding
: 3.3 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)31.36sResponse Time (max)31.36sResponse Time (total)31.36s
Combined
: 5.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)20.93sResponse Time (max)20.93sResponse Time (total)20.93s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.01sResponse Time (max)4.27sResponse Time (total)8.02s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)21.33sResponse Time (max)24.21sResponse Time (total)64.00s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.78sResponse Time (max)5.78sResponse Time (total)5.78s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.89sResponse Time (max)5.89sResponse Time (total)9.78s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.52sResponse Time (max)4.53sResponse Time (total)10.57s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)12.39sResponse Time (max)12.39sResponse Time (total)12.39s
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
A test is fully passed only if every run passed for that test.API error: 4Wrong answer: 3Response Time (avg)9.05sResponse Time (max)26.24sResponse Time (total)90.53s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)14.99sResponse Time (max)26.24sResponse Time (total)29.99s
Coding
: 3.0 A test is fully passed only if every run passed for that test.API error: 3Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)10.37sResponse Time (max)10.37sResponse Time (total)10.37s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)10.84sResponse Time (max)10.84sResponse Time (total)10.84s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)7.01sResponse Time (max)7.01sResponse Time (total)7.01s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.34sResponse Time (max)9.34sResponse Time (total)9.34s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.26sResponse Time (max)3.26sResponse Time (total)3.26s
Puzzle Solving
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.88sResponse Time (max)4.23sResponse Time (total)7.77s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)11.96sResponse Time (max)11.96sResponse Time (total)11.96s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
A test is fully passed only if every run passed for that test.Wrong answer: 2Did not follow instructions: 1Response Time (avg)68.14sResponse Time (max)280.52sResponse Time (total)1090.28s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 7.5 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)43.87sResponse Time (max)121.88sResponse Time (total)131.62s
Coding
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 5.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)280.52sResponse Time (max)280.52sResponse Time (total)280.52s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.16sResponse Time (max)8.54sResponse Time (total)14.31s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)127.58sResponse Time (max)133.93sResponse Time (total)382.74s
General Intelligence
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.25sResponse Time (max)5.25sResponse Time (total)5.25s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)64.03sResponse Time (max)124.45sResponse Time (total)128.06s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)46.68sResponse Time (max)134.22sResponse Time (total)140.04s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)7.73sResponse Time (max)7.73sResponse Time (total)7.73s
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
A test is fully passed only if every run passed for that test.API error: 8Wrong answer: 2Response Time (avg)15.25sResponse Time (max)43.55sResponse Time (total)182.96s…
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 8.3 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)11.69sResponse Time (max)19.37sResponse Time (total)35.08s
Coding
: 1.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 5.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)34.95sResponse Time (max)34.95sResponse Time (total)34.95s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)14.95sResponse Time (max)15.40sResponse Time (total)29.90s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 2API error: 1Response Time (avg)22.08sResponse Time (max)43.55sResponse Time (total)66.23s
General Intelligence
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Instructions following
: 6.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)3.40sResponse Time (max)3.40sResponse Time (total)3.40s
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.API error: 2Response Time (avg)7.52sResponse Time (max)7.52sResponse Time (total)7.52s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.87sResponse Time (max)5.87sResponse Time (total)5.87s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 6.9 A test is fully passed only if every run passed for that test.Extra formatting: 1Wrong answer: 1Response Time (avg)3.46sResponse Time (max)4.38sResponse Time (total)13.86s
Coding
: 3.3 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)27.11sResponse Time (max)27.11sResponse Time (total)27.11s
Combined
: 1.5 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.54sResponse Time (max)7.51sResponse Time (total)11.08s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.52sResponse Time (max)3.80sResponse Time (total)7.04s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 7.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)4.75sResponse Time (max)7.62sResponse Time (total)19.00s
Coding
: 1.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 2.3 A test is fully passed only if every run passed for that test.Timed out: 1Response Time (avg)30.53sResponse Time (max)30.53sResponse Time (total)30.53s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)23.16sResponse Time (max)26.55sResponse Time (total)46.33s
Instructions following
: 9.9 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.18sResponse Time (max)4.46sResponse Time (total)8.36s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)17.33sResponse Time (max)17.33sResponse Time (total)17.33s
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 4.0 A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)597msResponse Time (max)866msResponse Time (total)2.39s
Coding
: 1.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.14sResponse Time (max)1.14sResponse Time (total)1.14s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)6.48sResponse Time (max)6.48sResponse Time (total)6.48s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)601msResponse Time (max)634msResponse Time (total)1.20s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)611msResponse Time (max)616msResponse Time (total)1.83s
Instructions following
: 6.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)649msResponse Time (max)952msResponse Time (total)1.30s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)586msResponse Time (max)813msResponse Time (total)1.76s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.79sResponse Time (max)4.79sResponse Time (total)4.79s
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Coding
: 4.2 A test is fully passed only if every run passed for that test.API error: 2Wrong answer: 1Response Time (avg)1.39sResponse Time (max)1.39sResponse Time (total)1.39s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.81sResponse Time (max)3.81sResponse Time (total)3.81s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.04sResponse Time (max)1.05sResponse Time (total)2.08s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)927msResponse Time (max)1.17sResponse Time (total)2.78s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.03sResponse Time (max)1.17sResponse Time (total)2.07s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)2.79sResponse Time (max)2.79sResponse Time (total)2.79s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 6.6 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.19sResponse Time (max)2.04sResponse Time (total)4.75s
Coding
: 3.7 A test is fully passed only if every run passed for that test.API error: 2Wrong answer: 1Response Time (avg)1.30sResponse Time (max)1.30sResponse Time (total)1.30s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)3.70sResponse Time (max)3.70sResponse Time (total)3.70s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)979msResponse Time (max)1.02sResponse Time (total)1.96s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)925msResponse Time (max)1.16sResponse Time (total)2.77s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)987msResponse Time (max)1.13sResponse Time (total)1.97s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)2.83sResponse Time (max)2.83sResponse Time (total)2.83s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 4Response Time (avg)3.81sResponse Time (max)6.85sResponse Time (total)15.23s
Coding
: 1.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Combined
: 1.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)15.17sResponse Time (max)15.17sResponse Time (total)15.17s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.49sResponse Time (max)14.02sResponse Time (total)16.98s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)2.33sResponse Time (max)2.94sResponse Time (total)6.99s
Instructions following
: 6.4 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.82sResponse Time (max)2.92sResponse Time (total)5.65s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.02sResponse Time (max)6.02sResponse Time (total)6.02s
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Agentic
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Anti-AI Tricks
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)501msResponse Time (max)839msResponse Time (total)2.01s
Coding
: 1.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.22sResponse Time (max)1.22sResponse Time (total)1.22s
Combined
: 1.5 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)6.04sResponse Time (max)6.04sResponse Time (total)6.04s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)522msResponse Time (max)537msResponse Time (total)1.04s
General Intelligence
: 4.8 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)659msResponse Time (max)659msResponse Time (total)659ms
Instructions following
: 6.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)445msResponse Time (max)505msResponse Time (total)889ms
Puzzle Solving
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)473msResponse Time (max)502msResponse Time (total)1.42s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.63sResponse Time (max)4.63sResponse Time (total)4.63s
Trivia
: 0.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Global benchmark trends
See how current models trade score for cost, speed, output length, and category strength.