Agentic
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)47.42sResponse Time (max)47.42sResponse Time (total)47.42s
Anti-AI Tricks
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)828msResponse Time (max)1.95sResponse Time (total)3.31s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.19sResponse Time (max)1.97sResponse Time (total)3.56s
Combined
: 3.0 A test is fully passed only if every run passed for that test.Invalid tool call: 1Wrong answer: 1Response Time (avg)24.10sResponse Time (max)44.76sResponse Time (total)48.21s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.14sResponse Time (max)1.18sResponse Time (total)2.27s
Domain specific
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)552msResponse Time (max)675msResponse Time (total)1.66s
General Intelligence
: 4.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.31sResponse Time (max)1.31sResponse Time (total)1.31s
Instructions following
: 6.3 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)582msResponse Time (max)633msResponse Time (total)1.16s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.75sResponse Time (max)2.75sResponse Time (total)2.75s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)703msResponse Time (max)703msResponse Time (total)703ms
~$0.011Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.1095 h × 0.300 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.010823.
5.71s…
A test is fully passed only if every run passed for that test.Wrong answer: 14No answer: 1Response Time (avg)5.71sResponse Time (max)65.25sResponse Time (total)131.36s…
Agentic
: 5.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)65.25sResponse Time (max)65.25sResponse Time (total)65.25s
Anti-AI Tricks
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.70sResponse Time (max)4.03sResponse Time (total)6.82s
Coding
: 5.5 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.64sResponse Time (max)2.70sResponse Time (total)4.91s
Combined
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Wrong answer: 1Response Time (avg)11.98sResponse Time (max)16.02sResponse Time (total)23.95s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.60sResponse Time (max)1.71sResponse Time (total)3.19s
Domain specific
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)5.38sResponse Time (max)15.16sResponse Time (total)16.15s
General Intelligence
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)1.09sResponse Time (max)1.09sResponse Time (total)1.09s
Instructions following
: 7.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)612msResponse Time (max)637msResponse Time (total)1.22s
Puzzle Solving
: 6.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)1.30sResponse Time (max)2.37sResponse Time (total)3.89s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)4.35sResponse Time (max)4.35sResponse Time (total)4.35s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)540msResponse Time (max)540msResponse Time (total)540ms
Agentic
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)173.66sResponse Time (max)173.66sResponse Time (total)173.66s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.73sResponse Time (max)18.55sResponse Time (total)26.91s
Coding
: 7.4 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)192.20sResponse Time (max)370.86sResponse Time (total)576.59s
Combined
: 7.3 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)92.24sResponse Time (max)143.77sResponse Time (total)184.49s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)9.24sResponse Time (max)9.63sResponse Time (total)18.48s
Domain specific
: 3.5 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)326.96sResponse Time (max)674.08sResponse Time (total)980.89s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.72sResponse Time (max)3.99sResponse Time (total)7.44s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)19.51sResponse Time (max)39.77sResponse Time (total)58.54s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.11sResponse Time (max)5.11sResponse Time (total)5.11s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)325.04sResponse Time (max)325.04sResponse Time (total)325.04s
Agentic
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)187.85sResponse Time (max)187.85sResponse Time (total)187.85s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.93sResponse Time (max)5.18sResponse Time (total)11.70s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)40.50sResponse Time (max)72.14sResponse Time (total)121.51s
Combined
: 8.7 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)124.48sResponse Time (max)214.63sResponse Time (total)248.96s
Data parsing and extraction
: 6.5 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)8.76sResponse Time (max)10.36sResponse Time (total)17.53s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)76.98sResponse Time (max)144.07sResponse Time (total)230.93s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.54sResponse Time (max)2.67sResponse Time (total)5.07s
Puzzle Solving
: 8.3 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)12.62sResponse Time (max)29.62sResponse Time (total)37.85s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.API error: 1Response Time (avg)0msResponse Time (max)0msResponse Time (total)0ms
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)11.29sResponse Time (max)11.29sResponse Time (total)11.29s
Agentic
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)211.32sResponse Time (max)211.32sResponse Time (total)211.32s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)3.02sResponse Time (max)5.68sResponse Time (total)12.07s
Coding
: 7.7 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)140.92sResponse Time (max)376.04sResponse Time (total)422.75s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)90.63sResponse Time (max)143.71sResponse Time (total)181.27s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.03sResponse Time (max)9.09sResponse Time (total)16.06s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.Wrong answer: 2Response Time (avg)60.78sResponse Time (max)120.94sResponse Time (total)182.34s
Instructions following
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.43sResponse Time (max)2.87sResponse Time (total)4.86s
Puzzle Solving
: 7.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)4.91sResponse Time (max)8.81sResponse Time (total)14.74s
Tool Calling
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)8.42sResponse Time (max)8.42sResponse Time (total)8.42s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)12.64sResponse Time (max)12.64sResponse Time (total)12.64s
Agentic
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)159.62sResponse Time (max)159.62sResponse Time (total)159.62s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)1.81sResponse Time (max)2.74sResponse Time (total)7.24s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)45.53sResponse Time (max)98.91sResponse Time (total)136.59s
Combined
: 6.4 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)76.72sResponse Time (max)116.08sResponse Time (total)153.43s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.65sResponse Time (max)7.50sResponse Time (total)13.31s
Domain specific
: 2.9 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)17.29sResponse Time (max)18.55sResponse Time (total)51.86s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.04sResponse Time (max)2.85sResponse Time (total)4.09s
Puzzle Solving
: 8.2 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)2.23sResponse Time (max)2.98sResponse Time (total)6.69s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.75sResponse Time (max)5.75sResponse Time (total)5.75s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)24.39sResponse Time (max)24.39sResponse Time (total)24.39s
Agentic
: 6.1 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)137.50sResponse Time (max)137.50sResponse Time (total)137.50s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.03sResponse Time (max)2.72sResponse Time (total)8.13s
Coding
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)29.46sResponse Time (max)49.90sResponse Time (total)88.39s
Combined
: 6.9 A test is fully passed only if every run passed for that test.Invalid tool call: 1Response Time (avg)98.91sResponse Time (max)166.72sResponse Time (total)197.82s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)6.20sResponse Time (max)6.23sResponse Time (total)12.40s
Domain specific
: 4.1 A test is fully passed only if every run passed for that test.Wrong answer: 3Response Time (avg)23.34sResponse Time (max)42.75sResponse Time (total)70.03s
General Intelligence
: 4.7 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)5.00sResponse Time (max)5.00sResponse Time (total)5.00s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)2.32sResponse Time (max)3.23sResponse Time (total)4.64s
Puzzle Solving
: 8.7 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)1.72sResponse Time (max)2.26sResponse Time (total)5.15s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.85sResponse Time (max)5.85sResponse Time (total)5.85s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)9.26sResponse Time (max)9.26sResponse Time (total)9.26s
Agentic
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)132.27sResponse Time (max)132.27sResponse Time (total)132.27s
Anti-AI Tricks
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.10sResponse Time (max)21.48sResponse Time (total)32.40s
Coding
: 8.1 A test is fully passed only if every run passed for that test.Did not follow instructions: 1Response Time (avg)248.62sResponse Time (max)458.25sResponse Time (total)745.85s
Combined
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)185.57sResponse Time (max)333.21sResponse Time (total)371.15s
Data parsing and extraction
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)34.85sResponse Time (max)57.71sResponse Time (total)69.71s
Domain specific
: 5.3 A test is fully passed only if every run passed for that test.No answer: 1Wrong answer: 1Response Time (avg)526.74sResponse Time (max)640.85sResponse Time (total)1580.21s
Instructions following
: 9.8 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)8.24sResponse Time (max)10.18sResponse Time (total)16.49s
Puzzle Solving
: 7.6 A test is fully passed only if every run passed for that test.No answer: 1Response Time (avg)199.90sResponse Time (max)569.33sResponse Time (total)599.69s
Tool Calling
: 10.0 A test is fully passed only if every run passed for that test.No failed answers.Response Time (avg)5.62sResponse Time (max)5.62sResponse Time (total)5.62s
Trivia
: 3.0 A test is fully passed only if every run passed for that test.Wrong answer: 1Response Time (avg)263.98sResponse Time (max)263.98sResponse Time (total)263.98s
Global benchmark trends
See how current models trade score for cost, speed, output length, and category strength.