#1 🥇 #1
d1-ultra
9.5… Liquid
$0.005
…
566ms…
171/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 9
Response Time (avg) 566ms Response Time (max) - Response Time (total) 8.49s
…
Decisions
180
Wrong decisions
9
Accuracy
95.00%
Attempts
15/15
Perfect attempts
12/15
Execution errors
0
Response Time (avg)
566ms
Response Time (total)
8.49s
Wrong answer: 9
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 683ms Response Time (max) - Response Time (total) 2.05s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 570ms Response Time (max) - Response Time (total) 1.71s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 450ms Response Time (max) - Response Time (total) 1.35s
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 576ms Response Time (max) - Response Time (total) 1.73s
Chemistry : 75.0%
27/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 553ms Response Time (max) - Response Time (total) 1.66s
#2 🥈 #2
Solar Decide
9.1… Upstage
$0.008
…
12.02s…
163/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 17
Response Time (avg) 12.02s Response Time (max) - Response Time (total) 180.34s
…
Decisions
180
Wrong decisions
17
Accuracy
90.56%
Attempts
15/15
Perfect attempts
4/15
Execution errors
0
Response Time (avg)
12.02s
Response Time (total)
180.34s
Wrong answer: 17
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 5.43s Response Time (max) - Response Time (total) 16.28s
Policy : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 9.07s Response Time (max) - Response Time (total) 27.22s
Evidence : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 14.19s Response Time (max) - Response Time (total) 42.56s
Injection : 75.0%
27/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 11.66s Response Time (max) - Response Time (total) 34.99s
Chemistry : 94.4%
34/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 19.76s Response Time (max) - Response Time (total) 59.29s
#3 🥉 #3
Span-02
9.0… Respan
$0.002
…
1.23s…
162/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 18
Response Time (avg) 1.23s Response Time (max) - Response Time (total) 18.42s
…
Decisions
180
Wrong decisions
18
Accuracy
90.00%
Attempts
15/15
Perfect attempts
12/15
Execution errors
0
Response Time (avg)
1.23s
Response Time (total)
18.42s
Wrong answer: 18
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 2.65s Response Time (max) - Response Time (total) 7.96s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 585ms Response Time (max) - Response Time (total) 1.75s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 472ms Response Time (max) - Response Time (total) 1.42s
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 512ms Response Time (max) - Response Time (total) 1.54s
Chemistry : 50.0%
18/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.92s Response Time (max) - Response Time (total) 5.75s
#4 #4
Jev 1.13
9.0… Typesafe
$0.002
…
482ms…
162/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 18
Response Time (avg) 482ms Response Time (max) - Response Time (total) 7.23s
…
Decisions
180
Wrong decisions
18
Accuracy
90.00%
Attempts
15/15
Perfect attempts
12/15
Execution errors
0
Response Time (avg)
482ms
Response Time (total)
7.23s
Wrong answer: 18
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 401ms Response Time (max) - Response Time (total) 1.20s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 401ms Response Time (max) - Response Time (total) 1.20s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 314ms Response Time (max) - Response Time (total) 943ms
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 402ms Response Time (max) - Response Time (total) 1.21s
Chemistry : 50.0%
18/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 891ms Response Time (max) - Response Time (total) 2.67s
#5 #5
Decider V1.1 27B
8.8… Perplexity
$0.003
…
345ms…
159/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 21
Response Time (avg) 345ms Response Time (max) - Response Time (total) 5.17s
…
Decisions
180
Wrong decisions
21
Accuracy
88.33%
Attempts
15/15
Perfect attempts
12/15
Execution errors
0
Response Time (avg)
345ms
Response Time (total)
5.17s
Wrong answer: 21
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 362ms Response Time (max) - Response Time (total) 1.09s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 335ms Response Time (max) - Response Time (total) 1.01s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 359ms Response Time (max) - Response Time (total) 1.08s
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 350ms Response Time (max) - Response Time (total) 1.05s
Chemistry : 41.7%
15/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 318ms Response Time (max) - Response Time (total) 954ms
#6 #6
Jev 1.14
8.7… Typesafe
$0.002
…
276ms…
156/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 24
Response Time (avg) 276ms Response Time (max) - Response Time (total) 4.14s
…
Decisions
180
Wrong decisions
24
Accuracy
86.67%
Attempts
15/15
Perfect attempts
9/15
Execution errors
0
Response Time (avg)
276ms
Response Time (total)
4.14s
Wrong answer: 24
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 270ms Response Time (max) - Response Time (total) 811ms
Policy : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 260ms Response Time (max) - Response Time (total) 781ms
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 239ms Response Time (max) - Response Time (total) 718ms
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 310ms Response Time (max) - Response Time (total) 931ms
Chemistry : 41.7%
15/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 300ms Response Time (max) - Response Time (total) 900ms
#7 #7
D1
8.7… Liquid
$0.005
…
886ms…
156/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 24
Response Time (avg) 886ms Response Time (max) - Response Time (total) 13.29s
…
Decisions
180
Wrong decisions
24
Accuracy
86.67%
Attempts
15/15
Perfect attempts
9/15
Execution errors
0
Response Time (avg)
886ms
Response Time (total)
13.29s
Wrong answer: 24
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.87s Response Time (max) - Response Time (total) 5.60s
Policy : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 486ms Response Time (max) - Response Time (total) 1.46s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 413ms Response Time (max) - Response Time (total) 1.24s
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.16s Response Time (max) - Response Time (total) 3.48s
Chemistry : 41.7%
15/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 505ms Response Time (max) - Response Time (total) 1.52s
#15 #15
Decider V1.2
7.3… Perplexity
$0.001
…
570ms…
132/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 24
Response Time (avg) 570ms Response Time (max) - Response Time (total) 8.54s
…
Decisions
180
Wrong decisions
24
Accuracy
73.33%
Attempts
15/15
Perfect attempts
10/15
Execution errors
2
Response Time (avg)
570ms
Response Time (total)
8.54s
Wrong answer: 24
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 617ms Response Time (max) - Response Time (total) 1.85s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 644ms Response Time (max) - Response Time (total) 1.93s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 540ms Response Time (max) - Response Time (total) 1.62s
Injection : 33.3%
12/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 458ms Response Time (max) - Response Time (total) 1.37s
Chemistry : 33.3%
12/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 589ms Response Time (max) - Response Time (total) 1.77s
#8 #8
GPT-6 Luna Decisions
8.5… OpenAI
$0.004
…
665ms…
153/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 27
Response Time (avg) 665ms Response Time (max) - Response Time (total) 9.97s
…
Decisions
180
Wrong decisions
27
Accuracy
85.00%
Attempts
15/15
Perfect attempts
12/15
Execution errors
0
Response Time (avg)
665ms
Response Time (total)
9.97s
Wrong answer: 27
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 475ms Response Time (max) - Response Time (total) 1.43s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 991ms Response Time (max) - Response Time (total) 2.97s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 636ms Response Time (max) - Response Time (total) 1.91s
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 549ms Response Time (max) - Response Time (total) 1.65s
Chemistry : 25.0%
9/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 672ms Response Time (max) - Response Time (total) 2.02s
#9 #9
Clef Omni
8.5… Cloudflare
$0.007
…
782ms…
153/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 27
Response Time (avg) 782ms Response Time (max) - Response Time (total) 11.73s
…
Decisions
180
Wrong decisions
27
Accuracy
85.00%
Attempts
15/15
Perfect attempts
9/15
Execution errors
0
Response Time (avg)
782ms
Response Time (total)
11.73s
Wrong answer: 27
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 911ms Response Time (max) - Response Time (total) 2.73s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 748ms Response Time (max) - Response Time (total) 2.25s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.08s Response Time (max) - Response Time (total) 3.25s
Injection : 83.3%
30/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 432ms Response Time (max) - Response Time (total) 1.30s
Chemistry : 41.7%
15/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 734ms Response Time (max) - Response Time (total) 2.20s
#10 #10
Nimble v3 9B Local run Candidate responses were generated on local hardware. OpenRouter was used only for judging. nimble-v3:9b-q8_0 · Q8_0 · Ollama 0.40.5 · NVIDIA GeForce RTX 3090.
8.3… Bespoke Labs
~$0.001
Estimated GPU electricity cost only. It excludes the rest of the computer and hardware ownership. NVIDIA GeForce RTX 3090: 0.0021 h × 0.377 kW × €0.2896/kWh × 1.138 USD/EUR = ~$0.000263.
829ms…
150/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 30
Response Time (avg) 829ms Response Time (max) - Response Time (total) 12.43s
…
Decisions
180
Wrong decisions
30
Accuracy
83.33%
Attempts
15/15
Perfect attempts
9/15
Execution errors
0
Response Time (avg)
829ms
Response Time (total)
12.43s
Wrong answer: 30
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 606ms Response Time (max) - Response Time (total) 1.82s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 612ms Response Time (max) - Response Time (total) 1.84s
Evidence : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 612ms Response Time (max) - Response Time (total) 1.84s
Injection : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 723ms Response Time (max) - Response Time (total) 2.17s
Chemistry : 25.0%
9/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.59s Response Time (max) - Response Time (total) 4.77s
#11 #11
Microsoft-Decision-1
8.3… Microsoft
$0.001
…
953ms…
150/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 30
Response Time (avg) 953ms Response Time (max) - Response Time (total) 14.29s
…
Decisions
180
Wrong decisions
30
Accuracy
83.33%
Attempts
15/15
Perfect attempts
6/15
Execution errors
0
Response Time (avg)
953ms
Response Time (total)
14.29s
Wrong answer: 30
Routing : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.28s Response Time (max) - Response Time (total) 3.85s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.07s Response Time (max) - Response Time (total) 3.20s
Evidence : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 910ms Response Time (max) - Response Time (total) 2.73s
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 584ms Response Time (max) - Response Time (total) 1.75s
Chemistry : 33.3%
12/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 918ms Response Time (max) - Response Time (total) 2.75s
#12 #12
Drex v1.5
8.0… Nace AI
$0.001
…
837ms…
144/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 36
Response Time (avg) 837ms Response Time (max) - Response Time (total) 12.55s
…
Decisions
180
Wrong decisions
36
Accuracy
80.00%
Attempts
15/15
Perfect attempts
6/15
Execution errors
0
Response Time (avg)
837ms
Response Time (total)
12.55s
Wrong answer: 36
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 911ms Response Time (max) - Response Time (total) 2.73s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 974ms Response Time (max) - Response Time (total) 2.92s
Evidence : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 993ms Response Time (max) - Response Time (total) 2.98s
Injection : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 528ms Response Time (max) - Response Time (total) 1.59s
Chemistry : 16.7%
6/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 778ms Response Time (max) - Response Time (total) 2.34s
#13 #13
Decider Flash V1.2
7.8… Perplexity
$0.001
…
508ms…
141/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 39
Response Time (avg) 508ms Response Time (max) - Response Time (total) 7.62s
…
Decisions
180
Wrong decisions
39
Accuracy
78.33%
Attempts
15/15
Perfect attempts
6/15
Execution errors
0
Response Time (avg)
508ms
Response Time (total)
7.62s
Wrong answer: 39
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 562ms Response Time (max) - Response Time (total) 1.69s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 572ms Response Time (max) - Response Time (total) 1.72s
Evidence : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 450ms Response Time (max) - Response Time (total) 1.35s
Injection : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 460ms Response Time (max) - Response Time (total) 1.38s
Chemistry : 8.3%
3/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 495ms Response Time (max) - Response Time (total) 1.49s
#14 #14
Kev 4B
7.8… Jaredpalmer
$0.001
…
998ms…
141/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 39
Response Time (avg) 998ms Response Time (max) - Response Time (total) 14.96s
…
Decisions
180
Wrong decisions
39
Accuracy
78.33%
Attempts
15/15
Perfect attempts
0/15
Execution errors
0
Response Time (avg)
998ms
Response Time (total)
14.96s
Wrong answer: 39
Routing : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 949ms Response Time (max) - Response Time (total) 2.85s
Policy : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 985ms Response Time (max) - Response Time (total) 2.96s
Evidence : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 856ms Response Time (max) - Response Time (total) 2.57s
Injection : 83.3%
30/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.12s Response Time (max) - Response Time (total) 3.35s
Chemistry : 33.3%
12/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.08s Response Time (max) - Response Time (total) 3.24s
#16 #16
Tev1 4B Experimental
7.0… Togethercomputer
$0.006
…
885ms…
126/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 54
Response Time (avg) 885ms Response Time (max) - Response Time (total) 13.27s
…
Decisions
180
Wrong decisions
54
Accuracy
70.00%
Attempts
15/15
Perfect attempts
3/15
Execution errors
0
Response Time (avg)
885ms
Response Time (total)
13.27s
Wrong answer: 54
Routing : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 874ms Response Time (max) - Response Time (total) 2.62s
Policy : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 876ms Response Time (max) - Response Time (total) 2.63s
Evidence : 91.7%
33/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 892ms Response Time (max) - Response Time (total) 2.68s
Injection : 66.7%
24/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 890ms Response Time (max) - Response Time (total) 2.67s
Chemistry : 0.0%
0/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 891ms Response Time (max) - Response Time (total) 2.67s
#17 #17
Classify-1
6.9… Scaledown
$0.009
…
4.75s…
124/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 56
Response Time (avg) 4.75s Response Time (max) - Response Time (total) 71.31s
…
Decisions
180
Wrong decisions
56
Accuracy
68.89%
Attempts
15/15
Perfect attempts
6/15
Execution errors
0
Response Time (avg)
4.75s
Response Time (total)
71.31s
Wrong answer: 56
Routing : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 6.51s Response Time (max) - Response Time (total) 19.52s
Policy : 83.3%
30/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 7.21s Response Time (max) - Response Time (total) 21.63s
Evidence : 52.8%
19/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 5.02s Response Time (max) - Response Time (total) 15.05s
Injection : 100.0%
36/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.29s Response Time (max) - Response Time (total) 3.88s
Chemistry : 8.3%
3/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 3.75s Response Time (max) - Response Time (total) 11.24s
#18 #18
Solar Decide Flash
5.9… Upstage
$0.008
…
2.04s…
106/180
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
Wrong answer: 74
Response Time (avg) 2.04s Response Time (max) - Response Time (total) 30.59s
…
Decisions
180
Wrong decisions
74
Accuracy
58.89%
Attempts
15/15
Perfect attempts
0/15
Execution errors
0
Response Time (avg)
2.04s
Response Time (total)
30.59s
Wrong answer: 74
Routing : 77.8%
28/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.89s Response Time (max) - Response Time (total) 5.66s
Policy : 75.0%
27/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 1.21s Response Time (max) - Response Time (total) 3.64s
Evidence : 83.3%
30/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 2.13s Response Time (max) - Response Time (total) 6.38s
Injection : 50.0%
18/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 2.00s Response Time (max) - Response Time (total) 5.99s
Chemistry : 8.3%
3/36
Ranked by correct decisions; cost breaks ties. A perfect attempt needs all 12 answers correct. Missing coverage receives no rank. Partial credit applies: 3 of 12 correct answers gives 25%.
No failed answers.
Response Time (avg) 2.97s Response Time (max) - Response Time (total) 8.92s