Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Compared models

Last updated at: 2026-03-18

Metric MiniMax M2.7 MiniMax M2.7 medium Release: 2026-03-18 MiniMax M2.5 MiniMax M2.5 medium Release: 2026-02-12 Free Available GLM 5 GLM 5 medium Release: 2026-02-12 Gemini 3.1 Flash Lite Preview Gemini 3.1 Flash Lite Preview medium Release: 2026-03-03
Score 5.0 5.9 8.3 8.1
Rank #64 #50 #11 #14
Consistency 5.3 5.4 8.1 10.0
Tests Correct
Attempt pass rate 49.0% 60.8% 84.3% 70.6%
Flaky tests 10 10 4 0
Total Runs 51 51 51 51
Cost per result 2.398 4.987 0.896 0.413
Total Cost $0.072 $0.250 $0.108 $0.050
Input Price $0.300 / 1M $0.200 / 1M $0.720 / 1M $0.250 / 1M
Output Price $1.200 / 1M $1.200 / 1M $2.300 / 1M $1.500 / 1M
Output Tokens 4,517 107,044 19,833 1,737
Reasoning Tokens 47,612 206,422 38,523 26,346
Response Time (avg) 27.32s 39.65s 17.15s 3.70s
Response Time (max) 117.04s 237.27s 28.96s 14.93s
Response Time (total) 437.10s 396.47s 154.32s 62.97s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 7.9 6.3 83.3% 2 40.32s 3,010 17,716
MiniMax M2.5 7.9 6.3 83.3% 2 20.82s 286 45,344
GLM 5 10.0 10.0 100.0% 0 23.66s 480 7,056
Gemini 3.1 Flash Lite Preview 9.1 10.0 75.0% 0 2.33s 570 4,305
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 4.7 1.6 66.7% 1 41.03s 369 4,480
MiniMax M2.5 4.5 2.1 66.7% 1 60.39s 740 9,713
GLM 5 10.0 10.0 100.0% 0 28.96s 662 3,242
Gemini 3.1 Flash Lite Preview 10.0 10.0 100.0% 0 14.93s 327 7,347
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 6.3 5.8 66.7% 1 21.95s 187 5,882
MiniMax M2.5 4.6 1.7 66.7% 2 7.48s 266 3,835
GLM 5 7.1 5.6 83.3% 1 8.90s 567 3,734
Gemini 3.1 Flash Lite Preview 10.0 10.0 100.0% 0 2.29s 279 2,952
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 3.0 10.0 0.0% 0 19.00s 8 2,796
MiniMax M2.5 2.9 4.4 22.2% 2 237.27s 105,047 133,487
GLM 5 3.5 4.4 33.3% 2 0ms 13,176 14,137
Gemini 3.1 Flash Lite Preview 3.0 10.0 0.0% 0 4.21s 18 5,325
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 3.9 2.5 33.3% 1 38.70s 92 5,204
MiniMax M2.5 3.8 2.5 33.3% 1 6.63s 25 1,686
GLM 5 6.1 3.1 66.7% 1 14.69s 2,020 2,248
Gemini 3.1 Flash Lite Preview 10.0 10.0 100.0% 0 3.16s 96 1,488
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 3.7 1.8 50.0% 2 12.64s 213 2,457
MiniMax M2.5 8.1 6.8 83.3% 1 4.64s 252 1,873
GLM 5 10.0 10.0 100.0% 0 7.25s 1,001 2,129
Gemini 3.1 Flash Lite Preview 10.0 10.0 100.0% 0 1.91s 72 2,121
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 3.8 4.5 33.3% 2 25.62s 334 8,076
MiniMax M2.5 5.3 7.2 44.4% 1 11.54s 159 9,547
GLM 5 10.0 10.0 100.0% 0 15.64s 1,694 4,983
Gemini 3.1 Flash Lite Preview 7.7 10.0 66.7% 0 3.58s 141 1,896
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
MiniMax M2.7 4.7 1.6 66.7% 1 12.05s 304 1,001
MiniMax M2.5 10.0 10.0 100.0% 0 15.35s 269 937
GLM 5 10.0 10.0 100.0% 0 15.93s 233 994
Gemini 3.1 Flash Lite Preview 10.0 10.0 100.0% 0 3.80s 234 912

Quick Compare

Switch Comparison Pair

MiniMax M2.5mediumFree AvailablevsQwen3.5-35B-A3BnoneMiniMax M2.7mediumvsQwen3 Coder NextnoneMiniMax M2.5mediumFree AvailablevsHunter AlphanoneGemini 2.5 FlashnonevsMiniMax M2.5mediumFree AvailableMiniMax M2.7mediumvsGPT-4o-mininoneMiniMax M2.5mediumFree AvailablevsQwen3.5-122B-A10BnoneSeed-2.0-LitenonevsMiniMax M2.5mediumFree AvailableMiniMax M2.5mediumFree AvailablevsQwen3.5-FlashnoneMercury 2nonevsMiniMax M2.7mediumMiniMax M2.7mediumvsGPT-5.4 MininoneMiniMax M2.7mediumvsNemotron 3 Super 120b A12bnoneFree AvailableMiniMax M2.7mediumvsQwen3.5-9Bnone