Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

IBM: Granite 4.1 8B vs Mistral: Mistral Small 4

Last updated at: 2026-05-29

Metric Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01 Mistral Small 4 Mistral Small 4 none Release: 2026-03-16
Score 4.1 5.0
Rank #158 #140
Reliability 10.0 10.0
Consistency 10.0 9.5
Tests Correct
Attempt pass rate 10.0% 28.3%
Flaky tests 0 1
Total Runs 60 60
Cost per result 0.122 0.131
Total Cost $0.003 $0.007
Input Price $0.050 / 1M $0.150 / 1M
Output Price $0.100 / 1M $0.600 / 1M
Output Tokens 2,743 2,192
Reasoning Tokens 0 0
Response Time (avg) 719ms 629ms
Response Time (max) 2.17s 1.72s
Response Time (total) 14.37s 12.59s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.9 10.0 25.0% 0 844ms 903 0
Mistral Small 4 3.4 7.9 16.7% 1 395ms 182 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 5.2 10.0 0.0% 0 706ms 357 0
Mistral Small 4 4.0 9.5 0.0% 0 1.03s 610 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 1.88s 396 0
Mistral Small 4 3.0 10.0 0.0% 0 1.72s 496 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 575ms 195 0
Mistral Small 4 10.0 10.0 100.0% 0 822ms 261 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 357ms 24 0
Mistral Small 4 5.3 10.0 33.3% 0 367ms 28 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.0 10.0 0.0% 0 499ms 115 0
Mistral Small 4 4.0 10.0 0.0% 0 729ms 205 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.6 9.9 0.0% 0 344ms 66 0
Mistral Small 4 6.5 10.0 50.0% 0 380ms 69 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.2 10.0 0.0% 0 608ms 432 0
Mistral Small 4 3.1 9.9 0.0% 0 399ms 111 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 10.0 10.0 100.0% 0 2.17s 243 0
Mistral Small 4 10.0 10.0 100.0% 0 1.40s 213 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 306ms 12 0
Mistral Small 4 3.0 10.0 0.0% 0 397ms 17 0

Quick Compare

Switch Comparison Pair

Granite 4.1 8BnonevsQwen3.5-9BmediumMistral Small 4nonevsQwen3 Coder NextmediumGranite 4.1 8BnonevsGLM 4.7 FlashmediumMiniMax M2.7mediumvsMistral Small 4noneMistral Small 4nonevsElephant AlphamediumMiniMax M2.5mediumFree AvailablevsMistral Small 4noneMistral Small 4nonevsGLM 4.7 FlashmediumGranite 4.1 8BnonevsQwen3 Coder NextmediumMistral Small 4nonevsQwen3.5-9BmediumCobuddymediumvsMistral Small 4noneMistral Small 4nonevsOwl AlphamediumMistral Small 4nonevsgpt-oss-120bmediumFree Available