Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

IBM: Granite 4.1 8B vs Mistral: Mistral Small 4

Summary

Granite 4.1 8B vs Mistral Small 4 benchmark comparison: Mistral Small 4 leads on average score with 5.1 vs 4.0. Granite 4.1 8B has the lower benchmark cost at $0.003 vs $0.068. Granite 4.1 8B is faster at 728ms vs 9.40s, with pass rates of 9.5% vs 44.4%.

Recommended model: Granite 4.1 8B - It offers the best overall trade-off: a competitive score (4.0), lower cost than Mistral Small 4, and balanced response time.

Last updated at: 2026-06-12

Metric Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01 Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16
Score 4.0 5.1
Rank #163 #138
Reliability 10.0 10.0
Consistency 10.0 6.9
Tests Correct
Attempt pass rate 9.5% 44.4%
Flaky tests 0 8
Total Runs 63 63
Cost per result 0.131 1.344
Total Cost $0.003 $0.068
Input Price $0.050 / 1M $0.150 / 1M
Output Price $0.100 / 1M $0.600 / 1M
Total Input Tokens 46,285 42,576
Output Tokens 2,911 24,184
Reasoning Tokens 0 84,678
Response Time (avg) 728ms 9.40s
Response Time (max) 2.17s 59.15s
Response Time (total) 15.29s 197.39s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#163 IBM: Granite 4.1 8B

none
Cost
$0.001
Time
3.2s
Tokens
491 tok

#138 Mistral Small 4

medium
Cost
$0.006
Time
47.9s
Tokens
9,857 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 4.9 10.0 25.0% 0 844ms 645 903 0
Mistral Small 4 5.6 3.8 66.7% 3 2.67s 708 4,055 4,778
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 4.5 10.0 0.0% 0 775ms 8,344 525 0
Mistral Small 4 4.4 5.1 33.3% 2 39.98s 7,636 11,635 54,715
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 1.88s 19,089 396 0
Mistral Small 4 3.0 10.0 0.0% 0 25.25s 18,706 2,612 10,700
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 575ms 7,617 195 0
Mistral Small 4 7.3 5.9 83.3% 1 1.23s 6,171 335 723
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 357ms 768 24 0
Mistral Small 4 5.3 7.2 44.4% 1 6.11s 742 2,621 6,904
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 4.0 10.0 0.0% 0 499ms 528 115 0
Mistral Small 4 4.8 10.0 0.0% 0 2.05s 519 821 828
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.6 9.9 0.0% 0 344ms 687 66 0
Mistral Small 4 7.3 5.8 83.3% 1 1.38s 729 540 1,031
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.2 10.0 0.0% 0 608ms 672 432 0
Mistral Small 4 3.4 9.7 0.0% 0 2.17s 735 1,226 2,632
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 10.0 10.0 100.0% 0 2.17s 7,719 243 0
Mistral Small 4 10.0 10.0 100.0% 0 3.50s 6,420 321 810
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 306ms 216 12 0
Mistral Small 4 3.0 10.0 0.0% 0 5.92s 210 18 1,557

Quick Compare

Switch Comparison Pair

Mistral Small 4mediumvsMiMo-V2.5noneMistral Small 4mediumvsQwen3.5-9BnoneMistral Small 4mediumvsQwen3 Coder NextnoneMistral Small 4mediumvsQwen3.6 35B A3BnoneTrinity Large PreviewnonevsMistral Small 4mediumMistral Small 4mediumvsGPT-4o-mininoneDeepSeek V3.2nonevsMistral Small 4mediumMistral Small 4mediumvsQwen3.5-122B-A10BnoneMistral Small 4mediumvsGPT-5.4 MininoneMistral Small 4mediumvsGrok 4.20noneGranite 4.1 8BnonevsQwen3.5-9BmediumMistral Small 4mediumvsGLM 5 Turbonone