Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

IBM: Granite 4.1 8B vs Qwen: Qwen3.5-9B

Summary

Granite 4.1 8B vs Qwen3.5-9B benchmark comparison: Granite 4.1 8B leads on average score with 4.0 vs 3.8. Granite 4.1 8B has the lower benchmark cost at $0.003 vs $0.036. Granite 4.1 8B is faster at 728ms vs 82.24s, with pass rates of 9.5% vs 27.0%.

Recommended model: Granite 4.1 8B - It has the best score here (4.0), while costing about 13.7x less than Qwen3.5-9B.

Last updated at: 2026-06-12

Metric Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Score 4.0 3.8
Rank #163 #164
Reliability 10.0 6.7
Consistency 10.0 8.0
Tests Correct
Attempt pass rate 9.5% 27.0%
Flaky tests 0 5
Total Runs 63 63
Cost per result 0.131 1.187
Total Cost $0.003 $0.036
Input Price $0.050 / 1M $0.100 / 1M
Output Price $0.100 / 1M $0.150 / 1M
Total Input Tokens 46,285 17,070
Output Tokens 2,911 29,045
Reasoning Tokens 0 209,516
Response Time (avg) 728ms 82.24s
Response Time (max) 2.17s 226.38s
Response Time (total) 15.29s 1315.88s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#163 IBM: Granite 4.1 8B

none
Cost
$0.001
Time
3.2s
Tokens
491 tok

#164 Qwen3.5-9B

medium
Cost
$0.001
Time
35.9s
Tokens
3,030 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 4.9 10.0 25.0% 0 844ms 645 903 0
Qwen3.5-9B 5.1 5.8 50.0% 2 34.44s 369 2,621 12,411
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 4.5 10.0 0.0% 0 775ms 8,344 525 0
Qwen3.5-9B 2.9 10.0 0.0% 0 100.88s 2,396 7,890 41,129
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 1.88s 19,089 396 0
Qwen3.5-9B 3.0 10.0 0.0% 0 0ms 0 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 575ms 7,617 195 0
Qwen3.5-9B 3.6 5.6 33.3% 1 87.31s 4,722 1,383 32,113
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 357ms 768 24 0
Qwen3.5-9B 3.6 7.2 22.2% 1 137.75s 295 11,549 48,475
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 4.0 10.0 0.0% 0 499ms 528 115 0
Qwen3.5-9B 2.8 1.6 33.3% 1 226.38s 180 0 30,695
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.6 9.9 0.0% 0 344ms 687 66 0
Qwen3.5-9B 6.5 10.0 50.0% 0 5.75s 381 491 1,824
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.2 10.0 0.0% 0 608ms 672 432 0
Qwen3.5-9B 3.0 10.0 0.0% 0 32.27s 376 1,593 12,026
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 10.0 10.0 100.0% 0 2.17s 7,719 243 0
Qwen3.5-9B 10.0 10.0 100.0% 0 4.31s 8,283 444 1,149
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 306ms 216 12 0
Qwen3.5-9B 3.0 10.0 0.0% 0 177.02s 68 3,074 29,694

Quick Compare

Switch Comparison Pair

Granite 4.1 8BnonevsGLM 4.7 FlashmediumQwen3.5-9BmediumvsMiMo-V2-FlashnoneGranite 4.1 8BnonevsQwen3 Coder NextmediumGranite 4.1 8BnonevsMiniMax M2.5mediumMercury 2nonevsQwen3.5-9BmediumLing-2.6-1TnonevsQwen3.5-9BmediumCobuddymediumvsGranite 4.1 8Bnonegpt-oss-120bnoneFree AvailablevsQwen3.5-9BmediumGPT-5.4 NanononevsQwen3.5-9BmediumRing-2.6-1TnonevsQwen3.5-9BmediumLing-2.6-flashnonevsQwen3.5-9BmediumNemotron 3 SupernoneFree AvailablevsQwen3.5-9Bmedium