Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

IBM: Granite 4.1 8B vs Qwen: Qwen3.5-9B

Last updated at: 2026-05-01

Metric Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Score 4.3 4.4
Rank #135 #132
Reliability 10.0 N/A
Consistency 10.0 7.2
Tests Correct
Attempt pass rate 11.1% 33.3%
Flaky tests 0 6
Total Runs 54 54
Cost per result 0.110 0.976
Total Cost $0.003 $0.030
Input Price $0.050 / 1M $0.100 / 1M
Output Price $0.100 / 1M $0.150 / 1M
Output Tokens 2,712 24,291
Reasoning Tokens 0 172,597
Response Time (avg) 767ms 73.64s
Response Time (max) 2.17s 226.38s
Response Time (total) 13.81s 1104.60s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.9 10.0 25.0% 0 844ms 903 0
Qwen3.5-9B 5.1 5.8 50.0% 2 34.44s 2,621 12,411
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 6.9 10.0 0.0% 0 1.07s 339 0
Qwen3.5-9B 2.6 10.0 0.0% 0 135.61s 6,150 31,393
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 1.88s 396 0
Qwen3.5-9B 3.0 10.0 0.0% 0 0ms 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 575ms 195 0
Qwen3.5-9B 3.6 5.6 33.3% 1 87.31s 1,383 32,113
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 357ms 24 0
Qwen3.5-9B 3.6 7.2 22.2% 1 137.75s 11,549 48,475
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.0 10.0 0.0% 0 499ms 115 0
Qwen3.5-9B 2.8 1.6 33.3% 1 226.38s 0 30,695
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.6 9.9 0.0% 0 344ms 66 0
Qwen3.5-9B 6.4 5.8 66.7% 1 17.15s 599 4,517
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.7 10.0 0.0% 0 635ms 431 0
Qwen3.5-9B 3.1 10.0 0.0% 0 33.38s 1,545 11,844
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 10.0 10.0 100.0% 0 2.17s 243 0
Qwen3.5-9B 10.0 10.0 100.0% 0 4.31s 444 1,149

Quick Compare

Switch Comparison Pair

Nemotron 3 Nano Omni 30b A3b ReasoningnoneFree AvailablevsQwen3.5-9BmediumLing 2.6 1tnoneFree AvailablevsQwen3.5-9BmediumQwen3.5-9BmediumvsGrok 4.1 FastnoneQwen3.5-9BmediumvsMiMo-V2-FlashnoneGPT-5.4 NanononevsQwen3.5-9BmediumQwen3.5-9BmediumvsHY3 PreviewnoneFree AvailableGranite 4.1 8BnonevsGLM 4.7 FlashmediumMercury 2nonevsQwen3.5-9BmediumGranite 4.1 8BnonevsQwen3 Coder NextmediumLaguna Xs.2noneFree AvailablevsQwen3.5-9BmediumLaguna M.1noneFree AvailablevsQwen3.5-9BmediumGPT-4o-mininonevsQwen3.5-9Bmedium