Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

IBM: Granite 4.1 8B vs Qwen: Qwen3 Coder Next

Last updated at: 2026-05-01

Metric Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01 Qwen3 Coder Next Qwen3 Coder Next medium Release: 2026-02-03
Score 4.3 4.9
Rank #135 #123
Reliability 10.0 N/A
Consistency 10.0 8.7
Tests Correct
Attempt pass rate 11.1% 33.3%
Flaky tests 0 3
Total Runs 54 54
Cost per result 0.110 0.180
Total Cost $0.003 $0.008
Input Price $0.050 / 1M $0.140 / 1M
Output Price $0.100 / 1M $0.800 / 1M
Output Tokens 2,712 3,241
Reasoning Tokens 0 0
Response Time (avg) 767ms 10.75s
Response Time (max) 2.17s 81.80s
Response Time (total) 13.81s 129.01s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.9 10.0 25.0% 0 844ms 903 0
Qwen3 Coder Next 3.5 8.1 16.7% 1 8.64s 1,252 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 6.9 10.0 0.0% 0 1.07s 339 0
Qwen3 Coder Next 4.7 1.6 66.7% 1 1.69s 300 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 1.88s 396 0
Qwen3 Coder Next 3.0 10.0 0.0% 0 4.28s 317 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 575ms 195 0
Qwen3 Coder Next 6.5 10.0 50.0% 0 81.80s 246 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 357ms 24 0
Qwen3 Coder Next 5.3 10.0 33.3% 0 638ms 25 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.0 10.0 0.0% 0 499ms 115 0
Qwen3 Coder Next 6.3 3.4 66.7% 1 1.39s 142 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.6 9.9 0.0% 0 344ms 66 0
Qwen3 Coder Next 6.3 10.0 50.0% 0 7.34s 63 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.7 10.0 0.0% 0 635ms 431 0
Qwen3 Coder Next 3.1 10.0 0.0% 0 2.30s 641 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 10.0 10.0 100.0% 0 2.17s 243 0
Qwen3 Coder Next 10.0 10.0 100.0% 0 2.64s 255 0

Quick Compare

Switch Comparison Pair

Granite 4.1 8BnonevsQwen3.5-9BmediumMercury 2nonevsQwen3 Coder NextmediumLaguna Xs.2noneFree AvailablevsQwen3 Coder NextmediumLaguna M.1noneFree AvailablevsQwen3 Coder NextmediumGPT-4o-mininonevsQwen3 Coder NextmediumQwen3 Coder NextmediumvsMiMo-V2.5noneQwen3 Coder NextmediumvsHY3 PreviewnoneFree AvailableGPT-5.4 MininonevsQwen3 Coder NextmediumGPT-5.4 NanononevsQwen3 Coder NextmediumElephant AlphanonevsQwen3 Coder NextmediumMistral Small 4nonevsQwen3 Coder NextmediumNemotron 3 SupernoneFree AvailablevsQwen3 Coder Nextmedium