Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

IBM: Granite 4.1 8B vs OpenAI: GPT-5.4 Nano

Last updated at: 2026-05-01

Metric Granite 4.1 8B Granite 4.1 8B none Release: 2026-05-01 GPT-5.4 Nano GPT-5.4 Nano none Release: 2026-03-17
Score 4.3 4.6
Rank #135 #127
Reliability 10.0 N/A
Consistency 10.0 7.4
Tests Correct
Attempt pass rate 11.1% 33.3%
Flaky tests 0 6
Total Runs 54 54
Cost per result 0.110 0.299
Total Cost $0.003 $0.009
Input Price $0.050 / 1M $0.200 / 1M
Output Price $0.100 / 1M $1.250 / 1M
Output Tokens 2,712 2,762
Reasoning Tokens 0 0
Response Time (avg) 767ms 1.40s
Response Time (max) 2.17s 3.84s
Response Time (total) 13.81s 25.14s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.9 10.0 25.0% 0 844ms 903 0
GPT-5.4 Nano 3.5 8.0 16.7% 1 1.18s 800 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 6.9 10.0 0.0% 0 1.07s 339 0
GPT-5.4 Nano 7.1 3.7 66.7% 1 1.43s 577 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 1.88s 396 0
GPT-5.4 Nano 3.0 10.0 0.0% 0 3.84s 280 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 575ms 195 0
GPT-5.4 Nano 6.5 10.0 50.0% 0 1.11s 219 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.0 10.0 0.0% 0 357ms 24 0
GPT-5.4 Nano 2.9 4.4 22.2% 2 926ms 52 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 4.0 10.0 0.0% 0 499ms 115 0
GPT-5.4 Nano 3.8 2.5 33.3% 1 1.31s 180 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.6 9.9 0.0% 0 344ms 66 0
GPT-5.4 Nano 6.3 10.0 50.0% 0 787ms 84 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 3.7 10.0 0.0% 0 635ms 431 0
GPT-5.4 Nano 3.7 7.3 22.2% 1 1.29s 348 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Granite 4.1 8B 10.0 10.0 100.0% 0 2.17s 243 0
GPT-5.4 Nano 10.0 10.0 100.0% 0 3.40s 222 0

Quick Compare

Switch Comparison Pair

GPT-5.4 NanononevsGLM 4.7 FlashmediumGranite 4.1 8BnonevsQwen3.5-9BmediumGPT-5.4 NanononevsQwen3.5-9BmediumGPT-5.4 NanononevsQwen3 Coder NextmediumGranite 4.1 8BnonevsGLM 4.7 FlashmediumNemotron 3 Nano Omni 30b A3b ReasoningmediumFree AvailablevsGPT-5.4 NanononeGPT-5.4 NanononevsElephant AlphamediumGranite 4.1 8BnonevsQwen3 Coder NextmediumMiniMax M2.7mediumvsGPT-5.4 NanononeGranite 4.1 8BnonevsNemotron 3 Nano Omni 30b A3b ReasoningmediumFree AvailableGranite 4.1 8BnonevsElephant AlphamediumGranite 4.1 8BnonevsMiniMax M2.7medium