Navigate
AI BENCHY
Your ad here

AI BENCHY Compare

Google: Gemma 4 26B A4B vs Qwen: Qwen3.5-27B

Last updated at: 2026-04-04

Metric Gemma 4 26B A4B Gemma 4 26B A4B medium Release: 2026-04-03 Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24
Score 8.3 8.3
Rank #13 #14
Consistency 8.9 8.7
Tests Correct
Attempt pass rate 80.4% 80.4%
Flaky tests 2 3
Total Runs 51 51
Cost per result 0.189 3.885
Total Cost $0.025 $0.467
Input Price $0.130 / 1M $0.195 / 1M
Output Price $0.400 / 1M $1.560 / 1M
Output Tokens 12,412 2,125
Reasoning Tokens 39,955 223,335
Response Time (avg) 17.38s 52.01s
Response Time (max) 73.55s 163.96s
Response Time (total) 278.01s 884.10s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 6.20s 1,142 3,045
Qwen3.5-27B 8.7 7.9 91.7% 1 19.75s 569 31,505
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 9.6 10.0 100.0% 0 73.55s 5,415 13,112
Qwen3.5-27B 10.0 10.0 100.0% 0 163.96s 483 9,991
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 16.51s 1,567 2,827
Qwen3.5-27B 10.0 10.0 100.0% 0 30.26s 270 16,150
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 2.9 4.4 22.2% 2 23.62s 2,469 7,105
Qwen3.5-27B 5.3 10.0 33.3% 0 79.53s 43 52,368
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 29.76s 25 5,075
Qwen3.5-27B 6.1 3.1 66.7% 1 101.41s 70 23,147
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 17.54s 887 4,470
Qwen3.5-27B 10.0 10.0 100.0% 0 19.66s 97 11,638
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 7.9 9.6 66.7% 0 8.52s 457 3,065
Qwen3.5-27B 8.2 7.7 77.8% 1 64.61s 245 77,213
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Gemma 4 26B A4B 10.0 10.0 100.0% 0 9.01s 450 1,256
Qwen3.5-27B 10.0 10.0 100.0% 0 7.45s 348 1,323

Quick Compare

Switch Comparison Pair

Gemini 3.1 Flash Lite PreviewlowvsQwen3.5-27BmediumGemini 3 Flash PreviewnonevsQwen3.5-27BmediumGemini 3 Flash PreviewlowvsQwen3.5-27BmediumGemini 3.1 Flash Lite PreviewnonevsQwen3.5-27BmediumGPT-5.2 ChatnonevsQwen3.5-27BmediumGemma 4 26B A4BmediumvsGPT-5.2 ChatnoneGPT-5.3 ChatnonevsQwen3.5-27BmediumGemma 4 26B A4BmediumvsGPT-5.3 ChatnoneClaude Sonnet 4.6nonevsQwen3.5-27BmediumClaude Sonnet 4.6nonevsGemma 4 26B A4BmediumGemma 4 26B A4BmediumvsQwen3.5 Plus 2026-02-15noneGemma 4 31BnonevsQwen3.5-27Bmedium