Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Mistral: Mistral Small 4 vs Qwen: Qwen3.5-27B

Summary

Mistral Small 4 vs Qwen3.5-27B benchmark comparison: Qwen3.5-27B leads on average score with 5.9 vs 5.1. Mistral Small 4 has the lower benchmark cost at $0.007 vs $0.015. Mistral Small 4 is faster at 630ms vs 1.68s, with pass rates of 27.0% vs 38.1%.

Recommended model: Mistral Small 4 - Its score stays close to the best score here (5.1 vs 5.9), while costing about 2.1x less than Qwen3.5-27B.

Last updated at: 2026-06-12

Metric Mistral Small 4 Mistral Small 4 none Release: 2026-03-16 Qwen3.5-27B Qwen3.5-27B none Release: 2026-02-24
Score 5.1 5.9
Rank #136 #109
Reliability 10.0 10.0
Consistency 9.5 9.3
Tests Correct
Attempt pass rate 27.0% 38.1%
Flaky tests 1 2
Total Runs 63 63
Cost per result 0.139 0.249
Total Cost $0.007 $0.015
Input Price $0.150 / 1M $0.195 / 1M
Output Price $0.600 / 1M $1.560 / 1M
Total Input Tokens 37,309 44,478
Output Tokens 2,201 3,592
Reasoning Tokens 0 0
Response Time (avg) 630ms 1.68s
Response Time (max) 1.72s 9.39s
Response Time (total) 13.22s 35.25s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#136 Mistral Small 4

none
Cost
$0.002
Time
10.4s
Tokens
2,370 tok

#109 Qwen3.5-27B

none
Cost
$0.007
Time
42.9s
Tokens
4,273 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.4 7.9 16.7% 1 395ms 708 182 0
Qwen3.5-27B 4.8 10.0 25.0% 0 788ms 696 267 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.7 9.7 0.0% 0 901ms 7,636 619 0
Qwen3.5-27B 5.8 10.0 33.3% 0 1.80s 7,913 415 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.0 10.0 0.0% 0 1.72s 11,640 496 0
Qwen3.5-27B 2.8 1.6 33.3% 1 9.39s 16,918 1,461 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 10.0 10.0 100.0% 0 822ms 7,914 261 0
Qwen3.5-27B 10.0 10.0 100.0% 0 1.43s 7,794 243 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 5.3 10.0 33.3% 0 367ms 798 28 0
Qwen3.5-27B 3.0 10.0 0.0% 0 540ms 789 15 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 4.0 10.0 0.0% 0 729ms 519 205 0
Qwen3.5-27B 5.0 10.0 0.0% 0 2.51s 522 126 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 6.5 10.0 50.0% 0 380ms 729 69 0
Qwen3.5-27B 6.3 10.0 50.0% 0 1.03s 711 69 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.1 9.9 0.0% 0 399ms 735 111 0
Qwen3.5-27B 6.7 7.9 55.6% 1 1.38s 714 683 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 10.0 10.0 100.0% 0 1.40s 6,420 213 0
Qwen3.5-27B 10.0 10.0 100.0% 0 3.54s 8,211 303 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.0 10.0 0.0% 0 397ms 210 17 0
Qwen3.5-27B 3.0 10.0 0.0% 0 599ms 210 10 0

Quick Compare

Switch Comparison Pair

MiniMax M2.7mediumvsMistral Small 4noneCobuddymediumvsMistral Small 4noneGemini 3.1 Flash LiteminimalvsQwen3.5-27BnoneDeepSeek V4 ProhighvsQwen3.5-27BnoneGemma 4 31BmediumFree AvailablevsQwen3.5-27BnoneMiniMax M2.5mediumvsMistral Small 4noneNemotron 3 SupermediumFree AvailablevsQwen3.5-27BnoneMistral Small 4nonevsQwen3 Coder NextmediumGemini 3.1 Flash LitelowvsQwen3.5-27BnoneGemini 3.1 Flash Lite PreviewlowvsQwen3.5-27BnoneMiniMax M2.7mediumvsQwen3.5-27BnoneGPT-5 NanomediumvsQwen3.5-27Bnone