Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Mistral: Mistral Small 4 vs Qwen: Qwen3.5-9B

Summary

Mistral Small 4 vs Qwen3.5-9B benchmark comparison: Mistral Small 4 leads on average score with 5.3 vs 4.2. Qwen3.5-9B has the lower benchmark cost at $0.036 vs $0.068. Mistral Small 4 is faster at 9.40s vs 82.24s, with pass rates of 44.4% vs 27.0%.

Recommended model: Mistral Small 4 - It has the best score here (5.3), while responding about 8.7x faster than Qwen3.5-9B.

Last updated at: 2026-06-10

Metric Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16 Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Score 5.3 4.2
Rank #133 #162
Reliability 10.0 6.7
Consistency 6.9 8.0
Tests Correct
Attempt pass rate 44.4% 27.0%
Flaky tests 8 5
Total Runs 63 63
Cost per result 1.344 1.187
Total Cost $0.068 $0.036
Input Price $0.150 / 1M $0.100 / 1M
Output Price $0.600 / 1M $0.150 / 1M
Total Input Tokens 42,576 17,070
Output Tokens 24,184 29,045
Reasoning Tokens 84,678 209,516
Response Time (avg) 9.40s 82.24s
Response Time (max) 59.15s 226.38s
Response Time (total) 197.39s 1315.88s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#133 Mistral Small 4

medium
Cost
$0.006
Time
47.9s
Tokens
9,857 tok

#162 Qwen3.5-9B

medium
Cost
$0.001
Time
35.9s
Tokens
3,030 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 5.6 3.8 66.7% 3 2.67s 708 4,055 4,778
Qwen3.5-9B 5.1 5.8 50.0% 2 34.44s 369 2,621 12,411
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 4.4 5.1 33.3% 2 39.98s 7,636 11,635 54,715
Qwen3.5-9B 2.9 10.0 0.0% 0 100.88s 2,396 7,890 41,129
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.0 10.0 0.0% 0 25.25s 18,706 2,612 10,700
Qwen3.5-9B 3.0 10.0 0.0% 0 0ms 0 0 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 7.3 5.9 83.3% 1 1.23s 6,171 335 723
Qwen3.5-9B 3.6 5.6 33.3% 1 87.31s 4,722 1,383 32,113
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 5.3 7.2 44.4% 1 6.11s 742 2,621 6,904
Qwen3.5-9B 3.6 7.2 22.2% 1 137.75s 295 11,549 48,475
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 4.8 10.0 0.0% 0 2.05s 519 821 828
Qwen3.5-9B 2.8 1.6 33.3% 1 226.38s 180 0 30,695
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 7.3 5.8 83.3% 1 1.38s 729 540 1,031
Qwen3.5-9B 6.5 10.0 50.0% 0 5.75s 381 491 1,824
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.4 9.7 0.0% 0 2.17s 735 1,226 2,632
Qwen3.5-9B 3.0 10.0 0.0% 0 32.27s 376 1,593 12,026
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 10.0 10.0 100.0% 0 3.50s 6,420 321 810
Qwen3.5-9B 10.0 10.0 100.0% 0 4.31s 8,283 444 1,149
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Mistral Small 4 3.0 10.0 0.0% 0 5.92s 210 18 1,557
Qwen3.5-9B 3.0 10.0 0.0% 0 177.02s 68 3,074 29,694

Quick Compare

Switch Comparison Pair

Mistral Small 4mediumvsQwen3.5-122B-A10BnoneDeepSeek V3.2nonevsMistral Small 4mediumMistral Small 4mediumvsGLM 5 TurbononeLing-2.6-1TnonevsQwen3.5-9BmediumMistral Small 4mediumvsKimi K2.5noneMistral Small 4mediumvsQwen3.6 FlashnoneGranite 4.1 8BnonevsQwen3.5-9BmediumMistral Small 4mediumvsGrok 4.20noneMistral Small 4mediumvsgpt-oss-120bnoneFree AvailableMistral Small 4mediumvsGPT-5.4noneMistral Small 4mediumvsKimi K2.6noneFree AvailableMistral Small 4mediumvsMiMo-V2.5-Pronone