AI BENCHY Compare

Mistral: Mistral Small 4 vs Qwen: Qwen3.5-9B

Last updated at: 2026-05-01

Metric	Mistral Small 4 Mistral Small 4 none Release: 2026-03-16	Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02

Metric	Mistral Small 4 Mistral Small 4 none Release: 2026-03-16	Qwen3.5-9B Qwen3.5-9B medium Release: 2026-03-02
Score	5.2	4.4
Rank	#115	#132
Reliability	N/A	N/A
Consistency	9.5	7.2
Tests Correct
Attempt pass rate	31.5%	33.3%
Flaky tests	1	6
Total Runs	54	54
Cost per result	0.118	0.976
Total Cost	$0.006	$0.030
Input Price	$0.150 / 1M	$0.100 / 1M
Output Price	$0.600 / 1M	$0.150 / 1M
Output Tokens	2,207	24,291
Reasoning Tokens	0	172,597
Response Time (avg)	665ms	73.64s
Response Time (max)	1.72s	226.38s
Response Time (total)	11.97s	1104.60s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.4	7.9	16.7%	1		395ms	182	0
Qwen3.5-9B	5.1	5.8	50.0%	2		34.44s	2,621	12,411

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	4.5	9.0	0.0%	0		1.28s	583	0
Qwen3.5-9B	2.6	10.0	0.0%	0		135.61s	6,150	31,393

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.0	10.0	0.0%	0		1.72s	496	0
Qwen3.5-9B	3.0	10.0	0.0%	0		0ms	0	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	10.0	10.0	100.0%	0		822ms	261	0
Qwen3.5-9B	3.6	5.6	33.3%	1		87.31s	1,383	32,113

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	5.3	10.0	33.3%	0		367ms	28	0
Qwen3.5-9B	3.6	7.2	22.2%	1		137.75s	11,549	48,475

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	4.0	10.0	0.0%	0		729ms	205	0
Qwen3.5-9B	2.8	1.6	33.3%	1		226.38s	0	30,695

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	6.5	10.0	50.0%	0		380ms	69	0
Qwen3.5-9B	6.4	5.8	66.7%	1		17.15s	599	4,517

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.1	9.9	0.0%	0		589ms	170	0
Qwen3.5-9B	3.1	10.0	0.0%	0		33.38s	1,545	11,844

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	10.0	10.0	100.0%	0		1.40s	213	0
Qwen3.5-9B	10.0	10.0	100.0%	0		4.31s	444	1,149

Quick Compare

Switch Comparison Pair

Nemotron 3 Nano Omni 30b A3b ReasoningnoneFree AvailablevsQwen3.5-9Bmedium Mistral Small 4nonevsNemotron 3 Nano Omni 30b A3b ReasoningmediumFree Available Mistral Small 4nonevsElephant Alphamedium MiniMax M2.7mediumvsMistral Small 4none Ling 2.6 1tnoneFree AvailablevsQwen3.5-9Bmedium Granite 4.1 8BnonevsQwen3.5-9Bmedium Qwen3.5-9BmediumvsGrok 4.1 Fastnone Qwen3.5-9BmediumvsMiMo-V2-Flashnone GPT-5.4 NanononevsQwen3.5-9Bmedium Mistral Small 4nonevsQwen3 Coder Nextmedium Qwen3.5-9BmediumvsHY3 PreviewnoneFree Available Mercury 2nonevsQwen3.5-9Bmedium