AI BENCHY Compare

Mistral: Mistral Small 4 vs Qwen: Qwen3.5-35B-A3B

Last updated at: 2026-03-17

Metric	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16	Qwen3.5-35B-A3B Qwen3.5-35B-A3B none Release: 2026-02-24

Metric	Mistral Small 4 Mistral Small 4 medium Release: 2026-03-16	Qwen3.5-35B-A3B Qwen3.5-35B-A3B none Release: 2026-02-24
Rank	#55	#49
Score	5.6	5.9
Consistency	7.0	8.6
Cost per result	0.502	0.237
Total Cost	$0.026	$0.015
Tests Correct
Attempt pass rate	49.0%	47.1%
Flaky tests	6	3
Total Runs	51	51
Output Tokens	12,288	3,761
Reasoning Tokens	28,112	0
Response Time (avg)	4.18s	3.89s
Response Time (max)	25.25s	47.43s
Response Time (total)	71.03s	66.07s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	5.6	3.8	66.7%	3		2.67s	4,055	4,778
Qwen3.5-35B-A3B	3.4	7.9	16.7%	1		1.43s	574	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.0	10.0	0.0%	0		25.25s	2,612	10,700
Qwen3.5-35B-A3B	3.0	10.0	0.0%	0		47.43s	1,833	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	7.3	5.9	83.3%	1		1.23s	335	723
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		1.16s	243	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	5.3	7.2	44.4%	1		6.11s	2,621	6,904
Qwen3.5-35B-A3B	7.7	10.0	66.7%	0		485ms	15	0

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	4.8	10.0	0.0%	0		2.05s	821	828
Qwen3.5-35B-A3B	6.5	3.4	66.7%	1		1.19s	114	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	7.3	5.8	83.3%	1		1.38s	540	1,031
Qwen3.5-35B-A3B	6.3	10.0	50.0%	0		809ms	63	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.4	9.7	0.0%	0		2.00s	983	2,338
Qwen3.5-35B-A3B	3.9	7.4	22.2%	1		1.34s	655	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	10.0	10.0	100.0%	0		3.50s	321	810
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		2.30s	264	0

Quick Compare

Switch Comparison Pair

MiniMax M2.5mediumFree AvailablevsQwen3.5-35B-A3Bnone Mistral Small 4mediumvsGLM 5 Turbonone Mistral Small 4mediumvsGLM 4.7 Flashnone Mistral Small 4mediumvsQwen3.5-27Bnone Mistral Small 4mediumvsGPT-5.4none gpt-oss-120bmediumFree AvailablevsQwen3.5-35B-A3Bnone Mistral Small 4mediumvsQwen3.5-122B-A10Bnone Mistral Small 4mediumvsGrok 4.20 Betanone Mistral Small 4mediumvsKimi K2.5none Mistral Small 4mediumvsHunter Alphanone Trinity Large PreviewnoneFree AvailablevsMistral Small 4medium Gemini 2.5 FlashnonevsMistral Small 4medium