AI BENCHY Compare

Qwen: Qwen3.5-35B-A3B vs StepFun: Step 3.5 Flash

Summary

Step 3.5 Flash (medium) leads on average score with 6.6 vs 6.3. Step 3.5 Flash (medium) has the lower benchmark cost at $0.070 vs $0.401. Step 3.5 Flash (medium) is faster at 72.53s vs 72.57s, with pass rates of 69.8% vs 54.0%.

Recommended modelStep 3.5 Flash (medium)It has the best score here (6.6), while costing about 5.8x less than Qwen3.5-35B-A3B (medium).

Last updated at: 2026-07-14

Metric	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24	Step 3.5 Flash Step 3.5 Flash medium Release: 2026-02-01

Metric	Qwen3.5-35B-A3B Qwen3.5-35B-A3B medium Release: 2026-02-24	Step 3.5 Flash Step 3.5 Flash medium Release: 2026-02-01
Score	6.3	6.6
Rank	#108	#98
Reliability	10.0	10.0
Consistency	7.5	8.9
Tests Correct
Attempt pass rate	69.8%	54.0%
Flaky tests	6	1
Total Runs	63	60
Cost per result	5.162	0.198
Total Cost	$0.401	$0.070
Input Price	$0.140 / 1M	$0.100 / 1M
Output Price	$1.000 / 1M	$0.300 / 1M
Total Input Tokens	42,196	34,431
Output Tokens	40,630	91,587
Reasoning Tokens	353,577	195,973
Response Time (avg)	72.57s	72.53s
Response Time (max)	409.98s	453.94s
Response Time (total)	1524.04s	1015.47s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#108 Qwen3.5-35B-A3B

medium

Cost: $0.009
Time: 71.4s
Tokens: 8,631 tok

#98 Step 3.5 Flash

medium

Cost: $0.008
Time: 277.1s
Tokens: 23,695 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		21.13s	672	798	42,652
Step 3.5 Flash	10.0	10.0	100.0%	0		40.57s	694	20,391	24,176

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	5.9	9.3	33.3%	0		206.65s	4,106	23,844	111,462
Step 3.5 Flash	2.4	5.2	0.0%	0		258.38s	2,211	13,207	22,429

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	4.7	1.6	66.7%	1		75.34s	20,992	775	12,485
Step 3.5 Flash	10.0	10.0	100.0%	0		29.57s	13,638	1,176	12,984

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	7.3	5.9	83.3%	1		59.33s	6,061	235	19,493
Step 3.5 Flash	10.0	10.0	100.0%	0		15.01s	7,368	600	13,886

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	4.1	4.4	44.5%	2		88.34s	500	41	46,368
Step 3.5 Flash	5.3	7.2	44.4%	1		170.45s	673	45,350	90,436

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	2.8	1.6	33.3%	1		30.30s	172	20	3,753
Step 3.5 Flash	5.5	10.0	0.0%	0		22.39s	509	240	3,506

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		24.45s	699	97	17,361
Step 3.5 Flash	8.3	10.0	50.0%	0		4.78s	705	2,364	3,521

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	8.2	7.2	88.9%	1		33.13s	597	3,592	26,585
Step 3.5 Flash	5.3	10.0	33.3%	0		7.22s	711	5,630	10,861

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	10.0	10.0	100.0%	0		4.65s	8,193	309	1,365
Step 3.5 Flash	10.0	10.0	100.0%	0		11.91s	7,701	275	3,802

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-35B-A3B	3.0	10.0	0.0%	0		177.35s	204	10,919	72,053
Step 3.5 Flash	3.0	10.0	0.0%	0		108.45s	221	2,354	10,372

Quick Compare

Switch Comparison Pair