AI BENCHY Compare

Qwen: Qwen3.7 Max vs StepFun: Step 3.5 Flash

Last updated at: 2026-06-03

Metric	Qwen3.7 Max Qwen3.7 Max none Release: 2026-05-22	Step 3.5 Flash Step 3.5 Flash medium Release: 2026-02-01

Metric	Qwen3.7 Max Qwen3.7 Max none Release: 2026-05-22	Step 3.5 Flash Step 3.5 Flash medium Release: 2026-02-01
Score	7.9	7.4
Rank	#26	#55
Reliability	10.0	10.0
Consistency	10.0	9.4
Tests Correct
Attempt pass rate	70.0%	59.7%
Flaky tests	0	1
Total Runs	60	57
Cost per result	0.719	0.133
Total Cost	$0.051	$0.062
Input Price	$1.250 / 1M	$0.090 / 1M
Output Price	$3.750 / 1M	$0.300 / 1M
Total Input Tokens	34,266	33,555
Output Tokens	1,988	90,478
Reasoning Tokens	0	173,544
Response Time (avg)	1.30s	43.20s
Response Time (max)	3.92s	170.45s
Response Time (total)	25.95s	561.54s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	6.5	10.0	50.0%	0		1.08s	696	242	0
Step 3.5 Flash	10.0	10.0	100.0%	0		40.57s	694	20,391	24,176

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	6.8	10.0	50.0%	0		1.39s	5,070	576	0
Step 3.5 Flash	3.5	7.8	0.0%	0		62.83s	1,335	12,098	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	3.0	10.0	0.0%	0		2.17s	9,549	171	0
Step 3.5 Flash	10.0	10.0	100.0%	0		29.57s	13,638	1,176	12,984

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	10.0	10.0	100.0%	0		1.35s	7,794	243	0
Step 3.5 Flash	10.0	10.0	100.0%	0		15.01s	7,368	600	13,886

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	7.7	10.0	66.7%	0		975ms	789	15	0
Step 3.5 Flash	5.3	7.2	44.4%	1		170.45s	673	45,350	90,436

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	10.0	10.0	100.0%	0		1.04s	522	120	0
Step 3.5 Flash	5.5	10.0	0.0%	0		22.39s	509	240	3,506

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	10.0	10.0	100.0%	0		943ms	711	72	0
Step 3.5 Flash	8.3	10.0	50.0%	0		4.78s	705	2,364	3,521

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	10.0	10.0	100.0%	0		1.13s	714	314	0
Step 3.5 Flash	5.3	10.0	33.3%	0		7.22s	711	5,630	10,861

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	10.0	10.0	100.0%	0		3.92s	8,211	222	0
Step 3.5 Flash	10.0	10.0	100.0%	0		11.91s	7,701	275	3,802

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.7 Max	3.0	10.0	0.0%	0		856ms	210	13	0
Step 3.5 Flash	3.0	10.0	0.0%	0		108.45s	221	2,354	10,372

Quick Compare

Switch Comparison Pair

GPT-5.4mediumvsQwen3.7 Maxnone Qwen3.7 MaxnonevsStep 3.7 Flashmedium GPT-5.3 ChatnonevsStep 3.5 Flashmedium Qwen3.7 MaxnonevsGLM 5 Turbomedium Gemini 3.1 Flash LitelowvsStep 3.5 Flashmedium Gemini 3.5 FlashminimalvsQwen3.7 Maxnone Gemini 3 PRO PreviewmediumvsQwen3.7 Maxnone Gemma 4 26B A4BmediumFree AvailablevsQwen3.7 Maxnone Gemini 3.1 Flash Lite PreviewnonevsStep 3.5 Flashmedium Gemma 4 31BmediumFree AvailablevsQwen3.7 Maxnone Claude Opus 4.8nonevsStep 3.5 Flashmedium Qwen3.7 MaxnonevsGrok 4.3medium