AI BENCHY Compare

Qwen: Qwen3.5-Flash vs StepFun: Step 3.7 Flash

Summary

Step 3.7 Flash (high) leads on average score with 7.1 vs 6.8. Qwen3.5-Flash (medium) has the lower benchmark cost at $0.080 vs $1.148. Qwen3.5-Flash (medium) is faster at 63.29s vs 64.46s, with pass rates of 71.4% vs 63.5%.

Recommended modelQwen3.5-Flash (medium)Its score stays close to the best score here (6.8 vs 7.1), while costing about 14.4x less than Step 3.7 Flash (high).

Last updated at: 2026-07-14

Metric	Qwen3.5-Flash Qwen3.5-Flash medium Release: 2026-02-24	Step 3.7 Flash Step 3.7 Flash high Release: 2026-05-29

Metric	Qwen3.5-Flash Qwen3.5-Flash medium Release: 2026-02-24	Step 3.7 Flash Step 3.7 Flash high Release: 2026-05-29
Score	6.8	7.1
Rank	#87	#79
Reliability	10.0	10.0
Consistency	8.1	8.2
Tests Correct
Attempt pass rate	71.4%	63.5%
Flaky tests	5	4
Total Runs	63	63
Cost per result	0.871	10.434
Total Cost	$0.080	$1.148
Input Price	$0.065 / 1M	$0.200 / 1M
Output Price	$0.260 / 1M	$1.150 / 1M
Total Input Tokens	38,926	38,391
Output Tokens	2,088	991,355
Reasoning Tokens	294,598	0
Response Time (avg)	63.29s	64.46s
Response Time (max)	234.29s	364.99s
Response Time (total)	1265.85s	1353.57s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#87 Qwen3.5-Flash

medium

Cost: $0.002
Time: 25.8s
Tokens: 4,294 tok

#79 Step 3.7 Flash

high

Cost: $0.007
Time: 63.6s
Tokens: 6,030 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	10.0	10.0	100.0%	0		59.11s	672	383	32,992
Step 3.7 Flash	10.0	10.0	100.0%	0		13.40s	696	42,656	0

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	3.7	7.2	22.2%	1		58.87s	6,685	302	90,081
Step 3.7 Flash	4.0	6.0	22.2%	1		206.21s	6,057	327,340	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	10.0	10.0	100.0%	0		17.78s	14,934	483	8,270
Step 3.7 Flash	10.0	10.0	100.0%	0		13.01s	13,638	8,802	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	7.3	5.9	83.3%	1		56.99s	6,061	235	16,237
Step 3.7 Flash	10.0	10.0	100.0%	0		14.72s	7,368	23,113	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	5.3	7.2	44.4%	1		146.50s	581	58	43,615
Step 3.7 Flash	4.1	4.4	44.5%	2		149.64s	783	410,502	0

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	6.1	3.1	66.7%	1		40.05s	516	99	38,486
Step 3.7 Flash	5.5	10.0	0.0%	0		4.17s	510	2,862	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	10.0	10.0	100.0%	0		63.49s	699	98	14,139
Step 3.7 Flash	9.8	10.0	100.0%	0		1.52s	705	2,010	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	8.2	7.2	88.9%	1		27.61s	381	89	12,457
Step 3.7 Flash	5.3	7.2	44.4%	1		10.22s	711	25,422	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	10.0	10.0	100.0%	0		10.33s	8,193	309	1,284
Step 3.7 Flash	10.0	10.0	100.0%	0		2.79s	7,701	1,172	0

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.5-Flash	3.0	10.0	0.0%	0		48.98s	204	32	37,037
Step 3.7 Flash	3.0	10.0	0.0%	0		149.34s	222	147,476	0

Quick Compare

Switch Comparison Pair

Step 3.7 FlashhighvsGLM 5.1medium Gemini 3.5 FlashminimalvsQwen3.5-Flashmedium KAT-Coder-Pro V2.5nonevsQwen3.5-Flashmedium Laguna XS 2.1mediumFree AvailablevsStep 3.7 Flashhigh Step 3.7 FlashhighvsGLM 5.2none Gemini 3 Flash PreviewnonevsQwen3.5-Flashmedium Qwen3.7 PlusnonevsStep 3.7 Flashhigh Kimi K2.7 CodemediumvsStep 3.7 Flashhigh Gemma 4 26B A4BmediumFree AvailablevsStep 3.7 Flashhigh DeepSeek V4 PrononevsStep 3.7 Flashhigh Gemini 3.5 FlashnonevsStep 3.7 Flashhigh Claude Opus 4.8nonevsStep 3.7 Flashhigh