AI BENCHY Compare

Qwen: Qwen3.6 Plus vs StepFun: Step 3.7 Flash

Summary

Qwen3.6 Plus (medium) leads on average score with 7.8 vs 7.7. Qwen3.6 Plus (medium) has the lower benchmark cost at $0.294 vs $0.341. Step 3.7 Flash (low) is faster at 15.74s vs 30.70s, with pass rates of 69.8% vs 68.3%.

Recommended modelStep 3.7 Flash (low)Its score stays close to the best score here (7.7 vs 7.8), while responding about 1.9x faster than Qwen3.6 Plus (medium).

Last updated at: 2026-07-14

Metric	Qwen3.6 Plus Qwen3.6 Plus medium Release: 2026-04-20	Step 3.7 Flash Step 3.7 Flash low Release: 2026-05-29

Metric	Qwen3.6 Plus Qwen3.6 Plus medium Release: 2026-04-20	Step 3.7 Flash Step 3.7 Flash low Release: 2026-05-29
Score	7.8	7.7
Rank	#38	#51
Reliability	10.0	10.0
Consistency	9.3	8.4
Tests Correct
Attempt pass rate	69.8%	68.3%
Flaky tests	2	4
Total Runs	63	63
Cost per result	0.831	2.840
Total Cost	$0.294	$0.341
Input Price	$0.325 / 1M	$0.200 / 1M
Output Price	$1.950 / 1M	$1.150 / 1M
Total Input Tokens	41,565	40,101
Output Tokens	1,853	289,325
Reasoning Tokens	141,973	0
Response Time (avg)	30.70s	15.74s
Response Time (max)	201.68s	124.75s
Response Time (total)	613.99s	330.63s

Generation showcase

Hamster playing table tennis

Prompt: Create a detailed SVG illustration of a hamster playing table tennis.

#38 Qwen3.6 Plus

medium

Cost: $0.024
Time: 219.0s
Tokens: 12,235 tok

#51 Step 3.7 Flash

low

Invalid SVG

Cost: $0.004
Time: 25.3s
Tokens: 3,072 tok

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Category:

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	10.0	10.0	100.0%	0		9.90s	672	207	7,557
Step 3.7 Flash	8.7	7.9	91.7%	1		4.02s	756	10,896	0

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	6.1	7.8	44.4%	1		153.12s	7,098	58	50,586
Step 3.7 Flash	8.2	7.2	88.9%	1		9.46s	7,437	18,685	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	10.0	10.0	100.0%	0		34.95s	14,934	452	13,073
Step 3.7 Flash	10.0	10.0	100.0%	0		7.98s	13,683	6,426	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	10.0	10.0	100.0%	0		14.95s	7,782	270	10,706
Step 3.7 Flash	7.3	5.8	83.3%	1		2.29s	7,398	2,667	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	2.9	7.2	11.1%	1		29.59s	771	56	33,464
Step 3.7 Flash	5.3	7.2	44.4%	1		43.31s	828	104,487	0

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	5.1	10.0	0.0%	0		27.05s	516	111	5,232
Step 3.7 Flash	3.4	9.3	0.0%	0		7.00s	525	4,604	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	10.0	10.0	100.0%	0		7.54s	699	102	5,552
Step 3.7 Flash	9.8	10.0	100.0%	0		1.58s	735	1,857	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	10.0	10.0	100.0%	0		6.34s	696	309	6,712
Step 3.7 Flash	5.5	9.9	33.3%	0		1.84s	756	3,564	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	10.0	10.0	100.0%	0		5.87s	8,193	267	1,330
Step 3.7 Flash	10.0	10.0	100.0%	0		3.25s	7,746	1,360	0

Trivia	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Input Tokens	Output Tokens	Reasoning Tokens
Qwen3.6 Plus	3.0	10.0	0.0%	0		47.51s	204	21	7,761
Step 3.7 Flash	3.0	10.0	0.0%	0		124.75s	237	134,779	0

Quick Compare

Switch Comparison Pair

GPT-5.6 LunahighvsStep 3.7 Flashlow Claude Opus 4.6mediumvsStep 3.7 Flashlow Step 3.7 FlashlowvsGrok 4.3medium KAT-Coder-Pro V2.5highvsStep 3.7 Flashlow Qwen3.6 PlusmediumvsGrok 4.5low Qwen3.5-122B-A10BmediumvsStep 3.7 Flashlow Kimi K2.6mediumFree AvailablevsStep 3.7 Flashlow Gemini 3.1 Flash LitemediumvsStep 3.7 Flashlow Qwen3.5 Plus 2026-04-20mediumvsStep 3.7 Flashlow Gemini 3.1 Flash Lite PreviewmediumvsStep 3.7 Flashlow MiniMax M3mediumvsStep 3.7 Flashlow DeepSeek V4 ProhighvsStep 3.7 Flashlow