Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Qwen: Qwen3.5-27B vs StepFun: Step 3.7 Flash

Last updated at: 2026-05-29

Metric Qwen3.5-27B Qwen3.5-27B medium Release: 2026-02-24 Step 3.7 Flash Step 3.7 Flash medium Release: 2026-05-29
Score 7.9 7.9
Rank #28 #32
Reliability 10.0 9.9
Consistency 8.9 9.2
Tests Correct
Attempt pass rate 73.3% 71.7%
Flaky tests 3 2
Total Runs 60 58
Cost per result 4.532 2.663
Total Cost $0.590 $0.347
Input Price $0.195 / 1M $0.200 / 1M
Output Price $1.560 / 1M $1.150 / 1M
Output Tokens 2,569 294,481
Reasoning Tokens 304,894 0
Response Time (avg) 60.09s 18.32s
Response Time (max) 177.36s 113.98s
Response Time (total) 1201.89s 366.45s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 8.7 7.9 91.7% 1 19.75s 569 31,505
Step 3.7 Flash 8.7 7.9 91.7% 1 9.65s 32,185 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 7.0 9.8 50.0% 0 123.86s 416 64,993
Step 3.7 Flash 8.2 6.7 83.3% 1 10.64s 19,320 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 10.0 10.0 100.0% 0 163.96s 483 9,991
Step 3.7 Flash 10.0 10.0 100.0% 0 9.06s 7,106 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 10.0 10.0 100.0% 0 30.26s 270 16,150
Step 3.7 Flash 10.0 10.0 100.0% 0 2.75s 3,020 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 5.3 10.0 33.3% 0 79.53s 43 52,368
Step 3.7 Flash 7.7 10.0 66.7% 0 48.27s 70,347 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 6.1 3.1 66.7% 1 101.41s 70 23,147
Step 3.7 Flash 4.0 10.0 0.0% 0 6.85s 3,987 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 10.0 10.0 100.0% 0 19.66s 97 11,638
Step 3.7 Flash 9.8 10.0 100.0% 0 1.83s 2,166 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 8.2 7.7 77.8% 1 59.60s 242 70,096
Step 3.7 Flash 5.7 9.9 33.3% 0 6.19s 15,071 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 10.0 10.0 100.0% 0 7.45s 348 1,323
Step 3.7 Flash 10.0 10.0 100.0% 0 4.16s 2,115 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Output Tokens Reasoning Tokens
Qwen3.5-27B 3.0 10.0 0.0% 0 85.11s 31 23,683
Step 3.7 Flash 3.0 10.0 0.0% 0 113.98s 139,164 0

Quick Compare

Switch Comparison Pair

Qwen3.7 MaxnonevsStep 3.7 FlashmediumGemini 3.5 FlashminimalvsQwen3.5-27BmediumGPT-5.2 ChatnonevsStep 3.7 FlashmediumGemini 3.5 FlashminimalvsStep 3.7 FlashmediumGPT-5.2 ChatnonevsQwen3.5-27BmediumGemini 3 Flash PreviewnonevsStep 3.7 FlashmediumGemini 3 Flash PreviewnonevsQwen3.5-27BmediumDeepSeek V4 FlashhighFree AvailablevsStep 3.7 FlashmediumGemini 3.1 Flash Lite PreviewlowvsStep 3.7 FlashmediumDeepSeek V4 FlashhighFree AvailablevsQwen3.5-27BmediumGemini 3.1 Flash Lite PreviewlowvsQwen3.5-27BmediumGemini 3.1 Flash Lite PreviewnonevsStep 3.7 Flashmedium