Navigate
AI BENCHY
Advertise here

AI BENCHY Compare

Qwen: Qwen3.6 27B vs StepFun: Step 3.7 Flash

Last updated at: 2026-06-04

Metric Qwen3.6 27B Qwen3.6 27B medium Release: 2026-04-20 Step 3.7 Flash Step 3.7 Flash high Release: 2026-05-29
Score 6.8 7.0
Rank #78 #71
Reliability 10.0 10.0
Consistency 8.2 8.2
Tests Correct
Attempt pass rate 60.3% 63.5%
Flaky tests 5 4
Total Runs 63 63
Cost per result 3.361 10.434
Total Cost $0.444 $1.148
Input Price $0.290 / 1M $0.200 / 1M
Output Price $3.200 / 1M $1.150 / 1M
Total Input Tokens 39,376 38,391
Output Tokens 16,189 991,355
Reasoning Tokens 122,521 0
Response Time (avg) 59.71s 64.46s
Response Time (max) 168.22s 364.99s
Response Time (total) 1254.01s 1353.57s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 8.3 10.0 75.0% 0 12.62s 453 582 4,311
Step 3.7 Flash 10.0 10.0 100.0% 0 13.40s 696 42,656 0
Coding Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 7.7 10.0 66.7% 0 142.99s 5,051 7,968 43,367
Step 3.7 Flash 4.0 6.0 22.2% 1 206.21s 6,057 327,340 0
Combined Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 7.0 3.7 66.7% 1 83.07s 15,104 2,088 14,689
Step 3.7 Flash 10.0 10.0 100.0% 0 13.01s 13,638 8,802 0
Data parsing and extraction Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 3.5 1.4 50.0% 2 37.30s 7,778 568 9,404
Step 3.7 Flash 10.0 10.0 100.0% 0 14.72s 7,368 23,113 0
Domain specific Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 2.9 7.2 11.1% 1 73.38s 662 3,510 20,352
Step 3.7 Flash 4.1 4.4 44.5% 2 149.64s 783 410,502 0
General Intelligence Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 6.5 3.4 66.7% 1 39.53s 516 81 3,045
Step 3.7 Flash 5.5 10.0 0.0% 0 4.17s 510 2,862 0
Instructions following Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 10.0 10.0 100.0% 0 37.96s 699 346 6,548
Step 3.7 Flash 9.8 10.0 100.0% 0 1.52s 705 2,010 0
Puzzle Solving Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 7.7 10.0 66.7% 0 61.14s 696 255 12,044
Step 3.7 Flash 5.3 7.2 44.4% 1 10.22s 711 25,422 0
Tool Calling Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 10.0 10.0 100.0% 0 16.88s 8,213 390 2,954
Step 3.7 Flash 10.0 10.0 100.0% 0 2.79s 7,701 1,172 0
Trivia Score Consistency Attempt pass rate Flaky tests Tests Correct Response Time (avg) Input Tokens Output Tokens Reasoning Tokens
Qwen3.6 27B 3.0 10.0 0.0% 0 80.99s 204 401 5,807
Step 3.7 Flash 3.0 10.0 0.0% 0 149.34s 222 147,476 0

Quick Compare

Switch Comparison Pair

GPT-5.4 NanomediumvsStep 3.7 FlashhighClaude Opus 4.6mediumvsStep 3.7 FlashhighClaude Opus 4.8nonevsStep 3.7 FlashhighMiniMax M3mediumvsStep 3.7 FlashhighClaude Sonnet 4.6nonevsQwen3.6 27BmediumDeepSeek V3.2mediumvsStep 3.7 FlashhighQwen3.5-35B-A3BmediumvsStep 3.7 FlashhighStep 3.7 FlashhighvsGrok 4.20mediumSeed-2.0-MinimediumvsStep 3.7 FlashhighStep 3.7 FlashhighvsMiMo-V2-FlashmediumGPT-5.3 ChatnonevsStep 3.7 FlashhighQwen3.6 Max PreviewnonevsStep 3.7 Flashhigh