Compare Charts

Language:

❤️ Made by XCS

AI BENCHY Compare

Anthropic: Claude Sonnet 4.6 vs StepFun: Step 3.5 Flash

Model Name:

Last updated at : 2026-02-27 15:16

Summary

Metric	Anthropic: Claude Sonnet 4.6 none Release: Unknown release date	StepFun: Step 3.5 Flash medium Release: Unknown release date Free Available
Rank	#14	#11
Score	6.36	7.00
Consistency	9.50	8.32
Cost per result	1.325	0.000
Total Cost	$0.106	$0.000
Tests Correct
Wrong Tests	6	5
Attempt pass rate	59.5%	73.8%
Flaky tests	1	3
Output Tokens	2,637	60,502
Reasoning Tokens	0	117,044

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
Anthropic: Claude Sonnet 4.6	4.00	10.00	33.3%	0		1,199	0
StepFun: Step 3.5 Flash	10.00	10.00	100.0%	0		13,924	17,208

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
Anthropic: Claude Sonnet 4.6	10.00	10.00	100.0%	0		252	0
StepFun: Step 3.5 Flash	10.00	10.00	100.0%	0		535	11,548

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
Anthropic: Claude Sonnet 4.6	7.00	10.00	66.7%	0		54	0
StepFun: Step 3.5 Flash	4.00	7.21	44.4%	1		40,942	74,237

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
Anthropic: Claude Sonnet 4.6	5.50	10.00	50.0%	0		90	0
StepFun: Step 3.5 Flash	10.00	10.00	100.0%	0		2,121	3,274

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
Anthropic: Claude Sonnet 4.6	5.00	7.68	44.4%	1		595	0
StepFun: Step 3.5 Flash	2.00	4.96	33.3%	2		2,705	6,975

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
Anthropic: Claude Sonnet 4.6	10.00	10.00	100.0%	0		447	0
StepFun: Step 3.5 Flash	10.00	10.00	100.0%	0		275	3,802

Switch Comparison Pair

Compare Anthropic: Claude Sonnet 4.6 against...

Compare StepFun: Step 3.5 Flash against...