Compare Charts

Language:

❤️ Made by XCS

AI BENCHY Compare

OpenAI: GPT-4o-mini vs Qwen: Qwen3.5-Flash

Model Name:

Last updated at : 2026-02-27 15:16

Summary

Metric	OpenAI: GPT-4o-mini none Release: Unknown release date	Qwen: Qwen3.5-Flash none Release: Unknown release date
Rank	#28	#35
Score	4.86	3.63
Consistency	9.98	8.32
Cost per result	0.056	0.084
Total Cost	$0.003	$0.003
Tests Correct
Wrong Tests	9	11
Attempt pass rate	35.7%	33.3%
Flaky tests	0	3
Output Tokens	949	1,799
Reasoning Tokens	0	0

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
OpenAI: GPT-4o-mini	4.00	10.00	33.3%	0		180	0
Qwen: Qwen3.5-Flash	2.33	7.76	11.1%	1		687	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
OpenAI: GPT-4o-mini	10.00	10.00	100.0%	0		183	0
Qwen: Qwen3.5-Flash	5.38	5.81	83.3%	1		193	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
OpenAI: GPT-4o-mini	1.00	10.00	0.0%	0		15	0
Qwen: Qwen3.5-Flash	4.00	7.21	55.6%	1		14	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
OpenAI: GPT-4o-mini	5.50	10.00	50.0%	0		71	0
Qwen: Qwen3.5-Flash	3.50	9.99	0.0%	0		63	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
OpenAI: GPT-4o-mini	4.00	9.92	0.0%	0		295	0
Qwen: Qwen3.5-Flash	1.33	9.99	0.0%	0		578	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Output Tokens	Reasoning Tokens
OpenAI: GPT-4o-mini	10.00	10.00	100.0%	0		205	0
Qwen: Qwen3.5-Flash	10.00	10.00	100.0%	0		264	0

Switch Comparison Pair

Compare OpenAI: GPT-4o-mini against...

Compare Qwen: Qwen3.5-Flash against...