AI BENCHY Compare

Mistral: Mistral Small 4 vs OpenAI: GPT-4o-mini

Last updated at: 2026-05-01

Metric	Mistral Small 4 Mistral Small 4 none Release: 2026-03-16	GPT-4o-mini GPT-4o-mini none Release: 2024-07-18

Metric	Mistral Small 4 Mistral Small 4 none Release: 2026-03-16	GPT-4o-mini GPT-4o-mini none Release: 2024-07-18
Score	5.2	5.1
Rank	#115	#120
Reliability	N/A	N/A
Consistency	9.5	9.9
Tests Correct
Attempt pass rate	31.5%	27.8%
Flaky tests	1	0
Total Runs	54	54
Cost per result	0.118	0.098
Total Cost	$0.006	$0.005
Input Price	$0.150 / 1M	$0.150 / 1M
Output Price	$0.600 / 1M	$0.600 / 1M
Output Tokens	2,207	1,947
Reasoning Tokens	0	0
Response Time (avg)	665ms	2.00s
Response Time (max)	1.72s	7.58s
Response Time (total)	11.97s	21.99s

Top Models by Score

Score vs Total Cost

Response Time (avg)

Score vs Response Time (avg)

Total Output Tokens

Score vs Total Output Tokens

Category Breakdown

Anti-AI Tricks	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.4	7.9	16.7%	1		395ms	182	0
GPT-4o-mini	4.8	10.0	25.0%	0		1.34s	186	0

Coding	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	4.5	9.0	0.0%	0		1.28s	583	0
GPT-4o-mini	3.0	8.7	0.0%	0		2.55s	347	0

Combined	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.0	10.0	0.0%	0		1.72s	496	0
GPT-4o-mini	3.0	10.0	0.0%	0		7.58s	568	0

Data parsing and extraction	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	10.0	10.0	100.0%	0		822ms	261	0
GPT-4o-mini	10.0	10.0	100.0%	0		1.27s	183	0

Domain specific	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	5.3	10.0	33.3%	0		367ms	28	0
GPT-4o-mini	3.0	10.0	0.0%	0		637ms	15	0

General Intelligence	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	4.0	10.0	0.0%	0		729ms	205	0
GPT-4o-mini	4.0	10.0	0.0%	0		909ms	66	0

Instructions following	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	6.5	10.0	50.0%	0		380ms	69	0
GPT-4o-mini	6.3	10.0	50.0%	0		1.27s	69	0

Puzzle Solving	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	3.1	9.9	0.0%	0		589ms	170	0
GPT-4o-mini	3.7	10.0	0.0%	0		1.30s	308	0

Tool Calling	Score	Consistency	Attempt pass rate	Flaky tests	Tests Correct	Response Time (avg)	Output Tokens	Reasoning Tokens
Mistral Small 4	10.0	10.0	100.0%	0		1.40s	213	0
GPT-4o-mini	10.0	10.0	100.0%	0		2.51s	205	0

Quick Compare

Switch Comparison Pair

Mistral Small 4nonevsNemotron 3 Nano Omni 30b A3b ReasoningmediumFree Available Mistral Small 4nonevsElephant Alphamedium MiniMax M2.7mediumvsMistral Small 4none Nemotron 3 Nano Omni 30b A3b ReasoningmediumFree AvailablevsGPT-4o-mininone GPT-4o-mininonevsQwen3 Coder Nextmedium GPT-4o-mininonevsElephant Alphamedium MiniMax M2.7mediumvsGPT-4o-mininone Mistral Small 4nonevsQwen3 Coder Nextmedium MiniMax M2.5mediumFree AvailablevsMistral Small 4none GPT-4o-mininonevsGLM 4.7 Flashmedium MiniMax M2.5mediumFree AvailablevsGPT-4o-mininone Mistral Small 4mediumvsGPT-4o-mininone