Workflow Eval Detail

Burned-in Captions

Analyzes video frames to detect hardcoded captions baked into the visual content—useful for compliance checks and accessibility audits.

Latest Runcompleted

muxinc/ai

main·d35ae5a·@mux/ai v0.35.0

Cases

Avg Score

0.97

Avg Latency

5.79s

Avg Cost

$0.003

Avg Cost / Min

$0.0106/min

Avg Tokens

2,910

TL;DR

The workflow scored 0.9683 overall, with Anthropic leading quality, OpenAI gpt-5.1 leading latency, and OpenAI gpt-5.6-luna offering the lowest cost.

Best Quality

anthropic

claude-sonnet-4-5

Fastest

openai

gpt-5.1

Most Economical

openai

gpt-5.6-luna

What we measure

Each eval run captures efficacy, efficiency, and expense. We use this data to compare providers and track regressions over time.

Efficacy

Quality + correctness

Efficiency

Latency + token usage

Expense

Cost per request

Workflow snapshot

Suite statussuccess

Suite average score0.97

Suite duration2 minutes 1 second

Last suite runAug 25, 12:48 PM

Evaluation criteria

From eval tests

We evaluate caption detection accuracy, confidence calibration, and response integrity alongside speed and cost thresholds.

Captions Detected

[ DIRECTOR YELLING SCENE: 1, TAKE: 9 ]

Spatial Decomposition

Efficacy checks

Caption presence matches ground truth labels.
Confidence >0.8 when captions are expected.
Confidence is 0-1, language is string or null, and storyboard URL is HTTPS.

Efficiency targets

Latency: scores are normalized between 0 and 1. Under 5s earns 1.0; past 12s trends toward 0.
Token usage: scores are normalized between 0 and 1. Under 4,000 tokens earns 1.0; higher usage reduces the score.

Expense guardrails

Estimated cost under $0.012 per request for full score.
Usage data must include total tokens for cost analysis.

Provider breakdown

Run d35ae5a

Efficacy scoreHigher is better

LatencyLower is better

Token UsageLower is better

CostLower is better

Provider	Model	Cases	Avg Score	Avg Latency	Avg Tokens	Avg Cost	Avg Cost / Min
anthropic	claude-sonnet-4-5	3	1	2.72s	3,179	$0.0099	$0.0345/min
google	gemini-2.5-flash	3	0.99	6.27s	2,599	$0.0026	$0.0092/min
google	gemini-3-flash-preview	3	0.99	5.74s	3,170	$0.003	$0.0106/min
google	gemini-3.1-flash-lite	3	1	2.36s	2,624	$0.0007	$0.0024/min
openai	gpt-5-mini	3	0.9	17.14s	3,649	$0.0023	$0.0082/min
openai	gpt-5.1	3	0.9	1.73s	2,279	$0.0022	$0.0077/min
openai	gpt-5.6-luna	3	1	4.56s	2,871	$0.0003	$0.0012/min

Recent cases

Latest run · 21 cases

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset gIRjPqM

Score

Latency

Cost

$0.0088

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset gEvCHSJ

Score

Latency

2.65s

Cost

$0.0104

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset atuutlT

Score

Latency

2.51s

Cost

$0.0104

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset gIRjPqM

Score

Latency

5.03s

Cost

$0.0021

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset gEvCHSJ

Score

0.97

Latency

7.98s

Cost

$0.0038

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset atuutlT

Score

0.99

Latency

5.81s

Cost

$0.002

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset gIRjPqM

Score

Latency

3.95s

Cost

$0.0028

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset gEvCHSJ

Score

Latency

4.83s

Cost

$0.003

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset atuutlT

Score

0.96

Latency

8.44s

Cost

$0.0033

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset gIRjPqM

Score

Latency

1.6s

Cost

$0.0007

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset gEvCHSJ

Score

Latency

2.01s

Cost

$0.0007

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset atuutlT

Score

Latency

3.48s

Cost

$0.0007

openai ·gpt-5-miniAug 25, 12:50 PM

Asset gIRjPqM

Score

0.94

Latency

10.78s

Cost

$0.0018

openai ·gpt-5-miniAug 25, 12:50 PM

Asset gEvCHSJ

Score

0.84

Latency

27.31s

Cost

$0.0036

openai ·gpt-5-miniAug 25, 12:50 PM

Asset atuutlT

Score

0.92

Latency

13.33s

Cost

$0.0017

openai ·gpt-5.1Aug 25, 12:50 PM

Asset gIRjPqM

Score

Latency

1.37s

Cost

$0.0014

openai ·gpt-5.1Aug 25, 12:50 PM

Asset gEvCHSJ

Score

0.71

Latency

1.79s

Cost

$0.0033

openai ·gpt-5.1Aug 25, 12:50 PM

Asset atuutlT

Score

Latency

2.02s

Cost

$0.0019

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset gIRjPqM

Score

Latency

5.03s

Cost

$0.0004

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset gEvCHSJ

Score

0.99

Latency

5.55s

Cost

$0.0005

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset atuutlT

Score

Latency

3.1s

Cost

$0.0002