Workflow Eval Detail

Caption Translation

Converts captions into multiple languages, helping you reach global audiences without manual translation work.

Latest Runcompleted

muxinc/ai

main·d8d0122·@mux/ai v0.53.0

Cases

Avg Score

0.96

Avg Latency

9.6s

Avg Cost

$0.0069

Avg Cost / Min

$0.012/min

Avg Tokens

3,003

TL;DR

gpt-5.6-luna provides the strongest quality and lowest cost, while gemini-3.1-flash-lite provides the fastest execution with similarly high quality.

Best Quality

openai

gpt-5.6-luna

Fastest

google

gemini-3.1-flash-lite

Most Economical

openai

gpt-5.6-luna

What we measure

Each eval run captures efficacy, efficiency, and expense. We use this data to compare providers and track regressions over time.

Efficacy

Quality + correctness

Efficiency

Latency + token usage

Expense

Cost per request

Workflow snapshot

Suite statussuccess

Suite average score0.96

Suite duration4 minutes 17 seconds

Last suite runOct 16, 09:36 PM

Evaluation criteria

From eval tests

We validate VTT structure, translation faithfulness, and language code integrity, plus performance and budget targets.

English Translation

"Together we can reach more"

EN / ENG • Confidence 100%

Contextual Localization

Efficacy checks

Translated VTT starts with WEBVTT and keeps timestamps.
Cue count matches the original and translation differs.
Faithfulness scoring against the original transcript.
Language codes match ISO 639-1/3 and are consistent.
Response preserves asset ID and language fields.

Efficiency targets

Latency: scores are normalized between 0 and 1. Under 8s earns 1.0; past 15s trends toward 0.
Token usage: scores are normalized between 0 and 1. Under 2,500 tokens earns 1.0; higher usage reduces the score.

Expense guardrails

Estimated cost under $0.012 per request for full score.
Usage data must include total tokens for cost analysis.

Provider breakdown

Run d8d0122

Efficacy scoreHigher is better

LatencyLower is better

Token UsageLower is better

CostLower is better

Provider	Model	Cases	Avg Score	Avg Latency	Avg Tokens	Avg Cost	Avg Cost / Min
anthropic	claude-sonnet-4-5	3	0.99	7.17s	1,963	$0.0122	$0.0213/min
google	gemini-2.5-flash	3	0.98	7.05s	2,897	$0.0043	$0.0076/min
google	gemini-3-flash-preview	3	0.82	24.43s	7,668	$0.0197	$0.0344/min
google	gemini-3.1-flash-lite	3	1	1.93s	1,924	$0.0012	$0.0022/min
openai	gpt-5-mini	3	0.93	17.5s	3,228	$0.0042	$0.0073/min
openai	gpt-5.1	3	1	3.34s	1,602	$0.0058	$0.0102/min
openai	gpt-5.6-luna	3	1	5.76s	1,740	$0.0007	$0.0012/min

Recent cases

Latest run · 21 cases

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

6.56s

Cost

$0.012

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 88Lb01q

Score

0.99

Latency

6.59s

Cost

$0.0121

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 88Lb01q

Score

0.99

Latency

8.35s

Cost

$0.0126

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 88Lb01q

Score

0.99

Latency

5.96s

Cost

$0.0039

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

4.89s

Cost

$0.0031

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 88Lb01q

Score

0.95

Latency

10.31s

Cost

$0.006

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 88Lb01q

Score

0.77

Latency

34.33s

Cost

$0.0271

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 88Lb01q

Score

0.84

Latency

20.37s

Cost

$0.0169

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 88Lb01q

Score

0.85

Latency

18.59s

Cost

$0.0151

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

1.7s

Cost

$0.0012

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.05s

Cost

$0.0012

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.02s

Cost

$0.0012

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 88Lb01q

Score

0.96

Latency

13.94s

Cost

$0.0033

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 88Lb01q

Score

0.92

Latency

18.78s

Cost

$0.0046

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 88Lb01q

Score

0.92

Latency

19.77s

Cost

$0.0047

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

3.78s

Cost

$0.0057

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.88s

Cost

$0.0058

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

3.37s

Cost

$0.006

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

5.28s

Cost

$0.0007

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

6.77s

Cost

$0.0007

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

5.25s

Cost

$0.0007