Workflow Eval Detail

Caption Translation

Converts captions into multiple languages, helping you reach global audiences without manual translation work.

Latest Runcompleted

muxinc/ai

main·d35ae5a·@mux/ai v0.35.0

Cases

Avg Score

0.97

Avg Latency

10.73s

Avg Cost

$0.0054

Avg Cost / Min

$0.0094/min

Avg Tokens

2,502

TL;DR

The workflow achieved 96.9% overall accuracy; use gpt-5.1 for quality, gemini-3.1-flash-lite for latency, and gpt-5.6-luna for lowest cost.

Best Quality

openai

gpt-5.1

Fastest

google

gemini-3.1-flash-lite

Most Economical

openai

gpt-5.6-luna

What we measure

Each eval run captures efficacy, efficiency, and expense. We use this data to compare providers and track regressions over time.

Efficacy

Quality + correctness

Efficiency

Latency + token usage

Expense

Cost per request

Workflow snapshot

Suite statussuccess

Suite average score0.97

Suite duration4 minutes 45 seconds

Last suite runAug 25, 12:48 PM

Evaluation criteria

From eval tests

We validate VTT structure, translation faithfulness, and language code integrity, plus performance and budget targets.

English Translation

"Together we can reach more"

EN / ENG • Confidence 100%

Contextual Localization

Efficacy checks

Translated VTT starts with WEBVTT and keeps timestamps.
Cue count matches the original and translation differs.
Faithfulness scoring against the original transcript.
Language codes match ISO 639-1/3 and are consistent.
Response preserves asset ID and language fields.

Efficiency targets

Latency: scores are normalized between 0 and 1. Under 8s earns 1.0; past 15s trends toward 0.
Token usage: scores are normalized between 0 and 1. Under 2,500 tokens earns 1.0; higher usage reduces the score.

Expense guardrails

Estimated cost under $0.012 per request for full score.
Usage data must include total tokens for cost analysis.

Provider breakdown

Run d35ae5a

Efficacy scoreHigher is better

LatencyLower is better

Token UsageLower is better

CostLower is better

Provider	Model	Cases	Avg Score	Avg Latency	Avg Tokens	Avg Cost	Avg Cost / Min
anthropic	claude-sonnet-4-5	3	1	7.64s	1,964	$0.0122	$0.0213/min
google	gemini-2.5-flash	3	1	4.68s	2,252	$0.0027	$0.0048/min
google	gemini-3-flash-preview	3	0.9	15.62s	4,618	$0.0106	$0.0184/min
google	gemini-3.1-flash-lite	3	1	2.79s	1,921	$0.0012	$0.0021/min
openai	gpt-5-mini	3	0.89	34.36s	3,480	$0.0047	$0.0082/min
openai	gpt-5.1	3	1	5.35s	1,603	$0.0058	$0.0102/min
openai	gpt-5.6-luna	3	1	4.68s	1,675	$0.0006	$0.0011/min

Recent cases

Latest run · 21 cases

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

7.24s

Cost

$0.012

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

6.95s

Cost

$0.0121

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 88Lb01q

Score

0.99

Latency

8.72s

Cost

$0.0126

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

3.79s

Cost

$0.0021

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

5.16s

Cost

$0.0031

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

5.08s

Cost

$0.003

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 88Lb01q

Score

0.96

Latency

10.63s

Cost

$0.0066

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 88Lb01q

Score

0.88

Latency

17.78s

Cost

$0.0117

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 88Lb01q

Score

0.87

Latency

18.47s

Cost

$0.0134

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

3.02s

Cost

$0.0012

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

2.64s

Cost

$0.0012

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

2.71s

Cost

$0.0012

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 88Lb01q

Score

0.92

Latency

25.61s

Cost

$0.0036

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 88Lb01q

Score

0.89

Latency

31.95s

Cost

$0.0045

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 88Lb01q

Score

0.87

Latency

45.52s

Cost

$0.0059

openai ·gpt-5.1Aug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

6.18s

Cost

$0.0056

openai ·gpt-5.1Aug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

4.53s

Cost

$0.0058

openai ·gpt-5.1Aug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

5.34s

Cost

$0.0061

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

4.45s

Cost

$0.0006

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

4.75s

Cost

$0.0007

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset 88Lb01q

Score

Latency

4.85s

Cost

$0.0007