Workflow Eval Detail

Chapters

Automatically segments long-form video content into navigable chapters with timestamps and titles—enabling viewers to jump to key moments instantly.

Latest Runcompleted

muxinc/ai

main·d8d0122·@mux/ai v0.53.0

Cases

32

Avg Score

0.97

Avg Latency

5.67s

Avg Cost

$0.0052

Avg Cost / Min

$0.0006/min

Avg Tokens

4,329

TL;DR

Chapters performed strongly overall; use `gemini-3.1-flash-lite` for the best combined quality and latency, or `gpt-5.6-luna` to minimize cost.

Best Quality

google

gemini-3.1-flash-lite

Fastest

google

gemini-3.1-flash-lite

Most Economical

openai

gpt-5.6-luna

What we measure

Each eval run captures efficacy, efficiency, and expense. We use this data to compare providers and track regressions over time.

Efficacy

Quality + correctness

Efficiency

Latency + token usage

Expense

Cost per request

Workflow snapshot

Suite statussuccess

Suite average score0.96

Suite duration1 minute 15 seconds

Last suite runOct 16, 09:36 PM

Evaluation criteria

From eval tests

We evaluate chapter segmentation quality, timestamp accuracy, and title relevance alongside latency and cost metrics.

0:0020:00

Segmenting

→

↓

AI Chapters6 segments

1

Introduction & Welcome

0:00

2

Setting Up Your Environment

2:22

3

Core Concepts Deep Dive

6:25

4

Building the First Feature

10:12

5

Testing & Debugging

14:51

6

Deployment & Next Steps

18:44

Language: en

●Processing...

Temporal Segmentation

Efficacy checks

Chapter titles are non-empty and descriptive.
Start times are valid and monotonically increasing.
Chapter count is reasonable for video duration.
Language code matches expected ISO 639-1 format.
Semantic coherence between chapter titles and content.

Efficiency targets

Latency: scores are normalized between 0 and 1. Under 10s earns 1.0; past 25s trends toward 0.
Token usage: scores are normalized between 0 and 1. Under 5,000 tokens earns 1.0; higher usage reduces the score.

Expense guardrails

Estimated cost under $0.015 per request for full score.
Usage data must include total tokens for cost analysis.

Provider breakdown

Run d8d0122

Efficacy scoreHigher is better

LatencyLower is better

Token UsageLower is better

CostLower is better

Provider	Model	Cases	Avg Score	Avg Latency	Avg Tokens	Avg Cost	Avg Cost / Min
anthropic	claude-sonnet-4-5	5	0.99	3.8s	3,847	$0.0132	$0.0015/min
google	gemini-2.5-flash	5	0.98	7.53s	5,189	$0.0047	$0.0005/min
google	gemini-3-flash-preview	5	0.95	8.8s	5,625	$0.0078	$0.0007/min
google	gemini-3.1-flash-lite	5	0.99	1.31s	3,803	$0.0012	$0.0001/min
openai	gpt-5-mini	5	0.94	10.62s	4,510	$0.003	$0.0003/min
openai	gpt-5.1	4	0.98	2.39s	3,363	$0.0037	$0.0006/min
openai	gpt-5.6-luna	3	0.98	3.84s	3,399	$0.0003	$0.00/min

Recent cases

Latest run · 32 cases

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

3.98s

Cost

$0.0133

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

2.39s

Cost

$0.012

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

3.96s

Cost

$0.0134

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

4.37s

Cost

$0.0135

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

4.3s

Cost

$0.0139

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.96

Latency

6.25s

Cost

$0.0042

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.95

Latency

8.23s

Cost

$0.0051

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.99

Latency

7.43s

Cost

$0.0051

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.98

Latency

8.54s

Cost

$0.005

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

7.24s

Cost

$0.0041

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.96

Latency

5.71s

Cost

$0.0057

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.95

Latency

7.68s

Cost

$0.0069

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

4.61s

Cost

$0.0047

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.91

Latency

12.62s

Cost

$0.0108

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.91

Latency

13.39s

Cost

$0.0111

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

1.18s

Cost

$0.0012

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

1.15s

Cost

$0.001

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

1.52s

Cost

$0.0013

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

1.32s

Cost

$0.0013

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

1.37s

Cost

$0.0012

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.84

Latency

9.65s

Cost

$0.0033

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.96

Latency

7.8s

Cost

$0.0026

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.98

Latency

10.52s

Cost

$0.0026

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

12.21s

Cost

$0.0033

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

12.91s

Cost

$0.003

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 1XIUcA9

Score

0.95

Latency

3.09s

Cost

$0.0056

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

2.75s

Cost

$0.0047

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

1.73s

Cost

$0.002

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

2.01s

Cost

$0.0024

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

3.73s

Cost

$0.0003

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 1XIUcA9

Score

0.97

Latency

4.58s

Cost

$0.0002

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 1XIUcA9

Score

1

Latency

3.23s

Cost

$0.0003