Workflow Eval Detail

Chapters

Automatically segments long-form video content into navigable chapters with timestamps and titles—enabling viewers to jump to key moments instantly.

Latest Runcompleted

muxinc/ai

main·d35ae5a·@mux/ai v0.35.0

Cases

32

Avg Score

0.97

Avg Latency

7.11s

Avg Cost

$0.0049

Avg Cost / Min

$0.0006/min

Avg Tokens

4,248

TL;DR

Chapters performed strongly overall, with gpt-5.1 best for quality, gemini-3.1-flash-lite best for latency, and gpt-5.6-luna best for cost.

Best Quality

openai

gpt-5.1

Fastest

google

gemini-3.1-flash-lite

Most Economical

openai

gpt-5.6-luna

What we measure

Each eval run captures efficacy, efficiency, and expense. We use this data to compare providers and track regressions over time.

Efficacy

Quality + correctness

Efficiency

Latency + token usage

Expense

Cost per request

Workflow snapshot

Suite statussuccess

Suite average score0.96

Suite duration1 minute 56 seconds

Last suite runAug 25, 12:48 PM

Evaluation criteria

From eval tests

We evaluate chapter segmentation quality, timestamp accuracy, and title relevance alongside latency and cost metrics.

0:0020:00

Segmenting

→

↓

AI Chapters6 segments

1

Introduction & Welcome

0:00

2

Setting Up Your Environment

2:22

3

Core Concepts Deep Dive

6:25

4

Building the First Feature

10:12

5

Testing & Debugging

14:51

6

Deployment & Next Steps

18:44

Language: en

●Processing...

Temporal Segmentation

Efficacy checks

Chapter titles are non-empty and descriptive.
Start times are valid and monotonically increasing.
Chapter count is reasonable for video duration.
Language code matches expected ISO 639-1 format.
Semantic coherence between chapter titles and content.

Efficiency targets

Latency: scores are normalized between 0 and 1. Under 10s earns 1.0; past 25s trends toward 0.
Token usage: scores are normalized between 0 and 1. Under 5,000 tokens earns 1.0; higher usage reduces the score.

Expense guardrails

Estimated cost under $0.015 per request for full score.
Usage data must include total tokens for cost analysis.

Provider breakdown

Run d35ae5a

Efficacy scoreHigher is better

LatencyLower is better

Token UsageLower is better

CostLower is better

Provider	Model	Cases	Avg Score	Avg Latency	Avg Tokens	Avg Cost	Avg Cost / Min
anthropic	claude-sonnet-4-5	5	0.99	4.31s	3,831	$0.013	$0.0015/min
google	gemini-2.5-flash	5	0.98	6.77s	4,923	$0.004	$0.0004/min
google	gemini-3-flash-preview	5	0.96	10.01s	5,395	$0.0071	$0.0011/min
google	gemini-3.1-flash-lite	5	0.99	1.61s	3,794	$0.0012	$0.0001/min
openai	gpt-5-mini	5	0.93	18.05s	4,502	$0.0027	$0.0003/min
openai	gpt-5.1	4	0.99	2.8s	3,362	$0.0037	$0.0006/min
openai	gpt-5.6-luna	3	0.98	4.17s	3,418	$0.0003	$0.00/min

Recent cases

Latest run · 32 cases

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

4.47s

Cost

$0.013

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

2.83s

Cost

$0.012

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

4.61s

Cost

$0.0134

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

4.87s

Cost

$0.013

anthropic ·claude-sonnet-4-5Aug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

4.77s

Cost

$0.0138

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

5.26s

Cost

$0.0035

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.96

Latency

6.73s

Cost

$0.0041

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

9.44s

Cost

$0.0052

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

6.45s

Cost

$0.0042

google ·gemini-2.5-flashAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

5.98s

Cost

$0.0033

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.86

Latency

21.36s

Cost

$0.015

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

5.83s

Cost

$0.0039

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

5.9s

Cost

$0.0053

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.98

Latency

10.27s

Cost

$0.0056

google ·gemini-3-flash-previewAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

6.71s

Cost

$0.0059

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

1.47s

Cost

$0.0012

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

1.52s

Cost

$0.001

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

1.6s

Cost

$0.0012

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

1.66s

Cost

$0.0012

google ·gemini-3.1-flash-liteAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

1.81s

Cost

$0.0012

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.92

Latency

19.35s

Cost

$0.0028

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.95

Latency

13.79s

Cost

$0.0021

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.91

Latency

22.01s

Cost

$0.0033

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.92

Latency

19.67s

Cost

$0.0029

openai ·gpt-5-miniAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.95

Latency

15.45s

Cost

$0.0023

openai ·gpt-5.1Aug 25, 12:50 PM

Asset 1XIUcA9

Score

0.98

Latency

3.32s

Cost

$0.0054

openai ·gpt-5.1Aug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

2.2s

Cost

$0.0047

openai ·gpt-5.1Aug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

2.69s

Cost

$0.0022

openai ·gpt-5.1Aug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

2.98s

Cost

$0.0024

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.96

Latency

3.64s

Cost

$0.0003

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset 1XIUcA9

Score

0.97

Latency

4.82s

Cost

$0.0002

openai ·gpt-5.6-lunaAug 25, 12:50 PM

Asset 1XIUcA9

Score

1

Latency

4.06s

Cost

$0.0003