Workflow Eval Detail

Summarization

Generates concise summaries and smart tags from your content—perfect for search, discovery, and quick recaps.

Latest Runcompleted

muxinc/ai

main·d8d0122·@mux/ai v0.53.0

Cases

Avg Score

0.99

Avg Latency

4.83s

Avg Cost

$0.0037

Avg Cost / Min

$0.0066/min

Avg Tokens

3,433

TL;DR

Quality was high overall; choose claude-sonnet-4-5 for the highest score, gemini-3.1-flash-lite for lowest latency, or gpt-5.6-luna for lowest cost.

Best Quality

anthropic

claude-sonnet-4-5

Fastest

google

gemini-3.1-flash-lite

Most Economical

openai

gpt-5.6-luna

What we measure

Each eval run captures efficacy, efficiency, and expense. We use this data to compare providers and track regressions over time.

Efficacy

Quality + correctness

Efficiency

Latency + token usage

Expense

Cost per request

Workflow snapshot

Suite statussuccess

Suite average score0.96

Suite duration40.51s

Last suite runOct 16, 09:36 PM

Evaluation criteria

From eval tests

We score summary quality, tag relevance, and semantic similarity while tracking latency, token usage, and cost.

...and so when we look at the numbers...

...the growth trajectory has been...

...our teams have worked incredibly hard...

...which brings me to the next point...

Analyzing

→

↓

AI Summary8:00 duration

6 tags extracted

●Complete

Semantic Extraction

Efficacy checks

Title is non-empty, <=100 chars, and avoids filler starters.
Description is non-empty, <=1000 chars, and avoids meta phrases.
Tags are non-empty strings, unique, and <=10 items.
Title, description, and tags are semantically similar to references.
Response includes asset ID and HTTPS storyboard URL.

Efficiency targets

Latency: scores are normalized between 0 and 1. Under 8s earns 1.0; past 20s trends toward 0.
Token usage: scores are normalized between 0 and 1. Under 4,000 tokens earns 1.0; higher usage reduces the score.
Usage data must include input and output tokens > 0.

Expense guardrails

Estimated cost under $0.015 per request for full score.

Provider breakdown

Run d8d0122

Efficacy scoreHigher is better

LatencyLower is better

Token UsageLower is better

CostLower is better

Provider	Model	Cases	Avg Score	Avg Latency	Avg Tokens	Avg Cost	Avg Cost / Min
anthropic	claude-sonnet-4-5	4	0.99	5.66s	3,850	$0.0132	$0.0229/min
google	gemini-2.5-flash	4	0.99	6.75s	3,041	$0.0029	$0.0055/min
google	gemini-3-flash-preview	4	0.99	5.36s	3,471	$0.0033	$0.0053/min
google	gemini-3.1-flash-lite	4	0.99	2.32s	2,981	$0.0009	$0.0016/min
openai	gpt-5-mini	4	0.97	6.94s	4,457	$0.0018	$0.0034/min
openai	gpt-5.1	4	0.99	2.64s	2,514	$0.0026	$0.0074/min
openai	gpt-5.6-luna	3	0.99	3.96s	3,812	$0.0003	$0.0004/min

Recent cases

Latest run · 27 cases

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 88Lb01q

Score

0.98

Latency

6.58s

Cost

$0.0131

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

5.16s

Cost

$0.0129

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

5.28s

Cost

$0.0131

anthropic ·claude-sonnet-4-5Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

5.61s

Cost

$0.0136

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 88Lb01q

Score

0.95

Latency

6.66s

Cost

$0.0032

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

5.98s

Cost

$0.0026

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

8.26s

Cost

$0.0035

google ·gemini-2.5-flashOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

6.11s

Cost

$0.0025

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 88Lb01q

Score

0.97

Latency

5.4s

Cost

$0.0031

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

7.54s

Cost

$0.0046

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

3.78s

Cost

$0.0024

google ·gemini-3-flash-previewOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

4.7s

Cost

$0.003

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 88Lb01q

Score

0.96

Latency

2.36s

Cost

$0.0009

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.3s

Cost

$0.0009

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.34s

Cost

$0.0009

google ·gemini-3.1-flash-liteOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.27s

Cost

$0.0009

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 88Lb01q

Score

0.94

Latency

8.7s

Cost

$0.002

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 88Lb01q

Score

0.98

Latency

7.34s

Cost

$0.0019

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 88Lb01q

Score

0.99

Latency

5.54s

Cost

$0.0016

openai ·gpt-5-miniOct 16, 09:38 PM

Asset 88Lb01q

Score

0.98

Latency

6.18s

Cost

$0.0018

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 88Lb01q

Score

0.96

Latency

3.19s

Cost

$0.0042

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.72s

Cost

$0.0017

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.34s

Cost

$0.0017

openai ·gpt-5.1Oct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

2.31s

Cost

$0.0027

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 88Lb01q

Score

0.97

Latency

5.21s

Cost

$0.0003

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

3.3s

Cost

$0.0003

openai ·gpt-5.6-lunaOct 16, 09:38 PM

Asset 88Lb01q

Score

Latency

3.36s

Cost

$0.0003