vfxreview/DELIVERY_WORKFLOW.md

# Delivery Packages — Workflow Guide

How a finished shot becomes a dated folder ready to upload to the client.

There are two kinds of package. An **EXR delivery** is the shippable one and is
what most of this guide describes. A **MOV review** is a flat folder of preview
MOVs shared for a formal review — see
[MOV review packages](#mov-review-packages) at the end.

This is the operational guide. The design rationale and normative rules live in
[RenderPipeline2.md](RenderPipeline2.md) §11.

---

## The short version

1. A shot renders, validates, gets a preview, and **passes QC** → it becomes
   eligible for delivery.
2. On **Pipeline → Deliveries**, pick the project, see *"Episode 101 — Ready:
   18 shots"*, click **Build delivery**.
3. A render worker builds the folder on the delivery volume, verifies every
   file, and writes a manifest and CSV report.
4. A human uploads that folder to the client and clicks **Record upload**,
   pasting the link to where it now lives.
5. They send the client that link and click **Share link**, recording who it
   went to. Only then is the package `DELIVERED`.

Nothing is ever sent to the client automatically. Steps 4 and 5 are deliberately
manual — but until both are recorded the package sits in **Awaiting delivery**
at the top of the dashboard.

---

## What makes a shot eligible

An Export reaches `READY_FOR_DELIVERY` only by passing QC — from the AE panel
or the web. That is the single gate; there is no way to package an export that
has not been QC'd.

A render that failed automated validation can still get there: **Override
validation** on the Export detail page (admin/producer/supervisor, reason
required) resumes the pipeline at the preview stage, so the shot still goes
through QC like any other. The failed checks stay on record and keep appearing
as warnings in the QC queue, so whoever QCs it knows what was waved through.

Two rules worth knowing:

- **Newest version per shot only.** If a shot has two exports sitting ready, the
  older one is history and is not offered. Building refuses outright if you
  somehow select two versions of the same shot.
- **Version locking.** A package references the exact Export rows it was built
  from and copies the shot code and version string at creation time. Re-render a
  shot afterwards and the package is untouched — delivery history is immutable
  by construction.

---

## Building a package

**Pipeline → Deliveries**, choose a project. Each episode with ready shots shows
its count and the shots themselves, with frame counts and sizes.

**Build delivery** creates the package (`QUEUED`) and a `DELIVERY_BUILD` job on
the same queue the renders use. Requires **admin, producer or supervisor**.

Delivery jobs run at priority 10, which is inside the urgent threshold — so they
claim immediately even during work hours, when a render would wait for the
evening window. Packaging is file work with no After Effects involvement, so it
does not disturb an artist at the machine.

Watch the package status on the same page:

| Status | Meaning |
|---|---|
| `QUEUED` | Waiting for a worker to pick it up |
| `BUILDING` | A worker is creating folders and linking files |
| `READY` | Built and verified. Shown as *Built — not uploaded*, then *Uploaded — link not shared* |
| `DELIVERED` | The upload location and who the link was sent to are both on record |
| `FAILED` | Something did not verify — see the error and the per-shot list |
| `CANCELLED` | Abandoned before delivery |

---

## What lands on disk

Default layout (the studio's confirmed structure):

```
{deliveryRoot}/{episode}/{YYMMDD}_Delivery/
├── {SHOT_CODE}/
│   ├── {shot}_cmp_TT_{ver}.[#####].exr     ← the EXR sequence
│   └── {shot}_cmp_TT_{ver}.mov             ← slate/burn-in delivery MOV
├── manifest.json                            ← every file, size, checksum, source export
└── {YYMMDD}_Delivery_report.csv             ← one row per shot (client-facing)
```

EXRs and the MOV sit side by side in each shot folder — no `exr/` and `mov/`
subfolders, matching current practice.

The CSV carries: shot, version, frame range, frame count, file count, bytes,
checksum, MOV filename and status. Confirm these columns against the
production's delivery spec before the first real delivery.

---

## Hard links, and why the package appears instantly

Files are **hard-linked**, not copied, when the source and destination are on the
same volume. A hard link is a second name for the same data — not a shortcut and
not a copy — so an 18-shot EXR package builds in seconds and consumes
essentially no extra disk.

Verified on the studio SAN (`\\TRUENAS\vfx` over SMB): hard links work there, and
renders and deliveries live on the same share.

**Is that safe?** Deleting the delivery folder does not delete the render — the
data survives until the last name referring to it is gone. And nothing in the
pipeline ever rewrites an existing EXR: a new version is always a new directory.

Two escape hatches:

- A delivery root on a **different volume** falls back to copying automatically.
- `"copyMode": "always-copy"` in the project's delivery config forces real copies
  for a production that mandates physical separation.

The `linkMode` column on each item records what actually happened, so a package
never claims to have linked when it copied.

---

## Verification

Verification is proportional to what the builder actually did, which is what
keeps a package fast:

| Mode | How it verifies | Cost for a 25-shot episode |
|---|---|---|
| **Hard-linked** (default) | The entry *is* the source data, so the validation-time digest already describes it. Checks each file exists, and that no source was modified after it passed QC. | seconds |
| **Copied** | Read back in full and hashed; the sequence digest must match the one recorded at validation. | ~30 s per 2.8 GB shot |

Re-reading a hard link cannot disagree with validation — it is the same bytes
under a second name — and at ~100 MB/s over SMB it costs roughly half an hour
per episode to prove something that cannot be false. A copy is genuinely
different: it can be truncated or corrupted, so it is always read back.

Set `"verifyMode": "always-hash"` in the delivery config to read everything back
regardless, for a production that requires it contractually.

An item fails, and the whole package fails with it, when:

| Failure | Meaning |
|---|---|
| Checksum mismatch (copied items) | The delivered bytes differ from what passed QC |
| A source file modified after validation | The render changed after it passed QC — a link would carry the new bytes faithfully |
| Frame count differs from the export record | The render folder changed after QC — someone re-rendered in place |
| Source folder missing | The render is not where the export says it is |
| No matching files | The folder exists but holds nothing matching the output pattern |

**A package only reaches `READY` when every shot verified.** One bad shot means
`FAILED`, and no export is marked as packaged — there is no partial delivery.

A missing delivery MOV is the one non-fatal case: the EXRs are the deliverable,
so it is logged and the package continues without it.

> **Without validation there is nothing to verify against.** An export whose
> checksum was never recorded (validation disabled, or rendered before it
> shipped) still packages, but the digest is computed with nothing to compare it
> to. The ready list flags these as *"N without a validation checksum"*.

---

## Marking delivered

A built package is not a delivered one, so delivery is two recorded steps. Both
require admin, producer or supervisor.

1. **Record upload** — paste the link to the folder on the client's portal and
   confirm it matches the shot, frame and MOV counts shown. The link can be
   corrected until it is shared.
2. **Share link** — copy the generated message (link, shots, versions), send it,
   and record who it went to. This moves every packaged export to `DELIVERED`.

The second step is refused until the first is on record. Meanwhile the package
is listed under **Awaiting delivery** on the dashboard, amber after a day and
red after two.

This is still a human statement of fact — the client's portal has no API, so
the app cannot see the upload itself. What it guarantees is that nothing reads
as delivered without a link, a recipient, a name and a time behind it. Packages
delivered before this existed show as delivered with no evidence recorded.

---

## Delivery requests

When a client asks for something ("EXRs for 116"), log it so it cannot be
forgotten. Requires admin, producer or supervisor.

- **Dashboard → Awaiting delivery → Log request**, or the same button on
  **Pipeline → Deliveries**: pick the project, then a whole episode or a pasted
  list of shot codes (ours or the client's; a code that matches nothing is
  refused rather than skipped).
- **Shot Status → Bulk Actions → Log Delivery Request** for a ticked selection.

The shot list is a snapshot taken when the request is logged. The request then
sits in **Awaiting delivery** showing *"3 of 9 delivered"*, and on the
deliveries page every shot shows where it stands: not through QC, in QC, ready
to package, building, built, uploaded, delivered. **Build N ready** packages
the shots that have passed QC.

A request has no "done" button. A shot's line closes only when:

- a package containing it is shared (step 2 of marking delivered) after the
  request was logged, or
- it is marked **Delivered Outside Pipeline** with the upload link and who the
  link was sent to, dated on or after the day the request was logged. For a
  shot on an open request those two fields are required.

The request closes when every line has. A delivery made *before* the request
was logged does not count — a client asking again is asking for it again. The
only other way out is **Cancel request**, which needs a reason and stays in the
history.

The *Wanted* field (EXRs, MOVs, both) records what was asked for; it does not
change what closes a line. A MOV-only ask is closed through Delivered Outside
Pipeline, since a MOV review package has no delivered state.

---

## When something goes wrong

**A package failed.** Expand it: every shot shows its own status and error. Fix
the cause (usually a re-render after QC), then build a new package. Packages are
never deleted; a mistaken one is cancelled.

**The worker died mid-build.** The package is marked `FAILED` with *"check the
package folder before rebuilding"*. It is deliberately not retried
automatically — a half-written folder on the delivery volume is something a
human should look at first.

**Cancelling.** Only `DRAFT`, `QUEUED` and `FAILED` packages can be cancelled. A
`READY` or `DELIVERED` package stays in the history permanently; supersede it by
building a new one.

---

## Configuration

Per project, in `Project.deliveryConfig`; studio-wide defaults in the
`delivery.defaultConfig` SystemConfig key; built-in defaults otherwise. Merged
key by key, so a project can override just `deliveryRoot`.

```json
{
  "packageNameTemplate": "{yymmdd}_Delivery",
  "layout": {
    "packageDir": "{deliveryRoot}/{episode}/{packageName}",
    "shotDir":    "{packageDir}/{shotCode}",
    "reportFile": "{packageDir}/{packageName}_report.csv"
  },
  "deliveryRoot": "//SAN/DELIVERY",
  "includeMovs": true,
  "checksumAlgo": "xxh64",
  "reportFormats": ["csv", "json"],
  "copyMode": "auto",
  "verifyMode": "auto"
}
```

Tokens: `{showId} {episode} {yymmdd} {yyyymmdd} {packageName} {packageDir}
{deliveryRoot} {shotCode} {versionString} {packageNumber}`.

The resolved config is **snapshotted onto the package** at build time, so a
historic package always shows the rules it was built with, and a production with
a different spec is a config change rather than a code change.

`deliveryRoot` must be reachable *from the worker machine* — check with
`VFXReviewWorker.exe --check` if a build fails on paths.

---

## API

| # | Method | Path | Purpose |
|---|--------|------|---------|
| E16 | `POST` | `/api/deliveries` | Create a package and queue the build |
| E17 | `GET` | `/api/deliveries` | List packages / history |
| E18 | `GET` | `/api/deliveries/ready` | Ready exports grouped by episode |
| — | `POST` | `/api/deliveries/{id}/uploaded` | Record the upload: `{ uploadUrl, contentsConfirmed: true }` |
| E19 | `POST` | `/api/deliveries/{id}/delivered` | Record the share and mark delivered: `{ sharedWith, note? }` |
| — | `POST` | `/api/deliveries/{id}/cancel` | Cancel a package |
| — | `GET` | `/api/delivery-requests` | List requests with per-shot stages (`?projectId=`, `?open=1`) |
| — | `POST` | `/api/delivery-requests` | Log a request: `{ projectId, episode \| shotIds \| shotCodes, deliverable, requestedBy, dueDate?, note? }` |
| — | `POST` | `/api/delivery-requests/{id}/cancel` | Cancel a request: `{ reason }` |
| — | `POST` | `/api/ext/render/jobs/{jobId}/delivery` | Worker reports a build (API key) |

`POST /api/deliveries` takes `{ projectId, episode?, exportIds?, options? }`.
Omit `exportIds` to take every ready export in the episode; `options` accepts
`date`, `name`, `includeMovs` and `copyMode` to override the config for one
build.

---

## MOV review packages

For a formal review you share the preview MOVs, not the EXRs. That is its own
kind of package, built the same way and listed on the same page.

### Building one

**Shot Status**, tick the shots, then **Bulk Actions → Build MOV Review
Package**. A dialog shows exactly what would be built before anything happens:
the folder name, the version each shot contributes, and any shots being left
out. Requires **admin, producer or supervisor**.

The version each shot contributes is its **newest export that actually has a
preview MOV** — not necessarily its newest export. A shot whose latest version
is still rendering goes into the review as the version that can be watched.
A shot with no MOV at all is listed as left out, with the reason, rather than
quietly reducing the folder.

### What lands on disk

```
{deliveryRoot}/PRODUCER_REVIEWS/20260828_Producer_review/
├── EP6_VFX_012_v004.mov
├── EP6_VFX_013_v002.mov
└── UNG_106_010_040_cmp_TT_v001.mov
```

Flat — the MOVs sit beside each other so they can be scrubbed through in order.
Names follow the same client-naming rule as a delivery: the production's own
code where it has one, ours where it does not, so a shot reads identically in a
review folder and in the delivery that follows it.

The folder is **exactly the MOVs**: no CSV, no `manifest.json` beside them. The
manifest still goes back to the app, so the record of what was shared is kept
without cluttering the share.

Second review the same day is `20260828_Producer_review_02`, then `_03`. A
number is never reused, even if the package that took it was cancelled.

### What it does not do

**A review never moves a shot along.** Reviews happen throughout a shot's life,
long before QC passes it, and the same version is routinely shown twice — so
building one changes nothing about the exports in it, and there is no
**Mark delivered** on a review package. Shots stay exactly where they are in
the pipeline.

A **missing MOV fails the package** here, unlike in an EXR delivery where a
missing MOV is logged and the build continues. In a review the MOV is the
deliverable, and a review folder quietly one shot short is worse than one that
says so.

### Configuration

Two more keys in the same `deliveryConfig` (§11.6):

```json
{
  "reviewNameTemplate": "{yyyymmdd}_Producer_review",
  "layout": {
    "reviewDir": "{deliveryRoot}/PRODUCER_REVIEWS/{packageName}"
  }
}
```