docs: delivery packages workflow guide

RenderPipeline2.md §11 carries the design and the normative rules, but there was nothing telling someone how to actually build and ship a package. DELIVERY_WORKFLOW.md covers the path from "QC passed" to "client has it": eligibility, building, what lands on disk, verification and its failure modes, marking delivered, and configuration. It explains hard links properly — a second name for the same data, not a copy or a shortcut — since that is the part most likely to worry someone looking at a package that appeared instantly and used no disk. Also documents the DELIVERY_BUILD stage in the worker README, which previously mentioned deliveries only in passing, and adds a documentation index to the root README. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 18:47:54 +02:00
parent f4a2fee492
commit c4a85bac2b
3 changed files with 250 additions and 0 deletions
@@ -0,0 +1,219 @@
+# Delivery Packages — Workflow Guide
+
+How a finished shot becomes a dated folder ready to upload to the client.
+
+This is the operational guide. The design rationale and normative rules live in
+[RenderPipeline2.md](RenderPipeline2.md) §11.
+
+---
+
+## The short version
+
+1. A shot renders, validates, gets a preview, and **passes QC** → it becomes
+   eligible for delivery.
+2. On **Pipeline → Deliveries**, pick the project, see *"Episode 101 — Ready:
+   18 shots"*, click **Build delivery**.
+3. A render worker builds the folder on the delivery volume, verifies every
+   file, and writes a manifest and CSV report.
+4. A human uploads that folder to the client, then clicks **Mark delivered**.
+
+Nothing is ever sent to the client automatically. Step 4 is deliberately manual.
+
+---
+
+## What makes a shot eligible
+
+An Export reaches `READY_FOR_DELIVERY` only by passing QC — from the AE panel
+or the web. That is the single gate; there is no way to package an export that
+has not been QC'd.
+
+Two rules worth knowing:
+
+- **Newest version per shot only.** If a shot has two exports sitting ready, the
+  older one is history and is not offered. Building refuses outright if you
+  somehow select two versions of the same shot.
+- **Version locking.** A package references the exact Export rows it was built
+  from and copies the shot code and version string at creation time. Re-render a
+  shot afterwards and the package is untouched — delivery history is immutable
+  by construction.
+
+---
+
+## Building a package
+
+**Pipeline → Deliveries**, choose a project. Each episode with ready shots shows
+its count and the shots themselves, with frame counts and sizes.
+
+**Build delivery** creates the package (`QUEUED`) and a `DELIVERY_BUILD` job on
+the same queue the renders use. Requires **admin, producer or supervisor**.
+
+Delivery jobs run at priority 10, which is inside the urgent threshold — so they
+claim immediately even during work hours, when a render would wait for the
+evening window. Packaging is file work with no After Effects involvement, so it
+does not disturb an artist at the machine.
+
+Watch the package status on the same page:
+
+| Status | Meaning |
+|---|---|
+| `QUEUED` | Waiting for a worker to pick it up |
+| `BUILDING` | A worker is creating folders and linking files |
+| `READY` | Built and verified — upload it |
+| `DELIVERED` | You confirmed the client has it |
+| `FAILED` | Something did not verify — see the error and the per-shot list |
+| `CANCELLED` | Abandoned before delivery |
+
+---
+
+## What lands on disk
+
+Default layout (the studio's confirmed structure):
+
+```
+{deliveryRoot}/{episode}/{YYMMDD}_Delivery/
+├── {SHOT_CODE}/
+│   ├── {shot}_cmp_TT_{ver}.[#####].exr     ← the EXR sequence
+│   └── {shot}_cmp_TT_{ver}.mov             ← slate/burn-in delivery MOV
+├── manifest.json                            ← every file, size, checksum, source export
+└── {YYMMDD}_Delivery_report.csv             ← one row per shot (client-facing)
+```
+
+EXRs and the MOV sit side by side in each shot folder — no `exr/` and `mov/`
+subfolders, matching current practice.
+
+The CSV carries: shot, version, frame range, frame count, file count, bytes,
+checksum, MOV filename and status. Confirm these columns against the
+production's delivery spec before the first real delivery.
+
+---
+
+## Hard links, and why the package appears instantly
+
+Files are **hard-linked**, not copied, when the source and destination are on the
+same volume. A hard link is a second name for the same data — not a shortcut and
+not a copy — so an 18-shot EXR package builds in seconds and consumes
+essentially no extra disk.
+
+Verified on the studio SAN (`\\TRUENAS\vfx` over SMB): hard links work there, and
+renders and deliveries live on the same share.
+
+**Is that safe?** Deleting the delivery folder does not delete the render — the
+data survives until the last name referring to it is gone. And nothing in the
+pipeline ever rewrites an existing EXR: a new version is always a new directory.
+
+Two escape hatches:
+
+- A delivery root on a **different volume** falls back to copying automatically.
+- `"copyMode": "always-copy"` in the project's delivery config forces real copies
+  for a production that mandates physical separation.
+
+The `linkMode` column on each item records what actually happened, so a package
+never claims to have linked when it copied.
+
+---
+
+## Verification
+
+Every file is read back after packaging and hashed, and the sequence digest is
+compared against the one recorded at validation time. `fsutil hardlink list` does
+not work over SMB, so link identity is never queried — reading the bytes covers
+links and copies through one path.
+
+An item fails, and the whole package fails with it, when:
+
+| Failure | Meaning |
+|---|---|
+| Checksum mismatch | The delivered bytes differ from what passed QC |
+| Frame count differs from the export record | The render folder changed after QC — someone re-rendered in place |
+| Source folder missing | The render is not where the export says it is |
+| No matching files | The folder exists but holds nothing matching the output pattern |
+
+**A package only reaches `READY` when every shot verified.** One bad shot means
+`FAILED`, and no export is marked as packaged — there is no partial delivery.
+
+A missing delivery MOV is the one non-fatal case: the EXRs are the deliverable,
+so it is logged and the package continues without it.
+
+> **Without validation there is nothing to verify against.** An export whose
+> checksum was never recorded (validation disabled, or rendered before it
+> shipped) still packages, but the digest is computed with nothing to compare it
+> to. The ready list flags these as *"N without a validation checksum"*.
+
+---
+
+## Marking delivered
+
+Once you have uploaded the folder, **Mark delivered** moves every packaged
+export to `DELIVERED`. Requires admin, producer or supervisor.
+
+This is the end of the pipeline, and it is a human statement of fact — the app
+has no way to know the client received anything.
+
+---
+
+## When something goes wrong
+
+**A package failed.** Expand it: every shot shows its own status and error. Fix
+the cause (usually a re-render after QC), then build a new package. Packages are
+never deleted; a mistaken one is cancelled.
+
+**The worker died mid-build.** The package is marked `FAILED` with *"check the
+package folder before rebuilding"*. It is deliberately not retried
+automatically — a half-written folder on the delivery volume is something a
+human should look at first.
+
+**Cancelling.** Only `DRAFT`, `QUEUED` and `FAILED` packages can be cancelled. A
+`READY` or `DELIVERED` package stays in the history permanently; supersede it by
+building a new one.
+
+---
+
+## Configuration
+
+Per project, in `Project.deliveryConfig`; studio-wide defaults in the
+`delivery.defaultConfig` SystemConfig key; built-in defaults otherwise. Merged
+key by key, so a project can override just `deliveryRoot`.
+
+```json
+{
+  "packageNameTemplate": "{yymmdd}_Delivery",
+  "layout": {
+    "packageDir": "{deliveryRoot}/{episode}/{packageName}",
+    "shotDir":    "{packageDir}/{shotCode}",
+    "reportFile": "{packageDir}/{packageName}_report.csv"
+  },
+  "deliveryRoot": "//SAN/DELIVERY",
+  "includeMovs": true,
+  "checksumAlgo": "xxh64",
+  "reportFormats": ["csv", "json"],
+  "copyMode": "auto"
+}
+```
+
+Tokens: `{showId} {episode} {yymmdd} {yyyymmdd} {packageName} {packageDir}
+{deliveryRoot} {shotCode} {versionString} {packageNumber}`.
+
+The resolved config is **snapshotted onto the package** at build time, so a
+historic package always shows the rules it was built with, and a production with
+a different spec is a config change rather than a code change.
+
+`deliveryRoot` must be reachable *from the worker machine* — check with
+`VFXReviewWorker.exe --check` if a build fails on paths.
+
+---
+
+## API
+
+| # | Method | Path | Purpose |
+|---|--------|------|---------|
+| E16 | `POST` | `/api/deliveries` | Create a package and queue the build |
+| E17 | `GET` | `/api/deliveries` | List packages / history |
+| E18 | `GET` | `/api/deliveries/ready` | Ready exports grouped by episode |
+| E19 | `POST` | `/api/deliveries/{id}/delivered` | Mark delivered |
+| — | `POST` | `/api/deliveries/{id}/cancel` | Cancel a package |
+| — | `POST` | `/api/ext/render/jobs/{jobId}/delivery` | Worker reports a build (API key) |
+
+`POST /api/deliveries` takes `{ projectId, episode?, exportIds?, options? }`.
+Omit `exportIds` to take every ready export in the episode; `options` accepts
+`date`, `name`, `includeMovs` and `copyMode` to override the config for one
+build.
@@ -166,6 +166,19 @@ prisma/                  — schema.prisma, seed.ts

 ---

+## Documentation
+
+| Doc | Covers |
+|---|---|
+| [RenderPipeline2.md](RenderPipeline2.md) | Render pipeline specification — normative design, state machine, API contracts |
+| [DELIVERY_WORKFLOW.md](DELIVERY_WORKFLOW.md) | Building and shipping delivery packages (how-to) |
+| [VFXReviewConnector.md](VFXReviewConnector.md) | After Effects panel — setup, Shot Builder, QC, diagnostics |
+| [RenderWorker/README.md](RenderWorker/README.md) | Render worker service — packaging, install, validation, delivery builds |
+| [EXT_API_REFERENCE.md](EXT_API_REFERENCE.md) | External API (`/api/ext/*`) used by the panel and workers |
+| [Shot task workflow.md](Shot%20task%20workflow.md) | Shot and task lifecycle in the web app |
+
+---
+
 ## License

 MIT
@@ -89,6 +89,24 @@ Server-side switches (SystemConfig, no worker redeploy): `validation.enabled`,
 `validation.sampleEvery`, `validation.handleFrames`, and `validation.overrides` for per-check
 severity, e.g. `{"exr_type":"warn"}`.

+## Delivery builds
+
+The same service also claims `DELIVERY_BUILD` jobs: creating a dated package
+folder, hard-linking (or copying) each shot's EXRs and delivery MOV into it,
+verifying every file against the validation-time checksum, and writing
+`manifest.json` plus the CSV report.
+
+No After Effects involved, so these are claimed **even while an artist has AE
+open** — and at priority 10, inside the urgent threshold, so they run during the
+day rather than waiting for the render window. Someone is usually waiting on the
+folder.
+
+`deliveryRoot` has to be reachable from this machine under the service account:
+`--check` verifies `pathMappings` targets, which is where a build most often
+falls over.
+
+Full workflow: [DELIVERY_WORKFLOW.md](../DELIVERY_WORKFLOW.md).
+
 ### Checking a render by hand

 ```powershell