Files
vfxreview/DELIVERY_WORKFLOW.md
T
twotalesanimation e66e4868ea
Deploy / deploy (push) Successful in 6m13s
feat(validation): manual override for a failed run
Checks fail on renders that are genuinely fine — a plate the engine
cannot reach, an expectation that does not apply to this shot — and a
VALIDATION_FAILED export was a dead end, blocking delivery on a shot a
human can see is correct.

Override validation on the Export detail page resumes the pipeline where
a pass would have handed off: preview, then READY_FOR_QC. It never skips
QC and never delivers directly, so the frames are still looked at — the
override waives the automated check, not the human one.

The failed ValidationResult rows are left untouched rather than rewritten,
so the QC queue keeps listing them as warnings and whoever QCs the shot
sees what was waved through. The reason is required and recorded with the
checks it covered on the export's timeline, and the validation panel
keeps an "overridden" badge long after the event scrolls away.

Restricted to admin/producer/supervisor: waving a failed check through is
a supervisor's call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 11:19:45 +02:00

9.6 KiB

Delivery Packages — Workflow Guide

How a finished shot becomes a dated folder ready to upload to the client.

This is the operational guide. The design rationale and normative rules live in RenderPipeline2.md §11.


The short version

  1. A shot renders, validates, gets a preview, and passes QC → it becomes eligible for delivery.
  2. On Pipeline → Deliveries, pick the project, see "Episode 101 — Ready: 18 shots", click Build delivery.
  3. A render worker builds the folder on the delivery volume, verifies every file, and writes a manifest and CSV report.
  4. A human uploads that folder to the client, then clicks Mark delivered.

Nothing is ever sent to the client automatically. Step 4 is deliberately manual.


What makes a shot eligible

An Export reaches READY_FOR_DELIVERY only by passing QC — from the AE panel or the web. That is the single gate; there is no way to package an export that has not been QC'd.

A render that failed automated validation can still get there: Override validation on the Export detail page (admin/producer/supervisor, reason required) resumes the pipeline at the preview stage, so the shot still goes through QC like any other. The failed checks stay on record and keep appearing as warnings in the QC queue, so whoever QCs it knows what was waved through.

Two rules worth knowing:

  • Newest version per shot only. If a shot has two exports sitting ready, the older one is history and is not offered. Building refuses outright if you somehow select two versions of the same shot.
  • Version locking. A package references the exact Export rows it was built from and copies the shot code and version string at creation time. Re-render a shot afterwards and the package is untouched — delivery history is immutable by construction.

Building a package

Pipeline → Deliveries, choose a project. Each episode with ready shots shows its count and the shots themselves, with frame counts and sizes.

Build delivery creates the package (QUEUED) and a DELIVERY_BUILD job on the same queue the renders use. Requires admin, producer or supervisor.

Delivery jobs run at priority 10, which is inside the urgent threshold — so they claim immediately even during work hours, when a render would wait for the evening window. Packaging is file work with no After Effects involvement, so it does not disturb an artist at the machine.

Watch the package status on the same page:

Status Meaning
QUEUED Waiting for a worker to pick it up
BUILDING A worker is creating folders and linking files
READY Built and verified — upload it
DELIVERED You confirmed the client has it
FAILED Something did not verify — see the error and the per-shot list
CANCELLED Abandoned before delivery

What lands on disk

Default layout (the studio's confirmed structure):

{deliveryRoot}/{episode}/{YYMMDD}_Delivery/
├── {SHOT_CODE}/
│   ├── {shot}_cmp_TT_{ver}.[#####].exr     ← the EXR sequence
│   └── {shot}_cmp_TT_{ver}.mov             ← slate/burn-in delivery MOV
├── manifest.json                            ← every file, size, checksum, source export
└── {YYMMDD}_Delivery_report.csv             ← one row per shot (client-facing)

EXRs and the MOV sit side by side in each shot folder — no exr/ and mov/ subfolders, matching current practice.

The CSV carries: shot, version, frame range, frame count, file count, bytes, checksum, MOV filename and status. Confirm these columns against the production's delivery spec before the first real delivery.


Files are hard-linked, not copied, when the source and destination are on the same volume. A hard link is a second name for the same data — not a shortcut and not a copy — so an 18-shot EXR package builds in seconds and consumes essentially no extra disk.

Verified on the studio SAN (\\TRUENAS\vfx over SMB): hard links work there, and renders and deliveries live on the same share.

Is that safe? Deleting the delivery folder does not delete the render — the data survives until the last name referring to it is gone. And nothing in the pipeline ever rewrites an existing EXR: a new version is always a new directory.

Two escape hatches:

  • A delivery root on a different volume falls back to copying automatically.
  • "copyMode": "always-copy" in the project's delivery config forces real copies for a production that mandates physical separation.

The linkMode column on each item records what actually happened, so a package never claims to have linked when it copied.


Verification

Verification is proportional to what the builder actually did, which is what keeps a package fast:

Mode How it verifies Cost for a 25-shot episode
Hard-linked (default) The entry is the source data, so the validation-time digest already describes it. Checks each file exists, and that no source was modified after it passed QC. seconds
Copied Read back in full and hashed; the sequence digest must match the one recorded at validation. ~30 s per 2.8 GB shot

Re-reading a hard link cannot disagree with validation — it is the same bytes under a second name — and at ~100 MB/s over SMB it costs roughly half an hour per episode to prove something that cannot be false. A copy is genuinely different: it can be truncated or corrupted, so it is always read back.

Set "verifyMode": "always-hash" in the delivery config to read everything back regardless, for a production that requires it contractually.

An item fails, and the whole package fails with it, when:

Failure Meaning
Checksum mismatch (copied items) The delivered bytes differ from what passed QC
A source file modified after validation The render changed after it passed QC — a link would carry the new bytes faithfully
Frame count differs from the export record The render folder changed after QC — someone re-rendered in place
Source folder missing The render is not where the export says it is
No matching files The folder exists but holds nothing matching the output pattern

A package only reaches READY when every shot verified. One bad shot means FAILED, and no export is marked as packaged — there is no partial delivery.

A missing delivery MOV is the one non-fatal case: the EXRs are the deliverable, so it is logged and the package continues without it.

Without validation there is nothing to verify against. An export whose checksum was never recorded (validation disabled, or rendered before it shipped) still packages, but the digest is computed with nothing to compare it to. The ready list flags these as "N without a validation checksum".


Marking delivered

Once you have uploaded the folder, Mark delivered moves every packaged export to DELIVERED. Requires admin, producer or supervisor.

This is the end of the pipeline, and it is a human statement of fact — the app has no way to know the client received anything.


When something goes wrong

A package failed. Expand it: every shot shows its own status and error. Fix the cause (usually a re-render after QC), then build a new package. Packages are never deleted; a mistaken one is cancelled.

The worker died mid-build. The package is marked FAILED with "check the package folder before rebuilding". It is deliberately not retried automatically — a half-written folder on the delivery volume is something a human should look at first.

Cancelling. Only DRAFT, QUEUED and FAILED packages can be cancelled. A READY or DELIVERED package stays in the history permanently; supersede it by building a new one.


Configuration

Per project, in Project.deliveryConfig; studio-wide defaults in the delivery.defaultConfig SystemConfig key; built-in defaults otherwise. Merged key by key, so a project can override just deliveryRoot.

{
  "packageNameTemplate": "{yymmdd}_Delivery",
  "layout": {
    "packageDir": "{deliveryRoot}/{episode}/{packageName}",
    "shotDir":    "{packageDir}/{shotCode}",
    "reportFile": "{packageDir}/{packageName}_report.csv"
  },
  "deliveryRoot": "//SAN/DELIVERY",
  "includeMovs": true,
  "checksumAlgo": "xxh64",
  "reportFormats": ["csv", "json"],
  "copyMode": "auto",
  "verifyMode": "auto"
}

Tokens: {showId} {episode} {yymmdd} {yyyymmdd} {packageName} {packageDir} {deliveryRoot} {shotCode} {versionString} {packageNumber}.

The resolved config is snapshotted onto the package at build time, so a historic package always shows the rules it was built with, and a production with a different spec is a config change rather than a code change.

deliveryRoot must be reachable from the worker machine — check with VFXReviewWorker.exe --check if a build fails on paths.


API

# Method Path Purpose
E16 POST /api/deliveries Create a package and queue the build
E17 GET /api/deliveries List packages / history
E18 GET /api/deliveries/ready Ready exports grouped by episode
E19 POST /api/deliveries/{id}/delivered Mark delivered
— POST /api/deliveries/{id}/cancel Cancel a package
— POST /api/ext/render/jobs/{jobId}/delivery Worker reports a build (API key)

POST /api/deliveries takes { projectId, episode?, exportIds?, options? }. Omit exportIds to take every ready export in the episode; options accepts date, name, includeMovs and copyMode to override the config for one build.