diff --git a/RenderPipeline2.md b/RenderPipeline2.md new file mode 100644 index 0000000..9c97af4 --- /dev/null +++ b/RenderPipeline2.md @@ -0,0 +1,1380 @@ +# Render Pipeline 2 — Implementation Specification + +Status: Approved for implementation — **all design decisions resolved** (Section 18.1); Section 18.2 is a checklist of values to collect during implementation +Author: Lead Architect +Date: 2026-07-31 (rev 3 — final studio decisions folded in) +Source of truth for current state: `TECHNICAL_ARCHITECTURE_REPORT_CURRENT_STATE.md`, `EXT_API_REFERENCE.md` + +--- + +## 1. Purpose and Scope + +### 1.1 Goal + +Reduce the artist workflow to: + +``` +Artist finishes comp → clicks "Queue Export" → everything else is automatic +``` + +The pipeline automates: version increment, rendering (aerender), output validation, preview/thumbnail/metadata generation, QC queueing, and Netflix-style delivery package construction. The only remaining manual steps are: building the comp, visually QC-ing the render, and uploading the finished delivery folder to Content Hub (no Content Hub API exists today). + +### 1.2 What this document is + +A complete implementation blueprint. Future AI coding agents implement it phase-by-phase (Section 17) without needing further architectural decisions — every design question has been answered by the studio and recorded in Section 18.1; Section 18.2 lists the concrete values and verifications to collect during implementation. + +### 1.3 What this document is not + +- Not a redesign of VFXReview. Every existing model, route, and workflow stays intact. +- Not production code. Schema and payloads below are normative in shape, not character-exact. + +### 1.4 New capabilities introduced + +| Capability | Today | After | +|---|---|---| +| Render execution | Artist's AE render queue, manual | `RenderWorker` Windows service driving `aerender.exe` | +| Versioning | Artist clicks "Increment Version" in panel | Automatic on Queue Export | +| Output validation | None | Automatic frame/metadata/EXR validation | +| Preview MOV / thumbnail | AE render queue templates, manual | Automatic (worker, ffmpeg) | +| QC | Ad hoc | Explicit QC queue + Pass/Fail from AE panel | +| Delivery | PowerShell copy scripts from AE panel | `Build Netflix Delivery` button, DB-driven package builder | +| Delivery history | None | Permanent `DeliveryPackage` records | + +--- + +## 2. Guiding Principles and Reuse Map + +### 2.1 Principles + +1. **Extend, never replace.** New models reference existing ones by foreign key; no existing column is removed or repurposed. +2. **The database is the queue.** No Redis/RabbitMQ/BullMQ. PostgreSQL row-level atomic claims are sufficient at this scale (tens of jobs/day, a handful of workers). This is the single most important simplicity decision in this spec. +3. **The worker owns the filesystem; the server owns the truth.** Workers touch EXRs on the shared render root. The web app never reads render output directly — it trusts worker reports persisted in the DB. The app server (Docker) is never required to mount the SAN. +4. **Workers talk only HTTP.** Workers never get a `DATABASE_URL`. All state flows through `/api/ext/*` using the existing API-key auth. This is what makes "add more render machines later without changing the API" free. +5. **Previews become Versions.** The generated MOV preview is registered as a normal `Version` row on the shot's comp task. The entire existing review stack (player, comments, annotations, client portal) works on pipeline renders with zero changes. +6. **Idempotent, resumable, observable.** Every worker→server report is safe to repeat; every job can be retried; every state change is timestamped and attributable. + +### 2.2 Reuse map + +| Existing asset | Reused for | +|---|---| +| `/api/ext/*` + `API_SECRET_KEY` bearer auth | All worker and AE-panel endpoints (Section 15 adds per-machine keys as an option) | +| `Shot.shotVersion`, `Shot.exrOutput` | Remain the canonical "current version" mirror; pipeline updates them on Queue Export exactly as the panel's PATCH does today | +| `Version` model + upload/review flows | Preview MOVs are created as Versions; internal/client review unchanged | +| `lib/storage.ts` abstraction | Upload of preview MOV, thumbnail, metadata JSON, delivery reports | +| `/api/files/{...key}` | Serving previews/thumbnails as today | +| `lib/frame-utils.ts`, `lib/edl-utils.ts` | Frame counting, timecode math in validation and manifest building | +| `SystemConfig` table | Pipeline-wide settings (poll intervals, lease durations, tool paths defaults) | +| NextAuth sessions | All new internal UI pages and internal APIs | +| Existing ext API conventions (JSON errors, Zod validation, pagination shape) | All new endpoints | +| AE panel shot lookup / episode / shot list endpoints | Unchanged; panel keeps using them | + +### 2.3 Explicit non-goals + +- Uploading EXR sequences to object storage (they stay on the shared filesystem; studio decision 18.1-Q2). +- Deadline/Tractor-style generic render farm. This is an AE-only, purpose-built queue. +- Automatic Content Hub upload (no API exists; manual step stays). + +--- + +## 3. System Overview + +### 3.1 Component diagram + +```mermaid +flowchart LR + subgraph Workstation["Artist Workstation"] + AE["After Effects + VFXReview Panel (CEP/ExtendScript)"] + end + + subgraph Server["VFXReview (Next.js + Prisma + PostgreSQL)"] + EXT["/api/ext/* (API-key auth)"] + INT["/api/* internal (NextAuth)"] + DB[(PostgreSQL)] + ST["Storage abstraction lib/storage.ts"] + end + + subgraph RenderNode["Render Machine(s) — Windows"] + W["VFXReview RenderWorker (Windows Service)"] + AER["aerender.exe"] + FF["ffmpeg / oiiotool"] + end + + SAN[("Shared filesystem (projects, AEPs, render roots, delivery roots)")] + OBJ[("Object storage (Hetzner / S3-compat)")] + UI["Web UI (Render Queue, QC, Deliveries, Machines)"] + + AE -- "Queue Export, QC Pass/Fail" --> EXT + W -- "claim / progress / validate / complete / heartbeat" --> EXT + EXT --> DB + INT --> DB + UI --> INT + W --> AER + W --> FF + AER --> SAN + FF --> SAN + AE --> SAN + W -- "preview MOV / thumb / metadata upload" --> EXT + EXT --> ST --> OBJ +``` + +### 3.2 Happy-path sequence + +```mermaid +sequenceDiagram + autonumber + participant A as AE Panel + participant S as VFXReview API + participant W as RenderWorker + participant F as Shared FS + + A->>S: POST /api/ext/exports (manifest) + S->>S: increment Shot.shotVersion, create Export(QUEUED) + RenderJob(attempt 1) + S-->>A: exportId, versionString v004 + loop poll (default 10 s) + W->>S: POST /api/ext/render/jobs/claim + end + S-->>W: job + manifest (Export → RENDERING) + W->>F: launch aerender.exe, parse stdout + W->>S: PATCH progress (frame N of M, ETA) + W->>S: POST complete-render (Export → VALIDATING) + W->>F: scan EXR sequence, run checks + W->>S: POST validation results (pass) (Export → GENERATING_PREVIEW) + W->>F: ffmpeg MOV + thumbnail + metadata JSON + W->>S: presign + upload preview artifacts + W->>S: POST finalize (Export → READY_FOR_QC, Version row created) + A->>S: GET /api/ext/qc/queue → artist QCs in AE + A->>S: POST /api/ext/exports/{id}/qc {result: PASS} + S->>S: Export → READY_FOR_DELIVERY + Note over S: Coordinator clicks "Build Netflix Delivery" in web UI + S->>S: DeliveryPackage(QUEUED) + DeliveryItems (version-locked) + W->>S: claim build job + W->>F: hard-link/copy EXRs + MOVs, write report + manifest + W->>S: POST package complete (PACKAGED) + Note over S: Human uploads folder to Content Hub, clicks "Mark Delivered" + S->>S: Package DELIVERED, items' Exports → DELIVERED +``` + +--- + +## 4. Export Lifecycle State Machine + +The state machine lives on **`Export`** (not on `Shot`, not on `RenderJob`). `Shot.status` remains derived exactly as today; `RenderJob` has its own small execution status. "Working" is the implicit pre-state before any `Export` row exists. + +### 4.1 State diagram + +```mermaid +stateDiagram-v2 + [*] --> QUEUED : Queue Export (AE panel) + QUEUED --> RENDERING : worker claims job + QUEUED --> CANCELLED : user cancels + RENDERING --> RENDER_FAILED : aerender error / crash / lease expiry after max retries + RENDERING --> QUEUED : lease expired, retries remain (new attempt) + RENDERING --> VALIDATING : aerender exit 0 + RENDERING --> CANCELLED : user cancels (worker kills process) + RENDER_FAILED --> QUEUED : Retry (new RenderJob attempt) + VALIDATING --> VALIDATION_FAILED : any check fails + VALIDATING --> GENERATING_PREVIEW : all checks pass + VALIDATION_FAILED --> QUEUED : Retry (re-render) + GENERATING_PREVIEW --> PREVIEW_FAILED : ffmpeg/upload error + PREVIEW_FAILED --> GENERATING_PREVIEW : Retry preview only + GENERATING_PREVIEW --> READY_FOR_QC : preview Version registered + READY_FOR_QC --> QC_FAILED : QC Fail + READY_FOR_QC --> READY_FOR_DELIVERY : QC Pass + QC_FAILED --> [*] : artist fixes comp, queues new Export (this one → SUPERSEDED) + READY_FOR_DELIVERY --> PACKAGED : included in a built DeliveryPackage + PACKAGED --> DELIVERED : package marked delivered + READY_FOR_DELIVERY --> SUPERSEDED : newer Export passes QC + READY_FOR_QC --> SUPERSEDED : newer Export queued for same shot + QUEUED --> SUPERSEDED : newer Export queued for same shot + DELIVERED --> ARCHIVED : archival policy / manual + CANCELLED --> [*] + SUPERSEDED --> [*] + ARCHIVED --> [*] +``` + +### 4.2 Transition table (normative) + +| # | From | To | Trigger | Actor | Side effects | +|---|---|---|---|---|---| +| T1 | *(Working)* | QUEUED | `POST /api/ext/exports` | AE panel | `Shot.shotVersion` incremented; `Shot.exrOutput` updated; `RenderJob` attempt 1 created; any older non-terminal Export for the shot → SUPERSEDED | +| T2 | QUEUED | RENDERING | Atomic claim | Worker | `RenderJob.machineId/claimedAt/leaseExpiresAt` set | +| T3 | RENDERING | VALIDATING | `complete-render` report, exit 0 | Worker | render duration recorded | +| T4 | RENDERING | RENDER_FAILED | `fail` report or lease expiry with `attempt >= maxAttempts` | Worker / reaper | stderr tail + exit code stored on RenderJob | +| T5 | RENDERING | QUEUED | Lease expiry, `attempt < maxAttempts` | Server reaper | new `RenderJob` row, attempt+1 | +| T6 | RENDER_FAILED | QUEUED | Retry button / `retry` endpoint | User (web/panel) | new `RenderJob` attempt | +| T7 | VALIDATING | VALIDATION_FAILED | Validation report contains a FAIL | Worker | `ValidationResult` rows persisted; pipeline stops | +| T8 | VALIDATING | GENERATING_PREVIEW | All checks PASS/WARN | Worker | `ValidationResult` rows persisted | +| T9 | VALIDATION_FAILED | QUEUED | Retry (full re-render) | User | new attempt; old validation rows kept for history | +| T10 | GENERATING_PREVIEW | PREVIEW_FAILED | ffmpeg/upload failure | Worker | error stored | +| T11 | PREVIEW_FAILED | GENERATING_PREVIEW | Retry preview | User | validation is NOT re-run | +| T12 | GENERATING_PREVIEW | READY_FOR_QC | `finalize` report | Worker | `Version` row created (isLatest, **never client-shared**), thumbnail set, metadata JSON key stored; **no task/shot status change** — QC typically runs on already client-approved shots and must not disturb review state (Section 10) | +| T13 | READY_FOR_QC | QC_FAILED | `POST .../qc {FAIL}` | Artist (AE) or user (web) | `QCReview` row; task → CHANGES (drives existing Shot REVISIONS derivation) | +| T14 | READY_FOR_QC | READY_FOR_DELIVERY | `POST .../qc {PASS}` | Artist / user | `QCReview` row | +| T15 | READY_FOR_DELIVERY | PACKAGED | Delivery build completes | Worker | `DeliveryItem` links Export to package | +| T16 | PACKAGED | DELIVERED | "Mark Delivered" | User (web) | `DeliveryPackage.deliveredAt/By` set | +| T17 | QUEUED / READY_FOR_QC / READY_FOR_DELIVERY / QC_FAILED | SUPERSEDED | Newer Export queued (T1) or newer Export reaches READY_FOR_DELIVERY | Server | automatic; PACKAGED/DELIVERED Exports are never auto-superseded (delivery history is immutable) | +| T18 | QUEUED / RENDERING | CANCELLED | Cancel endpoint | User | worker told to kill via claim-refresh response | +| T19 | DELIVERED | ARCHIVED | Manual / retention job | Admin | terminal | + +Rules: + +- **Terminal states:** CANCELLED, SUPERSEDED, ARCHIVED. DELIVERED is terminal except T19. +- **Illegal transitions are rejected server-side** with `409 { "error": "Invalid transition RENDERING → READY_FOR_QC" }`. The server is the sole authority on state; workers *request* transitions. +- Every transition writes `Export.statusChangedAt` and appends to `ExportEvent` (Section 5.9) for auditability. + +--- + +## 5. Database Design + +All additions are new models plus **two nullable columns** on existing models. No existing column is modified or removed. Prisma-style definitions below are normative in shape; implementers map to house style (cuid ids, `createdAt`/`updatedAt` timestamps as in the existing schema). + +### 5.1 New enums + +```prisma +enum ExportStatus { + QUEUED + RENDERING + RENDER_FAILED + VALIDATING + VALIDATION_FAILED + GENERATING_PREVIEW + PREVIEW_FAILED + READY_FOR_QC + QC_FAILED + READY_FOR_DELIVERY + PACKAGED + DELIVERED + SUPERSEDED + ARCHIVED + CANCELLED +} + +enum RenderJobStatus { QUEUED CLAIMED RUNNING COMPLETED FAILED CANCELLED EXPIRED } +enum RenderJobType { AE_RENDER PREVIEW_ONLY DELIVERY_BUILD } +enum ValidationStatus { PASS FAIL WARN SKIPPED } +enum QCResult { PASS FAIL } +enum MachineStatus { ONLINE OFFLINE DISABLED } +enum DeliveryStatus { DRAFT QUEUED BUILDING READY DELIVERED FAILED CANCELLED } +``` + +### 5.2 Export + +The central pipeline entity: one row per "Queue Export" click. Carries the Section 4 state machine. + +```prisma +model Export { + id String @id @default(cuid()) + shotId String + shot Shot @relation(fields: [shotId], references: [id]) + projectId String + taskId String? // comp task the preview Version attaches to + versionId String? // preview Version created at READY_FOR_QC + versionNumber Int // 4 + versionString String // "v004" + status ExportStatus @default(QUEUED) + statusChangedAt DateTime @default(now()) + + // Manifest (denormalised for querying; full manifest JSON on RenderJob) + aepPath String + compName String + rendererType String // "aerender" + outputDir String // render root for this export + outputPattern String // "UNG_106_010_020_cmp_TT_v004.[####].exr" + frameStart Int + frameEnd Int + fps Float + width Int + height Int + colorspace String? // expected, e.g. "ACES - ACEScg" + + // Artifacts + deliveryMovPath String? // slate/burn-in delivery MOV on the SAN (Section 9); packaged in deliveries + previewMovKey String? // web H.264 transcode in object storage (also Version.fileUrl) + thumbnailKey String? + metadataKey String? // metadata JSON in object storage + exrFileCount Int? + exrTotalBytes BigInt? + checksum String? // sequence-level digest (xxHash of per-file hashes) + + submittedById String? // resolved User, else null + submittedByName String? // free text from panel config as fallback + supersededById String? // newer Export that replaced this one + + renderJobs RenderJob[] + validations ValidationResult[] + qcReviews QCReview[] + deliveryItems DeliveryItem[] + events ExportEvent[] + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([shotId, versionNumber]) + @@index([status]) + @@index([projectId, status]) +} +``` + +Design note — **why Export is separate from Version**: `Version` is the review-media entity (an MOV with comments/annotations/approvals). An Export is a *render lifecycle* that eventually *produces* a Version (T12). Overloading Version with render state would break the existing review flows this spec promises not to touch. `Export.versionId` links them 1:1 once the preview exists. + +Design note — **why Export ↔ RenderJob is 1:N**: each retry is a fresh `RenderJob` row (attempt N). This keeps a complete execution history (which machine, which logs, how long) without mutating past attempts, and makes "Render History" (Section 13) a trivial query. + +### 5.3 RenderJob + +One execution attempt of work by a worker. Also used (with `type: DELIVERY_BUILD`) for delivery package builds, so there is exactly one queue/claim/lease/heartbeat mechanism in the whole system. + +```prisma +model RenderJob { + id String @id @default(cuid()) + type RenderJobType @default(AE_RENDER) + exportId String? // set for AE_RENDER / PREVIEW_ONLY + export Export? @relation(fields: [exportId], references: [id]) + deliveryId String? // set for DELIVERY_BUILD + attempt Int @default(1) + maxAttempts Int @default(3) + status RenderJobStatus @default(QUEUED) + priority Int @default(50) // lower = sooner + manifest Json // full Render Manifest snapshot (Section 6.2) + + machineId String? + machine Machine? @relation(fields: [machineId], references: [id]) + claimedAt DateTime? + leaseExpiresAt DateTime? // claim + leaseSeconds; renewed by progress reports + startedAt DateTime? + finishedAt DateTime? + + progress Float @default(0) // 0..1 + currentFrame Int? + totalFrames Int? + etaSeconds Int? + exitCode Int? + errorMessage String? + logTail String? // last ~200 lines of aerender output + logFileKey String? // full log uploaded to object storage on finish/fail + + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@index([status, priority, createdAt]) + @@index([machineId, status]) +} +``` + +### 5.4 ValidationResult + +One row per check per validation run, so the UI can show exactly what failed and history is preserved across retries. + +```prisma +model ValidationResult { + id String @id @default(cuid()) + exportId String + export Export @relation(fields: [exportId], references: [id]) + renderJobId String // which attempt produced this run + checkName String // "frame_count", "missing_frames", "resolution", ... + status ValidationStatus + expected String? // "120" + actual String? // "119" + message String? // "Frame 1057 missing" + details Json? // e.g. list of missing frame numbers + createdAt DateTime @default(now()) + + @@index([exportId, renderJobId]) +} +``` + +### 5.5 QCReview + +```prisma +model QCReview { + id String @id @default(cuid()) + exportId String + export Export @relation(fields: [exportId], references: [id]) + result QCResult + notes String? + reviewedById String? // resolved User where possible + reviewerName String? // fallback for API-key reviews from the panel + source String // "AE_PANEL" | "WEB" + createdAt DateTime @default(now()) + + @@index([exportId]) +} +``` + +Multiple rows per Export are allowed (fail → re-QC after preview retry); the latest row is authoritative and drives T13/T14. + +### 5.6 Machine and WorkerHeartbeat + +```prisma +model Machine { + id String @id @default(cuid()) + name String @unique // "RENDER-01" + hostname String + status MachineStatus @default(OFFLINE) + enabled Boolean @default(true) // admin kill-switch: disabled machines cannot claim + lastSeenAt DateTime? + workerVersion String? + aeVersion String? // "2026 (24.x)" + capabilities Json? // { maxConcurrentJobs: 1, tools: {ffmpeg: "7.1", oiiotool: "2.5"} } + availability Json? // Section 7.10: { mode, windows, allowUrgentAnytime } — null = ALWAYS + renderNowUntil DateTime? // manual "Render Now" override (web toggle); claims allowed until this time + apiKeyHash String? // optional per-machine key (kept for the future; Section 15 — shared key for now) + renderJobs RenderJob[] + heartbeats WorkerHeartbeat[] + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt +} + +model WorkerHeartbeat { + id String @id @default(cuid()) + machineId String + machine Machine @relation(fields: [machineId], references: [id]) + createdAt DateTime @default(now()) + cpuPercent Float? + memPercent Float? + diskFreeGb Float? + currentJobId String? + + @@index([machineId, createdAt]) +} +``` + +Heartbeats also update `Machine.lastSeenAt/status` in place. A daily prune keeps 7 days of heartbeat rows (Section 7.8). `ONLINE` = heartbeat within 3× the heartbeat interval; the machine-monitor page derives this, and the reaper derives lease expiry independently of it. + +### 5.7 DeliveryPackage and DeliveryItem + +```prisma +model DeliveryPackage { + id String @id @default(cuid()) + projectId String + episode String // "101" + name String // "260805_Delivery" (per 18.1-Q6 convention) + packageNumber Int // sequential per project → "Package #15" + status DeliveryStatus @default(DRAFT) + rootPath String? // final path on delivery volume + configSnapshot Json // naming template + options at build time (immutable record) + manifest Json? // built manifest: files, sizes, checksums + reportKey String? // delivery report (object storage) — also written into the folder + errorMessage String? + createdById String + createdAt DateTime @default(now()) + builtAt DateTime? + deliveredAt DateTime? + deliveredById String? + items DeliveryItem[] + + @@unique([projectId, packageNumber]) + @@index([projectId, episode]) +} + +model DeliveryItem { + id String @id @default(cuid()) + packageId String + package DeliveryPackage @relation(fields: [packageId], references: [id]) + exportId String // version lock: the exact Export delivered + export Export @relation(fields: [exportId], references: [id]) + shotId String + shotCode String // denormalised: survives shot renames + versionString String // denormalised: "v004" + exrPath String? // path inside package + movPath String? + fileCount Int? + totalBytes BigInt? + checksum String? + status String @default("PENDING") // PENDING | COPIED | VERIFIED | FAILED + + @@unique([packageId, exportId]) +} +``` + +**Version locking:** the package references `Export` rows, and items denormalise `shotCode`/`versionString` at creation. Later re-renders of a shot never mutate a package — delivery history is immutable by construction. + +### 5.8 Changes to existing models (additive only) + +| Model | Addition | Purpose | +|---|---|---| +| `Shot` | `exports Export[]` back-relation | navigation only; no column | +| `Project` | `deliveryConfig Json?` (nullable) | per-production delivery naming/layout templates (Section 11.2); falls back to `SystemConfig` defaults when null | +| `Version` | *(no change)* | preview Versions are ordinary rows; `Export.versionId` points at them | +| `SystemConfig` | new keys (no schema change) | `render.leaseSeconds`, `render.pollSeconds`, `render.maxAttempts`, `delivery.defaultTemplate`, etc. | + +### 5.9 ExportEvent (audit log) + +```prisma +model ExportEvent { + id String @id @default(cuid()) + exportId String + export Export @relation(fields: [exportId], references: [id]) + fromStatus String? + toStatus String + actorType String // "WORKER" | "USER" | "SYSTEM" + actorId String? // machineId or userId + note String? + createdAt DateTime @default(now()) + + @@index([exportId, createdAt]) +} +``` + +Cheap, append-only, powers the timeline strip on the Export detail page and answers "why did this fail at 3 am". + +### 5.10 Entity-relationship diagram (new + touching models) + +```mermaid +erDiagram + Project ||--o{ Shot : has + Shot ||--o{ Version : has + Shot ||--o{ Task : has + Shot ||--o{ Export : has + Export ||--o{ RenderJob : attempts + Export ||--o{ ValidationResult : checks + Export ||--o{ QCReview : reviews + Export ||--o{ ExportEvent : audit + Export |o--o| Version : "produces preview" + Machine ||--o{ RenderJob : executes + Machine ||--o{ WorkerHeartbeat : reports + Project ||--o{ DeliveryPackage : has + DeliveryPackage ||--o{ DeliveryItem : contains + DeliveryItem }o--|| Export : "locks version of" + RenderJob }o--o| DeliveryPackage : "builds (type=DELIVERY_BUILD)" +``` + +--- + +## 6. API Design + +### 6.1 Conventions (inherited from existing ext API) + +- Base URL, auth headers (`Authorization: Bearer ` / `X-Api-Key`), JSON error shape `{ "error": "...", "details": [...] }`, and status-code table are identical to `EXT_API_REFERENCE.md`. +- New external namespaces: `/api/ext/exports`, `/api/ext/render`, `/api/ext/qc`, `/api/ext/workers`, `/api/ext/deliveries`. +- New internal (NextAuth-session) namespaces mirror them: `/api/render/*`, `/api/deliveries/*`, `/api/machines/*` for the web UI. Internal routes are thin wrappers over the same service-layer functions (`lib/render-pipeline/*.ts`) — one implementation, two auth fronts, matching the existing actions/route split. +- All state-changing worker endpoints are **idempotent**: repeating a report for a job already past that state returns `200 { "alreadyApplied": true }`, not an error — except contradictory reports (e.g. `complete` after `fail`), which return `409`. + +### 6.2 The Render Manifest + +Uploaded by the AE panel on Queue Export; snapshotted verbatim into `RenderJob.manifest`; handed to the worker on claim. The panel builds it from the existing `/api/ext/shots/lookup` payload plus local AE knowledge. + +```json +{ + "manifestVersion": 1, + "projectCode": "UNG_S1", + "shotCode": "UNG_106_010_020", + "shotId": "clxxxxxxxxxxxxxx", + "aepPath": "//SAN/projects/UNG_S1/106/UNG_106_010_020/ae/UNG_106_010_020_comp.aep", + "compName": "UNG_106_010_020_cmp", + "rendererType": "aerender", + "aeVersionHint": "2026", + "outputDir": "//SAN/renders/UNG_S1/106/UNG_106_010_020/v004", + "outputPattern": "UNG_106_010_020_cmp_TT_v004.[####].exr", + "outputModuleTemplate": "VFXR_EXR_SEQ", + "renderSettingsTemplate": "VFXR_FULL", + "frameStart": 1001, + "frameEnd": 1120, + "fps": 24, + "width": 3840, + "height": 2160, + "expected": { + "colorspace": "ACES - ACEScg", + "bitDepth": "half", + "exrCompression": "PIZ", + "alpha": false, + "timecodeStart": "00:59:49:12" + }, + "preview": { + "template": "default", + "burnins": true + } +} +``` + +The server validates the manifest with Zod, cross-checks `shotId`/`shotCode`/frame range against the DB, and **the server (not the panel) decides the new version number** — race-free single source of truth. + +### 6.3 Endpoint summary + +| # | Method | Path | Auth | Caller | Purpose | +|---|---|---|---|---|---| +| E1 | POST | `/api/ext/exports` | API key | AE panel | Queue Export (creates Export + RenderJob) | +| E2 | GET | `/api/ext/exports/latest?shotCode=&projectCode=` | API key | AE panel | Latest export + status for panel display | +| E3 | GET | `/api/ext/exports/{exportId}` | API key | panel/worker | Export detail incl. validations, QC | +| E4 | POST | `/api/ext/exports/{exportId}/qc` | API key | AE panel | QC Pass / QC Fail | +| E5 | GET | `/api/ext/qc/queue?projectCode=&episode=` | API key | AE panel | Exports in READY_FOR_QC | +| E6 | POST | `/api/ext/workers/register` | API key | worker | Register/upsert machine | +| E7 | POST | `/api/ext/workers/{machineId}/heartbeat` | API key | worker | Heartbeat + cancel-signal channel | +| E8 | POST | `/api/ext/render/jobs/claim` | API key | worker | Atomically claim next job | +| E9 | PATCH | `/api/ext/render/jobs/{jobId}/progress` | API key | worker | Progress/ETA; renews lease | +| E10 | POST | `/api/ext/render/jobs/{jobId}/fail` | API key | worker | Report failure | +| E11 | POST | `/api/ext/render/jobs/{jobId}/complete-render` | API key | worker | aerender finished OK → VALIDATING | +| E12 | POST | `/api/ext/render/jobs/{jobId}/validation` | API key | worker | Report all validation results | +| E13 | POST | `/api/ext/render/jobs/{jobId}/finalize` | API key | worker | Preview artifacts done → READY_FOR_QC | +| E14 | POST | `/api/ext/exports/{exportId}/retry` | API key | panel | Retry failed export | +| E15 | POST | `/api/render/jobs/{jobId}/retry` · `/cancel` | session | web UI | Retry / cancel | +| E16 | POST | `/api/deliveries` | session | web UI | Create package (DRAFT→QUEUED) | +| E17 | GET | `/api/deliveries?projectId=&episode=` | session | web UI | List packages / history | +| E18 | GET | `/api/deliveries/ready?projectId=&episode=` | session | web UI | READY_FOR_DELIVERY exports grouped by episode | +| E19 | POST | `/api/deliveries/{id}/delivered` | session | web UI | Mark Delivered | +| E20 | POST | `/api/ext/render/jobs/{jobId}/artifact-presign` | API key | worker | Presigned upload for preview/thumb/metadata/log | +| E21 | GET | `/api/render/queue` · `/api/render/exports` · `/api/machines` | session | web UI | Queue/monitoring pages (standard pagination) | + +### 6.4 Request/response examples (normative) + +**E1 — Queue Export** + +```http +POST /api/ext/exports +Authorization: Bearer +Content-Type: application/json + +{ "manifest": { ...Section 6.2..., "outputDir": "auto", "outputPattern": "auto" }, + "submittedByEmail": "jane@studio.com" } +``` + +`outputDir`/`outputPattern` may be `"auto"`: the server generates them from the shot's `exrOutput` convention and the *new* version number, and returns them; the panel does not guess paths for a version it doesn't know yet. + +```json +201 +{ + "export": { + "id": "exp_01", + "shotCode": "UNG_106_010_020", + "versionString": "v004", + "status": "QUEUED", + "outputDir": "//SAN/renders/UNG_S1/106/UNG_106_010_020/v004", + "outputPattern": "UNG_106_010_020_cmp_TT_v004.[####].exr" + }, + "renderJob": { "id": "rj_01", "attempt": 1, "priority": 50 }, + "shot": { "id": "clxxx", "shotVersion": "v004", "exrOutput": "UNG_106_010_020_cmp_TT_v004" }, + "superseded": ["exp_00"] +} +``` + +Errors: `409` if an active (QUEUED/RENDERING/VALIDATING/GENERATING_PREVIEW) Export exists for the shot and `force` is not set; `422` manifest validation failure. + +**E2 — Latest export for panel display** + +```http +GET /api/ext/exports/latest?shotCode=UNG_106_010_020&projectCode=UNG_S1 +``` + +```json +200 +{ + "export": { + "id": "exp_01", "versionString": "v004", "status": "READY_FOR_QC", + "statusChangedAt": "2026-08-01T12:04:11Z", + "renderJob": { "progress": 1, "machineName": "RENDER-01" }, + "validation": { "overall": "PASS", "failed": [] }, + "qc": { "latestResult": null }, + "previewUrl": "/api/files/videos/uuid-UNG_106_010_020_cmp_TT_v004.mov" + } +} +``` + +Returns `{ "export": null }` when the shot has never been exported. + +**E6 — Worker registration** (idempotent upsert on `name`) + +```http +POST /api/ext/workers/register +{ "name": "RENDER-01", "hostname": "render01.studio.local", + "workerVersion": "1.0.3", "aeVersion": "24.3", + "capabilities": { "maxConcurrentJobs": 1, "tools": { "ffmpeg": "7.1", "oiiotool": "2.5.4" } } } +``` + +```json +200 { "machine": { "id": "mac_01", "name": "RENDER-01", "enabled": true }, + "config": { "pollSeconds": 10, "heartbeatSeconds": 30, "leaseSeconds": 300 } } +``` + +Server-supplied `config` (from `SystemConfig`) means fleet tuning without touching worker installs. + +**E7 — Heartbeat** (also the cancel channel) + +```http +POST /api/ext/workers/mac_01/heartbeat +{ "cpuPercent": 82.5, "memPercent": 61.0, "diskFreeGb": 512.3, "currentJobId": "rj_01" } +``` + +```json +200 { "ok": true, "commands": [ { "type": "CANCEL_JOB", "jobId": "rj_01" } ] } +``` + +`commands` is normally `[]`. Piggybacking cancellation on the heartbeat avoids any server→worker connection (workers may be NAT-ed; no websockets needed). + +**E8 — Claim next job** (the critical atomic operation) + +```http +POST /api/ext/render/jobs/claim +{ "machineId": "mac_01", "types": ["AE_RENDER", "PREVIEW_ONLY", "DELIVERY_BUILD"] } +``` + +```json +200 +{ "job": { "id": "rj_01", "type": "AE_RENDER", "attempt": 1, "exportId": "exp_01", + "leaseExpiresAt": "2026-08-01T11:35:00Z", + "manifest": { ... } } } +``` + +`204 No Content` when the queue is empty. Server-side claim must be atomic under concurrent workers: + +```sql +UPDATE "RenderJob" SET status='CLAIMED', "machineId"=$1, + "claimedAt"=now(), "leaseExpiresAt"=now() + ($2 * interval '1 second') +WHERE id = ( + SELECT id FROM "RenderJob" + WHERE status='QUEUED' AND type = ANY($3) + ORDER BY priority ASC, "createdAt" ASC + FOR UPDATE SKIP LOCKED LIMIT 1 ) +RETURNING *; +``` + +(Implemented as `prisma.$queryRaw`/`$executeRaw`; `FOR UPDATE SKIP LOCKED` is the standard Postgres queue idiom and needs no extra infrastructure. Claiming also transitions the Export to RENDERING.) + +The claim additionally enforces **machine availability** (Section 7.10): outside a workstation's render window the server answers `204` even when jobs are queued — unless the machine has an active "Render Now" override (`renderNowUntil`) or the head-of-queue job is urgent priority (`priority <= 20` and the machine's `allowUrgentAnytime` is true). Centralising this in the claim endpoint means workers poll dumbly and all scheduling policy lives in the server/UI. + +**E9 — Progress** (renews lease; recommended every 10 s or every frame, whichever is coarser) + +```http +PATCH /api/ext/render/jobs/rj_01/progress +{ "machineId": "mac_01", "progress": 0.42, "currentFrame": 1051, + "totalFrames": 120, "etaSeconds": 96, "logTail": "PROGRESS: 0:00:02:03 (51): ..." } +``` + +```json +200 { "ok": true, "leaseExpiresAt": "2026-08-01T11:40:00Z", "cancelRequested": false } +``` + +**E10 — Fail** + +```http +POST /api/ext/render/jobs/rj_01/fail +{ "machineId": "mac_01", "stage": "RENDER", "exitCode": 9, + "errorMessage": "aerender ERROR: layer source file missing", + "logTail": "...", "retryable": true } +``` + +```json +200 { "job": { "status": "FAILED" }, "export": { "status": "RENDER_FAILED" }, + "autoRequeued": false } +``` + +If `retryable: true` and `attempt < maxAttempts`, the server auto-creates the next attempt and returns `"autoRequeued": true` with the new job id. `stage` is one of `RENDER | VALIDATION_RUNNER | PREVIEW | PACKAGE` (a crash *of the validation runner*, as opposed to a failed validation check, which uses E12). + +**E12 — Validation report** (one call carries the entire run) + +```http +POST /api/ext/render/jobs/rj_01/validation +{ "machineId": "mac_01", + "overall": "FAIL", + "sequence": { "fileCount": 119, "totalBytes": 5482196992, "checksum": "xxh64:9f2a..." }, + "checks": [ + { "checkName": "frame_count", "status": "FAIL", "expected": "120", "actual": "119" }, + { "checkName": "missing_frames", "status": "FAIL", "expected": "none", "actual": "1057", + "details": { "missing": [1057] } }, + { "checkName": "resolution", "status": "PASS", "expected": "3840x2160", "actual": "3840x2160" }, + { "checkName": "colourspace", "status": "PASS", "expected": "ACES - ACEScg", "actual": "ACES - ACEScg" }, + { "checkName": "bit_depth", "status": "PASS", "expected": "half", "actual": "half" } + ] } +``` + +```json +200 { "export": { "status": "VALIDATION_FAILED" } } +``` + +**E13 — Finalize** (after successful preview generation and artifact uploads) + +```http +POST /api/ext/render/jobs/rj_01/finalize +{ "machineId": "mac_01", + "artifacts": { "deliveryMovPath": "//SAN/renders/UNG_S1/106/UNG_106_010_020/v004/mov/UNG_106_010_020_cmp_TT_v004.mov", + "previewMovKey": "videos/uuid-UNG_..._v004.mov", + "thumbnailKey": "image/uuid-UNG_..._v004_thumb.jpg", + "metadataKey": "renders/uuid-UNG_..._v004_meta.json", + "logFileKey": "renders/uuid-rj_01.log" }, + "renderStats": { "renderSeconds": 812, "previewSeconds": 44 } } +``` + +```json +200 { "export": { "id": "exp_01", "status": "READY_FOR_QC" }, + "version": { "id": "ver_09", "versionNumber": 4, "isLatest": true } } +``` + +Server-side, finalize reuses the **existing version-creation service** (previous versions un-latested, new `Version` created with `fileUrl`/`thumbnailUrl`) but in *pipeline mode*: the Version is flagged not-client-visible and **no task or shot status is mutated**. Delivery QC runs on shots the client has usually already approved (Section 10) — a post-approval technical render must never resurface in the review flow or client portal. `finalize` also reports `deliveryMovPath` (the SAN path of the slate/burn-in delivery MOV) alongside the object-storage artifact keys. + +**E4 — QC** + +```http +POST /api/ext/exports/exp_01/qc +{ "result": "PASS", "notes": "Grain matched, edges clean", + "reviewerEmail": "jane@studio.com", "source": "AE_PANEL" } +``` + +```json +200 { "export": { "status": "READY_FOR_DELIVERY" }, + "qcReview": { "id": "qc_01", "result": "PASS", "createdAt": "..." } } +``` + +QC FAIL additionally sets the comp task to CHANGES (existing enum), which drives the existing Shot REVISIONS derivation — reviewers see it in today's dashboards with no UI change. + +**E5 — QC queue** + +```http +GET /api/ext/qc/queue?projectCode=UNG_S1&episode=106 +``` + +```json +200 { "items": [ { "exportId": "exp_01", "shotCode": "UNG_106_010_020", + "versionString": "v004", "readySince": "2026-08-01T12:04:11Z", + "outputDir": "//SAN/renders/UNG_S1/106/UNG_106_010_020/v004", + "previewUrl": "/api/files/videos/uuid-....mov", + "thumbnailUrl": "/api/files/image/uuid-...jpg" } ], + "total": 1 } +``` + +**E16 — Create delivery package** + +```http +POST /api/deliveries (NextAuth session) +{ "projectId": "cmp6l5...", "episode": "101", + "exportIds": ["exp_01", "exp_02"], // preselected from E18; server re-verifies + "options": { "includeMovs": true, "date": "2026-08-05" } } +``` + +```json +201 { "package": { "id": "dp_15", "packageNumber": 15, "name": "260805_Delivery", + "episode": "101", "rootPath": "//SAN/DELIVERY/101/260805_Delivery", + "status": "QUEUED", "itemCount": 18 }, + "buildJob": { "id": "rj_77", "type": "DELIVERY_BUILD" } } +``` + +Server verifies every export is READY_FOR_DELIVERY (else `422` listing offenders), snapshots `Project.deliveryConfig` into `configSnapshot`, creates items, and enqueues a `DELIVERY_BUILD` RenderJob that any worker can claim via E8. + +**E19 — Mark Delivered** + +```http +POST /api/deliveries/dp_15/delivered +{ "note": "Uploaded to Content Hub 2026-08-05 16:20 by Chris" } +``` + +```json +200 { "package": { "status": "DELIVERED", "deliveredAt": "...", "deliveredBy": "Chris" }, + "exportsUpdated": 18 } +``` + +--- + +## 7. Render Worker + +### 7.1 Technology choice + +**Recommendation: .NET 8 Windows Service (worker-service template).** Trade-offs considered: + +| Option | Pros | Cons | +|---|---|---| +| **.NET 8 Windows Service (recommended)** | First-class Windows Service lifecycle, single self-contained exe, robust process management (`System.Diagnostics.Process`), easy MSI/`sc.exe` install, no runtime to install | Different language than the Next.js codebase | +| Node.js + node-windows / NSSM | Same language as server; could share TS types | Fragile as a service; process supervision and single-file distribution weaker on Windows | +| Python + pywin32 | Studio pipeline familiarity | Packaging/service story weakest; dependency management on render nodes painful | + +The worker shares no code with the server anyway (it only speaks HTTP), so language uniformity buys little; operational robustness on Windows render nodes buys a lot. The API contract in Section 6 is the interface, which is why more machines can be added without any API change. + +### 7.2 Internal architecture + +```mermaid +flowchart TB + subgraph Worker["RenderWorker service"] + MAIN["Main loop"] --> REG["Registration (E6, startup)"] + MAIN --> HB["Heartbeat timer (E7, 30 s)"] + MAIN --> POLL["Claim poller (E8, 10 s, only when idle)"] + POLL --> EXEC["Job executor (one job at a time)"] + EXEC --> P1["Stage 1: aerender runner"] + EXEC --> P2["Stage 2: validation engine (Section 8)"] + EXEC --> P3["Stage 3: preview generator (Section 9)"] + EXEC --> P4["Alt: delivery builder (Section 11)"] + P1 --> REP["Reporter (E9-E13, retry with backoff, offline spool)"] + P2 --> REP + P3 --> REP + P4 --> REP + EXEC --> LOG["Rolling file logs %ProgramData%\\VFXReviewWorker\\logs"] + end +``` + +One job at a time per machine by default (`capabilities.maxConcurrentJobs: 1`) — aerender saturates a box; concurrency is a config knob, not an architecture change. + +### 7.3 aerender execution + +``` +aerender.exe -project "" -comp "" + -s -e + -RStemplate "" -OMtemplate "" + -output "\" + -mp -continueOnMissingFootage false +``` + +- stdout is parsed line-by-line: `PROGRESS: ... (N)` lines yield `currentFrame`; ETA = rolling average seconds/frame × frames remaining. +- stderr and non-progress stdout accumulate into the rolling `logTail` (last 200 lines) and full log file. +- Failure detection: non-zero exit code, `aerender ERROR` lines, or no progress line for `stallTimeoutSeconds` (default 600 — some frames legitimately take minutes) → kill process tree, report E10. +- `outputDir` is created by the worker before launch; a pre-existing non-empty dir for the same attempt is cleared (it can only contain output from a previous crashed attempt of this same immutable version — new versions always get new dirs). + +### 7.4 Job locking and lease renewal + +- Claiming (E8) is atomic server-side (`FOR UPDATE SKIP LOCKED`); a job can never be claimed twice. +- Every progress report renews the lease (`leaseSeconds`, default 300). +- A server-side **reaper** (see 7.7) requeues or fails jobs whose lease expired — this covers worker power loss, BSOD, and network partition with no worker cooperation. +- Workers include `machineId` in every job-scoped call; the server rejects reports from a machine that doesn't hold the job's lease (`409`) — a resurrected zombie worker cannot corrupt a reassigned job. + +### 7.5 Crash recovery (worker side) + +On service start: register (E6), then reconcile: if a local state file (`current-job.json`, written on claim, deleted on finish) names a job, query E3; if the server reassigned or failed it, discard local state and clean the orphaned output dir. Never resume a partial aerender — re-render from frame 1 of the range (attempt integrity beats partial-resume complexity; typical shots are ≤ 200 frames). + +### 7.6 Retries + +| Failure | Retry behaviour | +|---|---| +| aerender non-zero exit, `retryable: true` | Server auto-requeues up to `maxAttempts` (3); attempt N+1 is a fresh RenderJob | +| Deterministic errors (missing footage, missing comp) | Worker sets `retryable: false` → RENDER_FAILED immediately; humans fix the comp | +| Validation FAIL | Never auto-retried — re-rendering identical inputs reproduces the failure. Human retries (T9) after fixing cause | +| Preview failure | `PREVIEW_ONLY` retry re-runs ffmpeg only (validated EXRs are fine) | +| Report HTTP failures | Exponential backoff 1 s→60 s; reports spool to disk and replay in order — rendering continues during server downtime | + +### 7.7 Server-side reaper + +A small periodic task (Next.js route invoked by the existing container's cron-less pattern — recommend a `setInterval` in `instrumentation.ts` (Next.js server startup hook), 60 s): + +1. `RenderJob` CLAIMED/RUNNING with `leaseExpiresAt < now()` → mark EXPIRED; if attempts remain → new QUEUED attempt (T5), else Export → RENDER_FAILED (T4). +2. `Machine.lastSeenAt` older than 3× heartbeat → status OFFLINE. +3. Prune `WorkerHeartbeat` older than 7 days. + +This is the only "background job" the web app gains, and it is a single idempotent SQL sweep. (Mechanism confirmed as studio decision 18.1-Q8.) + +### 7.8 Configuration + +`C:\ProgramData\VFXReviewWorker\config.json` (machine-local, minimal — everything tunable lives in server `SystemConfig` and arrives via E6): + +```json +{ + "serverUrl": "https://review.twotalesvfx.com", + "apiKey": "", + "machineName": "RENDER-01", + "aerenderPath": "C:\\Program Files\\Adobe\\Adobe After Effects 2026\\Support Files\\aerender.exe", + "ffmpegPath": "C:\\pipeline\\bin\\ffmpeg.exe", + "oiiotoolPath": "C:\\pipeline\\bin\\oiiotool.exe", + "pathMappings": [ { "from": "//SAN/", "to": "S:/" } ] +} +``` + +`pathMappings` translates manifest UNC paths to local drive mappings — manifests store canonical UNC paths; each machine maps them locally. + +### 7.9 Logging + +- Rolling local logs (14 days) under `%ProgramData%\VFXReviewWorker\logs`. +- Per-job full aerender log uploaded via E20 on finish/fail (`RenderJob.logFileKey`) so the web UI can show complete logs without filesystem access. + +### 7.10 Workstation deployment, render windows and "Render Now" + +**Studio decision (18.1):** the worker runs on the two artist workstations — no dedicated render nodes initially. That makes availability scheduling a first-class feature, not an afterthought: + +| Mode (`Machine.availability.mode`) | Behaviour | +|---|---| +| `ALWAYS` | Machine claims whenever idle (default when `availability` is null; suits any future dedicated node) | +| `SCHEDULE` | Machine claims only inside configured windows, e.g. `{ "windows": [{ "days": ["mon","tue","wed","thu","fri"], "from": "19:00", "to": "08:00" }, { "days": ["sat","sun"], "from": "00:00", "to": "24:00" }] }` — the after-hours auto-render case | +| `MANUAL` | Machine never claims unless explicitly triggered | + +Overrides, in priority order: + +1. **Render Now** — a button on the Machine Monitoring page (and available to the artist locally via a small tray companion, optional) sets `renderNowUntil = now() + N hours`. The machine claims normally until it expires. This is the "urgent render during the day" path: the artist queues the export, then hits Render Now on their own machine. +2. **Urgent jobs** — jobs queued with urgent priority (`priority <= 20`, settable from the panel via a "queue as urgent" checkbox or from the Render Queue page) may be claimed even outside windows on machines with `allowUrgentAnytime: true`. +3. `Machine.enabled = false` beats everything (admin kill-switch, unchanged). + +Enforcement is entirely server-side in E8 (workers keep polling on their normal cadence; polling is a no-op HTTP call). Practical workstation considerations: + +- The service runs under a studio account with SAN access and works whether or not an artist is logged in. +- aerender competing with a working artist is prevented by the schedule, not by idle-detection heuristics — simple and predictable. If mid-day renders on the *other* artist's machine become routine, revisit with an idle-detection option then. +- Window times are per-machine config values collected at Phase 2 rollout (18.2-C5); accepted defaults: weekdays 19:00–08:00 + full weekends, Render Now = 4 h, urgent queueing for all users. + +--- + +## 8. Validation Engine + +Runs on the worker immediately after a successful render (Export = VALIDATING). Tools: **oiiotool/OpenImageIO** for EXR introspection (recommended over parsing with a hand-rolled EXR reader; ships as a static exe), plus plain directory scanning. + +### 8.1 Checks (normative list) + +| checkName | Method | FAIL condition | +|---|---|---| +| `file_count` | dir listing vs `frameEnd-frameStart+1` | count mismatch | +| `missing_frames` | parse frame numbers from filenames, diff against expected range | any gap | +| `duplicate_frames` | same parse | same frame number twice (e.g. differing padding) | +| `first_frame` / `last_frame` | min/max parsed frame | ≠ `frameStart` / `frameEnd` | +| `filename_consistency` | every file matches `outputPattern` regex (prefix, padding, extension) | any nonconforming file | +| `resolution` | oiiotool on first+middle+last frame | ≠ manifest `width`×`height` | +| `bit_depth` | EXR channel types | ≠ `expected.bitDepth` (half/float) | +| `exr_type` | header: scanline vs tiled, compression | ≠ `expected.exrCompression` (WARN if compression differs but readable; FAIL if not valid EXR) | +| `alpha_channel` | channel list contains A | presence ≠ `expected.alpha` | +| `colourspace` | EXR metadata (`chromaticities` / ACES container flag / colorspace attr written by AE OCIO) | ≠ `expected.colorspace` (working space is linear ACES2065-1 per 18.1-Q4); `colorPipeline` is required per project (18.1-Q12), so SKIPPED only on legacy projects predating pipeline enablement | +| `timecode_consistency` | EXR `timeCode` attr across sampled frames | non-monotonic, or first frame ≠ `expected.timecodeStart` when set | +| `zero_byte / corrupt` | every file size > 0; oiiotool header-read on sampled frames (first, last, every Nth) | any zero-byte or unreadable file | + +Full-sequence checks (count, gaps, names, sizes) run on every file; header checks sample first/middle/last + every 25th frame (fast, catches systemic issues; per-frame full decode is not worth minutes per shot — tunable via `SystemConfig: validation.sampleEvery`). + +### 8.2 Semantics + +- Any FAIL → whole run FAIL → Export = VALIDATION_FAILED, pipeline stops, exact reasons in `ValidationResult` rows (E12 example above shows a missing-frame report). +- WARN never stops the pipeline but is shown in QC and delivery UIs. +- Checksum: xxHash64 per file, sequence digest = xxHash64 of sorted per-file digests; stored on `Export.checksum` and reused by the delivery builder for copy verification (MD5 available for the client-facing manifest per 18.1-Q13). + +--- + +## 9. Preview Generation + +Runs on the worker after validation passes (Export = GENERATING_PREVIEW). + +**Studio decision (18.1):** previews are generated with the **existing AE slate/burn-in templates**, not ffmpeg drawtext. The studio already maintains: a slate + overlay template (1-frame slate at the head of the shot, overlays/burn-ins throughout), connector functions that build preview comps and pull slate/overlay data from the API, and the per-project colour chain to match reference: **linear ACES2065-1 → per-project log space → show LUT**. The pipeline reuses all of it headlessly. + +### 9.1 Stages + +1. **Build + render preview comp (AE, headless).** The worker launches `AfterFX.com -noui -r vfxr_build_preview.jsx`, passing a job-context JSON file (manifest + slate/overlay fields fetched from E3). The script is the connector's existing preview-comp code, packaged with the worker: it imports the *validated* EXR sequence, builds the preview comp from the studio template (head slate populated from API data, overlays/burn-ins throughout), applies the colour chain (linear → log → show LUT from `colorPipeline` config), queues it with the project's MOV output-module template, and renders the **delivery-grade preview MOV** to the SAN beside the EXRs (`Export.deliveryMovPath`). This is the "MOV LT Preview" that ships inside delivery packages (Section 11). +2. **Web transcode (ffmpeg).** The delivery MOV is transcoded to a web-playable H.264 (1920-wide, CRF 18) and a thumbnail JPEG (frame at 25% duration, 960-wide), uploaded via E20 presign. +3. **Metadata JSON.** Worker-composed (manifest + validation summary + render stats + checksums + tool/template versions), uploaded via E20. + +| Artifact | Produced by | Destination | +|---|---|---| +| Delivery MOV (slate + burn-ins, graded via show LUT, delivery codec from AE output template) | AE headless render | SAN → `Export.deliveryMovPath`; copied/linked into delivery packages | +| Web preview MOV (H.264) | ffmpeg transcode of the delivery MOV | object storage `videos/` → `Version.fileUrl` | +| Thumbnail | ffmpeg | `image/` → `Version.thumbnailUrl` + `Export.thumbnailKey` | +| Metadata JSON | worker | `renders/` → `Export.metadataKey` | + +Because the web preview is a transcode of the AE-generated MOV, review media and delivery media are guaranteed to show identical slates, burn-ins, and colour — one source of truth. + +### 9.2 Colour configuration + +`Project.deliveryConfig.colorPipeline` (per 18.1-Q4/Q12) — **required and unique per project**; enabling the pipeline on a project without it is a validation error, and values must never be carried over from another show: + +```json +{ "workingSpace": "ACES2065-1", + "logSpace": "", + "showLut": "//SAN/luts/UNG/UNG_show_v02.cube" } +``` + +The AE template consumes these via the existing connector transform-loading functions; validation's `colourspace` check expects `workingSpace` on the EXRs. + +### 9.3 Fallback engine + +`SystemConfig: preview.engine = "ae" | "ffmpeg"` (default `"ae"`). The ffmpeg path (no slate, drawtext burn-ins, LUT applied via `lut3d`) exists only as an emergency fallback if headless AE proves unreliable on a machine — it produces clearly-marked non-delivery-grade previews ("FALLBACK PREVIEW" burn-in) and never populates `deliveryMovPath`, so a fallback preview can never leak into a delivery package. + +Cost note: the AE preview pass roughly doubles machine time per export. This matches current studio practice (preview comps are rendered today) — it is now simply unattended. + +Finalize (E13) then creates the `Version` row server-side (Section 6.4) in pipeline mode — visible internally, never client-shared, no task/shot status changes. + +--- + +## 10. QC Workflow + +### 10.0 Purpose (studio decision, 18.1-Q11) + +This QC stage is **technical/final-delivery QC on approved shots** — the last human check before EXRs ship. The shot has usually already been client-approved through the existing review process; QC exists to catch what that process can't: mask slips, export glitches/artifacts, and technical faults (missing frames, incorrect metadata/timecodes, wrong format/resolution/colourspace). The automated validation (Section 8) covers the mechanical half; human QC covers the visual half. Consequently: + +- **QC PASS means "ready to upload" and nothing else.** It never shares anything to the client review platform, never changes approval state, never creates client-visible media. +- Pipeline-created preview Versions are internal-only, permanently (T12). + +### 10.1 Flow + +1. Export reaches READY_FOR_QC; it appears in the panel's QC queue (E5) and the web QC page. +2. Artist opens the lightweight QC AE project; selecting a queue entry makes the panel import the EXR sequence from `outputDir` into the QC comp (panel already has import-renders machinery) — frame-accurate, full-res QC in AE, not the H.264 preview. +3. Artist clicks **QC Pass** or **QC Fail** (optional notes) → E4. +4. PASS → READY_FOR_DELIVERY. FAIL → QC_FAILED + comp task → CHANGES; the artist fixes and Queue Export creates the next version (old Export → SUPERSEDED). + +### 10.2 Rules + +- QC is per-Export. A new Export always requires new QC. +- Web users can also QC from the Export detail page (same service function, session auth) — QC is not locked to AE. +- Self-QC is allowed (single-artist reality); the `QCReview` row records who, so a policy can be layered later without schema change. +- **No client-review side effects on PASS** (Section 10.0). QC FAIL still sets the comp task to CHANGES — the shot genuinely needs rework, and the existing REVISIONS derivation correctly surfaces that on internal dashboards. + +--- + +## 11. Delivery Builder + +### 11.1 Flow + +1. Web UI: user picks project + episode → E18 shows e.g. *"Episode 101 — Ready: 18 shots"* with the exact Export versions. +2. **Build Netflix Delivery** → E16 creates the package (QUEUED) + `DELIVERY_BUILD` job. +3. A worker claims it (same E8 queue), and for each item: create folders per template, **hard-link** EXRs (fallback copy), copy/link MOV, verify per-file checksums against `Export.checksum` data, write manifest + report, then report completion (package READY, exports PACKAGED). Progress via E9 (items done / total). +4. Human uploads the folder to Content Hub, clicks **Mark Delivered** (E19) → DELIVERED. + +### 11.2 Naming configuration — `Project.deliveryConfig` + +Default template encodes the studio's confirmed structure (18.1-Q6): `/DELIVERY/{episode}/{YYMMDD}_Delivery/{shotCode}/` with EXR sequence and the slate/burn-in preview MOV together in each shot folder. + +```json +{ + "packageNameTemplate": "{yymmdd}_Delivery", + "layout": { + "packageDir": "{deliveryRoot}/{episode}/{packageName}", + "shotDir": "{packageDir}/{shotCode}", + "reportFile": "{packageDir}/{packageName}_report.csv" + }, + "deliveryRoot": "//SAN/DELIVERY", + "includeMovs": true, + "checksumAlgo": "xxh64", + "reportFormats": ["csv", "json"], + "colorPipeline": { "workingSpace": "ACES2065-1", "logSpace": "ACEScct", "showLut": "//SAN/luts/UNG/UNG_show_v02.cube" }, + "preview": { "engine": "ae", "movOutputTemplate": "VFXR_PREVIEW_MOV" } +} +``` + +Per shot folder the builder places the EXR sequence files and the delivery MOV (`Export.deliveryMovPath`) side by side — no `exr/`/`mov/` subfolders, matching current practice. `{episode}` renders as `101` (season.episode as used today); `{yymmdd}` from the build date (overridable in E16 `options.date`). + +Tokens: `{showId} {episode} {yymmdd} {yyyymmdd} {packageName} {packageDir} {shotCode} {versionString} {packageNumber}`. Null config → `SystemConfig` default template. The resolved config is snapshotted into `DeliveryPackage.configSnapshot`, so historic packages always show the rules they were built with, and future productions with different specs are a config change, not a code change. + +### 11.3 Hard links vs copying + +**Recommendation: hard link when `deliveryRoot` and render root share an NTFS volume; transparent fallback to copy otherwise (cross-volume links are impossible).** Hard links make an 18-shot EXR package near-instant and ~zero extra disk; the risk (mutating a linked file changes both) is acceptable because rendered EXR versions are immutable by pipeline design — a version is never re-rendered in place (new version → new directory). Post-link verification: link target identity check; post-copy verification: size + xxHash against validation-time values. `DeliveryItem.status` → VERIFIED per item. Config override `"copyMode": "always-copy"` for productions that mandate physical separation. + +### 11.4 Manifest and delivery report + +- **Manifest** (`DeliveryPackage.manifest`, also `manifest.json` inside the package): every file with relative path, bytes, checksum, source Export id, plus tool/pipeline versions. +- **Delivery report** (CSV per `reportFormats`, uploaded via storage abstraction to `deliveries/` and written into the folder): one row per shot — shotCode, versionString, frame range, frame count, resolution, colourspace, EXR count/bytes, MOV name, QC reviewer/date, checksums. This is the client-facing document; columns confirmed against the delivery spec before first real delivery (18.2-C2) and configurable via `deliveryConfig`. + +### 11.5 Delivery history + +The Delivery Packages page (Section 13) lists every package permanently: `#15 · 101/260805_Delivery · Episode 101 · 18 shots · built 2026-08-05 · delivered 2026-08-05 by Chris`, expandable to items → Export → full render/validation/QC history. Packages are never deleted; a mistaken package is CANCELLED (DRAFT/QUEUED/FAILED only) or superseded by building a new one. + +--- + +## 12. AE Panel Changes + +The panel keeps all existing capabilities (episode/shot discovery, overlays, picture-lock pull, manual queue buttons remain as escape hatches). New/changed behaviour: + +### 12.1 Panel status header (always visible per selected shot) + +| Field | Source | +|---|---| +| Shot | existing `/api/ext/shots/lookup` | +| Current Version | `shot.shotVersion` (lookup) | +| Current Status | `shot.status` (lookup) | +| Latest Export + QC status | new E2 `exports/latest` — e.g. `v004 — READY_FOR_QC`, `v004 — RENDERING 42% (ETA 1:36 on RENDER-01)`, `v003 — QC FAILED: "grain mismatch"` | + +While an export is active the panel polls E2 every 15 s to live-update the header. + +### 12.2 Buttons and their API calls + +| Button | Behaviour | API calls | +|---|---|---| +| **Queue Export** | Save project (scripted), build manifest from lookup data + active comp, submit. No local rendering, no version click. Confirms: "Queued v004 on render farm." | `GET lookup` → `POST /api/ext/exports` (E1) | +| **Open Latest Export** | Import latest export's EXR sequence into the current project (existing import-renders code, pointed at E2's `outputDir`) | E2, then local import | +| **Load QC Queue** | Populate QC list (shot, version, ready-since, thumbnail); selection imports that render into the QC comp | `GET /api/ext/qc/queue` (E5), E3 | +| **QC Pass** | One click; optional notes field beside it | `POST /api/ext/exports/{id}/qc {PASS}` (E4) | +| **QC Fail** | Notes strongly encouraged (panel nags if empty) | E4 `{FAIL}` | +| **Retry Export** (shown only when latest export is in a *_FAILED state) | Re-queue | `POST /api/ext/exports/{id}/retry` (E14) | + +### 12.3 Removed from the artist's world + +Increment Version button (server does it on E1 — panel may keep it hidden behind an "advanced" toggle for emergencies), Prep Delivery PowerShell scripts (replaced by Section 11), manual preview-comp/MP4 queueing for review purposes (worker generates the review MOV). + +--- + +## 13. Web UI Changes + +All new pages use existing NextAuth sessions, existing layout/navigation, and the internal mirrors of the APIs (E15–E21). Suggested routes under `/(dashboard)`: + +| Page | Route | Contents | +|---|---|---| +| **Pipeline Dashboard** | `/pipeline` | Cards: queued / rendering / failed / ready-for-QC / ready-for-delivery counts; machines online; last 24 h throughput; recent failures | +| **Render Queue** | `/pipeline/queue` | Live table (poll 5 s): shot, version, status, progress bar, ETA, machine, attempt; row actions Retry/Cancel/Bump priority; filters project/episode/status | +| **Export detail** | `/pipeline/exports/[id]` | State timeline (ExportEvent), manifest, per-attempt logs (logFileKey), validation table, preview player (existing player component), QC history, delivery membership | +| **QC Queue** | `/pipeline/qc` | READY_FOR_QC grid with thumbnails; preview player; Pass/Fail with notes (same service as E4) | +| **Delivery Packages** | `/pipeline/deliveries` | Episode picker → "Ready: N shots" → Build button; package list with status chips; detail = items, report download, Mark Delivered | +| **Machine Monitoring** | `/pipeline/machines` | Machine cards: status, current job, CPU/mem/disk sparkline (WorkerHeartbeat), AE/worker versions; Enable/Disable toggle | +| **Validation Results** | inside Export detail + `/pipeline/validation` | Filterable failed-check browser across exports (spot systemic issues, e.g. every shot failing colourspace) | +| **Render History** | `/pipeline/history` | All RenderJobs: durations, machines, success rate; per-machine and per-episode aggregates | + +Small touches to existing pages: Shot detail gains an "Exports" tab (list of Exports with status chips); the existing version player is untouched (previews are Versions). + +--- + +## 14. Configuration Summary + +### 14.1 New SystemConfig keys (defaults) + +| Key | Default | Meaning | +|---|---|---| +| `render.pollSeconds` | 10 | worker claim poll | +| `render.heartbeatSeconds` | 30 | heartbeat interval | +| `render.leaseSeconds` | 300 | job lease | +| `render.maxAttempts` | 3 | auto-retry ceiling | +| `render.stallTimeoutSeconds` | 600 | no-progress kill | +| `validation.sampleEvery` | 25 | header-check sampling stride | +| `preview.maxWidth` | 1920 | preview scale | +| `delivery.defaultConfig` | JSON | fallback `deliveryConfig` | + +### 14.2 New environment variables + +None required server-side (deliberate — everything is `SystemConfig` or per-project). Worker config is file-based (7.8). + +--- + +## 15. Security and Authentication + +| Surface | Mechanism | +|---|---| +| Worker + panel endpoints (`/api/ext/*`) | Existing shared `API_SECRET_KEY` bearer/x-api-key — **confirmed studio decision (18.1-Q7): one key for now, all phases**. `Machine.apiKeyHash` stays in the schema so per-machine keys are a config feature, not a migration, if the fleet grows | +| Web UI + internal APIs | Existing NextAuth session; new pages respect existing role checks (delivery build + mark-delivered restricted to admin/coordinator roles per existing role model) | +| Attribution | `submittedByEmail`/`reviewerEmail` resolved to `User` rows by email where possible; stored as free text otherwise (panel is API-key-authed, not user-authed — acceptable per 18.1-Q7) | +| Middleware | `/api/ext/` is already allow-listed in middleware with per-route auth in handlers; new routes follow the identical pattern | +| Path safety | Server-generated output/delivery paths only from templates + validated tokens (`shotCode` regex, episode regex); worker refuses to delete/clear any directory outside configured render/delivery roots | + +--- + +## 16. Failure Handling and Operations Summary + +| Scenario | Behaviour | +|---|---| +| Worker crash / power loss mid-render | Lease expires → reaper requeues (T5) or fails after max attempts; orphan output cleaned by next attempt | +| Server down mid-render | Worker spools reports to disk, keeps rendering, replays in order on reconnect | +| Two workers race for one job | Impossible: `FOR UPDATE SKIP LOCKED` claim | +| Zombie worker reports on reassigned job | Rejected: machineId ≠ lease holder | +| Artist queues while a render is active | 409 unless `force` (which supersedes + cancels the active job) | +| aerender hangs silently | `stallTimeoutSeconds` kill → fail/retry | +| Validation fails at 3 am | Pipeline stops at VALIDATION_FAILED; exact check rows recorded; dashboard + (existing Slack webhook, optional Phase 3 nicety) surface it in the morning | +| Delivery volume out of space | Build job fails with explicit error; package FAILED; retryable after cleanup | +| Machine misbehaving | Admin toggles `enabled=false` → cannot claim; running job finishes or is cancelled | + +--- + +## 17. Implementation Plan + +Five independently deployable phases. Each phase leaves the previous workflow fully functional; artists can fall back to today's manual flow at any point until Phase 5 completes. + +### 17.1 Phase 1 — Render Queue (schema + queue API + panel Queue Export) + +| Aspect | Content | +|---|---| +| Objectives | Queue Export from panel creates Export/RenderJob; jobs visible in web UI; **no worker yet** (jobs sit QUEUED; a temporary "mark as done manually" admin action lets the studio keep using the old render path while the queue is validated) | +| DB | Migrations: all Section 5 enums + `Export`, `RenderJob`, `ExportEvent`; `Project.deliveryConfig` column; `@@unique([shotId, versionNumber])` | +| API | E1, E2, E3, E14, E21 (queue list), internal retry/cancel (E15); manifest Zod schema; service layer `lib/render-pipeline/exports.ts` | +| UI | Render Queue page (basic table), Export detail (manifest + events), Shot detail Exports tab | +| Panel | Queue Export button + status header (E2 polling); Increment Version hidden | +| Risks | Version-increment race between panel-PATCH legacy path and E1 → mitigate: E1 increments transactionally and panel stops PATCHing when the new button is used; duplicate `@@unique` guards | +| Testing | Unit: manifest validation, version increment transaction, state transitions incl. 409s. Integration: two concurrent E1 calls for one shot. Panel: queue against staging | +| Migration | Purely additive migration; deploy = normal `prisma migrate deploy` in entrypoint; no data backfill | + +### 17.2 Phase 2 — Render Worker + +| Aspect | Content | +|---|---| +| Objectives | The two artist workstations render queued jobs end-to-end to "EXRs on disk + job COMPLETED"; after-hours windows + Render Now override working; multi-machine ready by construction | +| DB | `Machine` (incl. `availability`, `renderNowUntil`), `WorkerHeartbeat` | +| API | E6–E11, E20; atomic claim SQL with availability gate (7.10); reaper task; internal machines API incl. Render Now toggle | +| UI | Machine Monitoring page (status, current job, Render Now button, window display); live progress/ETA in Render Queue | +| Worker | .NET service: registration, heartbeat, claim loop, aerender runner, progress parsing, lease renewal, crash recovery, spooled reporting, path mapping, installer + docs | +| Risks | Worker shares hardware with working artists → mitigated by schedule windows + server-side claim gate; AE version drift between the two workstations silently changes renders → registration reports `aeVersion`, dashboard warns on mismatch; stdout format differences across AE versions → progress parser tolerant, integration-tested per AE release; SAN permissions for the service account | +| Testing | Worker unit tests with recorded aerender transcripts (success, error, stall); kill -9 worker mid-render → verify reaper requeue; two workers + three jobs → no double claim; network-cut replay test; availability-window claim tests (in/out of window, Render Now, urgent priority) | +| Migration | Additive migration; install worker on one workstation first, second after a clean week; `render.maxAttempts=1` for first week (observe before auto-retrying) | + +### 17.3 Phase 3 — Validation + +| Aspect | Content | +|---|---| +| Objectives | Every render automatically validated; failures stop pipeline with exact reasons | +| DB | `ValidationResult` | +| API | E12; transitions T3→T7/T8 activated (Phase 2 interim: complete-render goes straight to a provisional READY_FOR_QC; Phase 3 inserts VALIDATING between) | +| UI | Validation table on Export detail; Validation Results browser; failure badge in queue | +| Worker | Validation engine (oiiotool integration, all Section 8.1 checks, checksums); oiiotool bundled with installer | +| Risks | Colourspace/timecode metadata may be absent in AE-written EXRs → those checks SKIPPED until per-project expectations configured (18.2-Q1); false FAILs block artists → per-check severity override in SystemConfig (`validation.overrides: {"exr_type": "warn"}`) | +| Testing | Golden EXR fixture sets: complete, gapped, duplicate, wrong-res, wrong-depth, no-alpha, corrupt, zero-byte; checksum determinism; large-sequence perf (500 frames < 60 s) | +| Migration | Additive; existing in-flight exports unaffected (transition map keys off manifest version) | + +### 17.4 Phase 4 — Preview + QC + +| Aspect | Content | +|---|---| +| Objectives | Automatic slate/burn-in delivery MOV + web preview + thumbnail + metadata; renders appear as internal-only Versions; QC queue live in panel and web | +| DB | `QCReview` | +| API | E13 (finalize incl. pipeline-mode Version creation via existing version service), E4, E5; QC internal routes | +| UI | QC Queue page; QC history on Export detail | +| Worker | Headless AE preview stage (`AfterFX.com -noui -r` driving the studio slate/overlay template + colour chain, Section 9.1), ffmpeg web transcode + thumbnail, artifact upload via presign; ffmpeg fallback engine | +| Panel | Load QC Queue, QC Pass/Fail, Open Latest Export; preview-comp JSX refactored so panel and worker share one script source | +| Risks | Headless AE (`-noui`) reliability on workstations is the phase's main unknown → prove it in week 1 with a spike before building around it (18.2-C4); preview colour must match reference and be supervisor-signed-off before artists trust it; pipeline-mode Version creation must not regress the manual upload path and must never leak client-visible media on approved shots → shared function + explicit regression tests for both | +| Testing | Preview parity: pipeline MOV vs artist-rendered preview of same shot (slate fields, burn-ins, colour) signed off by supervisor; Version integration: pipeline Version invisible in client portal on a shared shot; QC transition tests incl. task→CHANGES and no-side-effect PASS | +| Migration | Additive; announce to artists that preview comps/MOVs are now automatic | + +### 17.5 Phase 5 — Delivery Builder + +| Aspect | Content | +|---|---| +| Objectives | One-click dated Netflix package from DB; permanent delivery history; old PowerShell prep retired | +| DB | `DeliveryPackage`, `DeliveryItem` | +| API | E16–E19; DELIVERY_BUILD job type through existing claim path | +| UI | Delivery Packages page (episode picker, ready counts, build, history, mark delivered); Pipeline Dashboard completed | +| Worker | Delivery builder stage: template resolution, hard-link/copy, verification, manifest + report writing | +| Risks | Folder structure and checksum approach are decided (18.1-Q6/Q13); report columns still need confirming against the delivery spec before first real delivery (18.2-C2); hard links on non-NTFS delivery volume → auto-fallback copy is default-safe | +| Testing | Package build fixture (3 shots) → structure, links vs copies, checksum verification, report content; immutability: re-render a packaged shot → package untouched, new Export not auto-included; permission tests on build/mark-delivered roles | +| Migration | Additive; run one parallel delivery (old script + new builder) and diff the folders before cutover | + +### 17.6 Dependency graph + +```mermaid +flowchart LR + P1["Phase 1 Queue"] --> P2["Phase 2 Worker"] --> P3["Phase 3 Validation"] --> P4["Phase 4 Preview+QC"] --> P5["Phase 5 Delivery"] +``` + +Strictly sequential; each phase is shippable and useful on its own (P1: visibility; P2: hands-off rendering; P3: trust; P4: hands-off review; P5: hands-off delivery). + +--- + +## 18. Design Decisions — Resolved and Remaining + +### 18.1 Resolved decisions (studio input, 2026-07-31) + +The original open questions were answered by the studio. The decisions below are **normative** and already folded into the sections referenced. + +| Q | Topic | Decision | Where applied | +|---|---|---|---| +| Q1 | Render fleet | aerender runs on the **two artist workstations**. After-hours auto-render via per-machine schedule windows; **manual "Render Now" trigger** for urgent daytime renders; urgent-priority jobs may claim anytime | 7.10, 5.6 (`Machine.availability`, `renderNowUntil`), E8 gate | +| Q2 | EXR storage | EXRs are **never uploaded to object storage**; SAN only | 2.3 | +| Q3 | Export concurrency | **Single active Export per shot** (E1 409 + `force` supersede) | 6.4 E1 | +| Q4 | Colour management | Per-project `colorPipeline` config. ACES workflows: plates/renders **linear ACES2065-1**; reference match = **linear → per-project log → show LUT** | 9.2, 8.1 colourspace check | +| Q5 | Burn-ins / slates | Use the **existing AE slate + overlay templates**: 1-frame head slate, burn-ins throughout, connector functions pull slate/overlay data from the API and load the colour transforms. Worker runs this headlessly; ffmpeg is emergency fallback only | 9 (rewritten) | +| Q6 | Delivery structure | `/DELIVERY/{episode}/{YYMMDD}_Delivery/{shotCode}/` containing EXR sequence + slate/burn-in preview MOV together | 11.2 default config | +| Q7 | Auth | **One shared `API_SECRET_KEY` for now**; `Machine.apiKeyHash` remains as dormant schema support | 15 | +| Q8 | Reaper | As recommended: `instrumentation.ts` interval + defensive sweep on each E8 claim | 7.7 | +| Q9 | Outputs per shot | **One comp → one EXR sequence per version**; future `ExportOutput` child table is an additive migration if ever needed | 6.2 | +| Q10 | Retention | As recommended: pipeline never auto-deletes; "reclaimable space" report; humans delete | 16 | +| Q11 | QC semantics | QC is **technical/final-delivery QC on approved shots** (mask slips, export glitches, missing frames, metadata/timecode, format/res/colourspace). **PASS = ready to upload, nothing shared to client review**; pipeline Versions are internal-only | 10.0, T12/T14, 6.4 E13 | +| Q12 | Per-project colour values | `colorPipeline` (log space + show LUT) is **required and unique per project** — set at project setup, supervisor signs off one reference comparison before Phase 4 goes live on that project | 9.2 | +| Q13 | Delivery report / checksums | Dual support as recommended: `xxh64` for internal verification, `md5` available for the client-facing manifest via `checksumAlgo`; report columns confirmed against the delivery spec before the first real delivery (18.2-C2) | 11.4 | +| Q14 | Preview MOV codec | Lives in the AE output-module template; record the current template name in `deliveryConfig.preview.movOutputTemplate` at Phase 4 setup (18.2-C3) | 9.1, 11.2 | +| Q15 | Headless AE approach | As recommended: `AfterFX.com -noui` with the week-1 Phase 4 spike; documented fallbacks stand if the spike fails (18.2-C4) | 9.1, 17.4 | +| Q16 | Render windows / urgent policy | Defaults accepted: weekdays 19:00–08:00 + full weekends, Render Now = 4 h, urgent queueing available to all users; exact per-machine times collected at Phase 2 rollout (18.2-C5) | 7.10 | + +### 18.2 Implementation-time checklist + +No design decisions remain. These are concrete values to collect or verifications to run at the noted point — each traces back to a resolved decision in 18.1. + +| # | Item | When | What to do | +|---|---|---|---| +| C1 | Per-project colour values | Pipeline enablement per project (first: Phase 4 setup) | Enter the project's log space and show LUT path into `deliveryConfig.colorPipeline` — **required, unique per project** (Q12); supervisor signs off one pipeline-vs-reference comparison before artists rely on previews | +| C2 | Delivery report columns + client checksum requirement | Before the first real Phase 5 delivery | Confirm report columns and whether the client mandates MD5 sidecars against the production's delivery spec; set `checksumAlgo` accordingly (Q13) | +| C3 | Preview MOV output template | Phase 4 setup | Record the studio's current "LT Preview" AE output-module template name in `deliveryConfig.preview.movOutputTemplate` (Q14) | +| C4 | Headless AE spike | Week 1 of Phase 4, before further preview code | Render one preview via `AfterFX.com -noui -r` under the service account on **both** workstations, after hours, artist logged out. If it fails: fall back to panel-pre-built preview comp in the artist's AEP (plain `aerender` second comp), then ffmpeg engine as last resort (Q15) | +| C5 | Render windows + Render Now duration | Phase 2 rollout | Confirm per-machine window times against the accepted defaults (weekdays 19:00–08:00 + weekends, Render Now 4 h, urgent for all users) and enter into `Machine.availability` (Q16) | + +--- + +## 19. Document Map for Implementers + +| Section | Governs | Primary phase | +|---|---|---| +| 4 | All state transitions (server-enforced) | 1 | +| 5 | Schema migrations | 1, 2, 3, 4, 5 | +| 6 | API contracts + examples | 1, 2, 3, 4, 5 | +| 7 | Worker service | 2 | +| 8 | Validation checks | 3 | +| 9 | Preview artifacts | 4 | +| 10 | QC semantics | 4 | +| 11 | Delivery builder | 5 | +| 12 | AE panel | 1, 4 | +| 13 | Web UI | 1–5 | +| 17 | Sequencing, testing, migration | all | +| 18.1 | Resolved studio decisions (normative) | all | +| 18.2 | Implementation-time checklist (values/verifications) | as noted | + +End of specification. diff --git a/TECHNICAL_ARCHITECTURE_REPORT_CURRENT_STATE.md b/TECHNICAL_ARCHITECTURE_REPORT_CURRENT_STATE.md new file mode 100644 index 0000000..b745de8 --- /dev/null +++ b/TECHNICAL_ARCHITECTURE_REPORT_CURRENT_STATE.md @@ -0,0 +1,753 @@ +# Technical Architecture Report (Current State) + +Date: 2026-07-31 +Scope: Current implementation only (no redesign suggestions) + +--- + +## 1. Database + +Authoritative schema source: [prisma/schema.prisma](prisma/schema.prisma) + +### Full Prisma Schema +The full schema is defined in [prisma/schema.prisma](prisma/schema.prisma). + +### Models related to requested domains + +- Users/Auth + - User + - Account + - Session + - VerificationToken + - ClientAccess +- Projects/Episodes + - Project + - EpisodeDueDate + - Client +- Shots/Tasks/Versions/Reviews + - Shot + - ShotGroup + - Task + - Version + - Comment + - CommentReply + - Annotation + - Approval + - ReviewSession +- Files/Storage + - FootagePlate + - ShotReference + - SystemConfig +- Delivery/Export adjacent + - Shot fields: highResKey, highResFilename, exrOutput, shotVersion + - Version fields: fileUrl, fileName, proxyUrl, thumbnailUrl, posterUrl + +### Relationship summary + +- Client 1:N Project +- Project 1:N Shot +- Project 1:N Task +- Project 1:N ReviewSession +- Project 1:N EpisodeDueDate +- Shot 1:N Task +- Shot 1:N Version +- Shot 1:N FootagePlate +- Shot 1:N ShotReference +- Task 1:N Version +- Version 1:N Comment +- Version 1:N Annotation +- Version 1:N Approval +- Comment 1:N CommentReply +- User has many assigned/created entities across shots, tasks, versions, comments, approvals + +### Existing status enums + +From [prisma/schema.prisma](prisma/schema.prisma): + +- ProjectStatus: ACTIVE, ON_HOLD, COMPLETED, ARCHIVED +- ShotStatus: WAITING, IN_PROGRESS, INTERNAL_REVIEW, READY_FOR_CLIENT, CLIENT_REVIEW, REVISIONS, COMPLETE +- ShotApprovalStatus: PENDING, INTERNALLY_APPROVED, CLIENT_APPROVED +- TaskStatus: TODO, IN_PROGRESS, INTERNAL_REVIEW, CLIENT_REVIEW, CHANGES, DONE +- ApprovalStatus: PENDING_REVIEW, APPROVED, REJECTED, NEEDS_CHANGES +- ReviewStatus: PENDING, INTERNAL_APPROVED, CLIENT_APPROVED, NEEDS_CHANGES, FINAL_APPROVED + +Notes: +- There is no dedicated Delivery model. +- There is no dedicated Export model. +- Delivery/export state is represented by file pointers and shot/version metadata fields. + +--- + +## 2. API + +### 2.1 Shots + +#### External shot APIs + +- GET /api/ext/projects + - URL: /api/ext/projects + - Method: GET + - Purpose: List projects for pipeline tools + - Request body: None + - Response: projects[] with id, name, code, showId, projectType, status, dates, _count + - Source: [app/api/ext/projects/route.ts](app/api/ext/projects/route.ts) + +- GET /api/ext/projects/{projectCode}/episodes + - URL: /api/ext/projects/{projectCode}/episodes + - Method: GET + - Purpose: List distinct episodes and optionally shot payloads per episode + - Request body: None + - Response: project + episodes[]; optional shots[] includes exrOutput/timecodes + - Source: [app/api/ext/projects/[projectCode]/episodes/route.ts](app/api/ext/projects/%5BprojectCode%5D/episodes/route.ts) + +- GET /api/ext/projects/{projectCode}/shots + - URL: /api/ext/projects/{projectCode}/shots + - Method: GET + - Purpose: List shots by project code with filters/pagination + - Request body: None + - Response: project + pagination + shots[] + - Source: [app/api/ext/projects/[projectCode]/shots/route.ts](app/api/ext/projects/%5BprojectCode%5D/shots/route.ts) + +- GET /api/ext/shots + - URL: /api/ext/shots + - Method: GET + - Purpose: Legacy listing by projectId + - Request body: None + - Response: shots[] + total + - Source: [app/api/ext/shots/route.ts](app/api/ext/shots/route.ts) + +- POST /api/ext/shots + - URL: /api/ext/shots + - Method: POST + - Purpose: Create shot from external tool (JSON or multipart thumbnail) + - Request body: + - projectId, scene + - optional episode, description, artistId, priority, fps, frameStart, frameEnd, dueDate, thumbnailUrl, shotGroupName, shotCode + - optional thumbnail file (multipart) + - Response: created shot object + - Source: [app/api/ext/shots/route.ts](app/api/ext/shots/route.ts) + +- GET /api/ext/shots/lookup + - URL: /api/ext/shots/lookup + - Method: GET + - Purpose: Canonical shot lookup by shotCode (+ optional projectCode) + - Request body: None + - Response: shot object with project, artist, tasks, latest version, exrOutput, shotVersion, source/seq timecodes + - Source: [app/api/ext/shots/lookup/route.ts](app/api/ext/shots/lookup/route.ts) + +- GET /api/ext/shots/{shotId} + - URL: /api/ext/shots/{shotId} + - Method: GET + - Purpose: Shot detail by DB id, or byCode mode + - Request body: None + - Response: full shot detail with tasks/latest version/counts + - Source: [app/api/ext/shots/[shotId]/route.ts](app/api/ext/shots/%5BshotId%5D/route.ts) + +- PATCH /api/ext/shots/{shotId} + - URL: /api/ext/shots/{shotId} + - Method: PATCH + - Purpose: Update mutable shot field(s) from pipeline tools + - Request body: shotVersion (v###) currently supported + - Response: success + updated shot id/shotVersion + - Source: [app/api/ext/shots/[shotId]/route.ts](app/api/ext/shots/%5BshotId%5D/route.ts) + +#### Internal shot APIs + +- GET /api/shots/{shotId} + - URL: /api/shots/{shotId} + - Method: GET + - Purpose: Internal dashboard shot detail payload + - Request body: None + - Response: shot + tasks + artists + permissions flags + - Source: [app/api/shots/[shotId]/route.ts](app/api/shots/%5BshotId%5D/route.ts) + +- GET /api/projects/{projectId}/episodes + - URL: /api/projects/{projectId}/episodes + - Method: GET + - Purpose: Internal distinct episode list for project + - Request body: None + - Response: episodes[] + - Source: [app/api/projects/[projectId]/episodes/route.ts](app/api/projects/%5BprojectId%5D/episodes/route.ts) + +### 2.2 Reviews + +- GET /api/review-sessions + - Purpose: list review sessions +- POST /api/review-sessions + - Purpose: create review session token and portal link +- DELETE /api/review-sessions + - Purpose: deactivate review session +- Source: [app/api/review-sessions/route.ts](app/api/review-sessions/route.ts) + +- POST /api/client/{token}/auth + - Purpose: review password check + unlock cookie + - Source: [app/api/client/[token]/auth/route.ts](app/api/client/%5Btoken%5D/auth/route.ts) + +- GET /api/client/{token}/project + - Purpose: client portal project payload (shared shots/versions) + - Source: [app/api/client/[token]/project/route.ts](app/api/client/%5Btoken%5D/project/route.ts) + +- GET /api/client/{token}/versions/{versionId} + - Purpose: client review detail payload + - Source: [app/api/client/[token]/versions/[versionId]/route.ts](app/api/client/%5Btoken%5D/versions/%5BversionId%5D/route.ts) + +- POST /api/client/{token}/comment + - Purpose: client frame comment + - Request body: versionId, frameNumber, timestamp, text + - Response: created comment + - Source: [app/api/client/[token]/comment/route.ts](app/api/client/%5Btoken%5D/comment/route.ts) + +- POST /api/client/{token}/annotation + - Purpose: client annotation write + - Request body: versionId, frameNumber, drawingData, optional color + - Response: annotation + - Source: [app/api/client/[token]/annotation/route.ts](app/api/client/%5Btoken%5D/annotation/route.ts) + +- POST /api/client/{token}/approve + - Purpose: shot-level approve/changes and legacy version-level approval + - Request body: + - shot mode: shotId + action + - version mode: versionId + status + notes + - Response: success + - Source: [app/api/client/[token]/approve/route.ts](app/api/client/%5Btoken%5D/approve/route.ts) + +- GET /api/versions/{versionId}/comments + - Purpose: internal comments read + - Source: [app/api/versions/[versionId]/comments/route.ts](app/api/versions/%5BversionId%5D/comments/route.ts) + +- GET /api/versions/{versionId}/annotations + - Purpose: internal annotations read + - Source: [app/api/versions/[versionId]/annotations/route.ts](app/api/versions/%5BversionId%5D/annotations/route.ts) + +- GET /api/playlist + - Purpose: latest version per shot playlist + - Source: [app/api/playlist/route.ts](app/api/playlist/route.ts) + +### 2.3 File uploads + +- POST /api/upload + - Purpose: authenticated upload to Hetzner via app server + - Body: multipart file (+ type) + - Response: url, key + - Source: [app/api/upload/route.ts](app/api/upload/route.ts) + +- POST /api/upload/local + - Purpose: authenticated video upload path + - Body: multipart file + - Response: url, key + - Source: [app/api/upload/local/route.ts](app/api/upload/local/route.ts) + +- POST /api/upload/presign + - Purpose: direct browser->object-storage upload URL + - Body: fileName, contentType, optional folder + - Response: presignedUrl, key, url + - Source: [app/api/upload/presign/route.ts](app/api/upload/presign/route.ts) + +- GET/POST /api/uploadthing + - Purpose: UploadThing route handler passthrough (if configured) + - Source: [app/api/uploadthing/route.ts](app/api/uploadthing/route.ts) + +- POST /api/batch-upload/presign + - Purpose: presign high-res upload key + - Body: fileName + - Response: presignedUrl, key + - Source: [app/api/batch-upload/presign/route.ts](app/api/batch-upload/presign/route.ts) + +- POST /api/batch-upload/preview + - Purpose: classify upload actions before upload + - Body: projectId, fileNames[] + - Response: items[] with statuses (new-version, rename-and-upload, create-task, update-highres, etc) + - Source: [app/api/batch-upload/preview/route.ts](app/api/batch-upload/preview/route.ts) + +- POST /api/batch-upload/upload + - Purpose: commit highres key or create version records + - Body: + - update-highres: action, shotId, projectId, key, fileName + - version upload: action, shotId, projectId, file, task routing fields + - Response: success + action result + - Source: [app/api/batch-upload/upload/route.ts](app/api/batch-upload/upload/route.ts) + +### 2.4 EXRs / Rendering / Delivery / Storage / Metadata + +- GET /api/files/{...key} + - Purpose: file serving, range requests, local/Hetzner routing + - Source: [app/api/files/[...key]/route.ts](app/api/files/%5B...key%5D/route.ts) + +- GET/POST /api/admin/migration + - Purpose: local uploads migration status and per-key migration to Hetzner + - Source: [app/api/admin/migration/route.ts](app/api/admin/migration/route.ts) + +- POST/DELETE /api/storage-test + - Purpose: upload test object / delete test object + - Source: [app/api/storage-test/route.ts](app/api/storage-test/route.ts) + +- POST/DELETE /api/shots/{shotId}/highres + - Purpose: upload/remove high-res deliverable on shot + - Source: [app/api/shots/[shotId]/highres/route.ts](app/api/shots/%5BshotId%5D/highres/route.ts) + +- GET /api/shots/{shotId}/highres/download + - Purpose: internal presigned download URL for high-res + - Source: [app/api/shots/[shotId]/highres/download/route.ts](app/api/shots/%5BshotId%5D/highres/download/route.ts) + +- GET /api/client/{token}/shots/{shotId}/highres/download + - Purpose: client token-gated presigned high-res download URL + - Source: [app/api/client/[token]/shots/[shotId]/highres/download/route.ts](app/api/client/%5Btoken%5D/shots/%5BshotId%5D/highres/download/route.ts) + +- GET/POST/DELETE /api/shots/{shotId}/references + - Purpose: shot reference image management + - Source: [app/api/shots/[shotId]/references/route.ts](app/api/shots/%5BshotId%5D/references/route.ts) + +- GET /api/storyboard/pdf + - Purpose: server-rendered printable storyboard HTML with metadata options + - Source: [app/api/storyboard/pdf/route.ts](app/api/storyboard/pdf/route.ts) + +Notes: +- No dedicated REST endpoint that runs ffmpeg in this repository. +- No dedicated REST endpoint that performs EXR rendering on the server. +- No dedicated deliveries API namespace. + +--- + +## 3. Storage + +Primary storage abstraction: [lib/storage.ts](lib/storage.ts) + +### Provider modes + +- local +- uploadthing +- s3 +- r2 +- b2 +- minio + +### Dedicated high-res object storage path + +- Hetzner object storage helper methods are used for high-res and presigned direct uploads. +- Config source precedence: + - SystemConfig table keys + - env fallback + +### Folder/key layout in object storage (current code paths) + +- videos/ +- image/ +- highres/ +- storage-test/ + +### Local storage + +- LOCAL_UPLOAD_DIR (default ./uploads) +- Served via /api/files catch-all route + +### File naming conventions + +- Key format: {folder}/{uuid}-{sanitized-file-name} +- Sanitization done by sanitizeFileName to avoid URL/signature problems in object keys + +### EXR locations + +Within web app/runtime: +- EXR references are naming metadata (Shot.exrOutput) and file-serving pathing. + +Within DCC tooling docs/scripts: +- AE/Nuke workflows target shared export roots and per-shot folders. + +### Preview locations + +- Version media URLs stored in Version.fileUrl +- Accessed through app player routes and /api/files routing where applicable + +### Thumbnail locations + +- Shot.thumbnailUrl +- Version.thumbnailUrl +- Can point to /api/files/{key} or external provider URL + +Relevant sources: +- [lib/storage.ts](lib/storage.ts) +- [app/api/files/[...key]/route.ts](app/api/files/%5B...key%5D/route.ts) +- [actions/settings.ts](actions/settings.ts) + +--- + +## 4. Export Pipeline (Current) + +### AE panel + +Documented in repo: +- [VFXReviewConnector.md](VFXReviewConnector.md) +- [EXT_API_REFERENCE.md](EXT_API_REFERENCE.md) + +Implemented panel script also exists on the host AE installation (outside workspace), and behavior aligns with docs: +- Episodes/shots discovery via ext APIs +- Shot lookup API usage for metadata +- Overlay/slate essential property updates (burn-in style overlays) +- Queue EXR / Queue MP4 / Queue MOV render queue actions +- Queue EXR (review convention) +- Import exported EXR and import renders +- Pull picture lock using seq timecodes +- Increment shot version by PATCH call +- Delivery prep that copies/renames EXR files to delivery folder convention + +### External API endpoints used by DCC tools + +- GET /api/ext/projects/{projectCode}/episodes +- GET /api/ext/projects/{projectCode}/shots +- GET /api/ext/shots/lookup +- PATCH /api/ext/shots/{shotId} + +### Existing render scripts and pipelines + +- AE panel render queue automation (external script) +- Nuke connector script creating write-node outputs and shot scripts: + - [VFXReviewConnector.py](VFXReviewConnector.py) + +### ffmpeg scripts + +- None found in this repository. + +### Proxy generation + +- Version.proxyUrl field exists in schema. +- No implemented proxy-generation worker/function found. + +### Thumbnail generation + +- Upload/assignment flows exist. +- No server-side frame-extract thumbnail generator found. + +### Metadata extraction + +- EDL / picture-tracker CSV metadata parsing implemented: + - [lib/edl-utils.ts](lib/edl-utils.ts) + - [actions/shots.ts](actions/shots.ts) + +### Burn-in generation + +- Achieved via DCC overlay/slate layers and essential properties in AE/Nuke tool workflows. +- No ffmpeg burn-in path found. + +--- + +## 5. Review System + +### Current review workflow + +Core status derivation: +- [lib/shot-status.ts](lib/shot-status.ts) + +Server action orchestration: +- [actions/versions.ts](actions/versions.ts) +- [actions/approvals.ts](actions/approvals.ts) +- [actions/tasks.ts](actions/tasks.ts) +- [actions/shots.ts](actions/shots.ts) +- [actions/comments.ts](actions/comments.ts) + +### Internal review + +- Version upload: + - marks previous versions non-latest + - creates new latest version + - moves Task to INTERNAL_REVIEW + - recalculates Shot.status + +### Client review + +- Tokenized ReviewSession links +- Optional password gate with signed cookie unlock +- Client comment/annotation/approval endpoints +- Share/unshare semantics via shot and version visibility fields + +### Task status flow + +Observed statuses: +- TODO +- IN_PROGRESS +- INTERNAL_REVIEW +- CLIENT_REVIEW +- CHANGES +- DONE + +### Shot status flow + +Derived in priority order: +- REVISIONS (any task CHANGES) +- COMPLETE (shotApprovalStatus CLIENT_APPROVED) +- CLIENT_REVIEW / READY_FOR_CLIENT (internally approved + share flag) +- IN_PROGRESS (task TODO/IN_PROGRESS) +- INTERNAL_REVIEW (tasks exist) +- WAITING (no tasks) + +### Approval process + +- Version-level approvals create Approval rows and update Version.approvalStatus +- Shot-level client actions supported via client approve endpoint and shot actions + +Reference doc in repo: +- [Shot task workflow.md](Shot%20task%20workflow.md) + +--- + +## 6. Authentication + +### External tools + +- /api/ext/* routes authenticate using API_SECRET_KEY via: + - Authorization: Bearer + - x-api-key header + +### Client review links + +- tokenized route with ReviewSession lookup +- optional password hash validation and signed unlock cookie + +### App users + +- NextAuth credentials provider +- bcrypt password hash compare +- JWT session strategy + +### Middleware behavior + +- Route classes allowed through middleware auth gate: + - /api/ext/ + - /api/client/ + - /api/display/ + - /api/files/ + - /api/uploadthing +- Per-route auth still enforced in handlers + +Sources: +- [auth.ts](auth.ts) +- [auth.config.ts](auth.config.ts) +- [middleware.ts](middleware.ts) +- [lib/review-auth.ts](lib/review-auth.ts) + +--- + +## 7. Existing Background Jobs + +### Cron jobs +- None found. + +### Queues +- None found. + +### Workers +- None found. + +### Polling services +- Display devices poll: + - /api/display/events + - /api/dashboard/stats + +### Docker containers +- vfxreview app container +- postgres container +- Source: [docker-compose.yml](docker-compose.yml) + +### Scheduled tasks +- Startup migration commands in container entrypoint: + - [entrypoint.sh](entrypoint.sh) +- AE panel Prep Delivery launches background PowerShell copy scripts on workstation (outside web app runtime) + +--- + +## 8. After Effects Integration (Implemented Features) + +Repo-level references: +- [VFXReviewConnector.md](VFXReviewConnector.md) +- [EXT_API_REFERENCE.md](EXT_API_REFERENCE.md) + +Implemented capabilities observed/documented: + +- API calls + - episodes/shots listing and shot lookup + - shot version PATCH +- Authentication + - bearer API token in script +- Comp discovery + - shot code extraction from comp names and dropdown selection +- Render queue integration + - EXR Sequence + - REVIEW_PREVIEW (MP4) + - 4444 Tri (MOV) + - EXR review sequence queue variant +- Output path generation + - shot/version naming conventions + export root paths +- Shot lookup + - uses /api/ext/shots/lookup for metadata and naming decisions +- Upload features + - no direct upload-to-web API flow in panel docs/script (render/output handled in DCC/filesystem) +- Burn-in generation + - overlay and slate essential properties +- Review integration + - preview comp build and version increment sync + +--- + +## 9. Configuration + +### Environment variables + +Primary reference: [.env.example](.env.example) + +Observed vars in code/docs include: + +- DATABASE_URL +- NEXTAUTH_SECRET +- NEXTAUTH_URL +- NEXT_PUBLIC_APP_URL +- NEXT_PUBLIC_APP_NAME +- API_SECRET_KEY +- AUTH_SECRET +- STORAGE_PROVIDER +- LOCAL_UPLOAD_DIR +- AWS_ACCESS_KEY_ID +- AWS_SECRET_ACCESS_KEY +- AWS_REGION +- AWS_BUCKET_NAME +- R2_ACCESS_KEY_ID +- R2_SECRET_ACCESS_KEY +- R2_ACCOUNT_ID +- R2_BUCKET_NAME +- R2_PUBLIC_URL +- B2_APPLICATION_KEY_ID +- B2_APPLICATION_KEY +- B2_BUCKET_NAME +- B2_ENDPOINT +- MINIO_ENDPOINT +- MINIO_ACCESS_KEY +- MINIO_SECRET_KEY +- MINIO_BUCKET_NAME +- HETZNER_ENDPOINT +- HETZNER_ACCESS_KEY +- HETZNER_SECRET_KEY +- HETZNER_BUCKET_NAME +- UPLOADTHING_SECRET +- UPLOADTHING_APP_ID +- EMAIL_FROM +- EMAIL_SERVER_HOST +- EMAIL_SERVER_PORT +- EMAIL_SERVER_USER +- EMAIL_SERVER_PASSWORD +- SLACK_DEFAULT_WEBHOOK + +### Storage configuration + +- Provider abstraction in [lib/storage.ts](lib/storage.ts) +- Hetzner overrides in [actions/settings.ts](actions/settings.ts) and SystemConfig table + +### Render configuration + +- No server-side renderer configuration module found. +- DCC render templates are documented in [VFXReviewConnector.md](VFXReviewConnector.md). + +### ffmpeg configuration + +- None found in repository. + +### Object storage configuration + +- Implemented for AWS S3/R2/B2/MinIO + dedicated Hetzner helper path + +--- + +## 10. Existing Utility Functions (Reusable) + +### Metadata extraction + +- parseEdlCsv +- parsePictureTrackerCsv +- Source: [lib/edl-utils.ts](lib/edl-utils.ts) + +### File scanning + +- Local upload tree walker in migration route +- Nuke connector plate/render directory scanners +- Sources: + - [app/api/admin/migration/route.ts](app/api/admin/migration/route.ts) + - [VFXReviewConnector.py](VFXReviewConnector.py) + +### EXR sequence detection + +- Nuke connector helper sequence detectors and pattern conversion +- Source: [VFXReviewConnector.py](VFXReviewConnector.py) + +### MOV generation + +- No server-side MOV generation utility found. +- MOV outputs are DCC render queue outputs in external scripts/docs. + +### Checksums + +- No checksum utility found. + +### Frame counting + +- durationToFrameCount and frame math helpers +- Source: [lib/frame-utils.ts](lib/frame-utils.ts) + +### Timecode extraction/conversion + +- frameToTimecode / formatTimecode +- CSV timecode validation/parsing +- Sources: + - [lib/frame-utils.ts](lib/frame-utils.ts) + - [lib/utils.ts](lib/utils.ts) + - [lib/edl-utils.ts](lib/edl-utils.ts) + +--- + +## 11. High-Level Architecture Diagram + +```mermaid +flowchart LR + A[AE Panel / Nuke Connector\nExternal DCC tools] -->|API key auth| B[Next.js App Router APIs] + B --> C[Prisma ORM] + C --> D[(PostgreSQL)] + + B --> E[Storage Abstraction] + E --> F[(Hetzner Object Storage)] + E --> G[(S3/R2/B2/MinIO)] + B --> H[(Local uploads dir)] + + I[Internal reviewers\nNextAuth session] --> B + J[Client reviewers\nToken review sessions] --> B + + K[ESP32 display client] -->|x-display-key polling| B + + A --> L[Shared filesystem render roots\nEXR/MP4/MOV outputs] + L -->|served/linked via app metadata| B +``` + +--- + +## 12. Files Inspected (Primary) + +- [prisma/schema.prisma](prisma/schema.prisma) +- [lib/storage.ts](lib/storage.ts) +- [lib/shot-status.ts](lib/shot-status.ts) +- [lib/review-auth.ts](lib/review-auth.ts) +- [lib/edl-utils.ts](lib/edl-utils.ts) +- [lib/frame-utils.ts](lib/frame-utils.ts) +- [lib/utils.ts](lib/utils.ts) +- [auth.ts](auth.ts) +- [auth.config.ts](auth.config.ts) +- [middleware.ts](middleware.ts) +- [next.config.ts](next.config.ts) +- [.env.example](.env.example) +- [docker-compose.yml](docker-compose.yml) +- [Dockerfile](Dockerfile) +- [entrypoint.sh](entrypoint.sh) +- [EXT_API_REFERENCE.md](EXT_API_REFERENCE.md) +- [VFXReviewConnector.md](VFXReviewConnector.md) +- [VFXReviewConnector.py](VFXReviewConnector.py) +- [# VFXReview Connector for Nuke.md](#%20VFXReview%20Connector%20for%20Nuke.md) +- API handlers under [app/api](app/api) +- Server actions under [actions](actions) + +--- + +End of current-state report. diff --git a/app/(dashboard)/projects/[id]/shots/[shotId]/page.tsx b/app/(dashboard)/projects/[id]/shots/[shotId]/page.tsx index 9f4b32c..b473576 100644 --- a/app/(dashboard)/projects/[id]/shots/[shotId]/page.tsx +++ b/app/(dashboard)/projects/[id]/shots/[shotId]/page.tsx @@ -89,7 +89,7 @@ export default function ShotDetailPage() { const [isDuplicating, setIsDuplicating] = useState(false); const [isActioning, setIsActioning] = useState(false); const [highResDialogOpen, setHighResDialogOpen] = useState(false); - const [activeTab, setActiveTab] = useState<"tasks" | "reviews" | "footage" | "settings">("tasks"); + const [activeTab, setActiveTab] = useState<"tasks" | "reviews" | "footage" | "settings" | "exports">("tasks"); const [editingVersion, setEditingVersion] = useState(false); const [versionInput, setVersionInput] = useState(""); const [savingVersion, setSavingVersion] = useState(false); @@ -531,6 +531,18 @@ export default function ShotDetailPage() { Reviews +