Worker 1.7.0 (preview.highQuality, default on) now, after the AE render:
- encodes a High quality MP4 from the delivery MOV (full-res H.264 High,
CRF 14 capped at 100 Mbps) -> Version.hqFileUrl
- uploads the MOV itself -> Version.highResKey/Filename/Size
Both best-effort (the preview never fails over them), with a lease
keep-alive during the long encode/uploads; the export timeline records
what was attached.
Player: HQ switch when a High quality source exists; keeps frame, play
state, speed and zoom, remembered per browser, falls back to Standard if
the file won't load. ProRes itself stays download-only (browser support).
Downloads: the client portal's High Res button serves the version's
ProRes (client-code filename), falling back to the shot's manual file;
staff get a ProRes button on the review page.
Housekeeping (lib/render-pipeline/high-res.ts): HQ MP4 + ProRes are kept
while a version is latest, client-shared, approved or younger than
preview.highResRetentionDays (14); otherwise deleted from storage. Runs
per task after finalize and hourly from the reaper; deleting a version or
a reslate removes its files. The Standard MP4 is never touched.
Also: ProRes/Handles top-bar buttons go icon-only below lg so they don't
overlap the version selector.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The dashboard was a generic task-manager view of a studio that does not work
that way. Two of its five headline numbers counted Task.dueDate, which no
pipeline code writes - the only writer is a manual field on the task detail
page, so those cards read zero by construction. The rest were unscoped and
cumulative ("approved" across every project ever finished), the activity feed
was the notification table, and the shot queue sampled 15 rows by updatedAt,
a column the farm touches constantly. Nothing about failed exports, QC depth
or the render farm appeared at all, so a RENDER_FAILED was only visible if
someone thought to open the queue and filter for it.
Replaced with pipeline state, which the system maintains on its own:
- Blocked queue: the four failed statuses plus exports that stopped
progressing, each with the reason read off the actual cause - the failing
validation check, the QC note, the aerender error, or the machine that
claimed it and went quiet.
- Farm strip: machines, queue depth by job type, oldest unclaimed job, and
worker-version drift, since a worker's version decides which fixes it has.
- Five counters that can go down, scoped to live productions, each linking to
the rows it counted.
The queue page ignored ?status=, so the counters had nowhere to link; it now
takes an initial filter from the URL. Stat cards are hidden from CLIENT users
- the old ones were generic version counts, these are internal pipeline state.
Thresholds live in SystemConfig (dashboard.qcStaleHours, .stalledJobHours) so
they can be tuned without a redeploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Production fallout after enabling it: the trimmed EXR footage started
after the head handle as intended, but the overlay's burn-in timecode
(hardcoded to start at frame 1001) did not shift with it, so the
delivered burn-in no longer matched the footage. The picture-lock
reference render also never worked correctly in practice. Reverts the
AE panel checkbox, the preview-build script's trim/reference logic, and
the server/worker plumbing around it back to their pre-feature state
(82dfaee) — untrimmed previews, no picture-lock reference, overlay
timecode correct again.
The noPictureLock/piclockRefKey schema columns and their migration are
deliberately left in place: they're already applied in production, and
dropping an applied migration's file risks the next `prisma migrate
deploy`. They sit unused and harmless until this is revisited.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Previews used to ship the full padded (handle-included) render to
clients, with an artist manually pulling picture lock and hand-trimming
the comp to match. The automated preview stage now trims the client
comp to the picture-lock edit itself (a fixed handle count off each
end, per the studio's existing convention) and separately builds an
ungraded, full-duration picture-lock reference MP4 as a diagnostic
sidecar to check back against. A new "No Picture Lock" panel checkbox
is the single override for shots where picture lock is missing or
wrong, falling back to today's untrimmed behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Urgent already propagated — a preview inherits its render's priority, so
an urgent export stays ahead all the way to QC. The complaint was the
other half: at equal priority previews always ran last.
The queue orders by priority then age, and a preview job is only created
when its render finishes, so it is always newer than the renders queued
beside it. Ten shots ran as ten renders and then ten previews, and
nothing was reviewable until the end of the batch.
render.previewPriorityBoost (default 1) subtracts from the render's
priority when the preview is created, so a shot finishes end to end
before the next render starts. The boost never promotes a job across the
urgent threshold: a normal preview must not inherit permission to run
outside the render window that its render never had. Set it to 0 for the
old behaviour.
Worth knowing separately: a worker will not claim preview jobs at all
while an artist has AE open on that machine, which during the day is the
larger reason previews appear to wait.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Completes RenderPipeline2 phases 3-5. Additive throughout: with
validation.enabled=false the pipeline behaves exactly as before.
Validation (§8) runs on the worker straight after a render, while it
still holds the job and the files are local. Everything is read from the
EXR header — one 64 KB read per sampled frame, no pixel decode — so the
engine needs no oiiotool. The render job deliberately stays open through
the stage, so a worker that dies there is reaped and retried instead of
stranding the export in VALIDATING.
Timecode is the primary check and is anchored on the plate, never the
API: measured on the live show, Shot.timecodeStart is the editorial
in-point and sits one handle-length (8f) ahead of the plate, so checking
against it would pass the defective exports and fail the correct ones.
Reference precedence is manifest timing plate -> plate found on disk ->
API value, and the API rank is WARN-only. The panel now resolves plate
provenance by walking the comp's own footage layers, so the worker never
guesses which element carries the comp's timing.
Verified against real studio renders via the new --validate mode: it
catches both defect classes seen in shipped work — the +8 handle offset
and frame-derived timecode (00:00:41:17 = frame 1001 at 24fps).
QC (§10) is one service behind two front doors, API-key for the panel
and session for the web. PASS means ready to upload and nothing else;
FAIL sets the comp task to CHANGES. The panel gains a QC section that
imports the real EXR sequence at full res rather than the H.264.
Delivery (§11) rides the existing queue as a DELIVERY_BUILD job: hard
links with a copy fallback, verification by re-hashing delivered bytes
against the validation-time digest, manifest.json and the CSV report.
fsutil hardlink list is unavailable over SMB, so link identity is never
queried — reading the bytes back covers links and copies alike.
Also fixes ApiClient.ProgressAsync throwing on transport failure, which
would have aborted work in flight during a brief server outage; progress
reporting is best-effort by contract.
Removes the stale RenderWorker/ rule from the root .gitignore. The
worker's source is tracked like the rest of the repo and its own
.gitignore already covers bin/, obj/ and publish/; the rule silently
excluded new files while the existing 20 stayed tracked.
Not yet applied: migration 20260813000000 was hand-written because the
dev database was unavailable — run prisma migrate deploy and verify
before relying on it. None of this has run against a live server.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One "Queue Export" click now renders the EXR sequence, then rebuilds the shot
headlessly with the studio slate/overlay template to produce the delivery MOV
and review MP4. Implements RenderPipeline2 phases 1-2 plus the preview stage.
Server:
- New models Export, RenderJob, ExportEvent, Machine, WorkerHeartbeat, plus
Project.deliveryConfig and per-submission slate fields (Export.vfxScope,
Export.submissionNote, inherited from the shot's previous export).
Both migrations are purely additive; no existing column is touched.
- lib/render-pipeline: server-enforced state machine, transactional version
increment with supersede, atomic FOR UPDATE SKIP LOCKED claim gated by
machine availability windows, and a lease reaper run from instrumentation.ts.
- /api/ext/* endpoints for the panel and workers; session-auth mirrors under
/api/render and /api/machines for the web UI.
- Pipeline pages: render queue, export detail, machine monitoring, plus an
Exports tab on shot detail.
RenderWorker (.NET 8 Windows service, new):
- Registration, heartbeat as cancel channel, claim loop, aerender runner with
progress parsing and stall watchdog, crash recovery and disk-spooled
reporting that survives server downtime.
- Preview stage: headless AE assembles the preview comp into a throwaway AEP
with both output modules queued, then a single aerender pass renders them.
Preview jobs are not claimed while an interactive AE session is open, so an
artist's project is never taken over.
AE panel: Queue Export with live status polling, urgent flag, retry, and the
VFX Scope / Submission Note fields. Every existing panel action is unchanged.
Preview chaining ships disabled behind SystemConfig preview.enabled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
twotalesanimation