feat(pipeline): render queue, worker service and automated preview generation

One "Queue Export" click now renders the EXR sequence, then rebuilds the shot
headlessly with the studio slate/overlay template to produce the delivery MOV
and review MP4. Implements RenderPipeline2 phases 1-2 plus the preview stage.

Server:
- New models Export, RenderJob, ExportEvent, Machine, WorkerHeartbeat, plus
  Project.deliveryConfig and per-submission slate fields (Export.vfxScope,
  Export.submissionNote, inherited from the shot's previous export).
  Both migrations are purely additive; no existing column is touched.
- lib/render-pipeline: server-enforced state machine, transactional version
  increment with supersede, atomic FOR UPDATE SKIP LOCKED claim gated by
  machine availability windows, and a lease reaper run from instrumentation.ts.
- /api/ext/* endpoints for the panel and workers; session-auth mirrors under
  /api/render and /api/machines for the web UI.
- Pipeline pages: render queue, export detail, machine monitoring, plus an
  Exports tab on shot detail.

RenderWorker (.NET 8 Windows service, new):
- Registration, heartbeat as cancel channel, claim loop, aerender runner with
  progress parsing and stall watchdog, crash recovery and disk-spooled
  reporting that survives server downtime.
- Preview stage: headless AE assembles the preview comp into a throwaway AEP
  with both output modules queued, then a single aerender pass renders them.
  Preview jobs are not claimed while an interactive AE session is open, so an
  artist's project is never taken over.

AE panel: Queue Export with live status polling, urgent flag, retry, and the
VFX Scope / Submission Note fields. Every existing panel action is unchanged.

Preview chaining ships disabled behind SystemConfig preview.enabled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
twotalesanimation
2026-08-02 14:34:34 +02:00
parent 0d0f3e1a33
commit ae58dc0366
71 changed files with 11767 additions and 1 deletions
@@ -0,0 +1,57 @@
import { NextRequest, NextResponse } from "next/server";
import { randomUUID } from "crypto";
import { isExtAuthorized } from "@/lib/render-pipeline/ext-auth";
import { toErrorResponse, PipelineError } from "@/lib/render-pipeline/errors";
import { db } from "@/lib/db";
import { generateHetznerPresignedUploadUrl, sanitizeFileName } from "@/lib/storage";
// ── POST /api/ext/render/jobs/{jobId}/artifact-presign (E20) ─────────────────
//
// Presigned upload URL for worker artifacts. Kinds map to the existing object
// storage folder layout: preview → videos/, thumbnail → image/,
// metadata/log → renders/.
const KIND_FOLDERS: Record<string, string> = {
preview: "videos",
thumbnail: "image",
metadata: "renders",
log: "renders",
};
export async function POST(
req: NextRequest,
{ params }: { params: Promise<{ jobId: string }> }
) {
if (!isExtAuthorized(req)) {
return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
}
const { jobId } = await params;
let body: { machineId?: string; kind?: string; fileName?: string; contentType?: string };
try {
body = await req.json();
} catch {
return NextResponse.json({ error: "Invalid JSON" }, { status: 400 });
}
try {
if (!body.machineId) throw new PipelineError(400, "machineId is required");
if (!body.fileName) throw new PipelineError(400, "fileName is required");
if (!body.contentType) throw new PipelineError(400, "contentType is required");
const folder = KIND_FOLDERS[body.kind ?? ""];
if (!folder) {
throw new PipelineError(422, `kind must be one of: ${Object.keys(KIND_FOLDERS).join(", ")}`);
}
const job = await db.renderJob.findUnique({ where: { id: jobId }, select: { machineId: true } });
if (!job) throw new PipelineError(404, "Render job not found");
if (job.machineId !== body.machineId) {
throw new PipelineError(409, "Job is not held by this machine (lease reassigned?)");
}
const key = `${folder}/${randomUUID()}-${sanitizeFileName(body.fileName)}`;
const presignedUrl = await generateHetznerPresignedUploadUrl(key, body.contentType);
return NextResponse.json({ presignedUrl, key, url: `/api/files/${key}` });
} catch (err) {
const { body: errBody, status } = toErrorResponse(err);
return NextResponse.json(errBody, { status });
}
}