feat(pipeline): render queue, worker service and automated preview generation

One "Queue Export" click now renders the EXR sequence, then rebuilds the shot
headlessly with the studio slate/overlay template to produce the delivery MOV
and review MP4. Implements RenderPipeline2 phases 1-2 plus the preview stage.

Server:
- New models Export, RenderJob, ExportEvent, Machine, WorkerHeartbeat, plus
  Project.deliveryConfig and per-submission slate fields (Export.vfxScope,
  Export.submissionNote, inherited from the shot's previous export).
  Both migrations are purely additive; no existing column is touched.
- lib/render-pipeline: server-enforced state machine, transactional version
  increment with supersede, atomic FOR UPDATE SKIP LOCKED claim gated by
  machine availability windows, and a lease reaper run from instrumentation.ts.
- /api/ext/* endpoints for the panel and workers; session-auth mirrors under
  /api/render and /api/machines for the web UI.
- Pipeline pages: render queue, export detail, machine monitoring, plus an
  Exports tab on shot detail.

RenderWorker (.NET 8 Windows service, new):
- Registration, heartbeat as cancel channel, claim loop, aerender runner with
  progress parsing and stall watchdog, crash recovery and disk-spooled
  reporting that survives server downtime.
- Preview stage: headless AE assembles the preview comp into a throwaway AEP
  with both output modules queued, then a single aerender pass renders them.
  Preview jobs are not claimed while an interactive AE session is open, so an
  artist's project is never taken over.

AE panel: Queue Export with live status polling, urgent flag, retry, and the
VFX Scope / Submission Note fields. Every existing panel action is unchanged.

Preview chaining ships disabled behind SystemConfig preview.enabled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
twotalesanimation
2026-08-02 14:34:34 +02:00
parent 0d0f3e1a33
commit ae58dc0366
71 changed files with 11767 additions and 1 deletions
@@ -0,0 +1,80 @@
using System.Text.RegularExpressions;
namespace VFXReviewWorker;
/// <summary>
/// Parses aerender stdout line-by-line (§7.3): PROGRESS lines yield the
/// current frame; ERROR lines are collected; deterministic errors (missing
/// footage/comp/project) mark the job non-retryable — humans fix the comp.
/// </summary>
public static partial class ProgressParser
{
// e.g. "PROGRESS: 0:00:02:03 (51): 0 Seconds" / "PROGRESS: 2 (2): ..."
[GeneratedRegex(@"^PROGRESS:.*\((\d+)\)", RegexOptions.Compiled)]
private static partial Regex ProgressLine();
[GeneratedRegex(@"aerender\s+(SYNTAX\s+)?ERROR|^ERROR:", RegexOptions.Compiled | RegexOptions.IgnoreCase)]
private static partial Regex ErrorLine();
private static readonly string[] NonRetryablePatterns =
{
"missing footage",
"layer source file missing",
"file is missing",
"no comp was found",
"no composition",
"project file could not be opened",
"can not be opened",
"cannot be opened",
"the file format module could not parse the file",
"no render settings template",
"no output module template",
"syntax error",
"illegal argument",
};
/// <summary>Returns the rendered-frame ordinal (1-based within the range) or null.</summary>
public static int? ParseFrame(string line)
{
var m = ProgressLine().Match(line);
return m.Success && int.TryParse(m.Groups[1].Value, out var f) ? f : null;
}
public static bool IsErrorLine(string line) => ErrorLine().IsMatch(line);
/// <summary>Deterministic failures re-render identically — don't burn retries on them.</summary>
public static bool IsNonRetryableError(string line)
{
foreach (var p in NonRetryablePatterns)
{
if (line.Contains(p, StringComparison.OrdinalIgnoreCase)) return true;
}
return false;
}
}
/// <summary>ETA = rolling average seconds/frame × frames remaining (§7.3).</summary>
public sealed class EtaCalculator
{
private readonly int _windowSize;
private readonly Queue<double> _frameSeconds = new();
private DateTimeOffset? _lastFrameAt;
public EtaCalculator(int windowSize = 10) => _windowSize = windowSize;
public void RecordFrame(DateTimeOffset now)
{
if (_lastFrameAt is { } last)
{
_frameSeconds.Enqueue((now - last).TotalSeconds);
while (_frameSeconds.Count > _windowSize) _frameSeconds.Dequeue();
}
_lastFrameAt = now;
}
public int? EstimateSeconds(int framesRemaining)
{
if (_frameSeconds.Count == 0 || framesRemaining <= 0) return null;
return (int)Math.Round(_frameSeconds.Average() * framesRemaining);
}
}