fix(pipeline): let a finished render's preview outrank pending renders
Deploy / deploy (push) Successful in 6m11s
Deploy / deploy (push) Successful in 6m11s
Urgent already propagated — a preview inherits its render's priority, so an urgent export stays ahead all the way to QC. The complaint was the other half: at equal priority previews always ran last. The queue orders by priority then age, and a preview job is only created when its render finishes, so it is always newer than the renders queued beside it. Ten shots ran as ten renders and then ten previews, and nothing was reviewable until the end of the batch. render.previewPriorityBoost (default 1) subtracts from the render's priority when the preview is created, so a shot finishes end to end before the next render starts. The boost never promotes a job across the urgent threshold: a normal preview must not inherit permission to run outside the render window that its render never had. Set it to 0 for the old behaviour. Worth knowing separately: a worker will not claim preview jobs at all while an artist has AE open on that machine, which during the day is the larger reason previews appear to wait. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1131,6 +1131,14 @@ Cost note: the AE preview pass roughly doubles machine time per export. This mat
|
||||
|
||||
Finalize (E13) then creates the `Version` row server-side (Section 6.4) in pipeline mode — visible internally, never client-shared, no task/shot status changes.
|
||||
|
||||
### 9.3a Where a preview sits in the queue
|
||||
|
||||
The queue is ordered `priority ASC, createdAt ASC`, and a preview job is only created when its render *finishes* — so at equal priority it sorts behind every render queued alongside it. Ten shots queued together would run as ten renders and then ten previews, with nothing reviewable until the end.
|
||||
|
||||
`SystemConfig: render.previewPriorityBoost` (default 1) subtracts from the render's priority when the preview job is created, so a finished shot completes before the next render starts. Priority is otherwise inherited, which is what keeps an urgent export urgent all the way to READY_FOR_QC. The boost never promotes a job across `render.urgentPriorityThreshold`: a normal preview must not gain permission to run outside the render window that its render never had. Set the boost to `0` for the old phase behaviour.
|
||||
|
||||
Independently of priority, a worker **does not claim `PREVIEW_ONLY` jobs while an interactive After Effects is running** on that machine (§9.1) — a headless launch would hijack the artist's session. During the working day previews therefore queue regardless of rank, which is usually the larger effect on when they run.
|
||||
|
||||
### 9.4 Skipping the preview (`Export.skipPreview`)
|
||||
|
||||
Not every export is framed like its review. A deliverable rendered unscaled or without the punch-in shares a shot and a version history with the review, but a preview rebuilt from *its* frames would show the wrong picture — the earlier export's MOV is the correct one.
|
||||
|
||||
@@ -11,6 +11,14 @@ export const PIPELINE_CONFIG_DEFAULTS = {
|
||||
"render.maxAttempts": 3,
|
||||
"render.stallTimeoutSeconds": 600,
|
||||
"render.urgentPriorityThreshold": 20,
|
||||
/**
|
||||
* How far a preview job outranks the render it follows. A preview is only
|
||||
* created when its render finishes, so at equal priority it sorts behind
|
||||
* every render queued alongside it — 10 shots run as 10 renders then 10
|
||||
* previews, and nothing is reviewable until the end. A boost of 1 finishes
|
||||
* shots end to end instead. Set to 0 for the old phase behaviour.
|
||||
*/
|
||||
"render.previewPriorityBoost": 1,
|
||||
"render.outputRoot": "//SAN",
|
||||
// Validation stage (§8). Disabling it restores the pre-Phase-3 flow
|
||||
// (render → preview) without a redeploy, for a bad engine release.
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { previewPriority } from "./preview";
|
||||
|
||||
/**
|
||||
* A preview job is created when its render finishes, so it is always newer than
|
||||
* the renders queued beside it. With the queue ordered by priority then age,
|
||||
* equal priority means every pending render goes first — a batch of ten shots
|
||||
* runs as ten renders and then ten previews, and nothing is reviewable until
|
||||
* the end.
|
||||
*/
|
||||
describe("previewPriority", () => {
|
||||
const URGENT = 20;
|
||||
|
||||
it("outranks other work of the same priority so a shot finishes end to end", () => {
|
||||
expect(previewPriority(50, 1, URGENT)).toBe(49);
|
||||
});
|
||||
|
||||
it("keeps an urgent export's preview urgent", () => {
|
||||
// Urgent renders may claim outside the render window; their previews have
|
||||
// to be able to follow, or the shot stalls one step from being reviewable.
|
||||
expect(previewPriority(20, 1, URGENT)).toBe(19);
|
||||
expect(previewPriority(19, 1, URGENT)).toBeLessThanOrEqual(URGENT);
|
||||
});
|
||||
|
||||
it("never promotes a normal job across the urgent line", () => {
|
||||
// 21 - 1 would be 20, quietly granting the preview permission to run
|
||||
// outside the render window that its render never had.
|
||||
expect(previewPriority(21, 1, URGENT)).toBe(21);
|
||||
expect(previewPriority(25, 10, URGENT)).toBe(21);
|
||||
});
|
||||
|
||||
it("a boost of zero restores the old phase behaviour", () => {
|
||||
expect(previewPriority(50, 0, URGENT)).toBe(50);
|
||||
});
|
||||
|
||||
it("ignores a negative boost and never goes below zero", () => {
|
||||
expect(previewPriority(50, -5, URGENT)).toBe(50);
|
||||
expect(previewPriority(0, 5, URGENT)).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -177,6 +177,22 @@ export async function inheritPreviewMedia(
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* How a preview job is ranked against the render it follows.
|
||||
*
|
||||
* The preview is created when the render finishes, so with equal priority it
|
||||
* sorts behind every render queued at the same time — a batch runs as all the
|
||||
* renders, then all the previews, and nothing is reviewable until the end. The
|
||||
* boost lets a finished shot complete instead.
|
||||
*/
|
||||
export function previewPriority(renderPriority: number, boost: number, urgentThreshold: number): number {
|
||||
const boosted = renderPriority - Math.max(0, boost);
|
||||
// A boost must never promote a job across the urgent line: that would let it
|
||||
// claim outside the render window, which whoever queued it did not ask for.
|
||||
if (renderPriority > urgentThreshold && boosted <= urgentThreshold) return urgentThreshold + 1;
|
||||
return Math.max(0, boosted);
|
||||
}
|
||||
|
||||
/** Creates the PREVIEW_ONLY job that follows a successful EXR render. */
|
||||
export async function createPreviewJob(
|
||||
tx: Prisma.TransactionClient,
|
||||
@@ -185,13 +201,18 @@ export async function createPreviewJob(
|
||||
maxAttempts: number,
|
||||
priority: number
|
||||
) {
|
||||
const [boost, urgentThreshold] = await Promise.all([
|
||||
getConfigNumber("render.previewPriorityBoost"),
|
||||
getConfigNumber("render.urgentPriorityThreshold"),
|
||||
]);
|
||||
|
||||
return tx.renderJob.create({
|
||||
data: {
|
||||
type: "PREVIEW_ONLY",
|
||||
exportId: exportRow.id,
|
||||
attempt: 1,
|
||||
maxAttempts,
|
||||
priority,
|
||||
priority: previewPriority(priority, boost, urgentThreshold),
|
||||
manifest: manifest as unknown as Prisma.InputJsonValue,
|
||||
},
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user