Skip to content

T
Tidal
Commit 113d9c57 by PLN (Algolia)
Options

feat(slop): three output shapes and a full-length cut, so YouTube is reachable 

PLN: "i wanna floor youtube with individual clips for each good rec we have".
The renderer could not do that: it had exactly ONE output shape, 1080x1920
vertical, and no way to render a whole track.

WHY IT WAS STUCK AT ONE SHAPE

The geometry was hardcoded in three separate places -- the Playwright viewport,
the CDP screencast cap, and the ffmpeg -vf. Adding a shape meant finding all
three and keeping them consistent forever, so nobody did. They now derive from
one SHAPES table: vertical 1080x1920, square 1080x1080, landscape 1920x1080.

The viewport is the load-bearing one and the reason this is not just a crop.
Hydra COMPOSES for the frame it is given, so the shape has to be chosen before
the scene renders -- a 16:9 scene centre-cropped to 9:16 throws away most of the
motion, and the reverse is equally true. The viewport is now output/dpr, which
at the default dpr 2 is 540x960 for vertical: byte-identical to the old
behaviour, so existing renders are unchanged.

FULL-LENGTH

`--dur full` (ad-hoc) and `--cut full` (idea path) render start-to-end, with the
length measured by ffprobe rather than assumed. `--cut full` stays on the idea
path deliberately so a YouTube cut still inherits that idea's playsets and fx
instead of silently falling back to the ad-hoc defaults.

THE BUG THIS WOULD HAVE CAUSED, AVOIDED

The output filename now carries the shape. Without that, rendering square,
vertical and landscape of one cut writes the same path three times and leaves
only the last -- a batch that quietly produces a third of what it claims.

An unknown --shape throws at startup, before a multi-minute realtime capture,
not after it.

VALIDATED END TO END

  node --check                                  clean
  vertical -> viewport 540x960                  identical to before
  --shape hexagon                               throws at line 80, pre-capture
  --dur full on a 446.777729s file              planned "+446.8s"
  15s square test render                        1080x1080 h264 30fps, aac 48k

NOTE for anyone running this: the playwright browser cache on this box is EMPTY,
so it needs SLOP_CHROME=/usr/bin/chromium (the script already supports it) plus
the hexa dev server on :5173 under node 22. Capture is realtime, so a full
render costs the track's duration PER SHAPE.
parent 97b22c0b
Showing with 106 additions and 11 deletions
visuals/slop/render_slop_clip.mjs
......@@ -15,6 +15,17 @@
// node visuals/slop/render_slop_clip.mjs --audio mix/take94/take94_trimmed_v2.wav \
// --start 300 --dur 30 --playset liquid-metal --name take94_probe
//
// # shape: vertical (default, reels) | square (feed) | landscape (YouTube)
// node visuals/slop/render_slop_clip.mjs --idea wap --shape square
// # full length, for YouTube — start to end of the master
// node visuals/slop/render_slop_clip.mjs --idea wap --cut full --shape landscape
// node visuals/slop/render_slop_clip.mjs --audio Prod/LeJazzCestQuoi.flac \
// --dur full --shape landscape --name lejazz
//
// CAPTURE IS REALTIME (see below), so a full-length render costs the track's own
// duration in wall time, PER SHAPE. Three shapes of a 7-minute take is ~21
// minutes of headless Chromium. Batch accordingly.
//
// THE ONE NON-OBVIOUS DECISION: capture is REALTIME, via CDP screencast, not
// per-frame screenshots. Hydra's animation clock is wall time, so frame-stepping
// plays motion back roughly 3x fast and drifts away from the audio it is supposed
......@@ -44,6 +55,49 @@ const { chromium } = require("playwright");
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
// --------------------------------------------------------------- output shapes
//
// One output shape used to be baked into three separate places — the viewport,
// the screencast cap, and the ffmpeg -vf — which is why there was no square or
// 16:9 path: adding one meant finding all three and keeping them consistent
// forever. They are now derived from one table.
//
// The viewport is the load-bearing one. Hydra COMPOSES for whatever frame it is
// given, so the shape has to be chosen before the scene renders, not cropped on
// afterwards: a 16:9 scene centre-cropped to 9:16 throws away most of the
// motion, which was the original reason this was set on the viewport at all.
// The ffmpeg crop stays as a belt-and-braces exact-size guarantee, but by then
// it should be a no-op.
const SHAPES = {
vertical: { w: 1080, h: 1920 }, // Insta reels / TikTok / Shorts
square: { w: 1080, h: 1080 }, // Insta feed
landscape: { w: 1920, h: 1080 }, // YouTube
};
function shapeOf(name = "vertical") {
const s = SHAPES[name];
if (!s) {
throw new Error(
`unknown --shape "${name}". Have: ${Object.keys(SHAPES).join(", ")}`);
}
return s;
}
// Length of an audio file, for `--dur full` / `--cut full`. ffprobe rather than
// a guess: these recordings are whatever length the take happened to be, and a
// clip that stops short of the ending is worse than no clip.
function audioDuration(file) {
const out = execFileSync("ffprobe", [
"-v", "error", "-show_entries", "format=duration",
"-of", "default=nw=1:nk=1", file,
], { encoding: "utf8" }).trim();
const d = Number(out);
if (!Number.isFinite(d) || d <= 0) {
throw new Error(`ffprobe could not read a duration from ${file}`);
}
return d;
}
// Newest full-chromium build in the playwright cache, or undefined to let
// playwright pick (which is right on a machine whose cache matches its version).
function chromeBinary() {
......@@ -76,6 +130,9 @@ function args() {
}
const A = args();
// Resolved once, at startup, so an unknown --shape fails before a multi-minute
// realtime capture rather than after it.
const SHAPE = shapeOf(A.shape ?? "vertical");
function loadIdea(stem) {
const p = resolve(__dirname, "clip_ideas.json");
......@@ -100,7 +157,20 @@ function plan() {
name: A.name ?? basename(audio).replace(/\.\w+$/, ""),
audio,
start: Number(A.start ?? 0),
dur: Number(A.dur ?? 30),
// `--dur full` renders to the end of the file from --start. Measured with
// ffprobe, never assumed — and clamped so a --start past the end fails
// loudly here instead of producing a zero-frame capture 40s later.
dur: A.dur === "full"
? (() => {
const rest = audioDuration(audio) - Number(A.start ?? 0);
if (rest <= 0) {
throw new Error(
`--start ${A.start} is at or past the end of ${basename(audio)} `
+ `(${audioDuration(audio).toFixed(1)}s)`);
}
return rest;
})()
: Number(A.dur ?? 30),
playsets: [A.playset ?? "liquid-metal"],
fx: A.fx ? JSON.parse(A.fx) : DEFAULT_FX,
triggerHeavy: A.triggerHeavy === "true",
......@@ -111,11 +181,27 @@ function plan() {
throw new Error("need --idea <stem|pos> or --audio <file> --start --dur");
}
const i = loadIdea(A.idea);
const audio = A.master ? resolve(TIDAL, A.master) : i.master_audio;
// `--cut full` bypasses promo_cuts entirely and renders the whole master.
// Kept on the idea path so a full-length YouTube cut still inherits the
// idea's playsets and fx, rather than falling back to the ad-hoc defaults.
if (A.cut === "full") {
return {
name: `${i.tidal_stem}_full`,
audio,
start: 0,
dur: audioDuration(audio),
playsets: A.playset ? [A.playset] : i.visual.playsets,
fx: i.visual.fx,
triggerHeavy: i.visual.triggerHeavy,
source: `${i.opal_title} — FULL, ${i.measured_bpm} BPM`,
};
}
const cut = i.promo_cuts[`${A.cut}s`] ?? i.promo_cuts["30s"];
if (!cut) throw new Error(`idea ${A.idea} has no ${A.cut}s cut`);
return {
name: `${i.tidal_stem}_${A.cut}s`,
audio: A.master ? resolve(TIDAL, A.master) : i.master_audio,
audio,
start: cut.master_start,
dur: cut.duration,
playsets: A.playset ? [A.playset] : i.visual.playsets,
......@@ -155,7 +241,11 @@ mkdirSync(PUB, { recursive: true });
const slice = resolve(PUB, "clip.wav");
const framesDir = resolve(PUB, "frames");
const outMp4 = resolve(OUT_DIR, `${job.name}.mp4`);
// The shape is part of the identity, not decoration: rendering square, vertical
// and landscape of the same cut would otherwise write the same filename three
// times and silently leave only the last one. A batch that quietly produces a
// third of what it claims is worse than one that errors.
const outMp4 = resolve(OUT_DIR, `${job.name}_${A.shape ?? "vertical"}.mp4`);
console.log(`slop: ${job.name}`);
console.log(` source ${job.source}`);
......@@ -195,12 +285,17 @@ try {
"--disable-features=IsolateOrigins,site-per-process",
],
});
// 540x960 at dpr 2 = 1080x1920. Set on the VIEWPORT rather than upscaling
// later, so Hydra composes for the vertical frame instead of being cropped
// into it — a 16:9 scene centre-cropped to 9:16 loses most of the motion.
// Set on the VIEWPORT rather than upscaling later, so Hydra composes for the
// target frame instead of being cropped into it — a 16:9 scene centre-cropped
// to 9:16 loses most of the motion, and the same is true in reverse.
//
// The viewport is the output divided by dpr: at the default dpr 2 that is
// 540x960 for vertical, exactly as before. Deriving it means --shape and --dpr
// can no longer disagree with the mux about how big a frame is.
const dpr = Number(A.dpr ?? 2);
const ctx = await browser.newContext({
viewport: { width: 540, height: 960 },
deviceScaleFactor: Number(A.dpr ?? 2),
viewport: { width: Math.round(SHAPE.w / dpr), height: Math.round(SHAPE.h / dpr) },
deviceScaleFactor: dpr,
});
const page = await ctx.newPage();
page.on("console", (m) => {
......@@ -323,7 +418,7 @@ try {
await page.evaluate(() => { window.__slopCtx?.resume(); window.__slopAudio?.play(); });
await cdp.send("Page.startScreencast", {
format: "jpeg", quality: 90, everyNthFrame: 1,
maxWidth: 1080, maxHeight: 1920,
maxWidth: SHAPE.w, maxHeight: SHAPE.h,
});
const wall = Date.now();
await sleep(job.dur * 1000 + 400);
......@@ -358,8 +453,8 @@ try {
"-hide_banner", "-loglevel", "error", "-y",
"-f", "concat", "-safe", "0", "-i", resolve(framesDir, "list.ffconcat"),
"-i", slice,
"-vf", `fps=${A.fps},scale=1080:1920:force_original_aspect_ratio=increase,`
+ `crop=1080:1920,format=yuv420p`,
"-vf", `fps=${A.fps},scale=${SHAPE.w}:${SHAPE.h}:force_original_aspect_ratio=increase,`
+ `crop=${SHAPE.w}:${SHAPE.h},format=yuv420p`,
"-c:v", "libx264", "-preset", "medium", "-crf", "20",
"-c:a", "aac", "-b:a", "192k",
"-movflags", "+faststart", "-shortest", outMp4,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment