Skip to content

T
Tidal
Commit ecfa108c by PLN (Algolia)
Options

docs(foundry): API integration guide + phase-2 synthesis & research 

- README: 'Use as an API' — Python import / CLI subprocess / HTTP API, the
  path+catch.json contract, env overrides (for sibling tools e.g. the #89
  emotional-arc tool that consumes the demucs engine for stems)
- README: add 'link' to CLI reference
- research/{A,B,C}_*.md: the three strand reports (empirical profile,
  fundamentals rulebook, prior-art + recommended zero-dep stack)
- research/PHASE2_FINDINGS.md: strand-D synthesis — corrected per-stem model
  (loops cut per stem, often-but-not-always time-aligned), loopability rubric,
  Take/StemSlice model, correlation engine (§5·0 GT-at-scale), Hall of Fame
  tierlist, section-label grounding (↔ #89), 80/20 build order
- PHASE2_LOOPFINDER.md charter: grounded in the real flow + ground-truth pairs
parent 20bc875e
Showing with 1489 additions and 0 deletions
tools/foundry/README.md
......@@ -36,6 +36,7 @@ foundry.py sep <slug|url> # separate an existing catch (or fetch+sep a url)
foundry.py catch <url> # fetch + sep (the whole pipeline)
foundry.py list # ▣ separated / ▢ raw, per catch
foundry.py backends # ✓/✗ + hint per backend
foundry.py link <kit> # symlink Samples/<kit> → Dirt-Samples (s "<kit>")
foundry.py serve [--port] # launch the GUI
# separation knobs (sep / catch):
......@@ -56,6 +57,86 @@ cd tools/foundry && python3 -m pytest tests/ -q # pure engine: slug, parse, ar
---
## Use as an API (from another project)
The Foundry is the shared **demucs engine** for the wider toolbox — e.g. the
emotional-arc / section-analysis tool consumes it to get stems. Three ways to
integrate, cheapest first:
### 1. Python import (in-process — best for a sibling Python tool)
Add the foundry dir to `sys.path`, then call the same steps the CLI does. Runs
under system python3 (needs `pydantic`); the demucs subprocess uses its own venv.
```python
import sys; sys.path.insert(0, "/home/pln/Work/Sound/Tidal/tools/foundry")
from pathlib import Path
from engine import fetch, separate
from engine.model import Catch
ws = Path("~/Downloads/foundry").expanduser() # or set $FOUNDRY_HOME
catch = fetch.fetch("https://youtu.be/…", ws) # download → Catch
catch = separate.separate(catch, ws, backend="demucs", shifts=4,
progress=lambda p: print(f"{p}%"))
# stems are now on disk; resolve absolute paths from the Catch:
stems = {s.name: catch.dir(ws) / s.path for s in catch.stems}
bass_wav = stems["bass"] # → feed your analysis
# already have a catch? skip fetching:
catch = Catch.load(ws / "some_slug") # reads catch.json
```
Idempotency: `fetch` re-probes but re-downloads only if missing; check
`catch.separated` before re-separating. `separate.REGISTRY` lists backends;
`separate.get_backend("demucs").available()` → `(bool, hint)`.
### 2. CLI subprocess (language-agnostic)
```python
import subprocess, json
from pathlib import Path
F = "/home/pln/Work/Sound/Tidal/tools/foundry/foundry.py"
subprocess.run(["python3", F, "catch", url], check=True) # fetch + separate
# then read the catch.json the run wrote:
ws = Path("~/Downloads/foundry").expanduser()
slug = sorted(ws.iterdir())[-1].name # or derive from title
catch = json.loads((ws / slug / "catch.json").read_text())
```
### 3. HTTP API (for a running server / remote / a non-Python caller)
`python3 foundry.py serve` then poll jobs:
```
POST /api/fetch {url} → {id} # start download job
POST /api/separate {slug,backend,shifts} → {id} # start separation job
GET /api/job/<id> → {state,progress,msg,slug}
GET /api/catches → [Catch, …]
GET /api/backends → [{name,available,hint}]
GET /media/<slug>/<path> → audio (HTTP Range / 206 — seekable)
```
`state` goes `running → done|error`; on `done`, read `/api/catches` or
`/media/<slug>/stems/bass.wav`.
### The contract you can rely on
- Stems always land at `<workspace>/<slug>/stems/{drums,bass,other,vocals}.wav`
(normalized regardless of backend), 44.1 kHz WAV.
- `catch.json` is the source of truth (title, `source_url`, `backend`, `model`,
`stems[]`, `duration`) — parse it, don't scrape filenames.
- Override paths via env: `$FOUNDRY_HOME` (workspace), `$FOUNDRY_DEMUCS` (binary),
`$FOUNDRY_SAMPLES` / `$FOUNDRY_DIRT_SAMPLES` (publish targets).
- **Don't infer a stem's musical role from its name** (`other`≠melody,
`bass`≠sub) — analyze it. (Project rule; see CLAUDE.md.)
> Beta-testers: file what's awkward. The Python import path is the one to harden
> first — tell me if you'd rather have a `pip install -e`-able package.
---
## Engine roadmap — engine1 → engine2
The separation backend is a **registry** (`engine/separate.py`); the CLI/GUI ask
......
tools/foundry/research/A_loop_profile.md 0 → 100644
# Strand A — Empirical Loop Profile
> Status: **complete** · 2026-06-22
> Grounding: 135 kits / 1641 wavs + 3 ground-truth pairs (crimewave 24, diams_dj 16, humpty 9) + 702 .tidal files
---
## 1. Kit Population (135 kits, 1641 wavs)
| Metric | Value |
|---|---|
| Total kits | 135 |
| Total wav files | 1641 |
| Mean wavs / kit | 12.2 |
| Median wavs / kit | 6.0 |
| Stdev | 15.4 |
| Min / Max | 0 / 76 |
**Kit-size histogram:**
| Wavs per kit | Kits | % |
|---|---|---|
| 1 | 4 | 3% |
| 2–5 | 59 | 44% |
| 6–10 | 15 | 11% |
| 11–20 | 23 | 17% |
| 21–50 | 30 | 22% |
| 51+ | 4 | 3% |
Half the kits are small (2–5 wavs) — mini melodic or textural palettes.
The 22% in 21–50 and the 4 kits at 51+ are mature "played" kits with full role coverage.
**Kit usage in live .tidal files:**
| Usage | Kits |
|---|---|
| 0 files (unused / staged) | 29 |
| 1 file | 37 |
| 2–5 files | 35 |
| 6–20 files | 25 |
| 21+ files (core palette) | 9 |
106 of 135 kits appear in at least one .tidal file.
Mean 6.9 files / kit (among used kits); long tail: 9 "core palette" kits referenced 21+ times.
---
## 2. Duration Bimodality
The corpus is bimodal: one-shots (drums, FX, stabs) vs loops (harmonic/rhythmic content).
| Mode | Count | % | Duration range |
|---|---|---|---|
| One-shots (<2 s) | 187 | 11.4% | 18 ms – 1996 ms (mean 971 ms, median 867 ms) |
| Loops (≥2 s) | 1454 | 88.6% | 2.0 s – 531 s (mean 9.3 s, median 5.3 s) |
**Loop duration fine histogram:**
| Duration | Count | % | Interpretation @ 120–128 bpm |
|---|---|---|---|
| 2–3 s | 136 | 9.4% | < 1 bar (phrase fragment) |
| 3–4 s | 159 | 10.9% | ≈1–2 bar |
| **4–5 s** | **340** | **23.4%** | **≈2 bars @ 120–128 bpm — most common** |
| 5–6 s | 168 | 11.6% | 2–3 bars |
| 6–7 s | 150 | 10.3% | 3 bars |
| **7–8 s** | **154** | **10.6%** | **≈4 bars @ 120–128 bpm — second peak** |
| 8–9 s | 53 | 3.6% | 4+ bars |
| 9–10 s | 74 | 5.1% | ≈4–5 bars |
| 10–15 s | 120 | 8.3% | 5–8 bars |
| 15–30 s | 55 | 3.8% | 8–16 bars |
| 30+ s | 45 | 3.1% | full-section stems |
**Canonical bar-lengths at ParVagues tempos:**
| BPM | 2 bars | 4 bars | 8 bars |
|---|---|---|---|
| 120 | 4.00 s | 8.00 s | 16.00 s |
| 125 | 3.84 s | 7.68 s | 15.36 s |
| 128 | 3.75 s | 7.50 s | 15.00 s |
The two dominant duration peaks (4–5 s and 7–8 s) map directly to **2-bar and 4-bar loops at 120–128 bpm**.
The loop finder should try candidate lengths of [2, 4, 8] bars first.
---
## 3. Ground-Truth Pair Analysis
### crimewave (24 loops)
- Sources: 3 versions (Crystal Castles original ≈ 5.3 min; LZRTAG remix ≈ 4.3 min; LA Riots remix ≈ 4.8 min)
- Loop lengths: 3 groups by exact duration, corresponding to 3 source tempos:
- 7.88 s → 121.8 bpm (Crystal Castles original): 11 loops — synth variants, drum variants, bass variants, vocal lines
- 7.55 s → 127.2 bpm (LZRTAG remix): 4 loops — melodic/synth elements
- 7.47 s → 128.5 bpm (LA Riots remix): 9 loops — vocal phrase variations
- Kept: ~184 s across all 3 sources
- Coverage: comparing to longest source (315 s), kept 58% of raw source time — but that's misleading; loops repeat the same ~2–3 bar-windows from each version, with all stems exported per moment.
**Spectral profile (24 loops):**
| Feature | Mean | Stdev |
|---|---|---|
| RMS level | −19.2 dBFS | ±5.5 dB |
| Peak level | −5.0 dBFS | ±4.1 dB |
| Spectral centroid | 1159 Hz | ±1247 Hz (wide — all stems together) |
| Onset rate | 3.9 /s | ±1.4 |
| % percussive | 25% | ±28% |
| HNR | 9.2 | ±9.0 (wide — spans harmonic/noisy) |
| Tempo (librosa est.) | unstable (beat_track unreliable on short loops) | — |
### diams_dj (16 loops)
- Source: Diam's — DJ (4.5 min, 272 s)
- Loop length: all 7.8347 s exactly → 122.6 bpm, 4-bar cuts
- All 16 loops are from **one selected bar-window** in the source
- 16 elements = 9 vocal phrases (different lyrics/moments) + 7 guitar riff positions
- Kept: 125 s from 272 s = 46% coverage
**Spectral profile:**
| Feature | Mean | Stdev |
|---|---|---|
| RMS level | −18.3 dBFS | ±2.4 dB |
| Peak level | −1.5 dBFS | ±1.2 dB |
| Spectral centroid | 1246 Hz | ±643 Hz |
| Onset rate | 4.5 /s | ±1.1 |
| % percussive | 31% | ±13% |
### humpty (9 loops)
- Source: Humpty Dumpty (2.6 min, 154.6 s)
- Loop lengths: 2 groups:
- 4.784 s → 100.3 bpm: 2 elements (bass + keys)
- 4.693 s → 102.3 bpm: 7 elements (keys + brass + modal variations)
- 2 cut positions, all melodic — **no drums exported**
- Kept: 42 s from 154 s = 27% coverage
**Spectral profile:**
| Feature | Mean | Stdev |
|---|---|---|
| RMS level | −18.5 dBFS | ±3.2 dB |
| Peak level | −5.0 dBFS | ±2.8 dB |
| Spectral centroid | 663 Hz | ±237 Hz (bass-heavy / low-mid kit) |
| Onset rate | 3.4 /s | ±0.6 |
| % percussive | 9% | ±4% (highly harmonic) |
| HNR | 14.6 | ±13.1 (more harmonic than crimewave) |
---
## 4. Level / Loudness Profile (broad corpus sample, 48 files from 40 kits)
| Metric | Value |
|---|---|
| RMS mean | −23.8 dBFS |
| RMS median | −21.3 dBFS |
| RMS stdev | 8.3 dB |
| Peak mean | −6.5 dBFS |
| Peak median | −4.9 dBFS |
| RMS bucket | % |
|---|---|
| −6 to −12 dBFS | 4% |
| −12 to −18 dBFS | 19% |
| −18 to −24 dBFS | 33% |
| < −24 dBFS | 44% |
Loops tend to be **moderate-to-low level** (demucs stems are often quieter than the mix).
The loop finder does NOT need to normalize to a target; SuperDirt + gain pattern handles level shaping.
---
## 5. Spectral Character by Stem Role (broad corpus)
| Spectral centroid bucket | % of samples |
|---|---|
| < 500 Hz (bass register) | 8% |
| 500–2000 Hz (low-mid, harmonic core) | 54% |
| 2000–5000 Hz (mid, present) | 35% |
| > 5000 Hz (hi) | 2% |
Median centroid: 1691 Hz — the corpus skews **low-mid / harmonic**. High-centroid material (cymbals, noise) is a minority.
**Comparison across ground-truth kits by centroid:**
- crimewave: 1159 Hz (low-mid, mixed content)
- diams_dj: 1246 Hz (low-mid, guitar-led)
- humpty: 663 Hz (bass/low — pure harmonic kit)
---
## 6. The PLN Cutting Workflow (reverse-engineered)
Synthesized from exact duration clustering and filename analysis across all 3 ground-truth kits:
```
SOURCE TRACK
│
▼
[A] LISTEN → identify 1-4 "hook moments" (chorus, drop, breakdown, signature riff)
│ Typically 2-4 moments per 4-5 min track
▼
[B] SELECT → per moment: choose a 2-bar or 4-bar region, snapped to bar grid
│ Bar alignment is implied by loopAt usage in .tidal (loops play in time)
▼
[C] EXPORT → per region: export all 4 demucs stems (drums/bass/other/vocals)
│ Additional sub-splits if a stem has distinct phrases
▼
[D] LABEL → by semantic role + character, NOT by stem name
(e.g. "bassline1", "vocals_dark_lips", "synth_explose", "keys_brass3_MINOR")
▼
RESULT: 4–12 loops per "moment" × 1-4 moments = 4–48 loops/kit (observed: 9–24)
```
**What the human selects for (subjective, machine cannot decide):**
- The SPECIFIC bar-window with the best vibe / energy contrast
- Whether to export drums at all (humpty: skipped entirely)
- Which sub-phrases within a stem are distinct enough to keep separately
- Naming and character annotation
**What is mechanical and consistent (machine CAN automate):**
- Bar-grid alignment of cut endpoints
- Duration consistency within a source (all loops from same source version = same duration)
- Minimum duration: ≥ 2 bars (never a one-beat stab in loop context)
- Maximum practical duration: ≤ 8 bars for most content (98% < 15 s)
---
## 7. How Loops Are Played in TidalCycles
From analysis of the two confirmed ground-truth tracks:
**`vague_de_crime.tidal` (crimewave, 120 bpm):**
- `setcps (120/60/4)` = 120 bpm
- `loopAt 4`: stretches loop to exact 4 bars at current tempo
- `slice 16 (slow 4 $ run 16)`: chops loop into 16 slices, plays in order
- `chop 4` / `slice 4 "<0 1 2 3>"`: structured playback over 4 beats
- All 24 loop indices (0–23) are referenced across channels
**`bombe_dj.tidal` (diams_dj, 124 bpm):**
- `setcps (124/60/4)` = 124 bpm
- `loopAt 4` on 7.83 s loops = at 124 bpm, 4 bars = 7.74 s → 1.1% stretch (transparent)
- `slow 4 $ n "<9 10 11 12 …>"`: guitar riffs cycled over 4-bar phrases
- `slice 4 "<0 1 2 3>"` + `loopAt 4`: structured chop-and-play pattern
**Key implications for the loop finder:**
- `loopAt` tolerates ~1–5% timing error; bar-snapping at detected tempo is sufficient
- The loop finder's primary constraint is **bar boundary alignment**, not sample-perfect timing
- Loops are often chopped (`slice`, `chop`), so **loop start/end quality matters less than internal structure** — but a click at the seam is still audible if `loopAt` stretching fails
---
## 8. Key Findings Summary
### The target loop profile
A "ParVagues loop" is:
- **Duration**: 2-bar or 4-bar at 100–130 bpm → **3.7–8.0 s** for the dominant 85% of loop content
- **Level**: −12 to −28 dBFS RMS (moderate; demucs stems are not normalized)
- **Spectral**: skews low-mid (500–2000 Hz centroid), harmonic rather than noisy
- **Source coverage**: PLN keeps 27–58% of source duration, selecting 1–4 "moments"
- **Per-moment export**: 4–12 elements (all demucs stems + phrase variants) at same bar-window
- **Naming**: semantic role + energy character (not "drums.wav")
### The single most actionable finding for the loop-finder design
**Duration clusters within a kit = strong prior for bar-length detection.**
All loops from a single source version have bit-identical durations (±20 ms, ~0.3% — rounding in Audacity export). This means: if the loop finder detects a bar-length candidate (e.g. 7.83 s ≈ 4 bars at 122.6 bpm), it can **validate that tempo estimate against the actual loop file duration** with very high confidence. Conversely, the loop finder should output one candidate per (bar-length × source-tempo) combination, not per stem — because PLN expects to see the 4 stems of a bar-window as a coherent unit, not as 4 independent candidates.
**The finder's ranking target is not "best loop in isolation" but "best bar-window to extract all stems from."**
tools/foundry/research/B_fundamentals.md 0 → 100644
# Strand B — Sampling & Looping Fundamentals Rulebook
> **Charter context:** This rulebook governs the mechanical layer of the Foundry
> loop-finder (Phase 2). It covers everything the machine is allowed to decide
> autonomously: seam physics, bar alignment, stem-type policies, and SuperDirt
> file conventions. Human creative judgment is explicitly out of scope here —
> see the design philosophy table in `PHASE2_LOOPFINDER.md`.
>
> **Critical constraint throughout:** source material is livecoded (TidalCycles,
> no click track). Tempo drifts within and between tracks. Every rule that would
> assume a fixed global BPM is annotated with [DRIFT HAZARD].
---
## Section 1 — Seam Physics: Making the Boundary Inaudible
### 1.1 Zero-Crossing Alignment
**Rule B-1.1a — Both endpoints must land on a zero crossing.**
A "click" at a loop seam is an amplitude discontinuity: the waveform at the
end of loop iteration N abruptly jumps to a different amplitude at the start of
iteration N+1. The physical cause is the digital sample value changing
instantaneously by an amount audible as a broadband click. Snapping both the
loop-out and loop-in markers to zero-amplitude samples eliminates the worst-case
discontinuity.
> "Do look for crossings in the same direction (i.e. negative to positive) on
> both ends to help reduce slope discontinuity."
> — KVR Audio DSP Forum, "Avoiding Clicks In Loops" [1]
**Rule B-1.1b — Direction-matched zero crossings only.**
Snap to a zero crossing where the waveform slope is in the same direction on
both the loop-out and loop-in marker (both ascending, or both descending). A
slope mismatch at zero amplitude still produces an audible first-derivative
discontinuity (a softer pop, but audible in quiet material).
**Rule B-1.1c — Search window of ±10 ms around the candidate beat position.**
At 44.1 kHz that is ±441 samples — large enough to find a matching zero
crossing in most material, small enough not to wander into the next beat.
Tighten to ±5 ms (221 samples) for drums where transients are dense. If no
zero crossing is found in the window, fall back to Rule B-1.4 (crossfade).
Implementation sketch (Python/numpy):
```python
# find nearest zero crossing to sample index `pos`, within `window` samples
def nearest_zc(signal, pos, window=441, direction=None):
lo = max(0, pos - window)
hi = min(len(signal) - 1, pos + window)
indices = np.where(
(np.diff(np.sign(signal[lo:hi])) != 0)
)[0] + lo
if direction is not None:
# filter by slope sign
indices = [i for i in indices
if np.sign(signal[i+1] - signal[i]) == direction]
if not indices:
return pos # no crossing found — caller must crossfade
return indices[np.argmin(np.abs(np.array(indices) - pos))]
```
### 1.2 DC Offset Removal
**Rule B-1.2a — Remove DC offset before any loop-point search.**
DC offset shifts the entire waveform away from zero amplitude. A sample with
even a small DC component has no true zero crossings; every apparent zero
crossing is actually a non-zero sample. Result: every loop seam clicks or pops.
> "Playing back a sound effect with DC offset will produce a click at the
> beginning or end of the file. Since the waveform of a problematic DC offset
> file is shifted, the audio no longer intersects with the zero crossing line."
> — Digido.com, "DC Offset: Clicks and Pops" [2]
> "If you have a DC offset problem that's large enough to cause an audible pop,
> the most successful method is to apply an extremely-high quality linear phase
> high pass filter."
> — Digido.com [2]
**Rule B-1.2b — Apply a gentle linear-phase high-pass filter at ≤10 Hz.**
A very-low-cutoff HP filter removes DC and near-DC content without audibly
affecting bass frequencies.
> "Every highpass filter has infinite attenuation at 0 Hz (DC), you don't need
> a steep one. Steep filters will do more harm, because they introduce
> significant phase shift which blurs transients."
> — Digido.com [2]
Use a 1st-order or gentle 2nd-order Butterworth HP at 5–10 Hz. Avoid steep
brick-wall filters for this purpose. Alternative: compute the mean of the
signal and subtract it (DC-mean subtraction) — this is zero-phase and
equivalent for short blocks.
**Rule B-1.2c — DC offset threshold: flag stems where |mean(signal)| > 1e-4.**
Stems from demucs rarely have significant DC (demucs normalizes internally),
but re-encoded intermediates or analog-chain captures can acquire it. Flag and
auto-correct before loop-point search.
### 1.3 Equal-Power Crossfade at the Seam
Even with perfect zero-crossing alignment, a "hard splice" can still produce
a click if slope discontinuity remains. A short crossfade is the universal
fallback and should always be applied as insurance.
**Rule B-1.3a — Apply an equal-power (constant-power) crossfade, not a linear one.**
A linear crossfade causes a perceivable volume dip at the midpoint because
power is not preserved. An equal-power curve maintains perceived loudness:
```
gain_in(t) = sqrt(t) # fade in, t ∈ [0, 1]
gain_out(t) = sqrt(1 - t) # fade out, t ∈ [0, 1]
```
At the midpoint (t = 0.5): `gain_in = gain_out = 0.707`, so power is
`0.707² + 0.707² = 1.0`. [3]
The sin/cos variant is mathematically equivalent (sin²+cos²=1) and slightly
smoother at the edges but more expensive to compute [4]:
```
gain_in(t) = sin(t * π/2)
gain_out(t) = cos(t * π/2)
```
For the foundry's batch offline context, either variant is fine; prefer sqrt
for simplicity.
**Rule B-1.3b — Crossfade seam length: 10–30 ms (default: 20 ms) for loops;
1–5 ms for drum one-shots.**
- 20 ms at 44.1 kHz = 882 samples. This is the minimum that reliably hides
seam discontinuity in sustained material (pads, bass, chords).
- KVR forum consensus puts the minimum at 128–256 samples (~3–6 ms at 44.1 kHz)
for structurally significant material; 1 ms (44 samples) with an equal-power
curve eliminates the pop even at shorter crossfades [1].
- For material with audible pitch/timbre structure at the seam, keep the
crossfade short (≤10 ms) to prevent comb-filtering artifacts from the two
overlapping sections.
- For drones and pads, longer crossfades (up to 50 ms) are transparent; use
them to mask any harmonic mismatch at the boundary.
**Rule B-1.3c — Crossfade implementation: overlap-add, not mix-in-place.**
The crossfade region must borrow `fade_len` samples of "pre-roll" before the
loop-out point and `fade_len` samples of "post-roll" after the loop-in point.
Both must exist in the source audio. If the source audio has no pre/post room
(tight transient at bar 1 beat 1), fall back to a shorter fade or hard splice
at the zero crossing.
### 1.4 Loopability Score
**Rule B-1.4a — Compute a seam-discontinuity score; reject candidates above threshold.**
The primary loopability metric is the RMS of the amplitude difference vector
across the overlap region, normalized to the RMS of the loop body:
```
seam_error = rms(signal[loop_in : loop_in + K] - signal[loop_out - K : loop_out])
loop_rms = rms(signal[loop_in : loop_out])
loopability = 1 - clip(seam_error / (loop_rms + ε), 0, 1)
```
Higher `loopability` (→1.0) means the two ends are nearly identical in
waveform shape — the seam will be inaudible. A correlation-based variant
(Pearson r on the overlap region) is equivalent and sometimes more stable for
pitched material.
> "Calculate a score based on what provides the least harmonic distortion...
> apply frequency bias weighting to penalize lower frequency mismatches more
> heavily."
> — DSPRelated thread, "Calculating optimal loop point" [5]
**Rule B-1.4b — Secondary loopability metric: spectral continuity.**
Compute the spectral centroid difference between the last 50 ms of the loop
body and the first 50 ms. Timbre jumps (e.g., the loop out-point is in a
sparse section and the in-point is in a dense section) are audible even with
perfect amplitude matching. Flag candidates where central centroid delta
exceeds 20% of the loop's mean centroid.
---
## Section 2 — Musical Alignment: Bar and Beat Grid
### 2.1 The Core Formula
**Rule B-2.1a — Sample length for N bars at tempo T (BPM) and S (sample rate):**
```
samples = (N * 4 * 60 / T) * S # 4/4 time
seconds = N * 4 * 60 / T
```
Example: 2 bars at 130 BPM, 44100 Hz → `2 * 4 * 60 / 130 * 44100 = 81415 samples ≈ 1.846 s`.
Candidate loop lengths should be multiples of one beat: 1, 2, 4, 8 bars.
Half-bar loops (2 beats) are acceptable for percussive one-shots that repeat
at the bar.
**Rule B-2.1b — Quantize candidate endpoints to the nearest detected beat.**
Beat-synchronous cutting aligns the loop with the musical phrase. Even a 5 ms
misalignment (220 samples at 44.1 kHz) at 130 BPM causes the loop to drift one
full beat within ~37 repeats. [6]
**Rule B-2.1c [DRIFT HAZARD] — Never assume a global constant BPM.**
For livecoded material:
> "When running a beat tracker on signals with varying tempo directly, the
> result does not follow well because the beat tracker is assuming a single
> (average) tempo across the entire signal."
> — librosa documentation, "Beat tracking with time-varying tempo" [7]
Mitigation: use librosa's dynamic beat tracker or madmom's DBN beat tracker
(which models per-region local tempo). Compute a per-region BPM for each loop
candidate window rather than a global tempo. Treat any region where the beat
tracker confidence is below a threshold as "tempo-uncertain" and widen the zero-
crossing search window accordingly.
**Rule B-2.1d [DRIFT HAZARD] — Report estimated BPM as a range, not a point.**
A livecoded 130 BPM track may drift ±3 BPM within a single 8-bar phrase. Store
`bpm_mean`, `bpm_min`, `bpm_max` in the `LoopCandidate` model. Flag any
candidate with a range exceeding ±5 BPM as "tempo-unstable" in the UI — the
human must decide whether the drift is audible in context.
### 2.2 Phrase and Downbeat Alignment
**Rule B-2.2a — Prefer loop-in points at downbeats (beat 1 of a bar).**
Most musical phrases start on the downbeat. A loop that begins mid-bar will
feel rhythmically displaced when played with `s "<kit>"` in a Tidal pattern
that cycles at bar boundaries. Downbeat detection is a separate algorithm layer
(madmom's downbeat tracker is the state of the art); it is not the same as beat
tracking.
**Rule B-2.2b — Enumerate candidates at 1-, 2-, 4-, and 8-bar multiples.**
Starting from each detected downbeat, emit one candidate per standard loop
length. This gives the human a fast chooser grid without forcing a specific
length. [8]
**Rule B-2.2c — Flag loops shorter than 0.5 s as one-shot candidates, not loop candidates.**
See Section 4 (one-shot vs. loop policy).
---
## Section 3 — Stem-Specific Rules
### 3.1 Bass / Sub (bass.wav)
**Rule B-3.1a — Always mono-sum bass stems.**
Sub-bass energy below ~100 Hz must be mono-summed before export. Stereo width
in the sub creates phase cancellation on mono playback systems (club soundsystems,
phone speakers) and can cancel the sub entirely.
> "The low-end of your music, around 0–100 Hz, is the most problematic
> frequency range when it comes to mono compatibility... when summed to mono
> there's a huge risk of phase cancellation."
> — Black Ghost Audio, "How to Improve Mono Compatibility" [9]
> "In mono is assumed when loading sound files."
> — SuperDirt README [10]
Since SuperDirt loads all samples as mono by default, outputting a stereo bass
WAV is wasted data at best and phase-cancelled at worst. Export `bass.wav`
as a 1-channel (mono) file. Demucs `bass.wav` outputs are typically stereo —
apply `np.mean(audio, axis=0)` to collapse before writing.
**Rule B-3.1b — Validate: check correlation between stereo channels before summing.**
If the stereo bass stem has inter-channel correlation < 0.7, the channels
contain significantly different information; flag for human review before
collapsing. (Correlation < 0.5 indicates likely phase inversion or heavy
out-of-phase processing — warn strongly.)
**Rule B-3.1c — Loop points for bass: prefer cycle-aligned cuts at the sub fundamental.**
The sub-bass waveform period at, e.g., 50 Hz is 882 samples at 44.1 kHz.
Cutting mid-cycle produces a click that a 20 ms crossfade will mask, but the
seam will still audibly thump if the loop-out and loop-in are at opposite
phases of the sub cycle. After the bar-aligned beat snap, find the nearest
full-cycle zero crossing of the low-passed (100 Hz) signal. This is the
"sub-cycle snap" refinement.
### 3.2 Drums (drums.wav)
**Rule B-3.2a — Preserve transients: loop-in must be at or before the kick/snare
transient, not inside its attack.**
A transient truncated at the loop-in point sounds like a click AND destroys
the punch of the first beat. The loop-in point must be at a zero crossing
that precedes the first transient by at least 5 ms (220 samples), so the full
attack envelope is inside the loop.
> "Loop boundary crossfading may blur correctly captured transients very close
> to the loop beginning."
> — Loopy Pro Wiki, "Troubleshooting Transients" [11]
If the zero-crossing snap moves the loop-in past a transient, pull it back to
the previous zero crossing before the transient onset.
**Rule B-3.2b — Apply a shorter crossfade for drums: 1–5 ms (default: 3 ms).**
A 20 ms crossfade on a drum loop smears the first hit of each repetition. Use
3 ms equal-power crossfade unless the seam discontinuity score (B-1.4a) demands
more; cap at 10 ms for drums. [1]
**Rule B-3.2c — Onset-detect the drums stem to validate beat positions.**
Use librosa's `onset_detect` on the drum stem as a ground-truth beat grid. If
the detected onsets differ from the general beat tracker positions by >15 ms,
prefer the stem-derived onsets for drum-loop boundaries.
### 3.3 Pads / Other (other.wav)
**Rule B-3.3a — Allow longer crossfades for pads: up to 50 ms.**
Pad material has no transients to protect and often has significant spectral
evolution at phrase boundaries. A 30–50 ms equal-power crossfade is transparent
and effectively hides any timbral mismatch at the seam.
**Rule B-3.3b — Export pads as stereo if the source is stereo and the material
has genuine stereo width (mid-side correlation < 0.95).**
SuperDirt announces "mono is assumed" but SuperCollider can play stereo files —
it will upmix/distribute them. Since demucs `other.wav` often carries stereo
reverb tails and wide synth pads, exporting as stereo preserves that width.
Test the inter-channel correlation: if > 0.95 (nearly mono), collapse to mono.
**Rule B-3.3c — Pad loop ends must include the reverb tail.**
If the loop-out point clips a reverb tail, the next repetition starts with
an abrupt reverb restart that sounds unnatural. Detect tail activity by
tracking when the signal envelope falls below -40 dBFS, and snap the loop-out
point to include the full tail (or crossfade it into the in-point of the next
loop if the tail bleeds into the next bar).
---
## Section 4 — One-Shot vs. Loop Convention
**Rule B-4.1 — Duration threshold: < 0.75 s → classify as one-shot candidate.**
In the existing custom sample library the bimodal split is sub-second one-shots
vs. multi-bar loops. A stem or clip shorter than 0.75 s has no room for a
meaningful loop crossfade and should be exported as a one-shot (no loop
metadata, plays once on trigger).
> "One-shot sample... short in duration, ranging from a fraction of a second
> to a few seconds at max."
> — The Sample Lab [12]
**Rule B-4.2 — Loop candidates must be at least 1 bar at the estimated BPM.**
Below one bar, the loop repeats faster than the Tidal pattern cycle and
produces a stutter effect. The minimum loop length is therefore:
```
min_loop_s = 4 * 60 / bpm # 1 bar in 4/4
```
At 130 BPM: 1.846 s minimum. At 90 BPM: 2.667 s minimum.
**Rule B-4.3 — Naming convention: one-shots use plain index (`0.wav`, `1.wav`);
loops may use descriptive suffixes but must still sort to predictable indices.**
SuperDirt indexes samples within a kit folder alphabetically by filename
(behavior is OS-dependent — see note below). Name files so that alphabetical
order equals intended playback order:
```
00_kick_dry.wav
01_kick_room.wav
02_snare.wav
```
Zero-pad the numeric prefix to 2 digits minimum (3 if > 99 files per kit).
Do not rely on filename alone without the numeric prefix for ordering.
> [SuperDirt issue #131]: "Linux-based systems sort the former first, and
> others sort the latter first" — the sort order is OS-dependent and an
> unresolved issue in the (now-archived) SuperDirt repo. [13]
>
> Mitigation: use numeric-prefix names so that both alphabetical-by-name and
> alphabetical-by-full-path sorting produce the same order. This is the
> safest cross-platform convention.
**Rule B-4.4 — One-shots: zero-pad the tail to the next bar boundary.**
SuperDirt plays one-shots in full. If a kick sample ends abruptly, adding
20 ms of silence padding prevents the click from the buffer release. For
loops, no tail padding is needed (the loop crossfade handles it).
---
## Section 5 — SuperDirt File Format Requirements
### 5.1 Accepted Formats
**Rule B-5.1a — Export as 24-bit PCM WAV.**
SuperDirt/SuperCollider uses libsndfile for sample loading. libsndfile supports
16-bit and 24-bit PCM WAV, AIFF, and other formats. 24-bit gives 16 dB more
headroom than 16-bit (144 dB theoretical dynamic range vs. 96 dB) at no
practical storage penalty for loops. [14]
Do not export as 32-bit float WAV unless you specifically need bit-perfect
preservation of the demucs output (demucs emits 32-bit float internally);
SuperCollider handles 32-bit float WAV but it is wasteful for distribution
into a kit.
**Rule B-5.1b — Sample rate: match the SuperCollider server rate (typically 44100 Hz).**
SuperDirt requires the audio device to run at 44100 Hz or 48000 Hz. [15]
SuperCollider resamples on load if the file sample rate differs from the
server rate, but resampling costs CPU and degrades quality at the loop seam
(the resampled positions no longer land on the snapped zero crossings).
Export all loop WAVs at the same rate as the SuperCollider server — which is
almost always 44100 Hz in a standard TidalCycles setup.
> "Resampling is enabled by default in SuperCollider, so SuperCollider can run
> at a different sample rate than the hardware." [15]
> — but prefer matching rates to keep zero-crossing snaps exact.
Demucs outputs at the input file's sample rate. Most YouTube/streaming sources
are 44100 Hz; verify with `soundfile.info()` before processing.
**Rule B-5.1c — Channel count:**
- Bass stem → 1 channel (mono, always — see B-3.1a).
- Drums stem → 1 channel (mono) by default; stereo if the drum bus was
intentionally widened (rare in demucs output, correlation check per B-3.1b).
- Other/pads stem → 2 channels (stereo) unless correlation > 0.95.
- Vocals stem → 2 channels (stereo) for natural room/reverb.
### 5.2 Folder Layout and Indexing
**Rule B-5.2a — One kit = one flat directory under `~/Work/Sound/Samples/<kit>/`.**
The publish pipeline (`engine/publish.py`) symlinks this directory into
Dirt-Samples so that `s "<kit>"` resolves. No subdirectories inside a kit
folder (SuperDirt does not recurse).
**Rule B-5.2b — Files are indexed by SuperDirt in alphabetical filename order (with
OS-dependent caveats per B-4.3). The `n` parameter in Tidal selects by this index.**
```tidal
d1 $ s "mykit" -- plays index 0
d1 $ s "mykit" # n 3 -- plays index 3
d1 $ s "mykit" <| n (run 4) -- cycles through 0 1 2 3
```
The Foundry must write the kit manifest (which index maps to which loop) so
the human knows which `n` value to audition.
### 5.3 Normalization
**Rule B-5.3a — Normalize to -1 dBFS peak (not LUFS) for sample kit export.**
Kits are played by SuperDirt at whatever amplitude they are stored; Tidal's
`gain` parameter then adjusts at playback. Normalizing to near-0 dBFS peak
(-1 dBFS headroom for inter-sample peaks) ensures headroom is not wasted and
that the Tidal `gain` control has maximum range.
> "If you want to maximize level, normalize to peak, and set the value to
> somewhere just below 0 dBFS (maybe -0.7 dBFS)." [16]
Do NOT loudness-normalize to -14 LUFS for sample kits: that target is for
streaming masters, not raw sample material. Sample kits are not metered by
the streaming algorithms.
**Rule B-5.3b — Apply peak normalization per-file, not per-kit.**
Each WAV in the kit is normalized independently. Kit-level normalization is
the human's job via Tidal's `gain` — the machine just ensures no file is
clipping or excessively quiet.
**Rule B-5.3c — True-peak limit: -0.1 dBTP.**
Use a true-peak limiter (or numpy clip + 1 sample lookahead) rather than
sample-peak normalization. Inter-sample peaks above 0 dBFS cause clipping
on D/A conversion even when the sample values themselves are below 0 dBFS.
---
## Section 6 — Livecoded / Tempo-Drift Specific Rules
**Rule B-6.1 [DRIFT HAZARD] — Use a sliding-window BPM estimate, not a global one.**
Compute BPM in overlapping windows of 4–8 bars. For each candidate loop
region, use the local BPM from its own window to compute the expected bar
length. Drift of even 1 BPM over 4 bars causes a 30 ms timing error —
larger than the ±10 ms zero-crossing search window.
**Rule B-6.2 [DRIFT HAZARD] — Drift-tolerant bar-length estimate.**
Rather than computing `bar_length = 4 * 60 / bpm_global`, use the onset
positions directly:
```python
beats = librosa.beat.beat_track(y, sr, trim=False)[1]
bar_starts = beats[::4] # every 4th beat is a downbeat (4/4 assumption)
bar_lengths = np.diff(bar_starts) # in samples, variable across the piece
```
Loop candidates should span an integer number of actual inter-onset intervals,
not an integer number of mathematically computed bar lengths.
**Rule B-6.3 [DRIFT HAZARD] — Flag and score tempo-stable regions preferentially.**
A loop cut from a region where the local BPM variance is low (σ < 1 BPM over
the loop duration) will repeat more reliably in a Tidal pattern than one from
a high-drift region. Include `bpm_stability` (inverse of σ) as a positive
factor in the loopability score.
**Rule B-6.4 — Loops from livecoded stems are "free" loops, not tempo-locked.**
Unlike a sample-pack loop tagged with `120bpm`, these loops have an inherent
tempo that may not match the current Tidal session. Document the estimated BPM
in the Foundry's kit manifest (a `meta.json` alongside the WAV files) so the
human knows whether to stretch/pitch the loop or just play it as-is.
> TidalCycles does not auto-stretch samples to fit the current tempo; `speed`
> and `slow` modify playback rate but also shift pitch. There is no native
> tempo-sync stretching in SuperDirt. The human's judgment governs how to deploy
> a free loop. [17]
---
## Section 7 — Conflict Table and Trade-Offs
| Rule A | Rule B | Conflict | Resolution |
|--------|--------|----------|------------|
| B-1.1c: ±10 ms ZC search window | B-2.2a: snap to downbeat | If the nearest ZC is 12 ms from the downbeat, we breach one constraint | Prefer the musically-aligned downbeat; apply crossfade (B-1.3) to hide the seam. Never move more than 25 ms from the beat. |
| B-1.3b: 20 ms crossfade | B-3.2b: 3 ms crossfade for drums | Drums stem fed through the default pad crossfade | Per-stem defaults override the global default. Dispatch on stem type. |
| B-3.1a: mono-sum bass | B-3.3b: stereo pads | No conflict if stem type dispatches correctly | Identity check: bass.wav always mono, other.wav conditionally stereo. |
| B-1.4a: seam discontinuity score | B-2.2b: enumerate all bar-multiple candidates | Exhaustive enumeration is slow; scoring every candidate at 1/2/4/8 bars from every downbeat is O(N²) | Prune: only score candidates within a configurable "region of interest" around the detected high-activity section. Default: first 4 minutes. |
| B-5.1b: match SC server rate (44100) | Source at 48000 Hz | Resampling degrades ZC snap accuracy | Resample the whole stem to 44100 before loop-point search AND before export. Do not export at source rate. |
| B-6.2: drift-tolerant bar lengths | B-4.2: 1-bar minimum duration | At low BPM + high drift, `bar_lengths` array is noisy; minimum may not hold | Use median bar length, not mean, for minimum-duration check. Reject only if all detected bar lengths are < 0.5 s. |
---
## Sources
[1] KVR Audio DSP Forum, "Avoiding Clicks In Loops":
https://www.kvraudio.com/forum/viewtopic.php?t=325574
[2] Digido.com, "DC Offset: Clicks and Pops":
https://www.digido.com/ufaqs/dc-offset-clicks-pops/
(mirror/summary via integraudio.com: https://integraudio.com/how-to-remove-dc-offset/)
[3] Teedstuff blog, "Implementing a Constant Power Crossfade":
https://teedteed.wordpress.com/2019/05/06/implementing-a-constant-power-crossfade/
[4] KVR Audio, "I'm confused about equal power crossfading":
https://www.kvraudio.com/forum/viewtopic.php?t=347151
[5] DSPRelated forum, "Calculating optimal loop point in audio sample":
https://www.dsprelated.com/thread/15880/calculating-optimal-loop-point-in-audio-sample
(The music-dsp Columbia thread [https://music-dsp.music.columbia.narkive.com/C8IESiuZ/algorithms-for-finding-seamless-loops-in-audio]
covers equivalent ground but was inaccessible during research; the archived
version at web.archive.org is blocked from automated fetch.)
[6] CMUSE, "Loop Length Calculator for BPM, Bars, and Samples":
https://www.cmuse.org/loop-length-calculator
[7] librosa 0.11.0 documentation, "Beat tracking with time-varying tempo":
http://librosa.org/doc/0.11.0/auto_examples/plot_dynamic_beat.html
[8] Freesound Loop Dataset paper (Fonseca et al.):
https://arxiv.org/pdf/2008.11507
[9] Black Ghost Audio, "How to Improve Mono Compatibility":
https://www.blackghostaudio.com/blog/how-to-improve-mono-compatibility
[10] SuperDirt GitHub README (yaxu/SuperDirt):
https://github.com/yaxu/SuperDirt
("load samples (path can be passed) mono is assumed.")
[11] Loopy Pro Wiki, "Troubleshooting Transients":
https://wiki.loopypro.com/Troubleshooting_Transients
[12] The Sample Lab, "What are One-Shot Samples?":
https://thesamplelab.com/blogs/news/what-are-one-shot-samples
[13] SuperDirt GitHub issue #131, "Inconsistent sorting of files":
https://github.com/musikinformatik/SuperDirt/issues/131
(repo archived 2025-12-06; issue unresolved)
[14] Sound on Sound / general audio engineering: 24-bit vs 16-bit PCM.
https://blog.landr.com/sample-rate-bit-depth/
[15] TOPLAP forum, "Sample Rate Mismatch, server won't boot":
https://forum.toplap.org/t/sample-rate-mismatch-server-wont-boot/2048
[16] Normalization and peak targets for sample packs:
https://nolabelnoproducernolimits.com/indie/production/reaper/normalize/
[17] TidalCycles documentation, Custom Samples:
https://tidalcycles.org/docs/configuration/AudioSamples/audiosamples/
tools/foundry/research/C_prior_art.md 0 → 100644
# Strand C — Loop-Detection Algorithms & Prior Art
> Research date: 2026-06-22.
> Covers: beat/tempo tracking, loop-point / segment detection, seamless-cut tech, existing auto-loop tools.
> Context: demucs-separated stems, livecoded TidalCycles music with BPM drift, target use in `engine/loops.py`.
---
## 1. Beat / Tempo / Downbeat Tracking
### 1.1 librosa — `beat_track`, `plp`, `onset_strength`, `tempo`, `fourier_tempogram`
**How it works.**
`beat_track` (Ellis 2007 dynamic-programming) estimates a single global BPM via the autocorrelation of the onset-strength envelope, then picks beats consistent with that tempo using DP. It does support a *time-varying* `bpm` array (one value per onset frame), but the documentation is explicit: the algorithm "does not follow particularly well" when the tempo changes because it defaults to assuming a single average tempo across the signal. ([librosa docs — Dynamic Beat](http://librosa.org/doc/0.11.0/auto_examples/plot_dynamic_beat.html))
`plp` (Predominant Local Pulse, Grosche & Müller) analyses the onset-strength envelope in the *frequency domain* (Fourier tempogram) to find a locally-stable tempo **per frame**, then synthesises local half-waves whose peaks are the beat candidates. The librosa docs explicitly recommend `plp` over `beat_track` "when the tempo is expected to vary significantly over time" and for streaming use because it does not require the full signal. ([librosa docs — plp](https://librosa.org/doc/main/generated/librosa.beat.plp.html))
**Measured latency (system python3, librosa 0.11.0, 120 s synthetic signal, M²-era CPU):**
| Function | Wall time |
|---|---|
| `onset_strength` | 2.06 s |
| `plp` + peak-pick | 1.19 s |
| `beat_track` | 1.85 s |
| `MFCC` (13 coeffs) | 0.87 s |
| `chroma_cqt` | 0.76 s |
| Beat-sync `recurrence_matrix` (224×224) | 0.45 s |
| Foote novelty on beat-sync R | 0.013 s |
| ZCR | 0.008 s |
| **Full pipeline total** | **~5.4 s** |
**Suitability for BPM-drifting livecoded material.** `beat_track` is unsuitable for tracks where `setcps` changes mid-performance (e.g. 60→120→180→60 BPM). Its single-tempo model will snap to an average, producing misaligned beat frames. **`plp` is the preferred tracker** — frame-local tempo adapts across a `setcps` ramp. The extracted PLP maxima (via `scipy.signal.find_peaks`) form a variable-tempo beat grid without requiring the whole track up front. Caveats: PLP's time-frequency analysis still assumes locally stationary short windows (~384 frames at hop=512 → ~8 s context); abrupt single-step `setcps` jumps may cause a 1–2 beat slip at the transition boundary. Post-snap to the nearest onset-strength peak mitigates this.
**Dependency weight:** zero — already in the tide-table venv.
### 1.2 madmom — `DBNDownBeatTrackingProcessor`
**How it works.** Combines a bi-directional RNN (trained on annotated downbeat datasets) with a Dynamic Bayesian Network / HMM decoder that enforces metre consistency. Ranked #1 at MIREX 2016 downbeat tracking. ([madmom docs — downbeats](https://madmom.readthedocs.io/en/v0.16/modules/features/downbeats.html))
**Accuracy vs librosa.** Multiple comparisons and the BIFF.ai survey report madmom as the strongest *offline* downbeat tracker on classical and complex-rhythm recordings. It reliably emits (beat, downbeat) pairs. ([BIFF.ai rundown](https://biff.ai/a-rundown-of-open-source-beat-detection-models/))
**Suitability for livecoded material.** The RNN was trained on datasets with stable metre. BPM drift and `setcps` jumps are not in its training distribution. It assumes 4/4 or 3/4 and a constant metre within a track segment; a `slow 2` applied mid-pattern (halving effective BPM) may confuse the DBN state machine. The BIFF rundown notes madmom is "better suited to controlled environments rather than dynamic live scenarios."
**Latency.** Offline only; no streaming mode. Processing adds the RNN forward pass on top of signal processing: adds several seconds on long stems. BeatNet (below) wraps madmom's signal front-end with particle filtering and offers online/streaming modes but itself requires madmom as a dependency. ([BeatNet](https://github.com/mjhydri/BeatNet))
**Dependency weight: HIGH — NOT installed.** madmom requires Python ≤ 3.9 for its Cython extensions in the 0.16 line; installation on Python 3.12+ (likely the system python3 version) requires building from source and has known compatibility issues with NumPy 2.x. **Flag: do not add madmom to the tide-table venv without careful version-pinning and build testing.**
### 1.3 aubio
**How it works.** C library with Python bindings; beat tracker uses a phase-vocoder-based tempo estimator plus onset detector. Designed for real-time use (large buffer = higher accuracy at ~2048 samples, low latency at 512).
**Accuracy.** Within ±1% BPM on steady tracks; cannot correctly detect tempo changes smaller than 2.75%. ([aubio tempo issue #227](https://github.com/aubio/aubio/issues/227); cross-library comparison quoted in madmom paper). Prefers measurements around 107 BPM; may return half/double the real BPM on edge cases. In the BIFF survey madmom outperforms aubio on complex rhythms and classical material.
**Suitability for livecoded material.** Less suitable than librosa PLP: no per-frame tempo adaptation, assumes locally constant BPM window. Good for real-time onset-triggered applications (e.g. live VJ sync) but not the offline post-processing use case here.
**Dependency weight: NOT installed, C build required.** Not worth adding when librosa already covers the use case more robustly in our offline pipeline.
### 1.4 BeatNet (ISMIR 2021)
CRNN + particle filter for joint beat/downbeat/tempo/metre tracking. Handles streaming and real-time modes (latency < 50 ms). Best-of-all-worlds candidate for accuracy AND variable tempo — particle filtering can track abrupt BPM jumps that break the madmom DBN. ([BeatNet paper](https://arxiv.org/pdf/2108.03576))
**However:** requires madmom as a dependency (see 1.2 weight warning above). Not installed. Overkill for an offline batch loop-finder — the latency advantage is irrelevant for our use case.
### 1.5 "Beat This!" (2024)
A 2024 ISMIR paper proposes eliminating DBN post-processing entirely with a Transformer-based model that directly predicts beat activation — reportedly matching or beating madmom accuracy without the HMM overhead. ([arXiv 2407.21658](https://arxiv.org/pdf/2407.21658)). Promising for future adoption but currently requires a custom model checkpoint; not available as a pip drop-in and not installed.
---
## 2. Loop-Point / Segment Detection
### 2.1 Self-Similarity Matrix (SSM) + Foote Novelty
**Principle.** Build a feature matrix F (e.g. MFCC, chroma, or a concatenation) over time. Compute the SSM as pairwise cosine or Euclidean similarity between frames. Homogeneous sections form bright blocks along the diagonal; transitions create "checkerboard" patterns where two dissimilar blocks meet. Convolve the SSM diagonal with a checkerboard kernel to get a "novelty" function — peaks mark section boundaries (Foote 2000). ([Foote 2000 — FMP reference](https://www.audiolabs-erlangen.de/resources/MIR/FMP/C4/C4S4_NoveltySegmentation.html))
**Key implementation choice: operate on the beat-synchronous SSM, not the frame-level SSM.**
Our benchmark confirms that:
- Frame-level SSM (5168×5168 for 120 s) → Foote convolution: **26.96 s** — too slow for interactive use.
- Beat-synchronous SSM (224×224) → Foote convolution: **0.45 s + 0.013 s = <0.5 s total** — perfectly acceptable.
Beat-synchronous features (`librosa.util.sync`) are the standard mitigation for the quadratic SSM cost.
**For loops (not just boundaries).** The SSM's off-diagonal stripes directly encode repeating sections: a stripe at lag=L means the passage at time t looks like the passage at time t+L. Loop candidates = positions where the diagonal-lagged SSM is high AND the loop duration is musically useful (1/2/4/8 bars). The `librosa.segment.recurrence_matrix(mode='affinity')` already computes this in affinity form.
**Kernel size trade-off.** A kernel of k beats corresponds to a sensitivity window of ~k bars (at 4/4, 120 BPM). k=8 detects 2-bar repeats; k=16 detects 4-bar sections. Recommendation: run two passes (k=4 for 1-bar loops, k=16 for 4-bar loops).
**Variable-tempo robustness.** Beat-synchronous SSM absorbs tempo drift automatically — the beat axis is in *beats*, not seconds. A `slow 2` ramp shows up as longer beat intervals in the time axis but the SSM similarity at the feature level remains correct. **This is the central advantage of beat-sync over frame-sync for livecoded material.**
**Features to use.** From our L0/L1 stack (`reference_audio_feature_stack`):
- **MFCC (13)** — timbral identity; best for detecting when the *instrumentation* repeats.
- **Chroma / CENS** — harmonic identity; loop detection in tonal material.
- **Stacked MFCC+chroma (25 dims)** — covers both; used in the 2025 MSA study ([PLoS One 2025](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0312608)).
- For stems (demucs): drums stem → MFCC+RMS works; bass stem → MFCC+chroma; other/vocals → full stack. Do NOT assume MFCC alone on bass covers timbre (cf. `feedback_mastering_eda`).
**Accuracy.** The MSAF survey and the 2025 beat-MSA study report HR3F (boundary at 3 s tolerance) of ~69 % for the best beat-based methods on the SALAMI dataset. For loop *finding* (not boundary labelling) what matters is whether the *same* repeating region is flagged: SSM similarity > 0.7 on beat-sync MFCC/chroma is a reliable proxy. Perceptual validation by ear remains necessary (cf. project design philosophy).
### 2.2 Autocorrelation / Beat-Spectrum for Repeating Sections
**Principle.** The beat spectrum (Foote & Cooper 2001) = the mean of the SSM diagonals as a function of lag. Peaks at lag L mean the music repeats every L beats — directly encoding loop period candidates. `librosa.segment.recurrence_matrix` + `np.diag` sum is the quick implementation.
For *loop point* finding (not period estimation), autocorrelation of the waveform or of a feature envelope around a candidate region can score how well the tail of a loop matches its head — i.e. will it click-free? Cross-correlation of the waveform windows at candidate start/end gives a seam-coherence score that complements the SSM structural score.
Our micro-benchmark shows scoring 74 4-beat candidates in a 30 s stem takes < 1 ms — trivially fast even with hundreds of candidates.
### 2.3 Laplacian Segmentation (`librosa.segment.agglomerative`)
**Principle.** McFee & Ellis (2014) — embed beat-sync features in a graph Laplacian, cluster with agglomerative hierarchical clustering. Returns a flat label sequence (k clusters = k musical sections). Good for *labelling* sections (verse/chorus/bridge) but the k must be supplied; does not output boundary confidences, so less useful for ranking loop candidates.
**Suitability.** Useful downstream for *grouping* found loop candidates by musical section (e.g. "these 3 candidates all come from the same chorus"). Not the primary loop-finding engine. Timing: < 5 ms for beat-sync features on a 120 s stem.
### 2.4 Graph-Based Audio Looping (Roma, Tremblay, Green — DAFx 2021)
**Algorithm.** Builds a directed similarity graph from spectral frames where edge weight = inverse feature distance; traversal allows non-linear looping and granulation. Beat spectrum from SSM diagonals drives onset detection; cluster analysis finds stable homogeneous zones. ([DAFx 2021 paper](https://dafx2020.mdw.ac.at/proceedings/papers/DAFx20in21_paper_41.pdf))
**Suitability.** Designed for creative real-time granulation/looping (FluCoMa context), not for batch extraction of pre-cut WAVs. Its loop transitions are probabilistic (graph traversal), not deterministic start/end pairs. Intellectually interesting but not the right abstraction for our "candidate → human picks → export WAV" flow. **Defer.**
---
## 3. Seamless-Cut Technology
### 3.1 Zero-Crossing Snap
Standard: nudge the candidate start and end samples to the nearest zero-crossing in the waveform, preventing a DC-offset click at the loop seam. `librosa.zero_crossings` computes the ZCR boolean array; argwhere over a ±N sample window around the beat-frame sample gives the nearest ZC. Takes < 1 ms. ([DSP Related discussion](https://www.dsprelated.com/thread/15880/calculating-optimal-loop-point-in-audio-sample))
**Limitations.** ZC snap fails for sustained bass/drone material where the waveform rarely crosses zero (e.g. a low-frequency sine oscillating slowly). In that case, a short equal-power crossfade (2–10 ms) replaces ZC snap. Strand B owns the crossfade policy.
### 3.2 Beat-Synchronous Slicing
Start/end samples derived from `librosa.frames_to_samples(beat_frames)`. This guarantees bar-aligned boundaries independently of ZC snap; ZC snap is then a local ±N-sample refinement within the beat-aligned window. The two are complementary, not alternatives.
### 3.3 PyMusicLooper (arkrow)
An active open-source project specifically for finding seamless loop points. v3.0 (2023) added "10× faster core loop search" and explicit zero-crossing alignment. Uses beat detection internally (skips beat analysis in brute-force mode). The scoring mechanism is not documented publicly; the author warns "the algorithmically chosen best loop point may not be perceptually good" — they recommend interactive review. ([PyMusicLooper README](https://github.com/arkrow/PyMusicLooper))
**Assessment.** Designed for game-audio style looping (music that loops forever at a single point), not for extracting multiple diverse loop candidates from a stem for use as Tidal samples. The interactive single-loop paradigm mismatches our "surface 20 candidates, human picks" design. **Useful only as reference; don't adopt as dependency.**
---
## 4. Existing Auto-Loop Extractor Tools / Repos Summary
| Tool | Core algorithm | Tempo drift | Stem-aware | Multi-candidate | Verdict |
|---|---|---|---|---|---|
| PyMusicLooper | Beat detect + ZC + waveform corr | No | No | Semi (pruning) | Reference only |
| librosa_loopfinder (Kexanone) | librosa-based, details private | Unknown | No | Yes (win_length) | Light candidate |
| loop_extractor (CurtisVonRubenhoff) | TF/Spleeter + ML | Unknown | Yes (via Spleeter) | Yes | Python 3.10 lock-in; avoid |
| MSAF (urinieto) | Foote, Serrà sf, OLDA, scluster | No (fixed tempo) | No | Boundary list | Useful for boundary comparison |
| BeatNet | CRNN + particle filter | Yes | No | Beats only | Heavy dep (madmom) |
| FluCoMa graph looping | SSM graph traversal | Yes | Yes | Probabilistic | Creative tool, not batch |
---
## 5. Comparison Table — Techniques
| Technique | Accuracy (approx.) | Latency on 120 s stem | Handles BPM drift | New heavy deps | Recommended use |
|---|---|---|---|---|---|
| `librosa.beat.beat_track` | Moderate (single-BPM) | 1.85 s | No — assumes constant BPM | None | **Avoid as primary**; fallback only |
| `librosa.beat.plp` + peak-pick | Good for local-tempo | 1.19 s | **Yes — per-frame** | None | **Primary beat grid** |
| `librosa.onset.onset_strength` | N/A (input feature) | 2.06 s | N/A | None | Required input for plp |
| madmom DBNDownBeat | High (MIREX #1 2016) | Offline + RNN overhead | No — DBN assumes stable metre | **madmom (NOT installed)** | Defer — NumPy 2.x incompatible |
| aubio beat tracker | Moderate | Real-time capable | No — window-local only | aubio (NOT installed) | Skip — covered by plp |
| BeatNet | High + streaming | <50 ms per frame | **Yes — particle filter** | madmom + torch | Defer — heavy; overkill for offline |
| SSM + Foote novelty (frame-level) | Good for boundaries | 26.96 s conv | Beat-sync absorbs drift | None | Avoid frame-level; use beat-sync |
| SSM + Foote novelty (beat-sync) | Good for boundaries | 0.46 s total | **Yes — beat-sync normalises** | None | **Primary section/loop detector** |
| `librosa.segment.agglomerative` | Good (section labels) | <5 ms | Yes (beat-sync) | None | Supplementary (section grouping) |
| Autocorr seam score (waveform) | Perceptual proxy | <1 ms/candidate | N/A | None | **Primary loopability score** |
| ZC snap | N/A (click removal) | <1 ms | N/A | None | **Required post-processing** |
| PyMusicLooper | Game-audio loops | Seconds | Unknown | librosa + soundfile | Reference only |
---
## 6. Recommended v1 Stack
### Rationale
The constraints drive the stack:
1. **BPM drift is real** — `beat_track` (single-BPM) is unsafe; `plp` (per-frame) is the only installed tracker that handles drift.
2. **Stay in existing venv** — librosa 0.11.0 + numpy 2.4 + scipy 1.17.1 + sklearn 1.9.0 are installed. madmom is incompatible with NumPy 2.x and NOT installed; do not add it.
3. **Reuse L0/L1 features** — MFCC + chroma are already computed by `sample_features.py`; the loop finder should read from the same pipeline, not fork it.
4. **Beat-sync SSM is the sweet spot** — fast enough (<0.5 s on a 120 s stem), absorbs tempo drift, directly encodes repeating structure.
5. **Human picks** — the machine enumerates candidates and scores them; no auto-apply. The score must be *explanatory* (seam quality, SSM similarity, bar alignment) not opaque.
### The v1 Pipeline
```
stem.wav (demucs output, any orbit)
│
├─ 1. ONSET STRENGTH librosa.onset.onset_strength(y, sr)
│
├─ 2. VARIABLE-TEMPO BEAT GRID librosa.beat.plp(onset_envelope, sr)
│ → peak-pick via scipy.signal.find_peaks(pulse, distance=min_bpm_distance)
│ → beat_frames[] (variable spacing, handles setcps drift)
│
├─ 3. FEATURES librosa.feature.mfcc(y, sr, n_mfcc=13)
│ + librosa.feature.chroma_cqt(y, sr)
│ → stacked (25, n_frames)
│
├─ 4. BEAT-SYNC librosa.util.sync(feat, beat_frames)
│ → beat_feat (25, n_beats) ← tempo-drift-invariant representation
│
├─ 5. RECURRENCE MATRIX librosa.segment.recurrence_matrix(beat_feat,
│ mode='affinity', sym=True) → R (n_beats × n_beats)
│
├─ 6. FOOTE NOVELTY (two passes)
│ k=4 kernel → novelty_1bar (1-bar loop candidates)
│ k=16 kernel → novelty_4bar (4-bar / phrase loop candidates)
│ peak-pick on novelty → boundary_frames[]
│
├─ 7. CANDIDATE ENUMERATION
│ For each boundary pair (start_beat, end_beat) at bar lengths
│ {1, 2, 4, 8} bars:
│ - convert beat index → sample index
│ - duration filter (exclude < 0.25 s, > 16 s for 120 BPM baseline)
│
├─ 8. LOOPABILITY SCORE (per candidate)
│ a. SSM diagonal score = R[start_beat, end_beat] (0–1; how similar
│ are start and end contexts in feature space)
│ b. Seam coherence = cross-corr of waveform tail [end-N:end] vs
│ head [start:start+N] (N = 2048 samples)
│ c. ZCR at seam endpoints (proximity to zero crossing)
│ final_score = 0.5*SSM_score + 0.3*seam_coherence + 0.2*zcr_snap_proximity
│
├─ 9. ZC SNAP nudge start/end to nearest zero-crossing within ±512 samples
│
└─10. OUTPUT List[LoopCandidate] sorted by final_score descending
(top ~20, deduplicated by ≥50% overlap)
```
### Data model (sketch for strand D)
```python
@dataclass
class LoopCandidate:
start_sample: int # after ZC snap
end_sample: int # after ZC snap
start_beat: int # beat index in PLP grid
end_beat: int
bars: int # 1 / 2 / 4 / 8
bpm_local: float # median inter-beat interval in region → BPM
ssm_score: float # 0–1, recurrence similarity
seam_score: float # 0–1, waveform cross-corr at seam
zcr_snap_samples: int # how far we nudged start from raw beat edge
loopability: float # weighted composite
stem: str # "drums" / "bass" / "other" / "vocals" / "mix"
```
### What to defer (not v1)
| Technique | Why defer |
|---|---|
| madmom DBN downbeat | NumPy 2.x incompatible; madmom NOT installed; accuracy gain on livecoded music unclear |
| BeatNet | Requires madmom + torch model download; overkill for offline batch |
| Laplacian segmentation (as primary finder) | Needs k supplied; not self-contained |
| Graph-based looping (FluCoMa) | Wrong abstraction for our export flow |
| "Beat This!" Transformer | No pip-installable checkpoint yet |
| ML-based loopability (neural) | Insufficient labelled ParVagues loop training data; build ground truth first |
| Autocorr-based loop period detection | Subsumes into beat-spectrum diagonal of SSM — add only if SSM misses periodic structure in low-onset stems |
### Biggest risk
**BPM-drift at the `setcps` jump boundary** is the single hardest failure mode. PLP's local tempo adapts but needs ~8 s of context; at the exact frame of a `setcps` change, beat frames may be off by 1–2 beats, producing a slightly off-bar loop candidate. Mitigation: widen the ZC snap window at candidates near detected novelty peaks (which correlate with `setcps` transitions), and let the human reject/adjust in the UI. The SSM seam score and seam-coherence score will be *low* at bad-seam candidates, naturally demoting them in the ranking without needing to detect the tempo-change moment explicitly.
---
## 7. Sources
- Ellis, D.P.W. (2007). "Beat tracking by dynamic programming." *JNMR* 36(1). Via [librosa docs](https://librosa.org/doc/main/generated/librosa.beat.beat_track.html)
- Grosche, P. & Müller, M. (2011). "Extracting predominant local pulse information." *IEEE TASLP* 19(6). Via [librosa.beat.plp docs](https://librosa.org/doc/main/generated/librosa.beat.plp.html)
- Foote, J. (2000). "Automatic audio segmentation using a measure of audio novelty." *IEEE ICME*. [FMP reference](https://www.audiolabs-erlangen.de/resources/MIR/FMP/C4/C4S4_NoveltySegmentation.html)
- Böck, S. et al. (2016). "madmom: a new Python Audio and Music Signal Processing Library." *ACM MM*. [paper PDF](https://www.cp.jku.at/research/papers/Boeck_etal_ACMMM_2016.pdf)
- Böck, S. & Schedl, M. (2011). "Enhanced beat tracking with context-aware neural networks." *DAFx*. [madmom downbeats](https://madmom.readthedocs.io/en/v0.16/modules/features/downbeats.html)
- Hydri, M.J. et al. (2021). "BeatNet: CRNN and Particle Filtering for Online Joint Beat Downbeat and Meter Tracking." *ISMIR*. [arXiv 2108.03576](https://arxiv.org/pdf/2108.03576)
- BIFF.ai (2024). "A rundown of open source beat detection models." [biff.ai](https://biff.ai/a-rundown-of-open-source-beat-detection-models/)
- McFee, B. & Battenberg, E. (2024). "librosa 0.11.0: time-varying beat tracking." [librosa example](http://librosa.org/doc/0.11.0/auto_examples/plot_dynamic_beat.html)
- Roma, G., Tremblay, P.A. & Green, O. (2021). "Graph-Based Audio Looping and Granulation." *DAFx*. [proceedings](https://dafx2020.mdw.ac.at/proceedings/papers/DAFx20in21_paper_41.pdf)
- Nieto, O. & Bello, J.P. (2015). "MSAF: Music Structure Analysis Framework." *ISMIR LBD*. [paper](https://ismir2015.uma.es/LBD/LBD30.pdf)
- Tan, X. et al. (2025). "A music structure analysis method based on beat feature and improved residual networks." *PLoS One*. [DOI 10.1371/journal.pone.0312608](https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0312608)
- Arkrow (2023). "PyMusicLooper." [GitHub](https://github.com/arkrow/PyMusicLooper)
- Kexanone. "librosa_loopfinder." [GitHub](https://github.com/Kexanone/librosa_loopfinder)
- internal: `reference_drops_detection_prior_art.md` — Foote novelty on stem-activity SSM, BPM-drift footguns, v1 threshold values.
- internal: `reference_audio_feature_stack.md` — L0/L1/L2 hierarchy, MFCC/chroma discriminators, installed deps.
tools/foundry/research/PHASE2_FINDINGS.md 0 → 100644
# Phase 2 — Loop Finder: findings & design (strand D synthesis)
> Synthesizes strands **A** (empirical, `A_loop_profile.md`), **B** (fundamentals,
> `B_fundamentals.md`), **C** (prior art, `C_prior_art.md`). 2026-06-22.
> This is the design `engine/loops.py` (task #5) and the GUI panel (task #6)
> implement against. Validation plan = task #7.
---
## 0. The model (corrected by PLN, 2026-06-23)
The real flow: **demucs a whole track → loop-find 1..x loops *per stem*,
independently.** Cuts are *often* time-aligned across stems (the same `[x,y]`
window grabbed for bass + synth + drums) — but this is a **tendency, not a rule**:
the count varies per stem and some stems are skipped entirely (humpty exported no
drums). Strand A's "one window → 4 stems" was too rigid.
So the finder works **per stem**: generate 1..x ranked loop candidates for each
stem. Then **cross-stem time-alignment is a grouping signal, not a constraint** —
candidates from different stems that share a time window are clustered into a
loose **Take** (a time region) the human can audition together, but each stem
still contributes its own count (0..x), name, and keep/drop decision.
PLN picks 1–4 "hook moments" per source, cuts **2- or 4-bar regions** (strand A's
duration peaks hold), and names by role+character (`vocals_dark_lips`,
`keys_brass3_MINOR`), never by stem name. Level is kept as-cut (no normalize).
**Section labels ground the loops (PLN, 2026-06-23).** Song-structure analysis
(intro / verse / chorus / bridge / drop / outro) gives each loop a *grounding
tag* — it labels, identifies, and locates the cut ("this is the chorus synth",
"the drop bass"). The §1 beat-sync SSM + Foote novelty already segments the
track; labeling those segments by structural role is a small step on top, and the
**sibling emotional-arc / section tool (#89) already does section + arousal
analysis on stems** — reuse its output rather than rebuild. Feeds: the finder
(rank/locate by section), naming (task #13: `chorus_synth`, `drop_bass1`), and the
human's mental map of a source. The arc tool consumes the Foundry engine for
stems → the Foundry's finder consumes the arc tool's sections. Clean loop.
```
ParVagues loop, empirically (strand A):
duration 2 or 4 bars @ 100–130 bpm → 3.7–8.0 s covers 85% of loops
(peaks: 4–5 s = 2-bar, 7–8 s = 4-bar; one-shots <2 s are 11%)
level RMS −12…−28 dBFS — moderate; DO NOT normalize (SuperDirt gain shapes it)
spectral skews low-mid (500–2000 Hz centroid), harmonic > noisy
yield keeps 27–58% of source as 1–4 moments × 4–12 elements = 9–24 loops/kit
```
---
## 1. The pipeline (strand C — zero new deps, ~5.4 s per 120 s stem)
All `librosa` + `scipy` + `numpy`, already in the tide-table venv. **madmom is
deferred** (NumPy-2.x incompatible — build-chain risk); so are BeatNet, laplacian
segmentation, and "Beat This!".
1. **Beat grid — drift-tolerant.** `librosa.onset.onset_strength` →
`librosa.beat.plp` (predominant local pulse) → `scipy.signal.find_peaks`.
PLP gives per-frame local tempo, so it tracks `setcps` drift. **Never use
`beat_track` as primary** (single-BPM DP; A confirms it's unreliable on this
material).
2. **Beat-synchronous SSM.** MFCC(13) ⊕ chroma_cqt(12) → `librosa.util.sync` onto
the beat grid → `librosa.segment.recurrence_matrix(mode='affinity')`. ~224×224
for a 120 s stem (~0.45 s). The beat axis absorbs tempo drift for free.
3. **Foote checkerboard novelty** on the *beat-sync* SSM, two passes: k=4 (1-bar)
and k=16 (4-bar). 13 ms on 224×224. **Never run frame-level** (5168×5168 = 27 s).
4. **Candidate windows** at {1, 2, 4, 8} bars between novelty peaks (A: try
[2, 4, 8] first — the empirical sweet spot).
5. **Score** each window (§3) and **dedup**.
6. **Zero-crossing snap** (<1 ms) at export, after the human picks.
---
## 2. The mechanical rulebook (strand B — what the machine must honour)
| # | Rule | Param |
|---|------|-------|
| B1 | Snap endpoints to **same-direction** zero crossings | ±10 ms / ±441 samp |
| B2 | Remove DC offset first (≤10 Hz HP / mean-sub); flag \|mean\|>1e-4 | before ZC search |
| B3 | **Equal-power** crossfade at seam (`√t`/`√(1-t)`), not linear | 20 ms sustained, **3 ms drums** |
| B4 | Bar length `samples = bars*4*60/bpm*sr`; quantize ends to beats | — |
| B5 | **No global BPM** — PLP per-region; store bpm_mean/min/max; flag >±5 BPM as tempo-unstable | drift hazard |
| B6 | Bass → **mono-sum** (`mean(axis=0)`); check inter-channel corr first | demucs bass is stereo |
| B7 | Drums: loop-in ≥5 ms **before** first transient; short fade only | protect attack |
| B8 | Output **24-bit PCM WAV @ 44100 Hz**; resample *before* ZC search | SuperDirt/SC rate |
| B9 | One-shot if < 0.75 s; min loop = 1 bar at local BPM | classify |
| B10 | **Do NOT normalize** (A overrides B's −1 dBFS default: PLN keeps stems as-cut; SuperDirt gain handles level). Offer optional peak-norm as a toggle. | — |
**The central conflict (B + A):** bar-align (snap to downbeat) vs zero-crossing
(snap to nearest ZC). Resolution: **always honor the musical downbeat**, let the
equal-power crossfade absorb the residual seam, and **never move >25 ms from the
detected beat**. A confirms this is safe: `loopAt` in the `.tidal` tracks already
tolerates 1–5% timing error (e.g. bombe_dj stretches a 7.83 s loop 1.1% — transparent),
so **bar-boundary alignment, not sample-perfect timing, is the real constraint**.
---
## 3. Loopability score (composite, per window)
```
score = w_struct * ssm_diagonal_similarity # does this window repeat? (structural)
+ w_novel * boundary_novelty # Foote peak strength at the edges
+ w_seam * (1 - seam_discontinuity) # waveform cross-corr at the loop seam
+ w_zc * zc_proximity # how close a clean ZC is to the beat
- penalty * tempo_instability # demote windows near setcps jumps
```
Defaults `w_struct=0.4, w_novel=0.2, w_seam=0.3, w_zc=0.1` — tune against §5.
The seam term is computed on the **summed mix** (a click anywhere is audible).
**Tempo self-check (A's most actionable finding):** for a candidate at bar-length
L and tempo T, the *implied* duration must match a clean bar count within ±2%. PLN's
own loops cluster to bit-identical durations per source — so if PLP's T and the
{2,4,8}-bar grid don't produce a self-consistent L, the tempo estimate is suspect →
flag, widen ZC window, demote.
---
## 4. Data model & engine API (`engine/loops.py`)
```python
class StemSlice(BaseModel): # one stem, cut to the window
name: str # drums|bass|other|vocals
path: str # written wav (relative to catch dir, until exported)
rms_dbfs: float
onset_rate: float
centroid_hz: float
is_one_shot: bool
class Take(BaseModel): # a bar-window = the unit PLN selects
start_sample: int; end_sample: int
start_beat: int; end_beat: int
bars: int # 1|2|4|8
bpm: float; bpm_min: float; bpm_max: float
tempo_unstable: bool # bpm range > ±5
score: float # composite (§3)
seam_discontinuity: float; novelty: float; structural: float
crossfade_ms: float; zc_residual_ms: float
stems: list[StemSlice] # all 4 (or fewer) from THIS window
# pure-ish; reuses tide-table sample_features where it can
def find_takes(catch, workspace, *, bars=(2,4,8), top_n=8,
progress=None) -> list[Take]: ...
# export: honor B-rules, write to Samples/<kit>/, then publish.link_kit(kit)
def export_take(take, kit, *, keep=("drums","bass","other","vocals"),
names=None, peak_norm=False) -> list[Path]: ...
```
`export_take` is the only side-effecting step; it applies B1–B10, writes to
`~/Work/Sound/Samples/<kit>/`, and calls the already-built
`engine.publish.link_kit` to symlink into Dirt-Samples (`s "<kit>"`).
---
## 5. The rubric is corpus-wide — grade everything, not just 3 pairs
> Reframe (PLN, 2026-06-23): the 3 GT pairs are too thin, and the whole point is
> bigger — **every existing kit is a hand-cut loop that the rubric can grade.**
> The §3 score is not just the finder's internal ranker; it is a **loop grade**
> applicable to any wav. So build it as a shared, standalone instrument.
### 5·0. The Correlation Engine — ground truth at corpus scale (PLN, 2026-06-23)
Don't hand-pair 3 kits — **correlate every `Samples/**/*.wav` against every
`separated/htdemucs/**/*.wav`** to auto-recover each loop's provenance: which
source track, which stem, the start offset, and the length. Because a hand-cut
loop *is literally a slice of the stem*, matching is high-confidence:
- align by normalized cross-correlation on a cheap embedding (beat-sync
chroma+MFCC, or a downsampled mel fingerprint) to find the offset; confirm with
sample-domain xcorr on the matched span.
- only the ~recent demucs-derived kits will match (separated/ holds 27 tracks vs
135 kits — older kits predate this workflow); **quantify coverage**, don't
pretend full.
What this buys (all measured, not assumed):
- **provenance map** `sample → (track, stem, offset, len)` — dozens of GT pairs, free.
- **cross-stem alignment stats** — the actual answer to "how often are loops
time-aligned across stems?" (the §0 tendency, quantified).
- **the cut critique** — compare each human cut to its source: was the boundary
bar-aligned? on a clean ZC? this is exactly what the finder must reproduce.
- feeds both the tierlist (5b) and the finder recall test (5c).
### 5a. The Loop Grader (the katana — `feedback_build_katana_first`)
A pure function `grade(wav) -> LoopGrade` applying the A-profile + B-rulebook to
**any** sample: seam discontinuity (loop the file head-to-tail, measure the click),
ZC cleanliness at ends, DC offset, bar-length self-consistency (does duration map
to a clean {1,2,4,8}-bar count at a detected tempo within ±2%?), level sanity,
bass mono-compatibility, one-shot-vs-loop class. Output a 0–1 grade + per-rule
sub-scores + flags. **This is the same code the finder uses to score candidate
windows** (DRY): the finder grades hypothetical windows, the tierlist grades
existing files.
### 5b. Foundry Hall of Fame / tierlist
Run the grader over all 135 kits / 1641 wavs → rank into tiers (S/A/B/C/D). Two
payoffs:
- **QA on the library** — surface the few hand-cut loops with audible seams,
DC offset, or off-grid durations (fixable); celebrate the exemplars.
- **Validate the rubric itself** — usage is implicit ground truth: the **9
core-palette kits** (used 21+ times in `.tidal`) and the 106 used kits should
tier *higher* than the 29 unused/staged kits. If they don't, the rubric is
wrong, not the library. Correlate grade vs `.tidal` usage count.
Render as a tierlist viz in the Ship's Bridge language (later `/impeccable` pass);
the grader emits JSON the viz consumes (parsers-over-copy, `feedback_parsers_over_copy`).
PII: aggregate/representative, not full filename dumps (`feedback_pii_sensitivity`).
### 5c. Finder recall (still useful, now secondary)
The 3 GT pairs remain a sharp **finder** test: does `find_takes(top_n=N)` surface
the windows PLN kept? Match within ±1 bar; tempo within ±2% (crimewave
121.8/127.2/128.5, diams_dj 122.6, humpty ~102). Wire as pytest IT with GT
durations as fixtures.
**v1 targets:** grade↔usage correlation positive & significant across the corpus;
core-palette kits ≥ B-tier; finder recall@8 ≥ 0.7 on the 3 GT kits; tempo ±2% on ≥2/3.
---
## 6. GUI (task #6) — fast chooser, never autocrop
Per-catch **Loops** panel: wavesurfer.js waveform of the summed mix with novelty
peaks + ranked candidate windows shaded. Click a window → audition all 4 stems
looping in sync. Human prunes stems, names by role, hits **Forge** → `export_take`
→ auto-link. Suggestions are *never* auto-applied; a wrong rank costs one click to
ignore (the house rule).
## 7. Biggest risk & mitigation
PLP needs ~8 s of context to converge after a `setcps` jump, so beat frames near
tempo transitions can be off 1–2 beats. Mitigate: widen the ZC-snap window for
candidates landing near SSM novelty peaks, let the composite score demote off-seam
windows, and lean on the human final-pick as the ultimate safety net.
## 8. Build order (PLN: all of it, 80/20 first, clean-tested)
**Decided 2026-06-23.** Each stage feeds the next; rubric validated before the
finder leans on it; the showcase lands early.
1. **Loop grader** (`engine/grade.py`) — pure `grade(wav)->LoopGrade` over §2
rules + §3 score: seam, ZC, DC, bar-length self-consistency, level, bass-mono,
one-shot class. The katana — unit-tested, reused by finder + tierlist. *80/20:
ship the unambiguous rules first (seam/ZC/DC/duration), refine spectral after.*
2. **Hall of Fame tierlist** (`grade_corpus.py` → JSON → impeccable viz) — grade
all 135 kits, rank S/A/B/C/D, flag flawed cuts, correlate grade vs `.tidal`
usage (rubric validation: core-palette kits must tier high). Full Ship's Bridge
showcase, audition-on-click.
3. **Correlation engine** (`engine/correlate.py`) — §5·0 provenance recovery;
enriches the tierlist with the cut-critique + yields GT at scale for #7.
4. **Loop finder** (`engine/loops.py`) — §1 pipeline, **per-stem** candidates +
loose cross-stem grouping (§0), scored by the grader's loopability rubric (DRY).
Then `export_take` (B1–B10) + `publish.link_kit`.
5. **GUI loop panel** (task #6) → **finder recall IT** (task #7, uses #3's GT).
6. **engine2 Roformer** (tasks #8→#9) — independent; cleaner stems anytime.
Commit per stage (`feedback_commit_often`); branch, not master.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment