-
feat(foundry): correlation engine — recover loop provenance at scale (#12) · 7a79dc83
engine/correlate.py recovers where each hand-cut loop was cut from: a loop is a literal slice of a separated stem, so normalized cross-correlation of the waveform has a razor peak at the true offset (true slice ~1.0, foreign ~0.03). build_provenance.py runs it over the whole corpus → provenance.json. Result (honest): 482/1690 loops matched (29%) across 28 kits — only the recent demucs-derived kits have a source in separated/; the rest predate the workflow, and we quantify that rather than pretend. Recoveries are exact and name-confirming: bumbum←MC Fioti "Bum Bum Tam Tam", diams_dj←Diam's "DJ", like_sugar←Chaka Khan "Like Sugar", praise←Fatboy Slim. Cross-stem alignment 0.34 — the measured answer to §0's open question (loops share a window across stems about a third of the time). Two bugs caught and fixed during the build, both load-bearing: - NAIVE all-pairs full-res NCC took 56 MIN on 1690×120. Fix: coarse-to-fine — rank every stem by NCC on a frame-RMS *energy envelope* (phase-robust, ~40× cheaper), then run the precise sample-domain NCC only on the top-3 candidates. Minutes, not an hour. - SILENT-WINDOW BLOWUP: catastrophic cancellation in the sliding-variance cumsum drove the normaliser to ~0 in quiet stem regions, so silence scored ∞ and masqueraded as a perfect match (an early run reported a bogus 55%/0.69 — ~440 false positives). Fix: floor the window norm at 5% of the stem's global energy and clamp NCC to [-1,1]. Scores are now all ≤1.0 (max 1.000, median 0.969); the honest 29% is what survived. Regression test added. forkserver gotcha: py3.14's default mp start method re-imports the module per worker (empty globals), so the preloaded-stems COW trick silently fails — pinned mp fork context. 8 correlate tests (NCC offset/foreign/gain-invariance/silent-window, envelope prefilter, locate top-k). Feeds the tierlist cut-critique and the finder recall eval (#7).
PLN (Algolia) authored7a79dc83
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| __init__.py | Loading commit data... | |
| correlate.py | Loading commit data... | |
| fetch.py | Loading commit data... | |
| grade.py | Loading commit data... | |
| loops.py | Loading commit data... | |
| model.py | Loading commit data... | |
| naming.py | Loading commit data... | |
| publish.py | Loading commit data... | |
| separate.py | Loading commit data... |