Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
  • This project
    • Loading...
  • Sign in / Register
T
Tidal
  • Overview
    • Overview
    • Details
    • Activity
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 0
    • Issues 0
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • PLN
  • Tidal
  • Repository

Switch branch/tag
  • Tidal
  • tools
  • foundry
  • engine
  • correlate.py
Find file
BlameHistoryPermalink
  • PLN (Algolia)'s avatar
    feat(foundry): correlation engine — recover loop provenance at scale (#12) · 7a79dc83
    engine/correlate.py recovers where each hand-cut loop was cut from: a loop is a
    literal slice of a separated stem, so normalized cross-correlation of the waveform
    has a razor peak at the true offset (true slice ~1.0, foreign ~0.03). build_provenance.py
    runs it over the whole corpus → provenance.json.
    
    Result (honest): 482/1690 loops matched (29%) across 28 kits — only the recent
    demucs-derived kits have a source in separated/; the rest predate the workflow, and
    we quantify that rather than pretend. Recoveries are exact and name-confirming:
    bumbum←MC Fioti "Bum Bum Tam Tam", diams_dj←Diam's "DJ", like_sugar←Chaka Khan
    "Like Sugar", praise←Fatboy Slim. Cross-stem alignment 0.34 — the measured answer to
    §0's open question (loops share a window across stems about a third of the time).
    
    Two bugs caught and fixed during the build, both load-bearing:
    - NAIVE all-pairs full-res NCC took 56 MIN on 1690×120. Fix: coarse-to-fine —
      rank every stem by NCC on a frame-RMS *energy envelope* (phase-robust, ~40× cheaper),
      then run the precise sample-domain NCC only on the top-3 candidates. Minutes, not an hour.
    - SILENT-WINDOW BLOWUP: catastrophic cancellation in the sliding-variance cumsum
      drove the normaliser to ~0 in quiet stem regions, so silence scored ∞ and
      masqueraded as a perfect match (an early run reported a bogus 55%/0.69 — ~440 false
      positives). Fix: floor the window norm at 5% of the stem's global energy and clamp
      NCC to [-1,1]. Scores are now all ≤1.0 (max 1.000, median 0.969); the honest 29%
      is what survived. Regression test added.
    
    forkserver gotcha: py3.14's default mp start method re-imports the module per worker
    (empty globals), so the preloaded-stems COW trick silently fails — pinned mp fork
    context. 8 correlate tests (NCC offset/foreign/gain-invariance/silent-window, envelope
    prefilter, locate top-k). Feeds the tierlist cut-critique and the finder recall eval (#7).
    PLN (Algolia) authored Jun 23, 2026
    7a79dc83
correlate.py 5.68 KB
EditWeb IDE
×

Replace correlate.py

Attach a file by drag & drop or click to upload


Cancel
A new branch will be created in your fork and a new merge request will be started.