Commit 56c81e57 by PLN (Algolia)

feat(foundry): close-the-loop publish + phase-2 loop-finder charter

- engine/publish.py: link_kit() symlinks Samples/<kit> → Dirt-Samples/<kit>
  so a freshly-cut kit loads as s "<kit>" (325/508 Dirt-Samples entries are
  this pattern). Idempotent; repoints stale links; refuses to clobber real dirs.
- foundry.py link <kit> CLI; 5 new tests (15 total, all green)
- PHASE2_LOOPFINDER.md: research charter for the loop-finder (Sonnet-agent
  deep-research pass) — grounded in the REAL flow (yt → separated/htdemucs →
  hand-cut → Work/Sound/Samples → ln → Dirt-Samples → .tidal), with the
  crimewave/diams_dj/humpty stem→loop pairs as free ground truth
- README: export target corrected to Work/Sound/Samples + link step
parent 9eb2e66f
# Phase 2 — The Loop Finder · research charter
> Status: **research pending**. This charter scopes a deep-research pass (Sonnet
> 4.6 agents) that precedes any `engine/loops.py` code. engine1 (dl + stem) is
> live; this is the Audacity-replacement half.
## The goal, in one line
From a separated stem (or the source), surface **bar-aligned, click-free loop
candidates ranked by musical usefulness** — automating the *mechanical* drudgery
so PLN spends ears only on the *creative* call: which loop is the one.
## Design philosophy — automate the mechanical, reserve the ear
The machine is allowed to decide anything **objective and verifiable**; the human
keeps everything **subjective and musical**. This is the house rule already in
force for mastering (machine catches gross errors, ears judge feel —
`feedback_verify_own_renders`, `feedback_mastering_eda`).
| Machine owns (mechanical) | Human owns (creative) |
|---|---|
| tempo / beat / downbeat grid | *which* candidate is musical |
| bar-length region enumeration (1/2/4/8 bars) | the vibe / genre fit |
| zero-crossing snap (click-free in/out) | naming + final trim nudge |
| loopability score (seam discontinuity, self-similarity) | one-shot vs loop intent |
| energy / novelty ranking, dedup | what to actually keep |
A wrong machine guess must cost **zero** to override — candidates are suggestions
on a waveform, never auto-applied. The UI is a *fast chooser*, not an autocropper.
## The real flow this replaces (grounded 2026-06-22)
```
yt → ~/Downloads/separated/htdemucs/<track>/ (demucs stems; 27 kits, 9 recent)
→ hand-cut in Audacity
→ ~/Work/Sound/Samples/<kit>/ (named loops; 135 kits, 7 recent)
→ ln -s into Dirt-Samples/<kit> (so SuperDirt scans it)
→ used in .tidal as e.g. "crimewave", "diams_dj", "humpty"
```
`~/Work/Sound/Samples/` is the **custom sample root** loops land in — then a
symlink `Dirt-Samples/<kit> → Samples/<kit>` closes the loop so SuperDirt loads
it as `s "<kit>"` (325/508 of Dirt-Samples entries are exactly this). **Both
steps are already implemented mechanically** in `engine/publish.py` +
`foundry.py link <kit>`; the loop-export UI just calls them. Kits are bimodal
(sub-second one-shots ↔ multi-minute stems), so "loop" here spans both — strand
A must characterize the real distribution.
**Ground truth is free:** `crimewave` (24 wavs), `diams_dj` (16), `humpty` (9)
each pair a source stem (in `separated/`) with the loops PLN's ear actually cut
(in `Samples/`). That pairing is the validation set — what did the human keep,
and can the finder surface it in its top candidates?
## What we already have (reuse, don't reinvent)
- **tide-table feature stack**`sample_features.py`, `sample_classify.py`,
`sample_semantics.py` (CLAP), `sample_ontology.py`, `audio_lens.py`. The loop
ranker should sit ON this (`reference_audio_feature_stack`: L0 librosa → L1 MDA
→ L2 semantic), not grow a parallel feature path.
- **Foundry engine1** — stems already land at `<catch>/stems/{drums,bass,other,
vocals}.wav` with a `Catch` model + Range-serving GUI to hang a waveform on.
- **Prior art notes**`reference_drops_detection_prior_art` (Foote novelty on
stem-activity SSM, kick-snap, **livecoded BPM-drift footguns**) is directly
relevant: the same drift that breaks boundary detection breaks naive looping.
## Research questions (the three strands to fan out)
**A · Empirical — what is a "ParVagues loop"?**
EDA the **real custom sample root `~/Work/Sound/Samples/`** (135 kits, esp. the 7
recent + the demucs-derived ones: crimewave, diams_dj, humpty) — and the paired
source stems in `~/Downloads/separated/htdemucs/`. Characterize: per-kit wav
counts, the one-shot↔loop duration split, level, character, and *what the human
kept* vs the source stem. Cross-ref how the 702 `.tidal` tracks call these named
samples. Output: the target loop profile the finder should aim for, grounded in
what PLN actually cuts and plays.
**B · Theory — sampling & looping fundamentals.**
What makes a loop seamless and usable: zero-crossing & equal-power crossfade
seams, bar/phrase alignment, tempo-locking vs free, mono-sum for bass, sample-rate
/ bit-depth for Dirt, one-shot vs loop conventions. Output: the rulebook the
mechanical layer must honour.
**C · Prior art — loop-detection algorithms & libraries.**
Survey: librosa (beat_track, onset, tempo, plp), madmom (DBN downbeats), aubio,
self-similarity-matrix / Foote novelty for loop points, autocorrelation/beat-sync
for seamless cuts, existing "auto loop extractor" repos. Output: a shortlist of
techniques with accuracy/latency/dep trade-offs on **our livecoded, tempo-drifting
material**, plus the recommended stack for v1.
**D · Synthesis — the `engine/loops.py` design.**
Fold A+B+C into: the candidate-generation pipeline, the loopability score, the
`LoopCandidate` data model (start/end samples, bars, bpm, score, snap method), the
engine API, and the GUI panel (waveform + draggable regions + audition + "send →
`_samples/<pack>/` with a Tidal name"). Output: a design doc + a thin spike.
## Deliverables
1. `PHASE2_FINDINGS.md` — the synthesized research report (A+B+C), cited.
2. `engine/loops.py` design (API + `LoopCandidate` model) — in the report or a stub.
3. A **validation plan**: a few hand-picked ground-truth loops from real catches to
test the finder against (build the katana before cutting — `feedback_build_katana_first`).
## Constraints & footguns
- **BPM drift is real** (livecoded, no click) — never assume a global constant
tempo; detect per-region or track drift (`reference_drops_detection_prior_art`).
- **Validate by analysis, never by stem name**`other.wav` is not "melody",
`bass.wav` may carry a wobble not a sub (`feedback_mastering_eda`).
- Reuse the feature stack; keep deps to what the tide-table venv already has where
possible (librosa present); flag any new heavy dep (madmom) explicitly.
- Output WAVs Dirt can eat (sample rate / mono-sum policy decided in strand B).
...@@ -111,7 +111,10 @@ loop is musical, naming, the creative cut. ...@@ -111,7 +111,10 @@ loop is musical, naming, the creative cut.
- **Catch review** — the card list is the review surface; per-stem `<audio>` - **Catch review** — the card list is the review surface; per-stem `<audio>`
(Range-seekable). Phase 2 adds waveforms (wavesurfer.js) + loop regions. (Range-seekable). Phase 2 adds waveforms (wavesurfer.js) + loop regions.
- **Save / export** — today: `⬇ wav` per stem. Phase 2: "send loop → sample - **Save / export** — today: `⬇ wav` per stem. Phase 2: "send loop → sample
folder" with a Tidal-friendly name straight into `_samples/<pack>/`. folder" with a Tidal-friendly name into **`~/Work/Sound/Samples/<kit>/`** (the
real custom SuperDirt sample root — where crimewave/diams_dj/humpty already
live), then `foundry.py link <kit>` symlinks it into Dirt-Samples so it loads
as `s "<kit>"`. The link step is live today (`engine/publish.py`).
- **Settings** — engine + shifts live in the GUI opts bar; later: persisted - **Settings** — engine + shifts live in the GUI opts bar; later: persisted
defaults (engine, output sample-rate, target folder) in a `foundry.config.json`. defaults (engine, output sample-rate, target folder) in a `foundry.config.json`.
- **Analyse** — reuse `tools/analyze_samples.py` / tide-table `audio_lens.py` for - **Analyse** — reuse `tools/analyze_samples.py` / tide-table `audio_lens.py` for
......
"""publish — close the loop: make a sample kit loadable by SuperDirt.
ParVagues' custom kits live in `~/Work/Sound/Samples/<kit>/` and become playable
as `s "<kit>"` via a symlink in Dirt-Samples (325/508 of PLN's Dirt-Samples
entries are exactly this). The Foundry's last step links a freshly-cut kit so it
drops straight into a livecoding session — no manual `ln -s`.
from engine.publish import link_kit, samples_root, dirt_samples
"""
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path
def samples_root() -> Path:
"""Custom sample root SuperDirt kits live in. Override $FOUNDRY_SAMPLES."""
return Path(os.environ.get("FOUNDRY_SAMPLES", str(Path.home() / "Work" / "Sound" / "Samples")))
def dirt_samples() -> Path:
"""The Dirt-Samples dir SuperDirt scans. Override $FOUNDRY_DIRT_SAMPLES."""
return Path(os.environ.get(
"FOUNDRY_DIRT_SAMPLES",
str(Path.home() / ".local/share/SuperCollider/downloaded-quarks/Dirt-Samples"),
))
@dataclass
class LinkResult:
kit: str
src: Path # the real kit dir (in Samples)
link: Path # the symlink in Dirt-Samples
action: str # "created" | "already-linked" | "repointed" | "skipped-real-dir"
def link_kit(kit: str, *, samples: Path | None = None, dirt: Path | None = None) -> LinkResult:
"""Symlink Dirt-Samples/<kit> → Samples/<kit> so `s "<kit>"` resolves.
Idempotent. Refuses to clobber a *real* directory already at the target
(we never delete what we didn't create) — that's surfaced, not overwritten.
"""
samples = samples or samples_root()
dirt = dirt or dirt_samples()
src = (samples / kit).resolve()
if not src.is_dir():
raise FileNotFoundError(f"no kit dir to publish: {src}")
dirt.mkdir(parents=True, exist_ok=True)
link = dirt / kit
if link.is_symlink():
if link.resolve() == src:
return LinkResult(kit, src, link, "already-linked")
link.unlink() # repoint a stale symlink (safe: it's a link, not data)
link.symlink_to(src)
return LinkResult(kit, src, link, "repointed")
if link.exists(): # a real dir/file we didn't make — don't touch it
return LinkResult(kit, src, link, "skipped-real-dir")
link.symlink_to(src)
return LinkResult(kit, src, link, "created")
...@@ -11,6 +11,7 @@ shells out to dedicated venvs (demucs today, Roformer next). ...@@ -11,6 +11,7 @@ shells out to dedicated venvs (demucs today, Roformer next).
python3 foundry.py catch <url> # fetch + sep (the full pipeline) python3 foundry.py catch <url> # fetch + sep (the full pipeline)
python3 foundry.py list # list catches in the workspace python3 foundry.py list # list catches in the workspace
python3 foundry.py backends # show separation backends python3 foundry.py backends # show separation backends
python3 foundry.py link <kit> # link Samples/<kit> into Dirt-Samples
python3 foundry.py serve [--port 8765] # launch the web GUI python3 foundry.py serve [--port 8765] # launch the web GUI
Workspace defaults to $FOUNDRY_HOME or ~/Downloads/foundry; override with Workspace defaults to $FOUNDRY_HOME or ~/Downloads/foundry; override with
...@@ -85,6 +86,14 @@ def cmd_backends(a): ...@@ -85,6 +86,14 @@ def cmd_backends(a):
print(f"{'✓' if ok else '✗'} {name:10} {hint}") print(f"{'✓' if ok else '✗'} {name:10} {hint}")
def cmd_link(a):
from engine import publish as P
r = P.link_kit(a.kit)
print(f"{r.action}: {r.link} → {r.src}")
if r.action == "skipped-real-dir":
raise SystemExit(f"⚠ {r.link} is a real directory, not a symlink — left untouched")
def cmd_serve(a): def cmd_serve(a):
import server import server
server.run(workspace=a.workspace or F.default_workspace(), port=a.port) server.run(workspace=a.workspace or F.default_workspace(), port=a.port)
...@@ -109,6 +118,7 @@ def main(argv=None): ...@@ -109,6 +118,7 @@ def main(argv=None):
sp = sub.add_parser("catch"); sp.add_argument("url"); add_sep_opts(sp); sp.set_defaults(fn=cmd_catch) sp = sub.add_parser("catch"); sp.add_argument("url"); add_sep_opts(sp); sp.set_defaults(fn=cmd_catch)
sub.add_parser("list").set_defaults(fn=cmd_list) sub.add_parser("list").set_defaults(fn=cmd_list)
sub.add_parser("backends").set_defaults(fn=cmd_backends) sub.add_parser("backends").set_defaults(fn=cmd_backends)
sp = sub.add_parser("link"); sp.add_argument("kit"); sp.set_defaults(fn=cmd_link)
sp = sub.add_parser("serve"); sp.add_argument("--port", type=int, default=8765); sp.set_defaults(fn=cmd_serve) sp = sub.add_parser("serve"); sp.add_argument("--port", type=int, default=8765); sp.set_defaults(fn=cmd_serve)
a = p.parse_args(argv) a = p.parse_args(argv)
......
...@@ -90,3 +90,49 @@ def test_unknown_backend_raises(): ...@@ -90,3 +90,49 @@ def test_unknown_backend_raises():
import pytest import pytest
with pytest.raises(KeyError): with pytest.raises(KeyError):
S.get_backend("nope") S.get_backend("nope")
# ── publish: close-the-loop symlink into Dirt-Samples ───────────────────────
from engine import publish as P # noqa: E402
def test_link_kit_creates_symlink(tmp_path):
samples = tmp_path / "Samples"; dirt = tmp_path / "Dirt"
(samples / "crimewave").mkdir(parents=True)
r = P.link_kit("crimewave", samples=samples, dirt=dirt)
assert r.action == "created"
assert (dirt / "crimewave").is_symlink()
assert (dirt / "crimewave").resolve() == (samples / "crimewave").resolve()
def test_link_kit_idempotent(tmp_path):
samples = tmp_path / "Samples"; dirt = tmp_path / "Dirt"
(samples / "k").mkdir(parents=True)
P.link_kit("k", samples=samples, dirt=dirt)
assert P.link_kit("k", samples=samples, dirt=dirt).action == "already-linked"
def test_link_kit_repoints_stale(tmp_path):
samples = tmp_path / "Samples"; dirt = tmp_path / "Dirt"
(samples / "k").mkdir(parents=True)
(tmp_path / "old").mkdir()
dirt.mkdir()
(dirt / "k").symlink_to(tmp_path / "old")
r = P.link_kit("k", samples=samples, dirt=dirt)
assert r.action == "repointed"
assert (dirt / "k").resolve() == (samples / "k").resolve()
def test_link_kit_refuses_to_clobber_real_dir(tmp_path):
samples = tmp_path / "Samples"; dirt = tmp_path / "Dirt"
(samples / "k").mkdir(parents=True)
(dirt / "k").mkdir(parents=True) # a real dir we didn't create
r = P.link_kit("k", samples=samples, dirt=dirt)
assert r.action == "skipped-real-dir"
assert not (dirt / "k").is_symlink()
def test_link_kit_missing_source_raises(tmp_path):
import pytest
with pytest.raises(FileNotFoundError):
P.link_kit("nope", samples=tmp_path / "Samples", dirt=tmp_path / "Dirt")
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment