# The Foundry — TODO

Resume state for the URL→audio→stems→loops sampling tool. Snapshot **2026-08-21**.
Task IDs map to the session task board. Design: memory `project_foundry`.

## 🔵 In progress
- [ ] **#19 — Auto-tune loop** (create samples → grade → iterate sampling settings).
      PLN's closed-feedback main quest. **Harness BUILT & VALIDATED** (commit
      `6f85012`): `engine/autotune.py` (evaluate/search), `autotune.py` CLI,
      `loops.analyze_stem` `weights` override, 4 unit tests + a real run (baseline
      0.690 → best 0.763, **+10.5%** on 2 drum stems). Finder defaults (`loops.W`)
      deliberately UNCHANGED. **Exact next steps to resume cold:**
      1. Get real fuel: `cd tools/foundry && python3 foundry.py demos --all --sep`
         (fetches+separates the 10 sampling classics — SLOW, run as a durable bg
         job). Existing fuel: `~/Downloads/separated/htdemucs/*/{drums,bass,vocals,
         other}.wav` (27 tracks).
      2. Add the anti-gaming 2nd objective: recall-vs-provenance-GT (reuse
         `armada/tide-table/build_finder_eval.py` + `build_provenance.py`) so we
         don't just amplify the seam/zc confound. Make objective multi-objective
         (grade × recall) in `engine/autotune.py`.
      3. Per-stem-ROLE tuning: vocals want chop mode + different weights than drums
         (`--stems …/drums.wav` vs `…/vocals.wav`; analyze_chops path for vocals).
      4. Full campaign: more stems (`--max-stems`↑) + more configs (`--n`↑); factor
         window-feature extraction out of `analyze_stem` so weight sweeps are
         instant (extract-once/score-many — the scarcity-minded speedup).
      5. THEN adopt a VALIDATED profile as the new `loops.W` default, per role,
         with the lift numbers.
      Run: `python3 autotune.py --stems <glob> --n 24 --max-stems 4`.
      Objective so far = mean full-rubric grade × coverage.
      **VERIFY-RERANK now BUILT & validated end-to-end on the Super freak dub kit.**
      The finder scored the seam on the RAW window, but export zc-snaps (B1) — and the
      snap (plus the 3-dp start/end rounding export slices at) changes the wrap. Several
      proxy-seam≈1.0 windows CLICKED after the snap. Fix: `loops._verify_rerank` re-scores
      the ~3×top_n survivors' seam on the TRUE post-snap window (reusing `grade.seam_score`,
      DRY), swaps proxy→true seam in the composite, and hard-demotes true-seam <
      `VERIFY_SEAM_FLOOR`. New kwarg `analyze_stem(..., verify=True)` (default on; False
      reproduces the pre-#19 raw-proxy ranking for autotune baselines). Also fixed the
      sparse-material seam false-positive (`grade.seam_score`): the mean-|2nd-diff|
      denominator collapsed to ≈0 on sparse dub drums (129× click ratio on a clean loop);
      now 90th-percentile, amplitude-floored (2–3× on the same loop). Validation
      (drums+other, bars 1/2/4, BPM 132.5, export-faithful grades):
        drums top-3  BEFORE C/C/A  →  AFTER **S/S/A** (clicking @93.08 & @24.06 demoted)
        other top-3  BEFORE B/B/B  →  AFTER **S/S/S** (bad favorite @56.74 demoted out)
      +2 rerank regression tests + 1 sparse-percussion test; suite 57→60 green.

## 🟢 Landed (2026-08-21) — #20 batch, via producer stem packs

`engine/stempack.py` + `engine/roles.py` + `engine/kitcheck.py` + `kitgate.py` +
`restage.py`, branch `claude/fred-kits`, 14 commits, suite 86 → 112.
Full write-up: `armada/tasks/027-the-label-and-the-lens.md`, memory
`reference_stempack` + `feedback_label_and_lens`.

```bash
cd tools/foundry
python3 -m engine.stempack "<pack>" --jobs 9 --no-link --out /tmp/cut
python3 kitgate.py /tmp/cut/fred_kits.json            # then --link
```

## 🟡 Landed but DEFECTIVE (2026-08-22) — the Fred pack shipped with holes in it
> **Do not treat this pack as good.** See **#24** below: 12 of 111 loops contain dead
> air the grade never looked for. PLN found two of them in the first kit he opened.

150 samples · 43 kits · 9 tracks, staged at `~/Work/Sound/Samples/Fred/output/<kit>/`
(PLN's choice) and symlinked into Dirt-Samples. `s "fred_marea_drums" # n 3` resolves.
Record: `packs/fred/{fred_kits.json,.md,.tidal,kitcheck.md}`. Merged to `master`
(`0f27184`); branch `origin/claude/fred-kits` kept as history.

    S×84 A×28 B×38 · 111 loops + 39 chops · 150 of 150 distinct sounds
    restage 0/43 · 0 grade mismatches · kitcheck 0 failing · n indices verified
    bank-check: "No shadowed bank names" · rights 43/43 third_party w/ provenance

Ground truth held: MAREA declares `123BPM`, grid read **123.05**.
PLEASEMAKEITBETTER at 20% tempo confidence correctly shipped chops only.

**Six cuts, five discarded, each by looking at the output** — 161 (dead bars), 153
(rotation), 153 (**24 files were digital silence graded A/B**), aborted (staging root),
129 (silence filtered at export only, so BIGHEN's tonal kit lost 3 loops instead of
gaining 3 real ones). Full account: `armada/tasks/028-la-cale-and-the-empty-loop.md`,
memory `feedback_presence_is_a_precondition`.

**Audition first** (6 kitcheck advisories, weak bar periodicity — bar-aligned but not
repeating units): `marea_drums/10_marea_4b` and `11_marea_4b` (0.12 / 0.08 vs a 0.30
drums floor — the suspicious pair), then `angie_tonal/03_padstuff_2b`,
`angieinterlude_bass/00_pad_2b`, `bighen_tonal/01_synth_2b`, `yasminah_tonal/00_keys_4b`.
Plus 20 quiet / 17 wide / 2 marginal seams, reported not blocked. **Awaiting PLN's ears.**

To re-cut or cut another pack:

```bash
cd tools/foundry
python3 -u -m engine.stempack "<pack dir>" --no-link \
    --samples-root "$HOME/Work/Sound/Samples/<pack>/output" --out <fresh dir>
python3 restage.py --cuts <out>/fred_kits.json --dry-run    # expect 0
python3 kitgate.py <out>/fred_kits.json                     # 5 checks
python3 ../../armada/tide-table/register_fred_rights.py <out>/fred_kits.json
python3 kitgate.py <out>/fred_kits.json --link
python3 kitindex.py --root <staging root> --root "$HOME/Work/Sound/Samples" \
    --cuts <out>/fred_kits.json                             # the index La Cale reads
```

`--jobs` defaults to half the cores; children run at nice 19 / SCHED_IDLE.

## 🟢 Landed (2026-08-21) — #21 La Cale, the kit auditioner

`armada/ui/kits.html` + `src/kits/` + `tools/foundry/kitindex.py`. Browse every kit
under the sample root (the 111 that predate the Foundry included), audition a sample
untouched or looped, rack several on one bar grid at a chosen tempo, export a
paste-ready `.tidal` block. The rack plays `rate = dur / (bars * barlen)` — literally
`loopAt` — so what you hear is what the rig does. 13 vitest cases on the export.

```bash
cd armada/ui && nvm use 22 && npm ci && npm run dev     # /kits.html
python3 ../serve.py --dir dist --port 8792              # LAN / phone
cd ../../tools/foundry && python3 kitindex.py --all     # regenerate the index
```

Open follow-ups, none blocking: the Strudel tab emits text only (no `@strudel/web` —
the deck already plays the rack and two audio graphs could disagree); the rack has no
swing control wired to the UI though the engine supports it; and `?kits=fred_*` narrows
the rail but there is no way to save a rack as a named preset.

## 🔴 Open
- [ ] **#24 — Loops have dead air inside them; every presence check is an AGGREGATE.**
      **PLN's verdict, 2026-08-22: "tbh i dont think the loops are usable."** He opened
      one kit and found two defects, both graded **tier S with zero flags**:

      - `fred_angie_bass:0` (`00_bass_8b`, grade **0.991**) — *"it starts, then silence,
        then starts again"*. Beats 7-9 of 32 are **digital silence** (-52, -88, -88 dBFS).
        Beat map: `#######..#######################`
      - `fred_angie_bass:2` (`02_bass_2b`, grade **0.920**) — *"why is this not trimmed,
        silence before silence after?"* 528 ms head (1.17 beats) + 436 ms tail dead.
        The bass enters on beat 2, so `loopAt 2` puts silence on the downbeat.
        Beat map: `.#######`

      **Not isolated — 12 of 111 loops (11%), and 5 of 14 bass loops (36%):**

          fam    hit/total        the three shapes
          bass    5/14  35.7%     head>=1 beat: 4   tail>=1 beat: 5   interior gap>=2 beats: 5
          drums   4/45   8.9%     worst: bighen_bass/01_bass_8b  gap=5 beats, occ 62.5%, S 0.983
          vox     2/24   8.3%             yasminah_drums/07_hidrums_1b  head=3 of 4, occ 25%, S 0.909
          tonal   1/20   5.0%             dermot_bass/00_bass_8b  gap=4, occ 87.5%, S 0.948
          fx      0/8    0.0%

      **Root cause — the same bug one level down.** Silence-graded-A was fixed twice this
      session (at export, then before ranking) but *both* fixes are aggregates: file peak,
      whole-file RMS, `max(beat_peak)`. **An aggregate cannot see a hole.** A loud loop with
      a silent bar inside it passes every one of them. The dead-bar rejector in `kitcheck`
      has **bar** resolution, so a 2.5-beat hole straddling a bar boundary is invisible —
      angie:0's bars read -18.3 and -15.9 dBFS while the beats between them read -88.

      **The lens exists already**: `loops._beat_peaks` builds the exact per-beat table.
      New diagnostic committed: **`tools/foundry/beatocc.py`** — bins the envelope on the
      loop's own beat grid (bpm/bars from the manifest, never re-estimated) and reports
      *shape* (head/tail dead beats, longest interior run, occupancy) instead of level:

          cd tools/foundry
          python3 beatocc.py packs/fred/fred_kits.json                        # the 12
          python3 beatocc.py packs/fred/fred_kits.json --kit fred_angie_bass -v   # beat maps

      **Open decisions — needs data, do not guess:**
      1. **Occupancy floors must be PER FAMILY**, exactly like the periodicity floors.
         `marea_vox/06_breaths_1b` at 25% occupancy is a *breath*, not a defect; a bass
         loop at 62.5% is broken. Measure the medians before placing a floor.
      2. **Head/tail silence: trim, rotate, or reject?** Rotation can't fix angie:2 — the
         offset is 1.17 beats and `_rotate_to_downbeat` snaps to beats, and rolling only
         moves the hole to the end. Trimming changes the length so it's no longer
         `loopAt`-able at an integer bar count. Rejecting may be the honest answer.
      3. **Interior gaps: reject, or accept as a musical rest?** A 2-beat rest in a bassline
         is real music; 5 beats of nothing in an 8-bar loop is a bad window. The finder's
         window ranking should probably *penalise* dead beats rather than the gate rejecting
         them outright — that puts it in #19's weight-tuning loop.
      4. Whichever way it lands, `kitcheck`'s dead-bar check must move from **bar** to
         **beat** resolution, and `kitgate` must gain a shape check. Then **re-cut Fred**
         (`packs/fred/` has the exact command block above) and re-audition.

      Lesson banked: `feedback_presence_is_a_precondition`, `feedback_measure_the_time_axis`.

- [ ] **#22 — Point stempack at the rest.** Same driver, no new code:
      the 13 ACAPELLAS + 13 INSTRUMENTALS in `~/Work/Sound/Samples/Fred/` (already
      44.1 kHz, so no resample; acapellas want chops mode), and the ~27 demucs catches in
      `~/Downloads/separated/htdemucs/`. Each needs a `--prefix`.
- [ ] **#23 — Run `kitcheck`/`restage` over the whole existing corpus.** The dead-bar and
      periodicity tests are general, and `Samples/` has 163 kits that have never been
      audited; hand-cut kits also carry whatever level their source had.
      `python3 restage.py --all-under ~/Work/Sound/Samples --dry-run` is the first look.
      Expect "grades S, useless" to be non-rare.
- [ ] **#20b — Batch-explore the merged corpus** (the original #20 framing).
      Workflow is one-catch-at-a-time; build batch exploration:
      - Batch driver: `find_takes` over MANY source stems (the ~30 in
        `~/Downloads/separated/htdemucs` + new fetches) → grade → surface the best
        loops/chops corpus-wide ("what's worth sampling across everything pulled").
      - **Stem-role-aware**: skip near-silent stems, mode per role (chops for
        vocals, loops for rhythmic), rank by grade. (e.g. Loituma has no drums —
        bass + high voice are the interesting stems; grid-from-drums already falls
        back drums→bass→first.)
      - Surface in the Hall of Fame / a "fresh catches" view: best ungraded-yet
        candidates corpus-wide, click to audition + forge.
      - Ties to #19: once settings are tuned per role, batch-explore applies them
        at scale. Relates #11 tierlist (showcase pattern), #9 merge.

> Note: the Foundry engines are also productized over HTTP by the **Fourier**
> audio API — `/separate` (#30) wraps `engine.separate`, `/loops /grade /correlate`
> (#31) wrap the rest. See `armada/api/TODO.md`.
