Commit 2c5b68fd by PLN (Algolia)

fix(foundry): the probe was looking in the wrong place, with the wrong lens

Three separate errors, each of which quietly threw away good material.

**Where it looked.** Evenly-spaced probe windows sample the gaps. A stem is not
stationary — measured on ANGIE, even windows read HIHATS at -60.9 dBFS / 1.4%
active where the full file says -48.8 / 9.7%, and missed PAD STUFF's material
entirely (0.0% vs 6.9%) because that stem only plays in two sections. Windows are
now ranked by energy and chosen where the stem is actually playing.

**What it measured.** Usability was gated on overall RMS, which on a sparse stem
measures how much silence it contains, not whether it has material in it. ANGIE's
HIHATS stem is -48.8 dBFS RMS and a perfectly good hi-hat stem peaking at -14.7.
Level and activity now come from the whole file (it is only a frame-RMS pass) and
the gate is peak plus activity relative to the stem's own peak. Three ANGIE stems
and two MAREA stems come back from the dead.

**What it concluded.** "Mostly low energy, low centroid ⇒ bass" fires on a drum bus,
because a modern kick carries most of a drum mix's energy: ALL DRUMS reads lf 0.86,
centroid 216 Hz, indistinguishable from BASS on those features. So ALL DRUMS was
being filed into the bass kit AND mono-summed, and KICK with it. Onset density is
asked first now (ALL DRUMS 5.5/s vs BASS 1.1/s), and a `hits` claim survives a bass
measurement, because a kick IS low and slow and only the label says it is one hit.

And the authority question underneath all of it. The house rule is that a sound's
role is never INFERRED from its name; it is not that a producer's label about their
own stem is worthless. Those have very different evidential weight. Presence and
bass are decided by measurement — a stem under -40 dBFS peak has nothing in it, and
"all energy under 150 Hz, nothing above 2 kHz" is unambiguous. Everything else keeps
the label, because onset density cannot separate a rhythmic rhodes from a drum bus:
MAREA's KEYS2 reads 5.1 onsets/s and its sparse CLAP reads 1.0, so measurement alone
files both in the wrong kit, silently. Disagreements are reported, not resolved.

Also: register_fred_rights.py records a freshly cut third-party kit in the rights
ledger as third_party WITH its provenance, at the moment of cutting. Left alone it
would land as `unknown` at the next --init, which blocks correctly but blocks with
no reason attached.
parent 0bd4a75e
#!/usr/bin/env python3
"""Register freshly cut third-party sample kits in the rights ledger, blocked by default.
A kit cut from somebody else's stems is not releasable per-track until a human says so
(`reference_sample_rights_audit`). Left alone, a new `fred_*` bank would land in the
ledger as `unknown` the next time `rights_audit.py --init` ran — which does block, but
blocks with no reason attached, so whoever reads it later has to re-derive where the
audio came from. Recording the provenance at the moment of cutting is the whole point
of `feedback_metadata_provenance`: value, source, locator, date.
python3 register_fred_rights.py <fred_kits.json> [--source "..."]
"""
from __future__ import annotations
import argparse
import json
from datetime import date
from pathlib import Path
HERE = Path(__file__).resolve().parent
LEDGER = HERE / "rights_ledger.json"
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("cuts_json", type=Path)
ap.add_argument("--source", default="Fred again.. producer stem pack (fan dropbox)")
ap.add_argument("--status", default="third_party",
choices=["third_party", "unknown", "cleared", "original"])
a = ap.parse_args()
cuts = json.loads(a.cuts_json.read_text())
led = json.loads(LEDGER.read_text())
banks = led.setdefault("banks", {})
counts: dict[str, int] = {}
tracks: dict[str, set] = {}
for c in cuts:
counts[c["kit"]] = counts.get(c["kit"], 0) + 1
tracks.setdefault(c["kit"], set()).add(c["track"])
added = updated = 0
for kit, n in sorted(counts.items()):
note = f"cut by engine.stempack from {a.source} — source track(s): " \
f"{', '.join(sorted(tracks[kit]))}"
if kit in banks and banks[kit].get("status") not in (None, "unknown"):
updated += 1 # never overwrite a human verdict
banks[kit]["files"] = n
continue
banks[kit] = {"status": a.status, "origin": "local", "files": n,
"note": note, "decided": date.today().isoformat()}
added += 1
LEDGER.write_text(json.dumps(led, indent=1) + "\n")
print(f"{added} bank(s) registered as {a.status}; {updated} already had a verdict and were left alone")
print(f"ledger: {LEDGER}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
......@@ -137,6 +137,70 @@ GET /media/<slug>/<path> → audio (HTTP Range / 206 — seek
---
## Producer stem packs (`engine.stempack`)
Not every source needs separating. A producer stem pack — Fred again..'s dropbox is the
worked example — arrives already split, by the human who mixed it, into stems named
`MAREA MIX10 123BPM KIT STEM.wav`. That is better material than demucs can produce and
completely outside what `find_takes` accepts, so `engine/stempack.py` is the batch path
for it (TODO #20).
```bash
cd tools/foundry
python3 -m engine.stempack "<pack dir>" --tracks MAREA ANGIE --jobs 3 --out /tmp/cut
python3 -m engine.kitcheck /tmp/cut/fred_kits.json # audit what landed
```
One kit per **track × role family** (`fred_marea_drums`, `fred_marea_vox`), files named
`NN_<producer role>_<bars>b.wav`, linked into Dirt-Samples so `s "fred_marea_drums"`
resolves. `fred_kits.tidal` is a paste-ready cheat sheet with the `n` index of every
sample, because a kit you cannot address is a kit you will not use.
### Four things this had to get right
**Resample before analysing.** Packs like this ship at 96 kHz, and librosa's hop/window
defaults are sample-rate-relative — at 96 k every analysis frame spans half the musical
time and beat tracking wanders. MAREA declares its tempo in the filename, so it is free
ground truth: at 96 kHz the finder returned 123/123/124/**119**/**128.1**/**128.5** bpm
with a 0.661 top score; at 44.1 kHz it returned 123.0–123.1 on every candidate with
0.859, in a third of the time. Rate is a correctness input, not a performance knob.
**One tempo per track.** `analyze_stem` reports each candidate's bpm as the mean local
BPM inside its own window — right for detecting drift, wrong for deriving a bar length,
because then every loop gets its own tempo and two loops from one track do not layer.
Bar length comes from the *mode* of the local-BPM histogram, with a confidence (the
share of beats within 5 % of it). Below 35 % there is no pulse worth cutting bars
against and the track ships chops only — ME, which has no drum stem at all, is the case
that motivated it.
**Modes rank in separate pools.** `grade` scores anything under 0.75 s as a one-shot on
`0.5·level + 0.3·dc + 0.2·zc` — no seam, no bar term. Export already removes DC and snaps
zero crossings, so chops land near 1.0 while loops pay 0.35 on seam and 0.25 on bar
consistency. Head to head, chops take every slot: measured on ME, chops 0.993–0.999,
loops 0.841–0.881, and the vocal kit came out as three stabs with no loops in it.
**The label is evidence, not truth — and neither is the measurement.** `roles.py` maps
the pack's ~40 role tokens to a family and then measures the stem. Presence and bass are
decided by measurement (a stem under −40 dBFS peak has nothing in it; "all energy under
150 Hz, nothing above 2 kHz" is unambiguous). Everything else keeps the producer's label,
because onset density cannot separate a rhythmic rhodes from a drum bus — MAREA's KEYS2
reads 5.1 onsets/s and its sparse CLAP reads 1.0, so measurement alone would file both
in the wrong kit, silently. Disagreements are reported rather than resolved.
### What `kitcheck` catches that `grade` cannot
`grade` answers *is this mechanically a clean loop*. All of these pass it:
- **dead bars** — a 4-bar loop whose last two bars are a fade has a silent-to-silent
seam and an exact length, so it grades S. Per-bar RMS finds it.
- **not actually a loop** — the finder cuts on a beat grid, so its windows are always
bar-*aligned*; nothing checked they were a bar-length *unit*. Onset-envelope
autocorrelation at the bar lag does.
- **near-duplicates** — ALL DRUMS, KIT, MAIN DRUM LOOP and DRUM BREAKS are four views of
one groove, and the finder picks the same bar out of all four. Eight slots holding
three sounds is a worse kit than three. Dedup runs at export; `kitcheck` re-checks with
the same fingerprint and threshold so the two cannot disagree.
## Engine roadmap — engine1 → engine2
The separation backend is a **registry** (`engine/separate.py`); the CLI/GUI ask
......
......@@ -122,88 +122,109 @@ class StemProbe:
"""Cheap EDA that either backs the filename's claim or overrides it."""
rms_dbfs: float
peak_dbfs: float
active_frac: float # fraction of frames above -50 dBFS
active_frac: float # share of frames within 40 dB of this stem's own peak
active_rms_dbfs: float # how loud it is WHEN it plays
centroid_hz: float
lf_frac: float # energy below 150 Hz / total
onset_rate: float # onsets per second over ACTIVE time
hf_frac: float # energy above 2 kHz / total
onset_rate: float # onsets per second over the loud excerpts
measured_family: str
agrees: bool = field(default=False)
note: str = ""
@property
def usable(self) -> bool:
return self.active_frac >= 0.05 and self.rms_dbfs > -45.0
"""Is there material here at all?
Judged on PEAK and on how loud the stem is when it plays — not on overall RMS,
which on a sparse stem measures how much silence it contains. ANGIE's HIHATS
stem is −48.8 dBFS RMS and 9.7 % active, and it is a perfectly good hi-hat
stem peaking at −14.7 (`feedback_right_lens_per_control`).
"""
return self.peak_dbfs > -40.0 and self.active_frac >= 0.005
def _probe_windows(y: np.ndarray, sr: int, *, n: int = 4, win_s: float = 20.0
) -> np.ndarray:
"""Evenly-spaced excerpts, concatenated — the probe's view of a long stem.
Classifying a 5-minute stem does not need 5 minutes of spectra; it needs a fair
sample of them. Several spread windows rather than one long middle slice, because
a stem's character is not stationary — a pad that only enters at the drop reads as
silence if you look at one place (`feedback_measure_the_time_axis`). Short stems
fall through untouched.
def _loud_windows(db: np.ndarray, y: np.ndarray, sr: int, hop: int, *,
n: int = 4, win_s: float = 20.0) -> np.ndarray:
"""Excerpts chosen where the stem is PLAYING, concatenated.
Classifying a five-minute stem does not need five minutes of spectra, but the
cheap version — evenly spaced windows — samples the gaps. Measured on ANGIE:
even windows read HIHATS at −60.9 dBFS / 1.4 % active where the full file says
−48.8 / 9.7 %, and missed PAD STUFF's material completely (0.0 % vs 6.9 %),
because that stem only plays in two sections. A stem is not stationary, so where
you look decides what you find (`feedback_measure_the_time_axis`).
So: rank non-overlapping windows by energy and take the loudest few.
"""
win = int(win_s * sr)
if len(y) <= win * n:
return y
starts = np.linspace(0, len(y) - win, n).astype(int)
return np.concatenate([y[s:s + win] for s in starts])
per = max(1, win // hop)
nwin = len(db) // per
if nwin <= n:
return y
energy = np.array([db[i * per:(i + 1) * per].mean() for i in range(nwin)])
picks = sorted(np.argsort(energy)[-n:])
return np.concatenate([y[i * per * hop:i * per * hop + win] for i in picks])
def probe(y: np.ndarray, sr: int, claimed: str) -> StemProbe:
"""Measure a mono stem and decide what it actually is. Name is never trusted."""
import librosa
y = _probe_windows(y, sr)
eps = 1e-12
peak = 20 * np.log10(np.max(np.abs(y)) + eps)
rms = 20 * np.log10(np.sqrt(np.mean(y ** 2)) + eps)
hop = 512
frame_rms = librosa.feature.rms(y=y, hop_length=hop)[0]
frame_db = 20 * np.log10(frame_rms + eps)
active = frame_db > -50.0
peak = float(20 * np.log10(np.max(np.abs(y)) + eps))
rms = float(20 * np.log10(np.sqrt(np.mean(y ** 2)) + eps))
# Level and activity come from the WHOLE file — it is only a frame-RMS pass, and
# it is the number that decides whether a stem is worth cutting at all.
frame_db = 20 * np.log10(librosa.feature.rms(y=y, hop_length=hop)[0] + eps)
# "Active" relative to this stem's own peak, not an absolute floor: an isolated
# hi-hat stem is mostly silence by design, and its overall RMS is a statement about
# how much silence it contains, not about whether it has material in it.
active = (frame_db > peak - 40.0) & (frame_db > -60.0)
active_frac = float(active.mean()) if active.size else 0.0
active_rms = float(frame_db[active].mean()) if active.any() else -120.0
# spectral shape measured on ACTIVE frames only — a stem that is 80% silence
# (an FX hit, a one-bar riser) would otherwise report the centroid of its noise
# floor (`feedback_measure_the_time_axis`).
S = np.abs(librosa.stft(y, n_fft=2048, hop_length=hop))
if active.size and active.any():
S = S[:, : active.size][:, active[: S.shape[1]]] if S.shape[1] >= active.size \
else S[:, active[: S.shape[1]]]
if S.size == 0:
S = np.abs(librosa.stft(y, n_fft=2048, hop_length=hop))
# Spectral shape and onsets, on the loud excerpts only.
w = _loud_windows(frame_db, y, sr, hop)
S = np.abs(librosa.stft(w, n_fft=2048, hop_length=hop)) ** 2
freqs = librosa.fft_frequencies(sr=sr, n_fft=2048)
power = S ** 2
tot = power.sum() + eps
centroid = float((power.sum(axis=1) * freqs).sum() / tot)
lf_frac = float(power[freqs < 150].sum() / tot)
tot = S.sum() + eps
band = S.sum(axis=1)
centroid = float((band * freqs).sum() / tot)
lf_frac = float(band[freqs < 150].sum() / tot)
hf_frac = float(band[freqs > 2000].sum() / tot)
onsets = librosa.onset.onset_detect(y=y, sr=sr, units="time", backtrack=True)
active_s = max(active_frac * len(y) / sr, 1e-3)
onset_rate = float(len(onsets) / active_s)
onsets = librosa.onset.onset_detect(y=w, sr=sr, units="time", backtrack=True)
onset_rate = float(len(onsets) / max(len(w) / sr, 1.0))
# ── the verdict ──────────────────────────────────────────────────────────
# Ordered most-specific first. These thresholds are deliberately loose: the
# probe's job is to catch a stem that is nothing like its label, not to
# re-derive a taxonomy the producer already knows.
if lf_frac > 0.80 and centroid < 250:
measured = "bass"
elif onset_rate > 1.2 and centroid > 1200:
# PERCUSSIVE FIRST. The obvious test — "mostly low energy, low centroid ⇒ bass" —
# fires on a drum bus, because a modern kick carries most of a drum mix's ENERGY:
# measured on MAREA and ANGIE, ALL DRUMS reads lf_frac 0.84/0.86 and centroid
# 182/240 Hz, indistinguishable from a bass stem on those two features alone. What
# separates them is onset density (ALL DRUMS 6.6/s, BASS 1.2/s), so density is
# asked first and the spectral test only sees what is left.
if onset_rate > 3.0:
measured = "drums"
elif onset_rate > 1.2 and lf_frac > 0.5:
measured = "drums" # kick-led groove: percussive but dark
elif hf_frac > 0.25 and onset_rate > 0.8:
measured = "drums" # bright and struck: a clap/snare stem is sparse
elif lf_frac > 0.75 and hf_frac < 0.05:
measured = "bass"
elif onset_rate < 0.35:
measured = "fx" # sustained/sparse: pad, drone, texture
elif onset_rate > 1.2 and centroid > 1200:
measured = "drums"
else:
measured = "tonal"
p = StemProbe(rms_dbfs=round(rms, 2), peak_dbfs=round(peak, 2),
active_frac=round(active_frac, 3), centroid_hz=round(centroid, 1),
lf_frac=round(lf_frac, 3), onset_rate=round(onset_rate, 2),
active_frac=round(active_frac, 3), active_rms_dbfs=round(active_rms, 2),
centroid_hz=round(centroid, 1), lf_frac=round(lf_frac, 3),
hf_frac=round(hf_frac, 4), onset_rate=round(onset_rate, 2),
measured_family=measured)
# `vox` and `tonal` are not separable by these cheap features (a sung line and a
......@@ -217,20 +238,38 @@ def probe(y: np.ndarray, sr: int, claimed: str) -> StemProbe:
p.note = f"filename says {claimed}, measurement says {measured}"
if not p.usable:
p.note = (p.note + "; " if p.note else "") + \
f"near-silent (active {active_frac:.0%}, rms {rms:.0f} dBFS)"
f"no material (peak {peak:.0f} dBFS, active {active_frac:.1%})"
return p
def effective_family(name: StemName, p: StemProbe) -> str:
"""What we actually treat the stem as. Measurement wins on a hard contradiction.
"""What we actually treat the stem as — and how much authority measurement has.
The house rule is that a sound's role is never *inferred* from its name; it is not
that a producer's label about their own stem is worthless. These two claims have
very different evidential weight, and the split matters:
* **Presence is measured, always.** Whether there is material in a stem is decided
by the probe and nothing else.
* **Bass is measured.** "Almost all energy under 150 Hz, essentially nothing above
2 kHz, not densely struck" is unambiguous, and a stem like that is bass whatever
the filename says.
* **Everything else keeps the label.** Onset density cannot separate a rhythmic
rhodes from a drum bus — measured on MAREA, KEYS2 reads 5.1 onsets/s and would be
filed as drums, and the sparse CLAP stem reads 1.0/s and would be filed as tonal.
Both are wrong, and both would be wrong *silently*, which is worse than the
label's known imprecision.
`hits` is a *name-only* distinction (a KICK stem and an ALL DRUMS stem measure
almost identically — both are percussive and dark) so it survives a `drums`
measurement; that is the one place the label carries information the cheap
features cannot.
Every disagreement is reported either way, so a wrong label stays visible instead of
being quietly overwritten by a wrong measurement.
"""
if p.agrees:
return name.claimed_family
if name.claimed_family == "hits" and p.measured_family == "drums":
return "hits"
return p.measured_family
if not name.known:
# not the producer's word — our own fallback for a token like `MAREA STEM`.
# A guess loses to a measurement.
return p.measured_family
if (name.claimed_family in ("tonal", "fx") and p.measured_family == "bass"
and p.lf_frac > 0.75 and p.hf_frac < 0.05):
return "bass"
return name.claimed_family
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment