Commit bfc17e73 by PLN (Algolia)

feat(foundry): the gate now checks that `n 3` plays what the cheat sheet says

The cheat sheet numbers `n` by enumerating the manifest sorted by filename.
SuperDirt numbers it by enumerating the *folder* sorted by filename. Those two
agree only for as long as the folder holds exactly the manifest's files — and
`export_track` writes into a kit dir it never empties.

So re-cutting a pack is quietly unsafe. If a loop that was `00_kick_2b.wav` last
run comes out as `00_kick_4b.wav` this run, both files are on disk, the folder
listing gains an entry at position 0, and every index from there on addresses the
wrong sound. Nothing else notices: the manifest is right, the grades are right,
every file is a good loop, and the cheat sheet you paste from looks correct. You
find out on stage.

Noticed while looking at the run-3 kits and asking whether runs 1 and 2 had left
anything behind (they hadn't — all 58 files carried run-3 mtimes). The absence of
the bug today is not the absence of the bug.

check_indices compares the two listings **index for index**, not as sets, so it
distinguishes the two severities the report needs to separate: an orphan that
sorts last pollutes the bank but shifts nothing, while one that sorts early
breaks addressing from that point on — and it names the index where it breaks.
`--prune` deletes what the manifest does not list, scoped to manifest kit dirs
and to audio files only. Listing comes from `pvbanks.playable`, so "what
SuperDirt will load" keeps one definition across kitgate, bank-check and the
watcher — a checker and a loader that disagree here is the whole failure mode.

3 tests (clean / shifting orphan / trailing orphan); suite 112 → 115.
parent 7232003f
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
"""kitgate — the release gate for a freshly cut sample pack. Run before linking. """kitgate — the release gate for a freshly cut sample pack. Run before linking.
Cutting a pack produces audio; this decides whether it is fit to load into a running Cutting a pack produces audio; this decides whether it is fit to load into a running
rig. Four checks, three of them things a single earlier stage cannot answer alone: rig. Five checks, four of them things a single earlier stage cannot answer alone:
1. **grade, on the shipped bytes.** `engine.grade` re-run over the written files, not 1. **grade, on the shipped bytes.** `engine.grade` re-run over the written files, not
the candidate windows the finder scored. It is a separate code path reaching the same the candidate windows the finder scored. It is a separate code path reaching the same
...@@ -14,12 +14,15 @@ rig. Four checks, three of them things a single earlier stage cannot answer alon ...@@ -14,12 +14,15 @@ rig. Four checks, three of them things a single earlier stage cannot answer alon
the name — silently, with the only symptom being that an old track now plays the the name — silently, with the only symptom being that an old track now plays the
wrong sound. Checked BEFORE anything is linked, via `tools/pvbanks.py` so this agrees wrong sound. Checked BEFORE anything is linked, via `tools/pvbanks.py` so this agrees
with `bank-check.py` by construction (`feedback_one_parser_per_concept`). with `bank-check.py` by construction (`feedback_one_parser_per_concept`).
4. **Rights.** A kit cut from someone else's stems must not be releasable per-track by 4. **`n` indices.** The cheat sheet numbers `n` from the manifest; SuperDirt numbers it
from the folder. Export never empties a kit dir, so a re-cut can leave a stale file
behind and shift every index after it. Compared listing-to-listing, index for index.
5. **Rights.** A kit cut from someone else's stems must not be releasable per-track by
default. Reported here, ledgered by `armada/tide-table/register_fred_rights.py`. default. Reported here, ledgered by `armada/tide-table/register_fred_rights.py`.
python3 kitgate.py <cuts.json> [--link] [--source "..."] python3 kitgate.py <cuts.json> [--link] [--source "..."]
`--link` performs the Dirt-Samples symlink for every kit, but only if 1-3 pass. `--link` performs the Dirt-Samples symlink for every kit, but only if 1-4 pass.\n`--prune` first deletes kit-dir files the manifest does not list.
""" """
from __future__ import annotations from __future__ import annotations
...@@ -85,6 +88,50 @@ def check_shadowing(cuts: list[dict]) -> tuple[bool, list[str]]: ...@@ -85,6 +88,50 @@ def check_shadowing(cuts: list[dict]) -> tuple[bool, list[str]]:
return not clash, lines return not clash, lines
def check_indices(cuts: list[dict]) -> tuple[bool, list[str]]:
"""Does `n <i>` on disk address the file the cheat sheet says it does?
The cheat sheet numbers `n` by enumerating the manifest sorted by name, and
SuperDirt numbers it by enumerating the FOLDER sorted by name. Those two agree
only as long as the folder holds exactly the manifest's files — and export writes
into an existing kit dir without ever emptying it. So a re-cut that renames one
file (`00_kick_2b.wav` → `00_kick_4b.wav`) leaves both behind, and every index
from that point on addresses the wrong sound, silently, with a cheat sheet that
looks right. Compare the two listings index for index, not as sets.
Uses `pvbanks.playable` so "what SuperDirt will load" has one definition here,
in `bank-check.py`, and in the watcher.
"""
sys.path.insert(0, str(TIDAL_TOOLS))
try:
from pvbanks import playable
except Exception as e:
return True, [f" (skipped: {e})"]
by_kit: dict[str, list[dict]] = {}
for c in cuts:
by_kit.setdefault(c["kit"], []).append(c)
bad, orphans = [], 0
for kit, items in sorted(by_kit.items()):
kit_dir = Path(items[0]["path"]).parent
want = [c["name"] for c in sorted(items, key=lambda c: c["name"])]
have = [p.stem for p in playable(kit_dir)]
extra = [h for h in have if h not in want]
orphans += len(extra)
missing = [w for w in want if w not in have]
shift = next((i for i, (w, h) in enumerate(zip(want, have)) if w != h), None)
if extra or missing:
bad.append(f" ✗ {kit}: {len(extra)} orphan, {len(missing)} missing"
+ (f" — n {shift} onward addresses the wrong file" if shift is not None
else " — indices still line up")
+ (f" · orphans: {', '.join(extra[:4])}" if extra else ""))
lines = [f" {len(by_kit)} kit dirs hold exactly their {len(cuts)} manifest files"
if not bad else f" {orphans} orphan files across {len(bad)} kits"]
lines += bad[:20]
if orphans:
lines.append(" → re-run with --prune to delete files the manifest does not list")
return not bad, lines
def check_rights(cuts: list[dict], source: str) -> tuple[bool, list[str]]: def check_rights(cuts: list[dict], source: str) -> tuple[bool, list[str]]:
"""Are these banks recorded as NOT releasable per-track? Advisory, never blocking.""" """Are these banks recorded as NOT releasable per-track? Advisory, never blocking."""
ledger = (TIDAL_TOOLS.parent / "armada/tide-table/rights_ledger.json") ledger = (TIDAL_TOOLS.parent / "armada/tide-table/rights_ledger.json")
...@@ -105,21 +152,45 @@ def check_rights(cuts: list[dict], source: str) -> tuple[bool, list[str]]: ...@@ -105,21 +152,45 @@ def check_rights(cuts: list[dict], source: str) -> tuple[bool, list[str]]:
return True, lines return True, lines
def prune_orphans(cuts: list[dict]) -> int:
"""Delete kit-dir files the manifest does not list. Only ever inside a kit dir
named by the manifest, and only ever an audio file — we never delete what we did
not write."""
sys.path.insert(0, str(TIDAL_TOOLS))
from pvbanks import playable
keep: dict[Path, set[str]] = {}
for c in cuts:
keep.setdefault(Path(c["path"]).parent, set()).add(c["name"])
n = 0
for kit_dir, names in keep.items():
for f in playable(kit_dir):
if f.stem not in names:
f.unlink()
n += 1
return n
def main(argv=None) -> int: def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="kitgate", description=__doc__, ap = argparse.ArgumentParser(prog="kitgate", description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter) formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("cuts_json", type=Path) ap.add_argument("cuts_json", type=Path)
ap.add_argument("--link", action="store_true", help="symlink into Dirt-Samples if clean") ap.add_argument("--link", action="store_true", help="symlink into Dirt-Samples if clean")
ap.add_argument("--source", default="third-party producer stem pack") ap.add_argument("--source", default="third-party producer stem pack")
ap.add_argument("--prune", action="store_true",
help="delete files in the kit dirs that the manifest does not list "
"(leftovers from an earlier cut) before checking")
a = ap.parse_args(argv) a = ap.parse_args(argv)
cuts = json.loads(a.cuts_json.read_text()) cuts = json.loads(a.cuts_json.read_text())
if a.prune:
print(f"pruned {prune_orphans(cuts)} orphan files\n")
print(f"kitgate — {len(cuts)} samples in {len({c['kit'] for c in cuts})} kits\n") print(f"kitgate — {len(cuts)} samples in {len({c['kit'] for c in cuts})} kits\n")
results = {} results = {}
for label, fn in [("grade (re-run on the written files)", lambda: check_grades(cuts)), for label, fn in [("grade (re-run on the written files)", lambda: check_grades(cuts)),
("kitcheck", lambda: check_kits(cuts)), ("kitcheck", lambda: check_kits(cuts)),
("bank shadowing", lambda: check_shadowing(cuts)), ("bank shadowing", lambda: check_shadowing(cuts)),
("n indices (folder vs manifest)", lambda: check_indices(cuts)),
("rights", lambda: check_rights(cuts, a.source))]: ("rights", lambda: check_rights(cuts, a.source))]:
ok, lines = fn() ok, lines = fn()
results[label] = ok results[label] = ok
......
...@@ -263,3 +263,44 @@ def test_a_staged_kit_records_the_grade_of_what_it_actually_wrote(tmp_path, monk ...@@ -263,3 +263,44 @@ def test_a_staged_kit_records_the_grade_of_what_it_actually_wrote(tmp_path, monk
before = G.grade_array(clip.T, SR).grade before = G.grade_array(clip.T, SR).grade
after = G.grade_array((clip * head).T, SR).grade after = G.grade_array((clip * head).T, SR).grade
assert abs(before - after) > 0.02 # the grade really does move assert abs(before - after) > 0.02 # the grade really does move
# ── the gate's own listing check ─────────────────────────────────────────────
def _kitgate():
import importlib.util
spec = importlib.util.spec_from_file_location("kitgate", Path(__file__).parent.parent / "kitgate.py")
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)
return m
def test_a_clean_kit_dir_matches_its_manifest(tmp_path):
cuts = [_cut(_write(tmp_path, f"{i:02d}_kit_4b.wav", _click_track())) for i in range(3)]
ok, lines = _kitgate().check_indices(cuts)
assert ok, lines
def test_a_leftover_from_an_earlier_cut_shifts_every_index_after_it(tmp_path):
"""The failure this check exists for: export writes into a kit dir it never empties,
so re-cutting `00_kit_2b` as `00_kit_4b` leaves BOTH on disk. SuperDirt enumerates
the folder, so `n 1` now plays what the cheat sheet calls `n 0` — and nothing about
the manifest, the grades or the audio looks wrong."""
cuts = [_cut(_write(tmp_path, f"{i:02d}_kit_4b.wav", _click_track())) for i in range(3)]
_write(tmp_path, "00_kit_2b.wav", _click_track(bars=2)) # the stale leftover
g = _kitgate()
ok, lines = g.check_indices(cuts)
assert not ok
assert "orphan" in " ".join(lines)
assert "n 0 onward" in " ".join(lines) # it says WHERE the addressing breaks
assert g.prune_orphans(cuts) == 1
assert g.check_indices(cuts)[0]
def test_an_orphan_that_sorts_last_is_still_reported_but_indices_hold(tmp_path):
"""A trailing leftover pollutes the bank without moving any index — a different
severity, and the report has to say which one it is."""
cuts = [_cut(_write(tmp_path, f"{i:02d}_kit_4b.wav", _click_track())) for i in range(3)]
_write(tmp_path, "09_kit_4b.wav", _click_track())
ok, lines = _kitgate().check_indices(cuts)
assert not ok
assert "indices still line up" in " ".join(lines)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment