-
feat(foundry): auto-tune harness — search finder settings vs the grade rubric (#19) · 6f850128
PLN's closed feedback loop: generate candidate loops with a settings profile → grade them on the full quality rubric → score the profile → search for the settings that maximise quality. Auto-tune the composite weights instead of hand-guessing them. WHY: the finder/grader/correlator exist; the composite weights W were "provisional, calibrate with #7". This builds the machine that calibrates them against measured grade — and measures the lift honestly rather than asserting a tuning is better. WHAT: - engine/autotune.py — evaluate(stems, settings) runs the finder with a profile, grades every produced candidate on the FULL rubric (grade.py adds dc / level / bass-mono / bar-consistency — features the finder's own score does NOT use, so tuning pulls in signal beyond seam/zc), aggregates objective = mean_grade × coverage. search() sweeps random|grid and always includes the current defaults (loops.W) as a baseline row so the report shows lift, not just a number. - autotune.py — CLI (--stems glob --n --method --max-stems); PII-safe aggregated leaderboard (counts, never filenames). - engine/loops.py — analyze_stem now accepts an optional `weights` override (defaults unchanged); the grid is preloaded once per stem (weight-independent) so the sweep is cheap — a scarcity-minded speedup. VALIDATION: - 4/4 mocked harness unit tests (aggregation, empty-set, ranked+baseline-included, weights threaded through to the finder). - Real end-to-end run, 2 drums stems × 8 configs: baseline objective 0.690 → best 0.763 (+10.5%); the defaults ranked LAST of 8 — the search found real lift. NOT DONE ON PURPOSE: the finder defaults (loops.W) are UNCHANGED. The winning profile (seam 0.4, zc 0.05, bars=(4,8)) is from a 2-stem sample AND seam/zc are in both the finder score and the grade (a confound) — so it's suggestive, not a mandate. Remaining for #19: recall-vs-provenance-GT as an anti-gaming 2nd objective, per-stem-role tuning (vocals=chops want different weights than drums), a full-corpus + demo-corpus campaign, an extract-once/score-many speedup, THEN adopt a validated profile as the default.
PLN (Algolia) authored6f850128
| Name |
Last commit
|
Last update |
|---|---|---|
| .. | ||
| __init__.py | Loading commit data... | |
| autotune.py | Loading commit data... | |
| correlate.py | Loading commit data... | |
| demos.py | Loading commit data... | |
| fetch.py | Loading commit data... | |
| grade.py | Loading commit data... | |
| loops.py | Loading commit data... | |
| model.py | Loading commit data... | |
| naming.py | Loading commit data... | |
| publish.py | Loading commit data... | |
| separate.py | Loading commit data... |