Commit 3a7c660b by PLN (Algolia)

feat(tide-table): draft tracks.json from backlog.md — 25 gigs catalogued, now 33

Eight gigs had a page on the site and no tracks.json, which is the only thing
the catalog joins on, so eight real sets were invisible to every tool that
reads the corpus. All eight had a trusted setlist anchor in backlog.md
already; nothing needed PLN's ears, only a drafter.

build_gig_tracksjson.py gains build_from_backlog() and a --backlog-slug/--md
mode. Extended rather than rewritten: the existing segments-based path is
untouched, and parse_score() was lifted out of an inline closure so both modes
share one parser instead of growing a second opinion about what a score is.

gigs_total 25 -> 33, tracks_total 81, and recorded rose 40 -> 44 because the
new gigs brought their recordings into the map with them.

What these files deliberately do NOT contain, because the alternative is
inventing facts about PLN's own shows:

  - no `section` -- none of the eight has an ear-verified movement structure
  - no start_s/end_s/duration_s and no totalDuration_s -- no recording has
    been measured for any of them; each file says so in `durationBasis`
    rather than leaving the absence to be guessed at
  - `stage: null` where the frontmatter sets no stage
  - `bpm: null` where backlog.md carried no inline annotation

Title, date and venue come from content/lives/{year}/{slug}.md only, with
venue mapped from `address` exactly as the ear-verified OPAL-26 file does.

The 13 unresolved track names are IN the files, flagged `resolved: false` with
`file: null`, not dropped. A name silently disappearing is the failure this
whole pipeline exists to prevent, and six of the thirteen are opal-2025 alone
-- that gig is 11 of 17 and is the one worth a human pass:

  mephisteuf            TOP HATS
  toplap-solstice-2024  Deck the Hall
  38c3-house-of-tea     CCC0
  ensad                 Parce qu'Elle est la
  39c3-house-of-tea     TechnOrage
  le-vortex             TechnOrage
  opal-festival-2025    The Secret, 1er Septembre Pour Elle, Chere Mireille,
                        JEROME, Oct29 Love First, Cafes du plus chaud au plus
                        froid

The three `confirm:`-flagged gigs (algolia-fdlm, algorave-lyon, ete-surprise)
are still withheld by backlog_setlists.py's own design, pending PLN
confirming their anchors.

Suite: 480 passed, 2 skipped, 0 failed.
parent fbf3414d
......@@ -56,16 +56,183 @@ def style_of(score: str | None) -> str | None:
return parts[-1] if parts else None
def parse_score(rel: str) -> list[str] | None:
"""Read the sounds straight out of the .tidal when the view has no row.
catalog_view's track universe comes from `load_gigs()`, which globs the
already-published `content/lives/**/tracks.json`. So a track making its
FIRST catalogued appearance — which is precisely what a new gig page is
full of — cannot be in the view yet: the file that would add it is the one
being written here. Rebuilding the view does not fix it; that is a cycle,
not staleness.
The fix is not a second parser. `feedback_one_parser_per_concept`: this
calls the same `tidal_score.orbit_sounds` the view itself calls, so a new
track's sample list is produced by the one authority on what a score
loads, and cannot drift from the catalogued ones.
"""
path = REPO / rel
if not path.exists():
return None
try:
import tidal_score as ts
from sample_tfidf import sound_vocab
vocab, kind = sound_vocab()
return sorted({d["sound"] for d in
ts.orbit_sounds(path, vocab, kind).values()})
except Exception as exc: # a broken parse is not a zero
print(f"⚠ score parse failed for {rel}: {exc}")
return None
# 2026-09-05: mechanical catalogue pass for the 8 gigs recovered from
# backlog.md (backlog_setlists.py) that have a page but no tracks.json and no
# ear-verified recording to measure timecodes from. This mode is deliberately
# NARROWER than the segments-based one above: no start_s/end_s/duration_s (no
# recording was measured — feedback_metadata_vs_mastering), no `section` (PLN
# hasn't heard these sets — section is PLN-only, never inferred from a
# backlog sub-header). What backlog_setlists.py DID resolve mechanically
# (raw name, .tidal path, inline bpm/transition annotations) is kept; a raw
# name that failed to resolve is kept too, flagged `"resolved": false`, rather
# than silently dropped ([[feedback_absence_needs_proof]] / the brief's own
# "unresolved tracks must stay visible" rule).
DRAFTED_ON = "2026-09-05"
def _frontmatter(md_path: Path) -> dict:
"""Minimal frontmatter reader: the block between the two `---` lines,
parsed as YAML. Only the canonical fields (title/date/location/address/
stage/description) are used by build_from_backlog — never guessed,
always cited back to this file."""
import yaml
text = md_path.read_text()
if not text.startswith("---"):
return {}
end = text.index("\n---", 3)
return yaml.safe_load(text[3:end]) or {}
def build_from_backlog(slug: str, gig_name: str, backlog_json: Path,
md_path: Path, out_path: Path) -> tuple[dict, list[str]]:
"""Draft a tracks.json for one backlog-recovered gig. Returns (out, unresolved_raw_names)."""
backlog = json.loads(backlog_json.read_text())
g = backlog["gigs"].get(slug) or backlog["pending"].get(slug)
if g is None:
raise SystemExit(f"{slug!r} not found in {backlog_json} (gigs or pending)")
view = json.loads((HERE / "catalog_view.json").read_text())["tracks"]
sounds = {t["track"]: t.get("score_sounds") or [] for t in view}
fm = _frontmatter(md_path)
prov_locator = str(md_path)
def fmval(key):
v = fm.get(key)
return v if v not in (None, "") else None
rows, unresolved, no_sounds, first_appearance = [], [], [], []
for t in g["tracks"]:
score = t.get("track")
snd = sounds.get(score) if score else None
if score and snd is None:
snd = parse_score(score)
(no_sounds if snd is None else first_appearance).append(score)
row = {
"name": t["raw"],
"file": score,
"resolved": bool(score),
"bpm": t.get("bpm"),
"style": style_of(score),
"samples": sorted(snd) if snd else [],
}
if t.get("transition"):
row["transition"] = t["transition"] # mechanical, from backlog.md
rows.append(row)
if not score:
unresolved.append(t["raw"])
bpms = [r["bpm"] for r in rows if r["bpm"]]
resolved_meta = {"title": fmval("title"), "date": fmval("date"),
"venue": fmval("address"), "stage": fmval("stage")}
out = {
"_draft": [
f"Mechanically drafted from backlog.md by build_gig_tracksjson.py",
f"--backlog-slug on {DRAFTED_ON}, via backlog_setlists.py's anchor+alias",
"resolution. NOT ear-verified: no recording was measured for this gig,",
"so there are no start_s/end_s/duration_s and no `section` (movements) —",
"both would have to be guessed, and this tool never guesses. Any track",
"below with \"resolved\": false failed to match the catalog's alias index",
"and needs PLN's eyes (raw backlog.md name kept verbatim, not dropped).",
],
"gig": gig_name,
"title": fmval("title"),
"date": fmval("date"),
"venue": fmval("address"), # frontmatter field name for venue
"stage": fmval("stage"),
"trackCount": len(rows),
"bpmRange": [min(bpms), max(bpms)] if bpms else None,
"styleDistribution": {s: sum(1 for r in rows if r["style"] == s)
for s in sorted({r["style"] for r in rows if r["style"]})},
"samplePacks": sorted({x for r in rows for x in r["samples"]}),
"_needs_pln": (
[k for k in ("title", "date", "venue", "stage") if resolved_meta[k] is None]
+ [f'unresolved: "{n}"' for n in unresolved]
),
"_provenance": {
"tracklist": {"source": "backlog.md", "locator": g["header"],
"date": DRAFTED_ON},
"title": {"source": prov_locator, "locator": "frontmatter.title"},
"date": {"source": prov_locator, "locator": "frontmatter.date"},
"venue": {"source": prov_locator, "locator": "frontmatter.address"},
"stage": {"source": prov_locator, "locator": "frontmatter.stage"},
},
"durationBasis": (
"No ear-verified recording exists for this gig (or none has been "
"reviewed yet), so this tracklist carries NO per-track timecodes and "
"NO `section` (movements) — both are PLN-only and this draft never "
"invents them. Order and content come straight from backlog.md via "
"backlog_setlists.py, mechanically resolved to .tidal scores."
),
"tracks": rows,
}
out_path.write_text(json.dumps(out, indent=2, ensure_ascii=False) + "\n")
return out, unresolved
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("spec")
ap.add_argument("spec", nargs="?", help="segments-based draft spec json")
ap.add_argument("--out", required=True, help="draft tracks.json path")
ap.add_argument("--md-out", help="also draft the .md frontmatter skeleton")
ap.add_argument("--meta", help="JSON sidecar carrying the PLN-only fields, each "
"a bare value or {value, source, locator, date}")
ap.add_argument("--backlog-slug",
help="site slug (e.g. 2025/ensad) — draft from "
"backlog_setlists.json instead of segments; no "
"timecodes, no section (see build_from_backlog)")
ap.add_argument("--backlog-json", default=str(HERE / "backlog_setlists.json"))
ap.add_argument("--md", help="path to the gig's {slug}.md (backlog mode)")
a = ap.parse_args()
if a.backlog_slug:
if not a.md:
ap.error("--backlog-slug requires --md")
gig_name = a.backlog_slug.split("/", 1)[1]
out, unresolved = build_from_backlog(
a.backlog_slug, gig_name, Path(a.backlog_json), Path(a.md), Path(a.out))
print(f"✓ {a.out} ({out['trackCount']} tracks, "
f"{sum(1 for t in out['tracks'] if not t['resolved'])} unresolved)")
if unresolved:
print("⚠ UNRESOLVED (kept, flagged, need PLN): " + " · ".join(unresolved))
if out["_needs_pln"]:
missing_meta = [k for k in out["_needs_pln"] if not k.startswith("unresolved:")]
if missing_meta:
print(f"⚠ missing frontmatter field(s): {missing_meta}")
else:
print("✓ nothing left to invent — every non-derivable field is sourced")
return 0
meta = json.loads(Path(a.meta).read_text()) if a.meta else {}
prov: dict = {}
......@@ -101,34 +268,6 @@ def main() -> int:
view = json.loads((HERE / "catalog_view.json").read_text())["tracks"]
sounds = {t["track"]: t.get("score_sounds") or [] for t in view}
def parse_score(rel: str) -> list[str] | None:
"""Read the sounds straight out of the .tidal when the view has no row.
catalog_view's track universe comes from `load_gigs()`, which globs the
already-published `content/lives/**/tracks.json`. So a track making its
FIRST catalogued appearance — which is precisely what a new gig page is
full of — cannot be in the view yet: the file that would add it is the one
being written here. Rebuilding the view does not fix it; that is a cycle,
not staleness.
The fix is not a second parser. `feedback_one_parser_per_concept`: this
calls the same `tidal_score.orbit_sounds` the view itself calls, so a new
track's sample list is produced by the one authority on what a score
loads, and cannot drift from the catalogued ones.
"""
path = REPO / rel
if not path.exists():
return None
try:
import tidal_score as ts
from sample_tfidf import sound_vocab
vocab, kind = sound_vocab()
return sorted({d["sound"] for d in
ts.orbit_sounds(path, vocab, kind).values()})
except Exception as exc: # a broken parse is not a zero
print(f"⚠ score parse failed for {rel}: {exc}")
return None
rows, no_sounds, needs, first_appearance = [], [], [], []
for s in segs:
nom = nominal.get(s["perf_track"], {})
......
This source diff could not be displayed because it is too large. You can view the blob instead.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment