Commit 436d327f by PLN (Algolia)

fix(tide-table): stale line anchors, not the setlist soundcheck fix, dropped opal-2024 to 3

Two unrelated failing tests, investigated in order.

test_agreement_distribution_within_bounds asserted tracks_total == 73, a snapshot
from before yesterday's catalog rebuild (73 -> 81 tracks). Bumping it to 81 would
just re-freeze the same brittleness. Replaced it with
test_tracks_total_matches_the_generated_catalog: assert the live view's
tracks_total equals the CHECKED-IN catalog.generated.json's n_tracks. That file is
a 1:1 derivation of this same view, so the two can only disagree on a real
regression (corpus changed but catalog not regenerated, or the pipeline
dropped/duplicated a row) — never on honest growth. Verified it still fails by
corrupting n_tracks and re-running.

test_real_backlog_coverage_does_not_regress: opal-festival-2024's recovered
tracks fell from 13/14 to 3. The obvious suspect was e8f7066a (yesterday's
tools/setlist.py soundcheck filter) — ruled out by inspection (e8f7066a touches
only tools/setlist.py + its test, nothing under armada/tide-table) and by
bisection: replaying backlog_setlists.py's ANCHORS against every historical
backlog.md since the anchors were authored (5acf72f7, 2026-06-06) shows
n_resolved holding at 14 across ten intervening commits and only breaking at
49e1b78e ("update: post cosmic", 2026-08-23).

The real bug: ANCHORS keyed gigs by a raw 1-based LINE NUMBER pinned once against
a single backlog.md snapshot. backlog.md is PLN's running journal, not a stable
document, so every one of those ten commits that inserted lines earlier in the
file silently shifted every anchor below it. Opal 2024's anchor drifted through
blank lines and a lucky near-miss for months (the collected span still happened
to contain the same real content), then 49e1b78e pushed the drift past the
header entirely into the tail of the PREVIOUS gig's block, collecting 3
unrelated lines instead of the setlist.

Fix: anchor by the header's exact TEXT, resolved fresh against the live file on
every build() (resolve_anchors()), instead of a line number frozen at authoring
time. Self-healing against drift elsewhere in the file; raises loudly if a
heading is ever actually renamed or duplicated, rather than silently
mis-anchoring. The three anchors that carried an explicit end-of-block bound
(opal-2025/Latin-Heritage bleed, mephisteuf's commented block, the 38c3-toilet
cross-check) now store that bound as a line-count SPAN measured from the anchor,
since that span lives inside the block and doesn't move when the anchor does.

tools/setlist.py's soundcheck filter (e8f7066a) is untouched and still yields 15
entries for OPAL 2026 — never the culprit here, just an unrelated same-day
change that made a plausible but wrong prime suspect.

Full suite: 468 passed, 2 skipped (465 baseline + 2 fixed + 1 new regression
test), up from 2 failing.
parent 3c210a2a
...@@ -9,7 +9,7 @@ ...@@ -9,7 +9,7 @@
"slug": "2023/mephisteuf", "slug": "2023/mephisteuf",
"confirm": false, "confirm": false,
"header": "## Live @MephisTeuf 😈‍💻😈", "header": "## Live @MephisTeuf 😈‍💻😈",
"anchor_line": 710, "anchor_line": 722,
"n_tracks": 6, "n_tracks": 6,
"n_resolved": 5, "n_resolved": 5,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -62,7 +62,7 @@ ...@@ -62,7 +62,7 @@
"slug": "2024/opal-festival-2024", "slug": "2024/opal-festival-2024",
"confirm": false, "confirm": false,
"header": "## Opal <3🦉", "header": "## Opal <3🦉",
"anchor_line": 1031, "anchor_line": 1043,
"n_tracks": 14, "n_tracks": 14,
"n_resolved": 14, "n_resolved": 14,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -171,7 +171,7 @@ ...@@ -171,7 +171,7 @@
"slug": "2024/toplap-solstice-2024", "slug": "2024/toplap-solstice-2024",
"confirm": false, "confirm": false,
"header": "### TopLap Solstice Stream 2024", "header": "### TopLap Solstice Stream 2024",
"anchor_line": 1169, "anchor_line": 1181,
"n_tracks": 4, "n_tracks": 4,
"n_resolved": 3, "n_resolved": 3,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -210,7 +210,7 @@ ...@@ -210,7 +210,7 @@
"slug": "2024/38c3-house-of-tea", "slug": "2024/38c3-house-of-tea",
"confirm": false, "confirm": false,
"header": "#### Day 2 - HOUSE OF Tea", "header": "#### Day 2 - HOUSE OF Tea",
"anchor_line": 1194, "anchor_line": 1206,
"n_tracks": 12, "n_tracks": 12,
"n_resolved": 11, "n_resolved": 11,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -305,7 +305,7 @@ ...@@ -305,7 +305,7 @@
"slug": "2024/38c3-toilet", "slug": "2024/38c3-toilet",
"confirm": false, "confirm": false,
"header": "#### Day 3 - Hardcore in the Toilet Club", "header": "#### Day 3 - Hardcore in the Toilet Club",
"anchor_line": 1242, "anchor_line": 1254,
"n_tracks": 4, "n_tracks": 4,
"n_resolved": 3, "n_resolved": 3,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -344,7 +344,7 @@ ...@@ -344,7 +344,7 @@
"slug": "2025/ensad", "slug": "2025/ensad",
"confirm": false, "confirm": false,
"header": "# Set @ENSAD 2025", "header": "# Set @ENSAD 2025",
"anchor_line": 1441, "anchor_line": 1453,
"n_tracks": 9, "n_tracks": 9,
"n_resolved": 8, "n_resolved": 8,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -418,7 +418,7 @@ ...@@ -418,7 +418,7 @@
"slug": "2025/opal-festival-2025", "slug": "2025/opal-festival-2025",
"confirm": false, "confirm": false,
"header": "# OPAL 2025", "header": "# OPAL 2025",
"anchor_line": 1632, "anchor_line": 1644,
"n_tracks": 17, "n_tracks": 17,
"n_resolved": 11, "n_resolved": 11,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -548,7 +548,7 @@ ...@@ -548,7 +548,7 @@
"slug": "2025/39c3-house-of-tea", "slug": "2025/39c3-house-of-tea",
"confirm": false, "confirm": false,
"header": "# CCC 39c3 ALGORAVE <3", "header": "# CCC 39c3 ALGORAVE <3",
"anchor_line": 1786, "anchor_line": 1798,
"n_tracks": 14, "n_tracks": 14,
"n_resolved": 13, "n_resolved": 13,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -657,7 +657,7 @@ ...@@ -657,7 +657,7 @@
"slug": "2026/le-vortex", "slug": "2026/le-vortex",
"confirm": false, "confirm": false,
"header": "# VORTEX26", "header": "# VORTEX26",
"anchor_line": 1816, "anchor_line": 1828,
"n_tracks": 9, "n_tracks": 9,
"n_resolved": 8, "n_resolved": 8,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -733,7 +733,7 @@ ...@@ -733,7 +733,7 @@
"slug": "2024/algolia-fdlm", "slug": "2024/algolia-fdlm",
"confirm": true, "confirm": true,
"header": "## Algolia FDLM2024", "header": "## Algolia FDLM2024",
"anchor_line": 988, "anchor_line": 1000,
"n_tracks": 26, "n_tracks": 26,
"n_resolved": 26, "n_resolved": 26,
"suspect_bleed": true, "suspect_bleed": true,
...@@ -926,7 +926,7 @@ ...@@ -926,7 +926,7 @@
"slug": "2025/algorave-lyon", "slug": "2025/algorave-lyon",
"confirm": true, "confirm": true,
"header": "⠀⠀⠀⠹⣟⠀⠀⠀ GZ 2025 LYON ⠀⣰⠏⠀⠀", "header": "⠀⠀⠀⠹⣟⠀⠀⠀ GZ 2025 LYON ⠀⣰⠏⠀⠀",
"anchor_line": 1550, "anchor_line": 1562,
"n_tracks": 0, "n_tracks": 0,
"n_resolved": 0, "n_resolved": 0,
"suspect_bleed": false, "suspect_bleed": false,
...@@ -936,7 +936,7 @@ ...@@ -936,7 +936,7 @@
"slug": "2026/ete-surprise", "slug": "2026/ete-surprise",
"confirm": true, "confirm": true,
"header": "## SURPRISE IT'S A DANCEFLOOR", "header": "## SURPRISE IT'S A DANCEFLOOR",
"anchor_line": 1831, "anchor_line": 1843,
"n_tracks": 11, "n_tracks": 11,
"n_resolved": 8, "n_resolved": 8,
"suspect_bleed": false, "suspect_bleed": false,
......
...@@ -27,24 +27,39 @@ CATALOG_VIEW = HERE / "catalog_view.json" ...@@ -27,24 +27,39 @@ CATALOG_VIEW = HERE / "catalog_view.json"
OUT = HERE / "backlog_setlists.json" OUT = HERE / "backlog_setlists.json"
AS_OF = "2026-06-06" AS_OF = "2026-06-06"
# ── Authored gig anchors: (1-based line of the header in backlog.md) → site slug ── # ── Authored gig anchors: exact header TEXT in backlog.md → site slug ────────────
# `confirm:`-prefixed slugs are NOT yet trusted (ambiguous header / needs PLN's ear); # `confirm:`-prefixed slugs are NOT yet trusted (ambiguous header / needs PLN's ear);
# they are parsed and reported but withheld from the emitted artifact. # they are parsed and reported but withheld from the emitted artifact.
# value = slug OR (slug, end_line) to bound a block that would otherwise bleed # value = slug OR (slug, span) to bound a block that would otherwise bleed into
# into the next densely-packed setlist (the backlog has no blank-gap between them). # the next densely-packed setlist (the backlog has no blank-gap between them). `span`
# is a LINE COUNT measured from the anchor (not an absolute line) — it is drawn from
# content that lives INSIDE the anchored block, so it doesn't need updating every
# time unrelated journal entries shift the anchor itself.
#
# Anchors used to be keyed by raw line number, pinned once on 2026-06-06. backlog.md
# is PLN's running journal, not a stable document — 10 commits since then inserted
# ~50 lines earlier in the file, so every literal line number silently drifted.
# Opal 2024's anchor (originally line 1031) drifted through blank lines and a
# lucky near-miss for months, then on 2026-08-23 ('update: post cosmic', 49e1b78)
# the drift finally walked the anchor PAST its own header into the tail of the
# PREVIOUS gig's block — n_resolved silently fell from 14 to 3, caught only when
# the coverage-regression test was finally run again (2026-09-05). Anchors now key
# off the header's own TEXT, resolved fresh against the current file every build()
# — self-healing against drift elsewhere, and loud (RuntimeError) if a heading is
# ever actually renamed or duplicated, instead of silently mis-anchoring again.
ANCHORS = { ANCHORS = {
1441: "2025/ensad", # "# Set @ENSAD 2025" '# Set @ENSAD 2025': "2025/ensad",
1816: "2026/le-vortex", # "# VORTEX26" '# VORTEX26': "2026/le-vortex",
1031: "2024/opal-festival-2024", # "## Opal <3🦉" / OPAL 2024 '## Opal <3🦉': "2024/opal-festival-2024",
1632: ("2025/opal-festival-2025", 1667), # "# OPAL 2025" (bleeds into Latin Heritage) '# OPAL 2025': ("2025/opal-festival-2025", 35), # bleeds into Latin Heritage
710: ("2023/mephisteuf", 735), # "## Live @MephisTeuf" (commented block = cut) '## Live @MephisTeuf 😈\u200d💻😈': ("2023/mephisteuf", 25), # commented block = cut
1194: "2024/38c3-house-of-tea", # "#### Day 2 - HOUSE OF Tea" (under 38C3) '#### Day 2 - HOUSE OF Tea': "2024/38c3-house-of-tea", # (under 38C3)
1242: ("2024/38c3-toilet", 1253), # cross-check only (already has tracks.json) '#### Day 3 - Hardcore in the Toilet Club': ("2024/38c3-toilet", 11), # cross-check only (already has tracks.json)
1786: "2025/39c3-house-of-tea", # "# CCC 39c3 ALGORAVE ... Pour Un Thé Dansant" '# CCC 39c3 ALGORAVE <3': "2025/39c3-house-of-tea", # "... Pour Un Thé Dansant"
1169: "2024/toplap-solstice-2024", # "### TopLap Solstice Stream 2024" '### TopLap Solstice Stream 2024': "2024/toplap-solstice-2024",
988: "confirm:2024/algolia-fdlm", # "## Algolia FDLM2024" — already has tracks.json '## Algolia FDLM2024': "confirm:2024/algolia-fdlm", # already has tracks.json
1550: "confirm:2025/algorave-lyon", # "GZ 2025 LYON ALGORAVE" then "# LABENNE LIVE" (?) ' ⠀⠀⠀⠹⣟⠀⠀⠀ GZ 2025 LYON ⠀⣰⠏⠀⠀': "confirm:2025/algorave-lyon", # ASCII banner, no real heading nearby
1831: "confirm:2026/ete-surprise", # "## SURPRISE IT'S A DANCEFLOOR" — slug guess "## SURPRISE IT'S A DANCEFLOOR": "confirm:2026/ete-surprise", # slug guess
} }
# Still unanchored (gaps for #66, need PLN's ear / SC confirmation): 2022/algolia-fdlm-2022, # Still unanchored (gaps for #66, need PLN's ear / SC confirmation): 2022/algolia-fdlm-2022,
# 2024/38c3-chaos-music-club, 2025/39c3-toilet-rave, 2025/toplap-fromscratch-2025. The # 2024/38c3-chaos-music-club, 2025/39c3-toilet-rave, 2025/toplap-fromscratch-2025. The
...@@ -136,19 +151,37 @@ def clean_track_line(raw: str): ...@@ -136,19 +151,37 @@ def clean_track_line(raw: str):
return body, bpm, transition, True return body, bpm, transition, True
def _slug_end(val): def _slug_span(val):
"""ANCHORS value → (slug, end_line|None).""" """ANCHORS value → (slug, span|None). `span` is a line COUNT from the anchor
(see ANCHORS docstring above) — the caller turns it into an absolute end_line
once the anchor itself has been resolved against the live file."""
return val if isinstance(val, tuple) else (val, None) return val if isinstance(val, tuple) else (val, None)
def block_span(lines, start_idx, end_line=None): def resolve_anchors(lines):
"""ANCHORS' header TEXT → 1-based line number, resolved fresh against `lines`.
Never guesses: a header that isn't found (renamed) or that matches more than
once (duplicated) raises loudly, rather than silently mis-anchoring the way a
stale literal line number did for months (see ANCHORS docstring)."""
out = {}
for header, val in ANCHORS.items():
matches = [i for i, ln in enumerate(lines) if ln.rstrip() == header]
if len(matches) != 1:
raise RuntimeError(
f"backlog_setlists: anchor header {header!r} "
f"{'not found' if not matches else f'matches {len(matches)} lines'} "
f"in backlog.md — the heading itself changed; fix ANCHORS.")
out[matches[0] + 1] = val
return out
def block_span(lines, start_idx, anchor_idxs, end_line=None):
"""Collect a gig block. Headers in backlog.md are DECORATIVE (stylized banners, """Collect a gig block. Headers in backlog.md are DECORATIVE (stylized banners,
repeated `# PARVAGUES`, sub-labels) not hierarchical, so we don't scope by level. repeated `# PARVAGUES`, sub-labels) not hierarchical, so we don't scope by level.
From the anchor, walk to the next ANCHOR (or an explicit end_line), terminating From the anchor, walk to the next ANCHOR (or an explicit end_line), terminating
early once the setlist proper ends — a run of ≥3 blank lines after we've collected early once the setlist proper ends — a run of ≥3 blank lines after we've collected
≥2 items (journal prose resumes). Decorative headers are kept as section context.""" ≥2 items (journal prose resumes). Decorative headers are kept as section context."""
anchor_lines = sorted(k - 1 for k in ANCHORS) hard_end = next((a for a in anchor_idxs if a > start_idx), len(lines))
hard_end = next((a for a in anchor_lines if a > start_idx), len(lines))
if end_line is not None: if end_line is not None:
hard_end = min(hard_end, end_line - 1) hard_end = min(hard_end, end_line - 1)
out, items, blanks = [], 0, 0 out, items, blanks = [], 0, 0
...@@ -174,16 +207,19 @@ _LABEL = re.compile( ...@@ -174,16 +207,19 @@ _LABEL = re.compile(
def parse(lines, name_idx): def parse(lines, name_idx):
resolved = resolve_anchors(lines)
anchor_idxs = sorted(k - 1 for k in resolved)
gigs = {} gigs = {}
for ln1, val in sorted(ANCHORS.items()): for ln1, val in sorted(resolved.items()):
slug, end_line = _slug_end(val) slug, span = _slug_span(val)
end_line = ln1 + span if span is not None else None
confirm = slug.startswith("confirm:") confirm = slug.startswith("confirm:")
real_slug = slug.split(":", 1)[1] if confirm else slug real_slug = slug.split(":", 1)[1] if confirm else slug
i = ln1 - 1 i = ln1 - 1
header = lines[i].strip() header = lines[i].strip()
section = None section = None
tracks, unmatched = [], [] tracks, unmatched = [], []
for raw in block_span(lines, i, end_line): for raw in block_span(lines, i, anchor_idxs, end_line):
hm = re.match(r"\s*#{1,6}\s+(.+)$", raw) hm = re.match(r"\s*#{1,6}\s+(.+)$", raw)
if hm: # subsection label if hm: # subsection label
section = re.sub(r"[^\w\s'-]", "", deaccent(hm.group(1))).strip() or None section = re.sub(r"[^\w\s'-]", "", deaccent(hm.group(1))).strip() or None
......
...@@ -75,9 +75,27 @@ def test_eda_coverage_is_honest(view): ...@@ -75,9 +75,27 @@ def test_eda_coverage_is_honest(view):
# ── coverage regression guards (catch silent rot in the agreement numbers) ─── # ── coverage regression guards (catch silent rot in the agreement numbers) ───
def test_tracks_total_matches_the_generated_catalog(view):
"""tracks_total must equal the CHECKED-IN catalog's own count, not a frozen
number. catalog.generated.json (build_catalog.py) is a 1:1 derivation of this
same view (n_tracks == len(view["tracks"])), so the two can only disagree when
either the corpus grew/shrank and nobody regenerated the catalog, or the view
pipeline started dropping/duplicating a row — both real regressions.
A hardcoded snapshot (`== 73`) needed hand-editing every time the catalog
legitimately grew (it went 73 -> 81 on 2026-09-05 alone), and a lone floor
(`>= 73`) would happily let the corpus SHRINK by 8 tracks unnoticed. Deriving
the expectation from the catalog itself does neither: it tracks honest growth
for free and still fails the moment the two views disagree."""
import json
from pathlib import Path
catalog_path = Path(__file__).resolve().parent.parent / "catalog.generated.json"
catalog = json.loads(catalog_path.read_text())
assert view["stats"]["tracks_total"] == catalog["n_tracks"]
def test_agreement_distribution_within_bounds(view): def test_agreement_distribution_within_bounds(view):
s = view["stats"] s = view["stats"]
assert s["tracks_total"] == 73 # update consciously if catalog grows
assert s["ac_agree"] >= 45 # was 48 — alarm if it craters assert s["ac_agree"] >= 45 # was 48 — alarm if it craters
assert s["ac_conflict"] <= 4 # was 2 — alarm if conflicts spike assert s["ac_conflict"] <= 4 # was 2 — alarm if conflicts spike
assert s["ac_divergent"] <= 6 # was 3 — wrong-link suspects assert s["ac_divergent"] <= 6 # was 3 — wrong-link suspects
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment