Commit 1c83d75a by PLN (Algolia)

docs(archive): the suite that ran zero tests, and the phantom it hid

Structured archive entry. The keeper: two bugs hid each other for a month. One
file's module-level sys.exit aborted pytest collection for the whole directory,
so the obvious command ran zero tests while 273 passed file-by-file — and that
invisibility concealed the file's own month-old red check, whose NAME was a
complete bug report about a soundcheck being counted as OPAL's first track.

A test nobody can see failing is worse than one that does not exist, because its
name reads as reassurance. Fixing the harness surfaced a real correctness bug at
no extra cost.
parent cf896762
......@@ -1902,3 +1902,77 @@ uploaded "24 s short" while every other track "matched to the second".
note. Blocks on PLN for `--go` and for `/desire`. Unblocks the 8 remaining uploads,
then OPAL's empty `audio:` field on the website and the catalog release links
behind it. `sc_generation_lens.py` is reusable and CosmicFest will need it.
## Tests — the suite that ran zero tests, and the phantom it hid (2026-09-05)
**Description.** Two test files were flagged during a pre-compact sweep:
`test_orphan_orbits.py` failing 16-rows-vs-13-expected, and
`test_lcxl3_display.py` collecting zero tests. Fix them properly — not by bumping
a number or adding a skip.
**Done.** `21f83b5` and `e8f7066`.
- `test_orphan_orbits.py`: 9 failures → 41/41. Cause was a **stale fixture**, not
a parser bug, confirmed by `git log --follow` plus an independent grep of the
`.tidal` files rather than by trusting the parser under test. `2376e43`
("d10 is the riser", 2026-08-01) added a d10 safe-riser idiom to `bombe_dj`,
`wap`, `you_my_sunshine`, `mafia_sans_serif` and `desire` *after*
`HAND_MEASURED` froze on 2026-07-28, while `piment_bresilien` went the other way
(d10→d9, PLN's own request, `71bb9bc`/`5e5a37a`). So the
`vague_de_crime→bombe_dj` orphan set legitimately shrank from `{6,10}` to `{6}`.
Setlist counts are now **derived** from the setlist file via the same pipeline
`check-tracks.sh` uses.
- `test_lcxl3_display.py`: 0 collected → **30** ordinary pytest tests. It was
plain-script style (module-level `check()` calls). Baseline correctness
established by running the script directly *before* converting.
- `test_setlist.py`: guarded its module-level `sys.exit()` under `__main__` and
added an assertion so pytest sees its checks.
- `tools/setlist.py`: added `_NOT_THE_SET_RE` (`sound ?check|line ?check|balance`)
matched against the section heading, skips announced on stderr.
- Suite: **no tests ran (INTERNALERROR) → 274 passed, 2 skipped, 0 failed.**
Canary `test_gig_log.py` 169 passed throughout.
**Learnings.**
- **`pytest tools/tests/` was running ZERO tests.** `test_setlist.py` ended in a
bare module-level `sys.exit()`, which pytest raises during **collection**, so
the whole directory aborted with an `INTERNALERROR` — while 273 tests passed
when invoked file-by-file. Every test in the repo was invisible to the obvious
command. **A collected count of 0 is a failure, never a pass**, and per-file
runs hide a collection abort completely.
- **A test nobody can see failing is worse than a test that does not exist**,
because its name reads as reassurance. `test_setlist.py` had been red for a
month on a check whose name is a complete bug report: *"a deliberation note is
not a track (the quand_on_decolle phantom)"*.
- **And the bug was real.** OPAL 2026's backlog section opens `## SOUNDCHECK /
-- Quand on decolle`, and `entries()` took every list item under the gig
heading — so **every tool reading the setlist believed the set had 16 tracks
opening on Quand on décolle**. The recorded set has 15 and opens on Ceci n'est
pas Une Bombe (`segments_v3`/`v4` track 1).
- **The right fix was the inverse of the tempting one.** `entries()` carries a
comment explaining why it refuses to whitelist which of PLN's prose headings
are *part of* the set — the first attempt died on
`## Livecoding Techno DNB Nu-jazz`. A **blocklist** naming the one phase that is
definitionally not a performance is small, stable and safe, where a whitelist of
set sections is a list of the headings he happened to write that day.
- **`backlog.md` was never wrong** and was not touched. A soundcheck genuinely
belongs in his set list; it just isn't in his set. Fix the parser, never the
data.
- **A frozen count in a fixture is a time bomb in a live corpus.** `== 13` rots
every time PLN writes a track. Derive the expectation, or the test punishes the
author for working.
- **Two bugs hid each other**, which is why both survived a month: the
uncollectable file concealed its own failure *and* everybody else's. Fixing the
harness immediately surfaced a month-old correctness bug at no extra cost.
- **Every fix was verified by breaking it.** Neutering `_NOT_THE_SET_RE` made
`test_backlog_setlist_checks_all_pass` go red naming the phantom;
wrong-valuing a `HAND_MEASURED` entry produced the exact mismatch; editing
`rec_line`'s `divmod(total, 3600)``3601` failed
`test_rec_line_formats_hmmss_past_an_hour` with `REC 1:02:02` vs `1:02:03`.
That is the only evidence that counts.
- **Restoring a deliberate break can eat the fix.** `git checkout -- tools/setlist.py`
reverted the uncommitted fix along with the sabotage. Commit before you break
things, or re-apply and re-verify — which is what happened here.
**Deps.** Fell out of the first pre-compact's R5. Unblocks nothing directly but
makes every future change verifiable in one command. Leaves one unverified
follow-up: anything consuming `setlist.entries()` **positionally** was shifted by
one for a month — `setlist_to_segments.py` is the suspect.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment