Commit 5468507b by PLN (Algolia)

perf(ardour-sweep): derive the archive path instead of searching 25 G for it

The tool worked but could not answer its own question. Proving a missing source
is gone meant `rglob(name)` per file across the Freebox — 24 full traversals of a
25 G network mount for one session. It ran 8m15s without finishing, and on the
600s budget it would have returned "not found" purely because it ran out of time.
An unfinished search reported as an absence is how you drop the last reference to
a take that was actually recoverable.

The fix is not a faster search, it is not searching. The Freebox is a converged
mirror of $HOME (fbk), so an archived file's location is COMPUTABLE: mirror root +
the path relative to $HOME. One stat per file instead of a walk. Runtime went from
"8m15s, unfinished" to 0.40s, and the answer is now exact rather than best-effort.
It still checks the session's own dead/ and audiofiles dirs, since a file can be
moved aside rather than removed, and it refuses outright if no mirror is mounted —
"gone" is not a claim you get to make about audio when the SSOT is offline.

It also now reports HOW CURRENT the mirror is, because that changes what the
absence means. Here: the mirror last changed 2026-08-29 22:41, so it holds up to
Take100. Take101 is therefore absent locally AND absent from the mirror, but was
also never eligible for that backup — so the honest statement is "it is not
anywhere I can reach", not "it was archived and then lost". The tool now says that
distinction out loud rather than letting the operator read absence as proof.

Verdict for Tidal Live: 294 sources, 24 missing, all Take101, 12 orbits x L/R.
Would drop 24 sources and 24 regions. That take's .mid files survive; only the
audio is gone.

Two bugs of my own, fixed here and worth the note: the mirror loop bound `root`,
shadowing the XML tree and killing the region sweep with "'PosixPath' object has
no attribute 'iter'" — a local rebinding eating an outer name, which is this
repo's recurring self-inflicted wound. And an ElementTree truth test that Python
3.14 deprecates.

Verified the write refusal fires: with Ardour running it stops at "Refusing to
write: Ardour is running (pid [2708769])" and the session file's mtime is
unchanged. Applying it needs Ardour closed, by design.
parent 9bb6c294
...@@ -43,7 +43,7 @@ import time ...@@ -43,7 +43,7 @@ import time
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
from pathlib import Path from pathlib import Path
FREEBOX = Path("/mnt/freebox/PLN") MIRRORS = [Path("/mnt/freebox/PLN")] # converged $HOME mirrors; SSOT for audio
def ardour_running() -> list[int]: def ardour_running() -> list[int]:
...@@ -75,33 +75,49 @@ def source_path(session: Path, src: ET.Element) -> Path: ...@@ -75,33 +75,49 @@ def source_path(session: Path, src: ET.Element) -> Path:
return session.parent / "interchange" / session.stem / sub / name return session.parent / "interchange" / session.stem / sub / name
def find_elsewhere(names: set[str], session: Path, budget: float) -> dict[str, str]: def mirror_candidates(path: Path) -> list[Path]:
"""{name: first path} for every one of `names` that survives elsewhere. """Where a mirrored backup would keep `path`, by DERIVATION not search.
ONE walk per root, matching against a set — not one rglob per file. The first The Freebox is a converged mirror of $HOME (see fbk), so an archived file's
version did the latter and spent 24 full traversals of the Freebox over the location is computable: mirror root + the path relative to $HOME. That makes
network for a single session; it never finished. Cheap to write, and the kind the question one stat instead of a filesystem walk.
of thing that only shows up on the real corpus.
The deadline is checked per DIRECTORY, not per hit, so a fruitless search The first version of this tool searched instead — `rglob(name)` per missing
still stops on time. A search that ran out of budget is reported as such by file, 24 full traversals of a 25 G network mount. It ran for over eight
the caller — "I did not find it" and "I did not finish looking" are not the minutes without finishing and would have reported "not found" purely because
same answer, and only one of them justifies dropping a reference. it ran out of budget. When the archive's layout is known, derive the path;
only fall back to searching when it is not.
"""
out = []
try:
rel = path.relative_to(Path.home())
except ValueError:
return out
for root in MIRRORS:
if root.exists():
out.append(root / rel)
return out
def find_elsewhere(missing: list[tuple[str, Path]]) -> dict[str, str]:
"""{name: path} for every missing source that survives in a mirror.
`missing` is [(name, where Ardour looked)]. Also checks the session's own
tree, since a file can be moved aside into dead/ rather than deleted.
""" """
found: dict[str, str] = {} found: dict[str, str] = {}
deadline = time.monotonic() + budget for name, wanted in missing:
for root in (session.parent, FREEBOX): for cand in mirror_candidates(wanted):
if not root.exists(): if cand.exists():
continue found[name] = str(cand)
for dirpath, dirnames, filenames in os.walk(root, onerror=lambda e: None): break
if time.monotonic() > deadline: else:
return found # Moved aside rather than removed? Cheap, local, bounded.
# The session's own audiofiles dir is where we already looked. for alt in (wanted.parent.parent / "dead", wanted.parent):
for fn in filenames: hit = alt / name
if fn in names and fn not in found: if hit.exists() and hit != wanted:
found[fn] = str(Path(dirpath) / fn) found[name] = str(hit)
if len(found) == len(names): break
return found
return found return found
...@@ -113,8 +129,6 @@ def main(argv=None) -> int: ...@@ -113,8 +129,6 @@ def main(argv=None) -> int:
help="write the session (default is a dry run)") help="write the session (default is a dry run)")
ap.add_argument("--allow-archived", action="store_true", ap.add_argument("--allow-archived", action="store_true",
help="drop references even if the file survives elsewhere") help="drop references even if the file survives elsewhere")
ap.add_argument("--search-budget", type=float, default=60.0,
help="seconds to spend proving a file is gone (default 60)")
a = ap.parse_args(argv) a = ap.parse_args(argv)
session = a.session.expanduser() session = a.session.expanduser()
...@@ -141,29 +155,44 @@ def main(argv=None) -> int: ...@@ -141,29 +155,44 @@ def main(argv=None) -> int:
print(f" {take}: {len(ss)} source(s)") print(f" {take}: {len(ss)} source(s)")
# Prove they are gone everywhere before touching a reference. # Prove they are gone everywhere before touching a reference.
names = {(s.get("name") or "") for s in missing} pairs = [((s.get("name") or ""), source_path(session, s)) for s in missing]
t0 = time.monotonic() recoverable = find_elsewhere(pairs)
recoverable = find_elsewhere(names, session, a.search_budget)
spent = time.monotonic() - t0
print(f"\nsearched the session tree and {FREEBOX} in {spent:.0f}s")
if recoverable: if recoverable:
print(f"{len(recoverable)} missing file(s) SURVIVE elsewhere:", file=sys.stderr) print(f"\n{len(recoverable)} missing file(s) SURVIVE elsewhere:", file=sys.stderr)
for name, hit in list(recoverable.items())[:5]: for name, hit in list(recoverable.items())[:5]:
print(f" {name} -> {hit}", file=sys.stderr) print(f" {name} -> {hit}", file=sys.stderr)
if not a.allow_archived: if not a.allow_archived:
print("\nRefusing: restore these instead of dropping the reference " print("\nRefusing: restore these instead of dropping the reference "
"(or pass --allow-archived).", file=sys.stderr) "(or pass --allow-archived).", file=sys.stderr)
return 1 return 1
elif spent >= a.search_budget - 1: else:
print("the search ran OUT OF BUDGET before finishing — 'not found' here " # NB: not `root` — that name holds the XML tree here, and shadowing it
"means 'not yet found'.", file=sys.stderr) # made the region sweep below fail with "'PosixPath' has no attribute
if not a.allow_archived: # 'iter'". A local rebinding quietly eating an outer name is this repo's
print("Refusing on an unfinished search: raise --search-budget, or " # recurring self-inflicted wound.
"pass --allow-archived if you know these are gone.", for mroot in MIRRORS:
if not mroot.exists():
print(f"\nWARNING: {mroot} is not mounted — 'gone' is unproven.",
file=sys.stderr) file=sys.stderr)
if not a.allow_archived:
print("Refusing on an unverifiable archive: mount it, or pass "
"--allow-archived.", file=sys.stderr)
return 1 return 1
else: print("\nconfirmed gone: absent locally and absent from every mounted mirror.")
print("confirmed gone: nothing survives in the session tree or the Freebox.") # Say how current the mirror is. An empty answer from a STALE mirror is a
# weaker fact than it looks, and the operator should see which they have.
for cand in mirror_candidates(source_path(session, missing[0])):
d = cand.parent
if not d.exists():
continue
try:
newest = max((f.stat().st_mtime for f in d.iterdir()), default=0)
except OSError:
continue
print(f" mirror {d} last changed "
f"{time.strftime('%Y-%m-%d %H:%M', time.localtime(newest))} — if "
"that predates the take, its absence there proves nothing about "
"whether it was ever archived, only that it is not there now.")
ids = {s.get("id") for s in missing} ids = {s.get("id") for s in missing}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment