# L'Armada — completed-task archive (structured)

Long-term learning + blog/video source material. Each entry = a finished board task
captured in full (description · done · learnings · deps), appended by the `/pre-compact`
ritual before the task is cleared off the active board. Richer than the terse tracker
snapshot in `board-archive.md`. Newest first.

---

## #85 — Pack-level sample grouping (the grouping above the folder)
- **Description:** The Dirt-Samples FOLDER is a loose convention, not the true grouping —
  `rample*` is one bank sliced A0..S57, `808bd/cy/hc/mc` is one drum machine per voice,
  `*_commodore` is one chip sliced by role. PLN wanted the PACK granularity above the
  folder, so we can browse/aggregate at three grains (pack / folder / sample).
- **Done:** `sample_packs.py` + `sample_packs.json` (commit 96bc7bc), captain's log 016.
  A curated REGISTRY (types: drum_machine / bank / chip / source / genre) + an auto-grouper
  (alpha-prefix-before-digit + underscore head/tail tokens, ≥2 folders) that PROPOSES
  packs the registry missed — it caught `trance_*` and `voices_*`, folded in. Reads the
  already-grounded `sample_families.json` (no audio → fast, CPU-light). 22 packs over
  83/150 grounded folders; rolls per-sample family counts up to each pack.
- **Learnings:** Each grain answers a different question — folder = "what did the pack
  author call this slice?", pack = "where did it come from?", sample = "what does it
  actually sound like?". The signal isn't agreement, it's the MISMATCH: 15 of 22 packs
  span multiple audio families BY DESIGN (rample → 8 families, jungle → 11, one Commodore
  chip → bass/lead/pad/fx/synth). A pack defined by ORIGIN is *expected* to scatter across
  the timbre wheel; that scatter is a story (for Unwrapped viz / vibe-search scoping), not
  an error. A 1-folder "pack" is just a folder within the grounded subset — require ≥2
  (may gain siblings when #84 grounds all 718). Same discipline as the folder: a hint to
  compare against, never a law.
- **Deps:** Builds on the grounded `sample_families.json` (#69 resolver); feeds the
  Unwrapped viz (#64 lineage) and vibe-search. Re-run after #84.

## #65 — Fix serve.py Range broken-pipe crash on audio seek
- **Description:** `armada/serve.py copyfile()` threw `ConnectionResetError`/`BrokenPipeError`
  (Errno 104) when a browser aborts/seeks a Range request mid-stream (e.g. `proxy_take89.mp3`
  in the triangle drawer) — the 206 responses worked but the unhandled exception spammed the
  console.
- **Done:** Wrapped both copy branches of `copyfile()` and guaranteed the file handle closes
  in a `finally`; swallow `BrokenPipeError`/`ConnectionResetError`. Verified: a mid-stream
  abort (`curl --max-time` cut) leaves NO traceback and the server keeps serving (206
  follow-up OK). Commit **d70c3ee**.
- **Learnings:** A client disconnect on `<audio>` scrubbing is *normal*, not a failure —
  swallow it, don't log it. This was the keystone QOL fix: it de-frictions every audition
  task (#35 ear-verify, #43/#44 frag-audition, #20 best-take), so cheap-but-high-leverage.
- **Deps:** Blocked nothing formally; unblocks the ear-gating workflow ergonomically.

## #64 — "Information is Beautiful" dataviz of the generated corpus (EPIC)
- **Description:** Phase-2 infodesign of the tide-table corpus (after data-scientist
  exploration) — explorable insights served to a music aficionado.
- **Done:** `corpus.html` narrative + the **Unwrapped** explorable (vibe-search via CLAP,
  dominant-colour density field, zoom/pan) + landing `index.html` + honest style cloud +
  serve.py `/vibe` warm endpoint. Shipped across the prior session's commits; vibe-search,
  field-tuning, label-completion (1490/1501 samples), and zoom all landed.
- **Learnings:** Dots are the stars — the field stays subtle; **don't average hues** (→ mud),
  use the per-cell DOMINANT family colour + purity-fade + desaturate + lift. Cache the field
  offscreen, rebuild on colour/axis/filter change, never on hover. Per-one-shot CLAP
  genre/mood is unreliable (0.3s hits) but CLAP **embeddings are gold for relative
  similarity** (vibe-search / find-similar). Seed free-text search with the user's OWN words.
  DarkReader can fight a dark theme — suspect the inverter before re-tuning to a phantom.
- **Deps:** Spawned #63 (cross-app SPA landing — the tide-table index shipped; tying in the
  React surfaces remains). Set the foundation the emotion-engine work (#89–#92) built on.

---

# The Foundry — phase-2 loops chain (overnight, 2026-06-23)

One session built the entire url→stems→**loops** chain end to end: grader, tierlist,
correlation, finder, naming, GUI, plus engine2 (merge) and audio hardening. Every claim
below is measured; honest negatives kept honest.

## #8 — engine2 Roformer VRAM spike
- **Done:** `research/engine2_spike/spike.sh`; commit 1a201df.
- **Learnings:** Roformer FITS the 6GB RTX 2060 (peak 3051 MiB, ~0.55× realtime) — the
  wall was never VRAM, it was Python: py3.14 breaks (beartype rejects audio-separator's
  `Callable|None` hints); a py3.12 venv works (`~/.virtualenvs/audio-separator`).
- **Deps:** fed #9.

## #10 — Loop grader (the katana), engine/grade.py
- **Done:** pure `grade(wav)->LoopGrade` (S/A/B/C/D + per-rule subscores + flags); commit edb6225.
- **Learnings:** (1) librosa 0.11 tempo is `librosa.feature.tempo`, NOT `.rhythm.tempo` — a
  swallowed AttributeError flat-lined bar-consistency at 0.50 for every file. (2) seam =
  CONTINUITY = the 2nd difference `|x0-2x[-1]+x[-2]|` at the wrap, not the single-sample step
  (the step metric punishes a clean wrap at a zero crossing — a perfect sine scored 0.40).
- **Deps:** reused by #5, #7, #11, #18, #19.

## #11 — Hall of Fame tierlist (data + impeccable viz)
- **Done:** `build_tierlist.py` (1690 loops/97 kits) + `tierlist.html`; commits f88273b, 4c9a831.
- **Learnings:** grade↔.tidal-usage Spearman ρ=+0.34 (rubric points right). The CONFOUND is
  the insight: FX kits (risers 16×, weird_dialogs) tier LOW correctly — sweeps aren't loops,
  so the grader tells loops from transitions; custom `f*` palette validates from the other
  side (fsynth/fguitar A). Only 34/97 kits carry a usage token (folder-name vs `s`-token).
  Showed the confound in the scatter rather than hiding it (trust the instrument).

## #12 — Correlation engine (provenance recovery), engine/correlate.py
- **Done:** NCC sample↔stem matcher + `build_provenance.py`; 482/1690 matched (29%),
  cross-stem alignment 0.34; commit 7a79dc8.
- **Learnings:** a hand-cut loop is a literal slice → NCC peaks ~1.0 (foreign ~0.03). TWO
  load-bearing bugs: (a) naive all-pairs full-res took 56 MIN → fixed with coarse-to-fine
  (rank stems on a frame-RMS ENVELOPE, precise NCC on top-3, ~40× cheaper, minutes); (b)
  SILENT-WINDOW BLOWUP — catastrophic cancellation in the sliding-variance cumsum drove the
  normaliser to ~0 in quiet regions, so silence scored ∞ and faked perfect matches (a run
  reported a bogus 55%/0.69, ~440 false positives). Floor the window norm at 5% of global
  energy + clamp NCC to [-1,1]. Also: py3.14 mp defaults to FORKSERVER → COW-global trick
  fails (empty globals per worker); pin `mp.get_context("fork")`.

## #5 + #18 — Loop finder (engine/loops.py) + v2
- **Done:** §1 pipeline (PLP→beat-sync SSM→Foote→scored windows→Takes→export); grid-from-drums;
  chop mode; commits c17dd0d, de0cb11, cdf2f13, 0a2921a, 5ac5cd0.
- **Learnings:** bare PLP latches the eighth-note pulse → ~2× tempo (Loituma read 250 not 125);
  pin tempo_min/max to one octave + min peak distance. Per-stem PLP grids are WEAK on
  sparse-onset vocals → grid-from-drums (one grid from the rhythmic stem, reused for all)
  lifted recall 0.27→0.34 (vocals 0.27→0.50). Chop mode (onset sub-bar) WORKS but recall
  0.13 < loops 0.34 — PLN picks chops by TASTE from many onsets (recall is a harsh metric;
  onsets over-segment, floored chops at 0.25s). Loops=default, chops=opt-in.

## #7 — Finder recall harness, build_finder_eval.py
- **Done:** recall vs provenance GT (distinct-window, uncapped N + precision); commit 09bace6.
- **Learnings:** honest 0.27 recall / 0.24 precision on the 8 most-cut stems — BELOW the 0.70
  target, and that's diagnostic not failure: those are chop-heavy sources (14-42 sub-bar cuts
  /stem); the bar-finder can't reproduce sub-bar chops, and the finder is a SUGGESTER anyway.

## #9 — Merge-of-stems multimodel, engine/separate.py
- **Done:** MultiModelBackend cascade (Roformer vocals → demucs on the instrumental for
  drums/bass/other → merged 4-stem) + wired RoformerBackend + demucs model selector; commits
  853eeaf, 14e69fe.
- **Learnings:** there is NO 4-stem Roformer (all are 2-stem) — the win is an ENSEMBLE.
  htdemucs_ft is a free drop-in 4-stem A/B (vocals SDR 10.8 vs 9.9). Refactored backends to
  declare `produces` + `run_composite`. Verified on Loituma: 4 clean PCM_16 stems.

## #13/#16/#17 + audio fix — supporting wins
- **#13 naming** (engine/naming.py, 146070b): NN_role[_character] derived from the corpus
  (46% indexed, 74% underscore, kit-name not repeated).
- **#16 live MIDI panel** (c0fd5d4): SSE fan-out into the Bridge, colour-by-event.
- **#17 self-updating corpus essay** (24606d4): templated Story 02 from EDA + freshness badge;
  honesty fix — 4→14 gigs is "grew 3.5×", not the hand-copied "tripled".
- **AUDIO STREAMING fix** (7c3154a): stdlib http.server defaults to HTTP/1.0 → `<audio>`
  seeking (many small Range requests) is flaky in real browsers. Set
  `protocol_version="HTTP/1.1"` on the Foundry Handler AND armada RangeHandler (safe: every
  response sets Content-Length). Also preload="metadata" so durations show (0:00/0:00 read as
  "broken"), and offer the ORIGINAL alongside stems. A fleet-wide gotcha for any audio server.

- **Deps:** the chain feeds #19 (auto-tune settings against the grader) and #20 (batch-explore
  many sources). Engine registry now seeds the product/API/service direction PLN wants.

---

# Fourier — birth of the Audio-Intelligence API (2026-06-25)

Designed live with PLN via a long AskUserQuestion co-design (5 rounds), then built lean. The
product: a self-hosted, token-authed, async API in `armada/api/` exposing the fleet's engines
(emotion, samples, features, separation, loops) to hexa (Shipow) + future seats. Full design in
memory `project_fourier_api`. Named after Joseph Fourier — the transform behind `/spectrum` and the feature stack. [Codename renamed 2026-06-29 from the original "Douanier" (Le Douanier Rousseau).]

## #22 / #36 — Scaffold + emotion walking skeleton
- **Description:** stand up the service shell + ONE real endpoint to de-risk the three seams
  (engines import · service runs · client can auth) before building auth/jobs/cache.
- **Done:** FastAPI app (`app.py`) versioned `/v1`; `/v1/healthz` + synchronous
  `POST /v1/analyze/emotion` (upload → valence/arousal via the tide-table CLAP seam
  sample_semantics.embed_audios → emotion_ontology.score); `engines/` lazy-import adapter;
  runs in `~/.virtualenvs/fourier` (--system-site-packages). Commit 7f558a7.
- **Learnings:** lean rewire — the cheap "ears" don't need the job queue/quota for one trusted
  customer, so a walking skeleton (sync + stub token) reaches value in ~1 session vs 6 tasks.
  Arch PEP-668 blocks system pip → venv that INHERITS the multi-GB ML stack, adds only FastAPI.
  CLAP cold-load is ~34 s (→ later #28 warms it once in the worker). Real pipeline proven on a
  synthetic clip end-to-end.
- **Deps:** root of the whole Fourier tree.

## #23 — Bearer-token auth + scopes + the `fourier` CLI
- **Description:** replace the dev-token stub with a real self-hosted token store.
- **Done:** `db.py` (stdlib sqlite3, WAL for API+worker concurrency); `auth.py` — mint()
  returns plaintext ONCE, stores only sha256 (DB leak unreplayable), verify() → Principal with
  scope/expiry checks, `require_scope()` dependency (401/403); `fourier.py` admin CLI
  (token mint/list/revoke). `/v1/me` echoes identity. Commit fe9ca02.
- **Learnings:** scope='*' wildcard for hexa; dev-token kept as a documented local-only
  bootstrap (prod never sets it) so curl works without minting. 15/15 tests; mint expiry landed
  exactly 31 days out.
- **Deps:** unblocks #24 quota, #33 hexa token; gates every future engine route.

## #26 — Content-addressed cache (the margin lever)
- **Description:** identical audio analyzed once, served forever — the economic core.
- **Done:** `cache.py` — content_id = sha256 of DECODED PCM (re-encodes dedupe) + raw-bytes
  fallback; params_hash folds engine settings; get/put (JSON inline or result_ref path);
  yt-id/url→content_id aliases; LRU/size-cap gc() that unlinks evicted artifacts. Wired into
  `/v1/analyze/emotion` (`X-Fourier-Cache: hit|miss`). Commit 394bec0.
- **Learnings:** **measured 5663× on a real CLAP run** (10.6 s miss → 0.002 s hit). Cache keys
  on the engine so clap/light don't collide. 22/22 tests. This is the margin story vs cloud
  egress that justified self-hosting.
- **Deps:** underpins every engine endpoint + job idempotency; unblocks #27.

## (no task) — Sampling-classics demo sources (Foundry)
- **Done:** `tools/foundry/demos.json` — 10 classics (Amen/Funky Drummer/Apache breaks, Chic
  bass, Loituma vocal, Mozart no-drums, Rickroll, GM Flash, Humpty, Think) + loader + `foundry
  demos` CLI + `/api/demos` + UI quick-picks; `/api/fetch` accepts `ytsearch1:`. Commit fd1fae8.
- **Learnings:** only verified-id URLs ship as links (Loituma from the repo fixture, Rickroll);
  the rest are `ytsearch1:` queries so we never ship a guessed/broken video id. Pedagogically
  diverse on purpose — fuel for #19 autotune + #20 batch-explore.
- **Deps:** feeds #19 / #20.

## #37 — Refit for erable: CPU image + nech.pl platform contract
- **Description:** the SRE replied (SRE.md) that erable has NO GPU / ~2 GB RAM and CLAP/torch
  won't fit; ship a CPU-deployable image. Same letter reframed api.nech.pl as a multi-API
  PLATFORM (we're the `audio` sub-API).
- **Done:** Dockerfile (python:3.12-slim + ffmpeg/libsndfile + torch-free requirements-deploy.txt)
  → **~1 GB image**, <2 GB RAM, default engine=light. Conformed the app: public path `/audio/v1`
  (gateway strips it; routes at root + `root_path`), **central auth** (trust gateway-injected
  X-Tenant/X-Scopes, loopback-only; dropped our SQLite bearer/quota = #23/#24 superseded),
  `/metrics` Prometheus, `/separate` → 503 `compute_unavailable` seam. DEPLOY.md = erable
  contract. Commit 722dade.
- **Learnings:** the SRE EDGE WENT LIVE first (503 warming_up at the public URL) — deploy is
  now a one-command unblock. The container holds ONLY armada/api/ → every engine must be
  self-contained (no tide-table/Foundry import at runtime). 32/32 tests; validated in-container.
- **Deps:** unblocks #42 deploy + #33 onboard; superseded #23/#24.

## #38 — Modular ears building blocks: /features + /analyze/samples
- **Done:** two CPU/torch-free cached endpoints. `/features` (~35-dim spectral/MFCC/chroma/key/
  envelope + rhythm); `/analyze/samples` (per-sample EDA + role percs|bass|melodic|tops|atmos by
  MEASURED spectrum, never the name). `engines/feats.py` self-contained (vendored from
  sample_features/audio_lens). Shared `_cached()` helper (DRY across the 3 analyze routes). Commit 9eb2623.
- **Learnings:** vendoring fixed the librosa-0.11 `feature.rhythm.tempo` bug the tide-table
  original silently swallowed (→ `feature.tempo`). Role-by-measurement upholds feedback_mastering_eda.
  40/40 tests; validated in the torch-free container.
- **Deps:** established the vendoring pattern reused by #39/#41.

## #39 — /grade: the Foundry loop grader as a building block
- **Done:** `/grade` exposes the Foundry mechanical quality grader (composite 0..1 + S/A/B/C/D
  tier + seam/zc/bar/level/dc/bass sub-scores + flags). `engines/grade.py` is a BYTE-FOR-BYTE
  vendored copy; tests/test_grade_endpoint.py is a DRIFT GUARD (imports the Foundry canonical
  standalone, asserts identical grade/tier/sub + WEIGHTS/THRESH). Commit cd3caa7.
- **Learnings:** vendoring + a drift-guard test = DRY-safe deployment when the container can't
  hold the source repo (parsers-over-copy applied to a vendored copy). 43/43 tests.
- **Deps:** the CPU slice of #31 (/loops + /correlate still need a corpus/finder).

## #40 — Low-hanging endpoints: /onsets, /waveform, /analyze
- **Done:** `/onsets` (hit times + tempo + rate), `/waveform` (per-bin [min,max] + 0..1 RMS
  envelope — render-ready), `/analyze` (composite emotion+features+sample in one cached call).
  `engines/signal.py` self-contained librosa. Commit 8c2be60.
- **Learnings:** /onsets+/waveform are aimed at hexa's hydra VISUALS (rhythmic sync + reactive
  envelope) — the API's value isn't just analysis, it's render-ready signal. 50/50 tests.
- **Deps:** —

## #41 — /loudness, /spectrum (FFT-as-a-service), /naming
- **Done:** `/loudness` (BS.1770 integrated LUFS via pyloudnorm + 4×-oversample true-peak +
  crest + gain-to-target -14/-9); `/spectrum` (downsampled mel/linear spectrogram bands×frames,
  0..1, frames=1=avg spectrum — PLN's "FFT-as-a-service" idea); `/naming` (vendored Foundry
  namer, drift-guarded, + feats.character_of). pyloudnorm added to requirements-deploy. Commit f11bef9.
- **Learnings:** /spectrum is the canonical cache-as-margin building block — compute the FFT
  once, serve the bounded matrix forever. **11 engines total.** 58/58 tests.
- **Deps:** —

## #32 — Make it callable: OpenAPI snapshot + zero-dep TS client
- **Done:** `clients/openapi.json` (3.1 snapshot, servers pinned to api.nech.pl/audio/v1, +
  drift-guard test vs the live app), `clients/fourier.ts` (typed, zero-dep, one method/engine,
  `_cache` surfaced, FourierError; tsc --strict clean), `clients/README.md` (Vercel example +
  scope table + codegen one-liner). Commit 11c8f9d.
- **Learnings:** a hand-written zero-dep client beats generated bloat for ~11 simple multipart
  routes; the snapshot-drift test keeps hexa's spec honest as routes evolve. 60/60 tests.
- **Deps:** unblocks #33 (onboard hexa); needs #42 deploy to be live-callable.

## #42 — Deploy fourier:latest to erable → green /audio/v1/healthz
- **Done:** First real deploy of the CPU image to erable (pln@nech.pl). Image transferred via
  `docker save | ssh erable docker load`, run loopback-only on 127.0.0.1:9780. Registered
  /openapi.json + /docs as public gateway routes (nechapi 89586eb). DEPLOY.md gotchas captured
  (Tidal fbe1742). Green end-to-end: healthz/openapi/docs 200, authed 401 without token.
- **Learnings:** TWO erable traps — (1) `--security-opt seccomp=unconfined` is MANDATORY:
  Docker 19.03/kernel 4.9 default seccomp blocks the `clone3` syscall that python:3.12-slim's
  glibc 2.36 uses for pthread_create → container boots uvicorn then SIGSEGVs (exit 139) /
  "OpenBLAS pthread_create … Operation not permitted". (2) pin OPENBLAS/OMP/NUMEXPR/MKL
  _NUM_THREADS=1. The _platform service sidesteps #1 by being Alpine/musl on purpose. Data
  volume = ~/srv/fourier/data (no sudo for /srv; pure transient cache).
- **Deps:** unblocked #33; the keystone for the whole live API.

## #43 — Hierarchical scope convention (realm:domain:path) + central enforcement + CLI/docs
- **Done:** Convention `realm:domain:path` (realm=api|admin|future mail/data); required scope
  DERIVED from the path (api:audio:analyze:emotion), subtree wildcards (*, api:*, api:audio:*),
  non-wildcard=exact, api:* never grants admin:*. Canonical matcher `_platform/scopes.py` vendored
  byte-identical into `armada/api/scopes.py` (drift-guarded). Audio app: per-route scope strings
  replaced by ONE path-derived `require` dependency + footgun fix (missing X-Scopes → "" not "*").
  CENTRAL enforcement deployed in `_auth` (derive + 403). CLI validates scopes + `check` cmd.
  nechapi 104748f, Tidal 4197696. Verified: hexa api:*→200, admin admin:*→403 on audio, no-id→401.
- **Learnings:** PLN chose a top-level REALM (vs flat domain root) to future-proof for non-API
  capabilities (mail, personal data). Path-derived scopes = maintenance-free access control
  (add a route, its scope exists). The realm split is what lets a customer hold api:* (everything
  API) while admin stays a separate sealed plane. Tenant migration caveat: the active `maps`
  token has OLD-style geo:read/route — must become api:geo:* BEFORE geo joins the gateway.
- **Deps:** built on #37 platform reframe; enables clean multi-API + selling seats.

## #33 — Onboard Shipow / hexa (the MVP delivery)
- **Done:** hexa minted as tenant 3, plan pro, scope api:* (full platform access). Verified
  end-to-end through https://api.nech.pl/audio/v1 with his real token: /me, /features (cache
  miss→hit), /analyze/emotion. Branded onboarding PDF (clients/onboarding.html template →
  chromium --print-to-pdf; rendered PDF gitignored as it carries the token). Tidal b225ba7.
- **Learnings:** platform tokens DON'T expire (the schema has no expiry — revoke-by-prefix
  instead), so the old "31-day" idea didn't apply. Token plaintext is shown ONCE at mint (only
  the sha256 is stored). Onboarding artifact is transitional — PLN's stance: Scalar /docs +
  landing will replace the PDF; "only token distribution matters." Client SDK renamed
  Fourier→NechAPI (umbrella + .audio namespace) + header X-Fourier-Cache→X-Nech-Cache so the
  internal codename never reaches a consumer.
- **Deps:** was blocked by #42 (deploy) + #43 (scopes); closes the EPIC's MVP.

## #45 — Platform observability: usage analytics, security monitoring, uptime status landing
- **Done:** (1) CAPTURE — `_auth` logs EVERY attempt to usage_log (status 200/401/403 + reason +
  client ip; tenant_id nullable; quota counts 200 only); migration +reason +ip +health_checks.
  (2) ADMIN ANALYTICS (admin:*-gated, self-authed): /admin/usage (totals, by-tenant, top routes,
  14d series) + /admin/security (rejects by reason, offender IPs, recent rejects). (3) PUBLIC
  STATUS — api.nech.pl/ now renders per-API up/down + uptime% (24h/7d/30d) from a 60s background
  probe → health_checks; ca-certificates added for the https probe. nechapi e489d26. Verified:
  landing renders, admin endpoints return real data, hexa api:*→403 on /admin (plane sealed),
  rejects logged with source IP.
- **Learnings:** before this, _auth returned early on reject and logged NOTHING — a total
  security blind spot; logging the decision (not just authorized calls) is the foundation.
  Probing the PUBLIC healthz URL (not loopback) measures the real edge a caller sees — and a
  container can't reach the host's 127.0.0.1 anyway. nginx reload briefly fails one probe → a
  small transient uptime dip that self-heals. The admin/api realm split makes /admin trivially
  safe (an api:* token is auto-403'd).
- **Deps:** built on #43 (admin realm) + #45 capture; Grafana visual layer scheduled as #46.

## #34 — Ops: systemd units, observability, SRE deploy runbook
- **Description:** the ops half of the audio-API rollout — keep-alive units, observability,
  a deploy runbook for SRE.
- **Done:** closed as SUBSUMED by the platform's paved road (which appeared in a parallel SRE
  session between sessions). Observability shipped earlier (nechapi f89d7f9 — per-call capture,
  /admin analytics, public uptime status page); the systemd-restart + deploy-runbook half is now
  the golden base image (nechapi/py) + `nechapi ship` CLI + the onboarding runbook in the SRE
  repo, not bespoke units here. Reconciliation documented in armada/api/SRE.md (Tidal d228966).
- **Learnings:** the platform repo had DIVERGED — my 3 unpushed control-plane commits
  (observability, scope convention, audio-LIVE) vs origin's 10-ahead paved-road refactor (golden
  base, ship CLI, edge-cache tiers, geo/v1 + iris/v1 onboarded). Resolved by rebasing mine onto
  origin/main: only docs + the registry + the generated vhost conflicted; the core observability
  code (_platform/app.py, scopes.py, schema.sql) was untouched upstream and replayed clean. The
  generated vhost (api.nech.pl.conf) is re-rendered, never hand-merged. PARKED RISK flagged: geo
  is live with old-style scopes (geo:read) but my central enforcement derives api:geo:* — so if
  _platform is redeployed WITH central enforcement, geo tokens 403 until re-minted; enforcement
  stays parked (audio self-enforces) and that reconcile is SRE's.
- **Deps:** part of the EPIC #21 thread; observability lineage = #45.

## #35 — Test suite + CI for the API
- **Description:** a real test suite for the audio API plus a CI pipeline.
- **Done:** the 67-test suite already existed and was green, but only under a
  --system-site-packages venv (quietly host-dependent). Made it hermetic:
  requirements-test.txt pins the true test surface (fastapi/uvicorn/multipart + numpy + soundfile
  + librosa + pyloudnorm + httpx + pytest), NOT system-site-packages; .gitea/workflows/api-ci.yml
  runs it on git.plnech.fr for any push/PR touching armada/api/**. Tidal 213eb8a. Clean-venv
  rehearsal: 67 passed.
- **Learnings:** a system python bump to 3.14 had dropped fastapi/httpx from the old venv → the
  whole suite reported "no tests collected" (red for an ENV reason, not a code reason) — exactly
  what a site-packages-inheriting CI would hide. Rehearsing in a clean throwaway venv surfaced
  TWO deps the system venv had silently supplied: librosa (lazy-imported inside engine functions,
  not at module load — so a grep for eager imports missed it) and pyloudnorm (the LUFS engine).
  The heavy ML stack (torch/laion_clap/demucs) is NOT needed by the tests, so CI installs fast.
  CI is dormant until an act_runner is registered for the repo; the workflow is correct + locally
  rehearsed.
- **Deps:** part of EPIC #21; was a blocker on #21.

## #25 — Async job model + worker daemon (SQLite)
- **Description:** the async backbone so heavy work (sources/separate/loops) runs out-of-band and the API stays responsive.
- **Done:** committed 17d3f46. Four design decisions taken with PLN (AskUserQuestion w/ sketches): HYBRID sync/async (cheap analyses stay sync, heavy = jobs), PER-ENGINE submit (202 {job_id}) + SHARED poll (GET /jobs/{id}) + cancel (DELETE), SINGLE serialized worker, X-Accel-Redirect for artifacts. Built: jobs table (db.py), jobs.py (enqueue + born-done idempotent fast path, atomic claim, progress/finish/fail-requeue-under-cap, crash recovery, cooperative cancel), worker.py (serialized recover→claim→dispatch→webhook loop, SIGTERM-graceful, @jobs.handler registry), typed JobStatus endpoints, deploy/fourier-worker.service (systemd --user). 11 new tests (67→78), incl. a real 6-thread×25-job no-double-claim concurrency test. OpenAPI refreshed (15 ops) + @nech/api regenerated (getJob/cancelJob).
- **Learnings:** SQLite WAL serializes writers, so the claim race reduces to candidate-select → `UPDATE ... WHERE id=? AND status='pending'` + rowcount-0 retry — no locks needed, provably exactly-once across threads. typescript-fetch's generated instanceOf guard indexes a camel/snake key union that trips TS7053 under noImplicitAny → relaxed noImplicitAny for the 100%-generated package (emitted .d.ts stays fully typed). MID-BUILD: the btrfs disk hit ENOSPC at 98% (20G "free" but unallocated, metadata-full) — writes failed intermittently; cleared ~5.7G of regenerable caches (npm + ~/.cache browser/HF, user-approved) to continue. Disk still 97% — flagged for real cleanup.
- **Deps:** was blocked by scaffold + central auth (done); unblocks #29 → #30 → #31. Worker prod container-vs-host wiring is a paved-road residual (SRE/#34).

## #27 — Artifact store (local disk on erable; freebox deferred)
- **Description:** somewhere to keep the big rebuildable binaries (fetched sources, later stems/loops) that don't belong inline in SQLite, served efficiently to hexa.
- **Done:** committed c7e5392. `artifacts.py`: content-addressed layout `ARTIFACTS_ROOT/<cid>/<name>`, tracked as ordinary `cache` rows (kind→result_ref→path, bytes) so the **existing LRU `cache.gc()` evicts them for free** under `FOURIER_ARTIFACTS_MAX_MB` (1 GB). `resolve()` refuses path traversal / escapes outside the root. Serving = zero-copy nginx `X-Accel-Redirect` (`FOURIER_X_ACCEL`), `FileResponse` fallback in dev. `GET /artifacts/{cid}/{name}` endpoint (auth-gated, any tenant holding the opaque ref). DEPLOY.md §5b documents the internal nginx location. The cache schema's `result_ref` column + `gc()` unlink were pre-built in #25/#26 *for* this — #27 was mostly wiring.
- **Learnings:** PLN dropped freebox-as-SSOT ("forget freebox storage for now") → the store collapsed from "signed Range URLs over a remote mount" to "local dir + nginx internal location", much simpler. Reusing the cache table for artifacts means one eviction policy, one stats surface, one GC — no separate artifact lifecycle. The `_meter` middleware had to switch from `request.url.path` to the **route template** (`request.scope["route"].path_format`) or `/artifacts/{cid}` + `/jobs/{id}` would each mint an unbounded Prometheus series.
- **Deps:** was blocked-by-nothing once #25 landed; unblocks #30 (stems need somewhere to live). Go-live (nginx location + FOURIER_X_ACCEL) tracked in #47.

## #29 — /sources endpoint (yt-dlp fetch as a job)
- **Description:** turn a URL into analysis-ready audio — the front door of the heavy chain (sources→separate→loops).
- **Done:** committed c7e5392. `engines/sources.py` registers the `sources` job handler: yt-dlp → `bestaudio` → 44.1k WAV (PLN's choice: analysis-ready PCM over keep-source-codec), content-address, store as a `source` artifact, `put_alias(url→cid)`. `POST /sources` returns `202 {job_id}`; a re-fetched URL (alias hit + artifact still on disk) is a **born-`done` job** — no re-download. `normalize_url()` validates: http(s) only, lowercases host, drops fragment (so trivially-different URLs dedupe), and hard-blocks SSRF footguns (localhost, RFC1918/loopback/link-local/reserved IP literals, incl. the 169.254.169.254 cloud-metadata classic). The fetch is a thin seam (`_fetch_audio_wav`) so tests mock the network with a synthetic WAV — 17 new tests (78→95), zero network. healthz reports yt-dlp presence honestly.
- **Learnings:** PLN chose OPEN url scope (any http(s)) over an allowlist, explicitly leaning on "our apikey + nechapi monitoring" as the protection layer — so the code accepts any public URL but still refuses the SSRF set in-process (defense the gateway can't do). yt-dlp+ffmpeg are **worker-only** deps (the API container stays torch/yt-dlp-free) → added to requirements-deploy, not requirements-test. The idempotent born-done path falls out of the #25 `enqueue(..., result=...)` fast path + #26 aliases — the URL→cid alias table was designed in #26 *for* exactly this.
- **Deps:** blocked-by #25 (done); unblocks #30 (separate consumes the fetched WAV). Go-live (yt-dlp install on erable) tracked in #47.

## #48 — Gate /docs + /openapi.json behind a cross-cutting api:docs scope
- **Description:** PLN noticed `api.nech.pl/docs` "down" + asked if docs should be token-protected. Investigation found two things: (1) root `/docs` is 404 (the platform Scalar portal was never built; the 404 catch-all's "See /docs" hint dangles), and (2) the real issue — audio's `public_routes` exposed `/audio/v1/{docs,openapi.json}` ANONYMOUSLY: the full API surface advertised to the internet.
- **Done:** deployed live 2026-06-29. `required_for()` special-cases `docs|openapi.json|redoc` → `{realm}:docs` (canonical `nechapi/_platform/scopes.py` commit on branch `claude/docs-scope`, rebased onto upstream + merged to main `0ab148b`; vendored byte-identical into `armada/api/scopes.py`, commit `d4d27a4`). Dropped docs/openapi from `audio.public_routes` (kept `/healthz`), re-rendered the vhost. Rebuilt + shipped `nechapi-platform:latest` (Alpine, Postgres-backed so no token loss), recreated the container on `nechapi-net`, installed the re-rendered vhost (diffed-first: only my 2 blocks removed), `nginx -t && reload`. VERIFIED LIVE: anon /docs+/openapi.json → 401 (was 200), /healthz → 200; a minted `api:docs` token → /docs 200 but POST /analyze/emotion → 403 (decoupling proven); test token revoked. `api:*` (hexa/pln) keeps docs access via subtree.
- **Learnings:** the cross-cutting scope (`api:docs`, not per-API `api:audio:docs`) decouples "read the specs" from "call the API" — a docs-reader token can't invoke anything, and `api:*` still covers both via the subtree wildcard. The drift guard reads the MAIN `~/Work/nechapi` checkout, so the canonical + vendored copies must merge in LOCKSTEP — committing the vendored copy ahead of the canonical merge turns the local suite red (CI skips it when the canonical is absent). Resolved by reverting the premature vendor, staging on a branch, and re-vendoring only after the nechapi merge. Also re-learned the shared-repo race: origin/main had advanced (a parallel SRE `bin/nechapi` drift-guard commit) — rebased onto it (no overlap), and ALWAYS diff the rendered vhost vs the LIVE one before `sudo tee` so a parallel session's work isn't clobbered.
- **Deps:** spun out of the #29/#27 deploy session; touched the security control plane (gated). Follow-up: build the platform `/docs` Scalar portal (basic-auth RTFM/PLN per registry) + fix the 404 "See /docs" dangling hint.

## #44 — Official generated clients + private npm (Verdaccio)
- **Description:** ship a real, never-drifts client for hexa + future seats, distributed from a private registry — replacing the interim hand-written `nech.ts` drop-in.
- **Done:** deployed live 2026-06-29. (a) Generated `@nech/api` TS client from the OpenAPI snapshot (openapi-generator typescript-fetch → tsup ESM bundle; clean operationIds → `audio.analyzeEmotion()`; subpath exports). (b) Stood up **Verdaccio 6** on erable (loopback `:4873`, `~/srv/verdaccio/{storage,conf}`, `unless-stopped`) behind an LE-TLS nginx vhost **`npm.nech.pl`**; seeded publisher `pln` via the user API, locked signups (`max_users:-1`), creds 0600 at `~/.config/nechapi/verdaccio.pln.{pw,token}`. (c) Published `@nech/api@0.1.0` and install-verified as a consumer (AudioApi/Configuration import clean). (d) Retired `nech.ts` (`git rm`), updated README + onboarding.html to the `@nech/api` install path. Commits: client `ed77ee9`/regens, retire `851767f`; VERDACCIO.md marked DEPLOYED.
- **Learnings:** three Verdaccio gotchas the bare runbook missed — `listen: 0.0.0.0:4873` (else binds localhost INSIDE the container, dead `-p` map), `chown -R 10001:65533` the bind-mount (verdaccio uid), and `ssl_protocols TLSv1.2;` ONLY (erable's old OpenSSL rejects TLSv1.3 and fails `nginx -t`). Non-interactive publish: create the user via `PUT /-/user/org.couchdb.user:<name>` to get a token, then `npm publish --userconfig <temp .npmrc with _authToken>` — no interactive `npm adduser`/`login` needed. KEY DECISION: `@nech/*` reads are `$authenticated` (not public) — consistent with the bearer-gated `/openapi.json` (#48); so consumers need an npm READ token, a deliberate consistency call over install-convenience.
- **Deps:** unblocked by the OpenAPI snapshot (#32); part of EPIC #21. Follow-ups (in `armada/api/TODO.md`): a `nech_api` Python client; mint hexa a read token.

## #63 — The silent-orbit epidemic: `doNotReadYet` loads WAV headers only
- **Description:** for two evenings, orbits in `the_revolution_will_be_sampled` (d4/d5/d9-d12, varying per boot) made no sound. Events were visible in the Pulsar highlighter, no error appeared anywhere, and every static check — patterns, g* helpers, sample banks, libsndfile readability, orbit count, PipeWire links, Ardour routing, saved faders — pronounced the rig healthy while it was demonstrably not.
- **Done:** root cause identified 2026-07-28. SuperDirt `DirtSoundLibrary.sc:221` reads `Buffer.readWithInfo(server, path, onlyHeader: doNotReadYet)`. With `~dirt.doNotReadYet = true` (start_and_midi.scd:202) it reads **only the WAV header**: the buffer is allocated with a correct frame count and duration but contains NO AUDIO DATA. That is verbatim the `Buffer UGen: no buffer data` message. The deferred real read is what decides whether an orbit sounds, so which orbits die changes per boot, and a re-eval sometimes revives one by retriggering the read. Fix (eager-load the set's banks) spun out to #26 and deliberately NOT shipped at J-7 without index-integrity validation. Tooling shipped alongside: `872676a` (probe-chain contamination guard), `a43f428`+`d3de9bd` (remote control channel + CLI), `66af326` (eager package activation, which also closed #59).
- **Learnings:** THE METHODOLOGICAL ONE, worth more than the bug: every measurement used a 4-8 second window, and at that timescale a TIME-VARYING failure looks STATIC. "These two sample banks are dead" was overwhelmingly convincing and completely wrong; six hypotheses were built and demolished on top of it (banks, symlinks, 24-bit depth, filename characters, buffer exhaustion, xfade). The decisive observation took one command: play `arpy`, measure at t+3s (-37.0 dBFS rms, healthy) and again at t+26s (-inf, gone), no re-eval between. Patterns play, then die. Sampling ONE subject at TWO times would have found this in five minutes. Second lesson, more uncomfortable: probe-chain FABRICATED signal — contaminated capture taps reported five orbits at an identical +0.0 dBFS peak (twelve independent orbits don't do that; it was the mix measured five times), and that false table said d4/d5 had recovered while PLN's ears and Ardour's meters said otherwise. A measurement tool that invents signal outranks the human in an argument it should always lose; it now refuses to report unverified rows. Also: `find` does NOT follow symlinked directories without `-L`, which produced a confident "these files don't exist" about 69 files that do; and a bank name doesn't reveal its loader (`Dirt-Samples/kick` doesn't exist as a folder, yet `s "kick"` plays, because banks also come from `Samples/extra/*` and tidal-drum-machines).
- **Deps:** absorbed the `no buffer data` half of #45 (whose remaining 69 `zz._` AppleDouble warnings are harmless junk); handed the fix to #26, now the gig blocker; unblocked #59 as a side effect.

## #59 — "Sound broke after a Window Reload, fixed by a Tidal reboot"
- **Description:** a Pulsar `Window: Reload` reliably killed audio, and rebooting Tidal reliably restored it. Filed as a mysterious interaction between the reload and the audio stack.
- **Done:** fixed 2026-07-28 (`66af326`, pulsar-tidalcycles). Nothing was breaking. `activationHooks: ["source.tidalcycles:root-scope"]` plus `activationCommands` made Pulsar DEFER loading the package, so a reload landed in a window where the Tidal package was simply DORMANT — no ghci, none of its three UDP ports bound, and nothing saying so. `tidalcycles:reboot` happened to be the first *activationCommand* in the list, so "rebooting Tidal" worked by WAKING the package, not by fixing anything. Removed both keys; the package is now eager.
- **Learnings:** the remedy worked for a reason unrelated to the story we told about it, which is exactly why the cause stayed hidden for so long — a reliable workaround stops anyone asking why. Proof was structural rather than inferred: the renderer's `~/.pulsar/blob-store` contained no compiled tidalcycles code at all, so `lib/tidalcycles` had never been required (a good general check for "did this package actually load?"). Design consequence worth keeping: lazy activation and remote control are mutually exclusive on principle — a channel whose purpose is removing the manual keypress cannot itself require a keypress to come into existence. And shipping package JS always costs exactly one reload, because new code cannot be injected into a running Electron renderer.
- **Deps:** found while debugging why the new remote channel (#44/a43f428) wasn't listening; unblocks unattended launch work (#44).

---

## #26 — GIG-CRITICAL: silent orbits (root cause was OUR preload, not SuperDirt)

**Description.** For two evenings, orbits went silent mid-set. *Which* ones changed every
boot. The boot log looked perfect — every bank registered with the right file count — and
19,741 `Buffer UGen: no buffer data` messages piled up in one 6h session where no
performer would ever see them. Six orbits of `the_revolution_will_be_sampled` were dead on
stage. J-7 to OPAL 2026.

**Done.** `b4f6275` (branch `claude/preload-eager-banks`), in `tools/setlist_samples.py`'s
emitter, plus `setlist_opal2026.txt` and `gig-up.sh`. Also `0ae4116` (probe timeline +
check-mix session resolution) and, in the fork, `374d8a3` (stop swallowing GHC errors).

**Learnings.**

*It was a two-part interaction, and part two was ours.* Upstream, `doNotReadYet = true`
registers banks from WAV **headers only**; the audio read is fired *asynchronously* by
`readFileIfNecessary` (`DirtSoundLibrary.sc:268-274`), which returns immediately — so the
synth for the very event that triggered it gets a bufnum with no samples. That alone is a
first-hit glitch. What made it fatal: our generated `preload.scd` set
`doNotReadYet = false` and **never set it back**, and that flag gates the *only* caller of
`readFileIfNecessary` (`:259-261`). Flag false ⇒ a header-only buffer is never read. Not
late — **never**. So any bank the whitelist missed was permanently silent.

*And the whitelist missed banks by design.* `gig-up.sh` built it from the N
most-recently-**edited** tracks. `the_revolution_will_be_sampled` hadn't been touched
recently, so `like_sugar` and `the_revolution` were never warmed — exactly the banks
feeding the six dead orbits. Whitelist-by-mtime is also why *which* orbits died changed
every boot. The thing that made this feel haunted was file timestamps.

*The load-bearing line is one line.* Restoring `doNotReadYet = true` after warming demotes
the whitelist from load-bearing to a mere optimisation: a miss becomes a slow first hit
instead of a dead orbit. That is the only thing it is safe to be on stage.

*A reload REPLACES a bank, it does not append.* `addBuffer` defaults
`appendToExisting = false` and frees the old bank on the first file (`:42-46`), so a plain
re-load is index-stable — `like_sugar:21` keeps its meaning. An earlier session's worry
about *doubled* indices was wrong, and that wrong belief had made the fix look far more
delicate than it was. Measured: **0 of 1428** bank names even collide across the three
loaders.

*The count assertion caught a bug in itself on its first run,* which is the best possible
outcome for a guard: `jbk_kick: expected 508, got 254`. Exactly 2×, and the *tool* was
wrong. That folder symlinks into the rhadamanthe pack, which carries an AppleDouble
`._X.wav` beside every real `X.wav`; `glob('*.wav')` matches both, SuperDirt's
`pathMatch("*")` skips dotfiles, and `ls` hides them so the folder looks clean.

*Red herring worth remembering:* SuperDirt's "Required 0 MB of memory" is not evidence the
eager read failed — `memoryFootprint` counts frames from the header, so header-only
buffers already report full size. Only audio measurement can prove a preload worked.

**Validation (instrumental, no ears — PLN was at work).** Before: d4/d5/d9/d10/d11/d12 all
`-inf` at source. After: all 11 declared orbits carry signal *and still do* on a second
probe ~95 s later with no re-eval (stable to ~0.2 dB) — the two-timepoint check being
mandatory for this bug class. `no buffer data` 19,741-in-6h → **1** (a boot artifact at the
instant the preload completed). Preload asserts **47/47** banks, 2352 files, 2.2 s, 1.2 GB.
Also green: `perfect` 12/12, `gimme_acid` 10/10, `mafia_sans_serif` 7/7, `wap` 5/5. Bare
`s "like_sugar*4"` audible. Confirmed by PLN's ears later that day: *"i hear indeed all
sounds in the track, so it sounds good to me!"*

*Two harness failures during validation, both worth more than the fix.* A batch loop
returned four EMPTY results that read exactly like "nothing is playing" — cause: **zsh does
not word-split unquoted variables**, so `-d $ORB` passed one bogus argument and the tool
exited 2 with a usage error my grep hid. And the readiness gate passes ~4 s too early,
because `listening to Tidal` is printed by `~dirt.start` well before the warm-up. *When a
batch of measurements all come back empty, suspect the harness before the rig.*

**Deps.** Closed #26 and #63. Unmasked #61 (which then measured *clean* for the OPAL set —
all 41 seeded CCs cover it). Spawned #65 (an undefined `crackles` silencing nine orbits,
which is a compile error, not audio), #66, #67. Feeds #44 (readiness gate) and #24.

---

## #66 — Prove all 10 OPAL tracks playable + every control audibly effective

**Description.** PLN left the laptop running at J-7 and authorised autonomous work:
*"ensure all tracks from gig are playable. you can send fake midi events for each, thus
test all controls have impact on sound ;) even the crushes should change noticeably what
you see/'hear'"*. No ears in the room — every claim instrumental, every failure reported.

**Done.** `check-tracks.sh` end-to-end: **10/10 tracks, 91 orbits, zero silent, zero eaten
by an Ardour fader.** Two were broken at the start of the day — `desire.tidal` at **0/9**
(`off 0.125 (|+ note 12 . (|* gain 0.6))`: `.` binds tighter than the section, so it
composed a function into a ValueMap; one pair of parens → 9/9, `cfa56b6`) and
`vague_de_crime` (PLN's own `crackles` fix, saved). Every control measured effective with
the lens that suits it: `gF1` centroid 112→3681 Hz, `gF3` 384→2765 Hz (+529%), `gMute2`
−23 dB→silence, `crushbus` 125→189 Hz (+37%), `ply ^89` 8.1→4.7 onsets/s, `gMask` −33%
onsets, `gMute1`+`gM1` silencing four orbits at once. `ply ^91` on d7 reported
**INCONCLUSIVE** and left that way. Shipped `tools/control-lens.py`,
`tools/check-tracks.sh`, `tools/tempo-lens.py`; commits `cfa56b6` `708c2ae` `7b2e5b4`
`16429b0` `36466a4` `4836d74` `104fb03` `156444d`.

**Learnings — the instruments were wrong more often than the rig, six times, and the
arithmetic was never the bug.**
1. *rms is blind to a filter or a crusher* — they move timbre at constant loudness.
   `gMask` read "NO CHANGE, 0.5 dB": right sum, wrong question; on the density lens, −42%.
2. *`ply` reported "+inf %, EFFECTIVE" off a division by zero* — the envelope onset
   detector read 0.0/s on a sustained bass. The tool invented an effect, the one thing it
   promises never to do. Fixed with spectral flux + an INCONCLUSIVE floor.
3. *A decay crosses its own midpoint exactly once.* A healthy `mask "<t f!7>"` riser was
   called "DECAYING, almost certainly an xfade tail" twice. Two quartile averages cannot
   answer a monotonicity question; counting crossings can.
4. *One low FINAL bin is the window closing, not a fade* — `perfect`'s d4, steady for
   twelve bins, condemned on the thirteenth.
5. *A uniform +1.8% tempo bias* came from a threshold rule sliding down the left flank of
   the RIGHT peak — comparing points on one hill as if they were different hills. (I first
   blamed `np.correlate`'s unnormalised overlap and fixed that too; the bias didn't budge.
   **Fixing a real-but-irrelevant flaw is not evidence you found the cause.**)
6. *The observer was the effect.* Ardour's xrun counter climbed +816 in 25 s under the ply
   stress test — which reads exactly like "heavy patterns cause dropouts on gig week" and
   matched PLN's own crackle report. Remove the ~18 PipeWire capture nodes probe-chain
   creates, repeat the identical musical load: **zero xruns**, scsynth ERR = 1 all session.

*And the discipline that makes the rest survivable:* **A-B-A**. These patterns vary per
cycle by themselves (`d2` of `vague_de_crime` plays a different bar every cycle), so
re-measure the baseline at the END and treat any swing under 2× the self-drift as
inconclusive. Plus: take the baseline **twice** — a single closing repeat agreed with the
opening one by luck and reported drift 0.0% when the true within-condition spread was 33%.

*Also:* `you_my_sunshine`'s alarming "144 vs 166" tempo mismatch was **a commented-out
`setcps` line** — a parser miss masquerading as a data conflict, again. And the RUNBOOK's
pre-set checklist had been telling PLN to put the DJ filters DOWN for bypass; **centre is
bypass**, down is a hard low-pass.

**Deps.** Closed #65. Fed #62 (the live mixer is clean; the SAVED file still isn't),
#56 (event density exonerated as a crackle cause), #12 (one of three tempo questions
dissolved), #33. Spawned #68 (orphan orbits). Ran alongside #67 (LED painter) in a
parallel agent.

---

## #58 — Verify the DJF LED ramp live after reload

**Description.** The LCXL LED feedback had been built, wired into `tidal-remote.py boot`
and `gig-up.sh`, and shipped — but never once seen to light a physical LED. Every
validation to that point ran against a *fake* `aseqdump` stream. PLN's repeated reports
("still seeing static colors", "no touch-reactiveness actually neither on button nor
knobs") could not be distinguished from "the tool is completely broken" because nobody
on either side of the keyboard could observe the device.

**Done.** Closed 2026-07-28 by PLN on real hardware: *"i see hardware led btw, and press
states work! ... i see also filters and effects working as expected"*. The value→colour
ramp, the button press states and the effect colours are all confirmed live.

Supporting work landed the same night (`80b732b`): `tools/mock-lcxl.py`, a virtual
LaunchControl XL that decodes the same SysEx the device receives and renders it in the
terminal, so the convention can be designed, reviewed and regression-tested without
hardware — plus `value_ramp()`, PLN's six-step convention, and 8 regression tests.

**Learnings.**

- *The observability gap was the bug's hiding place.* For three days the LED thread was
  stuck not because the code was wrong but because neither party could see the output.
  Building the mock took under an hour and immediately explained the complaint that had
  survived two rounds of fixes.
- *"Correct" and "looks broken" can be the same picture.* The mock showed knobC 1-3
  painting **bright amber (63)** at rest — which is exactly `filter_colour(64)`, the DJ
  filters at centre/bypass, behaving precisely as designed. Bright amber at rest is
  visually indistinguishable from the factory yellow PLN was trying to escape. The code
  passed every test and still produced the exact experience it was meant to eliminate.
- *A second finding from the same run:* `control_colour()` returns `hue["full"]` for every
  value from 8 to 119, so a touched knob is one flat bright colour across its whole
  travel. Touch changes it once and then it never moves again — the literal definition of
  the "static colors" report.
- *The palette is the design constraint.* The LCXL is bicolor: 2 bits red × 2 bits green =
  16 states, of which roughly six read reliably on a dim stage. PLN independently asked
  for a six-step ramp. The hardware budget and his taste landed on the same number, which
  is a good sign that the convention is the right shape.
- *Transport, re-learned:* `amidi` works on a cold rig and fails on a live one, because
  SuperCollider's ALSA sequencer holds the rawmidi substream. `aseqsend` to the
  **sequencer** port coexists. The tool that works only when nothing else is running is
  the tool that fails at the only moment that matters.

**Deps.** Unblocked the remaining LED thread: #11 (colour convention — the two design
findings above are now its open questions) and #71 (latency: the LEDs work, but lag 1-2 s
behind fast fader moves, because every changed LED costs a `subprocess` fork+exec inside
the reader loop).

---

## #71 — LCXL LEDs lag 1-2s behind fast fader moves

**Description.** First night the LED feedback ran on real hardware, PLN's verdict was
"its great tbh i see also filters and effects working as expected :) but! its slow to
track if i move all fders quickly i see the animations lagging a sec or two behind".

**Done.** `128de62`. A `Painter` thread now owns the wire; the aseqdump reader only touches
the model. Dedupe on COLOUR, coalesce per frame, batch all dirty indices into one SysEx,
and cache port resolution for 2 s (dropped on any send failure). Added `--bench`, and
`tools/at/tests/test_lcxl_latency.py` (12 tests).

**Learnings.**
- *The bug was not what it looked like.* It looked like a slow device or a slow transport.
  It was that we deduped on the CC **value** while colour is a **step function** of value —
  six steps in `value_ramp`, seven in `filter_colour`. A 128-event fader sweep can change
  the board at most 6 times; we were asking the wire for ~20x the work that could possibly
  be visible. Fixing the *representation* mattered more than fixing the I/O.
- *Three forks per LED.* Every `send()` re-resolved the port with `aconnect -o` AND
  `amidi -l` before writing — ~1.7 ms each, measured. The uncached lookup was a deliberate
  earlier fix (a replug had moved the device 20:0 → 24:0), so the cache had to keep that
  property: cache for 2 s, and invalidate **on evidence** (a failed write) rather than on a
  timer. Real cost fell 7.70 ms → 2.72 ms per write.
- *A pipe queues, it does not drop.* All the I/O ran inside `for line in proc.stdout`, so a
  write in flight blocked reading the next MIDI event. Nothing was lost — everything simply
  arrived later, without bound. That unboundedness is why PLN perceived "a second or two"
  rather than a fixed delay. Latency that grows is a different diagnosis from latency that
  is constant.
- *Numbers, replaying the actual complaint.* 8 knobs + 8 faders swept together at 500
  events/s, transport modelled at its measured cost: legacy 744 wire msgs, 3 s of input
  taking 11.23 s (**+8.23 s overrun**); coalesced 48 msgs, +0.00 s, p50 7.9 ms / p99
  20.2 ms. Overrun *is* the visible lag — how far behind his hands the board finishes.
- *Rate-limit as a floor, not a tick.* Limiting the GAP between writes (rather than
  painting on a fixed clock) keeps an isolated button press instant while still bounding
  the burst rate. A fixed tick would have added latency to the common case.
- *The test that matters is the correctness one.* Fast-and-wrong beats nothing, so the
  suite decodes every SysEx the coalescer emitted through `mock-lcxl` and asserts the board
  that LANDS equals a full `build_frame` recomputation.

**Deps.** Followed #58 (which only became possible once LEDs were verified on hardware at
all). Surfaced #72. Its `Painter` became the template for the Bridge fix.

---

## #72 — Bridge MIDI stream dropped the NEWEST event

**Description.** Not reported by anyone — found by sweeping the rest of the gear for #71's
bug class, on the theory that a bug shaped like that rarely lives alone.

**Done.** `1c9df10`. Drop-OLDEST on a full subscriber queue; new `midistream.coalesce()`;
SSE batched into one write+flush per frame. Extracted `_fanout` for testability. 8 new
tests, 32 green in `tools/bridge`.

**Learnings.**
- *`except queue.Full: pass` is a silent lie.* It keeps 512 stale events and discards the
  one that just happened. For MIDI **state** the newest event IS the truth, so a dashboard
  tab that stalled for ~1.3 s at 400 CC/s would show frozen values for the rest of the set
  with nothing logged. Exactly #71's signature: nothing breaks, it just quietly stops
  telling you the truth.
- *State vs events is the whole design.* Continuous controls (CC, pitchbend, aftertouch)
  may be superseded by a newer value; notes may never be dropped. A fast monitor that loses
  a note is strictly worse than a slow one — so the tests assert every note survives a
  flood of 100 controller messages.
- *Preserve chronology.* A superseded value is overwritten **where it stood**, not moved to
  the end, so the monitor still reads as a timeline.
- *Wire format untouched.* Batching emits one `data:` line per event as before, so the
  browser needed no change — a perf fix with a zero-diff client.
- *Green tests on `parse_line` proved nothing about the queue behind it.* Had to extract a
  seam to test the drop policy at all. Same lesson as `feedback_verify_the_plumbing`.

**Deps.** Discovered by #71's sweep. Feeds #25/#42 (the Bridge as cockpit).

---

## #53 — Make master gain fixed pre-set, not live-MIDI (CC77)

**Description.** `midiGGlobal = orDef 0.769 "^77" * 1.3` in BootTidal.hs put a global gain
on one physical fader.

**Done.** `9647e44`. Now `midiGGlobal = 1.0`. Verified with `tools/check-boot.sh`: helper
block typechecks against tidal-1.9.5 under `ghc -fno-code`, the #55 seed block typechecks,
and all 13 g* helpers (incl. `midiGdef`, the one this edit could plausibly break) still
yield events against an EMPTY control map.

**Learnings.**
- *Two independent reasons, and the second was the payoff.* Safety: brushing fader 1 in the
  dark attenuated every midiG-using stream at once, silently. But the unlock was
  ergonomic — Tidal reading `^77` is what kept the whole surface one lane off. Killing it
  freed fader 1 and made fader N → dN possible (#46). A safety fix that also buys a feature
  is worth looking for.
- *Make the refactor provably inaudible.* The old untouched default evaluated to
  `0.769 * 1.3 = 0.9997`, so 1.0 moves every track by 0.003 dB. Choosing the constant to
  match the old default meant no track needed re-levelling.
- *Do not delete a definition the corpus calls.* PLN believed he no longer used `midiG`, and
  behaviourally he was right — with the global term gone it reduces to plain `gain (range
  lo hi cc)`. But `midiG'` appears in 167 files and `midiG` in 12, and deleting a referenced
  definition is a Haskell compile error, which takes the whole `let` block down and silences
  the entire track. Retiring the *name* is #73, post-gig, and the ORDER matters: rewrite the
  call sites first, delete last. Reversed, it is a 179-track mute-bomb.

**Deps.** Unblocked #46. Spawned #73.

---

## #75 — gig-log: a session recorder, so a run-through can be REVIEWED not just felt

**Description.** After a set, "did it glitch?", "did it get hot?" and "which controls did I
actually use?" were answered from memory. PLN asked for logging in place before his morning
run-through, which he records in Ardour: *"we def gotta ensure tomorrow morning when i start
you have logging in place so we can then review the perf and validate."*

**Done** (`4c57362`). `tools/gig-log.py record | mark | report | status | install | selftest`,
running as an enabled `gig-log.service`. 1 Hz JSONL: temps, fan, freq, core+package throttle
deltas, per-core cpu%, per-gear %cpu + RSS (scsynth/sclang/ArdourGUI/whole pulsar group),
PipeWire xruns per node, and every LCXL move. Wall-clock stamped so it aligns with the Ardour
take. `--report` renders sparklines, an xrun timeline, a gear table and a per-CC surface table
with first/last touch. 71 tests.

**Learnings.**
- **Measured, not asserted: 0.40% of one core** (270 pw-top lines parsed in 10 s), 0.79% as
  the live unit. My first selftest said 0.66% and was *flattering itself* — it never started
  the reader threads, so it measured a sampler with nothing to parse. Same mistake as the
  first `--bench` run in the LED work. Re-measured lower AND honest.
- **The logger must not be a firehose.** CC/pitchbend coalesced to one line per control per
  second with count + first/last/min/max — the #71/#72 fix applied to ourselves. Notes are
  never coalesced: a CC is a state, a note is an event.
- **Three wrong xrun baselines before the right one**, each caught by reading the report:
  first-sight → 72096 xruns on a 25 s *idle* session (pw-top prints a zero-filled table
  before the profiler has data); second-sight → silently wrong when TWO zero tables landed;
  finally **max over each node's first 5 sightings**, which needs no timing assumption
  because ERR is monotonic. A rule that is right *most* of the time is the worst kind for a
  gig log, because the one bad reading looks exactly like a disaster.
- **Ardour accumulates ~6 xruns/min with nothing playing** (67072 → 67123 → 67134 across
  captures minutes apart). Only the session delta may ever be quoted.
- Verified in production: **exactly 1.000 Hz, p99 gap 1.001 s, no drift** over 5654 samples.
- Fixed in the shared reader: `perf._read(None)` raised TypeError instead of returning the
  default, so a machine with no coretemp/dell_smm hwmon **crashed** the thermal read — the
  Bridge shares that path.
- `/proc/<pid>/stat` must be split after the LAST `)`: pulsar renderer comms look like
  `(pulsar) --type=renderer`, so splitting from the left silently reports the wrong column.

**Deps.** Unblocks the measurement half of #8 (thermals under load), #56 (Pulsar starving
audio) and #14 (rehearsal review). Feeds #44.

---

## #11 — LCXL colour, settled: red lows / bright-amber centre / green highs, and the ramp finally wired

**Description.** Two taste calls PLN settled by eye on the hardware: sunset the dim-amber
steps on the DJ filters (*"just have red in lows green in highs for clarity"*, *"i agree on
clarity > resolution"*), and make the knobs show more than on/off.

**Done** (`0b23f3d`). `filter_colour` reduced from seven steps to three palette corners —
red / bright amber at the 61-67 detent / green. `value_ramp` wired into `control_colour` for
knobs. Module header rewritten to describe the convention that now exists. +24 tests.

**Learnings.**
- *"wait 'brightness value 6 steps ramp'? I See only two states, dim and not dim"* — he was
  right. `value_ramp` was written, unit-tested, documented in the header, and **never
  called**; `control_colour` had three outcomes, so 30% and 90% were the same colour. The
  old knob rule had **no test at all**, which is exactly how a never-called ramp shipped.
- The old filter ramp spent four of seven steps on shades either side of centre, so the row
  read as a wash of oranges and the one thing you need — *which way is it cutting* — was the
  hardest bit to see. Three unambiguous states beat seven nobody can resolve on a dark stage.
- **The trade, stated so it cannot regress silently:** the ramp spends all three hues on
  VALUE, so hue no longer carries ROLE on rows A/B/C. Right way round — role is fixed by
  POSITION, a knob's setting has no other channel. Keeping "untouched" as a colour collided
  (dim green would mean both "rhythm, untouched" and "value ~3/4"); it moves to the HUD.

**Deps.** Closes the #11 taste call. #49/#51 (colour-by-function, legend) still open.

---

## #76 — panic red must track ^93, not chord membership

**Description.** PLN: *"why are [mutes] 1/2/3 resp red red green? these 3 buttons have same
roles, why diff colours?"*

**Done** (`0b23f3d`). `panic` threaded in from the model (`values[93]`); chord members paint
by role when clear; arming flashes red on all four as a global overlay applied last.

**Learnings.**
- **The panic chord is an OVERLOAD, not a dedicated function.** Hold 73+74+91+92 → SC
  edge-detects and flips persistent `^93`. Those buttons individually are gMute1/gMute2. The
  panic identity **does not exist at rest**, so painting it at rest is simply wrong.
- **The bug was in TWO places**, and the second was only found by *rendering the board*
  after fixing the first: `control_colour` tested chord membership before role, AND
  `parse_track` force-bound all four members to role "fx" via setdefault. A chord is not a
  binding. Verify by rendering, not by reasoning.
- `^93` owns no LED (CC93 is outside row F's 89-92), so those four buttons are the only
  place the armed state can live — and it is the highest-value LED in the rig, because it
  answers *"why is there no sound?"*. Hence overlay-over-dark, not overlay-over-role-only.
- The watch loop rebuilds the whole frame on a `^93` flip: one event changes four LEDs, so
  it is the one event a single-index paint cannot serve.

**Deps.** #34 (panic live-verify) still wants PLN's hands on the four buttons.

---

## #68 — orphan orbits: name the ghosts before the stage does

**Description.** PLN by ear: loaded vague_de_crime, played bombe_dj, *"heard diam's voice,
now still hearing the synth from crimewave... i hear it regardless of gains, but filters work
on them"*.

**Done** (`3ab9ae0`). `armada/setlist_opal2026.txt` (the set as data, resolved from
backlog codenames by locating each file) + `tools/orphan-orbits.py`
(chain / `--matrix` / `--pair` / `--silence` / `--json`) + 39 tests.

**Learnings.**
- `dN` replaces orbit N and says **nothing** about the others. **Both of PLN's symptoms are
  the proof and are diagnostic in general**: "regardless of gains" because the new track's
  gain lines address only its own orbits; "filters work on them" because gF3 is the global
  CC51 filter applied *inside* the ghost's own still-running pattern.
- **8 of 12 OPAL transitions leave ghosts, 16 orphaned orbits.** The ear-report reproduces
  exactly: vague_de_crime → bombe_dj leaves {6, 10}, and d6 IS the crimewave synth.
- **The find worth more than the report: Perfect <3 declares ALL TWELVE orbits, so its
  matrix column is entirely clean — a free RESET POINT.** Put it after the messiest stretch
  and ghosts stop propagating. A real ordering lever for #12.
- **Cross-validated**: the parser's orbit sets match the ten hand-measured on 2026-07-28.
  Tool and ear agree.
- **Stopped short of automating the fix, deliberately.** `eval-file` calls
  `atom.workspace.open(..., activatePane: true)` — it opens AND activates, so it would yank
  the editor away from the track being played mid-transition. Worse than the ghost.
- `silence`, never `# gain 0`: gain 0 leaves the pattern scheduled, burning a voice, and a
  later global gain change revives it.

**Deps.** Feeds #12 (order) and #13 (transitions). Fix choice needs PLN.

---

## #69 — the tray now says what gear is ALREADY UP

**Description.** PLN: *"i rightclick the perf indicator and see no gear status?"*

**Done** (`8bbcfd9`). A `Gear ▸ 3/6 up` submenu labelled ● running / ○ stopped / ✗ missing,
refreshed on menu-open only. `is_running` rewritten forkless. +11 tests.

**Learnings.**
- The state existed all along (`launchers.is_running`); `_build_menu` ran once at startup
  and read only `available`. **The safe action ("it's already up") looked identical to the
  dangerous one** — two SuperColliders is a zombie port 6010 and a silent rig.
- **THREE worse bugs surfaced while wiring it up, each found by checking the output:**
  1. `ARDOUR_SESSION` pointed at **"Tidal Multi"**, the ARCHIVE, not "Tidal Live". The tray's
     Ardour button opened the wrong session — the same mixup that produced a confidently
     wrong fader report on 2026-07-28, one layer down, 6 days before OPAL.
  2. Candidates were `ardour8/7/6`; the installed binary is `ardour9` (`ardour-9.2.0`). The
     entry reported "unavailable" while Ardour was running on the same machine.
  3. `pgrep -f ardour` matched the `tidal-ardour-autoroute.sh` helper **and any shell whose
     command line mentioned the word**. It could report Ardour UP while it was DOWN — worse
     than reporting nothing, because it is the state you act on. Now matches the executable
     basename, with full-line matching kept only for interpreted tools.
- **No forks:** one `pgrep` per launcher became a single shared /proc scan — 6 items in
  10.7 ms, and `snapshot()` is called by the web Bridge's poll too. Same lesson as the LED
  daemon's per-event fork.
- Glyphs not colour: a tray menu inherits the desktop palette, and this gets read in a dark
  room seconds before playing.

**Deps.** Unblocks #25 (the Bridge as single cockpit).

---

## 2026-07-29 — the day the seed was found to have never run (OPAL D-5)

### #79 — eval produces a SILENT pattern; the 4-cycle xfade masks it as a "drift to silence"
**Description.** PLN, playing: *"ctrl_enter on do it right, i hear the 4-bar pattern
each bar lower, 4th barely audible, its clearly a xfade"*, and on wap *"no bass? when
i move the knob C5 it starts sounding"*. Filed as an xfade bug.

**Done.** Root cause found and fixed in `2648074`. It was neither the xfade nor the
tracks. `pulsar-tidalcycles` splits BootTidal.hs on BLANK LINES, strips the `:{`/`:}`
it finds, and re-wraps each chunk in its own `:{ ... :}` — which holds exactly ONE
statement. The #55 control seed was `:{ let _seed = ... :}` followed by `mapM_` and
`putStrLn` with no blank lines, so all three fused into one statement and died with
a parse error. **The seed never ran once between 2026-07-27 and 2026-07-29.** Fix =
two blank lines, documented in-file as load-bearing. Verified live: the boot now
prints `[BootTidal] seeded 51 LCXL controls (#55)` and PLN confirmed it.

**Learnings.**
- *Every `dN` is `xfade N` with `xfadeIn 4`.* The four bars you hear after an eval
  are the PREVIOUS pattern leaving. A silent eval gets a graceful exit, so the bug
  presents as a drift. **Sound after ctrl+enter is evidence the LAST eval worked.**
- Measured with the new `silent-eval.py`: empty control map → **23 orbits silent
  across 10 of 10 buildable tracks**, including `desire` d1-d6, the entire track.
  With the seed → **zero**. Necessary and sufficient.
- **Why every guard was green.** `check-boot.sh` proved the helper block typechecks,
  the seed block typechecks, and the helpers emit events on an empty map. All true,
  all irrelevant — they test the Haskell; the failure was in how the file is CHUNKED
  AND FED. #61 had already suspected exactly this and was closed on that comfort.
  *Verifying that code is correct is not verifying that it runs.*
- Two hypotheses were killed from a FILE with no rig and nobody's ears, by the
  gig-log `controls` view: DJ filters parked at 51/59/80 (open) and gMask/gMute all
  at 0. That is the value of a parked-state readout.
- Guard added (`check-boot-blocks.py`, check-boot pass 4): replays Pulsar's chunking
  bug-for-bug — including that JS `.replace(str, str)` is NON-GLOBAL — and fails on
  parse errors only. Negative-tested: names block 26 at BootTidal.hs:663.

**Deps.** Blocked #77, #74, #13 — all now unblocked. Re-closed #61 with the real
answer.

### #82 — gig-log report: show each control's LAST VALUE
**Done.** `d1b21c9`. `left at` column, a `controls` subcommand translating each
parked value into what it MEANS, and `report --from/--to` windowing.
**Learnings.** Windowing has one non-obvious requirement: xrun/throttle are RUNNING
TOTALS, so a naive slice reports the whole session's count inside the window —
wrong, and wrong high; `slice_session()` rebases. Transcribing `gDJF` for the Hz
column surfaced that the lowpass half is LINEAR IN HERTZ, so the knob feels inert
then collapses in its last 2% — one gesture producing both "went quiet" and "sounds
lpf'd", the exact symptom pair #79 was filed with.
**Deps.** The keystone that unlocked #79.

### #78 — unmapped knobs must be DARK
**Done.** `2acbd9c`. Published current-track file; `--map` publishes as well as
paints; `--watch` follows it at 1 Hz and repaints.
**Learnings.** The colour code was never wrong — `build_frame` already paints only
bound CCs. The lie was WHICH BINDINGS it was handed: the watcher ran with no track,
so it painted the convention board (40/40 lit) regardless of what was loaded.
Measured: `do_it_right` binds 19/40, so **21 dead knobs were glowing**; wap 15;
desire 18. **The value ramp did not create the problem — it made a pre-existing lie
legible.** When a cosmetic change surfaces a complaint, check whether the cosmetics
exposed bad DATA before changing the cosmetics back.
**Deps.** Left #84 (publish from Pulsar's ctrl+enter path) as the honest remainder.

### #8 — THERMAL: 17024 package-throttle events
**Done.** Answered from the gig-log recording, no new work needed. **0 core and 0
package throttles** across 10h48m with audio, 72 °C peak. Heat is exonerated; the
17024 was an absolute counter including power excursions.
**Learnings.** Counters are deltas, not totals — and *measure the set, not the
corpus*: PLN corrected that only ~90 min of that session was playing, so every
aggregate over the full 10h48m described the wrong thing. That correction is what
motivated `report --from/--to`.

### #81 — "gig-log recorder is DEAD"
**Closed INVALID. The premise was false and the error was mine.**
**Learnings.** I asked systemd about `parvagues-gig-log`, a name I GUESSED; the unit
is `gig-log.service`, and it had been up 11 hours. A wrong unit name returns
"inactive", indistinguishable from a dead service. I then used that false negative to
resolve the report's deliberate ambiguity (*"still running, or it was killed"*) the
wrong way — **a broken check settling the very question it could not answer**. Two
lessons: read identifiers, don't guess them; and a tool must not offer the reader a
choice it can make itself. `a4dc37b` makes the report decide from DATA RECENCY (not
a process match, since `pgrep -f` has its own false-positive problem) and print
either `● RECORDING NOW` or `the recorder was KILLED`.

### #50 / #52 — LED colour tail (archived late, 2026-07-29)
Two small completed LED tasks that never got an archive entry and were cluttering
the board. Recorded here for completeness rather than value.

**#50 — Value-reactive LED colour for gF1/gF2/gF3 (now that DJF audio is fixed).**
A duplicate of #48, filed when the DJF audio bug was fixed and it became worth
doing "for real". Superseded by the settled colour language in #11/#76 and by
`81aecc9`, which corrected the report to model BOTH filter halves — see the
`reference_djf_two_halves_and_event_lag` memory: hard right is an 8 kHz high-pass,
not "open", so any value-reactive colour for row C must be bipolar.

**#52 — Fix button-LED reverting to factory yellow on repeat press.**
The "factory yellow" was never the device's default: it was a real Midi-Through
feedback loop echoing note velocity 127 back as an LED colour byte
(`reference_lcxl_midi_feedback_loop`). It regenerates on a Tidal reboot, so the
symptom returning is not a regression in the LED code.
