docs(log 023): the checks that could not fail

Four silences in one afternoon, with a gig that night, and every health check
reporting perfect: SC up with zero restarts, Tidal holding 6010, 48 graph links
present, master at -0.2 dB, interface clocking 48 kHz both ways. All true, all
useless.

What cracked it was PLN mentioning the system beeps had gone 'legato 0.2'. A
device that truncates a short sound is a device being restarted, and nothing in
the stack could see that because every lens sampled a moment and the fault lived
between them. trigger_time turned out to be the only restart counter anything
publishes: the UMC's playback stream had restarted 26 times in 15 minutes while
its capture stream — which drives the whole graph — never restarted once.

The log's real subject is the two tools I built to find it, and the fact that
both shipped blind. rig-pulse called SOUND on 'synths > 0 or peak > 1.5%' when
this rig idles at 71 synths and 17.7%, so it read healthy straight through a
silence its owner was complaining about. Then it learned the floor as a running
minimum, which fails because the floor rises. Meanwhile audio-lens compared
consecutive polls with a half-second minimum gap while polling every quarter
second, so its rate and stall checks never ran at all.

A check that never fires is indistinguishable from a check that passes, and the
only reason I found all three is that the rig kept obligingly breaking while I
watched.

armada/tasks/023-the-checks-that-could-not-fail.md

+---
+log: 023
+title: "The checks that could not fail"
+date: 2026-08-14
+task: "#129 #154 #155 #156 — an afternoon of 'd'un seul coup plus de son'"
+tags: [observability, debugging, midi, usb, tidal, rig]
+shareable: true
+---
+
+## Cap (what & why)
+
+*"Je jouais, d'un seul coup plus de son."* Four times in one afternoon, with a
+gig that night.
+
+Every check said the rig was fine. SuperCollider up with zero restarts and a
+clean log. Tidal's GHC alive, holding port 6010. The PipeWire graph fully
+linked, all 48 SuperCollider→Ardour connections present. Ardour's master at
+−0.2 dB, reaching both interface channels. The interface clocking a steady
+48 000 Hz on both directions.
+
+All true. All useless.
+
+## Manœuvre (how)
+
+The thing that actually cracked it was PLN's ear, not a check. He mentioned in
+passing that the system volume beeps had gone *"legato 0.2"* — truncated. A
+device that eats the head off a short sound is a device being stopped and
+restarted, and nothing in the stack could see that, because every lens sampled a
+moment and the fault lived between moments.
+
+So: build the lens that watches over time, then use it.
+
+## Prise (findings / artifacts)
+
+- **`trigger_time` is the only restart counter in the stack.** It changes solely
+  when a PCM stream is (re)started — no log line, no error, PipeWire reports the
+  node `running` on both sides of the event. It showed the UMC's **playback
+  stream restarting 26 times in 15 minutes while its capture stream never
+  restarted once** — and `pw-top` showed the capture node is the graph *driver*.
+  The graph clocked on serenely over an output that had gone away.
+- **The interface sat on a hub shared with a USB-C video adaptor**, negotiating
+  12 Mbps, with 43 kernel clock-source rejections in the same window. (#154)
+- **Three of four MIDI taps were watching the wrong device.** After a replug the
+  LCXL moved to ALSA client 20 and an Arturia KeyStep took 24 — and gig-log,
+  lcxl-leds and the HUD were all still bound to 24. They resolve by name, but
+  once, at start. `lcxl-leds.py:89` documents that exact hazard and does it
+  anyway. Fixed in prod by restarting both units. (#129, #155)
+- **The last stop was `^41`.** A bare `^NN` whose CC has never been received
+  yields no events at all; `midiOn` wraps it in `orDef 0`, so an untouched
+  button reads as *not pressed* and the pattern plays on 0% of cycles. And ^41
+  was FREED when gMask retired in July — nothing sends it any more, so it is a
+  guaranteed landmine sitting in five corpus files. Silence, by design,
+  indistinguishable from a crash. (#156)
+
+## Sel (the shareable learning)
+
+**I shipped a check that could not fail, one commit after writing a message
+about how dangerous those are.**
+
+`rig-pulse` was built to answer one question: when the sound stops, did Tidal
+stop asking or did the engine stop playing? Version one called it SOUND if the
+synth count was above zero or the DSP load above 1.5%. Reasonable numbers. They
+are also numbers this rig never goes below: it *idles* at 71 synths and 17.7%
+peak, because the global effects registered on all fourteen orbits live forever.
+So the tool reported SOUND continuously, straight through a silence its owner
+was actively complaining about, and could never once have fired.
+
+Version two learned the floor instead of assuming it — a running minimum. Also
+wrong, and only live data showed why: the floor *rises*, 71 synths at 15:02 and
+103 at 15:05, and a baseline that only ratchets down cannot track a rig that
+ratchets up.
+
+What finally worked was in the data the whole time. An idle SuperDirt is not
+approximately still, it is *exactly* still: five samples over eight seconds
+returned ugens=3531 every single time. A played one jitters 4144→4300 in two
+seconds. Compare samples to each other, ignore the level entirely, and the
+question of what the floor *is* stops mattering.
+
+The same bug was in the other tool simultaneously: its rate and stall checks
+compared consecutive polls and required a gap of half a second, while polling
+every quarter second. They never ran. A 900-second watch collected two samples.
+The restart detector carried the entire investigation while half the instrument
+was decoration.
+
+A check that never fires is indistinguishable from a check that passes. The only
+way to tell them apart is to run the thing against a state you *know* is bad —
+and I only found all three because the rig kept obligingly breaking.
+
+## Hameçon (hook)
+
+*"My monitoring said everything was healthy. It was right. It was also incapable
+of saying anything else."*