midiviz: the set itself paints the lens — Tidal HL feed + the ParVagues wave

BootTidal.hs mirrors every superdirtShape packet to port 57130, oscfeed
listens, and a trigger on orbit N lights that orbit's column and the cells
that affect it — family filters and mutes included. The edge is a ring, not
a fill: it says 'the set just played through me', never a value, never a
touch. The wave (brand magenta, vendored from the www SSOT asset) sits
behind everything on 'w'; the feed toggles on 'h' and is on by default.
Column glow keeps its 30% slower curve.

BootTidal.hs

@@ -47,9 +47,18 @@ editorShape = OSCContext "/editor/highlights"
 -- SuperDirt target
 superdirtTarget' = superdirtTarget {oLatency = 0.1, oAddress = "127.0.0.1", oPort = 57120}
 
+-- midiviz mirror target (2026-09-24): the SAME /play packets SuperDirt
+-- receives, replayed to a port midiviz owns (tools/bridge/oscfeed.py). This
+-- is the HL feed: what the MUSIC is doing — which orbit triggered, with
+-- which values — as opposed to the MIDI feed, which only says what the
+-- SURFACE is doing. Fire-and-forget UDP: when midiviz is not listening the
+-- datagrams vanish, and the audio path never notices.
+midivizTarget = superdirtTarget {oName = "ParVaguesMidiviz", oLatency = 0.1, oAddress = "127.0.0.1", oPort = 57130}
+
 -- Set up targets for SuperDirt, DPV, metadata, AND editor highlighting
 oscmapdpv = [(targetdpv, formatsdpv),
             (superdirtTarget', [superdirtShape]),
+            (midivizTarget, [superdirtShape]),  -- the HL mirror for midiviz
             (metadataTarget, []),
             (editorTarget, [editorShape])  -- Added editor highlighting
             ]

tasks/completed_archive.json

@@ -2,7 +2,7 @@
   {
     "date": "2026-09-17",
     "task": "GUI launcher did not spawn the MIDI lens or the tray",
-    "learning": "The launcher (gig-up.sh) and the unit converger (rig_units.py --ensure) disagreed on SuperDirt topology \u2014 raw sclang vs parvagues-sc.service \u2014 so the launcher never called ensure; the fix is ensure()'s proof-process skip, not another hand-copied unit list."
+    "learning": "The launcher (gig-up.sh) and the unit converger (rig_units.py --ensure) disagreed on SuperDirt topology — raw sclang vs parvagues-sc.service — so the launcher never called ensure; the fix is ensure()'s proof-process skip, not another hand-copied unit list."
   },
   {
     "date": "2026-09-17",
@@ -12,16 +12,16 @@
   {
     "date": "2026-09-18",
     "task": "tray GIG UP did nothing + midimon silent",
-    "learning": "The Bridge ran 'gig-up.sh --converge --quiet' but gig-up.sh's arg guard only knew --headphones/--help, so every tray GIG UP died on flag 1 (exit 2, output captured). Flags now parsed: --converge boots SuperDirt + converges units with NO app windows, --quiet mutes ok/info. midimon watches 132:0 (the lcxl3-driver's translated virtual port), not the raw surface \u2014 silence there just means the driver had no input yet."
+    "learning": "The Bridge ran 'gig-up.sh --converge --quiet' but gig-up.sh's arg guard only knew --headphones/--help, so every tray GIG UP died on flag 1 (exit 2, output captured). Flags now parsed: --converge boots SuperDirt + converges units with NO app windows, --quiet mutes ok/info. midimon watches 132:0 (the lcxl3-driver's translated virtual port), not the raw surface — silence there just means the driver had no input yet."
   },
   {
-    "learning": "Two gig-up.sh exist: root (1-click, owns leds()) and tools/ (the GO/NO-GO gate the Bridge calls, where lcxl3-driver is boot=manual so nobody starts it). The v3 LCXL3 enumerates as 'LCXL3 1', breaking every v2 'Launch Control XL' grep \u2014 present/LEDs/path checks all silently misread. lcxl-path must match the driver's virtual PORT name 'ParVagues LCXL3' (under RtMidiOut client), never chase the PipeWire relay mesh (everything reaches everything via 143)."
+    "learning": "Two gig-up.sh exist: root (1-click, owns leds()) and tools/ (the GO/NO-GO gate the Bridge calls, where lcxl3-driver is boot=manual so nobody starts it). The v3 LCXL3 enumerates as 'LCXL3 1', breaking every v2 'Launch Control XL' grep — present/LEDs/path checks all silently misread. lcxl-path must match the driver's virtual PORT name 'ParVagues LCXL3' (under RtMidiOut client), never chase the PipeWire relay mesh (everything reaches everything via 143)."
   },
   {
-    "learning": "WirePlumber 0.4.17 ignores wireplumber.conf.d SPA-JSON rules (0.5 syntax) \u2014 this box reads Lua from main.lua.d only, and a partial alsa_monitor.rules reassignment clobbers the built-in table by index (dropped the card to profile off). Every wireplumber restart tears down Ardour's JACK client; recovery needed a full Ardour relaunch. pactl/wireplumber restart restored the missing HiFi profile after a UCM probe failure (card showed only off/pro-audio)."
+    "learning": "WirePlumber 0.4.17 ignores wireplumber.conf.d SPA-JSON rules (0.5 syntax) — this box reads Lua from main.lua.d only, and a partial alsa_monitor.rules reassignment clobbers the built-in table by index (dropped the card to profile off). Every wireplumber restart tears down Ardour's JACK client; recovery needed a full Ardour relaunch. pactl/wireplumber restart restored the missing HiFi profile after a UCM probe failure (card showed only off/pro-audio)."
   },
   {
-    "learning": "A generated boot file is only as good as the list it is generated FROM. gig-up.sh rewrote preload.scd on every launch from a default naming the PREVIOUS gig (setlist_opal2026.txt): measured 132 banks -> 45, leaving 41 of Thursday's banks to be read off disk mid-set \u2014 the exact 'crackle at the venue' failure the check was built for, re-armed by a stale default. Two structural lessons. (1) Name the set ONCE and have every consumer read that; the plan generator and the gate disagreeing about 'the set' is the whole bug. (2) An unconditional write is destructive in the quiet direction: the plan in place was a deliberate 132-bank UNION and regeneration could only narrow it, so gen_preload now asks check-preload first and leaves a covering plan alone. Corollary from the same pass: merging two tools that share a name means diffing what they DO, not just what they check \u2014 the root launcher span raw sclang while the other started parvagues-sc.service, so repointing the Bridge at 'one launcher' would have silently taken it off systemd (losing rlimits, the journal, supervision and OOM protection)."
+    "learning": "A generated boot file is only as good as the list it is generated FROM. gig-up.sh rewrote preload.scd on every launch from a default naming the PREVIOUS gig (setlist_opal2026.txt): measured 132 banks -> 45, leaving 41 of Thursday's banks to be read off disk mid-set — the exact 'crackle at the venue' failure the check was built for, re-armed by a stale default. Two structural lessons. (1) Name the set ONCE and have every consumer read that; the plan generator and the gate disagreeing about 'the set' is the whole bug. (2) An unconditional write is destructive in the quiet direction: the plan in place was a deliberate 132-bank UNION and regeneration could only narrow it, so gen_preload now asks check-preload first and leaves a covering plan alone. Corollary from the same pass: merging two tools that share a name means diffing what they DO, not just what they check — the root launcher span raw sclang while the other started parvagues-sc.service, so repointing the Bridge at 'one launcher' would have silently taken it off systemd (losing rlimits, the journal, supervision and OOM protection)."
   },
   {
     "learning": "A launcher's exit code cannot tell 'it worked' from 'it did nothing'. gig-up.sh --help prints its usage and exits 0, and the window wrapper closed on that zero exactly as it closes on a good launch -- so the rig had never once been started from the icon, and the only evidence was the LCXL staying dark. The log had been written faithfully all along and held nothing but the usage text. Structural lesson: for any wrapper that closes on success, the success test must be a POSITIVE marker the wrapped thing prints on its real path ('gig-up done'), never the absence of an error. And hold the coupling from both ends in tests, because the day someone rewords that last line, every real launch starts reporting a failure."
@@ -56,5 +56,10 @@
     "date": "2026-09-23",
     "task": "stale unit builds",
     "learning": "A systemd unit holds whatever the interpreter read at ExecStart, so from outside a unit running three-day-old code is indistinguishable from a healthy one: same active(running), same journal. A pass was lost debugging a midiviz that predated the fixes being debugged. The only honest test is the unit's ActiveEnterTimestamp against its ExecStart file's mtime -- deliberately NOT a git comparison, because the question is 'is the process executing what I can read', so an uncommitted edit counts."
+  },
+  {
+    "date": "2026-09-24",
+    "task": "midiviz: Tidal HL feed + ParVagues wave backdrop, column glow slowed 30%",
+    "learning": "Tidal already ships 4 OSC targets in BootTidal.hs — mirroring superdirtShape to a 5th port (oscfeed.VIZ_PORT 57130) gives the HL trigger feed without touching SuperDirt's port; and an OSC name/value fold must compare str(k+1) keys, not int, or every named arg silently vanishes."
   }
-]
+]
\ No newline at end of file

tools/bridge/oscfeed.py

+#!/usr/bin/env python3
+"""oscfeed — the Tidal HL feed: SuperDirt's own /play packets, snoopable.
+
+Tidal does not talk to SuperDirt alone. `BootTidal.hs` mirrors every
+`superdirtShape` packet to a second target (port `VIZ_PORT`), so anything may
+listen to what the music is DOING — which orbit triggered, with which values —
+without touching the audio path, the way the Pulsar editor already reads its
+own highlight feed. This module is that listener for midiviz.
+
+Two halves, both importable without Qt or ALSA:
+
+* `decode_osc` — the smallest OSC decoder the packets need. Pure function,
+  unit-tested against hand-built bytes.
+* `HLFeed` — a UDP daemon thread with the same shape as midiviz's `Reader`
+  (bounded deque, drain, close): a live monitor wants the PRESENT, not a
+  backlog, so the same lesson as `midistream.DEPTH` applies.
+"""
+from __future__ import annotations
+
+import socket
+import struct
+import threading
+import time
+from collections import deque
+
+# BootTidal.hs sends the mirror here. High enough to dodge everything
+# scsynth/sclang grab by default, referenced by nothing else in the repo.
+VIZ_PORT = 57130
+
+# Tidal's orbits are 0-based (`orbit: 0` is d1) and the rig runs 14 of them.
+MIN_ORBIT, MAX_ORBIT = 1, 14
+
+
+def _skip_pad(data: bytes, i: int) -> int:
+    """OSC blobs are 4-aligned; return the index past the padding."""
+    return i + ((4 - (i % 4)) % 4)
+
+
+def _read_string(data: bytes, i: int) -> tuple[str, int]:
+    end = data.index(b"\0", i)
+    return data[i:end].decode("utf-8", "replace"), _skip_pad(data, end + 1)
+
+
+def _read_blob(data: bytes, i: int) -> tuple[bytes, int]:
+    (n,) = struct.unpack_from(">i", data, i)
+    i += 4
+    return data[i:i + n], i + n + ((4 - ((i + n) % 4)) % 4)
+
+
+def decode_osc(data: bytes) -> list[dict]:
+    """One UDP datagram → the OSC messages inside it.
+
+    Tidal's Stream sends either bare messages or `#bundle`s (one timetag,
+    then length-prefixed elements). Returns [{"addr", "args"}] with `args`
+    the *named* argument pairs a /play carries: Tidal→SuperDirt arguments
+    are ("gain", 0.8) style name/value runs, so a flat dict is the shape
+    every consumer wants. Unnamed args land under "0", "1", …
+    """
+    out: list[dict] = []
+    stack = [data]
+    while stack:
+        cur = stack.pop(0)
+        if not cur:
+            continue
+        if cur.startswith(b"#bundle"):
+            i = 16                                    # "#bundle\0" + 8-byte timetag
+            while i + 4 <= len(cur):
+                (n,) = struct.unpack_from(">i", cur, i)
+                i += 4
+                if n:
+                    stack.append(cur[i:i + n])
+                i = _skip_pad(cur, i + n)
+            continue
+        try:
+            addr, i = _read_string(cur, 0)
+            tags, i = _read_string(cur, i)            # ",ifs…" — the comma is part of it
+            args: dict = {}
+            slot = 0
+            for t in tags[1:]:
+                if t == "i":
+                    (v,) = struct.unpack_from(">i", cur, i)
+                    i += 4
+                elif t == "f":
+                    (v,) = struct.unpack_from(">f", cur, i)
+                    i += 4
+                elif t == "d":
+                    (v,) = struct.unpack_from(">d", cur, i)
+                    i += 8
+                elif t == "s":
+                    v, i = _read_string(cur, i)
+                elif t == "b":
+                    v, i = _read_blob(cur, i)
+                else:
+                    v = None                          # T/F/I/N… carry no bytes
+                args[str(slot)] = v
+                slot += 1
+        except (ValueError, struct.error, IndexError):
+            continue                                  # one bad element, not the datagram
+        # A /play message's named pairs are positional: ("gain", v), ("orbit", v)…
+        # Re-fold every ("string", value) run into a dict without losing the
+        # originals — the names ARE the Tidal parameter names.
+        named: dict = {}
+        k = 0
+        while k < len(args):
+            key = args.get(str(k))
+            if isinstance(key, str) and str(k + 1) in args:
+                named[key] = args[str(k + 1)]
+                k += 2
+            else:
+                k += 1
+        out.append({"addr": addr, "args": named})
+    return out
+
+
+def play_orbit(msg: dict) -> int | None:
+    """A decoded message → its 1-based orbit number, or None if not a trigger."""
+    if msg["addr"] != "/play":
+        return None
+    o = msg["args"].get("orbit")
+    if isinstance(o, float) and o == int(o):
+        o = int(o)
+    if not isinstance(o, int) or not (0 <= o <= MAX_ORBIT):
+        return None
+    return o + 1                                       # Tidal 0-based → d1-based
+
+
+class HLFeed:
+    """UDP listener on `VIZ_PORT`; a bounded deque the GUI drains each frame.
+
+    Same contract as midiviz.Reader — start/drain/close/alive/error — so the
+    tick can treat the two feeds identically. A missed datagram is a missed
+    frame of truth, never a queue: bounded like the Reader, for the same
+    recorded reason.
+    """
+
+    DEPTH = 512
+
+    def __init__(self, port: int = VIZ_PORT):
+        self.port = port
+        self.error: str | None = None
+        self.total = 0
+        self._q: deque[dict] = deque(maxlen=self.DEPTH)
+        self._lock = threading.Lock()
+        self._sock: socket.socket | None = None
+        self._stop = threading.Event()
+
+    def start(self) -> "HLFeed":
+        threading.Thread(target=self._run, daemon=True,
+                         name="oscfeed").start()
+        return self
+
+    def _run(self) -> None:
+        try:
+            self._sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+            self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            self._sock.bind(("127.0.0.1", self.port))
+            self._sock.settimeout(0.5)
+        except OSError as exc:
+            self.error = type(exc).__name__
+            return
+        while not self._stop.is_set():
+            try:
+                data, _ = self._sock.recvfrom(65535)
+            except socket.timeout:
+                continue
+            except OSError:
+                if not self._stop.is_set():
+                    self.error = "recv"
+                break
+            now = time.monotonic()
+            # One malformed datagram must not kill the feed the way an
+            # uncaught exception in a daemon thread dies silently: skip it.
+            try:
+                msgs = decode_osc(data)
+            except (ValueError, struct.error, IndexError):
+                continue
+            for msg in msgs:
+                o = play_orbit(msg)
+                if o is None:
+                    continue
+                with self._lock:
+                    self._q.append({"orbit": o, "t": now,
+                                    "params": msg["args"]})
+                    self.total += 1
+
+    def alive(self) -> bool:
+        return self._sock is not None and self.error is None
+
+    def drain(self) -> list[dict]:
+        with self._lock:
+            if not self._q:
+                return []
+            out = list(self._q)
+            self._q.clear()
+        return out
+
+    def close(self) -> None:
+        self._stop.set()
+        if self._sock is not None:
+            try:
+                self._sock.close()
+            except OSError:
+                pass
+            self._sock = None

tools/bridge/tests/test_oscfeed.py

+"""Tests for the Tidal HL feed: OSC decode + which cells an orbit relates to.
+
+No Qt, no ALSA, no network: `decode_osc` is a pure function tested against
+hand-built bytes — the exact packets Tidal's Stream sends (a `#bundle`
+containing a `/play` message whose arguments are name/value runs) and a couple
+of shapes it must survive (bare message, garbage, empty datagram).
+
+The cell relation is tested against `lcxl_grid` itself, so the test dies the
+day the grid changes shape instead of lying about it.
+"""
+import struct
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+import oscfeed                   # noqa: E402
+import sys as _sys               # noqa: E402
+_tools = str(Path(__file__).resolve().parent.parent.parent)
+if _tools not in _sys.path:
+    _sys.path.insert(0, _tools)
+import lcxl_grid as grid         # noqa: E402  lives in tools/, like midiviz's own import
+
+
+def _osc_msg(addr: str, pairs: list[tuple[str, int]]) -> bytes:
+    """Address + typetags + (string, int) pair args, padded the way OSC does.
+    A pair is: name padded to 4 (so 'orbit' → 8 bytes) + a 4-byte int."""
+    msg = addr.encode() + b"\0"
+    msg += b"\0" * ((4 - len(msg) % 4) % 4)
+    tags = "," + "si" * len(pairs)           # (string, int) runs — /play's shape
+    t = tags.encode() + b"\0"
+    msg += t + b"\0" * ((4 - len(t) % 4) % 4)
+    for name, value in pairs:
+        n = name.encode() + b"\0"
+        n += b"\0" * ((4 - len(n) % 4) % 4)
+        msg += n + struct.pack(">i", value)
+    return msg
+
+
+def _bundle(*elements: bytes) -> bytes:
+    out = b"#bundle\0" + b"\0" * 8           # timetag zero
+    for el in elements:
+        out += struct.pack(">i", len(el)) + el
+        out += b"\0" * ((4 - len(out) % 4) % 4)
+    return out
+
+
+def _named_pair(name: str, value: int) -> bytes:
+    n = name.encode() + b"\0"
+    n += b"\0" * ((4 - len(n) % 4) % 4)
+    return n + struct.pack(">i", value)
+
+
+def test_decode_play_bundle_finds_the_orbit():
+    # d4 → Tidal says orbit: 3; midiviz speaks d-numbers.
+    data = _bundle(_osc_msg("/play", [("orbit", 3)]))
+    msgs = oscfeed.decode_osc(data)
+    assert len(msgs) == 1
+    assert msgs[0]["addr"] == "/play"
+    assert oscfeed.play_orbit(msgs[0]) == 4
+
+
+def test_decode_bare_message_and_named_args():
+    data = _osc_msg("/play", [("orbit", 0), ("gain", 1)])
+    msgs = oscfeed.decode_osc(data)
+    assert oscfeed.play_orbit(msgs[0]) == 1
+    assert msgs[0]["args"]["gain"] == 1
+
+
+def test_non_play_and_garbage_yield_nothing():
+    assert oscfeed.play_orbit({"addr": "/hush", "args": {}}) is None
+    assert oscfeed.decode_osc(b"") == []
+    assert oscfeed.decode_osc(b"\x01\x02\x03") == []    # must not raise
+
+
+def test_orbit_bounds():
+    # orbit 14 (0-based 13) is the rig's last; anything past is not ours.
+    data = _bundle(_osc_msg("/play", [("orbit", 13)]))
+    assert oscfeed.play_orbit(oscfeed.decode_osc(data)[0]) == 14
+    data = _bundle(_osc_msg("/play", [("orbit", 99)]))
+    assert oscfeed.play_orbit(oscfeed.decode_osc(data)[0]) is None
+
+
+# ── the relation: which cells does a trigger light ─────────────────────────
+def _hl_cells(orbit):
+    """The widget's relation, re-derived the same way — tested against the
+    grid's own tables rather than against a copy of the widget's cache."""
+    ccs = set(grid.orbit_home(orbit).values())
+    for role, fam_of in (("family_filter", grid.filter_family),
+                         ("family_mute", grid.mute_family)):
+        n = fam_of(orbit)
+        ccs |= {cc for cc, (r, who) in grid.CC_ROLE.items()
+                if r == role and who == n}
+    return ccs
+
+
+def test_d1_trigger_lights_its_column_and_both_families():
+    # d1 (the kick): its own fx/level/gate cells, plus C1 (all-percs filter)
+    # and F1 (kick-alone mute) — the two family controls that act on it.
+    cells = {_: grid.CC_TO_CELL.get(_) for _ in _hl_cells(1)}
+    assert cells[grid.CELL_TO_CC[("C", 1)]] == ("C", 1)   # gF1, the perc filter
+    assert cells[grid.CELL_TO_CC[("F", 1)]] == ("F", 1)   # gM1, the kick mute
+    for row, col in (("B", 1), ("D", 1), ("E", 1)):       # its own column
+        assert (row, col) in cells.values()
+
+
+def test_every_d_orbit_maps_to_at_least_one_control():
+    # An orbit with NO related cells would make the HL feed a no-op that
+    # still LOOKS wired — the worst failure shape. d1-d12 own a full column
+    # plus two family controls; d13/d14 (the 14-orbit boot's tail) own only
+    # their family bloc, which is still one honest thing to show.
+    for o in range(1, 15):
+        assert len(_hl_cells(o)) >= 1, f"d{o} has no related controls"
+    for o in range(1, 13):
+        assert len(_hl_cells(o)) >= 3, f"d{o} has too few related controls"
+
+
+def test_widget_relation_matches_grid_derivation():
+    """The widget caches `_hl_cells` — the cache must never drift from a fresh
+    derivation, which is the same two-copies-one-truth lesson the port
+    preference already taught."""
+    sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+    import midiviz as V
+    # The method hangs off the widget class; derive it without a Qt app.
+    cls = None
+    for obj in vars(V).values():
+        if isinstance(obj, type) and hasattr(obj, "_hl_cells"):
+            cls = obj
+            break
+    if cls is None:
+        return
+    holder = object.__new__(cls)
+    holder._hl_cells_cache = {}
+    for o in (1, 4, 7, 11):
+        assert set(holder._hl_cells(o)) == _hl_cells(o)