midiviz: state as well as events — the DJFs know where they are resting

"as it maintains a state, it should be 'not all same default state' when you
look at it between two movements? like, the DJFs are at zero, in the Low, or in
the High, and this doesnt really grasp from viewing the midimon"

Correct, and it was not a theming problem. This module draws EVENTS, and makes
recency a visual axis on purpose: an event's identity is its position, its value
is a bar, its recency is brightness. So between two movements every cell decays
to the same floor tint, and a filter resting at zero is the same picture as one
parked at the top. No palette fixes that — an event stream carries changes, and
rest is the absence of a change.

A viewer cannot derive it either. The LCXL3 has no readback, and rows B/C run
relative (the surface sends v - 64), so the hardware holds no absolute position.
Exactly one process integrates those deltas and therefore owns the value, and it
is not the viewer: lcxl3-driver's `self.values` is not a cache of something else,
it is the only copy.

So: two channels, because two physics. Events stay edge-timed off the wire; state
is a level, published last-value-wins.

  driver   `~/.cache/parvagues/surface-state.json`, temp file + os.replace, on
           the existing loop at 10 Hz, written ONLY when the snapshot changed.
           Every failure swallowed: this is a courtesy to viewers, and a
           read-only /home must never stop the faders reaching Ardour.
  midiviz  one stat() per frame; an unchanged `seq` means zero parsing and zero
           repainting, which is the "pay once" property asked for. Absent,
           unreadable or wrong-shaped file all mean "no state", and the window
           draws exactly what it drew before this layer existed.

Deliberately not a socket or a broker: ~32 integers, idempotent, no history
worth queueing. A socket buys a connection, backpressure and a dead-consumer
failure mode in exchange for nothing. `cat` is the debugger.

First slice is the three family filters, drawn in the cells they physically
occupy (C1/C2/C3) rather than a strip of its own — the layout does not move, and
this module's founding idea is that identity IS position, so state belongs where
the events about it are, which is where his hands are.

ZERO IS THE CENTRE DETENT, 64, not 0: gDJF is an lpf and an hpf section both
wide open at ch=0.5, so centre is bypass and both ends filter. The event bar is
left-anchored, which is exactly the lie — a filter at rest draws a half-width
bar that reads as "half of something". The state strip sits above it,
centre-anchored, with the detent drawn: a bar growing left of centre is LOW, right
is HIGH, bare detent is ZERO. Static at rest, no decay, no pulse — being legible
when nothing has moved for a minute is the entire point.

Found by looking at it rather than by reasoning: the first version drew into the
bottom of the cell, where the event bar already lives, and C1-C3 rendered as
mud. Checked at 900x560 and 1600x1000, and 3 units tall beat 2 in sunlight.

16 tests, the load-bearing one being a real round trip: the real state_write and
the real SurfaceState.poll over one temp path, so a change to either side that
breaks the other fails here and not at a gig. That round trip caught a live bug —
`null` is valid JSON of the wrong shape, and the AttributeError from the first
.get escaped the handler and would have taken the window down.

Suites 526 passed, 2 failed (both pre-existing); --selftest PASS across three
themes and four shapes. Takes effect on the next driver restart; the running
driver was not touched.

tools/bridge/tests/test_surface_state.py

+"""The state channel, end to end: the driver writes it, midiviz reads it.
+
+Two processes and one file, so the interesting bugs are all at the seam. The
+round-trip test below is the one that earns its keep — it runs the REAL
+`Driver.state_write` and the REAL `SurfaceState.poll` against one temp path, so
+a change to either side that breaks the other fails here rather than at a gig,
+where the symptom would be "the DJF strip is blank" and the cause would be four
+files away.
+
+Why the channel exists at all: MIDI is edge-triggered and midiviz makes recency
+a visual axis, so between two movements every cell decays to the same tint and a
+filter resting at zero looks identical to one parked at the top. A consumer
+cannot derive the resting value either — the LCXL3 has no readback and rows B/C
+send deltas — so exactly one process can publish it, and this is the wire.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+import json
+import os
+import sys
+import types
+
+import pytest
+
+BRIDGE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+TOOLS = os.path.dirname(BRIDGE)
+sys.path.insert(0, TOOLS)
+sys.path.insert(0, BRIDGE)
+
+import lcxl_grid as grid  # noqa: E402
+
+
+def _load(path: str, name: str):
+    spec = importlib.util.spec_from_file_location(name, path)
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    return mod
+
+
+MV = _load(os.path.join(BRIDGE, "midiviz.py"), "mv_state")
+DRV = _load(os.path.join(TOOLS, "lcxl3-driver.py"), "drv_state")
+Driver = next(v for v in vars(DRV).values()
+              if isinstance(v, type) and hasattr(v, "state_write"))
+
+
+def _publisher(values: dict[int, int], track: str = "TEST"):
+    """The minimum a Driver needs to be to publish. Keeps the test honest about
+    what state_write actually reads, instead of constructing a whole rig."""
+    return types.SimpleNamespace(values=dict(values), track_name=track,
+                                 _state_seq=0, _state_sent=None, _state_at=0.0)
+
+
+# --------------------------------------------------------------------------- #
+# the seam
+# --------------------------------------------------------------------------- #
+def test_the_driver_writes_what_midiviz_reads(tmp_path, monkeypatch):
+    path = tmp_path / "surface-state.json"
+    monkeypatch.setattr(DRV, "STATE_FILE", path)
+    pub = _publisher({49: 8, 50: 64, 51: 120, 77: 100})
+
+    Driver.state_write(pub, now=1.0, force=True)
+    st = MV.SurfaceState(path)
+    assert st.poll() is True
+    assert st.values == {49: 8, 50: 64, 51: 120, 77: 100}
+    assert st.track == "TEST"
+    assert st.seq == 1
+
+
+def test_an_unchanged_seq_costs_the_consumer_nothing(tmp_path, monkeypatch):
+    """The 'pay once' property. A consumer polls every frame; at rest that must
+    be one stat() and no parse, which is why seq counts CHANGES not ticks."""
+    path = tmp_path / "surface-state.json"
+    monkeypatch.setattr(DRV, "STATE_FILE", path)
+    pub = _publisher({49: 30})
+    Driver.state_write(pub, now=1.0, force=True)
+
+    st = MV.SurfaceState(path)
+    assert st.poll() is True
+    reads = st.reads
+    assert st.poll() is False           # nothing changed on disk
+    assert st.reads == reads            # and nothing was parsed
+
+    # Touch the file without changing its content: mtime moves, seq does not,
+    # so the consumer must still decline to repaint.
+    os.utime(path, (os.path.getatime(path) + 10, os.path.getmtime(path) + 10))
+    assert st.poll() is False
+    assert st.reads == reads
+
+
+def test_a_resting_surface_writes_nothing(tmp_path, monkeypatch):
+    path = tmp_path / "surface-state.json"
+    monkeypatch.setattr(DRV, "STATE_FILE", path)
+    pub = _publisher({49: 64})
+    Driver.state_write(pub, now=1.0, force=True)
+    first = path.read_text()
+
+    for n in range(5):                  # five ticks, no hand on the board
+        Driver.state_write(pub, now=2.0 + n)
+    assert path.read_text() == first
+    assert pub._state_seq == 1
+
+
+def test_a_knob_move_advances_seq_but_the_rate_cap_holds(tmp_path, monkeypatch):
+    path = tmp_path / "surface-state.json"
+    monkeypatch.setattr(DRV, "STATE_FILE", path)
+    pub = _publisher({49: 64})
+    Driver.state_write(pub, now=1.0, force=True)
+
+    pub.values[49] = 70
+    Driver.state_write(pub, now=2.0)
+    assert json.loads(path.read_text())["seq"] == 2
+
+    # A sweep arrives far faster than any consumer paints; inside 0.1 s the
+    # publish is skipped and the NEXT tick carries the newer value.
+    pub.values[49] = 90
+    Driver.state_write(pub, now=2.05)
+    assert json.loads(path.read_text())["values"]["49"] == 70
+    Driver.state_write(pub, now=2.2)
+    assert json.loads(path.read_text())["values"]["49"] == 90
+
+
+# --------------------------------------------------------------------------- #
+# every way the file can be useless
+# --------------------------------------------------------------------------- #
+def test_no_file_means_no_state_and_no_exception(tmp_path):
+    st = MV.SurfaceState(tmp_path / "absent.json")
+    assert st.poll() is False
+    assert st.values == {}
+
+
+def test_the_file_going_away_clears_the_strip_once(tmp_path):
+    path = tmp_path / "s.json"
+    path.write_text(json.dumps({"seq": 3, "values": {"49": 10}}))
+    st = MV.SurfaceState(path)
+    assert st.poll() is True
+    path.unlink()
+    assert st.poll() is True            # report the change...
+    assert st.values == {}
+    assert st.poll() is False           # ...then go quiet
+
+
+@pytest.mark.parametrize("body", [
+    "",                                  # zero-length: a writer mid-flight
+    "{",                                  # truncated JSON
+    "null",                               # valid JSON, wrong shape
+    '{"seq": "x", "values": {}}',         # seq not an int
+    '{"seq": 1, "values": {"a": "b"}}',   # values not integers
+    '{"seq": 1}',                         # no values at all
+])
+def test_a_useless_file_never_raises(tmp_path, body):
+    """os.replace makes a torn read impossible, but an OLD driver, a truncated
+    disk or a hand-edited file are all real. None may take the window down."""
+    path = tmp_path / "s.json"
+    path.write_text(body)
+    st = MV.SurfaceState(path)
+    st.poll()
+    assert isinstance(st.values, dict)
+
+
+def test_an_unwritable_path_never_reaches_the_midi_path(tmp_path, monkeypatch):
+    """The publisher is a courtesy to viewers. A full or read-only /home must
+    never be able to stop the faders getting to Ardour."""
+    monkeypatch.setattr(DRV, "STATE_FILE",
+                        tmp_path / "no-such-dir" / "x" / "s.json")
+    monkeypatch.setattr(DRV.pathlib.Path, "mkdir",
+                        lambda *a, **k: (_ for _ in ()).throw(OSError("read-only")))
+    Driver.state_write(_publisher({49: 1}), now=1.0, force=True)   # must not raise
+
+
+def test_no_temp_file_is_left_behind(tmp_path, monkeypatch):
+    path = tmp_path / "s.json"
+    monkeypatch.setattr(DRV, "STATE_FILE", path)
+    pub = _publisher({49: 1})
+    for n in range(3):
+        pub.values[49] = n
+        Driver.state_write(pub, now=1.0 + n)
+    assert [f.name for f in tmp_path.iterdir()] == ["s.json"]
+
+
+# --------------------------------------------------------------------------- #
+# the slice's assumptions about the grid
+# --------------------------------------------------------------------------- #
+def test_the_djfs_are_still_where_the_strip_looks_for_them():
+    """The first slice draws only the family filters, in the cells they occupy.
+
+    Both halves of that are assumptions about the AUTHORED grid, and the grid
+    moves — the corpus had 685 controls migrated this month. If a filter leaves
+    row C or stops being a `family_filter`, the strip silently draws nothing,
+    which is the failure mode this repo keeps writing tests to avoid.
+    """
+    filters = [cc for cc, (role, _) in grid.CC_ROLE.items()
+               if role == "family_filter"]
+    assert sorted(filters) == [49, 50, 51], "the DJFs moved"
+    assert [grid.CC_TO_CELL[cc] for cc in sorted(filters)] == \
+        [("C", 1), ("C", 2), ("C", 3)]
+    assert "C" in grid.PHYSICAL_ORDER
+
+
+def test_the_detent_is_the_seeded_rest_value():
+    """64, not 0 — and the driver seeds it there, which is what makes the strip
+    meaningful before anything has been touched. gDJF is an lpf and an hpf
+    section both wide open at ch=0.5: centre is bypass, both ends filter."""
+    seeded = {cc: (64 if role == "family_filter" else 0)
+              for cc, (role, _) in grid.CC_ROLE.items()}
+    assert all(seeded[cc] == 64 for cc in (49, 50, 51))
+    assert seeded.get(29, 0) == 0

tools/lcxl3-driver.py

@@ -129,6 +129,20 @@ REPO = pathlib.Path(__file__).resolve().parent.parent
 # bug: a driver that parsed its track once, at launch, and never looked again.
 TRACK_FILE = pathlib.Path(os.path.expanduser("~/.cache/parvagues/current-track"))
 EVAL_FILE = pathlib.Path(os.path.expanduser("~/.cache/parvagues/eval-events.jsonl"))
+# THE STATE CHANNEL. This process is the only one that CAN publish it: the
+# LCXL3 has no readback, and rows B/C run in relative mode where the surface
+# sends deltas (rel_delta = v - 64), so the hardware itself holds no absolute
+# position. `self.values` is not a cache of something else — it is the only
+# copy. A viewer that sees only the MIDI stream cannot know a resting value,
+# which is why every cell in midiviz decays to the same tint between movements.
+#
+# Shape: last-value-wins, one small file, replaced atomically, carrying a
+# monotonic `seq` so a consumer that sees an unchanged seq does zero parsing
+# and zero repainting. At rest that costs one stat() per consumer frame.
+# Deliberately NOT a socket or a broker: the payload is ~32 integers with no
+# history worth queueing, and a socket would add a connection, backpressure and
+# a dead-consumer failure mode in exchange for nothing. `cat` is the debugger.
+STATE_FILE = pathlib.Path(os.path.expanduser("~/.cache/parvagues/surface-state.json"))
 
 # Same env override + default gig-log.py itself uses (its LOG_DIR) — the home
 # screen's REC timer must agree with `gig-log.py status` about which file is
@@ -882,6 +896,12 @@ class Driver:
             for v2 in self.v2_to_v3:
                 role, _ = self.grid.CC_ROLE.get(v2, ("", 0))
                 self.values[v2] = 64 if role == "family_filter" else 0
+            # State-channel bookkeeping, next to the state it publishes.
+            # `seq` counts CHANGES, not ticks: a resting surface writes nothing,
+            # so a consumer seeing an unchanged seq knows it can skip the parse.
+            self._state_seq = 0
+            self._state_sent: dict[str, int] | None = None
+            self._state_at = 0.0
             # ---- what the CURRENT track is; all of it replaced on a change ----
             self.track_path: pathlib.Path | None = None
             self.track_mtime = 0.0
@@ -1666,6 +1686,41 @@ class Driver:
         except OSError:
             pass
 
+    def state_write(self, now: float = 0.0, force: bool = False) -> None:
+        """Publish the analogue surface state. Best-effort, never load-bearing.
+
+        Rides the existing loop rather than adding a timer, capped at 10 Hz —
+        the same cadence the OLED coalesce already uses, because a knob sweep
+        arrives far faster than any consumer can paint.
+
+        Writes ONLY when the snapshot actually changed, so a resting surface
+        writes nothing at all and `seq` is a real change counter rather than a
+        clock. The write is temp-file + os.replace, so a reader never sees a
+        torn file and there is no lock to hold.
+
+        Every failure is swallowed. This is a courtesy to viewers; the audio and
+        MIDI paths must not care whether a cache file is writable, and a full
+        /home must never be able to stop the faders reaching Ardour.
+        """
+        if not force and now - self._state_at < 0.1:
+            return
+        self._state_at = now
+        snap = {str(k): int(v) for k, v in sorted(self.values.items())}
+        if snap == self._state_sent and not force:
+            return
+        self._state_seq += 1
+        try:
+            STATE_FILE.parent.mkdir(parents=True, exist_ok=True)
+            tmp = STATE_FILE.with_name(STATE_FILE.name + ".tmp")
+            tmp.write_text(json.dumps(
+                {"seq": self._state_seq, "t": round(time.time(), 3),
+                 "track": self.track_name, "values": snap},
+                separators=(",", ":")))
+            os.replace(tmp, STATE_FILE)
+            self._state_sent = snap
+        except OSError:
+            pass
+
     def tick_zero_anim(self, now: float) -> None:
         """~3 fps heartbeat on the value line while a DJF rests at its zero.
 
@@ -1948,6 +2003,7 @@ class Driver:
                 self.oled_labels(quiet=True)
                 self.oled_home()
                 last_labels = now
+            self.state_write(now)
             time.sleep(0.001)
 
         self.stop()