fix(install): only a running desktop stops the install; detect desktops by metapackage (#781)

* fix(install): detect desktops by their metapackages, not name prefixes

The Lite check matched any installed package starting with gnome/kde/
xfce/lxde, so standalone parts (gnome-keyring, xfce4-terminal,
lxde-icon-theme) rejected a Lite system. Match whole names of desktop
metapackages and session managers instead.

Also catch desktops the prefixes missed: Raspberry Pi OS Trixie replaced
raspberrypi-ui-mods with rpd-wayland-core / rpd-x-core, Debian tasksel
desktops (task-*-desktop), and multi-arch names (plasma-workspace:arm64).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

* docs(common): list espn_payload in the common README

#749 added src/common/espn_payload.py without a summary row or section,
so test_common_readme_lists_every_module fails on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ
(cherry picked from commit 8333f23e5c)

* fix(install): only a running desktop stops the install

A desktop costs the panel CPU only while it runs, so a running display
manager (checked with systemctl is-active, including the generic
display-manager alias, instead of a grep -q pipe under pipefail) still
stops the installer. Desktop packages or session files on a Pi that boots
to the console now print a warning and the install continues.

Adds installer OS-check tests for running, installed-only and Lite
systems, including the libblockdev and gnome-keyring false positives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZNWFWprcFYGfuf1JAyrBJ

---------

Co-authored-by: Claude <noreply@anthropic.com>

.gitattributes

@@ -10,3 +10,6 @@
 # Generated by scripts/build_css.py; collapsed in diffs, not hand-edited.
 web_interface/static/v3/tailwind.css linguist-generated=true
 web_interface/static/v3/plugin-frame.css linguist-generated=true
+
+# Installed as an executable (its shebang runs it) by install_service.sh.
+scripts/install/ledmatrix_refresh_units.py text eol=lf

.github/workflows/release-version-check.yml

@@ -31,7 +31,7 @@ jobs:
 
       - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
         with:
-          python-version: "3.12"
+          python-version: "3.13"
 
       # No dependencies: the script reads src/__init__.py and CHANGELOG.md only.
       - name: Assert the tag, CHANGELOG and src.__version__ agree

.github/workflows/test.yml

@@ -14,8 +14,14 @@ permissions:
 
 jobs:
   plugin-safety:
-    name: Plugin safety harness + unit tests
+    name: Plugin safety harness + unit tests (Python ${{ matrix.python-version }})
     runs-on: ubuntu-latest
+    # The two Pythons the installer supports: Raspberry Pi OS Bookworm ships
+    # 3.11 and Trixie 3.13.
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.13"]
     env:
       # The bundled fixture plugin gives the harness at least one real plugin
       # to render, and REQUIRE_PLUGINS turns "discovered zero plugins" into a
@@ -29,7 +35,7 @@ jobs:
 
       - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
         with:
-          python-version: "3.12"
+          python-version: ${{ matrix.python-version }}
           cache: pip
 
       - name: Install dependencies
@@ -43,8 +49,13 @@ jobs:
           pytest --no-cov test/plugins/
 
   unit-tests:
-    name: Core unit tests
+    name: Core unit tests (Python ${{ matrix.python-version }})
     runs-on: ubuntu-latest
+    # Bookworm's Python (3.11) and Trixie's (3.13); see plugin-safety.
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.13"]
     steps:
       - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
@@ -52,7 +63,7 @@ jobs:
 
       - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
         with:
-          python-version: "3.12"
+          python-version: ${{ matrix.python-version }}
           cache: pip
 
       - name: Install dependencies
@@ -84,7 +95,7 @@ jobs:
 
       - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
         with:
-          python-version: "3.12"
+          python-version: "3.13"
           cache: pip
 
       - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
@@ -123,7 +134,7 @@ jobs:
 
       - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
         with:
-          python-version: "3.12"
+          python-version: "3.13"
 
       # Downloads the pinned standalone Tailwind CLI (SHA-256 checked; no
       # Node), rebuilds static/v3/tailwind.css and plugin-frame.css from the
@@ -142,7 +153,7 @@ jobs:
 
       - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
         with:
-          python-version: "3.12"
+          python-version: "3.13"
           cache: pip
 
       # The runtime requirements are installed so mypy sees the real types of
@@ -181,7 +192,7 @@ jobs:
 
       - uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
         with:
-          python-version: "3.12"
+          python-version: "3.13"
 
       # Stdlib only; exits 0 whatever it finds.
       - name: Report method-family drift across the nine scoreboards

README.md

@@ -151,6 +151,11 @@ The system supports live, recent, and upcoming game information for multiple spo
   - **1GB models (Pi 3B / 3B+), the 512MB Pi Zero 2 W and other low-memory boards**: supported, but the `rpi-rgb-led-matrix` C++ build needs more memory than the Pi has. The installer detects this automatically, compiles with fewer parallel jobs, and adds a temporary swapfile for the build which it removes afterwards. Expect that step to take 15-25 minutes instead of 2-5, and leave at least **3GB free** on the SD card. If you manage swap yourself, opt out with `--skip-swap`. To pin the compiler down further, use `--build-jobs 1`. Once running, keep an eye on memory: see [docs/LOW_MEMORY_BOARDS.md](docs/LOW_MEMORY_BOARDS.md).
 
 
+### Operating system
+- **Raspberry Pi OS Lite, Trixie (Debian 13) or Bookworm (Debian 12)**, 64-bit recommended. Trixie is the current release and the one to pick for a new SD card; an existing Bookworm install works as it is, no upgrade needed. The installer checks this first and stops with directions on anything else (Bullseye and older, the desktop edition, other distributions).
+- **Python**: whatever the OS ships, 3.13 on Trixie and 3.11 on Bookworm. Don't install a different Python; the installer and the services use the system `python3`.
+- **Networking**: NetworkManager, the default on both. Choosing a WiFi network from the web page and the `LEDMatrix-Setup` hotspot need it; if you switched to dhcpcd in `raspi-config`, switch back (Advanced Options → Network Config → NetworkManager).
+
 ### RGB Matrix Bonnet / HAT
 - [Adafruit RGB Matrix Bonnet/HAT](https://www.adafruit.com/product/3211) – supports one “chain” of horizontally connected displays  
 - [Adafruit Triple LED Matrix Bonnet](https://www.adafruit.com/product/6358) – supports up to 3 vertical “chains” of horizontally connected displays *(use `regular` as hardware mapping)*  
@@ -249,7 +254,7 @@ These are not required and you can probably rig up something basic with stuff yo
 
 <img width="512" height="361" alt="Step 2 Other " src="https://github.com/user-attachments/assets/166a22e8-8067-48df-9f80-50c91f573356" />
 
-5. Then choose Raspbian OS (64-bit) Lite (Trixie)
+5. Then choose Raspbian OS (64-bit) Lite (Trixie). Bookworm Lite (listed as Legacy) also works; see [Operating system](#operating-system) below
 
 <img width="512" height="361" alt="Step 4 Trixie Lite 64" src="https://github.com/user-attachments/assets/3b8590ce-b810-4dfe-9253-26e0d4f8ed1e" />
 

docs/ADVANCED_FEATURES.md

@@ -649,7 +649,9 @@ When nothing is running on demand, `data.state` is
 > on-demand machinery is internal — drive it through the REST endpoints
 > above (or the web UI buttons). The API handlers
 > (`start_on_demand_display()` / `stop_on_demand_display()` in
-> `web_interface/blueprints/api_v3/display.py`) write a request into the cache
+> `web_interface/blueprints/api_v3/display.py`) send the request over the
+> display's control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)).
+> Only when the socket cannot carry it do they write it into the cache
 > manager under the `display_on_demand_request` key, which
 > `DisplayController._poll_on_demand_requests()`
 > (`src/display_controller.py`) picks up. A separate
@@ -747,8 +749,14 @@ keys helps troubleshoot stuck states.
   "timestamp": 1234567890.123
 }
 ```
-**Purpose:** Communication from web interface to display controller
-**When Set:** API endpoint receives request
+**Purpose:** Communication from web interface to display controller, as the
+fallback when the control socket cannot carry the request (deprecated; it
+will be removed in a later release)
+**When Set:** API endpoint receives a request and the display's control
+socket is unavailable (display stopped, or older than the socket or the
+command); some plugins also write it directly
+**Read:** once a second while the display serves the control socket (0.25 s
+without it), and only when the file changed since the last look
 **Auto-Cleared:** After processing or 1 hour TTL
 
 **2. display_on_demand_config** (No TTL)

docs/ARCHITECTURE.md

@@ -42,11 +42,11 @@ each other. They share three things:
 | State | Where | Written by | Read by |
 |---|---|---|---|
 | On-demand command | control socket `/run/ledmatrix/control.sock` ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)) | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py), via [`src/ipc/client.py`](../src/ipc/client.py) | display: [`src/ipc/server.py`](../src/ipc/server.py) acks; the render thread applies it in `_poll_on_demand_requests()` |
-| On-demand request (fallback) | cache `display_on_demand_request` | web, when the socket fails; four plugins write it directly | display: `_poll_on_demand_requests()` |
+| On-demand request (fallback) | cache `display_on_demand_request` | web, only when the socket could not carry the request (`should_fall_back`); four plugins write it directly | display: `_poll_on_demand_requests()`, a `stat()` every 1 s while the socket is up (0.25 s without), read only when the file changed |
 | On-demand state | cache `display_on_demand_state` | display: `_publish_on_demand_state()` | web: `/api/v3/display/on-demand/status` |
 | Current screen | cache `display_current_state` | display | web: `/api/v3/display/current-status` |
 | Plugin errors | cache `plugin_error_snapshot` | display: `ErrorSnapshotPublisher` ([`src/error_aggregator.py`](../src/error_aggregator.py)) | web: `read_error_report()` for `/api/v3/errors/*` |
-| Error clear | cache `plugin_error_clear_request` | web | display |
+| Error clear | control socket `errors.clear`; cache `plugin_error_clear_request` as the fallback | web: `POST /api/v3/errors/clear` | display: applied before the socket answers; the mailbox on the error publisher's 5 s tick, read only when the file changed |
 | Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
 | Fetch statistics (requests per plugin and host) | cache `fetch_stats_snapshot` | display: `FetchStatsPublisher` ([`src/common/fetch_service.py`](../src/common/fetch_service.py)), at most once a minute on change | web: `read_fetch_stats()` for `/api/v3/plugins/fetch-stats` |
 | Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
@@ -58,11 +58,16 @@ each other. They share three things:
 
 The on-demand start route starts `ledmatrix.service` when it is not running
 (`start_service`, on by default) but never restarts a running one. The routes
-send the command over the display's control socket and get an ack; when that
-fails (a stopped display, one older than the socket) they write the mailbox
-instead, which the display reads every `ON_DEMAND_POLL_INTERVAL` (0.25s), from
-its dwell sleep, its render loops and Vegas's interrupt check as well as the
-main loop. Both ways end in the same handler, `_handle_on_demand_request()`.
+send the command over the display's control socket and get an ack; only when
+the socket could not carry it (a stopped display, one older than the socket
+or the command) do they write the mailbox instead. A display that had the
+request and refused it is answered with the error, not posted a mailbox
+copy. The display looks at the mailbox every
+`MAILBOX_POLL_INTERVAL_WITH_SOCKET` (1 s) while it serves the socket, and
+every `ON_DEMAND_POLL_INTERVAL` (0.25 s) without one, from its dwell sleep,
+its render loops and Vegas's interrupt check as well as the main loop; a
+look is one `stat()` unless the file changed. Both ways end in the same
+handler, `_handle_on_demand_request()`.
 The socket's handlers only queue; see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)
 for the protocol, the permission model and the plan to retire the mailboxes.
 
@@ -138,7 +143,9 @@ loaded and when. Nothing else keeps plugin state:
 `DisplayController` right after it creates the `PluginManager`, writes the
 cache key `plugin_runtime_snapshot`: per plugin `loaded`, `state`, `error`
 (type, a redacted message of at most 200 characters, when, recoverable),
-`version` and `loaded_at`, plus `published_at`, `stale_after` and `running`.
+`version`, `loaded_at` and `modes` (the display modes `DisplayController`
+registered -- `plugin.modes` when the plugin computes them, else the
+manifest's), plus `published_at`, `stale_after` and `running`.
 The cache is on disk, usually the SD card, so it writes when something a
 reader sees changes -- throttled to once per 10 s -- and otherwise once a
 minute as a heartbeat. RUNNING, which every `update()` passes through, is
@@ -154,6 +161,9 @@ truth cannot leak into a response. `/api/v3/plugins/installed` returns
 `loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` per
 plugin and `data.runtime` (`status`, `published_at`, `age_seconds`);
 `/api/v3/plugins/state` returns the same beside the desired state.
+`PluginCatalog.get_plugin_display_modes` and `find_plugin_for_mode` prefer a
+live view's `modes` to the manifest's `display_modes`, so `/display/modes`
+and on-demand see modes a plugin generates from its config (#668).
 
 **Reconciliation**
 ([`state_reconciliation.py`](../src/plugin_system/state_reconciliation.py))

docs/EMULATOR_SETUP_GUIDE.md

@@ -17,13 +17,13 @@ The LEDMatrix emulator allows you to run and test LEDMatrix displays on your com
 ## Prerequisites
 
 ### System Requirements
-- Python 3.10 or higher
+- Python 3.11 or higher (3.11 and 3.13 are tested)
 - Windows, macOS, or Linux
 - At least 2GB RAM (4GB recommended)
 - Internet connection for plugin downloads
 
 ### Required Software
-- Python 3.10+
+- Python 3.11+
 - pip (Python package manager)
 - Git (for plugin management)
 

docs/FONT_MANAGER.md

@@ -203,6 +203,7 @@ Current methods:
 | `measure_text(text, font)` | `(width, height, baseline)` |
 | `get_font_height(font)` | Line height |
 | `register_plugin_fonts(plugin_id, font_manifest)` | Register a plugin's fonts (core calls it at load) |
+| `forget_plugin_fonts(plugin_id)` | Drop a plugin's manifest fonts and their cached objects (core calls it when a plugin unloads) |
 | `clear_cache()` | Drop cached fonts and metrics |
 | `font_catalog` (attribute) | Family name → file path |
 
@@ -218,5 +219,6 @@ Removed in 3.8.0, after logging a deprecation warning on first call since
 | `get_performance_stats()` | — |
 | `set_override()`, `remove_override()`, `get_overrides()` | a font field in your plugin's config schema |
 | `get_manager_fonts()`, `get_detected_fonts()` | — |
-| `get_plugin_fonts()`, `unregister_plugin_fonts()` | — |
+| `get_plugin_fonts()` | — |
+| `unregister_plugin_fonts()` | `forget_plugin_fonts()` (core calls it on unload) |
 | `add_font()`, `remove_font()`, `validate_font()` | the web UI's Fonts tab |

docs/GETTING_STARTED.md

@@ -15,6 +15,12 @@ This guide will help you set up your LEDMatrix display for the first time and ge
 - Power supply (5V, 4A minimum recommended)
 - MicroSD card (16GB minimum)
 
+**Software:**
+- Raspberry Pi OS Lite, Trixie (Debian 13) or Bookworm (Debian 12). Trixie
+  is the current release; Bookworm is listed as Legacy in Raspberry Pi
+  Imager. No other system is supported, and the installer says so up front.
+- The OS's own Python: 3.13 on Trixie, 3.11 on Bookworm
+
 **Network:**
 - WiFi network (or Ethernet cable)
 - Computer with web browser on same network
@@ -28,7 +34,8 @@ This guide will help you set up your LEDMatrix display for the first time and ge
 There is no prebuilt SD card image — you install LEDMatrix onto stock
 Raspberry Pi OS Lite yourself:
 
-1. Flash Raspberry Pi OS Lite to the MicroSD card (Raspberry Pi Imager)
+1. Flash Raspberry Pi OS Lite (Trixie, or Bookworm) to the MicroSD card
+   (Raspberry Pi Imager)
 2. Connect the LED matrix to your Raspberry Pi, insert the card, and
    power on
 3. SSH into the Pi and run the one-shot installer:
@@ -39,6 +46,17 @@ Raspberry Pi OS Lite yourself:
    [README Installation Steps / Quick Install](../README.md#installation-steps)
    for full details
 
+   The one-shot installer installs the newest release (the **stable** update
+   channel). To run the newest, unreleased code from `main` instead (the
+   **beta** channel), put `LEDMATRIX_CHANNEL=beta` in front of `bash`:
+   ```bash
+   curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | LEDMATRIX_CHANNEL=beta bash
+   ```
+   A manual clone starts on `main`; add `--beta` to `first_time_install.sh`
+   to stay on it, or leave it off and the first update after the next
+   release moves the device onto releases. You can switch channels later on
+   the General tab.
+
 **Expected Behavior after install:**
 - LED matrix will light up
 - A fresh install ships only the bundled `starlark-apps` and

docs/IPC_CONTROL_SOCKET.md

@@ -4,14 +4,21 @@ The display process serves a Unix socket that the web interface uses to send
 it commands and get an answer back. It replaces the cache-file "mailboxes" on
 the SD card one command at a time. Stage 1 carries on-demand start, stop and
 status. Stage 2 makes those commands land within a frame on every kind of
-screen, and adds `brightness.set` and `plugin.reload`. The file mailbox stays
-as a fallback for one release.
+screen, and adds `brightness.set` and `plugin.reload`. Stage 3 adds a state
+stream (`state.get`, `state.subscribe`), so the web interface reads what the
+display is doing from the socket instead of from cache files the display
+wrote to the SD card. Stage 4 makes the socket the only way a command goes
+while it works: the web interface writes a mailbox only when the socket
+cannot carry the request, `errors.clear` replaces the last command that
+always went through a mailbox, and the display looks at the mailboxes once
+a second, with a `stat()`. The file mailboxes and the cache keys stay as a
+fallback for one release.
 
 | | |
 |---|---|
 | Socket | `/run/ledmatrix/control.sock` (tmpfs) |
 | Served by | the display process ([`src/ipc/server.py`](../src/ipc/server.py)), started by `DisplayController.run()` |
-| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness) |
+| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop`, `POST /api/v3/plugins/update` (reload), `POST /api/v3/config/main` (brightness), `POST /api/v3/errors/clear`; and through [`web_interface/display_state.py`](../web_interface/display_state.py) (the state stream), `GET /api/v3/display/current-status`, `/display/on-demand/status`, `/plugins/installed` (`runtime`), `/plugins/state` and the reconciliations, `/health` (`display_loop`) |
 | Contract | [`src/ipc/contract.py`](../src/ipc/contract.py): messages, versions, framing and the socket path; both sides import it |
 | Override | `LEDMATRIX_CONTROL_SOCKET=/some/path.sock` for both processes, or `=off` to disable it |
 
@@ -36,6 +43,11 @@ running, the socket does not exist, and the web interface knows right away.
 
 ## Protocol (version 1)
 
+Stage 3 is still version 1: `state.get` and `state.subscribe` are new
+commands, and a stage-2 display answers them `unknown_command`, which the
+web interface treats as "no socket" and falls back from.
+
+
 **Framing.** One JSON object per line (newline-delimited JSON), UTF-8, at
 most 64 KiB per line (`MAX_MESSAGE_BYTES`). Senders encode with
 `ensure_ascii`, so a newline never appears inside a message. A connection
@@ -75,6 +87,19 @@ one. Clients branch on `error.code`, never on the message text.
 | `on_demand.status` | — | `{on_demand: {...}, current_mode, display_active}` | answered directly |
 | `brightness.set` | `{brightness: int 0-100}` | `{brightness, panel_brightness, dimmed, display_active}` | queued, awaited (2 s) |
 | `plugin.reload` | `{plugin_id}` | `{plugin_id, reloaded: true, version, modes}` | queued, awaited (10 s) |
+| `state.get` | `{since?, epoch?}` | a state snapshot (see "The state stream") | answered directly |
+| `state.subscribe` | — | a state snapshot, then pushed `state` / `tick` events | answered directly, then a stream |
+| `errors.clear` | `{cutoff: number}` (epoch seconds, finite, ≥ 0) | `{request_id, cutoff, cleared}` | answered directly, once applied |
+
+`errors.clear` (stage 4) forgets the plugin errors the display recorded at
+or before `cutoff` and rewrites its error snapshot (`plugin_error_snapshot`)
+before it answers, so the web interface's next read already has it. The
+request `id` is the clear's id, which the snapshot reports as
+`applied_clear_id`. It is answered on the connection thread by a handler the
+display registers (`ControlServer(handlers=...)`, the contract's
+`DIRECT_COMMANDS`): the error aggregator and its publisher have their own
+locks, and nothing the render thread owns is touched. A display that has no
+handler answers `unknown_command`, as an older display does.
 
 `duration` is a number of seconds, or a numeric string. `0`, `null` or `""`
 mean "until stopped". `pinned` must be a real boolean: the REST route has
@@ -130,6 +155,19 @@ web interface treats like any other socket failure and falls back from, and
 `hello` lists the commands a display knows. The version changes only when the
 envelope or the meaning of an existing command changes.
 
+**Events.** `state.subscribe` is the one command with more than one message
+in reply. After its response, the display pushes events on the same
+connection until either side hangs up:
+
+```json
+{"v": 1, "id": "<the subscribe id>", "event": "state", "result": {...a state snapshot...}}
+{"v": 1, "id": "<the subscribe id>", "event": "tick",  "result": {"version": 7, "epoch": "…", "pid": 812, "served_at": 1790000000.1, "changed": false, "loop": {...}, "volatile": {"display": {"last_updated": 1790000000.0}, "...": "..."}}}
+```
+
+An event has `event` where a response has `ok`, which is how a reader tells
+them apart. The client sends nothing after the subscribe; anything it does
+send is ignored.
+
 **Error codes:** `bad_json`, `bad_request`, `message_too_large`,
 `unsupported_version`, `unknown_command`, `invalid_args`, `busy` (queue full,
 or too many connections), `forbidden` (peer credentials refused), `internal`.
@@ -147,6 +185,177 @@ print(client.brightness_set(60))
 EOF
 ```
 
+## The state stream (stage 3)
+
+Before stage 3 the web interface learned what the display was doing by
+reading files the display kept writing:
+
+| What | Written by the display | How often | Medium |
+|---|---|---|---|
+| current mode, plugin, `is_display_active`, `on_demand_active` | `display_current_state` | every mode change, every flag change, and every 30 s | cache (SD card) |
+| on-demand session | `display_on_demand_state` | on each on-demand event | cache (SD card) |
+| plugin runtime snapshot (#690) | `plugin_runtime_snapshot` | on a change (at most every 10 s), else every 60 s | cache (SD card) |
+| render-loop liveness (#687) | `display-heartbeat.json` | every 5 s | tmpfs |
+
+Now the display also keeps the same state in memory and serves it on the
+socket.
+
+**The snapshot.** `state.get` and `state.subscribe` answer with one object:
+
+```json
+{"schema": 1, "version": 42, "epoch": "3f9c0d1e2a4b5c6d", "pid": 812,
+ "served_at": 1790000000.1, "changed": true,
+ "loop": {"heartbeat_age_seconds": 1.8, "armed": true, "stale_after": 60.0},
+ "state": {
+   "display":    {"mode": "nfl_live", "plugin_id": "football-scoreboard", "mode_index": 3,
+                  "total_modes": 9, "on_demand_active": false, "is_display_active": true,
+                  "last_updated": 1790000000.0},
+   "on_demand":  {"active": false, "status": "idle", "...": "as display_on_demand_state"},
+   "brightness": {"brightness": 80, "panel_brightness": 40, "dimmed": true},
+   "plugins":    {"schema": 1, "running": true, "published_at": 1789999998.5, "...": "as plugin_runtime_snapshot"},
+   "loop":       {"heartbeat_age_seconds": 1.8, "armed": true, "stale_after": 60.0}
+ }}
+```
+
+- `display` and `on_demand` are the dicts the cache keys hold, `plugins` is
+  the runtime snapshot (`build_runtime_snapshot`), and `brightness` is the
+  configured level, what the panel shows now, and whether the dim schedule
+  has it dimmed. A section not published yet is `null`.
+- `loop` is not published: the display measures it when it answers, from
+  the render thread's last beat in memory (`RenderWatchdog.liveness()`),
+  the same beat that writes the heartbeat file. So it keeps ageing while the
+  render thread is stuck, and the socket's connection threads still answer.
+  `heartbeat_age_seconds` is `null` until the loop has drawn its first frame.
+- `version` goes up whenever a section changes, ignoring the timestamps that
+  move on every publish (`last_updated`, `remaining`, `published_at`). It
+  counts within an `epoch`, one run of the display process, so a reader that
+  sees a new `epoch` has a restarted display.
+- `state.get` with `since` and `epoch` from an earlier answer gets just
+  `{changed: false, version, epoch, pid, served_at, loop, volatile}` while
+  nothing has changed. `volatile` is `{section: {key: value}}`: the current
+  values of those ignored timestamps, which the reader merges into the copy
+  it has. They don't make a new version, but they are still news:
+  `display.last_updated` is how a reader knows the render thread is still
+  publishing, and `plugins.published_at` the runtime publisher. Without
+  them a reader's copy kept the timestamps of the last real change, so a
+  mode on screen for over 120 s read as unknown.
+- A snapshot that would not fit in a message (hundreds of plugins) is sent
+  without `plugins`, and `truncated: ["plugins"]` says so. Readers then use
+  the cache for that section only.
+
+**The stream.** `state.subscribe` answers with the snapshot, then:
+
+- a `state` event (a full snapshot) whenever the version changes, and
+- a `tick` at least every 5 s (`SUBSCRIBE_KEEPALIVE_SECONDS`) when nothing
+  changed. It is the short `changed: false` answer, so it carries `loop`
+  (a stalled render loop shows up within one tick) and `volatile` (the
+  timestamps stay as fresh as the writers keep them), and it tells the
+  reader the connection is alive.
+
+A slow reader is never sent a backlog: each event is the latest version, so
+one that falls behind skips the versions in between. A reader that has heard
+nothing for 15 s (three keepalives) stops trusting its copy.
+
+**Who publishes, and when.** All of it is in memory, with no disk writes:
+
+- the render thread, at the places it already published the cache keys:
+  `display` and `brightness` on every pass of
+  `_publish_current_mode_state_if_changed()` (every loop pass, and every
+  `_service_pending_changes()` in a dwell, a scrolling screen or Vegas), and
+  `on_demand` in `_publish_on_demand_state()`. Every pass refreshes
+  `display.last_updated`, so a reader can tell when the render thread has
+  stopped publishing, just as the cache key's 120 s `max_age` does.
+- the plugin runtime publisher's thread, on every 5 s tick: the snapshot is
+  rebuilt when the state machine changed, otherwise only its `published_at`
+  moves. A change reaches subscribers within a tick, without the cache's
+  10 s throttle.
+
+Publishing is a hand-off, as the command queue is in the other direction.
+The hub (`StateHub` in [`src/ipc/server.py`](../src/ipc/server.py)) holds a
+lock only to swap a dict reference, compare it with the last one and bump the
+version. Every socket write happens on the subscriber's own connection
+thread. The render thread never waits for a reader.
+
+### Readers in the web interface
+
+[`web_interface/display_state.py`](../web_interface/display_state.py) holds
+one `state.subscribe` connection per web process
+(`src.ipc.client.StateSubscription`, a daemon thread, started on the first
+read and reconnecting with a backoff of 1 s up to 30 s). A route answers
+from the latest pushed snapshot in memory. Before the subscription has one,
+the route asks once with `state.get` (0.5 s timeout). When neither works, it
+reads the cache keys and the heartbeat file as before:
+
+| Route | From the socket | Fallback |
+|---|---|---|
+| `GET /api/v3/display/current-status` | `state.display` | `display_current_state` |
+| `GET /api/v3/display/on-demand/status` | `state.on_demand`, with `remaining` worked out from `expires_at` now | `display_on_demand_state` |
+| `GET /api/v3/plugins/installed` (`runtime`), `/plugins/state`, `POST /plugins/state/reconcile` and the startup reconciliation | `state.plugins` + `state.loop` | `plugin_runtime_snapshot` + `display-heartbeat.json` |
+| `GET /api/v3/health` (`checks.display_loop`) | `state.loop` | `display-heartbeat.json` |
+
+Each answer says where it came from: `source: "socket" | "cache"` (or
+`"heartbeat_file"` for the health check).
+
+The SSE display stream (`/api/v3/stream/display`) reads the preview frame
+file, not a cache key, so it does not change.
+
+**The same verdicts either way.** The socket's answers are judged by the
+rules the cache readers apply (#726):
+
+- the runtime view is `stalled` when the render loop's heartbeat age is at
+  least `HEARTBEAT_STALE_SECONDS` (60 s, the health check's threshold), and
+  then reports no per-plugin facts;
+- it is `stale` when the snapshot is older than its `stale_after` (the
+  publisher thread stopped);
+- with no beat yet, the snapshot is judged on its own;
+- there is no pid check, because the display that answered is alive;
+- a `display` section the render thread has not refreshed for 120 s reads
+  as unknown, as the cache key does once it ages out.
+
+The age a reader uses is the age the display measured, plus the time since
+the snapshot arrived.
+
+### Fewer SD writes
+
+The cache keys are still written, for one release, as the fallback. While
+the socket serves the readers, the display writes two of them less often.
+"Serves the readers" means a subscriber is connected, or a `state.get` came
+within the last 60 s (`StateHub.readers_active()`):
+
+- `display_current_state` is no longer written on every mode change: once
+  every 60 s (`CURRENT_STATE_RELAXED_REFRESH_SECONDS`, inside the readers'
+  120 s `max_age`), and at once when `is_display_active` or
+  `on_demand_active` changes.
+- `plugin_runtime_snapshot`'s refresh goes from 60 s to 120 s
+  (`RELAXED_REFRESH_INTERVAL`), and the snapshot says so in its own
+  `refresh_interval` and `stale_after` (360 s). Changes are still written at
+  once, at most every 10 s.
+
+`display_on_demand_state` is written only on events, so it is unchanged.
+The heartbeat file is on tmpfs, so it costs no SD writes, and it stays: the
+automatic update's health check reads it.
+
+This is safe because the relaxed rate only applies while readers are using
+the socket. If they stop (the web interface loses the socket, or is stopped),
+the next publish after the reader window writes a changed mode at once, and
+the runtime refresh goes back to 60 s. A fallback reader in that window sees
+a mode up to 60 s old, never one older than its `max_age`.
+
+Measured with fake clocks (`test_cache_writes_per_minute_with_and_without_socket_readers`
+in `test/test_state_stream_readers.py`), for a rotation of 15 s screens:
+
+| Key | Writes/min, no socket readers | Writes/min, socket readers |
+|---|---|---|
+| `display_current_state` | 4.0 | 1.0 |
+| `plugin_runtime_snapshot` | 1.0 | 0.5 |
+| Total | 5.0 | 1.5 |
+
+That is 70% fewer writes for these keys: about 2,200 a day instead of 7,200.
+Shorter screens save more, because the old rate followed the mode changes.
+A display that rarely changes mode (one plugin, a long live game) saves less. Plugin
+data caches, the error snapshot and font usage are written by other code
+and are not affected.
+
 ## How the display applies a command
 
 The server's threads never touch rendering. A connection thread parses the
@@ -173,7 +382,9 @@ place it reads the mailbox:
   Vegas iteration is on the stack (`_apply_pending_plugin_reloads`). Until
   then the current screen ends early, as it does for a WiFi notice: the
   frame loops, the dwell and Vegas's interrupt check all treat a pending
-  reload as a reason to stop (`_screen_preempted`).
+  reload as a reason to stop (the frame loops through the Arbiter's
+  mid-screen check, `Source.RELOAD`; the dwell through
+  `_plugin_reload_pending`).
 - Only the quick half of the reload runs on the render thread
   (`_start_plugin_reload`): the plugin's modes leave the rotation, its
   config subscription is dropped, and `PluginManager.detach_plugin` takes
@@ -196,7 +407,7 @@ place it reads the mailbox:
   unloads it mid-load; a disable saved meanwhile is applied once the
   reload is done. A second reload of the same plugin runs after the first.
 
-The 0.25 s floor on the mailbox read does not apply to the queue, because
+The floor on the mailbox read (0.25 s, 1 s since stage 4) does not apply to the queue, because
 draining it costs no disk read. A queued command also lets
 `_service_pending_changes()` skip its own floor.
 
@@ -224,7 +435,7 @@ Now the queue wakes the render thread:
 
 So a command lands within a millisecond or so on a static screen and in a
 dwell, and within one frame in Vegas and on a scrolling screen. The mailbox
-keeps its old delays. Commands still run only on the render thread: the
+is slower on purpose (see "The mailboxes now"). Commands still run only on the render thread: the
 connection threads only queue them and set the event. The one exception is
 the slow half of `plugin.reload` (tearing down and loading the plugin),
 which runs on its own thread. Every change to the display's state still
@@ -242,11 +453,74 @@ bookkeeping. A client's send to the render thread waking took 0.72 ms median
 Without a socket (Windows, `LEDMATRIX_CONTROL_SOCKET=off`) the waits are the
 plain sleeps they were.
 
-**Exactly once.** A command and a mailbox write for the same request share
-one `request_id`. If the client times out after the display queued the
-command and then also writes the mailbox, the display processes the request
-once. The existing `on_demand_request_id` and processed-id checks drop the
-second copy.
+**Exactly once.** Since stage 4 the web interface writes the mailbox only
+when the display never had the request (see "When the web interface falls
+back"), so a request goes one way or the other, never both. A command and a
+mailbox write for the same request still share one `request_id`, and the
+`on_demand_request_id` and processed-id checks still drop a second copy: an
+older web interface (before stage 4) wrote the mailbox after a reply timed
+out, too. The display takes such a copy out of the mailbox when it drops it.
+
+## When the web interface falls back (stage 4)
+
+The client tells a request the display never had from one it had and then
+failed. `ControlError.sent` is True once the whole request was written to a
+connected display; a refusal the display sends before reading anything
+(`forbidden`, too many connections) carries no request id, and leaves it
+False. `src.ipc.client.should_fall_back()` is the one rule every route uses:
+
+| What happened | Example reasons | Mailbox? | The route answers |
+|---|---|---|---|
+| The display never had it | `no_socket`, `refused`, `disabled`, `unsupported`, a connect or send that timed out, `forbidden` / `busy` at the door, `invalid_request` (refused by the client itself) | yes | success, `transport: "mailbox"`, `socket_error` |
+| A display too old to know it (the upgrade case) | `unknown_command`, `unsupported_version` | yes | as above |
+| The display had it and failed | `busy` (queue full), `invalid_args`, `internal`, a timeout or hang-up after the send, `bad_response` | no | `503` (`400` for `invalid_args`), `socket_error` |
+
+A display that had the request may have applied it (a reply that timed out),
+or would refuse the mailbox copy as well (bad arguments), or is stuck and
+would not read the mailbox either (a full queue). Writing the copy anyway
+only turned that into a "success". An on-demand stop with `stop_service`
+still stops the service, which ends on-demand whatever happened.
+
+Brightness and plugin reload never had a mailbox: without the socket, the
+config watcher applies the saved brightness and a reload becomes the
+restart banner, as before.
+
+### The mailboxes now
+
+| Mailbox | Written by | Read by the display | While the socket is up |
+|---|---|---|---|
+| `display_on_demand_request` | the web interface, only on fallback; plugins that predate `BasePlugin.request_on_demand()`, or run on a core without it | the render thread, `_poll_on_demand_requests()` | looked at every 1 s (`MAILBOX_POLL_INTERVAL_WITH_SOCKET`), 0.25 s without a socket |
+| `plugin_error_clear_request` | the web interface, only on fallback | the error publisher's thread, every 5 s tick | unchanged rate |
+
+A look is one `stat()` of the mailbox file (`CacheManager.file_signature`):
+`(inode, mtime, size)`, and every write renames a new file into place, so a
+new write always looks different. `MailboxWatch` reads the file only when
+that changed since the last look, so a mailbox that holds nothing new, or
+nothing at all, costs no open and no parse. A socket command never reads or
+deletes the on-demand mailbox. A start already processed is taken out of
+the mailbox instead of being re-read until it expires.
+
+A request that comes through the on-demand mailbox while the socket is up
+is logged once per writer (`came through the file mailbox although the
+control socket is up`), which names the plugins that still write it.
+
+### Plugins in the display process
+
+A plugin asks for the screen with `BasePlugin.request_on_demand()` and gives
+it back with `end_on_demand()` (see "On-demand display" in
+[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)). Neither goes through
+the socket or a file: `PluginManager` hands the mailbox-shaped request,
+marked `source: 'plugin'`, to `DisplayController.submit_plugin_on_demand`,
+which queues it in memory (at most `PLUGIN_ON_DEMAND_QUEUE_SIZE`, 32) from
+whatever thread the plugin called on, and wakes the render thread through
+the socket's queue flag (`ControlServer.wake()`). The render thread applies
+it in `_drain_control_commands`, after the socket's commands, through the
+same `_handle_on_demand_request`, so it lands within a frame like a socket
+command. Without a socket it lands on the next pending-changes pass (typically
+within 0.25 s). A plugin's stop ends only a session that plugin owns. The four
+plugins that wrote the mailbox (birdnet-go, mqtt-notifications, on-air,
+pomodoro-timer) use it where the core has it and write the mailbox
+otherwise.
 
 ## Robustness
 
@@ -263,9 +537,10 @@ block the render loop or crash it:
   and the connection is closed, because the next message boundary cannot be
   found. A client that disconnects mid-message is dropped silently. No
   exception from a handler leaves the connection thread.
-- **Full queue.** When the queue is full, the client gets `busy` and falls
-  back to the mailbox. A full queue means the render thread is stuck, and the
-  systemd watchdog deals with that.
+- **Full queue.** When the queue is full, the client gets `busy`, and the web
+  interface answers `503` rather than write the mailbox, which the stuck
+  render thread would not read either. A full queue means the render thread
+  is stuck, and the systemd watchdog deals with that.
 - **Awaited commands.** The wait for an awaited command's outcome happens on
   its connection thread and is bounded (`AWAIT_SECONDS`), so a stuck render
   thread costs that client `pending` and one connection slot for at most
@@ -279,7 +554,16 @@ block the render loop or crash it:
   process created.
 - **Never fatal.** If the server cannot start (Windows, no `AF_UNIX`, a bind
   failure, `LEDMATRIX_CONTROL_SOCKET=off`), it logs that and the display runs
-  as before. The web interface then uses the mailbox.
+  as before. The web interface then uses the mailbox, and reads the cache
+  keys and the heartbeat file.
+- **Subscribers (stage 3).** A `state.subscribe` connection gives its request
+  slot back and takes one of 4 subscriber slots (`MAX_SUBSCRIBERS`). A fifth
+  gets `busy`. So a few browsers' web processes holding streams can never
+  use up the 8 slots that commands need. Each subscriber has its own thread.
+  A send that cannot finish within the 2 s IO timeout (a reader that stopped
+  reading) drops that subscriber. Nothing else waits for it, and the render
+  thread only publishes to the hub. `close()` wakes every subscriber, so
+  they end at once.
 
 ## Security model
 
@@ -313,8 +597,10 @@ read its state, set the brightness, and reload a plugin the display is
 already running, all of which anyone who can reach the web UI can already do
 (the last by restarting the display). Nothing on the socket runs a shell,
 writes a file, or names a path, and `plugin.reload` cannot make the display
-import a plugin it was not running. Stage 2 changed none of the access rules
-above.
+import a plugin it was not running. Stages 2 and 3 changed none of the
+access rules above. The state stream carries what the cache keys already
+held, and those are readable by the same group. A subscriber goes through
+the same connect-time and peer-credential checks as any other connection.
 
 **Development.** A display that is not root and cannot write to
 `/run/ledmatrix`, such as `python3 run.py -e` from a checkout, serves the
@@ -352,29 +638,51 @@ device never touches the live display.
      "which sections changed" ack had no reader: the web interface knows
      what it saved. A reload from the socket thread would also run every
      config subscriber on a second thread beside the watcher's.
-3. **A state stream.** A `subscribe` command that keeps the connection open
-   and pushes events: mode changes, on-demand state (including the outcome of
-   an acked on-demand command, which today is only published), plugin
-   runtime state, the outcome of a reload that answered `pending`, and the
-   heartbeat. It replaces the polled `display_current_state`,
-   `plugin_runtime_snapshot` (#690) and `display-heartbeat.json` (#687) for
-   readers that hold a connection. The web interface relays it to its
-   existing SSE stream. The files remain for one release for older readers.
-   - The server's per-connection threads (8 at most) do not suit long-lived
-     subscribers. A subscriber needs its own bound and a writer that drops
-     events for a slow reader rather than blocking the display.
-   - Events are produced on the render thread, so publishing must be a
-     non-blocking hand-off, like the queue in the other direction.
-   - The store's install of an already-enabled plugin, and an uninstall that
-     keeps its config, still answer `restart_required`. With the stream they
-     can use a load/unload command and report the result the same way the
-     update route does now.
-4. **Retire the mailboxes.** After a release in which every device has had the
-   socket, the web interface stops writing `display_on_demand_request`, and
-   the display stops polling it, logging the plugins that still write it so
-   they can move to an in-process `request_display()`. The other cache keys
-   used as messages (`plugin_error_clear_request` and the remaining
-   `display_*` keys) move to the socket or to tmpfs.
+3. **A state stream (done).** `state.get` (a versioned snapshot) and
+   `state.subscribe` (the snapshot, then pushed changes and keepalive ticks)
+   carry the current mode, the on-demand state (including the outcome of an
+   acked on-demand command), the brightness, the plugin runtime snapshot and
+   the render loop's liveness, all served from memory (see "The state
+   stream"). The web interface's readers use it and fall back to the cache
+   keys and the heartbeat file. `display_current_state` and
+   `plugin_runtime_snapshot` are written less often while it serves them.
+   The keys remain for one release.
+   - Left for later: the outcome of a `plugin.reload` that answered
+     `pending` is visible only as the plugin's new `loaded_version` in
+     `state.plugins`, not as an event of its own.
+   - Left for later: the SSE display stream reads the preview frame, not
+     state, so nothing relays the stream to the browser yet. A browser still
+     polls the REST routes, which now answer from memory.
+   - Left for later: the store's install of an already-enabled plugin, and
+     an uninstall that keeps its config, still answer `restart_required`.
+     They can now use a load/unload command and report the result the same
+     way the update route does.
+4. **The mailboxes become a fallback (done).**
+   - The web interface writes a mailbox only when the socket could not carry
+     the request (`should_fall_back`); a display that had it and failed is
+     answered as that (see "When the web interface falls back").
+   - `errors.clear` replaces `plugin_error_clear_request` as the way a clear
+     reaches the display.
+   - The display looks at the on-demand mailbox once a second while the
+     socket is up, reads either mailbox only when its file changed, and logs
+     who still writes the on-demand one (see "The mailboxes now").
+   - Not changed, deliberately: config saves (the schedule, the dim
+     schedule, plugin settings) still reach the display through
+     `config.json` and its watcher, which is the setting itself rather than
+     a message; see `config.reload` under stage 2. The preview viewer marker
+     (`/tmp/led_matrix_preview_viewer`) is a presence signal the display
+     already stats at most once a second. Plugin health and metrics resets
+     write the persisted record the display publishes and do not reach the
+     running display (their routes say so); they are not mailboxes.
+5. **Remove the mailboxes (next release).** Once every device has run a
+   display with stage 4, the web interface stops writing both mailboxes and
+   the display stops reading them. The four plugins that wrote
+   `display_on_demand_request` now have an in-process way to ask for the
+   screen (`BasePlugin.request_on_demand()` / `end_on_demand()`, see
+   "Plugins in the display process"); they keep the mailbox write only as
+   their fallback on older cores. The display also stops writing `display_current_state`,
+   `display_on_demand_state` and `plugin_runtime_snapshot` once the web
+   interface no longer falls back to them.
 
 ## Checking it on a device
 
@@ -389,7 +697,18 @@ curl -s -X POST localhost:5000/api/v3/display/on-demand/start \
 If the response says `"transport": "mailbox"`, `socket_error` gives the
 reason. `no_socket` means the display is stopped or predates the socket.
 `refused` usually means the web user is not in the socket's group, which
-takes effect when the web service restarts after the user is added.
+takes effect when the web service restarts after the user is added. A `503`
+with `"transport": "socket"` means the display had the request and did not
+take it (`busy`, `timeout`, ...): nothing was written to the mailbox.
+
+An error clear:
+
+```bash
+curl -s -X POST localhost:5000/api/v3/errors/clear \
+  -H 'Content-Type: application/json' -d '{"all":true}'
+# ... "applied": true, "transport": "socket"
+sudo journalctl -u ledmatrix | grep -E "Cleared .* plugin error|file mailbox"
+```
 
 Brightness and a plugin reload:
 
@@ -405,3 +724,19 @@ sudo journalctl -u ledmatrix | grep -E "Brightness set|Reload(ing|ed) plugin"
 
 `unknown_command` in `brightness_socket_error` or `reload_error` means the
 display runs a stage-1 build: restart it once to pick up this one.
+
+The state stream:
+
+```bash
+curl -s localhost:5000/api/v3/display/current-status    # ... "source": "socket"
+curl -s localhost:5000/api/v3/health | python3 -m json.tool | grep -A3 display_loop
+python3 - <<'EOF'
+from src.ipc import client          # run from the project directory
+snap = client.state_get()
+print(snap['version'], snap['epoch'], snap['loop'], snap['state']['display'])
+EOF
+```
+
+`"source": "cache"` means the web interface could not use the socket: the
+display is stopped, predates stage 3, or the web user is not in the
+socket's group.

docs/PERMISSIONS.md

@@ -33,6 +33,9 @@ in again (services pick them up on restart).
 | `/run/ledmatrix/control.sock` | `root` : cache directory's group (`ledmatrix`) | `660` | The display's control socket; only root and that group can connect. See [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md#security-model) |
 | `scripts/fix_perms/safe_plugin_rm.sh`, `safe_pip_install.sh` | `root:root` | `755` | Run as root through sudo, so the web user must not be able to edit them |
 | `/etc/sudoers.d/ledmatrix_web`, `ledmatrix_wifi` | `root` | `440` | |
+| `/usr/local/sbin/ledmatrix-refresh-units` | `root:root` | `755` | Copy of `scripts/install/ledmatrix_refresh_units.py`, installed by `install_service.sh`. Outside the project so the web user cannot edit what sudo runs |
+| `/var/lib/ledmatrix/unit-backup/` | `root` | `700` | The units the last refresh replaced, for the automatic update's rollback |
+| `/etc/systemd/system/ledmatrix*.service`, `.path` | `root:root` | `644` | Readable so the web interface can compare them with the templates after an update |
 
 What keeps it that way at runtime:
 
@@ -86,6 +89,20 @@ password:
 - `journalctl -u ledmatrix.service *`, `-u ledmatrix *`, `-t ledmatrix *`,
   tagged `NOEXEC`: journalctl opens a pager on a terminal, and a shell
   escape from that pager would be a root shell
+- `/usr/local/sbin/ledmatrix-refresh-units ""` and
+  `/usr/local/sbin/ledmatrix-refresh-units --restore` — exactly these two
+  command lines (`""` means "no arguments"). After an update the first
+  installs the systemd units whose templates changed and runs
+  `systemctl daemon-reload`; the automatic update's rollback runs the second
+  to put the previous units back. The helper takes nothing from the caller:
+  the project folder and the web user come from the installed, root-owned
+  `ledmatrix.service` and `ledmatrix-web.service`. It only replaces the four
+  units `install_service.sh` installs, only if they are already installed,
+  and refuses a template that would change a unit's `User=` (root for the
+  display, the web user for the rest) or `WorkingDirectory=`, or that is a
+  symlink, not a regular file, or over 64 KB. It grants nothing new: the
+  templates are files the web user can edit, but so is `run.py`, which the
+  display service already runs as root.
 
 ### `/etc/sudoers.d/ledmatrix_wifi`
 
@@ -136,7 +153,9 @@ directory.
 | `safe_plugin_rm.sh`, `safe_pip_install.sh` | — | Called by the web interface through sudo | Not for manual use |
 
 To reinstall the sudoers rules, run
-`./scripts/install/configure_web_sudo.sh` (web rules) or
+`./scripts/install/configure_web_sudo.sh` (web rules; the
+`ledmatrix-refresh-units` rules also need the helper itself, which
+`sudo ./scripts/install/install_service.sh` installs) or
 `./scripts/install/configure_wifi_permissions.sh` (WiFi rules and polkit) as
 the web user, not with `sudo`.
 

docs/PLUGIN_API_REFERENCE.md

@@ -488,6 +488,78 @@ working for the plugin itself. `get_vegas_segment_width()` read the
 `vegas_panel_count` config value, which has never affected Vegas — a card's
 width comes from `get_vegas_content()` and `vegas_width_pct`.
 
+### On-demand display
+
+A plugin that reacts to something outside the rotation (an MQTT message, a
+timer, a detection) can take the screen for it, and give it back. Both
+methods are safe from any thread, including an MQTT callback: they only
+queue the request, and the display applies it on its render thread within a
+frame or so, exactly like an on-demand start or stop from the web interface.
+
+#### `request_on_demand(mode=None, duration=None, pinned=False) -> Optional[str]`
+
+Show this plugin now.
+
+- `mode`: one of the plugin's display modes; `None` for its first.
+- `duration`: seconds before the rotation resumes; `None` (or `0`) for no
+  limit, until `end_on_demand()` or the user stops it.
+- `pinned`: stay on `mode` instead of cycling through the plugin's other
+  modes.
+
+Returns the request id once the display has queued it, or `None` when
+there is no display in this process to ask (the web interface's plugin
+manager, `scripts/check_plugin.py`) or its queue is full. A bad argument
+(a `mode` that is not a string, a `duration` that is not a number) raises
+`ValueError`.
+
+#### `end_on_demand() -> Optional[str]`
+
+Give the screen back. Ends only a session this plugin owns: a session the
+user started for another plugin, or one that already ended, is left alone.
+Returns the request id once queued, or `None` as above.
+
+#### Older cores: feature detection
+
+These methods are new after core 3.8.0 (see `CHANGELOG.md`). Before them,
+plugins wrote the `display_on_demand_request` cache key (the "mailbox")
+themselves. The display reads it only once a second while the control
+socket is up, and it will be removed in a future release (see
+[IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md), stage 5). A plugin that
+must keep working on older cores checks for the method, and writes the
+mailbox only when the method is missing or answers `None`:
+
+```python
+import time, uuid
+
+def _show_alert(self):
+    if hasattr(self, "request_on_demand") and self.request_on_demand(
+            mode="my_alert", duration=15):
+        return
+    # Older core, or no display in this process: the mailbox, as before.
+    self.cache_manager.set("display_on_demand_request", {
+        "request_id": str(uuid.uuid4()), "action": "start",
+        "plugin_id": self.plugin_id, "mode": "my_alert",
+        "duration": 15, "pinned": False, "timestamp": time.time(),
+    })
+
+def _release(self):
+    if hasattr(self, "end_on_demand") and self.end_on_demand():
+        return
+    self.cache_manager.set("display_on_demand_request", {
+        "request_id": str(uuid.uuid4()), "action": "stop",
+        "plugin_id": self.plugin_id, "timestamp": time.time(),
+    })
+```
+
+Keep `ledmatrix_min_version` where it is: the fallback is what keeps the
+plugin working on older cores. A mailbox stop ends any on-demand session,
+whoever started it; `end_on_demand()` ends only the plugin's own.
+
+Both methods answer a request id only when the plugin manager returned a
+string, so a test that gives the plugin a `MagicMock()` plugin manager gets
+`None` and exercises the mailbox path. To test the new path, set
+`plugin_manager.request_on_demand.return_value = "some-id"`.
+
 > The full source for `BasePlugin` lives in
 > `src/plugin_system/base_plugin.py`. If a method here disagrees with the
 > source, the source wins — please open an issue or PR to fix the doc.
@@ -966,6 +1038,14 @@ if info:
     self.logger.info(f"Plugin: {info['name']}, Version: {info.get('version')}")
 ```
 
+#### `request_on_demand(plugin_id, mode=None, duration=None, pinned=False)` / `end_on_demand(plugin_id)`
+
+What `BasePlugin.request_on_demand()` and `end_on_demand()` call, with the
+plugin's own id. Call those instead; see
+[On-demand display](#on-demand-display). The display controller routes them
+to itself with `set_on_demand_handler()`; a plugin manager without a
+display behind it answers `None`.
+
 #### `get_all_plugin_info() -> List[Dict[str, Any]]`
 
 Get information for all plugins.

docs/REST_API_REFERENCE.md

@@ -159,14 +159,16 @@ there an unchecked checkbox — which the browser omits — is saved as
 }
 ```
 
-`restart_required` is always true here: display hardware, rotation,
-durations and general settings take effect when the display restarts, and
-the web UI shows its restart banner on the flag. (Plugin sections saved
-through this route reach the running plugin live, like
-`POST /plugins/config`.)
+`restart_required` is true when the save changed a setting that takes
+effect when the display restarts: display hardware, rotation order,
+timezone, general settings and the rest. The web UI shows its restart banner
+on the flag. It is false when the save changed only what the running display
+applies by itself, or nothing: `brightness`, the per-mode durations
+(`duration__<mode>`, `display.display_durations`) and plugin sections, which
+reach the running plugin live, like `POST /plugins/config`.
 
-A saved `brightness` is the exception: it reaches the panel without a
-restart. The route also sends it to the running display over the control
+A saved `brightness` reaches the panel without a restart. The route also
+sends it to the running display over the control
 socket (`brightness.set`), which puts it on the panel at once, and the
 response adds `"brightness_transport": "socket"`. Otherwise it is
 `"config"`, with `brightness_socket_error` giving the reason, and the
@@ -246,7 +248,10 @@ Replace the schedule configuration.
 ```
 
 A day whose `<day>_enabled` key is absent counts as enabled, with default
-times `07:00`-`23:00`. At least one day must be enabled.
+times `07:00`-`23:00`. An enabled schedule needs at least one day enabled; a
+disabled one (`"enabled": false`) may have every day off, as
+`config.template.json` ships it. A day that is off keeps the times sent for
+it, when they are valid `HH:MM`.
 
 **Response**:
 ```json
@@ -331,12 +336,23 @@ by the display process (stale after 120 seconds).
   "data": {
     "mode": "nfl_live",
     "plugin_id": "football-scoreboard",
-    "last_updated": 1234567890.123
+    "last_updated": 1234567890.123,
+    "source": "socket"
   }
 }
 ```
 
-When nothing has been published, every field is `null`.
+When nothing has been published, every field is `null`. `source` is
+`socket` when the answer came from the display's state stream over the
+control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), and `cache`
+when it came from the `display_current_state` cache key (no socket: the
+display is stopped or older, or this is Windows). A display whose render
+loop has not refreshed its state for 120 seconds is reported with every
+field `null`, either way. So is a stopped display: when the socket does not
+answer and the render loop's heartbeat
+(`/run/ledmatrix/display-heartbeat.json`) is absent, stale or from a process
+that is gone, the cache's last entry is not used. A display still beating
+without a socket, Windows, or a socket switched off reads the cache.
 
 ### List Display Modes
 
@@ -347,9 +363,11 @@ it. This is the list the force-display dialog offers.
 
 Send the reported `plugin_id` alongside `mode` when starting an on-demand
 display: `/display/on-demand/start` falls back to `find_plugin_for_mode` when
-`plugin_id` is omitted, and that lookup only sees modes declared in a static
-manifest — a plugin whose modes are generated (each installed Starlark app is
-one) returns 404 there.
+`plugin_id` is omitted. While the display is running, this list and that
+lookup use the modes the display registered, including ones a plugin generates
+from its config (each installed Starlark app, each soccer `custom_leagues`
+entry). With the display stopped, or for a plugin it has not loaded, both see
+only the modes its manifest declares.
 
 Triggers plugin discovery, which is otherwise lazy — so a caller that never
 opens the dashboard still gets the full list.
@@ -413,11 +431,16 @@ Get the current on-demand display state.
       "returncode": 0,
       "stdout": "active",
       "stderr": ""
-    }
+    },
+    "source": "socket"
   }
 }
 ```
 
+`source` is `socket` (the display's state stream, with `remaining` worked
+out at the time of the request) or `cache` (the `display_on_demand_state`
+cache key).
+
 With no on-demand request, `state` is
 `{"active": false, "status": "idle", "last_updated": null}`.
 
@@ -443,7 +466,7 @@ Request a specific plugin to display on-demand.
 - `mode` (string, optional): Display mode name (plugin_id inferred if not provided)
 - `duration` (number, optional): Duration in seconds (0 = until stopped)
 - `pinned` (boolean, optional): Pin display (pause rotation)
-- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within about a quarter of a second. When false and the service is stopped, the route returns 400.
+- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within a frame over its control socket (within about a second through the mailbox fallback). When false and the service is stopped, the route returns 400.
 
 **Response**:
 ```json
@@ -468,10 +491,19 @@ display's control socket acknowledged it (it is queued for the render thread,
 which wakes for it and applies it within a frame; see
 [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), `"mailbox"` means it was
 written to the cache mailbox the display polls, as before the socket existed.
-With `"mailbox"`, `socket_error` gives the reason the socket was not used
-(`no_socket` when the display is stopped or predates the socket, `timeout`,
-`refused`, `busy`, ...). Either way the request is applied the same way;
-`request_id` is the same id in both.
+The mailbox is used only when the socket could not carry the request. With
+`"mailbox"`, `socket_error` gives the reason (`no_socket` when the display is
+stopped or predates the socket, `refused`, a connect `timeout`,
+`unknown_command` from a display too old for the command, ...). Either way
+the request is applied the same way; `request_id` is the same id in both.
+
+When the display had the request and did not take it -- a full queue
+(`busy`), bad arguments (`invalid_args`), no answer after the request was
+sent (`timeout`, `closed`) -- the route answers `503` (`400` for
+`invalid_args`) with `status: "error"` and `data: {request_id, transport:
+"socket", socket_error}`, and writes nothing to the mailbox. The stop route
+does the same, except that with `stop_service: true` it still stops the
+service and answers success.
 
 ### Stop On-Demand Display
 
@@ -552,7 +584,8 @@ List all installed plugins with their status and metadata.
       "published_at": 1790000030.0,
       "age_seconds": 12.4,
       "stale_after": 180.0,
-      "heartbeat_age_seconds": 2.1
+      "heartbeat_age_seconds": 2.1,
+      "source": "socket"
     }
   }
 }
@@ -581,7 +614,10 @@ the display is hung or died), `stopped` (the display shut down) or `unknown`
 (nothing published yet). Unless it is `live`, every one of those fields is
 `null`. `heartbeat_age_seconds` is the heartbeat's age when it was taken into
 account, `null` otherwise (no heartbeat, as on the dev server, or one from
-another process). Health and metrics are at [`/plugins/health`](#get-plugin-health)
+another process). `runtime.source` is `socket` when the snapshot and the
+heartbeat age came from the display's state stream over the control socket,
+and `cache` when they came from the `plugin_runtime_snapshot` cache key and
+the heartbeat file; the rules above are the same for both. Health and metrics are at [`/plugins/health`](#get-plugin-health)
 and `/plugins/metrics`.
 
 `vegas_participation` is what Vegas mode does with the plugin: `"scroll"`,
@@ -2223,13 +2259,11 @@ error with `"all": true` (`max_age_hours` is then ignored).
 }
 ```
 
-The clear is asynchronous. The web interface records a request
-(`plugin_error_clear_request` in the shared cache), and the display service
-applies it within about 5 seconds, rebuilding its counts from the errors it
-keeps and republishing. Reads hide the cleared errors from the moment the
-request is recorded. Until the display service applies an age-based clear,
-`recent_errors` and `active_patterns` are already filtered but the counts
-are the old ones, and `clear_pending` is `true`.
+The clear goes to the display service over its control socket
+(`errors.clear`, see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), which
+applies it, rebuilding its counts from the errors it keeps, and republishes
+before it answers: `applied` is `true`, `transport` is `"socket"`, and
+`cleared_count` is the display's own count.
 
 ```json
 {
@@ -2237,17 +2271,30 @@ are the old ones, and `clear_pending` is `true`.
   "data": {
     "cleared_count": 13,
     "clear_requested": true,
+    "applied": true,
+    "transport": "socket",
     "request_id": "5f0c1e...",
     "cutoff": "2026-09-23T09:59:02.310000"
   },
-  "message": "Clear of all errors requested; the display service applies it within about 5 seconds"
+  "message": "Cleared all errors"
 }
 ```
 
-`cleared_count` is how many of the reported errors the clear hides. It is
+When the socket cannot carry it (the display is stopped, or older than
+`errors.clear`) the clear is asynchronous, as before: the web interface
+records a request (`plugin_error_clear_request` in the shared cache),
+`applied` is `false` and `transport` is `"mailbox"`, and the display service
+applies it within about 5 seconds. Reads hide the cleared errors from the
+moment the request is recorded. Until the display service applies an
+age-based clear, `recent_errors` and `active_patterns` are already filtered
+but the counts are the old ones, and `clear_pending` is `true`. Then
+`cleared_count` is how many of the reported errors the clear hides, and
 `null` when that cannot be known before the display service applies it (an
-age-based clear over more errors than the report lists). A request that
-could not be written to the shared cache answers `500`.
+age-based clear over more errors than the report lists).
+
+A request that could not be written to the shared cache answers `500`. A
+display that had the request and failed it (`internal`, a timeout after the
+request was sent) answers `503`, with `context.socket_error`.
 
 ---
 
@@ -2265,7 +2312,14 @@ display snapshot. `data.status` is `healthy` or `degraded`, with
 (with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
 frozen even if the service is active; the status turns `degraded`), or
 `not_reported` when the display writes none (not started yet, the dev server,
-Windows), which does not affect the status.
+Windows), which does not affect the status, or `stopped` (with `source:
+"service"`) when the display service is not active, the control socket does
+not answer and there is no live heartbeat; the status then turns
+`degraded`. A platform with no control socket (Windows) or a socket switched
+off never reports `stopped`. Its `source` is `socket` when the
+age came from the display's state stream over the control socket (measured
+in memory by the display) and `heartbeat_file` when it came from
+`/run/ledmatrix/display-heartbeat.json`.
 
 Open even when the web login is on, for uptime monitors; a caller that is not
 logged in (and has no token) then gets only `{"status": "success", "data":
@@ -2325,8 +2379,9 @@ Replace the dim schedule. `dim_brightness` is 0-100 (default 30). In
 `per-day` mode the days can be sent either as the `days` object that GET
 returns, or as the web form's flat fields (`monday_enabled`,
 `monday_start`, `monday_end`, ...). A day that is not sent counts as
-enabled with default times `20:00`-`07:00`; at least one day must be
-enabled.
+enabled with default times `20:00`-`07:00`. As for the schedule above, an
+enabled dim schedule needs at least one day enabled and a disabled one may
+have every day off.
 
 ---
 

docs/RUN_LOOP_REDESIGN.md

@@ -36,122 +36,189 @@ Each pass, in order:
 1. `loop_pass()` (watchdog). Apply a pending plugin enable/disable, then
    any plugin reloads the control socket asked for
    (`_apply_pending_plugin_reloads`; a pending reload ends the screen
-   before it, like a WiFi notice, through `_screen_preempted`). The static
-   screen's frame sleep and the dwell wait on the socket's queue instead of
-   sleeping (`_wait_frame_interval`, `_sleep_with_plugin_updates`); without
-   a socket, as in the golden traces, they are the plain sleeps.
+   before it, like a WiFi notice, as `Source.RELOAD` at the runner's
+   service points). The static screen's frame sleep and the dwell wait on
+   the socket's queue instead of sleeping (`_wait_frame_interval`,
+   `_sleep_with_plugin_updates`); without a socket, as in the golden
+   traces, they are the plain sleeps.
 2. With no modes: dwell 1 s, next pass.
 3. Poll on-demand requests and expiry, release plugins loaded only for
    on-demand, tick plugin updates, drop an expired WiFi notice, evaluate
    the schedule (an on-demand session overrides scheduled-off), apply the
    brightness target. Then gather the Arbiter's inputs
-   (`_arbiter_inputs`) and call `Arbiter.decide()`, which picks one of
-   steps 4-6 or returns `LEGACY` for steps 7-9 (stage 2).
+   (`_arbiter_inputs`) and call `Arbiter.decide()`.
 4. **Scheduled off:** blank, dwell up to 60 s. `_blank_while_scheduled_off`
 5. **Follower:** render one frame from the leader. `_run_follower_frame`
 6. **WiFi notice** (unless on-demand): draw it, dwell 0.5 s. `_show_wifi_notice`.
-   It is also polled mid-screen (`_wifi_notice_pending`): the frame loops,
-   the dwell sleep and an interrupted Vegas iteration end within about a
-   second when one arrives, and a screen cut short resumes after it.
-7. **Live priority** (unless on-demand, or Vegas keeps live content in the
-   ticker): switch to the next live mode, or resume the rotation. A game
-   that goes live during a screen is caught sooner, by
-   `_check_live_takeover` in the frame loops and the dwell sleep (at most
-   once a second, and not while a live mode is showing).
-8. **Vegas** (unless on-demand, or live content preempts it): run one
-   iteration of up to `max_cycle_duration`. A completed iteration ends the
-   pass, and so does one that yielded for a WiFi notice or the schedule.
-   Any other interrupted one falls through to step 9 in the same pass.
-9. **One screen:** pick the mode (`_resolve_active_mode`), the plugin
-   (`_plugin_for_mode`), draw the first frame through the executor
-   (`_dispatch_first_frame`). On no content, rotate at once
-   (`_note_empty_pass`, `_skip_failed_plugin_modes`). Otherwise work out the
-   bounds (`_track_dynamic_cycle`, `_resolve_durations`,
-   `_clamp_to_on_demand`) and the frame rate (`_needs_high_fps`), run the
-   125 Hz or 1 Hz frame loop, make up the minimum duration, then pick the
-   next mode (`_advance_after_screen`).
+   It also ends a running screen within about a second (the runner's
+   service points), and a screen cut short resumes after it.
+7. **The Sources below the notice:** read whether Vegas is on and make the
+   live-priority scan (`_arbiter_inputs_below_wifi`, where run() always
+   read them), and call `decide()` again. It answers OnDemand (the
+   session's current mode), Live (the next live mode, round-robin; a game
+   that goes live during a screen takes over at the next service point, at
+   most once a second), Vegas (`LEGACY`) or Rotation. `_take_plan` applies
+   the answer: a live claim or the resume when live priority ends, the
+   on-demand index.
+8. **Vegas** (`_run_vegas_iteration`): one iteration of up to
+   `max_cycle_duration`. A completed iteration ends the pass, and so does
+   one that yielded for the schedule, a reload or a WiFi notice. Any other
+   interrupted one asks `decide()` once more (`vegas_yielded`): a game that
+   stopped the ticker, or an on-demand session that started, shows next.
+9. **One screen:** pick the plugin (`_plugin_for_mode`) and hand the plan
+   to the `ScreenRunner` (`src/screen_runner.py`). It draws the first frame
+   through the executor (`_dispatch_first_frame`), has the controller fill
+   in the plugin's durations, dynamic flag and frame policy
+   (`_complete_plan`), runs the 125 Hz or 1 Hz frame loop with a service
+   point after each frame, makes up the minimum duration, and returns an
+   `Outcome`. On `PREEMPTED` the pass ends without advancing. On no
+   content, rotate at once (`_note_empty_pass`, `_skip_failed_plugin_modes`).
+   Otherwise `ArbiterState.after()` picks the next mode
+   (`_advance_after_screen`).
 
-The helpers named above were extracted in stage 1 without changing
-behaviour. Since stage 2 the choice between steps 4, 5, 6 and the rest is
-made by `Arbiter.decide()` in `src/display_arbiter.py`. The frame loops, the
-Vegas branch and every early exit are still inline in `run()`.
-
-## Target design
+## Design
 
 ```python
 def run(self):
     while True:
-        inputs = self._drain_inputs()                 # requests, schedule, config, sync
-        plan = self.arbiter.decide(self.state, inputs, clock.now())
-        outcome = self.runner.run(plan)               # ExitReason + elapsed
-        self.state = self.state.after(plan, outcome)  # rotation, on-demand index, live resume
+        inputs = self._arbiter_inputs()                     # schedule, follower, notice
+        plan = Arbiter.decide(self._arbiter_state(), inputs, now)
+        ...                                                 # off / follower / notice
+        plan = self._take_plan(Arbiter.decide(state, self._arbiter_inputs_below_wifi(inputs), now))
+        if plan.source is Source.LEGACY:                    # Vegas, until stage 4
+            plan = self._run_vegas_iteration(...)
+        outcome = runner.run(plan, plugin)                  # ExitReason + elapsed
+        if outcome.exit_reason is not ExitReason.PREEMPTED:
+            self._advance_after_screen(plan, outcome)       # ArbiterState.after
 ```
 
+The controller's attributes (`current_display_mode`, `current_mode_index`,
+`on_demand_*`, `_live_resume_index`) stay the record that the web UI, the
+control socket and the on-demand cache read. `_arbiter_state()` snapshots
+them into a frozen `ArbiterState`; the transitions are pure methods on it,
+and the controller writes their result back (`_adopt_state`).
+
 ### Sources
 
 Each kind of content is a Source. A Source looks at the state and the
 inputs and either offers a screen or passes. The Arbiter asks them in this
 order:
 
-| Order | Source | Offers a screen when | Today |
+| Order | Source | Offers a screen when | Code |
 |---|---|---|---|
-| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | step 4 |
-| 1 | Follower | a sync leader is driving this panel | step 5 |
-| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_resolve_active_mode` |
-| 3 | Wifi | a status message is pending and on-demand is not active | step 6 |
-| 4 | Live | a live-priority plugin has live content (round-robin across several) | step 7 |
-| 5 | Vegas | Vegas is enabled and nothing above wants the panel | step 8 |
-| 6 | Rotation | always: `available_modes[current_mode_index]` | step 9 |
+| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | `decide` |
+| 1 | Follower | a sync leader is driving this panel | `decide` |
+| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_on_demand_plan` |
+| 3 | Wifi | a status message is pending and on-demand is not active | `decide` |
+| 4 | Live | a live-priority plugin has live content (round-robin across several) | `live_pick` |
+| 5 | Vegas | Vegas is enabled and nothing above wants the panel | `LEGACY`, run by `_run_vegas_iteration` |
+| 6 | Rotation | always: the rotation's current mode | `rotation_plan` |
 
 ScheduledOff is a gate in front of the Sources because that is how it works
 today: a scheduled-off panel stays blank even for a follower, and only an
 on-demand session overrides it.
 
+The Rotation answers `state.current_mode`, not
+`available_modes[current_mode_index]`: the two agree except where something
+moved the panel off the list and the rotation carries on from there (a live
+mode no rotation entry names, or None after a session ended with no enabled
+mode to resume to), and `run()` always showed `current_display_mode`.
+
 ### Arbiter
 
 ```python
-Arbiter.decide(state, inputs, now) -> ScreenPlan
+Arbiter.decide(state, inputs, now, running=None) -> ScreenPlan
 ```
 
 `decide` is a pure function: it does no I/O, takes no locks and does not
 sleep. It can be tested with plain tables of (state, inputs, now) mapped to
-an expected plan. It returns a `ScreenPlan`:
+an expected plan.
+
+- `ArbiterState`: the current mode; the rotation and its index; the
+  on-demand session's modes, index, expiry and pin; the live resume point;
+  whether a mid-screen takeover has not shown yet. Transitions:
+  `next_on_demand`, `showing`, `claim_live`, `release_live`, `after`.
+- `ArbiterInputs`: whether the schedule has the panel on, an on-demand
+  session, a follower, the WiFi notice, the live modes (None where no scan
+  was made), whether Vegas is on and keeps live content in its ticker,
+  whether this pass's Vegas iteration has yielded, and (mid-screen) whether
+  a plugin reload is waiting.
+- `ScreenPlan`:
 
 | Field | Meaning |
 |---|---|
 | `source` | which Source won |
-| `mode`, `plugin` | what to draw (None for a blank or follower plan) |
-| `min_duration`, `max_duration` | from `_resolve_durations` and `_clamp_to_on_demand` |
+| `mode`, `plugin` | what to draw (None for a blank or follower plan); the plugin id once resolved |
+| `min_duration`, `max_duration` | from `_resolve_durations` and the on-demand bound (`on_demand_bound`), filled in after the first frame; an on-demand plan's `max_duration` is what is left of the session at `now` |
 | `dynamic` | run until the plugin's cycle completes, between min and max |
-| `frame_policy` | today `_needs_high_fps` (125 Hz or 1 Hz); see stage 5 |
+| `frame_policy` | `HIGH_FPS` or `STATIC`, today `_needs_high_fps`; see stage 5 |
 | `preemptible_by` | the Sources allowed to interrupt this plan mid-screen |
+| `notice`, `deadline`, `ends_live` | the WiFi notice; the on-demand expiry for the bound; "live priority just ended, resume the rotation first" |
+
+`decide` cannot ask a plugin anything, so the fields a plugin answers are
+filled in by the controller after the first frame, where they were always
+read (`_complete_plan`).
+
+With `running`, `decide` answers the mid-screen question instead: `running`
+itself while the screen holds, else the plan that ends it
+(`_hold_or_preempt`), in the order the frame loops always checked:
+
+1. Live: a game went live while a non-live screen runs. It is the one
+   preemption that changes the state (the rotation moves to the live mode
+   and remembers where it was), and it is claimed even when a WiFi notice
+   is also pending; the next pass shows the notice, then the game.
+2. The panel's mode moved under the screen (on-demand started, ended or
+   changed mode; the rotation was rebuilt).
+3. The schedule turned the panel off.
+4. A WiFi notice (unless on-demand outranks it), compared with its expiry.
+5. A plugin reload is waiting (between frames only).
+
+Every screen is preemptible by the gate, OnDemand, Wifi, Live, Rotation and
+a reload (`SCREEN_PREEMPTERS`), except that a live screen leaves Live out
+(`LIVE_PREEMPTERS`): live games take turns between screens. A follower and
+Vegas are looked at only between screens.
 
 ### ScreenRunner
 
 ```python
-ScreenRunner(clock: FrameClock).run(plan) -> Outcome(exit_reason, elapsed)
+ScreenRunner(clock: FrameClock, host: ScreenHost).run(plan, plugin) -> Outcome
 ```
 
 The ScreenRunner draws the first frame (`_dispatch_first_frame`), runs the
 frame loop that the plan's frame policy selects, services pending changes
 between frames, and returns one `ExitReason`:
 
-| ExitReason | Today's equivalent (golden-trace exit) |
+| ExitReason | Golden-trace exit |
 |---|---|
 | `DURATION` | target duration reached (`duration`) |
 | `CYCLE_COMPLETE` | dynamic plugin finished after its minimum (`cycle-complete`) |
-| `EMPTY` | first frame returned False (`empty`; `raised` when display() raised inside the executor) |
+| `EMPTY` | first frame returned False, or no plugin (`empty`; `raised` when display() raised inside the executor; `no-plugin`, `breaker`) |
 | `ERROR` | the dispatch itself raised (`error`) |
 | `DISPLAY_FALSE` | a later frame returned False (`display-false`) |
-| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `vegas-interrupt`, ...) |
+| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `live`, `wifi`, ...) |
+| `RELOAD` | a plugin reload is waiting: the screen ends early but counts as shown, and the rotation advances |
 
 `PREEMPTED` replaces the five `current_display_mode != active_mode` checks.
-The runner asks the Arbiter, at the throttled service points it already has,
-whether a Source in `plan.preemptible_by` now wants the panel.
+The runner asks its host at named service points (`Checkpoint`): `FRAME`
+after each frame (and when a socket command wakes the 1 Hz wait),
+`AFTER_LOOP` / `AFTER_COMPLETED_LOOP` when the frame loop ends,
+`after_dwell` after the make-up dwell, and `FINAL` before the rotation
+advances. Each is one `decide(..., running=plan)` call
+(`DisplayController._screen_check`). The checkpoint says whether a pending
+reload counts there and when the WiFi notice file is read (`NoticeRead`):
+the read is throttled to once a second and deletes an expired file, so it
+happens exactly where the loop always read it.
 
-`FrameClock` provides `now()` and `sleep()`. In production it is
-`time.monotonic`/`time.sleep`. In the golden traces it is the fake clock
-that the harness patches in today.
+In the 125 Hz loop the live-priority scan is made before the frame's sleep
+(`_screen_service`), at the moments it always was, and weighed by the
+service point after the sleep, where the loop always decided to end the
+screen.
+
+`FrameClock` provides `time()`, `perf_counter()` and `sleep()`, the shape of
+the `time` module. In production it is `_ModuleClock`, which looks up
+`src.display_controller.time` on each call, so the golden traces' fake clock
+drives the runner as it drove the inline loops. The runner's log lines use
+the controller's logger, so they keep their source in the journal.
 
 ## Stages
 
@@ -181,13 +248,14 @@ that the harness patches in today.
   - dynamic duration (cycle complete, plugin cap, global cap)
   - live priority taking over and handing back; live round-robin
   - on-demand start/stop/expiry; pinned on-demand; a session resumed after
-    a restart
+    a restart, and one that cannot resume (its plugin did not load); a
+    request naming a live mode the plugin's live check would drop
   - schedule off and dim, with an on-demand override during downtime
   - WiFi notice; sync follower
   - Vegas, with and without `live_in_ticker`
 - Each trace row is `[start, mode, duration, exit_reason, frames,
   force_clear]`. The exit reason is the event that decided what came next.
-- All 16 tests run in under a second. The goldens were generated from
+- All 18 tests run in under a second. The goldens were generated from
   main's `run()` before any code moved.
 - Vegas uses `FakeVegas`, which implements only the contract the controller
   depends on: `run_iteration()` returns True after its duration and False
@@ -249,37 +317,84 @@ What shipped:
   Source, the dwells, the expiry comparison, the snapshot's reads, each
   dispatch in `run()`); every one failed a test.
 
-### Stage 3: ScreenRunner and `PREEMPTED`
+### Stage 3: ScreenRunner and `PREEMPTED` (done; awaiting the ledpi soak)
 
-Move the two frame loops, the make-up dwell and the dynamic-duration exit
-into `ScreenRunner.run(plan)` with an injected `FrameClock`. Replace the
-five re-checks with `PREEMPTED`. Add the OnDemand, Live and Rotation Sources
-so `LEGACY` is left meaning only Vegas.
-
-Concretely, from where stage 2 left off:
+The plan, from where stage 2 left off:
 
 1. `ArbiterState` gains the rotation index, the on-demand mode list, index,
-   expiry and pin, and the live resume point (today `current_mode_index`,
-   `on_demand_*` and the live-priority stash). `ArbiterInputs` gains the
-   live modes (`_collect_live_modes`) and whether Vegas is enabled and keeps
-   live content in the ticker.
-2. OnDemand returns its current mode with `_clamp_to_on_demand`'s bound,
-   reading `now` for the expiry. Live returns the next live mode
-   (round-robin). Rotation returns `available_modes[current_mode_index]`.
-   `ScreenPlan` gains `mode`, `plugin`, `min_duration`, `max_duration`,
-   `dynamic`, `frame_policy` and `preemptible_by`.
-3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(plan,
-   outcome)` replaces `_advance_after_screen` and the live-resume
-   bookkeeping. Each mid-screen check asks `decide()` whether a Source in
-   `plan.preemptible_by` now wins, so `_screen_preempted`,
-   `_check_live_takeover` and `_wifi_notice_pending` become one call.
+   expiry and pin, and the live resume point. `ArbiterInputs` gains the
+   live modes and whether Vegas is enabled and keeps live content in the
+   ticker.
+2. OnDemand returns its current mode with the session's bound, reading
+   `now` for the expiry. Live returns the next live mode (round-robin).
+   Rotation returns the rotation's mode. `ScreenPlan` gains `mode`,
+   `plugin`, `min_duration`, `max_duration`, `dynamic`, `frame_policy` and
+   `preemptible_by`.
+3. `ScreenRunner.run(plan)` returns an `ExitReason`; `state.after(outcome)`
+   replaces `_advance_after_screen`'s step and the live-resume bookkeeping.
+   Each mid-screen check asks `decide()` whether a Source in
+   `plan.preemptible_by` now wins.
 4. The control socket (`_drain_control_commands`, `_wait_for_control`) and
    state publishing stay where they are; the runner calls them at its
    service points.
 
+What shipped, one commit each: the runner; then the OnDemand, Live and
+Rotation Sources; then one `decide()` call at the service points.
+
+- `src/screen_runner.py` (on the mypy ratchet): `ScreenRunner`,
+  `FrameClock`, `ExitReason`, `Outcome`, `Checkpoint`, `NoticeRead`,
+  `Screen` and the `ScreenHost` protocol, which `DisplayController`
+  implements through `_ScreenHost` (one-line forwards to its own methods).
+  The two frame loops, the make-up dwell and the dynamic-duration exit
+  moved in unchanged, pacing included.
+- `src/display_arbiter.py`: `Source` gains `ON_DEMAND`, `LIVE`, `ROTATION`
+  and `RELOAD`; `LEGACY` means only Vegas. `FramePolicy`. `ArbiterState`
+  and `ArbiterInputs` as listed under "Arbiter". The pure helpers
+  `on_demand_bound` (`_clamp_to_on_demand`), `live_pick`
+  (`_check_live_priority`'s pick), `live_takeover` (the mid-screen claim)
+  and `rotation_plan`.
+- A pass asks `decide()` twice: once with the inputs every pass reads, and
+  once, only when nothing above the notice took the panel, with the Vegas
+  check and the live scan, read where run() always read them (the scan
+  asks every live-priority plugin, and the Vegas check applies queued
+  Vegas config, so reading them earlier, on a follower or notice pass,
+  would be a change). A Vegas iteration that yields asks a third time.
+- `_resolve_active_mode`, `_clamp_to_on_demand` and `_screen_preempted` are
+  gone. `_apply_live_priority`, `_check_live_priority`,
+  `_check_live_takeover` and `_wifi_notice_pending` remain (Vegas, the
+  dwell sleep and the tests call them), built on the same pure rules.
+- `_sleep_with_plugin_updates` keeps its own break rules. It also serves
+  the blank, the notice and the idle wait, which are not screens, and its
+  rules are edge-triggered (an on-demand session starting on the mode
+  already showing ends a dwell but not a frame loop); folding them into
+  `decide()` would change behaviour.
+
+Behaviour, checked three ways:
+
+- Golden traces: unchanged, no regeneration.
+- Every harness run in the suite (67: the goldens plus the live-takeover,
+  WiFi+live, socket-wake, plugin-reload, schedule and tick tests) was
+  captured with every sleep, `display()` call, WiFi read, live scan,
+  publish, dwell and scroll-state call logged, and diffed against
+  `origin/main`. Identical, except:
+  - a Vegas pass used to scan the live plugins twice at the same instant
+    (step 7, then step 8's "is anything live?"); it scans once;
+  - `_apply_live_priority(None)` calls that changed nothing are not made;
+  - throttled WiFi reads that returned the cached answer (no side effect)
+    after a notice had already ended the screen are not made;
+  - in the 125 Hz loop the live scan still runs before the frame's sleep,
+    but the claim is made by the service point after it, so the "live"
+    state change happens 8 ms later. The screen ends at the same frame as
+    before.
+- Tables: `test/test_display_arbiter.py` (OnDemand, Live, Vegas/Rotation,
+  `after`, the 24-row mid-screen table, `live_takeover`) and
+  `test/test_screen_runner.py` (the runner on a scripted host and fake
+  clock; the controller's service point and the reads it makes). A
+  mutation run broke each moved or new piece once; see the PR.
+
 This stage touches frame pacing (the 8 ms deadline sleep, the 1 ms yield),
-so it needs a frame soak on ledpi, A/B against main. Coordinate with
-whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
+so it needs a frame soak on ledpi, A/B against main, before it merges.
+Coordinate with whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
 
 ### Stage 4: Vegas as a Source
 
@@ -344,3 +459,13 @@ PR that updates the affected trace and explains why. All six are fixed:
 
 A new one found later goes the same way: record it here with the trace that
 shows it, then fix it in its own PR, not inside a restructure stage.
+
+Open:
+
+- Vegas stops for a sync follower (its interrupt check includes
+  `is_follower_active`), but the yield path never looks at a follower, so a
+  full rotation screen (20 s in the test) runs before the next pass hands
+  the panel to the leader. Found by stage 3's mutation run;
+  `test_screen_runner.py::TestThroughRun::test_vegas_yielding_to_a_follower_shows_a_rotation_screen_first`
+  pins it. Stage 4, which drops the interrupt callback, is the natural
+  place to fix it.

docs/SCROLL_PERFORMANCE.md

@@ -246,8 +246,16 @@ mean exactly 10 ms, so a ticker stalling on half its frames still averages to a
 healthy 100 fps. The stats line reports the tail for that reason — read the
 percentiles, not the fps.
 
-Every scroller emits one line every 5 seconds covering *every* frame in that
-window, tagged with the plugin it came from:
+Every scroller summarises each 5-second window, covering *every* frame in it,
+in one line tagged with the plugin it came from. At the default log level the
+line reaches the journal only when it is worth reading: a **degraded** window
+(frame rate below 90% of the rate the window was locked to, i.e. 1 / its own
+median -- the same 0.9 Vegas's `Vegas FPS` line uses -- or more than 1% of its
+frames stalled), the first window after one (the recovery), and otherwise once
+every 5 minutes per scroller as a heartbeat, so silence means stopped rather
+than fine. Every window is logged at DEBUG: to see them all, run the display
+with `-d` or `LEDMATRIX_DEBUG=true` (see
+[CONFIG_DEBUGGING.md](CONFIG_DEBUGGING.md#enable-debug-logging)).
 
 ```bash
 journalctl -u ledmatrix --since "-10min" --no-pager | grep "Scroll frame stats"
@@ -286,6 +294,10 @@ journalctl -u ledmatrix --since "-3h" --no-pager | grep "Scroll frame stats" \
   | sort -k7 -rn
 ```
 
+At the default log level that ranks the windows the journal kept -- the
+degraded ones, recoveries and heartbeats -- so it over-weights bad windows;
+rank a debug run for an unbiased average, or soak the rig (below).
+
 The `$2 < 1000` guard drops windows whose median is a whole second or more.
 Those are not frames. Until the idle-gap fix in `log_frame_rate()`, the first
 frame of every scroll was timed against the end of the *previous* scroll, so

docs/SPORTS_UNIFICATION.md

@@ -87,10 +87,12 @@ more. Shared sports code lives in `src/common`:
 | `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
 | `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
 | `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
-| `sports_plugin_host.py` | next release | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
-| `sports_live_scroll.py` | next release | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
-| `sports_display_rules.py` | next release | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
-| `sports_font_path.py` | next release | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
+| `sports_plugin_host.py` | 3.8.0 | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
+| `sports_live_scroll.py` | 3.8.0 | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
+| `sports_display_rules.py` | 3.8.0 | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
+| `sports_font_path.py` | 3.8.0 | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
+| `sports_game_over.py` | 3.8.1 | `SportsGameOverMixin` — `_is_game_really_over`, with the `FINAL_PERIOD` seam (family 5) |
+| `sports_favorites.py` | 3.8.2 | `SportsFavoritesMixin`, `SportsUpcomingFavoritesMixin`, `SportsRecentFavoritesMixin` — `_is_favorite_game` and the favourites-only picks, on the `_favorite_key` seam (family 6) |
 
 Each is described in [src/common/README.md](../src/common/README.md).
 
@@ -102,7 +104,8 @@ modules taken from the plugin copies, each a **new module** rather than growth
 on an existing one: a plugin that deletes a method copy and relies on an older
 module having gained it fails at runtime with an `AttributeError`, while a
 missing module fails at load, where the version checks can see it.
-`sports_helpers.py` holds `_favorite_key`, the override point listed below.
+`sports_helpers.py` holds `_favorite_key`, the override point listed below;
+`sports_favorites.py` is what calls it.
 Each promoted module has a parity test that compares its bodies against the
 plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
 (`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
@@ -125,7 +128,7 @@ deprecation cycle.
 | `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
 | `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
 | `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
-| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
+| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching. `sports_favorites` compares it, and each `favorite_teams` entry, stripped and upper-cased; a `None` matches nothing | `game["<side>_abbr"]`. nrl returns the ESPN team id, `None` when it is missing |
 | `_config_schema_path()` | Plugin's `config_schema.json` — returning it routes `_get_layout_offset` through the `src.element_style` resolver (and gives it the defaults to compare against) | `None`, i.e. the classic inline `customization.layout` read |
 | `_font_root()` | Directory to resolve `assets/fonts` against | core install root |
 
@@ -134,8 +137,7 @@ constants rather than behavior:
 
 | Attribute | Meaning | Default |
 |---|---|---|
-| `FINAL_PERIOD` | Period at/after which a zero clock can mean "over" | `4` (hockey overrides to `3`) |
-| `CLOCK_COUNTS_DOWN` | Whether `0:00` means "expired" | `True` (soccer/afl/nrl override to `False` — their clocks count up, so `0:00` is kickoff) |
+| `FINAL_PERIOD` | Period from which a 0:00 clock ends a game (`sports_game_over`) | `None`: the clock never ends a game (afl, nrl, soccer, baseball, ufc). Hockey sets `3`; basketball, football and lacrosse `4` |
 | `COALESCE_SCORING_SEQUENCE` | Fold score increments arriving during an active celebration into that one celebration | `False` (football overrides to `True` — a touchdown lands as +6, then +1 for the extra point) |
 
 ### Why these are seams and not branches
@@ -146,11 +148,14 @@ so NRL matches favorites on team ID. Flattening every plugin to abbreviations
 would silently select the wrong club for NRL users. The base declares the seam,
 NRL fills it, and core never learns the string `"nrl"`.
 
-`CLOCK_COUNTS_DOWN` exists for the same reason in the opposite direction: a
+`FINAL_PERIOD` exists for the same reason in the opposite direction: a
 soccer clock reading `0:00` means the match has not kicked off, so running the
-clock-expiry branch there would evict live games.
+clock-expiry rule there would evict live games. Those sports declare `None`,
+and so do baseball (innings, not a clock) and ufc (a bout ends only on ESPN's
+final status). One attribute covers both questions, whether the clock can end
+a game and from which period, so no separate count-down flag was added.
 
-`COALESCE_SCORING_SEQUENCE` is the third of the same kind. In football one
+`COALESCE_SCORING_SEQUENCE` is another of the same kind. In football one
 scoring play arrives as two score updates, so the follow-up must be folded into
 the first celebration; in soccer two increments a few seconds apart are two real
 goals, and folding them would swallow one. Neither default is "right" — which is
@@ -298,6 +303,49 @@ Left in the plugins, though identical:
   renderers) is already core's, in `SportsHelpersMixin`; a renderer that
   wants it can inherit that.
 
+### Family 5: the game-over check (core done; adoption waits for a release)
+
+The pilot of the method below. ledmatrix-plugins `scripts/test_game_over_check.py`
+(#621) pinned 3,115 answers across the nine plugins first; the reconcile
+(ledmatrix-plugins #625) made the five bodies one and
+changed only the cells the owner's decisions under
+[Product decisions](#product-decisions-each-family-needs) explain: ufc's
+clock rule (65 cells), baseball's dormant one (53, every one a game with a
+`period` baseball's games never carry), and a level score at 0:00 (five
+cells in hockey, basketball, football and lacrosse). The harness renders
+were pixel-identical. `src/common/sports_game_over.py` holds the body;
+`test/test_sports_game_over_parity.py` compares it, and each plugin's
+`FINAL_PERIOD`, with the plugin copies.
+
+### Family 6: favourite matching (core done; adoption waits for a release)
+
+ledmatrix-plugins `scripts/test_favourite_matching.py` (#634) pinned 204 rows
+across the nine plugins first: `_is_favorite_game` on each manager role, the
+two selection methods, the real `update()` with favourites-only on and off,
+and the INFO summary; the reconcile extends it to 217 (a lower-case and a
+padded favourite through `update()`, and the live favourite boost). The reconcile (ledmatrix-plugins
+#635) made `_is_favorite_game` one body on `SportsCore`
+(afl and soccer's `SportsUpcoming` copies and five `SportsLive` copies, all
+redundant, are gone), added `_favorite_code` beside it, and gave nrl a
+`_favorite_key` override instead of its own copies. So that a lower-case
+favourite works on a favourites-only Upcoming board, the Upcoming `update()`'s
+favourites-only pre-filter and the basketball, hockey and lacrosse live boost
+now ask `_is_favorite_game` too (a one-line change each; `update()` itself is
+family 13). Of 3,897 cells only those the decisions above explain changed:
+case and spaces in eight plugins (30-40 each), the id-less duplicate fix (6-8
+each), nrl's key (6) and its "None" match (6), and the INFO line in baseball,
+football and ufc. The harness renders were byte-identical. `src/common/sports_favorites.py` holds the
+bodies, one mixin per carrying class; `test/test_sports_favorites_parity.py`
+compares them with the plugin copies and checks that only nrl overrides
+`_favorite_key`.
+
+Left for later families: the live screens' favourites-only filter
+(`_classify_live_game` and its inline copies) and favourites-first sort still
+compare abbreviations exactly, and
+`SportsCoreSharedMixin._round_robin_favorites` groups favourites by raw
+abbreviation (or by `_team_in` where a plugin has one) instead of through
+`_favorite_key`. The result-colour helpers also wait (decision above).
+
 ### Why the method changes
 
 Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins
@@ -333,8 +381,8 @@ game-over check); the report measures each method in it. The procedure:
      line in each plugin.
    - *A per-sport fact* (hockey ends in period 3; a soccer clock counts up).
      Make it a declared class constant or override point with a default, as
-     `FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`, `COALESCE_SCORING_SEQUENCE` and
-     `_favorite_key` are, and add it to the tables above. Never a sport-name
+     `FINAL_PERIOD`, `COALESCE_SCORING_SEQUENCE` and `_favorite_key` are,
+     and add it to the tables above. Never a sport-name
      branch: core must not learn sport names.
    - *A product difference*: anything a user can see (which games show, a
      colour, a date, a badge, how long a screen stays). The owner picks the
@@ -382,8 +430,8 @@ release.
 | # | Family | Methods (variants) | Why here |
 |---|---|---|---|
 | 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Core side done; see [Stage 4](#stage-4-the-identical-sweep-core-done-adoption-waits-for-a-release) |
-| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; its seams (`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`) were designed in B1. The pilot for the procedure |
-| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
+| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; one seam, `FINAL_PERIOD`. The pilot for the procedure. Reconciled to one body and promoted as `sports_game_over`; adoption waits for the release that ships it. See [Family 5](#family-5-the-game-over-check-core-done-adoption-waits-for-a-release) |
+| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam. Reconciled to one body each and promoted as `sports_favorites`; adoption waits for the release that ships it. See [Family 6](#family-6-favourite-matching-core-done-adoption-waits-for-a-release) |
 | 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
 | 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
 | 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
@@ -416,21 +464,28 @@ family 9 prepares.
 Owner calls to make before (or while) reconciling. Items marked *verify* are
 suspected behaviour that needs a payload or a rig to confirm first.
 
-- **5, game-over check.** Which rule each sport gets: the clock never ends a
-  game in afl, nrl and soccer (`CLOCK_COUNTS_DOWN = False`); hockey ends at
-  0:00 from period 3, basketball, football and lacrosse from period 4.
-  baseball and ufc share a copy that reads a missing clock as "0:00": dormant
-  in baseball (its games carry no `period`), and not triggered by ufc's round
-  breaks either. ESPN sends a break as `STATUS_END_OF_ROUND` with displayClock
-  `-`, not `0:00` (verified against recorded payloads; ledmatrix-plugins#580
-  pins it). Whatever rule ufc gets must not read `-` as `0:00`. Decide ufc's
-  rule: no clock rule (ESPN's `STATUS_FINAL` is the only end signal it needs;
-  this also closes a ~1 s window at the horn when the ticking clock reads
-  `0:00`), or its own final period.
-- **6, favourite matching.** NRL keeps matching favourites by team id
-  (abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
-  own copies of the selection methods. Six plugins log the recent-games
-  selection at INFO; baseball, football and ufc do not.
+- **5, game-over check. Decided 2026-10-05, done:** one seam,
+  `FINAL_PERIOD`: hockey 3; basketball, football and lacrosse 4; `None` (the
+  clock never ends a game) for afl, nrl and soccer (clocks that count up),
+  baseball (its games carry no `period`, so the old rule was dormant) and
+  ufc (a bout ends only on ESPN's final status, which also closes the ~1 s
+  window at the horn when the ticking clock reads `0:00`; ESPN's round-break
+  displayClock `-` was never a zero clock, ledmatrix-plugins#580). Only a
+  non-empty clock string counts (the baseball/ufc copy read a missing clock
+  as `0:00`). A score level at 0:00 is not over: the game stays live through
+  the break before overtime, and one that really ends tied ends on its final
+  status. Baseball keeps its postponed/suspended override in `BaseballLive`.
+- **6, favourite matching. Decided 2026-10-05, done:** each side of a game is
+  named by `_favorite_key` (the abbreviation; NRL overrides it with the ESPN
+  team id, and `None` for a missing id, which fixes a favourite typed "None"
+  matching every game without one) and compared with `favorite_teams`
+  stripped and upper-cased, so " bos" matches BOS. NRL's ambiguous "NEW"
+  still matches nothing and is logged; routing the result-colour helpers
+  (`side_is_favorite`, which tint both NEW clubs) through `_favorite_key` is
+  left for a later family. The recent-games selection logs at INFO in all
+  nine. ufc stays on the shared body, dormant: its favourites are fighters,
+  which its MMA managers match themselves (a follow-up). Fix ported: only a
+  game with an id can be a duplicate in the selection methods.
 - **7, other-games rotation.** football advances the rotation window under
   `_games_lock` (update() and display() both advance it; interleaved, a
   window of games is skipped) and fixes a favourites-only pool that recomposed

docs/TROUBLESHOOTING.md

@@ -84,6 +84,46 @@ python3 web_interface/start.py
 
 ### Installation & Build Issues
 
+#### "This version of Raspberry Pi OS is not supported"
+
+LEDMatrix installs on Raspberry Pi OS Lite **Trixie** (Debian 13, Python
+3.13) or **Bookworm** (Debian 12, Python 3.11). The installer checks
+`/etc/os-release` before it changes anything and stops on anything else.
+
+**Check what you have:**
+```bash
+grep -E '^(PRETTY_NAME|VERSION_ID)=' /etc/os-release
+python3 --version
+```
+
+**Solutions:**
+- `VERSION_ID="11"` (Bullseye) or older: flash a new card with Raspberry Pi
+  Imager, choosing Raspberry Pi OS Lite (64-bit). Trixie is recommended;
+  Bookworm (Legacy) also works. An in-place upgrade from Bullseye is not
+  supported by Raspberry Pi and is not worth the risk.
+- "A desktop is running": use the Lite image, not the desktop one, or boot
+  to the console with `sudo systemctl set-default multi-user.target` and
+  reboot. Desktop packages that are installed but not running only produce a
+  warning, and the install continues.
+- "python3 is Python 3.x; LEDMatrix needs Python 3.11 or newer": something
+  has replaced the system `python3`. Point it back at the OS's own Python
+  (`/usr/bin/python3` should be 3.11 on Bookworm, 3.13 on Trixie).
+- `sudo bash scripts/check_system_compatibility.sh` runs the same checks
+  without installing anything.
+
+#### "This Pi manages its network with dhcpcd, not NetworkManager"
+
+A warning, not an error: the install carries on and the display works. But
+choosing a WiFi network from the web page and the `LEDMatrix-Setup` hotspot
+both need NetworkManager, the default on Bookworm and Trixie. It appears
+when dhcpcd was selected in `raspi-config`. Switch back with a keyboard and
+screen attached (or over Ethernet), since the WiFi connection drops briefly:
+
+```bash
+sudo raspi-config   # Advanced Options -> Network Config -> NetworkManager
+sudo reboot
+```
+
 #### Step 6 fails: "Failed building wheel for rgbmatrix"
 
 **Symptoms:**
@@ -328,6 +368,49 @@ commit, then switches to releases on its own.
 3. **Local changes after a channel switch:** edits that no longer fit the new
    version are kept in the git stash rather than lost; `git stash list`
    shows them as "LEDMatrix autostash before update".
+4. **A new install is on a release, not `main`.** The one-shot installer
+   checks out the newest release. For the newest code instead, install with
+   `LEDMATRIX_CHANNEL=beta`:
+   ```bash
+   curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | LEDMATRIX_CHANNEL=beta bash
+   ```
+
+---
+
+#### Issue: "service settings ... are not applied yet" after an update
+
+**Symptoms:**
+- Update Code's message, or the web interface log, says an update changes
+  service settings that are not applied yet, and to run the installer
+- The display logs `ledmatrix.service differs from systemd/ledmatrix.service`
+  at startup
+
+**Explanation:** updates install the systemd units a new version changes
+through the root helper `/usr/local/sbin/ledmatrix-refresh-units`, which the
+installer sets up and grants to the web user in
+`/etc/sudoers.d/ledmatrix_web`. A device installed before that has neither,
+so the new unit settings (for example the display's watchdog) wait for a
+reinstall. The update itself is fine.
+
+**Solution:** re-run the installer once, as root:
+```bash
+cd ~/LEDMatrix
+sudo ./first_time_install.sh
+# or, lighter: install the units and helper, then the sudo rules
+sudo ./scripts/install/install_service.sh
+./scripts/install/configure_web_sudo.sh
+```
+Check it worked:
+```bash
+ls -l /usr/local/sbin/ledmatrix-refresh-units   # root root, rwxr-xr-x
+sudo -l | grep ledmatrix-refresh-units           # the two rules
+```
+A message that the helper **refused** a unit (`refusing to install it`)
+means a template in `systemd/` was edited so that it would run as another
+account or from another folder. The message names the template. Look at
+what changed with `git diff -- systemd/`, save any edit you want to keep,
+then restore only that file, for example
+`git checkout -- systemd/ledmatrix-web.service`.
 
 ---
 
@@ -364,9 +447,16 @@ commit, then switches to releases on its own.
 
 5. **Check required services:**
    ```bash
+   systemctl is-active NetworkManager   # must say "active"
    sudo systemctl status hostapd
    sudo systemctl status dnsmasq
    ```
+   On a fresh install `hostapd` shows as **masked**. That is expected, on
+   Bookworm and Trixie alike: Debian's hostapd package masks the service
+   when it is installed without a configuration, so the hotspot is brought
+   up through NetworkManager instead (look for `nmcli hotspot fallback` in
+   `journalctl -u ledmatrix-wifi-monitor`). If NetworkManager is not
+   active, see "This Pi manages its network with dhcpcd" above.
 
 6. **Manually enable AP mode:**
    ```bash
@@ -590,9 +680,10 @@ stack into the log, so it says which plugin was stuck.
    apart, so a plugin that hangs on every start does not restart the display
    hundreds of times an hour.
 
-4. **Is the watchdog installed?** Installs from before it keep their old unit
-   until the installer is re-run (a startup warning says the unit differs
-   from its template):
+4. **Is the watchdog installed?** Updates install new unit settings once the
+   installer has set up `ledmatrix-refresh-units`; installs from before that
+   keep their old unit until the installer is re-run (a startup warning says
+   the unit differs from its template):
    ```bash
    systemctl show -p WatchdogUSec ledmatrix   # 2min once running; 0 = not installed
    sudo ./scripts/install/install_service.sh

docs/WEB_FRONTEND_ARCHITECTURE.md

@@ -42,7 +42,8 @@ static/v3/js/
     registry.js         page lifecycle: init/destroy on htmx swaps
     api.js              fetch wrapper for /api/v3 (JSON envelope, login redirect)
     facade.js           window.LEDMatrix and deprecated aliases
-    (later) escape.js, notify.js, dialog.js, streams.js, visibility.js,
+    visibility.js       ctx.visibility: a page's timers run only while it is on screen
+    (later) escape.js, notify.js, dialog.js, streams.js,
             store.js (the one installed-plugin store), form/renderer.js
   pages/                one module per tab partial
     cache.js            export init(root, ctx), destroy(root, ctx)
@@ -81,6 +82,20 @@ The conventions the converted pages share:
   to the module's export of the same name and warns once.
 - **Timers are cleared in `destroy()`**, the one thing `ctx.signal` cannot
   undo by itself.
+- **Polling goes through `ctx.visibility`.** A refresh that repeats
+  (`ctx.visibility.every(ms, fn)`) or work that should run only while the
+  page is on screen (`ctx.visibility.whileVisible(start, stop)`) is
+  registered there, never with a bare `setInterval`. It runs only while the
+  page's tab is the active tab and the browser tab is visible, and it ends
+  when the page is destroyed, with no code in `destroy()`.
+- **A page reports its own htmx saves.** A form whose result a page module
+  shows (an `htmx:afterRequest` listener on the page root, in place of an
+  `hx-on` attribute naming a global) carries `data-reports-result`. `app.js`
+  then leaves the server's message to the page, as it does for a form with
+  an `hx-on` after-request handler, so a save shows one notification.
+- **Server data for the module goes in `data-*` attributes**, as JSON where
+  it is structured (`data-schedule-config='{{ schedule_config | tojson }}'`),
+  not templated into a script.
 
 `core/registry.js` handles the rest:
 
@@ -103,6 +118,7 @@ Each mount gets a `ctx` object:
 | `ctx.state` | A per-mount object for the page's own state |
 | `ctx.api` | Shared service from `boot.js` |
 | `ctx.notify` | Shared service from `boot.js` |
+| `ctx.visibility` | This page's handle on `core/visibility.js` (below), made per mount by `boot.js` through the registry's `mountContext` option |
 
 A page that passes `{ signal: ctx.signal }` to `addEventListener` and
 `fetch` needs no teardown code. Its listeners and in-flight requests go
@@ -111,6 +127,30 @@ example: its delete buttons use one delegated listener, rows are built with
 `textContent` rather than markup strings, and a newer load supersedes an
 older one.
 
+### Page visibility
+
+`core/visibility.js` gives each mounted page `ctx.visibility`:
+
+| Member | What it does |
+|---|---|
+| `whileVisible(start, stop)` | Runs `start()` when the page comes on screen (at once, if it mounts on screen) and `stop()` when it leaves. Returns a function that ends the registration, running `stop()` first if needed |
+| `every(ms, fn)` | `fn()` at once, then every `ms` while on screen. The interval is cleared while hidden and restarted, with an immediate `fn()`, when the page is back. Returns the same kind of end function |
+| `isVisible()` | True while the page is on screen |
+| `tab` | The tab the page belongs to: its name, or `forPage(ctx, { tab })` |
+
+"On screen" means the page's tab is the active tab and the browser tab is
+visible. Everything a page registered ends when its `ctx.signal` aborts,
+after `destroy()`, so a swapped-out partial leaves no interval behind.
+
+The answer comes from `window.LEDVisibility` (`app-shell.js`), read at call
+time, so the page modules and the classic partials that still call it
+(Overview, Logs, Tools) agree on the active tab, and the SSE streams keep
+pausing with them. Each registration takes its own `LEDVisibility` key, so
+registrations never replace each other or a classic partial's. Without
+`LEDVisibility` (a page outside `base.html`), the browser tab's visibility
+alone decides. Moving the tracker itself into the module (the shell table
+below) changes only `core/visibility.js`.
+
 ### One facade
 
 `window.LEDMatrix` is the only global the module code adds:
@@ -239,9 +279,9 @@ are the inline script in each partial today.
 | 3 | Operation History | 293 lines, now 0 | **Done in stage 2.** Read-only list; rows drawn with `textContent`, the search debounce cleared on destroy. The "Showing x to y" counters now also reset when nothing matches |
 | 4 | Config Editor (`raw_json.html`) | 212 lines, now 0 | **Done in stage 2.** Plain textareas (no CodeMirror on this page). It defined 5 globals after all (`formatJson`, `manualValidateJson`, `validateJSON`, `saveMainConfig`, `saveSecretsConfig`); nothing else used them, and they are deprecated aliases now. The live "Invalid JSON" line no longer puts the parser's message into `innerHTML` |
 | 5 | Backup & Restore | 232 lines, now 0 | **Done in stage 2.** Its 5 globals (`exportBackup`, `loadBackupList`, `validateRestoreFile`, `clearRestore`, `runRestore`) are deprecated aliases; the buttons are delegated `data-action`s. Uploads go through `ctx.api.request(..., { body: formData })` (`api.js` gained a raw `body` option) |
-| 6 | Schedule | 193 | 2 globals used as `hx-on` response handlers. Moves `hx-on` handlers into page listeners |
-| 7 | General | 147 | `webLogin` global and the security section. The first page that touches login |
-| 8 | Display | 231 | First page with `LEDVisibility` timers: those move to a `ctx.visibility` service that stops on destroy |
+| 6 | Schedule | 193 lines, now 0 | **Done in stage 3.** Its 2 `hx-on` response handlers (`handleScheduleResponse`, `handleDimScheduleResponse`) are one `htmx:afterRequest` listener on the page root, and deprecated aliases. The forms are marked `data-reports-result` so `app.js` does not repeat the server's message. The saved schedules reach the module as JSON in `data-schedule-config` / `data-dim-schedule-config` instead of being templated into the script |
+| 7 | General | 153 lines, now 0 | **Done in stage 3.** The Security section's three forms and two buttons are delegated `data-action`s (one submit and one click listener); `window.webLogin` is a deprecated alias of an object with its five methods. Login requests go through `ctx.api`, so the login redirect is quiet. The settings form keeps its `hx-on` call to the shared `showSaveResult`, as Rotation's does |
+| 8 | Display | 292 lines (2 scripts), now 0 | **Done in stage 4.** The first page with a timer: the 5 s multi-display sync poll is `ctx.visibility.every(5000, ...)` (above), so it runs only while the tab is on screen and stops when the partial is swapped out. Its one global, `updateSyncUI` (the Role menu's `onchange`), is a deprecated alias; the Advanced section's `onclick` is a delegated `data-action="toggle-section"` that calls the shared `toggleSection`. The status poll and the scroll-speed hint go through `ctx.api` with `ctx.signal`, as does the Vegas order widget's plugin-list request. The settings form keeps its `hx-on` call to `showSaveResult` and its `onsubmit` call to `fixInvalidNumberInputs`, as Rotation's does |
 | 9 | Overview | 410 (4 scripts) | First-run surface: Getting Started, update banner, live preview. Five globals |
 | 10 | WiFi | 364 | `x-data="wifiSetup()"` is defined by its own script. Moves to `Alpine.data()` registered from the module. AP-mode first screen, so it needs the AP-mode test on a real device |
 | 11 | Fonts | 681 | Large, but self-contained (6 globals) |
@@ -258,7 +298,7 @@ the order:
 | `showNotification` | 4 versions | `core/notify.js` |
 | The modal helper | `utils/dialog.js` | `core/dialog.js` |
 | SSE streams | `app-shell.js` | `core/streams.js` |
-| `LEDVisibility` | `app-shell.js` | `core/visibility.js` |
+| `LEDVisibility` | `app-shell.js` | `core/visibility.js` (the page-facing `ctx.visibility` is there since step 8; it reads the tracker from `app-shell.js`) |
 
 Each move leaves the old global as an alias. When the last inline script is
 gone, the script re-execution in `htmx-config.js` and the "HTMX never
@@ -278,12 +318,16 @@ Unit suites need only node. They import the shipped modules directly:
 
 | Suite | Kind | What it covers |
 |---|---|---|
-| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment |
+| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment, `mountContext` fields per mount |
+| `dom/test_visibility_service.js` | DOM: real `LEDVisibility` from `app-shell.js`, real registry, no server | `whileVisible` and `every` start and stop with the active tab and the browser tab's visibility; no interval runs while hidden or after a swap-out; one interval after five swaps; registrations never replace each other or a classic partial's; a destroyed page registers nothing; a throwing `start()` is contained; the no-`LEDVisibility` fallback |
 | `unit/test_core_modules.js` | Unit | `api.js` (envelope, errors, abort, login redirect, path check) and `facade.js` (facade, aliases) |
 | `dom/test_cache_page.js` | DOM: real partial, real API shape | No inline script; one request per swap and per Refresh after five swaps; a cancelled request draws nothing; hostile keys stay text; delete, empty, error, network and login states |
 | `dom/test_durations_page.js` | DOM: real partial, real widget, real API shape | One plugin-list request per swap; Move down moves one place after five swaps; the swap cancels a request in flight; a late-loading widget is waited for, and a page swapped away while waiting starts nothing; hostile names stay text |
 | `dom/test_operation_history_page.js` | DOM: real partial, real API shape | One history request per swap and per Refresh; the plugin filter filled once (from `PluginAPI`'s cache when loaded); paging, filters, debounced search, Clear (one DELETE), error/network/login states, cancel on swap; hostile ids, users and errors stay text |
 | `dom/test_raw_json_page.js` | DOM: real partial, real config | One POST per Save after five swaps, to the right file; Format and Validate act once; invalid JSON never sent and its message stays text; a save survives a swap and is still reported; the old globals' entry points |
+| `dom/test_schedule_page.js` | DOM: real partial, real widget | Both pickers drawn once per swap from the saved config; after five swaps each form's answer is one notification (message, fallback, refused, non-JSON, `null`), a request from outside the forms none; the brightness label; a late widget waited for, a page swapped away while waiting draws nothing; the old globals' entry points |
+| `dom/test_display_page.js` | DOM: real partial, real widget, real `LEDVisibility`, real API shape | After five swaps one page, one sync interval, the Vegas order drawn once and each control acting once (brightness, resolution, the two show/hide toggles, the Advanced toggle, one debounced hint request); the sync poll only while on screen and never after a swap-out; sync states and hostile peer names as text, failure and login answers; a late widget waited for; `updateSyncUI`'s entry point |
+| `dom/test_general_page.js` | DOM: real partial, real widget, real API shape | The timezone picker drawn once per swap with the saved zone; the settings form left to htmx; after five swaps each Security action makes one request (create, copy, revoke and its cancel, password and its mismatch); hostile token names stay text; refused, network and login answers; a create made before a swap is still reported and draws nothing; `webLogin`'s entry points |
 | `dom/test_backup_restore_page.js` | DOM: real partial, real API shape | One request per Refresh, Delete, Export (busy button ignores a second click), Inspect and Restore after five swaps; the upload's fields and the six restore options; reads cancelled by a swap, writes not; hostile file and host names stay text; the old globals' entry points |
 | `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; the converted pages are exactly the registered ones, each with its module, `init`, and one root in the rendered partial; a converted partial has no `<script>` and no `onclick`; every moved global is aliased in `boot.js` and exported by its module, and no template defines it any more |
 | `test/test_field_model_parity.py` | pytest | The model against the macro for every available schema |

first_time_install.sh

@@ -47,72 +47,133 @@ if echo "${DEVICE_MODEL:-}" | grep -qi "Raspberry Pi 5"; then
     echo "Raspberry Pi 5 detected — will verify RP1 library support."
 fi
 
-# Check OS version - must be Raspberry Pi OS Lite (Trixie)
+# Check OS version - must be Raspberry Pi OS Lite, Bookworm or Trixie.
+# The rules live in scripts/install/lib_os.sh, shared with
+# scripts/check_system_compatibility.sh.
 echo ""
 echo "Checking operating system requirements..."
 echo "----------------------------------------"
 OS_CHECK_FAILED=0
+OS_RELEASE=""
 
-if [ -f /etc/os-release ]; then
-    . /etc/os-release
-    echo "Detected OS: $PRETTY_NAME"
-    echo "Version ID: ${VERSION_ID:-unknown}"
+OS_LIB="$(cd "$(dirname "$0")" && pwd)/scripts/install/lib_os.sh"
+if [ ! -f "$OS_LIB" ]; then
+    echo "✗ ERROR: $OS_LIB is missing, so the operating system cannot be checked."
+    echo "  Your LEDMatrix download is incomplete. Download it again and re-run this script:"
+    echo "    git clone https://github.com/ChuckBuilds/LEDMatrix.git"
+    exit 1
+fi
+# shellcheck source=scripts/install/lib_os.sh
+. "$OS_LIB"
 
-    # Check if it's Raspberry Pi OS or Debian
-    if [[ "$ID" != "raspbian" ]] && [[ "$ID" != "debian" ]]; then
-        echo "✗ ERROR: This script requires Raspberry Pi OS (raspbian/debian)"
-        echo "  Detected OS ID: $ID"
-        OS_CHECK_FAILED=1
-    fi
+if [ -r "$LM_OS_RELEASE_FILE" ]; then
+    echo "Detected OS: $(lm_os_field PRETTY_NAME)"
+    OS_VERSION_ID=$(lm_os_field VERSION_ID)
+    echo "Version ID: ${OS_VERSION_ID:-unknown}"
 
-    # Check if it's Debian 13 (Trixie)
-    if [ "${VERSION_ID:-0}" != "13" ]; then
-        echo "✗ ERROR: This script requires Raspberry Pi OS Lite (Trixie) - Debian 13"
-        echo "  Detected version: ${VERSION_ID:-unknown}"
-        echo "  Please upgrade to Raspberry Pi OS Lite (Trixie) before continuing"
-        OS_CHECK_FAILED=1
+    if OS_RELEASE=$(lm_os_release); then
+        echo "✓ $(lm_release_label "$OS_RELEASE") detected"
     else
-        echo "✓ Debian 13 (Trixie) detected"
+        OS_ID=$(lm_os_field ID)
+        if [[ "$OS_ID" != "raspbian" ]] && [[ "$OS_ID" != "debian" ]]; then
+            echo "✗ ERROR: This script requires Raspberry Pi OS (raspbian/debian)"
+            echo "  Detected OS ID: ${OS_ID:-unknown}"
+        else
+            echo "✗ ERROR: This version of Raspberry Pi OS is not supported"
+            echo "  Detected version: ${OS_VERSION_ID:-unknown}"
+            echo "  Supported: Trixie (Debian 13) and Bookworm (Debian 12)"
+        fi
+        OS_CHECK_FAILED=1
     fi
 
-    # Check if it's the Lite version (no desktop environment)
-    # Check for desktop packages or desktop services
-    DESKTOP_DETECTED=0
-    if dpkg -l | grep -qE "^ii.*raspberrypi-ui-mods|^ii.*lxde|^ii.*xfce|^ii.*gnome|^ii.*kde"; then
-        DESKTOP_DETECTED=1
-    fi
-    if systemctl list-units --type=service --state=running 2>/dev/null | grep -qE "lightdm|gdm3|sddm|lxdm"; then
-        DESKTOP_DETECTED=1
+    # Check for a desktop. A desktop only competes with the panel for CPU while
+    # it runs, so a running display manager stops the install; desktop packages
+    # or session files on a Pi that boots to the console are only a warning.
+    DESKTOP_RUNNING=0
+    DESKTOP_INSTALLED=0
+    # display-manager is the alias every Debian display manager registers.
+    for dm in display-manager lightdm gdm gdm3 sddm lxdm; do
+        if systemctl is-active --quiet "$dm" 2>/dev/null; then
+            DESKTOP_RUNNING=1
+        fi
+    done
+    # grep without -q: -q exits at the first match, dpkg then dies of SIGPIPE,
+    # and pipefail turns a found desktop into "not found".
+    # Desktop metapackages and session managers, matched as whole installed
+    # package names: an unanchored ".*kde" matched libblockdev-* ("bloc-kde-v"),
+    # and a "gnome" prefix matched standalone parts such as gnome-keyring.
+    # Trixie replaced raspberrypi-ui-mods with the rpd-*-core metapackages.
+    DESKTOP_PACKAGES='raspberrypi-ui-mods|rpd-wayland-core|rpd-x-core'
+    DESKTOP_PACKAGES+='|lxde|lxde-core|lxsession|xfce4|xfce4-session'
+    DESKTOP_PACKAGES+='|gnome-shell|gnome-session|kde-plasma-desktop|plasma-desktop'
+    DESKTOP_PACKAGES+='|plasma-workspace|task-desktop|task-[a-z0-9]+-desktop'
+    if dpkg-query -W -f='${db:Status-Abbrev} ${binary:Package}\n' 2>/dev/null \
+            | grep -E "^ii +(${DESKTOP_PACKAGES})(:[a-z0-9]+)?$" >/dev/null; then
+        DESKTOP_INSTALLED=1
     fi
     if [ -d /usr/share/raspberrypi-ui-mods ] || [ -d /usr/share/xsessions ]; then
-        DESKTOP_DETECTED=1
+        DESKTOP_INSTALLED=1
     fi
 
-    if [ "$DESKTOP_DETECTED" -eq 1 ]; then
-        echo "✗ ERROR: Desktop environment detected - this script requires Raspberry Pi OS Lite"
-        echo "  Please use Raspberry Pi OS Lite (not the full desktop version)"
+    if [ "$DESKTOP_RUNNING" -eq 1 ]; then
+        echo "✗ ERROR: A desktop is running - this script requires Raspberry Pi OS Lite"
+        echo "  Please use Raspberry Pi OS Lite (not the full desktop version), or boot"
+        echo "  to the console: sudo systemctl set-default multi-user.target && sudo reboot"
         OS_CHECK_FAILED=1
+    elif [ "$DESKTOP_INSTALLED" -eq 1 ]; then
+        echo "⚠ WARNING: Desktop packages are installed, but no desktop is running."
+        echo "  Continuing. Keep the Pi booting to the console: a running desktop"
+        echo "  competes with the LED panel for CPU and can make it flicker."
     else
         echo "✓ Lite version confirmed (no desktop environment)"
     fi
 else
-    echo "✗ ERROR: Could not detect OS version (/etc/os-release not found)"
+    echo "✗ ERROR: Could not detect OS version ($LM_OS_RELEASE_FILE not found)"
     OS_CHECK_FAILED=1
 fi
 
+# Python: whatever python3 the release ships (3.11 on Bookworm, 3.13 on
+# Trixie). Checked only when python3 is already there -- Step 1 installs it
+# otherwise, and on a supported release that brings the release's own version.
+if [ "$OS_CHECK_FAILED" -eq 0 ]; then
+    if PYTHON3_VERSION=$(lm_python_version); then
+        case "$(lm_python_check "$PYTHON3_VERSION")" in
+            ok)
+                echo "✓ Python $PYTHON3_VERSION detected"
+                ;;
+            too-old)
+                echo "✗ ERROR: python3 is Python $PYTHON3_VERSION; LEDMatrix needs Python 3.$LM_PYTHON_MIN_MINOR or newer"
+                echo "  $(lm_release_label "$OS_RELEASE") ships Python $(lm_release_python "$OS_RELEASE"). Something on this"
+                echo "  system has changed which Python 'python3' runs; point it back at the system Python."
+                OS_CHECK_FAILED=1
+                ;;
+            *)
+                echo "⚠ python3 is Python $PYTHON3_VERSION, which LEDMatrix has not been tested with"
+                echo "  (tested: 3.$LM_PYTHON_MIN_MINOR to 3.$LM_PYTHON_MAX_MINOR). Continuing anyway."
+                ;;
+        esac
+    else
+        echo "python3 not found yet; Step 1 installs it."
+    fi
+fi
+
 if [ "$OS_CHECK_FAILED" -eq 1 ]; then
     echo ""
-    echo "Installation cannot continue. Please install Raspberry Pi OS Lite (Trixie) and try again."
-    echo ""
-    echo "To install Raspberry Pi OS Lite (Trixie):"
-    echo "  1. Download from: https://www.raspberrypi.com/software/operating-systems/"
-    echo "  2. Select 'Raspberry Pi OS Lite (64-bit)' with Debian 13 (Trixie)"
-    echo "  3. Flash to SD card using Raspberry Pi Imager"
-    echo "  4. Boot and run this script again"
+    echo "Installation cannot continue."
+    lm_print_supported_os_help
     exit 1
 fi
 
 echo "✓ OS requirements met"
+
+# WiFi setup (the web page's WiFi tab and the LEDMatrix-Setup hotspot) needs
+# NetworkManager. Both releases use it by default; say so plainly if this Pi
+# does not, but carry on -- the display itself does not depend on it.
+case "$(lm_network_stack)" in
+    networkmanager) echo "✓ NetworkManager is managing the network" ;;
+    dhcpcd) lm_print_dhcpcd_advice ;;
+    *) echo "⚠ Could not tell which service manages the network; WiFi setup from the web page needs NetworkManager" ;;
+esac
 echo ""
 
 # The user who ran the installer: SUDO_USER once we are running under sudo
@@ -190,6 +251,51 @@ _sync_rgb_submodule() {
     fi
     return 0
 }
+
+# LEDMatrix's own changes to the library live in patches/rpi-rgb-led-matrix/ and
+# are applied only for the build: _apply_rgb_patches before it, _revert_rgb_patches
+# after it, success or not. The checkout is left exactly as it was, so `git pull`
+# and _sync_rgb_submodule never meet local modifications in the submodule.
+# A patch that no longer applies (a submodule bump, a hand-edited checkout) is
+# reported and skipped -- the unpatched library still builds and works, so it is
+# never fatal. One that is already applied is left alone and not reverted.
+_RGB_APPLIED_PATCHES=()
+
+_apply_rgb_patches() {
+    local sub="$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master"
+    local dir="$PROJECT_ROOT_DIR/patches/rpi-rgb-led-matrix" patch name
+    _RGB_APPLIED_PATCHES=()
+    [ -d "$dir" ] || return 0
+    for patch in "$dir"/*.patch; do
+        [ -f "$patch" ] || continue
+        name=$(basename "$patch")
+        if _git_as_repo_owner -C "$sub" apply --check "$patch" >/dev/null 2>&1; then
+            if _git_as_repo_owner -C "$sub" apply "$patch"; then
+                _RGB_APPLIED_PATCHES+=("$patch")
+                echo "Applied library patch $name"
+            else
+                echo "⚠ Could not apply library patch $name; building without it"
+            fi
+        elif _git_as_repo_owner -C "$sub" apply --reverse --check "$patch" >/dev/null 2>&1; then
+            echo "Library patch $name is already applied"
+        else
+            echo "⚠ Library patch $name does not apply to this checkout; building without it"
+        fi
+    done
+    return 0
+}
+
+_revert_rgb_patches() {
+    local sub="$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" i
+    # Last applied first, in case two patches touch the same file.
+    for ((i = ${#_RGB_APPLIED_PATCHES[@]} - 1; i >= 0; i--)); do
+        if ! _git_as_repo_owner -C "$sub" apply --reverse "${_RGB_APPLIED_PATCHES[i]}"; then
+            echo "⚠ Could not revert $(basename "${_RGB_APPLIED_PATCHES[i]}"); restore the checkout with: git -C $sub checkout -- ."
+        fi
+    done
+    _RGB_APPLIED_PATCHES=()
+    return 0
+}
 # --- end rpi-rgb-led-matrix checkout helpers ---------------------------------
 
 # Determine the Project Root Directory (where this script is located)
@@ -223,6 +329,8 @@ SKIP_SWAP=${LEDMATRIX_SKIP_SWAP:-0}
 BUILD_JOBS_OVERRIDE=${LEDMATRIX_BUILD_JOBS:-}
 # Weekly automatic updates: 1 on, 0 off, empty = ask (interactive) or leave as is.
 AUTO_UPDATE=${LEDMATRIX_AUTO_UPDATE:-}
+# Update channel written to config.json: stable, beta, or empty = leave as is.
+UPDATE_CHANNEL=$(printf '%s' "${LEDMATRIX_CHANNEL:-}" | tr '[:upper:]' '[:lower:]')
 
 usage() {
     cat <<USAGE
@@ -240,12 +348,18 @@ Options:
       --enable-auto-update  Turn on weekly automatic updates (with health
                             check and automatic rollback)
       --no-auto-update      Leave weekly automatic updates off
+      --beta                Follow main, the newest code (the beta update
+                            channel). Without it, updates follow releases
+                            (stable). It sets the channel; it does not move
+                            this checkout -- the one-shot installer picks the
+                            version, and so does the next update.
   -h, --help                Show this help message and exit
 
 Environment variables (same effect as flags):
   LEDMATRIX_ASSUME_YES=1, RPI_RGB_FORCE_REBUILD=1, LEDMATRIX_SKIP_SOUND=1,
   LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1,
-  LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N, LEDMATRIX_AUTO_UPDATE=1|0
+  LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N, LEDMATRIX_AUTO_UPDATE=1|0,
+  LEDMATRIX_CHANNEL=stable|beta
 
 Low-memory devices:
   On a Pi with under 2GB of RAM the C++ build is limited to fewer parallel
@@ -265,6 +379,7 @@ while [ $# -gt 0 ]; do
         --skip-swap) SKIP_SWAP=1 ;;
         --enable-auto-update) AUTO_UPDATE=1 ;;
         --no-auto-update) AUTO_UPDATE=0 ;;
+        --beta) UPDATE_CHANNEL=beta ;;
         --build-jobs)
             shift
             if [ $# -eq 0 ]; then echo "--build-jobs requires a number"; usage; exit 1; fi
@@ -291,10 +406,12 @@ else
     lm_remove_build_swap() { return 0; }
 fi
 
-# Remove the temporary build swapfile no matter how the script ends. Step 6
-# tears it down itself; this is the backstop for the error path, since
-# on_error ends in `exit` and EXIT traps still run.
-trap 'lm_remove_build_swap' EXIT
+# Remove the temporary build swapfile, and take any library patches back out
+# of the submodule, no matter how the script ends. Step 6 does both itself;
+# this is the backstop for the error path (on_error ends in `exit` and EXIT
+# traps still run) and for an interrupted build. _revert_rgb_patches only
+# touches patches it applied, so running it twice is harmless.
+trap 'lm_remove_build_swap; _revert_rgb_patches' EXIT
 
 # Helpers
 retry() {
@@ -873,15 +990,25 @@ if [ -z "$AUTO_UPDATE" ] && [ "$ASSUME_YES" != "1" ] && [ -t 0 ]; then
     echo
     if [[ $REPLY =~ ^[Yy]$ ]]; then AUTO_UPDATE=1; else AUTO_UPDATE=0; fi
 fi
-if [ "$AUTO_UPDATE" = "1" ] || [ "$AUTO_UPDATE" = "0" ]; then
-    if python3 - "$PROJECT_ROOT_DIR/config/config.json" "$AUTO_UPDATE" <<'PY'
+case "$UPDATE_CHANNEL" in
+    stable|beta|"") ;;
+    *) echo "⚠ LEDMATRIX_CHANNEL=$UPDATE_CHANNEL is not stable or beta; leaving the update channel as it is"
+       UPDATE_CHANNEL="" ;;
+esac
+# The update channel, likewise only when asked for (--beta / LEDMATRIX_CHANNEL).
+if [ "$AUTO_UPDATE" = "1" ] || [ "$AUTO_UPDATE" = "0" ] || [ -n "$UPDATE_CHANNEL" ]; then
+    if python3 - "$PROJECT_ROOT_DIR/config/config.json" "$AUTO_UPDATE" "$UPDATE_CHANNEL" <<'PY'
 import json, os, sys, tempfile
-path, enabled = sys.argv[1], sys.argv[2] == "1"
+path, enabled = sys.argv[1], sys.argv[2]
+channel = sys.argv[3] if len(sys.argv) > 3 else ""
 with open(path, encoding="utf-8") as f:
     config = json.load(f)
 if not isinstance(config.get("auto_update"), dict):
     config["auto_update"] = {}
-config["auto_update"]["enabled"] = enabled
+if enabled in ("0", "1"):
+    config["auto_update"]["enabled"] = enabled == "1"
+if channel:
+    config["auto_update"]["channel"] = channel
 # Written beside the original and swapped in whole: the display service's
 # config watcher may be running and must never read a half-written file.
 original = os.stat(path)
@@ -902,9 +1029,11 @@ except BaseException:
     raise
 PY
     then
-        if [ "$AUTO_UPDATE" = "1" ]; then echo "✓ Weekly automatic updates enabled"; else echo "✓ Weekly automatic updates off"; fi
+        if [ "$AUTO_UPDATE" = "1" ]; then echo "✓ Weekly automatic updates enabled"
+        elif [ "$AUTO_UPDATE" = "0" ]; then echo "✓ Weekly automatic updates off"; fi
+        if [ -n "$UPDATE_CHANNEL" ]; then echo "✓ Update channel: $UPDATE_CHANNEL"; fi
     else
-        echo "⚠ Could not set auto_update in config/config.json; turn it on from the General tab instead"
+        echo "⚠ Could not set auto_update in config/config.json; set it from the General tab instead"
     fi
 fi
 
@@ -1226,9 +1355,11 @@ else
         fi
         BUILD_OUTPUT=$(mktemp)
         BUILD_SUCCESS=false
+        _apply_rgb_patches
         if run_rgbmatrix_build "$BUILD_JOBS" "$BUILD_OUTPUT"; then
             BUILD_SUCCESS=true
         fi
+        _revert_rgb_patches
         cat "$BUILD_OUTPUT" >> "$LOG_FILE"
         if [ "$BUILD_SUCCESS" != true ]; then
             print_rgbmatrix_build_failure "$BUILD_OUTPUT"
@@ -1349,15 +1480,16 @@ if ! command -v setcap >/dev/null 2>&1; then
     echo "⚠ setcap not found, skipping capability configuration"
     echo "  Install libcap2-bin if you need hardware timing capabilities"
 else
-    # Find the Python binary and resolve symlinks to get the real binary
+    # The binary the services run (ExecStart=/usr/bin/python3), symlinks
+    # resolved: python3.11 on Bookworm, python3.13 on Trixie. This used to
+    # prefer /usr/bin/python3.13 whenever it existed, which would set the
+    # capability on an interpreter the services never run if python3 pointed
+    # elsewhere.
     PYTHON_BIN=""
     PYTHON_VER=""
-    if [ -f "/usr/bin/python3.13" ]; then
-        PYTHON_BIN=$(readlink -f /usr/bin/python3.13)
-        PYTHON_VER="3.13"
-    elif [ -f "/usr/bin/python3" ]; then
+    if [ -f "/usr/bin/python3" ]; then
         PYTHON_BIN=$(readlink -f /usr/bin/python3)
-        PYTHON_VER=$(python3 --version 2>&1 | grep -oP '(?<=Python )\d+\.\d+' || echo "unknown")
+        PYTHON_VER=$(lm_python_version /usr/bin/python3) || PYTHON_VER="unknown"
     fi
 
     if [ -n "$PYTHON_BIN" ] && [ -f "$PYTHON_BIN" ]; then

mypy-clean.txt

@@ -36,8 +36,10 @@ src/common/sports_card.py
 src/common/sports_card_wrappers.py
 src/common/sports_celebration.py
 src/common/sports_display_rules.py
+src/common/sports_favorites.py
 src/common/sports_fetch.py
 src/common/sports_font_path.py
+src/common/sports_game_over.py
 src/common/sports_live_scroll.py
 src/common/sports_plugin_host.py
 src/common/sports_scroll.py
@@ -58,6 +60,7 @@ src/ipc/contract.py
 src/ipc/server.py
 src/logging_config.py
 src/logo_downloader.py
+src/malloc_tuning.py
 src/matrix_support.py
 src/pi5_matrix_support.py
 src/plugin_system/__init__.py
@@ -88,6 +91,7 @@ src/plugin_system/testing/vegas.py
 src/plugin_system/vegas_elements.py
 src/redaction.py
 src/scan_order.py
+src/screen_runner.py
 src/startup_validator.py
 src/vegas_mode/__init__.py
 src/vegas_mode/config.py

mypy.ini

@@ -6,8 +6,9 @@
 files = src
 exclude = (^|/)(test|__pycache__)/
 
-# Python version
-python_version = 3.10
+# Python version: the oldest the installer supports (Raspberry Pi OS
+# Bookworm ships 3.11; Trixie ships 3.13).
+python_version = 3.11
 
 # Platform (Linux/Raspberry Pi)
 platform = linux
@@ -103,8 +104,8 @@ ignore_missing_imports = True
 
 
 # numpy's own stubs (numpy>=2.3) use Python 3.12 `type` statements, which mypy
-# refuses to parse under python_version = 3.10 -- and 3.10 is the floor this
-# code has to run on, so it stays. Treat numpy as Any instead: skip it, and
+# refuses to parse under python_version = 3.11 -- and 3.11 (Bookworm) is the
+# floor this code has to run on, so it stays. Treat numpy as Any instead: skip it, and
 # follow_imports_for_stubs makes the skip apply to its .pyi files too.
 [mypy-numpy.*]
 follow_imports = skip

patches/rpi-rgb-led-matrix/0001-bulk-setimage.patch

@@ -0,0 +1,174 @@
+Faster SetImage for rpi-rgb-led-matrix (applied by first_time_install.sh at build
+time; the submodule itself stays at its pinned commit).
+
+Copying a frame into the panel buffer was the biggest CPU cost LEDMatrix owns on
+large panels: the binding's SetPixelsPillow walked the image column by column
+and called SetPixel per pixel, and each SetPixel read-modify-writes one word per
+PWM bit plane, 2KB apart, so consecutive pixels were a whole double-row apart and
+almost every write missed the cache. This patch:
+
+ * FrameCanvas gets its own SetPixelsPillow: row by row, one bulk SetPixels call
+   per row;
+ * Framebuffer::SetPixels clips once, looks colours up once per pixel, walks each
+   row's designators in order and writes the bit planes branch-free;
+ * the base Canvas.SetPixelsPillow loop (RGBMatrix.SetImage) is row-major.
+
+The bit-plane buffer is byte-identical to the old code's (882 memcmp checks over
+noise/gradient/solid/low-value/sparse images, clipped offsets, pwm 7/8/11,
+brightness 1/50/90/100, inverse colours, luminance correction off and a pixel
+mapper). Measured on a Pi 4 at 512x64: 6.0-6.3 ms -> 1.8 ms per frame through
+the Python binding; on hdpi (Pi 4, 4x128x64) frame copy 6.57 -> 2.21 ms and the
+display process 139% -> 103% of a core.
+
+LEDMatrix always draws into the canvas that is not on screen and swaps it in
+(DisplayManager.update_display), so the write order cannot show as tearing.
+
+Against hzeller/rpi-rgb-led-matrix 1ee4f76.
+
+diff --git a/bindings/python/rgbmatrix/core.pyx b/bindings/python/rgbmatrix/core.pyx
+index 230d87f..babc3bb 100644
+--- a/bindings/python/rgbmatrix/core.pyx
++++ b/bindings/python/rgbmatrix/core.pyx
+@@ -2,6 +2,7 @@
+ 
+ from libcpp cimport bool
+ from libc.stdint cimport uint8_t, uint32_t, uintptr_t
++from libc.stdlib cimport malloc, free
+ import cython
+ 
+ cdef extern from "Python.h":
+@@ -59,8 +60,9 @@ cdef class Canvas:
+ 
+         buffer = get_pillow_buffer(image_capsule)
+ 
+-        for col in range(max(0, -xstart), min(width, frame_width - xstart)):
+-            for row in range(max(0, -ystart), min(height, frame_height - ystart)):
++        # Row-major: walks both the image and the bitplane buffer sequentially.
++        for row in range(max(0, -ystart), min(height, frame_height - ystart)):
++            for col in range(max(0, -xstart), min(width, frame_width - xstart)):
+                 pixel = buffer[row][col]
+                 r = (pixel ) & 0xFF
+                 g = (pixel >> 8) & 0xFF
+@@ -86,6 +88,41 @@ cdef class FrameCanvas(Canvas):
+     def SetPixel(self, int x, int y, uint8_t red, uint8_t green, uint8_t blue):
+         (<cppinc.FrameCanvas*>self._getCanvas()).SetPixel(x, y, red, green, blue)
+ 
++    @cython.boundscheck(False)
++    @cython.wraparound(False)
++    def SetPixelsPillow(self, int xstart, int ystart, int width, int height, object image_capsule):
++        # Same result as Canvas.SetPixelsPillow(), but hands each image row
++        # to the C++ bulk FrameCanvas::SetPixels() instead of calling the
++        # virtual SetPixel() once per pixel.
++        cdef cppinc.FrameCanvas* my_canvas = <cppinc.FrameCanvas*>self._getCanvas()
++        cdef int col_start = max(0, -xstart)
++        cdef int col_end = min(width, my_canvas.width() - xstart)
++        cdef int row_start = max(0, -ystart)
++        cdef int row_end = min(height, my_canvas.height() - ystart)
++        cdef int row, col, pixel
++        cdef int *src
++        cdef cppinc.Color *line
++        cdef int **buffer
++
++        if col_end <= col_start or row_end <= row_start:
++            return
++        buffer = get_pillow_buffer(image_capsule)
++        line = <cppinc.Color*>malloc((col_end - col_start) * sizeof(cppinc.Color))
++        if line == NULL:
++            raise MemoryError()
++        try:
++            for row in range(row_start, row_end):
++                src = buffer[row]
++                for col in range(col_start, col_end):
++                    pixel = src[col]
++                    line[col - col_start].r = pixel & 0xFF
++                    line[col - col_start].g = (pixel >> 8) & 0xFF
++                    line[col - col_start].b = (pixel >> 16) & 0xFF
++                my_canvas.SetPixels(xstart + col_start, ystart + row,
++                                    col_end - col_start, 1, line)
++        finally:
++            free(line)
++
+ 
+     property width:
+         def __get__(self): return (<cppinc.FrameCanvas*>self._getCanvas()).width()
+diff --git a/bindings/python/rgbmatrix/cppinc.pxd b/bindings/python/rgbmatrix/cppinc.pxd
+index 8bec241..314332d 100644
+--- a/bindings/python/rgbmatrix/cppinc.pxd
++++ b/bindings/python/rgbmatrix/cppinc.pxd
+@@ -25,6 +25,7 @@ cdef extern from "led-matrix.h" namespace "rgb_matrix":
+         FrameCanvas *SwapOnVSync(FrameCanvas*, uint8_t)
+ 
+     cdef cppclass FrameCanvas(Canvas):
++        void SetPixels(int, int, int, int, Color*) nogil
+         bool SetPWMBits(uint8_t)
+         uint8_t pwmbits()
+         void SetBrightness(uint8_t)
+diff --git a/lib/framebuffer.cc b/lib/framebuffer.cc
+index 36d138b..aee62ca 100644
+--- a/lib/framebuffer.cc
++++ b/lib/framebuffer.cc
+@@ -807,11 +807,60 @@ void Framebuffer::SetPixel(int x, int y, uint8_t r, uint8_t g, uint8_t b) {
+   }
+ }
+ 
++// Bulk version of SetPixel(); produces exactly the same bitplane content.
++// Faster because it hoists the per-pixel work out of the loop: the color
++// mapping becomes one 256-entry table built per call (each channel maps
++// independently through the same function), the pixel designators of a row
++// are contiguous in the PixelDesignatorMap, and the bit-plane loop is
++// branchless (the color bits are effectively random, so the branches in
++// SetPixel() mispredict a lot).
+ void Framebuffer::SetPixels(int x, int y, int width, int height, Color *colors) {
+-  for (int iy = 0; iy < height; ++iy) {
+-    for (int ix = 0; ix < width; ++ix) {
+-      SetPixel(x + ix, y + iy, colors->r, colors->g, colors->b);
+-      ++colors;
++  PixelDesignatorMap *const mapper = *shared_mapper_;
++  const int ix_start = std::max(0, -x);
++  const int ix_end = std::min(width, mapper->width() - x);
++  const int iy_start = std::max(0, -y);
++  const int iy_end = std::min(height, mapper->height() - y);
++  if (ix_start >= ix_end || iy_start >= iy_end) return;
++
++  // Common case (luminance correction, no inversion): use the precomputed
++  // table directly; otherwise build one. Cheap enough to do per call, which
++  // matters for callers that send one row at a time.
++  uint16_t local_map[256];
++  const uint16_t *color_map;
++  if (do_luminance_correct_ && !inverse_color_) {
++    color_map = ColorLookupTable::GetLookup(brightness_).color;
++  } else {
++    for (int c = 0; c < 256; ++c) {
++      uint16_t unused1, unused2;
++      MapColors(c, 0, 0, &local_map[c], &unused1, &unused2);
++    }
++    color_map = local_map;
++  }
++
++  const int min_bit_plane = kBitPlanes - pwm_bits_;
++  gpio_bits_t *const plane_start = bitplane_buffer_ + columns_ * min_bit_plane;
++  for (int iy = iy_start; iy < iy_end; ++iy) {
++    const Color *c = colors + iy * width + ix_start;
++    const PixelDesignator *designator = mapper->get(x + ix_start, y + iy);
++    for (int ix = ix_start; ix < ix_end; ++ix, ++c, ++designator) {
++      const long pos = designator->gpio_word;
++      if (pos < 0) continue;  // non-used pixel marker.
++      const uint16_t red = color_map[c->r];
++      const uint16_t green = color_map[c->g];
++      const uint16_t blue = color_map[c->b];
++      const gpio_bits_t r_bits = designator->r_bit;
++      const gpio_bits_t g_bits = designator->g_bit;
++      const gpio_bits_t b_bits = designator->b_bit;
++      const gpio_bits_t designator_mask = designator->mask;
++      gpio_bits_t *bits = plane_start + pos;
++      for (int plane = min_bit_plane; plane < kBitPlanes; ++plane) {
++        const gpio_bits_t color_bits =
++          (r_bits & -(gpio_bits_t)((red >> plane) & 1))
++          | (g_bits & -(gpio_bits_t)((green >> plane) & 1))
++          | (b_bits & -(gpio_bits_t)((blue >> plane) & 1));
++        *bits = (*bits & designator_mask) | color_bits;
++        bits += columns_;
++      }
+     }
+   }
+ }

requirements.txt

@@ -1,5 +1,5 @@
 # LEDMatrix Core Dependencies
-# Compatible with Python 3.10, 3.11, 3.12, and 3.13
+# Compatible with Python 3.11, 3.12 and 3.13; CI tests 3.11 and 3.13
 # Tested on Raspbian OS 12 (Bookworm) and 13 (Trixie)
 
 # Image processing

run.py

@@ -14,6 +14,12 @@ project_dir = os.path.dirname(os.path.abspath(__file__))
 if project_dir not in sys.path:
     sys.path.insert(0, project_dir)
 
+# Cap glibc's malloc arenas before any thread exists (arenas already made
+# stay): the in-process twin of the unit's MALLOC_ARENA_MAX=2, for units
+# installed before that line. A no-op off glibc. See src/malloc_tuning.py.
+from src import malloc_tuning
+malloc_tuning.cap_arenas()
+
 # Under systemd the watchdog clock is already running, and start-up (plugin
 # loads, initial updates) takes far longer than the render loop's limit. Widen
 # it before anything slow is imported; the render loop narrows it again once

scripts/check_system_compatibility.sh

@@ -53,26 +53,35 @@ else
 fi
 echo ""
 
-# Check OS version
+# Check OS version. The supported releases come from the same library the
+# installer uses, so the two cannot disagree.
 echo "2. Checking Operating System Version..."
 echo "---------------------------------------"
-if [ -f /etc/os-release ]; then
-    . /etc/os-release
-    echo "OS: $PRETTY_NAME"
-    echo "Version ID: ${VERSION_ID:-unknown}"
+OS_LIB="$(cd "$(dirname "$0")" && pwd)/install/lib_os.sh"
+OS_LIB_LOADED=0
+OS_RELEASE=""
+if [ -f "$OS_LIB" ]; then
+    # shellcheck source=scripts/install/lib_os.sh
+    . "$OS_LIB"
+    OS_LIB_LOADED=1
+fi
 
-    # first_time_install.sh refuses anything but Raspberry Pi OS / Debian 13
-    # (Trixie), so anything else is an error here too, not a warning.
-    if [[ "$ID" == "raspbian" ]] || [[ "$ID" == "debian" ]]; then
-        if [ "${VERSION_ID:-0}" = "13" ]; then
-            print_success "Detected Debian 13 Trixie - supported"
-        elif [ "${VERSION_ID:-0}" = "12" ]; then
-            print_error "Debian 12 Bookworm is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
-        else
-            print_error "Debian/Raspbian ${VERSION_ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
-        fi
+if [ "$OS_LIB_LOADED" = "0" ]; then
+    print_error "$OS_LIB is missing - download LEDMatrix again"
+elif [ -r "$LM_OS_RELEASE_FILE" ]; then
+    OS_ID=$(lm_os_field ID)
+    OS_VERSION_ID=$(lm_os_field VERSION_ID)
+    echo "OS: $(lm_os_field PRETTY_NAME)"
+    echo "Version ID: ${OS_VERSION_ID:-unknown}"
+
+    # first_time_install.sh refuses anything else, so this is an error here
+    # too, not a warning.
+    if OS_RELEASE=$(lm_os_release); then
+        print_success "Detected $(lm_release_label "$OS_RELEASE") - supported"
+    elif [[ "$OS_ID" == "raspbian" ]] || [[ "$OS_ID" == "debian" ]]; then
+        print_error "Debian/Raspbian ${OS_VERSION_ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite, Trixie (Debian 13) or Bookworm (Debian 12)"
     else
-        print_error "${ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
+        print_error "${OS_ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite, Trixie (Debian 13) or Bookworm (Debian 12)"
     fi
 else
     print_error "Could not detect OS version"
@@ -92,7 +101,7 @@ if [ "$KERNEL_MAJOR" -ge "6" ]; then
     print_success "Kernel version is compatible (6.x or newer)"
     
     if [ "$KERNEL_MAJOR" -eq "6" ] && [ "$KERNEL_MINOR" -ge "12" ]; then
-        print_success "Running latest Trixie kernel (6.12 LTS)"
+        print_success "Running a 6.12 LTS or newer kernel"
     fi
 elif [ "$KERNEL_MAJOR" -eq "5" ] && [ "$KERNEL_MINOR" -ge "10" ]; then
     print_success "Kernel version is compatible (5.10+)"
@@ -104,25 +113,34 @@ echo ""
 # Check Python version
 echo "4. Checking Python Version..."
 echo "-----------------------------"
-if command -v python3 >/dev/null 2>&1; then
-    PYTHON_VERSION=$(python3 -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}")')
-    PYTHON_MAJOR=$(python3 -c 'import sys; print(sys.version_info.major)')
-    PYTHON_MINOR=$(python3 -c 'import sys; print(sys.version_info.minor)')
+if [ "$OS_LIB_LOADED" = "1" ] && command -v python3 >/dev/null 2>&1; then
+    PYTHON_VERSION=$(python3 -c 'import sys; print("%d.%d.%d" % sys.version_info[:3])')
+    PYTHON_MINOR_VERSION=$(lm_python_version) || PYTHON_MINOR_VERSION=""
+    PYTHON_RANGE="3.${LM_PYTHON_MIN_MINOR}-3.${LM_PYTHON_MAX_MINOR}"
 
     echo "Python: $PYTHON_VERSION"
 
-    if [ "$PYTHON_MAJOR" -eq "3" ]; then
-        if [ "$PYTHON_MINOR" -ge "10" ] && [ "$PYTHON_MINOR" -le "13" ]; then
-            print_success "Python version is supported (3.10-3.13)"
-        elif [ "$PYTHON_MINOR" -ge "14" ]; then
-            print_warning "Python 3.${PYTHON_MINOR} is very new - some packages may not be compatible yet"
-        else
-            # Pillow 12 and the pinned test tools need 3.10+, so this won't install.
-            print_error "Python 3.${PYTHON_MINOR} is too old - Python 3.10+ is required"
-        fi
-    else
-        print_error "Python 2.x detected - Python 3.10+ is required"
+    case "$(lm_python_check "$PYTHON_MINOR_VERSION")" in
+        ok)
+            print_success "Python version is supported ($PYTHON_RANGE)"
+            ;;
+        too-old)
+            # The rgbmatrix bindings declare requires-python >=3.11, so the
+            # display cannot be built on anything older.
+            print_error "Python $PYTHON_MINOR_VERSION is too old - Python 3.${LM_PYTHON_MIN_MINOR}+ is required"
+            ;;
+        too-new)
+            print_warning "Python $PYTHON_MINOR_VERSION is newer than LEDMatrix has been tested with ($PYTHON_RANGE)"
+            ;;
+        *)
+            print_warning "Could not read the Python version"
+            ;;
+    esac
+    if [ -n "$OS_RELEASE" ] && [ "$PYTHON_MINOR_VERSION" != "$(lm_release_python "$OS_RELEASE")" ]; then
+        print_warning "$(lm_release_label "$OS_RELEASE") ships Python $(lm_release_python "$OS_RELEASE"), but python3 runs $PYTHON_MINOR_VERSION"
     fi
+elif command -v python3 >/dev/null 2>&1; then
+    print_warning "Cannot check the Python version without $OS_LIB"
 else
     print_error "Python 3 not found - installation required"
 fi
@@ -268,6 +286,22 @@ if command -v ping >/dev/null 2>&1; then
 else
     print_warning "Ping command not available - cannot verify network"
 fi
+
+# WiFi setup from the web page and the LEDMatrix-Setup hotspot drive
+# NetworkManager, the default on both Bookworm and Trixie.
+if [ "$OS_LIB_LOADED" = "1" ]; then
+    case "$(lm_network_stack)" in
+        networkmanager)
+            print_success "NetworkManager manages the network (needed for WiFi setup)"
+            ;;
+        dhcpcd)
+            print_warning "dhcpcd manages the network - WiFi setup from the web page and the setup hotspot need NetworkManager (sudo raspi-config -> Advanced Options -> Network Config)"
+            ;;
+        *)
+            print_warning "Could not tell which service manages the network - WiFi setup from the web page needs NetworkManager"
+            ;;
+    esac
+fi
 echo ""
 
 # Print summary

scripts/install/README.md

@@ -5,11 +5,18 @@ This directory contains scripts for installing and configuring the LEDMatrix sys
 ## Scripts
 
 - **`one-shot-install.sh`** - Single-command installer; clones the
-  repo, checks prerequisites, then runs `first_time_install.sh`.
-  Invoked via `curl ... | bash` from the project root README.
+  repo, checks out the newest release (or `main` with
+  `LEDMATRIX_CHANNEL=beta`), checks prerequisites, then runs
+  `first_time_install.sh`. Invoked via `curl ... | bash` from the project
+  root README. Re-running it never moves a checkout to an older version.
 - **`install_service.sh`** - Installs, enables and starts the display
   service (`ledmatrix.service`), the web interface service
-  (`ledmatrix-web.service`) and the update-verify units (systemd)
+  (`ledmatrix-web.service`) and the update-verify units (systemd), and
+  installs `/usr/local/sbin/ledmatrix-refresh-units`
+- **`ledmatrix_refresh_units.py`** - Not run from here: `install_service.sh`
+  installs a root-owned copy as `/usr/local/sbin/ledmatrix-refresh-units`,
+  which updates run through sudo to install changed units (and the
+  automatic update's rollback, with `--restore`, to put them back)
 - **`install_web_service.sh`** - Installs only the web interface service
   and the update-verify units (systemd)
 - **`install_wifi_monitor.sh`** - Installs the WiFi monitor daemon service
@@ -34,6 +41,9 @@ Libraries (sourced, not run):
   script that renders a unit from `systemd/*.service`
 - **`lib_lowmem.sh`** - Build-job sizing and temporary swap for the C++
   build on low-memory Pis (`first_time_install.sh` Step 6)
+- **`lib_os.sh`** - Which releases (Bookworm, Trixie) and Python versions
+  (3.11-3.13) the installer accepts, and which service runs the network;
+  shared by `first_time_install.sh` and `scripts/check_system_compatibility.sh`
 
 ## Usage
 

scripts/install/configure_web_sudo.sh

@@ -138,6 +138,8 @@ echo "- View system logs via journalctl"
 echo "- Reboot and shutdown the system"
 echo "- Remove plugin directories (for update/uninstall when root-owned files block deletion)"
 echo "- Install plugin/base requirements.txt as root (so ledmatrix.service can see them)"
+echo "- Install the LEDMatrix systemd units an update changed, and restore them on rollback"
+echo "  (/usr/local/sbin/ledmatrix-refresh-units, installed by install_service.sh)"
 echo ""
 
 # Ask for confirmation

scripts/install/install_service.sh

@@ -143,6 +143,30 @@ for VERIFY_UNIT in ledmatrix-update-verify.service ledmatrix-update-verify.path;
     fi
 done
 
+# The helper updates run (through sudo, see lib_sudoers.sh) to install these
+# same units when a new version changes their templates, and to put the old
+# ones back if the automatic update rolls back. Root-owned and outside the
+# checkout, so the web user who owns the checkout cannot change what sudo runs.
+# Not fatal: without it, updates leave the units for the next reinstall.
+REFRESH_UNITS_SRC="$PROJECT_ROOT_DIR/scripts/install/ledmatrix_refresh_units.py"
+REFRESH_UNITS_DEST=/usr/local/sbin/ledmatrix-refresh-units
+if [ -f "$REFRESH_UNITS_SRC" ]; then
+    if sudo install -D -o root -g root -m 0755 "$REFRESH_UNITS_SRC" "$REFRESH_UNITS_DEST"; then
+        echo "Installed $REFRESH_UNITS_DEST (lets updates refresh these units)"
+    else
+        echo "WARNING: could not install $REFRESH_UNITS_DEST; updates will not refresh the systemd units" >&2
+    fi
+fi
+# The units above are copied from mktemp files, which are 0600. 0644 is what
+# first_time_install.sh (Step 8.1) sets, and lets the web interface compare
+# them with the templates after an update without root.
+for INSTALLED_UNIT in ledmatrix.service ledmatrix-web.service \
+        ledmatrix-update-verify.service ledmatrix-update-verify.path; do
+    if [ -f "/etc/systemd/system/$INSTALLED_UNIT" ]; then
+        sudo chmod 644 "/etc/systemd/system/$INSTALLED_UNIT" || true
+    fi
+done
+
 echo "Reloading systemd daemon for web service..."
 sudo systemctl daemon-reload
 

scripts/install/ledmatrix_refresh_units.py

@@ -0,0 +1,451 @@
+#!/usr/bin/python3 -I
+"""Refresh the installed LEDMatrix systemd units from the checkout's templates.
+
+Installed by scripts/install/install_service.sh as a root-owned copy,
+/usr/local/sbin/ledmatrix-refresh-units, and granted to the web interface's
+user by /etc/sudoers.d/ledmatrix_web (scripts/install/lib_sudoers.sh) with
+exactly two command lines:
+
+    ledmatrix-refresh-units              (no arguments)
+    ledmatrix-refresh-units --restore
+
+An update (Update Code, or the weekly automatic update) pulls new unit
+templates into systemd/, but the units systemd runs are the copies in
+/etc/systemd/system, which only the installer used to write. So a setting
+added to a template -- the render-loop watchdog, a memory limit -- never
+reached a device that was already installed. After an update the web
+interface runs this, and the next restart picks the new units up.
+
+* **No arguments:** render each installed unit from systemd/<unit> exactly as
+  install_service.sh does (__PROJECT_ROOT_DIR__ and __USER__ replaced
+  literally), and install the ones whose content differs (comments and blank
+  lines aside, as src/startup_validator.py compares them), then
+  ``systemctl daemon-reload``. The units replaced are saved first, so the
+  automatic update's rollback can put them back.
+* ``--restore``: put back the units the last refresh replaced, and
+  daemon-reload. Nothing saved means nothing to do.
+* ``--check``: print the units that would change, one per line. Needs no
+  root and changes nothing.
+
+What it trusts, and why. It takes no other input: the project directory and
+the web interface's user come from the installed, root-owned
+ledmatrix.service and ledmatrix-web.service, not from the caller, and sudo
+strips the caller's environment (``-I`` ignores the PYTHON* variables too).
+It only replaces units that are already installed, only the four
+install_service.sh installs, and only with a rendering that keeps each unit's
+User= (root for the display, the web user for the others) and
+WorkingDirectory=. The templates are files the web user can edit -- but so is
+run.py, which ledmatrix.service already runs as root, so a template grants
+nothing that user did not have; the checks keep a damaged or hostile template
+from changing who a unit runs as, and keep this from reading anything but a
+regular file under the checkout's systemd/ folder.
+
+Standard library only, and no imports from the checkout: the installed copy
+must not run code the web user can change.
+"""
+import json
+import os
+import re
+import stat
+import subprocess  # nosec B404 - fixed argv, no shell  # nosemgrep
+import sys
+import tempfile
+
+SYSTEMD_DIR = '/etc/systemd/system'
+#: Root-only: the units the last refresh replaced, for --restore.
+BACKUP_DIR = '/var/lib/ledmatrix/unit-backup'
+MANIFEST = 'manifest.json'
+INSTALLED_PATH = '/usr/local/sbin/ledmatrix-refresh-units'
+
+DISPLAY_UNIT = 'ledmatrix.service'
+WEB_UNIT = 'ledmatrix-web.service'
+VERIFY_SERVICE = 'ledmatrix-update-verify.service'
+VERIFY_PATH = 'ledmatrix-update-verify.path'
+#: What install_service.sh installs, in its order. Nothing else is touched.
+UNITS = (DISPLAY_UNIT, WEB_UNIT, VERIFY_SERVICE, VERIFY_PATH)
+
+MAX_TEMPLATE_BYTES = 64 * 1024
+_USER_RE = re.compile(r'^[a-z_][a-z0-9_-]{0,31}$')
+#: systemd expands % specifiers, and a quote, backslash or line break would
+#: be reinterpreted in a unit file (src/auto_update_setup.py refuses the same).
+#: (On Windows, where the tests also run, a backslash is the path separator.)
+_UNSAFE_PATH_CHARS = set('%"') | ({'\\'} if os.sep == '/' else set())
+
+EXIT_OK = 0
+EXIT_FAILED = 1
+EXIT_USAGE = 2
+
+
+class RefreshError(Exception):
+    """Why the units were left alone, in words for the web interface's log."""
+
+
+class UnitsUnreadable(RefreshError):
+    """An installed unit is not readable by this (unprivileged) user.
+
+    install_service.sh used to leave units mode 0600 (first_time_install.sh's
+    Step 8.1 makes them 0644), so ``--check`` as the web user cannot always
+    tell; the root helper itself can.
+    """
+
+
+def directive_values(text, key):
+    """Every value of ``key=`` in a unit's text, in order (systemd allows spaces around ``=``)."""
+    return [m.group(1).strip() for m in re.finditer(rf'^[ \t]*{key}[ \t]*=(.*)$', text or '', re.M)]
+
+
+def layout_problem(text, section, keys):
+    """What would make ``directive_values`` misread the unit as systemd reads it, or None.
+
+    A ``User=`` inside a backslash-continued line is part of the line before,
+    and one under [Unit] is ignored, so either could pass a check that systemd
+    then does not apply. Neither appears in the shipped templates.
+    """
+    current = None
+    for raw in (text or '').splitlines():
+        line = raw.strip()
+        if not line or line.startswith(('#', ';')):
+            continue
+        if line.endswith('\\'):
+            return 'continues a line with a backslash'
+        if line.startswith('[') and line.endswith(']'):
+            current = line[1:-1]
+            continue
+        key = line.split('=', 1)[0].strip()
+        if key in keys and current != section:
+            return f'sets {key}= outside [{section}]'
+    return None
+
+
+def unit_body(text):
+    """A unit's meaningful lines in order: no comments, no blank lines.
+
+    The same comparison src/startup_validator.py uses for its drift warning,
+    so what this refreshes is exactly what that warns about.
+    """
+    lines = []
+    for line in (text or '').splitlines():
+        line = line.strip()
+        if line and not line.startswith('#'):
+            lines.append(line)
+    return '\n'.join(lines)
+
+
+def render(template, project_root, user):
+    """install_service.sh's ``sed "s|__PROJECT_ROOT_DIR__|...|g; s|__USER__|...|g"``."""
+    return template.replace('__PROJECT_ROOT_DIR__', project_root).replace('__USER__', user)
+
+
+def _read_regular(path, limit=MAX_TEMPLATE_BYTES, dir_fd=None):
+    """A regular file's text, never through a symlink, a FIFO or a device."""
+    flags = os.O_RDONLY | getattr(os, 'O_NOFOLLOW', 0) | getattr(os, 'O_NONBLOCK', 0)
+    kwargs = {'dir_fd': dir_fd} if dir_fd is not None else {}
+    fd = os.open(path, flags, **kwargs)
+    try:
+        info = os.fstat(fd)
+        if not stat.S_ISREG(info.st_mode):
+            raise RefreshError(f'{path} is not a regular file')
+        if info.st_size > limit:
+            raise RefreshError(f'{path} is larger than {limit} bytes')
+        data = b''
+        while True:
+            chunk = os.read(fd, limit + 1 - len(data))
+            if not chunk:
+                break
+            data += chunk
+            if len(data) > limit:
+                raise RefreshError(f'{path} is larger than {limit} bytes')
+    finally:
+        os.close(fd)
+    if b'\0' in data:
+        raise RefreshError(f'{path} is not a text file')
+    try:
+        return data.decode('utf-8')
+    except UnicodeDecodeError as e:
+        raise RefreshError(f'{path} is not UTF-8') from e
+
+
+def _read_installed(systemd_dir, name):
+    path = os.path.join(systemd_dir, name)
+    try:
+        with open(path, 'r', encoding='utf-8') as f:
+            return f.read()
+    except FileNotFoundError:
+        return None
+    except PermissionError as e:
+        raise UnitsUnreadable(f'cannot read the installed {name}: {e}') from e
+    except (OSError, UnicodeDecodeError) as e:
+        raise RefreshError(f'cannot read the installed {name}: {e}') from e
+
+
+def _lookup_user(user):
+    try:
+        import pwd
+    except ImportError:  # not a POSIX host (the tests on Windows)
+        return True
+    try:
+        pwd.getpwnam(user)
+        return True
+    except KeyError:
+        return False
+
+
+class Refresher:
+    def __init__(self, systemd_dir=SYSTEMD_DIR, backup_dir=BACKUP_DIR, run=subprocess.run,
+                 is_root=None, user_exists=_lookup_user, log=None):
+        self.systemd_dir = systemd_dir
+        self.backup_dir = backup_dir
+        self.run = run
+        self.is_root = is_root or (lambda: hasattr(os, 'geteuid') and os.geteuid() == 0)
+        self.user_exists = user_exists
+        self.log = log or (lambda msg: print(msg, flush=True))
+
+    # -- what the installed units say -------------------------------------
+
+    def context(self, installed):
+        """(project root, web user) from the installed, root-owned units."""
+        display = installed.get(DISPLAY_UNIT)
+        if display is None:
+            raise RefreshError(f'{DISPLAY_UNIT} is not installed; run scripts/install/install_service.sh')
+        roots = directive_values(display, 'WorkingDirectory')
+        if len(roots) != 1:
+            raise RefreshError(f'the installed {DISPLAY_UNIT} does not name one WorkingDirectory')
+        root = roots[0]
+        if (not os.path.isabs(root) or any(ch in _UNSAFE_PATH_CHARS or ord(ch) < 32 for ch in root)
+                or os.path.normpath(root) != root):
+            raise RefreshError(f'the installed {DISPLAY_UNIT} runs from {root!r}, which cannot be used')
+        if not os.path.isdir(root):
+            raise RefreshError(f'{root} (the installed {DISPLAY_UNIT} WorkingDirectory) does not exist')
+
+        user = None
+        web = installed.get(WEB_UNIT)
+        if web is not None:
+            users = directive_values(web, 'User')
+            user = users[0] if len(users) == 1 else ('root' if not users else None)
+            if user is None or not _USER_RE.match(user) or not self.user_exists(user):
+                raise RefreshError(f'the installed {WEB_UNIT} runs as an account that cannot be used')
+            if directive_values(web, 'WorkingDirectory') != [root]:
+                raise RefreshError(f'the installed {WEB_UNIT} and {DISPLAY_UNIT} run from different folders')
+        return root, user
+
+    @staticmethod
+    def expected_user(name, web_user):
+        return 'root' if name == DISPLAY_UNIT else web_user
+
+    def _template(self, root, name):
+        """systemd/<name> under the checkout, as a regular file, never via a symlink."""
+        dir_flags = os.O_RDONLY | getattr(os, 'O_DIRECTORY', 0) | getattr(os, 'O_NOFOLLOW', 0)
+        if os.open in getattr(os, 'supports_dir_fd', set()):
+            try:
+                dfd = os.open(os.path.join(root, 'systemd'), dir_flags)
+            except OSError as e:
+                raise RefreshError(f'cannot open {root}/systemd: {e}') from e
+            try:
+                return _read_regular(name, dir_fd=dfd)
+            except FileNotFoundError:
+                return None
+            except OSError as e:
+                raise RefreshError(f'cannot read systemd/{name}: {e}') from e
+            finally:
+                os.close(dfd)
+        path = os.path.join(root, 'systemd', name)
+        if os.path.islink(os.path.join(root, 'systemd')):
+            raise RefreshError(f'{root}/systemd is a symlink')
+        try:
+            return _read_regular(path)
+        except FileNotFoundError:
+            return None
+        except OSError as e:
+            raise RefreshError(f'cannot read systemd/{name}: {e}') from e
+
+    def _validate(self, name, rendered, root, user):
+        problem = layout_problem(rendered, 'Service', ('User', 'WorkingDirectory'))
+        if problem:
+            raise RefreshError(f'systemd/{name} {problem}; refusing to install it')
+        if directive_values(rendered, 'User') != [user]:
+            raise RefreshError(f'systemd/{name} would not run as {user}; refusing to install it')
+        if directive_values(rendered, 'WorkingDirectory') != [root]:
+            raise RefreshError(f'systemd/{name} would not run from {root}; refusing to install it')
+
+    def plan(self):
+        """{unit: (installed text, new text)} for every installed unit that would change.
+
+        Raises RefreshError, and so changes nothing, if any unit cannot be
+        rendered safely: four units refreshed as a set or not at all.
+        """
+        installed = {name: _read_installed(self.systemd_dir, name) for name in UNITS}
+        root, web_user = self.context(installed)
+        changes = {}
+        for name in UNITS:
+            current = installed[name]
+            if current is None:
+                continue  # never installed here: installing is the installer's job
+            user = self.expected_user(name, web_user)
+            if user is None:
+                continue  # the web unit is not installed, so neither is its user
+            template = self._template(root, name)
+            if template is None:
+                continue  # a version without this unit leaves the installed one alone
+            rendered = render(template, root, user)
+            # A path unit runs nothing itself; what matters is what it starts.
+            if name.endswith('.service'):
+                self._validate(name, rendered, root, user)
+            else:
+                self._validate_path(name, rendered)
+            if unit_body(rendered) != unit_body(current):
+                changes[name] = (current, rendered)
+        return changes
+
+    def _validate_path(self, name, rendered):
+        problem = layout_problem(rendered, 'Path', ('Unit',))
+        if problem:
+            raise RefreshError(f'systemd/{name} {problem}; refusing to install it')
+        if directive_values(rendered, 'Unit') != [VERIFY_SERVICE]:
+            raise RefreshError(f'systemd/{name} does not start {VERIFY_SERVICE}; refusing to install it')
+        if directive_values(rendered, 'User'):
+            raise RefreshError(f'systemd/{name} sets User=; refusing to install it')
+
+    # -- writing ------------------------------------------------------------
+
+    def _write_unit(self, name, text):
+        fd, tmp = tempfile.mkstemp(dir=self.systemd_dir, prefix=f'.{name}.')
+        try:
+            with os.fdopen(fd, 'w', encoding='utf-8', newline='\n') as f:
+                f.write(text)
+            os.chmod(tmp, 0o644)
+            os.replace(tmp, os.path.join(self.systemd_dir, name))
+        except BaseException:
+            try:
+                os.unlink(tmp)
+            except OSError:
+                pass
+            raise
+
+    def _backup_dir(self):
+        """The backup folder, created root-only; refused if it is not a plain folder."""
+        os.makedirs(os.path.dirname(self.backup_dir), mode=0o755, exist_ok=True)
+        try:
+            os.mkdir(self.backup_dir, 0o700)
+        except FileExistsError:
+            pass
+        info = os.lstat(self.backup_dir)
+        if not stat.S_ISDIR(info.st_mode):
+            raise RefreshError(f'{self.backup_dir} is not a folder')
+        if hasattr(os, 'geteuid') and info.st_uid != os.geteuid():
+            raise RefreshError(f'{self.backup_dir} is not owned by root')
+        return self.backup_dir
+
+    def _clear_backup(self, folder):
+        for entry in os.listdir(folder):
+            path = os.path.join(folder, entry)
+            if os.path.isfile(path) or os.path.islink(path):
+                os.unlink(path)
+
+    def _systemctl(self, *args):
+        result = self.run(['systemctl', *args], capture_output=True, text=True, timeout=60)
+        if result.returncode != 0:
+            raise RefreshError(f'"systemctl {" ".join(args)}" failed: '
+                               f'{(result.stderr or result.stdout or "").strip()}')
+
+    def _restart_path_unit_if_active(self, names):
+        """A rewritten path unit watches the old path until it is restarted."""
+        if VERIFY_PATH not in names:
+            return
+        state = self.run(['systemctl', 'is-active', VERIFY_PATH], capture_output=True, text=True, timeout=30)
+        if (state.stdout or '').strip() == 'active':
+            self._systemctl('restart', VERIFY_PATH)
+
+    def refresh(self):
+        if not self.is_root():
+            raise RefreshError('must run as root (sudo)')
+        changes = self.plan()
+        folder = self._backup_dir()
+        # Always reset: the backup belongs to this refresh, so a --restore
+        # after an update that changed nothing restores nothing.
+        self._clear_backup(folder)
+        if not changes:
+            self.log('units: up to date')
+            return []
+        for name, (current, _) in changes.items():
+            with open(os.path.join(folder, name), 'w', encoding='utf-8', newline='\n') as f:
+                f.write(current)
+        with open(os.path.join(folder, MANIFEST), 'w', encoding='utf-8') as f:
+            json.dump({'units': sorted(changes)}, f)
+        try:
+            for name, (_, rendered) in changes.items():
+                self._write_unit(name, rendered)
+            self._systemctl('daemon-reload')
+        except BaseException:
+            # A failed refresh is reported as a failure, so the update records
+            # no units_refreshed and a rollback would not --restore. Put the
+            # replaced units back now, rather than leave a half-written set
+            # under the old code.
+            self._undo(changes, folder)
+            raise
+        self._restart_path_unit_if_active(changes)
+        self.log('units refreshed: ' + ' '.join(sorted(changes)))
+        return sorted(changes)
+
+    def _undo(self, changes, folder):
+        """Best effort: reinstall the units a failed refresh replaced."""
+        undone = True
+        for name, (current, _) in changes.items():
+            try:
+                self._write_unit(name, current)
+            except OSError as e:
+                undone = False
+                self.log(f'units: could not put back {name}: {e}')
+        try:
+            self.run(['systemctl', 'daemon-reload'], capture_output=True, text=True, timeout=60)
+        except (OSError, subprocess.SubprocessError) as e:
+            self.log(f'units: daemon-reload after putting units back failed: {e}')
+        if undone:
+            # Nothing is left to restore; keep the backup only if a unit could
+            # not be put back, so a manual --restore still can.
+            self._clear_backup(folder)
+
+    def restore(self):
+        if not self.is_root():
+            raise RefreshError('must run as root (sudo)')
+        folder = self._backup_dir()
+        try:
+            manifest = json.loads(_read_regular(os.path.join(folder, MANIFEST)))
+        except FileNotFoundError:
+            self.log('units: nothing to restore')
+            return []
+        names = [n for n in (manifest or {}).get('units', []) if n in UNITS]
+        for name in names:
+            self._write_unit(name, _read_regular(os.path.join(folder, name)))
+        self._systemctl('daemon-reload')
+        self._restart_path_unit_if_active(names)
+        self._clear_backup(folder)
+        self.log('units restored: ' + ' '.join(names))
+        return names
+
+
+def main(argv, refresher=None):
+    args = argv[1:]
+    if args not in ([], ['--restore'], ['--check']):
+        print('usage: ledmatrix-refresh-units [--restore | --check]', file=sys.stderr)
+        return EXIT_USAGE
+    refresher = refresher or Refresher()
+    try:
+        if args == ['--check']:
+            for name in sorted(refresher.plan()):
+                print(name)
+        elif args == ['--restore']:
+            refresher.restore()
+        else:
+            refresher.refresh()
+    except (RefreshError, OSError, subprocess.SubprocessError, ValueError) as e:
+        print(f'ledmatrix-refresh-units: {e}', file=sys.stderr)
+        return EXIT_FAILED
+    return EXIT_OK
+
+
+if __name__ == '__main__':
+    # Only as the installed program: sudo already sets a secure PATH, and
+    # this pins the one systemctl comes from. (Not in main(), which the
+    # tests call in-process.)
+    os.environ['PATH'] = '/usr/sbin:/usr/bin:/sbin:/bin'
+    sys.exit(main(sys.argv))

scripts/install/lib_os.sh

@@ -0,0 +1,138 @@
+#!/bin/bash
+# Which operating systems and Python versions LEDMatrix installs on.
+#
+# Sourced by first_time_install.sh and scripts/check_system_compatibility.sh,
+# so the installer and the compatibility checker cannot disagree about what
+# is supported. Pure functions: nothing here installs, changes or exits --
+# the callers decide what to do with the answers.
+#
+# Supported (Lite, no desktop):
+#   Raspberry Pi OS / Debian 12 "Bookworm" -- Python 3.11
+#   Raspberry Pi OS / Debian 13 "Trixie"   -- Python 3.13
+#
+# Everything the installer asks apt for (python3-pip, python3-venv,
+# python-dev-is-python3, python3-pil, python3-pil.imagetk, build-essential,
+# python3-setuptools, python3-wheel, cmake, ninja-build, git, curl, wget,
+# unzip, and hostapd, dnsmasq, network-manager for WiFi setup) has the same
+# name on both releases. Both ship a pip (23.0.1 and 25.1.1) that is PEP 668
+# "externally managed" and accepts --break-system-packages, and a cmake (3.25
+# and 3.31) new enough for the rgbmatrix build (3.22). So no step needs a
+# per-release branch today; if one ever does, the release name comes from
+# lm_os_release below.
+
+# Test hook: the os-release file to read.
+LM_OS_RELEASE_FILE="${LM_OS_RELEASE_FILE:-/etc/os-release}"
+
+# Oldest and newest python3 minor versions the installer accepts. 3.11 is
+# Bookworm's, and also the floor of the rgbmatrix bindings (requires-python
+# >=3.11 in rpi-rgb-led-matrix-master/pyproject.toml); 3.13 is Trixie's.
+LM_PYTHON_MIN_MINOR=11
+LM_PYTHON_MAX_MINOR=13
+
+# lm_os_field KEY -- one value from os-release with its quotes removed; empty
+# when the key or the file is missing. Parsed rather than sourced so that
+# os-release's ID, VERSION and friends do not land in the caller's variables.
+lm_os_field() {
+    [ -r "$LM_OS_RELEASE_FILE" ] || return 0
+    sed -n "/^$1=/{s/^$1=//;s/^[\"']//;s/[\"']\$//;p;q;}" "$LM_OS_RELEASE_FILE"
+}
+
+# lm_os_release -- print "bookworm" or "trixie" and succeed on a supported
+# release; print nothing and fail on anything else. VERSION_ID decides; the
+# codename is used only when VERSION_ID is missing.
+lm_os_release() {
+    local id version
+    id=$(lm_os_field ID)
+    version=$(lm_os_field VERSION_ID)
+    [ -n "$version" ] || version=$(lm_os_field VERSION_CODENAME)
+    case "$id" in
+        raspbian|debian) ;;
+        *) return 1 ;;
+    esac
+    case "$version" in
+        12|bookworm) echo bookworm ;;
+        13|trixie) echo trixie ;;
+        *) return 1 ;;
+    esac
+}
+
+# lm_release_label RELEASE -- how to name a release to a person.
+lm_release_label() {
+    case "$1" in
+        bookworm) echo "Debian 12 (Bookworm)" ;;
+        trixie) echo "Debian 13 (Trixie)" ;;
+        *) echo "$1" ;;
+    esac
+}
+
+# lm_release_python RELEASE -- the python3 version a release ships, e.g. 3.11.
+lm_release_python() {
+    case "$1" in
+        bookworm) echo 3.11 ;;
+        trixie) echo 3.13 ;;
+        *) return 1 ;;
+    esac
+}
+
+# lm_python_version [PYTHON] -- "3.11" and so on for python3 (or PYTHON);
+# prints nothing and fails when it cannot be run.
+lm_python_version() {
+    "${1:-python3}" -c 'import sys; print("%d.%d" % sys.version_info[:2])' 2>/dev/null
+}
+
+# lm_python_check VERSION -- print "ok", "too-old", "too-new" or "unknown"
+# for a version such as 3.11. Always succeeds, so it is safe under set -e.
+lm_python_check() {
+    local major minor
+    major=${1%%.*}
+    minor=${1#*.}
+    minor=${minor%%.*}
+    case "$major:$minor" in
+        *[!0-9:]*|:*|*:) echo unknown; return 0 ;;
+    esac
+    if [ "$major" -lt 3 ] || { [ "$major" -eq 3 ] && [ "$minor" -lt "$LM_PYTHON_MIN_MINOR" ]; }; then
+        echo too-old
+    elif [ "$major" -gt 3 ] || [ "$minor" -gt "$LM_PYTHON_MAX_MINOR" ]; then
+        echo too-new
+    else
+        echo ok
+    fi
+}
+
+# lm_network_stack -- which service runs the network: "networkmanager",
+# "dhcpcd" or "unknown". Raspberry Pi OS uses NetworkManager on both Bookworm
+# and Trixie; dhcpcd appears when someone switched back to it in raspi-config.
+lm_network_stack() {
+    if systemctl is-active --quiet NetworkManager 2>/dev/null; then
+        echo networkmanager
+    elif systemctl is-active --quiet dhcpcd 2>/dev/null; then
+        echo dhcpcd
+    else
+        echo unknown
+    fi
+}
+
+# lm_print_dhcpcd_advice -- the explanation for a Pi running dhcpcd. WiFi
+# setup from the web page and the LEDMatrix-Setup hotspot both drive
+# NetworkManager (nmcli). The installer does not switch the network stack
+# itself: doing that over SSH can cut the connection it is running on.
+lm_print_dhcpcd_advice() {
+    echo "⚠ This Pi manages its network with dhcpcd, not NetworkManager."
+    echo "  LEDMatrix installs and the display works, but choosing a WiFi network"
+    echo "  from the web page and the LEDMatrix-Setup hotspot both need NetworkManager."
+    echo "  To switch (with a keyboard and screen attached, or over Ethernet):"
+    echo "    sudo raspi-config  ->  Advanced Options  ->  Network Config  ->  NetworkManager"
+    echo "  then reboot."
+}
+
+# lm_print_supported_os_help -- what to do on an unsupported system.
+lm_print_supported_os_help() {
+    echo "LEDMatrix needs Raspberry Pi OS Lite: Trixie (Debian 13) or Bookworm (Debian 12)."
+    echo ""
+    echo "To install Raspberry Pi OS Lite:"
+    echo "  1. Download Raspberry Pi Imager from: https://www.raspberrypi.com/software/"
+    echo "  2. Choose 'Raspberry Pi OS Lite (64-bit)'. Trixie is the current version and"
+    echo "     is recommended; Bookworm (listed as Legacy) also works"
+    echo "  3. Flash it to the SD card"
+    echo "  4. Boot the Pi and run this script again"
+}

scripts/install/lib_sudoers.sh

@@ -10,6 +10,11 @@
 #
 # Add or remove a grant here and nowhere else.
 
+# Root-owned copy of scripts/install/ledmatrix_refresh_units.py, installed by
+# install_service.sh. Outside the checkout on purpose: the web user owns the
+# checkout, so a granted file inside it could be rewritten and run as root.
+LEDMATRIX_REFRESH_UNITS_PATH=/usr/local/sbin/ledmatrix-refresh-units
+
 # web_sudoers_rules WEB_USER PROJECT_ROOT SYSTEMCTL_PATH BASH_PATH REBOOT_PATH POWEROFF_PATH JOURNALCTL_PATH
 #
 # Print the ledmatrix_web sudoers rules to stdout.
@@ -58,6 +63,10 @@ $WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_pl
 # Install a requirements.txt as root via vetted helper, so packages are visible
 # to root-run ledmatrix.service (not just the web interface's own user).
 $WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh *
+# After an update, install the new systemd units (no arguments: "" allows none)
+# and, on the automatic update's rollback, put the previous ones back.
+$WEB_USER ALL=(ALL) NOPASSWD: $LEDMATRIX_REFRESH_UNITS_PATH ""
+$WEB_USER ALL=(ALL) NOPASSWD: $LEDMATRIX_REFRESH_UNITS_PATH --restore
 EOF
     if [ -n "$JOURNALCTL_PATH" ]; then
         cat << EOF

scripts/install/one-shot-install.sh

@@ -3,6 +3,10 @@
 # LED Matrix One-Shot Installation Script
 # This script provides a single-command installation experience
 # Usage: curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | bash
+#
+# A new install runs the newest release (the stable update channel). For the
+# newest code from main instead (the beta channel), set LEDMATRIX_CHANNEL=beta:
+#   curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | LEDMATRIX_CHANNEL=beta bash
 
 set -Eeuo pipefail
 
@@ -205,6 +209,114 @@ check_sudo() {
     print_success "Sudo access confirmed"
 }
 
+# --- release checkout helpers ------------------------------------------------
+# Which version an install runs. The rules are web_interface/update_channel.py's,
+# so the installer and the web interface's updates agree:
+#   stable (default)  the newest vX.Y.Z tag by semantic version; pre-releases
+#                     (v3.8.0-rc1), leading zeros and other tags are ignored
+#   beta              main, the newest code
+# Never backwards: an existing checkout moves to a release only when that
+# release contains its current commit (git merge-base --is-ancestor).
+# Never fatal: whatever goes wrong, the install carries on with the checkout
+# as it is.
+
+# Print "stable" or "beta": LEDMATRIX_CHANNEL when it is set, else the
+# existing install's auto_update.channel (CONFIG_FILE), else stable.
+_lm_channel() {
+    local config_file="${1:-}" value
+    value=$(printf '%s' "${LEDMATRIX_CHANNEL:-}" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')
+    case "$value" in
+        stable|beta) printf '%s\n' "$value"; return 0 ;;
+        "") ;;
+        *) print_warning "LEDMATRIX_CHANNEL=${LEDMATRIX_CHANNEL} is not stable or beta; using stable" >&2
+           printf 'stable\n'; return 0 ;;
+    esac
+    if [ -n "$config_file" ] && [ -f "$config_file" ] && command -v python3 >/dev/null 2>&1; then
+        value=$(python3 - "$config_file" 2>/dev/null <<'PY' || true
+import json, sys
+try:
+    with open(sys.argv[1], encoding="utf-8") as f:
+        section = json.load(f).get("auto_update")
+    value = section.get("channel") if isinstance(section, dict) else None
+    print(value.strip().lower() if isinstance(value, str) else "")
+except Exception:
+    print("")
+PY
+)
+        if [ "$value" = "beta" ]; then
+            printf 'beta\n'
+            return 0
+        fi
+    fi
+    printf 'stable\n'
+}
+
+# Print the newest release tag of the repository in the current directory,
+# or nothing when it has none.
+_lm_newest_release_tag() {
+    git tag --list 'v*' 2>/dev/null \
+        | grep -E '^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$' \
+        | sort -t. -k1.2,1n -k2,2n -k3,3n \
+        | tail -n 1 || true
+}
+
+# A fresh clone (on main): move to the newest release unless beta was asked for.
+_lm_checkout_release_after_clone() {
+    local channel tag
+    channel=$(_lm_channel "")
+    if [ "$channel" = "beta" ]; then
+        print_success "Beta channel: installing the newest code from main"
+        return 0
+    fi
+    tag=$(_lm_newest_release_tag)
+    if [ -z "$tag" ]; then
+        print_warning "No release found; installing the newest code from main"
+        return 0
+    fi
+    if git -c advice.detachedHead=false checkout --quiet --detach "${tag}^{commit}"; then
+        print_success "Installing release $tag (stable channel)"
+    else
+        print_warning "Could not check out release $tag; installing the newest code from main"
+    fi
+    return 0
+}
+
+# An existing checkout: move it forward along its channel, never backwards.
+# Returns 1 when it should be updated the way it always was (a fast-forward
+# pull of its branch): beta, or stable on a branch newer than every release.
+_lm_update_existing_checkout() {
+    local channel tag head tag_sha
+    channel=$(_lm_channel "config/config.json")
+    if [ "$channel" = "beta" ]; then
+        return 1
+    fi
+    if ! git fetch --quiet --tags --force origin >/dev/null 2>&1; then
+        print_warning "Could not fetch release tags; keeping the current version"
+        return 0
+    fi
+    tag=$(_lm_newest_release_tag)
+    head=$(git rev-parse --verify --quiet HEAD 2>/dev/null || true)
+    if [ -n "$tag" ] && [ -n "$head" ] && git merge-base --is-ancestor "$head" "$tag" 2>/dev/null; then
+        tag_sha=$(git rev-parse --verify --quiet "${tag}^{commit}" 2>/dev/null || true)
+        if [ "$head" = "$tag_sha" ]; then
+            print_success "Already on the newest release, $tag"
+        elif git -c advice.detachedHead=false checkout --quiet --detach "${tag}^{commit}"; then
+            print_success "Updated to release $tag (stable channel)"
+        else
+            print_warning "Could not move to release $tag (local changes?); keeping the current version"
+        fi
+        return 0
+    fi
+    if git symbolic-ref --quiet HEAD >/dev/null 2>&1; then
+        # Newer than the newest release (or no release yet): follow the branch
+        # until a release includes this version, as updates do.
+        return 1
+    fi
+    print_success "This checkout is newer than the newest release${tag:+ ($tag)}; leaving it as it is"
+    return 0
+}
+# --- end release checkout helpers --------------------------------------------
+
 # Main installation function
 main() {
     print_step "LED Matrix One-Shot Installation"
@@ -292,7 +404,10 @@ main() {
             
             # Try to safely update current branch first (fast-forward only to avoid unintended merges)
             PULL_SUCCESS=false
-            if git pull --ff-only origin "$CURRENT_BRANCH" >/dev/null 2>&1; then
+            # Stable: the newest release, if it contains this version.
+            if _lm_update_existing_checkout; then
+                PULL_SUCCESS=true
+            elif git pull --ff-only origin "$CURRENT_BRANCH" >/dev/null 2>&1; then
                 print_success "Repository updated successfully (branch: $CURRENT_BRANCH)"
                 PULL_SUCCESS=true
             else
@@ -323,10 +438,12 @@ main() {
             rm -rf "$REPO_DIR"
             print_success "Cloning repository..."
             retry git clone "$REPO_URL" "$REPO_DIR"
+            (cd "$REPO_DIR" && _lm_checkout_release_after_clone) || print_warning "Could not choose a release; installing the newest code from main"
         fi
     else
         print_success "Cloning repository to $REPO_DIR..."
         retry git clone "$REPO_URL" "$REPO_DIR"
+        (cd "$REPO_DIR" && _lm_checkout_release_after_clone) || print_warning "Could not choose a release; installing the newest code from main"
     fi
     
     # Verify repository is accessible
@@ -397,6 +514,7 @@ main() {
         sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 \
             LEDMATRIX_APT_UPDATED="${LEDMATRIX_APT_UPDATED:-0}" \
             LEDMATRIX_AUTO_UPDATE="${LEDMATRIX_AUTO_UPDATE:-}" \
+            LEDMATRIX_CHANNEL="${LEDMATRIX_CHANNEL:-}" \
             bash ./first_time_install.sh -y </dev/null
     fi
     INSTALL_EXIT_CODE=$?

scripts/install_dependencies_apt.py

@@ -4,13 +4,14 @@ Alternative dependency installer that tries apt packages first,
 then falls back to pip with --break-system-packages
 """
 
+import re
 import subprocess
 import sys
 import tempfile
 import warnings
 from collections import deque
 from pathlib import Path
-from typing import List, Tuple
+from typing import Dict, List, Tuple
 
 # How many trailing lines of a failed command's output to keep for the
 # end-of-run failure summary. Keeps the root cause near the end of the log,
@@ -81,6 +82,8 @@ def install_via_pip(package_name: str) -> Tuple[bool, str]:
 
     Returns (success, output).
     """
+    # pip knows PIL as Pillow; the others are asked for by their own name.
+    package_name = _dist_name(package_name)
     print(f"Installing {package_name} via pip...")
     success, output = _run([
         sys.executable, '-m', 'pip', 'install',
@@ -99,26 +102,66 @@ IMPORT_NAME_MAP = {
     'freetype-py': 'freetype',
 }
 
-# Minimum versions that must be met for an already-installed package to count
-# as satisfied. Debian Bookworm's python3-freetype is 2.3.0, below the
-# freetype-py>=2.5.1 pin in requirements.txt, so an import-only check would
-# wrongly skip the pip upgrade.
-MIN_VERSIONS = {
-    'freetype-py': (2, 5, 1),
+# The packages above are keyed by what main() lists; these are the ones whose
+# pip distribution name differs from that key.
+DIST_NAME_MAP = {
+    'PIL': 'Pillow',
 }
 
+REQUIREMENTS_FILE = Path(__file__).resolve().parent.parent / 'web_interface' / 'requirements.txt'
+
+
+def _version_tuple(text: str) -> tuple:
+    parts = []
+    for part in text.split('.'):
+        digits = ''.join(ch for ch in part if ch.isdigit())
+        if not digits:
+            break
+        parts.append(int(digits))
+    return tuple(parts)
+
+
+def _requirement_floors(path: Path = REQUIREMENTS_FILE) -> Dict[str, tuple]:
+    """``>=`` floors from a requirements file, keyed by lower-cased name.
+
+    The apt copies of these packages are older than the pins on both
+    supported releases -- Bookworm ships Flask and Werkzeug 2.2.2, Pillow 9.4,
+    requests 2.28, psutil 5.9, pytz 2022.7 and freetype-py 2.3; Trixie ships
+    Flask 3.1.1, Werkzeug 3.1.3, Pillow 11.1 and requests 2.32 --
+    so a package that merely imports is not enough. Read from the file rather
+    than copied here so the two cannot drift.
+    """
+    floors: Dict[str, tuple] = {}
+    try:
+        lines = path.read_text(encoding='utf-8').splitlines()
+    except OSError:
+        return floors
+    for line in lines:
+        match = re.match(r'\s*([A-Za-z0-9][A-Za-z0-9._-]*)[^#]*?>=\s*([0-9][0-9.]*)', line)
+        if match:
+            floors[match.group(1).lower()] = _version_tuple(match.group(2))
+    return floors
+
+
+def _dist_name(package_name: str) -> str:
+    return DIST_NAME_MAP.get(package_name, package_name)
+
+
+def _minimum_version(package_name: str) -> tuple:
+    """The required floor for ``package_name``, or () when there is none."""
+    return MIN_VERSIONS.get(_dist_name(package_name).lower(), ())
+
+
+# Minimum versions that must be met for an already-installed package to count
+# as satisfied.
+MIN_VERSIONS = _requirement_floors()
+
 
 def _installed_version_tuple(dist_name: str) -> tuple:
     """Return the installed distribution version as an int tuple, or () if unknown."""
     try:
         from importlib.metadata import version
-        parts = []
-        for part in version(dist_name).split('.'):
-            digits = ''.join(ch for ch in part if ch.isdigit())
-            if not digits:
-                break
-            parts.append(int(digits))
-        return tuple(parts)
+        return _version_tuple(version(dist_name))
     except Exception:
         return ()
 
@@ -134,9 +177,9 @@ def check_package_installed(package_name: str) -> bool:
             __import__(import_name)
         except ImportError:
             return False
-    minimum = MIN_VERSIONS.get(package_name)
+    minimum = _minimum_version(package_name)
     if minimum:
-        installed = _installed_version_tuple(package_name)
+        installed = _installed_version_tuple(_dist_name(package_name))
         if not installed or installed < minimum:
             print(f"{package_name} is installed but below the required "
                   f"{'.'.join(map(str, minimum))}; will upgrade via pip")
@@ -188,10 +231,11 @@ def main():
             continue
 
         # Try apt first, then pip. An apt install only counts if it also
-        # satisfies any minimum version (Debian's python3-freetype can be
-        # older than the freetype-py pin), otherwise fall through to pip.
+        # satisfies the requirements floor (the apt copies of most of these
+        # are older than the pins on both Bookworm and Trixie), otherwise
+        # fall through to pip.
         ok, apt_output = install_via_apt(package)
-        if ok and package in MIN_VERSIONS and not check_package_installed(package):
+        if ok and _minimum_version(package) and not check_package_installed(package):
             ok = False
             apt_output = f"apt version of {package} is below the required minimum"
         if not ok:

scripts/utils/auto_update_verify.py

@@ -15,7 +15,8 @@ The updater leaves data/auto_update_pending.json:
 
     {"status": "pending", "old_head": ..., "new_head": ...,
      "old_ref": "main" | "" (detached) | absent (older updaters),
-     "display_was_active": bool, "dependency_failures": [...], "created_at": ...}
+     "display_was_active": bool, "dependency_failures": [...],
+     "units_refreshed": bool (absent from older updaters), "created_at": ...}
 
 This moves its status to "verifying" and then to one of "success",
 "rolled_back" or "rollback_failed", with "reason" and "detail" saying why.
@@ -74,6 +75,10 @@ BASH_CANDIDATES = ('/usr/bin/bash', '/bin/bash')
 #: ...and, like it, moves to the next one only when sudo refused the command
 #: line (permission_utils.SUDO_REFUSAL_PHRASES), never after pip itself ran.
 SUDO_REFUSAL_PHRASES = ('a password is required', 'is not allowed to run', 'no tty present')
+#: The root-owned helper that installed the update's systemd units
+#: (web_interface/unit_refresh.py); ``--restore`` puts the previous ones back.
+REFRESH_UNITS_PATH = '/usr/local/sbin/ledmatrix-refresh-units'
+UNIT_RESTORE_TIMEOUT_SECONDS = 90
 
 #: The longest one health check can take: restart and wait, roll back
 #: (diff, reset, reinstalls), restart and wait again. A wait's last poll can
@@ -81,7 +86,8 @@ SUDO_REFUSAL_PHRASES = ('a password is required', 'is not allowed to run', 'no t
 _WAIT_WORST_SECONDS = (HEALTH_TIMEOUT_SECONDS + STABLE_SECONDS + WEB_CHECK_TIMEOUT_SECONDS
                        + 2 * SYSTEMCTL_QUERY_TIMEOUT_SECONDS + POLL_SECONDS)
 WORST_CASE_SECONDS = (2 * (2 * RESTART_TIMEOUT_SECONDS + _WAIT_WORST_SECONDS)
-                      + GIT_TIMEOUT_SECONDS + GIT_RESET_TIMEOUT_SECONDS + PIP_BUDGET_SECONDS)
+                      + GIT_TIMEOUT_SECONDS + GIT_RESET_TIMEOUT_SECONDS + UNIT_RESTORE_TIMEOUT_SECONDS
+                      + PIP_BUDGET_SECONDS)
 
 #: What a command that could not run at all reports: its callers only read
 #: these three fields, the same ones a completed subprocess has.
@@ -300,12 +306,26 @@ class Verifier:
         if result.returncode != 0:
             return False, (f'"git reset --hard {old}" failed: '
                            f'{(result.stderr or result.stdout or "").strip()}')
+        notes = []
+        # The update also installed its own systemd units: put the previous
+        # ones back before anything restarts onto the rolled-back code.
+        if pending.get('units_refreshed') and not self.restore_units():
+            notes.append('restoring the previous service settings failed; run '
+                         '"sudo ./scripts/install/install_service.sh" in the LEDMatrix folder')
         deadline = self.clock() + PIP_BUDGET_SECONDS
         failed = [rel for rel in requirements if not self.install_requirements(rel, deadline)]
         if failed:
-            return True, ('reinstalling the previous dependencies from ' + ', '.join(failed)
-                          + ' failed; run Install Base Requirements from the Tools tab')
-        return True, ''
+            notes.append('reinstalling the previous dependencies from ' + ', '.join(failed)
+                         + ' failed; run Install Base Requirements from the Tools tab')
+        return True, '; '.join(notes)
+
+    def restore_units(self):
+        """Reinstall the systemd units the update replaced. True on success."""
+        result = self._run(['sudo', '-n', REFRESH_UNITS_PATH, '--restore'],
+                           timeout=UNIT_RESTORE_TIMEOUT_SECONDS)
+        if result.returncode != 0:
+            self.log(f'restoring the previous systemd units failed: {(result.stderr or "").strip()}')
+        return result.returncode == 0
 
     # -- the check itself -------------------------------------------------
 

src/__init__.py

@@ -4,5 +4,5 @@ LEDMatrix Display System
 Core source package for the LED Matrix Display project.
 """
 
-__version__ = "3.8.0"
+__version__ = "3.8.2"
 

src/adaptive_layout.py

@@ -28,6 +28,7 @@ freetype.Face, so it drops straight into DisplayManager.draw_text().
 """
 
 import logging
+import weakref
 from collections import OrderedDict
 from dataclasses import dataclass
 from typing import Any, Dict, List, Optional, Sequence, Tuple, Union
@@ -332,9 +333,10 @@ class LayoutContext:
         # a plugin fitting changing text (a live game clock, a ticker) on a
         # 24/7 service would otherwise grow this without bound.
         self._fit_cache: "OrderedDict[Any, FitResult]" = OrderedDict()
-        # LRU-bounded (images are big). Entries hold a strong reference to
-        # the source image when keyed by id() so the id can't be recycled
-        # out from under the cache.
+        # LRU-bounded (images are big). An id()-keyed entry watches its
+        # source image through a weak reference and is dropped when the
+        # source is freed (see fit_image), so the id can't be recycled out
+        # from under the cache and the cache never keeps the source alive.
         self._image_cache: "OrderedDict[Any, Tuple[Any, Any]]" = OrderedDict()
 
     _IMAGE_CACHE_MAX = 64
@@ -536,8 +538,15 @@ class LayoutContext:
         cached per (image, box size, options) for this panel size.
 
         Prefer a stable ``cache_key`` (e.g. "logo:KC") for images that get
-        reloaded — the default id()-based key is safe (the entry pins the
-        source image) but misses across reloads of the same content.
+        reloaded — the default id()-based key misses across reloads of the
+        same content.
+
+        An id()-keyed entry lives only as long as its source image: it holds
+        a weak reference and is dropped when the source is freed. It used to
+        pin the source instead, so a plugin passing a freshly loaded image
+        each frame (``draw_image(Image.open(path), box)``, the documented
+        one-liner) never hit and kept the last 64 sources alive — ~64MB for
+        500x500 RGBA team logos, the median size under assets/sports.
         """
         from src.adaptive_images import fit_image as _fit_image
 
@@ -547,18 +556,31 @@ class LayoutContext:
         key = ("image", identity, img.size, box_w, box_h, mode,
                crop_to_ink, anchor, resample_name, upscale)
 
-        cached = self._image_cache.get(key)
-        if cached is not None:
-            self._image_cache.move_to_end(key)
+        cache = self._image_cache
+        cached = cache.get(key)
+        # An id()-keyed hit must still be this very image; the callback below
+        # normally removes a dead source's entry before its id can recur.
+        if cached is not None and (cache_key is not None or cached[1]() is img):
+            cache.move_to_end(key)
             return cached[0]
 
         result = _fit_image(img, (box_w, box_h), mode=mode,
                             crop_to_ink=crop_to_ink, anchor=anchor,
                             resample=resample, upscale=upscale)
-        # Pin the source only for id()-keyed entries (see docstring).
-        self._image_cache[key] = (result, img if cache_key is None else None)
-        while len(self._image_cache) > self._IMAGE_CACHE_MAX:
-            self._image_cache.popitem(last=False)
+        source = None
+        if cache_key is None:
+            def _forget(ref: Any, key: Any = key) -> None:
+                entry = cache.get(key)
+                if entry is not None and entry[1] is ref:
+                    cache.pop(key, None)
+            try:
+                source = weakref.ref(img, _forget)
+            except TypeError:
+                # Not weak-referenceable: pin it, as before.
+                source = lambda img=img: img  # noqa: E731
+        cache[key] = (result, source)
+        while len(cache) > self._IMAGE_CACHE_MAX:
+            cache.popitem(last=False)
         return result
 
     # ---- text utilities ------------------------------------------------

src/background_data_service.py

@@ -34,6 +34,7 @@ from src.common.fetch_service import (
     plugin_scope,
     share_connection_pool,
 )
+from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
 from src.common.espn_dates import (
     RANGE_RETRY_SECONDS,
     _note_range_rejected,
@@ -83,6 +84,10 @@ class FetchRequest:
     # the cache with the callbacks suppressed -- joiners waiting forever for a
     # fetch that did, in fact, succeed.
     commit_claimed: bool = False
+    # Trim an ESPN scoreboard response before it is cached and delivered
+    # (src/common/espn_payload.py). Set by whoever created the request; a
+    # submitter that joins the fetch gets the same payload.
+    slim_payload: bool = True
     result: Optional[Any] = None
     error: Optional[str] = None
     # The plugin that submitted the request, so the fetch service counts the
@@ -249,7 +254,8 @@ class BackgroundDataService:
                            timeout: Optional[int] = None,
                            max_retries: int = 3,
                            priority: int = 1,
-                           callback: Optional[Callable] = None) -> str:
+                           callback: Optional[Callable] = None,
+                           slim_payload: bool = True) -> str:
         """
         Submit a background fetch request.
         
@@ -265,6 +271,11 @@ class BackgroundDataService:
             priority: Accepted for compatibility and ignored; requests run in
                 submission order.
             callback: Optional callback function when request completes
+            slim_payload: Drop the parts of an ESPN scoreboard response no
+                scoreboard reads (stat leaders, athlete cards, links,
+                headlines, highlights) before caching it; see
+                src/common/espn_payload.py. Only ESPN /scoreboard URLs are
+                touched. Pass False to cache the response whole.
             
         Returns:
             Request ID for tracking the fetch operation
@@ -336,6 +347,7 @@ class BackgroundDataService:
             priority=priority,
             callback=callback,
             owner=owner,
+            slim_payload=slim_payload,
         )
         
         with self._lock:
@@ -497,6 +509,13 @@ class BackgroundDataService:
                 )
                 return result
 
+            # Most of an ESPN scoreboard response is never drawn, and the
+            # cached copy stays parsed in the memory tier while it is fresh.
+            # Trimmed before the write so the cache, request.result and the
+            # callbacks all see the same payload. See src/common/espn_payload.py.
+            if request.slim_payload and is_espn_scoreboard_url(request.url):
+                slim_scoreboard_payload(data)
+
             # Cache the data
             self.cache_manager.set(request.cache_key, data)
             

src/backup_manager.py

@@ -213,8 +213,9 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
     The plugins are the ``manifest.json`` files in the configured plugin
     directory (see :func:`_plugins_directory`), with the manifest's version;
     ``enabled`` is config.json's flag by the display's rule (a missing flag
-    is disabled). A restore reinstalls every listed plugin and takes enabled
-    state from the restored config.json, so ``enabled`` is informational.
+    is disabled). A restore installs each listed plugin that is missing and
+    takes enabled state from the restored config.json, so ``enabled`` is
+    informational.
 
     ``data/plugin_state.json`` is not read: it only ever repeated config's
     enabled flags and the manifests' versions, and is retired (nothing

src/base_odds_manager.py

@@ -20,6 +20,7 @@ from typing import Dict, Any, Optional, List, cast
 
 from src.common.api_helper import DEFAULT_HTTP_HEADERS
 from src.common.fetch_service import fetch_get, share_connection_pool
+from src.common.json_body import response_json
 
 
 
@@ -146,7 +147,7 @@ class BaseOddsManager:
             if _is_no_odds_marker(cached_data):
                 self.logger.debug("Cached no-odds marker for %s", cache_key)
                 return None
-            self.logger.debug(f"Using cached odds from ESPN for {cache_key}")
+            self.logger.debug("Using cached odds from ESPN for %s", cache_key)
             return cached_data
 
         if time.monotonic() < self._skip_network_until:
@@ -159,7 +160,7 @@ class BaseOddsManager:
                 self._skip_network_until - time.monotonic())
             return None
 
-        self.logger.debug(f"Cache miss - fetching fresh odds from ESPN for {cache_key}")
+        self.logger.debug("Cache miss - fetching fresh odds from ESPN for %s", cache_key)
 
         try:
             # Map league names to ESPN API format
@@ -173,26 +174,30 @@ class BaseOddsManager:
             
             espn_league = league_mapping.get(league, league)
             url = f"{self.base_url}/{sport}/leagues/{espn_league}/events/{event_id}/competitions/{event_id}/odds"
-            self.logger.debug(f"Requesting odds from URL: {url}")
+            self.logger.debug("Requesting odds from URL: %s", url)
             
             # The response cache may answer only inside this caller's own
             # interval, the age at which its cached odds expire anyway.
             response = fetch_get(self.session, url, timeout=self.request_timeout,
                                  cache_max_age=interval)
             response.raise_for_status()
-            raw_data = response.json()
+            raw_data = response_json(response)
 
             self._skip_network_until = 0.0   # reachable again
 
-            self.logger.debug(f"Received raw odds data from ESPN: {json.dumps(raw_data, indent=2)}")
+            # Guarded, not just %-style: the json.dumps argument would still be
+            # built for every response with DEBUG off.
+            if self.logger.isEnabledFor(logging.DEBUG):
+                self.logger.debug("Received raw odds data from ESPN: %s",
+                                  json.dumps(raw_data, indent=2))
             
             odds_data = self._extract_espn_data(raw_data)
             if odds_data:
-                self.logger.debug(f"Successfully extracted odds data: {odds_data}")
+                self.logger.debug("Successfully extracted odds data: %s", odds_data)
                 self.cache_manager.set(cache_key, odds_data, ttl=interval)
-                self.logger.debug(f"Saved odds data to cache for {cache_key} with TTL {interval}s")
+                self.logger.debug("Saved odds data to cache for %s with TTL %ss", cache_key, interval)
             else:
-                self.logger.debug(f"No odds data available for {cache_key}")
+                self.logger.debug("No odds data available for %s", cache_key)
                 # Cache the absence too, so the game is not re-requested
                 # on every update until the interval passes.
                 self.cache_manager.set(cache_key, {"no_odds": True}, ttl=interval)
@@ -226,12 +231,12 @@ class BaseOddsManager:
         Returns:
             Formatted odds data dictionary or None
         """
-        self.logger.debug(f"Extracting ESPN odds data. Data keys: {list(data.keys())}")
+        self.logger.debug("Extracting ESPN odds data. Data keys: %s", list(data.keys()))
         
         if "items" in data and data["items"]:
-            self.logger.debug(f"Found {len(data['items'])} items in odds data")
+            self.logger.debug("Found %d items in odds data", len(data['items']))
             item = data["items"][0]
-            self.logger.debug(f"First item keys: {list(item.keys())}")
+            self.logger.debug("First item keys: %s", list(item.keys()))
             
             # The ESPN API returns odds data directly in the item, not in a
             # providers array. ESPN sends explicit JSON nulls for absent
@@ -254,13 +259,17 @@ class BaseOddsManager:
                                     .get("pointSpread") or {}).get("value")
                 }
             }
-            self.logger.debug(f"Returning extracted odds data: {json.dumps(extracted_data, indent=2)}")
+            if self.logger.isEnabledFor(logging.DEBUG):
+                self.logger.debug("Returning extracted odds data: %s",
+                                  json.dumps(extracted_data, indent=2))
             return extracted_data
         
         # Check if this is a valid empty response or an unexpected structure
         if "count" in data and data["count"] == 0 and "items" in data and data["items"] == []:
             # This is a valid empty response - no odds available for this game
-            self.logger.debug(f"No odds available for this game. Response: {json.dumps(data, indent=2)}")
+            if self.logger.isEnabledFor(logging.DEBUG):
+                self.logger.debug("No odds available for this game. Response: %s",
+                                  json.dumps(data, indent=2))
             return None
         else:
             # This is an unexpected response structure

src/cache/disk_cache.py

@@ -4,6 +4,7 @@ Disk Cache
 Handles persistent disk-based caching with atomic writes and error recovery.
 """
 
+import hashlib
 import json
 import math
 import os
@@ -14,7 +15,7 @@ import tempfile
 import logging
 import threading
 import zlib
-from typing import Dict, Any, Optional, Protocol
+from typing import Dict, Any, Optional, Protocol, Tuple
 from datetime import datetime
 
 from src.common.path_safety import safe_path_component
@@ -31,6 +32,35 @@ except ImportError:  # pragma: no cover - exercised on hosts without the wheel
 # useful, and a half-written file was never useful.
 _ORPHAN_TEMP_MAX_AGE_SECONDS = 3600
 
+# Longest key, in UTF-8 bytes, used verbatim as a filename stem. ext4 caps a
+# name at 255 bytes and set()'s temp file is ".<stem>.json.<8 random>", 15
+# bytes longer than the stem, so anything near the cap could never be written:
+# the calendar plugin's key joins every calendar id and passed 300 bytes on a
+# real install, failing every write with ENAMETOOLONG. Longer keys keep this
+# many bytes as a readable prefix and end in a hash of the whole key.
+_MAX_KEY_FILENAME_BYTES = 200
+_KEY_HASH_CHARS = 16
+
+
+def _filename_stem(key: str) -> str:
+    """The filename stem for a key that is already a safe path component.
+
+    Short keys are used as they are, so every file already on disk keeps its
+    name. A long one becomes its first bytes plus a hash of the full key: the
+    prefix keeps the stem recognisable (and keeps the data-type words that
+    cleanup's retention lookup reads from it), the hash keeps two keys that
+    share a long prefix apart. The result is itself short, so a stem read back
+    from a filename -- which is how the web UI names a key it deletes -- maps to
+    the same file.
+    """
+    encoded = key.encode('utf-8')
+    if len(encoded) <= _MAX_KEY_FILENAME_BYTES:
+        return key
+    digest = hashlib.sha256(encoded).hexdigest()[:_KEY_HASH_CHARS]
+    keep = _MAX_KEY_FILENAME_BYTES - _KEY_HASH_CHARS - 1
+    prefix = encoded[:keep].decode('utf-8', errors='ignore')
+    return f"{prefix}-{digest}"
+
 
 
 class CacheStrategyProtocol(Protocol):
@@ -111,18 +141,91 @@ _HEAD_RE = re.compile(
 )
 
 
-def _stale_from_head(head: bytes, max_age: Optional[int], now: float) -> bool:
+# UNCHANGED RE-SAVES: THE FILE'S MTIME CARRIES THE NEWER TIMESTAMP
+# ----------------------------------------------------------------
+# Plugins re-save unchanged API data every update cycle, and every one of
+# those saves was a full rewrite on the SD card. DiskCache.set skips the write
+# when the payload matches the last one it wrote for the key -- but
+# CacheManager.set stamps each record with time.time(), so for set() the
+# payload never matched and the skip never fired.
+#
+# The digest now leaves out a header-first record's timestamp, so an unchanged
+# set() is skipped. What the skip must not do is make the record look older
+# than it is: the timestamp inside the file is from the last real write, and
+# a reader in another process (the web interface, with memory_ttl=0) or after
+# a restart would call fresh data stale. So the newer timestamp goes where it
+# costs no data write -- the file's mtime -- and readers take a record's age
+# from the newer of the two. The invariant that makes that safe:
+#
+#   a file's mtime is the timestamp of the newest record saved for its key
+#
+#   real write  mtime is set to the record's own timestamp, so a record saved
+#               with an old timestamp (data as of some earlier time) cannot
+#               borrow freshness from the moment it hit the disk
+#   skip        mtime is set to the skipped record's timestamp -- exactly what
+#               a rewrite would have stored, without the rewrite
+#
+# Readers of the on-disk timestamp, all of which go through _effective_timestamp:
+# DiskCache.get (the header check and the full parse; it also returns the
+# record with 'timestamp' set to the effective value, so CacheManager.get's
+# max_age path, the memory tier hydrated from disk, and any plugin reading
+# record['timestamp'] all see it). Readers that use mtime alone already see the
+# newer value: the retention sweep below, CacheManager.list_cache_files (the
+# web UI's cache list). Nothing else opens cache files: web_interface and
+# scripts reach them only through CacheManager.
+#
+# Something other than this class can also move an mtime forward -- a copy
+# without -p, an rsync without -t, a `touch`. (backup_manager.py does not
+# back up or restore the cache directory, so the in-tree restore cannot.) That
+# must not make old data fresh, so the lift is bounded: a reader never takes
+# the mtime as more than _MAX_TIMESTAMP_LIFT past the embedded timestamp, and
+# set() rewrites the file for real once a skip would need more than that, so
+# an honest lift never reaches the bound. A file copied a day after it was
+# written therefore reads at most an hour fresher than its contents say, and a
+# 30-second live-score record from yesterday stays stale. CacheManager.set
+# records written before this change have mtime == write time == embedded
+# timestamp, give or take the write itself, and read exactly as before; a
+# file an older version wrote or touched later than its embedded timestamp
+# says reads at most the same hour fresher, once, until it is next saved.
+
+#: Longest a skipped write may stand in for a real one, and so the furthest a
+#: file's mtime is ever trusted past the record's own timestamp. Unchanged data
+#: is rewritten at least this often, at most once an hour per key instead of
+#: once per update cycle.
+_MAX_TIMESTAMP_LIFT = 3600.0
+
+
+def _record_timestamp(value: Any) -> Optional[float]:
+    """A record's timestamp as a finite float, or None if it has no usable one."""
+    if isinstance(value, bool) or not isinstance(value, (int, float)):
+        return None
+    value = float(value)
+    return value if math.isfinite(value) else None
+
+
+def _effective_timestamp(embedded: float, mtime: Optional[float]) -> float:
+    """When a record was last saved: its timestamp, or the file's mtime if a
+    later unchanged save moved that forward -- never by more than
+    _MAX_TIMESTAMP_LIFT. See "UNCHANGED RE-SAVES" above."""
+    if mtime is None:
+        return embedded
+    return max(embedded, min(mtime, embedded + _MAX_TIMESTAMP_LIFT))
+
+
+def _stale_from_head(head: bytes, max_age: Optional[int], now: float,
+                     mtime: Optional[float] = None) -> bool:
     """True when a record's header alone shows it has expired.
 
     Mirrors the expiry rule in DiskCache.get: a per-entry ttl wins over the
     caller's max_age, and no limit at all means never stale. False whenever the
-    header cannot be read, so the full parse decides as it always did.
+    header cannot be read, so the full parse decides as it always did. ``mtime``
+    is the file's, which may carry a newer save than the header does.
     """
     match = _HEAD_RE.match(head)
     if not match:
         return False
     try:
-        timestamp = float(match.group(1))
+        timestamp = _effective_timestamp(float(match.group(1)), mtime)
         limit = max_age
         if match.group(2) is not None:
             ttl = float(match.group(2))
@@ -179,7 +282,7 @@ else:
 # --------------------------------------------
 # The display service runs as root and the web interface as the installing
 # user, and the web interface reads records only the display writes
-# (display_current_state, display_on_demand_state, plugin_metrics:*). Files are
+# (display_current_state, display_on_demand_state, plugin_metrics_snapshot). Files are
 # written 0660, so the web interface can read one only through its group.
 #
 # The installers rely on the directory's setgid bit to set that group. That is
@@ -248,11 +351,14 @@ class DiskCache:
         self.cache_dir = cache_dir
         self.logger = logger or logging.getLogger(__name__)
         self._lock = threading.Lock()
-        # key -> adler32 of the last payload successfully written to the
-        # primary cache path; lets set() skip rewriting identical data
-        # (per-process only — worst case another process rewrites, never
-        # a missed write). Guarded by _lock.
-        self._write_digests: Dict[str, int] = {}
+        # key -> what set() last put at the primary cache path: the adler32 of
+        # the payload (less a header-first timestamp), the timestamp the file
+        # holds (None for records without one), and the file's inode and size.
+        # Lets set() skip rewriting identical data. Per-process only, and the
+        # inode/size check means another process's write is never mistaken
+        # for ours -- worst case a redundant write, never a missed one.
+        # Guarded by _lock.
+        self._write_digests: Dict[str, Tuple[int, Optional[float], int, int]] = {}
     
     def get_cache_path(self, key: str) -> Optional[str]:
         """
@@ -267,6 +373,8 @@ class DiskCache:
         derives them), so rejecting anything with a path component turns
         away only inputs that could never have been written here.
 
+        A key too long to be a filename is shortened by _filename_stem.
+
         Args:
             key: Cache key
 
@@ -280,7 +388,7 @@ class DiskCache:
         if safe_key is None:
             self.logger.warning("Rejected unsafe cache key %r", key)
             return None
-        return os.path.join(self.cache_dir, f"{safe_key}.json")
+        return os.path.join(self.cache_dir, f"{_filename_stem(safe_key)}.json")
     
     def get(self, key: str, max_age: Optional[int] = 300) -> Optional[Dict[str, Any]]:
         """
@@ -301,31 +409,40 @@ class DiskCache:
         try:
             with self._lock:
                 with open(cache_path, 'rb') as f:
+                    # The open file's mtime, not the path's: the file a skipped
+                    # write touched is the one being read.
+                    mtime = os.fstat(f.fileno()).st_mtime
                     # Decide staleness from the header before paying for the
                     # parse. A stale read is the common case for the biggest
                     # records (a season schedule is re-fetched when its cache
                     # expires), and parsing 53MB to throw it away held the GIL
                     # for ~1.8s -- a visible freeze on the panel.
-                    if _stale_from_head(f.read(_HEAD_BYTES), max_age, time.time()):
+                    if _stale_from_head(f.read(_HEAD_BYTES), max_age, time.time(), mtime):
                         return None
                     f.seek(0)
                     record = _loads(f.read())
 
-            # Determine record timestamp (prefer embedded, else file mtime)
+            # Determine record timestamp: the embedded one, moved forward by a
+            # later unchanged save if there was one (see "UNCHANGED RE-SAVES"),
+            # else the file mtime.
             record_ts = None
             if isinstance(record, dict):
                 record_ts = record.get('timestamp')
             if record_ts is None:
-                try:
-                    record_ts = os.path.getmtime(cache_path)
-                except OSError:
-                    record_ts = None
-            
-            if record_ts is not None:
-                try:
-                    record_ts = float(record_ts)
-                except (TypeError, ValueError):
-                    record_ts = None
+                record_ts = mtime
+            else:
+                embedded_ts = _record_timestamp(record_ts)
+                if embedded_ts is None:
+                    try:
+                        record_ts = float(record_ts)
+                    except (TypeError, ValueError):
+                        record_ts = None
+                else:
+                    record_ts = _effective_timestamp(embedded_ts, mtime)
+                    if record_ts != embedded_ts:
+                        # Hand the record back as a rewrite would have left it,
+                        # so callers that age it themselves agree with us.
+                        record['timestamp'] = record_ts
 
             now = time.time()
 
@@ -403,7 +520,12 @@ class DiskCache:
             self.logger.warning("Cache data for key '%s' not serializable: %s", key, e)
             return
 
-        digest = zlib.adler32(payload)
+        timestamp = _record_timestamp(data.get('timestamp')) if isinstance(data, dict) else None
+        # A header-first record (CacheManager.set's layout) is compared without
+        # its timestamp, which differs on every save; see "UNCHANGED RE-SAVES".
+        # Any other layout is compared whole, as before.
+        head = _HEAD_RE.match(payload) if timestamp is not None else None
+        digest = zlib.adler32(memoryview(payload)[head.end(1):] if head else payload)
 
         try:
             # Atomic write to avoid partial/corrupt files
@@ -411,16 +533,10 @@ class DiskCache:
                 # Skip the disk entirely when this exact payload was already
                 # written for this key (plugins re-save unchanged API data
                 # every update cycle — each write is real SD-card wear).
-                # Refresh the file mtime so records that rely on it for TTL
-                # (no embedded 'timestamp') don't expire early; a metadata
-                # touch is journal-cheap compared to rewriting the data.
-                if self._write_digests.get(key) == digest:
-                    try:
-                        os.utime(cache_path, None)
-                        return
-                    except OSError:
-                        # File vanished or perms changed — fall through and write
-                        self._write_digests.pop(key, None)
+                # A metadata touch is journal-cheap compared to rewriting
+                # the data.
+                if self._skip_unchanged(key, cache_path, digest, timestamp):
+                    return
 
                 tmp_dir = os.path.dirname(cache_path)
                 # Try to create temp file in cache directory first
@@ -458,7 +574,7 @@ class DiskCache:
                                 # opened it in between was refused.
                                 _share_open_file(tmp_file.fileno(), _shared_group(tmp_dir))
                             os.replace(tmp_path, cache_path)
-                            self._write_digests[key] = digest
+                            self._remember_write(key, cache_path, digest, timestamp)
                         finally:
                             if os.path.exists(tmp_path):
                                 try:
@@ -471,13 +587,13 @@ class DiskCache:
                             with open(cache_path, 'wb') as cache_file:
                                 cache_file.write(payload)
                                 _share_open_file(cache_file.fileno(), _shared_group(tmp_dir))
-                            self._write_digests[key] = digest
+                            self._remember_write(key, cache_path, digest, timestamp)
                             self.logger.debug("Wrote cache for %s directly (non-atomic)", key)
                         except (IOError, OSError, PermissionError) as write_error:
                             # If direct write also fails, try fallback location
                             self.logger.warning("Direct write failed for key '%s' to %s: %s", key, cache_path, write_error)
                             raise  # Re-raise to trigger fallback logic
-                except (IOError, OSError, PermissionError):
+                except (IOError, OSError, PermissionError) as primary_error:
                     # Attempt one-time fallback write to user's home cache directory
                     try:
                         # Try user's home cache directory as fallback
@@ -503,11 +619,14 @@ class DiskCache:
                         self.logger.debug("Fallback cache write also failed for key '%s': %s", key, e2)
                     
                     # If all write attempts failed, log warning but don't raise exception
-                    # Cache is a performance optimization, not critical for operation
+                    # Cache is a performance optimization, not critical for operation.
+                    # Name the real error: this used to say "permission denied"
+                    # whatever happened, which sent a too-long filename off to
+                    # be debugged as a directory-ownership problem.
                     self.logger.warning(
-                        "Could not write cache for key '%s' to %s (permission denied). "
+                        "Could not write cache for key '%s' to %s (%s). "
                         "Cache will be unavailable for this key, but application will continue.",
-                        key, cache_path
+                        key, cache_path, primary_error.strerror or primary_error
                     )
                     return  # Exit gracefully without raising exception
         
@@ -520,6 +639,66 @@ class DiskCache:
             )
             return  # Exit gracefully without raising exception
     
+    def _skip_unchanged(self, key: str, cache_path: str, digest: int,
+                        timestamp: Optional[float]) -> bool:
+        """Stand in for a write of an unchanged record by touching the file.
+
+        True when the file already holds this record bar its timestamp and the
+        touch landed; False means write it. The touch sets mtime to the
+        record's timestamp -- what a rewrite would have stored -- or to now
+        for a record without one, whose age readers already take from mtime.
+        Caller holds _lock.
+        """
+        last = self._write_digests.get(key)
+        if last is None or last[0] != digest:
+            return False
+        _, written_ts, ino, size = last
+        if (timestamp is None) != (written_ts is None):
+            return False
+        if timestamp is not None and written_ts is not None:
+            # Never backwards (a rewrite would make the record older), and
+            # never further than readers will trust the mtime: past that the
+            # record is rewritten, so its own timestamp catches up.
+            if not written_ts <= timestamp <= written_ts + _MAX_TIMESTAMP_LIFT:
+                return False
+        try:
+            st = os.stat(cache_path)
+            if (st.st_ino, st.st_size) != (ino, size):
+                # Replaced since our write (another process, a restore):
+                # its contents are not the ones the digest describes.
+                self._write_digests.pop(key, None)
+                return False
+            # Setting an explicit time needs the file's owner; a file someone
+            # else wrote fails here and is rewritten (as our own file) instead.
+            os.utime(cache_path, None if timestamp is None else (timestamp, timestamp))
+            return True
+        except OSError:
+            # File vanished or perms changed — fall through and write
+            self._write_digests.pop(key, None)
+            return False
+
+    def _remember_write(self, key: str, cache_path: str, digest: int,
+                        timestamp: Optional[float]) -> None:
+        """After a real write: pin mtime to the record's timestamp and note
+        what was written, so the next unchanged save can be skipped.
+
+        Pinning keeps a record saved with an older timestamp from looking as
+        fresh as the moment it was written (see "UNCHANGED RE-SAVES"); for
+        CacheManager.set's records the two differ only by the write itself.
+        A timestamp in the future is left alone, mtime already being older.
+        Never raises: the data is on disk, and anything failing here only
+        costs the next save its skip. Caller holds _lock.
+        """
+        self._write_digests.pop(key, None)
+        try:
+            if timestamp is not None and timestamp <= time.time():
+                os.utime(cache_path, (timestamp, timestamp))
+            st = os.stat(cache_path)
+        except OSError as e:
+            self.logger.debug("Could not pin mtime of %s: %s", cache_path, e)
+            return
+        self._write_digests[key] = (digest, timestamp, st.st_ino, st.st_size)
+
     def clear(self, key: Optional[str] = None) -> None:
         """
         Clear cache entry or all entries.

src/cache_manager.py

@@ -28,7 +28,7 @@ import os
 import time
 from datetime import datetime
 import pytz
-from typing import Any, Dict, List, Optional
+from typing import Any, Dict, List, Optional, Tuple
 import logging
 import threading
 import tempfile
@@ -43,6 +43,72 @@ from src.logging_config import get_logger
 # it from either path.
 from src.cache.disk_cache import DateTimeEncoder  # noqa: F401 - deliberate re-export
 
+# CacheManager.config_manager not built yet (None means "not available").
+_UNSET: Any = object()
+
+
+def _outlived(record: Any, max_age: Optional[float], now: float) -> bool:
+    """Whether a record's own timestamp puts it past max_age.
+
+    The memory tier times an entry from when it was put there, and a record
+    loaded from disk is put there when it is read, not when it was written: a
+    record 290 s old, read after a restart, could be served for another
+    max_age from memory. This is the age check DiskCache.get makes, with the
+    same rule that a stored ttl wins over the caller's max_age. A record that
+    carries no timestamp is left to the memory tier's own clock.
+    """
+    if not isinstance(record, dict):
+        return False
+    stored_ttl = record.get('ttl')
+    if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
+            and stored_ttl >= 0:
+        max_age = stored_ttl
+    stamp = record.get('timestamp')
+    if max_age is None or stamp is None or isinstance(stamp, bool):
+        return False
+    try:
+        return now - float(stamp) > max_age
+    except (TypeError, ValueError):
+        return False
+
+
+_NOT_SEEN: Any = object()
+
+
+class MailboxWatch:
+    """Tells the poller of a mailbox key whether its file changed since the
+    last look, from one stat() (:meth:`CacheManager.file_signature`).
+
+    The display polls the mailboxes the web interface falls back to. Reading
+    one is an open and a JSON parse; with this a poll that finds the same file
+    (or none) costs a stat, and the file is read only after a new write. A
+    cache without ``file_signature`` (a test double) is read every time.
+    """
+
+    def __init__(self, key: str):
+        self.key = key
+        self._seen: Any = _NOT_SEEN
+
+    def changed(self, cache_manager: Any) -> bool:
+        """True when the poller should read the key now."""
+        signature = getattr(cache_manager, 'file_signature', None)
+        sig = signature(self.key) if callable(signature) else _NOT_SEEN
+        if sig is not None and not isinstance(sig, tuple):
+            return True   # cannot tell: read it
+        if sig is None:
+            self._seen = None
+            return False   # no file, nothing to read
+        if sig == self._seen:
+            return False
+        self._seen = sig
+        return True
+
+    def forget(self) -> None:
+        """Read the key on the next poll even if its file has not changed
+        (the last read failed)."""
+        self._seen = _NOT_SEEN
+
+
 class CacheManager:
     """Manages caching of API responses to reduce API calls."""
 
@@ -73,21 +139,19 @@ class CacheManager:
             self.logger.error("Could not find or create a writable cache directory. Caching will be disabled.")
             self.cache_dir = None
 
-        # Initialize config manager for sport-specific intervals
-        try:
-            from src.config_manager import ConfigManager
-            self.config_manager: Optional[Any] = ConfigManager()
-            self.config_manager.load_config()
-        except ImportError:
-            self.config_manager: Optional[Any] = None
-            self.logger.warning("ConfigManager not available, using default cache intervals")
+        # The config manager is built on first use of self.config_manager; see
+        # the property. Nothing in the cache reads it any more.
+        self._config_manager: Any = _UNSET
+        self._config_manager_lock = threading.Lock()
 
         # Initialize cache components using composition
         self._memory_cache_component = MemoryCache(
             max_size=default_max_size(), cleanup_interval=300.0
         )
         self._disk_cache_component = DiskCache(cache_dir=self.cache_dir, logger=self.logger)
-        self._strategy_component = CacheStrategy(config_manager=self.config_manager, logger=self.logger)
+        # No config manager: CacheStrategy keeps the parameter for callers but
+        # reads nothing from it, and passing ours would build it eagerly.
+        self._strategy_component = CacheStrategy(logger=self.logger)
         self._metrics_component = CacheMetrics(logger=self.logger)
         
         # Disk cleanup configuration
@@ -115,6 +179,44 @@ class CacheManager:
         if self.cache_dir:
             self.start_cleanup_thread()
 
+    @property
+    def config_manager(self) -> Optional[Any]:
+        """A loaded ConfigManager, built the first time it is asked for.
+
+        Every CacheManager used to build one and load the whole config in
+        __init__, for a cache strategy that stopped reading it -- startup paid
+        a config load (and the web interface another) per manager for nothing.
+        It is still public: the sports plugins resolve the global timezone and
+        display settings through ``cache_manager.config_manager``, and they get
+        the same object they always did, on first access instead of at
+        construction. None when ConfigManager cannot be imported, as before.
+        Assigning replaces it, as assigning the attribute always did.
+        """
+        # getattr: a manager made with __new__ (some tests) has no slot yet.
+        value = getattr(self, '_config_manager', _UNSET)
+        if value is not _UNSET:
+            return value
+        lock = getattr(self, '_config_manager_lock', None) or threading.Lock()
+        with lock:
+            value = getattr(self, '_config_manager', _UNSET)
+            if value is _UNSET:
+                try:
+                    from src.config_manager import ConfigManager
+                except ImportError:
+                    self.logger.warning("ConfigManager not available, using default cache intervals")
+                    value = None
+                else:
+                    value = ConfigManager()
+                    # Raises as it did from __init__; nothing is kept, so the
+                    # next access tries again.
+                    value.load_config()
+                self._config_manager = value
+        return value
+
+    @config_manager.setter
+    def config_manager(self, value: Optional[Any]) -> None:
+        self._config_manager = value
+
     def _get_writable_cache_dir(self) -> Optional[str]:
         """Tries to find or create a writable cache directory, preferring a system path when available."""
         # Attempt 1: System-wide persistent cache directory (preferred for services)
@@ -231,6 +333,24 @@ class CacheManager:
         """Get the path for a cache file."""
         return self._disk_cache_component.get_cache_path(key)
 
+    def file_signature(self, key: str) -> Optional[Tuple[int, int, int]]:
+        """``(st_ino, st_mtime_ns, st_size)`` of ``key``'s file, or None when
+        there is no file (the key is absent, or this cache has no disk tier).
+
+        One stat(), no read: a poller of a mailbox another process writes
+        compares it with the last one it saw and reads the file only when it
+        changed. Every write replaces the file (a temp file renamed into
+        place), so a new write always has a new inode, however fast it came.
+        """
+        path = self._get_cache_path(key)
+        if not path:
+            return None
+        try:
+            st = os.stat(path)
+        except OSError:
+            return None
+        return (st.st_ino, st.st_mtime_ns, st.st_size)
+
     def get_cached_data(self, key: str, max_age: int = 300, memory_ttl: Optional[int] = None) -> Optional[Dict[str, Any]]:
         """Get data from cache (memory first, then disk) honoring TTLs.
 
@@ -245,7 +365,11 @@ class CacheManager:
         # 1) Memory cache
         cached = self._memory_cache_component.get(key, max_age=in_memory_ttl)
         if cached is not None:
-            return cached
+            if not _outlived(cached, max_age, time.time()):
+                return cached
+            # Too old for this reader. Disk may hold a newer write (from the
+            # other process), and if it does not, the miss is the right answer.
+            self._memory_cache_component.clear(key)
 
         # 2) Disk cache
         record = self._disk_cache_component.get(key, max_age=max_age)
@@ -279,7 +403,9 @@ class CacheManager:
         # Check memory cache first (1 minute TTL)
         cached = self._memory_cache_component.get(key, max_age=60)
         if cached is not None:
-            return cached
+            if not _outlived(cached, 3600, time.time()):
+                return cached
+            self._memory_cache_component.clear(key)
         
         # Check disk cache
         data = self._disk_cache_component.get(key, max_age=3600)  # 1 hour for load_cache

src/common/README.md

@@ -17,7 +17,9 @@ Rules for the package:
 - `from src.common import ...` re-exports `APIHelper`, `ScrollHelper`,
   `LogoHelper`, `TextHelper`, `scroll_config` (plus `ScrollSettings`,
   `configure_scroll`, `resolve_scroll_settings`, `refresh_hz_from_config`) and
-  the adaptive layout names below ([`__init__.py`](__init__.py)).
+  the adaptive layout names below ([`__init__.py`](__init__.py)). Each is
+  imported on first use, so `import src.common` or a submodule import stays
+  cheap; add a new re-export to `_LAZY` there as well as `__all__`.
 
 ## Summary
 
@@ -26,6 +28,7 @@ Rules for the package:
 | [`api_helper`](#api_helper) | HTTP GET/POST with caching and rate limiting | Yes | — |
 | [`bdf_font`](#bdf_font) | Load and draw BDF bitmap fonts | Yes, if drawing BDF text directly | 3.5.0 |
 | [`espn_dates`](#espn_dates) | Fetch ESPN scoreboards across a date range | Yes (scoreboards) | 3.5.0 |
+| [`espn_payload`](#espn_payload) | Drop the parts of an ESPN scoreboard payload no scoreboard reads | No, core-internal (used by `BackgroundDataService`) | n/a |
 | [`favorite_team_check`](#favorite_team_check) | Log why a favourite team code shows nothing | Yes (scoreboards) | 3.6.0 |
 | [`fetch_service`](#fetch_service) | Pooled, merged, budgeted and counted HTTP for core fetch paths | No, core-internal (reached through `api_helper` and `espn_dates`) | n/a |
 | [`font_layout`](#font_layout) | Reproducible TrueType loading, crisp sizes | Yes | 3.4.0 |
@@ -42,8 +45,10 @@ Rules for the package:
 | [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
 | [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
 | [`sports_display_rules`](#sports_display_rules) | Which games a scoreboard shows, for how long, and its scorebug date line | Yes (scoreboards) | 3.8.0 |
+| [`sports_favorites`](#sports_favorites) | Which games involve a favourite team, and the favourites-only picks | Yes (scoreboards) | 3.8.2 |
 | [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
 | [`sports_font_path`](#sports_font_path) | Find a scoreboard's bundled font whatever the cwd | Yes (scoreboards) | 3.8.0 |
+| [`sports_game_over`](#sports_game_over) | Whether a game ESPN still lists as live has ended | Yes (scoreboards) | 3.8.1 |
 | [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
 | [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
 | [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 |
@@ -107,12 +112,27 @@ and the plugin test harness all use it. Most plugins get BDF text through
 [`espn_dates.py`](espn_dates.py). ESPN's site API rejects `dates=` ranges
 and truncates results when `limit` is above 500. `fetch_espn_scoreboard()`
 splits a range into month and day requests ESPN accepts and merges the
-results; `espn_date_chunks()`, `fetch_espn_date_chunks()`,
-`clamp_espn_limit()` and `merge_scoreboard_payloads()` are the pieces.
+results; `espn_date_chunks()`, `espn_request_chunks()`,
+`fetch_espn_date_chunks()`, `clamp_espn_limit()` and
+`merge_scoreboard_payloads()` are the pieces. A window's partial edge months
+are asked whole and trimmed to its days (US Eastern), and chunk requests share
+one process-wide cap of `ESPN_CHUNK_WORKERS` in flight.
 Every request goes through [`fetch_service`](#fetch_service), the chunks
 counted against the plugin that asked. Scoreboard plugins also bundle a copy
 for older cores.
 
+### espn_payload
+
+[`espn_payload.py`](espn_payload.py). Core-internal. ESPN scoreboard
+responses carry stat leaders, athlete cards, links, headlines and highlights
+that no scoreboard draws. `slim_scoreboard_payload(payload)` removes exactly
+those keys, in place, and leaves everything it does not know about alone;
+`is_espn_scoreboard_url(url)` says whether a URL is an ESPN site-API
+scoreboard. `BackgroundDataService` slims each scoreboard window before
+caching it, which cuts the five sports windows from ~40MB to ~12MB of parsed
+objects. Adding a key to the drop lists means first checking that nothing
+reads it.
+
 ### favorite_team_check
 
 [`favorite_team_check.py`](favorite_team_check.py).
@@ -274,6 +294,19 @@ list it before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
 `_effective_live_duration()`, the shorter dwell for a non-favourite live
 game).
 
+### sports_favorites
+
+[`sports_favorites.py`](sports_favorites.py). Sports family 6, one mixin per
+class that carried the methods: `SportsFavoritesMixin` (`SportsCore`:
+`_is_favorite_game(game)` and `_favorite_code(value)`),
+`SportsUpcomingFavoritesMixin` (`_select_games_for_display`) and
+`SportsRecentFavoritesMixin` (`_select_recent_games_for_display`). Each side
+of a game is named by `_favorite_key` (from `SportsHelpersMixin`; NRL
+overrides it with the team id) and compared with `favorite_teams` stripped and
+upper-cased. The selection methods give each favourite up to the per-team
+limit, count a game between two favourites for both, and treat only games
+with an id as possible duplicates.
+
 ### sports_fetch
 
 [`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
@@ -289,6 +322,16 @@ path as given when it exists (relative to the cwd), else
 `font_layout.resolve_asset_path(path)`. What the scoreboards'
 `_resolve_font_path` copies return on a core that ships it.
 
+### sports_game_over
+
+[`sports_game_over.py`](sports_game_over.py). `SportsGameOverMixin`:
+`_is_game_really_over(game)`, the `SportsLive` check that drops a game ESPN
+still lists as live (`SportsLiveSharedMixin._detect_stale_games` calls it).
+Over on a final period text, or on a 0:00 clock from period `FINAL_PERIOD`
+on unless the score is level. `FINAL_PERIOD` is a class attribute the host
+sets per sport; the default `None` means the clock never ends a game. List
+it before `SportsLiveSharedMixin`.
+
 ### sports_game_renderer
 
 [`sports_game_renderer.py`](sports_game_renderer.py).
@@ -386,7 +429,7 @@ Created by `DisplayController`; works with any plugin.
 `draw_multiline_text()`, `create_text_image()`.
 
 `draw_text_outlined(draw, xy, text, font, fill, outline_color=(0, 0, 0),
-offsets=OUTLINE_SQUARE)` (Unreleased) draws the text in `outline_color` at
+offsets=OUTLINE_SQUARE)` (3.8.1) draws the text in `outline_color` at
 each offset, then in `fill` on top: the same pixels as one `draw.text` per
 offset, but the string is rasterized once. `OUTLINE_SQUARE` is the
 eight-sided one-pixel outline the scoreboards draw, `OUTLINE_CROSS` the

src/common/__init__.py

@@ -6,45 +6,90 @@ This package provides reusable functionality for plugins and core modules:
 - Logo helpers
 - Text/scroll helpers
 - Adaptive layout and image helpers
+
+The names below are imported on first use (PEP 562), not when the package is
+imported. ``from src.common import ScrollHelper`` and
+``src.common.ScrollHelper`` work as before and return the same objects, but
+``import src.common`` -- or importing any submodule, such as
+``src.common.path_safety`` -- no longer loads numpy, requests and freetype
+along with every helper. The web interface imports src.common only for a few
+small modules and never needs those.
 """
 
-# Export commonly used utilities
-from src.common.api_helper import APIHelper
-from src.common.scroll_helper import ScrollHelper
-from src.common import scroll_config
-from src.common.scroll_config import (
-    ScrollSettings,
-    configure as configure_scroll,
-    resolve as resolve_scroll_settings,
-    refresh_hz_from_config,
-)
-from src.common.logo_helper import LogoHelper
-from src.common.text_helper import TextHelper
+import importlib
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
 
-# Adaptive layout & images (canonical homes: src.adaptive_layout /
-# src.adaptive_images — re-exported here so plugin authors find them in the
-# blessed-helpers package). See docs/ADAPTIVE_LAYOUT.md.
-from src.adaptive_layout import (
-    Region,
-    LayoutContext,
-    FontStep,
-    FontLadder,
-    LADDER_GRID,
-    LADDER_ARCADE,
-    FitResult,
-    draw_fitted_text,
-    ScoreboardRegions,
-    scoreboard_regions,
-    MediaRow,
-    media_row,
-)
-from src.adaptive_images import (
-    ImageFitResult,
-    fit_image,
-    draw_fitted_image,
-    RESAMPLE_LANCZOS,
-    RESAMPLE_NEAREST,
-)
+if TYPE_CHECKING:
+    # What mypy and editors see: the real names and their types.
+    from src.common.api_helper import APIHelper
+    from src.common.scroll_helper import ScrollHelper
+    from src.common import scroll_config
+    from src.common.scroll_config import (
+        ScrollSettings,
+        configure as configure_scroll,
+        resolve as resolve_scroll_settings,
+        refresh_hz_from_config,
+    )
+    from src.common.logo_helper import LogoHelper
+    from src.common.text_helper import TextHelper
+
+    # Adaptive layout & images (canonical homes: src.adaptive_layout /
+    # src.adaptive_images — re-exported here so plugin authors find them in the
+    # blessed-helpers package). See docs/ADAPTIVE_LAYOUT.md.
+    from src.adaptive_layout import (
+        Region,
+        LayoutContext,
+        FontStep,
+        FontLadder,
+        LADDER_GRID,
+        LADDER_ARCADE,
+        FitResult,
+        draw_fitted_text,
+        ScoreboardRegions,
+        scoreboard_regions,
+        MediaRow,
+        media_row,
+    )
+    from src.adaptive_images import (
+        ImageFitResult,
+        fit_image,
+        draw_fitted_image,
+        RESAMPLE_LANCZOS,
+        RESAMPLE_NEAREST,
+    )
+
+#: Exported name -> (module it lives in, attribute name there). An attribute
+#: of None means the name is the module itself. Keep in step with the
+#: TYPE_CHECKING imports above and with __all__.
+_LAZY: Dict[str, Tuple[str, Optional[str]]] = {
+    'APIHelper': ('src.common.api_helper', 'APIHelper'),
+    'ScrollHelper': ('src.common.scroll_helper', 'ScrollHelper'),
+    'scroll_config': ('src.common.scroll_config', None),
+    'ScrollSettings': ('src.common.scroll_config', 'ScrollSettings'),
+    'configure_scroll': ('src.common.scroll_config', 'configure'),
+    'resolve_scroll_settings': ('src.common.scroll_config', 'resolve'),
+    'refresh_hz_from_config': ('src.common.scroll_config', 'refresh_hz_from_config'),
+    'LogoHelper': ('src.common.logo_helper', 'LogoHelper'),
+    'TextHelper': ('src.common.text_helper', 'TextHelper'),
+    # adaptive layout & images
+    'Region': ('src.adaptive_layout', 'Region'),
+    'LayoutContext': ('src.adaptive_layout', 'LayoutContext'),
+    'FontStep': ('src.adaptive_layout', 'FontStep'),
+    'FontLadder': ('src.adaptive_layout', 'FontLadder'),
+    'LADDER_GRID': ('src.adaptive_layout', 'LADDER_GRID'),
+    'LADDER_ARCADE': ('src.adaptive_layout', 'LADDER_ARCADE'),
+    'FitResult': ('src.adaptive_layout', 'FitResult'),
+    'draw_fitted_text': ('src.adaptive_layout', 'draw_fitted_text'),
+    'ScoreboardRegions': ('src.adaptive_layout', 'ScoreboardRegions'),
+    'scoreboard_regions': ('src.adaptive_layout', 'scoreboard_regions'),
+    'MediaRow': ('src.adaptive_layout', 'MediaRow'),
+    'media_row': ('src.adaptive_layout', 'media_row'),
+    'ImageFitResult': ('src.adaptive_images', 'ImageFitResult'),
+    'fit_image': ('src.adaptive_images', 'fit_image'),
+    'draw_fitted_image': ('src.adaptive_images', 'draw_fitted_image'),
+    'RESAMPLE_LANCZOS': ('src.adaptive_images', 'RESAMPLE_LANCZOS'),
+    'RESAMPLE_NEAREST': ('src.adaptive_images', 'RESAMPLE_NEAREST'),
+}
 
 __all__ = [
     'APIHelper',
@@ -75,3 +120,25 @@ __all__ = [
     'RESAMPLE_LANCZOS',
     'RESAMPLE_NEAREST',
 ]
+
+
+def __getattr__(name: str) -> Any:
+    """Import an exported name on first access (PEP 562).
+
+    Only called for names not already in the module namespace, so after the
+    first access the cached value below is returned directly. Unknown names
+    raise AttributeError, which ``from src.common import <submodule>`` relies
+    on to fall through to importing the submodule.
+    """
+    try:
+        module_name, attr = _LAZY[name]
+    except KeyError:
+        raise AttributeError(f"module {__name__!r} has no attribute {name!r}") from None
+    module = importlib.import_module(module_name)  # nosemgrep: python.lang.security.audit.non-literal-import.non-literal-import -- module_name comes from the fixed _LAZY table
+    value = module if attr is None else getattr(module, attr)
+    globals()[name] = value
+    return value
+
+
+def __dir__() -> List[str]:
+    return sorted(set(globals()) | set(__all__))

src/common/api_helper.py

@@ -17,6 +17,7 @@ from src.common.espn_dates import (
     store_espn_scoreboard_cache,
 )
 from src.common.fetch_service import fetch_get, fetch_post, share_connection_pool
+from src.common.json_body import response_json
 from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
 
 import requests
@@ -157,7 +158,7 @@ class APIHelper:
             response.raise_for_status()
             
             # Parse JSON response
-            data: Dict[Any, Any] = response.json()
+            data: Dict[Any, Any] = response_json(response)
             
             # Cache response if cache key provided
             if cache_key and self.cache_manager:
@@ -304,7 +305,7 @@ class APIHelper:
             )
             response.raise_for_status()
             
-            return cast(Optional[Dict[Any, Any]], response.json())
+            return cast(Optional[Dict[Any, Any]], response_json(response))
             
         except requests.exceptions.RequestException as e:
             self.logger.error(f"POST request failed for {url}: {e}")

src/common/espn_dates.py

@@ -26,10 +26,30 @@ A month can hold more than 500 events (college baseball's March does), and
 ESPN answers that with exactly ``limit`` events and no hint that more exist. A
 month chunk that comes back full is therefore re-asked day by day.
 
+A window's *partial* edge months are asked for whole, too, once the window
+covers ``ESPN_MONTH_COVER_MIN_DAYS`` or more of their days, and the answer is
+trimmed back to the window's days. A scoreboard's default fortnight either side
+of today (29 days, two partial months) was 29 day requests per league; it is
+now 2. Trimming needs ESPN's "game day", which is the event's start in US
+Eastern time -- checked against the live API on 2026-10-03: 417 of 417 soccer
+events across five leagues and three months (one of them spanning the end of
+daylight saving) came back from exactly the day query their Eastern date
+names. A short window (a live poll's one or two days) stays day by day, so it
+never downloads a whole month to read a day of it.
+
+Chunk requests share one process-wide budget of ``ESPN_CHUNK_WORKERS`` in
+flight, however many windows are being fetched at once. Each window used to get
+its own six, so a scoreboard starting eight leagues -- each with a recent and
+an upcoming manager -- had ~40 requests in flight, every one beyond a session's
+pool a new connection and a new DNS lookup. On a Pi whose resolver could not
+keep up, that was ~90 ``NameResolutionError`` lines within a minute of every
+start.
+
 Once a range has been rejected, later ranges skip straight to chunks for
 ``RANGE_RETRY_SECONDS`` instead of spending a doomed request first -- live
 scoreboards ask every 30 seconds. After that the range is tried again, so the
-workaround retires itself if ESPN reverts.
+workaround retires itself if ESPN reverts. A process starts inside that
+period, as if a range had just been rejected.
 
 ONE CACHE KEY PER SCOREBOARD
 ----------------------------
@@ -55,7 +75,7 @@ import re
 import threading
 import time
 from concurrent.futures import ThreadPoolExecutor
-from datetime import date, datetime, timedelta
+from datetime import date, datetime, timedelta, tzinfo
 from functools import partial
 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
 
@@ -100,16 +120,57 @@ RANGE_RETRY_SECONDS = 6 * 60 * 60
 # pool_maxsize of 10 so the shared Session never has to discard connections.
 ESPN_CHUNK_WORKERS = 6
 
+#: An edge month the window covers at least this many days of is asked for
+#: whole and trimmed, instead of one request per day (see module docstring).
+#: Below it the days are cheaper than the month: a whole month is two to
+#: three times the bytes of the half of it a fortnight window holds.
+ESPN_MONTH_COVER_MIN_DAYS = 7
+
+# Every chunk request in the process holds one of these while it is in flight
+# -- the cap is per process, not per window (see module docstring).
+_chunk_slots = threading.BoundedSemaphore(ESPN_CHUNK_WORKERS)
+
+
+def _eastern_zone() -> Optional[tzinfo]:
+    """US Eastern, the zone ESPN's ``dates=YYYYMMDD`` means, or None when
+    this Python has no time zone data (no edge month is trimmed then)."""
+    zone: Optional[tzinfo] = None
+    try:
+        from zoneinfo import ZoneInfo
+        zone = ZoneInfo("America/New_York")
+    except Exception:  # noqa: BLE001 - no zoneinfo module or no tz database
+        zone = None
+    if zone is not None:
+        return zone
+    try:
+        import pytz
+        return cast(tzinfo, pytz.timezone("America/New_York"))
+    except Exception:  # noqa: BLE001
+        return None
+
+
+_EASTERN = _eastern_zone()
+
+# What _fetch_one_chunk returns for a month that came back at the cap.
+_CAPPED: Any = object()
+
 _range_lock = threading.Lock()
-_ranges_rejected_until = 0.0
+# A process starts out assuming ranges are still rejected, as they have been
+# since 2026-09-15, and tries one again RANGE_RETRY_SECONDS in. Starting
+# from "unknown" cost one doomed range request per window at every start --
+# eleven 400s at once from a soccer board, each fetching before any had
+# answered -- to learn what every start learns.
+_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
 
 __all__ = [
     "ESPN_MAX_LIMIT",
     "ESPN_CHUNK_WORKERS",
+    "ESPN_MONTH_COVER_MIN_DAYS",
     "RANGE_RETRY_SECONDS",
     "clamp_espn_limit",
     "parse_espn_date_range",
     "espn_date_chunks",
+    "espn_request_chunks",
     "merge_scoreboard_payloads",
     "fetch_espn_date_chunks",
     "fetch_espn_scoreboard",
@@ -220,6 +281,79 @@ def espn_date_chunks(start: date, end: date) -> List[str]:
     return chunks
 
 
+def espn_request_chunks(
+    start: date,
+    end: date,
+    month_cover_min_days: Optional[int] = None,
+) -> List[Tuple[str, Optional[Tuple[date, date]]]]:
+    """The requests that fetch ``[start, end]``, as ``(dates, trim)`` pairs.
+
+    :func:`espn_date_chunks`, except that a partial edge month with
+    ``month_cover_min_days`` (default ``ESPN_MONTH_COVER_MIN_DAYS``) or more
+    of its days in the window becomes one ``YYYYMM`` request whose ``trim``
+    is the first and last of those days: its events that start outside them
+    (US Eastern) are dropped. ``trim`` is None for every other request.
+    Without time zone data nothing can be trimmed, so the edge days stay day
+    requests.
+    """
+    if month_cover_min_days is None:
+        month_cover_min_days = ESPN_MONTH_COVER_MIN_DAYS
+    planned: List[Tuple[str, Optional[Tuple[date, date]]]] = []
+    run: List[str] = []
+
+    def flush() -> None:
+        if (_EASTERN is not None and month_cover_min_days > 0
+                and len(run) >= month_cover_min_days):
+            planned.append((run[0][:6], (_parse_day(run[0]), _parse_day(run[-1]))))
+        else:
+            planned.extend((day, None) for day in run)
+        run.clear()
+
+    for chunk in espn_date_chunks(start, end):
+        if run and (len(chunk) != 8 or chunk[:6] != run[0][:6]):
+            flush()
+        if len(chunk) == 8:
+            run.append(chunk)
+        else:
+            planned.append((chunk, None))
+    flush()
+    return planned
+
+
+def _parse_day(text: str) -> date:
+    return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
+
+
+def _eastern_day(stamp: Any) -> Optional[date]:
+    """The US Eastern date of an ESPN event ``date`` ("2026-10-10T11:30Z"),
+    or None when it cannot be read."""
+    if not isinstance(stamp, str) or _EASTERN is None:
+        return None
+    try:
+        moment = datetime.fromisoformat(stamp.strip().replace("Z", "+00:00"))
+    except ValueError:
+        return None
+    if moment.tzinfo is None:
+        return None
+    return moment.astimezone(_EASTERN).date()
+
+
+def _trim_to_days(payload: Any, first: date, last: date) -> Any:
+    """Drop the events of a month payload that start outside ``[first, last]``
+    (US Eastern). An event whose date cannot be read is kept: its day query
+    might well have returned it, and a game is never dropped on a guess.
+    """
+    if not isinstance(payload, dict) or not isinstance(payload.get("events"), list):
+        return payload
+    kept = []
+    for event in payload["events"]:
+        day = _eastern_day(event.get("date")) if isinstance(event, dict) else None
+        if day is None or first <= day <= last:
+            kept.append(event)
+    payload["events"] = kept
+    return payload
+
+
 def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
     """Fold chunk responses into one scoreboard payload.
 
@@ -250,37 +384,54 @@ def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
 def _fetch_one_chunk(
     session, url: str, params: Dict[str, Any], headers, timeout, logger, chunk: str,
     cache_max_age: Optional[float] = None,
-) -> Optional[Dict[str, Any]]:
+    trims: Optional[Dict[str, Tuple[date, date]]] = None,
+) -> Any:
     """GET a single ``dates=`` chunk, or None when it failed.
 
     One bad chunk must not sink the rest of the season, so every error is
     logged and swallowed here rather than raised to the gather below.
+
+    A month that comes back at the cap is truncated: it returns ``_CAPPED``,
+    its payload dropped here before it is ever held beside the others. A
+    month in ``trims`` loses its events outside the days given there.
+
+    The request holds one of the process-wide ``_chunk_slots`` while it runs.
     """
     try:
-        response = fetch_get(
-            session,
-            url,
-            params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
-            headers=headers,
-            timeout=timeout,
-            **_memo_kwargs(cache_max_age),
-        )
-        response.raise_for_status()
-        return cast(Optional[Dict[str, Any]], response_json(response))
+        with _chunk_slots:
+            response = fetch_get(
+                session,
+                url,
+                params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
+                headers=headers,
+                timeout=timeout,
+                **_memo_kwargs(cache_max_age),
+            )
+            response.raise_for_status()
+            payload = response_json(response)
     except Exception as exc:  # noqa: BLE001 - see docstring
         if logger:
             logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
         return None
+    if len(chunk) == 6 and isinstance(payload, dict):
+        if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
+            return _CAPPED
+        trim = (trims or {}).get(chunk)
+        if trim is not None:
+            payload = _trim_to_days(payload, *trim)
+    return payload
 
 
 def _fetch_chunks(
     session, url: str, params: Dict[str, Any], headers, timeout, logger,
     chunks: List[str], cache_max_age: Optional[float] = None,
-) -> List[Optional[Dict[str, Any]]]:
+    trims: Optional[Dict[str, Tuple[date, date]]] = None,
+) -> List[Any]:
     """Fetch every chunk, returning payloads positionally aligned with ``chunks``.
 
     Requests go out ``ESPN_CHUNK_WORKERS`` at a time because a cold season is
-    over a hundred of them. The order they come back in is not significant --
+    over a hundred of them -- and no more than that across every window the
+    process is fetching, which ``_fetch_one_chunk``'s slot enforces. The order they come back in is not significant --
     callers keep ``chunks`` order from the returned list -- but it does mean
     the session is shared across threads, which is why this only ever issues
     GETs and never touches session state.
@@ -293,7 +444,7 @@ def _fetch_chunks(
         return []
     fetch = partial(
         _fetch_one_chunk, session, url, params, headers, timeout, logger,
-        cache_max_age=cache_max_age,
+        cache_max_age=cache_max_age, trims=trims,
     )
     if len(chunks) == 1:
         return [fetch(chunks[0])]
@@ -340,7 +491,9 @@ def fetch_espn_date_chunks(
     if span is None:
         return None
 
-    chunks = espn_date_chunks(*span)
+    planned = espn_request_chunks(*span)
+    chunks = [chunk for chunk, _ in planned]
+    trims = {chunk: trim for chunk, trim in planned if trim is not None}
     if logger:
         logger.debug(
             "Fetching ESPN date range %s as %d month/day chunks",
@@ -349,32 +502,31 @@ def fetch_espn_date_chunks(
 
     results = _fetch_chunks(
         session, url, params, headers, timeout, logger, chunks, cache_max_age,
+        trims,
     )
     attempted = len(chunks)
 
-    # A month that came back at the cap is truncated; its days replace it in
-    # place, so merged events stay in chunk order however the requests raced.
+    # A month that came back at the cap is truncated; its days (only the
+    # window's, for a trimmed edge month) replace it in place, so merged
+    # events stay in chunk order however the requests raced. Its payload was
+    # already dropped in the worker: a capped college-baseball month is ~2MB
+    # of parsed JSON, and holding four of them through ~120 day requests added
+    # ~25MB to the peak -- more than the concurrency itself. Low-memory boards
+    # (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
     slots: List[Any] = results
     capped: Dict[int, List[str]] = {}
     for index, chunk in enumerate(chunks):
-        payload = slots[index]
-        if payload is None or len(chunk) != 6:
+        if slots[index] is not _CAPPED:
             continue
-        events = payload.get("events") if isinstance(payload, dict) else None
-        if len(events or []) >= ESPN_MAX_LIMIT:
-            if logger:
-                logger.info(
-                    "ESPN month %s hit the %d-event cap; re-asking it day by day",
-                    chunk, ESPN_MAX_LIMIT,
-                )
-            capped[index] = _days_of_month(chunk)
-            # Drop the truncated month now rather than after its days arrive:
-            # a capped college-baseball month is ~2MB of parsed JSON, and
-            # holding four of them through ~120 day requests added ~25MB to
-            # the peak -- more than the concurrency itself. Low-memory boards
-            # (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
-            slots[index] = None
-    payload = events = None
+        if logger:
+            logger.info(
+                "ESPN month %s hit the %d-event cap; re-asking it day by day",
+                chunk, ESPN_MAX_LIMIT,
+            )
+        trim = trims.get(chunk)
+        capped[index] = (_days_of_month(chunk) if trim is None
+                         else espn_date_chunks(*trim))
+        slots[index] = None
 
     if capped:
         days = [day for index in sorted(capped) for day in capped[index]]

src/common/espn_payload.py

@@ -0,0 +1,97 @@
+"""Drop the parts of an ESPN scoreboard payload no scoreboard reads.
+
+The sports scoreboards cache their Recent/Upcoming window (14 days back, 7
+ahead) as the raw ESPN response, and that record stays parsed in the memory
+cache for as long as it is fresh. Most of it is never drawn. Measured on hdpi
+(2026-10-02) the MLB window was 3.35MB of JSON and 13.5MB of Python objects,
+and the five windows together ~40MB, mostly in:
+
+* ``competitors[].leaders`` / ``competitions[].leaders`` -- per-team and
+  per-game stat leaders (28% of the MLB window)
+* ``competitors[].team.links`` / ``event.links`` -- web and app URLs
+* ``status.featuredAthletes`` and ``competitors[].probables`` -- athlete
+  cards with headshots and season stats
+* ``competitions[].headlines`` / ``highlights`` -- article and video blurbs
+  (28% of the college-football window)
+* ``competitions[].geoBroadcasts``
+
+None of those keys is read by core or by any plugin in ledmatrix-plugins
+(checked 2026-10-02 across every scoreboard, the odds ticker and the
+leaderboard), while everything that is read -- odds, records, linescores,
+situation, statistics, notes, broadcasts, venue -- is kept. Dropping them
+takes the five windows from ~40MB to ~12MB of parsed objects and the files from
+10.6MB to 3.0MB, so the reads that parse an expired window on the render
+thread get 3-4x cheaper too.
+
+:func:`slim_scoreboard_payload` changes the payload in place, and only ever
+removes the keys listed here: anything it does not know about is left alone.
+"""
+
+from typing import Any, Dict
+from urllib.parse import urlsplit
+
+# Per level of the payload, the keys removed. Kept deliberately explicit:
+# adding a key here means checking that nothing reads it first.
+_EVENT_DROP = ("links",)
+_COMPETITION_DROP = ("leaders", "headlines", "highlights", "geoBroadcasts")
+_STATUS_DROP = ("featuredAthletes",)
+_COMPETITOR_DROP = ("leaders", "probables")
+_TEAM_DROP = ("links",)
+
+
+def is_espn_scoreboard_url(url: Any) -> bool:
+    """Whether ``url`` is an ESPN site-API scoreboard endpoint."""
+    if not isinstance(url, str):
+        return False
+    try:
+        parts = urlsplit(url)
+    except ValueError:
+        return False
+    host = (parts.hostname or "").lower()
+    if host != "espn.com" and not host.endswith(".espn.com"):
+        return False
+    return parts.path.rstrip("/").endswith("/scoreboard")
+
+
+def _drop(obj: Any, keys) -> None:
+    if isinstance(obj, dict):
+        for key in keys:
+            obj.pop(key, None)
+
+
+def slim_scoreboard_payload(payload: Any) -> Any:
+    """Remove the unread parts of an ESPN scoreboard payload, in place.
+
+    Returns ``payload`` for convenience. Anything that is not shaped like a
+    scoreboard (not a dict, no ``events`` list, odd entries) is passed over
+    untouched rather than raising.
+    """
+    if not isinstance(payload, dict):
+        return payload
+    events = payload.get("events")
+    if not isinstance(events, list):
+        return payload
+    for event in events:
+        if not isinstance(event, dict):
+            continue
+        _drop(event, _EVENT_DROP)
+        competitions = event.get("competitions")
+        if not isinstance(competitions, list):
+            continue
+        for competition in competitions:
+            if not isinstance(competition, dict):
+                continue
+            _drop(competition, _COMPETITION_DROP)
+            _drop(competition.get("status"), _STATUS_DROP)
+            competitors = competition.get("competitors")
+            if not isinstance(competitors, list):
+                continue
+            for competitor in competitors:
+                if not isinstance(competitor, dict):
+                    continue
+                _drop(competitor, _COMPETITOR_DROP)
+                _drop(competitor.get("team"), _TEAM_DROP)
+    return payload
+
+
+__all__ = ["is_espn_scoreboard_url", "slim_scoreboard_payload"]

src/common/fetch_service.py

@@ -59,7 +59,10 @@ says how old with ``cache_max_age`` (``fetch_get(..., cache_max_age=ttl)``;
 Identical means what the validator store keys on: URL, query, effective
 headers and, for a session with cookies or auth, the session.
 
-**Counters.** Requests, merged requests, bytes, 304s, errors, HTTP errors,
+**Counters.** Requests, merged requests, bytes (``bytes`` decoded, as the
+caller reads them; ``wire_bytes`` as they crossed the network, which is
+what a metered connection pays for -- ESPN gzips, so the two differ ~14x),
+304s, errors, HTTP errors,
 adapter retries, throttled requests and seconds waited, plus requests
 answered without the network: ``memo_hits`` (the response cache) and
 ``cache_hits`` / ``legacy_cache_hits`` (a shared ESPN scoreboard cache entry,
@@ -201,6 +204,7 @@ _COUNTER_FIELDS = (
     "throttled",     # requests that waited for a host budget
     "overruns",      # requests that went after max_wait_seconds anyway
     "bytes",         # decoded response body bytes received
+    "wire_bytes",    # body bytes as they came off the socket (still compressed)
     "wait_seconds",  # time spent waiting for host budgets
     "memo_hits",     # answered from the response cache (max-age); nothing sent
     "cache_hits",    # scoreboard fetches answered from a shared ESPN cache entry
@@ -616,6 +620,30 @@ def _body_of(response: Any) -> Optional[bytes]:
     return content if isinstance(content, bytes) else None
 
 
+def _wire_bytes_of(response: Any, body: Optional[bytes]) -> int:
+    """How many body bytes came off the socket for ``response``: the
+    compressed size when the server sent gzip, which ESPN does for every
+    scoreboard (63 KB on the wire for an 865 KB college football Saturday).
+
+    urllib3's ``HTTPResponse.tell()`` counts the raw bytes read before
+    decoding. A response without one (a test double, an adapter that is not
+    urllib3) or one whose body was not read is counted at its decoded size,
+    or as 0, so the counter never claims less than it can prove.
+    """
+    if body is None:
+        return 0
+    raw = getattr(response, "raw", None)
+    tell = getattr(raw, "tell", None)
+    if callable(tell):
+        try:
+            read = tell()
+        except Exception:
+            read = None
+        if isinstance(read, int) and not isinstance(read, bool) and read > 0:
+            return read
+    return len(body)
+
+
 def _retries_of(response: Any) -> int:
     raw = getattr(response, "raw", None)
     retries = getattr(raw, "retries", None)
@@ -1117,6 +1145,7 @@ class FetchService:
                         http_errors=int(status is not None and status >= 400),
                         retries=_retries_of(response),
                         bytes=len(body) if body is not None else 0,
+                        wire_bytes=_wire_bytes_of(response, body),
                         throttled=int(waited > 0), overruns=int(overrun),
                         wait_seconds=waited)
         except Exception:

src/common/frame_timing.py

@@ -121,6 +121,7 @@ three times per threshold, so keep it to diagnostic runs, not soaks.
 
 from __future__ import annotations
 
+import atexit
 import copy
 import gc
 import json
@@ -213,10 +214,19 @@ class GcMonitor:
     lock: the render thread and the stats writer only read them.
 
     Install it once per process with :func:`install_gc_monitor`.
+
+    Collections still run while the interpreter shuts down, after module
+    globals such as ``time`` may already be torn down to ``None``. The clock
+    and ``sys.is_finalizing`` are bound here so the callback never looks a
+    global up, it does nothing once finalization has begun, and
+    :func:`install_gc_monitor` unregisters it at exit anyway.
     """
 
-    def __init__(self, threshold: float = GC_PAUSE_SECONDS):
+    def __init__(self, threshold: float = GC_PAUSE_SECONDS,
+                 clock: Callable[[], float] = time.perf_counter):
         self.threshold = threshold
+        self._clock = clock
+        self._is_finalizing = sys.is_finalizing
         self._started: Optional[float] = None
         #: Per generation (0, 1, 2), since the monitor was installed.
         self.collections = [0, 0, 0]
@@ -232,7 +242,9 @@ class GcMonitor:
         self.last_long: Optional[Tuple[float, float]] = None
 
     def __call__(self, phase: str, info: Dict[str, Any]) -> None:
-        now = time.perf_counter()
+        if self._is_finalizing():
+            return
+        now = self._clock()
         if phase == "start":
             self._started = now
             return
@@ -267,15 +279,38 @@ _gc_monitor_lock = threading.Lock()
 
 
 def install_gc_monitor() -> GcMonitor:
-    """The process's GcMonitor, installed in ``gc.callbacks`` on first call."""
+    """The process's GcMonitor, installed in ``gc.callbacks`` on first call.
+
+    It is unregistered at exit (:func:`uninstall_gc_monitor`), before the
+    interpreter tears module globals down.
+    """
     global _gc_monitor
     with _gc_monitor_lock:
         if _gc_monitor is None:
             _gc_monitor = GcMonitor()
             gc.callbacks.append(_gc_monitor)
+            atexit.register(uninstall_gc_monitor)
         return _gc_monitor
 
 
+def uninstall_gc_monitor() -> None:
+    """Take the process's GcMonitor out of ``gc.callbacks``; safe to repeat.
+
+    A recorder that still holds the monitor keeps its counters; they just
+    stop moving. The next :func:`install_gc_monitor` installs a fresh one.
+    """
+    global _gc_monitor
+    with _gc_monitor_lock:
+        monitor, _gc_monitor = _gc_monitor, None
+        if monitor is None:
+            return
+        atexit.unregister(uninstall_gc_monitor)
+        try:
+            gc.callbacks.remove(monitor)
+        except ValueError:
+            pass
+
+
 #: One presented frame's interval: (interval, blit, wait, hold, ops), where
 #: ops is the work noted before it (kind -> bytes) or None.
 _Frame = Tuple[float, float, float, int, Optional[Dict[str, int]]]

src/common/scroll_helper.py

@@ -28,6 +28,30 @@ import numpy as np
 # long over one frame, so a sample this large is an idle gap between scrolls.
 FPS_LOG_INTERVAL = 5.0
 
+# The stats line goes to INFO only when a window is worth an operator's
+# attention, as Vegas's FPS line does (src/vegas_mode/coordinator.py): every
+# 5s from every scroller was most of the journal on a healthy rig. A window is
+# degraded when its frame rate falls below this fraction of the rate it was
+# locked to (1 / its own median frame time; same 0.9 as Vegas) ...
+STATS_HEALTHY_FRACTION = 0.9
+# ... or when more than this share of its frames stalled (past 1.5x the
+# median). A 1% stall rate barely moves the mean, so the fps test alone would
+# miss the judder this line exists to show.
+STATS_DEGRADED_STALL_RATE = 0.01
+# A healthy scroller still logs at INFO this often, so silence in the journal
+# means stopped rather than fine. Every window is still logged at DEBUG.
+STATS_HEARTBEAT_INTERVAL = 300.0
+
+
+def frame_stats_degraded(stats: Dict[str, Any]) -> bool:
+    """Whether one frame_stats() window is worth logging at INFO."""
+    n = stats["frames"]
+    if n == 0 or stats["median"] <= 0:
+        return False
+    locked_fps = 1.0 / stats["median"]
+    return (stats["fps"] < locked_fps * STATS_HEALTHY_FRACTION
+            or stats["stalls"] > n * STATS_DEGRADED_STALL_RATE)
+
 
 def _rgb_pixels(item) -> np.ndarray:
     """An appended item's pixels as an RGB array, as pasting it would draw them."""
@@ -189,6 +213,11 @@ class ScrollHelper:
         # Every frame time since the last stats line, so the 5s summary can
         # report the tail rather than one arbitrary sample. Cleared on log.
         self._window: list = []
+        # INFO-level stats bookkeeping (see STATS_HEARTBEAT_INTERVAL). Kept
+        # across reset_scroll(): a heartbeat per scroll start would bring the
+        # chatter back. 0.0 so the first window after start-up is at INFO.
+        self._stats_last_info_log = 0.0
+        self._stats_was_degraded = False
         
         # Scrolling state management
         self.is_scrolling = False
@@ -532,7 +561,7 @@ class ScrollHelper:
         width = self.display_width
         strip_width = self.cached_array.shape[1]
 
-        if start_x + width + 1 <= strip_width:
+        if 0 <= start_x and start_x + width + 1 <= strip_width:
             # Slice the backing array directly. Going via
             # _get_visible_portion_integer would build two PIL images only for
             # them to be converted straight back to arrays, which measured 15x
@@ -540,9 +569,10 @@ class ScrollHelper:
             near = self.cached_array[:, start_x:start_x + width]
             far = self.cached_array[:, start_x + 1:start_x + 1 + width]
         else:
-            # Close enough to the end that one of the slices wraps; let the
-            # integer path handle that and pay the conversion. Continuous mode
-            # extends the strip before reaching here, so this is the rare case.
+            # One of the slices wraps (close to the end, or a strip narrower
+            # than the panel); let the integer path handle that and pay the
+            # conversion. Continuous mode extends the strip before reaching
+            # here, so this is the rare case.
             near = np.asarray(
                 self._get_visible_portion_integer(start_x, start_x + width))
             far = np.asarray(
@@ -572,31 +602,33 @@ class ScrollHelper:
         _size = (self.display_width, self.display_height)
         img_w = self.cached_array.shape[1]
 
-        if end_x <= img_w:
-            # Normal case: single contiguous slice (fastest path)
-            frame_array = np.ascontiguousarray(self.cached_array[:, start_x:end_x])
-            return Image.frombytes('RGB', _size, frame_array.tobytes())
+        if 0 <= start_x and end_x <= img_w:
+            # Normal case: single contiguous slice (fastest path). tobytes()
+            # on the column-slice view already returns C-order bytes, so
+            # ascontiguousarray() first only added a second full-frame copy.
+            return Image.frombytes(
+                'RGB', _size,
+                self.cached_array[:, start_x:end_x].tobytes())
+
+        # Ensure frame buffer is allocated for all non-simple paths
+        if self._frame_buffer is None or self._frame_buffer.shape != (self.display_height, self.display_width, 3):
+            self._frame_buffer = np.zeros((self.display_height, self.display_width, 3), dtype=np.uint8)
+
+        if img_w == 0:
+            self._frame_buffer[:] = 0
         else:
-            # Ensure frame buffer is allocated for all non-simple paths
-            if self._frame_buffer is None or self._frame_buffer.shape != (self.display_height, self.display_width, 3):
-                self._frame_buffer = np.zeros((self.display_height, self.display_width, 3), dtype=np.uint8)
+            # The frame runs off the strip, so it carries on from the head:
+            # frame column j is strip column (start_x + j) modulo the strip's
+            # width -- the tail and then the head, and a strip narrower than
+            # the panel repeated across it. Copying the tail and then the rest
+            # of the frame from the head assumed the head was that wide, and
+            # raised at every position for a strip narrower than the panel
+            # (Vegas composes one, with no lead-in, when its content is
+            # narrower than the chain).
+            np.take(self.cached_array, np.arange(start_x, end_x), axis=1,
+                    mode='wrap', out=self._frame_buffer)
 
-            width1 = img_w - start_x
-            if width1 > 0:
-                # Wrap-around: tail of image + head of image
-                self._frame_buffer[:, :width1] = self.cached_array[:, start_x:]
-                remaining_width = self.display_width - width1
-                self._frame_buffer[:, width1:] = self.cached_array[:, :remaining_width]
-            else:
-                # Edge case: start_x at or past image end — show from beginning,
-                # clamped to available width (scroll_position should wrap before
-                # reaching this state in normal operation).
-                available = min(self.display_width, img_w)
-                self._frame_buffer[:, :available] = self.cached_array[:, :available]
-                if available < self.display_width:
-                    self._frame_buffer[:, available:] = 0
-
-            return Image.frombytes('RGB', _size, self._frame_buffer.tobytes())
+        return Image.frombytes('RGB', _size, self._frame_buffer.tobytes())
     
     def calculate_dynamic_duration(self) -> int:
         """
@@ -1207,10 +1239,23 @@ class ScrollHelper:
             # as an idle gap. There is nothing to report, and reporting the
             # gap itself is the bug above.
             if self._window:
-                self.logger.info(
-                    "Scroll frame stats - %s",
-                    format_frame_stats(self._window),
-                )
+                # INFO when degraded, on the window that recovers from it, and
+                # as a slow heartbeat; DEBUG otherwise.
+                degraded = frame_stats_degraded(frame_stats(self._window))
+                if (degraded or self._stats_was_degraded
+                        or current_time - self._stats_last_info_log
+                        >= STATS_HEARTBEAT_INTERVAL):
+                    self.logger.info(
+                        "Scroll frame stats - %s",
+                        format_frame_stats(self._window),
+                    )
+                    self._stats_last_info_log = current_time
+                elif self.logger.isEnabledFor(logging.DEBUG):
+                    self.logger.debug(
+                        "Scroll frame stats - %s",
+                        format_frame_stats(self._window),
+                    )
+                self._stats_was_degraded = degraded
             self.last_fps_log_time = current_time
             self.frame_count = 0
             self._window = []

src/common/sports_card.py

@@ -18,7 +18,7 @@ the extra guard only stops a None size raising TypeError.
 """
 
 import logging
-from datetime import datetime, timezone
+from datetime import datetime, timedelta, timezone
 from typing import Any, Dict, Optional, Tuple
 from zoneinfo import ZoneInfo
 
@@ -338,10 +338,46 @@ def format_game_date(config: Optional[Dict[str, Any]], logger, date_text: str,
     if not raw:
         return ""
     fmt = str(scroll_card_option(config, "date_format", "abbrev") or "abbrev")
-    return _format_date_as(fmt, raw, lambda: weekday_for(config, logger, game))
+    return _format_date_as(fmt, raw, lambda: weekday_for(config, logger, game),
+                           game=game)
 
 
-def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
+def _printed_weekday(game: Optional[Dict], month: int, day: int) -> str:
+    """The weekday of the date a card prints as month/day, or '' if unknown.
+
+    The extractor prints "M/D" in the plugin's resolved zone (its own setting,
+    else the global one, else the system zone). The card cannot see that zone:
+    it is handed the plugin's config, whose ``timezone`` ships as "", so
+    card_tzinfo answers UTC and an evening kickoff in the Americas got the
+    next day's weekday ("Sat Oct 2" for a Friday game). Every zone is within
+    a day of UTC, so the printed date is the start's UTC date or a neighbour
+    of it; the one with that month and day is the date on the card.
+    """
+    if not isinstance(game, dict):
+        return ""
+    raw = game.get("start_time_utc") or game.get("start_time")
+    if not raw:
+        return ""
+    try:
+        start = raw if isinstance(raw, datetime) else datetime.fromisoformat(
+            str(raw).replace("Z", "+00:00"))
+        if start.utcoffset() is None:
+            return ""  # naive: no instant to place the date against
+        utc_day = start.astimezone(timezone.utc).date()
+    except (ValueError, TypeError, OverflowError):
+        return ""
+    for offset in (0, -1, 1):
+        try:
+            candidate = utc_day + timedelta(days=offset)
+        except OverflowError:
+            continue
+        if (candidate.month, candidate.day) == (month, day):
+            return WEEKDAY_ABBR[candidate.weekday()]
+    return ""
+
+
+def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR,
+                    game: Optional[Dict] = None) -> str:
     """Render a stripped, non-empty "M/D" *raw* in style *fmt*.
 
     The body both date formatters share. They differ in which setting names the
@@ -349,6 +385,9 @@ def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
     ``SportsCoreSharedMixin._format_game_date``), so those arrive as arguments:
     *weekday* is a zero-argument callable, only called for the "weekday" style.
     *months* lets the mixin keep reading its (overridable) ``_MONTH_ABBR``.
+    With *game*, the "weekday" style names the printed date's own weekday
+    (:func:`_printed_weekday`), and *weekday* is only the fallback for a
+    date its start time cannot place.
     """
     if fmt == "numeric":
         return raw
@@ -364,7 +403,7 @@ def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
     if fmt == "day_first":
         return f"{day} {name}"
     if fmt == "weekday":
-        day_name = weekday()
+        day_name = _printed_weekday(game, month, day) or weekday()
         return f"{day_name} {name} {day}" if day_name else f"{name} {day}"
     return f"{name} {day}"
 

src/common/sports_favorites.py

@@ -0,0 +1,263 @@
+"""Which games involve a favourite team, and which of them to show (sports family 6).
+
+The scoreboards' favourite matching, reconciled in ledmatrix-plugins
+(family 6) from seven ``_is_favorite_game`` bodies, two
+``_select_games_for_display`` and three ``_select_recent_games_for_display``
+into one each, and copied here under their existing names:
+
+- ``SportsFavoritesMixin`` (``SportsCore``): ``_is_favorite_game(game)``,
+  asked by ``SportsCoreSharedMixin._favorites_first``, the switch-mode
+  favourite boost (``SportsHelpersMixin._next_switch_index``), the
+  non-favourite live dwell (``SportsGameRulesMixin._effective_live_duration``)
+  and the plugins' live rotation; and ``_favorite_code(value)``, the
+  normalisation both sides of every comparison go through.
+- ``SportsUpcomingFavoritesMixin`` (``SportsUpcoming``):
+  ``_select_games_for_display``, the favourites-only pick of upcoming games.
+- ``SportsRecentFavoritesMixin`` (``SportsRecent``):
+  ``_select_recent_games_for_display``, the same for finished games, most
+  recent first.
+
+Each mixin carries only what its class already had, so no manager gains a
+method it did not have.
+
+THE RULE
+--------
+Each side of a game is named by ``_favorite_key(game, side)``, the override
+point ``SportsHelpersMixin`` (``src.common.sports_helpers``) has carried since
+3.5.0: the team abbreviation by default. A sport whose abbreviations are not
+unique overrides it -- NRL returns the ESPN team id (and None when the id is
+missing), because "NEW" is both Newcastle and New Zealand. That value and every
+entry of ``favorite_teams`` are compared as ``_favorite_code`` leaves them:
+stripped and upper-cased, a blank or missing value matching nothing. So
+" bos" in the config matches BOS.
+
+The selection methods give each favourite team up to the per-team limit
+(``upcoming_games_to_show`` / ``recent_games_to_show``); a game between two
+favourites counts for both. Only a game with an id can be a duplicate: two
+games without one are two games.
+
+A new module rather than more methods on ``sports_shared`` or
+``sports_helpers``, for the reason ``sports_helpers`` gives: a missing module
+fails at load, where the version checks see it; a missing method fails
+mid-update.
+
+WHAT A HOST MUST PROVIDE
+------------------------
+Derived by walking every ``self.<attr>`` the mixins read; the host-contract
+test in ``test/test_sports_favorites.py`` fails if a read is added without
+being listed here.
+
+- ``favorite_teams`` -- the resolved favourites list (``_is_favorite_game``).
+  The selection methods are handed the list instead.
+- ``_favorite_key`` -- ``SportsHelpersMixin`` supplies the default.
+- ``_favorite_code`` -- from ``SportsFavoritesMixin``, which the Upcoming and
+  Recent classes inherit through their ``SportsCore``.
+- ``logger`` -- the selection methods log each pick at DEBUG and a summary at
+  INFO.
+- ``upcoming_games_to_show`` (Upcoming) and ``recent_games_to_show`` (Recent)
+  -- the per-team limits.
+
+The methods read the game dict's ``id`` and ``start_time_utc`` (selection),
+whatever ``_favorite_key`` reads (``home_abbr`` / ``away_abbr`` by default),
+and ``home_abbr`` / ``away_abbr`` again for the DEBUG line; any may be missing.
+
+BASE ORDER
+----------
+No other mixin defines these methods, so the position in the bases does not
+change which body runs; a method on the plugin's own class still wins. The
+mixins have no ``__init__`` and no state.
+"""
+
+import logging
+from datetime import datetime, timezone
+from typing import Callable, Dict, List, Optional
+
+
+class SportsFavoritesMixin:
+    """``SportsCore``'s favourite check. See module docstring."""
+
+    # The host contract, declared for type checking only.
+    favorite_teams: List[str]
+    _favorite_key: Callable[[Dict, str], Optional[str]]
+
+    @staticmethod
+    def _favorite_code(value) -> Optional[str]:
+        """``value`` as favourites are compared: stripped and upper-cased.
+
+        None for a missing or blank value, which matches nothing.
+        """
+        if value is None:
+            return None
+        return str(value).strip().upper() or None
+
+    def _is_favorite_game(self, game: Dict) -> bool:
+        """Does either side of this game belong to a favourite team?
+
+        ``_favorite_key`` names each side (the abbreviation; nrl overrides it
+        with the ESPN team id), and both it and ``favorite_teams`` are compared
+        as ``_favorite_code`` normalises them, so " bos" matches BOS.
+        """
+        favorites = {self._favorite_code(team) for team in self.favorite_teams or ()}
+        favorites.discard(None)
+        return any(
+            self._favorite_code(self._favorite_key(game, side)) in favorites
+            for side in ("home", "away")
+        )
+
+
+class SportsUpcomingFavoritesMixin:
+    """``SportsUpcoming``'s favourites-only pick. See module docstring."""
+
+    # The host contract, declared for type checking only.
+    logger: logging.Logger
+    upcoming_games_to_show: int
+    _favorite_key: Callable[[Dict, str], Optional[str]]
+    _favorite_code: Callable[[object], Optional[str]]
+
+    def _select_games_for_display(
+        self, processed_games: List[Dict], favorite_teams: List[str]
+    ) -> List[Dict]:
+        """
+        Single-pass game selection with proper deduplication and counting.
+
+        When a game involves two favorite teams, it counts toward BOTH teams' limits.
+        This prevents unexpected game counts from the multi-pass algorithm.
+        Teams are matched as _is_favorite_game matches them. Only a game with
+        an id can be a duplicate: two games without one are two games.
+        """
+        sorted_games = sorted(
+            processed_games,
+            key=lambda g: g.get("start_time_utc")
+            or datetime.max.replace(tzinfo=timezone.utc),
+        )
+
+        if not favorite_teams:
+            return sorted_games
+
+        selected_games = []
+        selected_ids = set()
+        team_counts: Dict[Optional[str], int] = {
+            code: 0 for code in map(self._favorite_code, favorite_teams) if code
+        }
+
+        for game in sorted_games:
+            game_id = game.get("id")
+            if game_id is not None and game_id in selected_ids:
+                continue
+
+            home = self._favorite_code(self._favorite_key(game, "home"))
+            away = self._favorite_code(self._favorite_key(game, "away"))
+
+            home_fav = home in team_counts
+            away_fav = away in team_counts
+
+            if not home_fav and not away_fav:
+                continue
+
+            home_needs = home_fav and team_counts[home] < self.upcoming_games_to_show
+            away_needs = away_fav and team_counts[away] < self.upcoming_games_to_show
+
+            if home_needs or away_needs:
+                selected_games.append(game)
+                if game_id is not None:
+                    selected_ids.add(game_id)
+                if home_fav:
+                    team_counts[home] += 1
+                if away_fav:
+                    team_counts[away] += 1
+
+                self.logger.debug(
+                    f"Selected game {game.get('away_abbr')}@{game.get('home_abbr')}: "
+                    f"team_counts={team_counts}"
+                )
+
+            if all(c >= self.upcoming_games_to_show for c in team_counts.values()):
+                self.logger.debug("All favorite teams satisfied, stopping selection")
+                break
+
+        self.logger.info(
+            f"Selected {len(selected_games)} games for {len(favorite_teams)} "
+            f"favorite teams: {team_counts}"
+        )
+        return selected_games
+
+
+class SportsRecentFavoritesMixin:
+    """``SportsRecent``'s favourites-only pick. See module docstring."""
+
+    # The host contract, declared for type checking only.
+    logger: logging.Logger
+    recent_games_to_show: int
+    _favorite_key: Callable[[Dict, str], Optional[str]]
+    _favorite_code: Callable[[object], Optional[str]]
+
+    def _select_recent_games_for_display(
+        self, processed_games: List[Dict], favorite_teams: List[str]
+    ) -> List[Dict]:
+        """
+        Single-pass game selection for recent games with proper deduplication.
+
+        When a game involves two favorite teams, it counts toward BOTH teams' limits.
+        Games are sorted by most recent first.
+        Teams are matched as _is_favorite_game matches them. Only a game with
+        an id can be a duplicate: two games without one are two games.
+        """
+        sorted_games = sorted(
+            processed_games,
+            key=lambda g: g.get("start_time_utc")
+            or datetime.min.replace(tzinfo=timezone.utc),
+            reverse=True,
+        )
+
+        if not favorite_teams:
+            return sorted_games
+
+        selected_games = []
+        selected_ids = set()
+        team_counts: Dict[Optional[str], int] = {
+            code: 0 for code in map(self._favorite_code, favorite_teams) if code
+        }
+
+        for game in sorted_games:
+            game_id = game.get("id")
+            if game_id is not None and game_id in selected_ids:
+                continue
+
+            home = self._favorite_code(self._favorite_key(game, "home"))
+            away = self._favorite_code(self._favorite_key(game, "away"))
+
+            home_fav = home in team_counts
+            away_fav = away in team_counts
+
+            if not home_fav and not away_fav:
+                continue
+
+            home_needs = home_fav and team_counts[home] < self.recent_games_to_show
+            away_needs = away_fav and team_counts[away] < self.recent_games_to_show
+
+            if home_needs or away_needs:
+                selected_games.append(game)
+                if game_id is not None:
+                    selected_ids.add(game_id)
+                if home_fav:
+                    team_counts[home] += 1
+                if away_fav:
+                    team_counts[away] += 1
+
+                self.logger.debug(
+                    f"Selected recent game {game.get('away_abbr')}@{game.get('home_abbr')}: "
+                    f"team_counts={team_counts}"
+                )
+
+            if all(c >= self.recent_games_to_show for c in team_counts.values()):
+                self.logger.debug("All favorite teams satisfied, stopping selection")
+                break
+
+        self.logger.info(
+            f"Selected {len(selected_games)} recent games for {len(favorite_teams)} "
+            f"favorite teams: {team_counts}"
+        )
+        return selected_games
+
+
+__all__ = ["SportsFavoritesMixin", "SportsUpcomingFavoritesMixin", "SportsRecentFavoritesMixin"]

src/common/sports_game_over.py

@@ -0,0 +1,125 @@
+"""Whether a game ESPN still lists as live has in fact ended (sports family 5).
+
+``SportsGameOverMixin._is_game_really_over`` is the scoreboards'
+``SportsLive._is_game_really_over``, reconciled in ledmatrix-plugins
+#625 from five bodies into one and copied here under
+its existing name. ``SportsLiveSharedMixin._detect_stale_games``
+(``src.common.sports_shared``) calls it on every live game, and the plugins'
+live-priority filters call it too, to drop a game ESPN still reports as
+in progress.
+
+A game is over when its period text says final. From period ``FINAL_PERIOD``
+on, a clock reading 0:00 ends it too, unless the score is level: a tie at the
+end of regulation goes to overtime (or a shootout), and a game that does end
+tied says final. Only a clock *string* is read ("0:00" and ":00" are zero;
+":40", "0.0" and ESPN's "-" between MMA rounds are not), and a missing or
+unreadable score leaves the decision to the clock.
+
+``FINAL_PERIOD`` is the one per-sport fact, a class attribute rather than a
+sport-name branch. The scoreboards declare it on their ``SportsLive``:
+
+- 3: hockey;
+- 4: basketball, football, lacrosse;
+- ``None`` (this default; the clock never ends a game): afl, nrl and soccer,
+  whose clocks count up; baseball, which has innings; ufc, whose bouts end
+  only on ESPN's final status.
+
+A sport can still override the method and defer to it, as baseball's
+``BaseballLive`` does to end postponed and suspended games first.
+
+A new module rather than another method on ``sports_shared``, for the reason
+``sports_helpers`` gives: a missing module fails at load, where the version
+checks see it; a missing method fails mid-update.
+
+WHAT A HOST MUST PROVIDE
+------------------------
+Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
+test in ``test/test_sports_game_over.py`` fails if a read is added without
+being listed here.
+
+- ``logger`` -- a ``logging.Logger``; the method logs its verdict at DEBUG.
+- ``FINAL_PERIOD`` -- defaulted here to ``None``; set it on the host class.
+
+The method reads the game dict's ``away_abbr``, ``home_abbr``,
+``period_text``, ``period``, ``clock``, ``away_score`` and ``home_score``
+(``_extract_game_details_common``'s keys); any of them may be missing or
+null.
+
+BASE ORDER
+----------
+List the mixin before ``SportsLiveSharedMixin`` --
+``class SportsLive(SportsGameOverMixin, SportsLiveSharedMixin, SportsCore)`` --
+so the shared mixin's ``_detect_stale_games`` finds this method through the
+MRO. Neither shared mixin defines it, so the order does not change which body
+runs today; it keeps the method next to its caller should one ever be added
+there. A method on the plugin's own class still wins, and its ``super()``
+reaches this one. The mixin has no ``__init__`` and no state.
+"""
+
+import logging
+from typing import Dict, Optional
+
+
+class SportsGameOverMixin:
+    """The live manager's "is this game really over?" check. See module docstring."""
+
+    # The host contract, declared for type checking only.
+    logger: logging.Logger
+
+    #: Period from which a 0:00 clock ends a game; None: the clock never does.
+    FINAL_PERIOD: Optional[int] = None
+
+    def _is_game_really_over(self, game: Dict) -> bool:
+        """Whether a game ESPN still lists as live has in fact ended.
+
+        It has when its period text says final. From period ``FINAL_PERIOD``
+        on, a clock at 0:00 ends it too, unless the score is level: a tie at
+        the end of regulation goes to overtime, and a game that does end tied
+        says final. With ``FINAL_PERIOD = None`` the clock never ends a game.
+        """
+        game_str = f"{game.get('away_abbr')}@{game.get('home_abbr')}"
+
+        # ESPN can send the key as null, and .get()'s default only covers a
+        # missing key, so a None here crashed the whole live update.
+        raw_period_text = game.get("period_text")
+        period_text = raw_period_text.lower() if isinstance(raw_period_text, str) else ""
+        if "final" in period_text:
+            self.logger.debug(
+                f"_is_game_really_over({game_str}): "
+                f"returning True - 'final' in period_text='{period_text}'"
+            )
+            return True
+
+        # Same for a null or non-numeric period: treat it as period 0.
+        try:
+            period = int(game.get("period") or 0)
+        except (TypeError, ValueError, OverflowError):
+            period = 0
+        # Only a clock string is read: "0:00" and ":00" are zero; ":40" is not.
+        clock = game.get("clock")
+        clock_at_zero = isinstance(clock, str) and clock.replace(":", "").strip() in ("000", "00")
+
+        if self.FINAL_PERIOD is not None and period >= self.FINAL_PERIOD and clock_at_zero:
+            try:
+                tied = int(game["away_score"]) == int(game["home_score"])
+            except (KeyError, TypeError, ValueError, OverflowError):
+                tied = False  # a missing or unreadable score leaves it to the clock
+            if not tied:
+                self.logger.debug(
+                    f"_is_game_really_over({game_str}): "
+                    f"returning True - clock at 0:00 (clock='{clock}', period={period})"
+                )
+                return True
+            self.logger.debug(
+                f"_is_game_really_over({game_str}): "
+                f"returning False - tied at 0:00 (period={period}), overtime next"
+            )
+            return False
+
+        self.logger.debug(
+            f"_is_game_really_over({game_str}): returning False"
+        )
+        return False
+
+
+__all__ = ["SportsGameOverMixin"]

src/common/sports_helpers.py

@@ -34,8 +34,9 @@ first core release that ships it (see ``CHANGELOG.md``).
 
 ``_favorite_key`` is the one method not taken from the plugins: it is the
 override point from the since-removed ``src/base_classes`` sports core,
-carried here so later phases (shared celebrations and game selection) have a
-hardware-free home for the seam. No plugin defines it today and nothing in this module calls it.
+carried here as the hardware-free home for the seam. ``sports_favorites``
+calls it; nrl overrides it with the ESPN team id. Nothing in this module
+calls it.
 
 WHAT A HOST MUST PROVIDE
 ------------------------
@@ -47,8 +48,8 @@ listed here.
 - ``mode_config`` (dict) and ``logger`` -- ``_setting_int``. ``league`` is
   read with ``getattr`` for the warning text only.
 - ``games_list`` and ``current_game_index`` -- ``_next_switch_index``; plus
-  ``_is_favorite_game`` (called with a game), which stays per-plugin and is
-  only called when
+  ``_is_favorite_game`` (called with a game; ``sports_favorites`` has the
+  shared body), only called when
   ``favorite_rotation_boost`` is above 1. ``favorite_rotation_boost`` itself
   defaults to 1 on the mixin.
 - ``last_game_switch`` -- ``_reset_dwell_on_reentry``, read with ``getattr``
@@ -216,15 +217,12 @@ class SportsHelpersMixin:
         rather than a branch so core never has to learn the string "nrl"::
 
             def _favorite_key(self, game, side):
-                return str(game.get(f"{side}_id"))
+                team_id = game.get(f"{side}_id")
+                return None if team_id is None else str(team_id)
 
-        An override that stringifies should note that a missing id becomes the
-        literal ``"None"``, which would spuriously match a favorites list
-        containing that string. The default returns ``None`` for a missing
-        abbreviation, which never matches.
-
-        Carried from the since-removed ``src/base_classes`` sports core for
-        later phases; nothing in this module calls it yet.
+        It returns ``None`` for a missing id rather than ``str(None)``, which
+        would match a favourite typed "None". A ``None`` never matches.
+        ``sports_favorites`` compares the value stripped and upper-cased.
         """
         return game.get(f"{side}_abbr")
 

src/common/sports_scroll.py

@@ -323,7 +323,7 @@ class SportsScrollDisplay:
         :returns: True if a frame was drawn; False when there is no content or
             the frame could not be rendered.
         """
-        if not self.scroll_helper.cached_image:
+        if not self._has_strip():
             return False
 
         try:
@@ -416,7 +416,21 @@ class SportsScrollDisplay:
 
     def has_cached_content(self) -> bool:
         """Whether content is prepared and ready to scroll."""
-        return bool(self.scroll_helper.cached_image)
+        return self._has_strip()
+
+    def _has_strip(self) -> bool:
+        """Whether the helper holds a strip, without building its PIL image.
+
+        Reading ``cached_image`` after the strip was extended or trimmed builds
+        the image from the array and keeps it, so the strip is held twice;
+        display_scroll_frame asks this every frame. ``has_strip()`` answers
+        from the helper's bookkeeping. A helper without it (a plugin's own, a
+        test double) is asked the old way.
+        """
+        helper = self.scroll_helper
+        if callable(getattr(type(helper), "has_strip", None)):
+            return bool(helper.has_strip())
+        return bool(helper.cached_image)
 
     # ------------------------------------------------------------------
     # Live Vegas cards

src/common/sports_shared.py

@@ -57,7 +57,9 @@ Methods that stay per-plugin, because they are not identical across the eight
 ``_get_layout_offset``, ``_by_importance``, ``_other_games_window``,
 ``_upcoming_date_and_time_text``, ``_extract_game_details_common``,
 ``_load_division_team_ids``, ``_get_timezone``, ``_is_favorite_game``,
-``_is_game_really_over``, ``_is_ranked_game``, ``_passes_other_filters``.
+``_is_ranked_game``, ``_passes_other_filters``. (``_is_game_really_over``,
+which ``_detect_stale_games`` below calls, was here too until the plugins
+reconciled it; it is now ``src.common.sports_game_over``.)
 
 Of the fourteen shared class constants, thirteen are identical everywhere and
 live here. Only ``_SCORE_PROBE_TEXT`` varies -- afl and basketball reach three digits
@@ -120,6 +122,32 @@ _DEFAULT_LIVE_IDLE_MAX_SECONDS = 900
 _KICKOFF_GRACE_SECONDS = 900
 #: Fallback cadence around a kickoff when the manager has no update_interval.
 _KICKOFF_POLL_FLOOR = 30
+#: How many kickoffs after the current one a live manager remembers. Only the
+#: earliest few can matter before the next look refreshes the list, so this
+#: bounds the memory without dropping a kickoff the board would wait for.
+_KICKOFF_QUEUE_MAX = 8
+
+
+def _current_scheduled_start(host: Any, now: float) -> Optional[float]:
+    """The kickoff a live manager is honouring now, promoting the next queued one.
+
+    ``_next_scheduled_start_ts`` is the kickoff being honoured: the earliest
+    one ahead of us, or one that has just passed and is inside its grace.
+    Kickoffs behind it wait in ``_later_scheduled_starts``. When the current
+    one's grace runs out, the earliest queued kickoff that is not itself past
+    its grace takes over -- including one that has already passed, so a
+    second kickoff inside the first one's grace still gets a grace of its own.
+    """
+    current: Optional[float] = getattr(host, "_next_scheduled_start_ts", None)
+    if current and current > now - _KICKOFF_GRACE_SECONDS:
+        return current
+    queued: Optional[List[float]] = getattr(host, "_later_scheduled_starts", None)
+    if queued:
+        alive = sorted(s for s in queued if s > now - _KICKOFF_GRACE_SECONDS)
+        current = alive.pop(0) if alive else None
+        host._later_scheduled_starts = alive
+        host._next_scheduled_start_ts = current
+    return current if current and current > now - _KICKOFF_GRACE_SECONDS else None
 
 
 def _resolve_font_path(path: str) -> str:
@@ -360,14 +388,16 @@ class SportsCoreSharedMixin:
         The formatting is sports_card's. What differs from the card's
         ``format_game_date`` is passed in: the setting (``switch_date_format``,
         see :meth:`_switch_date_format`) and the weekday, which comes from
-        :meth:`_weekday_for` and so from this plugin's resolved timezone.
+        :meth:`_weekday_for` and so from this plugin's resolved timezone
+        when the game's start cannot place the printed date. The game goes
+        in too, so both formatters name the printed date's own weekday.
         """
         raw = str(date_text or "").strip()
         if not raw:
             return raw
         return _card._format_date_as(self._switch_date_format(), raw,
                                      lambda: self._weekday_for(game),
-                                     self._MONTH_ABBR)
+                                     self._MONTH_ABBR, game=game)
 
     def _weekday_for(self, game: Optional[Dict]) -> str:
         """Weekday abbreviation from the game's start time, or ''."""
@@ -1292,11 +1322,11 @@ class SportsLiveSharedMixin:
           otherwise look like another empty check and escalate the back-off
           again, right when the game is actually starting.
         """
-        start = getattr(self, "_next_scheduled_start_ts", None)
+        now = time.time()
+        start = _current_scheduled_start(self, now)
         if not start:
             return interval
         live = getattr(self, "update_interval", None) or _KICKOFF_POLL_FLOOR
-        now = time.time()
         if now < start:
             return max(live, min(interval, int(start - now)))
         if now - start <= _KICKOFF_GRACE_SECONDS:
@@ -1311,6 +1341,16 @@ class SportsLiveSharedMixin:
         already has. Self-correcting: a stored start that has passed is
         replaced by the next one offered, so a postponed game cannot pin the
         cadence to a kickoff that never happens.
+
+        Every pending kickoff is honoured, not just the first. A kickoff that
+        arrives while an earlier one is inside its grace is queued in
+        ``_later_scheduled_starts`` (the earliest _KICKOFF_QUEUE_MAX of them)
+        and takes over when that grace ends, with a grace of its own. Keeping
+        only the one kickoff dropped the second of two favourites starting
+        within the grace of each other: it was refused while the first held
+        the slot, and refused again once it had passed, so if ESPN had not
+        flipped it live by the end of the first grace the back-off went
+        straight back to its ceiling and the game was noticed up to that late.
         """
         if not isinstance(details, dict):
             return
@@ -1327,7 +1367,7 @@ class SportsLiveSharedMixin:
         now = time.time()
         if candidate <= now:
             return
-        current = getattr(self, "_next_scheduled_start_ts", None)
+        current = _current_scheduled_start(self, now)
         # A kickoff that has only just passed is *kept*, not replaced by the
         # next one on the card. Replacing it immediately is what made the grace
         # window in _clamp_to_scheduled_start dead code: the moment 13:00 came
@@ -1337,10 +1377,20 @@ class SportsLiveSharedMixin:
         # polled at 13:00:45, found nothing live because ESPN had not flipped
         # the status yet, and then went quiet for the next quarter of an hour,
         # which is the behaviour this whole clamp exists to prevent.
-        if (current is None
-                or current <= now - _KICKOFF_GRACE_SECONDS
-                or candidate < current):
+        #
+        # Nor is it forgotten: whichever kickoff loses is queued behind the
+        # one honoured now, so it gets its own grace when that one's ends.
+        if current is None:
             self._next_scheduled_start_ts = candidate
+            return
+        if candidate == current:
+            return
+        if candidate < current:
+            self._next_scheduled_start_ts, candidate = candidate, current
+        queued = getattr(self, "_later_scheduled_starts", None) or []
+        if candidate not in queued:
+            self._later_scheduled_starts = sorted(
+                [*queued, candidate])[:_KICKOFF_QUEUE_MAX]
 
     #: How long a game that finished live is still reported by
     #: finished_games_snapshot(): long enough for the recent-games list, which

src/common/sync_manager.py

@@ -29,7 +29,6 @@ import time
 import logging
 from enum import Enum
 from typing import Callable, Optional
-import numpy as np
 from PIL import Image
 
 from src.config_manager_atomic import _replace
@@ -434,6 +433,12 @@ class DisplaySyncManager:
             return
         if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
             return
+        # numpy is imported here, not at module level: the web interface
+        # imports this module for its constants (STATUS_FILE, SYNC_PORT) and
+        # would otherwise load numpy for nothing. Only a connected leader
+        # gets this far, and after the first frame the import is a
+        # sys.modules lookup.
+        import numpy as np
         try:
             arr = np.asarray(image.convert("RGB"), dtype=np.uint8)
             header = _RAW_MAGIC + _RAW_HEADER.pack(image.width, image.height)

src/config_manager.py

@@ -46,6 +46,35 @@ from src.common.permission_utils import (
     get_config_dir_mode
 )
 
+
+def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
+    """A deep copy of ``config`` that shares nothing with it.
+
+    load_config() hands one out per call, and the saves keep one, so the
+    cached config is never an object a caller holds. A web handler edits what
+    it loaded, validates, and may refuse the save; when the cache was that
+    same object, the refused edit stayed in it, and the next save of any
+    other setting wrote it to config.json -- a nested secret included, in
+    plain text, since it had never reached config_secrets.json to be
+    stripped.
+
+    The config is JSON data, so only its dicts and lists need copying; every
+    other value in it is immutable. On a Pi 4 with a real 60 KiB config this
+    takes 2.1 ms against copy.deepcopy's 6.8 ms, on a path ~30 handlers call
+    (a pickle round trip is no faster, 1.9 ms, and brings pickle into the
+    config path for nothing).
+    """
+    return _copy_containers(config)
+
+
+def _copy_containers(value: Any) -> Any:
+    if isinstance(value, dict):
+        return {key: _copy_containers(item) for key, item in value.items()}
+    if isinstance(value, list):
+        return [_copy_containers(item) for item in value]
+    return value
+
+
 class ConfigManager:
     """
     Reads and writes the main application configuration files.
@@ -126,9 +155,10 @@ class ConfigManager:
             validate_after_write=validate_after_write
         )
         
-        # Update in-memory config if save was successful
+        # Update in-memory config if save was successful. A copy: the caller
+        # still holds new_config_data (see _private_copy).
         if result.status == SaveResultStatus.SUCCESS:
-            self.config = new_config_data
+            self.config = _private_copy(new_config_data)
             # In-memory config now matches what was just written, so the
             # load_config fast path may return it. It still carries the
             # merged secrets that were stripped on disk; that matches a full
@@ -208,14 +238,16 @@ class ConfigManager:
 
         Fast path: when config.json, config_secrets.json and the template
         are all unchanged since the last successful load (mtime_ns + size),
-        the already-parsed self.config is returned without touching the
-        files — same aliasing semantics as the full path, which also
-        returns self.config.
+        a copy of the already-parsed self.config is returned without
+        touching the files.
+
+        Either way the caller gets its own copy (see _private_copy): editing
+        it changes nothing here until it is saved.
         """
         try:
             current_sig = self._files_signature()
             if self.config and self._loaded_sig == current_sig:
-                return self.config
+                return _private_copy(self.config)
 
             # Check if config file exists, if not create from template
             if not os.path.exists(self.config_path):
@@ -249,7 +281,7 @@ class ConfigManager:
             # Signature taken AFTER load + migration (migration may write the
             # config back), so it reflects exactly what was read/written.
             self._loaded_sig = self._files_signature()
-            return self.config
+            return _private_copy(self.config)
 
         except FileNotFoundError as e:
             # Only config.json can get here: a missing or unreadable secrets
@@ -355,8 +387,9 @@ class ConfigManager:
         try:
             atomic_write_json(self.config_path, config_to_write)
 
-            # Update the in-memory config to the new state (which includes secrets for runtime)
-            self.config = new_config_data
+            # Update the in-memory config to the new state (which includes
+            # secrets for runtime), as a copy -- see _private_copy
+            self.config = _private_copy(new_config_data)
             self._loaded_sig = self._files_signature()
             self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
             if secrets_content:

src/config_service.py

@@ -14,7 +14,7 @@ import json
 import time
 import threading
 from pathlib import Path
-from typing import Dict, Any, Optional, List, Callable
+from typing import Dict, Any, Optional, List, Callable, Tuple
 from collections import defaultdict
 import logging
 import hashlib
@@ -52,6 +52,17 @@ class ConfigService:
         
         # Thread safety
         self._lock: threading.RLock = threading.RLock()
+        # Held across a whole reload -- read, swap, notify -- so one reload's
+        # notifications finish before the next one's start. Subscribers run
+        # under this lock and never under _lock: the display's per-plugin
+        # subscriber can wait seconds for a busy plugin, and get_config(),
+        # subscribe() and unsubscribe() -- called from the render thread --
+        # must not wait behind it.
+        self._notify_lock: threading.RLock = threading.RLock()
+        # (key, callback, thread id) of the callback a notification is running,
+        # so unsubscribe() can wait for that one call; signalled on its return.
+        self._running_callback: Optional[Tuple[str, Callable[..., None], int]] = None
+        self._callback_done = threading.Condition(self._lock)
 
         # Current configuration
         self._current_config: Dict[str, Any] = {}
@@ -87,23 +98,24 @@ class ConfigService:
             True if config changed, False otherwise
         """
         try:
-            new_config = self.config_manager.load_config()
-            new_checksum = self._calculate_checksum(new_config)
+            with self._notify_lock:
+                new_config = self.config_manager.load_config()
+                new_checksum = self._calculate_checksum(new_config)
 
-            with self._lock:
-                # Check if config actually changed
-                if new_checksum == self._current_checksum:
-                    self.logger.debug("Configuration unchanged, skipping reload")
-                    return False
+                with self._lock:
+                    # Check if config actually changed
+                    if new_checksum == self._current_checksum:
+                        self.logger.debug("Configuration unchanged, skipping reload")
+                        return False
 
-                # Store old config for change detection
-                old_config = self._current_config.copy()
+                    # Store old config for change detection
+                    old_config = self._current_config.copy()
 
-                # Update current config
-                self._current_config = new_config
-                self._current_checksum = new_checksum
+                    # Update current config
+                    self._current_config = new_config
+                    self._current_checksum = new_checksum
 
-                # Notify subscribers
+                # Notify subscribers, outside _lock (see _notify_lock)
                 self._notify_subscribers(old_config, new_config)
 
                 self.logger.info(
@@ -127,16 +139,19 @@ class ConfigService:
         Args:
             old_config: Previous configuration
             new_config: New configuration
+
+        Called without _lock held. The subscriber lists are copied under it,
+        and each callback is checked against them again just before it runs.
         """
+        with self._lock:
+            subscribers = {key: list(callbacks) for key, callbacks in self._subscribers.items()}
+
         # Notify global subscribers (key: '*')
-        for callback in self._subscribers.get('*', []):
-            try:
-                callback(old_config, new_config)
-            except Exception as e:
-                self.logger.error("Error in global config change callback: %s", e, exc_info=True)
+        for callback in subscribers.get('*', []):
+            self._call_subscriber('*', callback, old_config, new_config)
 
         # Notify plugin-specific subscribers
-        for plugin_id in self._subscribers.keys():
+        for plugin_id, callbacks in subscribers.items():
             if plugin_id == '*':
                 continue
 
@@ -145,16 +160,42 @@ class ConfigService:
 
             # Only notify if plugin config actually changed
             if old_plugin_config != new_plugin_config:
-                for callback in self._subscribers[plugin_id]:
-                    try:
-                        callback(old_plugin_config, new_plugin_config)
-                    except Exception as e:
-                        self.logger.error(
-                            "Error in config change callback for %s: %s",
-                            plugin_id,
-                            e,
-                            exc_info=True
-                        )
+                for callback in callbacks:
+                    self._call_subscriber(plugin_id, callback,
+                                          old_plugin_config, new_plugin_config)
+
+    def _call_subscriber(
+        self,
+        key: str,
+        callback: Callable[[Dict[str, Any], Dict[str, Any]], None],
+        old_config: Dict[str, Any],
+        new_config: Dict[str, Any],
+    ) -> None:
+        """Run one callback, unless it was unsubscribed since the snapshot.
+
+        unsubscribe() promises that once it returns the callback is neither
+        running nor will run: the display unloads the plugin straight after.
+        """
+        with self._lock:
+            if callback not in self._subscribers.get(key, ()):
+                return
+            self._running_callback = (key, callback, threading.get_ident())
+        try:
+            callback(old_config, new_config)
+        except Exception as e:
+            if key == '*':
+                self.logger.error("Error in global config change callback: %s", e, exc_info=True)
+            else:
+                self.logger.error(
+                    "Error in config change callback for %s: %s",
+                    key,
+                    e,
+                    exc_info=True
+                )
+        finally:
+            with self._lock:
+                self._running_callback = None
+                self._callback_done.notify_all()
 
     def _check_file_changes(self) -> bool:
         """
@@ -276,6 +317,11 @@ class ConfigService:
         """
         Unsubscribe from configuration changes.
         
+        Once this returns the callback is not running and will not be called
+        again. A notification that is running this very callback is waited
+        for (unless the callback is the caller); one running any other
+        callback is not.
+
         Args:
             callback: Callback function to remove
             plugin_id: Optional plugin ID (must match subscription)
@@ -285,6 +331,10 @@ class ConfigService:
             if callback in self._subscribers[key]:
                 self._subscribers[key].remove(callback)
                 self.logger.debug("Unsubscribed from config changes for %s", key)
+            while (self._running_callback is not None
+                   and self._running_callback[:2] == (key, callback)
+                   and self._running_callback[2] != threading.get_ident()):
+                self._callback_done.wait()
     
     def shutdown(self) -> None:
         """Shutdown the configuration service."""

src/display_arbiter.py

@@ -1,41 +1,56 @@
 """What the panel shows next: the Arbiter of docs/RUN_LOOP_REDESIGN.md.
 
 ``Arbiter.decide(state, inputs, now)`` takes a snapshot that
-``DisplayController.run()`` gathers once per pass and returns a
-:class:`ScreenPlan` naming the Source that gets the panel. It is a pure
-function: no I/O, no clock reads (``now`` is passed in), no locks, and it
-changes nothing it is given. That is what lets a plain table of cases test
-the priority order, which used to exist only as the order of ``if`` blocks
-in ``run()``.
+``DisplayController.run()`` gathers and returns a :class:`ScreenPlan` naming
+the Source that gets the panel. It is a pure function: no I/O, no clock
+reads (``now`` is passed in), no locks, and it changes nothing it is given.
+That is what lets a plain table of cases test the priority order, which
+used to exist only as the order of ``if`` blocks in ``run()``.
 
-The full order is
+The order is
 
     ScheduledOff (a gate), Follower, OnDemand, Wifi, Live, Vegas, Rotation
 
-Stage 2 decides the gate, Follower and Wifi. Every other case returns a
-``LEGACY`` plan, meaning "carry on with run()'s existing code" (live
-priority, Vegas, then one rotation screen). OnDemand is in the order already
-because it outranks the WiFi notice: an active session is a ``LEGACY`` plan
-even when a notice is pending.
+Every Source but Vegas is decided here (stage 3). Vegas is the ``LEGACY``
+plan: the Arbiter picks it, but its iteration is still run()'s own code
+until stage 4.
 
-The Wifi Source's mid-screen rule, :func:`wifi_notice_preempts`, lives here
-too, so both of its answers -- at the top of a pass and between frames --
-come from one module.
+A pass asks twice: once with the inputs every pass reads (the gate,
+Follower, OnDemand, Wifi), and once more, only when nothing above the
+notice took the panel, with the inputs the Sources below it need (whether
+Vegas is on, the live-priority scan), read where run() always read them.
+
+``decide(..., running=plan)`` is the other question, asked by the
+ScreenRunner (src/screen_runner.py) at its service points: does a Source
+in ``plan.preemptible_by`` now take the panel from the screen that is
+running? The mid-screen rules are :func:`_hold_or_preempt`.
+
+The state transitions (the next on-demand mode, a live claim and its
+release, the rotation's step after a screen) are pure methods of
+:class:`ArbiterState`; the controller applies what they return.
 """
 
-from dataclasses import dataclass
+from dataclasses import dataclass, replace
 from enum import Enum
-from typing import Optional
+from typing import FrozenSet, Optional, Protocol, Tuple
 
 __all__ = [
     "Arbiter",
     "ArbiterInputs",
     "ArbiterState",
+    "FramePolicy",
+    "LIVE_PREEMPTERS",
     "SCHEDULED_OFF_DWELL",
+    "SCREEN_PREEMPTERS",
+    "ScreenEnd",
     "ScreenPlan",
     "Source",
     "WIFI_NOTICE_DWELL",
     "WifiNotice",
+    "live_pick",
+    "live_takeover",
+    "on_demand_bound",
+    "rotation_plan",
     "wifi_notice_preempts",
 ]
 
@@ -53,11 +68,41 @@ class Source(Enum):
 
     SCHEDULED_OFF = "scheduled-off"
     FOLLOWER = "follower"
+    ON_DEMAND = "on-demand"
     WIFI = "wifi"
-    # Not decided by the Arbiter yet: on-demand, live priority, Vegas and the
-    # rotation are still chosen by run()'s own code. Stage 3 adds the
-    # OnDemand, Live and Rotation Sources; stage 4 adds Vegas.
+    LIVE = "live"
+    # Vegas: the Arbiter picks it, but its iteration is still run()'s own
+    # code (and its interrupt callback a second copy of this order) until
+    # stage 4 makes it a Source driven frame by frame.
     LEGACY = "legacy"
+    ROTATION = "rotation"
+    # Not a screen: a plugin reload waits at the top of the loop. It ends a
+    # screen between frames (the screen counts as shown and the rotation
+    # moves on), and the next pass reloads before it draws.
+    RELOAD = "reload"
+
+
+class FramePolicy(Enum):
+    """How often a screen draws: today's two frame loops (see
+    DisplayController._needs_high_fps). Stage 5 lets plugins declare it."""
+
+    #: The 125 Hz loop, paced to an 8 ms deadline: scrolling plugins.
+    HIGH_FPS = "high-fps"
+    #: The 1 Hz loop.
+    STATIC = "static"
+
+
+class ScreenEnd(Protocol):
+    """What ArbiterState.after needs to know about how a screen ended
+    (screen_runner.Outcome, filled in by the controller)."""
+
+    @property
+    def on_demand_active(self) -> bool:
+        """An on-demand session was running when the screen ended."""
+
+    @property
+    def still_live(self) -> bool:
+        """The mode's plugin still had live content: hold the rotation."""
 
 
 @dataclass(frozen=True)
@@ -76,11 +121,122 @@ class WifiNotice:
 class ArbiterState:
     """What the Arbiter remembers between passes.
 
-    Nothing yet: the stage-2 Sources decide from the inputs alone. The
-    on-demand index, the rotation index and the live resume point move here
-    with their Sources in stage 3.
+    A snapshot of the controller's own fields, taken when decide() is
+    called (DisplayController._arbiter_state); the transitions below return
+    the next state, and the controller writes it back.
+
+    Attributes:
+        current_mode: The mode on the panel or about to be
+            (``current_display_mode``).
+        on_demand_modes: The on-demand session's modes, in the order it
+            shows them (a pinned mode already moved to the front when the
+            session started, by _apply_on_demand_pin).
+        on_demand_index: Which of them is showing.
+        on_demand_expires_at: When the session ends (wall clock), or None
+            for a session with no duration.
+        on_demand_pinned: The session was started pinned. Carried for the
+            snapshot; the pin itself is already in ``on_demand_modes``.
+        rotation: The rotation's modes (``available_modes``).
+        rotation_index: Where the rotation is (``current_mode_index``).
+        live_resume_index: Where the rotation was when live priority took
+            the panel, so it resumes there once nothing is live; None while
+            live priority holds nothing.
+        live_takeover_unshown: A mid-screen takeover chose current_mode and
+            it has not been shown yet, so the next pass must not advance the
+            live round-robin past it.
     """
 
+    current_mode: Optional[str] = None
+    on_demand_modes: Tuple[str, ...] = ()
+    on_demand_index: int = 0
+    on_demand_expires_at: Optional[float] = None
+    on_demand_pinned: bool = False
+    rotation: Tuple[str, ...] = ()
+    rotation_index: int = 0
+    live_resume_index: Optional[int] = None
+    live_takeover_unshown: bool = False
+
+    def next_on_demand(self) -> "ArbiterState":
+        """The session's next mode, wrapping round. Needs a mode list."""
+        index = (self.on_demand_index + 1) % len(self.on_demand_modes)
+        return replace(self, on_demand_index=index,
+                       current_mode=self.on_demand_modes[index])
+
+    def claim_live(self, mode: str) -> "ArbiterState":
+        """Live priority takes the panel for ``mode``.
+
+        The rotation's position is saved only on the first claim, not on
+        each re-check while the hold continues, so it resumes where live
+        priority interrupted it instead of after the live mode (which would
+        skip every mode between the two).
+        """
+        if self.current_mode == mode:
+            return self
+        resume = self.rotation_index if self.live_resume_index is None else self.live_resume_index
+        index = self.rotation.index(mode) if mode in self.rotation else self.rotation_index
+        return replace(self, current_mode=mode, rotation_index=index,
+                       live_resume_index=resume)
+
+    def after(self, outcome: "ScreenEnd") -> "ArbiterState":
+        """The state once a screen has run its course: the next mode.
+
+        An on-demand session moves to its next mode. Otherwise the rotation
+        advances -- unless the mode just shown is a live-priority mode that
+        is still live, which holds the panel. A session with no modes left
+        is ended by the controller before it asks (that is not pure: it
+        resumes the rotation and clears the cache).
+        """
+        if outcome.on_demand_active:
+            return self.next_on_demand() if self.on_demand_modes else self
+        if outcome.still_live or not self.rotation:
+            return self
+        index = (self.rotation_index + 1) % len(self.rotation)
+        return replace(self, rotation_index=index, current_mode=self.rotation[index])
+
+    def release_live(self) -> "ArbiterState":
+        """Nothing is live any more: the rotation resumes where it was."""
+        if self.live_resume_index is None or not self.rotation:
+            return self
+        index = self.live_resume_index % len(self.rotation)
+        return replace(self, current_mode=self.rotation[index], rotation_index=index,
+                       live_resume_index=None)
+
+    def showing(self, plan: "ScreenPlan") -> "ArbiterState":
+        """The state once ``plan`` is on the panel.
+
+        An on-demand plan puts the session's index on the mode it shows (an
+        index past the end of a shortened list starts it again at 0).
+        """
+        state = replace(self, current_mode=plan.mode)
+        if plan.source is Source.ON_DEMAND and self.on_demand_modes:
+            state = replace(state, on_demand_index=_on_demand_index(self))
+        return state
+
+
+def _on_demand_index(state: ArbiterState) -> int:
+    """The session's index, or 0 once it is past the end of its list."""
+    index = state.on_demand_index
+    return index if index < len(state.on_demand_modes) else 0
+
+
+def on_demand_bound(min_duration: float, max_duration: float,
+                    deadline: Optional[float],
+                    now: float) -> Optional[Tuple[float, float]]:
+    """Shorten a screen's (min, max) seconds to what is left of a timed
+    on-demand session ending at ``deadline``. None when nothing is left.
+
+    The OnDemand Source's bound, applied after the screen's first frame,
+    where it always was (``now`` is read then).
+    """
+    if deadline is None:
+        return min_duration, max_duration
+    remaining = max(0.0, deadline - now)
+    min_duration = min(min_duration, remaining)
+    max_duration = min(max_duration, remaining)
+    if max_duration <= 0:
+        return None
+    return min_duration, max_duration
+
 
 @dataclass(frozen=True)
 class ArbiterInputs:
@@ -95,57 +251,122 @@ class ArbiterInputs:
             when it could win (the panel is on, and neither a follower nor
             on-demand outranks it), because reading it has side effects: a
             1 Hz throttle and deleting an expired file.
+        live_modes: The modes with live content, from a live-priority scan,
+            in registration order; None when no scan was made (on-demand,
+            Vegas keeping live content in its ticker, a throttled
+            mid-screen check). A scan asks every live-priority plugin, so it
+            is made only where run() always made it.
+        vegas_enabled: Vegas mode is on (and no on-demand session holds it
+            off).
+        vegas_live_in_ticker: Vegas keeps live content in its ticker
+            instead of yielding the panel to it.
+        vegas_yielded: This pass's Vegas iteration has run and yielded, so
+            the Vegas Source passes and the screen it fell through to is
+            decided.
+        reload_pending: Mid-screen only: a plugin reload is waiting for the
+            top of the loop, at a service point where that ends the screen.
     """
 
     schedule_on: bool
     on_demand_active: bool
     follower_active: bool
     wifi_notice: Optional[WifiNotice] = None
+    live_modes: Optional[Tuple[str, ...]] = None
+    vegas_enabled: bool = False
+    vegas_live_in_ticker: bool = False
+    vegas_yielded: bool = False
+    reload_pending: bool = False
 
 
 @dataclass(frozen=True)
 class ScreenPlan:
     """The Arbiter's answer for one pass.
 
+    decide() is pure, so it cannot ask a plugin anything: the fields a
+    plugin answers (its durations, whether it runs a dynamic cycle, how
+    often it draws) are filled in by the controller after the screen's first
+    frame, when they have always been read (DisplayController.complete_plan).
+
     Attributes:
         source: The Source that gets the panel.
+        mode: The display mode to draw (None for a blank, follower or notice).
+        plugin: The id of the plugin drawing ``mode``, once resolved.
+        min_duration: Seconds the screen runs at least (dynamic duration).
         max_duration: How long the plan holds the panel, in seconds, at most
             (its dwell ends early when what the panel should show changes).
-            None when the Source paces itself: a follower frame, or LEGACY.
+            None when the Source paces itself (a follower frame, Vegas), and
+            for a rotation or live plan until its first frame.
+        dynamic: Run until the plugin's cycle completes, between min and max.
+        frame_policy: Which frame loop the screen runs.
+        preemptible_by: The Sources that may end the screen mid-way.
         notice: The WiFi notice to draw, for a WIFI plan.
+        deadline: For an on-demand plan, when the session ends (wall
+            clock): after the first frame the screen's durations are cut to
+            what is left (:func:`on_demand_bound`).
+        ends_live: Nothing is live any more and live priority had
+            interrupted the rotation: taking this plan resumes the rotation
+            where it was (ArbiterState.release_live) before it shows.
     """
 
     source: Source
+    mode: Optional[str] = None
+    plugin: Optional[str] = None
+    min_duration: Optional[float] = None
     max_duration: Optional[float] = None
+    dynamic: bool = False
+    frame_policy: Optional[FramePolicy] = None
+    preemptible_by: FrozenSet[Source] = frozenset()
     notice: Optional[WifiNotice] = None
+    deadline: Optional[float] = None
+    ends_live: bool = False
 
 
 SCHEDULED_OFF_PLAN = ScreenPlan(Source.SCHEDULED_OFF, max_duration=SCHEDULED_OFF_DWELL)
 FOLLOWER_PLAN = ScreenPlan(Source.FOLLOWER)
-LEGACY_PLAN = ScreenPlan(Source.LEGACY)
+RELOAD_PLAN = ScreenPlan(Source.RELOAD)
+
+#: What may end a screen mid-way: the schedule, an on-demand session
+#: starting or ending, a WiFi notice, a live game, the rotation moving
+#: under the screen, and a plugin reload. Not a follower or Vegas: those
+#: are only looked at between screens.
+SCREEN_PREEMPTERS: FrozenSet[Source] = frozenset(
+    {Source.SCHEDULED_OFF, Source.ON_DEMAND, Source.WIFI, Source.LIVE, Source.ROTATION,
+     Source.RELOAD})
+
+#: A live screen is not preempted by Live: live games take turns between
+#: screens, never mid-screen.
+LIVE_PREEMPTERS: FrozenSet[Source] = SCREEN_PREEMPTERS - {Source.LIVE}
 
 
 class Arbiter:
     """Decides which Source gets the panel. Stateless; see the module docstring."""
 
     @staticmethod
-    def decide(state: ArbiterState, inputs: ArbiterInputs, now: float) -> ScreenPlan:
+    def decide(state: ArbiterState, inputs: ArbiterInputs, now: float,
+               running: Optional[ScreenPlan] = None) -> ScreenPlan:
         """The plan for this pass, from the Sources in priority order.
 
+        With ``running``, the question is the ScreenRunner's at one of its
+        service points instead: does a Source in ``running.preemptible_by``
+        now take the panel from that screen? The answer is ``running``
+        itself (the same object) while it holds, else the plan that ends it.
+        See :func:`_hold_or_preempt` for the rules.
+
         Args:
-            state: What the Arbiter remembers between passes (nothing yet).
+            state: What the Arbiter remembers between passes.
             inputs: This pass's snapshot.
-            now: Wall-clock time of the snapshot. No stage-2 Source reads it:
-                the top-of-pass WiFi check takes the notice as read, and only
-                the mid-screen check (:func:`wifi_notice_preempts`) compares
-                it with the expiry. It is in the signature for the Sources
-                stage 3 adds (on-demand expiry, durations).
+            now: Wall-clock time of the snapshot. The OnDemand Source reads
+                it for what is left of a timed session, and the mid-screen
+                WiFi rule (:func:`wifi_notice_preempts`) to compare with the
+                notice's expiry. The top-of-pass WiFi check does not: it
+                takes the notice as read.
+            running: The screen on the panel, for a mid-screen check.
 
         Returns:
-            The winning Source's plan, or LEGACY_PLAN when the winner is one
-            run() still decides itself.
+            The winning Source's plan (LEGACY for Vegas), or ``running``.
         """
-        del state, now  # not read by the stage-2 Sources; see the docstring
+        if running is not None:
+            return _hold_or_preempt(state, inputs, now, running)
 
         # ScheduledOff is a gate, not a Source: a scheduled-off panel stays
         # blank even for a follower, and only an on-demand session overrides
@@ -157,17 +378,161 @@ class Arbiter:
         if inputs.follower_active:
             return FOLLOWER_PLAN
 
-        # 2. OnDemand: decided by run() until stage 3. It outranks the notice.
+        # 2. OnDemand: the session's current mode. It outranks the notice.
         if inputs.on_demand_active:
-            return LEGACY_PLAN
+            return _on_demand_plan(state, now)
 
         # 3. Wifi: a pending notice, held for one short dwell per pass.
         if inputs.wifi_notice is not None:
             return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL,
                               notice=inputs.wifi_notice)
 
-        # 4-6. Live, Vegas, Rotation: still run()'s own code.
-        return LEGACY_PLAN
+        # 4. Live: the next live game, round-robin across several. With
+        # nothing live, a rotation that live priority interrupted resumes.
+        ends_live = False
+        if _live_applies(inputs):
+            pick = live_pick(inputs.live_modes, state.current_mode,
+                             advance=not state.live_takeover_unshown)
+            if pick is not None:
+                return ScreenPlan(Source.LIVE, mode=pick, preemptible_by=LIVE_PREEMPTERS)
+            ends_live = state.live_resume_index is not None and bool(state.rotation)
+
+        # 5. Vegas: one iteration of the ticker, run by run()'s own code
+        # until stage 4. Passes once this pass's iteration has yielded.
+        if inputs.vegas_enabled and not inputs.vegas_yielded:
+            return ScreenPlan(Source.LEGACY, ends_live=ends_live)
+
+        # 6. Rotation: the rotation's current mode (after the resume, when
+        # live priority just ended).
+        return rotation_plan(state.release_live() if ends_live else state,
+                             ends_live=ends_live)
+
+
+def _hold_or_preempt(state: ArbiterState, inputs: ArbiterInputs, now: float,
+                     running: ScreenPlan) -> ScreenPlan:
+    """The mid-screen rules: ``running``, or the plan that ends it.
+
+    What the frame loops used to check one by one (_check_live_takeover,
+    then _screen_preempted with _wifi_notice_pending in it, before stage 3),
+    in their order:
+
+    1. Live: a game went live while a non-live screen runs (the inputs
+       carry a scan only when one was due, at most once a second). Checked
+       first because it is the one preemption that changes the state -- the
+       rotation moves to the live mode and remembers where it was -- and it
+       still happens when a WiFi notice is also pending: the next pass then
+       shows the notice, and the game after it.
+    2. The panel's mode moved under the screen: an on-demand session
+       started, ended or changed mode, or the rotation was rebuilt (a
+       plugin enabled, disabled or reloaded).
+    3. The schedule turned the panel off.
+    4. A WiFi notice arrived (unless on-demand outranks it), compared with
+       its expiry because the read throttle can hand back a stale one.
+    5. A plugin reload is waiting at the top of the loop.
+
+    A follower and Vegas are never mid-screen preemptions; they are looked
+    at between screens.
+    """
+    by = running.preemptible_by
+    if Source.LIVE in by:
+        takeover = live_takeover(state, inputs)
+        if takeover is not None:
+            return ScreenPlan(Source.LIVE, mode=takeover, preemptible_by=LIVE_PREEMPTERS)
+    if state.current_mode != running.mode:
+        source = Source.ON_DEMAND if inputs.on_demand_active else Source.ROTATION
+        if source in by:
+            return ScreenPlan(source, mode=state.current_mode, preemptible_by=SCREEN_PREEMPTERS)
+    if (Source.SCHEDULED_OFF in by and not inputs.schedule_on
+            and not inputs.on_demand_active):
+        return SCHEDULED_OFF_PLAN
+    notice = inputs.wifi_notice
+    if (Source.WIFI in by and notice is not None
+            and wifi_notice_preempts(notice, inputs.on_demand_active, now)):
+        return ScreenPlan(Source.WIFI, max_duration=WIFI_NOTICE_DWELL, notice=notice)
+    if Source.RELOAD in by and inputs.reload_pending:
+        return RELOAD_PLAN
+    return running
+
+
+def live_takeover(state: ArbiterState, inputs: ArbiterInputs) -> Optional[str]:
+    """The live mode that takes the panel mid-screen, or None.
+
+    The first live mode, when a scan found one and the panel is not on a
+    live mode already. Never while on-demand holds the panel, while it is
+    scheduled off, or while Vegas keeps live content in its ticker.
+    """
+    if not _live_applies(inputs) or inputs.on_demand_active or not inputs.schedule_on:
+        return None
+    live = inputs.live_modes
+    if not live or state.current_mode in live:
+        return None
+    return live[0]
+
+
+def _on_demand_plan(state: ArbiterState, now: float) -> ScreenPlan:
+    """The OnDemand Source: the session's current mode.
+
+    ``max_duration`` is what is left of a timed session at ``now`` (None
+    without a duration); ``deadline`` carries the expiry so the bound can be
+    applied again after the first frame. A session with no modes left (its
+    plugin was unloaded under it) gets a plan with no mode: the controller
+    ends the session and shows the rotation's mode instead.
+    """
+    modes = state.on_demand_modes
+    if not modes:
+        return ScreenPlan(Source.ON_DEMAND)
+    expires_at = state.on_demand_expires_at
+    remaining = None if expires_at is None else max(0.0, expires_at - now)
+    return ScreenPlan(Source.ON_DEMAND, mode=modes[_on_demand_index(state)],
+                      max_duration=remaining, deadline=expires_at,
+                      preemptible_by=SCREEN_PREEMPTERS)
+
+
+def rotation_plan(state: ArbiterState, ends_live: bool = False) -> ScreenPlan:
+    """The Rotation Source: the mode the rotation is on.
+
+    That is ``state.current_mode``, which is ``rotation[rotation_index]``
+    except where something moved the panel off the list and the rotation
+    carries on from there: a live mode no rotation entry names, or None
+    when a session ended with no enabled mode to resume to.
+    """
+    return ScreenPlan(Source.ROTATION, mode=state.current_mode, ends_live=ends_live,
+                      preemptible_by=SCREEN_PREEMPTERS)
+
+
+def _live_applies(inputs: ArbiterInputs) -> bool:
+    """Whether the Live Source has a say: a scan was made, and Vegas is not
+    keeping live content in its ticker (where the live plugin takes extra
+    turns in the marquee instead of the panel)."""
+    if inputs.live_modes is None:
+        return False
+    return not (inputs.vegas_enabled and inputs.vegas_live_in_ticker)
+
+
+def live_pick(live_modes: Optional[Tuple[str, ...]], current_mode: Optional[str],
+              advance: bool) -> Optional[str]:
+    """The live mode to show, or None when nothing is live.
+
+    When several plugins are live at once this round-robins between them, so
+    the panel alternates each dwell instead of pinning to the first one
+    registered. The mode on the panel is the cursor, so this stays right as
+    games start and end.
+
+    Args:
+        live_modes: The live modes, in registration order.
+        current_mode: The mode on the panel.
+        advance: True for the rotation's pick (the live mode after the one
+            showing). False for a peek (the one showing if it is still live,
+            else the first), which Vegas uses to ask whether anything is.
+    """
+    if not live_modes:
+        return None
+    if current_mode in live_modes:
+        if advance:
+            index = live_modes.index(current_mode)
+            return live_modes[(index + 1) % len(live_modes)]
+        return current_mode
+    return live_modes[0]
 
 
 def wifi_notice_preempts(notice: Optional[WifiNotice], on_demand_active: bool,

src/display_manager.py

@@ -57,7 +57,8 @@ import freetype
 
 from src.common import snapshot_policy
 from src import display_watchdog
-from src.common.frame_timing import FrameTimingRecorder, install_gc_monitor
+from src.common.frame_timing import (
+    FrameTimingRecorder, install_gc_monitor, uninstall_gc_monitor)
 
 if TYPE_CHECKING:
     from src.common.render_gate import RenderGate
@@ -316,6 +317,11 @@ class DisplayManager:
         # is handed to the writer; this only once it has been saved, so an
         # mtime touch never vouches for a frame still waiting to be written.
         self._saved_snapshot_digest: Optional[int] = None
+        # A changed frame reached _write_snapshot_if_due() inside the write
+        # interval and was skipped. Nothing writes it unless update_display()
+        # runs again, and a screen that draws once and holds never calls it
+        # again -- see write_owed_snapshot().
+        self._snapshot_owed = False
         self._snapshot_dir_prepared = False
         # Background writer used mid-scroll; see _write_snapshot_if_due.
         self._snapshot_cond = threading.Condition()
@@ -1407,6 +1413,8 @@ class DisplayManager:
         # The stall watchdog would otherwise outlive this manager.
         if getattr(self, 'frame_timing', None) is not None:
             self.frame_timing.close()
+            # Installed with the recorder; stop timing collections with it.
+            uninstall_gc_monitor()
         # Reset the singleton state when cleaning up
         DisplayManager._instance = None
 
@@ -1785,9 +1793,10 @@ class DisplayManager:
 
             if frame_checksum is not None:
                 digest = frame_checksum
+                frame_changed = digest != self._last_snapshot_digest
                 action = snapshot_policy.decide(
                     now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
-                    viewer_fresh, digest != self._last_snapshot_digest)
+                    viewer_fresh, frame_changed)
             else:
                 # Ask as if the frame had changed before paying to find out.
                 # decide() is monotone in frame_changed -- a SKIP for a
@@ -1799,22 +1808,36 @@ class DisplayManager:
                     now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
                     viewer_fresh, True)
                 if action is snapshot_policy.SnapshotAction.SKIP:
+                    # Not hashed, so not known to be unchanged: owed until a
+                    # later look finds it written or unchanged.
+                    self._snapshot_owed = True
                     return
                 digest = zlib.adler32(self.image.tobytes())
-                if digest == self._last_snapshot_digest:
+                frame_changed = digest != self._last_snapshot_digest
+                if not frame_changed:
                     # Unchanged after all: the decision an unchanged frame gets.
                     action = snapshot_policy.decide(
                         now, self._last_snapshot_ts,
                         self._last_snapshot_touch_ts, viewer_fresh, False)
             if action is snapshot_policy.SnapshotAction.SKIP:
+                # A changed frame inside the write interval stays owed: the
+                # next update_display() would write it, but a static screen
+                # may not make one -- write_owed_snapshot() covers that.
+                self._snapshot_owed = frame_changed
                 return
             if (action is snapshot_policy.SnapshotAction.TOUCH
                     and self._saved_snapshot_digest == digest):
                 # mtime bump only: keeps the health check (snapshot age)
                 # green without paying for a PNG encode of an unchanged frame
+                # (this frame is already on disk, so nothing is owed).
+                self._snapshot_owed = False
                 os.utime(self._snapshot_path, None)
                 self._last_snapshot_touch_ts = now
                 return
+            # Owed until the write below succeeds: if it raises, the frame
+            # stays owed and write_owed_snapshot() retries it, rather than a
+            # held screen leaving the preview stale after one failed write.
+            self._snapshot_owed = True
             # (A TOUCH for a frame that isn't on disk yet -- still queued, or
             # its write failed -- is written instead: touching would make the
             # older file on disk look current.)
@@ -1839,9 +1862,39 @@ class DisplayManager:
             self._last_snapshot_ts = now
             self._last_snapshot_touch_ts = now
             self._last_snapshot_digest = digest
+            self._snapshot_owed = False
         except Exception as e:
             self._log_snapshot_failure(e)
 
+    def write_owed_snapshot(self) -> None:
+        """Write a frame the snapshot throttle skipped, once it is due.
+
+        The preview snapshot is only ever written from update_display(), and
+        at most once per write interval (snapshot_policy). A frame pushed
+        inside that interval is skipped, and is written by the next
+        update_display() that comes after it -- but a screen that draws its
+        card once and then holds it makes no further call. Its frame was on
+        the panel and never in the preview: soccer's recent/upcoming cards
+        skip redundant redraws, and the first one after an on-demand start
+        (pushed a few milliseconds after the controller's clear) left
+        /api/v3/display/current and the web preview black for the whole
+        screen while the panel showed the card.
+
+        The render loop calls this after each frame. Cheap when nothing is
+        owed (one attribute read); otherwise the usual policy decides, so
+        the write still waits out the interval and an unchanged frame is
+        never re-encoded.
+        """
+        if not self._snapshot_owed:
+            return
+        try:
+            if self._writes_suppressed():
+                return
+            with self._update_lock:
+                self._write_snapshot_if_due()
+        except Exception as e:  # pylint: disable=broad-except
+            self._log_snapshot_failure(e)
+
     def _log_snapshot_failure(self, error: Exception) -> None:
         # Snapshot failures must never break display — but they must not
         # be silent either: the snapshot's mtime is the web UI's display

src/display_watchdog.py

@@ -216,6 +216,23 @@ class RenderWatchdog:
     def armed(self) -> bool:
         return self._armed
 
+    def liveness(self) -> Dict[str, Any]:
+        """The heartbeat, in memory: what the control socket's state stream
+        reports as ``loop``.
+
+        ``heartbeat_age_seconds`` is the age of the render thread's last beat,
+        the beat that writes the heartbeat file, so it ages at the same rate
+        and is judged by the same ``HEARTBEAT_STALE_SECONDS``. None until the
+        loop has drawn its first frame. Any thread may call this: it only
+        reads two attributes.
+        """
+        last = self._last_beat
+        age = None
+        if self._armed and last is not None:
+            age = max(self._clock() - last, 0.0)
+        return {'heartbeat_age_seconds': age, 'armed': self._armed,
+                'stale_after': HEARTBEAT_STALE_SECONDS}
+
     def _on_render_thread(self) -> bool:
         return self._render_thread is not None and threading.get_ident() == self._render_thread
 

src/dynamic_team_resolver.py

@@ -23,6 +23,7 @@ import requests
 from typing import Any, Dict, List
 
 from src.common.api_helper import DEFAULT_HTTP_HEADERS
+from src.common.json_body import response_json
 
 logger = logging.getLogger(__name__)
 
@@ -157,7 +158,7 @@ class DynamicTeamResolver:
             response = requests.get(rankings_url, headers=dict(DEFAULT_HTTP_HEADERS),
                                     timeout=self.request_timeout)
             response.raise_for_status()
-            data = response.json()
+            data = response_json(response)
             
             rankings = {}
             rankings_data = data.get('rankings', [])

src/error_aggregator.py

@@ -485,15 +485,18 @@ def record_error(
 # and only the display service's ever records anything (plugin_executor runs
 # the plugins there). The web interface therefore reads a snapshot the display
 # service publishes to the shared cache directory -- the same channel, and the
-# same file permissions, as display_current_state and plugin_metrics:*: files
+# same file permissions, as display_current_state and plugin_metrics_snapshot: files
 # are 0660 and carry the cache directory's group, so root writes and the web
 # user reads, and the other way round for the clear request.
 #
 #   ERROR_SNAPSHOT_KEY       written by the display service only
-#   ERROR_CLEAR_REQUEST_KEY  written by the web interface only
+#   ERROR_CLEAR_REQUEST_KEY  written by the web interface only, as a fallback
 #
-# A clear is asynchronous: the web interface records a request, and the
-# display service applies it (clear_before) on its next tick and republishes.
+# A clear goes over the control socket (``errors.clear``): the display applies
+# it (clear_before) and republishes the snapshot before it answers. Only when
+# the socket cannot carry it (no socket, or a display older than the command)
+# does the web interface record a request in the mailbox, which the display
+# applies on its next tick; its tick reads that file only when it changed.
 # Until it has, the web interface hides whatever the snapshot shows from
 # before the cutoff, so a clear takes effect for readers immediately and a
 # snapshot published just before the request cannot bring old errors back.
@@ -598,13 +601,43 @@ class ErrorSnapshotPublisher:
         self._published_version: Optional[int] = None
         self._last_attempt: Optional[float] = None
         self._applied_clear_id: Optional[str] = None
+        # The widest cutoff applied in this process: a clear request at or
+        # before it has nothing left to clear (see _pending_cutoff).
+        self._applied_clear_cutoff: Optional[float] = None
+        from src.cache_manager import MailboxWatch  # the display's cache, loaded already
+        self._mailbox = MailboxWatch(ERROR_CLEAR_REQUEST_KEY)
         self._tick_lock = threading.Lock()
         self._stop = threading.Event()
         self._thread: Optional[threading.Thread] = None
 
+    def _clear(self, request_id: str, cutoff: float) -> int:
+        """Apply one clear and remember it. Caller holds _tick_lock."""
+        cleared = 0
+        if math.isfinite(cutoff):
+            cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
+            _snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
+                                  cleared, request_id)
+            if self._applied_clear_cutoff is None or cutoff > self._applied_clear_cutoff:
+                self._applied_clear_cutoff = cutoff
+        # A malformed request is acknowledged too, so it is not retried forever.
+        self._applied_clear_id = request_id
+        return cleared
+
     def _apply_clear_request(self) -> bool:
-        """Honour a clear request we have not applied yet. True if one was."""
-        request = self.cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
+        """Honour a mailbox clear request we have not applied yet. True if one was.
+
+        The mailbox is the fallback for a web interface that could not use
+        the control socket (``errors.clear``, :meth:`clear_now`). It is read
+        only when its file changed since the last tick; otherwise a tick
+        costs one stat().
+        """
+        if not self._mailbox.changed(self.cache_manager):
+            return False
+        try:
+            request = self.cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
+        except Exception:
+            self._mailbox.forget()
+            raise
         if not isinstance(request, dict):
             return False
         request_id = request.get("request_id")
@@ -614,14 +647,30 @@ class ErrorSnapshotPublisher:
             cutoff = float(request.get("cutoff"))
         except (TypeError, ValueError):
             cutoff = float("nan")
-        if math.isfinite(cutoff):
-            cleared = self.aggregator.clear_before(datetime.fromtimestamp(cutoff))
-            _snapshot_logger.info("Cleared %d plugin error record(s) as requested (%s)",
-                                  cleared, request_id)
-        # A malformed request is acknowledged too, so it is not retried forever.
-        self._applied_clear_id = request_id
+        self._clear(request_id, cutoff)
         return True
 
+    def clear_now(self, request_id: str, cutoff: float) -> int:
+        """``errors.clear`` over the control socket: apply a clear at once and
+        republish the snapshot, so the web interface's next read has it.
+        Returns how many records were cleared. Raises when the snapshot
+        could not be written, so the caller is not told it worked."""
+        with self._tick_lock:
+            cleared = self._clear(request_id, float(cutoff))
+            self._publish(self.aggregator.version, self._clock())
+            return cleared
+
+    def _publish(self, version: int, now: float) -> None:
+        """Write the snapshot. Caller holds _tick_lock."""
+        # Stamp the attempt before writing: a cache that keeps failing
+        # is retried at the throttled rate, not on every tick.
+        self._last_attempt = now
+        snapshot = self.aggregator.build_snapshot()
+        snapshot["applied_clear_id"] = self._applied_clear_id
+        snapshot["applied_clear_cutoff"] = self._applied_clear_cutoff
+        self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
+        self._published_version = version
+
     def tick(self) -> bool:
         """Apply a pending clear and publish if due. True if a snapshot was written."""
         with self._tick_lock:
@@ -635,13 +684,7 @@ class ErrorSnapshotPublisher:
                     if (self._last_attempt is not None
                             and now - self._last_attempt < self.min_interval):
                         return False
-                # Stamp the attempt before writing: a cache that keeps failing
-                # is retried at the throttled rate, not on every tick.
-                self._last_attempt = now
-                snapshot = self.aggregator.build_snapshot()
-                snapshot["applied_clear_id"] = self._applied_clear_id
-                self.cache_manager.set(ERROR_SNAPSHOT_KEY, snapshot)
-                self._published_version = version
+                self._publish(version, now)
                 return True
             except Exception as err:  # never let reporting break the display
                 _snapshot_logger.debug("Could not publish the plugin error snapshot: %s",
@@ -693,6 +736,22 @@ def start_error_snapshot_publisher(cache_manager: Any) -> Optional[ErrorSnapshot
         return None
 
 
+def apply_error_clear(request_id: str, args: Any) -> Dict[str, Any]:
+    """The display's handler for ``errors.clear`` on the control socket.
+
+    ``args`` is the contract's ErrorsClearArgs (``cutoff``, epoch seconds).
+    Runs on the socket's connection thread: the aggregator and the publisher
+    have their own locks, and nothing here touches rendering. Returns
+    ErrorsClearResult once the clear is applied and the snapshot rewritten.
+    """
+    publisher = _snapshot_publisher
+    if publisher is None:
+        raise RuntimeError("the error snapshot publisher is not running")
+    cutoff = float(args.cutoff)
+    cleared = publisher.clear_now(request_id, cutoff)
+    return {"request_id": request_id, "cutoff": cutoff, "cleared": cleared}
+
+
 # --- Reading side (web interface) -------------------------------------------
 
 def read_error_report(cache_manager: Any) -> Tuple[Optional[Dict[str, Any]], Optional[Dict[str, Any]]]:
@@ -731,7 +790,15 @@ def _pending_cutoff(snapshot: Optional[Dict[str, Any]],
         cutoff = float(clear_request.get("cutoff"))
     except (TypeError, ValueError):
         return None
-    return cutoff if math.isfinite(cutoff) else None
+    if not math.isfinite(cutoff):
+        return None
+    # A wider clear has been applied since (over the control socket): this
+    # older request has nothing left to hide.
+    applied = snapshot.get("applied_clear_cutoff") if snapshot is not None else None
+    if (isinstance(applied, (int, float)) and not isinstance(applied, bool)
+            and applied >= cutoff):
+        return None
+    return cutoff
 
 
 def _is_after(item: Any, field_name: str, cutoff: float) -> bool:
@@ -831,13 +898,31 @@ def _count_cleared(summary: Dict[str, Any], cutoff: float) -> Optional[int]:
     return None
 
 
-def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
+#: ``send(request_id, cutoff)`` hands a clear to the display over the control
+#: socket and returns its ErrorsClearResult, or None when the socket could
+#: not carry it and the mailbox should be written instead. Any exception it
+#: raises reaches the caller: the display had the request and failed it.
+ClearSender = Callable[[str, float], Optional[Dict[str, Any]]]
+
+
+def request_error_clear(cache_manager: Any, cutoff: float,
+                        send: Optional[ClearSender] = None) -> Dict[str, Any]:
     """Ask the display service to forget errors recorded at or before ``cutoff``.
 
-    Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``
-    (see _count_cleared) and ``clear_requested``. Raises OSError when the
-    request did not reach the shared cache, since a cache without a usable
-    directory accepts set() and keeps nothing.
+    Over the control socket when ``send`` is given and carries it: the
+    display applies the clear and republishes its snapshot before it
+    answers, so nothing is written here. Otherwise (no socket, or a display
+    older than ``errors.clear``) a request is written to the
+    ``plugin_error_clear_request`` mailbox, which the display applies on
+    its next tick, and readers hide the cleared errors until then.
+
+    Returns ``request_id``, ``cutoff`` (ISO, local time), ``cleared_count``,
+    ``clear_requested``, ``applied`` (the display has already cleared them)
+    and ``transport`` (``socket`` or ``mailbox``). ``cleared_count`` is the
+    display's own count over the socket, else an estimate from the snapshot
+    (see _count_cleared). Raises OSError when a mailbox request did not reach
+    the shared cache, since a cache without a usable directory accepts set()
+    and keeps nothing.
 
     A request the display has not applied yet is only ever widened: a later,
     narrower one ("older than 24 hours" after "everything") overwriting it
@@ -848,8 +933,21 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
     if pending is not None:
         cutoff = max(cutoff, pending)
     before = error_summary_from_report(snapshot, clear_request)
+    request_id = uuid.uuid4().hex
+    answer = {
+        "clear_requested": True,
+        "request_id": request_id,
+        "cutoff": datetime.fromtimestamp(cutoff).isoformat(),
+    }
+    if send is not None:
+        result = send(request_id, cutoff)
+        if result is not None:
+            count = result.get("cleared")
+            return dict(answer, applied=True, transport="socket",
+                        cleared_count=count if isinstance(count, int) and not isinstance(count, bool)
+                        else _count_cleared(before, cutoff))
     request = {
-        "request_id": uuid.uuid4().hex,
+        "request_id": request_id,
         "cutoff": cutoff,
         "requested_at": time.time(),
     }
@@ -857,9 +955,5 @@ def request_error_clear(cache_manager: Any, cutoff: float) -> Dict[str, Any]:
     stored = cache_manager.get(ERROR_CLEAR_REQUEST_KEY, max_age=None, memory_ttl=0)
     if not isinstance(stored, dict) or stored.get("request_id") != request["request_id"]:
         raise OSError("the clear request was not stored in the shared cache")
-    return {
-        "cleared_count": _count_cleared(before, cutoff),
-        "clear_requested": True,
-        "request_id": request["request_id"],
-        "cutoff": datetime.fromtimestamp(cutoff).isoformat(),
-    }
+    return dict(answer, applied=False, transport="mailbox",
+                cleared_count=_count_cleared(before, cutoff))

src/font_manager.py

@@ -222,6 +222,39 @@ class FontManager:
             logger.error(f"Error registering fonts for plugin {plugin_id}: {e}", exc_info=True)
             return False
 
+    def forget_plugin_fonts(self, plugin_id: str) -> bool:
+        """Drop the fonts ``plugin_id``'s manifest registered: its manifest
+        and catalog, its ``plugin_id::family`` entries in font_catalog, and
+        cached font objects for those families. Called by core when a plugin
+        is unloaded, so a reload registers from its current manifest and a
+        removed plugin's fonts stop resolving.
+
+        FontManager takes no locks; like forget_manager_fonts this relies on
+        single dict operations being atomic and iterates snapshots, so a
+        render thread calling get_font() meanwhile cannot break it. Returns
+        True if the plugin had registered fonts.
+        """
+        prefix = f"{plugin_id}::"
+        manifest = self.plugin_fonts.pop(plugin_id, None)
+        catalog = self.plugin_font_catalogs.pop(plugin_id, None)
+        # Every namespaced entry, not just the families in the catalog: one
+        # whose file failed to load never made it into the catalog, and a
+        # caller may have added one directly.
+        for family in list(self.font_catalog):
+            if family.startswith(prefix):
+                self.font_catalog.pop(family, None)
+        # get_font() keys the cache f"{family}_{size_px}".
+        dropped = [key for key in list(self.font_cache) if key.startswith(prefix)]
+        for key in dropped:
+            self.font_cache.pop(key, None)
+        if dropped:
+            # Font objects someone may hold were dropped; see cache_generation.
+            self.cache_generation += 1
+        if manifest is None and catalog is None:
+            return False
+        logger.info("Forgot fonts of plugin %s", plugin_id)
+        return True
+
     def _validate_font_manifest(self, font_manifest: Dict[str, Any]) -> bool:
         """Validate the structure of a plugin's font manifest."""
         required_fields = ["fonts"]

src/ipc/client.py

@@ -3,27 +3,37 @@
 Every failure -- no socket (the display is stopped, or predates the socket),
 a refused or timed-out connection, a reply that breaks the contract, or an
 error the display returned -- raises :class:`ControlError` with a short
-``reason``, and the caller falls back to the file mailbox. Nothing here
-blocks for longer than ``timeout`` in total.
+``reason``. Nothing here blocks for longer than ``timeout`` in total.
+
+Whether the caller may then write the file mailbox instead is
+:func:`should_fall_back`: only when the display never took the request (it
+could not be reached, or it is too old to know the command). A display that
+took the request and then failed, refused or went quiet is answered as
+that, not posted a second time through the mailbox.
 """
 
 from __future__ import annotations
 
 import socket
+import threading
 import time
 import uuid
-from typing import Any, Dict, List, Mapping, Optional, Sequence
+from typing import Any, Callable, Dict, List, Mapping, Optional, Sequence
 
 from src.ipc.contract import (
     AWAIT_SECONDS,
     MAX_MESSAGE_BYTES,
     PROTOCOL_VERSION,
+    SUBSCRIBE_KEEPALIVE_SECONDS,
     SUPPORTED_VERSIONS,
     Command,
+    ErrorCode,
     FrameReader,
     ProtocolError,
     Request,
     Response,
+    StateEvent,
+    StateEventKind,
     client_socket_paths,
     decode_message,
     encode_message,
@@ -45,17 +55,49 @@ class ControlError(Exception):
     ``refused``, ``timeout``, ``closed``, ``bad_response``, ``invalid_request``.
     When the display answered with an error, ``reason`` is that error's
     :class:`~src.ipc.contract.ErrorCode` (``busy``, ``unknown_command``, ...).
+
+    ``sent`` is True once the whole request was written to a connected
+    display, which may then have acted on it. A refusal the display sends
+    before it reads anything (``forbidden``, too many connections) carries
+    no request id and leaves ``sent`` False.
     """
 
-    def __init__(self, reason: str, message: str = ''):
+    def __init__(self, reason: str, message: str = '', *, sent: bool = False):
         super().__init__(reason, message)
         self.reason = reason
         self.message = message
+        self.sent = sent
 
     def __str__(self) -> str:
         return f'{self.reason}: {self.message}' if self.message else self.reason
 
 
+#: Answers from a display that read the request but does not speak it: one
+#: older than the command (an upgrade in progress) or the protocol version.
+#: It did nothing, so the mailbox is the way to reach it.
+UPGRADE_REASONS = frozenset({ErrorCode.UNKNOWN_COMMAND, ErrorCode.UNSUPPORTED_VERSION})
+
+
+def should_fall_back(error: BaseException) -> bool:
+    """May the caller write the file mailbox after ``error``?
+
+    Yes when the display never took the request: there is no socket (the
+    display is stopped, predates the socket, or it is switched off), the
+    connection was refused or timed out, the display turned the connection
+    away before reading it, or it is too old to know the command
+    (:data:`UPGRADE_REASONS`). Also for an error that is not a
+    :class:`ControlError` (a bug in the client), as before.
+
+    No once the display had the request: a ``busy`` queue, ``invalid_args``,
+    an ``internal`` error, or a timeout or hang-up after the request was
+    sent. The display may have applied it, or would refuse it from the
+    mailbox too, so a second copy there only hides the failure.
+    """
+    if not isinstance(error, ControlError):
+        return True
+    return not error.sent or error.reason in UPGRADE_REASONS
+
+
 def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
             request_id: Optional[str] = None,
             timeout: float = DEFAULT_TIMEOUT_SECONDS,
@@ -89,11 +131,14 @@ def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
     # A refusal before the request was read (forbidden, too many
     # connections) carries no id.
     if response.id != request_id and not (response.id is None and not response.ok):
-        raise ControlError('bad_response', 'the reply is for a different request')
+        raise ControlError('bad_response', 'the reply is for a different request', sent=True)
     if not response.ok:
         error = response.error
+        # No id: refused at the door (forbidden, too many connections),
+        # before the display read the request.
         raise ControlError(error.code if error else 'bad_response',
-                           error.message if error else '')
+                           error.message if error else '',
+                           sent=response.id is not None)
     return dict(response.result or {})
 
 
@@ -138,26 +183,31 @@ def _connect(paths: Sequence[str], deadline: float) -> socket.socket:
 
 
 def _exchange(sock: socket.socket, payload: bytes, deadline: float) -> Response:
+    """Send ``payload`` and read the reply. A failure once the whole request
+    is written raises with ``sent=True``: the display may have it."""
+    sent = False
     try:
         sock.settimeout(_remaining(deadline))
         sock.sendall(payload)
+        sent = True
         reader = FrameReader(MAX_MESSAGE_BYTES)
         while True:
             sock.settimeout(_remaining(deadline))
             data = sock.recv(4096)
             if not data:
-                raise ControlError('closed', 'the display closed the connection')
+                raise ControlError('closed', 'the display closed the connection', sent=sent)
             lines = reader.feed(data)
             if lines:
                 return Response.from_dict(decode_message(lines[0]))
     except socket.timeout:
-        raise ControlError('timeout', 'no reply in time') from None
+        raise ControlError('timeout', 'no reply in time', sent=sent) from None
     except ProtocolError as e:
-        raise ControlError('bad_response', e.message) from None
-    except ControlError:
+        raise ControlError('bad_response', e.message, sent=sent) from None
+    except ControlError as e:
+        e.sent = e.sent or sent
         raise
     except OSError as e:
-        raise ControlError('closed', str(e)) from None
+        raise ControlError('closed', str(e), sent=sent) from None
 
 
 # -- commands ---------------------------------------------------------------------------
@@ -223,6 +273,21 @@ def plugin_reload(plugin_id: str, *, timeout: Optional[float] = None,
                    else timeout, paths=paths)
 
 
+def errors_clear(request_id: str, cutoff: float, *,
+                 timeout: float = DEFAULT_TIMEOUT_SECONDS,
+                 paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    """Have the display forget the plugin errors recorded at or before
+    ``cutoff`` (epoch seconds) and publish its error snapshot again.
+
+    Returns :class:`~src.ipc.contract.ErrorsClearResult` once it is done.
+    Raises :class:`ControlError`: ``unknown_command`` from a display older
+    than the command, which still reads the ``plugin_error_clear_request``
+    mailbox.
+    """
+    return request(Command.ERRORS_CLEAR, {'cutoff': cutoff}, request_id=request_id,
+                   timeout=timeout, paths=paths)
+
+
 def ping(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
          paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
     return request(Command.PING, {}, timeout=timeout, paths=paths)
@@ -233,3 +298,260 @@ def hello(client: str = 'web', *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
     """Version negotiation: the result's ``version`` is the one both sides speak."""
     return request(Command.HELLO, {'versions': list(SUPPORTED_VERSIONS), 'client': client},
                    timeout=timeout, paths=paths)
+
+
+# -- the state stream (stage 3) ---------------------------------------------------------
+
+def state_get(since: Optional[int] = None, epoch: Optional[str] = None, *,
+              timeout: float = DEFAULT_TIMEOUT_SECONDS,
+              paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    """The display's state now, as a :class:`~src.ipc.contract.StateSnapshot`.
+
+    With ``since``/``epoch`` from an earlier answer, an unchanged state comes
+    back in the short ``changed: false`` form. Raises :class:`ControlError`
+    (``unknown_command`` from a display older than stage 3).
+    """
+    args: Dict[str, Any] = {}
+    if since is not None:
+        args['since'] = since
+    if epoch is not None:
+        args['epoch'] = epoch
+    return request(Command.STATE_GET, args, timeout=timeout, paths=paths)
+
+
+def snapshot_age(snapshot: Mapping[str, Any], now_mono: Optional[float] = None) -> float:
+    """Seconds since ``snapshot`` arrived: ``received_mono`` (set by
+    :meth:`StateSubscription.latest`) to now; 0 for a one-shot answer."""
+    received = snapshot.get('received_mono')
+    if isinstance(received, (int, float)) and not isinstance(received, bool):
+        now_mono = time.monotonic() if now_mono is None else now_mono
+        return max(now_mono - float(received), 0.0)
+    return 0.0
+
+
+def snapshot_loop_age(snapshot: Mapping[str, Any],
+                      now_mono: Optional[float] = None) -> Optional[float]:
+    """The render loop's heartbeat age now, from a state snapshot: the age the
+    display measured when it answered, plus the time since the answer
+    arrived. None when the display has no beat to report yet."""
+    loop = snapshot.get('loop')
+    if not isinstance(loop, dict):
+        state = snapshot.get('state')
+        loop = state.get('loop') if isinstance(state, dict) else None
+    age = loop.get('heartbeat_age_seconds') if isinstance(loop, dict) else None
+    if not isinstance(age, (int, float)) or isinstance(age, bool):
+        return None
+    return max(float(age), 0.0) + snapshot_age(snapshot, now_mono)
+
+
+def _merge_volatile(state: Dict[str, Any], volatile: Any) -> None:
+    """Fold a tick's ``volatile`` values (``{section: {key: value}}``) into
+    ``state``, copying each section it touches.
+
+    These are the timestamps the hub leaves out of its version --
+    ``display.last_updated``, ``on_demand.last_updated``/``remaining``,
+    ``plugins.published_at`` -- and the readers judge freshness by them, so
+    a copy that only full ``state`` events updated would go stale while the
+    same mode stayed on screen. Only keys the section already has are taken:
+    a tick never adds a section or a key the last snapshot did not carry
+    (a section left out of a truncated snapshot stays out).
+    """
+    if not isinstance(volatile, dict):
+        return  # a display from before ticks carried them
+    for name, values in volatile.items():
+        section = state.get(name)
+        if not isinstance(section, dict) or not isinstance(values, dict):
+            continue
+        fresh = {k: v for k, v in values.items() if k in section}
+        if fresh:
+            state[name] = dict(section, **fresh)
+
+
+#: A subscription that has heard nothing for this long is not trusted: the
+#: display sends a tick at least every SUBSCRIBE_KEEPALIVE_SECONDS.
+SUBSCRIPTION_SILENCE_SECONDS = 3 * SUBSCRIBE_KEEPALIVE_SECONDS
+
+#: Reconnect backoff: the first retry, and the cap. A display that does not
+#: know state.subscribe (stage 2 or older) is retried at the cap.
+_RECONNECT_MIN_SECONDS = 1.0
+_RECONNECT_MAX_SECONDS = 30.0
+
+#: Failures that another try soon will not fix.
+_SLOW_RETRY_REASONS = frozenset({'unknown_command', 'unsupported_version', 'disabled',
+                                 'unsupported'})
+
+
+class StateSubscription:
+    """One ``state.subscribe`` connection, held on a daemon thread.
+
+    Keeps the latest snapshot the display pushed, so a reader answers from
+    memory (:meth:`latest`). Reconnects with a backoff when the display goes
+    away. Never raises into the caller: :meth:`latest` is None whenever the
+    copy cannot be vouched for (not connected, or silent for longer than
+    ``silence``), and the caller falls back.
+    """
+
+    def __init__(self, paths: Optional[Sequence[str]] = None, *,
+                 silence: float = SUBSCRIPTION_SILENCE_SECONDS,
+                 connect_timeout: float = DEFAULT_TIMEOUT_SECONDS,
+                 clock: Callable[[], float] = time.monotonic):
+        self._paths = list(paths) if paths is not None else None
+        self._silence = silence
+        self._connect_timeout = connect_timeout
+        self._clock = clock
+        self._lock = threading.Lock()
+        self._snapshot: Optional[Dict[str, Any]] = None
+        self._received: Optional[float] = None
+        self._connected = False
+        self._stop = threading.Event()
+        self._sock: Optional[socket.socket] = None
+        self._thread: Optional[threading.Thread] = None
+        #: The reason the last connection ended (a ControlError reason).
+        self.last_error: Optional[str] = None
+        #: Full snapshots received: the subscribe answer and each state event.
+        self.snapshots = 0
+
+    # -- the reader's side ---------------------------------------------------
+
+    @property
+    def connected(self) -> bool:
+        return self._connected
+
+    def latest(self) -> Optional[Dict[str, Any]]:
+        """A copy of the latest snapshot, with ``received_mono`` (this
+        process's monotonic clock when it arrived); None when not trusted."""
+        with self._lock:
+            if not self._connected or self._snapshot is None or self._received is None:
+                return None
+            if self._clock() - self._received > self._silence:
+                return None
+            snap = dict(self._snapshot)
+            snap['received_mono'] = self._received
+            return snap
+
+    # -- lifecycle -----------------------------------------------------------
+
+    def start(self) -> 'StateSubscription':
+        if self._thread is None or not self._thread.is_alive():
+            self._stop.clear()
+            self._thread = threading.Thread(target=self._run, name='ledmatrix-state-feed',
+                                            daemon=True)
+            self._thread.start()
+        return self
+
+    def stop(self, timeout: float = 2.0) -> None:
+        self._stop.set()
+        sock = self._sock
+        if sock is not None:
+            try:
+                sock.shutdown(socket.SHUT_RDWR)
+            except OSError:
+                pass
+        thread = self._thread
+        if thread is not None and thread is not threading.current_thread():
+            thread.join(timeout)
+        self._thread = None
+
+    # -- the feed thread -----------------------------------------------------
+
+    def _run(self) -> None:
+        backoff = _RECONNECT_MIN_SECONDS
+        while not self._stop.is_set():
+            snapshots = self.snapshots
+            try:
+                self._follow()
+            except ControlError as e:
+                self.last_error = e.reason
+                if e.reason in _SLOW_RETRY_REASONS:
+                    backoff = _RECONNECT_MAX_SECONDS
+            except Exception as e:  # pylint: disable=broad-except
+                self.last_error = type(e).__name__
+            finally:
+                with self._lock:
+                    self._connected = False
+                sock, self._sock = self._sock, None
+                if sock is not None:
+                    try:
+                        sock.close()
+                    except OSError:
+                        pass
+            if self.snapshots != snapshots:
+                # This connection got as far as the display's state: whatever
+                # ended it (a restart, most often), it was working, so the
+                # next try starts from the shortest wait again.
+                backoff = _RECONNECT_MIN_SECONDS
+            if self._stop.wait(backoff):
+                return
+            backoff = min(backoff * 2, _RECONNECT_MAX_SECONDS)
+
+    def _follow(self) -> None:
+        """Subscribe, then read events until the connection ends. Raises ControlError."""
+        if not socket_supported():
+            raise ControlError('unsupported', 'no Unix sockets on this platform')
+        candidates = list(self._paths) if self._paths is not None else client_socket_paths()
+        if not candidates:
+            raise ControlError('disabled', 'the control socket is turned off')
+        request_id = str(uuid.uuid4())
+        payload = encode_message(Request(id=request_id, cmd=Command.STATE_SUBSCRIBE,
+                                         args={}).to_dict())
+        sock = _connect(candidates, time.monotonic() + self._connect_timeout)
+        self._sock = sock
+        try:
+            sock.settimeout(self._connect_timeout)
+            sock.sendall(payload)
+            # A read waits for the next event; the display sends one at least
+            # every keepalive, so this much silence means it is gone.
+            sock.settimeout(self._silence)
+            reader = FrameReader(MAX_MESSAGE_BYTES)
+            first = True
+            while not self._stop.is_set():
+                data = sock.recv(65536)
+                if not data:
+                    raise ControlError('closed', 'the display closed the connection')
+                for line in reader.feed(data):
+                    obj = decode_message(line)
+                    if first:
+                        response = Response.from_dict(obj)
+                        if not response.ok:
+                            error = response.error
+                            raise ControlError(error.code if error else 'bad_response',
+                                               error.message if error else '')
+                        self._store(dict(response.result or {}), full=True)
+                        first = False
+                        continue
+                    event = StateEvent.from_dict(obj)
+                    self._store(event.result, full=event.event == StateEventKind.STATE)
+        except socket.timeout:
+            raise ControlError('timeout', 'the display went quiet') from None
+        except ProtocolError as e:
+            raise ControlError('bad_response', e.message) from None
+        except OSError as e:
+            if self._stop.is_set():
+                return
+            raise ControlError('closed', str(e)) from None
+
+    def _store(self, result: Dict[str, Any], full: bool) -> None:
+        now = self._clock()
+        with self._lock:
+            if full and isinstance(result.get('state'), dict):
+                self._snapshot = result
+                self.snapshots += 1
+            elif (self._snapshot is not None
+                  and result.get('epoch') == self._snapshot.get('epoch')):
+                # A tick: nothing changed but the render loop's liveness and
+                # the volatile keys (timestamps) the writers keep refreshing.
+                snap = dict(self._snapshot)
+                state = dict(snap.get('state') or {})
+                if result.get('version') == snap.get('version'):
+                    _merge_volatile(state, result.get('volatile'))
+                loop = result.get('loop')
+                if isinstance(loop, dict):
+                    state['loop'] = loop
+                    snap['loop'] = loop
+                snap['state'] = state
+                snap['served_at'] = result.get('served_at', snap.get('served_at'))
+                self._snapshot = snap
+            else:
+                return  # a tick before any state, or from another epoch
+            self._received = now
+            self._connected = True

src/ipc/contract.py

@@ -30,6 +30,11 @@ later the state stream). A few commands (:data:`AWAITED_COMMANDS`) are
 answered only once the render thread has applied them, or with ``pending``
 when it has not within :data:`AWAIT_SECONDS`.
 
+``state.subscribe`` is the one exception to "one response per request": its
+response is followed, on the same connection, by :class:`StateEvent` lines
+the display pushes until either side hangs up. Events carry ``event``
+instead of ``ok``.
+
 New commands are added within a protocol version: a display that does not
 know one answers ``unknown_command``, the client falls back, and ``hello``
 lists the commands a display knows. The version changes only when the
@@ -142,11 +147,15 @@ class Command:
     ON_DEMAND_STATUS = 'on_demand.status'
     BRIGHTNESS_SET = 'brightness.set'
     PLUGIN_RELOAD = 'plugin.reload'
+    STATE_GET = 'state.get'
+    STATE_SUBSCRIBE = 'state.subscribe'
+    ERRORS_CLEAR = 'errors.clear'
 
 
 #: Every command version 1 defines, in the order ``hello`` reports them.
-#: ``brightness.set`` and ``plugin.reload`` came in stage 2, within version 1
-#: (see the module docstring on adding commands).
+#: ``brightness.set`` and ``plugin.reload`` came in stage 2, ``state.get``
+#: and ``state.subscribe`` in stage 3, and ``errors.clear`` in stage 4, all
+#: within version 1 (see the module docstring on adding commands).
 COMMANDS: Tuple[str, ...] = (
     Command.HELLO,
     Command.PING,
@@ -155,8 +164,17 @@ COMMANDS: Tuple[str, ...] = (
     Command.ON_DEMAND_STATUS,
     Command.BRIGHTNESS_SET,
     Command.PLUGIN_RELOAD,
+    Command.STATE_GET,
+    Command.STATE_SUBSCRIBE,
+    Command.ERRORS_CLEAR,
 )
 
+#: Commands the connection thread answers itself, through a handler the
+#: display registers (``ControlServer(handlers=...)``), because they touch
+#: nothing the render thread owns. A display that registered none answers
+#: ``unknown_command``, and the client falls back as from an older display.
+DIRECT_COMMANDS = frozenset({Command.ERRORS_CLEAR})
+
 #: Commands that are queued for the render thread.
 QUEUED_COMMANDS = frozenset({Command.ON_DEMAND_START, Command.ON_DEMAND_STOP,
                              Command.BRIGHTNESS_SET, Command.PLUGIN_RELOAD})
@@ -173,6 +191,25 @@ AWAIT_SECONDS: Dict[str, float] = {
 }
 AWAITED_COMMANDS = frozenset(AWAIT_SECONDS)
 
+#: The state stream (stage 3). ``state.subscribe`` turns its connection into
+#: a one-way stream of :class:`StateEvent` lines. Subscribers have their own
+#: bound, separate from the short request connections, so they can never
+#: take the slots a command needs.
+MAX_SUBSCRIBERS = 4
+
+#: A subscriber hears from the display at least this often: a ``state``
+#: event when something changed, else a ``tick`` carrying the render loop's
+#: liveness and the latest volatile timestamps. A client that has heard nothing for a few of these treats its
+#: copy as unknown.
+SUBSCRIBE_KEEPALIVE_SECONDS = 5.0
+
+#: The shape of the ``state`` object in a state snapshot. Bumped only when a
+#: field changes meaning; new fields are added within a schema.
+STATE_SCHEMA = 1
+
+#: The sections of a state snapshot, in the order they are documented.
+STATE_SECTIONS: Tuple[str, ...] = ('display', 'on_demand', 'brightness', 'plugins', 'loop')
+
 #: Brightness, in percent, as the display's hardware setting takes it.
 MIN_BRIGHTNESS = 0
 MAX_BRIGHTNESS = 100
@@ -477,8 +514,91 @@ class PluginReloadArgs:
         return cls(plugin_id=plugin_id)
 
 
+def _optional_version(args: Mapping[str, Any], key: str) -> Optional[int]:
+    value = args.get(key)
+    if value is None:
+        return None
+    if not _is_int(value) or value < 0:
+        raise ProtocolError(ErrorCode.INVALID_ARGS, f'{key} must be a non-negative integer')
+    return value
+
+
+def _optional_epoch(args: Mapping[str, Any]) -> Optional[str]:
+    value = args.get('epoch')
+    if value is None or value == '':
+        return None
+    if not _valid_id(value):
+        raise ProtocolError(ErrorCode.INVALID_ARGS,
+                            f'epoch must be a printable string of 1-{MAX_ID_LENGTH} characters')
+    return str(value)
+
+
+@dataclass(frozen=True)
+class StateGetArgs:
+    """``state.get``: the display's state, as a versioned snapshot.
+
+    With ``since`` and the ``epoch`` it came from, the answer is only
+    ``{changed: false, version, epoch, served_at, loop, volatile}`` while the
+    state is still at that version, so a poller that already has it is sent
+    no state -- only the latest values of the keys that do not count as a
+    change (``volatile``, see :class:`StateSnapshot`).
+    """
+    since: Optional[int] = None
+    epoch: Optional[str] = None
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {'since': self.since, 'epoch': self.epoch}
+
+    @classmethod
+    def from_dict(cls, args: Mapping[str, Any]) -> 'StateGetArgs':
+        return cls(since=_optional_version(args, 'since'), epoch=_optional_epoch(args))
+
+
+@dataclass(frozen=True)
+class StateSubscribeArgs:
+    """``state.subscribe``: the snapshot now, then a push stream of changes.
+
+    The response is the snapshot ``state.get`` returns. After it the
+    connection carries only :class:`StateEvent` lines from the display: a
+    ``state`` event whenever the state changes (always the latest version,
+    so a reader that falls behind skips versions instead of queueing them),
+    and a ``tick`` at least every :data:`SUBSCRIBE_KEEPALIVE_SECONDS`.
+    """
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {}
+
+    @classmethod
+    def from_dict(cls, args: Mapping[str, Any]) -> 'StateSubscribeArgs':
+        return cls()
+
+
+@dataclass(frozen=True)
+class ErrorsClearArgs:
+    """``errors.clear``: forget the plugin errors recorded at or before
+    ``cutoff`` (seconds since the epoch), as ``POST /api/v3/errors/clear``
+    asks. The request id is the clear's id, which the display's error
+    snapshot then reports as ``applied_clear_id``.
+    """
+    cutoff: float
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {'cutoff': self.cutoff}
+
+    @classmethod
+    def from_dict(cls, args: Mapping[str, Any]) -> 'ErrorsClearArgs':
+        value = args.get('cutoff')
+        if isinstance(value, bool) or not isinstance(value, (int, float)):
+            raise ProtocolError(ErrorCode.INVALID_ARGS, 'cutoff must be a number of seconds')
+        if not math.isfinite(value) or value < 0:
+            raise ProtocolError(ErrorCode.INVALID_ARGS,
+                                'cutoff must be a finite, non-negative number of seconds')
+        return cls(cutoff=float(value))
+
+
 CommandArgs = Union[HelloArgs, OnDemandStartArgs, OnDemandStopArgs, NoArgs,
-                    BrightnessSetArgs, PluginReloadArgs]
+                    BrightnessSetArgs, PluginReloadArgs, StateGetArgs, StateSubscribeArgs,
+                    ErrorsClearArgs]
 
 #: The arguments of a command that goes on the render thread's queue.
 QueuedArgs = Union[OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs]
@@ -491,6 +611,9 @@ _ARG_TYPES: Dict[str, Any] = {
     Command.ON_DEMAND_STATUS: NoArgs,
     Command.BRIGHTNESS_SET: BrightnessSetArgs,
     Command.PLUGIN_RELOAD: PluginReloadArgs,
+    Command.STATE_GET: StateGetArgs,
+    Command.STATE_SUBSCRIBE: StateSubscribeArgs,
+    Command.ERRORS_CLEAR: ErrorsClearArgs,
 }
 
 
@@ -563,6 +686,112 @@ class PluginReloadResult(TypedDict):
     modes: List[str]
 
 
+class ErrorsClearResult(TypedDict):
+    """``errors.clear``, once applied and the error snapshot republished."""
+    request_id: str
+    cutoff: float
+    cleared: int
+
+
+class LoopState(TypedDict):
+    """``loop``: is the render loop still going round?
+
+    ``heartbeat_age_seconds`` is the age of the render thread's last beat,
+    measured in memory by the display when it answered -- the same beat that
+    writes ``display-heartbeat.json``. None until the loop has drawn its first
+    frame. At ``stale_after`` or more the loop is stalled: the threshold
+    ``/api/v3/health`` uses.
+    """
+    heartbeat_age_seconds: Optional[float]
+    armed: bool
+    stale_after: float
+
+
+class StateSnapshot(TypedDict, total=False):
+    """The answer to ``state.get`` and ``state.subscribe``, and the
+    ``result`` of a ``state`` event.
+
+    ``version`` counts changes to the state within one ``epoch`` (one run of
+    the display process): a reader that sees a new epoch starts over.
+    ``changed`` is False only for a ``state.get`` whose ``since`` is still
+    current, and then ``state`` is absent and ``volatile`` is there instead:
+    ``{section: {key: value}}``, the current values of the keys the version
+    ignores (``display.last_updated``, ``on_demand.last_updated`` and
+    ``remaining``, ``plugins.published_at``). A reader merges them into the
+    copy it has; they are how it can tell the writers are still publishing.
+    ``served_at`` is the display's wall clock when it answered. ``loop`` is
+    measured at that moment, so it is also inside ``state``.
+
+    ``state`` holds the sections in :data:`STATE_SECTIONS`:
+
+    * ``display``: what ``display_current_state`` holds (mode, plugin_id,
+      mode_index, total_modes, on_demand_active, is_display_active,
+      last_updated);
+    * ``on_demand``: what ``display_on_demand_state`` holds;
+    * ``brightness``: ``{brightness, panel_brightness, dimmed}``;
+    * ``plugins``: the plugin runtime snapshot (``plugin_runtime_snapshot``),
+      or None when there is none (or it was too large to send);
+    * ``loop``: :class:`LoopState`.
+
+    A section the display has not published yet is None.
+    """
+    schema: int
+    version: int
+    epoch: str
+    pid: int
+    served_at: float
+    changed: bool
+    state: Dict[str, Any]
+    volatile: Dict[str, Dict[str, Any]]
+    loop: LoopState
+
+
+class StateEventKind:
+    STATE = 'state'   # result: a full StateSnapshot, the latest version
+    TICK = 'tick'     # result: {version, epoch, pid, served_at, loop, volatile}; nothing changed
+
+
+@dataclass(frozen=True)
+class StateEvent:
+    """One message the display pushes to a subscriber.
+
+    ``{"v": 1, "id": "<the subscribe request's id>", "event": "state" | "tick",
+    "result": {...}}``. It has no ``ok``, which is how a reader tells it from
+    a response.
+    """
+    id: str
+    event: str
+    result: Dict[str, Any]
+    v: int = PROTOCOL_VERSION
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {'v': self.v, 'id': self.id, 'event': self.event, 'result': dict(self.result)}
+
+    @classmethod
+    def from_dict(cls, obj: Any) -> 'StateEvent':
+        """Validate an event. Raises :class:`ProtocolError` (BAD_REQUEST)."""
+        if not isinstance(obj, dict):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'an event must be a JSON object')
+        version = obj.get('v')
+        if not _is_int(version):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'v must be an integer')
+        raw_id = obj.get('id')
+        if not isinstance(raw_id, str):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'id must be a string')
+        event = obj.get('event')
+        if event not in (StateEventKind.STATE, StateEventKind.TICK):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'event must be "state" or "tick"')
+        result = obj.get('result')
+        if not isinstance(result, dict):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'result must be a JSON object')
+        return cls(id=raw_id, event=event, result=result, v=version)
+
+
+def is_event(obj: Any) -> bool:
+    """Whether a decoded message is a pushed event rather than a response."""
+    return isinstance(obj, dict) and 'event' in obj and 'ok' not in obj
+
+
 def negotiate_version(client_versions: Tuple[int, ...]) -> Optional[int]:
     """The highest version both sides speak, or None."""
     common = set(client_versions) & set(SUPPORTED_VERSIONS)

src/ipc/server.py

@@ -6,7 +6,9 @@ rendering: a command that changes the panel is validated, put on a bounded
 queue and acknowledged, and the render thread drains that queue at the point
 where it reads the file mailbox (``DisplayController._poll_on_demand_requests``),
 handing each command to the same code. Queries (``on_demand.status``) are
-answered from a snapshot callable the display provides.
+answered from a snapshot callable the display provides, and the few commands
+that touch nothing the render thread owns (``errors.clear``) by a handler the
+display registers, on the connection thread.
 
 The queue also wakes the render thread: :meth:`ControlServer.wait_for_command`
 is what it waits on in place of a sleep, so a command lands within a frame on
@@ -14,6 +16,13 @@ every kind of screen. An awaited command (``brightness.set``,
 ``plugin.reload``) carries a :class:`CommandOutcome` that the render thread
 fills in; its connection thread waits for that, bounded, before answering.
 
+The state stream (stage 3): the display publishes what it is doing into a
+:class:`StateHub`, in memory, and ``state.get`` / ``state.subscribe`` read
+it. A subscriber's connection gives back its request slot, takes one of
+:data:`~src.ipc.contract.MAX_SUBSCRIBERS`, and is pushed the latest version
+on every change plus a keepalive tick, from its own thread: publishing never
+waits for a reader, and a reader that stops reading is dropped.
+
 Robustness rules, because this runs inside the display process:
 
 * every connection has its own daemon thread, at most :data:`MAX_CLIENTS` at
@@ -37,6 +46,7 @@ server checks them again: root, its own user, or a member of that group.
 
 from __future__ import annotations
 
+import json
 import logging
 import os
 import queue
@@ -45,18 +55,24 @@ import stat
 import struct
 import threading
 import time
+import uuid
 from dataclasses import dataclass, field
-from typing import Any, Callable, Dict, FrozenSet, List, Mapping, Optional
+from typing import Any, Callable, Dict, FrozenSet, Iterable, List, Mapping, Optional, Tuple
 
 from src.ipc.contract import (
     AWAIT_SECONDS,
     AWAITED_COMMANDS,
     COMMANDS,
     DEFAULT_SOCKET_DIR,
+    DIRECT_COMMANDS,
     DEFAULT_SOCKET_PATH,
     MAX_MESSAGE_BYTES,
+    MAX_SUBSCRIBERS,
     PROTOCOL_VERSION,
     QUEUED_COMMANDS,
+    STATE_SCHEMA,
+    STATE_SECTIONS,
+    SUBSCRIBE_KEEPALIVE_SECONDS,
     SUPPORTED_VERSIONS,
     AckResult,
     BrightnessSetArgs,
@@ -72,6 +88,9 @@ from src.ipc.contract import (
     QueuedArgs,
     Request,
     Response,
+    StateEvent,
+    StateEventKind,
+    StateGetArgs,
     configured_socket_path,
     decode_message,
     dev_socket_path,
@@ -91,7 +110,8 @@ MAX_CLIENTS = 8
 
 #: Commands waiting for the render thread. It drains them at least every
 #: 0.25 s, so a full queue means the render thread is stuck, and the client
-#: is told ``busy`` (and falls back to the mailbox) instead of piling up work.
+#: is told ``busy`` instead of piling up work. The mailbox would not be read
+#: either, so the web interface reports the failure rather than fall back.
 QUEUE_SIZE = 16
 
 #: Timeout for one recv()/send() on a connection.
@@ -181,6 +201,241 @@ class QueuedCommand:
             self.outcome.fail(code, message)
 
 
+# -- the state stream (stage 3) ----------------------------------------------------------
+
+#: How long a ``state.get`` keeps the display counting its readers as served
+#: over the socket (:meth:`StateHub.readers_active`). A subscriber counts for
+#: as long as it is connected.
+READER_WINDOW_SECONDS = 60.0
+
+#: Room kept for the envelope (``v``, ``id``, ``event``) around a snapshot,
+#: within MAX_MESSAGE_BYTES.
+_ENVELOPE_ROOM = 512
+
+_MISSING = object()
+
+LoopProbe = Callable[[], Mapping[str, Any]]
+
+
+def _fingerprint(value: Optional[Mapping[str, Any]], volatile: Iterable[str]) -> Any:
+    """What a section's version is judged on: the value minus its volatile keys
+    (timestamps that move on every publish without anything changing)."""
+    if value is None:
+        return None
+    skip = frozenset(volatile)
+    return {k: v for k, v in value.items() if k not in skip} if skip else dict(value)
+
+
+def _volatile_values(sections: Mapping[str, Optional[Dict[str, Any]]],
+                     volatile: Mapping[str, FrozenSet[str]]) -> Dict[str, Dict[str, Any]]:
+    """``{section: {key: value}}``: the volatile keys each published section
+    has now. The sections are never mutated after publish (a publish swaps
+    in a new dict), so reading them outside the lock is safe."""
+    values: Dict[str, Dict[str, Any]] = {}
+    for name, keys in volatile.items():
+        value = sections.get(name)
+        if not keys or not isinstance(value, dict):
+            continue
+        present = {k: value[k] for k in keys if k in value}
+        if present:
+            values[name] = present
+    return values
+
+
+def _unknown_loop() -> Dict[str, Any]:
+    return {'heartbeat_age_seconds': None, 'armed': False, 'stale_after': None}
+
+
+def fit_snapshot(snapshot: Dict[str, Any]) -> Dict[str, Any]:
+    """``snapshot``, or a copy without the plugin runtime section when the
+    message would be over MAX_MESSAGE_BYTES (hundreds of plugins). The
+    reader then falls back to the cache for that section only; ``truncated``
+    says which was left out."""
+    state = snapshot.get('state')
+    if not isinstance(state, dict) or state.get('plugins') is None:
+        return snapshot
+    try:
+        size = len(json.dumps(snapshot, separators=(',', ':'), ensure_ascii=True,
+                              allow_nan=False))
+    except (TypeError, ValueError):
+        size = MAX_MESSAGE_BYTES
+    if size <= MAX_MESSAGE_BYTES - _ENVELOPE_ROOM:
+        return snapshot
+    logger.warning("State snapshot is %d bytes; sending it without the plugin runtime "
+                   "section", size)
+    trimmed = dict(snapshot)
+    trimmed['state'] = dict(state, plugins=None)
+    trimmed['truncated'] = ['plugins']
+    return trimmed
+
+
+class StateHub:
+    """The display's live state, in memory, for ``state.get`` and ``state.subscribe``.
+
+    Writers publish whole sections (:meth:`publish`): the render thread
+    publishes ``display``, ``on_demand`` and ``brightness``, and the plugin
+    runtime publisher's thread publishes ``plugins``. Each section has one
+    writer. ``loop`` is not published: it is measured when a reader asks
+    (``loop_probe``), so it keeps ageing while the render thread is stuck.
+
+    The version goes up when a section's value changes, ignoring the keys
+    the publisher names as volatile (timestamps). Those keys still carry
+    news -- ``display.last_updated`` is the render thread's proof of life --
+    so the short ``changed: false`` answer, which is what a subscriber's
+    tick carries, has their current values in ``volatile``; a reader merges
+    them into its copy. Publishing never blocks on
+    a reader: the lock is held only to swap a dict reference and compare it,
+    and every socket write happens on the reader's own thread, outside it.
+    A reader that is slow gets the latest version when it next asks, not
+    every version in between.
+    """
+
+    def __init__(self, loop_probe: Optional[LoopProbe] = None, *,
+                 clock: Callable[[], float] = time.monotonic,
+                 wall_clock: Callable[[], float] = time.time,
+                 epoch: Optional[str] = None, pid: Optional[int] = None,
+                 reader_window: float = READER_WINDOW_SECONDS):
+        self._cond = threading.Condition(threading.Lock())
+        self._sections: Dict[str, Optional[Dict[str, Any]]] = {}
+        self._fingerprints: Dict[str, Any] = {}
+        self._volatile: Dict[str, FrozenSet[str]] = {}
+        self._version = 0
+        self.epoch = epoch or uuid.uuid4().hex[:16]
+        self.pid = os.getpid() if pid is None else pid
+        self._loop_probe = loop_probe
+        self._clock = clock
+        self._wall_clock = wall_clock
+        self._reader_window = reader_window
+        self._last_read: Optional[float] = None
+        self._subscribers = 0
+
+    @property
+    def version(self) -> int:
+        return self._version
+
+    @property
+    def subscribers(self) -> int:
+        return self._subscribers
+
+    # -- writers -------------------------------------------------------------
+
+    def publish(self, section: str, value: Optional[Mapping[str, Any]],
+                volatile: Iterable[str] = ()) -> bool:
+        """Store a section's latest value; True when that is a new version.
+
+        The value is copied (one level), so the caller may reuse its dict.
+        """
+        stored = None if value is None else dict(value)
+        skip = frozenset(volatile)
+        fingerprint = _fingerprint(stored, skip)
+        with self._cond:
+            self._sections[section] = stored
+            self._volatile[section] = skip
+            if self._fingerprints.get(section, _MISSING) == fingerprint:
+                return False
+            self._fingerprints[section] = fingerprint
+            self._version += 1
+            self._cond.notify_all()
+        return True
+
+    def wake(self) -> None:
+        """Wake every waiting reader (the server is closing)."""
+        with self._cond:
+            self._cond.notify_all()
+
+    # -- readers -------------------------------------------------------------
+
+    def loop(self) -> Dict[str, Any]:
+        """The render loop's liveness now. Never raises."""
+        if self._loop_probe is None:
+            return _unknown_loop()
+        try:
+            return dict(self._loop_probe())
+        except Exception:  # pylint: disable=broad-except
+            logger.debug("Render loop liveness probe failed", exc_info=True)
+            return _unknown_loop()
+
+    def snapshot(self, since: Optional[int] = None,
+                 epoch: Optional[str] = None) -> Dict[str, Any]:
+        """The :class:`~src.ipc.contract.StateSnapshot` now.
+
+        ``since`` with this hub's ``epoch``, still the current version, gives
+        the short ``changed: false`` form, with ``volatile``: each section's
+        volatile keys at their latest values (the rest of the section is
+        what the reader already has).
+        """
+        with self._cond:
+            version = self._version
+            sections = dict(self._sections)
+            volatile = dict(self._volatile)
+        loop = self.loop()
+        result: Dict[str, Any] = {
+            'schema': STATE_SCHEMA,
+            'version': version,
+            'epoch': self.epoch,
+            'pid': self.pid,
+            'served_at': self._wall_clock(),
+            'loop': loop,
+        }
+        if since is not None and epoch == self.epoch and since == version:
+            result['changed'] = False
+            result['volatile'] = _volatile_values(sections, volatile)
+            return result
+        state: Dict[str, Any] = {name: sections.get(name) for name in STATE_SECTIONS
+                                 if name != 'loop'}
+        state['loop'] = loop
+        result['changed'] = True
+        result['state'] = state
+        return result
+
+    def wait_for_change(self, version: int, timeout: float,
+                        stop: Optional[threading.Event] = None) -> bool:
+        """Block up to ``timeout`` for a version other than ``version``."""
+        with self._cond:
+            self._cond.wait_for(
+                lambda: self._version != version or (stop is not None and stop.is_set()),
+                timeout)
+            return self._version != version
+
+    # -- who is reading ------------------------------------------------------
+
+    def note_read(self) -> None:
+        self._last_read = self._clock()
+
+    def subscriber_joined(self) -> None:
+        with self._cond:
+            self._subscribers += 1
+
+    def subscriber_left(self) -> None:
+        with self._cond:
+            self._subscribers = max(0, self._subscribers - 1)
+            self._last_read = self._clock()
+
+    def readers_active(self) -> bool:
+        """Is the socket serving state readers? A subscriber is connected, or a
+        ``state.get`` came within the reader window. The display uses this to
+        write the cache copies of the same state less often."""
+        if self._subscribers > 0:
+            return True
+        last = self._last_read
+        return last is not None and self._clock() - last < self._reader_window
+
+
+class _Slot:
+    """A connection slot, released once (a subscriber gives its back early)."""
+
+    def __init__(self, semaphore: threading.BoundedSemaphore):
+        self._semaphore = semaphore
+        self._held = True
+        self._lock = threading.Lock()
+
+    def release(self) -> None:
+        with self._lock:
+            if self._held:
+                self._held = False
+                self._semaphore.release()
+
+
 # -- peer credentials ------------------------------------------------------------------
 
 @dataclass(frozen=True)
@@ -301,6 +556,12 @@ def server_socket_path(environ: Optional[Mapping[str, str]] = None) -> Optional[
 
 StatusProvider = Callable[[], Dict[str, Any]]
 
+#: A handler for one of DIRECT_COMMANDS, ``(request_id, args) -> result``. It
+#: runs on the connection thread, so it must not touch what the render thread
+#: owns. It may raise ProtocolError to answer with that error's code; any
+#: other exception is answered ``internal``.
+DirectHandler = Callable[[str, Any], Mapping[str, Any]]
+
 
 class ControlServer:
     """Serves the control socket on background threads.
@@ -315,8 +576,17 @@ class ControlServer:
                  message_timeout: float = MESSAGE_TIMEOUT_SECONDS,
                  idle_timeout: float = IDLE_TIMEOUT_SECONDS,
                  check_peer: bool = True,
-                 await_seconds: Optional[Mapping[str, float]] = None):
+                 await_seconds: Optional[Mapping[str, float]] = None,
+                 state_hub: Optional[StateHub] = None,
+                 max_subscribers: int = MAX_SUBSCRIBERS,
+                 keepalive: float = SUBSCRIBE_KEEPALIVE_SECONDS,
+                 handlers: Optional[Mapping[str, DirectHandler]] = None):
         self.path = path
+        self.state_hub = state_hub
+        self._handlers: Dict[str, DirectHandler] = {
+            cmd: fn for cmd, fn in (handlers or {}).items() if cmd in DIRECT_COMMANDS}
+        self._subscriber_slots = threading.BoundedSemaphore(max_subscribers)
+        self._keepalive = keepalive
         self._await_seconds: Dict[str, float] = dict(AWAIT_SECONDS)
         if await_seconds:
             self._await_seconds.update(await_seconds)
@@ -378,6 +648,8 @@ class ControlServer:
         """Stop accepting and remove the socket file (only if it is still ours)."""
         self._stopping.set()
         self._close_socket()
+        if self.state_hub is not None:
+            self.state_hub.wake()   # subscribers see _stopping and hang up
         thread = self._thread
         if thread is not None and thread is not threading.current_thread():
             thread.join(timeout=2.0)
@@ -500,8 +772,22 @@ class ControlServer:
         """
         return self._pending.wait(timeout)
 
+    def wake(self) -> None:
+        """Wake the render thread as a queued command would, with nothing queued.
+
+        For work that reaches the display another way in the same process (a
+        plugin's on-demand request, ``DisplayController.submit_plugin_on_demand``):
+        the render thread returns from :meth:`wait_for_command` and drains,
+        and reads the caller's own queue there. Safe from any thread.
+        """
+        self._pending.set()
+
     def drain(self) -> List[QueuedCommand]:
-        """Every queued command, oldest first. Called from the render thread."""
+        """Every queued command, oldest first. Called from the render thread.
+
+        Clears the wake flag first, so anything queued (or woken for) while
+        this runs wakes the next wait again.
+        """
         commands: List[QueuedCommand] = []
         self._pending.clear()
         while True:
@@ -549,6 +835,7 @@ class ControlServer:
 
     def _serve(self, conn: socket.socket) -> None:
         """One connection: authenticate, then answer requests until it ends."""
+        slot = _Slot(self._slots)
         try:
             conn.settimeout(self._io_timeout)
             peer = peer_credentials(conn)
@@ -557,7 +844,7 @@ class ControlServer:
                                "this user or group %s", peer.pid, peer.uid, peer.gid, self._group)
                 self._send(conn, Response.failure(None, ErrorCode.FORBIDDEN, 'not permitted'))
                 return
-            self._read_requests(conn, peer)
+            self._read_requests(conn, peer, slot)
         except Exception:  # pylint: disable=broad-except
             logger.exception("Control socket connection failed")
         finally:
@@ -565,7 +852,7 @@ class ControlServer:
                 conn.close()
             except OSError:
                 pass
-            self._slots.release()
+            slot.release()
 
     def _peer_ok(self, peer: PeerCredentials) -> bool:
         groups = None
@@ -573,7 +860,8 @@ class ControlServer:
             groups = process_groups(peer.pid)
         return peer_allowed(peer, self._own_uid, self._group, groups)
 
-    def _read_requests(self, conn: socket.socket, peer: Optional[PeerCredentials]) -> None:
+    def _read_requests(self, conn: socket.socket, peer: Optional[PeerCredentials],
+                       slot: Optional[_Slot] = None) -> None:
         reader = FrameReader(MAX_MESSAGE_BYTES)
         idle_since = time.monotonic()
         message_started: Optional[float] = None
@@ -598,7 +886,13 @@ class ControlServer:
                 self._send(conn, Response.failure(None, e.code, e.message))
                 return  # can't find the next message boundary: hang up
             for line in lines:
-                if not self._send(conn, self.handle_line(line, peer)):
+                response, cmd = self._handle(line, peer)
+                if cmd == Command.STATE_SUBSCRIBE and response.ok:
+                    # The connection becomes a one-way stream; anything the
+                    # client sent after the subscribe is ignored.
+                    self._subscribe(conn, response, slot)
+                    return
+                if not self._send(conn, response):
                     return
             if reader.pending:
                 if message_started is None or lines:
@@ -624,20 +918,91 @@ class ControlServer:
     # -- requests --------------------------------------------------------------------
 
     def handle_line(self, line: bytes, peer: Optional[PeerCredentials] = None) -> Response:
-        """Answer one request line. Never raises."""
+        """Answer one request line. Never raises.
+
+        A ``state.subscribe`` answered here gets its snapshot only; the
+        stream that follows needs a connection (``_read_requests``).
+        """
+        return self._handle(line, peer)[0]
+
+    def _handle(self, line: bytes,
+                peer: Optional[PeerCredentials]) -> Tuple[Response, Optional[str]]:
+        """The response to one line, and the command it answered (when known)."""
         request_id: Optional[str] = None
+        cmd: Optional[str] = None
         try:
             obj = decode_message(line)
             raw_id = obj.get('id')
             request_id = raw_id if isinstance(raw_id, str) and len(raw_id) <= 128 else None
             request = Request.from_dict(obj)
             request_id = request.id
-            return self._dispatch(request, peer)
+            cmd = request.cmd
+            return self._dispatch(request, peer), cmd
         except ProtocolError as e:
-            return Response.failure(e.request_id or request_id, e.code, e.message)
+            return Response.failure(e.request_id or request_id, e.code, e.message), cmd
         except Exception:  # pylint: disable=broad-except
             logger.exception("Control socket handler failed")
-            return Response.failure(request_id, ErrorCode.INTERNAL, 'internal error')
+            return Response.failure(request_id, ErrorCode.INTERNAL, 'internal error'), cmd
+
+    # -- the state stream ----------------------------------------------------------
+
+    def _subscribe(self, conn: socket.socket, response: Response,
+                   slot: Optional[_Slot]) -> None:
+        """Answer a ``state.subscribe`` and push state events until it ends.
+
+        Subscribers have their own bound (MAX_SUBSCRIBERS) and give their
+        request slot back, so a few browsers watching never use up the slots
+        commands need. Everything here runs on this connection's thread: a
+        reader that does not keep up only stalls its own sends, and one that
+        stops reading for a whole IO timeout is dropped. The render thread
+        only ever publishes into the hub.
+        """
+        hub = self.state_hub
+        if hub is None or not self._subscriber_slots.acquire(blocking=False):
+            self._send(conn, Response.failure(response.id, ErrorCode.BUSY,
+                                              'too many state subscribers', v=response.v))
+            return
+        if slot is not None:
+            slot.release()
+        hub.subscriber_joined()
+        try:
+            if not self._send(conn, response):
+                return
+            result = response.result or {}
+            version = result.get('version', -1)
+            sub_id = response.id or ''
+            logger.debug("Control socket: state subscriber joined at version %s", version)
+            while not self._stopping.is_set():
+                hub.wait_for_change(version, self._keepalive, self._stopping)
+                if self._stopping.is_set():
+                    return
+                snap = hub.snapshot(since=version, epoch=hub.epoch)
+                if snap.get('changed'):
+                    version = snap['version']
+                    event = StateEvent(sub_id, StateEventKind.STATE, fit_snapshot(snap),
+                                       v=response.v)
+                else:
+                    event = StateEvent(sub_id, StateEventKind.TICK, snap, v=response.v)
+                if not self._send_event(conn, event):
+                    return
+        finally:
+            hub.subscriber_left()
+            self._subscriber_slots.release()
+
+    def _send_event(self, conn: socket.socket, event: StateEvent) -> bool:
+        try:
+            data = encode_message(event.to_dict())
+        except ProtocolError as e:
+            logger.error("Control socket state event not sent: %s", e.message)
+            return False
+        try:
+            conn.sendall(data)
+            return True
+        except socket.timeout:
+            logger.info("Control socket: dropping a state subscriber that stopped reading")
+            return False
+        except OSError:
+            return False
 
     def _dispatch(self, request: Request, peer: Optional[PeerCredentials]) -> Response:
         if request.cmd == Command.HELLO:
@@ -678,6 +1043,21 @@ class ControlServer:
                                         v=request.v)
             return Response.success(request.id, self._status_provider(), v=request.v)
 
+        if request.cmd in (Command.STATE_GET, Command.STATE_SUBSCRIBE):
+            hub = self.state_hub
+            if hub is None:
+                return Response.failure(request.id, ErrorCode.INTERNAL, 'no state available',
+                                        v=request.v)
+            if isinstance(args, StateGetArgs):
+                hub.note_read()
+                snap = hub.snapshot(since=args.since, epoch=args.epoch)
+            else:
+                snap = hub.snapshot()
+            return Response.success(request.id, fit_snapshot(snap), v=request.v)
+
+        if request.cmd in DIRECT_COMMANDS:
+            return self._direct(request, args)
+
         if request.cmd in QUEUED_COMMANDS and isinstance(args, (
                 OnDemandStartArgs, OnDemandStopArgs, BrightnessSetArgs, PluginReloadArgs)):
             awaited = request.cmd in AWAITED_COMMANDS
@@ -705,6 +1085,25 @@ class ControlServer:
         return Response.failure(request.id, ErrorCode.INTERNAL,
                                 f'{request.cmd} is not implemented', v=request.v)
 
+    def _direct(self, request: Request, args: Any) -> Response:
+        """A command the display answers on this thread (DIRECT_COMMANDS)."""
+        handler = self._handlers.get(request.cmd)
+        if handler is None:
+            # Answered as an older display would, so the client falls back.
+            return Response.failure(request.id, ErrorCode.UNKNOWN_COMMAND,
+                                    f'{request.cmd} is not served by this display',
+                                    v=request.v)
+        try:
+            result = handler(request.id, args)
+        except ProtocolError as e:
+            return Response.failure(request.id, e.code, e.message, v=request.v)
+        except Exception:  # pylint: disable=broad-except
+            logger.exception("Control socket: %s %s failed", request.cmd, request.id)
+            return Response.failure(request.id, ErrorCode.INTERNAL,
+                                    'the display failed to apply it', v=request.v)
+        logger.info("Control socket applied %s %s", request.cmd, request.id)
+        return Response.success(request.id, dict(result), v=request.v)
+
     def _await_outcome(self, request: Request, outcome: CommandOutcome) -> Response:
         """Answer an awaited command once the render thread has applied it.
 
@@ -728,22 +1127,27 @@ class ControlServer:
 
 def start_control_server(status_provider: Optional[StatusProvider] = None,
                          cache_dir: Optional[str] = None,
-                         environ: Optional[Mapping[str, str]] = None) -> Optional[ControlServer]:
+                         environ: Optional[Mapping[str, str]] = None,
+                         state_hub: Optional[StateHub] = None,
+                         handlers: Optional[Mapping[str, DirectHandler]] = None,
+                         ) -> Optional[ControlServer]:
     """Start the display's control socket, or return None when it can't run.
 
     None covers Windows, ``LEDMATRIX_CONTROL_SOCKET=off`` and any failure to
-    bind; in every case the web interface falls back to the file mailbox.
+    bind; in every case the web interface falls back to the file mailbox
+    and to the cache keys the display still writes.
     """
     path = server_socket_path(environ)
     if path is None:
         logger.debug("Control socket disabled or unsupported here; using the file mailbox only")
         return None
-    server = ControlServer(path, status_provider, resolve_socket_group(cache_dir))
+    server = ControlServer(path, status_provider, resolve_socket_group(cache_dir),
+                           state_hub=state_hub, handlers=handlers)
     return server if server.start() else None
 
 
 __all__ = [
-    'CommandOutcome', 'ControlServer', 'PeerCredentials', 'QueuedCommand', 'StatusProvider',
-    'peer_allowed', 'peer_credentials', 'process_groups', 'resolve_socket_group',
-    'server_socket_path', 'start_control_server', 'PROTOCOL_VERSION',
+    'CommandOutcome', 'ControlServer', 'PeerCredentials', 'QueuedCommand', 'StateHub',
+    'StatusProvider', 'fit_snapshot', 'peer_allowed', 'peer_credentials', 'process_groups',
+    'resolve_socket_group', 'server_socket_path', 'start_control_server', 'PROTOCOL_VERSION',
 ]

src/logo_downloader.py

@@ -21,6 +21,7 @@ from PIL.PngImagePlugin import PngInfo
 from requests.adapters import HTTPAdapter
 from urllib3.util.retry import Retry
 from src.common.api_helper import DEFAULT_HTTP_HEADERS
+from src.common.json_body import response_json
 from src.common.logo_helper import MAX_LOGO_BYTES
 from src.common.permission_utils import (
     ensure_directory_permissions,
@@ -481,7 +482,7 @@ class LogoDownloader:
             logger.info(f"Fetching team data for {league} from ESPN API...")
             response = self.session.get(api_url, params={'limit':1000},headers=self.headers, timeout=self.request_timeout)
             response.raise_for_status()
-            data: Dict = response.json()
+            data: Dict = response_json(response)
             
             logger.info(f"Successfully fetched team data for {league}")
             return data
@@ -505,7 +506,7 @@ class LogoDownloader:
             logger.info(f"Fetching team data for team {team_id} in {league} from ESPN API...")
             response = self.session.get(f"{api_url}/{team_id}", headers=self.headers, timeout=self.request_timeout)
             response.raise_for_status()
-            data: Dict = response.json()
+            data: Dict = response_json(response)
             
             logger.info(f"Successfully fetched team data for {team_id} in {league}")
             return data

src/malloc_tuning.py

@@ -0,0 +1,123 @@
+"""Keep glibc's malloc from holding on to memory the display has freed.
+
+The display process allocates and frees PIL images and numpy buffers all day
+from a dozen threads. glibc gives each allocating thread its own malloc arena
+(up to 8 x CPU count) and returns little of what is freed inside them to the
+OS, so resident memory climbs for hours while the live data stays flat. Two
+in-process remedies, both standard library only (ctypes) and both no-ops off
+Linux/glibc:
+
+* :func:`cap_arenas` -- ``mallopt(M_ARENA_MAX, 2)``, the in-process twin of the
+  unit's ``Environment=MALLOC_ARENA_MAX=2``. Units installed before that line
+  existed never got it (systemd runs the copy in /etc/systemd/system), so the
+  process applies it itself. Call it before any other thread starts: arenas
+  already created stay. A ``MALLOC_ARENA_MAX`` set in the environment wins.
+* :class:`MallocTrimmer` -- ``malloc_trim(0)`` at most every few minutes,
+  called from the render loop between screens, where no frame is being drawn.
+  glibc 2.8+ releases free pages from the middle of every arena, not only the
+  top of the main heap.
+
+Without glibc (macOS, Windows, musl, the dev server on any of them) nothing is
+loaded and every call returns False.
+"""
+import ctypes
+import logging
+import os
+import sys
+import time
+from typing import Any, Callable, Optional
+
+logger = logging.getLogger(__name__)
+
+#: glibc's mallopt() parameter number for the arena cap (malloc.h).
+M_ARENA_MAX = -8
+
+#: The arena cap applied when the environment does not set one; the same value
+#: as the unit's ``MALLOC_ARENA_MAX``.
+DEFAULT_ARENA_MAX = 2
+
+#: Seconds between malloc_trim() calls. A trim takes about 1-20 ms on a Pi 4,
+#: so this keeps it far from frame timing while still returning memory long
+#: before it piles up.
+TRIM_INTERVAL_SECONDS = 300.0
+
+_UNLOADED = object()
+_libc: Any = _UNLOADED
+
+
+def _load_libc() -> Optional[Any]:
+    """The process's C library if it is glibc with malloc_trim, else None."""
+    global _libc
+    if _libc is _UNLOADED:
+        _libc = None
+        if sys.platform.startswith('linux'):
+            try:
+                libc = ctypes.CDLL(None)
+                # gnu_get_libc_version is glibc-only, so musl (which has
+                # mallopt but no malloc_trim) is left alone as a whole.
+                if all(hasattr(libc, name) for name in
+                       ('gnu_get_libc_version', 'malloc_trim', 'mallopt')):
+                    libc.malloc_trim.argtypes = [ctypes.c_size_t]
+                    libc.malloc_trim.restype = ctypes.c_int
+                    libc.mallopt.argtypes = [ctypes.c_int, ctypes.c_int]
+                    libc.mallopt.restype = ctypes.c_int
+                    _libc = libc
+            except (OSError, AttributeError, TypeError):
+                logger.debug("glibc malloc controls unavailable", exc_info=True)
+    return _libc
+
+
+def cap_arenas(max_arenas: int = DEFAULT_ARENA_MAX) -> bool:
+    """Cap glibc's malloc arenas at ``max_arenas``. True when the cap was set.
+
+    Skipped when ``MALLOC_ARENA_MAX`` is in the environment: glibc has read it
+    already, and an operator who set it chose that value.
+    """
+    if os.environ.get('MALLOC_ARENA_MAX'):
+        return False
+    libc = _load_libc()
+    if libc is None:
+        return False
+    try:
+        return bool(libc.mallopt(M_ARENA_MAX, int(max_arenas)))
+    except Exception:  # pylint: disable=broad-except
+        logger.debug("mallopt(M_ARENA_MAX) failed", exc_info=True)
+        return False
+
+
+class MallocTrimmer:
+    """Calls ``malloc_trim(0)`` at most once per ``interval`` seconds.
+
+    :meth:`maybe_trim` is meant for an idle point of the render loop; it costs
+    one clock read when no trim is due. The first trim comes one interval
+    after construction, so start-up's allocations have settled.
+    """
+
+    def __init__(self, interval: float = TRIM_INTERVAL_SECONDS,
+                 clock: Callable[[], float] = time.monotonic) -> None:
+        self._interval = interval
+        self._clock = clock
+        self._libc = _load_libc()
+        self._next = clock() + interval
+
+    @property
+    def available(self) -> bool:
+        return self._libc is not None
+
+    def maybe_trim(self) -> bool:
+        """Trim if one is due. True when malloc_trim ran and released memory."""
+        if self._libc is None:
+            return False
+        now = self._clock()
+        if now < self._next:
+            return False
+        self._next = now + self._interval
+        try:
+            released = bool(self._libc.malloc_trim(0))
+        except Exception:  # pylint: disable=broad-except
+            logger.debug("malloc_trim failed; not trying again", exc_info=True)
+            self._libc = None
+            return False
+        logger.debug("malloc_trim(0) took %.1f ms, released=%s",
+                     (self._clock() - now) * 1000.0, released)
+        return released

src/plugin_system/__init__.py

@@ -3,15 +3,46 @@ LEDMatrix Plugin System
 
 This module provides the core plugin infrastructure for the LEDMatrix project.
 It enables dynamic loading, management, and discovery of display plugins.
+
+BasePlugin and PluginManager are imported on first use (PEP 562), not when
+the package is imported: the web interface imports several submodules
+(store_manager, schema_manager, ...) and never needs PluginManager, which
+pulls in the loader, executor and the shared helpers behind them.
+``from src.plugin_system import BasePlugin`` works as before and returns the
+same class.
 """
 
+import importlib
+from typing import TYPE_CHECKING, Any, Dict, List, Tuple
+
 __version__ = "1.0.0"
 
-from .base_plugin import BasePlugin
-from .plugin_manager import PluginManager
+if TYPE_CHECKING:
+    from .base_plugin import BasePlugin
+    from .plugin_manager import PluginManager
+
+#: Exported name -> (module it lives in, attribute name there).
+_LAZY: Dict[str, Tuple[str, str]] = {
+    'BasePlugin': ('src.plugin_system.base_plugin', 'BasePlugin'),
+    'PluginManager': ('src.plugin_system.plugin_manager', 'PluginManager'),
+}
 
 __all__ = [
     'BasePlugin',
     'PluginManager',
 ]
 
+
+def __getattr__(name: str) -> Any:
+    """Import an exported name on first access (PEP 562); see src.common."""
+    try:
+        module_name, attr = _LAZY[name]
+    except KeyError:
+        raise AttributeError(f"module {__name__!r} has no attribute {name!r}") from None
+    value = getattr(importlib.import_module(module_name), attr)  # nosemgrep: python.lang.security.audit.non-literal-import.non-literal-import -- module_name comes from the fixed _LAZY table
+    globals()[name] = value
+    return value
+
+
+def __dir__() -> List[str]:
+    return sorted(set(globals()) | set(__all__))

src/plugin_system/base_plugin.py

@@ -11,6 +11,7 @@ Stability: Stable - maintains backward compatibility
 from abc import ABC, abstractmethod
 from enum import Enum
 from typing import Dict, Any, Optional, List
+import math
 import os
 import sys
 from src.deprecation import deprecated, warn_deprecated
@@ -240,6 +241,26 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
     return legacy_vegas_participation(plugin)
 
 
+def finite_seconds(value: Any) -> Optional[float]:
+    """``value`` as seconds when it is a finite number or a numeric string,
+    else None. A bool is not a number here, though it is an int: True would
+    read as a one-second screen.
+
+    How the core reads a plugin's get_display_duration() -- the rotation
+    (DisplayController._get_display_duration) and the Vegas static pause --
+    which several plugins answer straight from config.json, so a value saved
+    as "20" or null arrives as a string or None. A number at or below zero is
+    returned as it is; each caller has its own rule for that.
+    """
+    if isinstance(value, bool):
+        return None
+    try:
+        seconds = float(value)
+    except (TypeError, ValueError, OverflowError):
+        return None
+    return seconds if math.isfinite(seconds) else None
+
+
 class BasePlugin(ABC):
     """
     Base class that all plugins must inherit from.
@@ -1070,6 +1091,65 @@ class BasePlugin(ABC):
         if callable(notify):
             notify(self.plugin_id)
 
+    def request_on_demand(self, mode: Optional[str] = None,
+                          duration: Optional[float] = None,
+                          pinned: bool = False) -> Optional[str]:
+        """
+        Take the screen now: show this plugin on demand. Safe from any thread.
+
+        For a plugin that reacts to something outside the rotation -- an MQTT
+        message, a timer, a detection -- and wants the panel for it. The
+        request goes straight to the display in this process and is applied
+        on its render thread within a frame or so, exactly like an on-demand
+        start from the web interface.
+
+        Args:
+            mode: One of this plugin's display modes; None for its first.
+            duration: Seconds to show it before the rotation resumes; None
+                (or zero) for no limit, until end_on_demand() or the user
+                stops it.
+            pinned: Stay on ``mode`` instead of cycling through the
+                plugin's other modes.
+
+        Returns:
+            The request id once the display has queued it, or None when
+            there is no display in this process to ask (the web interface,
+            scripts/check_plugin.py) or its queue is full. A plugin that
+            also runs on cores without this method writes the
+            ``display_on_demand_request`` mailbox on None, as before; see
+            "On-demand display" in docs/PLUGIN_API_REFERENCE.md.
+
+        Example::
+
+            if not (hasattr(self, 'request_on_demand')
+                    and self.request_on_demand(mode='my_alert', duration=15)):
+                self._write_on_demand_mailbox(...)   # older cores
+        """
+        request = getattr(getattr(self, 'plugin_manager', None), 'request_on_demand', None)
+        if not callable(request):
+            return None
+        request_id = request(self.plugin_id, mode=mode, duration=duration, pinned=pinned)
+        # Only a real id counts: a test's MagicMock manager answers a mock,
+        # which must read as "not taken" so the plugin's fallback runs.
+        return request_id if isinstance(request_id, str) else None
+
+    def end_on_demand(self) -> Optional[str]:
+        """
+        Give the screen back: end this plugin's on-demand session. Any thread.
+
+        Ends only a session this plugin owns. One the user started for
+        another plugin, or a session that already ended, is left alone. The
+        rotation resumes where it left off.
+
+        Returns:
+            The request id once queued, or None as request_on_demand() does.
+        """
+        end = getattr(getattr(self, 'plugin_manager', None), 'end_on_demand', None)
+        if not callable(end):
+            return None
+        request_id = end(self.plugin_id)
+        return request_id if isinstance(request_id, str) else None
+
     def get_vegas_participation(self) -> str:
         """
         How this plugin takes part in Vegas mode: ``'scroll'``, ``'pause'`` or

src/plugin_system/operation_types.py

@@ -48,12 +48,19 @@ class PluginOperation:
     completed_at: Optional[datetime] = None
     
     def to_dict(self) -> Dict[str, Any]:
-        """Convert operation to dictionary for serialization."""
+        """Convert operation to dictionary for serialization.
+
+        Parameters whose name starts with ``_`` are internal and left out:
+        PluginOperationQueue keeps the operation's callback there as
+        ``_callback`` until its worker runs it, and a pending operation's
+        status answered 500 because that function cannot be serialized.
+        """
         return {
             'operation_id': self.operation_id,
             'operation_type': self.operation_type.value,
             'plugin_id': self.plugin_id,
-            'parameters': self.parameters,
+            'parameters': {key: value for key, value in self.parameters.items()
+                           if not str(key).startswith('_')},
             'status': self.status.value,
             'progress': self.progress,
             'message': self.message,

src/plugin_system/plugin_catalog.py

@@ -15,7 +15,8 @@ reads through a catalog unchanged. It has nothing that runs a plugin: no
 ``load_plugin``, ``get_plugin`` or ``plugins``.
 
 Runtime state -- whether the display has a plugin loaded, its health, its
-errors -- is not here either. The display process publishes what it knows to
+errors -- is not here either, with one exception: given a ``runtime_source``,
+the mode lookups prefer the modes the running display registered. The display process publishes what it knows to
 the shared cache (health and resource metrics, the current mode, the error
 aggregator snapshot), and the web routes read those publications. What the
 display does not publish (which plugins it has loaded, its plugin state
@@ -26,8 +27,9 @@ See docs/ARCHITECTURE.md ("Web and display processes").
 
 import json
 import threading
+import time
 from pathlib import Path
-from typing import Any, Dict, List, Optional, Union, cast
+from typing import Any, Callable, Dict, List, Optional, Union, cast
 
 from src.common.permission_utils import (
     ensure_directory_permissions, get_plugin_dir_mode,
@@ -39,6 +41,10 @@ from src.plugin_system.plugin_dirs import (
 
 PathLike = Union[str, Path]
 
+#: How long one read of the display's runtime view answers mode lookups. A
+#: listing asks once per plugin; the cache copy is a file read each time.
+_RUNTIME_VIEW_TTL_SECONDS = 1.0
+
 
 class PluginCatalog:
     """Manifests, schemas, config and versions of the installed plugins.
@@ -49,10 +55,17 @@ class PluginCatalog:
     """
 
     def __init__(self, plugins_dir: PathLike, config_manager: Optional[Any] = None,
-                 schema_manager: Optional[Any] = None) -> None:
+                 schema_manager: Optional[Any] = None,
+                 runtime_source: Optional[Callable[[], Any]] = None) -> None:
         self.plugins_dir: Path = Path(plugins_dir)
         self.config_manager = config_manager
         self.schema_manager = schema_manager
+        # Returns the display's PluginRuntimeView
+        # (src/plugin_system/plugin_runtime.py). Its live view carries the
+        # modes the display registered, which the mode lookups below prefer
+        # to the manifest's. None: manifests only.
+        self.runtime_source = runtime_source
+        self._runtime_view_memo: Optional[tuple] = None
         self.logger = get_logger(__name__)
 
         # Guards plugin_manifests/plugin_directories: request threads read
@@ -172,23 +185,67 @@ class PluginCatalog:
             by_manifest=False)
         return str(plugin_dir) if plugin_dir is not None else None
 
-    def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
-        """The manifest's ``display_modes``, or [].
+    def _runtime_view(self) -> Any:
+        """The display's runtime view, read at most once a second; None
+        without a source or when reading it fails."""
+        if self.runtime_source is None:
+            return None
+        now = time.monotonic()
+        memo = self._runtime_view_memo
+        if memo is not None and now - memo[0] < _RUNTIME_VIEW_TTL_SECONDS:
+            return memo[1]
+        try:
+            view = self.runtime_source()
+        except Exception as exc:  # a lookup must still answer from manifests
+            self.logger.debug("Could not read the display's runtime view: %s", exc)
+            view = None
+        self._runtime_view_memo = (now, view)
+        return view
 
-        What the display actually rotates can differ: a plugin may compute
-        its modes at run time (``plugin.modes``). This is the declared list.
+    def _live_display_modes(self, plugin_id: str) -> Optional[List[str]]:
+        """The modes the running display registered for ``plugin_id``, or None."""
+        view = self._runtime_view()
+        if view is None:
+            return None
+        try:
+            modes = view.display_modes(plugin_id)
+        except Exception as exc:  # includes a source returning something else
+            self.logger.debug("Could not read display modes for %s: %s", plugin_id, exc)
+            return None
+        return list(modes) if isinstance(modes, list) and modes else None
+
+    def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
+        """The modes the display registered for the plugin, else the
+        manifest's ``display_modes``, else [].
+
+        A plugin may compute its modes at run time (``plugin.modes``): each
+        league soccer-scoreboard's ``custom_leagues`` adds is a mode no
+        manifest can list ahead of time (#668). The running display
+        publishes what it registered, and that wins while the display is
+        live and has the plugin loaded. Otherwise -- display stopped, plugin
+        disabled -- the declared list is the best answer there is.
         """
+        live = self._live_display_modes(plugin_id)
+        if live is not None:
+            return live
         with self._lock:
             manifest = self.plugin_manifests.get(plugin_id)
         modes = (manifest or {}).get('display_modes', [])
         return list(modes) if isinstance(modes, list) else []
 
     def find_plugin_for_mode(self, mode: str) -> Optional[str]:
-        """The plugin whose manifest declares ``mode`` (case-insensitive)."""
+        """The plugin that registered ``mode`` on the running display, else
+        the one whose manifest declares it (case-insensitive both ways)."""
         wanted = mode.strip().lower()
         with self._lock:
             manifests = dict(self.plugin_manifests)
+        for plugin_id in manifests:
+            live = self._live_display_modes(plugin_id)
+            if live and any(m.lower() == wanted for m in live):
+                return plugin_id
         for plugin_id, manifest in manifests.items():
+            if self._live_display_modes(plugin_id):
+                continue  # the display's list is the truth for this plugin
             modes = manifest.get('display_modes')
             if isinstance(modes, list) and any(
                     isinstance(m, str) and m.lower() == wanted for m in modes):

src/plugin_system/plugin_executor.py

@@ -92,7 +92,10 @@ class PluginExecutor:
                 with plugin_scope(plugin_id):
                     result_container['value'] = operation()
                 result_container['completed'] = True
-            except Exception as e:
+            except BaseException as e:  # pylint: disable=broad-except
+                # asyncio.CancelledError and SystemExit too: uncaught, one
+                # ended this thread with 'completed' unset, and an operation
+                # that failed at once was reported as timing out.
                 result_container['exception'] = e
                 result_container['completed'] = True
         

src/plugin_system/plugin_loader.py

@@ -199,9 +199,22 @@ def contained_plugin_dir(plugin_dir: Path, plugins_dir: Path) -> Optional[str]:
     name that came out of ``os.scandir()`` on the trusted root carries no
     taint, which is a real containment guarantee (and one CodeQL's
     path-injection query can follow), not a string sanitiser.
+
+    The entry looked for is the one ``plugin_dir`` itself names when it sits
+    directly in ``plugins_dir``: for a dev plugin symlinked in under its id,
+    the link's name. Resolving the link first and looking for the target's
+    folder name refused ``plugins/foo -> ~/.ledmatrix-dev-plugins/ledmatrix-foo``
+    (what ``dev_plugin_setup.sh link-github foo <url>`` makes), so the plugin
+    never loaded. Any other path is resolved and matched by its final name,
+    as before.
     """
-    plugin_dir_real = os.path.realpath(str(plugin_dir))
     plugins_dir_real = os.path.realpath(str(plugins_dir))
+    plugin_dir_abs = os.path.abspath(str(plugin_dir))
+    if os.path.realpath(os.path.dirname(plugin_dir_abs)) == plugins_dir_real:
+        matched_name = find_trusted_subdir(plugins_dir_real, os.path.basename(plugin_dir_abs))
+        if matched_name is not None:
+            return os.path.join(plugins_dir_real, matched_name)
+    plugin_dir_real = os.path.realpath(str(plugin_dir))
     matched_name = find_trusted_subdir(plugins_dir_real, os.path.basename(plugin_dir_real))
     if matched_name is None:
         return None
@@ -243,6 +256,10 @@ class PluginLoader:
         self.logger = logger or get_logger(__name__)
         self._loaded_modules: Dict[str, Any] = {}
         self._plugin_module_registry: Dict[str, set] = {}  # Maps plugin_id to set of module names
+        # plugin_id -> {dotted name: module} for the modules of the plugin's
+        # own packages (``providers.feed``). They keep their names while the
+        # plugin runs and are dropped with it; see _iter_plugin_submodules.
+        self._plugin_submodules: Dict[str, Dict[str, Any]] = {}
         # Lock to serialize module loading when plugins share module names
         # (e.g., scroll_display.py, game_renderer.py across sport plugins).
         # During exec_module, bare-name sub-modules temporarily appear in
@@ -449,6 +466,45 @@ class PluginLoader:
                 continue
         return result
 
+    @staticmethod
+    def _iter_plugin_submodules(
+        plugin_dir: Path, before_keys: set
+    ) -> list:
+        """Return dotted-name modules from plugin_dir added after before_keys.
+
+        The modules of a package the plugin ships (``providers.feed`` from
+        ``providers/feed.py``). _iter_plugin_bare_modules skips them, so the
+        bare ``providers`` was namespaced and dropped on unload while
+        ``providers.feed`` stayed in sys.modules: a reload after a store update
+        imported a fresh ``providers`` and then got the old ``feed`` back from
+        the cache, running the new manager.py against the old helpers until the
+        display restarted.
+
+        A module counts when its ``__file__`` -- or, for a namespace package,
+        which has none, every ``__path__`` entry -- is inside plugin_dir, so a
+        library the plugin imports (``requests.adapters``) never does.
+
+        Returns a list of (mod_name, module) tuples.
+        """
+        resolved_dir = plugin_dir.resolve()
+        result = []
+        for key in set(sys.modules.keys()) - before_keys:
+            if "." not in key:
+                continue
+            mod = sys.modules.get(key)
+            if mod is None:
+                continue
+            mod_file = getattr(mod, "__file__", None)
+            locations = [mod_file] if mod_file else list(getattr(mod, "__path__", None) or [])
+            if not locations:
+                continue
+            try:
+                if all(Path(loc).resolve().is_relative_to(resolved_dir) for loc in locations):
+                    result.append((key, mod))
+            except (ValueError, TypeError, OSError):
+                continue
+        return result
+
     def _evict_stale_bare_modules(self, plugin_dir: Path) -> dict:
         """Temporarily remove bare-name sys.modules entries from other plugins.
 
@@ -527,6 +583,13 @@ class PluginLoader:
         # Track for cleanup during unload
         self._plugin_module_registry[plugin_id] = namespaced_names
 
+        # The modules of the plugin's own packages keep their dotted names
+        # while it runs -- as they always have, so the package and its
+        # children stay a matching set in sys.modules -- and are dropped
+        # with the plugin by unregister_plugin_modules().
+        self._plugin_submodules[plugin_id] = dict(
+            self._iter_plugin_submodules(plugin_dir, before_keys))
+
         if namespaced_names:
             self.logger.info(
                 "Namespace-isolated %d module(s) for plugin %s",
@@ -537,10 +600,16 @@ class PluginLoader:
         """Remove namespaced sub-modules and cached module for a plugin from sys.modules.
 
         Called by PluginManager during unload to clean up all module entries
-        that were created when the plugin was loaded.
+        that were created when the plugin was loaded, including the dotted
+        modules of its packages. A dotted name is dropped only while it still
+        holds this plugin's module: the name is not namespaced, so another
+        plugin may have put its own there since.
         """
         for ns_name in self._plugin_module_registry.pop(plugin_id, set()):
             sys.modules.pop(ns_name, None)
+        for name, mod in self._plugin_submodules.pop(plugin_id, {}).items():
+            if sys.modules.get(name) is mod:
+                sys.modules.pop(name, None)
         self._loaded_modules.pop(plugin_id, None)
 
     def load_module(
@@ -646,11 +715,13 @@ class PluginLoader:
                     if evicted_name not in sys.modules:
                         sys.modules[evicted_name] = evicted_mod
                 # Clean up the partially-initialized main module and any
-                # bare-name sub-modules that were added during exec_module
-                # so they don't leak into subsequent plugin loads.
+                # bare-name or package sub-modules that were added during
+                # exec_module so they don't leak into subsequent plugin loads.
                 sys.modules.pop(module_name, None)
                 for key, _ in self._iter_plugin_bare_modules(plugin_dir, before_keys):
                     sys.modules.pop(key, None)
+                for key, _ in self._iter_plugin_submodules(plugin_dir, before_keys):
+                    sys.modules.pop(key, None)
                 raise
 
             self._loaded_modules[plugin_id] = module

src/plugin_system/plugin_local_files.py

@@ -0,0 +1,204 @@
+"""
+Files a plugin writes beside itself at runtime, which an update must keep.
+
+A store update replaces a plugin's directory with a fresh download and then
+deletes the old copy. Anything the plugin created there -- OAuth tokens, a
+client-secrets file, a PKCE verifier, cached state -- is in no release, so the
+fresh download does not contain it and deleting the old copy destroys it. On
+2026-10-04 updating calendar 1.2.9 -> 1.2.12 that way deleted its
+``token.pickle`` and ``credentials.json``, and the calendar stopped until they
+were restored from a backup.
+
+What counts as "the plugin's own local file" is the union of:
+
+* :data:`KNOWN_STATE_PATTERNS` -- secret and state files plugins are known to
+  write, kept even when a plugin forgot to gitignore them; and
+* whatever the plugin's own ``.gitignore`` (old copy or new) excludes. A file
+  the author ignores is by definition not part of a release.
+
+A file the new release ships is never overwritten: tracked content wins. Byte
+code (``__pycache__``, ``*.pyc``) and ``.git`` are never carried, since they
+belong to the old code rather than to the user.
+"""
+
+from __future__ import annotations
+
+import fnmatch
+import os
+import re
+import shutil
+from pathlib import Path
+from typing import Iterable, List, Optional, Pattern, Tuple
+
+__all__ = [
+    'KNOWN_STATE_PATTERNS',
+    'carry_over_local_files',
+    'is_known_state_file',
+    'local_files_to_keep',
+]
+
+# Basename globs. Kept even when the plugin's .gitignore does not list them.
+KNOWN_STATE_PATTERNS: Tuple[str, ...] = (
+    'token.pickle',
+    '*.pickle',
+    'token.json',
+    'credentials.json',
+    'config_secrets.json',
+    '.pkce_code_verifier',
+)
+
+_NEVER_CARRY_DIRS = frozenset({'.git', '__pycache__'})
+_NEVER_CARRY_SUFFIXES = ('.pyc', '.pyo')
+
+
+def is_known_state_file(rel_path: str) -> bool:
+    """True when ``rel_path``'s basename is a known secret/state file."""
+    name = rel_path.replace('\\', '/').rsplit('/', 1)[-1]
+    return any(fnmatch.fnmatchcase(name, p) for p in KNOWN_STATE_PATTERNS)
+
+
+class _GitIgnore:
+    """The subset of gitignore semantics plugin .gitignore files use.
+
+    Supports comments, ``!`` negation (last match wins), a trailing ``/`` for
+    directory-only patterns, anchoring by a leading or embedded ``/``, ``*``,
+    ``?``, ``[...]`` and ``**``. As in git, a file under an ignored directory
+    is ignored regardless of later negations.
+    """
+
+    def __init__(self, lines: Iterable[str]):
+        self._rules: List[Tuple[Pattern[str], bool, bool]] = []
+        for raw in lines:
+            line = raw.rstrip('\n').rstrip()
+            if not line or line.startswith('#'):
+                continue
+            negate = line.startswith('!')
+            if negate:
+                line = line[1:]
+            elif line.startswith('\\'):
+                line = line[1:]
+            dir_only = line.endswith('/')
+            line = line.rstrip('/')
+            if not line:
+                continue
+            anchored = '/' in line
+            line = line.lstrip('/')
+            body = self._translate(line)
+            regex = body if anchored else r'(?:.*/)?' + body
+            self._rules.append((re.compile(r'\A' + regex + r'\Z'), negate, dir_only))
+
+    @staticmethod
+    def _translate(pattern: str) -> str:
+        out, i, n = [], 0, len(pattern)
+        while i < n:
+            if pattern.startswith('**/', i):
+                out.append(r'(?:.*/)?')
+                i += 3
+            elif pattern.startswith('/**', i) and i + 3 == n:
+                out.append(r'/.*')
+                i += 3
+            elif pattern.startswith('**', i):
+                out.append(r'.*')
+                i += 2
+            elif pattern[i] == '*':
+                out.append(r'[^/]*')
+                i += 1
+            elif pattern[i] == '?':
+                out.append(r'[^/]')
+                i += 1
+            elif pattern[i] == '[':
+                end = pattern.find(']', i + 1)
+                if end == -1:
+                    out.append(re.escape('['))
+                    i += 1
+                else:
+                    cls = pattern[i + 1:end]
+                    if cls.startswith('!'):
+                        cls = '^' + cls[1:]
+                    out.append('[' + cls.replace('\\', '\\\\') + ']')
+                    i = end + 1
+            else:
+                out.append(re.escape(pattern[i]))
+                i += 1
+        return ''.join(out)
+
+    def _decide(self, rel: str, is_dir: bool) -> Optional[bool]:
+        verdict = None
+        for regex, negate, dir_only in self._rules:
+            if dir_only and not is_dir:
+                continue
+            if regex.match(rel):
+                verdict = not negate
+        return verdict
+
+    def ignores(self, rel_path: str) -> bool:
+        if not self._rules:
+            return False
+        parts = rel_path.replace('\\', '/').split('/')
+        for depth in range(1, len(parts)):
+            if self._decide('/'.join(parts[:depth]), True):
+                return True
+        return bool(self._decide('/'.join(parts), False))
+
+
+def _read_gitignore(plugin_dir: Path) -> List[str]:
+    try:
+        return (plugin_dir / '.gitignore').read_text(
+            encoding='utf-8', errors='replace').splitlines()
+    except OSError:
+        return []
+
+
+def local_files_to_keep(old_dir: Path, new_dir: Path) -> List[str]:
+    """Relative paths (``/``-separated) in ``old_dir`` to copy into ``new_dir``.
+
+    Regular files only; symlinks and anything the new release already ships
+    are skipped.
+    """
+    old_dir, new_dir = Path(old_dir), Path(new_dir)
+    ignore = _GitIgnore(_read_gitignore(old_dir) + _read_gitignore(new_dir))
+    keep: List[str] = []
+    for root, dirs, files in os.walk(old_dir):
+        dirs[:] = sorted(d for d in dirs if d not in _NEVER_CARRY_DIRS
+                         and not os.path.islink(os.path.join(root, d)))
+        rel_root = os.path.relpath(root, old_dir)
+        for name in sorted(files):
+            if name.endswith(_NEVER_CARRY_SUFFIXES):
+                continue
+            full = os.path.join(root, name)
+            if os.path.islink(full) or not os.path.isfile(full):
+                continue
+            rel = name if rel_root == '.' else f"{rel_root}/{name}".replace('\\', '/')
+            if not (is_known_state_file(rel) or ignore.ignores(rel)):
+                continue
+            if os.path.lexists(new_dir / rel):
+                continue
+            keep.append(rel)
+    return keep
+
+
+def carry_over_local_files(
+    old_dir: Path, new_dir: Path
+) -> Tuple[List[str], List[Tuple[str, str]]]:
+    """Copy the plugin's local files from ``old_dir`` into ``new_dir``.
+
+    Copies rather than moves, so ``old_dir`` stays a complete copy until the
+    caller deletes it. Returns ``(copied, failed)`` where ``failed`` pairs a
+    relative path with the error; the caller should keep ``old_dir`` when
+    anything failed.
+    """
+    copied: List[str] = []
+    failed: List[Tuple[str, str]] = []
+    try:
+        candidates = local_files_to_keep(old_dir, new_dir)
+    except OSError as e:
+        return copied, [('.', str(e))]
+    for rel in candidates:
+        dest = Path(new_dir) / rel
+        try:
+            dest.parent.mkdir(parents=True, exist_ok=True)
+            shutil.copy2(Path(old_dir) / rel, dest)
+            copied.append(rel)
+        except OSError as e:
+            failed.append((rel, str(e)))
+    return copied, failed

src/plugin_system/plugin_manager.py

@@ -15,6 +15,7 @@ import sys
 import time
 import threading
 import types
+import uuid
 from pathlib import Path
 from typing import Callable, Dict, List, NamedTuple, Optional, Any, Tuple, Union
 import logging
@@ -214,6 +215,9 @@ class PluginManager:
         # add_update_listener(). A tuple, replaced rather than mutated, so the
         # worker can iterate it without a lock.
         self._update_listeners: Tuple[Callable[[str], None], ...] = ()
+        # Where plugins' on-demand requests go: the display controller's
+        # submit_plugin_on_demand. See set_on_demand_handler().
+        self._on_demand_handler: Optional[Callable[[Dict[str, Any]], bool]] = None
         # Config changes that found the plugin's lock busy, latest per plugin,
         # with the instance they were meant for. See apply_config_change().
         self._deferred_config_changes: Dict[str, Tuple[Any, Dict[str, Any]]] = {}
@@ -597,11 +601,21 @@ class PluginManager:
             self.plugin_loader.unregister_plugin_modules(plugin_id)
         except Exception as e:  # pragma: no cover - defensive
             self.logger.debug("Could not drop modules of %s: %s", plugin_id, e)
-        try:
-            if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
-                self.font_manager.forget_manager_fonts(plugin_id)
-        except Exception as e:
-            self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
+        self._forget_plugin_fonts(plugin_id)
+
+    def _forget_plugin_fonts(self, plugin_id: str) -> None:
+        """Drop what the FontManager holds for a plugin: the fonts its
+        instance reported using (the Fonts tab's "Used by") and the fonts its
+        manifest registered. Never raises."""
+        if self.font_manager is None:
+            return
+        for name in ('forget_manager_fonts', 'forget_plugin_fonts'):
+            if not hasattr(self.font_manager, name):
+                continue
+            try:
+                getattr(self.font_manager, name)(plugin_id)
+            except Exception as e:
+                self.logger.debug("Could not forget fonts of %s (%s): %s", plugin_id, name, e)
     
     #: Config keys the **core** reads out of a plugin's own config block. The
     #: plugin never declares them, so a schema with
@@ -846,12 +860,9 @@ class PluginManager:
             # Delegate sub-module and cached-module cleanup to the loader
             self.plugin_loader.unregister_plugin_modules(plugin_id)
 
-            # Its font registrations go with it (the Fonts tab's "Used by").
-            try:
-                if self.font_manager is not None and hasattr(self.font_manager, 'forget_manager_fonts'):
-                    self.font_manager.forget_manager_fonts(plugin_id)
-            except Exception as e:
-                self.logger.debug("Could not forget fonts of %s: %s", plugin_id, e)
+            # Its font registrations go with it: the fonts it reported using
+            # and the ones its manifest registered.
+            self._forget_plugin_fonts(plugin_id)
 
             # Update state
             self.state_manager.set_state(plugin_id, PluginState.UNLOADED)
@@ -1163,7 +1174,7 @@ class PluginManager:
     def _record_update_failure(
         self,
         plugin_id: str,
-        exc: Optional[Exception] = None,
+        exc: Optional[BaseException] = None,
         log: bool = True,
         count_failure: bool = True,
     ) -> None:
@@ -1187,7 +1198,7 @@ class PluginManager:
         """
         failure_time = time.time()
         if exc is not None:
-            err: Exception = exc
+            err: BaseException = exc
             error_type = type(exc).__name__
         else:
             err = Exception(f"Plugin {plugin_id} execution failed (timeout or executor error)")
@@ -1653,7 +1664,7 @@ class PluginManager:
         finish_guard = threading.Lock()
         finished = {'done': False}
 
-        def _finish(success: bool, exc: Optional[Exception] = None) -> None:
+        def _finish(success: bool, exc: Optional[BaseException] = None) -> None:
             with finish_guard:
                 if finished['done']:
                     return
@@ -1727,7 +1738,13 @@ class PluginManager:
                     self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
                 else:
                     plugin_instance.update()
-            except Exception as exc:
+            except BaseException as exc:  # pylint: disable=broad-except
+                # BaseException, not just Exception: asyncio.CancelledError
+                # and SystemExit derive from it. Either one skipped _finish,
+                # so the plugin kept its lock and stayed RUNNING for good --
+                # never rescheduled, and every display() skipped as busy.
+                # Re-raised for the executor, which reports it as this
+                # update's failure.
                 _finish(False, exc=exc)
                 raise
             else:
@@ -1838,3 +1855,73 @@ class PluginManager:
             done = sorted(self._completed_updates)
             self._completed_updates.clear()
             return done
+
+    # -- on-demand requests from plugins -------------------------------------
+
+    def set_on_demand_handler(
+            self, handler: Optional[Callable[[Dict[str, Any]], bool]]) -> None:
+        """Route plugins' on-demand requests to ``handler`` (None: nowhere).
+
+        The display controller sets its ``submit_plugin_on_demand`` here
+        before any plugin loads. The handler takes a mailbox-shaped request
+        from any thread, queues it for the render thread and returns True,
+        or False when it could not. A plugin manager with no handler (the
+        web interface's, a test's, scripts/check_plugin.py's) has no screen
+        to give, so request_on_demand() there answers None.
+        """
+        self._on_demand_handler = handler
+
+    def request_on_demand(self, plugin_id: str, mode: Optional[str] = None,
+                          duration: Optional[float] = None,
+                          pinned: bool = False) -> Optional[str]:
+        """Ask the display to show ``plugin_id`` now. Safe from any thread.
+
+        BasePlugin.request_on_demand() lands here; see it for the arguments.
+        Returns the request id once the display has queued the request (it
+        is applied on the render thread within a frame or so), or None when
+        this process has no display to ask or its queue is full.
+        """
+        if not isinstance(plugin_id, str) or not plugin_id:
+            raise ValueError('plugin_id is required')
+        if mode is not None and (not isinstance(mode, str) or not mode):
+            raise ValueError('mode must be a non-empty string or None')
+        if duration is not None:
+            if isinstance(duration, bool) or not isinstance(duration, (int, float)):
+                raise ValueError('duration must be a number of seconds or None')
+            if not math.isfinite(duration) or duration <= 0:
+                duration = None   # the display reads these as "no limit" too
+            else:
+                duration = float(duration)
+        return self._submit_on_demand({
+            'action': 'start', 'plugin_id': plugin_id, 'mode': mode,
+            'duration': duration, 'pinned': bool(pinned)})
+
+    def end_on_demand(self, plugin_id: str) -> Optional[str]:
+        """Give the screen back, if ``plugin_id``'s on-demand session has it.
+
+        BasePlugin.end_on_demand() lands here. A session the plugin does not
+        own (the user started another plugin from the web interface, say) is
+        left alone. Returns the request id once queued, or None as
+        request_on_demand() does.
+        """
+        if not isinstance(plugin_id, str) or not plugin_id:
+            raise ValueError('plugin_id is required')
+        return self._submit_on_demand({'action': 'stop', 'plugin_id': plugin_id})
+
+    def _submit_on_demand(self, request: Dict[str, Any]) -> Optional[str]:
+        # __dict__.get: tests build bare managers with PluginManager.__new__.
+        handler = self.__dict__.get('_on_demand_handler')
+        if handler is None:
+            return None
+        request_id = str(uuid.uuid4())
+        request.update({'request_id': request_id, 'timestamp': time.time(),
+                        'source': 'plugin'})
+        try:
+            accepted = handler(request)
+        except Exception as exc:  # pylint: disable=broad-except
+            self._warn_rate_limited(
+                "on-demand-handler",
+                "The on-demand request from plugin %s failed: %r",
+                request.get('plugin_id'), exc)
+            return None
+        return request_id if accepted else None

src/plugin_system/plugin_runtime.py

@@ -36,6 +36,13 @@ tmpfs. A missing heartbeat (dev server, emulator, Windows, a display still
 starting up) or one from another process (a display restarted after a
 watchdog kill) says nothing, and the snapshot is judged on its own.
 
+The control socket. Where the display serves its state stream (stage 3,
+docs/IPC_CONTROL_SOCKET.md), every tick also hands the snapshot to it, in
+memory, and the web interface reads it there first
+(``view_from_socket_state``, judged by the same rules). While the socket
+serves those readers, the cache copy is their fallback and an unchanged
+snapshot is rewritten every ``RELAXED_REFRESH_INTERVAL`` instead.
+
 A dead publisher. systemd removes the heartbeat's directory when the
 service stops, so after a watchdog kill there is no heartbeat to go stale.
 The reader then asks whether the snapshot's ``pid`` still exists (POSIX
@@ -48,8 +55,8 @@ import math
 import os
 import threading
 import time
-from dataclasses import dataclass, field
-from typing import Any, Callable, Dict, Optional
+from dataclasses import dataclass, field, replace
+from typing import Any, Callable, Dict, List, Optional
 
 from src import display_watchdog
 from src.logging_config import get_logger
@@ -75,6 +82,15 @@ TICK_INTERVAL = 5.0
 #: A snapshot older than this is stale: three missed refreshes.
 STALE_AFTER = 3 * REFRESH_INTERVAL
 
+#: The refresh while the control socket serves the web interface's readers
+#: (``StateHub.readers_active``). The cache copy is then only their fallback,
+#: so an unchanged snapshot is rewritten half as often; the snapshot says so
+#: in its own ``refresh_interval`` and ``stale_after``.
+RELAXED_REFRESH_INTERVAL = 2 * REFRESH_INTERVAL
+
+#: The control socket's section for this snapshot (``state.plugins``).
+STATE_SECTION = "plugins"
+
 #: Bounds on a published ``stale_after``, so a corrupt value can make a
 #: reader neither trust a dead display for hours nor distrust a live one.
 _STALE_AFTER_MIN = 30.0
@@ -84,6 +100,9 @@ _ERROR_MESSAGE_CHARS = 200
 _ERROR_TYPE_CHARS = 80
 _ID_CHARS = 100
 _VERSION_CHARS = 40
+#: Bounds on a plugin's published ``modes``: a plugin computes them, so a
+#: runaway list must not bloat a file written to the SD card.
+_MAX_MODES = 200
 
 #: Reader statuses. Only LIVE carries runtime facts.
 LIVE = "live"
@@ -138,9 +157,19 @@ def summarize_error(error_info: Optional[Dict[str, Any]]) -> Optional[Dict[str,
     }
 
 
+def _published_modes(modes: Any) -> Optional[List[str]]:
+    """The registered display modes as a snapshot carries them, or None."""
+    if not isinstance(modes, list):
+        return None
+    # A name is a key the display matches exactly: drop one too long to
+    # carry whole rather than clip it into a different name.
+    return [m for m in modes if isinstance(m, str) and len(m) <= _ID_CHARS][:_MAX_MODES]
+
+
 def build_runtime_snapshot(state_manager: Any, *, started_at: float,
                            now: Optional[float] = None,
-                           running: bool = True) -> Dict[str, Any]:
+                           running: bool = True,
+                           refresh_interval: float = REFRESH_INTERVAL) -> Dict[str, Any]:
     """The snapshot for ``state_manager`` (a plugin_state.PluginStateManager).
 
     A stopped snapshot (``running=False``) lists no plugins: nothing is
@@ -156,14 +185,15 @@ def build_runtime_snapshot(state_manager: Any, *, started_at: float,
                 "error": summarize_error(record.get("error_info")),
                 "version": _clip(version, _VERSION_CHARS) if version else None,
                 "loaded_at": _epoch(record.get("loaded_at")),
+                "modes": _published_modes(record.get("modes")),
             }
     return {
         "schema": SNAPSHOT_SCHEMA,
         "running": running,
         "published_at": time.time() if now is None else now,
         "started_at": started_at,
-        "refresh_interval": REFRESH_INTERVAL,
-        "stale_after": STALE_AFTER,
+        "refresh_interval": refresh_interval,
+        "stale_after": 3 * refresh_interval,
         "pid": os.getpid(),
         "plugins": plugins,
     }
@@ -197,30 +227,87 @@ class PluginRuntimePublisher:
         self._tick_lock = threading.Lock()
         self._stop = threading.Event()
         self._thread: Optional[threading.Thread] = None
+        # The control socket's state stream (src/ipc/server.StateHub), when
+        # the display serves one: every tick also hands it the snapshot, in
+        # memory, and the cache refresh relaxes while it has readers.
+        self._hub: Any = None
+        self._hub_change: Optional[int] = None
+        self._hub_snapshot: Optional[Dict[str, Any]] = None
+        self.relaxed_refresh_interval = RELAXED_REFRESH_INTERVAL
 
-    def _write(self, running: bool) -> None:
-        snapshot = build_runtime_snapshot(self.state_manager, started_at=self.started_at,
-                                          now=self._wall_clock(), running=running)
+    def attach_hub(self, hub: Any) -> None:
+        """Also publish to the control socket's state hub, starting now."""
+        with self._tick_lock:
+            self._hub = hub
+            self._hub_change = None
+            self._hub_snapshot = None
+            try:
+                self._push_to_hub(self.state_manager.change_count)
+            except Exception as err:  # never let reporting break the display
+                logger.debug("Could not publish the plugin runtime state: %s", err,
+                             exc_info=True)
+
+    def _push_to_hub(self, change: int) -> None:
+        """The snapshot to the state hub: rebuilt when the state machine
+        changed, otherwise the last one with a new ``published_at``, which
+        the hub does not count as a new version. In memory, every tick, so
+        the socket's copy is never more than a tick old."""
+        hub = self._hub
+        if hub is None:
+            return
+        now = self._wall_clock()
+        if self._hub_snapshot is None or change != self._hub_change:
+            snapshot = build_runtime_snapshot(self.state_manager, started_at=self.started_at,
+                                              now=now)
+        else:
+            snapshot = dict(self._hub_snapshot, published_at=now)
+        hub.publish(STATE_SECTION, snapshot, volatile=("published_at",))
+        self._hub_snapshot = snapshot
+        self._hub_change = change
+
+    def _cache_refresh_interval(self) -> float:
+        """The cache refresh: relaxed while the socket serves the readers."""
+        hub = self._hub
+        try:
+            if hub is not None and hub.readers_active():
+                return self.relaxed_refresh_interval
+        except Exception:  # pylint: disable=broad-except
+            # The normal interval is the safe answer: it only writes more.
+            logger.debug("State hub readers_active() failed; using the normal refresh", exc_info=True)
+        return self.refresh_interval
+
+    def _write(self, running: bool, refresh_interval: Optional[float] = None) -> None:
+        snapshot = build_runtime_snapshot(
+            self.state_manager, started_at=self.started_at, now=self._wall_clock(),
+            running=running,
+            refresh_interval=self.refresh_interval if refresh_interval is None
+            else refresh_interval)
         self.cache_manager.set(PLUGIN_RUNTIME_KEY, snapshot)
 
     def tick(self) -> bool:
         """Publish if something changed (throttled) or the refresh is due.
-        True if a snapshot was written."""
+        True if a snapshot was written to the cache."""
         with self._tick_lock:
             try:
                 change = self.state_manager.change_count
+                try:
+                    self._push_to_hub(change)
+                except Exception as err:  # the cache copy still goes out below
+                    logger.debug("Could not publish the plugin runtime state: %s", err,
+                                 exc_info=True)
                 now = self._clock()
+                refresh = self._cache_refresh_interval()
                 since = None if self._last_attempt is None else now - self._last_attempt
                 if since is not None:
                     if change == self._published_change:
-                        if since < self.refresh_interval:
+                        if since < refresh:
                             return False
                     elif since < self.min_interval:
                         return False
                 # Stamp the attempt before writing: a cache that keeps failing
                 # is retried at the throttled rate, not on every tick.
                 self._last_attempt = now
-                self._write(running=True)
+                self._write(running=True, refresh_interval=refresh)
                 self._published_change = change
                 return True
             except Exception as err:  # never let reporting break the display
@@ -317,6 +404,9 @@ class PluginRuntimeView:
     plugins: Dict[str, Dict[str, Any]] = field(default_factory=dict)
     #: Age of the render loop's heartbeat, when it was taken into account.
     heartbeat_age_seconds: Optional[float] = None
+    #: Where the snapshot came from: ``cache`` (the shared cache file and the
+    #: heartbeat file) or ``socket`` (the control socket's state stream).
+    source: str = "cache"
 
     @property
     def live(self) -> bool:
@@ -339,6 +429,21 @@ class PluginRuntimeView:
             "loaded_at": record.get("loaded_at"),
         }
 
+    def display_modes(self, plugin_id: str) -> Optional[List[str]]:
+        """The display modes the display registered for ``plugin_id``: what
+        it rotates and accepts on-demand, including modes a plugin computes
+        from its config. None unless the view is live and the plugin is
+        loaded with its modes registered -- the caller then falls back to
+        the manifest's ``display_modes``."""
+        if not self.live:
+            return None
+        record = self.plugins.get(plugin_id)
+        modes = record.get("modes") if isinstance(record, dict) else None
+        if not isinstance(modes, list):
+            return None
+        modes = [m for m in modes if isinstance(m, str)]
+        return modes or None
+
     def describe(self) -> Dict[str, Any]:
         """The view's own status, for a response to carry beside the facts."""
         return {
@@ -348,6 +453,7 @@ class PluginRuntimeView:
             "stale_after": self.stale_after,
             "heartbeat_age_seconds": (None if self.heartbeat_age_seconds is None
                                       else round(self.heartbeat_age_seconds, 1)),
+            "source": self.source,
         }
 
 
@@ -445,6 +551,36 @@ def view_from_snapshot(snapshot: Any, now: Optional[float] = None,
     )
 
 
+def view_from_socket_state(snapshot: Any, now: Optional[float] = None,
+                           now_mono: Optional[float] = None) -> Optional[PluginRuntimeView]:
+    """Judge the ``plugins`` section of a control-socket state snapshot by
+    the same rules as the cache copy; None when it has none (an older
+    display, or a snapshot too large to carry it), so the caller reads the
+    cache instead.
+
+    The display measured its render loop's heartbeat age when it answered
+    (``state.loop``); that is the heartbeat here, aged by the time since the
+    answer arrived. A live snapshot with a stalled loop is ``stalled``, and a
+    snapshot older than its ``stale_after`` (the publisher thread stopped)
+    is ``stale``, exactly as for the cache. The display answered, so its
+    process is alive: there is no pid check.
+    """
+    from src.ipc.client import snapshot_loop_age  # stdlib-only module
+    if not isinstance(snapshot, dict):
+        return None
+    state = snapshot.get("state")
+    plugins = state.get(STATE_SECTION) if isinstance(state, dict) else None
+    if not isinstance(plugins, dict):
+        return None
+    now_mono = time.monotonic() if now_mono is None else now_mono
+    beat_age = snapshot_loop_age(snapshot, now_mono=now_mono)
+    heartbeat = None
+    if beat_age is not None:
+        heartbeat = {"pid": plugins.get("pid"), "mono": now_mono - beat_age}
+    view = view_from_snapshot(plugins, now=now, heartbeat=heartbeat, now_mono=now_mono)
+    return replace(view, source="socket")
+
+
 def read_plugin_runtime(cache_manager: Any, now: Optional[float] = None,
                         heartbeat_path: Optional[str] = None) -> PluginRuntimeView:
     """The display's latest snapshot, judged for staleness and against the

src/plugin_system/plugin_state.py

@@ -10,7 +10,7 @@ snapshot ``plugin_runtime.PluginRuntimePublisher`` publishes from it.
 import threading
 import time
 from enum import Enum
-from typing import Optional, Dict, Any
+from typing import Any, Dict, List, Optional
 from datetime import datetime
 import logging
 
@@ -231,6 +231,26 @@ class PluginStateManager:
             }
             self._note_change()
 
+    def record_modes(self, plugin_id: str, modes: List[str]) -> None:
+        """Record the display modes the display registered for ``plugin_id``.
+
+        Called by the DisplayController each time it registers the plugin.
+        These are the modes it actually rotates and accepts on-demand --
+        ``plugin.modes`` when the plugin computes them (a soccer league the
+        user added under ``custom_leagues``), else the manifest's list -- and
+        the web interface has no other way to learn them (#668). Kept on the
+        loaded record, so an unload or a reload's fresh record_loaded()
+        forgets them until the plugin is registered again.
+        """
+        with self._lock:
+            loaded = self._loaded.get(plugin_id)
+            if loaded is None:
+                return
+            modes = [str(m) for m in modes]
+            if loaded.get('modes') != modes:
+                loaded['modes'] = modes
+                self._note_change()
+
     def record_unloaded(self, plugin_id: str) -> None:
         """Forget the loaded record alone, keeping state and error info: for
         an unload that failed after the instance was already dropped."""
@@ -243,7 +263,8 @@ class PluginStateManager:
         section so a concurrent load or unload is seen whole or not at all.
 
         Per plugin: ``state`` (published_state()'s value), ``loaded``,
-        ``version`` and ``loaded_at`` (None unless loaded) and ``error_info``
+        ``version``, ``loaded_at`` and ``modes`` (None unless loaded; ``modes``
+        also None until the display registers it) and ``error_info``
         (a copy, or None).
         """
         with self._lock:
@@ -257,6 +278,7 @@ class PluginStateManager:
                     'loaded': loaded is not None,
                     'version': loaded['version'] if loaded else None,
                     'loaded_at': loaded['loaded_at'] if loaded else None,
+                    'modes': list(loaded['modes']) if loaded and 'modes' in loaded else None,
                     'error_info': dict(info) if info is not None else None,
                 }
             return records

src/plugin_system/resource_monitor.py

@@ -8,7 +8,7 @@ Provides resource limits and performance monitoring.
 import math
 import time
 import threading
-from typing import Dict, Optional, Any, Callable, cast
+from typing import Dict, Optional, Any, Callable, Set, cast
 from dataclasses import dataclass, field, fields
 
 from src.logging_config import get_logger
@@ -99,18 +99,33 @@ class ResourceMetrics:
     last_update_time: float = field(default_factory=time.time)
 
 
-#: How often a plugin's metrics are written to the cache, in seconds.
+#: How often the metrics snapshot is written to the cache, in seconds.
 #:
 #: Persisting on every call meant a small file rewritten roughly nine times a
 #: minute per plugin. On a rig with fourteen active plugins that was ~126
 #: writes a minute for metrics alone, and since each ~350-byte file costs a
 #: 4KB block plus an ext4 journal entry, it dominated the device's write
-#: volume -- on an SD card, which wears out.
+#: volume -- on an SD card, which wears out. Throttling each plugin's own
+#: record to once per 30 s still left two writes a minute per plugin, so all
+#: plugins now share one record (METRICS_SNAPSHOT_KEY), written at most once
+#: a minute: one write a minute however many plugins there are.
 #:
 #: The in-memory copy stays authoritative and exact; only the cross-process
-#: snapshot the web UI reads is delayed, and telemetry up to half a minute old
-#: is still a fair description of a long-running plugin.
-_METRICS_PERSIST_INTERVAL = 30.0
+#: snapshot the web UI reads is delayed, and telemetry up to a minute old is
+#: still a fair description of a long-running plugin.
+_METRICS_PERSIST_INTERVAL = 60.0
+
+#: The one cache record holding every plugin's metrics:
+#: ``{"schema": 1, "plugins": {plugin_id: <metrics record>}}``, each metrics
+#: record shaped as the per-plugin ``plugin_metrics:<id>`` records were. Those
+#: older records are still read for a plugin the snapshot does not have yet
+#: (an upgrade, or a plugin that has not run since), never written.
+METRICS_SNAPSHOT_KEY = "plugin_metrics_snapshot"
+_METRICS_SNAPSHOT_SCHEMA = 1
+
+#: A plugin with no call for this long is dropped from the snapshot -- what
+#: the cache's 30-day default retention did to its own record before.
+_METRICS_SNAPSHOT_ENTRY_MAX_AGE = 30 * 86400
 
 
 class PluginResourceMonitor:
@@ -140,10 +155,15 @@ class PluginResourceMonitor:
         self._metrics: Dict[str, ResourceMetrics] = {}
         self._limits: Dict[str, ResourceLimits] = {}
         self._bad_limits_warned: set = set()
-        # When each plugin's metrics last reached the cache. Metrics change on
-        # every call, so they cannot be de-duplicated the way health state can;
-        # they are rate-limited instead. See _METRICS_PERSIST_INTERVAL.
-        self._metrics_persisted_at: Dict[str, float] = {}
+        # When the metrics snapshot last reached the cache (monotonic), None
+        # until it has. Metrics change on every call, so they cannot be
+        # de-duplicated the way health state can; they are rate-limited
+        # instead. See _METRICS_PERSIST_INTERVAL.
+        self._snapshot_persisted_at: Optional[float] = None
+        # Plugins whose metrics this process recorded since the last snapshot
+        # write: only their entries are overwritten, the rest are kept as
+        # found on disk.
+        self._metrics_dirty: Set[str] = set()
 
         # Lock for thread-safe access
         self._lock = threading.Lock()
@@ -247,10 +267,14 @@ class PluginResourceMonitor:
         with self._lock:
             if force_reload or plugin_id not in self._metrics:
                 # Try to load from cache
-                cache_key = self._get_metrics_key(plugin_id)
-                cached = self.cache_manager.get(
-                    cache_key, max_age=None, memory_ttl=0 if force_reload else None
-                )
+                memory_ttl = 0 if force_reload else None
+                cached = self._read_snapshot(memory_ttl).get(plugin_id)
+                if cached is None:
+                    # Not in the snapshot: the per-plugin record an older
+                    # version wrote, if there is one.
+                    cached = self.cache_manager.get(
+                        self._get_metrics_key(plugin_id), max_age=None,
+                        memory_ttl=memory_ttl)
                 if cached:
                     metrics = self._metrics_from_cache(plugin_id, cached)
                 else:
@@ -498,12 +522,70 @@ class PluginResourceMonitor:
             summaries[plugin_id] = self.get_metrics_summary(plugin_id)
         return summaries
     
-    def _persist_metrics(self, plugin_id: str, metrics: ResourceMetrics,
-                         force: bool = False) -> None:
-        """Write a plugin's metrics to the cache, at most once per interval.
+    def _read_snapshot(self, memory_ttl: Optional[int] = None) -> Dict[str, Any]:
+        """The snapshot's per-plugin records, or {} if there is none usable.
 
         Caller must hold ``self._lock``.
         """
+        cached = self.cache_manager.get(
+            METRICS_SNAPSHOT_KEY, max_age=None, memory_ttl=memory_ttl)
+        if not isinstance(cached, dict) or cached.get('schema') != _METRICS_SNAPSHOT_SCHEMA:
+            return {}
+        plugins = cached.get('plugins')
+        return plugins if isinstance(plugins, dict) else {}
+
+    @staticmethod
+    def _metrics_record(metrics: ResourceMetrics) -> Dict[str, Any]:
+        """One plugin's entry in the snapshot."""
+        return {
+            'memory_mb': metrics.memory_mb,
+            'cpu_percent': metrics.cpu_percent,
+            'execution_time': metrics.execution_time,
+            'call_count': metrics.call_count,
+            'total_execution_time': metrics.total_execution_time,
+            'max_execution_time': metrics.max_execution_time,
+            'min_execution_time': (metrics.min_execution_time
+                                   if metrics.min_execution_time != float('inf')
+                                   else 0.0),
+            'last_update_time': metrics.last_update_time,
+        }
+
+    def _write_snapshot(self, drop: Optional[str] = None) -> None:
+        """Write the snapshot: what is on disk, with this process's recorded
+        plugins updated and ``drop`` removed.
+
+        Starting from the disk copy rather than from memory keeps the entries
+        of plugins this process has not run -- disabled ones, which the web UI
+        still shows -- and a reset made from the other process.
+
+        Caller must hold ``self._lock``.
+        """
+        plugins = dict(self._read_snapshot(memory_ttl=0))
+        if drop is not None:
+            plugins.pop(drop, None)
+        for plugin_id in self._metrics_dirty:
+            if plugin_id in self._metrics:
+                plugins[plugin_id] = self._metrics_record(self._metrics[plugin_id])
+        cutoff = time.time() - _METRICS_SNAPSHOT_ENTRY_MAX_AGE
+        for plugin_id, record in list(plugins.items()):
+            last = record.get('last_update_time') if isinstance(record, dict) else None
+            if isinstance(last, (int, float)) and last < cutoff:
+                del plugins[plugin_id]
+        self.cache_manager.set(METRICS_SNAPSHOT_KEY, {
+            'schema': _METRICS_SNAPSHOT_SCHEMA,
+            'plugins': plugins,
+        })
+        # Only once the write has landed, so a failed one is retried in full.
+        self._metrics_dirty.clear()
+
+    def _persist_metrics(self, plugin_id: str, metrics: ResourceMetrics,
+                         force: bool = False) -> None:
+        """Record that a plugin's metrics changed, and write the snapshot if
+        the last write is at least an interval old.
+
+        Caller must hold ``self._lock``.
+        """
+        self._metrics_dirty.add(plugin_id)
         # Monotonic, not wall clock: these devices have no RTC, so the clock
         # jumps by however far off boot-time was the moment NTP first syncs.
         # A forward jump would allow an early write, a backward one would
@@ -515,35 +597,26 @@ class PluginResourceMonitor:
         # single run -- the throttle swallowed the very first snapshot, which
         # is the one that matters most after a restart.
         now = time.monotonic()
-        last_written = self._metrics_persisted_at.get(plugin_id)
+        last_written = self._snapshot_persisted_at
         if (not force and last_written is not None
                 and now - last_written < _METRICS_PERSIST_INTERVAL):
             return
-        cache_key = self._get_metrics_key(plugin_id)
-        self.cache_manager.set(cache_key, {
-            'memory_mb': metrics.memory_mb,
-            'cpu_percent': metrics.cpu_percent,
-            'execution_time': metrics.execution_time,
-            'call_count': metrics.call_count,
-            'total_execution_time': metrics.total_execution_time,
-            'max_execution_time': metrics.max_execution_time,
-            'min_execution_time': (metrics.min_execution_time
-                                   if metrics.min_execution_time != float('inf')
-                                   else 0.0),
-            'last_update_time': metrics.last_update_time,
-        })
+        self._write_snapshot()
         # Only after the write lands. Marking it first would mean a failed
         # set() bought the next interval's silence without leaving a snapshot.
-        self._metrics_persisted_at[plugin_id] = now
+        self._snapshot_persisted_at = now
 
     def reset_metrics(self, plugin_id: str) -> None:
         """Reset metrics for a plugin."""
         with self._lock:
             if plugin_id in self._metrics:
                 self._metrics[plugin_id] = ResourceMetrics()
-                cache_key = self._get_metrics_key(plugin_id)
-                self.cache_manager.delete(cache_key)
+                self._metrics_dirty.discard(plugin_id)
+                self._write_snapshot(drop=plugin_id)
+                # The record an older version wrote, so the reader's fallback
+                # cannot bring the old numbers back.
+                self.cache_manager.delete(self._get_metrics_key(plugin_id))
                 # Let the next call persist immediately rather than leaving the
-                # deleted key absent for the rest of the interval.
-                self._metrics_persisted_at.pop(plugin_id, None)
+                # plugin absent from the snapshot for the rest of the interval.
+                self._snapshot_persisted_at = None
 

src/plugin_system/store_install.py

@@ -22,6 +22,7 @@ from src.plugin_system.plugin_loader import (
     contained_plugin_dir, requirements_to_install,
 )
 from src.plugin_system.plugin_dirs import BACKUP_MARKER
+from src.plugin_system.plugin_local_files import carry_over_local_files
 from src.plugin_system.repo_urls import (
     USER_AGENT, github_api_headers, github_owner_repo, normalize_repo_url,
 )
@@ -92,7 +93,9 @@ class _InstallMixin:
                 raise
 
             if installed:
-                self._discard_backup(plugin_id, backup_path, "install")
+                self._discard_backup(
+                    plugin_id, backup_path, "install",
+                    new_path=self._existing_install(plugin_id) or plugin_path)
                 return True
 
             self._restore_backup(plugin_id, plugin_path, backup_path, "Install")
@@ -133,8 +136,33 @@ class _InstallMixin:
             return f"could not set aside {plugin_path}: {e}"
         return None
 
-    def _discard_backup(self, plugin_id: str, backup_path: Path, action: str) -> None:
-        """Remove the set-aside copy after a successful (re)install."""
+    def _discard_backup(
+        self, plugin_id: str, backup_path: Path, action: str,
+        new_path: Optional[Path] = None,
+    ) -> None:
+        """Remove the set-aside copy after a successful (re)install.
+
+        With ``new_path`` (where the new copy landed), first carries the
+        plugin's own runtime files -- OAuth tokens, client secrets, anything
+        its .gitignore excludes -- from the old copy into the new one: no
+        release contains them, so deleting the old copy would destroy them.
+        See src/plugin_system/plugin_local_files.py. If any could not be
+        copied the old copy is kept, so nothing is lost.
+        """
+        if new_path is not None and new_path.is_dir():
+            copied, failed = carry_over_local_files(backup_path, new_path)
+            if copied:
+                self.logger.info(
+                    "Kept %d local file(s) of %s across the %s: %s",
+                    len(copied), plugin_id, action, ", ".join(copied))
+            if failed:
+                self.logger.error(
+                    "Could not carry %s's local files into the new copy (%s); "
+                    "the previous copy is kept at %s -- copy them back by hand",
+                    plugin_id,
+                    "; ".join(f"{rel}: {err}" for rel, err in failed),
+                    backup_path)
+                return
         if not self._safe_remove_directory(backup_path):
             self.logger.warning(
                 "%s of %s succeeded but the previous copy at %s could not be "
@@ -542,7 +570,8 @@ class _InstallMixin:
                     raise
                 temp_dir = None  # Prevent cleanup since we moved it
                 if backup_path is not None:
-                    self._discard_backup(plugin_id, backup_path, "install")
+                    self._discard_backup(
+                        plugin_id, backup_path, "install", new_path=final_path)
 
             # Install dependencies
             self._install_dependencies(final_path)

src/plugin_system/store_manager.py

@@ -138,6 +138,11 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
         # the registry cache expires. Only one thread fetches; others wait and
         # then get the result from the warm cache (double-checked locking).
         self._registry_fetch_lock = threading.Lock()
+        # refresh_registry_in_background: the one refresh thread, and when
+        # an offline one may be retried (see that method).
+        self._registry_refresh_lock = threading.Lock()
+        self._registry_refresh_thread: Optional[threading.Thread] = None
+        self._registry_refresh_retry_after = 0.0
 
         # Per-plugin locks for _reinstall_with_rollback: the web UI runs
         # Flask with threaded=True, so two overlapping requests for the
@@ -344,12 +349,26 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
         2. Fix permissions via os.chmod() then retry (works for same-owner files)
         3. Use sudo rm -rf as last resort (works for root-owned __pycache__, etc.)
 
+        A symlink -- a dev plugin linked in by scripts/dev/dev_plugin_setup.sh
+        -- is removed as a link, before any of that: rmtree refuses one, and
+        stage 2 would walk through it and chmod the developer's checkout.
+
         Args:
             path: Path to directory to remove
 
         Returns:
             True if directory was removed successfully, False otherwise
         """
+        if path.is_symlink():
+            # Checked before exists(), which follows the link: a dangling one
+            # would read as already removed and be left behind.
+            try:
+                path.unlink()
+                return True
+            except OSError as e:
+                self.logger.error(f"Could not remove the symlink {path}: {e}")
+                return False
+
         if not path.exists():
             return True  # Already removed
 

src/plugin_system/store_registry.py

@@ -7,6 +7,7 @@ methods reach shared state and helpers through ``self``.
 
 import json
 import requests
+import threading
 import time
 from concurrent.futures import ThreadPoolExecutor
 from datetime import datetime
@@ -985,10 +986,14 @@ class _RegistryMixin:
 
     def get_registry_info(self, plugin_id: str) -> Optional[Dict]:
         """
-        Get plugin information from the registry cache only (no GitHub API calls).
+        Get plugin information from the registry (plugins.json).
 
-        Use this for lightweight lookups where only registry fields are needed
-        (e.g., verified status, latest_version).
+        Makes no GitHub API calls, but it does go through `fetch_registry`:
+        when the in-memory copy is missing or older than
+        ``registry_cache_timeout`` it downloads plugins.json, and with no
+        network that waits out the timeout and retries. A caller that must
+        not block on the network (the installed-plugins list) uses
+        `get_cached_registry_info` instead.
 
         Args:
             plugin_id: Plugin identifier
@@ -999,3 +1004,52 @@ class _RegistryMixin:
         registry = self.fetch_registry()
         plugins = registry.get('plugins', []) or []
         return self._match_registry_entry(plugins, plugin_id)
+
+    def get_cached_registry_info(self, plugin_id: str) -> Optional[Dict]:
+        """The registry entry for ``plugin_id`` from the copy already in
+        memory, however old; never touches the network.
+
+        None when no registry has been loaded yet, or the plugin isn't in it.
+        When the copy is missing or past ``registry_cache_timeout`` this
+        starts `refresh_registry_in_background`, so a later call has it.
+        """
+        cache = getattr(self, 'registry_cache', None)
+        cache_time = getattr(self, 'registry_cache_time', None)
+        if (not cache or not cache_time
+                or (time.time() - cache_time) >= self.registry_cache_timeout):
+            self.refresh_registry_in_background()
+        plugins = cache.get('plugins') if isinstance(cache, dict) else None
+        if not isinstance(plugins, list):
+            return None
+        return self._match_registry_entry(
+            [p for p in plugins if isinstance(p, dict)], plugin_id)
+
+    def refresh_registry_in_background(self) -> bool:
+        """Fetch the registry on a daemon thread; True when one was started.
+
+        At most one runs at a time. After a fetch that left no registry in
+        memory (offline), no new one starts for ``_failure_backoff_seconds``,
+        so an offline Pi doesn't retry on every page load.
+        """
+        with self._registry_refresh_lock:
+            running = self._registry_refresh_thread
+            if running is not None and running.is_alive():
+                return False
+            if time.time() < self._registry_refresh_retry_after:
+                return False
+            thread = threading.Thread(
+                target=self._background_registry_refresh,
+                name='registry-refresh', daemon=True)
+            self._registry_refresh_thread = thread
+            thread.start()
+            return True
+
+    def _background_registry_refresh(self) -> None:
+        try:
+            self.fetch_registry()
+        except Exception as e:  # noqa: BLE001 - a background warm-up must not crash
+            self.logger.warning("Background registry refresh failed: %s", e)
+        if not getattr(self, 'registry_cache', None):
+            with self._registry_refresh_lock:
+                self._registry_refresh_retry_after = (
+                    time.time() + self._failure_backoff_seconds)

src/plugin_system/store_update.py

@@ -10,6 +10,9 @@ import subprocess  # nosec B404 - list-form argv only, no shell  # nosemgrep
 from pathlib import Path
 from typing import Dict, Optional, Tuple
 from src.plugin_system.plugin_dirs import BACKUP_MARKER
+from src.plugin_system.plugin_local_files import (
+    KNOWN_STATE_PATTERNS, is_known_state_file,
+)
 from src.plugin_system.repo_urls import same_repo
 
 
@@ -302,7 +305,11 @@ class _UpdateMixin:
                 installed = False
 
             if installed:
-                self._discard_backup(plugin_id, backup_path, "update")
+                # install_plugin may land the new copy under the manifest id
+                # rather than the old directory name.
+                self._discard_backup(
+                    plugin_id, backup_path, "update",
+                    new_path=self._existing_install(plugin_id) or plugin_path)
                 return True
 
             # Bad network, registry error...: the user keeps a working plugin.
@@ -509,8 +516,12 @@ class _UpdateMixin:
                             for line in untracked_result.stdout.strip().split('\n'):
                                 if line.startswith('??'):
                                     # Untracked file
-                                    file_path = line[3:].strip()
-                                    untracked_files.append(file_path)
+                                    file_path = line[3:].strip().strip('"')
+                                    # Tokens and secrets stay out of the
+                                    # stash (see below), so they alone are
+                                    # not a reason to stash.
+                                    if not is_known_state_file(file_path):
+                                        untracked_files.append(file_path)
                         
                         # Check for tracked file changes
                         status_result = subprocess.run(
@@ -537,9 +548,17 @@ class _UpdateMixin:
                     if has_changes:
                         self.logger.info(f"Stashing local changes in {plugin_id} before update")
                         try:
-                            # Use -u to include untracked files in stash
+                            # Use -u to include untracked files in stash --
+                            # except the plugin's tokens and secrets, which a
+                            # repo may have forgotten to gitignore. The stash
+                            # is never popped, so a stashed token.pickle would
+                            # vanish from the plugin and break it.
+                            stash_cmd = (
+                                ['git', '-C', str(plugin_path), 'stash', 'push', '-u',
+                                 '-m', f'LEDMatrix auto-stash before update {plugin_id}', '--', '.']
+                                + [f':(exclude,glob)**/{p}' for p in KNOWN_STATE_PATTERNS])
                             stash_result = subprocess.run(
-                                ['git', '-C', str(plugin_path), 'stash', 'push', '-u', '-m', f'LEDMatrix auto-stash before update {plugin_id}'],
+                                stash_cmd,
                                 capture_output=True,
                                 text=True,
                                 timeout=30,

src/screen_runner.py

@@ -0,0 +1,484 @@
+"""Runs one screen: the ScreenRunner of docs/RUN_LOOP_REDESIGN.md.
+
+``ScreenRunner.run(plan, plugin)`` draws a screen's first frame, runs the
+frame loop its plan's ``frame_policy`` picks (125 Hz or 1 Hz), makes up the
+minimum duration when the loop ended early, and returns one
+:class:`Outcome` saying why the screen ended. Everything that touches the
+plugin, the panel or the controller's state goes through a
+:class:`ScreenHost` (the DisplayController); everything that reads or waits
+on the clock goes through an injected :class:`FrameClock`. The runner itself
+holds no state between screens.
+
+What can end a screen early is decided at the runner's service points: after
+each frame, after the frame loop, and after the make-up dwell. At each one
+the host gathers a snapshot and asks the Arbiter, once, whether a Source in
+``plan.preemptible_by`` now wants the panel (:meth:`ScreenHost.check`). A yes
+is ``ExitReason.PREEMPTED``: the next pass of the loop decides what shows,
+and the rotation does not advance past the screen that was cut short.
+
+The frame pacing is the loop that used to be inline in
+``DisplayController.run()``, unchanged: the 125 Hz loop paces to an 8 ms
+deadline from the start of each frame (sleeping at least 1 ms, so a frame
+that overran still yields the GIL), and the 1 Hz loop sleeps a flat second
+between frames, woken early by a control socket command.
+"""
+
+import logging
+from dataclasses import dataclass
+from enum import Enum
+from typing import Any, NamedTuple, Optional, Protocol, Tuple
+
+from src.display_arbiter import FramePolicy, ScreenPlan, Source
+
+__all__ = [
+    "AFTER_COMPLETED_LOOP",
+    "AFTER_LOOP",
+    "Checkpoint",
+    "DYNAMIC_GRACE",
+    "ExitReason",
+    "FINAL",
+    "FRAME",
+    "FirstFrame",
+    "FrameClock",
+    "HIGH_FPS_INTERVAL",
+    "NoticeRead",
+    "Outcome",
+    "STATIC_INTERVAL",
+    "Screen",
+    "ScreenHost",
+    "ScreenRunner",
+    "after_dwell",
+]
+
+#: Seconds between frames in the high-FPS loop (125 Hz), for scrolling plugins.
+HIGH_FPS_INTERVAL = 0.008
+
+#: Seconds between frames in the static loop (1 Hz).
+STATIC_INTERVAL = 1.0
+
+#: A dynamic-duration screen ends on cycle completion only this long after its
+#: minimum, so timing jitter around the minimum can't end it early.
+DYNAMIC_GRACE = 0.5
+
+
+class ExitReason(Enum):
+    """Why a screen ended. The value is the golden traces' exit column where
+    one exists (test/test_run_loop_golden.py)."""
+
+    #: The screen ran its target duration.
+    DURATION = "duration"
+    #: A dynamic-duration plugin finished its cycle after its minimum.
+    CYCLE_COMPLETE = "cycle-complete"
+    #: The first frame had nothing to show (display() returned False or
+    #: raised inside the executor), or no plugin draws the mode.
+    EMPTY = "empty"
+    #: The first frame's dispatch itself raised.
+    ERROR = "error"
+    #: A later frame returned False (a dynamic-duration screen on the 1 Hz
+    #: loop keeps going instead).
+    DISPLAY_FALSE = "display-false"
+    #: Another Source took the panel, or an on-demand session ran out before
+    #: the screen began. The rotation does not advance.
+    PREEMPTED = "preempted"
+    #: A plugin reload is waiting for the top of the loop. The screen is cut
+    #: short but counts as shown: the rotation advances, and the next pass
+    #: reloads before it draws.
+    RELOAD = "reload"
+
+
+@dataclass(frozen=True)
+class Outcome:
+    """How a screen ended.
+
+    Attributes:
+        exit_reason: Why it ended.
+        elapsed: Seconds from the end of the first frame to the end.
+        preempted_by: For PREEMPTED, the plan that took the panel when the
+            Arbiter named one (None when the session simply ran out).
+        on_demand_active: Filled in by the controller when the screen is
+            over: an on-demand session was running at that moment.
+        still_live: Filled in by the controller: the mode's plugin still had
+            live content at that moment, which holds the rotation on it.
+    """
+
+    exit_reason: ExitReason
+    elapsed: float = 0.0
+    preempted_by: Optional[ScreenPlan] = None
+    on_demand_active: bool = False
+    still_live: bool = False
+
+
+class FrameClock(Protocol):
+    """The clocks the runner reads and the sleep it paces with.
+
+    The shape of the ``time`` module, so production passes it (through an
+    indirection that lets tests patch the module) and the golden traces pass
+    their fake clock.
+    """
+
+    def time(self) -> float:
+        """Wall-clock seconds: what screen durations are measured in."""
+
+    def perf_counter(self) -> float:
+        """A monotonic high-resolution clock: what the 8 ms pacing reads."""
+
+    def sleep(self, seconds: float) -> None:
+        """Block for ``seconds``."""
+
+
+class NoticeRead(Enum):
+    """When a service point reads the WiFi notice file.
+
+    The read is throttled to once a second and deletes an expired file, so
+    *when* it happens is behaviour: each service point reads it exactly
+    when the loop always did.
+    """
+
+    #: Not at all.
+    NEVER = "never"
+    #: Only if nothing cheaper has already ended the screen: no on-demand
+    #: session, the panel on, the mode unchanged and no live takeover.
+    IF_UNDECIDED = "if-undecided"
+    #: Whenever no on-demand session is running.
+    ALWAYS = "always"
+
+
+@dataclass(frozen=True)
+class Checkpoint:
+    """What one kind of service point considers.
+
+    Attributes:
+        name: For logs and tests.
+        notice: When the WiFi notice is read (see NoticeRead).
+        notice_counts: Whether a pending notice ends the screen here. After
+            the make-up dwell it does only while the screen had time left.
+        reload: Whether a pending plugin reload ends the screen here. Only
+            between frames: once the frame loop is over the screen is too.
+    """
+
+    name: str
+    notice: NoticeRead
+    reload: bool
+    notice_counts: bool = True
+
+
+#: Between frames: the frame loops' check, and the socket wake in the 1 Hz wait.
+FRAME = Checkpoint("frame", NoticeRead.IF_UNDECIDED, reload=True)
+#: After a frame loop that ended early (display() returned False, a reload).
+AFTER_LOOP = Checkpoint("after-loop", NoticeRead.IF_UNDECIDED, reload=False)
+#: After a frame loop that ran its course: only a mode change or the schedule.
+AFTER_COMPLETED_LOOP = Checkpoint("after-completed-loop", NoticeRead.NEVER, reload=False)
+#: The last look before the rotation advances.
+FINAL = Checkpoint("final", NoticeRead.NEVER, reload=False)
+
+
+def after_dwell(time_left: bool) -> Checkpoint:
+    """After the make-up dwell: a notice that cut it short ends the screen,
+    so the mode resumes after the notice instead of rotating past it."""
+    return Checkpoint("after-dwell", NoticeRead.ALWAYS, reload=False,
+                      notice_counts=time_left)
+
+
+class FirstFrame(NamedTuple):
+    """What the first frame's dispatch returned (see _dispatch_first_frame)."""
+
+    shown: bool
+    raised: bool
+    accepts_display_mode: bool
+
+
+@dataclass
+class Screen:
+    """One running screen: its completed plan, the plugin drawing it and
+    when it started. Mutable only in that the runner owns it."""
+
+    plan: ScreenPlan
+    plugin: Any
+    accepts_display_mode: bool
+    start: float
+
+    @property
+    def mode(self) -> Optional[str]:
+        return self.plan.mode
+
+
+class ScreenHost(Protocol):
+    """The controller's side of a screen. See DisplayController."""
+
+    def first_frame(self, plan: ScreenPlan, plugin: Any) -> FirstFrame:
+        """Draw the first frame through the plugin executor."""
+
+    def complete_plan(self, plan: ScreenPlan, plugin: Any) -> Optional[ScreenPlan]:
+        """The plan with the plugin's durations, dynamic flag and frame
+        policy, read after the first frame. None when an on-demand session
+        has no time left for it."""
+
+    def draw(self, screen: Screen) -> Any:
+        """One later frame: what display() returned."""
+
+    def after_frame(self, screen: Screen) -> None:
+        """After a frame that did not end the screen (the follower frame)."""
+
+    def tick(self) -> None:
+        """Plugin updates that have come due (throttled)."""
+
+    def service(self, screen: Screen) -> Optional[Tuple[str, ...]]:
+        """Apply pending changes (on-demand requests, schedule, brightness,
+        finished reloads). Returns the live modes when a live-priority scan
+        was due, else None."""
+
+    def wait_frame(self, interval: float, screen: Screen) -> Optional[ScreenPlan]:
+        """The 1 Hz loop's sleep between frames. The plan that takes the
+        panel when a control socket command ended the screen, else None."""
+
+    def check(self, screen: Screen, checkpoint: Checkpoint,
+              live_scan: Optional[Tuple[str, ...]] = None) -> Optional[ScreenPlan]:
+        """The service point: the plan that now takes the panel from this
+        screen, or None while it holds. One Arbiter.decide() call."""
+
+    def dwell(self, seconds: float) -> None:
+        """Sleep up to ``seconds``, servicing changes; returns early on one."""
+
+    def cycle_complete(self, screen: Screen) -> bool:
+        """The plugin's dynamic-duration cycle is complete."""
+
+
+class ScreenRunner:
+    """Runs one screen at a time for a ScreenHost. See the module docstring."""
+
+    def __init__(self, clock: FrameClock, host: ScreenHost,
+                 log: Optional[logging.Logger] = None):
+        self.clock = clock
+        self.host = host
+        # The controller passes its own logger, so these lines keep the
+        # source they always had in the journal.
+        self.log = log or logging.getLogger(__name__)
+
+    # -- the screen ------------------------------------------------------
+
+    def run(self, plan: ScreenPlan, plugin: Any) -> Outcome:
+        """Run ``plan``'s screen, drawn by ``plugin`` (None: nothing draws it)."""
+        if plugin is None:
+            return Outcome(ExitReason.EMPTY)
+        first = self.host.first_frame(plan, plugin)
+        if not first.shown:
+            return Outcome(ExitReason.ERROR if first.raised else ExitReason.EMPTY)
+        completed = self.host.complete_plan(plan, plugin)
+        if completed is None:
+            return Outcome(ExitReason.PREEMPTED)
+        screen = Screen(completed, plugin, first.accepts_display_mode,
+                        start=self.clock.time())
+
+        if completed.frame_policy is FramePolicy.HIGH_FPS:
+            reason, by = self._high_fps_loop(screen)
+        else:
+            reason, by = self._static_loop(screen)
+        if reason is ExitReason.PREEMPTED:
+            # The service point that ended the loop has decided; looking
+            # again now, at the same instant, gives the same answer.
+            return self._outcome(screen, reason, by)
+
+        loop_completed = reason in (ExitReason.DURATION, ExitReason.CYCLE_COMPLETE)
+        # LOAD-BEARING: a change the frame loop did not end on (a dwell
+        # inside it, a later frame returning False) must not fall into the
+        # make-up dwell below. It can run for the rest of the screen's
+        # duration, and a freshly requested on-demand mode would sit
+        # invisible for that long -- or be clobbered by a queued stop.
+        by = self.host.check(screen, AFTER_COMPLETED_LOOP if loop_completed else AFTER_LOOP)
+        if by is not None:
+            return self._outcome(screen, ExitReason.PREEMPTED, by)
+
+        # Honour the minimum duration when a static, non-dynamic screen's
+        # loop ended early. A screen cut short for a plugin reload is over:
+        # the dwell returns at once and the rotation advances.
+        if (not completed.dynamic and not loop_completed
+                and completed.frame_policy is not FramePolicy.HIGH_FPS):
+            elapsed = self.clock.time() - screen.start
+            remaining = max(0.0, self._max(screen) - elapsed)
+            if remaining > 0:
+                self.host.dwell(remaining)
+                time_left = self.clock.time() - screen.start < self._max(screen)
+                by = self.host.check(screen, after_dwell(time_left))
+                if by is not None:
+                    return self._outcome(screen, ExitReason.PREEMPTED, by)
+
+        if completed.dynamic:
+            self._log_dynamic_end(screen)
+
+        # The dwells above return early when a pending change (on-demand
+        # started or stopped, the panel scheduled off) has already decided
+        # what comes next; rotating now would skip it.
+        by = self.host.check(screen, FINAL)
+        if by is not None:
+            return self._outcome(screen, ExitReason.PREEMPTED, by)
+        return self._outcome(screen, reason, None)
+
+    def _outcome(self, screen: Screen, reason: ExitReason,
+                 by: Optional[ScreenPlan]) -> Outcome:
+        return Outcome(reason, self.clock.time() - screen.start, preempted_by=by)
+
+    @staticmethod
+    def _max(screen: Screen) -> float:
+        return float(screen.plan.max_duration or 0.0)
+
+    @staticmethod
+    def _min(screen: Screen) -> float:
+        return float(screen.plan.min_duration or 0.0)
+
+    @staticmethod
+    def _ended_by(by: ScreenPlan) -> ExitReason:
+        return ExitReason.RELOAD if by.source is Source.RELOAD else ExitReason.PREEMPTED
+
+    # -- the frame loops ---------------------------------------------------
+
+    def _high_fps_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
+        """Ultra-smooth frames for scrolling plugins (8 ms = 125 FPS)."""
+        clock, host, log = self.clock, self.host, self.log
+        interval = HIGH_FPS_INTERVAL
+        log.debug("Entering high-FPS loop for %s with display_interval=%.3fs (%.1f FPS)",
+                  screen.mode, interval, 1.0 / interval)
+        target = self._max(screen)
+        while True:
+            frame_start = clock.perf_counter()
+            try:
+                result = host.draw(screen)
+                if isinstance(result, bool) and not result:
+                    log.debug("Display returned False, breaking early")
+                    return ExitReason.DISPLAY_FALSE, None
+            except Exception:  # pylint: disable=broad-except
+                log.exception("Error during display update")
+
+            # Multi-display sync: send follower frame after each render
+            host.after_frame(screen)
+            host.tick()
+            # Throttled: one clock compare between passes. A live-priority
+            # scan, when one is due, happens here, before the sleep, as it
+            # always has; the Arbiter weighs it after the sleep.
+            live_scan = host.service(screen)
+
+            # Pace to the frame deadline rather than sleeping a flat
+            # interval on top of the work. display() has already blocked on
+            # the panel's vsync by this point, so an unconditional sleep is
+            # added to a wait that already happened. Measured on a 2x128x64
+            # chain at limit_refresh_rate_hz=100: ~4ms of render plus a flat
+            # 8ms put each iteration at ~12ms against a 10ms refresh grid, so
+            # every swap missed a refresh and the loop settled at 50fps where
+            # display_interval asks for 125 -- and with zero headroom, ~14% of
+            # frames slipped a further refresh, which is what reads as scroll
+            # stutter.
+            remaining = interval - (clock.perf_counter() - frame_start)
+            # Yield even when the frame overran its budget, so plugin update
+            # threads and the web UI are not starved of the GIL.
+            clock.sleep(remaining if remaining > 0 else 0.001)
+
+            by = host.check(screen, FRAME, live_scan)
+            if by is not None:
+                log.debug("Mode changed during high-FPS loop, breaking early")
+                return self._ended_by(by), by
+
+            elapsed = clock.time() - screen.start
+            if elapsed >= target:
+                log.debug("Reached high-FPS target duration %.2fs for mode %s",
+                          target, screen.mode)
+                return ExitReason.DURATION, None
+            if self._should_exit_dynamic(screen, elapsed):
+                log.debug("Dynamic duration cycle complete for %s after %.2fs",
+                          screen.mode, elapsed)
+                return ExitReason.CYCLE_COMPLETE, None
+
+    def _static_loop(self, screen: Screen) -> Tuple[ExitReason, Optional[ScreenPlan]]:
+        """One frame a second for everything else."""
+        clock, host, log = self.clock, self.host, self.log
+        interval = STATIC_INTERVAL
+        log.debug("Entering normal FPS loop for %s with display_interval=%.3fs",
+                  screen.mode, interval)
+        target = self._max(screen)
+        dynamic = screen.plan.dynamic
+        while True:
+            # Wakes for a control socket command and applies it at once,
+            # instead of up to a second later.
+            by = host.wait_frame(interval, screen)
+            if by is not None:
+                log.info("Mode changed during display loop from %s to %s (%s), "
+                         "breaking early", screen.mode, by.mode, by.source.value)
+                return self._ended_by(by), by
+            host.tick()
+
+            elapsed = clock.time() - screen.start
+            if elapsed >= target:
+                log.debug("Reached standard target duration %.2fs for mode %s",
+                          target, screen.mode)
+                return ExitReason.DURATION, None
+
+            try:
+                result = host.draw(screen)
+                if isinstance(result, bool) and not result:
+                    # A dynamic-duration screen doesn't end on False: it
+                    # keeps looping until its cycle completes or its maximum.
+                    if not dynamic:
+                        log.info("Display returned False for %s (no dynamic duration), "
+                                 "breaking early", screen.mode)
+                        return ExitReason.DISPLAY_FALSE, None
+                    log.debug("Display returned False for %s (dynamic duration enabled), "
+                              "continuing loop", screen.mode)
+            except Exception:  # pylint: disable=broad-except
+                log.exception("Error during display update")
+
+            # Multi-display sync: send follower frame after each render
+            host.after_frame(screen)
+
+            live_scan = host.service(screen)
+            by = host.check(screen, FRAME, live_scan)
+            if by is not None:
+                log.info("Mode changed during display loop from %s to %s (%s), "
+                         "breaking early", screen.mode, by.mode, by.source.value)
+                return self._ended_by(by), by
+
+            if self._should_exit_dynamic(screen, elapsed):
+                log.info("Dynamic duration cycle complete for %s after %.2fs",
+                         screen.mode, elapsed)
+                return ExitReason.CYCLE_COMPLETE, None
+
+    # -- dynamic duration --------------------------------------------------
+
+    def _should_exit_dynamic(self, screen: Screen, elapsed: float) -> bool:
+        if not screen.plan.dynamic:
+            return False
+        minimum = self._min(screen)
+        # A small grace period after min_duration prevents premature exits
+        # due to timing issues.
+        if elapsed < minimum + DYNAMIC_GRACE:
+            self.log.debug(
+                "_should_exit_dynamic: elapsed %.2fs < min_duration %.2fs + grace %.2fs, "
+                "returning False", elapsed, minimum, DYNAMIC_GRACE)
+            return False
+        cycle_complete = self.host.cycle_complete(screen)
+        self.log.debug(
+            "_should_exit_dynamic: elapsed %.2fs >= min %.2fs, cycle_complete=%s, returning %s",
+            elapsed, minimum + DYNAMIC_GRACE, cycle_complete, cycle_complete)
+        if cycle_complete:
+            self.log.debug("Cycle complete detected for %s after %.2fs (min: %.2fs, grace: %.2fs)",
+                           screen.mode, elapsed, minimum, DYNAMIC_GRACE)
+        return cycle_complete
+
+    def _log_dynamic_end(self, screen: Screen) -> None:
+        """How a dynamic-duration screen ended, for the log. Asks the plugin
+        once more whether its cycle is complete, as the loop always did."""
+        elapsed_total = self.clock.time() - screen.start
+        cycle_done = self.host.cycle_complete(screen)
+        minimum, maximum = self._min(screen), self._max(screen)
+        if cycle_done:
+            self.log.info(
+                "Dynamic duration cycle completed for %s after %.2fs "
+                "(target: %.2fs, min: %.2fs, max: %.2fs)",
+                screen.mode, elapsed_total, maximum, minimum, maximum)
+        elif elapsed_total >= maximum:
+            self.log.info(
+                "Dynamic duration cap reached before cycle completion for %s "
+                "(%.2fs/%ds, min: %.2fs)",
+                screen.mode, elapsed_total, int(maximum), minimum)
+        else:
+            self.log.debug(
+                "Dynamic duration cycle in progress for %s: %.2fs elapsed "
+                "(target: %.2fs, min: %.2fs, max: %.2fs)",
+                screen.mode, elapsed_total, maximum, minimum, maximum)

src/startup_validator.py

@@ -95,11 +95,13 @@ class StartupValidator:
     def _validate_systemd_units(self) -> None:
         """Warn when an installed unit has drifted from the repo's template.
 
-        Nothing re-applies these after the first install. `git pull` -- which is
-        what the web UI's update button runs -- brings a new template into the
-        checkout, but nothing copies it to /etc/systemd/system and nothing runs
-        `systemctl daemon-reload`, so the unit that actually runs is whatever
-        first_time_install.sh wrote on day one.
+        Before updates refreshed units, nothing re-applied these after the
+        first install: `git pull` brought a new template into the checkout,
+        but nothing copied it to /etc/systemd/system, so the unit that
+        actually ran was whatever first_time_install.sh wrote on day one.
+        Updates now install changed units through the root helper
+        ledmatrix-refresh-units (web_interface/unit_refresh.py) -- but only on
+        a device whose installer granted it, so this still catches the rest.
 
         That makes every hardening added to a unit inert on existing installs.
         Measured on one rig: the installed unit was thirteen days older than the
@@ -141,10 +143,11 @@ class StartupValidator:
 
                 if self._unit_body(expected) != self._unit_body(actual):
                     self.warnings.append(
-                        f"{installed.name} differs from {template_rel}; the "
-                        "installed unit is not refreshed by an update, so "
+                        f"{installed.name} differs from {template_rel}, so "
                         "settings added to the template are not in effect. "
-                        "Re-run scripts/install/install_service.sh to apply them."
+                        "Updates apply them only once the installer has granted "
+                        "ledmatrix-refresh-units: re-run "
+                        "scripts/install/install_service.sh (or first_time_install.sh) to apply them."
                     )
         except OSError as e:
             self.logger.debug("Could not compare systemd units: %s", e)

src/vegas_mode/coordinator.py

@@ -18,10 +18,11 @@ import math
 import sys
 import time
 import threading
-from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
+from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
 
 from src import display_watchdog
 from src.common import render_gate
+from src.plugin_system.base_plugin import finite_seconds
 from src.vegas_mode.config import VegasModeConfig
 from src.vegas_mode.elements import LiveEpochs
 from src.vegas_mode.plugin_adapter import PluginAdapter
@@ -53,6 +54,14 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
 #: every plugin. Game state doesn't change within a quarter second.
 _LIVE_PRIORITY_CHECK_INTERVAL = 0.25
 
+#: Seconds a static pause shows a plugin whose display duration can't be
+#: used, as long as the rotation shows it: 30 when get_display_duration()
+#: raises or answers something that is not a number
+#: (DisplayController._get_display_duration), 15 when it answers a number at
+#: or below zero (DisplayController._resolve_durations).
+_UNREADABLE_DURATION = 30.0
+_NOT_POSITIVE_DURATION = 15.0
+
 
 def _percentile(ordered: List[float], fraction: float) -> float:
     """Nearest-rank percentile of an already-sorted list.
@@ -92,6 +101,9 @@ class VegasModeCoordinator:
     _live_reason: Optional[str] = None
     # Set only while Vegas has changed the GIL switch interval; read with getattr.
     _saved_switch_interval: Optional[float]
+    #: Plugins already warned about a display duration the pause can't use,
+    #: so a bad setting logs once, not at every turn. Replaced, not mutated.
+    _duration_warned: FrozenSet[str] = frozenset()
 
     def __init__(
         self,
@@ -1010,7 +1022,7 @@ class VegasModeCoordinator:
             # Wait for the plugin's display duration. Monotonic, like the
             # iteration clock: an NTP step on an RTC-less Pi would otherwise
             # end the pause at once or stretch it by the correction.
-            duration = plugin.get_display_duration()
+            duration = self._static_pause_duration(plugin)
             start = time.monotonic()
 
             while time.monotonic() - start < duration:
@@ -1046,6 +1058,42 @@ class VegasModeCoordinator:
 
         return True
 
+    def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
+        """Seconds a static pause shows ``plugin``: its display duration,
+        read the way the rotation reads it.
+
+        Several plugins return their display_duration setting straight from
+        config.json, so one saved as "20" or null came back as a string or
+        None; comparing it with the clock raised, and the pause's broad
+        except ended the pause at every one of the plugin's turns. inf
+        paused until something interrupted it, and NaN, False, 0 or a
+        negative number ended the pause at once. A numeric string counts
+        (finite_seconds); anything else, or a raise, gets
+        _UNREADABLE_DURATION, and a number at or below zero
+        _NOT_POSITIVE_DURATION, logged once per plugin.
+        """
+        try:
+            value = plugin.get_display_duration()
+        except Exception as err:  # pylint: disable=broad-except
+            problem = f"get_display_duration() raised {type(err).__name__}: {err}"
+            fallback = _UNREADABLE_DURATION
+        else:
+            seconds = finite_seconds(value)
+            if seconds is not None and seconds > 0:
+                return seconds
+            if seconds is None:
+                problem = f"display duration {value!r} is not a number"
+                fallback = _UNREADABLE_DURATION
+            else:
+                problem = f"display duration {value!r} is not above zero"
+                fallback = _NOT_POSITIVE_DURATION
+        plugin_id = plugin.plugin_id
+        if plugin_id not in self._duration_warned:
+            self._duration_warned = self._duration_warned | {plugin_id}
+            logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
+                           plugin_id, problem, fallback)
+        return fallback
+
     def _end_static_pause(self) -> None:
         """End static pause and restore scroll state."""
         should_resume_scrolling = False

systemd/ledmatrix-web.service

@@ -19,6 +19,11 @@ Type=simple
 User=__USER__
 WorkingDirectory=__PROJECT_ROOT_DIR__
 Environment=USE_THREADING=1
+# Cap glibc's malloc arenas, as ledmatrix.service does: each allocating thread
+# can get its own arena, up to 8 x CPU count (24 on a 3-core Pi), and a grown
+# arena is never handed back to the OS. This threaded Flask process would hold
+# that memory the same way. See ledmatrix.service for the measurement.
+Environment=MALLOC_ARENA_MAX=2
 ExecStart=/usr/bin/python3 __PROJECT_ROOT_DIR__/scripts/utils/start_web_conditionally.py
 Restart=on-failure
 RestartSec=10

test/_run_loop_harness.py

@@ -143,12 +143,24 @@ class FakeCache:
     def __init__(self):
         self.data: Dict[str, Any] = {}
         self.cache_dir = "/nonexistent/run-loop-harness"
+        self._writes = 0
+        self._written: Dict[str, int] = {}
 
     def get(self, key, max_age=None, memory_ttl=None):
         return self.data.get(key)
 
     def set(self, key, data, ttl=None):
         self.data[key] = data
+        # Every write is a new file, as DiskCache's rename makes it.
+        self._writes += 1
+        self._written[key] = self._writes
+
+    def file_signature(self, key):
+        """CacheManager.file_signature: None without a file, else a value
+        that changes with every write."""
+        if key not in self.data:
+            return None
+        return (self._written.get(key, 0), 0, 0)
 
     def delete(self, key):
         self.data.pop(key, None)
@@ -724,11 +736,14 @@ class RunLoopHarness:
         self.clock.at(t, post)
 
     def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
-                          duration: Optional[float] = None, pinned: bool = False):
+                          duration: Optional[float] = None, pinned: bool = False,
+                          named_mode: Optional[str] = None):
         """Start with an on-demand session resumed from the cache, as after
         a restart: the state _select_startup_plugins restores, then
-        _populate_on_demand_modes_from_plugin, as __init__ calls it."""
+        _populate_on_demand_modes_from_plugin, as __init__ calls it. A
+        session that cannot resume is logged as ``on-demand-error``."""
         dc = self.controller
+        dc._on_demand_named_mode = named_mode
         dc.on_demand_active = True
         dc.on_demand_plugin_id = plugin_id
         dc.on_demand_mode = mode
@@ -739,6 +754,8 @@ class RunLoopHarness:
         dc.on_demand_status = 'active'
         dc.on_demand_schedule_override = True
         dc._populate_on_demand_modes_from_plugin()
+        if dc.on_demand_status == 'error':
+            self.log("on-demand-error", dc.on_demand_last_error)
 
     def wifi_message(self, t: float, message: str, duration: float = 5):
         def write():

test/conftest.py

@@ -328,6 +328,21 @@ def _hermetic_control_socket(monkeypatch):
     monkeypatch.setenv(SOCKET_PATH_ENV, 'off')
 
 
+@pytest.fixture(autouse=True)
+def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
+    """Keep updates' systemd unit refresh off the host.
+
+    perform_core_update runs web_interface/unit_refresh.py after any update
+    that moves HEAD, and several tests run the real one against a test clone.
+    On a device -- or a machine where install_service.sh was tried out -- it
+    would compare the clone's templates with the real /etc/systemd/system and
+    run the real sudo helper. Point it at a folder that does not exist: no units
+    installed, nothing to do. The unit refresh tests pass their own.
+    """
+    from web_interface import unit_refresh
+    monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
+
+
 @pytest.fixture(autouse=True)
 def reset_logging():
     """Reset logging configuration before each test."""

test/fixtures/run_loop_golden/on_demand_named_live.json

@@ -0,0 +1,29 @@
+{
+  "screens": [
+    [0.0, "clock", 5.0, "on-demand-start", 6, false],
+    [5.0, "sports_live", 15.0, "duration", 15, true],
+    [20.0, "sports_recent", 15.0, "duration", 15, true],
+    [35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
+    [40.0, "clock", 20.0, "duration", 20, true],
+    [60.0, "sports_live", 15.0, "display-false", 11, true],
+    [75.0, "sports_recent", 15.0, "duration", 15, true],
+    [90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
+    [100.0, "sports_live", 0.0, "empty", 1, true],
+    [100.0, "sports_recent", 15.0, "duration", 15, true],
+    [115.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [130.0, "sports_live", 0.0, "empty", 1, true],
+    [130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
+    [140.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [155.0, "clock", 5.0, "horizon", 5, true]
+  ],
+  "events": [
+    [5.0, "request", "start:n1"],
+    [5.0, "on-demand-start", "sports"],
+    [40.0, "request", "stop:n2"],
+    [40.0, "on-demand-requested-stop"],
+    [100.0, "request", "start:n3"],
+    [100.0, "on-demand-start", "sports"],
+    [140.0, "request", "stop:n4"],
+    [140.0, "on-demand-requested-stop"]
+  ]
+}

test/fixtures/run_loop_golden/on_demand_restore_failed.json

@@ -0,0 +1,10 @@
+{
+  "screens": [
+    [0.0, "clock", 20.0, "duration", 20, false],
+    [20.0, "weather", 20.0, "duration", 20, true],
+    [40.0, "clock", 20.0, "horizon", 20, true]
+  ],
+  "events": [
+    [0.0, "on-demand-error", "restore-failed"]
+  ]
+}

test/js/README.md

@@ -45,7 +45,12 @@ server has none.
 |---|---|---|
 | `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
 | `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin, never re-sends one that got any HTTP answer (the real `api_client.js` classifies a proxy 502 or a JSON error without `error_code` as `API_ERROR`), and counts a no-op update as already up to date in the summary. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
+| `unit/test_store_install.js` | no | The store's Install button, with the whole of `plugins_manager.js` run by `plugins_manager_sandbox.js` (a vm context, fake DOM and API): a fresh install reloads the list, then enables the id the plugin was installed as -- the answer's `plugin_id`, else the installed entry the store entry matches (Weather installs as `ledmatrix-weather`); a Reinstall leaves the enabled state alone |
+| `unit/test_install_polling.js` | no | How long Install waits for a queued install (sandbox): at least the server's 300 s dependency-install timeout; when it stops waiting it reloads the installed list and warns, rather than reporting a failure or enabling anything |
+| `unit/test_store_categories.js` | no | The store's category filter (sandbox): the template ships only All Categories, the rest come from the store's plugins (one per category whatever its case), choosing one filters to it, and a swapped-in select is refilled from the cache keeping the choice |
+| `unit/test_github_url_install.js` | no | Install Single Plugin (sandbox, the button as `plugins.html` ships it): no inline `onclick`, so a click or Enter sends exactly one `install-from-url` request and raises no error |
 | `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
+| `unit/test_plugin_order_list.js` | no | `widgets/plugin-order-list.js` (the Vegas and rotation order lists): a disabled plugin, which gets no row, keeps its slot in the saved order and its Vegas exclusion when the list rewrites its hidden inputs, around reordering and include/exclude; an uninstalled plugin's id is dropped, a failed plugin list leaves the inputs as saved, and only string ids are carried over, once each |
 | `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
 | `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
 | `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
@@ -53,6 +58,9 @@ server has none.
 | `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
 | `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
 | `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
+| `unit/test_overview_reconciliation_poll.js` | no | The Overview's reconciliation-banner poll from `partials/overview.html`, run in a vm: it gives up after a bounded number of requests when the status never says done, runs only while the Overview is on screen (`LEDVisibility`, its own key), and stops once the banner is shown |
+| `unit/test_display_partial_ids.js` | no | `js/pages/display.js` started on a fake root that answers only for the ids `partials/display.html` renders: every id it looks up (with every listener and timer it set fired) exists, and moving the brightness slider updates its label without throwing |
+| `unit/test_general_web_login_token.js` | no | `createToken` from `js/pages/general.js`, imported with a fake DOM and fetch: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
 | `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
 | `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
 | `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
@@ -61,6 +69,10 @@ server has none.
 | `dom/test_durations_page.js` | yes | The Rotation tab (`js/pages/durations.js`) with the real `plugin-order-list.js` widget: one plugin-list request per swap, one move per click after repeated swaps, a swap cancels the request in flight, a late widget is waited for |
 | `dom/test_operation_history_page.js` | yes | The Operation History tab (`js/pages/operation-history.js`): one request per swap and per Refresh, the plugin filter filled once, paging, filters, search, Clear, error/login states, hostile values stay text |
 | `dom/test_raw_json_page.js` | yes | The Config Editor tab (`js/pages/raw-json.js`): one POST per Save after repeated swaps, Format/Validate, invalid JSON never sent, a save survives a swap, the old global entry points |
+| `dom/test_schedule_page.js` | yes | The Schedule tab (`js/pages/schedule.js`) with the real `schedule-picker` widget: both pickers drawn once per swap from the saved config, one notification per save answer after repeated swaps, the brightness label, a late widget waited for, the old global entry points |
+| `dom/test_visibility_service.js` | yes (no server) | `js/core/visibility.js` with the real `LEDVisibility` from `app-shell.js` and the real registry: start/stop with the active tab and the browser tab's visibility, no interval while hidden or after a swap-out, registrations independent, the no-`LEDVisibility` fallback |
+| `dom/test_display_page.js` | yes | The Display tab (`js/pages/display.js`) with the real `plugin-order-list` widget and `LEDVisibility`: one page, one sync interval and one action per control after repeated swaps, the sync poll only while on screen and never after a swap-out, sync states as text, the debounced scroll-speed hint, `updateSyncUI`'s entry point |
+| `dom/test_general_page.js` | yes | The General tab (`js/pages/general.js`) with the real `timezone-selector` widget: the picker drawn once per swap, one request per Security action after repeated swaps, hostile token names stay text, refused/network/login answers, a write survives a swap, `webLogin`'s entry points |
 | `dom/test_backup_restore_page.js` | yes | The Backup & Restore tab (`js/pages/backup-restore.js`): one request per action after repeated swaps, the upload and restore options, reads cancelled and writes not on a swap, hostile names stay text, the old global entry points |
 | `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
 

test/js/dom/test_display_page.js

@@ -0,0 +1,325 @@
+// The Display tab as a page module (static/v3/js/pages/display.js), in a
+// real DOM (jsdom) with the real server-rendered partial, the real
+// plugin-order-list widget, the real window.LEDVisibility (app-shell.js)
+// behind ctx.visibility, and the real API's answer shapes. Built like
+// test_cache_page.js:
+//
+//   * the partial ships no <script> and no inline handlers; its root is
+//     data-page="display"
+//   * after five swaps: one mounted page, the Vegas order drawn once, each
+//     control acting once (brightness, resolution, Vegas and double-sided
+//     toggles, the Advanced section toggle, one debounced scroll-speed hint
+//     request)
+//   * the sync status is polled only while the Display tab is on screen and
+//     the browser tab visible: no interval runs while hidden, and none after
+//     the partial is swapped out
+//   * sync states drawn as text; a failed poll says "unavailable", a login
+//     redirect draws nothing
+//   * a widget that loads late is waited for, and a page swapped away while
+//     waiting starts nothing
+//   * window.updateSyncUI's entry point still works
+const http = require('http');
+const fs = require('fs');
+const path = require('path');
+const { pathToFileURL } = require('url');
+const { JSDOM, VirtualConsole } = require('jsdom');
+
+const BASE = process.env.BASE || 'http://localhost:5000';
+const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
+const get = p => new Promise((res, rej) =>
+  http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
+const load = f => import(pathToFileURL(path.join(JS, f)).href);
+const tick = ms => new Promise(r => setTimeout(r, ms || 0));
+
+let pass = 0, fail = 0;
+const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
+  : (fail++, console.log('  FAIL ' + l + (x !== undefined ? '  -> ' + JSON.stringify(x).slice(0, 300) : '')));
+
+(async () => {
+  const partial = await get('/partials/display');
+  const realSync = JSON.parse(await get('/api/v3/sync/status'));
+  const smooth = JSON.parse(await get('/api/v3/config/scroll-speed-advice?speed=50&min=1&max=200'));
+  const rough = JSON.parse(await get('/api/v3/config/scroll-speed-advice?speed=37&min=1&max=200'));
+  const { createRegistry } = await load('core/registry.js');
+  const { createApi } = await load('core/api.js');
+  const { createVisibility } = await load('core/visibility.js');
+  const displayPage = await load('pages/display.js');
+
+  console.log('\n── Display tab: page module (real DOM) ──');
+  ok('the partial ships no inline script', !/<script/i.test(partial));
+  ok('the partial has no inline handlers', !/\son(click|change|input)=/i.test(partial));
+  ok('the partial root is data-page="display"', /data-page="display"/.test(partial));
+  ok('the Advanced toggle names its action',
+     /data-action="toggle-section"\s+data-section="display-section-advanced-hardware"/.test(partial));
+  ok('the real sync status answers in the shape the page reads',
+     realSync.status === 'success' && realSync.data && typeof realSync.data.state === 'string', realSync);
+  ok('the real scroll-speed advice answers in the shape the page reads',
+     smooth.status === 'success' && smooth.data.applied && Array.isArray(rough.data.alternatives)
+       && rough.data.alternatives.length > 0, rough);
+
+  const errs = [];
+  const logged = [];
+  const vc = new VirtualConsole();
+  vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
+  vc.on('error', (...a) => logged.push(a.join(' ')));
+  const dom = new JSDOM(`<!doctype html><html><body><div id="display-content">${partial}</div></body></html>`,
+    { url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
+  const { window } = dom;
+  const doc = window.document;
+  const panel = doc.getElementById('display-content');
+
+  // The browser tab's visibility and the app's active tab, under test control.
+  let hidden = false;
+  Object.defineProperty(doc, 'hidden', { get: () => hidden, configurable: true });
+  const setHidden = v => { hidden = v; doc.dispatchEvent(new window.Event('visibilitychange')); };
+  const setTab = tab => doc.dispatchEvent(new window.CustomEvent('ledmatrix:tab-changed', { detail: { tab } }));
+  // Intervals, counted. Timeouts (the hint's debounce, the widget retry) are real.
+  const intervals = new Map();
+  let nextInterval = 1;
+  window.setInterval = (fn, ms) => { const id = nextInterval++; intervals.set(id, { fn, ms }); return id; };
+  window.clearInterval = id => { intervals.delete(id); };
+  const fireIntervals = () => [...intervals.values()].forEach(i => i.fn());
+
+  const HOSTILE = '<img src=x onerror="window.pwned=1">';
+  const plugins = [
+    { id: 'clock', name: 'Clock', enabled: true },
+    { id: 'weather', name: HOSTILE, enabled: true },
+  ];
+  let syncAnswer = { status: 'success', data: { role: 'leader', state: 'no_peer' } };
+  let syncMode = 'ok';
+  let advice = smooth;
+  const requests = [];
+  function fakeFetch(url, init = {}) {
+    requests.push(url);
+    const respond = (status, body, headers) => Promise.resolve({
+      status, ok: status >= 200 && status < 300,
+      headers: { get: h => (headers || {})[h] || null },
+      json: () => Promise.resolve(body),
+      text: () => Promise.resolve(JSON.stringify(body)),
+    });
+    if (url === '/api/v3/plugins/installed') return respond(200, { status: 'success', data: { plugins } });
+    if (url.startsWith('/api/v3/config/scroll-speed-advice?')) return respond(200, advice);
+    if (url === '/api/v3/sync/status') {
+      if (syncMode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
+      if (syncMode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
+      return respond(200, syncAnswer);
+    }
+    return respond(404, { status: 'error', message: 'unexpected ' + url });
+  }
+  window.fetch = fakeFetch;
+
+  // The shell: LEDVisibility (no Alpine here, so the active tab is the last
+  // ledmatrix:tab-changed; the SSE streams open stand-in EventSources), and
+  // the shared toggleSection the Advanced button calls.
+  window.getApp = () => null;
+  window.EventSource = class { addEventListener() {} removeEventListener() {} close() {} };
+  window.eval(fs.readFileSync(path.join(JS, 'app-shell.js'), 'utf8'));
+  const toggled = [];
+  window.toggleSection = id => toggled.push(id);
+  window.eval(fs.readFileSync(path.join(JS, 'widgets/plugin-order-list.js'), 'utf8'));
+  const widget = window.PluginOrderList;
+  ok('the widget script defines PluginOrderList', !!(widget && widget.init));
+
+  const visibility = createVisibility({ window });
+  const registry = createRegistry({
+    document: doc,
+    context: { api: createApi({ fetch: fakeFetch }), notify: () => {} },
+    mountContext: ctx => ({ visibility: visibility.forPage(ctx) }),
+  });
+  registry.register('display', displayPage);
+
+  const $ = id => doc.getElementById(id);
+  const root = () => doc.querySelector('[data-page="display"]');
+  const count = prefix => requests.filter(u => u.startsWith(prefix)).length;
+  const syncPolls = () => count('/api/v3/sync/status');
+  async function swap(html) {
+    panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
+    panel.innerHTML = html === undefined ? partial : html;
+    panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
+    await tick(20);
+  }
+  function fire(el, type) { el.dispatchEvent(new window.Event(type, { bubbles: true })); }
+  function setRole(role) { $('sync_role').value = role; fire($('sync_role'), 'change'); }
+
+  setTab('display');
+  await registry.start();
+  await tick(200);
+
+  // ── first load ──────────────────────────────────────────────────────────
+  ok('one plugin-list request on start', count('/api/v3/plugins/installed') === 1, requests);
+  ok('one scroll-speed hint request on start (after the debounce)', count('/api/v3/config/scroll-speed-advice') === 1, requests);
+  ok('the saved role is standalone: no sync request, no interval work',
+     $('sync_role').value === 'standalone' && syncPolls() === 0, [$('sync_role').value, syncPolls()]);
+  ok('the sync poll interval runs while the tab is on screen', intervals.size === 1
+     && [...intervals.values()][0].ms === 5000, intervals.size);
+  ok('the status bar is hidden for standalone', $('sync_status_bar').classList.contains('hidden'));
+
+  // ── five swaps ──────────────────────────────────────────────────────────
+  for (let i = 0; i < 5; i++) await swap();
+  await tick(200);
+  ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
+  ok('one sync interval, not six', intervals.size === 1, intervals.size);
+  ok('one plugin-list request per swap', count('/api/v3/plugins/installed') === 6, count('/api/v3/plugins/installed'));
+  ok('the Vegas order drawn once, not stacked', doc.querySelectorAll('#vegas_plugin_order .plugin-order-item').length === 2,
+     doc.querySelectorAll('#vegas_plugin_order .plugin-order-item').length);
+  ok('a hostile plugin name is shown as text', $('vegas_plugin_order').textContent.includes(HOSTILE)
+     && !doc.querySelector('#vegas_plugin_order img') && !window.pwned);
+
+  // ── the controls ────────────────────────────────────────────────────────
+  $('brightness').value = '42';
+  fire($('brightness'), 'input');
+  ok('the brightness value follows the slider', $('brightness-value').textContent === '42', $('brightness-value').textContent);
+
+  $('rows').value = '32'; $('cols').value = '64'; $('chain_length').value = '3'; $('parallel').value = '2';
+  fire($('parallel'), 'input');
+  ok('the resolution readout is cols x chain by rows x parallel',
+     $('display-resolution-value').textContent === '192 × 64 pixels', $('display-resolution-value').textContent);
+  $('orientation').value = '90';
+  fire($('orientation'), 'change');
+  ok('...swapped for a 90-degree orientation', $('display-resolution-value').textContent === '64 × 192 pixels',
+     $('display-resolution-value').textContent);
+  $('rows').value = '';
+  fire($('rows'), 'input');
+  ok('...and a dash while a field is empty', $('display-resolution-value').textContent === '—');
+
+  for (const [box, settings, shown] of [['vegas_scroll_enabled', 'vegas_scroll_settings', 'block'],
+                                         ['double_sided_enabled', 'double_sided_settings', 'grid']]) {
+    $(box).checked = true; fire($(box), 'change');
+    const on = $(settings).style.display;
+    $(box).checked = false; fire($(box), 'change');
+    ok(`${box} shows and hides its settings`, on === shown && $(settings).style.display === 'none',
+       [on, $(settings).style.display]);
+  }
+
+  root().querySelector('[data-action="toggle-section"]').click();
+  ok('the Advanced button toggles its section once', toggled.join() === 'display-section-advanced-hardware', toggled);
+
+  // ── the scroll-speed hint ───────────────────────────────────────────────
+  const hints = count('/api/v3/config/scroll-speed-advice');
+  advice = rough;
+  for (const v of ['36', '37', '38']) { $('vegas_scroll_speed').value = v; fire($('vegas_scroll_speed'), 'input'); }
+  ok('the speed value follows the slider', $('vegas_scroll_speed_value').textContent === '38');
+  await tick(250);
+  ok('three quick moves make one hint request', count('/api/v3/config/scroll-speed-advice') === hints + 1,
+     count('/api/v3/config/scroll-speed-advice') - hints);
+  ok('...for the last speed', requests.filter(u => u.includes('advice')).pop().includes('speed=38'));
+  const buttons = $('vegas_scroll_speed_hint').querySelectorAll('button');
+  ok('a rough speed offers the smooth ones', buttons.length === rough.data.alternatives.length
+     && /will run as/.test($('vegas_scroll_speed_hint').textContent), $('vegas_scroll_speed_hint').textContent);
+  buttons[0].click();
+  ok('picking one sets the slider', $('vegas_scroll_speed').value === String(Math.round(rough.data.alternatives[0].pixels_per_second))
+     && $('vegas_scroll_speed_value').textContent === $('vegas_scroll_speed').value, $('vegas_scroll_speed').value);
+  advice = smooth;
+  await tick(250);
+  ok('...and asks again', count('/api/v3/config/scroll-speed-advice') === hints + 2);
+  ok('a smooth speed says so', /^Smooth on this panel/.test($('vegas_scroll_speed_hint').textContent),
+     $('vegas_scroll_speed_hint').textContent);
+
+  // ── sync: the role ──────────────────────────────────────────────────────
+  setRole('leader');
+  await tick(20);
+  ok('choosing Leader shows the status bar', !$('sync_status_bar').classList.contains('hidden'));
+  ok('...hides Position', $('setting-display-sync_follower_position').style.display === 'none');
+  ok('...and asks for the status once', syncPolls() === 1, syncPolls());
+  ok('...drawn as text', $('sync_status_content').textContent.includes('No follower detected'),
+     $('sync_status_content').textContent);
+  setRole('follower');
+  await tick(20);
+  ok('choosing Follower shows Position', $('setting-display-sync_follower_position').style.display === '');
+
+  // ── sync: the poll runs only while on screen ────────────────────────────
+  let polls = syncPolls();
+  fireIntervals();
+  await tick(20);
+  ok('each interval tick polls once', syncPolls() === polls + 1, syncPolls() - polls);
+  setTab('logs');
+  ok('switching to another tab clears the interval', intervals.size === 0, intervals.size);
+  polls = syncPolls();
+  setTab('display');
+  await tick(20);
+  ok('switching back polls at once', syncPolls() === polls + 1 && intervals.size === 1, [syncPolls() - polls, intervals.size]);
+  setHidden(true);
+  ok('hiding the browser tab clears the interval', intervals.size === 0, intervals.size);
+  polls = syncPolls();
+  setHidden(false);
+  await tick(20);
+  ok('showing it polls at once', syncPolls() === polls + 1 && intervals.size === 1, [syncPolls() - polls, intervals.size]);
+
+  // ── sync: states ────────────────────────────────────────────────────────
+  async function poll() { fireIntervals(); await tick(20); return $('sync_status_content').textContent; }
+  syncAnswer = { status: 'success', data: { role: 'leader', state: 'connected', peer_ip: HOSTILE, peer_chain: 2 } };
+  ok('a connected follower, its address as text', (await poll()).includes('Follower connected — ' + HOSTILE)
+     && !$('sync_status_content').querySelector('img'), $('sync_status_content').textContent);
+  syncAnswer = { status: 'success', data: { role: 'leader', state: 'incompatible', error: 'rows differ ' + HOSTILE } };
+  ok('incompatible panels show the reason as text', (await poll()).includes('incompatible')
+     && !$('sync_error_detail').classList.contains('hidden') && $('sync_error_text').textContent === 'rows differ ' + HOSTILE
+     && !$('sync_error_detail').querySelector('img'));
+  syncAnswer = { status: 'success', data: realSync.data };
+  ok('the real server\'s answer is drawn', (await poll()).length > 0, $('sync_status_content').textContent);
+  syncMode = 'network';
+  ok('a failed poll says unavailable', (await poll()).includes('Sync status unavailable'));
+  syncAnswer = { status: 'success', data: { role: 'follower', state: 'follower', leader_ip: '10.0.0.2' } };
+  syncMode = 'ok';
+  ok('receiving from a leader', (await poll()).includes('Receiving from leader — 10.0.0.2'));
+  syncMode = 'login';
+  ok('a login redirect draws nothing', (await poll()).includes('Receiving from leader'));
+  syncMode = 'ok';
+
+  // ── standalone stops asking ─────────────────────────────────────────────
+  setRole('standalone');
+  polls = syncPolls();
+  fireIntervals();
+  await tick(20);
+  ok('standalone hides the bar and the poll asks nothing',
+     $('sync_status_bar').classList.contains('hidden') && syncPolls() === polls, syncPolls() - polls);
+
+  // ── window.updateSyncUI ─────────────────────────────────────────────────
+  $('sync_role').value = 'leader';
+  polls = syncPolls();
+  displayPage.updateSyncUI();
+  await tick(20);
+  ok('updateSyncUI() applies the role and asks once',
+     !$('sync_status_bar').classList.contains('hidden') && syncPolls() === polls + 1, syncPolls() - polls);
+
+  // ── swapped out ─────────────────────────────────────────────────────────
+  await swap('<p>another tab</p>');
+  ok('nothing left mounted', registry.list().length === 0, registry.list().length);
+  ok('no interval left running', intervals.size === 0, intervals.size);
+  polls = syncPolls();
+  setTab('overview');
+  setTab('display');
+  setHidden(true);
+  setHidden(false);
+  await tick(20);
+  ok('a swapped-out page never polls again', syncPolls() === polls && intervals.size === 0, syncPolls() - polls);
+  const hintsGone = count('/api/v3/config/scroll-speed-advice');
+  await swap();
+  $('vegas_scroll_speed').value = '40';
+  fire($('vegas_scroll_speed'), 'input');
+  await swap('<p>another tab</p>');
+  await tick(250);
+  ok('a hint still waiting out its debounce at the swap is never asked for',
+     count('/api/v3/config/scroll-speed-advice') === hintsGone, count('/api/v3/config/scroll-speed-advice') - hintsGone);
+
+  // ── the widget loads late ───────────────────────────────────────────────
+  delete window.PluginOrderList;
+  const beforeLate = count('/api/v3/plugins/installed');
+  await swap();
+  ok('no list request while the widget is missing', count('/api/v3/plugins/installed') === beforeLate);
+  window.PluginOrderList = widget;
+  await tick(150);
+  ok('the list starts once the widget arrives', count('/api/v3/plugins/installed') === beforeLate + 1);
+  delete window.PluginOrderList;
+  await swap();
+  const beforeGone = count('/api/v3/plugins/installed');
+  await swap('<p>another tab</p>');
+  window.PluginOrderList = widget;
+  await tick(250);
+  ok('a page swapped away while waiting starts nothing', count('/api/v3/plugins/installed') === beforeGone);
+
+  ok('no console errors', logged.length === 0, logged);
+  ok('no DOM errors', errs.length === 0, errs);
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/js/dom/test_durations_page.js

@@ -98,7 +98,11 @@ const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
 
   const lists = () => requests.filter(r => r.url === '/api/v3/plugins/installed').length;
   const $ = id => doc.getElementById(id);
-  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]');
+  // The rows' ids, in order. The input also keeps saved ids that have no row
+  // (a disabled plugin's place, see test/js/unit/test_plugin_order_list.js),
+  // and the saved order comes from whatever config the server has.
+  const SHOWN = plugins.filter(p => p.enabled).map(p => p.id);
+  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]').filter(id => SHOWN.includes(id));
   async function swap() {
     panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
     panel.innerHTML = partial;

test/js/dom/test_general_page.js

@@ -0,0 +1,272 @@
+// The General tab as a page module (static/v3/js/pages/general.js), in a real
+// DOM (jsdom) with the real server-rendered partial, the real timezone
+// widget and the real web-login endpoints' answer shapes. Built like
+// test_cache_page.js:
+//
+//   * the partial ships no <script> and no inline handlers; its root is
+//     data-page="general" and the Security section's forms and buttons name
+//     an action
+//   * the timezone picker is drawn once per swap-in, with the saved zone
+//   * after five swaps, each Security action makes exactly one request
+//   * a login change is a write: a swap does not cancel it, its result is
+//     still reported, and nothing is drawn into the page that has gone
+//   * token names reach the page as text
+//   * the settings form itself is left to htmx
+//   * window.webLogin's entry points still work
+const http = require('http');
+const fs = require('fs');
+const path = require('path');
+const { pathToFileURL } = require('url');
+const { JSDOM, VirtualConsole } = require('jsdom');
+
+const BASE = process.env.BASE || 'http://localhost:5000';
+const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
+const get = p => new Promise((res, rej) =>
+  http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
+const load = f => import(pathToFileURL(path.join(JS, f)).href);
+const tick = ms => new Promise(r => setTimeout(r, ms || 0));
+
+let pass = 0, fail = 0;
+const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
+  : (fail++, console.log('  FAIL ' + l + (x !== undefined ? '  -> ' + JSON.stringify(x).slice(0, 300) : '')));
+
+(async () => {
+  const partial = await get('/partials/general');
+  const realTokens = JSON.parse(await get('/api/v3/auth/tokens'));
+  const { createRegistry } = await load('core/registry.js');
+  const { createApi } = await load('core/api.js');
+  const generalPage = await load('pages/general.js');
+
+  console.log('\n── General tab: page module (real DOM) ──');
+  ok('the partial ships no inline script', !/<script/i.test(partial));
+  ok('the partial has no inline click or submit handlers', !/\son(click|submit|input)=/i.test(partial));
+  ok('the partial root is data-page="general"', /data-page="general"/.test(partial));
+  const security = /id="web-login-settings"/.test(partial);
+  ok('the server renders the Security section (it has a login store)', security);
+  ok('the real token list answers in the shape the section shows',
+     realTokens.status === 'success' && realTokens.data && Array.isArray(realTokens.data.tokens), realTokens);
+  ok('the Security forms name their action',
+     /<form[^>]*data-action="set-password"/.test(partial) && /<form[^>]*data-action="create-token"/.test(partial));
+  ok('the Copy button names its action', /data-action="copy-token"/.test(partial));
+
+  const errs = [];
+  const logged = [];
+  const vc = new VirtualConsole();
+  vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
+  vc.on('error', (...a) => logged.push(a.join(' ')));
+  const dom = new JSDOM(`<!doctype html><html><body><div id="general-content">${partial}</div></body></html>`,
+    { url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
+  const { window } = dom;
+  const doc = window.document;
+  const panel = doc.getElementById('general-content');
+  require('../led_escape').install(window);
+  window.eval(fs.readFileSync(path.join(JS, 'widgets/registry.js'), 'utf8'));
+  window.eval(fs.readFileSync(path.join(JS, 'widgets/timezone-selector.js'), 'utf8'));
+  const widgets = window.LEDMatrixWidgets;
+  ok('the widget scripts register timezone-selector', !!(widgets && widgets.get('timezone-selector')));
+
+  let confirmAnswer = true;
+  const confirms = [];
+  window.confirm = m => { confirms.push(m); return confirmAnswer; };
+  const reloads = [];
+  window.htmx = { ajax: (method, url, opts) => reloads.push([method, url, opts.target]) };
+
+  const HOSTILE = '<img src=x onerror="window.pwned=1">';
+  let mode = 'ok';
+  let nextId = 1;
+  const requests = [];
+  const pending = [];
+  function fakeFetch(url, init) {
+    requests.push({ url, method: init.method, body: init.body ? JSON.parse(init.body) : undefined });
+    const respond = (status, body, headers) => Promise.resolve({
+      status, ok: status >= 200 && status < 300,
+      headers: { get: h => (headers || {})[h] || null },
+      text: () => Promise.resolve(JSON.stringify(body)),
+    });
+    if (mode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
+    if (mode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
+    if (mode === 'refuse') return respond(400, { status: 'error', message: 'Give the token a name.' });
+    if (url === '/api/v3/auth/tokens' && init.method === 'POST') {
+      const id = 'tok' + (nextId++);
+      const name = JSON.parse(init.body).name;
+      const answer = () => respond(201, {
+        status: 'success', message: 'Token created. Copy it now: it is not shown again.',
+        data: { token: 'lmx_' + id, record: { id, name, prefix: 'lmx_' + id.slice(0, 3), created_at: '2026-10-04T00:00:00' } },
+      });
+      if (mode === 'hang') return new Promise(resolve => pending.push(() => resolve(answer())));
+      return answer();
+    }
+    if (url.startsWith('/api/v3/auth/tokens/') && init.method === 'DELETE') {
+      return respond(200, { status: 'success', message: 'Token revoked.', data: { tokens: [] } });
+    }
+    if (url === '/api/v3/auth/password') {
+      return respond(200, { status: 'success', message: 'Login is on. Other browsers now need the password.' });
+    }
+    return respond(404, { status: 'error', message: 'unexpected ' + url });
+  }
+  const notes = [];
+  const registry = createRegistry({
+    document: doc,
+    context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
+  });
+  registry.register('general', generalPage);
+
+  const $ = id => doc.getElementById(id);
+  const root = () => doc.querySelector('[data-page="general"]');
+  const timezoneWidgets = () => $('timezone_container').querySelectorAll('.timezone-selector-widget').length;
+  const rows = () => doc.querySelectorAll('#web-login-tokens [data-token-id]');
+  const calls = (method, prefix) => requests.filter(r => r.method === method && r.url.startsWith(prefix));
+  const lastNote = () => notes[notes.length - 1] || [];
+  function submit(form) {
+    const event = new window.Event('submit', { bubbles: true, cancelable: true });
+    form.dispatchEvent(event);
+    return event;
+  }
+  const form = action => root().querySelector(`form[data-action="${action}"]`);
+  async function swap(html) {
+    panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
+    panel.innerHTML = html === undefined ? partial : html;
+    panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
+    await tick(20);
+  }
+
+  await registry.start();
+  await tick(20);
+
+  // ── the timezone picker ─────────────────────────────────────────────────
+  const savedZone = $('timezone_container').dataset.timezone;
+  ok('the partial carries the saved timezone', !!savedZone, savedZone);
+  ok('the timezone picker is drawn once', timezoneWidgets() === 1, timezoneWidgets());
+  ok('...holding the saved zone', $('timezone_data') && $('timezone_data').value === savedZone,
+     $('timezone_data') && $('timezone_data').value);
+  ok('...posted as "timezone"', $('timezone_data') && $('timezone_data').name === 'timezone');
+
+  for (let i = 0; i < 5; i++) await swap();
+  ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
+  ok('the timezone picker is drawn once, not stacked', timezoneWidgets() === 1, timezoneWidgets());
+
+  // ── the settings form is htmx's ─────────────────────────────────────────
+  const settings = root().querySelector('form[hx-post="/api/v3/config/main"]');
+  ok('submitting the settings form is not prevented', settings && !submit(settings).defaultPrevented);
+  ok('...and makes no request of the page\'s own', requests.length === 0, requests.length);
+
+  if (security) {
+    // ── create a token ────────────────────────────────────────────────────
+    const before = rows().length;
+    const create = form('create-token');
+    create.querySelector('[name="name"]').value = HOSTILE;
+    create.setAttribute('data-dirty', '');
+    ok('Create token is handled by the page', submit(create).defaultPrevented);
+    await tick(20);
+    ok('one POST to /api/v3/auth/tokens', calls('POST', '/api/v3/auth/tokens').length === 1, requests);
+    ok('...with the name typed', calls('POST', '/api/v3/auth/tokens')[0].body.name === HOSTILE);
+    ok('a row is added', rows().length === before + 1, rows().length);
+    ok('the hostile token name is shown as text', root().querySelector('#web-login-tokens').textContent.includes(HOSTILE));
+    ok('...and created no element', !root().querySelector('#web-login-tokens img') && !window.pwned);
+    ok('the "No tokens yet" line is gone', !root().querySelector('#web-login-tokens [data-empty]'));
+    ok('the token is shown once', $('web-login-new-token-value').textContent === 'lmx_tok1'
+       && !$('web-login-new-token').classList.contains('hidden'));
+    ok('the form is clean again (no "Leave site?")', !create.hasAttribute('data-dirty'));
+    ok('one success notification', lastNote()[1] === 'success' && /Token created/.test(lastNote()[0]), notes);
+
+    // ── copy it (plain http: not a secure context, so it is selected) ────
+    root().querySelector('button[data-action="copy-token"]').click();
+    ok('Copy selects the token where the clipboard API is unavailable',
+       window.getSelection().toString() === 'lmx_tok1' && /Selected/.test(lastNote()[0]), lastNote());
+
+    // ── revoke it (the row drawn by the page, so delegation covers it) ───
+    confirmAnswer = false;
+    const added = rows()[rows().length - 1];
+    added.querySelector('button[data-action="revoke-token"]').click();
+    await tick(20);
+    ok('a cancelled Revoke sends nothing', calls('DELETE', '/api/v3/auth/tokens/').length === 0);
+    ok('...after asking with the token name as written', confirms.length === 1 && confirms[0].includes(HOSTILE), confirms);
+    confirmAnswer = true;
+    added.querySelector('button[data-action="revoke-token"]').click();
+    await tick(20);
+    ok('Revoke sends one DELETE for that token',
+       calls('DELETE', '/api/v3/auth/tokens/').length === 1 && calls('DELETE', '/api/v3/auth/tokens/')[0].url === '/api/v3/auth/tokens/tok1',
+       calls('DELETE', '/api/v3/auth/tokens/'));
+    ok('...and removes its row', rows().length === before, rows().length);
+
+    // ── the password ──────────────────────────────────────────────────────
+    const pw = form('set-password');
+    pw.querySelector('[name="new_password"]').value = 'correct horse battery';
+    pw.querySelector('[name="confirm_password"]').value = 'correct horse batterY';
+    submit(pw);
+    await tick(20);
+    ok('mismatched passwords are never sent', calls('POST', '/api/v3/auth/password').length === 0);
+    ok('...and say so', lastNote()[1] === 'error' && /do not match/.test(lastNote()[0]), lastNote());
+    pw.querySelector('[name="confirm_password"]').value = 'correct horse battery';
+    submit(pw);
+    await tick(20);
+    const sent = calls('POST', '/api/v3/auth/password');
+    ok('a matching password is sent once', sent.length === 1, sent.length);
+    ok('...with the current password only when the form has one',
+       sent[0] && sent[0].body.new_password === 'correct horse battery'
+         && (('current_password' in sent[0].body) === !!pw.querySelector('[name="current_password"]')), sent[0]);
+    ok('...and the section is reloaded once', reloads.length === 1 && reloads[0][1] === '/v3/partials/general'
+       && reloads[0][2] === '#general-content', reloads);
+
+    // ── refused, network failure, login redirect ──────────────────────────
+    mode = 'refuse';
+    submit(form('create-token'));
+    await tick(20);
+    ok('a refused request shows the server message', lastNote()[1] === 'error' && lastNote()[0] === 'Give the token a name.', lastNote());
+    mode = 'network';
+    submit(form('create-token'));
+    await tick(20);
+    ok('a network failure says the request failed', lastNote()[1] === 'error' && /^Request failed: /.test(lastNote()[0]), lastNote());
+    mode = 'login';
+    const quiet = notes.length;
+    submit(form('create-token'));
+    await tick(20);
+    ok('the login redirect shows nothing (the page is leaving)', notes.length === quiet, notes.slice(quiet));
+
+    // ── a write survives a swap ───────────────────────────────────────────
+    mode = 'hang';
+    await swap();
+    const rowsBefore = rows().length;
+    form('create-token').querySelector('[name="name"]').value = 'Late';
+    submit(form('create-token'));
+    await tick(5);
+    await swap();
+    pending.shift()();
+    await tick(20);
+    ok('a token created before a swap is still reported', lastNote()[1] === 'success', lastNote());
+    ok('...and draws nothing into the new page', rows().length === rowsBefore
+       && $('web-login-new-token').classList.contains('hidden'), rows().length);
+    mode = 'ok';
+
+    // ── window.webLogin ────────────────────────────────────────────────────
+    const viaAlias = calls('POST', '/api/v3/auth/tokens').length;
+    form('create-token').querySelector('[name="name"]').value = 'Alias';
+    await generalPage.webLogin.createToken(form('create-token'));
+    ok('webLogin.createToken(form) creates one token', calls('POST', '/api/v3/auth/tokens').length === viaAlias + 1);
+    ok('webLogin has the five old methods',
+       ['setPassword', 'disable', 'createToken', 'copyToken', 'revoke'].every(m => typeof generalPage.webLogin[m] === 'function'));
+  }
+
+  // ── the widget loads late ───────────────────────────────────────────────
+  delete window.LEDMatrixWidgets;
+  await swap();
+  ok('nothing drawn while the widget is missing', timezoneWidgets() === 0, timezoneWidgets());
+  window.LEDMatrixWidgets = widgets;
+  await tick(150);
+  ok('drawn once the widget arrives', timezoneWidgets() === 1, timezoneWidgets());
+
+  delete window.LEDMatrixWidgets;
+  await swap();
+  const kept = root();
+  await swap('<p>another tab</p>');
+  window.LEDMatrixWidgets = widgets;
+  await tick(250);
+  ok('a page swapped away while waiting draws nothing', kept.querySelectorAll('.timezone-selector-widget').length === 0);
+  ok('nothing left mounted', registry.list().length === 0, registry.list().length);
+
+  ok('no console errors', logged.length === 0, logged);
+  ok('no DOM errors', errs.length === 0, errs);
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/js/dom/test_schedule_page.js

@@ -0,0 +1,167 @@
+// The Schedule tab as a page module (static/v3/js/pages/schedule.js), in a
+// real DOM (jsdom) with the real server-rendered partial and the real
+// widget registry and schedule-picker widget. Built like test_cache_page.js:
+//
+//   * the partial ships no <script> and no inline handlers; its root is
+//     data-page="schedule" and carries both saved schedules as JSON
+//   * both pickers are drawn once per swap-in, from the saved config, however
+//     many swaps came first
+//   * each form's save is reported in exactly one notification (the forms
+//     are marked data-reports-result so app.js stays quiet)
+//   * the dim brightness label follows the slider
+//   * a widget that loads late is waited for, and a page swapped away while
+//     waiting draws nothing
+//   * the old globals' entry points still work
+const http = require('http');
+const fs = require('fs');
+const path = require('path');
+const { pathToFileURL } = require('url');
+const { JSDOM, VirtualConsole } = require('jsdom');
+
+const BASE = process.env.BASE || 'http://localhost:5000';
+const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
+const get = p => new Promise((res, rej) =>
+  http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
+const load = f => import(pathToFileURL(path.join(JS, f)).href);
+const tick = ms => new Promise(r => setTimeout(r, ms || 0));
+
+let pass = 0, fail = 0;
+const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
+  : (fail++, console.log('  FAIL ' + l + (x !== undefined ? '  -> ' + JSON.stringify(x).slice(0, 300) : '')));
+
+(async () => {
+  const partial = await get('/partials/schedule');
+  const { createRegistry } = await load('core/registry.js');
+  const schedulePage = await load('pages/schedule.js');
+
+  console.log('\n── Schedule tab: page module (real DOM) ──');
+  ok('the partial ships no inline script', !/<script/i.test(partial));
+  ok('the partial has no inline handlers', !/\son(click|input|change|submit)=/i.test(partial));
+  ok('the forms carry no hx-on handler', !/hx-on/i.test(partial));
+  ok('the partial root is data-page="schedule"', /data-page="schedule"/.test(partial));
+  ok('both forms are marked data-reports-result',
+     (partial.match(/<form[^>]*data-reports-result/g) || []).length === 2);
+
+  const errs = [];
+  const logged = [];
+  const vc = new VirtualConsole();
+  vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
+  vc.on('error', (...a) => logged.push(a.join(' ')));
+  const dom = new JSDOM(`<!doctype html><html><body><div id="schedule-content">${partial}</div></body></html>`,
+    { url: BASE + '/', virtualConsole: vc, runScripts: 'outside-only' });
+  const { window } = dom;
+  const doc = window.document;
+  const panel = doc.getElementById('schedule-content');
+  // base.html defines LEDEscape (app-early.js) before any tab loads; the
+  // widget escapes with it.
+  require('../led_escape').install(window);
+  window.eval(fs.readFileSync(path.join(JS, 'widgets/registry.js'), 'utf8'));
+  window.eval(fs.readFileSync(path.join(JS, 'widgets/schedule-picker.js'), 'utf8'));
+  const widgets = window.LEDMatrixWidgets;
+  ok('the widget scripts register schedule-picker', !!(widgets && widgets.get('schedule-picker')));
+
+  const notes = [];
+  const registry = createRegistry({
+    document: doc,
+    context: { api: null, notify: (m, t) => notes.push([m, t]) },
+  });
+  registry.register('schedule', schedulePage);
+
+  const $ = id => doc.getElementById(id);
+  const root = () => doc.querySelector('[data-page="schedule"]');
+  const saved = key => JSON.parse(root().dataset[key]);
+  const drawn = () => ['schedule_picker_container', 'dim_schedule_picker_container']
+    .map(id => $(id).querySelectorAll('.schedule-picker-widget').length);
+  async function swap(html) {
+    panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
+    panel.innerHTML = html === undefined ? partial : html;
+    panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
+    await tick(20);
+  }
+  function answer(formId, xhr) {
+    $(formId).dispatchEvent(new window.CustomEvent('htmx:afterRequest', {
+      bubbles: true, detail: { xhr, elt: $(formId), successful: xhr.status < 300 } }));
+  }
+
+  await registry.start();
+  await tick(20);
+
+  // ── first load ──────────────────────────────────────────────────────────
+  ok('both pickers drawn once', drawn().join() === '1,1', drawn());
+  const schedule = saved('scheduleConfig');
+  const dim = saved('dimScheduleConfig');
+  ok('the saved config reaches the page as JSON', schedule && typeof schedule === 'object' && dim && typeof dim === 'object');
+  const mode = cfg => cfg.mode ? cfg.mode.replace('-', '_') : (cfg.days ? 'per_day' : 'global');
+  ok('the display picker shows the saved mode', $('schedule_mode_value').value === mode(schedule),
+     [$('schedule_mode_value').value, schedule.mode]);
+  ok('the dim picker shows the saved mode', $('dim_schedule_mode_value').value === mode(dim),
+     [$('dim_schedule_mode_value').value, dim.mode]);
+  ok('the dim picker shows the saved start time',
+     $('dim_schedule_start_time_hidden').value === (dim.start_time || '20:00'), $('dim_schedule_start_time_hidden').value);
+
+  // ── repeated swaps ──────────────────────────────────────────────────────
+  for (let i = 0; i < 5; i++) await swap();
+  ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
+  ok('each picker drawn once, not stacked', drawn().join() === '1,1', drawn());
+
+  const okXhr = body => ({ status: 200, responseText: JSON.stringify(body) });
+  answer('schedule_form', okXhr({ status: 'success', message: 'Schedule configuration saved successfully' }));
+  ok('a schedule save is reported once', notes.length === 1, notes);
+  ok('...with the server message and status',
+     notes[0] && notes[0][0] === 'Schedule configuration saved successfully' && notes[0][1] === 'success', notes[0]);
+  answer('dim_schedule_form', okXhr({ status: 'success' }));
+  ok('a dim schedule save without a message says so',
+     notes.length === 2 && notes[1][0] === 'Dim schedule settings saved' && notes[1][1] === 'success', notes[1]);
+  answer('schedule_form', { status: 400, responseText: JSON.stringify({ status: 'error' }) });
+  ok('a refused save without a message says so',
+     notes.length === 3 && notes[2][0] === 'Error saving schedule' && notes[2][1] === 'error', notes[2]);
+  answer('dim_schedule_form', { status: 502, responseText: '<html>Bad gateway</html>' });
+  ok('a non-JSON answer is an error',
+     notes.length === 4 && notes[3][0] === 'Invalid response from server' && notes[3][1] === 'error', notes[3]);
+  answer('schedule_form', { status: 200, responseText: 'null' });
+  ok('a JSON null answer is an error, not a crash',
+     notes.length === 5 && notes[4][1] === 'error', notes[4]);
+  // An htmx request from elsewhere on the page (outside both forms) is not a save.
+  root().querySelector('.settings-filter').dispatchEvent(new window.CustomEvent('htmx:afterRequest', {
+    bubbles: true, detail: { xhr: okXhr({ status: 'success', message: 'x' }) } }));
+  ok('a request from outside the two forms reports nothing', notes.length === 5, notes.length);
+
+  // ── the brightness label ────────────────────────────────────────────────
+  $('dim_brightness').value = '42';
+  $('dim_brightness').dispatchEvent(new window.Event('input', { bubbles: true }));
+  ok('the dim brightness label follows the slider', $('dim_brightness_display').textContent === '42%',
+     $('dim_brightness_display').textContent);
+
+  // ── the widget loads late ───────────────────────────────────────────────
+  delete window.LEDMatrixWidgets;
+  await swap();
+  ok('nothing drawn while the widget is missing', drawn().join() === '0,0', drawn());
+  window.LEDMatrixWidgets = widgets;
+  await tick(150);
+  ok('drawn once the widget arrives', drawn().join() === '1,1', drawn());
+
+  delete window.LEDMatrixWidgets;
+  await swap();
+  const kept = root();
+  await swap('<p>another tab</p>');
+  window.LEDMatrixWidgets = widgets;
+  await tick(250);
+  ok('a page swapped away while waiting draws nothing',
+     kept.querySelectorAll('.schedule-picker-widget').length === 0);
+  ok('nothing left mounted', registry.list().length === 0, registry.list().length);
+
+  // ── the old globals ─────────────────────────────────────────────────────
+  await swap();
+  const before = notes.length;
+  schedulePage.handleScheduleResponse({ target: $('schedule_form'), detail: { xhr: okXhr({ status: 'success' }) } });
+  schedulePage.handleDimScheduleResponse({ target: $('dim_schedule_form'), detail: { xhr: okXhr({ status: 'success' }) } });
+  ok('handleScheduleResponse(event) and handleDimScheduleResponse(event) report once each',
+     notes.length === before + 2 && notes[before][0] === 'Schedule settings saved'
+       && notes[before + 1][0] === 'Dim schedule settings saved', notes.slice(before));
+
+  ok('no console errors', logged.length === 0, logged);
+  ok('no DOM errors', errs.length === 0, errs);
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/js/dom/test_tools_sections.js

@@ -98,8 +98,50 @@ const get = p => new Promise((res, rej) =>
   window.saveMqttBridge();
   await tick(150);
   ok('save includes password once typed', sent && sent.mqtt_password === 'typed-secret');
+
+  // A password with TLS off is refused unless allow_insecure_mqtt is set
+  // (CWE-319, api_v3/misc.py). The form has to be able to send it, or a
+  // plain-LAN broker with a password can never be saved from here.
+  const allowRow = () => $('mqtt-allow-insecure-row');
+  const shown = el => !!el && !el.classList.contains('hidden');
+  ok('allow-without-TLS control rendered', !!$('mqtt-allow-insecure'));
+  ok('allow-without-TLS starts as saved',
+     !!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === !!bridge.data.config.allow_insecure_mqtt);
+  ok('allow-without-TLS shown only while TLS is off',
+     shown(allowRow()) === !$('mqtt-tls').checked);
+  $('mqtt-tls').checked = true;
+  $('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
+  ok('ticking TLS hides it', !shown(allowRow()));
+  $('mqtt-tls').checked = false;
+  $('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
+  ok('unticking TLS shows it again', shown(allowRow()));
+
+  const setAllow = v => { if ($('mqtt-allow-insecure')) $('mqtt-allow-insecure').checked = v; };
+  setAllow(false);
+  window.saveMqttBridge();
+  await tick(150);
+  ok('save sends allow_insecure_mqtt false when unticked', !!sent && sent.allow_insecure_mqtt === false, sent);
+  setAllow(true);
+  window.saveMqttBridge();
+  await tick(150);
+  ok('save sends allow_insecure_mqtt true when ticked', !!sent && sent.allow_insecure_mqtt === true, sent);
   onPut = null;
 
+  // Prefilled from the saved settings, and hidden while TLS is saved on.
+  bridgePayload = JSON.parse(JSON.stringify(bridge));
+  bridgePayload.data.config.allow_insecure_mqtt = true;
+  bridgePayload.data.config.mqtt_tls = false;
+  window.loadMqttBridge();
+  await tick(150);
+  ok('a saved opt-in is prefilled', !!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === true);
+  bridgePayload.data.config.mqtt_tls = true;
+  window.loadMqttBridge();
+  await tick(150);
+  ok('hidden on load when TLS is saved on', !shown(allowRow()));
+  bridgePayload = bridge;
+  window.loadMqttBridge();
+  await tick(150);
+
   // ── Pixlet editor, idle ────────────────────────────────────────────────
   const appIds = (apps.data.apps || []).map(a => a.id);
   ok('editor lists the apps on disk',