fix(display): schedule windows end at the end time; on-demand ending in off hours blanks at once (#714)

Schedule and dim windows are half-open [start, end): on from the start time, off at exactly the end time, whatever second the check runs. An on-demand session that ends in scheduled-off hours (expiry or stop) clears the once-a-minute schedule gate, so the panel blanks within about a second. Golden: schedule; two test_display_pending_changes.py tests now say end_time 23:00.

Merged with #712 and #713: with all three in, docs/RUN_LOOP_REDESIGN.md's 'may be wrong' list is empty, so that section now records that all six items are fixed and by which PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -19,6 +19,216 @@ accepts both, but the store flags the old spelling as deprecated
 
 ## Unreleased
 
+### Shared fetch service (stage 1)
+
+Core's own HTTP fetch paths now go through one service, so the plugins that
+use them get pooling, merging, host budgets and per-plugin request counts
+without a code change. Return values, exceptions, cache keys, TTLs and retry
+policies are unchanged.
+
+- **What goes through it.** `APIHelper.get`/`post`, `fetch_espn_scoreboard`
+  and its date chunks (`src/common/espn_dates.py` -- every scoreboard's live,
+  recent and upcoming fetch, and `SportsFetchMixin._fetch_season_directly`),
+  `BackgroundDataService` and `BaseOddsManager.get_odds`. Plugins' own
+  `requests` calls are not covered yet.
+- **Shared connection pools.** Core sessions with the same retry policy mount
+  one shared adapter, so the odds managers (one per scoreboard league
+  manager), the background service and the APIHelpers reuse one connection
+  pool per host. Headers, cookies and auth stay per session.
+- **Merged requests.** Identical GETs in flight at once (same URL and query,
+  effective headers, timeout and retry policy) go out once; the others get a
+  copy of that response or the same exception. `BackgroundDataService`'s own
+  request opts out (`share_in_flight=False`): it cancels and replaces fetches,
+  and already merges by cache key.
+- **Host budgets.** Per-host token buckets, `fetch_service.rate_limits` in
+  `config.json` (new optional section in the template). ESPN hosts default to
+  20 requests/s with a burst of 200, far above normal traffic; no request waits
+  longer than `max_wait_seconds` (2 s). Other hosts are unthrottled.
+- **Conditional GET.** A response with `ETag` or `Last-Modified` is kept in a
+  small bounded store (64 entries, 4 MB, 1 MB each) and revalidated; a `304`
+  is returned to the caller as the original `200`. ESPN sends neither
+  validator today, so on ESPN this is dormant.
+- **Counters.** Requests, merged, bytes, 304s, errors, HTTP errors, adapter
+  retries, throttled requests and seconds waited, per plugin and per host.
+  Which plugin made a request comes from a context variable the plugin
+  executor and plugin loader set (carried across the background service's and
+  `espn_dates`' worker threads), or else from the plugin directory on the
+  stack, so a plugin's own threads count too. The display publishes them to
+  the shared cache at most once a minute on change; read them at
+  `GET /api/v3/plugins/fetch-stats`.
+- `fetch_service` is a core config section (`src/core_config_keys.py`).
+
+### New modules
+
+- `src/common/fetch_service.py` -- the fetch service above. Core-internal in
+  this release: plugins reach it through `APIHelper` and `espn_dates`, and
+  should not import it directly until a plugin-facing API ships (stage 3), so
+  it sets no `ledmatrix_min_version` floor.
+
+### Tooling
+
+- Golden trace tests for the display loop. `test/test_run_loop_golden.py`
+  runs the real `DisplayController.run()` against fake plugins on a fake
+  clock (`test/_run_loop_harness.py`), with no hardware and no real sleeps,
+  and compares which mode was shown, for how long and why it ended with
+  `test/fixtures/run_loop_golden/`. It has 15 scenarios: rotation,
+  empty and failing modes, dynamic duration, live priority, on-demand
+  (including pinned and resumed after a restart), the schedule and dim
+  schedule, WiFi notices, sync follower and Vegas. The whole file runs in
+  about a second. This is stage 1 of restructuring `run()`, described in
+  `docs/RUN_LOOP_REDESIGN.md`. The other part of stage 1 is internal and
+  changes no behaviour: twelve blocks of `run()` move into named helpers
+  (`_dispatch_first_frame`, `_resolve_durations`, `_resolve_active_mode`,
+  `_needs_high_fps`, `_advance_after_screen` and others), and the traces are
+  identical before and after the move.
+
+### Fixes
+
+- A plugin whose `display()` raises now opens its circuit breaker. The first
+  frame of each screen goes through the plugin executor, which caught the
+  exception and returned False. The display read that as "no content" and
+  recorded a success, which reset the plugin's failure streak, so the breaker
+  never tripped. The plugin stayed in rotation and logged a traceback on
+  every screen. The raise now counts as a failure, so after three in a row
+  the plugin leaves rotation until the cooldown ends, the same as a raising
+  `update()`. The display still moves straight on to the next mode. A hung
+  `display()` is still recorded once, as a hang.
+- A WiFi notice (such as "Connected to HomeNet" or "AP mode on") now shows
+  within about a second of being posted. It was only checked between
+  screens, so a 5 s notice posted during a 20 s screen expired before that
+  screen ended and never appeared. The screen it interrupts comes back in
+  full once the notice ends. When Vegas stops scrolling for a notice, the
+  notice is what shows next, and Vegas resumes after it; before, a rotation
+  screen showed instead and the notice expired behind it. An active
+  on-demand session still holds the panel until it ends.
+- A game that goes live now takes over the panel within about a second.
+  Live priority was only checked between screens, so a game that went live
+  during a 30 s screen waited for that screen to end. The frame loops and the
+  dwell sleep now check too, at most once a second, and not while an
+  on-demand session is running or a live game is already showing. When Vegas
+  stops for a live game, the game is the next screen. Before, one rotation
+  screen showed first and the game came after it. Each check also asks each
+  plugin `has_live_content()` once, where a plugin registered under several
+  modes used to be asked once per mode.
+- The display schedule turns the panel off at exactly the end time. A window
+  now runs from its start time up to, but not including, its end time: with
+  07:00-23:00 the panel is on at 07:00 and off at 23:00. Before, the end
+  minute counted as on, and because the schedule is checked once a minute,
+  the panel went off at 23:00 or at 23:01 depending on when in the minute
+  that check ran. Windows that cross midnight and per-day schedules follow
+  the same rule, and so does the dim schedule.
+- An on-demand session that ends during scheduled-off hours, by expiring or
+  being stopped, blanks the panel within about a second. It used to stay on
+  until the next minute, because the once-a-minute schedule check had
+  already run that minute and the session had overridden its answer.
+
+## 3.8.0
+
+Live Vegas elements: plugin content that keeps changing while it scrolls
+(scores on the scoreboards' cards, the flight map's gliding aircraft, the
+weather radar's loop), with live games kept in the ticker by default. Also
+stable/beta update channels, the display control socket, the systemd
+display watchdog, optional web login, ES-module web UI pages, sports
+consolidation stage 4, and the removal of the 35 plugin APIs deprecated
+since 3.5.0 (see Removed).
+
+### New modules
+
+A plugin may import these via `src.*` once it floors on 3.8.0 (and should
+guard the import, since the loader's version check is advisory).
+
+- `src/plugin_system/vegas_elements.py` -- `VegasElement`, the unit a
+  plugin's `get_vegas_elements()` returns (also re-exported from
+  `base_plugin`). See "Live Vegas elements" below.
+- `src/plugin_system/testing/vegas.py` -- the harness for those hooks:
+  `render_vegas_elements`, `check_vegas_elements`, `render_vegas_timeline`.
+- `src/common/sports_vegas.py` -- what a scoreboard needs for live cards:
+  `game_key`, `game_fingerprint`, `dedupe_games`, `VegasCardCache`,
+  `StickyOdds`, `finished_games`.
+- `src/common/sports_plugin_host.py`, `sports_live_scroll.py`,
+  `sports_display_rules.py` and `sports_font_path.py` -- sports
+  consolidation stage 4; see "New modules (sports consolidation stage 4)"
+  below.
+- `src/display_watchdog.py`, `src/plugin_system/plugin_catalog.py`,
+  `src/plugin_system/plugin_runtime.py`, `src/plugin_system/field_model.py`
+  and the `src/ipc/` package -- new core modules (described below) that
+  plugins do not normally import.
+
+### Web UI: ES modules and one form model (stage 1)
+
+- The web UI gains a native ES-module layer, loaded with
+  `<script type="module">` and served as-is (no bundler, nothing built on
+  the Pi): `static/v3/js/core/` (`boot.js`, `registry.js`, `api.js`,
+  `facade.js`) and `static/v3/js/pages/`. `window.LEDMatrix` is its one
+  global: `api`, `pages`, `notify`, `escape`, `widgets` and `deprecate`, the
+  last keeping old `window.*` names working as aliases that warn once.
+- Tab partials can become page modules: a partial whose root says
+  `data-page="<name>"` carries no inline script, and the page registry calls
+  the page's `init` once when htmx swaps it in and `destroy` when it is
+  swapped out, aborting a signal that removes its listeners and cancels its
+  requests. The Cache tab is converted as the reference
+  (`js/pages/cache.js`); `window.deleteCacheFile` remains as an alias.
+- Static `.js` files are always served as `text/javascript`, which module
+  scripts require, and a `.js` request without the `?v=` content version
+  (how modules import each other) is revalidated instead of cached as
+  immutable for a year.
+- `src/plugin_system/field_model.py`: `build_field_model(schema, config)`
+  describes a plugin's config form as one JSON field model. Nothing renders
+  from it yet; `test/test_field_model_parity.py` checks it names exactly the
+  form controls and starting values the `render_field` macro emits, for every
+  schema available (all 46 official plugins, when a checkout is present).
+- `docs/WEB_FRONTEND_ARCHITECTURE.md`: the target architecture, the
+  page-by-page migration order, and how forms switch to the model and to
+  JSON submit behind a flag.
+
+### Control socket (stage 1: on-demand)
+
+- **The display now serves a control socket**,
+  `/run/ledmatrix/control.sock`. It carries versioned JSON commands, one per
+  line, and every command gets an answer
+  ([docs/IPC_CONTROL_SOCKET.md](docs/IPC_CONTROL_SOCKET.md)).
+  - On-demand start, stop and status are the first commands, plus `hello`
+    (version negotiation) and `ping`.
+  - Start and stop are acknowledged once the render thread has them queued.
+    The render thread applies them through the same handler as the file
+    mailbox, at its next on-demand check. On a scrolling screen that is the
+    next frame (the mailbox waits up to 0.25 s). On a static screen it is up
+    to 1 s, the same as the mailbox.
+  - The server's threads never touch rendering. Garbage, oversize messages
+    and slow or vanishing clients are answered or dropped without blocking the
+    display.
+  - New core modules: `src/ipc/contract.py`, `server.py` and `client.py`.
+    They are internal, not a plugin API.
+- **`POST /api/v3/display/on-demand/start` and `/stop` try the socket
+  first.** On any failure (the display is stopped or predates the socket, a
+  timeout, a refusal), they write the `display_on_demand_request` mailbox
+  exactly as before. The response's new `transport` field says which path
+  was used (`"socket"` or `"mailbox"`), and `socket_error` gives the reason
+  for a fallback. Both paths carry the same `request_id`, so a request that
+  arrives both ways runs once. The mailbox, and the plugins that write it
+  directly, keep working for at least one more release.
+- **Permissions.** The socket is `0660` and owned by the group the two
+  services already share (the cache directory's group, `ledmatrix` on an
+  installed device). On Linux the server also checks each connection's
+  `SO_PEERCRED`: root, the display's own user, or a member of that group.
+  `/run/ledmatrix` comes from the existing `RuntimeDirectory=` (#687), or the
+  display creates it as root under an older unit, so no installer or unit
+  change is needed. `LEDMATRIX_CONTROL_SOCKET` overrides the path for both
+  processes, or turns the socket off with `off`. A non-root dev run uses a
+  private per-user path under the temp directory.
+
+### Scroll speed
+
+- The Vegas Scroll Speed slider now says what the panel will do with the speed
+  it is on, and offers the nearest smooth ones to click. Only speeds that advance
+  a whole number of pixels per refresh look smooth, and which those are depends
+  on the panel (`GET /api/v3/config/scroll-speed-advice`, built on
+  `scroll_config.speed_advice()`; it uses the refresh the display measured, not
+  the `limit_refresh_rate_hz` cap). The slider steps by 1 px/s instead of 5.
+- The default 50 px/s no longer snaps to a stepped 48 px/s (2 px every 5
+  refreshes, 24 fps) on a 120 Hz panel: `solve_crisp()` now prefers 60 or 40 px/s,
+  which move one pixel at a time. 100 Hz panels are unaffected.
+
 ### Update channels
 
 - Devices no longer pick up every merge to `main`. A new setting,
@@ -276,6 +486,14 @@ read any of them:
 
 ### Fixes
 
+- Quieter routine logging. Every rotation logged each mode twice
+  ("Switching to mode", then "Processing mode"), and a mode with nothing to
+  show added "display() returned False" and "No content to display". Those
+  three repeats are now DEBUG; "Switching to mode" stays INFO, and `--debug`
+  shows the rest. On ledpi this cut the display's journal lines by about 30%
+  (~105 to ~75 per 5 minutes). Each stored line costs roughly 9 KB of SD-card
+  writes through the persistent journal (display at INFO vs WARNING: about
+  190 KiB/min apart), so the saving is real but small.
 - Reinstalling a plugin by its registry id when it is installed under its
   manifest id (`weather` in `ledmatrix-weather/`) no longer deletes it when
   the install then fails. The safety copy was taken of `weather/`, which did
@@ -418,6 +636,15 @@ read any of them:
 
 ### Scrolling
 
+- A Vegas strip extension no longer costs a late frame. Appending the next
+  group rebuilt the whole strip (`np.concatenate`, 2-2.6ms for a 10-14k px
+  strip at 512x64 on a Pi 4) and trimming copied what was left (1.2-1.8ms),
+  so on hdpi every extension frame missed its refresh. The strip now lives in
+  a buffer with spare room (`ScrollHelper.STRIP_SPARE_FACTOR`): an append
+  writes only the new columns (~0.2ms), a trim only moves the start, and the
+  one full copy happens when the buffer is reallocated, about once every two
+  strip-lengths scrolled. A strip set from outside (the multi-display
+  follower's) is never written through.
 - A Vegas strip extension costs the render thread about a third of what it
   did. Appending the next group and trimming what has scrolled past each
   rebuilt the strip's PIL image from its numpy array in full
@@ -429,6 +656,14 @@ read any of them:
   twice. Assigning `cached_image` still stores exactly what was assigned.
   New `ScrollHelper.has_strip()` says whether there is a strip without
   building its image; the frame path and Vegas use it.
+- The frame after a Vegas strip extension is no longer late on a Pi 4. The
+  render thread also laid out every plugin block of the new group (joining
+  its rows, measuring the separation between each pair) and pasted the
+  blocks into one image, about 37ms on hdpi against ~3.75ms of slack. The
+  thread that fetches the group now does that as each plugin arrives, and
+  the extension only writes the prepared pixels into the strip
+  (`ScrollHelper.append_content` takes RGB arrays): 3.2ms. In a 4 x 8 minute
+  A/B soak, extension frames went from 10 of 10 late to 3 of 10.
 
 ### Tooling
 
@@ -458,18 +693,52 @@ read any of them:
 - The 35 plugin-facing methods deprecated in 3.5.0 are now removed in 3.8.0,
   not 3.7.0: 3.7.0 shipped with all of them still in place, still warning
   "will be removed in LEDMatrix 3.7.0". The warning, the docs and
-  `test/test_deprecation.py` now say 3.8.0. Nothing is removed yet.
+  `test/test_deprecation.py` now say 3.8.0. They are removed in this
+  release (see Removed, below).
 - New `scripts/plugin_api_usage.py` lists every `@deprecated` core method and
   scans core, the plugin monorepo and the registry's third-party plugins for
   calls and overrides, telling real uses from unrelated methods of the same
   name. Its output is `docs/DEPRECATIONS_3.8.md` (linked from
-  `docs/PLUGIN_API_REFERENCE.md#deprecated-apis`): 34 of the 35 are unused;
-  `CacheManager.get_memory_cache_stats` is still called by core's own
-  `log_memory_cache_stats()`, so it stays until that call migrates.
+  `docs/PLUGIN_API_REFERENCE.md#deprecated-apis`); no plugin uses any of
+  the 35.
 - `test/test_deprecation.py` fails while any `@deprecated` marker names a
   release at or below `src.__version__`, so a release can no longer ship
   warning about a removal it has already passed.
 
+### Removed
+
+The 35 plugin-facing methods deprecated in 3.5.0 (each has logged a warning
+on first call since, announced for 3.7.0 and then moved to 3.8.0) are gone.
+The usage scan (`docs/DEPRECATIONS_3.8.md`, re-run 2026-10-01) found no call
+or override of any of them in the 46 monorepo plugins or the 8 third-party
+plugins `plugins.json` lists, and core's own last callers went with them. A
+plugin that still calls one gets an `AttributeError`;
+`docs/PLUGIN_API_REFERENCE.md#deprecated-apis` lists what to use instead.
+
+- `CacheManager`: `has_data_changed`, `update_cache`, `setup_persistent_cache`,
+  `get_sport_live_interval`, `get_sport_key_from_cache_key`,
+  `get_background_cached_data`, `is_background_data_available`,
+  `record_cache_hit`, `record_cache_miss`, `record_fetch_time`,
+  `get_cache_metrics`, `log_cache_metrics`, `get_memory_cache_stats`. The
+  private change-detection helpers behind `has_data_changed`
+  (`_has_weather_changed` and friends, `_is_market_open`) went with it.
+- `DisplayManager`: `draw_weather_icon`, `draw_sun`, `draw_cloud`, `draw_rain`,
+  `draw_snow`, `draw_text_with_icons`, `get_scrolling_stats`, and with them
+  the `WEATHER_COLORS` table and the private `_draw_sun`/`_draw_cloud`/
+  `_draw_rain`/`_draw_snow`/`_draw_storm` helpers.
+  `VisualTestDisplayManager` (the plugin test harness) drops its copies of
+  the icon methods too, so a plugin's visual tests fail the way the real
+  display would instead of passing against methods that no longer exist.
+- `FontManager`: `set_override`, `remove_override`, `get_overrides`,
+  `add_font`, `remove_font`, `validate_font`, `get_font_catalog`,
+  `get_available_fonts`, `get_size_tokens`, `get_performance_stats`,
+  `get_manager_fonts`, `get_detected_fonts`, `get_plugin_fonts`,
+  `unregister_plugin_fonts`, plus the `size_tokens` attribute and the private
+  `_save_overrides` and `_clear_plugin_font_cache`. `resolve_font()` still
+  applies `config/font_overrides.json`.
+- `PluginManager.get_enabled_plugins` (check `enabled` on the entries in
+  `plugin_manager.plugins`).
+
 ### Web UI styling: a real Tailwind build
 
 - The web UI's utility classes now come from a generated
@@ -500,6 +769,37 @@ read any of them:
   in AP mode with no internet. They get a local `static/v3/plugin-frame.css`
   with the v2 palette they were written against.
 
+### New modules (sports consolidation stage 4)
+
+A plugin may import these via `src.*` once it floors on 3.8.0. All four hold code the
+scoreboard plugins carry as identical copies (checked at ledmatrix-plugins
+`56c4f15`), moved without behaviour change under the plugins' own names;
+each docstring lists what the host class must provide. Nothing in core uses
+them yet. The plugins delete their copies when they floor on 3.8.0.
+
+- `src/common/sports_plugin_host.py` — `SportsPluginHostMixin`, ten helpers
+  of the scoreboard plugin class (`manager.py`) identical in all nine:
+  `_dispatch_switch_refresh` (with `_SWITCH_REFRESH_MIN_GAP_SECONDS`),
+  `get_vegas_priority_weight`, `_favorite_team_is_live`,
+  `_favorite_scan_targets`, `_favorite_scan_games`, `_game_involves`,
+  `get_vegas_content_type`, `_dynamic_feature_enabled`,
+  `_get_total_games_for_manager` and `_build_manager_key`. List it before
+  `BasePlugin`: two of these override its defaults.
+- `src/common/sports_live_scroll.py` — `SportsLiveScrollMixin`, the eight
+  `manager.py` methods that rebuild a live scroll strip mid-cycle without
+  moving the marquee (`_live_scroll_needs_rebuild`,
+  `_preserving_scroll_position`, ...), with `LIVE_SCROLL_REBUILD_MIN_SECONDS`
+  and `LIVE_SCROLL_REBUILD_DUTY_DIVISOR`; identical in the eight scoreboards
+  with a strip (not ufc). `LIVE_VOLATILE_FIELDS` stays in each plugin.
+- `src/common/sports_display_rules.py` — `SportsCardOptionsMixin`
+  (`_card_option`, `_recent_date_text`; the eight team scoreboards; list it
+  before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
+  (`_filtered_or_all`, `_effective_live_duration`; all nine).
+- `src/common/sports_font_path.py` — `resolve_font_path`, what every
+  scoreboard's `_resolve_font_path` (nine `sports.py`, eight
+  `game_renderer.py`) returns on a core that ships it: the path as given when
+  it exists, else `font_layout.resolve_asset_path`.
+
 ## 3.7.0
 
 Sports consolidation stage 3 (#672). No behaviour change: nothing in core

config/config.template.json

@@ -174,6 +174,16 @@
     "plugin_system": {
         "plugins_directory": "plugin-repos"
     },
+    "fetch_service": {
+        "enabled": true,
+        "max_wait_seconds": 2,
+        "rate_limits": {
+            "*.espn.com": {
+                "per_second": 20,
+                "burst": 200
+            }
+        }
+    },
     "web-ui-info": {
         "enabled": true,
         "display_duration": 10

docs/ADVANCED_PLUGIN_DEVELOPMENT.md

@@ -27,7 +27,7 @@ Advanced patterns, examples, and best practices for developing LEDMatrix plugins
 
 The Display Manager's icon methods — `draw_weather_icon()`, `draw_sun()`,
 `draw_cloud()`, `draw_rain()`, `draw_snow()` and `draw_text_with_icons()` —
-are deprecated, removed in 3.8.0. Draw your own icons instead: render them
+were removed in 3.8.0. Draw your own icons instead: render them
 onto a PIL image and paste it onto `self.display_manager.image`, or ship
 icon images with the plugin. The weather plugin's `WeatherIcons` class is an
 example. See [Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis).
@@ -194,7 +194,7 @@ def update(self):
     sport_key = "nhl"
     cache_key = f"{self.plugin_id}_{sport_key}_games"
     
-    # get_background_cached_data() is deprecated, removed in 3.8.0 — use get()
+    # get_background_cached_data() was removed in 3.8.0 — use get()
     cached = self.cache_manager.get(cache_key, max_age=60)
     
     if cached:
@@ -596,8 +596,8 @@ def update(self):
 
 ```python
 def update(self):
-    # get_enabled_plugins() is deprecated, removed in 3.8.0 — check the
-    # instance's `enabled` flag instead
+    # get_enabled_plugins() was removed in 3.8.0 — check the instance's
+    # `enabled` flag instead
     weather_plugin = self.plugin_manager.get_plugin("weather")
     if weather_plugin is not None and weather_plugin.enabled:
         # Use weather data

docs/ARCHITECTURE.md

@@ -41,12 +41,14 @@ each other. They share three things:
 
 | State | Where | Written by | Read by |
 |---|---|---|---|
-| On-demand request | cache `display_on_demand_request` | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py) | display: `_poll_on_demand_requests()` |
+| On-demand command | control socket `/run/ledmatrix/control.sock` ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)) | web: `start_on_demand_display()` / `stop_on_demand_display()` in [`api_v3/display.py`](../web_interface/blueprints/api_v3/display.py), via [`src/ipc/client.py`](../src/ipc/client.py) | display: [`src/ipc/server.py`](../src/ipc/server.py) acks; the render thread applies it in `_poll_on_demand_requests()` |
+| On-demand request (fallback) | cache `display_on_demand_request` | web, when the socket fails; four plugins write it directly | display: `_poll_on_demand_requests()` |
 | On-demand state | cache `display_on_demand_state` | display: `_publish_on_demand_state()` | web: `/api/v3/display/on-demand/status` |
 | Current screen | cache `display_current_state` | display | web: `/api/v3/display/current-status` |
 | Plugin errors | cache `plugin_error_snapshot` | display: `ErrorSnapshotPublisher` ([`src/error_aggregator.py`](../src/error_aggregator.py)) | web: `read_error_report()` for `/api/v3/errors/*` |
 | Error clear | cache `plugin_error_clear_request` | web | display |
 | Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
+| Fetch statistics (requests per plugin and host) | cache `fetch_stats_snapshot` | display: `FetchStatsPublisher` ([`src/common/fetch_service.py`](../src/common/fetch_service.py)), at most once a minute on change | web: `read_fetch_stats()` for `/api/v3/plugins/fetch-stats` |
 | Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
 | Plugin runtime (loaded, state, last error, version) | cache `plugin_runtime_snapshot` | display: `PluginRuntimePublisher` ([`src/plugin_system/plugin_runtime.py`](../src/plugin_system/plugin_runtime.py)) | web: `read_plugin_runtime()` for `/api/v3/plugins/installed`, `/plugins/state`, reconciliation |
 | Preview frame | `/tmp/led_matrix_preview.png` | display: `DisplayManager`, gated by [`snapshot_policy`](../src/common/snapshot_policy.py) | web: display SSE stream, `/api/v3/health` (file age) |
@@ -55,9 +57,14 @@ each other. They share three things:
 | Render-loop heartbeat | `/run/ledmatrix/display-heartbeat.json` (tmpfs) | display: the render thread, via [`display_watchdog`](../src/display_watchdog.py) | web: `/api/v3/health` (`checks.display_loop`); the update health check |
 
 The on-demand start route starts `ledmatrix.service` when it is not running
-(`start_service`, on by default) but never restarts a running one: the display
-reads the mailbox every `ON_DEMAND_POLL_INTERVAL` (0.25s), from its dwell
-sleep, its render loops and Vegas's interrupt check as well as the main loop.
+(`start_service`, on by default) but never restarts a running one. The routes
+send the command over the display's control socket and get an ack; when that
+fails (a stopped display, one older than the socket) they write the mailbox
+instead, which the display reads every `ON_DEMAND_POLL_INTERVAL` (0.25s), from
+its dwell sleep, its render loops and Vegas's interrupt check as well as the
+main loop. Both ways end in the same handler, `_handle_on_demand_request()`.
+The socket's handlers only queue; see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)
+for the protocol, the permission model and the plan to retire the mailboxes.
 
 ### Web and display processes: who runs plugins
 
@@ -181,7 +188,8 @@ the scheduler), and sets up Vegas mode.
 enable/disable, poll on-demand requests, run scheduled plugin updates, check
 the on/off schedule and brightness, then show one screen. Priority is
 on-demand, then WiFi status messages, then live priority, then Vegas mode,
-then normal rotation.
+then normal rotation. [RUN_LOOP_REDESIGN.md](RUN_LOOP_REDESIGN.md) is the
+plan for restructuring this loop and lists its golden trace tests.
 
 - **Rotation.** `available_modes` is the ordered list of display modes;
   `current_mode_index` advances after each screen.
@@ -203,7 +211,10 @@ then normal rotation.
   to it, rotating between several live games.
 - **Schedule and dim schedule.** `_check_schedule()` reads `schedule`;
   `_check_dim_schedule()` reads `dim_schedule` and
-  `display.hardware.brightness`. Both are re-evaluated once a minute.
+  `display.hardware.brightness`. Both are re-evaluated once a minute, and
+  both windows are half-open: on (or dimmed) from the start time, off at
+  the end time. When an on-demand session ends, the on/off schedule is
+  re-checked at once rather than at the next minute.
 - **Long screens.** While a screen is showing (a dwell, a scroll, a Vegas
   iteration), `_service_pending_changes()` repeats the on-demand, schedule
   and brightness checks every 0.25 s, so a change does not wait for the

docs/CONFIG_REFERENCE.md

@@ -31,6 +31,13 @@ tooling against it.
 | `start_time` / `end_time` | `"HH:MM"`, `07:00`–`23:00` | Global-mode on/off times |
 | `days.<weekday>.{enabled,start_time,end_time}` | per-day objects | Per-day-mode overrides |
 
+The display is on from `start_time` up to, but not including, `end_time`:
+with `07:00`–`23:00` it turns on at 07:00 and off at 23:00. An end earlier
+than the start crosses midnight (`22:00`–`07:00` is on overnight). In
+per-day mode, the entry for the current day decides. An on-demand session
+keeps the display on during off hours; once it ends or is stopped, the
+display blanks within about a second.
+
 Read by `DisplayController._check_schedule()` (`src/display_controller.py`).
 Managed in the web UI under Schedule.
 
@@ -44,7 +51,9 @@ Same shape as `schedule` (the template sets its `mode` to `"global"`), plus:
 
 Read by `DisplayController._check_dim_schedule()` (`src/display_controller.py`;
 saved via `POST /api/v3/config/dim-schedule`). The display returns to
-`display.hardware.brightness` outside the window.
+`display.hardware.brightness` outside the window. The window has the same
+boundaries as `schedule`: dimmed from `start_time` up to, but not including,
+`end_time`.
 
 ## `display.hardware` — matrix panel hardware
 

docs/DEPRECATIONS_3.8.md

@@ -2,8 +2,8 @@
 
 Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it (see [How to re-run](#how-to-re-run)).
 
-- Scanned: 2026-09-30, core 3.7.0
-- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4327c2e4), 46 plugins
+- Scanned: 2026-10-01, core 3.7.0
+- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4de1d134), 46 plugins
 - Third-party plugins: 8 with their own repo in `plugins.json` (f1-live, gif-player, pga-tour-leaderboard, plex-marquee, ledmatrix-dresden-departures, tidbyt-baseball-scoreboard, sleeper-fantasy, ledmatrix-nascar)
 
 **37 deprecated methods: 36 unused, 1 still used, 0 need review.**
@@ -78,19 +78,19 @@ File paths are relative to the plugin's directory (core: the repo root).
 | `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:63 | test call | `assert cm.get_memory_cache_stats()["last_cleanup"] >= before` |
 | `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:68 | test call | `stats = cm.get_memory_cache_stats()` |
 | `DisplayManager.draw_sun` | core | src/plugin_system/testing/visual_display_manager.py:417 | unrelated | `def draw_sun(self, x: int, y: int, size: int = 16):` |
-| `DisplayManager.draw_cloud` | core | src/display_manager.py:1356 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
-| `DisplayManager.draw_cloud` | core | src/display_manager.py:1371 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
+| `DisplayManager.draw_cloud` | core | src/display_manager.py:1359 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
+| `DisplayManager.draw_cloud` | core | src/display_manager.py:1374 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
 | `DisplayManager.draw_cloud` | core | src/plugin_system/testing/visual_display_manager.py:421 | unrelated | `def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):` |
 | `DisplayManager.draw_cloud` | ledmatrix-weather | weather_icons.py:184 | unrelated | `def draw_cloud(draw: ImageDraw, x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)):` |
 | `DisplayManager.draw_rain` | core | src/plugin_system/testing/visual_display_manager.py:425 | unrelated | `def draw_rain(self, x: int, y: int, size: int = 16):` |
 | `DisplayManager.draw_snow` | core | src/plugin_system/testing/visual_display_manager.py:429 | unrelated | `def draw_snow(self, x: int, y: int, size: int = 16):` |
-| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1515 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
+| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1518 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
 | `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:510 | unrelated | `def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:` |
 | `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:533 | unrelated | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
-| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:76 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
-| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1265 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
-| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1544 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
-| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1635 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
+| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:84 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
+| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1280 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
+| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1559 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
+| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1650 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
 | `DisplayManager.draw_weather_icon` | ledmatrix-weather | weather_icons.py:168 | unrelated | `def draw_weather_icon(image: Image.Image, icon_code: str, x: int, y: int, size: int = DEFAULT_SIZE):` |
 | `DisplayManager.draw_text_with_icons` | core | src/plugin_system/testing/visual_display_manager.py:526 | unrelated | `def draw_text_with_icons(self, text: str, icons: List[tuple] = None,` |
 | `FontManager.get_font_catalog` | core tests | test/test_deprecation.py:229 | test call | `assert fm.get_font_catalog() == fm.font_catalog` |
@@ -106,12 +106,12 @@ File paths are relative to the plugin's directory (core: the repo root).
 
 | Source | Group | Python files | Hits |
 |---|---|---|---|
-| core | core | 164 | 20 |
-| core tests | core-tests | 323 | 17 |
+| core | core | 172 | 20 |
+| core tests | core-tests | 347 | 17 |
 | 7-segment-clock | monorepo | 3 | 0 |
-| afl-scoreboard | monorepo | 34 | 0 |
-| baseball-scoreboard | monorepo | 60 | 0 |
-| basketball-scoreboard | monorepo | 48 | 0 |
+| afl-scoreboard | monorepo | 35 | 0 |
+| baseball-scoreboard | monorepo | 61 | 0 |
+| basketball-scoreboard | monorepo | 49 | 0 |
 | birdnet-go | monorepo | 2 | 0 |
 | blackjack | monorepo | 7 | 2 |
 | calendar | monorepo | 5 | 1 |
@@ -121,37 +121,37 @@ File paths are relative to the plugin's directory (core: the repo root).
 | cricket-scoreboard | monorepo | 8 | 0 |
 | f1-scoreboard | monorepo | 15 | 0 |
 | fantasy-blitz | monorepo | 13 | 0 |
-| football-scoreboard | monorepo | 73 | 0 |
+| football-scoreboard | monorepo | 74 | 0 |
 | geochron | monorepo | 10 | 0 |
 | hello-world | monorepo | 2 | 0 |
-| hockey-scoreboard | monorepo | 51 | 0 |
+| hockey-scoreboard | monorepo | 52 | 0 |
 | incoming-packages | monorepo | 8 | 0 |
 | jellyfin-now-playing | monorepo | 4 | 0 |
-| lacrosse-scoreboard | monorepo | 39 | 0 |
+| lacrosse-scoreboard | monorepo | 40 | 0 |
 | ledmatrix-elections | monorepo | 12 | 0 |
-| ledmatrix-flights | monorepo | 45 | 0 |
+| ledmatrix-flights | monorepo | 48 | 0 |
 | ledmatrix-leaderboard | monorepo | 9 | 0 |
 | ledmatrix-music | monorepo | 11 | 0 |
 | ledmatrix-stocks | monorepo | 7 | 0 |
-| ledmatrix-weather | monorepo | 14 | 6 |
+| ledmatrix-weather | monorepo | 15 | 6 |
 | march-madness | monorepo | 4 | 0 |
 | masters-tournament | monorepo | 10 | 0 |
 | mqtt-notifications | monorepo | 4 | 0 |
 | news | monorepo | 6 | 0 |
 | nfl-draft | monorepo | 3 | 0 |
 | nfl-stat-leaders | monorepo | 8 | 0 |
-| nrl-scoreboard | monorepo | 29 | 0 |
+| nrl-scoreboard | monorepo | 30 | 0 |
 | odds-ticker | monorepo | 9 | 0 |
 | of-the-day | monorepo | 14 | 0 |
 | olympics | monorepo | 16 | 1 |
 | on-air | monorepo | 2 | 0 |
 | pomodoro-timer | monorepo | 3 | 0 |
-| soccer-scoreboard | monorepo | 46 | 0 |
+| soccer-scoreboard | monorepo | 47 | 0 |
 | static-image | monorepo | 3 | 0 |
 | stock-news | monorepo | 3 | 0 |
 | text-display | monorepo | 4 | 0 |
 | tide-display | monorepo | 3 | 0 |
-| ufc-scoreboard | monorepo | 34 | 0 |
+| ufc-scoreboard | monorepo | 38 | 0 |
 | web-ui-info | monorepo | 2 | 0 |
 | youtube-stats | monorepo | 5 | 0 |
 | f1-live | third-party | 10 | 0 |

docs/DEVELOPER_QUICK_REFERENCE.md

@@ -54,8 +54,8 @@ rows = self.layout.bounds.inset(1).split_v(3, 1, gap=1)
 self.draw_fit("12:34", rows[0])                 # largest crisp font that fits
 self.draw_image(logo, rows[1], mode="fill_height", crop_to_ink=True)
 
-# Weather icons: draw_weather_icon() is deprecated, removed in 3.8.0 —
-# draw your own icons (the weather plugin ships WeatherIcons)
+# Weather icons: draw_weather_icon() was removed in 3.8.0 — draw your
+# own icons (the weather plugin ships WeatherIcons)
 
 # Scrolling state
 display_manager.set_scrolling_state(True)
@@ -78,7 +78,7 @@ strategy = cache_manager.get_cache_strategy("weather")
 ```
 
 `get_background_cached_data()` (use `get()`) and `get_sport_live_interval()`
-are deprecated, removed in 3.8.0. See
+were removed in 3.8.0. See
 [Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis).
 
 ## Plugin Manager Quick Methods
@@ -87,8 +87,8 @@ are deprecated, removed in 3.8.0. See
 # Get plugins
 plugin = plugin_manager.get_plugin("plugin-id")
 all_plugins = plugin_manager.get_all_plugins()
-# get_enabled_plugins() is deprecated, removed in 3.8.0 — check `enabled`
-# on the entries in plugin_manager.plugins
+# get_enabled_plugins() was removed in 3.8.0 — check `enabled` on the
+# entries in plugin_manager.plugins
 
 # Get info
 info = plugin_manager.get_plugin_info("plugin-id")

docs/FONT_MANAGER.md

@@ -13,10 +13,9 @@
 BDF fonts in `assets/fonts/`, registers fonts that plugins ship, and records
 which plugin uses which font so the web UI can show it.
 
-Several methods are deprecated and will be removed in LEDMatrix 3.8.0; they
-log a warning on first call. They are listed in
-[Deprecated methods](#deprecated-methods) below, and the full set is pinned in
-[`test/test_deprecation.py`](../test/test_deprecation.py).
+Several methods were removed in LEDMatrix 3.8.0 after a release of
+deprecation warnings; [Removed methods](#removed-methods) below lists them
+with what to use instead.
 
 ## Getting the FontManager
 
@@ -128,8 +127,8 @@ font = self.font_manager.resolve_font(
 
 `resolve_font()` still honours `config/font_overrides.json` (a map of
 element key to `family` and/or `size_px`), which is read once at start-up.
-The methods that edit it — `set_override()`, `remove_override()`,
-`get_overrides()` — are deprecated, and there is no web UI or REST endpoint
+The methods that edited it — `set_override()`, `remove_override()`,
+`get_overrides()` — were removed in 3.8.0, and there is no web UI or REST endpoint
 for overrides (the override editor and `/api/v3/fonts/overrides` were
 removed). To let users choose a font, add a field to your plugin's config
 schema.
@@ -207,9 +206,10 @@ Current methods:
 | `clear_cache()` | Drop cached fonts and metrics |
 | `font_catalog` (attribute) | Family name → file path |
 
-### Deprecated methods
+### Removed methods
 
-Removed in 3.8.0. Each logs a warning on first call.
+Removed in 3.8.0, after logging a deprecation warning on first call since
+3.5.0.
 
 | Method | Use instead |
 |---|---|

docs/HOW_TO_RUN_TESTS.md

@@ -170,6 +170,31 @@ pytest test/test_config_manager.py
 pytest
 ```
 
+### Web UI JavaScript Tests
+
+The suites in `test/js` need node; the DOM ones also need jsdom and a running
+web interface (details in [`test/js/README.md`](../test/js/README.md)):
+
+```bash
+npm install --no-audit --no-fund --prefix test/js   # jsdom; node_modules is gitignored
+EMULATOR=true python3 web_interface/app.py           # in another shell
+BASE=http://localhost:5000 REQUIRE_DOM=1 node test/js/run_all.js
+```
+
+`pytest test/test_js_unit_suites.py` runs just the unit suites.
+
+### Plugin Config Form Parity
+
+`test/test_field_model_parity.py` checks `build_field_model` against the
+`render_field` macro for every plugin schema it finds
+([WEB_FRONTEND_ARCHITECTURE.md](WEB_FRONTEND_ARCHITECTURE.md)). It always
+covers `plugin-repos/` and the test fixtures; point it at a checkout of the
+official plugins to cover those too:
+
+```bash
+LEDMATRIX_MONOREPO_PLUGINS=../ledmatrix-plugins/plugins pytest test/test_field_model_parity.py
+```
+
 ### Debug a Failing Test
 
 ```bash

docs/IPC_CONTROL_SOCKET.md

@@ -0,0 +1,250 @@
+# Control socket (web → display)
+
+The display process serves a Unix socket that the web interface uses to send
+it commands and get an answer back. It replaces the cache-file "mailboxes" on
+the SD card one command at a time. Stage 1, described here, carries on-demand
+start/stop/status. The file mailbox stays as a fallback for one release.
+
+| | |
+|---|---|
+| Socket | `/run/ledmatrix/control.sock` (tmpfs) |
+| Served by | the display process ([`src/ipc/server.py`](../src/ipc/server.py)), started by `DisplayController.run()` |
+| Used by | the web interface ([`src/ipc/client.py`](../src/ipc/client.py)): `POST /api/v3/display/on-demand/start` and `/stop` |
+| Contract | [`src/ipc/contract.py`](../src/ipc/contract.py): messages, versions, framing and the socket path; both sides import it |
+| Override | `LEDMATRIX_CONTROL_SOCKET=/some/path.sock` for both processes, or `=off` to disable it |
+
+## Why
+
+Before the socket, the web interface sent commands by writing a cache key
+(`display_on_demand_request`) that the display read every 0.25 s.
+
+- **No acknowledgement.** The route answered "success" once the file was
+  written, whether or not a display was running to read it.
+- **Lost requests.** The display had to read the request and then delete it.
+  A request written between those two steps could be thrown away (see
+  `_consume_on_demand_request`). The cache has no atomic claim to prevent it.
+- **Fragile.** Each channel repeated its own permission, atomic-write,
+  staleness and in-memory-cache rules. Two of them caused bugs: a `memory_ttl`
+  bug ignored every on-demand request after the first for an hour, and a
+  stopped display was still reported as "active" for two minutes.
+
+The socket answers every command, carries one request per message (so nothing
+can overwrite it), and belongs to the display process. If the display is not
+running, the socket does not exist, and the web interface knows right away.
+
+## Protocol (version 1)
+
+**Framing.** One JSON object per line (newline-delimited JSON), UTF-8, at
+most 64 KiB per line (`MAX_MESSAGE_BYTES`). Senders encode with
+`ensure_ascii`, so a newline never appears inside a message. A connection
+can carry several requests. Each request gets exactly one response, in order.
+
+**Request**
+
+```json
+{"v": 1, "id": "5f0c…", "cmd": "on_demand.start",
+ "args": {"plugin_id": "clock", "mode": null, "duration": 30, "pinned": false}}
+```
+
+- `v` is the protocol version.
+- `id` is a printable string of 1-128 characters. It is echoed back in the
+  response, and for on-demand commands it is also the on-demand `request_id`.
+- `cmd` is a command name.
+- `args` is an object. It may be omitted when a command takes no arguments.
+
+**Response**
+
+```json
+{"v": 1, "id": "5f0c…", "ok": true,  "result": {"accepted": true, "request_id": "5f0c…", "queued": 1}}
+{"v": 1, "id": "5f0c…", "ok": false, "error": {"code": "busy", "message": "…"}}
+```
+
+`id` is `null` only when the request could not be parsed far enough to have
+one. Clients branch on `error.code`, never on the message text.
+
+**Commands**
+
+| `cmd` | `args` | `result` | Kind |
+|---|---|---|---|
+| `hello` | `{versions: [int], client?: str}` | `{version, versions, commands, max_message_bytes, server}` | answered directly |
+| `ping` | — | `{pong: true}` | answered directly |
+| `on_demand.start` | `{plugin_id?, mode?, duration?, pinned?}` (at least one of `plugin_id` and `mode`) | ack | queued |
+| `on_demand.stop` | — | ack | queued |
+| `on_demand.status` | — | `{on_demand: {...}, current_mode, display_active}` | answered directly |
+
+`duration` is a number of seconds, or a numeric string. `0`, `null` or `""`
+mean "until stopped". `pinned` must be a real boolean: the REST route has
+already converted strings like `"false"` before it sends the command. The
+`on_demand` object in `on_demand.status` is the same dict the display
+publishes to `display_on_demand_state`.
+
+**Acknowledgements.** A queued command is *accepted*, not *done*.
+`{"accepted": true, "request_id": …}` means the command is waiting in the
+render thread's queue, and the render thread will apply it at its next
+on-demand check. That is within one frame on a scrolling screen, 0.25 s
+during a dwell, and up to 1 s on a static screen, whose frame loop sleeps a
+second between frames. Except on a scrolling screen, where the mailbox waits
+up to 0.25 s, these are the mailbox's delays too: stage 1 adds
+acknowledgements, not speed. Any outcome is published as before
+(`display_on_demand_state`, and `status`/`error` for a bad plugin or mode),
+and it can be read with `on_demand.status`.
+
+**Versions.** Every request carries `v`. For any command except `hello`, a
+`v` the display does not speak gets `unsupported_version`. `hello` is checked
+by its `versions` list instead, and its result names the highest version both
+sides share, so a client can find out what a display supports before it
+relies on anything newer. Stage 1's client sends `v: 1` and falls back to the
+mailbox when the display refuses it. It does not send `hello` first, which
+saves a round trip.
+
+**Error codes:** `bad_json`, `bad_request`, `message_too_large`,
+`unsupported_version`, `unknown_command`, `invalid_args`, `busy` (queue full,
+or too many connections), `forbidden` (peer credentials refused), `internal`.
+
+Try it on a device:
+
+```bash
+python3 - <<'EOF'
+from src.ipc import client          # run from the project directory
+print(client.on_demand_status())
+EOF
+```
+
+## How the display applies a command
+
+The server's threads never touch rendering. A connection thread parses the
+request, validates it against the contract, and then does one of two things:
+
+- For a command that changes the panel, it puts a `QueuedCommand` on a
+  bounded queue (16 entries) and answers with the ack.
+- For a query, it answers from a status snapshot the display provides
+  (`DisplayController._control_status`). The snapshot only reads attributes.
+
+The render thread drains the queue in `_poll_on_demand_requests()`, the same
+place it reads the mailbox, and hands each command to
+`_handle_on_demand_request()`, which is the mailbox's own handler. The two
+paths share all of their code: activation, the processed-id guard, error
+publishing, and resuming the rotation afterwards. The 0.25 s floor on the
+mailbox read does not apply to the queue, because draining it costs no disk
+read. A queued command also lets `_service_pending_changes()` skip its own
+floor, so a long scrolling screen or a Vegas iteration takes the command at
+its next frame.
+
+**Exactly once.** A command and a mailbox write for the same request share
+one `request_id`. If the client times out after the display queued the
+command and then also writes the mailbox, the display processes the request
+once. The existing `on_demand_request_id` and processed-id checks drop the
+second copy.
+
+## Robustness
+
+All of this runs inside the display process, so nothing a client does may
+block the render loop or crash it:
+
+- **Bounded connections.** Each connection gets its own daemon thread, with
+  at most 8 at once. One more is answered `busy` and closed.
+- **Timeouts.** Each read and write times out after 2 s. A message must
+  arrive whole within 5 s of its first byte. An idle connection is closed
+  after 10 s. A slow or stuck client costs one thread for a few seconds.
+- **Malformed input.** A line that is not JSON gets `bad_json`, and the
+  connection carries on. A line longer than 64 KiB gets `message_too_large`,
+  and the connection is closed, because the next message boundary cannot be
+  found. A client that disconnects mid-message is dropped silently. No
+  exception from a handler leaves the connection thread.
+- **Full queue.** When the queue is full, the client gets `busy` and falls
+  back to the mailbox. A full queue means the render thread is stuck, and the
+  systemd watchdog deals with that.
+- **Startup.** The server binds under a temporary name, sets the mode and the
+  group, then renames the socket into place, so it never appears with the
+  umask's permissions. It removes a stale socket (a file that nothing is
+  listening on). It never removes a live socket or a file that is not a
+  socket. `close()` removes the socket only if it is still the one this
+  process created.
+- **Never fatal.** If the server cannot start (Windows, no `AF_UNIX`, a bind
+  failure, `LEDMATRIX_CONTROL_SOCKET=off`), it logs that and the display runs
+  as before. The web interface then uses the mailbox.
+
+## Security model
+
+The display runs as root and the web interface as the installing user (see
+[PERMISSIONS.md](PERMISSIONS.md)). The socket admits exactly those two, plus
+anything else in the group they share:
+
+1. **The directory.** `/run/ledmatrix` is created by `RuntimeDirectory=ledmatrix`
+   in `ledmatrix.service` (#687): root-owned, `0755`, on tmpfs, and removed
+   when the display stops. Under an older unit, the display creates the
+   directory itself as root, as it does for the heartbeat. No installer
+   change is needed.
+2. **The socket file.** The file is `root:<shared group>` with mode `0660`,
+   and the kernel refuses `connect()` to anyone without write permission on
+   it. The shared group is the cache directory's group whenever that
+   directory is group-writable. That is `ledmatrix` on an installed device
+   (`/var/cache/ledmatrix` is `root:ledmatrix 2775`), and it is the same rule
+   DiskCache uses for every file the two services share. Otherwise the group
+   is the project directory's (`get_shared_group_gid()`, which config files
+   use). With neither, the mode is `0600` and only root can connect.
+3. **Peer credentials.** Where the kernel reports them (`SO_PEERCRED`, on
+   Linux), the server checks every connection again. It accepts root, the
+   display's own user, or a member of the shared group: the peer's primary
+   gid, or a supplementary group read from `/proc/<pid>/status`. If `/proc`
+   is unreadable, it uses the group database. Any other peer gets `forbidden`
+   and is disconnected. This covers a socket mode that someone loosened by
+   hand.
+
+The commands are deliberately narrow. Stage 1 can start or stop on-demand
+display and read its state, which anyone who can reach the web UI can already
+do. Nothing on the socket runs a shell, writes a file, or names a path.
+
+**Development.** A display that is not root and cannot write to
+`/run/ledmatrix`, such as `python3 run.py -e` from a checkout, serves the
+socket at `$TMPDIR/ledmatrix-<uid>/control.sock`. That directory is private
+(`0700`), and the server refuses it if another user owns it. The web
+interface, run by the same user, looks there after `/run/ledmatrix`. The test
+suite sets `LEDMATRIX_CONTROL_SOCKET=off` (`test/conftest.py`), so a run on a
+device never touches the live display.
+
+## Stage plan
+
+1. **On-demand, with acks (this stage).** Contract, server, client.
+   `on_demand.start`/`stop`/`status`, `hello`, `ping`. The REST routes try the
+   socket first and report `transport: "socket" | "mailbox"` (plus
+   `socket_error` on fallback). The mailbox is unchanged, and the plugins that
+   write it directly (birdnet-go, mqtt-notifications, on-air, pomodoro-timer)
+   keep working.
+2. **Commands that are restarts or polls today.**
+   - `brightness.set`, transient and with no `config.json` write.
+   - `plugin.reload`, which replaces the `restart_required` answer from #688
+     with a live reload of the updated plugin on the render thread.
+   - `config.reload`, which applies a saved config without waiting for the 2 s
+     mtime poll and acks which sections changed.
+   - The dwell sleep and the static screen's 1 s frame sleep wait on the
+     queue instead of sleeping, so a command lands within milliseconds on
+     every kind of screen. Under WSL, with a static plugin on screen, a stop
+     takes 1.0 s by either path today.
+3. **A state stream.** A `subscribe` command that keeps the connection open
+   and pushes events: mode changes, on-demand state, plugin runtime state and
+   the heartbeat. It replaces the polled `display_current_state`,
+   `plugin_runtime_snapshot` (#690) and `display-heartbeat.json` (#687) for
+   readers that hold a connection. The web interface relays it to its
+   existing SSE stream. The files remain for one release for older readers.
+4. **Retire the mailboxes.** After a release in which every device has had the
+   socket, the web interface stops writing `display_on_demand_request`, and
+   the display stops polling it, logging the plugins that still write it so
+   they can move to an in-process `request_display()`. The other cache keys
+   used as messages (`plugin_error_clear_request` and the remaining
+   `display_*` keys) move to the socket or to tmpfs.
+
+## Checking it on a device
+
+```bash
+ls -l /run/ledmatrix/control.sock                 # srw-rw---- root ledmatrix
+sudo journalctl -u ledmatrix | grep "Control socket"
+curl -s -X POST localhost:5000/api/v3/display/on-demand/start \
+  -H 'Content-Type: application/json' -d '{"plugin_id":"clock","duration":20}'
+# ... "transport": "socket"
+```
+
+If the response says `"transport": "mailbox"`, `socket_error` gives the
+reason. `no_socket` means the display is stopped or predates the socket.
+`refused` usually means the web user is not in the socket's group, which
+takes effect when the web service restarts after the user is added.

docs/PERMISSIONS.md

@@ -29,6 +29,8 @@ in again (services pick them up on restart).
 | `assets/` | web user | dirs `755`, files `644` | Root writes downloaded logos regardless |
 | `/var/cache/ledmatrix/` | `root:ledmatrix` | `2775` (setgid) | Shared cache: see below |
 | Cache files | creator : `ledmatrix` | `660` | |
+| `/run/ledmatrix/` | `root` | `755` | tmpfs; `RuntimeDirectory=` in `ledmatrix.service`, removed when the display stops |
+| `/run/ledmatrix/control.sock` | `root` : cache directory's group (`ledmatrix`) | `660` | The display's control socket; only root and that group can connect. See [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md#security-model) |
 | `scripts/fix_perms/safe_plugin_rm.sh`, `safe_pip_install.sh` | `root:root` | `755` | Run as root through sudo, so the web user must not be able to edit them |
 | `/etc/sudoers.d/ledmatrix_web`, `ledmatrix_wifi` | `root` | `440` | |
 

docs/PLUGIN_API_REFERENCE.md

@@ -14,6 +14,7 @@ Complete API reference for plugin developers. This document describes all method
 - [Display Manager](#display-manager)
 - [Cache Manager](#cache-manager)
 - [Plugin Manager](#plugin-manager)
+- [Fetching data](#fetching-data)
 - [Deprecated APIs](#deprecated-apis)
 
 ---
@@ -628,18 +629,6 @@ self.display_manager.update_display()
 
 This is the canonical way to render arbitrary images.
 
-### Weather Icons (deprecated)
-
-> Deprecated, removed in 3.8.0 — draw your own icons (the weather plugin
-> ships `WeatherIcons`). See [Deprecated APIs](#deprecated-apis).
-
-- `draw_weather_icon(condition, x, y, size=16)` — icon for a condition
-  string such as `"clear"`, `"clouds"`, `"rain"`, `"snow"`, `"storm"`
-- `draw_sun(x, y, size=16)`, `draw_cloud(x, y, size=16, color=(200, 200, 200))`,
-  `draw_rain(x, y, size=16)`, `draw_snow(x, y, size=16)`
-- `draw_text_with_icons(text, icons=None, x=None, y=None, color=(255, 255, 255))`
-  — text plus a list of `(icon_type, x, y)` icons; calls `update_display()`
-
 ### Scrolling State Management
 
 For plugins that implement scrolling content, use these methods to coordinate with the display system.
@@ -730,20 +719,6 @@ Process any deferred updates if not currently scrolling. Called automatically by
 
 **Note**: Plugins typically don't need to call this directly.
 
-#### `get_scrolling_stats() -> dict`
-
-> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
-
-Get current scrolling statistics for debugging.
-
-**Returns**: Dictionary with scrolling state information
-
-**Example**:
-```python
-stats = self.display_manager.get_scrolling_stats()
-self.logger.debug(f"Scrolling: {stats['is_scrolling']}, Deferred: {stats['deferred_count']}")
-```
-
 ### Available Fonts
 
 The Display Manager provides several pre-loaded fonts:
@@ -873,27 +848,6 @@ Get data with automatic strategy detection from cache key.
 data = self.cache_manager.get_with_auto_strategy("nhl_live_scores")
 ```
 
-#### `get_background_cached_data(key: str, sport_key: Optional[str] = None) -> Optional[Dict[str, Any]]`
-
-> Deprecated, removed in 3.8.0 — use `get()`. See [Deprecated APIs](#deprecated-apis).
-
-Get background service cached data with sport-specific intervals.
-
-**Parameters**:
-- `key` (str): Cache key
-- `sport_key` (str, optional): Sport identifier (e.g., 'nhl', 'nba') for live interval lookup
-
-**Returns**: Cached data, or `None` if not found or stale
-
-**Example**:
-```python
-# Uses sport-specific live_update_interval from config
-games = self.cache_manager.get_background_cached_data(
-    "nhl_games",
-    sport_key="nhl"
-)
-```
-
 ### Strategy Methods
 
 #### `get_cache_strategy(data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]`
@@ -912,23 +866,6 @@ strategy = self.cache_manager.get_cache_strategy("sports_live", sport_key="nhl")
 max_age = strategy['max_age']  # Get configured max age
 ```
 
-#### `get_sport_live_interval(sport_key: str) -> int`
-
-> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
-
-Get the live_update_interval for a specific sport from config.
-
-**Parameters**:
-- `sport_key` (str): Sport identifier (e.g., 'nhl', 'nba')
-
-**Returns**: Live update interval in seconds
-
-**Example**:
-```python
-interval = self.cache_manager.get_sport_live_interval("nhl")
-# Returns configured live_update_interval for NHL
-```
-
 #### `get_data_type_from_key(key: str) -> str`
 
 Extract data type from cache key to determine appropriate cache strategy.
@@ -938,17 +875,6 @@ Extract data type from cache key to determine appropriate cache strategy.
 
 **Returns**: Inferred data type string
 
-#### `get_sport_key_from_cache_key(key: str) -> Optional[str]`
-
-> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
-
-Extract sport key from cache key for sport-specific strategies.
-
-**Parameters**:
-- `key` (str): Cache key
-
-**Returns**: Sport identifier, or `None` if not found
-
 ### Utility Methods
 
 #### `clear_cache(key: Optional[str] = None) -> None`
@@ -986,30 +912,6 @@ for file_info in files:
     self.logger.info(f"Cache: {file_info['key']}, Age: {file_info['age_display']}")
 ```
 
-### Metrics Methods (deprecated)
-
-#### `get_cache_metrics() -> Dict[str, Any]`
-
-> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
-
-Get cache performance metrics.
-
-**Returns**: Dictionary with cache statistics (`total_requests`, `cache_hit_rate`, `background_hit_rate`, `api_calls_saved`, `average_fetch_time`, etc.)
-
-**Example**:
-```python
-metrics = self.cache_manager.get_cache_metrics()
-self.logger.info(f"Cache hit rate: {metrics['cache_hit_rate']:.2%}")
-```
-
-#### `get_memory_cache_stats() -> Dict[str, Any]`
-
-> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
-
-Get memory cache statistics.
-
-**Returns**: Dictionary with memory cache stats (size, max_size, etc.)
-
 ---
 
 ## Plugin Manager
@@ -1048,14 +950,6 @@ for plugin_id, plugin in all_plugins.items():
     self.logger.info(f"Plugin {plugin_id} is loaded")
 ```
 
-#### `get_enabled_plugins() -> List[str]`
-
-> Deprecated, removed in 3.8.0 — check `enabled` on the instances in `plugin_manager.plugins`. See [Deprecated APIs](#deprecated-apis).
-
-Get list of enabled plugin IDs.
-
-**Returns**: List of plugin identifier strings
-
 #### `get_plugin_info(plugin_id: str) -> Optional[Dict[str, Any]]`
 
 Get plugin information including manifest and runtime info.
@@ -1137,6 +1031,63 @@ if weather is not None and weather.enabled:
 
 ---
 
+## Fetching data
+
+Use the core helpers for HTTP rather than a `requests.Session` of your own:
+`APIHelper` (`from src.common import APIHelper`) for JSON APIs, and
+`fetch_espn_scoreboard()` (`src.common.espn_dates`) or
+`BackgroundDataService` for ESPN scoreboards. Since the release after 3.7.0
+these go through the core **fetch service** (`src/common/fetch_service.py`),
+so a plugin that uses them gets the following with no code change. Return
+values, exceptions and retries are what they were.
+
+- **Shared connections.** Core sessions with the same retry policy share one
+  connection pool per host, instead of one pool per helper.
+- **Merged requests.** Identical GETs in flight at the same time (same URL
+  and query, headers, timeout and retry policy) go to the network once, and
+  every caller gets its own copy of the response, or the same exception.
+- **Host budgets.** A host can have a token-bucket budget. A request past it
+  waits for a token, but never longer than `max_wait_seconds` (2 s by
+  default). Only ESPN hosts have one by default (20 requests a second, burst
+  200), which normal use never reaches.
+- **Conditional GET.** When a server sends `ETag` or `Last-Modified`, the
+  next identical request revalidates, and a `304 Not Modified` comes back to
+  your code as the original `200` with its body. ESPN currently sends
+  neither, so this does nothing there.
+- **Counters.** Requests, merged requests, bytes, 304s, errors and time spent
+  waiting are counted per plugin and per host, and published for the web UI
+  at `GET /api/v3/plugins/fetch-stats` (see
+  [REST_API_REFERENCE.md](REST_API_REFERENCE.md#get-fetch-statistics)). A
+  request is counted against your plugin when it runs inside your
+  `update()`/`display()`, your constructor or `on_enable()`, or anywhere in
+  code under your plugin's directory, including threads you start.
+
+What is not covered yet: requests a plugin makes with its own `requests.get()`
+or `Session.get()` calls. They work as before but are invisible to the
+budgets and counters.
+
+The settings live in `config.json` under `fetch_service`, read when the
+display starts and on a config reload:
+
+```json
+"fetch_service": {
+    "enabled": true,
+    "max_wait_seconds": 2,
+    "rate_limits": {
+        "*.espn.com": {"per_second": 20, "burst": 200},
+        "api.example.com": {"per_second": 1, "burst": 5}
+    }
+}
+```
+
+`rate_limits` keys are a host or a `*.domain` pattern (which also matches
+the bare domain); `"per_second": 0` removes a budget. `"enabled": false`
+turns the whole service into a plain `session.get()`. Two further switches,
+`"single_flight": false` and `"conditional_get": false`, turn off merging and
+revalidation.
+
+---
+
 ## Best Practices
 
 ### Caching
@@ -1221,13 +1172,19 @@ if weather is not None and weather.enabled:
 
 ## Deprecated APIs
 
-These still work but log a warning the first time they are called
-(`journalctl -u ledmatrix` shows which one), and are **removed in 3.8.0**
-(first announced for 3.7.0, which shipped with them still in place).
-[DEPRECATIONS_3.8.md](DEPRECATIONS_3.8.md) is the usage scan behind that
-decision: which of these the official plugins, the registry's third-party
-plugins and core still call or override. Only methods that scan reports unused
-are removed in 3.8.0; the rest stay until their callers migrate.
+A deprecated method still works but logs a warning the first time it is
+called (`journalctl -u ledmatrix` shows which one), until the release that
+removes it. [DEPRECATIONS_3.8.md](DEPRECATIONS_3.8.md) is the usage scan
+behind each removal: which of the deprecated methods the official plugins,
+the registry's third-party plugins and core still call or override. Only
+methods that scan reports unused are removed; the rest stay until their
+callers migrate.
+
+### Removed in 3.8.0
+
+Deprecated in 3.5.0 with a warning on first call, and gone in 3.8.0:
+the scan found no caller in any official or third-party plugin. Calling one
+now raises `AttributeError`.
 
 | Object | Methods | Instead |
 |---|---|---|

docs/PLUGIN_DEVELOPMENT_GUIDE.md

@@ -519,15 +519,12 @@ When developing plugins, you'll need to use the APIs provided by the LEDMatrix s
 - `draw_text()` - Text rendering. For images, paste directly onto
   `display_manager.image` (a PIL Image) and call `update_display()`;
   there is no `draw_image()` helper method.
-- `draw_weather_icon()`, `draw_sun()`, `draw_cloud()` - Weather icons
-  (deprecated, removed in 3.8.0 — draw your own icons)
 - `get_text_width()`, `get_font_height()` - Text utilities
 - `set_scrolling_state()`, `defer_update()` - Scrolling state management
 
 **Cache Manager** (`self.cache_manager`):
 - `get()`, `set()`, `delete()` - Basic caching
 - `get_cached_data_with_strategy()` - Advanced caching with strategies
-- `get_background_cached_data()` - deprecated, removed in 3.8.0 — use `get()`
 
 **Plugin Manager** (`self.plugin_manager`):
 - `get_plugin()`, `get_all_plugins()` - Access other plugins

docs/README.md

@@ -72,6 +72,8 @@ Going deeper:
 ## Contributing to LEDMatrix itself
 
 - [ARCHITECTURE.md](ARCHITECTURE.md) — processes, display loop, plugin system, web UI; where to start reading
+- [WEB_FRONTEND_ARCHITECTURE.md](WEB_FRONTEND_ARCHITECTURE.md) — the web UI's ES modules, page lifecycle and form model, and the page-by-page migration to them
+- [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md) — the display's control socket: protocol, security model, stage plan
 - [DEVELOPMENT.md](DEVELOPMENT.md) — environment setup
 - [HOW_TO_RUN_TESTS.md](HOW_TO_RUN_TESTS.md) — running the test suite
 - [MULTI_ROOT_WORKSPACE_SETUP.md](MULTI_ROOT_WORKSPACE_SETUP.md) — multi-repo workspace

docs/REST_API_REFERENCE.md

@@ -447,13 +447,23 @@ Request a specific plugin to display on-demand.
     "mode": "nfl_live",
     "duration": 45,
     "pinned": true,
-    "service": { "active": true, "returncode": 0, "stdout": "", "stderr": "" }
+    "service": { "active": true, "returncode": 0, "stdout": "", "stderr": "" },
+    "transport": "socket"
   }
 }
 ```
 
 `service` is `null` when `start_service` is false.
 
+`transport` says how the request reached the display: `"socket"` means the
+display's control socket acknowledged it (it is queued for the render thread;
+see [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), `"mailbox"` means it was
+written to the cache mailbox the display polls, as before the socket existed.
+With `"mailbox"`, `socket_error` gives the reason the socket was not used
+(`no_socket` when the display is stopped or predates the socket, `timeout`,
+`refused`, `busy`, ...). Either way the request is applied the same way;
+`request_id` is the same id in both.
+
 ### Stop On-Demand Display
 
 **POST** `/api/v3/display/on-demand/stop`
@@ -476,11 +486,14 @@ Stop the current on-demand display.
   "status": "success",
   "data": {
     "request_id": "uuid-here",
-    "service": null
+    "service": null,
+    "transport": "socket"
   }
 }
 ```
 
+`transport` and `socket_error` are as for start.
+
 ---
 
 ## Plugins
@@ -967,6 +980,63 @@ Metrics for one plugin; `data` has the same fields as one entry above.
 
 Reset metrics for a plugin.
 
+### Get Fetch Statistics
+
+**GET** `/api/v3/plugins/fetch-stats`
+
+Network requests made through the core fetch service
+(`src/common/fetch_service.py`), per plugin and per host, cumulative since
+the display started. Read-only. The display publishes the counters at most
+once a minute when they change (every 10 minutes otherwise), so they can be
+up to a minute old. Requests a plugin makes with its own `requests` calls,
+outside `APIHelper`, `espn_dates`, `BackgroundDataService` and
+`BaseOddsManager`, are not counted yet.
+
+`data.status` is `live`, `stale` (no publish for longer than
+`stale_after`), `stopped` (the display exited; the last counters are kept)
+or `unknown` (nothing published; `data.data` is `null`).
+
+**Response**:
+```json
+{
+  "status": "success",
+  "data": {
+    "status": "live",
+    "age_seconds": 12.4,
+    "data": {
+      "schema": 1,
+      "running": true,
+      "published_at": 1790000000.0,
+      "stale_after": 720.0,
+      "since": 1789990000.0,
+      "totals": {"requests": 412, "merged": 3, "not_modified": 0,
+                 "errors": 1, "http_errors": 2, "retries": 0,
+                 "throttled": 0, "overruns": 0, "bytes": 18234011,
+                 "wait_seconds": 0.0},
+      "plugins": {
+        "football-scoreboard": {"requests": 240, "merged": 2, "bytes": 9120330,
+                                "hosts": {"site.api.espn.com": 180,
+                                          "sports.core.api.espn.com": 62},
+                                "...": "the other counters, as in totals"}
+      },
+      "hosts": {
+        "site.api.espn.com": {"requests": 301, "...": "as in totals"}
+      },
+      "validators": {"entries": 0, "bytes": 0},
+      "config": {"enabled": true, "single_flight": true,
+                 "conditional_get": true, "max_wait_seconds": 2.0,
+                 "rate_limits": {"*.espn.com": {"per_second": 20.0, "burst": 200.0}}}
+    }
+  }
+}
+```
+
+`requests` counts round trips sent (retries inside the HTTP adapter are in
+`retries`), `merged` requests answered by an identical one already in
+flight, `not_modified` 304s served from the stored body, `errors` transport
+failures and `http_errors` responses with status 400 or above. `bytes` is the
+decoded body size. `core` is everything no plugin made.
+
 ### Get/Set Plugin Limits
 
 **GET** `/api/v3/plugins/limits/<plugin_id>`

docs/RUN_LOOP_REDESIGN.md

@@ -0,0 +1,277 @@
+# Restructuring `DisplayController.run()`
+
+`run()` in [`src/display_controller.py`](../src/display_controller.py) decides
+what the panel shows and runs it. This document is the plan for turning it
+from one long loop into three parts with clear jobs: an **Arbiter** that
+decides, a **ScreenRunner** that runs one screen, and **Sources** that each
+know about one kind of content. It covers the target design, the stages that
+get there, and how each stage is checked.
+
+The goal is to change how the control flow is organised, not to move code
+into more files. Each stage ships as its own PR, and none of them changes
+what the panel shows unless that PR says so and updates the golden traces
+on purpose.
+
+## Why
+
+- **The priority order is written in branch order, twice.** It is
+  Follower, on-demand, WiFi notice, live priority, Vegas, rotation. In
+  `run()` that order exists only as the order of `if` blocks. Vegas
+  repeats part of it in its interrupt callback (`_check_vegas_interrupt`).
+- **Preemption is found by re-checking.** A screen ends early when
+  something else changed `current_display_mode` or `is_display_active`
+  underneath it. `run()` notices with five separate
+  `current_display_mode != active_mode` checks: after an empty pass, in each
+  of the two frame loops, after the frame loops, and before rotating.
+- **Most recent fixes were ordering bugs** between these branches (#618,
+  #644, #649, #652): a lost mode switch, rotating past an on-demand request,
+  spinning when every mode is empty.
+- **It could not be tested** without threads, real sleeps and stopping the
+  loop by raising from a patched method.
+
+## What `run()` does today
+
+Each pass, in order:
+
+1. `loop_pass()` (watchdog). Apply a pending plugin enable/disable.
+2. With no modes: dwell 1 s, next pass.
+3. Poll on-demand requests and expiry, release plugins loaded only for
+   on-demand, tick plugin updates, drop an expired WiFi notice, evaluate
+   the schedule (an on-demand session overrides scheduled-off), apply the
+   brightness target.
+4. **Scheduled off:** blank, dwell up to 60 s. `_blank_while_scheduled_off`
+5. **Follower:** render one frame from the leader. `_run_follower_frame`
+6. **WiFi notice** (unless on-demand): draw it, dwell 0.5 s. `_show_wifi_notice`.
+   It is also polled mid-screen (`_wifi_notice_pending`): the frame loops,
+   the dwell sleep and an interrupted Vegas iteration end within about a
+   second when one arrives, and a screen cut short resumes after it.
+7. **Live priority** (unless on-demand, or Vegas keeps live content in the
+   ticker): switch to the next live mode, or resume the rotation. A game
+   that goes live during a screen is caught sooner, by
+   `_check_live_takeover` in the frame loops and the dwell sleep (at most
+   once a second, and not while a live mode is showing).
+8. **Vegas** (unless on-demand, or live content preempts it): run one
+   iteration of up to `max_cycle_duration`. A completed iteration ends the
+   pass, and so does one that yielded for a WiFi notice or the schedule.
+   Any other interrupted one falls through to step 9 in the same pass.
+9. **One screen:** pick the mode (`_resolve_active_mode`), the plugin
+   (`_plugin_for_mode`), draw the first frame through the executor
+   (`_dispatch_first_frame`). On no content, rotate at once
+   (`_note_empty_pass`, `_skip_failed_plugin_modes`). Otherwise work out the
+   bounds (`_track_dynamic_cycle`, `_resolve_durations`,
+   `_clamp_to_on_demand`) and the frame rate (`_needs_high_fps`), run the
+   125 Hz or 1 Hz frame loop, make up the minimum duration, then pick the
+   next mode (`_advance_after_screen`).
+
+The helpers named above were extracted in stage 1 without changing
+behaviour. The frame loops, the Vegas branch and every early exit are still
+inline in `run()`.
+
+## Target design
+
+```python
+def run(self):
+    while True:
+        inputs = self._drain_inputs()                 # requests, schedule, config, sync
+        plan = self.arbiter.decide(self.state, inputs, clock.now())
+        outcome = self.runner.run(plan)               # ExitReason + elapsed
+        self.state = self.state.after(plan, outcome)  # rotation, on-demand index, live resume
+```
+
+### Sources
+
+Each kind of content is a Source. A Source looks at the state and the
+inputs and either offers a screen or passes. The Arbiter asks them in this
+order:
+
+| Order | Source | Offers a screen when | Today |
+|---|---|---|---|
+| gate | ScheduledOff | the schedule is off and no on-demand session overrides it | step 4 |
+| 1 | Follower | a sync leader is driving this panel | step 5 |
+| 2 | OnDemand | a session is active (its mode list, index, expiry and pin) | `_resolve_active_mode` |
+| 3 | Wifi | a status message is pending and on-demand is not active | step 6 |
+| 4 | Live | a live-priority plugin has live content (round-robin across several) | step 7 |
+| 5 | Vegas | Vegas is enabled and nothing above wants the panel | step 8 |
+| 6 | Rotation | always: `available_modes[current_mode_index]` | step 9 |
+
+ScheduledOff is a gate in front of the Sources because that is how it works
+today: a scheduled-off panel stays blank even for a follower, and only an
+on-demand session overrides it.
+
+### Arbiter
+
+```python
+Arbiter.decide(state, inputs, now) -> ScreenPlan
+```
+
+`decide` is a pure function: it does no I/O, takes no locks and does not
+sleep. It can be tested with plain tables of (state, inputs, now) mapped to
+an expected plan. It returns a `ScreenPlan`:
+
+| Field | Meaning |
+|---|---|
+| `source` | which Source won |
+| `mode`, `plugin` | what to draw (None for a blank or follower plan) |
+| `min_duration`, `max_duration` | from `_resolve_durations` and `_clamp_to_on_demand` |
+| `dynamic` | run until the plugin's cycle completes, between min and max |
+| `frame_policy` | today `_needs_high_fps` (125 Hz or 1 Hz); see stage 5 |
+| `preemptible_by` | the Sources allowed to interrupt this plan mid-screen |
+
+### ScreenRunner
+
+```python
+ScreenRunner(clock: FrameClock).run(plan) -> Outcome(exit_reason, elapsed)
+```
+
+The ScreenRunner draws the first frame (`_dispatch_first_frame`), runs the
+frame loop that the plan's frame policy selects, services pending changes
+between frames, and returns one `ExitReason`:
+
+| ExitReason | Today's equivalent (golden-trace exit) |
+|---|---|
+| `DURATION` | target duration reached (`duration`) |
+| `CYCLE_COMPLETE` | dynamic plugin finished after its minimum (`cycle-complete`) |
+| `EMPTY` | first frame returned False (`empty`; `raised` when display() raised inside the executor) |
+| `ERROR` | the dispatch itself raised (`error`) |
+| `DISPLAY_FALSE` | a later frame returned False (`display-false`) |
+| `PREEMPTED` | another Source took the panel (`on-demand-*`, `schedule-off`, `vegas-interrupt`, ...) |
+
+`PREEMPTED` replaces the five `current_display_mode != active_mode` checks.
+The runner asks the Arbiter, at the throttled service points it already has,
+whether a Source in `plan.preemptible_by` now wants the panel.
+
+`FrameClock` provides `now()` and `sleep()`. In production it is
+`time.monotonic`/`time.sleep`. In the golden traces it is the fake clock
+that the harness patches in today.
+
+## Stages
+
+| Stage | Change | Behaviour change | Verified by |
+|---|---|---|---|
+| 1 | Golden traces; extract helpers from `run()` | none | traces generated on main pass unchanged; mutation check |
+| 2 | Arbiter with Follower and Wifi Sources | none | traces unchanged; Arbiter unit tables; ledpi smoke |
+| 3 | ScreenRunner, FrameClock, ExitReason, `PREEMPTED`; OnDemand, Live, Rotation Sources | none | traces unchanged; ledpi frame soak A/B |
+| 4 | Vegas as a Source driven by `run_frame()` | none intended | traces against the real coordinator; ledpi Vegas soak A/B |
+| 5 | Plugins declare `frame_policy` | DEBUG instead of INFO for the FPS line | traces; soak on a static-heavy rotation |
+
+### Stage 1 (this PR)
+
+- `test/_run_loop_harness.py` builds a real `DisplayController` through
+  `__init__` on in-memory fakes (plugins, cache, config service, plugin
+  manager, sync manager, display manager). It swaps the module's `time` and
+  `datetime` for one fake clock and runs the real `run()` until a horizon.
+  The first frame of each screen still goes through the real
+  `PluginExecutor` and the per-plugin locks.
+- `test/test_run_loop_golden.py` has 15 scenarios, each compared with
+  `test/fixtures/run_loop_golden/<scenario>.json`:
+  - plain rotation (display_durations override, a high-FPS scroller, a
+    plugin whose `display()` takes no `display_mode`)
+  - empty modes and a mode with no plugin; an all-empty rotation (the 1 s
+    pause)
+  - plugin errors and the circuit breaker
+  - dynamic duration (cycle complete, plugin cap, global cap)
+  - live priority taking over and handing back; live round-robin
+  - on-demand start/stop/expiry; pinned on-demand; a session resumed after
+    a restart
+  - schedule off and dim, with an on-demand override during downtime
+  - WiFi notice; sync follower
+  - Vegas, with and without `live_in_ticker`
+- Each trace row is `[start, mode, duration, exit_reason, frames,
+  force_clear]`. The exit reason is the event that decided what came next.
+- All 16 tests run in under a second. The goldens were generated from
+  main's `run()` before any code moved.
+- Vegas uses `FakeVegas`, which implements only the contract the controller
+  depends on: `run_iteration()` returns True after its duration and False
+  when the interrupt or live check asks it to yield, checking at the real
+  coordinator's cadence. Running the real coordinator on the fake clock
+  belongs to stage 4.
+- Twelve helpers were extracted from `run()` (listed under "What `run()`
+  does today"). Breaking any one of them fails at least one golden trace.
+
+### Stage 2: Arbiter, starting with Follower and Wifi
+
+1. Add `ScreenPlan` and an `Arbiter` with the ScheduledOff gate, Follower
+   and Wifi. Every other case returns a `LEGACY` plan, which means "carry on
+   with the existing code" (steps 7-9).
+2. `run()` calls `decide()` after the bookkeeping in step 3 and dispatches
+   on `plan.source`: blank, `_run_follower_frame()`, the WiFi notice, or the
+   existing path. Inputs that Sources read (follower active, the pending
+   WiFi message, schedule state) are collected first, so `decide()` stays
+   pure.
+3. Unit-test `decide()` with tables. The golden traces must not change.
+   The Wifi Source must keep the mid-screen preemption described in step 6
+   of "What `run()` does today".
+
+Follower and Wifi go first because each is one self-contained branch that
+ends the pass. They prove the plumbing without touching the frame loops.
+
+### Stage 3: ScreenRunner and `PREEMPTED`
+
+Move the two frame loops, the make-up dwell and the dynamic-duration exit
+into `ScreenRunner.run(plan)` with an injected `FrameClock`. Replace the
+five re-checks with `PREEMPTED`. Add the OnDemand, Live and Rotation Sources
+so `LEGACY` is left meaning only Vegas.
+
+This stage touches frame pacing (the 8 ms deadline sleep, the 1 ms yield),
+so it needs a frame soak on ledpi, A/B against main. Coordinate with
+whoever owns scroll performance (`docs/SCROLL_PERFORMANCE.md`).
+
+### Stage 4: Vegas as a Source
+
+The controller calls `coordinator.run_frame()` once per frame from the
+ScreenRunner instead of handing over to `run_iteration()` for up to
+`max_cycle_duration`. The interrupt callback and the second copy of the
+priority order go away, because preemption becomes `PREEMPTED`. The
+`vegas-plugin-tick` thread that is spawned every 4 s becomes the
+controller's normal update tick. Extend the harness to drive the real
+coordinator on the fake clock, which means patching its `time` and running
+its prefetch inline. Verify with a Vegas soak on ledpi, A/B.
+
+### Stage 5: `frame_policy`
+
+Plugins declare `frame_policy` (STATIC, PERIODIC(hz), ANIMATED(fps),
+SCROLL). `_needs_high_fps` becomes the mapping for legacy plugins
+(`needs_high_fps`, the `static-image` special case, `enable_scrolling`),
+and its per-screen INFO line drops to DEBUG.
+
+## How each stage is verified
+
+- **Golden traces.** Run `python -m pytest test/test_run_loop_golden.py`;
+  it takes about a second. A refactoring stage must leave every trace
+  unchanged. A deliberate behaviour change regenerates them with
+  `LEDMATRIX_REGEN_GOLDEN=1` in its own commit, and the commit message
+  explains each changed row. A new scenario's golden is generated against
+  main's `run()` first, then checked against the branch.
+- **Mutation check.** Break each moved or new piece once, for example take
+  `max` of the caps instead of `min`, or skip the live hold. At least one
+  trace must fail each time. Stage 1 did this for all twelve helpers.
+- **Full suite.** Diff the FAILED/ERROR ids against a baseline run of main
+  in a separate worktree. The Windows host has a stable set of
+  pre-existing failures, so never compare against zero.
+- **ledpi soak** (stages 2-5). With the service running the branch:
+  `python3 scripts/frame_soak.py --preview` for 10 minutes on a scrolling
+  rotation, and on Vegas for stages 3-4. Alternate which build goes first.
+  Compare late-frame rate and freezes with main. Also check by hand that
+  on-demand start, stop and expiry, a live game taking over and handing
+  back, and the schedule turning the panel off and on all behave as before.
+
+## Behaviour the traces pin down that may be wrong
+
+Stage 1 recorded six behaviours as they were, each to be fixed in its own
+PR that updates the affected trace and explains why. All six are fixed:
+
+- A WiFi notice was only checked between screens, and Vegas yielded to one
+  and then showed a rotation screen instead. Notices now preempt within
+  about a second, and Vegas yields straight to them (#712; `wifi_notice`,
+  `vegas`).
+- A live game only took over between screens, and Vegas yielded to one and
+  then showed a rotation screen first. Games now take over within about a
+  second, and Vegas yields straight to them (#713; `live_priority`,
+  `vegas`).
+- An on-demand session that ended during scheduled-off kept the panel on
+  until the next minute, and a schedule window's end minute counted as on
+  only sometimes. Windows are now half-open `[start, end)`, and the panel
+  blanks as soon as on-demand ends in off hours (#714; `schedule`).
+
+A new one found later goes the same way: record it here with the trace that
+shows it, then fix it in its own PR, not inside a restructure stage.

docs/SCROLL_PERFORMANCE.md

@@ -73,6 +73,10 @@ Sample ladder for a 100 Hz panel:
  100.0 px/s  (1px every 1 refresh  = 100.0 fps, smooth)
 ```
 
+The Vegas **Scroll Speed** slider in the web UI shows the same thing live: a
+line under it says what your speed will run as on this panel, and links to the
+nearest smooth speeds.
+
 ### How a slow speed stays crisp
 
 `SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel
@@ -517,19 +521,25 @@ On the 2×128×64 chain above, which refreshes at about 130 Hz flat out
 
 ### What the display does about it
 
-At one pixel per refresh, the fastest crisp speed, the step is exactly one
-refresh's worth of motion, so it can be cancelled: show one half of the panel
+The step is the motion of one refresh, so it can be cancelled: show one half of the panel
 a refresh behind the other -- the half whose row at the seam lights at the
 start of each refresh. The two rows either side of the seam then show the same
 moment again. What is left is a
 lean of one pixel per half from top to bottom, continuous across the panel,
 which reads as nothing where the step read as a tear. `DisplayManager` does
-this while something scrolls at one frame per refresh
+this while something scrolls
 (`display.scan_order_compensation`, `"auto"` by default, `"off"` to disable;
 the geometry is in `src/scan_order.py`). The lagging rows come from the
 previous frame the display presented, so it works for Vegas and every plugin
 ticker without knowing how they scroll.
 
+A frame held for several refreshes (any crisp speed below the panel's full
+refresh rate, e.g. 60 px/s at 120 Hz) is presented as two swaps instead of one:
+the lagging half shows the previous frame for the first refresh and the new one
+for the rest, so it steps one refresh after the rest rather than one frame.
+That costs a second blit inside the refresh after the first swap, so it is
+skipped when a blit takes more than half a refresh.
+
 Checked on hdpi (4×128×64 on one chain, rotated 180, 2026-09-24) before it was
 written: `scan_mode: 1` (interlaced) made the step vanish but turned moving
 edges grainy, and halving the speed halved it, so it is the scan and not a torn
@@ -537,9 +547,6 @@ frame. With the compensation the step is gone at 90 px/s.
 
 It is left off where the row order is unknown or the maths does not hold:
 
-- **Slower speeds**, where each frame is held for two or more refreshes. The
-  offset there is half a pixel or less, and cancelling it would need a lag of
-  a fraction of a frame.
 - **Other layouts:** pixel mappers other than a 0 or 180 degree rotation
   (U-mapper, 90/270), non-zero `multiplexing`, interlaced `scan_mode`, and a
   canvas remapped to another height (double-sided mode).

docs/SPORTS_UNIFICATION.md

@@ -87,6 +87,10 @@ more. Shared sports code lives in `src/common`:
 | `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
 | `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
 | `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
+| `sports_plugin_host.py` | next release | `SportsPluginHostMixin` — the plugin class's (`manager.py`) identical helpers: Vegas weight, off-thread switch refresh |
+| `sports_live_scroll.py` | next release | `SportsLiveScrollMixin` — rebuild a live scroll strip mid-cycle, keeping the marquee's place |
+| `sports_display_rules.py` | next release | `SportsCardOptionsMixin`, `SportsGameRulesMixin` — scorebug date options, the no-favourites filter, non-favourite live dwell |
+| `sports_font_path.py` | next release | `resolve_font_path` — what the plugins' `_resolve_font_path` copies return |
 
 Each is described in [src/common/README.md](../src/common/README.md).
 
@@ -259,6 +263,41 @@ gave pixel-identical output for all 399 frames (192 harness screens across the
 nine plugins at the eight default sizes, 72 scroll/Vegas cards, 135
 celebration frames), with a parent-vs-parent rerun as the determinism control.
 
+### Stage 4: the identical sweep (core done; adoption waits for a release)
+
+Re-measured on ledmatrix-plugins `56c4f15` (2026-09-30) the report still
+lists 58 families identical in every copy. Stage 4 moves the ones that are
+identical across the nine, or across eight with the ninth lacking the
+method, into four new modules: `sports_plugin_host` (ten `manager.py`
+helpers, all nine), `sports_live_scroll` (eight `manager.py` methods, every
+plugin with a live strip, so not ufc), `sports_display_rules` (four
+`sports.py` methods, in two mixins because their carriers differ) and
+`sports_font_path`. The parity test (`test/test_sports_stage4_parity.py`)
+compares each with every plugin copy using this report's own normalisation,
+plus decorators and constant values, which the normalisation drops.
+
+`_resolve_font_path` was meant to be replaced by
+`font_layout.resolve_asset_path`, but that never looks in the cwd, and the
+plugins' copy does first, so the swap would change which font a process
+started from another checkout loads. `resolve_font_path` is the copy's
+behaviour on a core that ships it, checked path for path against all 17
+copies (`test/test_sports_font_path.py`).
+
+Left in the plugins, though identical:
+
+- `_get_timezone`, `_extract_game_details`, `_fetch_data` (nine): a
+  per-plugin import and the abstract contract, as in stage 3.
+- `_schema_font_size`, `_resolve_font_size` (eight renderers): they read the
+  plugin's own `_SCHEMA_PATH`, as in stage 3.
+- The 29 families carried by seven plugins or fewer: the afl/nrl/soccer
+  lineage's own helpers (`_swrr_advance`, `_refresh_switch_mode_managers`,
+  `_initialize_logo_dir`, ...), the multi-league helpers
+  (`_resolve_managers_for_mode`, `_extract_mode_type`, ...), and eleven
+  two-plugin helpers. Each is one lineage's code; most go when
+  family 13 or 14 reconciles the code around them. `_odds_color` (seven
+  renderers) is already core's, in `SportsHelpersMixin`; a renderer that
+  wants it can inherit that.
+
 ### Why the method changes
 
 Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins
@@ -342,7 +381,7 @@ release.
 
 | # | Family | Methods (variants) | Why here |
 |---|---|---|---|
-| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) is replaced by core's `font_layout.resolve_asset_path` rather than promoted |
+| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) becomes `sports_font_path.resolve_font_path`, not `font_layout.resolve_asset_path`, which skips the cwd. Core side done; see [Stage 4](#stage-4-the-identical-sweep-core-done-adoption-waits-for-a-release) |
 | 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; its seams (`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`) were designed in B1. The pilot for the procedure |
 | 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
 | 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |

docs/WEB_FRONTEND_ARCHITECTURE.md

@@ -0,0 +1,284 @@
+# Web frontend architecture
+
+This page covers where the web UI's JavaScript is going and how it gets
+there one page at a time. The UI is Flask + HTMX + Alpine.js. Templates
+live in `web_interface/templates/v3/` and static files in
+`web_interface/static/v3/`.
+
+Two rules hold at every step:
+
+- **The Pi never builds anything.** It serves the files that are committed.
+  CI builds the generated CSS (`scripts/build_css.py`, see #685) and checks
+  it. The JavaScript needs no build at all: it is native ES modules that the
+  browser loads as they are.
+- **Every page keeps working, and so does every plugin.** Third-party plugin
+  forms, `x-widget` scripts and plugin web UIs use the existing `window.*`
+  names. Each name keeps working as an alias until a release announces that
+  it will be removed.
+
+## Where it started
+
+- About 195 `window.*` globals. Their load order is held together by comments
+  repeated in the headers of `app-early.js`, `app-shell.js` and
+  `plugins_manager.js`.
+- About 5,700 lines of inline `<script>` in the tab partials.
+  `js/htmx-config.js` re-runs every one of them after every htmx swap, so
+  each partial's code had to cope with running twice.
+- The installed-plugin list is kept in four places.
+- Plugin config forms are drawn by the `render_field` macro in
+  `partials/plugin_config.html`, which is about 1,100 lines of Jinja. It
+  duplicates the JS widgets. The server then needs about 430 lines to
+  rebuild JSON from the flat dotted keys the form posts. The soccer form
+  renders to 1.2 MB of HTML.
+- `plugins_manager.js` is 3,800 lines. The owner decided that it needs a
+  namespace refactor before it is split, which is what this plan provides.
+
+## Target
+
+```
+static/v3/js/
+  core/                 ES modules ("type": "module" in core/package.json)
+    boot.js             entry point; base.html loads it with <script type="module">
+    registry.js         page lifecycle: init/destroy on htmx swaps
+    api.js              fetch wrapper for /api/v3 (JSON envelope, login redirect)
+    facade.js           window.LEDMatrix and deprecated aliases
+    (later) escape.js, notify.js, dialog.js, streams.js, visibility.js,
+            store.js (the one installed-plugin store), form/renderer.js
+  pages/                one module per tab partial
+    cache.js            export init(root, ctx), destroy(root, ctx)
+    ...
+```
+
+### The page lifecycle
+
+A converted partial has no `<script>`. Its root element names its page:
+
+```html
+<div class="..." data-page="cache"> ... </div>
+```
+
+`core/boot.js` registers each page with a loader:
+`registry.register('cache', () => import('../pages/cache.js'))`. A page's
+module is fetched only when its partial first appears.
+
+`core/registry.js` handles the rest:
+
+| Event | What the registry does |
+|---|---|
+| `htmx:beforeSwap` (on `document`, so it runs after the body-level handlers that can veto a swap) | If `detail.shouldSwap` is still true, destroys every mounted page inside the swap target |
+| `htmx:afterSwap` | Destroys any mounted page whose root has left the document, then mounts every `data-page` root not mounted yet |
+| `LEDMatrix.pages.refresh()` | Same as afterSwap. `loadPartialDirect` (the no-htmx fallback in `base.html`) calls it |
+| `start()` | Mounts whatever is already on the page. Module scripts run deferred, so a partial may arrive first |
+
+Mounting is idempotent: a root is never initialised twice.
+
+Each mount gets a `ctx` object:
+
+| Field | Contents |
+|---|---|
+| `ctx.root` | The page's root element |
+| `ctx.name` | The page name |
+| `ctx.signal` | An `AbortSignal` that is aborted after `destroy()` |
+| `ctx.state` | A per-mount object for the page's own state |
+| `ctx.api` | Shared service from `boot.js` |
+| `ctx.notify` | Shared service from `boot.js` |
+
+A page that passes `{ signal: ctx.signal }` to `addEventListener` and
+`fetch` needs no teardown code. Its listeners and in-flight requests go
+away when the partial is swapped out. `pages/cache.js` is the worked
+example: its delete buttons use one delegated listener, rows are built with
+`textContent` rather than markup strings, and a newer load supersedes an
+older one.
+
+### One facade
+
+`window.LEDMatrix` is the only global the module code adds:
+
+| Member | What it is |
+|---|---|
+| `api` | `core/api.js`: `get`/`post`/`put`/`del`. Resolves to the JSON body, rejects with an `ApiError` |
+| `pages` | `register`, `refresh`, `list` |
+| `notify(message, type)` | Calls `window.showNotification`, looked up at call time |
+| `escape` | Read-through to `window.LEDEscape` |
+| `widgets` | Read-through to `window.LEDMatrixWidgets` |
+| `deprecate(name, target, replacement)` | Keeps an old `window.*` name working. It warns once in the console, then forwards |
+
+`ApiError` carries `status`, `body`, `network` and `loginRequired`.
+
+`escape`, `widgets` and `notify` are read at call time. The classic scripts
+that define them are deferred, and a plugin may replace them.
+
+Login: `base.html` wraps `window.fetch` before any other script runs, and
+the wrapper sends a 401 with `X-LEDMatrix-Login` to the login page (#683).
+`api.js` calls `window.fetch` at call time, so its requests get the same
+redirect. It also rejects that answer quietly with `loginRequired`, so no
+error message flashes up while the page navigates away.
+
+### Serving modules from the Pi
+
+- **MIME type.** A browser runs a module only when it is served with a
+  JavaScript MIME type. `app.py` pins `.js` and `.mjs` to `text/javascript`
+  rather than trusting the host's mimetypes table, and
+  `test/web_interface/test_es_modules.py` checks it.
+- **Caching.** `url_for` adds `?v=<mtime>` to the entry script, but modules
+  import each other by plain relative URL, without the version. A static
+  `.js` request without `v` is therefore served `Cache-Control: no-cache`
+  (revalidated, so 304 when unchanged) instead of being cached as immutable
+  for a year. Versioned URLs keep the long cache. A later optimisation is an
+  import map that maps each module to its versioned URL.
+- **Load order.** `boot.js` loads after every classic script. Modules are
+  deferred and run in document order with the deferred classic scripts.
+  Nothing classic may depend on a module at load time. A classic script that
+  needs a module service calls `window.LEDMatrix` at run time.
+
+### One form model
+
+`src/plugin_system/field_model.py` provides
+`build_field_model(schema, config, plugin_id)`. It walks a plugin's schema
+once and returns a JSON tree with these keys for each field:
+
+- path, label, help, widget
+- starting value, default
+- constraints, options, secret flag
+- the exact form controls the macro posts today (`inputs`)
+- the JS widget it mounts (`mount`)
+
+`test/test_field_model_parity.py` renders the real macro for every schema it
+can find and checks that the model names the same controls, with the same
+starting values, in the same order, and the same widget mounts. The schemas
+come from `plugin-repos/`, `test/fixtures/plugins/`, the ledmatrix-plugins
+monorepo when a checkout is present, and a synthetic schema that reaches
+every branch of the macro. The test was mutation-checked when it was
+written. Each of these deliberate model bugs makes it fail:
+
+- dropping the checkbox-group sentinel
+- dropping a table's `00:00` time default
+- picking the first matching `<select>` option instead of the last
+- dropping the `None` quirk
+- missing all-hidden objects
+
+The model mirrors the macro's quirks on purpose. The parity run surfaced
+these:
+
+- 83 number fields whose schema default is `null` render `value="None"`.
+- Four array fields name an `x-widget` the core does not ship (`color`,
+  `tag-input`) and fall back to a comma-separated text box.
+- A list-typed `type` uses its first entry, so `["null", "string"]` draws a
+  text box.
+- Eleven objects with no properties and no widget render nothing.
+
+These get fixed once, in the renderer, after the switch below.
+
+## Switching forms to the model, behind a flag
+
+Stage 1 (this change) only proves the model is complete. Rendering does not
+change. The switch is staged so either path can be turned back on at any
+point:
+
+1. **Model endpoint.** `GET /api/v3/plugins/config/model?plugin_id=<id>`
+   returns `build_field_model(schema, prepared_masked_config)`. It uses the
+   same preparation as the partial: defaults merged, secrets masked.
+2. **Renderer module.** `core/form/renderer.js` walks the model. It draws
+   plain fields itself and hands every widget to `LEDMatrixWidgets` through
+   one `mount(el, field)` adapter. The adapter keeps plugin widgets' existing
+   `render(container, config, value, options)` signature (the hard
+   constraint in PRODUCT.md). `getValue()` results are assembled into one
+   JSON object.
+3. **Flag.** `plugin_config.html` renders the macro unless the form-model
+   flag is on. The flag is a `web_interface.form_model` setting in
+   `config.json` (default off), plus a per-browser override
+   (`localStorage.ledmatrixFormModel`) so a tester can compare both paths on
+   one device. With the flag on, the partial renders only a
+   `<div data-page="plugin-config" data-plugin-id="...">` root, and
+   `pages/plugin-config.js` fetches the model and renders it.
+4. **JSON submit.** With the flag on, Save posts
+   `Content-Type: application/json` to the existing
+   `POST /api/v3/plugins/config` JSON path (`plugin_config.py`, `is_json`).
+   That path already validates against the schema and keeps secrets. No
+   dotted keys, no `__rendered_section`, no checkbox reconstruction.
+5. **Save parity test.** This gates turning the flag on by default. For every
+   schema, posting the macro form's data and posting the renderer's JSON
+   must store the same config.
+6. **Retire.** Once the flag has been on by default for a release with no
+   regressions, the macro shrinks to a no-JS fallback for plain fields, and
+   the form-encoded reconstruction (`_parse_form_value_with_schema`,
+   `_set_nested_value`, `_set_missing_booleans_to_false` and friends in
+   `api_v3/__init__.py`) is deleted. The settings search index is then built
+   from the model instead of from rendered HTML.
+
+## Migration order
+
+Smallest and most isolated first. `plugins_manager.js` goes last. Line counts
+are the inline script in each partial today.
+
+| # | Page | Inline JS | Why it is here |
+|---|---|---|---|
+| 1 | Cache (`cache.html`) | 163 lines, now 0 | **Done in stage 1.** One endpoint pair, no globals other pages use. The reference conversion |
+| 2 | Rotation (`durations.html`) | 29 | Tiny. One htmx form |
+| 3 | Operation History | 293 | No globals, read-only list |
+| 4 | Config Editor (`raw_json.html`) | 212 | No globals. CodeMirror is set up and torn down in init/destroy |
+| 5 | Backup & Restore | 232 | 5 globals used only by its own `onclick`s; these become delegated listeners plus deprecated aliases |
+| 6 | Schedule | 193 | 2 globals used as `hx-on` response handlers. Moves `hx-on` handlers into page listeners |
+| 7 | General | 147 | `webLogin` global and the security section. The first page that touches login |
+| 8 | Display | 231 | First page with `LEDVisibility` timers: those move to a `ctx.visibility` service that stops on destroy |
+| 9 | Overview | 410 (4 scripts) | First-run surface: Getting Started, update banner, live preview. Five globals |
+| 10 | WiFi | 364 | `x-data="wifiSetup()"` is defined by its own script. Moves to `Alpine.data()` registered from the module. AP-mode first screen, so it needs the AP-mode test on a real device |
+| 11 | Fonts | 681 | Large, but self-contained (6 globals) |
+| 12 | Logs | 801 | 14 globals, a stream and timers. Uses the visibility service from step 8 |
+| 13 | Tools | 1,022 | 21 globals, MQTT bridge, Pixlet editor, diagnostics polling |
+| 14 | Starlark app config, plugin config (`plugin_config.html`) | 123 + 294 | Plugin panels sit inside an Alpine `x-if` that removes them without an htmx swap. The registry's sweep covers that on the next swap; this step adds a MutationObserver or an `x-if` hook. Then the form-model flag (above) |
+| 15 | Plugin Manager (`plugins.html` + `plugins_manager.js`) | 3,836-line file | Last. Split along the seams that already exist (installed grid, store, registries, Starlark section, on-demand) into `pages/plugins/*.js`. Its 42 globals become aliases. The four installed-plugin stores merge into one `core/store.js`, and `window.installedPlugins` becomes a getter over it |
+
+The shell moves in parallel, a service at a time, with no page depending on
+the order:
+
+| Service | Current home | New module |
+|---|---|---|
+| `showNotification` | 4 versions | `core/notify.js` |
+| The modal helper | `utils/dialog.js` | `core/dialog.js` |
+| SSE streams | `app-shell.js` | `core/streams.js` |
+| `LEDVisibility` | `app-shell.js` | `core/visibility.js` |
+
+Each move leaves the old global as an alias. When the last inline script is
+gone, the script re-execution in `htmx-config.js` and the "HTMX never
+loaded" fallbacks in `base.html` can go too (keep the captive-page path).
+
+## How the tests cover each step
+
+The JS suites live in `test/js` (see `test/js/README.md` and
+`docs/HOW_TO_RUN_TESTS.md`). In CI, the **Web UI JS tests** job installs
+jsdom, starts the web interface and runs `node test/js/run_all.js` with
+`REQUIRE_DOM=1`, so a skipped DOM suite fails the job.
+`test/test_js_unit_suites.py` also runs every unit suite under pytest.
+
+Unit suites need only node. They import the shipped modules directly:
+`core/package.json` and `pages/package.json` mark those directories
+`"type": "module"`.
+
+| Suite | Kind | What it covers |
+|---|---|---|
+| `unit/test_page_registry.js` | Unit, minimal DOM shim | The lifecycle: one init per root, destroy on swap, a veto keeps the page, swaps elsewhere leave it alone, the sweep, lazy loading, a destroy while loading, error containment |
+| `unit/test_core_modules.js` | Unit | `api.js` (envelope, errors, abort, login redirect, path check) and `facade.js` (facade, aliases) |
+| `dom/test_cache_page.js` | DOM: real partial, real API shape | No inline script; one request per swap and per Refresh after five swaps; a cancelled request draws nothing; hostile keys stay text; delete, empty, error, network and login states |
+| `test/web_interface/test_es_modules.py` | pytest | MIME type; `no-cache` without `?v` and immutable with it; `boot.js` loads last; every import resolves inside `core/` and `pages/`; every registered page has its module and exactly one partial root; a converted partial has no `<script>` |
+| `test/test_field_model_parity.py` | pytest | The model against the macro for every available schema |
+
+What each future step adds:
+
+- **A page conversion** adds `dom/test_<page>_page.js`, built like the cache
+  suite: the real partial from the server, the real API's payload shape, N
+  swaps followed by one action that must make exactly one request, the
+  destroy and cancel behaviour, and escaping. `test_es_modules.py` picks up
+  the new page automatically. A unit suite that today slices a function out
+  of a template or `plugins_manager.js` and `eval`s it is rewritten to
+  import the module once that code moves (stage C of the plan).
+- **A shell service move** adds a unit suite for the module and an alias
+  test showing the old global still works.
+- **The form switch** adds the save parity test (macro form data and
+  renderer JSON store the same config, for every schema) and a DOM suite
+  for `pages/plugin-config.js`. Both run with the flag on and off.
+- **`plugins_manager.js`.** The existing DOM suites (`test_installed_dom.js`,
+  `test_store_dom.js`, `test_no_double_fetch.js`) already test the real
+  Plugin Manager in jsdom. They stay green throughout the split and are the
+  gate for it, alongside the unit suites that pin its card rendering and
+  escaping.

mypy-clean.txt

@@ -22,6 +22,7 @@ src/common/api_helper.py
 src/common/bdf_font.py
 src/common/espn_dates.py
 src/common/favorite_team_check.py
+src/common/fetch_service.py
 src/common/font_layout.py
 src/common/frame_timing.py
 src/common/json_body.py
@@ -34,7 +35,11 @@ src/common/snapshot_policy.py
 src/common/sports_card.py
 src/common/sports_card_wrappers.py
 src/common/sports_celebration.py
+src/common/sports_display_rules.py
 src/common/sports_fetch.py
+src/common/sports_font_path.py
+src/common/sports_live_scroll.py
+src/common/sports_plugin_host.py
 src/common/sports_scroll.py
 src/common/sports_timezone.py
 src/common/sports_vegas.py
@@ -46,12 +51,17 @@ src/display_geometry.py
 src/dynamic_team_resolver.py
 src/exceptions.py
 src/font_usage.py
+src/ipc/__init__.py
+src/ipc/client.py
+src/ipc/contract.py
+src/ipc/server.py
 src/logging_config.py
 src/logo_downloader.py
 src/matrix_support.py
 src/pi5_matrix_support.py
 src/plugin_system/__init__.py
 src/plugin_system/compatibility.py
+src/plugin_system/field_model.py
 src/plugin_system/operation_history.py
 src/plugin_system/operation_queue.py
 src/plugin_system/operation_types.py

scripts/sports_drift_report.py

@@ -140,6 +140,14 @@ class _Canonical(ast.NodeTransformer):
         node.annotation = None
         return node
 
+    def visit_AnnAssign(self, node):
+        # ``x: T = v`` is ``x = v``; a bare ``x: T`` does nothing at runtime.
+        self.generic_visit(node)
+        if node.value is None:
+            return None
+        return ast.copy_location(
+            ast.Assign(targets=[node.target], value=node.value), node)
+
 
 class _Folded(_Canonical):
     """Canonical, plus sport names folded out of identifiers and strings."""

src/__init__.py

@@ -4,5 +4,5 @@ LEDMatrix Display System
 Core source package for the LED Matrix Display project.
 """
 
-__version__ = "3.7.0"
+__version__ = "3.8.0"
 

src/background_data_service.py

@@ -27,6 +27,13 @@ from concurrent.futures import ThreadPoolExecutor
 import pytz
 from src.cache_manager import CacheManager
 from src.common.json_body import response_json
+from src.common.fetch_service import (
+    current_plugin_id,
+    fetch_get,
+    get_fetch_service,
+    plugin_scope,
+    share_connection_pool,
+)
 from src.common.espn_dates import (
     RANGE_RETRY_SECONDS,
     _note_range_rejected,
@@ -78,6 +85,9 @@ class FetchRequest:
     commit_claimed: bool = False
     result: Optional[Any] = None
     error: Optional[str] = None
+    # The plugin that submitted the request, so the fetch service counts the
+    # worker's requests against it (fetch_service, caller identity).
+    owner: Optional[str] = None
 
 @dataclass
 class FetchResult:
@@ -119,6 +129,12 @@ class _ConnectionRetryingSession:
     def __init__(self, session):
         self._session = session
 
+    @property
+    def fetch_identity_session(self):
+        """The wrapped Session, whose headers and adapter the fetch service
+        reads to key this request (src/common/fetch_service.py)."""
+        return self._session
+
     def get(self, *args, **kwargs):
         for attempt in range(self.ATTEMPTS):
             try:
@@ -196,9 +212,12 @@ class BackgroundDataService:
         # connection errors three times, a dead network cost up to 16
         # connection attempts per request and held one of the few worker
         # threads for all of them.
+        #
+        # The adapter is the fetch service's shared no-retry one: the same
+        # max_retries=0, with the connection pool shared with the other core
+        # sessions that do not retry (the odds managers).
         self.session = requests.Session()
-        self.session.mount('http://', requests.adapters.HTTPAdapter(max_retries=0))
-        self.session.mount('https://', requests.adapters.HTTPAdapter(max_retries=0))
+        share_connection_pool(self.session, max_retries=0)
         
         # Default headers: core's shared set (real User-Agent, no hand-set
         # Accept-Encoding) -- see src/common/api_helper.py.
@@ -299,6 +318,10 @@ class BackgroundDataService:
         if url.split('?', 1)[0].rstrip('/').endswith('/scoreboard'):
             params = clamp_espn_limit(params)
 
+        # Who asked, resolved on the submitting thread: the worker thread
+        # runs no plugin code, so it could not tell (fetch_service).
+        owner = current_plugin_id()
+
         # Create fetch request
         request = FetchRequest(
             id=request_id,
@@ -311,7 +334,8 @@ class BackgroundDataService:
             timeout=timeout or self.request_timeout,
             max_retries=max_retries,
             priority=priority,
-            callback=callback
+            callback=callback,
+            owner=owner,
         )
         
         with self._lock:
@@ -330,6 +354,7 @@ class BackgroundDataService:
                 self.stats['deduplicated_requests'] = (
                     self.stats.get('deduplicated_requests', 0) + 1
                 )
+                get_fetch_service().note_merged(url, owner)
                 logger.info(
                     "Joined in-flight fetch %s for %s (cache_key=%s) instead of "
                     "starting a duplicate", existing_id, sport, cache_key
@@ -357,6 +382,11 @@ class BackgroundDataService:
         Returns:
             Fetch result with data or error information
         """
+        with plugin_scope(request.owner):
+            return self._fetch_data_worker_scoped(request)
+
+    def _fetch_data_worker_scoped(self, request: FetchRequest) -> FetchResult:
+        """_fetch_data_worker's body, run with the submitter as the caller."""
         start_time = time.time()
         result = FetchResult(request_id=request.id, success=False, retry_count=request.retry_count)
         
@@ -621,8 +651,14 @@ class BackgroundDataService:
         
         for attempt in range(request.max_retries + 1):
             try:
-                response = self.session.get(
+                # Not shared with an identical request in flight: this
+                # service cancels and replaces fetches, and a replacement
+                # must not join the one it replaced. Its own cache_key
+                # dedup already merges what should be merged.
+                response = fetch_get(
+                    self.session,
                     request.url,
+                    share_in_flight=False,
                     params=request.params,
                     headers=request.headers,
                     timeout=request.timeout

src/base_odds_manager.py

@@ -19,6 +19,7 @@ import json
 from typing import Dict, Any, Optional, List, cast
 
 from src.common.api_helper import DEFAULT_HTTP_HEADERS
+from src.common.fetch_service import fetch_get, share_connection_pool
 
 
 
@@ -59,7 +60,13 @@ class BaseOddsManager:
         # Deliberately no retry adapter, unlike api_helper: retries multiply
         # request_timeout, which is set to 5s precisely to stay inside that
         # budget. One try, then the cooldown below.
+        #
+        # Every scoreboard league manager builds one of these, so the session
+        # mounts the fetch service's shared no-retry adapter: the same single
+        # try, over one connection pool per host for all of them instead of
+        # one pool per instance.
         self.session = requests.Session()
+        share_connection_pool(self.session, max_retries=0)
         self.session.headers.update(DEFAULT_HTTP_HEADERS)
         
         # Configuration with defaults
@@ -168,7 +175,7 @@ class BaseOddsManager:
             url = f"{self.base_url}/{sport}/leagues/{espn_league}/events/{event_id}/competitions/{event_id}/odds"
             self.logger.debug(f"Requesting odds from URL: {url}")
             
-            response = self.session.get(url, timeout=self.request_timeout)
+            response = fetch_get(self.session, url, timeout=self.request_timeout)
             response.raise_for_status()
             raw_data = response.json()
 

src/cache_manager.py

@@ -37,7 +37,6 @@ from src.cache.disk_cache import DiskCache
 from src.cache.cache_strategy import CacheStrategy
 from src.cache.cache_metrics import CacheMetrics
 from src.logging_config import get_logger
-from src.deprecation import deprecated
 
 # Canonical implementation lives in src.cache.disk_cache; re-exported here
 # because this module's docstring documents it and external code may import
@@ -408,122 +407,6 @@ class CacheManager:
         """Get the cache directory path."""
         return self.cache_dir
 
-    @deprecated("3.8.0")
-    def has_data_changed(self, data_type: str, new_data: Dict[str, Any]) -> bool:
-        """Check if data has changed from cached version."""
-        cached_data = self.load_cache(data_type)
-        if not cached_data:
-            return True
-
-        if data_type == 'weather':
-            return self._has_weather_changed(cached_data, new_data)
-        elif data_type == 'stocks':
-            return self._has_stocks_changed(cached_data, new_data)
-        elif data_type == 'stock_news':
-            return self._has_news_changed(cached_data, new_data)
-        elif data_type == 'nhl':
-            return self._has_nhl_changed(cached_data, new_data)
-        elif data_type == 'mlb':
-            return self._has_mlb_changed(cached_data, new_data)
-        
-        return True
-
-    def _has_weather_changed(self, cached: Dict[str, Any], new: Dict[str, Any]) -> bool:
-        """Check if weather data has changed."""
-        # Handle new cache structure where data is nested under 'data' key
-        if 'data' in cached:
-            cached = cached['data']
-        
-        # Handle case where cached data might be the weather data directly
-        if 'current' in cached:
-            # This is the new structure with 'current' and 'forecast' keys
-            current_weather = cached.get('current', {})
-            if current_weather and 'main' in current_weather and 'weather' in current_weather:
-                cached_temp = round(current_weather['main']['temp'])
-                cached_condition = current_weather['weather'][0]['main']
-                return (cached_temp != new.get('temp') or 
-                        cached_condition != new.get('condition'))
-        
-        # Handle old structure where temp and condition are directly accessible
-        return (cached.get('temp') != new.get('temp') or 
-                cached.get('condition') != new.get('condition'))
-
-    def _has_stocks_changed(self, cached: Dict[str, Any], new: Dict[str, Any]) -> bool:
-        """Check if stock data has changed."""
-        if not self._is_market_open():
-            return False
-        return cached.get('price') != new.get('price')
-
-    def _has_news_changed(self, cached: Dict[str, Any], new: Dict[str, Any]) -> bool:
-        """Check if news data has changed."""
-        # Handle both dictionary and list formats
-        if isinstance(new, list):
-            # If new data is a list, cached data should also be a list
-            if not isinstance(cached, list):
-                return True
-            # Compare lengths and content
-            if len(cached) != len(new):
-                return True
-            # Compare titles since they're unique enough for our purposes
-            cached_titles = set(item.get('title', '') for item in cached)
-            new_titles = set(item.get('title', '') for item in new)
-            return cached_titles != new_titles
-        else:
-            # Original dictionary format handling
-            cached_headlines = set(h.get('id') for h in cached.get('headlines', []))
-            new_headlines = set(h.get('id') for h in new.get('headlines', []))
-            return not cached_headlines.issuperset(new_headlines)
-
-    def _has_nhl_changed(self, cached: Dict[str, Any], new: Dict[str, Any]) -> bool:
-        """Check if NHL data has changed."""
-        return (cached.get('game_status') != new.get('game_status') or
-                cached.get('score') != new.get('score'))
-
-    def _has_mlb_changed(self, cached: Dict[str, Any], new: Dict[str, Any]) -> bool:
-        """Check if MLB game data has changed."""
-        if not cached or not new:
-            return True
-            
-        # Check if any games have changed status or score
-        for game_id, new_game in new.items():
-            cached_game = cached.get(game_id)
-            if not cached_game:
-                return True
-                
-            # Check for score changes
-            if (new_game['away_score'] != cached_game['away_score'] or 
-                new_game['home_score'] != cached_game['home_score']):
-                return True
-                
-            # Check for status changes
-            if new_game['status'] != cached_game['status']:
-                return True
-                
-            # For live games, check inning and count
-            if new_game['status'] == 'in':
-                if (new_game['inning'] != cached_game['inning'] or 
-                    new_game['inning_half'] != cached_game['inning_half'] or
-                    new_game['balls'] != cached_game['balls'] or 
-                    new_game['strikes'] != cached_game['strikes'] or
-                    new_game['bases_occupied'] != cached_game['bases_occupied']):
-                    return True
-                    
-        return False
-
-    def _is_market_open(self) -> bool:
-        """Check if the US stock market is currently open."""
-        return self._strategy_component.is_market_open()
-
-    @deprecated("3.8.0", "use set()")
-    def update_cache(self, data_type: str, data: Dict[str, Any]) -> bool:
-        """Update cache with new data."""
-        cache_data = {
-            # Header first; see DiskCache's stale check.
-            'timestamp': time.time(),
-            'data': data,
-        }
-        return self.save_cache(data_type, cache_data)
-
     def get(self, key: str, max_age: Optional[int] = 300,
             memory_ttl: Optional[int] = None) -> Optional[Dict[str, Any]]:
         """Get data from cache if it exists and is not stale.
@@ -564,42 +447,6 @@ class CacheManager:
         cache_data['data'] = data
         self.save_cache(key, cache_data)
 
-    @deprecated("3.8.0")
-    def setup_persistent_cache(self) -> bool:
-        """
-        Set up a persistent cache directory with proper permissions.
-        This should be run once with sudo to create the directory.
-        """
-        try:
-            # Try to create /var/cache/ledmatrix with proper permissions
-            from pathlib import Path
-            from src.common.permission_utils import (
-                ensure_directory_permissions,
-                get_cache_dir_mode
-            )
-            cache_dir = '/var/cache/ledmatrix'
-            cache_dir_path = Path(cache_dir)
-            ensure_directory_permissions(cache_dir_path, get_cache_dir_mode())
-            
-            # Set ownership to the real user (not root)
-            real_user = os.environ.get('SUDO_USER')
-            if real_user:
-                import pwd
-                try:
-                    uid = pwd.getpwnam(real_user).pw_uid
-                    gid = pwd.getpwnam(real_user).pw_gid
-                    os.chown(cache_dir, uid, gid)
-                    self.logger.info(f"Set ownership of {cache_dir} to {real_user}")
-                except (OSError, KeyError) as e:
-                    self.logger.warning(f"Could not set ownership for {cache_dir}: {e}", exc_info=True)
-            
-            self.logger.info(f"Successfully set up persistent cache directory: {cache_dir}")
-            return True
-            
-        except (OSError, IOError, PermissionError) as e:
-            self.logger.error(f"Failed to set up persistent cache directory {cache_dir}: {e}", exc_info=True)
-            return False
-    
     def cleanup_disk_cache(self, force: bool = False) -> Dict[str, Any]:
         """
         Clean up expired disk cache files based on retention policies.
@@ -776,14 +623,6 @@ class CacheManager:
         else:
             self.logger.info("Disk cache cleanup thread stopped successfully") 
 
-    @deprecated("3.8.0")
-    def get_sport_live_interval(self, sport_key: str) -> int:
-        """
-        Get the live_update_interval for a specific sport from config.
-        Falls back to default values if config is not available.
-        """
-        return self._strategy_component.get_sport_live_interval(sport_key)
-
     def get_cache_strategy(self, data_type: str, sport_key: Optional[str] = None) -> Dict[str, Any]:
         """
         Get cache strategy for different data types.
@@ -798,13 +637,6 @@ class CacheManager:
         """
         return self._strategy_component.get_data_type_from_key(key)
 
-    @deprecated("3.8.0")
-    def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:
-        """
-        Extract sport key from cache key to determine appropriate live_update_interval.
-        """
-        return self._strategy_component.get_sport_key_from_cache_key(key)
-
     def get_cached_data_with_strategy(self, key: str, data_type: str = 'default') -> Optional[Dict[str, Any]]:
         """
         Get data from cache using data-type-specific strategy.
@@ -838,58 +670,6 @@ class CacheManager:
         data_type = self.get_data_type_from_key(key)
         return self.get_cached_data_with_strategy(key, data_type)
 
-    @deprecated("3.8.0", "use get()")
-    def get_background_cached_data(self, key: str, sport_key: Optional[str] = None) -> Optional[Dict[str, Any]]:
-        """
-        Get data from background service cache with appropriate strategy.
-        This method is specifically designed for Recent/Upcoming managers
-        to use data cached by the background service.
-        
-        Args:
-            key: Cache key to retrieve
-            sport_key: Sport key for determining appropriate cache strategy
-            
-        Returns:
-            Cached data if available and fresh, None otherwise
-        """
-        # Determine the appropriate cache strategy
-        data_type = self.get_data_type_from_key(key)
-        strategy = self.get_cache_strategy(data_type, sport_key)
-        
-        # For Recent/Upcoming managers, we want to use the background service cache
-        # which should have longer TTLs than the individual manager caches
-        max_age = strategy['max_age']
-        memory_ttl = strategy.get('memory_ttl', max_age)
-        
-        # Get the cached data
-        cached_data = self.get_cached_data(key, max_age, memory_ttl)
-        
-        if cached_data:
-            # Record cache hit for performance monitoring
-            self.record_cache_hit('background')
-            # Unwrap if stored in { 'data': ..., 'timestamp': ... } format
-            if isinstance(cached_data, dict) and 'data' in cached_data:
-                return cached_data['data']
-            return cached_data
-        
-        # Record cache miss for performance monitoring
-        self.record_cache_miss('background')
-        return None
-
-    @deprecated("3.8.0", "use get()")
-    def is_background_data_available(self, key: str, sport_key: Optional[str] = None) -> bool:
-        """
-        Check if background service has fresh data available.
-        This helps Recent/Upcoming managers determine if they should
-        wait for background data or fetch immediately.
-        """
-        data_type = self.get_data_type_from_key(key)
-        strategy = self.get_cache_strategy(data_type, sport_key)
-        
-        # Check if we have data that's still fresh according to background service TTL
-        cached_data = self.get_cached_data(key, strategy['max_age'])
-        return cached_data is not None
-
     def generate_sport_cache_key(self, sport: str, date_str: Optional[str] = None) -> str:
         """
         Centralized cache key generation for sports data.
@@ -906,45 +686,8 @@ class CacheManager:
             date_str = datetime.now(pytz.utc).strftime('%Y%m%d')
         return f"{sport}_{date_str}"
 
-    @deprecated("3.8.0")
-    def record_cache_hit(self, cache_type: str = 'regular') -> None:
-        """Record a cache hit for performance monitoring."""
-        self._metrics_component.record_hit(cache_type)
-
-    @deprecated("3.8.0")
-    def record_cache_miss(self, cache_type: str = 'regular') -> None:
-        """Record a cache miss for performance monitoring."""
-        self._metrics_component.record_miss(cache_type)
-
-    @deprecated("3.8.0")
-    def record_fetch_time(self, duration: float) -> None:
-        """Record fetch operation duration for performance monitoring."""
-        self._metrics_component.record_fetch_time(duration)
-
-    @deprecated("3.8.0")
-    def get_cache_metrics(self) -> Dict[str, Any]:
-        """Get current cache performance metrics."""
-        return self._metrics_component.get_metrics()
-
-    @deprecated("3.8.0")
-    def log_cache_metrics(self) -> None:
-        """Log current cache performance metrics."""
-        self._metrics_component.log_metrics()
-    
-    @deprecated("3.8.0")
-    def get_memory_cache_stats(self) -> Dict[str, Any]:
-        """
-        Get statistics about the memory cache.
-        
-        Returns:
-            Dictionary with memory cache statistics
-        """
-        return self._memory_cache_component.get_stats()
-    
     def log_memory_cache_stats(self) -> None:
         """Log current memory cache statistics."""
-        # Not get_memory_cache_stats(): that is deprecated, and core must not
-        # trip its own deprecation warning every time memory logging runs.
         stats = self._memory_cache_component.get_stats()
         self.logger.info(f"Memory Cache - Size: {stats['size']}/{stats['max_size']} "
                         f"({stats['usage_percent']:.1f}%), "

src/common/README.md

@@ -27,6 +27,7 @@ Rules for the package:
 | [`bdf_font`](#bdf_font) | Load and draw BDF bitmap fonts | Yes, if drawing BDF text directly | 3.5.0 |
 | [`espn_dates`](#espn_dates) | Fetch ESPN scoreboards across a date range | Yes (scoreboards) | 3.5.0 |
 | [`favorite_team_check`](#favorite_team_check) | Log why a favourite team code shows nothing | Yes (scoreboards) | 3.6.0 |
+| [`fetch_service`](#fetch_service) | Pooled, merged, budgeted and counted HTTP for core fetch paths | No, core-internal (reached through `api_helper` and `espn_dates`) | n/a |
 | [`font_layout`](#font_layout) | Reproducible TrueType loading, crisp sizes | Yes | 3.4.0 |
 | [`frame_timing`](#frame_timing) | Timing of every presented frame, stall watchdog | No, core-internal | n/a |
 | [`json_body`](#json_body) | Parse a response body as JSON, with orjson if installed | Optional (large payloads) | 3.5.0 |
@@ -40,9 +41,13 @@ Rules for the package:
 | [`sports_card`](#sports_card) | Scoreboard card settings, colours, fonts, dates | Yes (scoreboards) | 3.3.0 |
 | [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
 | [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
+| [`sports_display_rules`](#sports_display_rules) | Which games a scoreboard shows, for how long, and its scorebug date line | Yes (scoreboards) | 3.8.0 |
 | [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
+| [`sports_font_path`](#sports_font_path) | Find a scoreboard's bundled font whatever the cwd | Yes (scoreboards) | 3.8.0 |
 | [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
 | [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
+| [`sports_live_scroll`](#sports_live_scroll) | Rebuild a live scroll strip mid-cycle without moving it | Yes (scoreboards) | 3.8.0 |
+| [`sports_plugin_host`](#sports_plugin_host) | Helpers of a scoreboard's plugin class (`manager.py`) | Yes (scoreboards) | 3.8.0 |
 | [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 |
 | [`sports_shared`](#sports_shared) | Sport-independent `sports.py` methods | Yes (scoreboards) | 3.3.0 |
 | [`sports_vegas`](#sports_vegas) | Live Vegas cards: keys, card cache, sticky odds, finished games | Yes (scoreboards) | 3.8.0 |
@@ -104,7 +109,9 @@ and truncates results when `limit` is above 500. `fetch_espn_scoreboard()`
 splits a range into month and day requests ESPN accepts and merges the
 results; `espn_date_chunks()`, `fetch_espn_date_chunks()`,
 `clamp_espn_limit()` and `merge_scoreboard_payloads()` are the pieces.
-Scoreboard plugins also bundle a copy for older cores.
+Every request goes through [`fetch_service`](#fetch_service), the chunks
+counted against the plugin that asked. Scoreboard plugins also bundle a copy
+for older cores.
 
 ### favorite_team_check
 
@@ -117,6 +124,23 @@ says the league has nothing on yet; `reset()` re-arms it after a config edit.
 Diagnostics only: every failure is swallowed. Scoreboard plugins also bundle
 a copy for older cores.
 
+### fetch_service
+
+[`fetch_service.py`](fetch_service.py). Core-internal for now. Every core
+fetch path -- `APIHelper.get`/`post`, `espn_dates` (so every scoreboard's
+ESPN scoreboard fetch and `SportsFetchMixin`), `BackgroundDataService` and
+`BaseOddsManager` -- calls `fetch_get(session, url, ...)` instead of
+`session.get(url, ...)`. Same arguments, return value and exceptions; on top
+it shares one connection pool per host per retry policy
+(`share_connection_pool`), merges identical GETs in flight, applies per-host
+token buckets (`fetch_service.rate_limits` in config.json; ESPN gets 20/s,
+burst 200), revalidates with server-sent `ETag`/`Last-Modified` and counts
+requests per plugin and per host. The display publishes the counters
+(`FetchStatsPublisher`) for `GET /api/v3/plugins/fetch-stats`. Which plugin
+made a request comes from `plugin_scope()`, set by the plugin executor, or
+else from the plugin directory on the stack. See
+[docs/PLUGIN_API_REFERENCE.md](../../docs/PLUGIN_API_REFERENCE.md#fetching-data).
+
 ### font_layout
 
 [`font_layout.py`](font_layout.py). `load_truetype(path, size)` is
@@ -239,6 +263,16 @@ The colour helpers are free functions (`logo_palette()`, `lift_color()`,
 `mix_color()`, ...). Deciding *when* to celebrate stays in the plugin, which
 builds the celebration dict the docstring describes.
 
+### sports_display_rules
+
+[`sports_display_rules.py`](sports_display_rules.py). Two `SportsCore`
+mixins: `SportsCardOptionsMixin` (`_card_option()`, which never lets the
+upcoming scorebug lose both its date and time, and `_recent_date_text()`;
+list it before `SportsCoreSharedMixin`) and `SportsGameRulesMixin`
+(`_filtered_or_all()`, the no-favourites quality filter that fails open, and
+`_effective_live_duration()`, the shorter dwell for a non-favourite live
+game).
+
 ### sports_fetch
 
 [`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
@@ -247,6 +281,13 @@ methods that decide which requests a scoreboard makes --
 `_background_fetches_espn_ranges()`, `_needs_previous_day()` (the live
 lookback) and `_wants_live_odds()` (odds only for games near the screen).
 
+### sports_font_path
+
+[`sports_font_path.py`](sports_font_path.py). `resolve_font_path(path)`: the
+path as given when it exists (relative to the cwd), else
+`font_layout.resolve_asset_path(path)`. What the scoreboards'
+`_resolve_font_path` copies return on a core that ships it.
+
 ### sports_game_renderer
 
 [`sports_game_renderer.py`](sports_game_renderer.py).
@@ -264,6 +305,25 @@ what differs.
 `_odds_color` and `_upcoming_date_and_time_text` under their existing names.
 Nothing in core uses it.
 
+### sports_live_scroll
+
+[`sports_live_scroll.py`](sports_live_scroll.py). `SportsLiveScrollMixin`:
+keeps a live scroll strip current. It fingerprints the live games (the clock
+and the display pipeline's own keys excluded, via the host's
+`LIVE_VOLATILE_FIELDS`), rebuilds when they change, rate-limited by what a
+rebuild costs, and `_preserving_scroll_position()` keeps the marquee where
+it was. Pairs with `SportsPluginHostMixin`, whose `_dispatch_switch_refresh()`
+it uses.
+
+### sports_plugin_host
+
+[`sports_plugin_host.py`](sports_plugin_host.py). `SportsPluginHostMixin`:
+helpers of a scoreboard's `BasePlugin` subclass. `get_vegas_priority_weight()`
+(more Vegas slots while a favourite plays, found across every plugin's data
+shape), `_dispatch_switch_refresh()` (a manager refresh on a daemon thread, so
+`display()` never waits on the network), `get_vegas_content_type()` and small
+dynamic-duration helpers. List it before `BasePlugin`.
+
 ### sports_scroll
 
 [`sports_scroll.py`](sports_scroll.py). `SportsScrollDisplay` and

src/common/api_helper.py

@@ -11,10 +11,10 @@ import time
 from datetime import datetime
 from types import MappingProxyType
 from src.common.espn_dates import ESPN_MAX_LIMIT
+from src.common.fetch_service import fetch_get, fetch_post, share_connection_pool
 from typing import TYPE_CHECKING, Any, Dict, Mapping, Optional, cast
 
 import requests
-from requests.adapters import HTTPAdapter
 from urllib3.util.retry import Retry
 
 if TYPE_CHECKING:
@@ -45,7 +45,11 @@ class APIHelper:
 
     - Requests go through one ``requests.Session`` that retries GET, HEAD
       and OPTIONS on 429 and 5xx with exponential backoff, and sends
-      :data:`DEFAULT_HTTP_HEADERS`.
+      :data:`DEFAULT_HTTP_HEADERS`. Its connection pool is shared with every
+      other helper using the same retry policy, and requests go through the
+      core fetch service (``src/common/fetch_service.py``): identical GETs in
+      flight are merged, hosts with a budget are paced, and requests are
+      counted per plugin. Return values and errors are unchanged.
     - Consecutive requests from one helper are spaced at least
       ``set_rate_limit()`` seconds apart (1 second by default). A cache hit
       does not count.
@@ -81,9 +85,10 @@ class APIHelper:
             status_forcelist=[429, 500, 502, 503, 504],
             allowed_methods=["GET", "HEAD", "OPTIONS"]
         )
-        adapter = HTTPAdapter(max_retries=retry_strategy)
-        self.session.mount("https://", adapter)
-        self.session.mount("http://", adapter)
+        # The shared adapter for this retry policy: the same retries as a
+        # private HTTPAdapter(max_retries=retry_strategy), with the connection
+        # pool shared by every helper (fetch_service).
+        share_connection_pool(self.session, retry_strategy)
         
         self.session.headers.update({**DEFAULT_HTTP_HEADERS, 'Connection': 'keep-alive'})
         
@@ -128,7 +133,8 @@ class APIHelper:
                 request_headers.update(headers)
             
             # Make request
-            response = self.session.get(
+            response = fetch_get(
+                self.session,
                 url, 
                 params=params,
                 headers=request_headers,
@@ -255,7 +261,8 @@ class APIHelper:
             if headers:
                 request_headers.update(headers)
             
-            response = self.session.post(
+            response = fetch_post(
+                self.session,
                 url,
                 data=data,
                 json=json_data,

src/common/espn_dates.py

@@ -32,6 +32,7 @@ scoreboards ask every 30 seconds. After that the range is tried again, so the
 workaround retires itself if ESPN reverts.
 """
 
+import contextvars
 import threading
 import time
 from concurrent.futures import ThreadPoolExecutor
@@ -47,6 +48,21 @@ except ImportError:
     def response_json(response: Any) -> Any:
         return response.json()
 
+try:
+    # The core fetch service: counts, per-host budget, merging of identical
+    # requests. Same call, same result and errors as ``session.get``.
+    from src.common.fetch_service import fetch_get, pinned_caller
+except ImportError:
+    # Bundled copies on cores without it call the session directly.
+    import contextlib
+
+    def fetch_get(session: Any, url: str, *, share_in_flight: bool = True,
+                  **kwargs: Any) -> Any:
+        return session.get(url, **kwargs)
+
+    def pinned_caller() -> Any:
+        return contextlib.nullcontext()
+
 # Above this, ESPN returns a truncated list instead of an error. See module
 # docstring: 500 is the largest value measured to return complete data.
 ESPN_MAX_LIMIT = 500
@@ -195,7 +211,8 @@ def _fetch_one_chunk(
     logged and swallowed here rather than raised to the gather below.
     """
     try:
-        response = session.get(
+        response = fetch_get(
+            session,
             url,
             params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
             headers=headers,
@@ -220,6 +237,10 @@ def _fetch_chunks(
     callers keep ``chunks`` order from the returned list -- but it does mean
     the session is shared across threads, which is why this only ever issues
     GETs and never touches session state.
+
+    Each chunk runs in a copy of the caller's context, with the caller pinned
+    into it, so the fetch service counts the chunks against the plugin that
+    asked for the range rather than against the core.
     """
     if not chunks:
         return []
@@ -229,10 +250,15 @@ def _fetch_chunks(
     if len(chunks) == 1:
         return [fetch(chunks[0])]
     workers = min(ESPN_CHUNK_WORKERS, len(chunks))
+    with pinned_caller():
+        # One copy per chunk: a Context cannot be entered by two threads.
+        contexts = [contextvars.copy_context() for _ in chunks]
     with ThreadPoolExecutor(
         max_workers=workers, thread_name_prefix="espn-chunk",
     ) as pool:
-        return list(pool.map(fetch, chunks))
+        futures = [pool.submit(context.run, fetch, chunk)
+                   for context, chunk in zip(contexts, chunks)]
+        return [future.result() for future in futures]
 
 
 def fetch_espn_date_chunks(
@@ -363,7 +389,7 @@ def fetch_espn_scoreboard(
         # real error to log, without spending the chunks a second time.
         chunks_tried = True
 
-    response = session.get(url, params=params, headers=headers, timeout=timeout)
+    response = fetch_get(session, url, params=params, headers=headers, timeout=timeout)
     if is_range and response.status_code == 400 and not chunks_tried:
         _note_range_rejected()
         if logger:

src/common/scroll_config.py

@@ -157,7 +157,13 @@ def crisp_ladder(
 #: when 30 was asked for -- being 11% slow is worth far less than looking bad.
 _STEP_PENALTY = 0.05
 _SLOW_FPS_PENALTY = 0.25   # below 20fps
-_LOWISH_FPS_PENALTY = 0.10  # below 25fps
+_LOWISH_FPS_PENALTY = 0.16  # below 30fps, i.e. "slightly stepped"
+# Up to 30fps, matching CrispSpeed.steppiness: a measured 125.7Hz panel makes
+# 50.3px/s (2px every 5 refreshes) 25.1fps, which a 25fps cutoff let through.
+# 0.16, not less: asked for 50px/s on a 120Hz panel, 48px/s (2px every 5
+# refreshes, 24fps) costs 0.04 + 0.05 + this, and has to lose to both 60px/s
+# and 40px/s (1px, smooth, 20% off = 0.20). At 0.10 it won and shipped a
+# visibly stepped scroll to anyone asking for the default.
 
 
 def _quality_cost(candidate: "CrispSpeed", target: float) -> float:
@@ -173,7 +179,7 @@ def _quality_cost(candidate: "CrispSpeed", target: float) -> float:
     fps = candidate.frames_per_second
     if fps < 20:
         cost += _SLOW_FPS_PENALTY
-    elif fps < 25:
+    elif fps < 30:
         cost += _LOWISH_FPS_PENALTY
     return cost
 
@@ -474,3 +480,61 @@ def refresh_hz_from_config(global_config: Optional[Dict[str, Any]]) -> float:
     if not isinstance(hardware, dict):
         return DEFAULT_REFRESH_HZ
     return _coerce(hardware.get("limit_refresh_rate_hz")) or DEFAULT_REFRESH_HZ
+
+
+#: Smooth options offered next to a speed that is not one itself.
+_ADVICE_ALTERNATIVES = 2
+
+
+def speed_advice(
+    requested_pixels_per_second: float,
+    refresh_hz: float,
+    min_pixels_per_second: float = MIN_PIXELS_PER_SECOND,
+    max_pixels_per_second: float = MAX_PIXELS_PER_SECOND,
+) -> Dict[str, Any]:
+    """What the panel will do with a requested speed, for showing in a UI.
+
+    ``applied`` is what :func:`solve_crisp` picks, i.e. what really runs.
+    ``smooth`` is true when that is single-pixel-ish, 30fps-or-better motion.
+    ``alternatives`` are the smooth ladder entries nearest the request inside
+    the given range, for a click-to-apply suggestion; empty when the request
+    already is one.
+    """
+    hz = _coerce(refresh_hz) or DEFAULT_REFRESH_HZ
+    requested = max(MIN_PIXELS_PER_SECOND,
+                    min(MAX_PIXELS_PER_SECOND, _coerce(requested_pixels_per_second) or 0.0))
+    applied = solve_crisp(requested, hz)
+
+    def as_dict(c: CrispSpeed) -> Dict[str, Any]:
+        return {
+            "pixels_per_second": round(c.pixels_per_second, 1),
+            "pixels_per_frame": c.pixels_per_frame,
+            "frame_hold": c.frame_hold,
+            "frames_per_second": round(c.frames_per_second, 1),
+            "steppiness": c.steppiness,
+        }
+
+    smooth_ladder = [
+        c for c in crisp_ladder(hz)
+        if c.steppiness == "smooth"
+        and min_pixels_per_second <= c.pixels_per_second <= max_pixels_per_second
+    ]
+    # 2%: a UI hands over whole numbers, and 63 asked of a 62.9 px/s panel is
+    # as good as exact.
+    exact = abs(applied.pixels_per_second - requested) <= max(0.05, 0.02 * requested)
+    smooth = applied.steppiness == "smooth"
+    alternatives: List[CrispSpeed] = []
+    if not (exact and smooth):
+        alternatives = sorted(
+            smooth_ladder,
+            key=lambda c: abs(c.pixels_per_second - requested),
+        )[:_ADVICE_ALTERNATIVES]
+        alternatives.sort(key=lambda c: c.pixels_per_second)
+    return {
+        "requested": round(requested, 1),
+        "refresh_hz": round(hz, 1),
+        "applied": as_dict(applied),
+        "exact": exact,
+        "smooth": smooth,
+        "alternatives": [as_dict(c) for c in alternatives],
+    }

src/common/scroll_helper.py

@@ -18,7 +18,7 @@ Features:
 import logging
 import math
 import time
-from typing import Optional, Dict, Any
+from typing import Optional, Dict, Any, List, Tuple
 from PIL import Image
 import numpy as np
 
@@ -29,6 +29,15 @@ import numpy as np
 FPS_LOG_INTERVAL = 5.0
 
 
+def _rgb_pixels(item) -> np.ndarray:
+    """An appended item's pixels as an RGB array, as pasting it would draw them."""
+    if isinstance(item, np.ndarray):
+        return item
+    if item.mode != 'RGB':
+        item = item.convert('RGB')
+    return np.asarray(item)
+
+
 def frame_stats(frame_times: list) -> Dict[str, Any]:
     """Summary statistics over one window of frame durations (seconds).
 
@@ -114,6 +123,18 @@ class ScrollHelper:
         self.cached_image = None   # see the property below
         self.cached_array: Optional[np.ndarray] = None  # Numpy array cache for fast operations
         self.total_scroll_width = 0
+        # An extended strip lives in a buffer with spare room after it, and
+        # cached_array is a view of the buffer's live columns: an append writes
+        # only the new columns, and a trim only moves the view's start. See
+        # append_content. _strip_view is the view this helper last made; a
+        # cached_array that is anything else was set from outside and is not
+        # written through.
+        self._strip_buffer: Optional[np.ndarray] = None
+        self._strip_view: Optional[np.ndarray] = None
+        self._strip_start = 0
+        #: Bytes the last append_content / drop_scrolled_prefix copied, for
+        #: frame-timing attribution (src/common/frame_timing.py note_op).
+        self.last_copy_bytes = 0
         
         # Pre-allocated buffer for output frame (reused to avoid allocations)
         self._frame_buffer: Optional[np.ndarray] = None
@@ -249,6 +270,7 @@ class ScrollHelper:
             self.total_scroll_width = 0
             self.cached_image = Image.new('RGB', (self.display_width, self.display_height), (0, 0, 0))
             self.cached_array = np.array(self.cached_image)
+            self._forget_strip_buffer()
             self.scroll_position = 0.0
             self.total_distance_scrolled = 0.0
             self.scroll_complete = False
@@ -284,6 +306,7 @@ class ScrollHelper:
         self.cached_image = full_image
         # Convert to numpy array for fast operations
         self.cached_array = np.array(full_image)
+        self._forget_strip_buffer()
         actual_image_width = full_image.width
         self.total_scroll_width = actual_image_width
         
@@ -680,7 +703,10 @@ class ScrollHelper:
         strip also defers completion, which is the intent.
 
         Args:
-            content_items: Images to append, in order
+            content_items: Images to append, in order. An item may instead be
+                its pixels already as an RGB array (``np.asarray`` of an RGB
+                image), so a caller can do that conversion off the render
+                thread (Vegas prepares its blocks with the group).
             item_gap: Gap between appended items, and between the existing
                 content and the first appended item
             element_gap: Extra gap after each item, mirroring
@@ -695,40 +721,100 @@ class ScrollHelper:
         if self.cached_array is None or not self.has_strip():
             # Nothing to extend yet — this is just the first build.
             self.create_scrolling_image(
-                content_items, item_gap=item_gap, element_gap=element_gap, lead_gap=0)
+                [Image.fromarray(item) if isinstance(item, np.ndarray) else item
+                 for item in content_items],
+                item_gap=item_gap, element_gap=element_gap, lead_gap=0)
             return True
 
         gap = max(0, item_gap)
-        addition_width = (
-            sum(img.width for img in content_items)
-            + gap * len(content_items)          # one leading gap per item
-            + element_gap * len(content_items)
-        )
-
-        addition = Image.new('RGB', (addition_width, self.display_height), (0, 0, 0))
+        pieces = []
         x = 0
-        for img in content_items:
+        for item in content_items:
             x += gap                            # separate from whatever precedes
-            addition.paste(img, (x, 0))
-            x += img.width + element_gap
+            pixels = _rgb_pixels(item)
+            pieces.append((x, pixels))
+            x += pixels.shape[1] + element_gap
+        addition_width = x
 
-        # numpy concatenate, and no conversion back: the strip can be tens of
-        # thousands of columns wide and this runs on the render path. The PIL
-        # image is built from the array only if something reads it (see the
-        # cached_image property).
-        self.cached_array = np.concatenate(
-            (self.cached_array, np.array(addition)), axis=1)
+        # Each item is written straight into the spare room after the strip,
+        # when there is some: the strip can be tens of thousands of columns
+        # wide and this runs on the render thread, where copying all of it
+        # (2-3 ms at 512x64 on a Pi 4) -- or even laying the items out in an
+        # image of their own first (another 4-5 ms) -- cost the frame after
+        # every extension. The PIL image is built from the array only if
+        # something reads it (see cached_image).
+        self.cached_array = self._extended_strip(pieces, addition_width)
         self._defer_image()
         self.total_scroll_width = self.cached_array.shape[1]
         self.scroll_complete = False
 
-        self.logger.info(
+        # Debug: this runs on the render thread, and the caller (Vegas) logs
+        # each extension itself.
+        self.logger.debug(
             "Appended %d item(s) (%dpx) to scroll strip: now %dpx, position %.0f",
             len(content_items), addition_width, self.total_scroll_width,
             self.scroll_position
         )
         return True
 
+    #: Room an extended strip's buffer is given, as a multiple of what it
+    #: holds when (re)allocated. Trims free columns at the front and appends
+    #: use them at the back, so with 3x the buffer is reallocated -- the one
+    #: full copy -- about once every two strip-lengths scrolled.
+    STRIP_SPARE_FACTOR = 3.0
+
+    def _extended_strip(self, pieces: List[Tuple[int, np.ndarray]], added: int) -> np.ndarray:
+        """The strip with ``added`` black columns after it, ``pieces`` drawn in.
+
+        Each piece is ``(x, pixels)``, x counted from the old strip's end;
+        written in place when the buffer has the room.
+        """
+        live = self.cached_array
+        if live is None:
+            # append_content builds a first strip itself and never comes here.
+            raise RuntimeError("no strip to extend")
+        width = live.shape[1]
+        buffer = self._strip_buffer
+        if (live is self._strip_view and buffer is not None
+                and self._strip_start + width + added <= buffer.shape[1]):
+            end = self._strip_start + width
+            self.last_copy_bytes = 0
+        else:
+            total = width + added
+            buffer = np.empty((live.shape[0], max(total + 1, int(total * self.STRIP_SPARE_FACTOR)))
+                              + live.shape[2:], dtype=live.dtype)
+            buffer[:, :width] = live
+            self._strip_buffer = buffer
+            self._strip_start = 0
+            end = width
+            self.last_copy_bytes = live.nbytes
+        rows = buffer.shape[0]
+        region = buffer[:, end:end + added]
+        # Black only where no piece lands -- the gaps, and below a short
+        # piece: blanking the whole region first cost as much again as
+        # writing the pieces (1.8 ms at 512x64 on a Pi 4).
+        covered = 0
+        for x, pixels in pieces:
+            pixels = pixels[:rows]
+            cols = pixels.shape[1]
+            if x > covered:
+                region[:, covered:x] = 0
+            region[:pixels.shape[0], x:x + cols] = pixels
+            if pixels.shape[0] < rows:
+                region[pixels.shape[0]:, x:x + cols] = 0
+            covered = max(covered, x + cols)
+        if covered < added:
+            region[:, covered:] = 0
+        self.last_copy_bytes += region.nbytes
+        self._strip_view = buffer[:, self._strip_start:self._strip_start + width + added]
+        return self._strip_view
+
+    def _forget_strip_buffer(self) -> None:
+        """A new strip replaces the extended one: let its buffer go."""
+        self._strip_buffer = None
+        self._strip_view = None
+        self._strip_start = 0
+
     def drop_scrolled_prefix(self, keep_before: int = 0) -> int:
         """
         Discard columns that have already scrolled past, to bound memory.
@@ -766,9 +852,18 @@ class ScrollHelper:
         if cut <= 0:
             return 0
 
-        # .copy() so the original buffer is released rather than kept alive by
-        # a numpy view. The PIL image is deferred, as in append_content.
-        self.cached_array = self.cached_array[:, cut:].copy()
+        if self.cached_array is self._strip_view:
+            # Only the view's start moves; the columns behind it are reused
+            # when the buffer is next reallocated (append_content).
+            self._strip_view = self.cached_array[:, cut:]
+            self._strip_start += cut
+            self.cached_array = self._strip_view
+            self.last_copy_bytes = 0
+        else:
+            # Not a strip this helper extended: .copy() so the original
+            # buffer is released rather than kept alive by a view.
+            self.cached_array = self.cached_array[:, cut:].copy()
+            self.last_copy_bytes = self.cached_array.nbytes
         self._defer_image()
         self.total_scroll_width = self.cached_array.shape[1]
         self.scroll_position -= cut
@@ -881,6 +976,7 @@ class ScrollHelper:
         
         # Convert to numpy array for fast operations (required for get_visible_portion)
         self.cached_array = np.array(image)
+        self._forget_strip_buffer()
         
         # Update scroll width
         self.total_scroll_width = image.width
@@ -1138,6 +1234,7 @@ class ScrollHelper:
         """
         self.cached_image = None
         self.cached_array = None
+        self._forget_strip_buffer()
         self.total_scroll_width = 0
         self.scroll_position = 0.0
         self.total_distance_scrolled = 0.0

src/common/sports_display_rules.py

@@ -0,0 +1,164 @@
+"""Which games a scoreboard shows, for how long, and what its scorebug dates say.
+
+Four ``sports.py`` methods are identical (executable AST, docstrings
+stripped, decorators compared) in every scoreboard that carries them, and
+were copied here from ledmatrix-plugins ``56c4f15`` (origin/main,
+2026-09-30) under their existing names. They split into two mixins because
+their carriers differ, and a plugin should not gain an override it did not
+have:
+
+``SportsCardOptionsMixin`` -- afl, baseball, basketball, football, hockey,
+lacrosse, nrl and soccer (ufc draws no team scorebug):
+
+- ``_card_option`` -- reads one ``scroll_card`` key through
+  ``SportsCoreSharedMixin._card_option``, but never lets the upcoming
+  scorebug lose both its date and its time (the combination a settings-form
+  bug saved for a whole cohort of boards);
+- ``_recent_date_text`` -- the date line of the full-screen recent scorebug.
+
+``SportsGameRulesMixin`` -- all nine:
+
+- ``_filtered_or_all`` (all but football, which has no such method) -- the
+  quality and division filters on a board with no favourites, failing open
+  to every game rather than a blank panel;
+- ``_effective_live_duration`` (all but ufc, which has none) -- how long a
+  live game stays up: ``non_favorite_live_game_duration`` for a
+  non-favourite when favourites are set, else ``game_display_duration``.
+  afl, nrl and soccer carry it on ``SportsCore``, the other five on
+  ``SportsLive``; the bodies are the same.
+
+The plugin missing a method gains one it never calls, which changes nothing:
+nothing in that plugin, nor in core, calls it.
+
+A new module rather than more methods on ``sports_shared``, for the reason
+``sports_helpers`` gives: a missing module fails at load, where the version
+checks see it; a missing method fails mid-update.
+
+WHAT A HOST MUST PROVIDE
+------------------------
+Derived by walking every ``self.<attr>`` the mixins read; the host-contract
+test in ``test/test_sports_display_rules.py`` fails if a read is added
+without being listed here.
+
+``SportsCardOptionsMixin``:
+
+- ``SportsCoreSharedMixin`` (``src.common.sports_shared``) in the MRO
+  **after** this mixin: ``_card_option`` calls that mixin's ``_card_option``
+  and ``_switch_upcoming_center`` by name, and ``_recent_date_text`` its
+  ``_format_game_date``. List this mixin first --
+  ``class SportsCore(SportsCardOptionsMixin, SportsGameRulesMixin,
+  SportsFetchMixin, SportsCoreSharedMixin, SportsHelpersMixin, ABC)`` --
+  or ``SportsCoreSharedMixin._card_option`` wins and the rescue is lost.
+  Through it: ``config`` (the ``scroll_card`` block it reads).
+
+``SportsGameRulesMixin``:
+
+- ``_passes_other_filters(game)`` -- the plugin's own quality/division
+  filter (``_filtered_or_all``).
+- ``_check_ranking_coverage(games)`` -- from ``SportsCoreSharedMixin``.
+- ``favorite_teams``, ``game_display_duration`` and
+  ``_is_favorite_game(game)``; ``non_favorite_live_game_duration`` read with
+  ``getattr`` (``_effective_live_duration``).
+
+Neither mixin has an ``__init__`` or state. A method on the plugin's own
+class still wins over either.
+"""
+
+from typing import Any, Callable, Dict, List, Optional
+
+from src.common.sports_shared import SportsCoreSharedMixin
+
+
+class SportsCardOptionsMixin:
+    """The scorebug's ``scroll_card`` reads. See module docstring."""
+
+    # The host contract, declared for type checking only.
+    _format_game_date: Callable[..., str]
+
+    def _card_option(self, key: str, default: Any = None) -> Any:
+        """Read one scroll_card key, never blanking the upcoming scorebug.
+
+        With the middle set to "date and time" and both of those lines
+        switched off, the full-screen upcoming scorebug is two logos and
+        "Next Game" with nothing to say when the game is. Nobody picks that
+        on purpose -- "vs" and "none" are the settings for a card without the
+        stack -- yet a whole cohort of boards has it: switch_show_date/_time
+        shipped while the core's settings form still drew keys missing from
+        the saved config as unchecked boxes, so the next Save wrote both as
+        false (fixed in LEDMatrix #597). That one combination therefore reads
+        as both on. Hiding either line alone, or both under "vs" or "none",
+        is still honoured.
+        """
+        # The mixin named outright, not super(): tests lift this method onto
+        # stand-in classes that are not SportsCore subclasses.
+        base = SportsCoreSharedMixin._card_option
+        keys = ("switch_show_date", "switch_show_time")
+        value = base(self, key, default)  # type: ignore[arg-type]
+        if (key in keys and not value
+                and not any(base(self, k, True) for k in keys)  # type: ignore[arg-type]
+                and SportsCoreSharedMixin._switch_upcoming_center(self) == "date_time"):  # type: ignore[arg-type]
+            return True
+        return value
+
+    def _recent_date_text(self, game: Optional[Dict]) -> str:
+        """When a finished game was played, for the full-screen scorebug.
+
+        Formatted by switch_date_format, like the upcoming scorebug, so the
+        two dates on this display agree; its "numeric" default returns the
+        extractor's "9/23" unchanged. ``switch_recent_show_date`` (default
+        true) is the off switch.
+        """
+        if not self._card_option("switch_recent_show_date", True):
+            return ""
+        return self._format_game_date(str((game or {}).get("game_date") or ""), game)
+
+
+class SportsGameRulesMixin:
+    """Which games are worth showing, and for how long. See module docstring."""
+
+    # The host contract, declared for type checking only.
+    favorite_teams: List[str]
+    game_display_duration: float
+    _passes_other_filters: Callable[[Dict], bool]
+    _check_ranking_coverage: Callable[[List[Dict]], None]
+    _is_favorite_game: Callable[[Dict], bool]
+
+    def _filtered_or_all(self, games: List[Dict]) -> List[Dict]:
+        """The games worth watching, or all of them if that leaves none.
+
+        With no favourites configured every game selected is a non-favourite
+        game, so the quality and division settings have to apply here too. They
+        governed only the top-up slice, which this branch never uses, so a
+        board with an empty favourites list had both settings silently inert --
+        it could ask for ranked games only and still get the next N kickoffs.
+
+        Fails open as a whole, not just per check. `_passes_other_filters`
+        allows a game whose data could not be resolved, but a filter working
+        exactly as asked can still match nothing on a given day, and here there
+        is no favourite left to carry the mode -- an empty list is a blank
+        panel rather than a short one.
+        """
+        kept = [g for g in games if self._passes_other_filters(g)]
+        self._check_ranking_coverage(games)
+        return kept or games
+
+    def _effective_live_duration(self, game) -> float:
+        """How long the given live game should stay on screen before rotating.
+
+        Non-favorite live games use non_favorite_live_game_duration, but only
+        when it is set (> 0) AND favorite teams are configured. With no favorites
+        (or the knob at 0) every live game uses game_display_duration - identical
+        to the prior single-duration behavior. When show_favorite_teams_only is
+        on, non-favorite games are never shown, so this naturally never fires."""
+        non_fav = getattr(self, "non_favorite_live_game_duration", 0) or 0
+        if (
+            non_fav > 0
+            and self.favorite_teams
+            and game is not None
+            and not self._is_favorite_game(game)
+        ):
+            return non_fav
+        return self.game_display_duration
+
+
+__all__ = ["SportsCardOptionsMixin", "SportsGameRulesMixin"]

src/common/sports_font_path.py

@@ -0,0 +1,41 @@
+"""Where a scoreboard's bundled font file is, whatever the working directory.
+
+Every scoreboard's ``sports.py`` (nine) and ``game_renderer.py`` (eight)
+carries the same module-level ``_resolve_font_path``. It predates
+:func:`src.common.font_layout.resolve_asset_path`, and probes the core for
+it: the path as given when it exists (relative to the cwd), else the core's
+resolver (``FontManager._resolve_asset_path``, which delegates to
+``resolve_asset_path``), else the path joined to the install root, else the
+path unchanged so the caller's ``ImageFont.truetype`` raises and falls back
+as before.
+
+On every core this module ships in, the probe always finds the resolver, and
+the install-root join repeats what the resolver already tried. What is left
+is two steps, and :func:`resolve_font_path` is exactly those: the cwd first,
+then ``resolve_asset_path``. ``test/test_sports_font_path.py`` checks that
+against the plugins' own copies, path for path. It is the same rule as
+``sports_shared._resolve_font_path``, made public so a plugin can import it.
+
+Why not ``resolve_asset_path`` alone: it never consults the cwd, so a
+process started from another checkout would switch to the install root's
+fonts. Keeping the cwd first keeps that behaviour exactly.
+"""
+
+import os
+
+from src.common.font_layout import resolve_asset_path
+
+
+def resolve_font_path(path: str) -> str:
+    """``path`` if it exists, else :func:`resolve_asset_path` of it.
+
+    Absolute paths that exist come back untouched; a relative path is tried
+    against the cwd, then the install root; a path found nowhere comes back
+    unchanged, so the caller still raises and falls back.
+    """
+    if os.path.exists(path):
+        return path
+    return resolve_asset_path(path)
+
+
+__all__ = ["resolve_font_path"]

src/common/sports_live_scroll.py

@@ -0,0 +1,277 @@
+"""Keep a live scoreboard's scrolling strip current without restarting it.
+
+In scroll mode a scoreboard renders its games into one wide image and
+scrolls it past the panel. The strip used to be rebuilt only when a cycle
+completed, so a score changed mid-cycle stayed frozen in the pixels until the
+marquee finished. Eight scoreboards -- afl, baseball, basketball, football,
+hockey, lacrosse, nrl and soccer (ufc has no live strip) -- carry the same
+fix in their ``manager.py``: fingerprint the live games, rebuild when the
+fingerprint changes (rate-limited, and never for the clock alone), and keep
+the marquee's position across the rebuild. Its eight methods and two class
+constants are identical (executable AST, docstrings stripped, decorators
+compared) in all eight and were copied here from ledmatrix-plugins
+``56c4f15`` (origin/main, 2026-09-30) under their existing names:
+
+- ``_live_scroll_managers`` -- the live managers whose games are on the strip;
+- ``_refresh_live_scroll_managers`` -- let them refresh before they are
+  fingerprinted, off the render thread;
+- ``_live_scroll_fields``, ``_fingerprint_games`` and
+  ``_live_scroll_fingerprint`` -- what the strip was drawn from;
+- ``_live_scroll_needs_rebuild`` (with ``LIVE_SCROLL_REBUILD_MIN_SECONDS``
+  and ``LIVE_SCROLL_REBUILD_DUTY_DIVISOR``) and ``_note_live_scroll_built``
+  -- when to rebuild;
+- ``_preserving_scroll_position`` -- a context manager that keeps the marquee
+  where it was across a rebuild.
+
+``LIVE_VOLATILE_FIELDS`` stays in each plugin: afl, nrl and soccer also
+exclude ``period_text``, which embeds the clock in those sports.
+
+A separate module from ``sports_plugin_host`` because ufc has no live strip:
+it inherits that mixin and not this one, so none of this is in its MRO.
+
+WHAT A HOST MUST PROVIDE
+------------------------
+Derived by walking every ``self.<attr>`` / ``cls.<attr>`` the mixin reads;
+the host-contract test in ``test/test_sports_live_scroll.py`` fails if a read
+is added without being listed here.
+
+- ``LIVE_VOLATILE_FIELDS`` -- a class constant: the game-dict keys a rebuild
+  ignores (the clock, and what the display pipeline adds).
+- ``_live_scroll_fingerprints``, ``_live_scroll_rebuilt_at`` and
+  ``_live_scroll_rebuild_cost`` -- empty dicts the host creates in
+  ``__init__``, keyed by scroll key.
+- ``logger``.
+- ``_dispatch_switch_refresh(manager)`` -- from ``SportsPluginHostMixin``.
+- ``_league_registry`` (``{league: {"enabled": bool, "managers": {"live":
+  manager}}}``) or a ``_get_manager(mode_type)`` accessor, both read with
+  ``getattr`` -- ``_live_scroll_managers``. A host with neither gets no
+  managers, which leaves the feature inert rather than wrong.
+- ``_scroll_manager``, read with ``getattr`` --
+  ``_preserving_scroll_position`` asks it for the mode's scroll helper.
+
+Add it as a base of the plugin class beside ``SportsPluginHostMixin``, before
+``BasePlugin``: ``class SoccerScoreboardPlugin(SportsPluginHostMixin,
+SportsLiveScrollMixin, BasePlugin)``. The two define no name in common. No
+``__init__``; a method on the plugin's own class still wins over the mixin's.
+"""
+
+import logging
+import time
+from contextlib import contextmanager
+from typing import Any, Callable, ClassVar, Dict, FrozenSet, Iterator, List
+
+
+class SportsLiveScrollMixin:
+    """Mid-cycle rebuilds of a live scroll strip. See module docstring."""
+
+    # The host contract, declared for type checking only: these create no
+    # attributes, so the host's own values are what the methods read.
+    logger: logging.Logger
+    LIVE_VOLATILE_FIELDS: ClassVar[FrozenSet[str]]
+    _live_scroll_fingerprints: Dict[Any, Any]
+    _live_scroll_rebuilt_at: Dict[Any, float]
+    _live_scroll_rebuild_cost: Dict[Any, float]
+    _dispatch_switch_refresh: Callable[[Any], None]
+
+    #: Floor between mid-cycle strip rebuilds, and the duty-cycle cap that can
+    #: raise it.
+    #:
+    #: A rebuild re-renders every card into one wide image, on the render
+    #: thread, so the marquee is frozen for however long it takes. Measured on a
+    #: Pi 4: 28ms for one game, 139ms for five, 435ms for fifteen. A fixed 5s
+    #: floor is fine for one game and wrong for a full slate -- with fifteen
+    #: live games a pitch lands somewhere every second or so, the fingerprint
+    #: changes continuously, and 435ms every 5s is nearly a tenth of the time
+    #: spent not scrolling.
+    #:
+    #: So the floor also scales with what the last rebuild actually cost: never
+    #: spend more than 1/LIVE_SCROLL_REBUILD_DUTY_DIVISOR of wall time
+    #: rebuilding. Fifteen games self-limits to a rebuild every ~8.7s; one game
+    #: stays on the 5s floor. No per-sport tuning, and it adapts to slate size
+    #: and panel width on its own.
+    LIVE_SCROLL_REBUILD_MIN_SECONDS: ClassVar[float] = 5.0
+    LIVE_SCROLL_REBUILD_DUTY_DIVISOR: ClassVar[float] = 20.0
+
+    def _live_scroll_managers(self, league=None):
+        """The live managers whose games are on the strip.
+
+        Two shapes across the scoreboard lineage: a _league_registry (baseball,
+        basketball, hockey, lacrosse, soccer, football) and a _get_manager
+        accessor on the single-league plugins (afl, nrl). Anything else returns
+        nothing, which leaves this feature inert rather than wrong.
+        """
+        registry = getattr(self, "_league_registry", None)
+        if isinstance(registry, dict) and registry:
+            managers = []
+            for league_id, entry in registry.items():
+                if league is not None and league_id != league:
+                    continue
+                entry = entry or {}
+                if not entry.get("enabled", False):
+                    continue
+                manager = (entry.get("managers") or {}).get("live")
+                if manager is not None:
+                    managers.append(manager)
+            return managers
+        getter = getattr(self, "_get_manager", None)
+        if callable(getter):
+            try:
+                # pylint: disable=not-callable
+                # The lineages that lack _get_manager infer this as None, so a
+                # static checker calls it uncallable. callable() above is the
+                # runtime guard; the branch is simply dead in those plugins.
+                manager = getter("live")
+            except (AttributeError, KeyError, TypeError, ValueError, OSError):
+                return []
+            return [manager] if manager is not None else []
+        return []
+
+    def _refresh_live_scroll_managers(self, league=None) -> None:
+        """Let the live managers refresh before their games are fingerprinted.
+
+        Switch mode stays current because _try_manager_display() calls
+        _ensure_manager_updated() on every pass. Scroll mode had no equivalent:
+        its only refresh sat inside the block gated by the rebuild decision, and
+        that decision is computed from the data the refresh would replace. So
+        once the first strip was built nothing could change it, and the score on
+        the marquee stayed frozen until the process restarted.
+
+        The refresh runs off the render thread -- see _dispatch_switch_refresh().
+        This is called on every scroll frame, and a due manager.update() is a
+        network round trip: run inline, it froze the marquee for the length of
+        the ESPN request. The refreshed games land a few frames later, and the
+        fingerprint check that follows this call picks them up on the next frame
+        after they do. Dispatches for a manager are rate-limited, so the frames
+        where nothing is due cost a dict lookup and a clock read.
+
+        Deliberately NOT gated on mode_type == "live". A recent/upcoming strip
+        never rebuilds from the fingerprint (_live_scroll_needs_rebuild returns
+        early for those), so refreshing here looks like wasted work -- but with
+        live_priority the plugin only switches TO live mode once it knows live
+        games exist, and it learns that from these same managers. Refreshing
+        only while live mode is on screen would rebuild the same circularity one
+        level up, and a game that went live would wait for the background
+        plugin update -- an hour, on a rig that sets update_interval: 3600.
+        """
+        for manager in self._live_scroll_managers(league) or []:
+            try:
+                self._dispatch_switch_refresh(manager)
+            except (AttributeError, KeyError, TypeError, ValueError, OSError,
+                    RuntimeError) as exc:
+                # Narrow on purpose: the update itself runs on another thread,
+                # and _ensure_manager_updated() swallows whatever it raises, so
+                # anything arriving here is a lookup error or a thread that
+                # could not be started, not a fetch failure.
+                self.logger.debug("Live scroll refresh skipped: %s", exc)
+
+    @classmethod
+    def _live_scroll_fields(cls, game) -> tuple:
+        """One game as sorted ``(key, value)`` strings, minus the volatile keys."""
+        try:
+            items = list(game.items())
+        except AttributeError:
+            return (("<not-a-dict>", str(game)),)
+        return tuple(sorted((str(k), str(v)) for k, v in items
+                            if k not in cls.LIVE_VOLATILE_FIELDS))
+
+    @classmethod
+    def _fingerprint_games(cls, games) -> tuple:
+        """Order-independent fingerprint of a list of games."""
+        return tuple(sorted(cls._live_scroll_fields(g) for g in (games or [])))
+
+    def _live_scroll_fingerprint(self, league=None) -> tuple:
+        """Fingerprint of every live game the strip's managers hold now."""
+        games: List[Any] = []
+        for manager in self._live_scroll_managers(league):
+            games.extend(getattr(manager, "live_games", None) or [])
+        return self._fingerprint_games(games)
+
+    def _live_scroll_needs_rebuild(self, scroll_key, mode_type, league=None) -> bool:
+        """True when the live card would draw differently than the strip does.
+
+        _scroll_prepared is cleared only when the cycle *completes*, so a score
+        scored mid-cycle stayed frozen in the rendered strip until the marquee
+        finished -- minutes, for a long game list. Restarting the display forces
+        a rebuild, which is the workaround users find.
+        """
+        if mode_type != "live":
+            return False
+        known = self._live_scroll_fingerprints.get(scroll_key)
+        if known is None:
+            return False                      # nothing built yet; normal path
+        if self._live_scroll_fingerprint(league) == known:
+            return False
+        last = self._live_scroll_rebuilt_at.get(scroll_key, 0.0)
+        cost = self._live_scroll_rebuild_cost.get(scroll_key, 0.0)
+        floor = max(self.LIVE_SCROLL_REBUILD_MIN_SECONDS,
+                    cost * self.LIVE_SCROLL_REBUILD_DUTY_DIVISOR)
+        if time.time() - last < floor:
+            return False                      # deferred, not dropped
+        return True
+
+    def _note_live_scroll_built(self, scroll_key, mode_type, fingerprint=None,
+                                league=None) -> None:
+        """Record what the strip was built from.
+
+        Takes a fingerprint captured from the *managers* immediately before the
+        render, not one computed from the games handed to the renderer. Those
+        two are not comparable: _collect_games_for_scroll() decorates each game
+        with extra keys ("league", "status"), so a fingerprint taken from its
+        output can never equal one taken from the managers -- every check past
+        the rate limiter would rebuild, defeating the clock exclusion entirely.
+        That is not hypothetical; it is what the first version of this did, and
+        an end-to-end simulation caught it rebuilding on a bare clock tick.
+
+        Capturing before the render also closes the race a plain re-read would
+        open: a background update landing mid-render would otherwise be recorded
+        as though the strip already contained it.
+        """
+        if mode_type != "live":
+            return
+        self._live_scroll_fingerprints[scroll_key] = (
+            fingerprint if fingerprint is not None
+            else self._live_scroll_fingerprint(league))
+        self._live_scroll_rebuilt_at[scroll_key] = time.time()
+
+    @contextmanager
+    def _preserving_scroll_position(self, mode_type, active, scroll_key=None) -> Iterator[None]:
+        """Keep the marquee where it is across a mid-cycle rebuild.
+
+        ScrollHelper.set_scrolling_image() resets two counters and both matter:
+        scroll_position (without it the marquee snaps back to the start, which
+        looks worse than the stale score being fixed) and total_distance_scrolled
+        (without it the cycle restarts, so a game that keeps scoring could stop
+        the strip ever completing). Restored clamped to the new strip, since a
+        score gaining a digit changes its card's width by a few pixels.
+
+        A no-op unless `active` -- a first build should start at zero.
+        """
+        helper = None
+        if active and getattr(self, "_scroll_manager", None):
+            try:
+                helper = self._scroll_manager.get_scroll_display(mode_type).scroll_helper  # type: ignore[attr-defined]
+            except Exception:  # pragma: no cover - defensive
+                helper = None
+        position = getattr(helper, "scroll_position", None) if helper else None
+        distance = getattr(helper, "total_distance_scrolled", None) if helper else None
+        started = time.time()
+        try:
+            yield
+        finally:
+            # What this render cost, so the next floor can scale with it. Keyed by
+            # scroll_key, which is what _live_scroll_needs_rebuild() reads --
+            # they are only the same string in some of these plugins, and keying
+            # by mode_type made the duty cap silently inert in the rest.
+            self._live_scroll_rebuild_cost[scroll_key or mode_type] = time.time() - started
+            if helper is not None and position is not None:
+                width = max(getattr(helper, "total_scroll_width", 0) - 1, 0)
+                helper.scroll_position = min(position, width)
+                if distance is not None:
+                    helper.total_distance_scrolled = distance
+                helper.scroll_complete = False
+                self.logger.info(
+                    "[Scroll] Live card changed; rebuilt the %s strip in place "
+                    "at position %d", mode_type, int(helper.scroll_position))
+
+
+__all__ = ["SportsLiveScrollMixin"]

src/common/sports_plugin_host.py

@@ -0,0 +1,270 @@
+"""The scoreboard plugin class's helpers every ``manager.py`` copies.
+
+Each scoreboard's ``manager.py`` holds its ``BasePlugin`` subclass (the
+"host": ``SoccerScoreboardPlugin``, ``UFCScoreboardPlugin``, ...). Ten of its
+methods, and the class constant one of them reads, are identical
+(executable AST, docstrings stripped, decorators compared) in all nine
+scoreboards -- afl, baseball, basketball, football, hockey, lacrosse, nrl,
+soccer and ufc -- and were copied here from ledmatrix-plugins ``56c4f15``
+(origin/main, 2026-09-30) under their existing names:
+
+- ``_dispatch_switch_refresh`` (with ``_SWITCH_REFRESH_MIN_GAP_SECONDS``) --
+  run a manager's refresh on a daemon thread so ``display()`` never blocks
+  on the network;
+- ``get_vegas_priority_weight``, ``_favorite_team_is_live``,
+  ``_favorite_scan_targets``, ``_favorite_scan_games`` and
+  ``_game_involves`` -- how many Vegas slots the plugin asks for, and
+  whether a configured favourite is playing live;
+- ``get_vegas_content_type`` -- ``'multi'``: a scoreboard is a list of games;
+- ``_dynamic_feature_enabled``, ``_get_total_games_for_manager`` and
+  ``_build_manager_key`` -- small dynamic-duration helpers.
+
+This is stage 4 of the consolidation (docs/SPORTS_UNIFICATION.md): the
+families that needed no reconciling. The rest of ``manager.py`` has drifted
+and is reconciled one family per release before it moves.
+
+A new module rather than more methods on an existing mixin, for the reason
+``sports_helpers`` gives: a missing module fails at load, where the version
+checks see it; a missing method fails mid-frame.
+
+WHAT A HOST MUST PROVIDE
+------------------------
+Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
+test in ``test/test_sports_plugin_host.py`` fails if a read is added without
+being listed here.
+
+- ``_ensure_manager_updated(manager)`` -- ``_dispatch_switch_refresh`` runs
+  it on the thread it starts. It must swallow its own errors: nothing joins
+  the thread.
+- ``global_config``, ``has_live_priority()``, ``has_live_content()`` and
+  ``supports_dynamic_duration()`` -- all on ``BasePlugin``; the scoreboards
+  override the last three.
+- ``is_enabled`` -- set by each scoreboard's ``__init__`` (``BasePlugin``
+  calls its flag ``enabled``).
+- ``_switch_refresh_threads`` and ``_switch_refresh_at``, read with
+  ``getattr`` -- ``_dispatch_switch_refresh`` creates both on first use, so
+  a host need not.
+- The live managers it scans for favourites are found through ``vars(self)``
+  (``_favorite_scan_targets``): any attribute, or value of a dict
+  attribute, with ``favorite_teams`` (or ``favorite_fighters``) and
+  ``live_games`` (or ``live_matches``, or an ``active_celebration`` dict
+  holding a ``game``).
+
+Add it as a base of the plugin class, **before** ``BasePlugin``, e.g.
+``class SoccerScoreboardPlugin(SportsPluginHostMixin, BasePlugin)``:
+``get_vegas_priority_weight`` and ``get_vegas_content_type`` override
+``BasePlugin``'s defaults. A method on the plugin's own class still wins over
+the mixin's. The mixin has no ``__init__`` and creates no class attributes
+beyond its one constant.
+"""
+
+import logging
+import threading
+import time
+from typing import Any, Callable, ClassVar, Dict, Iterator, Optional
+
+
+class SportsPluginHostMixin:
+    """The scoreboard plugin class's identical helpers. See module docstring."""
+
+    # The host contract, declared for type checking only: these create no
+    # attributes, so the host's own values are what the methods read.
+    logger: logging.Logger
+    is_enabled: bool
+    global_config: Dict[str, Any]
+    _ensure_manager_updated: Callable[[Any], Any]
+    has_live_priority: Callable[[], bool]
+    has_live_content: Callable[[], bool]
+    supports_dynamic_duration: Callable[[], bool]
+    # Created on first use by _dispatch_switch_refresh, per instance.
+    _switch_refresh_threads: Dict[int, threading.Thread]
+    _switch_refresh_at: Dict[int, float]
+
+    #: Floor between two draw-time refresh dispatches for one manager. The
+    #: manager's own update() still decides whether anything is fetched; this
+    #: only stops display() starting a thread on every frame just to be told
+    #: the interval has not elapsed.
+    _SWITCH_REFRESH_MIN_GAP_SECONDS: ClassVar[float] = 5.0
+
+    def _dispatch_switch_refresh(self, manager) -> None:
+        """Run _ensure_manager_updated(manager) on a daemon thread.
+
+        Called from display(), so it must not block: when an update is due,
+        manager.update() fetches rankings and the schedule over the network,
+        and doing that inline stalled the frame for the length of the round
+        trip. The refreshed games land in the manager a few frames later --
+        still within the manager's own interval, which is the freshness the
+        switch path was missing.
+
+        At most one refresh per manager runs at a time, and dispatches for the
+        same manager are at least _SWITCH_REFRESH_MIN_GAP_SECONDS apart. Only
+        the render thread touches the two bookkeeping dicts, so they need no
+        lock; manager.update() stamps last_update before it fetches, so a
+        concurrent background plugin.update() for the same manager returns
+        early rather than fetching twice.
+        """
+        threads: Optional[Dict[int, threading.Thread]] = getattr(self, "_switch_refresh_threads", None)
+        if threads is None:
+            threads = self._switch_refresh_threads = {}
+        stamps: Optional[Dict[int, float]] = getattr(self, "_switch_refresh_at", None)
+        if stamps is None:
+            stamps = self._switch_refresh_at = {}
+
+        key = id(manager)
+        running = threads.get(key)
+        if running is not None and running.is_alive():
+            return
+        now = time.monotonic()
+        last = stamps.get(key)
+        if last is not None and now - last < self._SWITCH_REFRESH_MIN_GAP_SECONDS:
+            return
+        stamps[key] = now
+        thread = threading.Thread(
+            target=self._ensure_manager_updated,
+            args=(manager,),
+            daemon=True,
+            name="SwitchRefresh-%s" % type(manager).__name__,
+        )
+        threads[key] = thread
+        thread.start()
+
+    # ---- Vegas weighting: is a favourite playing? -----------------------
+    #
+    # With display.vegas_scroll.live_in_ticker set, the marquee keeps running
+    # through a live game and plugins can claim more than one slot per cycle.
+    # The core already gives any plugin with live content `live_weight`; this
+    # exists for the one thing the core cannot work out for itself, which is
+    # *whose* game is live. See PLUGIN_API_REFERENCE, "Vegas scroll hooks",
+    # and ADVANCED_FEATURES, "Live content in the ticker".
+
+    def get_vegas_priority_weight(self):
+        """Slots per Vegas cycle: more when a favorite team is playing.
+
+        Returns None when nothing is live, which leaves the decision to the
+        core rather than asserting a weight of 1 -- the core may have its own
+        reason to boost this plugin later.
+        """
+        try:
+            if not (self.has_live_priority() and self.has_live_content()):
+                return None
+            vegas = (self.global_config or {}).get('display', {}).get(
+                'vegas_scroll', {})
+            if self._favorite_team_is_live():
+                return vegas.get('favorite_live_weight', 5)
+            return vegas.get('live_weight', 3)
+        except Exception:
+            # Never let a weighting question break the rotation; the core
+            # treats an exception as weight 1 anyway, and None says the same
+            # thing more cheaply.
+            return None
+
+    def _favorite_team_is_live(self):
+        """Whether any live game or fight involves a configured favorite.
+
+        The sports plugins do not share one data shape, so this enumerates the
+        real ones rather than assuming. An earlier version looked only for an
+        attribute holding `live_games` alongside `favorite_teams`, which was
+        true of five plugins and quietly false for four others -- they simply
+        never reported a favorite, and no test noticed because the tests used
+        the assumed shape rather than each plugin's own.
+
+        Handled:
+
+        * managers held directly on the plugin *and* inside a dict such as
+          ``self._managers`` (nrl, afl)
+        * ``live_games`` (most) and ``live_matches`` (cricket)
+        * ``favorite_teams`` (most) and ``favorite_fighters`` (ufc)
+        * identifiers ``home_abbr``/``away_abbr``, ``home_id``/``away_id``,
+          ``fighter1_name``/``fighter2_name``, and cricket's nested
+          ``teams: [{name, abbr, short_name}]``
+        * ``active_celebration["game"]``, a snapshot the live manager keeps
+          precisely because the game leaves ``live_games`` while the
+          celebration is still on screen
+        """
+        for holder in self._favorite_scan_targets():
+            favorites = (getattr(holder, 'favorite_teams', None)
+                         or getattr(holder, 'favorite_fighters', None))
+            if not favorites:
+                continue
+            wanted = {str(f).strip().lower() for f in favorites if f}
+            if not wanted:
+                continue
+            for game in self._favorite_scan_games(holder):
+                if self._game_involves(game, wanted):
+                    return True
+        return False
+
+    def _favorite_scan_targets(self) -> Iterator[Any]:
+        """Objects that might carry live content: attributes, and dict values.
+
+        nrl and afl keep their per-league managers in a ``self._managers``
+        dict, so walking attribute values alone finds the dict and stops.
+        """
+        for value in list(vars(self).values()):
+            yield value
+            if isinstance(value, dict):
+                for nested in list(value.values()):
+                    yield nested
+
+    @staticmethod
+    def _favorite_scan_games(holder) -> Iterator[Dict[str, Any]]:
+        """Every game/fight on a holder that a favorite could be playing in."""
+        for attr in ('live_games', 'live_matches'):
+            for game in (getattr(holder, attr, None) or []):
+                if isinstance(game, dict):
+                    yield game
+        celebration = getattr(holder, 'active_celebration', None)
+        if isinstance(celebration, dict) and isinstance(celebration.get('game'), dict):
+            yield celebration['game']
+
+    @staticmethod
+    def _game_involves(game, wanted) -> bool:
+        """Whether a game/fight involves one of the wanted names."""
+        for field in ('home_abbr', 'away_abbr', 'home_id', 'away_id',
+                      'fighter1_name', 'fighter2_name'):
+            value = game.get(field)
+            if value is not None and str(value).strip().lower() in wanted:
+                return True
+        # Cricket nests its sides and matches on any of three names, by
+        # substring -- "india" should match "India Women". Mirrors that
+        # plugin's own _match_has_team rather than inventing a second rule.
+        for team in (game.get('teams') or []):
+            if not isinstance(team, dict):
+                continue
+            hay = " ".join(str(team.get(k) or '') for k in
+                           ('name', 'abbr', 'short_name')).lower()
+            if any(name in hay for name in wanted):
+                return True
+        return False
+
+    def get_vegas_content_type(self) -> str:
+        """Plugin provides multiple scrollable items (games)."""
+        return 'multi'
+
+    # ---- dynamic duration ------------------------------------------------
+
+    def _dynamic_feature_enabled(self) -> bool:
+        """Dynamic duration applies: the plugin is enabled and supports it."""
+        if not self.is_enabled:
+            return False
+        return self.supports_dynamic_duration()
+
+    @staticmethod
+    def _get_total_games_for_manager(manager) -> int:
+        """How many games a manager holds, from the first list it carries."""
+        if manager is None:
+            return 0
+        for attr in ("live_games", "games_list", "recent_games", "upcoming_games"):
+            value = getattr(manager, attr, None)
+            if isinstance(value, list):
+                return len(value)
+        return 0
+
+    @staticmethod
+    def _build_manager_key(mode_name: str, manager) -> str:
+        """``"<mode>:<manager class>"``, the key progress is tracked under."""
+        manager_name = manager.__class__.__name__ if manager else "None"
+        return f"{mode_name}:{manager_name}"
+
+
+__all__ = ["SportsPluginHostMixin"]

src/core_config_keys.py

@@ -29,6 +29,7 @@ CORE_CONFIG_KEYS = frozenset({
     'display',
     'sync',
     'plugin_system',
+    'fetch_service',
     # Older or optional core sections still found in existing config files.
     'logging',
     'network',

src/display_manager.py

@@ -52,7 +52,6 @@ import threading
 import time
 from collections import OrderedDict, deque
 from typing import Dict, Any, List, Optional, Tuple, TYPE_CHECKING
-import math
 import zlib
 import freetype
 
@@ -62,7 +61,6 @@ from src.common.frame_timing import FrameTimingRecorder
 
 if TYPE_CHECKING:
     from src.common.render_gate import RenderGate
-from src.deprecation import deprecated
 from src.logging_config import get_logger
 from src.common.permission_utils import (
     ensure_directory_permissions,
@@ -225,6 +223,11 @@ def _per_thread_canvas_attr(name: str) -> property:
 
 
 
+#: A held frame is only split when its blit takes less than this share of a
+#: refresh: the second blit has to land before the next vsync.
+_SPLIT_BLIT_FRACTION = 0.5
+
+
 class DisplayManager:
     """
     Singleton hardware abstraction layer for the RGB LED matrix.
@@ -973,28 +976,36 @@ class DisplayManager:
                 # mode the logical screen is first tiled across the full chain.
                 blit_started = time.perf_counter()
                 if self._double_sided is not None:
-                    self.offscreen_canvas.SetImage(self._composite_double_sided())
+                    segments = [(self._composite_double_sided(), self._frame_hold)]
                 else:
-                    self.offscreen_canvas.SetImage(self._scan_compensated(self.image))
-                blit_done = time.perf_counter()
-
-                # Swap buffers immediately. framerate_fraction holds the frame
-                # for N refreshes; SwapOnVSync blocks for all of them, which is
-                # what paces the render loop to the chosen frame rate.
+                    segments = self._scan_segments(self.image)
                 gate = self.render_gate
+                blit_time = swap_time = 0.0
+                # Usually one segment: the frame, held for _frame_hold
+                # refreshes. SwapOnVSync blocks for all of them, which is what
+                # paces the render loop to the chosen frame rate. Scan-order
+                # compensation on a held frame splits it, so the lagging rows
+                # change one refresh after the rest.
                 if gate is not None:
                     gate.before_swap(self._frame_hold)
-                self.matrix.SwapOnVSync(self.offscreen_canvas, self._frame_hold)
+                for index, (shown, hold) in enumerate(segments):
+                    if index:
+                        blit_started = time.perf_counter()
+                    self.offscreen_canvas.SetImage(shown)
+                    blit_done = time.perf_counter()
+                    blit_time += blit_done - blit_started
+                    self.matrix.SwapOnVSync(self.offscreen_canvas, hold)
+                    swap_time += time.perf_counter() - blit_done
+                    # Swap our canvas references
+                    self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
                 if gate is not None:
                     gate.after_swap(self._frame_hold)
                 presented_at = time.perf_counter()
+                self._last_blit_seconds = blit_time / len(segments)
                 self.frame_timing.record(
-                    blit_done - blit_started, presented_at - blit_done,
+                    blit_time, swap_time,
                     self._frame_hold, self.is_currently_scrolling(), presented_at)
 
-                # Swap our canvas references
-                self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
-
                 self._last_pushed_digest = digest
 
                 # Write a snapshot for the web preview (throttled)
@@ -1040,23 +1051,35 @@ class DisplayManager:
             ", ".join(f"rows {top}-{bottom - 1} show {lag} refresh(es) behind"
                       for top, bottom, lag in bands))
 
-    def _scan_compensated(self, image: Image.Image) -> Image.Image:
-        """The frame to present, with lagging rows taken from earlier frames.
+    def _scan_segments(self, image: Image.Image) -> List[Tuple[Image.Image, int]]:
+        """What to present for this frame: ``[(image, refreshes), ...]``.
 
-        Only mid-scroll at one frame per refresh: that is when consecutive
-        frames are consecutive refreshes. At a longer hold, or on a static
-        screen, the history is dropped and the frame goes out as it is.
+        Mid-scroll with compensation on, lagging rows are taken from earlier
+        refreshes (see src/scan_order.py). At one refresh per frame that is one
+        image. A frame held longer is split at the refresh where the lagging
+        rows catch up, so those rows step a refresh after the rest. The split
+        needs a second blit inside the refresh that follows the first swap, so
+        it is skipped when a blit is too slow to fit. A static screen goes out
+        as it is, and drops the history.
         """
+        hold = self._frame_hold
         bands = getattr(self, '_scan_lag_bands', None)
-        if not bands:
-            return image
-        if self._frame_hold != 1 or not self.is_currently_scrolling():
-            self._scan_history.clear()
-            return image
-        presented = scan_order.compose(image, self._scan_history, bands)
+        if not bands or not self.is_currently_scrolling():
+            if bands:
+                self._scan_history.clear()
+            return [(image, hold)]
+        if hold > 1:
+            blit = getattr(self, '_last_blit_seconds', 0.0)
+            if blit > _SPLIT_BLIT_FRACTION / max(1.0, self.refresh_hz):
+                self._scan_history.clear()
+                return [(image, hold)]
+        segments = [
+            (scan_order.compose(image, self._scan_history, bands, backs), count)
+            for backs, count in scan_order.refresh_plan(bands, hold)
+        ]
         # A copy: plugins draw into the same image object frame after frame.
         self._scan_history.appendleft(image.copy())
-        return presented
+        return segments
 
     def clear(self):
         """Clear the display completely."""
@@ -1323,203 +1346,6 @@ class DisplayManager:
         except Exception as e:
             logger.error(f"Error drawing text: {e}", exc_info=True)
 
-    @deprecated("3.8.0")
-    def draw_sun(self, x: int, y: int, size: int = 16):
-        """Draw a sun icon using yellow circles and lines."""
-        center = (x + size//2, y + size//2)
-        radius = size//3
-        
-        # Draw the center circle
-        self.draw.ellipse([center[0]-radius, center[1]-radius, 
-                          center[0]+radius, center[1]+radius], 
-                         fill=(255, 255, 0))  # Yellow
-        
-        # Draw the rays
-        ray_length = size//4
-        for angle in range(0, 360, 45):
-            rad = math.radians(angle)
-            start_x = center[0] + (radius * math.cos(rad))
-            start_y = center[1] + (radius * math.sin(rad))
-            end_x = center[0] + ((radius + ray_length) * math.cos(rad))
-            end_y = center[1] + ((radius + ray_length) * math.sin(rad))
-            self.draw.line([start_x, start_y, end_x, end_y], fill=(255, 255, 0), width=2)
-
-    @deprecated("3.8.0")
-    def draw_cloud(self, x: int, y: int, size: int = 16, color=(200, 200, 200)):
-        """Draw a cloud icon."""
-        # Draw multiple circles to form a cloud shape
-        self.draw.ellipse([x+size//4, y+size//3, x+size//4+size//2, y+size//3+size//2], fill=color)
-        self.draw.ellipse([x+size//2, y+size//3, x+size//2+size//2, y+size//3+size//2], fill=color)
-        self.draw.ellipse([x+size//3, y+size//6, x+size//3+size//2, y+size//6+size//2], fill=color)
-
-    @deprecated("3.8.0")
-    def draw_rain(self, x: int, y: int, size: int = 16):
-        """Draw rain icon with cloud and droplets."""
-        # Draw cloud
-        self.draw_cloud(x, y, size)
-        
-        # Draw rain drops
-        drop_color = (0, 0, 255)  # Blue
-        drop_size = size//6
-        for i in range(3):
-            drop_x = x + size//4 + (i * size//3)
-            drop_y = y + size//2
-            self.draw.line([drop_x, drop_y, drop_x, drop_y+drop_size], 
-                          fill=drop_color, width=2)
-
-    @deprecated("3.8.0")
-    def draw_snow(self, x: int, y: int, size: int = 16):
-        """Draw snow icon with cloud and snowflakes."""
-        # Draw cloud
-        self.draw_cloud(x, y, size)
-        
-        # Draw snowflakes
-        snow_color = (200, 200, 255)  # Light blue
-        for i in range(3):
-            center_x = x + size//4 + (i * size//3)
-            center_y = y + size//2 + size//4
-            # Draw a small star shape
-            for angle in range(0, 360, 60):
-                rad = math.radians(angle)
-                end_x = center_x + (size//8 * math.cos(rad))
-                end_y = center_y + (size//8 * math.sin(rad))
-                self.draw.line([center_x, center_y, end_x, end_y], 
-                             fill=snow_color, width=1)
-
-    # Weather icon color constants
-    WEATHER_COLORS = {
-        'sun': (255, 200, 0),    # Bright yellow
-        'cloud': (200, 200, 200), # Light gray
-        'rain': (0, 100, 255),    # Light blue
-        'snow': (220, 220, 255),  # Ice blue
-        'storm': (255, 255, 0)    # Lightning yellow
-    }
-
-    def _draw_sun(self, x: int, y: int, size: int) -> None:
-        """Draw a sun icon with rays."""
-        center_x, center_y = x + size//2, y + size//2
-        radius = size//4
-        ray_length = size//3
-        
-        # Draw the main sun circle
-        self.draw.ellipse([center_x - radius, center_y - radius, 
-                          center_x + radius, center_y + radius], 
-                         fill=self.WEATHER_COLORS['sun'])
-        
-        # Draw sun rays
-        for angle in range(0, 360, 45):
-            rad = math.radians(angle)
-            start_x = center_x + int((radius + 2) * math.cos(rad))
-            start_y = center_y + int((radius + 2) * math.sin(rad))
-            end_x = center_x + int((radius + ray_length) * math.cos(rad))
-            end_y = center_y + int((radius + ray_length) * math.sin(rad))
-            self.draw.line([start_x, start_y, end_x, end_y], 
-                         fill=self.WEATHER_COLORS['sun'], width=2)
-
-    def _draw_cloud(self, x: int, y: int, size: int) -> None:
-        """Draw a cloud using multiple circles."""
-        cloud_color = self.WEATHER_COLORS['cloud']
-        base_y = y + size//2
-        
-        # Draw main cloud body (3 overlapping circles)
-        circle_radius = size//4
-        positions = [
-            (x + size//3, base_y),           # Left circle
-            (x + size//2, base_y - size//6), # Top circle
-            (x + 2*size//3, base_y)          # Right circle
-        ]
-        
-        for cx, cy in positions:
-            self.draw.ellipse([cx - circle_radius, cy - circle_radius,
-                             cx + circle_radius, cy + circle_radius],
-                            fill=cloud_color)
-
-    def _draw_rain(self, x: int, y: int, size: int) -> None:
-        """Draw rain drops falling from a cloud."""
-        self._draw_cloud(x, y, size)
-        rain_color = self.WEATHER_COLORS['rain']
-        
-        # Draw rain drops at an angle
-        drop_size = size//8
-        drops = [
-            (x + size//4, y + 2*size//3),
-            (x + size//2, y + 3*size//4),
-            (x + 3*size//4, y + 2*size//3)
-        ]
-        
-        for dx, dy in drops:
-            # Draw angled rain drops
-            self.draw.line([dx, dy, dx - drop_size//2, dy + drop_size],
-                         fill=rain_color, width=2)
-
-    def _draw_snow(self, x: int, y: int, size: int) -> None:
-        """Draw snowflakes falling from a cloud."""
-        self._draw_cloud(x, y, size)
-        snow_color = self.WEATHER_COLORS['snow']
-        
-        # Draw snowflakes
-        flake_size = size//6
-        flakes = [
-            (x + size//4, y + 2*size//3),
-            (x + size//2, y + 3*size//4),
-            (x + 3*size//4, y + 2*size//3)
-        ]
-        
-        for fx, fy in flakes:
-            # Draw a snowflake (six-pointed star)
-            for angle in range(0, 360, 60):
-                rad = math.radians(angle)
-                end_x = fx + int(flake_size * math.cos(rad))
-                end_y = fy + int(flake_size * math.sin(rad))
-                self.draw.line([fx, fy, end_x, end_y],
-                             fill=snow_color, width=1)
-
-    def _draw_storm(self, x: int, y: int, size: int) -> None:
-        """Draw a storm cloud with lightning bolt."""
-        self._draw_cloud(x, y, size)
-        
-        # Draw lightning bolt
-        bolt_color = self.WEATHER_COLORS['storm']
-        bolt_points = [
-            (x + size//2, y + size//2),          # Top
-            (x + 3*size//5, y + 2*size//3),      # Middle right
-            (x + 2*size//5, y + 2*size//3),      # Middle left
-            (x + size//2, y + 5*size//6)         # Bottom
-        ]
-        self.draw.polygon(bolt_points, fill=bolt_color)
-
-    @deprecated("3.8.0")
-    def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:
-        """Draw a weather icon based on the condition."""
-        if condition.lower() in ['clear', 'sunny']:
-            self._draw_sun(x, y, size)
-        elif condition.lower() in ['clouds', 'cloudy', 'partly cloudy']:
-            self._draw_cloud(x, y, size)
-        elif condition.lower() in ['rain', 'drizzle', 'shower']:
-            self._draw_rain(x, y, size)
-        elif condition.lower() in ['snow', 'sleet', 'hail']:
-            self._draw_snow(x, y, size)
-        elif condition.lower() in ['thunderstorm', 'storm']:
-            self._draw_storm(x, y, size)
-        else:
-            self._draw_sun(x, y, size)
-        # Note: No update_display() here - let the caller handle the update
-
-    @deprecated("3.8.0")
-    def draw_text_with_icons(self, text: str, icons: List[tuple] = None, x: int = None, y: int = None, 
-                            color: tuple = (255, 255, 255)):
-        """Draw text with weather icons at specified positions."""
-        # Draw the text
-        self.draw_text(text, x, y, color)
-        
-        # Draw any icons
-        if icons:
-            for icon_type, icon_x, icon_y in icons:
-                self.draw_weather_icon(icon_type, icon_x, icon_y)
-        
-        # Update the display once after everything is drawn
-        self.update_display()
-
     def cleanup(self):
         """Clean up resources."""
         if hasattr(self, '_snapshot_cond'):
@@ -1831,18 +1657,6 @@ class DisplayManager:
         if removed_count > 0:
             logger.debug(f"Cleaned up {removed_count} expired deferred updates")
 
-    @deprecated("3.8.0")
-    def get_scrolling_stats(self) -> dict:
-        """Get current scrolling statistics for debugging."""
-        return {
-            'is_scrolling': self._scrolling_state['is_scrolling'],
-            'last_activity': self._scrolling_state['last_scroll_activity'],
-            'deferred_count': len(self._scrolling_state['deferred_updates']),
-            'inactivity_threshold': self._scrolling_state['scroll_inactivity_threshold'],
-            'max_deferred_updates': self._scrolling_state['max_deferred_updates'],
-            'deferred_update_ttl': self._scrolling_state['deferred_update_ttl']
-        }
-
     def _viewer_is_fresh(self, now: float) -> bool:
         """True when a browser preview is watching (marker file touched by
         the web SSE broadcaster). The marker is stat'd at most once per

src/font_manager.py

@@ -40,13 +40,7 @@ from pathlib import Path
 from PIL import ImageFont
 from src.common.bdf_font import load_bdf_face, read_bdf_native_size
 from src.common.font_layout import load_truetype, resolve_asset_path
-from src.common.permission_utils import (
-    ensure_directory_permissions,
-    get_assets_dir_mode,
-    get_config_dir_mode,
-)
-from typing import Dict, Tuple, Optional, Union, Any, List
-from src.deprecation import deprecated
+from typing import Dict, Tuple, Optional, Union, Any
 
 logger = logging.getLogger(__name__)
 
@@ -93,7 +87,7 @@ class FontManager:
         self.temp_font_dir = Path(tempfile.gettempdir()) / "ledmatrix_fonts"
         self.temp_font_dir.mkdir(exist_ok=True)
 
-        # Counters behind get_performance_stats().
+        # Font-load counters, kept up by get_font().
         self.performance_stats = {
             "cache_hits": 0,
             "cache_misses": 0,
@@ -109,12 +103,8 @@ class FontManager:
             "tom_thumb": "assets/fonts/tom-thumb.bdf"
         }
         
-        # Size tokens for convenience
-        self.size_tokens = {
-            "xs": 6, "sm": 8, "md": 10, "lg": 12, "xl": 14, "xxl": 16
-        }
-        
-        # Font overrides storage (for manual overrides)
+        # Per-element overrides read from config/font_overrides.json;
+        # resolve_font applies them.
         # Under the install root's config/ (which always exists), not the
         # cwd: the file itself may not exist yet, and resolve_asset_path
         # hands back a missing path unchanged.
@@ -187,26 +177,6 @@ class FontManager:
         if removed:
             self.manager_fonts_version += 1
 
-    @deprecated("3.8.0")
-    def get_manager_fonts(self, manager_id: Optional[str] = None) -> Dict[str, Any]:
-        """
-        Get registered fonts for a specific manager or all managers.
-        
-        Args:
-            manager_id: Optional manager ID, if None returns all
-            
-        Returns:
-            Dictionary of registered fonts
-        """
-        if manager_id:
-            return self.manager_fonts.get(manager_id, {})
-        return self.manager_fonts.copy()
-
-    @deprecated("3.8.0")
-    def get_detected_fonts(self) -> Dict[str, Dict[str, Any]]:
-        """Get all detected font usage across managers."""
-        return self.detected_fonts.copy()
-
     # ==================== Plugin Font Management ====================
 
     def register_plugin_fonts(self, plugin_id: str, font_manifest: Dict[str, Any],
@@ -433,51 +403,6 @@ class FontManager:
         search_dirs = [Path(resolve_asset_path(configured)), Path(resolve_asset_path("plugins"))]
         return resolve_plugin_dir(plugin_id, search_dirs, prefix=True)
 
-    @deprecated("3.8.0")
-    def unregister_plugin_fonts(self, plugin_id: str) -> bool:
-        """Unregister all fonts for a plugin."""
-        try:
-            if plugin_id in self.plugin_fonts:
-                # Remove from plugin catalogs
-                if plugin_id in self.plugin_font_catalogs:
-                    for family in self.plugin_font_catalogs[plugin_id]:
-                        namespaced_family = f"{plugin_id}::{family}"
-                        if namespaced_family in self.font_catalog:
-                            del self.font_catalog[namespaced_family]
-                    
-                    del self.plugin_font_catalogs[plugin_id]
-
-                # Remove plugin manifest
-                del self.plugin_fonts[plugin_id]
-
-                # Clear related cache entries
-                self._clear_plugin_font_cache(plugin_id)
-
-                logger.info(f"Unregistered fonts for plugin {plugin_id}")
-                return True
-
-            return False
-
-        except Exception as e:
-            logger.error(f"Error unregistering plugin fonts: {e}")
-            return False
-
-    def _clear_plugin_font_cache(self, plugin_id: str):
-        """Clear font cache entries for a specific plugin."""
-        keys_to_remove = [key for key in self.font_cache.keys() if key.startswith(f"{plugin_id}::")]
-        for key in keys_to_remove:
-            del self.font_cache[key]
-        if keys_to_remove:
-            # Font objects someone may hold were dropped; see cache_generation.
-            self.cache_generation += 1
-
-    @deprecated("3.8.0")
-    def get_plugin_fonts(self, plugin_id: str) -> List[str]:
-        """Get list of font families registered by a plugin."""
-        if plugin_id in self.plugin_font_catalogs:
-            return list(self.plugin_font_catalogs[plugin_id].keys())
-        return []
-
     # ==================== Font Resolution ====================
 
     def resolve_font(self, element_key: str, family: str, size_px: int, 
@@ -668,42 +593,6 @@ class FontManager:
             logger.error(f"Error getting font height: {e}", exc_info=True)
             return 12  # Default height
 
-    # ==================== Override Management ====================
-
-    @deprecated("3.8.0")
-    def set_override(self, element_key: str, family: str = None, size_px: int = None):
-        """Set font override for a specific element."""
-        if element_key not in self.font_overrides:
-            self.font_overrides[element_key] = {}
-
-        if family is not None:
-            self.font_overrides[element_key]["family"] = family
-        if size_px is not None:
-            self.font_overrides[element_key]["size_px"] = size_px
-
-        # Remove empty overrides
-        if not self.font_overrides[element_key]:
-            del self.font_overrides[element_key]
-        else:
-            self._save_overrides()
-
-        self.clear_cache()
-        logger.info(f"Font override set for {element_key}: {self.font_overrides.get(element_key, {})}")
-
-    @deprecated("3.8.0")
-    def remove_override(self, element_key: str):
-        """Remove font override for a specific element."""
-        if element_key in self.font_overrides:
-            del self.font_overrides[element_key]
-            self._save_overrides()
-            self.clear_cache()
-            logger.info(f"Font override removed for {element_key}")
-
-    @deprecated("3.8.0")
-    def get_overrides(self) -> Dict[str, Dict[str, str]]:
-        """Get current font overrides."""
-        return self.font_overrides.copy()
-
     # ==================== Font Discovery ====================
 
     @staticmethod
@@ -765,17 +654,6 @@ class FontManager:
             logger.warning(f"Could not load font overrides: {e}")
             self.font_overrides = {}
 
-    def _save_overrides(self):
-        """Save current font overrides to file."""
-        try:
-            font_overrides_path = Path(self.font_overrides_file)
-            ensure_directory_permissions(font_overrides_path.parent, get_config_dir_mode())
-            with open(self.font_overrides_file, 'w') as f:
-                json.dump(self.font_overrides, f, indent=2)
-            logger.info(f"Saved {len(self.font_overrides)} font overrides")
-        except Exception as e:
-            logger.error(f"Could not save font overrides: {e}")
-
     # ==================== Utility Methods ====================
 
     def clear_cache(self):
@@ -786,117 +664,3 @@ class FontManager:
         # without the bump they kept serving results for the dropped fonts.
         self.cache_generation += 1
         logger.info("Font cache cleared")
-
-    @deprecated("3.8.0", "read font_catalog")
-    def get_available_fonts(self) -> Dict[str, str]:
-        """Get dictionary of available font families and their paths."""
-        return self.font_catalog.copy()
-
-    @deprecated("3.8.0")
-    def get_size_tokens(self) -> Dict[str, int]:
-        """Get available size tokens."""
-        return self.size_tokens.copy()
-
-    @deprecated("3.8.0")
-    def get_performance_stats(self) -> Dict[str, Any]:
-        """Get performance statistics."""
-        uptime = time.time() - self.performance_stats["start_time"]
-        return {
-            "uptime_seconds": uptime,
-            "cache_hits": self.performance_stats["cache_hits"],
-            "cache_misses": self.performance_stats["cache_misses"],
-            "cache_hit_rate": (
-                self.performance_stats["cache_hits"] / 
-                (self.performance_stats["cache_hits"] + self.performance_stats["cache_misses"])
-                if (self.performance_stats["cache_hits"] + self.performance_stats["cache_misses"]) > 0 else 0
-            ),
-            "total_fonts_cached": len(self.font_cache),
-            "total_metrics_cached": len(self.metrics_cache),
-            "failed_loads": self.performance_stats["failed_loads"],
-            "total_fonts_available": len(self.font_catalog),
-            "plugin_fonts": len(self.plugin_fonts),
-            "manager_fonts": len(self.manager_fonts),
-            "detected_fonts": len(self.detected_fonts)
-        }
-
-    @deprecated("3.8.0", "read font_catalog")
-    def get_font_catalog(self) -> Dict[str, str]:
-        """Get the current font catalog."""
-        return self.font_catalog.copy()
-
-    @deprecated("3.8.0")
-    def add_font(self, font_file_path: str, family_name: str) -> bool:
-        """Add ``font_file_path`` to the catalog as ``family_name``. The file
-        stays where it is; only assets/fonts is created if it is missing."""
-        try:
-            # Validate font file
-            if not os.path.exists(font_file_path):
-                logger.error(f"Font file not found: {font_file_path}")
-                return False
-
-            # Check if family name already exists
-            if family_name in self.font_catalog:
-                logger.warning(f"Font family '{family_name}' already exists")
-                return False
-
-            fonts_dir = Path(resolve_asset_path("assets/fonts"))
-            ensure_directory_permissions(fonts_dir, get_assets_dir_mode())
-
-            # Add to catalog
-            self.font_catalog[family_name] = font_file_path
-            self.clear_cache()
-            logger.info(f"Added font {family_name}: {font_file_path}")
-            return True
-
-        except Exception as e:
-            logger.error(f"Error adding font {family_name}: {e}")
-            return False
-
-    @deprecated("3.8.0")
-    def remove_font(self, family_name: str) -> bool:
-        """Remove a font from the catalog."""
-        try:
-            if family_name not in self.font_catalog:
-                logger.warning(f"Font family '{family_name}' not found")
-                return False
-
-            # Check if font is currently in use
-            in_use = False
-            for override in self.font_overrides.values():
-                if override.get("family") == family_name:
-                    in_use = True
-                    break
-
-            if in_use:
-                logger.error(f"Cannot remove font '{family_name}' - it is currently in use")
-                return False
-
-            del self.font_catalog[family_name]
-            self.clear_cache()
-            logger.info(f"Removed font {family_name}")
-            return True
-
-        except Exception as e:
-            logger.error(f"Error removing font {family_name}: {e}")
-            return False
-
-    @deprecated("3.8.0")
-    def validate_font(self, font_path: str) -> Dict[str, Any]:
-        """Validate a font file."""
-        try:
-            if not os.path.exists(font_path):
-                return {"valid": False, "error": "Font file not found"}
-
-            if font_path.endswith('.bdf'):
-                # Try to load BDF font
-                freetype.Face(font_path)
-                return {"valid": True, "type": "bdf", "family": "unknown"}
-            elif font_path.endswith('.ttf'):
-                # Try to load TTF font
-                load_truetype(font_path, 12)
-                return {"valid": True, "type": "ttf", "family": "unknown"}
-            else:
-                return {"valid": False, "error": "Unsupported font format"}
-
-        except Exception as e:
-            return {"valid": False, "error": str(e)}

src/ipc/__init__.py

@@ -0,0 +1,11 @@
+"""The display process's control socket: web -> display commands with acks.
+
+- :mod:`src.ipc.contract` -- the versioned messages, the framing and where
+  the socket lives. Shared by both sides; standard library only.
+- :mod:`src.ipc.server` -- the display side: a threaded Unix-socket server
+  whose handlers only queue work for the render thread.
+- :mod:`src.ipc.client` -- the web side: one short-timeout request.
+
+See docs/IPC_CONTROL_SOCKET.md for the protocol, the security model and the
+stage plan.
+"""

src/ipc/client.py

@@ -0,0 +1,200 @@
+"""The web side of the control socket: one request, a short timeout, no retries.
+
+Every failure -- no socket (the display is stopped, or predates the socket),
+a refused or timed-out connection, a reply that breaks the contract, or an
+error the display returned -- raises :class:`ControlError` with a short
+``reason``, and the caller falls back to the file mailbox. Nothing here
+blocks for longer than ``timeout`` in total.
+"""
+
+from __future__ import annotations
+
+import socket
+import time
+import uuid
+from typing import Any, Dict, List, Mapping, Optional, Sequence
+
+from src.ipc.contract import (
+    MAX_MESSAGE_BYTES,
+    PROTOCOL_VERSION,
+    SUPPORTED_VERSIONS,
+    Command,
+    FrameReader,
+    ProtocolError,
+    Request,
+    Response,
+    client_socket_paths,
+    decode_message,
+    encode_message,
+    parse_args,
+    socket_supported,
+)
+
+#: Total budget for one request: connect, send and the reply. The display
+#: answers from a thread that does no rendering, normally within a few
+#: milliseconds; this only bounds a wedged one. The web route then falls back
+#: to the mailbox, so a timeout costs this much latency and nothing else.
+DEFAULT_TIMEOUT_SECONDS = 1.0
+
+
+class ControlError(Exception):
+    """The socket could not carry the request. ``reason`` is a short code.
+
+    Transport reasons: ``disabled``, ``unsupported``, ``no_socket``,
+    ``refused``, ``timeout``, ``closed``, ``bad_response``, ``invalid_request``.
+    When the display answered with an error, ``reason`` is that error's
+    :class:`~src.ipc.contract.ErrorCode` (``busy``, ``unknown_command``, ...).
+    """
+
+    def __init__(self, reason: str, message: str = ''):
+        super().__init__(reason, message)
+        self.reason = reason
+        self.message = message
+
+    def __str__(self) -> str:
+        return f'{self.reason}: {self.message}' if self.message else self.reason
+
+
+def request(cmd: str, args: Optional[Mapping[str, Any]] = None, *,
+            request_id: Optional[str] = None,
+            timeout: float = DEFAULT_TIMEOUT_SECONDS,
+            paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    """Send one command and return its ``result``. Raises :class:`ControlError`."""
+    args = dict(args or {})
+    request_id = request_id or str(uuid.uuid4())
+    try:
+        # Refuse locally what the display would refuse: a malformed id
+        # (callers may pass their own) or arguments that break the contract.
+        envelope = Request.from_dict({'v': PROTOCOL_VERSION, 'id': request_id,
+                                      'cmd': cmd, 'args': args})
+        parse_args(cmd, args)
+        payload = encode_message(envelope.to_dict())
+    except ProtocolError as e:
+        raise ControlError('invalid_request', e.message) from None
+
+    if not socket_supported():
+        raise ControlError('unsupported', 'no Unix sockets on this platform')
+    candidates: List[str] = list(paths) if paths is not None else client_socket_paths()
+    if not candidates:
+        raise ControlError('disabled', 'the control socket is turned off')
+
+    deadline = time.monotonic() + timeout
+    sock = _connect(candidates, deadline)
+    try:
+        response = _exchange(sock, payload, deadline)
+    finally:
+        sock.close()
+
+    # A refusal before the request was read (forbidden, too many
+    # connections) carries no id.
+    if response.id != request_id and not (response.id is None and not response.ok):
+        raise ControlError('bad_response', 'the reply is for a different request')
+    if not response.ok:
+        error = response.error
+        raise ControlError(error.code if error else 'bad_response',
+                           error.message if error else '')
+    return dict(response.result or {})
+
+
+def _remaining(deadline: float) -> float:
+    left = deadline - time.monotonic()
+    if left <= 0:
+        raise ControlError('timeout', 'no reply in time')
+    return left
+
+
+def _connect(paths: Sequence[str], deadline: float) -> socket.socket:
+    last = ControlError('no_socket', 'the display is not serving the control socket')
+    for path in paths:
+        sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        try:
+            sock.settimeout(_remaining(deadline))
+            sock.connect(path)
+            return sock
+        except (FileNotFoundError, NotADirectoryError):
+            sock.close()
+            continue
+        except ConnectionRefusedError:
+            sock.close()
+            last = ControlError('refused', f'nothing is listening at {path}')
+        except BlockingIOError:
+            # EAGAIN: the listen backlog is full -- a live but swamped display.
+            sock.close()
+            raise ControlError('busy', 'the display is not accepting connections') from None
+        except PermissionError:
+            sock.close()
+            last = ControlError('refused', f'no permission to connect to {path}')
+        except socket.timeout:
+            sock.close()
+            raise ControlError('timeout', 'connect timed out') from None
+        except ControlError:
+            sock.close()
+            raise
+        except OSError as e:
+            sock.close()
+            last = ControlError('refused', f'{path}: {e}')
+    raise last
+
+
+def _exchange(sock: socket.socket, payload: bytes, deadline: float) -> Response:
+    try:
+        sock.settimeout(_remaining(deadline))
+        sock.sendall(payload)
+        reader = FrameReader(MAX_MESSAGE_BYTES)
+        while True:
+            sock.settimeout(_remaining(deadline))
+            data = sock.recv(4096)
+            if not data:
+                raise ControlError('closed', 'the display closed the connection')
+            lines = reader.feed(data)
+            if lines:
+                return Response.from_dict(decode_message(lines[0]))
+    except socket.timeout:
+        raise ControlError('timeout', 'no reply in time') from None
+    except ProtocolError as e:
+        raise ControlError('bad_response', e.message) from None
+    except ControlError:
+        raise
+    except OSError as e:
+        raise ControlError('closed', str(e)) from None
+
+
+# -- commands ---------------------------------------------------------------------------
+
+def on_demand_start(request_id: str, plugin_id: Optional[str], mode: Optional[str],
+                    duration: Any = None, pinned: bool = False, *,
+                    timeout: float = DEFAULT_TIMEOUT_SECONDS,
+                    paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    """Ask the display to show a plugin now. Returns the ack; raises :class:`ControlError`.
+
+    ``request_id`` doubles as the on-demand request id, so a request that a
+    timed-out caller then also writes to the mailbox is processed only once.
+    """
+    args = {'plugin_id': plugin_id, 'mode': mode, 'duration': duration, 'pinned': pinned}
+    return request(Command.ON_DEMAND_START, args, request_id=request_id,
+                   timeout=timeout, paths=paths)
+
+
+def on_demand_stop(request_id: str, *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
+                   paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    """Ask the display to end on-demand. Returns the ack; raises :class:`ControlError`."""
+    return request(Command.ON_DEMAND_STOP, {}, request_id=request_id,
+                   timeout=timeout, paths=paths)
+
+
+def on_demand_status(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
+                     paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    """The display's live on-demand state. Raises :class:`ControlError`."""
+    return request(Command.ON_DEMAND_STATUS, {}, timeout=timeout, paths=paths)
+
+
+def ping(*, timeout: float = DEFAULT_TIMEOUT_SECONDS,
+         paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    return request(Command.PING, {}, timeout=timeout, paths=paths)
+
+
+def hello(client: str = 'web', *, timeout: float = DEFAULT_TIMEOUT_SECONDS,
+          paths: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+    """Version negotiation: the result's ``version`` is the one both sides speak."""
+    return request(Command.HELLO, {'versions': list(SUPPORTED_VERSIONS), 'client': client},
+                   timeout=timeout, paths=paths)

src/ipc/contract.py

@@ -0,0 +1,527 @@
+"""The control socket's contract: versioned messages, framing and location.
+
+Both processes import this module -- the display serves the socket
+(:mod:`src.ipc.server`) and the web interface calls it
+(:mod:`src.ipc.client`) -- so it is the one definition of what goes over the
+wire. Standard library only, and no import of the rest of ``src``.
+
+Wire format (protocol version 1)
+--------------------------------
+One JSON object per line (newline-delimited JSON), UTF-8, at most
+:data:`MAX_MESSAGE_BYTES` per line including the newline. Messages are
+encoded with ``ensure_ascii``, so a newline never appears inside one.
+
+Request::
+
+    {"v": 1, "id": "<1-128 chars>", "cmd": "on_demand.start", "args": {...}}
+
+Response, always carrying the request's ``id`` (``null`` when the request
+could not be parsed far enough to have one)::
+
+    {"v": 1, "id": "...", "ok": true,  "result": {...}}
+    {"v": 1, "id": "...", "ok": false, "error": {"code": "...", "message": "..."}}
+
+A connection may carry several requests; each gets exactly one response, in
+order. Commands that change what the panel shows are *acknowledged*, not
+completed: ``{"accepted": true, "request_id": ...}`` means the render thread
+has the command queued and will apply it at its next on-demand check. Its
+outcome is published the way it always was (``display_on_demand_state``,
+later the state stream).
+
+See docs/IPC_CONTROL_SOCKET.md for the full description.
+"""
+
+from __future__ import annotations
+
+import json
+import math
+import os
+import tempfile
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Mapping, Optional, Tuple, TypedDict, TypeGuard, Union
+
+
+# -- versions and limits -------------------------------------------------------
+
+#: The protocol version this code speaks by default.
+PROTOCOL_VERSION = 1
+
+#: Every version this code can speak; ``hello`` picks the highest common one.
+SUPPORTED_VERSIONS: Tuple[int, ...] = (1,)
+
+#: The largest message either side sends or accepts, newline included. A
+#: stage-1 message is well under 1 KiB; this only bounds a broken or hostile
+#: peer, so a reader never buffers more than this per connection.
+MAX_MESSAGE_BYTES = 64 * 1024
+
+#: Longest request id. Ids are also the on-demand ``request_id``, which the
+#: display logs and stores, so they are kept short.
+MAX_ID_LENGTH = 128
+
+#: Longest plugin id or mode name an on-demand command may carry.
+MAX_NAME_LENGTH = 128
+
+
+# -- where the socket lives ------------------------------------------------------
+
+#: ``RuntimeDirectory=ledmatrix`` in ledmatrix.service creates this (tmpfs,
+#: root-owned, 0755); a display under an older unit creates it itself, as it
+#: does for the heartbeat (src/display_watchdog.py).
+DEFAULT_SOCKET_DIR = '/run/ledmatrix'
+SOCKET_NAME = 'control.sock'
+DEFAULT_SOCKET_PATH = DEFAULT_SOCKET_DIR + '/' + SOCKET_NAME
+
+#: Overrides the socket path for both processes (a dev checkout, a second
+#: instance, tests). One of :data:`DISABLED_VALUES` turns the socket off: the
+#: display does not serve it and the web interface goes straight to the
+#: file mailbox.
+SOCKET_PATH_ENV = 'LEDMATRIX_CONTROL_SOCKET'
+DISABLED_VALUES = frozenset({'off', '0', 'false', 'no', 'none', 'disabled'})
+
+
+def socket_supported() -> bool:
+    """Whether this platform has Unix sockets at all (Windows Python does not)."""
+    import socket
+    return os.name == 'posix' and hasattr(socket, 'AF_UNIX')
+
+
+def socket_disabled(environ: Optional[Mapping[str, str]] = None) -> bool:
+    """True when :data:`SOCKET_PATH_ENV` switches the socket off."""
+    env = os.environ if environ is None else environ
+    value = (env.get(SOCKET_PATH_ENV) or '').strip()
+    return value.lower() in DISABLED_VALUES
+
+
+def configured_socket_path(environ: Optional[Mapping[str, str]] = None) -> Optional[str]:
+    """The path :data:`SOCKET_PATH_ENV` names, or None when it is unset or 'off'."""
+    env = os.environ if environ is None else environ
+    value = (env.get(SOCKET_PATH_ENV) or '').strip()
+    if not value or value.lower() in DISABLED_VALUES:
+        return None
+    return value
+
+
+def dev_socket_path(uid: Optional[int] = None) -> str:
+    """Where a display that cannot use /run/ledmatrix serves the socket.
+
+    A per-user directory under the temp dir, so a dev checkout run as an
+    ordinary user (``python3 run.py -e``) and its web interface, run by the
+    same user, find each other with no configuration.
+    """
+    if uid is None:
+        getuid = getattr(os, 'getuid', None)
+        uid = getuid() if getuid is not None else 0
+    return os.path.join(tempfile.gettempdir(), f'ledmatrix-{uid}', SOCKET_NAME)
+
+
+def client_socket_paths(environ: Optional[Mapping[str, str]] = None) -> List[str]:
+    """The paths a client tries, in order; empty when the socket is off."""
+    if socket_disabled(environ):
+        return []
+    configured = configured_socket_path(environ)
+    if configured:
+        return [configured]
+    return [DEFAULT_SOCKET_PATH, dev_socket_path()]
+
+
+# -- commands and error codes ----------------------------------------------------
+
+class Command:
+    """Command names. Dotted names group a feature's commands."""
+    HELLO = 'hello'
+    PING = 'ping'
+    ON_DEMAND_START = 'on_demand.start'
+    ON_DEMAND_STOP = 'on_demand.stop'
+    ON_DEMAND_STATUS = 'on_demand.status'
+
+
+#: Every command version 1 defines, in the order ``hello`` reports them.
+COMMANDS: Tuple[str, ...] = (
+    Command.HELLO,
+    Command.PING,
+    Command.ON_DEMAND_START,
+    Command.ON_DEMAND_STOP,
+    Command.ON_DEMAND_STATUS,
+)
+
+#: Commands that are queued for the render thread and answered with an ack.
+QUEUED_COMMANDS = frozenset({Command.ON_DEMAND_START, Command.ON_DEMAND_STOP})
+
+
+class ErrorCode:
+    """``error.code`` values. Clients branch on these, never on the message."""
+    BAD_JSON = 'bad_json'                        # a line that is not a JSON object
+    BAD_REQUEST = 'bad_request'                  # the envelope is malformed
+    MESSAGE_TOO_LARGE = 'message_too_large'      # over MAX_MESSAGE_BYTES
+    UNSUPPORTED_VERSION = 'unsupported_version'  # no version in common
+    UNKNOWN_COMMAND = 'unknown_command'
+    INVALID_ARGS = 'invalid_args'
+    BUSY = 'busy'                                # queue full / too many clients
+    FORBIDDEN = 'forbidden'                      # peer credentials refused
+    INTERNAL = 'internal'                        # a bug on the display side
+
+
+class ProtocolError(Exception):
+    """A message that breaks the contract. ``code`` is an :class:`ErrorCode`."""
+
+    def __init__(self, code: str, message: str, request_id: Optional[str] = None):
+        super().__init__(code, message, request_id)
+        self.code = code
+        self.message = message
+        self.request_id = request_id
+
+    def __str__(self) -> str:
+        return f'{self.code}: {self.message}'
+
+
+# -- the envelope ------------------------------------------------------------------
+
+def _is_int(value: Any) -> TypeGuard[int]:
+    return isinstance(value, int) and not isinstance(value, bool)
+
+
+def _valid_id(value: Any) -> bool:
+    return (isinstance(value, str) and 0 < len(value) <= MAX_ID_LENGTH
+            and value.isprintable())
+
+
+@dataclass(frozen=True)
+class Request:
+    """``{v, id, cmd, args}``."""
+    id: str
+    cmd: str
+    args: Dict[str, Any] = field(default_factory=dict)
+    v: int = PROTOCOL_VERSION
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {'v': self.v, 'id': self.id, 'cmd': self.cmd, 'args': dict(self.args)}
+
+    @classmethod
+    def from_dict(cls, obj: Any) -> 'Request':
+        """Validate an envelope. Raises :class:`ProtocolError`.
+
+        The version is checked by the server, not here, so that ``hello``
+        can negotiate across versions.
+        """
+        if not isinstance(obj, dict):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'a request must be a JSON object')
+        raw_id = obj.get('id')
+        request_id = raw_id if _valid_id(raw_id) else None
+        if request_id is None:
+            raise ProtocolError(ErrorCode.BAD_REQUEST,
+                                f'id must be a printable string of 1-{MAX_ID_LENGTH} characters')
+        version = obj.get('v')
+        if not _is_int(version):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'v must be an integer', request_id)
+        cmd = obj.get('cmd')
+        if not isinstance(cmd, str) or not cmd:
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'cmd must be a non-empty string', request_id)
+        args = obj.get('args', {})
+        if args is None:
+            args = {}
+        if not isinstance(args, dict):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'args must be a JSON object', request_id)
+        return cls(id=request_id, cmd=cmd, args=args, v=version)
+
+
+@dataclass(frozen=True)
+class ErrorInfo:
+    code: str
+    message: str
+
+    def to_dict(self) -> Dict[str, str]:
+        return {'code': self.code, 'message': self.message}
+
+
+@dataclass(frozen=True)
+class Response:
+    """``{v, id, ok, result}`` or ``{v, id, ok: false, error: {code, message}}``."""
+    id: Optional[str]
+    ok: bool
+    result: Optional[Dict[str, Any]] = None
+    error: Optional[ErrorInfo] = None
+    v: int = PROTOCOL_VERSION
+
+    @classmethod
+    def success(cls, request_id: Optional[str], result: Mapping[str, Any],
+                v: int = PROTOCOL_VERSION) -> 'Response':
+        return cls(id=request_id, ok=True, result=dict(result), v=v)
+
+    @classmethod
+    def failure(cls, request_id: Optional[str], code: str, message: str,
+                v: int = PROTOCOL_VERSION) -> 'Response':
+        return cls(id=request_id, ok=False, error=ErrorInfo(code, message), v=v)
+
+    def to_dict(self) -> Dict[str, Any]:
+        out: Dict[str, Any] = {'v': self.v, 'id': self.id, 'ok': self.ok}
+        if self.ok:
+            out['result'] = dict(self.result or {})
+        else:
+            error = self.error or ErrorInfo(ErrorCode.INTERNAL, 'unknown error')
+            out['error'] = error.to_dict()
+        return out
+
+    @classmethod
+    def from_dict(cls, obj: Any) -> 'Response':
+        """Validate a response. Raises :class:`ProtocolError` (BAD_REQUEST)."""
+        if not isinstance(obj, dict):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'a response must be a JSON object')
+        version = obj.get('v')
+        if not _is_int(version):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'v must be an integer')
+        raw_id = obj.get('id')
+        if raw_id is not None and not isinstance(raw_id, str):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'id must be a string or null')
+        ok = obj.get('ok')
+        if not isinstance(ok, bool):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'ok must be a boolean')
+        if ok:
+            result = obj.get('result', {})
+            if not isinstance(result, dict):
+                raise ProtocolError(ErrorCode.BAD_REQUEST, 'result must be a JSON object')
+            return cls(id=raw_id, ok=True, result=result, v=version)
+        error = obj.get('error')
+        if (not isinstance(error, dict) or not isinstance(error.get('code'), str)
+                or not isinstance(error.get('message', ''), str)):
+            raise ProtocolError(ErrorCode.BAD_REQUEST, 'error must be {code, message}')
+        return cls(id=raw_id, ok=False,
+                   error=ErrorInfo(error['code'], error.get('message', '')), v=version)
+
+
+# -- command arguments -------------------------------------------------------------
+
+def _optional_name(args: Mapping[str, Any], key: str) -> Optional[str]:
+    value = args.get(key)
+    if value is None or value == '':
+        return None
+    if not isinstance(value, str) or len(value) > MAX_NAME_LENGTH or not value.isprintable():
+        raise ProtocolError(ErrorCode.INVALID_ARGS,
+                            f'{key} must be a printable string of at most '
+                            f'{MAX_NAME_LENGTH} characters')
+    return value
+
+
+def _optional_duration(value: Any) -> Optional[float]:
+    """Seconds, or None for "until stopped". 0 means the same as None.
+
+    Numbers and numeric strings are accepted, the same as the REST route and
+    the file mailbox take them; anything else is refused rather than guessed.
+    """
+    if value is None or value == '':
+        return None
+    if isinstance(value, bool):
+        raise ProtocolError(ErrorCode.INVALID_ARGS, 'duration must be a number of seconds')
+    try:
+        seconds = float(value)
+    except (TypeError, ValueError):
+        raise ProtocolError(ErrorCode.INVALID_ARGS,
+                            'duration must be a number of seconds') from None
+    if not math.isfinite(seconds) or seconds < 0:
+        raise ProtocolError(ErrorCode.INVALID_ARGS,
+                            'duration must be a finite, non-negative number of seconds')
+    return seconds or None
+
+
+@dataclass(frozen=True)
+class HelloArgs:
+    """``hello``: the versions the client speaks, and a name for the logs."""
+    versions: Tuple[int, ...] = (PROTOCOL_VERSION,)
+    client: str = ''
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {'versions': list(self.versions), 'client': self.client}
+
+    @classmethod
+    def from_dict(cls, args: Mapping[str, Any]) -> 'HelloArgs':
+        versions = args.get('versions', [PROTOCOL_VERSION])
+        if (not isinstance(versions, list) or not versions or len(versions) > 32
+                or not all(_is_int(v) for v in versions)):
+            raise ProtocolError(ErrorCode.INVALID_ARGS, 'versions must be a list of integers')
+        client = args.get('client', '')
+        if not isinstance(client, str) or len(client) > MAX_NAME_LENGTH:
+            raise ProtocolError(ErrorCode.INVALID_ARGS, 'client must be a short string')
+        return cls(versions=tuple(versions), client=client)
+
+
+@dataclass(frozen=True)
+class OnDemandStartArgs:
+    """``on_demand.start``: show a plugin (or one of its modes) now.
+
+    The same fields the file mailbox carries. At least one of ``plugin_id``
+    and ``mode`` is required; the display resolves the other.
+    """
+    plugin_id: Optional[str] = None
+    mode: Optional[str] = None
+    duration: Optional[float] = None
+    pinned: bool = False
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {'plugin_id': self.plugin_id, 'mode': self.mode,
+                'duration': self.duration, 'pinned': self.pinned}
+
+    @classmethod
+    def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStartArgs':
+        plugin_id = _optional_name(args, 'plugin_id')
+        mode = _optional_name(args, 'mode')
+        if plugin_id is None and mode is None:
+            raise ProtocolError(ErrorCode.INVALID_ARGS, 'plugin_id or mode is required')
+        pinned = args.get('pinned', False)
+        if pinned is None:
+            pinned = False
+        if not isinstance(pinned, bool):
+            raise ProtocolError(ErrorCode.INVALID_ARGS, 'pinned must be a boolean')
+        return cls(plugin_id=plugin_id, mode=mode,
+                   duration=_optional_duration(args.get('duration')), pinned=pinned)
+
+
+@dataclass(frozen=True)
+class OnDemandStopArgs:
+    """``on_demand.stop``: end the on-demand session and resume rotation."""
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {}
+
+    @classmethod
+    def from_dict(cls, args: Mapping[str, Any]) -> 'OnDemandStopArgs':
+        return cls()
+
+
+@dataclass(frozen=True)
+class NoArgs:
+    """``ping`` and ``on_demand.status`` take no arguments (extra ones are ignored)."""
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {}
+
+    @classmethod
+    def from_dict(cls, args: Mapping[str, Any]) -> 'NoArgs':
+        return cls()
+
+
+CommandArgs = Union[HelloArgs, OnDemandStartArgs, OnDemandStopArgs, NoArgs]
+
+_ARG_TYPES: Dict[str, Any] = {
+    Command.HELLO: HelloArgs,
+    Command.PING: NoArgs,
+    Command.ON_DEMAND_START: OnDemandStartArgs,
+    Command.ON_DEMAND_STOP: OnDemandStopArgs,
+    Command.ON_DEMAND_STATUS: NoArgs,
+}
+
+
+def parse_args(cmd: str, args: Mapping[str, Any]) -> CommandArgs:
+    """Typed arguments for ``cmd``. Raises :class:`ProtocolError`."""
+    arg_type = _ARG_TYPES.get(cmd)
+    if arg_type is None:
+        raise ProtocolError(ErrorCode.UNKNOWN_COMMAND, f'unknown command: {cmd[:64]}')
+    parsed: CommandArgs = arg_type.from_dict(args)
+    return parsed
+
+
+def on_demand_request(request_id: str, args: Union[OnDemandStartArgs, OnDemandStopArgs],
+                      timestamp: float) -> Dict[str, Any]:
+    """The file-mailbox payload for a queued on-demand command.
+
+    The display hands socket commands to the same code that handles the
+    mailbox (``DisplayController._handle_on_demand_request``), so a command
+    behaves identically whichever way it arrived, and a request that came
+    both ways (a client that timed out and fell back) is processed once: the
+    request id is the same.
+    """
+    if isinstance(args, OnDemandStartArgs):
+        return {'request_id': request_id, 'action': 'start', 'plugin_id': args.plugin_id,
+                'mode': args.mode, 'duration': args.duration, 'pinned': args.pinned,
+                'timestamp': timestamp, 'source': 'socket'}
+    return {'request_id': request_id, 'action': 'stop', 'timestamp': timestamp,
+            'source': 'socket'}
+
+
+# -- results -----------------------------------------------------------------------
+
+class HelloResult(TypedDict):
+    version: int
+    versions: List[int]
+    commands: List[str]
+    max_message_bytes: int
+    server: str
+
+
+class PingResult(TypedDict):
+    pong: bool
+
+
+class AckResult(TypedDict):
+    """The answer to a queued command: the render thread will apply it."""
+    accepted: bool
+    request_id: str
+    queued: int
+
+
+def negotiate_version(client_versions: Tuple[int, ...]) -> Optional[int]:
+    """The highest version both sides speak, or None."""
+    common = set(client_versions) & set(SUPPORTED_VERSIONS)
+    return max(common) if common else None
+
+
+# -- framing -----------------------------------------------------------------------
+
+def encode_message(obj: Mapping[str, Any]) -> bytes:
+    """One newline-terminated JSON line. Raises :class:`ProtocolError` when too big."""
+    try:
+        text = json.dumps(obj, separators=(',', ':'), ensure_ascii=True, allow_nan=False)
+    except (TypeError, ValueError) as e:
+        raise ProtocolError(ErrorCode.BAD_REQUEST, f'message is not JSON-serialisable: {e}') from None
+    data = text.encode('ascii') + b'\n'
+    if len(data) > MAX_MESSAGE_BYTES:
+        raise ProtocolError(ErrorCode.MESSAGE_TOO_LARGE,
+                            f'message is {len(data)} bytes; the limit is {MAX_MESSAGE_BYTES}')
+    return data
+
+
+def decode_message(line: bytes) -> Dict[str, Any]:
+    """Parse one line (newline optional). Raises :class:`ProtocolError` (BAD_JSON)."""
+    try:
+        obj = json.loads(line.decode('utf-8'))
+    except ValueError:  # UnicodeDecodeError and JSONDecodeError are both ValueErrors
+        raise ProtocolError(ErrorCode.BAD_JSON, 'not valid UTF-8 JSON') from None
+    if not isinstance(obj, dict):
+        raise ProtocolError(ErrorCode.BAD_JSON, 'a message must be a JSON object')
+    return obj
+
+
+class FrameReader:
+    """Splits a byte stream into lines, never holding more than one message.
+
+    ``feed()`` returns the complete lines (without their newlines) the new
+    bytes finished, and raises :class:`ProtocolError` (MESSAGE_TOO_LARGE) as
+    soon as a line is longer than the limit, newline or not, so a peer that
+    never sends one cannot make the reader buffer without bound.
+    """
+
+    def __init__(self, max_bytes: int = MAX_MESSAGE_BYTES):
+        self._max = max_bytes
+        self._buffer = bytearray()
+
+    @property
+    def pending(self) -> int:
+        """Bytes of an unfinished message held."""
+        return len(self._buffer)
+
+    def feed(self, data: bytes) -> List[bytes]:
+        self._buffer.extend(data)
+        lines: List[bytes] = []
+        while True:
+            newline = self._buffer.find(b'\n')
+            if newline < 0:
+                break
+            if newline + 1 > self._max:
+                raise ProtocolError(ErrorCode.MESSAGE_TOO_LARGE,
+                                    f'message exceeds {self._max} bytes')
+            line = bytes(self._buffer[:newline])
+            del self._buffer[:newline + 1]
+            if line.strip():
+                lines.append(line)
+        if len(self._buffer) >= self._max:
+            raise ProtocolError(ErrorCode.MESSAGE_TOO_LARGE,
+                                f'message exceeds {self._max} bytes')
+        return lines

src/ipc/server.py

@@ -0,0 +1,643 @@
+"""The display side of the control socket.
+
+A small threaded server on a Unix stream socket (``/run/ledmatrix/control.sock``
+by default; see :mod:`src.ipc.contract` for the protocol). It never touches
+rendering: a command that changes the panel is validated, put on a bounded
+queue and acknowledged, and the render thread drains that queue at the point
+where it reads the file mailbox (``DisplayController._poll_on_demand_requests``),
+handing each command to the same code. Queries (``on_demand.status``) are
+answered from a snapshot callable the display provides.
+
+Robustness rules, because this runs inside the display process:
+
+* every connection has its own daemon thread, at most :data:`MAX_CLIENTS` at
+  once; one more is told ``busy`` and closed;
+* every read and write has a timeout, a message must arrive whole within
+  :data:`MESSAGE_TIMEOUT_SECONDS`, and an idle connection is closed after
+  :data:`IDLE_TIMEOUT_SECONDS` -- a slow or stuck client costs one thread for
+  a few seconds, never the render loop;
+* a line that is not JSON is answered with ``bad_json`` and the connection
+  carries on; a line over the size limit closes the connection; a client
+  that disconnects mid-message is simply dropped;
+* no exception from a handler leaves the connection thread.
+
+Who may connect (see docs/IPC_CONTROL_SOCKET.md, "Security model"): the
+socket file is ``0660`` and group-owned by the group the display and the web
+interface share -- the cache directory's group, the same rule DiskCache uses
+for the files it shares -- so the kernel refuses everyone else at connect().
+Where the kernel reports the peer's credentials (``SO_PEERCRED``, Linux) the
+server checks them again: root, its own user, or a member of that group.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import queue
+import socket
+import stat
+import struct
+import threading
+import time
+from dataclasses import dataclass
+from typing import Any, Callable, Dict, FrozenSet, List, Mapping, Optional, Union
+
+from src.ipc.contract import (
+    COMMANDS,
+    DEFAULT_SOCKET_DIR,
+    DEFAULT_SOCKET_PATH,
+    MAX_MESSAGE_BYTES,
+    PROTOCOL_VERSION,
+    QUEUED_COMMANDS,
+    SUPPORTED_VERSIONS,
+    AckResult,
+    Command,
+    ErrorCode,
+    FrameReader,
+    HelloArgs,
+    HelloResult,
+    OnDemandStartArgs,
+    OnDemandStopArgs,
+    ProtocolError,
+    Request,
+    Response,
+    configured_socket_path,
+    decode_message,
+    dev_socket_path,
+    encode_message,
+    negotiate_version,
+    on_demand_request,
+    parse_args,
+    socket_disabled,
+    socket_supported,
+)
+
+logger = logging.getLogger(__name__)
+
+#: Concurrent connections served. The web interface opens one per request
+#: and closes it; this only bounds a misbehaving client.
+MAX_CLIENTS = 8
+
+#: Commands waiting for the render thread. It drains them at least every
+#: 0.25 s, so a full queue means the render thread is stuck, and the client
+#: is told ``busy`` (and falls back to the mailbox) instead of piling up work.
+QUEUE_SIZE = 16
+
+#: Timeout for one recv()/send() on a connection.
+IO_TIMEOUT_SECONDS = 2.0
+
+#: A message must arrive whole within this long of its first byte.
+MESSAGE_TIMEOUT_SECONDS = 5.0
+
+#: A connection with no message in progress is closed after this long.
+IDLE_TIMEOUT_SECONDS = 10.0
+
+#: How often the accept loop wakes to notice close().
+_ACCEPT_POLL_SECONDS = 0.5
+
+_LISTEN_BACKLOG = 64
+
+
+# -- queued work ---------------------------------------------------------------------
+
+@dataclass(frozen=True)
+class QueuedCommand:
+    """A command waiting for the render thread."""
+    request_id: str
+    cmd: str
+    args: Union[OnDemandStartArgs, OnDemandStopArgs]
+    received_at: float          # time.time() when it was accepted
+    peer_uid: Optional[int] = None
+
+    def as_on_demand_request(self) -> Dict[str, Any]:
+        """The mailbox-shaped payload the display's on-demand handler takes."""
+        return on_demand_request(self.request_id, self.args, self.received_at)
+
+
+# -- peer credentials ------------------------------------------------------------------
+
+@dataclass(frozen=True)
+class PeerCredentials:
+    pid: int
+    uid: int
+    gid: int
+
+
+def peer_credentials(conn: socket.socket) -> Optional[PeerCredentials]:
+    """The connecting process's pid/uid/gid, where the kernel reports them.
+
+    ``SO_PEERCRED`` is Linux's; elsewhere this is None and the socket file's
+    mode is the only gate.
+    """
+    option = getattr(socket, 'SO_PEERCRED', None)
+    if option is None:
+        return None
+    try:
+        raw = conn.getsockopt(socket.SOL_SOCKET, option, struct.calcsize('3i'))
+        pid, uid, gid = struct.unpack('3i', raw)
+    except (OSError, struct.error):
+        return None
+    return PeerCredentials(pid=pid, uid=uid, gid=gid)
+
+
+def process_groups(pid: int) -> Optional[FrozenSet[int]]:
+    """A process's supplementary groups, from /proc; None when unreadable.
+
+    The web service's primary group is normally its user's own; the shared
+    group is a supplementary one, which ``SO_PEERCRED`` does not report.
+    """
+    try:
+        with open(f'/proc/{int(pid)}/status', 'r', encoding='ascii', errors='replace') as f:
+            for line in f:
+                if line.startswith('Groups:'):
+                    return frozenset(int(g) for g in line.split()[1:] if g.isdigit())
+    except (OSError, ValueError):
+        return None
+    return frozenset()
+
+
+def user_in_group(uid: int, gid: int) -> bool:
+    """Whether the account ``uid`` is listed in group ``gid`` (the group database)."""
+    try:
+        import grp
+        import pwd
+        name = pwd.getpwuid(uid).pw_name
+        group = grp.getgrgid(gid)
+    except (ImportError, KeyError, OSError):
+        return False
+    return name in group.gr_mem or pwd.getpwuid(uid).pw_gid == gid
+
+
+def peer_allowed(cred: PeerCredentials, own_uid: int, allowed_gid: Optional[int],
+                 groups: Optional[FrozenSet[int]] = None,
+                 in_group: Callable[[int, int], bool] = user_in_group) -> bool:
+    """The permission model: root, the server's own user, or the shared group.
+
+    ``groups`` are the peer's supplementary groups (from /proc); when they
+    could not be read the group database decides instead.
+    """
+    if cred.uid == 0 or cred.uid == own_uid:
+        return True
+    if allowed_gid is None:
+        return False
+    if cred.gid == allowed_gid:
+        return True
+    if groups is not None:
+        return allowed_gid in groups
+    return in_group(cred.uid, allowed_gid)
+
+
+def resolve_socket_group(cache_dir: Optional[str]) -> Optional[int]:
+    """The group the socket should belong to: the one the two services share.
+
+    The cache directory's group when the directory is group-writable --
+    the rule DiskCache applies to every file the display shares with the web
+    interface (``root:ledmatrix 2775`` on an installed device). Otherwise the
+    project directory's group (``get_shared_group_gid``), which config files
+    use. None when neither is known: then only root and the display's own
+    user can connect.
+    """
+    if cache_dir:
+        try:
+            st = os.stat(cache_dir)
+            if st.st_mode & stat.S_IWGRP:
+                return st.st_gid
+        except OSError:
+            pass
+    try:
+        from src.common.permission_utils import get_shared_group_gid
+        return get_shared_group_gid()
+    except ImportError:  # pragma: no cover - src is always importable here
+        return None
+
+
+def server_socket_path(environ: Optional[Mapping[str, str]] = None) -> Optional[str]:
+    """Where the display should serve the socket; None when it should not.
+
+    :data:`~src.ipc.contract.SOCKET_PATH_ENV` wins. Otherwise
+    /run/ledmatrix/control.sock when the display can create or write that
+    directory (root, which an installed display always is), and the per-user
+    dev path otherwise (an emulator run from a checkout).
+    """
+    if not socket_supported() or socket_disabled(environ):
+        return None
+    configured = configured_socket_path(environ)
+    if configured:
+        return configured
+    geteuid = getattr(os, 'geteuid', None)
+    if (geteuid is not None and geteuid() == 0) or os.access(DEFAULT_SOCKET_DIR, os.W_OK):
+        return DEFAULT_SOCKET_PATH
+    return dev_socket_path()
+
+
+# -- the server ------------------------------------------------------------------------
+
+StatusProvider = Callable[[], Dict[str, Any]]
+
+
+class ControlServer:
+    """Serves the control socket on background threads.
+
+    ``start()`` binds and starts accepting; ``drain()`` (render thread) takes
+    the queued commands; ``close()`` stops and removes the socket file.
+    """
+
+    def __init__(self, path: str, status_provider: Optional[StatusProvider] = None,
+                 group: Optional[int] = None, *, queue_size: int = QUEUE_SIZE,
+                 max_clients: int = MAX_CLIENTS, io_timeout: float = IO_TIMEOUT_SECONDS,
+                 message_timeout: float = MESSAGE_TIMEOUT_SECONDS,
+                 idle_timeout: float = IDLE_TIMEOUT_SECONDS,
+                 check_peer: bool = True):
+        self.path = path
+        self._status_provider = status_provider
+        self._group = group
+        self._queue: 'queue.Queue[QueuedCommand]' = queue.Queue(maxsize=queue_size)
+        self._pending = threading.Event()
+        self._slots = threading.BoundedSemaphore(max_clients)
+        self._io_timeout = io_timeout
+        self._message_timeout = message_timeout
+        self._idle_timeout = idle_timeout
+        self._check_peer = check_peer
+        self._sock: Optional[socket.socket] = None
+        self._identity: Optional[tuple] = None   # (st_dev, st_ino) of our socket file
+        self._thread: Optional[threading.Thread] = None
+        self._stopping = threading.Event()
+        self._own_uid = os.geteuid() if hasattr(os, 'geteuid') else -1
+
+    # -- lifecycle -------------------------------------------------------------
+
+    @property
+    def running(self) -> bool:
+        return self._thread is not None and self._thread.is_alive()
+
+    @property
+    def socket_mode(self) -> int:
+        """0660 with a shared group; 0600 (the display's user only) without one."""
+        return 0o660 if self._group is not None else 0o600
+
+    def start(self) -> bool:
+        """Bind and start serving. False (logged) when the socket cannot be served.
+
+        Never raises: without the socket the web interface uses the file
+        mailbox, exactly as before.
+        """
+        if not socket_supported():
+            logger.debug("Control socket not started: no Unix sockets on this platform")
+            return False
+        try:
+            self._prepare_directory()
+            if not self._clear_stale_socket():
+                return False
+            self._bind()
+        except OSError as e:
+            logger.warning("Control socket not started at %s (%s); the web interface "
+                           "will use the file mailbox", self.path, e)
+            self._close_socket()
+            return False
+        self._stopping.clear()
+        self._thread = threading.Thread(target=self._accept_loop, name='ledmatrix-ipc',
+                                        daemon=True)
+        self._thread.start()
+        logger.info("Control socket listening at %s (mode %o, group %s)",
+                    self.path, self.socket_mode,
+                    self._group if self._group is not None else 'none')
+        return True
+
+    def close(self) -> None:
+        """Stop accepting and remove the socket file (only if it is still ours)."""
+        self._stopping.set()
+        self._close_socket()
+        thread = self._thread
+        if thread is not None and thread is not threading.current_thread():
+            thread.join(timeout=2.0)
+        self._thread = None
+        if self._identity is not None:
+            try:
+                st = os.lstat(self.path)
+                if (st.st_dev, st.st_ino) == self._identity:
+                    os.unlink(self.path)
+            except OSError:
+                pass
+            self._identity = None
+
+    def _close_socket(self) -> None:
+        sock, self._sock = self._sock, None
+        if sock is not None:
+            try:
+                sock.close()
+            except OSError:
+                pass
+
+    def _prepare_directory(self) -> None:
+        directory = os.path.dirname(os.path.abspath(self.path))
+        if self.path == dev_socket_path():
+            # The dev path is in the shared temp dir: private to this user,
+            # and refused if someone else got there first.
+            os.makedirs(directory, mode=0o700, exist_ok=True)
+            self._check_private_directory(directory)
+        elif not os.path.isdir(directory):
+            # /run/ledmatrix under a unit that predates RuntimeDirectory= (the
+            # display is root and makes it, as it does for the heartbeat), or
+            # a configured path. 0755: the web interface only needs to reach
+            # the socket; the socket's own mode decides who may connect.
+            os.makedirs(directory, mode=0o755, exist_ok=True)
+
+    def _check_private_directory(self, directory: str) -> None:
+        """Refuse a dev directory someone else made (it lives in a shared /tmp)."""
+        st = os.lstat(directory)
+        if stat.S_ISLNK(st.st_mode) or not stat.S_ISDIR(st.st_mode):
+            raise OSError(f'{directory} is not a plain directory')
+        if hasattr(os, 'geteuid') and st.st_uid != os.geteuid():
+            raise OSError(f'{directory} belongs to uid {st.st_uid}, not this user')
+
+    def _clear_stale_socket(self) -> bool:
+        """Remove a socket left by a display that died; never a live or foreign file."""
+        try:
+            st = os.lstat(self.path)
+        except FileNotFoundError:
+            return True
+        if not stat.S_ISSOCK(st.st_mode):
+            logger.error("Control socket not started: %s exists and is not a socket", self.path)
+            return False
+        probe = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        probe.settimeout(0.5)
+        try:
+            probe.connect(self.path)
+        except OSError:
+            os.unlink(self.path)   # nothing listening: a previous display's leftover
+            return True
+        finally:
+            probe.close()
+        logger.warning("Control socket not started: another process is serving %s", self.path)
+        return False
+
+    def _bind(self) -> None:
+        """Bind under a temporary name, set mode and group, then rename into place.
+
+        The rename makes the socket appear with its final permissions, never
+        briefly with the process umask's.
+        """
+        tmp = f'{self.path}.{os.getpid()}.tmp'
+        try:
+            os.unlink(tmp)
+        except FileNotFoundError:
+            pass
+        sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        self._sock = sock
+        try:
+            sock.bind(tmp)
+            os.chmod(tmp, self.socket_mode)
+            if self._group is not None and hasattr(os, 'chown'):
+                try:
+                    os.chown(tmp, -1, self._group)
+                except OSError as e:
+                    # Not root and not in the group (a dev run): only this user
+                    # (and root) can connect, which is what a dev run needs.
+                    logger.debug("Could not give the control socket group %s: %s",
+                                 self._group, e)
+            # The backlog is only the kernel's queue in front of accept();
+            # MAX_CLIENTS still bounds what is served. A short one makes a
+            # burst of clients fail connect() with EAGAIN instead of being
+            # answered (busy or otherwise).
+            sock.listen(_LISTEN_BACKLOG)
+            sock.settimeout(_ACCEPT_POLL_SECONDS)
+            os.rename(tmp, self.path)
+        except BaseException:
+            try:
+                os.unlink(tmp)
+            except OSError:
+                pass
+            raise
+        st = os.lstat(self.path)
+        self._identity = (st.st_dev, st.st_ino)
+
+    # -- the render thread's side ------------------------------------------------
+
+    @property
+    def has_pending(self) -> bool:
+        """Cheap check for queued commands, for the render thread's fast path."""
+        return self._pending.is_set()
+
+    def drain(self) -> List[QueuedCommand]:
+        """Every queued command, oldest first. Called from the render thread."""
+        commands: List[QueuedCommand] = []
+        self._pending.clear()
+        while True:
+            try:
+                commands.append(self._queue.get_nowait())
+            except queue.Empty:
+                break
+        return commands
+
+    # -- serving -------------------------------------------------------------------
+
+    def _accept_loop(self) -> None:
+        while not self._stopping.is_set():
+            sock = self._sock
+            if sock is None:
+                break
+            try:
+                conn, _ = sock.accept()
+            except socket.timeout:
+                continue
+            except OSError as e:
+                if self._stopping.is_set():
+                    break
+                logger.warning("Control socket accept failed: %s", e)
+                time.sleep(0.1)
+                continue
+            if not self._slots.acquire(blocking=False):
+                self._refuse(conn, ErrorCode.BUSY, 'too many connections')
+                continue
+            try:
+                threading.Thread(target=self._serve, args=(conn,), name='ledmatrix-ipc-conn',
+                                 daemon=True).start()
+            except RuntimeError:  # can't start a thread: shed the client
+                self._slots.release()
+                self._refuse(conn, ErrorCode.BUSY, 'server overloaded')
+
+    def _refuse(self, conn: socket.socket, code: str, message: str) -> None:
+        try:
+            conn.settimeout(0.2)
+            conn.sendall(encode_message(Response.failure(None, code, message).to_dict()))
+        except OSError:
+            pass
+        finally:
+            conn.close()
+
+    def _serve(self, conn: socket.socket) -> None:
+        """One connection: authenticate, then answer requests until it ends."""
+        try:
+            conn.settimeout(self._io_timeout)
+            peer = peer_credentials(conn)
+            if self._check_peer and peer is not None and not self._peer_ok(peer):
+                logger.warning("Control socket refused pid %d (uid %d, gid %d): not root, "
+                               "this user or group %s", peer.pid, peer.uid, peer.gid, self._group)
+                self._send(conn, Response.failure(None, ErrorCode.FORBIDDEN, 'not permitted'))
+                return
+            self._read_requests(conn, peer)
+        except Exception:  # pylint: disable=broad-except
+            logger.exception("Control socket connection failed")
+        finally:
+            try:
+                conn.close()
+            except OSError:
+                pass
+            self._slots.release()
+
+    def _peer_ok(self, peer: PeerCredentials) -> bool:
+        groups = None
+        if peer.uid not in (0, self._own_uid) and self._group is not None:
+            groups = process_groups(peer.pid)
+        return peer_allowed(peer, self._own_uid, self._group, groups)
+
+    def _read_requests(self, conn: socket.socket, peer: Optional[PeerCredentials]) -> None:
+        reader = FrameReader(MAX_MESSAGE_BYTES)
+        idle_since = time.monotonic()
+        message_started: Optional[float] = None
+        while not self._stopping.is_set():
+            now = time.monotonic()
+            if message_started is not None and now - message_started > self._message_timeout:
+                logger.debug("Control socket: dropping a client too slow to send a message")
+                return
+            if message_started is None and now - idle_since > self._idle_timeout:
+                return
+            try:
+                data = conn.recv(4096)
+            except socket.timeout:
+                continue
+            except OSError:
+                return
+            if not data:
+                return  # closed, possibly mid-message: nothing to answer
+            try:
+                lines = reader.feed(data)
+            except ProtocolError as e:
+                self._send(conn, Response.failure(None, e.code, e.message))
+                return  # can't find the next message boundary: hang up
+            for line in lines:
+                if not self._send(conn, self.handle_line(line, peer)):
+                    return
+            if reader.pending:
+                if message_started is None or lines:
+                    message_started = time.monotonic()
+            else:
+                message_started = None
+                idle_since = time.monotonic()
+
+    def _send(self, conn: socket.socket, response: Response) -> bool:
+        try:
+            data = encode_message(response.to_dict())
+        except ProtocolError as e:
+            # A status snapshot too big (or not JSON) to send is a display bug.
+            logger.error("Control socket response not sent: %s", e.message)
+            data = encode_message(Response.failure(
+                response.id, ErrorCode.INTERNAL, 'response could not be encoded').to_dict())
+        try:
+            conn.sendall(data)
+            return True
+        except OSError:
+            return False
+
+    # -- requests --------------------------------------------------------------------
+
+    def handle_line(self, line: bytes, peer: Optional[PeerCredentials] = None) -> Response:
+        """Answer one request line. Never raises."""
+        request_id: Optional[str] = None
+        try:
+            obj = decode_message(line)
+            raw_id = obj.get('id')
+            request_id = raw_id if isinstance(raw_id, str) and len(raw_id) <= 128 else None
+            request = Request.from_dict(obj)
+            request_id = request.id
+            return self._dispatch(request, peer)
+        except ProtocolError as e:
+            return Response.failure(e.request_id or request_id, e.code, e.message)
+        except Exception:  # pylint: disable=broad-except
+            logger.exception("Control socket handler failed")
+            return Response.failure(request_id, ErrorCode.INTERNAL, 'internal error')
+
+    def _dispatch(self, request: Request, peer: Optional[PeerCredentials]) -> Response:
+        if request.cmd == Command.HELLO:
+            # Exempt from the envelope version check: this is how a client
+            # that speaks other versions finds out which ones we share.
+            hello = HelloArgs.from_dict(request.args)
+            version = negotiate_version(hello.versions)
+            if version is None:
+                return Response.failure(
+                    request.id, ErrorCode.UNSUPPORTED_VERSION,
+                    f'no common protocol version; this display speaks {list(SUPPORTED_VERSIONS)}')
+            result: HelloResult = {
+                'version': version,
+                'versions': list(SUPPORTED_VERSIONS),
+                'commands': list(COMMANDS),
+                'max_message_bytes': MAX_MESSAGE_BYTES,
+                'server': 'ledmatrix-display',
+            }
+            return Response.success(request.id, dict(result), v=version)
+
+        if request.v not in SUPPORTED_VERSIONS:
+            return Response.failure(
+                request.id, ErrorCode.UNSUPPORTED_VERSION,
+                f'protocol version {request.v} is not supported; '
+                f'this display speaks {list(SUPPORTED_VERSIONS)}')
+
+        try:
+            args = parse_args(request.cmd, request.args)
+        except ProtocolError as e:
+            return Response.failure(request.id, e.code, e.message, v=request.v)
+
+        if request.cmd == Command.PING:
+            return Response.success(request.id, {'pong': True}, v=request.v)
+
+        if request.cmd == Command.ON_DEMAND_STATUS:
+            if self._status_provider is None:
+                return Response.failure(request.id, ErrorCode.INTERNAL, 'no status available',
+                                        v=request.v)
+            return Response.success(request.id, self._status_provider(), v=request.v)
+
+        if request.cmd in QUEUED_COMMANDS and isinstance(args, (OnDemandStartArgs,
+                                                               OnDemandStopArgs)):
+            command = QueuedCommand(request_id=request.id, cmd=request.cmd, args=args,
+                                    received_at=time.time(),
+                                    peer_uid=peer.uid if peer is not None else None)
+            try:
+                self._queue.put_nowait(command)
+            except queue.Full:
+                logger.warning("Control socket queue full; refusing %s %s",
+                               request.cmd, request.id)
+                return Response.failure(request.id, ErrorCode.BUSY,
+                                        'the display is not taking commands right now',
+                                        v=request.v)
+            self._pending.set()
+            ack: AckResult = {'accepted': True, 'request_id': request.id,
+                              'queued': self._queue.qsize()}
+            logger.info("Control socket accepted %s %s", request.cmd, request.id)
+            return Response.success(request.id, dict(ack), v=request.v)
+
+        # A command in COMMANDS with no handler here is a bug in this module.
+        return Response.failure(request.id, ErrorCode.INTERNAL,
+                                f'{request.cmd} is not implemented', v=request.v)
+
+
+def start_control_server(status_provider: Optional[StatusProvider] = None,
+                         cache_dir: Optional[str] = None,
+                         environ: Optional[Mapping[str, str]] = None) -> Optional[ControlServer]:
+    """Start the display's control socket, or return None when it can't run.
+
+    None covers Windows, ``LEDMATRIX_CONTROL_SOCKET=off`` and any failure to
+    bind; in every case the web interface falls back to the file mailbox.
+    """
+    path = server_socket_path(environ)
+    if path is None:
+        logger.debug("Control socket disabled or unsupported here; using the file mailbox only")
+        return None
+    server = ControlServer(path, status_provider, resolve_socket_group(cache_dir))
+    return server if server.start() else None
+
+
+__all__ = [
+    'ControlServer', 'PeerCredentials', 'QueuedCommand', 'StatusProvider',
+    'peer_allowed', 'peer_credentials', 'process_groups', 'resolve_socket_group',
+    'server_socket_path', 'start_control_server', 'PROTOCOL_VERSION',
+]

src/plugin_system/field_model.py

@@ -0,0 +1,740 @@
+"""
+One field model for a plugin's config form, built from its schema and config.
+
+Today a plugin's settings form is drawn by the ``render_field`` macro in
+``web_interface/templates/v3/partials/plugin_config.html``: about 1,100 lines
+of Jinja that walk the schema, pick a control per property (or hand it to a
+JS widget through an inline ``<script>``) and post flat dotted form keys that
+the server rebuilds into JSON. The JS widgets duplicate much of that, and the
+two drift.
+
+:func:`build_field_model` walks the schema once, the way the macro does, and
+returns a plain, JSON-serialisable tree describing every field: its dotted
+path, label, help, widget, starting value, default, constraints and options,
+and -- so that the model is provably complete before anything renders from
+it -- the exact form controls the macro emits for it today (``inputs``) and
+the JS widget it mounts (``mount``). ``test/test_field_model_parity.py``
+renders the real macro for every plugin schema it can find and checks that
+the names and starting values of those controls match the model exactly.
+
+Nothing renders from this yet. The plan (docs/WEB_FRONTEND_ARCHITECTURE.md):
+one ES-module renderer walks this model and mounts every field through the
+widget registry, the form posts JSON to the existing JSON save path, and the
+macro and the dotted-key reconstruction retire.
+
+The model mirrors the macro's behaviour, quirks included (an enum check runs
+before a number's, a list-typed ``type`` uses its first entry, a number whose
+default is ``null`` renders the text ``None``), because parity is the point of
+this stage. Fixing those is a renderer change for later, made once in one
+place.
+
+Shape (all keys always present unless noted)::
+
+    {
+      "version": 1,
+      "plugin_id": "...",
+      "rendered_sections": ["key", ...],   # the __rendered_section hidden inputs
+      "fields": [Field, ...],              # basic tier, in form order
+      "advanced_fields": [Field, ...],     # flat "x-advanced": true fields
+      "schemaless": false,                 # true: no schema, fields from config
+    }
+
+    Field = {
+      "key", "path", "id", "label", "help",
+      "type":      the macro's field type (first entry of a list type),
+      "widget":    what draws it: checkbox, select, number, text, csv-text,
+                   section, schedule-picker, time-range, style-editor,
+                   toggle-switch, slider, number-input, file-upload,
+                   checkbox-group, google-calendar-picker, day-selector,
+                   custom-feeds, array-table, color-picker, json-file-manager,
+                   any string widget the macro mounts (text-input, ...), or a
+                   plugin-supplied x-widget,
+      "x_widget":  the schema's x-widget, or None,
+      "value":     the value the form starts with,
+      "default":   the schema default (key absent when the schema has none),
+      "secret":    true for "x-secret" fields,
+      "advanced":  true in the Advanced Settings section,
+      "constraints": {minimum, maximum, ...} as declared,
+      "options":   [{"value", "label"}] for selects and checkbox groups,
+      "inputs":    [Input, ...] form controls the server renders,
+      "mount":     {"widget", "name", "value", "config", "plugin_widget"} or None,
+      "children":  [Field, ...] for sections (and a style-editor's fallback),
+      "columns" / "rows" / "max_items" for array-table and custom-feeds,
+      "stale_values" for checkbox groups, "error" for a mis-declared widget,
+    }
+
+    Input = {"name", "control", "value", "encoding"[, "checked"][, "options"]}
+      control:  hidden | text | number | url | date | time | checkbox | select
+      encoding: how ``value`` is written into the HTML today --
+                text  str(value)                  json  JSON text
+                csv   ", ".join(str(item))        bool  "true"/"false"
+                For a checkbox, ``value`` is its value attribute and
+                ``checked`` its state; for a select, ``value`` is the option
+                the browser submits and ``options`` the option values.
+
+Secret fields: pass the config *after* ``mask_secret_fields`` (as the route
+does); the model copies values verbatim.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any, Dict, Iterator, List, Optional, Tuple
+
+FIELD_MODEL_VERSION = 1
+
+#: String x-widgets the macro mounts as JS widgets (plugin_config.html's
+#: ``str_widget in [...]`` list). Any other x-widget on a string is a
+#: plugin-supplied widget, loaded through ensureWidget over a text fallback.
+STRING_WIDGETS = (
+    'text-input', 'textarea', 'select-dropdown', 'toggle-switch', 'radio-group',
+    'date-picker', 'time-picker', 'slider', 'color-picker', 'email-input',
+    'url-input', 'password-input', 'font-selector', 'file-upload-single',
+    'plugin-file-manager', 'google-oauth',
+)
+
+_CONSTRAINT_KEYS = (
+    'minimum', 'maximum', 'exclusiveMinimum', 'exclusiveMaximum', 'multipleOf',
+    'minLength', 'maxLength', 'pattern', 'format', 'minItems', 'maxItems',
+    'uniqueItems',
+)
+
+_MISSING = object()
+
+
+# ── Jinja semantics the macro relies on ─────────────────────────────────────
+
+def _is_string(value: Any) -> bool:
+    return isinstance(value, str)
+
+
+def _is_iterable(value: Any) -> bool:
+    """Jinja's ``is iterable``: anything ``iter()`` accepts (dicts included)."""
+    try:
+        iter(value)
+    except TypeError:
+        return False
+    return True
+
+
+def _is_list_like(value: Any) -> bool:
+    """``value is iterable and value is not string``."""
+    return value is not None and _is_iterable(value) and not _is_string(value)
+
+
+_WORD_SPLIT = re.compile(r'([-\s({\[<]+)')
+
+
+def _title(text: Any) -> str:
+    """Jinja's ``title`` filter (not str.title: it keeps "2xl" lower case)."""
+    return ''.join(
+        item[0].upper() + item[1:].lower()
+        for item in _WORD_SPLIT.split(str(text)) if item)
+
+
+def _humanise(key: Any) -> str:
+    """``key|replace('_', ' ')|title``."""
+    return _title(str(key).replace('_', ' '))
+
+
+def _x_options(prop: Dict[str, Any]) -> Dict[str, Any]:
+    return prop.get('x-options') or prop.get('x_options') or {}
+
+
+def _x_widget(prop: Dict[str, Any]) -> Optional[str]:
+    return prop.get('x-widget') or prop.get('x_widget')
+
+
+def is_hidden(prop: Any) -> bool:
+    """The macro's ``prop_is_hidden``: "x-display": "hidden", or an object
+    whose every child is hidden."""
+    if not isinstance(prop, dict):
+        return False
+    if prop.get('x-display') == 'hidden':
+        return True
+    children = prop.get('properties')
+    if isinstance(children, dict) and children:
+        return all(is_hidden(child) for child in children.values())
+    return False
+
+
+def field_type(prop: Dict[str, Any]) -> Any:
+    """The macro's field type: a string ``type``, the first entry of a list
+    ``type`` (so ``["null", "integer"]`` is ``"null"``), else ``"string"``.
+
+    Usually a string; a malformed list type hands back whatever its first
+    entry is, as the macro does."""
+    declared = prop.get('type')
+    if _is_string(declared):
+        return declared
+    if declared and _is_list_like(declared):
+        return next(iter(declared))
+    return 'string'
+
+
+def _column_type(col_def: Dict[str, Any]) -> Any:
+    """array-table's column type: the first non-"null" entry of a list type."""
+    raw = col_def.get('type', 'string')
+    if _is_list_like(raw):
+        rest = [entry for entry in raw if entry != 'null']
+        return rest[0] if rest and rest[0] else 'string'
+    return raw or 'string'
+
+
+def _array_value(value: Any, prop: Dict[str, Any]) -> Any:
+    """The value most array widgets draw: the stored list, else a list
+    default, else []."""
+    if _is_list_like(value):
+        return value
+    default = prop.get('default', _MISSING)
+    if default is not _MISSING and _is_list_like(default):
+        return default
+    return []
+
+
+def _constraints(prop: Dict[str, Any]) -> Dict[str, Any]:
+    return {key: prop[key] for key in _CONSTRAINT_KEYS if key in prop}
+
+
+def _input(name: str, control: str, value: Any, encoding: str = 'text',
+           **extra: Any) -> Dict[str, Any]:
+    item = {'name': name, 'control': control, 'value': value, 'encoding': encoding}
+    item.update(extra)
+    return item
+
+
+def _select_value(options: List[Any], matches: List[Any]) -> Any:
+    """What a single <select> submits: the last selected option, else the first."""
+    if matches:
+        return matches[-1]
+    return options[0] if options else None
+
+
+# ── fields ──────────────────────────────────────────────────────────────────
+
+def _base_node(key: str, prop: Dict[str, Any], value: Any, full_key: str,
+               plugin_id: str) -> Dict[str, Any]:
+    node: Dict[str, Any] = {
+        'key': key,
+        'path': full_key,
+        'id': f"{plugin_id}-{full_key}".replace('.', '-').replace('_', '-'),
+        'label': prop.get('title') or _humanise(key),
+        'help': prop.get('description') or '',
+        'type': field_type(prop),
+        'widget': None,
+        'x_widget': _x_widget(prop),
+        'value': value,
+        'secret': bool(prop.get('x-secret')),
+        'advanced': False,
+        'constraints': _constraints(prop),
+        'options': [],
+        'inputs': [],
+        'mount': None,
+        'children': [],
+    }
+    if 'default' in prop:
+        node['default'] = prop['default']
+    return node
+
+
+def _mount(widget: str, name: Optional[str], value: Any,
+           config: Optional[Dict[str, Any]] = None,
+           plugin_widget: bool = False) -> Dict[str, Any]:
+    return {'widget': widget, 'name': name, 'value': value,
+            'config': config or {}, 'plugin_widget': plugin_widget}
+
+
+def _build_field(key: str, prop: Any, value: Any, prefix: str,
+                 plugin_id: str) -> Optional[Dict[str, Any]]:
+    """``render_field``: one property, or None when the macro draws nothing."""
+    if not isinstance(prop, dict) or is_hidden(prop):
+        return None
+    # A key the saved config doesn't have renders its schema default.
+    if value is None and 'default' in prop:
+        value = prop['default']
+    full_key = f"{prefix}.{key}" if prefix else key
+    node = _base_node(key, prop, value, full_key, plugin_id)
+    ftype = node['type']
+
+    if ftype == 'object':
+        return _object_field(node, key, prop, value, prefix, full_key, plugin_id)
+    if ftype == 'boolean':
+        _boolean_field(node, prop, value, full_key)
+    elif prop.get('enum'):
+        _enum_field(node, prop, value, full_key)
+    elif ftype in ('number', 'integer'):
+        _number_field(node, prop, value, full_key, ftype)
+    elif ftype == 'array':
+        _array_field(node, prop, value, full_key, plugin_id)
+    else:
+        _string_field(node, prop, value, full_key, ftype)
+    return node
+
+
+def _object_field(node: Dict[str, Any], key: str, prop: Dict[str, Any], value: Any,
+                  prefix: str, full_key: str, plugin_id: str) -> Optional[Dict[str, Any]]:
+    widget = _x_widget(prop)
+    obj_value = value if value is not None else {}
+    if widget in ('schedule-picker', 'time-range'):
+        node['widget'] = widget
+        node['value'] = obj_value
+        node['inputs'].append(_input(full_key, 'hidden', obj_value, 'json'))
+        node['mount'] = _mount(widget, None, obj_value, {'x-options': _x_options(prop)})
+        return node
+    if widget == 'style-editor':
+        # The widget renders its own inputs under full_key; until it loads,
+        # and wherever it declines a block, the nested section is the form.
+        node['widget'] = 'style-editor'
+        node['value'] = obj_value
+        node['mount'] = _mount('style-editor', full_key, obj_value, {'schema': prop})
+        node['children'] = [_section(key, prop, value, prefix, plugin_id)]
+        return node
+    if prop.get('properties'):
+        return _section(key, prop, value, prefix, plugin_id)
+    return None  # an object with no properties and no widget draws nothing
+
+
+def _section(key: str, prop: Dict[str, Any], value: Any, prefix: str,
+             plugin_id: str) -> Dict[str, Any]:
+    """``render_nested_section``: a collapsible block of child fields."""
+    full_key = f"{prefix}.{key}" if prefix else key
+    # Only a dict can be looked into; a legacy boolean is the block's
+    # `enabled` switch (schema_manager.legacy_bool_as_object).
+    properties = prop.get('properties') or {}
+    if isinstance(value, dict):
+        nested_value = value
+    elif isinstance(value, bool) and 'enabled' in properties:
+        nested_value = {'enabled': value}
+    else:
+        nested_value = {}
+    node = _base_node(key, prop, nested_value, full_key, plugin_id)
+    node['widget'] = 'section'
+    node['id'] = f"{plugin_id}-section-{full_key}".replace('.', '-').replace('_', '-')
+    order = prop['x-propertyOrder'] if 'x-propertyOrder' in prop else list(properties.keys())
+    for nested_key in order:
+        if nested_key in properties and not is_hidden(properties[nested_key]):
+            child = _build_field(nested_key, properties[nested_key],
+                                 nested_value[nested_key] if nested_key in nested_value else None,
+                                 full_key, plugin_id)
+            if child is not None:
+                node['children'].append(child)
+    return node
+
+
+def _boolean_field(node, prop, value, full_key):
+    if _x_widget(prop) == 'toggle-switch':
+        node['widget'] = 'toggle-switch'
+        node['mount'] = _mount('toggle-switch', full_key,
+                               value if value is not None else False,
+                               {'type': 'boolean', 'x-options': _x_options(prop)})
+    else:
+        node['widget'] = 'checkbox'
+        node['inputs'].append(_input(full_key, 'checkbox', 'true', checked=bool(value)))
+
+
+def _enum_field(node, prop, value, full_key):
+    options = list(prop['enum'])
+    labels = _x_options(prop).get('labels') or {}
+    node['widget'] = 'select'
+    node['options'] = [{'value': option,
+                        'label': labels.get(option, _humanise(option))
+                        if _hashable(option) else _humanise(option)}
+                       for option in options]
+    posted = _select_value(options, [option for option in options if value == option])
+    node['inputs'].append(_input(full_key, 'select', posted, options=options))
+
+
+def _hashable(value: Any) -> bool:
+    try:
+        hash(value)
+    except TypeError:
+        return False
+    return True
+
+
+def _number_field(node, prop, value, full_key, ftype):
+    widget = _x_widget(prop)
+    if widget in ('slider', 'number-input'):
+        node['widget'] = widget
+        node['mount'] = _mount(widget, full_key, value, {
+            'type': ftype,
+            'minimum': prop.get('minimum'),
+            'maximum': prop.get('maximum'),
+            'x-options': _x_options(prop),
+        })
+    else:
+        node['widget'] = 'number'
+        node['inputs'].append(_input(full_key, 'number', _text_value(value, prop)))
+
+
+def _text_value(value: Any, prop: Dict[str, Any]) -> Any:
+    """``value if value is not none else (prop.default if defined else '')``."""
+    if value is not None:
+        return value
+    return prop['default'] if 'default' in prop else ''
+
+
+def _array_field(node, prop, value, full_key, plugin_id):
+    items = prop.get('items') or {}
+    widget = _x_widget(prop) or (
+        'array-table' if (items.get('type') == 'object' and items.get('properties')) else None)
+
+    if widget == 'file-upload':
+        upload = prop.get('x-upload-config') or {}
+        images = _array_value(value, prop)
+        node.update(widget='file-upload', value=images)
+        node['constraints'].update({
+            'max_files': upload.get('max_files', 10),
+            'allowed_types': upload.get('allowed_types',
+                                        ['image/png', 'image/jpeg', 'image/bmp', 'image/gif']),
+            'max_size_mb': upload.get('max_size_mb', 5),
+            'plugin_id': upload.get('plugin_id', plugin_id),
+            'endpoint': upload.get('endpoint', '/api/v3/plugins/assets/upload'),
+            'file_type': upload.get('file_type', 'image'),
+        })
+        node['inputs'].append(_input(full_key, 'hidden', images, 'json'))
+    elif widget == 'checkbox-group':
+        _checkbox_group(node, prop, value, full_key)
+    elif widget == 'google-calendar-picker':
+        selected = _calendar_value(value, prop)
+        node.update(widget=widget, value=selected)
+        node['mount'] = _mount(widget, full_key, selected, {})
+    elif widget == 'day-selector':
+        days = _array_value(value, prop)
+        node.update(widget=widget, value=days)
+        node['mount'] = _mount(widget, full_key, days, {'x-options': _x_options(prop)})
+    elif widget == 'custom-feeds':
+        _custom_feeds(node, prop, value, full_key, items)
+    elif widget == 'array-table':
+        _array_table(node, prop, value, full_key, items)
+    elif widget == 'color-picker':
+        _color_picker(node, prop, value, full_key)
+    else:
+        # The comma-separated text input; any other x-widget lands here too.
+        default = prop.get('default', _MISSING)
+        values = value if value is not None else ([] if default is _MISSING else default)
+        node.update(widget='csv-text', value=values)
+        node['inputs'].append(_input(full_key, 'text',
+                                     values if _is_list_like(values) else '',
+                                     'csv' if _is_list_like(values) else 'text'))
+
+
+def _checkbox_group(node, prop, value, full_key):
+    selected = _array_value(value, prop)
+    items = prop.get('items') or {}
+    options = items.get('enum') or []
+    labels = (prop.get('x-options') or {}).get('labels') or {}
+    # A saved value that is no longer an option is dropped (and reported), so
+    # the save does not fail validation on a value nobody can see.
+    stale = [v for v in selected if v not in options] if options else []
+    if options:
+        selected = [v for v in selected if v in options]
+    node.update(widget='checkbox-group', value=selected, stale_values=stale)
+    node['options'] = [{'value': option,
+                        'label': labels.get(option, _humanise(option))
+                        if _hashable(option) else _humanise(option)}
+                       for option in options]
+    for option in options:
+        node['inputs'].append(_input(f"{full_key}[]", 'checkbox', option,
+                                     checked=option in selected))
+    node['inputs'].append(_input(f"{full_key}_data", 'hidden', selected, 'json'))
+    # Sentinel: posts the field even when every box is unchecked.
+    node['inputs'].append(_input(f"{full_key}[]", 'hidden', ''))
+
+
+def _calendar_value(value: Any, prop: Dict[str, Any]) -> Any:
+    """google-calendar-picker accepts a legacy comma-separated string."""
+    if value is not None and _is_string(value) and value:
+        return [part.strip() for part in value.split(',')]
+    if _is_list_like(value):
+        return value
+    default = prop.get('default', _MISSING)
+    if default is not _MISSING and _is_string(default) and default:
+        return [part.strip() for part in default.split(',')]
+    if default is not _MISSING and _is_list_like(default):
+        return default
+    return []
+
+
+def _custom_feeds(node, prop, value, full_key, items):
+    node['widget'] = 'custom-feeds'
+    item_properties = items.get('properties', {})
+    if not (item_properties.get('name') and item_properties.get('url')):
+        node['error'] = "Custom feeds widget requires 'name' and 'url' properties in items schema."
+        return
+    feeds = _array_value(value, prop)
+    node.update(value=feeds, rows=feeds, max_items=prop.get('maxItems', 50))
+    for index, item in enumerate(feeds):
+        base = f"{full_key}.{index}"
+        node['inputs'].append(_input(f"{base}.name", 'text', item.get('name', '')))
+        node['inputs'].append(_input(f"{base}.url", 'url', item.get('url', '')))
+        logo = item.get('logo') or {}
+        logo_path = logo.get('path', '')
+        if logo_path:
+            node['inputs'].append(_input(f"{base}.logo.path", 'hidden', logo_path))
+            if logo.get('id'):
+                node['inputs'].append(_input(f"{base}.logo.id", 'hidden', logo.get('id')))
+        enabled = bool(item.get('enabled', True))
+        node['inputs'].append(_input(f"{base}.enabled", 'hidden', enabled, 'bool'))
+        node['inputs'].append(_input(f"{base}.enabled", 'checkbox', 'true', checked=enabled))
+
+
+def _table_columns(prop: Dict[str, Any], item_properties: Dict[str, Any]) -> List[str]:
+    """x-columns minus hidden ones, else the first four simple properties."""
+    x_columns = prop.get('x-columns')
+    if x_columns:
+        return [name for name in x_columns if not is_hidden(item_properties.get(name))]
+    columns: List[str] = []
+    for name, col_def in item_properties.items():
+        if (col_def.get('type') not in ['object', 'array'] and len(columns) < 4
+                and not is_hidden(col_def)):
+            columns.append(name)
+    return columns
+
+
+def _array_table(node, prop, value, full_key, items):
+    item_properties = items.get('properties', {})
+    rows = _array_value(value, prop)
+    columns = _table_columns(prop, item_properties)
+    advanced = {k: v for k, v in item_properties.items()
+                if k not in columns and k != 'id' and not is_hidden(v)}
+    node.update(widget='array-table', value=rows, rows=rows,
+                max_items=prop.get('maxItems', 50))
+    node['columns'] = [{
+        'key': name,
+        'label': (item_properties.get(name) or {}).get('title', _humanise(name)),
+        'type': _column_type(item_properties.get(name) or {}),
+        'x_widget': ((item_properties.get(name) or {}).get('x-widget')
+                     or (item_properties.get(name) or {}).get('x_widget', '')),
+    } for name in columns]
+    node['advanced_columns'] = list(advanced)
+
+    for index, item in enumerate(rows):
+        base = f"{full_key}.{index}"
+        for name in columns:
+            node['inputs'].extend(_table_cell(f"{base}.{name}", item_properties.get(name, {}),
+                                              item.get(name, item_properties.get(name, {}).get('default', ''))))
+        # Hidden item properties have no control, but a posted row replaces
+        # the stored item wholesale, so their stored values are carried.
+        for k, v in item_properties.items():
+            if is_hidden(v) and k in item and item[k] is not None:
+                node['inputs'].append(_input(f"{base}.{k}", 'hidden', item[k], 'json'))
+        if advanced:
+            node['inputs'].extend(_advanced_cells(base, advanced, item))
+
+
+def _table_cell(name: str, col_def: Dict[str, Any], col_value: Any) -> List[Dict[str, Any]]:
+    col_type = _column_type(col_def)
+    col_widget = col_def.get('x-widget') or col_def.get('x_widget', '')
+    col_enum = col_def.get('enum', [])
+    if col_type == 'boolean':
+        return [_input(name, 'hidden', bool(col_value), 'bool'),
+                _input(name, 'checkbox', 'true', checked=bool(col_value))]
+    if col_type in ('integer', 'number'):
+        return [_input(name, 'number', col_value if col_value is not None else '')]
+    if col_enum:
+        options = [opt for opt in col_enum if opt is not None]
+        matches = [opt for opt in options
+                   if col_value == opt or (col_value is None and col_def.get('default') == opt)]
+        return [_input(name, 'select', _select_value(options, matches), options=options)]
+    if col_widget == 'date-picker':
+        return [_input(name, 'date', col_value if col_value is not None else '')]
+    if col_widget == 'time-picker':
+        return [_input(name, 'time', col_value if col_value is not None else '00:00')]
+    return [_input(name, 'text', col_value if col_value is not None else '')]
+
+
+def _advanced_cells(base: str, advanced: Dict[str, Any], item: Dict[str, Any]) -> List[Dict[str, Any]]:
+    """The row's hidden inputs for properties edited in the row editor."""
+    cells: List[Dict[str, Any]] = []
+    for prop_name, prop_schema in advanced.items():
+        if prop_schema.get('type', 'string') == 'object' and prop_schema.get('properties'):
+            stored = item.get(prop_name)
+            sub_obj = stored if isinstance(stored, dict) else {}
+            container = item.get(prop_name, {})
+            for sub_name, sub_schema in prop_schema.get('properties', {}).items():
+                name = f"{base}.{prop_name}.{sub_name}"
+                if is_hidden(sub_schema):
+                    if sub_name in sub_obj and sub_obj[sub_name] is not None:
+                        cells.append(_input(name, 'hidden', sub_obj[sub_name], 'json'))
+                    continue
+                sub_val = container.get(sub_name) if isinstance(container, dict) else None
+                final = sub_val if sub_val is not None else sub_schema.get('default')
+                cells.append(_input(name, 'hidden', final if final is not None else ''))
+        else:
+            stored = item.get(prop_name)
+            final = stored if stored is not None else prop_schema.get('default')
+            cells.append(_input(f"{base}.{prop_name}", 'hidden', final if final is not None else ''))
+    return cells
+
+
+def _color_picker(node, prop, value, full_key):
+    default = prop.get('default', _MISSING)
+    if _is_list_like(value):
+        rgb = value
+    elif default is not _MISSING and _is_list_like(default):
+        rgb = default
+    else:
+        rgb = [255, 255, 255]
+    channels = [rgb[i] if len(rgb) > i else 255 for i in range(3)]
+    node.update(widget='color-picker', value=rgb)
+    for index, channel in enumerate(channels):
+        node['inputs'].append(_input(f"{full_key}.{index}", 'number', channel))
+
+
+def _string_field(node, prop, value, full_key, ftype):
+    widget = _x_widget(prop)
+    text = _text_value(value, prop)
+    node['value'] = text
+    if widget == 'file-upload':
+        upload = prop.get('x-upload-config') or {}
+        node['widget'] = 'file-upload'
+        node['constraints'].update({
+            'upload_endpoint': upload.get('upload_endpoint', ''),
+            'target_filename': upload.get('target_filename', 'file.json'),
+            'max_size_mb': upload.get('max_size_mb', 1),
+            'allowed_extensions': upload.get('allowed_extensions', ['.json']),
+        })
+        node['inputs'].append(_input(full_key, 'hidden', text))
+    elif widget == 'json-file-manager':
+        # An iframe of the plugin's own file manager; it saves on its own.
+        node['widget'] = 'json-file-manager'
+    elif widget in STRING_WIDGETS:
+        node['widget'] = widget
+        node['mount'] = _mount(widget, full_key, text, {
+            'type': ftype,
+            'enum': prop.get('enum') or [],
+            'minimum': prop.get('minimum'),
+            'maximum': prop.get('maximum'),
+            'x-options': _x_options(prop),
+            'x-upload-config': prop.get('x-upload-config') or prop.get('x_upload_config') or {},
+            'x-widget-config': prop.get('x-widget-config') or prop.get('x_widget_config') or {},
+        })
+    else:
+        node['widget'] = widget or 'text'
+        node['inputs'].append(_input(full_key, 'text', text))
+        if widget:
+            # A plugin-supplied widget (manifest "widgets"); the text input
+            # stays as the fallback until it renders.
+            node['mount'] = _mount(widget, full_key, text, {
+                'type': ftype,
+                'enum': prop.get('enum') or [],
+                'x-options': _x_options(prop),
+                'x-widget-config': prop.get('x-widget-config') or prop.get('x_widget_config') or {},
+            }, plugin_widget=True)
+
+
+# ── the form ────────────────────────────────────────────────────────────────
+
+def _schemaless_field(key: str, value: Any) -> Dict[str, Any]:
+    """A plugin with no schema: one plain control per stored key."""
+    node = _base_node(key, {}, value, key, '')
+    node['id'] = 'fallback-field-' + str(key).replace(' ', '-')
+    if value is True or value is False:
+        node['widget'] = 'checkbox'
+        node['type'] = 'boolean'
+        # No value attribute, so a checked box posts "on".
+        node['inputs'].append(_input(key, 'checkbox', 'on', checked=bool(value)))
+    elif isinstance(value, (int, float, complex)):
+        node['widget'] = 'number'
+        node['type'] = 'number'
+        node['inputs'].append(_input(key, 'number', value))
+    else:
+        node['widget'] = 'text'
+        node['inputs'].append(_input(key, 'text', value))
+    return node
+
+
+def build_field_model(schema: Any, config: Any, plugin_id: str = '') -> Dict[str, Any]:
+    """The field model for one plugin's config form.
+
+    ``schema`` is the plugin's config schema as the route loads it
+    (``SchemaManager.load_schema``, so style elements are expanded);
+    ``config`` is the plugin's section after defaults are merged and secrets
+    masked, exactly what ``plugin_config.html`` is rendered with.
+    """
+    config = config if isinstance(config, dict) else {}
+    model: Dict[str, Any] = {
+        'version': FIELD_MODEL_VERSION,
+        'plugin_id': plugin_id,
+        'rendered_sections': [],
+        'fields': [],
+        'advanced_fields': [],
+        'schemaless': False,
+    }
+    properties = schema.get('properties') if isinstance(schema, dict) else None
+    if not properties:
+        model['schemaless'] = True
+        model['fields'] = [_schemaless_field(key, value)
+                           for key, value in config.items() if key not in ['enabled']]
+        return model
+
+    order = schema['x-propertyOrder'] if 'x-propertyOrder' in schema else list(properties.keys())
+    basic: List[str] = []
+    advanced: List[str] = []
+    for key in order:
+        if key in properties and key != 'enabled' and not is_hidden(properties[key]):
+            prop = properties[key]
+            declared = prop.get('type') if isinstance(prop, dict) else None
+            is_object = declared is not None and _is_iterable(declared) and 'object' in declared
+            if isinstance(prop, dict) and prop.get('x-advanced') and not is_object:
+                advanced.append(key)
+            else:
+                basic.append(key)
+    model['rendered_sections'] = basic + advanced
+
+    for tier, keys in (('fields', basic), ('advanced_fields', advanced)):
+        for key in keys:
+            node = _build_field(key, properties[key], config[key] if key in config else None,
+                                '', plugin_id)
+            if node is not None:
+                node['advanced'] = tier == 'advanced_fields'
+                model[tier].append(node)
+    return model
+
+
+# ── walking the model ───────────────────────────────────────────────────────
+
+def iter_fields(model: Dict[str, Any]) -> Iterator[Dict[str, Any]]:
+    """Every field node, depth first, in form order."""
+    def walk(nodes):
+        for node in nodes:
+            yield node
+            yield from walk(node.get('children') or [])
+    yield from walk(model.get('fields') or [])
+    yield from walk(model.get('advanced_fields') or [])
+
+
+def form_inputs(model: Dict[str, Any]) -> List[Dict[str, Any]]:
+    """Every server-rendered form control, in document order, starting with
+    the ``__rendered_section`` hidden inputs."""
+    inputs = [_input('__rendered_section', 'hidden', key)
+              for key in model.get('rendered_sections') or []]
+    for node in iter_fields(model):
+        inputs.extend(node.get('inputs') or [])
+    return inputs
+
+
+def widget_mounts(model: Dict[str, Any]) -> List[Dict[str, Any]]:
+    """Every JS widget the form mounts, in document order.
+
+    A style-editor's fallback section is drawn before the editor's own
+    script, so a node's children come before its own mount.
+    """
+    mounts: List[Dict[str, Any]] = []
+
+    def walk(nodes):
+        for node in nodes:
+            walk(node.get('children') or [])
+            if node.get('mount'):
+                mounts.append(node['mount'])
+    walk(model.get('fields') or [])
+    walk(model.get('advanced_fields') or [])
+    return mounts
+
+
+def field_names(model: Dict[str, Any]) -> List[Tuple[str, str]]:
+    """(name, source) for every posted name: 'form' controls and named 'widget' mounts."""
+    names = [(item['name'], 'form') for item in form_inputs(model)]
+    names += [(mount['name'], 'widget') for mount in widget_mounts(model) if mount['name']]
+    return names

src/plugin_system/plugin_executor.py

@@ -10,6 +10,7 @@ from typing import Any, Dict, Optional, Callable
 from threading import Thread
 import logging
 
+from src.common.fetch_service import plugin_scope
 from src.exceptions import PluginError
 from src.logging_config import get_logger
 from src.error_aggregator import record_error
@@ -83,7 +84,10 @@ class PluginExecutor:
         
         def target():
             try:
-                result_container['value'] = operation()
+                # Fetches made by the operation (and by threads the core
+                # starts from it) are counted against this plugin.
+                with plugin_scope(plugin_id):
+                    result_container['value'] = operation()
                 result_container['completed'] = True
             except Exception as e:
                 result_container['exception'] = e
@@ -173,7 +177,8 @@ class PluginExecutor:
         force_clear: bool = False,
         display_mode: Optional[str] = None,
         timeout: Optional[float] = None,
-        accepts_display_mode: Optional[bool] = None
+        accepts_display_mode: Optional[bool] = None,
+        raise_errors: bool = False
     ) -> bool:
         """
         Execute plugin display() method with error handling.
@@ -187,9 +192,18 @@ class PluginExecutor:
             accepts_display_mode: Whether plugin.display() takes a
                 display_mode keyword. Pass it when the caller already knows;
                 None falls back to inspecting the callable.
+            raise_errors: Re-raise the PluginError wrapping an exception
+                display() raised, instead of returning False. False alone
+                cannot tell "no content" from "raised", and a caller that
+                feeds the circuit breaker needs that difference. The error
+                is still logged and recorded first. A timeout still returns
+                False either way.
             
         Returns:
             True if display succeeded, False otherwise
+
+        Raises:
+            PluginError: Only with ``raise_errors``, when display() raised.
         """
         try:
             start_time = time.monotonic()
@@ -245,6 +259,8 @@ class PluginExecutor:
             return False
         except PluginError:
             # Already logged and recorded in execute_with_timeout
+            if raise_errors:
+                raise
             return False
         except Exception as e:
             self.logger.error(

src/plugin_system/plugin_manager.py

@@ -32,7 +32,7 @@ from src.plugin_system.schema_manager import (
 from src.plugin_system.plugin_dirs import (
     ManifestStatus, PluginDirectoryIndex, resolve_plugin_dir,
 )
-from src.deprecation import deprecated
+from src.common.fetch_service import plugin_scope, register_plugin_directory
 from src.common.permission_utils import (
     ensure_directory_permissions,
     get_plugin_dir_mode
@@ -424,6 +424,11 @@ class PluginManager:
             # Update mapping if found via search
             if plugin_id not in self.plugin_directories:
                 self.plugin_directories[plugin_id] = plugin_dir
+
+            # Code under this directory is this plugin's: the fetch service
+            # counts a request against it even from a thread the plugin
+            # started itself (src/common/fetch_service.py, caller identity).
+            register_plugin_directory(plugin_id, plugin_dir)
             
             # Get plugin config
             if self.config_manager:
@@ -463,18 +468,20 @@ class PluginManager:
                 config = dict(config)
                 config['enabled'] = True
             
-            # Use PluginLoader to load plugin
-            plugin_instance, _module = self.plugin_loader.load_plugin(
-                plugin_id=plugin_id,
-                manifest=manifest,
-                plugin_dir=plugin_dir,
-                config=config,
-                display_manager=self.display_manager,
-                cache_manager=self.cache_manager,
-                plugin_manager=self,
-                install_deps=True,
-                plugins_dir=self.plugins_dir,
-            )
+            # Use PluginLoader to load plugin. Fetches the constructor makes
+            # count against the plugin.
+            with plugin_scope(plugin_id):
+                plugin_instance, _module = self.plugin_loader.load_plugin(
+                    plugin_id=plugin_id,
+                    manifest=manifest,
+                    plugin_dir=plugin_dir,
+                    config=config,
+                    display_manager=self.display_manager,
+                    cache_manager=self.cache_manager,
+                    plugin_manager=self,
+                    install_deps=True,
+                    plugins_dir=self.plugins_dir,
+                )
             
             # Register plugin-shipped fonts with the FontManager (if any).
             # Plugin manifests can declare a "fonts" block that ships custom
@@ -528,7 +535,8 @@ class PluginManager:
                 # Call on_enable if plugin is enabled
                 if hasattr(plugin_instance, 'on_enable'):
                     try:
-                        plugin_instance.on_enable()
+                        with plugin_scope(plugin_id):
+                            plugin_instance.on_enable()
                     except Exception:
                         # Undo the registration above before the outer
                         # handler marks it ERROR: left in self.plugins, the
@@ -873,16 +881,6 @@ class PluginManager:
         """
         return self.plugins.copy()
     
-    @deprecated("3.8.0", "check each plugin's enabled flag in plugins")
-    def get_enabled_plugins(self) -> List[str]:
-        """
-        Get list of enabled plugin IDs.
-        
-        Returns:
-            List of plugin IDs that are currently enabled
-        """
-        return [pid for pid, plugin in self.plugins.items() if plugin.enabled]
-    
     def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
         """
         Get information about a plugin (manifest + runtime info).

src/plugin_system/testing/visual_display_manager.py

@@ -14,9 +14,7 @@ PIL Image canvas and draws text using the actual project fonts.
 MAINTENANCE WARNING: this class is a deliberate fork of
 src/display_manager.py so it can run without hardware. It mirrors
 these DisplayManager methods by name and behavior: _load_fonts,
-get_font_height, get_text_width, draw_text,
-draw_text_with_icons, draw_weather_icon (and the _draw_sun/_draw_cloud/
-_draw_rain/_draw_snow/_draw_storm family), format_date_with_ordinal,
+get_font_height, get_text_width, draw_text, format_date_with_ordinal,
 capture_mode, set_scrolling_state, is_currently_scrolling,
 process_deferred_updates, update_display, render_size, offscreen. A behavior
 change to any of those in DisplayManager must be mirrored here, or
@@ -26,13 +24,12 @@ BDF text is not mirrored: both classes load BDF faces and draw BDF glyphs
 through src/common/bdf_font.py, so those pixels cannot drift.
 """
 
-import math
 import os
 import time
 import warnings
 from contextlib import contextmanager
 from pathlib import Path
-from typing import Any, List, Optional, Tuple
+from typing import Any, Optional, Tuple
 
 from PIL import Image, ImageDraw, ImageFont
 from src.common.bdf_font import draw_bdf_text, load_bdf_face
@@ -63,15 +60,6 @@ class VisualTestDisplayManager:
     no emulator dependency.
     """
 
-    # Weather icon color constants (same as DisplayManager)
-    WEATHER_COLORS = {
-        'sun': (255, 200, 0),
-        'cloud': (200, 200, 200),
-        'rain': (0, 100, 255),
-        'snow': (220, 220, 255),
-        'storm': (255, 255, 0),
-    }
-
     def __init__(self, width: int = 128, height: int = 32):
         self._width = width
         self._height = height
@@ -410,129 +398,6 @@ class VisualTestDisplayManager:
                 return font.size
             return 8
 
-    # ------------------------------------------------------------------
-    # Weather drawing helpers
-    # ------------------------------------------------------------------
-
-    def draw_sun(self, x: int, y: int, size: int = 16):
-        """Draw a sun icon using yellow circles and lines."""
-        self._draw_sun(x, y, size)
-
-    def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):
-        """Draw a cloud icon."""
-        self._draw_cloud(x, y, size, color)
-
-    def draw_rain(self, x: int, y: int, size: int = 16):
-        """Draw rain icon with cloud and droplets."""
-        self._draw_rain(x, y, size)
-
-    def draw_snow(self, x: int, y: int, size: int = 16):
-        """Draw snow icon with cloud and snowflakes."""
-        self._draw_snow(x, y, size)
-
-    def _draw_sun(self, x: int, y: int, size: int) -> None:
-        """Draw a sun icon with rays (internal weather icon version)."""
-        center_x, center_y = x + size // 2, y + size // 2
-        radius = size // 4
-        ray_length = size // 3
-        self.draw.ellipse(
-            [center_x - radius, center_y - radius,
-             center_x + radius, center_y + radius],
-            fill=self.WEATHER_COLORS['sun'],
-        )
-        for angle in range(0, 360, 45):
-            rad = math.radians(angle)
-            start_x = center_x + int((radius + 2) * math.cos(rad))
-            start_y = center_y + int((radius + 2) * math.sin(rad))
-            end_x = center_x + int((radius + ray_length) * math.cos(rad))
-            end_y = center_y + int((radius + ray_length) * math.sin(rad))
-            self.draw.line([start_x, start_y, end_x, end_y], fill=self.WEATHER_COLORS['sun'], width=2)
-
-    def _draw_cloud(self, x: int, y: int, size: int, color: Optional[Tuple[int, int, int]] = None) -> None:
-        """Draw a cloud using multiple circles (internal weather icon version)."""
-        cloud_color = color if color is not None else self.WEATHER_COLORS['cloud']
-        base_y = y + size // 2
-        circle_radius = size // 4
-        positions = [
-            (x + size // 3, base_y),
-            (x + size // 2, base_y - size // 6),
-            (x + 2 * size // 3, base_y),
-        ]
-        for cx, cy in positions:
-            self.draw.ellipse(
-                [cx - circle_radius, cy - circle_radius,
-                 cx + circle_radius, cy + circle_radius],
-                fill=cloud_color,
-            )
-
-    def _draw_rain(self, x: int, y: int, size: int) -> None:
-        """Draw rain drops falling from a cloud."""
-        self._draw_cloud(x, y, size)
-        rain_color = self.WEATHER_COLORS['rain']
-        drop_size = size // 8
-        drops = [
-            (x + size // 4, y + 2 * size // 3),
-            (x + size // 2, y + 3 * size // 4),
-            (x + 3 * size // 4, y + 2 * size // 3),
-        ]
-        for dx, dy in drops:
-            self.draw.line([dx, dy, dx - drop_size // 2, dy + drop_size], fill=rain_color, width=2)
-
-    def _draw_snow(self, x: int, y: int, size: int) -> None:
-        """Draw snowflakes falling from a cloud."""
-        self._draw_cloud(x, y, size)
-        snow_color = self.WEATHER_COLORS['snow']
-        flake_size = size // 6
-        flakes = [
-            (x + size // 4, y + 2 * size // 3),
-            (x + size // 2, y + 3 * size // 4),
-            (x + 3 * size // 4, y + 2 * size // 3),
-        ]
-        for fx, fy in flakes:
-            for angle in range(0, 360, 60):
-                rad = math.radians(angle)
-                end_x = fx + int(flake_size * math.cos(rad))
-                end_y = fy + int(flake_size * math.sin(rad))
-                self.draw.line([fx, fy, end_x, end_y], fill=snow_color, width=1)
-
-    def _draw_storm(self, x: int, y: int, size: int) -> None:
-        """Draw a storm cloud with lightning bolt."""
-        self._draw_cloud(x, y, size)
-        bolt_color = self.WEATHER_COLORS['storm']
-        bolt_points = [
-            (x + size // 2, y + size // 2),
-            (x + 3 * size // 5, y + 2 * size // 3),
-            (x + 2 * size // 5, y + 2 * size // 3),
-            (x + size // 2, y + 5 * size // 6),
-        ]
-        self.draw.polygon(bolt_points, fill=bolt_color)
-
-    def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:
-        """Draw a weather icon based on the condition."""
-        cond = condition.lower()
-        if cond in ('clear', 'sunny'):
-            self._draw_sun(x, y, size)
-        elif cond in ('clouds', 'cloudy', 'partly cloudy'):
-            self._draw_cloud(x, y, size)
-        elif cond in ('rain', 'drizzle', 'shower'):
-            self._draw_rain(x, y, size)
-        elif cond in ('snow', 'sleet', 'hail'):
-            self._draw_snow(x, y, size)
-        elif cond in ('thunderstorm', 'storm'):
-            self._draw_storm(x, y, size)
-        else:
-            self._draw_sun(x, y, size)
-
-    def draw_text_with_icons(self, text: str, icons: List[tuple] = None,
-                             x: int = None, y: int = None,
-                             color: tuple = (255, 255, 255)):
-        """Draw text with weather icons at specified positions."""
-        self.draw_text(text, x, y, color)
-        if icons:
-            for icon_type, icon_x, icon_y in icons:
-                self.draw_weather_icon(icon_type, icon_x, icon_y)
-        self.update_display()
-
     # ------------------------------------------------------------------
     # Scrolling state (no-op interface compat)
     # ------------------------------------------------------------------

src/scan_order.py

@@ -23,6 +23,10 @@ above it near the end, the section below must show one more refresh of lag to
 stay continuous with it (and one less where the order jumps the other way).
 Stacked parallel chains are lit simultaneously, so each further half adds one.
 
+A frame held for several refreshes (a slower, crisp scroll) is presented as a
+sequence of swaps instead of one long hold, so the lagging half can step one
+refresh after the rest: see :func:`refresh_plan`.
+
 Only layouts whose physical row order is known are compensated: plain chains,
 parallel chains, and a 0 or 180 degree rotation. Other pixel mappers
 (U-mapper, 90/270 rotation, ...), special multiplexing and interlaced scan are
@@ -109,19 +113,47 @@ def scan_lag_bands(hardware: Mapping[str, Any], height: int,
     return [band for band in bands if band[2] > 0] or None
 
 
-def compose(image: Image.Image, history: Sequence[Image.Image],
-            bands: Sequence[Band]) -> Image.Image:
-    """``image`` with each band taken from the frame ``lag`` refreshes back.
+def refresh_plan(bands: Sequence[Band], hold: int) -> List[Tuple[Tuple[int, ...], int]]:
+    """How to present one frame that is held for ``hold`` refreshes.
 
-    ``history[0]`` is the previous frame. A band whose frame is not available
-    yet (the first frames of a scroll) is left current. Returns ``image`` itself
-    when nothing changes, so the caller pays for a copy only when it must.
+    A band lagging ``lag`` refreshes shows, on refresh ``r`` of the frame, what
+    the panel showed ``lag`` refreshes earlier: the current frame once
+    ``r >= lag``, else a frame ``ceil((lag - r) / hold)`` back. At one refresh
+    per frame that is just ``lag`` frames back. Held longer, the lagging band
+    steps one refresh after the rest instead of one frame, which is the only
+    way to cancel the offset: it is a fraction of a frame there.
+
+    Returns ``[(frames_back_per_band, refreshes), ...]`` in order, merging
+    neighbouring refreshes that show the same thing so each costs one swap.
+    """
+    plan: List[Tuple[Tuple[int, ...], int]] = []
+    for r in range(max(1, hold)):
+        backs = tuple(max(0, -((r - lag) // max(1, hold))) for _, _, lag in bands)
+        if plan and plan[-1][0] == backs:
+            plan[-1] = (backs, plan[-1][1] + 1)
+        else:
+            plan.append((backs, 1))
+    return plan
+
+
+def compose(image: Image.Image, history: Sequence[Image.Image],
+            bands: Sequence[Band],
+            backs: Optional[Sequence[int]] = None) -> Image.Image:
+    """``image`` with each band taken from an earlier frame.
+
+    ``history[0]`` is the previous frame. ``backs`` is how many frames back each
+    band is taken from (0 = the current one); by default that is the band's lag,
+    which is right when every frame is held for one refresh. A band whose frame
+    is not available yet (the first frames of a scroll) is left current.
+    Returns ``image`` itself when nothing changes, so the caller pays for a copy
+    only when it must.
     """
     out = image
-    for top, bottom, lag in bands:
-        if lag > len(history):
+    for i, (top, bottom, lag) in enumerate(bands):
+        back = lag if backs is None else backs[i]
+        if back <= 0 or back > len(history):
             continue
-        source = history[lag - 1]
+        source = history[back - 1]
         if source.size != image.size:
             continue
         if out is image:

src/vegas_mode/live_worker.py

@@ -369,6 +369,7 @@ class VegasWorker(threading.Thread):
             member = p.stream_manager.fetch_group_member(
                 job.pending.pop(0), offscreen_only=True)
             if member is not None:
+                p.prepare_group_member(member)
                 job.group.append(member)
         if not job.pending:
             self._group_job = None

src/vegas_mode/render_pipeline.py

@@ -13,6 +13,7 @@ import threading
 from collections import deque
 from contextlib import nullcontext
 from typing import Optional, List, Any, Dict, Deque, Tuple
+import numpy as np
 from PIL import Image
 
 from src.common.scroll_config import solve_crisp
@@ -77,6 +78,19 @@ def join_plugin_rows(
     return block, layout
 
 
+class PreparedBlock:
+    """One plugin's block, joined and turned into pixels ahead of the strip."""
+
+    __slots__ = ('images', 'block', 'layout', 'pixels')
+
+    def __init__(self, images: List[Image.Image], config: VegasModeConfig) -> None:
+        # Held so the id() it is filed under cannot be reused while it waits.
+        self.images = images
+        self.block, self.layout = join_plugin_rows(images, config)
+        block = self.block if self.block.mode == 'RGB' else self.block.convert('RGB')
+        self.pixels = np.asarray(block)
+
+
 class RenderPipeline:
     """
     High-performance render pipeline for Vegas scroll mode.
@@ -115,6 +129,17 @@ class RenderPipeline:
     # without __init__ (tests).
     _static_markers: Tuple[Tuple[int, str], ...] = ()
 
+    # Blocks joined off the render thread by whichever thread fetched the
+    # group (prepare_group_member): id(images) -> PreparedBlock. Laying a
+    # plugin's rows out and turning the block into pixels cost the frame
+    # after every extension ~35 ms on a Pi 4 when done there. Only producer
+    # threads add entries and only extend_scroll_content takes them; a
+    # reset drops the lot. Made on first use, so pipelines built without
+    # __init__ (tests) work too.
+    _prepared_blocks: Optional[Dict[int, 'PreparedBlock']] = None
+    #: Blocks kept waiting at most; a group is a handful of plugins.
+    PREPARED_BLOCKS_MAX = 64
+
     # Live elements in the strip (see "live element records" below). Replaced,
     # never mutated, like _static_markers, and class-level for the same reason.
     _elements: Tuple[ElementRecord, ...] = ()
@@ -466,6 +491,11 @@ class RenderPipeline:
         if note is not None:
             note(kind, nbytes)
 
+    def _copied_bytes(self) -> int:
+        """Bytes the scroll helper's last append or trim copied (the strip, if it cannot say)."""
+        copied = getattr(self.scroll_helper, 'last_copy_bytes', None)
+        return int(copied) if isinstance(copied, int) else self._strip_nbytes()
+
     def _strip_nbytes(self) -> int:
         array = self.scroll_helper.cached_array
         return int(array.nbytes) if array is not None else 0
@@ -593,6 +623,8 @@ class RenderPipeline:
                 try:
                     with gate.yielding() if gate is not None else nullcontext():
                         group = self.stream_manager.take_next_group(offscreen_only=True)
+                        for member in group or ():
+                            self.prepare_group_member(member)
                 except Exception:
                     logger.exception("Background prefetch failed")
                     group = []
@@ -660,7 +692,7 @@ class RenderPipeline:
             element_gap=0,
         )
         if appended:
-            self._note_op('extend', self._strip_nbytes())
+            self._note_op('extend', self._copied_bytes())
             logger.info(
                 "[%s] Appended deferred content: strip now %dpx, %dpx ahead",
                 plugin_id, self.scroll_helper.total_scroll_width,
@@ -672,6 +704,35 @@ class RenderPipeline:
         """Whether any canvas-bound plugins are still queued."""
         return bool(self._deferred_queue)
 
+    def prepare_group_member(self, member) -> None:
+        """Join one fetched ``(plugin_id, images)`` ahead of the strip.
+
+        Called by the thread that fetched it (the prefetch thread or the live
+        worker, under the render gate), so the extension that appends it only
+        has to copy its pixels into the strip. Never raises: a member left
+        unprepared is joined at the extension instead, as before.
+        """
+        try:
+            images = member[1]
+            if not images:
+                return
+            blocks = self._prepared_blocks
+            if blocks is None:
+                blocks = self._prepared_blocks = {}
+            if len(blocks) >= self.PREPARED_BLOCKS_MAX:
+                blocks.clear()      # left by groups that were never appended
+            blocks[id(images)] = PreparedBlock(images, self.config)
+        except Exception:  # pylint: disable=broad-except
+            logger.debug("Could not prepare a Vegas block ahead", exc_info=True)
+
+    def _take_prepared_block(self, images: List[Image.Image]) -> Optional[PreparedBlock]:
+        """The block prepared for exactly these images, if there is one."""
+        blocks = self._prepared_blocks
+        if not blocks:
+            return None
+        prepared = blocks.pop(id(images), None)
+        return prepared if prepared is not None and prepared.images is images else None
+
     def _claim_prepared_group(self):
         """Take the prefetched group, if one is ready."""
         with self._prefetch_lock:
@@ -714,6 +775,8 @@ class RenderPipeline:
             for pid, images in grouped:
                 if is_static is not None and is_static(pid):
                     statics.append((sum(1 for _p, imgs in content if imgs), pid))
+                    if images:
+                        self._take_prepared_block(images)   # never appended
                 else:
                     content.append((pid, images))
             grouped = content
@@ -752,10 +815,18 @@ class RenderPipeline:
 
             blocks = []
             layouts = []
+            items = []
             total_rows = 0
             for _plugin_id, images in grouped:
                 total_rows += len(images)
-                block, layout = self._join_plugin_rows_with_layout(images)
+                prepared = self._take_prepared_block(images)
+                if prepared is not None:
+                    block, layout = prepared.block, prepared.layout
+                    items.append(prepared.pixels)
+                else:
+                    # Fetched inline, or by a thread that did not prepare it.
+                    block, layout = self._join_plugin_rows_with_layout(images)
+                    items.append(block)
                 blocks.append(block)
                 layouts.append(layout)
 
@@ -764,13 +835,13 @@ class RenderPipeline:
                 # append_content is about to build a strip from scratch.
                 self._reset_records()
             appended = self.scroll_helper.append_content(
-                content_items=blocks,
+                content_items=items,
                 item_gap=self.config.separator_width,
                 element_gap=0,
             )
             if not appended:
                 return False
-            moved = self._strip_nbytes()
+            moved = self._copied_bytes()
 
             # Where each block starts, laid out as append_content does: a
             # separator before every block, or -- when there was no strip to
@@ -789,9 +860,10 @@ class RenderPipeline:
                 self._static_markers = tuple(
                     (max(0, x - cut), pid) for x, pid in self._static_markers)
             self._forget_trimmed_records(cut)
-            # The append built the whole strip anew, and a trim copies what is
-            # left of it again: both land in the frame after this one.
-            self._note_op('extend', moved + (self._strip_nbytes() if cut else 0))
+            # Both land in the frame after this one: the append's new columns
+            # (the whole strip when its buffer had to be reallocated), and a
+            # trim's copy, if it made one.
+            self._note_op('extend', moved + (self._copied_bytes() if cut else 0))
 
             self._segments_in_scroll = [pid for pid, _ in grouped]
             self.stats['composition_count'] += 1
@@ -1404,6 +1476,7 @@ class RenderPipeline:
             self._prefetch_generation += 1
             self._prepared_group = None
             self._deferred_queue = []
+            self._prepared_blocks = None
         self._static_markers = ()
         self._stop_live_worker()
         self._reset_records()

test/_run_loop_harness.py

@@ -0,0 +1,821 @@
+"""Drive the real DisplayController.run() on a fake clock and record a trace.
+
+The golden trace tests (test_run_loop_golden.py) use this to pin down what
+run() does today -- which mode is on the panel, for how long, and why it
+left -- so that the loop can be restructured (docs/RUN_LOOP_REDESIGN.md)
+without changing any of it.
+
+What is real and what is fake
+-----------------------------
+Real: DisplayController itself (constructed through __init__, then run()),
+PluginExecutor (each screen's first frame still goes through its thread),
+the per-plugin display locks, and every controller method run() calls.
+
+Fake, so the run is deterministic and takes milliseconds:
+
+* the clock -- ``src.display_controller.time`` and ``datetime`` are replaced
+  by one FakeClock; sleeping only advances it. Scripted events (an on-demand
+  request, a WiFi notice, live content starting) fire as it passes them.
+* plugins -- FakePlugin, whose content, liveness and dynamic-duration answers
+  are functions of the fake clock.
+* the plugin manager, cache, config service, display manager and sync
+  manager -- in-memory stand-ins with no threads.
+* the Vegas coordinator -- FakeVegas implements only the contract the
+  controller relies on (run_iteration() returning True when it ran its
+  duration and False when interrupted, the interrupt and live checks it
+  calls back into). The real coordinator spawns threads and renders a strip;
+  driving it on the fake clock is part of stage 4 (Vegas as a Source).
+
+The run ends when the fake clock passes the scenario's horizon: the clock
+raises StopRun, a BaseException, which run()'s ``except Exception`` lets
+through after its ``finally`` has run cleanup().
+
+How the trace is read
+---------------------
+Everything observable is appended to one ordered event log. reduce_trace()
+folds it into screens: a screen starts at the first display() call of a
+loop pass (a "pass" is one call of the watchdog's loop_pass(), at the top of
+run()'s loop), or at the first follower / Vegas / WiFi / blank frame. Its
+exit reason is the first reason-bearing event logged before the next screen
+starts, else ``duration``.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import threading
+from datetime import datetime, timezone
+from pathlib import Path
+from types import SimpleNamespace
+from typing import Any, Callable, Dict, List, Optional, Tuple
+from unittest.mock import MagicMock, patch
+
+from src.common.sync_manager import SyncRole
+from src.plugin_system.plugin_executor import PluginExecutor
+
+GOLDEN_DIR = Path(__file__).parent / "fixtures" / "run_loop_golden"
+
+#: Monday 2026-01-05 22:59:30 UTC. The schedule scenario's windows are set
+#: around 23:00; every other scenario has no schedule, so the date is moot.
+T0 = datetime(2026, 1, 5, 22, 59, 30, tzinfo=timezone.utc).timestamp()
+
+#: Loop passes allowed without the clock moving before the run is called a
+#: spin. run() must sleep somewhere on every few passes.
+SPIN_LIMIT = 500
+
+
+class StopRun(BaseException):
+    """Ends a harness run. A BaseException so run()'s handlers pass it on."""
+
+
+class SpinError(BaseException):
+    """run() went round SPIN_LIMIT times without the clock moving.
+
+    A BaseException for the same reason as StopRun: run() would log and
+    swallow anything less, and the test would see a short trace."""
+
+
+# ---------------------------------------------------------------------------
+# Clock
+# ---------------------------------------------------------------------------
+
+class FakeClock:
+    """time.time/monotonic/perf_counter all read ``now``; sleep() advances it.
+
+    Alarms are (time, callback) pairs fired, in time order, by the sleep that
+    carries the clock past them. Reaching the horizon raises StopRun.
+    """
+
+    def __init__(self, start: float, horizon: float):
+        self.start = start
+        self.now = start
+        self.horizon = start + horizon
+        self._alarms: List[Tuple[float, int, Callable[[], None]]] = []
+        self._seq = 0
+        self.passes_since_advance = 0
+
+    def rel(self) -> float:
+        return self.now - self.start
+
+    def at(self, t: float, callback: Callable[[], None]) -> None:
+        self._alarms.append((self.start + t, self._seq, callback))
+        self._seq += 1
+        self._alarms.sort()
+
+    def time(self) -> float:
+        return self.now
+
+    def sleep(self, seconds: float) -> None:
+        target = self.now + max(0.0, seconds)
+        while self._alarms and self._alarms[0][0] <= target:
+            when, _, callback = self._alarms.pop(0)
+            self.now = max(self.now, when)
+            callback()
+        self.now = target
+        if seconds > 0:
+            self.passes_since_advance = 0
+        if self.now >= self.horizon:
+            raise StopRun()
+
+    def time_module(self) -> SimpleNamespace:
+        return SimpleNamespace(time=self.time, monotonic=self.time,
+                               perf_counter=self.time, sleep=self.sleep)
+
+    def datetime_class(self):
+        clock = self
+
+        class FakeDateTime(datetime):
+            @classmethod
+            def now(cls, tz=None):  # type: ignore[override]
+                return datetime.fromtimestamp(clock.now, tz or timezone.utc)
+
+        return FakeDateTime
+
+
+# ---------------------------------------------------------------------------
+# Fakes
+# ---------------------------------------------------------------------------
+
+class FakeCache:
+    """The in-memory slice of CacheManager that run() and its helpers use."""
+
+    def __init__(self):
+        self.data: Dict[str, Any] = {}
+        self.cache_dir = "/nonexistent/run-loop-harness"
+
+    def get(self, key, max_age=None, memory_ttl=None):
+        return self.data.get(key)
+
+    def set(self, key, data, ttl=None):
+        self.data[key] = data
+
+    def delete(self, key):
+        self.data.pop(key, None)
+
+    def clear_cache(self, key=None):
+        if key is None:
+            self.data.clear()
+        else:
+            self.data.pop(key, None)
+
+    def __getattr__(self, name):
+        # Anything else (stats, cleanup hooks) is a no-op.
+        return lambda *a, **k: None
+
+
+class FakeConfigService:
+    def __init__(self, config):
+        self.config = config
+
+    def get_config(self):
+        return self.config
+
+    def subscribe(self, *a, **k):
+        pass
+
+    def unsubscribe(self, *a, **k):
+        pass
+
+    def shutdown(self):
+        pass
+
+
+class FakeSync:
+    """A standalone sync manager whose follower state follows the script."""
+
+    role = SyncRole.STANDALONE
+
+    def __init__(self, harness: "RunLoopHarness"):
+        self._h = harness
+        self.follower_windows: List[Tuple[float, float]] = []
+
+    def is_follower_active(self) -> bool:
+        t = self._h.clock.rel()
+        return any(a <= t < b for a, b in self.follower_windows)
+
+    def get_latest_scroll_x(self):
+        return None
+
+    def get_latest_frame(self):
+        return "leader-frame"
+
+    def stop(self):
+        pass
+
+    def __getattr__(self, name):
+        return lambda *a, **k: None
+
+
+class FakeHealthTracker:
+    """Circuit breaker stand-in: opens after two consecutive failures and
+    stays open (no wall-clock cooldown, which would not be deterministic)."""
+
+    def __init__(self, harness: "RunLoopHarness"):
+        self._h = harness
+        self.failures: Dict[str, int] = {}
+
+    def should_skip_plugin(self, plugin_id):
+        skip = self.failures.get(plugin_id, 0) >= 2
+        if skip:
+            self._h.log("breaker-open", plugin_id, quiet=True)
+        return skip
+
+    def record_success(self, plugin_id):
+        self.failures[plugin_id] = 0
+
+    def record_failure(self, plugin_id, exc=None):
+        self.failures[plugin_id] = self.failures.get(plugin_id, 0) + 1
+        self._h.log("health-failure", plugin_id)
+
+
+class FakePluginManager:
+    def __init__(self):
+        self.plugins: Dict[str, Any] = {}
+        self.plugin_manifests: Dict[str, Any] = {}
+        self.plugin_last_update: Dict[str, float] = {}
+        self.health_tracker = None
+        self.resource_monitor = None
+        self.state_manager = None
+        self.plugin_executor = PluginExecutor()
+        self.no_lock: set = set()
+        self._locks: Dict[str, threading.Lock] = {}
+        self.hangs: List[str] = []
+
+    def discover_plugins(self):
+        return []
+
+    def discovered_plugin_ids(self):
+        return set(self.plugins)
+
+    def load_plugin(self, plugin_id, force_enabled=False):
+        return False
+
+    def get_plugin(self, plugin_id):
+        return self.plugins.get(plugin_id)
+
+    def unload_plugin(self, plugin_id):
+        self.plugins.pop(plugin_id, None)
+        return True
+
+    def get_plugin_lock(self, plugin_id):
+        if plugin_id in self.no_lock:
+            return None  # as when loading failed part-way
+        return self._locks.setdefault(plugin_id, threading.Lock())
+
+    def record_display_hang(self, plugin_id, seconds):
+        self.hangs.append(plugin_id)
+
+    def note_display_duration(self, plugin_id, seconds):
+        pass
+
+    def run_scheduled_updates(self):
+        pass
+
+    def run_scheduled_updates_with_changes(self):
+        return []
+
+    def stop_update_worker(self):
+        pass
+
+
+class FakePlugin:
+    """A plugin whose answers are functions of the harness clock.
+
+    Args:
+        plugin_id: The plugin id.
+        modes: Its display modes, registered in this order.
+        duration: get_display_duration().
+        content: ``content(t, mode) -> bool``: what display() returns.
+            Defaults to always True.
+        live: ``(start, end)`` seconds during which has_live_content() is
+            True; get_live_modes() then names its modes ending in ``_live``.
+        live_priority: has_live_priority().
+        dynamic: Enables dynamic duration. Keys: ``cap`` (the plugin's cap),
+            ``cycle`` (get_cycle_duration()), ``complete_after`` (seconds
+            after reset_cycle_state() that is_cycle_complete() turns True;
+            None means never).
+        needs_high_fps / enable_scrolling: Set as attributes only when given,
+            since run() tests for their presence.
+        raises: display() raises RuntimeError.
+        first_frame_only: display() returns True on a screen's first frame
+            and False on every later one.
+    """
+
+    def __init__(self, plugin_id: str, modes: List[str], duration: float = 30,
+                 content: Optional[Callable[[float, str], bool]] = None,
+                 live: Optional[Tuple[float, float]] = None,
+                 live_priority: bool = False,
+                 dynamic: Optional[Dict[str, Any]] = None,
+                 needs_high_fps: Optional[bool] = None,
+                 enable_scrolling: Optional[bool] = None,
+                 raises: bool = False,
+                 first_frame_only: bool = False):
+        self.plugin_id = plugin_id
+        self.modes = list(modes)
+        self.duration = duration
+        self.content = content
+        self.live = live
+        self.live_priority = live_priority
+        self.dynamic = dynamic
+        self.raises = raises
+        self.first_frame_only = first_frame_only
+        if needs_high_fps is not None:
+            self.needs_high_fps = needs_high_fps
+        if enable_scrolling is not None:
+            self.enable_scrolling = enable_scrolling
+        self._h: Optional["RunLoopHarness"] = None
+        self._reset_at: Optional[float] = None
+
+    # -- display -----------------------------------------------------------
+    def display(self, display_mode=None, force_clear=False):
+        assert self._h is not None
+        return self._h.on_display(self, display_mode or self.modes[0], force_clear)
+
+    def get_display_duration(self):
+        return self.duration
+
+    # -- live --------------------------------------------------------------
+    def _is_live(self) -> bool:
+        if not self.live or self._h is None:
+            return False
+        t = self._h.clock.rel()
+        return self.live[0] <= t < self.live[1]
+
+    def has_live_priority(self):
+        return self.live_priority
+
+    def has_live_content(self):
+        return self._is_live()
+
+    def get_live_modes(self):
+        return [m for m in self.modes if m.endswith("_live")]
+
+    # -- dynamic duration ----------------------------------------------------
+    def supports_dynamic_duration(self):
+        return bool(self.dynamic)
+
+    def get_dynamic_duration_cap(self):
+        return (self.dynamic or {}).get("cap")
+
+    def get_cycle_duration(self, display_mode=None):
+        return (self.dynamic or {}).get("cycle")
+
+    def reset_cycle_state(self):
+        assert self._h is not None
+        self._reset_at = self._h.clock.rel()
+        self._h.log("cycle-reset", self.plugin_id)
+
+    def is_cycle_complete(self):
+        if not self.dynamic:
+            return True
+        after = self.dynamic.get("complete_after")
+        if after is None or self._reset_at is None or self._h is None:
+            return False
+        done = self._h.clock.rel() - self._reset_at >= after
+        if done:
+            self._h.log("cycle-complete", self.plugin_id, quiet=True)
+        return done
+
+
+class LegacyFakePlugin(FakePlugin):
+    """display() without a display_mode parameter, as older plugins have."""
+
+    def display(self, force_clear=False):  # type: ignore[override]
+        assert self._h is not None
+        return self._h.on_display(self, self.modes[0], force_clear)
+
+
+class FakeVegas:
+    """The coordinator contract DisplayController relies on, nothing more.
+
+    run_iteration() renders frames at 125 Hz on the fake clock for
+    ``cycle`` seconds and returns True, or returns False as soon as the
+    interrupt checker (every 10 frames) or the live-priority checker (every
+    0.25 s) asks it to yield -- the same cadence the real coordinator uses.
+    A live-priority pause is lifted by the next call, as in the real one.
+    """
+
+    FRAME = 1.0 / 125
+    INTERRUPT_EVERY = 10
+    LIVE_EVERY = 0.25
+
+    def __init__(self, harness: "RunLoopHarness", cycle: float = 30.0,
+                 live_in_ticker: bool = False):
+        self._h = harness
+        self.cycle = cycle
+        self.is_enabled = True
+        self.vegas_config = SimpleNamespace(live_in_ticker=live_in_ticker)
+        self.render_pipeline = None
+        self._interrupt: Optional[Callable[[], bool]] = None
+        self._live: Optional[Callable[[], Any]] = None
+        self._paused_for_live = False
+
+    def set_live_priority_checker(self, fn):
+        self._live = fn
+
+    def set_interrupt_checker(self, fn, check_interval=10):
+        self._interrupt = fn
+
+    def apply_pending_config_if_idle(self):
+        pass
+
+    def cleanup(self):
+        pass
+
+    def run_iteration(self) -> bool:
+        h = self._h
+        clock = h.clock
+        if self._paused_for_live:
+            self._paused_for_live = False
+        h.log("vegas-start", None, quiet=True)
+        start = clock.now
+        last_live = None
+        frames = 0
+        while True:
+            now = clock.now
+            if (self._live and not self.vegas_config.live_in_ticker
+                    and (last_live is None or now - last_live >= self.LIVE_EVERY)):
+                last_live = now
+                if self._live():
+                    self._paused_for_live = True
+                    h.log("vegas-live")
+                    return False
+            h.log("vegas-frame", None, quiet=True)
+            clock.sleep(self.FRAME)
+            frames += 1
+            if self._interrupt and frames % self.INTERRUPT_EVERY == 0 and self._interrupt():
+                h.log("vegas-interrupt")
+                return False
+            if clock.now - start >= self.cycle:
+                return True
+
+
+# ---------------------------------------------------------------------------
+# Harness
+# ---------------------------------------------------------------------------
+
+#: Events that can end a screen, as they appear in the trace.
+REASON_EVENTS = {
+    "schedule-off", "schedule-on", "live", "live-ended", "on-demand-start",
+    "on-demand-requested-stop", "on-demand-expired",
+    "on-demand-no-modes-available", "vegas-live", "vegas-interrupt",
+    "cycle-complete", "display-false",
+}
+
+_SEGMENT_FOR = {
+    "follower-frame": "<follower>",
+    "vegas-frame": "<vegas>",
+    "wifi": "<wifi>",
+    "blank": "<off>",
+}
+
+
+class RunLoopHarness:
+    """Build a DisplayController on fakes, run it, and return its trace."""
+
+    def __init__(self, tmp_path: Path, horizon: float):
+        self.clock = FakeClock(T0, horizon)
+        self.events: List[Tuple[float, str, Any, Dict[str, Any]]] = []
+        self.tmp_path = tmp_path
+        # The controller keeps this very dict as self.config, so a scenario
+        # can edit what run() reads live (durations, schedules). Values
+        # __init__ copies out (global_dynamic_config) are set on the
+        # controller instead.
+        self.config: Dict[str, Any] = {
+            "timezone": "UTC",
+            "display": {"hardware": {"brightness": 90}},
+        }
+        self.cache = FakeCache()
+        self.pm = FakePluginManager()
+        self.sync = FakeSync(self)
+        self.dm = self._display_manager()
+        self._displayed_this_pass = False
+        self.controller = self._build()
+
+    # -- event log -----------------------------------------------------------
+    def log(self, kind: str, subject: Any = None, quiet: bool = False, **data):
+        data["quiet"] = quiet
+        self.events.append((round(self.clock.rel(), 3), kind, subject, data))
+
+    def on_display(self, plugin: FakePlugin, mode: str, force_clear: bool):
+        first = not self._displayed_this_pass
+        self._displayed_this_pass = True
+        if plugin.raises:
+            self.log("first" if first else "frame", mode, quiet=True,
+                     clear=bool(force_clear), result="raised")
+            raise RuntimeError(f"{plugin.plugin_id} display() failed")
+        if plugin.first_frame_only:
+            result = first
+        elif plugin.content is None:
+            result = True
+        else:
+            result = bool(plugin.content(self.clock.rel(), mode))
+        self.log("first" if first else "frame", mode, quiet=True,
+                 clear=bool(force_clear), result=result)
+        return result
+
+    # -- construction ----------------------------------------------------------
+    def _display_manager(self):
+        dm = MagicMock(name="DisplayManager")
+        dm.width = 128
+        dm.height = 32
+        dm._sync_render_allowed = False
+        dm.set_brightness = MagicMock(side_effect=self._on_set_brightness)
+        dm.update_display = MagicMock(side_effect=self._on_update_display)
+        dm.get_font_height = MagicMock(return_value=8)
+        return dm
+
+    def _on_set_brightness(self, value):
+        self.log("brightness", value)
+        return True
+
+    def _on_update_display(self):
+        if getattr(self.dm, "_sync_render_allowed", False):
+            self.log("follower-frame", None, quiet=True)
+        elif not self.controller.is_display_active:
+            self.log("blank", None, quiet=True)
+
+    def _build(self):
+        from src import display_controller as dc_mod
+
+        clock = self.clock
+        env = {"LEDMATRIX_HOT_RELOAD": "false", "EMULATOR": "true"}
+        with patch.dict(os.environ, env), \
+                patch.object(dc_mod, "time", clock.time_module()), \
+                patch.object(dc_mod, "datetime", clock.datetime_class()), \
+                patch.object(dc_mod, "ConfigManager", MagicMock()), \
+                patch.object(dc_mod, "ConfigService", lambda **kw: FakeConfigService(self.config)), \
+                patch.object(dc_mod, "CacheManager", lambda: self.cache), \
+                patch.object(dc_mod, "DisplayManager", lambda config: self.dm), \
+                patch.object(dc_mod, "FontManager", MagicMock()), \
+                patch.object(dc_mod, "DisplaySyncManager", lambda **kw: self.sync), \
+                patch("src.plugin_system.PluginManager", lambda **kw: self.pm), \
+                patch("src.error_aggregator.start_error_snapshot_publisher", lambda cm: None), \
+                patch("src.font_usage.start_font_usage_publisher", lambda *a, **k: None), \
+                patch("src.plugin_system.plugin_runtime.start_plugin_runtime_publisher",
+                      lambda *a, **k: None), \
+                patch("src.auto_update_setup.ensure_update_helper", lambda config: None):
+            controller = dc_mod.DisplayController()
+
+        # __init__ wires real health/resource monitors; swap in the fake
+        # breaker so failures and skips are deterministic.
+        self.pm.health_tracker = FakeHealthTracker(self)
+        self.pm.resource_monitor = None
+        controller.wifi_status_file = self.tmp_path / "wifi_status.json"
+        self._instrument(controller)
+        return controller
+
+    def _instrument(self, dc) -> None:
+        """Log the controller's decisions without changing any of them.
+
+        Each wrapper calls straight through to the real method; only methods
+        that exist both before and after the stage-1 extraction are wrapped,
+        so the same harness records the same trace from either.
+        """
+        h = self
+
+        def wrap(name, before, after):
+            real = getattr(dc, name)
+
+            def wrapper(*args, **kwargs):
+                token = before(*args, **kwargs)
+                result = real(*args, **kwargs)
+                after(token, *args, **kwargs)
+                return result
+            setattr(dc, name, wrapper)
+
+        wrap("_evaluate_schedule",
+             lambda: dc.is_display_active,
+             lambda was: (h.log("schedule-off") if was and not dc.is_display_active
+                          else h.log("schedule-on") if not was and dc.is_display_active
+                          else None))
+        wrap("_activate_on_demand",
+             lambda request: None,
+             lambda _, request: h.log("on-demand-start", request.get("plugin_id"))
+             if dc.on_demand_active else h.log("on-demand-error", dc.on_demand_last_error))
+        wrap("_clear_on_demand",
+             lambda reason=None: dc.on_demand_active,
+             lambda was, reason=None: h.log(f"on-demand-{reason}") if was else None)
+        wrap("_apply_live_priority",
+             lambda mode: dc.current_display_mode,
+             lambda prev, mode: (None if dc.current_display_mode == prev
+                                 else h.log("live" if mode else "live-ended",
+                                            dc.current_display_mode)))
+        real_note = dc._note_empty_pass
+
+        def note_empty_pass():
+            h.log("empty", dc.current_display_mode, quiet=True)
+            return real_note()
+        dc._note_empty_pass = note_empty_pass
+
+        real_wifi = dc._display_wifi_status_message
+
+        def display_wifi(status):
+            shown = real_wifi(status)
+            if shown:
+                h.log("wifi", status.get("message"), quiet=True)
+            return shown
+        dc._display_wifi_status_message = display_wifi
+
+    # -- scenario setup ------------------------------------------------------
+    def add_plugin(self, plugin: FakePlugin, lock: bool = True) -> FakePlugin:
+        """Register a plugin the way _register_loaded_plugin leaves things."""
+        dc = self.controller
+        plugin._h = self
+        self.pm.plugins[plugin.plugin_id] = plugin
+        if not lock:
+            self.pm.no_lock.add(plugin.plugin_id)
+        dc.plugin_display_modes[plugin.plugin_id] = list(plugin.modes)
+        for mode in plugin.modes:
+            if mode not in dc.available_modes:
+                dc.available_modes.append(mode)
+            dc.plugin_modes[mode] = plugin
+            dc.mode_to_plugin_id[mode] = plugin.plugin_id
+        return plugin
+
+    def add_mode_without_plugin(self, mode: str) -> None:
+        self.controller.available_modes.append(mode)
+
+    def on_demand_request(self, t: float, request_id: str, action: str = "start", **fields):
+        def post():
+            self.log("request", f"{action}:{request_id}")
+            self.cache.set("display_on_demand_request",
+                           {"request_id": request_id, "action": action, **fields})
+        self.clock.at(t, post)
+
+    def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
+                          duration: Optional[float] = None, pinned: bool = False):
+        """Start with an on-demand session resumed from the cache, as after
+        a restart: the state _select_startup_plugins restores, then
+        _populate_on_demand_modes_from_plugin, as __init__ calls it."""
+        dc = self.controller
+        dc.on_demand_active = True
+        dc.on_demand_plugin_id = plugin_id
+        dc.on_demand_mode = mode
+        dc.on_demand_duration = duration
+        dc.on_demand_pinned = pinned
+        dc.on_demand_requested_at = self.clock.now
+        dc.on_demand_expires_at = self.clock.now + duration if duration else None
+        dc.on_demand_status = 'active'
+        dc.on_demand_schedule_override = True
+        dc._populate_on_demand_modes_from_plugin()
+
+    def wifi_message(self, t: float, message: str, duration: float = 5):
+        def write():
+            self.log("wifi-file", message)
+            self.controller.wifi_status_file.write_text(json.dumps(
+                {"message": message, "timestamp": self.clock.now, "duration": duration}),
+                encoding="utf-8")
+        self.clock.at(t, write)
+
+    def enable_vegas(self, cycle: float = 30.0, live_in_ticker: bool = False) -> FakeVegas:
+        """Install FakeVegas, wired up as _initialize_vegas_mode wires the real one."""
+        dc = self.controller
+        vegas = FakeVegas(self, cycle=cycle, live_in_ticker=live_in_ticker)
+        vegas.set_live_priority_checker(dc._check_live_priority)
+        vegas.set_interrupt_checker(
+            lambda: dc._check_vegas_interrupt() or dc.sync_manager.is_follower_active(),
+            check_interval=10)
+        dc.vegas_coordinator = vegas
+        return vegas
+
+    # -- running -------------------------------------------------------------
+    def run(self) -> Dict[str, Any]:
+        from src import display_controller as dc_mod
+        from src import display_watchdog
+
+        clock = self.clock
+        watchdog = display_watchdog.watchdog
+        real_loop_pass = watchdog.loop_pass
+
+        def loop_pass():
+            self._displayed_this_pass = False
+            self.log("pass", None, quiet=True)
+            clock.passes_since_advance += 1
+            if clock.passes_since_advance > SPIN_LIMIT:
+                raise SpinError(f"run() spun {SPIN_LIMIT} passes at t={clock.rel():.3f}")
+            return real_loop_pass()
+
+        with patch.object(dc_mod, "time", clock.time_module()), \
+                patch.object(dc_mod, "datetime", clock.datetime_class()), \
+                patch.object(watchdog, "loop_pass", loop_pass):
+            try:
+                self.controller.run()
+            except StopRun:
+                pass
+            else:
+                # run() only returns after catching something itself.
+                raise AssertionError(
+                    f"run() returned at t={clock.rel():.3f} before the horizon")
+        return reduce_trace(self.events, round(clock.horizon - clock.start, 3))
+
+
+def reduce_trace(events, horizon: float) -> Dict[str, Any]:
+    """Fold the event log into screens and the notable events."""
+    screens: List[Dict[str, Any]] = []
+    notable: List[List[Any]] = []
+    cur: Optional[Dict[str, Any]] = None
+    # What happened in the current loop pass, for attributing an empty pass.
+    shown_this_pass = False
+    failed_this_pass = False
+    breaker_this_pass = False
+
+    def start(t, mode, clear=None):
+        nonlocal cur
+        cur = {"t": t, "mode": mode, "frames": 0, "clear": clear, "exit": None}
+        screens.append(cur)
+
+    for t, kind, subject, data in events:
+        if not data.get("quiet"):
+            notable.append([t, kind] + ([subject] if subject is not None else []))
+        if kind == "pass":
+            shown_this_pass = failed_this_pass = breaker_this_pass = False
+        elif kind in ("first", "frame"):
+            if kind == "first" or cur is None:
+                start(t, subject, data["clear"])
+                shown_this_pass = True
+                cur["result"] = data["result"]
+            cur["frames"] += 1
+            if kind == "frame" and data["result"] is False and cur["exit"] is None:
+                cur["exit"] = "display-false"
+        elif kind in _SEGMENT_FOR:
+            segment = _SEGMENT_FOR[kind]
+            if cur is None or cur["mode"] != segment or cur["exit"] is not None:
+                start(t, segment)
+            cur["frames"] += 1
+        elif kind == "vegas-start":
+            start(t, "<vegas>")
+        elif kind == "health-failure":
+            failed_this_pass = True
+        elif kind == "breaker-open":
+            breaker_this_pass = True
+        elif kind == "empty":
+            if shown_this_pass and cur is not None and cur["exit"] is None:
+                # display() ran and had nothing (False) or raised.
+                cur["exit"] = "raised" if cur.get("result") == "raised" else "empty"
+            else:
+                # Never reached display(): no plugin, the breaker is open, or
+                # the dispatch itself raised.
+                start(t, subject)
+                cur["exit"] = ("error" if failed_this_pass
+                               else "breaker" if breaker_this_pass else "no-plugin")
+        elif kind in REASON_EVENTS and cur is not None and cur["exit"] is None:
+            cur["exit"] = kind
+
+    rows = []
+    for i, screen in enumerate(screens):
+        end = screens[i + 1]["t"] if i + 1 < len(screens) else horizon
+        nxt = screens[i + 1] if i + 1 < len(screens) else None
+        # A WiFi notice logs no event at the moment it takes the panel (the
+        # file is written earlier), so a screen followed by one is labelled
+        # "wifi". Its duration column shows whether it was cut short.
+        exit_reason = screen["exit"] or (
+            "horizon" if nxt is None
+            else "wifi" if nxt["mode"] == "<wifi>" and screen["mode"] != "<wifi>"
+            else "duration")
+        rows.append([screen["t"], screen["mode"], round(end - screen["t"], 3),
+                     exit_reason, screen["frames"], screen["clear"]])
+    return {"screens": rows, "events": notable}
+
+
+# ---------------------------------------------------------------------------
+# Golden files
+# ---------------------------------------------------------------------------
+
+def dump_golden(trace: Dict[str, Any]) -> str:
+    """One screen or event per line, so a diff points at the row that moved."""
+    def block(name, rows, last=False):
+        end = "" if last else ","
+        if not rows:
+            return [f'  "{name}": []{end}']
+        return [f'  "{name}": [',
+                ",\n".join("    " + json.dumps(row) for row in rows),
+                f"  ]{end}"]
+
+    lines = (["{"] + block("screens", trace["screens"])
+             + block("events", trace["events"], last=True) + ["}"])
+    return "\n".join(lines) + "\n"
+
+
+def check_golden(name: str, trace: Dict[str, Any]) -> None:
+    """Compare against test/fixtures/run_loop_golden/<name>.json.
+
+    LEDMATRIX_REGEN_GOLDEN=1 rewrites the file instead. Only do that for a
+    deliberate behaviour change, and say why in the commit.
+    """
+    path = GOLDEN_DIR / f"{name}.json"
+    text = dump_golden(trace)
+    if os.environ.get("LEDMATRIX_REGEN_GOLDEN") == "1":
+        GOLDEN_DIR.mkdir(parents=True, exist_ok=True)
+        path.write_text(text, encoding="utf-8", newline="\n")
+        return
+    assert path.exists(), f"no golden trace {path}; run with LEDMATRIX_REGEN_GOLDEN=1"
+    expected = json.loads(path.read_text(encoding="utf-8"))
+    actual = json.loads(text)
+    if actual != expected:
+        import difflib
+        diff = "\n".join(difflib.unified_diff(
+            dump_golden(expected).splitlines(), text.splitlines(),
+            "golden", "actual", lineterm="", n=2))
+        raise AssertionError(f"run() trace for {name!r} changed:\n{diff}")

test/conftest.py

@@ -315,6 +315,19 @@ def _hermetic_display_watchdog(monkeypatch):
                         display_watchdog.RenderWatchdog(environ={}, heartbeat_dir=None))
 
 
+@pytest.fixture(autouse=True)
+def _hermetic_control_socket(monkeypatch):
+    """Keep the control socket (src/ipc) off the host.
+
+    DisplayController.run() would serve /run/ledmatrix/control.sock -- or
+    find the live display's already there, when the suite runs on a device
+    -- and the web routes would send on-demand commands to that display.
+    Off by default; the socket tests point it at a tmp_path of their own.
+    """
+    from src.ipc.contract import SOCKET_PATH_ENV
+    monkeypatch.setenv(SOCKET_PATH_ENV, 'off')
+
+
 @pytest.fixture(autouse=True)
 def reset_logging():
     """Reset logging configuration before each test."""

test/fixtures/api_v3_url_map.json

@@ -192,6 +192,15 @@
       "POST"
     ]
   ],
+  [
+    "/api/v3/config/scroll-speed-advice",
+    "api_v3.get_scroll_speed_advice",
+    [
+      "GET",
+      "HEAD",
+      "OPTIONS"
+    ]
+  ],
   [
     "/api/v3/config/secrets",
     "api_v3.get_secrets_config",
@@ -458,6 +467,15 @@
       "POST"
     ]
   ],
+  [
+    "/api/v3/plugins/fetch-stats",
+    "api_v3.get_fetch_stats",
+    [
+      "GET",
+      "HEAD",
+      "OPTIONS"
+    ]
+  ],
   [
     "/api/v3/plugins/health",
     "api_v3.get_plugin_health",

test/fixtures/run_loop_golden/all_empty.json

@@ -0,0 +1,14 @@
+{
+  "screens": [
+    [0.0, "a", 0.0, "empty", 1, false],
+    [0.0, "b", 0.0, "empty", 1, true],
+    [0.0, "c", 1.0, "empty", 1, true],
+    [1.0, "a", 1.0, "empty", 1, true],
+    [2.0, "b", 1.0, "empty", 1, true],
+    [3.0, "c", 1.0, "empty", 1, true],
+    [4.0, "a", 1.0, "empty", 1, true],
+    [5.0, "b", 1.0, "empty", 1, true],
+    [6.0, "c", 6.0, "horizon", 6, true]
+  ],
+  "events": []
+}

test/fixtures/run_loop_golden/dynamic_duration.json

@@ -0,0 +1,24 @@
+{
+  "screens": [
+    [0.0, "scroller", 20.008, "cycle-complete", 2502, false],
+    [20.008, "news", 40.0, "duration", 40, true],
+    [60.008, "board", 11.0, "cycle-complete", 12, true],
+    [71.008, "clock", 10.0, "duration", 10, true],
+    [81.008, "scroller", 20.007, "cycle-complete", 2502, true],
+    [101.015, "news", 40.0, "duration", 40, true],
+    [141.015, "board", 11.0, "cycle-complete", 12, true],
+    [152.015, "clock", 10.0, "duration", 10, true],
+    [162.015, "scroller", 20.008, "cycle-complete", 2502, true],
+    [182.023, "news", 37.977, "horizon", 38, true]
+  ],
+  "events": [
+    [0.0, "cycle-reset", "scroller"],
+    [20.008, "cycle-reset", "news"],
+    [60.008, "cycle-reset", "board"],
+    [81.008, "cycle-reset", "scroller"],
+    [101.015, "cycle-reset", "news"],
+    [141.015, "cycle-reset", "board"],
+    [162.015, "cycle-reset", "scroller"],
+    [182.023, "cycle-reset", "news"]
+  ]
+}

test/fixtures/run_loop_golden/empty_modes.json

@@ -0,0 +1,22 @@
+{
+  "screens": [
+    [0.0, "clock", 10.0, "duration", 10, false],
+    [10.0, "empty", 0.0, "empty", 1, true],
+    [10.0, "ghost", 0.0, "no-plugin", 0, null],
+    [10.0, "flaky", 12.0, "display-false", 2, true],
+    [22.0, "clock", 10.0, "duration", 10, true],
+    [32.0, "empty", 0.0, "empty", 1, true],
+    [32.0, "ghost", 0.0, "no-plugin", 0, null],
+    [32.0, "flaky", 12.0, "display-false", 2, true],
+    [44.0, "clock", 10.0, "duration", 10, true],
+    [54.0, "empty", 0.0, "empty", 1, true],
+    [54.0, "ghost", 0.0, "no-plugin", 0, null],
+    [54.0, "flaky", 12.0, "display-false", 2, true],
+    [66.0, "clock", 10.0, "duration", 10, true],
+    [76.0, "empty", 0.0, "empty", 1, true],
+    [76.0, "ghost", 0.0, "no-plugin", 0, null],
+    [76.0, "flaky", 12.0, "display-false", 2, true],
+    [88.0, "clock", 2.0, "horizon", 2, true]
+  ],
+  "events": []
+}

test/fixtures/run_loop_golden/follower.json

@@ -0,0 +1,10 @@
+{
+  "screens": [
+    [0.0, "clock", 20.0, "duration", 20, false],
+    [20.0, "weather", 20.0, "duration", 20, true],
+    [40.0, "<follower>", 10.017, "duration", 601, null],
+    [50.017, "clock", 20.0, "duration", 20, true],
+    [70.017, "weather", 9.983, "horizon", 10, true]
+  ],
+  "events": []
+}

test/fixtures/run_loop_golden/live_priority.json

@@ -0,0 +1,21 @@
+{
+  "screens": [
+    [0.0, "clock", 20.0, "duration", 20, false],
+    [20.0, "weather", 20.0, "duration", 20, true],
+    [40.0, "sports_recent", 10.0, "live", 11, true],
+    [50.0, "sports_live", 20.0, "duration", 20, true],
+    [70.0, "sports_live", 20.0, "duration", 20, false],
+    [90.0, "sports_live", 20.0, "live-ended", 20, false],
+    [110.0, "sports_recent", 20.0, "duration", 20, true],
+    [130.0, "sports_live", 0.0, "empty", 1, true],
+    [130.0, "clock", 20.0, "duration", 20, true],
+    [150.0, "weather", 20.0, "duration", 20, true],
+    [170.0, "sports_recent", 20.0, "duration", 20, true],
+    [190.0, "sports_live", 0.0, "empty", 1, true],
+    [190.0, "clock", 10.0, "horizon", 10, true]
+  ],
+  "events": [
+    [50.0, "live", "sports_live"],
+    [110.0, "live-ended", "sports_recent"]
+  ]
+}

test/fixtures/run_loop_golden/live_round_robin.json

@@ -0,0 +1,20 @@
+{
+  "screens": [
+    [0.0, "nfl_live", 15.0, "duration", 15, true],
+    [15.0, "nfl_live", 15.0, "live", 15, false],
+    [30.0, "nhl_live", 15.0, "live", 15, true],
+    [45.0, "nfl_live", 15.0, "live", 15, true],
+    [60.0, "nhl_live", 15.0, "duration", 15, true],
+    [75.0, "nhl_live", 15.0, "duration", 15, false],
+    [90.0, "nhl_live", 15.0, "duration", 15, false],
+    [105.0, "clock", 15.0, "duration", 15, true],
+    [120.0, "nfl_live", 15.0, "duration", 15, true],
+    [135.0, "nhl_live", 15.0, "horizon", 15, true]
+  ],
+  "events": [
+    [0.0, "live", "nfl_live"],
+    [30.0, "live", "nhl_live"],
+    [45.0, "live", "nfl_live"],
+    [60.0, "live", "nhl_live"]
+  ]
+}

test/fixtures/run_loop_golden/on_demand.json

@@ -0,0 +1,30 @@
+{
+  "screens": [
+    [0.0, "clock", 20.0, "duration", 20, false],
+    [20.0, "weather", 5.0, "on-demand-start", 6, true],
+    [25.0, "sports_recent", 15.0, "duration", 15, true],
+    [40.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [55.0, "sports_recent", 15.0, "duration", 15, true],
+    [70.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [85.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
+    [95.0, "weather", 20.0, "duration", 20, true],
+    [115.0, "sports_recent", 15.0, "duration", 15, true],
+    [130.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [145.0, "clock", 5.0, "on-demand-start", 6, true],
+    [150.0, "weather", 20.0, "duration", 20, true],
+    [170.0, "weather", 10.0, "on-demand-expired", 10, true],
+    [180.0, "clock", 20.0, "duration", 20, true],
+    [200.0, "weather", 20.0, "duration", 20, true],
+    [220.0, "sports_recent", 15.0, "duration", 15, true],
+    [235.0, "sports_upcoming", 5.0, "horizon", 5, true]
+  ],
+  "events": [
+    [25.0, "request", "start:r1"],
+    [25.0, "on-demand-start", "sports"],
+    [95.0, "request", "stop:r2"],
+    [95.0, "on-demand-requested-stop"],
+    [150.0, "request", "start:r3"],
+    [150.0, "on-demand-start", "weather"],
+    [180.0, "on-demand-expired"]
+  ]
+}

test/fixtures/run_loop_golden/on_demand_pinned.json

@@ -0,0 +1,29 @@
+{
+  "screens": [
+    [0.0, "clock", 12.0, "on-demand-start", 13, false],
+    [12.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [27.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [42.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [57.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [72.0, "sports_upcoming", 8.0, "on-demand-start", 9, true],
+    [80.0, "app_a", 0.0, "empty", 1, true],
+    [80.0, "app_b", 10.0, "duration", 10, true],
+    [90.0, "app_a", 0.0, "empty", 1, true],
+    [90.0, "app_b", 10.0, "duration", 10, true],
+    [100.0, "app_a", 0.0, "empty", 1, true],
+    [100.0, "app_b", 10.0, "duration", 10, true],
+    [110.0, "app_a", 0.0, "empty", 1, true],
+    [110.0, "app_b", 10.0, "on-demand-requested-stop", 10, true],
+    [120.0, "clock", 20.0, "duration", 20, true],
+    [140.0, "sports_recent", 15.0, "duration", 15, true],
+    [155.0, "sports_upcoming", 5.0, "horizon", 5, true]
+  ],
+  "events": [
+    [12.0, "request", "start:p1"],
+    [12.0, "on-demand-start", "sports"],
+    [80.0, "request", "start:p2"],
+    [80.0, "on-demand-start", "starlark"],
+    [120.0, "request", "stop:p3"],
+    [120.0, "on-demand-requested-stop"]
+  ]
+}

test/fixtures/run_loop_golden/on_demand_restored.json

@@ -0,0 +1,14 @@
+{
+  "screens": [
+    [0.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [15.0, "sports_recent", 15.0, "duration", 15, true],
+    [30.0, "sports_upcoming", 10.0, "on-demand-expired", 10, true],
+    [40.0, "clock", 20.0, "duration", 20, true],
+    [60.0, "weather", 20.0, "duration", 20, true],
+    [80.0, "sports_recent", 15.0, "duration", 15, true],
+    [95.0, "sports_upcoming", 5.0, "horizon", 5, true]
+  ],
+  "events": [
+    [40.0, "on-demand-expired"]
+  ]
+}

test/fixtures/run_loop_golden/plain_rotation.json

@@ -0,0 +1,17 @@
+{
+  "screens": [
+    [0.0, "clock", 15.0, "duration", 15, false],
+    [15.0, "weather_now", 20.0, "duration", 20, true],
+    [35.0, "weather_forecast", 20.0, "duration", 20, true],
+    [55.0, "ticker", 10.008, "duration", 1252, true],
+    [65.008, "legacy", 5.0, "duration", 5, true],
+    [70.008, "clock", 15.0, "duration", 15, true],
+    [85.008, "weather_now", 20.0, "duration", 20, true],
+    [105.008, "weather_forecast", 20.0, "duration", 20, true],
+    [125.008, "ticker", 10.008, "duration", 1252, true],
+    [135.016, "legacy", 5.0, "duration", 5, true],
+    [140.016, "clock", 15.0, "duration", 15, true],
+    [155.016, "weather_now", 4.984, "horizon", 5, true]
+  ],
+  "events": []
+}

test/fixtures/run_loop_golden/plugin_error.json

@@ -0,0 +1,29 @@
+{
+  "screens": [
+    [0.0, "clock", 10.0, "duration", 10, false],
+    [10.0, "broken_a", 0.0, "error", 0, null],
+    [10.0, "weather", 10.0, "duration", 10, true],
+    [20.0, "crashy", 0.0, "raised", 1, true],
+    [20.0, "clock", 10.0, "duration", 10, true],
+    [30.0, "broken_a", 0.0, "error", 0, null],
+    [30.0, "weather", 10.0, "duration", 10, true],
+    [40.0, "crashy", 0.0, "raised", 1, true],
+    [40.0, "clock", 10.0, "duration", 10, true],
+    [50.0, "broken_a", 0.0, "breaker", 0, null],
+    [50.0, "broken_b", 0.0, "breaker", 0, null],
+    [50.0, "weather", 10.0, "duration", 10, true],
+    [60.0, "crashy", 0.0, "breaker", 0, null],
+    [60.0, "clock", 10.0, "duration", 10, true],
+    [70.0, "broken_a", 0.0, "breaker", 0, null],
+    [70.0, "broken_b", 0.0, "breaker", 0, null],
+    [70.0, "weather", 10.0, "duration", 10, true],
+    [80.0, "crashy", 0.0, "breaker", 0, null],
+    [80.0, "clock", 10.0, "horizon", 10, true]
+  ],
+  "events": [
+    [10.0, "health-failure", "broken"],
+    [20.0, "health-failure", "crashy"],
+    [30.0, "health-failure", "broken"],
+    [40.0, "health-failure", "crashy"]
+  ]
+}

test/fixtures/run_loop_golden/schedule.json

@@ -0,0 +1,27 @@
+{
+  "screens": [
+    [0.0, "clock", 20.0, "duration", 20, false],
+    [20.0, "weather", 20.0, "duration", 20, true],
+    [40.0, "clock", 20.0, "duration", 20, true],
+    [60.0, "weather", 20.0, "duration", 20, true],
+    [80.0, "clock", 10.0, "schedule-off", 11, true],
+    [90.0, "<off>", 80.0, "on-demand-start", 3, null],
+    [170.0, "weather", 20.0, "on-demand-expired", 20, true],
+    [190.0, "<off>", 140.0, "schedule-on", 3, null],
+    [330.0, "clock", 20.0, "duration", 20, true],
+    [350.0, "weather", 20.0, "duration", 20, true],
+    [370.0, "clock", 20.0, "duration", 20, true],
+    [390.0, "weather", 10.0, "horizon", 10, true]
+  ],
+  "events": [
+    [30.0, "brightness", 30],
+    [90.0, "schedule-off"],
+    [170.0, "request", "start:s1"],
+    [170.0, "on-demand-start", "weather"],
+    [170.0, "schedule-on"],
+    [170.0, "brightness", 90],
+    [190.0, "on-demand-expired"],
+    [190.0, "schedule-off"],
+    [330.0, "schedule-on"]
+  ]
+}

test/fixtures/run_loop_golden/vegas.json

@@ -0,0 +1,27 @@
+{
+  "screens": [
+    [0.0, "<vegas>", 30.008, "duration", 3751, null],
+    [30.008, "<vegas>", 30.007, "duration", 3751, null],
+    [60.015, "<vegas>", 10.24, "vegas-live", 1280, null],
+    [70.255, "sports_live", 20.0, "duration", 20, true],
+    [90.255, "sports_live", 20.0, "display-false", 11, false],
+    [110.255, "<vegas>", 30.008, "duration", 3751, null],
+    [140.263, "<vegas>", 10.0, "on-demand-start", 1250, null],
+    [150.263, "clock", 20.0, "duration", 20, true],
+    [170.263, "clock", 5.0, "on-demand-expired", 5, true],
+    [175.263, "<vegas>", 24.959, "vegas-interrupt", 3120, null],
+    [200.222, "<wifi>", 3.0, "duration", 6, null],
+    [203.222, "<vegas>", 30.008, "duration", 3751, null],
+    [233.23, "<vegas>", 26.77, "horizon", 3347, null]
+  ],
+  "events": [
+    [70.255, "vegas-live"],
+    [70.255, "live", "sports_live"],
+    [150.0, "request", "start:v1"],
+    [150.263, "on-demand-start", "clock"],
+    [150.263, "vegas-interrupt"],
+    [175.263, "on-demand-expired"],
+    [200.0, "wifi-file", "Connected to HomeNet"],
+    [200.222, "vegas-interrupt"]
+  ]
+}

test/fixtures/run_loop_golden/vegas_live_in_ticker.json

@@ -0,0 +1,9 @@
+{
+  "screens": [
+    [0.0, "<vegas>", 30.008, "duration", 3751, null],
+    [30.008, "<vegas>", 30.007, "duration", 3751, null],
+    [60.015, "<vegas>", 30.008, "duration", 3751, null],
+    [90.023, "<vegas>", 9.977, "horizon", 1248, null]
+  ],
+  "events": []
+}

test/fixtures/run_loop_golden/wifi_notice.json

@@ -0,0 +1,21 @@
+{
+  "screens": [
+    [0.0, "clock", 20.0, "duration", 20, false],
+    [20.0, "weather", 5.0, "wifi", 6, true],
+    [25.0, "<wifi>", 5.0, "duration", 10, null],
+    [30.0, "weather", 20.0, "duration", 20, true],
+    [50.0, "clock", 20.0, "on-demand-start", 20, true],
+    [70.0, "clock", 10.0, "on-demand-expired", 10, true],
+    [80.0, "<wifi>", 15.0, "duration", 30, null],
+    [95.0, "clock", 20.0, "duration", 20, true],
+    [115.0, "weather", 20.0, "duration", 20, true],
+    [135.0, "clock", 15.0, "horizon", 15, true]
+  ],
+  "events": [
+    [25.0, "wifi-file", "Connected to HomeNet"],
+    [60.0, "request", "start:w1"],
+    [60.0, "on-demand-start", "clock"],
+    [65.0, "wifi-file", "AP mode on"],
+    [80.0, "on-demand-expired"]
+  ]
+}

test/js/README.md

@@ -51,10 +51,13 @@ server has none.
 | `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
 | `unit/test_inline_handler_escaping.js` | no | The store, saved-repository and custom-registry inline `onclick` handlers and the live `window.updateImageList` from `plugins_manager.js`: a registry id, URL or uploaded file name carrying `'`, `"` or entities adds no attributes and reaches the handler intact, and the store's View button opens only http(s) links |
 | `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
+| `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
+| `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
 | `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
 | `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
 | `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
 | `dom/test_no_double_fetch.js` | yes | Loads the **whole** `plugins_manager.js` and counts requests: typing in the store search must filter the cached list, not refetch `/api/v3/plugins/store/list` |
+| `dom/test_cache_page.js` | yes | The Cache tab as a page module (`js/pages/cache.js`) on the real partial: no inline script, one request per swap and per Refresh after repeated swaps, a cancelled request draws nothing, hostile keys stay text, delete/empty/error/login states |
 | `dom/test_tools_sections.js` | yes | The Tools tab's MQTT bridge and Pixlet editor sections: form prefill, the write-only password (blank means unchanged), the running-session banner and countdown, and that the editor link points at the host you loaded the page from |
 
 Point the DOM suites at a rig with a full plugin set when it matters — a dev box

test/js/dom/test_cache_page.js

@@ -0,0 +1,196 @@
+// The Cache tab as a page module (static/v3/js/pages/cache.js), in a real DOM
+// (jsdom) with the real server-rendered partial and the real API's payload
+// shape. The reference conversion for docs/WEB_FRONTEND_ARCHITECTURE.md, so
+// this pins what every converted page must do:
+//
+//   * the partial ships no <script>; its root is data-page="cache"
+//   * the page starts once per swap-in and stops on swap-out: repeated htmx
+//     swaps leave exactly one live set of listeners (one request per Refresh
+//     click, however many times the tab was reloaded)
+//   * a request still in flight when the page is swapped away is cancelled
+//     and draws nothing
+//   * server data reaches the page as text, never as markup
+const http = require('http');
+const path = require('path');
+const { pathToFileURL } = require('url');
+const { JSDOM, VirtualConsole } = require('jsdom');
+
+const BASE = process.env.BASE || 'http://localhost:5000';
+const JS = path.resolve(__dirname, '../../../web_interface/static/v3/js');
+const get = p => new Promise((res, rej) =>
+  http.get(BASE + p, r => { let d = ''; r.on('data', c => d += c); r.on('end', () => res(d)); }).on('error', rej));
+const load = f => import(pathToFileURL(path.join(JS, f)).href);
+const tick = ms => new Promise(r => setTimeout(r, ms || 0));
+
+let pass = 0, fail = 0;
+const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
+  : (fail++, console.log('  FAIL ' + l + (x !== undefined ? '  -> ' + JSON.stringify(x).slice(0, 300) : '')));
+
+(async () => {
+  const partial = await get('/partials/cache');
+  const real = JSON.parse(await get('/api/v3/cache/list'));
+  const { createRegistry } = await load('core/registry.js');
+  const { createApi } = await load('core/api.js');
+  const cachePage = await load('pages/cache.js');
+
+  console.log('\n── Cache tab: page module (real DOM) ──');
+  ok('the partial ships no inline script', !/<script/i.test(partial));
+  ok('the partial root is data-page="cache"', /data-page="cache"/.test(partial));
+  ok('the real API answers in the shape the page reads',
+     real.status === 'success' && real.data && Array.isArray(real.data.cache_files), real);
+
+  // Real shape, plus entries the page must treat as text.
+  const HOSTILE = '<img src=x onerror="window.pwned=1">\'"&';
+  const sample = Object.assign({}, real.data, {
+    cache_dir: real.data.cache_dir || '/var/cache/ledmatrix',
+    cache_files: [
+      { key: 'weather_current', filename: 'weather_current.json', age_seconds: 12,
+        age_display: '12s', size_display: '1.2 KB', modified_datetime: '2026-09-30T12:00:00' },
+      { key: HOSTILE, filename: HOSTILE + '.json', age_seconds: 7200,
+        age_display: '2h', size_display: '3 B', modified_datetime: '2026-09-30T10:00:00' },
+    ],
+  });
+
+  const errs = [];
+  const vc = new VirtualConsole();
+  vc.on('jsdomError', e => errs.push(String(e.message || e).split('\n')[0]));
+  vc.on('error', (...a) => errs.push(a.join(' ')));
+  const dom = new JSDOM(`<!doctype html><html><body><div id="cache-content">${partial}</div></body></html>`,
+    { url: BASE + '/', virtualConsole: vc });
+  const { window } = dom;
+  const doc = window.document;
+  const panel = doc.getElementById('cache-content');
+
+  // Controllable API.
+  let listBody = { status: 'success', data: sample };
+  let listMode = 'ok';
+  const requests = [];
+  const pending = [];
+  function fakeFetch(url, init) {
+    requests.push({ url, method: init.method, body: init.body });
+    const respond = (status, body, headers = {}) => Promise.resolve({
+      status, ok: status >= 200 && status < 300,
+      headers: { get: n => headers[n] || null },
+      text: () => Promise.resolve(JSON.stringify(body)),
+    });
+    if (url === '/api/v3/cache/delete') return respond(200, { status: 'success', message: 'Deleted it' });
+    if (listMode === 'network') return Promise.reject(new TypeError('Failed to fetch'));
+    if (listMode === 'login') return respond(401, { status: 'error' }, { 'X-LEDMatrix-Login': '/login' });
+    if (listMode === 'hang') {
+      return new Promise((resolve, reject) => {
+        pending.push(resolve);
+        init.signal.addEventListener('abort', () => {
+          const e = new Error('aborted'); e.name = 'AbortError'; reject(e);
+        });
+      });
+    }
+    return respond(200, listBody);
+  }
+  const notes = [];
+  const registry = createRegistry({
+    document: doc,
+    context: { api: createApi({ fetch: fakeFetch }), notify: (m, t) => notes.push([m, t]) },
+  });
+  registry.register('cache', cachePage);
+
+  const lists = () => requests.filter(r => r.url === '/api/v3/cache/list').length;
+  const $ = id => doc.getElementById(id);
+  const visible = id => !$(id).classList.contains('hidden');
+  // What htmx does around a swap of the tab panel.
+  async function swap() {
+    panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
+    panel.innerHTML = partial;
+    panel.dispatchEvent(new window.CustomEvent('htmx:afterSwap', { bubbles: true, detail: { target: panel } }));
+    await tick(20);
+  }
+
+  await registry.start();
+  await tick(20);
+
+  // ── first load ──────────────────────────────────────────────────────────
+  ok('one list request on start', lists() === 1, lists());
+  const rows = doc.querySelectorAll('#cache-files-tbody tr');
+  ok('one row per cache file', rows.length === 2, rows.length);
+  ok('cache directory shown', $('cache-dir').textContent === sample.cache_dir, $('cache-dir').textContent);
+  ok('hostile key is shown as text', rows[1].textContent.includes(HOSTILE));
+  ok('...and created no element', !doc.querySelector('#cache-files-tbody img') && !window.pwned);
+  const buttons = [...doc.querySelectorAll('#cache-files-tbody button[data-cache-key]')];
+  ok('delete buttons carry the exact key', buttons.map(b => b.dataset.cacheKey).join('|') === 'weather_current|' + HOSTILE);
+  ok('delete buttons have no inline handler', buttons.every(b => !b.getAttribute('onclick')));
+  ok('fresh entries are green, old ones red',
+     rows[0].querySelector('.text-green-600') && rows[1].querySelector('.text-red-600'));
+
+  // ── repeated swaps ──────────────────────────────────────────────────────
+  const oldRefresh = $('refresh-cache-btn');
+  for (let i = 0; i < 5; i++) await swap();
+  ok('one list request per swap', lists() === 6, lists());
+  ok('one mounted page after five swaps', registry.list().length === 1, registry.list().length);
+  const before = lists();
+  $('refresh-cache-btn').click();
+  await tick(20);
+  ok('Refresh makes exactly one request (no duplicate listeners)', lists() === before + 1, lists() - before);
+  oldRefresh.click();
+  await tick(20);
+  ok('a swapped-out button does nothing', lists() === before + 1, lists() - before);
+
+  // ── delete ──────────────────────────────────────────────────────────────
+  let asked = null;
+  window.confirm = msg => { asked = msg; return false; };
+  doc.querySelector('#cache-files-tbody button[data-cache-key]').click();
+  await tick(20);
+  ok('delete asks first', asked && asked.includes('weather_current'), asked);
+  ok('cancel sends nothing', !requests.some(r => r.url === '/api/v3/cache/delete'));
+  window.confirm = () => true;
+  const listsBeforeDelete = lists();
+  doc.querySelectorAll('#cache-files-tbody button[data-cache-key]')[1].click();
+  await tick(30);
+  const del = requests.filter(r => r.url === '/api/v3/cache/delete');
+  ok('one delete request', del.length === 1, del.length);
+  ok('it posts the exact key as JSON', del[0] && del[0].method === 'POST' && JSON.parse(del[0].body).key === HOSTILE);
+  ok('the server\'s message is shown', notes.some(n => n[0] === 'Deleted it' && n[1] === 'success'), notes);
+  ok('the list reloads after a delete', lists() === listsBeforeDelete + 1, lists() - listsBeforeDelete);
+  const viaAlias = await cachePage.deleteCacheFile('weather_current');
+  ok('the old deleteCacheFile(key) entry point still works', viaAlias === true);
+
+  // ── states ──────────────────────────────────────────────────────────────
+  listBody = { status: 'success', data: { cache_dir: null, cache_files: [] } };
+  $('refresh-cache-btn').click(); await tick(20);
+  ok('empty state shown', visible('cache-empty') && !visible('cache-error') && !doc.querySelector('#cache-files-tbody tr'));
+  ok('a missing cache directory says so', $('cache-dir').textContent === 'Not configured');
+  ok('a missing cache directory is greyed', $('cache-dir').classList.contains('text-gray-500'));
+  listBody = { status: 'success', data: { cache_dir: '/var/cache/ledmatrix', cache_files: [] } };
+  $('refresh-cache-btn').click(); await tick(20);
+  ok('a directory that appears later is not greyed', $('cache-dir').textContent === '/var/cache/ledmatrix'
+     && !$('cache-dir').classList.contains('text-gray-500'));
+
+  listBody = { status: 'error', message: 'Cache unavailable' };
+  $('refresh-cache-btn').click(); await tick(20);
+  ok('an API error shows its message', visible('cache-error') && $('cache-error-message').textContent === 'Cache unavailable',
+     $('cache-error-message').textContent);
+
+  listMode = 'network';
+  $('refresh-cache-btn').click(); await tick(20);
+  ok('a network failure says so', $('cache-error-message').textContent === 'Error loading cache files: Failed to fetch',
+     $('cache-error-message').textContent);
+
+  listMode = 'ok'; listBody = { status: 'success', data: sample };
+  await swap();
+  listMode = 'login';
+  $('cache-error').classList.add('hidden');
+  $('refresh-cache-btn').click(); await tick(20);
+  ok('the login redirect draws no error', !visible('cache-error'));
+
+  // ── in flight when swapped away ─────────────────────────────────────────
+  listMode = 'hang';
+  $('refresh-cache-btn').click(); await tick(5);
+  ok('a request is in flight', pending.length >= 1);
+  listMode = 'ok';
+  await swap();
+  ok('the new page drew its own list', doc.querySelectorAll('#cache-files-tbody tr').length === 2);
+  ok('the cancelled request drew nothing', !visible('cache-error'));
+
+  ok('no DOM errors', errs.length === 0, errs);
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/js/run_all.js

@@ -22,9 +22,10 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
               'unit/test_style_editor_layout_leaf_collision.js',
               'unit/test_update_all.js', 'unit/test_inline_handler_escaping.js',
               'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js',
-              'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js'];
+              'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js',
+              'unit/test_page_registry.js', 'unit/test_core_modules.js'];
 const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
-             'dom/test_tools_sections.js'];
+             'dom/test_tools_sections.js', 'dom/test_cache_page.js'];
 
 function reachable(url) {
   return new Promise(res => {

test/js/unit/test_core_modules.js

@@ -0,0 +1,143 @@
+// core/api.js and core/facade.js (web_interface/static/v3/js/core/).
+//
+// api.js: one fetch wrapper. Resolves to the parsed JSON body; rejects with
+// an ApiError for HTTP errors, {"status": "error"} bodies, unreadable bodies
+// and network failures; passes an AbortError through untouched; and turns the
+// optional web login's 401 + X-LEDMatrix-Login (#683) into a quiet
+// `loginRequired` error, since base.html's fetch wrapper is already sending
+// the browser to the login page.
+//
+// facade.js: window.LEDMatrix, and deprecated aliases for moved globals.
+//
+// Plain node: imports the shipped ES modules, no DOM needed.
+
+const path = require('path');
+const { pathToFileURL } = require('url');
+
+const CORE = path.resolve(__dirname, '../../../web_interface/static/v3/js/core');
+const load = f => import(pathToFileURL(path.join(CORE, f)).href);
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  -> ' + JSON.stringify(extra) : '')));
+
+function response(status, body, headers = {}) {
+  const text = typeof body === 'string' ? body : JSON.stringify(body);
+  return {
+    status, ok: status >= 200 && status < 300,
+    headers: { get: n => headers[n] !== undefined ? headers[n] : null },
+    text: () => Promise.resolve(text),
+  };
+}
+async function rejection(promise) {
+  try { await promise; return null; } catch (e) { return e; }
+}
+
+(async () => {
+  const { createApi, ApiError, isLoginRedirect, isAbort } = await load('api.js');
+  const { createFacade, installFacade, defineDeprecatedAlias, FACADE_VERSION } = await load('facade.js');
+  const { createRegistry } = await load('registry.js');
+
+  console.log('\n1. api: requests go out as JSON, through fetch at call time');
+  {
+    const calls = [];
+    const api = createApi({ fetch: (url, init) => { calls.push([url, init]); return Promise.resolve(response(200, { status: 'success', data: { n: 1 } })); } });
+    const body = await api.get('/api/v3/cache/list');
+    ok('resolves to the parsed body', body.data.n === 1, body);
+    ok('GET has no body', calls[0][1].method === 'GET' && calls[0][1].body === undefined);
+    await api.post('/api/v3/cache/delete', { key: 'a"b' });
+    ok('POST sends JSON', calls[1][1].headers['Content-Type'] === 'application/json' && JSON.parse(calls[1][1].body).key === 'a"b');
+    const controller = new AbortController();
+    await api.get('/api/v3/x', { signal: controller.signal });
+    ok('the signal is passed to fetch', calls[2][1].signal === controller.signal);
+
+    // Default: window.fetch looked up per call, so base.html's login wrapper
+    // (installed before any module runs, or replaced later) is the one used.
+    const seen = [];
+    globalThis.fetch = () => { seen.push('first'); return Promise.resolve(response(200, { status: 'success' })); };
+    const live = createApi();
+    await live.get('/api/v3/a');
+    globalThis.fetch = () => { seen.push('second'); return Promise.resolve(response(200, { status: 'success' })); };
+    await live.get('/api/v3/b');
+    ok('uses whatever window.fetch is at call time', seen.join() === 'first,second', seen);
+  }
+
+  console.log('\n2. api: errors');
+  {
+    const api = r => createApi({ fetch: () => (r instanceof Error ? Promise.reject(r) : Promise.resolve(r)) });
+    let e = await rejection(api(response(500, { status: 'error', message: 'Disk full' })).get('/api/v3/x'));
+    ok('HTTP error carries status and message', e instanceof ApiError && e.status === 500 && e.message === 'Disk full', e && e.message);
+    e = await rejection(api(response(200, { status: 'error', message: 'Nope' })).get('/api/v3/x'));
+    ok('a 200 with status "error" is an error', e instanceof ApiError && e.status === 200 && e.message === 'Nope' && e.body.status === 'error');
+    e = await rejection(api(response(502, '<html>Bad gateway</html>')).get('/api/v3/x'));
+    ok('a non-JSON error page says the status', e instanceof ApiError && e.status === 502 && e.message === 'HTTP 502', e && e.message);
+    e = await rejection(api(response(200, 'not json')).get('/api/v3/x'));
+    ok('an unreadable 200 is an error', e instanceof ApiError && /Unreadable/.test(e.message));
+    e = await rejection(api(new TypeError('Failed to fetch')).get('/api/v3/x'));
+    ok('a network failure is flagged', e instanceof ApiError && e.network && e.status === 0 && e.message === 'Failed to fetch');
+    const abort = new Error('aborted'); abort.name = 'AbortError';
+    e = await rejection(api(abort).get('/api/v3/x'));
+    ok('an abort passes through untouched', e === abort && isAbort(e));
+  }
+
+  console.log('\n3. api: the optional web login (#683)');
+  {
+    const login = response(401, { status: 'error', message: 'Login required' }, { 'X-LEDMatrix-Login': '/login?next=/' });
+    ok('isLoginRedirect matches the wrapper in base.html', isLoginRedirect(login));
+    ok('...not a protocol-relative URL', !isLoginRedirect(response(401, {}, { 'X-LEDMatrix-Login': '//evil.example/' })));
+    ok('...not a 401 without the header', !isLoginRedirect(response(401, {})));
+    ok('...not another status', !isLoginRedirect(response(403, {}, { 'X-LEDMatrix-Login': '/login' })));
+    const e = await rejection(createApi({ fetch: () => Promise.resolve(login) }).get('/api/v3/x'));
+    ok('rejects quietly with loginRequired', e instanceof ApiError && e.loginRequired && e.status === 401);
+  }
+
+  console.log('\n4. api: only this server\'s paths');
+  {
+    const api = createApi({ fetch: () => Promise.resolve(response(200, { status: 'success' })) });
+    for (const bad of ['//evil.example/x', 'https://evil.example/x', 'api/v3/x', '/a b', '/a\\b']) {
+      const e = await rejection(api.get(bad));
+      ok(`refuses ${JSON.stringify(bad)}`, e instanceof TypeError, e && e.message);
+    }
+  }
+
+  console.log('\n5. facade: window.LEDMatrix');
+  {
+    const warnings = [];
+    const win = { console: { warn: m => warnings.push(m), log() {}, error() {} } };
+    const api = createApi({ fetch: () => Promise.resolve(response(200, { status: 'success' })) });
+    const reg = createRegistry({ document: { addEventListener() {}, removeEventListener() {}, querySelectorAll: () => [] } });
+    const facade = installFacade(win, createFacade(win, api, reg));
+    ok('installed as window.LEDMatrix', win.LEDMatrix === facade && facade.version === FACADE_VERSION);
+    ok('exposes api and pages', facade.api === api && typeof facade.pages.register === 'function' && typeof facade.pages.refresh === 'function');
+    ok('is frozen', Object.isFrozen(facade) && Object.isFrozen(facade.pages));
+    win.LEDEscape = { html: s => s };
+    win.LEDMatrixWidgets = { get() {} };
+    ok('escape and widgets read through at call time', facade.escape === win.LEDEscape && facade.widgets === win.LEDMatrixWidgets);
+    const notes = [];
+    win.showNotification = (m, t) => notes.push([m, t]);
+    facade.notify('saved', 'success');
+    win.showNotification = (m, t) => notes.push(['replaced', m, t]);
+    facade.notify('again');
+    ok('notify uses the current showNotification', JSON.stringify(notes) === JSON.stringify([['saved', 'success'], ['replaced', 'again', 'info']]), notes);
+  }
+
+  console.log('\n6. facade: deprecated aliases keep old globals working');
+  {
+    const warnings = [];
+    const win = {};
+    const logger = { warn: m => warnings.push(m) };
+    const calls = [];
+    defineDeprecatedAlias(win, 'deleteCacheFile', function(key) { calls.push([this, key]); return 'done'; }, 'the Delete buttons', logger);
+    ok('a function alias forwards its arguments and result', win.deleteCacheFile('k1') === 'done' && calls[0][1] === 'k1');
+    win.deleteCacheFile('k2');
+    ok('warns once, naming the replacement', warnings.length === 1 && /deleteCacheFile/.test(warnings[0]) && /the Delete buttons/.test(warnings[0]), warnings);
+    defineDeprecatedAlias(win, 'oldThing', { a: 1 }, 'LEDMatrix.thing', logger);
+    ok('a value alias is a getter', win.oldThing.a === 1 && warnings.length === 2);
+    Object.defineProperty(win, 'locked', { value: 1, configurable: false });
+    ok('a non-configurable global is left alone', defineDeprecatedAlias(win, 'locked', () => 2, null, logger) === false && win.locked === 1);
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/js/unit/test_html_escaping.js

@@ -115,8 +115,8 @@ const ESCAPERS = [
    'templates/v3/partials/tools.html', 'function phEscape(s) {', 'phEscape', false],
   ['logs.html (escapeHtml)',
    'templates/v3/partials/logs.html', 'function escapeHtml(text) {', 'escapeHtml', false],
-  ['cache.html (escapeHtml)',
-   'templates/v3/partials/cache.html', 'function escapeHtml(text) {', 'escapeHtml', false],
+  // cache.html has no script any more: js/pages/cache.js builds its rows with
+  // textContent, and test/js/dom/test_cache_page.js checks a hostile key.
 ];
 
 // The breakout payload: closes a double-quoted attribute and opens an event

test/js/unit/test_page_registry.js

@@ -0,0 +1,247 @@
+// The page lifecycle (web_interface/static/v3/js/core/registry.js).
+//
+// A converted partial's root carries data-page="<name>"; the registry calls
+// the page module's init(root, ctx) once when the root appears and
+// destroy(root, ctx) when htmx swaps it away, aborting ctx.signal so every
+// listener the page registered with it goes too. This is what replaces the
+// inline <script> blocks that htmx-config.js re-ran on every swap.
+//
+// Imports the shipped ES module directly (js/core/package.json marks the
+// directory "type": "module"). The DOM is a minimal shim, so this needs only
+// node and runs under test/test_js_unit_suites.py as well as run_all.js.
+
+const path = require('path');
+const { pathToFileURL } = require('url');
+
+const CORE = path.resolve(__dirname, '../../../web_interface/static/v3/js/core');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  -> ' + JSON.stringify(extra) : '')));
+
+// ── DOM shim: just what the registry touches ───────────────────────────────
+class El extends EventTarget {
+  constructor(tag, attrs = {}) {
+    super();
+    this.tagName = tag.toUpperCase();
+    this.attrs = new Map(Object.entries(attrs));
+    this.children = [];
+    this.parentNode = null;
+  }
+  getAttribute(n) { return this.attrs.has(n) ? this.attrs.get(n) : null; }
+  setAttribute(n, v) { this.attrs.set(n, String(v)); }
+  appendChild(c) { if (c.parentNode) c.remove(); c.parentNode = this; this.children.push(c); return c; }
+  remove() { if (this.parentNode) { this.parentNode.children = this.parentNode.children.filter(x => x !== this); this.parentNode = null; } }
+  replaceChildren(...nodes) { this.children.slice().forEach(c => c.remove()); nodes.forEach(n => this.appendChild(n)); }
+  *descendants() { for (const c of this.children) { yield c; yield* c.descendants(); } }
+  // Only the one selector the registry uses: [attr]
+  matches(sel) { const m = /^\[([\w-]+)\]$/.exec(sel); return !!m && this.attrs.has(m[1]); }
+  querySelectorAll(sel) { return [...this.descendants()].filter(e => e.matches(sel)); }
+  contains(other) { for (let n = other; n; n = n.parentNode) if (n === this) return true; return false; }
+  get isConnected() { let n = this; while (n.parentNode) n = n.parentNode; return n instanceof Doc; }
+}
+class Doc extends El {
+  constructor() { super('#document'); this.documentElement = this.appendChild(new El('html')); this.body = this.documentElement.appendChild(new El('body')); }
+}
+const event = (type, detail) => { const e = new Event(type); e.detail = detail; return e; };
+// htmx fires its events on the target and they bubble to the document, where
+// the registry listens. Node's EventTarget has no tree, so walk it here.
+function fire(target, type, detail) {
+  for (let n = target; n; n = n.parentNode) n.dispatchEvent(event(type, detail));
+}
+const tick = () => new Promise(r => setTimeout(r, 0));
+
+// A page module that records its lifecycle, and checks ctx.signal works.
+function recorder(log) {
+  return {
+    init(root, ctx) {
+      log.push(['init', root.getAttribute('id'), ctx.name]);
+      ctx.state.clicks = 0;
+      root.addEventListener('click', () => { ctx.state.clicks++; log.push(['click', root.getAttribute('id')]); }, { signal: ctx.signal });
+      ctx.signal.addEventListener('abort', () => log.push(['aborted', root.getAttribute('id')]));
+      if (ctx.service) log.push(['service', ctx.service]);
+    },
+    destroy(root, ctx) { log.push(['destroy', root.getAttribute('id'), ctx.signal.aborted]); },
+  };
+}
+
+(async () => {
+  const { createRegistry, PAGE_ATTRIBUTE } = await import(pathToFileURL(path.join(CORE, 'registry.js')).href);
+  const quiet = { error: () => {}, warn: () => {} };
+
+  console.log('\n1. mounts on start, once per root, with the shared context');
+  {
+    const doc = new Doc();
+    const panel = doc.body.appendChild(new El('div', { id: 'cache-content' }));
+    const root = panel.appendChild(new El('div', { id: 'a', [PAGE_ATTRIBUTE]: 'demo' }));
+    const log = [];
+    const reg = createRegistry({ document: doc, context: { service: 'api' }, logger: quiet });
+    reg.register('demo', recorder(log));
+    await reg.start();
+    ok('init ran once on start', log.filter(e => e[0] === 'init').length === 1, log);
+    ok('ctx carries the page name', log[0][2] === 'demo', log);
+    ok('ctx carries the shared services', log.some(e => e[0] === 'service' && e[1] === 'api'), log);
+    await reg.refresh(); await reg.scan(); await reg.mount(root);
+    ok('refresh/scan/mount again do not re-init', log.filter(e => e[0] === 'init').length === 1, log);
+    root.dispatchEvent(new Event('click'));
+    ok('the page listener works', log.filter(e => e[0] === 'click').length === 1, log);
+    ok('list() reports the mounted page', reg.list().length === 1 && reg.list()[0].initialised === true, reg.list().length);
+  }
+
+  console.log('\n2. an htmx swap destroys the old page and starts the new one');
+  {
+    const doc = new Doc();
+    const panel = doc.body.appendChild(new El('div', { id: 'panel' }));
+    const first = panel.appendChild(new El('div', { id: 'first', [PAGE_ATTRIBUTE]: 'demo' }));
+    const log = [];
+    const reg = createRegistry({ document: doc, logger: quiet });
+    reg.register('demo', recorder(log));
+    await reg.start();
+
+    for (let i = 0; i < 5; i++) {
+      fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
+      panel.replaceChildren(new El('div', { id: 'swap' + i, [PAGE_ATTRIBUTE]: 'demo' }));
+      fire(panel, 'htmx:afterSwap', { target: panel });
+      await tick();
+    }
+    const inits = log.filter(e => e[0] === 'init').map(e => e[1]);
+    const destroys = log.filter(e => e[0] === 'destroy').map(e => e[1]);
+    ok('one init per swapped-in root', JSON.stringify(inits) === JSON.stringify(['first', 'swap0', 'swap1', 'swap2', 'swap3', 'swap4']), inits);
+    ok('one destroy per swapped-out root', JSON.stringify(destroys) === JSON.stringify(['first', 'swap0', 'swap1', 'swap2', 'swap3']), destroys);
+    ok('destroy runs before the signal is aborted', log.filter(e => e[0] === 'destroy').every(e => e[2] === false), log);
+    ok('every destroyed page had its signal aborted', log.filter(e => e[0] === 'aborted').length === 5, log);
+    ok('only the live page is mounted', reg.list().length === 1 && reg.list()[0].root.getAttribute('id') === 'swap4');
+    // The old root's listener was registered with ctx.signal: gone.
+    first.dispatchEvent(new Event('click'));
+    ok('a destroyed page no longer hears its own events', !log.some(e => e[0] === 'click' && e[1] === 'first'), log);
+  }
+
+  console.log('\n3. a vetoed swap (shouldSwap false, e.g. an error response) keeps the page');
+  {
+    const doc = new Doc();
+    const panel = doc.body.appendChild(new El('div'));
+    panel.appendChild(new El('div', { id: 'keep', [PAGE_ATTRIBUTE]: 'demo' }));
+    const log = [];
+    const reg = createRegistry({ document: doc, logger: quiet });
+    reg.register('demo', recorder(log));
+    await reg.start();
+    fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: false });
+    fire(panel, 'htmx:afterSwap', { target: panel });
+    ok('not destroyed', !log.some(e => e[0] === 'destroy'), log);
+    ok('still mounted', reg.list().length === 1);
+  }
+
+  console.log('\n4. a swap elsewhere leaves the page alone');
+  {
+    const doc = new Doc();
+    const a = doc.body.appendChild(new El('div'));
+    const b = doc.body.appendChild(new El('div'));
+    a.appendChild(new El('div', { id: 'a-page', [PAGE_ATTRIBUTE]: 'demo' }));
+    const log = [];
+    const reg = createRegistry({ document: doc, logger: quiet });
+    reg.register('demo', recorder(log));
+    await reg.start();
+    fire(b, 'htmx:beforeSwap', { target: b, shouldSwap: true });
+    b.replaceChildren(new El('p'));
+    fire(b, 'htmx:afterSwap', { target: b });
+    ok('the other panel\'s page is untouched', log.filter(e => e[0] !== 'service').map(e => e[0]).join() === 'init', log);
+  }
+
+  console.log('\n5. content removed without htmx (Alpine x-if, outerHTML) is swept on the next swap or refresh');
+  {
+    const doc = new Doc();
+    const panel = doc.body.appendChild(new El('div'));
+    const root = panel.appendChild(new El('div', { id: 'gone', [PAGE_ATTRIBUTE]: 'demo' }));
+    const log = [];
+    const reg = createRegistry({ document: doc, logger: quiet });
+    reg.register('demo', recorder(log));
+    await reg.start();
+    root.remove();
+    ok('nothing happens until the registry looks', !log.some(e => e[0] === 'destroy'));
+    await reg.refresh();
+    ok('refresh() destroys a detached root', log.some(e => e[0] === 'destroy' && e[1] === 'gone'), log);
+    // loadPartialDirect inserts HTML without htmx events and calls refresh().
+    panel.appendChild(new El('div', { id: 'direct', [PAGE_ATTRIBUTE]: 'demo' }));
+    await reg.refresh();
+    ok('refresh() starts a root inserted without htmx', log.some(e => e[0] === 'init' && e[1] === 'direct'), log);
+  }
+
+  console.log('\n6. lazy page modules: loaded on first use, once');
+  {
+    const doc = new Doc();
+    const panel = doc.body.appendChild(new El('div'));
+    const log = [];
+    let loads = 0;
+    const reg = createRegistry({ document: doc, logger: quiet });
+    reg.register('lazy', () => { loads++; return Promise.resolve({ default: recorder(log) }); });
+    await reg.start();
+    ok('not loaded while no partial uses it', loads === 0);
+    for (let i = 0; i < 3; i++) {
+      fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
+      panel.replaceChildren(new El('div', { id: 'l' + i, [PAGE_ATTRIBUTE]: 'lazy' }));
+      fire(panel, 'htmx:afterSwap', { target: panel });
+      await tick(); await tick();
+    }
+    ok('loader called once', loads === 1, loads);
+    ok('a default export works', log.filter(e => e[0] === 'init').length === 3, log);
+
+    // Swapped away while its module is still loading: never initialised.
+    let release;
+    const slowLog = [];
+    reg.register('slow', () => new Promise(r => { release = r; }));
+    fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
+    panel.replaceChildren(new El('div', { id: 's', [PAGE_ATTRIBUTE]: 'slow' }));
+    fire(panel, 'htmx:afterSwap', { target: panel });
+    fire(panel, 'htmx:beforeSwap', { target: panel, shouldSwap: true });
+    panel.replaceChildren(new El('p'));
+    fire(panel, 'htmx:afterSwap', { target: panel });
+    release(recorder(slowLog));
+    await tick(); await tick();
+    ok('a page destroyed before its module arrived never runs init', slowLog.length === 0, slowLog);
+  }
+
+  console.log('\n7. a page registered after its partial arrived still starts');
+  {
+    const doc = new Doc();
+    doc.body.appendChild(new El('div', { id: 'early', [PAGE_ATTRIBUTE]: 'late' }));
+    const log = [];
+    const reg = createRegistry({ document: doc, logger: quiet });
+    await reg.start();
+    reg.register('late', recorder(log));
+    await tick();
+    ok('init ran on register', log.some(e => e[0] === 'init' && e[1] === 'early'), log);
+  }
+
+  console.log('\n8. a failing page is contained');
+  {
+    const doc = new Doc();
+    doc.body.appendChild(new El('div', { id: 'bad', [PAGE_ATTRIBUTE]: 'bad' }));
+    doc.body.appendChild(new El('div', { id: 'good', [PAGE_ATTRIBUTE]: 'demo' }));
+    const errors = [];
+    const log = [];
+    const reg = createRegistry({ document: doc, logger: { error: (...a) => errors.push(a.join(' ')), warn() {} } });
+    reg.register('bad', { init() { throw new Error('boom'); }, destroy() { log.push(['bad-destroy']); } });
+    reg.register('demo', recorder(log));
+    await reg.start();
+    ok('the error is logged with the page name', errors.length === 1 && /bad/.test(errors[0]), errors);
+    ok('the other page still started', log.some(e => e[0] === 'init' && e[1] === 'good'), log);
+    reg.stop();
+    ok('destroy is not called for a page whose init failed', !log.some(e => e[0] === 'bad-destroy'), log);
+    ok('stop() destroys every page', log.some(e => e[0] === 'destroy' && e[1] === 'good') && reg.list().length === 0, log);
+  }
+
+  console.log('\n9. register() rejects mistakes loudly');
+  {
+    const reg = createRegistry({ document: new Doc(), logger: quiet });
+    const throws = fn => { try { fn(); return false; } catch (e) { return true; } };
+    ok('no name', throws(() => reg.register('', { init() {} })));
+    ok('no init and not a loader', throws(() => reg.register('x', {})));
+    reg.register('dup', { init() {} });
+    ok('a duplicate name', throws(() => reg.register('dup', { init() {} })));
+    ok('has()', reg.has('dup') && !reg.has('nope'));
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/plugins/test_visual_rendering.py

@@ -196,42 +196,3 @@ class TestVisualDisplayManager:
         assert '11th' in result
 
 
-class TestWeatherDrawing:
-    """Test weather icon rendering."""
-
-    def test_draw_sun(self):
-        vdm = VisualTestDisplayManager(width=128, height=32)
-        vdm.draw_sun(0, 0, 16)
-        pixels = list(vdm.image.getdata())
-        non_black = [p for p in pixels if p != (0, 0, 0)]
-        assert len(non_black) > 0
-
-    def test_draw_cloud(self):
-        vdm = VisualTestDisplayManager(width=128, height=32)
-        vdm.draw_cloud(0, 0, 16)
-        pixels = list(vdm.image.getdata())
-        non_black = [p for p in pixels if p != (0, 0, 0)]
-        assert len(non_black) > 0
-
-    def test_draw_rain(self):
-        vdm = VisualTestDisplayManager(width=128, height=32)
-        vdm.draw_rain(0, 0, 16)
-        pixels = list(vdm.image.getdata())
-        non_black = [p for p in pixels if p != (0, 0, 0)]
-        assert len(non_black) > 0
-
-    def test_draw_snow(self):
-        vdm = VisualTestDisplayManager(width=128, height=32)
-        vdm.draw_snow(0, 0, 16)
-        pixels = list(vdm.image.getdata())
-        non_black = [p for p in pixels if p != (0, 0, 0)]
-        assert len(non_black) > 0
-
-    def test_draw_weather_icon_dispatches(self):
-        vdm = VisualTestDisplayManager(width=128, height=32)
-        for condition in ['clear', 'cloudy', 'rain', 'snow', 'storm', 'unknown']:
-            vdm.clear()
-            vdm.draw_weather_icon(condition, 0, 0, 16)
-            pixels = list(vdm.image.getdata())
-            non_black = [p for p in pixels if p != (0, 0, 0)]
-            assert len(non_black) > 0, f"draw_weather_icon('{condition}') should render pixels"

test/test_api_v3_on_demand_socket.py

@@ -0,0 +1,206 @@
+"""POST /display/on-demand/start and /stop: control socket first, mailbox fallback.
+
+The routes hand the request to the display over the control socket
+(src/ipc) and get an acknowledgement. On any failure -- no socket (a stopped
+display, or one older than the socket), a timeout, a refusal, a bug in the
+client -- they write the file mailbox exactly as they did before the socket
+existed. These tests pin both paths, that exactly one of them is used, that
+the response says which, and that the request id is the same either way (the
+display deduplicates on it).
+
+The socket client is patched at the route's module attribute; the last class
+runs a real server on a temp socket (Linux/macOS only).
+"""
+
+import os
+import sys
+from pathlib import Path
+from unittest.mock import patch
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
+
+from src.ipc import client as control_client  # noqa: E402
+from src.ipc import contract as c  # noqa: E402
+
+START_URL = "/api/v3/display/on-demand/start"
+STOP_URL = "/api/v3/display/on-demand/stop"
+MAILBOX = "display_on_demand_request"
+CLIENT = "web_interface.blueprints.api_v3.display.control_client"
+
+
+@pytest.fixture
+def service(api_v3_module):
+    """A running display service; records systemctl calls and mailbox writes."""
+    api_v3_module.api_v3.plugin_catalog = None
+    api_v3_module.api_v3.config_manager = None
+    state = {"active": True}
+    calls = []
+
+    def status():
+        return {"active": state["active"]}
+
+    def systemctl(args):
+        calls.append(("systemctl", args[-2]))
+        if args[-2:] == ["start", "ledmatrix.service"]:
+            state["active"] = True
+        return {"returncode": 0, "stdout": "", "stderr": ""}
+
+    cache = api_v3_module.api_v3.cache_manager
+    cache.set.side_effect = lambda key, value, *a, **kw: calls.append(("cache", key))
+    with patch("web_interface.blueprints.api_v3._get_display_service_status",
+               side_effect=status), \
+         patch("web_interface.blueprints.api_v3.display._get_display_service_status",
+               side_effect=status), \
+         patch("web_interface.blueprints.api_v3._run_systemctl_command",
+               side_effect=systemctl), \
+         patch("web_interface.blueprints.api_v3.display._stop_display_service"):
+        yield {"state": state, "cache": cache, "calls": calls}
+
+
+def _mailbox_writes(cache):
+    return [call.args[1] for call in cache.set.call_args_list
+            if call.args and call.args[0] == MAILBOX]
+
+
+def _ack(request_id, *a, **kw):
+    return {"accepted": True, "request_id": request_id, "queued": 1}
+
+
+class TestSocketPath:
+    def test_start_goes_over_the_socket_and_skips_the_mailbox(self, api_v3_client, service):
+        with patch(f"{CLIENT}.on_demand_start", side_effect=_ack) as start:
+            resp = api_v3_client.post(START_URL, json={
+                "plugin_id": "weather", "mode": "weather_current",
+                "duration": 60, "pinned": True})
+        assert resp.status_code == 200, resp.get_json()
+        data = resp.get_json()["data"]
+        assert data["transport"] == "socket"
+        assert "socket_error" not in data
+        assert _mailbox_writes(service["cache"]) == []
+        start.assert_called_once_with(data["request_id"], "weather", "weather_current", 60, True)
+
+    def test_a_callers_request_id_is_passed_through(self, api_v3_client, service):
+        with patch(f"{CLIENT}.on_demand_start", side_effect=_ack) as start:
+            data = api_v3_client.post(START_URL, json={
+                "plugin_id": "weather", "request_id": "ha-123"}).get_json()["data"]
+        assert data["request_id"] == "ha-123"
+        assert start.call_args.args[0] == "ha-123"
+
+    def test_stop_goes_over_the_socket(self, api_v3_client, service):
+        with patch(f"{CLIENT}.on_demand_stop", side_effect=_ack) as stop:
+            data = api_v3_client.post(STOP_URL, json={}).get_json()["data"]
+        assert data["transport"] == "socket"
+        stop.assert_called_once_with(data["request_id"])
+        assert _mailbox_writes(service["cache"]) == []
+
+
+class TestMailboxFallback:
+    @pytest.mark.parametrize("reason", [
+        "no_socket", "refused", "timeout", "closed", "bad_response", "invalid_request",
+        "busy", "unknown_command", "unsupported_version", "disabled", "unsupported",
+    ])
+    def test_any_socket_failure_writes_the_mailbox_as_before(
+            self, api_v3_client, service, reason):
+        with patch(f"{CLIENT}.on_demand_start",
+                   side_effect=control_client.ControlError(reason, "x")):
+            resp = api_v3_client.post(START_URL, json={
+                "plugin_id": "weather", "mode": "weather_current",
+                "duration": 60, "pinned": True})
+        assert resp.status_code == 200
+        data = resp.get_json()["data"]
+        assert data["transport"] == "mailbox"
+        assert data["socket_error"] == reason
+        [write] = _mailbox_writes(service["cache"])
+        assert write["request_id"] == data["request_id"]
+        assert write["action"] == "start"
+        assert (write["plugin_id"], write["mode"], write["duration"], write["pinned"]) == \
+            ("weather", "weather_current", 60, True)
+
+    def test_a_client_bug_still_falls_back(self, api_v3_client, service):
+        with patch(f"{CLIENT}.on_demand_start", side_effect=RuntimeError("boom")):
+            resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
+        assert resp.status_code == 200
+        assert resp.get_json()["data"]["socket_error"] == "internal"
+        assert len(_mailbox_writes(service["cache"])) == 1
+
+    def test_an_unknown_reason_is_reported_as_other(self, api_v3_client, service):
+        # Only known codes are echoed back; anything else stays server-side.
+        with patch(f"{CLIENT}.on_demand_start",
+                   side_effect=control_client.ControlError("/run/secret/path", "x")):
+            data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
+        assert data["transport"] == "mailbox"
+        assert data["socket_error"] == "other"
+        assert len(_mailbox_writes(service["cache"])) == 1
+
+    def test_every_display_error_code_is_reportable(self):
+        from web_interface.blueprints.api_v3 import display
+        codes = {v for k, v in vars(c.ErrorCode).items() if not k.startswith("_")}
+        assert codes <= set(display._REPORTABLE_SOCKET_REASONS)
+
+    def test_stop_falls_back(self, api_v3_client, service):
+        with patch(f"{CLIENT}.on_demand_stop",
+                   side_effect=control_client.ControlError("timeout")):
+            data = api_v3_client.post(STOP_URL, json={}).get_json()["data"]
+        assert data["transport"] == "mailbox"
+        [write] = _mailbox_writes(service["cache"])
+        assert write == {"request_id": data["request_id"], "action": "stop",
+                         "timestamp": write["timestamp"]}
+
+    def test_a_stopped_display_gets_the_mailbox_before_it_is_started(
+            self, api_v3_client, service):
+        service["state"]["active"] = False
+        with patch(f"{CLIENT}.on_demand_start",
+                   side_effect=control_client.ControlError("no_socket")):
+            resp = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
+        assert resp.status_code == 200
+        assert service["calls"] == [("cache", MAILBOX), ("systemctl", "start")]
+
+    def test_the_socket_is_off_in_the_test_suite(self, api_v3_client, service):
+        # conftest's _hermetic_control_socket: a suite run on a device must
+        # not drive the live display.
+        assert os.environ[c.SOCKET_PATH_ENV] == "off"
+        data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
+        assert data["transport"] == "mailbox"
+        assert data["socket_error"] in ("disabled", "unsupported")   # Linux, Windows
+
+
+@pytest.mark.skipif(not c.socket_supported(), reason="AF_UNIX sockets are Linux/macOS only")
+class TestRealSocket:
+    @pytest.fixture
+    def live(self, monkeypatch):
+        import shutil
+        import tempfile
+        from src.ipc.server import ControlServer
+        d = tempfile.mkdtemp(prefix="lmipc-")
+        path = os.path.join(d, "control.sock")
+        server = ControlServer(path, status_provider=dict)
+        assert server.start()
+        monkeypatch.setenv(c.SOCKET_PATH_ENV, path)
+        yield server
+        server.close()
+        shutil.rmtree(d, ignore_errors=True)
+
+    def test_start_is_acked_and_queued(self, api_v3_client, service, live):
+        data = api_v3_client.post(START_URL, json={
+            "plugin_id": "weather", "duration": "30"}).get_json()["data"]
+        assert data["transport"] == "socket"
+        [cmd] = live.drain()
+        payload = cmd.as_on_demand_request()
+        assert payload["request_id"] == data["request_id"]
+        assert payload["plugin_id"] == "weather" and payload["duration"] == 30.0
+        assert _mailbox_writes(service["cache"]) == []
+
+    def test_stop_is_acked_and_queued(self, api_v3_client, service, live):
+        data = api_v3_client.post(STOP_URL, json={}).get_json()["data"]
+        assert data["transport"] == "socket"
+        assert [x.request_id for x in live.drain()] == [data["request_id"]]
+
+    def test_a_display_that_went_away_falls_back(self, api_v3_client, service, live):
+        live.close()
+        data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
+        assert data["transport"] == "mailbox" and data["socket_error"] == "no_socket"
+        assert len(_mailbox_writes(service["cache"])) == 1

test/test_cache_manager_memory_tier.py

@@ -40,11 +40,9 @@ def test_cleanup_and_stats_follow_a_replaced_component(cm):
     assert cm._memory_cache_component.get("stale") is None
     assert cm._memory_cache_component.get("fresh") == {"v": 1}
 
-    stats = cm.get_memory_cache_stats()
-    assert stats["size"] == 1
-    assert stats["max_size"] == 7
-    assert stats["cleanup_interval"] == 11.0
-    assert stats["usage_percent"] == pytest.approx(100 / 7)
+    with patch.object(cm.logger, "info") as info:
+        cm.log_memory_cache_stats()
+    assert "Size: 1/7 (14.3%)" in info.call_args[0][0]
 
 
 def test_periodic_cleanup_is_throttled_and_records_its_run(cm):
@@ -60,16 +58,16 @@ def test_periodic_cleanup_is_throttled_and_records_its_run(cm):
     before = time.time()
     cm.get_cached_data("missing")          # triggers the periodic sweep
     assert mem.size() == 0
-    assert cm.get_memory_cache_stats()["last_cleanup"] >= before
+    assert mem.get_stats()["last_cleanup"] >= before
 
 
-def test_stats_have_the_documented_shape(cm):
+def test_memory_stats_log_reads_the_live_tier(cm):
     cm.set("k", {"v": 1})
-    stats = cm.get_memory_cache_stats()
-    assert set(stats) == {"size", "max_size", "usage_percent",
-                          "last_cleanup", "cleanup_interval"}
-    assert stats["size"] == 1
-    assert stats["max_size"] == cm._memory_cache_component.max_size()
+    with patch.object(cm.logger, "info") as info:
+        cm.log_memory_cache_stats()
+    message = info.call_args[0][0]
+    assert f"Size: 1/{cm._memory_cache_component.max_size()}" in message
+    assert "Last cleanup:" in message
 
 
 def test_listing_the_cache_dir_does_not_hold_the_memory_lock(cm, tmp_path):

test/test_deprecation.py

@@ -20,31 +20,10 @@ from src.deprecation import deprecated
 
 REPO = Path(__file__).resolve().parents[1]
 
-#: Everything deprecated for removal in 3.8.0 (first announced for 3.7.0,
-#: which shipped with all of them still in place). docs/DEPRECATIONS_3.8.md
-#: says which are unused. Removing one of these, or deprecating another,
-#: should be a deliberate edit here too.
-DEPRECATED = {
-    "src.cache_manager.CacheManager": [
-        "has_data_changed", "update_cache", "setup_persistent_cache",
-        "get_sport_live_interval", "get_sport_key_from_cache_key",
-        "get_background_cached_data", "is_background_data_available",
-        "record_cache_hit", "record_cache_miss", "record_fetch_time",
-        "get_cache_metrics", "log_cache_metrics", "get_memory_cache_stats",
-    ],
-    "src.display_manager.DisplayManager": [
-        "draw_sun", "draw_cloud", "draw_rain", "draw_snow", "draw_weather_icon",
-        "draw_text_with_icons", "get_scrolling_stats",
-    ],
-    "src.font_manager.FontManager": [
-        "get_manager_fonts", "get_detected_fonts", "unregister_plugin_fonts",
-        "get_plugin_fonts", "set_override", "remove_override", "get_overrides",
-        "get_available_fonts", "get_size_tokens", "get_performance_stats",
-        "get_font_catalog", "add_font", "remove_font", "validate_font",
-    ],
-    "src.plugin_system.plugin_manager.PluginManager": ["get_enabled_plugins"],
-}
-
+#: Everything still deprecated. (The 35 methods deprecated for 3.8.0 were
+#: removed in it: docs/DEPRECATIONS_3.8.md found them unused.) Removing one
+#: of these, or deprecating another, should be a deliberate edit here too.
+#:
 #: Deprecated with Vegas participation, for removal in 3.9.0: core never read
 #: them (src.plugin_system.base_plugin.VEGAS_LEGACY_REMOVAL).
 DEPRECATED_3_9 = {
@@ -55,7 +34,7 @@ DEPRECATED_3_9 = {
 
 #: Every pinned marker: (class path, method) -> the release that removes it.
 PINNED = {(path, name): removal
-          for removal, table in (("3.8.0", DEPRECATED), ("3.9.0", DEPRECATED_3_9))
+          for removal, table in (("3.9.0", DEPRECATED_3_9),)
           for path, names in table.items() for name in names}
 
 
@@ -132,7 +111,57 @@ def test_usage_script_lists_exactly_the_pinned_markers(usage_script):
     assert found == {(path, name, removal) for (path, name), removal in PINNED.items()}
 
 
-def test_usage_script_tells_uses_from_name_collisions(usage_script, tmp_path):
+#: A stand-in core for the scanner tests below, so they keep working whichever
+#: real markers exist (the 3.8.0 ones they were written against are gone).
+FAKE_CORE = {
+    "src/cache_manager.py": """\
+        class CacheManager:
+            @deprecated("9.9.0", "use set()")
+            def update_cache(self, key, data):
+                pass
+        """,
+    "src/display_manager.py": """\
+        class DisplayManager:
+            @deprecated("9.9.0")
+            def draw_sun(self, x, y):
+                pass
+
+            @deprecated("9.9.0")
+            def draw_cloud(self, x, y):
+                pass
+
+            @deprecated("9.9.0")
+            def draw_rain(self, x, y):
+                self.draw_cloud(x, y)
+
+            @deprecated("9.9.0")
+            def draw_snow(self, x, y):
+                pass
+
+            @deprecated("9.9.0")
+            def get_scrolling_stats(self):
+                return {}
+        """,
+    "src/font_manager.py": """\
+        class FontManager:
+            @deprecated("9.9.0")
+            def add_font(self, path, name):
+                return True
+        """,
+}
+
+
+@pytest.fixture
+def fake_core(tmp_path):
+    root = tmp_path / "core"
+    for rel, source in FAKE_CORE.items():
+        path = root / rel
+        path.parent.mkdir(parents=True, exist_ok=True)
+        path.write_text(textwrap.dedent(source), encoding="utf-8")
+    return root
+
+
+def test_usage_script_tells_uses_from_name_collisions(usage_script, fake_core, tmp_path):
     """Calls on the owning object and overrides count; same-named methods of
     unrelated classes and hits in test files do not."""
     plugin = tmp_path / "demo"
@@ -163,7 +192,7 @@ def test_usage_script_tells_uses_from_name_collisions(usage_script, tmp_path):
             display_manager.draw_snow.assert_not_called()
         """), encoding="utf-8")
 
-    markers = usage_script.find_markers(REPO)
+    markers = usage_script.find_markers(fake_core)
     source = usage_script.Source("demo", "monorepo", plugin)
     usage_script.scan_tree(source, [plugin], plugin, markers, core=False)
     kinds = {key: sorted(("test " if h.test else "") + h.kind for h in hits)
@@ -186,11 +215,12 @@ def test_usage_script_tells_uses_from_name_collisions(usage_script, tmp_path):
     assert status["DisplayManager.draw_snow"][0] == "unused"     # a test mock only
 
 
-def test_usage_script_follows_calls_between_deprecated_core_methods(usage_script):
+def test_usage_script_follows_calls_between_deprecated_core_methods(usage_script, fake_core):
     """draw_rain calls draw_cloud; with no outside callers both are unused."""
-    markers = usage_script.find_markers(REPO)
-    core = usage_script.Source("core", "core", REPO)
-    usage_script.scan_tree(core, [REPO / "src" / "display_manager.py"], REPO, markers, core=True)
+    markers = usage_script.find_markers(fake_core)
+    core = usage_script.Source("core", "core", fake_core)
+    usage_script.scan_tree(core, [fake_core / "src" / "display_manager.py"], fake_core,
+                           markers, core=True)
     kinds = {h.kind for h in core.hits["DisplayManager.draw_cloud"]}
     assert kinds == {"internal"}
     assert usage_script.verdicts(markers, [core])["DisplayManager.draw_cloud"][0] == "unused"
@@ -219,11 +249,3 @@ def test_first_call_warns_and_logs_then_stays_quiet(fresh, caplog):
     assert caught[0].filename == __file__  # points at the caller
     assert sum("will be removed in LEDMatrix 9.9.9" in r.message for r in caplog.records) == 1
     assert old.__name__ == "old" and old.__doc__ == "Doc."
-
-
-def test_decorated_methods_still_work(fresh):
-    from src.font_manager import FontManager
-    fm = FontManager({})
-    with warnings.catch_warnings():
-        warnings.simplefilter("ignore")
-        assert fm.get_font_catalog() == fm.font_catalog

test/test_display_controller_schedule.py

@@ -6,7 +6,8 @@ test_display_controller_optimizations.py::TestScheduleMinuteGate already
 covers the once-per-minute gating; this file covers what it doesn't:
 midnight-crossing windows, mode selection (global / per-day / legacy
 inference), per-day disabled days, invalid time strings, unknown
-timezones, boundary equality, and the transition-tracking flags.
+timezones, the half-open [start, end) boundaries, on-demand ending during
+scheduled-off, and the transition-tracking flags.
 
 Both methods read only self.config and a handful of instance attributes,
 so a bare stub via object.__new__ (the test_display_controller_vegas_tick
@@ -41,12 +42,13 @@ def make_controller(config=None, *, normal_brightness=90):
 
 
 def at(time_str, day="monday"):
-    """Context manager patching the controller module's clock."""
+    """Patch the controller module's clock to ``HH:MM`` or ``HH:MM:SS``."""
     patcher = patch("src.display_controller.datetime")
     mock_dt = patcher.start()
     mock_dt.strptime = datetime.strptime
+    fmt = "%H:%M:%S" if time_str.count(":") == 2 else "%H:%M"
     mock_dt.now.return_value.time.return_value = (
-        datetime.strptime(time_str, "%H:%M").time())
+        datetime.strptime(time_str, fmt).time())
     mock_dt.now.return_value.strftime.return_value.lower.return_value = day
     mock_dt.now.return_value.hour = int(time_str.split(":")[0])
     mock_dt.now.return_value.minute = int(time_str.split(":")[1])
@@ -98,10 +100,13 @@ class TestScheduleWindows:
         assert check_at(dc, "20:00") is False
         assert check_at(dc, "08:59") is False
 
-    def test_boundaries_are_inclusive(self):
+    def test_window_is_half_open(self):
+        # [start, end): on from the start minute, off at the end minute.
         dc = make_controller(self._config("09:00", "17:00"))
-        assert check_at(dc, "09:00") is True   # now == start
-        assert check_at(dc, "17:00") is True   # now == end
+        assert check_at(dc, "08:59:59") is False
+        assert check_at(dc, "09:00") is True    # now == start
+        assert check_at(dc, "16:59:59") is True
+        assert check_at(dc, "17:00") is False   # now == end
 
     def test_midnight_crossing_window(self):
         # 21:00 -> 07:00: active late evening AND early morning, inactive
@@ -110,8 +115,11 @@ class TestScheduleWindows:
         assert check_at(dc, "23:00") is True
         assert check_at(dc, "03:00") is True
         assert check_at(dc, "12:00") is False
-        assert check_at(dc, "21:00") is True   # boundary
-        assert check_at(dc, "07:00") is True   # boundary
+        assert check_at(dc, "20:59:59") is False
+        assert check_at(dc, "21:00") is True    # start
+        assert check_at(dc, "00:00") is True    # midnight itself
+        assert check_at(dc, "06:59:59") is True
+        assert check_at(dc, "07:00") is False   # end
 
     def test_no_schedule_config_is_always_active(self):
         dc = make_controller({"timezone": "UTC"})
@@ -296,3 +304,185 @@ class TestDimSchedule:
         assert dc._was_dimmed is True
         dim_at(dc, "12:00")
         assert dc._was_dimmed is False
+
+
+def check_in_same_minute(dc, time_str, day="monday"):
+    """Run _check_schedule WITHOUT resetting the minute gate, as the loop does."""
+    p = at(time_str, day)
+    try:
+        dc._check_schedule()
+    finally:
+        p.stop()
+    return dc.is_display_active
+
+
+class TestEndMinuteBoundary:
+    """The panel goes off at the end minute whichever second the check runs.
+
+    The loop evaluates the schedule once per clock minute, on the first check
+    in it. With a closed [start, end] window only a check at hh:mm:00.000 saw
+    the end minute as inside, so the panel went off at the start or the end
+    of that minute depending on timing.
+    """
+
+    WINDOWS = {
+        "same_day": ({"start_time": "09:00", "end_time": "17:00"},
+                     "monday", "16:59", "17:00"),
+        "midnight_crossing": ({"start_time": "22:00", "end_time": "07:00"},
+                              "monday", "06:59", "07:00"),
+        "per_day_midnight_crossing": (
+            {"mode": "per-day", "start_time": "09:00", "end_time": "17:00",
+             "days": {"wednesday": {"enabled": True, "start_time": "22:00",
+                                    "end_time": "07:00"}}},
+            "wednesday", "06:59", "07:00"),
+    }
+
+    def _controller(self, window):
+        return make_controller({"schedule": {"enabled": True, **window},
+                                "timezone": "UTC"})
+
+    @pytest.mark.parametrize("name", sorted(WINDOWS))
+    @pytest.mark.parametrize("second", ["00", "59"])
+    def test_off_for_the_whole_end_minute(self, name, second):
+        window, day, last_on, end = self.WINDOWS[name]
+        dc = self._controller(window)
+        assert check_at(dc, f"{last_on}:59", day) is True
+        # First check of the end minute, at :00 or at :59.
+        assert check_in_same_minute(dc, f"{end}:{second}", day) is False
+
+    @pytest.mark.parametrize("name", sorted(WINDOWS))
+    def test_gated_minute_keeps_the_off_answer(self, name):
+        window, day, last_on, end = self.WINDOWS[name]
+        dc = self._controller(window)
+        assert check_at(dc, f"{last_on}:30", day) is True
+        assert check_in_same_minute(dc, f"{end}:00", day) is False
+        assert check_in_same_minute(dc, f"{end}:59", day) is False
+
+    @pytest.mark.parametrize("second", ["00", "59"])
+    def test_on_for_the_whole_start_minute(self, second):
+        dc = self._controller({"start_time": "22:00", "end_time": "07:00"})
+        assert check_at(dc, "21:59:59") is False
+        assert check_in_same_minute(dc, f"22:00:{second}") is True
+
+
+class TestOnDemandEndsDuringScheduledOff:
+    """An on-demand session ending in off hours blanks the panel at once,
+    not when the once-a-minute schedule check next runs."""
+
+    def _controller(self):
+        dc = make_controller({"schedule": {"enabled": True,
+                                           "start_time": "07:00",
+                                           "end_time": "23:00"},
+                              "timezone": "UTC"})
+        dc.on_demand_active = False
+        dc.on_demand_schedule_override = False
+        return dc
+
+    def _evaluate(self, dc, time_str):
+        p = at(time_str)
+        try:
+            dc._evaluate_schedule()
+        finally:
+            p.stop()
+        return dc.is_display_active
+
+    def test_session_end_in_off_hours_blanks_within_the_minute(self):
+        dc = self._controller()
+        assert self._evaluate(dc, "23:30:05") is False
+        dc.on_demand_active = True
+        assert self._evaluate(dc, "23:30:10") is True    # override
+        assert dc.on_demand_schedule_override is True
+        dc._reset_on_demand_fields()                      # expired or stopped
+        assert self._evaluate(dc, "23:30:40") is False   # same minute
+        assert dc.on_demand_schedule_override is False
+
+    def test_session_end_in_on_hours_stays_on(self):
+        dc = self._controller()
+        assert self._evaluate(dc, "12:00:05") is True
+        dc.on_demand_active = True
+        assert self._evaluate(dc, "12:00:10") is True
+        dc._reset_on_demand_fields()
+        assert self._evaluate(dc, "12:00:40") is True
+
+
+class TestOnDemandEndsDuringScheduledOffRunLoop:
+    """The same through the real run() loop (test/_run_loop_harness.py).
+
+    The harness clock starts at 22:59:30; the schedule below is off from
+    23:01 (t=90) until 23:05 (t=330). The sessions end mid-minute, so the
+    old behaviour (on until the next minute) would show as a gap."""
+
+    def _harness(self, tmp_path):
+        from test._run_loop_harness import FakePlugin, RunLoopHarness
+        h = RunLoopHarness(tmp_path, horizon=260)
+        h.config["schedule"] = {"enabled": True, "start_time": "23:05",
+                                "end_time": "23:01"}
+        h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+        h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+        return h
+
+    @staticmethod
+    def _first_off_after(trace, t):
+        return [row for row in trace["screens"]
+                if row[1] == "<off>" and row[0] >= t][0]
+
+    def test_expiry_blanks_at_once(self, tmp_path):
+        h = self._harness(tmp_path)
+        # 15 s from t=170 ends at t=185, 23:02:35.
+        h.on_demand_request(170, "x1", plugin_id="weather", duration=15)
+        trace = h.run()
+        session = [r for r in trace["screens"] if r[0] == 170.0][0]
+        assert session[1:4] == ["weather", 15.0, "on-demand-expired"]
+        assert self._first_off_after(trace, 170)[0] == 185.0
+
+    def test_stop_blanks_at_once(self, tmp_path):
+        h = self._harness(tmp_path)
+        h.on_demand_request(170, "x1", plugin_id="weather")
+        h.on_demand_request(181, "x2", action="stop")  # 23:02:31
+        trace = h.run()
+        assert 181.0 <= self._first_off_after(trace, 170)[0] <= 182.0
+
+
+class TestDimBoundaries:
+    """The dim schedule shares _in_window, so it is half-open too."""
+
+    def _config(self, start, end, **extra):
+        return {"dim_schedule": {"enabled": True, "start_time": start,
+                                 "end_time": end, "dim_brightness": 25,
+                                 **extra},
+                "timezone": "UTC"}
+
+    def test_same_day_dim_window_is_half_open(self):
+        dc = make_controller(self._config("13:00", "14:00"))
+        assert dim_at(dc, "12:59:59") == 90
+        assert dim_at(dc, "13:00") == 25
+        assert dim_at(dc, "13:59:59") == 25
+        assert dim_at(dc, "14:00:00") == 90
+        assert dim_at(dc, "14:00:59") == 90
+
+    def test_midnight_crossing_dim_window(self):
+        dc = make_controller(self._config("20:00", "07:00"))
+        assert dim_at(dc, "19:59:59") == 90
+        assert dim_at(dc, "20:00") == 25
+        assert dim_at(dc, "00:00") == 25
+        assert dim_at(dc, "06:59:59") == 25
+        assert dim_at(dc, "07:00:00") == 90
+        assert dim_at(dc, "07:00:59") == 90
+
+    def test_per_day_dim_end_minute(self):
+        dc = make_controller(self._config("20:00", "07:00", mode="per-day", days={
+            "friday": {"enabled": True, "start_time": "23:00",
+                       "end_time": "06:00"},
+        }))
+        assert dim_at(dc, "05:59:59", day="friday") == 25
+        assert dim_at(dc, "06:00:00", day="friday") == 90
+        assert dim_at(dc, "06:00:59", day="friday") == 90
+
+    def test_dim_end_minute_checked_late_in_the_minute(self):
+        dc = make_controller(self._config("20:00", "07:00"))
+        assert dim_at(dc, "06:59:30") == 25
+        p = at("07:00:59")  # first check of the end minute; gate not reset
+        try:
+            assert dc._check_dim_schedule() == 90
+        finally:
+            p.stop()

test/test_display_exception_breaker.py

@@ -0,0 +1,303 @@
+"""A plugin whose display() raises must count as a circuit-breaker failure.
+
+The first frame of every screen goes through PluginExecutor.execute_display,
+which catches whatever display() raises and reports False. run() read that
+False as "no content" and called record_success() on it, so a plugin that
+raised on every screen reset its own failure streak each time and the breaker
+never opened. It stayed in rotation, logging a traceback per screen, forever.
+
+These tests drive the real run() on a fake clock with the real executor and
+the real health tracker.
+"""
+
+import copy
+import threading
+import types
+from unittest.mock import MagicMock
+
+import pytest
+
+from src.exceptions import PluginError
+from src.plugin_system.plugin_executor import PluginExecutor
+from src.plugin_system.plugin_health import CircuitState, PluginHealthTracker
+
+THRESHOLD = 3
+COOLDOWN = 300.0
+SCREEN_SECONDS = 10
+
+
+class FakeClock:
+    """Moves only when the code under test sleeps; runs events as it passes them."""
+
+    def __init__(self, start=10_000.0):
+        self.t = start
+        self._events = []
+
+    def now(self):
+        return self.t
+
+    def sleep(self, seconds):
+        self.t += max(seconds, 0.0005)
+        while self._events and self._events[0][0] <= self.t:
+            _, fn = self._events.pop(0)
+            fn()
+
+    def after(self, seconds, fn):
+        self._events.append((self.t + seconds, fn))
+        self._events.sort(key=lambda e: e[0])
+
+    def module(self):
+        return types.SimpleNamespace(time=self.now, monotonic=self.now,
+                                     perf_counter=self.now, sleep=self.sleep)
+
+
+class _Stop(KeyboardInterrupt):
+    """Ends run(): it catches KeyboardInterrupt and cleans up."""
+
+
+class _Cache:
+    def __init__(self):
+        self.store = {}
+
+    def set(self, key, data, ttl=None, **kwargs):
+        self.store[key] = copy.deepcopy(data)
+
+    def get(self, key, max_age=None, memory_ttl=None, **kwargs):
+        return copy.deepcopy(self.store.get(key))
+
+
+class _Plugin:
+    """A static plugin. ``outcomes`` scripts each screen's first frame in
+    turn: True/False is returned, an exception instance is raised; once the
+    script runs out it returns True. Later frames of a screen return True.
+
+    A screen's first frame is the one PluginExecutor dispatches, on its own
+    thread; the render loop's later frames run on the calling thread.
+    """
+
+    needs_high_fps = False
+
+    def __init__(self, plugin_id, clock, outcomes=()):
+        self.plugin_id = plugin_id
+        self._clock = clock
+        self._outcomes = list(outcomes)
+        self._caller = threading.current_thread()
+        self.first_frames = []  # (time, outcome) of each executor dispatch
+        self.calls = 0
+
+    def display(self, force_clear=False):
+        if threading.current_thread() is self._caller:
+            return True  # a later frame of a screen that started fine
+        self.calls += 1
+        outcome = self._outcomes.pop(0) if self._outcomes else True
+        self.first_frames.append((self._clock.t, outcome))
+        if isinstance(outcome, BaseException):
+            raise outcome
+        return outcome
+
+
+@pytest.fixture
+def clock(monkeypatch):
+    c = FakeClock()
+    fake_time = c.module()
+    monkeypatch.setattr('src.display_controller.time', fake_time)
+    # The breaker's cooldown is wall-clock; put it on the same clock.
+    monkeypatch.setattr('src.plugin_system.plugin_health.time', fake_time)
+    return c
+
+
+@pytest.fixture
+def tracker():
+    return PluginHealthTracker(_Cache(), failure_threshold=THRESHOLD,
+                               cooldown_period=COOLDOWN)
+
+
+@pytest.fixture
+def controller(test_display_controller, clock, tracker):
+    c = test_display_controller
+    c._refresh_config_cache({'display': {'hardware': {'brightness': 90}}})
+    c.current_brightness = 90
+    c.is_display_active = True
+    c._check_wifi_status_message = MagicMock(return_value=None)
+    c._cleanup_expired_wifi_status = MagicMock()
+    c.cache_manager.get = MagicMock(return_value=None)
+    c.cache_manager.set = MagicMock()
+    c.cache_manager.delete = MagicMock()
+    c.display_manager.set_brightness = MagicMock(return_value=True)
+    c.display_manager.update_display = MagicMock()
+
+    pm = c.plugin_manager
+    # The real executor: its exception handling is what is under test.
+    pm.plugin_executor = PluginExecutor(default_timeout=5.0)
+    pm.health_tracker = tracker
+    locks = {}
+    pm.get_plugin_lock = lambda pid: locks.setdefault(pid, threading.Lock())
+    pm.record_display_hang = MagicMock()
+    pm.note_display_duration = MagicMock()
+    return c
+
+
+def _install(c, *plugins):
+    c.plugin_modes.clear()
+    c.mode_to_plugin_id.clear()
+    c.plugin_display_modes.clear()
+    for plugin in plugins:
+        c.plugin_modes[plugin.plugin_id] = plugin
+        c.mode_to_plugin_id[plugin.plugin_id] = plugin.plugin_id
+        c.plugin_display_modes[plugin.plugin_id] = [plugin.plugin_id]
+    c.available_modes = [p.plugin_id for p in plugins]
+    c.current_mode_index = 0
+    c.current_display_mode = c.available_modes[0]
+    c.config.setdefault('display', {})['display_durations'] = {
+        p.plugin_id: SCREEN_SECONDS for p in plugins}
+
+
+def _run_for(c, clock, seconds):
+    def stop():
+        raise _Stop()
+    clock.after(seconds, stop)
+    c.run()
+
+
+def _boom():
+    return RuntimeError("display() failed")
+
+
+class TestRaisingDisplayOpensTheBreaker:
+    def test_opens_at_the_threshold_and_leaves_rotation(self, controller, clock, tracker):
+        c = controller
+        crashy = _Plugin('crashy', clock, [_boom() for _ in range(100)])
+        good = _Plugin('good', clock)
+        _install(c, good, crashy)
+
+        _run_for(c, clock, 200)
+
+        state = tracker.get_health_state('crashy')
+        assert state['circuit_state'] == CircuitState.OPEN.value
+        assert state['consecutive_failures'] == THRESHOLD
+        assert state['last_error'].endswith("display() failed")
+        # Exactly THRESHOLD raises reached display(); the open breaker kept
+        # it out of every later pass, well inside the cooldown.
+        assert crashy.calls == THRESHOLD
+        # The display kept moving: the healthy plugin went on being shown.
+        assert len(good.first_frames) > THRESHOLD + 2
+        assert tracker.get_health_state('good')['consecutive_failures'] == 0
+
+    def test_back_in_rotation_after_the_cooldown(self, controller, clock, tracker):
+        c = controller
+        crashy = _Plugin('crashy', clock, [_boom() for _ in range(THRESHOLD)])
+        good = _Plugin('good', clock)
+        _install(c, good, crashy)
+
+        _run_for(c, clock, COOLDOWN + 100)
+
+        # Half-open after the cooldown, one attempt succeeded, circuit closed.
+        assert crashy.calls > THRESHOLD
+        state = tracker.get_health_state('crashy')
+        assert state['circuit_state'] == CircuitState.CLOSED.value
+        assert state['consecutive_failures'] == 0
+        opened_at = crashy.first_frames[THRESHOLD - 1][0]
+        retried_at = crashy.first_frames[THRESHOLD][0]
+        assert retried_at - opened_at >= COOLDOWN
+
+    def test_one_success_resets_the_streak(self, controller, clock, tracker):
+        c = controller
+        script = [_boom(), _boom(), True, _boom(), _boom(), True]
+        flaky = _Plugin('flaky', clock, script)
+        good = _Plugin('good', clock)
+        _install(c, good, flaky)
+
+        _run_for(c, clock, 6 * 2 * SCREEN_SECONDS + 5)
+
+        assert flaky.calls >= len(script)
+        assert [o if o is True else 'raised' for _, o in flaky.first_frames[:6]] == [
+            'raised', 'raised', True, 'raised', 'raised', True]
+        state = tracker.get_health_state('flaky')
+        assert state['circuit_state'] == CircuitState.CLOSED.value
+        assert state['consecutive_failures'] == 0
+        assert state['total_failures'] == 4
+
+    def test_no_content_is_still_not_a_failure(self, controller, clock, tracker):
+        c = controller
+        empty = _Plugin('empty', clock, [False] * 100)
+        good = _Plugin('good', clock)
+        _install(c, good, empty)
+
+        _run_for(c, clock, 200)
+
+        state = tracker.get_health_state('empty')
+        assert state['circuit_state'] == CircuitState.CLOSED.value
+        assert state.get('total_failures', 0) == 0
+        assert empty.calls > THRESHOLD
+
+
+class TestHangIsNotCountedTwice:
+    def test_a_timed_out_display_records_only_the_hang(self, controller, clock, tracker):
+        c = controller
+        c.plugin_manager.plugin_executor = PluginExecutor(default_timeout=0.05)
+        release = threading.Event()
+
+        class _Hung(_Plugin):
+            def display(self, force_clear=False):
+                self.calls += 1
+                release.wait(2.0)  # real time: outlives the executor's timeout
+                return True
+
+        hung = _Hung('hung', clock)
+        good = _Plugin('good', clock)
+        _install(c, hung, good)
+        failures = []
+        real_record_failure = tracker.record_failure
+        tracker.record_failure = lambda pid, err=None: (
+            failures.append(pid), real_record_failure(pid, err))
+        try:
+            _run_for(c, clock, SCREEN_SECONDS - 1)
+        finally:
+            release.set()
+
+        c.plugin_manager.record_display_hang.assert_called_once()
+        assert c.plugin_manager.record_display_hang.call_args.args[0] == 'hung'
+        # The hang path records the failure (PluginManager._record_hang); the
+        # dispatch adds neither a failure nor a success on top.
+        assert failures == []
+        assert tracker.get_health_state('hung').get('total_successes', 0) == 0
+
+
+class TestExecutorRaiseErrors:
+    def _plugin(self, display):
+        return types.SimpleNamespace(display=display)
+
+    def test_default_still_returns_false(self):
+        def display(force_clear=False):
+            raise ValueError("bad")
+        assert PluginExecutor().execute_display(
+            self._plugin(display), 'p', accepts_display_mode=False) is False
+
+    def test_raise_errors_surfaces_the_plugin_error(self):
+        def display(force_clear=False):
+            raise ValueError("bad")
+        with pytest.raises(PluginError) as info:
+            PluginExecutor().execute_display(
+                self._plugin(display), 'p', accepts_display_mode=False,
+                raise_errors=True)
+        assert isinstance(info.value.__cause__, ValueError)
+
+    def test_raise_errors_leaves_a_timeout_as_false(self):
+        done = threading.Event()
+
+        def display(force_clear=False):
+            done.wait(1.0)
+            return True
+        try:
+            assert PluginExecutor(default_timeout=0.05).execute_display(
+                self._plugin(display), 'p', accepts_display_mode=False,
+                raise_errors=True) is False
+        finally:
+            done.set()
+
+    def test_raise_errors_passes_results_through(self):
+        executor = PluginExecutor()
+        for value, expected in ((True, True), (False, False), (None, True)):
+            assert executor.execute_display(
+                self._plugin(lambda force_clear=False, v=value: v), 'p',
+                accepts_display_mode=False, raise_errors=True) is expected

test/test_display_pending_changes.py

@@ -477,7 +477,7 @@ class TestRunLoopBlanksWhenVegasHandsBack:
         c._cleanup_expired_wifi_status = MagicMock()
         c._refresh_config_cache({
             'display': {'hardware': {'brightness': 90}},
-            'schedule': {'enabled': True, 'start_time': '07:00', 'end_time': '22:59'},
+            'schedule': {'enabled': True, 'start_time': '07:00', 'end_time': '23:00'},
         })
         c.vegas_coordinator = vegas_coordinator(c)
         c.vegas_coordinator._pending_config_update = False
@@ -518,7 +518,7 @@ class TestRunLoopBlanksWhenVegasHandsBack:
         c._refresh_config_cache({
             'display': {'hardware': {'brightness': 90},
                         'display_durations': {'ticker': 120}},
-            'schedule': {'enabled': True, 'start_time': '07:00', 'end_time': '22:59'},
+            'schedule': {'enabled': True, 'start_time': '07:00', 'end_time': '23:00'},
         })
         c.plugin_manager.plugin_executor.execute_display.side_effect = (
             lambda target, plugin_id, force_clear=False, display_mode=None, **kw:

test/test_fetch_service.py

@@ -0,0 +1,878 @@
+"""The core fetch service: merging, host budgets, conditional GET, counters.
+
+No network: every request goes to a fake transport -- a real
+``requests.Session`` subclass whose ``get`` answers from a handler -- so the
+service sees real ``requests.Response`` objects, real header merging and real
+adapters, and nothing leaves the machine. Clocks and sleeps are injected.
+
+What callers already rely on (return values, exceptions, retries) is pinned
+by the existing suites, which run unchanged through the service:
+test_api_helper.py, test_background_data_service*.py,
+test_background_fetch_dedupe.py, test_base_odds_manager.py,
+test_odds_request_budget.py, test_espn_dates.py and test_sports_fetch.py.
+"""
+
+import importlib.util
+import json
+import threading
+import time
+
+import pytest
+import requests
+from requests.structures import CaseInsensitiveDict
+from urllib3.util.retry import Retry
+
+from src.common import fetch_service as fs
+from src.common.fetch_service import (
+    FetchService,
+    FetchStatsPublisher,
+    TokenBucket,
+    current_plugin_id,
+    plugin_scope,
+    read_fetch_stats,
+    register_plugin_directory,
+    unregister_plugin_directory,
+)
+
+
+# --- fakes -----------------------------------------------------------------------
+
+class FakeClock:
+    def __init__(self, start=1000.0):
+        self.t = start
+        self.sleeps = []
+
+    def now(self):
+        return self.t
+
+    def sleep(self, seconds):
+        self.sleeps.append(seconds)
+        self.t += seconds
+
+    def advance(self, seconds):
+        self.t += seconds
+
+
+def make_response(status=200, body=b'{"ok": 1}', headers=None, url="https://api.test/x"):
+    response = requests.Response()
+    response.status_code = status
+    response._content = body
+    response.headers = CaseInsensitiveDict(headers or {})
+    response.url = url
+    response.encoding = "utf-8"
+    response.reason = "OK" if status < 400 else "Error"
+    return response
+
+
+class FakeSession(requests.Session):
+    """A Session whose get() answers from ``handler(url, kwargs)``."""
+
+    def __init__(self, handler=None, gate=None):
+        super().__init__()
+        self.handler = handler or (lambda url, kwargs: make_response(url=url))
+        self.gate = gate
+        self.calls = []
+        self.started = threading.Event()
+        self._calls_lock = threading.Lock()
+
+    def get(self, url, **kwargs):
+        with self._calls_lock:
+            self.calls.append((url, kwargs))
+        self.started.set()
+        if self.gate is not None:
+            assert self.gate.wait(5), "test gate never opened"
+        return self.handler(url, kwargs)
+
+
+@pytest.fixture
+def clock():
+    return FakeClock()
+
+
+@pytest.fixture
+def service(clock):
+    return FetchService({"rate_limits": {}}, clock=clock.now, sleep=clock.sleep)
+
+
+@pytest.fixture
+def global_service(monkeypatch, clock):
+    """A fresh process-wide service, for code that calls fetch_get()."""
+    svc = FetchService({"rate_limits": {}}, clock=clock.now, sleep=clock.sleep)
+    monkeypatch.setattr(fs, "_service", svc)
+    return svc
+
+
+def _counters(svc, plugin=None, host=None):
+    snap = svc.snapshot()
+    if plugin is not None:
+        return snap["plugins"].get(plugin, {})
+    if host is not None:
+        return snap["hosts"].get(host, {})
+    return snap["totals"]
+
+
+# --- the call itself is unchanged -----------------------------------------------------
+
+class TestPassThrough:
+
+    def test_session_get_sees_exactly_the_callers_arguments(self, service):
+        session = FakeSession()
+        response = service.get(session, "https://api.test/x", params={"a": 1},
+                               headers={"X-Y": "z"}, timeout=7)
+        assert session.calls == [("https://api.test/x",
+                                  {"params": {"a": 1}, "headers": {"X-Y": "z"}, "timeout": 7})]
+        assert response.json() == {"ok": 1}
+
+    def test_no_kwargs_the_caller_did_not_pass(self, service):
+        session = FakeSession()
+        service.get(session, "https://api.test/x", timeout=5)
+        assert session.calls[0][1] == {"timeout": 5}
+
+    def test_the_transport_exception_reaches_the_caller_unchanged(self, service):
+        boom = requests.ConnectionError("down")
+
+        def handler(url, kwargs):
+            raise boom
+
+        with pytest.raises(requests.ConnectionError) as caught:
+            service.get(FakeSession(handler), "https://api.test/x")
+        assert caught.value is boom
+
+    def test_an_http_error_response_is_returned_not_raised(self, service):
+        session = FakeSession(lambda url, kw: make_response(503, b"busy"))
+        response = service.get(session, "https://api.test/x")
+        assert response.status_code == 503
+        with pytest.raises(requests.HTTPError):
+            response.raise_for_status()
+
+    def test_disabled_is_a_plain_session_get(self, clock):
+        svc = FetchService({"enabled": False, "rate_limits": {"api.test": {"per_second": 1, "burst": 1}}},
+                           clock=clock.now, sleep=clock.sleep)
+        session = FakeSession()
+        for _ in range(3):
+            svc.get(session, "https://api.test/x")
+        assert len(session.calls) == 3
+        assert clock.sleeps == []
+        assert _counters(svc)["requests"] == 0
+
+    def test_a_test_double_session_still_works(self, service):
+        from unittest.mock import MagicMock
+        session = MagicMock()
+        session.get.return_value.json.return_value = {"a": 1}
+        assert service.get(session, "https://api.test/x", timeout=3).json() == {"a": 1}
+        session.get.assert_called_once_with("https://api.test/x", timeout=3)
+
+    def test_session_none_uses_the_pooled_session_for_the_host(self, service, monkeypatch):
+        seen = []
+        monkeypatch.setattr(requests.Session, "get",
+                            lambda self, url, **kw: seen.append(self) or make_response())
+        service.get(None, "https://a.test/1")
+        service.get(None, "https://a.test/2")
+        service.get(None, "https://b.test/1")
+        assert seen[0] is seen[1] is service.session_for("https://a.test/")
+        assert seen[2] is not seen[0]
+
+
+# --- single-flight ------------------------------------------------------------------------
+
+def _wait_for_waiters(svc, count, timeout=5):
+    deadline = time.monotonic() + timeout
+    while time.monotonic() < deadline:
+        with svc._lock:
+            flights = list(svc._inflight.values())
+        if flights and flights[0].waiters >= count:
+            return
+        time.sleep(0.005)
+    raise AssertionError(f"only {flights[0].waiters if flights else 0} of {count} callers joined")
+
+
+class TestSingleFlight:
+
+    def test_concurrent_identical_gets_go_out_once(self, service):
+        gate = threading.Event()
+        session = FakeSession(gate=gate)
+        results = []
+
+        def call():
+            results.append(service.get(session, "https://api.test/x",
+                                       params={"d": "1"}, timeout=5))
+
+        threads = [threading.Thread(target=call) for _ in range(5)]
+        threads[0].start()
+        assert session.started.wait(5)
+        for t in threads[1:]:
+            t.start()
+        _wait_for_waiters(service, 4)
+        gate.set()
+        for t in threads:
+            t.join(5)
+
+        assert len(session.calls) == 1
+        assert len(results) == 5
+        assert all(r.json() == {"ok": 1} for r in results)
+        # Each caller gets its own Response object to mutate.
+        assert len({id(r) for r in results}) == 5
+        totals = _counters(service)
+        assert totals["requests"] == 1
+        assert totals["merged"] == 4
+
+    def test_merged_callers_get_the_leaders_exception(self, service):
+        gate = threading.Event()
+
+        def handler(url, kwargs):
+            raise requests.Timeout("slow")
+
+        session = FakeSession(handler, gate=gate)
+        errors = []
+
+        def call():
+            try:
+                service.get(session, "https://api.test/x", timeout=5)
+            except requests.Timeout as err:
+                errors.append(err)
+
+        threads = [threading.Thread(target=call) for _ in range(3)]
+        threads[0].start()
+        assert session.started.wait(5)
+        for t in threads[1:]:
+            t.start()
+        _wait_for_waiters(service, 2)
+        gate.set()
+        for t in threads:
+            t.join(5)
+
+        assert len(session.calls) == 1
+        assert len(errors) == 3
+        totals = _counters(service)
+        assert totals["errors"] == 1 and totals["merged"] == 2
+
+    @pytest.mark.parametrize("second", [
+        {"params": {"d": "2"}},                       # another query
+        {"params": {"d": "1"}, "timeout": 9},         # another timeout
+        {"params": {"d": "1"}, "headers": {"Accept": "text/plain"}},  # another representation
+    ])
+    def test_requests_that_could_answer_differently_are_not_merged(self, service, second):
+        gate = threading.Event()
+        session = FakeSession(gate=gate)
+        first = threading.Thread(target=lambda: service.get(
+            session, "https://api.test/x", params={"d": "1"}, timeout=5))
+        first.start()
+        assert session.started.wait(5)
+        other = threading.Thread(target=lambda: service.get(
+            session, "https://api.test/x", **{"timeout": 5, **second}))
+        other.start()
+        deadline = time.monotonic() + 5
+        while len(session.calls) < 2 and time.monotonic() < deadline:
+            time.sleep(0.005)
+        gate.set()
+        first.join(5)
+        other.join(5)
+        assert len(session.calls) == 2
+        assert _counters(service)["merged"] == 0
+
+    def test_different_retry_policies_are_not_merged(self, service):
+        gate = threading.Event()
+        retrying = FakeSession(gate=gate)
+        retrying.mount("https://", requests.adapters.HTTPAdapter(max_retries=Retry(total=3)))
+        plain = FakeSession(gate=gate)
+        a = threading.Thread(target=lambda: service.get(retrying, "https://api.test/x"))
+        a.start()
+        assert retrying.started.wait(5)
+        b = threading.Thread(target=lambda: service.get(plain, "https://api.test/x"))
+        b.start()
+        assert plain.started.wait(5)
+        gate.set()
+        a.join(5)
+        b.join(5)
+        assert len(retrying.calls) == len(plain.calls) == 1
+
+    def test_sessions_with_the_same_policy_and_headers_share_a_flight(self, service):
+        gate = threading.Event()
+        one, two = FakeSession(gate=gate), FakeSession(gate=gate)
+        a = threading.Thread(target=lambda: service.get(one, "https://api.test/x", timeout=5))
+        a.start()
+        assert one.started.wait(5)
+        b = threading.Thread(target=lambda: service.get(two, "https://api.test/x", timeout=5))
+        b.start()
+        _wait_for_waiters(service, 1)
+        gate.set()
+        a.join(5)
+        b.join(5)
+        assert len(one.calls) == 1 and two.calls == []
+
+    def test_a_session_with_cookies_only_merges_with_itself(self, service):
+        gate = threading.Event()
+        cookied, plain = FakeSession(gate=gate), FakeSession(gate=gate)
+        cookied.cookies.set("sid", "secret")
+        a = threading.Thread(target=lambda: service.get(cookied, "https://api.test/x"))
+        a.start()
+        assert cookied.started.wait(5)
+        b = threading.Thread(target=lambda: service.get(plain, "https://api.test/x"))
+        b.start()
+        assert plain.started.wait(5)
+        gate.set()
+        a.join(5)
+        b.join(5)
+        assert len(cookied.calls) == len(plain.calls) == 1
+
+    def test_sequential_identical_gets_each_go_out(self, service):
+        session = FakeSession()
+        service.get(session, "https://api.test/x")
+        service.get(session, "https://api.test/x")
+        assert len(session.calls) == 2
+
+    def test_streamed_requests_are_never_merged(self, service):
+        gate = threading.Event()
+        session = FakeSession(gate=gate)
+        a = threading.Thread(target=lambda: service.get(session, "https://api.test/x", stream=True))
+        a.start()
+        assert session.started.wait(5)
+        b = threading.Thread(target=lambda: service.get(session, "https://api.test/x", stream=True))
+        b.start()
+        deadline = time.monotonic() + 5
+        while len(session.calls) < 2 and time.monotonic() < deadline:
+            time.sleep(0.005)
+        gate.set()
+        a.join(5)
+        b.join(5)
+        assert len(session.calls) == 2
+
+
+# --- token buckets ---------------------------------------------------------------------------
+
+class TestTokenBucket:
+
+    def test_burst_then_one_token_per_interval(self, clock):
+        bucket = TokenBucket(per_second=2, burst=3, clock=clock.now)
+        assert [bucket.reserve(10)[0] for _ in range(3)] == [0.0, 0.0, 0.0]
+        assert bucket.reserve(10) == (0.5, False)
+        assert bucket.reserve(10) == (1.0, False)
+
+    def test_tokens_refill_with_time_up_to_the_burst(self, clock):
+        bucket = TokenBucket(per_second=2, burst=3, clock=clock.now)
+        for _ in range(3):
+            bucket.reserve(10)
+        clock.advance(100)
+        assert [bucket.reserve(10)[0] for _ in range(3)] == [0.0, 0.0, 0.0]
+        assert bucket.reserve(10)[0] == 0.5
+
+    def test_a_wait_is_capped_at_max_wait(self, clock):
+        bucket = TokenBucket(per_second=1, burst=1, clock=clock.now)
+        bucket.reserve(0.5)
+        assert bucket.reserve(0.5) == (0.5, True)
+        # The debt never runs further than max_wait either.
+        assert bucket.reserve(0.5) == (0.5, True)
+        clock.advance(10)
+        assert bucket.reserve(0.5) == (0.0, False)
+
+
+class TestHostBudgets:
+
+    def test_requests_past_the_budget_wait(self, clock):
+        svc = FetchService({"rate_limits": {"api.test": {"per_second": 1, "burst": 2}},
+                            "max_wait_seconds": 10}, clock=clock.now, sleep=clock.sleep)
+        session = FakeSession()
+        for _ in range(4):
+            svc.get(session, "https://api.test/x")
+        assert clock.sleeps == [1.0, 1.0]
+        host = _counters(svc, host="api.test")
+        assert host["throttled"] == 2 and host["wait_seconds"] == 2.0
+        assert host["requests"] == 4
+
+    def test_other_hosts_are_not_throttled(self, clock):
+        svc = FetchService({"rate_limits": {"api.test": {"per_second": 1, "burst": 1}}},
+                           clock=clock.now, sleep=clock.sleep)
+        session = FakeSession()
+        for _ in range(5):
+            svc.get(session, "https://elsewhere.test/x")
+        assert clock.sleeps == []
+
+    def test_each_host_has_its_own_bucket(self, clock):
+        svc = FetchService({"rate_limits": {"*.espn.com": {"per_second": 1, "burst": 1}},
+                            "max_wait_seconds": 10}, clock=clock.now, sleep=clock.sleep)
+        session = FakeSession()
+        svc.get(session, "https://site.api.espn.com/a")
+        svc.get(session, "https://sports.core.api.espn.com/a")
+        assert clock.sleeps == []
+        svc.get(session, "https://site.api.espn.com/a")
+        assert clock.sleeps == [1.0]
+
+    def test_wildcard_matches_the_bare_domain_and_subdomains_only(self, clock):
+        svc = FetchService({"rate_limits": {"*.espn.com": {"per_second": 5, "burst": 9}}},
+                           clock=clock.now, sleep=clock.sleep)
+        assert svc._limit_for("espn.com") == (5.0, 9.0)
+        assert svc._limit_for("site.api.espn.com") == (5.0, 9.0)
+        assert svc._limit_for("notespn.com") is None
+
+    def test_the_default_budget_covers_espn_and_a_cold_season_burst(self, clock):
+        svc = FetchService(clock=clock.now, sleep=clock.sleep)
+        session = FakeSession()
+        for _ in range(200):
+            svc.get(session, "https://site.api.espn.com/x")
+        assert clock.sleeps == []
+        svc.get(session, "https://site.api.espn.com/x")
+        assert clock.sleeps == [pytest.approx(0.05)]
+        svc.get(session, "https://api.example.org/x")
+        assert len(clock.sleeps) == 1
+
+    def test_zero_per_second_removes_a_budget(self, clock):
+        svc = FetchService({"rate_limits": {"*.espn.com": {"per_second": 0, "burst": 1}}},
+                           clock=clock.now, sleep=clock.sleep)
+        session = FakeSession()
+        for _ in range(5):
+            svc.get(session, "https://site.api.espn.com/x")
+        assert clock.sleeps == []
+
+    def test_a_merged_caller_spends_no_token(self, clock):
+        svc = FetchService({"rate_limits": {"api.test": {"per_second": 1, "burst": 1}},
+                            "max_wait_seconds": 10}, clock=clock.now, sleep=clock.sleep)
+        gate = threading.Event()
+        session = FakeSession(gate=gate)
+        a = threading.Thread(target=lambda: svc.get(session, "https://api.test/x"))
+        a.start()
+        assert session.started.wait(5)
+        b = threading.Thread(target=lambda: svc.get(session, "https://api.test/x"))
+        b.start()
+        _wait_for_waiters(svc, 1)
+        gate.set()
+        a.join(5)
+        b.join(5)
+        assert clock.sleeps == []
+
+
+# --- conditional GET ----------------------------------------------------------------------------
+
+class Versioned:
+    """A server with one resource and an ETag, honouring If-None-Match."""
+
+    def __init__(self, validator="etag"):
+        self.version = 1
+        self.validator = validator
+        self.seen = []
+
+    def body(self):
+        return json.dumps({"version": self.version}).encode()
+
+    def tag(self):
+        if self.validator == "etag":
+            return {"ETag": f'"v{self.version}"'}
+        return {"Last-Modified": f"Thu, 01 Oct 2026 00:00:0{self.version} GMT"}
+
+    def __call__(self, url, kwargs):
+        headers = CaseInsensitiveDict(kwargs.get("headers") or {})
+        self.seen.append(dict(headers))
+        current = self.tag()
+        if (headers.get("If-None-Match") == current.get("ETag") and "ETag" in current) or \
+                (headers.get("If-Modified-Since") == current.get("Last-Modified")
+                 and "Last-Modified" in current):
+            return make_response(304, b"", headers={**current, "Date": "now"}, url=url)
+        return make_response(200, self.body(),
+                             headers={**current, "Content-Type": "application/json"}, url=url)
+
+
+class TestConditionalGet:
+
+    @pytest.mark.parametrize("validator,header", [("etag", "If-None-Match"),
+                                                  ("last-modified", "If-Modified-Since")])
+    def test_a_304_returns_the_stored_body_as_a_200(self, service, validator, header):
+        server = Versioned(validator)
+        session = FakeSession(server)
+        first = service.get(session, "https://api.test/x", timeout=5)
+        second = service.get(session, "https://api.test/x", timeout=5)
+
+        assert header not in server.seen[0]
+        assert header in server.seen[1]
+        assert second.status_code == 200
+        assert second.json() == first.json() == {"version": 1}
+        assert second.headers["Content-Type"] == "application/json"
+        second.raise_for_status()
+        totals = _counters(service)
+        assert totals["requests"] == 2
+        assert totals["not_modified"] == 1
+        assert totals["bytes"] == len(server.body())  # the 304 carried none
+
+    def test_a_changed_resource_is_fetched_and_stored_again(self, service):
+        server = Versioned()
+        session = FakeSession(server)
+        service.get(session, "https://api.test/x")
+        server.version = 2
+        changed = service.get(session, "https://api.test/x")
+        assert changed.json() == {"version": 2}
+        again = service.get(session, "https://api.test/x")
+        assert again.json() == {"version": 2}
+        assert server.seen[2]["If-None-Match"] == '"v2"'
+
+    def test_no_validators_no_conditional_request(self, service):
+        session = FakeSession()  # answers 200 with no ETag/Last-Modified
+        service.get(session, "https://api.test/x", headers={"A": "1"})
+        service.get(session, "https://api.test/x", headers={"A": "1"})
+        assert session.calls[1][1] == {"headers": {"A": "1"}}
+        assert service.snapshot()["validators"]["entries"] == 0
+
+    def test_a_200_without_validators_drops_the_stored_one(self, service):
+        server = Versioned()
+        session = FakeSession(server)
+        service.get(session, "https://api.test/x")
+        session.handler = lambda url, kw: make_response(200, b'{"new": 1}', url=url)
+        service.get(session, "https://api.test/x")
+        assert service.snapshot()["validators"]["entries"] == 0
+
+    def test_a_callers_own_conditional_request_is_left_alone(self, service):
+        server = Versioned()
+        session = FakeSession(server)
+        service.get(session, "https://api.test/x")
+        raw = service.get(session, "https://api.test/x", headers={"If-None-Match": '"v1"'})
+        assert raw.status_code == 304
+
+    def test_validators_are_per_representation(self, service):
+        server = Versioned()
+        session = FakeSession(server)
+        service.get(session, "https://api.test/x", params={"d": "1"})
+        service.get(session, "https://api.test/x", params={"d": "2"})
+        assert "If-None-Match" not in server.seen[1]
+
+    def test_a_body_too_big_for_the_store_is_not_kept(self, clock):
+        svc = FetchService({"rate_limits": {}, "validator_store": {"max_entry_bytes": 4}},
+                           clock=clock.now, sleep=clock.sleep)
+        server = Versioned()
+        session = FakeSession(server)
+        svc.get(session, "https://api.test/x")
+        svc.get(session, "https://api.test/x")
+        assert "If-None-Match" not in server.seen[1]
+
+    def test_the_store_evicts_least_recently_used_past_its_budget(self, clock):
+        svc = FetchService({"rate_limits": {}, "validator_store": {"max_entries": 2}},
+                           clock=clock.now, sleep=clock.sleep)
+        session = FakeSession(Versioned())
+        for path in ("a", "b", "c"):
+            svc.get(session, f"https://api.test/{path}")
+        assert svc.snapshot()["validators"]["entries"] == 2
+
+    def test_off_switch(self, clock):
+        svc = FetchService({"rate_limits": {}, "conditional_get": False},
+                           clock=clock.now, sleep=clock.sleep)
+        server = Versioned()
+        session = FakeSession(server)
+        svc.get(session, "https://api.test/x")
+        svc.get(session, "https://api.test/x")
+        assert "If-None-Match" not in server.seen[1]
+
+
+# --- counters and caller identity ----------------------------------------------------------------
+
+class TestCounters:
+
+    def test_per_plugin_and_per_host(self, service):
+        session = FakeSession()
+        with plugin_scope("weather"):
+            service.get(session, "https://api.weather.test/now")
+            service.get(session, "https://api.weather.test/later")
+        service.get(session, "https://site.api.espn.com/x")
+        snap = service.snapshot()
+        assert snap["plugins"]["weather"]["requests"] == 2
+        assert snap["plugins"]["weather"]["hosts"] == {"api.weather.test": 2}
+        assert snap["plugins"]["core"]["requests"] == 1
+        assert snap["hosts"]["api.weather.test"]["requests"] == 2
+        assert snap["hosts"]["site.api.espn.com"]["requests"] == 1
+        assert snap["totals"]["bytes"] == 3 * len(b'{"ok": 1}')
+
+    def test_errors_and_http_errors(self, service):
+        def handler(url, kwargs):
+            if url.endswith("/down"):
+                raise requests.ConnectionError("down")
+            return make_response(404, b"nope", url=url)
+
+        session = FakeSession(handler)
+        with pytest.raises(requests.ConnectionError):
+            service.get(session, "https://api.test/down")
+        service.get(session, "https://api.test/missing")
+        totals = _counters(service)
+        assert totals["requests"] == 2
+        assert totals["errors"] == 1
+        assert totals["http_errors"] == 1
+
+    def test_every_change_bumps_the_change_count(self, service):
+        before = service.change_count
+        service.get(FakeSession(), "https://api.test/x")
+        assert service.change_count > before
+
+    def test_post_is_counted_and_never_merged(self, service):
+        class PostSession(FakeSession):
+            def post(self, url, **kwargs):
+                self.calls.append((url, kwargs))
+                return make_response(201, b"{}", url=url)
+
+        session = PostSession()
+        with plugin_scope("poster"):
+            response = service.post(session, "https://api.test/x", json={"a": 1})
+        assert response.status_code == 201
+        assert session.calls == [("https://api.test/x", {"json": {"a": 1}})]
+        assert _counters(service, plugin="poster")["requests"] == 1
+
+
+class TestCallerIdentity:
+
+    def test_scope_wins_and_nests(self):
+        assert current_plugin_id() is None
+        with plugin_scope("outer"):
+            assert current_plugin_id() == "outer"
+            with plugin_scope("inner"):
+                assert current_plugin_id() == "inner"
+            with plugin_scope(None):
+                assert current_plugin_id() == "outer"
+        assert current_plugin_id() is None
+
+    def test_a_plugins_own_thread_is_named_by_its_source_directory(self, tmp_path, service):
+        plugin_dir = tmp_path / "my-plugin"
+        plugin_dir.mkdir()
+        (plugin_dir / "fetcher.py").write_text(
+            "import threading\n"
+            "def fetch_in_thread(service, session, url):\n"
+            "    t = threading.Thread(target=lambda: service.get(session, url))\n"
+            "    t.start()\n"
+            "    t.join(5)\n",
+            encoding="utf-8")
+        spec = importlib.util.spec_from_file_location("_fs_test_fetcher", plugin_dir / "fetcher.py")
+        module = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(module)
+        register_plugin_directory("my-plugin", plugin_dir)
+        try:
+            module.fetch_in_thread(service, FakeSession(), "https://api.test/x")
+        finally:
+            unregister_plugin_directory("my-plugin")
+        assert _counters(service, plugin="my-plugin")["requests"] == 1
+        assert "core" not in service.snapshot()["plugins"]
+
+    def test_the_executor_scopes_a_plugin_operation(self):
+        from src.plugin_system.plugin_executor import PluginExecutor
+        seen = PluginExecutor().execute_with_timeout(current_plugin_id, plugin_id="clock")
+        assert seen == "clock"
+
+    def test_background_fetches_count_against_the_submitter(self, global_service):
+        from unittest.mock import MagicMock
+        from src.background_data_service import BackgroundDataService
+
+        cache = MagicMock()
+        cache.get.return_value = None
+        bds = BackgroundDataService(cache, max_workers=1, request_timeout=5)
+        bds.session = FakeSession(lambda url, kw: make_response(body=b'{"events": []}', url=url))
+        try:
+            with plugin_scope("football-scoreboard"):
+                request_id = bds.submit_fetch_request(
+                    "nfl", 2026, "https://site.api.espn.com/apis/site/v2/sports/football/nfl/scoreboard",
+                    cache_key="fs_test_nfl", params={"dates": "2026"})
+            deadline = time.monotonic() + 5
+            while not bds.is_request_complete(request_id) and time.monotonic() < deadline:
+                time.sleep(0.01)
+            assert bds.get_result(request_id).success
+        finally:
+            bds.shutdown(wait=True)
+        assert _counters(global_service, plugin="football-scoreboard")["requests"] == 1
+
+    def test_espn_chunks_on_worker_threads_count_against_the_caller(self, global_service):
+        from src.common.espn_dates import espn_date_chunks, fetch_espn_date_chunks, parse_espn_date_range
+
+        session = FakeSession(lambda url, kw: make_response(body=b'{"events": []}', url=url))
+        dates = "20260801-20261015"
+        with plugin_scope("baseball-scoreboard"):
+            fetch_espn_date_chunks(session, "https://site.api.espn.com/s/scoreboard",
+                                   params={"dates": dates})
+        chunks = len(espn_date_chunks(*parse_espn_date_range(dates)))
+        assert chunks > 1
+        assert len(session.calls) == chunks
+        assert _counters(global_service, plugin="baseball-scoreboard")["requests"] == chunks
+        assert "core" not in global_service.snapshot()["plugins"]
+
+    def test_api_helper_goes_through_the_service(self, global_service):
+        from src.common.api_helper import APIHelper
+
+        helper = APIHelper()
+        helper.set_rate_limit(0)
+        helper.session = FakeSession(lambda url, kw: make_response(body=b'{"a": 1}', url=url))
+        with plugin_scope("nfl-draft"):
+            assert helper.get("https://api.test/x") == {"a": 1}
+        assert _counters(global_service, plugin="nfl-draft")["requests"] == 1
+
+    def test_odds_go_through_the_service(self, global_service):
+        from unittest.mock import MagicMock
+        from src.base_odds_manager import BaseOddsManager
+
+        cache = MagicMock()
+        cache.get_with_auto_strategy.return_value = None
+        manager = BaseOddsManager(cache)
+        manager.session = FakeSession(
+            lambda url, kw: make_response(body=b'{"count": 0, "items": []}', url=url))
+        with plugin_scope("odds-ticker"):
+            assert manager.get_odds("football", "nfl", "401") is None
+        assert manager.session.calls[0][1] == {"timeout": manager.request_timeout}
+        assert _counters(global_service, plugin="odds-ticker")["requests"] == 1
+
+
+# --- pooling -------------------------------------------------------------------------------------------
+
+class TestConnectionPool:
+
+    def test_core_sessions_with_one_policy_share_one_adapter(self, global_service):
+        from unittest.mock import MagicMock
+        from src.background_data_service import BackgroundDataService
+        from src.base_odds_manager import BaseOddsManager
+
+        odds_a = BaseOddsManager(MagicMock()).session.get_adapter("https://x.test")
+        odds_b = BaseOddsManager(MagicMock()).session.get_adapter("https://x.test")
+        bds = BackgroundDataService(MagicMock(), max_workers=1)
+        try:
+            assert odds_a is odds_b is bds.session.get_adapter("https://x.test")
+            assert odds_a.max_retries.total == 0
+        finally:
+            bds.shutdown(wait=False)
+
+    def test_a_different_retry_policy_gets_its_own_adapter(self, global_service):
+        from src.common.api_helper import APIHelper
+        helper_adapter = APIHelper().session.get_adapter("https://x.test")
+        assert helper_adapter is APIHelper().session.get_adapter("https://x.test")
+        assert helper_adapter is not global_service.shared_adapter(0)
+        assert helper_adapter.max_retries.total == 3
+        assert helper_adapter.max_retries.status_forcelist == [429, 500, 502, 503, 504]
+        assert APIHelper(max_retries=1).session.get_adapter("https://x.test") is not helper_adapter
+
+    def test_the_pooled_session_keeps_no_cookies(self, service):
+        import http.client
+        import io
+        from types import SimpleNamespace
+        from requests.cookies import extract_cookies_to_jar
+
+        def offer_cookie(session):
+            msg = http.client.parse_headers(io.BytesIO(b"Set-Cookie: sid=1; Path=/" + b"\r\n" * 2))
+            raw = SimpleNamespace(_original_response=SimpleNamespace(msg=msg))
+            request = requests.Request("GET", "https://api.test/").prepare()
+            extract_cookies_to_jar(session.cookies, request, raw)
+            return len(session.cookies)
+
+        assert offer_cookie(requests.Session()) == 1        # what a private Session does
+        assert offer_cookie(service.session_for("https://api.test/")) == 0
+
+
+# --- configuration ----------------------------------------------------------------------------------
+
+class TestConfigure:
+
+    def test_reapplying_the_same_section_keeps_the_validator_store(self, clock):
+        config = {"rate_limits": {}}
+        svc = FetchService(config, clock=clock.now, sleep=clock.sleep)
+        svc.get(FakeSession(Versioned()), "https://api.test/x")
+        svc.configure(dict(config))
+        assert svc.snapshot()["validators"]["entries"] == 1
+        svc.configure({"rate_limits": {"api.test": {"per_second": 1}}})
+        assert svc.snapshot()["validators"]["entries"] == 0
+
+    @pytest.mark.parametrize("bad", [
+        "nonsense",
+        {"rate_limits": "nonsense"},
+        {"rate_limits": {"api.test": "fast"}},
+        {"rate_limits": {"api.test": {"per_second": -1}}},
+        {"max_wait_seconds": "long"},
+    ])
+    def test_bad_values_fall_back_without_raising(self, clock, bad):
+        svc = FetchService(bad, clock=clock.now, sleep=clock.sleep)
+        assert svc.describe_config()["max_wait_seconds"] == 2.0
+        svc.get(FakeSession(), "https://api.test/x")
+
+    def test_the_template_section_is_what_the_code_defaults_to(self):
+        import os
+        root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+        with open(os.path.join(root, "config", "config.template.json"), encoding="utf-8") as fh:
+            template = json.load(fh)["fetch_service"]
+        for key, value in template.items():
+            assert fs.DEFAULT_CONFIG[key] == value
+
+
+# --- publishing and reading -----------------------------------------------------------------------------
+
+class SharedCache:
+    def __init__(self):
+        self.entries = {}
+        self.writes = 0
+
+    def get(self, key, max_age=None, memory_ttl=None):
+        return self.entries.get(key)
+
+    def set(self, key, value, *args, **kwargs):
+        self.writes += 1
+        self.entries[key] = json.loads(json.dumps(value))
+
+
+class TestPublisher:
+
+    def _publisher(self, service, clock, cache):
+        return FetchStatsPublisher(cache, service, clock=clock.now, wall_clock=lambda: 5000.0)
+
+    def test_on_change_at_most_once_a_minute(self, service, clock):
+        cache = SharedCache()
+        publisher = self._publisher(service, clock, cache)
+        assert publisher.tick() is True                  # first publish
+        assert publisher.tick() is False                 # nothing changed
+        service.get(FakeSession(), "https://api.test/x")
+        clock.advance(30)
+        assert publisher.tick() is False                 # changed, but too soon
+        clock.advance(30)
+        assert publisher.tick() is True
+        assert cache.writes == 2
+        snap = cache.entries[fs.FETCH_STATS_KEY]
+        assert snap["running"] is True
+        assert snap["totals"]["requests"] == 1
+
+    def test_heartbeat_when_nothing_changes(self, service, clock):
+        cache = SharedCache()
+        publisher = self._publisher(service, clock, cache)
+        publisher.tick()
+        clock.advance(fs.REFRESH_INTERVAL - 1)
+        assert publisher.tick() is False
+        clock.advance(1)
+        assert publisher.tick() is True
+
+    def test_stop_publishes_stopped(self, service, clock):
+        cache = SharedCache()
+        publisher = self._publisher(service, clock, cache)
+        publisher.stop()
+        assert cache.entries[fs.FETCH_STATS_KEY]["running"] is False
+
+    def test_a_failing_cache_never_raises(self, service, clock):
+        class Broken(SharedCache):
+            def set(self, *a, **k):
+                raise OSError("disk full")
+
+        assert self._publisher(service, clock, Broken()).tick() is False
+
+    def test_reader_statuses(self, service, clock):
+        cache = SharedCache()
+        assert read_fetch_stats(cache)["status"] == "unknown"
+        assert read_fetch_stats(None)["status"] == "unknown"
+        publisher = self._publisher(service, clock, cache)
+        publisher.tick()
+        assert read_fetch_stats(cache, now=5010.0)["status"] == "live"
+        assert read_fetch_stats(cache, now=5000.0 + fs.STALE_AFTER + 1)["status"] == "stale"
+        publisher.stop()
+        view = read_fetch_stats(cache, now=5010.0)
+        assert view["status"] == "stopped"
+        assert view["data"]["totals"]["requests"] == 0
+
+
+def test_the_web_route_returns_the_published_counters(clock):
+    from test._api_v3_test_helpers import build_app
+    from web_interface.blueprints import api_v3 as module
+
+    svc = FetchService({"rate_limits": {}}, clock=clock.now, sleep=clock.sleep)
+    with plugin_scope("weather"):
+        svc.get(FakeSession(), "https://api.test/x")
+    cache = SharedCache()
+    FetchStatsPublisher(cache, svc, wall_clock=time.time).tick()
+
+    original = getattr(module.api_v3, "cache_manager", None)
+    module.api_v3.cache_manager = cache
+    try:
+        body = build_app(module.api_v3).test_client().get("/api/v3/plugins/fetch-stats").get_json()
+    finally:
+        module.api_v3.cache_manager = original
+    assert body["status"] == "success"
+    assert body["data"]["status"] == "live"
+    assert body["data"]["data"]["plugins"]["weather"]["requests"] == 1

test/test_field_model_parity.py

@@ -0,0 +1,480 @@
+"""
+build_field_model() against the real render_field macro, for every schema.
+
+The field model (src/plugin_system/field_model.py) is meant to replace the
+1,100-line ``render_field`` macro in plugin_config.html as the one description
+of a plugin's config form. Before anything renders from it, it has to be
+complete: for each schema, the model must name exactly the form controls the
+macro draws today, with the same starting values, and the same JS widgets
+with the same names and values.
+
+This test renders the macro (the real template, through a Flask Jinja
+environment so ``tojson`` behaves as in the app) and parses the form:
+
+* every named control inside the <form>: (name, control, submitted text,
+  checked) in document order. A <select> contributes the option a browser
+  would submit (the last ``selected`` one, else the first).
+* every inline widget script: (widget, name, JSON value).
+
+and checks both lists equal what the model predicts, in order.
+
+Schemas covered:
+* every plugin under plugin-repos/ and test/fixtures/plugins/,
+* the official plugins monorepo, read-only, when a checkout is found: the
+  directory named by $LEDMATRIX_MONOREPO_PLUGINS, else
+  ../ledmatrix-plugins/plugins next to this checkout, else
+  ~/.ledmatrix-dev-plugins/ledmatrix-plugins/plugins (dev_plugin_setup.sh),
+* SYNTHETIC below: one schema reaching every branch of the macro, with a
+  config that fills its tables, so CI covers every widget without the
+  monorepo.
+
+Each schema is rendered twice: with nothing stored (the macro's own default
+fallback) and with the config the route really renders -- schema defaults
+merged (prepare_plugin_config) and secrets masked.
+"""
+
+import html as html_lib
+import json
+import os
+import re
+from html.parser import HTMLParser
+from pathlib import Path
+
+import pytest
+from flask import Flask
+
+from src.element_style import expand_style_elements
+from src.plugin_system.field_model import (
+    build_field_model, field_names, form_inputs, iter_fields, widget_mounts,
+)
+from src.plugin_system.schema_manager import plugin_config_defaults, prepare_plugin_config
+from src.web_interface.secret_helpers import mask_secret_fields
+
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+TEMPLATES = PROJECT_ROOT / "web_interface" / "templates"
+
+
+# ── schema sources ──────────────────────────────────────────────────────────
+
+def _monorepo_plugins_dir():
+    candidates = []
+    if os.environ.get("LEDMATRIX_MONOREPO_PLUGINS"):
+        candidates.append(Path(os.environ["LEDMATRIX_MONOREPO_PLUGINS"]))
+    candidates.append(PROJECT_ROOT.parent / "ledmatrix-plugins" / "plugins")
+    candidates.append(Path.home() / ".ledmatrix-dev-plugins" / "ledmatrix-plugins" / "plugins")
+    for candidate in candidates:
+        if candidate.is_dir() and any(candidate.glob("*/config_schema.json")):
+            return candidate
+    return None
+
+
+MONOREPO = _monorepo_plugins_dir()
+
+
+def _schema_files():
+    found = []
+    for base, label in ((PROJECT_ROOT / "plugin-repos", "plugin-repos"),
+                        (PROJECT_ROOT / "test" / "fixtures" / "plugins", "fixtures"),
+                        (MONOREPO, "monorepo")):
+        if base is None or not base.is_dir():
+            continue
+        for path in sorted(base.glob("*/config_schema.json")):
+            found.append((f"{label}/{path.parent.name}", path))
+    return found
+
+
+SCHEMA_FILES = _schema_files()
+
+
+# Every branch of render_field / render_nested_section, plus a config that
+# gives the row-based widgets rows to draw.
+SYNTHETIC = {
+    "type": "object",
+    "x-propertyOrder": ["display_duration", "label", "mode", "count", "ratio",
+                        "brightness", "zoom", "dup_enum", "tags", "days", "teams", "calendars",
+                        "images", "feeds", "bad_feeds", "rows", "events", "colour", "credentials",
+                        "files", "password", "picker", "plugin_widget", "nullable",
+                        "nullable_number", "toggle", "flag", "schedule", "window",
+                        "customization", "nested", "legacy", "empty_object", "hidden_one",
+                        "hidden_object", "fancy_advanced", "not_advanced_object", "union"],
+    "properties": {
+        "enabled": {"type": "boolean", "default": True},
+        "display_duration": {"type": "number", "default": 15, "minimum": 1},
+        "label": {"type": "string", "default": "Hello \"world\" & <you>", "title": "Label"},
+        "mode": {"type": "string", "enum": ["vs", "abbrev", "full_name"], "default": "abbrev",
+                 "x-options": {"labels": {"vs": "vs."}}},
+        "count": {"type": "integer", "default": 3, "enum": [1, 3, 5]},
+        "ratio": {"type": "number", "minimum": 0, "maximum": 1},
+        "brightness": {"type": "integer", "default": 50, "x-widget": "slider",
+                       "minimum": 0, "maximum": 100},
+        "zoom": {"type": "number", "x-widget": "number-input", "default": None},
+        # 1 == 1.0, so both options are marked selected; a browser submits the last.
+        "dup_enum": {"type": "number", "enum": [1, 1.0, 2], "default": 1},
+        "tags": {"type": "array", "items": {"type": "string"}, "default": ["a", "b"]},
+        "days": {"type": "array", "x-widget": "day-selector", "items": {"type": "string"},
+                 "default": ["mon", "fri"]},
+        "teams": {"type": "array", "x-widget": "checkbox-group",
+                  "items": {"type": "string", "enum": ["NYY", "BOS", "LAD"]},
+                  "x-options": {"labels": {"NYY": "Yankees"}}, "default": ["NYY"]},
+        "calendars": {"type": "array", "x-widget": "google-calendar-picker",
+                      "default": "primary, work"},
+        "images": {"type": "array", "x-widget": "file-upload",
+                   "x-upload-config": {"max_files": 3}, "items": {"type": "object"}},
+        "feeds": {"type": "array", "x-widget": "custom-feeds", "items": {
+            "type": "object", "properties": {
+                "name": {"type": "string"}, "url": {"type": "string"},
+                "logo": {"type": "object", "properties": {
+                    "path": {"type": "string"}, "id": {"type": "string"}}},
+                "enabled": {"type": "boolean", "default": True}}}},
+        "bad_feeds": {"type": "array", "x-widget": "custom-feeds",
+                      "items": {"type": "object", "properties": {"title": {"type": "string"}}}},
+        "rows": {"type": "array", "items": {"type": "object", "properties": {
+            "id": {"type": "string", "x-display": "hidden"},
+            "symbol": {"type": "string", "description": "Ticker"},
+            "shares": {"type": ["null", "integer"], "minimum": 0},
+            "side": {"type": "string", "enum": ["buy", "sell", None], "default": "buy"},
+            "active": {"type": "boolean", "default": True},
+            "on": {"type": "string", "x-widget": "date-picker"},
+            "at": {"type": "string", "x-widget": "time-picker"},
+            "logo": {"type": "string", "x-widget": "file-upload-single"},
+            "layout": {"type": "object", "properties": {
+                "x": {"type": "integer", "default": 0},
+                "secret_offset": {"type": "integer", "x-display": "hidden"},
+                "y": {"type": "integer"}}},
+            "note": {"type": "string", "default": "n/a"},
+            "odd": {"type": ["object", "null"], "properties": {"a": {"type": "string"}}},
+        }}},
+        "events": {"type": "array", "x-columns": ["title", "on", "at", "logo", "kind", "gone"],
+                   "items": {"type": "object", "properties": {
+                       "title": {"type": "string", "default": "Untitled"},
+                       "on": {"type": "string", "x-widget": "date-picker"},
+                       "at": {"type": "string", "x-widget": "time-picker"},
+                       "logo": {"type": "string", "x-widget": "file-upload-single"},
+                       "kind": {"type": "string", "enum": ["a", "b"], "default": "b"},
+                       "secret": {"type": "string", "x-display": "hidden"}}}},
+        "colour": {"type": "array", "x-widget": "color-picker", "default": [10, 20, 30]},
+        "credentials": {"type": "string", "x-widget": "file-upload",
+                        "x-upload-config": {"target_filename": "creds.json"}},
+        "files": {"type": "string", "x-widget": "json-file-manager"},
+        "password": {"type": "string", "x-widget": "password-input", "x-secret": True,
+                     "default": "hunter2"},
+        "picker": {"type": "string", "x-widget": "font-selector", "default": "4x6"},
+        "plugin_widget": {"type": "string", "x-widget": "custom-leagues", "default": "eng.1"},
+        "nullable": {"type": ["null", "string"], "default": None},
+        "nullable_number": {"type": "integer", "default": None},
+        "toggle": {"type": "boolean", "x-widget": "toggle-switch"},
+        "flag": {"type": "boolean", "default": False, "x-advanced": True},
+        "schedule": {"type": "object", "x-widget": "schedule-picker",
+                     "properties": {"enabled": {"type": "boolean"}}},
+        "window": {"type": "object", "x-widget": "time-range", "default": {"start": "07:00"}},
+        "customization": {"type": "object", "x-widget": "style-editor", "properties": {
+            "score_text": {"type": "object", "properties": {
+                "font": {"type": "string", "default": "PressStart2P"},
+                "text_color": {"type": "array", "x-widget": "color-picker",
+                               "default": [255, 0, 0]}}},
+            "favorite_result_colors": {"type": "boolean", "default": True}}},
+        "nested": {"type": "object", "title": "Nested", "x-propertyOrder": ["b", "a", "missing"],
+                   "properties": {
+                       "a": {"type": "string", "default": "x"},
+                       "b": {"type": "object", "properties": {
+                           "deep": {"type": "integer", "default": 7}}}}},
+        "legacy": {"type": "object", "properties": {
+            "enabled": {"type": "boolean"}, "seconds": {"type": "integer", "default": 30}}},
+        "empty_object": {"type": "object"},
+        "hidden_one": {"type": "string", "x-display": "hidden", "default": "zzz"},
+        "hidden_object": {"type": "object", "properties": {
+            "inner": {"type": "string", "x-display": "hidden"}}},
+        "fancy_advanced": {"type": "integer", "default": 1, "x-advanced": True},
+        "not_advanced_object": {"type": "object", "x-advanced": True, "properties": {
+            "inner": {"type": "boolean", "default": True}}},
+        "union": {"type": ["boolean", "object"], "properties": {
+            "enabled": {"type": "boolean"}}},
+    },
+}
+
+SYNTHETIC_CONFIG = {
+    "label": "stored 'quote'",
+    "teams": ["BOS", "SEA"],  # SEA is no longer an option
+    "images": [{"id": "img-1", "path": "assets/a.png", "filename": "a.png",
+                "schedule": {"enabled": True, "mode": "weekly"}}],
+    "feeds": [
+        {"name": "News", "url": "https://example.com/rss",
+         "logo": {"path": "assets/logo.png", "id": "logo-1"}, "enabled": False},
+        {"name": "Blog", "url": "https://example.com/blog"},
+    ],
+    "bad_feeds": [{"title": "ignored"}],
+    "rows": [
+        {"id": "row-1", "symbol": "AAPL", "shares": 10, "side": "sell", "active": False,
+         "on": "2026-01-02", "layout": {"x": 3, "secret_offset": 9}, "odd": {"a": "b"}},
+        {"symbol": "MSFT", "shares": None, "at": "09:30", "logo": "assets/m.png"},
+    ],
+    "events": [
+        {"title": "Launch", "on": "2026-03-04", "at": "18:00", "logo": "assets/l.png",
+         "kind": "a", "secret": "s3"},
+        {"on": None, "at": None, "logo": None, "kind": None},
+    ],
+    "colour": [1, 2],
+    "legacy": True,
+    "union": True,
+    "zoom": 2.5,
+}
+
+# Every branch the macro has, so a schema set that stops reaching one fails.
+MACRO_WIDGETS = {
+    "checkbox", "toggle-switch", "select", "number", "slider", "number-input",
+    "file-upload", "checkbox-group", "google-calendar-picker", "day-selector",
+    "custom-feeds", "array-table", "color-picker", "csv-text", "text",
+    "json-file-manager", "password-input", "font-selector", "custom-leagues",
+    "schedule-picker", "time-range", "style-editor", "section",
+}
+
+
+# ── rendering the macro ─────────────────────────────────────────────────────
+
+_app = Flask("field_model_parity", template_folder=str(TEMPLATES))
+
+
+def _render(schema, config, plugin_id):
+    plugin = {"id": plugin_id, "name": plugin_id, "description": "", "enabled": True,
+              "author": "test", "version": "1.0.0"}
+    with _app.app_context():
+        return _app.jinja_env.get_template("v3/partials/plugin_config.html").render(
+            plugin=plugin, schema=schema, config=config, web_ui_actions=[])
+
+
+class _FormParser(HTMLParser):
+    """Named controls and widget scripts inside the config <form>."""
+
+    def __init__(self):
+        super().__init__(convert_charrefs=True)
+        self.depth = 0
+        self.controls = []
+        self.scripts = []
+        self._select = None
+        self._in_script = False
+        self._script = []
+
+    def handle_starttag(self, tag, attrs):
+        a = dict(attrs)
+        if tag == "form" and (a.get("id") or "").startswith("plugin-config-form-"):
+            self.depth += 1
+            return
+        if not self.depth:
+            return
+        if tag == "script":
+            self._in_script, self._script = True, []
+        elif tag == "input" and a.get("name") is not None:
+            kind = (a.get("type") or "text").lower()
+            if kind == "checkbox":
+                value = a.get("value") if a.get("value") is not None else "on"
+            else:
+                value = a.get("value") if a.get("value") is not None else ""
+            self.controls.append({"name": a["name"], "control": kind, "text": value,
+                                  "checked": "checked" in a if kind == "checkbox" else None})
+        elif tag == "select" and a.get("name") is not None:
+            self._select = {"name": a["name"], "control": "select", "options": [],
+                            "selected": [], "checked": None}
+        elif tag == "option" and self._select is not None:
+            self._select["options"].append(a.get("value"))
+            if "selected" in a:
+                self._select["selected"].append(a.get("value"))
+
+    def handle_endtag(self, tag):
+        if tag == "form" and self.depth:
+            self.depth -= 1
+        elif tag == "script" and self._in_script:
+            self._in_script = False
+            self.scripts.append("".join(self._script))
+        elif tag == "select" and self._select is not None:
+            s = self._select
+            text = s["selected"][-1] if s["selected"] else (s["options"][0] if s["options"] else "")
+            self.controls.append({"name": s["name"], "control": "select", "text": text,
+                                  "checked": None, "options": s["options"]})
+            self._select = None
+
+    def handle_data(self, data):
+        if self._in_script:
+            self._script.append(data)
+
+
+_VALUE_RE = re.compile(r"^\s*var value = (?:fallback \? fallback\.value : )?(.*);\s*$", re.M)
+_NAME_RE = re.compile(r"\bname: '([^']*)'")
+_WIDGET_RE = re.compile(r"LEDMatrixWidgets\.get\('([^']+)'\)")
+_PLUGIN_WIDGET_RE = re.compile(r"var WIDGET = (\".*?\");")
+
+
+def _script_mount(script):
+    plugin = _PLUGIN_WIDGET_RE.search(script)
+    widget = json.loads(plugin.group(1)) if plugin else None
+    if widget is None:
+        found = _WIDGET_RE.search(script)
+        widget = found.group(1) if found else None
+    if widget is None:
+        return None
+    name = _NAME_RE.search(script)
+    value = _VALUE_RE.search(script)
+    return (widget,
+            html_lib.unescape(name.group(1)) if name else None,
+            _canon(json.loads(value.group(1))) if value else None)
+
+
+def _parse_form(markup):
+    parser = _FormParser()
+    parser.feed(markup)
+    parser.close()
+    controls = [(c["name"], c["control"], c["text"], c["checked"], tuple(c.get("options") or ()))
+                for c in parser.controls]
+    mounts = [m for m in (_script_mount(s) for s in parser.scripts) if m]
+    return controls, mounts
+
+
+# ── what the model predicts ─────────────────────────────────────────────────
+
+def _canon(value):
+    """JSON round trip: tuples become lists, so equality is JSON equality."""
+    return json.loads(json.dumps(value))
+
+
+def _as_text(item):
+    value, encoding = item["value"], item["encoding"]
+    if encoding == "json":
+        return value  # compared after parsing, see _expected_controls
+    if encoding == "csv":
+        return ", ".join(str(v) for v in value)
+    if encoding == "bool":
+        return "true" if value else "false"
+    return str(value)
+
+
+def _expected_controls(model):
+    out = []
+    for item in form_inputs(model):
+        options = tuple(str(o) for o in item.get("options") or ())
+        out.append((item["name"], item["control"], _as_text(item),
+                    item.get("checked") if item["control"] == "checkbox" else None, options))
+    return out
+
+
+def _normalise_json_controls(controls, model_inputs):
+    """Compare JSON-encoded inputs by value, not by spelling."""
+    result = []
+    for control, item in zip(controls, model_inputs):
+        if item["encoding"] == "json" and control[0] == item["name"]:
+            try:
+                parsed = json.loads(control[2])
+            except ValueError:
+                parsed = control[2]
+            control = (control[0], control[1], parsed, control[3], control[4])
+        result.append(control)
+    return result + list(controls[len(model_inputs):])
+
+
+def _expected_mounts(model):
+    return [(m["widget"], m["name"], _canon(m["value"])) for m in widget_mounts(model)]
+
+
+# ── cases ───────────────────────────────────────────────────────────────────
+
+def _route_config(schema, stored):
+    """The config plugin_config.html is rendered with (pages_v3)."""
+    config = prepare_plugin_config(stored, schema, plugin_config_defaults(schema))
+    return mask_secret_fields(config, schema.get("properties") or {})
+
+
+def _cases():
+    cases = [("synthetic", "stored", SYNTHETIC, SYNTHETIC_CONFIG),
+             ("synthetic", "route", SYNTHETIC, _route_config(SYNTHETIC, SYNTHETIC_CONFIG)),
+             ("synthetic", "empty", SYNTHETIC, {}),
+             ("schemaless", "stored", {}, {"enabled": True, "a": True, "b": 2.5, "c": "x"})]
+    for label, path in SCHEMA_FILES:
+        schema = expand_style_elements(json.loads(path.read_text(encoding="utf-8")))
+        cases.append((label, "empty", schema, {}))
+        cases.append((label, "route", schema, _route_config(schema, {})))
+    return cases
+
+
+CASES = _cases()
+
+
+def _check(schema, config, plugin_id):
+    markup = _render(schema, config, plugin_id)
+    model = build_field_model(schema, config, plugin_id)
+    controls, mounts = _parse_form(markup)
+    inputs = form_inputs(model)
+    expected = _expected_controls(model)
+    expected = [(n, c, _canon(t) if i["encoding"] == "json" else t, k, o)
+                for (n, c, t, k, o), i in zip(expected, inputs)]
+    assert _normalise_json_controls(controls, inputs) == expected
+    assert mounts == _expected_mounts(model)
+    # The headline property: the same set of posted names.
+    rendered = {c[0] for c in controls} | {m[1] for m in mounts if m[1]}
+    assert rendered == {name for name, _ in field_names(model)}
+    return model
+
+
+@pytest.mark.parametrize("label,variant,schema,config", CASES,
+                         ids=[f"{c[0]}[{c[1]}]" for c in CASES])
+def test_model_matches_the_macro(label, variant, schema, config):
+    plugin_id = label.split("/")[-1]
+    _check(schema, json.loads(json.dumps(config)), plugin_id)
+
+
+def test_every_macro_branch_is_reached():
+    """The cases above must exercise every widget path the macro has."""
+    seen = set()
+    for _label, _variant, schema, config in CASES:
+        model = build_field_model(schema, json.loads(json.dumps(config)), "p")
+        seen |= {node["widget"] for node in iter_fields(model)}
+    assert MACRO_WIDGETS <= seen, sorted(MACRO_WIDGETS - seen)
+
+
+def test_the_local_schemas_are_all_covered():
+    """plugin-repos/ and the fixtures are always in the parity set."""
+    labels = {label for label, _ in SCHEMA_FILES}
+    for base, prefix in ((PROJECT_ROOT / "plugin-repos", "plugin-repos"),
+                         (PROJECT_ROOT / "test" / "fixtures" / "plugins", "fixtures")):
+        for path in base.glob("*/config_schema.json"):
+            assert f"{prefix}/{path.parent.name}" in labels
+
+
+def test_the_synthetic_model_reads_as_documented():
+    """Spot checks of the model itself, beyond parity with the HTML."""
+    model = build_field_model(SYNTHETIC, json.loads(json.dumps(SYNTHETIC_CONFIG)), "demo")
+    by_path = {}
+    for node in iter_fields(model):
+        # First wins: a style-editor shares its path with its fallback section.
+        by_path.setdefault(node["path"], node)
+
+    assert "enabled" not in by_path  # the header toggle owns it
+    assert "hidden_one" not in by_path and "hidden_object" not in by_path
+    assert model["rendered_sections"][-2:] == ["flag", "fancy_advanced"]
+    assert [n["path"] for n in model["advanced_fields"]] == ["flag", "fancy_advanced"]
+    assert by_path["not_advanced_object"]["advanced"] is False
+
+    assert by_path["mode"]["widget"] == "select"
+    assert by_path["mode"]["options"][0] == {"value": "vs", "label": "vs."}
+    assert by_path["count"]["widget"] == "select"  # enum wins over integer
+    assert by_path["teams"]["stale_values"] == ["SEA"]
+    assert by_path["teams"]["value"] == ["BOS"]
+    assert by_path["calendars"]["mount"]["value"] == ["primary", "work"]
+    assert by_path["legacy"]["value"] == {"enabled": True}
+    assert by_path["legacy.enabled"]["inputs"][0]["checked"] is True
+    assert by_path["nested.b.deep"]["value"] == 7
+    assert [c["key"] for c in by_path["nested"]["children"]] == ["b", "a"]
+    assert by_path["password"]["secret"] is True
+    assert by_path["plugin_widget"]["mount"]["plugin_widget"] is True
+    assert by_path["customization"]["widget"] == "style-editor"
+    assert by_path["customization.score_text.text_color"]["widget"] == "color-picker"
+    assert [c["key"] for c in by_path["rows"]["columns"]] == ["symbol", "shares", "side", "active"]
+    assert by_path["rows"]["advanced_columns"] == ["on", "at", "logo", "layout", "note", "odd"]
+    assert by_path["bad_feeds"]["error"]
+    assert "default" not in by_path["ratio"] and by_path["ratio"]["value"] is None
+    json.dumps(model)  # plain JSON all the way down
+
+
+def test_monorepo_coverage_is_reported():
+    """Not a gate: say which monorepo the parity run used (or that it was absent)."""
+    count = sum(1 for label, _ in SCHEMA_FILES if label.startswith("monorepo/"))
+    if MONOREPO is None:
+        pytest.skip("no ledmatrix-plugins checkout found; set LEDMATRIX_MONOREPO_PLUGINS")
+    assert count == len(list(MONOREPO.glob("*/config_schema.json")))

test/test_font_manager.py

@@ -150,16 +150,6 @@ class TestCacheLifecycle:
         fm.clear_cache()
         assert fm.cache_generation == gen_before + 1
 
-    def test_clearing_a_plugins_cached_fonts_bumps_generation(self, fm):
-        fm.font_cache["demo::tiny_8"] = object()
-        gen_before = fm.cache_generation
-        fm._clear_plugin_font_cache("demo")
-        assert "demo::tiny_8" not in fm.font_cache
-        assert fm.cache_generation == gen_before + 1
-        # Nothing to drop, nothing to rebuild.
-        fm._clear_plugin_font_cache("demo")
-        assert fm.cache_generation == gen_before + 1
-
 
 class TestPluginFonts:
     """plugin:// sources resolve against the plugin's own directory, which

test/test_ipc_contract.py

@@ -0,0 +1,275 @@
+"""The control socket's contract (src/ipc/contract.py): messages and framing.
+
+Pure data, so every test here runs on every platform. What they pin:
+
+* a request and a response survive encode -> decode -> parse unchanged, and
+  the on-demand arguments carry exactly what the file mailbox carries;
+* the envelope and the arguments refuse what the display could not act on
+  (missing ids, wrong types, a non-finite duration) with a stable error code;
+* framing never holds more than one message's worth of bytes, however the
+  bytes arrive;
+* where the socket is looked for, and how it is switched off.
+"""
+
+import json
+import math
+
+import pytest
+
+from src.ipc import contract as c
+from src.ipc.contract import (
+    Command, ErrorCode, FrameReader, OnDemandStartArgs, OnDemandStopArgs,
+    ProtocolError, Request, Response,
+)
+
+
+def _wire(obj):
+    """Encode then decode, as one side's bytes reach the other."""
+    data = c.encode_message(obj)
+    assert data.endswith(b'\n') and data.count(b'\n') == 1
+    return c.decode_message(data)
+
+
+class TestRoundTrip:
+    def test_request(self):
+        req = Request(id='abc-1', cmd=Command.ON_DEMAND_START,
+                      args={'plugin_id': 'clock', 'mode': None, 'duration': 30.0,
+                            'pinned': True})
+        back = Request.from_dict(_wire(req.to_dict()))
+        assert back == req
+        assert back.v == c.PROTOCOL_VERSION
+
+    def test_success_response(self):
+        resp = Response.success('abc-1', {'accepted': True, 'request_id': 'abc-1', 'queued': 1})
+        back = Response.from_dict(_wire(resp.to_dict()))
+        assert back == resp
+        assert back.ok and back.error is None
+
+    def test_failure_response(self):
+        resp = Response.failure('abc-1', ErrorCode.BUSY, 'queue full')
+        wire = _wire(resp.to_dict())
+        assert wire == {'v': 1, 'id': 'abc-1', 'ok': False,
+                        'error': {'code': 'busy', 'message': 'queue full'}}
+        assert Response.from_dict(wire) == resp
+
+    def test_failure_without_an_id(self):
+        wire = _wire(Response.failure(None, ErrorCode.BAD_JSON, 'nope').to_dict())
+        assert wire['id'] is None
+        assert Response.from_dict(wire).id is None
+
+    def test_start_args_round_trip(self):
+        args = OnDemandStartArgs(plugin_id='clock', mode='clock_main', duration=45.0,
+                                 pinned=True)
+        assert OnDemandStartArgs.from_dict(_wire(args.to_dict())) == args
+
+    def test_encoded_messages_are_ascii_single_lines(self):
+        data = c.encode_message({'v': 1, 'id': 'x', 'cmd': 'ping',
+                                 'args': {'text': 'line1\nline2 café'}})
+        assert data.count(b'\n') == 1
+        data.decode('ascii')
+        assert c.decode_message(data)['args']['text'] == 'line1\nline2 café'
+
+
+class TestEnvelopeValidation:
+    @pytest.mark.parametrize('obj', [
+        [], 'x', 1, None,
+    ])
+    def test_not_an_object(self, obj):
+        with pytest.raises(ProtocolError) as e:
+            Request.from_dict(obj)
+        assert e.value.code == ErrorCode.BAD_REQUEST
+
+    @pytest.mark.parametrize('bad_id', [None, '', 7, 'x' * (c.MAX_ID_LENGTH + 1), 'a\nb'])
+    def test_bad_id(self, bad_id):
+        with pytest.raises(ProtocolError) as e:
+            Request.from_dict({'v': 1, 'id': bad_id, 'cmd': 'ping'})
+        assert e.value.code == ErrorCode.BAD_REQUEST
+        assert e.value.request_id is None
+
+    @pytest.mark.parametrize('v', [None, '1', 1.0, True])
+    def test_bad_version_type_keeps_the_id(self, v):
+        with pytest.raises(ProtocolError) as e:
+            Request.from_dict({'v': v, 'id': 'r1', 'cmd': 'ping'})
+        assert e.value.code == ErrorCode.BAD_REQUEST
+        assert e.value.request_id == 'r1'
+
+    def test_missing_cmd(self):
+        with pytest.raises(ProtocolError) as e:
+            Request.from_dict({'v': 1, 'id': 'r1'})
+        assert e.value.code == ErrorCode.BAD_REQUEST
+
+    def test_args_default_to_empty(self):
+        assert Request.from_dict({'v': 1, 'id': 'r', 'cmd': 'ping'}).args == {}
+        assert Request.from_dict({'v': 1, 'id': 'r', 'cmd': 'ping', 'args': None}).args == {}
+
+    def test_args_must_be_an_object(self):
+        with pytest.raises(ProtocolError) as e:
+            Request.from_dict({'v': 1, 'id': 'r', 'cmd': 'ping', 'args': [1]})
+        assert e.value.code == ErrorCode.BAD_REQUEST
+
+    def test_an_unknown_version_parses(self):
+        # The server, not the parser, decides about versions, so that hello
+        # can negotiate.
+        assert Request.from_dict({'v': 99, 'id': 'r', 'cmd': 'hello'}).v == 99
+
+    @pytest.mark.parametrize('obj', [
+        {'v': 1, 'id': 'r', 'ok': 'yes'},
+        {'v': 1, 'id': 'r', 'ok': False},
+        {'v': 1, 'id': 'r', 'ok': False, 'error': {'message': 'x'}},
+        {'v': 1, 'id': 5, 'ok': True},
+        {'v': 1, 'id': 'r', 'ok': True, 'result': [1]},
+        {'id': 'r', 'ok': True},
+    ])
+    def test_malformed_responses(self, obj):
+        with pytest.raises(ProtocolError):
+            Response.from_dict(obj)
+
+
+class TestOnDemandArgs:
+    def test_plugin_or_mode_is_required(self):
+        with pytest.raises(ProtocolError) as e:
+            OnDemandStartArgs.from_dict({'duration': 10})
+        assert e.value.code == ErrorCode.INVALID_ARGS
+
+    def test_mode_alone_is_enough(self):
+        assert OnDemandStartArgs.from_dict({'mode': 'nfl_live'}).mode == 'nfl_live'
+
+    @pytest.mark.parametrize('raw, seconds', [
+        (None, None), ('', None), (0, None), (45, 45.0), (2.5, 2.5), ('30', 30.0),
+    ])
+    def test_duration(self, raw, seconds):
+        assert OnDemandStartArgs.from_dict({'plugin_id': 'p', 'duration': raw}).duration == seconds
+
+    @pytest.mark.parametrize('raw', [-1, 'soon', True, [5], math.inf, math.nan, 'inf'])
+    def test_bad_duration(self, raw):
+        with pytest.raises(ProtocolError) as e:
+            OnDemandStartArgs.from_dict({'plugin_id': 'p', 'duration': raw})
+        assert e.value.code == ErrorCode.INVALID_ARGS
+
+    @pytest.mark.parametrize('pinned', ['true', 1, 'false'])
+    def test_pinned_must_be_a_real_boolean(self, pinned):
+        # The web route coerces "false" to False before it gets here; the
+        # contract does not guess (bool("false") is True).
+        with pytest.raises(ProtocolError):
+            OnDemandStartArgs.from_dict({'plugin_id': 'p', 'pinned': pinned})
+
+    @pytest.mark.parametrize('name', [5, 'x' * (c.MAX_NAME_LENGTH + 1), 'a\nb'])
+    def test_bad_names(self, name):
+        with pytest.raises(ProtocolError):
+            OnDemandStartArgs.from_dict({'plugin_id': name})
+
+    def test_unknown_command(self):
+        with pytest.raises(ProtocolError) as e:
+            c.parse_args('reboot', {})
+        assert e.value.code == ErrorCode.UNKNOWN_COMMAND
+
+    @pytest.mark.parametrize('cmd', c.COMMANDS)
+    def test_every_command_has_an_argument_type(self, cmd):
+        args = {'plugin_id': 'p'} if cmd == Command.ON_DEMAND_START else {}
+        c.parse_args(cmd, args)
+
+    def test_hello_versions(self):
+        assert c.HelloArgs.from_dict({'versions': [1, 2], 'client': 'web'}).versions == (1, 2)
+        for bad in ([], ['1'], 'x', [True]):
+            with pytest.raises(ProtocolError):
+                c.HelloArgs.from_dict({'versions': bad})
+
+    def test_negotiation(self):
+        assert c.negotiate_version((1,)) == 1
+        assert c.negotiate_version((1, 7)) == 1
+        assert c.negotiate_version((7,)) is None
+
+
+class TestMailboxShape:
+    """Socket commands are handed to the mailbox's own handler, so they must
+    look exactly like what the web route writes to the mailbox."""
+
+    def test_start(self):
+        args = OnDemandStartArgs(plugin_id='clock', mode='clock_main', duration=60.0,
+                                 pinned=True)
+        payload = c.on_demand_request('rid', args, 123.0)
+        assert payload == {'request_id': 'rid', 'action': 'start', 'plugin_id': 'clock',
+                           'mode': 'clock_main', 'duration': 60.0, 'pinned': True,
+                           'timestamp': 123.0, 'source': 'socket'}
+
+    def test_stop(self):
+        payload = c.on_demand_request('rid', OnDemandStopArgs(), 5.0)
+        assert payload['action'] == 'stop' and payload['request_id'] == 'rid'
+
+
+class TestFraming:
+    def test_one_message_in_pieces(self):
+        data = c.encode_message({'v': 1, 'id': 'a', 'cmd': 'ping'})
+        reader = FrameReader()
+        out = []
+        for i in range(len(data)):
+            out += reader.feed(data[i:i + 1])
+        assert [json.loads(x) for x in out] == [{'v': 1, 'id': 'a', 'cmd': 'ping'}]
+        assert reader.pending == 0
+
+    def test_several_messages_in_one_chunk(self):
+        data = b''.join(c.encode_message({'n': n}) for n in range(3))
+        assert [json.loads(x)['n'] for x in FrameReader().feed(data)] == [0, 1, 2]
+
+    def test_blank_lines_are_skipped(self):
+        assert FrameReader().feed(b'\n\r\n  \n{"a":1}\n') == [b'{"a":1}']
+
+    def test_a_line_over_the_limit_is_refused(self):
+        reader = FrameReader(max_bytes=32)
+        with pytest.raises(ProtocolError) as e:
+            reader.feed(b'x' * 40 + b'\n')
+        assert e.value.code == ErrorCode.MESSAGE_TOO_LARGE
+
+    def test_a_line_that_never_ends_is_refused_at_the_limit(self):
+        reader = FrameReader(max_bytes=32)
+        reader.feed(b'x' * 31)
+        with pytest.raises(ProtocolError) as e:
+            reader.feed(b'x')
+        assert e.value.code == ErrorCode.MESSAGE_TOO_LARGE
+
+    def test_exactly_the_limit_is_allowed(self):
+        reader = FrameReader(max_bytes=8)
+        assert reader.feed(b'1234567\n') == [b'1234567']
+
+    def test_encode_refuses_an_oversize_message(self):
+        with pytest.raises(ProtocolError) as e:
+            c.encode_message({'blob': 'x' * c.MAX_MESSAGE_BYTES})
+        assert e.value.code == ErrorCode.MESSAGE_TOO_LARGE
+
+    def test_encode_refuses_non_json(self):
+        with pytest.raises(ProtocolError):
+            c.encode_message({'x': math.nan})
+        with pytest.raises(ProtocolError):
+            c.encode_message({'x': object()})
+
+    @pytest.mark.parametrize('line', [b'{', b'[1,2]', b'"x"', b'\xff\xfe', b'null'])
+    def test_decode_garbage(self, line):
+        with pytest.raises(ProtocolError) as e:
+            c.decode_message(line)
+        assert e.value.code == ErrorCode.BAD_JSON
+
+
+class TestSocketLocation:
+    def test_default(self):
+        paths = c.client_socket_paths({})
+        assert paths[0] == '/run/ledmatrix/control.sock'
+        assert len(paths) == 2 and paths[1].endswith('control.sock')
+
+    def test_configured_path_is_the_only_one_tried(self):
+        assert c.client_socket_paths({c.SOCKET_PATH_ENV: '/x/y.sock'}) == ['/x/y.sock']
+
+    @pytest.mark.parametrize('value', ['off', 'OFF', '0', 'false', 'disabled', ' none '])
+    def test_switched_off(self, value):
+        env = {c.SOCKET_PATH_ENV: value}
+        assert c.socket_disabled(env)
+        assert c.client_socket_paths(env) == []
+        assert c.configured_socket_path(env) is None
+
+    def test_dev_path_is_per_user(self):
+        assert c.dev_socket_path(1000) != c.dev_socket_path(1001)
+        assert 'ledmatrix-1000' in c.dev_socket_path(1000)
+
+    def test_the_default_dir_is_the_heartbeats(self):
+        # One RuntimeDirectory= serves both (#687).
+        from src import display_watchdog
+        assert c.DEFAULT_SOCKET_DIR == display_watchdog.HEARTBEAT_DIR

test/test_ipc_display_hook.py

@@ -0,0 +1,229 @@
+"""DisplayController's side of the control socket.
+
+The server's handlers only queue; the render thread drains the queue where
+it reads the file mailbox (_poll_on_demand_requests) and hands each command
+to the mailbox's own handler (_handle_on_demand_request). These tests pin
+that hook:
+
+* a socket command is applied by the same code as a mailbox request, with
+  its request id, and without waiting for the mailbox's 0.25 s read floor;
+* a request that arrives both ways (a client that timed out after the
+  command was queued, then wrote the mailbox) is activated once;
+* a command that fails is contained, and the ones after it still run;
+* cleanup closes the socket; a disabled socket changes nothing.
+"""
+
+import os
+import time
+from unittest.mock import MagicMock
+
+import pytest
+
+from src.ipc import client
+from src.ipc import contract as c
+from src.ipc.contract import Command, OnDemandStartArgs, OnDemandStopArgs
+from src.ipc.server import QueuedCommand
+
+
+def _start(rid, plugin_id='clock', **kw):
+    return QueuedCommand(request_id=rid, cmd=Command.ON_DEMAND_START,
+                         args=OnDemandStartArgs(plugin_id=plugin_id, **kw),
+                         received_at=time.time())
+
+
+def _stop(rid):
+    return QueuedCommand(request_id=rid, cmd=Command.ON_DEMAND_STOP,
+                         args=OnDemandStopArgs(), received_at=time.time())
+
+
+class FakeServer:
+    def __init__(self, *commands):
+        self.commands = list(commands)
+        self.closed = False
+
+    @property
+    def has_pending(self):
+        return bool(self.commands)
+
+    def drain(self):
+        out, self.commands = self.commands, []
+        return out
+
+    def close(self):
+        self.closed = True
+
+
+@pytest.fixture
+def controller(test_display_controller):
+    c_ = test_display_controller
+    c_.on_demand_active = False
+    c_.on_demand_request_id = None
+    c_._last_on_demand_poll = None
+    mailbox = {'value': None}
+
+    def fake_get(key, *a, **kw):
+        if key == 'display_on_demand_request':
+            return mailbox['value']
+        return None
+
+    def fake_delete(key):
+        if key == 'display_on_demand_request':
+            mailbox['value'] = None
+
+    c_.cache_manager.get = MagicMock(side_effect=fake_get)
+    c_.cache_manager.set = MagicMock()
+    c_.cache_manager.delete = MagicMock(side_effect=fake_delete)
+    c_._activate_on_demand = MagicMock()
+    c_.mailbox = mailbox
+    return c_
+
+
+class TestDrain:
+    def test_a_socket_start_goes_through_the_mailbox_handler(self, controller):
+        controller._control_server = FakeServer(_start('sock-1', duration=30.0, pinned=True))
+        controller._poll_on_demand_requests()
+        controller._activate_on_demand.assert_called_once()
+        request = controller._activate_on_demand.call_args.args[0]
+        assert request['request_id'] == 'sock-1'
+        assert request['action'] == 'start'
+        assert request['plugin_id'] == 'clock'
+        assert request['duration'] == 30.0 and request['pinned'] is True
+        assert controller.on_demand_request_id == 'sock-1'
+        # The same restart-replay guard as a mailbox request.
+        controller.cache_manager.set.assert_any_call(
+            'display_on_demand_processed_id', 'sock-1', ttl=3600)
+
+    def test_socket_commands_skip_the_mailbox_floor(self, controller):
+        server = FakeServer()
+        controller._control_server = server
+        controller._poll_on_demand_requests()          # reads the mailbox, sets the floor
+        reads = controller.cache_manager.get.call_count
+        server.commands.append(_start('quick'))
+        controller._poll_on_demand_requests()          # within the floor
+        controller._activate_on_demand.assert_called_once()
+        mailbox_reads = [call for call in controller.cache_manager.get.call_args_list[reads:]
+                         if call.args[0] == 'display_on_demand_request']
+        # Only _consume_on_demand_request's compare-before-delete re-read.
+        assert len(mailbox_reads) <= 1
+
+    def test_a_request_that_came_both_ways_is_activated_once(self, controller):
+        controller._control_server = FakeServer(_start('dup'))
+        controller.mailbox['value'] = {'request_id': 'dup', 'action': 'start',
+                                       'plugin_id': 'clock'}
+        controller._poll_on_demand_requests()
+        controller._last_on_demand_poll = None
+        controller._poll_on_demand_requests()
+        controller._activate_on_demand.assert_called_once()
+        assert controller.mailbox['value'] is None, "the duplicate was left in the mailbox"
+
+    def test_a_fallback_write_landing_later_is_ignored(self, controller):
+        controller._control_server = FakeServer(_start('late'))
+        controller._poll_on_demand_requests()
+        controller.mailbox['value'] = {'request_id': 'late', 'action': 'start',
+                                       'plugin_id': 'clock'}
+        controller._last_on_demand_poll = None
+        controller._poll_on_demand_requests()
+        controller._activate_on_demand.assert_called_once()
+
+    def test_the_mailbox_still_works_alongside(self, controller):
+        controller._control_server = FakeServer()
+        controller.mailbox['value'] = {'request_id': 'mb', 'action': 'start', 'plugin_id': 'p'}
+        controller._poll_on_demand_requests()
+        assert controller._activate_on_demand.call_args.args[0]['request_id'] == 'mb'
+
+    def test_a_socket_stop_ends_on_demand(self, controller):
+        controller.on_demand_active = True
+        controller._clear_on_demand = MagicMock()
+        controller._control_server = FakeServer(_stop('halt'))
+        controller._poll_on_demand_requests()
+        controller._clear_on_demand.assert_called_once_with(reason='requested-stop')
+
+    def test_commands_run_in_arrival_order(self, controller):
+        seen = []
+        controller._activate_on_demand = MagicMock(
+            side_effect=lambda r: seen.append(r['request_id']))
+        controller._control_server = FakeServer(_start('a'), _start('b'), _start('c'))
+        controller._poll_on_demand_requests()
+        assert seen == ['a', 'b', 'c']
+
+    def test_a_failing_command_is_contained(self, controller):
+        calls = []
+
+        def activate(request):
+            calls.append(request['request_id'])
+            if request['request_id'] == 'bad':
+                raise RuntimeError('plugin exploded')
+
+        controller._activate_on_demand = MagicMock(side_effect=activate)
+        controller._control_server = FakeServer(_start('bad'), _start('good'))
+        controller._poll_on_demand_requests()
+        assert calls == ['bad', 'good']
+
+    def test_no_server_means_mailbox_only(self, controller):
+        controller._control_server = None
+        controller._poll_on_demand_requests()
+        controller._activate_on_demand.assert_not_called()
+
+
+class TestPendingChangesFloor:
+    def test_a_queued_command_skips_the_floor(self, controller):
+        server = FakeServer()
+        controller._control_server = server
+        controller._service_pending_changes()
+        server.commands.append(_start('now'))
+        controller._service_pending_changes()        # well inside the 0.25 s floor
+        controller._activate_on_demand.assert_called_once()
+
+    def test_nothing_queued_keeps_the_floor(self, controller):
+        controller._control_server = FakeServer()
+        controller._poll_on_demand_requests = MagicMock()
+        controller._service_pending_changes()
+        controller._service_pending_changes()
+        assert controller._poll_on_demand_requests.call_count == 1
+
+
+class TestLifecycle:
+    def test_status_snapshot(self, controller):
+        controller.current_display_mode = 'clock_main'
+        controller.on_demand_active = True
+        controller.on_demand_plugin_id = 'clock'
+        controller.on_demand_expires_at = None
+        status = controller._control_status()
+        assert status['current_mode'] == 'clock_main'
+        assert status['on_demand']['active'] is True
+        assert status['on_demand']['plugin_id'] == 'clock'
+        c.encode_message(status)   # it has to fit on the wire
+
+    def test_cleanup_closes_the_socket(self, controller):
+        server = FakeServer()
+        controller._control_server = server
+        controller.cleanup()
+        assert server.closed
+        assert controller._control_server is None
+
+    def test_disabled_socket_starts_nothing(self, controller):
+        # conftest sets LEDMATRIX_CONTROL_SOCKET=off for every test.
+        controller._start_control_server()
+        assert controller._control_server is None
+
+    @pytest.mark.skipif(not c.socket_supported(), reason='AF_UNIX sockets are Linux/macOS only')
+    def test_end_to_end(self, controller, monkeypatch):
+        import shutil
+        import tempfile
+        d = tempfile.mkdtemp(prefix='lmipc-')
+        path = os.path.join(d, 'control.sock')
+        monkeypatch.setenv(c.SOCKET_PATH_ENV, path)
+        try:
+            controller._start_control_server()
+            assert controller._control_server is not None
+            ack = client.on_demand_start('e2e', 'clock', None, 15, False, paths=[path])
+            assert ack['accepted'] is True and ack['request_id'] == 'e2e'
+            status = client.on_demand_status(paths=[path])
+            assert 'on_demand' in status and 'current_mode' in status
+            controller._service_pending_changes()
+            request = controller._activate_on_demand.call_args.args[0]
+            assert request['request_id'] == 'e2e' and request['duration'] == 15.0
+            controller.cleanup()
+            assert not os.path.exists(path)
+        finally:
+            shutil.rmtree(d, ignore_errors=True)

test/test_ipc_server.py

@@ -0,0 +1,570 @@
+"""The display side of the control socket (src/ipc/server.py).
+
+Two layers:
+
+* ``handle_line`` and the permission model are plain functions of their
+  input, tested on every platform: every request gets exactly one answer,
+  garbage is answered rather than raised, queued commands are acked with
+  their request id, and the render thread drains them in order.
+* The socket itself (``TestLiveSocket``, ``TestPermissions``) needs AF_UNIX,
+  so those tests are skipped on Windows and run on Linux (CI, WSL, a Pi): a
+  real server on a tmp_path socket, driven by the real client and by raw
+  sockets that misbehave -- garbage, oversize lines, a client that hangs up
+  mid-message, one that never finishes -- while the server keeps serving.
+"""
+
+import json
+import os
+import socket
+import stat
+import threading
+import time
+
+import pytest
+
+from src.ipc import client
+from src.ipc import contract as c
+from src.ipc import server as srv
+from src.ipc.contract import Command, ErrorCode
+from src.ipc.server import ControlServer, PeerCredentials, peer_allowed
+
+needs_unix_sockets = pytest.mark.skipif(not c.socket_supported(),
+                                        reason='AF_UNIX sockets are Linux/macOS only')
+
+
+def _line(obj):
+    return json.dumps(obj).encode()
+
+
+def _req(cmd, args=None, rid='r1', v=1):
+    return _line({'v': v, 'id': rid, 'cmd': cmd, 'args': args or {}})
+
+
+@pytest.fixture
+def status():
+    return {'on_demand': {'active': False, 'status': 'idle'}, 'current_mode': 'clock'}
+
+
+@pytest.fixture
+def server(status, tmp_path):
+    """A server that is never started: handle_line and drain only."""
+    return ControlServer(str(tmp_path / 'unused.sock'), status_provider=lambda: dict(status),
+                         queue_size=3)
+
+
+class TestHandleLine:
+    def test_ping(self, server):
+        resp = server.handle_line(_req(Command.PING))
+        assert resp.ok and resp.id == 'r1' and resp.result == {'pong': True}
+
+    def test_hello_negotiates(self, server):
+        resp = server.handle_line(_req(Command.HELLO, {'versions': [1, 5], 'client': 't'}, v=5))
+        assert resp.ok
+        assert resp.result['version'] == 1
+        assert resp.result['commands'] == list(c.COMMANDS)
+        assert resp.result['max_message_bytes'] == c.MAX_MESSAGE_BYTES
+        assert resp.v == 1
+
+    def test_hello_with_nothing_in_common(self, server):
+        resp = server.handle_line(_req(Command.HELLO, {'versions': [9]}, v=9))
+        assert not resp.ok and resp.error.code == ErrorCode.UNSUPPORTED_VERSION
+
+    def test_other_commands_need_a_supported_version(self, server):
+        resp = server.handle_line(_req(Command.PING, v=2))
+        assert not resp.ok and resp.error.code == ErrorCode.UNSUPPORTED_VERSION
+        assert resp.id == 'r1'
+
+    @pytest.mark.parametrize('line, code, rid', [
+        (b'not json', ErrorCode.BAD_JSON, None),
+        (b'[1]', ErrorCode.BAD_JSON, None),
+        (b'\xff', ErrorCode.BAD_JSON, None),
+        (_line({'v': 1, 'cmd': 'ping'}), ErrorCode.BAD_REQUEST, None),
+        (_line({'v': 'one', 'id': 'q', 'cmd': 'ping'}), ErrorCode.BAD_REQUEST, 'q'),
+        (_req('shutdown_the_pi'), ErrorCode.UNKNOWN_COMMAND, 'r1'),
+        (_req(Command.ON_DEMAND_START, {}), ErrorCode.INVALID_ARGS, 'r1'),
+        (_req(Command.ON_DEMAND_START, {'plugin_id': 'p', 'duration': 'x'}),
+         ErrorCode.INVALID_ARGS, 'r1'),
+    ])
+    def test_garbage_is_answered_not_raised(self, server, line, code, rid):
+        resp = server.handle_line(line)
+        assert not resp.ok
+        assert resp.error.code == code
+        assert resp.id == rid
+        assert server.drain() == []
+
+    def test_start_is_queued_and_acked(self, server):
+        resp = server.handle_line(_req(Command.ON_DEMAND_START,
+                                       {'plugin_id': 'clock', 'duration': 30, 'pinned': True},
+                                       rid='abc'))
+        assert resp.ok
+        assert resp.result == {'accepted': True, 'request_id': 'abc', 'queued': 1}
+        assert server.has_pending
+        [cmd] = server.drain()
+        assert not server.has_pending
+        payload = cmd.as_on_demand_request()
+        assert payload['request_id'] == 'abc'
+        assert payload['action'] == 'start'
+        assert payload['plugin_id'] == 'clock'
+        assert payload['duration'] == 30.0 and payload['pinned'] is True
+
+    def test_stop_is_queued_and_acked(self, server):
+        resp = server.handle_line(_req(Command.ON_DEMAND_STOP, rid='s1'))
+        assert resp.ok and resp.result['request_id'] == 's1'
+        assert [x.as_on_demand_request()['action'] for x in server.drain()] == ['stop']
+
+    def test_drain_keeps_arrival_order(self, server):
+        for rid in ('a', 'b', 'c'):
+            server.handle_line(_req(Command.ON_DEMAND_START, {'plugin_id': 'p'}, rid=rid))
+        assert [x.request_id for x in server.drain()] == ['a', 'b', 'c']
+        assert server.drain() == []
+
+    def test_a_full_queue_says_busy_and_queues_nothing_more(self, server):
+        for rid in ('a', 'b', 'c'):
+            assert server.handle_line(_req(Command.ON_DEMAND_STOP, rid=rid)).ok
+        resp = server.handle_line(_req(Command.ON_DEMAND_STOP, rid='d'))
+        assert not resp.ok and resp.error.code == ErrorCode.BUSY and resp.id == 'd'
+        assert [x.request_id for x in server.drain()] == ['a', 'b', 'c']
+
+    def test_status_answers_from_the_provider_without_queueing(self, server, status):
+        status['on_demand']['active'] = True
+        resp = server.handle_line(_req(Command.ON_DEMAND_STATUS))
+        assert resp.ok and resp.result['on_demand']['active'] is True
+        assert not server.has_pending
+
+    def test_a_failing_status_provider_is_an_internal_error(self, tmp_path):
+        def boom():
+            raise RuntimeError('render thread mid-update')
+        s = ControlServer(str(tmp_path / 'x.sock'), status_provider=boom)
+        resp = s.handle_line(_req(Command.ON_DEMAND_STATUS, rid='z'))
+        assert not resp.ok and resp.error.code == ErrorCode.INTERNAL and resp.id == 'z'
+
+
+class TestPermissionModel:
+    """root, the display's own user, or the shared group -- nobody else."""
+    OWN, GROUP = 0, 990
+
+    @pytest.mark.parametrize('cred, groups, allowed', [
+        (PeerCredentials(1, 0, 0), None, True),                      # root
+        (PeerCredentials(1, 1000, 1000), frozenset({990}), True),    # web user, in group
+        (PeerCredentials(1, 1000, 990), frozenset(), True),          # primary group
+        (PeerCredentials(1, 1001, 1001), frozenset({27, 44}), False),
+        (PeerCredentials(1, 65534, 65534), frozenset(), False),       # nobody
+    ])
+    def test_model(self, cred, groups, allowed):
+        assert peer_allowed(cred, self.OWN, self.GROUP, groups) is allowed
+
+    def test_own_user_without_a_group(self):
+        assert peer_allowed(PeerCredentials(1, 1000, 1000), 1000, None, frozenset())
+        assert not peer_allowed(PeerCredentials(1, 1001, 1001), 1000, None, frozenset({1}))
+
+    def test_group_database_decides_when_proc_is_unreadable(self):
+        seen = []
+
+        def in_group(uid, gid):
+            seen.append((uid, gid))
+            return uid == 1000
+
+        cred = PeerCredentials(1, 1000, 1000)
+        assert peer_allowed(cred, 0, 990, None, in_group=in_group)
+        assert not peer_allowed(PeerCredentials(1, 1001, 1001), 0, 990, None, in_group=in_group)
+        assert seen == [(1000, 990), (1001, 990)]
+
+    @pytest.mark.skipif(os.name != 'posix', reason='POSIX permission bits')
+    def test_socket_group_follows_a_shared_cache_dir(self, tmp_path):
+        shared = tmp_path / 'cache'
+        shared.mkdir()
+        os.chmod(shared, 0o2775)
+        assert srv.resolve_socket_group(str(shared)) == shared.stat().st_gid
+
+    @pytest.mark.skipif(os.name != 'posix', reason='POSIX permission bits')
+    def test_a_private_cache_dir_falls_back_to_the_project_group(self, tmp_path, monkeypatch):
+        private = tmp_path / 'cache'
+        private.mkdir()
+        os.chmod(private, 0o755)
+        from src.common import permission_utils
+        monkeypatch.setattr(permission_utils, 'get_shared_group_gid', lambda: 4242)
+        assert srv.resolve_socket_group(str(private)) == 4242
+
+    def test_mode_is_group_only_with_a_group(self, tmp_path):
+        assert ControlServer(str(tmp_path / 'a'), group=990).socket_mode == 0o660
+        assert ControlServer(str(tmp_path / 'b'), group=None).socket_mode == 0o600
+
+
+class TestWhereTheServerListens:
+    def test_off_means_no_server(self):
+        assert srv.server_socket_path({c.SOCKET_PATH_ENV: 'off'}) is None
+        assert srv.start_control_server(environ={c.SOCKET_PATH_ENV: 'off'}) is None
+
+    @needs_unix_sockets
+    def test_configured(self):
+        assert srv.server_socket_path({c.SOCKET_PATH_ENV: '/tmp/x.sock'}) == '/tmp/x.sock'
+
+    @needs_unix_sockets
+    def test_unprivileged_dev_run_uses_the_per_user_path(self, monkeypatch):
+        monkeypatch.setattr(os, 'geteuid', lambda: 1000)
+        monkeypatch.setattr(os, 'access', lambda p, m: False)
+        assert srv.server_socket_path({}) == c.dev_socket_path()
+
+    @needs_unix_sockets
+    def test_root_uses_run(self, monkeypatch):
+        monkeypatch.setattr(os, 'geteuid', lambda: 0)
+        assert srv.server_socket_path({}) == c.DEFAULT_SOCKET_PATH
+
+    @pytest.mark.skipif(c.socket_supported(), reason='Windows only')
+    def test_windows_skips_cleanly(self):
+        assert srv.server_socket_path({}) is None
+        assert ControlServer('x.sock').start() is False
+        with pytest.raises(client.ControlError) as e:
+            client.ping(paths=['x.sock'])
+        assert e.value.reason == 'unsupported'
+
+
+# -- the real socket --------------------------------------------------------------------
+
+@pytest.fixture
+def sock_path(tmp_path_factory):
+    # AF_UNIX paths are limited to ~107 bytes; pytest's tmp_path can be longer.
+    import tempfile
+    d = tempfile.mkdtemp(prefix='lmipc-')
+    yield os.path.join(d, 'control.sock')
+    import shutil
+    shutil.rmtree(d, ignore_errors=True)
+
+
+@pytest.fixture
+def live(sock_path, status):
+    servers = []
+
+    def make(**kwargs):
+        kwargs.setdefault('status_provider', lambda: dict(status))
+        s = ControlServer(sock_path, **kwargs)
+        assert s.start()
+        servers.append(s)
+        return s
+
+    yield make
+    for s in servers:
+        s.close()
+
+
+def _raw(path, timeout=2.0):
+    s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+    s.settimeout(timeout)
+    s.connect(path)
+    return s
+
+
+def _read_line(s):
+    buf = b''
+    while not buf.endswith(b'\n'):
+        try:
+            chunk = s.recv(1)   # one byte at a time: never eat the next line
+        except ConnectionResetError:
+            break
+        if not chunk:
+            break
+        buf += chunk
+    return json.loads(buf) if buf.endswith(b'\n') else None
+
+
+@needs_unix_sockets
+class TestLiveSocket:
+    def test_client_round_trip(self, live, sock_path):
+        live()
+        assert client.ping(paths=[sock_path]) == {'pong': True}
+        assert client.hello(paths=[sock_path])['version'] == 1
+        assert client.on_demand_status(paths=[sock_path])['current_mode'] == 'clock'
+
+    def test_ack_path(self, live, sock_path):
+        server = live()
+        ack = client.on_demand_start('req-1', 'clock', None, 20, True, paths=[sock_path])
+        assert ack == {'accepted': True, 'request_id': 'req-1', 'queued': 1}
+        ack = client.on_demand_stop('req-2', paths=[sock_path])
+        assert ack['request_id'] == 'req-2'
+        assert [(x.request_id, x.cmd) for x in server.drain()] == [
+            ('req-1', Command.ON_DEMAND_START), ('req-2', Command.ON_DEMAND_STOP)]
+
+    def test_socket_file_mode_and_cleanup(self, live, sock_path):
+        server = live(group=os.getgid())
+        st = os.lstat(sock_path)
+        assert stat.S_ISSOCK(st.st_mode)
+        assert stat.S_IMODE(st.st_mode) == 0o660
+        assert st.st_gid == os.getgid()
+        assert not [f for f in os.listdir(os.path.dirname(sock_path)) if f.endswith('.tmp')]
+        server.close()
+        assert not os.path.exists(sock_path)
+
+    def test_without_a_group_only_the_owner_may_connect(self, live, sock_path):
+        live(group=None)
+        assert stat.S_IMODE(os.lstat(sock_path).st_mode) == 0o600
+
+    def test_garbage_then_a_good_request_on_one_connection(self, live, sock_path):
+        live()
+        s = _raw(sock_path)
+        try:
+            s.sendall(b'this is not json\n')
+            assert _read_line(s)['error']['code'] == ErrorCode.BAD_JSON
+            s.sendall(_req(Command.PING, rid='after') + b'\n')
+            resp = _read_line(s)
+            assert resp['ok'] and resp['id'] == 'after'
+        finally:
+            s.close()
+
+    def test_two_requests_in_one_write(self, live, sock_path):
+        live()
+        s = _raw(sock_path)
+        try:
+            s.sendall(_req(Command.PING, rid='a') + b'\n' + _req(Command.PING, rid='b') + b'\n')
+            assert _read_line(s)['id'] == 'a'
+            assert _read_line(s)['id'] == 'b'
+        finally:
+            s.close()
+
+    def test_oversize_is_refused_and_the_server_lives_on(self, live, sock_path):
+        live()
+        s = _raw(sock_path)
+        try:
+            try:
+                # Exactly the limit with no newline: the server has read it
+                # all when it refuses, so its answer is not lost to a reset.
+                s.sendall(b'{"pad":"' + b'x' * (c.MAX_MESSAGE_BYTES - 8))
+            except OSError:
+                pass  # the server may hang up before we finish writing
+            resp = _read_line(s)
+            assert resp['error']['code'] == ErrorCode.MESSAGE_TOO_LARGE
+            assert s.recv(10) == b''          # and hung up
+        finally:
+            s.close()
+        assert client.ping(paths=[sock_path]) == {'pong': True}
+
+    def test_a_client_that_hangs_up_mid_message(self, live, sock_path):
+        server = live()
+        s = _raw(sock_path)
+        s.sendall(b'{"v":1,"id":"half","cmd":"on_demand.st')
+        s.close()
+        time.sleep(0.2)
+        assert client.ping(paths=[sock_path]) == {'pong': True}
+        assert server.drain() == []
+
+    def test_a_slow_client_is_dropped_and_blocks_nobody(self, live, sock_path):
+        live(io_timeout=0.2, message_timeout=0.5)
+        slow = _raw(sock_path, timeout=3)
+        try:
+            slow.sendall(b'{"v":1,')           # ...and never finishes
+            t0 = time.monotonic()
+            assert client.ping(paths=[sock_path]) == {'pong': True}
+            assert time.monotonic() - t0 < 0.5, 'a slow client held up another'
+            assert slow.recv(100) == b''       # hung up on, not answered
+        finally:
+            slow.close()
+
+    def test_an_idle_connection_is_closed(self, live, sock_path):
+        live(io_timeout=0.1, idle_timeout=0.3)
+        s = _raw(sock_path, timeout=3)
+        try:
+            assert s.recv(100) == b''
+        finally:
+            s.close()
+
+    def test_too_many_clients_are_told_busy(self, live, sock_path):
+        live(max_clients=2, io_timeout=0.2, idle_timeout=5)
+        held = [_raw(sock_path) for _ in range(2)]
+        try:
+            time.sleep(0.1)
+            extra = _raw(sock_path)
+            try:
+                assert _read_line(extra)['error']['code'] == ErrorCode.BUSY
+            finally:
+                extra.close()
+        finally:
+            for s in held:
+                s.close()
+        time.sleep(0.3)
+        assert client.ping(paths=[sock_path]) == {'pong': True}
+
+    def test_many_concurrent_clients(self, live, sock_path):
+        server = live(queue_size=64)
+        errors = []
+
+        def go(n):
+            try:
+                client.on_demand_start(f'r{n}', 'p', None, paths=[sock_path], timeout=3)
+            except client.ControlError as e:  # busy is allowed under load
+                if e.reason != ErrorCode.BUSY:
+                    errors.append(e)
+
+        threads = [threading.Thread(target=go, args=(n,)) for n in range(20)]
+        for t in threads:
+            t.start()
+        for t in threads:
+            t.join()
+        assert errors == []
+        assert 0 < len(server.drain()) <= 20
+
+    def test_a_stale_socket_is_replaced(self, sock_path, status):
+        dead = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        dead.bind(sock_path)
+        dead.close()                    # file left behind, nothing listening
+        s = ControlServer(sock_path, status_provider=lambda: status)
+        try:
+            assert s.start()
+            assert client.ping(paths=[sock_path]) == {'pong': True}
+        finally:
+            s.close()
+
+    def test_a_live_socket_is_not_stolen(self, live, sock_path, status):
+        live()
+        second = ControlServer(sock_path, status_provider=lambda: status)
+        assert second.start() is False
+        assert client.ping(paths=[sock_path]) == {'pong': True}
+
+    def test_a_regular_file_is_never_removed(self, sock_path):
+        with open(sock_path, 'w') as f:
+            f.write('precious')
+        assert ControlServer(sock_path).start() is False
+        with open(sock_path) as f:
+            assert f.read() == 'precious'
+
+    def test_close_leaves_a_successor_s_socket_alone(self, sock_path, status):
+        first = ControlServer(sock_path, status_provider=lambda: status)
+        assert first.start()
+        first._close_socket()                 # dead, but still owns the path
+        os.unlink(sock_path)
+        second = ControlServer(sock_path, status_provider=lambda: status)
+        assert second.start()
+        try:
+            first.close()                     # must not unlink second's file
+            assert client.ping(paths=[sock_path]) == {'pong': True}
+        finally:
+            second.close()
+
+    def test_client_reasons(self, sock_path, tmp_path):
+        with pytest.raises(client.ControlError) as e:
+            client.ping(paths=[sock_path])
+        assert e.value.reason == 'no_socket'
+        dead = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        dead.bind(sock_path)
+        try:
+            with pytest.raises(client.ControlError) as e:
+                client.ping(paths=[sock_path])
+            assert e.value.reason == 'refused'
+        finally:
+            dead.close()
+        with pytest.raises(client.ControlError) as e:
+            client.ping(paths=[])
+        assert e.value.reason == 'disabled'
+        with pytest.raises(client.ControlError) as e:
+            client.on_demand_start('x', None, None, paths=[sock_path])
+        assert e.value.reason == 'invalid_request'
+
+    def test_a_display_that_never_answers_times_out(self, sock_path):
+        mute = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        mute.bind(sock_path)
+        mute.listen(1)                  # accepts at the kernel, never replies
+        try:
+            t0 = time.monotonic()
+            with pytest.raises(client.ControlError) as e:
+                client.ping(paths=[sock_path], timeout=0.3)
+            assert e.value.reason == 'timeout'
+            assert time.monotonic() - t0 < 1.0
+        finally:
+            mute.close()
+
+    def test_the_dev_directory_must_be_private(self, monkeypatch, tmp_path):
+        d = tmp_path / 'shared'
+        d.mkdir()
+        target = d / 'control.sock'
+        monkeypatch.setattr(srv, 'dev_socket_path', lambda: str(target))
+        monkeypatch.setattr(os, 'geteuid', lambda: os.getuid() + 1)   # "someone else's"
+        assert ControlServer(str(target)).start() is False
+
+
+@needs_unix_sockets
+@pytest.mark.skipif(not hasattr(socket, 'SO_PEERCRED'), reason='SO_PEERCRED is Linux-only')
+class TestPermissions:
+    def test_peer_credentials_are_read(self, live, sock_path):
+        server = live()
+        s = _raw(sock_path)
+        try:
+            s.sendall(_req(Command.PING) + b'\n')
+            assert _read_line(s)['ok']
+        finally:
+            s.close()
+        a, b = socket.socketpair(socket.AF_UNIX)
+        try:
+            cred = srv.peer_credentials(a)
+            assert cred.uid == os.geteuid() and cred.pid == os.getpid()
+        finally:
+            a.close()
+            b.close()
+        assert srv.process_groups(os.getpid()) == frozenset(os.getgroups())
+        assert server.running
+
+    @pytest.mark.skipif(hasattr(os, 'geteuid') and os.geteuid() == 0,
+                        reason='root may always connect')
+    def test_a_peer_outside_the_model_is_refused(self, live, sock_path):
+        server = live(group=None)
+        # Pretend the display runs as someone else: this process is then
+        # neither root, the display's user, nor in its (absent) group.
+        server._own_uid = os.geteuid() + 12345
+        s = _raw(sock_path)
+        try:
+            resp = _read_line(s)
+            assert resp['error']['code'] == ErrorCode.FORBIDDEN
+            assert s.recv(10) == b''
+        finally:
+            s.close()
+        with pytest.raises(client.ControlError) as e:
+            client.on_demand_stop('nope', paths=[sock_path])
+        assert e.value.reason == ErrorCode.FORBIDDEN
+        assert server.drain() == []
+
+    @pytest.mark.skipif(not (hasattr(os, 'geteuid') and os.geteuid() == 0),
+                        reason='needs root to switch users (run under WSL as root, or on a Pi)')
+    def test_the_kernel_enforces_the_group(self, live, sock_path):
+        """The real deployment shape: root serves, an unprivileged user connects.
+
+        nobody in the socket's group gets in; nobody outside it gets EACCES
+        from connect() -- the kernel's check, before any byte is read.
+        """
+        import pwd
+        nobody = pwd.getpwnam('nobody')
+        allowed_gid = nobody.pw_gid
+        live(group=allowed_gid)
+        os.chmod(os.path.dirname(sock_path), 0o755)
+
+        def try_as(gid):
+            r, w = os.pipe()
+            pid = os.fork()
+            if pid == 0:  # child: drop to nobody with only `gid`
+                os.close(r)
+                try:
+                    os.setgroups([])
+                    os.setgid(gid)
+                    os.setuid(nobody.pw_uid)
+                    result = json.dumps(client.ping(paths=[sock_path]))
+                except client.ControlError as e:
+                    result = 'error:' + e.reason
+                except Exception as e:  # report anything else to the parent
+                    result = 'crash:' + repr(e)
+                os.write(w, result.encode())
+                os._exit(0)
+            os.close(w)
+            out = b''
+            while True:
+                chunk = os.read(r, 4096)
+                if not chunk:
+                    break
+                out += chunk
+            os.close(r)
+            os.waitpid(pid, 0)
+            return out.decode()
+
+        assert try_as(allowed_gid) == '{"pong": true}'
+        other_gid = allowed_gid - 1 if allowed_gid > 1 else allowed_gid + 1
+        assert try_as(other_gid) == 'error:refused'
+        # Someone loosens the mode by hand: the kernel lets the outsider
+        # connect, and SO_PEERCRED still turns it away.
+        os.chmod(sock_path, 0o666)
+        assert try_as(other_gid) == 'error:forbidden'
+        assert try_as(allowed_gid) == '{"pong": true}'

test/test_run_loop_golden.py

@@ -0,0 +1,222 @@
+"""Golden traces of DisplayController.run(): what is shown, for how long, and why.
+
+Each scenario runs the real run() loop against fake plugins on a fake clock
+(see test/_run_loop_harness.py) and compares the screens it produced with
+test/fixtures/run_loop_golden/<scenario>.json. A trace row is
+
+    [start_s, mode, duration_s, exit_reason, frames, force_clear_on_first_frame]
+
+and ``events`` lists what else happened (requests, live changes, schedule,
+brightness) with its time.
+
+These pin down today's behaviour so run() can be restructured into an
+Arbiter / ScreenRunner / Sources (docs/RUN_LOOP_REDESIGN.md) without changing
+it. A diff here is a behaviour change: if it is intended, regenerate with
+LEDMATRIX_REGEN_GOLDEN=1 and explain the change in the commit message.
+"""
+
+import os
+
+import pytest
+
+os.environ.setdefault("EMULATOR", "true")
+
+from test._run_loop_harness import (  # noqa: E402
+    FakePlugin,
+    LegacyFakePlugin,
+    RunLoopHarness,
+    check_golden,
+)
+
+
+def scenario_plain_rotation(h: RunLoopHarness):
+    # clock: duration from display_durations, which beats the plugin's own.
+    # weather: the plugin's own duration. ticker: scrolls, so high-FPS.
+    # legacy: display() without display_mode.
+    h.config["display"]["display_durations"] = {"clock": 15}
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=99))
+    h.add_plugin(FakePlugin("weather", ["weather_now", "weather_forecast"], duration=20))
+    h.add_plugin(FakePlugin("ticker", ["ticker"], duration=10, enable_scrolling=True))
+    h.add_plugin(LegacyFakePlugin("legacy", ["legacy"], duration=5))
+
+
+def scenario_empty_modes(h: RunLoopHarness):
+    # empty: never has content, skipped at once. ghost: a mode with no
+    # plugin behind it. flaky: content on the first frame only, so the
+    # 1 s loop breaks early and the dwell is made up by sleeping.
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=10))
+    h.add_plugin(FakePlugin("empty", ["empty"], duration=10, content=lambda t, m: False))
+    h.add_mode_without_plugin("ghost")
+    h.add_plugin(FakePlugin("flaky", ["flaky"], duration=12, first_frame_only=True))
+
+
+def scenario_all_empty(h: RunLoopHarness):
+    # Nothing to show anywhere: one rotation of empty passes, then a 1 s
+    # pause per pass instead of a spin.
+    h.add_plugin(FakePlugin("a", ["a"], content=lambda t, m: False))
+    h.add_plugin(FakePlugin("b", ["b"], content=lambda t, m: False))
+    h.add_plugin(FakePlugin("c", ["c"], content=lambda t, m: t >= 6))
+
+
+def scenario_plugin_error(h: RunLoopHarness):
+    # broken's dispatch raises (no display lock: loading failed part-way),
+    # so all its modes are skipped together; two failures open the breaker.
+    # crashy's display() raises inside the executor: an empty pass
+    # ("raised") that also counts as a breaker failure, so after two raises
+    # it is skipped by the breaker. Its modes are not skipped together.
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=10))
+    h.add_plugin(FakePlugin("broken", ["broken_a", "broken_b"], duration=10), lock=False)
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=10))
+    h.add_plugin(FakePlugin("crashy", ["crashy"], duration=10, raises=True))
+
+
+def scenario_dynamic_duration(h: RunLoopHarness):
+    # Read once at startup, so set where __init__ left it.
+    h.controller.global_dynamic_config = {"max_duration_seconds": 50}
+    # scroller: high-FPS, completes its cycle 20 s after each reset.
+    h.add_plugin(FakePlugin("scroller", ["scroller"], duration=10, needs_high_fps=True,
+                            dynamic={"cap": None, "complete_after": 20}))
+    # news: 1 s loop, asks for 45 s but its own cap is 40; never completes.
+    h.add_plugin(FakePlugin("news", ["news"], duration=10,
+                            dynamic={"cap": 40, "cycle": 45, "complete_after": None}))
+    # board: no cap of its own, so the global 50 s applies; done after 5 s,
+    # but the 10 s minimum (+0.5 s grace) holds it.
+    h.add_plugin(FakePlugin("board", ["board"], duration=10,
+                            dynamic={"cap": None, "complete_after": 5}))
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=10))
+
+
+def scenario_live_priority(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+    h.add_plugin(FakePlugin(
+        "sports", ["sports_recent", "sports_live"], duration=20,
+        live=(50, 110), live_priority=True,
+        content=lambda t, mode: mode != "sports_live" or 50 <= t < 110))
+
+
+def scenario_live_round_robin(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=15))
+    h.add_plugin(FakePlugin("nfl", ["nfl_live"], duration=15, live=(0, 70), live_priority=True))
+    h.add_plugin(FakePlugin("nhl", ["nhl_live"], duration=15, live=(20, 100), live_priority=True))
+
+
+def scenario_on_demand(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+    h.add_plugin(FakePlugin("sports", ["sports_recent", "sports_upcoming"], duration=15))
+    # Mid-way through clock's first screen; then stopped by request.
+    h.on_demand_request(25, "r1", plugin_id="sports")
+    h.on_demand_request(95, "r2", action="stop")
+    # A timed request that expires on its own.
+    h.on_demand_request(150, "r3", plugin_id="weather", duration=30)
+
+
+def scenario_on_demand_pinned(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("sports", ["sports_recent", "sports_upcoming"], duration=15))
+    h.on_demand_request(12, "p1", plugin_id="sports", mode="sports_upcoming", pinned=True)
+    # An on-demand mode with nothing to show is skipped like any other.
+    h.add_plugin(FakePlugin("starlark", ["app_a", "app_b"], duration=10,
+                            content=lambda t, mode: mode != "app_a"))
+    h.on_demand_request(80, "p2", plugin_id="starlark")
+    h.on_demand_request(120, "p3", action="stop")
+
+
+def scenario_on_demand_restored(h: RunLoopHarness):
+    # A restart during an on-demand session resumes it: the first screen is
+    # the saved mode (with a full clear), not the rotation's first mode, and
+    # the rotation starts from the top once it expires.
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+    h.add_plugin(FakePlugin("sports", ["sports_recent", "sports_upcoming"], duration=15))
+    h.restore_on_demand("sports", mode="sports_upcoming", duration=40)
+
+
+def scenario_schedule(h: RunLoopHarness):
+    # The clock starts at 22:59:30. Off from 23:01 until 23:05 (the window
+    # spans midnight); dimmed from 23:00 until 23:01.
+    h.config["schedule"] = {"enabled": True, "start_time": "23:05", "end_time": "23:01"}
+    h.config["dim_schedule"] = {"enabled": True, "start_time": "23:00",
+                                "end_time": "23:01", "dim_brightness": 30}
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+    # An on-demand request during scheduled downtime overrides it; when it
+    # expires the panel blanks at once, not at the next minute.
+    h.on_demand_request(170, "s1", plugin_id="weather", duration=20)
+
+
+def scenario_wifi_notice(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+    # Posted mid-screen: it preempts the screen at its next frame, stays up
+    # until it expires, and the interrupted mode then comes back in full.
+    h.wifi_message(25, "Connected to HomeNet", duration=5)
+    # While on-demand is active the notice waits.
+    h.on_demand_request(60, "w1", plugin_id="clock", duration=20)
+    h.wifi_message(65, "AP mode on", duration=30)
+
+
+def scenario_follower(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+    # Only checked at the top of a pass, so it takes over when the screen
+    # running at t=35 ends, and hands back the pass after it ends.
+    h.sync.follower_windows = [(35, 50)]
+
+
+def scenario_vegas(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin(
+        "sports", ["sports_live"], duration=20, live=(70, 100), live_priority=True,
+        content=lambda t, mode: 70 <= t < 100))
+    h.enable_vegas(cycle=30)
+    # On-demand takes the panel from Vegas mid-iteration, then hands back.
+    h.on_demand_request(150, "v1", plugin_id="clock", duration=25)
+    h.wifi_message(200, "Connected to HomeNet", duration=3)
+
+
+def scenario_vegas_live_in_ticker(h: RunLoopHarness):
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20, live=(10, 50),
+                            live_priority=True))
+    h.enable_vegas(cycle=30, live_in_ticker=True)
+
+
+SCENARIOS = {
+    "plain_rotation": (scenario_plain_rotation, 160),
+    "empty_modes": (scenario_empty_modes, 90),
+    "all_empty": (scenario_all_empty, 12),
+    "plugin_error": (scenario_plugin_error, 90),
+    "dynamic_duration": (scenario_dynamic_duration, 220),
+    "live_priority": (scenario_live_priority, 200),
+    "live_round_robin": (scenario_live_round_robin, 150),
+    "on_demand": (scenario_on_demand, 240),
+    "on_demand_pinned": (scenario_on_demand_pinned, 160),
+    "on_demand_restored": (scenario_on_demand_restored, 100),
+    "schedule": (scenario_schedule, 400),
+    "wifi_notice": (scenario_wifi_notice, 150),
+    "follower": (scenario_follower, 80),
+    "vegas": (scenario_vegas, 260),
+    "vegas_live_in_ticker": (scenario_vegas_live_in_ticker, 100),
+}
+
+
+@pytest.mark.parametrize("name", sorted(SCENARIOS))
+def test_run_loop_golden_trace(name, tmp_path):
+    build, horizon = SCENARIOS[name]
+    harness = RunLoopHarness(tmp_path, horizon=horizon)
+    build(harness)
+    trace = harness.run()
+    check_golden(name, trace)
+
+
+def test_traces_are_repeatable(tmp_path):
+    """Two runs of the busiest scenario give the identical trace."""
+    traces = []
+    for i in range(2):
+        (tmp_path / str(i)).mkdir()
+        harness = RunLoopHarness(tmp_path / str(i), horizon=240)
+        scenario_on_demand(harness)
+        traces.append(harness.run())
+    assert traces[0] == traces[1]

test/test_run_loop_live_takeover.py

@@ -0,0 +1,218 @@
+"""Live priority takes the panel promptly, through the real run() loop.
+
+Two behaviours the golden traces recorded (docs/RUN_LOOP_REDESIGN.md):
+
+* A game that went live mid-screen waited for that screen to end. Now the
+  frame loops and the dwell sleep check, at most once a second, and switch.
+* When Vegas yielded to live content, one rotation screen showed before the
+  game. Now the game is what shows next.
+
+These run the real DisplayController.run() on the fake clock from
+test/_run_loop_harness.py. Each trace row is
+[start, mode, duration, exit_reason, frames, force_clear].
+"""
+
+import os
+
+os.environ.setdefault("EMULATOR", "true")
+
+from test._run_loop_harness import FakePlugin, RunLoopHarness  # noqa: E402
+
+
+def _run(tmp_path, horizon, build):
+    harness = RunLoopHarness(tmp_path, horizon=horizon)
+    build(harness)
+    return harness, harness.run()["screens"]
+
+
+def _first(rows, mode):
+    return next(row for row in rows if row[1] == mode)
+
+
+def _counting(plugin):
+    """Count has_live_content() calls, with the fake-clock time of each."""
+    calls = []
+    real = plugin.has_live_content
+
+    def has_live_content():
+        calls.append(plugin._h.clock.rel())
+        return real()
+    plugin.has_live_content = has_live_content
+    return calls
+
+
+class TestMidScreenTakeover:
+    def test_live_game_cuts_a_one_hz_screen_short(self, tmp_path):
+        def build(h):
+            h.add_plugin(FakePlugin("clock", ["clock"], duration=30))
+            h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20,
+                                    live=(12.5, 100), live_priority=True))
+        _, rows = _run(tmp_path, 60, build)
+        clock = rows[0]
+        assert clock[1] == "clock" and clock[3] == "live"
+        live = _first(rows, "sports_live")
+        # Taken over at the first check after 12.5 s, not at 30 s.
+        assert 12.5 <= live[0] <= 13.5
+
+    def test_live_game_cuts_a_scrolling_screen_short(self, tmp_path):
+        def build(h):
+            h.add_plugin(FakePlugin("ticker", ["ticker"], duration=30, needs_high_fps=True))
+            h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20,
+                                    live=(7.2, 100), live_priority=True))
+        _, rows = _run(tmp_path, 40, build)
+        assert rows[0][1] == "ticker" and rows[0][3] == "live"
+        assert 7.2 <= _first(rows, "sports_live")[0] <= 8.3
+
+    def test_live_game_cuts_a_make_up_dwell_short(self, tmp_path):
+        # display() returns False after the first frame, so the 1 Hz loop
+        # breaks and the rest of the 30 s is a dwell sleep.
+        def build(h):
+            h.add_plugin(FakePlugin("flaky", ["flaky"], duration=30, first_frame_only=True))
+            h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20,
+                                    live=(10, 100), live_priority=True))
+        _, rows = _run(tmp_path, 50, build)
+        assert rows[0][1] == "flaky"
+        assert 10 <= _first(rows, "sports_live")[0] <= 11
+
+    def test_on_demand_is_never_preempted(self, tmp_path):
+        def build(h):
+            h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+            h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+            h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20,
+                                    live=(10, 200), live_priority=True))
+            h.on_demand_request(2, "od", plugin_id="weather", duration=40)
+        _, rows = _run(tmp_path, 60, build)
+        on_demand = [row for row in rows if 2 <= row[0] < 42]
+        assert on_demand and all(row[1] == "weather" for row in on_demand)
+        # Not even interrupted and restarted: each on-demand screen runs out.
+        assert all(row[3] != "live" for row in on_demand)
+        assert on_demand[0][2] == 20.0
+        # Once the session expires, the live game takes over.
+        after = [row for row in rows if row[0] >= 42]
+        assert after[0][1] == "sports_live"
+
+    def test_simultaneous_games_still_take_turns(self, tmp_path):
+        # Both go live during the clock screen. The takeover shows the first
+        # one; the next pass must not advance the round-robin past it.
+        def build(h):
+            h.add_plugin(FakePlugin("clock", ["clock"], duration=30))
+            h.add_plugin(FakePlugin("nfl", ["nfl_live"], duration=15,
+                                    live=(10, 200), live_priority=True))
+            h.add_plugin(FakePlugin("nhl", ["nhl_live"], duration=15,
+                                    live=(10, 200), live_priority=True))
+        _, rows = _run(tmp_path, 75, build)
+        modes = [row[1] for row in rows]
+        assert modes[:5] == ["clock", "nfl_live", "nhl_live", "nfl_live", "nhl_live"]
+        assert 10 <= rows[1][0] <= 11
+
+
+class TestTakeoverCheck:
+    """_check_live_takeover() on its own, on a controller built by the harness."""
+
+    def _controller(self, tmp_path, current="clock"):
+        h = RunLoopHarness(tmp_path, horizon=10)
+        h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+        sports = h.add_plugin(FakePlugin("sports", ["sports_recent", "sports_live"],
+                                         duration=20, live=(0, 100), live_priority=True))
+        dc = h.controller
+        dc.current_display_mode = current
+        dc.current_mode_index = dc.available_modes.index(current)
+        return h, dc, _counting(sports)
+
+    def test_switches_to_the_live_mode(self, tmp_path):
+        _, dc, calls = self._controller(tmp_path)
+        dc._check_live_takeover()
+        assert dc.current_display_mode == "sports_live"
+        assert dc.force_change is True
+        assert dc._live_takeover_unshown is True
+        # The rotation resumes from the screen that was cut short.
+        assert dc._live_resume_index == 0
+        assert len(calls) == 1  # once per plugin, not per mode key
+
+    def test_on_demand_session_is_left_alone(self, tmp_path):
+        _, dc, calls = self._controller(tmp_path)
+        dc.on_demand_active = True
+        dc._check_live_takeover()
+        assert dc.current_display_mode == "clock"
+        assert calls == []
+
+    def test_scheduled_off_is_left_alone(self, tmp_path):
+        _, dc, calls = self._controller(tmp_path)
+        dc.is_display_active = False
+        dc._check_live_takeover()
+        assert dc.current_display_mode == "clock"
+        assert calls == []
+
+    def test_vegas_keeping_live_in_the_ticker_is_left_alone(self, tmp_path):
+        h, dc, calls = self._controller(tmp_path)
+        h.enable_vegas(live_in_ticker=True)
+        dc._check_live_takeover()
+        assert dc.current_display_mode == "clock"
+        assert calls == []
+
+    def test_live_screen_already_showing_is_not_rescanned(self, tmp_path):
+        _, dc, calls = self._controller(tmp_path, current="sports_live")
+        dc._collect_live_modes()  # the scan that put the live mode up
+        dc._last_live_scan = None  # throttle out of the way
+        dc._check_live_takeover()
+        assert dc.current_display_mode == "sports_live"
+        assert dc._live_takeover_unshown is False
+        assert len(calls) == 1
+
+
+class TestLiveContentPollingCost:
+    def test_at_most_once_a_second_during_a_rotation_screen(self, tmp_path):
+        holder = {}
+
+        def build(h):
+            h.add_plugin(FakePlugin("clock", ["clock"], duration=30, needs_high_fps=True))
+            # Two mode keys on one plugin: still asked once per scan.
+            sports = h.add_plugin(FakePlugin("sports", ["sports_recent", "sports_live"],
+                                             duration=20, live_priority=True,
+                                             content=lambda t, m: m != "sports_live"))
+            holder["calls"] = _counting(sports)
+        _run(tmp_path, 29, build)
+        calls = holder["calls"]
+        # A 125 Hz screen, 29 s long: about one scan a second, never two
+        # within a second of each other.
+        assert len(calls) <= 30
+        assert all(b - a >= 0.99 for a, b in zip(calls, calls[1:]))
+
+    def test_not_rescanned_while_a_live_game_is_showing(self, tmp_path):
+        holder = {}
+
+        def build(h):
+            h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+            sports = h.add_plugin(FakePlugin("sports", ["sports_live"], duration=30,
+                                             live=(0, 200), live_priority=True))
+            holder["calls"] = _counting(sports)
+        _, rows = _run(tmp_path, 90, build)
+        assert all(row[1] == "sports_live" for row in rows)
+        # Per 30 s live screen: the scan before it and the hold check after
+        # it, as before -- nothing from inside the screen.
+        assert len(holder["calls"]) <= 2 * len(rows)
+
+
+class TestVegasYieldsToLive:
+    def test_live_game_shows_next_without_a_rotation_screen(self, tmp_path):
+        def build(h):
+            h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+            h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+            h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20,
+                                    live=(40, 200), live_priority=True))
+            h.enable_vegas(cycle=30)
+        _, rows = _run(tmp_path, 80, build)
+        assert rows[0][1] == "<vegas>"
+        yielded = next(i for i, row in enumerate(rows) if row[3] == "vegas-live")
+        nxt = rows[yielded + 1]
+        assert nxt[1] == "sports_live"
+        assert nxt[0] == rows[yielded][0] + rows[yielded][2]
+
+    def test_live_in_ticker_keeps_the_ticker(self, tmp_path):
+        def build(h):
+            h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+            h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20,
+                                    live=(10, 200), live_priority=True))
+            h.enable_vegas(cycle=30, live_in_ticker=True)
+        _, rows = _run(tmp_path, 70, build)
+        assert all(row[1] == "<vegas>" for row in rows)

test/test_run_loop_wifi_and_live.py

@@ -0,0 +1,157 @@
+"""A WiFi notice and a live game that both want the panel at once.
+
+The two preempt the current screen independently (_wifi_notice_pending and
+_check_live_takeover, both polled from the frame loops and the dwell sleep),
+so these pin down how they combine. The documented priority is follower,
+on-demand, WiFi, live, Vegas, rotation: the notice shows first, then the
+game, with no rotation screen in between, and the scheduled-off panel shows
+neither. Runs the real run() loop on the fake clock of
+test/_run_loop_harness.py. Each trace row is
+[start, mode, duration, exit_reason, frames, force_clear].
+"""
+
+import os
+
+import pytest
+
+os.environ.setdefault("EMULATOR", "true")
+
+from test._run_loop_harness import FakePlugin, RunLoopHarness  # noqa: E402
+
+
+def _run(tmp_path, horizon, build):
+    harness = RunLoopHarness(tmp_path, horizon=horizon)
+    build(harness)
+    return harness.run()
+
+
+def _sports(h, live, **kwargs):
+    h.add_plugin(FakePlugin("sports", ["sports_live"], duration=20, live=live,
+                            live_priority=True, **kwargs))
+
+
+def _notice_then_game(rows, after, posted, expires):
+    """Check the rows from index `after` on: notice, then game, nothing else.
+
+    The notice is up within about a second of being posted and stays up
+    until it expires (it may be redrawn across pass boundaries, so it can
+    span several rows). The game follows it directly.
+    """
+    wifi = []
+    i = after
+    while rows[i][1] == "<wifi>":
+        wifi.append(rows[i])
+        i += 1
+    assert wifi, rows
+    assert posted <= wifi[0][0] <= posted + 1.25
+    # Continuous: each notice row starts where the one before it ended.
+    for prev, cur in zip(wifi, wifi[1:]):
+        assert cur[0] == pytest.approx(prev[0] + prev[2])
+    assert wifi[-1][0] + wifi[-1][2] >= expires
+    game = rows[i]
+    assert game[1] == "sports_live"
+    assert game[0] == pytest.approx(wifi[-1][0] + wifi[-1][2])
+    return i
+
+
+@pytest.mark.parametrize("live_at, wifi_at", [(10.2, 10.4), (10.4, 10.2)],
+                         ids=["game-first", "notice-first"])
+def test_both_during_a_static_screen(tmp_path, live_at, wifi_at):
+    def build(h):
+        h.add_plugin(FakePlugin("clock", ["clock"], duration=30))
+        h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+        _sports(h, (live_at, 60))
+        h.wifi_message(wifi_at, "Connected to HomeNet", duration=5)
+    trace = _run(tmp_path, 90, build)
+    rows = trace["screens"]
+
+    # The 1 Hz loop's next check after both arrive ends clock's screen.
+    assert rows[0][1] == "clock" and rows[0][2] <= 11.0
+    _notice_then_game(rows, 1, wifi_at, wifi_at + 5)
+    # The game is never on the panel before the notice.
+    first_game = next(row for row in rows if row[1] == "sports_live")
+    first_wifi = next(row for row in rows if row[1] == "<wifi>")
+    assert first_wifi[0] < first_game[0]
+    # Once the game ends, the rotation resumes at the screen it cut short.
+    after_game = next(row for row in rows if row[0] >= 60 and row[1] != "sports_live")
+    assert after_game[1] == "clock"
+
+
+def test_both_at_once_during_a_scrolling_screen(tmp_path):
+    def build(h):
+        h.add_plugin(FakePlugin("ticker", ["ticker"], duration=30, enable_scrolling=True))
+        h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+        _sports(h, (10.0, 60))
+        h.wifi_message(10.0, "AP mode on", duration=4)
+    rows = _run(tmp_path, 90, build)["screens"]
+
+    assert rows[0][1] == "ticker" and rows[0][0] + rows[0][2] <= 11.0
+    _notice_then_game(rows, 1, 10.0, 14.0)
+    assert "weather" not in [row[1] for row in rows if row[0] < 60]
+
+
+def test_vegas_yields_to_both_with_no_rotation_screen(tmp_path):
+    def build(h):
+        h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+        _sports(h, (40, 70), content=lambda t, mode: 40 <= t < 70)
+        h.enable_vegas(cycle=30)
+        h.wifi_message(40, "Connected to HomeNet", duration=3)
+    trace = _run(tmp_path, 110, build)
+    rows = trace["screens"]
+
+    yielded = next(i for i, row in enumerate(rows)
+                   if row[1] == "<vegas>" and row[3] in ("vegas-live", "vegas-interrupt"))
+    # Vegas's live check (4 Hz) can see the game before the notice file's
+    # 1 Hz stat sees the notice; then the game is up for at most a second
+    # before the notice preempts it.
+    i = yielded + 1
+    if rows[i][1] == "sports_live":
+        assert rows[i][2] <= 1.0 and rows[i][3] == "wifi"
+        i += 1
+    i = _notice_then_game(rows, i, 40, 43)
+    # Neither the rotation nor the ticker runs while the game is live.
+    during = [row[1] for row in rows[yielded + 1:] if row[0] < 70]
+    assert set(during) <= {"sports_live", "<wifi>"}
+    assert rows[-1][1] == "<vegas>"
+
+
+def test_vegas_stopped_for_a_game_shows_a_known_notice_first(tmp_path):
+    # The notice is already posted when Vegas stops for the game: each Vegas
+    # frame runs its live check before its interrupt check, so the game can
+    # be what stops it. Without the interrupt check the notice is only
+    # learned after the yield, which pins the order the yield path checks
+    # them in: the notice first.
+    def build(h):
+        h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+        _sports(h, (40, 70), content=lambda t, mode: 40 <= t < 70)
+        vegas = h.enable_vegas(cycle=30)
+        vegas.set_interrupt_checker(lambda: False)
+        h.wifi_message(39.5, "Connected to HomeNet", duration=4)
+    rows = _run(tmp_path, 110, build)["screens"]
+
+    yielded = next(i for i, row in enumerate(rows) if row[3] == "vegas-live")
+    assert 40.0 <= rows[yielded][0] + rows[yielded][2] <= 40.3
+    _notice_then_game(rows, yielded + 1, 40.0, 43.5)
+
+
+def test_scheduled_off_shows_neither(tmp_path):
+    # The harness clock starts at 22:59:30: off from 23:00 (t=30) to 23:05
+    # (t=330). The notice and the game both arrive at t=120, well inside it
+    # (whichever way the end minute is counted).
+    def build(h):
+        h.config["schedule"] = {"enabled": True, "start_time": "23:05", "end_time": "23:00"}
+        h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+        h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+        _sports(h, (120, 400))
+        h.wifi_message(120, "AP mode on", duration=30)
+    trace = _run(tmp_path, 380, build)
+    rows = trace["screens"]
+
+    off = next(i for i, row in enumerate(rows) if row[1] == "<off>")
+    assert rows[off][0] <= 90.0
+    assert rows[off][0] + rows[off][2] == 330.0 and rows[off][3] == "schedule-on"
+    assert not any(row[1] == "<wifi>" for row in rows)
+    live_events = [e for e in trace["events"] if e[1] == "live"]
+    assert live_events and all(e[0] >= 330.0 for e in live_events)
+    # The game, still live when the panel comes back, is what shows.
+    assert rows[off + 1][1] == "sports_live"

test/test_scan_order.py

@@ -73,6 +73,27 @@ def _frame(shade):
     return Image.new("RGB", (8, 64), (shade, shade, shade))
 
 
+class TestRefreshPlan:
+    BANDS = [(32, 64, 1)]
+
+    def test_one_refresh_per_frame_is_a_plain_lag(self):
+        assert scan_order.refresh_plan(self.BANDS, 1) == [((1,), 1)]
+
+    def test_a_held_frame_lags_only_its_first_refresh(self):
+        assert scan_order.refresh_plan(self.BANDS, 2) == [((1,), 1), ((0,), 1)]
+        assert scan_order.refresh_plan(self.BANDS, 5) == [((1,), 1), ((0,), 4)]
+
+    def test_a_lag_longer_than_the_hold_reaches_further_back(self):
+        # Three halves down a stack, held for two refreshes.
+        plan = scan_order.refresh_plan([(0, 8, 3)], 2)
+        assert plan == [((2,), 1), ((1,), 1)]
+
+    def test_every_refresh_of_the_frame_is_accounted_for(self):
+        for hold in range(1, 9):
+            plan = scan_order.refresh_plan([(0, 8, 1), (8, 16, 2)], hold)
+            assert sum(count for _, count in plan) == hold
+
+
 class TestCompose:
     def test_a_band_comes_from_the_frame_that_many_refreshes_back(self):
         now, previous = _frame(30), _frame(20)
@@ -133,10 +154,28 @@ class TestUpdateDisplay:
         assert shown.getpixel((0, 0)) == (20, 0, 0)
         assert shown.getpixel((0, 31)) == (10, 0, 0)
 
-    def test_not_on_a_static_screen_or_a_held_frame(self, dm):
+    def test_not_on_a_static_screen(self, dm):
         dm._scan_lag_bands = [(16, 32, 1)]
         self._push(dm, 10)
         assert self._push(dm, 20).getpixel((0, 31)) == (20, 0, 0)   # not scrolling
+
+    def test_a_held_frame_is_split_so_the_lagging_half_steps_a_refresh_late(self, dm):
+        dm._scan_lag_bands = [(16, 32, 1)]
         dm.set_scrolling_state(True, 2)
-        self._push(dm, 30)
-        assert self._push(dm, 40).getpixel((0, 31)) == (40, 0, 0)   # hold 2
+        self._push(dm, 10)
+        before = len(dm._presented)
+        self._push(dm, 20)
+        first, second = dm._presented[before:]
+        assert first.getpixel((0, 0)) == (20, 0, 0)
+        assert first.getpixel((0, 31)) == (10, 0, 0)     # lagging half: still old
+        assert second.getpixel((0, 31)) == (20, 0, 0)    # caught up a refresh later
+
+    def test_a_slow_blit_is_not_split(self, dm):
+        dm._scan_lag_bands = [(16, 32, 1)]
+        dm.set_scrolling_state(True, 3)
+        self._push(dm, 10)
+        dm._last_blit_seconds = 1.0   # far longer than a refresh
+        before = len(dm._presented)
+        self._push(dm, 20)
+        assert len(dm._presented) - before == 1
+        assert dm._presented[-1].getpixel((0, 31)) == (20, 0, 0)

test/test_scroll_config.py

@@ -13,6 +13,7 @@ from src.common.scroll_config import (  # noqa: E402
     MAX_PIXELS_PER_FRAME,
     crisp_ladder,
     solve_crisp,
+    speed_advice,
     MAX_PIXELS_PER_SECOND,
     MIN_PIXELS_PER_SECOND,
     ScrollSettings,
@@ -465,3 +466,41 @@ class TestFrameHoldIsReportedNotApplied:
             display_manager=dm)
         assert dm.calls == [], "configure() must not apply the hold itself"
         assert settings.frame_hold == 4, "but it must report what to apply"
+
+
+class TestSpeedAdvice:
+    def test_default_speed_on_a_120hz_panel_is_not_left_stepped(self):
+        """50 px/s used to snap to 48 (2px every 5 refreshes, 24fps)."""
+        got = solve_crisp(50, 120)
+        assert got.steppiness == "smooth"
+        assert got.pixels_per_frame == 1
+
+    def test_unchanged_choices_on_a_100hz_panel(self):
+        assert solve_crisp(50, 100).pixels_per_second == pytest.approx(50.0)
+        assert solve_crisp(60, 100).pixels_per_second == pytest.approx(66.667, abs=0.01)
+
+    def test_smooth_exact_speed_needs_no_alternatives(self):
+        advice = speed_advice(60, 120)
+        assert advice["exact"] and advice["smooth"]
+        assert advice["alternatives"] == []
+
+    def test_off_ladder_speed_offers_the_nearest_smooth_ones(self):
+        advice = speed_advice(50, 120, 10, 200)
+        assert advice["applied"]["steppiness"] == "smooth"
+        offered = [a["pixels_per_second"] for a in advice["alternatives"]]
+        assert offered == [40.0, 60.0]
+        assert all(a["steppiness"] == "smooth" for a in advice["alternatives"])
+
+    def test_alternatives_stay_inside_the_requested_range(self):
+        advice = speed_advice(50, 120, 45, 200)
+        assert all(45 <= a["pixels_per_second"] <= 200 for a in advice["alternatives"])
+
+    def test_a_whole_number_near_the_panels_speed_counts_as_exact(self):
+        """The UI sends 63 for a 62.9 px/s panel."""
+        assert speed_advice(63, 125.74)["exact"]
+
+    def test_a_measured_rate_does_not_let_a_stepped_speed_through(self):
+        """125.74Hz: 50.3px/s is 2px every 5 refreshes at 25.1fps."""
+        got = solve_crisp(50, 125.74)
+        assert got.steppiness == "smooth"
+        assert got.pixels_per_frame == 1

test/test_scroll_helper_in_place.py

@@ -0,0 +1,146 @@
+"""ScrollHelper extends and trims a strip in place (src/common/scroll_helper.py).
+
+Every extension of the Vegas strip used to rebuild it whole (np.concatenate)
+and every trim copied what was left: 3.5-4.5 ms on the render thread at 512x64
+on a Pi 4, so the frame after each extension was late. The strip now lives in
+a buffer with spare room: an append writes only the new columns, a trim only
+moves the view's start, and a full copy happens only when the buffer is
+reallocated. What a frame shows must not change at all.
+"""
+import random
+import sys
+from pathlib import Path
+
+import numpy as np
+import pytest
+from PIL import Image
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from src.common.scroll_helper import ScrollHelper  # noqa: E402
+
+W, H = 64, 16
+
+
+def _items(rng, n):
+    out = []
+    for _ in range(n):
+        width = rng.randint(5, 60)
+        out.append(Image.frombytes("RGB", (width, H),
+                                   bytes(rng.randrange(256) for _ in range(width * H * 3))))
+    return out
+
+
+def _helper():
+    helper = ScrollHelper(W, H)
+    helper.create_scrolling_image(_items(random.Random(1), 4), item_gap=3, lead_gap=0)
+    return helper
+
+
+def _reference_append(strip, items, gap):
+    """What append_content used to do."""
+    width = sum(i.width for i in items) + gap * len(items)
+    addition = Image.new("RGB", (width, H))
+    x = 0
+    for item in items:
+        x += gap
+        addition.paste(item, (x, 0))
+        x += item.width
+    return np.concatenate((strip, np.array(addition)), axis=1)
+
+
+def test_an_append_writes_into_the_buffer_and_copies_only_the_new_columns():
+    helper = _helper()
+    helper.append_content(_items(random.Random(2), 2), item_gap=3)    # allocates
+    buffer = helper._strip_buffer
+    before = helper.cached_array.shape[1]
+    items = _items(random.Random(3), 2)
+    helper.append_content(items, item_gap=3)
+    assert helper._strip_buffer is buffer
+    assert np.shares_memory(helper.cached_array, buffer)
+    added = helper.cached_array.shape[1] - before
+    assert helper.last_copy_bytes == added * H * 3
+
+
+def test_a_trim_copies_nothing():
+    helper = _helper()
+    helper.append_content(_items(random.Random(2), 3), item_gap=3)
+    helper.scroll_position = 120.0
+    cut = helper.drop_scrolled_prefix()
+    assert cut == 120 and helper.last_copy_bytes == 0
+    assert np.shares_memory(helper.cached_array, helper._strip_buffer)
+
+
+def test_the_buffer_is_reallocated_when_the_room_runs_out():
+    helper = _helper()
+    helper.append_content(_items(random.Random(2), 1), item_gap=3)
+    first = helper._strip_buffer
+    rng = random.Random(4)
+    while helper._strip_buffer is first:
+        helper.append_content(_items(rng, 3), item_gap=3)
+    assert helper.last_copy_bytes == helper.cached_array.nbytes
+    assert helper._strip_start == 0
+
+
+def test_a_strip_set_from_outside_is_never_written_through():
+    # The multi-display follower adopts a read-only array straight from an image.
+    helper = _helper()
+    helper.append_content(_items(random.Random(2), 1), item_gap=3)
+    adopted = np.asarray(Image.new("RGB", (300, H), (9, 9, 9)))
+    helper.cached_array = adopted
+    helper.total_scroll_width = 300
+    helper.append_content(_items(random.Random(3), 1), item_gap=3)
+    assert not np.shares_memory(helper.cached_array, adopted)
+    assert (adopted == 9).all()
+    helper.cached_array = adopted
+    helper.scroll_position = 100.0
+    helper.drop_scrolled_prefix()
+    assert not np.shares_memory(helper.cached_array, adopted)
+
+
+def test_a_new_strip_lets_the_old_buffer_go():
+    helper = _helper()
+    helper.append_content(_items(random.Random(2), 1), item_gap=3)
+    helper.create_scrolling_image(_items(random.Random(5), 2), item_gap=3, lead_gap=0)
+    assert helper._strip_buffer is None
+    helper.append_content(_items(random.Random(2), 1), item_gap=3)
+    helper.clear_cache()
+    assert helper._strip_buffer is None and helper._strip_view is None
+
+
+@pytest.mark.parametrize("seed", range(12))
+def test_every_frame_matches_the_old_copying_strip(seed):
+    """Random appends, trims, scrolling and patches, against a strip kept the old way."""
+    rng = random.Random(seed)
+    helper = _helper()
+    reference = helper.cached_array.copy()
+    for _ in range(60):
+        op = rng.random()
+        if op < 0.35:
+            items = _items(rng, rng.randint(1, 3))
+            gap = rng.randint(0, 6)
+            helper.append_content(items, item_gap=gap)
+            reference = _reference_append(reference, items, gap)
+        elif op < 0.55:
+            keep = rng.randint(0, W)
+            before = helper.scroll_position
+            cut = helper.drop_scrolled_prefix(keep_before=keep)
+            reference = reference[:, cut:].copy()
+            assert helper.scroll_position == before - cut
+        elif op < 0.7 and helper.cached_array.shape[1] > 8:
+            x = rng.randrange(helper.cached_array.shape[1] - 4)
+            pixels = np.full((H, 4, 3), rng.randrange(256), dtype=np.uint8)
+            helper.patch_columns(x, pixels)
+            reference[:, x:x + 4] = pixels
+        else:
+            limit = max(0, helper.cached_array.shape[1] - W - 1)
+            helper.scroll_position = float(rng.randint(0, limit)) if limit else 0.0
+        assert helper.cached_array.shape == reference.shape
+        assert (helper.cached_array == reference).all()
+        assert helper.total_scroll_width == reference.shape[1]
+        frame = np.asarray(helper.get_visible_portion())
+        x = int(helper.scroll_position)
+        if x + W <= reference.shape[1]:
+            assert (frame == reference[:, x:x + W]).all()
+    # The lazily built image is the strip as it stands.
+    assert (np.asarray(helper.cached_image) == reference).all()

test/test_sports_display_rules.py

@@ -0,0 +1,203 @@
+"""src.common.sports_display_rules: behaviour and host contract.
+
+Ported from the scoreboards' tests of the same methods (hockey's
+test_switch_show_date_time.py, baseball's test_recent_game_date.py, the
+test_non_favorite_live_duration.py copies), against stub hosts composed the
+way the plugins compose ``SportsCore``: the new mixins first, then
+``SportsCoreSharedMixin``.
+"""
+
+import ast
+from pathlib import Path
+
+import pytest
+
+from src.common import sports_display_rules
+from src.common.sports_display_rules import SportsCardOptionsMixin, SportsGameRulesMixin
+from src.common.sports_shared import SportsCoreSharedMixin
+
+
+class Core(SportsCardOptionsMixin, SportsGameRulesMixin, SportsCoreSharedMixin):
+    """A SportsCore stand-in in the documented base order."""
+
+    def __init__(self, scroll_card=None, favorites=(), non_fav=0, duration=15,
+                 passes=lambda g: True, quality="any"):
+        self.config = {"scroll_card": dict(scroll_card or {})}
+        self.favorite_teams = list(favorites)
+        self.non_favorite_live_game_duration = non_fav
+        self.game_display_duration = duration
+        self._passes = passes
+        self.other_games_min_quality = quality
+        self.coverage_checked = []
+
+    def _passes_other_filters(self, game):
+        return self._passes(game)
+
+    def _check_ranking_coverage(self, games):
+        self.coverage_checked.append(list(games))
+
+    def _is_favorite_game(self, game):
+        return game.get("home_abbr") in self.favorite_teams
+
+
+# ---------------------------------------------------------------------------
+# _card_option
+# ---------------------------------------------------------------------------
+
+class TestCardOption:
+    def test_ordinary_keys_read_through(self):
+        core = Core({"vs_text": "@"})
+        assert core._card_option("vs_text", "VS") == "@"
+        assert core._card_option("missing", 7) == 7
+
+    def test_both_lines_off_under_date_time_reads_as_both_on(self):
+        core = Core({"switch_show_date": False, "switch_show_time": False})
+        assert core._card_option("switch_show_date", True) is True
+        assert core._card_option("switch_show_time", True) is True
+
+    @pytest.mark.parametrize("center", ["vs", "none"])
+    def test_both_off_is_honoured_without_the_stack(self, center):
+        core = Core({"switch_show_date": False, "switch_show_time": False,
+                     "switch_upcoming_center": center})
+        assert core._card_option("switch_show_date", True) is False
+        assert core._card_option("switch_show_time", True) is False
+
+    def test_one_line_off_is_honoured(self):
+        core = Core({"switch_show_date": False, "switch_show_time": True})
+        assert core._card_option("switch_show_date", True) is False
+        assert core._card_option("switch_show_time", True) is True
+
+    def test_inherit_follows_the_card_center(self):
+        core = Core({"switch_show_date": False, "switch_show_time": False,
+                     "switch_upcoming_center": "inherit", "upcoming_center": "vs"})
+        assert core._card_option("switch_show_date", True) is False
+
+    def test_it_must_come_before_the_shared_mixin(self):
+        """In the other order the shared reader wins and the rescue is lost."""
+
+        class Wrong(SportsCoreSharedMixin, SportsCardOptionsMixin):
+            pass
+
+        wrong = Wrong()
+        wrong.config = {"scroll_card": {"switch_show_date": False, "switch_show_time": False}}
+        assert wrong._card_option("switch_show_date", True) is False
+        assert Core._card_option is SportsCardOptionsMixin._card_option
+
+    def test_works_lifted_onto_a_stand_in(self):
+        """Plugin tests lift it onto classes that are not SportsCore subclasses."""
+
+        class StandIn:
+            config = {"scroll_card": {"switch_show_date": False, "switch_show_time": False}}
+            _card_option = SportsCardOptionsMixin._card_option
+            _switch_upcoming_center = SportsCoreSharedMixin._switch_upcoming_center
+
+        assert StandIn()._card_option("switch_show_time", True) is True
+
+
+class TestRecentDateText:
+    def test_numeric_default_is_the_extractors_text(self):
+        assert Core()._recent_date_text({"game_date": "9/23"}) == "9/23"
+
+    def test_follows_switch_date_format(self):
+        core = Core({"switch_date_format": "abbrev"})
+        assert core._recent_date_text({"game_date": "9/23"}) == "Sep 23"
+
+    def test_the_off_switch(self):
+        core = Core({"switch_recent_show_date": False})
+        assert core._recent_date_text({"game_date": "9/23"}) == ""
+
+    @pytest.mark.parametrize("game", [None, {}, {"game_date": None}])
+    def test_no_date_is_empty(self, game):
+        assert Core()._recent_date_text(game) == ""
+
+
+# ---------------------------------------------------------------------------
+# _filtered_or_all
+# ---------------------------------------------------------------------------
+
+class TestFilteredOrAll:
+    def test_keeps_what_passes(self):
+        games = [{"id": 1, "ok": True}, {"id": 2, "ok": False}]
+        core = Core(passes=lambda g: g["ok"])
+        assert core._filtered_or_all(games) == [games[0]]
+
+    def test_fails_open_when_nothing_passes(self):
+        games = [{"id": 1}, {"id": 2}]
+        assert Core(passes=lambda g: False)._filtered_or_all(games) == games
+
+    def test_ranking_coverage_is_checked_on_every_game(self):
+        games = [{"id": 1}, {"id": 2}]
+        core = Core(passes=lambda g: g["id"] == 1)
+        core._filtered_or_all(games)
+        assert core.coverage_checked == [games]
+
+
+# ---------------------------------------------------------------------------
+# _effective_live_duration
+# ---------------------------------------------------------------------------
+
+class TestEffectiveLiveDuration:
+    FAV = {"home_abbr": "DAL"}
+    OTHER = {"home_abbr": "NYG"}
+
+    def test_non_favourite_gets_the_shorter_dwell(self):
+        core = Core(favorites=["DAL"], non_fav=5, duration=20)
+        assert core._effective_live_duration(self.OTHER) == 5
+        assert core._effective_live_duration(self.FAV) == 20
+
+    def test_no_favourites_means_one_duration(self):
+        assert Core(non_fav=5, duration=20)._effective_live_duration(self.OTHER) == 20
+
+    @pytest.mark.parametrize("knob", [0, None])
+    def test_the_knob_off(self, knob):
+        core = Core(favorites=["DAL"], non_fav=knob, duration=20)
+        assert core._effective_live_duration(self.OTHER) == 20
+
+    def test_no_game(self):
+        assert Core(favorites=["DAL"], non_fav=5, duration=20)._effective_live_duration(None) == 20
+
+    def test_a_host_without_the_knob(self):
+        core = Core(favorites=["DAL"], duration=20)
+        del core.non_favorite_live_game_duration
+        assert core._effective_live_duration(self.OTHER) == 20
+
+
+# ---------------------------------------------------------------------------
+# Host contract
+# ---------------------------------------------------------------------------
+
+def _self_reads(class_name):
+    tree = ast.parse(Path(sports_display_rules.__file__).read_text(encoding="utf-8"))
+    cls = next(n for n in tree.body if isinstance(n, ast.ClassDef) and n.name == class_name)
+    names = set()
+    for node in ast.walk(cls):
+        if (isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load)
+                and isinstance(node.value, ast.Name) and node.value.id == "self"):
+            names.add(node.attr)
+        if (isinstance(node, ast.Call) and isinstance(node.func, ast.Name)
+                and node.func.id == "getattr" and len(node.args) >= 2
+                and isinstance(node.args[0], ast.Name) and node.args[0].id == "self"
+                and isinstance(node.args[1], ast.Constant)):
+            names.add(node.args[1].value)
+    return names
+
+
+class TestHostContract:
+    @pytest.mark.parametrize("mixin", [SportsCardOptionsMixin, SportsGameRulesMixin])
+    def test_every_host_read_is_documented(self, mixin):
+        needed = _self_reads(mixin.__name__) - set(dir(mixin))
+        undocumented = sorted(n for n in needed if f"``{n}" not in sports_display_rules.__doc__)
+        assert undocumented == [], f"read but not in the host contract: {undocumented}"
+
+    def test_the_mixins_create_no_attributes(self):
+        for name in ("_format_game_date", "favorite_teams", "game_display_duration",
+                     "_passes_other_filters", "_check_ranking_coverage", "_is_favorite_game"):
+            assert not hasattr(SportsCardOptionsMixin, name)
+            assert not hasattr(SportsGameRulesMixin, name)
+        for mixin in (SportsCardOptionsMixin, SportsGameRulesMixin):
+            assert "__init__" not in vars(mixin)
+
+    def test_the_two_define_no_name_in_common(self):
+        a = {n for n in vars(SportsCardOptionsMixin) if not n.startswith("__")}
+        b = {n for n in vars(SportsGameRulesMixin) if not n.startswith("__")}
+        assert a & b == set()

test/test_sports_font_path.py

@@ -0,0 +1,120 @@
+"""src.common.sports_font_path: the plugins' ``_resolve_font_path``, path for path.
+
+The plugins' copy probes the core for ``FontManager._resolve_asset_path``
+and falls back to its own install-root join; ``resolve_font_path`` is what
+that comes to on a core that ships it. The bodies differ, so instead of an
+AST comparison this runs both on the same paths -- found in the cwd only,
+under the install root only, in both, absolute, and nowhere -- from a
+temporary cwd, and requires the same string back. The plugin copies are read
+from LEDMATRIX_PLUGINS (every sports.py and game_renderer.py that still has
+one); without it, the comparison is against the copy transcribed below.
+"""
+
+import ast
+import os
+from pathlib import Path
+
+import pytest
+
+from src.common.font_layout import resolve_asset_path
+from src.common.sports_font_path import resolve_font_path
+
+REPO = Path(__file__).resolve().parents[1]
+BUNDLED = "assets/fonts/PressStart2P-Regular.ttf"
+
+#: ledmatrix-plugins 56c4f15, plugins/*-scoreboard/sports.py (docstring and
+#: comments dropped). The same body is in every sports.py and game_renderer.py.
+TRANSCRIBED = '''
+def _resolve_font_path(path: str) -> str:
+    if os.path.exists(path):
+        return path
+    try:
+        import src.font_manager as _core_fonts
+        manager = getattr(_core_fonts, "FontManager", None)
+        resolver = getattr(manager, "_resolve_asset_path", None)
+        if resolver is not None:
+            resolved = resolver(path)
+            if resolved and os.path.exists(resolved):
+                return resolved
+        root = os.path.dirname(os.path.dirname(os.path.abspath(_core_fonts.__file__)))
+        candidate = os.path.join(root, path)
+        if os.path.exists(candidate):
+            return candidate
+    except (ImportError, AttributeError, OSError):
+        return path
+    return path
+'''
+
+
+def _compile(source: str):
+    namespace = {"os": os}
+    exec(compile(source, "<plugin copy>", "exec"), namespace)  # nosec B102 - test-only, source is a plugin file  # nosemgrep
+    return namespace["_resolve_font_path"]
+
+
+def _plugin_copies():
+    """(label, function) for every plugin copy, or the transcription."""
+    raw = os.environ.get("LEDMATRIX_PLUGINS")
+    root = Path(raw) if raw else None
+    if root is not None and (root / "plugins").is_dir():
+        root = root / "plugins"
+    copies = []
+    if root is not None and root.is_dir():
+        for path in sorted(root.glob("*-scoreboard/*.py")):
+            if path.name not in ("sports.py", "game_renderer.py"):
+                continue
+            tree = ast.parse(path.read_text(encoding="utf-8"))
+            for node in tree.body:
+                if isinstance(node, ast.FunctionDef) and node.name == "_resolve_font_path":
+                    copies.append((f"{path.parent.name}/{path.name}",
+                                   _compile(ast.unparse(node))))
+    if not copies:
+        copies.append(("transcribed", _compile(TRANSCRIBED)))
+    return copies
+
+
+COPIES = _plugin_copies()
+
+
+@pytest.fixture
+def elsewhere(tmp_path, monkeypatch):
+    """A cwd that is not the install root, holding one font of its own and a
+    shadow of a bundled one."""
+    (tmp_path / "cwd_only.ttf").write_bytes(b"x")
+    shadow = tmp_path / BUNDLED
+    shadow.parent.mkdir(parents=True)
+    shadow.write_bytes(b"x")
+    monkeypatch.chdir(tmp_path)
+    return tmp_path
+
+
+def _cases(cwd: Path):
+    return [
+        "cwd_only.ttf",                        # in the cwd only
+        BUNDLED,                               # in both: the cwd wins
+        "assets/fonts/4x6-font.ttf",           # under the install root only
+        str(REPO / BUNDLED),                   # absolute, exists
+        str(cwd / "missing.ttf"),              # absolute, missing
+        "assets/fonts/no-such-font.ttf",       # relative, nowhere
+        "",                                    # empty
+    ]
+
+
+@pytest.mark.parametrize("label,copy", COPIES, ids=[c[0] for c in COPIES])
+def test_same_answer_as_the_plugin_copy(label, copy, elsewhere):
+    for path in _cases(elsewhere):
+        assert resolve_font_path(path) == copy(path), (label, path)
+
+
+def test_the_cwd_comes_first(elsewhere):
+    assert resolve_font_path(BUNDLED) == BUNDLED
+    assert resolve_asset_path(BUNDLED) != BUNDLED      # what dropping it would change
+
+
+def test_the_install_root_after_it(elsewhere):
+    found = resolve_font_path("assets/fonts/4x6-font.ttf")
+    assert Path(found).is_absolute() and Path(found).is_file()
+
+
+def test_nowhere_comes_back_unchanged(elsewhere):
+    assert resolve_font_path("assets/fonts/no-such-font.ttf") == "assets/fonts/no-such-font.ttf"

test/test_sports_live_scroll.py

@@ -0,0 +1,349 @@
+"""src.common.sports_live_scroll: behaviour and host contract.
+
+Ported from the eight scoreboards' test_live_scroll_refresh.py, against a
+stub host carrying only the documented contract: what counts as a change
+(the clock and the display pipeline's decoration do not), the rate limit and
+its duty-cycle scaling, the marquee keeping its place across a rebuild, both
+manager shapes (a league registry; afl/nrl's ``_get_manager``), and the
+refresh that runs before the fingerprint.
+"""
+
+import ast
+import threading
+import time
+from pathlib import Path
+
+import pytest
+
+from src.common import sports_live_scroll
+from src.common.sports_live_scroll import SportsLiveScrollMixin
+from src.common.sports_plugin_host import SportsPluginHostMixin
+
+KEY = "live"
+
+
+def game(gid="1", home="2", away="1", **extra):
+    g = {"id": gid, "home_score": home, "away_score": away,
+         "period": 3, "period_text": "3rd",
+         "clock": "12:04", "status_text": "12:04 - 3rd",
+         "is_final": False, "is_halftime": False,
+         "home_abbr": "AAA", "away_abbr": "BBB"}
+    g.update(extra)
+    return g
+
+
+class _Manager:
+    def __init__(self, games=()):
+        self.live_games = list(games)
+
+
+class _Helper:
+    """Stands in for ScrollHelper, including the reset that makes this hard."""
+
+    def __init__(self):
+        self.scroll_position = 0.0
+        self.total_distance_scrolled = 0.0
+        self.total_scroll_width = 5000
+        self.scroll_complete = True
+
+    def set_scrolling_image(self, width=5000):
+        self.total_scroll_width = width
+        self.scroll_position = 0.0
+        self.total_distance_scrolled = 0.0
+        self.scroll_complete = False
+
+
+class _Log:
+    def __init__(self):
+        self.lines = []
+
+    def info(self, msg, *args):
+        self.lines.append(msg % args)
+
+    def debug(self, msg, *args):
+        self.lines.append(msg % args)
+
+
+class Host(SportsLiveScrollMixin):
+    """The documented contract (registry shape)."""
+
+    LIVE_VOLATILE_FIELDS = frozenset({"clock", "status_text", "display_clock",
+                                      "league", "status"})
+
+    def __init__(self, games=(), helper=None, second_league_games=()):
+        self._league_registry = {
+            "primary": {"enabled": True, "managers": {"live": _Manager(games)}},
+            "disabled": {"enabled": False,
+                         "managers": {"live": _Manager(second_league_games)}},
+        }
+        self._live_scroll_fingerprints = {}
+        self._live_scroll_rebuilt_at = {}
+        self._live_scroll_rebuild_cost = {}
+        self.logger = _Log()
+        self.dispatched = []
+
+        class _SM:
+            def get_scroll_display(self, mode_type):
+                return type("SD", (), {"scroll_helper": helper})()
+
+        self._scroll_manager = _SM() if helper else None
+
+    def _dispatch_switch_refresh(self, manager):
+        self.dispatched.append(manager)
+
+    def _games(self):
+        return self._league_registry["primary"]["managers"]["live"].live_games
+
+    def _set(self, games):
+        self._league_registry["primary"]["managers"]["live"].live_games = list(games)
+
+
+def fresh(games=(), **kw):
+    host = Host(games, **kw)
+    host._note_live_scroll_built(KEY, "live", host._live_scroll_fingerprint())
+    host._live_scroll_rebuilt_at[KEY] = 0.0        # past the rate-limit floor
+    return host
+
+
+class TestManagers:
+    def test_the_enabled_league_only(self):
+        host = Host([game()], second_league_games=[game(gid="9")])
+        assert [m.live_games for m in host._live_scroll_managers()] == [host._games()]
+
+    def test_one_league_by_name(self):
+        host = Host([game()])
+        assert len(host._live_scroll_managers("primary")) == 1
+        assert host._live_scroll_managers("nope") == []
+
+    def test_the_single_league_shape(self):
+        host = Host([game()])
+        host._league_registry = None
+        only = _Manager([game()])
+        host._get_manager = lambda mode: only if mode == "live" else None
+        assert host._live_scroll_managers() == [only]
+
+    def test_a_failing_accessor_is_no_managers(self):
+        host = Host()
+        host._league_registry = {}
+
+        def broken(mode):
+            raise KeyError(mode)
+
+        host._get_manager = broken
+        assert host._live_scroll_managers() == []
+
+    def test_neither_shape_is_inert(self):
+        host = Host()
+        host._league_registry = None
+        assert host._live_scroll_managers() == []
+
+
+class TestWhatCountsAsAChange:
+    def test_nothing_changed(self):
+        assert not fresh([game()])._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_the_clock_ticking_is_not_a_rebuild(self):
+        host = fresh([game()])
+        host._set([game(clock="11:58", status_text="11:58 - 3rd")])
+        assert not host._live_scroll_needs_rebuild(KEY, "live")
+
+    @pytest.mark.parametrize("change", [
+        {"home": "3"}, {"period_text": "OT", "period": 4}, {"is_final": True},
+        {"is_halftime": True}, {"situation": "power play"},
+        {"some_new_field_a_card_draws": "x"}])
+    def test_anything_else_is(self, change):
+        host = fresh([game()])
+        host._set([game(**change)])
+        assert host._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_a_second_game_going_live(self):
+        host = fresh([game()])
+        host._set([game(), game(gid="2")])
+        assert host._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_the_pipelines_decoration_is_not_a_change(self):
+        host = fresh([game()])
+        host._set([dict(game(), league="nhl", status={"state": "in"})])
+        assert not host._live_scroll_needs_rebuild(KEY, "live")
+        host = fresh([dict(game(), league="nhl", status={"state": "in"})])
+        host._set([game()])
+        assert not host._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_the_hosts_volatile_fields_are_the_ones_read(self):
+        """afl, nrl and soccer also ignore period_text; that stays theirs."""
+
+        class ClockInLabel(Host):
+            LIVE_VOLATILE_FIELDS = Host.LIVE_VOLATILE_FIELDS | {"period_text"}
+
+        host = ClockInLabel([game()])
+        host._note_live_scroll_built(KEY, "live")
+        host._live_scroll_rebuilt_at[KEY] = 0.0
+        host._set([game(period_text="3rd 11:58")])
+        assert not host._live_scroll_needs_rebuild(KEY, "live")
+
+    @pytest.mark.parametrize("mode", ["recent", "upcoming"])
+    def test_other_modes_never_rebuild(self, mode):
+        host = fresh([game()])
+        host._set([game(home="5")])
+        assert not host._live_scroll_needs_rebuild(KEY, mode)
+
+    def test_a_first_build_is_not_a_change(self):
+        assert not Host([game()])._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_a_non_dict_game_still_fingerprints(self):
+        assert Host._fingerprint_games(["odd", None]) == tuple(sorted(
+            ((("<not-a-dict>", "odd"),), (("<not-a-dict>", "None"),))))
+
+
+class TestRateLimit:
+    def test_a_change_inside_the_floor_is_deferred_not_lost(self):
+        host = Host([game()])
+        host._note_live_scroll_built(KEY, "live", host._live_scroll_fingerprint())
+        host._set([game(home="3")])
+        assert not host._live_scroll_needs_rebuild(KEY, "live")
+        host._live_scroll_rebuilt_at[KEY] = 0.0
+        assert host._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_an_expensive_rebuild_raises_the_floor(self):
+        host = fresh([game()])
+        host._live_scroll_rebuild_cost[KEY] = 0.463      # 0.463 x 20 = 9.3s
+        host._live_scroll_rebuilt_at[KEY] = time.time() - 6.0
+        host._set([game(home="9")])
+        assert not host._live_scroll_needs_rebuild(KEY, "live")
+        host._live_scroll_rebuilt_at[KEY] = time.time() - 10.0
+        assert host._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_a_cheap_rebuild_stays_on_the_minimum(self):
+        host = fresh([game()])
+        host._live_scroll_rebuild_cost[KEY] = 0.029
+        host._live_scroll_rebuilt_at[KEY] = time.time() - 6.0
+        host._set([game(home="9")])
+        assert host._live_scroll_needs_rebuild(KEY, "live")
+
+    def test_the_constants(self):
+        assert SportsLiveScrollMixin.LIVE_SCROLL_REBUILD_MIN_SECONDS == 5.0
+        assert SportsLiveScrollMixin.LIVE_SCROLL_REBUILD_DUTY_DIVISOR == 20.0
+
+    def test_noting_a_non_live_build_records_nothing(self):
+        host = Host([game()])
+        host._note_live_scroll_built(KEY, "recent")
+        assert host._live_scroll_fingerprints == {} and host._live_scroll_rebuilt_at == {}
+
+
+class TestPreservingScrollPosition:
+    def test_position_and_progress_survive(self):
+        helper = _Helper()
+        host = Host([game()], helper=helper)
+        helper.scroll_position = helper.total_distance_scrolled = 812.0
+        with host._preserving_scroll_position("live", active=True):
+            helper.set_scrolling_image()
+        assert helper.scroll_position == 812.0
+        assert helper.total_distance_scrolled == 812.0
+        assert helper.scroll_complete is False
+        assert any("rebuilt the live strip in place at position 812" in line
+                   for line in host.logger.lines)
+
+    def test_clamped_to_a_shorter_strip(self):
+        helper = _Helper()
+        host = Host([game()], helper=helper)
+        helper.scroll_position = 1300.0
+        with host._preserving_scroll_position("live", active=True):
+            helper.set_scrolling_image(width=1200)
+        assert helper.scroll_position == 1199
+
+    def test_a_first_build_starts_at_zero(self):
+        helper = _Helper()
+        host = Host([game()], helper=helper)
+        helper.scroll_position = 500.0
+        with host._preserving_scroll_position("live", active=False):
+            helper.set_scrolling_image()
+        assert helper.scroll_position == 0.0
+
+    def test_no_scroll_manager_is_survivable_and_still_costed(self):
+        host = Host([game()], helper=None)
+        with host._preserving_scroll_position("live", active=True, scroll_key="nhl_live"):
+            pass
+        assert "nhl_live" in host._live_scroll_rebuild_cost
+
+    def test_the_cost_is_keyed_by_mode_without_a_scroll_key(self):
+        host = Host([game()])
+        with host._preserving_scroll_position("live", active=False):
+            pass
+        assert set(host._live_scroll_rebuild_cost) == {"live"}
+
+
+class TestRefresh:
+    def test_every_live_manager_is_dispatched(self):
+        host = Host([game()])
+        host._refresh_live_scroll_managers()
+        assert host.dispatched == [host._league_registry["primary"]["managers"]["live"]]
+
+    def test_a_dispatch_error_is_logged_not_raised(self):
+        host = Host([game()])
+
+        def broken(manager):
+            raise RuntimeError("can't start new thread")
+
+        host._dispatch_switch_refresh = broken
+        host._refresh_live_scroll_managers()
+        assert any("Live scroll refresh skipped" in line for line in host.logger.lines)
+
+    def test_with_the_host_mixin_it_runs_off_thread(self):
+        """The real pairing: _dispatch_switch_refresh from sports_plugin_host."""
+
+        class Plugin(SportsPluginHostMixin, SportsLiveScrollMixin):
+            LIVE_VOLATILE_FIELDS = Host.LIVE_VOLATILE_FIELDS
+
+            def __init__(self):
+                self.manager = _Manager([game()])
+                self._league_registry = {"a": {"enabled": True,
+                                               "managers": {"live": self.manager}}}
+                self.logger = _Log()
+                self.updated = threading.Event()
+
+            def _ensure_manager_updated(self, manager):
+                self.updated.set()
+
+        plugin = Plugin()
+        plugin._refresh_live_scroll_managers()
+        assert plugin.updated.wait(5)
+
+
+# ---------------------------------------------------------------------------
+# Host contract
+# ---------------------------------------------------------------------------
+
+def _self_reads():
+    tree = ast.parse(Path(sports_live_scroll.__file__).read_text(encoding="utf-8"))
+    cls = next(n for n in tree.body
+               if isinstance(n, ast.ClassDef) and n.name == "SportsLiveScrollMixin")
+    names = set()
+    for node in ast.walk(cls):
+        if (isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load)
+                and isinstance(node.value, ast.Name) and node.value.id in ("self", "cls")):
+            names.add(node.attr)
+        if (isinstance(node, ast.Call) and isinstance(node.func, ast.Name)
+                and node.func.id == "getattr" and len(node.args) >= 2
+                and isinstance(node.args[0], ast.Name) and node.args[0].id == "self"
+                and isinstance(node.args[1], ast.Constant)):
+            names.add(node.args[1].value)
+    return names
+
+
+class TestHostContract:
+    def test_every_host_read_is_documented(self):
+        needed = _self_reads() - set(dir(SportsLiveScrollMixin))
+        undocumented = sorted(n for n in needed if f"``{n}" not in sports_live_scroll.__doc__)
+        assert undocumented == [], f"read but not in the host contract: {undocumented}"
+
+    def test_the_mixin_creates_no_attributes_of_its_own(self):
+        for name in ("logger", "LIVE_VOLATILE_FIELDS", "_live_scroll_fingerprints",
+                     "_live_scroll_rebuilt_at", "_live_scroll_rebuild_cost",
+                     "_dispatch_switch_refresh"):
+            assert not hasattr(SportsLiveScrollMixin, name)
+        assert "__init__" not in vars(SportsLiveScrollMixin)
+
+    def test_no_name_in_common_with_the_host_mixin(self):
+        ours = {n for n in vars(SportsLiveScrollMixin) if not n.startswith("__")}
+        theirs = {n for n in vars(SportsPluginHostMixin) if not n.startswith("__")}
+        assert ours & theirs == set()

test/test_sports_plugin_host.py

@@ -0,0 +1,280 @@
+"""src.common.sports_plugin_host: behaviour and host contract.
+
+Ported from the scoreboards' own tests of the same methods
+(test_vegas_priority_weight.py in each, test_switch_refresh_off_render_thread.py
+in baseball, basketball, football, hockey, lacrosse and ufc), against a stub
+host carrying only the documented contract. Each plugin's data shape is
+covered: managers as attributes and in a dict (nrl, afl), favourite fighters
+(ufc), team ids (nrl), cricket's nested sides, and the celebration snapshot.
+"""
+
+import ast
+import threading
+import time
+from pathlib import Path
+
+import pytest
+
+from src.common import sports_plugin_host
+from src.common.sports_plugin_host import SportsPluginHostMixin
+
+
+class Host(SportsPluginHostMixin):
+    """The documented contract, and nothing else the mixin could lean on."""
+
+    def __init__(self, live_priority=True, live_content=True, vegas=None,
+                 enabled=True, dynamic=True):
+        self.has_live_priority = lambda: live_priority
+        self.has_live_content = lambda: live_content
+        self.global_config = {"display": {"vegas_scroll": vegas or {}}}
+        self.is_enabled = enabled
+        self.supports_dynamic_duration = lambda: dynamic
+        self.refreshed = []
+        self.release = threading.Event()
+        self.release.set()
+
+    def _ensure_manager_updated(self, manager):
+        self.release.wait(5)
+        self.refreshed.append(manager)
+
+
+class LiveManager:
+    def __init__(self, games=(), favorites=(), attr="live_games",
+                 fav_attr="favorite_teams", celebrating=None):
+        setattr(self, attr, list(games))
+        setattr(self, fav_attr, list(favorites))
+        if celebrating is not None:
+            self.active_celebration = {"game": celebrating, "started_at": 0}
+
+
+def _game(home="DAL", away="PHI", **extra):
+    return {"home_abbr": home, "away_abbr": away, **extra}
+
+
+# ---------------------------------------------------------------------------
+# get_vegas_priority_weight
+# ---------------------------------------------------------------------------
+
+class TestVegasPriorityWeight:
+    def test_nothing_live_has_no_opinion(self):
+        assert Host(live_content=False).get_vegas_priority_weight() is None
+        assert Host(live_priority=False).get_vegas_priority_weight() is None
+
+    def test_a_live_game_without_a_favourite_gets_the_live_weight(self):
+        host = Host(vegas={"live_weight": 3, "favorite_live_weight": 5})
+        host.nfl_live = LiveManager([_game()], ["NYG"])
+        assert host.get_vegas_priority_weight() == 3
+
+    def test_a_favourite_playing_gets_the_favourite_weight(self):
+        host = Host(vegas={"live_weight": 3, "favorite_live_weight": 7})
+        host.nfl_live = LiveManager([_game()], ["dal"])
+        assert host.get_vegas_priority_weight() == 7
+
+    def test_defaults_when_the_config_says_nothing(self):
+        host = Host()
+        host.nfl_live = LiveManager([_game()], ["NYG"])
+        assert host.get_vegas_priority_weight() == 3
+        host.nfl_live.favorite_teams = ["DAL"]
+        assert host.get_vegas_priority_weight() == 5
+
+    def test_matching_ignores_case_and_space(self):
+        host = Host()
+        host.nfl_live = LiveManager([_game()], ["  dAl "])
+        assert host.get_vegas_priority_weight() == 5
+
+    def test_managers_inside_a_dict_are_found(self):
+        """nrl and afl keep their managers in ``self._managers``."""
+        host = Host()
+        host._managers = {"live": LiveManager([_game()], ["PHI"])}
+        assert host.get_vegas_priority_weight() == 5
+
+    def test_a_later_manager_is_still_found(self):
+        host = Host()
+        host.a = LiveManager([], ["DAL"])
+        host.b = LiveManager([_game()], ["DAL"])
+        assert host.get_vegas_priority_weight() == 5
+
+    def test_junk_in_the_game_list_is_skipped(self):
+        host = Host()
+        host.nfl_live = LiveManager(["not-a-dict", None], ["DAL"])
+        assert host.get_vegas_priority_weight() == 3
+
+    def test_an_exception_is_none_not_a_raise(self):
+        host = Host()
+        host.has_live_content = lambda: (_ for _ in ()).throw(RuntimeError("boom"))
+        assert host.get_vegas_priority_weight() is None
+
+
+class TestFavoriteTeamIsLive:
+    def test_ufc_fighters(self):
+        host = Host()
+        host.ufc_live = LiveManager(
+            [{"fighter1_name": "Jon Jones", "fighter2_name": "Stipe Miocic"}],
+            ["jon jones"], fav_attr="favorite_fighters")
+        assert host._favorite_team_is_live() is True
+
+    def test_nrl_team_ids(self):
+        host = Host()
+        host._managers = {"live": LiveManager([_game(home_id="17", away_id="9")], ["9"])}
+        assert host._favorite_team_is_live() is True
+
+    def test_cricket_nested_sides_match_by_substring(self):
+        host = Host()
+        host.cricket = LiveManager(
+            [{"teams": [{"name": "India Women", "abbr": "INDW"}, "junk"]}],
+            ["india"], attr="live_matches")
+        assert host._favorite_team_is_live() is True
+
+    def test_a_celebrating_game_counts_after_it_leaves_live_games(self):
+        host = Host()
+        host.nfl_live = LiveManager([], ["DAL"], celebrating=_game())
+        assert host._favorite_team_is_live() is True
+
+    def test_no_favourites_or_only_blank_ones(self):
+        host = Host()
+        host.nfl_live = LiveManager([_game()], ["", None])
+        assert host._favorite_team_is_live() is False
+        host.nfl_live.favorite_teams = []
+        assert host._favorite_team_is_live() is False
+
+    def test_scan_targets_walk_attributes_and_dict_values(self):
+        host = Host()
+        inner = object()
+        host.plain = 1
+        host.bag = {"x": inner}
+        targets = list(host._favorite_scan_targets())
+        assert 1 in targets and host.bag in targets and inner in targets
+
+
+# ---------------------------------------------------------------------------
+# _dispatch_switch_refresh
+# ---------------------------------------------------------------------------
+
+class TestDispatchSwitchRefresh:
+    def test_runs_off_the_calling_thread(self):
+        host = Host()
+        host.release.clear()
+        manager = object()
+        started = time.monotonic()
+        host._dispatch_switch_refresh(manager)
+        assert time.monotonic() - started < 1.0       # did not wait on the update
+        assert host.refreshed == []
+        host.release.set()
+        host._switch_refresh_threads[id(manager)].join(5)
+        assert host.refreshed == [manager]
+
+    def test_one_refresh_per_manager_at_a_time(self):
+        host = Host()
+        host.release.clear()
+        manager = object()
+        host._dispatch_switch_refresh(manager)
+        first = host._switch_refresh_threads[id(manager)]
+        host._switch_refresh_at[id(manager)] = 0.0    # past the gap, still running
+        host._dispatch_switch_refresh(manager)
+        assert host._switch_refresh_threads[id(manager)] is first
+        host.release.set()
+        first.join(5)
+        assert host.refreshed == [manager]
+
+    def test_dispatches_are_rate_limited(self):
+        host = Host()
+        manager = object()
+        host._dispatch_switch_refresh(manager)
+        host._switch_refresh_threads[id(manager)].join(5)
+        host._dispatch_switch_refresh(manager)        # inside the gap
+        assert host.refreshed == [manager]
+        host._switch_refresh_at[id(manager)] -= host._SWITCH_REFRESH_MIN_GAP_SECONDS
+        host._dispatch_switch_refresh(manager)
+        host._switch_refresh_threads[id(manager)].join(5)
+        assert host.refreshed == [manager, manager]
+
+    def test_threads_are_daemons_named_for_the_manager(self):
+        host = Host()
+
+        class NFLLiveManager:
+            pass
+
+        manager = NFLLiveManager()
+        host._dispatch_switch_refresh(manager)
+        thread = host._switch_refresh_threads[id(manager)]
+        thread.join(5)
+        assert thread.daemon and thread.name == "SwitchRefresh-NFLLiveManager"
+
+    def test_the_gap_is_a_class_setting(self):
+        assert SportsPluginHostMixin._SWITCH_REFRESH_MIN_GAP_SECONDS == 5.0
+
+
+# ---------------------------------------------------------------------------
+# The small ones
+# ---------------------------------------------------------------------------
+
+class TestSmallHelpers:
+    def test_content_type_is_multi(self):
+        assert Host().get_vegas_content_type() == "multi"
+
+    @pytest.mark.parametrize("enabled,dynamic,expected", [
+        (True, True, True), (True, False, False), (False, True, False)])
+    def test_dynamic_feature_enabled(self, enabled, dynamic, expected):
+        assert Host(enabled=enabled, dynamic=dynamic)._dynamic_feature_enabled() is expected
+
+    def test_total_games_takes_the_first_list(self):
+        manager = type("M", (), {"live_games": None, "games_list": [1, 2],
+                                 "recent_games": [1, 2, 3]})()
+        assert Host._get_total_games_for_manager(manager) == 2
+        assert Host._get_total_games_for_manager(None) == 0
+        assert Host._get_total_games_for_manager(object()) == 0
+
+    def test_manager_key(self):
+        class NHLRecentManager:
+            pass
+
+        assert Host._build_manager_key("nhl_recent", NHLRecentManager()) == "nhl_recent:NHLRecentManager"
+        assert Host._build_manager_key("nhl_recent", None) == "nhl_recent:None"
+
+
+def test_it_overrides_base_plugin_when_listed_first():
+    from src.plugin_system.base_plugin import BasePlugin
+
+    class Plugin(SportsPluginHostMixin, BasePlugin):
+        def update(self):
+            pass
+
+        def display(self, force_clear=False):
+            pass
+
+    assert Plugin.get_vegas_content_type is SportsPluginHostMixin.get_vegas_content_type
+    assert Plugin.get_vegas_priority_weight is SportsPluginHostMixin.get_vegas_priority_weight
+
+
+# ---------------------------------------------------------------------------
+# Host contract
+# ---------------------------------------------------------------------------
+
+def _self_reads(module, class_name):
+    """Every ``self.X`` / ``cls.X`` / ``getattr(self, "X")`` a mixin reads."""
+    tree = ast.parse(Path(module.__file__).read_text(encoding="utf-8"))
+    cls = next(n for n in tree.body if isinstance(n, ast.ClassDef) and n.name == class_name)
+    names = set()
+    for node in ast.walk(cls):
+        if (isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load)
+                and isinstance(node.value, ast.Name) and node.value.id in ("self", "cls")):
+            names.add(node.attr)
+        if (isinstance(node, ast.Call) and isinstance(node.func, ast.Name)
+                and node.func.id == "getattr" and len(node.args) >= 2
+                and isinstance(node.args[0], ast.Name) and node.args[0].id == "self"
+                and isinstance(node.args[1], ast.Constant)):
+            names.add(node.args[1].value)
+    return names
+
+
+class TestHostContract:
+    def test_every_host_read_is_documented(self):
+        needed = _self_reads(sports_plugin_host, "SportsPluginHostMixin") - set(dir(SportsPluginHostMixin))
+        undocumented = sorted(n for n in needed if f"``{n}" not in sports_plugin_host.__doc__)
+        assert undocumented == [], f"read but not in the host contract: {undocumented}"
+
+    def test_the_mixin_creates_no_attributes_of_its_own(self):
+        for name in ("logger", "is_enabled", "global_config", "_ensure_manager_updated",
+                     "has_live_priority", "has_live_content", "supports_dynamic_duration"):
+            assert not hasattr(SportsPluginHostMixin, name)
+        assert "__init__" not in vars(SportsPluginHostMixin)

test/test_sports_stage4_parity.py

@@ -0,0 +1,176 @@
+"""The stage 4 sports modules still match every plugin copy that remains.
+
+``sports_plugin_host``, ``sports_live_scroll`` and ``sports_display_rules``
+were copied from the scoreboard plugins, which delete their copies once they
+floor on the release that ships these. Until each has, a copy that changes on
+its own is a fix one side has and the other lacks.
+
+Point LEDMATRIX_PLUGINS at a ledmatrix-plugins checkout and every method here
+is compared with every plugin copy using ``scripts/sports_drift_report.py``'s
+own normalisation -- the AST with docstrings, decorators and annotations
+dropped, which is how the report decided these families are identical -- and,
+because that normalisation drops them, the decorators are compared as well
+(``@staticmethod`` vs ``@classmethod`` vs ``@contextmanager`` is behaviour).
+Class constants are compared by value. A copy that is gone counts as adopted
+when the plugin's file names the module. Without the variable this skips:
+core CI has no plugins checkout.
+
+``sports_font_path`` is compared by behaviour instead (its body is the
+plugins' probe with the dead branches removed); see test_sports_font_path.py.
+"""
+
+import ast
+import importlib.util
+import os
+from pathlib import Path
+
+import pytest
+
+from src.common import sports_display_rules, sports_live_scroll, sports_plugin_host
+
+REPO = Path(__file__).resolve().parents[1]
+
+ALL = ("afl", "baseball", "basketball", "football", "hockey", "lacrosse",
+       "nrl", "soccer", "ufc")
+NO_UFC = tuple(s for s in ALL if s != "ufc")
+
+
+def _is_plugin_class(name: str) -> bool:
+    return name.endswith("ScoreboardPlugin")
+
+
+#: (module, mixin, plugin file, which plugin classes may hold a copy,
+#:  {promoted name: the plugins that carry it}).
+#: A name's carriers are the plugins whose copy was compared when it moved;
+#: the others never had one, and must not grow one either.
+PROMOTED = [
+    (sports_plugin_host, "SportsPluginHostMixin", "manager.py", _is_plugin_class,
+     {name: ALL for name in (
+         "_SWITCH_REFRESH_MIN_GAP_SECONDS", "_dispatch_switch_refresh",
+         "get_vegas_priority_weight", "_favorite_team_is_live",
+         "_favorite_scan_targets", "_favorite_scan_games", "_game_involves",
+         "get_vegas_content_type", "_dynamic_feature_enabled",
+         "_get_total_games_for_manager", "_build_manager_key")}),
+    (sports_live_scroll, "SportsLiveScrollMixin", "manager.py", _is_plugin_class,
+     {name: NO_UFC for name in (
+         "LIVE_SCROLL_REBUILD_MIN_SECONDS", "LIVE_SCROLL_REBUILD_DUTY_DIVISOR",
+         "_live_scroll_managers", "_refresh_live_scroll_managers",
+         "_live_scroll_fields", "_fingerprint_games", "_live_scroll_fingerprint",
+         "_live_scroll_needs_rebuild", "_note_live_scroll_built",
+         "_preserving_scroll_position")}),
+    (sports_display_rules, "SportsCardOptionsMixin", "sports.py",
+     lambda name: name == "SportsCore",
+     {"_card_option": NO_UFC, "_recent_date_text": NO_UFC}),
+    (sports_display_rules, "SportsGameRulesMixin", "sports.py",
+     lambda name: name in ("SportsCore", "SportsLive"),
+     {"_filtered_or_all": tuple(s for s in ALL if s != "football"),
+      "_effective_live_duration": NO_UFC}),
+]
+
+
+def _drift_report():
+    """scripts/sports_drift_report.py, loaded by path (scripts/ is no package)."""
+    spec = importlib.util.spec_from_file_location(
+        "sports_drift_report", REPO / "scripts" / "sports_drift_report.py")
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+DRIFT = _drift_report()
+
+
+def _plugins_root():
+    root = DRIFT.resolve_plugins_dir(os.environ.get("LEDMATRIX_PLUGINS"))
+    if root is None:
+        pytest.skip("set LEDMATRIX_PLUGINS to a ledmatrix-plugins checkout to "
+                    "compare these modules against the plugin copies")
+    return root
+
+
+def _members(tree, wanted):
+    """{name: node} for the functions and constants of the classes ``wanted`` accepts."""
+    found = {}
+    for node in tree.body:
+        if not (isinstance(node, ast.ClassDef) and wanted(node.name)):
+            continue
+        for item in node.body:
+            if isinstance(item, (ast.FunctionDef, ast.AsyncFunctionDef)):
+                found.setdefault(item.name, []).append(item)
+            elif isinstance(item, (ast.Assign, ast.AnnAssign)) and item.value is not None:
+                target = item.targets[0] if isinstance(item, ast.Assign) else item.target
+                if isinstance(target, ast.Name):
+                    found.setdefault(target.id, []).append(item)
+    return found
+
+
+def _fingerprint(node):
+    """What must agree: the drift report's body digest plus the decorators,
+    or a constant's value."""
+    if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
+        return ("def", DRIFT._digest(node, DRIFT._Canonical()),
+                tuple(ast.unparse(d) for d in node.decorator_list))
+    return ("value", ast.dump(node.value))
+
+
+def _core_members(module, mixin):
+    tree = ast.parse(Path(module.__file__).read_text(encoding="utf-8"))
+    return {name: nodes[0] for name, nodes in _members(tree, lambda n: n == mixin).items()}
+
+
+CASES = [(module.__name__.rsplit(".", 1)[1], mixin, name)
+         for module, mixin, _file, _cls, carriers in PROMOTED
+         for name in sorted(carriers)]
+
+
+def test_every_promoted_name_has_a_parity_case():
+    """A method added to a mixin without a row above would go unchecked."""
+    for module, mixin, _file, _cls, carriers in PROMOTED:
+        assert sorted(_core_members(module, mixin)) == sorted(carriers), mixin
+
+
+@pytest.mark.parametrize("module_name,mixin,name", CASES, ids=lambda v: str(v))
+def test_every_remaining_plugin_copy_matches(module_name, mixin, name):
+    root = _plugins_root()
+    module, _mixin, filename, wanted, by_name = next(
+        row for row in PROMOTED if row[1] == mixin)
+    carriers = by_name[name]
+    ours = _fingerprint(_core_members(module, mixin)[name])
+    drifted, missing, extra = [], [], []
+    for sport in ALL:
+        path = root / f"{sport}-scoreboard" / filename
+        source = path.read_text(encoding="utf-8")
+        copies = _members(ast.parse(source), wanted).get(name, [])
+        if sport not in carriers:
+            if copies:
+                extra.append(sport)
+            continue
+        if not copies:
+            # Gone is fine once the plugin uses the module; otherwise the
+            # finder is not seeing its copy.
+            if module.__name__ not in source:
+                missing.append(sport)
+            continue
+        drifted += [sport for c in copies if _fingerprint(c) != ours]
+    assert missing == [], f"{name} not found in: {missing}"
+    assert extra == [], (
+        f"{name} appeared in {extra}, which had no copy when it moved; "
+        f"decide whether {module_name} should cover it")
+    assert drifted == [], (
+        f"{name} in {module_name} differs from the copy in: {drifted}. "
+        f"Port the change to both, or stop treating it as shared.")
+
+
+def test_the_drift_report_still_calls_them_identical():
+    """The report's own verdict, per family, while any copy is left."""
+    root = _plugins_root()
+    families = DRIFT.build(root, ("sports.py", "manager.py"))
+    rows = {(r["file"], r["family"]): r
+            for r in (DRIFT.summarise(k, v) for k, v in families.items())}
+    not_identical = []
+    for _module, _mixin, filename, _cls, carriers in PROMOTED:
+        for name in carriers:
+            row = rows.get((filename, name))
+            if row is not None and row["worst_class_variants"] != 1:
+                not_identical.append(f"{filename}::{name}")
+    assert not_identical == []

test/test_vegas_live_property.py

@@ -91,6 +91,16 @@ class _DM:
 def _pipeline(groups):
     p = RenderPipeline(VegasModeConfig(continuous_scroll=True, lead_in_width=0,
                                        separator_width=12), _DM(), _Stream(groups))
+    start_prefetch = p.start_prefetch
+
+    def prefetch_now():
+        # Done before the next extension, so both twins append the same
+        # groups the same way (prepared ahead) whatever the thread timing.
+        start_prefetch()
+        if p._prefetch_thread is not None:
+            p._prefetch_thread.join(5)
+
+    p.start_prefetch = prefetch_now
     assert p.compose_scroll_content()
     return p
 

test/test_vegas_live_worker.py

@@ -93,7 +93,8 @@ def _pipeline(gate=True, **cfg):
         display_width=W, display_height=H, frame_interval=0.01,
         config=VegasModeConfig(**cfg),
         display_manager=SimpleNamespace(render_gate=_Gate() if gate else None),
-        stream_manager=_Stream(adapter), _prefetch_thread=None)
+        stream_manager=_Stream(adapter), _prefetch_thread=None, prepared=[])
+    p.prepare_group_member = p.prepared.append
     return p, adapter
 
 
@@ -364,6 +365,9 @@ def test_a_group_is_fetched_a_member_at_a_time_and_published():
         assert p._prepared_group is None
         worker._run(worker._pick(NOW))
     assert p._prepared_group == [("a", ["img-a"]), ("b", ["img-b"]), ("c", ["img-c"])]
+    # Each member was laid out for the strip here, as it arrived, not by the
+    # render thread at the extension.
+    assert p.prepared == p._prepared_group
     assert p.stream_manager.plans == [None]
     assert all(offscreen for _pid, offscreen in p.stream_manager.fetched)
     assert worker._pick(NOW) is None      # the slot is full