feat(scroll): report a panel that cannot reach its refresh cap, and suggest one it can hold

Scroll speeds are solved against display.hardware.limit_refresh_rate_hz,
which is only a ceiling. A panel that cannot reach it still moves whole
pixels per frame, but every scroll runs slow by the shortfall and the
"smooth" ladder is the cap's, not the panel's. A user rig (Pi 4, 2x128x64,
adafruit-hat-pwm, pwm_bits 9, gpio_slowdown 5) measured 107.6-113.1 Hz under
a 120 Hz cap: 60 px/s ran at 55, and nothing said why.

- scroll_config: refresh_shortfall() (more than 3% under the planned rate),
  holdable_cap() (a multiple of 10, 5% under the measurement, since the
  measurement is the fast end of an uncapped panel's drift), and
  describe_refresh_shortfall().
- FrameTimingRecorder.plan_refresh(): once the measured period has held for
  three trusted windows, a shortfall is logged once as a warning naming the
  cap to use. DisplayManager calls it only for a real panel, not the
  emulator or the fallback canvas. The stats file records
  planned_refresh_hz (additive).
- GET /api/v3/config/refresh-rate, plus a hint under the Display tab's
  Limit Refresh Rate field with a button that fills in the suggested cap.
- _panel_refresh_hz (behind the Vegas slider's advice) ignores a measurement
  written under a different cap, so a changed cap stops being advised from
  the old rate before the display restarts.

Verified on ledpi with a temporary 200 Hz cap: the warning logged about a
minute after the restart ("about 132 Hz ... Set Limit Refresh Rate to
120 Hz"), the endpoint returned the same shortfall, and the Display tab
showed the hint; its button filled in 120. ledpi was restored afterwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -19,6 +19,63 @@ accepts both, but the store flags the old spelling as deprecated
 
 ## Unreleased
 
+### Scroll speed: a panel slower than its refresh cap is reported
+
+- Scroll speeds are solved against `limit_refresh_rate_hz`, so a panel that
+  cannot reach its cap ran every scroll slow by the shortfall, with no sign
+  why (one Pi 4 on a 120 Hz cap refreshed at ~110 Hz: 60 px/s ran at 55).
+  Once the display has measured the real rate over three windows of
+  scrolling, a panel more than 3% short of the cap is logged once, as a
+  warning from `src.common.frame_timing` that names a cap it can hold (a
+  multiple of 10, 5% under the measurement). The Display tab shows the same
+  under Limit Refresh Rate, with a button that fills it in, from the new
+  `GET /api/v3/config/refresh-rate`. Not checked in the emulator or on the
+  fallback canvas.
+- The frame-stats file records `planned_refresh_hz` (additive), and the
+  scroll-speed advice behind the Vegas slider ignores a measurement written
+  under a different cap. Until now, after the cap changed, the slider kept
+  advising from the old rate until the display restarted.
+- New in `src.common.scroll_config`: `refresh_shortfall()`, `holdable_cap()`
+  and `describe_refresh_shortfall()`.
+
+### Fixed
+
+- The web preview and `/api/v3/display/current` no longer stay black for a
+  whole screen that draws its card once and then holds it. The snapshot is
+  written from `update_display()` at most once per write interval, so a frame
+  pushed inside that interval was skipped and left for the next
+  `update_display()` -- which such a screen never makes. Soccer's
+  recent/upcoming cards skip redundant redraws, and the first one after an
+  on-demand start lands a few milliseconds after the start's clear wrote a
+  black frame: on ledpi the preview showed 0 lit pixels for the whole 15 s
+  while the panel showed the card. `DisplayManager` now remembers a skipped
+  changed frame, and the render loop writes it (`write_owed_snapshot()`)
+  once the interval has passed. The cadence is unchanged, and nothing extra
+  runs when no frame is owed.
+
+### ESPN date-range fetches: fewer requests, fewer at once
+
+A soccer board (8 leagues, ESPN rejecting `dates=` ranges) logged ~90
+`NameResolutionError` lines and an `update() timed out` at every start on a
+Pi: each league's fortnight-either-side window was 29 day requests, fetched
+by several managers at once, ~40 in flight. Measured against live ESPN with
+soccer-scoreboard 2.39.2, alternating runs: **~450 requests per start, peak
+~45 in flight, ~75 DNS lookups -> 46 requests, peak 13, ~30 lookups**.
+
+- `fetch_espn_date_chunks()` asks for a window's partial edge month whole
+  when the window covers `ESPN_MONTH_COVER_MIN_DAYS` (7) or more of its days,
+  and trims the answer to the window's days by each event's US Eastern start
+  date -- the day ESPN's `dates=YYYYMMDD` means (417 of 417 live soccer
+  events matched). A 29-day window spanning two months is 2 requests instead
+  of 29. Short windows (a live poll's 1-2 days) stay day by day. A trimmed
+  month that comes back at the 500-event cap re-asks only the window's days.
+  An event with no readable date is kept. New: `espn_request_chunks()`.
+- Chunk requests share one process-wide cap of `ESPN_CHUNK_WORKERS` (6) in
+  flight, across every window being fetched, instead of six per window.
+- A new process starts as if a range had just been rejected, so it no longer
+  spends one doomed 400 per window at every start (eleven at once from a
+  soccer board); the range is still retried `RANGE_RETRY_SECONDS` in.
+
 ### Cheap per-frame and per-fetch savings
 
 - `BaseOddsManager.get_odds()` no longer pretty-prints every odds response
@@ -505,6 +562,54 @@ policies are unchanged.
   stored `ttl` was stretched the same way. A memory hit is now also checked against
   the record's own timestamp, and a stale one falls through to disk, which
   returns a newer write if there is one.
+- An on-demand request that names a `*_live` mode now shows that mode. On
+  ledpi, `{"plugin_id": "football-scoreboard", "mode": "ncaa_fb_live"}` with
+  15 college games on answered 200 and showed `nfl_recent`. The session's
+  mode list kept a live mode only when the plugin's `has_live_content()`
+  said so. That method answers the live-priority question, and the sports
+  plugins answer it for favourite teams only. A mode the request names
+  (not one resolved from a bare plugin id) now leads the session, with the
+  plugin's other modes after it. If it has nothing to draw, the session
+  moves on to the next of those modes, like any empty on-demand mode. The
+  name is saved with the session (`named_mode` in
+  `display_on_demand_config`), so a restart resumes on it.
+- A restart during an on-demand session whose plugin then fails to load no
+  longer leaves a session with no modes. On ledpi, `clock-simple` failed
+  config validation after a crash. The display logged `No valid display
+  modes found for on-demand plugin 'clock-simple' after restoration` and
+  kept reporting the session as active until its first pass ended it as
+  `idle`. The cached request stayed behind for the next restart. The session
+  now ends at startup with status `error` and error `restore-failed`, which
+  `/display/on-demand/status` reports, and the cached request is dropped. The
+  same applies when the plugin system itself fails to start.
+- `POST /api/v3/config/schedule` and `/config/dim-schedule` accept a
+  disabled per-day schedule with every day off. That is the shape
+  `config.template.json` ships, so posting back what GET returned on a fresh
+  install answered 400 "At least one day must be enabled". An enabled per-day
+  schedule still needs a day on. A day that is off now keeps the times it
+  was posted with (the schedule picker sends them). Before, saving dropped
+  them, so turning the day back on showed the defaults.
+- `POST /api/v3/config/main` answers `restart_required: true` only when the
+  save changed a setting the running display does not apply by itself.
+  Brightness (`brightness.set` and the config watcher), the per-mode
+  durations and plugin sections are applied live. A brightness-only save,
+  such as the MQTT bridge's slider, or a save that changed nothing, no longer
+  shows the restart banner. Hardware, rotation order, timezone and every
+  other setting still ask for the restart.
+- `GET /api/v3/health` reports `degraded` when the display service is
+  stopped. Before, only the sub-checks changed, and the overall status stayed
+  `healthy` for as long as the last preview frame was under 60 s old.
+  `checks.display_loop.status` is now `stopped` when three things agree:
+  systemd says the service is not active, the control socket does not
+  answer, and there is no live heartbeat. Where the platform has no socket
+  (Windows) or it is switched off, nothing changes.
+- `GET /api/v3/display/current-status` no longer reports the stopped
+  display's last state (`is_display_active: true`) from the cache for up to
+  120 s. When the control socket does not answer and the render loop's
+  heartbeat is absent, stale, or from a process that is gone (#726's rules),
+  the answer is unknown, with every field `null`. A display that still beats
+  without a socket, Windows and a socket switched off read the cache as
+  before. New `web_interface.display_state.display_gone()`.
 - The garbage-collection timer (`GcMonitor`, above) no longer prints
   `Exception ignored while calling GC callback ... 'NoneType' object has no
   attribute 'perf_counter'` when the display service or a test run exits.
@@ -618,6 +723,57 @@ policies are unchanged.
   the plugin leaves rotation until the cooldown ends, the same as a raising
   `update()`. The display still moves straight on to the next mode. A hung
   `display()` is still recorded once, as a hang.
+- A plugin settings save that failed validation no longer leaks into the next
+  save. `ConfigManager.load_config()` returned its cached config itself (the
+  fast path from #410), so the form save's edits went into the cache before
+  validation ran, and a refused save left them there. The next save of any
+  other setting (another plugin's, a plugin toggle, the schedule) wrote them
+  to config.json: the refused value, and a nested secret typed into the same
+  form (`mqtt.password`, `league.espn_s2`, `flightaware.api_key`) in plain
+  text, because it had never reached config_secrets.json to be stripped.
+  The form also reloaded showing the refused values. `load_config()` now
+  returns a private copy, and the saves keep one, so nothing a caller edits
+  reaches the cache unless it is saved. The copy duplicates only the dicts
+  and lists (every other JSON value is immutable): 2.1 ms for a real 60 KiB
+  config on a Pi 4, against 6.8 ms for `copy.deepcopy`.
+- `GET /api/v3/plugins/config` no longer returns secrets. It sent back the
+  plugin's section with config_secrets.json merged in, API keys and tokens
+  in plain text: the masking #276 added was dropped in #330. It also took
+  any id, so `?plugin_id=web_auth` returned the login's cookie-signing key
+  and password hash and `?plugin_id=github` the Plugin Store token. Secret
+  fields now come back blank, as the settings page renders them, and a
+  plugin with no schema has its credential-named fields blanked, as
+  `GET /config/main` does. Blank rather than the `••••••••` of
+  `GET /config/secrets`, because the save reads a blank secret as
+  "unchanged", so a client can post the response back without erasing
+  one. Core sections and malformed ids get a 400, as they already did from
+  reset and uninstall.
+- Plugin settings with a table (a list of rows, such as geochron's cities
+  or the countdowns) save again when a text cell is blank or holds only
+  digits. A row posts its cells as `cities.0.timezone`, and the schema
+  lookup stopped at the list, so each cell was parsed with no schema: a
+  blank optional text cell became null, and a name like "2027" became a
+  number. Either failed validation, and every save of the page failed for
+  as long as the row existed. A plugin with a secret in its rows could not
+  be saved from the page at all, since the secret cell is drawn blank. The
+  lookup now steps from the index into the list's item schema.
+- A plugin whose API key is required and has no default (youtube-stats)
+  can be saved from its settings page without typing the key in again. The
+  page draws a stored secret blank and posts the blank back; for a required
+  secret the save read that blank as null, failed validation, and refused
+  every save of the page. A blank secret field now means "unchanged", as it
+  already did for an optional one.
+- `POST /api/v3/plugins/config` refuses a core section or a malformed
+  plugin id with a 400, as reset and uninstall already did.
+  `{"plugin_id": "display", ...}` merged unvalidated values into the core
+  display section (and added `"enabled": true` to it), and an id that was
+  not a string answered with a 500.
+- A plugin text setting saves what was typed when that looks like a
+  boolean or JSON. The form save tried `true`/`false` and `[...]`/`{...}`
+  before it looked at the schema, so a text field holding "true", "False",
+  "[1, 2]" or "{}" was stored as a boolean, list or object, and the save
+  failed validation. Text fields, nullable ones included, are now taken as
+  typed; other types convert as before.
 - A WiFi notice (such as "Connected to HomeNet" or "AP mode on") now shows
   within about a second of being posted. It was only checked between
   screens, so a 5 s notice posted during a 20 s screen expired before that
@@ -626,6 +782,42 @@ policies are unchanged.
   notice is what shows next, and Vegas resumes after it; before, a rotation
   screen showed instead and the notice expired behind it. An active
   on-demand session still holds the panel until it ends.
+- The Config Editor tab no longer shows API keys and tokens in plain
+  text. Its `config_secrets.json` editor (`/partials/raw-json`) was filled
+  with the file as it is on disk, so while the web login is off (the
+  default) anyone who could reach the port could read every credential,
+  although `GET /api/v3/config/secrets` masks them. The editor now shows the
+  same masked values. Saving it unchanged changes nothing, because the save
+  drops the masks and merges onto the stored file; to change a secret,
+  replace its mask. A list of secrets still needs every entry's real value
+  to be changed. The `config.json` editor is unchanged: its save writes the
+  file as given, so a mask there would be stored.
+- A disabled plugin keeps its place in the rotation order and its Vegas
+  exclusion when the Display or Rotation & Durations tab is saved. The order
+  lists show enabled plugins only and rewrite their hidden inputs from those
+  rows as soon as they are drawn, so any save of either tab stored the lists
+  without the disabled plugin. Once re-enabled, it came back at the end of
+  the rotation and scrolling in Vegas again. A disabled plugin's saved id
+  now stays in its saved place (`widgets/plugin-order-list.js`); the id of
+  a plugin that is no longer installed is still dropped.
+- Restoring a backup with "Reinstall missing plugins" installs only the
+  plugins that are missing. Every plugin the backup listed was sent to the
+  store's install, which replaces an installed copy with a fresh download,
+  so a restore onto the same device re-downloaded all of them in one
+  request. A plugin installed from its own URL is not in the registry, so
+  its "reinstall" failed and the restore answered "Restore failed" while
+  the plugin sat there installed. An installed plugin, found by the store's
+  own lookup (registry aliases included), is now listed under Skipped as
+  `plugin:<id> (installed)`.
+- `POST /api/v3/config/main` answers a JSON body that does not parse with
+  400 `Invalid JSON in request body`, as `/config/raw/main` does, and an
+  empty JSON body with 400 `No data provided`. Both were a 500
+  `CONFIG_SAVE_FAILED` suggesting file permissions and disk space, with a
+  traceback logged at ERROR: `get_json()` raised inside the handler's
+  catch-all.
+- Fonts restored from a backup show up in the Fonts tab and the font
+  pickers straight away. The font catalog is cached for five minutes, and
+  upload and delete cleared it but a restore did not.
 - A game that goes live now takes over the panel within about a second.
   Live priority was only checked between screens, so a game that went live
   during a 30 s screen waited for that screen to end. The frame loops and the
@@ -635,6 +827,45 @@ policies are unchanged.
   screen showed first and the game came after it. Each check also asks each
   plugin `has_live_content()` once, where a plugin registered under several
   modes used to be asked once per mode.
+- A plugin action whose params hold `true`, `false` or `null` runs again.
+  `/api/v3/plugins/action` wrote the params into the source of the wrapper
+  that runs the plugin's script, and those JSON words are not Python, so the
+  wrapper stopped with a NameError and the action answered "Action failed".
+  The plugin file manager's category toggle sends `"enabled": true`, so
+  turning a category on or off in of-the-day always failed. The params now
+  reach the wrapper on its stdin; the script still receives them as JSON on
+  its own stdin, as before.
+- An on-demand request that `/api/v3/display/on-demand/start` refuses no
+  longer runs later. With the display stopped the request goes to the
+  display's mailbox, and the display reads that mailbox for an hour without
+  looking at a request's age. So with "Start display service" unticked, the
+  answer was "Display service is not running", yet the next time the
+  display was started it ran that plugin, pinned if the request said so.
+  The same happened after "Failed to start display service". On either
+  refusal the route now takes its request back out of the mailbox, unless a
+  newer one has replaced it. A request the display acknowledges over the
+  control socket is now a success whatever systemd reports: a display run
+  by hand or in the emulator was told "not running" for a request it had
+  already taken, and with "Start display service" ticked the route tried to
+  start the service beside it.
+- `/api/v3/plugins/operation/<id>` reports a queued operation as `pending`
+  instead of answering 500. The queue keeps an operation's callback among
+  its parameters until it runs, and the status route tried to send that
+  function as JSON. An install queued behind another plugin's install
+  failed every status poll until the first one finished. Parameters whose
+  name starts with `_` are internal and are no longer in the answer.
+- A second click on Install while that plugin is still installing, or an
+  Uninstall during its install, now answers 409 "already has an install,
+  update or uninstall in progress" instead of 500 "An error occurred". The
+  first operation carried on either way. The uninstall route also stopped
+  recording a failed uninstall in the operation history for an uninstall
+  that never started.
+- `/api/v3/plugins/<plugin_id>/static/<path>` serves images and other
+  binary files. It opened every file as UTF-8 text, so a plugin's icon or
+  preview image answered 500 `UnicodeDecodeError`. Files are now sent as
+  they are on disk, an image with its own content type; HTML, JavaScript,
+  CSS, JSON and other text keep the types they had. The path checks are
+  unchanged.
 - The display schedule turns the panel off at exactly the end time. A window
   now runs from its start time up to, but not including, its end time: with
   07:00-23:00 the panel is on at 07:00 and off at 23:00. Before, the end
@@ -642,6 +873,31 @@ policies are unchanged.
   the panel went off at 23:00 or at 23:01 depending on when in the minute
   that check ran. Windows that cross midnight and per-day schedules follow
   the same rule, and so does the dim schedule.
+- The MQTT bridge settings on the Tools tab can save a broker password with
+  TLS off. The server refuses that unless `allow_insecure_mqtt` is set, and
+  the form had no way to set it, so a password-protected broker on a home
+  network without TLS could not be saved from the web UI, and once such a
+  password was stored every later save failed too. While "Use TLS" is
+  unchecked the form now shows "Allow without TLS (trusted network)",
+  prefilled from the saved settings. It is off until ticked, so the server
+  still refuses a cleartext password by default.
+- The Overview's plugin-config warning check stops polling. It asked
+  `/api/v3/plugins/reconciliation-status` every 2 s until startup
+  reconciliation reported done, and the route reports not done whenever its
+  status file is missing: reconciliation raised before writing it, or /tmp
+  was cleaned under a long-running web service. The page then sent that
+  request every 2 s for as long as it stayed open, whichever tab was showing.
+  It now gives up after a minute and only polls while the Overview is on
+  screen.
+- Moving the Brightness slider on the Display tab no longer throws an error
+  in the browser console on every step. Its handler also updated a "LED
+  brightness" line that was removed from the page in #387; the lookup is
+  gone.
+- Creating an API token on the General tab no longer leaves the page asking
+  "Leave site?" on reload. The unsaved-changes guard marks a form when you
+  type in it and clears the mark only after an htmx save, and the token form
+  saves with a plain request, so it stayed marked after the token was
+  created. It is cleared once the token is saved.
 - An on-demand session that ends during scheduled-off hours, by expiring or
   being stopped, blanks the panel within about a second. It used to stay on
   until the next minute, because the once-a-minute schedule check had

docs/REST_API_REFERENCE.md

@@ -159,14 +159,16 @@ there an unchecked checkbox — which the browser omits — is saved as
 }
 ```
 
-`restart_required` is always true here: display hardware, rotation,
+`restart_required` is true when the save changed a setting that takes
-durations and general settings take effect when the display restarts, and
+effect when the display restarts: display hardware, rotation order,
-the web UI shows its restart banner on the flag. (Plugin sections saved
+timezone, general settings and the rest. The web UI shows its restart banner
-through this route reach the running plugin live, like
+on the flag. It is false when the save changed only what the running display
-`POST /plugins/config`.)
+applies by itself, or nothing: `brightness`, the per-mode durations
+(`duration__<mode>`, `display.display_durations`) and plugin sections, which
+reach the running plugin live, like `POST /plugins/config`.
 
-A saved `brightness` is the exception: it reaches the panel without a
+A saved `brightness` reaches the panel without a restart. The route also
-restart. The route also sends it to the running display over the control
+sends it to the running display over the control
 socket (`brightness.set`), which puts it on the panel at once, and the
 response adds `"brightness_transport": "socket"`. Otherwise it is
 `"config"`, with `brightness_socket_error` giving the reason, and the
@@ -246,7 +248,10 @@ Replace the schedule configuration.
 ```
 
 A day whose `<day>_enabled` key is absent counts as enabled, with default
-times `07:00`-`23:00`. At least one day must be enabled.
+times `07:00`-`23:00`. An enabled schedule needs at least one day enabled; a
+disabled one (`"enabled": false`) may have every day off, as
+`config.template.json` ships it. A day that is off keeps the times sent for
+it, when they are valid `HH:MM`.
 
 **Response**:
 ```json
@@ -343,7 +348,11 @@ control socket ([IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md)), and `cache`
 when it came from the `display_current_state` cache key (no socket: the
 display is stopped or older, or this is Windows). A display whose render
 loop has not refreshed its state for 120 seconds is reported with every
-field `null`, either way.
+field `null`, either way. So is a stopped display: when the socket does not
+answer and the render loop's heartbeat
+(`/run/ledmatrix/display-heartbeat.json`) is absent, stale or from a process
+that is gone, the cache's last entry is not used. A display still beating
+without a socket, Windows, or a socket switched off reads the cache.
 
 ### List Display Modes
 
@@ -2281,7 +2290,11 @@ display snapshot. `data.status` is `healthy` or `degraded`, with
 (with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
 frozen even if the service is active; the status turns `degraded`), or
 `not_reported` when the display writes none (not started yet, the dev server,
-Windows), which does not affect the status. Its `source` is `socket` when the
+Windows), which does not affect the status, or `stopped` (with `source:
+"service"`) when the display service is not active, the control socket does
+not answer and there is no live heartbeat; the status then turns
+`degraded`. A platform with no control socket (Windows) or a socket switched
+off never reports `stopped`. Its `source` is `socket` when the
 age came from the display's state stream over the control socket (measured
 in memory by the display) and `heartbeat_file` when it came from
 `/run/ledmatrix/display-heartbeat.json`.
@@ -2344,8 +2357,9 @@ Replace the dim schedule. `dim_brightness` is 0-100 (default 30). In
 `per-day` mode the days can be sent either as the `days` object that GET
 returns, or as the web form's flat fields (`monday_enabled`,
 `monday_start`, `monday_end`, ...). A day that is not sent counts as
-enabled with default times `20:00`-`07:00`; at least one day must be
+enabled with default times `20:00`-`07:00`. As for the schedule above, an
-enabled.
+enabled dim schedule needs at least one day enabled and a disabled one may
+have every day off.
 
 ---
 

docs/RUN_LOOP_REDESIGN.md

@@ -181,13 +181,14 @@ that the harness patches in today.
   - dynamic duration (cycle complete, plugin cap, global cap)
   - live priority taking over and handing back; live round-robin
   - on-demand start/stop/expiry; pinned on-demand; a session resumed after
-    a restart
+    a restart, and one that cannot resume (its plugin did not load); a
+    request naming a live mode the plugin's live check would drop
   - schedule off and dim, with an on-demand override during downtime
   - WiFi notice; sync follower
   - Vegas, with and without `live_in_ticker`
 - Each trace row is `[start, mode, duration, exit_reason, frames,
   force_clear]`. The exit reason is the event that decided what came next.
-- All 16 tests run in under a second. The goldens were generated from
+- All 18 tests run in under a second. The goldens were generated from
   main's `run()` before any code moved.
 - Vegas uses `FakeVegas`, which implements only the contract the controller
   depends on: `run_iteration()` returns True after its duration and False

docs/SCROLL_PERFORMANCE.md

@@ -77,6 +77,31 @@ The Vegas **Scroll Speed** slider in the web UI shows the same thing live: a
 line under it says what your speed will run as on this panel, and links to the
 nearest smooth speeds.
 
+### A panel that cannot reach its cap
+
+Speeds are solved against `limit_refresh_rate_hz`, the configured cap, but a
+cap is only a ceiling: a long chain, a high `pwm_bits` or a big
+`gpio_slowdown` can leave the panel below it. One Pi 4 driving 2×128×64 on
+`adafruit-hat-pwm` with `pwm_bits 9` and `gpio_slowdown 5` measured
+107.6–113.1 Hz under a 120 Hz cap. Frames still move whole pixels, but
+every scroll runs that much slower than configured (60 px/s ran at 55 px/s),
+and the smooth speeds are the cap's rather than the panel's.
+
+The display measures the real rate from its own frames. About a minute
+into scrolling, a panel more than 3% short of its cap is logged once:
+
+```
+WARNING - src.common.frame_timing - The panel refreshes at about 113 Hz, below
+the 120 Hz that scroll speeds are planned for ... Set Limit Refresh Rate to
+100 Hz (web UI, Display tab), which this panel can hold, and restart.
+```
+
+The Display tab says the same under **Limit Refresh Rate**, with a button
+that fills in the suggested cap (`GET /api/v3/config/refresh-rate`). The
+suggestion is a multiple of 10 at least 5% under the measurement, because
+an uncapped panel drifts and the measurement is the fast end of it. A cap the
+panel holds also stops the drift.
+
 ### How a slow speed stays crisp
 
 `SwapOnVSync(canvas, framerate_fraction)` holds each frame for N panel

src/backup_manager.py

@@ -213,8 +213,9 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
     The plugins are the ``manifest.json`` files in the configured plugin
     directory (see :func:`_plugins_directory`), with the manifest's version;
     ``enabled`` is config.json's flag by the display's rule (a missing flag
-    is disabled). A restore reinstalls every listed plugin and takes enabled
+    is disabled). A restore installs each listed plugin that is missing and
-    state from the restored config.json, so ``enabled`` is informational.
+    takes enabled state from the restored config.json, so ``enabled`` is
+    informational.
 
     ``data/plugin_state.json`` is not read: it only ever repeated config's
     enabled flags and the manifests' versions, and is retired (nothing

src/common/README.md

@@ -109,8 +109,11 @@ and the plugin test harness all use it. Most plugins get BDF text through
 [`espn_dates.py`](espn_dates.py). ESPN's site API rejects `dates=` ranges
 and truncates results when `limit` is above 500. `fetch_espn_scoreboard()`
 splits a range into month and day requests ESPN accepts and merges the
-results; `espn_date_chunks()`, `fetch_espn_date_chunks()`,
+results; `espn_date_chunks()`, `espn_request_chunks()`,
-`clamp_espn_limit()` and `merge_scoreboard_payloads()` are the pieces.
+`fetch_espn_date_chunks()`, `clamp_espn_limit()` and
+`merge_scoreboard_payloads()` are the pieces. A window's partial edge months
+are asked whole and trimmed to its days (US Eastern), and chunk requests share
+one process-wide cap of `ESPN_CHUNK_WORKERS` in flight.
 Every request goes through [`fetch_service`](#fetch_service), the chunks
 counted against the plugin that asked. Scoreboard plugins also bundle a copy
 for older cores.

src/common/espn_dates.py

@@ -26,10 +26,30 @@ A month can hold more than 500 events (college baseball's March does), and
 ESPN answers that with exactly ``limit`` events and no hint that more exist. A
 month chunk that comes back full is therefore re-asked day by day.
 
+A window's *partial* edge months are asked for whole, too, once the window
+covers ``ESPN_MONTH_COVER_MIN_DAYS`` or more of their days, and the answer is
+trimmed back to the window's days. A scoreboard's default fortnight either side
+of today (29 days, two partial months) was 29 day requests per league; it is
+now 2. Trimming needs ESPN's "game day", which is the event's start in US
+Eastern time -- checked against the live API on 2026-10-03: 417 of 417 soccer
+events across five leagues and three months (one of them spanning the end of
+daylight saving) came back from exactly the day query their Eastern date
+names. A short window (a live poll's one or two days) stays day by day, so it
+never downloads a whole month to read a day of it.
+
+Chunk requests share one process-wide budget of ``ESPN_CHUNK_WORKERS`` in
+flight, however many windows are being fetched at once. Each window used to get
+its own six, so a scoreboard starting eight leagues -- each with a recent and
+an upcoming manager -- had ~40 requests in flight, every one beyond a session's
+pool a new connection and a new DNS lookup. On a Pi whose resolver could not
+keep up, that was ~90 ``NameResolutionError`` lines within a minute of every
+start.
+
 Once a range has been rejected, later ranges skip straight to chunks for
 ``RANGE_RETRY_SECONDS`` instead of spending a doomed request first -- live
 scoreboards ask every 30 seconds. After that the range is tried again, so the
-workaround retires itself if ESPN reverts.
+workaround retires itself if ESPN reverts. A process starts inside that
+period, as if a range had just been rejected.
 
 ONE CACHE KEY PER SCOREBOARD
 ----------------------------
@@ -55,7 +75,7 @@ import re
 import threading
 import time
 from concurrent.futures import ThreadPoolExecutor
-from datetime import date, datetime, timedelta
+from datetime import date, datetime, timedelta, tzinfo
 from functools import partial
 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple, cast
 
@@ -100,16 +120,57 @@ RANGE_RETRY_SECONDS = 6 * 60 * 60
 # pool_maxsize of 10 so the shared Session never has to discard connections.
 ESPN_CHUNK_WORKERS = 6
 
+#: An edge month the window covers at least this many days of is asked for
+#: whole and trimmed, instead of one request per day (see module docstring).
+#: Below it the days are cheaper than the month: a whole month is two to
+#: three times the bytes of the half of it a fortnight window holds.
+ESPN_MONTH_COVER_MIN_DAYS = 7
+
+# Every chunk request in the process holds one of these while it is in flight
+# -- the cap is per process, not per window (see module docstring).
+_chunk_slots = threading.BoundedSemaphore(ESPN_CHUNK_WORKERS)
+
+
+def _eastern_zone() -> Optional[tzinfo]:
+    """US Eastern, the zone ESPN's ``dates=YYYYMMDD`` means, or None when
+    this Python has no time zone data (no edge month is trimmed then)."""
+    zone: Optional[tzinfo] = None
+    try:
+        from zoneinfo import ZoneInfo
+        zone = ZoneInfo("America/New_York")
+    except Exception:  # noqa: BLE001 - no zoneinfo module or no tz database
+        zone = None
+    if zone is not None:
+        return zone
+    try:
+        import pytz
+        return cast(tzinfo, pytz.timezone("America/New_York"))
+    except Exception:  # noqa: BLE001
+        return None
+
+
+_EASTERN = _eastern_zone()
+
+# What _fetch_one_chunk returns for a month that came back at the cap.
+_CAPPED: Any = object()
+
 _range_lock = threading.Lock()
-_ranges_rejected_until = 0.0
+# A process starts out assuming ranges are still rejected, as they have been
+# since 2026-09-15, and tries one again RANGE_RETRY_SECONDS in. Starting
+# from "unknown" cost one doomed range request per window at every start --
+# eleven 400s at once from a soccer board, each fetching before any had
+# answered -- to learn what every start learns.
+_ranges_rejected_until = time.monotonic() + RANGE_RETRY_SECONDS
 
 __all__ = [
     "ESPN_MAX_LIMIT",
     "ESPN_CHUNK_WORKERS",
+    "ESPN_MONTH_COVER_MIN_DAYS",
     "RANGE_RETRY_SECONDS",
     "clamp_espn_limit",
     "parse_espn_date_range",
     "espn_date_chunks",
+    "espn_request_chunks",
     "merge_scoreboard_payloads",
     "fetch_espn_date_chunks",
     "fetch_espn_scoreboard",
@@ -220,6 +281,79 @@ def espn_date_chunks(start: date, end: date) -> List[str]:
     return chunks
 
 
+def espn_request_chunks(
+    start: date,
+    end: date,
+    month_cover_min_days: Optional[int] = None,
+) -> List[Tuple[str, Optional[Tuple[date, date]]]]:
+    """The requests that fetch ``[start, end]``, as ``(dates, trim)`` pairs.
+
+    :func:`espn_date_chunks`, except that a partial edge month with
+    ``month_cover_min_days`` (default ``ESPN_MONTH_COVER_MIN_DAYS``) or more
+    of its days in the window becomes one ``YYYYMM`` request whose ``trim``
+    is the first and last of those days: its events that start outside them
+    (US Eastern) are dropped. ``trim`` is None for every other request.
+    Without time zone data nothing can be trimmed, so the edge days stay day
+    requests.
+    """
+    if month_cover_min_days is None:
+        month_cover_min_days = ESPN_MONTH_COVER_MIN_DAYS
+    planned: List[Tuple[str, Optional[Tuple[date, date]]]] = []
+    run: List[str] = []
+
+    def flush() -> None:
+        if (_EASTERN is not None and month_cover_min_days > 0
+                and len(run) >= month_cover_min_days):
+            planned.append((run[0][:6], (_parse_day(run[0]), _parse_day(run[-1]))))
+        else:
+            planned.extend((day, None) for day in run)
+        run.clear()
+
+    for chunk in espn_date_chunks(start, end):
+        if run and (len(chunk) != 8 or chunk[:6] != run[0][:6]):
+            flush()
+        if len(chunk) == 8:
+            run.append(chunk)
+        else:
+            planned.append((chunk, None))
+    flush()
+    return planned
+
+
+def _parse_day(text: str) -> date:
+    return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
+
+
+def _eastern_day(stamp: Any) -> Optional[date]:
+    """The US Eastern date of an ESPN event ``date`` ("2026-10-10T11:30Z"),
+    or None when it cannot be read."""
+    if not isinstance(stamp, str) or _EASTERN is None:
+        return None
+    try:
+        moment = datetime.fromisoformat(stamp.strip().replace("Z", "+00:00"))
+    except ValueError:
+        return None
+    if moment.tzinfo is None:
+        return None
+    return moment.astimezone(_EASTERN).date()
+
+
+def _trim_to_days(payload: Any, first: date, last: date) -> Any:
+    """Drop the events of a month payload that start outside ``[first, last]``
+    (US Eastern). An event whose date cannot be read is kept: its day query
+    might well have returned it, and a game is never dropped on a guess.
+    """
+    if not isinstance(payload, dict) or not isinstance(payload.get("events"), list):
+        return payload
+    kept = []
+    for event in payload["events"]:
+        day = _eastern_day(event.get("date")) if isinstance(event, dict) else None
+        if day is None or first <= day <= last:
+            kept.append(event)
+    payload["events"] = kept
+    return payload
+
+
 def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
     """Fold chunk responses into one scoreboard payload.
 
@@ -250,37 +384,54 @@ def merge_scoreboard_payloads(payloads: List[Any]) -> Dict[str, Any]:
 def _fetch_one_chunk(
     session, url: str, params: Dict[str, Any], headers, timeout, logger, chunk: str,
     cache_max_age: Optional[float] = None,
-) -> Optional[Dict[str, Any]]:
+    trims: Optional[Dict[str, Tuple[date, date]]] = None,
+) -> Any:
     """GET a single ``dates=`` chunk, or None when it failed.
 
     One bad chunk must not sink the rest of the season, so every error is
     logged and swallowed here rather than raised to the gather below.
+
+    A month that comes back at the cap is truncated: it returns ``_CAPPED``,
+    its payload dropped here before it is ever held beside the others. A
+    month in ``trims`` loses its events outside the days given there.
+
+    The request holds one of the process-wide ``_chunk_slots`` while it runs.
     """
     try:
-        response = fetch_get(
+        with _chunk_slots:
-            session,
+            response = fetch_get(
-            url,
+                session,
-            params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
+                url,
-            headers=headers,
+                params=dict(params, dates=chunk, limit=ESPN_MAX_LIMIT),
-            timeout=timeout,
+                headers=headers,
-            **_memo_kwargs(cache_max_age),
+                timeout=timeout,
-        )
+                **_memo_kwargs(cache_max_age),
-        response.raise_for_status()
+            )
-        return cast(Optional[Dict[str, Any]], response_json(response))
+            response.raise_for_status()
+            payload = response_json(response)
     except Exception as exc:  # noqa: BLE001 - see docstring
         if logger:
             logger.warning("ESPN chunk %s failed, skipping it: %s", chunk, exc)
         return None
+    if len(chunk) == 6 and isinstance(payload, dict):
+        if len(payload.get("events") or []) >= ESPN_MAX_LIMIT:
+            return _CAPPED
+        trim = (trims or {}).get(chunk)
+        if trim is not None:
+            payload = _trim_to_days(payload, *trim)
+    return payload
 
 
 def _fetch_chunks(
     session, url: str, params: Dict[str, Any], headers, timeout, logger,
     chunks: List[str], cache_max_age: Optional[float] = None,
-) -> List[Optional[Dict[str, Any]]]:
+    trims: Optional[Dict[str, Tuple[date, date]]] = None,
+) -> List[Any]:
     """Fetch every chunk, returning payloads positionally aligned with ``chunks``.
 
     Requests go out ``ESPN_CHUNK_WORKERS`` at a time because a cold season is
-    over a hundred of them. The order they come back in is not significant --
+    over a hundred of them -- and no more than that across every window the
+    process is fetching, which ``_fetch_one_chunk``'s slot enforces. The order they come back in is not significant --
     callers keep ``chunks`` order from the returned list -- but it does mean
     the session is shared across threads, which is why this only ever issues
     GETs and never touches session state.
@@ -293,7 +444,7 @@ def _fetch_chunks(
         return []
     fetch = partial(
         _fetch_one_chunk, session, url, params, headers, timeout, logger,
-        cache_max_age=cache_max_age,
+        cache_max_age=cache_max_age, trims=trims,
     )
     if len(chunks) == 1:
         return [fetch(chunks[0])]
@@ -340,7 +491,9 @@ def fetch_espn_date_chunks(
     if span is None:
         return None
 
-    chunks = espn_date_chunks(*span)
+    planned = espn_request_chunks(*span)
+    chunks = [chunk for chunk, _ in planned]
+    trims = {chunk: trim for chunk, trim in planned if trim is not None}
     if logger:
         logger.debug(
             "Fetching ESPN date range %s as %d month/day chunks",
@@ -349,32 +502,31 @@ def fetch_espn_date_chunks(
 
     results = _fetch_chunks(
         session, url, params, headers, timeout, logger, chunks, cache_max_age,
+        trims,
     )
     attempted = len(chunks)
 
-    # A month that came back at the cap is truncated; its days replace it in
+    # A month that came back at the cap is truncated; its days (only the
-    # place, so merged events stay in chunk order however the requests raced.
+    # window's, for a trimmed edge month) replace it in place, so merged
+    # events stay in chunk order however the requests raced. Its payload was
+    # already dropped in the worker: a capped college-baseball month is ~2MB
+    # of parsed JSON, and holding four of them through ~120 day requests added
+    # ~25MB to the peak -- more than the concurrency itself. Low-memory boards
+    # (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
     slots: List[Any] = results
     capped: Dict[int, List[str]] = {}
     for index, chunk in enumerate(chunks):
-        payload = slots[index]
+        if slots[index] is not _CAPPED:
-        if payload is None or len(chunk) != 6:
             continue
-        events = payload.get("events") if isinstance(payload, dict) else None
+        if logger:
-        if len(events or []) >= ESPN_MAX_LIMIT:
+            logger.info(
-            if logger:
+                "ESPN month %s hit the %d-event cap; re-asking it day by day",
-                logger.info(
+                chunk, ESPN_MAX_LIMIT,
-                    "ESPN month %s hit the %d-event cap; re-asking it day by day",
+            )
-                    chunk, ESPN_MAX_LIMIT,
+        trim = trims.get(chunk)
-                )
+        capped[index] = (_days_of_month(chunk) if trim is None
-            capped[index] = _days_of_month(chunk)
+                         else espn_date_chunks(*trim))
-            # Drop the truncated month now rather than after its days arrive:
+        slots[index] = None
-            # a capped college-baseball month is ~2MB of parsed JSON, and
-            # holding four of them through ~120 day requests added ~25MB to
-            # the peak -- more than the concurrency itself. Low-memory boards
-            # (docs/LOW_MEMORY_BOARDS.md) have under 200MB of headroom.
-            slots[index] = None
-    payload = events = None
 
     if capped:
         days = [day for index in sorted(capped) for day in capped[index]]

src/common/frame_timing.py

@@ -135,6 +135,8 @@ import time
 import traceback
 from typing import Any, Callable, Dict, List, Optional, Tuple, TypedDict
 
+from src.common import scroll_config
+
 logger = logging.getLogger(__name__)
 
 #: Bumped when a field changes meaning, so a reader can refuse stale files.
@@ -179,6 +181,12 @@ MAX_REFRESH_DROP = 0.2
 #: trusted -- about a second of scrolling.
 MIN_FRAMES_FOR_REFRESH = 90
 
+#: Trusted windows, counting the one that adopted the period, before a panel
+#: slower than its cap is reported. The estimate can still fall (the refresh
+#: rate rise) by up to MAX_REFRESH_DROP per window early on; the warning
+#: should not name a rate one more window would have corrected.
+REFRESH_CHECK_WINDOWS = 3
+
 FLUSH_INTERVAL = 10.0
 
 #: A scroll's last frame older than this is a stall worth a stack dump.
@@ -441,6 +449,12 @@ class FrameTimingRecorder:
             1.0 / refresh_hz if refresh_hz and refresh_hz > 0 else None)
         # The first estimate, until a second window agrees with it.
         self._refresh_candidate: Optional[float] = None
+        # The rate scroll speeds are solved against; see plan_refresh().
+        self.planned_refresh_hz: Optional[float] = None
+        # Trusted windows seen since the period was adopted, until the
+        # shortfall check has run.
+        self._refresh_windows = 0
+        self._shortfall_checked = True
         self.totals: Dict[str, Any] = {
             "static_frames": 0,
             "scroll_frames": 0,
@@ -639,7 +653,12 @@ class FrameTimingRecorder:
                     self._refresh_candidate = estimate
             elif current * (1.0 - MAX_REFRESH_DROP) <= estimate < current:
                 self.refresh_period = estimate
+            if self.refresh_period is not None:
+                self._refresh_windows += 1
         period = self.refresh_period
+        if (period and not self._shortfall_checked
+                and self._refresh_windows >= REFRESH_CHECK_WINDOWS):
+            self._check_refresh_shortfall(1.0 / period)
 
         histograms = self.histograms
         for frame in batch:
@@ -688,6 +707,25 @@ class FrameTimingRecorder:
                 elif missed <= -1:
                     totals["early_frames"] += 1
 
+    def plan_refresh(self, hz: Optional[float]) -> None:
+        """Say what rate scroll speeds are solved against, before frames arrive.
+
+        ``DisplayManager.refresh_hz``: the configured cap. Once the measured
+        rate has held for :data:`REFRESH_CHECK_WINDOWS` windows, a panel that
+        falls short of it is logged once, with a cap it can hold (see
+        :func:`src.common.scroll_config.refresh_shortfall`). The display
+        manager calls this only for a real panel.
+        """
+        self.planned_refresh_hz = hz
+        self._shortfall_checked = not hz
+
+    def _check_refresh_shortfall(self, measured_hz: float) -> None:
+        """Log, once, a panel that cannot reach the rate speeds assume."""
+        self._shortfall_checked = True
+        shortfall = scroll_config.refresh_shortfall(measured_hz, self.planned_refresh_hz)
+        if shortfall:
+            logger.warning(scroll_config.describe_refresh_shortfall(shortfall))
+
     def snapshot(self) -> Dict[str, Any]:
         """The JSON document: cumulative since this process started."""
         if not self._binding_checked:
@@ -704,6 +742,9 @@ class FrameTimingRecorder:
             "bucket_ms": BUCKET_MS,
             "freeze_seconds": FREEZE_SECONDS,
             "measured_refresh_hz": round(1.0 / period, 2) if period else None,
+            # Additive: what scroll speeds were solved against, so a reader can
+            # tell a stale file (written under another cap) from this one.
+            "planned_refresh_hz": self.planned_refresh_hz,
             "binding_releases_gil": self._binding_gil,
             "info": info,
             "totals": copy.deepcopy(self.totals),

src/common/scroll_config.py

@@ -538,3 +538,66 @@ def speed_advice(
         "smooth": smooth,
         "alternatives": [as_dict(c) for c in alternatives],
     }
+
+
+#: A measured refresh this far below the rate speeds are planned for means
+#: the panel cannot reach its cap. Smaller gaps are the cap's own slack and
+#: the estimate's: one rig measured 99.95 Hz under a 100 Hz cap.
+REFRESH_SHORTFALL = 0.03
+
+#: How far under the measured rate a suggested cap sits. The measurement is
+#: the fast end of the panel's refreshes (frame_timing takes the 10th
+#: percentile of intervals), and an uncapped panel drifts: one read
+#: 107.6-113.1 Hz over 15 seconds. A cap inside that band would not hold.
+CAP_HEADROOM = 0.05
+
+
+def holdable_cap(measured_hz: Any) -> Optional[int]:
+    """A refresh cap the panel can hold: a multiple of 10, 5% under what it measured.
+
+    A multiple of 10 because its whole-pixel speeds are round numbers (a
+    100 Hz cap gives 50 and 100 px/s). None without a usable measurement, or
+    when the panel is too slow for any cap of 10 Hz or more.
+    """
+    hz = _coerce(measured_hz)
+    if hz is None:
+        return None
+    cap = int(hz * (1.0 - CAP_HEADROOM) // 10) * 10
+    return cap if cap >= 10 else None
+
+
+def refresh_shortfall(measured_hz: Any, planned_hz: Any) -> Optional[Dict[str, Any]]:
+    """When the panel refreshes measurably slower than speeds are planned for.
+
+    ``planned_hz`` is what :func:`configure` solves against -- the
+    ``limit_refresh_rate_hz`` cap, or :data:`DEFAULT_REFRESH_HZ` when it is 0.
+    A panel that cannot reach it still moves whole pixels per frame, but every
+    speed runs slow by the shortfall and the ladder of smooth speeds is the
+    cap's, not the panel's. None when there is no measurement, or the panel
+    reaches the cap (or beats it, as some do by a few Hz).
+    """
+    measured, planned = _coerce(measured_hz), _coerce(planned_hz)
+    if measured is None or planned is None:
+        return None
+    if measured >= planned * (1.0 - REFRESH_SHORTFALL):
+        return None
+    return {
+        "measured_hz": round(measured, 1),
+        "planned_hz": round(planned, 1),
+        "suggested_cap_hz": holdable_cap(measured),
+        "slow_percent": round((1.0 - measured / planned) * 100),
+    }
+
+
+def describe_refresh_shortfall(shortfall: Dict[str, Any]) -> str:
+    """One log line for :func:`refresh_shortfall`'s answer."""
+    text = (
+        f"The panel refreshes at about {shortfall['measured_hz']:.0f} Hz, below "
+        f"the {shortfall['planned_hz']:.0f} Hz that scroll speeds are planned "
+        f"for (display.hardware.limit_refresh_rate_hz), so every scroll runs "
+        f"about {shortfall['slow_percent']}% slower than configured and the "
+        f"smooth speeds are worked out for a rate this panel never reaches.")
+    if shortfall.get("suggested_cap_hz"):
+        text += (f" Set Limit Refresh Rate to {shortfall['suggested_cap_hz']} Hz "
+                 f"(web UI, Display tab), which this panel can hold, and restart.")
+    return text

src/config_manager.py

@@ -46,6 +46,35 @@ from src.common.permission_utils import (
     get_config_dir_mode
 )
 
+
+def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
+    """A deep copy of ``config`` that shares nothing with it.
+
+    load_config() hands one out per call, and the saves keep one, so the
+    cached config is never an object a caller holds. A web handler edits what
+    it loaded, validates, and may refuse the save; when the cache was that
+    same object, the refused edit stayed in it, and the next save of any
+    other setting wrote it to config.json -- a nested secret included, in
+    plain text, since it had never reached config_secrets.json to be
+    stripped.
+
+    The config is JSON data, so only its dicts and lists need copying; every
+    other value in it is immutable. On a Pi 4 with a real 60 KiB config this
+    takes 2.1 ms against copy.deepcopy's 6.8 ms, on a path ~30 handlers call
+    (a pickle round trip is no faster, 1.9 ms, and brings pickle into the
+    config path for nothing).
+    """
+    return _copy_containers(config)
+
+
+def _copy_containers(value: Any) -> Any:
+    if isinstance(value, dict):
+        return {key: _copy_containers(item) for key, item in value.items()}
+    if isinstance(value, list):
+        return [_copy_containers(item) for item in value]
+    return value
+
+
 class ConfigManager:
     """
     Reads and writes the main application configuration files.
@@ -126,9 +155,10 @@ class ConfigManager:
             validate_after_write=validate_after_write
         )
         
-        # Update in-memory config if save was successful
+        # Update in-memory config if save was successful. A copy: the caller
+        # still holds new_config_data (see _private_copy).
         if result.status == SaveResultStatus.SUCCESS:
-            self.config = new_config_data
+            self.config = _private_copy(new_config_data)
             # In-memory config now matches what was just written, so the
             # load_config fast path may return it. It still carries the
             # merged secrets that were stripped on disk; that matches a full
@@ -208,14 +238,16 @@ class ConfigManager:
 
         Fast path: when config.json, config_secrets.json and the template
         are all unchanged since the last successful load (mtime_ns + size),
-        the already-parsed self.config is returned without touching the
+        a copy of the already-parsed self.config is returned without
-        files — same aliasing semantics as the full path, which also
+        touching the files.
-        returns self.config.
+
+        Either way the caller gets its own copy (see _private_copy): editing
+        it changes nothing here until it is saved.
         """
         try:
             current_sig = self._files_signature()
             if self.config and self._loaded_sig == current_sig:
-                return self.config
+                return _private_copy(self.config)
 
             # Check if config file exists, if not create from template
             if not os.path.exists(self.config_path):
@@ -249,8 +281,8 @@ class ConfigManager:
             # Signature taken AFTER load + migration (migration may write the
             # config back), so it reflects exactly what was read/written.
             self._loaded_sig = self._files_signature()
-            return self.config
+            return _private_copy(self.config)
-            
+
         except FileNotFoundError as e:
             # Only config.json can get here: a missing or unreadable secrets
             # file is handled where it is read.
@@ -355,8 +387,9 @@ class ConfigManager:
         try:
             atomic_write_json(self.config_path, config_to_write)
 
-            # Update the in-memory config to the new state (which includes secrets for runtime)
+            # Update the in-memory config to the new state (which includes
-            self.config = new_config_data
+            # secrets for runtime), as a copy -- see _private_copy
+            self.config = _private_copy(new_config_data)
             self._loaded_sig = self._files_signature()
             self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
             if secrets_content:

src/display_controller.py

@@ -368,6 +368,10 @@ class DisplayController:
         self.on_demand_last_error: Optional[str] = None
         self.on_demand_last_event: Optional[str] = None
         self.on_demand_schedule_override = False
+        # The mode the request named, when it named one (not a mode resolved
+        # from a bare plugin id). Shown even when the plugin's live checks
+        # would leave it out of the session (_on_demand_modes_for_plugin).
+        self._on_demand_named_mode: Optional[str] = None
         # Plugins that are disabled in config and loaded only because an
         # on-demand request named them. The main loop unloads each one once
         # on-demand has moved off it (_release_on_demand_plugins).
@@ -561,6 +565,17 @@ class DisplayController:
         except Exception:  # pylint: disable=broad-except
             logger.exception("Plugin system initialization failed")
             self.plugin_manager = None
+            # A restored session has no plugin to resume on. It may have been
+            # read already (on_demand_active) or not yet, if initialization
+            # failed before the restore ran; either way, end it visibly.
+            try:
+                cached_session = self.cache_manager.get('display_on_demand_config',
+                                                        max_age=3600)
+            except Exception:  # pylint: disable=broad-except
+                cached_session = None
+            if self.on_demand_active or cached_session:
+                self.cache_manager.clear_cache('display_on_demand_config')
+                self._set_on_demand_error('restore-failed')
             # Its state machine no longer describes what runs; let the last
             # snapshot go stale (readers then say unknown) rather than keep
             # refreshing it.
@@ -1215,6 +1230,13 @@ class DisplayController:
                 note = getattr(self.plugin_manager, 'note_display_duration', None)
                 if note is not None and plugin_id:
                     note(plugin_id, time.monotonic() - started)
+                # A screen that drew once and holds makes no more
+                # update_display() calls, so a frame the preview throttle
+                # skipped would otherwise never reach the snapshot.
+                write_owed = getattr(getattr(self, 'display_manager', None),
+                                     'write_owed_snapshot', None)
+                if write_owed is not None:
+                    write_owed()
 
     def _health_tracker(self):
         """The plugin circuit breaker, or None when it is not enabled."""
@@ -1665,6 +1687,7 @@ class DisplayController:
         self.on_demand_expires_at = None
         self.on_demand_pinned = False
         self.on_demand_schedule_override = False
+        self._on_demand_named_mode = None
         # While the session ran, _evaluate_schedule may have forced
         # is_display_active on over a scheduled-off answer. Drop the minute
         # gate so the next _check_schedule recomputes it; otherwise the panel
@@ -1831,6 +1854,7 @@ class DisplayController:
         self.on_demand_pinned = on_demand_config.get('pinned', False)
         self.on_demand_requested_at = on_demand_config.get('requested_at')
         self.on_demand_expires_at = on_demand_config.get('expires_at')
+        self._on_demand_named_mode = on_demand_config.get('named_mode')
         self.on_demand_status = 'active'
         self.on_demand_schedule_override = True
         logger.info("On-demand mode detected during initialization: resuming on plugin '%s'; "
@@ -2317,13 +2341,25 @@ class DisplayController:
                 return modes[0]
         return plugin_id
 
-    def _on_demand_modes_for_plugin(self, plugin_id: str) -> List[str]:
+    def _on_demand_modes_for_plugin(self, plugin_id: str,
+                                    named_mode: Optional[str] = None) -> List[str]:
         """Every loaded display mode belonging to `plugin_id`, in rotation order.
 
         Live modes that actually have content lead, then the rest, then live
         modes with nothing to show -- so an on-demand request for a sports
         plugin opens on a game in progress rather than an empty live screen.
         Returns an empty list when the plugin has no loaded modes.
+
+        `named_mode` is a mode the request asked for by name. It is always
+        in the list, first when the checks below would have dropped it.
+        Those checks ask has_live_content(), which is the live-priority
+        question -- "should this plugin take the panel from the rotation?"
+        -- and the sports plugins answer it for favourite teams only. Asking
+        for ncaa_fb_live with fifteen games on and no favourite playing got
+        a 200 and nfl_recent on the panel. The plugin's display() is what
+        knows whether the mode has anything to draw; when it has not, the
+        session moves to the plugin's next mode like any empty on-demand
+        mode.
         """
         plugin_modes = self.plugin_display_modes.get(plugin_id, [])
         if not plugin_modes:
@@ -2364,6 +2400,15 @@ class DisplayController:
             # Only live modes available but no content - use them anyway
             ordered_modes = live_modes
 
+        if named_mode and named_mode in available_plugin_modes:
+            # The named mode leads whether or not the live check kept it: a
+            # second live mode with content is already in the list, but behind
+            # the first, so the session would rotate away before reaching it.
+            if named_mode not in ordered_modes:
+                logger.info("On-demand: showing %s as requested; plugin '%s' reports no "
+                            "live-priority content for it", named_mode, plugin_id)
+            ordered_modes = [named_mode] + [m for m in ordered_modes if m != named_mode]
+
         return ordered_modes
 
     def _apply_on_demand_pin(self, ordered_modes: List[str], resolved_mode: Optional[str],
@@ -2392,10 +2437,20 @@ class DisplayController:
 
         plugin_id = self.on_demand_plugin_id
 
-        ordered_modes = self._on_demand_modes_for_plugin(plugin_id)
+        ordered_modes = self._on_demand_modes_for_plugin(plugin_id, self._on_demand_named_mode)
         if not ordered_modes:
-            logger.warning("No valid display modes found for on-demand plugin '%s' after restoration", plugin_id)
+            # The plugin did not load this time (seen on a rig: its config
-            self.on_demand_modes = []
+            # failed validation after the crash that caused the restart), so
+            # there is nothing to resume. Leaving the session active with no
+            # modes published it as active for a plugin that was not running
+            # until the first pass ended it as an ordinary 'idle', and kept
+            # the cached request for the next restart to trip over. End it
+            # as a failure the status endpoint reports, and drop the cache.
+            logger.error("On-demand session for plugin '%s' cannot resume after the "
+                         "restart: the plugin has no loaded display modes (did it "
+                         "fail to load?); ending it", plugin_id)
+            self.cache_manager.clear_cache('display_on_demand_config')
+            self._set_on_demand_error('restore-failed')
             return
 
         # A restart must not silently un-pin: the pin is part of the request
@@ -2596,7 +2651,10 @@ class DisplayController:
         if resolved_mode in self.available_modes:
             self.current_mode_index = self.available_modes.index(resolved_mode)
 
-        ordered_modes = self._on_demand_modes_for_plugin(resolved_plugin_id)
+        # Named: the request gave this mode itself, rather than a plugin id
+        # (or a mode the plugin doesn't have) that resolved to a default.
+        named_mode = resolved_mode if mode == resolved_mode else None
+        ordered_modes = self._on_demand_modes_for_plugin(resolved_plugin_id, named_mode)
         if not ordered_modes:
             logger.error("No valid display modes found for plugin '%s'", resolved_plugin_id)
             self._set_on_demand_error("no-modes")
@@ -2613,6 +2671,7 @@ class DisplayController:
         self.on_demand_requested_at = now
         self.on_demand_expires_at = (now + duration) if duration else None
         self.on_demand_pinned = pinned
+        self._on_demand_named_mode = named_mode
         self.on_demand_status = 'active'
         self.on_demand_last_error = None
         self.on_demand_last_event = 'started'
@@ -2641,6 +2700,7 @@ class DisplayController:
             'mode': resolved_mode,
             'duration': duration,
             'pinned': pinned,
+            'named_mode': named_mode,
             'requested_at': now,
             'expires_at': self.on_demand_expires_at
         }

src/display_manager.py

@@ -317,6 +317,11 @@ class DisplayManager:
         # is handed to the writer; this only once it has been saved, so an
         # mtime touch never vouches for a frame still waiting to be written.
         self._saved_snapshot_digest: Optional[int] = None
+        # A changed frame reached _write_snapshot_if_due() inside the write
+        # interval and was skipped. Nothing writes it unless update_display()
+        # runs again, and a screen that draws once and holds never calls it
+        # again -- see write_owed_snapshot().
+        self._snapshot_owed = False
         self._snapshot_dir_prepared = False
         # Background writer used mid-scroll; see _write_snapshot_if_due.
         self._snapshot_cond = threading.Condition()
@@ -388,6 +393,11 @@ class DisplayManager:
         
         self._setup_matrix()
         logger.info("Matrix setup completed in %.3f seconds", time.time() - start_time)
+        # Only a real panel's swaps wait on its refresh: the emulator and the
+        # fallback canvas pace themselves, so "slower than the cap" would be
+        # noise there.
+        if self.matrix is not None and os.environ.get('EMULATOR', 'false') != 'true':
+            self.frame_timing.plan_refresh(self.refresh_hz)
         self._setup_scan_order_compensation()
         
         font_time = time.time()
@@ -1496,7 +1506,9 @@ class DisplayManager:
         fractional-pixel motion. See src/common/scroll_config.py.
 
         Note this is the configured *cap*, not necessarily what the panel
-        achieves -- scripts/scroll_speeds.py --measure reports the real rate.
+        achieves -- scripts/scroll_speeds.py --measure reports the real rate,
+        and the frame-timing recorder logs a warning, with a cap the panel can
+        hold, once it has measured a panel that falls short of this.
         """
         hardware = (self.config.get('display') or {}).get('hardware') or {}
         try:
@@ -1788,9 +1800,10 @@ class DisplayManager:
 
             if frame_checksum is not None:
                 digest = frame_checksum
+                frame_changed = digest != self._last_snapshot_digest
                 action = snapshot_policy.decide(
                     now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
-                    viewer_fresh, digest != self._last_snapshot_digest)
+                    viewer_fresh, frame_changed)
             else:
                 # Ask as if the frame had changed before paying to find out.
                 # decide() is monotone in frame_changed -- a SKIP for a
@@ -1802,22 +1815,36 @@ class DisplayManager:
                     now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
                     viewer_fresh, True)
                 if action is snapshot_policy.SnapshotAction.SKIP:
+                    # Not hashed, so not known to be unchanged: owed until a
+                    # later look finds it written or unchanged.
+                    self._snapshot_owed = True
                     return
                 digest = zlib.adler32(self.image.tobytes())
-                if digest == self._last_snapshot_digest:
+                frame_changed = digest != self._last_snapshot_digest
+                if not frame_changed:
                     # Unchanged after all: the decision an unchanged frame gets.
                     action = snapshot_policy.decide(
                         now, self._last_snapshot_ts,
                         self._last_snapshot_touch_ts, viewer_fresh, False)
             if action is snapshot_policy.SnapshotAction.SKIP:
+                # A changed frame inside the write interval stays owed: the
+                # next update_display() would write it, but a static screen
+                # may not make one -- write_owed_snapshot() covers that.
+                self._snapshot_owed = frame_changed
                 return
             if (action is snapshot_policy.SnapshotAction.TOUCH
                     and self._saved_snapshot_digest == digest):
                 # mtime bump only: keeps the health check (snapshot age)
                 # green without paying for a PNG encode of an unchanged frame
+                # (this frame is already on disk, so nothing is owed).
+                self._snapshot_owed = False
                 os.utime(self._snapshot_path, None)
                 self._last_snapshot_touch_ts = now
                 return
+            # Owed until the write below succeeds: if it raises, the frame
+            # stays owed and write_owed_snapshot() retries it, rather than a
+            # held screen leaving the preview stale after one failed write.
+            self._snapshot_owed = True
             # (A TOUCH for a frame that isn't on disk yet -- still queued, or
             # its write failed -- is written instead: touching would make the
             # older file on disk look current.)
@@ -1842,9 +1869,39 @@ class DisplayManager:
             self._last_snapshot_ts = now
             self._last_snapshot_touch_ts = now
             self._last_snapshot_digest = digest
+            self._snapshot_owed = False
         except Exception as e:
             self._log_snapshot_failure(e)
 
+    def write_owed_snapshot(self) -> None:
+        """Write a frame the snapshot throttle skipped, once it is due.
+
+        The preview snapshot is only ever written from update_display(), and
+        at most once per write interval (snapshot_policy). A frame pushed
+        inside that interval is skipped, and is written by the next
+        update_display() that comes after it -- but a screen that draws its
+        card once and then holds it makes no further call. Its frame was on
+        the panel and never in the preview: soccer's recent/upcoming cards
+        skip redundant redraws, and the first one after an on-demand start
+        (pushed a few milliseconds after the controller's clear) left
+        /api/v3/display/current and the web preview black for the whole
+        screen while the panel showed the card.
+
+        The render loop calls this after each frame. Cheap when nothing is
+        owed (one attribute read); otherwise the usual policy decides, so
+        the write still waits out the interval and an unchanged frame is
+        never re-encoded.
+        """
+        if not self._snapshot_owed:
+            return
+        try:
+            if self._writes_suppressed():
+                return
+            with self._update_lock:
+                self._write_snapshot_if_due()
+        except Exception as e:  # pylint: disable=broad-except
+            self._log_snapshot_failure(e)
+
     def _log_snapshot_failure(self, error: Exception) -> None:
         # Snapshot failures must never break display — but they must not
         # be silent either: the snapshot's mtime is the web UI's display

src/plugin_system/operation_types.py

@@ -48,12 +48,19 @@ class PluginOperation:
     completed_at: Optional[datetime] = None
     
     def to_dict(self) -> Dict[str, Any]:
-        """Convert operation to dictionary for serialization."""
+        """Convert operation to dictionary for serialization.
+
+        Parameters whose name starts with ``_`` are internal and left out:
+        PluginOperationQueue keeps the operation's callback there as
+        ``_callback`` until its worker runs it, and a pending operation's
+        status answered 500 because that function cannot be serialized.
+        """
         return {
             'operation_id': self.operation_id,
             'operation_type': self.operation_type.value,
             'plugin_id': self.plugin_id,
-            'parameters': self.parameters,
+            'parameters': {key: value for key, value in self.parameters.items()
+                           if not str(key).startswith('_')},
             'status': self.status.value,
             'progress': self.progress,
             'message': self.message,

test/_run_loop_harness.py

@@ -724,11 +724,14 @@ class RunLoopHarness:
         self.clock.at(t, post)
 
     def restore_on_demand(self, plugin_id: str, mode: Optional[str] = None,
-                          duration: Optional[float] = None, pinned: bool = False):
+                          duration: Optional[float] = None, pinned: bool = False,
+                          named_mode: Optional[str] = None):
         """Start with an on-demand session resumed from the cache, as after
         a restart: the state _select_startup_plugins restores, then
-        _populate_on_demand_modes_from_plugin, as __init__ calls it."""
+        _populate_on_demand_modes_from_plugin, as __init__ calls it. A
+        session that cannot resume is logged as ``on-demand-error``."""
         dc = self.controller
+        dc._on_demand_named_mode = named_mode
         dc.on_demand_active = True
         dc.on_demand_plugin_id = plugin_id
         dc.on_demand_mode = mode
@@ -739,6 +742,8 @@ class RunLoopHarness:
         dc.on_demand_status = 'active'
         dc.on_demand_schedule_override = True
         dc._populate_on_demand_modes_from_plugin()
+        if dc.on_demand_status == 'error':
+            self.log("on-demand-error", dc.on_demand_last_error)
 
     def wifi_message(self, t: float, message: str, duration: float = 5):
         def write():

test/fixtures/api_v3_url_map.json

@@ -175,6 +175,15 @@
       "POST"
     ]
   ],
+  [
+    "/api/v3/config/refresh-rate",
+    "api_v3.get_refresh_rate",
+    [
+      "GET",
+      "HEAD",
+      "OPTIONS"
+    ]
+  ],
   [
     "/api/v3/config/schedule",
     "api_v3.get_schedule_config",

test/fixtures/run_loop_golden/on_demand_named_live.json

@@ -0,0 +1,29 @@
+{
+  "screens": [
+    [0.0, "clock", 5.0, "on-demand-start", 6, false],
+    [5.0, "sports_live", 15.0, "duration", 15, true],
+    [20.0, "sports_recent", 15.0, "duration", 15, true],
+    [35.0, "sports_upcoming", 5.0, "on-demand-requested-stop", 6, true],
+    [40.0, "clock", 20.0, "duration", 20, true],
+    [60.0, "sports_live", 15.0, "display-false", 11, true],
+    [75.0, "sports_recent", 15.0, "duration", 15, true],
+    [90.0, "sports_upcoming", 10.0, "on-demand-start", 11, true],
+    [100.0, "sports_live", 0.0, "empty", 1, true],
+    [100.0, "sports_recent", 15.0, "duration", 15, true],
+    [115.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [130.0, "sports_live", 0.0, "empty", 1, true],
+    [130.0, "sports_recent", 10.0, "on-demand-requested-stop", 11, true],
+    [140.0, "sports_upcoming", 15.0, "duration", 15, true],
+    [155.0, "clock", 5.0, "horizon", 5, true]
+  ],
+  "events": [
+    [5.0, "request", "start:n1"],
+    [5.0, "on-demand-start", "sports"],
+    [40.0, "request", "stop:n2"],
+    [40.0, "on-demand-requested-stop"],
+    [100.0, "request", "start:n3"],
+    [100.0, "on-demand-start", "sports"],
+    [140.0, "request", "stop:n4"],
+    [140.0, "on-demand-requested-stop"]
+  ]
+}

test/fixtures/run_loop_golden/on_demand_restore_failed.json

@@ -0,0 +1,10 @@
+{
+  "screens": [
+    [0.0, "clock", 20.0, "duration", 20, false],
+    [20.0, "weather", 20.0, "duration", 20, true],
+    [40.0, "clock", 20.0, "horizon", 20, true]
+  ],
+  "events": [
+    [0.0, "on-demand-error", "restore-failed"]
+  ]
+}

test/js/README.md

@@ -50,6 +50,7 @@ server has none.
 | `unit/test_store_categories.js` | no | The store's category filter (sandbox): the template ships only All Categories, the rest come from the store's plugins (one per category whatever its case), choosing one filters to it, and a swapped-in select is refilled from the cache keeping the choice |
 | `unit/test_github_url_install.js` | no | Install Single Plugin (sandbox, the button as `plugins.html` ships it): no inline `onclick`, so a click or Enter sends exactly one `install-from-url` request and raises no error |
 | `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
+| `unit/test_plugin_order_list.js` | no | `widgets/plugin-order-list.js` (the Vegas and rotation order lists): a disabled plugin, which gets no row, keeps its slot in the saved order and its Vegas exclusion when the list rewrites its hidden inputs, around reordering and include/exclude; an uninstalled plugin's id is dropped, a failed plugin list leaves the inputs as saved, and only string ids are carried over, once each |
 | `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
 | `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
 | `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
@@ -57,6 +58,9 @@ server has none.
 | `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
 | `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
 | `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
+| `unit/test_overview_reconciliation_poll.js` | no | The Overview's reconciliation-banner poll from `partials/overview.html`, run in a vm: it gives up after a bounded number of requests when the status never says done, runs only while the Overview is on screen (`LEDVisibility`, its own key), and stops once the banner is shown |
+| `unit/test_display_partial_ids.js` | no | `partials/display.html`: every literal `getElementById()` in its inline scripts names an id the partial renders, and moving the brightness slider (the shipped script, in a vm with a fake DOM) updates its label without throwing |
+| `unit/test_general_web_login_token.js` | no | `window.webLogin.createToken` from `partials/general.html`, run in a vm: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
 | `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
 | `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
 | `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |

test/js/dom/test_durations_page.js

@@ -98,7 +98,11 @@ const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
 
   const lists = () => requests.filter(r => r.url === '/api/v3/plugins/installed').length;
   const $ = id => doc.getElementById(id);
-  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]');
+  // The rows' ids, in order. The input also keeps saved ids that have no row
+  // (a disabled plugin's place, see test/js/unit/test_plugin_order_list.js),
+  // and the saved order comes from whatever config the server has.
+  const SHOWN = plugins.filter(p => p.enabled).map(p => p.id);
+  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]').filter(id => SHOWN.includes(id));
   async function swap() {
     panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
     panel.innerHTML = partial;

test/js/dom/test_tools_sections.js

@@ -98,8 +98,50 @@ const get = p => new Promise((res, rej) =>
   window.saveMqttBridge();
   await tick(150);
   ok('save includes password once typed', sent && sent.mqtt_password === 'typed-secret');
+
+  // A password with TLS off is refused unless allow_insecure_mqtt is set
+  // (CWE-319, api_v3/misc.py). The form has to be able to send it, or a
+  // plain-LAN broker with a password can never be saved from here.
+  const allowRow = () => $('mqtt-allow-insecure-row');
+  const shown = el => !!el && !el.classList.contains('hidden');
+  ok('allow-without-TLS control rendered', !!$('mqtt-allow-insecure'));
+  ok('allow-without-TLS starts as saved',
+     !!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === !!bridge.data.config.allow_insecure_mqtt);
+  ok('allow-without-TLS shown only while TLS is off',
+     shown(allowRow()) === !$('mqtt-tls').checked);
+  $('mqtt-tls').checked = true;
+  $('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
+  ok('ticking TLS hides it', !shown(allowRow()));
+  $('mqtt-tls').checked = false;
+  $('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
+  ok('unticking TLS shows it again', shown(allowRow()));
+
+  const setAllow = v => { if ($('mqtt-allow-insecure')) $('mqtt-allow-insecure').checked = v; };
+  setAllow(false);
+  window.saveMqttBridge();
+  await tick(150);
+  ok('save sends allow_insecure_mqtt false when unticked', !!sent && sent.allow_insecure_mqtt === false, sent);
+  setAllow(true);
+  window.saveMqttBridge();
+  await tick(150);
+  ok('save sends allow_insecure_mqtt true when ticked', !!sent && sent.allow_insecure_mqtt === true, sent);
   onPut = null;
 
+  // Prefilled from the saved settings, and hidden while TLS is saved on.
+  bridgePayload = JSON.parse(JSON.stringify(bridge));
+  bridgePayload.data.config.allow_insecure_mqtt = true;
+  bridgePayload.data.config.mqtt_tls = false;
+  window.loadMqttBridge();
+  await tick(150);
+  ok('a saved opt-in is prefilled', !!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === true);
+  bridgePayload.data.config.mqtt_tls = true;
+  window.loadMqttBridge();
+  await tick(150);
+  ok('hidden on load when TLS is saved on', !shown(allowRow()));
+  bridgePayload = bridge;
+  window.loadMqttBridge();
+  await tick(150);
+
   // ── Pixlet editor, idle ────────────────────────────────────────────────
   const appIds = (apps.data.apps || []).map(a => a.id);
   ok('editor lists the apps on disk',

test/js/run_all.js

@@ -17,6 +17,7 @@ const fs = require('fs');
 
 const BASE = process.env.BASE || 'http://localhost:5000';
 const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
+              'unit/test_plugin_order_list.js',
               'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
               'unit/test_style_editor_layout_leaf_columns.js',
               'unit/test_style_editor_layout_leaf_collision.js',
@@ -28,7 +29,10 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
               'unit/test_inline_handler_escaping.js',
               'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js',
               'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js',
-              'unit/test_page_registry.js', 'unit/test_core_modules.js'];
+              'unit/test_page_registry.js', 'unit/test_core_modules.js',
+              'unit/test_overview_reconciliation_poll.js',
+              'unit/test_display_partial_ids.js',
+              'unit/test_general_web_login_token.js'];
 const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
              'dom/test_tools_sections.js', 'dom/test_cache_page.js',
              'dom/test_durations_page.js', 'dom/test_operation_history_page.js',

test/js/unit/test_display_partial_ids.js

@@ -0,0 +1,108 @@
+// The Display tab's inline script must only look up elements the partial
+// renders.
+//
+// Its brightness slider handler also wrote to #brightness-display, a "LED
+// brightness: N%" line that #387 removed from partials/display.html. The
+// lookup returned null, so every movement of the slider threw a TypeError.
+// This checks every literal getElementById() in the partial's inline scripts
+// against the ids its markup renders, and runs the shipped script in a vm
+// with a fake DOM (null for an id the markup lacks, as in a browser) to move
+// the slider.
+//
+// No jsdom and no server needed.
+
+const fs = require('fs');
+const path = require('path');
+const vm = require('vm');
+
+const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/display.html');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
+
+const html = fs.readFileSync(PARTIAL, 'utf8');
+const blocks = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)];
+const scripts = blocks.map(m => m[1]);
+// The markup is what lies between the script blocks (sliced around them, not
+// a replace(), which CodeQL reads as an incomplete HTML sanitizer).
+let markup = '';
+let from = 0;
+for (const m of blocks) {
+  markup += html.slice(from, m.index);
+  from = m.index + m[0].length;
+}
+markup += html.slice(from);
+const rendered = new Set([...markup.matchAll(/\bid="([^"{}]+)"/g)].map(m => m[1]));
+
+console.log('\n── Display partial: element lookups ──');
+
+// 1. Static: every literal lookup names an id the partial renders.
+const lookups = scripts.flatMap(s => [...s.matchAll(/getElementById\('([^']+)'\)/g)].map(m => m[1]));
+const missing = [...new Set(lookups.filter(id => !rendered.has(id)))];
+ok('the inline scripts look elements up', lookups.length > 0, lookups.length);
+ok('every looked-up id is rendered by the partial', missing.length === 0, missing);
+
+// 2. Behaviour: moving the brightness slider updates its label and throws nothing.
+function fakeElement(id) {
+  const listeners = {};
+  const classes = new Set();
+  return {
+    id, value: '', textContent: '', min: '', max: '', checked: false,
+    style: {}, dataset: {}, className: '',
+    classList: {
+      add: c => classes.add(c), remove: c => classes.delete(c),
+      toggle: (c, on) => (on === undefined ? (classes.has(c) ? classes.delete(c) : classes.add(c)) : (on ? classes.add(c) : classes.delete(c))),
+      contains: c => classes.has(c),
+    },
+    addEventListener: (type, fn) => { (listeners[type] ||= []).push(fn); },
+    dispatchEvent() { return true; },
+    appendChild() {},
+    listeners,
+  };
+}
+
+const main = scripts.find(s => s.includes("getElementById('brightness')"));
+ok('found the script that wires the brightness slider', !!main);
+if (main) {
+  const elements = new Map();
+  const document = {
+    readyState: 'complete',
+    hidden: false,
+    getElementById: id => {
+      if (!rendered.has(id)) return null;
+      if (!elements.has(id)) elements.set(id, fakeElement(id));
+      return elements.get(id);
+    },
+    createElement: () => fakeElement(''),
+    createTextNode: () => ({}),
+    addEventListener() {},
+  };
+  const window = {
+    LEDEscape: { html: v => String(v), attr: v => String(v) },
+    LEDVisibility: { onActive() {} },
+  };
+  const context = {
+    window, document, console, URLSearchParams,
+    fetch: () => new Promise(() => {}),
+    setTimeout: () => 0, clearTimeout() {}, setInterval: () => 0, clearInterval() {},
+  };
+  vm.createContext(context);
+  let loadError = null;
+  try { vm.runInContext(main, context); } catch (e) { loadError = e; }
+  ok('the script loads', !loadError, loadError && String(loadError));
+
+  const slider = elements.get('brightness');
+  const handlers = (slider && slider.listeners.input) || [];
+  ok('the slider has an input handler', handlers.length > 0);
+  let thrown = null;
+  slider.value = '42';
+  try { handlers.forEach(fn => fn.call(slider, { target: slider })); } catch (e) { thrown = e; }
+  ok('moving the slider throws nothing', !thrown, thrown && String(thrown));
+  ok('...and shows the new value', elements.get('brightness-value').textContent === '42',
+     elements.get('brightness-value').textContent);
+}
+
+console.log(`\n${pass} passed, ${fail} failed\n`);
+process.exit(fail ? 1 : 0);

test/js/unit/test_general_web_login_token.js

@@ -0,0 +1,107 @@
+// Creating an API token on the General tab must leave its form clean.
+//
+// app.js marks a form data-dirty on any input in it and clears the mark only
+// after a successful htmx request; its beforeunload handler then asks "Leave
+// site?" while any visible form is still dirty. The token form posts with
+// fetch (window.webLogin.createToken in partials/general.html), so after a
+// token was created the form stayed dirty and reloading the page while the
+// General tab was open prompted about changes that had been saved.
+//
+// Runs the shipped inline script in a vm with a fake fetch and DOM -- no jsdom
+// and no server needed.
+
+const fs = require('fs');
+const path = require('path');
+const vm = require('vm');
+
+const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/general.html');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
+
+function webLoginScript() {
+  const html = fs.readFileSync(PARTIAL, 'utf8');
+  const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
+  const found = scripts.find(s => s.includes('window.webLogin = {'));
+  if (!found) throw new Error('webLogin script not found in general.html');
+  return found;
+}
+
+function el() {
+  const classes = new Set(['hidden']);
+  return {
+    textContent: '', dataset: {}, style: {}, className: '',
+    classList: { add: c => classes.add(c), remove: c => classes.delete(c), contains: c => classes.has(c) },
+    appendChild() {}, addEventListener() {}, querySelector: () => null,
+  };
+}
+
+function load(answer) {
+  const elements = {
+    'web-login-tokens': el(),
+    'web-login-new-token-value': el(),
+    'web-login-new-token': el(),
+  };
+  const notes = [];
+  const window = { showNotification: (m, t) => notes.push([m, t]), alert() {}, confirm: () => true };
+  const context = {
+    window, console,
+    document: {
+      getElementById: id => elements[id] || null,
+      createElement: () => el(),
+      querySelectorAll: () => [],
+    },
+    fetch: () => Promise.resolve({
+      ok: answer.ok, status: answer.ok ? 200 : 400,
+      json: () => Promise.resolve(answer.body),
+    }),
+  };
+  vm.createContext(context);
+  vm.runInContext(webLoginScript(), context);
+  return { webLogin: context.window.webLogin, elements, notes };
+}
+
+function dirtyForm() {
+  const attrs = new Map([['data-dirty', '']]);
+  return {
+    querySelector: sel => (sel === '[name="name"]' ? { value: 'Home Assistant' } : null),
+    reset() {},
+    hasAttribute: name => attrs.has(name),
+    setAttribute: (name, value) => attrs.set(name, String(value)),
+    removeAttribute: name => attrs.delete(name),
+  };
+}
+
+const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
+
+(async () => {
+  console.log('\n── General tab: API token form ──');
+
+  {
+    const t = load({ ok: true, body: {
+      status: 'success', message: 'Token created',
+      data: { token: 'lmx_secret', record: { id: 't1', name: 'Home Assistant', prefix: 'lmx_sec' } },
+    } });
+    const form = dirtyForm();
+    t.webLogin.createToken(form);
+    await flush();
+    ok('the new token is shown', t.elements['web-login-new-token-value'].textContent === 'lmx_secret');
+    ok('a created token leaves the form clean (no "Leave site?" on reload)',
+       !form.hasAttribute('data-dirty'));
+  }
+
+  {
+    const t = load({ ok: false, body: { status: 'error', message: 'Name is required' } });
+    const form = dirtyForm();
+    t.webLogin.createToken(form);
+    await flush();
+    ok('a refused request reports the error', t.notes.some(([m, type]) => type === 'error' && /Name is required/.test(m)),
+       t.notes);
+    ok('...and keeps the form dirty: nothing was saved', form.hasAttribute('data-dirty'));
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed\n`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });

test/js/unit/test_overview_reconciliation_poll.js

@@ -0,0 +1,137 @@
+// The Overview's "Plugin Config Warning" poll must end.
+//
+// The banner script in partials/overview.html asks
+// /api/v3/plugins/reconciliation-status every 2 s until startup reconciliation
+// says it is done. The route answers done: false whenever its status file is
+// missing -- reconciliation raised before writing it, or /tmp was cleaned
+// under a long-running web service -- so the poll used to run every 2 s for
+// as long as the page stayed open, on every tab. It now gives up after a
+// bounded number of tries and runs only while the Overview is on screen
+// (LEDVisibility, like the other partials' pollers).
+//
+// Runs the shipped inline script in a vm with fake timers, fetch and DOM --
+// no jsdom and no server needed.
+
+const fs = require('fs');
+const path = require('path');
+const vm = require('vm');
+
+const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/overview.html');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
+
+function bannerScript() {
+  const html = fs.readFileSync(PARTIAL, 'utf8');
+  const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
+  const found = scripts.find(s => s.includes('ledmatrix-recon-dismissed'));
+  if (!found) throw new Error('reconciliation banner script not found in overview.html');
+  return found;
+}
+
+const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
+
+function load({ payload, visibility = true }) {
+  const timers = new Map();
+  let nextId = 1;
+  const calls = [];
+  const banner = { style: { setProperty() {} }, dataset: {} };
+  const text = { textContent: '' };
+  const registrations = [];
+  const window = {};
+  if (visibility) {
+    window.LEDVisibility = {
+      onActive(tab, start, stop, key) { registrations.push({ tab, start, stop, key }); start(); },
+    };
+  }
+  const context = {
+    window,
+    document: {
+      getElementById: id => ({ 'reconciliation-banner': banner, 'reconciliation-banner-text': text })[id] || null,
+    },
+    sessionStorage: { getItem: () => null, setItem() {} },
+    fetch: (url) => {
+      calls.push(url);
+      return Promise.resolve({ json: () => Promise.resolve(payload()) });
+    },
+    setTimeout: (fn) => { const id = nextId++; timers.set(id, fn); return id; },
+    clearTimeout: (id) => { timers.delete(id); },
+  };
+  vm.createContext(context);
+  vm.runInContext(bannerScript(), context);
+  const fireTimers = async () => {
+    const due = [...timers.entries()];
+    timers.clear();
+    due.forEach(([, fn]) => fn());
+    await flush();
+  };
+  return { calls, timers, registrations, banner, text, window, fireTimers };
+}
+
+(async () => {
+  console.log('\n── Overview reconciliation poll ──');
+
+  // 1. A status file that never says done: the poll stops on its own.
+  {
+    const t = load({ payload: () => ({ status: 'success', data: { done: false, unresolved: [] } }) });
+    await flush();
+    for (let i = 0; i < 200; i++) await t.fireTimers();
+    ok('a status that never turns done stops being polled', t.timers.size === 0,
+       { pending: t.timers.size, requests: t.calls.length });
+    ok('...after a bounded number of requests (at most 30, a minute at 2 s)',
+       t.calls.length > 1 && t.calls.length <= 30, t.calls.length);
+  }
+
+  // 2. Runs only while the Overview is on screen.
+  {
+    const t = load({ payload: () => ({ status: 'success', data: { done: false, unresolved: [] } }) });
+    await flush();
+    const reg = t.registrations[0];
+    ok('registers with LEDVisibility for the overview tab', !!reg && reg.tab === 'overview', reg && reg.tab);
+    ok('under its own key, so it does not replace another overview poller',
+       !!reg && !!reg.key && reg.key !== 'overview', reg && reg.key);
+    ok('first request goes out at once', t.calls.length === 1, t.calls.length);
+    if (reg) {
+      reg.stop();
+      ok('leaving the tab cancels the pending retry', t.timers.size === 0, t.timers.size);
+      for (let i = 0; i < 5; i++) await t.fireTimers();
+      ok('no requests while another tab is active', t.calls.length === 1, t.calls.length);
+      reg.start();
+      await flush();
+      ok('coming back asks again at once', t.calls.length === 2, t.calls.length);
+      ok('...and keeps polling', t.timers.size === 1, t.timers.size);
+    }
+  }
+
+  // 3. A finished reconciliation with findings shows the banner and stops.
+  {
+    let done = false;
+    const t = load({ payload: () => (done
+      ? { status: 'success', data: { done: true, unresolved: [{ plugin_id: 'clock', type: 'plugin_missing_on_disk' }] } }
+      : { status: 'success', data: { done: false, unresolved: [] } }) });
+    await flush();
+    await t.fireTimers();
+    done = true;
+    await t.fireTimers();
+    ok('the banner names the finding once reconciliation is done',
+       t.text.textContent.includes('clock'), t.text.textContent);
+    const before = t.calls.length;
+    for (let i = 0; i < 5; i++) await t.fireTimers();
+    ok('no more requests once it is done', t.calls.length === before && t.timers.size === 0,
+       { before, after: t.calls.length, pending: t.timers.size });
+  }
+
+  // 4. Without LEDVisibility (base.html always has it) it still runs, bounded.
+  {
+    const t = load({ visibility: false, payload: () => ({ status: 'success', data: { done: false } }) });
+    await flush();
+    ok('runs without LEDVisibility', t.calls.length === 1, t.calls.length);
+    for (let i = 0; i < 200; i++) await t.fireTimers();
+    ok('...and is still bounded', t.timers.size === 0 && t.calls.length <= 30, t.calls.length);
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed\n`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });

test/js/unit/test_plugin_order_list.js

@@ -0,0 +1,189 @@
+// The shared plugin order list (widgets/plugin-order-list.js) keeps what it
+// does not show.
+//
+// It lists enabled plugins only, and rewrites its hidden inputs from those
+// rows as soon as it has drawn them. A disabled plugin's place in the order
+// and its Vegas exclusion used to vanish from the inputs on that rewrite, so
+// any later save of the Display or Rotation & Durations tab stored them
+// without it: re-enabled, the plugin came back at the end of the rotation and
+// scrolling in Vegas again. An uninstalled plugin's id is still dropped, as
+// before, so the lists don't collect ids nothing can show. Runs the shipped
+// widget in a vm with a minimal fake DOM -- no jsdom and no server needed, so
+// it runs under test/test_js_unit_suites.py too.
+
+const fs = require('fs');
+const path = require('path');
+const vm = require('vm');
+const WIDGET = path.resolve(__dirname, '../../../web_interface/static/v3/js/widgets/plugin-order-list.js');
+
+let pass = 0, fail = 0;
+const ok = (label, cond, extra) => cond
+  ? (pass++, console.log('  ok   ' + label))
+  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
+const same = (a, b) => JSON.stringify(a) === JSON.stringify(b);
+
+class FakeElement {
+  constructor(tag) {
+    this.tagName = tag.toUpperCase();
+    this.children = [];
+    this.parent = null;
+    this.dataset = {};
+    this.style = {};
+    this.className = '';
+    this.value = '';
+    this.checked = false;
+    this.listeners = {};
+    this._text = '';
+  }
+  appendChild(child) {
+    if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
+    child.parent = this;
+    this.children.push(child);
+    return child;
+  }
+  insertBefore(child, ref) {
+    if (!ref) return this.appendChild(child);
+    if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
+    child.parent = this;
+    this.children.splice(this.children.indexOf(ref), 0, child);
+    return child;
+  }
+  get previousElementSibling() {
+    const siblings = this.parent ? this.parent.children : [];
+    return siblings[siblings.indexOf(this) - 1] || null;
+  }
+  get nextElementSibling() {
+    const siblings = this.parent ? this.parent.children : [];
+    const i = siblings.indexOf(this);
+    return i < 0 ? null : siblings[i + 1] || null;
+  }
+  set textContent(value) { this._text = value; this.children = []; }
+  get textContent() { return this._text; }
+  setAttribute() {}
+  focus() {}
+  addEventListener(type, fn) { (this.listeners[type] ||= []).push(fn); }
+  fire(type, event) { (this.listeners[type] || []).forEach(fn => fn.call(this, event || {})); }
+  descendants() { return this.children.flatMap(c => [c, ...c.descendants()]); }
+  querySelectorAll(selector) {
+    const cls = selector.replace(/^\./, '');
+    return this.descendants().filter(e => e.className.split(/\s+/).includes(cls));
+  }
+  querySelector(selector) { return this.querySelectorAll(selector)[0] || null; }
+}
+
+/** Run the widget over `plugins` with the given saved inputs; resolves once it has drawn. */
+async function mount({ plugins, order, excluded, fetchFails }) {
+  const els = {
+    list: new FakeElement('div'),
+    order: Object.assign(new FakeElement('input'), { value: JSON.stringify(order) }),
+  };
+  if (excluded !== undefined) {
+    els.excluded = Object.assign(new FakeElement('input'), { value: JSON.stringify(excluded) });
+  }
+  const context = {
+    // The widget logs a failed list; expected there, so kept off the output.
+    console: fetchFails ? Object.assign({}, console, { error: () => {} }) : console,
+    window: {},
+    document: {
+      getElementById: (id) => els[id] || null,
+      createElement: (tag) => new FakeElement(tag),
+      createTextNode: (text) => new FakeElement('#text'),
+    },
+    fetch: () => (fetchFails ? Promise.reject(new Error('service restarting')) : Promise.resolve({
+      json: () => Promise.resolve({ status: 'success', data: { plugins } }),
+    })),
+  };
+  vm.createContext(context);
+  vm.runInContext(fs.readFileSync(WIDGET, 'utf8'), context);
+  context.window.PluginOrderList.init({
+    containerId: 'list', orderInputId: 'order',
+    excludedInputId: excluded !== undefined ? 'excluded' : undefined,
+  });
+  await new Promise(resolve => setTimeout(resolve, 0));
+  const rows = () => els.list.querySelectorAll('.plugin-order-item');
+  return {
+    rows,
+    rowIds: () => rows().map(r => r.dataset.pluginId),
+    order: () => JSON.parse(els.order.value),
+    excluded: () => JSON.parse(els.excluded.value),
+    row: (id) => rows().find(r => r.dataset.pluginId === id),
+  };
+}
+
+const PLUGINS = [
+  { id: 'weather', name: 'Weather', enabled: true },
+  { id: 'clock', name: 'Clock', enabled: false },
+  { id: 'stocks', name: 'Stocks', enabled: true },
+];
+
+(async () => {
+  console.log('\nVegas: a disabled plugin keeps its place and its exclusion');
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 'clock', 'stocks'], excluded: ['clock'] });
+    ok('only enabled plugins get a row', same(t.rowIds(), ['weather', 'stocks']), t.rowIds());
+    ok('drawing the list keeps the disabled plugin in the order, in its place',
+       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
+    ok('drawing the list keeps its exclusion', same(t.excluded(), ['clock']), t.excluded());
+
+    // Move Stocks up: the rows swap, and Clock stays in its saved slot.
+    const up = t.row('stocks').querySelectorAll('.plugin-order-move')[0];
+    up.fire('click');
+    ok('reordering the rows fills the other slots in the new order',
+       same(t.order(), ['stocks', 'clock', 'weather']), t.order());
+
+    const include = t.row('weather').querySelector('.plugin-order-include');
+    include.checked = false;
+    include.fire('change');
+    ok('unchecking a row adds it, and the disabled exclusion stays',
+       same([...t.excluded()].sort(), ['clock', 'weather']), t.excluded());
+    include.checked = true;
+    include.fire('change');
+    ok('checking it again removes only that one', same(t.excluded(), ['clock']), t.excluded());
+  }
+
+  console.log('\nRotation order: the same, without exclusions');
+  {
+    const plugins = [
+      { id: 'clock', enabled: true },
+      { id: 'off', enabled: false },
+      { id: 'weather', enabled: true },
+      { id: 'new', enabled: true },
+    ];
+    const t = await mount({ plugins, order: ['clock', 'off', 'weather'] });
+    ok('the disabled plugin keeps its slot; a plugin not in the saved order goes last',
+       same(t.order(), ['clock', 'off', 'weather', 'new']), t.order());
+  }
+
+  console.log('\nAn uninstalled plugin is dropped; a failed list keeps everything');
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
+                            excluded: ['gone', 'clock'] });
+    ok('the disabled plugin is kept and the uninstalled one dropped from the order',
+       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
+    ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
+  }
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
+                            excluded: ['gone', 'clock'], fetchFails: true });
+    // No installed list, so nothing can be told apart: no rows, and the
+    // inputs keep what was saved, uninstalled ids included.
+    ok('a failed plugin list draws no rows', t.rowIds().length === 0, t.rowIds());
+    ok('and leaves the saved order as it was',
+       same(t.order(), ['weather', 'gone', 'clock', 'stocks']), t.order());
+    ok('and the saved exclusions', same(t.excluded(), ['gone', 'clock']), t.excluded());
+  }
+
+  console.log('\nOnly what the server would accept is carried over');
+  {
+    const t = await mount({ plugins: PLUGINS, order: ['weather', 7, 'clock', null, 'clock', 'stocks'],
+                            excluded: ['clock', 3, 'clock'] });
+    // /config/main refuses a list holding anything but strings, which would
+    // block every later Display save; a repeated id is kept once.
+    ok('non-string and repeated saved ids are dropped from the order',
+       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
+    ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
+  }
+
+  console.log(`\n${pass} passed, ${fail} failed`);
+  process.exit(fail ? 1 : 0);
+})().catch(e => { console.error(e); process.exit(1); });

test/test_api_v3_on_demand_restart.py

@@ -37,6 +37,7 @@ from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401
 START_URL = "/api/v3/display/on-demand/start"
 STOP_URL = "/api/v3/display/on-demand/stop"
 MAILBOX = "display_on_demand_request"
+DISPLAY = "web_interface.blueprints.api_v3.display"
 
 
 @pytest.fixture
@@ -150,6 +151,102 @@ class TestStartWhileTheServiceIsStopped:
         assert response.get_json()["status"] == "error"
 
 
+class _Mailbox:
+    """The CacheManager calls the routes make, over a dict."""
+
+    def __init__(self):
+        self.entries = {}
+
+    def set(self, key, value, ttl=None):
+        self.entries[key] = value
+
+    def get(self, key, max_age=300, memory_ttl=None):
+        return self.entries.get(key)
+
+    def delete(self, key):
+        self.entries.pop(key, None)
+
+
+class TestARefusedStartLeavesNoRequestBehind:
+    """A start the route answers with an error must not run later.
+
+    The request was posted (to the mailbox, with the display stopped) before
+    the route refused it, and the display reads the mailbox for an hour
+    without looking at a request's age. So "Display service is not running"
+    (start_service off) or "Failed to start display service" left the
+    request waiting, and the next time the display started -- minutes later,
+    by hand -- it ran that plugin, pinned if the request said so.
+
+    A socket acknowledgement is the other side of it: the display answered,
+    so it is running and has the request, whatever systemd says (a display
+    run by hand or in the emulator has no active unit). That is a success,
+    not "not running", and no unit is started beside it.
+    """
+
+    @pytest.fixture
+    def mailbox(self, api_v3_module, service):
+        box = _Mailbox()
+        api_v3_module.api_v3.cache_manager = box
+        service["state"]["active"] = False
+        return box
+
+    @pytest.mark.parametrize("body", [
+        {"plugin_id": "weather", "start_service": False},
+        {"plugin_id": "weather"},                           # start_service defaults on
+    ])
+    def test_a_socket_ack_is_a_success_whatever_systemd_says(
+            self, api_v3_client, service, mailbox, body):
+        with patch(f"{DISPLAY}.control_client.on_demand_start",
+                   side_effect=lambda request_id, *a: {"accepted": True}):
+            response = api_v3_client.post(START_URL, json=body)
+        assert response.status_code == 200, response.get_json()
+        assert response.get_json()["data"]["transport"] == "socket"
+        assert MAILBOX not in mailbox.entries
+        assert _systemctl_verbs(service["systemctl"]) == [], (
+            "a unit was started beside a display that answered the socket")
+
+    def test_without_start_service_the_request_is_taken_back(
+            self, api_v3_client, service, mailbox):
+        response = api_v3_client.post(START_URL, json={
+            "plugin_id": "weather", "pinned": True, "start_service": False})
+        assert response.status_code == 400
+        assert response.get_json()["status"] == "error"
+        assert MAILBOX not in mailbox.entries
+
+    def test_a_start_that_fails_takes_its_request_back(self, api_v3_client, service, mailbox):
+        service["systemctl"].side_effect = lambda args: {
+            "returncode": 1, "stdout": "", "stderr": "denied"}
+        response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
+        assert response.status_code == 500
+        assert MAILBOX not in mailbox.entries
+
+    def test_a_newer_request_is_left_alone_on_the_400(self, api_v3_client, service, mailbox):
+        newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
+
+        def stopped_and_another_post_lands(*args):
+            mailbox.entries[MAILBOX] = newer
+            return {"active": False}
+
+        with patch(f"{DISPLAY}._get_display_service_status",
+                   side_effect=stopped_and_another_post_lands):
+            response = api_v3_client.post(START_URL, json={
+                "plugin_id": "weather", "start_service": False})
+        assert response.status_code == 400
+        assert mailbox.entries[MAILBOX] is newer
+
+    def test_a_newer_request_is_left_alone_on_the_500(self, api_v3_client, service, mailbox):
+        newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
+
+        def start_fails_after_another_post(args):
+            mailbox.entries[MAILBOX] = newer
+            return {"returncode": 1, "stdout": "", "stderr": "denied"}
+
+        service["systemctl"].side_effect = start_fails_after_another_post
+        response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
+        assert response.status_code == 500
+        assert mailbox.entries[MAILBOX] is newer
+
+
 class TestStop:
     def test_stop_posts_a_stop_request_and_leaves_the_service_running(
             self, api_v3_client, service):

test/test_api_v3_operation_status_pending.py

@@ -0,0 +1,83 @@
+"""GET /api/v3/plugins/operation/<id> answers for an operation still waiting.
+
+PluginOperationQueue keeps an operation's callback in its parameters, under
+``_callback``, until the worker takes it to run. PluginOperation.to_dict()
+returned the parameters as they were, so for a pending operation the route
+handed jsonify a function and answered 500 "A system error occurred". That
+is every poll of an install queued behind another plugin's: the second of
+two installs read as broken until the first one finished.
+"""
+
+import json
+import sys
+import threading
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
+
+from src.plugin_system.operation_queue import PluginOperationQueue  # noqa: E402
+from src.plugin_system.operation_types import (  # noqa: E402
+    OperationType, PluginOperation,
+)
+
+
+def _callback(op):
+    return {"success": True, "message": "done"}
+
+
+class TestToDict:
+    def test_private_parameters_are_left_out(self):
+        op = PluginOperation(OperationType.INSTALL, "demo",
+                             parameters={"_callback": _callback, "branch": "main"})
+        assert op.to_dict()["parameters"] == {"branch": "main"}
+        json.dumps(op.to_dict())   # serializable
+
+    def test_the_operation_keeps_its_callback_for_the_worker(self):
+        op = PluginOperation(OperationType.INSTALL, "demo",
+                             parameters={"_callback": _callback})
+        op.to_dict()
+        assert op.parameters["_callback"] is _callback
+
+    def test_the_other_fields_are_unchanged(self):
+        op = PluginOperation(OperationType.UNINSTALL, "demo", operation_id="op-1")
+        assert op.to_dict() == {
+            "operation_id": "op-1", "operation_type": "uninstall", "plugin_id": "demo",
+            "parameters": {}, "status": "pending", "progress": 0.0, "message": "",
+            "error": None, "result": None,
+            "created_at": op.created_at.isoformat(), "started_at": None,
+            "completed_at": None,
+        }
+
+
+class TestTheRoute:
+    @pytest.fixture
+    def busy_queue(self, api_v3_module):
+        """A real queue whose worker is held by another plugin's operation."""
+        queue = PluginOperationQueue(max_history=10)
+        api_v3_module.api_v3.operation_queue = queue
+        started, release = threading.Event(), threading.Event()
+
+        def blocker(op):
+            started.set()
+            release.wait(10)
+            return {"success": True, "message": "done"}
+
+        queue.enqueue_operation(OperationType.INSTALL, "busy", operation_callback=blocker)
+        assert started.wait(5)
+        yield queue
+        release.set()
+        queue.shutdown()
+
+    def test_a_pending_operation_reports_pending(self, api_v3_client, busy_queue):
+        op_id = busy_queue.enqueue_operation(
+            OperationType.INSTALL, "demo", operation_callback=_callback)
+        response = api_v3_client.get(f"/api/v3/plugins/operation/{op_id}")
+        assert response.status_code == 200, response.get_json()
+        data = response.get_json()["data"]
+        assert data["status"] == "pending"
+        assert data["plugin_id"] == "demo"
+        assert "_callback" not in data["parameters"]

test/test_api_v3_partial_main_save.py

@@ -253,6 +253,32 @@ class TestVegasCycleDurations:
         assert saved['config']['display']['display_durations'] == {'clock': 45}
 
 
+class TestMalformedBody:
+    """A JSON body that does not parse is the caller's mistake: a 400.
+
+    get_json() raised Werkzeug's BadRequest inside the handler's try, whose
+    catch-all answered 500 CONFIG_SAVE_FAILED with "check file permissions"
+    advice and logged a traceback at ERROR.
+    """
+
+    def test_is_a_400_in_the_raw_routes_shape(self, api_v3_client, saved, api_v3_module):
+        api_v3_module.api_v3.config_manager.get_raw_file_content.return_value = {}
+        resp = api_v3_client.post('/api/v3/config/main', data='{not json',
+                                  content_type='application/json')
+        assert resp.status_code == 400
+        assert resp.get_json() == {'status': 'error', 'message': 'Invalid JSON in request body'}
+        assert 'config' not in saved
+        raw = api_v3_client.post('/api/v3/config/raw/main', data='{not json',
+                                 content_type='application/json')
+        assert (raw.status_code, raw.get_json()) == (400, resp.get_json())
+
+    def test_an_empty_json_post_is_still_no_data(self, api_v3_client, saved):
+        resp = api_v3_client.post('/api/v3/config/main', data='',
+                                  content_type='application/json')
+        assert resp.status_code == 400
+        assert resp.get_json()['message'] == 'No data provided'
+
+
 class TestRawSaveStartsAutoUpdateSetup:
     @pytest.fixture
     def raw_env(self, api_v3_module, monkeypatch):

test/test_api_v3_plugin_action_params.py

@@ -0,0 +1,93 @@
+"""POST /api/v3/plugins/action hands ``params`` to the plugin's script intact.
+
+The route runs the script through a generated wrapper, and the params went
+into that wrapper as Python source: ``params = {json.dumps(params)}``. JSON is
+not Python. ``true``, ``false`` and ``null`` are undefined names there, so any
+params holding a boolean or a null died with a NameError before the script
+ran. The plugin file manager's category toggle sends ``{"category_name": ...,
+"enabled": true}``, so of-the-day's category toggle failed every time with
+"Action failed".
+
+The script's side of the contract is unchanged and pinned here too: the
+params arrive on stdin as one JSON document, LEDMATRIX_ROOT is set, and what
+the script prints to stdout is what the route parses.
+"""
+
+import json
+import subprocess
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
+
+ACTION_URL = "/api/v3/plugins/action"
+
+# The action script: report what it was handed, as JSON on stdout.
+ECHO_SCRIPT = (
+    "import json, os, sys\n"
+    "raw = sys.stdin.read()\n"
+    "print(json.dumps({'status': 'success', 'got': json.loads(raw),\n"
+    "                  'root': os.environ.get('LEDMATRIX_ROOT')}))\n"
+)
+
+
+@pytest.fixture
+def echo_plugin(tmp_path, api_v3_module, monkeypatch):
+    plugin_dir = tmp_path / "demo"
+    plugin_dir.mkdir()
+    (plugin_dir / "manifest.json").write_text(json.dumps({
+        "id": "demo",
+        "web_ui_actions": [{"id": "toggle", "type": "script", "script": "echo.py"}],
+    }), encoding="utf-8")
+    (plugin_dir / "echo.py").write_text(ECHO_SCRIPT, encoding="utf-8")
+    api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = str(plugin_dir)
+
+    # The route runs `python3`; use this interpreter, so the test does not
+    # depend on what that name resolves to here.
+    real_run = subprocess.run
+
+    def run(cmd, *args, **kwargs):
+        if isinstance(cmd, list) and cmd and cmd[0] == "python3":
+            cmd = [sys.executable] + cmd[1:]
+        return real_run(cmd, *args, **kwargs)
+
+    monkeypatch.setattr(subprocess, "run", run)
+    return plugin_dir
+
+
+@pytest.mark.parametrize("params", [
+    {"category_name": "jokes", "enabled": True},   # the file manager's toggle
+    {"category_name": "jokes", "enabled": False},
+    {"filename": None},
+    {"nested": {"list": [1, None, True, 2.5], "empty": {}}},
+    {"text": "café ✓ \U0001F600"},
+    {"text": "he said \"hi\" and 'bye' \\ ''' \"\"\" \n\t end"},
+], ids=["true", "false", "null", "nested", "unicode", "quotes"])
+def test_the_script_receives_the_params_it_was_sent(api_v3_client, echo_plugin, params):
+    response = api_v3_client.post(ACTION_URL, json={
+        "plugin_id": "demo", "action_id": "toggle", "params": params})
+    body = response.get_json()
+    assert response.status_code == 200, body
+    assert body["got"] == params
+
+
+def test_a_param_cannot_run_code_in_the_wrapper(api_v3_client, echo_plugin, tmp_path):
+    marker = tmp_path / "PWNED"
+    hostile = "\"}\nopen(%r, 'w').write('ran')\n#" % str(marker)
+    params = {"name": hostile, "flag": True}
+    response = api_v3_client.post(ACTION_URL, json={
+        "plugin_id": "demo", "action_id": "toggle", "params": params})
+    assert response.status_code == 200, response.get_json()
+    assert response.get_json()["got"] == params
+    assert not marker.exists(), "a param value ran as code"
+
+
+def test_the_script_still_gets_ledmatrix_root(api_v3_client, echo_plugin, api_v3_module):
+    response = api_v3_client.post(ACTION_URL, json={
+        "plugin_id": "demo", "action_id": "toggle", "params": {"enabled": True}})
+    assert response.status_code == 200, response.get_json()
+    assert response.get_json()["root"] == str(api_v3_module.PROJECT_ROOT)

test/test_api_v3_plugin_operation_conflict.py

@@ -0,0 +1,89 @@
+"""A second install or uninstall while one is in progress is a 409, not a 500.
+
+PluginOperationQueue refuses a second operation for a plugin that already
+has one waiting or running (test_operation_queue_pending_and_trim.py), and
+says so by raising ValueError. /plugins/install let that escape to the
+blueprint's catch-all, so a double-clicked Install answered 500 "An error
+occurred; see logs for details" while the first install carried on.
+/plugins/uninstall caught it in its own catch-all: a 500 "Failed to
+uninstall plugin", and an "uninstall failed" entry in the operation
+history for an uninstall that never started.
+"""
+
+import sys
+import threading
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
+
+from src.plugin_system.operation_queue import PluginOperationQueue  # noqa: E402
+
+INSTALL = "/api/v3/plugins/install"
+UNINSTALL = "/api/v3/plugins/uninstall"
+
+
+@pytest.fixture
+def installing(api_v3_module, tmp_path):
+    """A real queue with an install of "clock" running and held there."""
+    queue = PluginOperationQueue(max_history=10)
+    api_v3_module.api_v3.operation_queue = queue
+    started, release = threading.Event(), threading.Event()
+
+    def slow_install(plugin_id, branch=None):
+        started.set()
+        release.wait(10)
+        return True
+
+    store = api_v3_module.api_v3.plugin_store_manager
+    store.install_plugin.side_effect = slow_install
+    store.get_registry_info.return_value = None
+    store.plugins_dir = str(tmp_path)
+    api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = None
+    yield {"queue": queue, "started": started, "store": store}
+    release.set()
+    queue.shutdown()
+
+
+def _start_first_install(client, installing):
+    response = client.post(INSTALL, json={"plugin_id": "clock"})
+    assert response.status_code == 200, response.get_json()
+    assert installing["started"].wait(5)
+
+
+def _failed_history(api_v3_module):
+    return [c for c in api_v3_module.api_v3.operation_history.record_operation.call_args_list
+            if c.kwargs.get("status") == "failed"]
+
+
+def test_a_second_install_click_is_a_conflict(api_v3_client, api_v3_module, installing):
+    _start_first_install(api_v3_client, installing)
+    response = api_v3_client.post(INSTALL, json={"plugin_id": "clock"})
+    assert response.status_code == 409, response.get_json()
+    body = response.get_json()
+    assert body["status"] == "error"
+    assert body["error_code"] == "PLUGIN_OPERATION_CONFLICT"
+    assert "clock" in body["message"]
+    assert installing["store"].install_plugin.call_count == 1
+    assert _failed_history(api_v3_module) == []
+
+
+def test_an_uninstall_during_the_install_is_a_conflict(api_v3_client, api_v3_module,
+                                                        installing):
+    _start_first_install(api_v3_client, installing)
+    response = api_v3_client.post(UNINSTALL, json={"plugin_id": "clock"})
+    assert response.status_code == 409, response.get_json()
+    assert response.get_json()["error_code"] == "PLUGIN_OPERATION_CONFLICT"
+    assert _failed_history(api_v3_module) == [], (
+        "an uninstall that never started was recorded as failed")
+    api_v3_module.api_v3.plugin_store_manager.uninstall_plugin.assert_not_called()
+
+
+def test_another_plugin_is_still_queued(api_v3_client, installing):
+    _start_first_install(api_v3_client, installing)
+    response = api_v3_client.post(INSTALL, json={"plugin_id": "weather"})
+    assert response.status_code == 200, response.get_json()
+    assert response.get_json()["data"]["operation_id"]

test/test_api_v3_plugin_static_files.py

@@ -0,0 +1,73 @@
+"""GET /api/v3/plugins/<plugin_id>/static/<path> serves binary files too.
+
+The route opened every file as UTF-8 text, so an image -- what the API
+reference says it is for, plugin previews and icons -- failed to decode and
+answered 500 "UnicodeDecodeError". Files are now sent as bytes. The text
+types the route always set are unchanged, and the path checks are pinned in
+test_path_traversal_guards.py::TestServePluginStatic.
+"""
+
+import json
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
+
+PNG = (b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01"
+       b"\x08\x06\x00\x00\x00\x1f\x15\xc4\x89")
+
+
+@pytest.fixture
+def plugin_dir(tmp_path, api_v3_module):
+    d = tmp_path / "demo"
+    (d / "web_ui").mkdir(parents=True)
+    (d / "manifest.json").write_text(json.dumps({"id": "demo"}), encoding="utf-8")
+    api_v3_module.api_v3.plugin_catalog.get_plugin_directory.side_effect = (
+        lambda pid: str(d) if pid == "demo" else None)
+    return d
+
+
+def _get(client, path):
+    return client.get(f"/api/v3/plugins/demo/static/{path}")
+
+
+def test_an_image_is_served_as_its_bytes(api_v3_client, plugin_dir):
+    (plugin_dir / "web_ui" / "icon.png").write_bytes(PNG)
+    response = _get(api_v3_client, "web_ui/icon.png")
+    assert response.status_code == 200, response.get_json(silent=True)
+    assert response.mimetype == "image/png"
+    assert response.data == PNG
+
+
+def test_an_unknown_binary_file_is_served_too(api_v3_client, plugin_dir):
+    blob = bytes(range(256))
+    (plugin_dir / "data.bin").write_bytes(blob)
+    response = _get(api_v3_client, "data.bin")
+    assert response.status_code == 200, response.get_json(silent=True)
+    assert response.data == blob
+
+
+@pytest.mark.parametrize("name,mimetype", [
+    ("page.html", "text/html"),
+    ("app.js", "application/javascript"),
+    ("style.css", "text/css"),
+    ("data.json", "application/json"),
+    ("notes.txt", "text/plain"),
+    ("README.md", "text/plain"),
+    ("helper.py", "text/plain"),
+])
+def test_text_files_keep_their_types(api_v3_client, plugin_dir, name, mimetype):
+    content = "caf\u00e9 \u2713 <p>hi</p>\n"
+    (plugin_dir / name).write_bytes(content.encode("utf-8"))
+    response = _get(api_v3_client, name)
+    assert response.status_code == 200
+    assert response.mimetype == mimetype
+    assert response.data == content.encode("utf-8")
+
+
+def test_a_missing_file_is_still_a_404(api_v3_client, plugin_dir):
+    assert _get(api_v3_client, "nope.png").status_code == 404

test/test_api_v3_status_accuracy.py

@@ -0,0 +1,326 @@
+"""Four web answers that disagreed with the rig they describe (found on ledpi).
+
+1. POST /config/schedule refused the schedule GET returns on a fresh install
+   (config.template.json: per-day, every day off, schedule disabled) with
+   "At least one day must be enabled", as did /config/dim-schedule. A
+   disabled schedule needs no enabled day.
+2. A brightness-only POST /config/main answered ``restart_required: true``,
+   though the display applies brightness live (brightness.set over the
+   socket, and the config watcher). The flag now says whether anything
+   changed that the running display does not pick up by itself.
+3. /health stayed "healthy" with the display service stopped: only the
+   sub-checks changed. Service inactive, no socket and no live heartbeat
+   is now ``display_loop: stopped`` and "degraded".
+4. /display/current-status kept answering ``is_display_active: true`` from
+   the cache for up to 120 s after the display stopped. With no socket and
+   no live heartbeat it is now unknown.
+"""
+
+import copy
+import json
+import os
+import sys
+import time
+from pathlib import Path
+from unittest.mock import patch
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
+
+from src import display_watchdog  # noqa: E402
+from src.ipc import client as control_client  # noqa: E402
+from web_interface import display_state  # noqa: E402
+
+REPO = Path(__file__).resolve().parent.parent
+TEMPLATE = json.loads((REPO / 'config' / 'config.template.json').read_text(encoding='utf-8'))
+
+
+@pytest.fixture
+def store(api_v3_module, monkeypatch):
+    state = {'config': {}, 'saves': 0}
+    api_v3_module.api_v3.config_manager.load_config.side_effect = \
+        lambda *a, **k: copy.deepcopy(state['config'])
+
+    def fake_save(_manager, config, **_kwargs):
+        state['config'] = copy.deepcopy(config)
+        state['saves'] += 1
+        return True, ''
+
+    monkeypatch.setattr(api_v3_module, '_save_config_atomic', fake_save)
+    return state
+
+
+# --- 1. schedules ---------------------------------------------------------------
+
+SCHEDULE_ROUTES = [('/api/v3/config/schedule', 'schedule'),
+                   ('/api/v3/config/dim-schedule', 'dim_schedule')]
+
+
+@pytest.mark.parametrize('route,section', SCHEDULE_ROUTES)
+def test_the_templates_disabled_per_day_schedule_saves_back(api_v3_client, store,
+                                                            route, section):
+    stored = copy.deepcopy(TEMPLATE[section])
+    stored['mode'] = 'per-day'
+    assert stored['enabled'] is False
+    assert not any(day['enabled'] for day in stored['days'].values())
+    store['config'] = {section: copy.deepcopy(stored)}
+
+    read = api_v3_client.get(route).get_json()['data']
+    resp = api_v3_client.post(route, json=read)
+
+    assert resp.status_code == 200, resp.get_json()
+    saved = store['config'][section]
+    assert saved['enabled'] is False and saved['mode'] == 'per-day'
+    # The disabled days keep their times: switching one on finds them.
+    assert saved['days'] == stored['days']
+
+
+@pytest.mark.parametrize('route,section', SCHEDULE_ROUTES)
+def test_an_enabled_per_day_schedule_still_needs_a_day(api_v3_client, store, route, section):
+    body = copy.deepcopy(TEMPLATE[section])
+    body.update(enabled=True, mode='per-day')
+    resp = api_v3_client.post(route, json=body)
+    assert resp.status_code == 400
+    assert 'At least one day must be enabled' in resp.get_json()['message']
+    assert store['saves'] == 0
+
+
+@pytest.mark.parametrize('route', [r for r, _ in SCHEDULE_ROUTES])
+def test_the_pickers_form_post_with_every_day_off_saves(api_v3_client, store, route):
+    """What schedule-picker.js posts: flat hidden inputs, booleans as strings,
+    times for every day."""
+    body = {'enabled': 'false', 'mode': 'per_day', 'start_time': '07:00', 'end_time': '23:00'}
+    for day in ('monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday'):
+        body.update({f'{day}_enabled': 'false', f'{day}_start': '06:30', f'{day}_end': '22:15'})
+    resp = api_v3_client.post(route, json=body)
+    assert resp.status_code == 200, resp.get_json()
+
+
+def test_an_invalid_time_on_a_disabled_day_is_dropped_not_refused(api_v3_client, store):
+    body = {'enabled': False, 'mode': 'per-day',
+            'days': {'monday': {'enabled': False, 'start_time': 'soon', 'end_time': '22:00'}}}
+    resp = api_v3_client.post('/api/v3/config/schedule', json=body)
+    assert resp.status_code == 200, resp.get_json()
+    assert store['config']['schedule']['days']['monday'] == {'enabled': False,
+                                                             'end_time': '22:00'}
+
+
+# --- 2. restart_required on /config/main ------------------------------------------
+
+STORED_MAIN = {
+    'timezone': 'America/Chicago',
+    'display': {
+        'hardware': {'rows': 32, 'cols': 64, 'chain_length': 2, 'brightness': 90,
+                     'disable_hardware_pulsing': False, 'inverse_colors': False,
+                     'show_refresh_rate': False},
+        'runtime': {'gpio_slowdown': 4},
+        'display_durations': {'clock': 15},
+        'use_short_date_format': False,
+    },
+}
+
+
+@pytest.fixture
+def main_store(store):
+    store['config'] = copy.deepcopy(STORED_MAIN)
+    return store
+
+
+def _save_main(client, body):
+    with patch('web_interface.blueprints.api_v3.control_client.brightness_set',
+               side_effect=control_client.ControlError('no_socket', 'x')):
+        resp = client.post('/api/v3/config/main', data=json.dumps(body),
+                           content_type='application/json')
+    assert resp.status_code == 200, resp.get_json()
+    return resp.get_json()
+
+
+def test_a_brightness_only_save_needs_no_restart(api_v3_client, main_store):
+    body = _save_main(api_v3_client, {'brightness': 40})
+    assert main_store['config']['display']['hardware']['brightness'] == 40
+    assert body['restart_required'] is False
+
+
+def test_a_brightness_save_on_a_config_without_a_display_section(api_v3_client, store):
+    """The route creates display.hardware and display.runtime on the way;
+    empty sections are not a change."""
+    store['config'] = {}
+    assert _save_main(api_v3_client, {'brightness': 40})['restart_required'] is False
+
+
+def test_the_display_form_with_only_brightness_changed_needs_no_restart(api_v3_client,
+                                                                       main_store):
+    hw = STORED_MAIN['display']['hardware']
+    body = {'__form_section': 'display', 'rows': 32, 'cols': 64, 'chain_length': 2,
+            'brightness': 55, 'gpio_slowdown': 4}
+    body.update({k: 'on' for k in ('disable_hardware_pulsing', 'inverse_colors',
+                                   'show_refresh_rate') if hw[k]})
+    assert _save_main(api_v3_client, body)['restart_required'] is False
+
+
+def test_a_mode_duration_needs_no_restart(api_v3_client, main_store):
+    body = _save_main(api_v3_client, {'duration__clock': 40})
+    assert main_store['config']['display']['display_durations']['clock'] == 40
+    assert body['restart_required'] is False
+
+
+@pytest.mark.parametrize('change', [{'rows': 64}, {'brightness': 40, 'chain_length': 3},
+                                    {'gpio_slowdown': 2}, {'timezone': 'UTC'}])
+def test_a_setting_the_display_reads_at_startup_still_needs_one(api_v3_client, main_store,
+                                                                change):
+    assert _save_main(api_v3_client, change)['restart_required'] is True
+
+
+def test_restart_needed_compares_leaves():
+    from web_interface.blueprints.api_v3.config import restart_needed
+    before = {'display': {'hardware': {'brightness': 90, 'rows': 32}}}
+    assert not restart_needed(before, copy.deepcopy(before))
+    assert not restart_needed(before, {'display': {'hardware': {'brightness': 10, 'rows': 32},
+                                                   'runtime': {}}})
+    assert restart_needed(before, {'display': {'hardware': {'brightness': 90}}})  # removed
+    assert not restart_needed({}, {'clock': {'enabled': True}}, live_paths=[('clock',)])
+    assert restart_needed({}, {'clockwork': {'enabled': True}}, live_paths=[('clock',)])
+
+
+# --- 3 and 4. a stopped display -----------------------------------------------------
+
+@pytest.fixture
+def no_display(monkeypatch, tmp_path):
+    """A Pi whose display service has stopped: the socket is expected here
+    but does not answer, and systemd took the heartbeat's directory away."""
+    monkeypatch.setattr(display_state, 'socket_supported', lambda: True)
+    monkeypatch.setattr(display_state, 'client_socket_paths', lambda: [str(tmp_path / 'gone')])
+    monkeypatch.setattr(display_state, 'read_state', lambda: None)
+    path = tmp_path / 'display-heartbeat.json'
+    monkeypatch.setattr(display_watchdog, 'HEARTBEAT_PATH', str(path))
+
+    def beat(age, pid=None):
+        path.write_text(json.dumps({'pid': os.getpid() if pid is None else pid,
+                                    'mono': time.monotonic() - age,
+                                    'wall': time.time() - age}))
+    return beat
+
+
+@pytest.fixture
+def service(monkeypatch):
+    status = {'active': False, 'returncode': 3, 'stdout': 'inactive', 'stderr': ''}
+    monkeypatch.setattr('web_interface.blueprints.api_v3.misc._get_display_service_status',
+                        lambda: dict(status))
+    return status
+
+
+@pytest.fixture
+def fresh_preview(tmp_path, monkeypatch):
+    """The preview frame the display left behind, under 60 s old: on its own
+    it kept the hardware check "connected"."""
+    from web_interface import display_preview
+    snapshot = tmp_path / 'preview.png'
+    snapshot.write_bytes(b'png')
+    monkeypatch.setattr(display_preview, 'SNAPSHOT_PATH', str(snapshot))
+
+
+def _health(client):
+    resp = client.get('/api/v3/health')
+    assert resp.status_code == 200, resp.get_json()
+    return resp.get_json()['data']
+
+
+class TestHealth:
+    def test_a_stopped_display_service_is_degraded(self, api_v3_client, no_display, service,
+                                                   fresh_preview):
+        data = _health(api_v3_client)
+        assert data['services']['display_service']['status'] == 'inactive'
+        assert data['checks']['display_loop']['status'] == 'stopped'
+        assert data['status'] == 'degraded'
+
+    def test_a_service_still_starting_is_not(self, api_v3_client, no_display, service,
+                                             fresh_preview):
+        """Active, before its socket and first heartbeat: not stopped."""
+        service.update(active=True, stdout='active', returncode=0)
+        data = _health(api_v3_client)
+        assert data['checks']['display_loop']['status'] == 'not_reported'
+        assert data['status'] == 'healthy'
+
+    def test_a_display_run_by_hand_is_not_stopped(self, api_v3_client, no_display, service,
+                                                  fresh_preview):
+        """The service is off but a display process beats (sudo python3 run.py)."""
+        no_display(age=2)
+        data = _health(api_v3_client)
+        assert data['checks']['display_loop']['status'] == 'running'
+        assert data['status'] == 'healthy'
+
+    @pytest.mark.parametrize('platform', ['no_unix_sockets', 'socket_off'])
+    def test_without_a_socket_to_expect_nothing_changes(self, api_v3_client, no_display,
+                                                        service, fresh_preview, monkeypatch,
+                                                        platform):
+        """Windows and the dev server (no systemd unit), or the socket
+        deliberately off: no heartbeat is no signal, as before."""
+        if platform == 'no_unix_sockets':
+            monkeypatch.setattr(display_state, 'socket_supported', lambda: False)
+        else:
+            monkeypatch.setattr(display_state, 'client_socket_paths', lambda: [])
+        service.update(returncode=-1, stdout='', stderr='systemctl not found')
+        data = _health(api_v3_client)
+        assert data['checks']['display_loop']['status'] == 'not_reported'
+        assert data['status'] == 'healthy'
+
+    def test_the_status_only_answer_says_degraded(self, api_v3_client, no_display, service,
+                                                  fresh_preview, monkeypatch):
+        monkeypatch.setattr('web_interface.blueprints.api_v3.misc.request_is_authenticated',
+                            lambda: False)
+        resp = api_v3_client.get('/api/v3/health')
+        assert resp.get_json()['data'] == {'status': 'degraded'}
+
+
+class TestCurrentStatus:
+    CACHED = {'mode': 'clock', 'plugin_id': 'clock', 'is_display_active': True,
+              'on_demand_active': False, 'last_updated': None}
+
+    @pytest.fixture
+    def cached(self, api_v3_module):
+        entry = dict(self.CACHED, last_updated=time.time() - 30)
+        cache = api_v3_module.api_v3.cache_manager
+        cache.get.side_effect = lambda key, *a, **kw: (
+            dict(entry) if key == 'display_current_state' else None)
+        return entry
+
+    def _status(self, client):
+        resp = client.get('/api/v3/display/current-status')
+        assert resp.status_code == 200
+        return resp.get_json()['data']
+
+    def test_a_stopped_display_is_not_reported_active(self, api_v3_client, no_display, cached):
+        data = self._status(api_v3_client)
+        assert not data.get('is_display_active')
+        assert data['mode'] is None and data['last_updated'] is None
+        assert data['source'] == 'cache'
+
+    def test_a_stale_heartbeat_is_not_active_either(self, api_v3_client, no_display, cached):
+        no_display(age=display_watchdog.HEARTBEAT_STALE_SECONDS + 5)
+        assert self._status(api_v3_client)['mode'] is None
+
+    @pytest.mark.skipif(os.name != 'posix', reason='process_exists answers only on POSIX')
+    def test_a_heartbeat_from_a_dead_process_is_not_active(self, api_v3_client, no_display,
+                                                           cached):
+        no_display(age=1, pid=2 ** 22 + 12345)
+        assert self._status(api_v3_client)['mode'] is None
+
+    def test_a_live_heartbeat_without_a_socket_reads_the_cache(self, api_v3_client,
+                                                               no_display, cached):
+        """An older display with no socket, still running."""
+        no_display(age=2)
+        data = self._status(api_v3_client)
+        assert data['mode'] == 'clock' and data['is_display_active'] is True
+
+    @pytest.mark.parametrize('platform', ['no_unix_sockets', 'socket_off'])
+    def test_without_a_socket_to_expect_the_cache_answers(self, api_v3_client, no_display,
+                                                          cached, monkeypatch, platform):
+        if platform == 'no_unix_sockets':
+            monkeypatch.setattr(display_state, 'socket_supported', lambda: False)
+        else:
+            monkeypatch.setattr(display_state, 'client_socket_paths', lambda: [])
+        data = self._status(api_v3_client)
+        assert data['mode'] == 'clock' and data['is_display_active'] is True

test/test_config_load_cache.py

@@ -58,7 +58,8 @@ class TestFastPath:
         for _ in range(10):
             again = m.load_config()
         assert counts["n"] == 0, "fast path must not re-open any config file"
-        assert again is first  # same aliasing semantics as the full path
+        assert again == first
+        assert again is not first  # each caller gets its own copy, see below
 
     def test_config_change_triggers_reload(self, mgr):
         m, config, secrets, template = mgr
@@ -98,6 +99,33 @@ class TestFastPath:
         assert m.load_config()["timezone"] == "America/New_York"
 
 
+class TestCallersGetACopy:
+    """A web handler edits what load_config returned, then validates. When
+    validation failed, the edit stayed in the cache the fast path serves, and
+    the next unrelated save wrote it -- a nested secret included, in plain
+    text, because it had never reached config_secrets.json to be stripped."""
+
+    def test_editing_a_loaded_config_does_not_change_the_next_load(self, mgr):
+        m, config, secrets, template = mgr
+        loaded = m.load_config()
+        loaded["display"]["brightness"] = 1
+        loaded["weather"]["api_key"] = "typed-but-never-saved"
+        again = m.load_config()
+        assert again["display"]["brightness"] == 90
+        assert again["weather"]["api_key"] == "sek"
+
+    def test_the_full_path_also_returns_a_copy(self, mgr):
+        m, config, secrets, template = mgr
+        m.load_config()["display"]["brightness"] = 1  # first load: full path
+        assert m.load_config()["display"]["brightness"] == 90
+
+    def test_an_edit_never_reaches_a_later_save(self, mgr):
+        m, config, secrets, template = mgr
+        m.load_config()["display"]["new_secret"] = "hunter2"  # then bailed out
+        m.save_config(m.load_config())  # some other handler saves
+        assert "hunter2" not in config.read_text()
+
+
 class TestSaveCoherence:
     def test_save_config_then_load_returns_saved_data(self, mgr, monkeypatch):
         m, config, secrets, template = mgr
@@ -111,6 +139,15 @@ class TestSaveCoherence:
         assert loaded["weather"]["api_key"] == "sek"  # secrets survive in memory
         assert counts["n"] == 0  # signature refreshed by save; no re-read
 
+    def test_the_saved_dict_does_not_become_the_cache(self, mgr):
+        m, config, secrets, template = mgr
+        m.load_config()
+        new = {"display": {"brightness": 42}, "timezone": "UTC",
+               "weather": {"api_key": "sek"}}
+        m.save_config(new)
+        new["display"]["brightness"] = 7  # the caller keeps using its dict
+        assert m.load_config()["display"]["brightness"] == 42
+
     def test_cross_process_save_is_picked_up(self, mgr):
         """Another process writing config.json (different mtime) must bust
         this process's fast path — the core cross-process guarantee."""

test/test_config_manager_secrets.py

@@ -143,7 +143,10 @@ class TestLoadFastPath:
         manager = make_manager(tmp_path, config={"timezone": "UTC"})
         first = manager.load_config()
         second = manager.load_config()
-        assert second is first  # same aliased dict, no re-read
+        # A copy of the cached dict, never the dict itself; that it is not
+        # re-read is test_config_load_cache's test_unchanged_files_are_not_reread
+        assert second == first
+        assert second is not first
 
     def test_touching_secrets_file_invalidates_cache(self, tmp_path):
         manager = make_manager(

test/test_espn_dates.py

@@ -489,3 +489,171 @@ class TestConcurrency:
 
         assert live["peak"] <= espn_dates.ESPN_CHUNK_WORKERS
         assert live["peak"] > 1, "chunks should actually overlap"
+
+
+class TestEdgeMonths:
+    """A window's partial edge months are asked whole and trimmed.
+
+    The default scoreboard window -- a fortnight either side of today -- spans
+    two partial months, so it used to cost 29 day requests per league. ESPN's
+    ``dates=YYYYMMDD`` means a US Eastern day (verified against the live API
+    on 2026-10-03, 417 of 417 soccer events), so a month answer trimmed to
+    the window's Eastern days is what the day requests returned.
+    """
+
+    def test_a_fortnight_either_side_is_two_requests(self):
+        planned = espn_dates.espn_request_chunks(date(2026, 9, 20), date(2026, 10, 18))
+        assert planned == [
+            ("202609", (date(2026, 9, 20), date(2026, 9, 30))),
+            ("202610", (date(2026, 10, 1), date(2026, 10, 18))),
+        ]
+
+    def test_a_live_polls_two_days_stay_two_days(self):
+        planned = espn_dates.espn_request_chunks(date(2026, 10, 2), date(2026, 10, 3))
+        assert planned == [("20261002", None), ("20261003", None)]
+
+    def test_the_threshold_is_inclusive(self):
+        n = espn_dates.ESPN_MONTH_COVER_MIN_DAYS
+        short = espn_dates.espn_request_chunks(date(2026, 10, 1), date(2026, 10, n - 1))
+        assert [chunk for chunk, _ in short] == [
+            "202610%02d" % day for day in range(1, n)]
+        enough = espn_dates.espn_request_chunks(date(2026, 10, 1), date(2026, 10, n))
+        assert enough == [("202610", (date(2026, 10, 1), date(2026, 10, n)))]
+
+    def test_whole_months_and_short_edges_are_unchanged(self):
+        planned = espn_dates.espn_request_chunks(date(2026, 8, 30), date(2026, 10, 2))
+        assert planned == [
+            ("20260830", None), ("20260831", None), ("202609", None),
+            ("20261001", None), ("20261002", None),
+        ]
+
+    def test_without_time_zone_data_edges_stay_days(self, monkeypatch):
+        monkeypatch.setattr(espn_dates, "_EASTERN", None)
+        planned = espn_dates.espn_request_chunks(date(2026, 9, 20), date(2026, 10, 18))
+        assert len(planned) == 29
+        assert all(trim is None for _, trim in planned)
+
+    @pytest.mark.parametrize("start,end", [
+        (date(2026, 9, 20), date(2026, 10, 18)),
+        (date(2026, 1, 25), date(2026, 3, 3)),
+        (date(2026, 12, 20), date(2027, 1, 9)),
+        (date(2026, 10, 5), date(2026, 10, 9)),
+    ])
+    def test_the_planned_requests_still_cover_every_day_exactly_once(self, start, end):
+        covered = []
+        for chunk, trim in espn_dates.espn_request_chunks(start, end):
+            if trim is None:
+                covered.extend(days_covered_by([chunk]))
+            else:
+                assert chunk == trim[0].strftime("%Y%m") == trim[1].strftime("%Y%m")
+                covered.extend(trim[0] + timedelta(days=offset)
+                               for offset in range((trim[1] - trim[0]).days + 1))
+        expected = [start + timedelta(days=offset) for offset in range((end - start).days + 1)]
+        assert covered == expected
+
+    def test_a_trimmed_month_keeps_only_the_windows_eastern_days(self):
+        september = [
+            # 03:30Z on the 20th is still the 19th in New York: outside.
+            {"id": "before", "date": "2026-09-20T03:30Z"},
+            {"id": "first", "date": "2026-09-20T14:00Z"},
+            {"id": "late", "date": "2026-09-30T23:30Z"},
+        ]
+        october = [
+            {"id": "oct1", "date": "2026-10-01T19:00Z"},
+            # 03:30Z on the 19th is the evening of the 18th in New York: inside.
+            {"id": "last", "date": "2026-10-19T03:30Z"},
+            {"id": "after", "date": "2026-10-19T14:00Z"},
+            {"id": "undated"},
+        ]
+        session = FakeSession({"202609": september, "202610": october})
+
+        data = fetch_espn_date_chunks(session, URL, params={"dates": "20260920-20261018"})
+
+        assert sorted(call["dates"] for call in session.calls) == ["202609", "202610"]
+        # An event with no readable date is kept, never dropped on a guess.
+        assert [e["id"] for e in data["events"]] == ["first", "late", "oct1", "last", "undated"]
+
+    def test_eastern_standard_time_is_honoured_after_the_clocks_change(self):
+        # 2026-11-01 ends daylight saving: Eastern is UTC-5 from then on.
+        november = [
+            {"id": "out", "date": "2026-11-15T04:30Z"},  # Nov 14, 23:30 EST
+            {"id": "in", "date": "2026-11-15T05:30Z"},   # Nov 15, 00:30 EST
+        ]
+        session = FakeSession({"202611": november})
+        data = fetch_espn_date_chunks(session, URL, params={"dates": "20261115-20261121"})
+        assert [e["id"] for e in data["events"]] == ["in"]
+
+    def test_a_capped_edge_month_re_asks_only_the_windows_days(self):
+        full = [{"id": "cap%d" % i, "date": "2026-10-05T18:00Z"} for i in range(ESPN_MAX_LIMIT)]
+        by_chunk = {"202610": full}
+        by_chunk.update({"202610%02d" % day: [{"id": "o%02d" % day}] for day in range(1, 32)})
+        session = FakeSession(by_chunk)
+
+        data = fetch_espn_date_chunks(session, URL, params={"dates": "20261001-20261010"})
+
+        sent = [call["dates"] for call in session.calls]
+        assert sent[0] == "202610"
+        assert sorted(sent[1:]) == ["202610%02d" % day for day in range(1, 11)]
+        assert [e["id"] for e in data["events"]] == ["o%02d" % day for day in range(1, 11)]
+
+
+class TestProcessWideChunkCap:
+    """The chunk cap holds across windows, not per window.
+
+    A soccer board starting eight leagues fetches sixteen windows at once.
+    With a pool of ``ESPN_CHUNK_WORKERS`` each, ~40 requests were in flight
+    and every one past a session's pool opened a connection -- and a DNS
+    lookup. On ledpi that was ~90 NameResolutionErrors per start.
+    """
+
+    def test_concurrent_windows_share_one_budget(self):
+        live = {"now": 0, "peak": 0}
+        guard = threading.Lock()
+
+        class CountingSession(FakeSession):
+            def get(self, url, params=None, headers=None, timeout=None):
+                with guard:
+                    live["now"] += 1
+                    live["peak"] = max(live["peak"], live["now"])
+                try:
+                    time.sleep(0.01)
+                    return super().get(url, params=params, headers=headers, timeout=timeout)
+                finally:
+                    with guard:
+                        live["now"] -= 1
+
+        sessions = [CountingSession() for _ in range(6)]
+        # Six leagues, so the fetch service cannot merge them into one, on a
+        # host with no token bucket: earlier tests may have spent ESPN's
+        # burst, and a bucket paced at 20/s would serialise these by itself.
+        threads = [
+            threading.Thread(target=fetch_espn_date_chunks,
+                             args=(session, "https://scores.example.test/league%d" % index),
+                             kwargs={"params": {"dates": "20260101-20261231"}})
+            for index, session in enumerate(sessions)
+        ]
+        for thread in threads:
+            thread.start()
+        for thread in threads:
+            thread.join(timeout=30)
+
+        assert all(len(session.calls) == 12 for session in sessions)
+        assert live["peak"] <= espn_dates.ESPN_CHUNK_WORKERS
+        assert live["peak"] > 1, "chunks should still overlap"
+
+
+def test_a_fresh_process_skips_the_doomed_range_request():
+    """Every start used to spend one 400 per window learning that ranges are
+    still rejected -- eleven at once from a soccer board. A new process now
+    starts inside the retry period instead."""
+    import subprocess
+    import sys
+    from pathlib import Path
+
+    out = subprocess.run(
+        [sys.executable, "-c",
+         "import src.common.espn_dates as e; print(e._ranges_known_rejected())"],
+        cwd=str(Path(__file__).resolve().parents[1]),
+        capture_output=True, text=True, timeout=60,
+    )
+    assert out.stdout.strip() == "True", out.stderr

test/test_fetch_service.py

@@ -670,14 +670,14 @@ class TestCallerIdentity:
         assert _counters(global_service, plugin="football-scoreboard")["requests"] == 1
 
     def test_espn_chunks_on_worker_threads_count_against_the_caller(self, global_service):
-        from src.common.espn_dates import espn_date_chunks, fetch_espn_date_chunks, parse_espn_date_range
+        from src.common.espn_dates import espn_request_chunks, fetch_espn_date_chunks, parse_espn_date_range
 
         session = FakeSession(lambda url, kw: make_response(body=b'{"events": []}', url=url))
         dates = "20260801-20261015"
         with plugin_scope("baseball-scoreboard"):
             fetch_espn_date_chunks(session, "https://site.api.espn.com/s/scoreboard",
                                    params={"dates": dates})
-        chunks = len(espn_date_chunks(*parse_espn_date_range(dates)))
+        chunks = len(espn_request_chunks(*parse_espn_date_range(dates)))
         assert chunks > 1
         assert len(session.calls) == chunks
         assert _counters(global_service, plugin="baseball-scoreboard")["requests"] == chunks

test/test_frame_timing.py

@@ -807,3 +807,54 @@ def test_a_process_with_the_gc_monitor_exits_cleanly():
     assert proc.returncode == 0, proc.stderr
     assert "Exception ignored" not in proc.stderr
     assert "installed at exit: False" in proc.stdout
+
+
+SLOW = 1 / 110.0  # a panel that cannot reach a 120 Hz cap
+
+
+def _windows(recorder, n, interval, start=0.0):
+    for i in range(n):
+        _feed(recorder, [interval] * 200, start=start + 50.0 * i)
+        _aggregate(recorder)
+
+
+def _shortfall_warnings(caplog):
+    return [r for r in caplog.records
+            if r.name == "src.common.frame_timing" and "Limit Refresh Rate" in r.getMessage()]
+
+
+def test_a_panel_slower_than_its_cap_is_reported_once(tmp_path, caplog):
+    r = _recorder(tmp_path)
+    r.plan_refresh(120.0)
+    caplog.set_level("WARNING")
+    _windows(r, 3, SLOW)          # adopted on the 2nd window, checked on the 4th
+    assert _shortfall_warnings(caplog) == []
+    _windows(r, 3, SLOW, start=1000.0)
+    warnings = _shortfall_warnings(caplog)
+    assert len(warnings) == 1
+    assert "about 110 Hz" in warnings[0].getMessage()
+    assert "to 100 Hz" in warnings[0].getMessage()
+
+
+def test_a_panel_that_reaches_its_cap_is_not_reported(tmp_path, caplog):
+    r = _recorder(tmp_path)
+    r.plan_refresh(100.0)
+    caplog.set_level("WARNING")
+    _windows(r, 6, PERIOD)
+    assert _shortfall_warnings(caplog) == []
+
+
+def test_without_a_planned_rate_nothing_is_checked(tmp_path, caplog):
+    # The emulator and the fallback canvas: DisplayManager never calls
+    # plan_refresh(), since their frames are not paced by a panel.
+    r = _recorder(tmp_path)
+    caplog.set_level("WARNING")
+    _windows(r, 6, SLOW)
+    assert _shortfall_warnings(caplog) == []
+
+
+def test_the_snapshot_records_the_planned_rate(tmp_path):
+    r = _recorder(tmp_path)
+    assert r.snapshot()["planned_refresh_hz"] is None
+    r.plan_refresh(120.0)
+    assert r.snapshot()["planned_refresh_hz"] == 120.0

test/test_mqtt_bridge_config_endpoint.py

@@ -136,3 +136,22 @@ class TestCleartextCredentialsNeedAnExplicitOptIn:
                              "allow_insecure_mqtt": "false"})
         assert r.status_code == 400
 
+    def test_the_settings_read_reports_the_opt_in(self, client, monkeypatch):
+        """The Tools form prefills its "Allow without TLS" box from the GET.
+
+        Off until someone saves it on, so an untouched form sends false and
+        the guard above still refuses a cleartext password.
+        """
+        c, _ = client
+        monkeypatch.setattr(misc, "_mqtt_bridge_service_state",
+                            lambda: {"installed": False, "active": False, "enabled": False})
+
+        def read():
+            return c.get("/api/v3/integrations/mqtt-bridge").get_json()["data"]["config"]
+
+        assert read()["allow_insecure_mqtt"] is False
+        r = c.put(URL, json={"mqtt_password": "hunter2", "mqtt_tls": False,
+                             "allow_insecure_mqtt": True})
+        assert r.status_code == 200, r.get_json()
+        assert read()["allow_insecure_mqtt"] is True
+

test/test_on_demand_live_and_restore.py

@@ -0,0 +1,162 @@
+"""Two on-demand edges seen on a rig.
+
+  * A request naming a ``*_live`` mode got HTTP 200 and a different mode on
+    the panel. The session's mode list kept live modes only when the plugin's
+    has_live_content() said so, and that is the live-priority question,
+    which the sports plugins answer for favourite teams only: fifteen college
+    games on, no favourite playing, and ``ncaa_fb_live`` became
+    ``nfl_recent``.
+  * A restart during a session whose plugin then failed to load (its config
+    no longer validated) logged "No valid display modes found ... after
+    restoration" and left the session active with no modes: published as
+    active for a plugin that was not running, with its cached request kept
+    for the next restart.
+"""
+
+from unittest.mock import MagicMock
+
+import pytest
+
+SPORTS_MODES = ['nfl_live', 'nfl_recent', 'nfl_upcoming',
+                'ncaa_fb_live', 'ncaa_fb_recent', 'ncaa_fb_upcoming']
+
+
+def _sports_plugin(has_live_content=False):
+    plugin = MagicMock(spec=['display', 'has_live_content', 'has_live_priority',
+                             'get_live_modes'])
+    plugin.has_live_content.return_value = has_live_content
+    plugin.has_live_priority.return_value = True
+    plugin.get_live_modes.return_value = []
+    return plugin
+
+
+def _register(controller, plugin_id, modes, plugin):
+    controller.plugin_display_modes[plugin_id] = list(modes)
+    for mode in modes:
+        controller.plugin_modes[mode] = plugin
+        controller.mode_to_plugin_id[mode] = plugin_id
+        if mode not in controller.available_modes:
+            controller.available_modes.append(mode)
+
+
+@pytest.fixture
+def football(test_display_controller):
+    c = test_display_controller
+    _register(c, 'football-scoreboard', SPORTS_MODES, _sports_plugin())
+    return c
+
+
+class TestANamedLiveModeIsShown:
+
+    def test_it_is_the_first_screen(self, football):
+        football._activate_on_demand({'plugin_id': 'football-scoreboard',
+                                      'mode': 'ncaa_fb_live'})
+        assert football.on_demand_active
+        assert football.current_display_mode == 'ncaa_fb_live'
+        assert football.on_demand_mode == 'ncaa_fb_live'
+
+    def test_the_plugins_other_modes_follow_it(self, football):
+        football._activate_on_demand({'plugin_id': 'football-scoreboard',
+                                      'mode': 'ncaa_fb_live'})
+        assert football.on_demand_modes[0] == 'ncaa_fb_live'
+        assert set(football.on_demand_modes[1:]) == {
+            'nfl_recent', 'nfl_upcoming', 'ncaa_fb_recent', 'ncaa_fb_upcoming'}
+
+    def test_pinned_holds_it(self, football):
+        football._activate_on_demand({'plugin_id': 'football-scoreboard',
+                                      'mode': 'ncaa_fb_live', 'pinned': True})
+        assert football.on_demand_modes == ['ncaa_fb_live']
+
+    def test_a_bare_plugin_request_still_skips_quiet_live_modes(self, football):
+        """Only a mode asked for by name is kept: a plugin-only request
+        resolves to the plugin's first mode (nfl_live), and opening on an
+        empty live screen there is what the ordering exists to avoid."""
+        football._activate_on_demand({'plugin_id': 'football-scoreboard'})
+        assert not any(m.endswith('_live') for m in football.on_demand_modes)
+
+    def test_a_named_second_live_mode_with_content_leads(self, test_display_controller):
+        """With live content both live modes are kept, nfl_live first; a
+        request naming ncaa_fb_live must still open on it, not rotate away."""
+        c = test_display_controller
+        _register(c, 'football-scoreboard', SPORTS_MODES, _sports_plugin(has_live_content=True))
+        c._activate_on_demand({'plugin_id': 'football-scoreboard', 'mode': 'ncaa_fb_live'})
+        assert c.on_demand_modes[0] == 'ncaa_fb_live'
+        assert c.on_demand_modes.count('ncaa_fb_live') == 1
+        assert 'nfl_live' in c.on_demand_modes[1:]
+
+    def test_the_named_mode_survives_a_restart(self, football):
+        football._activate_on_demand({'plugin_id': 'football-scoreboard',
+                                      'mode': 'ncaa_fb_live'})
+        saved = football.cache_manager.set.call_args_list[-1]
+        assert saved.args[0] == 'display_on_demand_config'
+        config = saved.args[1]
+        assert config['named_mode'] == 'ncaa_fb_live'
+
+        football._reset_on_demand_fields()
+        football._select_startup_plugins(['football-scoreboard'], config)
+        football._populate_on_demand_modes_from_plugin()
+        assert football.on_demand_modes[football.on_demand_mode_index] == 'ncaa_fb_live'
+
+
+class TestARestoreWithNothingToResume:
+
+    @pytest.fixture
+    def restored(self, test_display_controller):
+        c = test_display_controller
+        c.config['clock-simple'] = {'enabled': True}
+        c._select_startup_plugins(['clock-simple'],
+                                  {'plugin_id': 'clock-simple', 'mode': 'clock-simple'})
+        assert c.on_demand_active
+        # The plugin's load then fails: nothing is registered for it.
+        c.cache_manager.clear_cache.reset_mock()
+        c._populate_on_demand_modes_from_plugin()
+        return c
+
+    def test_the_session_ends(self, restored):
+        assert not restored.on_demand_active
+        assert restored.on_demand_plugin_id is None
+        assert not restored.on_demand_schedule_override
+
+    def test_it_is_reported_as_an_error(self, restored):
+        assert restored.on_demand_status == 'error'
+        assert restored.on_demand_last_error == 'restore-failed'
+        # The last on-demand state write, not the last write of any key: the
+        # font-usage publisher thread writes its own key at its own pace.
+        published = [c for c in restored.cache_manager.set.call_args_list
+                     if c.args and c.args[0] == 'display_on_demand_state'][-1]
+        assert published.args[1]['status'] == 'error'
+        assert published.args[1]['error'] == 'restore-failed'
+
+    def test_the_cached_request_is_dropped(self, restored):
+        restored.cache_manager.clear_cache.assert_any_call('display_on_demand_config')
+
+
+def test_a_plugin_system_failure_ends_a_cached_session_not_yet_restored(
+        mock_config_manager, mock_display_manager, mock_cache_manager,
+        test_config_with_plugins, emulator_mode):
+    """Initialization can fail before the cached session is read, with
+    on_demand_active still False: the session must still end, visibly."""
+    from unittest.mock import patch
+    from src.display_controller import DisplayController
+
+    mock_config_manager.get_config.return_value = test_config_with_plugins
+    mock_config_manager.load_config.return_value = test_config_with_plugins
+    mock_cache_manager._memory_cache['display_on_demand_config'] = {
+        'plugin_id': 'clock-simple', 'mode': 'clock-simple'}
+    with patch('src.display_controller.ConfigManager', return_value=mock_config_manager), \
+         patch('src.display_controller.DisplayManager', return_value=mock_display_manager), \
+         patch('src.display_controller.CacheManager', return_value=mock_cache_manager), \
+         patch('src.display_controller.FontManager'), \
+         patch('src.plugin_system.PluginManager', side_effect=RuntimeError("boom")):
+        controller = DisplayController()
+        try:
+            assert controller.plugin_manager is None
+            assert not controller.on_demand_active
+            assert controller.on_demand_status == 'error'
+            assert controller.on_demand_last_error == 'restore-failed'
+            mock_cache_manager.clear_cache.assert_any_call('display_on_demand_config')
+        finally:
+            try:
+                controller.cleanup()
+            except Exception:
+                pass

test/test_run_loop_golden.py

@@ -133,6 +133,32 @@ def scenario_on_demand_restored(h: RunLoopHarness):
     h.restore_on_demand("sports", mode="sports_upcoming", duration=40)
 
 
+def scenario_on_demand_named_live(h: RunLoopHarness):
+    # Games are on until t=70, but none involves a favourite, so
+    # has_live_content() (the live-priority answer) stays False throughout.
+    # A request naming sports_live still opens on it (it opened on
+    # sports_recent); asked for again after the games end, it has nothing to
+    # draw and the session moves on to the plugin's next mode.
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin(
+        "sports", ["sports_live", "sports_recent", "sports_upcoming"], duration=15,
+        live_priority=True,
+        content=lambda t, mode: mode != "sports_live" or t < 70))
+    h.on_demand_request(5, "n1", plugin_id="sports", mode="sports_live")
+    h.on_demand_request(40, "n2", action="stop")
+    h.on_demand_request(100, "n3", plugin_id="sports", mode="sports_live")
+    h.on_demand_request(140, "n4", action="stop")
+
+
+def scenario_on_demand_restore_failed(h: RunLoopHarness):
+    # A restart during a session whose plugin then fails to load: the
+    # session ends as an error before the first screen, and the rotation
+    # runs normally from the top.
+    h.add_plugin(FakePlugin("clock", ["clock"], duration=20))
+    h.add_plugin(FakePlugin("weather", ["weather"], duration=20))
+    h.restore_on_demand("gone", mode="gone", duration=40)
+
+
 def scenario_schedule(h: RunLoopHarness):
     # The clock starts at 22:59:30. Off from 23:01 until 23:05 (the window
     # spans midnight); dimmed from 23:00 until 23:01.
@@ -194,6 +220,8 @@ SCENARIOS = {
     "on_demand": (scenario_on_demand, 240),
     "on_demand_pinned": (scenario_on_demand_pinned, 160),
     "on_demand_restored": (scenario_on_demand_restored, 100),
+    "on_demand_named_live": (scenario_on_demand_named_live, 160),
+    "on_demand_restore_failed": (scenario_on_demand_restore_failed, 60),
     "schedule": (scenario_schedule, 400),
     "wifi_notice": (scenario_wifi_notice, 150),
     "follower": (scenario_follower, 80),

test/test_scroll_config.py

@@ -21,6 +21,7 @@ from src.common.scroll_config import (  # noqa: E402
     refresh_hz_from_config,
     resolve,
 )
+from src.common import scroll_config  # noqa: E402
 
 
 class FakeHelper:
@@ -504,3 +505,46 @@ class TestSpeedAdvice:
         got = solve_crisp(50, 125.74)
         assert got.steppiness == "smooth"
         assert got.pixels_per_frame == 1
+
+
+class TestRefreshShortfall:
+    """A panel that cannot reach its cap runs every scroll slow."""
+
+    def test_the_ledmatrix_rig_is_told_to_cap_at_100(self):
+        # Pi 4, 2x128x64 on adafruit-hat-pwm under a 120 Hz cap: measured
+        # 107.6-113.1 Hz, and frame_timing reports the fast end.
+        s = scroll_config.refresh_shortfall(113.1, 120)
+        assert s == {"measured_hz": 113.1, "planned_hz": 120.0,
+                     "suggested_cap_hz": 100, "slow_percent": 6}
+
+    def test_a_panel_that_holds_its_cap_is_fine(self):
+        assert scroll_config.refresh_shortfall(99.95, 100) is None
+        assert scroll_config.refresh_shortfall(97.5, 100) is None
+
+    def test_a_panel_that_beats_its_cap_is_fine(self):
+        assert scroll_config.refresh_shortfall(125.7, 120) is None
+
+    def test_nothing_measured_says_nothing(self):
+        assert scroll_config.refresh_shortfall(None, 120) is None
+        assert scroll_config.refresh_shortfall(0, 120) is None
+        assert scroll_config.refresh_shortfall("fast", 120) is None
+
+    def test_the_suggestion_leaves_headroom_under_the_measurement(self):
+        assert scroll_config.holdable_cap(113.1) == 100
+        assert scroll_config.holdable_cap(95.0) == 90
+        # 5% under 105 is 99.75: 100 would sit inside the panel's drift.
+        assert scroll_config.holdable_cap(105.0) == 90
+        assert scroll_config.holdable_cap(9.0) is None
+        assert scroll_config.holdable_cap(None) is None
+
+    def test_the_log_line_names_the_cap_to_use(self):
+        text = scroll_config.describe_refresh_shortfall(
+            scroll_config.refresh_shortfall(113.1, 120))
+        assert "about 113 Hz" in text and "120 Hz" in text
+        assert "6% slower" in text
+        assert "Set Limit Refresh Rate to 100 Hz" in text
+
+    def test_no_suggestion_for_a_panel_too_slow_for_any_cap(self):
+        text = scroll_config.describe_refresh_shortfall(
+            scroll_config.refresh_shortfall(9.0, 100))
+        assert "Set Limit Refresh Rate" not in text

test/test_snapshot_owed_frame.py

@@ -0,0 +1,183 @@
+"""A frame the preview throttle skipped still reaches the snapshot.
+
+The preview snapshot (/api/v3/display/current, the web UI's live preview) is
+only written from update_display(), at most once per write interval. A screen
+that draws its card once and then holds it -- soccer's recent/upcoming cards
+skip redundant redraws -- pushes exactly one frame. When that push lands inside
+the interval, e.g. a few milliseconds after the on-demand start's clear wrote a
+black frame, the throttle skips it and nothing ever writes it: on ledpi the
+preview stayed black for soccer's whole 15 s screen while the panel showed the
+card, and the next screen "rendered immediately".
+
+Runs the real DisplayManager on the emulator, like test_display_dirty_tracking.
+"""
+
+import os
+import sys
+import types
+
+os.environ["EMULATOR"] = "true"
+
+import pytest
+from PIL import Image
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
+
+
+@pytest.fixture(scope="module")
+def dm(tmp_path_factory):
+    from src.display_manager import DisplayManager
+    DisplayManager._instance = None
+    manager = DisplayManager({
+        "display": {
+            "hardware": {"rows": 32, "cols": 64, "chain_length": 2,
+                         "parallel": 1, "brightness": 90},
+            "runtime": {"gpio_slowdown": 0},
+        },
+    }, suppress_test_pattern=True)
+    manager._snapshot_path = str(
+        tmp_path_factory.mktemp("owed_snapshot") / "led_matrix_preview.png")
+    yield manager
+    DisplayManager._instance = None
+
+
+@pytest.fixture
+def viewer(dm, monkeypatch, tmp_path):
+    """A preview is open (1 s write interval); fresh snapshot bookkeeping."""
+    monkeypatch.setattr(dm, "_viewer_is_fresh", lambda now: True)
+    dm._viewer_was_fresh = True
+    dm._snapshot_path = str(tmp_path / "snap.png")
+    dm._last_snapshot_ts = 0.0
+    dm._last_snapshot_touch_ts = 0.0
+    dm._last_snapshot_digest = None
+    dm._saved_snapshot_digest = None
+    dm._snapshot_owed = False
+    dm.set_scrolling_state(False)
+    return dm
+
+
+def _lit(path):
+    with Image.open(path) as img:
+        return sum(1 for p in img.convert("RGB").getdata() if max(p) > 20)
+
+
+def _age_last_write(dm, seconds=2.0):
+    """As if `seconds` had passed since the last snapshot write."""
+    dm._last_snapshot_ts -= seconds
+    dm._last_snapshot_touch_ts -= seconds
+
+
+def _clear_then_draw_card(dm):
+    """The on-demand start's clear, then the card a few ms later."""
+    dm.clear()
+    dm.update_display()                       # black frame: written
+    assert _lit(dm._snapshot_path) == 0
+    dm.draw.rectangle([4, 4, 40, 20], fill=(255, 255, 0))
+    dm.update_display()                       # the card: inside the interval
+
+
+def _controller(dm):
+    from src import display_controller as dc_module
+    controller = dc_module.DisplayController.__new__(dc_module.DisplayController)
+    controller.plugin_manager = None
+    controller.display_manager = dm
+    return controller
+
+
+class _HoldingPlugin:
+    """Already showing its card: display() returns True and draws nothing."""
+
+    plugin_id = "holding"
+
+    def __init__(self):
+        self.calls = 0
+
+    def display(self, display_mode=None, force_clear=False):
+        self.calls += 1
+        return True
+
+
+def test_a_held_card_reaches_the_preview_on_the_next_frame(viewer):
+    dm = viewer
+    _clear_then_draw_card(dm)
+    assert _lit(dm._snapshot_path) == 0       # the throttle skipped the card
+
+    controller = _controller(dm)
+    plugin = _HoldingPlugin()
+    _age_last_write(dm)
+    # The render loop's next frame: the plugin draws nothing and makes no
+    # update_display() call, as soccer's switch cards do.
+    assert controller._display_once(plugin, "soccer_eng.1_recent", True) is True
+    assert plugin.calls == 1
+    assert _lit(dm._snapshot_path) > 0
+
+
+def test_the_owed_write_still_waits_out_the_interval(viewer, monkeypatch):
+    dm = viewer
+    _clear_then_draw_card(dm)
+    saves = []
+    monkeypatch.setattr(dm, "_save_snapshot", lambda image: saves.append(image))
+    dm.write_owed_snapshot()                  # still inside the interval
+    assert saves == []
+    _age_last_write(dm)
+    dm.write_owed_snapshot()
+    assert len(saves) == 1
+    # Written: nothing is owed, so later frames do no work and the unchanged
+    # frame is not encoded again.
+    assert dm._snapshot_owed is False
+    _age_last_write(dm)
+    dm.write_owed_snapshot()
+    assert len(saves) == 1
+
+
+def test_a_failed_owed_write_stays_owed_and_is_retried(viewer, monkeypatch):
+    dm = viewer
+    _clear_then_draw_card(dm)
+    _age_last_write(dm)
+    attempts = []
+
+    def failing_save(image):
+        attempts.append(image)
+        raise OSError("disk full")
+
+    monkeypatch.setattr(dm, "_save_snapshot", failing_save)
+    dm.write_owed_snapshot()                  # the write fails
+    assert len(attempts) == 1
+    assert dm._snapshot_owed is True          # still owed: a held screen
+    saves = []                                # makes no update_display()
+    monkeypatch.setattr(dm, "_save_snapshot", lambda image: saves.append(image))
+    dm.write_owed_snapshot()                  # retried on the next frame
+    assert len(saves) == 1
+    assert dm._snapshot_owed is False
+
+
+def test_nothing_owed_after_a_frame_that_was_written(viewer, monkeypatch):
+    dm = viewer
+    dm.draw.rectangle([0, 0, 8, 8], fill=(0, 255, 0))
+    dm.update_display()                       # due: written at once
+    assert dm._snapshot_owed is False
+    calls = []
+    monkeypatch.setattr(dm, "_write_snapshot_if_due",
+                        lambda *a, **k: calls.append(a))
+    dm.write_owed_snapshot()
+    assert calls == []
+
+
+def test_an_unchanged_frame_inside_the_interval_is_not_owed(viewer):
+    dm = viewer
+    dm.draw.rectangle([0, 0, 8, 8], fill=(0, 0, 255))
+    dm.update_display()
+    dm.update_display()                       # same frame, inside the interval
+    assert dm._snapshot_owed is False
+
+
+def test_a_controller_without_the_hook_still_draws():
+    """Controllers built without a display manager (tests) are unaffected."""
+    from src import display_controller as dc_module
+    controller = dc_module.DisplayController.__new__(dc_module.DisplayController)
+    controller.plugin_manager = None
+    plugin = _HoldingPlugin()
+    assert controller._display_once(plugin, "x", True) is True
+    controller.display_manager = types.SimpleNamespace()
+    assert controller._display_once(plugin, "x", True) is True
+    assert plugin.calls == 2

test/test_state_stream_readers.py

@@ -612,6 +612,12 @@ class TestEndToEnd:
         cached['display_current_state'] = {'mode': 'from-cache', 'last_updated': 1}
         path = str(tmp_path / 'control.sock')
         monkeypatch.setenv(c.SOCKET_PATH_ENV, path)
+        # The display is this process, and its render loop is beating: the
+        # cache is then still its answer once the socket goes.
+        heartbeat = tmp_path / 'display-heartbeat.json'
+        heartbeat.write_text(json.dumps({'pid': os.getpid(), 'mono': time.monotonic(),
+                                         'wall': time.time()}))
+        monkeypatch.setattr(display_watchdog, 'HEARTBEAT_PATH', str(heartbeat))
         hub = _hub_with_everything()
         server = ControlServer(path, state_hub=hub, keepalive=0.2)
         assert server.start()
@@ -638,3 +644,8 @@ class TestEndToEnd:
                 break
             time.sleep(0.05)
         assert (data['mode'], data['source']) == ('from-cache', 'cache')
+        # Stopped: systemd takes the heartbeat's directory with it, and the
+        # cache's last answer is no longer anyone's.
+        heartbeat.unlink()
+        data = _data(client, '/api/v3/display/current-status')
+        assert (data['mode'], data['source']) == (None, 'cache')

test/web_interface/test_api_v3_backup_restore.py

@@ -50,11 +50,13 @@ class FakeResult:
         self.plugins_to_install = plugins_to_install or []
         self.plugins_installed = []
         self.plugins_failed = []
+        self.skipped = []
 
     def to_dict(self):
         return {
             "success": self.success,
             "restored": self.restored,
+            "skipped": self.skipped,
             "errors": self.errors,
             "plugins_installed": self.plugins_installed,
             "plugins_failed": self.plugins_failed,
@@ -286,6 +288,109 @@ class TestPluginReinstall:
         assert body["data"]["plugins_failed"][0]["error"] == "Store manager unavailable"
 
 
+class TestInstalledPluginsAreNotReinstalled:
+    """"Reinstall missing plugins" installs only what is missing.
+
+    Every plugin the backup listed went to install_plugin, which replaces an
+    installed copy with a fresh download: restoring onto the same device
+    re-downloaded all of them inside the request. One installed from its own
+    URL is not in the registry, so its "reinstall" returned False and the
+    whole restore answered 500 "Restore failed" with the plugin still there.
+    """
+
+    @staticmethod
+    def _installed(tmp_path, *names):
+        found = {}
+        for name in names:
+            (tmp_path / name).mkdir()
+            found[name] = tmp_path / name
+        return lambda plugin_id: found.get(plugin_id)
+
+    def test_an_installed_plugin_is_skipped_and_a_missing_one_installed(
+            self, client, restore, tmp_path):
+        restore.return_value = FakeResult(
+            plugins_to_install=[{"plugin_id": "clock"}, {"plugin_id": "weather"}])
+        store = api_v3.plugin_store_manager
+        store._existing_install.side_effect = self._installed(tmp_path, "clock")
+        store.install_plugin.return_value = True
+        response = post(client)
+        assert response.status_code == 200
+        store.install_plugin.assert_called_once_with("weather")
+        data = response.get_json()["data"]
+        assert data["plugins_installed"] == ["weather"]
+        assert data["plugins_failed"] == []
+        assert "plugin:clock (installed)" in data["skipped"]
+
+    def test_an_installed_plugin_the_store_cannot_install_is_not_a_failure(
+            self, client, restore, tmp_path):
+        restore.return_value = FakeResult(plugins_to_install=[{"plugin_id": "my-3p"}])
+        store = api_v3.plugin_store_manager
+        store._existing_install.side_effect = self._installed(tmp_path, "my-3p")
+        store.install_plugin.return_value = False
+        response = post(client)
+        assert response.status_code == 200
+        assert response.get_json()["data"]["plugins_failed"] == []
+        store.install_plugin.assert_not_called()
+
+    @pytest.fixture
+    def real_store(self, tmp_path):
+        from src.plugin_system.store_manager import PluginStoreManager
+        plugins_dir = tmp_path / "plugin-repos"
+        for folder, manifest_id in (("ledmatrix-weather", "ledmatrix-weather"),
+                                    ("my-3p", "my-3p")):
+            (plugins_dir / folder).mkdir(parents=True)
+            (plugins_dir / folder / "manifest.json").write_text(
+                json.dumps({"id": manifest_id, "version": "1.0.0"}))
+        store = PluginStoreManager(plugins_dir=str(plugins_dir),
+                                   uninstalled_registry_path=str(tmp_path / "uninstalled.json"))
+        # The official weather plugin's registry id differs from the id it
+        # installs under; my-3p was installed from its own URL.
+        registry = {"plugins": [{
+            "id": "weather", "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins",
+            "plugin_path": "plugins/ledmatrix-weather"}]}
+        store.registry_cache = registry
+        store.fetch_registry = lambda *a, **k: registry
+        store.install_plugin = MagicMock(return_value=True)
+        api_v3.plugin_store_manager = store
+        return store
+
+    def test_with_the_real_store_aliases_and_third_party_installs_count(
+            self, client, restore, real_store):
+        restore.return_value = FakeResult(plugins_to_install=[
+            {"plugin_id": "weather"}, {"plugin_id": "my-3p"}, {"plugin_id": "clock"}])
+        response = post(client)
+        assert response.status_code == 200
+        real_store.install_plugin.assert_called_once_with("clock")
+        skipped = response.get_json()["data"]["skipped"]
+        assert "plugin:weather (installed)" in skipped
+        assert "plugin:my-3p (installed)" in skipped
+
+
+class TestFontsCatalogCache:
+    """The Fonts tab's catalog is cached for 5 minutes (fonts.py).
+
+    Upload and delete clear it; a restore did not, so restored fonts were
+    missing from the Fonts tab and every font picker until it expired.
+    """
+
+    @pytest.fixture
+    def cached_catalog(self):
+        from web_interface.cache import delete_cached, get_cached, set_cached
+        set_cached('fonts_catalog', {'fonts': ['5x7.bdf']}, ttl_seconds=300)
+        yield lambda: get_cached('fonts_catalog', ttl_seconds=300)
+        delete_cached('fonts_catalog')
+
+    def test_a_restore_that_restored_fonts_clears_it(self, client, restore, cached_catalog):
+        restore.return_value = FakeResult(restored=["config", "fonts (2)"])
+        assert post(client).status_code == 200
+        assert cached_catalog() is None
+
+    def test_a_restore_without_fonts_keeps_it(self, client, restore, cached_catalog):
+        restore.return_value = FakeResult(restored=["config"])
+        assert post(client).status_code == 200
+        assert cached_catalog() == {'fonts': ['5x7.bdf']}
+
+
 class TestFailureReporting:
     def test_restore_errors_produce_a_500(self, client, restore):
         restore.return_value = FakeResult(

test/web_interface/test_api_v3_config_raw.py

@@ -13,7 +13,9 @@ tmp_path so the assertions are against files on disk rather than mock
 calls.
 """
 
+import html
 import json
+import re
 import sys
 from pathlib import Path
 from unittest.mock import MagicMock
@@ -221,3 +223,66 @@ class TestRawEndpointsBypassSecretSeparation:
         env.client.post(MAIN, json={"weather": {"api_key": "PLAINTEXT-KEY"}})
         # Nothing was moved aside into the secrets file.
         assert not env.secrets_file.exists() or "PLAINTEXT-KEY" not in env.secrets_file.read_text()
+
+
+class TestConfigEditorRoundTrip:
+    """The Config Editor tab (/partials/raw-json) and the save it posts to.
+
+    The secrets editor is shown masked, like GET /config/secrets: the page is
+    served to anyone who can reach the port while the optional web login is
+    off. Its save strips the masks and merges onto the stored file, so a
+    masked editor saved back as it is changes nothing.
+    """
+
+    STORED = {
+        "github": {"api_token": "ghp_REAL_TOKEN_1234"},
+        "ledmatrix-weather": {"api_key": "WEATHER_KEY_abcdef", "units_id": 42},
+        "calendar": {"accounts": [{"name": "home", "token": "CAL_TOKEN_9"}]},
+        "youtube": {"api_key": "YOUR_YOUTUBE_API_KEY", "channel_secret": ""},
+    }
+    REAL_VALUES = ("ghp_REAL_TOKEN_1234", "WEATHER_KEY_abcdef", "CAL_TOKEN_9")
+
+    @pytest.fixture
+    def editor(self, env, monkeypatch):
+        from web_interface.blueprints import pages_v3 as pages_module
+        env.secrets_file.write_text(json.dumps(self.STORED))
+        monkeypatch.setattr(pages_module.pages_v3, "config_manager",
+                            env.config_manager, raising=False)
+        app = Flask(__name__, template_folder=str(project_root / "web_interface" / "templates"))
+        app.config["TESTING"] = True
+        app.register_blueprint(pages_module.pages_v3)
+        app.register_blueprint(api_v3, url_prefix="/api/v3")
+        return app.test_client()
+
+    @staticmethod
+    def _secrets_textarea(client):
+        page = client.get("/partials/raw-json")
+        assert page.status_code == 200
+        match = re.search(r'<textarea id="secrets-config-editor"[^>]*>(.*?)</textarea>',
+                          page.get_data(as_text=True), re.S)
+        assert match, "the secrets editor is missing from the partial"
+        return html.unescape(match.group(1))
+
+    def test_the_editor_shows_no_secret_value(self, editor):
+        text = self._secrets_textarea(editor)
+        for value in self.REAL_VALUES:
+            assert value not in text
+        shown = json.loads(text)
+        assert shown["github"]["api_token"] == "\u2022" * 8
+        # Same shape as the file, and "not set" still reads as not set.
+        assert shown["calendar"]["accounts"][0]["name"] == "\u2022" * 8
+        assert shown["youtube"] == {"api_key": "YOUR_YOUTUBE_API_KEY", "channel_secret": ""}
+
+    def test_saving_it_back_unchanged_keeps_every_secret(self, editor, env):
+        shown = json.loads(self._secrets_textarea(editor))
+        response = editor.post(SECRETS, json=shown)
+        assert response.status_code == 200
+        assert json.loads(env.secrets_file.read_text()) == self.STORED
+
+    def test_editing_one_secret_changes_only_that_one(self, editor, env):
+        shown = json.loads(self._secrets_textarea(editor))
+        shown["ledmatrix-weather"]["api_key"] = "NEW_WEATHER_KEY"
+        assert editor.post(SECRETS, json=shown).status_code == 200
+        expected = json.loads(json.dumps(self.STORED))
+        expected["ledmatrix-weather"]["api_key"] = "NEW_WEATHER_KEY"
+        assert json.loads(env.secrets_file.read_text()) == expected

test/web_interface/test_api_v3_helpers.py

@@ -169,6 +169,10 @@ class TestGetSchemaProperty:
             },
             "fifa.world": {"type": "object",
                            "properties": {"enabled": {"type": "boolean"}}},
+            "cities": {"type": "array",
+                       "items": {"type": "object",
+                                 "properties": {"timezone": {"type": "string"}}}},
+            "color": {"type": ["array", "null"], "items": {"type": "integer"}},
         }
     }
 
@@ -185,6 +189,15 @@ class TestGetSchemaProperty:
         prop = _get_schema_property(self.SCHEMA, "fifa.world.enabled")
         assert prop == {"type": "boolean"}
 
+    def test_an_index_steps_into_the_array_items(self):
+        # How a table row posts its cells
+        assert _get_schema_property(self.SCHEMA, "cities.0.timezone") == {"type": "string"}
+        assert _get_schema_property(self.SCHEMA, "color.2") == {"type": "integer"}
+
+    def test_a_non_index_under_an_array_is_not_found(self):
+        assert _get_schema_property(self.SCHEMA, "cities.timezone") is None
+        assert _get_schema_property(self.SCHEMA, "cities.0.nope") is None
+
     def test_missing_path_returns_none(self):
         assert _get_schema_property(self.SCHEMA, "nope.nope") is None
 

test/web_interface/test_api_v3_refresh_rate.py

@@ -0,0 +1,63 @@
+"""GET /api/v3/config/refresh-rate: the cap, the measured rate, a cap to hold."""
+import json
+from unittest.mock import MagicMock
+
+import pytest
+from flask import Flask
+
+from web_interface.blueprints.api_v3 import api_v3
+
+
+@pytest.fixture
+def client(monkeypatch, tmp_path):
+    stats = tmp_path / "stats.json"
+    monkeypatch.setattr("src.common.frame_timing.default_stats_path", lambda: str(stats))
+    manager = MagicMock()
+    manager.load_config.return_value = {
+        "display": {"hardware": {"limit_refresh_rate_hz": 120}}}
+    monkeypatch.setattr(api_v3, "config_manager", manager, raising=False)
+    app = Flask(__name__)
+    app.register_blueprint(api_v3, url_prefix="/api/v3")
+    c = app.test_client()
+    c.stats_path = stats
+    return c
+
+
+def _get(client):
+    body = client.get("/api/v3/config/refresh-rate").get_json()
+    assert body["status"] == "success"
+    return body["data"]
+
+
+def test_nothing_measured_yet(client):
+    data = _get(client)
+    assert data == {"planned_hz": 120.0, "measured_hz": None, "shortfall": None}
+
+
+def test_a_panel_short_of_its_cap_gets_a_cap_it_can_hold(client):
+    client.stats_path.write_text(json.dumps(
+        {"measured_refresh_hz": 110.4, "planned_refresh_hz": 120.0}))
+    data = _get(client)
+    assert data["measured_hz"] == 110.4
+    assert data["shortfall"]["suggested_cap_hz"] == 100
+    assert data["shortfall"]["slow_percent"] == 8
+
+
+def test_a_panel_at_its_cap_has_no_shortfall(client):
+    client.stats_path.write_text(json.dumps(
+        {"measured_refresh_hz": 121.3, "planned_refresh_hz": 120.0}))
+    assert _get(client)["shortfall"] is None
+
+
+def test_a_file_written_under_another_cap_is_stale(client):
+    # The cap was changed to 120 but the display still runs under 100 Hz.
+    client.stats_path.write_text(json.dumps(
+        {"measured_refresh_hz": 99.9, "planned_refresh_hz": 100.0}))
+    data = _get(client)
+    assert data["measured_hz"] is None
+    assert data["shortfall"] is None
+
+
+def test_a_file_from_a_display_too_old_to_record_its_cap_still_counts(client):
+    client.stats_path.write_text(json.dumps({"measured_refresh_hz": 110.4}))
+    assert _get(client)["shortfall"]["suggested_cap_hz"] == 100

test/web_interface/test_plugin_config_endpoints.py

@@ -0,0 +1,393 @@
+"""GET and POST /plugins/config against a real ConfigManager and SchemaManager.
+
+Each class is one bug, reproduced through the endpoint the settings form and
+API clients use, with assertions on config.json and config_secrets.json.
+"""
+
+import json
+from unittest.mock import MagicMock
+
+import pytest
+from flask import Flask
+
+from src.config_manager import ConfigManager
+from src.plugin_system.schema_manager import SchemaManager
+from web_interface.blueprints.api_v3 import api_v3
+
+PLUGIN_ID = "demo"
+OTHER_ID = "other"
+
+SCHEMA = {
+    "type": "object",
+    "properties": {
+        "enabled": {"type": "boolean", "default": True},
+        "api_key": {"type": "string", "x-secret": True, "default": ""},
+        "city": {"type": "string", "default": "Austin"},
+        "mqtt": {
+            "type": "object",
+            "properties": {
+                "host": {"type": "string", "default": ""},
+                "port": {"type": "integer", "default": 1883,
+                         "minimum": 1, "maximum": 65535},
+                "password": {"type": "string", "x-secret": True, "default": ""},
+            },
+        },
+        "accounts": {
+            "type": "array",
+            "default": [],
+            "items": {
+                "type": "object",
+                "properties": {
+                    "name": {"type": "string"},
+                    "token": {"type": "string", "x-secret": True},
+                },
+            },
+        },
+    },
+}
+
+OTHER_SCHEMA = {
+    "type": "object",
+    "properties": {
+        "enabled": {"type": "boolean", "default": True},
+        "label": {"type": "string", "default": "x"},
+    },
+}
+
+STORED = {
+    PLUGIN_ID: {"enabled": True, "city": "Paris",
+                "mqtt": {"host": "broker", "port": 1883},
+                "accounts": [{"name": "a"}, {"name": "b"}]},
+    OTHER_ID: {"enabled": True, "label": "hello"},
+}
+
+STORED_SECRETS = {
+    PLUGIN_ID: {"api_key": "TOPSECRET",
+                "accounts": [{"token": "TOK-A"}, {"token": "TOK-B"}]},
+}
+
+_ATTRS = ('config_manager', 'plugin_catalog', 'plugin_store_manager',
+          'saved_repositories_manager', 'schema_manager',
+          'operation_queue', 'operation_history', 'cache_manager')
+
+
+@pytest.fixture
+def env(tmp_path):
+    config_file = tmp_path / "config.json"
+    secrets_file = tmp_path / "config_secrets.json"
+    plugins_dir = tmp_path / "plugins"
+    for plugin_id, schema in ((PLUGIN_ID, SCHEMA), (OTHER_ID, OTHER_SCHEMA)):
+        plugin_dir = plugins_dir / plugin_id
+        plugin_dir.mkdir(parents=True)
+        (plugin_dir / "config_schema.json").write_text(json.dumps(schema))
+        (plugin_dir / "manifest.json").write_text(json.dumps({"id": plugin_id}))
+    config_file.write_text(json.dumps(STORED))
+    secrets_file.write_text(json.dumps(STORED_SECRETS))
+
+    sentinel = object()
+    originals = {name: getattr(api_v3, name, sentinel) for name in _ATTRS}
+
+    config_manager = ConfigManager(config_path=str(config_file),
+                                   secrets_path=str(secrets_file))
+    config_manager.template_path = str(tmp_path / "no-template.json")
+    plugin_manager = MagicMock()
+    plugin_manager.plugin_manifests = {PLUGIN_ID: {"id": PLUGIN_ID},
+                                       OTHER_ID: {"id": OTHER_ID}}
+    plugin_manager.plugins_dir = plugins_dir
+
+    for name in _ATTRS:
+        setattr(api_v3, name, MagicMock())
+    api_v3.config_manager = config_manager
+    api_v3.schema_manager = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path)
+    api_v3.plugin_catalog = plugin_manager
+    api_v3.operation_queue = None
+
+    app = Flask(__name__)
+    app.config["TESTING"] = True
+    app.register_blueprint(api_v3, url_prefix="/api/v3")
+
+    class Env:
+        client = app.test_client()
+
+        @staticmethod
+        def use_schema(schema, plugin_id=PLUGIN_ID):
+            (plugins_dir / plugin_id / "config_schema.json").write_text(json.dumps(schema))
+
+        @staticmethod
+        def store(section, plugin_id=PLUGIN_ID):
+            main = json.loads(config_file.read_text())
+            main[plugin_id] = section
+            config_file.write_text(json.dumps(main))
+
+        @staticmethod
+        def main():
+            return json.loads(config_file.read_text())
+
+        @staticmethod
+        def secrets():
+            return json.loads(secrets_file.read_text())
+
+        @staticmethod
+        def post_form(data, plugin_id=PLUGIN_ID):
+            return Env.client.post(f"/api/v3/plugins/config?plugin_id={plugin_id}",
+                                   data=data)
+
+        @staticmethod
+        def post_json(config, plugin_id=PLUGIN_ID):
+            return Env.client.post("/api/v3/plugins/config",
+                                   json={"plugin_id": plugin_id, "config": config})
+
+    yield Env
+
+    for name, original in originals.items():
+        if original is sentinel:
+            if hasattr(api_v3, name):
+                delattr(api_v3, name)
+        else:
+            setattr(api_v3, name, original)
+
+
+class TestARejectedSaveLeavesNothingBehind:
+    """The form save edited the cached config load_config hands out, then
+    failed validation. The cache kept the edit, and the next save of any
+    other setting wrote it to config.json -- the rejected value, and a
+    nested secret typed into the same form in plain text."""
+
+    REJECTED = {"mqtt.host": "broker", "mqtt.port": "99999",
+                "mqtt.password": "hunter2", "__rendered_section": ["mqtt"]}
+
+    def test_the_rejected_values_never_reach_config_json(self, env):
+        assert env.post_form(self.REJECTED).status_code == 400
+
+        resp = env.post_json({"label": "bye"}, plugin_id=OTHER_ID)
+        assert resp.status_code == 200, resp.get_json()
+
+        main = env.main()
+        assert main[OTHER_ID]["label"] == "bye"
+        assert main[PLUGIN_ID]["mqtt"] == {"host": "broker", "port": 1883}
+        assert "hunter2" not in json.dumps(main)
+
+    def test_the_form_reloads_with_the_stored_values(self, env):
+        assert env.post_form(self.REJECTED).status_code == 400
+        assert api_v3.config_manager.load_config()[PLUGIN_ID]["mqtt"]["port"] == 1883
+
+
+class TestGetMasksSecrets:
+    """GET /plugins/config returned the section with config_secrets.json
+    merged in, secrets and all: the masking #276 added was lost when the
+    route was rewritten. The settings page and GET /config/secrets mask."""
+
+    def test_secrets_come_back_blank(self, env):
+        data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
+        assert data["api_key"] == ""
+        assert data["accounts"] == [{"name": "a", "token": ""}, {"name": "b", "token": ""}]
+        assert data["city"] == "Paris"
+
+    def test_posting_the_response_back_keeps_every_secret(self, env):
+        data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
+        resp = env.post_json(data)
+        assert resp.status_code == 200, resp.get_json()
+        assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
+        assert "TOPSECRET" not in json.dumps(env.main())
+
+    def test_the_settings_form_posting_masked_fields_keeps_every_secret(self, env):
+        # The page renders secrets blank (pages_v3 masks the same way)
+        resp = env.post_form({
+            "api_key": "", "city": "Lyon", "mqtt.host": "broker", "mqtt.port": "1883",
+            "mqtt.password": "", "__rendered_section": ["api_key", "city", "mqtt"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
+        assert env.main()[PLUGIN_ID]["city"] == "Lyon"
+
+    def test_a_plugin_without_a_schema_has_credential_named_fields_blanked(self, env, tmp_path):
+        (tmp_path / "plugins" / "bare").mkdir()
+        env.store({"enabled": True, "station": "KAUS"}, plugin_id="bare")
+        secrets = env.secrets()
+        secrets["bare"] = {"api_token": "BARE-TOKEN"}
+        (tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
+        data = env.client.get("/api/v3/plugins/config?plugin_id=bare").get_json()["data"]
+        assert data["api_token"] == ""
+        assert data["station"] == "KAUS"
+
+    @pytest.mark.parametrize("section", ["web_auth", "github", "display"])
+    def test_a_core_section_is_refused(self, env, tmp_path, section):
+        secrets = env.secrets()
+        secrets["web_auth"] = {"cookie_secret": "COOKIE-KEY", "password_hash": "HASH"}
+        secrets["github"] = {"api_token": "ghp_TOKEN"}
+        (tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
+        env.store({"hardware": {"rows": 32}}, plugin_id="display")
+        resp = env.client.get(f"/api/v3/plugins/config?plugin_id={section}")
+        assert resp.status_code == 400
+        body = resp.get_data(as_text=True)
+        assert "COOKIE-KEY" not in body and "ghp_TOKEN" not in body
+
+
+ROWS_SCHEMA = {
+    "type": "object",
+    "properties": {
+        "enabled": {"type": "boolean", "default": True},
+        "cities": {
+            "type": "array",
+            "x-widget": "array-table",
+            "default": [],
+            "items": {
+                "type": "object",
+                "properties": {
+                    "name": {"type": "string"},
+                    "timezone": {"type": "string"},
+                    "lat": {"type": "number"},
+                    "show": {"type": "boolean", "default": True},
+                },
+                "required": ["name", "lat"],
+            },
+        },
+    },
+}
+
+
+class TestArrayRowCellsFollowTheItemSchema:
+    """A table row posts its cells as ``cities.0.timezone``. The schema
+    lookup stopped at the array, so each cell was parsed blind: a blank
+    optional text cell became null and a text cell holding digits became a
+    number, and either failed validation -- every save of the page, for as
+    long as the row existed (geochron's city without a timezone, a countdown
+    named "2027")."""
+
+    ROW = {"cities.0.name": "Tokyo", "cities.0.timezone": "Asia/Tokyo",
+           "cities.0.lat": "35.68", "cities.0.show": "true",
+           "__rendered_section": ["cities"]}
+
+    @pytest.fixture(autouse=True)
+    def _rows(self, env):
+        env.use_schema(ROWS_SCHEMA)
+        env.store({"enabled": True, "cities": [
+            {"name": "Tokyo", "timezone": "Asia/Tokyo", "lat": 35.68, "show": True}]})
+
+    def test_a_blank_optional_text_cell_saves(self, env):
+        resp = env.post_form({**self.ROW, "cities.0.timezone": ""})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.main()[PLUGIN_ID]["cities"][0]["timezone"] == ""
+
+    def test_a_text_cell_of_digits_stays_text(self, env):
+        resp = env.post_form({**self.ROW, "cities.0.name": "2027"})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.main()[PLUGIN_ID]["cities"][0]["name"] == "2027"
+
+    def test_number_and_boolean_cells_still_convert(self, env):
+        resp = env.post_form({**self.ROW, "cities.0.show": "false"})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.main()[PLUGIN_ID]["cities"] == [
+            {"name": "Tokyo", "timezone": "Asia/Tokyo", "lat": 35.68, "show": False}]
+
+
+class TestMaskedSecretCellsInARow:
+    """The same lookup: a row's secret cell, rendered blank, came back as
+    null and failed validation, so a plugin with secrets in a list could not
+    be saved from its settings page at all."""
+
+    def test_the_stored_tokens_survive_a_save_of_the_form(self, env):
+        resp = env.post_form({
+            "city": "Lyon", "accounts.0.name": "a", "accounts.0.token": "",
+            "accounts.1.name": "b", "accounts.1.token": "",
+            "__rendered_section": ["city", "accounts"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
+        assert env.main()[PLUGIN_ID]["accounts"] == [{"name": "a"}, {"name": "b"}]
+
+
+class TestABlankSecretIsLeftAsStored:
+    """The form renders a secret blank and posts the blank back. For a
+    required secret with no default (youtube-stats' api_key) the blank was
+    read as null, failed validation, and blocked every save of the page
+    until the key was typed in again."""
+
+    @pytest.fixture(autouse=True)
+    def _required_secret(self, env):
+        schema = json.loads(json.dumps(SCHEMA))
+        del schema["properties"]["api_key"]["default"]
+        schema["required"] = ["api_key"]
+        env.use_schema(schema)
+
+    def test_saving_other_settings_keeps_the_stored_secret(self, env):
+        resp = env.post_form({"api_key": "", "city": "Lyon",
+                              "__rendered_section": ["api_key", "city"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.main()[PLUGIN_ID]["city"] == "Lyon"
+        assert env.secrets()[PLUGIN_ID]["api_key"] == "TOPSECRET"
+
+    def test_a_new_secret_is_still_saved(self, env):
+        resp = env.post_form({"api_key": "NEW-KEY", "city": "Lyon",
+                              "__rendered_section": ["api_key", "city"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.secrets()[PLUGIN_ID]["api_key"] == "NEW-KEY"
+
+    def test_a_changed_secret_then_left_blank_stays_changed(self, env):
+        # The second save must not write back what the first one's load
+        # had merged in (the old key)
+        env.post_form({"api_key": "NEW-KEY", "__rendered_section": ["api_key"]})
+        resp = env.post_form({"api_key": "", "city": "Nice",
+                              "__rendered_section": ["api_key", "city"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.secrets()[PLUGIN_ID]["api_key"] == "NEW-KEY"
+
+    def test_a_blank_list_secret_is_left_as_stored_too(self, env, tmp_path):
+        schema = json.loads(json.dumps(SCHEMA))
+        schema["properties"]["tokens"] = {"type": "array", "x-secret": True,
+                                          "items": {"type": "string"}, "default": []}
+        env.use_schema(schema)
+        secrets = env.secrets()
+        secrets[PLUGIN_ID]["tokens"] = ["t1", "t2"]
+        (tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
+        resp = env.post_form({"tokens": "", "city": "Lyon",
+                              "__rendered_section": ["tokens", "city"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.secrets()[PLUGIN_ID]["tokens"] == ["t1", "t2"]
+
+
+class TestSaveRefusesWhatIsNotAPluginId:
+    """GET and reset refuse a core section or a malformed id; the save took
+    any of them. ``{"plugin_id": "display"}`` merged unvalidated values into
+    the core display section, and an id that was not a string raised a
+    TypeError, answered as a 500."""
+
+    def test_a_core_section_is_refused_and_left_alone(self, env):
+        env.store({"hardware": {"rows": 32}}, plugin_id="display")
+        resp = env.post_json({"hardware": {"rows": "banana"}}, plugin_id="display")
+        assert resp.status_code == 400
+        assert env.main()["display"] == {"hardware": {"rows": 32}}
+
+    def test_the_form_save_refuses_one_too(self, env):
+        resp = env.post_form({"password_hash": "x"}, plugin_id="web_auth")
+        assert resp.status_code == 400
+        assert "web_auth" not in env.main()
+
+    @pytest.mark.parametrize("plugin_id", [["demo"], {"id": "demo"}, 7, "", "../demo"])
+    def test_a_malformed_id_is_a_400(self, env, plugin_id):
+        resp = env.post_json({"city": "Lyon"}, plugin_id=plugin_id)
+        assert resp.status_code == 400
+
+
+class TestTextFieldsKeepWhatWasTyped:
+    """A text field holding "true", "False", "[1, 2]" or "{}" was converted
+    to a boolean, list or object before the schema's type was consulted, and
+    the save then failed validation for a perfectly good string."""
+
+    @pytest.mark.parametrize("typed", ["true", "False", "[1, 2]", "{}", "42"])
+    def test_a_text_field(self, env, typed):
+        resp = env.post_form({"city": typed, "__rendered_section": ["city"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.main()[PLUGIN_ID]["city"] == typed
+
+    def test_a_nullable_text_field(self, env):
+        schema = json.loads(json.dumps(SCHEMA))
+        schema["properties"]["nickname"] = {"type": ["string", "null"], "default": None}
+        env.use_schema(schema)
+        resp = env.post_form({"nickname": "false", "__rendered_section": ["nickname"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.main()[PLUGIN_ID]["nickname"] == "false"
+
+    def test_other_types_still_convert(self, env):
+        resp = env.post_form({"mqtt.host": "true", "mqtt.port": "8883",
+                              "__rendered_section": ["mqtt"]})
+        assert resp.status_code == 200, resp.get_json()
+        assert env.main()[PLUGIN_ID]["mqtt"] == {"host": "true", "port": 8883}

test/web_interface/test_web_process_runs_no_plugin_code.py

@@ -221,7 +221,9 @@ class TestTheWebProcessNeverRunsAPlugin:
     def test_saving_its_section_through_the_main_config(self, web):
         body = web.post("/api/v3/config/main", {PLUGIN_ID: {"message": "via main"}})
         assert web.stored()["message"] == "via main"
-        assert body["restart_required"] is True
+        # The display's config watcher hands the section to the running
+        # plugin (on_config_change), as for /plugins/config: no restart.
+        assert body["restart_required"] is False
         assert web.ran() == []
 
     def test_resetting_its_config(self, web):

web_interface/blueprints/api_v3/__init__.py

@@ -957,6 +957,19 @@ def _get_schema_property(schema, key_path):
                     i = j
                     matched = True
                     break
+                # Through an array to its items: a table row posts its cells
+                # as "cities.0.timezone", where the index names no property.
+                # Stopping here left each cell parsed with no schema at all,
+                # so a blank text cell became null and "2027" a number.
+                items = prop.get('items') if _schema_type_is(prop, 'array') else None
+                if isinstance(items, dict) and parts[j].isdigit():
+                    if j + 1 == len(parts):
+                        return items
+                    if 'properties' in items:
+                        current = items['properties']
+                        i = j + 1
+                        matched = True
+                        break
                 # Matched a non-object before consuming the path — can't go deeper.
                 return None
         if not matched:
@@ -1040,6 +1053,16 @@ def _parse_form_value_with_schema(value, key_path, schema):
 
     # Handle None/empty values
     if value is None or (isinstance(value, str) and value.strip() == ''):
+        # The form draws a stored secret blank, so a blank secret means
+        # "unchanged", and "" is what the save drops as unchanged
+        # (remove_empty_secrets). A required one with no default fell
+        # through to None below, failed validation, and blocked every save
+        # of the page until the secret was typed in again. Not _SKIP_FIELD:
+        # that keeps the merged value from load_config(), which the save
+        # would then write back to config_secrets.json. Text secrets only:
+        # a list or object one gets its empty value below, dropped the same.
+        if prop and prop.get('x-secret') and prop.get('type', 'string') == 'string':
+            return ""
         # A nullable field left blank means null, not an empty container.
         # This is the inherit sentinel for per-mode style overrides: an
         # empty list there would read as "the user chose no colour" rather
@@ -1074,6 +1097,14 @@ def _parse_form_value_with_schema(value, key_path, schema):
     if isinstance(value, str):
         stripped = value.strip()
 
+        # A text field keeps what was typed. The guesses below ran first, so
+        # "true", "False", "[1, 2]" or "{}" in a text field became a boolean,
+        # list or object, and the save failed validation for a good string.
+        declared = prop.get('type') if isinstance(prop, dict) else None
+        if declared == 'string' or (isinstance(declared, list) and
+                                    [t for t in declared if t != 'null'] == ['string']):
+            return value
+
         # Check for boolean strings
         if stripped.lower() == 'true':
             return True

web_interface/blueprints/api_v3/backup.py

@@ -16,6 +16,7 @@ import web_interface.blueprints.api_v3 as _pkg
 # as module attributes, and a value binding would not see the patch.
 # Several are also called from helpers that live in __init__, so the
 # package is the only patch point that covers every caller.
+from web_interface.cache import delete_cached
 
 
 @api_v3.route('/backup/preview', methods=['GET'])
@@ -85,6 +86,17 @@ _RESTORE_OPTION_KEYS = frozenset((
     'restore_config', 'restore_secrets', 'restore_wifi', 'restore_fonts',
     'restore_plugin_uploads', 'reinstall_plugins',
 ))
+def _installed_path(psm, plugin_id):
+    """Where the store finds ``plugin_id`` installed, or None.
+
+    The same lookup install_plugin makes to decide that a copy exists: the
+    id, or an id the registry proves is the same plugin (``aliases``, the
+    ``plugin_path`` name), never a bare ``ledmatrix-<id>`` folder.
+    """
+    found = psm._existing_install(plugin_id)
+    return found if isinstance(found, Path) and found.exists() else None
+
+
 @api_v3.route('/backup/restore', methods=['POST'])
 def backup_restore():
     """Restore a backup ZIP with optional RestoreOptions."""
@@ -134,6 +146,10 @@ def backup_restore():
             os.unlink(tmp_path)
         except OSError:
             pass
+    # Restored fonts reach the Fonts tab through a catalog cached for five
+    # minutes (fonts.py); upload and delete clear it, and so must this.
+    if any(str(item).startswith('fonts') for item in result.restored):
+        delete_cached('fonts_catalog')
 
     # Reinstall plugins if requested and store manager available
     if options.reinstall_plugins and result.plugins_to_install:
@@ -143,6 +159,15 @@ def backup_restore():
             if not pid:
                 continue
             try:
+                # Only what is missing. install_plugin replaces an installed
+                # copy with a fresh download, so restoring onto the same
+                # device re-downloaded every plugin, and one installed from
+                # its own URL (not in the registry) "failed" and failed the
+                # whole restore while it sat there installed. The store's
+                # own lookup, so registry aliases count as installed too.
+                if psm and _installed_path(psm, pid) is not None:
+                    result.skipped.append(f'plugin:{pid} (installed)')
+                    continue
                 if psm and hasattr(psm, 'install_plugin'):
                     ok = psm.install_plugin(pid)
                     if ok:

web_interface/blueprints/api_v3/config.py

@@ -17,6 +17,8 @@ from src.pi5_matrix_support import is_raspberry_pi_5
 from web_interface.cache import invalidate_cache
 from web_interface.auth import SECTION as _WEB_AUTH_SECTION, strip_auth_section
 import web_interface.blueprints.api_v3 as _pkg
+import copy
+from typing import Any, Dict, Iterable, Tuple
 
 # Read through the module rather than bound by value: tests patch these
 # as module attributes, and a value binding would not see the patch.
@@ -33,6 +35,50 @@ FORM_SECTION_FIELD = '__form_section'
 GENERAL_FIELDS = ('timezone', 'city', 'state', 'country', 'web_display_autostart',
                   'plugins_directory', 'auto_update_enabled', 'auto_update_channel')
 
+#: Settings in config.json the running display applies without a restart,
+#: as key paths (a path covers everything under it). Brightness goes over the
+#: control socket (brightness.set) and the config watcher's refresh
+#: (DisplayController._refresh_config_cache); the per-mode durations are read
+#: from the live config each time a mode starts (_get_display_duration).
+#: Plugin sections are live as well (each plugin's on_config_change), and
+#: save_main_config adds the ones a request saves.
+LIVE_CONFIG_PATHS: Tuple[Tuple[str, ...], ...] = (
+    ('display', 'hardware', 'brightness'),
+    ('display', 'display_durations'),
+)
+
+
+_MISSING = object()
+
+
+def _config_leaves(config: Any, prefix: Tuple[str, ...] = ()) -> Dict[Tuple[str, ...], Any]:
+    """Every non-dict value in ``config``, by key path. An empty dict has none,
+    so a section created empty on the way to a field is not a change."""
+    if not isinstance(config, dict):
+        return {prefix: config}
+    leaves: Dict[Tuple[str, ...], Any] = {}
+    for key, value in config.items():
+        leaves.update(_config_leaves(value, prefix + (str(key),)))
+    return leaves
+
+
+def restart_needed(before: Dict[str, Any], after: Dict[str, Any],
+                   live_paths: Iterable[Tuple[str, ...]] = LIVE_CONFIG_PATHS) -> bool:
+    """Does going from config ``before`` to ``after`` need a display restart?
+
+    True when anything changed outside ``live_paths``. A save that changes
+    only live settings, or nothing at all, does not.
+    """
+    live = tuple(live_paths)
+    old, new = _config_leaves(before), _config_leaves(after)
+    for path in set(old) | set(new):
+        if old.get(path, _MISSING) == new.get(path, _MISSING):
+            continue
+        if not any(path[:len(prefix)] == prefix for prefix in live):
+            return True
+    return False
+
+
 #: Top-level fields save_main_config stores somewhere of its own (location,
 #: plugin_system, ...), never as a config key of the same name.
 _MAPPED_TOP_LEVEL_FIELDS = GENERAL_FIELDS + (
@@ -72,6 +118,22 @@ def _day_setting(data, day, flat_key, nested_key):
     return False, None
 
 
+def _disabled_day_times(data, day, start_key, end_key):
+    """The times posted for a day that is off, the valid ones.
+
+    Nothing reads them while the day is off, but the schedule picker posts
+    them and GET returns them, so keeping them means turning the day back on
+    finds what was there. An invalid one is dropped rather than refused, for
+    the same reason.
+    """
+    times = {}
+    for field, key in (('start_time', start_key), ('end_time', end_key)):
+        value = _day_setting(data, day, key, field)[1]
+        if value and _validate_time_format(value)[0]:
+            times[field] = value
+    return times
+
+
 @api_v3.route('/config/main', methods=['GET'])
 def get_main_config():
     """Get main configuration, with credentials redacted."""
@@ -96,11 +158,16 @@ def _panel_refresh_hz(config):
     cap = scroll_config.refresh_hz_from_config(config)
     try:
         with open(frame_timing.default_stats_path(), encoding='utf-8') as fh:
-            measured = float(json.load(fh).get('measured_refresh_hz') or 0)
+            stats = json.load(fh)
+        measured = float(stats.get('measured_refresh_hz') or 0)
+        planned = float(stats.get('planned_refresh_hz') or 0)
     except (OSError, ValueError, TypeError, AttributeError):
+        measured = planned = 0.0
+    # Reject a stale file from a previous hardware config: one written under
+    # another cap (the display has not restarted since it changed), or, from
+    # a display too old to record its cap, a measurement far off this one.
+    if planned and abs(planned - cap) > 0.5:
         measured = 0.0
-    # Reject a stale file from a previous hardware config: a measurement far
-    # off the cap says the config changed since it was written.
     if measured > 0 and 0.5 * cap <= measured <= 1.5 * cap:
         return measured, 'measured'
     return cap, 'configured'
@@ -127,6 +194,29 @@ def get_scroll_speed_advice():
     return jsonify({'status': 'success', 'data': advice})
 
 
+@api_v3.route('/config/refresh-rate', methods=['GET'])
+def get_refresh_rate():
+    """The refresh cap, what the panel measured, and a cap it can hold.
+
+    Backs the hint under the Display tab's Limit Refresh Rate field. Scroll
+    speeds are solved against the cap, so a panel that cannot reach it runs
+    every scroll slow; ``shortfall`` (None when the panel keeps up, or nothing
+    has been measured yet) says by how much and suggests a cap.
+    """
+    from src.common import scroll_config
+    if not api_v3.config_manager:
+        return jsonify({'status': 'error', 'message': 'Config manager not initialized'}), 500
+    config = api_v3.config_manager.load_config()
+    planned = scroll_config.refresh_hz_from_config(config)
+    hz, source = _panel_refresh_hz(config)
+    measured = hz if source == 'measured' else None
+    return jsonify({'status': 'success', 'data': {
+        'planned_hz': planned,
+        'measured_hz': round(measured, 1) if measured else None,
+        'shortfall': scroll_config.refresh_shortfall(measured, planned),
+    }})
+
+
 @api_v3.route('/config/schedule', methods=['GET'])
 def get_schedule_config():
     """Get current schedule configuration"""
@@ -257,11 +347,16 @@ def save_schedule_config():
 
                     day_config['start_time'] = start_time
                     day_config['end_time'] = end_time
+                else:
+                    day_config.update(_disabled_day_times(data, day, start_key, end_key))
 
                 schedule_config['days'][day] = day_config
 
-            # Validate that at least one day is enabled in per-day mode
+            # An enabled per-day schedule needs a day to be on. A disabled
-            if enabled_days_count == 0:
+            # one does not: every day off with the schedule off is what
+            # config.template.json ships, so refusing it meant a fresh
+            # install could not post back the schedule GET returned.
+            if enabled_days_count == 0 and enabled_value:
                 return error_response(
                     ErrorCode.VALIDATION_ERROR,
                     "At least one day must be enabled in per-day schedule mode",
@@ -465,11 +560,13 @@ def save_dim_schedule_config():
 
                     day_config['start_time'] = start_time
                     day_config['end_time'] = end_time
+                else:
+                    day_config.update(_disabled_day_times(data, day, start_key, end_key))
 
                 dim_schedule_config['days'][day] = day_config
 
-            # Validate that at least one day is enabled in per-day mode
+            # As for the on/off schedule: only an enabled one needs a day on.
-            if enabled_days_count == 0:
+            if enabled_days_count == 0 and enabled_value:
                 return error_response(
                     ErrorCode.VALIDATION_ERROR,
                     "At least one day must be enabled in per-day dim schedule mode",
@@ -507,7 +604,11 @@ def save_main_config():
         # Try to get JSON data first, fallback to form data
         data = None
         if request.is_json:
-            data = request.get_json()
+            # silent=True, as in save_raw_main_config: get_json() raised
+            # Werkzeug's BadRequest into the catch-all below, a 500.
+            data = request.get_json(silent=True)
+            if data is None and request.get_data():
+                return jsonify({'status': 'error', 'message': 'Invalid JSON in request body'}), 400
             if data is not None and not isinstance(data, dict):
                 return jsonify({'status': 'error', 'message': 'Request body must be a JSON object'}), 400
         else:
@@ -553,6 +654,8 @@ def save_main_config():
 
         # Merge with existing config (similar to original implementation)
         current_config = api_v3.config_manager.load_config()
+        # What was stored, to tell which settings this save changed.
+        stored_config = copy.deepcopy(current_config)
         was_auto_update_enabled = bool((current_config.get('auto_update') or {}).get('enabled'))
 
         is_general_update = any(k in data for k in GENERAL_FIELDS)
@@ -1190,13 +1293,15 @@ def save_main_config():
                 message = f'{message}. {note}'
         except Exception:
             logger.warning("Automatic update setup could not be started", exc_info=True)
-        # Display hardware, rotation/durations and general settings take
+        # Display hardware, rotation order and general settings take effect
-        # effect after a display restart; the UI shows its restart banner on
+        # after a display restart; the UI shows its restart banner on this
-        # this flag.
+        # flag. Brightness, mode durations and plugin settings are applied by
-        extra = {'restart_required': True}
+        # the running display (LIVE_CONFIG_PATHS), so a save that changed
-        # Brightness is the exception: the display applies a saved one
+        # only those -- or nothing -- does not ask for one.
-        # without a restart. Over the control socket it lands at once,
+        live_paths = LIVE_CONFIG_PATHS + tuple((plugin_id,) for plugin_id in plugin_keys_to_remove)
-        # instead of when the config watcher next looks (up to ~2 s).
+        extra = {'restart_required': restart_needed(stored_config, current_config, live_paths)}
+        # Over the control socket a saved brightness lands at once, instead
+        # of when the config watcher next looks (up to ~2 s).
         if 'brightness' in data:
             saved = (current_config.get('display', {}).get('hardware', {}) or {}).get('brightness')
             if isinstance(saved, int) and not isinstance(saved, bool):

web_interface/blueprints/api_v3/display.py

@@ -69,6 +69,26 @@ def _deliver_on_demand(payload):
     return 'mailbox', reason
 
 
+def _withdraw_on_demand(request_id):
+    """Take a start request the route has refused back out of the mailbox.
+
+    The display reads the mailbox for an hour without looking at a
+    request's age, so one left there after an error answer ran whenever the
+    display next started. Only this request is removed: the mailbox is
+    re-read and cleared only while it still holds this request_id, as the
+    display's _consume_on_demand_request does, so a newer request posted in
+    the meantime stays for the display to take.
+    """
+    cache = _cache_manager()
+    try:
+        current = cache.get('display_on_demand_request', max_age=3600, memory_ttl=0)
+        if isinstance(current, dict) and current.get('request_id') == request_id:
+            cache.delete('display_on_demand_request')
+    except Exception:  # the route is answering an error already
+        logger.warning("Could not withdraw on-demand request %s from the mailbox",
+                       request_id, exc_info=True)
+
+
 @api_v3.route('/display/current', methods=['GET'])
 def get_display_current():
     """The latest display preview, as the /stream/display SSE stream sends it.
@@ -259,9 +279,26 @@ def start_on_demand_display():
     }
     transport, socket_error = _deliver_on_demand(request_payload)
 
+    # A socket acknowledgement is the display itself answering: it is
+    # running and has the request queued, whatever systemd says (a display
+    # run by hand or in the emulator has no active unit). So nothing is
+    # checked or started for it -- that answered "not running" for a request
+    # that had already taken effect. The service is still reported the way
+    # _ensure_display_service_running reports a running one.
+    if transport == 'socket':
+        service_result = (dict(_get_display_service_status(), started=False)
+                          if start_service else None)
+        return _on_demand_started(request_id, resolved_plugin, resolved_mode,
+                                  duration, pinned, service_result, transport,
+                                  socket_error)
+
     service_status = _get_display_service_status()
 
     if not service_status.get('active') and not start_service:
+        # The request is in the mailbox, and the display reads it whenever
+        # it next starts: taken back out, or a request answered with this
+        # error ran later anyway.
+        _withdraw_on_demand(request_id)
         return jsonify({
             'status': 'error',
             'message': 'Display service is not running. Please start the display service or enable "Start Service" option.',
@@ -285,16 +322,25 @@ def start_on_demand_display():
         service_result = _ensure_display_service_running()
         # Check if service actually started
         if service_result and not service_result.get('active'):
+            _withdraw_on_demand(request_id)
             return jsonify({
                 'status': 'error',
                 'message': 'Failed to start display service. Please check service logs or start it manually.',
                 'service_result': service_result
             }), 500
 
+    return _on_demand_started(request_id, resolved_plugin, resolved_mode,
+                              duration, pinned, service_result, transport,
+                              socket_error)
+
+
+def _on_demand_started(request_id, plugin_id, mode, duration, pinned,
+                       service_result, transport, socket_error):
+    """The success answer of /display/on-demand/start."""
     response_data = {
         'request_id': request_id,
-        'plugin_id': resolved_plugin,
+        'plugin_id': plugin_id,
-        'mode': resolved_mode,
+        'mode': mode,
         'duration': duration,
         'pinned': pinned,
         'service': service_result,
@@ -340,15 +386,22 @@ def get_current_display_status():
     Read from the display's state stream over the control socket when it is
     available (``source: "socket"``). Otherwise from what the display
     publishes to the shared cache (display_controller._publish_current_mode_state)
-    when the active mode changes (``source: "cache"``).
+    when the active mode changes (``source: "cache"``). Unknown (every field
+    None) when the socket and the heartbeat both say the display is gone
+    (display_state.display_gone).
     """
-    state = display_state.current_status(display_state.read_state())
+    snapshot = display_state.read_state()
+    state = display_state.current_status(snapshot)
     source = 'socket'
     if state is None:
         source = 'cache'
-        cache = _cache_manager()
+        # A stopped display leaves its last answer in the cache, where it
-        # memory_ttl=0: written by the display service; see get_on_demand_status.
+        # read as on (is_display_active: true) for the 120 s max_age. With
-        state = cache.get('display_current_state', max_age=120, memory_ttl=0)
+        # no socket and no live heartbeat there is no display behind it.
+        if not display_state.display_gone(snapshot):
+            cache = _cache_manager()
+            # memory_ttl=0: written by the display service; see get_on_demand_status.
+            state = cache.get('display_current_state', max_age=120, memory_ttl=0)
     if state is None:
         state = {
             'mode': None,

web_interface/blueprints/api_v3/misc.py

@@ -102,6 +102,7 @@ def get_health():
         # the only signal, as it always was.
         # The display reports the same beat's age over the control socket's
         # state stream, measured in memory; the file is the fallback.
+        snapshot = None
         try:
             snapshot = display_state.read_state()
             if snapshot is not None:
@@ -132,6 +133,26 @@ def get_health():
                 'error': 'see logs for details'
             }
 
+        # A stopped display service. The heartbeat's absence alone says
+        # nothing (the dev server, the emulator and Windows write none), so
+        # the overall status stayed "healthy" with the display down until the
+        # last preview frame it left aged past 60 s (hardware: stale). Together
+        # the three signals are definite: systemd says the service is not
+        # active, the control socket does not answer, and there is no live
+        # heartbeat (display_state.display_gone, which is never true where
+        # the platform has no socket or it is switched off).
+        try:
+            if (not display_service_status.get('active')
+                    and display_state.display_gone(snapshot)):
+                health_status['checks']['display_loop'] = {
+                    'status': 'stopped',
+                    'note': 'The display service is not running',
+                    'source': 'service',
+                }
+        except Exception:
+            logger.warning("Health check could not tell whether the display is stopped",
+                           exc_info=True)
+
         # Check hardware connectivity (if display manager available)
         try:
             snapshot_path = display_preview.SNAPSHOT_PATH

web_interface/blueprints/api_v3/plugin_assets.py

@@ -3,8 +3,12 @@
 Routes decorate the shared `api_v3` Blueprint from the package `__init__`,
 so their endpoint names do not depend on which module they live in.
 """
+import mimetypes
+
+from flask import send_file
+
 from web_interface.blueprints.api_v3 import (
-    PROJECT_ROOT, Response, _plugin_directory, api_v3, datetime, hashlib,
+    PROJECT_ROOT, _plugin_directory, api_v3, datetime, hashlib,
     json, jsonify, logger, os, request, uuid,
 )
 from src.common.path_safety import (
@@ -231,8 +235,8 @@ def serve_plugin_static(plugin_id, file_path):
     if not requested_file.exists() or not requested_file.is_file():
         return jsonify({'status': 'error', 'message': 'File not found'}), 404
 
-    # Determine content type
+    # Determine content type. Text keeps the types this route always set;
-    content_type = 'text/plain'
+    # anything else (an icon, a preview image) gets its own.
     name = requested_file.name
     if name.endswith('.html'):
         content_type = 'text/html'
@@ -242,12 +246,14 @@ def serve_plugin_static(plugin_id, file_path):
         content_type = 'text/css'
     elif name.endswith('.json'):
         content_type = 'application/json'
+    else:
+        guessed = mimetypes.guess_type(name)[0]
+        content_type = ('text/plain' if not guessed or guessed.startswith('text/')
+                        else guessed)
 
-    # Read and return file
+    # Sent as bytes. Opening it as UTF-8 text failed to decode any binary
-    with open(requested_file, 'r', encoding='utf-8') as f:
+    # file, so an image answered 500 UnicodeDecodeError.
-        content = f.read()
+    return send_file(requested_file, mimetype=content_type)
-
-    return Response(content, mimetype=content_type)
 
 
 @api_v3.route('/plugins/assets/delete', methods=['POST'])

web_interface/blueprints/api_v3/plugin_config.py

@@ -9,13 +9,15 @@ from web_interface.blueprints.api_v3 import (
     _enhance_schema_with_core_properties, _non_plugin_id_error,
     _filter_config_by_schema, _get_schema_property,
     _hidden_array_item_property, _plugin_directory,
-    _parse_form_value_with_schema, _schema_allows_null, _schema_type_is,
+    _parse_form_value_with_schema, _redact_credentials, _schema_allows_null,
-    _set_missing_booleans_to_false, _set_nested_value, api_v3, datetime,
+    _schema_type_is, _set_missing_booleans_to_false, _set_nested_value, api_v3,
-    deep_merge, error_response, exception_error_response, find_secret_fields,
+    datetime, deep_merge, error_response, exception_error_response,
-    json, jsonify, logger, merge_secrets, os, remove_empty_secrets, request,
+    find_secret_fields, json, jsonify, logger, merge_secrets, os,
-    separate_secrets, success_response, validate_request_json,
+    remove_empty_secrets, request, separate_secrets, success_response,
+    validate_request_json,
 )
 from src.web_interface.config_arrays import coerce_array_shapes
+from src.web_interface.secret_helpers import mask_secret_fields
 from src.web_interface.validators import dedup_unique_arrays
 import web_interface.blueprints.api_v3 as _pkg
 # Read through the module rather than bound by value: tests patch these
@@ -43,6 +45,12 @@ def get_plugin_config():
                 context={'missing_params': ['plugin_id']},
                 status_code=400
             )
+        # load_config() merges config_secrets.json in, core sections
+        # included: ?plugin_id=web_auth returned the login's cookie key and
+        # password hash, and ?plugin_id=github the Plugin Store token.
+        id_error = _non_plugin_id_error(plugin_id)
+        if id_error:
+            return id_error
 
         # Get plugin configuration from config manager
         main_config = api_v3.config_manager.load_config()
@@ -52,12 +60,13 @@ def get_plugin_config():
         # missing fields, reading legacy booleans as objects first: what the
         # plugin runs with, and what posts back through the JSON save
         schema_mgr = api_v3.schema_manager
+        schema = None
         if schema_mgr:
             try:
                 from src.plugin_system.schema_manager import prepare_plugin_config
+                schema = schema_mgr.load_schema(plugin_id, use_cache=True)
                 defaults = schema_mgr.generate_default_config(plugin_id, use_cache=True)
-                plugin_config = prepare_plugin_config(
+                plugin_config = prepare_plugin_config(plugin_config, schema, defaults)
-                    plugin_config, schema_mgr.load_schema(plugin_id, use_cache=True), defaults)
             except Exception as e:
                 # Log but don't fail - defaults merge is best effort
                 logger.warning("Could not merge defaults for %s: %s", plugin_id, e)
@@ -158,6 +167,17 @@ def get_plugin_config():
                 'display_duration': 30
             }
 
+        # Secrets go out blank, as the settings page renders them (#276 added
+        # this; #330 dropped it). Blank, not the bullets GET /config/secrets
+        # uses: the save reads a blank secret as "unchanged", so this
+        # response posts back without erasing one.
+        properties = schema.get('properties') if isinstance(schema, dict) else None
+        if isinstance(properties, dict):
+            plugin_config = mask_secret_fields(plugin_config, properties)
+        else:
+            # No schema to mark them: blank whatever is named like one
+            plugin_config = _redact_credentials(plugin_config)
+
         return success_response(data=plugin_config)
     except Exception as e:
         return exception_error_response(e, ErrorCode.CONFIG_LOAD_FAILED)
@@ -183,6 +203,12 @@ def save_plugin_config():
             if error:
                 return error
             plugin_id = data['plugin_id']
+            # As reset and uninstall do: {"plugin_id": "display"} merged
+            # unvalidated values into the core display section, and an id
+            # that was not a string raised a TypeError, answered as a 500.
+            id_error = _non_plugin_id_error(plugin_id)
+            if id_error:
+                return id_error
             submitted_config = data.get('config', {})
             if not isinstance(submitted_config, dict):
                 return error_response(
@@ -201,6 +227,9 @@ def save_plugin_config():
                     'plugin_id required in query string',
                     status_code=400
                 )
+            id_error = _non_plugin_id_error(plugin_id)
+            if id_error:
+                return id_error
 
             # Load existing config as base (partial form updates should merge, not replace)
             existing_config = {}

web_interface/blueprints/api_v3/plugin_store.py

@@ -81,6 +81,27 @@ def _listed_plugin_dir(base: Path, name: str) -> Optional[Path]:
     return None
 
 
+def _enqueue_or_conflict(operation_type, plugin_id, callback):
+    """``(operation_id, None)``, or ``(None, a 409 response)``.
+
+    The queue raises ValueError when the plugin already has an operation
+    waiting or running -- a double-clicked Install, an uninstall during an
+    install. That is the caller's timing, not a server fault: it reached
+    the client as a 500, and the uninstall route recorded a failed
+    uninstall that had never started.
+    """
+    try:
+        return api_v3.operation_queue.enqueue_operation(
+            operation_type, plugin_id, operation_callback=callback), None
+    except ValueError:
+        return None, error_response(
+            ErrorCode.PLUGIN_OPERATION_CONFLICT,
+            f'Plugin {plugin_id} already has an install, update or uninstall '
+            'in progress; wait for it to finish, then try again',
+            status_code=409
+        )
+
+
 @api_v3.route('/plugins/update', methods=['POST'])
 def update_plugin():
     """Update plugin"""
@@ -402,11 +423,10 @@ def uninstall_plugin():
                                                 preserve_config=preserve_config)}
 
             # Enqueue operation
-            operation_id = api_v3.operation_queue.enqueue_operation(
+            operation_id, conflict = _enqueue_or_conflict(
-                OperationType.UNINSTALL,
+                OperationType.UNINSTALL, plugin_id, uninstall_callback)
-                plugin_id,
+            if conflict:
-                operation_callback=uninstall_callback
+                return conflict
-            )
 
             return success_response(
                 data={'operation_id': operation_id},
@@ -538,11 +558,10 @@ def install_plugin():
                 raise Exception(error_msg)
 
         # Enqueue operation
-        operation_id = api_v3.operation_queue.enqueue_operation(
+        operation_id, conflict = _enqueue_or_conflict(
-            OperationType.INSTALL,
+            OperationType.INSTALL, plugin_id, install_callback)
-            plugin_id,
+        if conflict:
-            operation_callback=install_callback
+            return conflict
-        )
 
         branch_msg = f" (branch: {branch})" if branch else ""
         return success_response(

web_interface/blueprints/api_v3/plugins.py

@@ -439,6 +439,10 @@ sys.exit(proc.returncode)
                     import tempfile
                     import json as json_lib
 
+                    # The params reach the wrapper on its stdin, never in
+                    # its source: written there as `params = <JSON>`, a
+                    # true, false or null was an undefined name and the
+                    # wrapper died with a NameError before the script ran.
                     params_json = json_lib.dumps(action_params)
                     with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as wrapper:
                         wrapper.write(f'''import sys
@@ -449,6 +453,9 @@ import json
 # Set LEDMATRIX_ROOT
 os.environ['LEDMATRIX_ROOT'] = r"{PROJECT_ROOT}"
 
+# The params, as JSON on this wrapper's own stdin
+params = json.loads(sys.stdin.read())
+
 # Run the script and provide params as JSON via stdin
 proc = subprocess.Popen(
     [sys.executable, r"{script_file}"],
@@ -460,7 +467,6 @@ proc = subprocess.Popen(
 )
 
 # Send params as JSON to stdin
-params = {params_json}
 stdout, _ = proc.communicate(input=json.dumps(params), timeout=120)
 print(stdout)
 sys.exit(proc.returncode)
@@ -470,6 +476,7 @@ sys.exit(proc.returncode)
                     try:
                         result = subprocess.run(
                             ['python3', wrapper_path],
+                            input=params_json,
                             capture_output=True,
                             text=True,
                             timeout=120,

web_interface/blueprints/pages_v3.py

@@ -11,7 +11,7 @@ _SAFE_PLUGIN_ID_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
 _SAFE_WEB_UI_FILE_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}\.html$')
 _SAFE_WIDGET_NAME_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
 _SAFE_WIDGET_SCRIPT_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}\.js$')
-from src.web_interface.secret_helpers import mask_secret_fields
+from src.web_interface.secret_helpers import mask_all_secret_values, mask_secret_fields
 from src.plugin_system.schema_manager import plugin_config_defaults, prepare_plugin_config
 from src.common.path_safety import resolve_under, safe_path_component
 from src.pi5_matrix_support import is_raspberry_pi_5
@@ -623,9 +623,14 @@ def _load_raw_json_partial():
         main_config_data = pages_v3.config_manager.get_raw_file_content('main')
         # The web login section (password and token hashes) is managed in
         # General > Security, never in this editor; its save keeps it.
+        # The rest is masked, as GET /api/v3/config/secrets masks it: this
+        # page is served to anyone who can reach the port while the web
+        # login is off, and it was handing them every credential in the
+        # file. The save strips the masks and merges onto the stored file
+        # (save_raw_secrets_config), so a value left masked stays as it is.
         from web_interface.auth import strip_auth_section
-        secrets_config_data = strip_auth_section(
+        secrets_config_data = mask_all_secret_values(strip_auth_section(
-            pages_v3.config_manager.get_raw_file_content('secrets'))
+            pages_v3.config_manager.get_raw_file_content('secrets')))
         main_config_json = json.dumps(main_config_data, indent=4)
         secrets_config_json = json.dumps(secrets_config_data, indent=4)
 

web_interface/display_state.py

@@ -133,6 +133,39 @@ def on_demand_state(snapshot: Optional[Dict[str, Any]],
     return state
 
 
+def display_gone(snapshot: Optional[Dict[str, Any]]) -> bool:
+    """Is there positively no display behind a fallback to the cache?
+
+    True only when the socket should be there (this platform has one and it
+    is not switched off) but gave no ``snapshot``, and the render loop's
+    heartbeat file says nothing is running either: it is absent (systemd
+    removes its directory when the service stops), stale, or written by a
+    process that no longer exists -- #726's rules for the runtime snapshot.
+    Then what the display last left in the cache is a dead process's answer.
+
+    False whenever the answer is in doubt: a snapshot came in, the socket is
+    off or unsupported (Windows, the test suite, a deliberate ``off``), or a
+    live heartbeat says the display is running without a socket (an older
+    display). Those read the cache exactly as before.
+    """
+    if snapshot is not None:
+        return False
+    if not socket_supported() or not client_socket_paths():
+        return False
+    from src import display_watchdog
+    from src.plugin_system.plugin_runtime import process_exists
+    heartbeat = display_watchdog.read_heartbeat(display_watchdog.HEARTBEAT_PATH)
+    if heartbeat is None:
+        return True
+    age = display_watchdog.heartbeat_age(heartbeat)
+    if age is None or age >= display_watchdog.HEARTBEAT_STALE_SECONDS:
+        return True
+    pid = heartbeat.get('pid')
+    if isinstance(pid, int) and not isinstance(pid, bool) and process_exists(pid) is False:
+        return True
+    return False
+
+
 def loop_heartbeat_age(snapshot: Optional[Dict[str, Any]]) -> Optional[float]:
     """The render loop's heartbeat age now; None when the display has no
     beat to report yet (or there is no snapshot)."""
@@ -141,5 +174,5 @@ def loop_heartbeat_age(snapshot: Optional[Dict[str, Any]]) -> Optional[float]:
     return control_client.snapshot_loop_age(snapshot)
 
 
-__all__ = ['current_status', 'loop_heartbeat_age', 'on_demand_state', 'read_state',
+__all__ = ['current_status', 'display_gone', 'loop_heartbeat_age', 'on_demand_state',
-           'stop_subscription']
+           'read_state', 'stop_subscription']

web_interface/static/v3/js/widgets/plugin-order-list.js

@@ -18,7 +18,9 @@
  *   });
  *
  * The container re-renders from /api/v3/plugins/installed each init; the
- * hidden input(s) must already hold the saved order/exclusions (JSON).
+ * hidden input(s) must already hold the saved order/exclusions (JSON). Saved
+ * ids of disabled plugins (installed, but without a row) stay in them, in
+ * their saved places; ids of plugins no longer installed are dropped.
  */
 (function() {
     'use strict';
@@ -39,17 +41,60 @@
         const excludedInput = options.excludedInputId ? document.getElementById(options.excludedInputId) : null;
         if (!container || !orderInput) return;
 
+        // The saved lists as the inputs held them when the rows were drawn.
+        // Only enabled plugins get a row, and the inputs are rewritten from
+        // the rows, so a disabled plugin's place and exclusion have to be
+        // carried over from these: dropped, the next Display or Durations
+        // save stored the lists without it, and once re-enabled it came back
+        // at the end of the rotation and scrolling in Vegas again.
+        let savedOrder = [];
+        let savedExcluded = [];
+        // Every installed plugin's id, enabled or not, from the same
+        // response. A saved id outside it belongs to an uninstalled plugin
+        // and is dropped, as every save used to; without the list, nothing
+        // is dropped.
+        let installedIds = null;
+
+        // Saved ids of installed plugins with no row, once each. Only
+        // strings: /config/main refuses a list holding anything else, which
+        // would block every save.
+        function unlisted(saved, rowIds) {
+            const seen = new Set(rowIds);
+            return saved.filter(id => {
+                if (typeof id !== 'string' || seen.has(id)) return false;
+                if (installedIds && !installedIds.has(id)) return false;
+                seen.add(id);
+                return true;
+            });
+        }
+
         function syncInputs() {
-            const order = [];
+            const rowIds = [];
             const excluded = [];
             container.querySelectorAll('.plugin-order-item').forEach(item => {
                 const pluginId = item.dataset.pluginId;
-                order.push(pluginId);
+                rowIds.push(pluginId);
                 const checkbox = item.querySelector('.plugin-order-include');
                 if (checkbox && !checkbox.checked) excluded.push(pluginId);
             });
-            orderInput.value = JSON.stringify(order);
+            // An id without a row keeps its saved slot; the rows fill the
-            if (excludedInput) excludedInput.value = JSON.stringify(excluded);
+            // other slots in their current order, and any rows left over
+            // (plugins not in the saved order) go last.
+            const kept = new Set(unlisted(savedOrder, rowIds));
+            const order = [];
+            let next = 0;
+            savedOrder.forEach(id => {
+                if (kept.has(id)) {
+                    order.push(id);
+                    kept.delete(id);
+                } else if (rowIds.includes(id) && next < rowIds.length) {
+                    order.push(rowIds[next++]);
+                }
+            });
+            orderInput.value = JSON.stringify(order.concat(rowIds.slice(next)));
+            if (excludedInput) {
+                excludedInput.value = JSON.stringify(excluded.concat(unlisted(savedExcluded, rowIds)));
+            }
         }
 
         function setupDragAndDrop() {
@@ -104,6 +149,7 @@
             .then(data => {
                 const allPlugins = (data.data && data.data.plugins) || data.plugins || [];
                 const plugins = allPlugins.filter(p => p.enabled);
+                installedIds = new Set(allPlugins.map(p => p && p.id));
                 if (plugins.length === 0) {
                     const empty = document.createElement('p');
                     empty.className = 'text-sm text-gray-500 italic';
@@ -125,6 +171,8 @@
                 // (e.g. a saved value of "null"); normalize to arrays.
                 if (!Array.isArray(currentOrder)) currentOrder = [];
                 if (!Array.isArray(excluded)) excluded = [];
+                savedOrder = currentOrder;
+                savedExcluded = excluded;
 
                 // Saved order first, then any newly enabled plugins.
                 const orderedPlugins = [];

web_interface/templates/v3/partials/display.html

@@ -318,6 +318,7 @@
                            min="0"
                            max="1000"
                            class="form-control">
+                    <p id="limit_refresh_rate_hz_hint" class="mt-1 text-xs text-amber-700" aria-live="polite"></p>
                 </div>
             </div>
 
@@ -884,7 +885,6 @@ With this off a live game takes over the whole display with the full-screen scor
 // Update brightness display
 document.getElementById('brightness').addEventListener('input', function() {
     document.getElementById('brightness-value').textContent = this.value;
-    document.getElementById('brightness-display').textContent = this.value;
 });
 
 
@@ -916,6 +916,41 @@ document.getElementById('brightness').addEventListener('input', function() {
         });
     }
 
+    // Say so when the panel cannot reach its refresh cap. Scroll speeds are
+    // worked out against the cap, so every scroll then runs slow, and the
+    // display has measured a cap the panel can hold.
+    (function refreshRateHint() {
+        const hint = document.getElementById('limit_refresh_rate_hz_hint');
+        const input = document.getElementById('limit_refresh_rate_hz');
+        if (!hint || !input) return;
+        fetch('/api/v3/config/refresh-rate')
+            .then(function(r) { return r.json(); })
+            .then(function(body) {
+                const s = body.status === 'success' && body.data.shortfall;
+                hint.textContent = '';
+                if (!s) return;
+                hint.appendChild(document.createTextNode(
+                    'This panel refreshes at about ' + Math.round(s.measured_hz) +
+                    ' Hz, below this ' + Math.round(s.planned_hz) + ' Hz cap, so scrolls run about ' +
+                    s.slow_percent + '% slower than set.' + (s.suggested_cap_hz ? ' ' : '')));
+                if (!s.suggested_cap_hz) return;
+                const btn = document.createElement('button');
+                btn.type = 'button';
+                btn.className = 'underline font-medium';
+                btn.textContent = 'Use ' + s.suggested_cap_hz + ' Hz';
+                btn.addEventListener('click', function() {
+                    input.value = s.suggested_cap_hz;
+                    input.dispatchEvent(new Event('input', {bubbles: true}));
+                    input.dispatchEvent(new Event('change', {bubbles: true}));
+                    hint.textContent = 'Save, then restart the display, to apply ' +
+                        s.suggested_cap_hz + ' Hz.';
+                });
+                hint.appendChild(btn);
+                hint.appendChild(document.createTextNode(', a cap it can hold.'));
+            })
+            .catch(function() { hint.textContent = ''; });
+    })();
+
     // Declared before first use: let is not hoisted usably.
     let scrollHintTimer = null;
     let scrollHintSeq = 0;

web_interface/templates/v3/partials/general.html

@@ -404,6 +404,10 @@
                 document.getElementById('web-login-new-token-value').textContent = res.d.data.token;
                 document.getElementById('web-login-new-token').classList.remove('hidden');
                 form.reset();
+                // app.js marks a form dirty on input and clears the mark only
+                // after an htmx save; this one posts with fetch, so clear it
+                // here or a reload asks "Leave site?" about a saved token.
+                form.removeAttribute('data-dirty');
                 notify(res.d.message || 'Token created', 'success');
             }).catch(function(err) { notify('Request failed: ' + err.message, 'error'); });
         },

web_interface/templates/v3/partials/overview.html

@@ -55,19 +55,40 @@
 <script>
 (function () {
     var DISMISS_KEY = 'ledmatrix-recon-dismissed';
+    // Startup reconciliation is done within seconds of the web service
+    // starting. The route also answers done: false when its status file is
+    // missing (reconciliation raised before writing it, or /tmp was cleaned
+    // under a long-running service), which used to keep this polling every
+    // 2 s for as long as the page was open, on every tab. So: give up after
+    // a minute, and poll only while the Overview is on screen.
+    var POLL_MS = 2000;
+    var MAX_POLLS = 30;
     var _recon_timer = null;
+    var _polls = 0;
+    var _finished = false;   // done, given up, or dismissed
+    var _active = false;
+    var _inFlight = false;
 
     function checkReconciliation() {
+        _recon_timer = null;
+        _inFlight = true;
         fetch('/api/v3/plugins/reconciliation-status')
             .then(function (r) { return r.json(); })
             .then(function (resp) {
+                _inFlight = false;
+                if (_finished) return;
                 var d = resp.data || {};
                 if (!d.done) {
-                    // Reconciliation still running — poll again shortly
+                    // Reconciliation still running (or it never wrote its
-                    _recon_timer = setTimeout(checkReconciliation, 2000);
+                    // status): ask again shortly, a bounded number of times.
+                    if (++_polls >= MAX_POLLS) {
+                        _finished = true;
+                        return;
+                    }
+                    if (_active) _recon_timer = setTimeout(checkReconciliation, POLL_MS);
                     return;
                 }
-                _recon_timer = null;
+                _finished = true;
                 if (!d.unresolved || d.unresolved.length === 0) return;
                 var key = d.unresolved.map(function (i) { return i.plugin_id; }).sort().join(',');
                 if (sessionStorage.getItem(DISMISS_KEY) === key) return;
@@ -102,13 +123,33 @@
                 banner.dataset.dismissKey = key;
                 banner.style.setProperty('display', 'flex', 'important');
             })
-            .catch(function () {});
+            .catch(function () { _inFlight = false; });
+    }
+
+    function startReconciliationPoll() {
+        _active = true;
+        if (!_finished && _recon_timer === null && !_inFlight) checkReconciliation();
+    }
+    function stopReconciliationPoll() {
+        _active = false;
+        if (_recon_timer !== null) {
+            clearTimeout(_recon_timer);
+            _recon_timer = null;
+        }
+    }
+    // Keyed apart from any other Overview registration, which a shared key
+    // would replace.
+    if (window.LEDVisibility) {
+        window.LEDVisibility.onActive('overview', startReconciliationPoll,
+                                      stopReconciliationPoll, 'overview-reconciliation');
+    } else {
+        startReconciliationPoll();
     }
-    checkReconciliation();
 
     window.dismissReconciliationBanner = function () {
         var banner = document.getElementById('reconciliation-banner');
         banner.style.setProperty('display', 'none', 'important');
+        _finished = true;
         if (_recon_timer !== null) {
             clearTimeout(_recon_timer);
             _recon_timer = null;

web_interface/templates/v3/partials/tools.html

@@ -1093,6 +1093,12 @@
                 Use TLS
                 <span class="text-xs text-gray-500">(a password without TLS crosses the network in the clear)</span>
             </label>
+            <label id="mqtt-allow-insecure-row" class="${c.mqtt_tls ? 'hidden' : 'flex'} items-center gap-2 text-sm text-gray-700">
+                <input id="mqtt-allow-insecure" type="checkbox" ${c.allow_insecure_mqtt ? 'checked' : ''}
+                       class="rounded border-gray-300">
+                Allow without TLS (trusted network)
+                <span class="text-xs text-gray-500">(needed to save a password while TLS is off)</span>
+            </label>
 
             <div class="flex items-center justify-between gap-4 pt-2">
                 <p class="text-xs text-gray-500">
@@ -1132,6 +1138,15 @@
         if (clearBtn) clearBtn.addEventListener('click', () => clearMqttPassword());
         const clearTokenBtn = document.getElementById('mqtt-clear-api-token');
         if (clearTokenBtn) clearTokenBtn.addEventListener('click', () => clearMqttApiToken());
+        // The cleartext opt-in only means something while TLS is off.
+        const tlsBox = document.getElementById('mqtt-tls');
+        const allowRow = document.getElementById('mqtt-allow-insecure-row');
+        if (tlsBox && allowRow) {
+            tlsBox.addEventListener('change', () => {
+                allowRow.classList.toggle('hidden', tlsBox.checked);
+                allowRow.classList.toggle('flex', !tlsBox.checked);
+            });
+        }
     }
 
     window.loadMqttBridge = function() {
@@ -1160,6 +1175,9 @@
             on_demand_duration: val('mqtt-duration') === '' ? null : val('mqtt-duration'),
             log_level: val('mqtt-log-level'),
             mqtt_tls: !!(document.getElementById('mqtt-tls') || {}).checked,
+            // The server refuses a password with TLS off unless this is set
+            // (CWE-319); it is off until the user ticks it.
+            allow_insecure_mqtt: !!(document.getElementById('mqtt-allow-insecure') || {}).checked,
         };
         // Only send a password when one was typed; blank means "leave it alone".
         const pw = val('mqtt-password');