perf(fetch): cache ESPN scoreboard windows without the parts nothing reads

The sports scoreboards cache their Recent/Upcoming window as the raw ESPN
response, and it stays parsed in the memory tier while fresh. Measured on
hdpi, most of it is never drawn: per-team stat leaders, athlete cards
(featuredAthletes, probables), team and event links, headlines, video
highlights and geo broadcasts. None of those keys is read by core or by any
plugin in ledmatrix-plugins.

BackgroundDataService now drops them from an ESPN /scoreboard response
before caching and delivering it (src/common/espn_payload.py), keeping
everything else. On the five hdpi windows that is 10.6MB -> 3.0MB of JSON
and ~40MB -> ~12MB of parsed objects, and parsing an expired window gets
3-4x cheaper. submit_fetch_request(slim_payload=False) caches a response
whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkfgXMqqwn2w4NN7LRzhxy

CHANGELOG.md

@@ -618,57 +618,6 @@ policies are unchanged.
   the plugin leaves rotation until the cooldown ends, the same as a raising
   `update()`. The display still moves straight on to the next mode. A hung
   `display()` is still recorded once, as a hang.
-- A plugin settings save that failed validation no longer leaks into the next
-  save. `ConfigManager.load_config()` returned its cached config itself (the
-  fast path from #410), so the form save's edits went into the cache before
-  validation ran, and a refused save left them there. The next save of any
-  other setting (another plugin's, a plugin toggle, the schedule) wrote them
-  to config.json: the refused value, and a nested secret typed into the same
-  form (`mqtt.password`, `league.espn_s2`, `flightaware.api_key`) in plain
-  text, because it had never reached config_secrets.json to be stripped.
-  The form also reloaded showing the refused values. `load_config()` now
-  returns a private copy, and the saves keep one, so nothing a caller edits
-  reaches the cache unless it is saved. The copy duplicates only the dicts
-  and lists (every other JSON value is immutable): 2.1 ms for a real 60 KiB
-  config on a Pi 4, against 6.8 ms for `copy.deepcopy`.
-- `GET /api/v3/plugins/config` no longer returns secrets. It sent back the
-  plugin's section with config_secrets.json merged in, API keys and tokens
-  in plain text: the masking #276 added was dropped in #330. It also took
-  any id, so `?plugin_id=web_auth` returned the login's cookie-signing key
-  and password hash and `?plugin_id=github` the Plugin Store token. Secret
-  fields now come back blank, as the settings page renders them, and a
-  plugin with no schema has its credential-named fields blanked, as
-  `GET /config/main` does. Blank rather than the `••••••••` of
-  `GET /config/secrets`, because the save reads a blank secret as
-  "unchanged", so a client can post the response back without erasing
-  one. Core sections and malformed ids get a 400, as they already did from
-  reset and uninstall.
-- Plugin settings with a table (a list of rows, such as geochron's cities
-  or the countdowns) save again when a text cell is blank or holds only
-  digits. A row posts its cells as `cities.0.timezone`, and the schema
-  lookup stopped at the list, so each cell was parsed with no schema: a
-  blank optional text cell became null, and a name like "2027" became a
-  number. Either failed validation, and every save of the page failed for
-  as long as the row existed. A plugin with a secret in its rows could not
-  be saved from the page at all, since the secret cell is drawn blank. The
-  lookup now steps from the index into the list's item schema.
-- A plugin whose API key is required and has no default (youtube-stats)
-  can be saved from its settings page without typing the key in again. The
-  page draws a stored secret blank and posts the blank back; for a required
-  secret the save read that blank as null, failed validation, and refused
-  every save of the page. A blank secret field now means "unchanged", as it
-  already did for an optional one.
-- `POST /api/v3/plugins/config` refuses a core section or a malformed
-  plugin id with a 400, as reset and uninstall already did.
-  `{"plugin_id": "display", ...}` merged unvalidated values into the core
-  display section (and added `"enabled": true` to it), and an id that was
-  not a string answered with a 500.
-- A plugin text setting saves what was typed when that looks like a
-  boolean or JSON. The form save tried `true`/`false` and `[...]`/`{...}`
-  before it looked at the schema, so a text field holding "true", "False",
-  "[1, 2]" or "{}" was stored as a boolean, list or object, and the save
-  failed validation. Text fields, nullable ones included, are now taken as
-  typed; other types convert as before.
 - A WiFi notice (such as "Connected to HomeNet" or "AP mode on") now shows
   within about a second of being posted. It was only checked between
   screens, so a 5 s notice posted during a 20 s screen expired before that
@@ -677,42 +626,6 @@ policies are unchanged.
   notice is what shows next, and Vegas resumes after it; before, a rotation
   screen showed instead and the notice expired behind it. An active
   on-demand session still holds the panel until it ends.
-- The Config Editor tab no longer shows API keys and tokens in plain
-  text. Its `config_secrets.json` editor (`/partials/raw-json`) was filled
-  with the file as it is on disk, so while the web login is off (the
-  default) anyone who could reach the port could read every credential,
-  although `GET /api/v3/config/secrets` masks them. The editor now shows the
-  same masked values. Saving it unchanged changes nothing, because the save
-  drops the masks and merges onto the stored file; to change a secret,
-  replace its mask. A list of secrets still needs every entry's real value
-  to be changed. The `config.json` editor is unchanged: its save writes the
-  file as given, so a mask there would be stored.
-- A disabled plugin keeps its place in the rotation order and its Vegas
-  exclusion when the Display or Rotation & Durations tab is saved. The order
-  lists show enabled plugins only and rewrite their hidden inputs from those
-  rows as soon as they are drawn, so any save of either tab stored the lists
-  without the disabled plugin. Once re-enabled, it came back at the end of
-  the rotation and scrolling in Vegas again. A disabled plugin's saved id
-  now stays in its saved place (`widgets/plugin-order-list.js`); the id of
-  a plugin that is no longer installed is still dropped.
-- Restoring a backup with "Reinstall missing plugins" installs only the
-  plugins that are missing. Every plugin the backup listed was sent to the
-  store's install, which replaces an installed copy with a fresh download,
-  so a restore onto the same device re-downloaded all of them in one
-  request. A plugin installed from its own URL is not in the registry, so
-  its "reinstall" failed and the restore answered "Restore failed" while
-  the plugin sat there installed. An installed plugin, found by the store's
-  own lookup (registry aliases included), is now listed under Skipped as
-  `plugin:<id> (installed)`.
-- `POST /api/v3/config/main` answers a JSON body that does not parse with
-  400 `Invalid JSON in request body`, as `/config/raw/main` does, and an
-  empty JSON body with 400 `No data provided`. Both were a 500
-  `CONFIG_SAVE_FAILED` suggesting file permissions and disk space, with a
-  traceback logged at ERROR: `get_json()` raised inside the handler's
-  catch-all.
-- Fonts restored from a backup show up in the Fonts tab and the font
-  pickers straight away. The font catalog is cached for five minutes, and
-  upload and delete cleared it but a restore did not.
 - A game that goes live now takes over the panel within about a second.
   Live priority was only checked between screens, so a game that went live
   during a 30 s screen waited for that screen to end. The frame loops and the
@@ -722,45 +635,6 @@ policies are unchanged.
   screen showed first and the game came after it. Each check also asks each
   plugin `has_live_content()` once, where a plugin registered under several
   modes used to be asked once per mode.
-- A plugin action whose params hold `true`, `false` or `null` runs again.
-  `/api/v3/plugins/action` wrote the params into the source of the wrapper
-  that runs the plugin's script, and those JSON words are not Python, so the
-  wrapper stopped with a NameError and the action answered "Action failed".
-  The plugin file manager's category toggle sends `"enabled": true`, so
-  turning a category on or off in of-the-day always failed. The params now
-  reach the wrapper on its stdin; the script still receives them as JSON on
-  its own stdin, as before.
-- An on-demand request that `/api/v3/display/on-demand/start` refuses no
-  longer runs later. With the display stopped the request goes to the
-  display's mailbox, and the display reads that mailbox for an hour without
-  looking at a request's age. So with "Start display service" unticked, the
-  answer was "Display service is not running", yet the next time the
-  display was started it ran that plugin, pinned if the request said so.
-  The same happened after "Failed to start display service". On either
-  refusal the route now takes its request back out of the mailbox, unless a
-  newer one has replaced it. A request the display acknowledges over the
-  control socket is now a success whatever systemd reports: a display run
-  by hand or in the emulator was told "not running" for a request it had
-  already taken, and with "Start display service" ticked the route tried to
-  start the service beside it.
-- `/api/v3/plugins/operation/<id>` reports a queued operation as `pending`
-  instead of answering 500. The queue keeps an operation's callback among
-  its parameters until it runs, and the status route tried to send that
-  function as JSON. An install queued behind another plugin's install
-  failed every status poll until the first one finished. Parameters whose
-  name starts with `_` are internal and are no longer in the answer.
-- A second click on Install while that plugin is still installing, or an
-  Uninstall during its install, now answers 409 "already has an install,
-  update or uninstall in progress" instead of 500 "An error occurred". The
-  first operation carried on either way. The uninstall route also stopped
-  recording a failed uninstall in the operation history for an uninstall
-  that never started.
-- `/api/v3/plugins/<plugin_id>/static/<path>` serves images and other
-  binary files. It opened every file as UTF-8 text, so a plugin's icon or
-  preview image answered 500 `UnicodeDecodeError`. Files are now sent as
-  they are on disk, an image with its own content type; HTML, JavaScript,
-  CSS, JSON and other text keep the types they had. The path checks are
-  unchanged.
 - The display schedule turns the panel off at exactly the end time. A window
   now runs from its start time up to, but not including, its end time: with
   07:00-23:00 the panel is on at 07:00 and off at 23:00. Before, the end
@@ -768,31 +642,6 @@ policies are unchanged.
   the panel went off at 23:00 or at 23:01 depending on when in the minute
   that check ran. Windows that cross midnight and per-day schedules follow
   the same rule, and so does the dim schedule.
-- The MQTT bridge settings on the Tools tab can save a broker password with
-  TLS off. The server refuses that unless `allow_insecure_mqtt` is set, and
-  the form had no way to set it, so a password-protected broker on a home
-  network without TLS could not be saved from the web UI, and once such a
-  password was stored every later save failed too. While "Use TLS" is
-  unchecked the form now shows "Allow without TLS (trusted network)",
-  prefilled from the saved settings. It is off until ticked, so the server
-  still refuses a cleartext password by default.
-- The Overview's plugin-config warning check stops polling. It asked
-  `/api/v3/plugins/reconciliation-status` every 2 s until startup
-  reconciliation reported done, and the route reports not done whenever its
-  status file is missing: reconciliation raised before writing it, or /tmp
-  was cleaned under a long-running web service. The page then sent that
-  request every 2 s for as long as it stayed open, whichever tab was showing.
-  It now gives up after a minute and only polls while the Overview is on
-  screen.
-- Moving the Brightness slider on the Display tab no longer throws an error
-  in the browser console on every step. Its handler also updated a "LED
-  brightness" line that was removed from the page in #387; the lookup is
-  gone.
-- Creating an API token on the General tab no longer leaves the page asking
-  "Leave site?" on reload. The unsaved-changes guard marks a form when you
-  type in it and clears the mark only after an htmx save, and the token form
-  saves with a plain request, so it stayed marked after the token was
-  created. It is cleared once the token is saved.
 - An on-demand session that ends during scheduled-off hours, by expiring or
   being stopped, blanks the panel within about a second. It used to stay on
   until the next minute, because the once-a-minute schedule check had
@@ -885,25 +734,6 @@ policies are unchanged.
   a runtime publisher that stops still goes `stale`, and a subscription that
   goes quiet still falls back to the cache. The cache path's 120 s rule is
   unchanged.
-- A plugin that pauses the Vegas scroll gets its pause when its display
-  duration is not a plain number. Several plugins (clock-simple, calendar,
-  countdown) return `display_duration` as it is in config.json, so a value
-  saved as `"20"` or `null` (the raw config editor, a hand edit) reached the
-  pause as a string or None; comparing it with the clock raised, and the
-  plugin flashed up and the scroll went straight on, at every one of its
-  turns. `inf` held the pause until something interrupted it, and 0, a
-  negative number or NaN ended it at once. The pause now reads the duration
-  as the rotation does (`finite_seconds()` in `base_plugin`): a numeric
-  string counts, anything else that is not a finite number (or a
-  `get_display_duration()` that raises) pauses for 30 s, and a number at or
-  below zero for 15 s, with one warning per plugin.
-- Reinstalling Weather, Music, Stocks or Leaderboard from the Plugin Store
-  while it is enabled asks for a display restart, as reinstalling any other
-  enabled plugin does. `POST /api/v3/plugins/install` looked for the
-  plugin's `enabled` flag under the store id (`weather`), but its config
-  section is under the id its manifest declares (`ledmatrix-weather`), so
-  `restart_required` was always false and the display kept running the
-  copy it had loaded. The check now uses the installed id.
 
 ### Scrolling
 

src/background_data_service.py

@@ -34,6 +34,7 @@ from src.common.fetch_service import (
     plugin_scope,
     share_connection_pool,
 )
+from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
 from src.common.espn_dates import (
     RANGE_RETRY_SECONDS,
     _note_range_rejected,
@@ -83,6 +84,10 @@ class FetchRequest:
     # the cache with the callbacks suppressed -- joiners waiting forever for a
     # fetch that did, in fact, succeed.
     commit_claimed: bool = False
+    # Trim an ESPN scoreboard response before it is cached and delivered
+    # (src/common/espn_payload.py). Set by whoever created the request; a
+    # submitter that joins the fetch gets the same payload.
+    slim_payload: bool = True
     result: Optional[Any] = None
     error: Optional[str] = None
     # The plugin that submitted the request, so the fetch service counts the
@@ -249,7 +254,8 @@ class BackgroundDataService:
                            timeout: Optional[int] = None,
                            max_retries: int = 3,
                            priority: int = 1,
-                           callback: Optional[Callable] = None) -> str:
+                           callback: Optional[Callable] = None,
+                           slim_payload: bool = True) -> str:
         """
         Submit a background fetch request.
         
@@ -265,6 +271,11 @@ class BackgroundDataService:
             priority: Accepted for compatibility and ignored; requests run in
                 submission order.
             callback: Optional callback function when request completes
+            slim_payload: Drop the parts of an ESPN scoreboard response no
+                scoreboard reads (stat leaders, athlete cards, links,
+                headlines, highlights) before caching it; see
+                src/common/espn_payload.py. Only ESPN /scoreboard URLs are
+                touched. Pass False to cache the response whole.
             
         Returns:
             Request ID for tracking the fetch operation
@@ -336,6 +347,7 @@ class BackgroundDataService:
             priority=priority,
             callback=callback,
             owner=owner,
+            slim_payload=slim_payload,
         )
         
         with self._lock:
@@ -497,6 +509,13 @@ class BackgroundDataService:
                 )
                 return result
 
+            # Most of an ESPN scoreboard response is never drawn, and the
+            # cached copy stays parsed in the memory tier while it is fresh.
+            # Trimmed before the write so the cache, request.result and the
+            # callbacks all see the same payload. See src/common/espn_payload.py.
+            if request.slim_payload and is_espn_scoreboard_url(request.url):
+                slim_scoreboard_payload(data)
+
             # Cache the data
             self.cache_manager.set(request.cache_key, data)
             

src/backup_manager.py

@@ -213,9 +213,8 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
     The plugins are the ``manifest.json`` files in the configured plugin
     directory (see :func:`_plugins_directory`), with the manifest's version;
     ``enabled`` is config.json's flag by the display's rule (a missing flag
-    is disabled). A restore installs each listed plugin that is missing and
-    takes enabled state from the restored config.json, so ``enabled`` is
-    informational.
+    is disabled). A restore reinstalls every listed plugin and takes enabled
+    state from the restored config.json, so ``enabled`` is informational.
 
     ``data/plugin_state.json`` is not read: it only ever repeated config's
     enabled flags and the manifests' versions, and is retired (nothing

src/common/espn_payload.py

@@ -0,0 +1,97 @@
+"""Drop the parts of an ESPN scoreboard payload no scoreboard reads.
+
+The sports scoreboards cache their Recent/Upcoming window (14 days back, 7
+ahead) as the raw ESPN response, and that record stays parsed in the memory
+cache for as long as it is fresh. Most of it is never drawn. Measured on hdpi
+(2026-10-02) the MLB window was 3.35MB of JSON and 13.5MB of Python objects,
+and the five windows together ~40MB, mostly in:
+
+* ``competitors[].leaders`` / ``competitions[].leaders`` -- per-team and
+  per-game stat leaders (28% of the MLB window)
+* ``competitors[].team.links`` / ``event.links`` -- web and app URLs
+* ``status.featuredAthletes`` and ``competitors[].probables`` -- athlete
+  cards with headshots and season stats
+* ``competitions[].headlines`` / ``highlights`` -- article and video blurbs
+  (28% of the college-football window)
+* ``competitions[].geoBroadcasts``
+
+None of those keys is read by core or by any plugin in ledmatrix-plugins
+(checked 2026-10-02 across every scoreboard, the odds ticker and the
+leaderboard), while everything that is read -- odds, records, linescores,
+situation, statistics, notes, broadcasts, venue -- is kept. Dropping them
+takes the five windows from ~40MB to ~12MB of parsed objects and the files from
+10.6MB to 3.0MB, so the reads that parse an expired window on the render
+thread get 3-4x cheaper too.
+
+:func:`slim_scoreboard_payload` changes the payload in place, and only ever
+removes the keys listed here: anything it does not know about is left alone.
+"""
+
+from typing import Any, Dict
+from urllib.parse import urlsplit
+
+# Per level of the payload, the keys removed. Kept deliberately explicit:
+# adding a key here means checking that nothing reads it first.
+_EVENT_DROP = ("links",)
+_COMPETITION_DROP = ("leaders", "headlines", "highlights", "geoBroadcasts")
+_STATUS_DROP = ("featuredAthletes",)
+_COMPETITOR_DROP = ("leaders", "probables")
+_TEAM_DROP = ("links",)
+
+
+def is_espn_scoreboard_url(url: Any) -> bool:
+    """Whether ``url`` is an ESPN site-API scoreboard endpoint."""
+    if not isinstance(url, str):
+        return False
+    try:
+        parts = urlsplit(url)
+    except ValueError:
+        return False
+    host = (parts.hostname or "").lower()
+    if host != "espn.com" and not host.endswith(".espn.com"):
+        return False
+    return parts.path.rstrip("/").endswith("/scoreboard")
+
+
+def _drop(obj: Any, keys) -> None:
+    if isinstance(obj, dict):
+        for key in keys:
+            obj.pop(key, None)
+
+
+def slim_scoreboard_payload(payload: Any) -> Any:
+    """Remove the unread parts of an ESPN scoreboard payload, in place.
+
+    Returns ``payload`` for convenience. Anything that is not shaped like a
+    scoreboard (not a dict, no ``events`` list, odd entries) is passed over
+    untouched rather than raising.
+    """
+    if not isinstance(payload, dict):
+        return payload
+    events = payload.get("events")
+    if not isinstance(events, list):
+        return payload
+    for event in events:
+        if not isinstance(event, dict):
+            continue
+        _drop(event, _EVENT_DROP)
+        competitions = event.get("competitions")
+        if not isinstance(competitions, list):
+            continue
+        for competition in competitions:
+            if not isinstance(competition, dict):
+                continue
+            _drop(competition, _COMPETITION_DROP)
+            _drop(competition.get("status"), _STATUS_DROP)
+            competitors = competition.get("competitors")
+            if not isinstance(competitors, list):
+                continue
+            for competitor in competitors:
+                if not isinstance(competitor, dict):
+                    continue
+                _drop(competitor, _COMPETITOR_DROP)
+                _drop(competitor.get("team"), _TEAM_DROP)
+    return payload
+
+
+__all__ = ["is_espn_scoreboard_url", "slim_scoreboard_payload"]

src/config_manager.py

@@ -46,35 +46,6 @@ from src.common.permission_utils import (
     get_config_dir_mode
 )
 
-
-def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
-    """A deep copy of ``config`` that shares nothing with it.
-
-    load_config() hands one out per call, and the saves keep one, so the
-    cached config is never an object a caller holds. A web handler edits what
-    it loaded, validates, and may refuse the save; when the cache was that
-    same object, the refused edit stayed in it, and the next save of any
-    other setting wrote it to config.json -- a nested secret included, in
-    plain text, since it had never reached config_secrets.json to be
-    stripped.
-
-    The config is JSON data, so only its dicts and lists need copying; every
-    other value in it is immutable. On a Pi 4 with a real 60 KiB config this
-    takes 2.1 ms against copy.deepcopy's 6.8 ms, on a path ~30 handlers call
-    (a pickle round trip is no faster, 1.9 ms, and brings pickle into the
-    config path for nothing).
-    """
-    return _copy_containers(config)
-
-
-def _copy_containers(value: Any) -> Any:
-    if isinstance(value, dict):
-        return {key: _copy_containers(item) for key, item in value.items()}
-    if isinstance(value, list):
-        return [_copy_containers(item) for item in value]
-    return value
-
-
 class ConfigManager:
     """
     Reads and writes the main application configuration files.
@@ -155,10 +126,9 @@ class ConfigManager:
             validate_after_write=validate_after_write
         )
         
-        # Update in-memory config if save was successful. A copy: the caller
-        # still holds new_config_data (see _private_copy).
+        # Update in-memory config if save was successful
         if result.status == SaveResultStatus.SUCCESS:
-            self.config = _private_copy(new_config_data)
+            self.config = new_config_data
             # In-memory config now matches what was just written, so the
             # load_config fast path may return it. It still carries the
             # merged secrets that were stripped on disk; that matches a full
@@ -238,16 +208,14 @@ class ConfigManager:
 
         Fast path: when config.json, config_secrets.json and the template
         are all unchanged since the last successful load (mtime_ns + size),
-        a copy of the already-parsed self.config is returned without
-        touching the files.
-
-        Either way the caller gets its own copy (see _private_copy): editing
-        it changes nothing here until it is saved.
+        the already-parsed self.config is returned without touching the
+        files — same aliasing semantics as the full path, which also
+        returns self.config.
         """
         try:
             current_sig = self._files_signature()
             if self.config and self._loaded_sig == current_sig:
-                return _private_copy(self.config)
+                return self.config
 
             # Check if config file exists, if not create from template
             if not os.path.exists(self.config_path):
@@ -281,8 +249,8 @@ class ConfigManager:
             # Signature taken AFTER load + migration (migration may write the
             # config back), so it reflects exactly what was read/written.
             self._loaded_sig = self._files_signature()
-            return _private_copy(self.config)
-
+            return self.config
+            
         except FileNotFoundError as e:
             # Only config.json can get here: a missing or unreadable secrets
             # file is handled where it is read.
@@ -387,9 +355,8 @@ class ConfigManager:
         try:
             atomic_write_json(self.config_path, config_to_write)
 
-            # Update the in-memory config to the new state (which includes
-            # secrets for runtime), as a copy -- see _private_copy
-            self.config = _private_copy(new_config_data)
+            # Update the in-memory config to the new state (which includes secrets for runtime)
+            self.config = new_config_data
             self._loaded_sig = self._files_signature()
             self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
             if secrets_content:

src/display_controller.py

@@ -25,6 +25,7 @@ import os
 import inspect
 import signal
 import json
+import math
 import threading
 import types
 from collections import deque
@@ -56,7 +57,6 @@ from src.ipc.contract import (
     PluginReloadResult,
 )
 from src.ipc.server import ControlServer, QueuedCommand, StateHub, start_control_server
-from src.plugin_system.base_plugin import finite_seconds
 from src.vegas_mode.render_pipeline import SYNC_SEND_INTERVAL
 
 # Get logger with consistent configuration
@@ -90,6 +90,19 @@ _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS = 2.0
 DEFAULT_DYNAMIC_DURATION_CAP = 180.0
 
 
+def _finite_seconds(value: Any) -> Optional[float]:
+    """``value`` as seconds when it is a finite number or a numeric string,
+    else None. A bool is not a number here, though it is an int: True would
+    read as a one-second screen."""
+    if isinstance(value, bool):
+        return None
+    try:
+        seconds = float(value)
+    except (TypeError, ValueError, OverflowError):
+        return None
+    return seconds if math.isfinite(seconds) else None
+
+
 class _PluginReloadJob:
     """A ``plugin.reload`` whose slow half runs off the render thread.
 
@@ -1379,7 +1392,7 @@ class DisplayController:
         except Exception as err:  # pylint: disable=broad-except
             problem = f"get_display_duration() raised {type(err).__name__}: {err}"
         else:
-            seconds = finite_seconds(value)
+            seconds = _finite_seconds(value)
             if seconds is not None:
                 return seconds
             problem = f"display duration {value!r} is not a number"

src/plugin_system/base_plugin.py

@@ -11,7 +11,6 @@ Stability: Stable - maintains backward compatibility
 from abc import ABC, abstractmethod
 from enum import Enum
 from typing import Dict, Any, Optional, List
-import math
 import os
 import sys
 from src.deprecation import deprecated, warn_deprecated
@@ -241,26 +240,6 @@ def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) ->
     return legacy_vegas_participation(plugin)
 
 
-def finite_seconds(value: Any) -> Optional[float]:
-    """``value`` as seconds when it is a finite number or a numeric string,
-    else None. A bool is not a number here, though it is an int: True would
-    read as a one-second screen.
-
-    How the core reads a plugin's get_display_duration() -- the rotation
-    (DisplayController._get_display_duration) and the Vegas static pause --
-    which several plugins answer straight from config.json, so a value saved
-    as "20" or null arrives as a string or None. A number at or below zero is
-    returned as it is; each caller has its own rule for that.
-    """
-    if isinstance(value, bool):
-        return None
-    try:
-        seconds = float(value)
-    except (TypeError, ValueError, OverflowError):
-        return None
-    return seconds if math.isfinite(seconds) else None
-
-
 class BasePlugin(ABC):
     """
     Base class that all plugins must inherit from.

src/plugin_system/operation_types.py

@@ -48,19 +48,12 @@ class PluginOperation:
     completed_at: Optional[datetime] = None
     
     def to_dict(self) -> Dict[str, Any]:
-        """Convert operation to dictionary for serialization.
-
-        Parameters whose name starts with ``_`` are internal and left out:
-        PluginOperationQueue keeps the operation's callback there as
-        ``_callback`` until its worker runs it, and a pending operation's
-        status answered 500 because that function cannot be serialized.
-        """
+        """Convert operation to dictionary for serialization."""
         return {
             'operation_id': self.operation_id,
             'operation_type': self.operation_type.value,
             'plugin_id': self.plugin_id,
-            'parameters': {key: value for key, value in self.parameters.items()
-                           if not str(key).startswith('_')},
+            'parameters': self.parameters,
             'status': self.status.value,
             'progress': self.progress,
             'message': self.message,

src/vegas_mode/coordinator.py

@@ -18,11 +18,10 @@ import math
 import sys
 import time
 import threading
-from typing import Optional, Dict, Any, FrozenSet, List, Callable, TYPE_CHECKING
+from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
 
 from src import display_watchdog
 from src.common import render_gate
-from src.plugin_system.base_plugin import finite_seconds
 from src.vegas_mode.config import VegasModeConfig
 from src.vegas_mode.elements import LiveEpochs
 from src.vegas_mode.plugin_adapter import PluginAdapter
@@ -54,14 +53,6 @@ _FPS_HEARTBEAT_INTERVAL = 300.0
 #: every plugin. Game state doesn't change within a quarter second.
 _LIVE_PRIORITY_CHECK_INTERVAL = 0.25
 
-#: Seconds a static pause shows a plugin whose display duration can't be
-#: used, as long as the rotation shows it: 30 when get_display_duration()
-#: raises or answers something that is not a number
-#: (DisplayController._get_display_duration), 15 when it answers a number at
-#: or below zero (DisplayController._resolve_durations).
-_UNREADABLE_DURATION = 30.0
-_NOT_POSITIVE_DURATION = 15.0
-
 
 def _percentile(ordered: List[float], fraction: float) -> float:
     """Nearest-rank percentile of an already-sorted list.
@@ -101,9 +92,6 @@ class VegasModeCoordinator:
     _live_reason: Optional[str] = None
     # Set only while Vegas has changed the GIL switch interval; read with getattr.
     _saved_switch_interval: Optional[float]
-    #: Plugins already warned about a display duration the pause can't use,
-    #: so a bad setting logs once, not at every turn. Replaced, not mutated.
-    _duration_warned: FrozenSet[str] = frozenset()
 
     def __init__(
         self,
@@ -1022,7 +1010,7 @@ class VegasModeCoordinator:
             # Wait for the plugin's display duration. Monotonic, like the
             # iteration clock: an NTP step on an RTC-less Pi would otherwise
             # end the pause at once or stretch it by the correction.
-            duration = self._static_pause_duration(plugin)
+            duration = plugin.get_display_duration()
             start = time.monotonic()
 
             while time.monotonic() - start < duration:
@@ -1058,42 +1046,6 @@ class VegasModeCoordinator:
 
         return True
 
-    def _static_pause_duration(self, plugin: 'BasePlugin') -> float:
-        """Seconds a static pause shows ``plugin``: its display duration,
-        read the way the rotation reads it.
-
-        Several plugins return their display_duration setting straight from
-        config.json, so one saved as "20" or null came back as a string or
-        None; comparing it with the clock raised, and the pause's broad
-        except ended the pause at every one of the plugin's turns. inf
-        paused until something interrupted it, and NaN, False, 0 or a
-        negative number ended the pause at once. A numeric string counts
-        (finite_seconds); anything else, or a raise, gets
-        _UNREADABLE_DURATION, and a number at or below zero
-        _NOT_POSITIVE_DURATION, logged once per plugin.
-        """
-        try:
-            value = plugin.get_display_duration()
-        except Exception as err:  # pylint: disable=broad-except
-            problem = f"get_display_duration() raised {type(err).__name__}: {err}"
-            seconds = _UNREADABLE_DURATION
-        else:
-            seconds = finite_seconds(value)
-            if seconds is not None and seconds > 0:
-                return seconds
-            if seconds is None:
-                problem = f"display duration {value!r} is not a number"
-                seconds = _UNREADABLE_DURATION
-            else:
-                problem = f"display duration {value!r} is not above zero"
-                seconds = _NOT_POSITIVE_DURATION
-        plugin_id = plugin.plugin_id
-        if plugin_id not in self._duration_warned:
-            self._duration_warned = self._duration_warned | {plugin_id}
-            logger.warning("[%s] %s; its static pause lasts %.0fs (logged once)",
-                           plugin_id, problem, seconds)
-        return seconds
-
     def _end_static_pause(self) -> None:
         """End static pause and restore scroll state."""
         should_resume_scrolling = False

test/js/README.md

@@ -50,7 +50,6 @@ server has none.
 | `unit/test_store_categories.js` | no | The store's category filter (sandbox): the template ships only All Categories, the rest come from the store's plugins (one per category whatever its case), choosing one filters to it, and a swapped-in select is refilled from the cache keeping the choice |
 | `unit/test_github_url_install.js` | no | Install Single Plugin (sandbox, the button as `plugins.html` ships it): no inline `onclick`, so a click or Enter sends exactly one `install-from-url` request and raises no error |
 | `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
-| `unit/test_plugin_order_list.js` | no | `widgets/plugin-order-list.js` (the Vegas and rotation order lists): a disabled plugin, which gets no row, keeps its slot in the saved order and its Vegas exclusion when the list rewrites its hidden inputs, around reordering and include/exclude; an uninstalled plugin's id is dropped, a failed plugin list leaves the inputs as saved, and only string ids are carried over, once each |
 | `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
 | `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
 | `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
@@ -58,9 +57,6 @@ server has none.
 | `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
 | `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
 | `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
-| `unit/test_overview_reconciliation_poll.js` | no | The Overview's reconciliation-banner poll from `partials/overview.html`, run in a vm: it gives up after a bounded number of requests when the status never says done, runs only while the Overview is on screen (`LEDVisibility`, its own key), and stops once the banner is shown |
-| `unit/test_display_partial_ids.js` | no | `partials/display.html`: every literal `getElementById()` in its inline scripts names an id the partial renders, and moving the brightness slider (the shipped script, in a vm with a fake DOM) updates its label without throwing |
-| `unit/test_general_web_login_token.js` | no | `window.webLogin.createToken` from `partials/general.html`, run in a vm: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
 | `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
 | `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
 | `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |

test/js/dom/test_durations_page.js

@@ -98,11 +98,7 @@ const ok = (l, c, x) => c ? (pass++, console.log('  ok   ' + l))
 
   const lists = () => requests.filter(r => r.url === '/api/v3/plugins/installed').length;
   const $ = id => doc.getElementById(id);
-  // The rows' ids, in order. The input also keeps saved ids that have no row
-  // (a disabled plugin's place, see test/js/unit/test_plugin_order_list.js),
-  // and the saved order comes from whatever config the server has.
-  const SHOWN = plugins.filter(p => p.enabled).map(p => p.id);
-  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]').filter(id => SHOWN.includes(id));
+  const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]');
   async function swap() {
     panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
     panel.innerHTML = partial;

test/js/dom/test_tools_sections.js

@@ -98,50 +98,8 @@ const get = p => new Promise((res, rej) =>
   window.saveMqttBridge();
   await tick(150);
   ok('save includes password once typed', sent && sent.mqtt_password === 'typed-secret');
-
-  // A password with TLS off is refused unless allow_insecure_mqtt is set
-  // (CWE-319, api_v3/misc.py). The form has to be able to send it, or a
-  // plain-LAN broker with a password can never be saved from here.
-  const allowRow = () => $('mqtt-allow-insecure-row');
-  const shown = el => !!el && !el.classList.contains('hidden');
-  ok('allow-without-TLS control rendered', !!$('mqtt-allow-insecure'));
-  ok('allow-without-TLS starts as saved',
-     !!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === !!bridge.data.config.allow_insecure_mqtt);
-  ok('allow-without-TLS shown only while TLS is off',
-     shown(allowRow()) === !$('mqtt-tls').checked);
-  $('mqtt-tls').checked = true;
-  $('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
-  ok('ticking TLS hides it', !shown(allowRow()));
-  $('mqtt-tls').checked = false;
-  $('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
-  ok('unticking TLS shows it again', shown(allowRow()));
-
-  const setAllow = v => { if ($('mqtt-allow-insecure')) $('mqtt-allow-insecure').checked = v; };
-  setAllow(false);
-  window.saveMqttBridge();
-  await tick(150);
-  ok('save sends allow_insecure_mqtt false when unticked', !!sent && sent.allow_insecure_mqtt === false, sent);
-  setAllow(true);
-  window.saveMqttBridge();
-  await tick(150);
-  ok('save sends allow_insecure_mqtt true when ticked', !!sent && sent.allow_insecure_mqtt === true, sent);
   onPut = null;
 
-  // Prefilled from the saved settings, and hidden while TLS is saved on.
-  bridgePayload = JSON.parse(JSON.stringify(bridge));
-  bridgePayload.data.config.allow_insecure_mqtt = true;
-  bridgePayload.data.config.mqtt_tls = false;
-  window.loadMqttBridge();
-  await tick(150);
-  ok('a saved opt-in is prefilled', !!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === true);
-  bridgePayload.data.config.mqtt_tls = true;
-  window.loadMqttBridge();
-  await tick(150);
-  ok('hidden on load when TLS is saved on', !shown(allowRow()));
-  bridgePayload = bridge;
-  window.loadMqttBridge();
-  await tick(150);
-
   // ── Pixlet editor, idle ────────────────────────────────────────────────
   const appIds = (apps.data.apps || []).map(a => a.id);
   ok('editor lists the apps on disk',

test/js/run_all.js

@@ -17,7 +17,6 @@ const fs = require('fs');
 
 const BASE = process.env.BASE || 'http://localhost:5000';
 const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
-              'unit/test_plugin_order_list.js',
               'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
               'unit/test_style_editor_layout_leaf_columns.js',
               'unit/test_style_editor_layout_leaf_collision.js',
@@ -29,10 +28,7 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
               'unit/test_inline_handler_escaping.js',
               'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js',
               'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js',
-              'unit/test_page_registry.js', 'unit/test_core_modules.js',
-              'unit/test_overview_reconciliation_poll.js',
-              'unit/test_display_partial_ids.js',
-              'unit/test_general_web_login_token.js'];
+              'unit/test_page_registry.js', 'unit/test_core_modules.js'];
 const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
              'dom/test_tools_sections.js', 'dom/test_cache_page.js',
              'dom/test_durations_page.js', 'dom/test_operation_history_page.js',

test/js/unit/test_display_partial_ids.js

@@ -1,108 +0,0 @@
-// The Display tab's inline script must only look up elements the partial
-// renders.
-//
-// Its brightness slider handler also wrote to #brightness-display, a "LED
-// brightness: N%" line that #387 removed from partials/display.html. The
-// lookup returned null, so every movement of the slider threw a TypeError.
-// This checks every literal getElementById() in the partial's inline scripts
-// against the ids its markup renders, and runs the shipped script in a vm
-// with a fake DOM (null for an id the markup lacks, as in a browser) to move
-// the slider.
-//
-// No jsdom and no server needed.
-
-const fs = require('fs');
-const path = require('path');
-const vm = require('vm');
-
-const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/display.html');
-
-let pass = 0, fail = 0;
-const ok = (label, cond, extra) => cond
-  ? (pass++, console.log('  ok   ' + label))
-  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
-
-const html = fs.readFileSync(PARTIAL, 'utf8');
-const blocks = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)];
-const scripts = blocks.map(m => m[1]);
-// The markup is what lies between the script blocks (sliced around them, not
-// a replace(), which CodeQL reads as an incomplete HTML sanitizer).
-let markup = '';
-let from = 0;
-for (const m of blocks) {
-  markup += html.slice(from, m.index);
-  from = m.index + m[0].length;
-}
-markup += html.slice(from);
-const rendered = new Set([...markup.matchAll(/\bid="([^"{}]+)"/g)].map(m => m[1]));
-
-console.log('\n── Display partial: element lookups ──');
-
-// 1. Static: every literal lookup names an id the partial renders.
-const lookups = scripts.flatMap(s => [...s.matchAll(/getElementById\('([^']+)'\)/g)].map(m => m[1]));
-const missing = [...new Set(lookups.filter(id => !rendered.has(id)))];
-ok('the inline scripts look elements up', lookups.length > 0, lookups.length);
-ok('every looked-up id is rendered by the partial', missing.length === 0, missing);
-
-// 2. Behaviour: moving the brightness slider updates its label and throws nothing.
-function fakeElement(id) {
-  const listeners = {};
-  const classes = new Set();
-  return {
-    id, value: '', textContent: '', min: '', max: '', checked: false,
-    style: {}, dataset: {}, className: '',
-    classList: {
-      add: c => classes.add(c), remove: c => classes.delete(c),
-      toggle: (c, on) => (on === undefined ? (classes.has(c) ? classes.delete(c) : classes.add(c)) : (on ? classes.add(c) : classes.delete(c))),
-      contains: c => classes.has(c),
-    },
-    addEventListener: (type, fn) => { (listeners[type] ||= []).push(fn); },
-    dispatchEvent() { return true; },
-    appendChild() {},
-    listeners,
-  };
-}
-
-const main = scripts.find(s => s.includes("getElementById('brightness')"));
-ok('found the script that wires the brightness slider', !!main);
-if (main) {
-  const elements = new Map();
-  const document = {
-    readyState: 'complete',
-    hidden: false,
-    getElementById: id => {
-      if (!rendered.has(id)) return null;
-      if (!elements.has(id)) elements.set(id, fakeElement(id));
-      return elements.get(id);
-    },
-    createElement: () => fakeElement(''),
-    createTextNode: () => ({}),
-    addEventListener() {},
-  };
-  const window = {
-    LEDEscape: { html: v => String(v), attr: v => String(v) },
-    LEDVisibility: { onActive() {} },
-  };
-  const context = {
-    window, document, console, URLSearchParams,
-    fetch: () => new Promise(() => {}),
-    setTimeout: () => 0, clearTimeout() {}, setInterval: () => 0, clearInterval() {},
-  };
-  vm.createContext(context);
-  let loadError = null;
-  try { vm.runInContext(main, context); } catch (e) { loadError = e; }
-  ok('the script loads', !loadError, loadError && String(loadError));
-
-  const slider = elements.get('brightness');
-  const handlers = (slider && slider.listeners.input) || [];
-  ok('the slider has an input handler', handlers.length > 0);
-  let thrown = null;
-  slider.value = '42';
-  try { handlers.forEach(fn => fn.call(slider, { target: slider })); } catch (e) { thrown = e; }
-  ok('moving the slider throws nothing', !thrown, thrown && String(thrown));
-  ok('...and shows the new value', elements.get('brightness-value').textContent === '42',
-     elements.get('brightness-value').textContent);
-}
-
-console.log(`\n${pass} passed, ${fail} failed\n`);
-process.exit(fail ? 1 : 0);

test/js/unit/test_general_web_login_token.js

@@ -1,107 +0,0 @@
-// Creating an API token on the General tab must leave its form clean.
-//
-// app.js marks a form data-dirty on any input in it and clears the mark only
-// after a successful htmx request; its beforeunload handler then asks "Leave
-// site?" while any visible form is still dirty. The token form posts with
-// fetch (window.webLogin.createToken in partials/general.html), so after a
-// token was created the form stayed dirty and reloading the page while the
-// General tab was open prompted about changes that had been saved.
-//
-// Runs the shipped inline script in a vm with a fake fetch and DOM -- no jsdom
-// and no server needed.
-
-const fs = require('fs');
-const path = require('path');
-const vm = require('vm');
-
-const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/general.html');
-
-let pass = 0, fail = 0;
-const ok = (label, cond, extra) => cond
-  ? (pass++, console.log('  ok   ' + label))
-  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
-
-function webLoginScript() {
-  const html = fs.readFileSync(PARTIAL, 'utf8');
-  const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
-  const found = scripts.find(s => s.includes('window.webLogin = {'));
-  if (!found) throw new Error('webLogin script not found in general.html');
-  return found;
-}
-
-function el() {
-  const classes = new Set(['hidden']);
-  return {
-    textContent: '', dataset: {}, style: {}, className: '',
-    classList: { add: c => classes.add(c), remove: c => classes.delete(c), contains: c => classes.has(c) },
-    appendChild() {}, addEventListener() {}, querySelector: () => null,
-  };
-}
-
-function load(answer) {
-  const elements = {
-    'web-login-tokens': el(),
-    'web-login-new-token-value': el(),
-    'web-login-new-token': el(),
-  };
-  const notes = [];
-  const window = { showNotification: (m, t) => notes.push([m, t]), alert() {}, confirm: () => true };
-  const context = {
-    window, console,
-    document: {
-      getElementById: id => elements[id] || null,
-      createElement: () => el(),
-      querySelectorAll: () => [],
-    },
-    fetch: () => Promise.resolve({
-      ok: answer.ok, status: answer.ok ? 200 : 400,
-      json: () => Promise.resolve(answer.body),
-    }),
-  };
-  vm.createContext(context);
-  vm.runInContext(webLoginScript(), context);
-  return { webLogin: context.window.webLogin, elements, notes };
-}
-
-function dirtyForm() {
-  const attrs = new Map([['data-dirty', '']]);
-  return {
-    querySelector: sel => (sel === '[name="name"]' ? { value: 'Home Assistant' } : null),
-    reset() {},
-    hasAttribute: name => attrs.has(name),
-    setAttribute: (name, value) => attrs.set(name, String(value)),
-    removeAttribute: name => attrs.delete(name),
-  };
-}
-
-const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
-
-(async () => {
-  console.log('\n── General tab: API token form ──');
-
-  {
-    const t = load({ ok: true, body: {
-      status: 'success', message: 'Token created',
-      data: { token: 'lmx_secret', record: { id: 't1', name: 'Home Assistant', prefix: 'lmx_sec' } },
-    } });
-    const form = dirtyForm();
-    t.webLogin.createToken(form);
-    await flush();
-    ok('the new token is shown', t.elements['web-login-new-token-value'].textContent === 'lmx_secret');
-    ok('a created token leaves the form clean (no "Leave site?" on reload)',
-       !form.hasAttribute('data-dirty'));
-  }
-
-  {
-    const t = load({ ok: false, body: { status: 'error', message: 'Name is required' } });
-    const form = dirtyForm();
-    t.webLogin.createToken(form);
-    await flush();
-    ok('a refused request reports the error', t.notes.some(([m, type]) => type === 'error' && /Name is required/.test(m)),
-       t.notes);
-    ok('...and keeps the form dirty: nothing was saved', form.hasAttribute('data-dirty'));
-  }
-
-  console.log(`\n${pass} passed, ${fail} failed\n`);
-  process.exit(fail ? 1 : 0);
-})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });

test/js/unit/test_overview_reconciliation_poll.js

@@ -1,137 +0,0 @@
-// The Overview's "Plugin Config Warning" poll must end.
-//
-// The banner script in partials/overview.html asks
-// /api/v3/plugins/reconciliation-status every 2 s until startup reconciliation
-// says it is done. The route answers done: false whenever its status file is
-// missing -- reconciliation raised before writing it, or /tmp was cleaned
-// under a long-running web service -- so the poll used to run every 2 s for
-// as long as the page stayed open, on every tab. It now gives up after a
-// bounded number of tries and runs only while the Overview is on screen
-// (LEDVisibility, like the other partials' pollers).
-//
-// Runs the shipped inline script in a vm with fake timers, fetch and DOM --
-// no jsdom and no server needed.
-
-const fs = require('fs');
-const path = require('path');
-const vm = require('vm');
-
-const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/overview.html');
-
-let pass = 0, fail = 0;
-const ok = (label, cond, extra) => cond
-  ? (pass++, console.log('  ok   ' + label))
-  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
-
-function bannerScript() {
-  const html = fs.readFileSync(PARTIAL, 'utf8');
-  const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
-  const found = scripts.find(s => s.includes('ledmatrix-recon-dismissed'));
-  if (!found) throw new Error('reconciliation banner script not found in overview.html');
-  return found;
-}
-
-const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
-
-function load({ payload, visibility = true }) {
-  const timers = new Map();
-  let nextId = 1;
-  const calls = [];
-  const banner = { style: { setProperty() {} }, dataset: {} };
-  const text = { textContent: '' };
-  const registrations = [];
-  const window = {};
-  if (visibility) {
-    window.LEDVisibility = {
-      onActive(tab, start, stop, key) { registrations.push({ tab, start, stop, key }); start(); },
-    };
-  }
-  const context = {
-    window,
-    document: {
-      getElementById: id => ({ 'reconciliation-banner': banner, 'reconciliation-banner-text': text })[id] || null,
-    },
-    sessionStorage: { getItem: () => null, setItem() {} },
-    fetch: (url) => {
-      calls.push(url);
-      return Promise.resolve({ json: () => Promise.resolve(payload()) });
-    },
-    setTimeout: (fn) => { const id = nextId++; timers.set(id, fn); return id; },
-    clearTimeout: (id) => { timers.delete(id); },
-  };
-  vm.createContext(context);
-  vm.runInContext(bannerScript(), context);
-  const fireTimers = async () => {
-    const due = [...timers.entries()];
-    timers.clear();
-    due.forEach(([, fn]) => fn());
-    await flush();
-  };
-  return { calls, timers, registrations, banner, text, window, fireTimers };
-}
-
-(async () => {
-  console.log('\n── Overview reconciliation poll ──');
-
-  // 1. A status file that never says done: the poll stops on its own.
-  {
-    const t = load({ payload: () => ({ status: 'success', data: { done: false, unresolved: [] } }) });
-    await flush();
-    for (let i = 0; i < 200; i++) await t.fireTimers();
-    ok('a status that never turns done stops being polled', t.timers.size === 0,
-       { pending: t.timers.size, requests: t.calls.length });
-    ok('...after a bounded number of requests (at most 30, a minute at 2 s)',
-       t.calls.length > 1 && t.calls.length <= 30, t.calls.length);
-  }
-
-  // 2. Runs only while the Overview is on screen.
-  {
-    const t = load({ payload: () => ({ status: 'success', data: { done: false, unresolved: [] } }) });
-    await flush();
-    const reg = t.registrations[0];
-    ok('registers with LEDVisibility for the overview tab', !!reg && reg.tab === 'overview', reg && reg.tab);
-    ok('under its own key, so it does not replace another overview poller',
-       !!reg && !!reg.key && reg.key !== 'overview', reg && reg.key);
-    ok('first request goes out at once', t.calls.length === 1, t.calls.length);
-    if (reg) {
-      reg.stop();
-      ok('leaving the tab cancels the pending retry', t.timers.size === 0, t.timers.size);
-      for (let i = 0; i < 5; i++) await t.fireTimers();
-      ok('no requests while another tab is active', t.calls.length === 1, t.calls.length);
-      reg.start();
-      await flush();
-      ok('coming back asks again at once', t.calls.length === 2, t.calls.length);
-      ok('...and keeps polling', t.timers.size === 1, t.timers.size);
-    }
-  }
-
-  // 3. A finished reconciliation with findings shows the banner and stops.
-  {
-    let done = false;
-    const t = load({ payload: () => (done
-      ? { status: 'success', data: { done: true, unresolved: [{ plugin_id: 'clock', type: 'plugin_missing_on_disk' }] } }
-      : { status: 'success', data: { done: false, unresolved: [] } }) });
-    await flush();
-    await t.fireTimers();
-    done = true;
-    await t.fireTimers();
-    ok('the banner names the finding once reconciliation is done',
-       t.text.textContent.includes('clock'), t.text.textContent);
-    const before = t.calls.length;
-    for (let i = 0; i < 5; i++) await t.fireTimers();
-    ok('no more requests once it is done', t.calls.length === before && t.timers.size === 0,
-       { before, after: t.calls.length, pending: t.timers.size });
-  }
-
-  // 4. Without LEDVisibility (base.html always has it) it still runs, bounded.
-  {
-    const t = load({ visibility: false, payload: () => ({ status: 'success', data: { done: false } }) });
-    await flush();
-    ok('runs without LEDVisibility', t.calls.length === 1, t.calls.length);
-    for (let i = 0; i < 200; i++) await t.fireTimers();
-    ok('...and is still bounded', t.timers.size === 0 && t.calls.length <= 30, t.calls.length);
-  }
-
-  console.log(`\n${pass} passed, ${fail} failed\n`);
-  process.exit(fail ? 1 : 0);
-})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });

test/js/unit/test_plugin_order_list.js

@@ -1,189 +0,0 @@
-// The shared plugin order list (widgets/plugin-order-list.js) keeps what it
-// does not show.
-//
-// It lists enabled plugins only, and rewrites its hidden inputs from those
-// rows as soon as it has drawn them. A disabled plugin's place in the order
-// and its Vegas exclusion used to vanish from the inputs on that rewrite, so
-// any later save of the Display or Rotation & Durations tab stored them
-// without it: re-enabled, the plugin came back at the end of the rotation and
-// scrolling in Vegas again. An uninstalled plugin's id is still dropped, as
-// before, so the lists don't collect ids nothing can show. Runs the shipped
-// widget in a vm with a minimal fake DOM -- no jsdom and no server needed, so
-// it runs under test/test_js_unit_suites.py too.
-
-const fs = require('fs');
-const path = require('path');
-const vm = require('vm');
-const WIDGET = path.resolve(__dirname, '../../../web_interface/static/v3/js/widgets/plugin-order-list.js');
-
-let pass = 0, fail = 0;
-const ok = (label, cond, extra) => cond
-  ? (pass++, console.log('  ok   ' + label))
-  : (fail++, console.log('  FAIL ' + label + (extra !== undefined ? '  ' + JSON.stringify(extra) : '')));
-const same = (a, b) => JSON.stringify(a) === JSON.stringify(b);
-
-class FakeElement {
-  constructor(tag) {
-    this.tagName = tag.toUpperCase();
-    this.children = [];
-    this.parent = null;
-    this.dataset = {};
-    this.style = {};
-    this.className = '';
-    this.value = '';
-    this.checked = false;
-    this.listeners = {};
-    this._text = '';
-  }
-  appendChild(child) {
-    if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
-    child.parent = this;
-    this.children.push(child);
-    return child;
-  }
-  insertBefore(child, ref) {
-    if (!ref) return this.appendChild(child);
-    if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
-    child.parent = this;
-    this.children.splice(this.children.indexOf(ref), 0, child);
-    return child;
-  }
-  get previousElementSibling() {
-    const siblings = this.parent ? this.parent.children : [];
-    return siblings[siblings.indexOf(this) - 1] || null;
-  }
-  get nextElementSibling() {
-    const siblings = this.parent ? this.parent.children : [];
-    const i = siblings.indexOf(this);
-    return i < 0 ? null : siblings[i + 1] || null;
-  }
-  set textContent(value) { this._text = value; this.children = []; }
-  get textContent() { return this._text; }
-  setAttribute() {}
-  focus() {}
-  addEventListener(type, fn) { (this.listeners[type] ||= []).push(fn); }
-  fire(type, event) { (this.listeners[type] || []).forEach(fn => fn.call(this, event || {})); }
-  descendants() { return this.children.flatMap(c => [c, ...c.descendants()]); }
-  querySelectorAll(selector) {
-    const cls = selector.replace(/^\./, '');
-    return this.descendants().filter(e => e.className.split(/\s+/).includes(cls));
-  }
-  querySelector(selector) { return this.querySelectorAll(selector)[0] || null; }
-}
-
-/** Run the widget over `plugins` with the given saved inputs; resolves once it has drawn. */
-async function mount({ plugins, order, excluded, fetchFails }) {
-  const els = {
-    list: new FakeElement('div'),
-    order: Object.assign(new FakeElement('input'), { value: JSON.stringify(order) }),
-  };
-  if (excluded !== undefined) {
-    els.excluded = Object.assign(new FakeElement('input'), { value: JSON.stringify(excluded) });
-  }
-  const context = {
-    // The widget logs a failed list; expected there, so kept off the output.
-    console: fetchFails ? Object.assign({}, console, { error: () => {} }) : console,
-    window: {},
-    document: {
-      getElementById: (id) => els[id] || null,
-      createElement: (tag) => new FakeElement(tag),
-      createTextNode: (text) => new FakeElement('#text'),
-    },
-    fetch: () => (fetchFails ? Promise.reject(new Error('service restarting')) : Promise.resolve({
-      json: () => Promise.resolve({ status: 'success', data: { plugins } }),
-    })),
-  };
-  vm.createContext(context);
-  vm.runInContext(fs.readFileSync(WIDGET, 'utf8'), context);
-  context.window.PluginOrderList.init({
-    containerId: 'list', orderInputId: 'order',
-    excludedInputId: excluded !== undefined ? 'excluded' : undefined,
-  });
-  await new Promise(resolve => setTimeout(resolve, 0));
-  const rows = () => els.list.querySelectorAll('.plugin-order-item');
-  return {
-    rows,
-    rowIds: () => rows().map(r => r.dataset.pluginId),
-    order: () => JSON.parse(els.order.value),
-    excluded: () => JSON.parse(els.excluded.value),
-    row: (id) => rows().find(r => r.dataset.pluginId === id),
-  };
-}
-
-const PLUGINS = [
-  { id: 'weather', name: 'Weather', enabled: true },
-  { id: 'clock', name: 'Clock', enabled: false },
-  { id: 'stocks', name: 'Stocks', enabled: true },
-];
-
-(async () => {
-  console.log('\nVegas: a disabled plugin keeps its place and its exclusion');
-  {
-    const t = await mount({ plugins: PLUGINS, order: ['weather', 'clock', 'stocks'], excluded: ['clock'] });
-    ok('only enabled plugins get a row', same(t.rowIds(), ['weather', 'stocks']), t.rowIds());
-    ok('drawing the list keeps the disabled plugin in the order, in its place',
-       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
-    ok('drawing the list keeps its exclusion', same(t.excluded(), ['clock']), t.excluded());
-
-    // Move Stocks up: the rows swap, and Clock stays in its saved slot.
-    const up = t.row('stocks').querySelectorAll('.plugin-order-move')[0];
-    up.fire('click');
-    ok('reordering the rows fills the other slots in the new order',
-       same(t.order(), ['stocks', 'clock', 'weather']), t.order());
-
-    const include = t.row('weather').querySelector('.plugin-order-include');
-    include.checked = false;
-    include.fire('change');
-    ok('unchecking a row adds it, and the disabled exclusion stays',
-       same([...t.excluded()].sort(), ['clock', 'weather']), t.excluded());
-    include.checked = true;
-    include.fire('change');
-    ok('checking it again removes only that one', same(t.excluded(), ['clock']), t.excluded());
-  }
-
-  console.log('\nRotation order: the same, without exclusions');
-  {
-    const plugins = [
-      { id: 'clock', enabled: true },
-      { id: 'off', enabled: false },
-      { id: 'weather', enabled: true },
-      { id: 'new', enabled: true },
-    ];
-    const t = await mount({ plugins, order: ['clock', 'off', 'weather'] });
-    ok('the disabled plugin keeps its slot; a plugin not in the saved order goes last',
-       same(t.order(), ['clock', 'off', 'weather', 'new']), t.order());
-  }
-
-  console.log('\nAn uninstalled plugin is dropped; a failed list keeps everything');
-  {
-    const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
-                            excluded: ['gone', 'clock'] });
-    ok('the disabled plugin is kept and the uninstalled one dropped from the order',
-       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
-    ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
-  }
-  {
-    const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
-                            excluded: ['gone', 'clock'], fetchFails: true });
-    // No installed list, so nothing can be told apart: no rows, and the
-    // inputs keep what was saved, uninstalled ids included.
-    ok('a failed plugin list draws no rows', t.rowIds().length === 0, t.rowIds());
-    ok('and leaves the saved order as it was',
-       same(t.order(), ['weather', 'gone', 'clock', 'stocks']), t.order());
-    ok('and the saved exclusions', same(t.excluded(), ['gone', 'clock']), t.excluded());
-  }
-
-  console.log('\nOnly what the server would accept is carried over');
-  {
-    const t = await mount({ plugins: PLUGINS, order: ['weather', 7, 'clock', null, 'clock', 'stocks'],
-                            excluded: ['clock', 3, 'clock'] });
-    // /config/main refuses a list holding anything but strings, which would
-    // block every later Display save; a repeated id is kept once.
-    ok('non-string and repeated saved ids are dropped from the order',
-       same(t.order(), ['weather', 'clock', 'stocks']), t.order());
-    ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
-  }
-
-  console.log(`\n${pass} passed, ${fail} failed`);
-  process.exit(fail ? 1 : 0);
-})().catch(e => { console.error(e); process.exit(1); });

test/test_api_v3_install_restart_installed_id.py

@@ -1,123 +0,0 @@
-"""POST /plugins/install asks for a restart by the id the plugin installed as.
-
-A store install needs a display restart when config.json already enables the
-plugin (a reinstall, or a config carried over): the display loads a plugin
-when its ``enabled`` flag changes, and this flag did not. The route read the
-flag under the registry id it was given. An aliased entry installs under
-another id -- ``weather`` installs a directory whose manifest declares
-``ledmatrix-weather``, and its config section is ``ledmatrix-weather`` -- so
-reinstalling an enabled Weather never reported that a restart was needed,
-and the display kept running the old copy.
-"""
-
-import json
-from unittest.mock import MagicMock
-
-import pytest
-
-from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401
-
-INSTALL = "/api/v3/plugins/install"
-
-
-@pytest.fixture
-def store(api_v3_module, tmp_path):
-    """The store installs registry entry ``weather`` as ``installed_id``."""
-    manager = api_v3_module.api_v3.plugin_store_manager
-    manager.install_plugin.return_value = True
-    manager.get_registry_info.return_value = None
-    manager._find_plugin_path.return_value = None
-
-    def installs_as(installed_id):
-        path = tmp_path / installed_id
-        path.mkdir()
-        (path / "manifest.json").write_text(json.dumps({"id": installed_id}),
-                                            encoding="utf-8")
-        manager._find_plugin_path.side_effect = (
-            lambda pid: path if pid == "weather" else None)
-
-    manager.installs_as = installs_as
-    return manager
-
-
-@pytest.fixture
-def config(api_v3_module):
-    """config.json with an ``enabled`` flag for each plugin id given."""
-    def sections(enabled):
-        api_v3_module.api_v3.config_manager.load_config.return_value = {
-            plugin_id: {"enabled": flag} for plugin_id, flag in enabled.items()}
-    return sections
-
-
-@pytest.fixture
-def queued(api_v3_module):
-    queue = MagicMock()
-
-    def enqueue(operation_type, plugin_id, operation_callback=None):
-        queue.callback_result = operation_callback(MagicMock())
-        return "op-1"
-
-    queue.enqueue_operation.side_effect = enqueue
-    api_v3_module.api_v3.operation_queue = queue
-    return queue
-
-
-def _direct(client):
-    return client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
-
-
-def _queued(client, queue):
-    client.post(INSTALL, json={"plugin_id": "weather"})
-    return queue.callback_result
-
-
-class TestDirectInstall:
-    def test_an_aliased_install_enabled_under_its_installed_id_asks_for_a_restart(
-            self, api_v3_client, store, config):
-        store.installs_as("ledmatrix-weather")
-        config({"ledmatrix-weather": True})
-        body = _direct(api_v3_client)
-        assert body["status"] == "success"
-        assert body["restart_required"] is True
-        assert body["restart_message"]
-
-    def test_an_enabled_section_under_the_registry_id_alone_does_not(
-            self, api_v3_client, store, config):
-        """The display knows the plugin as ledmatrix-weather; nothing runs
-        under a section called weather."""
-        store.installs_as("ledmatrix-weather")
-        config({"weather": True})
-        assert _direct(api_v3_client)["restart_required"] is False
-
-    def test_an_aliased_install_that_is_not_enabled_needs_no_restart(
-            self, api_v3_client, store, config):
-        store.installs_as("ledmatrix-weather")
-        config({"ledmatrix-weather": False})
-        assert _direct(api_v3_client)["restart_required"] is False
-
-    def test_an_install_under_its_own_id_is_unchanged(self, api_v3_client, store, config):
-        store.installs_as("weather")
-        config({"weather": True})
-        assert _direct(api_v3_client)["restart_required"] is True
-
-    def test_an_install_that_cannot_be_found_uses_the_requested_id(
-            self, api_v3_client, store, config):
-        config({"weather": True})
-        assert _direct(api_v3_client)["restart_required"] is True
-
-
-class TestQueuedInstall:
-    def test_an_aliased_install_enabled_under_its_installed_id_asks_for_a_restart(
-            self, api_v3_client, store, config, queued):
-        store.installs_as("ledmatrix-weather")
-        config({"ledmatrix-weather": True})
-        result = _queued(api_v3_client, queued)
-        assert result["success"] is True
-        assert result["restart_required"] is True
-        assert result["restart_message"]
-
-    def test_an_enabled_section_under_the_registry_id_alone_does_not(
-            self, api_v3_client, store, config, queued):
-        store.installs_as("ledmatrix-weather")
-        config({"weather": True})
-        assert _queued(api_v3_client, queued)["restart_required"] is False

test/test_api_v3_on_demand_restart.py

@@ -37,7 +37,6 @@ from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401
 START_URL = "/api/v3/display/on-demand/start"
 STOP_URL = "/api/v3/display/on-demand/stop"
 MAILBOX = "display_on_demand_request"
-DISPLAY = "web_interface.blueprints.api_v3.display"
 
 
 @pytest.fixture
@@ -151,102 +150,6 @@ class TestStartWhileTheServiceIsStopped:
         assert response.get_json()["status"] == "error"
 
 
-class _Mailbox:
-    """The CacheManager calls the routes make, over a dict."""
-
-    def __init__(self):
-        self.entries = {}
-
-    def set(self, key, value, ttl=None):
-        self.entries[key] = value
-
-    def get(self, key, max_age=300, memory_ttl=None):
-        return self.entries.get(key)
-
-    def delete(self, key):
-        self.entries.pop(key, None)
-
-
-class TestARefusedStartLeavesNoRequestBehind:
-    """A start the route answers with an error must not run later.
-
-    The request was posted (to the mailbox, with the display stopped) before
-    the route refused it, and the display reads the mailbox for an hour
-    without looking at a request's age. So "Display service is not running"
-    (start_service off) or "Failed to start display service" left the
-    request waiting, and the next time the display started -- minutes later,
-    by hand -- it ran that plugin, pinned if the request said so.
-
-    A socket acknowledgement is the other side of it: the display answered,
-    so it is running and has the request, whatever systemd says (a display
-    run by hand or in the emulator has no active unit). That is a success,
-    not "not running", and no unit is started beside it.
-    """
-
-    @pytest.fixture
-    def mailbox(self, api_v3_module, service):
-        box = _Mailbox()
-        api_v3_module.api_v3.cache_manager = box
-        service["state"]["active"] = False
-        return box
-
-    @pytest.mark.parametrize("body", [
-        {"plugin_id": "weather", "start_service": False},
-        {"plugin_id": "weather"},                           # start_service defaults on
-    ])
-    def test_a_socket_ack_is_a_success_whatever_systemd_says(
-            self, api_v3_client, service, mailbox, body):
-        with patch(f"{DISPLAY}.control_client.on_demand_start",
-                   side_effect=lambda request_id, *a: {"accepted": True}):
-            response = api_v3_client.post(START_URL, json=body)
-        assert response.status_code == 200, response.get_json()
-        assert response.get_json()["data"]["transport"] == "socket"
-        assert MAILBOX not in mailbox.entries
-        assert _systemctl_verbs(service["systemctl"]) == [], (
-            "a unit was started beside a display that answered the socket")
-
-    def test_without_start_service_the_request_is_taken_back(
-            self, api_v3_client, service, mailbox):
-        response = api_v3_client.post(START_URL, json={
-            "plugin_id": "weather", "pinned": True, "start_service": False})
-        assert response.status_code == 400
-        assert response.get_json()["status"] == "error"
-        assert MAILBOX not in mailbox.entries
-
-    def test_a_start_that_fails_takes_its_request_back(self, api_v3_client, service, mailbox):
-        service["systemctl"].side_effect = lambda args: {
-            "returncode": 1, "stdout": "", "stderr": "denied"}
-        response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
-        assert response.status_code == 500
-        assert MAILBOX not in mailbox.entries
-
-    def test_a_newer_request_is_left_alone_on_the_400(self, api_v3_client, service, mailbox):
-        newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
-
-        def stopped_and_another_post_lands(*args):
-            mailbox.entries[MAILBOX] = newer
-            return {"active": False}
-
-        with patch(f"{DISPLAY}._get_display_service_status",
-                   side_effect=stopped_and_another_post_lands):
-            response = api_v3_client.post(START_URL, json={
-                "plugin_id": "weather", "start_service": False})
-        assert response.status_code == 400
-        assert mailbox.entries[MAILBOX] is newer
-
-    def test_a_newer_request_is_left_alone_on_the_500(self, api_v3_client, service, mailbox):
-        newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
-
-        def start_fails_after_another_post(args):
-            mailbox.entries[MAILBOX] = newer
-            return {"returncode": 1, "stdout": "", "stderr": "denied"}
-
-        service["systemctl"].side_effect = start_fails_after_another_post
-        response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
-        assert response.status_code == 500
-        assert mailbox.entries[MAILBOX] is newer
-
-
 class TestStop:
     def test_stop_posts_a_stop_request_and_leaves_the_service_running(
             self, api_v3_client, service):

test/test_api_v3_operation_status_pending.py

@@ -1,83 +0,0 @@
-"""GET /api/v3/plugins/operation/<id> answers for an operation still waiting.
-
-PluginOperationQueue keeps an operation's callback in its parameters, under
-``_callback``, until the worker takes it to run. PluginOperation.to_dict()
-returned the parameters as they were, so for a pending operation the route
-handed jsonify a function and answered 500 "A system error occurred". That
-is every poll of an install queued behind another plugin's: the second of
-two installs read as broken until the first one finished.
-"""
-
-import json
-import sys
-import threading
-from pathlib import Path
-
-import pytest
-
-sys.path.insert(0, str(Path(__file__).parent.parent))
-
-from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
-
-from src.plugin_system.operation_queue import PluginOperationQueue  # noqa: E402
-from src.plugin_system.operation_types import (  # noqa: E402
-    OperationType, PluginOperation,
-)
-
-
-def _callback(op):
-    return {"success": True, "message": "done"}
-
-
-class TestToDict:
-    def test_private_parameters_are_left_out(self):
-        op = PluginOperation(OperationType.INSTALL, "demo",
-                             parameters={"_callback": _callback, "branch": "main"})
-        assert op.to_dict()["parameters"] == {"branch": "main"}
-        json.dumps(op.to_dict())   # serializable
-
-    def test_the_operation_keeps_its_callback_for_the_worker(self):
-        op = PluginOperation(OperationType.INSTALL, "demo",
-                             parameters={"_callback": _callback})
-        op.to_dict()
-        assert op.parameters["_callback"] is _callback
-
-    def test_the_other_fields_are_unchanged(self):
-        op = PluginOperation(OperationType.UNINSTALL, "demo", operation_id="op-1")
-        assert op.to_dict() == {
-            "operation_id": "op-1", "operation_type": "uninstall", "plugin_id": "demo",
-            "parameters": {}, "status": "pending", "progress": 0.0, "message": "",
-            "error": None, "result": None,
-            "created_at": op.created_at.isoformat(), "started_at": None,
-            "completed_at": None,
-        }
-
-
-class TestTheRoute:
-    @pytest.fixture
-    def busy_queue(self, api_v3_module):
-        """A real queue whose worker is held by another plugin's operation."""
-        queue = PluginOperationQueue(max_history=10)
-        api_v3_module.api_v3.operation_queue = queue
-        started, release = threading.Event(), threading.Event()
-
-        def blocker(op):
-            started.set()
-            release.wait(10)
-            return {"success": True, "message": "done"}
-
-        queue.enqueue_operation(OperationType.INSTALL, "busy", operation_callback=blocker)
-        assert started.wait(5)
-        yield queue
-        release.set()
-        queue.shutdown()
-
-    def test_a_pending_operation_reports_pending(self, api_v3_client, busy_queue):
-        op_id = busy_queue.enqueue_operation(
-            OperationType.INSTALL, "demo", operation_callback=_callback)
-        response = api_v3_client.get(f"/api/v3/plugins/operation/{op_id}")
-        assert response.status_code == 200, response.get_json()
-        data = response.get_json()["data"]
-        assert data["status"] == "pending"
-        assert data["plugin_id"] == "demo"
-        assert "_callback" not in data["parameters"]

test/test_api_v3_partial_main_save.py

@@ -253,32 +253,6 @@ class TestVegasCycleDurations:
         assert saved['config']['display']['display_durations'] == {'clock': 45}
 
 
-class TestMalformedBody:
-    """A JSON body that does not parse is the caller's mistake: a 400.
-
-    get_json() raised Werkzeug's BadRequest inside the handler's try, whose
-    catch-all answered 500 CONFIG_SAVE_FAILED with "check file permissions"
-    advice and logged a traceback at ERROR.
-    """
-
-    def test_is_a_400_in_the_raw_routes_shape(self, api_v3_client, saved, api_v3_module):
-        api_v3_module.api_v3.config_manager.get_raw_file_content.return_value = {}
-        resp = api_v3_client.post('/api/v3/config/main', data='{not json',
-                                  content_type='application/json')
-        assert resp.status_code == 400
-        assert resp.get_json() == {'status': 'error', 'message': 'Invalid JSON in request body'}
-        assert 'config' not in saved
-        raw = api_v3_client.post('/api/v3/config/raw/main', data='{not json',
-                                 content_type='application/json')
-        assert (raw.status_code, raw.get_json()) == (400, resp.get_json())
-
-    def test_an_empty_json_post_is_still_no_data(self, api_v3_client, saved):
-        resp = api_v3_client.post('/api/v3/config/main', data='',
-                                  content_type='application/json')
-        assert resp.status_code == 400
-        assert resp.get_json()['message'] == 'No data provided'
-
-
 class TestRawSaveStartsAutoUpdateSetup:
     @pytest.fixture
     def raw_env(self, api_v3_module, monkeypatch):

test/test_api_v3_plugin_action_params.py

@@ -1,93 +0,0 @@
-"""POST /api/v3/plugins/action hands ``params`` to the plugin's script intact.
-
-The route runs the script through a generated wrapper, and the params went
-into that wrapper as Python source: ``params = {json.dumps(params)}``. JSON is
-not Python. ``true``, ``false`` and ``null`` are undefined names there, so any
-params holding a boolean or a null died with a NameError before the script
-ran. The plugin file manager's category toggle sends ``{"category_name": ...,
-"enabled": true}``, so of-the-day's category toggle failed every time with
-"Action failed".
-
-The script's side of the contract is unchanged and pinned here too: the
-params arrive on stdin as one JSON document, LEDMATRIX_ROOT is set, and what
-the script prints to stdout is what the route parses.
-"""
-
-import json
-import subprocess
-import sys
-from pathlib import Path
-
-import pytest
-
-sys.path.insert(0, str(Path(__file__).parent.parent))
-
-from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
-
-ACTION_URL = "/api/v3/plugins/action"
-
-# The action script: report what it was handed, as JSON on stdout.
-ECHO_SCRIPT = (
-    "import json, os, sys\n"
-    "raw = sys.stdin.read()\n"
-    "print(json.dumps({'status': 'success', 'got': json.loads(raw),\n"
-    "                  'root': os.environ.get('LEDMATRIX_ROOT')}))\n"
-)
-
-
-@pytest.fixture
-def echo_plugin(tmp_path, api_v3_module, monkeypatch):
-    plugin_dir = tmp_path / "demo"
-    plugin_dir.mkdir()
-    (plugin_dir / "manifest.json").write_text(json.dumps({
-        "id": "demo",
-        "web_ui_actions": [{"id": "toggle", "type": "script", "script": "echo.py"}],
-    }), encoding="utf-8")
-    (plugin_dir / "echo.py").write_text(ECHO_SCRIPT, encoding="utf-8")
-    api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = str(plugin_dir)
-
-    # The route runs `python3`; use this interpreter, so the test does not
-    # depend on what that name resolves to here.
-    real_run = subprocess.run
-
-    def run(cmd, *args, **kwargs):
-        if isinstance(cmd, list) and cmd and cmd[0] == "python3":
-            cmd = [sys.executable] + cmd[1:]
-        return real_run(cmd, *args, **kwargs)
-
-    monkeypatch.setattr(subprocess, "run", run)
-    return plugin_dir
-
-
-@pytest.mark.parametrize("params", [
-    {"category_name": "jokes", "enabled": True},   # the file manager's toggle
-    {"category_name": "jokes", "enabled": False},
-    {"filename": None},
-    {"nested": {"list": [1, None, True, 2.5], "empty": {}}},
-    {"text": "café ✓ \U0001F600"},
-    {"text": "he said \"hi\" and 'bye' \\ ''' \"\"\" \n\t end"},
-], ids=["true", "false", "null", "nested", "unicode", "quotes"])
-def test_the_script_receives_the_params_it_was_sent(api_v3_client, echo_plugin, params):
-    response = api_v3_client.post(ACTION_URL, json={
-        "plugin_id": "demo", "action_id": "toggle", "params": params})
-    body = response.get_json()
-    assert response.status_code == 200, body
-    assert body["got"] == params
-
-
-def test_a_param_cannot_run_code_in_the_wrapper(api_v3_client, echo_plugin, tmp_path):
-    marker = tmp_path / "PWNED"
-    hostile = "\"}\nopen(%r, 'w').write('ran')\n#" % str(marker)
-    params = {"name": hostile, "flag": True}
-    response = api_v3_client.post(ACTION_URL, json={
-        "plugin_id": "demo", "action_id": "toggle", "params": params})
-    assert response.status_code == 200, response.get_json()
-    assert response.get_json()["got"] == params
-    assert not marker.exists(), "a param value ran as code"
-
-
-def test_the_script_still_gets_ledmatrix_root(api_v3_client, echo_plugin, api_v3_module):
-    response = api_v3_client.post(ACTION_URL, json={
-        "plugin_id": "demo", "action_id": "toggle", "params": {"enabled": True}})
-    assert response.status_code == 200, response.get_json()
-    assert response.get_json()["root"] == str(api_v3_module.PROJECT_ROOT)

test/test_api_v3_plugin_operation_conflict.py

@@ -1,89 +0,0 @@
-"""A second install or uninstall while one is in progress is a 409, not a 500.
-
-PluginOperationQueue refuses a second operation for a plugin that already
-has one waiting or running (test_operation_queue_pending_and_trim.py), and
-says so by raising ValueError. /plugins/install let that escape to the
-blueprint's catch-all, so a double-clicked Install answered 500 "An error
-occurred; see logs for details" while the first install carried on.
-/plugins/uninstall caught it in its own catch-all: a 500 "Failed to
-uninstall plugin", and an "uninstall failed" entry in the operation
-history for an uninstall that never started.
-"""
-
-import sys
-import threading
-from pathlib import Path
-
-import pytest
-
-sys.path.insert(0, str(Path(__file__).parent.parent))
-
-from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
-
-from src.plugin_system.operation_queue import PluginOperationQueue  # noqa: E402
-
-INSTALL = "/api/v3/plugins/install"
-UNINSTALL = "/api/v3/plugins/uninstall"
-
-
-@pytest.fixture
-def installing(api_v3_module, tmp_path):
-    """A real queue with an install of "clock" running and held there."""
-    queue = PluginOperationQueue(max_history=10)
-    api_v3_module.api_v3.operation_queue = queue
-    started, release = threading.Event(), threading.Event()
-
-    def slow_install(plugin_id, branch=None):
-        started.set()
-        release.wait(10)
-        return True
-
-    store = api_v3_module.api_v3.plugin_store_manager
-    store.install_plugin.side_effect = slow_install
-    store.get_registry_info.return_value = None
-    store.plugins_dir = str(tmp_path)
-    api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = None
-    yield {"queue": queue, "started": started, "store": store}
-    release.set()
-    queue.shutdown()
-
-
-def _start_first_install(client, installing):
-    response = client.post(INSTALL, json={"plugin_id": "clock"})
-    assert response.status_code == 200, response.get_json()
-    assert installing["started"].wait(5)
-
-
-def _failed_history(api_v3_module):
-    return [c for c in api_v3_module.api_v3.operation_history.record_operation.call_args_list
-            if c.kwargs.get("status") == "failed"]
-
-
-def test_a_second_install_click_is_a_conflict(api_v3_client, api_v3_module, installing):
-    _start_first_install(api_v3_client, installing)
-    response = api_v3_client.post(INSTALL, json={"plugin_id": "clock"})
-    assert response.status_code == 409, response.get_json()
-    body = response.get_json()
-    assert body["status"] == "error"
-    assert body["error_code"] == "PLUGIN_OPERATION_CONFLICT"
-    assert "clock" in body["message"]
-    assert installing["store"].install_plugin.call_count == 1
-    assert _failed_history(api_v3_module) == []
-
-
-def test_an_uninstall_during_the_install_is_a_conflict(api_v3_client, api_v3_module,
-                                                        installing):
-    _start_first_install(api_v3_client, installing)
-    response = api_v3_client.post(UNINSTALL, json={"plugin_id": "clock"})
-    assert response.status_code == 409, response.get_json()
-    assert response.get_json()["error_code"] == "PLUGIN_OPERATION_CONFLICT"
-    assert _failed_history(api_v3_module) == [], (
-        "an uninstall that never started was recorded as failed")
-    api_v3_module.api_v3.plugin_store_manager.uninstall_plugin.assert_not_called()
-
-
-def test_another_plugin_is_still_queued(api_v3_client, installing):
-    _start_first_install(api_v3_client, installing)
-    response = api_v3_client.post(INSTALL, json={"plugin_id": "weather"})
-    assert response.status_code == 200, response.get_json()
-    assert response.get_json()["data"]["operation_id"]

test/test_api_v3_plugin_static_files.py

@@ -1,73 +0,0 @@
-"""GET /api/v3/plugins/<plugin_id>/static/<path> serves binary files too.
-
-The route opened every file as UTF-8 text, so an image -- what the API
-reference says it is for, plugin previews and icons -- failed to decode and
-answered 500 "UnicodeDecodeError". Files are now sent as bytes. The text
-types the route always set are unchanged, and the path checks are pinned in
-test_path_traversal_guards.py::TestServePluginStatic.
-"""
-
-import json
-import sys
-from pathlib import Path
-
-import pytest
-
-sys.path.insert(0, str(Path(__file__).parent.parent))
-
-from test._api_v3_test_helpers import api_v3_client, api_v3_module  # noqa: F401,E402
-
-PNG = (b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01"
-       b"\x08\x06\x00\x00\x00\x1f\x15\xc4\x89")
-
-
-@pytest.fixture
-def plugin_dir(tmp_path, api_v3_module):
-    d = tmp_path / "demo"
-    (d / "web_ui").mkdir(parents=True)
-    (d / "manifest.json").write_text(json.dumps({"id": "demo"}), encoding="utf-8")
-    api_v3_module.api_v3.plugin_catalog.get_plugin_directory.side_effect = (
-        lambda pid: str(d) if pid == "demo" else None)
-    return d
-
-
-def _get(client, path):
-    return client.get(f"/api/v3/plugins/demo/static/{path}")
-
-
-def test_an_image_is_served_as_its_bytes(api_v3_client, plugin_dir):
-    (plugin_dir / "web_ui" / "icon.png").write_bytes(PNG)
-    response = _get(api_v3_client, "web_ui/icon.png")
-    assert response.status_code == 200, response.get_json(silent=True)
-    assert response.mimetype == "image/png"
-    assert response.data == PNG
-
-
-def test_an_unknown_binary_file_is_served_too(api_v3_client, plugin_dir):
-    blob = bytes(range(256))
-    (plugin_dir / "data.bin").write_bytes(blob)
-    response = _get(api_v3_client, "data.bin")
-    assert response.status_code == 200, response.get_json(silent=True)
-    assert response.data == blob
-
-
-@pytest.mark.parametrize("name,mimetype", [
-    ("page.html", "text/html"),
-    ("app.js", "application/javascript"),
-    ("style.css", "text/css"),
-    ("data.json", "application/json"),
-    ("notes.txt", "text/plain"),
-    ("README.md", "text/plain"),
-    ("helper.py", "text/plain"),
-])
-def test_text_files_keep_their_types(api_v3_client, plugin_dir, name, mimetype):
-    content = "caf\u00e9 \u2713 <p>hi</p>\n"
-    (plugin_dir / name).write_bytes(content.encode("utf-8"))
-    response = _get(api_v3_client, name)
-    assert response.status_code == 200
-    assert response.mimetype == mimetype
-    assert response.data == content.encode("utf-8")
-
-
-def test_a_missing_file_is_still_a_404(api_v3_client, plugin_dir):
-    assert _get(api_v3_client, "nope.png").status_code == 404

test/test_config_load_cache.py

@@ -58,8 +58,7 @@ class TestFastPath:
         for _ in range(10):
             again = m.load_config()
         assert counts["n"] == 0, "fast path must not re-open any config file"
-        assert again == first
-        assert again is not first  # each caller gets its own copy, see below
+        assert again is first  # same aliasing semantics as the full path
 
     def test_config_change_triggers_reload(self, mgr):
         m, config, secrets, template = mgr
@@ -99,33 +98,6 @@ class TestFastPath:
         assert m.load_config()["timezone"] == "America/New_York"
 
 
-class TestCallersGetACopy:
-    """A web handler edits what load_config returned, then validates. When
-    validation failed, the edit stayed in the cache the fast path serves, and
-    the next unrelated save wrote it -- a nested secret included, in plain
-    text, because it had never reached config_secrets.json to be stripped."""
-
-    def test_editing_a_loaded_config_does_not_change_the_next_load(self, mgr):
-        m, config, secrets, template = mgr
-        loaded = m.load_config()
-        loaded["display"]["brightness"] = 1
-        loaded["weather"]["api_key"] = "typed-but-never-saved"
-        again = m.load_config()
-        assert again["display"]["brightness"] == 90
-        assert again["weather"]["api_key"] == "sek"
-
-    def test_the_full_path_also_returns_a_copy(self, mgr):
-        m, config, secrets, template = mgr
-        m.load_config()["display"]["brightness"] = 1  # first load: full path
-        assert m.load_config()["display"]["brightness"] == 90
-
-    def test_an_edit_never_reaches_a_later_save(self, mgr):
-        m, config, secrets, template = mgr
-        m.load_config()["display"]["new_secret"] = "hunter2"  # then bailed out
-        m.save_config(m.load_config())  # some other handler saves
-        assert "hunter2" not in config.read_text()
-
-
 class TestSaveCoherence:
     def test_save_config_then_load_returns_saved_data(self, mgr, monkeypatch):
         m, config, secrets, template = mgr
@@ -139,15 +111,6 @@ class TestSaveCoherence:
         assert loaded["weather"]["api_key"] == "sek"  # secrets survive in memory
         assert counts["n"] == 0  # signature refreshed by save; no re-read
 
-    def test_the_saved_dict_does_not_become_the_cache(self, mgr):
-        m, config, secrets, template = mgr
-        m.load_config()
-        new = {"display": {"brightness": 42}, "timezone": "UTC",
-               "weather": {"api_key": "sek"}}
-        m.save_config(new)
-        new["display"]["brightness"] = 7  # the caller keeps using its dict
-        assert m.load_config()["display"]["brightness"] == 42
-
     def test_cross_process_save_is_picked_up(self, mgr):
         """Another process writing config.json (different mtime) must bust
         this process's fast path — the core cross-process guarantee."""

test/test_config_manager_secrets.py

@@ -143,10 +143,7 @@ class TestLoadFastPath:
         manager = make_manager(tmp_path, config={"timezone": "UTC"})
         first = manager.load_config()
         second = manager.load_config()
-        # A copy of the cached dict, never the dict itself; that it is not
-        # re-read is test_config_load_cache's test_unchanged_files_are_not_reread
-        assert second == first
-        assert second is not first
+        assert second is first  # same aliased dict, no re-read
 
     def test_touching_secrets_file_invalidates_cache(self, tmp_path):
         manager = make_manager(

test/test_espn_payload.py

@@ -0,0 +1,162 @@
+"""Tests for src/common/espn_payload.py and its use by BackgroundDataService."""
+
+import copy
+import time
+from unittest.mock import MagicMock, Mock, patch
+
+import pytest
+
+from src.background_data_service import BackgroundDataService, shutdown_background_service
+from src.common.espn_payload import is_espn_scoreboard_url, slim_scoreboard_payload
+
+SCOREBOARD = "https://site.api.espn.com/apis/site/v2/sports/baseball/mlb/scoreboard"
+
+
+def _event():
+    """One event carrying every key the slimming drops and a sample of the
+    keys scoreboards read, at the depth ESPN puts them."""
+    competitor = {
+        "id": "10",
+        "homeAway": "home",
+        "score": "5",
+        "team": {"abbreviation": "NYY", "logo": "https://a/l.png",
+                 "links": [{"href": "https://espn.com/team"}]},
+        "records": [{"summary": "90-60"}],
+        "linescores": [{"value": 1}],
+        "statistics": [{"name": "hits", "displayValue": "9"}],
+        "leaders": [{"name": "avg", "leaders": [{"athlete": {"id": "1"}}]}],
+        "probables": [{"athlete": {"id": "2"}, "statistics": []}],
+    }
+    return {
+        "id": "401",
+        "date": "2026-10-01T23:05Z",
+        "links": [{"href": "https://espn.com/game"}],
+        "status": {"type": {"state": "post"}},
+        "competitions": [{
+            "status": {"type": {"state": "post", "shortDetail": "Final"},
+                       "featuredAthletes": [{"athlete": {"id": "3"}}]},
+            "competitors": [competitor, dict(copy.deepcopy(competitor), homeAway="away")],
+            "odds": [{"details": "NYY -150", "overUnder": 8.5}],
+            "situation": {"outs": 2},
+            "notes": [{"headline": "Game 1"}],
+            "broadcasts": [{"names": ["FOX"]}],
+            "venue": {"fullName": "Yankee Stadium"},
+            "leaders": [{"name": "hits"}],
+            "headlines": [{"description": "recap"}],
+            "highlights": [{"links": {"source": {}}}],
+            "geoBroadcasts": [{"media": {"shortName": "FOX"}}],
+        }],
+    }
+
+
+class TestSlimScoreboardPayload:
+    def test_drops_exactly_the_listed_keys(self):
+        payload = {"leagues": [{"id": "10"}], "events": [_event()]}
+        slim_scoreboard_payload(payload)
+        event = payload["events"][0]
+        competition = event["competitions"][0]
+        assert "links" not in event
+        for key in ("leaders", "headlines", "highlights", "geoBroadcasts"):
+            assert key not in competition
+        assert "featuredAthletes" not in competition["status"]
+        for competitor in competition["competitors"]:
+            assert "leaders" not in competitor
+            assert "probables" not in competitor
+            assert "links" not in competitor["team"]
+
+    def test_keeps_everything_else_unchanged(self):
+        """Removing the dropped keys from the original by hand gives exactly
+        the slimmed payload: nothing else moved, changed or went missing."""
+        original = {"leagues": [{"id": "10"}], "events": [_event(), _event()]}
+        expected = copy.deepcopy(original)
+        for event in expected["events"]:
+            del event["links"]
+            competition = event["competitions"][0]
+            for key in ("leaders", "headlines", "highlights", "geoBroadcasts"):
+                del competition[key]
+            del competition["status"]["featuredAthletes"]
+            for competitor in competition["competitors"]:
+                del competitor["leaders"], competitor["probables"]
+                del competitor["team"]["links"]
+        assert slim_scoreboard_payload(original) == expected
+
+    def test_in_place_and_returns_payload(self):
+        payload = {"events": [_event()]}
+        assert slim_scoreboard_payload(payload) is payload
+
+    @pytest.mark.parametrize("payload", [
+        None, [], "x", {}, {"events": None}, {"events": "x"},
+        {"events": [None, 1, "x", {"competitions": None}]},
+        {"events": [{"competitions": [None, {"status": None, "competitors": None}]}]},
+        {"events": [{"competitions": [{"competitors": [None, {"team": None}]}]}]},
+    ])
+    def test_odd_shapes_pass_through(self, payload):
+        before = copy.deepcopy(payload)
+        assert slim_scoreboard_payload(payload) == before
+
+
+class TestIsEspnScoreboardUrl:
+    @pytest.mark.parametrize("url", [
+        SCOREBOARD,
+        SCOREBOARD + "/",
+        "http://site.api.espn.com/apis/site/v2/sports/football/college-football/scoreboard",
+    ])
+    def test_scoreboards(self, url):
+        assert is_espn_scoreboard_url(url)
+
+    @pytest.mark.parametrize("url", [
+        None, "", 12,
+        "https://site.api.espn.com/apis/site/v2/sports/baseball/mlb/teams",
+        "https://site.api.espn.com/apis/site/v2/sports/football/nfl/summary",
+        "https://example.com/scoreboard",
+        "https://espn.com.evil.example/apis/x/scoreboard",
+        "https://notespn.com/apis/x/scoreboard",
+    ])
+    def test_not_scoreboards(self, url):
+        assert not is_espn_scoreboard_url(url)
+
+
+@pytest.fixture
+def service():
+    shutdown_background_service()
+    cache = MagicMock()
+    cache.get.return_value = None
+    svc = BackgroundDataService(cache, max_workers=1, request_timeout=5)
+    yield svc
+    svc.shutdown(wait=False)
+    shutdown_background_service()
+
+
+def _run(service, url, **kwargs):
+    response = Mock(status_code=200)
+    response.json.return_value = {"events": [_event()]}
+    response.raise_for_status.return_value = None
+    delivered = []
+    with patch.object(service.session, "get", return_value=response):
+        req_id = service.submit_fetch_request(
+            sport="mlb", year=2026, url=url, cache_key="mlb_schedule_window_14_7",
+            callback=lambda result: delivered.append(result.data), **kwargs)
+        deadline = time.time() + 5
+        while not service.is_request_complete(req_id) and time.time() < deadline:
+            time.sleep(0.02)
+    cached = service.cache_manager.set.call_args[0][1]
+    return cached, delivered
+
+
+class TestBackgroundServiceSlims:
+    def test_espn_scoreboard_is_cached_and_delivered_slimmed(self, service):
+        cached, delivered = _run(service, SCOREBOARD)
+        competition = cached["events"][0]["competitions"][0]
+        assert "leaders" not in competition
+        assert "probables" not in competition["competitors"][0]
+        assert competition["odds"] and competition["situation"]
+        # The callback sees the very payload that was cached.
+        assert delivered and delivered[0] is cached
+
+    def test_opt_out_caches_whole_response(self, service):
+        cached, _ = _run(service, SCOREBOARD, slim_payload=False)
+        assert cached == {"events": [_event()]}
+
+    def test_other_urls_untouched(self, service):
+        cached, _ = _run(service, "https://example.com/feed")
+        assert cached == {"events": [_event()]}

test/test_mqtt_bridge_config_endpoint.py

@@ -136,22 +136,3 @@ class TestCleartextCredentialsNeedAnExplicitOptIn:
                              "allow_insecure_mqtt": "false"})
         assert r.status_code == 400
 
-    def test_the_settings_read_reports_the_opt_in(self, client, monkeypatch):
-        """The Tools form prefills its "Allow without TLS" box from the GET.
-
-        Off until someone saves it on, so an untouched form sends false and
-        the guard above still refuses a cleartext password.
-        """
-        c, _ = client
-        monkeypatch.setattr(misc, "_mqtt_bridge_service_state",
-                            lambda: {"installed": False, "active": False, "enabled": False})
-
-        def read():
-            return c.get("/api/v3/integrations/mqtt-bridge").get_json()["data"]["config"]
-
-        assert read()["allow_insecure_mqtt"] is False
-        r = c.put(URL, json={"mqtt_password": "hunter2", "mqtt_tls": False,
-                             "allow_insecure_mqtt": True})
-        assert r.status_code == 200, r.get_json()
-        assert read()["allow_insecure_mqtt"] is True
-

test/test_vegas_static_mode.py

@@ -219,8 +219,7 @@ class TestCoordinatorStaticPause:
     def _plugin(self):
         plugin = MagicMock()
         plugin.plugin_id = 'clock'
-        # A moment: zero would pause 15 s, as the rotation shows it.
-        plugin.get_display_duration.return_value = 0.01
+        plugin.get_display_duration.return_value = 0
         return plugin
 
     def test_trigger_comes_from_the_pipeline(self):

test/test_vegas_static_pause_duration.py

@@ -1,197 +0,0 @@
-"""A Vegas static pause lasts as long as the rotation shows the plugin.
-
-The pause asked the plugin for get_display_duration() and compared the
-answer with the clock. Several plugins (clock-simple, calendar, countdown)
-return their display_duration setting as it is in config.json, so one saved
-as "20" or null -- the raw config editor, a hand edit -- reached that
-comparison as a string or None. The TypeError went to the pause's broad
-except, which ended the pause: the plugin flashed up and the scroll went on,
-at every one of its turns. inf paused until something interrupted it, and
-NaN, False, 0 or a negative number ended the pause at once.
-
-The pause now reads the answer the way the rotation does since #739, with
-the same helper (base_plugin.finite_seconds): a numeric string counts;
-anything else that is not a finite number, or a raise, gets the rotation's
-30 s; a number at or below zero gets its 15 s.
-"""
-
-import logging
-import os
-import threading
-from types import SimpleNamespace
-from unittest.mock import MagicMock
-
-os.environ.setdefault("EMULATOR", "true")
-
-import pytest
-
-from src.vegas_mode import coordinator
-
-NOT_NUMBERS = [None, '', 'twenty', True, False, float('nan'), float('inf'),
-               'inf', '1e400', [20], {'seconds': 20}]
-NOT_ABOVE_ZERO = [0, -5, '-5', '0']
-NUMBERS = [('20', 20.0), (' 7.5 ', 7.5), (12, 12.0), (12.5, 12.5)]
-
-
-class FakeClock:
-    """time.monotonic/time.sleep for the pause loop: sleeping moves the clock."""
-
-    #: A pause still going after this long never ends (inf did that).
-    LIMIT = 3600.0
-
-    def __init__(self):
-        self.now = 0.0
-
-    def monotonic(self):
-        return self.now
-
-    def sleep(self, seconds):
-        self.now += seconds
-        if self.now > self.LIMIT:
-            raise RuntimeError("the static pause never ended")
-
-
-@pytest.fixture
-def clock(monkeypatch):
-    fake = FakeClock()
-    monkeypatch.setattr(coordinator, 'time', fake)
-    return fake
-
-
-def _plugin(duration, plugin_id='clock-simple'):
-    plugin = MagicMock()
-    plugin.plugin_id = plugin_id
-    plugin.get_display_duration.return_value = duration
-    return plugin
-
-
-def _coord(*plugins):
-    coord = coordinator.VegasModeCoordinator.__new__(coordinator.VegasModeCoordinator)
-    coord.render_pipeline = MagicMock()
-    coord.render_pipeline.get_scroll_position.return_value = 0
-    coord.display_manager = MagicMock()
-    locks = {plugin.plugin_id: threading.Lock() for plugin in plugins}
-    coord.plugin_manager = SimpleNamespace(get_plugin_lock=locks.__getitem__)
-    coord._state_lock = threading.Lock()
-    coord._static_pause_active = False
-    coord._saved_scroll_position = None
-    coord._should_stop = False
-    coord._live_priority_active = False
-    coord._live_priority_check = None
-    coord._interrupt_check = None
-    coord.stats = {'static_pauses': 0}
-    return coord
-
-
-def _pause(coord, plugin, clock):
-    """One static pause: (whether it completed, how long it lasted)."""
-    start = clock.now
-    completed = coord._handle_static_pause(plugin)
-    return completed, clock.now - start
-
-
-class TestPauseLength:
-    @pytest.mark.parametrize('value, seconds', NUMBERS)
-    def test_numbers_and_numeric_strings_are_used(self, clock, value, seconds):
-        plugin = _plugin(value)
-        completed, lasted = _pause(_coord(plugin), plugin, clock)
-        assert completed is True
-        assert lasted == pytest.approx(seconds, abs=0.15)
-
-    @pytest.mark.parametrize('value', NOT_NUMBERS, ids=repr)
-    def test_anything_but_a_finite_number_pauses_for_30s(self, clock, value):
-        plugin = _plugin(value)
-        completed, lasted = _pause(_coord(plugin), plugin, clock)
-        assert completed is True
-        assert lasted == pytest.approx(30.0, abs=0.15)
-        plugin.display.assert_called_once_with(force_clear=True)
-
-    @pytest.mark.parametrize('value', NOT_ABOVE_ZERO, ids=repr)
-    def test_a_number_not_above_zero_pauses_for_15s(self, clock, value):
-        plugin = _plugin(value)
-        completed, lasted = _pause(_coord(plugin), plugin, clock)
-        assert completed is True
-        assert lasted == pytest.approx(15.0, abs=0.15)
-
-    def test_a_raising_get_display_duration_pauses_for_30s(self, clock):
-        plugin = _plugin(None)
-        plugin.get_display_duration.side_effect = KeyError('display_duration')
-        completed, lasted = _pause(_coord(plugin), plugin, clock)
-        assert completed is True
-        assert lasted == pytest.approx(30.0, abs=0.15)
-
-    def test_a_good_value_after_a_bad_one_is_used(self, clock):
-        plugin = _plugin(None)
-        coord = _coord(plugin)
-        assert _pause(coord, plugin, clock)[1] == pytest.approx(30.0, abs=0.15)
-        plugin.get_display_duration.return_value = 45
-        assert _pause(coord, plugin, clock)[1] == pytest.approx(45.0, abs=0.15)
-
-    def test_the_pause_can_still_be_interrupted(self, clock):
-        plugin = _plugin('twenty')
-        coord = _coord(plugin)
-        coord._interrupt_check = lambda: clock.now >= 5
-        completed, lasted = _pause(coord, plugin, clock)
-        assert completed is False
-        assert lasted == pytest.approx(5.0, abs=0.15)
-
-
-class TestWarning:
-    def test_logged_once_per_plugin(self, clock, caplog):
-        clock_plugin = _plugin('twenty')
-        calendar = _plugin(None, plugin_id='calendar')
-        coord = _coord(clock_plugin, calendar)
-        with caplog.at_level(logging.WARNING, logger='src.vegas_mode.coordinator'):
-            for _ in range(3):
-                for plugin in (clock_plugin, calendar):
-                    coord._handle_static_pause(plugin)
-        warnings = [r.getMessage() for r in caplog.records
-                    if 'display duration' in r.getMessage()]
-        assert len(warnings) == 2
-        assert any('clock-simple' in m and "'twenty'" in m for m in warnings)
-        assert any('calendar' in m and 'None' in m for m in warnings)
-
-
-class TestFiniteSeconds:
-    """The shared rule: what counts as a number of seconds."""
-
-    @pytest.mark.parametrize('value, seconds', NUMBERS + [(0, 0.0), ('-5', -5.0)])
-    def test_numbers_and_numeric_strings(self, value, seconds):
-        from src.plugin_system.base_plugin import finite_seconds
-        result = finite_seconds(value)
-        assert result == seconds and isinstance(result, float)
-
-    @pytest.mark.parametrize('value', NOT_NUMBERS + [pytest.param(10 ** 400, id='10**400')],
-                             ids=repr)
-    def test_anything_else_is_none(self, value):
-        from src.plugin_system.base_plugin import finite_seconds
-        assert finite_seconds(value) is None
-
-
-def _rotation_seconds(plugin):
-    """How long the rotation shows ``plugin`` (no dynamic duration, no
-    Rotation & Durations override): the two calls run() makes for a screen.
-    """
-    from src.display_controller import DisplayController
-    dc = object.__new__(DisplayController)
-    dc.config = {}
-    dc.plugin_modes = {'mode': plugin}
-    return dc._resolve_durations(plugin, 'mode', dc._get_display_duration('mode'), False)[1]
-
-
-class TestSameAsTheRotation:
-    """The pause and the rotation share finite_seconds; this pins their
-    fallbacks (30 s, 15 s) to each other too."""
-
-    @pytest.mark.parametrize('value', [value for value, _ in NUMBERS]
-                             + NOT_NUMBERS + NOT_ABOVE_ZERO, ids=repr)
-    def test_the_pause_lasts_as_long_as_the_rotation_shows_it(self, clock, value):
-        plugin = _plugin(value)
-        expected = _rotation_seconds(plugin)
-        assert _pause(_coord(plugin), plugin, clock)[1] == pytest.approx(expected, abs=0.15)
-
-    def test_a_raise_too(self, clock):
-        plugin = _plugin(None)
-        plugin.get_display_duration.side_effect = KeyError('display_duration')
-        expected = _rotation_seconds(plugin)
-        assert _pause(_coord(plugin), plugin, clock)[1] == pytest.approx(expected, abs=0.15)

test/web_interface/test_api_v3_backup_restore.py

@@ -50,13 +50,11 @@ class FakeResult:
         self.plugins_to_install = plugins_to_install or []
         self.plugins_installed = []
         self.plugins_failed = []
-        self.skipped = []
 
     def to_dict(self):
         return {
             "success": self.success,
             "restored": self.restored,
-            "skipped": self.skipped,
             "errors": self.errors,
             "plugins_installed": self.plugins_installed,
             "plugins_failed": self.plugins_failed,
@@ -288,109 +286,6 @@ class TestPluginReinstall:
         assert body["data"]["plugins_failed"][0]["error"] == "Store manager unavailable"
 
 
-class TestInstalledPluginsAreNotReinstalled:
-    """"Reinstall missing plugins" installs only what is missing.
-
-    Every plugin the backup listed went to install_plugin, which replaces an
-    installed copy with a fresh download: restoring onto the same device
-    re-downloaded all of them inside the request. One installed from its own
-    URL is not in the registry, so its "reinstall" returned False and the
-    whole restore answered 500 "Restore failed" with the plugin still there.
-    """
-
-    @staticmethod
-    def _installed(tmp_path, *names):
-        found = {}
-        for name in names:
-            (tmp_path / name).mkdir()
-            found[name] = tmp_path / name
-        return lambda plugin_id: found.get(plugin_id)
-
-    def test_an_installed_plugin_is_skipped_and_a_missing_one_installed(
-            self, client, restore, tmp_path):
-        restore.return_value = FakeResult(
-            plugins_to_install=[{"plugin_id": "clock"}, {"plugin_id": "weather"}])
-        store = api_v3.plugin_store_manager
-        store._existing_install.side_effect = self._installed(tmp_path, "clock")
-        store.install_plugin.return_value = True
-        response = post(client)
-        assert response.status_code == 200
-        store.install_plugin.assert_called_once_with("weather")
-        data = response.get_json()["data"]
-        assert data["plugins_installed"] == ["weather"]
-        assert data["plugins_failed"] == []
-        assert "plugin:clock (installed)" in data["skipped"]
-
-    def test_an_installed_plugin_the_store_cannot_install_is_not_a_failure(
-            self, client, restore, tmp_path):
-        restore.return_value = FakeResult(plugins_to_install=[{"plugin_id": "my-3p"}])
-        store = api_v3.plugin_store_manager
-        store._existing_install.side_effect = self._installed(tmp_path, "my-3p")
-        store.install_plugin.return_value = False
-        response = post(client)
-        assert response.status_code == 200
-        assert response.get_json()["data"]["plugins_failed"] == []
-        store.install_plugin.assert_not_called()
-
-    @pytest.fixture
-    def real_store(self, tmp_path):
-        from src.plugin_system.store_manager import PluginStoreManager
-        plugins_dir = tmp_path / "plugin-repos"
-        for folder, manifest_id in (("ledmatrix-weather", "ledmatrix-weather"),
-                                    ("my-3p", "my-3p")):
-            (plugins_dir / folder).mkdir(parents=True)
-            (plugins_dir / folder / "manifest.json").write_text(
-                json.dumps({"id": manifest_id, "version": "1.0.0"}))
-        store = PluginStoreManager(plugins_dir=str(plugins_dir),
-                                   uninstalled_registry_path=str(tmp_path / "uninstalled.json"))
-        # The official weather plugin's registry id differs from the id it
-        # installs under; my-3p was installed from its own URL.
-        registry = {"plugins": [{
-            "id": "weather", "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins",
-            "plugin_path": "plugins/ledmatrix-weather"}]}
-        store.registry_cache = registry
-        store.fetch_registry = lambda *a, **k: registry
-        store.install_plugin = MagicMock(return_value=True)
-        api_v3.plugin_store_manager = store
-        return store
-
-    def test_with_the_real_store_aliases_and_third_party_installs_count(
-            self, client, restore, real_store):
-        restore.return_value = FakeResult(plugins_to_install=[
-            {"plugin_id": "weather"}, {"plugin_id": "my-3p"}, {"plugin_id": "clock"}])
-        response = post(client)
-        assert response.status_code == 200
-        real_store.install_plugin.assert_called_once_with("clock")
-        skipped = response.get_json()["data"]["skipped"]
-        assert "plugin:weather (installed)" in skipped
-        assert "plugin:my-3p (installed)" in skipped
-
-
-class TestFontsCatalogCache:
-    """The Fonts tab's catalog is cached for 5 minutes (fonts.py).
-
-    Upload and delete clear it; a restore did not, so restored fonts were
-    missing from the Fonts tab and every font picker until it expired.
-    """
-
-    @pytest.fixture
-    def cached_catalog(self):
-        from web_interface.cache import delete_cached, get_cached, set_cached
-        set_cached('fonts_catalog', {'fonts': ['5x7.bdf']}, ttl_seconds=300)
-        yield lambda: get_cached('fonts_catalog', ttl_seconds=300)
-        delete_cached('fonts_catalog')
-
-    def test_a_restore_that_restored_fonts_clears_it(self, client, restore, cached_catalog):
-        restore.return_value = FakeResult(restored=["config", "fonts (2)"])
-        assert post(client).status_code == 200
-        assert cached_catalog() is None
-
-    def test_a_restore_without_fonts_keeps_it(self, client, restore, cached_catalog):
-        restore.return_value = FakeResult(restored=["config"])
-        assert post(client).status_code == 200
-        assert cached_catalog() == {'fonts': ['5x7.bdf']}
-
-
 class TestFailureReporting:
     def test_restore_errors_produce_a_500(self, client, restore):
         restore.return_value = FakeResult(

test/web_interface/test_api_v3_config_raw.py

@@ -13,9 +13,7 @@ tmp_path so the assertions are against files on disk rather than mock
 calls.
 """
 
-import html
 import json
-import re
 import sys
 from pathlib import Path
 from unittest.mock import MagicMock
@@ -223,66 +221,3 @@ class TestRawEndpointsBypassSecretSeparation:
         env.client.post(MAIN, json={"weather": {"api_key": "PLAINTEXT-KEY"}})
         # Nothing was moved aside into the secrets file.
         assert not env.secrets_file.exists() or "PLAINTEXT-KEY" not in env.secrets_file.read_text()
-
-
-class TestConfigEditorRoundTrip:
-    """The Config Editor tab (/partials/raw-json) and the save it posts to.
-
-    The secrets editor is shown masked, like GET /config/secrets: the page is
-    served to anyone who can reach the port while the optional web login is
-    off. Its save strips the masks and merges onto the stored file, so a
-    masked editor saved back as it is changes nothing.
-    """
-
-    STORED = {
-        "github": {"api_token": "ghp_REAL_TOKEN_1234"},
-        "ledmatrix-weather": {"api_key": "WEATHER_KEY_abcdef", "units_id": 42},
-        "calendar": {"accounts": [{"name": "home", "token": "CAL_TOKEN_9"}]},
-        "youtube": {"api_key": "YOUR_YOUTUBE_API_KEY", "channel_secret": ""},
-    }
-    REAL_VALUES = ("ghp_REAL_TOKEN_1234", "WEATHER_KEY_abcdef", "CAL_TOKEN_9")
-
-    @pytest.fixture
-    def editor(self, env, monkeypatch):
-        from web_interface.blueprints import pages_v3 as pages_module
-        env.secrets_file.write_text(json.dumps(self.STORED))
-        monkeypatch.setattr(pages_module.pages_v3, "config_manager",
-                            env.config_manager, raising=False)
-        app = Flask(__name__, template_folder=str(project_root / "web_interface" / "templates"))
-        app.config["TESTING"] = True
-        app.register_blueprint(pages_module.pages_v3)
-        app.register_blueprint(api_v3, url_prefix="/api/v3")
-        return app.test_client()
-
-    @staticmethod
-    def _secrets_textarea(client):
-        page = client.get("/partials/raw-json")
-        assert page.status_code == 200
-        match = re.search(r'<textarea id="secrets-config-editor"[^>]*>(.*?)</textarea>',
-                          page.get_data(as_text=True), re.S)
-        assert match, "the secrets editor is missing from the partial"
-        return html.unescape(match.group(1))
-
-    def test_the_editor_shows_no_secret_value(self, editor):
-        text = self._secrets_textarea(editor)
-        for value in self.REAL_VALUES:
-            assert value not in text
-        shown = json.loads(text)
-        assert shown["github"]["api_token"] == "\u2022" * 8
-        # Same shape as the file, and "not set" still reads as not set.
-        assert shown["calendar"]["accounts"][0]["name"] == "\u2022" * 8
-        assert shown["youtube"] == {"api_key": "YOUR_YOUTUBE_API_KEY", "channel_secret": ""}
-
-    def test_saving_it_back_unchanged_keeps_every_secret(self, editor, env):
-        shown = json.loads(self._secrets_textarea(editor))
-        response = editor.post(SECRETS, json=shown)
-        assert response.status_code == 200
-        assert json.loads(env.secrets_file.read_text()) == self.STORED
-
-    def test_editing_one_secret_changes_only_that_one(self, editor, env):
-        shown = json.loads(self._secrets_textarea(editor))
-        shown["ledmatrix-weather"]["api_key"] = "NEW_WEATHER_KEY"
-        assert editor.post(SECRETS, json=shown).status_code == 200
-        expected = json.loads(json.dumps(self.STORED))
-        expected["ledmatrix-weather"]["api_key"] = "NEW_WEATHER_KEY"
-        assert json.loads(env.secrets_file.read_text()) == expected

test/web_interface/test_api_v3_helpers.py

@@ -169,10 +169,6 @@ class TestGetSchemaProperty:
             },
             "fifa.world": {"type": "object",
                            "properties": {"enabled": {"type": "boolean"}}},
-            "cities": {"type": "array",
-                       "items": {"type": "object",
-                                 "properties": {"timezone": {"type": "string"}}}},
-            "color": {"type": ["array", "null"], "items": {"type": "integer"}},
         }
     }
 
@@ -189,15 +185,6 @@ class TestGetSchemaProperty:
         prop = _get_schema_property(self.SCHEMA, "fifa.world.enabled")
         assert prop == {"type": "boolean"}
 
-    def test_an_index_steps_into_the_array_items(self):
-        # How a table row posts its cells
-        assert _get_schema_property(self.SCHEMA, "cities.0.timezone") == {"type": "string"}
-        assert _get_schema_property(self.SCHEMA, "color.2") == {"type": "integer"}
-
-    def test_a_non_index_under_an_array_is_not_found(self):
-        assert _get_schema_property(self.SCHEMA, "cities.timezone") is None
-        assert _get_schema_property(self.SCHEMA, "cities.0.nope") is None
-
     def test_missing_path_returns_none(self):
         assert _get_schema_property(self.SCHEMA, "nope.nope") is None
 

test/web_interface/test_plugin_config_endpoints.py

@@ -1,393 +0,0 @@
-"""GET and POST /plugins/config against a real ConfigManager and SchemaManager.
-
-Each class is one bug, reproduced through the endpoint the settings form and
-API clients use, with assertions on config.json and config_secrets.json.
-"""
-
-import json
-from unittest.mock import MagicMock
-
-import pytest
-from flask import Flask
-
-from src.config_manager import ConfigManager
-from src.plugin_system.schema_manager import SchemaManager
-from web_interface.blueprints.api_v3 import api_v3
-
-PLUGIN_ID = "demo"
-OTHER_ID = "other"
-
-SCHEMA = {
-    "type": "object",
-    "properties": {
-        "enabled": {"type": "boolean", "default": True},
-        "api_key": {"type": "string", "x-secret": True, "default": ""},
-        "city": {"type": "string", "default": "Austin"},
-        "mqtt": {
-            "type": "object",
-            "properties": {
-                "host": {"type": "string", "default": ""},
-                "port": {"type": "integer", "default": 1883,
-                         "minimum": 1, "maximum": 65535},
-                "password": {"type": "string", "x-secret": True, "default": ""},
-            },
-        },
-        "accounts": {
-            "type": "array",
-            "default": [],
-            "items": {
-                "type": "object",
-                "properties": {
-                    "name": {"type": "string"},
-                    "token": {"type": "string", "x-secret": True},
-                },
-            },
-        },
-    },
-}
-
-OTHER_SCHEMA = {
-    "type": "object",
-    "properties": {
-        "enabled": {"type": "boolean", "default": True},
-        "label": {"type": "string", "default": "x"},
-    },
-}
-
-STORED = {
-    PLUGIN_ID: {"enabled": True, "city": "Paris",
-                "mqtt": {"host": "broker", "port": 1883},
-                "accounts": [{"name": "a"}, {"name": "b"}]},
-    OTHER_ID: {"enabled": True, "label": "hello"},
-}
-
-STORED_SECRETS = {
-    PLUGIN_ID: {"api_key": "TOPSECRET",
-                "accounts": [{"token": "TOK-A"}, {"token": "TOK-B"}]},
-}
-
-_ATTRS = ('config_manager', 'plugin_catalog', 'plugin_store_manager',
-          'saved_repositories_manager', 'schema_manager',
-          'operation_queue', 'operation_history', 'cache_manager')
-
-
-@pytest.fixture
-def env(tmp_path):
-    config_file = tmp_path / "config.json"
-    secrets_file = tmp_path / "config_secrets.json"
-    plugins_dir = tmp_path / "plugins"
-    for plugin_id, schema in ((PLUGIN_ID, SCHEMA), (OTHER_ID, OTHER_SCHEMA)):
-        plugin_dir = plugins_dir / plugin_id
-        plugin_dir.mkdir(parents=True)
-        (plugin_dir / "config_schema.json").write_text(json.dumps(schema))
-        (plugin_dir / "manifest.json").write_text(json.dumps({"id": plugin_id}))
-    config_file.write_text(json.dumps(STORED))
-    secrets_file.write_text(json.dumps(STORED_SECRETS))
-
-    sentinel = object()
-    originals = {name: getattr(api_v3, name, sentinel) for name in _ATTRS}
-
-    config_manager = ConfigManager(config_path=str(config_file),
-                                   secrets_path=str(secrets_file))
-    config_manager.template_path = str(tmp_path / "no-template.json")
-    plugin_manager = MagicMock()
-    plugin_manager.plugin_manifests = {PLUGIN_ID: {"id": PLUGIN_ID},
-                                       OTHER_ID: {"id": OTHER_ID}}
-    plugin_manager.plugins_dir = plugins_dir
-
-    for name in _ATTRS:
-        setattr(api_v3, name, MagicMock())
-    api_v3.config_manager = config_manager
-    api_v3.schema_manager = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path)
-    api_v3.plugin_catalog = plugin_manager
-    api_v3.operation_queue = None
-
-    app = Flask(__name__)
-    app.config["TESTING"] = True
-    app.register_blueprint(api_v3, url_prefix="/api/v3")
-
-    class Env:
-        client = app.test_client()
-
-        @staticmethod
-        def use_schema(schema, plugin_id=PLUGIN_ID):
-            (plugins_dir / plugin_id / "config_schema.json").write_text(json.dumps(schema))
-
-        @staticmethod
-        def store(section, plugin_id=PLUGIN_ID):
-            main = json.loads(config_file.read_text())
-            main[plugin_id] = section
-            config_file.write_text(json.dumps(main))
-
-        @staticmethod
-        def main():
-            return json.loads(config_file.read_text())
-
-        @staticmethod
-        def secrets():
-            return json.loads(secrets_file.read_text())
-
-        @staticmethod
-        def post_form(data, plugin_id=PLUGIN_ID):
-            return Env.client.post(f"/api/v3/plugins/config?plugin_id={plugin_id}",
-                                   data=data)
-
-        @staticmethod
-        def post_json(config, plugin_id=PLUGIN_ID):
-            return Env.client.post("/api/v3/plugins/config",
-                                   json={"plugin_id": plugin_id, "config": config})
-
-    yield Env
-
-    for name, original in originals.items():
-        if original is sentinel:
-            if hasattr(api_v3, name):
-                delattr(api_v3, name)
-        else:
-            setattr(api_v3, name, original)
-
-
-class TestARejectedSaveLeavesNothingBehind:
-    """The form save edited the cached config load_config hands out, then
-    failed validation. The cache kept the edit, and the next save of any
-    other setting wrote it to config.json -- the rejected value, and a
-    nested secret typed into the same form in plain text."""
-
-    REJECTED = {"mqtt.host": "broker", "mqtt.port": "99999",
-                "mqtt.password": "hunter2", "__rendered_section": ["mqtt"]}
-
-    def test_the_rejected_values_never_reach_config_json(self, env):
-        assert env.post_form(self.REJECTED).status_code == 400
-
-        resp = env.post_json({"label": "bye"}, plugin_id=OTHER_ID)
-        assert resp.status_code == 200, resp.get_json()
-
-        main = env.main()
-        assert main[OTHER_ID]["label"] == "bye"
-        assert main[PLUGIN_ID]["mqtt"] == {"host": "broker", "port": 1883}
-        assert "hunter2" not in json.dumps(main)
-
-    def test_the_form_reloads_with_the_stored_values(self, env):
-        assert env.post_form(self.REJECTED).status_code == 400
-        assert api_v3.config_manager.load_config()[PLUGIN_ID]["mqtt"]["port"] == 1883
-
-
-class TestGetMasksSecrets:
-    """GET /plugins/config returned the section with config_secrets.json
-    merged in, secrets and all: the masking #276 added was lost when the
-    route was rewritten. The settings page and GET /config/secrets mask."""
-
-    def test_secrets_come_back_blank(self, env):
-        data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
-        assert data["api_key"] == ""
-        assert data["accounts"] == [{"name": "a", "token": ""}, {"name": "b", "token": ""}]
-        assert data["city"] == "Paris"
-
-    def test_posting_the_response_back_keeps_every_secret(self, env):
-        data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
-        resp = env.post_json(data)
-        assert resp.status_code == 200, resp.get_json()
-        assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
-        assert "TOPSECRET" not in json.dumps(env.main())
-
-    def test_the_settings_form_posting_masked_fields_keeps_every_secret(self, env):
-        # The page renders secrets blank (pages_v3 masks the same way)
-        resp = env.post_form({
-            "api_key": "", "city": "Lyon", "mqtt.host": "broker", "mqtt.port": "1883",
-            "mqtt.password": "", "__rendered_section": ["api_key", "city", "mqtt"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
-        assert env.main()[PLUGIN_ID]["city"] == "Lyon"
-
-    def test_a_plugin_without_a_schema_has_credential_named_fields_blanked(self, env, tmp_path):
-        (tmp_path / "plugins" / "bare").mkdir()
-        env.store({"enabled": True, "station": "KAUS"}, plugin_id="bare")
-        secrets = env.secrets()
-        secrets["bare"] = {"api_token": "BARE-TOKEN"}
-        (tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
-        data = env.client.get("/api/v3/plugins/config?plugin_id=bare").get_json()["data"]
-        assert data["api_token"] == ""
-        assert data["station"] == "KAUS"
-
-    @pytest.mark.parametrize("section", ["web_auth", "github", "display"])
-    def test_a_core_section_is_refused(self, env, tmp_path, section):
-        secrets = env.secrets()
-        secrets["web_auth"] = {"cookie_secret": "COOKIE-KEY", "password_hash": "HASH"}
-        secrets["github"] = {"api_token": "ghp_TOKEN"}
-        (tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
-        env.store({"hardware": {"rows": 32}}, plugin_id="display")
-        resp = env.client.get(f"/api/v3/plugins/config?plugin_id={section}")
-        assert resp.status_code == 400
-        body = resp.get_data(as_text=True)
-        assert "COOKIE-KEY" not in body and "ghp_TOKEN" not in body
-
-
-ROWS_SCHEMA = {
-    "type": "object",
-    "properties": {
-        "enabled": {"type": "boolean", "default": True},
-        "cities": {
-            "type": "array",
-            "x-widget": "array-table",
-            "default": [],
-            "items": {
-                "type": "object",
-                "properties": {
-                    "name": {"type": "string"},
-                    "timezone": {"type": "string"},
-                    "lat": {"type": "number"},
-                    "show": {"type": "boolean", "default": True},
-                },
-                "required": ["name", "lat"],
-            },
-        },
-    },
-}
-
-
-class TestArrayRowCellsFollowTheItemSchema:
-    """A table row posts its cells as ``cities.0.timezone``. The schema
-    lookup stopped at the array, so each cell was parsed blind: a blank
-    optional text cell became null and a text cell holding digits became a
-    number, and either failed validation -- every save of the page, for as
-    long as the row existed (geochron's city without a timezone, a countdown
-    named "2027")."""
-
-    ROW = {"cities.0.name": "Tokyo", "cities.0.timezone": "Asia/Tokyo",
-           "cities.0.lat": "35.68", "cities.0.show": "true",
-           "__rendered_section": ["cities"]}
-
-    @pytest.fixture(autouse=True)
-    def _rows(self, env):
-        env.use_schema(ROWS_SCHEMA)
-        env.store({"enabled": True, "cities": [
-            {"name": "Tokyo", "timezone": "Asia/Tokyo", "lat": 35.68, "show": True}]})
-
-    def test_a_blank_optional_text_cell_saves(self, env):
-        resp = env.post_form({**self.ROW, "cities.0.timezone": ""})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.main()[PLUGIN_ID]["cities"][0]["timezone"] == ""
-
-    def test_a_text_cell_of_digits_stays_text(self, env):
-        resp = env.post_form({**self.ROW, "cities.0.name": "2027"})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.main()[PLUGIN_ID]["cities"][0]["name"] == "2027"
-
-    def test_number_and_boolean_cells_still_convert(self, env):
-        resp = env.post_form({**self.ROW, "cities.0.show": "false"})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.main()[PLUGIN_ID]["cities"] == [
-            {"name": "Tokyo", "timezone": "Asia/Tokyo", "lat": 35.68, "show": False}]
-
-
-class TestMaskedSecretCellsInARow:
-    """The same lookup: a row's secret cell, rendered blank, came back as
-    null and failed validation, so a plugin with secrets in a list could not
-    be saved from its settings page at all."""
-
-    def test_the_stored_tokens_survive_a_save_of_the_form(self, env):
-        resp = env.post_form({
-            "city": "Lyon", "accounts.0.name": "a", "accounts.0.token": "",
-            "accounts.1.name": "b", "accounts.1.token": "",
-            "__rendered_section": ["city", "accounts"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
-        assert env.main()[PLUGIN_ID]["accounts"] == [{"name": "a"}, {"name": "b"}]
-
-
-class TestABlankSecretIsLeftAsStored:
-    """The form renders a secret blank and posts the blank back. For a
-    required secret with no default (youtube-stats' api_key) the blank was
-    read as null, failed validation, and blocked every save of the page
-    until the key was typed in again."""
-
-    @pytest.fixture(autouse=True)
-    def _required_secret(self, env):
-        schema = json.loads(json.dumps(SCHEMA))
-        del schema["properties"]["api_key"]["default"]
-        schema["required"] = ["api_key"]
-        env.use_schema(schema)
-
-    def test_saving_other_settings_keeps_the_stored_secret(self, env):
-        resp = env.post_form({"api_key": "", "city": "Lyon",
-                              "__rendered_section": ["api_key", "city"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.main()[PLUGIN_ID]["city"] == "Lyon"
-        assert env.secrets()[PLUGIN_ID]["api_key"] == "TOPSECRET"
-
-    def test_a_new_secret_is_still_saved(self, env):
-        resp = env.post_form({"api_key": "NEW-KEY", "city": "Lyon",
-                              "__rendered_section": ["api_key", "city"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.secrets()[PLUGIN_ID]["api_key"] == "NEW-KEY"
-
-    def test_a_changed_secret_then_left_blank_stays_changed(self, env):
-        # The second save must not write back what the first one's load
-        # had merged in (the old key)
-        env.post_form({"api_key": "NEW-KEY", "__rendered_section": ["api_key"]})
-        resp = env.post_form({"api_key": "", "city": "Nice",
-                              "__rendered_section": ["api_key", "city"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.secrets()[PLUGIN_ID]["api_key"] == "NEW-KEY"
-
-    def test_a_blank_list_secret_is_left_as_stored_too(self, env, tmp_path):
-        schema = json.loads(json.dumps(SCHEMA))
-        schema["properties"]["tokens"] = {"type": "array", "x-secret": True,
-                                          "items": {"type": "string"}, "default": []}
-        env.use_schema(schema)
-        secrets = env.secrets()
-        secrets[PLUGIN_ID]["tokens"] = ["t1", "t2"]
-        (tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
-        resp = env.post_form({"tokens": "", "city": "Lyon",
-                              "__rendered_section": ["tokens", "city"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.secrets()[PLUGIN_ID]["tokens"] == ["t1", "t2"]
-
-
-class TestSaveRefusesWhatIsNotAPluginId:
-    """GET and reset refuse a core section or a malformed id; the save took
-    any of them. ``{"plugin_id": "display"}`` merged unvalidated values into
-    the core display section, and an id that was not a string raised a
-    TypeError, answered as a 500."""
-
-    def test_a_core_section_is_refused_and_left_alone(self, env):
-        env.store({"hardware": {"rows": 32}}, plugin_id="display")
-        resp = env.post_json({"hardware": {"rows": "banana"}}, plugin_id="display")
-        assert resp.status_code == 400
-        assert env.main()["display"] == {"hardware": {"rows": 32}}
-
-    def test_the_form_save_refuses_one_too(self, env):
-        resp = env.post_form({"password_hash": "x"}, plugin_id="web_auth")
-        assert resp.status_code == 400
-        assert "web_auth" not in env.main()
-
-    @pytest.mark.parametrize("plugin_id", [["demo"], {"id": "demo"}, 7, "", "../demo"])
-    def test_a_malformed_id_is_a_400(self, env, plugin_id):
-        resp = env.post_json({"city": "Lyon"}, plugin_id=plugin_id)
-        assert resp.status_code == 400
-
-
-class TestTextFieldsKeepWhatWasTyped:
-    """A text field holding "true", "False", "[1, 2]" or "{}" was converted
-    to a boolean, list or object before the schema's type was consulted, and
-    the save then failed validation for a perfectly good string."""
-
-    @pytest.mark.parametrize("typed", ["true", "False", "[1, 2]", "{}", "42"])
-    def test_a_text_field(self, env, typed):
-        resp = env.post_form({"city": typed, "__rendered_section": ["city"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.main()[PLUGIN_ID]["city"] == typed
-
-    def test_a_nullable_text_field(self, env):
-        schema = json.loads(json.dumps(SCHEMA))
-        schema["properties"]["nickname"] = {"type": ["string", "null"], "default": None}
-        env.use_schema(schema)
-        resp = env.post_form({"nickname": "false", "__rendered_section": ["nickname"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.main()[PLUGIN_ID]["nickname"] == "false"
-
-    def test_other_types_still_convert(self, env):
-        resp = env.post_form({"mqtt.host": "true", "mqtt.port": "8883",
-                              "__rendered_section": ["mqtt"]})
-        assert resp.status_code == 200, resp.get_json()
-        assert env.main()[PLUGIN_ID]["mqtt"] == {"host": "true", "port": 8883}

web_interface/blueprints/api_v3/__init__.py

@@ -957,19 +957,6 @@ def _get_schema_property(schema, key_path):
                     i = j
                     matched = True
                     break
-                # Through an array to its items: a table row posts its cells
-                # as "cities.0.timezone", where the index names no property.
-                # Stopping here left each cell parsed with no schema at all,
-                # so a blank text cell became null and "2027" a number.
-                items = prop.get('items') if _schema_type_is(prop, 'array') else None
-                if isinstance(items, dict) and parts[j].isdigit():
-                    if j + 1 == len(parts):
-                        return items
-                    if 'properties' in items:
-                        current = items['properties']
-                        i = j + 1
-                        matched = True
-                        break
                 # Matched a non-object before consuming the path — can't go deeper.
                 return None
         if not matched:
@@ -1053,16 +1040,6 @@ def _parse_form_value_with_schema(value, key_path, schema):
 
     # Handle None/empty values
     if value is None or (isinstance(value, str) and value.strip() == ''):
-        # The form draws a stored secret blank, so a blank secret means
-        # "unchanged", and "" is what the save drops as unchanged
-        # (remove_empty_secrets). A required one with no default fell
-        # through to None below, failed validation, and blocked every save
-        # of the page until the secret was typed in again. Not _SKIP_FIELD:
-        # that keeps the merged value from load_config(), which the save
-        # would then write back to config_secrets.json. Text secrets only:
-        # a list or object one gets its empty value below, dropped the same.
-        if prop and prop.get('x-secret') and prop.get('type', 'string') == 'string':
-            return ""
         # A nullable field left blank means null, not an empty container.
         # This is the inherit sentinel for per-mode style overrides: an
         # empty list there would read as "the user chose no colour" rather
@@ -1097,14 +1074,6 @@ def _parse_form_value_with_schema(value, key_path, schema):
     if isinstance(value, str):
         stripped = value.strip()
 
-        # A text field keeps what was typed. The guesses below ran first, so
-        # "true", "False", "[1, 2]" or "{}" in a text field became a boolean,
-        # list or object, and the save failed validation for a good string.
-        declared = prop.get('type') if isinstance(prop, dict) else None
-        if declared == 'string' or (isinstance(declared, list) and
-                                    [t for t in declared if t != 'null'] == ['string']):
-            return value
-
         # Check for boolean strings
         if stripped.lower() == 'true':
             return True

web_interface/blueprints/api_v3/backup.py

@@ -16,7 +16,6 @@ import web_interface.blueprints.api_v3 as _pkg
 # as module attributes, and a value binding would not see the patch.
 # Several are also called from helpers that live in __init__, so the
 # package is the only patch point that covers every caller.
-from web_interface.cache import delete_cached
 
 
 @api_v3.route('/backup/preview', methods=['GET'])
@@ -86,17 +85,6 @@ _RESTORE_OPTION_KEYS = frozenset((
     'restore_config', 'restore_secrets', 'restore_wifi', 'restore_fonts',
     'restore_plugin_uploads', 'reinstall_plugins',
 ))
-def _installed_path(psm, plugin_id):
-    """Where the store finds ``plugin_id`` installed, or None.
-
-    The same lookup install_plugin makes to decide that a copy exists: the
-    id, or an id the registry proves is the same plugin (``aliases``, the
-    ``plugin_path`` name), never a bare ``ledmatrix-<id>`` folder.
-    """
-    found = psm._existing_install(plugin_id)
-    return found if isinstance(found, Path) and found.exists() else None
-
-
 @api_v3.route('/backup/restore', methods=['POST'])
 def backup_restore():
     """Restore a backup ZIP with optional RestoreOptions."""
@@ -146,10 +134,6 @@ def backup_restore():
             os.unlink(tmp_path)
         except OSError:
             pass
-    # Restored fonts reach the Fonts tab through a catalog cached for five
-    # minutes (fonts.py); upload and delete clear it, and so must this.
-    if any(str(item).startswith('fonts') for item in result.restored):
-        delete_cached('fonts_catalog')
 
     # Reinstall plugins if requested and store manager available
     if options.reinstall_plugins and result.plugins_to_install:
@@ -159,15 +143,6 @@ def backup_restore():
             if not pid:
                 continue
             try:
-                # Only what is missing. install_plugin replaces an installed
-                # copy with a fresh download, so restoring onto the same
-                # device re-downloaded every plugin, and one installed from
-                # its own URL (not in the registry) "failed" and failed the
-                # whole restore while it sat there installed. The store's
-                # own lookup, so registry aliases count as installed too.
-                if psm and _installed_path(psm, pid) is not None:
-                    result.skipped.append(f'plugin:{pid} (installed)')
-                    continue
                 if psm and hasattr(psm, 'install_plugin'):
                     ok = psm.install_plugin(pid)
                     if ok:

web_interface/blueprints/api_v3/config.py

@@ -507,11 +507,7 @@ def save_main_config():
         # Try to get JSON data first, fallback to form data
         data = None
         if request.is_json:
-            # silent=True, as in save_raw_main_config: get_json() raised
-            # Werkzeug's BadRequest into the catch-all below, a 500.
-            data = request.get_json(silent=True)
-            if data is None and request.get_data():
-                return jsonify({'status': 'error', 'message': 'Invalid JSON in request body'}), 400
+            data = request.get_json()
             if data is not None and not isinstance(data, dict):
                 return jsonify({'status': 'error', 'message': 'Request body must be a JSON object'}), 400
         else:

web_interface/blueprints/api_v3/display.py

@@ -69,26 +69,6 @@ def _deliver_on_demand(payload):
     return 'mailbox', reason
 
 
-def _withdraw_on_demand(request_id):
-    """Take a start request the route has refused back out of the mailbox.
-
-    The display reads the mailbox for an hour without looking at a
-    request's age, so one left there after an error answer ran whenever the
-    display next started. Only this request is removed: the mailbox is
-    re-read and cleared only while it still holds this request_id, as the
-    display's _consume_on_demand_request does, so a newer request posted in
-    the meantime stays for the display to take.
-    """
-    cache = _cache_manager()
-    try:
-        current = cache.get('display_on_demand_request', max_age=3600, memory_ttl=0)
-        if isinstance(current, dict) and current.get('request_id') == request_id:
-            cache.delete('display_on_demand_request')
-    except Exception:  # the route is answering an error already
-        logger.warning("Could not withdraw on-demand request %s from the mailbox",
-                       request_id, exc_info=True)
-
-
 @api_v3.route('/display/current', methods=['GET'])
 def get_display_current():
     """The latest display preview, as the /stream/display SSE stream sends it.
@@ -279,26 +259,9 @@ def start_on_demand_display():
     }
     transport, socket_error = _deliver_on_demand(request_payload)
 
-    # A socket acknowledgement is the display itself answering: it is
-    # running and has the request queued, whatever systemd says (a display
-    # run by hand or in the emulator has no active unit). So nothing is
-    # checked or started for it -- that answered "not running" for a request
-    # that had already taken effect. The service is still reported the way
-    # _ensure_display_service_running reports a running one.
-    if transport == 'socket':
-        service_result = (dict(_get_display_service_status(), started=False)
-                          if start_service else None)
-        return _on_demand_started(request_id, resolved_plugin, resolved_mode,
-                                  duration, pinned, service_result, transport,
-                                  socket_error)
-
     service_status = _get_display_service_status()
 
     if not service_status.get('active') and not start_service:
-        # The request is in the mailbox, and the display reads it whenever
-        # it next starts: taken back out, or a request answered with this
-        # error ran later anyway.
-        _withdraw_on_demand(request_id)
         return jsonify({
             'status': 'error',
             'message': 'Display service is not running. Please start the display service or enable "Start Service" option.',
@@ -322,25 +285,16 @@ def start_on_demand_display():
         service_result = _ensure_display_service_running()
         # Check if service actually started
         if service_result and not service_result.get('active'):
-            _withdraw_on_demand(request_id)
             return jsonify({
                 'status': 'error',
                 'message': 'Failed to start display service. Please check service logs or start it manually.',
                 'service_result': service_result
             }), 500
 
-    return _on_demand_started(request_id, resolved_plugin, resolved_mode,
-                              duration, pinned, service_result, transport,
-                              socket_error)
-
-
-def _on_demand_started(request_id, plugin_id, mode, duration, pinned,
-                       service_result, transport, socket_error):
-    """The success answer of /display/on-demand/start."""
     response_data = {
         'request_id': request_id,
-        'plugin_id': plugin_id,
-        'mode': mode,
+        'plugin_id': resolved_plugin,
+        'mode': resolved_mode,
         'duration': duration,
         'pinned': pinned,
         'service': service_result,

web_interface/blueprints/api_v3/plugin_assets.py

@@ -3,12 +3,8 @@
 Routes decorate the shared `api_v3` Blueprint from the package `__init__`,
 so their endpoint names do not depend on which module they live in.
 """
-import mimetypes
-
-from flask import send_file
-
 from web_interface.blueprints.api_v3 import (
-    PROJECT_ROOT, _plugin_directory, api_v3, datetime, hashlib,
+    PROJECT_ROOT, Response, _plugin_directory, api_v3, datetime, hashlib,
     json, jsonify, logger, os, request, uuid,
 )
 from src.common.path_safety import (
@@ -235,8 +231,8 @@ def serve_plugin_static(plugin_id, file_path):
     if not requested_file.exists() or not requested_file.is_file():
         return jsonify({'status': 'error', 'message': 'File not found'}), 404
 
-    # Determine content type. Text keeps the types this route always set;
-    # anything else (an icon, a preview image) gets its own.
+    # Determine content type
+    content_type = 'text/plain'
     name = requested_file.name
     if name.endswith('.html'):
         content_type = 'text/html'
@@ -246,14 +242,12 @@ def serve_plugin_static(plugin_id, file_path):
         content_type = 'text/css'
     elif name.endswith('.json'):
         content_type = 'application/json'
-    else:
-        guessed = mimetypes.guess_type(name)[0]
-        content_type = ('text/plain' if not guessed or guessed.startswith('text/')
-                        else guessed)
 
-    # Sent as bytes. Opening it as UTF-8 text failed to decode any binary
-    # file, so an image answered 500 UnicodeDecodeError.
-    return send_file(requested_file, mimetype=content_type)
+    # Read and return file
+    with open(requested_file, 'r', encoding='utf-8') as f:
+        content = f.read()
+
+    return Response(content, mimetype=content_type)
 
 
 @api_v3.route('/plugins/assets/delete', methods=['POST'])

web_interface/blueprints/api_v3/plugin_config.py

@@ -9,15 +9,13 @@ from web_interface.blueprints.api_v3 import (
     _enhance_schema_with_core_properties, _non_plugin_id_error,
     _filter_config_by_schema, _get_schema_property,
     _hidden_array_item_property, _plugin_directory,
-    _parse_form_value_with_schema, _redact_credentials, _schema_allows_null,
-    _schema_type_is, _set_missing_booleans_to_false, _set_nested_value, api_v3,
-    datetime, deep_merge, error_response, exception_error_response,
-    find_secret_fields, json, jsonify, logger, merge_secrets, os,
-    remove_empty_secrets, request, separate_secrets, success_response,
-    validate_request_json,
+    _parse_form_value_with_schema, _schema_allows_null, _schema_type_is,
+    _set_missing_booleans_to_false, _set_nested_value, api_v3, datetime,
+    deep_merge, error_response, exception_error_response, find_secret_fields,
+    json, jsonify, logger, merge_secrets, os, remove_empty_secrets, request,
+    separate_secrets, success_response, validate_request_json,
 )
 from src.web_interface.config_arrays import coerce_array_shapes
-from src.web_interface.secret_helpers import mask_secret_fields
 from src.web_interface.validators import dedup_unique_arrays
 import web_interface.blueprints.api_v3 as _pkg
 # Read through the module rather than bound by value: tests patch these
@@ -45,12 +43,6 @@ def get_plugin_config():
                 context={'missing_params': ['plugin_id']},
                 status_code=400
             )
-        # load_config() merges config_secrets.json in, core sections
-        # included: ?plugin_id=web_auth returned the login's cookie key and
-        # password hash, and ?plugin_id=github the Plugin Store token.
-        id_error = _non_plugin_id_error(plugin_id)
-        if id_error:
-            return id_error
 
         # Get plugin configuration from config manager
         main_config = api_v3.config_manager.load_config()
@@ -60,13 +52,12 @@ def get_plugin_config():
         # missing fields, reading legacy booleans as objects first: what the
         # plugin runs with, and what posts back through the JSON save
         schema_mgr = api_v3.schema_manager
-        schema = None
         if schema_mgr:
             try:
                 from src.plugin_system.schema_manager import prepare_plugin_config
-                schema = schema_mgr.load_schema(plugin_id, use_cache=True)
                 defaults = schema_mgr.generate_default_config(plugin_id, use_cache=True)
-                plugin_config = prepare_plugin_config(plugin_config, schema, defaults)
+                plugin_config = prepare_plugin_config(
+                    plugin_config, schema_mgr.load_schema(plugin_id, use_cache=True), defaults)
             except Exception as e:
                 # Log but don't fail - defaults merge is best effort
                 logger.warning("Could not merge defaults for %s: %s", plugin_id, e)
@@ -167,17 +158,6 @@ def get_plugin_config():
                 'display_duration': 30
             }
 
-        # Secrets go out blank, as the settings page renders them (#276 added
-        # this; #330 dropped it). Blank, not the bullets GET /config/secrets
-        # uses: the save reads a blank secret as "unchanged", so this
-        # response posts back without erasing one.
-        properties = schema.get('properties') if isinstance(schema, dict) else None
-        if isinstance(properties, dict):
-            plugin_config = mask_secret_fields(plugin_config, properties)
-        else:
-            # No schema to mark them: blank whatever is named like one
-            plugin_config = _redact_credentials(plugin_config)
-
         return success_response(data=plugin_config)
     except Exception as e:
         return exception_error_response(e, ErrorCode.CONFIG_LOAD_FAILED)
@@ -203,12 +183,6 @@ def save_plugin_config():
             if error:
                 return error
             plugin_id = data['plugin_id']
-            # As reset and uninstall do: {"plugin_id": "display"} merged
-            # unvalidated values into the core display section, and an id
-            # that was not a string raised a TypeError, answered as a 500.
-            id_error = _non_plugin_id_error(plugin_id)
-            if id_error:
-                return id_error
             submitted_config = data.get('config', {})
             if not isinstance(submitted_config, dict):
                 return error_response(
@@ -227,9 +201,6 @@ def save_plugin_config():
                     'plugin_id required in query string',
                     status_code=400
                 )
-            id_error = _non_plugin_id_error(plugin_id)
-            if id_error:
-                return id_error
 
             # Load existing config as base (partial form updates should merge, not replace)
             existing_config = {}

web_interface/blueprints/api_v3/plugin_store.py

@@ -81,27 +81,6 @@ def _listed_plugin_dir(base: Path, name: str) -> Optional[Path]:
     return None
 
 
-def _enqueue_or_conflict(operation_type, plugin_id, callback):
-    """``(operation_id, None)``, or ``(None, a 409 response)``.
-
-    The queue raises ValueError when the plugin already has an operation
-    waiting or running -- a double-clicked Install, an uninstall during an
-    install. That is the caller's timing, not a server fault: it reached
-    the client as a 500, and the uninstall route recorded a failed
-    uninstall that had never started.
-    """
-    try:
-        return api_v3.operation_queue.enqueue_operation(
-            operation_type, plugin_id, operation_callback=callback), None
-    except ValueError:
-        return None, error_response(
-            ErrorCode.PLUGIN_OPERATION_CONFLICT,
-            f'Plugin {plugin_id} already has an install, update or uninstall '
-            'in progress; wait for it to finish, then try again',
-            status_code=409
-        )
-
-
 @api_v3.route('/plugins/update', methods=['POST'])
 def update_plugin():
     """Update plugin"""
@@ -423,10 +402,11 @@ def uninstall_plugin():
                                                 preserve_config=preserve_config)}
 
             # Enqueue operation
-            operation_id, conflict = _enqueue_or_conflict(
-                OperationType.UNINSTALL, plugin_id, uninstall_callback)
-            if conflict:
-                return conflict
+            operation_id = api_v3.operation_queue.enqueue_operation(
+                OperationType.UNINSTALL,
+                plugin_id,
+                operation_callback=uninstall_callback
+            )
 
             return success_response(
                 data={'operation_id': operation_id},
@@ -530,13 +510,11 @@ def install_plugin():
                     )
 
                 branch_msg = f" (branch: {branch})" if branch else ""
-                # plugin_id: the id to enable it by, and the id its config
-                # section is under (see _installed_plugin_id).
-                installed_id = _installed_plugin_id(plugin_id)
+                # plugin_id: the id to enable it by (see _installed_plugin_id).
                 return {'success': True,
                         'message': f'Plugin {plugin_id} installed successfully{branch_msg}',
-                        'plugin_id': installed_id,
-                        **_store_restart_fields('install', _plugin_enabled_in_config(installed_id))}
+                        'plugin_id': _installed_plugin_id(plugin_id),
+                        **_store_restart_fields('install', _plugin_enabled_in_config(plugin_id))}
             else:
                 error_msg = f'Failed to install plugin {plugin_id}'
                 if branch:
@@ -560,10 +538,11 @@ def install_plugin():
                 raise Exception(error_msg)
 
         # Enqueue operation
-        operation_id, conflict = _enqueue_or_conflict(
-            OperationType.INSTALL, plugin_id, install_callback)
-        if conflict:
-            return conflict
+        operation_id = api_v3.operation_queue.enqueue_operation(
+            OperationType.INSTALL,
+            plugin_id,
+            operation_callback=install_callback
+        )
 
         branch_msg = f" (branch: {branch})" if branch else ""
         return success_response(
@@ -590,11 +569,10 @@ def install_plugin():
                 )
 
             branch_msg = f" (branch: {branch})" if branch else ""
-            installed_id = _installed_plugin_id(plugin_id)
             return success_response(
                 message=f'Plugin installed successfully{branch_msg}',
-                extra={'plugin_id': installed_id,
-                       **_store_restart_fields('install', _plugin_enabled_in_config(installed_id))})
+                extra={'plugin_id': _installed_plugin_id(plugin_id),
+                       **_store_restart_fields('install', _plugin_enabled_in_config(plugin_id))})
         else:
             error_msg = f'Failed to install plugin {plugin_id}'
             if branch:

web_interface/blueprints/api_v3/plugins.py

@@ -439,10 +439,6 @@ sys.exit(proc.returncode)
                     import tempfile
                     import json as json_lib
 
-                    # The params reach the wrapper on its stdin, never in
-                    # its source: written there as `params = <JSON>`, a
-                    # true, false or null was an undefined name and the
-                    # wrapper died with a NameError before the script ran.
                     params_json = json_lib.dumps(action_params)
                     with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as wrapper:
                         wrapper.write(f'''import sys
@@ -453,9 +449,6 @@ import json
 # Set LEDMATRIX_ROOT
 os.environ['LEDMATRIX_ROOT'] = r"{PROJECT_ROOT}"
 
-# The params, as JSON on this wrapper's own stdin
-params = json.loads(sys.stdin.read())
-
 # Run the script and provide params as JSON via stdin
 proc = subprocess.Popen(
     [sys.executable, r"{script_file}"],
@@ -467,6 +460,7 @@ proc = subprocess.Popen(
 )
 
 # Send params as JSON to stdin
+params = {params_json}
 stdout, _ = proc.communicate(input=json.dumps(params), timeout=120)
 print(stdout)
 sys.exit(proc.returncode)
@@ -476,7 +470,6 @@ sys.exit(proc.returncode)
                     try:
                         result = subprocess.run(
                             ['python3', wrapper_path],
-                            input=params_json,
                             capture_output=True,
                             text=True,
                             timeout=120,

web_interface/blueprints/pages_v3.py

@@ -11,7 +11,7 @@ _SAFE_PLUGIN_ID_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
 _SAFE_WEB_UI_FILE_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}\.html$')
 _SAFE_WIDGET_NAME_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
 _SAFE_WIDGET_SCRIPT_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}\.js$')
-from src.web_interface.secret_helpers import mask_all_secret_values, mask_secret_fields
+from src.web_interface.secret_helpers import mask_secret_fields
 from src.plugin_system.schema_manager import plugin_config_defaults, prepare_plugin_config
 from src.common.path_safety import resolve_under, safe_path_component
 from src.pi5_matrix_support import is_raspberry_pi_5
@@ -623,14 +623,9 @@ def _load_raw_json_partial():
         main_config_data = pages_v3.config_manager.get_raw_file_content('main')
         # The web login section (password and token hashes) is managed in
         # General > Security, never in this editor; its save keeps it.
-        # The rest is masked, as GET /api/v3/config/secrets masks it: this
-        # page is served to anyone who can reach the port while the web
-        # login is off, and it was handing them every credential in the
-        # file. The save strips the masks and merges onto the stored file
-        # (save_raw_secrets_config), so a value left masked stays as it is.
         from web_interface.auth import strip_auth_section
-        secrets_config_data = mask_all_secret_values(strip_auth_section(
-            pages_v3.config_manager.get_raw_file_content('secrets')))
+        secrets_config_data = strip_auth_section(
+            pages_v3.config_manager.get_raw_file_content('secrets'))
         main_config_json = json.dumps(main_config_data, indent=4)
         secrets_config_json = json.dumps(secrets_config_data, indent=4)
 

web_interface/static/v3/js/widgets/plugin-order-list.js

@@ -18,9 +18,7 @@
  *   });
  *
  * The container re-renders from /api/v3/plugins/installed each init; the
- * hidden input(s) must already hold the saved order/exclusions (JSON). Saved
- * ids of disabled plugins (installed, but without a row) stay in them, in
- * their saved places; ids of plugins no longer installed are dropped.
+ * hidden input(s) must already hold the saved order/exclusions (JSON).
  */
 (function() {
     'use strict';
@@ -41,60 +39,17 @@
         const excludedInput = options.excludedInputId ? document.getElementById(options.excludedInputId) : null;
         if (!container || !orderInput) return;
 
-        // The saved lists as the inputs held them when the rows were drawn.
-        // Only enabled plugins get a row, and the inputs are rewritten from
-        // the rows, so a disabled plugin's place and exclusion have to be
-        // carried over from these: dropped, the next Display or Durations
-        // save stored the lists without it, and once re-enabled it came back
-        // at the end of the rotation and scrolling in Vegas again.
-        let savedOrder = [];
-        let savedExcluded = [];
-        // Every installed plugin's id, enabled or not, from the same
-        // response. A saved id outside it belongs to an uninstalled plugin
-        // and is dropped, as every save used to; without the list, nothing
-        // is dropped.
-        let installedIds = null;
-
-        // Saved ids of installed plugins with no row, once each. Only
-        // strings: /config/main refuses a list holding anything else, which
-        // would block every save.
-        function unlisted(saved, rowIds) {
-            const seen = new Set(rowIds);
-            return saved.filter(id => {
-                if (typeof id !== 'string' || seen.has(id)) return false;
-                if (installedIds && !installedIds.has(id)) return false;
-                seen.add(id);
-                return true;
-            });
-        }
-
         function syncInputs() {
-            const rowIds = [];
+            const order = [];
             const excluded = [];
             container.querySelectorAll('.plugin-order-item').forEach(item => {
                 const pluginId = item.dataset.pluginId;
-                rowIds.push(pluginId);
+                order.push(pluginId);
                 const checkbox = item.querySelector('.plugin-order-include');
                 if (checkbox && !checkbox.checked) excluded.push(pluginId);
             });
-            // An id without a row keeps its saved slot; the rows fill the
-            // other slots in their current order, and any rows left over
-            // (plugins not in the saved order) go last.
-            const kept = new Set(unlisted(savedOrder, rowIds));
-            const order = [];
-            let next = 0;
-            savedOrder.forEach(id => {
-                if (kept.has(id)) {
-                    order.push(id);
-                    kept.delete(id);
-                } else if (rowIds.includes(id) && next < rowIds.length) {
-                    order.push(rowIds[next++]);
-                }
-            });
-            orderInput.value = JSON.stringify(order.concat(rowIds.slice(next)));
-            if (excludedInput) {
-                excludedInput.value = JSON.stringify(excluded.concat(unlisted(savedExcluded, rowIds)));
-            }
+            orderInput.value = JSON.stringify(order);
+            if (excludedInput) excludedInput.value = JSON.stringify(excluded);
         }
 
         function setupDragAndDrop() {
@@ -149,7 +104,6 @@
             .then(data => {
                 const allPlugins = (data.data && data.data.plugins) || data.plugins || [];
                 const plugins = allPlugins.filter(p => p.enabled);
-                installedIds = new Set(allPlugins.map(p => p && p.id));
                 if (plugins.length === 0) {
                     const empty = document.createElement('p');
                     empty.className = 'text-sm text-gray-500 italic';
@@ -171,8 +125,6 @@
                 // (e.g. a saved value of "null"); normalize to arrays.
                 if (!Array.isArray(currentOrder)) currentOrder = [];
                 if (!Array.isArray(excluded)) excluded = [];
-                savedOrder = currentOrder;
-                savedExcluded = excluded;
 
                 // Saved order first, then any newly enabled plugins.
                 const orderedPlugins = [];

web_interface/templates/v3/partials/display.html

@@ -884,6 +884,7 @@ With this off a live game takes over the whole display with the full-screen scor
 // Update brightness display
 document.getElementById('brightness').addEventListener('input', function() {
     document.getElementById('brightness-value').textContent = this.value;
+    document.getElementById('brightness-display').textContent = this.value;
 });
 
 

web_interface/templates/v3/partials/general.html

@@ -404,10 +404,6 @@
                 document.getElementById('web-login-new-token-value').textContent = res.d.data.token;
                 document.getElementById('web-login-new-token').classList.remove('hidden');
                 form.reset();
-                // app.js marks a form dirty on input and clears the mark only
-                // after an htmx save; this one posts with fetch, so clear it
-                // here or a reload asks "Leave site?" about a saved token.
-                form.removeAttribute('data-dirty');
                 notify(res.d.message || 'Token created', 'success');
             }).catch(function(err) { notify('Request failed: ' + err.message, 'error'); });
         },

web_interface/templates/v3/partials/overview.html

@@ -55,40 +55,19 @@
 <script>
 (function () {
     var DISMISS_KEY = 'ledmatrix-recon-dismissed';
-    // Startup reconciliation is done within seconds of the web service
-    // starting. The route also answers done: false when its status file is
-    // missing (reconciliation raised before writing it, or /tmp was cleaned
-    // under a long-running service), which used to keep this polling every
-    // 2 s for as long as the page was open, on every tab. So: give up after
-    // a minute, and poll only while the Overview is on screen.
-    var POLL_MS = 2000;
-    var MAX_POLLS = 30;
     var _recon_timer = null;
-    var _polls = 0;
-    var _finished = false;   // done, given up, or dismissed
-    var _active = false;
-    var _inFlight = false;
 
     function checkReconciliation() {
-        _recon_timer = null;
-        _inFlight = true;
         fetch('/api/v3/plugins/reconciliation-status')
             .then(function (r) { return r.json(); })
             .then(function (resp) {
-                _inFlight = false;
-                if (_finished) return;
                 var d = resp.data || {};
                 if (!d.done) {
-                    // Reconciliation still running (or it never wrote its
-                    // status): ask again shortly, a bounded number of times.
-                    if (++_polls >= MAX_POLLS) {
-                        _finished = true;
-                        return;
-                    }
-                    if (_active) _recon_timer = setTimeout(checkReconciliation, POLL_MS);
+                    // Reconciliation still running — poll again shortly
+                    _recon_timer = setTimeout(checkReconciliation, 2000);
                     return;
                 }
-                _finished = true;
+                _recon_timer = null;
                 if (!d.unresolved || d.unresolved.length === 0) return;
                 var key = d.unresolved.map(function (i) { return i.plugin_id; }).sort().join(',');
                 if (sessionStorage.getItem(DISMISS_KEY) === key) return;
@@ -123,33 +102,13 @@
                 banner.dataset.dismissKey = key;
                 banner.style.setProperty('display', 'flex', 'important');
             })
-            .catch(function () { _inFlight = false; });
-    }
-
-    function startReconciliationPoll() {
-        _active = true;
-        if (!_finished && _recon_timer === null && !_inFlight) checkReconciliation();
-    }
-    function stopReconciliationPoll() {
-        _active = false;
-        if (_recon_timer !== null) {
-            clearTimeout(_recon_timer);
-            _recon_timer = null;
-        }
-    }
-    // Keyed apart from any other Overview registration, which a shared key
-    // would replace.
-    if (window.LEDVisibility) {
-        window.LEDVisibility.onActive('overview', startReconciliationPoll,
-                                      stopReconciliationPoll, 'overview-reconciliation');
-    } else {
-        startReconciliationPoll();
+            .catch(function () {});
     }
+    checkReconciliation();
 
     window.dismissReconciliationBanner = function () {
         var banner = document.getElementById('reconciliation-banner');
         banner.style.setProperty('display', 'none', 'important');
-        _finished = true;
         if (_recon_timer !== null) {
             clearTimeout(_recon_timer);
             _recon_timer = null;

web_interface/templates/v3/partials/tools.html

@@ -1093,12 +1093,6 @@
                 Use TLS
                 <span class="text-xs text-gray-500">(a password without TLS crosses the network in the clear)</span>
             </label>
-            <label id="mqtt-allow-insecure-row" class="${c.mqtt_tls ? 'hidden' : 'flex'} items-center gap-2 text-sm text-gray-700">
-                <input id="mqtt-allow-insecure" type="checkbox" ${c.allow_insecure_mqtt ? 'checked' : ''}
-                       class="rounded border-gray-300">
-                Allow without TLS (trusted network)
-                <span class="text-xs text-gray-500">(needed to save a password while TLS is off)</span>
-            </label>
 
             <div class="flex items-center justify-between gap-4 pt-2">
                 <p class="text-xs text-gray-500">
@@ -1138,15 +1132,6 @@
         if (clearBtn) clearBtn.addEventListener('click', () => clearMqttPassword());
         const clearTokenBtn = document.getElementById('mqtt-clear-api-token');
         if (clearTokenBtn) clearTokenBtn.addEventListener('click', () => clearMqttApiToken());
-        // The cleartext opt-in only means something while TLS is off.
-        const tlsBox = document.getElementById('mqtt-tls');
-        const allowRow = document.getElementById('mqtt-allow-insecure-row');
-        if (tlsBox && allowRow) {
-            tlsBox.addEventListener('change', () => {
-                allowRow.classList.toggle('hidden', tlsBox.checked);
-                allowRow.classList.toggle('flex', !tlsBox.checked);
-            });
-        }
     }
 
     window.loadMqttBridge = function() {
@@ -1175,9 +1160,6 @@
             on_demand_duration: val('mqtt-duration') === '' ? null : val('mqtt-duration'),
             log_level: val('mqtt-log-level'),
             mqtt_tls: !!(document.getElementById('mqtt-tls') || {}).checked,
-            // The server refuses a password with TLS off unless this is set
-            // (CWE-319); it is off until the user ticks it.
-            allow_insecure_mqtt: !!(document.getElementById('mqtt-allow-insecure') || {}).checked,
         };
         // Only send a password when one was typed; blank means "leave it alone".
         const pw = val('mqtt-password');