Compare commits

..
Author SHA1 Message Date
ChuckBuildsandClaude Opus 5 094090d138 fix(display): consume the reconcile request before serving it
Addresses the second review finding: a lost update on
_pending_plugin_reconcile.

The flag was cleared after a successful reconcile. Reconcile has already read
its config by that point, so a config change arriving mid-flight set a flag
that the trailing clear then erased -- a request that was never served, and
the newest config never reconciled. That is the same "my save did nothing"
symptom this PR exists to remove, so leaving it would have undercut the fix.

Consume the request before running it instead, and re-arm only on a retryable
failure. A change that lands during reconcile now stays set and is picked up
on the next pass.

The per-frame read stays lock-free. It is a fast path that can only produce a
false negative -- the watcher setting the flag just after it is read is seen
on the next iteration -- never a false positive that loses a request. The lock
is taken only when a reconcile is actually pending or a config change arrives.

Extracted _service_pending_reconcile() so the sequence is testable rather than
buried in run()'s loop; the review asked for a regression test that invokes
the subscriber during reconciliation, which is not reachable otherwise.

Tests: 4 new, covering a request racing in mid-reconcile, the quiet success,
the retryable-failure re-arm, and not reconciling when nothing is pending.
Two of them fail against the previous clear-after-success semantics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-22 09:33:44 -04:00
ChuckBuildsandClaude Opus 5 7156d31491 fix(display): snapshot the plugin mappings under their locks
Addresses the review finding on the cross-thread reads.

_enabled_plugin_not_running runs on the config-watcher thread and read two
mappings the render thread mutates. Catching RuntimeError was not a fix: it
turned a torn read into a coin flip between an unnecessary discovery scan and
a missed retry, which is the bug this PR exists to remove.

Both reads are now snapshots taken under the lock that guards their writes:

- plugin_manifests via a new PluginManager.discovered_plugin_ids(), which
  copies the ids while holding the existing _discovery_lock. Discovery
  rebuilds that mapping entry by entry, so an unsynchronised reader can see
  it half-populated.
- plugin_display_modes under a new controller lock, taken at the only two
  sites that mutate it (_register_loaded_plugin / _unregister_plugin).

The locks are never nested -- each snapshot is taken and released before the
next -- so this cannot deadlock against discovery, which holds _discovery_lock
while it rebuilds.

No cost on the per-frame path. Both mutation sites run during reconcile, which
is rare, and every hot-path read of plugin_display_modes is on the render
thread itself, same thread as the writes, so those stay lock-free.

Tests: the accessor returns a snapshot rather than a live view, and actually
takes the discovery lock (proved from a second thread, since an RLock is
reentrant on the owning one) so a later refactor cannot quietly drop it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-22 08:48:28 -04:00
ChuckBuildsandClaude Opus 5 4143aa958c fix(display): retry a plugin that is enabled but failed to load
A plugin whose validate_config() returns False is treated as a hard load
failure. The API then reports enabled=true, loaded=false, error=null: the
plugin is simply absent, with nothing saying why. hockey-scoreboard sat in
that state on a live rig for four days.

The recovery path existed but could not be reached. _reconcile_enabled_plugins
computes to_add = desired - current, and a plugin that failed to load is never
in current, so it stays in to_add and would be retried. But the reconcile is
queued by _enabled_set_changed(), which compares only top-level `enabled`
flags -- and the edit that actually fixes such a plugin (enabling a league,
filling in an API key) is nested inside the plugin's own config section. No
top-level flag changes, so no reconcile is queued, and the save that should
have fixed it does nothing. Only toggling some unrelated plugin -- which does
change a top-level flag -- queues the global reconcile that recovers it.

Add a second gate: queue a reconcile when a discovered plugin is enabled in
config but absent from the running set.

It is deliberately narrow rather than "reconcile on any config change".
Reconcile calls discover_plugins(), a ~39-manifest filesystem scan, and it
runs on the render thread; doing that on every config save would trade this
bug for a frame hitch. Gating on plugin_manifests also keeps non-plugin
sections that carry their own `enabled` flag (schedule, display) from
queueing a reconcile they can never satisfy. In the steady state -- every
enabled plugin loaded -- the new check is False and costs nothing.

The same valid-but-unconfigured => hard-fail shape still exists in
text-display, youtube-stats, birdnet-go, ledmatrix-flights and
mqtt-notifications; this makes all of them recoverable without a restart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
2026-08-22 08:09:11 -04:00
ChuckandClaude Opus 5 6b74506695 fix(sports): fetch odds for the games shown, not the whole schedule window (#494)
* fix(sports): fetch odds for the games shown, not the whole window

SportsUpcoming.update() walked every upcoming game in the schedule
window and called _fetch_odds() on each one inside that collection
loop, narrowing to upcoming_games_to_show only afterwards. Each call is
a separate sequential ESPN request.

The comment sitting above it said odds were fetched "only for games that
will be displayed". The only narrowing it actually applied was
show_favorite_teams_only, which is not the default, so in the usual
configuration nothing narrowed it at all.

Measured on devpi, where the football plugin has the same shape:

  467 odds requests in one 35s burst, 467 distinct events
  315 NFL + 152 college-football -- roughly a whole season
  plugin football-scoreboard operation timed out after 30.0s

The burst repeats each time the 1h odds TTL expires: 67 -> 327 -> 957 ->
1261 requests/hour across four consecutive hours. Between expiries the
cache works and the rate is zero, so this is a thundering herd on
expiry, not a caching failure.

The fetch now runs after selection, over team_games -- the list already
cut to upcoming_games_to_show. This mirrors the fix the football plugin
already carries; the shared base class never got it.

SportsLive is deliberately left as it is: it walks the raw event list
because it has to find which games are live, but only fetches odds for a
game that has already passed the is_live/is_halftime test, so its
fan-out is bounded by how many games are actually in progress. The test
pins that distinction rather than assuming it.

The test reads the AST rather than the source text, and asserts the full
set of call sites, so a new one has to be classified deliberately
instead of inheriting whichever behaviour it happens to land in. Writing
it that way is what turned up the SportsLive site, which I had missed.

Verified: reverting the fix fails the test with the offending iterable
named ("iterates over 'events'"). 525 passed, 9 skipped across the sports
and odds suites.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

* test(sports): check the odds guard structurally, not by its text

Review caught that _guards_above() collected an `if` test even when the
call sat in that if's `else`, so moving _fetch_odds() into the else of
the is_live/is_halftime test would still pass -- while fetching odds for
exactly the non-live games the guard exists to exclude.

Verifying that turned up a wider hole in the same assertion. It matched
substrings of the *unparsed source*, so a negated condition satisfied it
too:

    if not (details["is_live"] or details["is_halftime"]):
        self._fetch_odds(details)      # every non-live game

Both names still appear in that text, so `"is_live" in guards` held and
the test passed on code doing the opposite of what it claims to check.

The guard test is now structural. It walks the AST for an enclosing `if`
whose *body* (never its `else`) contains the call, and whose test
references both names without either sitting under a `not`.

Verified by mutation: fetching odds for non-live games now fails with
"does not sit in the true branch of a test requiring the game to be in
progress". Moving the call into the else of the *favourites* test still
passes, which is correct -- the game there is still live, so the
in-progress contract holds and the fan-out stays bounded by how many
games are actually in play.

525 passed, 9 skipped across the sports and odds suites.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 16:24:15 -04:00
ChuckandClaude Opus 5 5f29243e87 fix(config): make the device location the default for plugin location fields (#490)
* fix(config): make the device location the default for plugin location fields

A user in Kansas City reported their radar centred on Dallas, TX with
nothing in config.json to explain it.

The radar is the `ledmatrix-weather` plugin's `radar` mode, and it centres
on the same coordinates as every other weather mode: `forecast_data`
lat/lon, geocoded from the plugin's own `location_city` /
`location_state` / `location_country`. Those ship with schema defaults of
Dallas / Texas / US. A user who never opened the weather plugin's config
form therefore has no `location_city` on disk, and `PluginManager` merges
the schema default in at load time — so the whole plugin (not just the
radar) silently runs on Dallas. Radar is just the only mode that draws a
recognisable map and gives the mismatch away.

Meanwhile the device-wide `location` block that General settings writes
was read by nothing at all, despite its own help text promising it was
"used for weather, sunrise/sunset, and other location-based content".

`SchemaManager.generate_default_config()` now substitutes the device
`location` into the three fully-namespaced `location_*` keys before
handing defaults back, so the promise holds:

- Only `location_city` / `location_state` / `location_country` are
  substituted. A bare `state` key is left alone — `ledmatrix-elections`
  uses it for a two-letter code, and rewriting it would break that plugin.
- A value the user saved on the plugin still wins: this replaces the
  schema default, and `merge_with_defaults` puts user config on top.
- The substitution is applied on the way out of the defaults cache rather
  than into it, so changing the device location takes effect immediately.
- No config manager, no `location` block, or an unreadable config all
  fall back to the plugin's own schema defaults.

Every caller benefits: the plugin loader, the config form (which now
pre-fills the user's real city), config save, and reset-to-defaults.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GNLrSZ32FNKpHRaduKEJsg

* docs(web): name the exact plugin keys the device location seeds

Review follow-up. The General settings help text said the device location
was "the default for every plugin that asks for a city", which overstates
what the code does: only the fully-namespaced `location_city` /
`location_state` / `location_country` keys are substituted. A plugin with
a bare `city` key gets nothing — deliberately, since `ledmatrix-elections`
uses `state` for a two-letter code. The tips now name the exact keys.

Worth noting for anyone editing these: `ui.help_tip(...)` takes a
single-quoted Jinja string, so an apostrophe in the tip text has to be
escaped or written around. The wording here avoids them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GNLrSZ32FNKpHRaduKEJsg

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-21 16:22:48 -04:00
ChuckandClaude Opus 5 1fbe244e49 fix(plugins): say when discovery skips a directory (#489)
* fix(plugins): say when discovery skips a directory

A plugin can be enabled in config, enabled in plugin state, present on disk
with a valid manifest and an importable entry point -- and simply absent from
the running process, with nothing anywhere to say why.

That is not hypothetical. hockey-scoreboard on a live rig is enabled in both
places, imports cleanly when loaded by hand, and is listed in the Vegas plugin
order, but is not among the 22 plugins the process actually holds. Establishing
even that much meant comparing cache-file mtimes to find it had last run three
days earlier. The journal had nothing, because discovery does not report what
it declines to load.

Two paths were silent. A directory with no manifest.json was skipped without
comment, which is defensible until it is the thing you are trying to explain.
Quieter still, a manifest that parsed but carried no "id" was read
successfully and then dropped on the floor -- no warning, no trace, and the
plugin simply does not exist as far as the rest of the system is concerned.

Both now log a warning naming the directory and the reason.

This does not explain the rig above; its manifest has an id. It makes the next
occurrence diagnosable from the journal instead of from file timestamps.

Reverting the change fails both tests. 65 plugin-system tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

* fix(plugins): warn once per directory, not once per scan

Self-review catch. Discovery runs on every web UI page load and every config
reconcile, so warning unconditionally about an unloadable directory would put
a line in the journal each time someone opened a page -- the same log-volume
problem this change exists to help diagnose.

The skip is now reported once per directory per process. The diagnostic value
is unchanged: the reason a plugin is missing still appears in the journal,
once, where before it appeared nowhere.

Test added covering five consecutive scans producing one warning.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

* fix(plugins): one unusable manifest no longer aborts the whole scan

json.load accepts any JSON value, so a manifest.json holding null, [],
"text" or 42 parses without complaint and then raises AttributeError on
manifest.get('id'). Nothing catches that: the outer handler around the
scan takes OSError and PermissionError only.

So a single malformed manifest did not skip that one directory -- it
aborted _scan_directory_for_plugins outright, and every other plugin on
disk, however healthy, silently failed to register. Reproduced with
three directories, the middle one holding `null`:

    SCAN ABORTED -> AttributeError: 'NoneType' object has no attribute 'get'
      the two valid plugins never registered

That is the same failure this PR set out to fix, in its most severe
form: a plugin enabled in config, enabled in plugin state, present on
disk, and absent from the running process with nothing to say why --
except here it takes every other plugin with it.

A manifest that is not a JSON object is now skipped like any other
unusable directory, named once, with what it actually was:

    Skipping bad-null: its manifest.json is NoneType, not a JSON object
    Skipping bad-list: its manifest.json is list, not a JSON object
    scan returned: ['aaa-good', 'zzz-good']

Verified: removing the guard fails 6 of the 10 tests. Covers null, list,
string, int and bool, and asserts the healthy plugins either side of the
bad one still register.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 16:19:33 -04:00
15 changed files with 956 additions and 210 deletions
+1 -1
View File
@@ -18,7 +18,7 @@ tooling against it.
| `web_display_autostart` | bool, `true` | Whether the web interface service starts with the system | `scripts/utils/start_web_conditionally.py` |
| `timezone` | string, `"America/New_York"` | IANA timezone for schedules and displays | `ConfigManager.get_timezone()` |
| `target_fps` | int, `100` | Frame-rate ceiling for plugin rendering | `src/plugin_system/base_plugin.py`, `src/common/sports_scroll.py` |
| `location` | object | `city` / `state` / `country`, offered to plugins that need a location (weather, etc.) | plugins via merged config |
| `location` | object | `city` / `state` / `country`. Supplies the **default** for a plugin's own `location_city` / `location_state` / `location_country` setting, so weather, radar and friends follow this device without being configured twice. A value saved on the plugin itself still overrides it. | `SchemaManager.apply_device_location()`, then plugins via merged config |
## `schedule` — display on/off hours
+15 -2
View File
@@ -145,8 +145,7 @@ class SportsUpcoming(SportsCore):
if (game['home_abbr'] in self.favorite_teams or
game['away_abbr'] in self.favorite_teams):
favorite_games_found += 1
if self.show_odds:
self._fetch_odds(game)
# Odds are NOT fetched here -- see after selection below.
# Enhanced logging for debugging
self.logger.info(f"Found {all_upcoming_games} total upcoming games in data")
@@ -190,6 +189,20 @@ class SportsUpcoming(SportsCore):
# Limit to the specified number of upcoming games
team_games = team_games[:self.upcoming_games_to_show]
# Odds are fetched here, for the games that survived selection,
# rather than in the loop that collects them. That loop walks every
# upcoming game in the schedule window, and for a college league
# the window is enormous -- a live rig logged 946 upcoming games in
# one cycle and displayed 1 of them. The comment up there claimed
# odds were fetched "only for games that will be displayed", but
# the only narrowing it applied was show_favorite_teams_only, which
# is not the default; in the usual case nothing narrowed it at all
# and every game cost a separate ESPN request on a Pi that is also
# driving the panel.
if self.show_odds:
for game in team_games:
self._fetch_odds(game)
# Log changes or periodically
should_log = (
current_time - self.last_log_time >= self.log_interval or
-4
View File
@@ -328,10 +328,6 @@ class ScrollHelper:
elapsed_time = current_time - (self.scroll_start_time or current_time)
# The image already includes display_width padding, so we only need total_scroll_width
required_total_distance = self.total_scroll_width
# Progress telemetry, emitted every few seconds for the whole of
# every scroll. It says how far along a marquee is, which is what
# you turn debug on to watch and not something an operator needs
# in the journal on a device that scrolls all day.
self.logger.debug(
"Scroll progress: elapsed=%.2fs, target=%.2fs, total_scrolled=%.0f/%d px (%.1f%%)",
elapsed_time,
+81 -5
View File
@@ -181,6 +181,16 @@ class DisplayController:
self.plugin_modes = {} # mode -> plugin_instance mapping for plugin-first dispatch
self.mode_to_plugin_id: Dict[str, str] = {}
self.plugin_display_modes: Dict[str, List[str]] = {}
# plugin_display_modes is mutated only by _register_loaded_plugin /
# _unregister_plugin on the render thread, but the config-watcher
# thread reads it in _enabled_plugin_not_running. Both mutation sites
# run during reconcile (rare), so this lock never touches the per-frame
# path -- the hot-path reads are same-thread as the writes.
self._plugin_modes_lock = threading.Lock()
# Guards the consume-and-clear of _pending_plugin_reconcile. Only taken
# when a reconcile is actually pending or a config change arrives, both
# rare -- the per-frame path just reads the bool.
self._reconcile_flag_lock = threading.Lock()
# Per-plugin config-change callbacks, kept so we can unsubscribe a
# plugin when it is disabled live.
self._plugin_config_callbacks: Dict[str, Callable] = {}
@@ -463,7 +473,9 @@ class DisplayController:
self._refresh_config_cache(new_config)
# If a plugin was enabled/disabled, flag a reconcile for the main
# loop to apply (loading/unloading off the watcher thread is unsafe).
if self._enabled_set_changed(old_config, new_config):
if (self._enabled_set_changed(old_config, new_config)
or self._enabled_plugin_not_running(new_config)):
with self._reconcile_flag_lock:
self._pending_plugin_reconcile = True
self.config_service.subscribe(_controller_config_change)
@@ -1749,11 +1761,12 @@ class DisplayController:
# rebuilding available_modes happens here on the render thread so
# it can't race with rendering. Deferred while on-demand is active
# (the flag stays set) so we don't fight its temporary-enable.
# The lock-free read is a fast path only; it can be a false
# negative (the watcher setting the flag just after it is read
# is seen next iteration), never a false positive that loses a
# request.
if self._pending_plugin_reconcile and not self.on_demand_active:
# Only clear the flag on success -- a retryable failure
# (e.g. discovery) leaves it set so the request isn't lost.
if self._reconcile_enabled_plugins():
self._pending_plugin_reconcile = False
self._service_pending_reconcile()
if not self.available_modes:
# Nothing to render yet. Re-check _pending_plugin_reconcile
@@ -2813,6 +2826,7 @@ class DisplayController:
logger.debug("Using manifest display_modes for %s: %s", plugin_id, display_modes)
if not (isinstance(display_modes, list) and display_modes):
display_modes = [plugin_id]
with self._plugin_modes_lock:
self.plugin_display_modes[plugin_id] = list(display_modes)
# Subscribe to config changes for per-plugin hot-reload. Bind plugin_id
@@ -2847,6 +2861,7 @@ class DisplayController:
def _unregister_plugin(self, plugin_id: str) -> None:
"""Remove a plugin's modes, config subscription and instance, then
unload it. Used by live disable hot-reload."""
with self._plugin_modes_lock:
modes = self.plugin_display_modes.pop(plugin_id, [])
for mode in modes:
if mode in self.available_modes:
@@ -2892,6 +2907,67 @@ class DisplayController:
}
return enabled_map(old_config) != enabled_map(new_config)
def _service_pending_reconcile(self) -> None:
"""Consume a pending reconcile request and run it.
The request is consumed BEFORE reconciling, not cleared after. Clearing
after would drop any config change that lands while reconcile is
running: reconcile has already read its config by then, so the clear
erases a request it never served and the newest config never
reconciles -- the same "your save did nothing" failure this whole path
exists to prevent. Consuming first means such a request stays set and
is picked up on the next pass.
A retryable failure (e.g. discovery) re-arms the flag.
"""
with self._reconcile_flag_lock:
pending = self._pending_plugin_reconcile
self._pending_plugin_reconcile = False
if pending and not self._reconcile_enabled_plugins():
with self._reconcile_flag_lock:
self._pending_plugin_reconcile = True
def _enabled_plugin_not_running(self, new_config: Dict[str, Any]) -> bool:
"""True when a discovered plugin is enabled in config but not running.
``_enabled_set_changed`` compares only top-level ``enabled`` flags, which
misses the case that strands a plugin: one whose ``validate_config()``
returned False is absent from the running set, and the edit that fixes it
(enabling a league, filling in an API key) lives *nested* inside that
plugin's own section. No top-level flag changes, so no reconcile is
queued, and the save that should have fixed it appears to do nothing --
only toggling some unrelated plugin recovers it. hockey-scoreboard sat
enabled-but-absent on a live rig for four days this way.
Deliberately narrow: it fires only for ids the plugin manager has
actually discovered, so non-plugin sections that carry their own
``enabled`` flag (``schedule``, ``display``, ...) don't queue a reconcile
on every save. In the steady state -- everything enabled is loaded --
this is False and costs nothing. That matters because reconcile runs
``discover_plugins()`` on the render thread, where a needless
filesystem scan per config save would show up as a frame hitch.
Runs on the config-watcher thread, so both mappings it reads are
snapshotted under the lock that guards their writes.
"""
if self.plugin_manager is None:
return False
# Two snapshots, each taken under its own lock and never nested, so a
# half-written mapping is never observed and this can't deadlock
# against discovery (which holds the discovery lock while rebuilding).
try:
known = self.plugin_manager.discovered_plugin_ids()
except AttributeError:
# Older manager without the accessor: fall back to a plain read.
known = set(getattr(self.plugin_manager, 'plugin_manifests', ()) or ())
with self._plugin_modes_lock:
running = set(self.plugin_display_modes)
for key, value in new_config.items():
if (key in known and isinstance(value, dict)
and value.get('enabled', False) and key not in running):
return True
return False
def _reconcile_enabled_plugins(self) -> bool:
"""Load/unload plugins so the running set matches the enabled set in
config. Runs on the main display thread (never the config-watcher
+63 -7
View File
@@ -71,11 +71,15 @@ class PluginManager:
self.plugin_loader = PluginLoader(logger=self.logger)
self.plugin_executor = PluginExecutor(default_timeout=30.0, logger=self.logger)
self.state_manager = PluginStateManager(logger=self.logger)
self.schema_manager = SchemaManager(plugins_dir=self.plugins_dir, logger=self.logger)
self.schema_manager = SchemaManager(plugins_dir=self.plugins_dir, logger=self.logger,
config_manager=self.config_manager)
# Lock protecting plugin_manifests and plugin_directories from
# concurrent mutation (background reconciliation) and reads (requests).
self._discovery_lock = threading.RLock()
#: Directories already reported as unloadable, so the warning is
#: emitted once rather than on every discovery scan.
self._skip_reported: set = set()
# Lock protecting plugin_last_update from concurrent mutation/iteration.
# It's written from run_scheduled_updates()/update_all_plugins() (main
@@ -195,18 +199,59 @@ class PluginManager:
continue
manifest_path = item / "manifest.json"
if manifest_path.exists():
if not manifest_path.exists():
# Once per directory per process. Discovery runs on every
# web UI page load and every config reconcile, so warning
# unconditionally would put a line in the journal each
# time someone opened a page -- the same log-volume
# problem this is meant to help diagnose.
# A directory here that carries no manifest is not a
# plugin. Said once, because the alternative is a plugin
# that is enabled in config, enabled in plugin state,
# present on disk, and simply absent from the running
# process with nothing anywhere to say why. Working that
# out afterwards means reading cache-file mtimes.
if item.name not in self._skip_reported:
self._skip_reported.add(item.name)
self.logger.warning(
"Skipping %s: no manifest.json, so it cannot be "
"loaded as a plugin", item.name)
continue
try:
with open(manifest_path, 'r', encoding='utf-8') as f:
manifest = json.load(f)
plugin_id = manifest.get('id')
if plugin_id:
plugin_ids.append(plugin_id)
new_manifests[plugin_id] = manifest
new_directories[plugin_id] = item
except (json.JSONDecodeError, PermissionError, OSError) as e:
self.logger.warning("Error reading manifest from %s: %s", manifest_path, e, exc_info=True)
continue
# json.load accepts any JSON value, so a manifest holding
# null, [] or "text" parses and then raises AttributeError on
# .get(). Nothing here catches that -- the outer handler takes
# OSError/PermissionError only -- so a single malformed
# manifest aborted the whole scan and every other plugin on
# disk, however healthy, silently failed to register.
if not isinstance(manifest, dict):
if item.name not in self._skip_reported:
self._skip_reported.add(item.name)
self.logger.warning(
"Skipping %s: its manifest.json is %s, not a JSON "
"object", item.name, type(manifest).__name__)
continue
plugin_id = manifest.get('id')
if not plugin_id:
# Parsed but unusable. This was the quietest path of all:
# the manifest is read successfully and then dropped.
if item.name not in self._skip_reported:
self._skip_reported.add(item.name)
self.logger.warning(
"Skipping %s: its manifest.json has no \"id\", so "
"there is nothing to register it under", item.name)
continue
plugin_ids.append(plugin_id)
new_manifests[plugin_id] = manifest
new_directories[plugin_id] = item
except (OSError, PermissionError) as e:
self.logger.error("Error scanning directory %s: %s", directory, e, exc_info=True)
@@ -586,6 +631,17 @@ class PluginManager:
return self.load_plugin(plugin_id)
def discovered_plugin_ids(self) -> set:
"""Snapshot of the discovered plugin ids, taken under the discovery lock.
Callers on other threads (the config watcher) must not iterate
``plugin_manifests`` directly: discovery rebuilds it entry by entry, so
an unsynchronised reader can see a half-populated mapping or raise
"dictionary changed size during iteration".
"""
with self._discovery_lock:
return set(self.plugin_manifests)
def get_plugin(self, plugin_id: str) -> Optional[Any]:
"""
Get a loaded plugin instance by ID.
+87 -4
View File
@@ -26,7 +26,25 @@ class SchemaManager:
- Cache invalidation on plugin changes
"""
def __init__(self, plugins_dir: Optional[Path] = None, project_root: Optional[Path] = None, logger: Optional[logging.Logger] = None):
# Plugin config keys that mean "where this device is". A plugin declaring
# any of these in its schema gets the device-wide ``location`` block from
# config.json as the *default* for that field, instead of whatever city the
# plugin author happened to ship. A value the user set on the plugin itself
# always wins -- this only ever replaces the schema default, so an explicit
# per-plugin location is still honoured.
#
# Only these fully-namespaced keys are substituted. A bare ``state`` or
# ``city`` key is deliberately left alone: plugins use those for unrelated
# things (ledmatrix-elections' ``state`` is a two-letter code, not a place
# name), and silently rewriting them would break those plugins.
DEVICE_LOCATION_KEYS: Dict[str, str] = {
'location_city': 'city',
'location_state': 'state',
'location_country': 'country',
}
def __init__(self, plugins_dir: Optional[Path] = None, project_root: Optional[Path] = None,
logger: Optional[logging.Logger] = None, config_manager: Optional[Any] = None):
"""
Initialize the Schema Manager.
@@ -34,10 +52,14 @@ class SchemaManager:
plugins_dir: Base plugins directory path
project_root: Project root directory path
logger: Optional logger instance
config_manager: Optional config manager, used to resolve the
device-wide ``location`` that seeds plugin location defaults.
Omitting it simply leaves schema defaults untouched.
"""
self.logger = logger or logging.getLogger(__name__)
self.plugins_dir = plugins_dir
self.project_root = project_root or Path.cwd()
self.config_manager = config_manager
# Schema cache: plugin_id -> schema dict
self._schema_cache: Dict[str, Dict[str, Any]] = {}
@@ -212,10 +234,70 @@ class SchemaManager:
return defaults
def get_device_location(self) -> Optional[Dict[str, Any]]:
"""
Return the device-wide ``location`` block from config.json, or None.
This is the City/State/Country the user sets once under General
settings. Returns None when there is no config manager wired, the
config can't be read, or no location has been configured.
"""
if self.config_manager is None:
return None
try:
config = self.config_manager.load_config()
except Exception as e:
# A config that can't be read must never stop defaults being
# generated -- the plugin's own schema defaults still apply.
self.logger.debug(f"Could not read device location from config: {e}")
return None
if not isinstance(config, dict):
return None
location = config.get('location')
return location if isinstance(location, dict) else None
def apply_device_location(self, defaults: Dict[str, Any]) -> Dict[str, Any]:
"""
Replace location-shaped schema defaults with the device's own location.
Without this, a plugin that ships ``"location_city": "Dallas"`` as its
schema default silently reports Dallas weather (and centres its radar
there) for every user who never opened that plugin's config form --
even though they set their real city under General settings. The
substituted value is still only a *default*: ``merge_with_defaults``
lets any per-plugin value the user saved win over it.
Mutates and returns ``defaults`` for convenience.
"""
if not defaults:
return defaults
if not any(key in defaults for key in self.DEVICE_LOCATION_KEYS):
return defaults
location = self.get_device_location()
if not location:
return defaults
for key, field in self.DEVICE_LOCATION_KEYS.items():
if key not in defaults:
continue
value = location.get(field)
# Only a non-empty string is a real answer; a blank or missing
# field means "not configured", which leaves the schema default.
if isinstance(value, str) and value.strip():
defaults[key] = value.strip()
return defaults
def generate_default_config(self, plugin_id: str, use_cache: bool = True) -> Dict[str, Any]:
"""
Generate default configuration for a plugin from its schema.
Location fields (see ``DEVICE_LOCATION_KEYS``) default to the device's
configured location rather than the plugin author's. That substitution
is applied on the way out rather than being cached, so changing the
device location takes effect without invalidating the defaults cache.
Args:
plugin_id: Plugin identifier
use_cache: If True, return cached defaults if available
@@ -225,7 +307,7 @@ class SchemaManager:
"""
# Check cache first
if use_cache and plugin_id in self._defaults_cache:
return self._defaults_cache[plugin_id].copy()
return self.apply_device_location(self._defaults_cache[plugin_id].copy())
schema = self.load_schema(plugin_id, use_cache=use_cache)
if not schema:
@@ -249,10 +331,11 @@ class SchemaManager:
if 'live_priority' not in defaults:
defaults['live_priority'] = schema.get('properties', {}).get('live_priority', {}).get('default', False)
# Cache the defaults
# Cache the defaults *before* the device location is layered on, so a
# later change to the device location is picked up by the next call.
self._defaults_cache[plugin_id] = defaults.copy()
return defaults
return self.apply_device_location(defaults)
def validate_config_against_schema(self, config: Dict[str, Any], schema: Dict[str, Any],
plugin_id: Optional[str] = None) -> Tuple[bool, List[str]]:
+7 -71
View File
@@ -31,18 +31,6 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
#: Degradation threshold, as a fraction of target_fps. A marquee jitters a
#: little all the time, so "anything under target" would report constantly and
#: mean nothing; 90% of target is the point where a shortfall is real. At a
#: 60fps target that is 54fps -- 55fps is a normal wobble and stays at DEBUG,
#: which is deliberate, not an off-by-one.
_FPS_HEALTHY_FRACTION = 0.9
#: A healthy marquee still reports this often, so silence means stopped
#: rather than fine.
_FPS_HEARTBEAT_INTERVAL = 300.0
def _percentile(ordered: List[float], fraction: float) -> float:
"""Nearest-rank percentile of an already-sorted list.
@@ -108,11 +96,6 @@ class VegasModeCoordinator:
self._is_active = False
self._is_paused = False
self._should_stop = False
# Frame-rate health, tracked across run_iteration() calls so the
# heartbeat is one-per-interval rather than one-per-cycle, and so a
# recovery spanning two cycles is still reported. Reset on start().
self._fps_last_health_log = 0.0
self._fps_was_degraded = False
self._state_lock = threading.Lock()
# Live priority tracking
@@ -265,11 +248,6 @@ class VegasModeCoordinator:
self._is_active = True
self._should_stop = False
self._start_time = time.time()
# A fresh run starts with a clean health slate: no stale
# "was degraded" from the previous run, and a heartbeat that is
# due immediately so the first sample confirms the marquee is up.
self._fps_last_health_log = 0.0
self._fps_was_degraded = False
# Line up the next group immediately, so the first extension is already
# warm rather than stalling the scroll to fetch it.
@@ -417,18 +395,8 @@ class VegasModeCoordinator:
duration = self.render_pipeline.get_dynamic_duration()
start_time = time.time()
frame_count = 0
fps_log_interval = 5.0 # Sample FPS every 5 seconds
# Health state lives on the coordinator, not here: run_iteration() is
# called once per cycle, so locals reset every few seconds. That made
# `last_fps_health_log = 0.0` fire the "heartbeat" on the first sample
# of every iteration rather than once per interval, and a recovery
# that crossed an iteration boundary was never reported at all --
# was_degraded had already gone back to False.
# Monotonic, and deliberately not start_time: start_time is wall
# clock and is used below to report the iteration's duration. Mixing
# the two here would make every delta hugely negative and silence the
# frame-rate reporting altogether.
last_fps_log_time = time.monotonic()
fps_log_interval = 5.0 # Log FPS every 5 seconds
last_fps_log_time = start_time
fps_frame_count = 0
# A mean hides stutter completely. At 120fps a five-second window is
# ~600 frames, so a 200ms freeze -- plainly visible on a marquee --
@@ -440,13 +408,7 @@ class VegasModeCoordinator:
logger.info("Starting Vegas iteration for %.1fs", duration)
while True:
# Monotonic, like the FPS window below. These devices have no RTC,
# so the wall clock jumps by however wrong boot time was the moment
# NTP first syncs. A backward jump makes frame_elapsed negative,
# and `frame_interval - frame_elapsed` then sleeps for longer than
# the whole budget -- the render loop stalls for the size of the
# correction. A forward jump inflates p99 and worst-frame instead.
frame_started = time.monotonic()
frame_started = time.time()
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
@@ -474,7 +436,7 @@ class VegasModeCoordinator:
# quarter of the budget spent not rendering. Subtracting the work
# already done keeps the pacing target while reclaiming that time,
# and yields the GIL either way so other threads still run.
frame_elapsed = time.monotonic() - frame_started
frame_elapsed = time.time() - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
# Measured before the sleep: time spent working, not pacing.
@@ -486,42 +448,16 @@ class VegasModeCoordinator:
frame_count += 1
fps_frame_count += 1
# Periodic FPS logging. Reported at INFO only when the frame rate
# is actually worth an operator's attention -- a shortfall against
# target, or the recovery from one -- with a slow heartbeat so a
# healthy marquee still shows a pulse.
#
# Measured over two hours on a running rig: 1410 samples, 98.5%
# of them within 10% of target. The 1.5% that were not included a
# reading of 8.6fps against a target of 60 -- a real stall, and
# completely invisible inside 1389 lines reading "59.6".
# Monotonic: every use of this value in the block below is a
# duration, and these devices have no RTC, so the wall clock jumps
# by however wrong boot time was the moment NTP first syncs. That
# would not only mis-fire the heartbeat, it would corrupt the
# frame rate itself, since fps is frames divided by this delta.
current_time = time.monotonic()
# Periodic FPS logging
current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
p99 = _percentile(sorted(frame_times), 0.99)
target = self.vegas_config.target_fps
degraded = target > 0 and fps < target * _FPS_HEALTHY_FRACTION
due = (current_time - self._fps_last_health_log
>= _FPS_HEARTBEAT_INTERVAL)
if degraded or self._fps_was_degraded or due:
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, target, fps_frame_count,
fps, self.vegas_config.target_fps, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
self._fps_last_health_log = current_time
else:
logger.debug(
"Vegas FPS: %.1f (target: %d, frames: %d) p99 %.1fms worst %.1fms",
fps, target, fps_frame_count,
p99 * 1000.0, frame_worst * 1000.0
)
self._fps_was_degraded = degraded
last_fps_log_time = current_time
fps_frame_count = 0
frame_worst = 0.0
@@ -6,6 +6,7 @@ These tests cover the reconcile path that loads/unloads plugins and rebuilds
the dispatch maps on the main thread when the enabled set changes.
"""
import copy
from unittest.mock import MagicMock
@@ -253,3 +254,182 @@ class TestEnabledSetChanged:
{"a": {"enabled": True, "duration": 30}},
{"a": {"enabled": True, "duration": 45}},
) is False
class TestEnabledPluginNotRunning:
"""A plugin that fails validate_config() is enabled but absent, and the
config edit that fixes it is nested inside the plugin's own section -- so
the top-level ``enabled`` comparison never sees it. These cover the second
gate that queues a reconcile in that case.
"""
def test_nested_edit_is_invisible_to_the_enabled_set_check(self, test_display_controller):
"""The original gate: proves why a second one is needed."""
controller = test_display_controller
old = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": False}}}
new = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": True}}}
# Enabling a league changes no top-level flag.
assert controller._enabled_set_changed(old, new) is False
def test_queues_reconcile_when_enabled_plugin_is_absent(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {} # failed to load
cfg = {"hockey-scoreboard": {"enabled": True, "nhl": {"enabled": True}}}
assert controller._enabled_plugin_not_running(cfg) is True
def test_quiet_when_every_enabled_plugin_is_running(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
cfg = {"hockey-scoreboard": {"enabled": True}}
assert controller._enabled_plugin_not_running(cfg) is False
def test_disabled_plugin_does_not_queue(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {}
cfg = {"hockey-scoreboard": {"enabled": False}}
assert controller._enabled_plugin_not_running(cfg) is False
def test_non_plugin_sections_do_not_queue(self, test_display_controller):
"""``schedule``/``display`` carry their own ``enabled`` and are never
in plugin_display_modes -- without the manifest check they would queue
a reconcile, and therefore a filesystem scan, on every config save."""
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
cfg = {
"hockey-scoreboard": {"enabled": True},
"schedule": {"enabled": True},
"display": {"enabled": True},
}
assert controller._enabled_plugin_not_running(cfg) is False
def test_non_dict_section_is_ignored(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {}
assert controller._enabled_plugin_not_running({"hockey-scoreboard": "nonsense"}) is False
def test_no_plugin_manager_is_quiet(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager = None
assert controller._enabled_plugin_not_running({"x": {"enabled": True}}) is False
class TestReconcileQueuedThroughSubscriber:
"""End-to-end through the real config-change subscriber, not the helper.
Without the second gate this is the four-day-outage path: the plugin is
enabled, absent, and the save that enables its league sets no flag.
"""
@staticmethod
def _subscriber(controller):
subs = controller.config_service._subscribers['*']
for cb in subs:
if getattr(cb, '__name__', '') == '_controller_config_change':
return cb
raise AssertionError(f"controller subscriber not found among {subs}")
@staticmethod
def _configs(controller, plugin_section_old, plugin_section_new):
"""Build two full configs differing only inside the plugin section --
the subscriber refreshes its cache from these, so they must be real."""
base = copy.deepcopy(controller.config)
old = copy.deepcopy(base)
new = copy.deepcopy(base)
old["hockey-scoreboard"] = plugin_section_old
new["hockey-scoreboard"] = plugin_section_new
return old, new
def test_nested_edit_queues_reconcile_for_absent_plugin(self, test_display_controller):
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {} # validate_config() said False
controller._pending_plugin_reconcile = False
old, new = self._configs(
controller,
{"enabled": True, "nhl": {"enabled": False}},
{"enabled": True, "nhl": {"enabled": True}},
)
# The original gate is blind to this edit ...
assert controller._enabled_set_changed(old, new) is False
self._subscriber(controller)(old, new)
# ... but the reconcile is queued anyway.
assert controller._pending_plugin_reconcile is True
def test_steady_state_does_not_queue_reconcile(self, test_display_controller):
"""Everything enabled is running: an unrelated edit must not queue a
reconcile, or every config save drags a filesystem scan onto the
render thread."""
controller = test_display_controller
controller.plugin_manager.plugin_manifests = {"hockey-scoreboard": {}}
controller.plugin_manager.discovered_plugin_ids.return_value = {"hockey-scoreboard"}
controller.plugin_display_modes = {"hockey-scoreboard": ["nhl"]}
controller._pending_plugin_reconcile = False
old, new = self._configs(
controller,
{"enabled": True, "scroll_speed": 1},
{"enabled": True, "scroll_speed": 2},
)
self._subscriber(controller)(old, new)
assert controller._pending_plugin_reconcile is False
class TestPendingReconcileNotLost:
"""A config change arriving *during* reconcile must not be discarded.
The flag used to be cleared after a successful reconcile. Reconcile has
already read its config by then, so that clear erased a request it never
served and the newest config never reconciled -- the same "my save did
nothing" symptom this path exists to prevent.
"""
def test_request_arriving_during_reconcile_survives(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
def reconcile_and_race():
# The watcher thread queues another change while we are mid-flight.
with controller._reconcile_flag_lock:
controller._pending_plugin_reconcile = True
return True
controller._reconcile_enabled_plugins = reconcile_and_race
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is True, \
"a config change landing during reconcile was discarded"
def test_flag_cleared_on_a_quiet_success(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
controller._reconcile_enabled_plugins = lambda: True
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is False
def test_retryable_failure_rearms(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = True
controller._reconcile_enabled_plugins = lambda: False
controller._service_pending_reconcile()
assert controller._pending_plugin_reconcile is True
def test_no_reconcile_when_nothing_pending(self, test_display_controller):
controller = test_display_controller
controller._pending_plugin_reconcile = False
calls = []
controller._reconcile_enabled_plugins = lambda: calls.append(1) or True
controller._service_pending_reconcile()
assert calls == []
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env python3
"""Discovery must say when it skips a directory.
A plugin can be enabled in config, enabled in plugin state, present on disk
with a valid entry point -- and simply absent from the running process, with
nothing in the journal to say why. Working that out afterwards meant comparing
cache-file mtimes to find when it had last run.
Two paths were silent. A directory with no manifest.json was ignored, and --
quieter still -- a manifest that parsed but carried no "id" was read
successfully and then dropped on the floor.
"""
import json
import logging
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
def _manager(tmp_path):
pm = PluginManager.__new__(PluginManager)
pm.plugins_dir = tmp_path
pm.logger = logging.getLogger("test.discovery")
pm.plugin_manifests = {}
pm.plugin_directories = {}
pm._discovery_lock = __import__("threading").RLock()
pm._skip_reported = set()
pm.schema_manager = MagicMock()
return pm
def test_a_directory_without_a_manifest_is_reported(tmp_path, caplog):
(tmp_path / "not-a-plugin").mkdir()
pm = _manager(tmp_path)
with caplog.at_level(logging.WARNING, logger="test.discovery"):
pm._scan_directory_for_plugins(tmp_path)
joined = " ".join(r.message for r in caplog.records)
assert "not-a-plugin" in joined and "manifest" in joined, (
f"skip was silent; log said: {joined!r}")
def test_a_manifest_without_an_id_is_reported(tmp_path, caplog):
d = tmp_path / "idless"
d.mkdir()
(d / "manifest.json").write_text(json.dumps({"name": "No Id", "version": "1.0.0"}))
pm = _manager(tmp_path)
with caplog.at_level(logging.WARNING, logger="test.discovery"):
pm._scan_directory_for_plugins(tmp_path)
joined = " ".join(r.message for r in caplog.records)
assert "idless" in joined and "id" in joined, (
f"a parsed-but-unusable manifest vanished silently; log said: {joined!r}")
def test_a_good_plugin_still_registers(tmp_path, caplog):
d = tmp_path / "real-plugin"
d.mkdir()
(d / "manifest.json").write_text(json.dumps(
{"id": "real-plugin", "name": "Real", "version": "1.0.0"}))
pm = _manager(tmp_path)
pm._scan_directory_for_plugins(tmp_path)
assert "real-plugin" in pm.plugin_manifests, "a valid plugin was not registered"
def test_the_warning_does_not_repeat_on_every_scan(tmp_path, caplog):
"""Discovery runs on every web UI page load and every config reconcile.
Warning unconditionally would put a line in the journal each time someone
opened a page -- the same log-volume problem this is meant to help
diagnose.
"""
(tmp_path / "not-a-plugin").mkdir()
pm = _manager(tmp_path)
with caplog.at_level(logging.WARNING, logger="test.discovery"):
for _ in range(5):
pm._scan_directory_for_plugins(tmp_path)
hits = [r for r in caplog.records if "not-a-plugin" in r.message]
assert len(hits) == 1, f"warned {len(hits)} times across 5 scans"
def _plugin(tmp_path, name, body):
d = tmp_path / name
d.mkdir()
(d / "manifest.json").write_text(json.dumps(body))
return d
VALID = {"name": "V", "version": "1.0.0", "class_name": "X", "display_modes": ["m"]}
@pytest.mark.parametrize("body", [None, [1, 2], "not an object", 42, True])
def test_a_manifest_that_is_not_an_object_is_skipped_not_fatal(tmp_path, caplog, body):
"""json.load accepts any JSON value, not just objects.
manifest.get('id') then raised AttributeError, which nothing here caught --
the outer handler takes OSError/PermissionError only. A single malformed
manifest aborted the entire scan, so every other plugin on disk, however
healthy, silently failed to register.
"""
_plugin(tmp_path, "aaa-good", dict(VALID, id="aaa-good"))
_plugin(tmp_path, "mmm-bad", body)
_plugin(tmp_path, "zzz-good", dict(VALID, id="zzz-good"))
pm = _manager(tmp_path)
with caplog.at_level(logging.WARNING, logger="test.discovery"):
found = pm._scan_directory_for_plugins(tmp_path)
assert sorted(found) == ["aaa-good", "zzz-good"], (
"one unusable manifest took the healthy plugins down with it")
joined = " ".join(r.message for r in caplog.records)
assert "mmm-bad" in joined, f"the skip was silent; log said: {joined!r}"
def test_the_bad_manifest_is_named_with_what_it_actually_was(tmp_path, caplog):
_plugin(tmp_path, "listy", [1, 2])
pm = _manager(tmp_path)
with caplog.at_level(logging.WARNING, logger="test.discovery"):
pm._scan_directory_for_plugins(tmp_path)
joined = " ".join(r.message for r in caplog.records)
assert "listy" in joined and "list" in joined, (
f"the warning does not say what the manifest was: {joined!r}")
@@ -0,0 +1,63 @@
"""Tests for PluginManager.discovered_plugin_ids().
The config-watcher thread needs the set of discovered plugin ids while the
render thread may be rebuilding plugin_manifests. Iterating that dict directly
can observe a half-populated mapping or raise "dictionary changed size during
iteration", so the accessor snapshots it under the discovery lock.
"""
import tempfile
import threading
from pathlib import Path
import pytest
from src.plugin_system.plugin_manager import PluginManager
@pytest.fixture
def pm():
with tempfile.TemporaryDirectory() as tmp:
yield PluginManager(plugins_dir=str(Path(tmp) / "plugins"))
def test_returns_the_discovered_ids(pm):
pm.plugin_manifests = {"clock-simple": {}, "hockey-scoreboard": {}}
assert pm.discovered_plugin_ids() == {"clock-simple", "hockey-scoreboard"}
def test_empty_when_nothing_discovered(pm):
pm.plugin_manifests = {}
assert pm.discovered_plugin_ids() == set()
def test_is_a_snapshot_not_a_live_view(pm):
"""The caller iterates the result on another thread; it must not alias
the mapping discovery is still writing to."""
pm.plugin_manifests = {"clock-simple": {}}
snapshot = pm.discovered_plugin_ids()
pm.plugin_manifests["hockey-scoreboard"] = {}
assert snapshot == {"clock-simple"}
def test_takes_the_discovery_lock(pm):
"""Guards against the lock being dropped in a later refactor: with the
lock held by another thread the call must block rather than read."""
pm.plugin_manifests = {"clock-simple": {}}
finished = threading.Event()
def call():
pm.discovered_plugin_ids()
finished.set()
pm._discovery_lock.acquire()
try:
# RLock is reentrant per-thread, so use a *different* thread to prove
# the accessor actually waits on it.
t = threading.Thread(target=call, daemon=True)
t.start()
assert not finished.wait(timeout=0.3), "accessor did not take the discovery lock"
finally:
pm._discovery_lock.release()
t.join(timeout=2)
assert finished.is_set()
+179
View File
@@ -0,0 +1,179 @@
"""
Tests for the device-location default: a plugin that ships a location field in
its schema must default to the device's configured City/State/Country, not to
whatever place the plugin author hard-coded.
The bug this pins: ledmatrix-weather ships ``"location_city": "Dallas"`` as a
schema default, so a user who set Kansas City under General settings but never
opened the weather plugin's own config form got Dallas weather — and a radar
centred on Dallas with nothing in config.json to explain it.
"""
import json
import pytest
from src.plugin_system.schema_manager import SchemaManager
class FakeConfigManager:
"""Minimal stand-in exposing the load_config() SchemaManager relies on."""
def __init__(self, config):
self.config = config
self.load_count = 0
def load_config(self):
self.load_count += 1
return self.config
class ExplodingConfigManager:
def load_config(self):
raise OSError("config.json is unreadable")
WEATHER_SCHEMA = {
"type": "object",
"properties": {
"location_city": {"type": "string", "default": "Dallas"},
"location_state": {"type": "string", "default": "Texas"},
"location_country": {"type": "string", "default": "US"},
"units": {"type": "string", "default": "imperial"},
},
}
def write_plugin(plugins_dir, plugin_id, schema):
plugin_dir = plugins_dir / plugin_id
plugin_dir.mkdir(parents=True, exist_ok=True)
(plugin_dir / "config_schema.json").write_text(json.dumps(schema))
return plugin_dir
@pytest.fixture
def plugins_dir(tmp_path):
d = tmp_path / "plugin-repos"
d.mkdir()
return d
def make_sm(plugins_dir, tmp_path, location):
config = {} if location is None else {"location": location}
cm = FakeConfigManager(config)
sm = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path,
config_manager=cm)
return sm, cm
class TestDeviceLocationDefaults:
def test_device_location_replaces_plugin_default(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm, _ = make_sm(plugins_dir, tmp_path,
{"city": "Kansas City", "state": "Missouri", "country": "US"})
defaults = sm.generate_default_config("ledmatrix-weather")
assert defaults["location_city"] == "Kansas City"
assert defaults["location_state"] == "Missouri"
assert defaults["location_country"] == "US"
# Non-location defaults are untouched.
assert defaults["units"] == "imperial"
def test_user_set_plugin_value_still_wins(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm, _ = make_sm(plugins_dir, tmp_path,
{"city": "Kansas City", "state": "Missouri", "country": "US"})
defaults = sm.generate_default_config("ledmatrix-weather")
merged = sm.merge_with_defaults({"location_city": "Denver"}, defaults)
assert merged["location_city"] == "Denver"
# Fields the user did not override still follow the device.
assert merged["location_state"] == "Missouri"
def test_blank_and_missing_device_fields_leave_schema_default(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm, _ = make_sm(plugins_dir, tmp_path, {"city": "Kansas City", "state": " "})
defaults = sm.generate_default_config("ledmatrix-weather")
assert defaults["location_city"] == "Kansas City"
assert defaults["location_state"] == "Texas" # blank -> not configured
assert defaults["location_country"] == "US" # absent -> schema default
def test_no_device_location_configured_is_a_no_op(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm, _ = make_sm(plugins_dir, tmp_path, None)
defaults = sm.generate_default_config("ledmatrix-weather")
assert defaults["location_city"] == "Dallas"
def test_no_config_manager_is_a_no_op(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path)
assert sm.generate_default_config("ledmatrix-weather")["location_city"] == "Dallas"
def test_unreadable_config_falls_back_to_schema_defaults(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path,
config_manager=ExplodingConfigManager())
assert sm.generate_default_config("ledmatrix-weather")["location_city"] == "Dallas"
class TestScopedToNamespacedKeys:
def test_bare_state_key_is_not_rewritten(self, plugins_dir, tmp_path):
"""ledmatrix-elections' ``state`` is a two-letter code, not a place name."""
write_plugin(plugins_dir, "ledmatrix-elections", {
"type": "object",
"properties": {
"state": {"type": "string", "default": "CA"},
"city": {"type": "string", "default": "Springfield"},
},
})
sm, _ = make_sm(plugins_dir, tmp_path,
{"city": "Kansas City", "state": "Missouri", "country": "US"})
defaults = sm.generate_default_config("ledmatrix-elections")
assert defaults["state"] == "CA"
assert defaults["city"] == "Springfield"
def test_plugin_without_location_fields_never_reads_config(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "clock-simple", {
"type": "object",
"properties": {"format": {"type": "string", "default": "12h"}},
})
sm, cm = make_sm(plugins_dir, tmp_path, {"city": "Kansas City"})
defaults = sm.generate_default_config("clock-simple")
assert defaults["format"] == "12h"
assert cm.load_count == 0
class TestCachingStaysFresh:
def test_location_change_is_picked_up_through_the_defaults_cache(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm, cm = make_sm(plugins_dir, tmp_path, {"city": "Kansas City"})
assert sm.generate_default_config("ledmatrix-weather")["location_city"] == "Kansas City"
cm.config["location"]["city"] = "Omaha"
# Second call is served from the defaults cache, but must not serve a
# stale location.
assert sm.generate_default_config("ledmatrix-weather")["location_city"] == "Omaha"
def test_cached_defaults_are_not_mutated_by_the_overlay(self, plugins_dir, tmp_path):
write_plugin(plugins_dir, "ledmatrix-weather", WEATHER_SCHEMA)
sm, cm = make_sm(plugins_dir, tmp_path, {"city": "Kansas City"})
sm.generate_default_config("ledmatrix-weather")
assert sm._defaults_cache["ledmatrix-weather"]["location_city"] == "Dallas"
cm.config.pop("location")
assert sm.generate_default_config("ledmatrix-weather")["location_city"] == "Dallas"
+136
View File
@@ -0,0 +1,136 @@
"""Odds must be fetched for the games shown, not every game in the window.
SportsUpcoming.update() collected every upcoming game in the schedule window
and called _fetch_odds() on each one *inside* that collection loop, narrowing
to upcoming_games_to_show only afterwards. The comment there said odds were
fetched "only for games that will be displayed", but the sole narrowing it
applied was show_favorite_teams_only, which is not the default -- so in the
usual configuration nothing narrowed it at all.
Measured on a live rig: a college league produced 946 upcoming games in one
cycle and displayed 1 of them. The same shape on the football plugin produced
a burst of 467 sequential ESPN requests that ran for 35s and blew that
plugin's 30s update budget, and it repeats every time the 1h odds TTL expires.
SportsLive is deliberately different: it walks the raw event list because it
has to find which games are live, but only fetches odds for a game that has
already passed the is_live/is_halftime test, so the fan-out is bounded by how
many games are actually in progress.
"""
import ast
from pathlib import Path
import pytest
MODES = (Path(__file__).resolve().parent.parent
/ "src" / "base_classes" / "sports" / "modes.py")
TREE = ast.parse(MODES.read_text(encoding="utf-8"))
def _fetch_sites():
"""(class name, method name, lineno) for every self._fetch_odds(...) call."""
calls = [n.lineno for n in ast.walk(TREE)
if isinstance(n, ast.Call) and isinstance(n.func, ast.Attribute)
and n.func.attr == "_fetch_odds"]
sites = []
for cls in [n for n in ast.walk(TREE) if isinstance(n, ast.ClassDef)]:
for fn in [n for n in cls.body if isinstance(n, ast.FunctionDef)]:
for lineno in calls:
if fn.lineno <= lineno <= (fn.end_lineno or fn.lineno):
sites.append((cls.name, fn.name, lineno))
assert len(sites) == len(calls), "a _fetch_odds call sits outside any method"
return sites
def _innermost_loop_iterable(lineno):
best = None
for node in ast.walk(TREE):
if isinstance(node, ast.For) and \
node.lineno <= lineno <= (node.end_lineno or node.lineno):
if best is None or node.lineno > best.lineno:
best = node
return None if best is None else ast.unparse(best.iter)
def _spans(body, lineno):
"""True when `lineno` falls inside this list of statements."""
return any(n.lineno <= lineno <= (n.end_lineno or n.lineno) for n in body)
def _parents(tree):
table = {}
for node in ast.walk(tree):
for child in ast.iter_child_nodes(node):
table[child] = node
return table
PARENTS = _parents(TREE)
def _mentions_positively(test, names):
"""True when `test` references every name, none of them under a `not`.
Structural, not textual. Matching the unparsed source would accept
`not (details["is_live"] or details["is_halftime"])` -- which selects
exactly the non-live games this guard exists to exclude -- because the
names still appear in the text.
"""
found = set()
for node in ast.walk(test):
if not (isinstance(node, ast.Constant) and node.value in names):
continue
negated = False
cursor = node
while cursor is not test and cursor in PARENTS:
cursor = PARENTS[cursor]
if isinstance(cursor, ast.UnaryOp) and isinstance(cursor.op, ast.Not):
negated = True
break
if not negated:
found.add(node.value)
return found >= set(names)
def _guarded_by_positive(lineno, names):
"""True when some enclosing `if` runs this line only if `names` hold.
Only the TRUE branch counts: an `if` whose `else` contains the call would
otherwise look like a guard while doing the opposite.
"""
for node in ast.walk(TREE):
if isinstance(node, ast.If) and _spans(node.body, lineno) \
and _mentions_positively(node.test, names):
return True
return False
def test_every_fetch_site_is_accounted_for():
"""A new call site must be classified deliberately, not inherited silently."""
found = {(cls, fn) for cls, fn, _ in _fetch_sites()}
assert found == {("SportsUpcoming", "update"), ("SportsLive", "update")}, (
f"unexpected _fetch_odds call sites: {sorted(found)}. Each one is a "
"sequential ESPN request per game -- classify it here on purpose.")
def test_upcoming_fetches_only_the_selected_games():
for cls, _fn, lineno in _fetch_sites():
if cls != "SportsUpcoming":
continue
iterable = _innermost_loop_iterable(lineno)
assert iterable == "team_games", (
f"SportsUpcoming._fetch_odds at line {lineno} iterates over "
f"{iterable!r}. It must run over team_games -- already narrowed to "
"upcoming_games_to_show -- not over every event in the schedule "
"window. Each item costs one sequential ESPN request.")
def test_live_only_fetches_for_games_actually_in_progress():
for cls, _fn, lineno in _fetch_sites():
if cls != "SportsLive":
continue
assert _guarded_by_positive(lineno, {"is_live", "is_halftime"}), (
f"SportsLive._fetch_odds at line {lineno} does not sit in the true "
"branch of a test requiring the game to be in progress. Without "
"that, it fans out across the whole event list -- one sequential "
"ESPN request per game.")
-99
View File
@@ -1,99 +0,0 @@
"""Frame pacing and FPS health reporting must not depend on the wall clock.
These devices have no RTC, so the system clock jumps by however wrong boot
time was the moment NTP first syncs. The render loop sleeps the *remainder*
of each frame budget:
frame_elapsed = <now> - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
With a wall-clock `now`, a backward jump makes frame_elapsed negative, so
`frame_interval - frame_elapsed` exceeds the whole budget and the render loop
stalls for the size of the correction. A forward jump instead inflates the
p99 and worst-frame numbers the telemetry reports.
"""
import ast
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
COORD = (Path(__file__).resolve().parent.parent
/ "src" / "vegas_mode" / "coordinator.py")
TREE = ast.parse(COORD.read_text(encoding="utf-8"))
def _assignments_of(name):
"""Every `name = <expr>` in the module, as unparsed source."""
out = []
for node in ast.walk(TREE):
if isinstance(node, ast.Assign):
for target in node.targets:
if isinstance(target, ast.Name) and target.id == name:
out.append((node.lineno, ast.unparse(node.value)))
return out
def test_per_frame_timestamps_are_monotonic():
for name in ("frame_started", "frame_elapsed"):
assigns = _assignments_of(name)
assert assigns, f"{name} is no longer assigned -- has the loop changed?"
for lineno, expr in assigns:
assert "time.time()" not in expr, (
f"{name} at line {lineno} uses the wall clock ({expr!r}). A "
"backward NTP step makes the per-frame delta negative and the "
"loop then sleeps longer than the whole frame budget.")
assert "time.monotonic()" in expr, (
f"{name} at line {lineno} is {expr!r}, expected monotonic")
def test_the_fps_window_is_monotonic():
for lineno, expr in _assignments_of("current_time"):
assert "time.monotonic()" in expr, (
f"current_time at line {lineno} is {expr!r}; fps is frames divided "
"by this delta, so a clock step would corrupt the rate itself")
def test_health_state_is_not_reset_every_iteration():
"""run_iteration() runs once per cycle -- locals here reset every few seconds.
As locals, `last_fps_health_log = 0.0` made the 300s heartbeat fire on the
first sample of every iteration, and a recovery spanning two iterations was
never reported because was_degraded had already gone back to False.
"""
run_iteration = next(
(n for n in ast.walk(TREE)
if isinstance(n, ast.FunctionDef) and n.name == "run_iteration"), None)
assert run_iteration is not None, "run_iteration() not found"
local_names = {t.id for n in ast.walk(run_iteration)
if isinstance(n, ast.Assign)
for t in n.targets if isinstance(t, ast.Name)}
for leaked in ("last_fps_health_log", "was_degraded"):
assert leaked not in local_names, (
f"{leaked} is a local of run_iteration() again, so it resets every "
"cycle -- the heartbeat degenerates to once per iteration")
body = ast.unparse(run_iteration)
assert "self._fps_last_health_log" in body and "self._fps_was_degraded" in body, (
"the health state should live on the coordinator, across iterations")
def test_start_clears_stale_health_state():
"""A new run must not inherit "was degraded" from the previous one."""
start = next((n for n in ast.walk(TREE)
if isinstance(n, ast.FunctionDef) and n.name == "start"), None)
assert start is not None, "start() not found"
body = ast.unparse(start)
assert "self._fps_last_health_log" in body and "self._fps_was_degraded" in body, (
"start() does not reset the FPS health state")
def test_the_degraded_threshold_is_documented():
"""The 90% band is deliberate; say so where the constant is defined."""
source = COORD.read_text(encoding="utf-8")
idx = source.index("_FPS_HEALTHY_FRACTION = ")
preamble = source[max(0, idx - 700):idx]
assert "90%" in preamble or "0.9" in preamble, (
"the degradation threshold is not explained at its definition, so "
"'below target' reads as a bug rather than a deliberate band")
+2 -1
View File
@@ -118,7 +118,8 @@ saved_repositories_manager = SavedRepositoriesManager()
schema_manager = SchemaManager(
plugins_dir=plugins_dir,
project_root=project_root,
logger=None
logger=None,
config_manager=config_manager
)
# Initialize operation queue for plugin operations
@@ -95,7 +95,7 @@
<!-- Location Information -->
<div class="grid grid-cols-1 md:grid-cols-3 xl:grid-cols-3 2xl:grid-cols-3 gap-4">
<div class="form-group" id="setting-general-city" data-setting-key="location.city">
<label for="city" class="block text-sm font-medium text-gray-700">City{{ ui.help_tip('City used for weather, sunrise/sunset, and other location-based content.\nExample: Dallas.', 'City') }}</label>
<label for="city" class="block text-sm font-medium text-gray-700">City{{ ui.help_tip('City used for weather, sunrise/sunset, radar, and other location-based content.\nExample: Kansas City.\nUsed as the default for the location_city setting on plugins that have one; a value saved on the plugin itself overrides it.', 'City') }}</label>
<input type="text"
id="city"
name="city"
@@ -104,7 +104,7 @@
</div>
<div class="form-group" id="setting-general-state" data-setting-key="location.state">
<label for="state" class="block text-sm font-medium text-gray-700">State{{ ui.help_tip('State or region for your location.\nExample: Texas. Improves location-lookup accuracy.', 'State') }}</label>
<label for="state" class="block text-sm font-medium text-gray-700">State{{ ui.help_tip('State or region for your location.\nExample: Missouri. Improves location-lookup accuracy.\nUsed as the default for the location_state setting on plugins that have one.', 'State') }}</label>
<input type="text"
id="state"
name="state"
@@ -113,7 +113,7 @@
</div>
<div class="form-group" id="setting-general-country" data-setting-key="location.country">
<label for="country" class="block text-sm font-medium text-gray-700">Country{{ ui.help_tip('Country code or name for your location.\nExample: US. Used with City and State for weather and geolocation.', 'Country') }}</label>
<label for="country" class="block text-sm font-medium text-gray-700">Country{{ ui.help_tip('Country code or name for your location.\nExample: US. Used with City and State for weather, radar, and geolocation.\nUsed as the default for the location_country setting on plugins that have one.', 'Country') }}</label>
<input type="text"
id="country"
name="country"