Compare commits

..
Author SHA1 Message Date
Claude 8a0854472e Tag plugin logs structurally and surface the active plugin in System Logs
- get_logger() now returns a PluginLoggerAdapter when given a plugin_id,
  so every plugin log call is stamped with plugin_id automatically instead
  of only calls that explicitly passed extra={'plugin_id': ...}. This makes
  the "[Plugin: x]" prefix reliable in the journalctl-backed log stream.
- display_controller publishes the currently active mode/plugin to the
  shared cache whenever it changes, exposed via a new
  GET /api/v3/display/current-status endpoint.
- System Logs page: adds a "Now showing" banner backed by that endpoint, a
  plugin filter dropdown (populated from parsed log lines), a plugin badge
  per log entry, and fixes log parsing to handle the short-iso timestamp
  format journalctl actually returns (the old regex only matched syslog
  timestamps, so level/plugin extraction silently never ran).
2026-07-16 18:50:25 +00:00
66f9950a30 chore(ci): add security-audit tooling scripts (workflow files pushed separately) (#414)
* chore(ci): add security-audit workflow and plugin security-proof scripts

- scripts/prove_security.py, audit_plugins.py, generate_report.py --
  automated checks (dangerous eval()/exec() calls, dependency scanning,
  report generation) for plugins.
- .github/workflows/security-audit.yml + bandit.yaml -- CI wiring for
  the above plus gitleaks secret scanning and bandit static analysis.
- .github/workflows/tests.yml -- pytest matrix across Python 3.10-3.12.

Also fixes two Codacy findings while these files are freshly landing:
- prove_security.py: dropped a pointless f-string prefix with no
  placeholders.
- security-audit.yml: pinned gitleaks/gitleaks-action to a full commit
  SHA (matching this repo's existing pinning convention in test.yml)
  instead of the floating v2 tag.

Split out of the original chore/dead-code-removal commit, which had
accidentally bundled this in alongside unrelated dead-code deletions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore: drop workflow files -- pushed separately (needs workflow OAuth scope)

* fix(security-tooling): address PR review findings across bandit.yaml, audit_plugins.py, generate_report.py, prove_security.py

bandit.yaml:
- Removed scripts/prove_security.py's file-level exclusion. Ran bandit
  directly to get ground truth: the real false positive is B105 (dict key
  "PASS" misread as password-like), not the eval/exec pattern the old
  comment claimed. Added a targeted # nosec B105 there, and found+fixed
  the identical pattern already present in generate_report.py.
- Left the repo-wide B607 skip as-is: confirmed via AST scan that properly
  narrowing it touches 100+ bare-name subprocess call sites across
  wifi_manager.py, store_manager.py, permission_utils.py, app.py, and
  start.py -- none of which are part of this PR. Out of proportion to fix
  here; flagged as a dedicated follow-up.

scripts/audit_plugins.py:
- SyntaxError/OSError while scanning a plugin file now report CRITICAL
  (blocking) instead of WARNING/INFO -- a file that couldn't be parsed or
  read was never actually checked for danger, so it must not silently
  pass the audit.
- --plugin <name> now tracks whether the requested plugin was found across
  all PLUGIN_BASE_DIRS and exits 1 with a clear error if not, instead of
  silently scanning zero plugins and reporting success.
- The AST visitor now tracks import aliases (import subprocess as sp;
  from builtins import eval as e) and resolves them before checking
  against dangerous APIs, closing a straightforward evasion of every
  PLUGIN-001 through PLUGIN-005 check. Verified against both aliased and
  unaliased evasion patterns.

scripts/generate_report.py:
- _md_table_row now escapes pipe characters and normalizes newlines in
  every cell, so scanner-controlled content (a matched secret, a bandit
  issue_text) can't corrupt the Markdown table structure.
- _load now distinguishes "artifact missing/malformed" from "valid empty
  result": each summarizer returns an availability flag, and main() now
  reports INCOMPLETE (not PASSED) with exit code 1 when any artifact is
  unavailable, instead of silently folding it in as 0 findings.
- Gitleaks suppression now uses exact-match placeholder values (pulled
  from the actual config_secrets.template.json) plus a template-path
  allowlist, replacing broad substring checks that could hide a real
  secret containing something like "example.com" as part of its value.

scripts/prove_security.py:
- T1b (dangerous plugin calls): a file that fails to parse/read now
  reports CRITICAL with the exception details instead of being silently
  swallowed by `except (SyntaxError, OSError): pass`.
- T6 (Docker hardening): base images must now be pinned to an @sha256
  digest; a specific tag like python:3.12 is mutable and is now correctly
  flagged as unpinned, not just missing tags or :latest.
- T2a (API surface): no config mechanism for enforcing local-only access
  exists in this codebase today (app.py hardcodes host='0.0.0.0'), so the
  "environment-aware" check as described isn't buildable without adding
  new config infrastructure -- out of scope here. Applied the achievable
  part: upgraded from INFO to WARNING, since enforcement can never
  currently be confirmed.
- T1a (zip-slip): replaced the whole-file substring check with an AST
  walk that finds every extract()/extractall() call and confirms an
  is_relative_to() guard + "Zip-slip detected" log precede it in the same
  function. Verified it still passes on the real store_manager.py (both
  the per-member and validate-then-bulk-extract call sites) and correctly
  flags a synthetic unguarded extractall().
- T3a (hardcoded secrets): violation details no longer include the
  matched credential text -- only file, line, pattern type, and a
  redacted SHA-256 fingerprint, so a real finding doesn't get published
  into CI logs/artifacts/PR comments with wider exposure than the
  original leak. Verified with a synthetic secret that no raw content
  reaches the output.

Validated: all four files compile; bandit scans all three scripts clean
(2 legitimate targeted suppressions, 0 unaddressed findings); each new/
changed code path exercised directly (alias evasion, unmatched --plugin,
missing/malformed/valid-empty artifacts, digest-pinning, zip-slip
guard/no-guard, secret redaction); full audit_plugins.py -> prove_security.py
-> generate_report.py pipeline run end-to-end producing a correct report.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(security-tooling): address follow-up review findings on PR #414

scripts/generate_report.py:
- Added an explicit `object` type annotation to _md_sanitize_cell's value
  parameter -- it deliberately accepts any stringifiable value (calls
  str(value) unconditionally), so `object` reflects its actual contract
  more accurately than leaving it untyped.

scripts/prove_security.py:
- Dockerfile FROM-line parsing: renamed the comprehension variable `l` to
  `line` (ambiguous single-letter name). More importantly, fixed a real
  false-positive: `FROM --platform=<platform> <image>` was reading the
  --platform= flag itself as the image token, so a properly digest-pinned
  image behind a platform flag was incorrectly reported as unpinned.
  Verified against platform+digest, platform+tag-only, and digest+AS-alias
  Dockerfiles.

scripts/audit_plugins.py:
- Consolidated visit_Call's dangerous-API detection: previously, alias
  resolution only covered ast.Name calls for eval/exec/compile and
  ast.Attribute calls for subprocess/os.system, missing from-imported
  subprocess/os functions called as bare names (from subprocess import
  run as prun; prun(cmd, shell=True) or from os import system as s;
  s(cmd)). Added _resolve_call_target() to resolve both call shapes to a
  single fully-qualified target, then run all five PLUGIN-00x checks
  against that one resolved value. Verified against 10 evasion
  combinations (from-import aliases, direct/attribute calls, aliased
  module imports) and confirmed zero false positives on benign os/
  subprocess usage without shell=True.

Validated: all three files compile, bandit scans clean (same 2 legitimate
suppressions as before, 0 new findings), audit_plugins.py/prove_security.py
re-run against the real repo with no regressions from the prior fix pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 14:33:17 -04:00
4abcd0e4f9 Fix PermissionError reading config_secrets.json in web interface (#416)
ledmatrix.service (main display) runs as root while ledmatrix-web.service
runs as the non-root install user (install_web_service.sh). Both
config_manager.py and config_manager_atomic.py only chmod'd
config_secrets.json to 0o640 without ever fixing its group, so a file
written by the root service ended up group-owned by root and unreadable
by the web user, crashing the settings page with a raw PermissionError.

Add ensure_shared_group_ownership() to chgrp secrets/config files (best
effort, root-only) to the project directory's owning group whenever they
are created or saved, and self-heal existing files on load. Also make
get_raw_file_content() tolerate an unreadable secrets file the same way
load_config() already does, degrading to empty secrets instead of a 500.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-15 14:25:58 -04:00
2a1c47fa76 chore: remove dead modules and unused dependencies (~1,180 LOC) (#412)
* chore: remove dead modules and unused dependencies (~1,180 LOC)

Deletions, each re-verified with a fresh repo-wide grep (core, web,
scripts, docs, plugin monorepo) immediately before removal:

Modules with zero live importers:
- src/background_cache_mixin.py + src/generic_cache_mixin.py (134+150
  LOC — referenced only by each other)
- src/font_test_manager.py (134 LOC)
- src/image_utils.py (22 LOC, self-documented deprecated)
- src/layout_manager.py (408 LOC — only its own test imported it) +
  test/test_layout_manager.py
- src/common/basketball_plugin_example.py (328 LOC sample)

requirements.txt entries with zero importers in core (pre-plugin-era
manager deps): icalevents, geopy, timezonefinder, unidecode. Plus the
google-auth trio (google-auth-oauthlib, google-auth-httplib2,
google-api-python-client): their only importer is the calendar PLUGIN,
which declares all three in its own requirements.txt (verified in the
monorepo and on an installed copy) — the plugin dependency installer
owns them. Existing venvs are unaffected (removal doesn't uninstall);
fresh installs get them when calendar is installed.

Two stale references cleaned (a comment in test_pillow_compat.py, a
directory listing in HOW_TO_RUN_TESTS.md). Full suite green except the
two documented pre-existing failures (circuit_breaker mock drift, fixed
in #400; clock-simple 64x32 overflow, pre-dates this series); all core
entry modules verified importing cleanly under the emulator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: remove content accidentally bundled into the dead-code-removal commit

The previous commit's git add/commit swept in a lot of unrelated,
unreviewed work alongside the intended dead-code deletions: a new Plugin
Composer web UI, new security-audit/CI tooling, a new march-madness
plugin, and 23 local-development-only symlinks under plugin-repos/ (per
scripts/setup_plugin_repos.py's own docstring, these are meant to be
generated locally, never committed -- .gitignore has no entry for them,
which is how they slipped in).

Removed here, split into their own PRs instead (except plugin-repos/*
symlinks and march-madness/ncaa_logos, which are dropped rather than
carried forward -- see PR discussion):
- web_interface/blueprints/composer.py + composer-app.js +
  composer-canvas.js + composer.html + manager.py.j2
- scripts/prove_security.py, audit_plugins.py, generate_report.py
- .github/workflows/security-audit.yml, .github/workflows/tests.yml,
  bandit.yaml
- All plugin-repos/* symlinks (local dev artifacts, not meant to be
  committed at all)
- plugin-repos/march-madness/* and the 4 new assets/sports/ncaa_logos/*
  PNGs it needed (left out of every split PR pending a decision on
  whether march-madness belongs in the core repo or the plugin monorepo)

This PR now contains only what its title describes: the dead-code
removal from the previous commit, untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:58:31 -04:00
9db1d2391a chore(assets): add 4 NCAA team logos needed by march-madness (COLGATE, LEHIGH, MICHIGAN, RUTGERS) (#415)
march-madness (already live in ledmatrix-plugins) loads team logos from
this shared assets/sports/ncaa_logos/<ABBR>.png cache at runtime
(manager.py:233) -- these 4 were missing.

Split out of PR #412 (chore/dead-code-removal), which had accidentally
bundled these in alongside unrelated dead-code deletions.


Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 09:57:52 -04:00
14a59c863c perf(wifi): one status fetch per monitor tick instead of three (#411)
* perf(wifi): one status fetch per monitor tick instead of three

The wifi monitor daemon fetched WiFi status + ethernet state before its
AP-mode check, the check internally fetched the same state again (with
retry), and the daemon fetched a third time afterwards — each fetch is
several nmcli subprocess forks, every 30s, forever, even on a perfectly
healthy link.

check_and_manage_ap_mode's decision logic is extracted to
_manage_ap_mode(status, ethernet, ap_active); the new
check_and_manage_ap_mode_with_state() runs the single (retrying) fetch
battery and returns (changed, status, ethernet, ap_active_after) — the
post-state is derivable because state only ever flips via one enable or
one disable. The original bool-returning method delegates, so existing
callers are untouched. The daemon's dead pre-fetch is removed and its
post-check reads use the returned state; the AP-enable retry semantics
(_get_wifi_status_with_retry) are preserved exactly. ~10-15 forks/30s
drops to ~4-6.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(wifi): add missing type hints on AP-mode state helpers, fix unused-var lint in tests

- check_and_manage_ap_mode_with_state now declares its
  Tuple[bool, WiFiStatus, bool, bool] return type instead of being untyped.
- _manage_ap_mode's status/ethernet_connected/ap_active parameters are now
  typed (WiFiStatus, bool, bool), matching its existing -> bool return hint.
- test_wifi_check_state.py: rename unused unpacked variables (status/
  ethernet/ap_after) to underscore-prefixed names at the three call sites
  that don't use them, leaving the used ones untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:22:37 -04:00
bff13129c4 perf(config): mtime-signature fast path for load_config (#410)
load_config re-read and re-parsed config.json, the secrets file, AND the
template (running the recursive migration diff) on every call — with
~30 call sites in web request handlers, some hit 2-3x per request.

Fast path: stat all three files (mtime_ns + size); when unchanged since
the last successful load, return the already-parsed self.config (same
aliasing semantics as before). The signature is taken AFTER load +
migration so a migration write-back doesn't retrigger, and both save
paths refresh it. Cross-process freshness is preserved by construction:
a save from the other process bumps the file mtime, so the next load
here re-reads — verified by a dedicated test. Same-second edits are
caught by mtime_ns plus a size check.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 08:49:43 -04:00
6499794c12 fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache (#409)
* fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache

ledmatrix-leaderboard's data_fetcher.py calls these two real-CacheManager
methods (src/cache_manager.py:313,817), but MockCacheManager had neither --
update() always hit an AttributeError, caught by a broad except and logged,
so the harness rendered an empty-but-green leaderboard on every test run
without ever exercising real standings data.

Both mocks delegate to the existing get()/set() -- a mock doesn't need the
real strategy's per-data-type max_age/market-hours timing, plugins under
test just need the methods to exist and round-trip whatever was cached.

* fix(testing): clear get_cached_data_with_strategy_calls in MockCacheManager.reset()

reset() cleared get_calls/set_calls/delete_calls but not the newer
get_cached_data_with_strategy_calls tracker, so a reused mock (e.g. across
test cases sharing a fixture) retained stale strategy-call records after
reset().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-14 08:33:08 -04:00
ChuckandGitHub 3d347a368a fix(testing): stop plugin enabled:false schema defaults from silently disabling harness tests (#408)
check_plugin.py, render_plugin.py, and the pytest plugin matrix each built
config as {"enabled": True} then merged in config_schema.json's defaults on
top, letting a plugin's own enabled:false default (a reasonable choice for
a seasonal/opt-in plugin -- 15 of 23 real plugins ship one) silently win.
Every harness/CI render of those plugins was testing "disabled, do
nothing" rather than real behavior.

Extract build_full_config() into testing/loading.py (already the shared
home for plugin-discovery/config-default logic) and use it from all three
call sites: schema defaults, then a forced enabled=True, then harness.json's
config, then the caller's explicit config -- so a test can still
deliberately disable a plugin on purpose, it just can't happen by accident
via the plugin's own shipped schema default anymore.
2026-07-14 08:21:35 -04:00
0aca40cf3a perf(plugins): run scheduled updates off the render thread (#407)
* perf(plugins): run scheduled updates off the render thread

plugin update() executed inline in the render loop — execute_update's
internal thread.join(timeout=30) blocked it, so one slow plugin HTTP
fetch froze scrolling for the whole fetch (up to 30s; DNS-retry storms
made this a regular occurrence on flaky networks).

Scheduling stays on the render thread and keeps every existing gate
(enabled, circuit breaker, can_execute, interval); due updates are now
enqueued to a single background worker (serialized — same one-at-a-time
execution as before, no thundering herd). RUNNING is set at enqueue so
can_execute blocks re-entry alongside the pending-set dedup.

Per-plugin locks make the old implicit update/display no-overlap
guarantee explicit: the worker holds the plugin's lock through its
update; the display side try-locks and, when the plugin is mid-update,
holds the last frame for that iteration — reported as success so a
mid-update skip never advances the rotation. Unlike before, the
guarantee now also holds across the post-timeout window (previously the
lingering update thread overlapped display()). Deadlock-free by
construction: the worker takes one lock; display never blocks.

Timeout semantics unchanged (lingering daemon thread documented).
Kill switch: plugin_system.synchronous_updates: true restores the
inline path.

8 new concurrency tests (non-blocking scheduler, overlap assertion
under a hammering display loop, lock release on failure/timeout paths,
dedup, kill switch); 4-min devpi soak clean (updates completing,
rotation advancing, no stuck RUNNING states).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(plugins): fix skipped-frame health/force_change tracking, config validation, and lock-lifetime gaps in async updates

Addresses PR #407 review findings:
- display_controller: only clear force_change / record health success
  when display() actually ran this frame, not when the frame was skipped
  because the plugin's lock was busy (a skip must preserve a pending
  mode-switch force_clear).
- plugin_manager: replace bool() coercion of synchronous_updates with
  explicit isinstance validation of plugin_system/synchronous_updates,
  failing safe to synchronous mode (with a logged reason) on malformed
  config instead of silently defaulting to async.
- plugin_manager: _update_worker_loop now acquires the plugin lock before
  looking up its instance and re-checks under the lock, so an unloaded
  plugin's lifecycle state is never resurrected to ENABLED.
- plugin_manager + display_controller: move lock ownership (and, for
  updates, RUNNING/pending lifecycle bookkeeping) into the actual update()/
  display() call itself rather than the timeout-wrapped caller, so the
  lock stays held for the real operation's duration even after
  PluginExecutor's own join(timeout) elapses and a lingering daemon thread
  keeps running in the background.
- DisplayController.cleanup() now stops the update worker before tearing
  down display/cache resources; stop_update_worker() logs when the join
  times out instead of failing silently.
- test_async_plugin_updates: rewrite test_unloaded_while_queued_is_harmless
  to exercise the public unload_plugin() lifecycle (via a deterministic
  blocker) instead of deleting pm.plugins directly, and add a regression
  test proving the plugin lock stays held through PluginExecutor's own
  timeout while the real update() call is still running.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 08:08:51 -04:00
9837315308 perf(display): dirty tracking in update_display + plugin FPS declaration (#406)
* perf(display): dirty tracking in update_display + plugin FPS declaration

update_display now skips SetImage+SwapOnVSync when the frame is
byte-identical to the last pushed one (adler32 digest) AND brightness
is unchanged — brightness is part of the digest, and set_brightness
additionally resets it, so a dim-schedule change can never be skipped.
clear() resets the digest (it writes to the matrix directly). Skipping
a swap is hardware-safe: the panel refreshes the current frame from the
driver's own thread; swaps only change content.

Kill switch: display.dirty_tracking: false restores always-push.

display_controller's high-FPS decision gains a precedence step: a
plugin exposing needs_high_fps is honored first (so static-image can
declare False for still PNGs and stop burning a 125fps loop on them);
static-image without the attribute keeps its historical forced
high-FPS (GIF back-compat); scrolling logic is otherwise unchanged.

Verified with 7 tests against the real DisplayManager on
RGBMatrixEmulator (identical-frame skip, pixel-change push, clear and
brightness invalidation, snapshot-through-skip, kill switch) plus the
202-test display/controller/vegas suites, and a clean devpi deploy.
Audit: every SetImage/SwapOnVSync/Clear/brightness call site is inside
display_manager — no external writer can bypass the digest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(display): serialize update_display, narrow brightness exception, log fixes

CodeRabbit review on #406, verified against current code:

- update_display() can genuinely be called from background threads (some
  sports base classes call it directly from inside update() for an
  immediate "live" refresh), not just the render loop — confirmed via the
  existing follower-mode gating wrapper in display_controller.py, which
  exists specifically because "background plugin threads" can reach it.
  Without a lock, two callers could both pass the digest check before
  either writes _last_pushed_digest back, causing a redundant push, or
  interleave the offscreen/current canvas swap. Added self._update_lock
  (RLock, in case of re-entrant callers) around the full method body so
  every call site is automatically covered — no caller changes needed.
  (No prior lock existed to reuse on DisplayManager; this adds one.)
- Narrowed the brightness-read exception handler to AttributeError,
  matching the established pattern in get_brightness()/set_brightness()
  — a getattr() with a default already swallows AttributeError, so the
  only case this guards is the property getter itself raising, and the
  established pattern treats that as an expected, specific failure mode
  rather than something to blanket-catch.
- FPS-check debug log now includes the plugin_id already in scope
  (previously only active_mode) and a "[DisplayController]" prefix for
  grep-ability, matching the sibling log two lines below it.
- test_display_dirty_tracking.py: dm fixture and test_config_flag_wires_through
  now reset the DisplayManager singleton on teardown, matching the pattern
  test_display_manager.py already uses elsewhere in the same file family.
- test_snapshot_still_written_on_skip previously only exercised the
  non-skip (push) path despite its name; now performs a second update that
  meets the skip conditions (identical frame) and asserts the snapshot is
  still written even though the panel push itself is skipped.

All 7 dirty-tracking tests pass, plus the full display_manager/
display_controller/vegas suite (140 passed). Full repo suite has only the
5 known pre-existing failures (double-sided config x2, state_reconciliation
x2, and test_circuit_breaker's conftest.py mock signature drift — the
latter fixed in #400, which this branch's base predates).

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 10:31:20 -04:00
c1fa5094be fix(store): plugin updates keep the old install until the new one succeeds (#405)
* fix(store): plugin updates keep the old install until the new one succeeds

Both reinstall paths in update_plugin — the monorepo-migration remote
switch AND the routine archive update every store user hits — deleted
the installed plugin directory BEFORE downloading its replacement. A
mid-update failure (bad network, registry error) permanently destroyed
the plugin. Seen in the field: a Pi with broken DNS lost 12 plugins in
one update pass during the monorepo migration.

New _reinstall_with_rollback: rename the old install aside (using the
'.standalone-backup-' name pattern plugin discovery already excludes),
run install_plugin, remove the aside on success — restore it on ANY
failure, clearing partial-download debris first. A stale aside from a
previous crash is cleared before starting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(store): serialize concurrent updates per plugin, check cleanup results

CodeRabbit review on #405 flagged two things in
_reinstall_with_rollback, both verified against current code:

- Real race: the web UI runs Flask with threaded=True and there's a
  single update route, so two overlapping requests for the same
  plugin_id (double-click, two tabs) can interleave. The loser could
  rename the winner's in-progress install aside mid-download, deleting
  its own rollback safety net — worse than the bug this function
  exists to fix. Added a lazy per-plugin_id lock dict (mirrors the
  plugin_manager per-plugin lock pattern) held for the whole function.
- _safe_remove_directory's return value was ignored at both call
  sites. Stale-aside cleanup failure now aborts cleanly instead of
  falling through to a rename that would fail anyway with a less
  useful error; post-success backup-removal failure now logs instead
  of failing silently (still returns True — the update itself
  succeeded, and the next update self-heals the leftover aside).

Left the third nitpick (test_stale_aside_from_previous_crash_is_cleared)
addressed by asserting the stale dir is actually gone and that
install_plugin was reached, rather than just the end-to-end result.

Added a concurrency regression test asserting install_plugin never
runs for the same plugin_id while another call is in flight.

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:32:37 -04:00
4d49b0f892 perf(display): snapshot mirror — viewer gating, digest skip, keepalive (#404)
The display service PNG-encoded its frame to /tmp/led_matrix_preview.png
at 5 fps, 24/7 — identical frames, no viewers, per-call imports and a
chmod every write. On the devpi baseline the display service idles at
~92% CPU; this was one of its biggest fixed costs.

- New pure policy (src/common/snapshot_policy.py, unit-tested off-Pi):
  WRITE changed frames at full rate only while a viewer is watching,
  at a 30s idle cadence otherwise; NEVER re-encode unchanged frames —
  bump mtime (os.utime) every 20s instead, keeping the health check's
  snapshot-age liveness proxy (60s threshold in api_v3) green. Cross-
  referencing comments guard the two constants.
- Viewer detection: the web SSE display broadcaster (which only runs
  while browsers are subscribed) touches /tmp/led_matrix_preview_viewer
  each loop; the display service stats it at most 1/s. On viewer
  arrival the write clock resets so the first frame lands within ~1s.
- Hoisted the per-call pathlib/permission_utils imports; directory
  permissions ensured once instead of every frame.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:28:30 -04:00
efe76d3add perf: hot-path micro fixes in the render loop (#403)
* perf: hot-path micro fixes in the render loop

- _check_wifi_status_message stat'd the status file on every render
  iteration (60+ fps) for a message whose lifetime is seconds; throttle
  the check to 1 Hz with a cached result.
- Demote the per-iteration "Display active, processing mode" INFO to
  DEBUG and convert the remaining eager f-string logs to lazy % args —
  the devpi baseline showed ~9 journald lines/sec, which is both noise
  and SD-card wear.
- Vegas cycle-end blank frame: hoist the inline PIL import and reuse a
  preallocated buffer instead of allocating per cycle wrap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: initialise wifi-status throttle state in __init__

Codacy (pylint access-member-before-definition) on #403: the throttled
early-return read _wifi_status_last_result relying on the non-local
invariant that the first call always passes the throttle window and
assigns it. Correct at runtime, but fragile — initialise both throttle
fields in the constructor and drop the getattr fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:23:25 -04:00
273d9962d1 fix(cache): stop fsync-hammering the SD card on unchanged data (#402)
DiskCache.set wrote every key as mkstemp -> json.dump(indent=4) ->
flush+fsync -> replace -> chmod, on the persistent cache dir — dozens
of force-flushed SD writes per minute on an API-heavy install, mostly
rewriting identical data every plugin update cycle.

- Serialize once, compact (no indent): cache files are machine-read
  only; indenting multiplied the bytes written.
- Skip the disk when the payload for a key is unchanged (adler32 map,
  per-process); refresh the file mtime instead so records relying on
  mtime for TTL don't expire early. Self-heals if the file was removed
  externally (expiry cleanup).
- Drop the per-write fsync: os.replace already guarantees readers never
  see a torn file, and cache data is re-fetchable — the flush bought
  nothing but card wear.

API unchanged; DateTimeEncoder round-trip covered by tests.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:00:35 -04:00
9e3b5f366e fix(core): harden text-measurement caches; surface snapshot failures (#400)
* fix(core): harden text-measurement caches; surface snapshot failures

Deep-dive findings, all three latent on every 24/7 install:

- font_manager.metrics_cache and display_manager._text_width_cache were
  unbounded dicts keyed by (text, id(font)). Two problems: keys embed
  the measured TEXT, so ever-changing strings (a clock, a live score, a
  ticker) grow them without limit; and id()-keying without holding a
  reference means a garbage-collected font's id can be recycled by a
  DIFFERENT font, silently returning wrong widths/metrics (classic
  plugins create fonts per render, so this is reachable). Both caches
  are now LRU-bounded (1024) and pin the font in the entry so its id
  stays valid. metrics_cache also keyed on the text itself instead of
  hash(text), removing a collision path.

- _write_snapshot_if_due logged failures at DEBUG — invisible at the
  default level. The snapshot's mtime is the web UI's display mirror
  AND its hardware-liveness proxy, so a quiet failure freezes the
  mirror and makes health checks lie (seen in the field: a stale
  root-owned /tmp file froze it for a day). Failures now WARN, rate-
  limited to once per 5 minutes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* test: sync mock cache manager signature with CacheManager.get

test_circuit_breaker has been failing on main: plugin_health passes
memory_ttl= to cache_manager.get(), and the conftest mock's signature
was never updated — the same component/double drift class as the
monitored_update bug (#392).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 08:59:41 -04:00
6edd80d9f3 fix(schedule): stop stray 'days' data from overriding Global schedule (#399)
save_schedule_config never persisted the schedule's 'mode' field, and
_check_schedule inferred per-day vs global purely from whether a 'days'
dict was present for the current day. Config migration
(_merge_template_defaults) re-adds the template's 'schedule.days' (all
days disabled by default) whenever it's missing from the user's saved
config - which is exactly the case after saving Global mode, since that
save path intentionally pops 'days'. The result: a user on Global mode
would get their schedule silently reinterpreted as per-day, with today's
day disabled, blanking the display.

Persist 'mode' on save and have _check_schedule honor it explicitly
(mirroring how _check_dim_schedule already does), so a resurrected
'days' dict can't override an explicit Global selection. Falls back to
the old inference behavior only when no 'mode' is recorded.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 10:53:33 -04:00
1c7a0cef66 fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation (#398)
* fix(vegas): restore live plugin-update refresh dropped by sync refactor

Investigating a user report that Vegas scroll mode doesn't update scores
or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live
score change reached the ticker within a few seconds instead of waiting
for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed
plugin_last_update timestamps to detect which plugins got fresh data and
called coordinator.mark_plugin_updated() for each, and should_recompose()
checked has_pending_updates_for_visible_segments() to trigger an immediate
hot-swap.

PR #330 (May 14, multi-display wireless sync) refactored both call sites
while adding sync support and silently deleted this entire mechanism --
not just gated it behind the new sync-mode deferral it legitimately
needed, but removed it outright. The result: VegasModeCoordinator.
mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_
segments() have been fully implemented but never called from anywhere
since. Vegas mode's only remaining freshness sources are a 5s content
cache TTL (fine) and full recompose at cycle boundaries, which depending
on min/max_cycle_duration can be minutes away -- so live scores/status
can sit stale far longer than a user would expect from a "live" ticker.

Fix:
- Restored _tick_plugin_updates_for_vegas() in display_controller.py,
  wired as the Vegas coordinator's update callback in place of the plain
  _tick_plugin_updates(). Diffs plugin_last_update before/after the tick
  and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each
  plugin that actually got new data (rather than returning the list, since
  the callback interface no longer consumes a return value).
- Restored the has_pending_updates_for_visible_segments() check in
  render_pipeline.should_recompose(), positioned after (not instead of)
  the sync-mode early return PR #330 added, so standalone installations
  regain immediate refresh while synced leader/follower pairs correctly
  keep deferring hot-swaps to cycle boundaries as PR #330 intended.

Test plan:
- Added test_display_controller_vegas_tick.py and
  test_vegas_render_pipeline_recompose.py -- neither area had any prior
  test coverage, which is very likely why this regression went unnoticed
  for ~2.5 months.
- Verified both new test files fail against the pre-fix code (swapped in
  the current main versions of both files) with exactly the expected
  errors -- AttributeError for the deleted method, and the recompose
  assertion returning False instead of True -- then pass against the fix.
- Confirmed the sync-mode deferral this restoration must not break still
  holds: test_sync_active_defers_pending_updates_to_cycle_boundary.
- Full related suite (test_vegas_plugin_adapter, test_vegas_config,
  test_display_controller_plugin_toggle, test_display_controller_
  optimizations, test_plugin_system): 108 passed, 1 pre-existing failure
  unrelated to this change (test_circuit_breaker, stale mock signature).
- Full CI plugin-safety suite (test_harness, test_visual_rendering,
  test_plugin_matrix): 52 passed, 2 pre-existing skips.

* fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation

_tick_plugin_updates_for_vegas() snapshotted and later re-iterated
plugin_manager.plugin_last_update from the Vegas background update-tick
thread while the main render loop (or other callers) could mutate the
same dict concurrently — a real race (unprotected dict iteration/mutation
across threads), not just a style nit.

Move the snapshot/update/diff into a new locked
PluginManager.run_scheduled_updates_with_changes() so all reads and
mutations of plugin_last_update happen under one lock, and update
DisplayController to use it. The lock is only held around the dict
accesses, not the update pass itself, so slow plugin update() calls don't
serialize against other callers.

Also add a regression test covering that the Vegas coordinator is wired
to the Vegas-aware tick callback rather than the plain one.

Skipped as not worth the change:
- Narrowing the broad `except Exception` around
  vc.mark_plugin_updated(plugin_id) to specific types: it's a deliberate
  per-plugin isolation boundary (matches the same pattern used elsewhere
  in this file for plugin/coordinator calls) and there's no documented,
  stable set of exceptions that call can raise to narrow to.
- Adding an inactive-DisplaySyncManager test to
  test_vegas_render_pipeline_recompose.py: verified
  VegasModeCoordinator.set_sync_manager() already normalizes a
  SyncRole.STANDALONE manager to None before handing it to the render
  pipeline (src/vegas_mode/coordinator.py:152-156), so should_recompose()'s
  `is not None` check is correct in practice; the suggested case is
  already covered by that normalization.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 10:52:18 -04:00
ChuckandGitHub 6052a60d22 fix(vegas): restore live plugin-update refresh dropped by sync refactor (#395)
Investigating a user report that Vegas scroll mode doesn't update scores
or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live
score change reached the ticker within a few seconds instead of waiting
for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed
plugin_last_update timestamps to detect which plugins got fresh data and
called coordinator.mark_plugin_updated() for each, and should_recompose()
checked has_pending_updates_for_visible_segments() to trigger an immediate
hot-swap.

PR #330 (May 14, multi-display wireless sync) refactored both call sites
while adding sync support and silently deleted this entire mechanism --
not just gated it behind the new sync-mode deferral it legitimately
needed, but removed it outright. The result: VegasModeCoordinator.
mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_
segments() have been fully implemented but never called from anywhere
since. Vegas mode's only remaining freshness sources are a 5s content
cache TTL (fine) and full recompose at cycle boundaries, which depending
on min/max_cycle_duration can be minutes away -- so live scores/status
can sit stale far longer than a user would expect from a "live" ticker.

Fix:
- Restored _tick_plugin_updates_for_vegas() in display_controller.py,
  wired as the Vegas coordinator's update callback in place of the plain
  _tick_plugin_updates(). Diffs plugin_last_update before/after the tick
  and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each
  plugin that actually got new data (rather than returning the list, since
  the callback interface no longer consumes a return value).
- Restored the has_pending_updates_for_visible_segments() check in
  render_pipeline.should_recompose(), positioned after (not instead of)
  the sync-mode early return PR #330 added, so standalone installations
  regain immediate refresh while synced leader/follower pairs correctly
  keep deferring hot-swaps to cycle boundaries as PR #330 intended.

Test plan:
- Added test_display_controller_vegas_tick.py and
  test_vegas_render_pipeline_recompose.py -- neither area had any prior
  test coverage, which is very likely why this regression went unnoticed
  for ~2.5 months.
- Verified both new test files fail against the pre-fix code (swapped in
  the current main versions of both files) with exactly the expected
  errors -- AttributeError for the deleted method, and the recompose
  assertion returning False instead of True -- then pass against the fix.
- Confirmed the sync-mode deferral this restoration must not break still
  holds: test_sync_active_defers_pending_updates_to_cycle_boundary.
- Full related suite (test_vegas_plugin_adapter, test_vegas_config,
  test_display_controller_plugin_toggle, test_display_controller_
  optimizations, test_plugin_system): 108 passed, 1 pre-existing failure
  unrelated to this change (test_circuit_breaker, stale mock signature).
- Full CI plugin-safety suite (test_harness, test_visual_rendering,
  test_plugin_matrix): 52 passed, 2 pre-existing skips.
2026-07-12 10:40:34 -04:00
7f7f0d6464 feat: adaptive layout system — size-aware regions, crisp font ladders, image fitting (#393)
* feat(layout): adaptive layout & font scaling system for plugins

Add src/adaptive_layout.py — opt-in core helpers so plugins render
legibly on any panel size without hand-tuned per-display layouts:

- Region: integer rect algebra (bands/columns/weighted splits/centering)
  that partitions space so text bands can't overlap by construction
- Font ladders: ordered (family, size) steps known to render crisply
  (LADDER_GRID: X11 BDFs at native sizes; LADDER_ARCADE: PressStart2P at
  8px multiples) — fitting walks the ladder instead of scaling pixel
  fonts fractionally
- LayoutContext: breakpoint tiers, geometry scale vs. a declared design
  size, and cached fit_text/fit_lines/font_for_rows queries

Generalizes the three patterns proven in the field: f1-scoreboard's
scale factor, masters-tournament's tiers, baseball-scoreboard's font
fallback ladder.

Wiring: BasePlugin gains a lazy .layout property and draw_fit();
FontManager gains get_native_bdf_size() and a cache_generation counter;
manifest schema gains display.design_size and requires.display_size
max_width/max_height; 96x48 joins DEFAULT_TEST_SIZES; the bounds-check
harness records negative-coordinate draws; TextHelper's broken
measurement helpers are fixed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): adaptive image fitting + composite region helpers

Add src/adaptive_images.py — the image counterpart to fit_text:
- fit_image(img, box, mode=contain|cover|fill_height|stretch,
  crop_to_ink, anchor, resample, upscale) promoting the proven plugin
  patterns (football's crop-to-ink fill-height logos, masters' cover
  crop + NEAREST flags, static-image's letterbox). Upscales by default —
  thumbnail()'s downscale-only behavior is why imagery stays tiny on
  big panels.
- draw_fitted_image() pastes aligned within a Region with alpha mask.
- One central Pillow>=9.1 RESAMPLE shim replacing ~15 plugin copies.

LayoutContext.fit_image() caches results per (identity, box size,
options) with a 64-entry LRU; id()-keyed entries pin the source image.
BasePlugin.draw_image() is the one-liner adoption path beside draw_fit.

Composites in adaptive_layout.py: Region.offset() (user x/y-offset
passthrough), scoreboard_regions() (the two-logos-plus-score card math
duplicated across six sports plugins, logo_slot = min(H, W//2)), and
media_row() (art-left/text-right).

Fix LogoHelper's size-blind cache key (stale sizes on panel change);
deprecation note on dead image_utils.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(harness): scale-up fill check, config variants, multi-size dev gallery

Quality gates for adaptive layout:

- fill_metrics()/check_scale_up() in the safety harness: overflow catches
  content too big for a panel, but nothing caught content that stays tiny
  on panels >= 2x the plugin's declared design size. The check measures
  lit-content extents and warns (or fails, when a plugin opts into
  "fill_check": "strict" in test/harness.json) below 50% coverage on the
  doubled axis. Warn-only by default so no existing plugin breaks.

- harness.json "variants": extra runs with config overlays and their own
  golden dirs, so an opt-in mode (e.g. layout_mode: adaptive) is golden-
  tested beside the classic default. check_plugin.py loops base + variants
  and labels variant results mode@name.

- Dev preview server: GET /api/sizes (harness size sample), POST
  /api/render-matrix (render at up to 12 sizes in one call), size-preset
  dropdown, and an "All Sizes" side-by-side gallery in the preview UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(plugins): adaptive-lib discoverability + advisory version compat warning

Discoverability: re-export the adaptive layout/image API from src.common
(the blessed-helpers package plugin authors already know) — canonical
paths stay src.adaptive_layout / src.adaptive_images so nothing breaks.
Document it in src/common/README.md and cross-link ADAPTIVE_LAYOUT.md
from the developer docs authors actually read (quick reference, API
reference, advanced dev, font manager, dev preview, plugin dev guide);
ADAPTIVE_LAYOUT.md gains adaptive-images, composite-layouts and
preserving-user-customization sections.

Compat: PluginLoader now logs one advisory warning (never raises) when a
plugin's manifest declares a min LEDMatrix version newer than the running
core, checking the min_ledmatrix_version / requires.* / versions[]
spellings found in the wild. Guarded against stale core version numbers.

src/__init__.py __version__ bumped 1.0.0 -> 3.1.0 to match the latest
release tag (v3.1.0) — it had never been updated and the compat check
needs a truthful number. NOTE: verify this matches the intended release
numbering before the next tag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add measure_font_crispness — verify a ladder rung isn't blurry

PIL antialiases TTF outlines by default; a 'pixel-style' font only
rasterizes without antialiasing at specific sizes (for PressStart2P:
exact multiples of its 8px design grid). A ladder rung at an unverified
size silently renders blurry on an LED panel — this exact bug shipped in
both text-display's and football-scoreboard's custom TTF ladders
(non-8-multiple PressStart2P sizes, and '5by7.regular'/'4x6-font' at
sizes that were never actually crisp).

measure_font_crispness(font, sample_text) renders the sample and reports
the fraction of ink-bbox pixels that are neither pure black nor pure
white. BDF fonts (real bitmaps) always score 0.0; TTF ladders should be
verified against this before shipping — see the new
TestFontFitting::test_ladder_arcade_is_crisp pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add fit_text_proportional — proportional sizing vs. always-maximize

fit_text always picks the largest ladder rung that fits its box. That's
right when an element owns dedicated space, but wrong when several
independently-fitted elements need to stay visually harmonious as the
panel grows: a score's box might have generous room while a neighboring
logo scales by a fixed geometry factor via px() — fit_text lets the score
balloon out of proportion (even overlapping the logo) even though its
individual pick is technically correct.

fit_text_proportional(text, box, base_size_px, ladder) instead targets
base_size_px * self.scale (the same scale factor px() already uses),
picking the nearest ladder rung at or below that target, still capped to
what fits the box, floored at the smallest rung when the target is below
every rung. Refactored the shared largest-that-fits/ellipsize walk into
_walk_ladder() so fit_text and fit_text_proportional don't duplicate it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): fit_text_proportional gains an axis-specific scale override

self.scale (min(width_ratio, height_ratio)) is the right conservative
default for anything whose aspect ratio matters, but a caller whose
surrounding composition already scales along a single axis — e.g.
football-scoreboard's logo_slot = min(height, width // 2), which tracks
height alone — needs text sized the same way, or it reads as
under-scaled next to logos that grew on a panel that only got taller
(128x32 -> 128x64: self.scale stays 1.0 since width didn't grow, but
logos still double).

fit_text_proportional(..., scale=None) now accepts an explicit override;
None keeps the existing self.scale default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(layout): scoreboard_regions reserves real center space at 2:1 aspect ratios

logo_slot = min(height, width // 2) has a blind spot: at exactly 2:1
aspect ratio (width == 2 * height -- a very common shape: two, four, or
more square modules stacked into a taller panel) width // 2 and height
are equal, so the two logo slots claim the ENTIRE width and leave zero
pixels for a center column, no matter how large the panel gets. Not a
'small panel' problem -- 96x48, 128x64, and 256x128 (all exactly 2:1) hit
it identically, while the 128x32 design baseline and panels like 192x48
or 256x32 never do, because height is already the tighter constraint
there.

Two new parameters fix it in the one shared helper every scoreboard-style
plugin composes through:

- min_center_fraction / min_center_design_px reserve at least
  max(width * fraction, design_px * ctx.scale) for the center column,
  capping logo_slot further when needed. The scaled design-px term
  matters on small panels where a flat fraction alone reserves too little
  absolute space.
- score_bleed_fraction extends the score's own fit box (not the logo
  slots themselves) a controlled amount into each side -- the same way
  real broadcast scoreboards let a big score number's edges cross into
  the team marks flanking it. Without this the reserve alone can still be
  too narrow for a short score to render without truncating.

score_area is now genuinely narrower than the full card width (previously
identical to status_band/detail_band, which still span the full width and
overlay the logos -- short text there was never the problem).

Verified against the full harness size spread: a real game score like
'17-21' never needs ellipsis at any tested 2:1-or-tighter aspect ratio
(test_score_never_needs_ellipsis_for_a_short_score), and wide panels
(128x32/192x48/256x32-style) are provably unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: document scoreboard_regions' center-reserve and score-bleed params

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: address CodeRabbit review on PR #393

- docs: scope the self.layout note to BasePlugin subclasses (others build
  a LayoutContext directly) and make explicit that adaptive layout is
  opt-in — classic rendering stays unless a plugin adopts the APIs.
- dev_server: broaden the render-request catch (a bad manifest.json now
  returns a clean 400 instead of an unhandled 500) and stop echoing raw
  exception text in the loader-failure responses — full tracebacks go to
  the dev server's console log instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): allowlist plugin_id before any path lookup

CodeQL (py/path-injection): plugin_id arrives in request input and flows
into filesystem paths via find_plugin_dir. Gate it with the same
^[a-zA-Z0-9_-]{1,64}$ allowlist the web UI's pages_v3 uses, at the
single choke point every route resolves through.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): lexical containment check on resolved plugin dirs

CodeQL doesn't recognize the interprocedural allowlist as a
path-injection barrier; add the canonical one — normalize (without
following symlinks, since dev plugins are commonly symlinked into
plugins/) and require the result to stay inside the search dir.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): inline normpath containment barrier before render

CodeQL doesn't credit the sanitization inside find_plugin_dir along
this flow; apply its documented barrier (normpath + startswith against
the allowed roots) inline in _parse_render_request, on the exact path
that reaches the render/load sinks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): derive plugin dir from trusted directory listings

CodeQL's barrier-guard recognition doesn't see a startswith check
inside an any() comprehension, so the normalize-and-prefix approach
still flagged. Break the taint outright instead: after lookup, re-derive
the directory by enumerating the search dirs (iterdir) and matching by
path equality — the Path used for all downstream file access is built
solely from trusted listings, never from request input.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): use os.scandir for path-injection barrier, redact stack traces from render responses

CodeQL doesn't model Path.iterdir() as a taint-clearing enumeration the
way it does os.scandir() -- _trusted_plugin_dir's iterdir-based rebuild
still traced plugin_id through to the manifest.json open(). Switched to
scandir, matching the pattern already verified clean on PR #396.

Also stops surfacing raw exception text (update()/display() failures)
in the JSON render response -- logs full detail server-side via
exc_info instead, returning only the exception class name to the
client. And drops path values from three plugin_loader debug/error
logs that CodeQL flags as clear-text-logging of externally-influenced
data, keeping plugin_id (not flagged) for context.

* fix(dev-server): remove conditional-reassignment ambiguity in plugin_dir resolution

CodeQL's path-injection flow still traced through _parse_render_request
after the scandir fix -- the tainted find_plugin_dir() result and the
scandir-derived _trusted_plugin_dir() result shared the same variable
name (plugin_dir), reassigned only on the truthy branch. That merge
point apparently isn't treated as a barrier by the flow analysis, so it
kept tracing the pre-reassignment value through to the manifest open().

Split into two distinct names -- candidate_dir (tainted, used only to
call _trusted_plugin_dir) and trusted_dir (the only name used for any
downstream file access) -- so there's no reassigned variable for the
flow to walk through.

* fix: remove unused imports flagged by Codacy

Union in adaptive_images.py and field in adaptive_layout.py are both
imported but never used -- the last two Codacy findings on this PR,
matching the same fix already applied on PR #396.

* fix(layout): bound the fit cache; never alias the source image in fits

Two latent issues found in a self-review pass:

- LayoutContext._fit_cache was an unbounded dict (the image cache got an
  LRU cap, the text-fit cache didn't). Cache keys embed the fitted TEXT,
  so a plugin fitting changing strings — a live game clock, a ticker —
  on a 24/7 service grows it forever. Now LRU-bounded at 512 entries via
  the same pattern as the image cache.

- fit_image returned the caller's ORIGINAL image object when the source
  was already RGBA at target size (contain/fill_height, no ink crop).
  ImageFitResult is documented as an independent copy, and LayoutContext
  caches results — an aliased image lets later mutations of the source
  corrupt cached fits (or vice versa). Copy in that branch.

Both covered by new regression tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 10:38:52 -04:00
63 changed files with 4399 additions and 4297 deletions
Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

+29
View File
@@ -0,0 +1,29 @@
# bandit.yaml — LEDMatrix bandit configuration
# https://bandit.readthedocs.io/en/latest/config.html
#
# Skips are justified by the specific codebase context documented below.
# Do not remove skips without updating the justification comment.
skips:
# B104: Binding to all interfaces (0.0.0.0)
# Intentional — the Flask server binds 0.0.0.0 for LAN access on a Raspberry Pi.
# This is not internet-facing and is documented in web_interface/app.py.
- B104
# B603: subprocess call without shell=True
# All subprocess.run() calls in this codebase use list arguments (confirmed by
# grep — zero uses of shell=True in src/ or web_interface/). List args prevent
# shell injection. See src/common/permission_utils.py for the primary usage.
- B603
# B607: Starting a process with a partial executable path
# The subprocess calls invoke system utilities (systemctl, sudo, git) by name.
# These are fixed-list invocations, not user-controlled, and rely on PATH.
- B607
exclude_dirs:
- tests
- test
- venv
- .venv
- rpi-rgb-led-matrix-master
-1
View File
@@ -248,7 +248,6 @@ test/
├── test_config_service.py # Config service tests
├── test_config_validation_edge_cases.py # Config edge cases
├── test_font_manager.py # Font manager tests
├── test_layout_manager.py # Layout manager tests
├── test_text_helper.py # Text helper tests
├── test_error_handling.py # Error handling tests
├── test_error_aggregator.py # Error aggregation tests
-7
View File
@@ -8,16 +8,11 @@ numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
# Timezone handling
pytz>=2024.2,<2025.0 # Updated for latest timezone data
timezonefinder>=6.5.0,<7.0.0 # Updated for better performance and accuracy
geopy>=2.4.1,<3.0.0
# HTTP requests
requests>=2.33.0,<3.0.0
# Google API integration
google-auth-oauthlib>=1.2.0,<2.0.0
google-auth-httplib2>=0.2.0,<1.0.0
google-api-python-client>=2.147.0,<3.0.0
# Font rendering
freetype-py>=2.5.1,<3.0.0
@@ -29,10 +24,8 @@ spotipy>=2.25.2,<3.0.0
Flask>=3.1.3,<4.0.0
# Text processing
unidecode>=1.3.8,<2.0.0
# Calendar integration
icalevents>=0.1.27,<1.0.0
# WebSocket support
python-socketio>=5.14.0,<6.0.0
+344
View File
@@ -0,0 +1,344 @@
#!/usr/bin/env python3
"""
LEDMatrix Plugin Security Auditor
Performs AST-based security analysis of all Python files in plugin directories.
Designed to run in CI — exits non-zero on CRITICAL findings only.
Usage:
python scripts/audit_plugins.py
python scripts/audit_plugins.py --verbose
python scripts/audit_plugins.py --plugin hello-world
python scripts/audit_plugins.py --output results.json
"""
import ast
import argparse
import json
import sys
from dataclasses import dataclass, asdict
from pathlib import Path
from datetime import datetime, timezone
PROJECT_ROOT = Path(__file__).resolve().parent.parent
PLUGIN_BASE_DIRS = [
PROJECT_ROOT / "plugins",
PROJECT_ROOT / "plugin-repos",
]
# ─────────────────────────────────────────────────────────────────────────────
# Finding dataclass
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class Finding:
plugin_id: str
file: str
line: int
severity: str # CRITICAL | WARNING | INFO
rule: str
message: str
def to_dict(self) -> dict:
return asdict(self)
# ─────────────────────────────────────────────────────────────────────────────
# AST visitor
# ─────────────────────────────────────────────────────────────────────────────
class _PluginVisitor(ast.NodeVisitor):
"""Collect security findings from a single plugin Python file."""
def __init__(self, filepath: Path, plugin_id: str):
self.filepath = filepath
self.plugin_id = plugin_id
self.findings: list[Finding] = []
# Local name -> real dotted path, so aliased imports and from-imports
# of dangerous APIs (import subprocess as sp; from builtins import
# eval as e) are still recognized in visit_Call below.
self._aliases: dict[str, str] = {}
def _add(self, node: ast.AST, severity: str, rule: str, message: str) -> None:
self.findings.append(Finding(
plugin_id=self.plugin_id,
file=str(self.filepath.relative_to(PROJECT_ROOT)),
line=getattr(node, "lineno", 0),
severity=severity,
rule=rule,
message=message,
))
def _resolve(self, local_name: str) -> str:
"""Resolve a local name through recorded import aliases to its real
dotted path (e.g. "sp" -> "subprocess"); unresolved names pass through
unchanged."""
return self._aliases.get(local_name, local_name)
def _resolve_call_target(self, func: ast.expr) -> str | None:
"""Resolve a Call's func node to a fully-qualified dotted target,
covering a direct name (bare builtin, aliased import, or
from-import: from builtins import eval as e; from subprocess
import run; from os import system as s) and module-attribute
access (subprocess.run, sp.run, os.system, o.system) uniformly.
Returns None for call shapes this doesn't attempt to resolve."""
if isinstance(func, ast.Name):
return self._resolve(func.id)
if isinstance(func, ast.Attribute) and isinstance(func.value, ast.Name):
base = self._resolve(func.value.id)
return f"{base}.{func.attr}"
return None
def visit_Call(self, node: ast.Call) -> None:
target = self._resolve_call_target(node.func)
if target is None:
self.generic_visit(node)
return
leaf = target.rsplit(".", 1)[-1]
# eval() / exec() / compile() — arbitrary code execution, whether a
# bare call, an aliased import, or a from-import
# (from builtins import eval as e; e(...))
if leaf == "eval":
self._add(node, "CRITICAL", "PLUGIN-001",
"eval() call — arbitrary code execution risk")
elif leaf == "exec":
self._add(node, "CRITICAL", "PLUGIN-002",
"exec() call — arbitrary code execution risk")
elif leaf == "compile":
self._add(node, "WARNING", "PLUGIN-003",
"compile() call — dynamic code compilation")
# subprocess.*(shell=True), whether subprocess.run(...), sp.run(...),
# or a from-import (from subprocess import run; run(..., shell=True))
if target in {
"subprocess.run", "subprocess.call", "subprocess.Popen",
"subprocess.check_call", "subprocess.check_output",
}:
for kw in node.keywords:
if (kw.arg == "shell" and
isinstance(kw.value, ast.Constant) and
kw.value.value is True):
self._add(node, "WARNING", "PLUGIN-004",
f"subprocess.{leaf}(shell=True) — "
f"shell injection risk if args include user input")
# os.system(), whether os.system(...), o.system(...), or a
# from-import (from os import system as s; s(...))
if target == "os.system":
self._add(node, "WARNING", "PLUGIN-005",
"os.system() call — prefer subprocess with list args")
self.generic_visit(node)
def visit_Import(self, node: ast.Import) -> None:
for alias in node.names:
if alias.asname:
local, real = alias.asname, alias.name
else:
# `import os.path` binds the top-level name `os`, not `os.path`
local = real = alias.name.split(".")[0]
self._aliases[local] = real
self._check_import(node, alias.name)
self.generic_visit(node)
def visit_ImportFrom(self, node: ast.ImportFrom) -> None:
if node.module:
for alias in node.names:
local = alias.asname or alias.name
self._aliases[local] = f"{node.module}.{alias.name}"
self._check_import(node, node.module)
self.generic_visit(node)
def _check_import(self, node: ast.AST, module_name: str) -> None:
dangerous = {
"ctypes": ("WARNING", "PLUGIN-010", "ctypes import — native code execution"),
"cffi": ("WARNING", "PLUGIN-011", "cffi import — native code execution"),
"pickle": ("WARNING", "PLUGIN-012",
"pickle import — deserialization can execute arbitrary code"),
"marshal": ("WARNING", "PLUGIN-013",
"marshal import — deserialization risk"),
}
for mod, (severity, rule, msg) in dangerous.items():
if module_name == mod or module_name.startswith(mod + "."):
self._add(node, severity, rule, msg)
# ─────────────────────────────────────────────────────────────────────────────
# Per-plugin audit
# ─────────────────────────────────────────────────────────────────────────────
def audit_plugin(plugin_dir: Path) -> list[Finding]:
"""Audit a single plugin directory. Returns all findings."""
findings: list[Finding] = []
plugin_id = plugin_dir.name
# Check for required files
for required_file, rule, msg in [
("manifest.json", "PLUGIN-020",
"manifest.json missing — plugin may be incomplete"),
("config_schema.json", "PLUGIN-021",
"config_schema.json missing — no input validation schema declared"),
]:
if not (plugin_dir / required_file).exists():
findings.append(Finding(
plugin_id=plugin_id,
file=str((plugin_dir / required_file).relative_to(PROJECT_ROOT)),
line=0,
severity="WARNING",
rule=rule,
message=msg,
))
# AST analysis of all Python files
for py_file in sorted(plugin_dir.rglob("*.py")):
try:
source = py_file.read_text(encoding="utf-8")
tree = ast.parse(source, filename=str(py_file))
visitor = _PluginVisitor(py_file, plugin_id)
visitor.visit(tree)
findings.extend(visitor.findings)
except SyntaxError as exc:
# A file the visitor can't even parse is a file we can't verify
# is safe -- this must block the audit, not just warn.
findings.append(Finding(
plugin_id=plugin_id,
file=str(py_file.relative_to(PROJECT_ROOT)),
line=getattr(exc, "lineno", 0) or 0,
severity="CRITICAL",
rule="PLUGIN-030",
message=f"Python syntax error — cannot be parsed: {exc}",
))
except OSError as exc:
# Same reasoning as SyntaxError: an unreadable file was never
# actually scanned, so it must block rather than pass silently.
findings.append(Finding(
plugin_id=plugin_id,
file=str(py_file.relative_to(PROJECT_ROOT)),
line=0,
severity="CRITICAL",
rule="PLUGIN-031",
message=f"Could not read file: {exc}",
))
return findings
# ─────────────────────────────────────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="LEDMatrix plugin security auditor",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--plugin", "-p", default=None,
help="Audit a specific plugin ID only")
parser.add_argument("--output", "-o", default=None,
help="Write JSON results to this file")
parser.add_argument("--verbose", "-v", action="store_true",
help="Show all findings, not just summary")
args = parser.parse_args()
print("=" * 60)
print("LEDMatrix Plugin Security Audit")
print(f"Project root: {PROJECT_ROOT}")
print("=" * 60)
all_findings: list[Finding] = []
plugins_scanned = 0
plugin_found = args.plugin is None
for base_dir in PLUGIN_BASE_DIRS:
if not base_dir.exists():
if args.verbose:
print(f" ⏭️ Skipping {base_dir.name}/ (directory not found)")
continue
base_label = base_dir.relative_to(PROJECT_ROOT)
print(f"\n Scanning {base_label}/")
for plugin_dir in sorted(base_dir.iterdir()):
if not plugin_dir.is_dir():
continue
if plugin_dir.name.startswith((".", "_")):
continue
if args.plugin and plugin_dir.name != args.plugin:
continue
if args.plugin:
plugin_found = True
findings = audit_plugin(plugin_dir)
all_findings.extend(findings)
plugins_scanned += 1
critical = [f for f in findings if f.severity == "CRITICAL"]
warnings = [f for f in findings if f.severity == "WARNING"]
if critical:
icon, label = "🚨", "CRITICAL"
elif warnings:
icon, label = "⚠️ ", "WARN "
else:
icon, label = "", "PASS "
print(f" {icon} [{label}] {plugin_dir.name}"
f"{len(critical)} critical, {len(warnings)} warnings")
if args.verbose:
for f in findings:
severity_icon = {"CRITICAL": "🚨", "WARNING": "⚠️ ", "INFO": ""}.get(
f.severity, " "
)
print(f" {severity_icon} {f.rule} {f.file}:{f.line}{f.message}")
if args.plugin and not plugin_found:
print(f"\n 🚨 Plugin '{args.plugin}' not found in any of "
f"{[str(d.relative_to(PROJECT_ROOT)) for d in PLUGIN_BASE_DIRS]}"
f"nothing was audited")
return 1
# Summary
critical_findings = [f for f in all_findings if f.severity == "CRITICAL"]
warning_findings = [f for f in all_findings if f.severity == "WARNING"]
print(f"\n{'=' * 60}")
print(f" Plugins scanned : {plugins_scanned}")
print(f" CRITICAL : {len(critical_findings)}")
print(f" WARNING : {len(warning_findings)}")
if critical_findings:
print("\n 🚨 CRITICAL findings:")
for f in critical_findings:
print(f" {f.plugin_id} | {Path(f.file).name}:{f.line} | {f.message}")
# Write JSON output
if args.output:
output_data = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"plugins_scanned": plugins_scanned,
"summary": {
"critical": len(critical_findings),
"warnings": len(warning_findings),
},
"findings": [f.to_dict() for f in all_findings],
}
Path(args.output).write_text(
json.dumps(output_data, indent=2), encoding="utf-8"
)
print(f"\n Results written to: {args.output}")
if critical_findings:
print("\n 🚨 Blocking — CRITICAL issues must be resolved")
return 1
print("\n ✅ No critical issues found")
return 0
if __name__ == "__main__":
sys.exit(main())
+6 -7
View File
@@ -37,7 +37,7 @@ os.environ['EMULATOR'] = 'true'
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_config_defaults, load_harness_spec, load_manifest,
build_full_config, find_plugin_dir, load_harness_spec, load_manifest,
)
from src.plugin_system.testing.harness import ( # noqa: E402
RenderResult, render_plugin_matrix, compare_to_goldens, write_goldens,
@@ -97,12 +97,11 @@ def check_one(plugin_id: str, search_dirs: List[str], sizes, mock_data: Dict,
# matrix path does; explicit CLI flags still override the file.
spec = load_harness_spec(plugin_dir)
# config_schema defaults (real-install behavior), then harness.json config,
# then CLI --config — most specific wins.
full_config = {"enabled": True}
full_config.update(load_config_defaults(plugin_dir))
full_config.update(spec.get("config", {}))
full_config.update(config)
# config_schema defaults (real-install behavior, with enabled forced True
# so a plugin's own enabled:false default can't accidentally disable
# testing), then harness.json config, then CLI --config — most specific
# wins.
full_config = build_full_config(plugin_dir, spec, config)
# Precedence: CLI flag > LEDMATRIX_TEST_SIZES env > harness.json > default.
effective_sizes = sizes if sizes else resolve_test_sizes(spec.get("sizes"))
+1 -1
View File
@@ -55,7 +55,7 @@ def main():
failures += not check("draw.textbbox",
lambda: draw.textbbox((0, 0), "Test", font=font))
print("\nResampling (used in logo_helper, image_utils, sports base):")
print("\nResampling (used in logo_helper, sports base):")
logo = Image.new('RGBA', (200, 200), (255, 128, 0, 200))
failures += not check("Image.Resampling.LANCZOS exists",
lambda: str(Image.Resampling.LANCZOS))
+54 -8
View File
@@ -200,16 +200,62 @@ def _render_once(plugin_id, plugin_dir, manifest, config, mock_data, width, heig
skip_update):
"""Render one plugin at one size. Returns the /api/render response dict.
Thin wrapper over the shared render service (also used by the web UI's
config-page preview); a fresh plugin instance per call, mirroring the
safety harness, so sizes never share state.
A fresh plugin instance per call, mirroring the safety harness, so sizes
never share state.
"""
from src.plugin_system.testing.render_service import render_plugin_once
from src.plugin_system.testing import VisualTestDisplayManager, MockCacheManager, MockPluginManager
from src.plugin_system.plugin_loader import PluginLoader
return render_plugin_once(
plugin_id, plugin_dir, manifest=manifest, config=config,
mock_data=mock_data, width=width, height=height,
skip_update=skip_update)
display_manager = VisualTestDisplayManager(width=width, height=height)
cache_manager = MockCacheManager()
plugin_manager = MockPluginManager()
# Pre-populate cache with mock data
for key, value in mock_data.items():
cache_manager.set(key, value)
loader = PluginLoader()
errors = []
warnings = []
plugin_instance, _module = loader.load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=plugin_dir,
config=config,
display_manager=display_manager,
cache_manager=cache_manager,
plugin_manager=plugin_manager,
install_deps=False,
)
start_time = time.time()
# Run update()
if not skip_update:
try:
plugin_instance.update()
except Exception as e:
logger.warning("update() raised for plugin %s", plugin_id, exc_info=True)
warnings.append(f"update() raised: {type(e).__name__} — see server log")
# Run display()
try:
plugin_instance.display(force_clear=True)
except Exception as e:
logger.warning("display() raised for plugin %s", plugin_id, exc_info=True)
errors.append(f"display() raised: {type(e).__name__} — see server log")
render_time_ms = round((time.time() - start_time) * 1000, 1)
return {
'image': f'data:image/png;base64,{display_manager.get_image_base64()}',
'width': width,
'height': height,
'render_time_ms': render_time_ms,
'errors': errors,
'warnings': warnings,
}
def _trusted_plugin_dir(plugin_dir: Path) -> Optional[Path]:
+356
View File
@@ -0,0 +1,356 @@
#!/usr/bin/env python3
"""
Security Report Generator
Aggregates JSON output from all CI security audit jobs into a single
Markdown report suitable for PR comments and artifact storage.
Expected artifact layout (from actions/download-artifact@v4):
<artifact-dir>/
sast-results/
bandit-results.json
semgrep-results.json
dependency-audit-results/
pip-audit-results.json
safety-results.json
secrets-scan-results/
gitleaks-results.json
security-proofs-results/
security-proofs-results.json
plugin-audit-results/
plugin-audit-results.json
Usage:
python scripts/generate_report.py --artifact-dir audit-artifacts/ --output report.md
python scripts/generate_report.py --artifact-dir audit-artifacts/ --output report.md --verbose
"""
import argparse
import json
import sys
from pathlib import Path
from datetime import datetime, timezone
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# Gitleaks matches exactly equal to one of these (not a substring match -- a
# real secret that merely contains one of these words as part of its actual
# value must still be reported) are known template placeholders.
_GITLEAKS_SUPPRESS_EXACT_VALUES = {
"YOUR_YOUTUBE_API_KEY",
"YOUR_YOUTUBE_CHANNEL_ID",
"YOUR_GITHUB_PERSONAL_ACCESS_TOKEN",
}
# Findings in these files are suppressed regardless of value -- they are
# template/example files that are expected to only ever contain placeholders.
_GITLEAKS_SUPPRESS_PATHS = [
"config_secrets.template.json",
"config.template.json",
]
# ─────────────────────────────────────────────────────────────────────────────
# Helpers
# ─────────────────────────────────────────────────────────────────────────────
def _load(path: Path) -> tuple[dict | list | None, str | None]:
"""Load a JSON artifact file.
Returns (data, error): error is None on success (data is whatever was
parsed, which may legitimately be an empty list/dict for a clean scan);
otherwise error is a human-readable reason the artifact is unavailable,
distinguishing "missing/malformed artifact" from "valid empty result" so
callers don't silently treat a broken CI job as a clean pass.
"""
if not path.exists():
return None, f"artifact not found: {path}"
try:
return json.loads(path.read_text(encoding="utf-8")), None
except (json.JSONDecodeError, OSError) as exc:
return None, f"could not read/parse {path}: {exc}"
def _md_sanitize_cell(value: object) -> str:
"""Escape/normalize a value so scanner-controlled content (a matched
secret, a bandit issue_text, a file path) can't alter the Markdown
table's structure: pipes would add bogus columns, newlines would break
out of the row (or forge a fake header/separator line)."""
text = str(value)
text = text.replace("\\", "\\\\").replace("|", "\\|")
text = text.replace("\r\n", " ").replace("\n", " ").replace("\r", " ")
return text
def _md_table_row(*cells: str) -> str:
return "| " + " | ".join(_md_sanitize_cell(c) for c in cells) + " |"
# ─────────────────────────────────────────────────────────────────────────────
# Per-tool summarizers
# Returns: (markdown_lines: list[str], critical_count: int, available: bool)
# `available=False` means the artifact was missing or malformed -- distinct
# from a valid scan that simply found nothing -- so the caller can report
# INCOMPLETE instead of silently counting it as a clean pass.
# ─────────────────────────────────────────────────────────────────────────────
def _summarize_bandit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "sast-results" / "bandit-results.json")
if error:
return [f"_bandit results unavailable: {error}_"], 0, False
results = data.get("results", [])
high = [r for r in results if r.get("issue_severity") == "HIGH"]
medium = [r for r in results if r.get("issue_severity") == "MEDIUM"]
low = [r for r in results if r.get("issue_severity") == "LOW"]
lines = [
f"**Bandit**: {len(high)} HIGH · {len(medium)} MEDIUM · {len(low)} LOW"
]
if high:
lines += [
"",
"| Severity | File | Line | Issue |",
"| --- | --- | --- | --- |",
]
for r in high[:10]:
fname = Path(r.get("filename", "")).name
lines.append(_md_table_row(
"HIGH", f"`{fname}`",
str(r.get("line_number", "?")),
r.get("issue_text", "")
))
if len(high) > 10:
lines.append(f"_… and {len(high) - 10} more HIGH findings_")
return lines, len(high), True
def _summarize_pip_audit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "dependency-audit-results" / "pip-audit-results.json")
if error:
return [f"_pip-audit results unavailable: {error}_"], 0, False
# pip-audit JSON format: {"dependencies": [{"name": ..., "vulns": [...]}]}
vulns: list[dict] = []
for dep in data.get("dependencies", []):
for v in dep.get("vulns", []):
vulns.append({"package": dep.get("name", "?"), **v})
lines = [f"**pip-audit**: {len(vulns)} vulnerabilities found"]
if vulns:
lines += ["", "| Package | ID | Fix |", "| --- | --- | --- |"]
for v in vulns[:10]:
fix = v.get("fix_versions", ["none"])
fix_str = ", ".join(fix) if fix else "none"
lines.append(_md_table_row(
v.get("package", "?"),
v.get("id", "?"),
fix_str,
))
# Treat known vulnerabilities as warnings, not critical (they may be unavoidable)
return lines, 0, True
def _summarize_gitleaks(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "secrets-scan-results" / "gitleaks-results.json")
if error:
return [f"_gitleaks results unavailable: {error}_"], 0, False
if not isinstance(data, list):
data = []
real_findings = []
suppressed = 0
for finding in data:
secret_val = str(finding.get("Secret", "") or finding.get("Match", ""))
file_name = Path(finding.get("File", "")).name
if (secret_val in _GITLEAKS_SUPPRESS_EXACT_VALUES
or file_name in _GITLEAKS_SUPPRESS_PATHS):
suppressed += 1
else:
real_findings.append(finding)
lines = [
f"**Gitleaks**: {len(real_findings)} finding(s) "
f"({suppressed} suppressed as template placeholders)"
]
if real_findings:
lines += ["", "| Rule | File | Line | Description |", "| --- | --- | --- | --- |"]
for f in real_findings[:10]:
fname = Path(f.get("File", "")).name
lines.append(_md_table_row(
f.get("RuleID", "?"),
f"`{fname}`",
str(f.get("StartLine", "?")),
f.get("Description", ""),
))
critical = len(real_findings) # any real secret is critical
return lines, critical, True
def _summarize_security_proofs(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "security-proofs-results" / "security-proofs-results.json")
if error:
return [f"_security proofs results unavailable: {error}_"], 0, False
if not isinstance(data, list):
data = []
critical = [r for r in data if r.get("severity") == "CRITICAL"]
warnings = [r for r in data if r.get("severity") == "WARNING"]
passed = [r for r in data if r.get("severity") == "PASS"]
skipped = [r for r in data if r.get("severity") == "SKIP"]
lines = [
f"**Security Proofs**: "
f"{len(passed)} PASS · {len(warnings)} WARN · "
f"{len(critical)} CRITICAL · {len(skipped)} SKIP",
"",
]
_icon = {"PASS": "", "INFO": "", "WARNING": "⚠️", # nosec B105 - severity labels, not credentials
"CRITICAL": "🚨", "SKIP": "⏭️"}
for r in data:
icon = _icon.get(r.get("severity", ""), "")
lines.append(
f"- {icon} **{r.get('test_id', '?')}**: {r.get('message', '')}"
)
if r.get("details") and r.get("severity") in ("CRITICAL", "WARNING"):
lines.append(f" - _{r['details']}_")
return lines, len(critical), True
def _summarize_plugin_audit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "plugin-audit-results" / "plugin-audit-results.json")
if error:
return [f"_plugin audit results unavailable: {error}_"], 0, False
summary = data.get("summary", {})
findings = data.get("findings", [])
critical_findings = [f for f in findings if f.get("severity") == "CRITICAL"]
warning_findings = [f for f in findings if f.get("severity") == "WARNING"]
lines = [
f"**Plugin Audit**: {data.get('plugins_scanned', '?')} plugins scanned — "
f"{summary.get('critical', 0)} CRITICAL · {summary.get('warnings', 0)} WARNINGS"
]
if critical_findings:
lines += ["", "| Plugin | File | Line | Rule | Message |",
"| --- | --- | --- | --- | --- |"]
for f in critical_findings[:10]:
fname = Path(f.get("file", "")).name
lines.append(_md_table_row(
f.get("plugin_id", "?"),
f"`{fname}`",
str(f.get("line", "?")),
f.get("rule", "?"),
f.get("message", ""),
))
if warning_findings and not critical_findings:
lines.append(f"\n_{len(warning_findings)} warning(s) found — see artifact for details_")
return lines, summary.get("critical", 0), True
# ─────────────────────────────────────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="Generate consolidated security audit report",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--artifact-dir", required=True,
help="Directory containing downloaded CI artifacts")
parser.add_argument("--output", "-o", required=True,
help="Output Markdown file path")
parser.add_argument("--verbose", "-v", action="store_true")
args = parser.parse_args()
artifact_dir = Path(args.artifact_dir)
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
bandit_lines, bandit_crit, bandit_ok = _summarize_bandit(artifact_dir)
pip_audit_lines, pip_audit_crit, pip_audit_ok = _summarize_pip_audit(artifact_dir)
gitleaks_lines, gitleaks_crit, gitleaks_ok = _summarize_gitleaks(artifact_dir)
proofs_lines, proofs_crit, proofs_ok = _summarize_security_proofs(artifact_dir)
plugins_lines, plugins_crit, plugins_ok = _summarize_plugin_audit(artifact_dir)
unavailable_tools = [
name for name, ok in [
("bandit", bandit_ok), ("pip-audit", pip_audit_ok),
("gitleaks", gitleaks_ok), ("security-proofs", proofs_ok),
("plugin-audit", plugins_ok),
] if not ok
]
total_critical = bandit_crit + pip_audit_crit + gitleaks_crit + proofs_crit + plugins_crit
if unavailable_tools:
# A missing/malformed artifact means that tool's checks never
# actually ran -- this must not be reported as a clean PASS just
# because the *artifacts that did load* found nothing.
overall = "INCOMPLETE ⚠️"
elif total_critical > 0:
overall = "ACTION REQUIRED 🚨"
else:
overall = "PASSED ✅"
def section(title: str, lines: list[str]) -> str:
return f"### {title}\n\n" + "\n".join(lines) + "\n"
incomplete_note = (
f"\n_⚠️ Incomplete: results unavailable for {', '.join(unavailable_tools)} "
f"— see the corresponding section(s) below for details_\n"
if unavailable_tools else ""
)
report = f"""## 🔒 Security Audit — {overall}
_Generated: {timestamp}_
{incomplete_note}
| Critical | High/Warn | Overall |
| :---: | :---: | :---: |
| {'🚨 ' + str(total_critical) if total_critical else '✅ 0'} | ⚠️ see below | {overall} |
---
{section('SAST — Bandit', bandit_lines)}
{section('Dependencies — pip-audit', pip_audit_lines)}
{section('Secrets — Gitleaks', gitleaks_lines)}
{section('LEDMatrix Security Proofs', proofs_lines)}
{section('Plugin Security Audit', plugins_lines)}
---
_Total critical findings: **{total_critical}**_
"""
output_path = Path(args.output)
output_path.write_text(report, encoding="utf-8")
if args.verbose:
print(f" Report written to: {output_path}")
print(f" Status: {overall}")
print(f" Critical findings: {total_critical}")
print(f" bandit={bandit_crit} pip-audit={pip_audit_crit} "
f"gitleaks={gitleaks_crit} proofs={proofs_crit} plugins={plugins_crit}")
if unavailable_tools:
print(f" Unavailable: {', '.join(unavailable_tools)}")
if unavailable_tools:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+593
View File
@@ -0,0 +1,593 @@
#!/usr/bin/env python3
"""
LEDMatrix Security Proof Tests
Automated proofs that run in CI to verify security properties hold on every
commit. Inspired by the Huntarr security review approach of using standard
tooling to confirm specific vulnerability classes are absent.
Usage:
python scripts/prove_security.py
python scripts/prove_security.py --verbose
python scripts/prove_security.py --output results.json
Exit code: 1 only if CRITICAL findings are detected. Warnings are reported
but do not block CI.
"""
import ast
import argparse
import hashlib
import json
import re
import sys
from dataclasses import dataclass, asdict
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# ─────────────────────────────────────────────────────────────────────────────
# Result dataclass
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class TestResult:
test_id: str
severity: str # PASS | INFO | WARNING | CRITICAL | SKIP
message: str
details: str = ""
def to_dict(self) -> dict:
return asdict(self)
@property
def icon(self) -> str:
return {
"PASS": "", # nosec B105 - severity label, not a credential
"INFO": "",
"WARNING": "⚠️ ",
"CRITICAL": "🚨",
"SKIP": "⏭️ ",
}.get(self.severity, "")
# ─────────────────────────────────────────────────────────────────────────────
# T1: Plugin Loading / Zip Slip
# ─────────────────────────────────────────────────────────────────────────────
def test_t1a_zip_slip_protection() -> TestResult:
"""
Verify that zip-slip protection actually guards zip extraction in
store_manager.py.
A whole-file substring check for "is_relative_to"/"Zip-slip detected"
would pass even if the guard existed somewhere unrelated, or covered
only one of several extract()/extractall() call sites. Instead, this
walks the AST: for every extract()/extractall() call, it confirms an
is_relative_to() check (and the "Zip-slip detected" log) appears
earlier in that same enclosing function -- validate-then-bulk-extract
(validate every member, then call extractall() only after all passed)
counts as protecting the call, since it covers the same member list.
"""
store_manager = PROJECT_ROOT / "src" / "plugin_system" / "store_manager.py"
if not store_manager.exists():
return TestResult("T1a", "CRITICAL",
"store_manager.py not found",
f"Expected at {store_manager}")
content = store_manager.read_text(encoding="utf-8")
try:
tree = ast.parse(content, filename=str(store_manager))
except SyntaxError as exc:
return TestResult("T1a", "CRITICAL",
"store_manager.py could not be parsed",
str(exc))
extraction_sites = 0
unprotected: list[str] = []
for func in ast.walk(tree):
if not isinstance(func, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
extract_calls = [
node for node in ast.walk(func)
if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute)
and node.func.attr in ("extract", "extractall")
]
if not extract_calls:
continue
extraction_sites += len(extract_calls)
guard_lines = [
n.lineno for n in ast.walk(func)
if isinstance(n, ast.Attribute) and n.attr == "is_relative_to"
]
has_zip_slip_log = any(
isinstance(n, ast.Constant) and isinstance(n.value, str)
and "Zip-slip detected" in n.value
for n in ast.walk(func)
)
for call in extract_calls:
guarded = has_zip_slip_log and any(g < call.lineno for g in guard_lines)
if not guarded:
unprotected.append(
f"{func.name}() line {call.lineno}: {call.func.attr}() call not "
f"clearly preceded by an is_relative_to() guard + Zip-slip log "
f"in the same function"
)
if extraction_sites == 0:
return TestResult("T1a", "WARNING",
"No zipfile extract()/extractall() calls found in store_manager.py",
"Verify plugin installation no longer extracts zip archives, "
"or that this check still targets the right file")
if unprotected:
return TestResult("T1a", "CRITICAL",
f"{len(unprotected)} of {extraction_sites} zip extraction "
f"call(s) not clearly guarded",
"; ".join(unprotected))
return TestResult("T1a", "PASS",
"Zip-slip protection verified",
f"All {extraction_sites} extract()/extractall() call(s) in "
f"store_manager.py are preceded by an is_relative_to() guard "
f"with a Zip-slip log in the same function")
def test_t1b_dangerous_plugin_calls() -> list[TestResult]:
"""
Scan plugin directories for dangerous function calls (eval, exec).
These represent arbitrary code execution risks in plugin code.
"""
results = []
plugin_dirs = [
PROJECT_ROOT / "plugins",
PROJECT_ROOT / "plugin-repos",
]
violations: list[str] = []
files_scanned = 0
scan_errors: list[str] = []
for base in plugin_dirs:
if not base.exists():
continue
for plugin_dir in sorted(base.iterdir()):
if not plugin_dir.is_dir() or plugin_dir.name.startswith(('.', '_')):
continue
for py_file in plugin_dir.rglob("*.py"):
files_scanned += 1
try:
source = py_file.read_text(encoding="utf-8")
tree = ast.parse(source, filename=str(py_file))
for node in ast.walk(tree):
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
if node.func.id in ("eval", "exec"):
rel = py_file.relative_to(PROJECT_ROOT)
violations.append(
f"{rel}:{node.lineno}{node.func.id}() call")
except (SyntaxError, OSError) as exc:
# A file we couldn't parse/read was never actually
# scanned for eval()/exec() -- that must block this
# test, not silently pass as if it were clean.
rel = py_file.relative_to(PROJECT_ROOT)
scan_errors.append(f"{rel}{type(exc).__name__}: {exc}")
if scan_errors:
results.append(TestResult(
"T1b", "CRITICAL",
f"{len(scan_errors)} plugin file(s) could not be scanned for eval()/exec()",
"; ".join(scan_errors[:10])
))
if violations:
results.append(TestResult(
"T1b", "CRITICAL",
f"Dangerous function calls found in plugins ({len(violations)} instance(s))",
"; ".join(violations[:10])
))
elif not scan_errors:
results.append(TestResult(
"T1b", "PASS",
"No eval()/exec() calls found in plugins",
f"{files_scanned} plugin Python files scanned"
))
return results
# ─────────────────────────────────────────────────────────────────────────────
# T2: API Surface Inventory
# ─────────────────────────────────────────────────────────────────────────────
def test_t2a_api_surface_inventory() -> TestResult:
"""
Document the API surface area.
This app intentionally has no authentication (local-only Raspberry Pi
design, documented in web_interface/app.py). This test produces an
inventory for audit purposes and warns only if the design-intent comment
is removed from app.py (which would indicate someone deleted the rationale
without adding auth, rather than a deliberate undocumented change).
"""
api_file = PROJECT_ROOT / "web_interface" / "blueprints" / "api_v3.py"
app_file = PROJECT_ROOT / "web_interface" / "app.py"
if not api_file.exists():
return TestResult("T2a", "WARNING", "api_v3.py not found", str(api_file))
api_content = api_file.read_text(encoding="utf-8")
routes = re.findall(r"@api_v3\.route\('([^']+)'", api_content)
csrf_documented = False
if app_file.exists():
app_content = app_file.read_text(encoding="utf-8")
csrf_documented = "CSRF protection disabled for local-only" in app_content
summary = (
f"{len(routes)} API routes in api_v3.py. "
f"No auth decorators (intentional local-only design). "
f"CSRF disabled: {'YES — design intent documented in app.py' if csrf_documented else 'YES — but design intent comment NOT found in app.py'}. "
f"Rate limiting: 1000/min."
)
if not csrf_documented:
return TestResult(
"T2a", "WARNING",
"CSRF is disabled but the design-intent comment is missing from app.py",
"Add the rationale comment back, or add proper CSRF protection if "
"the app is now internet-facing"
)
# There is currently no config mechanism that actually enforces the
# local-only boundary the design-intent comment describes -- app.py
# hardcodes host='0.0.0.0' unconditionally, so nothing here can confirm
# this deployment is in fact LAN-only. Reporting this as mere INFO
# understates that: an unauthenticated, CSRF-disabled API surface is a
# real risk the moment this ever runs somewhere other than a home LAN,
# documented rationale or not.
return TestResult(
"T2a", "WARNING",
"API surface has no auth and CSRF disabled; enforcement of the "
"documented local-only boundary cannot be confirmed",
summary
)
# ─────────────────────────────────────────────────────────────────────────────
# T3: Secrets & Credential Handling
# ─────────────────────────────────────────────────────────────────────────────
# Patterns that suggest real credentials (must be >8 chars, not placeholders)
_SECRET_PATTERNS = [
(r'(?i)password\s*=\s*["\'](?!none|empty|placeholder|example|test|default|""|'')[^"\']{8,}["\']', "WARNING", "password"),
(r'(?i)api[_-]?key\s*=\s*["\'](?!none|empty|placeholder|YOUR_|example|test)[^"\']{16,}["\']', "WARNING", "api_key"),
(r'(?i)secret\s*=\s*["\'](?!none|empty|placeholder|YOUR_|example|test)[^"\']{16,}["\']', "WARNING", "secret"),
# Real GitHub token pattern
(r'ghp_[a-zA-Z0-9]{36}', "CRITICAL", "github_token"),
# Generic long bearer tokens
(r'Bearer\s+[a-zA-Z0-9\-_\.]{32,}', "WARNING", "bearer_token"),
]
_TEMPLATE_SKIP_STRINGS = [
"YOUR_", "PLACEHOLDER", "_HERE", "example.com", "config_secrets.template",
"prove_security", # this file itself
]
_SCAN_DIRS = ["src", "web_interface", "scripts"]
def test_t3a_hardcoded_secrets() -> TestResult:
"""Scan source code for hardcoded credentials."""
violations: list[str] = []
for dir_name in _SCAN_DIRS:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
# Skip test files and this script
if "test" in str(py_file).lower() or "prove_security" in str(py_file):
continue
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
for pattern, severity, pattern_type in _SECRET_PATTERNS:
for match in re.finditer(pattern, content):
line_content = match.group(0)
# Skip lines containing template placeholder strings.
# line_content is only used for this in-memory check --
# it must never be stored or included in output below.
if any(skip in line_content for skip in _TEMPLATE_SKIP_STRINGS):
continue
rel = py_file.relative_to(PROJECT_ROOT)
line_no = content[: match.start()].count("\n") + 1
# Redacted fingerprint lets the same finding be recognized
# across scans without ever reporting the matched
# credential itself (which would otherwise get published
# into CI logs, JSON artifacts, and PR comments -- wider
# exposure than the original leak).
fingerprint = hashlib.sha256(line_content.encode()).hexdigest()[:12]
violations.append(
f"[{severity}] {rel}:{line_no}{pattern_type} "
f"(fingerprint {fingerprint})"
)
critical_violations = [v for v in violations if "[CRITICAL]" in v]
if critical_violations:
return TestResult(
"T3a", "CRITICAL",
f"Hardcoded secrets found ({len(critical_violations)} critical)",
"; ".join(critical_violations[:5])
)
if violations:
return TestResult(
"T3a", "WARNING",
f"Potential hardcoded secrets found ({len(violations)} instance(s))",
"; ".join(violations[:5])
)
return TestResult("T3a", "PASS", "No hardcoded secrets detected",
f"Scanned {', '.join(_SCAN_DIRS)}")
def test_t3b_plaintext_password_storage() -> TestResult:
"""
Check for user account password storage without hashing.
The LEDMatrix app has no user account system, so this should produce INFO.
It would only CRITICAL if someone added user auth and stored passwords without hashing.
We require all three of: a password *variable assignment or DB operation*,
a clear storage call (INSERT / db commit / ORM save), and no hashing lib present
— to avoid false positives from files that contain 'password' for WiFi handling
and '.save()' for image/file saving in unrelated functions.
"""
hashing_libs = ["bcrypt", "argon2", "pbkdf2", "scrypt",
"generate_password_hash", "hashpw", "make_password"]
# Patterns that indicate password being stored in a database / ORM context.
# Must be specific enough to avoid matching set.add(), file.save(), etc.
db_storage_patterns = ["INSERT INTO", "db.session", "session.add(", "session.commit(", "orm.save"]
password_storage_found = False
for dir_name in _SCAN_DIRS:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
# Require DB/ORM context specifically — not just any .save() call
if ("password" in content.lower() and
any(store in content for store in db_storage_patterns) and
not any(h in content for h in hashing_libs)):
password_storage_found = True
if password_storage_found:
return TestResult(
"T3b", "CRITICAL",
"Potential plaintext password storage in database/ORM detected",
"Found password + database storage operations without a recognized hashing library"
)
return TestResult("T3b", "INFO",
"No plaintext password storage detected",
"App has no user account system — expected result")
# ─────────────────────────────────────────────────────────────────────────────
# T4: Path Traversal
# ─────────────────────────────────────────────────────────────────────────────
def test_t4a_path_traversal() -> TestResult:
"""
Verify static file serving uses send_from_directory (safe) rather than
open() with user-supplied paths. Also checks for extractall() calls that
lack the is_relative_to() guard.
"""
issues: list[str] = []
app_file = PROJECT_ROOT / "web_interface" / "app.py"
if app_file.exists():
content = app_file.read_text(encoding="utf-8")
# The file-serve route should use send_from_directory or commonpath
if "send_from_directory" not in content and "commonpath" not in content:
issues.append("app.py: file-serve routes may not use send_from_directory/commonpath")
# Check all extractall() calls have a preceding is_relative_to guard
for py_file in (PROJECT_ROOT / "src").rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
if "extractall(" in content and "is_relative_to" not in content:
rel = py_file.relative_to(PROJECT_ROOT)
issues.append(f"{rel}: extractall() without is_relative_to() guard")
if issues:
return TestResult(
"T4a", "WARNING",
f"Potential path traversal patterns found ({len(issues)})",
"; ".join(issues)
)
return TestResult("T4a", "PASS",
"Path traversal mitigations verified",
"send_from_directory/commonpath used for file serving; "
"extractall() calls have is_relative_to() guards")
# ─────────────────────────────────────────────────────────────────────────────
# T5: Auth Bypass Patterns
# ─────────────────────────────────────────────────────────────────────────────
def test_t5a_auth_bypass_patterns() -> TestResult:
"""
Look for broken auth bypass patterns — not the intentional no-auth design
(T2a covers that), but patterns that suggest auth was INTENDED to exist
but has an exploitable bypass: broad substring matching, debug-mode skips,
or if-True conditions.
"""
bypass_signals = [
(r'if\s+True\s*:', "if True: bypass"),
(r'if\s+debug\s*:', "debug-mode auth skip"),
(r'request\.path\s+in\s+', "substring path matching in auth (Huntarr pattern)"),
(r'EXEMPT_ROUTES\s*=', "exempt routes list"),
]
findings: list[str] = []
for dir_name in ["src", "web_interface"]:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
for pattern, label in bypass_signals:
if re.search(pattern, content):
# Only flag if the file also contains auth-related terms
if any(auth in content.lower() for auth in
["auth", "login", "authenticate", "token", "permission"]):
rel = py_file.relative_to(PROJECT_ROOT)
findings.append(f"{rel}: {label}")
if findings:
return TestResult(
"T5a", "WARNING",
f"Potential auth bypass patterns found ({len(findings)})",
"; ".join(findings[:5])
)
return TestResult("T5a", "PASS",
"No auth bypass patterns detected",
"Checked src/ and web_interface/ for bypass signals")
# ─────────────────────────────────────────────────────────────────────────────
# T6: Docker / Container Hardening
# ─────────────────────────────────────────────────────────────────────────────
def test_t6_docker_hardening() -> TestResult:
"""Container security — skipped if no Dockerfile exists."""
dockerfile = PROJECT_ROOT / "Dockerfile"
if not dockerfile.exists():
return TestResult("T6", "SKIP",
"No Dockerfile found — container security scan not applicable",
"If Docker support is added in future, enable hadolint/trivy scanning "
"in .github/workflows/security-audit.yml")
content = dockerfile.read_text(encoding="utf-8")
issues: list[str] = []
# Check for non-root USER directive
user_lines = [l for l in content.splitlines() if l.strip().startswith("USER")]
if not user_lines or user_lines[-1].strip() == "USER root":
issues.append("Container runs as root — use USER directive to drop privileges")
# Check for pinned base image tags. A tag (even a specific version, not
# just :latest) is mutable -- the same tag can point to a different
# image later. Only a @sha256 digest is truly immutable/reproducible.
from_lines = [line for line in content.splitlines() if line.strip().startswith("FROM")]
for from_line in from_lines:
parts = from_line.split()
# FROM [--platform=<platform>] <image> [AS <name>] -- skip an
# optional --platform= flag so it's never mistaken for the image
# token itself (which would falsely report it as unpinned).
image_parts = [p for p in parts[1:] if not p.startswith("--platform=")]
if image_parts:
image = image_parts[0]
if "@sha256:" not in image:
issues.append(f"Base image not pinned to a digest: {image}")
if issues:
return TestResult("T6", "WARNING",
f"Dockerfile hardening issues ({len(issues)})",
"; ".join(issues))
return TestResult("T6", "PASS", "Dockerfile hardening checks passed", "")
# ─────────────────────────────────────────────────────────────────────────────
# Runner
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="LEDMatrix security proof tests",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--output", "-o", default=None,
help="Write JSON results to this file")
parser.add_argument("--verbose", "-v", action="store_true",
help="Show details for each check")
args = parser.parse_args()
print("=" * 60)
print("LEDMatrix Security Proof Tests")
print(f"Project root: {PROJECT_ROOT}")
print("=" * 60)
all_results: list[TestResult] = []
# Run all test groups
all_results.append(test_t1a_zip_slip_protection())
all_results.extend(test_t1b_dangerous_plugin_calls())
all_results.append(test_t2a_api_surface_inventory())
all_results.append(test_t3a_hardcoded_secrets())
all_results.append(test_t3b_plaintext_password_storage())
all_results.append(test_t4a_path_traversal())
all_results.append(test_t5a_auth_bypass_patterns())
all_results.append(test_t6_docker_hardening())
# Print results
print()
for r in all_results:
line = f" {r.icon} [{r.severity:<8}] {r.test_id}: {r.message}"
print(line)
if args.verbose and r.details:
print(f" {r.details}")
# Tally
critical = [r for r in all_results if r.severity == "CRITICAL"]
warnings = [r for r in all_results if r.severity == "WARNING"]
passed = [r for r in all_results if r.severity == "PASS"]
skipped = [r for r in all_results if r.severity == "SKIP"]
print()
print(f" Results: {len(passed)} PASS {len(warnings)} WARN "
f"{len(critical)} CRITICAL {len(skipped)} SKIP")
# Write JSON output
if args.output:
output_data = [r.to_dict() for r in all_results]
Path(args.output).write_text(
json.dumps(output_data, indent=2), encoding="utf-8"
)
print(f" Results written to: {args.output}")
if critical:
print(f"\n 🚨 {len(critical)} CRITICAL issue(s) found — blocking")
return 1
if warnings:
print(f"\n ⚠️ {len(warnings)} warning(s) found — non-blocking")
print("\n ✅ All checks passed (warnings are non-blocking)")
return 0
if __name__ == "__main__":
sys.exit(main())
+2 -5
View File
@@ -28,7 +28,7 @@ os.environ['EMULATOR'] = 'true'
# Import logger after path setup so src.logging_config is importable
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_manifest, load_config_defaults,
build_full_config, find_plugin_dir, load_manifest,
)
logger = get_logger("[Render Plugin]")
@@ -83,16 +83,13 @@ def main() -> int:
manifest = load_manifest(Path(plugin_dir))
# Parse config: start with schema defaults, then apply overrides
config_defaults = load_config_defaults(Path(plugin_dir))
try:
user_config = json.loads(args.config)
except json.JSONDecodeError as e:
logger.error("Invalid JSON config: %s", e)
return 1
config = {'enabled': True}
config.update(config_defaults)
config.update(user_config)
config = build_full_config(Path(plugin_dir), cli_config=user_config)
# Load mock data if provided
mock_data = {}
+12 -16
View File
@@ -78,21 +78,17 @@ class WiFiMonitorDaemon:
while self.running:
try:
# Get current status before checking
status = self.wifi_manager.get_wifi_status()
ethernet_connected = self.wifi_manager._is_ethernet_connected()
# Check WiFi status and manage AP mode
state_changed = self.wifi_manager.check_and_manage_ap_mode()
# Get updated status after check
updated_status = self.wifi_manager.get_wifi_status()
updated_ethernet = self.wifi_manager._is_ethernet_connected()
# One combined check that also returns the state it observed —
# the previous flow fetched status before AND after the check
# on top of the check's own internal fetch, each one several
# nmcli subprocess forks, every 30s, forever.
(state_changed, updated_status, updated_ethernet,
ap_active) = self.wifi_manager.check_and_manage_ap_mode_with_state()
current_state = {
'connected': updated_status.connected,
'ethernet_connected': updated_ethernet,
'ap_active': updated_status.ap_mode_active,
'ap_active': ap_active,
'ssid': updated_status.ssid
}
@@ -109,7 +105,7 @@ class WiFiMonitorDaemon:
else:
logger.debug("Ethernet not connected")
if updated_status.ap_mode_active:
if ap_active:
logger.info(f"AP mode ACTIVE - SSID: {ap_ssid} (IP: 192.168.4.1)")
else:
logger.debug("AP mode inactive")
@@ -123,16 +119,16 @@ class WiFiMonitorDaemon:
# Log periodic status (less verbose)
if updated_status.connected:
logger.debug(f"Status check: WiFi={updated_status.ssid} ({updated_status.signal}%), "
f"Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
f"Ethernet={updated_ethernet}, AP={ap_active}")
else:
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={ap_active}")
# Escalating recovery: if nmcli reports connected but actual internet
# is unreachable for several consecutive checks, restart NetworkManager.
# This is done HERE (not inside check_and_manage_ap_mode) to keep the
# AP-enable trigger clean and avoid false-positive AP enables from
# transient packet loss on otherwise working WiFi.
if updated_status.connected and not updated_status.ap_mode_active:
if updated_status.connected and not ap_active:
if not self.wifi_manager.check_internet_connectivity():
self._consecutive_internet_failures += 1
logger.warning(
-134
View File
@@ -1,134 +0,0 @@
"""
Background Cache Mixin for Sports Managers
This mixin provides common caching functionality to eliminate code duplication
across all sports managers. It implements the background service cache pattern
where Recent/Upcoming managers consume data from the background service cache.
"""
import time
from typing import Dict, Optional, Any, Callable
class BackgroundCacheMixin:
"""
Mixin class that provides background service cache functionality to sports managers.
This mixin eliminates code duplication by providing a common implementation
for the background service cache pattern used across all sports managers.
Note: For non-sports managers (weather, stocks, news, etc.), use
GenericCacheMixin instead. See src/generic_cache_mixin.py for details.
"""
def _fetch_data_with_background_cache(self,
sport_key: str,
api_fetch_method: Callable,
live_manager_class: type = None) -> Optional[Dict]:
"""
Common logic for fetching data with background service cache support.
This method implements the background service cache pattern:
1. Live managers always fetch fresh data
2. Recent/Upcoming managers try background cache first
3. Fallback to direct API call if background data unavailable
Args:
sport_key: Sport identifier (e.g., 'nba', 'nfl', 'ncaa_fb')
api_fetch_method: Method to call for direct API fetch
live_manager_class: Class to check if this is a live manager
Returns:
Cached or fresh data from API
"""
start_time = time.time()
cache_hit = False
cache_source = None
try:
# For Live managers, always fetch fresh data
if live_manager_class and isinstance(self, live_manager_class):
self.logger.info(f"[{sport_key.upper()}] Live manager - fetching fresh data")
result = api_fetch_method(use_cache=False)
cache_source = "live_fresh"
else:
# For Recent/Upcoming managers, try background service cache first
cache_key = self.cache_manager.generate_sport_cache_key(sport_key)
# Check if background service has fresh data
if self.cache_manager.is_background_data_available(cache_key, sport_key):
cached_data = self.cache_manager.get_background_cached_data(cache_key, sport_key)
if cached_data:
self.logger.info(f"[{sport_key.upper()}] Using background service cache for {cache_key}")
result = cached_data
cache_hit = True
cache_source = "background_cache"
else:
self.logger.warning(f"[{sport_key.upper()}] Background cache check passed but no data returned for {cache_key}")
result = None
cache_source = "background_miss"
else:
self.logger.info(f"[{sport_key.upper()}] Background data not available for {cache_key}")
result = None
cache_source = "background_unavailable"
# Fallback to direct API call if background data not available
if result is None:
self.logger.info(f"[{sport_key.upper()}] Fetching directly from API for {cache_key}")
result = api_fetch_method(use_cache=True)
cache_source = "api_fallback"
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
# Log performance metrics
self._log_fetch_performance(sport_key, duration, cache_hit, cache_source)
return result
except Exception as e:
duration = time.time() - start_time
self.logger.error(f"[{sport_key.upper()}] Error in background cache fetch after {duration:.2f}s: {e}")
self.cache_manager.record_fetch_time(duration)
raise
def _log_fetch_performance(self, sport_key: str, duration: float, cache_hit: bool, cache_source: str):
"""
Log detailed performance metrics for fetch operations.
Args:
sport_key: Sport identifier
duration: Fetch operation duration in seconds
cache_hit: Whether this was a cache hit
cache_source: Source of the data (background_cache, api_fallback, etc.)
"""
# Log basic performance info
self.logger.info(f"[{sport_key.upper()}] Fetch completed in {duration:.2f}s "
f"(cache_hit={cache_hit}, source={cache_source})")
# Log detailed metrics every 10 operations
if hasattr(self, '_fetch_count'):
self._fetch_count += 1
else:
self._fetch_count = 1
if self._fetch_count % 10 == 0:
metrics = self.cache_manager.get_cache_metrics()
self.logger.info(f"[{sport_key.upper()}] Cache Performance Summary - "
f"Hit Rate: {metrics['cache_hit_rate']:.2%}, "
f"Background Hit Rate: {metrics['background_hit_rate']:.2%}, "
f"API Calls Saved: {metrics['api_calls_saved']}")
def get_cache_performance_summary(self) -> Dict[str, Any]:
"""
Get cache performance summary for this manager.
Returns:
Dictionary containing cache performance metrics
"""
return self.cache_manager.get_cache_metrics()
def log_cache_performance(self):
"""Log current cache performance metrics."""
self.cache_manager.log_cache_metrics()
+48 -9
View File
@@ -10,6 +10,7 @@ import time
import tempfile
import logging
import threading
import zlib
from typing import Dict, Any, Optional, Protocol
from datetime import datetime
@@ -53,6 +54,11 @@ class DiskCache:
self.cache_dir = cache_dir
self.logger = logger or logging.getLogger(__name__)
self._lock = threading.Lock()
# key -> adler32 of the last payload successfully written to the
# primary cache path; lets set() skip rewriting identical data
# (per-process only — worst case another process rewrites, never
# a missed write). Guarded by _lock.
self._write_digests: Dict[str, int] = {}
def get_cache_path(self, key: str) -> Optional[str]:
"""
@@ -155,10 +161,35 @@ class DiskCache:
cache_path = self.get_cache_path(key)
if not cache_path:
return
# Serialize once, compact (no indent): the payload is reused by every
# write path below, and cache files are machine-read only — indenting
# them just multiplied the bytes written to the SD card.
try:
payload = json.dumps(data, cls=DateTimeEncoder)
except (TypeError, ValueError) as e:
self.logger.warning("Cache data for key '%s' not serializable: %s", key, e)
return
digest = zlib.adler32(payload.encode('utf-8'))
try:
# Atomic write to avoid partial/corrupt files
with self._lock:
# Skip the disk entirely when this exact payload was already
# written for this key (plugins re-save unchanged API data
# every update cycle — each write is real SD-card wear).
# Refresh the file mtime so records that rely on it for TTL
# (no embedded 'timestamp') don't expire early; a metadata
# touch is journal-cheap compared to rewriting the data.
if self._write_digests.get(key) == digest:
try:
os.utime(cache_path, None)
return
except OSError:
# File vanished or perms changed — fall through and write
self._write_digests.pop(key, None)
tmp_dir = os.path.dirname(cache_path)
# Try to create temp file in cache directory first
# If that fails due to permissions, fall back to direct write
@@ -181,13 +212,17 @@ class DiskCache:
fd = None
if tmp_path and fd is not None:
# Use atomic write with temp file
# Atomic write with temp file. No fsync: os.replace
# already guarantees readers never see a torn file,
# and cache data is re-fetchable — forcing a disk
# flush per write was the single biggest SD-card
# wear source (dozens of fsyncs/min on API-heavy
# installs) for data that can be re-downloaded.
try:
with os.fdopen(fd, 'w', encoding='utf-8') as tmp_file:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
tmp_file.flush()
os.fsync(tmp_file.fileno())
tmp_file.write(payload)
os.replace(tmp_path, cache_path)
self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -203,9 +238,8 @@ class DiskCache:
# Fallback: direct write (not atomic, but better than failing)
try:
with open(cache_path, 'w', encoding='utf-8') as cache_file:
json.dump(data, cache_file, indent=4, cls=DateTimeEncoder)
cache_file.flush()
os.fsync(cache_file.fileno())
cache_file.write(payload)
self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -229,9 +263,12 @@ class DiskCache:
pass
if os.path.isdir(fallback_dir) and os.access(fallback_dir, os.W_OK):
# NOTE: no digest record here — the fallback file
# is a different path, so future sets must keep
# retrying the primary location.
fallback_path = os.path.join(fallback_dir, os.path.basename(cache_path))
with open(fallback_path, 'w', encoding='utf-8') as tmp_file:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
tmp_file.write(payload)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(fallback_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -272,6 +309,7 @@ class DiskCache:
with self._lock:
if key:
self._write_digests.pop(key, None)
cache_path = self.get_cache_path(key)
if cache_path and os.path.exists(cache_path):
try:
@@ -280,6 +318,7 @@ class DiskCache:
self.logger.warning("Could not remove cache file %s: %s", cache_path, e)
else:
# Clear all cache files
self._write_digests.clear()
if os.path.exists(self.cache_dir):
for filename in os.listdir(self.cache_dir):
if filename.endswith('.json'):
-328
View File
@@ -1,328 +0,0 @@
"""
Example: Basketball Plugin using LEDMatrix Common Helpers
This example shows how to refactor the basketball plugin to use the
ledmatrix-common package for cleaner, more maintainable code.
"""
from pathlib import Path
from typing import Any, Dict, List, Optional
# Import common helpers
from src.common import (
LogoHelper, TextHelper, APIHelper, DisplayHelper,
GameHelper, ConfigHelper
)
from src.plugin_system.base_plugin import BasePlugin
class BasketballPluginManager(BasePlugin):
"""
Basketball scoreboard plugin using LEDMatrix Common helpers.
This version is much cleaner and more maintainable than the original
because it delegates common functionality to the shared helpers.
"""
def __init__(
self,
plugin_id: str,
config: Dict[str, Any],
display_manager,
cache_manager,
plugin_manager
):
"""Initialize the basketball plugin with common helpers."""
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# Get display dimensions
self.display_width = display_manager.matrix.width
self.display_height = display_manager.matrix.height
# Initialize common helpers
self._init_helpers()
# Load configuration
self._load_config()
# State tracking
self.current_games = []
self.current_game = None
# Log initialization
enabled_leagues = [k for k, v in self.league_configs.items() if v['enabled']]
self.logger.info(f"Basketball plugin initialized with leagues: {enabled_leagues}")
def _init_helpers(self):
"""Initialize all common helpers."""
# Logo helper for team logos
self.logo_helper = LogoHelper(
display_width=self.display_width,
display_height=self.display_height,
logger=self.logger
)
# Text helper for rendering
self.text_helper = TextHelper(logger=self.logger)
self.fonts = self.text_helper.load_fonts()
# API helper for ESPN data
self.api_helper = APIHelper(
cache_manager=self.cache_manager,
logger=self.logger
)
# Display helper for layouts
self.display_helper = DisplayHelper(
display_width=self.display_width,
display_height=self.display_height,
logger=self.logger
)
# Game helper for data processing
self.game_helper = GameHelper(
timezone_str=self.config.get('timezone', 'UTC'),
logger=self.logger
)
# Config helper for configuration management
self.config_helper = ConfigHelper(logger=self.logger)
def _load_config(self):
"""Load and validate configuration."""
# Get basketball-specific config
basketball_config = self.config_helper.get_sports_config(self.config, 'basketball')
# Build league configurations
self.league_configs = {
'nba': {
'enabled': basketball_config.get('nba_enabled', True),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/nba/scoreboard',
'logo_dir': Path('assets/sports/nba_logos'),
'favorite_teams': basketball_config.get('nba_favorite_teams', []),
'display_modes': {
'nba_live': basketball_config.get('nba_display_modes_live', True),
'nba_recent': basketball_config.get('nba_display_modes_recent', True),
'nba_upcoming': basketball_config.get('nba_display_modes_upcoming', True),
},
},
'wnba': {
'enabled': basketball_config.get('wnba_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/wnba/scoreboard',
'logo_dir': Path('assets/sports/wnba_logos'),
'favorite_teams': basketball_config.get('wnba_favorite_teams', []),
'display_modes': {
'wnba_live': basketball_config.get('wnba_display_modes_live', True),
'wnba_recent': basketball_config.get('wnba_display_modes_recent', True),
'wnba_upcoming': basketball_config.get('wnba_display_modes_upcoming', True),
},
},
'ncaam': {
'enabled': basketball_config.get('ncaam_basketball_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/scoreboard',
'logo_dir': Path('assets/sports/ncaa_logos'),
'favorite_teams': basketball_config.get('ncaam_basketball_favorite_teams', []),
'display_modes': {
'ncaam_basketball_live': basketball_config.get('ncaam_basketball_display_modes_live', True),
'ncaam_basketball_recent': basketball_config.get('ncaam_basketball_display_modes_recent', True),
'ncaam_basketball_upcoming': basketball_config.get('ncaam_basketball_display_modes_upcoming', True),
},
},
'ncaaw': {
'enabled': basketball_config.get('ncaaw_basketball_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/scoreboard',
'logo_dir': Path('assets/sports/ncaa_logos'),
'favorite_teams': basketball_config.get('ncaaw_basketball_favorite_teams', []),
'display_modes': {
'ncaaw_basketball_live': basketball_config.get('ncaaw_basketball_display_modes_live', True),
'ncaaw_basketball_recent': basketball_config.get('ncaaw_basketball_display_modes_recent', True),
'ncaaw_basketball_upcoming': basketball_config.get('ncaaw_basketball_display_modes_upcoming', True),
},
},
}
def update(self) -> None:
"""Update game data for all enabled leagues."""
try:
all_games = []
for league_key, league_config in self.league_configs.items():
if not league_config['enabled']:
continue
games = self._fetch_league_games(league_key, league_config)
for game in games:
game['league_key'] = league_key
game['league_config'] = league_config
all_games.extend(games)
self.current_games = all_games
self.logger.debug(f"Updated basketball data: {len(all_games)} total games")
except Exception as e:
self.logger.error(f"Error updating basketball data: {e}", exc_info=True)
def _fetch_league_games(self, league_key: str, league_config: Dict) -> List[Dict]:
"""Fetch games for a specific league using API helper."""
try:
# Use API helper to fetch ESPN data with caching
data = self.api_helper.fetch_espn_scoreboard(
sport='basketball',
league=league_key,
cache_key=f"basketball_{league_key}",
cache_ttl=300 # 5 minutes cache
)
if not data:
return []
# Use game helper to process events
events = data.get('events', [])
games = self.game_helper.process_games(events, sport='basketball')
# Add logo paths to games
for game in games:
logo_dir = league_config['logo_dir']
game['home_logo_path'] = logo_dir / f"{game['home_abbr']}.png"
game['away_logo_path'] = logo_dir / f"{game['away_abbr']}.png"
return games
except Exception as e:
self.logger.error(f"Error fetching {league_key} games: {e}", exc_info=True)
return []
def display(self, force_clear: bool = False, display_mode: str = None) -> None:
"""Display basketball games using display helper."""
try:
mode = display_mode or self._determine_display_mode()
if not mode:
self._display_no_games()
return
# Filter games for mode
filtered_games = self._filter_games_for_mode(mode)
if not filtered_games:
self._display_no_games()
return
# Display first game
self.current_game = filtered_games[0]
self._draw_scorebug_layout(self.current_game, force_clear)
except Exception as e:
self.logger.error(f"Error displaying game: {e}", exc_info=True)
def _determine_display_mode(self) -> Optional[str]:
"""Determine display mode based on available games."""
# Priority: live > recent > upcoming
for game in self.current_games:
if game.get('is_live'):
return f"{game['league_key']}_live"
for game in self.current_games:
if game.get('is_final'):
return f"{game['league_key']}_recent"
for game in self.current_games:
if game.get('is_upcoming'):
return f"{game['league_key']}_upcoming"
return None
def _filter_games_for_mode(self, mode: str) -> List[Dict]:
"""Filter games based on display mode."""
filtered = []
for game in self.current_games:
league_config = game.get('league_config', {})
display_modes = league_config.get('display_modes', {})
if mode in display_modes and display_modes[mode]:
if 'live' in mode and game.get('is_live'):
filtered.append(game)
elif 'recent' in mode and game.get('is_final'):
filtered.append(game)
elif 'upcoming' in mode and game.get('is_upcoming'):
filtered.append(game)
return filtered[:5]
def _draw_scorebug_layout(self, game: Dict, force_clear: bool = False) -> None:
"""Draw the basketball scorebug layout using display helper."""
try:
# Load logos using logo helper
home_logo = self.logo_helper.load_logo(
game['home_abbr'],
game['home_logo_path']
)
away_logo = self.logo_helper.load_logo(
game['away_abbr'],
game['away_logo_path']
)
if not home_logo or not away_logo:
self.logger.error("Failed to load logos")
self._display_error("Logo Error")
return
# Use display helper to create scorebug layout
final_img = self.display_helper.draw_scorebug_layout(
game_data=game,
fonts=self.fonts,
home_logo=home_logo,
away_logo=away_logo
)
# Display the image
self.display_manager.image.paste(final_img, (0, 0))
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error drawing scorebug: {e}", exc_info=True)
def _display_no_games(self) -> None:
"""Display 'no games' message using display helper."""
try:
img = self.display_helper.draw_no_data_message("No Games")
self.display_manager.image = img.copy()
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error displaying no games: {e}", exc_info=True)
def _display_error(self, message: str) -> None:
"""Display error message using display helper."""
try:
img = self.display_helper.draw_error_message(message)
self.display_manager.image = img.copy()
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error displaying error message: {e}", exc_info=True)
def get_display_duration(self) -> float:
"""Get display duration."""
return self.config.get('display_duration', 15)
def cleanup(self) -> None:
"""Cleanup resources."""
self.current_games = []
self.logger.info("Basketball plugin cleaned up")
# Example usage and benefits:
"""
Benefits of using LEDMatrix Common helpers:
1. **Cleaner Code**: The plugin is much shorter and more readable
2. **Reusable Components**: Common functionality is shared across plugins
3. **Better Testing**: Each helper can be tested independently
4. **Easier Maintenance**: Bug fixes in helpers benefit all plugins
5. **Consistent Behavior**: All plugins use the same underlying logic
6. **Reduced Dependencies**: Plugins don't need to import LEDMatrix core
7. **Better Error Handling**: Centralized error handling in helpers
8. **Configuration Management**: Consistent config handling across plugins
The original basketball plugin was 326 lines. This version is much cleaner
and delegates most functionality to the common helpers, making it easier to
maintain and extend.
"""
+54
View File
@@ -146,6 +146,60 @@ def ensure_file_permissions(path: Path, mode: int = 0o644) -> None:
raise
_shared_group_gid_cache: Optional[int] = None
def get_shared_group_gid() -> Optional[int]:
"""
Return the gid that should own config/secrets files shared between the
root-run ``ledmatrix.service`` (main display) and the non-root user that
``ledmatrix-web.service`` runs as (see install_web_service.sh, which sets
``User=$SUDO_USER``).
Resolved once from the project root directory's current group (normally
the login user's group from the initial ``git clone``), since that user
is stable across reinstalls unlike any single file's ownership.
Returns:
The gid, or None if it cannot be determined.
"""
global _shared_group_gid_cache
if _shared_group_gid_cache is not None:
return _shared_group_gid_cache
try:
project_root = Path(__file__).resolve().parent.parent.parent
_shared_group_gid_cache = project_root.stat().st_gid
return _shared_group_gid_cache
except OSError:
return None
def ensure_shared_group_ownership(path: Path) -> None:
"""
Best-effort chgrp of ``path`` to the shared group (see
:func:`get_shared_group_gid`) when running as root.
Only root can change a file's group to one the calling process isn't a
member of, which is exactly the case that causes the web interface
(running as a non-root user) to get ``PermissionError`` reading files
the root-run display service just wrote with a 0o640/2775 mode: the mode
is group-readable, but without this the group is root's, not the web
user's. Silently does nothing if not running as root or on any error —
this is a hardening step, not a required one.
"""
if os.geteuid() != 0:
return
gid = get_shared_group_gid()
if gid is None:
return
try:
if path.exists() and path.stat().st_gid != gid:
os.chown(path, -1, gid)
logger.debug(f"Set shared group ownership (gid {gid}) on {path}")
except OSError as e:
logger.debug(f"Could not set shared group ownership on {path}: {e}")
def get_config_file_mode(file_path: Path) -> int:
"""
Return appropriate permission mode for config files.
+68
View File
@@ -0,0 +1,68 @@
"""Snapshot write policy for the display preview mirror.
The display service mirrors frames to /tmp/led_matrix_preview.png, which
serves two consumers with different needs:
- The web UI's live preview (SSE reader in web_interface/app.py) wants
fresh frames — but only while a browser is actually watching.
- The health check (web_interface/blueprints/api_v3.py, hardware status)
uses the file's AGE as a liveness proxy: age >= 60s reads as degraded.
PNG-encoding every frame at 5 fps forever — identical frames, no viewers —
was one of the biggest fixed CPU costs on the Pi. This module is the pure
decision logic (extracted so it's unit-testable off-Pi; display_manager
imports rgbmatrix unconditionally and can't be):
WRITE — encode + atomically replace the snapshot file
TOUCH — os.utime only: keeps the health-check mtime fresh and lets
the SSE reader (mtime-gated) resend at a low rate, without
paying for a PNG encode of an unchanged frame
SKIP — do nothing
Policy:
- With a fresh viewer marker: changed frames write at up to 1/VIEWER_INTERVAL.
- Without viewers: changed frames still write at 1/IDLE_INTERVAL so the
preview page shows something recent on open.
- Unchanged frames are never re-encoded; the mtime is touched every
TOUCH_INTERVAL so the health check (60s threshold) never degrades.
If any constant here changes, re-check the health threshold in
api_v3.py (get_hardware_status) — TOUCH_INTERVAL must stay well under it.
"""
from enum import Enum
# Snapshot cadence with a browser preview open (seconds).
VIEWER_INTERVAL = 0.2
# Snapshot cadence with no viewers — cheap freshness for page-open (seconds).
IDLE_INTERVAL = 30.0
# Max age of the last write/touch before bumping mtime for the health
# check. MUST stay well under api_v3's 60s degraded threshold.
TOUCH_INTERVAL = 20.0
# A viewer marker older than this no longer counts as a live viewer.
VIEWER_MARKER_FRESH_SEC = 5.0
class SnapshotAction(Enum):
WRITE = "write"
TOUCH = "touch"
SKIP = "skip"
def decide(now: float, last_write_ts: float, last_touch_ts: float,
viewer_fresh: bool, frame_changed: bool) -> SnapshotAction:
"""Decide what to do with the current frame.
Args:
now: current monotonic-ish timestamp (same clock as the ts args)
last_write_ts: when a frame was last actually encoded+written
last_touch_ts: when the file mtime was last bumped (write or touch)
viewer_fresh: a browser preview is currently watching
frame_changed: the frame differs from the last WRITTEN frame
"""
interval = VIEWER_INTERVAL if viewer_fresh else IDLE_INTERVAL
if frame_changed and (now - last_write_ts) >= interval:
return SnapshotAction.WRITE
if (now - max(last_write_ts, last_touch_ts)) >= TOUCH_INTERVAL:
return SnapshotAction.TOUCH
return SnapshotAction.SKIP
+71 -5
View File
@@ -38,6 +38,7 @@ from src.config_manager_atomic import (
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
ensure_shared_group_ownership,
get_config_file_mode,
get_config_dir_mode
)
@@ -56,6 +57,13 @@ class ConfigManager:
self.secrets_path: str = secrets_path or "config/config_secrets.json"
self.template_path: str = "config/config.template.json"
self.config: Dict[str, Any] = {}
# (mtime_ns, size) signature of (config, secrets, template) at the
# last successful load. load_config() skips the full re-read (3 file
# parses + recursive template migration) when nothing changed —
# ~30 web request handlers call it, some 2-3x per request. Cross-
# process freshness is preserved: another process's save bumps the
# mtime, so the next load here re-reads.
self._loaded_sig: Optional[tuple] = None
self.logger: logging.Logger = get_logger(__name__)
# Initialize atomic config manager
@@ -122,6 +130,14 @@ class ConfigManager:
# Update in-memory config if save was successful
if result.status == SaveResultStatus.SUCCESS:
self.config = new_config_data
# In-memory config now matches what was just written; refresh
# the load signature so the fast path stays valid. NOTE: the
# in-memory copy includes merged secrets; the on-disk file has
# them stripped — the fast path returning self.config preserves
# exactly the pre-cache behavior (load-after-save also returned
# the secret-merged self.config only after re-reading secrets;
# here secrets file is unchanged, so contents are equivalent).
self._loaded_sig = self._files_signature()
self.logger.info(f"Configuration successfully saved atomically to {os.path.abspath(self.config_path)}")
elif result.status == SaveResultStatus.ROLLED_BACK:
# Reload config from file after rollback
@@ -179,13 +195,36 @@ class ConfigManager:
atomic_mgr = self._get_atomic_manager()
return atomic_mgr.validate_config_file(config_path)
def _files_signature(self) -> tuple:
"""(mtime_ns, size) of config/secrets/template, None for missing —
cheap staleness probe (3 stats) for the load_config fast path."""
sig = []
for path in (self.config_path, self.secrets_path, self.template_path):
try:
st = os.stat(path)
sig.append((st.st_mtime_ns, st.st_size))
except OSError:
sig.append(None)
return tuple(sig)
def load_config(self) -> Dict[str, Any]:
"""Load configuration from JSON files."""
"""Load configuration from JSON files.
Fast path: when config.json, config_secrets.json and the template
are all unchanged since the last successful load (mtime_ns + size),
the already-parsed self.config is returned without touching the
files — same aliasing semantics as the full path, which also
returns self.config.
"""
try:
current_sig = self._files_signature()
if self.config and self._loaded_sig == current_sig:
return self.config
# Check if config file exists, if not create from template
if not os.path.exists(self.config_path):
self._create_config_from_template()
# Load main config
self.logger.info(f"Attempting to load config from: {os.path.abspath(self.config_path)}")
with open(self.config_path, 'r') as f:
@@ -196,6 +235,11 @@ class ConfigManager:
# Load and merge secrets if they exist (be permissive on errors)
if os.path.exists(self.secrets_path):
# Self-heal stale group ownership (e.g. the root-run display
# service wrote this file before the web user was granted
# group access) before every load attempt; no-op unless
# running as root and the group is already wrong.
ensure_shared_group_ownership(Path(self.secrets_path))
try:
with open(self.secrets_path, 'r') as f:
secrets = json.load(f)
@@ -205,7 +249,10 @@ class ConfigManager:
self.logger.warning(f"Secrets file not readable ({self.secrets_path}): {e}. Continuing without secrets.")
except (json.JSONDecodeError, OSError) as e:
self.logger.warning(f"Error reading secrets file ({self.secrets_path}): {e}. Continuing without secrets.")
# Signature taken AFTER load + migration (migration may write the
# config back), so it reflects exactly what was read/written.
self._loaded_sig = self._files_signature()
return self.config
except FileNotFoundError as e:
@@ -264,7 +311,8 @@ class ConfigManager:
json.dump(config_to_write, f, indent=4)
# Update the in-memory config to the new state (which includes secrets for runtime)
self.config = new_config_data
self.config = new_config_data
self._loaded_sig = self._files_signature()
self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
if secrets_content:
self.logger.info("Secret values were preserved in memory and not written to the main config file.")
@@ -321,6 +369,7 @@ class ConfigManager:
# Set proper file permissions after creation
config_path_obj = Path(self.config_path)
ensure_file_permissions(config_path_obj, get_config_file_mode(config_path_obj))
ensure_shared_group_ownership(config_path_obj)
self.logger.info(f"Created config.json from template at {os.path.abspath(self.config_path)}")
@@ -433,6 +482,11 @@ class ConfigManager:
self.logger.error(error_msg)
raise ConfigError(error_msg, config_path=path_to_load)
if file_type == "secrets":
# Best-effort self-heal: no-op unless running as root and the
# group is stale (see load_config for why this can happen).
ensure_shared_group_ownership(Path(path_to_load))
try:
with open(path_to_load, 'r') as f:
return json.load(f)
@@ -440,7 +494,18 @@ class ConfigManager:
error_msg = f"Error parsing {file_type} configuration file: {path_to_load}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
except (IOError, OSError, PermissionError) as e:
except PermissionError as e:
if file_type == "secrets":
# Match load_config()'s tolerance: a secrets file the web
# process can't read (e.g. written 0640 by the root-run
# display service before the group was fixed up) shouldn't
# 500 the settings page — degrade to "no secrets" instead.
self.logger.warning(f"Secrets file not readable ({path_to_load}): {e}. Returning empty secrets.")
return {}
error_msg = f"Error loading {file_type} configuration file {path_to_load}: {str(e)}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
except (IOError, OSError) as e:
error_msg = f"Error loading {file_type} configuration file {path_to_load}: {str(e)}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
@@ -497,6 +562,7 @@ class ConfigManager:
# Ensure final file has correct permissions
try:
ensure_file_permissions(path_obj, file_mode)
ensure_shared_group_ownership(path_obj)
except OSError as perm_error:
# If we can't set permissions but file was written, log warning but don't fail
self.logger.warning(
+8
View File
@@ -17,6 +17,7 @@ from enum import Enum
from src.exceptions import ConfigError
from src.logging_config import get_logger
from src.common.permission_utils import ensure_shared_group_ownership
class SaveResultStatus(Enum):
@@ -410,6 +411,13 @@ class AtomicConfigManager:
# This is important because temp files may have different permissions
# and we need root service to be able to read config.json
os.chmod(destination, target_mode)
# Also fix group ownership when this save is running as root
# (the display service): 0o640 alone only helps the non-root web
# user read a root-written secrets file if its group already
# matches the web user's group, which isn't guaranteed. See
# permission_utils.ensure_shared_group_ownership for why.
ensure_shared_group_ownership(destination)
except Exception as e:
raise ConfigError(f"Error during atomic move: {e}") from e
+251 -44
View File
@@ -23,6 +23,9 @@ Entry point: :func:`main` — instantiates :class:`DisplayController` and calls
import time
import os
import json
import threading
import types
from contextlib import contextmanager
from pathlib import Path
from typing import Dict, Any, List, Optional, Callable
from datetime import datetime
@@ -199,6 +202,10 @@ class DisplayController:
self.wifi_status_file = WIFI_STATUS_FILE
self.wifi_status_active = False
self.wifi_status_expires_at: Optional[float] = None
# _check_wifi_status_message throttle state (checked at frame rate,
# stat'd at most once per second)
self._wifi_status_check_ts = 0.0
self._wifi_status_last_result: Optional[Dict[str, Any]] = None
# Plugin display() signature cache — must be initialised before the plugin
# loading loop below so the .pop() invalidation at load time is always safe.
@@ -502,7 +509,10 @@ class DisplayController:
# Run plugin updates inside the Vegas loop so the inter-iteration
# gap is <1 ms (nothing left for _tick_plugin_updates() to do).
self.vegas_coordinator.set_update_callback(self._tick_plugin_updates)
# Use the Vegas-aware variant so plugins that got fresh data are
# hot-swapped into the scroll promptly instead of waiting for the
# next full cycle.
self.vegas_coordinator.set_update_callback(self._tick_plugin_updates_for_vegas)
# Wire multi-display sync into Vegas render pipeline
follower_pos = self.config.get("sync", {}).get("follower_position", "left")
@@ -621,18 +631,28 @@ class DisplayController:
current_day = current_time.strftime('%A').lower() # e.g. 'monday'
current_time_only = current_time.time()
# Check if per-day schedule is configured
days_config = schedule_config.get('days')
# Determine which schedule to use
# Determine which schedule to use. Respect an explicit 'mode' field
# (like the dim schedule does) so a stray/legacy 'days' dict left over
# from config migration or a prior per-day setup can't silently
# override a user's Global schedule selection.
mode = schedule_config.get('mode')
mode_normalized = mode.replace('_', '-') if mode else None
use_per_day = False
if days_config:
# Check if days dict is not empty and contains current day
if days_config and current_day in days_config:
if mode_normalized == 'global':
use_per_day = False
elif mode_normalized == 'per-day':
use_per_day = bool(days_config and current_day in days_config)
elif days_config:
# No explicit mode recorded (legacy config) - fall back to
# inferring from presence of a 'days' dict for the current day.
if current_day in days_config:
use_per_day = True
elif days_config:
# Days dict exists but doesn't have current day - fall back to global
else:
logger.debug("Per-day schedule exists but %s not configured, using global schedule", current_day)
if use_per_day:
@@ -828,6 +848,42 @@ class DisplayController:
if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_failure(plugin_id, exc)
def _tick_plugin_updates_for_vegas(self) -> None:
"""Run scheduled plugin updates and tell Vegas mode which plugins
actually got fresh data, so it can hot-swap them into the scroll
without waiting for a full cycle to complete.
Used as the Vegas coordinator's update callback instead of the plain
_tick_plugin_updates() so that a live score change is reflected in
the ticker within a few seconds rather than at the next cycle
boundary (which, depending on min/max_cycle_duration, can be
minutes away). Restores wiring that PR #299 added and PR #330's
sync-mode refactor inadvertently dropped: coordinator.mark_plugin_updated()
has been unreachable dead code since.
Delegates the before/after plugin_last_update snapshot to
PluginManager.run_scheduled_updates_with_changes() so the snapshot,
update pass, and diff are lock-protected against this callback's own
background update-tick thread racing the main render loop.
"""
if not self.plugin_manager or not hasattr(self.plugin_manager, "run_scheduled_updates_with_changes"):
self._tick_plugin_updates()
return
updated = self.plugin_manager.run_scheduled_updates_with_changes()
vc = getattr(self, "vegas_coordinator", None)
if vc is None:
return
if updated:
logger.info("Vegas update tick: %d plugin(s) updated: %s", len(updated), updated)
for plugin_id in updated:
try:
vc.mark_plugin_updated(plugin_id)
except Exception: # pylint: disable=broad-except
logger.exception("Error marking plugin %s updated for Vegas", plugin_id)
def _tick_plugin_updates(self):
"""Run scheduled plugin updates if the plugin manager supports them."""
if not self.plugin_manager:
@@ -839,6 +895,30 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error running scheduled plugin updates")
@contextmanager
def _display_lock_or_skip(self, plugin_id):
"""Try-lock guard keeping a plugin's display() off its in-flight update().
Yields True when display may run (lock held, released on exit) or
when no lock support exists (older plugin manager). Yields False when
the plugin's update() is currently executing on the background
worker — the caller should treat the frame as displayed (the panel
holds the last pushed frame) rather than as a plugin failure, so a
mid-update skip never advances the rotation.
"""
pm = self.plugin_manager
if not pm or not hasattr(pm, 'get_plugin_lock') or not plugin_id:
yield True
return
lock = pm.get_plugin_lock(plugin_id)
if not lock.acquire(blocking=False):
yield False
return
try:
yield True
finally:
lock.release()
_FOLLOWER_SEND_INTERVAL = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
def _follower_rebuild_scroll_image(self) -> None:
@@ -1053,6 +1133,29 @@ class DisplayController:
remaining = self.on_demand_expires_at - time.time()
return max(0.0, remaining)
def _publish_current_mode_state(self) -> None:
"""Publish the currently active display mode/plugin to cache for the web UI."""
try:
state = {
'mode': self.current_display_mode,
'plugin_id': self.mode_to_plugin_id.get(self.current_display_mode),
'mode_index': self.current_mode_index,
'total_modes': len(self.available_modes),
'on_demand_active': self.on_demand_active,
'is_display_active': self.is_display_active,
'last_updated': time.time(),
}
self.cache_manager.set('display_current_state', state)
self._last_published_mode = self.current_display_mode
except (OSError, RuntimeError, ValueError, TypeError) as err:
logger.error("Failed to publish current display state: %s", err, exc_info=True)
def _publish_current_mode_state_if_changed(self) -> None:
"""Publish current mode state only when it actually changed, to avoid
writing to the shared cache on every render tick."""
if self.current_display_mode != getattr(self, '_last_published_mode', None):
self._publish_current_mode_state()
def _publish_on_demand_state(self) -> None:
"""Publish current on-demand state to cache for external consumers."""
try:
@@ -1572,6 +1675,7 @@ class DisplayController:
logger.info("Starting display with cached data (fast startup mode)")
self.current_display_mode = self.available_modes[self.current_mode_index] if self.available_modes else 'none'
logger.info(f"Initial mode set to: {self.current_display_mode} (index: {self.current_mode_index}, total modes: {len(self.available_modes)})")
self._publish_current_mode_state()
while True:
# Apply plugin enable/disable edits saved via the web UI. The
@@ -1632,10 +1736,12 @@ class DisplayController:
logger.debug(f"Error clearing display when inactive: {e}")
logger.info(f"Display not active (is_display_active={self.is_display_active}), sleeping...")
self._publish_current_mode_state()
self._sleep_with_plugin_updates(60)
continue
logger.info(f"Display active, processing mode: {self.current_display_mode}")
self._publish_current_mode_state_if_changed()
logger.debug("Display active, processing mode: %s", self.current_display_mode)
# Plugins update on their own schedules - no forced sync updates needed
# Each plugin has its own update_interval and background services
@@ -1803,7 +1909,7 @@ class DisplayController:
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
should_skip = self.plugin_manager.health_tracker.should_skip_plugin(plugin_id)
if should_skip:
logger.info(f"Skipping plugin {plugin_id} due to circuit breaker (mode: {active_mode})")
logger.info("Skipping plugin %s due to circuit breaker (mode: %s)", plugin_id, active_mode)
display_result = False
# Skip to next mode - let existing logic handle it
manager_to_display = None
@@ -1826,6 +1932,7 @@ class DisplayController:
plugin_id = getattr(manager_to_display, 'plugin_id', active_mode)
try:
logger.debug(f"Calling display() for {active_mode} with force_clear={self.force_change}")
can_display = False
if hasattr(manager_to_display, 'display'):
# Opt #1: look up (or compute once) whether display() accepts display_mode
_cache_key = plugin_id
@@ -1836,14 +1943,64 @@ class DisplayController:
)
_accepts_display_mode = self._plugin_accepts_display_mode[_cache_key]
# Use PluginExecutor for safe execution with timeout
if self.plugin_manager and hasattr(self.plugin_manager, 'plugin_executor'):
result = self.plugin_manager.plugin_executor.execute_display(
manager_to_display,
plugin_id,
force_clear=self.force_change,
display_mode=active_mode if _accepts_display_mode else None
)
pm = self.plugin_manager
display_lock = None
can_display = True
if pm and hasattr(pm, 'get_plugin_lock'):
display_lock = pm.get_plugin_lock(plugin_id)
can_display = display_lock.acquire(blocking=False)
if not can_display:
# update() in flight on the worker — hold
# the last frame; not a plugin failure
result = True
elif pm and hasattr(pm, 'plugin_executor'):
# PluginExecutor's own thread.join(timeout) can
# return before the real display() call
# finishes (a lingering daemon thread keeps
# running it) -- so the lock is released from
# inside the wrapped call itself, whichever
# thread actually finishes it, rather than
# here when this dispatch merely returns.
release_guard = threading.Lock()
released = {'done': False}
def _release_display_lock():
with release_guard:
if released['done']:
return
released['done'] = True
if display_lock is not None:
display_lock.release()
if _accepts_display_mode:
def _display_target(display_mode=None, force_clear=False):
try:
return manager_to_display.display(
display_mode=display_mode, force_clear=force_clear)
finally:
_release_display_lock()
else:
def _display_target(force_clear=False):
try:
return manager_to_display.display(force_clear=force_clear)
finally:
_release_display_lock()
try:
result = self.plugin_manager.plugin_executor.execute_display(
types.SimpleNamespace(display=_display_target),
plugin_id,
force_clear=self.force_change,
display_mode=active_mode if _accepts_display_mode else None
)
except Exception: # pragma: no cover - defensive;
# execute_display catches everything
# internally, but guarantee the lock is
# never leaked if something unexpected
# slips through.
_release_display_lock()
raise
# execute_display returns bool, convert to expected format
if result:
result = True # Success
@@ -1851,23 +2008,31 @@ class DisplayController:
result = False # Failed
else:
# Fallback to direct call if executor not available
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=self.force_change)
else:
result = manager_to_display.display(force_clear=self.force_change)
try:
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=self.force_change)
else:
result = manager_to_display.display(force_clear=self.force_change)
finally:
if display_lock is not None:
display_lock.release()
logger.debug(f"display() returned: {result} (type: {type(result)})")
# Check if display() returned a boolean (new behavior)
if isinstance(result, bool):
display_result = result
if not display_result:
logger.info(f"Plugin {plugin_id} display() returned False for mode {active_mode}")
logger.info("Plugin %s display() returned False for mode %s", plugin_id, active_mode)
# Record success if display completed without exception
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_success(plugin_id)
self.force_change = False
# Record success only when display() actually ran this
# frame -- a skipped frame (lock busy) held the last
# frame, not a real success, and must not clear
# force_change or the pending mode-switch clear will
# be lost when display() finally does run.
if can_display:
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_success(plugin_id)
self.force_change = False
except Exception as exc: # pylint: disable=broad-except
logger.exception("Error displaying %s", self.current_display_mode)
# Record failure
@@ -2072,10 +2237,23 @@ class DisplayController:
# For plugins, call display multiple times to allow game rotation
if manager_to_display and hasattr(manager_to_display, 'display'):
# Check if plugin needs high FPS (like stock ticker)
# Always enable high-FPS for static-image plugin (for GIF animation support)
# High-FPS decision, in precedence order:
# 1. A plugin that declares needs_high_fps knows best
# (e.g. static-image sets it False for still PNGs,
# True for animated GIFs).
# 2. Back-compat: older static-image versions without
# the attribute keep the historical forced high-FPS
# (GIF support).
# 3. Otherwise scrolling plugins get high FPS.
plugin_id = getattr(manager_to_display, 'plugin_id', None)
if plugin_id == 'static-image':
declared = getattr(manager_to_display, 'needs_high_fps', None)
if declared is not None:
needs_high_fps = bool(declared)
logger.debug(
"[DisplayController] FPS check for %s (plugin=%s) - "
"plugin declares needs_high_fps=%s",
active_mode, plugin_id, needs_high_fps)
elif plugin_id == 'static-image':
needs_high_fps = True
logger.debug("FPS check - static-image plugin: forcing high-FPS mode for GIF support")
else:
@@ -2139,11 +2317,16 @@ class DisplayController:
while True:
try:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
with self._display_lock_or_skip(plugin_id) as can_display:
if can_display:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
else:
# update() in flight — hold the last frame
result = True
if isinstance(result, bool) and not result:
logger.debug("Display returned False, breaking early")
break
@@ -2203,11 +2386,16 @@ class DisplayController:
break
try:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
with self._display_lock_or_skip(plugin_id) as can_display:
if can_display:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
else:
# update() in flight — hold the last frame
result = True
if isinstance(result, bool) and not result:
# For dynamic duration plugins, don't exit on False - keep looping
# until cycle is complete or max duration is reached
@@ -2354,6 +2542,16 @@ class DisplayController:
Returns None on any error or if message is expired/invalid.
"""
try:
# Throttle the existence stat to ~1 Hz: this runs on every render
# iteration (60+ fps), and the file usually doesn't exist — the
# status message's lifetime is measured in seconds anyway.
# Both attributes are initialised in __init__.
now = time.time()
if (now - self._wifi_status_check_ts) < 1.0:
return self._wifi_status_last_result
self._wifi_status_check_ts = now
self._wifi_status_last_result = None
# Check if file exists
if not self.wifi_status_file or not self.wifi_status_file.exists():
return None
@@ -2404,13 +2602,14 @@ class DisplayController:
pass
return None
# Message is valid and not expired
return {
# Message is valid and not expired — cache for the throttle window
self._wifi_status_last_result = {
'message': message,
'timestamp': timestamp,
'duration': duration,
'expires_at': expires_at
}
return self._wifi_status_last_result
except Exception as e:
# Catch-all for any unexpected errors - log but don't break the display
@@ -2714,6 +2913,14 @@ class DisplayController:
def cleanup(self):
"""Clean up resources."""
# Stop the async update worker first so no in-flight update() call
# is still touching display/cache-backed resources while they're
# torn down below.
if self.plugin_manager and hasattr(self.plugin_manager, 'stop_update_worker'):
try:
self.plugin_manager.stop_update_worker()
except Exception as e:
logger.warning("Error stopping plugin update worker: %s", e)
# Shutdown config service if it exists
if hasattr(self, 'config_service'):
try:
+184 -51
View File
@@ -31,13 +31,25 @@ if os.getenv("EMULATOR", "false") == "true":
else:
from rgbmatrix import RGBMatrix, RGBMatrixOptions
from contextlib import contextmanager
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
import threading
import time
from typing import Dict, Any, List, Optional
from collections import OrderedDict
from typing import Dict, Any, List, Optional, Tuple
import logging
import math
import zlib
import freetype
from src.common import snapshot_policy
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
get_assets_dir_mode,
get_assets_file_mode,
)
# Get logger without configuring
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO) # Set to INFO level
@@ -180,14 +192,45 @@ class DisplayManager:
# the logical image is blitted to the matrix unchanged.
self._double_sided = None # dict {copies, axis, logical_width, logical_height} or None
self._physical_image = None # full-chain buffer reused each frame when tiling
# Text-width measurement cache: (text, id(font)) -> pixel_width
# Text-width measurement cache: (text, id(font)) -> (width, font_ref)
# Avoids re-measuring the same string+font on every display() call.
# LRU-bounded: keys embed the TEXT, so changing strings (a clock, a
# live score) would otherwise grow it forever on a 24/7 service.
# Entries hold a strong reference to the font so its id() can't be
# recycled by a different font object — an id-keyed cache without
# the reference can return the WRONG width after garbage collection.
# Cleared on _load_fonts() so stale entries don't survive a font reload.
self._text_width_cache: Dict[tuple, int] = {}
# Snapshot settings for web preview integration (service writes, web reads)
self._text_width_cache: "OrderedDict[tuple, Tuple[int, Any]]" = OrderedDict()
self._TEXT_WIDTH_CACHE_MAX = 1024
# Snapshot mirror for web preview + health check (service writes, web
# reads). Cadence/skip decisions live in src/common/snapshot_policy.py:
# full rate only while the web SSE broadcaster keeps the viewer marker
# fresh; unchanged frames are never re-encoded, only mtime-touched.
self._snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path intentional; web UI reads same path
self._snapshot_min_interval_sec = 0.2 # max ~5 fps
self._viewer_marker_path = "/tmp/led_matrix_preview_viewer" # nosec B108 - touched by web SSE broadcaster
self._last_snapshot_ts = 0.0
self._last_snapshot_touch_ts = 0.0
self._last_snapshot_digest: Optional[int] = None
self._snapshot_dir_prepared = False
self._viewer_check_ts = 0.0
self._viewer_fresh = False
self._viewer_was_fresh = False
# Snapshot failures are logged as warnings, rate-limited so a
# persistent failure (e.g. an unwritable file) can't spam the log —
# but is never silent: the snapshot's mtime doubles as the web UI's
# hardware-liveness signal, so a quiet failure makes health checks lie.
self._snapshot_fail_log_ts = 0.0
# Dirty tracking: (image digest, brightness) of the last frame pushed
# to the panel; update_display() skips identical pushes. Kill switch:
# display.dirty_tracking: false.
self._dirty_tracking_enabled = bool(
self.config.get('display', {}).get('dirty_tracking', True))
self._last_pushed_digest = None
# Serializes update_display(): plugins can call it directly from
# background threads (see docstring on update_display), not just the
# render loop. RLock in case a caller within the critical section
# ever re-enters (e.g. via a nested draw callback).
self._update_lock = threading.RLock()
# Scrolling state tracking for graceful updates
self._scrolling_state = {
@@ -418,6 +461,10 @@ class DisplayManager:
try:
# RGBMatrix accepts brightness as a property
self.matrix.brightness = brightness
# Brightness applies on the next swap — force a re-push even if
# the image itself is unchanged (belt-and-braces: brightness is
# also part of the dirty-tracking digest when readable).
self._last_pushed_digest = None
logger.info(f"[BRIGHTNESS] Display brightness set to {brightness}%")
return True
except AttributeError as e:
@@ -509,33 +556,70 @@ class DisplayManager:
return phys
def update_display(self):
"""Update the display using double buffering with proper sync."""
"""Update the display using double buffering with proper sync.
Skips the panel push entirely when the frame is byte-identical to
the last pushed one (same image digest AND same brightness) static
content re-rendered every second, and 125 fps loops between actual
scroll steps, otherwise re-walk the full framebuffer for nothing.
The panel keeps refreshing the current frame from its own thread,
so skipping a swap never blanks or freezes the hardware.
Correctness hinges on invalidation: clear() resets the digest (it
writes to the matrix directly), and brightness is PART of the digest
so a dim-schedule change is never skipped. Disable via config
``display.dirty_tracking: false`` if a redraw issue is ever suspected.
Serialized via ``_update_lock``: plugins can call this directly from
background threads (e.g. sports base classes push an immediate
"live" refresh from inside update()), so without a lock two callers
could both pass the digest check before either writes it back,
double-pushing a frame, or interleave the offscreen/current canvas
swap below. The lock is scoped to this method, so callers never
need to know about it.
"""
try:
if self.matrix is None:
# Fallback mode - no actual hardware to update
logger.debug("Update display called in fallback mode (no hardware)")
# Still write a snapshot so the web UI can preview
with self._update_lock:
if self.matrix is None:
# Fallback mode - no actual hardware to update
logger.debug("Update display called in fallback mode (no hardware)")
# Still write a snapshot so the web UI can preview
self._write_snapshot_if_due()
return
if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
digest = None
if self._dirty_tracking_enabled:
try:
brightness = getattr(self.matrix, 'brightness', None)
except AttributeError:
brightness = None
digest = (zlib.adler32(self.image.tobytes()), brightness)
if digest == self._last_pushed_digest:
# Nothing changed since the last push — the panel is
# already showing exactly this frame.
self._write_snapshot_if_due()
return
# Copy the current image to the offscreen canvas. In double-sided
# mode the logical screen is first tiled across the full chain.
if self._double_sided is not None:
self.offscreen_canvas.SetImage(self._composite_double_sided())
else:
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
# Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
self._last_pushed_digest = digest
# Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due()
return
if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
# Copy the current image to the offscreen canvas. In double-sided
# mode the logical screen is first tiled across the full chain.
if self._double_sided is not None:
self.offscreen_canvas.SetImage(self._composite_double_sided())
else:
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
# Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
# Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due()
except Exception as e:
logger.error(f"Error updating display: {e}")
@@ -569,6 +653,9 @@ class DisplayManager:
# Clear both canvases and the underlying matrix to ensure no artifacts.
# Failures are non-fatal — the image buffer is already black above, so
# the next update_display() call will push clean content regardless.
# The matrix content no longer matches the last pushed digest,
# so dirty tracking must not skip the next push.
self._last_pushed_digest = None
try:
self.offscreen_canvas.Clear()
except (RuntimeError, OSError) as e:
@@ -699,12 +786,15 @@ class DisplayManager:
Results are cached by (text, font identity) so plugins that measure
the same string every frame (e.g. to centre a score) pay only one
measurement per unique (text, font) pair.
measurement per unique (text, font) pair. The entry keeps the font
alive so its id() can't be recycled, and the cache is LRU-bounded so
ever-changing text (clocks, tickers) can't grow it without limit.
"""
cache_key = (text, id(font))
cached = self._text_width_cache.get(cache_key)
if cached is not None:
return cached
self._text_width_cache.move_to_end(cache_key)
return cached[0]
try:
if isinstance(font, freetype.Face):
@@ -719,7 +809,9 @@ class DisplayManager:
logger.error("Error getting text width: %s", e)
return 0
self._text_width_cache[cache_key] = width
self._text_width_cache[cache_key] = (width, font)
while len(self._text_width_cache) > self._TEXT_WIDTH_CACHE_MAX:
self._text_width_cache.popitem(last=False)
return width
def get_font_height(self, font):
@@ -1128,27 +1220,56 @@ class DisplayManager:
'deferred_update_ttl': self._scrolling_state['deferred_update_ttl']
}
def _viewer_is_fresh(self, now: float) -> bool:
"""True when a browser preview is watching (marker file touched by
the web SSE broadcaster). The marker is stat'd at most once per
second at 125 fps loops a per-call stat would be pure overhead."""
if (now - self._viewer_check_ts) >= 1.0:
self._viewer_check_ts = now
try:
marker_age = now - os.stat(self._viewer_marker_path).st_mtime
self._viewer_fresh = marker_age < snapshot_policy.VIEWER_MARKER_FRESH_SEC
except OSError:
self._viewer_fresh = False
return self._viewer_fresh
def _write_snapshot_if_due(self) -> None:
"""Write the current image to a PNG snapshot file at a limited frequency."""
"""Mirror the current frame to the preview snapshot when the policy
says it's worth it — see src/common/snapshot_policy.py. Unchanged
frames are never re-encoded; without viewers the cadence drops to
the idle keepalive."""
try:
now = time.time()
if (now - self._last_snapshot_ts) < self._snapshot_min_interval_sec:
viewer_fresh = self._viewer_is_fresh(now)
if viewer_fresh and not self._viewer_was_fresh:
# A preview just opened: let the next changed frame through
# immediately instead of waiting out the idle interval.
self._last_snapshot_ts = 0.0
self._viewer_was_fresh = viewer_fresh
digest = zlib.adler32(self.image.tobytes())
action = snapshot_policy.decide(
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, digest != self._last_snapshot_digest)
if action is snapshot_policy.SnapshotAction.SKIP:
return
# Ensure directory exists with proper permissions
from pathlib import Path
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
get_assets_dir_mode,
get_assets_file_mode
)
if action is snapshot_policy.SnapshotAction.TOUCH:
# mtime bump only: keeps the health check (snapshot age)
# green without paying for a PNG encode of an unchanged frame
os.utime(self._snapshot_path, None)
self._last_snapshot_touch_ts = now
return
# WRITE: ensure directory permissions once, not per frame
snapshot_path_obj = Path(self._snapshot_path)
# Only ensure permissions on non-system directories
# Never modify /tmp permissions - it has special system permissions (1777)
# that must not be changed or it breaks apt and other system tools
parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
if not self._snapshot_dir_prepared:
# Never modify /tmp permissions - it has special system
# permissions (1777) that must not be changed or it breaks
# apt and other system tools
parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
self._snapshot_dir_prepared = True
# Write atomically: temp then replace
tmp_path = f"{self._snapshot_path}.tmp"
self.image.save(tmp_path, format='PNG')
@@ -1163,6 +1284,18 @@ class DisplayManager:
except Exception:
pass
self._last_snapshot_ts = now
self._last_snapshot_touch_ts = now
self._last_snapshot_digest = digest
except Exception as e:
# Snapshot failures should never break display; log at debug to avoid noise
logger.debug(f"Snapshot write skipped: {e}")
# Snapshot failures must never break display — but they must not
# be silent either: the snapshot's mtime is the web UI's display
# mirror AND its hardware-liveness proxy, so a quietly failing
# write freezes the mirror and makes health checks lie (seen in
# the field: a stale root-owned /tmp file froze it for a day).
# Warn at most once per 5 minutes to avoid log spam.
if (now - self._snapshot_fail_log_ts) > 300:
self._snapshot_fail_log_ts = now
logger.warning("Snapshot write failing (web preview/health "
"mirror is stale): %s", e)
else:
logger.debug(f"Snapshot write skipped: {e}")
-552
View File
@@ -1,552 +0,0 @@
"""Universal per-element style resolution for plugin customization.
Plugins expose per-element user customization under ``config['customization']``:
"customization": {
"score_text": {"font": "PressStart2P-Regular.ttf", "font_size": 10,
"text_color": [255, 255, 255]},
"layout": {"score": {"x_offset": 2, "y_offset": 0}}
}
Before this module, every plugin re-implemented the same three pieces
a font loader, an x/y-offset reader, and (for adaptive layout mode) a
"did the user actually override this?" check. The loaders diverged four
ways across the sports plugins and music, the offset reader was copied
twice, and the override check is subtle enough that it shipped broken
twice: the web UI's save flow (schema_manager.merge_with_defaults) writes
the FULL schema default object into config.json on every save, and the
plugin manager merges defaults into ``config`` again before instantiation,
so a key being *present* never means the user set it. The only correct
test is "present AND different from the schema default", which requires
knowing the schema defaults previously a hand-maintained dict per plugin.
This module is that logic, once:
resolver = ElementStyleResolver(config, schema_defaults)
style = resolver.style('score_text', classic_font='PressStart2P-Regular.ttf',
classic_size=10)
style.font # loaded PIL font, ready for draw.text
style.user_forced # True only for a genuine user override
dx, dy = resolver.offset('score')
``BasePlugin.element_style()`` wires this up automatically (schema defaults
come from the plugin's own config_schema.json via the schema manager).
Standalone helper classes (e.g. a plugin's GameRenderer) should receive a
resolver from their owning plugin rather than build one themselves.
Deliberately pure PIL + stdlib: no imports from the plugin system or web
layer, so it is usable from any renderer and trivially testable.
"""
import json
import logging
import os
from dataclasses import dataclass
from typing import Any, Dict, Optional, Tuple, Union
from PIL import ImageFont
logger = logging.getLogger(__name__)
# Font-family aliases accepted in customization configs. Filenames pass
# through unchanged. (Supersedes the per-plugin copies in the baseball
# plugin; keep names in sync with the web UI's /fonts/catalog so the
# font-selector widget and this loader agree.)
FONT_ALIASES: Dict[str, str] = {
"press_start": "PressStart2P-Regular.ttf",
"four_by_six": "4x6-font.ttf",
"five_by_seven": "5x7.bdf",
}
DEFAULT_FONTS_DIR = os.path.join("assets", "fonts")
DEFAULT_FALLBACK_FONT = "PressStart2P-Regular.ttf"
PILFont = Union[ImageFont.FreeTypeFont, ImageFont.ImageFont]
def resolve_font_name(font_name: str) -> str:
"""Resolve a font family alias to its filename, leaving filenames as-is."""
return FONT_ALIASES.get(font_name, font_name)
def extract_schema_defaults(schema: Dict[str, Any]) -> Dict[str, Any]:
"""Nested defaults dict from a JSON Schema (mirrors
SchemaManager.extract_defaults_from_schema, kept here so this module
stays importable without the plugin system the parity test in
test_element_style.py guards against drift).
An object property carrying its own ``default`` short-circuits recursion,
and array-typed properties without one default to ``[]`` (or a
single-item list when the items schema declares a default), matching
the schema manager's behavior exactly.
"""
defaults: Dict[str, Any] = {}
for key, prop in (schema.get("properties") or {}).items():
if not isinstance(prop, dict):
continue
if "default" in prop:
defaults[key] = prop["default"]
elif prop.get("type") == "object" and "properties" in prop:
nested = extract_schema_defaults(prop)
if nested:
defaults[key] = nested
elif prop.get("type") == "array" and "items" in prop:
items = prop.get("items")
if isinstance(items, dict) and "default" in items and \
not (items.get("type") == "object" and "properties" in items):
defaults[key] = [items["default"]]
else:
defaults[key] = []
return defaults
def defaults_from_schema_file(schema_path: str) -> Dict[str, Any]:
"""Schema defaults straight from a plugin's own config_schema.json.
Plugins that hand a resolver to standalone helper classes should build
it with this, pointed at their own schema file it works identically
in production, the test harness, and the dev server, unlike the plugin
manager's schema manager (absent under mocks). x-style-elements
declarations are expanded first, so declared elements' defaults are
included exactly as the web UI's schema manager sees them. Returns {}
on any error.
"""
try:
with open(schema_path, "r", encoding="utf-8") as f:
schema = json.load(f)
if isinstance(schema, dict):
return extract_schema_defaults(expand_style_elements(schema))
except Exception as e:
logger.debug("Could not load schema defaults from %s: %s",
schema_path, e)
return {}
# ---------------------------------------------------------------------------
# x-style-elements schema expansion
# ---------------------------------------------------------------------------
#
# A plugin declares its styleable display elements ONCE, compactly, on its
# customization object instead of hand-copying ~50-line property blocks:
#
# "customization": {
# "type": "object",
# "x-style-elements": {
# "score_text": {
# "title": "Game Score",
# "font": {"default": "PressStart2P-Regular.ttf"},
# "size": {"default": 10, "min": 4, "max": 16},
# "color": true, # or {"default": [r,g,b]}
# "offsets": true
# }
# }
# }
#
# expand_style_elements() turns each declaration into full font/font_size/
# text_color/layout-offset property blocks (marked "x-style-managed": true)
# using widgets the web config form already renders. The declaration stays
# in the schema — it doubles as the element registry for tooling. Expansion
# is idempotent, and a hand-written property block for the same element
# always wins over the generated one.
#
# SchemaManager.load_schema() applies this at serve time (so the web form,
# save path, validation, and defaults generation all see the expanded
# shape), and defaults_from_schema_file() applies it when plugins read
# their own schema — one implementation, no drift.
def get_style_elements(schema: Dict[str, Any]) -> Dict[str, Any]:
"""The x-style-elements declaration from a schema ({} if none)."""
try:
decl = schema.get("properties", {}).get("customization", {}).get("x-style-elements")
return decl if isinstance(decl, dict) else {}
except AttributeError:
return {}
def expand_style_elements(schema: Dict[str, Any]) -> Dict[str, Any]:
"""Expand x-style-elements into full customization property blocks.
Returns the schema unchanged (same object) when there is nothing to
expand; otherwise returns an expanded DEEP COPY, leaving the input
untouched. Never raises on any error the original schema is returned
so a malformed declaration can't take a plugin down.
"""
import copy
try:
declarations = get_style_elements(schema)
if not declarations:
return schema
schema = copy.deepcopy(schema)
customization = schema["properties"]["customization"]
properties = customization.setdefault("properties", {})
order = customization.get("x-propertyOrder")
offset_elements = []
for element_key, declaration in declarations.items():
if not isinstance(declaration, dict):
continue
if declaration.get("offsets") is True:
offset_elements.append((element_key, declaration))
if element_key in properties:
# Hand-written (or previously expanded) block wins.
continue
properties[element_key] = _style_element_block(element_key, declaration)
if isinstance(order, list) and element_key not in order:
# Keep generated elements ahead of the layout section.
insert_at = order.index("layout") if "layout" in order else len(order)
order.insert(insert_at, element_key)
if offset_elements:
_expand_offset_blocks(properties, order, offset_elements)
return schema
except Exception as e:
logger.error("x-style-elements expansion failed: %s", e)
return schema
def _style_element_block(element_key: str, declaration: Dict[str, Any]) -> Dict[str, Any]:
"""One generated customization.<element> property block."""
title = declaration.get("title") or element_key.replace("_", " ").title()
font_decl = declaration.get("font") if isinstance(declaration.get("font"), dict) else {}
size_decl = declaration.get("size") if isinstance(declaration.get("size"), dict) else {}
block_properties: Dict[str, Any] = {
"font": {
"type": "string",
"title": "Font Family",
"description": "Select the font to use",
"x-widget": "font-selector",
"default": font_decl.get("default", DEFAULT_FALLBACK_FONT),
},
"font_size": {
"type": "integer",
"title": "Font Size",
"description": ("Font size in pixels (BDF fonts are fixed-size "
"and ignore this)"),
"minimum": size_decl.get("min", 4),
"maximum": size_decl.get("max", 32),
"default": size_decl.get("default", 8),
},
}
block_order = ["font", "font_size"]
color_decl = declaration.get("color")
if color_decl:
default_color = [255, 255, 255]
if isinstance(color_decl, dict) and isinstance(color_decl.get("default"), list):
default_color = color_decl["default"]
# The default doubles as the "untouched" sentinel: the resolver only
# honors a color that DIFFERS from it, so untouched saves (the web
# form always posts the RGB inputs) can't clobber a plugin's
# semantic/state-dependent colors.
block_properties["text_color"] = {
"type": "array",
"title": "Text Color",
"description": "RGB color as [red, green, blue] (0-255 each)",
"items": {"type": "integer", "minimum": 0, "maximum": 255},
"minItems": 3,
"maxItems": 3,
"x-widget": "color-picker",
"default": default_color,
}
block_order.append("text_color")
return {
"type": "object",
"title": title,
"description": f"Style settings for {title}",
"x-style-managed": True,
"properties": block_properties,
"x-propertyOrder": block_order,
"additionalProperties": False,
}
def _expand_offset_blocks(properties: Dict[str, Any], order,
offset_elements) -> None:
"""Generate customization.layout.<element> x/y offset blocks."""
layout = properties.get("layout")
if not isinstance(layout, dict):
layout = {
"type": "object",
"title": "Layout Positioning",
"description": ("Adjust X,Y coordinate offsets for elements. "
"Values are relative to default positions; "
"negative moves left/up, positive right/down."),
"x-style-managed": True,
"properties": {},
"additionalProperties": False,
}
properties["layout"] = layout
if isinstance(order, list) and "layout" not in order:
order.append("layout")
layout_properties = layout.setdefault("properties", {})
layout_order = layout.get("x-propertyOrder")
for element_key, declaration in offset_elements:
if element_key in layout_properties:
continue # hand-written layout entry wins
title = declaration.get("title") or element_key.replace("_", " ").title()
layout_properties[element_key] = {
"type": "object",
"title": title,
"x-style-managed": True,
"properties": {
"x_offset": {
"type": "integer",
"title": "X Offset",
"description": "Horizontal offset in pixels (default: 0)",
"default": 0,
},
"y_offset": {
"type": "integer",
"title": "Y Offset",
"description": "Vertical offset in pixels (default: 0)",
"default": 0,
},
},
"additionalProperties": False,
}
if isinstance(layout_order, list) and element_key not in layout_order:
layout_order.append(element_key)
def load_font(font_name: str, size: int, *,
fonts_dir: str = DEFAULT_FONTS_DIR,
fallback_font: str = DEFAULT_FALLBACK_FONT) -> PILFont:
"""Load a font by name at a pixel size, never raising.
Resolution order:
1. alias -> filename (``FONT_ALIASES``)
2. ``ImageFont.truetype`` handles .ttf/.otf, and .bdf too (FreeType
loads BDF strikes at their native size; a non-native size raises
"invalid pixel size" and falls through)
3. for .bdf: a pre-converted ``.pil`` sidecar via ``ImageFont.load``
4. ``fallback_font`` at the requested size
5. ``ImageFont.load_default()``
"""
font_name = resolve_font_name(font_name or "")
font_path = os.path.join(fonts_dir, font_name)
lower = font_name.lower()
if os.path.exists(font_path):
try:
return ImageFont.truetype(font_path, size)
except Exception as e:
logger.debug("truetype failed for %s@%s: %s", font_name, size, e)
if lower.endswith(".bdf"):
pil_path = font_path.rsplit(".", 1)[0] + ".pil"
if os.path.exists(pil_path):
try:
return ImageFont.load(pil_path)
except Exception as e:
logger.debug("PIL sidecar failed for %s: %s", pil_path, e)
logger.warning(
"BDF font %s could not be loaded at size %s (BDF fonts are "
"fixed-size; font_size must match the native size). Falling "
"back to %s.", font_name, size, fallback_font)
else:
logger.warning("Font file not found: %s, falling back to %s",
font_path, fallback_font)
fallback_path = os.path.join(fonts_dir, resolve_font_name(fallback_font))
try:
return ImageFont.truetype(fallback_path, size)
except Exception as e:
logger.warning("Fallback font %s failed (%s); using PIL default",
fallback_font, e)
return ImageFont.load_default()
@dataclass(frozen=True)
class ElementStyle:
"""Resolved style for one display element."""
font: PILFont
font_name: str
font_size: int
#: The user's color when they genuinely changed it, else the plugin's
#: classic color (which may be state-dependent — e.g. a score that turns
#: gold on a touchdown — so an untouched schema default must never
#: clobber it; the web form always posts the color inputs).
color: Optional[Tuple[int, int, int]]
#: Additive (dx, dy) translation from customization.layout offsets.
offset: Tuple[int, int]
#: True when the configured value genuinely differs from the schema
#: default (NOT merely present — saved configs always contain defaults).
user_forced_font: bool
user_forced_size: bool
user_forced_color: bool = False
@property
def user_forced(self) -> bool:
"""True when the user pinned this element's font or size; adaptive
layouts must use the font as-is instead of ladder-fitting. (Color is
deliberately excluded it never affects sizing.)"""
return self.user_forced_font or self.user_forced_size
def _as_int(value: Any, default: int) -> int:
"""Int coercion tolerant of floats and numeric strings from configs."""
if value is None:
return default
if isinstance(value, bool):
return default
if isinstance(value, (int, float)):
return int(value)
try:
return int(float(value))
except (TypeError, ValueError):
return default
def _as_color(value: Any) -> Optional[Tuple[int, int, int]]:
"""[r, g, b] list/tuple -> tuple; anything else -> None."""
if isinstance(value, (list, tuple)) and len(value) == 3:
try:
return tuple(max(0, min(255, int(c))) for c in value)
except (TypeError, ValueError):
return None
return None
class ElementStyleResolver:
"""Resolves per-element fonts, colors and offsets from a plugin config.
``schema_defaults`` is the nested defaults dict extracted from the
plugin's config_schema.json (SchemaManager.extract_defaults_from_schema).
It is the reference for the user-override check: a configured value
equal to its schema default is treated as untouched, because the save
flow persists all defaults. When ``schema_defaults`` is empty (older
cores, unit tests), the check degrades to comparing against the
``classic_*`` values the caller supplies.
"""
def __init__(self, config: Optional[Dict[str, Any]],
schema_defaults: Optional[Dict[str, Any]] = None, *,
fonts_dir: str = DEFAULT_FONTS_DIR,
fallback_font: str = DEFAULT_FALLBACK_FONT):
self._config = config if isinstance(config, dict) else {}
self._defaults = schema_defaults if isinstance(schema_defaults, dict) else {}
self._fonts_dir = fonts_dir
self._fallback_font = fallback_font
self._cache: Dict[Any, ElementStyle] = {}
# -- internals ----------------------------------------------------
def _element_config(self, element_key: str) -> Dict[str, Any]:
cust = self._config.get("customization")
if not isinstance(cust, dict):
return {}
element = cust.get(element_key)
return element if isinstance(element, dict) else {}
def _element_defaults(self, element_key: str) -> Dict[str, Any]:
cust = self._defaults.get("customization")
if not isinstance(cust, dict):
return {}
element = cust.get(element_key)
return element if isinstance(element, dict) else {}
# -- public API ---------------------------------------------------
def style(self, element_key: str, *, classic_font: str, classic_size: int,
classic_color: Optional[Tuple[int, int, int]] = None) -> ElementStyle:
"""Resolve the style for one element.
``classic_font``/``classic_size``/``classic_color`` are the plugin's
hardcoded defaults for this element used when the config has no
value, and as the override reference when schema defaults are
unavailable.
"""
cache_key = (element_key, classic_font, classic_size, classic_color)
cached = self._cache.get(cache_key)
if cached is not None:
return cached
element_cfg = self._element_config(element_key)
element_defaults = self._element_defaults(element_key)
configured_font = element_cfg.get("font")
configured_size = element_cfg.get("font_size")
# Reference for "did the user change it": schema default when known,
# else the plugin's classic default.
reference_font = element_defaults.get("font", classic_font)
reference_size = _as_int(element_defaults.get("font_size"), classic_size)
user_forced_font = (configured_font is not None
and configured_font != reference_font)
user_forced_size = (configured_size is not None
and _as_int(configured_size, reference_size) != reference_size)
font_name = configured_font if configured_font is not None else classic_font
font_size = _as_int(configured_size, classic_size)
font = load_font(font_name, font_size, fonts_dir=self._fonts_dir,
fallback_font=self._fallback_font)
# Color follows the same provenance rule as fonts: the web form
# always posts the RGB inputs, so a saved config carries the schema
# default whether or not the user touched it — only a value that
# DIFFERS from the schema default is a real override. Otherwise keep
# classic_color, which may be state-dependent (semantic colors like
# a gold touchdown score) and must not be clobbered by a default.
configured_color = _as_color(element_cfg.get("text_color"))
default_color = _as_color(element_defaults.get("text_color"))
if configured_color is None:
user_forced_color = False
elif default_color is None:
# no schema default to compare against — presence is intent
user_forced_color = True
else:
user_forced_color = configured_color != default_color
color = configured_color if user_forced_color else classic_color
resolved = ElementStyle(
font=font, font_name=font_name, font_size=font_size,
color=color, offset=self.offset(element_key),
user_forced_font=user_forced_font, user_forced_size=user_forced_size,
user_forced_color=user_forced_color,
)
self._cache[cache_key] = resolved
return resolved
def offset_value(self, element_key: str, axis: str, default: int = 0) -> int:
"""One offset axis for an element (e.g. 'x_offset', 'away_x_offset').
Reads ``customization.layout.<element>`` first (the deployed sports
convention), falling back to ``customization.<element>`` for plugins
that keep offsets on the element itself.
"""
cust = self._config.get("customization")
if not isinstance(cust, dict):
return default
layout = cust.get("layout")
if isinstance(layout, dict):
element = layout.get(element_key)
if isinstance(element, dict) and axis in element:
return _as_int(element.get(axis), default)
element = cust.get(element_key)
if isinstance(element, dict) and axis in element:
return _as_int(element.get(axis), default)
return default
def offset(self, element_key: str) -> Tuple[int, int]:
"""(dx, dy) additive translation for an element; (0, 0) when unset."""
return (self.offset_value(element_key, "x_offset"),
self.offset_value(element_key, "y_offset"))
def is_for(self, config: Optional[Dict[str, Any]]) -> bool:
"""True when this resolver was built over exactly this config object.
Callers that cache a resolver (plugins whose config dict gets
REPLACED on config change) use this to decide when to rebuild
preferred over reaching into the private ``_config``.
"""
return self._config is config
def clear_cache(self) -> None:
self._cache.clear()
+18 -5
View File
@@ -35,6 +35,7 @@ import urllib.request
import zipfile
import tempfile
import time
from collections import OrderedDict
from pathlib import Path
from PIL import ImageFont
from typing import Dict, Tuple, Optional, Union, Any, List
@@ -58,7 +59,13 @@ class FontManager:
# Font discovery and catalog
self.font_catalog: Dict[str, str] = {} # family_name -> file_path
self.font_cache: Dict[str, Union[ImageFont.FreeTypeFont, freetype.Face]] = {} # (family, size) -> font
self.metrics_cache: Dict[str, Tuple[int, int, int]] = {} # (text, font_id) -> (width, height, baseline)
# (text, id(font)) -> ((width, height, baseline), font_ref).
# LRU-bounded — keys embed the measured TEXT, so changing strings
# (clocks, live scores) would otherwise grow it forever. Entries
# keep the font alive so its id() can't be recycled by a different
# font object (which would silently return wrong metrics).
self.metrics_cache: "OrderedDict[Any, Tuple[Tuple[int, int, int], Any]]" = OrderedDict()
self._METRICS_CACHE_MAX = 1024
# Plugin font management
self.plugin_fonts: Dict[str, Dict[str, Any]] = {} # plugin_id -> font_manifest
@@ -555,10 +562,14 @@ class FontManager:
Returns:
Tuple of (width, height, baseline_offset)
"""
cache_key = f"{hash(text)}_{id(font)}"
# Key on the text itself (hash(text) could collide) + font identity;
# the entry below keeps the font referenced so the id stays valid.
cache_key = (text, id(font))
if cache_key in self.metrics_cache:
return self.metrics_cache[cache_key]
cached = self.metrics_cache.get(cache_key)
if cached is not None:
self.metrics_cache.move_to_end(cache_key)
return cached[0]
try:
if isinstance(font, freetype.Face):
@@ -595,7 +606,9 @@ class FontManager:
baseline = 10
result = (width, height, baseline)
self.metrics_cache[cache_key] = result
self.metrics_cache[cache_key] = (result, font)
while len(self.metrics_cache) > self._METRICS_CACHE_MAX:
self.metrics_cache.popitem(last=False)
return result
def get_font_height(self, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> int:
-135
View File
@@ -1,135 +0,0 @@
import os
import freetype
from PIL import ImageDraw, ImageFont
import logging
from typing import Dict, Any
from src.display_manager import DisplayManager
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class FontTestManager:
"""Manager for testing fonts with easy BDF/TTF switching."""
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager):
self.display_manager = display_manager
self.config = config
self.logger = logging.getLogger('FontTest')
# FONT CONFIGURATION - EASY SWITCHING
# Set to 'bdf' or 'ttf' to switch font types
self.font_type = 'bdf' # Change this to 'ttf' to use TTF font
# Font configurations
self.font_configs = {
'bdf': {
'path': "assets/fonts/cozette.bdf",
'display_name': "Cozette BTF",
'description': "BTF font Test"
},
'ttf': {
'path': "assets/fonts/5by7.regular.ttf",
'display_name': "5by7 TTF",
'description': "TTF font test"
}
}
# Get current font configuration
self.current_config = self.font_configs[self.font_type]
self.font_path = self.current_config['path']
# Verify font exists
if not os.path.exists(self.font_path):
self.logger.error(f"Font file not found: {self.font_path}")
raise FileNotFoundError(f"Font file not found: {self.font_path}")
# Load the font based on type
if self.font_type == 'bdf':
self._load_bdf_font()
else:
self._load_ttf_font()
self.logger.info(f"Initialized FontTestManager with {self.current_config['description']}")
def _load_bdf_font(self):
"""Load BDF font using freetype."""
try:
self.face = freetype.Face(self.font_path)
self.logger.info(f"Successfully loaded BDF font from {self.font_path}")
except Exception as e:
self.logger.error(f"Failed to load BDF font: {e}")
raise
def _load_ttf_font(self):
"""Load TTF font using PIL."""
try:
self.font = ImageFont.truetype(self.font_path, 8) # Size 8 for 5x7 font
self.logger.info(f"Successfully loaded TTF font from {self.font_path}")
except Exception as e:
self.logger.error(f"Failed to load TTF font: {e}")
raise
def update(self):
"""No update needed for static display."""
def display(self, force_clear: bool = False):
"""Display the font with sample text."""
try:
# Clear the display
self.display_manager.clear()
# Draw font name at the top
self.display_manager.draw_text(self.current_config['display_name'], y=2, color=(255, 255, 255))
# Draw sample text
draw = ImageDraw.Draw(self.display_manager.image)
sample_text = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
# Calculate starting position
x = 10 # Start 10 pixels from the left
y = 10 # Start 10 pixels from the top
# Draw text based on font type
if self.font_type == 'bdf':
self._draw_bdf_text(draw, sample_text, x, y)
else:
self._draw_ttf_text(draw, sample_text, x, y)
# Update the display once
self.display_manager.update_display()
# Log that display is complete
self.logger.info("Font test display complete.")
except Exception as e:
self.logger.error(f"Error displaying font test: {e}", exc_info=True)
def _draw_bdf_text(self, draw, text, x, y):
"""Draw text using BDF font."""
for char in text:
# Load the glyph
self.face.load_char(char)
bitmap = self.face.glyph.bitmap
# Draw the glyph
for i in range(bitmap.rows):
for j in range(bitmap.width):
try:
# Get the byte containing the pixel
byte_index = i * bitmap.pitch + (j // 8)
if byte_index < len(bitmap.buffer):
byte = bitmap.buffer[byte_index]
# Check if the specific bit is set
if byte & (1 << (7 - (j % 8))):
draw.point((x + j, y + i), fill=(255, 255, 255))
except IndexError:
self.logger.warning(f"Index out of range for char '{char}' at position ({i}, {j})")
continue
# Move to next character position
x += self.face.glyph.advance.x >> 6
def _draw_ttf_text(self, draw, text, x, y):
"""Draw text using TTF font."""
draw.text((x, y), text, font=self.font, fill=(255, 255, 255))
-150
View File
@@ -1,150 +0,0 @@
"""
Generic Cache Mixin for Any Manager
This mixin provides caching functionality that can be used by any manager
that needs to cache data, not just sports managers. It's a more general
version of BackgroundCacheMixin that works for weather, stocks, news, etc.
"""
import time
from typing import Dict, Optional, Any, Callable
class GenericCacheMixin:
"""
Generic mixin class that provides caching functionality to any manager.
This mixin can be used by weather, stock, news, or any other manager
that needs to cache data with performance monitoring.
Note: For sports managers that need background service cache integration,
use BackgroundCacheMixin instead. See src/background_cache_mixin.py for details.
"""
def _fetch_data_with_cache(self,
cache_key: str,
api_fetch_method: Callable,
cache_ttl: int = 300,
force_refresh: bool = False) -> Optional[Dict]:
"""
Generic caching pattern for any manager.
Args:
cache_key: Unique cache key for this data
api_fetch_method: Method to call for fresh data
cache_ttl: Time-to-live in seconds (default: 5 minutes)
force_refresh: Skip cache and fetch fresh data
Returns:
Cached or fresh data from API
"""
start_time = time.time()
cache_hit = False
cache_source = None
try:
# Check cache first (unless forcing refresh)
if not force_refresh:
cached_data = self.cache_manager.get_cached_data(cache_key, cache_ttl)
if cached_data:
self.logger.info(f"Using cached data for {cache_key}")
cache_hit = True
cache_source = "cache"
self.cache_manager.record_cache_hit('regular')
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
self._log_fetch_performance(cache_key, duration, cache_hit, cache_source)
return cached_data
# Fetch fresh data
self.logger.info(f"Fetching fresh data for {cache_key}")
result = api_fetch_method()
cache_source = "api_fresh"
# Store in cache if we got data
if result:
self.cache_manager.save_cache(cache_key, result)
self.cache_manager.record_cache_miss('regular')
else:
self.logger.warning(f"No data returned for {cache_key}")
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
# Log performance
self._log_fetch_performance(cache_key, duration, cache_hit, cache_source)
return result
except Exception as e:
duration = time.time() - start_time
self.logger.error(f"Error fetching data for {cache_key} after {duration:.2f}s: {e}")
self.cache_manager.record_fetch_time(duration)
raise
def _log_fetch_performance(self, cache_key: str, duration: float, cache_hit: bool, cache_source: str):
"""
Log detailed performance metrics for fetch operations.
Args:
cache_key: Cache key that was accessed
duration: Fetch operation duration in seconds
cache_hit: Whether this was a cache hit
cache_source: Source of the data (cache, api_fresh, etc.)
"""
# Log basic performance info
self.logger.info(f"Fetch completed for {cache_key} in {duration:.2f}s "
f"(cache_hit={cache_hit}, source={cache_source})")
# Log detailed metrics every 10 operations
if hasattr(self, '_fetch_count'):
self._fetch_count += 1
else:
self._fetch_count = 1
if self._fetch_count % 10 == 0:
metrics = self.cache_manager.get_cache_metrics()
self.logger.info(f"Cache Performance Summary - "
f"Hit Rate: {metrics['cache_hit_rate']:.2%}, "
f"API Calls Saved: {metrics['api_calls_saved']}, "
f"Avg Fetch Time: {metrics['average_fetch_time']:.2f}s")
def get_cache_performance_summary(self) -> Dict[str, Any]:
"""
Get cache performance summary for this manager.
Returns:
Dictionary containing cache performance metrics
"""
return self.cache_manager.get_cache_metrics()
def log_cache_performance(self):
"""Log current cache performance metrics."""
self.cache_manager.log_cache_metrics()
def clear_cache_for_key(self, cache_key: str):
"""Clear cache for a specific key."""
self.cache_manager.clear_cache(cache_key)
self.logger.info(f"Cleared cache for {cache_key}")
def get_cache_info(self, cache_key: str) -> Dict[str, Any]:
"""
Get information about a cached item.
Args:
cache_key: Cache key to check
Returns:
Dictionary with cache information
"""
# This would need to be implemented in CacheManager
# For now, just return basic info
return {
'key': cache_key,
'exists': self.cache_manager.get_cached_data(cache_key, 0) is not None,
'ttl': 'unknown' # Would need to be implemented
}
-22
View File
@@ -1,22 +0,0 @@
"""Deprecated: use src/adaptive_images.py (fit_image) instead.
This module predates the adaptive image system and has no known callers.
It is kept only so any out-of-tree code importing it keeps working.
"""
import logging
from PIL import Image
logger = logging.getLogger(__name__)
def scale_to_max_dimensions(img, max_width, max_height):
h_to_w_ratio = img.height / img.width
w_to_h_ratio = img.width / img.height
if img.height > max_height:
img = img.resize((int(max_height * w_to_h_ratio), max_height), Image.Resampling.LANCZOS)
if img.width > max_width:
img = img.resize((max_width, int(max_width * h_to_w_ratio)), Image.Resampling.LANCZOS)
return img
-409
View File
@@ -1,409 +0,0 @@
"""
Layout Manager for LED Matrix Display
Handles custom layouts, element positioning, and display composition.
"""
import json
import os
import logging
from typing import Dict, List, Any
from datetime import datetime
logger = logging.getLogger(__name__)
class LayoutManager:
def __init__(self, display_manager=None, config_path="config/custom_layouts.json"):
self.display_manager = display_manager
self.config_path = config_path
self.layouts = self.load_layouts()
self.current_layout = None
def load_layouts(self) -> Dict[str, Any]:
"""Load saved layouts from file."""
try:
if os.path.exists(self.config_path):
with open(self.config_path, 'r') as f:
return json.load(f)
return {}
except Exception as e:
logger.error(f"Error loading layouts: {e}")
return {}
def save_layouts(self) -> bool:
"""Save layouts to file."""
try:
from pathlib import Path
from src.common.permission_utils import (
ensure_directory_permissions,
get_config_dir_mode
)
config_path_obj = Path(self.config_path)
ensure_directory_permissions(config_path_obj.parent, get_config_dir_mode())
with open(self.config_path, 'w') as f:
json.dump(self.layouts, f, indent=2)
return True
except Exception as e:
logger.error(f"Error saving layouts: {e}")
return False
def create_layout(self, name: str, elements: List[Dict], description: str = "") -> bool:
"""Create a new layout."""
try:
self.layouts[name] = {
'elements': elements,
'description': description,
'created': datetime.now().isoformat(),
'modified': datetime.now().isoformat()
}
return self.save_layouts()
except Exception as e:
logger.error(f"Error creating layout '{name}': {e}")
return False
def update_layout(self, name: str, elements: List[Dict], description: str = None) -> bool:
"""Update an existing layout."""
try:
if name not in self.layouts:
return False
self.layouts[name]['elements'] = elements
self.layouts[name]['modified'] = datetime.now().isoformat()
if description is not None:
self.layouts[name]['description'] = description
return self.save_layouts()
except Exception as e:
logger.error(f"Error updating layout '{name}': {e}")
return False
def delete_layout(self, name: str) -> bool:
"""Delete a layout."""
try:
if name in self.layouts:
del self.layouts[name]
return self.save_layouts()
return False
except Exception as e:
logger.error(f"Error deleting layout '{name}': {e}")
return False
def get_layout(self, name: str) -> Dict[str, Any]:
"""Get a specific layout."""
return self.layouts.get(name, {})
def list_layouts(self) -> List[str]:
"""Get list of all layout names."""
return list(self.layouts.keys())
def set_current_layout(self, name: str) -> bool:
"""Set the current active layout."""
if name in self.layouts:
self.current_layout = name
return True
return False
def render_layout(self, layout_name: str = None, data_context: Dict = None) -> bool:
"""Render a layout to the display."""
if not self.display_manager:
logger.error("No display manager available")
return False
layout_name = layout_name or self.current_layout
if not layout_name or layout_name not in self.layouts:
logger.error(f"Layout '{layout_name}' not found")
return False
try:
# Clear the display
self.display_manager.clear()
# Get layout elements
elements = self.layouts[layout_name]['elements']
# Render each element
for element in elements:
self.render_element(element, data_context or {})
# Update the display
self.display_manager.update_display()
return True
except Exception as e:
logger.error(f"Error rendering layout '{layout_name}': {e}")
return False
def render_element(self, element: Dict, data_context: Dict) -> None:
"""Render a single element."""
element_type = element.get('type')
x = element.get('x', 0)
y = element.get('y', 0)
properties = element.get('properties', {})
try:
if element_type == 'text':
self._render_text_element(x, y, properties, data_context)
elif element_type == 'weather_icon':
self._render_weather_icon_element(x, y, properties, data_context)
elif element_type == 'rectangle':
self._render_rectangle_element(x, y, properties)
elif element_type == 'line':
self._render_line_element(x, y, properties)
elif element_type == 'clock':
self._render_clock_element(x, y, properties)
elif element_type == 'data_text':
self._render_data_text_element(x, y, properties, data_context)
else:
logger.warning(f"Unknown element type: {element_type}")
except Exception as e:
logger.error(f"Error rendering element {element_type}: {e}")
def _render_text_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a text element."""
text = properties.get('text', 'Sample Text')
color = tuple(properties.get('color', [255, 255, 255]))
font_size = properties.get('font_size', 'normal')
# Support template variables in text
text = self._process_template_text(text, data_context)
# Select font
if font_size == 'small':
font = self.display_manager.small_font
elif font_size == 'large':
font = self.display_manager.regular_font
else:
font = self.display_manager.regular_font
self.display_manager.draw_text(text, x, y, color, font=font)
def _render_weather_icon_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a weather icon element."""
condition = properties.get('condition', 'sunny')
size = properties.get('size', 16)
# Use weather data from context if available
if 'weather' in data_context and 'condition' in data_context['weather']:
condition = data_context['weather']['condition'].lower()
self.display_manager.draw_weather_icon(condition, x, y, size)
def _render_rectangle_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a rectangle element."""
width = properties.get('width', 10)
height = properties.get('height', 10)
color = tuple(properties.get('color', [255, 255, 255]))
filled = properties.get('filled', False)
if filled:
self.display_manager.draw.rectangle(
[x, y, x + width, y + height],
fill=color
)
else:
self.display_manager.draw.rectangle(
[x, y, x + width, y + height],
outline=color
)
def _render_line_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a line element."""
x2 = properties.get('x2', x + 10)
y2 = properties.get('y2', y)
color = tuple(properties.get('color', [255, 255, 255]))
width = properties.get('width', 1)
self.display_manager.draw.line([x, y, x2, y2], fill=color, width=width)
def _render_clock_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a clock element."""
format_str = properties.get('format', '%H:%M')
color = tuple(properties.get('color', [255, 255, 255]))
current_time = datetime.now().strftime(format_str)
self.display_manager.draw_text(current_time, x, y, color)
def _render_data_text_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a data-driven text element."""
data_key = properties.get('data_key', '')
format_str = properties.get('format', '{value}')
color = tuple(properties.get('color', [255, 255, 255]))
default_value = properties.get('default', 'N/A')
# Extract data from context
value = self._get_nested_value(data_context, data_key, default_value)
# Format the text
try:
text = format_str.format(value=value)
except (ValueError, TypeError, KeyError, IndexError):
text = str(value)
self.display_manager.draw_text(text, x, y, color)
def _process_template_text(self, text: str, data_context: Dict) -> str:
"""Process template variables in text."""
try:
# Simple template processing - replace {key} with values from context
for key, value in data_context.items():
placeholder = f"{{{key}}}"
if placeholder in text:
text = text.replace(placeholder, str(value))
return text
except Exception as e:
logger.error(f"Error processing template text: {e}")
return text
def _get_nested_value(self, data: Dict, key: str, default=None):
"""Get a nested value from a dictionary using dot notation."""
try:
keys = key.split('.')
value = data
for k in keys:
value = value[k]
return value
except (KeyError, TypeError):
return default
def create_preset_layouts(self) -> None:
"""Create some preset layouts for common use cases."""
# Basic clock layout
clock_layout = [
{
'type': 'clock',
'x': 10,
'y': 10,
'properties': {
'format': '%H:%M',
'color': [255, 255, 255]
}
},
{
'type': 'clock',
'x': 10,
'y': 20,
'properties': {
'format': '%m/%d',
'color': [100, 100, 255]
}
}
]
self.create_layout('basic_clock', clock_layout, 'Simple clock with date')
# Weather layout
weather_layout = [
{
'type': 'weather_icon',
'x': 5,
'y': 5,
'properties': {
'condition': 'sunny',
'size': 20
}
},
{
'type': 'data_text',
'x': 30,
'y': 8,
'properties': {
'data_key': 'weather.temperature',
'format': '{value}°',
'color': [255, 200, 0],
'default': '--°'
}
},
{
'type': 'data_text',
'x': 30,
'y': 18,
'properties': {
'data_key': 'weather.condition',
'format': '{value}',
'color': [200, 200, 200],
'default': 'Unknown'
}
}
]
self.create_layout('weather_display', weather_layout, 'Weather icon with temperature and condition')
# Mixed dashboard layout
dashboard_layout = [
{
'type': 'clock',
'x': 2,
'y': 2,
'properties': {
'format': '%H:%M',
'color': [255, 255, 255]
}
},
{
'type': 'weather_icon',
'x': 50,
'y': 2,
'properties': {
'size': 16
}
},
{
'type': 'data_text',
'x': 70,
'y': 5,
'properties': {
'data_key': 'weather.temperature',
'format': '{value}°',
'color': [255, 200, 0],
'default': '--°'
}
},
{
'type': 'line',
'x': 0,
'y': 15,
'properties': {
'x2': 128,
'y2': 15,
'color': [100, 100, 100]
}
},
{
'type': 'data_text',
'x': 2,
'y': 18,
'properties': {
'data_key': 'stocks.AAPL.price',
'format': 'AAPL: ${value}',
'color': [0, 255, 0],
'default': 'AAPL: N/A'
}
}
]
self.create_layout('dashboard', dashboard_layout, 'Mixed dashboard with clock, weather, and stocks')
logger.info("Created preset layouts")
def get_layout_preview(self, layout_name: str) -> Dict[str, Any]:
"""Get a preview representation of a layout."""
if layout_name not in self.layouts:
return {}
layout = self.layouts[layout_name]
elements = layout['elements']
# Create a simple preview representation
preview = {
'name': layout_name,
'description': layout.get('description', ''),
'element_count': len(elements),
'elements': []
}
for element in elements:
preview['elements'].append({
'type': element.get('type'),
'position': f"({element.get('x', 0)}, {element.get('y', 0)})",
'properties': list(element.get('properties', {}).keys())
})
return preview
+26 -8
View File
@@ -139,23 +139,41 @@ def setup_logging(
sys.stderr.write(f"Warning: Could not set up file logging to {log_file}: {e}\n")
def get_logger(name: str, plugin_id: Optional[str] = None) -> logging.Logger:
class PluginLoggerAdapter(logging.LoggerAdapter):
"""LoggerAdapter that stamps every record with its plugin_id.
A plain `logging.Logger` attribute (the old approach) is never copied
onto individual `LogRecord`s, so `ContextualFormatter`/`StructuredFormatter`
only ever saw `plugin_id` on calls that explicitly passed
`extra={'plugin_id': ...}` (i.e. `log_with_context`). This adapter injects
it into `extra` on every call, so `self.logger.info(...)` in plugin code
is tagged automatically.
"""
def process(self, msg, kwargs):
extra = dict(kwargs.get('extra') or {})
extra.setdefault('plugin_id', self.extra.get('plugin_id'))
kwargs['extra'] = extra
return msg, kwargs
def get_logger(name: str, plugin_id: Optional[str] = None):
"""
Get a logger with consistent configuration.
Args:
name: Logger name (typically __name__)
plugin_id: Optional plugin ID for automatic context
Returns:
Configured logger instance
Configured logger instance (or a PluginLoggerAdapter when plugin_id
is given, which supports the same .debug/.info/.warning/.error API)
"""
logger = logging.getLogger(name)
# Add plugin_id as attribute for formatters
if plugin_id:
logger.plugin_id = plugin_id
return PluginLoggerAdapter(logger, {'plugin_id': plugin_id})
return logger
+3 -62
View File
@@ -86,7 +86,9 @@ class BasePlugin(ABC):
self.display_manager: Any = display_manager
self.cache_manager: Any = cache_manager
self.plugin_manager: Any = plugin_manager
self.logger: logging.Logger = get_logger(f"plugin.{plugin_id}", plugin_id=plugin_id)
# get_logger returns a PluginLoggerAdapter here (plugin_id given), which
# stamps every record with plugin_id so it survives into formatted output.
self.logger = get_logger(f"plugin.{plugin_id}", plugin_id=plugin_id)
self.enabled: bool = config.get("enabled", True)
self.logger.info("Initialized plugin: %s", plugin_id)
@@ -270,64 +272,6 @@ class BasePlugin(ABC):
return (int(width), int(height))
return DEFAULT_DESIGN_SIZE
# -------------------------------------------------------------------------
# Element style resolution (per-element user customization)
# -------------------------------------------------------------------------
@property
def style_resolver(self) -> Any:
"""
ElementStyleResolver for this plugin's customization config.
Lazily built with the schema defaults from this plugin's own
config_schema.json (via the plugin manager's schema manager), so
"did the user override this font?" is answered correctly even though
saved configs always contain the schema defaults. Rebuilt when the
config changes (see on_config_change). Pass it into standalone helper
classes (game renderers etc.) instead of letting them build their own.
See src/element_style.py.
"""
from src.element_style import ElementStyleResolver
cached = getattr(self, "_style_resolver", None)
if cached is not None:
return cached
schema_defaults: Dict[str, Any] = {}
schema_manager = getattr(self.plugin_manager, "schema_manager", None)
if schema_manager is not None:
try:
schema = schema_manager.load_schema(self.plugin_id)
if schema:
schema_defaults = schema_manager.extract_defaults_from_schema(schema)
except Exception as e:
self.logger.debug("Schema defaults unavailable for %s: %s",
self.plugin_id, e)
resolver = ElementStyleResolver(self.config, schema_defaults)
self._style_resolver = resolver
return resolver
def element_style(self, element_key: str, *, classic_font: str,
classic_size: int,
classic_color: Optional[tuple] = None) -> Any:
"""
Resolved font/color/offset for one display element, honoring the
user's customization.<element> config. classic_* are this plugin's
hardcoded defaults for the element.
Example:
style = self.element_style('title_text',
classic_font='PressStart2P-Regular.ttf',
classic_size=8)
draw.text((x + style.offset[0], y + style.offset[1]),
title, font=style.font,
fill=style.color or (255, 255, 255))
"""
return self.style_resolver.style(element_key, classic_font=classic_font,
classic_size=classic_size,
classic_color=classic_color)
def get_display_duration(self) -> float:
"""
Get the display duration for this plugin instance.
@@ -790,9 +734,6 @@ class BasePlugin(ABC):
# Update simple flags
self.enabled = self.config.get("enabled", self.enabled)
# Invalidate the cached style resolver — it captured the old config
self._style_resolver = None
def get_info(self) -> Dict[str, Any]:
"""
Return plugin info for display in web UI.
+269 -45
View File
@@ -8,12 +8,13 @@ API Version: 1.0.0
"""
import json
import queue
import sys
import time
import threading
import types
from pathlib import Path
from typing import Dict, List, Optional, Any
from typing import Dict, List, Optional, Any, Tuple
import logging
from src.exceptions import PluginError, ConfigError
from src.logging_config import get_logger
@@ -76,6 +77,12 @@ class PluginManager:
# concurrent mutation (background reconciliation) and reads (requests).
self._discovery_lock = threading.RLock()
# Lock protecting plugin_last_update from concurrent mutation/iteration.
# It's written from run_scheduled_updates()/update_all_plugins() (main
# loop) and read/diffed by run_scheduled_updates_with_changes(), which
# Vegas mode calls from its own background update-tick thread.
self._plugin_last_update_lock = threading.RLock()
# Active plugins
self.plugins: Dict[str, Any] = {}
self.plugin_manifests: Dict[str, Dict[str, Any]] = {}
@@ -91,6 +98,50 @@ class PluginManager:
# Health tracking (optional, set by display_controller if available)
self.health_tracker = None
self.resource_monitor = None
# --- Asynchronous plugin updates -------------------------------
# update() used to run inline in the render loop (execute_update's
# internal thread.join(timeout=30) blocked it), so one slow plugin
# HTTP fetch froze scrolling for the whole fetch. Scheduling still
# happens on the render thread (run_scheduled_updates), but
# execution moves to this single background worker. Per-plugin
# locks keep a plugin's update() and display() mutually exclusive —
# today's implicit guarantee, now explicit (and, unlike today,
# also held across the post-timeout window).
# Kill switch: plugin_system.synchronous_updates: true restores the
# inline path.
self._update_queue: "queue.Queue[Optional[Tuple[str, float]]]" = queue.Queue()
self._pending_updates: set = set()
self._pending_lock = threading.Lock()
self._plugin_locks: Dict[str, threading.Lock] = {}
self._plugin_locks_guard = threading.Lock()
self._update_worker: Optional[threading.Thread] = None
self._synchronous_updates = False
if self.config_manager is not None:
try:
cfg = self.config_manager.get_config() or {}
except (OSError, ValueError) as exc:
self.logger.warning(
"Could not load config to check plugin_system.synchronous_updates "
"(%s: %s); defaulting to synchronous updates", type(exc).__name__, exc)
self._synchronous_updates = True
else:
plugin_system_cfg = cfg.get('plugin_system', {})
if not isinstance(plugin_system_cfg, dict):
self.logger.warning(
"config plugin_system must be a mapping, got %s; "
"defaulting to synchronous updates",
type(plugin_system_cfg).__name__)
self._synchronous_updates = True
else:
sync_value = plugin_system_cfg.get('synchronous_updates', False)
if not isinstance(sync_value, bool):
self.logger.warning(
"config plugin_system.synchronous_updates must be a boolean, "
"got %r; defaulting to synchronous updates", sync_value)
self._synchronous_updates = True
else:
self._synchronous_updates = sync_value
# Ensure plugins directory exists with proper permissions
try:
@@ -317,7 +368,8 @@ class PluginManager:
# Store plugin instance
self.plugins[plugin_id] = plugin_instance
self.plugin_last_update[plugin_id] = 0.0
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = 0.0
# Invalidate cached interval so next tick re-derives it for this plugin
self._update_interval_cache.pop(plugin_id, None)
@@ -429,7 +481,8 @@ class PluginManager:
# Remove from active plugins
del self.plugins[plugin_id]
self.plugin_last_update.pop(plugin_id, None)
with self._plugin_last_update_lock:
self.plugin_last_update.pop(plugin_id, None)
self._update_interval_cache.pop(plugin_id, None)
# Remove main module from sys.modules if present
@@ -698,7 +751,8 @@ class PluginManager:
'recoverable': True,
}
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
self.plugin_last_update[plugin_id] = failure_time
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = failure_time
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info, error=err)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, err)
@@ -731,49 +785,218 @@ class PluginManager:
if interval is None:
continue
last_update = self.plugin_last_update.get(plugin_id, 0.0)
with self._plugin_last_update_lock:
last_update = self.plugin_last_update.get(plugin_id, 0.0)
if last_update == 0.0 or (current_time - last_update) >= interval:
# Update state to RUNNING
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
try:
# Use PluginExecutor for safe execution
success = False
if self.resource_monitor:
# If resource monitor exists, wrap the call
def monitored_update():
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
# SimpleNamespace stores `update` as an *instance*
# attribute, so attribute lookup returns the plain
# function object as-is. A dynamically-built class
# (`type(..., {'update': monitored_update})`) instead
# stores it as a *class* attribute, which the
# descriptor protocol turns into a bound method on
# access -- silently prepending the instance as an
# implicit first argument to a function that takes
# none, raising "monitored_update() takes 0
# positional arguments but 1 was given" on every call.
success = self.plugin_executor.execute_update(
types.SimpleNamespace(update=monitored_update),
plugin_id
)
else:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
self.plugin_last_update[plugin_id] = current_time
self.state_manager.record_update(plugin_id)
# Update state back to ENABLED
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
# Record success
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
if self._synchronous_updates:
# Kill-switch path: the original inline execution
# (blocks the caller until update() completes/times out)
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
self._execute_update_now(plugin_id, plugin_instance, current_time)
else:
self._enqueue_update(plugin_id, current_time)
def get_plugin_lock(self, plugin_id: str) -> threading.Lock:
"""Per-plugin lock keeping update() and display() mutually exclusive.
The update worker holds it for the duration of a plugin's update();
the display side acquires it non-blocking and skips that frame's
display() call when the plugin is mid-update.
"""
with self._plugin_locks_guard:
lock = self._plugin_locks.get(plugin_id)
if lock is None:
lock = threading.Lock()
self._plugin_locks[plugin_id] = lock
return lock
def _enqueue_update(self, plugin_id: str, scheduled_time: float) -> None:
"""Queue a due update for the background worker (dedup while pending)."""
with self._pending_lock:
if plugin_id in self._pending_updates:
return
self._pending_updates.add(plugin_id)
# RUNNING is set at enqueue time so can_execute() blocks re-entry and
# the web UI shows the truthful state while the item waits its turn.
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
self._ensure_update_worker()
self._update_queue.put((plugin_id, scheduled_time))
def _ensure_update_worker(self) -> None:
if self._update_worker is not None and self._update_worker.is_alive():
return
self._update_worker = threading.Thread(
target=self._update_worker_loop, name='plugin-update-worker',
daemon=True)
self._update_worker.start()
def _update_worker_loop(self) -> None:
"""Single worker: dispatches queued updates off the render thread
(matching the old inline behavior no thundering herd of
concurrent fetches).
The plugin's lock is acquired here, before its instance is looked
up, and the instance is re-fetched under the lock a concurrent
unload_plugin() can't leave this loop about to run update() on an
instance that's already been torn down. The lock — and RUNNING/
pending lifecycle state is released by the update itself once the
real update() call genuinely finishes (see _execute_update_now),
which can be after this dispatch returns if PluginExecutor's own
timeout elapses first.
"""
while True:
item = self._update_queue.get()
if item is None: # shutdown sentinel
return
plugin_id, scheduled_time = item
lock = self.get_plugin_lock(plugin_id)
lock.acquire()
plugin_instance = self.plugins.get(plugin_id)
if plugin_instance is None: # unloaded while queued; its
# lifecycle state was already cleared by unload_plugin —
# leave it alone rather than resurrecting it to ENABLED
lock.release()
with self._pending_lock:
self._pending_updates.discard(plugin_id)
continue
try:
self._execute_update_now(plugin_id, plugin_instance,
scheduled_time, lock=lock)
except Exception: # pylint: disable=broad-except
# _execute_update_now guarantees the lock/pending bookkeeping
# is released via its own _finish() before returning or
# raising; this is a last-resort log only.
self.logger.exception("update worker: unexpected error for %s",
plugin_id)
def stop_update_worker(self, timeout: float = 5.0) -> None:
"""Signal the worker to exit (used by cleanup; thread is a daemon)."""
if self._update_worker is not None and self._update_worker.is_alive():
self._update_queue.put(None)
self._update_worker.join(timeout=timeout)
if self._update_worker.is_alive():
self.logger.warning(
"Update worker did not stop within %.1fs; it is a daemon "
"thread and will be abandoned on shutdown", timeout)
def _execute_update_now(self, plugin_id: str, plugin_instance: Any,
scheduled_time: float,
lock: Optional[threading.Lock] = None) -> None:
"""Execute a plugin's update() via PluginExecutor, then bookkeep.
Caller is responsible for having set RUNNING state.
On the synchronous path (``lock=None``) this is the original,
unchanged inline behavior. On the async worker path, PluginExecutor's
internal thread.join(timeout) blocks only the calling thread -- on
timeout the lingering daemon update-thread keeps running the real
plugin.update() call unkillable in the background. So that the
plugin's lock (and its RUNNING/pending lifecycle state) stays held
for that real duration rather than just this bounded wait, ownership
of both is carried by the wrapped update callable itself, released
from whichever thread actually finishes it -- see _finish() below.
"""
finish_guard = threading.Lock()
finished = {'done': False}
def _finish(success: bool, exc: Optional[Exception] = None) -> None:
with finish_guard:
if finished['done']:
return
finished['done'] = True
try:
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = scheduled_time
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id, exc=exc)
finally:
if lock is not None:
lock.release()
with self._pending_lock:
self._pending_updates.discard(plugin_id)
if lock is None:
# Synchronous / no-lock path: unchanged behavior.
try:
if self.resource_monitor:
def monitored_update():
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
# SimpleNamespace stores `update` as an *instance*
# attribute, so attribute lookup returns the plain
# function object as-is. A dynamically-built class
# (`type(..., {'update': monitored_update})`) instead
# stores it as a *class* attribute, which the
# descriptor protocol turns into a bound method on
# access -- silently prepending the instance as an
# implicit first argument to a function that takes
# none, raising "monitored_update() takes 0
# positional arguments but 1 was given" on every call.
success = self.plugin_executor.execute_update(
types.SimpleNamespace(update=monitored_update),
plugin_id
)
else:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
_finish(success)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
_finish(False, exc=exc)
return
# Async worker path: the real update() call -- through the resource
# monitor, if configured -- owns finishing the lock/lifecycle
# bookkeeping, from whichever thread actually runs it to completion.
def _target_update() -> None:
try:
if self.resource_monitor:
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
else:
plugin_instance.update()
except Exception as exc:
_finish(False, exc=exc)
raise
else:
_finish(True)
try:
self.plugin_executor.execute_update(
types.SimpleNamespace(update=_target_update), plugin_id)
except Exception as exc: # pragma: no cover - defensive; execute_update
# catches everything internally, but guarantee _finish still
# runs (releasing the lock) if something unexpected slips through.
self.logger.exception("Unexpected error dispatching update for %s: %s", plugin_id, exc)
_finish(False, exc=exc)
def run_scheduled_updates_with_changes(self, current_time: Optional[float] = None) -> List[str]:
"""
Like run_scheduled_updates(), but also returns the plugin_ids whose
plugin_last_update timestamp actually advanced during this call.
The before/after snapshots and the update pass itself are each
individually lock-protected against concurrent plugin_last_update
mutation (Vegas mode calls this from its own background
update-tick thread, racing the main render loop's plugin updates),
so callers get an atomic "who got fresh data" answer without
reaching into plugin_last_update themselves. The lock is not held
across the update pass so slow/blocking plugin update() calls don't
serialize against other plugin_last_update readers.
"""
with self._plugin_last_update_lock:
old_times = dict(self.plugin_last_update)
self.run_scheduled_updates(current_time)
with self._plugin_last_update_lock:
return [
plugin_id for plugin_id, new_time in self.plugin_last_update.items()
if new_time > old_times.get(plugin_id, 0.0)
]
def update_all_plugins(self) -> None:
"""
@@ -797,7 +1020,8 @@ class PluginManager:
try:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
self.plugin_last_update[plugin_id] = time.time()
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = time.time()
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else:
+3 -39
View File
@@ -110,17 +110,12 @@ class SchemaManager:
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Validate schema structure (basic check)
if not isinstance(schema, dict):
self.logger.error(f"Invalid schema format for {plugin_id}: not a dictionary")
return None
# Expand x-style-elements declarations BEFORE caching, so every
# consumer (config form GET, save path, validation, defaults
# generation) sees the identical expanded shape.
schema = self._expand_style_elements(schema)
# Cache the schema
self._schema_cache[plugin_id] = schema
@@ -137,41 +132,10 @@ class SchemaManager:
self.logger.error(f"Error loading schema for {plugin_id}: {e}")
return None
# ------------------------------------------------------------------
# x-style-elements expansion
# ------------------------------------------------------------------
# Plugins declare styleable display elements compactly via an
# "x-style-elements" object on their customization schema; load_schema
# expands each declaration into full font/font_size/text_color/offset
# property blocks before caching, so the config form, save path,
# validation, and defaults generation all see the same expanded shape.
# The single implementation lives in src.element_style (pure, also used
# by plugins reading their own schema file) — see that module for the
# declaration format.
@staticmethod
def get_style_elements(schema: Dict[str, Any]) -> Dict[str, Any]:
"""The x-style-elements declaration from a schema ({} if none)."""
from src.element_style import get_style_elements
return get_style_elements(schema)
def _expand_style_elements(self, schema: Dict[str, Any]) -> Dict[str, Any]:
"""Expand x-style-elements declarations (no-op without any).
Never raises; on any failure the original schema is returned so a
malformed declaration can't take a plugin down.
"""
try:
from src.element_style import expand_style_elements
return expand_style_elements(schema)
except Exception as e:
self.logger.error(f"x-style-elements expansion failed: {e}")
return schema
def invalidate_cache(self, plugin_id: Optional[str] = None) -> None:
"""
Invalidate schema cache for a plugin or all plugins.
Args:
plugin_id: Plugin identifier to invalidate, or None to clear all
"""
+93 -9
View File
@@ -142,9 +142,28 @@ class PluginStoreManager:
# then get the result from the warm cache (double-checked locking).
self._registry_fetch_lock = threading.Lock()
# Per-plugin locks for _reinstall_with_rollback: the web UI runs
# Flask with threaded=True, so two overlapping requests for the
# same plugin_id (double-click, two browser tabs) would otherwise
# both rename the same directory aside — one succeeds, and the
# loser can end up renaming the winner's in-progress install aside
# mid-download, stealing its own rollback safety net. Keyed by
# plugin_id so unrelated plugins still update concurrently.
self._reinstall_locks: Dict[str, threading.Lock] = {}
self._reinstall_locks_guard = threading.Lock()
# Ensure plugins directory exists
self.plugins_dir.mkdir(exist_ok=True)
def _get_reinstall_lock(self, plugin_id: str) -> threading.Lock:
"""Lazily create (or fetch) the per-plugin reinstall lock."""
with self._reinstall_locks_guard:
lock = self._reinstall_locks.get(plugin_id)
if lock is None:
lock = threading.Lock()
self._reinstall_locks[plugin_id] = lock
return lock
def _record_cache_backoff(self, cache_dict: Dict, cache_key: str,
cache_timeout: int, payload: Any) -> None:
"""Bump a cache entry's timestamp so subsequent lookups hit the
@@ -2263,6 +2282,74 @@ class PluginStoreManager:
self.logger.error(f"Error uninstalling plugin {plugin_id}: {e}")
return False
def _reinstall_with_rollback(self, plugin_id: str, plugin_path: Path) -> bool:
"""Replace an installed plugin with a fresh install, atomically.
The old install is renamed aside (not deleted) until the new install
succeeds, then removed; on ANY install failure the old directory is
restored. This is the difference between a failed update and a
destroyed plugin: the previous delete-then-install flow permanently
removed plugins whenever the download failed mid-update (seen in the
field during the monorepo migration on a Pi with broken DNS every
old-remote plugin was deleted and none could be re-downloaded).
The aside name embeds '.standalone-backup-' so plugin discovery
(plugin_manager._scan_directory_for_plugins) ignores it even though
it still contains a manifest.json.
Held for the whole operation under a per-plugin_id lock: two
overlapping requests for the same plugin (double-click, two
browser tabs the web UI runs Flask with threaded=True) must not
interleave their renames, or the second could steal the first's
rollback safety net mid-install. Other plugin_ids are unaffected.
"""
with self._get_reinstall_lock(plugin_id):
backup_path = plugin_path.with_name(
f"{plugin_path.name}.standalone-backup-migrating")
# A stale aside from a previous crash would block the rename
if backup_path.exists():
if not self._safe_remove_directory(backup_path):
self.logger.error(
f"Could not clear stale backup for {plugin_id} at "
f"{backup_path}; leaving old install in place")
return False
try:
plugin_path.rename(backup_path)
except OSError as e:
self.logger.error(
f"Could not set aside old plugin directory for {plugin_id}: {e}")
return False
try:
installed = self.install_plugin(plugin_id)
except Exception as e:
self.logger.error(f"Reinstall of {plugin_id} raised: {e}")
installed = False
if installed:
if not self._safe_remove_directory(backup_path):
self.logger.warning(
f"Update of {plugin_id} succeeded but the old backup "
f"at {backup_path} could not be removed; it will be "
f"cleared on the next update")
return True
# Install failed (bad network, registry error...) — put the old
# version back so the user still has a working plugin.
self.logger.error(
f"Reinstall of {plugin_id} failed; restoring previous version")
try:
if plugin_path.exists():
# partial download debris from the failed install
self._safe_remove_directory(plugin_path)
backup_path.rename(plugin_path)
self.logger.info(f"Restored previous install of {plugin_id}")
except OSError as e:
self.logger.error(
f"CRITICAL: could not restore {plugin_id} from {backup_path}: {e}. "
f"The previous install is preserved there — rename it back manually.")
return False
def update_plugin(self, plugin_id: str) -> bool:
"""
Update a plugin to the latest commit on its upstream branch.
@@ -2325,10 +2412,7 @@ class PluginStoreManager:
f"Plugin {resolved_id} git remote ({local_remote}) differs from registry ({registry_repo}). "
f"Reinstalling from registry to migrate to new source."
)
if not self._safe_remove_directory(plugin_path):
self.logger.error(f"Failed to remove old plugin directory for {resolved_id}")
return False
return self.install_plugin(resolved_id)
return self._reinstall_with_rollback(resolved_id, plugin_path)
# Check if already up to date
if remote_sha and local_sha and remote_sha.startswith(local_sha):
@@ -2632,11 +2716,11 @@ class PluginStoreManager:
# Plugin is not a git repo but is in registry and has a newer version - reinstall
self.logger.info(f"Plugin {plugin_id} not installed via git; re-installing latest archive (registry id: {registry_id})")
# Remove directory and reinstall fresh
if not self._safe_remove_directory(plugin_path):
self.logger.error(f"Failed to remove old plugin directory for {plugin_id}")
return False
return self.install_plugin(registry_id)
# Reinstall with the old version kept aside until the new
# download succeeds — this is the path every routine store
# update takes, and a mid-update network failure must not
# destroy the user's plugin.
return self._reinstall_with_rollback(registry_id, plugin_path)
except Exception as e:
import traceback
+24
View File
@@ -88,3 +88,27 @@ def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
with open(mock_path, 'r') as mf:
spec['mock_data_contents'] = json.load(mf)
return spec
def build_full_config(
plugin_dir: Union[str, Path],
spec: Optional[Dict[str, Any]] = None,
cli_config: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""Build the config a plugin sees under test.
Merge order: config_schema.json defaults, then a forced ``enabled: True``,
then harness.json's config overlay, then the caller's explicit config --
most specific wins. `enabled` is re-asserted *after* the schema defaults
so a plugin that reasonably ships `enabled: false` (e.g. a seasonal or
opt-in plugin) can't silently make every harness run test "disabled, do
nothing" by accident -- callers that genuinely want to test the disabled
path can still do so via `cli_config={"enabled": False}`.
"""
spec = spec or {}
config: Dict[str, Any] = {}
config.update(load_config_defaults(plugin_dir))
config["enabled"] = True
config.update(spec.get("config", {}))
config.update(cli_config or {})
return config
+20
View File
@@ -71,6 +71,7 @@ class MockCacheManager:
self.get_calls = []
self.set_calls = []
self.delete_calls = []
self.get_cached_data_with_strategy_calls = []
# Real temp dir for plugins that write/read files under cache_dir.
# Registered for cleanup so each mock instance doesn't leak a tmp dir.
self.cache_dir = tempfile.mkdtemp(prefix="ledmatrix-mock-cache-")
@@ -108,6 +109,24 @@ class MockCacheManager:
self.delete_calls.append(key)
if key in self._cache:
del self._cache[key]
def get_cached_data_with_strategy(self, key: str, data_type: str = 'default') -> Optional[Any]:
"""Mock of CacheManager.get_cached_data_with_strategy (src/cache_manager.py).
The real method picks a max_age/memory_ttl strategy per data_type
(and extends it during market-closed hours for market data) before
delegating to get_cached_data(). None of that timing nuance matters
for a mock -- plugins under test just need the method to exist and
return whatever was cached, so this delegates straight to get().
"""
self.get_cached_data_with_strategy_calls.append({'key': key, 'data_type': data_type})
return self.get(key)
def save_cache(self, key: str, data: Any) -> None:
"""Mock of CacheManager.save_cache (src/cache_manager.py) -- the
write-side counterpart to get_cached_data_with_strategy, used by the
same real-CacheManager-oriented plugins. Delegates to set()."""
self.set(key, data)
if key in self._cache_timestamps:
del self._cache_timestamps[key]
@@ -118,6 +137,7 @@ class MockCacheManager:
self.get_calls = []
self.set_calls = []
self.delete_calls = []
self.get_cached_data_with_strategy_calls = []
class MockConfigManager:
-121
View File
@@ -1,121 +0,0 @@
"""Headless single-render service for plugins.
Renders one plugin instance at one panel size to an in-memory PIL image
no hardware, no singletons, no pip (install_deps is always False). Shared
by the dev server's /api/render endpoints and the production web UI's
config-page live preview.
A fresh plugin instance is created per call (mirroring the safety
harness), so repeated renders never share instance state. The plugin's
module does stay imported in the process module-level globals persist
across calls, which is fine for previewing but worth knowing.
"""
import json
import logging
import time
from pathlib import Path
from typing import Any, Dict, Optional
logger = logging.getLogger(__name__)
def render_plugin_once(plugin_id: str, plugin_dir: Path,
manifest: Optional[Dict[str, Any]] = None,
config: Optional[Dict[str, Any]] = None,
mock_data: Optional[Dict[str, Any]] = None,
width: int = 128, height: int = 32,
skip_update: bool = True) -> Dict[str, Any]:
"""Render one plugin at one size. Returns a response-shaped dict:
{'image': 'data:image/png;base64,...', 'width', 'height',
'render_time_ms', 'errors': [...], 'warnings': [...]}
``skip_update`` defaults to True: update() may block on live network
(sports APIs, Spotify) callers that want real data should prime
``mock_data`` (e.g. from the plugin's test/harness.json fixture, see
``load_harness_spec``) or explicitly pass skip_update=False.
Raises on plugin load failure; update()/display() exceptions are
captured into warnings/errors instead so a broken render still shows
whatever was drawn.
"""
from src.plugin_system.plugin_loader import PluginLoader
from src.plugin_system.testing import (
MockCacheManager, MockPluginManager, VisualTestDisplayManager)
plugin_dir = Path(plugin_dir)
if manifest is None:
with open(plugin_dir / 'manifest.json', 'r', encoding='utf-8') as f:
manifest = json.load(f)
config = config or {'enabled': True}
mock_data = mock_data or {}
display_manager = VisualTestDisplayManager(width=width, height=height)
cache_manager = MockCacheManager()
plugin_manager = MockPluginManager()
# Pre-populate cache with mock data
for key, value in mock_data.items():
cache_manager.set(key, value)
loader = PluginLoader()
errors = []
warnings = []
plugin_instance, _module = loader.load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=plugin_dir,
config=config,
display_manager=display_manager,
cache_manager=cache_manager,
plugin_manager=plugin_manager,
install_deps=False,
)
start_time = time.time()
# Exception detail policy (matches the dev server's convention): full
# tracebacks go to the server log; clients get only the exception class
# name — CodeQL flags raw exception text in responses as stack-trace
# exposure, and the log is where developers look anyway.
try:
if not skip_update:
try:
plugin_instance.update()
except Exception as e:
logger.warning("update() raised for plugin %s", plugin_id,
exc_info=True)
warnings.append(f"update() raised: {type(e).__name__} — see server log")
try:
plugin_instance.display(force_clear=True)
except Exception as e:
logger.warning("display() raised for plugin %s", plugin_id,
exc_info=True)
errors.append(f"display() raised: {type(e).__name__} — see server log")
render_time_ms = round((time.time() - start_time) * 1000, 1)
# Capture BEFORE cleanup — a plugin's cleanup may clear the canvas
image_b64 = display_manager.get_image_base64()
finally:
# The instance is throwaway, but its __init__ may have opened
# sessions or started threads (music's clients, sports API
# sessions). In a long-running web process, previews without
# cleanup would leak those per request.
try:
plugin_instance.cleanup()
except Exception as e:
logger.warning("cleanup() raised for plugin %s", plugin_id,
exc_info=True)
warnings.append(f"cleanup() raised: {type(e).__name__} — see server log")
return {
'image': f'data:image/png;base64,{image_b64}',
'width': width,
'height': height,
'render_time_ms': render_time_ms,
'errors': errors,
'warnings': warnings,
}
+24 -4
View File
@@ -66,6 +66,10 @@ class RenderPipeline:
else display_manager.height
)
# Reusable blank frame for cycle-end pushes (allocated lazily,
# re-blacked before each reuse)
self._blank_frame = None
# ScrollHelper for optimized scrolling
self.scroll_helper = ScrollHelper(
self.display_width,
@@ -234,11 +238,19 @@ class RenderPipeline:
)
# Push blank immediately so the hardware never shows any
# post-wrap content while the coordinator recomposes the
# next cycle (~100 ms).
# next cycle (~100 ms). The blank is allocated once and
# reused across cycle wraps (fresh paste each time in case
# a consumer drew on the previous one).
try:
from PIL import Image as _Image
blank = _Image.new('RGB', (self.display_width, self.display_height))
self.display_manager.image = blank
if self._blank_frame is None or self._blank_frame.size != (
self.display_width, self.display_height):
self._blank_frame = Image.new(
'RGB', (self.display_width, self.display_height))
else:
self._blank_frame.paste(
(0, 0, 0),
(0, 0, self.display_width, self.display_height))
self.display_manager.image = self._blank_frame
self.display_manager.update_display()
except Exception:
logger.exception("Failed to write blank frame to display at cycle end")
@@ -297,6 +309,8 @@ class RenderPipeline:
Returns True when:
- Cycle is complete and we should start fresh
- Staging buffer has new content
- A plugin currently visible in the scroll has pending updated data
(e.g. a live score changed) standalone (non-sync) mode only
"""
if self._cycle_complete:
return True
@@ -314,6 +328,12 @@ class RenderPipeline:
if buffer_status['staging_count'] > 0:
return True
# Trigger recompose when pending updates affect visible segments, so
# live score/status changes reach the display within a few seconds
# instead of waiting for the next full cycle.
if self.stream_manager.has_pending_updates_for_visible_segments():
return True
return False
def hot_swap_content(self) -> bool:
+24
View File
@@ -2564,11 +2564,35 @@ address=/detectportal.firefox.com/192.168.4.1
Returns:
True if AP mode state changed, False otherwise
"""
changed, _status, _ethernet, _ap = self.check_and_manage_ap_mode_with_state()
return changed
def check_and_manage_ap_mode_with_state(self) -> Tuple[bool, WiFiStatus, bool, bool]:
"""Like check_and_manage_ap_mode, but also returns the state it
observed, so callers (the wifi monitor daemon) don't have to re-run
the same nmcli subprocess battery before AND after the check
each status fetch is several process forks.
Returns:
(state_changed, WiFiStatus, ethernet_connected, ap_active_after)
"""
try:
# Get status with retry for more reliable detection
status = self._get_wifi_status_with_retry()
ethernet_connected = self._is_ethernet_connected()
ap_active = self._is_ap_mode_active()
changed = self._manage_ap_mode(status, ethernet_connected, ap_active)
# State only ever changes via one enable or one disable, so the
# post-state is the inverse of the pre-state when changed.
ap_after = (not ap_active) if changed else ap_active
return changed, status, ethernet_connected, ap_after
except Exception as e:
logger.error(f"Error checking AP mode: {e}", exc_info=True)
return False, WiFiStatus(connected=False), False, False
def _manage_ap_mode(self, status: WiFiStatus, ethernet_connected: bool, ap_active: bool) -> bool:
"""AP-mode decision logic against an already-fetched state snapshot."""
try:
auto_enable = self.config.get("auto_enable_ap_mode", True) # Default: True (safe due to grace period)
# Log current state for debugging
+5 -1
View File
@@ -38,7 +38,11 @@ def mock_cache_manager():
mock._memory_cache_timestamps = {}
mock.cache_dir = "/tmp/test_cache"
def mock_get(key: str, max_age: int = 300) -> Optional[Dict]:
def mock_get(key: str, max_age: Optional[int] = 300,
memory_ttl: Optional[int] = None) -> Optional[Dict]:
# Signature mirrors CacheManager.get — keep in sync or callers
# passing keyword args (health tracker, resource monitor) break
# only in tests, hiding real-API compatibility.
return mock._memory_cache.get(key)
def mock_set(key: str, data: Dict, ttl: Optional[int] = None) -> None:
+58
View File
@@ -253,3 +253,61 @@ class TestCheckPluginHonorsHarnessJson:
)
assert captured["freeze_time"] == "2030-01-01 00:00:00"
assert captured["config"]["timezone"] == "America/New_York"
class TestBuildFullConfigForcesEnabled:
"""Regression: a plugin's own config_schema.json may reasonably default
enabled to False (e.g. a seasonal or opt-in plugin) -- march-madness and
14 other real plugins do. The harness must still test it as enabled
unless a caller explicitly asks otherwise, or every render silently
becomes a same-shaped "disabled, do nothing" no-op."""
def _make_plugin_with_disabled_default(self, tmp_path):
pdir = tmp_path / "plugins" / "demo-seasonal"
pdir.mkdir(parents=True)
(pdir / "manifest.json").write_text(json.dumps({
"id": "demo-seasonal", "name": "Demo Seasonal", "version": "1.0.0",
"author": "test", "entry_point": "manager.py",
"class_name": "DemoSeasonal", "display_modes": ["demo-seasonal"],
"compatible_versions": ["*"],
}))
(pdir / "config_schema.json").write_text(json.dumps({
"type": "object",
"properties": {"enabled": {"type": "boolean", "default": False}},
}))
return pdir
def test_schema_disabled_default_does_not_win(self, tmp_path):
from src.plugin_system.testing.loading import build_full_config
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
config = build_full_config(plugin_dir)
assert config["enabled"] is True
def test_harness_json_config_can_still_disable(self, tmp_path):
from src.plugin_system.testing.loading import build_full_config
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
config = build_full_config(plugin_dir, spec={"config": {"enabled": False}})
assert config["enabled"] is False
def test_explicit_cli_config_can_still_disable(self, tmp_path):
from src.plugin_system.testing.loading import build_full_config
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
config = build_full_config(plugin_dir, cli_config={"enabled": False})
assert config["enabled"] is False
def test_check_one_renders_a_schema_disabled_plugin_as_enabled(self, tmp_path, monkeypatch):
"""End-to-end: check_plugin.py's check_one() must not blank-render a
plugin just because its own schema defaults enabled to False."""
mod = _load_check_plugin_cli()
plugin_dir = self._make_plugin_with_disabled_default(tmp_path)
captured = {}
monkeypatch.setattr(mod, "render_plugin_matrix",
lambda **kw: captured.update(kw) or [])
monkeypatch.setattr(mod, "compare_to_goldens", lambda *a, **k: [])
mod.check_one(
plugin_id="demo-seasonal", search_dirs=[str(tmp_path / "plugins")],
sizes=None, mock_data={}, config={}, run_update=True,
out_dir=None, update_golden=False, golden_dir_override=None,
freeze_time=None,
)
assert captured["config"]["enabled"] is True
+2 -4
View File
@@ -22,7 +22,7 @@ import pytest
from src.plugin_system.testing.harness import (
render_plugin_matrix, compare_to_goldens,
)
from src.plugin_system.testing.loading import load_config_defaults, load_harness_spec
from src.plugin_system.testing.loading import build_full_config, load_harness_spec
from src.plugin_system.testing.sizes import resolve_test_sizes
PROJECT_ROOT = Path(__file__).resolve().parents[2]
@@ -81,9 +81,7 @@ def test_plugin_renders_across_sizes_and_screens(plugin_id: str) -> None:
plugin_dir = _PLUGINS[plugin_id]
spec = load_harness_spec(plugin_dir)
config = {"enabled": True}
config.update(load_config_defaults(plugin_dir))
config.update(spec.get("config", {}))
config = build_full_config(plugin_dir, spec)
# Sizes: LEDMATRIX_TEST_SIZES env (test on real hardware) wins, then the
# plugin's own harness.json "sizes", else the default representative sample.
+270
View File
@@ -0,0 +1,270 @@
"""Tests for asynchronous plugin updates (plugin_manager background worker).
The invariants that keep this change safe:
1. run_scheduled_updates returns immediately a slow update() can never
again freeze the render loop (the original defect: 30s scroll freezes).
2. A plugin's update() and display() are NEVER concurrent — the per-plugin
lock makes the old implicit no-overlap guarantee explicit, and it stays
held through PluginExecutor's own timeout: a lingering, still-running
update() keeps the lock even after PluginExecutor gives up waiting on it.
3. Failure/timeout bookkeeping is unchanged (same executor, same
_record_update_failure path, same last-update stamping).
4. The kill switch (plugin_system.synchronous_updates) restores the
inline path exactly.
"""
import os
import sys
import threading
import time
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
from src.plugin_system.plugin_state import PluginState # noqa: E402
class SlowPlugin:
"""Fake plugin whose update() sleeps and records overlap violations."""
def __init__(self, update_seconds=0.5):
self.enabled = True
self.update_seconds = update_seconds
self.update_calls = 0
self.display_calls = 0
self.in_update = False
self.overlap_detected = False
def update(self):
self.in_update = True
self.update_calls += 1
time.sleep(self.update_seconds)
self.in_update = False
return True
def display(self, force_clear=False):
if self.in_update:
self.overlap_detected = True
self.display_calls += 1
return True
@pytest.fixture
def pm(tmp_path):
manager = PluginManager(plugins_dir=str(tmp_path), config_manager=None,
display_manager=None, cache_manager=None)
yield manager
manager.stop_update_worker()
def _install(pm, plugin, plugin_id="slow-plugin"):
pm.plugins[plugin_id] = plugin
pm._update_interval_cache[plugin_id] = 0.01 # always due
# load_plugin normally registers state; can_execute() gates on it
pm.state_manager.set_state(plugin_id, PluginState.ENABLED)
return plugin_id
class TestSchedulerNonBlocking:
def test_run_scheduled_updates_returns_immediately(self, pm):
plugin_id = _install(pm, SlowPlugin(update_seconds=2.0))
start = time.monotonic()
pm.run_scheduled_updates()
elapsed = time.monotonic() - start
assert elapsed < 0.1, f"scheduler blocked for {elapsed:.2f}s"
# the update actually runs in the background
deadline = time.monotonic() + 5
while pm.plugins[plugin_id].update_calls == 0 and time.monotonic() < deadline:
time.sleep(0.05)
assert pm.plugins[plugin_id].update_calls == 1
def test_no_double_enqueue_while_pending(self, pm):
plugin_id = _install(pm, SlowPlugin(update_seconds=0.8))
for _ in range(20):
pm.run_scheduled_updates()
time.sleep(0.01)
time.sleep(1.5) # let the single queued update finish
assert pm.plugins[plugin_id].update_calls == 1
class TestUpdateDisplayExclusion:
def test_display_lock_held_during_update(self, pm):
plugin_id = _install(pm, SlowPlugin(update_seconds=0.6))
pm.run_scheduled_updates()
# give the worker a moment to take the lock and enter update()
deadline = time.monotonic() + 2
while not pm.plugins[plugin_id].in_update and time.monotonic() < deadline:
time.sleep(0.01)
lock = pm.get_plugin_lock(plugin_id)
assert lock.acquire(blocking=False) is False, \
"lock must be held while update() runs"
# and released afterwards
deadline = time.monotonic() + 3
while pm.plugins[plugin_id].in_update and time.monotonic() < deadline:
time.sleep(0.05)
time.sleep(0.1)
assert lock.acquire(blocking=False) is True
lock.release()
def test_no_overlap_under_hammering_display_loop(self, pm):
"""Simulate the render loop's try-lock display pattern at high rate
while updates fire the plugin itself asserts no overlap."""
plugin = SlowPlugin(update_seconds=0.15)
plugin_id = _install(pm, plugin)
stop = threading.Event()
def render_loop():
while not stop.is_set():
lock = pm.get_plugin_lock(plugin_id)
if lock.acquire(blocking=False):
try:
plugin.display()
finally:
lock.release()
time.sleep(0.002)
renderer = threading.Thread(target=render_loop, daemon=True)
renderer.start()
try:
for _ in range(6):
pm.plugin_last_update.pop(plugin_id, None) # force due
pm.run_scheduled_updates()
time.sleep(0.3)
finally:
stop.set()
renderer.join(timeout=2)
assert plugin.update_calls >= 3
assert plugin.display_calls > 10
assert plugin.overlap_detected is False
def test_lock_held_through_timeout_until_real_update_finishes(self, pm):
"""PluginExecutor's join(timeout) can return before the real
update() call does -- the lingering daemon thread keeps running it.
The plugin's lock must stay held for that whole real duration, not
just PluginExecutor's bounded wait, or display() could run
concurrently with a still-executing update()."""
plugin = SlowPlugin(update_seconds=0.3)
plugin_id = _install(pm, plugin)
pm.plugin_executor.default_timeout = 0.05 # times out well before
# update_seconds elapses
pm.run_scheduled_updates()
deadline = time.monotonic() + 2
while not plugin.in_update and time.monotonic() < deadline:
time.sleep(0.01)
assert plugin.in_update is True
# PluginExecutor's own timeout has now elapsed, but the real
# update() (0.3s) is still running in its lingering daemon thread.
time.sleep(0.15)
assert plugin.in_update is True, "test setup: update should still be running"
lock = pm.get_plugin_lock(plugin_id)
assert lock.acquire(blocking=False) is False, \
"lock must stay held through PluginExecutor's timeout while the real update() runs"
# Once the real update() genuinely finishes, the lock is released.
deadline = time.monotonic() + 2
while plugin.in_update and time.monotonic() < deadline:
time.sleep(0.02)
time.sleep(0.1)
assert lock.acquire(blocking=False) is True
lock.release()
def test_state_returns_to_enabled_after_update(self, pm):
"""RUNNING is set at enqueue (blocks re-entry via can_execute) and
must return to an executable state once the update finishes."""
plugin_id = _install(pm, SlowPlugin(update_seconds=0.1))
pm.run_scheduled_updates()
# while queued/running, re-entry is blocked
assert pm.state_manager.can_execute(plugin_id) is False
deadline = time.monotonic() + 3
while time.monotonic() < deadline:
if (pm.plugins[plugin_id].update_calls
and pm.state_manager.can_execute(plugin_id)):
break
time.sleep(0.05)
assert pm.plugins[plugin_id].update_calls == 1
assert pm.state_manager.can_execute(plugin_id) is True
class TestFailurePaths:
def test_update_failure_routes_through_failure_bookkeeping(self, pm):
class FailingPlugin(SlowPlugin):
def update(self):
self.update_calls += 1
raise RuntimeError("boom")
plugin_id = _install(pm, FailingPlugin())
pm.run_scheduled_updates()
deadline = time.monotonic() + 3
while pm.plugins[plugin_id].update_calls == 0 and time.monotonic() < deadline:
time.sleep(0.05)
time.sleep(0.2)
# failure stamped so the interval gate holds (no hot retry loop)
assert pm.plugin_last_update.get(plugin_id, 0) > 0
# lock released after failure
assert pm.get_plugin_lock(plugin_id).acquire(blocking=False) is True
pm.get_plugin_lock(plugin_id).release()
def test_unloaded_while_queued_is_harmless(self, pm):
"""Exercise the public unload_plugin() lifecycle rather than
deleting pm.plugins directly: queue the target's update behind a
deterministic blocker (occupying the single worker), unload the
target while its item still sits queued, then release the blocker
and confirm the target's update never ran and its state stayed
unloaded rather than being resurrected to ENABLED."""
blocker_event = threading.Event()
class BlockerPlugin(SlowPlugin):
def update(self):
self.update_calls += 1
blocker_event.wait(timeout=5)
return True
blocker_id = _install(pm, BlockerPlugin(), plugin_id="blocker-plugin")
target = SlowPlugin(update_seconds=0.05)
target_id = _install(pm, target, plugin_id="slow-plugin")
# Dispatch the blocker first so it occupies the single worker
# thread, then enqueue the target behind it -- deterministically
# queued, not yet started.
pm._enqueue_update(blocker_id, time.time())
deadline = time.monotonic() + 2
while pm.plugins[blocker_id].update_calls == 0 and time.monotonic() < deadline:
time.sleep(0.01)
assert pm.plugins[blocker_id].update_calls == 1
pm._enqueue_update(target_id, time.time())
assert target_id in pm._pending_updates
assert pm.unload_plugin(target_id) is True
assert target_id not in pm.plugins
blocker_event.set() # let the blocker finish; worker moves on to
# the target's queued item
deadline = time.monotonic() + 3
while target_id in pm._pending_updates and time.monotonic() < deadline:
time.sleep(0.02)
assert target.update_calls == 0, "update() must not run for an unloaded plugin"
assert target_id not in pm._pending_updates
assert pm.state_manager.get_state(target_id) == PluginState.UNLOADED
class TestKillSwitch:
def test_synchronous_mode_blocks_like_before(self, pm):
pm._synchronous_updates = True
plugin_id = _install(pm, SlowPlugin(update_seconds=0.4))
start = time.monotonic()
pm.run_scheduled_updates()
elapsed = time.monotonic() - start
assert elapsed >= 0.4, "synchronous mode must run inline"
assert pm.plugins[plugin_id].update_calls == 1
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+58
View File
@@ -400,3 +400,61 @@ class TestDiskCache:
assert stats['fetch_count'] == 3
assert stats['total_fetch_time'] == 1.8
assert stats['average_fetch_time'] == pytest.approx(0.6, abs=0.01)
class TestDiskCacheWriteEconomy:
"""SD-card wear guards: identical payloads skip the disk, files are
compact, and TTL semantics survive the skip (see PR: fix/diskcache-sd-wear)."""
def test_identical_set_skips_rewrite(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v"})
path = cache.get_cache_path("k")
first = os.stat(path)
os.utime(path, (first.st_atime - 100, first.st_mtime - 100)) # age it
aged_mtime = os.stat(path).st_mtime
ino_before = os.stat(path).st_ino
cache.set("k", {"data": "v"}) # identical payload
after = os.stat(path)
# mtime refreshed (TTL for mtime-based records preserved)...
assert after.st_mtime > aged_mtime
# ...but the file was NOT rewritten (same inode: no replace happened)
assert after.st_ino == ino_before
def test_changed_data_rewrites(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v1"})
cache.set("k", {"data": "v2"})
assert cache.get("k") == {"data": "v2"}
def test_clear_resets_digest(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v"})
cache.clear("k")
assert cache.get("k") is None
cache.set("k", {"data": "v"}) # same payload after clear must WRITE
assert cache.get("k") == {"data": "v"}
def test_skip_self_heals_when_file_deleted_externally(self, tmp_path):
import os
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"data": "v"})
os.remove(cache.get_cache_path("k")) # e.g. expiry cleanup
cache.set("k", {"data": "v"}) # digest matches but file is gone
assert cache.get("k") == {"data": "v"}
def test_files_are_compact_json(self, tmp_path):
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"a": 1, "b": [1, 2, 3]})
raw = open(cache.get_cache_path("k")).read()
assert "\n" not in raw.strip() # no indent
assert cache.get("k") == {"a": 1, "b": [1, 2, 3]}
def test_datetime_round_trip_still_works(self, tmp_path):
from datetime import datetime
cache = DiskCache(cache_dir=str(tmp_path))
cache.set("k", {"when": datetime(2026, 7, 12, 10, 30)})
assert cache.get("k") == {"when": "2026-07-12T10:30:00"}
+129
View File
@@ -0,0 +1,129 @@
"""Tests for load_config's mtime fast path (src/config_manager.py).
load_config used to re-read + re-parse config.json, the template (with a
recursive migration diff) and secrets on EVERY call ~30 web request
handlers call it, some 2-3x per request. The fast path skips all of it
when the three files' (mtime_ns, size) signatures are unchanged.
The invariant that matters most: cross-process freshness a save from
the web process must be picked up by the display process's next load.
That's guaranteed because the signature is re-stat'd on every call.
"""
import json
import os
import sys
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.config_manager import ConfigManager # noqa: E402
@pytest.fixture
def mgr(tmp_path):
config = tmp_path / "config.json"
secrets = tmp_path / "secrets.json"
template = tmp_path / "template.json"
config.write_text(json.dumps({"display": {"brightness": 90}, "timezone": "UTC"}))
secrets.write_text(json.dumps({"weather": {"api_key": "sek"}}))
template.write_text(json.dumps({"display": {"brightness": 90}, "timezone": "UTC"}))
m = ConfigManager(config_path=str(config), secrets_path=str(secrets))
m.template_path = str(template)
return m, config, secrets, template
def _count_opens(monkeypatch, mgr_paths):
"""Count open() calls hitting the config files."""
counts = {"n": 0}
real_open = open
def counting_open(file, *args, **kwargs):
if str(file) in mgr_paths:
counts["n"] += 1
return real_open(file, *args, **kwargs)
import builtins
monkeypatch.setattr(builtins, "open", counting_open)
return counts
class TestFastPath:
def test_unchanged_files_are_not_reread(self, mgr, monkeypatch):
m, config, secrets, template = mgr
first = m.load_config()
assert first["weather"]["api_key"] == "sek" # secrets merged
counts = _count_opens(monkeypatch, {str(config), str(secrets), str(template)})
for _ in range(10):
again = m.load_config()
assert counts["n"] == 0, "fast path must not re-open any config file"
assert again is first # same aliasing semantics as the full path
def test_config_change_triggers_reload(self, mgr):
m, config, secrets, template = mgr
m.load_config()
data = json.loads(config.read_text())
data["display"]["brightness"] = 55
config.write_text(json.dumps(data))
os.utime(config, (os.stat(config).st_atime, os.stat(config).st_mtime + 2))
assert m.load_config()["display"]["brightness"] == 55
def test_secrets_change_triggers_reload(self, mgr):
m, config, secrets, template = mgr
m.load_config()
secrets.write_text(json.dumps({"weather": {"api_key": "NEW"}}))
os.utime(secrets, (os.stat(secrets).st_atime, os.stat(secrets).st_mtime + 2))
assert m.load_config()["weather"]["api_key"] == "NEW"
def test_template_change_triggers_reload_and_migration(self, mgr):
m, config, secrets, template = mgr
m.load_config()
template.write_text(json.dumps({
"display": {"brightness": 90}, "timezone": "UTC",
"brand_new_key": {"added": True}}))
os.utime(template, (os.stat(template).st_atime, os.stat(template).st_mtime + 2))
reloaded = m.load_config()
assert reloaded.get("brand_new_key") == {"added": True}
def test_same_second_edit_detected_via_mtime_ns_or_size(self, mgr):
"""Coarse-mtime same-second edits: size difference still busts it."""
m, config, secrets, template = mgr
m.load_config()
st = os.stat(config)
data = json.loads(config.read_text())
data["timezone"] = "America/New_York" # different byte length
config.write_text(json.dumps(data))
os.utime(config, (st.st_atime, st.st_mtime)) # force same mtime
assert m.load_config()["timezone"] == "America/New_York"
class TestSaveCoherence:
def test_save_config_then_load_returns_saved_data(self, mgr, monkeypatch):
m, config, secrets, template = mgr
m.load_config()
new = {"display": {"brightness": 42}, "timezone": "UTC",
"weather": {"api_key": "sek"}}
m.save_config(new)
counts = _count_opens(monkeypatch, {str(config), str(secrets), str(template)})
loaded = m.load_config()
assert loaded["display"]["brightness"] == 42
assert loaded["weather"]["api_key"] == "sek" # secrets survive in memory
assert counts["n"] == 0 # signature refreshed by save; no re-read
def test_cross_process_save_is_picked_up(self, mgr):
"""Another process writing config.json (different mtime) must bust
this process's fast path — the core cross-process guarantee."""
m, config, secrets, template = mgr
m.load_config()
other = ConfigManager(config_path=str(config), secrets_path=str(secrets))
other.template_path = str(template)
other.load_config()
other.save_config({"display": {"brightness": 11}, "timezone": "UTC",
"weather": {"api_key": "sek"}})
os.utime(config, (os.stat(config).st_atime, os.stat(config).st_mtime + 2))
assert m.load_config()["display"]["brightness"] == 11
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
@@ -0,0 +1,88 @@
"""
Regression tests for DisplayController._tick_plugin_updates_for_vegas().
PR #299 added logic to detect which plugins actually got fresh data on a
scheduled-update tick and notify Vegas mode via
vegas_coordinator.mark_plugin_updated(), so a live score change reaches the
scroll within seconds instead of waiting for a full cycle. PR #330's
multi-display sync refactor deleted this method (folding the callback back
to the plain _tick_plugin_updates(), which reports nothing), silently
orphaning VegasModeCoordinator.mark_plugin_updated() -- it has had zero
callers since.
"""
from typing import Dict, List, Optional
from unittest.mock import MagicMock
from src.display_controller import DisplayController
def _make_controller(updated: Optional[List[str]] = None, vegas_coordinator: Optional[MagicMock] = None) -> DisplayController:
dc = object.__new__(DisplayController)
dc.plugin_manager = MagicMock()
dc.plugin_manager.run_scheduled_updates_with_changes.return_value = list(updated or [])
dc.vegas_coordinator = vegas_coordinator
return dc
class TestTickPluginUpdatesForVegas:
def test_marks_only_plugins_whose_timestamp_advanced(self):
vc = MagicMock()
dc = _make_controller(updated=["stock-news"], vegas_coordinator=vc)
dc._tick_plugin_updates_for_vegas()
vc.mark_plugin_updated.assert_called_once_with("stock-news")
def test_no_advance_marks_nothing(self):
vc = MagicMock()
dc = _make_controller(updated=[], vegas_coordinator=vc)
dc._tick_plugin_updates_for_vegas()
vc.mark_plugin_updated.assert_not_called()
def test_no_vegas_coordinator_does_not_raise(self):
dc = _make_controller(updated=["stock-news"], vegas_coordinator=None)
dc._tick_plugin_updates_for_vegas() # must not raise
def test_mark_plugin_updated_exception_does_not_propagate(self):
"""One plugin's mark_plugin_updated failing must not stop the tick
or crash the update loop it runs in."""
vc = MagicMock()
vc.mark_plugin_updated.side_effect = [RuntimeError("boom"), None]
dc = _make_controller(updated=["a", "b"], vegas_coordinator=vc)
dc._tick_plugin_updates_for_vegas() # must not raise
assert vc.mark_plugin_updated.call_count == 2
class TestVegasCoordinatorCallbackWiring:
def test_initialize_wires_vegas_aware_tick_as_update_callback(self):
"""The Vegas coordinator must be given the Vegas-aware
_tick_plugin_updates_for_vegas as its update callback, not the plain
_tick_plugin_updates() -- that's the exact wiring PR #330 dropped."""
dc = object.__new__(DisplayController)
dc.config = {"display": {"vegas_scroll": {"enabled": True}}, "sync": {}}
dc.display_manager = MagicMock()
dc.plugin_manager = MagicMock()
dc.sync_manager = MagicMock()
dc._check_live_priority = MagicMock()
dc._check_vegas_interrupt = MagicMock(return_value=False)
fake_coordinator = MagicMock()
import src.display_controller as dc_module
original_imported = dc_module._vegas_mode_imported
original_class = dc_module.VegasModeCoordinator
try:
dc_module._vegas_mode_imported = True
dc_module.VegasModeCoordinator = MagicMock(return_value=fake_coordinator)
dc._initialize_vegas_mode()
finally:
dc_module._vegas_mode_imported = original_imported
dc_module.VegasModeCoordinator = original_class
fake_coordinator.set_update_callback.assert_called_once_with(dc._tick_plugin_updates_for_vegas)
+162
View File
@@ -0,0 +1,162 @@
"""Tests for update_display dirty tracking (src/display_manager.py).
Runs against RGBMatrixEmulator (EMULATOR=true), exercising the REAL
DisplayManager not a mock so the skip logic, its invalidation hooks,
and the kill switch are verified off-Pi.
The invariants:
- identical frames are pushed exactly once (SwapOnVSync not re-called)
- ANY pixel change pushes
- clear() and set_brightness() invalidate (the two paths that alter panel
state outside the digest's view)
- the kill switch (display.dirty_tracking: false) restores always-push
"""
import os
import sys
import time
os.environ["EMULATOR"] = "true"
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
@pytest.fixture(scope="module")
def dm():
"""One real DisplayManager on the emulator (it's a process singleton)."""
from src.display_manager import DisplayManager
DisplayManager._instance = None
DisplayManager._initialized = False
manager = DisplayManager({
"display": {
"hardware": {"rows": 32, "cols": 64, "chain_length": 2,
"parallel": 1, "brightness": 90},
"runtime": {"gpio_slowdown": 0},
},
}, suppress_test_pattern=True)
yield manager
DisplayManager._instance = None
DisplayManager._initialized = False
class _SwapSpy:
"""Counts SwapOnVSync calls through the real matrix object."""
def __init__(self, matrix):
self.matrix = matrix
self.count = 0
self._orig = matrix.SwapOnVSync
def __enter__(self):
def counting(canvas):
self.count += 1
return self._orig(canvas)
self.matrix.SwapOnVSync = counting
return self
def __exit__(self, *exc):
self.matrix.SwapOnVSync = self._orig
class TestDirtyTracking:
def test_identical_frames_push_once(self, dm):
dm.draw.rectangle([0, 0, 10, 10], fill=(255, 0, 0))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
dm.update_display()
assert spy.count == 1
def test_pixel_change_pushes(self, dm):
dm.update_display()
with _SwapSpy(dm.matrix) as spy:
dm.draw.point((5, 5), fill=(0, 255, 0))
dm.update_display()
dm.update_display() # unchanged again
assert spy.count == 1
def test_clear_invalidates(self, dm):
dm.draw.rectangle([0, 0, 20, 20], fill=(0, 0, 255))
dm.update_display()
dm.clear() # writes to the matrix directly; digest must reset
with _SwapSpy(dm.matrix) as spy:
dm.update_display() # black frame after clear must still push
assert spy.count == 1
def test_brightness_change_forces_push(self, dm):
dm.draw.rectangle([0, 0, 20, 20], fill=(200, 200, 200))
dm.update_display()
with _SwapSpy(dm.matrix) as spy:
dm.update_display() # identical -> skipped
assert spy.count == 0
dm.set_brightness(40) # dim schedule scenario
dm.update_display() # same image, new brightness -> push
assert spy.count == 1
dm.set_brightness(90)
def test_snapshot_still_written_on_skip(self, dm, tmp_path):
"""The web preview mirror must keep working through skipped panel
pushes: _write_snapshot_if_due() still runs on the dirty-tracking
skip path and applies its own write/touch policy rather than being
bypassed entirely (see src/common/snapshot_policy.py an unchanged
frame is touched, not re-encoded, once TOUCH_INTERVAL elapses)."""
dm._snapshot_path = str(tmp_path / "snap.png")
dm._last_snapshot_ts = 0.0
dm._last_snapshot_touch_ts = 0.0
dm._last_snapshot_digest = None
dm.draw.rectangle([0, 0, 30, 8], fill=(255, 255, 0))
dm.update_display() # push + snapshot write (first frame)
assert os.path.exists(dm._snapshot_path)
first_mtime = os.path.getmtime(dm._snapshot_path)
# Age the write/touch bookkeeping past TOUCH_INTERVAL so the next
# identical frame is due for a touch, then push it again: dirty
# tracking must skip the panel write, but the snapshot mirror must
# still get its mtime bumped so the health check doesn't go stale.
from src.common import snapshot_policy
stale_ts = time.time() - snapshot_policy.TOUCH_INTERVAL - 1.0
dm._last_snapshot_ts = stale_ts
dm._last_snapshot_touch_ts = stale_ts
with _SwapSpy(dm.matrix) as spy:
dm.update_display() # identical frame -> panel push skipped
assert spy.count == 0
assert os.path.getmtime(dm._snapshot_path) > first_mtime
class TestKillSwitch:
def test_dirty_tracking_can_be_disabled(self, dm):
dm._dirty_tracking_enabled = False
try:
dm.draw.rectangle([0, 0, 10, 10], fill=(1, 2, 3))
with _SwapSpy(dm.matrix) as spy:
dm.update_display()
dm.update_display()
dm.update_display()
assert spy.count == 3 # always-push, exactly the old behavior
finally:
dm._dirty_tracking_enabled = True
dm._last_pushed_digest = None
def test_config_flag_wires_through(self):
from src.display_manager import DisplayManager
DisplayManager._instance = None
DisplayManager._initialized = False
try:
manager = DisplayManager({
"display": {
"hardware": {"rows": 32, "cols": 64, "chain_length": 1,
"parallel": 1},
"runtime": {"gpio_slowdown": 0},
"dirty_tracking": False,
},
}, suppress_test_pattern=True)
assert manager._dirty_tracking_enabled is False
finally:
DisplayManager._instance = None
DisplayManager._initialized = False
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
-519
View File
@@ -1,519 +0,0 @@
"""Tests for src/element_style.py — universal per-element style resolution.
The load-bearing behavior is the user-override check: saved configs ALWAYS
contain the schema defaults (merge_with_defaults runs at save time and again
before plugin instantiation), so "key present" must never be read as "user
set it". Only "present and different from the schema default" counts.
"""
import json
import os
import sys
import pytest
from PIL import ImageFont
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.element_style import ( # noqa: E402
ElementStyle,
ElementStyleResolver,
FONT_ALIASES,
defaults_from_schema_file,
extract_schema_defaults,
load_font,
resolve_font_name,
)
PRESS_START = "PressStart2P-Regular.ttf"
FOUR_BY_SIX = "4x6-font.ttf"
FIVE_BY_SEVEN_BDF = "5x7.bdf"
# ---------------------------------------------------------------------------
# load_font
# ---------------------------------------------------------------------------
class TestLoadFont:
def test_ttf(self):
font = load_font(PRESS_START, 8)
assert isinstance(font, ImageFont.FreeTypeFont)
assert font.size == 8
def test_bdf_at_native_size(self):
"""FreeType loads BDF strikes directly at their native size."""
font = load_font(FIVE_BY_SEVEN_BDF, 7)
assert isinstance(font, ImageFont.FreeTypeFont)
def test_bdf_at_wrong_size_falls_back(self):
"""BDF fonts are fixed-size; a non-native size falls back to the
fallback font at the requested size rather than raising."""
font = load_font(FIVE_BY_SEVEN_BDF, 14)
assert isinstance(font, ImageFont.FreeTypeFont)
assert font.size == 14 # fallback font honored the requested size
def test_alias_resolves(self):
assert resolve_font_name("press_start") == PRESS_START
font = load_font("press_start", 16)
assert isinstance(font, ImageFont.FreeTypeFont)
assert font.size == 16
def test_filename_passes_through_alias(self):
assert resolve_font_name(FOUR_BY_SIX) == FOUR_BY_SIX
def test_missing_file_falls_back(self):
font = load_font("no-such-font.ttf", 10)
assert isinstance(font, ImageFont.FreeTypeFont)
assert font.size == 10
@pytest.mark.parametrize("garbage", ["", None, "../../etc/passwd", "x" * 300])
def test_garbage_never_raises(self, garbage):
font = load_font(garbage, 8)
assert font is not None
def test_everything_missing_uses_pil_default(self):
font = load_font("nope.ttf", 8, fonts_dir="/nonexistent",
fallback_font="also-nope.ttf")
assert font is not None # ImageFont.load_default()
def test_aliases_cover_the_baseball_set(self):
"""The centralized map must be a superset of the per-plugin copies
it replaces (baseball game_renderer.py + sports.py)."""
assert FONT_ALIASES["press_start"] == PRESS_START
assert FONT_ALIASES["four_by_six"] == FOUR_BY_SIX
assert FONT_ALIASES["five_by_seven"] == FIVE_BY_SEVEN_BDF
# ---------------------------------------------------------------------------
# user_forced provenance
# ---------------------------------------------------------------------------
SCHEMA_DEFAULTS = {
"customization": {
"score_text": {"font": PRESS_START, "font_size": 10},
}
}
def _style(config, defaults=SCHEMA_DEFAULTS):
return ElementStyleResolver(config, defaults).style(
"score_text", classic_font=PRESS_START, classic_size=10)
class TestUserForced:
def test_absent_is_not_forced(self):
style = _style({})
assert not style.user_forced
assert style.font_name == PRESS_START
assert style.font_size == 10
def test_schema_default_present_is_not_forced(self):
"""THE bug this module exists to fix: the save flow writes schema
defaults into every saved config, so their presence means nothing."""
config = {"customization": {"score_text": {
"font": PRESS_START, "font_size": 10}}}
style = _style(config)
assert not style.user_forced
def test_different_font_is_forced(self):
config = {"customization": {"score_text": {
"font": FOUR_BY_SIX, "font_size": 10}}}
style = _style(config)
assert style.user_forced_font
assert not style.user_forced_size
assert style.user_forced
def test_different_size_is_forced(self):
config = {"customization": {"score_text": {
"font": PRESS_START, "font_size": 14}}}
style = _style(config)
assert style.user_forced_size
assert not style.user_forced_font
assert style.font_size == 14
def test_string_size_equal_to_default_is_not_forced(self):
config = {"customization": {"score_text": {"font_size": "10"}}}
assert not _style(config).user_forced
def test_without_schema_defaults_compares_against_classic(self):
"""Degraded mode (old cores, tests): classic_* is the reference."""
config = {"customization": {"score_text": {
"font": PRESS_START, "font_size": 10}}}
style = _style(config, defaults={})
assert not style.user_forced
forced = _style({"customization": {"score_text": {"font_size": 12}}},
defaults={})
assert forced.user_forced
def test_schema_default_differing_from_classic_wins_as_reference(self):
"""When the schema declares a different default than the classic_*
args, the schema is the reference a config equal to the schema
default is untouched."""
defaults = {"customization": {"score_text": {
"font": FOUR_BY_SIX, "font_size": 6}}}
config = {"customization": {"score_text": {
"font": FOUR_BY_SIX, "font_size": 6}}}
style = ElementStyleResolver(config, defaults).style(
"score_text", classic_font=PRESS_START, classic_size=10)
assert not style.user_forced
def test_unknown_element_uses_classic(self):
style = ElementStyleResolver({}, SCHEMA_DEFAULTS).style(
"no_such_element", classic_font=FOUR_BY_SIX, classic_size=6)
assert not style.user_forced
assert style.font_name == FOUR_BY_SIX
assert style.font_size == 6
def test_malformed_customization_is_tolerated(self):
for bad in [{"customization": "oops"},
{"customization": {"score_text": "oops"}},
{"customization": {"score_text": {"font_size": "huge"}}},
None]:
style = _style(bad)
assert not style.user_forced
assert style.font_size == 10
# ---------------------------------------------------------------------------
# color
# ---------------------------------------------------------------------------
class TestColor:
"""Color provenance mirrors fonts: the web form ALWAYS posts the RGB
inputs, so a saved config carries the schema-default color whether or
not the user touched it. Only a value differing from the schema default
is an override; otherwise the plugin's classic color survives — critical
for state-dependent colors (a score that turns gold on a touchdown)."""
COLOR_DEFAULTS = {"customization": {"score_text": {
"font": PRESS_START, "font_size": 10, "text_color": [255, 255, 255]}}}
def _color_style(self, config, defaults=None, classic_color=(255, 215, 0)):
return ElementStyleResolver(config, defaults or self.COLOR_DEFAULTS).style(
"score_text", classic_font=PRESS_START, classic_size=10,
classic_color=classic_color)
def test_absent_returns_classic_color(self):
style = self._color_style({})
assert style.color == (255, 215, 0)
assert not style.user_forced_color
def test_absent_with_no_classic_is_none(self):
assert _style({}).color is None
def test_schema_default_present_keeps_classic_color(self):
"""A saved config always contains the default — it must not clobber
the plugin's (possibly semantic) classic color."""
config = {"customization": {"score_text": {"text_color": [255, 255, 255]}}}
style = self._color_style(config)
assert style.color == (255, 215, 0)
assert not style.user_forced_color
def test_changed_color_is_an_override(self):
config = {"customization": {"score_text": {"text_color": [0, 128, 255]}}}
style = self._color_style(config)
assert style.color == (0, 128, 255)
assert style.user_forced_color
def test_present_without_schema_default_is_an_override(self):
"""Hand-written schemas without a text_color default: presence is
intent (there is nothing to compare against)."""
config = {"customization": {"score_text": {"text_color": [0, 128, 255]}}}
style = self._color_style(config, defaults=SCHEMA_DEFAULTS)
assert style.color == (0, 128, 255)
assert style.user_forced_color
def test_values_clamped(self):
config = {"customization": {"score_text": {"text_color": [300, -5, 128]}}}
assert self._color_style(config).color == (255, 0, 128)
def test_color_never_affects_user_forced_sizing(self):
config = {"customization": {"score_text": {"text_color": [0, 128, 255]}}}
style = self._color_style(config)
assert style.user_forced_color
assert not style.user_forced
@pytest.mark.parametrize("bad", [[1, 2], [1, 2, 3, 4], "red",
["a", "b", "c"], 255, None])
def test_malformed_falls_back_to_classic(self, bad):
config = {"customization": {"score_text": {"text_color": bad}}}
style = self._color_style(config, classic_color=(1, 2, 3))
assert style.color == (1, 2, 3)
assert not style.user_forced_color
# ---------------------------------------------------------------------------
# offsets
# ---------------------------------------------------------------------------
class TestOffsets:
def test_unset_is_zero(self):
assert ElementStyleResolver({}).offset("score") == (0, 0)
def test_layout_section(self):
"""The deployed sports convention: customization.layout.<element>."""
config = {"customization": {"layout": {"score": {
"x_offset": 3, "y_offset": -2}}}}
assert ElementStyleResolver(config).offset("score") == (3, -2)
def test_element_section_fallback(self):
config = {"customization": {"score": {"x_offset": 5}}}
assert ElementStyleResolver(config).offset("score") == (5, 0)
def test_layout_section_wins_over_element_section(self):
config = {"customization": {
"layout": {"score": {"x_offset": 1}},
"score": {"x_offset": 9, "y_offset": 9},
}}
resolver = ElementStyleResolver(config)
assert resolver.offset_value("score", "x_offset") == 1
# y_offset absent from layout section -> element section supplies it
assert resolver.offset_value("score", "y_offset") == 9
@pytest.mark.parametrize("raw,expected", [
(2, 2), (2.7, 2), ("3", 3), ("2.0", 2), ("-4", -4),
(None, 0), ("junk", 0), ([], 0), (True, 0),
])
def test_coercion_matches_sports_helper(self, raw, expected):
"""Same tolerance as the sports.py/_get_layout_offset copies this
replaces: int/float/numeric-string pass, anything else -> default."""
config = {"customization": {"layout": {"e": {"x_offset": raw}}}}
assert ElementStyleResolver(config).offset_value("e", "x_offset") == expected
def test_custom_axis_names(self):
"""Football's records use away_x_offset/home_x_offset."""
config = {"customization": {"layout": {"records": {
"away_x_offset": 4, "home_x_offset": -4}}}}
resolver = ElementStyleResolver(config)
assert resolver.offset_value("records", "away_x_offset") == 4
assert resolver.offset_value("records", "home_x_offset") == -4
def test_style_carries_offset(self):
config = {"customization": {"layout": {"score_text": {
"x_offset": 2, "y_offset": 1}}}}
assert _style(config).offset == (2, 1)
# ---------------------------------------------------------------------------
# caching
# ---------------------------------------------------------------------------
class TestCaching:
def test_same_call_is_cached(self):
resolver = ElementStyleResolver({}, SCHEMA_DEFAULTS)
a = resolver.style("score_text", classic_font=PRESS_START, classic_size=10)
b = resolver.style("score_text", classic_font=PRESS_START, classic_size=10)
assert a is b
def test_clear_cache(self):
resolver = ElementStyleResolver({}, SCHEMA_DEFAULTS)
a = resolver.style("score_text", classic_font=PRESS_START, classic_size=10)
resolver.clear_cache()
b = resolver.style("score_text", classic_font=PRESS_START, classic_size=10)
assert a is not b
# PIL fonts compare by identity; compare the value fields
assert (a.font_name, a.font_size, a.color, a.offset, a.user_forced) == \
(b.font_name, b.font_size, b.color, b.offset, b.user_forced)
# ---------------------------------------------------------------------------
# schema default extraction
# ---------------------------------------------------------------------------
class TestSchemaDefaults:
def test_matches_schema_manager_extraction(self):
"""The pure helper must agree with SchemaManager.extract_defaults_from_schema
on a real plugin-style schema it exists so plugins get the same
answer in harness contexts where the schema manager is absent."""
from src.plugin_system.schema_manager import SchemaManager
schema = {
"type": "object",
"properties": {
"enabled": {"type": "boolean", "default": True},
"customization": {
"type": "object",
"properties": {
"score_text": {
"type": "object",
"properties": {
"font": {"type": "string", "default": PRESS_START},
"font_size": {"type": "integer", "default": 10},
"y_percent": {"type": "number"},
},
},
"layout": {
"type": "object",
"properties": {
"score": {
"type": "object",
"properties": {
"x_offset": {"type": "integer", "default": 0},
},
},
},
},
},
},
"opaque_with_default": {"type": "object", "default": {},
"properties": {"x": {"default": 1}}},
# array shapes the schema manager special-cases — parity
# must hold for these too (found divergent in review)
"plain_array": {"type": "array", "items": {"type": "string"}},
"obj_array": {"type": "array", "items": {
"type": "object", "properties": {"x": {"default": 1}}}},
"item_default_array": {"type": "array",
"items": {"type": "string", "default": "a"}},
},
}
pure = extract_schema_defaults(schema)
managed = SchemaManager().extract_defaults_from_schema(schema)
assert pure == managed
assert pure["customization"]["score_text"]["font"] == PRESS_START
# object-level default short-circuits recursion (both must agree)
assert pure["opaque_with_default"] == {}
assert pure["plain_array"] == []
assert pure["obj_array"] == []
assert pure["item_default_array"] == ["a"]
def test_defaults_from_schema_file(self, tmp_path):
schema_path = tmp_path / "config_schema.json"
schema_path.write_text(json.dumps({
"type": "object",
"properties": {
"customization": {
"type": "object",
"properties": {
"title_text": {
"type": "object",
"properties": {
"font": {"type": "string", "default": PRESS_START},
},
},
},
},
},
}))
defaults = defaults_from_schema_file(str(schema_path))
assert defaults["customization"]["title_text"]["font"] == PRESS_START
def test_defaults_from_missing_or_bad_file(self, tmp_path):
assert defaults_from_schema_file("/nonexistent/schema.json") == {}
bad = tmp_path / "bad.json"
bad.write_text("{not json")
assert defaults_from_schema_file(str(bad)) == {}
# ---------------------------------------------------------------------------
# BasePlugin integration
# ---------------------------------------------------------------------------
class _StubSchemaManager:
"""Schema manager double exposing the two methods the resolver path uses."""
def __init__(self, schema):
self._schema = schema
def load_schema(self, plugin_id, use_cache=True):
return self._schema
def extract_defaults_from_schema(self, schema, prefix=""):
# Mirror the real nested-dict extraction for this simple shape
def walk(props):
out = {}
for key, spec in props.get("properties", {}).items():
if "default" in spec:
out[key] = spec["default"]
elif spec.get("type") == "object" and "properties" in spec:
nested = walk(spec)
if nested:
out[key] = nested
return out
return walk(schema)
def _make_plugin(config, schema=None):
from src.plugin_system.base_plugin import BasePlugin
from src.plugin_system.testing.mocks import (
MockCacheManager, MockPluginManager)
from src.plugin_system.testing.visual_display_manager import (
VisualTestDisplayManager)
class _Plugin(BasePlugin):
def update(self):
return True
def display(self, force_clear=False):
return None
plugin_manager = MockPluginManager()
if schema is not None:
plugin_manager.schema_manager = _StubSchemaManager(schema)
return _Plugin("test-plugin", config,
VisualTestDisplayManager(64, 32),
MockCacheManager(), plugin_manager)
TEST_SCHEMA = {
"type": "object",
"properties": {
"customization": {
"type": "object",
"properties": {
"score_text": {
"type": "object",
"properties": {
"font": {"type": "string", "default": PRESS_START},
"font_size": {"type": "integer", "default": 10},
},
},
},
},
},
}
class TestBasePluginIntegration:
def test_element_style_with_schema_defaults(self):
"""The full path: saved config carries schema defaults, plugin's
element_style still reports not-forced."""
config = {"enabled": True, "customization": {"score_text": {
"font": PRESS_START, "font_size": 10}}}
plugin = _make_plugin(config, schema=TEST_SCHEMA)
style = plugin.element_style("score_text", classic_font=PRESS_START,
classic_size=10)
assert not style.user_forced
def test_element_style_detects_real_override(self):
config = {"enabled": True, "customization": {"score_text": {
"font": PRESS_START, "font_size": 14}}}
plugin = _make_plugin(config, schema=TEST_SCHEMA)
style = plugin.element_style("score_text", classic_font=PRESS_START,
classic_size=10)
assert style.user_forced_size
assert style.font_size == 14
def test_works_without_schema_manager(self):
"""MockPluginManager has no schema_manager attribute by default —
the resolver degrades to classic-default comparison, no crash."""
config = {"enabled": True, "customization": {"score_text": {
"font_size": 12}}}
plugin = _make_plugin(config, schema=None)
style = plugin.element_style("score_text", classic_font=PRESS_START,
classic_size=10)
assert style.user_forced_size # 12 != classic 10
def test_resolver_is_cached_and_invalidated_on_config_change(self):
plugin = _make_plugin({"enabled": True}, schema=TEST_SCHEMA)
first = plugin.style_resolver
assert plugin.style_resolver is first
plugin.on_config_change({"enabled": True, "customization": {
"score_text": {"font_size": 14}}})
second = plugin.style_resolver
assert second is not first
style = plugin.element_style("score_text", classic_font=PRESS_START,
classic_size=10)
assert style.user_forced_size
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
-392
View File
@@ -1,392 +0,0 @@
"""
Tests for LayoutManager.
Tests layout creation, management, rendering, and element positioning.
"""
import pytest
import json
from unittest.mock import MagicMock
from src.layout_manager import LayoutManager
class TestLayoutManager:
"""Test LayoutManager functionality."""
@pytest.fixture
def tmp_layout_file(self, tmp_path):
"""Create a temporary layout file."""
layout_file = tmp_path / "custom_layouts.json"
return str(layout_file)
@pytest.fixture
def mock_display_manager(self):
"""Create a mock display manager."""
dm = MagicMock()
dm.clear = MagicMock()
dm.update_display = MagicMock()
dm.draw_text = MagicMock()
dm.draw_weather_icon = MagicMock()
dm.small_font = MagicMock()
dm.regular_font = MagicMock()
return dm
@pytest.fixture
def layout_manager(self, tmp_layout_file, mock_display_manager):
"""Create a LayoutManager instance."""
return LayoutManager(
display_manager=mock_display_manager,
config_path=tmp_layout_file
)
def test_init(self, tmp_layout_file, mock_display_manager):
"""Test LayoutManager initialization."""
lm = LayoutManager(
display_manager=mock_display_manager,
config_path=tmp_layout_file
)
assert lm.display_manager == mock_display_manager
assert lm.config_path == tmp_layout_file
assert lm.layouts == {}
assert lm.current_layout is None
def test_load_layouts_file_exists(self, tmp_path, mock_display_manager):
"""Test loading layouts from existing file."""
layout_file = tmp_path / "custom_layouts.json"
layout_data = {
"test_layout": {
"elements": [{"type": "text", "x": 0, "y": 0}],
"description": "Test layout"
}
}
with open(layout_file, 'w') as f:
json.dump(layout_data, f)
lm = LayoutManager(
display_manager=mock_display_manager,
config_path=str(layout_file)
)
assert "test_layout" in lm.layouts
assert lm.layouts["test_layout"]["description"] == "Test layout"
def test_load_layouts_file_not_exists(self, tmp_layout_file, mock_display_manager):
"""Test loading layouts when file doesn't exist."""
lm = LayoutManager(
display_manager=mock_display_manager,
config_path=tmp_layout_file
)
assert lm.layouts == {}
def test_create_layout(self, layout_manager):
"""Test creating a new layout."""
elements = [{"type": "text", "x": 10, "y": 20, "properties": {"text": "Hello"}}]
result = layout_manager.create_layout("test_layout", elements, "Test description")
assert result is True
assert "test_layout" in layout_manager.layouts
assert layout_manager.layouts["test_layout"]["elements"] == elements
assert layout_manager.layouts["test_layout"]["description"] == "Test description"
assert "created" in layout_manager.layouts["test_layout"]
assert "modified" in layout_manager.layouts["test_layout"]
def test_update_layout(self, layout_manager):
"""Test updating an existing layout."""
# Create a layout first
elements1 = [{"type": "text", "x": 0, "y": 0}]
layout_manager.create_layout("test_layout", elements1, "Original")
# Update it
elements2 = [{"type": "text", "x": 10, "y": 20}]
result = layout_manager.update_layout("test_layout", elements2, "Updated")
assert result is True
assert layout_manager.layouts["test_layout"]["elements"] == elements2
assert layout_manager.layouts["test_layout"]["description"] == "Updated"
assert "modified" in layout_manager.layouts["test_layout"]
def test_update_layout_not_exists(self, layout_manager):
"""Test updating a non-existent layout."""
elements = [{"type": "text", "x": 0, "y": 0}]
result = layout_manager.update_layout("nonexistent", elements)
assert result is False
def test_delete_layout(self, layout_manager):
"""Test deleting a layout."""
elements = [{"type": "text", "x": 0, "y": 0}]
layout_manager.create_layout("test_layout", elements)
result = layout_manager.delete_layout("test_layout")
assert result is True
assert "test_layout" not in layout_manager.layouts
def test_delete_layout_not_exists(self, layout_manager):
"""Test deleting a non-existent layout."""
result = layout_manager.delete_layout("nonexistent")
assert result is False
def test_get_layout(self, layout_manager):
"""Test getting a specific layout."""
elements = [{"type": "text", "x": 0, "y": 0}]
layout_manager.create_layout("test_layout", elements)
layout = layout_manager.get_layout("test_layout")
assert layout is not None
assert layout["elements"] == elements
def test_get_layout_not_exists(self, layout_manager):
"""Test getting a non-existent layout."""
layout = layout_manager.get_layout("nonexistent")
assert layout == {}
def test_list_layouts(self, layout_manager):
"""Test listing all layouts."""
layout_manager.create_layout("layout1", [])
layout_manager.create_layout("layout2", [])
layout_manager.create_layout("layout3", [])
layouts = layout_manager.list_layouts()
assert len(layouts) == 3
assert "layout1" in layouts
assert "layout2" in layouts
assert "layout3" in layouts
def test_set_current_layout(self, layout_manager):
"""Test setting the current layout."""
layout_manager.create_layout("test_layout", [])
result = layout_manager.set_current_layout("test_layout")
assert result is True
assert layout_manager.current_layout == "test_layout"
def test_set_current_layout_not_exists(self, layout_manager):
"""Test setting a non-existent layout as current."""
result = layout_manager.set_current_layout("nonexistent")
assert result is False
assert layout_manager.current_layout is None
def test_render_layout(self, layout_manager, mock_display_manager):
"""Test rendering a layout."""
elements = [
{"type": "text", "x": 0, "y": 0, "properties": {"text": "Hello"}},
{"type": "text", "x": 10, "y": 10, "properties": {"text": "World"}}
]
layout_manager.create_layout("test_layout", elements)
result = layout_manager.render_layout("test_layout")
assert result is True
mock_display_manager.clear.assert_called_once()
mock_display_manager.update_display.assert_called_once()
assert mock_display_manager.draw_text.call_count == 2
def test_render_layout_no_display_manager(self, tmp_layout_file):
"""Test rendering without display manager."""
lm = LayoutManager(display_manager=None, config_path=tmp_layout_file)
lm.create_layout("test_layout", [])
result = lm.render_layout("test_layout")
assert result is False
def test_render_layout_not_exists(self, layout_manager):
"""Test rendering a non-existent layout."""
result = layout_manager.render_layout("nonexistent")
assert result is False
def test_render_element_text(self, layout_manager, mock_display_manager):
"""Test rendering a text element."""
element = {
"type": "text",
"x": 10,
"y": 20,
"properties": {
"text": "Hello",
"color": [255, 0, 0],
"font_size": "small"
}
}
layout_manager.render_element(element, {})
mock_display_manager.draw_text.assert_called_once()
call_args = mock_display_manager.draw_text.call_args
assert call_args[0][0] == "Hello" # text
assert call_args[0][1] == 10 # x
assert call_args[0][2] == 20 # y
def test_render_element_weather_icon(self, layout_manager, mock_display_manager):
"""Test rendering a weather icon element."""
element = {
"type": "weather_icon",
"x": 10,
"y": 20,
"properties": {
"condition": "sunny",
"size": 16
}
}
layout_manager.render_element(element, {})
mock_display_manager.draw_weather_icon.assert_called_once_with("sunny", 10, 20, 16)
def test_render_element_weather_icon_from_context(self, layout_manager, mock_display_manager):
"""Test rendering weather icon with data from context."""
element = {
"type": "weather_icon",
"x": 10,
"y": 20,
"properties": {"size": 16}
}
data_context = {
"weather": {
"condition": "cloudy"
}
}
layout_manager.render_element(element, data_context)
mock_display_manager.draw_weather_icon.assert_called_once_with("cloudy", 10, 20, 16)
def test_render_element_rectangle(self, layout_manager, mock_display_manager):
"""Test rendering a rectangle element."""
element = {
"type": "rectangle",
"x": 10,
"y": 20,
"properties": {
"width": 50,
"height": 30,
"color": [255, 0, 0],
"filled": True
}
}
# Mock the draw object and rectangle method
mock_draw = MagicMock()
mock_display_manager.draw = mock_draw
layout_manager.render_element(element, {})
# Verify rectangle was drawn
mock_draw.rectangle.assert_called_once()
def test_render_element_unknown_type(self, layout_manager):
"""Test rendering an unknown element type."""
element = {
"type": "unknown_type",
"x": 0,
"y": 0,
"properties": {}
}
# Should not raise an exception
layout_manager.render_element(element, {})
def test_process_template_text(self, layout_manager):
"""Test template text processing."""
text = "Hello {name}, temperature is {temp}°F"
data_context = {
"name": "World",
"temp": 72
}
result = layout_manager._process_template_text(text, data_context)
assert result == "Hello World, temperature is 72°F"
def test_process_template_text_no_context(self, layout_manager):
"""Test template text with missing context."""
text = "Hello {name}"
data_context = {}
result = layout_manager._process_template_text(text, data_context)
# Should leave template as-is or handle gracefully
assert "{name}" in result or result == "Hello "
def test_save_layouts_error_handling(self, layout_manager):
"""Test error handling when saving layouts."""
# Create a layout
layout_manager.create_layout("test", [])
# Make save fail by using invalid path
layout_manager.config_path = "/nonexistent/directory/layouts.json"
result = layout_manager.save_layouts()
# Should handle error gracefully
assert result is False
def test_render_element_line(self, layout_manager, mock_display_manager):
"""Test rendering a line element."""
element = {
"type": "line",
"x": 10,
"y": 20,
"properties": {
"x2": 50,
"y2": 30,
"color": [255, 0, 0],
"width": 2
}
}
mock_draw = MagicMock()
mock_display_manager.draw = mock_draw
layout_manager.render_element(element, {})
mock_draw.line.assert_called_once()
def test_render_element_clock(self, layout_manager, mock_display_manager):
"""Test rendering a clock element."""
element = {
"type": "clock",
"x": 10,
"y": 20,
"properties": {
"format": "%H:%M",
"color": [255, 255, 255]
}
}
layout_manager.render_element(element, {})
mock_display_manager.draw_text.assert_called_once()
def test_render_element_data_text(self, layout_manager, mock_display_manager):
"""Test rendering a data text element."""
element = {
"type": "data_text",
"x": 10,
"y": 20,
"properties": {
"data_key": "weather.temperature",
"format": "Temp: {value}°F",
"color": [255, 255, 255],
"default": "N/A"
}
}
data_context = {
"weather": {
"temperature": 72
}
}
layout_manager.render_element(element, data_context)
mock_display_manager.draw_text.assert_called_once()
+8
View File
@@ -123,6 +123,14 @@ class TestPluginManager:
pm.run_scheduled_updates(current_time=time.time())
# Updates now execute on the background worker (the scheduler
# returns immediately) — wait for completion before asserting.
deadline = time.time() + 5
while (plugin_instance.update.call_count == 0
and time.time() < deadline):
time.sleep(0.02)
pm.stop_update_worker()
plugin_instance.update.assert_called_once()
assert "test_plugin" in pm.plugin_last_update
assert pm.state_manager.get_state("test_plugin") == PluginState.ENABLED
-273
View File
@@ -1,273 +0,0 @@
"""Tests for x-style-elements schema expansion.
A plugin declares styleable elements once, compactly; expansion generates
the full customization property blocks at schema-load time. The invariants
that matter:
- idempotent (expand(expand(s)) == expand(s)) and the input is never mutated
- both load paths (cached GET, uncached save) see the identical shape
- generated defaults flow into generate_default_config, and saving twice is
round-trip stable (merge_with_defaults produces no churn)
- hand-written property blocks for the same element always win
- defaults_from_schema_file (what plugins use to build resolvers from their
RAW schema file) agrees exactly with the schema manager's expanded view
"""
import copy
import json
import os
import sys
import jsonschema
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.element_style import ( # noqa: E402
defaults_from_schema_file,
expand_style_elements,
extract_schema_defaults,
get_style_elements,
)
from src.plugin_system.schema_manager import SchemaManager # noqa: E402
PRESS_START = "PressStart2P-Regular.ttf"
def _declared_schema():
return {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"enabled": {"type": "boolean", "default": True},
"customization": {
"type": "object",
"title": "Display Customization",
"x-style-elements": {
"score_text": {
"title": "Game Score",
"font": {"default": PRESS_START},
"size": {"default": 10, "min": 4, "max": 16},
"color": True,
"offsets": True,
},
"detail_text": {
"font": {"default": "4x6-font.ttf"},
"size": {"default": 6},
},
},
"properties": {},
"additionalProperties": False,
},
},
}
class TestExpansionShape:
def test_generates_element_blocks(self):
expanded = expand_style_elements(_declared_schema())
cust = expanded["properties"]["customization"]["properties"]
score = cust["score_text"]
assert score["x-style-managed"] is True
assert score["title"] == "Game Score"
assert score["properties"]["font"]["default"] == PRESS_START
assert score["properties"]["font"]["x-widget"] == "font-selector"
assert score["properties"]["font_size"]["default"] == 10
assert score["properties"]["font_size"]["minimum"] == 4
assert score["properties"]["font_size"]["maximum"] == 16
assert score["properties"]["text_color"]["x-widget"] == "color-picker"
assert score["properties"]["text_color"]["default"] == [255, 255, 255]
assert score["additionalProperties"] is False
def test_color_and_offsets_are_optional(self):
expanded = expand_style_elements(_declared_schema())
cust = expanded["properties"]["customization"]["properties"]
detail = cust["detail_text"]
assert "text_color" not in detail["properties"]
assert detail["title"] == "Detail Text" # prettified from the key
layout = cust["layout"]["properties"]
assert "score_text" in layout
assert "detail_text" not in layout
def test_offsets_block_shape(self):
expanded = expand_style_elements(_declared_schema())
layout = expanded["properties"]["customization"]["properties"]["layout"]
assert layout["x-style-managed"] is True
entry = layout["properties"]["score_text"]
assert entry["properties"]["x_offset"]["default"] == 0
assert entry["properties"]["y_offset"]["default"] == 0
def test_declared_color_default(self):
schema = _declared_schema()
decl = schema["properties"]["customization"]["x-style-elements"]
decl["score_text"]["color"] = {"default": [255, 215, 0]}
expanded = expand_style_elements(schema)
color = (expanded["properties"]["customization"]["properties"]
["score_text"]["properties"]["text_color"])
assert color["default"] == [255, 215, 0]
def test_declaration_survives_expansion(self):
"""The declaration is the element registry for tooling — it must
remain readable from the expanded schema."""
expanded = expand_style_elements(_declared_schema())
assert set(get_style_elements(expanded)) == {"score_text", "detail_text"}
assert set(SchemaManager.get_style_elements(expanded)) == {
"score_text", "detail_text"}
def test_no_declaration_returns_same_object(self):
schema = {"type": "object", "properties": {"enabled": {"default": True}}}
assert expand_style_elements(schema) is schema
def test_valid_draft7(self):
jsonschema.Draft7Validator.check_schema(
expand_style_elements(_declared_schema()))
def test_property_order_updated_when_present(self):
schema = _declared_schema()
schema["properties"]["customization"]["x-propertyOrder"] = []
expanded = expand_style_elements(schema)
order = expanded["properties"]["customization"]["x-propertyOrder"]
# generated elements before layout (the template only renders keys
# in x-propertyOrder when one exists)
assert set(order) == {"score_text", "detail_text", "layout"}
assert order.index("score_text") < order.index("layout")
def test_malformed_declaration_is_harmless(self):
schema = _declared_schema()
schema["properties"]["customization"]["x-style-elements"] = {
"bad": "not a dict", "score_text": {"size": {"default": 10}}}
expanded = expand_style_elements(schema)
cust = expanded["properties"]["customization"]["properties"]
assert "bad" not in cust
assert "score_text" in cust
class TestExpansionInvariants:
def test_idempotent(self):
once = expand_style_elements(_declared_schema())
twice = expand_style_elements(once)
assert once == twice
def test_input_never_mutated(self):
schema = _declared_schema()
snapshot = copy.deepcopy(schema)
expand_style_elements(schema)
assert schema == snapshot
def test_hand_written_block_wins(self):
schema = _declared_schema()
hand_written = {
"type": "object",
"properties": {"font": {"type": "string", "default": "custom.ttf"}},
}
schema["properties"]["customization"]["properties"]["score_text"] = \
copy.deepcopy(hand_written)
expanded = expand_style_elements(schema)
assert (expanded["properties"]["customization"]["properties"]["score_text"]
== hand_written)
def test_hand_written_layout_entry_wins(self):
schema = _declared_schema()
schema["properties"]["customization"]["properties"]["layout"] = {
"type": "object",
"properties": {"score_text": {"type": "object", "properties": {
"x_offset": {"type": "integer", "default": 5}}}},
}
expanded = expand_style_elements(schema)
layout = expanded["properties"]["customization"]["properties"]["layout"]
assert layout["properties"]["score_text"]["properties"]["x_offset"]["default"] == 5
class TestSchemaManagerIntegration:
def _manager_with_schema(self, tmp_path, schema):
plugin_dir = tmp_path / "test-plugin"
plugin_dir.mkdir()
(plugin_dir / "config_schema.json").write_text(json.dumps(schema))
return SchemaManager(plugins_dir=tmp_path)
def test_load_schema_expands(self, tmp_path):
mgr = self._manager_with_schema(tmp_path, _declared_schema())
loaded = mgr.load_schema("test-plugin")
assert "score_text" in loaded["properties"]["customization"]["properties"]
def test_cached_and_uncached_loads_agree(self, tmp_path):
"""The save path uses use_cache=False while the form GET uses the
cache they must see the identical expanded shape."""
mgr = self._manager_with_schema(tmp_path, _declared_schema())
cached = mgr.load_schema("test-plugin", use_cache=True)
again = mgr.load_schema("test-plugin", use_cache=True)
uncached = mgr.load_schema("test-plugin", use_cache=False)
assert cached == uncached == again
def test_disk_file_untouched(self, tmp_path):
schema = _declared_schema()
mgr = self._manager_with_schema(tmp_path, schema)
mgr.load_schema("test-plugin")
on_disk = json.loads(
(tmp_path / "test-plugin" / "config_schema.json").read_text())
assert on_disk == schema
assert "score_text" not in on_disk["properties"]["customization"]["properties"]
def test_defaults_include_generated_elements(self, tmp_path):
mgr = self._manager_with_schema(tmp_path, _declared_schema())
defaults = mgr.generate_default_config("test-plugin")
assert defaults["customization"]["score_text"]["font"] == PRESS_START
assert defaults["customization"]["score_text"]["font_size"] == 10
assert defaults["customization"]["score_text"]["text_color"] == [255, 255, 255]
assert defaults["customization"]["layout"]["score_text"]["x_offset"] == 0
def test_save_twice_is_round_trip_stable(self, tmp_path):
"""merge_with_defaults(merged, defaults) must be a fixed point —
saving a config twice can't keep growing/altering it."""
mgr = self._manager_with_schema(tmp_path, _declared_schema())
defaults = mgr.generate_default_config("test-plugin")
user_config = {"enabled": True, "customization": {
"score_text": {"font_size": 14}}}
merged_once = mgr.merge_with_defaults(user_config, defaults)
merged_twice = mgr.merge_with_defaults(merged_once, defaults)
assert merged_once == merged_twice
assert merged_once["customization"]["score_text"]["font_size"] == 14
class TestResolverParity:
def test_defaults_from_schema_file_matches_manager_view(self, tmp_path):
"""Plugins build resolvers from their RAW schema file; the web UI
merges defaults from the EXPANDED schema. Both must produce the
same defaults or override detection diverges between contexts."""
schema = _declared_schema()
plugin_dir = tmp_path / "test-plugin"
plugin_dir.mkdir()
schema_path = plugin_dir / "config_schema.json"
schema_path.write_text(json.dumps(schema))
mgr = SchemaManager(plugins_dir=tmp_path)
manager_defaults = mgr.extract_defaults_from_schema(
mgr.load_schema("test-plugin"))
raw_file_defaults = defaults_from_schema_file(str(schema_path))
assert raw_file_defaults == manager_defaults
def test_resolver_treats_generated_defaults_as_untouched(self, tmp_path):
"""End to end: a config saved through the web UI (all generated
defaults baked in) must not read as a user override, and the
schema-default color must not clobber a classic color."""
from src.element_style import ElementStyleResolver
schema_path = tmp_path / "config_schema.json"
schema_path.write_text(json.dumps(_declared_schema()))
defaults = defaults_from_schema_file(str(schema_path))
saved_config = {"enabled": True, "customization": {
"score_text": {"font": PRESS_START, "font_size": 10,
"text_color": [255, 255, 255]},
"layout": {"score_text": {"x_offset": 0, "y_offset": 0}},
}}
resolver = ElementStyleResolver(saved_config, defaults)
style = resolver.style("score_text", classic_font=PRESS_START,
classic_size=10, classic_color=(255, 215, 0))
assert not style.user_forced
assert not style.user_forced_color
assert style.color == (255, 215, 0) # semantic classic color survives
assert style.offset == (0, 0)
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+93
View File
@@ -0,0 +1,93 @@
"""Tests for the snapshot write policy (src/common/snapshot_policy.py).
The invariants that matter:
- unchanged frames are NEVER re-encoded (the old code PNG-encoded identical
frames at 5 fps, 24/7)
- the file mtime never goes stale enough to trip the health check's 60s
degraded threshold (api_v3 get_hardware_status)
- a viewer gets full cadence; no viewer drops to the idle keepalive
"""
import os
import sys
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.common.snapshot_policy import ( # noqa: E402
IDLE_INTERVAL,
TOUCH_INTERVAL,
VIEWER_INTERVAL,
SnapshotAction,
decide,
)
class TestViewerCadence:
def test_changed_frame_with_viewer_writes_at_full_rate(self):
assert decide(now=100.0, last_write_ts=100.0 - VIEWER_INTERVAL,
last_touch_ts=0, viewer_fresh=True,
frame_changed=True) is SnapshotAction.WRITE
def test_changed_frame_with_viewer_respects_min_interval(self):
assert decide(now=100.0, last_write_ts=100.0 - VIEWER_INTERVAL / 2,
last_touch_ts=100.0, viewer_fresh=True,
frame_changed=True) is SnapshotAction.SKIP
def test_unchanged_frame_with_viewer_never_writes(self):
"""A static screen with a viewer must not burn PNG encodes."""
assert decide(now=100.0, last_write_ts=90.0, last_touch_ts=90.0,
viewer_fresh=True,
frame_changed=False) is SnapshotAction.SKIP
class TestIdleCadence:
def test_changed_frame_without_viewer_waits_for_idle_interval(self):
assert decide(now=100.0, last_write_ts=100.0 - IDLE_INTERVAL / 2,
last_touch_ts=100.0, viewer_fresh=False,
frame_changed=True) is SnapshotAction.SKIP
def test_changed_frame_without_viewer_writes_at_idle_rate(self):
assert decide(now=100.0, last_write_ts=100.0 - IDLE_INTERVAL,
last_touch_ts=0, viewer_fresh=False,
frame_changed=True) is SnapshotAction.WRITE
class TestHealthKeepalive:
def test_stale_mtime_gets_touched(self):
"""Whatever else happens, mtime must be bumped within TOUCH_INTERVAL
so the health check (60s threshold) never reads the display as dead."""
assert decide(now=100.0, last_write_ts=100.0 - TOUCH_INTERVAL,
last_touch_ts=100.0 - TOUCH_INTERVAL, viewer_fresh=False,
frame_changed=False) is SnapshotAction.TOUCH
def test_touch_applies_with_viewer_too(self):
"""Viewer watching a static screen: no writes, but health stays green."""
assert decide(now=100.0, last_write_ts=100.0 - TOUCH_INTERVAL - 1,
last_touch_ts=100.0 - TOUCH_INTERVAL - 1, viewer_fresh=True,
frame_changed=False) is SnapshotAction.TOUCH
def test_recent_touch_suppresses_another(self):
assert decide(now=100.0, last_write_ts=0.0,
last_touch_ts=100.0 - TOUCH_INTERVAL / 2, viewer_fresh=False,
frame_changed=False) is SnapshotAction.SKIP
def test_touch_interval_stays_under_health_threshold(self):
"""api_v3's hardware status treats snapshot age >= 60s as degraded.
Keep a 2x margin so scheduling jitter can't trip it."""
assert TOUCH_INTERVAL <= 30
def test_worst_case_mtime_age_is_bounded(self):
"""Simulate any interleaving: from any state, within one policy call
after TOUCH_INTERVAL elapses, mtime gets refreshed (WRITE or TOUCH)."""
for viewer in (True, False):
for changed in (True, False):
action = decide(now=1000.0, last_write_ts=900.0,
last_touch_ts=900.0, viewer_fresh=viewer,
frame_changed=changed)
assert action in (SnapshotAction.WRITE, SnapshotAction.TOUCH)
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+171
View File
@@ -0,0 +1,171 @@
"""Tests for atomic plugin updates (store_manager._reinstall_with_rollback).
Regression for a field data-loss incident: update_plugin's reinstall paths
(monorepo migration AND routine archive updates) deleted the installed
plugin BEFORE downloading its replacement a mid-update network failure
permanently destroyed the plugin. Seen live: a Pi with broken DNS lost 12
plugins from one update pass.
"""
import json
import os
import sys
import threading
import time
from pathlib import Path
from unittest.mock import patch
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.plugin_system.store_manager import PluginStoreManager # noqa: E402
PLUGIN_ID = "rollback-test-plugin"
@pytest.fixture
def store(tmp_path):
mgr = PluginStoreManager(plugins_dir=str(tmp_path))
plugin_dir = tmp_path / PLUGIN_ID
plugin_dir.mkdir()
(plugin_dir / "manifest.json").write_text(json.dumps(
{"id": PLUGIN_ID, "name": "Rollback Test", "version": "1.0.0"}))
(plugin_dir / "manager.py").write_text("# old version marker\n")
return mgr, plugin_dir
class TestReinstallWithRollback:
def test_failed_install_restores_old_version(self, store):
"""The whole point: a failed download must leave the old install."""
mgr, plugin_dir = store
with patch.object(mgr, "install_plugin", return_value=False):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is False
assert plugin_dir.exists()
assert "old version marker" in (plugin_dir / "manager.py").read_text()
# no aside debris left behind
leftovers = [p for p in plugin_dir.parent.iterdir()
if "standalone-backup" in p.name]
assert leftovers == []
def test_install_exception_restores_old_version(self, store):
mgr, plugin_dir = store
with patch.object(mgr, "install_plugin",
side_effect=RuntimeError("network down")):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is False
assert plugin_dir.exists()
assert "old version marker" in (plugin_dir / "manager.py").read_text()
def test_successful_install_removes_aside(self, store):
mgr, plugin_dir = store
def fake_install(plugin_id):
new_dir = plugin_dir # same path, new content
new_dir.mkdir(exist_ok=True)
(new_dir / "manager.py").write_text("# new version\n")
(new_dir / "manifest.json").write_text(json.dumps(
{"id": PLUGIN_ID, "name": "Rollback Test", "version": "2.0.0"}))
return True
with patch.object(mgr, "install_plugin", side_effect=fake_install):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is True
assert "new version" in (plugin_dir / "manager.py").read_text()
leftovers = [p for p in plugin_dir.parent.iterdir()
if "standalone-backup" in p.name]
assert leftovers == []
def test_partial_download_debris_is_replaced_by_old_version(self, store):
"""A failed install that left a partial directory must still roll back."""
mgr, plugin_dir = store
def fake_partial_install(plugin_id):
plugin_dir.mkdir(exist_ok=True)
(plugin_dir / "half-downloaded.tmp").write_text("junk")
return False
with patch.object(mgr, "install_plugin", side_effect=fake_partial_install):
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
assert ok is False
assert "old version marker" in (plugin_dir / "manager.py").read_text()
assert not (plugin_dir / "half-downloaded.tmp").exists()
def test_stale_aside_from_previous_crash_is_cleared(self, store):
mgr, plugin_dir = store
stale = plugin_dir.parent / f"{PLUGIN_ID}.standalone-backup-migrating"
stale.mkdir()
(stale / "old.txt").write_text("stale")
with patch.object(mgr, "install_plugin", return_value=False) as mock_install:
ok = mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir)
# The reinstall itself still fails (mocked) and the old install is
# restored, but the stale aside must not have survived — otherwise
# it would have blocked this run's own rename (or a future one).
assert not stale.exists()
mock_install.assert_called_once_with(PLUGIN_ID)
assert ok is False
assert plugin_dir.exists()
assert "old version marker" in (plugin_dir / "manager.py").read_text()
def test_concurrent_updates_for_same_plugin_are_serialized(self, store):
"""Two overlapping requests for the same plugin_id (double-click,
two browser tabs the web UI runs Flask with threaded=True) must
not interleave: the loser must wait for the winner to finish
rather than renaming the winner's in-progress install aside and
stealing its rollback safety net."""
mgr, plugin_dir = store
active = 0
max_active = 0
guard = threading.Lock()
def fake_install(plugin_id):
nonlocal active, max_active
with guard:
active += 1
max_active = max(max_active, active)
time.sleep(0.05)
plugin_dir.mkdir(exist_ok=True)
(plugin_dir / "manager.py").write_text("# new version\n")
(plugin_dir / "manifest.json").write_text(json.dumps(
{"id": PLUGIN_ID, "name": "Rollback Test", "version": "2.0.0"}))
with guard:
active -= 1
return True
results = []
def worker():
results.append(mgr._reinstall_with_rollback(PLUGIN_ID, plugin_dir))
with patch.object(mgr, "install_plugin", side_effect=fake_install):
threads = [threading.Thread(target=worker) for _ in range(2)]
for t in threads:
t.start()
for t in threads:
t.join(timeout=5)
assert max_active == 1, "install_plugin ran concurrently for the same plugin_id"
assert results == [True, True]
assert plugin_dir.exists()
assert "new version" in (plugin_dir / "manager.py").read_text()
leftovers = [p for p in plugin_dir.parent.iterdir()
if "standalone-backup" in p.name]
assert leftovers == []
def test_aside_name_is_invisible_to_discovery(self, store, tmp_path):
"""The aside still contains a manifest.json — discovery must skip it
(relies on the existing '.standalone-backup-' exclusion)."""
mgr, plugin_dir = store
from src.plugin_system.plugin_manager import PluginManager
aside = plugin_dir.parent / f"{PLUGIN_ID}.standalone-backup-migrating"
plugin_dir.rename(aside)
pm = PluginManager(plugins_dir=str(tmp_path), config_manager=None,
display_manager=None, cache_manager=None)
found = pm._scan_directory_for_plugins(Path(tmp_path))
assert PLUGIN_ID not in found
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+45
View File
@@ -0,0 +1,45 @@
"""
Unit tests for src/plugin_system/testing/mocks.py.
MockCacheManager/MockPluginManager stand in for the real production
managers under the plugin safety harness -- a missing method here isn't a
harness bug in the abstract, it's a plugin silently failing to render
under test (confirmed on ledmatrix-leaderboard, which calls
get_cached_data_with_strategy() and previously hit an AttributeError that
its own broad except swallowed, producing an empty-but-green render).
"""
from src.plugin_system.testing.mocks import MockCacheManager
class TestMockCacheManagerStrategyMethod:
def test_get_cached_data_with_strategy_returns_cached_value(self):
cm = MockCacheManager()
cm.set("standings_nfl", {"teams": ["KC", "BUF"]})
result = cm.get_cached_data_with_strategy("standings_nfl", "sports_live")
assert result == {"teams": ["KC", "BUF"]}
def test_get_cached_data_with_strategy_returns_none_when_missing(self):
cm = MockCacheManager()
assert cm.get_cached_data_with_strategy("missing_key") is None
def test_get_cached_data_with_strategy_defaults_data_type(self):
cm = MockCacheManager()
cm.set("k", "v")
assert cm.get_cached_data_with_strategy("k") == "v"
def test_calls_are_tracked(self):
cm = MockCacheManager()
cm.get_cached_data_with_strategy("k", "sports_live")
assert cm.get_cached_data_with_strategy_calls == [{"key": "k", "data_type": "sports_live"}]
def test_save_cache_is_readable_via_strategy_lookup(self):
cm = MockCacheManager()
cm.save_cache("standings_nfl", {"teams": ["KC", "BUF"]})
assert cm.get_cached_data_with_strategy("standings_nfl") == {"teams": ["KC", "BUF"]}
def test_reset_clears_strategy_call_tracking(self):
cm = MockCacheManager()
cm.get_cached_data_with_strategy("k", "sports_live")
cm.reset()
assert cm.get_cached_data_with_strategy_calls == []
@@ -0,0 +1,72 @@
"""
Regression tests for RenderPipeline.should_recompose()'s pending-updates check.
PR #299 added a check so a plugin's live score/status change (a "pending
update" in StreamManager) triggers a hot-swap within a few seconds instead
of waiting for a full scroll cycle to complete. PR #330 (multi-display sync)
refactored should_recompose() and dropped that check entirely -- not just
gated behind the new sync-mode deferral it added, but removed outright, so
even standalone (non-sync) installations silently lost live-refresh and fell
back to waiting for full cycle boundaries (which, depending on
min/max_cycle_duration, can be minutes).
"""
from unittest.mock import MagicMock
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.render_pipeline import RenderPipeline
class FakeDisplayManager:
width = 64
height = 32
def _make_pipeline(sync_manager=None):
stream_manager = MagicMock()
stream_manager.get_buffer_status.return_value = {'staging_count': 0}
pipeline = RenderPipeline(VegasModeConfig(), FakeDisplayManager(), stream_manager)
pipeline.sync_manager = sync_manager
return pipeline, stream_manager
class TestShouldRecompose:
def test_cycle_complete_always_recomposes(self):
pipeline, stream_manager = _make_pipeline()
pipeline._cycle_complete = True
stream_manager.has_pending_updates_for_visible_segments.return_value = False
assert pipeline.should_recompose() is True
def test_no_pending_updates_no_staging_does_not_recompose(self):
pipeline, stream_manager = _make_pipeline()
stream_manager.has_pending_updates_for_visible_segments.return_value = False
assert pipeline.should_recompose() is False
def test_pending_updates_on_visible_segment_triggers_recompose(self):
"""The actual regression: a live-updated plugin currently in view
must trigger a recompose instead of waiting for cycle end."""
pipeline, stream_manager = _make_pipeline()
stream_manager.has_pending_updates_for_visible_segments.return_value = True
assert pipeline.should_recompose() is True
def test_staging_buffer_content_triggers_recompose(self):
pipeline, stream_manager = _make_pipeline()
stream_manager.get_buffer_status.return_value = {'staging_count': 1}
stream_manager.has_pending_updates_for_visible_segments.return_value = False
assert pipeline.should_recompose() is True
def test_sync_active_defers_pending_updates_to_cycle_boundary(self):
"""Sync-mode deferral (PR #330's actual intent) must still hold:
pending updates alone must NOT trigger a mid-cycle hot-swap when a
follower display is attached, since that causes a visible
freeze+jump on the follower. This must keep working after
restoring the non-sync pending-updates check above."""
pipeline, stream_manager = _make_pipeline(sync_manager=MagicMock())
stream_manager.has_pending_updates_for_visible_segments.return_value = True
assert pipeline.should_recompose() is False
def test_sync_active_still_recomposes_on_cycle_complete(self):
pipeline, stream_manager = _make_pipeline(sync_manager=MagicMock())
pipeline._cycle_complete = True
stream_manager.has_pending_updates_for_visible_segments.return_value = True
assert pipeline.should_recompose() is True
+92
View File
@@ -0,0 +1,92 @@
"""Tests for check_and_manage_ap_mode_with_state (src/wifi_manager.py).
The wifi monitor daemon used to fetch WiFi status before AND after each
check on top of the check's own internal fetch — every fetch is several
nmcli subprocess forks, every 30s, forever. The new API returns the state
the check observed, so the daemon runs exactly one fetch battery per tick.
"""
import os
import sys
from unittest.mock import MagicMock, patch
import pytest
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
from src.wifi_manager import WiFiManager, WiFiStatus # noqa: E402
@pytest.fixture
def wm(tmp_path):
with patch.object(WiFiManager, "_load_config", return_value={}, create=True):
manager = WiFiManager.__new__(WiFiManager)
# minimal attribute setup without running the real __init__
manager.config = {"auto_enable_ap_mode": True}
manager._disconnected_checks = 0
manager._disconnected_checks_required = 3
manager._ap_enabled_at = None
return manager
def _wire(wm, connected, ethernet, ap_active):
wm._get_wifi_status_with_retry = MagicMock(
return_value=WiFiStatus(connected=connected, ssid="net" if connected else None))
wm._is_ethernet_connected = MagicMock(return_value=ethernet)
wm._is_ap_mode_active = MagicMock(return_value=ap_active)
wm.enable_ap_mode = MagicMock(return_value=(True, "ok"))
wm.disable_ap_mode = MagicMock(return_value=(True, "ok"))
wm.scan_networks = MagicMock(return_value=([], False))
wm._save_cached_scan = MagicMock()
wm._FORCE_AP_FLAG_PATH = MagicMock()
wm._FORCE_AP_FLAG_PATH.exists.return_value = False
class TestWithState:
def test_single_fetch_per_call(self, wm):
_wire(wm, connected=True, ethernet=False, ap_active=False)
wm.check_and_manage_ap_mode_with_state()
assert wm._get_wifi_status_with_retry.call_count == 1
assert wm._is_ethernet_connected.call_count == 1
assert wm._is_ap_mode_active.call_count == 1
def test_returns_observed_state(self, wm):
_wire(wm, connected=True, ethernet=True, ap_active=False)
changed, status, ethernet, ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is False
assert status.connected is True
assert ethernet is True
assert ap_after is False
def test_ap_after_inverts_on_disable(self, wm):
"""WiFi reconnects while AP is up -> auto-disable -> ap_after False."""
_wire(wm, connected=True, ethernet=False, ap_active=True)
changed, _status, _ethernet, ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is True
assert ap_after is False
wm.disable_ap_mode.assert_called_once()
def test_ap_after_inverts_on_enable(self, wm):
"""Grace period exhausted with nothing connected -> enable -> True."""
_wire(wm, connected=False, ethernet=False, ap_active=False)
wm._disconnected_checks = 2 # this call is the 3rd
changed, _status, _ethernet, ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is True
assert ap_after is True
wm.enable_ap_mode.assert_called_once()
def test_bool_wrapper_is_back_compatible(self, wm):
_wire(wm, connected=True, ethernet=False, ap_active=False)
assert wm.check_and_manage_ap_mode() is False
_wire(wm, connected=True, ethernet=False, ap_active=True)
assert wm.check_and_manage_ap_mode() is True
def test_exception_path_never_raises(self, wm):
wm._get_wifi_status_with_retry = MagicMock(side_effect=RuntimeError("nmcli gone"))
changed, status, _ethernet, _ap_after = wm.check_and_manage_ap_mode_with_state()
assert changed is False
assert status.connected is False
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
-252
View File
@@ -1,252 +0,0 @@
"""Tests for POST /api/v3/plugins/preview — the config-page live preview.
The endpoint renders a plugin headlessly (pure PIL, no hardware, no pip)
with a CANDIDATE config: either the current form state (parsed by the same
parse_plugin_config_form used by save, so preview and save can never
disagree) or a JSON config body.
"""
import base64
import io
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from flask import Flask
from PIL import Image
project_root = Path(__file__).parent.parent.parent
sys.path.insert(0, str(project_root))
from web_interface.blueprints import api_v3 as api_v3_module # noqa: E402
from web_interface.blueprints.api_v3 import api_v3 # noqa: E402
PLUGIN_ID = "preview-test-plugin"
MANAGER_PY = '''
from PIL import ImageFont
from src.plugin_system.base_plugin import BasePlugin
class PreviewTestPlugin(BasePlugin):
def update(self):
return True
def display(self, force_clear=False):
if force_clear:
self.display_manager.clear()
text = self.config.get("message", "hello")
self.display_manager.draw.text((1, 1), text, fill=(255, 255, 255))
self.display_manager.update_display()
'''
MANIFEST = {
"id": PLUGIN_ID,
"name": "Preview Test Plugin",
"version": "1.0.0",
"class_name": "PreviewTestPlugin",
"entry_point": "manager.py",
"display_modes": ["preview_test"],
}
SCHEMA = {
"type": "object",
"properties": {
"enabled": {"type": "boolean", "default": True},
"message": {"type": "string", "default": "hello"},
},
}
@pytest.fixture
def plugin_dir(tmp_path):
plugin = tmp_path / PLUGIN_ID
plugin.mkdir()
(plugin / "manager.py").write_text(MANAGER_PY)
(plugin / "manifest.json").write_text(json.dumps(MANIFEST))
(plugin / "config_schema.json").write_text(json.dumps(SCHEMA))
return plugin
@pytest.fixture
def client(plugin_dir, tmp_path):
from src.plugin_system.schema_manager import SchemaManager
test_app = Flask(__name__)
test_app.register_blueprint(api_v3, url_prefix="/api/v3")
config_manager = MagicMock()
config_manager.load_config.return_value = {
"display": {"hardware": {"cols": 64, "chain_length": 2,
"rows": 32, "parallel": 1}},
PLUGIN_ID: {"enabled": False, "message": "saved"},
}
plugin_manager = MagicMock()
plugin_manager.plugins_dir = str(tmp_path)
old = (getattr(api_v3_module.api_v3, "config_manager", None),
getattr(api_v3_module.api_v3, "plugin_manager", None),
getattr(api_v3_module.api_v3, "schema_manager", None))
api_v3_module.api_v3.config_manager = config_manager
api_v3_module.api_v3.plugin_manager = plugin_manager
api_v3_module.api_v3.schema_manager = SchemaManager(plugins_dir=tmp_path)
with test_app.test_client() as c:
yield c
(api_v3_module.api_v3.config_manager,
api_v3_module.api_v3.plugin_manager,
api_v3_module.api_v3.schema_manager) = old
def _decode_image(data_url):
assert data_url.startswith("data:image/png;base64,")
raw = base64.b64decode(data_url.split(",", 1)[1])
return Image.open(io.BytesIO(raw))
class TestPreviewEndpoint:
def test_json_body_renders_at_default_panel_size(self, client):
"""No width/height -> the user's real panel (64*2 x 32*1)."""
resp = client.post(f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}",
json={"config": {"message": "hi"}})
assert resp.status_code == 200
data = resp.get_json()["data"]
img = _decode_image(data["image"])
assert img.size == (128, 32)
assert data["errors"] == []
def test_explicit_size(self, client):
resp = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=64&height=64",
json={"config": {}})
assert resp.status_code == 200
img = _decode_image(resp.get_json()["data"]["image"])
assert img.size == (64, 64)
def test_form_encoding_matches_json(self, client):
"""The form path (what HTMX posts) and the JSON path must render
the same candidate config identically."""
via_json = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=128&height=32",
json={"config": {"message": "same"}})
via_form = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=128&height=32",
data={"message": "same"})
a = _decode_image(via_json.get_json()["data"]["image"])
b = _decode_image(via_form.get_json()["data"]["image"])
assert list(a.getdata()) == list(b.getdata())
def test_candidate_config_wins_over_saved(self, client):
"""The preview must show the UNSAVED form state, not the saved
config ('saved' vs 'candidate' render differently)."""
saved = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=128&height=32",
json={"config": {}})
candidate = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=128&height=32",
json={"config": {"message": "candidate"}})
a = _decode_image(saved.get_json()["data"]["image"])
b = _decode_image(candidate.get_json()["data"]["image"])
assert list(a.getdata()) != list(b.getdata())
def test_disabled_plugin_still_previews(self, client):
"""Saved config has enabled: False — preview forces enabled."""
resp = client.post(f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}",
json={"config": {}})
assert resp.status_code == 200
assert resp.get_json()["data"]["errors"] == []
def test_preview_size_form_field(self, client):
"""The UI size selector posts __preview_size=WxH via hx-vals (htmx
caches hx-post's path, so it can't ride the query string). It must
set the render size and must NOT leak into the candidate config."""
resp = client.post(f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}",
data={"message": "hi", "__preview_size": "64x64"})
assert resp.status_code == 200
data = resp.get_json()["data"]
img = _decode_image(data["image"])
assert img.size == (64, 64)
assert data["errors"] == []
def test_query_args_beat_preview_size_field(self, client):
resp = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=128&height=32",
data={"message": "hi", "__preview_size": "64x64"})
img = _decode_image(resp.get_json()["data"]["image"])
assert img.size == (128, 32)
def test_malformed_preview_size_falls_back_to_panel(self, client):
resp = client.post(f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}",
data={"message": "hi", "__preview_size": "bogus x"})
img = _decode_image(resp.get_json()["data"]["image"])
assert img.size == (128, 32) # cols*chain x rows*parallel
def test_json_candidate_deep_merges_onto_saved_config(self, client):
"""A partial JSON candidate must not wipe saved sibling values in
the same nested section (form path and save both deep-merge)."""
# Saved config has message "saved"; posting an unrelated nested key
# must not discard it — render must still differ from a candidate
# that explicitly changes message.
keep_saved = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=128&height=32",
json={"config": {}})
explicit = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=128&height=32",
json={"config": {"message": "saved"}})
a = _decode_image(keep_saved.get_json()["data"]["image"])
b = _decode_image(explicit.get_json()["data"]["image"])
assert list(a.getdata()) == list(b.getdata())
def test_hanging_plugin_times_out(self, client, plugin_dir, monkeypatch):
"""A plugin whose display() hangs must not pin the web worker.
Uses its own plugin id: the loader caches the module per id, so
reusing PLUGIN_ID would run the already-imported (non-hanging) code
when this test follows others in the suite.
"""
from web_interface.blueprints import api_v3 as api_v3_module
monkeypatch.setattr(api_v3_module, "PREVIEW_RENDER_TIMEOUT_SEC", 1)
hang_id = "preview-hang-plugin"
hang_dir = plugin_dir.parent / hang_id
hang_dir.mkdir()
(hang_dir / "manager.py").write_text(MANAGER_PY.replace(
"self.display_manager.update_display()",
"import time; time.sleep(10); self.display_manager.update_display()"))
manifest = dict(MANIFEST, id=hang_id, name="Hang Plugin")
(hang_dir / "manifest.json").write_text(json.dumps(manifest))
(hang_dir / "config_schema.json").write_text(json.dumps(SCHEMA))
resp = client.post(f"/api/v3/plugins/preview?plugin_id={hang_id}",
json={"config": {}})
assert resp.status_code == 504
def test_htmx_gets_html_fragment(self, client):
resp = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=64&height=32",
data={"message": "hi"}, headers={"HX-Request": "true"})
assert resp.status_code == 200
assert resp.mimetype == "text/html"
body = resp.get_data(as_text=True)
assert "<img" in body and "data:image/png;base64," in body
def test_unknown_plugin_404(self, client):
resp = client.post("/api/v3/plugins/preview?plugin_id=nope",
json={"config": {}})
assert resp.status_code == 404
def test_missing_plugin_id_400(self, client):
resp = client.post("/api/v3/plugins/preview", json={"config": {}})
assert resp.status_code == 400
def test_absurd_size_rejected(self, client):
resp = client.post(
f"/api/v3/plugins/preview?plugin_id={PLUGIN_ID}&width=99999&height=32",
json={"config": {}})
assert resp.status_code == 400
if __name__ == "__main__":
sys.exit(pytest.main([__file__, "-v"]))
+16 -1
View File
@@ -608,9 +608,23 @@ def display_preview_generator():
import base64
from PIL import Image
import io
snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path matches display_manager; only read here
# Viewer marker: this generator only runs while the broadcaster has
# subscribers (it exits with no clients), so touching the marker each
# loop tells the DISPLAY service a browser is actually watching — it
# only pays for full-rate PNG snapshot encodes while this stays fresh
# (see src/common/snapshot_policy.py).
viewer_marker_path = "/tmp/led_matrix_preview_viewer" # nosec B108 - fixed path matches display_manager
last_modified = None
def _touch_viewer_marker():
try:
with open(viewer_marker_path, 'a'):
pass
os.utime(viewer_marker_path, None)
except OSError:
pass # display side treats a missing marker as "no viewer"
# Get display dimensions from config
try:
@@ -627,6 +641,7 @@ def display_preview_generator():
while True:
try:
_touch_viewer_marker()
# Check if snapshot file exists and has been modified
if os.path.exists(snapshot_path):
current_modified = os.path.getmtime(snapshot_path)
File diff suppressed because it is too large Load Diff
-6
View File
@@ -706,12 +706,6 @@ def _load_plugin_config_partial(plugin_id):
try:
with open(schema_path, 'r', encoding='utf-8') as f:
schema = json.load(f)
# Expand x-style-elements declarations into full property
# blocks — the same expansion SchemaManager.load_schema
# applies on the API paths. The form must render the exact
# shape the save path parses and validates against.
from src.element_style import expand_style_elements
schema = expand_style_elements(schema)
except Exception as e:
logger.warning("Could not load schema for plugin: %s", e)
+167 -31
View File
@@ -4,6 +4,17 @@
<p class="mt-1 text-sm text-gray-600">View real-time logs from the LED matrix service for troubleshooting.</p>
</div>
<!-- Currently displayed plugin -->
<div id="current-plugin-banner" class="hidden mb-4 flex items-center gap-2 text-sm bg-blue-50 border border-blue-200 text-blue-800 rounded-lg px-3 py-2">
<i class="fas fa-tv"></i>
<span>Now showing: <strong id="current-plugin-mode">-</strong>
<span id="current-plugin-id-wrap" class="hidden">(plugin: <code id="current-plugin-id" class="bg-blue-100 px-1 rounded"></code>)</span>
</span>
<button id="current-plugin-filter-btn" class="hidden ml-auto btn bg-blue-600 hover:bg-blue-700 text-white px-2 py-0.5 rounded text-xs">
Filter to this plugin
</button>
</div>
<!-- Controls -->
<div class="flex flex-wrap items-center justify-between gap-4 mb-6">
<div class="flex items-center space-x-4">
@@ -27,6 +38,11 @@
<option value="INFO">Info & Above</option>
</select>
<!-- Plugin Filter -->
<select id="log-plugin-filter" class="form-control text-sm">
<option value="">All Plugins</option>
</select>
<!-- Search -->
<div class="relative">
<input type="text" id="log-search" placeholder="Search logs..." class="form-control text-sm pl-8 pr-4 py-1 w-48">
@@ -139,6 +155,7 @@ window._filteredLogs = [];
const realtimeToggle = document.getElementById('log-realtime-toggle');
const refreshBtn = document.getElementById('refresh-logs-btn');
const levelFilter = document.getElementById('log-level-filter');
const pluginFilter = document.getElementById('log-plugin-filter');
const searchInput = document.getElementById('log-search');
const autoscrollToggle = document.getElementById('log-autoscroll');
const clearBtn = document.getElementById('clear-logs-btn');
@@ -160,6 +177,11 @@ window._filteredLogs = [];
levelFilter.parentNode.replaceChild(newFilter, levelFilter);
newFilter.addEventListener('change', filterLogs);
}
if (pluginFilter) {
const newFilter = pluginFilter.cloneNode(true);
pluginFilter.parentNode.replaceChild(newFilter, pluginFilter);
newFilter.addEventListener('change', filterLogs);
}
if (searchInput) {
const newInput = searchInput.cloneNode(true);
searchInput.parentNode.replaceChild(newInput, searchInput);
@@ -180,6 +202,24 @@ window._filteredLogs = [];
downloadBtn.parentNode.replaceChild(newBtn, downloadBtn);
newBtn.addEventListener('click', downloadLogs);
}
const currentPluginFilterBtn = document.getElementById('current-plugin-filter-btn');
if (currentPluginFilterBtn) {
const newBtn = currentPluginFilterBtn.cloneNode(true);
currentPluginFilterBtn.parentNode.replaceChild(newBtn, currentPluginFilterBtn);
newBtn.addEventListener('click', function() {
const pluginFilterEl = document.getElementById('log-plugin-filter');
if (pluginFilterEl && window._currentPluginId) {
pluginFilterEl.value = window._currentPluginId;
filterLogs();
}
});
}
refreshCurrentPluginStatus();
if (window._currentPluginPollTimer) {
clearInterval(window._currentPluginPollTimer);
}
window._currentPluginPollTimer = setInterval(refreshCurrentPluginStatus, 5000);
// Handle window resize for responsive height
window.addEventListener('resize', function() {
@@ -284,37 +324,25 @@ function processLogs(logsText, append = false) {
// Skip empty lines
if (!line.trim()) return;
// Try to parse journalctl format: "MMM DD HH:MM:SS hostname service[pid]: message"
// Example: "Oct 13 14:23:45 raspberrypi ledmatrix[1234]: INFO: Starting display"
// journalctl (--output=short-iso) emits: "YYYY-MM-DDTHH:MM:SS+ZZZZ hostname service[pid]: message"
// Example: "2024-01-15T10:23:45+0000 raspberrypi ledmatrix[1234]: INFO - plugin.nhl_scoreboard - [Plugin: nhl_scoreboard] Updated scores"
// Also accept the older syslog-style "MMM DD HH:MM:SS" timestamp for compatibility.
let timestamp = '';
let level = 'INFO';
let message = line;
// Extract timestamp (first part before hostname)
const timestampMatch = line.match(/^([A-Z][a-z]{2}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})/);
if (timestampMatch) {
timestamp = timestampMatch[1];
// Find the message part (after service name and pid)
const messageMatch = line.match(/:\s*(.+)$/);
if (messageMatch) {
message = messageMatch[1];
// Detect log level from message
if (message.match(/\b(ERROR|CRITICAL|FATAL)\b/i)) {
level = 'ERROR';
} else if (message.match(/\b(WARNING|WARN)\b/i)) {
level = 'WARNING';
} else if (message.match(/\bDEBUG\b/i)) {
level = 'DEBUG';
} else if (message.match(/\bINFO\b/i)) {
level = 'INFO';
}
// Clean up level prefix from message if it exists
message = message.replace(/^(ERROR|WARNING|WARN|INFO|DEBUG):\s*/i, '');
}
let plugin = '';
let rest = null;
const isoMatch = line.match(/^(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:[+-]\d{2}:?\d{2}|Z))\s+(.*)$/);
const syslogMatch = !isoMatch && line.match(/^([A-Z][a-z]{2}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+(.*)$/);
if (isoMatch) {
timestamp = isoMatch[1];
rest = isoMatch[2];
} else if (syslogMatch) {
timestamp = syslogMatch[1];
rest = syslogMatch[2];
} else {
// If no timestamp, use current time
timestamp = new Date().toLocaleString('en-US', {
@@ -327,14 +355,58 @@ function processLogs(logsText, append = false) {
});
}
if (rest !== null) {
// Find the message part (after hostname + service name/pid)
const messageMatch = rest.match(/:\s*(.+)$/);
if (messageMatch) {
message = messageMatch[1];
// Detect log level from message
if (message.match(/\b(ERROR|CRITICAL|FATAL)\b/i)) {
level = 'ERROR';
} else if (message.match(/\b(WARNING|WARN)\b/i)) {
level = 'WARNING';
} else if (message.match(/\bDEBUG\b/i)) {
level = 'DEBUG';
} else if (message.match(/\bINFO\b/i)) {
level = 'INFO';
}
// Clean up level prefix from message if it exists
message = message.replace(/^(ERROR|WARNING|WARN|INFO|DEBUG)\s*[-:]\s*/i, '');
// journalctl already carries its own timestamp; strip the app's
// internal "YYYY-MM-DD HH:MM:SS.mmm - LEVEL - logger.name - "
// prefix (see ContextualFormatter in src/logging_config.py) so
// it isn't duplicated in the displayed message.
message = message.replace(
/^\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}(?:\.\d+)?\s*-\s*(ERROR|WARNING|WARN|INFO|DEBUG|CRITICAL)\s*-\s*[\w.]+\s*-\s*/i,
''
);
// Extract the "[Plugin: <id>]" context tag emitted by
// ContextualFormatter for every plugin logger (see
// src/logging_config.py PluginLoggerAdapter), and pull it out
// of the displayed message into its own field.
const pluginMatch = message.match(/^\[Plugin:\s*([^\]]+)\]\s*/);
if (pluginMatch) {
plugin = pluginMatch[1].trim();
message = message.replace(pluginMatch[0], '');
}
} else {
message = rest;
}
}
const logEntry = {
timestamp: timestamp,
level: level,
plugin: plugin,
message: message,
raw: line,
id: Date.now() + Math.random()
};
// Don't add duplicate entries when appending
if (!append || !window._allLogs.find(log => log.raw === line)) {
window._allLogs.push(logEntry);
@@ -346,9 +418,26 @@ function processLogs(logsText, append = false) {
window._allLogs = window._allLogs.slice(-window._MAX_LOGS);
}
updatePluginFilterOptions();
filterLogs();
}
function updatePluginFilterOptions() {
const pluginFilterEl = document.getElementById('log-plugin-filter');
if (!pluginFilterEl) return;
const previousValue = pluginFilterEl.value;
const plugins = Array.from(new Set(window._allLogs.map(log => log.plugin).filter(Boolean))).sort();
pluginFilterEl.innerHTML = '<option value="">All Plugins</option>' +
plugins.map(p => `<option value="${escapeHtml(p)}">${escapeHtml(p)}</option>`).join('');
// Restore previous selection if it's still a valid option
if (previousValue && plugins.includes(previousValue)) {
pluginFilterEl.value = previousValue;
}
}
function renderLogs() {
if (window._filteredLogs.length === 0) {
showEmptyState();
@@ -364,10 +453,14 @@ function renderLogs() {
window._filteredLogs.forEach(log => {
const logElement = document.createElement('div');
logElement.className = `log-entry py-1 px-2 hover:bg-gray-800 rounded transition-colors duration-150 ${getLogLevelClass(log.level)}`;
const pluginBadge = log.plugin
? `<span class="log-plugin flex-shrink-0 px-2 py-0.5 rounded text-xs font-semibold bg-purple-700 text-white" title="Plugin: ${escapeHtml(log.plugin)}">${escapeHtml(log.plugin)}</span>`
: '';
logElement.innerHTML = `
<div class="flex items-start gap-3 text-xs font-mono">
<span class="log-timestamp text-gray-400 flex-shrink-0 w-32">${escapeHtml(log.timestamp)}</span>
<span class="log-level flex-shrink-0 px-2 py-0.5 rounded text-xs font-semibold ${getLogLevelBadgeClass(log.level)}">${log.level}</span>
${pluginBadge}
<span class="log-message flex-1 ${getLogLevelTextClass(log.level)} break-words">${escapeHtml(log.message)}</span>
</div>
`;
@@ -410,10 +503,12 @@ function getLogLevelTextClass(level) {
function filterLogs() {
const levelFilterEl = document.getElementById('log-level-filter');
const pluginFilterEl = document.getElementById('log-plugin-filter');
const searchEl = document.getElementById('log-search');
if (!levelFilterEl || !searchEl) return;
const levelFilter = levelFilterEl.value;
const pluginFilter = pluginFilterEl ? pluginFilterEl.value : '';
const searchTerm = searchEl.value.toLowerCase();
window._filteredLogs = window._allLogs.filter(log => {
@@ -430,8 +525,14 @@ function filterLogs() {
}
}
// Plugin filter
if (pluginFilter && log.plugin !== pluginFilter) {
return false;
}
// Search filter
if (searchTerm && !log.message.toLowerCase().includes(searchTerm)) {
if (searchTerm && !log.message.toLowerCase().includes(searchTerm) &&
!(log.plugin && log.plugin.toLowerCase().includes(searchTerm))) {
return false;
}
@@ -617,10 +718,45 @@ function escapeHtml(text) {
return div.innerHTML;
}
function refreshCurrentPluginStatus() {
fetch('/api/v3/display/current-status')
.then(response => response.json())
.then(data => {
if (data.status !== 'success' || !data.data) return;
const state = data.data;
const banner = document.getElementById('current-plugin-banner');
const modeEl = document.getElementById('current-plugin-mode');
const idWrap = document.getElementById('current-plugin-id-wrap');
const idEl = document.getElementById('current-plugin-id');
const filterBtn = document.getElementById('current-plugin-filter-btn');
if (!banner || !modeEl) return;
window._currentPluginId = state.plugin_id || null;
modeEl.textContent = state.mode || 'unknown';
if (state.plugin_id) {
idEl.textContent = state.plugin_id;
idWrap.classList.remove('hidden');
if (filterBtn) filterBtn.classList.remove('hidden');
} else {
idWrap.classList.add('hidden');
if (filterBtn) filterBtn.classList.add('hidden');
}
banner.classList.remove('hidden');
})
.catch(() => {
// Silently ignore - banner just stays hidden/stale
});
}
// Cleanup on page unload
window.addEventListener('beforeunload', function() {
if (window._logsEventSource) {
window._logsEventSource.close();
}
if (window._currentPluginPollTimer) {
clearInterval(window._currentPluginPollTimer);
window._currentPluginPollTimer = null;
}
});
</script>
@@ -994,51 +994,8 @@
<p class="text-xs text-amber-600">Plugin is disabled, but on-demand will temporarily enable it.</p>
{% endif %}
</div>
{# Live Preview — renders the plugin headlessly with the CURRENT
(unsaved) form values, at the real panel size or a chosen one #}
<div class="mt-4 pt-4 border-t border-gray-200 space-y-3">
<div class="flex items-center gap-2">
<i class="fas fa-eye text-blue-500"></i>
<span class="text-sm font-semibold text-gray-900">Live Preview</span>
</div>
<div class="flex flex-wrap items-center gap-2">
{# No name attribute: must not be submitted with the config
save. The size travels via the button's hx-vals (read at
request time — htmx caches hx-post's path at process
time, so mutating the attribute onchange doesn't work). #}
<select id="preview-size-{{ plugin.id }}"
aria-label="Preview panel size"
class="text-sm rounded-md border-gray-300 py-1.5">
<option value="">My panel size</option>
<option value="64x32">64 &times; 32</option>
<option value="128x32">128 &times; 32</option>
<option value="128x64">128 &times; 64</option>
<option value="192x48">192 &times; 48</option>
<option value="256x128">256 &times; 128</option>
</select>
<button type="button"
id="preview-btn-{{ plugin.id }}"
hx-post="/api/v3/plugins/preview?plugin_id={{ plugin.id }}"
hx-include="closest form"
hx-vals='js:{"__preview_size": document.getElementById("preview-size-{{ plugin.id }}").value}'
hx-target="#plugin-preview-{{ plugin.id }}"
hx-swap="innerHTML"
hx-indicator="#preview-indicator-{{ plugin.id }}"
class="px-3 py-2 text-sm bg-blue-600 hover:bg-blue-700 text-white rounded-md flex items-center gap-2 transition-colors">
<i class="fas fa-tv"></i>
<span>Preview</span>
</button>
<span id="preview-indicator-{{ plugin.id }}" class="htmx-indicator text-xs text-gray-500">
<i class="fas fa-spinner fa-spin"></i> rendering&hellip;
</span>
</div>
<div id="plugin-preview-{{ plugin.id }}">
<p class="text-xs text-gray-400">Renders this plugin with the current (unsaved) settings &mdash; try it before you save.</p>
</div>
</div>
</div>
{# Configuration Form Panel #}
<div class="bg-gray-50 rounded-lg p-4">
<h3 class="text-md font-medium text-gray-900 mb-3">Configuration</h3>