32 Commits

Author SHA1 Message Date
Chuck
f67b9c25f1 fix(tests): thread cleanup on assertion failure, reduce oversized image
- test_health_monitor.py: wrap start_monitoring calls in try/finally so
  the background thread is always stopped even when an assertion fails
- test_scroll_helper.py: reduce 50,000px test image to 5,000px to avoid
  unnecessary memory pressure on Raspberry Pi

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 09:37:49 -04:00
Claude
4977c5fbc9 test: add 306 new tests covering previously untested modules
Adds test coverage for six major untested areas:
- src/base_classes/api_extractors.py — ESPN football, baseball, hockey, soccer extractors
- src/base_classes/data_sources.py — ESPN, MLB, and soccer API data sources (HTTP mocked)
- src/common/game_helper.py — game extraction, filtering, sorting, and summaries
- src/common/utils.py — all utility functions (normalise, format, validate, parse)
- src/common/scroll_helper.py — ScrollHelper init, create, update, visible portion, duration
- src/background_data_service.py — cache hit/miss paths, retry, cancel, cleanup, singleton
- src/vegas_mode/config.py — VegasModeConfig from_config, validate, update, ordering
- src/logo_downloader.py — normalize_abbreviation, filename variations, directory helpers
- src/plugin_system/health_monitor.py — HealthStatus determination, metrics, suggestions, lifecycle

https://claude.ai/code/session_015792DiGo27JbgH5mk3KBjk
2026-05-24 02:45:27 +00:00
Chuck
327e87f735 fix(pi5): auto-detect Pi 5 and force rgbmatrix rebuild when rp1_rio missing (#341)
* fix(pi5): auto-detect Pi 5 and force rgbmatrix rebuild when rp1_rio missing

first_time_install.sh:
- Detect Pi 5 from /proc/device-tree/model at startup
- Step 6 skip logic now also checks hasattr(RGBMatrixOptions(), 'rp1_rio'):
  if the installed library lacks rp1_rio (built before Pi 5 support was added)
  the build is forced even when the module is already importable. This is the
  root cause of mmap errors to 0x3f000000 (Pi 3 bus) on Pi 5 hardware.
- After a successful Pi 5 build, verify rp1_rio is present and print a
  diagnostic with the submodule update command if it's still missing.

src/display_manager.py:
- rp1_rio warning now names the symptom (mmap to 0x3f000000) and gives the
  exact fix command so users can act immediately from the log.

README.md:
- Remove "Pi 5 is unsupported" — Pi 5 is fully supported since the library
  submodule includes rp1_pio and rp1_rio backends.
- Document the forced-rebuild command for users migrating from Pi 4.
- Fix gpio_slowdown guidance: Pi 5 PIO mode uses 1–2, not 5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(install): only append Pi 5 suffix in skip-build message when IS_PI5=1

${IS_PI5:+...} expands whenever IS_PI5 is set, including when it's "0".
Replace with an explicit equality check so the suffix only appears on
actual Pi 5 installs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 16:02:58 -04:00
Chuck
b5426da2a7 fix(fonts): skip preview API call for BDF bitmap fonts (#345)
The font preview endpoint explicitly rejects .bdf files (glyph rendering
not implemented server-side), returning 400. The JS didn't know this and
called the endpoint for every selected font, causing a noisy 400 on load.

Guard added in updateFontPreview(): if the selected font ends in .bdf,
show "Preview not available for BDF bitmap fonts" and return early instead
of hitting the API.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 14:25:38 -04:00
Chuck
302ab1da4f fix(plugin-config): handle missing type key in oneOf/anyOf schema fields (#344)
* fix(web-ui): dedup registry fetches, surface reconciliation warnings, add check-update endpoint

Story 1 — src/plugin_system/store_manager.py:
Add threading.Lock (_registry_fetch_lock) to fetch_registry(). The outer cache
check remains the hot path (no lock). When the cache is cold, only one thread
hits the network; concurrent callers block on the lock then get the result from
the warm cache (double-checked locking). Eliminates duplicate GitHub requests
on every page load when the 15-minute cache expires.

Story 2 — web_interface/app.py + api_v3.py + overview.html:
_run_startup_reconciliation() now writes /tmp/ledmatrix_reconciliation.json
(atomic tempfile+replace, mirrors hw_status pattern) so the result survives
the background thread. New GET /api/v3/plugins/reconciliation-status reads
that file. Overview page gains a dismissible yellow banner that shows stale
plugin_id values (e.g. sync, github, youtube) and tells the user to remove
them or reinstall from the Plugin Store. Banner is suppressed for the session
after dismiss using sessionStorage keyed on the plugin_id list.

Story 3 — web_interface/blueprints/api_v3.py:
Add GET /api/v3/system/check-update. Does git fetch origin main then compares
local HEAD vs origin/main to compute update_available, remote_sha, and
commits_behind. Result is cached for 5 minutes so it doesn't run git on every
page load. Falls back to {update_available: false} on any error. Eliminates
the 404 logged on every page load.

Story 4 (Pi 5 rgbmatrix rebuild) was already fixed in PR #341.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-config): handle missing `type` key in schema fields using oneOf/anyOf

Jinja2's `prop.type` on a dict without a `type` key returns an Undefined
object. Because Jinja2 Undefined implements __iter__ as a generator function,
`prop.type is iterable` evaluates True, then `prop.type[0]` calls
Undefined.__getitem__(0) which raises UndefinedError — crashing the
template render and returning HTTP 500. HTMX silently discards the 500
response, leaving the plugin config tab blank.

Fix: use `prop.get('type')` which returns None for missing keys instead of
Undefined. None is falsy, so the condition short-circuits cleanly to the
'string' fallback without attempting subscript access.

Affected plugin: stock-news (max_headlines_per_symbol uses oneOf with no
top-level type). Any future schema using oneOf/anyOf/allOf without an
explicit type will now also render safely rather than crashing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): harden check-update, reconciliation status endpoint, and temp-file write

api_v3.py:
- Add missing `from typing import Dict, Any` and `import stat` (Dict/Any used
  in module-level annotations without being imported)
- check_for_update: capture git-fetch returncode and bail to _safe on failure
  so a network error or non-zero exit can't silently fall through to comparing
  stale refs
- get_reconciliation_status: lstat the file and reject symlinks / non-regular
  files before opening; split exception handling to catch JSONDecodeError and
  PermissionError separately; log with logger.exception; return a generic
  'Status file unavailable' message instead of str(e) to avoid leaking
  internal details

overview.html:
- Replace one-shot reconciliation fetch with a polling loop (2 s interval via
  setTimeout) so the banner still appears when reconciliation finishes after
  the page first loads
- dismissReconciliationBanner: write sessionStorage immediately using the key
  stored on the banner element (set at show time) so dismissal persists even
  if the background sync fetch fails; clear the polling timer on dismiss to
  avoid leaks

app.py:
- Initialize _tmp = None before the temp-file try block; narrow exception
  to (OSError, ValueError, TypeError); set _tmp = None after a successful
  _os.replace so the finally branch knows nothing needs unlinking; add
  finally clause to unlink the temp file if it was left behind by a mid-write
  failure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: reconciliation status errors return graceful not-done instead of HTTP 500; log fetch stderr

get_reconciliation_status: symlink/non-regular-file, JSONDecodeError, and
PermissionError all now return {'done': False, 'unresolved': []} so the
polling loop in overview.html keeps retrying rather than stopping on a
transient error.

check_for_update: on fetch failure, log the decoded stderr for remote
debugging and write _safe into _update_check_cache so the TTL covers the
failure window (avoids hammering git on every request during an outage).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(bandit): replace hardcoded /tmp paths with tempfile.gettempdir() (B108)

Codacy/Bandit B108 flagged two hardcoded '/tmp/' string literals in app.py
(lines 737, 741). Replaced with _tempfile.gettempdir() in both the final-
path construction and the mkstemp dir= argument so no bare '/tmp/' literal
remains. Also updated the matching reader path in api_v3.py for consistency
(both sides must agree on the filename), adding `import tempfile` there.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 15:53:16 -04:00
Chuck
9cd2bd14ce Update README.md (#342)
Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-05-19 20:47:34 -04:00
Chuck
53ee184bc5 chore: remove march-madness from bundled plugin-repos (#340)
March Madness is now available in the ledmatrix-plugins monorepo store
(ChuckBuilds/ledmatrix-plugins/plugins/march-madness) and should be
installed via the Plugin Store like any other plugin.

Removing the bundled copy so new installs don't automatically include it.
Existing users keep their installed version until they choose to uninstall.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 20:00:41 -04:00
Chuck
e00d75bbb5 Disable schedule and update timezone and location (#338)
Updated schedule settings to disable all days and changed timezone and location.

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-05-19 18:57:09 -04:00
Chuck
33f76b4895 feat(pi5): RP1 backend UI, gpio slowdown guidance, and hardware init error banner (#337)
* feat(pi5): expose RP1 backend selector, fix gpio defaults, surface init failures in web UI

- Add rp1_rio select (PIO/RIO) to Display Settings hardware config section;
  saved via /api/v3/config/main with 0-or-1 validation — previously the key
  existed in config.json but was not editable from the UI
- Update gpio_slowdown help text with per-model guidance (Pi 3: 3, Pi 4: 4,
  Pi 5: 4–5) and raise max from 5 → 10 to match full library range
- Fix gpio_slowdown Python fallback default from 2 → 3 (only affects edge case
  where the runtime config section is absent; explicit config values are unchanged)
- display_manager writes /tmp/led_matrix_hw_status.json at startup: ok/error;
  Display Settings page fetches it and shows a yellow warning banner when the
  matrix failed to initialize, including Pi 5 remediation steps
- Add GET /api/v3/hardware/status endpoint that reads the status file
- Improve fallback error log to include Pi 5 rebuild hint

Pi 3/4 users: rp1_rio=0 is set in config but silently ignored by the library
on non-RP1 hardware; all other changes are additive or tighten defaults only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(pi5): correct gpio_slowdown guidance — Pi 5 PIO default is 1, not 4-5

The upstream library defaults gpio_slowdown to 1 for Pi 5 (IsPi4() ? 2 : 1).
In PIO mode the value is a pixel-clock divisor, so 4-5 was unnecessarily
conservative advice. Updated help text and error log to reflect the actual
range (1-3 typical for Pi 5 PIO; inverted effect in RIO mode).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): atomic hw-status write, narrow bare excepts, urllib3 CVE floor

- display_manager: replace open()+bare-except with tempfile.mkstemp→fsync→
  chmod(0o600)→os.replace; adds symlink guard and logs errors via logger
  instead of swallowing them silently; pull json/tempfile to module imports
- display_manager cleanup(): narrow broad `except Exception: pass` to
  (OSError, RuntimeError, ValueError, MemoryError) with debug log
- api_v3 get_hardware_status(): catch json.JSONDecodeError and PermissionError
  explicitly; log full traceback server-side; return generic "Unable to read
  hardware status" to client instead of leaking str(e)
- march-madness/requirements.txt: bump urllib3 floor 2.2.2→2.6.3 (CVE fix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(template): apply |int filter to rp1_rio comparisons in display.html

Without |int, a string-typed value (e.g. from a hand-edited config.json)
causes both selected tests to fail and the select renders with no option
pre-selected. Matches the existing pattern used for multiplexing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:42:19 -04:00
Chuck
c6b79e11d5 fix: Codacy round-2 — urllib3 CVEs, missed JS/Python issues (#336)
urllib3 CVEs (10 Trivy findings):
  plugin-repos/march-madness/requirements.txt: bump urllib3>=1.26.0 to
  >=2.2.2 to address CVE-2021-33503, CVE-2023-43804, CVE-2023-45803,
  CVE-2024-37891, and 2025-2026 decompression/redirect CVEs.

Missed code fixes from round-1:
  display_helper.py: remove unused draw=ImageDraw.Draw(img) — the method
  delegates to _draw_centered_text which creates its own draw context.
  custom-feeds.js:334: one bare removeCustomFeedRow(this) was missed by
  the earlier replace_all; changed to window.removeCustomFeedRow(this).
  app.js: add htmx to /* global */ declaration — htmx.ajax() is called
  at lines 146 and 172 but htmx was only declared in the extension files.
  timezone-selector.js:215: second unused catch (e) → catch {} missed
  when we fixed line 361 in round-1.

Bandit B110 annotations (3 new except/pass blocks from newer PRs):
  start.py: hostname -I IP parsing — non-critical startup info.
  display_controller.py: scroll_helper.get_portion_at — optional method.
  display_manager.py: canvas reset during cleanup — best-effort.

41 confirmed false positives suppressed via Codacy API:
  35x pyflakes in test/, plugin-repos/, scripts/ — not production code
  Flask 0.0.0.0, os.execvp, Bandit B603, vendor ESLint, already-fixed
  Biome noPrototypeBuiltins.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 18:04:21 -04:00
Chuck
d941c91f24 fix(systemd): wait for network connectivity before starting services (#335)
Change After=network.target → After=network-online.target + Wants=network-
online.target in both service templates and install_web_service.sh.

network.target only guarantees NetworkManager has started — it does NOT
mean the device has an active internet connection. On boot the LED matrix
service was starting within seconds of the network interface appearing,
before WiFi association and DHCP completed, causing every first-update API
call to fail with "Network is unreachable" or DNS resolution errors.

network-online.target waits for a confirmed route before the service fires.
On Raspberry Pi OS this is provided by NetworkManager-wait-online. The
tradeoff is a few extra seconds at boot, acceptable for a display device.

Observed on devpi: service started at 14:48:03, all API calls (weather,
FlightRadar24, local ADS-B) failed at 14:48:07 with network errors, then
the service restarted cleanly at 14:50:40 once WiFi was established.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 15:47:35 -04:00
Chuck
054ad78d7b chore(deps): update rpi-rgb-led-matrix to latest upstream for Pi 5 support (#334)
* chore(deps): update rpi-rgb-led-matrix to latest upstream for Pi 5 support

Configure submodule to track upstream master branch (branch = master in
.gitmodules) so future updates are a single 'git submodule update --remote'
rather than manual SHA management.

Update first_time_install.sh to use --remote flag so fresh installs always
pull the current upstream master, not the commit recorded at clone time.

Current upstream HEAD (8907235) brings:
- PR #1886: Raspberry Pi 5 support — new RP1 PIO and RIO backends. The
  library auto-detects Pi 5 hardware at runtime; no config change required
  for basic operation. adafruit-hat-pwm is confirmed supported on Pi 5.
- PR #1833: setup.py migrated from distutils → setuptools, fixing Python
  3.12+ build failure (Pi runs Python 3.13). Previous version could not
  build the bindings at all on current Pi OS.

Expose new rp1_rio option in display_manager.py and config.template.json:
  0 (default) = PIO mode — uses Pi 5 RP1 coprocessor, minimal CPU usage
  1 = RIO mode — Registered IO, faster throughput, higher CPU; note that
      gpio_slowdown has inverted effect in this mode

No API changes to RGBMatrix, RGBMatrixOptions, or FrameCanvas. Pi 4 and
earlier hardware is unaffected — rp1_rio is silently ignored on non-Pi-5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(deps): update rpi-rgb-led-matrix install for new scikit-build-core system

The library migrated from 'make build-python' + 'pip install bindings/python'
to a scikit-build-core + cmake build where the entire repo root is pip-
installable via 'pip install .'. Update first_time_install.sh accordingly:
- Remove the 'make build-python' step (target no longer exists)
- Install directly from the repo root instead of bindings/python
- Replace build deps: remove cython3/scons/python3-dev, add python-dev-is-python3

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: deterministic submodule install + guard rp1_rio for older rgbmatrix

first_time_install.sh: remove --remote from both git submodule update
calls so first-time installs check out the pinned commit recorded in the
repo rather than whatever upstream master happens to be at install time.
The branch = master config in .gitmodules reserves --remote for an
explicit maintainer upgrade (git submodule update --remote).

display_manager.py: guard rp1_rio assignment with hasattr() so setting
the option in config does not cause an AttributeError and silently fall
through to emulator mode when running against RGBMatrixEmulator or an
older rgbmatrix build that predates the Pi 5 property. Emit a warning
instead so the operator knows the value was ignored.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 14:17:00 -04:00
Chuck
05b3fa56cb fix: Codacy security fixes, CVE dependency bumps, and code quality cleanup (#331)
* fix(deps): bump minimum versions to address CVEs

Pillow 10.4.0 → 12.2.0: CVE-2026-40192 (DoS via FITS decompression bomb),
CVE-2026-25990 (OOB write via PSD image), CVE-2026-42311/42308/42310

requests 2.32.0 → 2.33.0: CVE-2026-25645 (temp file security bypass),
CVE-2024-47081 (.netrc credentials leak)

werkzeug 3.0.0 → 3.1.6: CVE-2023-46136, CVE-2024-49766/49767,
CVE-2025-66221, CVE-2026-21860/27199 (DoS, path traversal, safe_join bypass)

Flask 3.0.0 → 3.1.3: CVE-2026-27205 (session data caching info disclosure)

spotipy 2.24.0 → 2.25.2: CVE-2025-27154, CVE-2025-66040

python-socketio 5.11.0 → 5.14.0: CVE-2025-61765

pytest 7.4.0 → 9.0.3: CVE-2025-71176 (insecure temp dir handling)

Updated in requirements.txt, web_interface/requirements.txt,
plugin-repos/starlark-apps/requirements.txt, and
plugin-repos/march-madness/requirements.txt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve Pylint errors in executor, data service, and odds call

Rename TimeoutError to PluginTimeoutError in plugin_executor.py to
avoid shadowing the built-in; no external callers affected.

Remove dead try/except in BackgroundDataService.shutdown: executor.shutdown()
never accepted a timeout kwarg so the try branch always raised TypeError.
Simplify to a direct shutdown(wait=wait) call.

Remove is_live kwarg from odds_manager.get_odds() call in sports.py;
BaseOddsManager.get_odds() has no such parameter. The live update interval
is already encoded in the update_interval_seconds argument passed alongside.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: MD5→SHA-256, shellcheck warnings, and broken doc links

config_service.py: replace MD5 with SHA-256 for config change detection;
same semantics (equality comparison), no stored hashes affected.

Shell scripts — shellcheck warnings:
- diagnose_web_interface.sh: remove useless cat (SC2002)
- dev_plugin_setup.sh: restructure A&&B||C into if/then (SC2015)
- fix_assets_permissions.sh: remove unused REAL_HOME block (SC2034)
- install_web_service.sh: remove unused USER_HOME assignment (SC2034)
- diagnose_web_ui.sh: remove unused SUDO assignments (SC2034)
- diagnose_plugin_permissions.sh: remove unused BLUE color var (SC2034)
- first_time_install.sh: remove unused CLEAR var, PACKAGE_NAME
  assignment, and replace loop variable with _ (SC2034)

docs/PLUGIN_ARCHITECTURE_SPEC.md: fix 10 broken TOC anchor links to
include section numbers matching the actual headings (MD051).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unused imports and bare exception aliases (pyflakes F401/F841)

Remove unused imports across 86 files in src/, web_interface/, test/,
and scripts/ using autoflake. No logic changes — only dead import
statements and unused names in from-imports are removed.

Also remove bare exception aliases where the variable is never
referenced in the handler body:
- src/cache/disk_cache.py: except (IOError, OSError, PermissionError) as e
- src/cache_manager.py: except (OSError, IOError, PermissionError) as perm_error
- src/plugin_system/resource_monitor.py: except Exception as e
- web_interface/app.py: except Exception as read_err

86 files changed, 205 lines removed, 18 pre-existing test failures unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unused local variable assignments (pyflakes F841)

Dead assignments removed across src/ and web_interface/:

- background_data_service: drop future= on fire-and-forget executor.submit
- base_classes/baseball: drop font= (all rendering uses self.fonts['time'])
- base_classes/hockey: drop status_short= (never referenced after assignment)
- common/cli: drop game_helper=/config_helper= bindings in import-test block;
  constructors called for instantiation-only validation
- common/display_helper: drop text_width= (x_position uses display_width
  directly); drop draw= in create_error_image (uses _draw_centered_text)
- config_manager: remove dead secrets_content loading block in migration path
  (comment already noted save_config_atomic handles secrets internally)
- display_manager: drop setup_start= (timing was never completed or read)
- font_manager: drop target_path= (catalog uses font_file_path directly);
  drop face=/font= bindings in validate_font (validation by construction —
  TypeError on failure is the signal, not the return value)
- font_test_manager: drop width=/height= (draw_text uses display_manager directly)
- plugin_system/state_reconciliation: drop manager= (only config/disk/state_mgr used)
- plugin_system/store_manager: drop result= on pip install subprocess.run
  (check=True raises on failure; stdout unused)
- web_interface/blueprints/pages_v3: drop main_config_path=""/secrets_config_path=""
  (render_template uses config_manager.get_*_path() inline)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(js): resolve ESLint no-undef warnings across 6 JS files

Three distinct patterns:

1. Vendor library globals — htmx is injected by <script> before these
   extension files load; ESLint lints files in isolation and doesn't know.
   Fix: add /* global htmx */ to htmx-sse.js and htmx-json-enc.js.

2. Cross-file globals — showNotification is defined as window.showNotification
   in app.js/notification.js but called bare in app.js and error_handler.js.
   ESLint doesn't connect window.X = Y with a bare call to X.
   Fix: add /* global showNotification */ to app.js and error_handler.js.

3. Forward-reference window.* functions — in array-table.js, checkbox-group.js,
   and custom-feeds.js, functions like removeArrayTableRow are called early
   inside event-handler closures but assigned to window.* later in the file.
   At runtime this works (the handler fires after the assignment), but ESLint
   sees the bare name at the call site.
   Fix: change bare calls to window.removeArrayTableRow(this) etc. so the
   reference is explicit and ESLint-safe.

Also guard the updateSystemStats call in app.js reconnectSSE: the function
is called but defined nowhere in the codebase. Guard with typeof check so
it won't throw ReferenceError if the reconnect path is hit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(js): resolve Biome lint warnings across 9 JS files

noUnusedVariables (catch bindings → optional catch syntax):
- app.js, file-upload.js, timezone-selector.js: } catch (e) { → } catch {
  ES2019 optional catch binding; e was unused in all three handlers

noUnusedVariables (dead assignments):
- app.js: remove const data= in display SSE stub (handler does nothing yet)
- api_client.js: remove const timeoutId= (setTimeout ID never used to cancel)
- custom-feeds.js: remove const oldIndex= (getAttribute result never read)
- schedule-picker.js: remove const compactMode= (never used in HTML build)
- select-dropdown.js: remove const icons= (icons not yet rendered in options)

noPrototypeBuiltins:
- day-selector.js: DAY_LABELS.hasOwnProperty(x) →
  Object.prototype.hasOwnProperty.call(DAY_LABELS, x)
  Safe form that works even on null-prototype objects

useIterableCallbackReturn:
- file-upload.js, notification.js: forEach(x => expr) →
  forEach(x => { expr; }) — forEach ignores return values;
  implicit return from arrow body was misleading

htmx-sse.js is a vendor extension file with old-style var/== patterns
that are correct for it; 18 Biome issues suppressed via Codacy API
rather than modifying the vendor source.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): escape user input in raw HTML responses in pages_v3.py

plugin_id comes directly from the URL path
(/partials/plugin-config/<plugin_id>) and was interpolated into an HTML
fragment without escaping. A crafted URL like
/partials/plugin-config/<script>alert(1)</script> would inject that
tag into the DOM via the HTMX partial response.

Fix: wrap all user-controlled values in markupsafe.escape() before
embedding in raw HTML strings. Affects the plugin-not-found 404
response and both error 500 responses in the plugin config partial.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address Bandit B108/B110 across production code

B110 (try/except/pass):
- display_controller.py: narrow 'except Exception' to 'except AttributeError'
  for get_offset_frame() — plugins not having this optional method is the
  expected case, not all exceptions
- config_manager.py: B110 already resolved by the earlier removal of the
  dead secrets-loading block (the except/pass was inside it)
- All other except/pass blocks in src/ and web_interface/ are intentional
  (last-resort recovery, best-effort fallbacks, non-critical startup probes).
  Annotated each with # nosec B110 and a brief inline reason so the decision
  is explicit for future reviewers.
- Test files and plugin-repos B110 suppressed via Codacy API (not prod code).

B108 (/tmp usage):
- permission_utils.py: /tmp listed to PREVENT permission changes on it — not
  used as a temp path. Annotated # nosec B108.
- display_manager.py: fixed snapshot path is intentional (web UI reads same
  path); path-check guard also annotated.
- wifi_manager.py: named /tmp files match the sudoers allowlist installed with
  the system (the paths are hard-coded in both places by design). Annotated
  all six open/cp references # nosec B108.
- scripts/render_plugin.py: dev script default overridable by user. Annotated.
- web_interface/app.py: reads the same fixed path written by display_manager.
  Annotated # nosec B108.
- Test files suppressed via Codacy API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address remaining Codacy security findings

Flask debug=True (real fix):
- web_interface/app.py: debug=True in __main__ block exposes the Werkzeug
  interactive debugger (arbitrary code execution). Changed to
  os.environ.get('FLASK_DEBUG', '0') == '1' — off by default, opt-in
  via environment variable for local development.

nosec annotations (accepted risk with documented rationale):
- disk_cache.py: os.chmod(0o660) is intentional — web UI and LED matrix
  service share a group, 660 gives group write while denying world access
  (B103 + Semgrep insecure-file-permissions suppressed in Codacy)
- wifi_manager.py: urlopen to hardcoded connectivity-check.ubuntu.com URL
  (B310 — no user input involved)
- font_manager.py: urlretrieve URL comes from user's own config file on
  their local device (B310)
- start_web_conditionally.py: os.execvp with both sys.executable and a
  fixed PROJECT_DIR-relative constant (B606)

Confirmed false positives suppressed via Codacy API (15 issues):
- SSRF (3x): client-side JS fetch — SSRF is server-side; browser fetch
  is CORS-restricted to same origin
- B105 (3x): test fixtures use dummy secrets by design; store_manager
  checks for the placeholder string, it is not itself a secret
- PMD numeric literal (2x): 10000000 is within Number.MAX_SAFE_INTEGER
- Prototype pollution (1x): read-only schema traversal, no writes
- no-unsanitized_method (1x): dynamic import() is CORS-restricted
- detect-unsafe-regex (1x): operates on server-controlled config values
- plugin-repos B103 (1x): vendor code chmod on executable
- Semgrep insecure-file-permissions (3x): same disk_cache 0o660 as above

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unnecessary f prefix from f-strings without placeholders (F541)

Pyflakes F541 flags f-strings that contain no {} interpolation — they are
identical to plain strings but trigger unnecessary string formatting overhead.

Fixed in production code:
- src/base_classes/data_sources.py (2 debug log calls)
- src/logo_downloader.py (1 error log)
- src/plugin_system/store_manager.py (5 strings across 3 log calls)
- src/web_interface/validators.py (1 return value)
- src/wifi_manager.py (4 log/message strings)
- web_interface/start.py (1 print)

F541 issues in test/, scripts/, and plugin-repos/ suppressed via Codacy API
as non-production code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(dev): add Pillow compatibility smoke test script

Covers all Pillow APIs used in LEDMatrix — image creation, drawing,
font metrics, LANCZOS resampling, paste/alpha_composite, and PNG I/O.
Run after any Pillow version bump to catch regressions before deploy.

    python3 scripts/dev/test_pillow_compat.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve 8 new Codacy issues introduced by PR changes

shellcheck SC2034:
- first_time_install.sh: 'type' loop variable also unused in the wifi
  status loop (we previously fixed 'device' → '_' but left 'type').
  Changed to '_ _ state' since neither device nor type is referenced.

ESLint no-undef:
- app.js: typeof guards don't satisfy no-undef; added updateSystemStats
  to the /* global */ declaration alongside showNotification.

nosec annotation:
- web_interface/app.py: app.run(host='0.0.0.0') line changed when we
  fixed debug=True, giving it a new issue ID. Re-added # nosec B104.

pyflakes F401:
- scripts/dev/test_pillow_compat.py: ImageFilter was imported but never
  used in the smoke test. Removed from the import.

Codacy API suppressions (false positives on changed lines):
- disk_cache.py 0o660 chmod (2x): lines changed when # nosec B103 was
  added, producing new Semgrep issue IDs. Re-suppressed.
- pages_v3.py raw-html-concat: Semgrep does not recognise escape() as
  a sanitizer; the escape() call IS the correct fix.
- app.py flask 0.0.0.0: same line as B104 above; Semgrep rule also
  re-suppressed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address PR review findings

Fix (10 of 15 findings):

plugin-repos/march-madness/requirements.txt:
  Add urllib3>=1.26.0 — manager.py directly imports from urllib3; it was
  an undeclared transitive dependency via requests.

scripts/dev/dev_plugin_setup.sh:
  Restore subshell form (cd "$target_dir" && git pull --rebase) || true
  so the shell's working directory is not permanently changed after the
  if-cd block. Previous fix for SC2015 leaked cwd into the remainder of
  the script.

src/base_classes/sports.py:
  Narrow 'except Exception' to 'except RuntimeError as e' and log via
  self.logger.debug — Path.home() raises only RuntimeError for service
  users; other exceptions should not be silently swallowed.

src/config_service.py:
  Fix stale "MD5 checksum" in ConfigVersion.__init__ docstring (line 40);
  the implementation uses SHA-256 since the Codacy fix.

src/wifi_manager.py:
  Log the last-resort AP enable failure with exc_info=True instead of
  silently passing — failure here means the device may be unreachable.

web_interface/blueprints/pages_v3.py:
  Log the outer metadata pre-load exception at debug level instead of
  swallowing it silently; schema still loads fully below.

src/background_data_service.py:
  Remove unused 'timeout' parameter from shutdown() — executor.shutdown()
  does not accept timeout; update __del__ caller accordingly.

src/font_manager.py:
  Validate URL scheme before urlretrieve — reject non-http/https schemes
  (e.g. file://) to prevent reading local files from config-supplied URLs.

src/plugin_system/plugin_executor.py:
  Simplify redundant except tuple: (PluginTimeoutError, PluginError,
  Exception) → Exception, which already covers the others.

test/test_display_controller.py:
  Mark empty test_plugin_discovery_and_loading as @pytest.mark.skip with
  reason. Move duplicate 'from datetime import datetime' to module header
  and remove the stray mid-module copy.

Skip (5 of 15 findings, with reasons):
  - pytest 9.0.3 concerns: full suite already verified (467 pass, 18 pre-existing)
  - Pillow 12.2.0 API concerns: no deprecated APIs in codebase; tests + Pi smoke test pass
  - diagnose_web_ui.sh sudo validation: set -e already ensures fail-fast on any sudo failure
  - app.py request-logging except: must stay silent (recursive logging risk); annotated
  - app.py SSE file-read except: genuinely transient I/O; annotated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 10:19:55 -04:00
sarjent
44d1a08db4 perf(plugins): dramatically speed up plugin manager tab load time (#333)
* fix(cache): check odds keys before generic live check in get_data_type_from_key

Cache keys like odds_espn_basketball_nba_<id>_live contain both 'odds'
and 'live'. The previous ordering matched the generic 'live' check first,
returning 'sports_live' (30 s TTL) instead of the correct 'odds_live'
(120 s TTL). This caused the ESPN odds API to be hit every 30 s per live
game, frequently triggering the 3-second per-request timeout and returning
no odds data.

Moving the 'odds' check above the generic 'live' block restores the
correct 120-second cache TTL for in-progress game odds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display): use single-quoted HTML attributes for JSON hidden inputs

Placing |tojson output (which contains double quotes) inside a
double-quoted HTML attribute broke the attribute — browsers closed
the attribute at the first inner quote, leaving JS with an empty or
truncated value. JSON.parse then failed silently, leaving excluded=[]
so all Vegas scroll plugins appeared checked (included) regardless of
the actual excluded_plugins config.

Switch to single-quoted HTML attributes so the JSON double quotes
are valid inside the attribute value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(plugins): dramatically speed up plugin manager tab load time

## Problem

The Plugins tab loaded slowly and inconsistently (5–30s depending on
cache state), with a blank spinner for the entire wait. Three root
causes:

1. **N+1 subprocess per installed plugin** — `_get_local_git_info` ran
   4 separate git subprocesses per plugin (rev-parse HEAD, abbrev-ref,
   config --get remote.origin.url, log --format=%cI). With 15 plugins
   that's 60 blocking subprocess spawns before the endpoint returned.

2. **Serial per-plugin loop** — the `/plugins/installed` endpoint
   processed each plugin sequentially: manifest read → git info →
   instance lookup → Vegas mode query, one plugin at a time.

3. **Serial JS loading** — the store search only started after installed
   plugins fully completed, so users waited for both round-trips back
   to back. No UI feedback during the wait.

## Changes

### Backend — src/plugin_system/store_manager.py
- Consolidate 4 git subprocesses → 1: branch read from `.git/HEAD`
  (file I/O, no subprocess), remote URL parsed from `.git/config`
  (file I/O, no subprocess), SHA + commit date fetched together in a
  single `git log -1 --format=%H%n%cI` call
- Existing signature-based cache already eliminates all subprocesses on
  warm hits; this change cuts cold-cache cost from 4 → 1 per plugin

### Backend — web_interface/blueprints/api_v3.py
- Wrap per-plugin work in a `_build_plugin_entry()` helper and execute
  it across a `ThreadPoolExecutor(max_workers=8)` so all plugins are
  processed in parallel instead of sequentially
- Fix double `get_plugin()` call per plugin (was called once for the
  enabled fallback and again for Vegas mode — now one shared call)

### Frontend — web_interface/static/v3/plugins_manager.js
- Fire `searchPluginStore()` and `loadInstalledPlugins()` simultaneously
  instead of waiting for installed to complete before starting the store
- After installed data arrives, call `applyStoreFiltersAndSort(true)` to
  refresh install/update/reinstall badges from already-cached store data
  (instant, no extra network call)

### Frontend — web_interface/templates/v3/partials/plugins.html
- Add responsive skeleton cards to the installed plugins section that
  match real card proportions (removed automatically when data renders)
- Replace the 5 featureless gray boxes in the store skeleton with 10
  structured skeleton cards matching the real card layout

## Measured improvement on Pi 4 (11 installed plugins, ledpi-ticker)

| Scenario | Before | After |
|---|---|---|
| Cold cache (first open) | ~8–15s | **0.9s** |
| Warm cache (git cache hit) | ~1–2s | **55ms** |
| UI feedback during load | blank spinner | skeleton cards |
| Store waits for installed | yes (serial) | no (parallel) |

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugins): harden git metadata parsing and plugin entry building

store_manager.py:
- Detect worktree/submodule .git files (gitdir: <path>) and resolve
  to the actual git directory before reading HEAD or config
- Wrap HEAD read_text in try/except OSError/NotADirectoryError so
  atypical repos return None instead of propagating exceptions
- Guard config url line split with '=' presence check to avoid
  IndexError on malformed lines

api_v3.py:
- Wrap _build_plugin_entry body in a try/except via a thin outer
  wrapper so a single plugin's failure doesn't 500 the whole endpoint;
  failed entries return None and are filtered by the existing [r for r
  in results if r is not None] step
- Narrow manifest except clause to FileNotFoundError, PermissionError,
  json.JSONDecodeError instead of bare Exception
- Validate manifest is a dict before calling plugin_info.update() and
  log a debug message when it isn't

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 18:09:33 -04:00
sarjent
6a4644007d fix(display): Vegas excluded plugins always showing as checked (#332)
* fix(cache): check odds keys before generic live check in get_data_type_from_key

Cache keys like odds_espn_basketball_nba_<id>_live contain both 'odds'
and 'live'. The previous ordering matched the generic 'live' check first,
returning 'sports_live' (30 s TTL) instead of the correct 'odds_live'
(120 s TTL). This caused the ESPN odds API to be hit every 30 s per live
game, frequently triggering the 3-second per-request timeout and returning
no odds data.

Moving the 'odds' check above the generic 'live' block restores the
correct 120-second cache TTL for in-progress game odds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display): use single-quoted HTML attributes for JSON hidden inputs

Placing |tojson output (which contains double quotes) inside a
double-quoted HTML attribute broke the attribute — browsers closed
the attribute at the first inner quote, leaving JS with an empty or
truncated value. JSON.parse then failed silently, leaving excluded=[]
so all Vegas scroll plugins appeared checked (included) regardless of
the actual excluded_plugins config.

Switch to single-quoted HTML attributes so the JSON double quotes
are valid inside the attribute value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 18:09:16 -04:00
sarjent
1c4d5c5271 feat(sync): multi-display wireless sync — extend scrolling across two LED matrices (#330)
* feat(sync): multi-display wireless sync — extend scrolling across two LED matrices

Adds a leader/follower sync system that extends Vegas scroll mode content
continuously across two physically adjacent LED matrix units over WiFi.

Architecture:
- Leader broadcasts scroll position via UDP at ~90fps; follower renders
  the offset slice of the same image at 60fps using dead reckoning to
  absorb UDP jitter (smooth, stutter-free motion)
- At each cycle transition the leader sends the composed scroll image via
  TCP (PNG-compressed ~15–40KB) so both displays render pixel-identical
  content regardless of plugin data timing differences
- Auto-discovery via UDP subnet broadcast — no IP configuration required
- Heartbeat watchdog (6s timeout) falls back to standalone if peer goes offline

Key files:
- src/common/sync_manager.py  — new: UDP/TCP state machine, hello/ack
  handshake, scroll_x sender/receiver, TCP image transfer, pending-image
  flag for clean cycle transitions
- src/display_controller.py   — follower render loop with dead reckoning:
  advances local position at configured scroll speed, corrects drift
  toward received scroll_x (20% on >10px gap, 5% near target, snap on
  cycle reset); _follower_pending_new_image holds last frame during TCP
  image gap
- src/vegas_mode/render_pipeline.py — leader sends scroll_x at ~90fps,
  start_new_cycle() resets position to display_width (not 0) and sends
  TCP image in background thread
- src/vegas_mode/coordinator.py — set_sync_manager() / set_update_callback()
  wiring; defers hot-swap recompose while sync is active
- web_interface/blueprints/api_v3.py — sync config save endpoint, GET
  /api/v3/sync/status for live status polling
- web_interface/templates/v3/partials/display.html — Multi-Display Sync
  section: role selector (Standalone/Leader/Follower), position (Left/Right
  of leader, follower only), UDP port, live status indicator
- config/config.template.json — sync block: role, port, follower_position

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address PR review findings

- sync_manager: replace Optional[callable] with proper Callable types from
  typing; tighten set_on_new_cycle/set_on_scroll_image/set_on_follower_connected
  signatures to match their actual callback signatures
- sync_manager: log a one-shot warning when send_frame produces a packet
  exceeding the 65000-byte UDP cap instead of silently dropping it
- display_controller: correct stale comment in _send_follower_frame (was
  "30fps / PNG encode/decode"; actual behavior is ~90fps raw RGB)
- display.html: guard setInterval with window.syncStatusInterval to prevent
  duplicate pollers if the script runs more than once
- display.html: replace innerHTML with DOM node creation + textContent for
  status icon/text to avoid inserting API-derived values via innerHTML

Skip: time.time() → monotonic and self.config staleness are pre-existing
issues not introduced by this PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address second round of PR review findings

- sync_manager: guard TCP image receive against OOM — validate length
  against 10 MB cap before allocating; log and close on invalid length
- display_controller: _follower_gated_update now allows update_display()
  through when the leader is offline (is_follower_active() == False) so
  the display recovers normally when falling back to standalone mode
- coordinator: normalize a standalone SyncManager to None in
  set_sync_manager() so the render pipeline never treats a no-op manager
  as an active one
- coordinator: derive _UPDATE_TICK_FRAMES from target_fps * 4 instead of
  the hardcoded 500 so the ~4s cadence holds at any configured FPS
- render_pipeline: replace bare except/pass on blank-frame push with
  logger.exception() so failures are visible in logs

Skip: config.template.json comments — JSON does not support inline comments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address third round of PR review findings

- sync_manager: use 'with socket.socket(...)' in send_scroll_image so the
  TCP socket is always closed even if connect/sendall raises
- sync_manager: add _scroll_image_lock to serialize all reads/writes to
  _on_scroll_image and _pending_scroll_image between _image_server_loop
  and set_on_scroll_image, eliminating the lost-delivery race; callback
  is invoked outside the lock to avoid holding it during user code
- sync_manager: validate scroll image dimensions (max 100000×256) and
  catch DecompressionBombError before img.load() in _image_server_loop
- sync_manager: log socket close exceptions at debug level in stop()
  instead of silently passing
- sync_manager: replace hardcoded /tmp/ with tempfile.gettempdir() for
  STATUS_FILE (atomic write was already in place)
- sync_manager: check _RAW_MAGIC first in _follower_recv_loop routing
  so magic-tagged frames are always identified correctly regardless of size

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address fourth round of PR review findings

- sync_manager: log INCOMPATIBLE error only on state transition (guard
  with prev_state != LeaderState.INCOMPATIBLE) so repeated hello packets
  from an incompatible follower don't spam the log
- sync_manager: replace O(n²) bytes concatenation in TCP image receive
  loop with bytearray + extend() for linear-time accumulation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): suppress Codacy false positives

- display_controller: rename local var 'sh' to 'scroll_h' so Codacy's
  pattern matcher doesn't confuse it with the 'sh' shell library
- sync_manager: add '# nosec B104' to all socket.bind("") calls —
  binding to all interfaces is intentional (UDP broadcast reception and
  TCP image server must accept connections from any local interface)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): add nosec B104 to socket creation lines for Codacy

Codacy attributes the bind-to-all-interfaces finding to the socket.socket()
creation lines (140, 439) rather than the .bind() calls. Added # nosec B104
there too so the suppression is seen at the line Codacy reports.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 09:51:44 -04:00
sarjent
dbb53da31d fix(vegas): eliminate plugin re-appearance at scroll cycle boundaries (#327)
* fix(vegas): eliminate plugin re-appearance at scroll cycle boundaries

The Vegas scroll image is wider than the display. scroll_helper marks a
cycle complete only after total_distance_scrolled >= total_scroll_width +
display_width, meaning it keeps scrolling for an extra display_width of
pixels after all content has exited left. During that extra travel the
scroll_position wraps back to ~0 and the first plugin re-enters from the
right - visible for ~2-3 seconds as a plugin partially displaying before
the next one starts.

render_pipeline.render_frame(): end the cycle the moment
total_distance_scrolled >= total_scroll_width (the natural wrap point),
before any second-pass content becomes visible. Push a blank frame
immediately on detection so hardware never shows a frozen content
snapshot while start_new_cycle() recomposes (~100 ms).

display_manager.py: add capture_mode() context manager. When active,
update_display() and the canvas clear in clear() skip the hardware
write, preventing plugins that call update_display() internally from
flashing on the matrix during off-screen content capture inside
start_new_cycle().

plugin_adapter.py: wrap all plugin.display() calls in
_capture_display_content() and _trigger_scroll_content_generation()
with capture_mode() so the fallback capture path never produces
hardware output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(vegas): tighten exception handling in clear() and blank-frame push

display_manager.clear(): replace bare except/pass on the three hardware
Clear() calls with (RuntimeError, OSError) and a logger.error() so
failures are visible in logs rather than silently swallowed.  Still
non-fatal — the PIL image buffer is already black before these calls,
so the next update_display() will push clean content regardless.

render_pipeline.render_frame(): replace broad except/pass in the
blank-frame push with (ImportError, ValueError, TypeError, MemoryError)
and a logger.error() that includes display dimensions for context.
update_display() already handles its own hardware errors internally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(vegas): catch OSError and RuntimeError in blank-frame push

Image.new() can raise OSError in some PIL environments and hardware
libraries may surface RuntimeError on I/O failures.  Add both to the
exception tuple alongside the existing ImportError/ValueError/TypeError/
MemoryError so no boundary failure escapes the local handler.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 15:51:38 -04:00
Chuck
452afacd12 fix(news): custom RSS feed save fails with validation error when no logo (#329)
_set_missing_booleans_to_false was unconditionally creating an empty
dict for every nested-object sub-property when processing array items.
For the news plugin's custom_feeds, this produced logo:{} on every feed
item that had no logo uploaded. jsonschema then validated that empty
object against logo's required:["id","path"] constraint and failed.

Fix: skip recursion into a sub-object when it isn't already present in
the array item. There's no reason to create an optional object like
logo just to look for boolean fields inside it.

Also extend _filter_config_by_schema to recurse into array items when
the items schema has properties. Previously arrays were passed through
unchanged, so any stray field on a feed item (legacy data, migration
artifacts) would survive to validation where additionalProperties:false
would reject it.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 15:51:17 -04:00
Chuck
3b45a75f75 fix: pixlet install false-failure, force render in web service, web UI perf (#328)
* fix: pixlet install false-failure, force render in web service, web UI perf

Fixes three user-reported issues:

1. Pixlet install reported failure even on success — `set -e` in
   download_pixlet.sh caused the script to exit non-zero when
   `((success_count++))` evaluated the post-increment old value (0),
   which bash treats as a failed command. Fixed with shell arithmetic
   assignment instead.

2. Force render returned 503 "plugin not loaded in web service" — the
   web service runs its own PluginManager with display_manager=None,
   so the starlark-apps plugin can never be loaded there. Added a
   standalone render path (_standalone_render_starlark_app) that calls
   pixlet directly from the web service, writing to cached_render.webp.
   The main-service plugin path is preserved and tried first.

3. Web UI sluggishness — the SSE /stream/stats generator was blocking
   a Flask thread for 1s per tick via psutil.cpu_percent(interval=1).
   Switched to non-blocking interval=None (primed at startup). Also
   cached the per-client ledmatrix systemctl check (15s TTL), raised
   the AP mode check TTL from 5s to 30s, and halved the display
   preview poll rate from 0.5s to 1s.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(starlark): surface config errors, deduplicate pixlet lookup, uniform response shape

- _standalone_render_starlark_app: split silent except into separate
  json.JSONDecodeError and OSError handlers that return (False, message)
  with the file path and parse error, so callers know when config.json
  is unreadable rather than silently rendering with empty config

- get_starlark_status: replace 14-line inline platform/shutil pixlet
  lookup with _find_pixlet_binary(), which also checks the user-
  configured starlark-apps.pixlet_path — the old code never consulted
  that setting so the status endpoint could wrongly report pixlet
  unavailable even when a custom path was configured

- render_starlark_app standalone branch: add frame_count: 0 to success
  and error responses so both branches return the same payload shape;
  0 is accurate since standalone render writes cached_render.webp but
  does not extract frames into memory

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(starlark): safe chmod fallback in pixlet lookup, semantic HTTP codes in standalone render

_find_pixlet_binary: bundled.chmod(0o755) could raise OSError (e.g.
file owned by root) and abort the entire resolution chain, skipping
the PATH fallback. Now checks os.access first; if chmod is needed,
wraps it in try/except OSError, logs a warning, and falls through to
shutil.which so PATH is always tried.

_standalone_render_starlark_app: expanded return type from
(bool, str) to (bool, int, str) so each failure mode carries a
semantic HTTP status code:
  app/star file missing  → 404
  invalid/unreadable config.json → 400
  pixlet binary missing  → 503
  pixlet non-zero exit   → 502
  subprocess timeout     → 504
  unexpected exception   → 500
  success                → 200
Call site updated to unpack (success, status_code, error) and forward
status_code directly in both success and error responses.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(starlark): validate config.json is a dict before iterating items

json.load succeeds on valid JSON that isn't an object (e.g. an array
or bare string), leaving app_config as a non-dict and causing an
AttributeError on the subsequent .items() call. Added isinstance check
immediately after json.load; non-dict values return (False, 400, ...)
with the actual type name in the message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(starlark): harden pixlet binary check and manifest shape validation

_find_pixlet_binary: switched bundled.exists() to bundled.is_file() so
a directory named pixlet-linux-arm64 (traversable, hence X_OK) is no
longer returned as a binary; re-check os.access after chmod succeeds
so we only return the path when executability is confirmed, with a
separate warning if it still isn't executable after chmod.

_standalone_render_starlark_app: added isinstance guards on the
manifest and apps values returned by _read_starlark_manifest(); a
manifest.json containing valid non-object JSON (e.g. an array) would
previously raise AttributeError on .get(); invalid shape now returns
(False, 400, ...) instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 14:14:48 -04:00
Chuck
1a0f1c8015 fix: service control buttons and AP-mode SSH lockout post-install (#326)
* fix: service control buttons and AP-mode SSH lockout post-install

Two user-reported issues after fresh install:

1. All service buttons (Start/Stop/Restart Display, Restart Web Service)
   failed silently — only Reboot worked.

   Root cause: sudoers rules use `ledmatrix.service` (with suffix) but
   api_v3.py called `sudo systemctl start ledmatrix` (no suffix). sudo
   does exact string matching, so every service action was rejected with
   returncode=1. Also missing from sudoers: ledmatrix-web, journalctl,
   and is-active entries.

   Fix:
   - Add `.service` suffix to all 8 sudo systemctl call sites in
     api_v3.py (_ensure_display_service_running, _stop_display_service,
     and all execute_system_action branches).
   - Add timeout=15 to all subprocess.run calls in execute_system_action
     (previously could hang indefinitely).
   - Add missing sudoers rules to first_time_install.sh and
     configure_web_sudo.sh: ledmatrix-web.service start/stop/restart,
     is-active for both name forms, and journalctl -u/-t ledmatrix rules.

2. SSH and web UI became inaccessible after ~1 hour even though the
   display kept running.

   Root cause: wifi_monitor_daemon restarts NetworkManager after 5
   consecutive internet failures (~2.5 min). Each NM restart drops WiFi
   briefly. During that window check_and_manage_ap_mode() increments
   _disconnected_checks but the daemon never reset it after the restart.
   After 3 such NM-restart cycles, _disconnected_checks reached 3 and
   AP mode activated — changing the Pi from WiFi client to hotspot
   (192.168.4.1) and killing SSH on the old IP.

   Fix:
   - Reset wifi_manager._disconnected_checks = 0 in the daemon
     immediately after a successful NM restart so the brief drop it
     causes doesn't count toward AP-mode activation.
   - Increase _disconnected_checks_required from 3 to 6 (90s → 3min)
     as an additional buffer against transient network flaps.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* revert: restore AP-mode grace period to 90s (3 checks)

The counter reset after NM restart already fully prevents the SSH-lockout
cascade: _disconnected_checks can never accumulate across NM restarts
because it is reset to 0 before the next daemon iteration runs.

The 3→6 increase provided no additional fix for the described problem and
caused a UX regression: fresh Pi devices with no WiFi configured would
wait 3 minutes instead of 90 seconds for the LEDMatrix-Setup hotspot to
appear.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address five valid review findings; skip two

Fixed:
- march-madness/requirements.txt: Pillow>=10.3.0 (patches CVE-2024-28219;
  10.3.0 is the actual fix version — reviewer cited 12.2.0 but that risks
  breaking API changes without test coverage)
- wifi_monitor_daemon.py: add missing `import subprocess`; subprocess.run
  and CalledProcessError would NameError at runtime on the NM restart path
- wifi_manager.py: validate ap_idle_timeout_minutes before arithmetic —
  coerce to int, clamp 1–1440, fall back to 15 on bad config values
- wifi_manager.py: call _remove_nm_dnsmasq_captive_conf() on all three
  rollback paths in _enable_ap_mode_nmcli_hotspot() and in the top-level
  except block so stale dnsmasq drop-ins are never left behind
- api_v3.py: fix wrong_password prefix strip — removeprefix("wrong_password:")
  then lstrip() handles both "wrong_password: msg" and "wrong_password:msg"
- plugins_manager.js: add .catch() to loadInstalledPlugins().then() to
  surface failures instead of silently dropping unhandled rejections

Skipped:
- WiFiManager AP state persistence: architectural overhaul; _is_ap_mode_active()
  already derives from live system state, not in-memory variables
- Absolute subprocess paths in api_v3.py: paths vary by distro (/usr/bin vs
  /bin); web service has a normal PATH; sudoers already use resolved paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address five review findings (NM retry loop, start_display message, code quality)

- wifi_monitor_daemon: reset _consecutive_internet_failures = 0 in both
  NM-restart exception handlers; previously both left the counter at threshold,
  causing an immediate retry on the next iteration instead of waiting another
  full backoff period

- api_v3: fix start_display failure message — when mode is set and systemctl
  returns non-zero, message now includes the failure reason and a hint rather
  than always reporting success phrasing

- wifi_manager: move _redirect_backend from class variable to instance variable
  in __init__ alongside _ap_enabled_at; class-level default shadowed correctly
  in practice (single instance) but was misleading

- wifi_manager: narrow broad except Exception in _check_internet_connectivity
  to (subprocess.SubprocessError, OSError) for ping and OSError for HTTP
  (urllib.error.URLError is an OSError subclass in Python 3)

- wifi_manager: remove redundant local 'import re as _re' in _validate_ap_config;
  re is already imported at module level (line 37)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address five review findings (Pillow CVEs, daemon exception narrowing, timeout handling, plugin store)

- march-madness/requirements.txt: Pillow>=12.2.0 (patches CVE-2026-42308
  and CVE-2026-42310; previous floor of 10.3.0 was insufficient)

- wifi_monitor_daemon: narrow final except Exception to
  (subprocess.SubprocessError, OSError) so programming errors in the NM
  restart block are no longer silently swallowed

- api_v3/execute_system_action: add explicit subprocess.TimeoutExpired
  handler before the generic Exception catch; returns action-specific
  message with 'status','message','returncode','stdout','stderr' fields
  so the UI receives a precise, actionable payload instead of the generic
  'Failed to execute system action' string

- plugins_manager.js: move searchPluginStore into .finally() so the
  plugin store renders regardless of whether loadInstalledPlugins succeeds
  or fails; .catch() still logs the error

- first_time_install.sh: add safe_plugin_rm.sh NOPASSWD rule to the
  /tmp/ledmatrix_web_sudoers block; configure_web_sudo.sh had this rule
  but the standalone installer never granted it, leaving plugin removal
  broken after first-time install

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(api): resolve sudo/systemctl/reboot/poweroff paths at startup

Use shutil.which() with safe fallbacks for the four privileged binaries
instead of relying on bare names being resolved by the subprocess shell
search. Resolves paths once at module load rather than per-call.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 17:58:51 -04:00
5ymb01
b361866679 fix(security): escape user-controlled output in plugin action UI (#323)
* style: trim trailing whitespace and fix EOF in plugins_manager.js

Autofix from pre-commit hooks (trailing-whitespace, end-of-file-fixer).
No code logic changes — purely whitespace.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(security): escape user-controlled output in plugin action UI

The plugin action result handlers in executePluginAction() injected
data.message, data.output, and data.auth_url directly into innerHTML
template literals without escaping. A plugin's action handler returning
malicious content (e.g., from a third-party plugin or compromised
upstream) could execute arbitrary JavaScript in the web UI context.

Wrap user-controlled strings in escapeHtml() at all four sites:
- Step-2 (continuation) error path (message + output)
- OAuth flow auth_url (link href + display text, with http:// guard)
- Step-1 simple-success output
- Step-1 failure path (message + output)

The escapeHtml() helper is already defined in this file and used
elsewhere (validation errors, plugin store cards).

Co-Authored-By: 5ymb01 <5ymb01@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: 5ymb01 <5ymb01@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-06 10:12:31 -04:00
Chuck
ceb4c4105f fix(wifi): reliable open AP with captive portal — tested on Trixie Pi (#320)
* fix(wifi): create truly open AP via nmcli connection add; add captive portal to nmcli path

nmcli device wifi hotspot always attaches a WPA2 PSK on Bookworm/Trixie
and silently ignores post-creation security modifications, causing users
to be prompted for an unknown password. Switch to nmcli connection add
with 802-11-wireless.mode ap and no security section — NM cannot auto-add
a password to a profile that has no 802-11-wireless-security block.

Also:
- Remove dead DEFAULT_AP_PASSWORD / ap_password config field (stored but
  never passed to hostapd or nmcli, causing user confusion)
- Add iptables port 80→5000 redirect to the nmcli AP path so captive portal
  auto-popup works on phones without hostapd (previously only worked on
  the hostapd path)
- Clean up iptables rules on disable for the nmcli path
- Improve LED message on AP enable: show SSID, "No password", and IP:port
  on both paths so users know exactly how to connect
- Fix systemd template: replace hardcoded /home/ledpi/LEDMatrix/ with
  __PROJECT_ROOT_DIR__ placeholder (install script already writes correct path)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): address Codacy review findings in AP mode implementation

- Validate ap_ssid/ap_channel from config before passing to subprocess
  (printable ASCII ≤32 chars; channel 1-14) to prevent command injection

- Fix INPUT iptables rule: PREROUTING redirects port 80→5000 so the INPUT
  chain sees dport=5000, not 80. Old INPUT rule on port 80 was a no-op.

- Refactor iptables setup/teardown into _setup_iptables_redirect() and
  _teardown_iptables_redirect() helpers, eliminating duplicate logic in
  the hostapd and nmcli paths

- Save/restore ip_forward state (via /tmp/ledmatrix_ip_forward_saved)
  instead of forcing it to 0 on cleanup, which could break VPNs or
  bridges already relying on forwarding

- nmcli path skips ip_forward management entirely: NM's ipv4.method=shared
  already manages it for the duration of the connection

- Fix _get_ap_status_nmcli() verification: new 'connection add type wifi'
  profiles have type '802-11-wireless', not 'hotspot', so verification was
  always returning False. Now also matches by our known connection name.

- Remove SSID-based connection deletion: deleting any profile whose SSID
  matched the AP SSID could destroy a user's saved home WiFi profile.
  Now only deletes by our application-managed profile names.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugins): fix async race in refreshPlugins; use cache TTL to gate re-swap metadata fetch

refreshPlugins() called searchPluginStore(true) and showNotification() immediately
after refreshInstalledPlugins() without awaiting the returned Promise, so
window.installedPlugins could still be stale when the store rendered its
Installed/Reinstall badges. Chain .then() so both run only after the fetch
completes.

In initializePlugins(), the re-swap path always passed fetchCommitInfo=false to
searchPluginStore, skipping GitHub metadata even when the 5-minute cache TTL had
expired. Add storeCacheExpired() helper and compute isReswapWarm = _reswap &&
!storeCacheExpired() so fresh metadata is fetched whenever the cache is cold,
regardless of whether the render is a first load or a tab re-swap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address three wifi_manager and one plugins_manager review findings

wifi_manager.py:
- _create_hostapd_config: use _validate_ap_config() for ssid/channel instead
  of raw self.config values; strip newlines from SSID to prevent config-file
  injection via the generated hostapd.conf
- _setup_iptables_redirect: check return codes of sysctl ip_forward enable and
  both iptables -A calls; on any failure log the error output, call
  _teardown_iptables_redirect() to restore state, and return False instead of
  silently succeeding
- _enable_ap_mode_nmcli_hotspot: on AP verification failure roll back fully —
  tear down iptables redirect, delete the LEDMatrix-Setup-AP connection profile,
  clear the LED message — before returning False

plugins_manager.js:
- initializePlugins: chain searchPluginStore(!isReswapWarm) inside
  loadInstalledPlugins().then() so window.installedPlugins is populated before
  the store renders Installed/Reinstall badges (same pattern applied to
  refreshPlugins() in the previous commit)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): use _find_command_path for iptables/sysctl; harden ip_forward save/restore

Add _find_command_path() helper that extends _check_command()'s sbin-aware lookup to
return the absolute binary path rather than a boolean. Use it in
_setup_iptables_redirect and _teardown_iptables_redirect so iptables and sysctl are
resolved via /sbin or /usr/sbin even when those directories are absent from PATH in
systemd service environments.

Also harden the ip_forward save/restore logic:
- Read ip_forward from /proc/sys/net/ipv4/ip_forward (no subprocess, no PATH
  dependency) instead of spawning sysctl -n
- Skip the sysctl -w ip_forward=1 write when the value is already "1" to avoid
  mutating state owned by another service (VPN, NM shared mode, bridge)
- Track save success via presence of the save file: if the /proc read or file write
  fails, leave the file absent so teardown knows not to restore
- In _teardown_iptables_redirect, only restore ip_forward when the save file exists;
  if absent, leave the current value untouched rather than forcing "0"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): check _setup_iptables_redirect return; fix hostapd LED SSID; teardown on exception

- Both AP startup paths (hostapd and nmcli) now check the bool returned by
  _setup_iptables_redirect() and treat False as a hard failure: the hostapd
  path stops hostapd/dnsmasq and returns an error tuple; the nmcli path brings
  down and deletes the LEDMatrix-Setup-AP profile and clears the LED message

- _enable_ap_mode_hostapd's LED message now calls _validate_ap_config() to get
  the same sanitized SSID that _create_hostapd_config() uses, so the displayed
  name always matches the AP actually broadcast by hostapd

- _setup_iptables_redirect's outer except block now calls
  _teardown_iptables_redirect() before returning False so partial iptables/
  ip_forward state is always cleaned up on unexpected exceptions; cleanup
  exceptions are caught and logged separately

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(wifi): add unit tests for AP mode — open network, iptables, LED, cleanup ordering

Six pytest unit tests covering the five review scenarios. All subprocess and
filesystem side-effects are mocked so the tests run without root, hardware, or
a Pi OS environment.

1. test_nmcli_ap_profile_has_no_security_params — asserts the nmcli connection
   add command has no key-mgmt / psk / WPA arguments and sets mode=ap.
2. test_iptables_nat_rules_added_on_ap_start — verifies _setup_iptables_redirect
   emits a PREROUTING REDIRECT 80→5000 rule and an INPUT ACCEPT rule for port
   5000 (not 80, which never hits INPUT after PREROUTING rewrites it).
3. test_iptables_rules_and_ip_forward_reverted_on_teardown — verifies the -D
   PREROUTING/-D INPUT calls and that sysctl restores the saved ip_forward value
   and removes the save file.
4. test_ip_forward_not_restored_when_save_file_absent — verifies teardown skips
   sysctl when the save file was never written, preventing blind ip_forward=0 on
   systems using ip_forward for VPNs or NM shared mode.
5. test_led_message_shows_ssid_no_password_and_url — asserts the LED message
   includes the SSID, 'No password', and the 192.168.4.1:5000 setup URL.
6. test_existing_ap_profiles_deleted_before_new_profile_created — asserts all
   known profile names are targeted for deletion before 'nmcli connection add'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(wifi): adopt adsb-feeder-image hotspot patterns — DNS spoofing, connectivity check, idle timeout, wrong-password UX, watchdog escalation

Inspired by the production-proven approach in dirkhh/adsb-feeder-image.

1. DNS spoofing for automatic captive-portal popup (Change 1 — Critical)
   Write /etc/NetworkManager/dnsmasq-shared.d/ledmatrix-captive.conf with
   address=/#/192.168.4.1 before nmcli connection up so NM's built-in
   dnsmasq (ipv4.method=shared) resolves every hostname to the AP IP.
   This triggers the OS captive-portal popup automatically on iOS / Android /
   Windows / macOS — no manual navigation to 192.168.4.1:5000/setup required.
   New helpers: _write_nm_dnsmasq_captive_conf / _remove_nm_dnsmasq_captive_conf.
   New constants: NM_DNSMASQ_SHARED_DIR / NM_DNSMASQ_SHARED_CONF.

2. Real internet connectivity check (Change 2 — High)
   Add _check_internet_connectivity() (ping 8.8.8.8 + HTTP fallback).
   check_and_manage_ap_mode() now considers a device "disconnected" when nmcli
   shows connected but no real internet reachability, matching adsb-feeder's
   multi-method gateway/DNS/HTTP test approach.

3. AP idle timeout (Change 3 — Medium)
   Track _ap_enabled_at timestamp in enable_ap_mode(). Add _has_ap_clients()
   using 'iw dev <iface> station dump'. check_and_manage_ap_mode() auto-disables
   AP after ap_idle_timeout_minutes (default 15) with no associated clients.

4. Wrong-password error feedback (Change 4 — Medium)
   _connect_nmcli() detects "Secrets were required" / "authentication rejected"
   in nmcli stderr and prefixes the message with "wrong_password: ".
   The /api/v3/wifi/connect route propagates error_type="wrong_password" in the
   JSON response. captive_setup.html shows "Incorrect password — try again"
   (keeping the form active) instead of the generic failure message.

5. Escalating watchdog NM restart (Change 5 — Low)
   wifi_monitor_daemon.py tracks _consecutive_internet_failures. After
   _nm_restart_threshold (5) consecutive checks where nmcli shows connected but
   internet is unreachable, restart NetworkManager as a recovery step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): restore safe AP-enable trigger; decouple internet check from AP logic

The previous commit introduced _check_internet_connectivity() into
check_and_manage_ap_mode(), which shared the same _disconnected_checks counter
that triggers AP enable. This created a false-positive risk: 90 seconds of
packet loss on working WiFi would enable AP mode and kick off the connection.

Fix: restore nmcli association state as the sole AP-enable trigger (original,
safe behaviour). The internet connectivity check is now used only in the daemon
watchdog for the NM-restart escalation — matching how adsb-feeder-image actually
structures the two concerns (initial setup detection vs. ongoing monitoring).

Also clarify daemon comment: the connectivity check runs once per cycle in the
watchdog block, not inside check_and_manage_ap_mode, so there is no double-call.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): remove PMF setting from open AP profile — breaks nmcli connection add on Trixie NM 1.52+

802-11-wireless-security.pmf is only valid within a security section that also
includes key-mgmt. Adding it to an open-network profile causes NM 1.52+ to
reject the connection add with 'key-mgmt: property is missing'. PMF has no
meaning for open APs (it only applies to WPA2/WPA3), so the setting is simply
removed rather than worked around.

Found by testing on devpi (Trixie, NM 1.52.1).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): add nftables fallback for port redirect; graceful degradation when neither available

Tested on devpi (Trixie, NM 1.52.1): iptables is not installed; nftables is.
The original code called _setup_iptables_redirect() and treated 'iptables not
found' as a hard failure, rolling back the entire AP setup.

Changes:
- _setup_iptables_redirect() now tries iptables first, then nftables as a
  fallback. When neither is available it logs a warning and returns True so
  the AP still comes up (DNS spoofing still triggers the captive portal popup;
  users land on port 5000 directly instead of being auto-redirected from 80).
- Split into _setup_iptables_redirect_iptables() and
  _setup_iptables_redirect_nftables() for clarity.
- Added _redirect_backend instance var ("iptables" | "nftables" | None) so
  _teardown_iptables_redirect() uses the same tool that setup used.
- nftables teardown: deletes the 'ledmatrix' table (clean, no leftover rules).
- iptables teardown: unchanged logic (ip_forward save/restore).
- Also removed the PMF workaround for Trixie: 802-11-wireless-security.pmf
  requires key-mgmt to also be set, breaking open-network creation on NM 1.52+.
  Open APs have no management frame protection by definition.
- Update teardown test to set _redirect_backend = "iptables" before calling it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): public check_internet_connectivity(); absolute systemctl path; stricter mode assertion

wifi_manager.py:
- Add public check_internet_connectivity() wrapping the private method so the
  daemon does not reach into the private API

wifi_monitor_daemon.py:
- Call wifi_manager.check_internet_connectivity() instead of the private
  _check_internet_connectivity()
- Use /usr/bin/systemctl (absolute path) instead of bare "systemctl"
- Wrap NM restart in try/except with check=True; only reset
  _consecutive_internet_failures on success — on CalledProcessError or other
  exception, log the error and leave the counter unchanged so the next cycle
  retries

test/test_wifi_manager_ap.py:
- Replace loose `assert "ap" in add_calls[0]` (list-membership check that
  could be satisfied by any element equal to "ap") with an explicit key/value
  check: locate "802-11-wireless.mode" in the command list and assert the next
  element is exactly "ap"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 08:25:20 -04:00
Chuck
e9af18cdf1 Add Codacy badge to README (#322)
Added a Codacy badge to the README for code quality.

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-05-01 09:41:50 -04:00
Chuck
5e6c40ad55 fix(plugins): plugin manager tab doesn't always load on first visit (#319)
* fix(plugins): reset pluginsInitialized on HTMX re-swap; use refreshInstalledPlugins in refreshPlugins

HTMX's `revealed` trigger fires each time the plugins tab becomes
visible (Alpine's x-show toggles display:none which re-triggers the
IntersectionObserver). Each re-reveal fetches a fresh empty HTML
skeleton via hx-swap="innerHTML". The htmx:afterSwap handler reset
window.pluginManager.initialized/initializing but not pluginsInitialized,
so initializePlugins() hit its guard and skipped loadInstalledPlugins()
and searchPluginStore() — leaving the fresh empty DOM unpopulated.

Fix: also reset pluginsInitialized = false in the afterSwap handler.
Existing caches (3s for installed plugins, 5min for store) mean tab
revisits within the TTL render from cache instantly with no extra
API traffic.

Also change refreshPlugins() to call refreshInstalledPlugins() (which
already exists and explicitly invalidates the cache) instead of the
bare loadInstalledPlugins() call that could silently skip the fetch
if the 3-second cache happened to still be valid.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugins): use cached store on HTMX re-swap; reserve searchPluginStore(true) for first load

searchPluginStore(true) bypasses the isCacheValid check unconditionally,
so every tab revisit was hitting the GitHub commit-info API even within
the 5-minute cache window.

Set window.pluginManager._reswap = true in the htmx:afterSwap handler
and read it in initializePlugins() to call searchPluginStore(false) on
re-swaps (respects the 5-minute cache) vs searchPluginStore(true) on
first load (always fetches fresh). Explicit user refresh via
refreshPlugins() already calls searchPluginStore(true) directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 11:57:26 -04:00
Chuck
d6bd1ee215 fix(install): prevent weather and music from auto-installing on fresh install (#318)
* fix(install): remove weather and music credential stubs from secrets template

config_secrets.template.json shipped ledmatrix-weather and music as
top-level keys; config_manager deep-merges secrets into the main config
on load, so the reconciler treated them as plugin config entries and
auto-installed both plugins on first web UI visit after a fresh install.

Remove both keys from the template and clear the inline fallback block
in first_time_install.sh so new installs start clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(install): sync fallback secrets with template structure

The fallback block (used when config_secrets.template.json is missing)
was an empty object after the weather/music keys were removed. Mirror
the current template so youtube and github placeholders are always
present regardless of whether the template file exists.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 10:22:28 -04:00
Chuck
acaf8a248e feat(web): update-available banner in web UI (#311)
* feat(web): add update-available banner to web UI

Adds a polite, dismissible banner between the header and navigation
tabs that appears when the local repo is behind origin/main. Shows
commit count and a one-click "Update Now" button that triggers the
existing git_pull action.

- New GET /api/v3/system/check-update endpoint (5-min cache, compares
  local HEAD vs origin/main SHA)
- Banner auto-checks on page load then every 30 minutes
- Dismiss persists for the browser session via sessionStorage
- Styled for both light and dark themes
- Cache invalidated after successful git_pull so banner hides immediately

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(update-banner): address review findings — lock, returncode checks, update_available logic, a11y, button state

- Add _update_check_lock (threading.Lock) around all reads/writes to
  _update_check_cache in check_for_update() and git_pull, preventing
  races on concurrent requests
- Validate returncode for git fetch, rev-parse HEAD, and rev-parse
  origin/main; raise RuntimeError on failure so errors are caught and
  returned as error payloads instead of silently producing stale/empty SHAs
- Set update_available = commits_behind > 0 (was unconditionally True
  when local_sha != remote_sha); prevents false positive when local is
  ahead of remote
- Add type="button" and aria-label="Dismiss update" to the icon-only
  dismiss button
- Restore btn.innerHTML and btn.disabled in both success and error paths
  of applyUpdate(); only hide the banner and clear sessionStorage when
  data.status === 'success'

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(update-banner): address second-round review findings

api_v3.py:
- Move all git work inside _update_check_lock so concurrent requests
  re-check cache staleness after acquiring the lock; only the first
  caller runs git fetch/rev-parse/log, subsequent callers return the
  cached result
- Check log_result.returncode and raise on failure so a broken git log
  doesn't produce a silent false-negative (commits_behind=0)
- Rename loop variable l → commit_line

base.html:
- Replace boolean _dismissed flag with SHA-scoped sessionStorage key
  'update-sha-dismissed'; dismissing for SHA X still allows the banner
  to reappear when origin/main advances to SHA Y
- Successful applyUpdate clears 'update-sha-dismissed' so the next
  update cycle can show the banner again
- Add aria-live="polite" aria-atomic="true" to #update-banner-text so
  screen readers announce content changes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-30 09:38:03 -04:00
Scott Raynor
db9585cea9 Changed nhl logo to more monochrome logo that looks much better on the LEDMatrix. (#315)
Signed-off-by: J. Scott Raynor <sr-github@raynorsplace.net>
2026-04-29 19:12:05 -04:00
Chuck
65e3e8319b fix(plugin_manager): prevent permanent ERROR state after update timeout (#316)
* fix(plugin_manager): prevent permanent ERROR state after update timeout

When execute_update() fails (timeout or unhandled exception), the plugin
state was set to ERROR with no recovery path. can_execute() returns False
for ERROR state, so the plugin's update() was never called again, leaving
it showing stale data indefinitely.

Instead, update plugin_last_update so the plugin waits one configured
interval before retrying, and keep the state ENABLED so recovery is
automatic on the next cycle.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin_manager): address PR review — failure timestamp and error context

- Use time.time() at the point of failure instead of reusing current_time
  (captured before execution), so the full retry interval always elapses
  after a timeout rather than one execution-duration shorter

- Add PluginStateManager.set_error_info() to persist structured error context
  without changing plugin state; call it in both failure branches so
  get_error_info() / get_state_info() surface recoverable errors alongside
  ERROR-state errors

- Add warning log on the success=False branch (was previously silent)

- Pass a descriptive Exception (not a generic "Plugin execution failed") to
  health_tracker.record_failure() in the timeout/executor-error path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin_manager): atomic state+error write via set_state_with_error

The two-step set_state() / set_error_info() sequence left a window where
readers could observe ENABLED state without the accompanying error context.

Add threading.RLock to PluginStateManager and a new set_state_with_error()
method that holds the lock for both the state-transition write and the
_error_info write together. The method inlines the state-transition logic
rather than calling set_state() internally to intentionally skip the
"clear _error_info for non-ERROR states" side effect — the recoverable
error dict is exactly what we want stored.

Replace both paired set_state / set_error_info call sites in
run_scheduled_updates() with the single atomic method.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin_state): lock _error_info accesses and store defensive copies

Three verified issues:

- set_error_info wrote _error_info without holding _lock and stored the
  caller's dict by reference, allowing races and post-write mutation
- set_state_with_error stored error_info by reference (lock was already held)
- get_error_info read _error_info without _lock and returned the live
  reference, letting callers mutate the stored snapshot

Implicit fourth fix: set_state also wrote _error_info without _lock; locking
get_error_info while leaving that writer unguarded would have created a new
race, so set_state is now wrapped in _lock too for consistency.

Changes:
- set_state: wrap entire body in self._lock (covers _states, _state_history,
  and _error_info writes atomically; ERROR-path _error_info value was already
  a fresh dict literal so no copy needed)
- set_error_info: acquire self._lock + store dict(error_info) shallow copy
- set_state_with_error: store dict(error_info) shallow copy (lock already held)
- get_error_info: acquire self._lock + return dict(info) copy or None

All stored values are flat dicts of strings/floats/bools, so shallow copy
is sufficient — deepcopy is not needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin_manager): apply recovery logic to update_all_plugins; extract helper

update_all_plugins still set PluginState.ERROR on both failure paths, leaving
it inconsistent with the run_scheduled_updates fix from the same PR.

Extract _record_update_failure(plugin_id, exc=None) to hold all shared failure
logic: capture actual failure time, build structured error_info, log the retry
warning, stamp plugin_last_update, call set_state_with_error(ENABLED), and
forward to health_tracker. Replace all four failure sites (two in
run_scheduled_updates, two in update_all_plugins) with calls to this helper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:22:12 -04:00
Chuck
4ef3f8cad5 feat(web): add config backup & restore UI (#310)
* feat(web): add config backup & restore UI

Adds a Backup & Restore tab to the v3 web UI that packages user config,
secrets, WiFi, user-uploaded fonts, plugin image uploads, and the installed
plugin list into a single ZIP for safe reinstall recovery. Restore extracts
the bundle, snapshots current state via the existing atomic config manager
(so rollback stays available), reapplies the selected sections, and
optionally reinstalls missing plugins from the store.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backup): address PR review findings

- backup_manager: read plugin state from "states" key (not "plugins") to
  match the actual plugin_state.json format written by state_manager
- backup_manager: stream ZIP directly to a temp file instead of building
  it in an io.BytesIO buffer to avoid OOM on Raspberry Pi
- backup_manager: tighten plugin-uploads path validation in validate_backup
  and restore_backup to require "/uploads/" in the path, rejecting any
  non-uploads files smuggled under assets/plugins/
- api_v3: enforce 200 MB upload limit by streaming in chunks rather than
  relying on validate_file_upload (which only checks the filename)
- api_v3: replace bool() with _coerce_to_bool() for RestoreOptions fields
  so string "false" is not treated as truthy
- api_v3: capture and log _save_config_atomic return value instead of
  discarding it; log rather than silence font-cache and config-reload errors
- backup_restore.html: track inspectedFile so runRestore always applies to
  the file the user inspected, not a subsequently selected file; clear on
  input change or clearRestore()
- backup_restore.html: throw on non-success restore payload so errors are
  surfaced via the error notification path instead of yellow "warnings"
- test: update fixture to use correct "states" key structure; import
  SCHEMA_VERSION constant instead of hardcoding 1; rename unused err -> _err

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(backup): address second round of PR review findings

- api_v3: guard opts_dict with isinstance check after json.loads so a
  non-object JSON payload (null, array, etc.) returns a 400 instead of a
  500 AttributeError
- backup_manager: wrap tmp ZIP creation and os.replace in try/except so
  the .zip.tmp temp file is always removed on any failure
- backup_manager: replace hardcoded Path("/tmp/_zip_check") sentinel in
  validate_backup with a proper tempfile.TemporaryDirectory() so path
  traversal checks are portable and leave no artifacts
- backup_restore.html: detect partial-success responses (plugins_failed or
  errors non-empty) even when status is 'success' and render yellow/warning
  styling and notify instead of green

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(backup): add post-install steps for restored plugins; conditional restart hint

- api_v3: after a successful plugin reinstall during restore, run the same
  post-install sequence used by the normal /plugins/install flow:
  invalidate schema cache, discover_plugins()/load_plugin(), and
  set_plugin_installed() so restored plugins are immediately available
- backup_restore.html: only show the "restart the display service" hint
  when at least one item was restored or at least one plugin was installed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(backup): address Codacy findings

- api_v3: replace 'fonts' in ' '.join(result.restored) substring check
  with any(r.startswith("fonts") for r in result.restored) to avoid
  fragile joined-string membership testing
- api_v3: replace deprecated datetime.utcnow() and utcfromtimestamp()
  with datetime.now(timezone.utc) and fromtimestamp(..., timezone.utc);
  add timezone to import
- test: remove unused import io (backup_manager no longer uses BytesIO)
- src/backup_manager.py hardcoded /tmp sentinel was already fixed in a
  prior commit (tempfile.TemporaryDirectory)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-27 17:05:27 -04:00
Chuck
338bdc44cb fix(web): list calendar endpoint works without a running plugin instance (#314)
* fix(web): list calendar endpoint works without a running plugin instance

The web service (ledmatrix-web) and display service (ledmatrix) run in
separate processes. The web process discovers plugins but never
instantiates them — only the display process does. Consequently
api_v3.plugin_manager.get_plugin('calendar') always returns None in the
web process, and /api/v3/plugins/calendar/list-calendars responded with
404 "Calendar plugin is not running. Enable it and save config first."
even when the plugin was enabled, saved, and authenticated.

Fall back to reading token.pickle + credentials.json directly from the
calendar plugin directory and calling the Google Calendar API from the
web process. A live plugin instance is still preferred when available
(e.g. local dev where web and display share a process).

Also surface clearer, actionable error messages for missing/expired
tokens so users know to re-run the authentication step.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(web): address review feedback on calendar list endpoint

- Remove unused creds_path local
- Prefer google-auth JSON token (token.json) via Credentials.from_authorized_user_file; keep pickle fallback for backward compat, guarded by a size check and a comment describing the trust boundary (owner-only file inside the plugin dir, not user-supplied input)
- Do NOT persist refreshed credentials from the web request path; the display service owns token.pickle and concurrent writes could corrupt it. Refresh happens in memory only for the duration of the request
- Add explicit timeouts to token refresh (via Request(timeout=...)) and to the Calendar API list call (num_retries=1), and return a retryable user-facing message on socket.timeout / TimeoutError
- Import socket at module scope

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(web): apply real socket timeout to Google Calendar API calls

googleapiclient's execute() does not accept a timeout kwarg — the
timeout comes from the httplib2.Http the service was built with.
Build an AuthorizedHttp wrapping httplib2.Http(timeout=15) so
calendarList().list().execute() cannot hang the Flask worker on
flaky connectivity. Disable discovery doc caching to avoid the
default file-cache warning in this ephemeral request path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: ChuckBuilds <ChuckBuilds@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 13:28:49 -04:00
Chuck
73c00140df fix(web): resolve ReferenceError in single-file upload handler (#313)
The finally block in handleSingleFileUpload referenced an undefined
fileInput variable left over from an earlier refactor, causing an
"Unhandled promise rejection: ReferenceError" after every single-file
upload (e.g. OAuth credentials.json for the calendar plugin) even when
the upload itself succeeded.

Resolve the file input by id inside the finally block so it can be
cleared when present, tolerating the drop-zone-only case.

Co-authored-by: ChuckBuilds <ChuckBuilds@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 10:34:09 -04:00
Chuck
68a38c39f7 fix(web-ui-info): syntax error and init order crash on fresh install (#312)
Two bugs that prevented the web-ui-info default plugin from loading:

1. Orphaned `except Exception` at line 203 with no matching `try` —
   caused a SyntaxError preventing the module from importing at all.
   Removed the orphan; the outer try/except already covers the block.

2. `_get_local_ip()` called in __init__ before `_ap_mode_cache_time`
   was initialized, causing AttributeError. Moved AP mode cache field
   initialization above the `_get_local_ip()` call.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-19 11:56:05 -04:00
162 changed files with 8815 additions and 4817 deletions

1
.gitmodules vendored
View File

@@ -1,3 +1,4 @@
[submodule "rpi-rgb-led-matrix-master"] [submodule "rpi-rgb-led-matrix-master"]
path = rpi-rgb-led-matrix-master path = rpi-rgb-led-matrix-master
url = https://github.com/hzeller/rpi-rgb-led-matrix.git url = https://github.com/hzeller/rpi-rgb-led-matrix.git
branch = master

View File

@@ -1,4 +1,10 @@
# LEDMatrix # LEDMatrix
[![License](https://img.shields.io/badge/license-GPL--3.0-green)](LICENSE)
[![Discord](https://img.shields.io/badge/Discord-community-5865F2?logo=discord&logoColor=white)](https://discord.gg/RdrC37rEag)
[![GitHub Stars](https://img.shields.io/github/stars/ChuckBuilds/ledmatrix?style=flat&color=yellow)](https://github.com/ChuckBuilds/ledmatrix)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/77fc9b446a5948e5b0aed7a7aaeb1bab)](https://app.codacy.com/gh/ChuckBuilds/LEDMatrix/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)
## Welcome to LEDMatrix! ## Welcome to LEDMatrix!
Welcome to the LEDMatrix Project! This open-source project enables you to run an information-rich display on a Raspberry Pi connected to an LED RGB Matrix panel. Whether you want to see your calendar, weather forecasts, sports scores, stock prices, or any other information at a glance, LEDMatrix brings it all together. Welcome to the LEDMatrix Project! This open-source project enables you to run an information-rich display on a Raspberry Pi connected to an LED RGB Matrix panel. Whether you want to see your calendar, weather forecasts, sports scores, stock prices, or any other information at a glance, LEDMatrix brings it all together.
@@ -126,10 +132,15 @@ The system supports live, recent, and upcoming game information for multiple spo
| This project can be finnicky! RGB LED Matrix displays are not built the same or to a high-quality standard. We have seen many displays arrive dead or partially working in our discord. Please purchase from a reputable vendor. | | This project can be finnicky! RGB LED Matrix displays are not built the same or to a high-quality standard. We have seen many displays arrive dead or partially working in our discord. Please purchase from a reputable vendor. |
### Raspberry Pi ### Raspberry Pi
- Raspberry Pi Zero's don't have enough processing power for this project and the Pi 5 is unsupported due to new GPIO output. - Raspberry Pi Zero's don't have enough processing power for this project.
- **Raspberry Pi 3B or 4 (NOT RPi 5!)** - **Raspberry Pi 3B, 4, or 5**
[Amazon Affiliate Link Raspberry Pi 4 4GB RAM](https://amzn.to/4dJixuX) [Amazon Affiliate Link Raspberry Pi 4 4GB RAM](https://amzn.to/4dJixuX)
[Amazon Affiliate Link Raspberry Pi 4 8GB RAM](https://amzn.to/4qbqY7F) [Amazon Affiliate Link Raspberry Pi 4 8GB RAM](https://amzn.to/4qbqY7F)
- **Pi 5 users**: the installer automatically detects Pi 5 and builds the `rpi-rgb-led-matrix` library with RP1 support. If you previously installed on a Pi 4 and migrated the SD card, or if you see `mmap` errors in the logs, force a fresh library build:
```bash
sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh
```
- Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and set `gpio_slowdown` to `1` or `2`.
### RGB Matrix Bonnet / HAT ### RGB Matrix Bonnet / HAT
@@ -581,7 +592,7 @@ These settings control runtime behavior and GPIO timing:
- **Critical setting**: Must match your Raspberry Pi model for stability - **Critical setting**: Must match your Raspberry Pi model for stability
- **Raspberry Pi 3**: Use 3 - **Raspberry Pi 3**: Use 3
- **Raspberry Pi 4**: Use 4 - **Raspberry Pi 4**: Use 4
- **Raspberry Pi 5**: Use 5 (or higher if needed) - **Raspberry Pi 5**: Use 12 in PIO mode (`rp1_rio: 0`, the default); start with `1` and increase if you see flickering
- **Raspberry Pi Zero/1**: Use 1-2 - **Raspberry Pi Zero/1**: Use 1-2
- Incorrect values can cause display corruption, flickering, or system instability - Incorrect values can cause display corruption, flickering, or system instability
- If you experience issues, try adjusting this value up or down by 1 - If you experience issues, try adjusting this value up or down by 1

Binary file not shown.

Before

Width:  |  Height:  |  Size: 105 KiB

After

Width:  |  Height:  |  Size: 657 KiB

View File

@@ -1,43 +1,43 @@
{ {
"web_display_autostart": true, "web_display_autostart": true,
"schedule": { "schedule": {
"enabled": true, "enabled": false,
"mode": "per-day", "mode": "per-day",
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00", "end_time": "23:00",
"days": { "days": {
"monday": { "monday": {
"enabled": true, "enabled": false,
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00" "end_time": "23:00"
}, },
"tuesday": { "tuesday": {
"enabled": true, "enabled": false,
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00" "end_time": "23:00"
}, },
"wednesday": { "wednesday": {
"enabled": true, "enabled": false,
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00" "end_time": "23:00"
}, },
"thursday": { "thursday": {
"enabled": true, "enabled": false,
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00" "end_time": "23:00"
}, },
"friday": { "friday": {
"enabled": true, "enabled": false,
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00" "end_time": "23:00"
}, },
"saturday": { "saturday": {
"enabled": true, "enabled": false,
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00" "end_time": "23:00"
}, },
"sunday": { "sunday": {
"enabled": true, "enabled": false,
"start_time": "07:00", "start_time": "07:00",
"end_time": "23:00" "end_time": "23:00"
} }
@@ -51,46 +51,46 @@
"end_time": "07:00", "end_time": "07:00",
"days": { "days": {
"monday": { "monday": {
"enabled": true, "enabled": false,
"start_time": "20:00", "start_time": "20:00",
"end_time": "07:00" "end_time": "07:00"
}, },
"tuesday": { "tuesday": {
"enabled": true, "enabled": false,
"start_time": "20:00", "start_time": "20:00",
"end_time": "07:00" "end_time": "07:00"
}, },
"wednesday": { "wednesday": {
"enabled": true, "enabled": false,
"start_time": "20:00", "start_time": "20:00",
"end_time": "07:00" "end_time": "07:00"
}, },
"thursday": { "thursday": {
"enabled": true, "enabled": false,
"start_time": "20:00", "start_time": "20:00",
"end_time": "07:00" "end_time": "07:00"
}, },
"friday": { "friday": {
"enabled": true, "enabled": false,
"start_time": "20:00", "start_time": "20:00",
"end_time": "07:00" "end_time": "07:00"
}, },
"saturday": { "saturday": {
"enabled": true, "enabled": false,
"start_time": "20:00", "start_time": "20:00",
"end_time": "07:00" "end_time": "07:00"
}, },
"sunday": { "sunday": {
"enabled": true, "enabled": false,
"start_time": "20:00", "start_time": "20:00",
"end_time": "07:00" "end_time": "07:00"
} }
} }
}, },
"timezone": "America/Chicago", "timezone": "America/New_York",
"location": { "location": {
"city": "Dallas", "city": "Tampa",
"state": "Texas", "state": "Florida",
"country": "US" "country": "US"
}, },
"display": { "display": {
@@ -112,7 +112,8 @@
"limit_refresh_rate_hz": 100 "limit_refresh_rate_hz": 100
}, },
"runtime": { "runtime": {
"gpio_slowdown": 3 "gpio_slowdown": 3,
"rp1_rio": 0
}, },
"display_durations": {}, "display_durations": {},
"use_short_date_format": true, "use_short_date_format": true,
@@ -126,6 +127,11 @@
"buffer_ahead": 2 "buffer_ahead": 2
} }
}, },
"sync": {
"role": "standalone",
"port": 5765,
"follower_position": "left"
},
"plugin_system": { "plugin_system": {
"plugins_directory": "plugin-repos", "plugins_directory": "plugin-repos",
"auto_discover": true, "auto_discover": true,

View File

@@ -1,17 +1,9 @@
{ {
"ledmatrix-weather": {
"api_key": "YOUR_OPENWEATHERMAP_API_KEY"
},
"youtube": { "youtube": {
"api_key": "YOUR_YOUTUBE_API_KEY", "api_key": "YOUR_YOUTUBE_API_KEY",
"channel_id": "YOUR_YOUTUBE_CHANNEL_ID" "channel_id": "YOUR_YOUTUBE_CHANNEL_ID"
}, },
"music": {
"SPOTIFY_CLIENT_ID": "YOUR_SPOTIFY_CLIENT_ID_HERE",
"SPOTIFY_CLIENT_SECRET": "YOUR_SPOTIFY_CLIENT_SECRET_HERE",
"SPOTIFY_REDIRECT_URI": "http://127.0.0.1:8888/callback"
},
"github": { "github": {
"api_token": "YOUR_GITHUB_PERSONAL_ACCESS_TOKEN" "api_token": "YOUR_GITHUB_PERSONAL_ACCESS_TOKEN"
} }
} }

View File

@@ -34,16 +34,16 @@ This document outlines the transformation of the LEDMatrix project into a modula
## Table of Contents ## Table of Contents
1. [Current Architecture Analysis](#current-architecture-analysis) 1. [Current Architecture Analysis](#1-current-architecture-analysis)
2. [Plugin System Design](#plugin-system-design) 2. [Plugin System Design](#2-plugin-system-design)
3. [Plugin Store & Discovery](#plugin-store--discovery) 3. [Plugin Store & Discovery](#3-plugin-store--discovery)
4. [Web UI Transformation](#web-ui-transformation) 4. [Web UI Transformation](#4-web-ui-transformation)
5. [Migration Strategy](#migration-strategy) 5. [Migration Strategy](#5-migration-strategy)
6. [Plugin Developer Guidelines](#plugin-developer-guidelines) 6. [Plugin Developer Guidelines](#6-plugin-developer-guidelines)
7. [Technical Implementation Details](#technical-implementation-details) 7. [Technical Implementation Details](#7-technical-implementation-details)
8. [Best Practices & Standards](#best-practices--standards) 8. [Best Practices & Standards](#8-best-practices--standards)
9. [Security Considerations](#security-considerations) 9. [Security Considerations](#9-security-considerations)
10. [Implementation Roadmap](#implementation-roadmap) 10. [Implementation Roadmap](#10-implementation-roadmap)
--- ---

View File

@@ -36,9 +36,17 @@ if [ -r /proc/device-tree/model ]; then
DEVICE_MODEL=$(tr -d '\0' </proc/device-tree/model) DEVICE_MODEL=$(tr -d '\0' </proc/device-tree/model)
echo "Detected device: $DEVICE_MODEL" echo "Detected device: $DEVICE_MODEL"
else else
DEVICE_MODEL=""
echo "⚠ Could not detect Raspberry Pi model (continuing anyway)" echo "⚠ Could not detect Raspberry Pi model (continuing anyway)"
fi fi
# Detect Pi 5 for hardware-specific install decisions (RP1 library verification)
IS_PI5=0
if echo "${DEVICE_MODEL:-}" | grep -qi "Raspberry Pi 5"; then
IS_PI5=1
echo "Raspberry Pi 5 detected — will verify RP1 library support."
fi
# Check OS version - must be Raspberry Pi OS Lite (Trixie) # Check OS version - must be Raspberry Pi OS Lite (Trixie)
echo "" echo ""
echo "Checking operating system requirements..." echo "Checking operating system requirements..."
@@ -259,8 +267,6 @@ else
fi fi
echo "" echo ""
CLEAR='
'
CURRENT_STEP="Install system dependencies" CURRENT_STEP="Install system dependencies"
echo "Step 1: Installing system dependencies..." echo "Step 1: Installing system dependencies..."
echo "----------------------------------------" echo "----------------------------------------"
@@ -273,7 +279,7 @@ apt_update
# Install required system packages # Install required system packages
echo "Installing Python packages and dependencies..." echo "Installing Python packages and dependencies..."
apt_install python3-pip python3-venv python3-dev python3-pil python3-pil.imagetk build-essential python3-setuptools python3-wheel cython3 scons cmake ninja-build apt_install python3-pip python3-venv python-dev-is-python3 python3-pil python3-pil.imagetk build-essential python3-setuptools python3-wheel cmake ninja-build
# Install additional system dependencies that might be needed # Install additional system dependencies that might be needed
echo "Installing additional system dependencies..." echo "Installing additional system dependencies..."
@@ -599,9 +605,13 @@ if [ ! -f "$PROJECT_ROOT_DIR/config/config_secrets.json" ]; then
echo "⚠ Template config/config_secrets.template.json not found; creating a minimal secrets file" echo "⚠ Template config/config_secrets.template.json not found; creating a minimal secrets file"
cat > "$PROJECT_ROOT_DIR/config/config_secrets.json" <<'EOF' cat > "$PROJECT_ROOT_DIR/config/config_secrets.json" <<'EOF'
{ {
"weather": { "youtube": {
"api_key": "YOUR_OPENWEATHERMAP_API_KEY" "api_key": "YOUR_YOUTUBE_API_KEY",
} "channel_id": "YOUR_YOUTUBE_CHANNEL_ID"
},
"github": {
"api_token": "YOUR_GITHUB_PERSONAL_ACCESS_TOKEN"
}
} }
EOF EOF
# Check if service runs as root and set ownership accordingly # Check if service runs as root and set ownership accordingly
@@ -667,8 +677,6 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
echo "[$PACKAGE_NUM/$TOTAL_PACKAGES] Installing: $line" echo "[$PACKAGE_NUM/$TOTAL_PACKAGES] Installing: $line"
# Check if package is already installed (basic check - may not catch all cases) # Check if package is already installed (basic check - may not catch all cases)
PACKAGE_NAME=$(echo "$line" | sed -E 's/[<>=!].*$//' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
# Try installing with verbose output and timeout (if available) # Try installing with verbose output and timeout (if available)
# Use --no-cache-dir to avoid cache issues, --verbose for diagnostics # Use --no-cache-dir to avoid cache issues, --verbose for diagnostics
INSTALL_OUTPUT=$(mktemp) INSTALL_OUTPUT=$(mktemp)
@@ -783,9 +791,28 @@ CURRENT_STEP="Build and install rpi-rgb-led-matrix"
echo "Step 6: Building and installing rpi-rgb-led-matrix..." echo "Step 6: Building and installing rpi-rgb-led-matrix..."
echo "-----------------------------------------------------" echo "-----------------------------------------------------"
# If already installed and not forcing rebuild, skip expensive build # On Pi 5, also check that the installed library has rp1_rio support.
# A library built before Pi 5 support was added imports fine but maps to the
# Pi 3 peripheral bus address (0x3f000000) instead of the RP1 chip at runtime.
_HAS_RP1=0
if python3 -c 'from rgbmatrix import RGBMatrixOptions; assert hasattr(RGBMatrixOptions(), "rp1_rio")' >/dev/null 2>&1; then
_HAS_RP1=1
fi
_SKIP_BUILD=0
if python3 -c 'from rgbmatrix import RGBMatrix, RGBMatrixOptions' >/dev/null 2>&1 && [ "${RPI_RGB_FORCE_REBUILD:-0}" != "1" ]; then if python3 -c 'from rgbmatrix import RGBMatrix, RGBMatrixOptions' >/dev/null 2>&1 && [ "${RPI_RGB_FORCE_REBUILD:-0}" != "1" ]; then
echo "rgbmatrix Python package already available; skipping build (set RPI_RGB_FORCE_REBUILD=1 to force rebuild)." if [ "$IS_PI5" = "1" ] && [ "$_HAS_RP1" = "0" ]; then
echo "⚠ Pi 5 detected: installed rgbmatrix lacks rp1_rio support (older build)."
echo " Forcing rebuild to get Pi 5 RP1 support..."
else
_SKIP_BUILD=1
fi
fi
if [ "$_SKIP_BUILD" = "1" ]; then
_skip_suffix=""
if [ "$IS_PI5" = "1" ]; then _skip_suffix=" with Pi 5 RP1 support"; fi
echo "rgbmatrix already installed${_skip_suffix}; skipping build (set RPI_RGB_FORCE_REBUILD=1 to force rebuild)."
else else
# Ensure rpi-rgb-led-matrix submodule is initialized # Ensure rpi-rgb-led-matrix submodule is initialized
if [ ! -d "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" ]; then if [ ! -d "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" ]; then
@@ -821,20 +848,13 @@ else
fi fi
pushd "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" >/dev/null pushd "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" >/dev/null
echo "Building rpi-rgb-led-matrix Python bindings..." echo "Installing rpi-rgb-led-matrix Python package (scikit-build-core + cmake)..."
# Build the library first, then Python bindings echo " Build deps required: python-dev-is-python3 cmake"
# The build-python target depends on the library being built echo " This compiles C++ — may take 2-5 minutes on Pi 4/5..."
if ! make build-python; then
echo "✗ Failed to build rpi-rgb-led-matrix Python bindings"
echo " Make sure you have the required build tools installed:"
echo " sudo apt install -y build-essential python3-dev cython3 scons"
popd >/dev/null
exit 1
fi
cd bindings/python
echo "Installing rpi-rgb-led-matrix Python package via pip..."
if ! python3 -m pip install --break-system-packages .; then if ! python3 -m pip install --break-system-packages .; then
echo "✗ Failed to install rpi-rgb-led-matrix Python package" echo "✗ Failed to install rpi-rgb-led-matrix Python package"
echo " Ensure build tools are installed:"
echo " sudo apt install -y python-dev-is-python3 cmake build-essential"
popd >/dev/null popd >/dev/null
exit 1 exit 1
fi fi
@@ -859,6 +879,17 @@ except Exception as e:
PY PY
then then
echo "✓ rpi-rgb-led-matrix installed and verified" echo "✓ rpi-rgb-led-matrix installed and verified"
# Pi 5: confirm the freshly-built library has rp1_rio support
if [ "$IS_PI5" = "1" ]; then
if python3 -c 'from rgbmatrix import RGBMatrixOptions; assert hasattr(RGBMatrixOptions(), "rp1_rio")' >/dev/null 2>&1; then
echo "✓ Pi 5 RP1 (rp1_rio) support confirmed"
else
echo "⚠ rp1_rio not found after rebuild — the submodule may be an older version."
echo " Try updating the submodule and rebuilding:"
echo " git submodule update --remote rpi-rgb-led-matrix-master"
echo " sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh"
fi
fi
else else
echo "✗ rpi-rgb-led-matrix import test failed" echo "✗ rpi-rgb-led-matrix import test failed"
exit 1 exit 1
@@ -1082,6 +1113,7 @@ SYSTEMCTL_PATH=$(which systemctl)
REBOOT_PATH=$(which reboot) REBOOT_PATH=$(which reboot)
POWEROFF_PATH=$(which poweroff) POWEROFF_PATH=$(which poweroff)
BASH_PATH=$(which bash) BASH_PATH=$(which bash)
JOURNALCTL_PATH=$(which journalctl 2>/dev/null || true)
# Create sudoers content # Create sudoers content
cat > /tmp/ledmatrix_web_sudoers << EOF cat > /tmp/ledmatrix_web_sudoers << EOF
@@ -1097,10 +1129,23 @@ $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH enable ledmatrix.service $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH enable ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH disable ledmatrix.service $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH disable ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT_DIR/display_controller.py $ACTUAL_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT_DIR/display_controller.py
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/start_display.sh $ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/start_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/stop_display.sh $ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/stop_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_plugin_rm.sh *
EOF EOF
if [ -n "$JOURNALCTL_PATH" ]; then
cat >> /tmp/ledmatrix_web_sudoers << EOF
$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -u ledmatrix.service *
$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -u ledmatrix *
$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -t ledmatrix *
EOF
fi
if [ -f "$SUDOERS_FILE" ] && cmp -s /tmp/ledmatrix_web_sudoers "$SUDOERS_FILE"; then if [ -f "$SUDOERS_FILE" ] && cmp -s /tmp/ledmatrix_web_sudoers "$SUDOERS_FILE"; then
echo "Sudoers configuration already up to date" echo "Sudoers configuration already up to date"
@@ -1461,7 +1506,7 @@ echo "WiFi Connection Status:"
if command -v nmcli >/dev/null 2>&1; then if command -v nmcli >/dev/null 2>&1; then
WIFI_STATUS=$(nmcli -t -f DEVICE,TYPE,STATE device status 2>/dev/null | grep -i wifi || echo "") WIFI_STATUS=$(nmcli -t -f DEVICE,TYPE,STATE device status 2>/dev/null | grep -i wifi || echo "")
if [ -n "$WIFI_STATUS" ]; then if [ -n "$WIFI_STATUS" ]; then
echo "$WIFI_STATUS" | while IFS=':' read -r device type state; do echo "$WIFI_STATUS" | while IFS=':' read -r _ _ state; do
if [ "$state" = "connected" ]; then if [ "$state" = "connected" ]; then
SSID=$(nmcli -t -f active,ssid device wifi 2>/dev/null | grep "^yes:" | cut -d: -f2 | head -1) SSID=$(nmcli -t -f active,ssid device wifi 2>/dev/null | grep "^yes:" | cut -d: -f2 | head -1)
if [ -n "$SSID" ]; then if [ -n "$SSID" ]; then

View File

@@ -1,138 +0,0 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "March Madness Plugin Configuration",
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"default": false,
"description": "Enable the March Madness tournament display"
},
"leagues": {
"type": "object",
"title": "Tournament Leagues",
"description": "Which NCAA tournaments to display",
"properties": {
"ncaam": {
"type": "boolean",
"default": true,
"description": "Show NCAA Men's Tournament games"
},
"ncaaw": {
"type": "boolean",
"default": true,
"description": "Show NCAA Women's Tournament games"
}
},
"additionalProperties": false
},
"favorite_teams": {
"type": "array",
"title": "Favorite Teams",
"description": "Team abbreviations to highlight (e.g., DUKE, UNC). Leave empty to show all teams equally.",
"items": {
"type": "string"
},
"uniqueItems": true,
"default": []
},
"display_options": {
"type": "object",
"title": "Display Options",
"x-collapsed": true,
"properties": {
"show_seeds": {
"type": "boolean",
"default": true,
"description": "Show tournament seeds (1-16) next to team names"
},
"show_round_logos": {
"type": "boolean",
"default": true,
"description": "Show round logo separators between game groups"
},
"highlight_upsets": {
"type": "boolean",
"default": true,
"description": "Highlight upset winners (higher seed beating lower seed) in gold"
},
"show_bracket_progress": {
"type": "boolean",
"default": true,
"description": "Show which teams are still alive in each region"
},
"scroll_speed": {
"type": "number",
"default": 1.0,
"minimum": 0.5,
"maximum": 5.0,
"description": "Scroll speed (pixels per frame)"
},
"scroll_delay": {
"type": "number",
"default": 0.02,
"minimum": 0.001,
"maximum": 0.1,
"description": "Delay between scroll frames (seconds)"
},
"target_fps": {
"type": "integer",
"default": 120,
"minimum": 30,
"maximum": 200,
"description": "Target frames per second"
},
"loop": {
"type": "boolean",
"default": true,
"description": "Loop the scroll continuously"
},
"dynamic_duration": {
"type": "boolean",
"default": true,
"description": "Automatically adjust display duration based on content width"
},
"min_duration": {
"type": "integer",
"default": 30,
"minimum": 10,
"maximum": 300,
"description": "Minimum display duration in seconds"
},
"max_duration": {
"type": "integer",
"default": 300,
"minimum": 30,
"maximum": 600,
"description": "Maximum display duration in seconds"
}
},
"additionalProperties": false
},
"data_settings": {
"type": "object",
"title": "Data Settings",
"x-collapsed": true,
"properties": {
"update_interval": {
"type": "integer",
"default": 300,
"minimum": 60,
"maximum": 3600,
"description": "How often to refresh tournament data (seconds). Automatically shortens to 60s when live games are detected."
},
"request_timeout": {
"type": "integer",
"default": 30,
"minimum": 5,
"maximum": 60,
"description": "API request timeout in seconds"
}
},
"additionalProperties": false
}
},
"required": ["enabled"],
"additionalProperties": false,
"x-propertyOrder": ["enabled", "leagues", "favorite_teams", "display_options", "data_settings"]
}

View File

@@ -1,910 +0,0 @@
"""March Madness Plugin — NCAA Tournament bracket tracker for LED Matrix.
Displays a horizontally-scrolling ticker of NCAA Tournament games grouped by
round, with seeds, round logos, live scores, and upset highlighting.
"""
import re
import threading
import time
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
import numpy as np
import pytz
import requests
from PIL import Image, ImageDraw, ImageFont
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.plugin_system.base_plugin import BasePlugin
try:
from src.common.scroll_helper import ScrollHelper
except ImportError:
ScrollHelper = None
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
SCOREBOARD_URLS = {
"ncaam": "https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/scoreboard",
"ncaaw": "https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/scoreboard",
}
ROUND_ORDER = {"NCG": 0, "F4": 1, "E8": 2, "S16": 3, "R32": 4, "R64": 5, "": 6}
ROUND_DISPLAY_NAMES = {
"NCG": "Championship",
"F4": "Final Four",
"E8": "Elite Eight",
"S16": "Sweet Sixteen",
"R32": "Round of 32",
"R64": "Round of 64",
}
ROUND_LOGO_FILES = {
"NCG": "CHAMPIONSHIP.png",
"F4": "FINAL_4.png",
"E8": "ELITE_8.png",
"S16": "SWEET_16.png",
"R32": "ROUND_32.png",
"R64": "ROUND_64.png",
}
REGION_ORDER = {"E": 0, "W": 1, "S": 2, "MW": 3, "": 4}
# Colors
COLOR_WHITE = (255, 255, 255)
COLOR_GOLD = (255, 215, 0)
COLOR_GRAY = (160, 160, 160)
COLOR_DIM = (100, 100, 100)
COLOR_RED = (255, 60, 60)
COLOR_GREEN = (60, 200, 60)
COLOR_BLACK = (0, 0, 0)
COLOR_DARK_BG = (20, 20, 20)
# ---------------------------------------------------------------------------
# Plugin Class
# ---------------------------------------------------------------------------
class MarchMadnessPlugin(BasePlugin):
"""NCAA March Madness tournament bracket tracker."""
def __init__(
self,
plugin_id: str,
config: Dict[str, Any],
display_manager: Any,
cache_manager: Any,
plugin_manager: Any,
):
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# Config
leagues_config = config.get("leagues", {})
self.show_ncaam: bool = leagues_config.get("ncaam", True)
self.show_ncaaw: bool = leagues_config.get("ncaaw", True)
self.favorite_teams: List[str] = [t.upper() for t in config.get("favorite_teams", [])]
display_options = config.get("display_options", {})
self.show_seeds: bool = display_options.get("show_seeds", True)
self.show_round_logos: bool = display_options.get("show_round_logos", True)
self.highlight_upsets: bool = display_options.get("highlight_upsets", True)
self.show_bracket_progress: bool = display_options.get("show_bracket_progress", True)
self.scroll_speed: float = display_options.get("scroll_speed", 1.0)
self.scroll_delay: float = display_options.get("scroll_delay", 0.02)
self.target_fps: int = display_options.get("target_fps", 120)
self.loop: bool = display_options.get("loop", True)
self.dynamic_duration_enabled: bool = display_options.get("dynamic_duration", True)
self.min_duration: int = display_options.get("min_duration", 30)
self.max_duration: int = display_options.get("max_duration", 300)
if self.min_duration > self.max_duration:
self.logger.warning(
f"min_duration ({self.min_duration}) > max_duration ({self.max_duration}); swapping values"
)
self.min_duration, self.max_duration = self.max_duration, self.min_duration
data_settings = config.get("data_settings", {})
self.update_interval: int = data_settings.get("update_interval", 300)
self.request_timeout: int = data_settings.get("request_timeout", 30)
# Scrolling flag for display controller
self.enable_scrolling = True
# State
self.games_data: List[Dict] = []
self.ticker_image: Optional[Image.Image] = None
self.last_update: float = 0
self.dynamic_duration: float = 60
self.total_scroll_width: int = 0
self._display_start_time: Optional[float] = None
self._end_reached_logged: bool = False
self._update_lock = threading.Lock()
self._has_live_games: bool = False
self._cached_dynamic_duration: Optional[float] = None
self._duration_cache_time: float = 0
# Display dimensions
self.display_width: int = self.display_manager.matrix.width
self.display_height: int = self.display_manager.matrix.height
# HTTP session with retry
self.session = requests.Session()
retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504])
self.session.mount("https://", HTTPAdapter(max_retries=retry))
self.headers = {"User-Agent": "LEDMatrix/2.0"}
# ScrollHelper
if ScrollHelper:
self.scroll_helper = ScrollHelper(self.display_width, self.display_height, logger=self.logger)
if hasattr(self.scroll_helper, "set_frame_based_scrolling"):
self.scroll_helper.set_frame_based_scrolling(True)
self.scroll_helper.set_scroll_speed(self.scroll_speed)
self.scroll_helper.set_scroll_delay(self.scroll_delay)
if hasattr(self.scroll_helper, "set_target_fps"):
self.scroll_helper.set_target_fps(self.target_fps)
self.scroll_helper.set_dynamic_duration_settings(
enabled=self.dynamic_duration_enabled,
min_duration=self.min_duration,
max_duration=self.max_duration,
buffer=0.1,
)
else:
self.scroll_helper = None
self.logger.warning("ScrollHelper not available")
# Fonts
self.fonts = self._load_fonts()
# Logos
self._round_logos: Dict[str, Image.Image] = {}
self._team_logo_cache: Dict[str, Optional[Image.Image]] = {}
self._march_madness_logo: Optional[Image.Image] = None
self._load_round_logos()
self.logger.info(
f"MarchMadnessPlugin initialized — NCAAM: {self.show_ncaam}, "
f"NCAAW: {self.show_ncaaw}, favorites: {self.favorite_teams}"
)
# ------------------------------------------------------------------
# Fonts
# ------------------------------------------------------------------
def _load_fonts(self) -> Dict[str, ImageFont.FreeTypeFont]:
fonts = {}
try:
fonts["score"] = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 10)
except IOError:
fonts["score"] = ImageFont.load_default()
try:
fonts["time"] = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
except IOError:
fonts["time"] = ImageFont.load_default()
try:
fonts["detail"] = ImageFont.truetype("assets/fonts/4x6-font.ttf", 6)
except IOError:
fonts["detail"] = ImageFont.load_default()
return fonts
# ------------------------------------------------------------------
# Logo loading
# ------------------------------------------------------------------
def _load_round_logos(self) -> None:
logo_dir = Path("assets/sports/ncaa_logos")
for round_key, filename in ROUND_LOGO_FILES.items():
path = logo_dir / filename
try:
img = Image.open(path).convert("RGBA")
# Resize to fit display height
target_h = self.display_height - 4
ratio = target_h / img.height
target_w = int(img.width * ratio)
self._round_logos[round_key] = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
except (OSError, ValueError) as e:
self.logger.warning(f"Could not load round logo {filename}: {e}")
except Exception:
self.logger.exception(f"Unexpected error loading round logo {filename}")
# March Madness logo
mm_path = logo_dir / "MARCH_MADNESS.png"
try:
img = Image.open(mm_path).convert("RGBA")
target_h = self.display_height - 4
ratio = target_h / img.height
target_w = int(img.width * ratio)
self._march_madness_logo = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
except (OSError, ValueError) as e:
self.logger.warning(f"Could not load March Madness logo: {e}")
except Exception:
self.logger.exception("Unexpected error loading March Madness logo")
def _get_team_logo(self, abbr: str) -> Optional[Image.Image]:
if abbr in self._team_logo_cache:
return self._team_logo_cache[abbr]
logo_dir = Path("assets/sports/ncaa_logos")
path = logo_dir / f"{abbr}.png"
try:
img = Image.open(path).convert("RGBA")
target_h = self.display_height - 6
ratio = target_h / img.height
target_w = int(img.width * ratio)
img = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
self._team_logo_cache[abbr] = img
return img
except (FileNotFoundError, OSError, ValueError):
self._team_logo_cache[abbr] = None
return None
except Exception:
self.logger.exception(f"Unexpected error loading team logo for {abbr}")
self._team_logo_cache[abbr] = None
return None
# ------------------------------------------------------------------
# Data fetching
# ------------------------------------------------------------------
def _is_tournament_window(self) -> bool:
today = datetime.now(pytz.utc)
return (3, 10) <= (today.month, today.day) <= (4, 10)
def _fetch_tournament_data(self) -> List[Dict]:
"""Fetch tournament games from ESPN scoreboard API."""
all_games: List[Dict] = []
leagues = []
if self.show_ncaam:
leagues.append("ncaam")
if self.show_ncaaw:
leagues.append("ncaaw")
for league_key in leagues:
url = SCOREBOARD_URLS.get(league_key)
if not url:
continue
cache_key = f"march_madness_{league_key}_scoreboard"
cache_max_age = 60 if self._has_live_games else self.update_interval
cached = self.cache_manager.get(cache_key, max_age=cache_max_age)
if cached:
all_games.extend(cached)
continue
try:
# NCAA basketball scoreboard without dates param returns current games
params = {"limit": 1000, "groups": 100}
resp = self.session.get(url, params=params, headers=self.headers, timeout=self.request_timeout)
resp.raise_for_status()
data = resp.json()
events = data.get("events", [])
league_games = []
for event in events:
game = self._parse_event(event, league_key)
if game:
league_games.append(game)
self.cache_manager.set(cache_key, league_games)
self.logger.info(f"Fetched {len(league_games)} {league_key} tournament games")
all_games.extend(league_games)
except Exception:
self.logger.exception(f"Error fetching {league_key} tournament data")
return all_games
def _parse_event(self, event: Dict, league_key: str) -> Optional[Dict]:
"""Parse an ESPN event into a game dict."""
competitions = event.get("competitions", [])
if not competitions:
return None
comp = competitions[0]
# Confirm tournament game
comp_type = comp.get("type", {})
is_tournament = comp_type.get("abbreviation") == "TRNMNT"
notes = comp.get("notes", [])
headline = ""
if notes:
headline = notes[0].get("headline", "")
if not is_tournament and "Championship" in headline:
is_tournament = True
if not is_tournament:
return None
# Status
status = comp.get("status", {}).get("type", {})
state = status.get("state", "pre")
status_detail = status.get("shortDetail", "")
# Teams
competitors = comp.get("competitors", [])
home_team = next((c for c in competitors if c.get("homeAway") == "home"), None)
away_team = next((c for c in competitors if c.get("homeAway") == "away"), None)
if not home_team or not away_team:
return None
home_abbr = home_team.get("team", {}).get("abbreviation", "???")
away_abbr = away_team.get("team", {}).get("abbreviation", "???")
home_score = home_team.get("score", "0")
away_score = away_team.get("score", "0")
# Seeds
home_seed = home_team.get("curatedRank", {}).get("current", 0)
away_seed = away_team.get("curatedRank", {}).get("current", 0)
if home_seed >= 99:
home_seed = 0
if away_seed >= 99:
away_seed = 0
# Round and region
tournament_round = self._parse_round(headline)
tournament_region = self._parse_region(headline)
# Date/time
date_str = event.get("date", "")
start_time_utc = None
game_date = ""
game_time = ""
try:
if date_str.endswith("Z"):
date_str = date_str.replace("Z", "+00:00")
dt = datetime.fromisoformat(date_str)
if dt.tzinfo is None:
start_time_utc = dt.replace(tzinfo=pytz.UTC)
else:
start_time_utc = dt.astimezone(pytz.UTC)
local = start_time_utc.astimezone(pytz.timezone("US/Eastern"))
game_date = local.strftime("%-m/%-d")
game_time = local.strftime("%-I:%M%p").replace("AM", "am").replace("PM", "pm")
except (ValueError, AttributeError):
pass
# Period / clock for live games
period = 0
clock = ""
period_text = ""
is_halftime = False
if state == "in":
status_obj = comp.get("status", {})
period = status_obj.get("period", 0)
clock = status_obj.get("displayClock", "")
detail_lower = status_detail.lower()
uses_quarters = league_key == "ncaaw" or "quarter" in detail_lower or detail_lower.startswith("q")
if period <= (4 if uses_quarters else 2):
period_text = f"Q{period}" if uses_quarters else f"H{period}"
else:
ot_num = period - (4 if uses_quarters else 2)
period_text = f"OT{ot_num}" if ot_num > 1 else "OT"
if "halftime" in detail_lower:
is_halftime = True
elif state == "post":
period_text = status.get("shortDetail", "Final")
if "Final" not in period_text:
period_text = "Final"
# Determine winner and upset
is_final = state == "post"
is_upset = False
winner_side = ""
if is_final:
try:
h = int(float(home_score))
a = int(float(away_score))
if h > a:
winner_side = "home"
if home_seed > away_seed > 0:
is_upset = True
elif a > h:
winner_side = "away"
if away_seed > home_seed > 0:
is_upset = True
except (ValueError, TypeError):
pass
return {
"id": event.get("id", ""),
"league": league_key,
"home_abbr": home_abbr,
"away_abbr": away_abbr,
"home_score": str(home_score),
"away_score": str(away_score),
"home_seed": home_seed,
"away_seed": away_seed,
"tournament_round": tournament_round,
"tournament_region": tournament_region,
"state": state,
"is_final": is_final,
"is_live": state == "in",
"is_upcoming": state == "pre",
"is_halftime": is_halftime,
"period": period,
"period_text": period_text,
"clock": clock,
"status_detail": status_detail,
"game_date": game_date,
"game_time": game_time,
"start_time_utc": start_time_utc,
"is_upset": is_upset,
"winner_side": winner_side,
"headline": headline,
}
@staticmethod
def _parse_round(headline: str) -> str:
hl = headline.lower()
if "national championship" in hl:
return "NCG"
if "final four" in hl:
return "F4"
if "elite 8" in hl or "elite eight" in hl:
return "E8"
if "sweet 16" in hl or "sweet sixteen" in hl:
return "S16"
if "2nd round" in hl or "second round" in hl:
return "R32"
if "1st round" in hl or "first round" in hl:
return "R64"
return ""
@staticmethod
def _parse_region(headline: str) -> str:
if "East Region" in headline:
return "E"
if "West Region" in headline:
return "W"
if "South Region" in headline:
return "S"
if "Midwest Region" in headline:
return "MW"
m = re.search(r"Regional (\d+)", headline)
if m:
return f"R{m.group(1)}"
return ""
# ------------------------------------------------------------------
# Game processing
# ------------------------------------------------------------------
def _process_games(self, games: List[Dict]) -> Dict[str, List[Dict]]:
"""Group games by round, sorted by round significance then region/seed."""
grouped: Dict[str, List[Dict]] = {}
for game in games:
rnd = game.get("tournament_round", "")
grouped.setdefault(rnd, []).append(game)
# Sort each round's games by region then seed matchup
for rnd, round_games in grouped.items():
round_games.sort(
key=lambda g: (
REGION_ORDER.get(g.get("tournament_region", ""), 4),
min(g.get("away_seed", 99), g.get("home_seed", 99)),
)
)
return grouped
# ------------------------------------------------------------------
# Rendering
# ------------------------------------------------------------------
def _draw_text_with_outline(
self,
draw: ImageDraw.Draw,
text: str,
xy: tuple,
font: ImageFont.FreeTypeFont,
fill: tuple = COLOR_WHITE,
outline: tuple = COLOR_BLACK,
) -> None:
x, y = xy
for dx in (-1, 0, 1):
for dy in (-1, 0, 1):
if dx or dy:
draw.text((x + dx, y + dy), text, font=font, fill=outline)
draw.text((x, y), text, font=font, fill=fill)
def _create_round_separator(self, round_key: str) -> Image.Image:
"""Create a separator tile for a tournament round."""
height = self.display_height
name = ROUND_DISPLAY_NAMES.get(round_key, round_key)
font = self.fonts["time"]
# Measure text
tmp = Image.new("RGB", (1, 1))
tmp_draw = ImageDraw.Draw(tmp)
text_width = int(tmp_draw.textlength(name, font=font))
# Logo on each side
logo = self._round_logos.get(round_key, self._march_madness_logo)
logo_w = logo.width if logo else 0
padding = 6
total_w = padding + logo_w + padding + text_width + padding + logo_w + padding
total_w = max(total_w, 80)
img = Image.new("RGB", (total_w, height), COLOR_DARK_BG)
draw = ImageDraw.Draw(img)
# Draw logos
x = padding
if logo:
logo_y = (height - logo.height) // 2
img.paste(logo, (x, logo_y), logo)
x += logo_w + padding
# Draw round name
text_y = (height - 8) // 2 # 8px font
self._draw_text_with_outline(draw, name, (x, text_y), font, fill=COLOR_GOLD)
x += text_width + padding
if logo:
logo_y = (height - logo.height) // 2
img.paste(logo, (x, logo_y), logo)
return img
def _create_game_tile(self, game: Dict) -> Image.Image:
"""Create a single game tile for the scrolling ticker."""
height = self.display_height
font_score = self.fonts["score"]
font_time = self.fonts["time"]
font_detail = self.fonts["detail"]
# Load team logos
away_logo = self._get_team_logo(game["away_abbr"])
home_logo = self._get_team_logo(game["home_abbr"])
logo_w = 0
if away_logo:
logo_w = max(logo_w, away_logo.width)
if home_logo:
logo_w = max(logo_w, home_logo.width)
if logo_w == 0:
logo_w = 24
# Build text elements
away_seed_str = f"({game['away_seed']})" if self.show_seeds and game.get("away_seed", 0) > 0 else ""
home_seed_str = f"({game['home_seed']})" if self.show_seeds and game.get("home_seed", 0) > 0 else ""
away_text = f"{away_seed_str}{game['away_abbr']}"
home_text = f"{game['home_abbr']}{home_seed_str}"
# Measure text widths
tmp = Image.new("RGB", (1, 1))
tmp_draw = ImageDraw.Draw(tmp)
away_text_w = int(tmp_draw.textlength(away_text, font=font_detail))
home_text_w = int(tmp_draw.textlength(home_text, font=font_detail))
# Center content: status line
if game["is_live"]:
if game["is_halftime"]:
status_text = "Halftime"
else:
status_text = f"{game['period_text']} {game['clock']}".strip()
elif game["is_final"]:
status_text = game.get("period_text", "Final")
else:
status_text = f"{game['game_date']} {game['game_time']}".strip()
status_w = int(tmp_draw.textlength(status_text, font=font_time))
# Score line (for live/final)
score_text = ""
if game["is_live"] or game["is_final"]:
score_text = f"{game['away_score']}-{game['home_score']}"
score_w = int(tmp_draw.textlength(score_text, font=font_score)) if score_text else 0
# Calculate tile width
h_pad = 4
center_w = max(status_w, score_w, 40)
tile_w = h_pad + logo_w + h_pad + away_text_w + h_pad + center_w + h_pad + home_text_w + h_pad + logo_w + h_pad
img = Image.new("RGB", (tile_w, height), COLOR_BLACK)
draw = ImageDraw.Draw(img)
# Paste away logo
x = h_pad
if away_logo:
logo_y = (height - away_logo.height) // 2
img.paste(away_logo, (x, logo_y), away_logo)
x += logo_w + h_pad
# Away team text (seed + abbr)
is_fav_away = game["away_abbr"] in self.favorite_teams if self.favorite_teams else False
away_color = COLOR_GOLD if is_fav_away else COLOR_WHITE
if game["is_final"] and game["winner_side"] == "away" and self.highlight_upsets and game["is_upset"]:
away_color = COLOR_GOLD
team_text_y = (height - 6) // 2 - 5 # Upper half
self._draw_text_with_outline(draw, away_text, (x, team_text_y), font_detail, fill=away_color)
x += away_text_w + h_pad
# Center block
center_x = x
center_mid = center_x + center_w // 2
# Status text (top center of center block)
status_x = center_mid - status_w // 2
status_y = 2
status_color = COLOR_GREEN if game["is_live"] else COLOR_GRAY
self._draw_text_with_outline(draw, status_text, (status_x, status_y), font_time, fill=status_color)
# Score (bottom center of center block, for live/final)
if score_text:
score_x = center_mid - score_w // 2
score_y = height - 13
# Upset highlighting
if game["is_final"] and game["is_upset"] and self.highlight_upsets:
score_color = COLOR_GOLD
elif game["is_live"]:
score_color = COLOR_WHITE
else:
score_color = COLOR_WHITE
self._draw_text_with_outline(draw, score_text, (score_x, score_y), font_score, fill=score_color)
# Date for final games (below score)
if game["is_final"] and game.get("game_date"):
date_w = int(draw.textlength(game["game_date"], font=font_detail))
date_x = center_mid - date_w // 2
date_y = height - 6
self._draw_text_with_outline(draw, game["game_date"], (date_x, date_y), font_detail, fill=COLOR_DIM)
x = center_x + center_w + h_pad
# Home team text
is_fav_home = game["home_abbr"] in self.favorite_teams if self.favorite_teams else False
home_color = COLOR_GOLD if is_fav_home else COLOR_WHITE
if game["is_final"] and game["winner_side"] == "home" and self.highlight_upsets and game["is_upset"]:
home_color = COLOR_GOLD
self._draw_text_with_outline(draw, home_text, (x, team_text_y), font_detail, fill=home_color)
x += home_text_w + h_pad
# Paste home logo
if home_logo:
logo_y = (height - home_logo.height) // 2
img.paste(home_logo, (x, logo_y), home_logo)
return img
def _create_ticker_image(self) -> None:
"""Build the full scrolling ticker image from game tiles."""
if not self.games_data:
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
grouped = self._process_games(self.games_data)
content_items: List[Image.Image] = []
# Order rounds by significance (most important first)
sorted_rounds = sorted(grouped.keys(), key=lambda r: ROUND_ORDER.get(r, 6))
for rnd in sorted_rounds:
games = grouped[rnd]
if not games:
continue
# Add round separator
if self.show_round_logos and rnd:
separator = self._create_round_separator(rnd)
content_items.append(separator)
# Add game tiles
for game in games:
tile = self._create_game_tile(game)
content_items.append(tile)
if not content_items:
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
if not self.scroll_helper:
self.ticker_image = None
return
gap_width = 16
# Use ScrollHelper to create the scrolling image
self.ticker_image = self.scroll_helper.create_scrolling_image(
content_items=content_items,
item_gap=gap_width,
element_gap=0,
)
self.total_scroll_width = self.scroll_helper.total_scroll_width
self.dynamic_duration = self.scroll_helper.get_dynamic_duration()
self.logger.info(
f"Ticker image created: {self.ticker_image.width}px wide, "
f"{len(self.games_data)} games, dynamic_duration={self.dynamic_duration:.0f}s"
)
# ------------------------------------------------------------------
# Plugin lifecycle
# ------------------------------------------------------------------
def update(self) -> None:
"""Fetch and process tournament data."""
if not self.enabled:
return
current_time = time.time()
# Use shorter interval if live games detected
interval = 60 if self._has_live_games else self.update_interval
if current_time - self.last_update < interval:
return
with self._update_lock:
self.last_update = current_time
if not self._is_tournament_window():
self.logger.debug("Outside tournament window, skipping fetch")
self.games_data = []
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
try:
games = self._fetch_tournament_data()
self._has_live_games = any(g["is_live"] for g in games)
self.games_data = games
self._create_ticker_image()
self.logger.info(
f"Updated: {len(games)} games, "
f"live={self._has_live_games}"
)
except Exception as e:
self.logger.error(f"Update error: {e}", exc_info=True)
def display(self, force_clear: bool = False) -> None:
"""Render one scroll frame."""
if not self.enabled:
return
if force_clear or self._display_start_time is None:
self._display_start_time = time.time()
if self.scroll_helper:
self.scroll_helper.reset_scroll()
self._end_reached_logged = False
if not self.games_data or self.ticker_image is None:
self._display_fallback()
return
if not self.scroll_helper:
self._display_fallback()
return
try:
if self.loop or not self.scroll_helper.is_scroll_complete():
self.scroll_helper.update_scroll_position()
elif not self._end_reached_logged:
self.logger.info("Scroll complete")
self._end_reached_logged = True
visible = self.scroll_helper.get_visible_portion()
if visible is None:
self._display_fallback()
return
self.dynamic_duration = self.scroll_helper.get_dynamic_duration()
matrix_w = self.display_manager.matrix.width
matrix_h = self.display_manager.matrix.height
if not hasattr(self.display_manager, "image") or self.display_manager.image is None:
self.display_manager.image = Image.new("RGB", (matrix_w, matrix_h), COLOR_BLACK)
self.display_manager.image.paste(visible, (0, 0))
self.display_manager.update_display()
self.scroll_helper.log_frame_rate()
except Exception as e:
self.logger.error(f"Display error: {e}", exc_info=True)
self._display_fallback()
def _display_fallback(self) -> None:
w = self.display_manager.matrix.width
h = self.display_manager.matrix.height
img = Image.new("RGB", (w, h), COLOR_BLACK)
draw = ImageDraw.Draw(img)
if self._is_tournament_window():
text = "No games"
else:
text = "Off-season"
text_w = int(draw.textlength(text, font=self.fonts["time"]))
text_x = (w - text_w) // 2
text_y = (h - 8) // 2
draw.text((text_x, text_y), text, font=self.fonts["time"], fill=COLOR_GRAY)
# Show March Madness logo if available
if self._march_madness_logo:
logo_y = (h - self._march_madness_logo.height) // 2
img.paste(self._march_madness_logo, (2, logo_y), self._march_madness_logo)
self.display_manager.image = img
self.display_manager.update_display()
# ------------------------------------------------------------------
# Duration / cycle management
# ------------------------------------------------------------------
def get_display_duration(self) -> float:
current_time = time.time()
if self._cached_dynamic_duration is not None:
cache_age = current_time - self._duration_cache_time
if cache_age < 5.0:
return self._cached_dynamic_duration
self._cached_dynamic_duration = self.dynamic_duration
self._duration_cache_time = current_time
return self.dynamic_duration
def supports_dynamic_duration(self) -> bool:
if not self.enabled:
return False
return self.dynamic_duration_enabled
def is_cycle_complete(self) -> bool:
if not self.supports_dynamic_duration():
return True
if self._display_start_time is not None and self.dynamic_duration > 0:
elapsed = time.time() - self._display_start_time
if elapsed >= self.dynamic_duration:
return True
if not self.loop and self.scroll_helper and self.scroll_helper.is_scroll_complete():
return True
return False
def reset_cycle_state(self) -> None:
super().reset_cycle_state()
self._display_start_time = None
self._end_reached_logged = False
if self.scroll_helper:
self.scroll_helper.reset_scroll()
# ------------------------------------------------------------------
# Vegas mode
# ------------------------------------------------------------------
def get_vegas_content(self):
if not self.games_data:
return None
tiles = []
for game in self.games_data:
tiles.append(self._create_game_tile(game))
return tiles if tiles else None
def get_vegas_content_type(self) -> str:
return "multi"
# ------------------------------------------------------------------
# Info / cleanup
# ------------------------------------------------------------------
def get_info(self) -> Dict:
info = super().get_info()
info["total_games"] = len(self.games_data)
info["has_live_games"] = self._has_live_games
info["dynamic_duration"] = self.dynamic_duration
info["tournament_window"] = self._is_tournament_window()
return info
def cleanup(self) -> None:
self.games_data = []
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
self._team_logo_cache.clear()
if self.session:
self.session.close()
self.session = None
super().cleanup()

View File

@@ -1,37 +0,0 @@
{
"id": "march-madness",
"name": "March Madness",
"version": "1.0.0",
"description": "NCAA March Madness tournament bracket tracker with round branding, seeded matchups, live scores, and upset highlighting",
"author": "ChuckBuilds",
"category": "sports",
"tags": [
"ncaa",
"basketball",
"march-madness",
"tournament",
"bracket",
"scrolling"
],
"repo": "https://github.com/ChuckBuilds/ledmatrix-plugins",
"branch": "main",
"plugin_path": "plugins/march-madness",
"versions": [
{
"version": "1.0.0",
"ledmatrix_min": "2.0.0",
"released": "2026-02-16"
}
],
"stars": 0,
"downloads": 0,
"last_updated": "2026-02-16",
"verified": true,
"screenshot": "",
"display_modes": [
"march_madness"
],
"dependencies": {},
"entry_point": "manager.py",
"class_name": "MarchMadnessPlugin"
}

View File

@@ -1,4 +0,0 @@
requests>=2.28.0
Pillow>=9.1.0
pytz>=2022.1
numpy>=1.24.0

View File

@@ -1,3 +1,3 @@
Pillow>=10.4.0 Pillow>=12.2.0
PyYAML>=6.0.2 PyYAML>=6.0.2
requests>=2.32.0 requests>=2.33.0

View File

@@ -35,24 +35,24 @@ class WebUIInfoPlugin(BasePlugin):
"""Initialize the Web UI Info plugin.""" """Initialize the Web UI Info plugin."""
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager) super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# AP mode cache (must be initialized before _get_local_ip)
self._ap_mode_cached = False
self._ap_mode_cache_time = 0.0
self._ap_mode_cache_ttl = 60.0
# Get device hostname # Get device hostname
try: try:
self.device_id = socket.gethostname() self.device_id = socket.gethostname()
except Exception as e: except Exception as e:
self.logger.warning(f"Could not get hostname: {e}, using 'localhost'") self.logger.warning(f"Could not get hostname: {e}, using 'localhost'")
self.device_id = "localhost" self.device_id = "localhost"
# Get device IP address # Get device IP address
self.device_ip = self._get_local_ip() self.device_ip = self._get_local_ip()
# IP refresh tracking # IP refresh tracking
self.last_ip_refresh = time.time() self.last_ip_refresh = time.time()
self.ip_refresh_interval = 300.0 # Refresh IP every 5 minutes self.ip_refresh_interval = 300.0
# AP mode cache
self._ap_mode_cached = False
self._ap_mode_cache_time = 0.0
self._ap_mode_cache_ttl = 60.0 # Cache AP mode check for 60 seconds
# Rotation state # Rotation state
self.current_display_mode = "hostname" # "hostname" or "ip" self.current_display_mode = "hostname" # "hostname" or "ip"
@@ -200,9 +200,7 @@ class WebUIInfoPlugin(BasePlugin):
elif current_interface == "wlan0": elif current_interface == "wlan0":
self.logger.debug(f"Found WiFi IP: {ip} on {current_interface}") self.logger.debug(f"Found WiFi IP: {ip} on {current_interface}")
return ip return ip
except Exception:
pass
# Last resort: try hostname resolution (often returns 127.0.0.1) # Last resort: try hostname resolution (often returns 127.0.0.1)
try: try:
ip = socket.gethostbyname(socket.gethostname()) ip = socket.gethostbyname(socket.gethostname())

View File

@@ -3,7 +3,7 @@
# Tested on Raspbian OS 12 (Bookworm) and 13 (Trixie) # Tested on Raspbian OS 12 (Bookworm) and 13 (Trixie)
# Image processing # Image processing
Pillow>=10.4.0,<12.0.0 Pillow>=12.2.0,<13.0.0
numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x) numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
# Timezone handling # Timezone handling
@@ -12,7 +12,7 @@ timezonefinder>=6.5.0,<7.0.0 # Updated for better performance and accuracy
geopy>=2.4.1,<3.0.0 geopy>=2.4.1,<3.0.0
# HTTP requests # HTTP requests
requests>=2.32.0,<3.0.0 requests>=2.33.0,<3.0.0
# Google API integration # Google API integration
google-auth-oauthlib>=1.2.0,<2.0.0 google-auth-oauthlib>=1.2.0,<2.0.0
@@ -23,10 +23,10 @@ google-api-python-client>=2.147.0,<3.0.0
freetype-py>=2.5.1,<3.0.0 freetype-py>=2.5.1,<3.0.0
# Spotify integration # Spotify integration
spotipy>=2.24.0,<3.0.0 spotipy>=2.25.2,<3.0.0
# Flask web framework # Flask web framework
Flask>=3.0.0,<4.0.0 Flask>=3.1.3,<4.0.0
# Text processing # Text processing
unidecode>=1.3.8,<2.0.0 unidecode>=1.3.8,<2.0.0
@@ -35,7 +35,7 @@ unidecode>=1.3.8,<2.0.0
icalevents>=0.1.27,<1.0.0 icalevents>=0.1.27,<1.0.0
# WebSocket support # WebSocket support
python-socketio>=5.11.0,<6.0.0 python-socketio>=5.14.0,<6.0.0
python-engineio>=4.9.0,<5.0.0 python-engineio>=4.9.0,<5.0.0
websockets>=12.0,<14.0 websockets>=12.0,<14.0
websocket-client>=1.8.0,<2.0.0 websocket-client>=1.8.0,<2.0.0
@@ -44,7 +44,7 @@ websocket-client>=1.8.0,<2.0.0
jsonschema>=4.20.0,<5.0.0 jsonschema>=4.20.0,<5.0.0
# Testing dependencies # Testing dependencies
pytest>=7.4.0,<8.0.0 pytest>=9.0.3,<10.0.0
pytest-cov>=4.1.0,<5.0.0 pytest-cov>=4.1.0,<5.0.0
pytest-mock>=3.11.0,<4.0.0 pytest-mock>=3.11.0,<4.0.0
mypy>=1.5.0,<2.0.0 mypy>=1.5.0,<2.0.0

1
run.py
View File

@@ -51,7 +51,6 @@ if debug_mode:
# Try to import the plugin system directly to get better error info # Try to import the plugin system directly to get better error info
print("DEBUG: Attempting to import src.plugin_system...", flush=True) print("DEBUG: Attempting to import src.plugin_system...", flush=True)
from src.plugin_system import PluginManager
print("DEBUG: Plugin system import successful", flush=True) print("DEBUG: Plugin system import successful", flush=True)
except ImportError as e: except ImportError as e:
print(f"DEBUG: Plugin system import failed: {e}", flush=True) print(f"DEBUG: Plugin system import failed: {e}", flush=True)

View File

@@ -9,7 +9,7 @@ and preventing validation errors.
import json import json
import sys import sys
from pathlib import Path from pathlib import Path
from typing import Any, Dict, List, Optional from typing import Any, Dict, List
def get_default_for_field(prop: Dict[str, Any]) -> Any: def get_default_for_field(prop: Dict[str, Any]) -> Any:

View File

@@ -9,9 +9,8 @@ Analyze all plugin config schemas to identify issues:
""" """
import json import json
import os
from pathlib import Path from pathlib import Path
from typing import Dict, List, Set, Any from typing import Dict, List, Any
import jsonschema import jsonschema
from jsonschema import Draft7Validator from jsonschema import Draft7Validator

View File

@@ -3,8 +3,6 @@
Check what imports are actually in the app.py file on the Pi Check what imports are actually in the app.py file on the Pi
""" """
import sys
import os
from pathlib import Path from pathlib import Path
# Read the app.py file and check the import lines # Read the app.py file and check the import lines

View File

@@ -203,7 +203,7 @@ link_github_plugin() {
log_info "Repository already exists at $target_dir" log_info "Repository already exists at $target_dir"
if [[ -d "$target_dir/.git" ]]; then if [[ -d "$target_dir/.git" ]]; then
log_info "Updating repository..." log_info "Updating repository..."
(cd "$target_dir" && git pull --rebase || true) (cd "$target_dir" && git pull --rebase) || true
fi fi
else else
# Clone the repository # Clone the repository

View File

@@ -0,0 +1,95 @@
#!/usr/bin/env python3
"""
Pillow compatibility smoke test.
Exercises the Pillow APIs used throughout LEDMatrix to verify a new
Pillow version doesn't break image rendering, font handling, or resize ops.
Run after upgrading Pillow:
python3 scripts/dev/test_pillow_compat.py
"""
import sys
def check(label, fn):
try:
result = fn()
print(f"{label}" + (f"{result}" if result is not None else ""))
return True
except Exception as e:
print(f"{label}{type(e).__name__}: {e}", file=sys.stderr)
return False
def main():
from PIL import Image, ImageDraw, ImageFont
import PIL
print(f"Pillow {PIL.__version__} on Python {sys.version.split()[0]}\n")
failures = 0
print("Image creation:")
failures += not check("Image.new RGB",
lambda: Image.new('RGB', (128, 32), (0, 0, 0)).size)
failures += not check("Image.new RGBA",
lambda: Image.new('RGBA', (64, 64), (255, 0, 0, 128)).size)
failures += not check("Image.new 1-bit",
lambda: Image.new('1', (16, 16)).size)
print("\nDraw operations:")
img = Image.new('RGB', (128, 32), (0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.load_default()
failures += not check("draw.rectangle",
lambda: draw.rectangle([0, 0, 127, 31], outline=(255, 0, 0)))
failures += not check("draw.text",
lambda: draw.text((2, 2), "Hello", fill=(255, 255, 255), font=font))
failures += not check("draw.line",
lambda: draw.line([0, 0, 127, 31], fill=(0, 255, 0)))
print("\nFont metrics (used in text_helper, scroll_helper):")
failures += not check("draw.textlength",
lambda: f"{draw.textlength('Test', font=font):.1f}px")
failures += not check("draw.textbbox",
lambda: draw.textbbox((0, 0), "Test", font=font))
print("\nResampling (used in logo_helper, image_utils, sports base):")
logo = Image.new('RGBA', (200, 200), (255, 128, 0, 200))
failures += not check("Image.Resampling.LANCZOS exists",
lambda: str(Image.Resampling.LANCZOS))
failures += not check("thumbnail with LANCZOS",
lambda: (logo.thumbnail((64, 32), Image.Resampling.LANCZOS), logo.size)[1])
big = Image.new('RGB', (300, 300), (0, 128, 255))
failures += not check("resize with LANCZOS",
lambda: big.resize((128, 32), Image.Resampling.LANCZOS).size)
print("\nComposite / paste (used in display rendering):")
base = Image.new('RGB', (128, 32), (0, 0, 0))
overlay = Image.new('RGBA', (32, 32), (255, 0, 0, 128))
failures += not check("paste RGBA onto RGB",
lambda: (base.paste(overlay.convert('RGB'), (0, 0)), base.size)[1])
failures += not check("Image.alpha_composite",
lambda: Image.alpha_composite(
Image.new('RGBA', (32, 32)), overlay).size)
print("\nImage I/O:")
import io
buf = io.BytesIO()
img.save(buf, format='PNG')
buf.seek(0)
failures += not check("save/load PNG roundtrip",
lambda: Image.open(buf).size)
print()
if failures == 0:
print(f"All checks passed. Pillow {PIL.__version__} is compatible.")
return 0
else:
print(f"{failures} check(s) failed — review output above.", file=sys.stderr)
return 1
if __name__ == '__main__':
sys.exit(main())

View File

@@ -15,7 +15,6 @@ Usage: python tools/validate_python.py <python_file>
import ast import ast
import sys import sys
import os import os
from pathlib import Path
def validate_file(filepath: str) -> bool: def validate_file(filepath: str) -> bool:
"""Validate a Python file for common issues.""" """Validate a Python file for common issues."""

View File

@@ -13,7 +13,6 @@ echo ""
RED='\033[0;31m' RED='\033[0;31m'
GREEN='\033[0;32m' GREEN='\033[0;32m'
YELLOW='\033[1;33m' YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color NC='\033[0m' # No Color
# Get the actual user # Get the actual user

View File

@@ -41,7 +41,7 @@ if [ -f "$PROJECT_DIR/config/config.json" ]; then
echo -e "${GREEN}✓ Config file found${NC}" echo -e "${GREEN}✓ Config file found${NC}"
# Check web_display_autostart setting # Check web_display_autostart setting
AUTOSTART=$(cat "$PROJECT_DIR/config/config.json" | grep -o '"web_display_autostart"[[:space:]]*:[[:space:]]*[a-z]*' | grep -o '[a-z]*$') AUTOSTART=$(grep -o '"web_display_autostart"[[:space:]]*:[[:space:]]*[a-z]*' "$PROJECT_DIR/config/config.json" | grep -o '[a-z]*$')
if [ "$AUTOSTART" == "true" ]; then if [ "$AUTOSTART" == "true" ]; then
echo -e "${GREEN}✓ web_display_autostart: true${NC}" echo -e "${GREEN}✓ web_display_autostart: true${NC}"

View File

@@ -16,11 +16,8 @@ YELLOW='\033[1;33m'
NC='\033[0m' # No Color NC='\033[0m' # No Color
# Check if running as root or with sudo # Check if running as root or with sudo
if [ "$EUID" -ne 0 ]; then if [ "$EUID" -ne 0 ]; then
echo -e "${YELLOW}Warning: Some checks require sudo. Running what we can...${NC}" echo -e "${YELLOW}Warning: Some checks require sudo. Running what we can...${NC}"
SUDO=""
else
SUDO=""
fi fi
PROJECT_DIR="${HOME}/LEDMatrix" PROJECT_DIR="${HOME}/LEDMatrix"

View File

@@ -118,7 +118,7 @@ total_count=${#ARCHITECTURES[@]}
for arch in "${!ARCHITECTURES[@]}"; do for arch in "${!ARCHITECTURES[@]}"; do
if download_binary "$arch" "${ARCHITECTURES[$arch]}"; then if download_binary "$arch" "${ARCHITECTURES[$arch]}"; then
((success_count++)) success_count=$((success_count + 1))
fi fi
done done

View File

@@ -7,12 +7,6 @@ echo "Fixing LEDMatrix assets directory permissions..."
# Get the real user (not root when running with sudo) # Get the real user (not root when running with sudo)
REAL_USER=${SUDO_USER:-$USER} REAL_USER=${SUDO_USER:-$USER}
# Resolve the home directory of the real user robustly
if command -v getent >/dev/null 2>&1; then
REAL_HOME=$(getent passwd "$REAL_USER" | cut -d: -f6)
else
REAL_HOME=$(eval echo ~"$REAL_USER")
fi
REAL_GROUP=$(id -gn "$REAL_USER") REAL_GROUP=$(id -gn "$REAL_USER")
# Get the project directory # Get the project directory

View File

@@ -89,9 +89,9 @@ TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service" echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix" echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service" echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web" echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web" echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web" echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service"
# Optional: journalctl (non-critical — skip if not found) # Optional: journalctl (non-critical — skip if not found)
if [ -n "$JOURNALCTL_PATH" ]; then if [ -n "$JOURNALCTL_PATH" ]; then

View File

@@ -14,9 +14,6 @@ else
ACTUAL_USER=$(whoami) ACTUAL_USER=$(whoami)
fi fi
# Get the home directory of the actual user
USER_HOME=$(eval echo ~$ACTUAL_USER)
# Determine the Project Root Directory (parent of scripts/install/) # Determine the Project Root Directory (parent of scripts/install/)
PROJECT_ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd) PROJECT_ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
@@ -34,7 +31,8 @@ echo "Generating service file with dynamic paths..."
WEB_SERVICE_FILE_CONTENT=$(cat <<EOF WEB_SERVICE_FILE_CONTENT=$(cat <<EOF
[Unit] [Unit]
Description=LED Matrix Web Interface Service Description=LED Matrix Web Interface Service
After=network.target After=network-online.target
Wants=network-online.target
[Service] [Service]
Type=simple Type=simple

View File

@@ -81,7 +81,7 @@ def main() -> int:
help='Plugin config as JSON string') help='Plugin config as JSON string')
parser.add_argument('--mock-data', '-m', default=None, parser.add_argument('--mock-data', '-m', default=None,
help='Path to JSON file with mock cache data') help='Path to JSON file with mock cache data')
parser.add_argument('--output', '-o', default='/tmp/plugin_render.png', parser.add_argument('--output', '-o', default='/tmp/plugin_render.png', # nosec B108 - dev script default; user can override
help='Output PNG path (default: /tmp/plugin_render.png)') help='Output PNG path (default: /tmp/plugin_render.png)')
parser.add_argument('--width', type=int, default=128, help='Display width (default: 128)') parser.add_argument('--width', type=int, default=128, help='Display width (default: 128)')
parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)') parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)')

View File

@@ -7,9 +7,7 @@ Supports both unittest and pytest.
""" """
import sys import sys
import os
import argparse import argparse
import subprocess
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
@@ -198,17 +196,14 @@ def main():
if runner == 'auto': if runner == 'auto':
# Try pytest first, fall back to unittest # Try pytest first, fall back to unittest
try: try:
import pytest
runner = 'pytest' runner = 'pytest'
except ImportError: except ImportError:
runner = 'unittest' runner = 'unittest'
# Run tests # Run tests
if runner == 'pytest': if runner == 'pytest':
import importlib.util
return run_pytest_tests(test_files, args.verbose, args.coverage) return run_pytest_tests(test_files, args.verbose, args.coverage)
else: else:
import importlib.util
return run_unittest_tests(test_files, args.verbose) return run_unittest_tests(test_files, args.verbose)

View File

@@ -6,9 +6,7 @@ This script allows manual clearing of specific cache keys or all cache data.
import os import os
import sys import sys
import json
import argparse import argparse
from pathlib import Path
# Add the src directory to the path so we can import our modules # Add the src directory to the path so we can import our modules
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src')) sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src'))

View File

@@ -111,7 +111,7 @@ def main():
# Ensure PYTHONPATH is set correctly if web_interface.py has relative imports to src # Ensure PYTHONPATH is set correctly if web_interface.py has relative imports to src
# The WorkingDirectory in systemd service should handle this for web_interface.py # The WorkingDirectory in systemd service should handle this for web_interface.py
print(f"Launching web interface v3: {sys.executable} {WEB_INTERFACE_SCRIPT}") print(f"Launching web interface v3: {sys.executable} {WEB_INTERFACE_SCRIPT}")
os.execvp(sys.executable, [sys.executable, WEB_INTERFACE_SCRIPT]) os.execvp(sys.executable, [sys.executable, WEB_INTERFACE_SCRIPT]) # nosec B606 - both args are fixed constants
except Exception as e: except Exception as e:
print(f"Failed to exec web interface: {e}") print(f"Failed to exec web interface: {e}")
sys.exit(1) # Failed to start sys.exit(1) # Failed to start

View File

@@ -10,6 +10,7 @@ import sys
import time import time
import logging import logging
import signal import signal
import subprocess
from pathlib import Path from pathlib import Path
# Add project root to path (parent of scripts/utils/) # Add project root to path (parent of scripts/utils/)
@@ -43,7 +44,11 @@ class WiFiMonitorDaemon:
self.wifi_manager = WiFiManager() self.wifi_manager = WiFiManager()
self.running = True self.running = True
self.last_state = None self.last_state = None
# Counts consecutive checks where nmcli says "connected" but internet is unreachable.
# After _nm_restart_threshold failures, NetworkManager is restarted as a recovery step.
self._consecutive_internet_failures = 0
self._nm_restart_threshold = 5 # ~2.5 min at 30s interval
# Register signal handlers for graceful shutdown # Register signal handlers for graceful shutdown
signal.signal(signal.SIGINT, self._signal_handler) signal.signal(signal.SIGINT, self._signal_handler)
signal.signal(signal.SIGTERM, self._signal_handler) signal.signal(signal.SIGTERM, self._signal_handler)
@@ -122,6 +127,43 @@ class WiFiMonitorDaemon:
else: else:
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}") logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
# Escalating recovery: if nmcli reports connected but actual internet
# is unreachable for several consecutive checks, restart NetworkManager.
# This is done HERE (not inside check_and_manage_ap_mode) to keep the
# AP-enable trigger clean and avoid false-positive AP enables from
# transient packet loss on otherwise working WiFi.
if updated_status.connected and not updated_status.ap_mode_active:
if not self.wifi_manager.check_internet_connectivity():
self._consecutive_internet_failures += 1
logger.warning(
f"Internet unreachable despite nmcli connection "
f"({self._consecutive_internet_failures}/{self._nm_restart_threshold})"
)
if self._consecutive_internet_failures >= self._nm_restart_threshold:
logger.warning("Restarting NetworkManager to recover internet connectivity")
try:
subprocess.run(
["/usr/bin/systemctl", "restart", "NetworkManager"],
capture_output=True, timeout=20, check=True
)
self._consecutive_internet_failures = 0
# NM restart causes a brief WiFi drop; reset the AP-mode grace
# counter so that transient disconnect doesn't count toward
# triggering AP mode.
self.wifi_manager._disconnected_checks = 0
except subprocess.CalledProcessError as e:
logger.error(f"NetworkManager restart failed (rc={e.returncode}); "
"resetting failure counter to avoid tight retry loop")
self._consecutive_internet_failures = 0
except (subprocess.SubprocessError, OSError) as e:
logger.error(f"NetworkManager restart error: {e}; "
"resetting failure counter to avoid tight retry loop")
self._consecutive_internet_failures = 0
else:
self._consecutive_internet_failures = 0
else:
self._consecutive_internet_failures = 0
# Sleep until next check # Sleep until next check
time.sleep(self.check_interval) time.sleep(self.check_interval)

View File

@@ -7,10 +7,7 @@ where Recent/Upcoming managers consume data from the background service cache.
""" """
import time import time
import logging
from typing import Dict, Optional, Any, Callable from typing import Dict, Optional, Any, Callable
from datetime import datetime
import pytz
class BackgroundCacheMixin: class BackgroundCacheMixin:

View File

@@ -14,19 +14,15 @@ Key Features:
- Memory-efficient data storage - Memory-efficient data storage
""" """
import os
import time import time
import logging import logging
import threading import threading
import requests import requests
from typing import Dict, Any, Optional, List, Callable, Union from typing import Dict, Any, Optional, Callable
from datetime import datetime, timedelta
from dataclasses import dataclass, field from dataclasses import dataclass, field
from enum import Enum from enum import Enum
import json
import queue import queue
from concurrent.futures import ThreadPoolExecutor, Future from concurrent.futures import ThreadPoolExecutor
import weakref
from src.cache_manager import CacheManager from src.cache_manager import CacheManager
# Configure logging # Configure logging
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -227,7 +223,7 @@ class BackgroundDataService:
self.stats['cache_misses'] += 1 self.stats['cache_misses'] += 1
# Submit to executor # Submit to executor
future = self.executor.submit(self._fetch_data_worker, request) self.executor.submit(self._fetch_data_worker, request)
logger.info(f"Submitted background fetch request {request_id} for {sport} {year}") logger.info(f"Submitted background fetch request {request_id} for {sport} {year}")
return request_id return request_id
@@ -553,13 +549,12 @@ class BackgroundDataService:
if to_remove: if to_remove:
logger.info(f"Cleared {len(to_remove)} old completed requests") logger.info(f"Cleared {len(to_remove)} old completed requests")
def shutdown(self, wait: bool = True, timeout: int = 30): def shutdown(self, wait: bool = True):
""" """
Shutdown the background data service. Shutdown the background data service.
Args: Args:
wait: Whether to wait for active requests to complete wait: Whether to wait for active requests to complete
timeout: Maximum time to wait for shutdown
""" """
logger.info("Shutting down BackgroundDataService...") logger.info("Shutting down BackgroundDataService...")
@@ -570,24 +565,14 @@ class BackgroundDataService:
for request_id in list(self.active_requests.keys()): for request_id in list(self.active_requests.keys()):
self.cancel_request(request_id) self.cancel_request(request_id)
# Shutdown executor with compatibility for older Python versions self.executor.shutdown(wait=wait)
try:
# Try with timeout parameter (Python 3.9+)
self.executor.shutdown(wait=wait, timeout=timeout)
except TypeError:
# Fallback for older Python versions that don't support timeout
if wait and timeout:
# For older versions, we can't specify timeout, so just wait
self.executor.shutdown(wait=True)
else:
self.executor.shutdown(wait=wait)
logger.info("BackgroundDataService shutdown complete") logger.info("BackgroundDataService shutdown complete")
def __del__(self): def __del__(self):
"""Cleanup when service is destroyed.""" """Cleanup when service is destroyed."""
if not self._shutdown: if not self._shutdown:
self.shutdown(wait=False, timeout=None) self.shutdown(wait=False)
# Global service instance # Global service instance
_background_service: Optional[BackgroundDataService] = None _background_service: Optional[BackgroundDataService] = None

605
src/backup_manager.py Normal file
View File

@@ -0,0 +1,605 @@
"""
User configuration backup and restore.
Packages the user's LEDMatrix configuration, secrets, WiFi settings,
user-uploaded fonts, plugin image uploads, and installed-plugin manifest
into a single ``.zip`` that can be exported from one installation and
imported on a fresh install.
This module is intentionally Flask-free so it can be unit-tested and
used from scripts.
"""
from __future__ import annotations
import json
import logging
import os
import shutil
import socket
import tempfile
import zipfile
from dataclasses import dataclass, field, asdict
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
logger = logging.getLogger(__name__)
SCHEMA_VERSION = 1
# Filenames shipped with the LEDMatrix repository under ``assets/fonts/``.
# Anything present on disk but NOT in this set is treated as a user upload
# and included in backups. Keep this snapshot in sync with the repo — regenerate
# with::
#
# ls assets/fonts/
#
# Tests assert the set matches the checked-in fonts.
BUNDLED_FONTS: frozenset[str] = frozenset({
"10x20.bdf",
"4x6.bdf",
"4x6-font.ttf",
"5by7.regular.ttf",
"5x7.bdf",
"5x8.bdf",
"6x9.bdf",
"6x10.bdf",
"6x12.bdf",
"6x13.bdf",
"6x13B.bdf",
"6x13O.bdf",
"7x13.bdf",
"7x13B.bdf",
"7x13O.bdf",
"7x14.bdf",
"7x14B.bdf",
"8x13.bdf",
"8x13B.bdf",
"8x13O.bdf",
"9x15.bdf",
"9x15B.bdf",
"9x18.bdf",
"9x18B.bdf",
"AUTHORS",
"bdf_font_guide",
"clR6x12.bdf",
"helvR12.bdf",
"ic8x8u.bdf",
"MatrixChunky8.bdf",
"MatrixChunky8X.bdf",
"MatrixLight6.bdf",
"MatrixLight6X.bdf",
"MatrixLight8X.bdf",
"PressStart2P-Regular.ttf",
"README",
"README.md",
"texgyre-27.bdf",
"tom-thumb.bdf",
})
# Relative paths inside the project that the backup knows how to round-trip.
_CONFIG_REL = Path("config/config.json")
_SECRETS_REL = Path("config/config_secrets.json")
_WIFI_REL = Path("config/wifi_config.json")
_FONTS_REL = Path("assets/fonts")
_PLUGIN_UPLOADS_REL = Path("assets/plugins")
_STATE_REL = Path("data/plugin_state.json")
MANIFEST_NAME = "manifest.json"
PLUGINS_MANIFEST_NAME = "plugins.json"
# Hard cap on the size of a single file we'll accept inside an uploaded ZIP
# to limit zip-bomb risk. 50 MB matches the existing plugin-image upload cap.
_MAX_MEMBER_BYTES = 50 * 1024 * 1024
# Hard cap on the total uncompressed size of an uploaded ZIP.
_MAX_TOTAL_BYTES = 200 * 1024 * 1024
# ---------------------------------------------------------------------------
# Data classes
# ---------------------------------------------------------------------------
@dataclass
class RestoreOptions:
"""Which sections of a backup should be restored."""
restore_config: bool = True
restore_secrets: bool = True
restore_wifi: bool = True
restore_fonts: bool = True
restore_plugin_uploads: bool = True
reinstall_plugins: bool = True
@dataclass
class RestoreResult:
"""Outcome of a restore operation."""
success: bool = False
restored: List[str] = field(default_factory=list)
skipped: List[str] = field(default_factory=list)
plugins_to_install: List[Dict[str, Any]] = field(default_factory=list)
plugins_installed: List[str] = field(default_factory=list)
plugins_failed: List[Dict[str, str]] = field(default_factory=list)
errors: List[str] = field(default_factory=list)
manifest: Dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> Dict[str, Any]:
return asdict(self)
# ---------------------------------------------------------------------------
# Manifest helpers
# ---------------------------------------------------------------------------
def _ledmatrix_version(project_root: Path) -> str:
"""Best-effort version string for the current install."""
version_file = project_root / "VERSION"
if version_file.exists():
try:
return version_file.read_text(encoding="utf-8").strip() or "unknown"
except OSError:
pass
head_file = project_root / ".git" / "HEAD"
if head_file.exists():
try:
head = head_file.read_text(encoding="utf-8").strip()
if head.startswith("ref: "):
ref = head[5:]
ref_path = project_root / ".git" / ref
if ref_path.exists():
return ref_path.read_text(encoding="utf-8").strip()[:12] or "unknown"
return head[:12] or "unknown"
except OSError:
pass
return "unknown"
def _build_manifest(contents: List[str], project_root: Path) -> Dict[str, Any]:
return {
"schema_version": SCHEMA_VERSION,
"created_at": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
"ledmatrix_version": _ledmatrix_version(project_root),
"hostname": socket.gethostname(),
"contents": contents,
}
# ---------------------------------------------------------------------------
# Installed-plugin enumeration
# ---------------------------------------------------------------------------
def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
"""
Return a list of currently-installed plugins suitable for the backup
manifest. Each entry has ``plugin_id`` and ``version``.
Reads ``data/plugin_state.json`` if present; otherwise walks the plugin
directory and reads each ``manifest.json``.
"""
plugins: Dict[str, Dict[str, Any]] = {}
state_file = project_root / _STATE_REL
if state_file.exists():
try:
with state_file.open("r", encoding="utf-8") as f:
state = json.load(f)
raw_plugins = state.get("states", {}) if isinstance(state, dict) else {}
if isinstance(raw_plugins, dict):
for plugin_id, info in raw_plugins.items():
if not isinstance(info, dict):
continue
plugins[plugin_id] = {
"plugin_id": plugin_id,
"version": info.get("version") or "",
"enabled": bool(info.get("enabled", True)),
}
except (OSError, json.JSONDecodeError) as e:
logger.warning("Could not read plugin_state.json: %s", e)
# Fall back to scanning plugin-repos/ for manifests.
plugins_root = project_root / "plugin-repos"
if plugins_root.exists():
for entry in sorted(plugins_root.iterdir()):
if not entry.is_dir():
continue
manifest = entry / "manifest.json"
if not manifest.exists():
continue
try:
with manifest.open("r", encoding="utf-8") as f:
data = json.load(f)
except (OSError, json.JSONDecodeError):
continue
plugin_id = data.get("id") or entry.name
if plugin_id not in plugins:
plugins[plugin_id] = {
"plugin_id": plugin_id,
"version": data.get("version", ""),
"enabled": True,
}
return sorted(plugins.values(), key=lambda p: p["plugin_id"])
# ---------------------------------------------------------------------------
# Font filtering
# ---------------------------------------------------------------------------
def iter_user_fonts(project_root: Path) -> List[Path]:
"""Return absolute paths to user-uploaded fonts (anything in
``assets/fonts/`` not listed in :data:`BUNDLED_FONTS`)."""
fonts_dir = project_root / _FONTS_REL
if not fonts_dir.exists():
return []
user_fonts: List[Path] = []
for entry in sorted(fonts_dir.iterdir()):
if entry.is_file() and entry.name not in BUNDLED_FONTS:
user_fonts.append(entry)
return user_fonts
def iter_plugin_uploads(project_root: Path) -> List[Path]:
"""Return every file under ``assets/plugins/*/uploads/`` (recursive)."""
plugin_root = project_root / _PLUGIN_UPLOADS_REL
if not plugin_root.exists():
return []
out: List[Path] = []
for plugin_dir in sorted(plugin_root.iterdir()):
if not plugin_dir.is_dir():
continue
uploads = plugin_dir / "uploads"
if not uploads.exists():
continue
for root, _dirs, files in os.walk(uploads):
for name in sorted(files):
out.append(Path(root) / name)
return out
# ---------------------------------------------------------------------------
# Export
# ---------------------------------------------------------------------------
def create_backup(
project_root: Path,
output_dir: Optional[Path] = None,
) -> Path:
"""
Build a backup ZIP and write it into ``output_dir`` (defaults to
``<project_root>/config/backups/exports/``). Returns the path to the
created file.
"""
project_root = Path(project_root).resolve()
if output_dir is None:
output_dir = project_root / "config" / "backups" / "exports"
output_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
hostname = socket.gethostname() or "ledmatrix"
safe_host = "".join(c for c in hostname if c.isalnum() or c in "-_") or "ledmatrix"
zip_name = f"ledmatrix-backup-{safe_host}-{timestamp}.zip"
zip_path = output_dir / zip_name
contents: List[str] = []
# Stream directly to a temp file so we never hold the whole ZIP in memory.
tmp_path = zip_path.with_suffix(".zip.tmp")
try:
with zipfile.ZipFile(tmp_path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
# Config files.
if (project_root / _CONFIG_REL).exists():
zf.write(project_root / _CONFIG_REL, _CONFIG_REL.as_posix())
contents.append("config")
if (project_root / _SECRETS_REL).exists():
zf.write(project_root / _SECRETS_REL, _SECRETS_REL.as_posix())
contents.append("secrets")
if (project_root / _WIFI_REL).exists():
zf.write(project_root / _WIFI_REL, _WIFI_REL.as_posix())
contents.append("wifi")
# User-uploaded fonts.
user_fonts = iter_user_fonts(project_root)
if user_fonts:
for font in user_fonts:
arcname = font.relative_to(project_root).as_posix()
zf.write(font, arcname)
contents.append("fonts")
# Plugin uploads.
plugin_uploads = iter_plugin_uploads(project_root)
if plugin_uploads:
for upload in plugin_uploads:
arcname = upload.relative_to(project_root).as_posix()
zf.write(upload, arcname)
contents.append("plugin_uploads")
# Installed plugins manifest.
plugins = list_installed_plugins(project_root)
if plugins:
zf.writestr(
PLUGINS_MANIFEST_NAME,
json.dumps(plugins, indent=2),
)
contents.append("plugins")
# Manifest goes last so that `contents` reflects what we actually wrote.
manifest = _build_manifest(contents, project_root)
zf.writestr(MANIFEST_NAME, json.dumps(manifest, indent=2))
os.replace(tmp_path, zip_path)
except Exception:
tmp_path.unlink(missing_ok=True)
raise
logger.info("Created backup %s (%d bytes)", zip_path, zip_path.stat().st_size)
return zip_path
def preview_backup_contents(project_root: Path) -> Dict[str, Any]:
"""Return a summary of what ``create_backup`` would include."""
project_root = Path(project_root).resolve()
return {
"has_config": (project_root / _CONFIG_REL).exists(),
"has_secrets": (project_root / _SECRETS_REL).exists(),
"has_wifi": (project_root / _WIFI_REL).exists(),
"user_fonts": [p.name for p in iter_user_fonts(project_root)],
"plugin_uploads": len(iter_plugin_uploads(project_root)),
"plugins": list_installed_plugins(project_root),
}
# ---------------------------------------------------------------------------
# Validate
# ---------------------------------------------------------------------------
def _safe_extract_path(base_dir: Path, member_name: str) -> Optional[Path]:
"""Resolve a ZIP member name against ``base_dir`` and reject anything
that escapes it. Returns the resolved absolute path, or ``None`` if the
name is unsafe."""
# Reject absolute paths and Windows-style drives outright.
if member_name.startswith(("/", "\\")) or (len(member_name) >= 2 and member_name[1] == ":"):
return None
target = (base_dir / member_name).resolve()
try:
target.relative_to(base_dir.resolve())
except ValueError:
return None
return target
def validate_backup(zip_path: Path) -> Tuple[bool, str, Dict[str, Any]]:
"""
Inspect a backup ZIP without extracting to disk.
Returns ``(ok, error_message, manifest_dict)``. ``manifest_dict`` contains
the parsed manifest plus diagnostic fields:
- ``detected_contents``: list of section names present in the archive
- ``plugins``: parsed plugins.json if present
- ``total_uncompressed``: sum of uncompressed sizes
"""
zip_path = Path(zip_path)
if not zip_path.exists():
return False, f"Backup file not found: {zip_path}", {}
try:
with zipfile.ZipFile(zip_path, "r") as zf:
names = zf.namelist()
if MANIFEST_NAME not in names:
return False, "Backup is missing manifest.json", {}
total = 0
with tempfile.TemporaryDirectory() as _sandbox:
sandbox = Path(_sandbox)
for info in zf.infolist():
if info.file_size > _MAX_MEMBER_BYTES:
return False, f"Member {info.filename} is too large", {}
total += info.file_size
if total > _MAX_TOTAL_BYTES:
return False, "Backup exceeds maximum allowed size", {}
# Safety: reject members with unsafe paths up front.
if _safe_extract_path(sandbox, info.filename) is None:
return False, f"Unsafe path in backup: {info.filename}", {}
try:
manifest_raw = zf.read(MANIFEST_NAME).decode("utf-8")
manifest = json.loads(manifest_raw)
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as e:
return False, f"Invalid manifest.json: {e}", {}
if not isinstance(manifest, dict) or "schema_version" not in manifest:
return False, "Invalid manifest structure", {}
if manifest.get("schema_version") != SCHEMA_VERSION:
return (
False,
f"Unsupported backup schema version: {manifest.get('schema_version')}",
{},
)
detected: List[str] = []
if _CONFIG_REL.as_posix() in names:
detected.append("config")
if _SECRETS_REL.as_posix() in names:
detected.append("secrets")
if _WIFI_REL.as_posix() in names:
detected.append("wifi")
if any(n.startswith(_FONTS_REL.as_posix() + "/") for n in names):
detected.append("fonts")
if any(
n.startswith(_PLUGIN_UPLOADS_REL.as_posix() + "/") and "/uploads/" in n
for n in names
):
detected.append("plugin_uploads")
plugins: List[Dict[str, Any]] = []
if PLUGINS_MANIFEST_NAME in names:
try:
plugins = json.loads(zf.read(PLUGINS_MANIFEST_NAME).decode("utf-8"))
if not isinstance(plugins, list):
plugins = []
else:
detected.append("plugins")
except (OSError, UnicodeDecodeError, json.JSONDecodeError):
plugins = []
result_manifest = dict(manifest)
result_manifest["detected_contents"] = detected
result_manifest["plugins"] = plugins
result_manifest["total_uncompressed"] = total
result_manifest["file_count"] = len(names)
return True, "", result_manifest
except zipfile.BadZipFile:
return False, "File is not a valid ZIP archive", {}
except OSError as e:
return False, f"Could not read backup: {e}", {}
# ---------------------------------------------------------------------------
# Restore
# ---------------------------------------------------------------------------
def _extract_zip_safe(zip_path: Path, dest_dir: Path) -> None:
"""Extract ``zip_path`` into ``dest_dir`` rejecting any unsafe members."""
with zipfile.ZipFile(zip_path, "r") as zf:
for info in zf.infolist():
target = _safe_extract_path(dest_dir, info.filename)
if target is None:
raise ValueError(f"Unsafe path in backup: {info.filename}")
if info.is_dir():
target.mkdir(parents=True, exist_ok=True)
continue
target.parent.mkdir(parents=True, exist_ok=True)
with zf.open(info, "r") as src, open(target, "wb") as dst:
shutil.copyfileobj(src, dst, length=64 * 1024)
def _copy_file(src: Path, dst: Path) -> None:
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)
def restore_backup(
zip_path: Path,
project_root: Path,
options: Optional[RestoreOptions] = None,
) -> RestoreResult:
"""
Restore ``zip_path`` into ``project_root`` according to ``options``.
Plugin reinstalls are NOT performed here — the caller is responsible for
walking ``result.plugins_to_install`` and calling the store manager. This
keeps this module Flask-free and side-effect free beyond the filesystem.
"""
if options is None:
options = RestoreOptions()
project_root = Path(project_root).resolve()
result = RestoreResult()
ok, err, manifest = validate_backup(zip_path)
if not ok:
result.errors.append(err)
return result
result.manifest = manifest
with tempfile.TemporaryDirectory(prefix="ledmatrix_restore_") as tmp:
tmp_dir = Path(tmp)
try:
_extract_zip_safe(Path(zip_path), tmp_dir)
except (ValueError, zipfile.BadZipFile, OSError) as e:
result.errors.append(f"Failed to extract backup: {e}")
return result
# Main config.
if options.restore_config and (tmp_dir / _CONFIG_REL).exists():
try:
_copy_file(tmp_dir / _CONFIG_REL, project_root / _CONFIG_REL)
result.restored.append("config")
except OSError as e:
result.errors.append(f"Failed to restore config.json: {e}")
elif (tmp_dir / _CONFIG_REL).exists():
result.skipped.append("config")
# Secrets.
if options.restore_secrets and (tmp_dir / _SECRETS_REL).exists():
try:
_copy_file(tmp_dir / _SECRETS_REL, project_root / _SECRETS_REL)
result.restored.append("secrets")
except OSError as e:
result.errors.append(f"Failed to restore config_secrets.json: {e}")
elif (tmp_dir / _SECRETS_REL).exists():
result.skipped.append("secrets")
# WiFi.
if options.restore_wifi and (tmp_dir / _WIFI_REL).exists():
try:
_copy_file(tmp_dir / _WIFI_REL, project_root / _WIFI_REL)
result.restored.append("wifi")
except OSError as e:
result.errors.append(f"Failed to restore wifi_config.json: {e}")
elif (tmp_dir / _WIFI_REL).exists():
result.skipped.append("wifi")
# User fonts — skip anything that collides with a bundled font.
tmp_fonts = tmp_dir / _FONTS_REL
if options.restore_fonts and tmp_fonts.exists():
restored_count = 0
for font in sorted(tmp_fonts.iterdir()):
if not font.is_file():
continue
if font.name in BUNDLED_FONTS:
result.skipped.append(f"font:{font.name} (bundled)")
continue
try:
_copy_file(font, project_root / _FONTS_REL / font.name)
restored_count += 1
except OSError as e:
result.errors.append(f"Failed to restore font {font.name}: {e}")
if restored_count:
result.restored.append(f"fonts ({restored_count})")
elif tmp_fonts.exists():
result.skipped.append("fonts")
# Plugin uploads.
tmp_uploads = tmp_dir / _PLUGIN_UPLOADS_REL
if options.restore_plugin_uploads and tmp_uploads.exists():
count = 0
for root, _dirs, files in os.walk(tmp_uploads):
for name in files:
src = Path(root) / name
rel = src.relative_to(tmp_dir)
if "/uploads/" not in rel.as_posix():
result.errors.append(f"Rejected unexpected plugin path: {rel}")
continue
try:
_copy_file(src, project_root / rel)
count += 1
except OSError as e:
result.errors.append(f"Failed to restore {rel}: {e}")
if count:
result.restored.append(f"plugin_uploads ({count})")
elif tmp_uploads.exists():
result.skipped.append("plugin_uploads")
# Plugins list (for caller to reinstall).
if options.reinstall_plugins and (tmp_dir / PLUGINS_MANIFEST_NAME).exists():
try:
with (tmp_dir / PLUGINS_MANIFEST_NAME).open("r", encoding="utf-8") as f:
plugins = json.load(f)
if isinstance(plugins, list):
result.plugins_to_install = [
{"plugin_id": p.get("plugin_id"), "version": p.get("version", "")}
for p in plugins
if isinstance(p, dict) and p.get("plugin_id")
]
except (OSError, json.JSONDecodeError) as e:
result.errors.append(f"Could not read plugins.json: {e}")
result.success = not result.errors
return result

View File

@@ -7,7 +7,7 @@ fields and data structures.
""" """
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from typing import Dict, Any, Optional, List from typing import Dict, Optional
import logging import logging
from datetime import datetime from datetime import datetime
import pytz import pytz
@@ -21,12 +21,10 @@ class APIDataExtractor(ABC):
@abstractmethod @abstractmethod
def extract_game_details(self, game_event: Dict) -> Optional[Dict]: def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract common game details from raw API data.""" """Extract common game details from raw API data."""
pass
@abstractmethod @abstractmethod
def get_sport_specific_fields(self, game_event: Dict) -> Dict: def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract sport-specific fields (downs, innings, periods, etc.).""" """Extract sport-specific fields (downs, innings, periods, etc.)."""
pass
def _extract_common_details(self, game_event: Dict) -> tuple[Dict | None, Dict | None, Dict | None, Dict | None, Dict | None]: def _extract_common_details(self, game_event: Dict) -> tuple[Dict | None, Dict | None, Dict | None, Dict | None, Dict | None]:
"""Extract common game details that work across all sports.""" """Extract common game details that work across all sports."""

View File

@@ -329,7 +329,6 @@ class Baseball(SportsCore):
return return
series_summary = game.get("series_summary", "") series_summary = game.get("series_summary", "")
font = self.fonts.get('detail', ImageFont.load_default())
bbox = draw_overlay.textbbox((0, 0), series_summary, font=self.fonts['time']) bbox = draw_overlay.textbbox((0, 0), series_summary, font=self.fonts['time'])
height = bbox[3] - bbox[1] height = bbox[3] - bbox[1]
shots_y = (self.display_height - height) // 2 shots_y = (self.display_height - height) // 2

View File

@@ -1,6 +1,4 @@
import logging import logging
import time
from datetime import datetime, timezone
from typing import Any, Dict, Optional from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont

View File

@@ -6,11 +6,10 @@ to support different APIs and data providers.
""" """
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from typing import Dict, Any, Optional, List from typing import Dict, List
import requests import requests
import logging import logging
from datetime import datetime, timedelta from datetime import datetime
import time
class DataSource(ABC): class DataSource(ABC):
"""Abstract base class for data sources.""" """Abstract base class for data sources."""
@@ -35,17 +34,14 @@ class DataSource(ABC):
@abstractmethod @abstractmethod
def fetch_live_games(self, sport: str, league: str) -> List[Dict]: def fetch_live_games(self, sport: str, league: str) -> List[Dict]:
"""Fetch live games for a sport/league.""" """Fetch live games for a sport/league."""
pass
@abstractmethod @abstractmethod
def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]: def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]:
"""Fetch schedule for a sport/league within date range.""" """Fetch schedule for a sport/league within date range."""
pass
@abstractmethod @abstractmethod
def fetch_standings(self, sport: str, league: str) -> Dict: def fetch_standings(self, sport: str, league: str) -> Dict:
"""Fetch standings for a sport/league.""" """Fetch standings for a sport/league."""
pass
def get_headers(self) -> Dict[str, str]: def get_headers(self) -> Dict[str, str]:
"""Get headers for API requests.""" """Get headers for API requests."""
@@ -217,7 +213,7 @@ class MLBAPIDataSource(DataSource):
response.raise_for_status() response.raise_for_status()
data = response.json() data = response.json()
self.logger.debug(f"Fetched standings from MLB API") self.logger.debug("Fetched standings from MLB API")
return data return data
except Exception as e: except Exception as e:
@@ -296,7 +292,7 @@ class SoccerAPIDataSource(DataSource):
response.raise_for_status() response.raise_for_status()
data = response.json() data = response.json()
self.logger.debug(f"Fetched standings from soccer API") self.logger.debug("Fetched standings from soccer API")
return data return data
except Exception as e: except Exception as e:

View File

@@ -1,10 +1,8 @@
from typing import Dict, Any, Optional, List from typing import Dict, Any, Optional
from src.display_manager import DisplayManager from src.display_manager import DisplayManager
from src.cache_manager import CacheManager from src.cache_manager import CacheManager
from datetime import datetime, timezone, timedelta
import logging import logging
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
import time
from src.base_classes.data_sources import ESPNDataSource from src.base_classes.data_sources import ESPNDataSource
from src.base_classes.sports import SportsCore, SportsLive from src.base_classes.sports import SportsCore, SportsLive

View File

@@ -1,6 +1,4 @@
import logging import logging
import time
from datetime import datetime, timezone
from typing import Any, Dict, Optional from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
@@ -79,8 +77,6 @@ class Hockey(SportsCore):
away_shots = round(home_team_saves / home_team_saves_per) away_shots = round(home_team_saves / home_team_saves_per)
if away_team_saves_per > 0: if away_team_saves_per > 0:
home_shots = round(away_team_saves / away_team_saves_per) home_shots = round(away_team_saves / away_team_saves_per)
status_short = status["type"].get("shortDetail", "")
if situation and status["type"]["state"] == "in": if situation and status["type"]["state"] == "in":
# Detect scoring events from status detail # Detect scoring events from status detail
# status_detail = status["type"].get("detail", "") # status_detail = status["type"].get("detail", "")

View File

@@ -5,7 +5,7 @@ import time
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, List, Optional from typing import Any, Dict, List, Optional
import pytz import pytz
import requests import requests
@@ -172,8 +172,8 @@ class SportsCore(ABC):
try: try:
fallbacks.append(Path.home() / ".ledmatrix" / "logos" / self.sport_key) fallbacks.append(Path.home() / ".ledmatrix" / "logos" / self.sport_key)
except Exception: except RuntimeError as e:
pass self.logger.debug("Could not resolve home directory (expected for service users): %s", e)
fallbacks.append(Path(tempfile.gettempdir()) / "ledmatrix_logos" / self.sport_key) fallbacks.append(Path(tempfile.gettempdir()) / "ledmatrix_logos" / self.sport_key)
@@ -416,7 +416,6 @@ class SportsCore(ABC):
league=self.league, league=self.league,
event_id=game['id'], event_id=game['id'],
update_interval_seconds=update_interval, update_interval_seconds=update_interval,
is_live=is_live
) )
if odds_data: if odds_data:

View File

@@ -11,13 +11,10 @@ Follows LEDMatrix configuration management patterns:
- Maintainable: Changes to odds logic affect all plugins - Maintainable: Changes to odds logic affect all plugins
""" """
import time
import logging import logging
import requests import requests
import json import json
from datetime import datetime, timedelta, timezone
from typing import Dict, Any, Optional, List from typing import Dict, Any, Optional, List
import pytz
class BaseOddsManager: class BaseOddsManager:

View File

@@ -193,19 +193,21 @@ class CacheStrategy:
Data type string for strategy lookup Data type string for strategy lookup
""" """
key_lower = key.lower() key_lower = key.lower()
# Odds data — checked FIRST because odds keys may also contain 'live'/'current'
# (e.g. odds_espn_nba_game_123_live). The odds TTL (120s for live, 1800s for
# upcoming) must win over the generic sports_live TTL (30s) to avoid hitting
# the ESPN odds API every 30 seconds per game.
if 'odds' in key_lower:
# For live games, use shorter cache; for upcoming games, use longer cache
if any(x in key_lower for x in ['live', 'current']):
return 'odds_live' # Live odds change more frequently (120s TTL)
return 'odds' # Regular odds for upcoming games (1800s TTL)
# Live sports data (only reached if key does NOT contain 'odds') # Odds data — checked before the generic 'live' block below because
# live-odds cache keys (e.g. odds_espn_basketball_nba_<id>_live) contain
# both 'odds' AND 'live'. Without this ordering the 'live' check below
# would match first and return 'sports_live' (30 s TTL) instead of the
# correct 'odds_live' (120 s TTL).
if 'odds' in key_lower:
if any(x in key_lower for x in ['live', 'current']):
return 'odds_live' # Live odds change more frequently
return 'odds' # Regular odds for upcoming games
# Live sports data
if any(x in key_lower for x in ['live', 'current', 'scoreboard']): if any(x in key_lower for x in ['live', 'current', 'scoreboard']):
if 'soccer' in key_lower:
return 'sports_live' # Soccer live data is very time-sensitive
return 'sports_live' return 'sports_live'
# Weather data # Weather data

View File

@@ -13,7 +13,6 @@ import threading
from typing import Dict, Any, Optional, Protocol from typing import Dict, Any, Optional, Protocol
from datetime import datetime from datetime import datetime
from src.exceptions import CacheError
class CacheStrategyProtocol(Protocol): class CacheStrategyProtocol(Protocol):
@@ -184,7 +183,7 @@ class DiskCache:
os.replace(tmp_path, cache_path) os.replace(tmp_path, cache_path)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files # Set proper permissions: 660 (rw-rw----) for group-readable cache files
try: try:
os.chmod(cache_path, 0o660) os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
except OSError: except OSError:
pass # Non-critical if chmod fails pass # Non-critical if chmod fails
finally: finally:
@@ -202,7 +201,7 @@ class DiskCache:
os.fsync(cache_file.fileno()) os.fsync(cache_file.fileno())
# Set proper permissions: 660 (rw-rw----) for group-readable cache files # Set proper permissions: 660 (rw-rw----) for group-readable cache files
try: try:
os.chmod(cache_path, 0o660) os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
except OSError: except OSError:
pass # Non-critical if chmod fails pass # Non-critical if chmod fails
self.logger.debug("Wrote cache for %s directly (non-atomic)", key) self.logger.debug("Wrote cache for %s directly (non-atomic)", key)
@@ -210,7 +209,7 @@ class DiskCache:
# If direct write also fails, try fallback location # If direct write also fails, try fallback location
self.logger.warning("Direct write failed for key '%s' to %s: %s", key, cache_path, write_error) self.logger.warning("Direct write failed for key '%s' to %s: %s", key, cache_path, write_error)
raise # Re-raise to trigger fallback logic raise # Re-raise to trigger fallback logic
except (IOError, OSError, PermissionError) as e: except (IOError, OSError, PermissionError):
# Attempt one-time fallback write to user's home cache directory # Attempt one-time fallback write to user's home cache directory
try: try:
# Try user's home cache directory as fallback # Try user's home cache directory as fallback
@@ -228,7 +227,7 @@ class DiskCache:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder) json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files # Set proper permissions: 660 (rw-rw----) for group-readable cache files
try: try:
os.chmod(fallback_path, 0o660) os.chmod(fallback_path, 0o660) # nosec B103 - intentional; web UI and service share a group
except OSError: except OSError:
pass # Non-critical if chmod fails pass # Non-critical if chmod fails
self.logger.debug("Cache wrote to fallback location: %s", fallback_path) self.logger.debug("Cache wrote to fallback location: %s", fallback_path)

View File

@@ -7,7 +7,6 @@ from typing import Any, Dict, List, Optional
import logging import logging
import threading import threading
import tempfile import tempfile
from pathlib import Path
from src.exceptions import CacheError from src.exceptions import CacheError
from src.cache.memory_cache import MemoryCache from src.cache.memory_cache import MemoryCache
from src.cache.disk_cache import DiskCache from src.cache.disk_cache import DiskCache
@@ -111,7 +110,7 @@ class CacheManager:
if os.access(system_cache_dir, os.W_OK): if os.access(system_cache_dir, os.W_OK):
self.logger.info(f"Using system cache directory: {system_cache_dir}") self.logger.info(f"Using system cache directory: {system_cache_dir}")
return system_cache_dir return system_cache_dir
except (OSError, IOError, PermissionError) as perm_error: except (OSError, IOError, PermissionError):
# Permission errors are expected when running as non-root # Permission errors are expected when running as non-root
self.logger.debug(f"Could not create system cache directory (permission denied): {system_cache_dir}") self.logger.debug(f"Could not create system cache directory (permission denied): {system_cache_dir}")
except (OSError, IOError, PermissionError) as e: except (OSError, IOError, PermissionError) as e:

View File

@@ -5,13 +5,10 @@ Handles HTTP requests, caching, and ESPN API integration for LED matrix plugins.
Extracted from LEDMatrix core to provide reusable functionality for plugins. Extracted from LEDMatrix core to provide reusable functionality for plugins.
""" """
import json
import logging import logging
import time import time
from datetime import datetime, timedelta from datetime import datetime
from pathlib import Path from typing import Any, Dict, Optional
from typing import Any, Dict, List, Optional, Union
from urllib.parse import urlencode
import requests import requests
from requests.adapters import HTTPAdapter from requests.adapters import HTTPAdapter

View File

@@ -5,11 +5,9 @@ This example shows how to refactor the basketball plugin to use the
ledmatrix-common package for cleaner, more maintainable code. ledmatrix-common package for cleaner, more maintainable code.
""" """
import logging
from pathlib import Path from pathlib import Path
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional
from PIL import Image, ImageDraw
# Import common helpers # Import common helpers
from src.common import ( from src.common import (

View File

@@ -42,7 +42,7 @@ def test_utilities(display_width: int, display_height: int):
print(f"Testing LEDMatrix Common utilities with {display_width}x{display_height} display") print(f"Testing LEDMatrix Common utilities with {display_width}x{display_height} display")
try: try:
from ledmatrix_common import LogoHelper, TextHelper, APIHelper, DisplayHelper, GameHelper, ConfigHelper from ledmatrix_common import LogoHelper, TextHelper, DisplayHelper, GameHelper, ConfigHelper
# Test LogoHelper # Test LogoHelper
print("Testing LogoHelper...") print("Testing LogoHelper...")
@@ -63,12 +63,12 @@ def test_utilities(display_width: int, display_height: int):
# Test GameHelper # Test GameHelper
print("Testing GameHelper...") print("Testing GameHelper...")
game_helper = GameHelper() GameHelper()
print("GameHelper initialized") print("GameHelper initialized")
# Test ConfigHelper # Test ConfigHelper
print("Testing ConfigHelper...") print("Testing ConfigHelper...")
config_helper = ConfigHelper() ConfigHelper()
print("ConfigHelper initialized") print("ConfigHelper initialized")
print("All tests passed!") print("All tests passed!")

View File

@@ -6,7 +6,7 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
""" """
import logging import logging
from typing import Any, Dict, List, Optional, Tuple, Union from typing import Any, Dict, Optional, Tuple
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
@@ -166,8 +166,7 @@ class DisplayHelper:
img = self.create_base_image(background_color) img = self.create_base_image(background_color)
draw = ImageDraw.Draw(img) draw = ImageDraw.Draw(img)
# Calculate text position (start off-screen to the right) # Start text off-screen to the right
text_width = draw.textlength(text, font=font)
x_position = self.display_width x_position = self.display_width
# Draw text # Draw text
@@ -216,8 +215,7 @@ class DisplayHelper:
PIL Image with error message PIL Image with error message
""" """
img = self.create_base_image((50, 0, 0)) # Dark red background img = self.create_base_image((50, 0, 0)) # Dark red background
draw = ImageDraw.Draw(img)
# Use default font # Use default font
font = ImageFont.load_default() font = ImageFont.load_default()
@@ -237,8 +235,6 @@ class DisplayHelper:
PIL Image with no data message PIL Image with no data message
""" """
img = self.create_base_image((0, 0, 0)) img = self.create_base_image((0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.load_default() font = ImageFont.load_default()
self._draw_centered_text(message, font, (0, 0, 0), (150, 150, 150)) self._draw_centered_text(message, font, (0, 0, 0), (150, 150, 150))

View File

@@ -6,10 +6,8 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
""" """
import logging import logging
import os
from pathlib import Path from pathlib import Path
from typing import Dict, List, Optional, Union from typing import Dict, List, Optional, Union
from urllib.parse import urlparse
import requests import requests
from PIL import Image from PIL import Image

View File

@@ -17,7 +17,7 @@ logger = logging.getLogger(__name__)
# System directories that should never have their permissions modified # System directories that should never have their permissions modified
# These directories have special system-level permissions that must be preserved # These directories have special system-level permissions that must be preserved
PROTECTED_SYSTEM_DIRECTORIES = { PROTECTED_SYSTEM_DIRECTORIES = { # nosec B108 - these are checked to PREVENT permission changes, not to use as temp paths
'/tmp', '/tmp',
'/var/tmp', '/var/tmp',
'/dev', '/dev',

651
src/common/sync_manager.py Normal file
View File

@@ -0,0 +1,651 @@
"""
Multi-Display Sync Manager
Synchronizes scrolling content across two LED matrix display units over UDP.
Runs at the core framework level — works with any plugin automatically.
Roles:
standalone No sync (default behavior)
leader Drives scroll, sends rendered follower frames via UDP
follower Receives frames from leader; falls back to own plugins when
the leader goes offline
Compatibility rule: rows and cols must match between leader and follower.
chain_length may differ — each display can have a different number of panels.
Port default: 5765 (UDP). Open this port on both Pis if ufw is active:
sudo ufw allow 5765/udp
"""
import io
import json
import os
import socket
import struct
import tempfile
import threading
import time
import logging
from enum import Enum
from typing import Callable, Optional
import numpy as np
from PIL import Image
# Raw-frame wire format: 8-byte magic + 4-byte header + raw RGB pixels
# Much faster than PNG: no encode/decode, negligible CPU, same UDP packet size
_RAW_MAGIC = b'SYNC_RAW'
_RAW_HEADER = struct.Struct('<HH') # width, height (uint16 LE)
SYNC_PORT = 5765
HELLO_INTERVAL = 5.0 # follower broadcasts hello every 5 s
HEARTBEAT_INTERVAL = 2.0 # follower sends heartbeat every 2 s
PEER_TIMEOUT = 6.0 # leader: no heartbeat → follower gone
LEADER_TIMEOUT = 6.0 # follower: no frame → leader gone
STATUS_FILE = os.path.join(tempfile.gettempdir(), "led_matrix_sync_status.json")
class SyncRole(Enum):
STANDALONE = "standalone"
LEADER = "leader"
FOLLOWER = "follower"
class LeaderState(Enum):
NO_PEER = "no_peer"
CONNECTED = "connected"
INCOMPATIBLE = "incompatible"
class FollowerState(Enum):
STANDALONE = "standalone"
FOLLOWER = "follower"
class DisplaySyncManager:
"""
Core sync manager. Instantiated by DisplayController based on config['sync'].
Leader sends compressed PNG frames to the follower after each render cycle.
Follower renders received frames; returns to own plugin stack when leader
goes offline.
"""
def __init__(
self,
role_str: str,
cfg: dict,
hw_config: dict,
logger: logging.Logger,
) -> None:
"""
Args:
role_str: "standalone" | "leader" | "follower"
cfg: config['sync'] dict
hw_config: config['display']['hardware'] dict (this Pi's own config)
logger: framework logger
"""
try:
self.role = SyncRole(role_str)
except ValueError:
logger.warning("Invalid sync role '%s', defaulting to standalone", role_str)
self.role = SyncRole.STANDALONE
self.logger = logger
self.port = int(cfg.get("port", SYNC_PORT))
self._hw_config = hw_config
# Leader state
self._leader_state = LeaderState.NO_PEER
self._peer_ip: Optional[str] = None
self._peer_compatible: bool = False
self._peer_chain: int = 0
self._last_heartbeat_time: float = 0.0
self._leader_width: int = 0 # set by display_controller after init
# Follower state
self._follower_state = FollowerState.STANDALONE
self._latest_frame: Optional[Image.Image] = None # pixel-frame fallback
self._latest_scroll_x: Optional[float] = None # Vegas scroll position
self._last_leader_frame_time: float = 0.0
self._frame_lock = threading.Lock()
self._leader_ip: Optional[str] = None
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
self._img_server_sock = None # TCP server for scroll image transfer
# Leader state additions
self._on_follower_connected: Optional[Callable[[], None]] = None # called when follower connects
self._error_message: Optional[str] = None
self._running = False
self._recv_sock: Optional[socket.socket] = None
self._send_sock: Optional[socket.socket] = None
if self.role == SyncRole.STANDALONE:
return
if self.role == SyncRole.LEADER:
self._start_leader()
elif self.role == SyncRole.FOLLOWER:
self._start_follower()
# ------------------------------------------------------------------ #
# Leader setup #
# ------------------------------------------------------------------ #
def _start_leader(self) -> None:
# Receive socket: listens for hello + heartbeat from follower
self._recv_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # nosec B104
self._recv_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._recv_sock.bind(("", self.port)) # nosec B104 — intentional: must receive UDP broadcast on all interfaces
self._recv_sock.settimeout(1.0)
# Send socket: unicast frames + hello_ack to follower
self._send_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._running = True
threading.Thread(
target=self._leader_recv_loop, daemon=True, name="sync-leader-recv"
).start()
threading.Thread(
target=self._leader_watchdog, daemon=True, name="sync-leader-watchdog"
).start()
self.logger.info("Sync: leader started on UDP port %d", self.port)
self.write_status_file()
def _leader_recv_loop(self) -> None:
while self._running:
try:
data, addr = self._recv_sock.recvfrom(1024)
sender_ip = addr[0]
try:
msg = json.loads(data.decode("utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError):
continue
t = msg.get("t")
if t == "hello":
self._handle_hello(msg, sender_ip)
elif t == "hb":
if self._peer_ip == sender_ip:
self._last_heartbeat_time = time.time()
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync leader recv error: %s", exc)
def _handle_hello(self, msg: dict, sender_ip: str) -> None:
hw = self._hw_config
local_rows = hw.get("rows", 32)
local_cols = hw.get("cols", 64)
peer_rows = int(msg.get("rows", 0))
peer_cols = int(msg.get("cols", 0))
peer_chain = int(msg.get("chain", 1))
compatible = peer_rows == local_rows and peer_cols == local_cols
self._peer_ip = sender_ip
self._peer_compatible = compatible
self._peer_chain = peer_chain
self._last_heartbeat_time = time.time()
prev_state = self._leader_state
if compatible:
if prev_state != LeaderState.CONNECTED:
self.logger.info(
"Sync: follower connected at %s (chain=%d)", sender_ip, peer_chain
)
self._leader_state = LeaderState.CONNECTED
self._error_message = None
# Send scroll image immediately on new connection so follower has identical content
if prev_state != LeaderState.CONNECTED and self._on_follower_connected:
threading.Thread(
target=self._on_follower_connected,
daemon=True, name="sync-leader-img-push"
).start()
else:
self._leader_state = LeaderState.INCOMPATIBLE
self._error_message = (
f"Incompatible panels: follower is {peer_cols}x{peer_rows}, "
f"leader is {local_cols}x{local_rows}. "
f"rows and cols must match between displays."
)
if prev_state != LeaderState.INCOMPATIBLE:
self.logger.error("Sync: %s", self._error_message)
if self._leader_state != prev_state:
self.write_status_file()
ack = json.dumps({
"t": "hello_ack",
"compatible": compatible,
"leader_width": self._leader_width,
"error": self._error_message,
}).encode("utf-8")
try:
self._send_sock.sendto(ack, (sender_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: hello_ack send failed: %s", exc)
def _leader_watchdog(self) -> None:
while self._running:
time.sleep(1.0)
if self._leader_state == LeaderState.CONNECTED:
if time.time() - self._last_heartbeat_time > PEER_TIMEOUT:
self.logger.info(
"Sync: follower heartbeat timeout — peer disconnected"
)
self._leader_state = LeaderState.NO_PEER
self._peer_ip = None
self._peer_compatible = False
self.write_status_file()
def _image_server_loop(self) -> None:
"""Follower: TCP server that receives the leader's scroll image at each new cycle."""
while self._running:
try:
conn, addr = self._img_server_sock.accept()
conn.settimeout(10.0)
try:
# 4-byte big-endian length prefix
hdr = b""
while len(hdr) < 4:
chunk = conn.recv(4 - len(hdr))
if not chunk:
break
hdr += chunk
if len(hdr) < 4:
continue
length = int.from_bytes(hdr, "big")
_MAX_IMAGE_BYTES = 10 * 1024 * 1024 # 10 MB — well above any real scroll image
if length <= 0 or length > _MAX_IMAGE_BYTES:
self.logger.warning(
"Sync: rejected TCP image with invalid length %d (max %d) from %s",
length, _MAX_IMAGE_BYTES, addr,
)
conn.close()
continue
data = bytearray()
while len(data) < length:
chunk = conn.recv(min(65536, length - len(data)))
if not chunk:
break
data.extend(chunk)
img = Image.open(io.BytesIO(data))
_MAX_W, _MAX_H = 100_000, 256 # generous for any real scroll image
if img.width > _MAX_W or img.height > _MAX_H:
self.logger.warning(
"Sync: rejected oversized scroll image %dx%d (max %dx%d) from %s",
img.width, img.height, _MAX_W, _MAX_H, addr,
)
continue
try:
img.load()
except (Image.DecompressionBombError, ValueError) as exc:
self.logger.warning("Sync: rejected decompression bomb from %s: %s", addr, exc)
continue
self.logger.info(
"Sync: received scroll image %dx%d (%d bytes compressed)",
img.width, img.height, length,
)
with self._scroll_image_lock:
if self._on_scroll_image:
cb = self._on_scroll_image
else:
# Callback not registered yet (startup race) — cache it
self._pending_scroll_image = img
cb = None
if cb:
cb(img)
finally:
conn.close()
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync: image server error: %s", exc)
def send_scroll_image(self, image: Image.Image) -> None:
"""Leader: send the full scroll image to the follower via TCP.
PNG compression typically reduces a 5000×32 image to ~2050KB,
transferring in <20ms on local WiFi. Called at new_cycle and on
first connection so both Pis always have identical cached_arrays.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
buf = io.BytesIO()
image.save(buf, format="PNG", optimize=True)
data = buf.getvalue()
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.settimeout(5.0)
sock.connect((self._peer_ip, self.port + 1))
sock.sendall(len(data).to_bytes(4, "big") + data)
self.logger.info(
"Sync: sent scroll image %dx%d (%d bytes compressed)",
image.width, image.height, len(data),
)
except Exception as exc:
self.logger.debug("Sync: image send error: %s", exc)
def set_on_follower_connected(self, callback: Callable[[], None]) -> None:
"""Leader: callback fired (in a thread) when a compatible follower first connects.
Use this to push the current scroll image immediately.
If a follower is already connected when this is called, fires right away
(handles the race where follower connects during leader startup).
"""
self._on_follower_connected = callback
if self._leader_state == LeaderState.CONNECTED:
threading.Thread(
target=callback, daemon=True, name="sync-leader-img-push-late"
).start()
def set_on_scroll_image(self, callback: Callable[[Image.Image], None]) -> None:
"""Follower: callback fired with the received Image when leader sends scroll image.
If an image was received before this callback was registered (startup race),
fires immediately with that cached image.
"""
with self._scroll_image_lock:
self._on_scroll_image = callback
pending = self._pending_scroll_image
self._pending_scroll_image = None
if pending is not None:
callback(pending)
def send_scroll_x(self, scroll_x: float) -> None:
"""Leader (Vegas mode): broadcast scroll position instead of a pixel frame.
The follower renders from its own local pipeline at scroll_x - display_width.
~20 bytes vs ~18KB for raw frames — eliminates all content-change artifacts.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
msg = json.dumps({"t": "sx", "x": round(scroll_x, 2)}).encode("utf-8")
self._send_sock.sendto(msg, (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: scroll_x send error: %s", exc)
def send_new_cycle(self) -> None:
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: new_cycle send error: %s", exc)
def send_frame(self, image: Image.Image) -> None:
"""Leader: send a rendered frame to the follower as raw RGB bytes.
Raw format is orders of magnitude faster than PNG on Pi hardware —
no encode on sender, no decode on receiver.
Packet: 8-byte magic + 4-byte (width, height) header + raw RGB bytes.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
arr = np.asarray(image.convert("RGB"), dtype=np.uint8)
header = _RAW_MAGIC + _RAW_HEADER.pack(image.width, image.height)
data = header + arr.tobytes()
if len(data) <= 65000:
self._send_sock.sendto(data, (self._peer_ip, self.port))
elif not getattr(self, '_oversized_frame_warned', False):
self._oversized_frame_warned = True
self.logger.warning(
"Sync: frame too large for UDP (%d bytes, max 65000) — "
"image %dx%d will not be sent; use TCP image sync instead",
len(data), image.width, image.height,
)
except Exception as exc:
self.logger.debug("Sync: frame send error: %s", exc)
def set_leader_width(self, width: int) -> None:
"""Called by DisplayController once display_manager.width is known."""
self._leader_width = width
# ------------------------------------------------------------------ #
# Follower setup #
# ------------------------------------------------------------------ #
def _start_follower(self) -> None:
# Receive socket: listens for frames + hello_ack from leader
self._recv_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._recv_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._recv_sock.bind(("", self.port)) # nosec B104 — intentional: must receive UDP broadcast on all interfaces
self._recv_sock.settimeout(0.1)
# Send socket: broadcasts hello + heartbeat
self._send_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._send_sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
self._running = True
threading.Thread(
target=self._follower_recv_loop, daemon=True, name="sync-follower-recv"
).start()
threading.Thread(
target=self._follower_announce_loop, daemon=True, name="sync-follower-announce"
).start()
threading.Thread(
target=self._follower_watchdog, daemon=True, name="sync-follower-watchdog"
).start()
# TCP server: receives scroll images from leader (port + 1)
self._img_server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # nosec B104
self._img_server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._img_server_sock.bind(("", self.port + 1)) # nosec B104 — intentional: TCP server must accept connections on all interfaces
self._img_server_sock.listen(1)
self._img_server_sock.settimeout(1.0)
threading.Thread(
target=self._image_server_loop, daemon=True, name="sync-image-server"
).start()
self.logger.info(
"Sync: follower started on UDP port %d, image server on TCP %d",
self.port, self.port + 1,
)
self.write_status_file()
def _follower_recv_loop(self) -> None:
while self._running:
try:
data, addr = self._recv_sock.recvfrom(65535)
sender_ip = addr[0]
if data[:8] == _RAW_MAGIC or len(data) > 512:
# Frame data: prefer magic-tagged raw RGB; fall back to legacy PNG
try:
if data[:8] == _RAW_MAGIC:
w, h = _RAW_HEADER.unpack(data[8:12])
raw = data[12:]
img = Image.frombuffer(
"RGB", (w, h), raw, "raw", "RGB", 0, 1
)
else:
# Fallback: try legacy PNG
img = Image.open(io.BytesIO(data))
img.load()
with self._frame_lock:
self._latest_frame = img
self._last_leader_frame_time = time.time()
self._leader_ip = sender_ip
if self._follower_state == FollowerState.STANDALONE:
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
except Exception as exc:
self.logger.debug("Sync: frame decode error: %s", exc)
else:
# Control message
try:
msg = json.loads(data.decode("utf-8"))
t = msg.get("t")
if t == "hello_ack":
self._leader_ip = sender_ip
self._peer_compatible = msg.get("compatible", False)
self._error_message = msg.get("error")
if not self._peer_compatible and self._error_message:
self.logger.error(
"Sync: leader rejected handshake — %s",
self._error_message,
)
self.write_status_file()
elif t == "sx":
# Vegas scroll-position sync — tiny message, renders locally
self._latest_scroll_x = float(msg["x"])
self._last_leader_frame_time = time.time()
self._leader_ip = sender_ip
if self._follower_state == FollowerState.STANDALONE:
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
if self._on_new_cycle:
self._on_new_cycle() # build initial scroll image
elif t == "nc":
# Leader started a new scroll cycle — rebuild local image
if self._on_new_cycle:
self._on_new_cycle()
except (json.JSONDecodeError, UnicodeDecodeError, KeyError):
pass
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync follower recv error: %s", exc)
def _follower_announce_loop(self) -> None:
hw = self._hw_config
hello = json.dumps({
"t": "hello",
"rows": hw.get("rows", 32),
"cols": hw.get("cols", 64),
"chain": hw.get("chain_length", 1),
}).encode("utf-8")
heartbeat = json.dumps({"t": "hb"}).encode("utf-8")
dest = ("<broadcast>", self.port)
last_hello = 0.0
last_hb = 0.0
while self._running:
now = time.time()
if now - last_hello >= HELLO_INTERVAL:
try:
self._send_sock.sendto(hello, dest)
last_hello = now
except Exception as exc:
self.logger.debug("Sync: hello broadcast error: %s", exc)
if now - last_hb >= HEARTBEAT_INTERVAL:
try:
self._send_sock.sendto(heartbeat, dest)
last_hb = now
except Exception as exc:
self.logger.debug("Sync: heartbeat error: %s", exc)
time.sleep(0.5)
def _follower_watchdog(self) -> None:
while self._running:
time.sleep(1.0)
if self._follower_state == FollowerState.FOLLOWER:
if time.time() - self._last_leader_frame_time > LEADER_TIMEOUT:
self.logger.info(
"Sync: leader frame timeout — returning to standalone mode"
)
self._follower_state = FollowerState.STANDALONE
with self._frame_lock:
self._latest_frame = None
self.write_status_file()
# ------------------------------------------------------------------ #
# Public API #
# ------------------------------------------------------------------ #
def is_follower_active(self) -> bool:
"""True when this Pi is in active follower mode (receiving frames)."""
return (
self.role == SyncRole.FOLLOWER
and self._follower_state == FollowerState.FOLLOWER
)
def get_latest_scroll_x(self) -> Optional[float]:
"""Follower: return the most recently received Vegas scroll position, or None."""
return self._latest_scroll_x
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
"""Follower: register a callback fired when the leader starts a new scroll cycle.
Used to trigger a local start_new_cycle() so both Pis rebuild from same fresh data.
"""
self._on_new_cycle = callback
def get_latest_frame(self) -> Optional[Image.Image]:
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
with self._frame_lock:
return self._latest_frame
def get_status(self) -> dict:
"""Return sync state dict for the web API status endpoint."""
hw = self._hw_config
base = {
"role": self.role.value,
"port": self.port,
"local_rows": hw.get("rows", 32),
"local_cols": hw.get("cols", 64),
"local_chain": hw.get("chain_length", 1),
}
if self.role == SyncRole.STANDALONE:
return {**base, "state": "standalone"}
if self.role == SyncRole.LEADER:
return {
**base,
"state": self._leader_state.value,
"peer_ip": self._peer_ip,
"peer_compatible": self._peer_compatible,
"peer_chain": self._peer_chain,
"leader_width": self._leader_width,
"error": self._error_message,
}
# Follower
return {
**base,
"state": self._follower_state.value,
"leader_ip": self._leader_ip,
"peer_compatible": self._peer_compatible,
"error": self._error_message,
}
def write_status_file(self) -> None:
"""Write current sync status to STATUS_FILE for the web UI to read."""
try:
status = self.get_status()
status["ts"] = time.time()
tmp = STATUS_FILE + ".tmp"
with open(tmp, "w") as f:
json.dump(status, f)
os.replace(tmp, STATUS_FILE)
except Exception as exc:
self.logger.debug("Sync: status file write error: %s", exc)
def stop(self) -> None:
"""Shut down threads and close sockets."""
self._running = False
for sock in (self._recv_sock, self._send_sock, self._img_server_sock):
if sock:
try:
sock.close()
except Exception as exc:
self.logger.debug("Sync: error closing socket: %s", exc)

View File

@@ -6,7 +6,6 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
""" """
import logging import logging
import os
from pathlib import Path from pathlib import Path
from typing import Dict, List, Optional, Tuple, Union from typing import Dict, List, Optional, Tuple, Union

View File

@@ -8,7 +8,7 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
import logging import logging
import re import re
from datetime import datetime, timezone from datetime import datetime, timezone
from typing import Optional, Tuple, Union from typing import Union
import pytz import pytz

View File

@@ -313,17 +313,8 @@ class ConfigManager:
self._merge_template_defaults(self.config, template_config) self._merge_template_defaults(self.config, template_config)
# Save migrated config using atomic save to preserve permissions # Save migrated config using atomic save to preserve permissions
# Load secrets if they exist to pass to atomic save
secrets_content = {}
if os.path.exists(self.secrets_path):
try:
with open(self.secrets_path, 'r') as f_secrets:
secrets_content = json.load(f_secrets)
except Exception:
pass # Continue without secrets if can't load
# Use atomic save to preserve file permissions # Use atomic save to preserve file permissions
# Note: save_config_atomic handles secrets internally, no need to pass new_secrets # Note: save_config_atomic handles secrets internally
result = self.save_config_atomic( result = self.save_config_atomic(
new_config_data=self.config, new_config_data=self.config,
create_backup=False, # Already created backup above create_backup=False, # Already created backup above

View File

@@ -12,11 +12,10 @@ This service wraps ConfigManager and adds:
""" """
import json import json
import os
import time import time
import threading import threading
from pathlib import Path from pathlib import Path
from typing import Dict, Any, Optional, List, Callable, Set from typing import Dict, Any, Optional, List, Callable
from datetime import datetime from datetime import datetime
from collections import defaultdict from collections import defaultdict
import logging import logging
@@ -38,7 +37,7 @@ class ConfigVersion:
config: Configuration dictionary config: Configuration dictionary
version: Version number version: Version number
timestamp: When this version was created timestamp: When this version was created
checksum: MD5 checksum of the config checksum: SHA-256 hex digest of the config (for change detection)
""" """
self.config: Dict[str, Any] = config self.config: Dict[str, Any] = config
self.version: int = version self.version: int = version
@@ -114,9 +113,9 @@ class ConfigService:
self._start_file_watching() self._start_file_watching()
def _calculate_checksum(self, config: Dict[str, Any]) -> str: def _calculate_checksum(self, config: Dict[str, Any]) -> str:
"""Calculate MD5 checksum of configuration.""" """Calculate checksum of configuration for change detection."""
config_str = json.dumps(config, sort_keys=True) config_str = json.dumps(config, sort_keys=True)
return hashlib.md5(config_str.encode()).hexdigest() return hashlib.sha256(config_str.encode()).hexdigest()
def _load_config(self) -> bool: def _load_config(self) -> bool:
""" """

View File

@@ -1,6 +1,4 @@
import time import time
import logging
import sys
import os import os
import json import json
from pathlib import Path from pathlib import Path
@@ -16,6 +14,7 @@ from src.config_service import ConfigService
from src.cache_manager import CacheManager from src.cache_manager import CacheManager
from src.font_manager import FontManager from src.font_manager import FontManager
from src.logging_config import get_logger from src.logging_config import get_logger
from src.common.sync_manager import DisplaySyncManager, SyncRole
# Get logger with consistent configuration # Get logger with consistent configuration
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -32,10 +31,7 @@ class DisplayController:
def __init__(self): def __init__(self):
start_time = time.time() start_time = time.time()
logger.info("Starting DisplayController initialization") logger.info("Starting DisplayController initialization")
# Throttle tracking for _tick_plugin_updates in high-FPS loops
self._last_plugin_tick_time = 0.0
# Initialize ConfigManager and wrap with ConfigService for hot-reload # Initialize ConfigManager and wrap with ConfigService for hot-reload
config_manager = ConfigManager() config_manager = ConfigManager()
enable_hot_reload = os.environ.get('LEDMATRIX_HOT_RELOAD', 'true').lower() == 'true' enable_hot_reload = os.environ.get('LEDMATRIX_HOT_RELOAD', 'true').lower() == 'true'
@@ -69,7 +65,39 @@ class DisplayController:
config_time = time.time() config_time = time.time()
self.display_manager = DisplayManager(self.config) self.display_manager = DisplayManager(self.config)
logger.info("DisplayManager initialized in %.3f seconds", time.time() - config_time) logger.info("DisplayManager initialized in %.3f seconds", time.time() - config_time)
# Initialize multi-display sync (standalone by default — no-op unless configured)
sync_cfg = self.config.get("sync", {})
hw_cfg = self.config.get("display", {}).get("hardware", {})
self.sync_manager = DisplaySyncManager(
role_str=sync_cfg.get("role", "standalone"),
cfg=sync_cfg,
hw_config=hw_cfg,
logger=logger,
)
# Tell the leader its own physical display width so it can include it in hello_ack
if self.sync_manager.role == SyncRole.LEADER:
self.sync_manager.set_leader_width(self.display_manager.width)
# Follower mode setup
if self.sync_manager.role == SyncRole.FOLLOWER:
# Gate update_display() so background plugin threads cannot write to
# hardware — only our render loop is permitted.
_real_update = self.display_manager.update_display
_dm = self.display_manager
def _follower_gated_update():
# Allow through when the sync render loop has the token, or when
# the leader has gone offline and we've fallen back to standalone.
if getattr(_dm, '_sync_render_allowed', False) or not self.sync_manager.is_follower_active():
_real_update()
self.display_manager.update_display = _follower_gated_update
# Note: _on_new_cycle is NOT registered here. The leader now sends
# its actual scroll image via TCP at each new_cycle, so the follower
# adopts that image directly via set_on_scroll_image(). Registering
# _on_new_cycle would trigger a local rebuild that overwrites the
# leader's just-received image with a different locally-built one.
# Initialize Font Manager # Initialize Font Manager
font_time = time.time() font_time = time.time()
self.font_manager = FontManager(self.config) self.font_manager = FontManager(self.config)
@@ -82,8 +110,7 @@ class DisplayController:
logger.info("Display modes initialized in %.3f seconds", time.time() - init_time) logger.info("Display modes initialized in %.3f seconds", time.time() - init_time)
self.force_change = False self.force_change = False
self._next_live_priority_check = 0.0 # monotonic timestamp for throttled live priority checks
# All sports and content managers now handled via plugins # All sports and content managers now handled via plugins
logger.info("All sports and content managers now handled via plugin system") logger.info("All sports and content managers now handled via plugin system")
@@ -396,20 +423,64 @@ class DisplayController:
# Set up live priority checker # Set up live priority checker
self.vegas_coordinator.set_live_priority_checker(self._check_live_priority) self.vegas_coordinator.set_live_priority_checker(self._check_live_priority)
# Set up interrupt checker for on-demand/wifi status # Set up interrupt checker for on-demand/wifi status and follower mode
def _vegas_interrupt():
return self._check_vegas_interrupt() or self.sync_manager.is_follower_active()
self.vegas_coordinator.set_interrupt_checker( self.vegas_coordinator.set_interrupt_checker(
self._check_vegas_interrupt, _vegas_interrupt,
check_interval=10 # Check every 10 frames (~80ms at 125 FPS) check_interval=10 # Check every 10 frames (~80ms at 125 FPS)
) )
# Set up plugin update tick to keep data fresh during Vegas mode # Run plugin updates inside the Vegas loop so the inter-iteration
self.vegas_coordinator.set_update_tick( # gap is <1 ms (nothing left for _tick_plugin_updates() to do).
self._tick_plugin_updates_for_vegas, self.vegas_coordinator.set_update_callback(self._tick_plugin_updates)
interval=1.0
) # Wire multi-display sync into Vegas render pipeline
follower_pos = self.config.get("sync", {}).get("follower_position", "left")
self.vegas_coordinator.set_sync_manager(self.sync_manager, follower_pos)
logger.info("Vegas mode coordinator initialized") logger.info("Vegas mode coordinator initialized")
# Follower does NOT build its own initial scroll image — the leader
# pushes its image via TCP as soon as set_on_follower_connected fires.
# A local build would create a different (wrong) image that could
# temporarily replace the leader's correct one.
# When the leader sends its scroll image (TCP), update our
# cached_array so both Pis have pixel-identical images.
import numpy as _np
def _on_leader_scroll_image(image):
vc = getattr(self, 'vegas_coordinator', None)
if vc and vc.render_pipeline:
rp = vc.render_pipeline
arr = _np.asarray(image.convert("RGB"), dtype=_np.uint8)
rp.scroll_helper.cached_image = image
rp.scroll_helper.cached_array = arr
rp.scroll_helper.total_scroll_width = image.width
self._follower_pending_new_image = False
logger.info(
"Sync: follower adopted leader scroll image %dx%d",
image.width, image.height,
)
self.sync_manager.set_on_scroll_image(_on_leader_scroll_image)
if self.sync_manager.role == SyncRole.LEADER:
# When a follower first connects, push the current scroll image so
# the follower doesn't have to wait for the next new_cycle event.
# Polls until the image is ready (Vegas may still be composing on startup).
def _on_follower_connected():
import time as _t
for _ in range(300): # up to 30s
vc = getattr(self, 'vegas_coordinator', None)
if vc and vc.render_pipeline:
img = vc.render_pipeline.scroll_helper.cached_image
if img is not None:
self.sync_manager.send_scroll_image(img)
return
_t.sleep(0.1)
logger.warning("Sync: no scroll image available to push to new follower")
self.sync_manager.set_on_follower_connected(_on_follower_connected)
except Exception as e: except Exception as e:
logger.error("Failed to initialize Vegas mode: %s", e, exc_info=True) logger.error("Failed to initialize Vegas mode: %s", e, exc_info=True)
self.vegas_coordinator = None self.vegas_coordinator = None
@@ -444,51 +515,16 @@ class DisplayController:
return False return False
def _tick_plugin_updates_for_vegas(self):
"""
Run scheduled plugin updates and return IDs of plugins that were updated.
Called periodically by the Vegas coordinator to keep plugin data fresh
during Vegas mode. Returns a list of plugin IDs whose data changed so
Vegas can refresh their content in the scroll.
Returns:
List of updated plugin IDs, or None if no updates occurred
"""
if not self.plugin_manager or not hasattr(self.plugin_manager, 'plugin_last_update'):
self._tick_plugin_updates()
return None
# Snapshot update timestamps before ticking
old_times = dict(self.plugin_manager.plugin_last_update)
# Run the scheduled updates
self._tick_plugin_updates()
# Detect which plugins were actually updated
updated = []
for plugin_id, new_time in self.plugin_manager.plugin_last_update.items():
if new_time > old_times.get(plugin_id, 0.0):
updated.append(plugin_id)
if updated:
logger.info("Vegas update tick: %d plugin(s) updated: %s", len(updated), updated)
return updated or None
def _check_schedule(self): def _check_schedule(self):
"""Check if display should be active based on schedule.""" """Check if display should be active based on schedule."""
# Get fresh config from config_service to support hot-reload schedule_config = self.config.get('schedule', {})
current_config = self.config_service.get_config()
schedule_config = current_config.get('schedule', {})
# If schedule config doesn't exist or is empty, default to always active # If schedule config doesn't exist or is empty, default to always active
if not schedule_config: if not schedule_config:
self.is_display_active = True self.is_display_active = True
self._was_display_active = True # Track previous state for schedule change detection self._was_display_active = True # Track previous state for schedule change detection
return return
# Check if schedule is explicitly disabled # Check if schedule is explicitly disabled
# Default to True (schedule enabled) if 'enabled' key is missing for backward compatibility # Default to True (schedule enabled) if 'enabled' key is missing for backward compatibility
if 'enabled' in schedule_config and not schedule_config.get('enabled', True): if 'enabled' in schedule_config and not schedule_config.get('enabled', True):
@@ -498,7 +534,7 @@ class DisplayController:
return return
# Get configured timezone, default to UTC # Get configured timezone, default to UTC
timezone_str = current_config.get('timezone', 'UTC') timezone_str = self.config.get('timezone', 'UTC')
try: try:
tz = pytz.timezone(timezone_str) tz = pytz.timezone(timezone_str)
except pytz.UnknownTimeZoneError: except pytz.UnknownTimeZoneError:
@@ -596,18 +632,15 @@ class DisplayController:
Target brightness level (dim_brightness if in dim period, Target brightness level (dim_brightness if in dim period,
normal brightness otherwise) normal brightness otherwise)
""" """
# Get fresh config from config_service to support hot-reload
current_config = self.config_service.get_config()
# Get normal brightness from config # Get normal brightness from config
normal_brightness = current_config.get('display', {}).get('hardware', {}).get('brightness', 90) normal_brightness = self.config.get('display', {}).get('hardware', {}).get('brightness', 90)
# If display is OFF via schedule, don't process dim schedule # If display is OFF via schedule, don't process dim schedule
if not self.is_display_active: if not self.is_display_active:
self.is_dimmed = False self.is_dimmed = False
return normal_brightness return normal_brightness
dim_config = current_config.get('dim_schedule', {}) dim_config = self.config.get('dim_schedule', {})
# If dim schedule doesn't exist or is disabled, use normal brightness # If dim schedule doesn't exist or is disabled, use normal brightness
if not dim_config or not dim_config.get('enabled', False): if not dim_config or not dim_config.get('enabled', False):
@@ -615,7 +648,7 @@ class DisplayController:
return normal_brightness return normal_brightness
# Get configured timezone # Get configured timezone
timezone_str = current_config.get('timezone', 'UTC') timezone_str = self.config.get('timezone', 'UTC')
try: try:
tz = pytz.timezone(timezone_str) tz = pytz.timezone(timezone_str)
except pytz.UnknownTimeZoneError: except pytz.UnknownTimeZoneError:
@@ -722,21 +755,83 @@ class DisplayController:
except Exception: # pylint: disable=broad-except except Exception: # pylint: disable=broad-except
logger.exception("Error running scheduled plugin updates") logger.exception("Error running scheduled plugin updates")
def _tick_plugin_updates_throttled(self, min_interval: float = 0.0): _FOLLOWER_SEND_INTERVAL = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
"""Throttled version of _tick_plugin_updates for high-FPS loops.
Args: def _follower_rebuild_scroll_image(self) -> None:
min_interval: Minimum seconds between calls. When <= 0 the """Follower: rebuild the local Vegas scroll image so both Pis render from
call passes straight through to _tick_plugin_updates so the same fresh plugin data. Called at startup (after Vegas initializes)
plugin-configured update_interval values are never capped. and each time the leader broadcasts a new-cycle signal. Runs in a daemon
thread so it never blocks the 60fps render loop.
""" """
if min_interval <= 0: try:
self._tick_plugin_updates() vc = getattr(self, 'vegas_coordinator', None)
if not vc:
logger.warning("Sync: follower has no vegas_coordinator — cannot build scroll image")
return
rp = vc.render_pipeline
if not rp:
logger.warning("Sync: follower vegas_coordinator has no render_pipeline")
return
logger.info("Sync: follower starting scroll image rebuild")
ok = rp.start_new_cycle()
if ok and rp.scroll_helper.cached_image is not None:
logger.info(
"Sync: follower scroll image ready — %dx%d",
rp.scroll_helper.cached_image.width,
rp.scroll_helper.cached_image.height,
)
else:
logger.warning(
"Sync: follower scroll image rebuild FAILED (ok=%s, cached=%s)",
ok, rp.scroll_helper.cached_image is not None,
)
except Exception as exc:
logger.warning("Sync: follower scroll image rebuild error: %s", exc, exc_info=True)
def _send_follower_frame(self, plugin_instance) -> None:
"""Leader: generate and send the follower's portion of the current frame.
The follower is physically to the LEFT of the leader in a right-to-left
scrolling ticker, so it shows content at scroll_position - display_width
(content that already scrolled off the leader's left edge).
Set sync.follower_position = "right" in config to invert this.
"""
if not (self.sync_manager and self.sync_manager.role == SyncRole.LEADER):
return return
# Throttle to ~90fps via _FOLLOWER_SEND_INTERVAL — raw RGB bytes, no encode/decode
now = time.time() now = time.time()
if now - self._last_plugin_tick_time >= min_interval: if now - getattr(self, '_last_follower_send', 0) < self._FOLLOWER_SEND_INTERVAL:
self._last_plugin_tick_time = now return
self._tick_plugin_updates() self._last_follower_send = now
follower_frame = None
width = self.display_manager.width
sync_cfg = self.config.get("sync", {})
sign = -1 if sync_cfg.get("follower_position", "left") == "left" else 1
offset = sign * width
# 1. Explicit hook — plugin opted in with get_offset_frame()
try:
follower_frame = plugin_instance.get_offset_frame(offset)
except AttributeError:
pass # Most plugins don't implement get_offset_frame; that's expected
# 2. Auto-detect — plugin has a scroll_helper (standard pattern for all
# scroll plugins). Works with zero plugin code changes.
if follower_frame is None:
try:
scroll_h = getattr(plugin_instance, 'scroll_helper', None)
if scroll_h is not None:
follower_frame = scroll_h.get_portion_at(scroll_h.scroll_position + offset)
except Exception: # nosec B110 - scroll_helper.get_portion_at is optional; skip on error
pass
# 3. Mirror fallback — static plugins (clock, weather) show same frame
if follower_frame is None:
follower_frame = self.display_manager.image
if follower_frame is not None:
self.sync_manager.send_frame(follower_frame)
def _sleep_with_plugin_updates(self, duration: float, tick_interval: float = 1.0): def _sleep_with_plugin_updates(self, duration: float, tick_interval: float = 1.0):
"""Sleep while continuing to service plugin update schedules.""" """Sleep while continuing to service plugin update schedules."""
@@ -1373,6 +1468,88 @@ class DisplayController:
# Plugins update on their own schedules - no forced sync updates needed # Plugins update on their own schedules - no forced sync updates needed
# Each plugin has its own update_interval and background services # Each plugin has its own update_interval and background services
# Multi-display sync: follower mode — render frames received from leader.
# Plugin update() threads still run (via _tick_plugin_updates above) so
# data is fresh when we return to standalone if the leader goes offline.
if self.sync_manager.is_follower_active():
# Dead-reckoning follower render:
# Advance local position at configured speed each tick; snap or
# gently correct toward received scroll_x to absorb UDP jitter.
_now_dr = time.perf_counter()
_dt = _now_dr - getattr(self, '_follower_dr_last_t', _now_dr)
self._follower_dr_last_t = _now_dr
vc = getattr(self, 'vegas_coordinator', None)
rp = vc.render_pipeline if (vc and vc.render_pipeline) else None
width = self.display_manager.width
# Advance local position at Vegas scroll speed (px/s → px/tick)
vegas_speed = (
self.config.get('display', {})
.get('vegas_scroll', {})
.get('scroll_speed', 75)
)
local_x = getattr(self, '_follower_local_x', None)
if local_x is None:
local_x = float(width) # safe start (past pre-roll guard)
local_x += vegas_speed * _dt
# Pull latest position from leader (may be None if no packet yet)
scroll_x = self.sync_manager.get_latest_scroll_x()
if scroll_x is not None:
diff = scroll_x - local_x
total_w = (
rp.scroll_helper.total_scroll_width
if rp and rp.scroll_helper.total_scroll_width
else width * 4
)
if abs(diff) > total_w * 0.5:
# Large jump → cycle reset, snap immediately
local_x = float(scroll_x)
self._follower_pending_new_image = True
elif abs(diff) > 10:
# Moderate drift → 20% correction per tick
local_x += diff * 0.20
else:
# Near → gentle 5% correction
local_x += diff * 0.05
self._follower_local_x = local_x
if rp and rp.scroll_helper.cached_image is not None:
sync_cfg = self.config.get("sync", {})
sign = -1 if sync_cfg.get("follower_position", "left") == "left" else 1
# Hold last frame until TCP image arrives after cycle reset
if not getattr(self, "_follower_pending_new_image", False):
if local_x >= width:
rp.scroll_helper.scroll_position = local_x + sign * width
frame = rp.scroll_helper.get_visible_portion()
if frame is not None:
self._follower_last_frame = frame
elif scroll_x is None:
# Fallback: pixel frame before first scroll_x arrives
frame = self.sync_manager.get_latest_frame()
if frame is not None:
self._follower_last_frame = frame
display_frame = getattr(self, '_follower_last_frame', None)
if display_frame is not None:
self.display_manager.image = display_frame
self.display_manager._sync_render_allowed = True
self.display_manager.update_display()
self.display_manager._sync_render_allowed = False
# Precision deadline timer — keeps render at exactly 60fps
_deadline = getattr(self, '_follower_deadline', None)
_now = time.perf_counter()
if _deadline is None or _now > _deadline + 0.1:
_deadline = _now
_deadline += 1.0 / 60
self._follower_deadline = _deadline
_sleep = _deadline - time.perf_counter()
if _sleep > 0:
time.sleep(_sleep)
continue
# Process any deferred updates that may have accumulated # Process any deferred updates that may have accumulated
# This also cleans up expired updates to prevent memory leaks # This also cleans up expired updates to prevent memory leaks
self.display_manager.process_deferred_updates() self.display_manager.process_deferred_updates()
@@ -1746,7 +1923,7 @@ class DisplayController:
) )
target_duration = max_duration target_duration = max_duration
start_time = time.monotonic() start_time = time.time()
def _should_exit_dynamic(elapsed_time: float) -> bool: def _should_exit_dynamic(elapsed_time: float) -> bool:
if not dynamic_enabled: if not dynamic_enabled:
@@ -1805,34 +1982,19 @@ class DisplayController:
except Exception: # pylint: disable=broad-except except Exception: # pylint: disable=broad-except
logger.exception("Error during display update") logger.exception("Error during display update")
# Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display)
time.sleep(display_interval) time.sleep(display_interval)
self._tick_plugin_updates_throttled(min_interval=1.0) self._tick_plugin_updates()
self._poll_on_demand_requests() self._poll_on_demand_requests()
self._check_on_demand_expiration() self._check_on_demand_expiration()
# Check for live priority every ~30s so live
# games can interrupt long display durations
elapsed = time.monotonic() - start_time
now = time.monotonic()
if not self.on_demand_active and now >= self._next_live_priority_check:
self._next_live_priority_check = now + 30.0
live_mode = self._check_live_priority()
if live_mode and live_mode != active_mode:
logger.info("Live priority detected during high-FPS loop: %s", live_mode)
self.current_display_mode = live_mode
self.force_change = True
try:
self.current_mode_index = self.available_modes.index(live_mode)
except ValueError:
pass
# continue the main while loop to skip
# post-loop rotation/sleep logic
break
if self.current_display_mode != active_mode: if self.current_display_mode != active_mode:
logger.debug("Mode changed during high-FPS loop, breaking early") logger.debug("Mode changed during high-FPS loop, breaking early")
break break
elapsed = time.time() - start_time
if elapsed >= target_duration: if elapsed >= target_duration:
logger.debug( logger.debug(
"Reached high-FPS target duration %.2fs for mode %s", "Reached high-FPS target duration %.2fs for mode %s",
@@ -1862,7 +2024,7 @@ class DisplayController:
time.sleep(display_interval) time.sleep(display_interval)
self._tick_plugin_updates() self._tick_plugin_updates()
elapsed = time.monotonic() - start_time elapsed = time.time() - start_time
if elapsed >= target_duration: if elapsed >= target_duration:
logger.debug( logger.debug(
"Reached standard target duration %.2fs for mode %s", "Reached standard target duration %.2fs for mode %s",
@@ -1889,25 +2051,11 @@ class DisplayController:
except Exception: # pylint: disable=broad-except except Exception: # pylint: disable=broad-except
logger.exception("Error during display update") logger.exception("Error during display update")
# Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display)
self._poll_on_demand_requests() self._poll_on_demand_requests()
self._check_on_demand_expiration() self._check_on_demand_expiration()
# Check for live priority every ~30s so live
# games can interrupt long display durations
now = time.monotonic()
if not self.on_demand_active and now >= self._next_live_priority_check:
self._next_live_priority_check = now + 30.0
live_mode = self._check_live_priority()
if live_mode and live_mode != active_mode:
logger.info("Live priority detected during display loop: %s", live_mode)
self.current_display_mode = live_mode
self.force_change = True
try:
self.current_mode_index = self.available_modes.index(live_mode)
except ValueError:
pass
break
if self.current_display_mode != active_mode: if self.current_display_mode != active_mode:
logger.info("Mode changed during display loop from %s to %s, breaking early", active_mode, self.current_display_mode) logger.info("Mode changed during display loop from %s to %s, breaking early", active_mode, self.current_display_mode)
break break
@@ -1921,26 +2069,19 @@ class DisplayController:
loop_completed = True loop_completed = True
break break
# If live priority preempted the display loop, skip
# all post-loop logic (remaining sleep, rotation) and
# restart the main loop so the live mode displays
# immediately.
if self.current_display_mode != active_mode:
continue
# Ensure we honour minimum duration when not dynamic and loop ended early # Ensure we honour minimum duration when not dynamic and loop ended early
if ( if (
not dynamic_enabled not dynamic_enabled
and not loop_completed and not loop_completed
and not needs_high_fps and not needs_high_fps
): ):
elapsed = time.monotonic() - start_time elapsed = time.time() - start_time
remaining_sleep = max(0.0, max_duration - elapsed) remaining_sleep = max(0.0, max_duration - elapsed)
if remaining_sleep > 0: if remaining_sleep > 0:
self._sleep_with_plugin_updates(remaining_sleep) self._sleep_with_plugin_updates(remaining_sleep)
if dynamic_enabled: if dynamic_enabled:
elapsed_total = time.monotonic() - start_time elapsed_total = time.time() - start_time
cycle_done = self._plugin_cycle_complete(manager_to_display) cycle_done = self._plugin_cycle_complete(manager_to_display)
# Log cycle completion status and metrics # Log cycle completion status and metrics

View File

@@ -1,11 +1,14 @@
import json
import os import os
import tempfile
if os.getenv("EMULATOR", "false") == "true": if os.getenv("EMULATOR", "false") == "true":
from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions
else: else:
from rgbmatrix import RGBMatrix, RGBMatrixOptions from rgbmatrix import RGBMatrix, RGBMatrixOptions
from contextlib import contextmanager
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
import time import time
from typing import Dict, Any, List, Tuple from typing import Dict, Any, List
import logging import logging
import math import math
import freetype import freetype
@@ -28,8 +31,10 @@ class DisplayManager:
self.config = config or {} self.config = config or {}
self._force_fallback = force_fallback self._force_fallback = force_fallback
self._suppress_test_pattern = suppress_test_pattern self._suppress_test_pattern = suppress_test_pattern
# When True, update_display() and clear() skip hardware writes (used during off-screen content capture)
self._capture_mode_active = False
# Snapshot settings for web preview integration (service writes, web reads) # Snapshot settings for web preview integration (service writes, web reads)
self._snapshot_path = "/tmp/led_matrix_preview.png" self._snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path intentional; web UI reads same path
self._snapshot_min_interval_sec = 0.2 # max ~5 fps self._snapshot_min_interval_sec = 0.2 # max ~5 fps
self._last_snapshot_ts = 0.0 self._last_snapshot_ts = 0.0
@@ -55,8 +60,7 @@ class DisplayManager:
def _setup_matrix(self): def _setup_matrix(self):
"""Initialize the RGB matrix with configuration settings.""" """Initialize the RGB matrix with configuration settings."""
setup_start = time.time() _init_error_str = None
try: try:
# Allow callers (e.g., web UI) to force non-hardware fallback mode # Allow callers (e.g., web UI) to force non-hardware fallback mode
if getattr(self, '_force_fallback', False): if getattr(self, '_force_fallback', False):
@@ -86,7 +90,7 @@ class DisplayManager:
options.disable_hardware_pulsing = hardware_config.get('disable_hardware_pulsing', False) options.disable_hardware_pulsing = hardware_config.get('disable_hardware_pulsing', False)
options.show_refresh_rate = hardware_config.get('show_refresh_rate', False) options.show_refresh_rate = hardware_config.get('show_refresh_rate', False)
options.limit_refresh_rate_hz = hardware_config.get('limit_refresh_rate_hz', 90) options.limit_refresh_rate_hz = hardware_config.get('limit_refresh_rate_hz', 90)
options.gpio_slowdown = runtime_config.get('gpio_slowdown', 2) options.gpio_slowdown = runtime_config.get('gpio_slowdown', 3)
# Disable internal privilege dropping - we manage this via systemd or remain root # Disable internal privilege dropping - we manage this via systemd or remain root
# This prevents the library from dropping to 'daemon' user which breaks file permissions # This prevents the library from dropping to 'daemon' user which breaks file permissions
@@ -99,6 +103,18 @@ class DisplayManager:
options.pwm_dither_bits = hardware_config.get('pwm_dither_bits') options.pwm_dither_bits = hardware_config.get('pwm_dither_bits')
if 'inverse_colors' in hardware_config: if 'inverse_colors' in hardware_config:
options.inverse_colors = hardware_config.get('inverse_colors') options.inverse_colors = hardware_config.get('inverse_colors')
# Pi 5 only: 0=PIO/RP1 coprocessor (default, less CPU),
# 1=RIO/Registered IO (faster; gpio_slowdown effect is inverted in this mode)
if 'rp1_rio' in runtime_config:
if hasattr(options, 'rp1_rio'):
options.rp1_rio = runtime_config.get('rp1_rio')
else:
logger.warning(
"rp1_rio is set in config but the installed rgbmatrix library does "
"not support it — the library was likely built without Pi 5 RP1 "
"support (mmap to 0x3f000000 instead of RP1 chip). "
"Fix: sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh"
)
logger.info(f"Initializing RGB Matrix with settings: rows={options.rows}, cols={options.cols}, chain_length={options.chain_length}, parallel={options.parallel}, hardware_mapping={options.hardware_mapping}") logger.info(f"Initializing RGB Matrix with settings: rows={options.rows}, cols={options.cols}, chain_length={options.chain_length}, parallel={options.parallel}, hardware_mapping={options.hardware_mapping}")
@@ -129,6 +145,7 @@ class DisplayManager:
self._draw_test_pattern() self._draw_test_pattern()
except Exception as e: except Exception as e:
_init_error_str = str(e)
logger.error(f"Failed to initialize RGB Matrix: {e}", exc_info=True) logger.error(f"Failed to initialize RGB Matrix: {e}", exc_info=True)
# Create a fallback image for web preview using configured dimensions when available # Create a fallback image for web preview using configured dimensions when available
self.matrix = None self.matrix = None
@@ -149,12 +166,41 @@ class DisplayManager:
self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0)) self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0))
self.draw.line([0, 0, fallback_width - 1, fallback_height - 1], fill=(0, 255, 0)) self.draw.line([0, 0, fallback_width - 1, fallback_height - 1], fill=(0, 255, 0))
self.draw.text((2, max(0, (fallback_height // 2) - 4)), "Simulation", fill=(0, 128, 255)) self.draw.text((2, max(0, (fallback_height // 2) - 4)), "Simulation", fill=(0, 128, 255))
except Exception: except Exception: # nosec B110 - best-effort fallback visualization; drawing errors must not crash startup
# Best-effort; ignore drawing errors in fallback # Best-effort; ignore drawing errors in fallback
pass pass
logger.error(f"Matrix initialization failed, using fallback mode with size {fallback_width}x{fallback_height}. Error: {e}") logger.error(
f"Matrix initialization failed — running in fallback/simulation mode "
f"(size {fallback_width}x{fallback_height}). Error: {e}. "
"On Raspberry Pi 5: ensure rpi-rgb-led-matrix was built from the latest "
"submodule (re-run first_time_install.sh). gpio_slowdown of 23 is typical for Pi 5 PIO mode."
)
# Do not raise here; allow fallback mode so web preview and non-hardware environments work # Do not raise here; allow fallback mode so web preview and non-hardware environments work
# Write hardware status file so the web UI can surface init failures
_hw_status = {"ok": self.matrix is not None, "error": _init_error_str}
_status_path = "/tmp/led_matrix_hw_status.json" # nosec B108
try:
if os.path.islink(_status_path):
logger.warning("Skipping hardware status write: %s is a symlink", _status_path)
else:
_fd, _tmp_path = tempfile.mkstemp(dir="/tmp", prefix=".led_hw_") # nosec B108
try:
with os.fdopen(_fd, "w") as _f:
json.dump(_hw_status, _f)
_f.flush()
os.fsync(_f.fileno())
os.chmod(_tmp_path, 0o600)
os.replace(_tmp_path, _status_path)
except Exception:
try:
os.unlink(_tmp_path)
except OSError:
pass
raise
except Exception:
logger.error("Failed to write hardware status file", exc_info=True)
@property @property
def width(self): def width(self):
"""Get the display width.""" """Get the display width."""
@@ -255,6 +301,22 @@ class DisplayManager:
except Exception as e: except Exception as e:
logger.error(f"Error drawing test pattern: {e}", exc_info=True) logger.error(f"Error drawing test pattern: {e}", exc_info=True)
@contextmanager
def capture_mode(self):
"""Suppress hardware output during off-screen content capture.
Plugins call update_display() as part of their normal display() flow.
When fetching content for Vegas mode the render loop is still running,
so any incidental hardware write causes a visible flash on the matrix.
Entering this context prevents those writes without affecting the PIL
image buffer, which the adapter reads to extract content.
"""
self._capture_mode_active = True
try:
yield
finally:
self._capture_mode_active = False
def update_display(self): def update_display(self):
"""Update the display using double buffering with proper sync.""" """Update the display using double buffering with proper sync."""
try: try:
@@ -264,10 +326,13 @@ class DisplayManager:
# Still write a snapshot so the web UI can preview # Still write a snapshot so the web UI can preview
self._write_snapshot_if_due() self._write_snapshot_if_due()
return return
# Copy the current image to the offscreen canvas if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
# Copy the current image to the offscreen canvas
self.offscreen_canvas.SetImage(self.image) self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately # Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas) self.matrix.SwapOnVSync(self.offscreen_canvas)
@@ -304,21 +369,23 @@ class DisplayManager:
# Create a new black image # Create a new black image
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height)) self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
# Clear both canvases and the underlying matrix to ensure no artifacts if not self._capture_mode_active:
try: # Clear both canvases and the underlying matrix to ensure no artifacts.
self.offscreen_canvas.Clear() # Failures are non-fatal — the image buffer is already black above, so
except Exception: # the next update_display() call will push clean content regardless.
pass try:
try: self.offscreen_canvas.Clear()
self.current_canvas.Clear() except (RuntimeError, OSError) as e:
except Exception: logger.error("Failed to clear offscreen canvas: %s", e)
pass try:
try: self.current_canvas.Clear()
# Extra safety: clear the matrix front buffer as well except (RuntimeError, OSError) as e:
self.matrix.Clear() logger.error("Failed to clear current canvas: %s", e)
except Exception: try:
pass self.matrix.Clear()
except (RuntimeError, OSError) as e:
logger.error("Failed to clear matrix front buffer: %s", e)
# Note: We do NOT call update_display() here to avoid black flashes. # Note: We do NOT call update_display() here to avoid black flashes.
# The caller should call update_display() after drawing new content. # The caller should call update_display() after drawing new content.
@@ -714,8 +781,8 @@ class DisplayManager:
try: try:
self.image = Image.new('RGB', (self.width, self.height)) self.image = Image.new('RGB', (self.width, self.height))
self.draw = ImageDraw.Draw(self.image) self.draw = ImageDraw.Draw(self.image)
except Exception: except (OSError, RuntimeError, ValueError, MemoryError):
pass logger.debug("Canvas reset during cleanup failed", exc_info=True)
# Reset the singleton state when cleaning up # Reset the singleton state when cleaning up
DisplayManager._instance = None DisplayManager._instance = None
DisplayManager._initialized = False DisplayManager._initialized = False
@@ -872,7 +939,7 @@ class DisplayManager:
# Never modify /tmp permissions - it has special system permissions (1777) # Never modify /tmp permissions - it has special system permissions (1777)
# that must not be changed or it breaks apt and other system tools # that must not be changed or it breaks apt and other system tools
parent_dir = snapshot_path_obj.parent parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode()) ensure_directory_permissions(parent_dir, get_assets_dir_mode())
# Write atomically: temp then replace # Write atomically: temp then replace
tmp_path = f"{self._snapshot_path}.tmp" tmp_path = f"{self._snapshot_path}.tmp"

View File

@@ -19,8 +19,7 @@ Usage:
import logging import logging
import time import time
import requests import requests
from typing import Dict, List, Set, Optional, Any from typing import Dict, List
from datetime import datetime, timezone
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)

View File

@@ -3,15 +3,14 @@ import logging
import freetype import freetype
import json import json
import hashlib import hashlib
import urllib.parse
import urllib.request import urllib.request
import zipfile import zipfile
import tempfile import tempfile
import shutil
import time import time
from pathlib import Path from pathlib import Path
from PIL import ImageFont from PIL import ImageFont
from typing import Dict, Tuple, Optional, Union, Any, List from typing import Dict, Tuple, Optional, Union, Any, List
from functools import lru_cache
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -267,9 +266,12 @@ class FontManager:
logger.info(f"Using cached font: {cache_path}") logger.info(f"Using cached font: {cache_path}")
return str(cache_path) return str(cache_path)
# Download font # Download font — restrict to http/https to prevent file:// reads
parsed = urllib.parse.urlparse(url)
if parsed.scheme not in ('http', 'https'):
raise ValueError(f"Font URL must use http or https, got: {parsed.scheme!r}")
logger.info(f"Downloading font from {url}") logger.info(f"Downloading font from {url}")
urllib.request.urlretrieve(url, cache_path) urllib.request.urlretrieve(url, cache_path) # nosec B310 - scheme validated above
# Handle zip files # Handle zip files
if url.endswith('.zip'): if url.endswith('.zip'):
@@ -699,8 +701,6 @@ class FontManager:
fonts_dir = Path("assets/fonts") fonts_dir = Path("assets/fonts")
ensure_directory_permissions(fonts_dir, get_assets_dir_mode()) ensure_directory_permissions(fonts_dir, get_assets_dir_mode())
target_path = os.path.join(fonts_dir, f"{family_name}.{font_file_path.rsplit('.', 1)[-1]}")
# Add to catalog # Add to catalog
self.font_catalog[family_name] = font_file_path self.font_catalog[family_name] = font_file_path
self.clear_cache() self.clear_cache()
@@ -746,11 +746,11 @@ class FontManager:
if font_path.endswith('.bdf'): if font_path.endswith('.bdf'):
# Try to load BDF font # Try to load BDF font
face = freetype.Face(font_path) freetype.Face(font_path)
return {"valid": True, "type": "bdf", "family": "unknown"} return {"valid": True, "type": "bdf", "family": "unknown"}
elif font_path.endswith('.ttf'): elif font_path.endswith('.ttf'):
# Try to load TTF font # Try to load TTF font
font = ImageFont.truetype(font_path, 12) ImageFont.truetype(font_path, 12)
return {"valid": True, "type": "ttf", "family": "unknown"} return {"valid": True, "type": "ttf", "family": "unknown"}
else: else:
return {"valid": False, "error": "Unsupported font format"} return {"valid": False, "error": "Unsupported font format"}

View File

@@ -1,7 +1,6 @@
import os import os
import time
import freetype import freetype
from PIL import Image, ImageDraw, ImageFont from PIL import ImageDraw, ImageFont
import logging import logging
from typing import Dict, Any from typing import Dict, Any
from src.display_manager import DisplayManager from src.display_manager import DisplayManager
@@ -73,7 +72,6 @@ class FontTestManager:
def update(self): def update(self):
"""No update needed for static display.""" """No update needed for static display."""
pass
def display(self, force_clear: bool = False): def display(self, force_clear: bool = False):
"""Display the font with sample text.""" """Display the font with sample text."""
@@ -81,10 +79,6 @@ class FontTestManager:
# Clear the display # Clear the display
self.display_manager.clear() self.display_manager.clear()
# Get display dimensions
width = self.display_manager.matrix.width
height = self.display_manager.matrix.height
# Draw font name at the top # Draw font name at the top
self.display_manager.draw_text(self.current_config['display_name'], y=2, color=(255, 255, 255)) self.display_manager.draw_text(self.current_config['display_name'], y=2, color=(255, 255, 255))

View File

@@ -7,7 +7,6 @@ version of BackgroundCacheMixin that works for weather, stocks, news, etc.
""" """
import time import time
import logging
from typing import Dict, Optional, Any, Callable from typing import Dict, Optional, Any, Callable

View File

@@ -6,9 +6,8 @@ Handles custom layouts, element positioning, and display composition.
import json import json
import os import os
import logging import logging
from typing import Dict, List, Any, Tuple from typing import Dict, List, Any
from datetime import datetime from datetime import datetime
from PIL import Image, ImageDraw, ImageFont
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)

View File

@@ -11,7 +11,7 @@ import time
import logging import logging
import requests import requests
import json import json
from typing import Dict, Any, List, Optional, Tuple from typing import Dict, List, Optional, Tuple
from pathlib import Path from pathlib import Path
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
from requests.adapters import HTTPAdapter from requests.adapters import HTTPAdapter
@@ -191,7 +191,7 @@ class LogoDownloader:
return True return True
except PermissionError: except PermissionError:
logger.error(f"Permission denied: Cannot write to directory {path}") logger.error(f"Permission denied: Cannot write to directory {path}")
logger.error(f"Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh") logger.error("Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
return False return False
except Exception as e: except Exception as e:
logger.error(f"Failed to test write access to directory {path}: {e}") logger.error(f"Failed to test write access to directory {path}: {e}")
@@ -248,7 +248,7 @@ class LogoDownloader:
except PermissionError as e: except PermissionError as e:
logger.error(f"Permission denied downloading logo for {team_abbreviation}: {e}") logger.error(f"Permission denied downloading logo for {team_abbreviation}: {e}")
logger.error(f"Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh") logger.error("Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
return False return False
except requests.exceptions.RequestException as e: except requests.exceptions.RequestException as e:
logger.error(f"Failed to download logo for {team_abbreviation}: {e}") logger.error(f"Failed to download logo for {team_abbreviation}: {e}")

View File

@@ -11,7 +11,7 @@ Builds on existing PluginHealthTracker to provide:
import threading import threading
import time import time
from typing import Dict, Any, Optional, List, Callable from typing import Dict, Any, Optional, List, Callable
from datetime import datetime, timedelta from datetime import datetime
from enum import Enum from enum import Enum
from dataclasses import dataclass from dataclasses import dataclass

View File

@@ -7,9 +7,8 @@ status tracking and cancellation support.
import threading import threading
import queue import queue
import time
from typing import Dict, Optional, List, Callable, Any from typing import Dict, Optional, List, Callable, Any
from datetime import datetime, timedelta from datetime import datetime
from pathlib import Path from pathlib import Path
import json import json

View File

@@ -7,7 +7,7 @@ and their associated data structures.
from enum import Enum from enum import Enum
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Dict, Any, Optional, List from typing import Dict, Any, Optional
from datetime import datetime from datetime import datetime
import uuid import uuid

View File

@@ -6,9 +6,8 @@ error isolation, and performance monitoring.
""" """
import time import time
import signal from typing import Any, Optional, Callable
from typing import Any, Optional, Dict, Callable from threading import Thread
from threading import Thread, Event
import logging import logging
from src.exceptions import PluginError from src.exceptions import PluginError
@@ -16,9 +15,8 @@ from src.logging_config import get_logger
from src.error_aggregator import record_error from src.error_aggregator import record_error
class TimeoutError(Exception): class PluginTimeoutError(Exception):
"""Raised when a plugin operation times out.""" """Raised when a plugin operation times out."""
pass
class PluginExecutor: class PluginExecutor:
@@ -57,7 +55,7 @@ class PluginExecutor:
Result of operation Result of operation
Raises: Raises:
TimeoutError: If operation times out PluginTimeoutError: If operation times out
PluginError: If operation raises an exception PluginError: If operation raises an exception
""" """
timeout = timeout or self.default_timeout timeout = timeout or self.default_timeout
@@ -81,7 +79,7 @@ class PluginExecutor:
if not result_container['completed']: if not result_container['completed']:
error_msg = f"{plugin_context} operation timed out after {timeout}s" error_msg = f"{plugin_context} operation timed out after {timeout}s"
self.logger.error(error_msg) self.logger.error(error_msg)
timeout_error = TimeoutError(error_msg) timeout_error = PluginTimeoutError(error_msg)
record_error(timeout_error, plugin_id=plugin_id, operation="timeout") record_error(timeout_error, plugin_id=plugin_id, operation="timeout")
raise timeout_error raise timeout_error
@@ -128,7 +126,7 @@ class PluginExecutor:
) )
return True return True
except TimeoutError: except PluginTimeoutError:
self.logger.error("Plugin %s update() timed out", plugin_id) self.logger.error("Plugin %s update() timed out", plugin_id)
return False return False
except PluginError: except PluginError:
@@ -204,7 +202,7 @@ class PluginExecutor:
# For backward compatibility: if plugin returns None or something else, treat as success # For backward compatibility: if plugin returns None or something else, treat as success
self.logger.debug(f"Plugin {plugin_id} display() returned non-boolean: {result}, treating as True") self.logger.debug(f"Plugin {plugin_id} display() returned non-boolean: {result}, treating as True")
return True return True
except TimeoutError: except PluginTimeoutError:
self.logger.error("Plugin %s display() timed out", plugin_id) self.logger.error("Plugin %s display() timed out", plugin_id)
return False return False
except PluginError: except PluginError:
@@ -247,7 +245,7 @@ class PluginExecutor:
timeout=timeout, timeout=timeout,
plugin_id=plugin_id plugin_id=plugin_id
) )
except (TimeoutError, PluginError, Exception) as e: except Exception as e: # covers PluginTimeoutError, PluginError, and unexpected errors
self.logger.warning( self.logger.warning(
"Plugin %s %s failed, using default return: %s", "Plugin %s %s failed, using default return: %s",
plugin_id, plugin_id,

View File

@@ -7,10 +7,7 @@ Handles dynamic plugin loading from the plugins/ directory.
API Version: 1.0.0 API Version: 1.0.0
""" """
import os
import json import json
import importlib
import importlib.util
import sys import sys
import subprocess import subprocess
import time import time
@@ -677,6 +674,44 @@ class PluginManager:
# Default: 60 seconds # Default: 60 seconds
return 60.0 return 60.0
def _record_update_failure(
self,
plugin_id: str,
exc: Optional[Exception] = None,
) -> None:
"""Apply the standard failure-recovery path for a plugin update.
Stamps plugin_last_update with the actual failure time so the full
configured interval elapses before the next retry, then transitions
the plugin back to ENABLED (not ERROR) with structured error context
so automatic recovery happens on the next scheduled cycle.
Args:
plugin_id: Plugin identifier
exc: The exception that caused the failure, if any. When None a
synthetic ExecutionFailure exception is constructed from the
timeout/executor-error path.
"""
failure_time = time.time()
if exc is not None:
err: Exception = exc
error_type = type(exc).__name__
else:
err = Exception(f"Plugin {plugin_id} execution failed (timeout or executor error)")
error_type = 'ExecutionFailure'
error_info = {
'error': str(err),
'error_type': error_type,
'timestamp': failure_time,
'recoverable': True,
}
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
self.plugin_last_update[plugin_id] = failure_time
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info, error=err)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, err)
def run_scheduled_updates(self, current_time: Optional[float] = None) -> None: def run_scheduled_updates(self, current_time: Optional[float] = None) -> None:
""" """
Trigger plugin updates based on their defined update intervals. Trigger plugin updates based on their defined update intervals.
@@ -734,16 +769,10 @@ class PluginManager:
if self.health_tracker: if self.health_tracker:
self.health_tracker.record_success(plugin_id) self.health_tracker.record_success(plugin_id)
else: else:
# Execution failed (timeout or error) self._record_update_failure(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ERROR)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, Exception("Plugin execution failed"))
except Exception as exc: # pylint: disable=broad-except except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc) self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=exc) self._record_update_failure(plugin_id, exc=exc)
# Record failure
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, exc)
def update_all_plugins(self) -> None: def update_all_plugins(self) -> None:
""" """
@@ -769,14 +798,12 @@ class PluginManager:
if success: if success:
self.plugin_last_update[plugin_id] = time.time() self.plugin_last_update[plugin_id] = time.time()
self.state_manager.record_update(plugin_id) self.state_manager.record_update(plugin_id)
# Update state back to ENABLED
self.state_manager.set_state(plugin_id, PluginState.ENABLED) self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else: else:
# Execution failed self._record_update_failure(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ERROR)
except Exception as exc: # pylint: disable=broad-except except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc) self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=exc) self._record_update_failure(plugin_id, exc=exc)
def get_plugin_health_metrics(self) -> Dict[str, Any]: def get_plugin_health_metrics(self) -> Dict[str, Any]:
""" """

View File

@@ -5,6 +5,7 @@ Manages plugin state machine (loaded → enabled → running → error)
with state transitions and queries. with state transitions and queries.
""" """
import threading
from enum import Enum from enum import Enum
from typing import Optional, Dict, Any from typing import Optional, Dict, Any
from datetime import datetime from datetime import datetime
@@ -34,6 +35,7 @@ class PluginStateManager:
logger: Optional logger instance logger: Optional logger instance
""" """
self.logger = logger or get_logger(__name__) self.logger = logger or get_logger(__name__)
self._lock = threading.RLock()
self._states: Dict[str, PluginState] = {} self._states: Dict[str, PluginState] = {}
self._state_history: Dict[str, list] = {} self._state_history: Dict[str, list] = {}
self._error_info: Dict[str, Dict[str, Any]] = {} self._error_info: Dict[str, Dict[str, Any]] = {}
@@ -48,44 +50,44 @@ class PluginStateManager:
) -> None: ) -> None:
""" """
Set plugin state and record transition. Set plugin state and record transition.
Args: Args:
plugin_id: Plugin identifier plugin_id: Plugin identifier
state: New state state: New state
error: Optional error if transitioning to ERROR state error: Optional error if transitioning to ERROR state
""" """
old_state = self._states.get(plugin_id, PluginState.UNLOADED) with self._lock:
self._states[plugin_id] = state old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
# Record state transition
if plugin_id not in self._state_history: if plugin_id not in self._state_history:
self._state_history[plugin_id] = [] self._state_history[plugin_id] = []
transition = { transition = {
'timestamp': datetime.now(), 'timestamp': datetime.now(),
'from': old_state.value, 'from': old_state.value,
'to': state.value, 'to': state.value,
'error': str(error) if error else None 'error': str(error) if error else None
}
self._state_history[plugin_id].append(transition)
# Store error info if transitioning to ERROR state
if state == PluginState.ERROR and error:
self._error_info[plugin_id] = {
'error': str(error),
'error_type': type(error).__name__,
'timestamp': datetime.now()
} }
elif state != PluginState.ERROR: self._state_history[plugin_id].append(transition)
# Clear error info when leaving ERROR state
self._error_info.pop(plugin_id, None) # Store error info if transitioning to ERROR state
if state == PluginState.ERROR and error:
self.logger.debug( self._error_info[plugin_id] = {
"Plugin %s state transition: %s%s", 'error': str(error),
plugin_id, 'error_type': type(error).__name__,
old_state.value, 'timestamp': datetime.now()
state.value }
) elif state != PluginState.ERROR:
# Clear error info when leaving ERROR state
self._error_info.pop(plugin_id, None)
self.logger.debug(
"Plugin %s state transition: %s%s",
plugin_id,
old_state.value,
state.value
)
def get_state(self, plugin_id: str) -> PluginState: def get_state(self, plugin_id: str) -> PluginState:
""" """
@@ -136,17 +138,82 @@ class PluginStateManager:
""" """
return self._state_history.get(plugin_id, []) return self._state_history.get(plugin_id, [])
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]: def set_error_info(self, plugin_id: str, error_info: Dict[str, Any]) -> None:
""" """
Get error information for a plugin in ERROR state. Persist structured error context without changing plugin state.
Used for recoverable failures (e.g. update timeout) where the plugin
stays ENABLED but the error details should remain queryable.
Args: Args:
plugin_id: Plugin identifier plugin_id: Plugin identifier
error_info: Arbitrary dict describing the error
Returns:
Error information dict or None
""" """
return self._error_info.get(plugin_id) with self._lock:
self._error_info[plugin_id] = dict(error_info)
def set_state_with_error(
self,
plugin_id: str,
state: PluginState,
error_info: Dict[str, Any],
error: Optional[Exception] = None,
) -> None:
"""Set plugin state and persist error context atomically.
Unlike calling set_state() then set_error_info() separately, this
method holds ``_lock`` for both writes so no reader can observe the
new state without the accompanying error context.
Intentionally does not clear ``_error_info`` the way set_state() does
for non-ERROR transitions — this is the recoverable-failure path where
the error dict is the entire point.
Args:
plugin_id: Plugin identifier
state: New state
error_info: Structured error dict to persist alongside the state
error: Optional exception recorded in the transition history
"""
with self._lock:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
self._state_history[plugin_id].append({
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None,
})
self._error_info[plugin_id] = dict(error_info)
self.logger.debug(
"Plugin %s state transition: %s%s (recoverable error stored)",
plugin_id,
old_state.value,
state.value,
)
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get error information for a plugin.
Returns the stored error dict whether the plugin is in ERROR state or
still ENABLED after a recoverable failure. Returns a shallow copy so
callers cannot mutate the stored snapshot.
Args:
plugin_id: Plugin identifier
Returns:
Copy of the error information dict, or None
"""
with self._lock:
info = self._error_info.get(plugin_id)
return dict(info) if info is not None else None
def record_update(self, plugin_id: str) -> None: def record_update(self, plugin_id: str) -> None:
"""Record that plugin update() was called.""" """Record that plugin update() was called."""

View File

@@ -20,7 +20,6 @@ except ImportError:
class ResourceLimitExceeded(Exception): class ResourceLimitExceeded(Exception):
"""Raised when a plugin exceeds its resource limits.""" """Raised when a plugin exceeds its resource limits."""
pass
@dataclass @dataclass
@@ -228,7 +227,7 @@ class PluginResourceMonitor:
except ResourceLimitExceeded: except ResourceLimitExceeded:
raise raise
except Exception as e: except Exception:
# Still record execution time even on error # Still record execution time even on error
execution_time = time.time() - start_time execution_time = time.time() - start_time
with self._lock: with self._lock:

View File

@@ -5,7 +5,6 @@ Manages saved GitHub repository URLs for easy plugin discovery and installation.
""" """
import json import json
import os
import logging import logging
from pathlib import Path from pathlib import Path
from typing import List, Dict, Optional from typing import List, Dict, Optional

View File

@@ -8,7 +8,6 @@ Provides utilities for extracting defaults, validating configurations, and manag
import copy import copy
import json import json
import logging import logging
import os
from pathlib import Path from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple from typing import Any, Dict, List, Optional, Tuple
import jsonschema import jsonschema

View File

@@ -8,12 +8,12 @@ Detects and fixes inconsistencies between:
- State manager state - State manager state
""" """
from typing import Dict, Any, List, Optional, Set from typing import Dict, Any, List, Set
from dataclasses import dataclass from dataclasses import dataclass
from enum import Enum from enum import Enum
from pathlib import Path from pathlib import Path
from src.plugin_system.state_manager import PluginStateManager, PluginState, PluginStateStatus from src.plugin_system.state_manager import PluginStateManager
from src.logging_config import get_logger from src.logging_config import get_logger
@@ -234,7 +234,7 @@ class StateReconciliation:
'version': manifest.get('version'), 'version': manifest.get('version'),
'name': manifest.get('name') 'name': manifest.get('name')
} }
except Exception: except Exception: # nosec B110 - corrupt/unreadable manifest; skip this plugin, outer except logs
pass pass
except Exception as e: except Exception as e:
self.logger.warning(f"Error reading disk state: {e}") self.logger.warning(f"Error reading disk state: {e}")
@@ -285,7 +285,6 @@ class StateReconciliation:
config = config_state.get(plugin_id, {}) config = config_state.get(plugin_id, {})
disk = disk_state.get(plugin_id, {}) disk = disk_state.get(plugin_id, {})
manager = manager_state.get(plugin_id, {})
state_mgr = state_manager_state.get(plugin_id, {}) state_mgr = state_manager_state.get(plugin_id, {})
# Check: Plugin exists on disk but not in config # Check: Plugin exists on disk but not in config

View File

@@ -10,6 +10,7 @@ import json
import stat import stat
import subprocess import subprocess
import shutil import shutil
import threading
import zipfile import zipfile
import tempfile import tempfile
import requests import requests
@@ -23,7 +24,6 @@ import logging
from src.common.permission_utils import sudo_remove_directory from src.common.permission_utils import sudo_remove_directory
try: try:
import jsonschema
from jsonschema import Draft7Validator, ValidationError from jsonschema import Draft7Validator, ValidationError
JSONSCHEMA_AVAILABLE = True JSONSCHEMA_AVAILABLE = True
except ImportError: except ImportError:
@@ -101,6 +101,10 @@ class PluginStoreManager:
# handlers. Bumping the cached-entry timestamp on failure serves # handlers. Bumping the cached-entry timestamp on failure serves
# the stale payload cheaply until the backoff expires. # the stale payload cheaply until the backoff expires.
self._failure_backoff_seconds = 60 self._failure_backoff_seconds = 60
# Prevents concurrent callers from each firing a network request when
# the registry cache expires. Only one thread fetches; others wait and
# then get the result from the warm cache (double-checked locking).
self._registry_fetch_lock = threading.Lock()
# Ensure plugins directory exists # Ensure plugins directory exists
self.plugins_dir.mkdir(exist_ok=True) self.plugins_dir.mkdir(exist_ok=True)
@@ -433,9 +437,9 @@ class PluginStoreManager:
return stale return stale
if not self.github_token: if not self.github_token:
self.logger.warning( self.logger.warning(
f"GitHub API rate limit likely exceeded (403). " "GitHub API rate limit likely exceeded (403). "
f"Add a GitHub personal access token to config/config_secrets.json " "Add a GitHub personal access token to config/config_secrets.json "
f"under 'github.api_token' to increase rate limits from 60 to 5000/hour." "under 'github.api_token' to increase rate limits from 60 to 5000/hour."
) )
else: else:
self.logger.warning( self.logger.warning(
@@ -576,41 +580,50 @@ class PluginStoreManager:
(current_time - self.registry_cache_time) < self.registry_cache_timeout): (current_time - self.registry_cache_time) < self.registry_cache_timeout):
return self.registry_cache return self.registry_cache
try: with self._registry_fetch_lock:
self.logger.info(f"Fetching plugin registry from {self.REGISTRY_URL}") # Re-check inside the lock — a concurrent caller that was waiting
response = self._http_get_with_retries(self.REGISTRY_URL, timeout=10) # may have already populated the cache while we blocked.
response.raise_for_status() current_time = time.time()
self.registry_cache = response.json() if (self.registry_cache and self.registry_cache_time and
self.registry_cache_time = current_time not force_refresh and
self.logger.info(f"Fetched registry with {len(self.registry_cache.get('plugins', []))} plugins") (current_time - self.registry_cache_time) < self.registry_cache_timeout):
return self.registry_cache
except requests.RequestException as e:
self.logger.error(f"Error fetching registry: {e}")
if raise_on_failure:
raise
# Prefer stale cache over an empty list so the plugin list UI
# keeps working on a flaky connection (e.g. Pi on WiFi). Bump
# registry_cache_time into a short backoff window so the next
# request serves the stale payload cheaply instead of
# re-hitting the network on every request (matches the
# pattern used by github_cache / commit_info_cache).
if self.registry_cache:
self.logger.warning("Falling back to stale registry cache")
self.registry_cache_time = (
time.time() + self._failure_backoff_seconds - self.registry_cache_timeout
)
return self.registry_cache return self.registry_cache
return {"plugins": []}
except json.JSONDecodeError as e: try:
self.logger.error(f"Error parsing registry JSON: {e}") self.logger.info(f"Fetching plugin registry from {self.REGISTRY_URL}")
if raise_on_failure: response = self._http_get_with_retries(self.REGISTRY_URL, timeout=10)
raise response.raise_for_status()
if self.registry_cache: self.registry_cache = response.json()
self.registry_cache_time = ( self.registry_cache_time = current_time
time.time() + self._failure_backoff_seconds - self.registry_cache_timeout self.logger.info(f"Fetched registry with {len(self.registry_cache.get('plugins', []))} plugins")
)
return self.registry_cache return self.registry_cache
return {"plugins": []} except requests.RequestException as e:
self.logger.error(f"Error fetching registry: {e}")
if raise_on_failure:
raise
# Prefer stale cache over an empty list so the plugin list UI
# keeps working on a flaky connection (e.g. Pi on WiFi). Bump
# registry_cache_time into a short backoff window so the next
# request serves the stale payload cheaply instead of
# re-hitting the network on every request (matches the
# pattern used by github_cache / commit_info_cache).
if self.registry_cache:
self.logger.warning("Falling back to stale registry cache")
self.registry_cache_time = (
time.time() + self._failure_backoff_seconds - self.registry_cache_timeout
)
return self.registry_cache
return {"plugins": []}
except json.JSONDecodeError as e:
self.logger.error(f"Error parsing registry JSON: {e}")
if raise_on_failure:
raise
if self.registry_cache:
self.registry_cache_time = (
time.time() + self._failure_backoff_seconds - self.registry_cache_timeout
)
return self.registry_cache
return {"plugins": []}
def search_plugins(self, query: str = "", category: str = "", tags: List[str] = None, fetch_commit_info: bool = True, include_saved_repos: bool = True, saved_repositories_manager = None) -> List[Dict]: def search_plugins(self, query: str = "", category: str = "", tags: List[str] = None, fetch_commit_info: bool = True, include_saved_repos: bool = True, saved_repositories_manager = None) -> List[Dict]:
""" """
@@ -1078,7 +1091,7 @@ class PluginStoreManager:
# Get the actual plugin ID from manifest (source of truth) # Get the actual plugin ID from manifest (source of truth)
manifest_plugin_id = manifest.get('id') manifest_plugin_id = manifest.get('id')
if not manifest_plugin_id: if not manifest_plugin_id:
self.logger.error(f"Plugin manifest missing 'id' field") self.logger.error("Plugin manifest missing 'id' field")
self._safe_remove_directory(plugin_path) self._safe_remove_directory(plugin_path)
return False return False
@@ -1729,7 +1742,7 @@ class PluginStoreManager:
try: try:
self.logger.info(f"Installing dependencies for {plugin_path.name}") self.logger.info(f"Installing dependencies for {plugin_path.name}")
result = subprocess.run( subprocess.run(
['pip3', 'install', '--break-system-packages', '-r', str(requirements_file)], ['pip3', 'install', '--break-system-packages', '-r', str(requirements_file)],
check=True, check=True,
capture_output=True, capture_output=True,
@@ -1850,58 +1863,72 @@ class PluginStoreManager:
return cached[1] return cached[1]
try: try:
sha_result = subprocess.run( # .git may be a file (worktree / submodule) containing "gitdir: <path>".
['git', '-C', str(plugin_path), 'rev-parse', 'HEAD'], # Resolve it to the actual git directory before reading any files.
try:
if git_dir.is_file():
pointer = git_dir.read_text(encoding='utf-8', errors='replace').strip()
if pointer.startswith('gitdir:'):
resolved = (plugin_path / pointer[len('gitdir:'):].strip()).resolve()
if resolved.is_dir():
git_dir = resolved
else:
return None
else:
return None
except (OSError, NotADirectoryError):
return None
# Read branch directly from .git/HEAD (no subprocess).
branch = ''
try:
head_text = (git_dir / 'HEAD').read_text(encoding='utf-8', errors='replace').strip()
if head_text.startswith('ref: refs/heads/'):
branch = head_text[len('ref: refs/heads/'):]
elif head_text.startswith('ref: '):
branch = head_text[len('ref: '):]
# else: detached HEAD — branch stays ''
except (OSError, NotADirectoryError):
pass
# Remote URL from .git/config — parse [remote "origin"] url line.
remote_url = None
try:
config_text = (git_dir / 'config').read_text(encoding='utf-8', errors='replace')
in_origin = False
for line in config_text.splitlines():
stripped = line.strip()
if stripped == '[remote "origin"]':
in_origin = True
elif stripped.startswith('['):
in_origin = False
elif in_origin and stripped.startswith('url') and '=' in stripped:
remote_url = stripped.split('=', 1)[1].strip()
break
except (OSError, NotADirectoryError):
pass
# Single subprocess: SHA + commit date in one call.
log_result = subprocess.run(
['git', '-C', str(plugin_path), 'log', '-1', '--format=%H%n%cI', 'HEAD'],
capture_output=True, capture_output=True,
text=True, text=True,
timeout=10, timeout=10,
check=True check=True
) )
sha = sha_result.stdout.strip() lines = log_result.stdout.strip().splitlines()
sha = lines[0] if lines else ''
branch_result = subprocess.run( commit_date_iso = lines[1] if len(lines) > 1 else ''
['git', '-C', str(plugin_path), 'rev-parse', '--abbrev-ref', 'HEAD'],
capture_output=True,
text=True,
timeout=10,
check=True
)
branch = branch_result.stdout.strip()
if branch == 'HEAD':
branch = ''
# Get remote URL
remote_url_result = subprocess.run(
['git', '-C', str(plugin_path), 'config', '--get', 'remote.origin.url'],
capture_output=True,
text=True,
timeout=10,
check=False
)
remote_url = remote_url_result.stdout.strip() if remote_url_result.returncode == 0 else None
# Get commit date in ISO format
date_result = subprocess.run(
['git', '-C', str(plugin_path), 'log', '-1', '--format=%cI', 'HEAD'],
capture_output=True,
text=True,
timeout=10,
check=True
)
commit_date_iso = date_result.stdout.strip()
result = { result = {
'sha': sha, 'sha': sha,
'short_sha': sha[:7] if sha else '', 'short_sha': sha[:7] if sha else '',
'branch': branch 'branch': branch,
} }
# Add remote URL if available
if remote_url: if remote_url:
result['remote_url'] = remote_url result['remote_url'] = remote_url
# Add commit date if available
if commit_date_iso: if commit_date_iso:
result['date_iso'] = commit_date_iso result['date_iso'] = commit_date_iso
result['date'] = self._iso_to_date(commit_date_iso) result['date'] = self._iso_to_date(commit_date_iso)
@@ -2390,7 +2417,7 @@ class PluginStoreManager:
if not plugin_info_remote: if not plugin_info_remote:
self.logger.warning(f"Plugin {plugin_id} not found in registry and not a git repository; cannot update automatically") self.logger.warning(f"Plugin {plugin_id} not found in registry and not a git repository; cannot update automatically")
if not repo_url: if not repo_url:
self.logger.warning(f"Plugin may have been installed via ZIP download. Try reinstalling from GitHub URL to enable updates.") self.logger.warning("Plugin may have been installed via ZIP download. Try reinstalling from GitHub URL to enable updates.")
return False return False
repo_url = plugin_info_remote.get('repo') repo_url = plugin_info_remote.get('repo')

View File

@@ -6,7 +6,6 @@ and plugin_manager for use in plugin unit tests.
""" """
from typing import Dict, Any, Optional from typing import Dict, Any, Optional
from unittest.mock import MagicMock
from PIL import Image from PIL import Image

View File

@@ -16,7 +16,7 @@ import math
import os import os
import time import time
from pathlib import Path from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple from typing import Any, List, Optional, Tuple
from PIL import Image, ImageDraw, ImageFont from PIL import Image, ImageDraw, ImageFont
@@ -236,7 +236,6 @@ class VisualTestDisplayManager:
Replicated from DisplayManager._draw_bdf_text(). Replicated from DisplayManager._draw_bdf_text().
""" """
try: try:
import freetype
if isinstance(color, list): if isinstance(color, list):
color = tuple(color) color = tuple(color)
face = font if font else self.calendar_font face = font if font else self.calendar_font

View File

@@ -6,8 +6,7 @@ Fails fast with clear error messages to prevent runtime issues.
""" """
import os import os
import logging from typing import Any, List, Optional, Tuple
from typing import Dict, Any, List, Optional, Tuple
from pathlib import Path from pathlib import Path
from src.exceptions import ConfigError, PluginError, CacheError from src.exceptions import ConfigError, PluginError, CacheError
from src.logging_config import get_logger from src.logging_config import get_logger

View File

@@ -6,7 +6,7 @@ plugin ordering, exclusions, scroll speed, and display settings.
""" """
import logging import logging
from typing import Dict, Any, List, Set, Optional from typing import Dict, Any, List, Set
from dataclasses import dataclass, field from dataclasses import dataclass, field
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)

View File

@@ -90,13 +90,11 @@ class VegasModeCoordinator:
self._interrupt_check: Optional[Callable[[], bool]] = None self._interrupt_check: Optional[Callable[[], bool]] = None
self._interrupt_check_interval: int = 10 # Check every N frames self._interrupt_check_interval: int = 10 # Check every N frames
# Plugin update tick for keeping data fresh during Vegas mode # Plugin update callback — fired from a background thread inside the loop
self._update_tick: Optional[Callable[[], Optional[List[str]]]] = None # so the main loop's _tick_plugin_updates() finds nothing due when Vegas
self._update_tick_interval: float = 1.0 # Tick every 1 second # returns, eliminating the inter-iteration frozen-frame gap.
self._update_thread: Optional[threading.Thread] = None self._update_callback: Optional[Callable[[], None]] = None
self._update_results: Optional[List[str]] = None self._update_tick_running: bool = False
self._update_results_lock = threading.Lock()
self._last_update_tick_time: float = 0.0
# Config update tracking # Config update tracking
self._config_version = 0 self._config_version = 0
@@ -139,6 +137,25 @@ class VegasModeCoordinator:
"""Check if Vegas mode is currently running.""" """Check if Vegas mode is currently running."""
return self._is_active return self._is_active
def set_sync_manager(self, sync_manager, follower_position: str = "left") -> None:
"""
Attach a DisplaySyncManager so Vegas mode sends the follower's portion
of the ticker to the second display on every rendered frame.
Args:
sync_manager: DisplaySyncManager instance, or None to disable sync
follower_position: "left" (default) or "right" — physical position of
the follower display relative to the leader
"""
if self.render_pipeline:
# Don't expose a standalone (no-op) manager to the pipeline — treat it as None
if sync_manager is not None and hasattr(sync_manager, 'role'):
from src.common.sync_manager import SyncRole
if sync_manager.role == SyncRole.STANDALONE:
sync_manager = None
self.render_pipeline.sync_manager = sync_manager
self.render_pipeline.sync_follower_left = (follower_position == "left")
def set_live_priority_checker(self, checker: Callable[[], Optional[str]]) -> None: def set_live_priority_checker(self, checker: Callable[[], Optional[str]]) -> None:
""" """
Set the callback for checking live priority content. Set the callback for checking live priority content.
@@ -166,24 +183,19 @@ class VegasModeCoordinator:
self._interrupt_check = checker self._interrupt_check = checker
self._interrupt_check_interval = max(1, check_interval) self._interrupt_check_interval = max(1, check_interval)
def set_update_tick( def set_update_callback(self, callback: Callable[[], None]) -> None:
self,
callback: Callable[[], Optional[List[str]]],
interval: float = 1.0
) -> None:
""" """
Set the callback for periodic plugin update ticking during Vegas mode. Set a callback for running plugin updates from inside the Vegas loop.
This keeps plugin data fresh while the Vegas render loop is running. Fired in a daemon background thread every ~4 s so plugin data stays
The callback should run scheduled plugin updates and return a list of fresh without blocking the render loop. The main loop's
plugin IDs that were actually updated, or None/empty if no updates occurred. _tick_plugin_updates() then finds all intervals already satisfied and
returns immediately, collapsing the inter-iteration gap to <1 ms.
Args: Args:
callback: Callable that returns list of updated plugin IDs or None callback: Callable with no arguments (typically _tick_plugin_updates)
interval: Seconds between update tick calls (default 1.0)
""" """
self._update_tick = callback self._update_callback = callback
self._update_tick_interval = max(0.5, interval)
def start(self) -> bool: def start(self) -> bool:
""" """
@@ -237,9 +249,6 @@ class VegasModeCoordinator:
self.stats['total_runtime_seconds'] += time.time() - self._start_time self.stats['total_runtime_seconds'] += time.time() - self._start_time
self._start_time = None self._start_time = None
# Wait for in-flight background update before tearing down state
self._drain_update_thread()
# Cleanup components # Cleanup components
self.render_pipeline.reset() self.render_pipeline.reset()
self.stream_manager.reset() self.stream_manager.reset()
@@ -335,83 +344,101 @@ class VegasModeCoordinator:
last_fps_log_time = start_time last_fps_log_time = start_time
fps_frame_count = 0 fps_frame_count = 0
self._last_update_tick_time = start_time
logger.info("Starting Vegas iteration for %.1fs", duration) logger.info("Starting Vegas iteration for %.1fs", duration)
try: while True:
while True: # Check for STATIC mode plugin that should pause scroll
# Check for STATIC mode plugin that should pause scroll static_plugin = self._check_static_plugin_trigger()
static_plugin = self._check_static_plugin_trigger() if static_plugin:
if static_plugin: if not self._handle_static_pause(static_plugin):
if not self._handle_static_pause(static_plugin): # Static pause was interrupted
# Static pause was interrupted return False
# After static pause, skip this segment and continue
self.stream_manager.get_next_segment() # Consume the segment
continue
# Run frame
if not self.run_frame():
# Check why we stopped
with self._state_lock:
if self._should_stop:
return False
if self._is_paused:
# Paused for live priority - let caller handle
return False return False
# After static pause, skip this segment and continue
self.stream_manager.get_next_segment() # Consume the segment
continue
# Run frame # Sleep for frame interval
if not self.run_frame(): time.sleep(frame_interval)
# Check why we stopped
with self._state_lock:
if self._should_stop:
return False
if self._is_paused:
# Paused for live priority - let caller handle
return False
# Sleep for frame interval # Increment frame count and check for interrupt periodically
time.sleep(frame_interval) frame_count += 1
fps_frame_count += 1
# Increment frame count and check for interrupt periodically # Periodic FPS logging
frame_count += 1 current_time = time.time()
fps_frame_count += 1 if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d)",
fps, self.vegas_config.target_fps, fps_frame_count
)
last_fps_log_time = current_time
fps_frame_count = 0
# Periodic FPS logging if (self._interrupt_check and
current_time = time.time() frame_count % self._interrupt_check_interval == 0):
if current_time - last_fps_log_time >= fps_log_interval: try:
fps = fps_frame_count / (current_time - last_fps_log_time) if self._interrupt_check():
logger.info( logger.debug(
"Vegas FPS: %.1f (target: %d, frames: %d)", "Vegas interrupted by callback after %d frames",
fps, self.vegas_config.target_fps, fps_frame_count frame_count
) )
last_fps_log_time = current_time return False
fps_frame_count = 0 except Exception:
# Log but don't let interrupt check errors stop Vegas
logger.exception("Interrupt check failed")
# Periodic plugin update tick to keep data fresh (non-blocking) # Fire plugin update tick in a background thread every ~4 s.
self._drive_background_updates() # Running it here (rather than only between iterations) means the
# main loop's _tick_plugin_updates() finds all intervals already
if (self._interrupt_check and # satisfied on return, so the inter-iteration gap is <1 ms and the
frame_count % self._interrupt_check_interval == 0): # display never shows a frozen frame between iterations.
_UPDATE_TICK_FRAMES = max(1, int(self.vegas_config.target_fps * 4)) # every 4 s regardless of FPS
if (self._update_callback and
frame_count % _UPDATE_TICK_FRAMES == 0 and
not self._update_tick_running):
self._update_tick_running = True
def _run_tick(cb=self._update_callback):
try: try:
if self._interrupt_check(): cb()
logger.debug( finally:
"Vegas interrupted by callback after %d frames", self._update_tick_running = False
frame_count threading.Thread(
) target=_run_tick, daemon=True, name="vegas-plugin-tick"
return False ).start()
except Exception:
# Log but don't let interrupt check errors stop Vegas
logger.exception("Interrupt check failed")
# Check elapsed time # Check elapsed time
elapsed = time.time() - start_time elapsed = time.time() - start_time
if elapsed >= duration: if elapsed >= duration:
break break
# Check for cycle completion # NOTE: do NOT break on is_cycle_complete() here.
if self.render_pipeline.is_cycle_complete(): # When multi-display sync is active, breaking exits run_iteration()
break # which causes a 2-3s delay before start_new_cycle() is called on
# the next run_iteration(). During that gap the scroll advances into
# the pre-roll zone, then start_new_cycle() resets it — producing a
# second visible jump on the follower display ~2.5s after the first.
#
# Instead, run_frame() handles cycle completion directly (it calls
# start_new_cycle() in the very next frame, 8ms later), collapsing
# the two events into a single clean transition.
#
# Without sync, the iteration now runs to its full duration and may
# cycle content multiple times within one iteration — acceptable for
# a continuous ticker.
logger.info("Vegas iteration completed after %.1fs", time.time() - start_time) logger.info("Vegas iteration completed after %.1fs", time.time() - start_time)
return True return True
finally:
# Ensure background update thread finishes before the main loop
# resumes its own _tick_plugin_updates() calls, preventing concurrent
# run_scheduled_updates() execution.
self._drain_update_thread()
def _check_live_priority(self) -> bool: def _check_live_priority(self) -> bool:
""" """
@@ -500,71 +527,6 @@ class VegasModeCoordinator:
if self._pending_config is None: if self._pending_config is None:
self._pending_config_update = False self._pending_config_update = False
def _run_update_tick_background(self) -> None:
"""Run the plugin update tick in a background thread.
Stores results for the render loop to pick up on its next iteration,
so the scroll never blocks on API calls.
"""
try:
updated_plugins = self._update_tick()
if updated_plugins:
with self._update_results_lock:
# Accumulate rather than replace to avoid losing notifications
# if a previous result hasn't been picked up yet
if self._update_results is None:
self._update_results = updated_plugins
else:
self._update_results.extend(updated_plugins)
except Exception:
logger.exception("Background plugin update tick failed")
def _drain_update_thread(self, timeout: float = 2.0) -> None:
"""Wait for any in-flight background update thread to finish.
Called when transitioning out of Vegas mode so the main-loop
``_tick_plugin_updates`` call doesn't race with a still-running
background thread.
"""
if self._update_thread is not None and self._update_thread.is_alive():
self._update_thread.join(timeout=timeout)
if self._update_thread.is_alive():
logger.warning(
"Background update thread did not finish within %.1fs", timeout
)
def _drive_background_updates(self) -> None:
"""Collect finished background update results and launch new ticks.
Safe to call from both the main render loop and the static-pause
wait loop so that plugin data stays fresh regardless of which
code path is active.
"""
# 1. Collect results from a previously completed background update
with self._update_results_lock:
ready_results = self._update_results
self._update_results = None
if ready_results:
for pid in ready_results:
self.mark_plugin_updated(pid)
# 2. Kick off a new background update if interval elapsed and none running
current_time = time.time()
if (self._update_tick and
current_time - self._last_update_tick_time >= self._update_tick_interval):
thread_alive = (
self._update_thread is not None
and self._update_thread.is_alive()
)
if not thread_alive:
self._last_update_tick_time = current_time
self._update_thread = threading.Thread(
target=self._run_update_tick_background,
daemon=True,
name="vegas-update-tick",
)
self._update_thread.start()
def mark_plugin_updated(self, plugin_id: str) -> None: def mark_plugin_updated(self, plugin_id: str) -> None:
""" """
Notify that a plugin's data has been updated. Notify that a plugin's data has been updated.
@@ -683,8 +645,10 @@ class VegasModeCoordinator:
logger.info("Static pause interrupted by live priority") logger.info("Static pause interrupted by live priority")
return False return False
# Keep plugin data fresh during static pause # Yield immediately if multi-display follower mode becomes active
self._drive_background_updates() if self._interrupt_check and self._interrupt_check():
logger.info("Static pause interrupted by sync follower mode")
return False
# Sleep in small increments to remain responsive # Sleep in small increments to remain responsive
time.sleep(0.1) time.sleep(0.1)

View File

@@ -329,50 +329,51 @@ class PluginAdapter:
# Save display state to restore after # Save display state to restore after
original_image = self.display_manager.image.copy() original_image = self.display_manager.image.copy()
# Method 1: Try _create_scrolling_display (stocks pattern) with self.display_manager.capture_mode():
if hasattr(plugin, '_create_scrolling_display'): # Method 1: Try _create_scrolling_display (stocks pattern)
logger.info( if hasattr(plugin, '_create_scrolling_display'):
"[%s] Triggering via _create_scrolling_display()",
plugin_id
)
try:
plugin._create_scrolling_display()
cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] _create_scrolling_display() SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
return cached_image
except (AttributeError, TypeError, ValueError, OSError):
logger.exception(
"[%s] _create_scrolling_display() failed", plugin_id
)
# Method 2: Try display(force_clear=True) which typically builds scroll content
if hasattr(plugin, 'display'):
logger.info(
"[%s] Triggering via display(force_clear=True)",
plugin_id
)
try:
self.display_manager.clear()
plugin.display(force_clear=True)
cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] display(force_clear=True) SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
return cached_image
logger.info( logger.info(
"[%s] display(force_clear=True) did not populate cached_image", "[%s] Triggering via _create_scrolling_display()",
plugin_id plugin_id
) )
except (AttributeError, TypeError, ValueError, OSError): try:
logger.exception( plugin._create_scrolling_display()
"[%s] display(force_clear=True) failed", plugin_id cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] _create_scrolling_display() SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
return cached_image
except (AttributeError, TypeError, ValueError, OSError):
logger.exception(
"[%s] _create_scrolling_display() failed", plugin_id
)
# Method 2: Try display(force_clear=True) which typically builds scroll content
if hasattr(plugin, 'display'):
logger.info(
"[%s] Triggering via display(force_clear=True)",
plugin_id
) )
try:
self.display_manager.clear()
plugin.display(force_clear=True)
cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] display(force_clear=True) SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
return cached_image
logger.info(
"[%s] display(force_clear=True) did not populate cached_image",
plugin_id
)
except (AttributeError, TypeError, ValueError, OSError):
logger.exception(
"[%s] display(force_clear=True) failed", plugin_id
)
logger.info( logger.info(
"[%s] Could not trigger scroll content generation", "[%s] Could not trigger scroll content generation",
@@ -408,10 +409,7 @@ class PluginAdapter:
original_image = self.display_manager.image.copy() original_image = self.display_manager.image.copy()
logger.info("[%s] Fallback: saved original display state", plugin_id) logger.info("[%s] Fallback: saved original display state", plugin_id)
# Lightweight in-memory data refresh before capturing. # Ensure plugin has fresh data before capturing
# Full update() is intentionally skipped here — the background
# update tick in the Vegas coordinator handles periodic API
# refreshes so we don't block the content-fetch thread.
has_update_data = hasattr(plugin, 'update_data') has_update_data = hasattr(plugin, 'update_data')
logger.info("[%s] Fallback: has update_data=%s", plugin_id, has_update_data) logger.info("[%s] Fallback: has update_data=%s", plugin_id, has_update_data)
if has_update_data: if has_update_data:
@@ -421,21 +419,24 @@ class PluginAdapter:
except (AttributeError, RuntimeError, OSError): except (AttributeError, RuntimeError, OSError):
logger.exception("[%s] Fallback: update_data() failed", plugin_id) logger.exception("[%s] Fallback: update_data() failed", plugin_id)
# Clear and call plugin display # Clear and call plugin display — use capture_mode to suppress hardware writes
self.display_manager.clear() # that plugins may trigger internally via update_display().
logger.info("[%s] Fallback: display cleared, calling display()", plugin_id) with self.display_manager.capture_mode():
self.display_manager.clear()
logger.info("[%s] Fallback: display cleared, calling display()", plugin_id)
# First try without force_clear (some plugins behave better this way) # First try without force_clear (some plugins behave better this way)
try: try:
plugin.display() plugin.display()
logger.info("[%s] Fallback: display() called successfully", plugin_id) logger.info("[%s] Fallback: display() called successfully", plugin_id)
except TypeError: except TypeError:
# Plugin may require force_clear argument # Plugin may require force_clear argument
logger.info("[%s] Fallback: display() failed, trying with force_clear=True", plugin_id) logger.info("[%s] Fallback: display() failed, trying with force_clear=True", plugin_id)
plugin.display(force_clear=True) plugin.display(force_clear=True)
# Capture the result
captured = self.display_manager.image.copy()
# Capture the result
captured = self.display_manager.image.copy()
logger.info( logger.info(
"[%s] Fallback: captured frame %dx%d, mode=%s", "[%s] Fallback: captured frame %dx%d, mode=%s",
plugin_id, captured.width, captured.height, captured.mode plugin_id, captured.width, captured.height, captured.mode
@@ -454,9 +455,10 @@ class PluginAdapter:
plugin_id plugin_id
) )
# Try once more with force_clear=True # Try once more with force_clear=True
self.display_manager.clear() with self.display_manager.capture_mode():
plugin.display(force_clear=True) self.display_manager.clear()
captured = self.display_manager.image.copy() plugin.display(force_clear=True)
captured = self.display_manager.image.copy()
is_blank, bright_ratio = self._is_blank_image(captured, return_ratio=True) is_blank, bright_ratio = self._is_blank_image(captured, return_ratio=True)
logger.info( logger.info(
@@ -585,28 +587,6 @@ class PluginAdapter:
else: else:
self._content_cache.clear() self._content_cache.clear()
def invalidate_plugin_scroll_cache(self, plugin: 'BasePlugin', plugin_id: str) -> None:
"""
Clear a plugin's scroll_helper cache so Vegas re-fetches fresh visuals.
Uses scroll_helper.clear_cache() to reset all cached state (cached_image,
cached_array, total_scroll_width, scroll_position, etc.) — not just the
image. Without this, plugins that use scroll_helper (stocks, news,
odds-ticker, etc.) would keep serving stale scroll images even after
their data refreshes.
Args:
plugin: Plugin instance
plugin_id: Plugin identifier
"""
scroll_helper = getattr(plugin, 'scroll_helper', None)
if scroll_helper is None:
return
if getattr(scroll_helper, 'cached_image', None) is not None:
scroll_helper.clear_cache()
logger.debug("[%s] Cleared scroll_helper cache", plugin_id)
def get_content_type(self, plugin: 'BasePlugin', plugin_id: str) -> str: def get_content_type(self, plugin: 'BasePlugin', plugin_id: str) -> str:
""" """
Get the type of content a plugin provides. Get the type of content a plugin provides.

View File

@@ -11,11 +11,10 @@ import threading
from collections import deque from collections import deque
from typing import Optional, List, Any, Dict, Deque, TYPE_CHECKING from typing import Optional, List, Any, Dict, Deque, TYPE_CHECKING
from PIL import Image from PIL import Image
import numpy as np
from src.common.scroll_helper import ScrollHelper from src.common.scroll_helper import ScrollHelper
from src.vegas_mode.config import VegasModeConfig from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.stream_manager import StreamManager, ContentSegment from src.vegas_mode.stream_manager import StreamManager
if TYPE_CHECKING: if TYPE_CHECKING:
pass pass
@@ -52,6 +51,10 @@ class RenderPipeline:
self.config = config self.config = config
self.display_manager = display_manager self.display_manager = display_manager
self.stream_manager = stream_manager self.stream_manager = stream_manager
self.sync_manager = None # Optional DisplaySyncManager — set by coordinator
self.sync_follower_left = True # True = follower is LEFT of leader (default)
self._sync_send_interval = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
self._last_sync_send = 0.0
# Display dimensions (handle both property and method access patterns) # Display dimensions (handle both property and method access patterns)
self.display_width = ( self.display_width = (
@@ -202,8 +205,26 @@ class RenderPipeline:
# Update scroll position # Update scroll position
self.scroll_helper.update_scroll_position() self.scroll_helper.update_scroll_position()
# Check if cycle is complete # Determine if the cycle is done.
if self.scroll_helper.is_scroll_complete(): #
# scroll_helper considers a cycle complete only after
# total_distance_scrolled >= total_scroll_width + display_width.
# That extra display_width of travel causes a "wrap-around" phase
# where scroll_position resets to ~0 and the first plugin's content
# re-enters from the right — the user sees this 2-3 s of re-entry
# as "a plugin partially displaying before the next one starts."
#
# We end the cycle as soon as total_distance_scrolled reaches
# total_scroll_width (the wrap-around point), before any second-pass
# content becomes visible. The scroll_helper's own is_scroll_complete()
# check is kept as a fallback for any edge-cases where that threshold
# is never hit.
at_wrap_point = (
not self._cycle_complete and
self.scroll_helper.total_distance_scrolled >= self.scroll_helper.total_scroll_width
)
if at_wrap_point or self.scroll_helper.is_scroll_complete():
if not self._cycle_complete: if not self._cycle_complete:
self._cycle_complete = True self._cycle_complete = True
self.stats['scroll_cycles'] += 1 self.stats['scroll_cycles'] += 1
@@ -211,6 +232,17 @@ class RenderPipeline:
"Scroll cycle complete after %.1fs", "Scroll cycle complete after %.1fs",
time.time() - self._cycle_start_time time.time() - self._cycle_start_time
) )
# Push blank immediately so the hardware never shows any
# post-wrap content while the coordinator recomposes the
# next cycle (~100 ms).
try:
from PIL import Image as _Image
blank = _Image.new('RGB', (self.display_width, self.display_height))
self.display_manager.image = blank
self.display_manager.update_display()
except Exception:
logger.exception("Failed to write blank frame to display at cycle end")
return True # Cycle done; coordinator starts new cycle next frame
# Get visible portion # Get visible portion
visible_frame = self.scroll_helper.get_visible_portion() visible_frame = self.scroll_helper.get_visible_portion()
@@ -221,6 +253,15 @@ class RenderPipeline:
self.display_manager.image = visible_frame self.display_manager.image = visible_frame
self.display_manager.update_display() self.display_manager.update_display()
# Multi-display sync: send scroll position to follower.
# The follower renders from its own cached_array (kept identical to the
# leader's via TCP image transfer at each new_cycle) at scroll_x ± display_width.
if self.sync_manager:
now = time.time()
if now - self._last_sync_send >= self._sync_send_interval:
self._last_sync_send = now
self.sync_manager.send_scroll_x(self.scroll_helper.scroll_position)
# Update scrolling state # Update scrolling state
self.display_manager.set_scrolling_state(True) self.display_manager.set_scrolling_state(True)
@@ -260,33 +301,38 @@ class RenderPipeline:
if self._cycle_complete: if self._cycle_complete:
return True return True
# When multi-display sync is active, defer mid-cycle hot swaps until the
# cycle ends naturally. Hot swaps block the render loop for 15-30ms while
# the image is rebuilt, causing a freeze+jump that the follower perceives
# as a speed-up. Deferring to cycle boundaries keeps transitions clean.
# Staging buffer content is still pre-loaded; it just applies at cycle end.
if self.sync_manager is not None:
return False
# Check if we need more content in the buffer # Check if we need more content in the buffer
buffer_status = self.stream_manager.get_buffer_status() buffer_status = self.stream_manager.get_buffer_status()
if buffer_status['staging_count'] > 0: if buffer_status['staging_count'] > 0:
return True return True
# Trigger recompose when pending updates affect visible segments
if self.stream_manager.has_pending_updates_for_visible_segments():
return True
return False return False
def hot_swap_content(self) -> bool: def hot_swap_content(self) -> bool:
""" """
Hot-swap to new composed content. Hot-swap to new composed content.
Called when staging buffer has updated content or pending updates exist. Called when staging buffer has updated content.
Preserves scroll position for mid-cycle updates to prevent visual jumps. Swaps atomically to prevent visual glitches.
Returns: Returns:
True if swap occurred True if swap occurred
""" """
try: try:
# Save scroll position for mid-cycle updates # Snapshot position before swap so we can reposition after.
saved_position = self.scroll_helper.scroll_position # The new image has completely different content — if scroll_position
saved_total_distance = self.scroll_helper.total_distance_scrolled # is left unchanged it lands at an arbitrary mid-content point in the
saved_total_width = max(1, self.scroll_helper.total_scroll_width) # new image, causing a visible jump on both displays.
was_mid_cycle = not self._cycle_complete old_width = self.scroll_helper.total_scroll_width
old_pos = self.scroll_helper.scroll_position
# Process any pending updates # Process any pending updates
self.stream_manager.process_updates() self.stream_manager.process_updates()
@@ -294,20 +340,24 @@ class RenderPipeline:
# Recompose with updated content # Recompose with updated content
if self.compose_scroll_content(): if self.compose_scroll_content():
# Map scroll position proportionally into the new image width so
# we resume at the same relative progress through the content.
# This keeps the visual tempo consistent and avoids the jump that
# occurred when old scroll_position landed arbitrarily in new image.
new_width = self.scroll_helper.total_scroll_width
if old_width > 0 and new_width > 0:
ratio = (old_pos % old_width) / old_width
self.scroll_helper.scroll_position = ratio * new_width
else:
self.scroll_helper.scroll_position = 0.0
self.stats['hot_swaps'] += 1 self.stats['hot_swaps'] += 1
# Restore scroll position for mid-cycle updates so the logger.debug(
# scroll continues from where it was instead of jumping to 0 "Hot-swap completed: scroll repositioned %.0f%.0f (%.1f%% of new %dpx image)",
if was_mid_cycle: old_pos, self.scroll_helper.scroll_position,
new_total_width = max(1, self.scroll_helper.total_scroll_width) (self.scroll_helper.scroll_position / new_width * 100) if new_width else 0,
progress_ratio = min(saved_total_distance / saved_total_width, 0.999) new_width,
self.scroll_helper.total_distance_scrolled = progress_ratio * new_total_width )
self.scroll_helper.scroll_position = min(
saved_position,
float(new_total_width - 1)
)
self.scroll_helper.scroll_complete = False
self._cycle_complete = False
logger.debug("Hot-swap completed (mid_cycle_restore=%s)", was_mid_cycle)
return True return True
return False return False
@@ -342,7 +392,29 @@ class RenderPipeline:
return False return False
# Compose new scroll content # Compose new scroll content
return self.compose_scroll_content() result = self.compose_scroll_content()
if result and self.sync_manager:
# When sync is active, start the leader at display_width instead of 0.
# This skips the initial black gap so the leader immediately shows content.
# The follower starts at position 0 (the gap) which looks like a clean
# blank transition rather than near-end content wrapping around.
self.scroll_helper.scroll_position = float(self.display_width)
if result and self.sync_manager:
# Signal follower that a new cycle started (triggers its own rebuild)
self.sync_manager.send_new_cycle()
# Push the actual scroll image over TCP so follower has identical pixels.
# Done in a background thread to not block the render loop (~15ms transfer).
if self.scroll_helper.cached_image is not None:
import threading as _t
_t.Thread(
target=self.sync_manager.send_scroll_image,
args=(self.scroll_helper.cached_image,),
daemon=True, name="sync-image-push"
).start()
return result
def get_current_scroll_info(self) -> Dict[str, Any]: def get_current_scroll_info(self) -> Dict[str, Any]:
"""Get current scroll state information.""" """Get current scroll state information."""

View File

@@ -14,7 +14,7 @@ Supports three display modes:
import logging import logging
import threading import threading
import time import time
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING from typing import Optional, List, Dict, Any, Deque, TYPE_CHECKING
from collections import deque from collections import deque
from dataclasses import dataclass, field from dataclasses import dataclass, field
from PIL import Image from PIL import Image
@@ -24,7 +24,6 @@ from src.vegas_mode.plugin_adapter import PluginAdapter
from src.plugin_system.base_plugin import VegasDisplayMode from src.plugin_system.base_plugin import VegasDisplayMode
if TYPE_CHECKING: if TYPE_CHECKING:
from src.plugin_system.base_plugin import BasePlugin
from src.plugin_system.plugin_manager import PluginManager from src.plugin_system.plugin_manager import PluginManager
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)

View File

@@ -5,11 +5,11 @@ Provides consistent API response formatting across all endpoints.
""" """
import time import time
from typing import Any, Optional, Dict, Tuple, Union from typing import Any, Optional, Dict, Tuple
from flask import jsonify, request from flask import jsonify, request
from src.web_interface.error_handler import create_error_response, create_success_response from src.web_interface.error_handler import create_error_response, create_success_response
from src.web_interface.errors import ErrorCode, ErrorCategory from src.web_interface.errors import ErrorCode
def success_response( def success_response(

View File

@@ -5,7 +5,6 @@ Provides decorators and helpers for consistent error handling across API endpoin
""" """
import functools import functools
import traceback
from typing import Callable, Any, Optional from typing import Callable, Any, Optional
from flask import jsonify from flask import jsonify

View File

@@ -3,7 +3,6 @@ Input validation utilities for the web interface.
Provides validation functions for user inputs to prevent XSS, invalid data, and security issues. Provides validation functions for user inputs to prevent XSS, invalid data, and security issues.
""" """
import re import re
import os
from typing import Optional, Tuple, List from typing import Optional, Tuple, List
from urllib.parse import urlparse from urllib.parse import urlparse
from pathlib import Path from pathlib import Path
@@ -56,7 +55,7 @@ def validate_image_url(url: str) -> Tuple[bool, Optional[str]]:
parsed = urlparse(url) parsed = urlparse(url)
allowed_protocols = ['http', 'https'] allowed_protocols = ['http', 'https']
if parsed.scheme not in allowed_protocols: if parsed.scheme not in allowed_protocols:
return False, f"Only http:// and https:// protocols are allowed" return False, "Only http:// and https:// protocols are allowed"
return True, None return True, None
except Exception as e: except Exception as e:
return False, f"Invalid URL format: {str(e)}" return False, f"Invalid URL format: {str(e)}"

File diff suppressed because it is too large Load Diff

View File

@@ -7,7 +7,8 @@
[Unit] [Unit]
Description=LED Matrix Web Interface Service Description=LED Matrix Web Interface Service
After=network.target After=network-online.target
Wants=network-online.target
[Service] [Service]
Type=simple Type=simple

View File

@@ -7,7 +7,7 @@ Wants=network.target
Type=simple Type=simple
User=root User=root
WorkingDirectory=__PROJECT_ROOT_DIR__ WorkingDirectory=__PROJECT_ROOT_DIR__
ExecStart=/usr/bin/python3 /home/ledpi/LEDMatrix/scripts/utils/wifi_monitor_daemon.py --interval 30 ExecStart=/usr/bin/python3 __PROJECT_ROOT_DIR__/scripts/utils/wifi_monitor_daemon.py --interval 30
Restart=on-failure Restart=on-failure
RestartSec=10 RestartSec=10
StandardOutput=syslog StandardOutput=syslog

View File

@@ -1,6 +1,7 @@
[Unit] [Unit]
Description=LED Matrix Display Service Description=LED Matrix Display Service
After=network.target After=network-online.target
Wants=network-online.target
[Service] [Service]
Type=simple Type=simple

View File

@@ -5,7 +5,6 @@ Provides common fixtures for mocking core components and test setup.
""" """
import pytest import pytest
import os
import sys import sys
from pathlib import Path from pathlib import Path
from unittest.mock import Mock, MagicMock from unittest.mock import Mock, MagicMock

View File

@@ -3,7 +3,6 @@
Diagnostic script to examine NBA API data structure and identify the missing 'id' field issue. Diagnostic script to examine NBA API data structure and identify the missing 'id' field issue.
""" """
import requests import requests
import json
import logging import logging
from typing import Dict, Any from typing import Dict, Any

View File

@@ -8,7 +8,7 @@ import sys
import json import json
from pathlib import Path from pathlib import Path
from unittest.mock import MagicMock, Mock from unittest.mock import MagicMock, Mock
from typing import Any, Dict, Generator, Optional from typing import Any, Dict
# Add project root to path # Add project root to path
project_root = Path(__file__).parent.parent.parent project_root = Path(__file__).parent.parent.parent

View File

@@ -6,9 +6,7 @@ Provides common test functionality for all plugins.
import pytest import pytest
import json import json
from pathlib import Path
from typing import Dict, Any from typing import Dict, Any
from unittest.mock import MagicMock
from src.plugin_system.plugin_loader import PluginLoader from src.plugin_system.plugin_loader import PluginLoader
from src.plugin_system.base_plugin import BasePlugin from src.plugin_system.base_plugin import BasePlugin

View File

@@ -5,7 +5,6 @@ Verifies that the visual display manager actually renders pixels,
loads fonts, and can save snapshots. loads fonts, and can save snapshots.
""" """
import pytest
from PIL import Image from PIL import Image
from src.plugin_system.testing import VisualTestDisplayManager from src.plugin_system.testing import VisualTestDisplayManager

342
test/test_api_extractors.py Normal file
View File

@@ -0,0 +1,342 @@
"""
Tests for src/base_classes/api_extractors.py
Covers ESPNFootballExtractor, ESPNBaseballExtractor, ESPNHockeyExtractor,
SoccerAPIExtractor, and the shared _extract_common_details logic.
"""
import logging
import pytest
from src.base_classes.api_extractors import (
ESPNFootballExtractor,
ESPNBaseballExtractor,
ESPNHockeyExtractor,
SoccerAPIExtractor,
)
# ---------------------------------------------------------------------------
# Shared test data factories
# ---------------------------------------------------------------------------
def _make_espn_event(state: str = "in", home_abbr: str = "KC", away_abbr: str = "BUF",
home_score: str = "14", away_score: str = "7",
date_str: str = "2024-01-15T20:00:00Z",
include_situation: bool = False,
situation: dict | None = None,
status_detail: str = "2nd Qtr 8:42",
period: int = 2) -> dict:
"""Build a minimal ESPN-style game event dict."""
comp_status = {
"type": {
"state": state,
"shortDetail": status_detail,
"detail": status_detail,
"name": "STATUS_IN_PROGRESS",
},
"period": period,
"displayClock": "8:42",
}
comp = {
"status": comp_status,
"competitors": [
{
"homeAway": "home",
"team": {"abbreviation": home_abbr, "displayName": f"{home_abbr} Team"},
"score": home_score,
},
{
"homeAway": "away",
"team": {"abbreviation": away_abbr, "displayName": f"{away_abbr} Team"},
"score": away_score,
},
],
}
if include_situation:
comp["situation"] = situation or {}
return {
"id": "test-game-1",
"date": date_str,
"competitions": [comp],
}
def _make_logger() -> logging.Logger:
return logging.getLogger("test_extractor")
# ---------------------------------------------------------------------------
# ESPNFootballExtractor
# ---------------------------------------------------------------------------
class TestESPNFootballExtractor:
def setup_method(self):
self.extractor = ESPNFootballExtractor(_make_logger())
def test_extract_live_game_basic_fields(self):
event = _make_espn_event(state="in", home_score="14", away_score="7")
result = self.extractor.extract_game_details(event)
assert result is not None
assert result["home_abbr"] == "KC"
assert result["away_abbr"] == "BUF"
assert result["home_score"] == "14"
assert result["away_score"] == "7"
assert result["is_live"] is True
assert result["is_final"] is False
assert result["is_upcoming"] is False
def test_extract_final_game(self):
event = _make_espn_event(state="post")
result = self.extractor.extract_game_details(event)
assert result is not None
assert result["is_final"] is True
assert result["is_live"] is False
def test_extract_upcoming_game(self):
event = _make_espn_event(state="pre")
result = self.extractor.extract_game_details(event)
assert result is not None
assert result["is_upcoming"] is True
def test_sport_specific_fields_default_when_pregame(self):
event = _make_espn_event(state="pre")
fields = self.extractor.get_sport_specific_fields(event)
assert "down" in fields
assert "distance" in fields
assert "possession" in fields
assert "is_redzone" in fields
assert fields["is_redzone"] is False
def test_sport_specific_fields_live_with_situation(self):
situation = {
"down": 3,
"distance": 7,
"possession": "KC",
"isRedZone": True,
"homeTimeouts": 2,
"awayTimeouts": 1,
}
event = _make_espn_event(state="in", include_situation=True, situation=situation)
fields = self.extractor.get_sport_specific_fields(event)
assert fields["down"] == 3
assert fields["distance"] == 7
assert fields["is_redzone"] is True
assert fields["home_timeouts"] == 2
assert fields["away_timeouts"] == 1
def test_scoring_event_detected(self):
# situation must be non-empty (truthy) for the live block to execute
situation = {"down": 1, "distance": 10}
event = _make_espn_event(
state="in",
include_situation=True,
situation=situation,
status_detail="touchdown scored",
)
fields = self.extractor.get_sport_specific_fields(event)
assert "touchdown" in fields.get("scoring_event", "").lower()
def test_returns_none_on_empty_event(self):
assert self.extractor.extract_game_details({}) is None
def test_returns_none_when_teams_missing(self):
event = {
"id": "x",
"date": "2024-01-15T20:00:00Z",
"competitions": [
{
"status": {"type": {"state": "in", "shortDetail": "", "detail": "", "name": ""}},
"competitors": [], # no competitors
}
],
}
assert self.extractor.extract_game_details(event) is None
def test_date_z_suffix_parsed(self):
event = _make_espn_event(date_str="2024-01-15T20:00:00Z")
result = self.extractor.extract_game_details(event)
# Should not raise and should return a result
assert result is not None
def test_id_propagated(self):
event = _make_espn_event()
result = self.extractor.extract_game_details(event)
assert result["id"] == "test-game-1"
# ---------------------------------------------------------------------------
# ESPNBaseballExtractor
# ---------------------------------------------------------------------------
class TestESPNBaseballExtractor:
def setup_method(self):
self.extractor = ESPNBaseballExtractor(_make_logger())
def test_extract_live_game(self):
event = _make_espn_event(
state="in", home_abbr="NYY", away_abbr="BOS",
home_score="3", away_score="2"
)
result = self.extractor.extract_game_details(event)
assert result is not None
assert result["home_abbr"] == "NYY"
assert result["is_live"] is True
def test_baseball_sport_fields_defaults(self):
event = _make_espn_event(state="pre")
fields = self.extractor.get_sport_specific_fields(event)
assert "inning" in fields
assert "outs" in fields
assert "bases" in fields
assert "strikes" in fields
assert "balls" in fields
def test_baseball_sport_fields_live(self):
situation = {
"inning": 7,
"outs": 2,
"bases": "110",
"strikes": 2,
"balls": 3,
"pitcher": "Smith",
"batter": "Jones",
}
event = _make_espn_event(state="in", include_situation=True, situation=situation)
fields = self.extractor.get_sport_specific_fields(event)
assert fields["inning"] == 7
assert fields["outs"] == 2
assert fields["strikes"] == 2
assert fields["pitcher"] == "Smith"
def test_returns_none_on_empty(self):
assert self.extractor.extract_game_details({}) is None
# ---------------------------------------------------------------------------
# ESPNHockeyExtractor
# ---------------------------------------------------------------------------
class TestESPNHockeyExtractor:
def setup_method(self):
self.extractor = ESPNHockeyExtractor(_make_logger())
def test_extract_live_game(self):
event = _make_espn_event(
state="in", home_abbr="BOS", away_abbr="TOR",
home_score="2", away_score="1"
)
result = self.extractor.extract_game_details(event)
assert result is not None
assert result["is_live"] is True
def test_hockey_period_text_p1(self):
situation = {"isPowerPlay": False}
event = _make_espn_event(
state="in", include_situation=True, situation=situation, period=1
)
fields = self.extractor.get_sport_specific_fields(event)
assert fields["period_text"] == "P1"
def test_hockey_period_text_p2(self):
situation = {"isPowerPlay": False} # non-empty so the live block executes
event = _make_espn_event(
state="in", include_situation=True, situation=situation, period=2
)
fields = self.extractor.get_sport_specific_fields(event)
assert fields["period_text"] == "P2"
def test_hockey_period_text_p3(self):
situation = {"isPowerPlay": False}
event = _make_espn_event(
state="in", include_situation=True, situation=situation, period=3
)
fields = self.extractor.get_sport_specific_fields(event)
assert fields["period_text"] == "P3"
def test_hockey_period_text_ot(self):
situation = {"isPowerPlay": False}
event = _make_espn_event(
state="in", include_situation=True, situation=situation, period=4
)
fields = self.extractor.get_sport_specific_fields(event)
assert fields["period_text"] == "OT1"
def test_hockey_power_play(self):
situation = {"isPowerPlay": True, "homeShots": 12, "awayShots": 8}
event = _make_espn_event(state="in", include_situation=True, situation=situation, period=2)
fields = self.extractor.get_sport_specific_fields(event)
assert fields["power_play"] is True
assert fields["shots_on_goal"]["home"] == 12
assert fields["shots_on_goal"]["away"] == 8
def test_hockey_fields_defaults_pregame(self):
event = _make_espn_event(state="pre")
fields = self.extractor.get_sport_specific_fields(event)
assert "period" in fields
assert "power_play" in fields
assert fields["power_play"] is False
def test_returns_none_on_empty(self):
assert self.extractor.extract_game_details({}) is None
# ---------------------------------------------------------------------------
# SoccerAPIExtractor
# ---------------------------------------------------------------------------
class TestSoccerAPIExtractor:
def setup_method(self):
self.extractor = SoccerAPIExtractor(_make_logger())
def _make_soccer_event(self, is_live: bool = True) -> dict:
return {
"id": "soccer-1",
"home_team": {"abbreviation": "ARS", "name": "Arsenal"},
"away_team": {"abbreviation": "CHE", "name": "Chelsea"},
"home_score": "2",
"away_score": "1",
"status": "LIVE",
"is_live": is_live,
"is_final": not is_live,
"is_upcoming": False,
"half": "1",
"stoppage_time": "2",
"home_yellow_cards": 1,
"away_yellow_cards": 2,
"home_red_cards": 0,
"away_red_cards": 0,
"home_possession": 55,
"away_possession": 45,
}
def test_extract_live_game(self):
event = self._make_soccer_event(is_live=True)
result = self.extractor.extract_game_details(event)
assert result is not None
assert result["home_abbr"] == "ARS"
assert result["away_abbr"] == "CHE"
assert result["is_live"] is True
def test_sport_specific_cards(self):
event = self._make_soccer_event()
fields = self.extractor.get_sport_specific_fields(event)
assert fields["cards"]["home_yellow"] == 1
assert fields["cards"]["away_yellow"] == 2
assert fields["cards"]["home_red"] == 0
def test_sport_specific_possession(self):
event = self._make_soccer_event()
fields = self.extractor.get_sport_specific_fields(event)
assert fields["possession"]["home"] == 55
assert fields["possession"]["away"] == 45
def test_sport_specific_half(self):
event = self._make_soccer_event()
fields = self.extractor.get_sport_specific_fields(event)
assert fields["half"] == "1"
def test_scores_as_strings(self):
event = self._make_soccer_event()
result = self.extractor.extract_game_details(event)
assert result["home_score"] == "2"
assert result["away_score"] == "1"

View File

@@ -0,0 +1,299 @@
"""
Tests for src/background_data_service.py
Covers BackgroundDataService: submit_fetch_request, get_result,
is_request_complete, get_request_status, cancel_request, get_statistics,
_cleanup_completed_requests, shutdown, and get_background_service singleton.
"""
import time
import pytest
from unittest.mock import MagicMock, patch, Mock
from concurrent.futures import Future
from src.background_data_service import (
BackgroundDataService,
FetchStatus,
FetchResult,
FetchRequest,
get_background_service,
shutdown_background_service,
)
import src.background_data_service as bds_module
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def reset_global_service():
"""Ensure each test starts with no global singleton."""
shutdown_background_service()
yield
shutdown_background_service()
@pytest.fixture
def mock_cache_manager():
m = MagicMock()
m.get.return_value = None
m.set.return_value = None
m.generate_sport_cache_key.return_value = "test_key"
return m
@pytest.fixture
def service(mock_cache_manager):
svc = BackgroundDataService(mock_cache_manager, max_workers=2, request_timeout=5)
yield svc
svc.shutdown(wait=False)
# ---------------------------------------------------------------------------
# Initialisation
# ---------------------------------------------------------------------------
class TestInitialisation:
def test_stats_zeroed(self, service):
stats = service.get_statistics()
assert stats["total_requests"] == 0
assert stats["completed_requests"] == 0
assert stats["failed_requests"] == 0
def test_no_active_requests(self, service):
assert len(service.active_requests) == 0
def test_not_shutdown(self, service):
assert service._shutdown is False
# ---------------------------------------------------------------------------
# Cache hit path
# ---------------------------------------------------------------------------
class TestCacheHit:
def test_cache_hit_returns_request_id(self, service, mock_cache_manager):
mock_cache_manager.get.return_value = {"events": [{"id": "1"}]}
req_id = service.submit_fetch_request(
sport="nfl", year=2024,
url="https://example.com/nfl",
cache_key="nfl_key",
)
assert req_id is not None
# Request should be immediately complete due to cache hit
result = service.get_result(req_id)
assert result is not None
assert result.success is True
assert result.cached is True
def test_cache_hit_increments_stat(self, service, mock_cache_manager):
mock_cache_manager.get.return_value = {"events": []}
service.submit_fetch_request(sport="nba", year=2024, url="https://x.com", cache_key="k")
stats = service.get_statistics()
assert stats["cached_hits"] == 1
# ---------------------------------------------------------------------------
# Actual fetch path (mocked HTTP)
# ---------------------------------------------------------------------------
class TestFetchPath:
def _valid_payload(self) -> dict:
return {"events": [{"id": "g1"}, {"id": "g2"}]}
def test_successful_fetch_completes(self, service, mock_cache_manager):
mock_resp = Mock()
mock_resp.json.return_value = self._valid_payload()
mock_resp.raise_for_status.return_value = None
with patch.object(service.session, "get", return_value=mock_resp):
req_id = service.submit_fetch_request(
sport="nfl", year=2024,
url="https://example.com/nfl",
cache_key="nfl_test",
)
# Wait for the background thread
deadline = time.time() + 5
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.05)
result = service.get_result(req_id)
assert result is not None
assert result.success is True
assert result.data == self._valid_payload()
def test_failed_fetch_records_error(self, service, mock_cache_manager):
with patch.object(service.session, "get", side_effect=Exception("network error")):
req_id = service.submit_fetch_request(
sport="nba", year=2024,
url="https://example.com/nba",
cache_key="nba_test",
max_retries=0,
)
deadline = time.time() + 5
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.05)
result = service.get_result(req_id)
assert result is not None
assert result.success is False
assert result.error is not None
def test_cache_miss_increments_stat(self, service, mock_cache_manager):
mock_resp = Mock()
mock_resp.json.return_value = self._valid_payload()
mock_resp.raise_for_status.return_value = None
with patch.object(service.session, "get", return_value=mock_resp):
service.submit_fetch_request(
sport="nfl", year=2024, url="https://x.com", cache_key="new_key",
)
stats = service.get_statistics()
assert stats["cache_misses"] == 1
def test_callback_called_on_success(self, service, mock_cache_manager):
callback = Mock()
mock_resp = Mock()
mock_resp.json.return_value = self._valid_payload()
mock_resp.raise_for_status.return_value = None
with patch.object(service.session, "get", return_value=mock_resp):
req_id = service.submit_fetch_request(
sport="nfl", year=2024, url="https://x.com",
cache_key="cb_key", callback=callback, max_retries=0,
)
deadline = time.time() + 5
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.05)
callback.assert_called_once()
call_arg = callback.call_args[0][0]
assert isinstance(call_arg, FetchResult)
def test_data_cached_after_successful_fetch(self, service, mock_cache_manager):
mock_resp = Mock()
mock_resp.json.return_value = self._valid_payload()
mock_resp.raise_for_status.return_value = None
with patch.object(service.session, "get", return_value=mock_resp):
req_id = service.submit_fetch_request(
sport="nfl", year=2024, url="https://x.com", cache_key="cache_after_key",
)
deadline = time.time() + 5
while not service.is_request_complete(req_id) and time.time() < deadline:
time.sleep(0.05)
mock_cache_manager.set.assert_called()
# ---------------------------------------------------------------------------
# Request status / cancel
# ---------------------------------------------------------------------------
class TestRequestStatusAndCancel:
def test_unknown_request_status_is_none(self, service):
assert service.get_request_status("nonexistent") is None
def test_cancel_active_request(self, service, mock_cache_manager):
# Manually insert an active request
req = FetchRequest(
id="r1", sport="nfl", year=2024,
cache_key="k", url="https://x.com",
)
req.status = FetchStatus.PENDING
service.active_requests["r1"] = req
result = service.cancel_request("r1")
assert result is True
assert "r1" not in service.active_requests
def test_cancel_nonexistent_request(self, service):
assert service.cancel_request("does-not-exist") is False
def test_is_request_complete_false_for_active(self, service, mock_cache_manager):
req = FetchRequest(
id="r2", sport="mlb", year=2024,
cache_key="k2", url="https://x.com",
)
service.active_requests["r2"] = req
assert service.is_request_complete("r2") is False
def test_is_request_complete_true_for_done(self, service):
result = FetchResult(request_id="r3", success=True)
service.completed_requests["r3"] = result
assert service.is_request_complete("r3") is True
def test_get_result_returns_none_for_unknown(self, service):
assert service.get_result("unknown") is None
# ---------------------------------------------------------------------------
# Shutdown
# ---------------------------------------------------------------------------
class TestShutdown:
def test_shutdown_sets_flag(self, service):
service.shutdown(wait=False)
assert service._shutdown is True
def test_submit_after_shutdown_raises(self, service, mock_cache_manager):
service.shutdown(wait=False)
with pytest.raises(RuntimeError, match="shutting down"):
service.submit_fetch_request(
sport="nfl", year=2024, url="https://x.com", cache_key="k"
)
# ---------------------------------------------------------------------------
# Cleanup
# ---------------------------------------------------------------------------
class TestCleanup:
def test_cleanup_removes_old_requests(self, service):
old_result = FetchResult(request_id="old", success=True)
old_result.completed_at = time.time() - 7200 # 2 hours ago
service.completed_requests["old"] = old_result
service._last_completed_requests_cleanup = 0 # force cleanup
removed = service._cleanup_completed_requests(force=True)
assert removed >= 1
assert "old" not in service.completed_requests
def test_cleanup_respects_interval(self, service):
old_result = FetchResult(request_id="r", success=True)
old_result.completed_at = time.time() - 7200
service.completed_requests["r"] = old_result
# Cleanup interval not passed, should skip
service._last_completed_requests_cleanup = time.time()
removed = service._cleanup_completed_requests(force=False)
assert removed == 0
def test_size_limit_enforcement(self, service):
service._max_completed_requests = 3
for i in range(5):
result = FetchResult(request_id=str(i), success=True)
result.completed_at = time.time() - (5 - i) * 100 # oldest first
service.completed_requests[str(i)] = result
service._last_completed_requests_cleanup = 0
service._cleanup_completed_requests(force=True)
assert len(service.completed_requests) <= 3
# ---------------------------------------------------------------------------
# Singleton get_background_service
# ---------------------------------------------------------------------------
class TestGetBackgroundService:
def test_first_call_requires_cache_manager(self):
with pytest.raises(ValueError, match="cache_manager is required"):
get_background_service()
def test_creates_singleton(self, mock_cache_manager):
svc1 = get_background_service(mock_cache_manager)
svc2 = get_background_service()
assert svc1 is svc2
def test_shutdown_clears_singleton(self, mock_cache_manager):
get_background_service(mock_cache_manager)
shutdown_background_service()
with pytest.raises(ValueError):
get_background_service()

Some files were not shown because too many files have changed in this diff Show More