18 Commits

Author SHA1 Message Date
Chuck
eba2d4a711 docs: address CodeRabbit review comments on #306
Reviewed all 12 CodeRabbit comments on PR #306, verified each against
the current code, and fixed the 11 valid ones. The 12th finding is a
real code bug (cache_manager.delete() calls in api_helper.py and
resource_monitor.py) that's already in the planned follow-up code-fix
PR, so it stays out of this docs PR.

Fixed:

.cursor/plugins_guide.md, .cursor/README.md, .cursorrules
- I claimed "there is no --emulator flag" in 3 places. Verified in
  run.py:19-20 that the -e/--emulator flag is defined and functional
  (it sets os.environ["EMULATOR"]="true" before the display imports).
  Other docs I didn't touch (.cursor/plugin_templates/QUICK_START.md,
  docs/PLUGIN_DEVELOPMENT_GUIDE.md) already use the flag correctly.
  Replaced all 3 wrong statements with accurate guidance that
  both forms work and explains the CLI flag's relationship to the
  env var.

.cursorrules, docs/GETTING_STARTED.md, docs/WEB_INTERFACE_GUIDE.md,
docs/PLUGIN_DEVELOPMENT_GUIDE.md
- Four places claimed "the plugin loader also falls back to plugins/".
  Verified that PluginManager.discover_plugins()
  (src/plugin_system/plugin_manager.py:154) only scans the
  configured directory — no fallback. The fallback to plugins/
  exists only in two narrower places: store_manager.py:1700-1718
  (store install/update/uninstall operations) and
  schema_manager.py:70-80 (schema lookup for the web UI form
  generator). Rewrote all four mentions with the precise scope.
  Added a recommendation to set plugin_system.plugins_directory
  to "plugins" for the smoothest dev workflow with
  dev_plugin_setup.sh symlinks.

docs/FONT_MANAGER.md
- The "Status" warning told plugin authors to use
  display_manager.font_manager.resolve_font(...) as a workaround for
  loading plugin fonts. Verified in src/font_manager.py that
  resolve_font() takes a family name, not a file path — so the
  workaround as written doesn't actually work. Rewrote to tell
  authors to load the font directly with PIL or freetype-py in their
  plugin.
- The same section said "the user-facing font override system in the
  Fonts tab still works for any element that's been registered via
  register_manager_font()". Verified in
  web_interface/blueprints/api_v3.py:5404-5428 that
  /api/v3/fonts/overrides is a placeholder implementation that
  returns empty arrays and contains "would integrate with the actual
  font system" comments — the Fonts tab does not have functional
  integration with register_manager_font() or the override system.
  Removed the false claim and added an explicit note that the tab
  is a placeholder.

docs/ADVANCED_FEATURES.md:523
- The on-demand section said REST/UI calls write a request "into the
  cache manager (display_on_demand_config key)". Wrong — verified
  via grep that api_v3.py:1622 and :1687 write to
  display_on_demand_request, and display_on_demand_config is only
  written by the controller during activation
  (display_controller.py:1195, cleared at :1221). Corrected the key
  name and added controller file:line references so future readers
  can verify.

docs/ADVANCED_FEATURES.md:803
- "Plugins using the background service" paragraph listed all
  scoreboard plugins but an orphaned " MLB (baseball)" bullet
  remained below from the old version of the section. Removed the
  orphan and added "baseball/MLB" to the inline list for clarity.

web_interface/README.md
- The POST /api/v3/system/action action list was incomplete. Verified
  in web_interface/app.py:1383,1386 that enable_autostart and
  disable_autostart are valid actions. Added both.
- The Plugin Store section was missing
  GET /api/v3/plugins/store/github-status (verified at
  api_v3.py:3296). Added it.
- The SSE line-range reference was app.py:607-615 but line 619
  contains the "Exempt SSE streams from CSRF and add rate limiting"
  block that's semantically part of the same feature. Extended the
  range to 607-619.

docs/GETTING_STARTED.md
- Rows/Columns step said "Columns: 64 or 96 (match your hardware)".
  The web UI's validation accepts any integer in 16-128. Clarified
  that 64 and 96 are the common bundled-hardware values but the
  valid range is wider.

Not addressed (out of scope for docs PR):

- .cursorrules:184 CodeRabbit comment flagged the non-existent
  cache_manager.delete() calls in src/common/api_helper.py:287 and
  src/plugin_system/resource_monitor.py:343. These are real CODE
  bugs, not doc bugs, and they're the first item in the planned
  post-docs-refresh code-cleanup PR (see
  /home/chuck/.claude/plans/warm-imagining-river.md). The docs in
  this PR correctly state that delete() doesn't exist on
  CacheManager — the fix belongs in the follow-up code PR that
  either adds a delete() shim or updates the two callers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 19:11:41 -04:00
Chuck
ef579dd191 docs: fix broken file references found by path-existence crosscheck
Ran a doc-vs-filesystem crosscheck: extracted every backtick-quoted
path with a file extension or known directory prefix from docs/*.md
and verified each exists. After filtering false positives
(placeholder paths, config keys mistaken for paths, paths inside
docs that already have historical-status banners), found 4 real
broken references — 3 fixed in docs, 1 fixed by creating the missing
file:

docs/HOW_TO_RUN_TESTS.md:339
- Claimed ".github/workflows/tests.yml" exists and runs pytest on
  multiple Python versions in CI. There is no such workflow.
  The only GitHub Actions file is security-audit.yml (bandit + semgrep).
- Pytest runs locally but is NOT gated on PRs.
- Replaced the fictional CI section with the actual state and a
  note explaining how someone could contribute a real test workflow.

docs/MIGRATION_GUIDE.md:92
- Referenced scripts/fix_perms/README.md "(if exists)" — the
  hedge betrays that the writer wasn't sure. The README didn't
  exist. The 6 scripts in scripts/fix_perms/ were never documented.
- Created the missing scripts/fix_perms/README.md from scratch
  with one-line descriptions of all 6 scripts (fix_assets,
  fix_cache, fix_plugin, fix_web, fix_nhl_cache, safe_plugin_rm)
  + when-to-use-each guidance + usage examples.
- Updated MIGRATION_GUIDE link to drop the "(if exists)" hedge
  since the file now exists.

docs/FONT_MANAGER.md:376
- "See test/font_manager_example.py for a complete working example"
  — that file does not exist. Verified by listing test/ directory.
- Replaced with a pointer to src/font_manager.py itself and the
  existing scoreboard base classes in src/base_classes/ that
  actually use the font manager API in production.

Path-existence check methodology:
- Walked docs/ recursively, regex-extracted backtick-quoted paths
  matching either /\.(py|sh|json|yml|yaml|md|txt|service|html|js|css|ttf|bdf|png)/
  or paths starting with known directory prefixes (scripts/, src/,
  config/, web_interface/, systemd/, assets/, docs/, test/, etc.)
- Filtered out URLs, absolute paths (placeholders), and paths
  without slashes (likely not relative refs).
- Checked existence relative to project root.
- Out of 80 unique relative paths in docs/, 32 didn't exist on
  disk. Most were false positives (configkeys mistaken for paths,
  example placeholders like 'assets/myfont.ttf', historical
  references inside docs that already have status banners). The 4
  above were genuine broken refs.

This pattern is reusable for future iterations and worth wiring
into CI (link checker like lychee, scoped to fenced code paths
rather than just markdown links, would catch the same class).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 16:22:48 -04:00
Chuck
b15bbb6f4b docs(requirements): document optional dependencies (scipy, psutil, Flask-Limiter)
A doc-vs-code crosscheck of every Python import in src/ and
web_interface/ against requirements.txt found 3 packages that the
code uses but requirements.txt doesn't list. Verified with grep that
all 3 are wrapped in try/except blocks with documented fallback
paths, so they're optional features rather than missing required
deps:

- scipy           src/common/scroll_helper.py:26
                  → from scipy.ndimage import shift; HAS_SCIPY flag.
                  Used for sub-pixel interpolation in scrolling.
                  Falls back to a simpler shift algorithm without it.

- psutil          src/plugin_system/resource_monitor.py:15
                  → import psutil; PSUTIL_AVAILABLE flag. Used for
                  per-plugin CPU/memory monitoring. Silently no-ops
                  without it.

- flask-limiter   web_interface/app.py:42-43
                  → from flask_limiter import Limiter; wrapped at the
                  caller. Used for accidental-abuse rate limiting on
                  the web interface (not security). Web interface
                  starts without rate limiting when missing.

These were latent in two ways:
1. A user reading requirements.txt thinks they have the full feature
   set after `pip install -r requirements.txt`, but they don't get
   smoother scrolling, plugin resource monitoring, or rate limiting.
2. A contributor who deletes one of the packages from their dev env
   wouldn't know which feature they just lost — the fallbacks are
   silent.

Added an "Optional dependencies" section at the bottom of
requirements.txt with the version constraint, the file:line where
each is used, the feature it enables, and the install command. The
comment-only format means `pip install -r requirements.txt` still
gives the minimal-feature install (preserving current behavior),
while users who want the full feature set can copy the explicit
pip install commands.

Other findings from the same scan that came back as false positives
or known issues:
- web_interface_v2: dead pattern flagged in earlier iteration
  (still no real implementation; affects 11+ plugins via the same
  try/except dead-fallback pattern)
- urllib3: comes with `requests` transitively
- All 'src.', 'web_interface.', 'rgbmatrix', 'RGBMatrixEmulator'
  imports: internal modules
- base_plugin / plugin_manager / store_manager / mocks /
  visual_display_manager: relative imports to local modules
- freetype: false positive (freetype-py is in requirements.txt
  under the package name)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 16:07:16 -04:00
Chuck
40fcd1ed9f docs: fix README_NBA_LOGOS and PLUGIN_CONFIGURATION_GUIDE
scripts/README_NBA_LOGOS.md
- "python download_nba_logos.py" — wrong on two counts. The script
  is at scripts/download_nba_logos.py (not the project root), and
  "python" is Python 2 on most systems. Replaced all 4 occurrences
  with "python3 scripts/download_nba_logos.py".
- The doc framed itself as the way to set up "the NBA leaderboard".
  The basketball/leaderboard functionality is now in the
  basketball-scoreboard and ledmatrix-leaderboard plugins (in the
  ledmatrix-plugins repo), which auto-download logos on first run.
  Reframed the script as a pre-population utility for offline / dev
  use cases.
- Bumped the documented Python minimum from 3.7 to 3.9 to match
  the rest of the project.

docs/PLUGIN_CONFIGURATION_GUIDE.md
- The "Plugin Manifest" example was missing 3 fields the plugin
  loader actually requires: id, entry_point, and class_name. A
  contributor copying this manifest verbatim would get
  PluginError("No class_name in manifest") at load time — the same
  loader bug already found in stock-news. Added all three.
- The same example showed config_schema as an inline object. The
  loader expects config_schema to be a file path string (e.g.
  "config_schema.json") with the actual schema in a separate JSON
  file — verified earlier in this audit. Fixed.
- Added a paragraph explaining the loader's required fields and
  the case-sensitivity rule on class_name (the bug that broke
  hello-world's manifest before this PR fixed it).
- "Plugin Manager Class" example had the wrong constructor
  signature: (config, display_manager, cache_manager, font_manager).
  The real BasePlugin.__init__ at base_plugin.py:53-60 takes
  (plugin_id, config, display_manager, cache_manager, plugin_manager).
  A copy-pasted example would TypeError on instantiation. Fixed,
  including a comment noting which attributes BasePlugin sets up.
- Renamed the example class from MyPluginManager to MyPlugin to
  match the project convention (XxxPlugin / XxxScoreboardPlugin
  in actual plugins).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 14:07:39 -04:00
Chuck
33d023bbd5 docs(widgets): list the 20 undocumented built-in widgets
The widget registry README documented 3 widgets (file-upload,
checkbox-group, custom-feeds) but the directory contains 23 registered
widgets total. A plugin author reading this doc would think those 3
were the only built-in options and either reach for a custom widget
unnecessarily or settle for a generic text input.

Verified the actual list with:
  grep -h "register('" web_interface/static/v3/js/widgets/*.js \
    | sed -E "s|.*register\\('([^']+)'.*|\\1|" | sort -u

Added an "Other Built-in Widgets" section after the 3 detailed
sections, listing the remaining 20 with one-line descriptions
organized by category:
- Inputs (6): text-input, textarea, number-input, email-input,
  url-input, password-input
- Selectors (7): select-dropdown, radio-group, toggle-switch,
  slider, color-picker, font-selector, timezone-selector
- Date/time/scheduling (4): date-picker, day-selector, time-range,
  schedule-picker
- Composite/data-source (2): array-table, google-calendar-picker
- Internal (2): notification, base-widget

Pointed at the .js source files as the canonical source for each
widget's exact schema and options — keeps this list low-maintenance
since I'm not duplicating each widget's full options table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 13:52:14 -04:00
Chuck
62da1d2b09 docs: fix bare /api/plugins paths in PLUGIN_CONFIGURATION_TABS
Found 5 more bare /api/plugins/* paths in PLUGIN_CONFIGURATION_TABS.md
that I missed in the round 2 sweep — they're inside data flow diagrams
and prose ("loaded via /api/plugins/installed", etc.) so the earlier
grep over Markdown code blocks didn't catch them. Fixed all 5 to use
/api/v3/plugins/* (the api_v3 blueprint mount path verified at
web_interface/app.py:144).

Also added a status banner noting that the "Implementation Details"
section references the pre-v3 file layout (web_interface_v2.py,
templates/index_v2.html) which no longer exists. The current
implementation is in web_interface/app.py, blueprints/api_v3.py, and
templates/v3/. Same kind of historical drift I flagged in
PLUGIN_ARCHITECTURE_SPEC.md and the PLUGIN_CUSTOM_ICONS_FEATURE doc.
The user-facing parts of the doc (Overview, Features, Form Generation
Process) are still accurate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 13:37:52 -04:00
Chuck
f4dbde51bd Customize bug report template for LEDMatrix hardware
The bug_report.md template was the GitHub default and asked
"Desktop (OS/Browser/Version)" and "Smartphone (Device/OS)" — neither
of which is relevant for a project that runs on a Raspberry Pi with
hardware LED panels. A user filing a bug under the old template was
giving us none of the information we'd actually need to triage it.

Replaced with a LEDMatrix-aware template that prompts for:
- Pi model, OS/kernel, panel type, HAT/Bonnet, PWM jumper status,
  display chain dimensions
- LEDMatrix git commit / release tag
- Plugin id and version (if the bug is plugin-related)
- Relevant config snippet (with redaction reminder for API keys)
- journalctl log excerpt with the exact command to capture it
- Optional photo of the actual display for visual issues

Kept feature_request.md as-is — generic content there is fine.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 13:22:41 -04:00
Chuck
38773044e9 Add CODE_OF_CONDUCT, SECURITY, PR template; link them from README
Tier 1 organizational files that any open-source project at
LEDMatrix's maturity is expected to have. None of these existed
before. They're additive — no existing content was rewritten.

CODE_OF_CONDUCT.md
- Contributor Covenant 2.1 (the de facto standard for open-source
  projects). Mentions both the Discord and the GitHub Security
  Advisories channel for reporting violations.

SECURITY.md
- Private vulnerability disclosure flow with two channels: GitHub
  Security Advisories (preferred) and Discord DM.
- Documents the project's known security model as intentional
  rather than vulnerabilities: no web UI auth, plugins run
  unsandboxed, display service runs as root for GPIO access,
  config_secrets.json is plaintext. These match the limitations
  already called out in PLUGIN_QUICK_REFERENCE.md and the audit
  flagging from earlier in this PR.
- Out-of-scope section points users at upstream
  (rpi-rgb-led-matrix, third-party plugins) so reports land in the
  right place.

.github/PULL_REQUEST_TEMPLATE.md
- 10-line checklist that prompts for the things that would have
  caught the bugs in this very PR: did you load the changed plugin
  once, did you update docs alongside code, are there any plugin
  compatibility implications.
- Linked from CONTRIBUTING.md for the full flow.

README.md
- Added a License section near the bottom (the README previously
  said nothing about the license despite the project being GPL-3.0).
- Added a Contributing section pointing at CONTRIBUTING.md and
  SECURITY.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 12:52:48 -04:00
Chuck
44cd3e8c2f Add LICENSE (GPL-3.0) and CONTRIBUTING.md
LICENSE
- The repository previously had no LICENSE file. The README and every
  downstream plugin README already reference GPL-3.0 ("same as
  LEDMatrix project"), but the canonical license text was missing —
  contributors had no formal record of what they were contributing
  under, and GitHub couldn't auto-detect the license for the repo
  banner.
- Added the canonical GPL-3.0 text from
  https://www.gnu.org/licenses/gpl-3.0.txt (verbatim, 674 lines).
- Compatibility verified: rpi-rgb-led-matrix is GPL-2.0-or-later
  (per its COPYING file and README; the "or any later version" clause
  in lib/*.h headers makes GPL-3.0 distribution legal).

CONTRIBUTING.md
- The repository had no CONTRIBUTING file. New contributors had to
  reconstruct the dev setup from DEVELOPMENT.md, PLUGIN_DEVELOPMENT_GUIDE.md,
  SUBMISSION.md, and the root README.
- Added a single page covering: dev environment setup (preview
  server, emulator, hardware), running tests, PR submission flow,
  commit message convention, plugin contribution pointer, and the
  license terms contributors are agreeing to.

> Note for the maintainer: I (the AI assistant doing this audit) am
> selecting GPL-3.0 because every reference in the existing
> documentation already says GPL-3.0 — this commit just makes that
> declaration legally binding by adding the actual file. Please
> confirm during PR review that GPL-3.0 is what you want; if you
> prefer a different license, revert this commit before merging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 12:21:46 -04:00
Chuck
8b838ff366 docs: fix .cursorrules — the file Cursor auto-loads to learn the API
This is the file that Cursor reads to learn how plugin development
works. Stale entries here directly mislead AI-assisted plugin authors
on every new plugin. Several of the same bug patterns I've been
fixing in the user-facing docs were here too.

Display Manager section (highest impact)
- "draw_image(image, x, y): Draw PIL Image" — that method doesn't
  exist on DisplayManager. Same bug already fixed in
  PLUGIN_API_REFERENCE.md, PLUGIN_DEVELOPMENT_GUIDE.md,
  ledmatrix-stocks/README.md, and .cursor/plugin_templates/QUICK_START.md.
  Removed the bullet and replaced it with a paragraph explaining the
  real pattern: paste onto display_manager.image directly, then
  update_display(). Includes the transparency-mask form.
- Added the small_font/centered args to draw_text() since they're
  the ones that matter most for new plugin authors
- Added draw_weather_icon since it's commonly used

Cache Manager section
- "delete(key): Remove cached value" — there's no delete() method
  on CacheManager. The real method is clear_cache(key=None) (also
  removes everything when called without args). Same bug as before.
- Added get_cached_data_with_strategy and get_background_cached_data
  since contributors will hit these when working on sports plugins

Plugin System Overview
- "loaded from the plugins/ directory" — clarified that the default
  is plugin-repos/ (per config.template.json:130) with plugins/ as
  the dev fallback used by scripts/dev/dev_plugin_setup.sh

Plugin Development Workflow
- ./dev_plugin_setup.sh -> ./scripts/dev/dev_plugin_setup.sh (×2)
- Manual setup step "Create directory in plugins/<plugin-id>/" ->
  plugin-repos/<plugin-id>/ as the canonical location
- "Use emulator: python run.py --emulator or ./run_emulator.sh"
  — the --emulator flag doesn't exist; ./run_emulator.sh isn't at
  root (it lives at scripts/dev/run_emulator.sh). Replaced with the
  real options: scripts/dev_server.py for dev preview, or
  EMULATOR=true python3 run.py for the full emulator path.

Configuration Management
- "Reference secrets via config_secrets key in main config" — this
  is the same fictional reference syntax I just fixed in
  .cursor/plugins_guide.md. Verified in src/config_manager.py:162-172
  that secrets are deep-merged into the main config; there's no
  separate reference field. Replaced with a clear explanation of
  the deep-merge approach.

Code Organization
- "plugins/<plugin-id>/" -> the canonical location is
  plugin-repos/<plugin-id>/ (or its dev-time symlink in plugins/)
- "see plugins/hockey-scoreboard/ as reference" — the canonical
  source for example plugins is the ledmatrix-plugins repo. Updated
  the pointer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 10:22:41 -04:00
Chuck
93e2d29af6 docs: fix .cursor/ helper docs
The .cursor/ directory holds the dev-side helper docs that Cursor and
contributors using AI tooling rely on to bootstrap plugin development.
Several of them had the same bug patterns as the user-facing docs.

.cursor/plugin_templates/QUICK_START.md
- "Adding Image Rendering" section showed
  display_manager.draw_image(image, x=0, y=0). That method doesn't
  exist on DisplayManager (same bug as PLUGIN_API_REFERENCE.md and
  PLUGIN_DEVELOPMENT_GUIDE.md). Replaced with the canonical
  display_manager.image.paste((x,y)) pattern, including the
  transparency-mask form.

.cursor/plugins_guide.md
- 10 occurrences of ./dev_plugin_setup.sh — the script lives at
  scripts/dev/dev_plugin_setup.sh, so anyone copy-pasting these
  examples gets "command not found". Bulk fixed via sed.
- "Test with emulator: python run.py --emulator" — there's no
  --emulator flag. Replaced with the real options:
  EMULATOR=true python3 run.py for the full display, or
  scripts/dev_server.py for the dev preview.
- Secrets management section showed a fictional
  "config_secrets": { "api_key": "my-plugin.api_key" } reference
  field. Verified in src/config_manager.py:162-172 that secrets are
  loaded by deep-merging config_secrets.json into the main config.
  There is no separate reference field — just put the secret under
  the same plugin namespace and read it from the merged config.
  Rewrote the section with the real pattern.
- "ssh pi@raspberrypi" -> "ssh ledpi@your-pi-ip" (consistent with
  the rest of LEDMatrix docs which use ledpi as the default user)

.cursor/README.md
- Same ./dev_plugin_setup.sh -> ./scripts/dev/dev_plugin_setup.sh
  fix (×6 occurrences via replace_all).
- Same "python run.py --emulator" -> "EMULATOR=true python3 run.py"
  fix. Also added a pointer to scripts/dev_server.py for previewing
  plugins without running the full display.
- "Example Plugins: plugins/hockey-scoreboard/" — the canonical
  source is the ledmatrix-plugins repo. Installed copies land in
  plugin-repos/ or plugins/. Updated the line to point at the
  ledmatrix-plugins repo and explain both local locations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 10:08:46 -04:00
Chuck
a62d4529fb docs: flag aspirational/regressed features in plugin docs
These docs describe features that exist as documented in the doc but
either never wired up or regressed when v3 shipped. Each gets a clear
status banner so plugin authors don't waste time chasing features that
don't actually work.

FONT_MANAGER.md
- The "For Plugin Developers / Plugin Font Registration" section
  documents adding a "fonts" block to manifest.json that gets
  registered via FontManager.register_plugin_fonts(). The method
  exists at src/font_manager.py:150 but is **never called from
  anywhere** in the codebase (verified: zero callers). A plugin
  shipping a manifest "fonts" block has its fonts silently ignored.
  Added a status warning and a note about how to actually ship plugin
  fonts (regular files in the plugin dir, loaded directly).

PLUGIN_IMPLEMENTATION_SUMMARY.md
- Added a top-level status banner.
- Architecture diagram referenced src/plugin_system/registry_manager.py
  (which doesn't exist) and listed plugins/ as the install location.
  Replaced with the real file list (plugin_loader, schema_manager,
  health_monitor, operation_queue, state_manager) and pointed at
  plugin-repos/ as the default install location.
- "Dependency Management: Virtual Environments" — verified there's no
  per-plugin venv. Removed the bullet and added a note that plugin
  Python deps install into the system Python environment, with no
  conflict resolution.
- "Permission System: File Access Control / Network Access /
  Resource Limits / CPU and memory constraints" — none of these
  exist. There's a resource_monitor.py and health_monitor.py for
  metrics/warnings, but no hard caps or sandboxing. Replaced the
  section with what's actually implemented and a clear note that
  plugins run in the same process with full file/network access.

PLUGIN_CUSTOM_ICONS.md and PLUGIN_CUSTOM_ICONS_FEATURE.md
- The custom-icon feature was implemented in the v2 web interface
  via a getPluginIcon() helper in templates/index_v2.html that read
  the manifest "icon" field. When the v3 web interface was built,
  that helper wasn't ported. Verified in
  web_interface/templates/v3/base.html:515 and :774, plugin tab
  icons are hardcoded to `fas fa-puzzle-piece`. The "icon" field in
  plugin manifests is currently silently ignored (verified with grep
  across web_interface/ and src/plugin_system/ — zero non-action-
  related reads of plugin.icon or manifest.icon).
- Added a status banner to both docs noting the regression so plugin
  authors don't think their custom icons are broken in their own
  plugin code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 09:39:46 -04:00
Chuck
b577668568 docs: clarify plugin paths and fix systemd manual install bug
PLUGIN_DEVELOPMENT_GUIDE.md
- Added a "Plugin directory note" callout near the top explaining
  the plugins/ vs plugin-repos/ split:
  - Dev workflow uses plugins/ (where dev_plugin_setup.sh creates
    symlinks)
  - Production / Plugin Store uses plugin-repos/ (the configurable
    default per config.template.json:130)
  - The plugin loader falls back to plugins/ so dev symlinks are
    picked up automatically (schema_manager.py:77)
  - User can set plugins_directory to "plugins" in the General tab
    if they want both to share a directory

CLAUDE.md
- The Project Structure section had plugins/ and plugin-repos/
  exactly reversed:
  - Old: "plugins/ - Installed plugins directory (gitignored)"
         "plugin-repos/ - Development symlinks to monorepo plugin dirs"
  - Real: plugin-repos/ is the canonical Plugin Store install
    location and is not gitignored. plugins/* IS gitignored
    (verified in .gitignore) and is the legacy/dev location used by
    scripts/dev/dev_plugin_setup.sh.
  Reversed the descriptions and added line refs.

systemd/README.md
- "Manual Installation" section told users to copy the unit file
  directly to /etc/systemd/system/. Verified the unit file in
  systemd/ledmatrix.service contains __PROJECT_ROOT_DIR__
  placeholders that the install scripts substitute at install time.
  A user following the manual steps would get a service that fails
  to start with "WorkingDirectory=__PROJECT_ROOT_DIR__" errors.
  Added a clear warning and a sed snippet that substitutes the
  placeholder before installing.

src/common/README.md
- Was missing 2 of the 11 utility modules in the directory
  (verified with ls): permission_utils.py and cli.py. Added brief
  descriptions for both.

Out-of-scope code bug found while auditing (flagged but not fixed):
- scripts/dev/dev_plugin_setup.sh:9 sets PROJECT_ROOT="$SCRIPT_DIR"
  which resolves to scripts/dev/, not the project root. This means
  the script's PLUGINS_DIR resolves to scripts/dev/plugins/ instead
  of the project's plugins/ — confirmed by the existence of
  scripts/dev/plugins/of-the-day/ from prior runs. Real fix is to
  set PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)". Not fixing in
  this docs PR.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 09:25:01 -04:00
Chuck
2f3433cebc docs: fix misc remaining docs (architecture, dev quickref, sub-dir READMEs)
PLUGIN_ARCHITECTURE_SPEC.md
- Added a banner at the top noting this is a historical design doc
  written before the plugin system shipped. The doc is ~1900 lines
  with 13 stale /api/plugins/* paths (real is /api/v3/plugins/*),
  references to web_interface_v2.py (current is app.py), and a
  Migration Strategy / Implementation Roadmap that's now history.
  Banner points readers at the current docs
  (PLUGIN_DEVELOPMENT_GUIDE, PLUGIN_API_REFERENCE,
  REST_API_REFERENCE) without needing to retrofit every section.

PLUGIN_CONFIG_ARCHITECTURE.md
- 10 occurrences of /api/plugins/* missing /v3 prefix. Bulk fixed.

DEVELOPER_QUICK_REFERENCE.md
- cache_manager.delete("key") -> cache_manager.clear_cache("key")
  with comment noting delete() doesn't exist. Same bug already
  documented in PLUGIN_API_REFERENCE.md.

SSH_UNAVAILABLE_AFTER_INSTALL.md
- 4 occurrences of port 5001 -> 5000 in AP-mode and Ethernet/WiFi
  recovery instructions.

PLUGIN_CUSTOM_ICONS_FEATURE.md
- Port 5001 -> 5000.

CONFIG_DEBUGGING.md
- Documented /api/v3/config/plugin/<id> and /api/v3/config/validate
  endpoints don't exist. Replaced with the real endpoints:
  /api/v3/config/main, /api/v3/plugins/schema?plugin_id=,
  /api/v3/plugins/config?plugin_id=. Added a note that validation
  runs server-side automatically on POST.

STARLARK_APPS_GUIDE.md
- "Plugins -> Starlark Apps" UI navigation path doesn't exist (5
  occurrences). Replaced with the real path: Plugin Manager tab,
  then the per-plugin Starlark Apps tab in the second nav row.
- "Navigate to Plugins" install step -> Plugin Manager tab.

web_interface/README.md
- Documented several endpoints that don't exist in the api_v3
  blueprint:
  - GET /api/v3/plugins (list) -> /api/v3/plugins/installed
  - GET /api/v3/plugins/<id> -> doesn't exist
  - POST /api/v3/plugins/<id>/config -> POST /api/v3/plugins/config
  - GET /api/v3/plugins/<id>/enable + /disable -> POST /api/v3/plugins/toggle
  - GET /api/v3/store/plugins -> /api/v3/plugins/store/list
  - POST /api/v3/store/install/<id> -> POST /api/v3/plugins/install
  - POST /api/v3/store/uninstall/<id> -> POST /api/v3/plugins/uninstall
  - POST /api/v3/store/update/<id> -> POST /api/v3/plugins/update
  - POST /api/v3/display/start/stop/restart -> POST /api/v3/system/action
  - GET /api/v3/display/status -> GET /api/v3/system/status
- Also fixed config/secrets.json -> config/config_secrets.json
- Replaced the per-section endpoint duplication with a current real
  endpoint list and a pointer to docs/REST_API_REFERENCE.md.
- Documented that SSE stream endpoints are defined directly on the
  Flask app at app.py:607-615, not in the api_v3 blueprint.

scripts/install/README.md
- Was missing 3 of the 9 install scripts in the directory:
  one-shot-install.sh, configure_wifi_permissions.sh, and
  debug_install.sh. Added them with brief descriptions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 09:19:47 -04:00
Chuck
b374bfa8c6 docs: fix plugin config + store + dependency docs
PLUGIN_STORE_GUIDE.md
- 19 occurrences of port 5050 -> 5000
- All API paths missing /v3 (e.g. /api/plugins/install ->
  /api/v3/plugins/install). Bulk fix.

PLUGIN_REGISTRY_SETUP_GUIDE.md
- Same port + /api/v3 fixes (3 occurrences each)
- "Go to Plugin Store tab" -> "Open the Plugin Manager tab and scroll
  to the Install from GitHub section" (the real flow for registry
  setup is the GitHub install section, not the Plugin Store search)

PLUGIN_CONFIG_QUICK_START.md
- Port 5001 -> 5000 (5001 is the dev_server.py default, not the web UI)
- "Plugin Store tab" install flow -> real Plugin Manager + Plugin Store
  section + per-plugin tab in second nav row
- Removed reference to PLUGIN_CONFIG_TABS_SUMMARY.md (archived doc)

PLUGIN_CONFIGURATION_TABS.md
- "Plugin Management vs Configuration" section confusingly described
  a "Plugins Tab" that doesn't exist as a single thing. Rewrote to
  describe the real two-piece structure: Plugin Manager tab (browse,
  install, toggle) vs per-plugin tabs (configure individual plugins).

PLUGIN_DEPENDENCY_GUIDE.md
- Port 5001 -> 5000

PLUGIN_DEPENDENCY_TROUBLESHOOTING.md
- Wrong port (8080) and wrong UI nav ("Plugin Store or Plugin
  Management"). Fixed to the real flow.

PLUGIN_QUICK_REFERENCE.md
- "Plugin Location: ./plugins/ directory" -> default is plugin-repos/
  (verified in config/config.template.json:130 and
  display_controller.py:132). plugins/ is a fallback.
- File structure diagram showed plugins/ -> plugin-repos/.
- Web UI install flow: "Plugin Store tab" -> "Plugin Manager tab ->
  Plugin Store section". Also fixed Configure ⚙️ button (doesn't
  exist) and "Drag and drop reorder" (not implemented).
- API examples: replaced ad-hoc Python pseudocode with real curl
  examples against /api/v3/plugins/* endpoints. Pointed at
  REST_API_REFERENCE.md for the full list.
- "Migration Path Phase 1-5" was a roadmap written before the plugin
  system shipped. The plugin system is now stable and live. Removed
  the migration phases as they're history, not a roadmap.
- "Quick Migration" section called scripts/migrate_to_plugins.py
  which doesn't exist anywhere in the repo. Removed.
- "Plugin Registry Structure" referenced
  ChuckBuilds/ledmatrix-plugin-registry which doesn't exist. The
  real registry is ChuckBuilds/ledmatrix-plugins. Fixed.
- "Next Steps" / "Questions to Resolve" sections were
  pre-implementation planning notes. Replaced with a "Known
  Limitations" section that documents the actually-real gaps
  (sandboxing, resource limits, ratings, auto-updates).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 22:10:05 -04:00
Chuck
49287bdd1a docs: fix ADVANCED_FEATURES and REST_API_REFERENCE
REST_API_REFERENCE.md
- Wrong path: /fonts/delete/<font_family> -> /fonts/<font_family>
  (verified the real DELETE route in
  web_interface/blueprints/api_v3.py).
- Diffed the documented routes against the real api_v3 blueprint
  (92 routes vs the 71 documented). Added missing sections:
  - Error tracking (/errors/summary, /errors/plugin/<id>, /errors/clear)
  - Health (/health)
  - Schedule dim/power (/config/dim-schedule GET/POST)
  - Plugin-specific endpoints (calendar/list-calendars,
    of-the-day/json/upload+delete, plugins/<id>/static/<path>)
  - Starlark Apps (12 endpoints: status, install-pixlet, apps CRUD,
    repository browse/install, upload)
  - Font preview (/fonts/preview)
- Updated table of contents with the new sections.
- Added a footer note that the API blueprint mounts at /api/v3
  (app.py:144) and that SSE stream endpoints are defined directly on
  the Flask app at app.py:607-615.

ADVANCED_FEATURES.md
- Vegas Scroll Mode section was actually accurate (verified all
  config keys match src/vegas_mode/config.py:15-30).

- On-Demand Display section had multiple bugs:
  - 5 occurrences of port 5050 -> 5000
  - All API paths missing /v3 (e.g. /api/display/on-demand/start
    should be /api/v3/display/on-demand/start)
  - "Settings -> Plugin Management -> Show Now Button" UI flow doesn't
    exist. Real flow: open the plugin's tab in the second nav row,
    click Run On-Demand / Stop On-Demand.
  - "Python API Methods" section showed
    controller.show_on_demand() / clear_on_demand() /
    is_on_demand_active() / get_on_demand_info() — none of these
    methods exist on DisplayController. The on-demand machinery is
    all internal (_set_on_demand_*, _activate_on_demand, etc) and
    is driven through the cache_manager. Replaced the section with
    a note pointing to the REST API.
  - All Use Case Examples used the same fictional Python calls.
    Replaced with curl examples against the real API.

- Cache Management section claimed "On-demand display uses Redis cache
  keys". LEDMatrix doesn't use Redis — verified with grep that
  src/cache_manager.py has no redis import. The cache is file-based,
  managed by CacheManager (file at /var/cache/ledmatrix/ or fallback
  paths). Rewrote the manual recovery section:
  - Removed redis-cli commands
  - Replaced cache.delete() Python calls with cache.clear_cache()
    (the real public method per the same bug already flagged in
    PLUGIN_API_REFERENCE.md)
  - Replaced "Settings -> Cache Management" with the real Cache tab
  - Documented the actual cache directory candidates

- Background Data Service section:
  - Used "nfl_scoreboard" as the plugin id in the example.
    The real plugin is "football-scoreboard" (handles both NFL and
    NCAA). Fixed.
  - "Implementation Status: Phase 1 NFL only / Phase 2 planned"
    section was severely outdated. The background service is now
    used by all sports scoreboards (football, hockey, baseball,
    basketball, soccer, lacrosse, F1, UFC), the odds ticker, and
    the leaderboard plugin. Replaced with a current "Plugins using
    the background service" note.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 21:55:34 -04:00
Chuck
1d31465df0 docs: fix WEB_INTERFACE_GUIDE and WIFI_NETWORK_SETUP
WEB_INTERFACE_GUIDE.md
- Web UI port: 5050 -> 5000 (4 occurrences)
- Tab list was almost entirely fictional. Documented tabs:
  General Settings, Display Settings, Durations, Sports Configuration,
  Plugin Management, Plugin Store, Font Management. None of these
  exist. Real tabs (verified in web_interface/templates/v3/base.html:
  935-1000): Overview, General, WiFi, Schedule, Display, Config Editor,
  Fonts, Logs, Cache, Operation History, plus Plugin Manager and
  per-plugin tabs in the second nav row. Rewrote the navigation
  section, the General/Display/Plugin sections, and the Common Tasks
  walkthroughs to match.
- Quick Actions list referenced "Test Display" button (doesn't exist).
  Replaced with the real button list verified in
  partials/overview.html:88-152: Start/Stop Display, Restart Display
  Service, Restart Web Service, Update Code, Reboot, Shutdown.
- API endpoints used /api/* paths. The api_v3 blueprint mounts at
  /api/v3 (web_interface/app.py:144), so the real paths are
  /api/v3/config/main, /api/v3/system/status, etc. Fixed.
- Removed bogus "Sports Configuration tab" walkthrough; sports
  favorites live inside each scoreboard plugin's own tab now.
- Plugin directory listed as /plugins/. Real default is plugin-repos/
  (verified in config/config.template.json:130 and
  display_controller.py:132); plugins/ is a fallback.
- Removed "Swipe navigation between tabs" mobile claim (not implemented).

WIFI_NETWORK_SETUP.md
- 21 occurrences of port 5050 -> 5000.
- All /api/wifi/* curl examples used the wrong path. The real wifi
  API routes are at /api/v3/wifi/* (api_v3.py:6367-6609). Fixed.
- ap_password default was documented as "" (empty/open network) but
  config/wifi_config.json ships with "ledmatrix123". Updated the
  Quick Start, Configuration table, AP Mode Settings section, and
  Security Recommendations to match. Also clarified that setting
  ap_password to "" is the way to make it an open network.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 21:39:11 -04:00
Chuck
2a7a318cf7 docs: refresh and correct stale documentation across repo
Walked the README and docs/ tree against current code and fixed several
real bugs and many stale references. Highlights:

User-facing
- README.md: web interface install instructions referenced
  install_web_service.sh at the repo root, but it actually lives at
  scripts/install/install_web_service.sh.
- docs/GETTING_STARTED.md: every web UI port reference said 5050, but
  the real server in web_interface/start.py:123 binds 5000. Same bug
  was duplicated in docs/TROUBLESHOOTING.md (17 occurrences). Fixed
  both.
- docs/GETTING_STARTED.md: rewrote tab-by-tab instructions. The doc
  referenced "Plugin Store", "Plugin Management", "Sports Configuration",
  "Durations", and "Font Management" tabs - none of which exist. Real
  tabs (verified in web_interface/templates/v3/base.html) are: Overview,
  General, WiFi, Schedule, Display, Config Editor, Fonts, Logs, Cache,
  Operation History, Plugin Manager (+ per-plugin tabs).
- docs/GETTING_STARTED.md: removed references to a "Test Display"
  button (doesn't exist) and "Show Now" / "Stop" plugin buttons. Real
  controls are "Run On-Demand" / "Stop On-Demand" inside each plugin's
  tab (partials/plugin_config.html:792).
- docs/TROUBLESHOOTING.md: removed dead reference to
  troubleshoot_weather.sh (doesn't exist anywhere in the repo); weather
  is now a plugin in ledmatrix-plugins.

Developer-facing
- docs/PLUGIN_API_REFERENCE.md: documented draw_image() doesn't exist
  on DisplayManager. Real plugins paste onto display_manager.image
  directly (verified in src/base_classes/{baseball,basketball,football,
  hockey}.py). Replaced with the canonical pattern.
- docs/PLUGIN_API_REFERENCE.md: documented cache_manager.delete() doesn't
  exist. Real method is clear_cache(key=None). Updated the section.
- docs/PLUGIN_API_REFERENCE.md: added 10 missing BasePlugin methods that
  the doc never mentioned: dynamic-duration hooks, live-priority hooks,
  and the full Vegas-mode interface.
- docs/PLUGIN_DEVELOPMENT_GUIDE.md: same draw_image fix.
- docs/DEVELOPMENT.md: corrected the "Plugin Submodules" section. Plugins
  are NOT git submodules - .gitmodules only contains
  rpi-rgb-led-matrix-master. Plugins are installed at runtime into the
  plugins directory configured by plugin_system.plugins_directory
  (default plugin-repos/). Both internal links in this doc were also
  broken (missing relative path adjustment).
- docs/HOW_TO_RUN_TESTS.md: removed pytest-timeout from install line
  (not in requirements.txt) and corrected the test/integration/ path
  (real integration tests are at test/web_interface/integration/).
  Replaced the fictional file structure diagram with the real one.
- docs/EMULATOR_SETUP_GUIDE.md: clone URL was a placeholder; default
  pixel_size was documented as 16 but emulator_config.json ships with 5.

Index
- docs/README.md: rewrote. Old index claimed "16-17 files after
  consolidation" but docs/ actually has 38 .md files. Four were missing
  from the index entirely (CONFIG_DEBUGGING, DEV_PREVIEW,
  PLUGIN_ERROR_HANDLING, STARLARK_APPS_GUIDE). Trimmed the navel-gazing
  consolidation/statistics sections.

Out of scope but worth flagging:
- src/plugin_system/resource_monitor.py:343 and src/common/api_helper.py:287
  call cache_manager.delete(key) but no such method exists on
  CacheManager. Both call sites would AttributeError at runtime if hit.
  Not fixed in this docs PR - either add a delete() shim or convert
  callers to clear_cache().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 20:45:19 -04:00
182 changed files with 5284 additions and 14559 deletions

View File

@@ -1,7 +0,0 @@
---
exclude_paths:
- "plugin-repos/**"
- "plugins/**"
- "assets/**"
- "test/**"
- "scripts/debug/**"

View File

@@ -195,9 +195,8 @@ Located in: `src/cache_manager.py`
**Key Methods:**
- `get(key, max_age=300)`: Get cached value (returns None if missing/stale)
- `set(key, value, ttl=None)`: Cache a value
- `delete(key)` / `clear_cache(key=None)`: Remove a single cache entry,
or (for `clear_cache` with no argument) every cached entry. `delete`
is an alias for `clear_cache(key)`.
- `clear_cache(key=None)`: Remove a cache entry, or all entries if `key`
is omitted. There is no `delete()` method.
- `get_cached_data_with_strategy(key, data_type)`: Cache get with
data-type-aware TTL strategy
- `get_background_cached_data(key, sport_key)`: Cache get for the

1
.gitmodules vendored
View File

@@ -1,4 +1,3 @@
[submodule "rpi-rgb-led-matrix-master"]
path = rpi-rgb-led-matrix-master
url = https://github.com/hzeller/rpi-rgb-led-matrix.git
branch = master

View File

@@ -1,10 +1,4 @@
# LEDMatrix
[![License](https://img.shields.io/badge/license-GPL--3.0-green)](LICENSE)
[![Discord](https://img.shields.io/badge/Discord-community-5865F2?logo=discord&logoColor=white)](https://discord.gg/RdrC37rEag)
[![GitHub Stars](https://img.shields.io/github/stars/ChuckBuilds/ledmatrix?style=flat&color=yellow)](https://github.com/ChuckBuilds/ledmatrix)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/77fc9b446a5948e5b0aed7a7aaeb1bab)](https://app.codacy.com/gh/ChuckBuilds/LEDMatrix/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)
## Welcome to LEDMatrix!
Welcome to the LEDMatrix Project! This open-source project enables you to run an information-rich display on a Raspberry Pi connected to an LED RGB Matrix panel. Whether you want to see your calendar, weather forecasts, sports scores, stock prices, or any other information at a glance, LEDMatrix brings it all together.
@@ -132,15 +126,10 @@ The system supports live, recent, and upcoming game information for multiple spo
| This project can be finnicky! RGB LED Matrix displays are not built the same or to a high-quality standard. We have seen many displays arrive dead or partially working in our discord. Please purchase from a reputable vendor. |
### Raspberry Pi
- Raspberry Pi Zero's don't have enough processing power for this project.
- **Raspberry Pi 3B, 4, or 5**
- Raspberry Pi Zero's don't have enough processing power for this project and the Pi 5 is unsupported due to new GPIO output.
- **Raspberry Pi 3B or 4 (NOT RPi 5!)**
[Amazon Affiliate Link Raspberry Pi 4 4GB RAM](https://amzn.to/4dJixuX)
[Amazon Affiliate Link Raspberry Pi 4 8GB RAM](https://amzn.to/4qbqY7F)
- **Pi 5 users**: the installer automatically detects Pi 5 and builds the `rpi-rgb-led-matrix` library with RP1 support. If you previously installed on a Pi 4 and migrated the SD card, or if you see `mmap` errors in the logs, force a fresh library build:
```bash
sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh
```
- Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and set `gpio_slowdown` to `1` or `2`.
### RGB Matrix Bonnet / HAT
@@ -592,7 +581,7 @@ These settings control runtime behavior and GPIO timing:
- **Critical setting**: Must match your Raspberry Pi model for stability
- **Raspberry Pi 3**: Use 3
- **Raspberry Pi 4**: Use 4
- **Raspberry Pi 5**: Use 12 in PIO mode (`rp1_rio: 0`, the default); start with `1` and increase if you see flickering
- **Raspberry Pi 5**: Use 5 (or higher if needed)
- **Raspberry Pi Zero/1**: Use 1-2
- Incorrect values can cause display corruption, flickering, or system instability
- If you experience issues, try adjusting this value up or down by 1

Binary file not shown.

Before

Width:  |  Height:  |  Size: 657 KiB

After

Width:  |  Height:  |  Size: 105 KiB

View File

@@ -1,43 +1,43 @@
{
"web_display_autostart": true,
"schedule": {
"enabled": false,
"enabled": true,
"mode": "per-day",
"start_time": "07:00",
"end_time": "23:00",
"days": {
"monday": {
"enabled": false,
"enabled": true,
"start_time": "07:00",
"end_time": "23:00"
},
"tuesday": {
"enabled": false,
"enabled": true,
"start_time": "07:00",
"end_time": "23:00"
},
"wednesday": {
"enabled": false,
"enabled": true,
"start_time": "07:00",
"end_time": "23:00"
},
"thursday": {
"enabled": false,
"enabled": true,
"start_time": "07:00",
"end_time": "23:00"
},
"friday": {
"enabled": false,
"enabled": true,
"start_time": "07:00",
"end_time": "23:00"
},
"saturday": {
"enabled": false,
"enabled": true,
"start_time": "07:00",
"end_time": "23:00"
},
"sunday": {
"enabled": false,
"enabled": true,
"start_time": "07:00",
"end_time": "23:00"
}
@@ -51,46 +51,46 @@
"end_time": "07:00",
"days": {
"monday": {
"enabled": false,
"enabled": true,
"start_time": "20:00",
"end_time": "07:00"
},
"tuesday": {
"enabled": false,
"enabled": true,
"start_time": "20:00",
"end_time": "07:00"
},
"wednesday": {
"enabled": false,
"enabled": true,
"start_time": "20:00",
"end_time": "07:00"
},
"thursday": {
"enabled": false,
"enabled": true,
"start_time": "20:00",
"end_time": "07:00"
},
"friday": {
"enabled": false,
"enabled": true,
"start_time": "20:00",
"end_time": "07:00"
},
"saturday": {
"enabled": false,
"enabled": true,
"start_time": "20:00",
"end_time": "07:00"
},
"sunday": {
"enabled": false,
"enabled": true,
"start_time": "20:00",
"end_time": "07:00"
}
}
},
"timezone": "America/New_York",
"timezone": "America/Chicago",
"location": {
"city": "Tampa",
"state": "Florida",
"city": "Dallas",
"state": "Texas",
"country": "US"
},
"display": {
@@ -112,8 +112,7 @@
"limit_refresh_rate_hz": 100
},
"runtime": {
"gpio_slowdown": 3,
"rp1_rio": 0
"gpio_slowdown": 3
},
"display_durations": {},
"use_short_date_format": true,
@@ -127,11 +126,6 @@
"buffer_ahead": 2
}
},
"sync": {
"role": "standalone",
"port": 5765,
"follower_position": "left"
},
"plugin_system": {
"plugins_directory": "plugin-repos",
"auto_discover": true,

View File

@@ -1,9 +1,17 @@
{
"ledmatrix-weather": {
"api_key": "YOUR_OPENWEATHERMAP_API_KEY"
},
"youtube": {
"api_key": "YOUR_YOUTUBE_API_KEY",
"channel_id": "YOUR_YOUTUBE_CHANNEL_ID"
},
"music": {
"SPOTIFY_CLIENT_ID": "YOUR_SPOTIFY_CLIENT_ID_HERE",
"SPOTIFY_CLIENT_SECRET": "YOUR_SPOTIFY_CLIENT_SECRET_HERE",
"SPOTIFY_REDIRECT_URI": "http://127.0.0.1:8888/callback"
},
"github": {
"api_token": "YOUR_GITHUB_PERSONAL_ACCESS_TOKEN"
}
}
}

View File

@@ -62,7 +62,7 @@ display_manager.defer_update(lambda: self.update_cache(), priority=0)
# Basic caching
cached = cache_manager.get("key", max_age=3600)
cache_manager.set("key", data)
cache_manager.delete("key") # alias for clear_cache(key)
cache_manager.clear_cache("key") # there is no delete() method
# Advanced caching
data = cache_manager.get_cached_data_with_strategy("key", data_type="weather")

View File

@@ -138,28 +138,29 @@ font = self.font_manager.resolve_font(
## For Plugin Developers
> **Note**: plugins that ship their own fonts via a `"fonts"` block
> in `manifest.json` are registered automatically during plugin load
> (`src/plugin_system/plugin_manager.py` calls
> `FontManager.register_plugin_fonts()`). The `plugin://…` source
> URIs documented below are resolved relative to the plugin's
> install directory.
> ⚠️ **Status**: the plugin-font registration described below is
> implemented in `src/font_manager.py:150` (`register_plugin_fonts()`)
> but is **not currently wired into the plugin loader**. Adding a
> `"fonts"` block to your plugin's `manifest.json` will silently have
> no effect — the FontManager method exists but nothing calls it.
>
> The **Fonts** tab in the web UI that lists detected
> manager-registered fonts is still a **placeholder
> implementation**fonts that managers register through
> `register_manager_font()` do not yet appear there. The
> programmatic per-element override workflow described in
> [Manual Font Overrides](#manual-font-overrides) below
> (`set_override()` / `remove_override()` / the
> `config/font_overrides.json` store) **does** work today and is
> the supported way to override a font for an element until the
> Fonts tab is wired up. If you can't wait and need a workaround
> right now, you can also just load the font directly with PIL
> (or `freetype-py` for BDF) inside your plugin's `manager.py`
> and skip the override system entirely.
> Until that's connected, plugin authors who need a custom font
> should load it directly with PIL (or `freetype-py` for BDF) in
> their plugin's `manager.py``FontManager.resolve_font(family=…,
> size_px=…)` takes a **family name**, not a file path, so it can't
> be used to pull a font from your plugin directory. The
> `plugin://…` source URIs described below are only honored by
> `register_plugin_fonts()` itself, which isn't wired up.
>
> The `/api/v3/fonts/overrides` endpoints and the **Fonts** tab in
> the web UI are currently **placeholder implementations** — they
> return empty arrays and contain "would integrate with the actual
> font system" comments. Manually registered manager fonts do
> **not** yet flow into that tab. If you need an override today,
> load the font directly in your plugin and skip the
> override system.
### Plugin Font Registration
### Plugin Font Registration (planned)
In your plugin's `manifest.json`:

View File

@@ -336,15 +336,15 @@ pytest --cov=src --cov-report=html
## Continuous Integration
The repo runs
[`.github/workflows/security-audit.yml`](../.github/workflows/security-audit.yml)
(bandit + semgrep) on every push. A pytest CI workflow at
`.github/workflows/tests.yml` is queued to land alongside this
PR ([ChuckBuilds/LEDMatrix#307](https://github.com/ChuckBuilds/LEDMatrix/pull/307));
the workflow file itself was held back from that PR because the
push token lacked the GitHub `workflow` scope, so it needs to be
committed separately by a maintainer. Once it's in, this section
will be updated to describe what the job runs.
There is currently no CI test workflow in this repo — `pytest` runs
locally but is not gated on PRs. The only GitHub Actions workflow is
[`.github/workflows/security-audit.yml`](../.github/workflows/security-audit.yml),
which runs bandit and semgrep on every push.
If you'd like to add a test workflow, the recommended setup is a
`.github/workflows/tests.yml` that runs `pytest` against the
supported Python versions (3.10, 3.11, 3.12, 3.13 per
`requirements.txt`). Open an issue or PR if you want to contribute it.
## Best Practices

View File

@@ -34,16 +34,16 @@ This document outlines the transformation of the LEDMatrix project into a modula
## Table of Contents
1. [Current Architecture Analysis](#1-current-architecture-analysis)
2. [Plugin System Design](#2-plugin-system-design)
3. [Plugin Store & Discovery](#3-plugin-store--discovery)
4. [Web UI Transformation](#4-web-ui-transformation)
5. [Migration Strategy](#5-migration-strategy)
6. [Plugin Developer Guidelines](#6-plugin-developer-guidelines)
7. [Technical Implementation Details](#7-technical-implementation-details)
8. [Best Practices & Standards](#8-best-practices--standards)
9. [Security Considerations](#9-security-considerations)
10. [Implementation Roadmap](#10-implementation-roadmap)
1. [Current Architecture Analysis](#current-architecture-analysis)
2. [Plugin System Design](#plugin-system-design)
3. [Plugin Store & Discovery](#plugin-store--discovery)
4. [Web UI Transformation](#web-ui-transformation)
5. [Migration Strategy](#migration-strategy)
6. [Plugin Developer Guidelines](#plugin-developer-guidelines)
7. [Technical Implementation Details](#technical-implementation-details)
8. [Best Practices & Standards](#best-practices--standards)
9. [Security Considerations](#security-considerations)
10. [Implementation Roadmap](#implementation-roadmap)
---

View File

@@ -1,5 +1,16 @@
# Plugin Custom Icons Guide
> ⚠️ **Status:** the `icon` field in `manifest.json` is currently
> **not honored by the v3 web interface**. Plugin tab icons are
> hardcoded to `fas fa-puzzle-piece` in
> `web_interface/templates/v3/base.html:515` and `:774`. The icon
> field was originally read by a `getPluginIcon()` helper in the v2
> templates, but that helper wasn't ported to v3. Setting `icon` in a
> manifest is harmless (it's just ignored) so plugin authors can leave
> it in place for when this regression is fixed.
>
> Tracking issue: see the LEDMatrix repo for the open ticket.
## Overview
Plugins can specify custom icons that appear next to their name in the web interface tabs. This makes your plugin instantly recognizable and adds visual polish to the UI.

View File

@@ -1,12 +1,13 @@
# Plugin Custom Icons Feature
> **Note:** this doc was originally written against the v2 web
> interface. The v3 web interface now honors the same `icon` field
> in `manifest.json` — the API passes it through at
> `web_interface/blueprints/api_v3.py` and the three plugin-tab
> render sites in `web_interface/templates/v3/base.html` read it
> with a `fas fa-puzzle-piece` fallback. The guidance below still
> applies; only the referenced template/helper names differ.
> ⚠️ **Status:** this doc describes the v2 web interface
> implementation of plugin custom icons. The feature **regressed when
> the v3 web interface was built** — the `getPluginIcon()` helper
> referenced below lived in `templates/index_v2.html` (which is now
> archived) and was not ported to the v3 templates. Plugin tab icons
> in v3 are hardcoded to `fas fa-puzzle-piece`
> (`web_interface/templates/v3/base.html:515` and `:774`). The
> `icon` field in `manifest.json` is currently silently ignored.
## What Was Implemented

View File

@@ -10,98 +10,6 @@ The LEDMatrix Widget Registry system allows plugins to use reusable UI component
## Available Core Widgets
### Plugin File Manager Widget (`plugin-file-manager`)
Full inline file management UI for plugins that manage files via the `web_ui_actions` system. Renders a card grid, upload zone, create/delete modals, and an entry table editor — entirely inline, no iframe.
`plugin_id` is **automatically injected** from template context. File operations call `/api/v3/plugins/action` immediately on user action; no Save Configuration needed.
**Schema Configuration:**
```json
{
"file_manager": {
"type": "null",
"title": "Data Files",
"x-widget": "plugin-file-manager",
"x-widget-config": {
"actions": {
"list": "list-files",
"get": "get-file",
"save": "save-file",
"upload": "upload-file",
"delete": "delete-file",
"create": "create-file",
"toggle": "toggle-category"
},
"upload_hint": "JSON files with day numbers 1365 as keys",
"directory_label": "my_data/",
"create_fields": [
{ "key": "category_name", "label": "Category Name",
"placeholder": "e.g., my_words", "pattern": "^[a-z0-9_]+$",
"hint": "Lowercase letters, numbers, underscores" },
{ "key": "display_name", "label": "Display Name",
"placeholder": "e.g., My Words", "hint": "Optional" }
]
}
}
}
```
**`list` is required** — the widget calls it on render to populate the file grid; omitting it leaves the widget stuck in a loading state. All other actions are optional — omit any key to hide its UI element (e.g., no `create` = no New File button, no `toggle` = no enable/disable switch).
The edit view auto-detects whether file content is tabular (object-of-objects with uniform keys) and shows a paginated table editor with inline cells. Otherwise falls back to a JSON textarea.
**Used by:** of-the-day
---
### Time Picker Widget (`time-picker`)
Single time selection using the browser's native time input. Returns a string in `HH:MM` (24-hour) format. Generic — works in any plugin without configuration.
**Schema Configuration:**
```json
{
"target_time": {
"type": "string",
"x-widget": "time-picker",
"default": "00:00",
"x-options": {
"placeholder": "Select time",
"clearable": true
}
}
}
```
**Used by:** countdown
---
### File Upload Single Widget (`file-upload-single`)
Single-image upload for string fields. Uploads to the plugin's asset folder (`assets/plugins/<plugin_id>/uploads/`) and sets the string field value to the returned relative path. Shows a thumbnail preview and a clear button. The `plugin_id` is **automatically injected** from the template context — no need to specify it in the schema.
**Schema Configuration:**
```json
{
"image_path": {
"type": "string",
"x-widget": "file-upload-single",
"x-upload-config": {
"allowed_types": ["image/png", "image/jpeg", "image/bmp", "image/gif"],
"max_size_mb": 5
}
}
}
```
Note: Unlike `file-upload` (array-level), this widget is for a single `string` field. It is ideal for per-item images inside `array-table` rows.
**Used by:** countdown
---
### File Upload Widget (`file-upload`)
Upload and manage image files with drag-and-drop support, preview, delete, and scheduling.

View File

@@ -36,17 +36,9 @@ if [ -r /proc/device-tree/model ]; then
DEVICE_MODEL=$(tr -d '\0' </proc/device-tree/model)
echo "Detected device: $DEVICE_MODEL"
else
DEVICE_MODEL=""
echo "⚠ Could not detect Raspberry Pi model (continuing anyway)"
fi
# Detect Pi 5 for hardware-specific install decisions (RP1 library verification)
IS_PI5=0
if echo "${DEVICE_MODEL:-}" | grep -qi "Raspberry Pi 5"; then
IS_PI5=1
echo "Raspberry Pi 5 detected — will verify RP1 library support."
fi
# Check OS version - must be Raspberry Pi OS Lite (Trixie)
echo ""
echo "Checking operating system requirements..."
@@ -267,6 +259,8 @@ else
fi
echo ""
CLEAR='
'
CURRENT_STEP="Install system dependencies"
echo "Step 1: Installing system dependencies..."
echo "----------------------------------------"
@@ -279,7 +273,7 @@ apt_update
# Install required system packages
echo "Installing Python packages and dependencies..."
apt_install python3-pip python3-venv python-dev-is-python3 python3-pil python3-pil.imagetk build-essential python3-setuptools python3-wheel cmake ninja-build
apt_install python3-pip python3-venv python3-dev python3-pil python3-pil.imagetk build-essential python3-setuptools python3-wheel cython3 scons cmake ninja-build
# Install additional system dependencies that might be needed
echo "Installing additional system dependencies..."
@@ -605,13 +599,9 @@ if [ ! -f "$PROJECT_ROOT_DIR/config/config_secrets.json" ]; then
echo "⚠ Template config/config_secrets.template.json not found; creating a minimal secrets file"
cat > "$PROJECT_ROOT_DIR/config/config_secrets.json" <<'EOF'
{
"youtube": {
"api_key": "YOUR_YOUTUBE_API_KEY",
"channel_id": "YOUR_YOUTUBE_CHANNEL_ID"
},
"github": {
"api_token": "YOUR_GITHUB_PERSONAL_ACCESS_TOKEN"
}
"weather": {
"api_key": "YOUR_OPENWEATHERMAP_API_KEY"
}
}
EOF
# Check if service runs as root and set ownership accordingly
@@ -677,6 +667,8 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
echo "[$PACKAGE_NUM/$TOTAL_PACKAGES] Installing: $line"
# Check if package is already installed (basic check - may not catch all cases)
PACKAGE_NAME=$(echo "$line" | sed -E 's/[<>=!].*$//' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
# Try installing with verbose output and timeout (if available)
# Use --no-cache-dir to avoid cache issues, --verbose for diagnostics
INSTALL_OUTPUT=$(mktemp)
@@ -791,28 +783,9 @@ CURRENT_STEP="Build and install rpi-rgb-led-matrix"
echo "Step 6: Building and installing rpi-rgb-led-matrix..."
echo "-----------------------------------------------------"
# On Pi 5, also check that the installed library has rp1_rio support.
# A library built before Pi 5 support was added imports fine but maps to the
# Pi 3 peripheral bus address (0x3f000000) instead of the RP1 chip at runtime.
_HAS_RP1=0
if python3 -c 'from rgbmatrix import RGBMatrixOptions; assert hasattr(RGBMatrixOptions(), "rp1_rio")' >/dev/null 2>&1; then
_HAS_RP1=1
fi
_SKIP_BUILD=0
# If already installed and not forcing rebuild, skip expensive build
if python3 -c 'from rgbmatrix import RGBMatrix, RGBMatrixOptions' >/dev/null 2>&1 && [ "${RPI_RGB_FORCE_REBUILD:-0}" != "1" ]; then
if [ "$IS_PI5" = "1" ] && [ "$_HAS_RP1" = "0" ]; then
echo "⚠ Pi 5 detected: installed rgbmatrix lacks rp1_rio support (older build)."
echo " Forcing rebuild to get Pi 5 RP1 support..."
else
_SKIP_BUILD=1
fi
fi
if [ "$_SKIP_BUILD" = "1" ]; then
_skip_suffix=""
if [ "$IS_PI5" = "1" ]; then _skip_suffix=" with Pi 5 RP1 support"; fi
echo "rgbmatrix already installed${_skip_suffix}; skipping build (set RPI_RGB_FORCE_REBUILD=1 to force rebuild)."
echo "rgbmatrix Python package already available; skipping build (set RPI_RGB_FORCE_REBUILD=1 to force rebuild)."
else
# Ensure rpi-rgb-led-matrix submodule is initialized
if [ ! -d "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" ]; then
@@ -848,13 +821,20 @@ else
fi
pushd "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" >/dev/null
echo "Installing rpi-rgb-led-matrix Python package (scikit-build-core + cmake)..."
echo " Build deps required: python-dev-is-python3 cmake"
echo " This compiles C++ — may take 2-5 minutes on Pi 4/5..."
echo "Building rpi-rgb-led-matrix Python bindings..."
# Build the library first, then Python bindings
# The build-python target depends on the library being built
if ! make build-python; then
echo "✗ Failed to build rpi-rgb-led-matrix Python bindings"
echo " Make sure you have the required build tools installed:"
echo " sudo apt install -y build-essential python3-dev cython3 scons"
popd >/dev/null
exit 1
fi
cd bindings/python
echo "Installing rpi-rgb-led-matrix Python package via pip..."
if ! python3 -m pip install --break-system-packages .; then
echo "✗ Failed to install rpi-rgb-led-matrix Python package"
echo " Ensure build tools are installed:"
echo " sudo apt install -y python-dev-is-python3 cmake build-essential"
popd >/dev/null
exit 1
fi
@@ -879,17 +859,6 @@ except Exception as e:
PY
then
echo "✓ rpi-rgb-led-matrix installed and verified"
# Pi 5: confirm the freshly-built library has rp1_rio support
if [ "$IS_PI5" = "1" ]; then
if python3 -c 'from rgbmatrix import RGBMatrixOptions; assert hasattr(RGBMatrixOptions(), "rp1_rio")' >/dev/null 2>&1; then
echo "✓ Pi 5 RP1 (rp1_rio) support confirmed"
else
echo "⚠ rp1_rio not found after rebuild — the submodule may be an older version."
echo " Try updating the submodule and rebuilding:"
echo " git submodule update --remote rpi-rgb-led-matrix-master"
echo " sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh"
fi
fi
else
echo "✗ rpi-rgb-led-matrix import test failed"
exit 1
@@ -1113,7 +1082,6 @@ SYSTEMCTL_PATH=$(which systemctl)
REBOOT_PATH=$(which reboot)
POWEROFF_PATH=$(which poweroff)
BASH_PATH=$(which bash)
JOURNALCTL_PATH=$(which journalctl 2>/dev/null || true)
# Create sudoers content
cat > /tmp/ledmatrix_web_sudoers << EOF
@@ -1129,23 +1097,10 @@ $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH enable ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH disable ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT_DIR/display_controller.py
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/start_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/stop_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_plugin_rm.sh *
EOF
if [ -n "$JOURNALCTL_PATH" ]; then
cat >> /tmp/ledmatrix_web_sudoers << EOF
$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -u ledmatrix.service *
$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -u ledmatrix *
$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -t ledmatrix *
EOF
fi
if [ -f "$SUDOERS_FILE" ] && cmp -s /tmp/ledmatrix_web_sudoers "$SUDOERS_FILE"; then
echo "Sudoers configuration already up to date"
@@ -1506,7 +1461,7 @@ echo "WiFi Connection Status:"
if command -v nmcli >/dev/null 2>&1; then
WIFI_STATUS=$(nmcli -t -f DEVICE,TYPE,STATE device status 2>/dev/null | grep -i wifi || echo "")
if [ -n "$WIFI_STATUS" ]; then
echo "$WIFI_STATUS" | while IFS=':' read -r _ _ state; do
echo "$WIFI_STATUS" | while IFS=':' read -r device type state; do
if [ "$state" = "connected" ]; then
SSID=$(nmcli -t -f active,ssid device wifi 2>/dev/null | grep "^yes:" | cut -d: -f2 | head -1)
if [ -n "$SSID" ]; then

View File

@@ -0,0 +1,138 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "March Madness Plugin Configuration",
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"default": false,
"description": "Enable the March Madness tournament display"
},
"leagues": {
"type": "object",
"title": "Tournament Leagues",
"description": "Which NCAA tournaments to display",
"properties": {
"ncaam": {
"type": "boolean",
"default": true,
"description": "Show NCAA Men's Tournament games"
},
"ncaaw": {
"type": "boolean",
"default": true,
"description": "Show NCAA Women's Tournament games"
}
},
"additionalProperties": false
},
"favorite_teams": {
"type": "array",
"title": "Favorite Teams",
"description": "Team abbreviations to highlight (e.g., DUKE, UNC). Leave empty to show all teams equally.",
"items": {
"type": "string"
},
"uniqueItems": true,
"default": []
},
"display_options": {
"type": "object",
"title": "Display Options",
"x-collapsed": true,
"properties": {
"show_seeds": {
"type": "boolean",
"default": true,
"description": "Show tournament seeds (1-16) next to team names"
},
"show_round_logos": {
"type": "boolean",
"default": true,
"description": "Show round logo separators between game groups"
},
"highlight_upsets": {
"type": "boolean",
"default": true,
"description": "Highlight upset winners (higher seed beating lower seed) in gold"
},
"show_bracket_progress": {
"type": "boolean",
"default": true,
"description": "Show which teams are still alive in each region"
},
"scroll_speed": {
"type": "number",
"default": 1.0,
"minimum": 0.5,
"maximum": 5.0,
"description": "Scroll speed (pixels per frame)"
},
"scroll_delay": {
"type": "number",
"default": 0.02,
"minimum": 0.001,
"maximum": 0.1,
"description": "Delay between scroll frames (seconds)"
},
"target_fps": {
"type": "integer",
"default": 120,
"minimum": 30,
"maximum": 200,
"description": "Target frames per second"
},
"loop": {
"type": "boolean",
"default": true,
"description": "Loop the scroll continuously"
},
"dynamic_duration": {
"type": "boolean",
"default": true,
"description": "Automatically adjust display duration based on content width"
},
"min_duration": {
"type": "integer",
"default": 30,
"minimum": 10,
"maximum": 300,
"description": "Minimum display duration in seconds"
},
"max_duration": {
"type": "integer",
"default": 300,
"minimum": 30,
"maximum": 600,
"description": "Maximum display duration in seconds"
}
},
"additionalProperties": false
},
"data_settings": {
"type": "object",
"title": "Data Settings",
"x-collapsed": true,
"properties": {
"update_interval": {
"type": "integer",
"default": 300,
"minimum": 60,
"maximum": 3600,
"description": "How often to refresh tournament data (seconds). Automatically shortens to 60s when live games are detected."
},
"request_timeout": {
"type": "integer",
"default": 30,
"minimum": 5,
"maximum": 60,
"description": "API request timeout in seconds"
}
},
"additionalProperties": false
}
},
"required": ["enabled"],
"additionalProperties": false,
"x-propertyOrder": ["enabled", "leagues", "favorite_teams", "display_options", "data_settings"]
}

View File

@@ -0,0 +1,910 @@
"""March Madness Plugin — NCAA Tournament bracket tracker for LED Matrix.
Displays a horizontally-scrolling ticker of NCAA Tournament games grouped by
round, with seeds, round logos, live scores, and upset highlighting.
"""
import re
import threading
import time
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
import numpy as np
import pytz
import requests
from PIL import Image, ImageDraw, ImageFont
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.plugin_system.base_plugin import BasePlugin
try:
from src.common.scroll_helper import ScrollHelper
except ImportError:
ScrollHelper = None
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
SCOREBOARD_URLS = {
"ncaam": "https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/scoreboard",
"ncaaw": "https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/scoreboard",
}
ROUND_ORDER = {"NCG": 0, "F4": 1, "E8": 2, "S16": 3, "R32": 4, "R64": 5, "": 6}
ROUND_DISPLAY_NAMES = {
"NCG": "Championship",
"F4": "Final Four",
"E8": "Elite Eight",
"S16": "Sweet Sixteen",
"R32": "Round of 32",
"R64": "Round of 64",
}
ROUND_LOGO_FILES = {
"NCG": "CHAMPIONSHIP.png",
"F4": "FINAL_4.png",
"E8": "ELITE_8.png",
"S16": "SWEET_16.png",
"R32": "ROUND_32.png",
"R64": "ROUND_64.png",
}
REGION_ORDER = {"E": 0, "W": 1, "S": 2, "MW": 3, "": 4}
# Colors
COLOR_WHITE = (255, 255, 255)
COLOR_GOLD = (255, 215, 0)
COLOR_GRAY = (160, 160, 160)
COLOR_DIM = (100, 100, 100)
COLOR_RED = (255, 60, 60)
COLOR_GREEN = (60, 200, 60)
COLOR_BLACK = (0, 0, 0)
COLOR_DARK_BG = (20, 20, 20)
# ---------------------------------------------------------------------------
# Plugin Class
# ---------------------------------------------------------------------------
class MarchMadnessPlugin(BasePlugin):
"""NCAA March Madness tournament bracket tracker."""
def __init__(
self,
plugin_id: str,
config: Dict[str, Any],
display_manager: Any,
cache_manager: Any,
plugin_manager: Any,
):
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# Config
leagues_config = config.get("leagues", {})
self.show_ncaam: bool = leagues_config.get("ncaam", True)
self.show_ncaaw: bool = leagues_config.get("ncaaw", True)
self.favorite_teams: List[str] = [t.upper() for t in config.get("favorite_teams", [])]
display_options = config.get("display_options", {})
self.show_seeds: bool = display_options.get("show_seeds", True)
self.show_round_logos: bool = display_options.get("show_round_logos", True)
self.highlight_upsets: bool = display_options.get("highlight_upsets", True)
self.show_bracket_progress: bool = display_options.get("show_bracket_progress", True)
self.scroll_speed: float = display_options.get("scroll_speed", 1.0)
self.scroll_delay: float = display_options.get("scroll_delay", 0.02)
self.target_fps: int = display_options.get("target_fps", 120)
self.loop: bool = display_options.get("loop", True)
self.dynamic_duration_enabled: bool = display_options.get("dynamic_duration", True)
self.min_duration: int = display_options.get("min_duration", 30)
self.max_duration: int = display_options.get("max_duration", 300)
if self.min_duration > self.max_duration:
self.logger.warning(
f"min_duration ({self.min_duration}) > max_duration ({self.max_duration}); swapping values"
)
self.min_duration, self.max_duration = self.max_duration, self.min_duration
data_settings = config.get("data_settings", {})
self.update_interval: int = data_settings.get("update_interval", 300)
self.request_timeout: int = data_settings.get("request_timeout", 30)
# Scrolling flag for display controller
self.enable_scrolling = True
# State
self.games_data: List[Dict] = []
self.ticker_image: Optional[Image.Image] = None
self.last_update: float = 0
self.dynamic_duration: float = 60
self.total_scroll_width: int = 0
self._display_start_time: Optional[float] = None
self._end_reached_logged: bool = False
self._update_lock = threading.Lock()
self._has_live_games: bool = False
self._cached_dynamic_duration: Optional[float] = None
self._duration_cache_time: float = 0
# Display dimensions
self.display_width: int = self.display_manager.matrix.width
self.display_height: int = self.display_manager.matrix.height
# HTTP session with retry
self.session = requests.Session()
retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504])
self.session.mount("https://", HTTPAdapter(max_retries=retry))
self.headers = {"User-Agent": "LEDMatrix/2.0"}
# ScrollHelper
if ScrollHelper:
self.scroll_helper = ScrollHelper(self.display_width, self.display_height, logger=self.logger)
if hasattr(self.scroll_helper, "set_frame_based_scrolling"):
self.scroll_helper.set_frame_based_scrolling(True)
self.scroll_helper.set_scroll_speed(self.scroll_speed)
self.scroll_helper.set_scroll_delay(self.scroll_delay)
if hasattr(self.scroll_helper, "set_target_fps"):
self.scroll_helper.set_target_fps(self.target_fps)
self.scroll_helper.set_dynamic_duration_settings(
enabled=self.dynamic_duration_enabled,
min_duration=self.min_duration,
max_duration=self.max_duration,
buffer=0.1,
)
else:
self.scroll_helper = None
self.logger.warning("ScrollHelper not available")
# Fonts
self.fonts = self._load_fonts()
# Logos
self._round_logos: Dict[str, Image.Image] = {}
self._team_logo_cache: Dict[str, Optional[Image.Image]] = {}
self._march_madness_logo: Optional[Image.Image] = None
self._load_round_logos()
self.logger.info(
f"MarchMadnessPlugin initialized — NCAAM: {self.show_ncaam}, "
f"NCAAW: {self.show_ncaaw}, favorites: {self.favorite_teams}"
)
# ------------------------------------------------------------------
# Fonts
# ------------------------------------------------------------------
def _load_fonts(self) -> Dict[str, ImageFont.FreeTypeFont]:
fonts = {}
try:
fonts["score"] = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 10)
except IOError:
fonts["score"] = ImageFont.load_default()
try:
fonts["time"] = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
except IOError:
fonts["time"] = ImageFont.load_default()
try:
fonts["detail"] = ImageFont.truetype("assets/fonts/4x6-font.ttf", 6)
except IOError:
fonts["detail"] = ImageFont.load_default()
return fonts
# ------------------------------------------------------------------
# Logo loading
# ------------------------------------------------------------------
def _load_round_logos(self) -> None:
logo_dir = Path("assets/sports/ncaa_logos")
for round_key, filename in ROUND_LOGO_FILES.items():
path = logo_dir / filename
try:
img = Image.open(path).convert("RGBA")
# Resize to fit display height
target_h = self.display_height - 4
ratio = target_h / img.height
target_w = int(img.width * ratio)
self._round_logos[round_key] = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
except (OSError, ValueError) as e:
self.logger.warning(f"Could not load round logo {filename}: {e}")
except Exception:
self.logger.exception(f"Unexpected error loading round logo {filename}")
# March Madness logo
mm_path = logo_dir / "MARCH_MADNESS.png"
try:
img = Image.open(mm_path).convert("RGBA")
target_h = self.display_height - 4
ratio = target_h / img.height
target_w = int(img.width * ratio)
self._march_madness_logo = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
except (OSError, ValueError) as e:
self.logger.warning(f"Could not load March Madness logo: {e}")
except Exception:
self.logger.exception("Unexpected error loading March Madness logo")
def _get_team_logo(self, abbr: str) -> Optional[Image.Image]:
if abbr in self._team_logo_cache:
return self._team_logo_cache[abbr]
logo_dir = Path("assets/sports/ncaa_logos")
path = logo_dir / f"{abbr}.png"
try:
img = Image.open(path).convert("RGBA")
target_h = self.display_height - 6
ratio = target_h / img.height
target_w = int(img.width * ratio)
img = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
self._team_logo_cache[abbr] = img
return img
except (FileNotFoundError, OSError, ValueError):
self._team_logo_cache[abbr] = None
return None
except Exception:
self.logger.exception(f"Unexpected error loading team logo for {abbr}")
self._team_logo_cache[abbr] = None
return None
# ------------------------------------------------------------------
# Data fetching
# ------------------------------------------------------------------
def _is_tournament_window(self) -> bool:
today = datetime.now(pytz.utc)
return (3, 10) <= (today.month, today.day) <= (4, 10)
def _fetch_tournament_data(self) -> List[Dict]:
"""Fetch tournament games from ESPN scoreboard API."""
all_games: List[Dict] = []
leagues = []
if self.show_ncaam:
leagues.append("ncaam")
if self.show_ncaaw:
leagues.append("ncaaw")
for league_key in leagues:
url = SCOREBOARD_URLS.get(league_key)
if not url:
continue
cache_key = f"march_madness_{league_key}_scoreboard"
cache_max_age = 60 if self._has_live_games else self.update_interval
cached = self.cache_manager.get(cache_key, max_age=cache_max_age)
if cached:
all_games.extend(cached)
continue
try:
# NCAA basketball scoreboard without dates param returns current games
params = {"limit": 1000, "groups": 100}
resp = self.session.get(url, params=params, headers=self.headers, timeout=self.request_timeout)
resp.raise_for_status()
data = resp.json()
events = data.get("events", [])
league_games = []
for event in events:
game = self._parse_event(event, league_key)
if game:
league_games.append(game)
self.cache_manager.set(cache_key, league_games)
self.logger.info(f"Fetched {len(league_games)} {league_key} tournament games")
all_games.extend(league_games)
except Exception:
self.logger.exception(f"Error fetching {league_key} tournament data")
return all_games
def _parse_event(self, event: Dict, league_key: str) -> Optional[Dict]:
"""Parse an ESPN event into a game dict."""
competitions = event.get("competitions", [])
if not competitions:
return None
comp = competitions[0]
# Confirm tournament game
comp_type = comp.get("type", {})
is_tournament = comp_type.get("abbreviation") == "TRNMNT"
notes = comp.get("notes", [])
headline = ""
if notes:
headline = notes[0].get("headline", "")
if not is_tournament and "Championship" in headline:
is_tournament = True
if not is_tournament:
return None
# Status
status = comp.get("status", {}).get("type", {})
state = status.get("state", "pre")
status_detail = status.get("shortDetail", "")
# Teams
competitors = comp.get("competitors", [])
home_team = next((c for c in competitors if c.get("homeAway") == "home"), None)
away_team = next((c for c in competitors if c.get("homeAway") == "away"), None)
if not home_team or not away_team:
return None
home_abbr = home_team.get("team", {}).get("abbreviation", "???")
away_abbr = away_team.get("team", {}).get("abbreviation", "???")
home_score = home_team.get("score", "0")
away_score = away_team.get("score", "0")
# Seeds
home_seed = home_team.get("curatedRank", {}).get("current", 0)
away_seed = away_team.get("curatedRank", {}).get("current", 0)
if home_seed >= 99:
home_seed = 0
if away_seed >= 99:
away_seed = 0
# Round and region
tournament_round = self._parse_round(headline)
tournament_region = self._parse_region(headline)
# Date/time
date_str = event.get("date", "")
start_time_utc = None
game_date = ""
game_time = ""
try:
if date_str.endswith("Z"):
date_str = date_str.replace("Z", "+00:00")
dt = datetime.fromisoformat(date_str)
if dt.tzinfo is None:
start_time_utc = dt.replace(tzinfo=pytz.UTC)
else:
start_time_utc = dt.astimezone(pytz.UTC)
local = start_time_utc.astimezone(pytz.timezone("US/Eastern"))
game_date = local.strftime("%-m/%-d")
game_time = local.strftime("%-I:%M%p").replace("AM", "am").replace("PM", "pm")
except (ValueError, AttributeError):
pass
# Period / clock for live games
period = 0
clock = ""
period_text = ""
is_halftime = False
if state == "in":
status_obj = comp.get("status", {})
period = status_obj.get("period", 0)
clock = status_obj.get("displayClock", "")
detail_lower = status_detail.lower()
uses_quarters = league_key == "ncaaw" or "quarter" in detail_lower or detail_lower.startswith("q")
if period <= (4 if uses_quarters else 2):
period_text = f"Q{period}" if uses_quarters else f"H{period}"
else:
ot_num = period - (4 if uses_quarters else 2)
period_text = f"OT{ot_num}" if ot_num > 1 else "OT"
if "halftime" in detail_lower:
is_halftime = True
elif state == "post":
period_text = status.get("shortDetail", "Final")
if "Final" not in period_text:
period_text = "Final"
# Determine winner and upset
is_final = state == "post"
is_upset = False
winner_side = ""
if is_final:
try:
h = int(float(home_score))
a = int(float(away_score))
if h > a:
winner_side = "home"
if home_seed > away_seed > 0:
is_upset = True
elif a > h:
winner_side = "away"
if away_seed > home_seed > 0:
is_upset = True
except (ValueError, TypeError):
pass
return {
"id": event.get("id", ""),
"league": league_key,
"home_abbr": home_abbr,
"away_abbr": away_abbr,
"home_score": str(home_score),
"away_score": str(away_score),
"home_seed": home_seed,
"away_seed": away_seed,
"tournament_round": tournament_round,
"tournament_region": tournament_region,
"state": state,
"is_final": is_final,
"is_live": state == "in",
"is_upcoming": state == "pre",
"is_halftime": is_halftime,
"period": period,
"period_text": period_text,
"clock": clock,
"status_detail": status_detail,
"game_date": game_date,
"game_time": game_time,
"start_time_utc": start_time_utc,
"is_upset": is_upset,
"winner_side": winner_side,
"headline": headline,
}
@staticmethod
def _parse_round(headline: str) -> str:
hl = headline.lower()
if "national championship" in hl:
return "NCG"
if "final four" in hl:
return "F4"
if "elite 8" in hl or "elite eight" in hl:
return "E8"
if "sweet 16" in hl or "sweet sixteen" in hl:
return "S16"
if "2nd round" in hl or "second round" in hl:
return "R32"
if "1st round" in hl or "first round" in hl:
return "R64"
return ""
@staticmethod
def _parse_region(headline: str) -> str:
if "East Region" in headline:
return "E"
if "West Region" in headline:
return "W"
if "South Region" in headline:
return "S"
if "Midwest Region" in headline:
return "MW"
m = re.search(r"Regional (\d+)", headline)
if m:
return f"R{m.group(1)}"
return ""
# ------------------------------------------------------------------
# Game processing
# ------------------------------------------------------------------
def _process_games(self, games: List[Dict]) -> Dict[str, List[Dict]]:
"""Group games by round, sorted by round significance then region/seed."""
grouped: Dict[str, List[Dict]] = {}
for game in games:
rnd = game.get("tournament_round", "")
grouped.setdefault(rnd, []).append(game)
# Sort each round's games by region then seed matchup
for rnd, round_games in grouped.items():
round_games.sort(
key=lambda g: (
REGION_ORDER.get(g.get("tournament_region", ""), 4),
min(g.get("away_seed", 99), g.get("home_seed", 99)),
)
)
return grouped
# ------------------------------------------------------------------
# Rendering
# ------------------------------------------------------------------
def _draw_text_with_outline(
self,
draw: ImageDraw.Draw,
text: str,
xy: tuple,
font: ImageFont.FreeTypeFont,
fill: tuple = COLOR_WHITE,
outline: tuple = COLOR_BLACK,
) -> None:
x, y = xy
for dx in (-1, 0, 1):
for dy in (-1, 0, 1):
if dx or dy:
draw.text((x + dx, y + dy), text, font=font, fill=outline)
draw.text((x, y), text, font=font, fill=fill)
def _create_round_separator(self, round_key: str) -> Image.Image:
"""Create a separator tile for a tournament round."""
height = self.display_height
name = ROUND_DISPLAY_NAMES.get(round_key, round_key)
font = self.fonts["time"]
# Measure text
tmp = Image.new("RGB", (1, 1))
tmp_draw = ImageDraw.Draw(tmp)
text_width = int(tmp_draw.textlength(name, font=font))
# Logo on each side
logo = self._round_logos.get(round_key, self._march_madness_logo)
logo_w = logo.width if logo else 0
padding = 6
total_w = padding + logo_w + padding + text_width + padding + logo_w + padding
total_w = max(total_w, 80)
img = Image.new("RGB", (total_w, height), COLOR_DARK_BG)
draw = ImageDraw.Draw(img)
# Draw logos
x = padding
if logo:
logo_y = (height - logo.height) // 2
img.paste(logo, (x, logo_y), logo)
x += logo_w + padding
# Draw round name
text_y = (height - 8) // 2 # 8px font
self._draw_text_with_outline(draw, name, (x, text_y), font, fill=COLOR_GOLD)
x += text_width + padding
if logo:
logo_y = (height - logo.height) // 2
img.paste(logo, (x, logo_y), logo)
return img
def _create_game_tile(self, game: Dict) -> Image.Image:
"""Create a single game tile for the scrolling ticker."""
height = self.display_height
font_score = self.fonts["score"]
font_time = self.fonts["time"]
font_detail = self.fonts["detail"]
# Load team logos
away_logo = self._get_team_logo(game["away_abbr"])
home_logo = self._get_team_logo(game["home_abbr"])
logo_w = 0
if away_logo:
logo_w = max(logo_w, away_logo.width)
if home_logo:
logo_w = max(logo_w, home_logo.width)
if logo_w == 0:
logo_w = 24
# Build text elements
away_seed_str = f"({game['away_seed']})" if self.show_seeds and game.get("away_seed", 0) > 0 else ""
home_seed_str = f"({game['home_seed']})" if self.show_seeds and game.get("home_seed", 0) > 0 else ""
away_text = f"{away_seed_str}{game['away_abbr']}"
home_text = f"{game['home_abbr']}{home_seed_str}"
# Measure text widths
tmp = Image.new("RGB", (1, 1))
tmp_draw = ImageDraw.Draw(tmp)
away_text_w = int(tmp_draw.textlength(away_text, font=font_detail))
home_text_w = int(tmp_draw.textlength(home_text, font=font_detail))
# Center content: status line
if game["is_live"]:
if game["is_halftime"]:
status_text = "Halftime"
else:
status_text = f"{game['period_text']} {game['clock']}".strip()
elif game["is_final"]:
status_text = game.get("period_text", "Final")
else:
status_text = f"{game['game_date']} {game['game_time']}".strip()
status_w = int(tmp_draw.textlength(status_text, font=font_time))
# Score line (for live/final)
score_text = ""
if game["is_live"] or game["is_final"]:
score_text = f"{game['away_score']}-{game['home_score']}"
score_w = int(tmp_draw.textlength(score_text, font=font_score)) if score_text else 0
# Calculate tile width
h_pad = 4
center_w = max(status_w, score_w, 40)
tile_w = h_pad + logo_w + h_pad + away_text_w + h_pad + center_w + h_pad + home_text_w + h_pad + logo_w + h_pad
img = Image.new("RGB", (tile_w, height), COLOR_BLACK)
draw = ImageDraw.Draw(img)
# Paste away logo
x = h_pad
if away_logo:
logo_y = (height - away_logo.height) // 2
img.paste(away_logo, (x, logo_y), away_logo)
x += logo_w + h_pad
# Away team text (seed + abbr)
is_fav_away = game["away_abbr"] in self.favorite_teams if self.favorite_teams else False
away_color = COLOR_GOLD if is_fav_away else COLOR_WHITE
if game["is_final"] and game["winner_side"] == "away" and self.highlight_upsets and game["is_upset"]:
away_color = COLOR_GOLD
team_text_y = (height - 6) // 2 - 5 # Upper half
self._draw_text_with_outline(draw, away_text, (x, team_text_y), font_detail, fill=away_color)
x += away_text_w + h_pad
# Center block
center_x = x
center_mid = center_x + center_w // 2
# Status text (top center of center block)
status_x = center_mid - status_w // 2
status_y = 2
status_color = COLOR_GREEN if game["is_live"] else COLOR_GRAY
self._draw_text_with_outline(draw, status_text, (status_x, status_y), font_time, fill=status_color)
# Score (bottom center of center block, for live/final)
if score_text:
score_x = center_mid - score_w // 2
score_y = height - 13
# Upset highlighting
if game["is_final"] and game["is_upset"] and self.highlight_upsets:
score_color = COLOR_GOLD
elif game["is_live"]:
score_color = COLOR_WHITE
else:
score_color = COLOR_WHITE
self._draw_text_with_outline(draw, score_text, (score_x, score_y), font_score, fill=score_color)
# Date for final games (below score)
if game["is_final"] and game.get("game_date"):
date_w = int(draw.textlength(game["game_date"], font=font_detail))
date_x = center_mid - date_w // 2
date_y = height - 6
self._draw_text_with_outline(draw, game["game_date"], (date_x, date_y), font_detail, fill=COLOR_DIM)
x = center_x + center_w + h_pad
# Home team text
is_fav_home = game["home_abbr"] in self.favorite_teams if self.favorite_teams else False
home_color = COLOR_GOLD if is_fav_home else COLOR_WHITE
if game["is_final"] and game["winner_side"] == "home" and self.highlight_upsets and game["is_upset"]:
home_color = COLOR_GOLD
self._draw_text_with_outline(draw, home_text, (x, team_text_y), font_detail, fill=home_color)
x += home_text_w + h_pad
# Paste home logo
if home_logo:
logo_y = (height - home_logo.height) // 2
img.paste(home_logo, (x, logo_y), home_logo)
return img
def _create_ticker_image(self) -> None:
"""Build the full scrolling ticker image from game tiles."""
if not self.games_data:
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
grouped = self._process_games(self.games_data)
content_items: List[Image.Image] = []
# Order rounds by significance (most important first)
sorted_rounds = sorted(grouped.keys(), key=lambda r: ROUND_ORDER.get(r, 6))
for rnd in sorted_rounds:
games = grouped[rnd]
if not games:
continue
# Add round separator
if self.show_round_logos and rnd:
separator = self._create_round_separator(rnd)
content_items.append(separator)
# Add game tiles
for game in games:
tile = self._create_game_tile(game)
content_items.append(tile)
if not content_items:
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
if not self.scroll_helper:
self.ticker_image = None
return
gap_width = 16
# Use ScrollHelper to create the scrolling image
self.ticker_image = self.scroll_helper.create_scrolling_image(
content_items=content_items,
item_gap=gap_width,
element_gap=0,
)
self.total_scroll_width = self.scroll_helper.total_scroll_width
self.dynamic_duration = self.scroll_helper.get_dynamic_duration()
self.logger.info(
f"Ticker image created: {self.ticker_image.width}px wide, "
f"{len(self.games_data)} games, dynamic_duration={self.dynamic_duration:.0f}s"
)
# ------------------------------------------------------------------
# Plugin lifecycle
# ------------------------------------------------------------------
def update(self) -> None:
"""Fetch and process tournament data."""
if not self.enabled:
return
current_time = time.time()
# Use shorter interval if live games detected
interval = 60 if self._has_live_games else self.update_interval
if current_time - self.last_update < interval:
return
with self._update_lock:
self.last_update = current_time
if not self._is_tournament_window():
self.logger.debug("Outside tournament window, skipping fetch")
self.games_data = []
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
try:
games = self._fetch_tournament_data()
self._has_live_games = any(g["is_live"] for g in games)
self.games_data = games
self._create_ticker_image()
self.logger.info(
f"Updated: {len(games)} games, "
f"live={self._has_live_games}"
)
except Exception as e:
self.logger.error(f"Update error: {e}", exc_info=True)
def display(self, force_clear: bool = False) -> None:
"""Render one scroll frame."""
if not self.enabled:
return
if force_clear or self._display_start_time is None:
self._display_start_time = time.time()
if self.scroll_helper:
self.scroll_helper.reset_scroll()
self._end_reached_logged = False
if not self.games_data or self.ticker_image is None:
self._display_fallback()
return
if not self.scroll_helper:
self._display_fallback()
return
try:
if self.loop or not self.scroll_helper.is_scroll_complete():
self.scroll_helper.update_scroll_position()
elif not self._end_reached_logged:
self.logger.info("Scroll complete")
self._end_reached_logged = True
visible = self.scroll_helper.get_visible_portion()
if visible is None:
self._display_fallback()
return
self.dynamic_duration = self.scroll_helper.get_dynamic_duration()
matrix_w = self.display_manager.matrix.width
matrix_h = self.display_manager.matrix.height
if not hasattr(self.display_manager, "image") or self.display_manager.image is None:
self.display_manager.image = Image.new("RGB", (matrix_w, matrix_h), COLOR_BLACK)
self.display_manager.image.paste(visible, (0, 0))
self.display_manager.update_display()
self.scroll_helper.log_frame_rate()
except Exception as e:
self.logger.error(f"Display error: {e}", exc_info=True)
self._display_fallback()
def _display_fallback(self) -> None:
w = self.display_manager.matrix.width
h = self.display_manager.matrix.height
img = Image.new("RGB", (w, h), COLOR_BLACK)
draw = ImageDraw.Draw(img)
if self._is_tournament_window():
text = "No games"
else:
text = "Off-season"
text_w = int(draw.textlength(text, font=self.fonts["time"]))
text_x = (w - text_w) // 2
text_y = (h - 8) // 2
draw.text((text_x, text_y), text, font=self.fonts["time"], fill=COLOR_GRAY)
# Show March Madness logo if available
if self._march_madness_logo:
logo_y = (h - self._march_madness_logo.height) // 2
img.paste(self._march_madness_logo, (2, logo_y), self._march_madness_logo)
self.display_manager.image = img
self.display_manager.update_display()
# ------------------------------------------------------------------
# Duration / cycle management
# ------------------------------------------------------------------
def get_display_duration(self) -> float:
current_time = time.time()
if self._cached_dynamic_duration is not None:
cache_age = current_time - self._duration_cache_time
if cache_age < 5.0:
return self._cached_dynamic_duration
self._cached_dynamic_duration = self.dynamic_duration
self._duration_cache_time = current_time
return self.dynamic_duration
def supports_dynamic_duration(self) -> bool:
if not self.enabled:
return False
return self.dynamic_duration_enabled
def is_cycle_complete(self) -> bool:
if not self.supports_dynamic_duration():
return True
if self._display_start_time is not None and self.dynamic_duration > 0:
elapsed = time.time() - self._display_start_time
if elapsed >= self.dynamic_duration:
return True
if not self.loop and self.scroll_helper and self.scroll_helper.is_scroll_complete():
return True
return False
def reset_cycle_state(self) -> None:
super().reset_cycle_state()
self._display_start_time = None
self._end_reached_logged = False
if self.scroll_helper:
self.scroll_helper.reset_scroll()
# ------------------------------------------------------------------
# Vegas mode
# ------------------------------------------------------------------
def get_vegas_content(self):
if not self.games_data:
return None
tiles = []
for game in self.games_data:
tiles.append(self._create_game_tile(game))
return tiles if tiles else None
def get_vegas_content_type(self) -> str:
return "multi"
# ------------------------------------------------------------------
# Info / cleanup
# ------------------------------------------------------------------
def get_info(self) -> Dict:
info = super().get_info()
info["total_games"] = len(self.games_data)
info["has_live_games"] = self._has_live_games
info["dynamic_duration"] = self.dynamic_duration
info["tournament_window"] = self._is_tournament_window()
return info
def cleanup(self) -> None:
self.games_data = []
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
self._team_logo_cache.clear()
if self.session:
self.session.close()
self.session = None
super().cleanup()

View File

@@ -0,0 +1,37 @@
{
"id": "march-madness",
"name": "March Madness",
"version": "1.0.0",
"description": "NCAA March Madness tournament bracket tracker with round branding, seeded matchups, live scores, and upset highlighting",
"author": "ChuckBuilds",
"category": "sports",
"tags": [
"ncaa",
"basketball",
"march-madness",
"tournament",
"bracket",
"scrolling"
],
"repo": "https://github.com/ChuckBuilds/ledmatrix-plugins",
"branch": "main",
"plugin_path": "plugins/march-madness",
"versions": [
{
"version": "1.0.0",
"ledmatrix_min": "2.0.0",
"released": "2026-02-16"
}
],
"stars": 0,
"downloads": 0,
"last_updated": "2026-02-16",
"verified": true,
"screenshot": "",
"display_modes": [
"march_madness"
],
"dependencies": {},
"entry_point": "manager.py",
"class_name": "MarchMadnessPlugin"
}

View File

@@ -0,0 +1,4 @@
requests>=2.28.0
Pillow>=9.1.0
pytz>=2022.1
numpy>=1.24.0

View File

@@ -22,6 +22,5 @@
"Pillow>=10.0.0",
"PyYAML>=6.0",
"requests>=2.31.0"
],
"local_only": true
]
}

View File

@@ -1,3 +1,3 @@
Pillow>=12.2.0
Pillow>=10.4.0
PyYAML>=6.0.2
requests>=2.33.0
requests>=2.32.0

View File

@@ -35,24 +35,24 @@ class WebUIInfoPlugin(BasePlugin):
"""Initialize the Web UI Info plugin."""
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# AP mode cache (must be initialized before _get_local_ip)
self._ap_mode_cached = False
self._ap_mode_cache_time = 0.0
self._ap_mode_cache_ttl = 60.0
# Get device hostname
try:
self.device_id = socket.gethostname()
except Exception as e:
self.logger.warning(f"Could not get hostname: {e}, using 'localhost'")
self.device_id = "localhost"
# Get device IP address
self.device_ip = self._get_local_ip()
# IP refresh tracking
self.last_ip_refresh = time.time()
self.ip_refresh_interval = 300.0
self.ip_refresh_interval = 300.0 # Refresh IP every 5 minutes
# AP mode cache
self._ap_mode_cached = False
self._ap_mode_cache_time = 0.0
self._ap_mode_cache_ttl = 60.0 # Cache AP mode check for 60 seconds
# Rotation state
self.current_display_mode = "hostname" # "hostname" or "ip"
@@ -200,7 +200,9 @@ class WebUIInfoPlugin(BasePlugin):
elif current_interface == "wlan0":
self.logger.debug(f"Found WiFi IP: {ip} on {current_interface}")
return ip
except Exception:
pass
# Last resort: try hostname resolution (often returns 127.0.0.1)
try:
ip = socket.gethostbyname(socket.gethostname())

View File

@@ -3,7 +3,7 @@
# Tested on Raspbian OS 12 (Bookworm) and 13 (Trixie)
# Image processing
Pillow>=12.2.0,<13.0.0
Pillow>=10.4.0,<12.0.0
numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
# Timezone handling
@@ -12,7 +12,7 @@ timezonefinder>=6.5.0,<7.0.0 # Updated for better performance and accuracy
geopy>=2.4.1,<3.0.0
# HTTP requests
requests>=2.33.0,<3.0.0
requests>=2.32.0,<3.0.0
# Google API integration
google-auth-oauthlib>=1.2.0,<2.0.0
@@ -23,10 +23,10 @@ google-api-python-client>=2.147.0,<3.0.0
freetype-py>=2.5.1,<3.0.0
# Spotify integration
spotipy>=2.25.2,<3.0.0
spotipy>=2.24.0,<3.0.0
# Flask web framework
Flask>=3.1.3,<4.0.0
Flask>=3.0.0,<4.0.0
# Text processing
unidecode>=1.3.8,<2.0.0
@@ -35,7 +35,7 @@ unidecode>=1.3.8,<2.0.0
icalevents>=0.1.27,<1.0.0
# WebSocket support
python-socketio>=5.14.0,<6.0.0
python-socketio>=5.11.0,<6.0.0
python-engineio>=4.9.0,<5.0.0
websockets>=12.0,<14.0
websocket-client>=1.8.0,<2.0.0
@@ -44,7 +44,7 @@ websocket-client>=1.8.0,<2.0.0
jsonschema>=4.20.0,<5.0.0
# Testing dependencies
pytest>=9.0.3,<10.0.0
pytest>=7.4.0,<8.0.0
pytest-cov>=4.1.0,<5.0.0
pytest-mock>=3.11.0,<4.0.0
mypy>=1.5.0,<2.0.0

1
run.py
View File

@@ -51,6 +51,7 @@ if debug_mode:
# Try to import the plugin system directly to get better error info
print("DEBUG: Attempting to import src.plugin_system...", flush=True)
from src.plugin_system import PluginManager
print("DEBUG: Plugin system import successful", flush=True)
except ImportError as e:
print(f"DEBUG: Plugin system import failed: {e}", flush=True)

View File

@@ -9,7 +9,7 @@ and preventing validation errors.
import json
import sys
from pathlib import Path
from typing import Any, Dict, List
from typing import Any, Dict, List, Optional
def get_default_for_field(prop: Dict[str, Any]) -> Any:

View File

@@ -9,8 +9,9 @@ Analyze all plugin config schemas to identify issues:
"""
import json
import os
from pathlib import Path
from typing import Dict, List, Any
from typing import Dict, List, Set, Any
import jsonschema
from jsonschema import Draft7Validator

View File

@@ -3,6 +3,8 @@
Check what imports are actually in the app.py file on the Pi
"""
import sys
import os
from pathlib import Path
# Read the app.py file and check the import lines

View File

@@ -67,9 +67,8 @@ def main():
print(" 📍 Will run on: http://0.0.0.0:5000")
print(" ⏹️ Press Ctrl+C to stop")
# Run the app (debug mode controlled by env var to satisfy security scanners)
_debug = os.environ.get('LEDMATRIX_FLASK_DEBUG', '0') == '1'
app.run(host='0.0.0.0', port=5000, debug=_debug)
# Run the app (this should start the server)
app.run(host='0.0.0.0', port=5000, debug=True)
except KeyboardInterrupt:
print("\n ⏹️ Server stopped by user")

View File

@@ -6,7 +6,7 @@
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
PROJECT_ROOT="$SCRIPT_DIR"
PLUGINS_DIR="$PROJECT_ROOT/plugins"
CONFIG_FILE="$PROJECT_ROOT/dev_plugins.json"
DEFAULT_DEV_DIR="$HOME/.ledmatrix-dev-plugins"
@@ -203,7 +203,7 @@ link_github_plugin() {
log_info "Repository already exists at $target_dir"
if [[ -d "$target_dir/.git" ]]; then
log_info "Updating repository..."
(cd "$target_dir" && git pull --rebase) || true
(cd "$target_dir" && git pull --rebase || true)
fi
else
# Clone the repository

View File

@@ -0,0 +1 @@
/home/chuck/.ledmatrix-dev-plugins/ledmatrix-of-the-day

View File

@@ -1,95 +0,0 @@
#!/usr/bin/env python3
"""
Pillow compatibility smoke test.
Exercises the Pillow APIs used throughout LEDMatrix to verify a new
Pillow version doesn't break image rendering, font handling, or resize ops.
Run after upgrading Pillow:
python3 scripts/dev/test_pillow_compat.py
"""
import sys
def check(label, fn):
try:
result = fn()
print(f"{label}" + (f"{result}" if result is not None else ""))
return True
except Exception as e:
print(f"{label}{type(e).__name__}: {e}", file=sys.stderr)
return False
def main():
from PIL import Image, ImageDraw, ImageFont
import PIL
print(f"Pillow {PIL.__version__} on Python {sys.version.split()[0]}\n")
failures = 0
print("Image creation:")
failures += not check("Image.new RGB",
lambda: Image.new('RGB', (128, 32), (0, 0, 0)).size)
failures += not check("Image.new RGBA",
lambda: Image.new('RGBA', (64, 64), (255, 0, 0, 128)).size)
failures += not check("Image.new 1-bit",
lambda: Image.new('1', (16, 16)).size)
print("\nDraw operations:")
img = Image.new('RGB', (128, 32), (0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.load_default()
failures += not check("draw.rectangle",
lambda: draw.rectangle([0, 0, 127, 31], outline=(255, 0, 0)))
failures += not check("draw.text",
lambda: draw.text((2, 2), "Hello", fill=(255, 255, 255), font=font))
failures += not check("draw.line",
lambda: draw.line([0, 0, 127, 31], fill=(0, 255, 0)))
print("\nFont metrics (used in text_helper, scroll_helper):")
failures += not check("draw.textlength",
lambda: f"{draw.textlength('Test', font=font):.1f}px")
failures += not check("draw.textbbox",
lambda: draw.textbbox((0, 0), "Test", font=font))
print("\nResampling (used in logo_helper, image_utils, sports base):")
logo = Image.new('RGBA', (200, 200), (255, 128, 0, 200))
failures += not check("Image.Resampling.LANCZOS exists",
lambda: str(Image.Resampling.LANCZOS))
failures += not check("thumbnail with LANCZOS",
lambda: (logo.thumbnail((64, 32), Image.Resampling.LANCZOS), logo.size)[1])
big = Image.new('RGB', (300, 300), (0, 128, 255))
failures += not check("resize with LANCZOS",
lambda: big.resize((128, 32), Image.Resampling.LANCZOS).size)
print("\nComposite / paste (used in display rendering):")
base = Image.new('RGB', (128, 32), (0, 0, 0))
overlay = Image.new('RGBA', (32, 32), (255, 0, 0, 128))
failures += not check("paste RGBA onto RGB",
lambda: (base.paste(overlay.convert('RGB'), (0, 0)), base.size)[1])
failures += not check("Image.alpha_composite",
lambda: Image.alpha_composite(
Image.new('RGBA', (32, 32)), overlay).size)
print("\nImage I/O:")
import io
buf = io.BytesIO()
img.save(buf, format='PNG')
buf.seek(0)
failures += not check("save/load PNG roundtrip",
lambda: Image.open(buf).size)
print()
if failures == 0:
print(f"All checks passed. Pillow {PIL.__version__} is compatible.")
return 0
else:
print(f"{failures} check(s) failed — review output above.", file=sys.stderr)
return 1
if __name__ == '__main__':
sys.exit(main())

View File

@@ -15,6 +15,7 @@ Usage: python tools/validate_python.py <python_file>
import ast
import sys
import os
from pathlib import Path
def validate_file(filepath: str) -> bool:
"""Validate a Python file for common issues."""

View File

@@ -13,6 +13,7 @@ echo ""
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Get the actual user

View File

@@ -41,7 +41,7 @@ if [ -f "$PROJECT_DIR/config/config.json" ]; then
echo -e "${GREEN}✓ Config file found${NC}"
# Check web_display_autostart setting
AUTOSTART=$(grep -o '"web_display_autostart"[[:space:]]*:[[:space:]]*[a-z]*' "$PROJECT_DIR/config/config.json" | grep -o '[a-z]*$')
AUTOSTART=$(cat "$PROJECT_DIR/config/config.json" | grep -o '"web_display_autostart"[[:space:]]*:[[:space:]]*[a-z]*' | grep -o '[a-z]*$')
if [ "$AUTOSTART" == "true" ]; then
echo -e "${GREEN}✓ web_display_autostart: true${NC}"

View File

@@ -16,8 +16,11 @@ YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Check if running as root or with sudo
if [ "$EUID" -ne 0 ]; then
if [ "$EUID" -ne 0 ]; then
echo -e "${YELLOW}Warning: Some checks require sudo. Running what we can...${NC}"
SUDO=""
else
SUDO=""
fi
PROJECT_DIR="${HOME}/LEDMatrix"

View File

@@ -118,7 +118,7 @@ total_count=${#ARCHITECTURES[@]}
for arch in "${!ARCHITECTURES[@]}"; do
if download_binary "$arch" "${ARCHITECTURES[$arch]}"; then
success_count=$((success_count + 1))
((success_count++))
fi
done

View File

@@ -7,6 +7,12 @@ echo "Fixing LEDMatrix assets directory permissions..."
# Get the real user (not root when running with sudo)
REAL_USER=${SUDO_USER:-$USER}
# Resolve the home directory of the real user robustly
if command -v getent >/dev/null 2>&1; then
REAL_HOME=$(getent passwd "$REAL_USER" | cut -d: -f6)
else
REAL_HOME=$(eval echo ~"$REAL_USER")
fi
REAL_GROUP=$(id -gn "$REAL_USER")
# Get the project directory

View File

@@ -89,9 +89,9 @@ TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web"
# Optional: journalctl (non-critical — skip if not found)
if [ -n "$JOURNALCTL_PATH" ]; then

View File

@@ -14,6 +14,9 @@ else
ACTUAL_USER=$(whoami)
fi
# Get the home directory of the actual user
USER_HOME=$(eval echo ~$ACTUAL_USER)
# Determine the Project Root Directory (parent of scripts/install/)
PROJECT_ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
@@ -31,8 +34,7 @@ echo "Generating service file with dynamic paths..."
WEB_SERVICE_FILE_CONTENT=$(cat <<EOF
[Unit]
Description=LED Matrix Web Interface Service
After=network-online.target
Wants=network-online.target
After=network.target
[Service]
Type=simple

View File

@@ -81,7 +81,7 @@ def main() -> int:
help='Plugin config as JSON string')
parser.add_argument('--mock-data', '-m', default=None,
help='Path to JSON file with mock cache data')
parser.add_argument('--output', '-o', default='/tmp/plugin_render.png', # nosec B108 - dev script default; user can override
parser.add_argument('--output', '-o', default='/tmp/plugin_render.png',
help='Output PNG path (default: /tmp/plugin_render.png)')
parser.add_argument('--width', type=int, default=128, help='Display width (default: 128)')
parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)')

View File

@@ -7,7 +7,9 @@ Supports both unittest and pytest.
"""
import sys
import os
import argparse
import subprocess
from pathlib import Path
from typing import Optional
@@ -196,14 +198,17 @@ def main():
if runner == 'auto':
# Try pytest first, fall back to unittest
try:
import pytest
runner = 'pytest'
except ImportError:
runner = 'unittest'
# Run tests
if runner == 'pytest':
import importlib.util
return run_pytest_tests(test_files, args.verbose, args.coverage)
else:
import importlib.util
return run_unittest_tests(test_files, args.verbose)

View File

@@ -6,7 +6,9 @@ This script allows manual clearing of specific cache keys or all cache data.
import os
import sys
import json
import argparse
from pathlib import Path
# Add the src directory to the path so we can import our modules
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src'))

View File

@@ -111,7 +111,7 @@ def main():
# Ensure PYTHONPATH is set correctly if web_interface.py has relative imports to src
# The WorkingDirectory in systemd service should handle this for web_interface.py
print(f"Launching web interface v3: {sys.executable} {WEB_INTERFACE_SCRIPT}")
os.execvp(sys.executable, [sys.executable, WEB_INTERFACE_SCRIPT]) # nosec B606 - both args are fixed constants
os.execvp(sys.executable, [sys.executable, WEB_INTERFACE_SCRIPT])
except Exception as e:
print(f"Failed to exec web interface: {e}")
sys.exit(1) # Failed to start

View File

@@ -10,7 +10,6 @@ import sys
import time
import logging
import signal
import subprocess
from pathlib import Path
# Add project root to path (parent of scripts/utils/)
@@ -44,11 +43,7 @@ class WiFiMonitorDaemon:
self.wifi_manager = WiFiManager()
self.running = True
self.last_state = None
# Counts consecutive checks where nmcli says "connected" but internet is unreachable.
# After _nm_restart_threshold failures, NetworkManager is restarted as a recovery step.
self._consecutive_internet_failures = 0
self._nm_restart_threshold = 5 # ~2.5 min at 30s interval
# Register signal handlers for graceful shutdown
signal.signal(signal.SIGINT, self._signal_handler)
signal.signal(signal.SIGTERM, self._signal_handler)
@@ -127,43 +122,6 @@ class WiFiMonitorDaemon:
else:
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
# Escalating recovery: if nmcli reports connected but actual internet
# is unreachable for several consecutive checks, restart NetworkManager.
# This is done HERE (not inside check_and_manage_ap_mode) to keep the
# AP-enable trigger clean and avoid false-positive AP enables from
# transient packet loss on otherwise working WiFi.
if updated_status.connected and not updated_status.ap_mode_active:
if not self.wifi_manager.check_internet_connectivity():
self._consecutive_internet_failures += 1
logger.warning(
f"Internet unreachable despite nmcli connection "
f"({self._consecutive_internet_failures}/{self._nm_restart_threshold})"
)
if self._consecutive_internet_failures >= self._nm_restart_threshold:
logger.warning("Restarting NetworkManager to recover internet connectivity")
try:
subprocess.run(
["/usr/bin/systemctl", "restart", "NetworkManager"],
capture_output=True, timeout=20, check=True
)
self._consecutive_internet_failures = 0
# NM restart causes a brief WiFi drop; reset the AP-mode grace
# counter so that transient disconnect doesn't count toward
# triggering AP mode.
self.wifi_manager._disconnected_checks = 0
except subprocess.CalledProcessError as e:
logger.error(f"NetworkManager restart failed (rc={e.returncode}); "
"resetting failure counter to avoid tight retry loop")
self._consecutive_internet_failures = 0
except (subprocess.SubprocessError, OSError) as e:
logger.error(f"NetworkManager restart error: {e}; "
"resetting failure counter to avoid tight retry loop")
self._consecutive_internet_failures = 0
else:
self._consecutive_internet_failures = 0
else:
self._consecutive_internet_failures = 0
# Sleep until next check
time.sleep(self.check_interval)

View File

@@ -7,7 +7,10 @@ where Recent/Upcoming managers consume data from the background service cache.
"""
import time
import logging
from typing import Dict, Optional, Any, Callable
from datetime import datetime
import pytz
class BackgroundCacheMixin:

View File

@@ -14,15 +14,19 @@ Key Features:
- Memory-efficient data storage
"""
import os
import time
import logging
import threading
import requests
from typing import Dict, Any, Optional, Callable
from typing import Dict, Any, Optional, List, Callable, Union
from datetime import datetime, timedelta
from dataclasses import dataclass, field
from enum import Enum
import json
import queue
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import ThreadPoolExecutor, Future
import weakref
from src.cache_manager import CacheManager
# Configure logging
logger = logging.getLogger(__name__)
@@ -223,7 +227,7 @@ class BackgroundDataService:
self.stats['cache_misses'] += 1
# Submit to executor
self.executor.submit(self._fetch_data_worker, request)
future = self.executor.submit(self._fetch_data_worker, request)
logger.info(f"Submitted background fetch request {request_id} for {sport} {year}")
return request_id
@@ -549,12 +553,13 @@ class BackgroundDataService:
if to_remove:
logger.info(f"Cleared {len(to_remove)} old completed requests")
def shutdown(self, wait: bool = True):
def shutdown(self, wait: bool = True, timeout: int = 30):
"""
Shutdown the background data service.
Args:
wait: Whether to wait for active requests to complete
timeout: Maximum time to wait for shutdown
"""
logger.info("Shutting down BackgroundDataService...")
@@ -565,14 +570,24 @@ class BackgroundDataService:
for request_id in list(self.active_requests.keys()):
self.cancel_request(request_id)
self.executor.shutdown(wait=wait)
# Shutdown executor with compatibility for older Python versions
try:
# Try with timeout parameter (Python 3.9+)
self.executor.shutdown(wait=wait, timeout=timeout)
except TypeError:
# Fallback for older Python versions that don't support timeout
if wait and timeout:
# For older versions, we can't specify timeout, so just wait
self.executor.shutdown(wait=True)
else:
self.executor.shutdown(wait=wait)
logger.info("BackgroundDataService shutdown complete")
def __del__(self):
"""Cleanup when service is destroyed."""
if not self._shutdown:
self.shutdown(wait=False)
self.shutdown(wait=False, timeout=None)
# Global service instance
_background_service: Optional[BackgroundDataService] = None

View File

@@ -1,605 +0,0 @@
"""
User configuration backup and restore.
Packages the user's LEDMatrix configuration, secrets, WiFi settings,
user-uploaded fonts, plugin image uploads, and installed-plugin manifest
into a single ``.zip`` that can be exported from one installation and
imported on a fresh install.
This module is intentionally Flask-free so it can be unit-tested and
used from scripts.
"""
from __future__ import annotations
import json
import logging
import os
import shutil
import socket
import tempfile
import zipfile
from dataclasses import dataclass, field, asdict
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
logger = logging.getLogger(__name__)
SCHEMA_VERSION = 1
# Filenames shipped with the LEDMatrix repository under ``assets/fonts/``.
# Anything present on disk but NOT in this set is treated as a user upload
# and included in backups. Keep this snapshot in sync with the repo — regenerate
# with::
#
# ls assets/fonts/
#
# Tests assert the set matches the checked-in fonts.
BUNDLED_FONTS: frozenset[str] = frozenset({
"10x20.bdf",
"4x6.bdf",
"4x6-font.ttf",
"5by7.regular.ttf",
"5x7.bdf",
"5x8.bdf",
"6x9.bdf",
"6x10.bdf",
"6x12.bdf",
"6x13.bdf",
"6x13B.bdf",
"6x13O.bdf",
"7x13.bdf",
"7x13B.bdf",
"7x13O.bdf",
"7x14.bdf",
"7x14B.bdf",
"8x13.bdf",
"8x13B.bdf",
"8x13O.bdf",
"9x15.bdf",
"9x15B.bdf",
"9x18.bdf",
"9x18B.bdf",
"AUTHORS",
"bdf_font_guide",
"clR6x12.bdf",
"helvR12.bdf",
"ic8x8u.bdf",
"MatrixChunky8.bdf",
"MatrixChunky8X.bdf",
"MatrixLight6.bdf",
"MatrixLight6X.bdf",
"MatrixLight8X.bdf",
"PressStart2P-Regular.ttf",
"README",
"README.md",
"texgyre-27.bdf",
"tom-thumb.bdf",
})
# Relative paths inside the project that the backup knows how to round-trip.
_CONFIG_REL = Path("config/config.json")
_SECRETS_REL = Path("config/config_secrets.json")
_WIFI_REL = Path("config/wifi_config.json")
_FONTS_REL = Path("assets/fonts")
_PLUGIN_UPLOADS_REL = Path("assets/plugins")
_STATE_REL = Path("data/plugin_state.json")
MANIFEST_NAME = "manifest.json"
PLUGINS_MANIFEST_NAME = "plugins.json"
# Hard cap on the size of a single file we'll accept inside an uploaded ZIP
# to limit zip-bomb risk. 50 MB matches the existing plugin-image upload cap.
_MAX_MEMBER_BYTES = 50 * 1024 * 1024
# Hard cap on the total uncompressed size of an uploaded ZIP.
_MAX_TOTAL_BYTES = 200 * 1024 * 1024
# ---------------------------------------------------------------------------
# Data classes
# ---------------------------------------------------------------------------
@dataclass
class RestoreOptions:
"""Which sections of a backup should be restored."""
restore_config: bool = True
restore_secrets: bool = True
restore_wifi: bool = True
restore_fonts: bool = True
restore_plugin_uploads: bool = True
reinstall_plugins: bool = True
@dataclass
class RestoreResult:
"""Outcome of a restore operation."""
success: bool = False
restored: List[str] = field(default_factory=list)
skipped: List[str] = field(default_factory=list)
plugins_to_install: List[Dict[str, Any]] = field(default_factory=list)
plugins_installed: List[str] = field(default_factory=list)
plugins_failed: List[Dict[str, str]] = field(default_factory=list)
errors: List[str] = field(default_factory=list)
manifest: Dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> Dict[str, Any]:
return asdict(self)
# ---------------------------------------------------------------------------
# Manifest helpers
# ---------------------------------------------------------------------------
def _ledmatrix_version(project_root: Path) -> str:
"""Best-effort version string for the current install."""
version_file = project_root / "VERSION"
if version_file.exists():
try:
return version_file.read_text(encoding="utf-8").strip() or "unknown"
except OSError:
pass
head_file = project_root / ".git" / "HEAD"
if head_file.exists():
try:
head = head_file.read_text(encoding="utf-8").strip()
if head.startswith("ref: "):
ref = head[5:]
ref_path = project_root / ".git" / ref
if ref_path.exists():
return ref_path.read_text(encoding="utf-8").strip()[:12] or "unknown"
return head[:12] or "unknown"
except OSError:
pass
return "unknown"
def _build_manifest(contents: List[str], project_root: Path) -> Dict[str, Any]:
return {
"schema_version": SCHEMA_VERSION,
"created_at": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
"ledmatrix_version": _ledmatrix_version(project_root),
"hostname": socket.gethostname(),
"contents": contents,
}
# ---------------------------------------------------------------------------
# Installed-plugin enumeration
# ---------------------------------------------------------------------------
def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
"""
Return a list of currently-installed plugins suitable for the backup
manifest. Each entry has ``plugin_id`` and ``version``.
Reads ``data/plugin_state.json`` if present; otherwise walks the plugin
directory and reads each ``manifest.json``.
"""
plugins: Dict[str, Dict[str, Any]] = {}
state_file = project_root / _STATE_REL
if state_file.exists():
try:
with state_file.open("r", encoding="utf-8") as f:
state = json.load(f)
raw_plugins = state.get("states", {}) if isinstance(state, dict) else {}
if isinstance(raw_plugins, dict):
for plugin_id, info in raw_plugins.items():
if not isinstance(info, dict):
continue
plugins[plugin_id] = {
"plugin_id": plugin_id,
"version": info.get("version") or "",
"enabled": bool(info.get("enabled", True)),
}
except (OSError, json.JSONDecodeError) as e:
logger.warning("Could not read plugin_state.json: %s", e)
# Fall back to scanning plugin-repos/ for manifests.
plugins_root = project_root / "plugin-repos"
if plugins_root.exists():
for entry in sorted(plugins_root.iterdir()):
if not entry.is_dir():
continue
manifest = entry / "manifest.json"
if not manifest.exists():
continue
try:
with manifest.open("r", encoding="utf-8") as f:
data = json.load(f)
except (OSError, json.JSONDecodeError):
continue
plugin_id = data.get("id") or entry.name
if plugin_id not in plugins:
plugins[plugin_id] = {
"plugin_id": plugin_id,
"version": data.get("version", ""),
"enabled": True,
}
return sorted(plugins.values(), key=lambda p: p["plugin_id"])
# ---------------------------------------------------------------------------
# Font filtering
# ---------------------------------------------------------------------------
def iter_user_fonts(project_root: Path) -> List[Path]:
"""Return absolute paths to user-uploaded fonts (anything in
``assets/fonts/`` not listed in :data:`BUNDLED_FONTS`)."""
fonts_dir = project_root / _FONTS_REL
if not fonts_dir.exists():
return []
user_fonts: List[Path] = []
for entry in sorted(fonts_dir.iterdir()):
if entry.is_file() and entry.name not in BUNDLED_FONTS:
user_fonts.append(entry)
return user_fonts
def iter_plugin_uploads(project_root: Path) -> List[Path]:
"""Return every file under ``assets/plugins/*/uploads/`` (recursive)."""
plugin_root = project_root / _PLUGIN_UPLOADS_REL
if not plugin_root.exists():
return []
out: List[Path] = []
for plugin_dir in sorted(plugin_root.iterdir()):
if not plugin_dir.is_dir():
continue
uploads = plugin_dir / "uploads"
if not uploads.exists():
continue
for root, _dirs, files in os.walk(uploads):
for name in sorted(files):
out.append(Path(root) / name)
return out
# ---------------------------------------------------------------------------
# Export
# ---------------------------------------------------------------------------
def create_backup(
project_root: Path,
output_dir: Optional[Path] = None,
) -> Path:
"""
Build a backup ZIP and write it into ``output_dir`` (defaults to
``<project_root>/config/backups/exports/``). Returns the path to the
created file.
"""
project_root = Path(project_root).resolve()
if output_dir is None:
output_dir = project_root / "config" / "backups" / "exports"
output_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
hostname = socket.gethostname() or "ledmatrix"
safe_host = "".join(c for c in hostname if c.isalnum() or c in "-_") or "ledmatrix"
zip_name = f"ledmatrix-backup-{safe_host}-{timestamp}.zip"
zip_path = output_dir / zip_name
contents: List[str] = []
# Stream directly to a temp file so we never hold the whole ZIP in memory.
tmp_path = zip_path.with_suffix(".zip.tmp")
try:
with zipfile.ZipFile(tmp_path, "w", compression=zipfile.ZIP_DEFLATED) as zf:
# Config files.
if (project_root / _CONFIG_REL).exists():
zf.write(project_root / _CONFIG_REL, _CONFIG_REL.as_posix())
contents.append("config")
if (project_root / _SECRETS_REL).exists():
zf.write(project_root / _SECRETS_REL, _SECRETS_REL.as_posix())
contents.append("secrets")
if (project_root / _WIFI_REL).exists():
zf.write(project_root / _WIFI_REL, _WIFI_REL.as_posix())
contents.append("wifi")
# User-uploaded fonts.
user_fonts = iter_user_fonts(project_root)
if user_fonts:
for font in user_fonts:
arcname = font.relative_to(project_root).as_posix()
zf.write(font, arcname)
contents.append("fonts")
# Plugin uploads.
plugin_uploads = iter_plugin_uploads(project_root)
if plugin_uploads:
for upload in plugin_uploads:
arcname = upload.relative_to(project_root).as_posix()
zf.write(upload, arcname)
contents.append("plugin_uploads")
# Installed plugins manifest.
plugins = list_installed_plugins(project_root)
if plugins:
zf.writestr(
PLUGINS_MANIFEST_NAME,
json.dumps(plugins, indent=2),
)
contents.append("plugins")
# Manifest goes last so that `contents` reflects what we actually wrote.
manifest = _build_manifest(contents, project_root)
zf.writestr(MANIFEST_NAME, json.dumps(manifest, indent=2))
os.replace(tmp_path, zip_path)
except Exception:
tmp_path.unlink(missing_ok=True)
raise
logger.info("Created backup %s (%d bytes)", zip_path, zip_path.stat().st_size)
return zip_path
def preview_backup_contents(project_root: Path) -> Dict[str, Any]:
"""Return a summary of what ``create_backup`` would include."""
project_root = Path(project_root).resolve()
return {
"has_config": (project_root / _CONFIG_REL).exists(),
"has_secrets": (project_root / _SECRETS_REL).exists(),
"has_wifi": (project_root / _WIFI_REL).exists(),
"user_fonts": [p.name for p in iter_user_fonts(project_root)],
"plugin_uploads": len(iter_plugin_uploads(project_root)),
"plugins": list_installed_plugins(project_root),
}
# ---------------------------------------------------------------------------
# Validate
# ---------------------------------------------------------------------------
def _safe_extract_path(base_dir: Path, member_name: str) -> Optional[Path]:
"""Resolve a ZIP member name against ``base_dir`` and reject anything
that escapes it. Returns the resolved absolute path, or ``None`` if the
name is unsafe."""
# Reject absolute paths and Windows-style drives outright.
if member_name.startswith(("/", "\\")) or (len(member_name) >= 2 and member_name[1] == ":"):
return None
target = (base_dir / member_name).resolve()
try:
target.relative_to(base_dir.resolve())
except ValueError:
return None
return target
def validate_backup(zip_path: Path) -> Tuple[bool, str, Dict[str, Any]]:
"""
Inspect a backup ZIP without extracting to disk.
Returns ``(ok, error_message, manifest_dict)``. ``manifest_dict`` contains
the parsed manifest plus diagnostic fields:
- ``detected_contents``: list of section names present in the archive
- ``plugins``: parsed plugins.json if present
- ``total_uncompressed``: sum of uncompressed sizes
"""
zip_path = Path(zip_path)
if not zip_path.exists():
return False, f"Backup file not found: {zip_path}", {}
try:
with zipfile.ZipFile(zip_path, "r") as zf:
names = zf.namelist()
if MANIFEST_NAME not in names:
return False, "Backup is missing manifest.json", {}
total = 0
with tempfile.TemporaryDirectory() as _sandbox:
sandbox = Path(_sandbox)
for info in zf.infolist():
if info.file_size > _MAX_MEMBER_BYTES:
return False, f"Member {info.filename} is too large", {}
total += info.file_size
if total > _MAX_TOTAL_BYTES:
return False, "Backup exceeds maximum allowed size", {}
# Safety: reject members with unsafe paths up front.
if _safe_extract_path(sandbox, info.filename) is None:
return False, f"Unsafe path in backup: {info.filename}", {}
try:
manifest_raw = zf.read(MANIFEST_NAME).decode("utf-8")
manifest = json.loads(manifest_raw)
except (OSError, UnicodeDecodeError, json.JSONDecodeError):
return False, "Invalid manifest.json", {}
if not isinstance(manifest, dict) or "schema_version" not in manifest:
return False, "Invalid manifest structure", {}
if manifest.get("schema_version") != SCHEMA_VERSION:
return (
False,
f"Unsupported backup schema version: {manifest.get('schema_version')}",
{},
)
detected: List[str] = []
if _CONFIG_REL.as_posix() in names:
detected.append("config")
if _SECRETS_REL.as_posix() in names:
detected.append("secrets")
if _WIFI_REL.as_posix() in names:
detected.append("wifi")
if any(n.startswith(_FONTS_REL.as_posix() + "/") for n in names):
detected.append("fonts")
if any(
n.startswith(_PLUGIN_UPLOADS_REL.as_posix() + "/") and "/uploads/" in n
for n in names
):
detected.append("plugin_uploads")
plugins: List[Dict[str, Any]] = []
if PLUGINS_MANIFEST_NAME in names:
try:
plugins = json.loads(zf.read(PLUGINS_MANIFEST_NAME).decode("utf-8"))
if not isinstance(plugins, list):
plugins = []
else:
detected.append("plugins")
except (OSError, UnicodeDecodeError, json.JSONDecodeError):
plugins = []
result_manifest = dict(manifest)
result_manifest["detected_contents"] = detected
result_manifest["plugins"] = plugins
result_manifest["total_uncompressed"] = total
result_manifest["file_count"] = len(names)
return True, "", result_manifest
except zipfile.BadZipFile:
return False, "File is not a valid ZIP archive", {}
except OSError:
return False, "Could not read backup", {}
# ---------------------------------------------------------------------------
# Restore
# ---------------------------------------------------------------------------
def _extract_zip_safe(zip_path: Path, dest_dir: Path) -> None:
"""Extract ``zip_path`` into ``dest_dir`` rejecting any unsafe members."""
with zipfile.ZipFile(zip_path, "r") as zf:
for info in zf.infolist():
target = _safe_extract_path(dest_dir, info.filename)
if target is None:
raise ValueError(f"Unsafe path in backup: {info.filename}")
if info.is_dir():
target.mkdir(parents=True, exist_ok=True)
continue
target.parent.mkdir(parents=True, exist_ok=True)
with zf.open(info, "r") as src, open(target, "wb") as dst:
shutil.copyfileobj(src, dst, length=64 * 1024)
def _copy_file(src: Path, dst: Path) -> None:
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)
def restore_backup(
zip_path: Path,
project_root: Path,
options: Optional[RestoreOptions] = None,
) -> RestoreResult:
"""
Restore ``zip_path`` into ``project_root`` according to ``options``.
Plugin reinstalls are NOT performed here — the caller is responsible for
walking ``result.plugins_to_install`` and calling the store manager. This
keeps this module Flask-free and side-effect free beyond the filesystem.
"""
if options is None:
options = RestoreOptions()
project_root = Path(project_root).resolve()
result = RestoreResult()
ok, err, manifest = validate_backup(zip_path)
if not ok:
result.errors.append(err)
return result
result.manifest = manifest
with tempfile.TemporaryDirectory(prefix="ledmatrix_restore_") as tmp:
tmp_dir = Path(tmp)
try:
_extract_zip_safe(Path(zip_path), tmp_dir)
except (ValueError, zipfile.BadZipFile, OSError) as e:
result.errors.append(f"Failed to extract backup: {e}")
return result
# Main config.
if options.restore_config and (tmp_dir / _CONFIG_REL).exists():
try:
_copy_file(tmp_dir / _CONFIG_REL, project_root / _CONFIG_REL)
result.restored.append("config")
except OSError as e:
result.errors.append(f"Failed to restore config.json: {e}")
elif (tmp_dir / _CONFIG_REL).exists():
result.skipped.append("config")
# Secrets.
if options.restore_secrets and (tmp_dir / _SECRETS_REL).exists():
try:
_copy_file(tmp_dir / _SECRETS_REL, project_root / _SECRETS_REL)
result.restored.append("secrets")
except OSError as e:
result.errors.append(f"Failed to restore config_secrets.json: {e}")
elif (tmp_dir / _SECRETS_REL).exists():
result.skipped.append("secrets")
# WiFi.
if options.restore_wifi and (tmp_dir / _WIFI_REL).exists():
try:
_copy_file(tmp_dir / _WIFI_REL, project_root / _WIFI_REL)
result.restored.append("wifi")
except OSError as e:
result.errors.append(f"Failed to restore wifi_config.json: {e}")
elif (tmp_dir / _WIFI_REL).exists():
result.skipped.append("wifi")
# User fonts — skip anything that collides with a bundled font.
tmp_fonts = tmp_dir / _FONTS_REL
if options.restore_fonts and tmp_fonts.exists():
restored_count = 0
for font in sorted(tmp_fonts.iterdir()):
if not font.is_file():
continue
if font.name in BUNDLED_FONTS:
result.skipped.append(f"font:{font.name} (bundled)")
continue
try:
_copy_file(font, project_root / _FONTS_REL / font.name)
restored_count += 1
except OSError as e:
result.errors.append(f"Failed to restore font {font.name}: {e}")
if restored_count:
result.restored.append(f"fonts ({restored_count})")
elif tmp_fonts.exists():
result.skipped.append("fonts")
# Plugin uploads.
tmp_uploads = tmp_dir / _PLUGIN_UPLOADS_REL
if options.restore_plugin_uploads and tmp_uploads.exists():
count = 0
for root, _dirs, files in os.walk(tmp_uploads):
for name in files:
src = Path(root) / name
rel = src.relative_to(tmp_dir)
if "/uploads/" not in rel.as_posix():
result.errors.append(f"Rejected unexpected plugin path: {rel}")
continue
try:
_copy_file(src, project_root / rel)
count += 1
except OSError as e:
result.errors.append(f"Failed to restore {rel}: {e}")
if count:
result.restored.append(f"plugin_uploads ({count})")
elif tmp_uploads.exists():
result.skipped.append("plugin_uploads")
# Plugins list (for caller to reinstall).
if options.reinstall_plugins and (tmp_dir / PLUGINS_MANIFEST_NAME).exists():
try:
with (tmp_dir / PLUGINS_MANIFEST_NAME).open("r", encoding="utf-8") as f:
plugins = json.load(f)
if isinstance(plugins, list):
result.plugins_to_install = [
{"plugin_id": p.get("plugin_id"), "version": p.get("version", "")}
for p in plugins
if isinstance(p, dict) and p.get("plugin_id")
]
except (OSError, json.JSONDecodeError) as e:
result.errors.append(f"Could not read plugins.json: {e}")
result.success = not result.errors
return result

View File

@@ -7,7 +7,7 @@ fields and data structures.
"""
from abc import ABC, abstractmethod
from typing import Dict, Optional
from typing import Dict, Any, Optional, List
import logging
from datetime import datetime
import pytz
@@ -21,10 +21,12 @@ class APIDataExtractor(ABC):
@abstractmethod
def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract common game details from raw API data."""
pass
@abstractmethod
def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract sport-specific fields (downs, innings, periods, etc.)."""
pass
def _extract_common_details(self, game_event: Dict) -> tuple[Dict | None, Dict | None, Dict | None, Dict | None, Dict | None]:
"""Extract common game details that work across all sports."""

View File

@@ -329,6 +329,7 @@ class Baseball(SportsCore):
return
series_summary = game.get("series_summary", "")
font = self.fonts.get('detail', ImageFont.load_default())
bbox = draw_overlay.textbbox((0, 0), series_summary, font=self.fonts['time'])
height = bbox[3] - bbox[1]
shots_y = (self.display_height - height) // 2

View File

@@ -1,4 +1,6 @@
import logging
import time
from datetime import datetime, timezone
from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont

View File

@@ -6,10 +6,11 @@ to support different APIs and data providers.
"""
from abc import ABC, abstractmethod
from typing import Dict, List
from typing import Dict, Any, Optional, List
import requests
import logging
from datetime import datetime
from datetime import datetime, timedelta
import time
class DataSource(ABC):
"""Abstract base class for data sources."""
@@ -34,14 +35,17 @@ class DataSource(ABC):
@abstractmethod
def fetch_live_games(self, sport: str, league: str) -> List[Dict]:
"""Fetch live games for a sport/league."""
pass
@abstractmethod
def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]:
"""Fetch schedule for a sport/league within date range."""
pass
@abstractmethod
def fetch_standings(self, sport: str, league: str) -> Dict:
"""Fetch standings for a sport/league."""
pass
def get_headers(self) -> Dict[str, str]:
"""Get headers for API requests."""
@@ -213,7 +217,7 @@ class MLBAPIDataSource(DataSource):
response.raise_for_status()
data = response.json()
self.logger.debug("Fetched standings from MLB API")
self.logger.debug(f"Fetched standings from MLB API")
return data
except Exception as e:
@@ -292,7 +296,7 @@ class SoccerAPIDataSource(DataSource):
response.raise_for_status()
data = response.json()
self.logger.debug("Fetched standings from soccer API")
self.logger.debug(f"Fetched standings from soccer API")
return data
except Exception as e:

View File

@@ -1,8 +1,10 @@
from typing import Dict, Any, Optional
from typing import Dict, Any, Optional, List
from src.display_manager import DisplayManager
from src.cache_manager import CacheManager
from datetime import datetime, timezone, timedelta
import logging
from PIL import Image, ImageDraw, ImageFont
import time
from src.base_classes.data_sources import ESPNDataSource
from src.base_classes.sports import SportsCore, SportsLive

View File

@@ -1,4 +1,6 @@
import logging
import time
from datetime import datetime, timezone
from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont
@@ -77,6 +79,8 @@ class Hockey(SportsCore):
away_shots = round(home_team_saves / home_team_saves_per)
if away_team_saves_per > 0:
home_shots = round(away_team_saves / away_team_saves_per)
status_short = status["type"].get("shortDetail", "")
if situation and status["type"]["state"] == "in":
# Detect scoring events from status detail
# status_detail = status["type"].get("detail", "")

View File

@@ -5,7 +5,7 @@ import time
from abc import ABC, abstractmethod
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
from typing import Any, Callable, Dict, List, Optional
import pytz
import requests
@@ -172,8 +172,8 @@ class SportsCore(ABC):
try:
fallbacks.append(Path.home() / ".ledmatrix" / "logos" / self.sport_key)
except RuntimeError as e:
self.logger.debug("Could not resolve home directory (expected for service users): %s", e)
except Exception:
pass
fallbacks.append(Path(tempfile.gettempdir()) / "ledmatrix_logos" / self.sport_key)
@@ -416,6 +416,7 @@ class SportsCore(ABC):
league=self.league,
event_id=game['id'],
update_interval_seconds=update_interval,
is_live=is_live
)
if odds_data:

View File

@@ -11,10 +11,21 @@ Follows LEDMatrix configuration management patterns:
- Maintainable: Changes to odds logic affect all plugins
"""
import time
import logging
import requests
import json
from datetime import datetime, timedelta, timezone
from typing import Dict, Any, Optional, List
import pytz
# Import the API counter function from web interface
try:
from web_interface_v2 import increment_api_counter
except ImportError:
# Fallback if web interface is not available
def increment_api_counter(kind: str, count: int = 1):
pass
class BaseOddsManager:
@@ -120,7 +131,9 @@ class BaseOddsManager:
response = requests.get(url, timeout=self.request_timeout)
response.raise_for_status()
raw_data = response.json()
# Increment API counter for odds data
increment_api_counter('odds', 1)
self.logger.debug(f"Received raw odds data from ESPN: {json.dumps(raw_data, indent=2)}")
odds_data = self._extract_espn_data(raw_data)

View File

@@ -193,21 +193,19 @@ class CacheStrategy:
Data type string for strategy lookup
"""
key_lower = key.lower()
# Odds data — checked before the generic 'live' block below because
# live-odds cache keys (e.g. odds_espn_basketball_nba_<id>_live) contain
# both 'odds' AND 'live'. Without this ordering the 'live' check below
# would match first and return 'sports_live' (30 s TTL) instead of the
# correct 'odds_live' (120 s TTL).
# Odds data — checked FIRST because odds keys may also contain 'live'/'current'
# (e.g. odds_espn_nba_game_123_live). The odds TTL (120s for live, 1800s for
# upcoming) must win over the generic sports_live TTL (30s) to avoid hitting
# the ESPN odds API every 30 seconds per game.
if 'odds' in key_lower:
# For live games, use shorter cache; for upcoming games, use longer cache
if any(x in key_lower for x in ['live', 'current']):
return 'odds_live' # Live odds change more frequently
return 'odds' # Regular odds for upcoming games
return 'odds_live' # Live odds change more frequently (120s TTL)
return 'odds' # Regular odds for upcoming games (1800s TTL)
# Live sports data
# Live sports data (only reached if key does NOT contain 'odds')
if any(x in key_lower for x in ['live', 'current', 'scoreboard']):
if 'soccer' in key_lower:
return 'sports_live' # Soccer live data is very time-sensitive
return 'sports_live'
# Weather data

View File

@@ -13,6 +13,7 @@ import threading
from typing import Dict, Any, Optional, Protocol
from datetime import datetime
from src.exceptions import CacheError
class CacheStrategyProtocol(Protocol):
@@ -183,7 +184,7 @@ class DiskCache:
os.replace(tmp_path, cache_path)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
os.chmod(cache_path, 0o660)
except OSError:
pass # Non-critical if chmod fails
finally:
@@ -201,7 +202,7 @@ class DiskCache:
os.fsync(cache_file.fileno())
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
os.chmod(cache_path, 0o660)
except OSError:
pass # Non-critical if chmod fails
self.logger.debug("Wrote cache for %s directly (non-atomic)", key)
@@ -209,7 +210,7 @@ class DiskCache:
# If direct write also fails, try fallback location
self.logger.warning("Direct write failed for key '%s' to %s: %s", key, cache_path, write_error)
raise # Re-raise to trigger fallback logic
except (IOError, OSError, PermissionError):
except (IOError, OSError, PermissionError) as e:
# Attempt one-time fallback write to user's home cache directory
try:
# Try user's home cache directory as fallback
@@ -227,7 +228,7 @@ class DiskCache:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(fallback_path, 0o660) # nosec B103 - intentional; web UI and service share a group
os.chmod(fallback_path, 0o660)
except OSError:
pass # Non-critical if chmod fails
self.logger.debug("Cache wrote to fallback location: %s", fallback_path)

View File

@@ -7,6 +7,7 @@ from typing import Any, Dict, List, Optional
import logging
import threading
import tempfile
from pathlib import Path
from src.exceptions import CacheError
from src.cache.memory_cache import MemoryCache
from src.cache.disk_cache import DiskCache
@@ -110,7 +111,7 @@ class CacheManager:
if os.access(system_cache_dir, os.W_OK):
self.logger.info(f"Using system cache directory: {system_cache_dir}")
return system_cache_dir
except (OSError, IOError, PermissionError):
except (OSError, IOError, PermissionError) as perm_error:
# Permission errors are expected when running as non-root
self.logger.debug(f"Could not create system cache directory (permission denied): {system_cache_dir}")
except (OSError, IOError, PermissionError) as e:
@@ -319,43 +320,18 @@ class CacheManager:
return None
def clear_cache(self, key: Optional[str] = None) -> None:
"""Clear cache entries.
Pass a non-empty ``key`` to remove a single entry, or pass
``None`` (the default) to clear every cached entry. An empty
string is rejected to prevent accidental whole-cache wipes
from callers that pass through unvalidated input.
"""
if key is None:
"""Clear cache for a specific key or all keys."""
if key:
# Clear specific key
self._memory_cache_component.clear(key)
self._disk_cache_component.clear(key)
self.logger.info("Cleared cache for key: %s", key)
else:
# Clear all keys
memory_count = self._memory_cache_component.size()
self._memory_cache_component.clear()
self._disk_cache_component.clear()
self.logger.info("Cleared all cache: %d memory entries", memory_count)
return
if not isinstance(key, str) or not key:
raise ValueError(
"clear_cache(key) requires a non-empty string; "
"pass key=None to clear all entries"
)
# Clear specific key
self._memory_cache_component.clear(key)
self._disk_cache_component.clear(key)
self.logger.info("Cleared cache for key: %s", key)
def delete(self, key: str) -> None:
"""Remove a single cache entry.
Thin wrapper around :meth:`clear_cache` that **requires** a
non-empty string key — unlike ``clear_cache(None)`` it never
wipes every entry. Raises ``ValueError`` on ``None`` or an
empty string.
"""
if key is None or not isinstance(key, str) or not key:
raise ValueError("delete(key) requires a non-empty string key")
self.clear_cache(key)
def list_cache_files(self) -> List[Dict[str, Any]]:
"""List all cache files with metadata (key, age, size, path).

View File

@@ -5,10 +5,13 @@ Handles HTTP requests, caching, and ESPN API integration for LED matrix plugins.
Extracted from LEDMatrix core to provide reusable functionality for plugins.
"""
import json
import logging
import time
from datetime import datetime
from typing import Any, Dict, Optional
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Dict, List, Optional, Union
from urllib.parse import urlencode
import requests
from requests.adapters import HTTPAdapter

View File

@@ -5,9 +5,11 @@ This example shows how to refactor the basketball plugin to use the
ledmatrix-common package for cleaner, more maintainable code.
"""
import logging
from pathlib import Path
from typing import Any, Dict, List, Optional
from PIL import Image, ImageDraw
# Import common helpers
from src.common import (

View File

@@ -42,7 +42,7 @@ def test_utilities(display_width: int, display_height: int):
print(f"Testing LEDMatrix Common utilities with {display_width}x{display_height} display")
try:
from ledmatrix_common import LogoHelper, TextHelper, DisplayHelper, GameHelper, ConfigHelper
from ledmatrix_common import LogoHelper, TextHelper, APIHelper, DisplayHelper, GameHelper, ConfigHelper
# Test LogoHelper
print("Testing LogoHelper...")
@@ -63,12 +63,12 @@ def test_utilities(display_width: int, display_height: int):
# Test GameHelper
print("Testing GameHelper...")
GameHelper()
game_helper = GameHelper()
print("GameHelper initialized")
# Test ConfigHelper
print("Testing ConfigHelper...")
ConfigHelper()
config_helper = ConfigHelper()
print("ConfigHelper initialized")
print("All tests passed!")

View File

@@ -6,7 +6,7 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
"""
import logging
from typing import Any, Dict, Optional, Tuple
from typing import Any, Dict, List, Optional, Tuple, Union
from PIL import Image, ImageDraw, ImageFont
@@ -166,7 +166,8 @@ class DisplayHelper:
img = self.create_base_image(background_color)
draw = ImageDraw.Draw(img)
# Start text off-screen to the right
# Calculate text position (start off-screen to the right)
text_width = draw.textlength(text, font=font)
x_position = self.display_width
# Draw text
@@ -215,7 +216,8 @@ class DisplayHelper:
PIL Image with error message
"""
img = self.create_base_image((50, 0, 0)) # Dark red background
draw = ImageDraw.Draw(img)
# Use default font
font = ImageFont.load_default()
@@ -235,6 +237,8 @@ class DisplayHelper:
PIL Image with no data message
"""
img = self.create_base_image((0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.load_default()
self._draw_centered_text(message, font, (0, 0, 0), (150, 150, 150))

View File

@@ -6,8 +6,10 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
"""
import logging
import os
from pathlib import Path
from typing import Dict, List, Optional, Union
from urllib.parse import urlparse
import requests
from PIL import Image

View File

@@ -17,7 +17,7 @@ logger = logging.getLogger(__name__)
# System directories that should never have their permissions modified
# These directories have special system-level permissions that must be preserved
PROTECTED_SYSTEM_DIRECTORIES = { # nosec B108 - these are checked to PREVENT permission changes, not to use as temp paths
PROTECTED_SYSTEM_DIRECTORIES = {
'/tmp',
'/var/tmp',
'/dev',

View File

@@ -1,651 +0,0 @@
"""
Multi-Display Sync Manager
Synchronizes scrolling content across two LED matrix display units over UDP.
Runs at the core framework level — works with any plugin automatically.
Roles:
standalone No sync (default behavior)
leader Drives scroll, sends rendered follower frames via UDP
follower Receives frames from leader; falls back to own plugins when
the leader goes offline
Compatibility rule: rows and cols must match between leader and follower.
chain_length may differ — each display can have a different number of panels.
Port default: 5765 (UDP). Open this port on both Pis if ufw is active:
sudo ufw allow 5765/udp
"""
import io
import json
import os
import socket
import struct
import tempfile
import threading
import time
import logging
from enum import Enum
from typing import Callable, Optional
import numpy as np
from PIL import Image
# Raw-frame wire format: 8-byte magic + 4-byte header + raw RGB pixels
# Much faster than PNG: no encode/decode, negligible CPU, same UDP packet size
_RAW_MAGIC = b'SYNC_RAW'
_RAW_HEADER = struct.Struct('<HH') # width, height (uint16 LE)
SYNC_PORT = 5765
HELLO_INTERVAL = 5.0 # follower broadcasts hello every 5 s
HEARTBEAT_INTERVAL = 2.0 # follower sends heartbeat every 2 s
PEER_TIMEOUT = 6.0 # leader: no heartbeat → follower gone
LEADER_TIMEOUT = 6.0 # follower: no frame → leader gone
STATUS_FILE = os.path.join(tempfile.gettempdir(), "led_matrix_sync_status.json")
class SyncRole(Enum):
STANDALONE = "standalone"
LEADER = "leader"
FOLLOWER = "follower"
class LeaderState(Enum):
NO_PEER = "no_peer"
CONNECTED = "connected"
INCOMPATIBLE = "incompatible"
class FollowerState(Enum):
STANDALONE = "standalone"
FOLLOWER = "follower"
class DisplaySyncManager:
"""
Core sync manager. Instantiated by DisplayController based on config['sync'].
Leader sends compressed PNG frames to the follower after each render cycle.
Follower renders received frames; returns to own plugin stack when leader
goes offline.
"""
def __init__(
self,
role_str: str,
cfg: dict,
hw_config: dict,
logger: logging.Logger,
) -> None:
"""
Args:
role_str: "standalone" | "leader" | "follower"
cfg: config['sync'] dict
hw_config: config['display']['hardware'] dict (this Pi's own config)
logger: framework logger
"""
try:
self.role = SyncRole(role_str)
except ValueError:
logger.warning("Invalid sync role '%s', defaulting to standalone", role_str)
self.role = SyncRole.STANDALONE
self.logger = logger
self.port = int(cfg.get("port", SYNC_PORT))
self._hw_config = hw_config
# Leader state
self._leader_state = LeaderState.NO_PEER
self._peer_ip: Optional[str] = None
self._peer_compatible: bool = False
self._peer_chain: int = 0
self._last_heartbeat_time: float = 0.0
self._leader_width: int = 0 # set by display_controller after init
# Follower state
self._follower_state = FollowerState.STANDALONE
self._latest_frame: Optional[Image.Image] = None # pixel-frame fallback
self._latest_scroll_x: Optional[float] = None # Vegas scroll position
self._last_leader_frame_time: float = 0.0
self._frame_lock = threading.Lock()
self._leader_ip: Optional[str] = None
self._on_new_cycle: Optional[Callable[[], None]] = None # called when leader starts new cycle
self._on_scroll_image: Optional[Callable[[Image.Image], None]] = None # called with Image when received
self._pending_scroll_image: Optional[Image.Image] = None # image received before callback set
self._scroll_image_lock = threading.Lock() # guards _on_scroll_image / _pending_scroll_image
self._img_server_sock = None # TCP server for scroll image transfer
# Leader state additions
self._on_follower_connected: Optional[Callable[[], None]] = None # called when follower connects
self._error_message: Optional[str] = None
self._running = False
self._recv_sock: Optional[socket.socket] = None
self._send_sock: Optional[socket.socket] = None
if self.role == SyncRole.STANDALONE:
return
if self.role == SyncRole.LEADER:
self._start_leader()
elif self.role == SyncRole.FOLLOWER:
self._start_follower()
# ------------------------------------------------------------------ #
# Leader setup #
# ------------------------------------------------------------------ #
def _start_leader(self) -> None:
# Receive socket: listens for hello + heartbeat from follower
self._recv_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # nosec B104
self._recv_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._recv_sock.bind(("", self.port)) # nosec B104 — intentional: must receive UDP broadcast on all interfaces
self._recv_sock.settimeout(1.0)
# Send socket: unicast frames + hello_ack to follower
self._send_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._running = True
threading.Thread(
target=self._leader_recv_loop, daemon=True, name="sync-leader-recv"
).start()
threading.Thread(
target=self._leader_watchdog, daemon=True, name="sync-leader-watchdog"
).start()
self.logger.info("Sync: leader started on UDP port %d", self.port)
self.write_status_file()
def _leader_recv_loop(self) -> None:
while self._running:
try:
data, addr = self._recv_sock.recvfrom(1024)
sender_ip = addr[0]
try:
msg = json.loads(data.decode("utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError):
continue
t = msg.get("t")
if t == "hello":
self._handle_hello(msg, sender_ip)
elif t == "hb":
if self._peer_ip == sender_ip:
self._last_heartbeat_time = time.time()
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync leader recv error: %s", exc)
def _handle_hello(self, msg: dict, sender_ip: str) -> None:
hw = self._hw_config
local_rows = hw.get("rows", 32)
local_cols = hw.get("cols", 64)
peer_rows = int(msg.get("rows", 0))
peer_cols = int(msg.get("cols", 0))
peer_chain = int(msg.get("chain", 1))
compatible = peer_rows == local_rows and peer_cols == local_cols
self._peer_ip = sender_ip
self._peer_compatible = compatible
self._peer_chain = peer_chain
self._last_heartbeat_time = time.time()
prev_state = self._leader_state
if compatible:
if prev_state != LeaderState.CONNECTED:
self.logger.info(
"Sync: follower connected at %s (chain=%d)", sender_ip, peer_chain
)
self._leader_state = LeaderState.CONNECTED
self._error_message = None
# Send scroll image immediately on new connection so follower has identical content
if prev_state != LeaderState.CONNECTED and self._on_follower_connected:
threading.Thread(
target=self._on_follower_connected,
daemon=True, name="sync-leader-img-push"
).start()
else:
self._leader_state = LeaderState.INCOMPATIBLE
self._error_message = (
f"Incompatible panels: follower is {peer_cols}x{peer_rows}, "
f"leader is {local_cols}x{local_rows}. "
f"rows and cols must match between displays."
)
if prev_state != LeaderState.INCOMPATIBLE:
self.logger.error("Sync: %s", self._error_message)
if self._leader_state != prev_state:
self.write_status_file()
ack = json.dumps({
"t": "hello_ack",
"compatible": compatible,
"leader_width": self._leader_width,
"error": self._error_message,
}).encode("utf-8")
try:
self._send_sock.sendto(ack, (sender_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: hello_ack send failed: %s", exc)
def _leader_watchdog(self) -> None:
while self._running:
time.sleep(1.0)
if self._leader_state == LeaderState.CONNECTED:
if time.time() - self._last_heartbeat_time > PEER_TIMEOUT:
self.logger.info(
"Sync: follower heartbeat timeout — peer disconnected"
)
self._leader_state = LeaderState.NO_PEER
self._peer_ip = None
self._peer_compatible = False
self.write_status_file()
def _image_server_loop(self) -> None:
"""Follower: TCP server that receives the leader's scroll image at each new cycle."""
while self._running:
try:
conn, addr = self._img_server_sock.accept()
conn.settimeout(10.0)
try:
# 4-byte big-endian length prefix
hdr = b""
while len(hdr) < 4:
chunk = conn.recv(4 - len(hdr))
if not chunk:
break
hdr += chunk
if len(hdr) < 4:
continue
length = int.from_bytes(hdr, "big")
_MAX_IMAGE_BYTES = 10 * 1024 * 1024 # 10 MB — well above any real scroll image
if length <= 0 or length > _MAX_IMAGE_BYTES:
self.logger.warning(
"Sync: rejected TCP image with invalid length %d (max %d) from %s",
length, _MAX_IMAGE_BYTES, addr,
)
conn.close()
continue
data = bytearray()
while len(data) < length:
chunk = conn.recv(min(65536, length - len(data)))
if not chunk:
break
data.extend(chunk)
img = Image.open(io.BytesIO(data))
_MAX_W, _MAX_H = 100_000, 256 # generous for any real scroll image
if img.width > _MAX_W or img.height > _MAX_H:
self.logger.warning(
"Sync: rejected oversized scroll image %dx%d (max %dx%d) from %s",
img.width, img.height, _MAX_W, _MAX_H, addr,
)
continue
try:
img.load()
except (Image.DecompressionBombError, ValueError) as exc:
self.logger.warning("Sync: rejected decompression bomb from %s: %s", addr, exc)
continue
self.logger.info(
"Sync: received scroll image %dx%d (%d bytes compressed)",
img.width, img.height, length,
)
with self._scroll_image_lock:
if self._on_scroll_image:
cb = self._on_scroll_image
else:
# Callback not registered yet (startup race) — cache it
self._pending_scroll_image = img
cb = None
if cb:
cb(img)
finally:
conn.close()
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync: image server error: %s", exc)
def send_scroll_image(self, image: Image.Image) -> None:
"""Leader: send the full scroll image to the follower via TCP.
PNG compression typically reduces a 5000×32 image to ~2050KB,
transferring in <20ms on local WiFi. Called at new_cycle and on
first connection so both Pis always have identical cached_arrays.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
buf = io.BytesIO()
image.save(buf, format="PNG", optimize=True)
data = buf.getvalue()
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.settimeout(5.0)
sock.connect((self._peer_ip, self.port + 1))
sock.sendall(len(data).to_bytes(4, "big") + data)
self.logger.info(
"Sync: sent scroll image %dx%d (%d bytes compressed)",
image.width, image.height, len(data),
)
except Exception as exc:
self.logger.debug("Sync: image send error: %s", exc)
def set_on_follower_connected(self, callback: Callable[[], None]) -> None:
"""Leader: callback fired (in a thread) when a compatible follower first connects.
Use this to push the current scroll image immediately.
If a follower is already connected when this is called, fires right away
(handles the race where follower connects during leader startup).
"""
self._on_follower_connected = callback
if self._leader_state == LeaderState.CONNECTED:
threading.Thread(
target=callback, daemon=True, name="sync-leader-img-push-late"
).start()
def set_on_scroll_image(self, callback: Callable[[Image.Image], None]) -> None:
"""Follower: callback fired with the received Image when leader sends scroll image.
If an image was received before this callback was registered (startup race),
fires immediately with that cached image.
"""
with self._scroll_image_lock:
self._on_scroll_image = callback
pending = self._pending_scroll_image
self._pending_scroll_image = None
if pending is not None:
callback(pending)
def send_scroll_x(self, scroll_x: float) -> None:
"""Leader (Vegas mode): broadcast scroll position instead of a pixel frame.
The follower renders from its own local pipeline at scroll_x - display_width.
~20 bytes vs ~18KB for raw frames — eliminates all content-change artifacts.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
msg = json.dumps({"t": "sx", "x": round(scroll_x, 2)}).encode("utf-8")
self._send_sock.sendto(msg, (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: scroll_x send error: %s", exc)
def send_new_cycle(self) -> None:
"""Leader: signal that a new scroll cycle has started so follower rebuilds its image."""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
self._send_sock.sendto(b'{"t":"nc"}', (self._peer_ip, self.port))
except Exception as exc:
self.logger.debug("Sync: new_cycle send error: %s", exc)
def send_frame(self, image: Image.Image) -> None:
"""Leader: send a rendered frame to the follower as raw RGB bytes.
Raw format is orders of magnitude faster than PNG on Pi hardware —
no encode on sender, no decode on receiver.
Packet: 8-byte magic + 4-byte (width, height) header + raw RGB bytes.
"""
if self.role != SyncRole.LEADER:
return
if self._leader_state != LeaderState.CONNECTED or not self._peer_ip:
return
try:
arr = np.asarray(image.convert("RGB"), dtype=np.uint8)
header = _RAW_MAGIC + _RAW_HEADER.pack(image.width, image.height)
data = header + arr.tobytes()
if len(data) <= 65000:
self._send_sock.sendto(data, (self._peer_ip, self.port))
elif not getattr(self, '_oversized_frame_warned', False):
self._oversized_frame_warned = True
self.logger.warning(
"Sync: frame too large for UDP (%d bytes, max 65000) — "
"image %dx%d will not be sent; use TCP image sync instead",
len(data), image.width, image.height,
)
except Exception as exc:
self.logger.debug("Sync: frame send error: %s", exc)
def set_leader_width(self, width: int) -> None:
"""Called by DisplayController once display_manager.width is known."""
self._leader_width = width
# ------------------------------------------------------------------ #
# Follower setup #
# ------------------------------------------------------------------ #
def _start_follower(self) -> None:
# Receive socket: listens for frames + hello_ack from leader
self._recv_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._recv_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._recv_sock.bind(("", self.port)) # nosec B104 — intentional: must receive UDP broadcast on all interfaces
self._recv_sock.settimeout(0.1)
# Send socket: broadcasts hello + heartbeat
self._send_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._send_sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
self._running = True
threading.Thread(
target=self._follower_recv_loop, daemon=True, name="sync-follower-recv"
).start()
threading.Thread(
target=self._follower_announce_loop, daemon=True, name="sync-follower-announce"
).start()
threading.Thread(
target=self._follower_watchdog, daemon=True, name="sync-follower-watchdog"
).start()
# TCP server: receives scroll images from leader (port + 1)
self._img_server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # nosec B104
self._img_server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._img_server_sock.bind(("", self.port + 1)) # nosec B104 — intentional: TCP server must accept connections on all interfaces
self._img_server_sock.listen(1)
self._img_server_sock.settimeout(1.0)
threading.Thread(
target=self._image_server_loop, daemon=True, name="sync-image-server"
).start()
self.logger.info(
"Sync: follower started on UDP port %d, image server on TCP %d",
self.port, self.port + 1,
)
self.write_status_file()
def _follower_recv_loop(self) -> None:
while self._running:
try:
data, addr = self._recv_sock.recvfrom(65535)
sender_ip = addr[0]
if data[:8] == _RAW_MAGIC or len(data) > 512:
# Frame data: prefer magic-tagged raw RGB; fall back to legacy PNG
try:
if data[:8] == _RAW_MAGIC:
w, h = _RAW_HEADER.unpack(data[8:12])
raw = data[12:]
img = Image.frombuffer(
"RGB", (w, h), raw, "raw", "RGB", 0, 1
)
else:
# Fallback: try legacy PNG
img = Image.open(io.BytesIO(data))
img.load()
with self._frame_lock:
self._latest_frame = img
self._last_leader_frame_time = time.time()
self._leader_ip = sender_ip
if self._follower_state == FollowerState.STANDALONE:
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
except Exception as exc:
self.logger.debug("Sync: frame decode error: %s", exc)
else:
# Control message
try:
msg = json.loads(data.decode("utf-8"))
t = msg.get("t")
if t == "hello_ack":
self._leader_ip = sender_ip
self._peer_compatible = msg.get("compatible", False)
self._error_message = msg.get("error")
if not self._peer_compatible and self._error_message:
self.logger.error(
"Sync: leader rejected handshake — %s",
self._error_message,
)
self.write_status_file()
elif t == "sx":
# Vegas scroll-position sync — tiny message, renders locally
self._latest_scroll_x = float(msg["x"])
self._last_leader_frame_time = time.time()
self._leader_ip = sender_ip
if self._follower_state == FollowerState.STANDALONE:
self._follower_state = FollowerState.FOLLOWER
self.logger.info(
"Sync: leader active at %s — switching to follower mode",
sender_ip,
)
self.write_status_file()
if self._on_new_cycle:
self._on_new_cycle() # build initial scroll image
elif t == "nc":
# Leader started a new scroll cycle — rebuild local image
if self._on_new_cycle:
self._on_new_cycle()
except (json.JSONDecodeError, UnicodeDecodeError, KeyError):
pass
except socket.timeout:
continue
except Exception as exc:
self.logger.debug("Sync follower recv error: %s", exc)
def _follower_announce_loop(self) -> None:
hw = self._hw_config
hello = json.dumps({
"t": "hello",
"rows": hw.get("rows", 32),
"cols": hw.get("cols", 64),
"chain": hw.get("chain_length", 1),
}).encode("utf-8")
heartbeat = json.dumps({"t": "hb"}).encode("utf-8")
dest = ("<broadcast>", self.port)
last_hello = 0.0
last_hb = 0.0
while self._running:
now = time.time()
if now - last_hello >= HELLO_INTERVAL:
try:
self._send_sock.sendto(hello, dest)
last_hello = now
except Exception as exc:
self.logger.debug("Sync: hello broadcast error: %s", exc)
if now - last_hb >= HEARTBEAT_INTERVAL:
try:
self._send_sock.sendto(heartbeat, dest)
last_hb = now
except Exception as exc:
self.logger.debug("Sync: heartbeat error: %s", exc)
time.sleep(0.5)
def _follower_watchdog(self) -> None:
while self._running:
time.sleep(1.0)
if self._follower_state == FollowerState.FOLLOWER:
if time.time() - self._last_leader_frame_time > LEADER_TIMEOUT:
self.logger.info(
"Sync: leader frame timeout — returning to standalone mode"
)
self._follower_state = FollowerState.STANDALONE
with self._frame_lock:
self._latest_frame = None
self.write_status_file()
# ------------------------------------------------------------------ #
# Public API #
# ------------------------------------------------------------------ #
def is_follower_active(self) -> bool:
"""True when this Pi is in active follower mode (receiving frames)."""
return (
self.role == SyncRole.FOLLOWER
and self._follower_state == FollowerState.FOLLOWER
)
def get_latest_scroll_x(self) -> Optional[float]:
"""Follower: return the most recently received Vegas scroll position, or None."""
return self._latest_scroll_x
def set_on_new_cycle(self, callback: Callable[[], None]) -> None:
"""Follower: register a callback fired when the leader starts a new scroll cycle.
Used to trigger a local start_new_cycle() so both Pis rebuild from same fresh data.
"""
self._on_new_cycle = callback
def get_latest_frame(self) -> Optional[Image.Image]:
"""Follower: return the most recently received pixel frame (non-Vegas fallback)."""
with self._frame_lock:
return self._latest_frame
def get_status(self) -> dict:
"""Return sync state dict for the web API status endpoint."""
hw = self._hw_config
base = {
"role": self.role.value,
"port": self.port,
"local_rows": hw.get("rows", 32),
"local_cols": hw.get("cols", 64),
"local_chain": hw.get("chain_length", 1),
}
if self.role == SyncRole.STANDALONE:
return {**base, "state": "standalone"}
if self.role == SyncRole.LEADER:
return {
**base,
"state": self._leader_state.value,
"peer_ip": self._peer_ip,
"peer_compatible": self._peer_compatible,
"peer_chain": self._peer_chain,
"leader_width": self._leader_width,
"error": self._error_message,
}
# Follower
return {
**base,
"state": self._follower_state.value,
"leader_ip": self._leader_ip,
"peer_compatible": self._peer_compatible,
"error": self._error_message,
}
def write_status_file(self) -> None:
"""Write current sync status to STATUS_FILE for the web UI to read."""
try:
status = self.get_status()
status["ts"] = time.time()
tmp = STATUS_FILE + ".tmp"
with open(tmp, "w") as f:
json.dump(status, f)
os.replace(tmp, STATUS_FILE)
except Exception as exc:
self.logger.debug("Sync: status file write error: %s", exc)
def stop(self) -> None:
"""Shut down threads and close sockets."""
self._running = False
for sock in (self._recv_sock, self._send_sock, self._img_server_sock):
if sock:
try:
sock.close()
except Exception as exc:
self.logger.debug("Sync: error closing socket: %s", exc)

View File

@@ -6,6 +6,7 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
"""
import logging
import os
from pathlib import Path
from typing import Dict, List, Optional, Tuple, Union

View File

@@ -8,7 +8,7 @@ Extracted from LEDMatrix core to provide reusable functionality for plugins.
import logging
import re
from datetime import datetime, timezone
from typing import Union
from typing import Optional, Tuple, Union
import pytz

View File

@@ -313,8 +313,17 @@ class ConfigManager:
self._merge_template_defaults(self.config, template_config)
# Save migrated config using atomic save to preserve permissions
# Load secrets if they exist to pass to atomic save
secrets_content = {}
if os.path.exists(self.secrets_path):
try:
with open(self.secrets_path, 'r') as f_secrets:
secrets_content = json.load(f_secrets)
except Exception:
pass # Continue without secrets if can't load
# Use atomic save to preserve file permissions
# Note: save_config_atomic handles secrets internally
# Note: save_config_atomic handles secrets internally, no need to pass new_secrets
result = self.save_config_atomic(
new_config_data=self.config,
create_backup=False, # Already created backup above

View File

@@ -12,10 +12,11 @@ This service wraps ConfigManager and adds:
"""
import json
import os
import time
import threading
from pathlib import Path
from typing import Dict, Any, Optional, List, Callable
from typing import Dict, Any, Optional, List, Callable, Set
from datetime import datetime
from collections import defaultdict
import logging
@@ -37,7 +38,7 @@ class ConfigVersion:
config: Configuration dictionary
version: Version number
timestamp: When this version was created
checksum: SHA-256 hex digest of the config (for change detection)
checksum: MD5 checksum of the config
"""
self.config: Dict[str, Any] = config
self.version: int = version
@@ -113,9 +114,9 @@ class ConfigService:
self._start_file_watching()
def _calculate_checksum(self, config: Dict[str, Any]) -> str:
"""Calculate checksum of configuration for change detection."""
"""Calculate MD5 checksum of configuration."""
config_str = json.dumps(config, sort_keys=True)
return hashlib.sha256(config_str.encode()).hexdigest()
return hashlib.md5(config_str.encode()).hexdigest()
def _load_config(self) -> bool:
"""

View File

@@ -1,4 +1,6 @@
import time
import logging
import sys
import os
import json
from pathlib import Path
@@ -14,7 +16,6 @@ from src.config_service import ConfigService
from src.cache_manager import CacheManager
from src.font_manager import FontManager
from src.logging_config import get_logger
from src.common.sync_manager import DisplaySyncManager, SyncRole
# Get logger with consistent configuration
logger = get_logger(__name__)
@@ -31,7 +32,10 @@ class DisplayController:
def __init__(self):
start_time = time.time()
logger.info("Starting DisplayController initialization")
# Throttle tracking for _tick_plugin_updates in high-FPS loops
self._last_plugin_tick_time = 0.0
# Initialize ConfigManager and wrap with ConfigService for hot-reload
config_manager = ConfigManager()
enable_hot_reload = os.environ.get('LEDMATRIX_HOT_RELOAD', 'true').lower() == 'true'
@@ -65,39 +69,7 @@ class DisplayController:
config_time = time.time()
self.display_manager = DisplayManager(self.config)
logger.info("DisplayManager initialized in %.3f seconds", time.time() - config_time)
# Initialize multi-display sync (standalone by default — no-op unless configured)
sync_cfg = self.config.get("sync", {})
hw_cfg = self.config.get("display", {}).get("hardware", {})
self.sync_manager = DisplaySyncManager(
role_str=sync_cfg.get("role", "standalone"),
cfg=sync_cfg,
hw_config=hw_cfg,
logger=logger,
)
# Tell the leader its own physical display width so it can include it in hello_ack
if self.sync_manager.role == SyncRole.LEADER:
self.sync_manager.set_leader_width(self.display_manager.width)
# Follower mode setup
if self.sync_manager.role == SyncRole.FOLLOWER:
# Gate update_display() so background plugin threads cannot write to
# hardware — only our render loop is permitted.
_real_update = self.display_manager.update_display
_dm = self.display_manager
def _follower_gated_update():
# Allow through when the sync render loop has the token, or when
# the leader has gone offline and we've fallen back to standalone.
if getattr(_dm, '_sync_render_allowed', False) or not self.sync_manager.is_follower_active():
_real_update()
self.display_manager.update_display = _follower_gated_update
# Note: _on_new_cycle is NOT registered here. The leader now sends
# its actual scroll image via TCP at each new_cycle, so the follower
# adopts that image directly via set_on_scroll_image(). Registering
# _on_new_cycle would trigger a local rebuild that overwrites the
# leader's just-received image with a different locally-built one.
# Initialize Font Manager
font_time = time.time()
self.font_manager = FontManager(self.config)
@@ -110,7 +82,8 @@ class DisplayController:
logger.info("Display modes initialized in %.3f seconds", time.time() - init_time)
self.force_change = False
self._next_live_priority_check = 0.0 # monotonic timestamp for throttled live priority checks
# All sports and content managers now handled via plugins
logger.info("All sports and content managers now handled via plugin system")
@@ -423,64 +396,20 @@ class DisplayController:
# Set up live priority checker
self.vegas_coordinator.set_live_priority_checker(self._check_live_priority)
# Set up interrupt checker for on-demand/wifi status and follower mode
def _vegas_interrupt():
return self._check_vegas_interrupt() or self.sync_manager.is_follower_active()
# Set up interrupt checker for on-demand/wifi status
self.vegas_coordinator.set_interrupt_checker(
_vegas_interrupt,
self._check_vegas_interrupt,
check_interval=10 # Check every 10 frames (~80ms at 125 FPS)
)
# Run plugin updates inside the Vegas loop so the inter-iteration
# gap is <1 ms (nothing left for _tick_plugin_updates() to do).
self.vegas_coordinator.set_update_callback(self._tick_plugin_updates)
# Wire multi-display sync into Vegas render pipeline
follower_pos = self.config.get("sync", {}).get("follower_position", "left")
self.vegas_coordinator.set_sync_manager(self.sync_manager, follower_pos)
# Set up plugin update tick to keep data fresh during Vegas mode
self.vegas_coordinator.set_update_tick(
self._tick_plugin_updates_for_vegas,
interval=1.0
)
logger.info("Vegas mode coordinator initialized")
# Follower does NOT build its own initial scroll image — the leader
# pushes its image via TCP as soon as set_on_follower_connected fires.
# A local build would create a different (wrong) image that could
# temporarily replace the leader's correct one.
# When the leader sends its scroll image (TCP), update our
# cached_array so both Pis have pixel-identical images.
import numpy as _np
def _on_leader_scroll_image(image):
vc = getattr(self, 'vegas_coordinator', None)
if vc and vc.render_pipeline:
rp = vc.render_pipeline
arr = _np.asarray(image.convert("RGB"), dtype=_np.uint8)
rp.scroll_helper.cached_image = image
rp.scroll_helper.cached_array = arr
rp.scroll_helper.total_scroll_width = image.width
self._follower_pending_new_image = False
logger.info(
"Sync: follower adopted leader scroll image %dx%d",
image.width, image.height,
)
self.sync_manager.set_on_scroll_image(_on_leader_scroll_image)
if self.sync_manager.role == SyncRole.LEADER:
# When a follower first connects, push the current scroll image so
# the follower doesn't have to wait for the next new_cycle event.
# Polls until the image is ready (Vegas may still be composing on startup).
def _on_follower_connected():
import time as _t
for _ in range(300): # up to 30s
vc = getattr(self, 'vegas_coordinator', None)
if vc and vc.render_pipeline:
img = vc.render_pipeline.scroll_helper.cached_image
if img is not None:
self.sync_manager.send_scroll_image(img)
return
_t.sleep(0.1)
logger.warning("Sync: no scroll image available to push to new follower")
self.sync_manager.set_on_follower_connected(_on_follower_connected)
except Exception as e:
logger.error("Failed to initialize Vegas mode: %s", e, exc_info=True)
self.vegas_coordinator = None
@@ -515,16 +444,51 @@ class DisplayController:
return False
def _tick_plugin_updates_for_vegas(self):
"""
Run scheduled plugin updates and return IDs of plugins that were updated.
Called periodically by the Vegas coordinator to keep plugin data fresh
during Vegas mode. Returns a list of plugin IDs whose data changed so
Vegas can refresh their content in the scroll.
Returns:
List of updated plugin IDs, or None if no updates occurred
"""
if not self.plugin_manager or not hasattr(self.plugin_manager, 'plugin_last_update'):
self._tick_plugin_updates()
return None
# Snapshot update timestamps before ticking
old_times = dict(self.plugin_manager.plugin_last_update)
# Run the scheduled updates
self._tick_plugin_updates()
# Detect which plugins were actually updated
updated = []
for plugin_id, new_time in self.plugin_manager.plugin_last_update.items():
if new_time > old_times.get(plugin_id, 0.0):
updated.append(plugin_id)
if updated:
logger.info("Vegas update tick: %d plugin(s) updated: %s", len(updated), updated)
return updated or None
def _check_schedule(self):
"""Check if display should be active based on schedule."""
schedule_config = self.config.get('schedule', {})
# Get fresh config from config_service to support hot-reload
current_config = self.config_service.get_config()
schedule_config = current_config.get('schedule', {})
# If schedule config doesn't exist or is empty, default to always active
if not schedule_config:
self.is_display_active = True
self._was_display_active = True # Track previous state for schedule change detection
return
# Check if schedule is explicitly disabled
# Default to True (schedule enabled) if 'enabled' key is missing for backward compatibility
if 'enabled' in schedule_config and not schedule_config.get('enabled', True):
@@ -534,7 +498,7 @@ class DisplayController:
return
# Get configured timezone, default to UTC
timezone_str = self.config.get('timezone', 'UTC')
timezone_str = current_config.get('timezone', 'UTC')
try:
tz = pytz.timezone(timezone_str)
except pytz.UnknownTimeZoneError:
@@ -632,15 +596,18 @@ class DisplayController:
Target brightness level (dim_brightness if in dim period,
normal brightness otherwise)
"""
# Get fresh config from config_service to support hot-reload
current_config = self.config_service.get_config()
# Get normal brightness from config
normal_brightness = self.config.get('display', {}).get('hardware', {}).get('brightness', 90)
normal_brightness = current_config.get('display', {}).get('hardware', {}).get('brightness', 90)
# If display is OFF via schedule, don't process dim schedule
if not self.is_display_active:
self.is_dimmed = False
return normal_brightness
dim_config = self.config.get('dim_schedule', {})
dim_config = current_config.get('dim_schedule', {})
# If dim schedule doesn't exist or is disabled, use normal brightness
if not dim_config or not dim_config.get('enabled', False):
@@ -648,7 +615,7 @@ class DisplayController:
return normal_brightness
# Get configured timezone
timezone_str = self.config.get('timezone', 'UTC')
timezone_str = current_config.get('timezone', 'UTC')
try:
tz = pytz.timezone(timezone_str)
except pytz.UnknownTimeZoneError:
@@ -755,83 +722,21 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error running scheduled plugin updates")
_FOLLOWER_SEND_INTERVAL = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
def _tick_plugin_updates_throttled(self, min_interval: float = 0.0):
"""Throttled version of _tick_plugin_updates for high-FPS loops.
def _follower_rebuild_scroll_image(self) -> None:
"""Follower: rebuild the local Vegas scroll image so both Pis render from
the same fresh plugin data. Called at startup (after Vegas initializes)
and each time the leader broadcasts a new-cycle signal. Runs in a daemon
thread so it never blocks the 60fps render loop.
Args:
min_interval: Minimum seconds between calls. When <= 0 the
call passes straight through to _tick_plugin_updates so
plugin-configured update_interval values are never capped.
"""
try:
vc = getattr(self, 'vegas_coordinator', None)
if not vc:
logger.warning("Sync: follower has no vegas_coordinator — cannot build scroll image")
return
rp = vc.render_pipeline
if not rp:
logger.warning("Sync: follower vegas_coordinator has no render_pipeline")
return
logger.info("Sync: follower starting scroll image rebuild")
ok = rp.start_new_cycle()
if ok and rp.scroll_helper.cached_image is not None:
logger.info(
"Sync: follower scroll image ready — %dx%d",
rp.scroll_helper.cached_image.width,
rp.scroll_helper.cached_image.height,
)
else:
logger.warning(
"Sync: follower scroll image rebuild FAILED (ok=%s, cached=%s)",
ok, rp.scroll_helper.cached_image is not None,
)
except Exception as exc:
logger.warning("Sync: follower scroll image rebuild error: %s", exc, exc_info=True)
def _send_follower_frame(self, plugin_instance) -> None:
"""Leader: generate and send the follower's portion of the current frame.
The follower is physically to the LEFT of the leader in a right-to-left
scrolling ticker, so it shows content at scroll_position - display_width
(content that already scrolled off the leader's left edge).
Set sync.follower_position = "right" in config to invert this.
"""
if not (self.sync_manager and self.sync_manager.role == SyncRole.LEADER):
if min_interval <= 0:
self._tick_plugin_updates()
return
# Throttle to ~90fps via _FOLLOWER_SEND_INTERVAL — raw RGB bytes, no encode/decode
now = time.time()
if now - getattr(self, '_last_follower_send', 0) < self._FOLLOWER_SEND_INTERVAL:
return
self._last_follower_send = now
follower_frame = None
width = self.display_manager.width
sync_cfg = self.config.get("sync", {})
sign = -1 if sync_cfg.get("follower_position", "left") == "left" else 1
offset = sign * width
# 1. Explicit hook — plugin opted in with get_offset_frame()
try:
follower_frame = plugin_instance.get_offset_frame(offset)
except AttributeError:
pass # Most plugins don't implement get_offset_frame; that's expected
# 2. Auto-detect — plugin has a scroll_helper (standard pattern for all
# scroll plugins). Works with zero plugin code changes.
if follower_frame is None:
try:
scroll_h = getattr(plugin_instance, 'scroll_helper', None)
if scroll_h is not None:
follower_frame = scroll_h.get_portion_at(scroll_h.scroll_position + offset)
except Exception: # nosec B110 - scroll_helper.get_portion_at is optional; skip on error
pass
# 3. Mirror fallback — static plugins (clock, weather) show same frame
if follower_frame is None:
follower_frame = self.display_manager.image
if follower_frame is not None:
self.sync_manager.send_frame(follower_frame)
if now - self._last_plugin_tick_time >= min_interval:
self._last_plugin_tick_time = now
self._tick_plugin_updates()
def _sleep_with_plugin_updates(self, duration: float, tick_interval: float = 1.0):
"""Sleep while continuing to service plugin update schedules."""
@@ -1468,88 +1373,6 @@ class DisplayController:
# Plugins update on their own schedules - no forced sync updates needed
# Each plugin has its own update_interval and background services
# Multi-display sync: follower mode — render frames received from leader.
# Plugin update() threads still run (via _tick_plugin_updates above) so
# data is fresh when we return to standalone if the leader goes offline.
if self.sync_manager.is_follower_active():
# Dead-reckoning follower render:
# Advance local position at configured speed each tick; snap or
# gently correct toward received scroll_x to absorb UDP jitter.
_now_dr = time.perf_counter()
_dt = _now_dr - getattr(self, '_follower_dr_last_t', _now_dr)
self._follower_dr_last_t = _now_dr
vc = getattr(self, 'vegas_coordinator', None)
rp = vc.render_pipeline if (vc and vc.render_pipeline) else None
width = self.display_manager.width
# Advance local position at Vegas scroll speed (px/s → px/tick)
vegas_speed = (
self.config.get('display', {})
.get('vegas_scroll', {})
.get('scroll_speed', 75)
)
local_x = getattr(self, '_follower_local_x', None)
if local_x is None:
local_x = float(width) # safe start (past pre-roll guard)
local_x += vegas_speed * _dt
# Pull latest position from leader (may be None if no packet yet)
scroll_x = self.sync_manager.get_latest_scroll_x()
if scroll_x is not None:
diff = scroll_x - local_x
total_w = (
rp.scroll_helper.total_scroll_width
if rp and rp.scroll_helper.total_scroll_width
else width * 4
)
if abs(diff) > total_w * 0.5:
# Large jump → cycle reset, snap immediately
local_x = float(scroll_x)
self._follower_pending_new_image = True
elif abs(diff) > 10:
# Moderate drift → 20% correction per tick
local_x += diff * 0.20
else:
# Near → gentle 5% correction
local_x += diff * 0.05
self._follower_local_x = local_x
if rp and rp.scroll_helper.cached_image is not None:
sync_cfg = self.config.get("sync", {})
sign = -1 if sync_cfg.get("follower_position", "left") == "left" else 1
# Hold last frame until TCP image arrives after cycle reset
if not getattr(self, "_follower_pending_new_image", False):
if local_x >= width:
rp.scroll_helper.scroll_position = local_x + sign * width
frame = rp.scroll_helper.get_visible_portion()
if frame is not None:
self._follower_last_frame = frame
elif scroll_x is None:
# Fallback: pixel frame before first scroll_x arrives
frame = self.sync_manager.get_latest_frame()
if frame is not None:
self._follower_last_frame = frame
display_frame = getattr(self, '_follower_last_frame', None)
if display_frame is not None:
self.display_manager.image = display_frame
self.display_manager._sync_render_allowed = True
self.display_manager.update_display()
self.display_manager._sync_render_allowed = False
# Precision deadline timer — keeps render at exactly 60fps
_deadline = getattr(self, '_follower_deadline', None)
_now = time.perf_counter()
if _deadline is None or _now > _deadline + 0.1:
_deadline = _now
_deadline += 1.0 / 60
self._follower_deadline = _deadline
_sleep = _deadline - time.perf_counter()
if _sleep > 0:
time.sleep(_sleep)
continue
# Process any deferred updates that may have accumulated
# This also cleans up expired updates to prevent memory leaks
self.display_manager.process_deferred_updates()
@@ -1923,7 +1746,7 @@ class DisplayController:
)
target_duration = max_duration
start_time = time.time()
start_time = time.monotonic()
def _should_exit_dynamic(elapsed_time: float) -> bool:
if not dynamic_enabled:
@@ -1982,19 +1805,34 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error during display update")
# Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display)
time.sleep(display_interval)
self._tick_plugin_updates()
self._tick_plugin_updates_throttled(min_interval=1.0)
self._poll_on_demand_requests()
self._check_on_demand_expiration()
# Check for live priority every ~30s so live
# games can interrupt long display durations
elapsed = time.monotonic() - start_time
now = time.monotonic()
if not self.on_demand_active and now >= self._next_live_priority_check:
self._next_live_priority_check = now + 30.0
live_mode = self._check_live_priority()
if live_mode and live_mode != active_mode:
logger.info("Live priority detected during high-FPS loop: %s", live_mode)
self.current_display_mode = live_mode
self.force_change = True
try:
self.current_mode_index = self.available_modes.index(live_mode)
except ValueError:
pass
# continue the main while loop to skip
# post-loop rotation/sleep logic
break
if self.current_display_mode != active_mode:
logger.debug("Mode changed during high-FPS loop, breaking early")
break
elapsed = time.time() - start_time
if elapsed >= target_duration:
logger.debug(
"Reached high-FPS target duration %.2fs for mode %s",
@@ -2024,7 +1862,7 @@ class DisplayController:
time.sleep(display_interval)
self._tick_plugin_updates()
elapsed = time.time() - start_time
elapsed = time.monotonic() - start_time
if elapsed >= target_duration:
logger.debug(
"Reached standard target duration %.2fs for mode %s",
@@ -2051,11 +1889,25 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error during display update")
# Multi-display sync: send follower frame after each render
self._send_follower_frame(manager_to_display)
self._poll_on_demand_requests()
self._check_on_demand_expiration()
# Check for live priority every ~30s so live
# games can interrupt long display durations
now = time.monotonic()
if not self.on_demand_active and now >= self._next_live_priority_check:
self._next_live_priority_check = now + 30.0
live_mode = self._check_live_priority()
if live_mode and live_mode != active_mode:
logger.info("Live priority detected during display loop: %s", live_mode)
self.current_display_mode = live_mode
self.force_change = True
try:
self.current_mode_index = self.available_modes.index(live_mode)
except ValueError:
pass
break
if self.current_display_mode != active_mode:
logger.info("Mode changed during display loop from %s to %s, breaking early", active_mode, self.current_display_mode)
break
@@ -2069,19 +1921,26 @@ class DisplayController:
loop_completed = True
break
# If live priority preempted the display loop, skip
# all post-loop logic (remaining sleep, rotation) and
# restart the main loop so the live mode displays
# immediately.
if self.current_display_mode != active_mode:
continue
# Ensure we honour minimum duration when not dynamic and loop ended early
if (
not dynamic_enabled
and not loop_completed
and not needs_high_fps
):
elapsed = time.time() - start_time
elapsed = time.monotonic() - start_time
remaining_sleep = max(0.0, max_duration - elapsed)
if remaining_sleep > 0:
self._sleep_with_plugin_updates(remaining_sleep)
if dynamic_enabled:
elapsed_total = time.time() - start_time
elapsed_total = time.monotonic() - start_time
cycle_done = self._plugin_cycle_complete(manager_to_display)
# Log cycle completion status and metrics

View File

@@ -1,14 +1,11 @@
import json
import os
import tempfile
if os.getenv("EMULATOR", "false") == "true":
from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions
else:
from rgbmatrix import RGBMatrix, RGBMatrixOptions
from contextlib import contextmanager
from PIL import Image, ImageDraw, ImageFont
import time
from typing import Dict, Any, List
from typing import Dict, Any, List, Tuple
import logging
import math
import freetype
@@ -31,10 +28,8 @@ class DisplayManager:
self.config = config or {}
self._force_fallback = force_fallback
self._suppress_test_pattern = suppress_test_pattern
# When True, update_display() and clear() skip hardware writes (used during off-screen content capture)
self._capture_mode_active = False
# Snapshot settings for web preview integration (service writes, web reads)
self._snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path intentional; web UI reads same path
self._snapshot_path = "/tmp/led_matrix_preview.png"
self._snapshot_min_interval_sec = 0.2 # max ~5 fps
self._last_snapshot_ts = 0.0
@@ -60,7 +55,8 @@ class DisplayManager:
def _setup_matrix(self):
"""Initialize the RGB matrix with configuration settings."""
_init_error_str = None
setup_start = time.time()
try:
# Allow callers (e.g., web UI) to force non-hardware fallback mode
if getattr(self, '_force_fallback', False):
@@ -90,7 +86,7 @@ class DisplayManager:
options.disable_hardware_pulsing = hardware_config.get('disable_hardware_pulsing', False)
options.show_refresh_rate = hardware_config.get('show_refresh_rate', False)
options.limit_refresh_rate_hz = hardware_config.get('limit_refresh_rate_hz', 90)
options.gpio_slowdown = runtime_config.get('gpio_slowdown', 3)
options.gpio_slowdown = runtime_config.get('gpio_slowdown', 2)
# Disable internal privilege dropping - we manage this via systemd or remain root
# This prevents the library from dropping to 'daemon' user which breaks file permissions
@@ -103,18 +99,6 @@ class DisplayManager:
options.pwm_dither_bits = hardware_config.get('pwm_dither_bits')
if 'inverse_colors' in hardware_config:
options.inverse_colors = hardware_config.get('inverse_colors')
# Pi 5 only: 0=PIO/RP1 coprocessor (default, less CPU),
# 1=RIO/Registered IO (faster; gpio_slowdown effect is inverted in this mode)
if 'rp1_rio' in runtime_config:
if hasattr(options, 'rp1_rio'):
options.rp1_rio = runtime_config.get('rp1_rio')
else:
logger.warning(
"rp1_rio is set in config but the installed rgbmatrix library does "
"not support it — the library was likely built without Pi 5 RP1 "
"support (mmap to 0x3f000000 instead of RP1 chip). "
"Fix: sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh"
)
logger.info(f"Initializing RGB Matrix with settings: rows={options.rows}, cols={options.cols}, chain_length={options.chain_length}, parallel={options.parallel}, hardware_mapping={options.hardware_mapping}")
@@ -145,7 +129,6 @@ class DisplayManager:
self._draw_test_pattern()
except Exception as e:
_init_error_str = str(e)
logger.error(f"Failed to initialize RGB Matrix: {e}", exc_info=True)
# Create a fallback image for web preview using configured dimensions when available
self.matrix = None
@@ -166,41 +149,12 @@ class DisplayManager:
self.draw.rectangle([0, 0, fallback_width - 1, fallback_height - 1], outline=(255, 0, 0))
self.draw.line([0, 0, fallback_width - 1, fallback_height - 1], fill=(0, 255, 0))
self.draw.text((2, max(0, (fallback_height // 2) - 4)), "Simulation", fill=(0, 128, 255))
except Exception: # nosec B110 - best-effort fallback visualization; drawing errors must not crash startup
except Exception:
# Best-effort; ignore drawing errors in fallback
pass
logger.error(
f"Matrix initialization failed — running in fallback/simulation mode "
f"(size {fallback_width}x{fallback_height}). Error: {e}. "
"On Raspberry Pi 5: ensure rpi-rgb-led-matrix was built from the latest "
"submodule (re-run first_time_install.sh). gpio_slowdown of 23 is typical for Pi 5 PIO mode."
)
logger.error(f"Matrix initialization failed, using fallback mode with size {fallback_width}x{fallback_height}. Error: {e}")
# Do not raise here; allow fallback mode so web preview and non-hardware environments work
# Write hardware status file so the web UI can surface init failures
_hw_status = {"ok": self.matrix is not None, "error": _init_error_str}
_status_path = "/tmp/led_matrix_hw_status.json" # nosec B108
try:
if os.path.islink(_status_path):
logger.warning("Skipping hardware status write: %s is a symlink", _status_path)
else:
_fd, _tmp_path = tempfile.mkstemp(dir="/tmp", prefix=".led_hw_") # nosec B108
try:
with os.fdopen(_fd, "w") as _f:
json.dump(_hw_status, _f)
_f.flush()
os.fsync(_f.fileno())
os.chmod(_tmp_path, 0o644)
os.replace(_tmp_path, _status_path)
except Exception:
try:
os.unlink(_tmp_path)
except OSError:
pass
raise
except Exception:
logger.error("Failed to write hardware status file", exc_info=True)
@property
def width(self):
"""Get the display width."""
@@ -301,22 +255,6 @@ class DisplayManager:
except Exception as e:
logger.error(f"Error drawing test pattern: {e}", exc_info=True)
@contextmanager
def capture_mode(self):
"""Suppress hardware output during off-screen content capture.
Plugins call update_display() as part of their normal display() flow.
When fetching content for Vegas mode the render loop is still running,
so any incidental hardware write causes a visible flash on the matrix.
Entering this context prevents those writes without affecting the PIL
image buffer, which the adapter reads to extract content.
"""
self._capture_mode_active = True
try:
yield
finally:
self._capture_mode_active = False
def update_display(self):
"""Update the display using double buffering with proper sync."""
try:
@@ -326,13 +264,10 @@ class DisplayManager:
# Still write a snapshot so the web UI can preview
self._write_snapshot_if_due()
return
if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
# Copy the current image to the offscreen canvas
# Copy the current image to the offscreen canvas
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
@@ -369,23 +304,21 @@ class DisplayManager:
# Create a new black image
self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
self.draw = ImageDraw.Draw(self.image)
if not self._capture_mode_active:
# Clear both canvases and the underlying matrix to ensure no artifacts.
# Failures are non-fatal — the image buffer is already black above, so
# the next update_display() call will push clean content regardless.
try:
self.offscreen_canvas.Clear()
except (RuntimeError, OSError) as e:
logger.error("Failed to clear offscreen canvas: %s", e)
try:
self.current_canvas.Clear()
except (RuntimeError, OSError) as e:
logger.error("Failed to clear current canvas: %s", e)
try:
self.matrix.Clear()
except (RuntimeError, OSError) as e:
logger.error("Failed to clear matrix front buffer: %s", e)
# Clear both canvases and the underlying matrix to ensure no artifacts
try:
self.offscreen_canvas.Clear()
except Exception:
pass
try:
self.current_canvas.Clear()
except Exception:
pass
try:
# Extra safety: clear the matrix front buffer as well
self.matrix.Clear()
except Exception:
pass
# Note: We do NOT call update_display() here to avoid black flashes.
# The caller should call update_display() after drawing new content.
@@ -781,8 +714,8 @@ class DisplayManager:
try:
self.image = Image.new('RGB', (self.width, self.height))
self.draw = ImageDraw.Draw(self.image)
except (OSError, RuntimeError, ValueError, MemoryError):
logger.debug("Canvas reset during cleanup failed", exc_info=True)
except Exception:
pass
# Reset the singleton state when cleaning up
DisplayManager._instance = None
DisplayManager._initialized = False
@@ -939,7 +872,7 @@ class DisplayManager:
# Never modify /tmp permissions - it has special system permissions (1777)
# that must not be changed or it breaks apt and other system tools
parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
if parent_dir and str(parent_dir) != '/tmp':
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
# Write atomically: temp then replace
tmp_path = f"{self._snapshot_path}.tmp"

View File

@@ -19,7 +19,8 @@ Usage:
import logging
import time
import requests
from typing import Dict, List
from typing import Dict, List, Set, Optional, Any
from datetime import datetime, timezone
logger = logging.getLogger(__name__)

View File

@@ -3,14 +3,15 @@ import logging
import freetype
import json
import hashlib
import urllib.parse
import urllib.request
import zipfile
import tempfile
import shutil
import time
from pathlib import Path
from PIL import ImageFont
from typing import Dict, Tuple, Optional, Union, Any, List
from functools import lru_cache
logger = logging.getLogger(__name__)
@@ -266,12 +267,9 @@ class FontManager:
logger.info(f"Using cached font: {cache_path}")
return str(cache_path)
# Download font — restrict to http/https to prevent file:// reads
parsed = urllib.parse.urlparse(url)
if parsed.scheme not in ('http', 'https'):
raise ValueError(f"Font URL must use http or https, got: {parsed.scheme!r}")
# Download font
logger.info(f"Downloading font from {url}")
urllib.request.urlretrieve(url, cache_path) # nosec B310 - scheme validated above
urllib.request.urlretrieve(url, cache_path)
# Handle zip files
if url.endswith('.zip'):
@@ -701,6 +699,8 @@ class FontManager:
fonts_dir = Path("assets/fonts")
ensure_directory_permissions(fonts_dir, get_assets_dir_mode())
target_path = os.path.join(fonts_dir, f"{family_name}.{font_file_path.rsplit('.', 1)[-1]}")
# Add to catalog
self.font_catalog[family_name] = font_file_path
self.clear_cache()
@@ -746,11 +746,11 @@ class FontManager:
if font_path.endswith('.bdf'):
# Try to load BDF font
freetype.Face(font_path)
face = freetype.Face(font_path)
return {"valid": True, "type": "bdf", "family": "unknown"}
elif font_path.endswith('.ttf'):
# Try to load TTF font
ImageFont.truetype(font_path, 12)
font = ImageFont.truetype(font_path, 12)
return {"valid": True, "type": "ttf", "family": "unknown"}
else:
return {"valid": False, "error": "Unsupported font format"}

View File

@@ -1,6 +1,7 @@
import os
import time
import freetype
from PIL import ImageDraw, ImageFont
from PIL import Image, ImageDraw, ImageFont
import logging
from typing import Dict, Any
from src.display_manager import DisplayManager
@@ -72,6 +73,7 @@ class FontTestManager:
def update(self):
"""No update needed for static display."""
pass
def display(self, force_clear: bool = False):
"""Display the font with sample text."""
@@ -79,6 +81,10 @@ class FontTestManager:
# Clear the display
self.display_manager.clear()
# Get display dimensions
width = self.display_manager.matrix.width
height = self.display_manager.matrix.height
# Draw font name at the top
self.display_manager.draw_text(self.current_config['display_name'], y=2, color=(255, 255, 255))

View File

@@ -7,6 +7,7 @@ version of BackgroundCacheMixin that works for weather, stocks, news, etc.
"""
import time
import logging
from typing import Dict, Optional, Any, Callable

View File

@@ -6,8 +6,9 @@ Handles custom layouts, element positioning, and display composition.
import json
import os
import logging
from typing import Dict, List, Any
from typing import Dict, List, Any, Tuple
from datetime import datetime
from PIL import Image, ImageDraw, ImageFont
logger = logging.getLogger(__name__)

View File

@@ -11,7 +11,7 @@ import time
import logging
import requests
import json
from typing import Dict, List, Optional, Tuple
from typing import Dict, Any, List, Optional, Tuple
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
from requests.adapters import HTTPAdapter
@@ -43,9 +43,6 @@ class LogoDownloader:
'ncaaw': 'https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/teams', # Alias for basketball plugin
'ncaa_baseball': 'https://site.api.espn.com/apis/site/v2/sports/baseball/college-baseball/teams',
'ncaam_hockey': 'https://site.api.espn.com/apis/site/v2/sports/hockey/mens-college-hockey/teams',
'ncaaw_hockey': 'https://site.api.espn.com/apis/site/v2/sports/hockey/womens-college-hockey/teams',
'ncaam_lacrosse': 'https://site.api.espn.com/apis/site/v2/sports/lacrosse/mens-college-lacrosse/teams',
'ncaaw_lacrosse': 'https://site.api.espn.com/apis/site/v2/sports/lacrosse/womens-college-lacrosse/teams',
# Soccer leagues
'soccer_eng.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/eng.1/teams',
'soccer_esp.1': 'https://site.api.espn.com/apis/site/v2/sports/soccer/esp.1/teams',
@@ -76,8 +73,6 @@ class LogoDownloader:
'ncaa_baseball': 'assets/sports/ncaa_logos',
'ncaam_hockey': 'assets/sports/ncaa_logos',
'ncaaw_hockey': 'assets/sports/ncaa_logos',
'ncaam_lacrosse': 'assets/sports/ncaa_logos',
'ncaaw_lacrosse': 'assets/sports/ncaa_logos',
# Soccer leagues - all use the same soccer_logos directory
'soccer_eng.1': 'assets/sports/soccer_logos',
'soccer_esp.1': 'assets/sports/soccer_logos',
@@ -191,7 +186,7 @@ class LogoDownloader:
return True
except PermissionError:
logger.error(f"Permission denied: Cannot write to directory {path}")
logger.error("Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
logger.error(f"Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
return False
except Exception as e:
logger.error(f"Failed to test write access to directory {path}: {e}")
@@ -248,7 +243,7 @@ class LogoDownloader:
except PermissionError as e:
logger.error(f"Permission denied downloading logo for {team_abbreviation}: {e}")
logger.error("Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
logger.error(f"Please run: sudo ./scripts/fix_perms/fix_assets_permissions.sh")
return False
except requests.exceptions.RequestException as e:
logger.error(f"Failed to download logo for {team_abbreviation}: {e}")

View File

@@ -11,7 +11,7 @@ Builds on existing PluginHealthTracker to provide:
import threading
import time
from typing import Dict, Any, Optional, List, Callable
from datetime import datetime
from datetime import datetime, timedelta
from enum import Enum
from dataclasses import dataclass

View File

@@ -7,8 +7,9 @@ status tracking and cancellation support.
import threading
import queue
import time
from typing import Dict, Optional, List, Callable, Any
from datetime import datetime
from datetime import datetime, timedelta
from pathlib import Path
import json

View File

@@ -7,7 +7,7 @@ and their associated data structures.
from enum import Enum
from dataclasses import dataclass, field
from typing import Dict, Any, Optional
from typing import Dict, Any, Optional, List
from datetime import datetime
import uuid

View File

@@ -6,8 +6,9 @@ error isolation, and performance monitoring.
"""
import time
from typing import Any, Optional, Callable
from threading import Thread
import signal
from typing import Any, Optional, Dict, Callable
from threading import Thread, Event
import logging
from src.exceptions import PluginError
@@ -15,8 +16,9 @@ from src.logging_config import get_logger
from src.error_aggregator import record_error
class PluginTimeoutError(Exception):
class TimeoutError(Exception):
"""Raised when a plugin operation times out."""
pass
class PluginExecutor:
@@ -55,7 +57,7 @@ class PluginExecutor:
Result of operation
Raises:
PluginTimeoutError: If operation times out
TimeoutError: If operation times out
PluginError: If operation raises an exception
"""
timeout = timeout or self.default_timeout
@@ -79,7 +81,7 @@ class PluginExecutor:
if not result_container['completed']:
error_msg = f"{plugin_context} operation timed out after {timeout}s"
self.logger.error(error_msg)
timeout_error = PluginTimeoutError(error_msg)
timeout_error = TimeoutError(error_msg)
record_error(timeout_error, plugin_id=plugin_id, operation="timeout")
raise timeout_error
@@ -126,7 +128,7 @@ class PluginExecutor:
)
return True
except PluginTimeoutError:
except TimeoutError:
self.logger.error("Plugin %s update() timed out", plugin_id)
return False
except PluginError:
@@ -202,7 +204,7 @@ class PluginExecutor:
# For backward compatibility: if plugin returns None or something else, treat as success
self.logger.debug(f"Plugin {plugin_id} display() returned non-boolean: {result}, treating as True")
return True
except PluginTimeoutError:
except TimeoutError:
self.logger.error("Plugin %s display() timed out", plugin_id)
return False
except PluginError:
@@ -245,7 +247,7 @@ class PluginExecutor:
timeout=timeout,
plugin_id=plugin_id
)
except Exception as e: # covers PluginTimeoutError, PluginError, and unexpected errors
except (TimeoutError, PluginError, Exception) as e:
self.logger.warning(
"Plugin %s %s failed, using default return: %s",
plugin_id,

View File

@@ -5,11 +5,9 @@ Handles plugin module imports, dependency installation, and class instantiation.
Extracted from PluginManager to improve separation of concerns.
"""
import hashlib
import json
import importlib
import importlib.util
import os
import sys
import subprocess
import threading
@@ -70,11 +68,6 @@ class PluginLoader:
Returns:
Path to plugin directory or None if not found
"""
# Sanitize plugin_id — os.path.basename is a CodeQL-recognized path sanitizer
plugin_id = os.path.basename(plugin_id or '')
if not plugin_id:
return None
# Strategy 1: Use mapping from discovery
if plugin_directories and plugin_id in plugin_directories:
plugin_dir = plugin_directories[plugin_id]
@@ -82,16 +75,14 @@ class PluginLoader:
self.logger.debug("Using plugin directory from discovery mapping: %s", plugin_dir)
return plugin_dir
# Strategy 2: Direct paths — resolve and validate they stay within plugins_dir
plugins_dir_resolved = plugins_dir.resolve()
for _candidate_name in (plugin_id, f"ledmatrix-{plugin_id}"):
_candidate = (plugins_dir_resolved / _candidate_name).resolve()
try:
_candidate.relative_to(plugins_dir_resolved)
except ValueError:
continue
if _candidate.exists():
return _candidate
# Strategy 2: Direct paths
plugin_dir = plugins_dir / plugin_id
if plugin_dir.exists():
return plugin_dir
plugin_dir = plugins_dir / f"ledmatrix-{plugin_id}"
if plugin_dir.exists():
return plugin_dir
# Strategy 3: Case-insensitive search
normalized_id = plugin_id.lower()
@@ -139,123 +130,51 @@ class PluginLoader:
self,
plugin_dir: Path,
plugin_id: str,
plugins_dir: Optional[Path] = None,
timeout: int = 300
) -> bool:
"""
Install plugin dependencies from requirements.txt.
Args:
plugin_dir: Plugin directory path
plugin_id: Plugin identifier
plugins_dir: Trusted base plugins directory for path containment check
timeout: Installation timeout in seconds
Returns:
True if dependencies installed or not needed, False on error
"""
plugin_id = os.path.basename(plugin_id or '')
if not plugin_id:
return False
# Resolve to a canonical absolute path (normalises .. and symlinks)
plugin_dir_real = os.path.realpath(str(plugin_dir))
if plugins_dir is not None:
# Reconstruct the plugin path from a trusted base + a sanitised
# directory name. os.path.basename() is CodeQL's recognised
# py/path-injection sanitiser: it strips all directory components
# so the result cannot contain traversal sequences. Joining it
# with the resolved, trusted plugins_dir produces a path that
# CodeQL considers untainted.
plugins_dir_real = os.path.realpath(str(plugins_dir))
safe_dir_name = os.path.basename(plugin_dir_real)
if not safe_dir_name:
self.logger.error("Could not determine plugin directory name for %s", plugin_id)
return False
safe_plugin_dir = os.path.join(plugins_dir_real, safe_dir_name)
if not os.path.isdir(safe_plugin_dir):
self.logger.error(
"Plugin directory for %s not found inside plugins dir", plugin_id
)
return False
else:
safe_plugin_dir = plugin_dir_real
if not os.path.isdir(safe_plugin_dir):
self.logger.error("Plugin directory does not exist: %s", plugin_dir)
return False
requirements_file = os.path.join(safe_plugin_dir, "requirements.txt")
marker_file = os.path.join(safe_plugin_dir, ".dependencies_installed")
if not os.path.isfile(requirements_file):
requirements_file = plugin_dir / "requirements.txt"
if not requirements_file.exists():
return True # No dependencies needed
try:
with open(requirements_file, 'rb') as fh:
current_hash = hashlib.sha256(fh.read()).hexdigest()
except OSError as e:
self.logger.error("Failed to read requirements.txt for %s: %s", plugin_id, e)
return False
# Skip if requirements.txt hasn't changed since last install
if os.path.isfile(marker_file):
try:
with open(marker_file, 'r', encoding='utf-8') as fh:
stored_hash = fh.read().strip()
except OSError as e:
self.logger.warning(
"Could not read dependency marker for %s (%s), will reinstall dependencies",
plugin_id, e
)
else:
if stored_hash == current_hash:
self.logger.debug("Dependencies already installed for %s (requirements unchanged)", plugin_id)
return True
self.logger.info("Requirements changed for %s, reinstalling dependencies", plugin_id)
# Check if already installed
marker_path = plugin_dir / ".dependencies_installed"
if marker_path.exists():
self.logger.debug("Dependencies already installed for %s", plugin_id)
return True
try:
self.logger.info("Installing dependencies for plugin %s...", plugin_id)
result = subprocess.run(
[sys.executable, "-m", "pip", "install", "--break-system-packages", "-r", requirements_file],
[sys.executable, "-m", "pip", "install", "--break-system-packages", "-r", str(requirements_file)],
capture_output=True,
text=True,
timeout=timeout,
check=False
)
if result.returncode == 0:
try:
with open(marker_file, 'w', encoding='utf-8') as fh:
fh.write(current_hash)
ensure_file_permissions(Path(marker_file), get_plugin_file_mode())
except OSError as marker_err:
self.logger.debug("Could not write dependency marker for %s: %s", plugin_id, marker_err)
# Mark as installed
marker_path.touch()
# Set proper file permissions after creating marker
ensure_file_permissions(marker_path, get_plugin_file_mode())
self.logger.info("Dependencies installed successfully for %s", plugin_id)
return True
else:
stderr = result.stderr or ""
# uninstall-no-record-file means the package is already present at the
# system level (e.g. installed via dnf/apt without a pip RECORD file).
# pip can't replace it, but it IS installed — write the marker so we
# don't retry on every restart.
if "uninstall-no-record-file" in stderr:
self.logger.warning(
"Dependencies for %s include system-managed packages (no pip RECORD). "
"Assuming they are satisfied: %s",
plugin_id, stderr.strip()
)
try:
with open(marker_file, 'w', encoding='utf-8') as fh:
fh.write(current_hash)
ensure_file_permissions(Path(marker_file), get_plugin_file_mode())
except OSError as marker_err:
self.logger.debug("Could not write dependency marker for %s: %s", plugin_id, marker_err)
return True
self.logger.warning(
"Dependency installation returned non-zero exit code for %s: %s",
plugin_id,
stderr
result.stderr
)
return False
except subprocess.TimeoutExpired:
@@ -430,20 +349,9 @@ class PluginLoader:
Returns:
Loaded module or None on error
"""
plugin_id = os.path.basename(plugin_id or '')
if not plugin_id:
raise PluginError("Invalid plugin ID")
try:
plugin_dir_resolved = plugin_dir.resolve(strict=True)
except OSError:
raise PluginError("Plugin directory not found", plugin_id=plugin_id)
entry_file = (plugin_dir_resolved / entry_point).resolve()
try:
entry_file.relative_to(plugin_dir_resolved)
except ValueError:
raise PluginError("Invalid entry point path", plugin_id=plugin_id)
entry_file = plugin_dir / entry_point
if not entry_file.exists():
error_msg = f"Entry point file not found for plugin {plugin_id}"
error_msg = f"Entry point file not found: {entry_file} for plugin {plugin_id}"
self.logger.error(error_msg)
raise PluginError(error_msg, plugin_id=plugin_id, context={'entry_file': str(entry_file)})
@@ -593,12 +501,11 @@ class PluginLoader:
display_manager: Any,
cache_manager: Any,
plugin_manager: Any,
install_deps: bool = True,
plugins_dir: Optional[Path] = None,
install_deps: bool = True
) -> Tuple[Any, Any]:
"""
Complete plugin loading process.
Args:
plugin_id: Plugin identifier
manifest: Plugin manifest
@@ -608,22 +515,16 @@ class PluginLoader:
cache_manager: Cache manager instance
plugin_manager: Plugin manager instance
install_deps: Whether to install dependencies
plugins_dir: Trusted base plugins directory forwarded to install_dependencies
Returns:
Tuple of (plugin_instance, module)
Raises:
PluginError: If loading fails
"""
# Install dependencies if needed
if install_deps:
if not self.install_dependencies(plugin_dir, plugin_id, plugins_dir=plugins_dir):
raise PluginError(
f"Dependency installation failed for plugin {plugin_id} in {plugin_dir}",
plugin_id=plugin_id,
context={'plugin_dir': str(plugin_dir)},
)
self.install_dependencies(plugin_dir, plugin_id)
# Load module
entry_point = manifest.get('entry_point', 'manager.py')

View File

@@ -7,7 +7,10 @@ Handles dynamic plugin loading from the plugins/ directory.
API Version: 1.0.0
"""
import os
import json
import importlib
import importlib.util
import sys
import subprocess
import time
@@ -350,29 +353,12 @@ class PluginManager:
display_manager=self.display_manager,
cache_manager=self.cache_manager,
plugin_manager=self,
install_deps=True,
plugins_dir=self.plugins_dir,
install_deps=True
)
# Store module
self.plugin_modules[plugin_id] = module
# Register plugin-shipped fonts with the FontManager (if any).
# Plugin manifests can declare a "fonts" block that ships custom
# fonts with the plugin; FontManager.register_plugin_fonts handles
# the actual loading. Wired here so manifest declarations take
# effect without requiring plugin code changes.
font_manifest = manifest.get('fonts')
if font_manifest and self.font_manager is not None and hasattr(
self.font_manager, 'register_plugin_fonts'
):
try:
self.font_manager.register_plugin_fonts(plugin_id, font_manifest)
except Exception as e:
self.logger.warning(
"Failed to register fonts for plugin %s: %s", plugin_id, e
)
# Validate configuration
if hasattr(plugin_instance, 'validate_config'):
try:
@@ -675,44 +661,6 @@ class PluginManager:
# Default: 60 seconds
return 60.0
def _record_update_failure(
self,
plugin_id: str,
exc: Optional[Exception] = None,
) -> None:
"""Apply the standard failure-recovery path for a plugin update.
Stamps plugin_last_update with the actual failure time so the full
configured interval elapses before the next retry, then transitions
the plugin back to ENABLED (not ERROR) with structured error context
so automatic recovery happens on the next scheduled cycle.
Args:
plugin_id: Plugin identifier
exc: The exception that caused the failure, if any. When None a
synthetic ExecutionFailure exception is constructed from the
timeout/executor-error path.
"""
failure_time = time.time()
if exc is not None:
err: Exception = exc
error_type = type(exc).__name__
else:
err = Exception(f"Plugin {plugin_id} execution failed (timeout or executor error)")
error_type = 'ExecutionFailure'
error_info = {
'error': str(err),
'error_type': error_type,
'timestamp': failure_time,
'recoverable': True,
}
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
self.plugin_last_update[plugin_id] = failure_time
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info, error=err)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, err)
def run_scheduled_updates(self, current_time: Optional[float] = None) -> None:
"""
Trigger plugin updates based on their defined update intervals.
@@ -770,10 +718,16 @@ class PluginManager:
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id)
# Execution failed (timeout or error)
self.state_manager.set_state(plugin_id, PluginState.ERROR)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, Exception("Plugin execution failed"))
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
self._record_update_failure(plugin_id, exc=exc)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=exc)
# Record failure
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, exc)
def update_all_plugins(self) -> None:
"""
@@ -799,12 +753,14 @@ class PluginManager:
if success:
self.plugin_last_update[plugin_id] = time.time()
self.state_manager.record_update(plugin_id)
# Update state back to ENABLED
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else:
self._record_update_failure(plugin_id)
# Execution failed
self.state_manager.set_state(plugin_id, PluginState.ERROR)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
self._record_update_failure(plugin_id, exc=exc)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=exc)
def get_plugin_health_metrics(self) -> Dict[str, Any]:
"""

View File

@@ -5,7 +5,6 @@ Manages plugin state machine (loaded → enabled → running → error)
with state transitions and queries.
"""
import threading
from enum import Enum
from typing import Optional, Dict, Any
from datetime import datetime
@@ -35,7 +34,6 @@ class PluginStateManager:
logger: Optional logger instance
"""
self.logger = logger or get_logger(__name__)
self._lock = threading.RLock()
self._states: Dict[str, PluginState] = {}
self._state_history: Dict[str, list] = {}
self._error_info: Dict[str, Dict[str, Any]] = {}
@@ -50,44 +48,44 @@ class PluginStateManager:
) -> None:
"""
Set plugin state and record transition.
Args:
plugin_id: Plugin identifier
state: New state
error: Optional error if transitioning to ERROR state
"""
with self._lock:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
transition = {
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
# Record state transition
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
transition = {
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None
}
self._state_history[plugin_id].append(transition)
# Store error info if transitioning to ERROR state
if state == PluginState.ERROR and error:
self._error_info[plugin_id] = {
'error': str(error),
'error_type': type(error).__name__,
'timestamp': datetime.now()
}
self._state_history[plugin_id].append(transition)
# Store error info if transitioning to ERROR state
if state == PluginState.ERROR and error:
self._error_info[plugin_id] = {
'error': str(error),
'error_type': type(error).__name__,
'timestamp': datetime.now()
}
elif state != PluginState.ERROR:
# Clear error info when leaving ERROR state
self._error_info.pop(plugin_id, None)
self.logger.debug(
"Plugin %s state transition: %s%s",
plugin_id,
old_state.value,
state.value
)
elif state != PluginState.ERROR:
# Clear error info when leaving ERROR state
self._error_info.pop(plugin_id, None)
self.logger.debug(
"Plugin %s state transition: %s%s",
plugin_id,
old_state.value,
state.value
)
def get_state(self, plugin_id: str) -> PluginState:
"""
@@ -138,82 +136,17 @@ class PluginStateManager:
"""
return self._state_history.get(plugin_id, [])
def set_error_info(self, plugin_id: str, error_info: Dict[str, Any]) -> None:
"""
Persist structured error context without changing plugin state.
Used for recoverable failures (e.g. update timeout) where the plugin
stays ENABLED but the error details should remain queryable.
Args:
plugin_id: Plugin identifier
error_info: Arbitrary dict describing the error
"""
with self._lock:
self._error_info[plugin_id] = dict(error_info)
def set_state_with_error(
self,
plugin_id: str,
state: PluginState,
error_info: Dict[str, Any],
error: Optional[Exception] = None,
) -> None:
"""Set plugin state and persist error context atomically.
Unlike calling set_state() then set_error_info() separately, this
method holds ``_lock`` for both writes so no reader can observe the
new state without the accompanying error context.
Intentionally does not clear ``_error_info`` the way set_state() does
for non-ERROR transitions — this is the recoverable-failure path where
the error dict is the entire point.
Args:
plugin_id: Plugin identifier
state: New state
error_info: Structured error dict to persist alongside the state
error: Optional exception recorded in the transition history
"""
with self._lock:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
if plugin_id not in self._state_history:
self._state_history[plugin_id] = []
self._state_history[plugin_id].append({
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None,
})
self._error_info[plugin_id] = dict(error_info)
self.logger.debug(
"Plugin %s state transition: %s%s (recoverable error stored)",
plugin_id,
old_state.value,
state.value,
)
def get_error_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get error information for a plugin.
Returns the stored error dict whether the plugin is in ERROR state or
still ENABLED after a recoverable failure. Returns a shallow copy so
callers cannot mutate the stored snapshot.
Get error information for a plugin in ERROR state.
Args:
plugin_id: Plugin identifier
Returns:
Copy of the error information dict, or None
Error information dict or None
"""
with self._lock:
info = self._error_info.get(plugin_id)
return dict(info) if info is not None else None
return self._error_info.get(plugin_id)
def record_update(self, plugin_id: str) -> None:
"""Record that plugin update() was called."""

View File

@@ -20,6 +20,7 @@ except ImportError:
class ResourceLimitExceeded(Exception):
"""Raised when a plugin exceeds its resource limits."""
pass
@dataclass
@@ -227,7 +228,7 @@ class PluginResourceMonitor:
except ResourceLimitExceeded:
raise
except Exception:
except Exception as e:
# Still record execution time even on error
execution_time = time.time() - start_time
with self._lock:

View File

@@ -5,6 +5,7 @@ Manages saved GitHub repository URLs for easy plugin discovery and installation.
"""
import json
import os
import logging
from pathlib import Path
from typing import List, Dict, Optional

View File

@@ -8,6 +8,7 @@ Provides utilities for extracting defaults, validating configurations, and manag
import copy
import json
import logging
import os
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
import jsonschema

View File

@@ -8,12 +8,12 @@ Detects and fixes inconsistencies between:
- State manager state
"""
from typing import Dict, Any, List, Set
from typing import Dict, Any, List, Optional
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from src.plugin_system.state_manager import PluginStateManager
from src.plugin_system.state_manager import PluginStateManager, PluginState, PluginStateStatus
from src.logging_config import get_logger
@@ -86,38 +86,16 @@ class StateReconciliation:
self.plugins_dir = Path(plugins_dir)
self.store_manager = store_manager
self.logger = get_logger(__name__)
# Plugin IDs that failed auto-repair and should NOT be retried this
# process lifetime. Prevents the infinite "attempt to reinstall missing
# plugin" loop when a config entry references a plugin that isn't in
# the registry (e.g. legacy 'github', 'youtube' entries). A process
# restart — or an explicit user-initiated reconcile with force=True —
# clears this so recovery is possible after the underlying issue is
# fixed.
self._unrecoverable_missing_on_disk: Set[str] = set()
def reconcile_state(self, force: bool = False) -> ReconciliationResult:
def reconcile_state(self) -> ReconciliationResult:
"""
Perform state reconciliation.
Compares state from all sources and fixes safe inconsistencies.
Args:
force: If True, clear the unrecoverable-plugin cache before
reconciling so previously-failed auto-repairs are retried.
Intended for user-initiated reconcile requests after the
underlying issue (e.g. registry update) has been fixed.
Returns:
ReconciliationResult with findings and fixes
"""
if force and self._unrecoverable_missing_on_disk:
self.logger.info(
"Force reconcile requested; clearing %d cached unrecoverable plugin(s)",
len(self._unrecoverable_missing_on_disk),
)
self._unrecoverable_missing_on_disk.clear()
self.logger.info("Starting state reconciliation")
inconsistencies = []
@@ -185,19 +163,13 @@ class StateReconciliation:
message=f"Reconciliation failed: {str(e)}"
)
# Top-level config keys that are NOT plugins.
# Includes both config.json structural keys and config_secrets.json top-level
# keys (load_config() deep-merges secrets in, so secrets keys appear here too).
# Top-level config keys that are NOT plugins
_SYSTEM_CONFIG_KEYS = frozenset({
'web_display_autostart', 'timezone', 'location', 'display',
'plugin_system', 'vegas_scroll_speed', 'vegas_separator_width',
'vegas_target_fps', 'vegas_buffer_ahead', 'vegas_plugin_order',
'vegas_excluded_plugins', 'vegas_scroll_enabled', 'logging',
'dim_schedule', 'network', 'system', 'schedule',
# Multi-display sync config (config.json structural key)
'sync',
# Secrets file top-level keys (merged in by load_config)
'github', 'youtube',
})
def _get_config_state(self) -> Dict[str, Dict[str, Any]]:
@@ -240,7 +212,7 @@ class StateReconciliation:
'version': manifest.get('version'),
'name': manifest.get('name')
}
except Exception: # nosec B110 - corrupt/unreadable manifest; skip this plugin, outer except logs
except Exception:
pass
except Exception as e:
self.logger.warning(f"Error reading disk state: {e}")
@@ -291,6 +263,7 @@ class StateReconciliation:
config = config_state.get(plugin_id, {})
disk = disk_state.get(plugin_id, {})
manager = manager_state.get(plugin_id, {})
state_mgr = state_manager_state.get(plugin_id, {})
# Check: Plugin exists on disk but not in config
@@ -307,26 +280,7 @@ class StateReconciliation:
# Check: Plugin in config but not on disk
if config.get('exists_in_config') and not disk.get('exists_on_disk'):
# Skip plugins that previously failed auto-repair in this process.
# Re-attempting wastes CPU (network + git clone each request) and
# spams the logs with the same "Plugin not found in registry"
# error. The entry is still surfaced as MANUAL_FIX_REQUIRED so the
# UI can show it, but no auto-repair will run.
previously_unrecoverable = plugin_id in self._unrecoverable_missing_on_disk
# Also refuse to re-install a plugin that the user just uninstalled
# through the UI — prevents a race where the reconciler fires
# between file removal and config cleanup and resurrects the
# plugin the user just deleted.
recently_uninstalled = (
self.store_manager is not None
and hasattr(self.store_manager, 'was_recently_uninstalled')
and self.store_manager.was_recently_uninstalled(plugin_id)
)
can_repair = (
self.store_manager is not None
and not previously_unrecoverable
and not recently_uninstalled
)
can_repair = self.store_manager is not None
inconsistencies.append(Inconsistency(
plugin_id=plugin_id,
inconsistency_type=InconsistencyType.PLUGIN_MISSING_ON_DISK,
@@ -340,15 +294,15 @@ class StateReconciliation:
# Check: Enabled state mismatch
config_enabled = config.get('enabled', False)
state_mgr_enabled = state_mgr.get('enabled')
if state_mgr_enabled is not None and config_enabled != state_mgr_enabled:
inconsistencies.append(Inconsistency(
plugin_id=plugin_id,
inconsistency_type=InconsistencyType.PLUGIN_ENABLED_MISMATCH,
description=f"Plugin {plugin_id} enabled state mismatch: config={config_enabled}, state_manager={state_mgr_enabled}",
fix_action=FixAction.AUTO_FIX,
current_state={'enabled': state_mgr_enabled},
expected_state={'enabled': config_enabled},
current_state={'enabled': config_enabled},
expected_state={'enabled': state_mgr_enabled},
can_auto_fix=True
))
@@ -371,23 +325,15 @@ class StateReconciliation:
return self._auto_repair_missing_plugin(inconsistency.plugin_id)
elif inconsistency.inconsistency_type == InconsistencyType.PLUGIN_ENABLED_MISMATCH:
# config.json is the user-editable source of truth for enabled state.
# Bring the state manager in sync with config rather than the reverse,
# so that manual config edits (or the state left behind after an
# uninstall+reinstall cycle) don't silently override the user's intent.
config_enabled = inconsistency.expected_state.get('enabled')
success = self.state_manager.set_plugin_enabled(inconsistency.plugin_id, config_enabled)
if success:
self.logger.info(
f"Fixed: Synced state manager enabled={config_enabled} for "
f"{inconsistency.plugin_id} to match config"
)
else:
self.logger.warning(
f"Failed to sync state manager enabled={config_enabled} for "
f"{inconsistency.plugin_id}"
)
return success
# Sync enabled state from state manager to config
expected_enabled = inconsistency.expected_state.get('enabled')
config = self.config_manager.load_config()
if inconsistency.plugin_id not in config:
config[inconsistency.plugin_id] = {}
config[inconsistency.plugin_id]['enabled'] = expected_enabled
self.config_manager.save_config(config)
self.logger.info(f"Fixed: Synced enabled state for {inconsistency.plugin_id}")
return True
except Exception as e:
self.logger.error(f"Error fixing inconsistency: {e}", exc_info=True)
@@ -396,13 +342,7 @@ class StateReconciliation:
return False
def _auto_repair_missing_plugin(self, plugin_id: str) -> bool:
"""Attempt to reinstall a missing plugin from the store.
On failure, records plugin_id in ``_unrecoverable_missing_on_disk`` so
subsequent reconciliation passes within this process do not retry and
spam the log / CPU. A process restart (or an explicit ``force=True``
reconcile) is required to clear the cache.
"""
"""Attempt to reinstall a missing plugin from the store."""
if not self.store_manager:
return False
@@ -411,43 +351,6 @@ class StateReconciliation:
if plugin_id.startswith('ledmatrix-'):
candidates.append(plugin_id[len('ledmatrix-'):])
# Cheap pre-check: is any candidate actually present in the registry
# at all? If not, we know up-front this is unrecoverable and can skip
# the expensive install_plugin path (which does a forced GitHub fetch
# before failing).
#
# IMPORTANT: we must pass raise_on_failure=True here. The default
# fetch_registry() silently falls back to a stale cache or an empty
# dict on network failure, which would make it impossible to tell
# "plugin genuinely not in registry" from "I can't reach the
# registry right now" — in the second case we'd end up poisoning
# _unrecoverable_missing_on_disk with every config entry on a fresh
# boot with no cache.
registry_has_candidate = False
try:
registry = self.store_manager.fetch_registry(raise_on_failure=True)
registry_ids = {
p.get('id') for p in (registry.get('plugins', []) or []) if p.get('id')
}
registry_has_candidate = any(c in registry_ids for c in candidates)
except Exception as e:
# If we can't reach the registry, treat this as transient — don't
# mark unrecoverable, let the next pass try again.
self.logger.warning(
"[AutoRepair] Could not read registry to check %s: %s", plugin_id, e
)
return False
if not registry_has_candidate:
self.logger.warning(
"[AutoRepair] %s not present in registry; marking unrecoverable "
"(will not retry this session). Reinstall from the Plugin Store "
"or remove the stale config entry to clear this warning.",
plugin_id,
)
self._unrecoverable_missing_on_disk.add(plugin_id)
return False
for candidate_id in candidates:
try:
self.logger.info("[AutoRepair] Attempting to reinstall missing plugin: %s", candidate_id)
@@ -463,11 +366,6 @@ class StateReconciliation:
except Exception as e:
self.logger.error("[AutoRepair] Error reinstalling %s: %s", candidate_id, e, exc_info=True)
self.logger.warning(
"[AutoRepair] Could not reinstall %s from store; marking unrecoverable "
"(will not retry this session).",
plugin_id,
)
self._unrecoverable_missing_on_disk.add(plugin_id)
self.logger.warning("[AutoRepair] Could not reinstall %s from store", plugin_id)
return False

View File

@@ -5,28 +5,24 @@ Handles plugin discovery, installation, updates, and uninstallation
from both the official registry and custom GitHub repositories.
"""
import hashlib
import os
import json
import stat
import subprocess
import shutil
import threading
import zipfile
import tempfile
import requests
import time
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime
from pathlib import Path
from typing import List, Dict, Optional, Any, Tuple
from typing import List, Dict, Optional, Any
import logging
from urllib.parse import urlparse
from src.common.permission_utils import sudo_remove_directory
try:
import jsonschema
from jsonschema import Draft7Validator, ValidationError
JSONSCHEMA_AVAILABLE = True
except ImportError:
@@ -56,93 +52,19 @@ class PluginStoreManager:
self.registry_cache = None
self.registry_cache_time = None # Timestamp of when registry was cached
self.github_cache = {} # Cache for GitHub API responses
self.cache_timeout = 3600 # 1 hour cache timeout (repo info: stars, default_branch)
# 15 minutes for registry cache. Long enough that the plugin list
# endpoint on a warm cache never hits the network, short enough that
# new plugins show up within a reasonable window. See also the
# stale-cache fallback in fetch_registry for transient network
# failures.
self.registry_cache_timeout = 900
self.cache_timeout = 3600 # 1 hour cache timeout
self.registry_cache_timeout = 300 # 5 minutes for registry cache
self.commit_info_cache = {} # Cache for latest commit info: {key: (timestamp, data)}
# 30 minutes for commit/manifest caches. Plugin Store users browse
# the catalog via /plugins/store/list which fetches commit info and
# manifest data per plugin. 5-min TTLs meant every fresh browse on
# a Pi4 paid for ~3 HTTP requests x N plugins (30-60s serial). 30
# minutes keeps the cache warm across a realistic session while
# still picking up upstream updates within a reasonable window.
self.commit_cache_timeout = 1800
self.commit_cache_timeout = 300 # 5 minutes (same as registry)
self.manifest_cache = {} # Cache for GitHub manifest fetches: {key: (timestamp, data)}
self.manifest_cache_timeout = 1800
self.manifest_cache_timeout = 300 # 5 minutes
self.github_token = self._load_github_token()
self._token_validation_cache = {} # Cache for token validation results: {token: (is_valid, timestamp, error_message)}
self._token_validation_cache_timeout = 300 # 5 minutes cache for token validation
# Per-plugin tombstone timestamps for plugins that were uninstalled
# recently via the UI. Used by the state reconciler to avoid
# resurrecting a plugin the user just deleted when reconciliation
# races against the uninstall operation. Cleared after ``_uninstall_tombstone_ttl``.
self._uninstall_tombstones: Dict[str, float] = {}
self._uninstall_tombstone_ttl = 300 # 5 minutes
# Cache for _get_local_git_info: {plugin_path_str: (signature, data)}
# where ``signature`` is a tuple of (head_mtime, resolved_ref_mtime,
# head_contents) so a fast-forward update to the current branch
# (which touches .git/refs/heads/<branch> but NOT .git/HEAD) still
# invalidates the cache. Before this cache, every
# /plugins/installed request fired 4 git subprocesses per plugin,
# which pegged the CPU on a Pi4 with a dozen plugins. The cached
# ``data`` dict is the same shape returned by ``_get_local_git_info``
# itself (sha / short_sha / branch / optional remote_url, date_iso,
# date) — all string-keyed strings.
self._git_info_cache: Dict[str, Tuple[Tuple, Dict[str, str]]] = {}
# How long to wait before re-attempting a failed GitHub metadata
# fetch after we've already served a stale cache hit. Without this,
# a single expired-TTL + network-error would cause every subsequent
# request to re-hit the network (and fail again) until the network
# actually came back — amplifying the failure and blocking request
# handlers. Bumping the cached-entry timestamp on failure serves
# the stale payload cheaply until the backoff expires.
self._failure_backoff_seconds = 60
# Prevents concurrent callers from each firing a network request when
# the registry cache expires. Only one thread fetches; others wait and
# then get the result from the warm cache (double-checked locking).
self._registry_fetch_lock = threading.Lock()
# Ensure plugins directory exists
self.plugins_dir.mkdir(exist_ok=True)
def _record_cache_backoff(self, cache_dict: Dict, cache_key: str,
cache_timeout: int, payload: Any) -> None:
"""Bump a cache entry's timestamp so subsequent lookups hit the
cache rather than re-failing over the network.
Used by the stale-on-error fallbacks in the GitHub metadata fetch
paths. Without this, a cache entry whose TTL just expired would
cause every subsequent request to re-hit the network and fail
again until the network actually came back. We write a synthetic
timestamp ``(now + backoff - cache_timeout)`` so the cache-valid
check ``(now - ts) < cache_timeout`` succeeds for another
``backoff`` seconds.
"""
synthetic_ts = time.time() + self._failure_backoff_seconds - cache_timeout
cache_dict[cache_key] = (synthetic_ts, payload)
def mark_recently_uninstalled(self, plugin_id: str) -> None:
"""Record that ``plugin_id`` was just uninstalled by the user."""
self._uninstall_tombstones[plugin_id] = time.time()
def was_recently_uninstalled(self, plugin_id: str) -> bool:
"""Return True if ``plugin_id`` has an active uninstall tombstone."""
ts = self._uninstall_tombstones.get(plugin_id)
if ts is None:
return False
if time.time() - ts > self._uninstall_tombstone_ttl:
# Expired — clean up so the dict doesn't grow unbounded.
self._uninstall_tombstones.pop(plugin_id, None)
return False
return True
def _load_github_token(self) -> Optional[str]:
"""
Load GitHub API token from config_secrets.json if available.
@@ -359,8 +281,7 @@ class PluginStoreManager:
# Extract owner/repo from URL
try:
# Handle different URL formats
_parsed_url = urlparse(repo_url)
if _parsed_url.hostname in ('github.com', 'www.github.com'):
if 'github.com' in repo_url:
parts = repo_url.strip('/').split('/')
if len(parts) >= 2:
owner = parts[-2]
@@ -387,25 +308,7 @@ class PluginStoreManager:
if self.github_token:
headers['Authorization'] = f'token {self.github_token}'
try:
response = requests.get(api_url, headers=headers, timeout=10)
except requests.RequestException as req_err:
# Network error: prefer a stale cache hit over an
# empty default so the UI keeps working on a flaky
# Pi WiFi link. Bump the cached entry's timestamp
# into a short backoff window so subsequent
# requests serve the stale payload cheaply instead
# of re-hitting the network on every request.
if cache_key in self.github_cache:
_, stale = self.github_cache[cache_key]
self._record_cache_backoff(self.github_cache, cache_key, self.cache_timeout, stale)
self.logger.warning(
"GitHub repo info fetch failed for %s (%s); serving stale cache.",
cache_key, req_err,
)
return stale
raise
response = requests.get(api_url, headers=headers, timeout=10)
if response.status_code == 200:
data = response.json()
pushed_at = data.get('pushed_at', '') or data.get('updated_at', '')
@@ -425,25 +328,12 @@ class PluginStoreManager:
self.github_cache[cache_key] = (time.time(), repo_info)
return repo_info
elif response.status_code == 403:
# Rate limit or authentication issue. If we have a
# previously-cached value, serve it rather than
# returning empty defaults — a stale star count is
# better than a reset to zero. Apply the same
# failure-backoff bump as the network-error path
# so we don't hammer the API with repeat requests
# while rate-limited.
if cache_key in self.github_cache:
_, stale = self.github_cache[cache_key]
self._record_cache_backoff(self.github_cache, cache_key, self.cache_timeout, stale)
self.logger.warning(
"GitHub API 403 for %s; serving stale cache.", cache_key,
)
return stale
# Rate limit or authentication issue
if not self.github_token:
self.logger.warning(
"GitHub API rate limit likely exceeded (403). "
"Add a GitHub personal access token to config/config_secrets.json "
"under 'github.api_token' to increase rate limits from 60 to 5000/hour."
f"GitHub API rate limit likely exceeded (403). "
f"Add a GitHub personal access token to config/config_secrets.json "
f"under 'github.api_token' to increase rate limits from 60 to 5000/hour."
)
else:
self.logger.warning(
@@ -452,10 +342,6 @@ class PluginStoreManager:
)
else:
self.logger.warning(f"GitHub API request failed: {response.status_code} for {api_url}")
if cache_key in self.github_cache:
_, stale = self.github_cache[cache_key]
self._record_cache_backoff(self.github_cache, cache_key, self.cache_timeout, stale)
return stale
return {
'stars': 0,
@@ -522,10 +408,9 @@ class PluginStoreManager:
# Try to find plugins.json in common locations
# First try root directory
registry_urls = []
# Extract owner/repo from URL
_parsed_repo_url = urlparse(repo_url)
if _parsed_repo_url.hostname in ('github.com', 'www.github.com'):
if 'github.com' in repo_url:
parts = repo_url.split('/')
if len(parts) >= 2:
owner = parts[-2]
@@ -557,78 +442,37 @@ class PluginStoreManager:
self.logger.error(f"Error fetching registry from URL: {e}", exc_info=True)
return None
def fetch_registry(self, force_refresh: bool = False, raise_on_failure: bool = False) -> Dict:
def fetch_registry(self, force_refresh: bool = False) -> Dict:
"""
Fetch the plugin registry from GitHub.
Args:
force_refresh: Force refresh even if cached
raise_on_failure: If True, re-raise network / JSON errors instead
of silently falling back to stale cache / empty dict. UI
callers prefer the stale-fallback default so the plugin
list keeps working on flaky WiFi; the state reconciler
needs the explicit failure signal so it can distinguish
"plugin genuinely not in registry" from "I couldn't reach
the registry at all" and not mark everything unrecoverable.
Returns:
Registry data with list of available plugins
Raises:
requests.RequestException / json.JSONDecodeError when
``raise_on_failure`` is True and the fetch fails.
"""
# Check if cache is still valid (within timeout)
current_time = time.time()
if (self.registry_cache and self.registry_cache_time and
not force_refresh and
if (self.registry_cache and self.registry_cache_time and
not force_refresh and
(current_time - self.registry_cache_time) < self.registry_cache_timeout):
return self.registry_cache
with self._registry_fetch_lock:
# Re-check inside the lock — a concurrent caller that was waiting
# may have already populated the cache while we blocked.
current_time = time.time()
if (self.registry_cache and self.registry_cache_time and
not force_refresh and
(current_time - self.registry_cache_time) < self.registry_cache_timeout):
return self.registry_cache
try:
self.logger.info(f"Fetching plugin registry from {self.REGISTRY_URL}")
response = self._http_get_with_retries(self.REGISTRY_URL, timeout=10)
response.raise_for_status()
self.registry_cache = response.json()
self.registry_cache_time = current_time
self.logger.info(f"Fetched registry with {len(self.registry_cache.get('plugins', []))} plugins")
return self.registry_cache
except requests.RequestException as e:
self.logger.error(f"Error fetching registry: {e}")
if raise_on_failure:
raise
# Prefer stale cache over an empty list so the plugin list UI
# keeps working on a flaky connection (e.g. Pi on WiFi). Bump
# registry_cache_time into a short backoff window so the next
# request serves the stale payload cheaply instead of
# re-hitting the network on every request (matches the
# pattern used by github_cache / commit_info_cache).
if self.registry_cache:
self.logger.warning("Falling back to stale registry cache")
self.registry_cache_time = (
time.time() + self._failure_backoff_seconds - self.registry_cache_timeout
)
return self.registry_cache
return {"plugins": []}
except json.JSONDecodeError as e:
self.logger.error(f"Error parsing registry JSON: {e}")
if raise_on_failure:
raise
if self.registry_cache:
self.registry_cache_time = (
time.time() + self._failure_backoff_seconds - self.registry_cache_timeout
)
return self.registry_cache
return {"plugins": []}
try:
self.logger.info(f"Fetching plugin registry from {self.REGISTRY_URL}")
response = self._http_get_with_retries(self.REGISTRY_URL, timeout=10)
response.raise_for_status()
self.registry_cache = response.json()
self.registry_cache_time = current_time
self.logger.info(f"Fetched registry with {len(self.registry_cache.get('plugins', []))} plugins")
return self.registry_cache
except requests.RequestException as e:
self.logger.error(f"Error fetching registry: {e}")
return {"plugins": []}
except json.JSONDecodeError as e:
self.logger.error(f"Error parsing registry JSON: {e}")
return {"plugins": []}
def search_plugins(self, query: str = "", category: str = "", tags: List[str] = None, fetch_commit_info: bool = True, include_saved_repos: bool = True, saved_repositories_manager = None) -> List[Dict]:
"""
@@ -673,95 +517,68 @@ class PluginStoreManager:
except Exception as e:
self.logger.warning(f"Failed to fetch plugins from saved repository {repo_url}: {e}")
# First pass: apply cheap filters (category/tags/query) so we only
# fetch GitHub metadata for plugins that will actually be returned.
filtered: List[Dict] = []
results = []
for plugin in plugins:
# Category filter
if category and plugin.get('category') != category:
continue
# Tags filter (match any tag)
if tags and not any(tag in plugin.get('tags', []) for tag in tags):
continue
# Query search (case-insensitive)
if query:
query_lower = query.lower()
searchable_text = ' '.join([
plugin.get('name', ''),
plugin.get('description', ''),
plugin.get('id', ''),
plugin.get('author', ''),
plugin.get('author', '')
]).lower()
if query_lower not in searchable_text:
continue
filtered.append(plugin)
def _enrich(plugin: Dict) -> Dict:
"""Enrich a single plugin with GitHub metadata.
Called concurrently from a ThreadPoolExecutor. Each underlying
HTTP helper (``_get_github_repo_info`` / ``_get_latest_commit_info``
/ ``_fetch_manifest_from_github``) is thread-safe — they use
``requests`` and write their own cache keys on Python dicts,
which is atomic under the GIL for single-key assignments.
"""
# Enhance plugin data with GitHub metadata
enhanced_plugin = plugin.copy()
# Get real GitHub stars
repo_url = plugin.get('repo', '')
if not repo_url:
return enhanced_plugin
if repo_url:
github_info = self._get_github_repo_info(repo_url)
enhanced_plugin['stars'] = github_info.get('stars', plugin.get('stars', 0))
enhanced_plugin['default_branch'] = github_info.get('default_branch', plugin.get('branch', 'main'))
enhanced_plugin['last_updated_iso'] = github_info.get('last_commit_iso')
enhanced_plugin['last_updated'] = github_info.get('last_commit_date')
github_info = self._get_github_repo_info(repo_url)
enhanced_plugin['stars'] = github_info.get('stars', plugin.get('stars', 0))
enhanced_plugin['default_branch'] = github_info.get('default_branch', plugin.get('branch', 'main'))
enhanced_plugin['last_updated_iso'] = github_info.get('last_commit_iso')
enhanced_plugin['last_updated'] = github_info.get('last_commit_date')
if fetch_commit_info:
branch = plugin.get('branch') or github_info.get('default_branch', 'main')
if fetch_commit_info:
branch = plugin.get('branch') or github_info.get('default_branch', 'main')
commit_info = self._get_latest_commit_info(repo_url, branch)
if commit_info:
enhanced_plugin['last_commit'] = commit_info.get('short_sha')
enhanced_plugin['last_commit_sha'] = commit_info.get('sha')
enhanced_plugin['last_updated'] = commit_info.get('date') or enhanced_plugin.get('last_updated')
enhanced_plugin['last_updated_iso'] = commit_info.get('date_iso') or enhanced_plugin.get('last_updated_iso')
enhanced_plugin['last_commit_message'] = commit_info.get('message')
enhanced_plugin['last_commit_author'] = commit_info.get('author')
enhanced_plugin['branch'] = commit_info.get('branch', branch)
enhanced_plugin['last_commit_branch'] = commit_info.get('branch')
commit_info = self._get_latest_commit_info(repo_url, branch)
if commit_info:
enhanced_plugin['last_commit'] = commit_info.get('short_sha')
enhanced_plugin['last_commit_sha'] = commit_info.get('sha')
enhanced_plugin['last_updated'] = commit_info.get('date') or enhanced_plugin.get('last_updated')
enhanced_plugin['last_updated_iso'] = commit_info.get('date_iso') or enhanced_plugin.get('last_updated_iso')
enhanced_plugin['last_commit_message'] = commit_info.get('message')
enhanced_plugin['last_commit_author'] = commit_info.get('author')
enhanced_plugin['branch'] = commit_info.get('branch', branch)
enhanced_plugin['last_commit_branch'] = commit_info.get('branch')
# Fetch manifest from GitHub for additional metadata (description, etc.)
plugin_subpath = plugin.get('plugin_path', '')
manifest_rel = f"{plugin_subpath}/manifest.json" if plugin_subpath else "manifest.json"
github_manifest = self._fetch_manifest_from_github(repo_url, branch, manifest_rel)
if github_manifest:
if 'last_updated' in github_manifest and not enhanced_plugin.get('last_updated'):
enhanced_plugin['last_updated'] = github_manifest['last_updated']
if 'description' in github_manifest:
enhanced_plugin['description'] = github_manifest['description']
# Intentionally NO per-plugin manifest.json fetch here.
# The registry's plugins.json already carries ``description``
# (it is generated from each plugin's manifest by
# ``update_registry.py``), and ``last_updated`` is filled in
# from the commit info above. An earlier implementation
# fetched manifest.json per plugin anyway, which meant one
# extra HTTPS round trip per result; on a Pi4 with a flaky
# WiFi link the tail retries of that one extra call
# (_http_get_with_retries does 3 attempts with exponential
# backoff) dominated wall time even after parallelization.
results.append(enhanced_plugin)
return enhanced_plugin
# Fan out the per-plugin GitHub enrichment. The previous
# implementation did this serially, which on a Pi4 with ~15 plugins
# and a fresh cache meant 30+ HTTP requests in strict sequence (the
# "connecting to display" hang reported by users). With a thread
# pool, latency is dominated by the slowest request rather than
# their sum. Workers capped at 10 to stay well under the
# unauthenticated GitHub rate limit burst and avoid overwhelming a
# Pi's WiFi link. For a small number of plugins the pool is
# essentially free.
if not filtered:
return []
# Not worth the pool overhead for tiny workloads. Parenthesized to
# make Python's default ``and`` > ``or`` precedence explicit: a
# single plugin, OR a small batch where we don't need commit info.
if (len(filtered) == 1) or ((not fetch_commit_info) and (len(filtered) < 4)):
return [_enrich(p) for p in filtered]
max_workers = min(10, len(filtered))
with ThreadPoolExecutor(max_workers=max_workers, thread_name_prefix='plugin-search') as executor:
# executor.map preserves input order, which the UI relies on.
return list(executor.map(_enrich, filtered))
return results
def _fetch_manifest_from_github(self, repo_url: str, branch: str = "master", manifest_path: str = "manifest.json", force_refresh: bool = False) -> Optional[Dict]:
"""
@@ -780,8 +597,7 @@ class PluginStoreManager:
try:
# Convert repo URL to raw content URL
# https://github.com/user/repo -> https://raw.githubusercontent.com/user/repo/branch/manifest.json
_parsed_manifest_url = urlparse(repo_url)
if _parsed_manifest_url.hostname in ('github.com', 'www.github.com'):
if 'github.com' in repo_url:
# Handle different URL formats
repo_url = repo_url.rstrip('/')
if repo_url.endswith('.git'):
@@ -860,28 +676,7 @@ class PluginStoreManager:
last_error = None
for branch_name in branches_to_try:
api_url = f"https://api.github.com/repos/{owner}/{repo}/commits/{branch_name}"
try:
response = requests.get(api_url, headers=headers, timeout=10)
except requests.RequestException as req_err:
# Network failure: fall back to a stale cache hit if
# available so the plugin store UI keeps populating
# commit info on a flaky WiFi link. Bump the cached
# timestamp into the backoff window so we don't
# re-retry on every request.
if cache_key in self.commit_info_cache:
_, stale = self.commit_info_cache[cache_key]
if stale is not None:
self._record_cache_backoff(
self.commit_info_cache, cache_key,
self.commit_cache_timeout, stale,
)
self.logger.warning(
"GitHub commit fetch failed for %s (%s); serving stale cache.",
cache_key, req_err,
)
return stale
last_error = str(req_err)
continue
response = requests.get(api_url, headers=headers, timeout=10)
if response.status_code == 200:
commit_data = response.json()
commit_sha_full = commit_data.get('sha', '')
@@ -911,23 +706,7 @@ class PluginStoreManager:
if last_error:
self.logger.debug(f"Unable to fetch commit info for {repo_url}: {last_error}")
# All branches returned a non-200 response (e.g. 404 on every
# candidate, or a transient 5xx). If we already had a good
# cached value, prefer serving that — overwriting it with
# None here would wipe out commit info the UI just showed
# on the previous request. Bump the timestamp into the
# backoff window so subsequent lookups hit the cache.
if cache_key in self.commit_info_cache:
_, prior = self.commit_info_cache[cache_key]
if prior is not None:
self._record_cache_backoff(
self.commit_info_cache, cache_key,
self.commit_cache_timeout, prior,
)
return prior
# No prior good value — cache the negative result so we don't
# hammer a plugin that genuinely has no reachable commits.
# Cache negative result to avoid repeated failing calls
self.commit_info_cache[cache_key] = (time.time(), None)
except Exception as e:
@@ -1097,7 +876,7 @@ class PluginStoreManager:
# Get the actual plugin ID from manifest (source of truth)
manifest_plugin_id = manifest.get('id')
if not manifest_plugin_id:
self.logger.error("Plugin manifest missing 'id' field")
self.logger.error(f"Plugin manifest missing 'id' field")
self._safe_remove_directory(plugin_path)
return False
@@ -1748,7 +1527,7 @@ class PluginStoreManager:
try:
self.logger.info(f"Installing dependencies for {plugin_path.name}")
subprocess.run(
result = subprocess.run(
['pip3', 'install', '--break-system-packages', '-r', str(requirements_file)],
check=True,
capture_output=True,
@@ -1756,12 +1535,6 @@ class PluginStoreManager:
timeout=300
)
self.logger.info(f"Dependencies installed successfully for {plugin_path.name}")
# Write hash marker so plugin_loader skips redundant pip run on next startup
try:
current_hash = hashlib.sha256(requirements_file.read_bytes()).hexdigest()
(plugin_path / ".dependencies_installed").write_text(current_hash, encoding='utf-8')
except OSError as marker_err:
self.logger.debug("Could not write dependency marker for %s: %s", plugin_path.name, marker_err)
return True
except subprocess.CalledProcessError as e:
@@ -1787,166 +1560,69 @@ class PluginStoreManager:
self.logger.error(f"Unexpected error installing dependencies for {plugin_path.name}: {e}", exc_info=True)
return False
def _git_cache_signature(self, git_dir: Path) -> Optional[Tuple]:
"""Build a cache signature that invalidates on the kind of updates
a plugin user actually cares about.
Caching on ``.git/HEAD`` mtime alone is not enough: a ``git pull``
that fast-forwards the current branch updates
``.git/refs/heads/<branch>`` (or ``.git/packed-refs``) but leaves
HEAD's contents and mtime untouched. And the cached ``result``
dict includes ``remote_url`` — a value read from ``.git/config`` —
so a config-only change (e.g. a monorepo-migration re-pointing
``remote.origin.url``) must also invalidate the cache.
Signature components:
- HEAD contents (catches detach / branch switch)
- HEAD mtime
- if HEAD points at a ref, that ref file's mtime (catches
fast-forward / reset on the current branch)
- packed-refs mtime as a coarse fallback for repos using packed refs
- .git/config contents + mtime (catches remote URL changes and
any other config-only edit that affects what the cached
``remote_url`` field should contain)
Returns ``None`` if HEAD cannot be read at all (caller will skip
the cache and take the slow path).
"""
head_file = git_dir / 'HEAD'
try:
head_mtime = head_file.stat().st_mtime
head_contents = head_file.read_text(encoding='utf-8', errors='replace').strip()
except OSError:
return None
ref_mtime = None
if head_contents.startswith('ref: '):
ref_path = head_contents[len('ref: '):].strip()
# ``ref_path`` looks like ``refs/heads/main``. It lives either
# as a loose file under .git/ or inside .git/packed-refs.
loose_ref = git_dir / ref_path
try:
ref_mtime = loose_ref.stat().st_mtime
except OSError:
ref_mtime = None
packed_refs_mtime = None
if ref_mtime is None:
try:
packed_refs_mtime = (git_dir / 'packed-refs').stat().st_mtime
except OSError:
packed_refs_mtime = None
config_mtime = None
config_contents = None
config_file = git_dir / 'config'
try:
config_mtime = config_file.stat().st_mtime
config_contents = config_file.read_text(encoding='utf-8', errors='replace').strip()
except OSError:
config_mtime = None
config_contents = None
return (
head_contents, head_mtime,
ref_mtime, packed_refs_mtime,
config_contents, config_mtime,
)
def _get_local_git_info(self, plugin_path: Path) -> Optional[Dict[str, str]]:
"""Return local git branch, commit hash, and commit date if the plugin is a git checkout.
Results are cached keyed on a signature that includes HEAD
contents plus the mtime of HEAD AND the resolved ref (or
packed-refs). Repeated calls skip the four ``git`` subprocesses
when nothing has changed, and a ``git pull`` that fast-forwards
the branch correctly invalidates the cache.
"""
"""Return local git branch, commit hash, and commit date if the plugin is a git checkout."""
git_dir = plugin_path / '.git'
if not git_dir.exists():
return None
cache_key = str(plugin_path)
signature = self._git_cache_signature(git_dir)
if signature is not None:
cached = self._git_info_cache.get(cache_key)
if cached is not None and cached[0] == signature:
return cached[1]
try:
# .git may be a file (worktree / submodule) containing "gitdir: <path>".
# Resolve it to the actual git directory before reading any files.
try:
if git_dir.is_file():
pointer = git_dir.read_text(encoding='utf-8', errors='replace').strip()
if pointer.startswith('gitdir:'):
resolved = (plugin_path / pointer[len('gitdir:'):].strip()).resolve()
if resolved.is_dir():
git_dir = resolved
else:
return None
else:
return None
except (OSError, NotADirectoryError):
return None
# Read branch directly from .git/HEAD (no subprocess).
branch = ''
try:
head_text = (git_dir / 'HEAD').read_text(encoding='utf-8', errors='replace').strip()
if head_text.startswith('ref: refs/heads/'):
branch = head_text[len('ref: refs/heads/'):]
elif head_text.startswith('ref: '):
branch = head_text[len('ref: '):]
# else: detached HEAD — branch stays ''
except (OSError, NotADirectoryError):
pass
# Remote URL from .git/config — parse [remote "origin"] url line.
remote_url = None
try:
config_text = (git_dir / 'config').read_text(encoding='utf-8', errors='replace')
in_origin = False
for line in config_text.splitlines():
stripped = line.strip()
if stripped == '[remote "origin"]':
in_origin = True
elif stripped.startswith('['):
in_origin = False
elif in_origin and stripped.startswith('url') and '=' in stripped:
remote_url = stripped.split('=', 1)[1].strip()
break
except (OSError, NotADirectoryError):
pass
# Single subprocess: SHA + commit date in one call.
log_result = subprocess.run(
['git', '-C', str(plugin_path), 'log', '-1', '--format=%H%n%cI', 'HEAD'],
sha_result = subprocess.run(
['git', '-C', str(plugin_path), 'rev-parse', 'HEAD'],
capture_output=True,
text=True,
timeout=10,
check=True
)
lines = log_result.stdout.strip().splitlines()
sha = lines[0] if lines else ''
commit_date_iso = lines[1] if len(lines) > 1 else ''
sha = sha_result.stdout.strip()
branch_result = subprocess.run(
['git', '-C', str(plugin_path), 'rev-parse', '--abbrev-ref', 'HEAD'],
capture_output=True,
text=True,
timeout=10,
check=True
)
branch = branch_result.stdout.strip()
if branch == 'HEAD':
branch = ''
# Get remote URL
remote_url_result = subprocess.run(
['git', '-C', str(plugin_path), 'config', '--get', 'remote.origin.url'],
capture_output=True,
text=True,
timeout=10,
check=False
)
remote_url = remote_url_result.stdout.strip() if remote_url_result.returncode == 0 else None
# Get commit date in ISO format
date_result = subprocess.run(
['git', '-C', str(plugin_path), 'log', '-1', '--format=%cI', 'HEAD'],
capture_output=True,
text=True,
timeout=10,
check=True
)
commit_date_iso = date_result.stdout.strip()
result = {
'sha': sha,
'short_sha': sha[:7] if sha else '',
'branch': branch,
'branch': branch
}
# Add remote URL if available
if remote_url:
result['remote_url'] = remote_url
# Add commit date if available
if commit_date_iso:
result['date_iso'] = commit_date_iso
result['date'] = self._iso_to_date(commit_date_iso)
if signature is not None:
self._git_info_cache[cache_key] = (signature, result)
return result
except subprocess.CalledProcessError as err:
self.logger.debug(f"Failed to read git info for {plugin_path.name}: {err}")
@@ -2429,7 +2105,7 @@ class PluginStoreManager:
if not plugin_info_remote:
self.logger.warning(f"Plugin {plugin_id} not found in registry and not a git repository; cannot update automatically")
if not repo_url:
self.logger.warning("Plugin may have been installed via ZIP download. Try reinstalling from GitHub URL to enable updates.")
self.logger.warning(f"Plugin may have been installed via ZIP download. Try reinstalling from GitHub URL to enable updates.")
return False
repo_url = plugin_info_remote.get('repo')

View File

@@ -6,6 +6,7 @@ and plugin_manager for use in plugin unit tests.
"""
from typing import Dict, Any, Optional
from unittest.mock import MagicMock
from PIL import Image

View File

@@ -16,7 +16,7 @@ import math
import os
import time
from pathlib import Path
from typing import Any, List, Optional, Tuple
from typing import Any, Dict, List, Optional, Tuple
from PIL import Image, ImageDraw, ImageFont
@@ -236,6 +236,7 @@ class VisualTestDisplayManager:
Replicated from DisplayManager._draw_bdf_text().
"""
try:
import freetype
if isinstance(color, list):
color = tuple(color)
face = font if font else self.calendar_font

View File

@@ -6,7 +6,8 @@ Fails fast with clear error messages to prevent runtime issues.
"""
import os
from typing import Any, List, Optional, Tuple
import logging
from typing import Dict, Any, List, Optional, Tuple
from pathlib import Path
from src.exceptions import ConfigError, PluginError, CacheError
from src.logging_config import get_logger

View File

@@ -6,7 +6,7 @@ plugin ordering, exclusions, scroll speed, and display settings.
"""
import logging
from typing import Dict, Any, List, Set
from typing import Dict, Any, List, Set, Optional
from dataclasses import dataclass, field
logger = logging.getLogger(__name__)

View File

@@ -90,11 +90,13 @@ class VegasModeCoordinator:
self._interrupt_check: Optional[Callable[[], bool]] = None
self._interrupt_check_interval: int = 10 # Check every N frames
# Plugin update callback — fired from a background thread inside the loop
# so the main loop's _tick_plugin_updates() finds nothing due when Vegas
# returns, eliminating the inter-iteration frozen-frame gap.
self._update_callback: Optional[Callable[[], None]] = None
self._update_tick_running: bool = False
# Plugin update tick for keeping data fresh during Vegas mode
self._update_tick: Optional[Callable[[], Optional[List[str]]]] = None
self._update_tick_interval: float = 1.0 # Tick every 1 second
self._update_thread: Optional[threading.Thread] = None
self._update_results: Optional[List[str]] = None
self._update_results_lock = threading.Lock()
self._last_update_tick_time: float = 0.0
# Config update tracking
self._config_version = 0
@@ -137,25 +139,6 @@ class VegasModeCoordinator:
"""Check if Vegas mode is currently running."""
return self._is_active
def set_sync_manager(self, sync_manager, follower_position: str = "left") -> None:
"""
Attach a DisplaySyncManager so Vegas mode sends the follower's portion
of the ticker to the second display on every rendered frame.
Args:
sync_manager: DisplaySyncManager instance, or None to disable sync
follower_position: "left" (default) or "right" — physical position of
the follower display relative to the leader
"""
if self.render_pipeline:
# Don't expose a standalone (no-op) manager to the pipeline — treat it as None
if sync_manager is not None and hasattr(sync_manager, 'role'):
from src.common.sync_manager import SyncRole
if sync_manager.role == SyncRole.STANDALONE:
sync_manager = None
self.render_pipeline.sync_manager = sync_manager
self.render_pipeline.sync_follower_left = (follower_position == "left")
def set_live_priority_checker(self, checker: Callable[[], Optional[str]]) -> None:
"""
Set the callback for checking live priority content.
@@ -183,19 +166,24 @@ class VegasModeCoordinator:
self._interrupt_check = checker
self._interrupt_check_interval = max(1, check_interval)
def set_update_callback(self, callback: Callable[[], None]) -> None:
def set_update_tick(
self,
callback: Callable[[], Optional[List[str]]],
interval: float = 1.0
) -> None:
"""
Set a callback for running plugin updates from inside the Vegas loop.
Set the callback for periodic plugin update ticking during Vegas mode.
Fired in a daemon background thread every ~4 s so plugin data stays
fresh without blocking the render loop. The main loop's
_tick_plugin_updates() then finds all intervals already satisfied and
returns immediately, collapsing the inter-iteration gap to <1 ms.
This keeps plugin data fresh while the Vegas render loop is running.
The callback should run scheduled plugin updates and return a list of
plugin IDs that were actually updated, or None/empty if no updates occurred.
Args:
callback: Callable with no arguments (typically _tick_plugin_updates)
callback: Callable that returns list of updated plugin IDs or None
interval: Seconds between update tick calls (default 1.0)
"""
self._update_callback = callback
self._update_tick = callback
self._update_tick_interval = max(0.5, interval)
def start(self) -> bool:
"""
@@ -249,6 +237,9 @@ class VegasModeCoordinator:
self.stats['total_runtime_seconds'] += time.time() - self._start_time
self._start_time = None
# Wait for in-flight background update before tearing down state
self._drain_update_thread()
# Cleanup components
self.render_pipeline.reset()
self.stream_manager.reset()
@@ -344,101 +335,83 @@ class VegasModeCoordinator:
last_fps_log_time = start_time
fps_frame_count = 0
self._last_update_tick_time = start_time
logger.info("Starting Vegas iteration for %.1fs", duration)
while True:
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
if static_plugin:
if not self._handle_static_pause(static_plugin):
# Static pause was interrupted
return False
# After static pause, skip this segment and continue
self.stream_manager.get_next_segment() # Consume the segment
continue
# Run frame
if not self.run_frame():
# Check why we stopped
with self._state_lock:
if self._should_stop:
return False
if self._is_paused:
# Paused for live priority - let caller handle
try:
while True:
# Check for STATIC mode plugin that should pause scroll
static_plugin = self._check_static_plugin_trigger()
if static_plugin:
if not self._handle_static_pause(static_plugin):
# Static pause was interrupted
return False
# After static pause, skip this segment and continue
self.stream_manager.get_next_segment() # Consume the segment
continue
# Sleep for frame interval
time.sleep(frame_interval)
# Run frame
if not self.run_frame():
# Check why we stopped
with self._state_lock:
if self._should_stop:
return False
if self._is_paused:
# Paused for live priority - let caller handle
return False
# Increment frame count and check for interrupt periodically
frame_count += 1
fps_frame_count += 1
# Sleep for frame interval
time.sleep(frame_interval)
# Periodic FPS logging
current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d)",
fps, self.vegas_config.target_fps, fps_frame_count
)
last_fps_log_time = current_time
fps_frame_count = 0
# Increment frame count and check for interrupt periodically
frame_count += 1
fps_frame_count += 1
if (self._interrupt_check and
frame_count % self._interrupt_check_interval == 0):
try:
if self._interrupt_check():
logger.debug(
"Vegas interrupted by callback after %d frames",
frame_count
)
return False
except Exception:
# Log but don't let interrupt check errors stop Vegas
logger.exception("Interrupt check failed")
# Periodic FPS logging
current_time = time.time()
if current_time - last_fps_log_time >= fps_log_interval:
fps = fps_frame_count / (current_time - last_fps_log_time)
logger.info(
"Vegas FPS: %.1f (target: %d, frames: %d)",
fps, self.vegas_config.target_fps, fps_frame_count
)
last_fps_log_time = current_time
fps_frame_count = 0
# Fire plugin update tick in a background thread every ~4 s.
# Running it here (rather than only between iterations) means the
# main loop's _tick_plugin_updates() finds all intervals already
# satisfied on return, so the inter-iteration gap is <1 ms and the
# display never shows a frozen frame between iterations.
_UPDATE_TICK_FRAMES = max(1, int(self.vegas_config.target_fps * 4)) # every 4 s regardless of FPS
if (self._update_callback and
frame_count % _UPDATE_TICK_FRAMES == 0 and
not self._update_tick_running):
self._update_tick_running = True
def _run_tick(cb=self._update_callback):
# Periodic plugin update tick to keep data fresh (non-blocking)
self._drive_background_updates()
if (self._interrupt_check and
frame_count % self._interrupt_check_interval == 0):
try:
cb()
finally:
self._update_tick_running = False
threading.Thread(
target=_run_tick, daemon=True, name="vegas-plugin-tick"
).start()
if self._interrupt_check():
logger.debug(
"Vegas interrupted by callback after %d frames",
frame_count
)
return False
except Exception:
# Log but don't let interrupt check errors stop Vegas
logger.exception("Interrupt check failed")
# Check elapsed time
elapsed = time.time() - start_time
if elapsed >= duration:
break
# Check elapsed time
elapsed = time.time() - start_time
if elapsed >= duration:
break
# NOTE: do NOT break on is_cycle_complete() here.
# When multi-display sync is active, breaking exits run_iteration()
# which causes a 2-3s delay before start_new_cycle() is called on
# the next run_iteration(). During that gap the scroll advances into
# the pre-roll zone, then start_new_cycle() resets it — producing a
# second visible jump on the follower display ~2.5s after the first.
#
# Instead, run_frame() handles cycle completion directly (it calls
# start_new_cycle() in the very next frame, 8ms later), collapsing
# the two events into a single clean transition.
#
# Without sync, the iteration now runs to its full duration and may
# cycle content multiple times within one iteration — acceptable for
# a continuous ticker.
# Check for cycle completion
if self.render_pipeline.is_cycle_complete():
break
logger.info("Vegas iteration completed after %.1fs", time.time() - start_time)
return True
logger.info("Vegas iteration completed after %.1fs", time.time() - start_time)
return True
finally:
# Ensure background update thread finishes before the main loop
# resumes its own _tick_plugin_updates() calls, preventing concurrent
# run_scheduled_updates() execution.
self._drain_update_thread()
def _check_live_priority(self) -> bool:
"""
@@ -527,6 +500,71 @@ class VegasModeCoordinator:
if self._pending_config is None:
self._pending_config_update = False
def _run_update_tick_background(self) -> None:
"""Run the plugin update tick in a background thread.
Stores results for the render loop to pick up on its next iteration,
so the scroll never blocks on API calls.
"""
try:
updated_plugins = self._update_tick()
if updated_plugins:
with self._update_results_lock:
# Accumulate rather than replace to avoid losing notifications
# if a previous result hasn't been picked up yet
if self._update_results is None:
self._update_results = updated_plugins
else:
self._update_results.extend(updated_plugins)
except Exception:
logger.exception("Background plugin update tick failed")
def _drain_update_thread(self, timeout: float = 2.0) -> None:
"""Wait for any in-flight background update thread to finish.
Called when transitioning out of Vegas mode so the main-loop
``_tick_plugin_updates`` call doesn't race with a still-running
background thread.
"""
if self._update_thread is not None and self._update_thread.is_alive():
self._update_thread.join(timeout=timeout)
if self._update_thread.is_alive():
logger.warning(
"Background update thread did not finish within %.1fs", timeout
)
def _drive_background_updates(self) -> None:
"""Collect finished background update results and launch new ticks.
Safe to call from both the main render loop and the static-pause
wait loop so that plugin data stays fresh regardless of which
code path is active.
"""
# 1. Collect results from a previously completed background update
with self._update_results_lock:
ready_results = self._update_results
self._update_results = None
if ready_results:
for pid in ready_results:
self.mark_plugin_updated(pid)
# 2. Kick off a new background update if interval elapsed and none running
current_time = time.time()
if (self._update_tick and
current_time - self._last_update_tick_time >= self._update_tick_interval):
thread_alive = (
self._update_thread is not None
and self._update_thread.is_alive()
)
if not thread_alive:
self._last_update_tick_time = current_time
self._update_thread = threading.Thread(
target=self._run_update_tick_background,
daemon=True,
name="vegas-update-tick",
)
self._update_thread.start()
def mark_plugin_updated(self, plugin_id: str) -> None:
"""
Notify that a plugin's data has been updated.
@@ -645,10 +683,8 @@ class VegasModeCoordinator:
logger.info("Static pause interrupted by live priority")
return False
# Yield immediately if multi-display follower mode becomes active
if self._interrupt_check and self._interrupt_check():
logger.info("Static pause interrupted by sync follower mode")
return False
# Keep plugin data fresh during static pause
self._drive_background_updates()
# Sleep in small increments to remain responsive
time.sleep(0.1)

View File

@@ -329,51 +329,50 @@ class PluginAdapter:
# Save display state to restore after
original_image = self.display_manager.image.copy()
with self.display_manager.capture_mode():
# Method 1: Try _create_scrolling_display (stocks pattern)
if hasattr(plugin, '_create_scrolling_display'):
logger.info(
"[%s] Triggering via _create_scrolling_display()",
plugin_id
)
try:
plugin._create_scrolling_display()
cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] _create_scrolling_display() SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
return cached_image
except (AttributeError, TypeError, ValueError, OSError):
logger.exception(
"[%s] _create_scrolling_display() failed", plugin_id
)
# Method 2: Try display(force_clear=True) which typically builds scroll content
if hasattr(plugin, 'display'):
logger.info(
"[%s] Triggering via display(force_clear=True)",
plugin_id
)
try:
self.display_manager.clear()
plugin.display(force_clear=True)
cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] display(force_clear=True) SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
return cached_image
# Method 1: Try _create_scrolling_display (stocks pattern)
if hasattr(plugin, '_create_scrolling_display'):
logger.info(
"[%s] Triggering via _create_scrolling_display()",
plugin_id
)
try:
plugin._create_scrolling_display()
cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] display(force_clear=True) did not populate cached_image",
plugin_id
"[%s] _create_scrolling_display() SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
except (AttributeError, TypeError, ValueError, OSError):
logger.exception(
"[%s] display(force_clear=True) failed", plugin_id
return cached_image
except (AttributeError, TypeError, ValueError, OSError):
logger.exception(
"[%s] _create_scrolling_display() failed", plugin_id
)
# Method 2: Try display(force_clear=True) which typically builds scroll content
if hasattr(plugin, 'display'):
logger.info(
"[%s] Triggering via display(force_clear=True)",
plugin_id
)
try:
self.display_manager.clear()
plugin.display(force_clear=True)
cached_image = getattr(scroll_helper, 'cached_image', None)
if cached_image is not None and isinstance(cached_image, Image.Image):
logger.info(
"[%s] display(force_clear=True) SUCCESS: %dx%d",
plugin_id, cached_image.width, cached_image.height
)
return cached_image
logger.info(
"[%s] display(force_clear=True) did not populate cached_image",
plugin_id
)
except (AttributeError, TypeError, ValueError, OSError):
logger.exception(
"[%s] display(force_clear=True) failed", plugin_id
)
logger.info(
"[%s] Could not trigger scroll content generation",
@@ -409,7 +408,10 @@ class PluginAdapter:
original_image = self.display_manager.image.copy()
logger.info("[%s] Fallback: saved original display state", plugin_id)
# Ensure plugin has fresh data before capturing
# Lightweight in-memory data refresh before capturing.
# Full update() is intentionally skipped here — the background
# update tick in the Vegas coordinator handles periodic API
# refreshes so we don't block the content-fetch thread.
has_update_data = hasattr(plugin, 'update_data')
logger.info("[%s] Fallback: has update_data=%s", plugin_id, has_update_data)
if has_update_data:
@@ -419,24 +421,21 @@ class PluginAdapter:
except (AttributeError, RuntimeError, OSError):
logger.exception("[%s] Fallback: update_data() failed", plugin_id)
# Clear and call plugin display — use capture_mode to suppress hardware writes
# that plugins may trigger internally via update_display().
with self.display_manager.capture_mode():
self.display_manager.clear()
logger.info("[%s] Fallback: display cleared, calling display()", plugin_id)
# Clear and call plugin display
self.display_manager.clear()
logger.info("[%s] Fallback: display cleared, calling display()", plugin_id)
# First try without force_clear (some plugins behave better this way)
try:
plugin.display()
logger.info("[%s] Fallback: display() called successfully", plugin_id)
except TypeError:
# Plugin may require force_clear argument
logger.info("[%s] Fallback: display() failed, trying with force_clear=True", plugin_id)
plugin.display(force_clear=True)
# Capture the result
captured = self.display_manager.image.copy()
# First try without force_clear (some plugins behave better this way)
try:
plugin.display()
logger.info("[%s] Fallback: display() called successfully", plugin_id)
except TypeError:
# Plugin may require force_clear argument
logger.info("[%s] Fallback: display() failed, trying with force_clear=True", plugin_id)
plugin.display(force_clear=True)
# Capture the result
captured = self.display_manager.image.copy()
logger.info(
"[%s] Fallback: captured frame %dx%d, mode=%s",
plugin_id, captured.width, captured.height, captured.mode
@@ -455,10 +454,9 @@ class PluginAdapter:
plugin_id
)
# Try once more with force_clear=True
with self.display_manager.capture_mode():
self.display_manager.clear()
plugin.display(force_clear=True)
captured = self.display_manager.image.copy()
self.display_manager.clear()
plugin.display(force_clear=True)
captured = self.display_manager.image.copy()
is_blank, bright_ratio = self._is_blank_image(captured, return_ratio=True)
logger.info(
@@ -587,6 +585,28 @@ class PluginAdapter:
else:
self._content_cache.clear()
def invalidate_plugin_scroll_cache(self, plugin: 'BasePlugin', plugin_id: str) -> None:
"""
Clear a plugin's scroll_helper cache so Vegas re-fetches fresh visuals.
Uses scroll_helper.clear_cache() to reset all cached state (cached_image,
cached_array, total_scroll_width, scroll_position, etc.) — not just the
image. Without this, plugins that use scroll_helper (stocks, news,
odds-ticker, etc.) would keep serving stale scroll images even after
their data refreshes.
Args:
plugin: Plugin instance
plugin_id: Plugin identifier
"""
scroll_helper = getattr(plugin, 'scroll_helper', None)
if scroll_helper is None:
return
if getattr(scroll_helper, 'cached_image', None) is not None:
scroll_helper.clear_cache()
logger.debug("[%s] Cleared scroll_helper cache", plugin_id)
def get_content_type(self, plugin: 'BasePlugin', plugin_id: str) -> str:
"""
Get the type of content a plugin provides.

View File

@@ -11,10 +11,11 @@ import threading
from collections import deque
from typing import Optional, List, Any, Dict, Deque, TYPE_CHECKING
from PIL import Image
import numpy as np
from src.common.scroll_helper import ScrollHelper
from src.vegas_mode.config import VegasModeConfig
from src.vegas_mode.stream_manager import StreamManager
from src.vegas_mode.stream_manager import StreamManager, ContentSegment
if TYPE_CHECKING:
pass
@@ -51,10 +52,6 @@ class RenderPipeline:
self.config = config
self.display_manager = display_manager
self.stream_manager = stream_manager
self.sync_manager = None # Optional DisplaySyncManager — set by coordinator
self.sync_follower_left = True # True = follower is LEFT of leader (default)
self._sync_send_interval = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
self._last_sync_send = 0.0
# Display dimensions (handle both property and method access patterns)
self.display_width = (
@@ -205,26 +202,8 @@ class RenderPipeline:
# Update scroll position
self.scroll_helper.update_scroll_position()
# Determine if the cycle is done.
#
# scroll_helper considers a cycle complete only after
# total_distance_scrolled >= total_scroll_width + display_width.
# That extra display_width of travel causes a "wrap-around" phase
# where scroll_position resets to ~0 and the first plugin's content
# re-enters from the right — the user sees this 2-3 s of re-entry
# as "a plugin partially displaying before the next one starts."
#
# We end the cycle as soon as total_distance_scrolled reaches
# total_scroll_width (the wrap-around point), before any second-pass
# content becomes visible. The scroll_helper's own is_scroll_complete()
# check is kept as a fallback for any edge-cases where that threshold
# is never hit.
at_wrap_point = (
not self._cycle_complete and
self.scroll_helper.total_distance_scrolled >= self.scroll_helper.total_scroll_width
)
if at_wrap_point or self.scroll_helper.is_scroll_complete():
# Check if cycle is complete
if self.scroll_helper.is_scroll_complete():
if not self._cycle_complete:
self._cycle_complete = True
self.stats['scroll_cycles'] += 1
@@ -232,17 +211,6 @@ class RenderPipeline:
"Scroll cycle complete after %.1fs",
time.time() - self._cycle_start_time
)
# Push blank immediately so the hardware never shows any
# post-wrap content while the coordinator recomposes the
# next cycle (~100 ms).
try:
from PIL import Image as _Image
blank = _Image.new('RGB', (self.display_width, self.display_height))
self.display_manager.image = blank
self.display_manager.update_display()
except Exception:
logger.exception("Failed to write blank frame to display at cycle end")
return True # Cycle done; coordinator starts new cycle next frame
# Get visible portion
visible_frame = self.scroll_helper.get_visible_portion()
@@ -253,15 +221,6 @@ class RenderPipeline:
self.display_manager.image = visible_frame
self.display_manager.update_display()
# Multi-display sync: send scroll position to follower.
# The follower renders from its own cached_array (kept identical to the
# leader's via TCP image transfer at each new_cycle) at scroll_x ± display_width.
if self.sync_manager:
now = time.time()
if now - self._last_sync_send >= self._sync_send_interval:
self._last_sync_send = now
self.sync_manager.send_scroll_x(self.scroll_helper.scroll_position)
# Update scrolling state
self.display_manager.set_scrolling_state(True)
@@ -301,38 +260,33 @@ class RenderPipeline:
if self._cycle_complete:
return True
# When multi-display sync is active, defer mid-cycle hot swaps until the
# cycle ends naturally. Hot swaps block the render loop for 15-30ms while
# the image is rebuilt, causing a freeze+jump that the follower perceives
# as a speed-up. Deferring to cycle boundaries keeps transitions clean.
# Staging buffer content is still pre-loaded; it just applies at cycle end.
if self.sync_manager is not None:
return False
# Check if we need more content in the buffer
buffer_status = self.stream_manager.get_buffer_status()
if buffer_status['staging_count'] > 0:
return True
# Trigger recompose when pending updates affect visible segments
if self.stream_manager.has_pending_updates_for_visible_segments():
return True
return False
def hot_swap_content(self) -> bool:
"""
Hot-swap to new composed content.
Called when staging buffer has updated content.
Swaps atomically to prevent visual glitches.
Called when staging buffer has updated content or pending updates exist.
Preserves scroll position for mid-cycle updates to prevent visual jumps.
Returns:
True if swap occurred
"""
try:
# Snapshot position before swap so we can reposition after.
# The new image has completely different content — if scroll_position
# is left unchanged it lands at an arbitrary mid-content point in the
# new image, causing a visible jump on both displays.
old_width = self.scroll_helper.total_scroll_width
old_pos = self.scroll_helper.scroll_position
# Save scroll position for mid-cycle updates
saved_position = self.scroll_helper.scroll_position
saved_total_distance = self.scroll_helper.total_distance_scrolled
saved_total_width = max(1, self.scroll_helper.total_scroll_width)
was_mid_cycle = not self._cycle_complete
# Process any pending updates
self.stream_manager.process_updates()
@@ -340,24 +294,20 @@ class RenderPipeline:
# Recompose with updated content
if self.compose_scroll_content():
# Map scroll position proportionally into the new image width so
# we resume at the same relative progress through the content.
# This keeps the visual tempo consistent and avoids the jump that
# occurred when old scroll_position landed arbitrarily in new image.
new_width = self.scroll_helper.total_scroll_width
if old_width > 0 and new_width > 0:
ratio = (old_pos % old_width) / old_width
self.scroll_helper.scroll_position = ratio * new_width
else:
self.scroll_helper.scroll_position = 0.0
self.stats['hot_swaps'] += 1
logger.debug(
"Hot-swap completed: scroll repositioned %.0f%.0f (%.1f%% of new %dpx image)",
old_pos, self.scroll_helper.scroll_position,
(self.scroll_helper.scroll_position / new_width * 100) if new_width else 0,
new_width,
)
# Restore scroll position for mid-cycle updates so the
# scroll continues from where it was instead of jumping to 0
if was_mid_cycle:
new_total_width = max(1, self.scroll_helper.total_scroll_width)
progress_ratio = min(saved_total_distance / saved_total_width, 0.999)
self.scroll_helper.total_distance_scrolled = progress_ratio * new_total_width
self.scroll_helper.scroll_position = min(
saved_position,
float(new_total_width - 1)
)
self.scroll_helper.scroll_complete = False
self._cycle_complete = False
logger.debug("Hot-swap completed (mid_cycle_restore=%s)", was_mid_cycle)
return True
return False
@@ -392,29 +342,7 @@ class RenderPipeline:
return False
# Compose new scroll content
result = self.compose_scroll_content()
if result and self.sync_manager:
# When sync is active, start the leader at display_width instead of 0.
# This skips the initial black gap so the leader immediately shows content.
# The follower starts at position 0 (the gap) which looks like a clean
# blank transition rather than near-end content wrapping around.
self.scroll_helper.scroll_position = float(self.display_width)
if result and self.sync_manager:
# Signal follower that a new cycle started (triggers its own rebuild)
self.sync_manager.send_new_cycle()
# Push the actual scroll image over TCP so follower has identical pixels.
# Done in a background thread to not block the render loop (~15ms transfer).
if self.scroll_helper.cached_image is not None:
import threading as _t
_t.Thread(
target=self.sync_manager.send_scroll_image,
args=(self.scroll_helper.cached_image,),
daemon=True, name="sync-image-push"
).start()
return result
return self.compose_scroll_content()
def get_current_scroll_info(self) -> Dict[str, Any]:
"""Get current scroll state information."""

View File

@@ -14,7 +14,7 @@ Supports three display modes:
import logging
import threading
import time
from typing import Optional, List, Dict, Any, Deque, TYPE_CHECKING
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
from collections import deque
from dataclasses import dataclass, field
from PIL import Image
@@ -24,6 +24,7 @@ from src.vegas_mode.plugin_adapter import PluginAdapter
from src.plugin_system.base_plugin import VegasDisplayMode
if TYPE_CHECKING:
from src.plugin_system.base_plugin import BasePlugin
from src.plugin_system.plugin_manager import PluginManager
logger = logging.getLogger(__name__)

View File

@@ -5,11 +5,11 @@ Provides consistent API response formatting across all endpoints.
"""
import time
from typing import Any, Optional, Dict, Tuple
from typing import Any, Optional, Dict, Tuple, Union
from flask import jsonify, request
from src.web_interface.error_handler import create_error_response, create_success_response
from src.web_interface.errors import ErrorCode
from src.web_interface.errors import ErrorCode, ErrorCategory
def success_response(

View File

@@ -5,6 +5,7 @@ Provides decorators and helpers for consistent error handling across API endpoin
"""
import functools
import traceback
from typing import Callable, Any, Optional
from flask import jsonify

Some files were not shown because too many files have changed in this diff Show More