Compare commits

..
Author SHA1 Message Date
68d03ea260 feat: universal element style system — resolver, schema expansion, live preview (#394)
* feat(element-style): universal per-element style resolver

One shared implementation of the three things every customizable plugin
re-invented: a font loader, the customization.layout x/y-offset reader,
and the "did the user actually override this font?" check.

The override check is the load-bearing piece: the web UI's save flow
writes full schema defaults into config.json on every save, and the
plugin manager merges defaults again before instantiation, so key
presence never means user intent. The resolver compares against the
plugin's own schema defaults (via schema_manager), degrading to
caller-supplied classic defaults when unavailable. This retires the
hand-maintained _CLASSIC_FONT_DEFAULTS dicts that shipped broken twice.

The loader is a superset of the four per-plugin variants: alias
resolution (baseball), truetype for TTF/OTF/BDF (FreeType loads BDF at
native size), .pil sidecar fallback for BDF (football), fallback font,
PIL default — never raises. Also introduces the text_color convention
([r,g,b], absent = keep the plugin's hardcoded color).

BasePlugin gains a lazy style_resolver property (invalidated on config
change) and element_style() sugar; standalone helpers receive the
resolver from their owning plugin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* feat(element-style): x-style-elements schema expansion + color provenance

Plugins can now declare styleable display elements once, compactly, on
their customization schema ("x-style-elements") instead of hand-copying
the ~50-line font/font_size/text_color/offset property blocks (currently
duplicated 52x across the plugin monorepo). SchemaManager.load_schema
expands declarations before caching, so the config form, save path,
validation, and defaults generation all see the same shape; the single
expansion implementation lives in src.element_style and is also applied
by defaults_from_schema_file, keeping the web UI's view and a plugin's
raw-schema-file view of the defaults provably identical (parity test).

Generated blocks use only widgets the config form already renders
(font-selector, color-picker, number inputs) and update x-propertyOrder
when present (the template only renders listed keys). Expansion is
idempotent, never mutates its input or the cached/on-disk schema, and a
hand-written block for the same element always wins.

Color gets the same provenance rule as fonts: the web form always posts
the RGB inputs, so a saved config carries the schema-default color
whether or not the user touched it — the resolver now only honors a
color that DIFFERS from the schema default, and keeps the plugin's
classic (possibly state-dependent, e.g. gold-on-touchdown) color
otherwise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* feat(web): live plugin preview on the config page

Adds POST /api/v3/plugins/preview: renders a plugin headlessly with the
CANDIDATE (unsaved) config and returns a PNG — so users can see exactly
what the panel will show before saving, at their real panel size (from
display.hardware) or a chosen test size.

Two extractions make it drift-proof rather than parallel-implemented:
- dev_server's _render_once moves to
  src/plugin_system/testing/render_service.py (pure PIL via
  VisualTestDisplayManager, install_deps=False always — safe in the web
  process, which never touches display hardware); the dev server now
  wraps it.
- save_plugin_config's ~350-line form->config conversion is extracted
  verbatim as parse_plugin_config_form and shared by the preview
  endpoint, so preview and save can never interpret the form
  differently.

update() is skipped by default (no network on the request thread);
plugins with a test/harness.json get their mock-data fixture primed
instead, and ?skip_update=0 opts into a live update. The config page
gains a Live Preview panel (HTMX hx-include of the existing form, size
selector, pixelated img fragment) — works for every plugin with zero
per-plugin code, including the x-style-elements font/size/color fields.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(web): expand x-style-elements in the config-page form render too

pages_v3's plugin-config partial loads config_schema.json directly from
disk rather than through SchemaManager.load_schema, so declared style
elements expanded everywhere EXCEPT the form the user actually sees.
Found live on the devpi: the API served the expanded schema and the
save path validated against it, but the config page rendered no
font/color/offset fields. Apply the same (idempotent) expansion here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(web): preview size selector — htmx caches hx-post, use hx-vals

Found live on the devpi: selecting a preview size did nothing because
htmx snapshots the request path when it processes the button, so the
size dropdown's onchange mutation of hx-post never took effect. The size
now travels as a __preview_size=WxH form field attached via hx-vals
(evaluated at request time); the endpoint honors it (query args still
win for API callers) and strips it before form parsing so it can't leak
into the candidate config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(web): allowlist plugin_id in the preview endpoint's dir lookup

Same defense as the dev server's find_plugin_dir (and pages_v3's
existing pattern): plugin_id arrives in request input and is used to
build filesystem paths — reject anything outside ^[a-zA-Z0-9_-]{1,64}$
before touching the filesystem.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: harden preview + resolver edges found in self-review

- extract_schema_defaults now matches SchemaManager's array handling
  ([] for arrays without defaults, [item] for item-level defaults) —
  the documented parity guarantee was false for array-typed properties;
  parity test extended to cover them.
- render_plugin_once calls plugin cleanup() in a finally (image captured
  first — cleanup may clear the canvas): preview instances could leak
  sessions/threads per request in the long-running web process.
- Preview JSON path deep-merges the candidate onto the saved config,
  matching the form path — a shallow update() silently dropped saved
  sibling values in any nested section the candidate touched.
- Preview render is bounded (15s, 504 on timeout, daemonized worker):
  a hanging plugin display() no longer pins a web worker forever.
- ElementStyleResolver.is_for(config): public staleness check for
  callers that cache a resolver, instead of poking _config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(render-service): adopt the dev server's exception-detail policy

Port c6962701's convention into the shared render service (which
supersedes the inline _render_once it patched): full tracebacks to the
server log via exc_info, only the exception class name to the client.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 10:40:03 -04:00
ChuckandClaude Fable 5 a55504c0af fix(layout): bound the fit cache; never alias the source image in fits
Two latent issues found in a self-review pass:

- LayoutContext._fit_cache was an unbounded dict (the image cache got an
  LRU cap, the text-fit cache didn't). Cache keys embed the fitted TEXT,
  so a plugin fitting changing strings — a live game clock, a ticker —
  on a 24/7 service grows it forever. Now LRU-bounded at 512 entries via
  the same pattern as the image cache.

- fit_image returned the caller's ORIGINAL image object when the source
  was already RGBA at target size (contain/fill_height, no ink crop).
  ImageFitResult is documented as an independent copy, and LayoutContext
  caches results — an aliased image lets later mutations of the source
  corrupt cached fits (or vice versa). Copy in that branch.

Both covered by new regression tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam
2026-07-12 10:19:47 -04:00
ChuckBuilds bd17fad578 fix: remove unused imports flagged by Codacy
Union in adaptive_images.py and field in adaptive_layout.py are both
imported but never used -- the last two Codacy findings on this PR,
matching the same fix already applied on PR #396.
2026-07-12 10:07:18 -04:00
ChuckBuilds 7d5044c315 fix(dev-server): remove conditional-reassignment ambiguity in plugin_dir resolution
CodeQL's path-injection flow still traced through _parse_render_request
after the scandir fix -- the tainted find_plugin_dir() result and the
scandir-derived _trusted_plugin_dir() result shared the same variable
name (plugin_dir), reassigned only on the truthy branch. That merge
point apparently isn't treated as a barrier by the flow analysis, so it
kept tracing the pre-reassignment value through to the manifest open().

Split into two distinct names -- candidate_dir (tainted, used only to
call _trusted_plugin_dir) and trusted_dir (the only name used for any
downstream file access) -- so there's no reassigned variable for the
flow to walk through.
2026-07-12 09:53:09 -04:00
ChuckBuilds c696270182 fix(dev-server): use os.scandir for path-injection barrier, redact stack traces from render responses
CodeQL doesn't model Path.iterdir() as a taint-clearing enumeration the
way it does os.scandir() -- _trusted_plugin_dir's iterdir-based rebuild
still traced plugin_id through to the manifest.json open(). Switched to
scandir, matching the pattern already verified clean on PR #396.

Also stops surfacing raw exception text (update()/display() failures)
in the JSON render response -- logs full detail server-side via
exc_info instead, returning only the exception class name to the
client. And drops path values from three plugin_loader debug/error
logs that CodeQL flags as clear-text-logging of externally-influenced
data, keeping plugin_id (not flagged) for context.
2026-07-12 09:43:36 -04:00
ChuckandClaude Fable 5 a11c59698a fix(dev-server): derive plugin dir from trusted directory listings
CodeQL's barrier-guard recognition doesn't see a startswith check
inside an any() comprehension, so the normalize-and-prefix approach
still flagged. Break the taint outright instead: after lookup, re-derive
the directory by enumerating the search dirs (iterdir) and matching by
path equality — the Path used for all downstream file access is built
solely from trusted listings, never from request input.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam
2026-07-11 12:45:17 -04:00
ChuckandClaude Fable 5 149f1daa1e fix(dev-server): inline normpath containment barrier before render
CodeQL doesn't credit the sanitization inside find_plugin_dir along
this flow; apply its documented barrier (normpath + startswith against
the allowed roots) inline in _parse_render_request, on the exact path
that reaches the render/load sinks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam
2026-07-11 12:41:35 -04:00
ChuckandClaude Fable 5 68d5540985 fix(dev-server): lexical containment check on resolved plugin dirs
CodeQL doesn't recognize the interprocedural allowlist as a
path-injection barrier; add the canonical one — normalize (without
following symlinks, since dev plugins are commonly symlinked into
plugins/) and require the result to stay inside the search dir.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam
2026-07-11 12:38:46 -04:00
ChuckandClaude Fable 5 72b443d541 fix(dev-server): allowlist plugin_id before any path lookup
CodeQL (py/path-injection): plugin_id arrives in request input and flows
into filesystem paths via find_plugin_dir. Gate it with the same
^[a-zA-Z0-9_-]{1,64}$ allowlist the web UI's pages_v3 uses, at the
single choke point every route resolves through.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam
2026-07-11 12:30:17 -04:00
ChuckandClaude Fable 5 c5f5e25150 fix: address CodeRabbit review on PR #393
- docs: scope the self.layout note to BasePlugin subclasses (others build
  a LayoutContext directly) and make explicit that adaptive layout is
  opt-in — classic rendering stays unless a plugin adopts the APIs.
- dev_server: broaden the render-request catch (a bad manifest.json now
  returns a clean 400 instead of an unhandled 500) and stop echoing raw
  exception text in the loader-failure responses — full tracebacks go to
  the dev server's console log instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam
2026-07-11 10:40:11 -04:00
ChuckandClaude Fable 5 278d757de0 docs: document scoreboard_regions' center-reserve and score-bleed params
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 6b47599c4a fix(layout): scoreboard_regions reserves real center space at 2:1 aspect ratios
logo_slot = min(height, width // 2) has a blind spot: at exactly 2:1
aspect ratio (width == 2 * height -- a very common shape: two, four, or
more square modules stacked into a taller panel) width // 2 and height
are equal, so the two logo slots claim the ENTIRE width and leave zero
pixels for a center column, no matter how large the panel gets. Not a
'small panel' problem -- 96x48, 128x64, and 256x128 (all exactly 2:1) hit
it identically, while the 128x32 design baseline and panels like 192x48
or 256x32 never do, because height is already the tighter constraint
there.

Two new parameters fix it in the one shared helper every scoreboard-style
plugin composes through:

- min_center_fraction / min_center_design_px reserve at least
  max(width * fraction, design_px * ctx.scale) for the center column,
  capping logo_slot further when needed. The scaled design-px term
  matters on small panels where a flat fraction alone reserves too little
  absolute space.
- score_bleed_fraction extends the score's own fit box (not the logo
  slots themselves) a controlled amount into each side -- the same way
  real broadcast scoreboards let a big score number's edges cross into
  the team marks flanking it. Without this the reserve alone can still be
  too narrow for a short score to render without truncating.

score_area is now genuinely narrower than the full card width (previously
identical to status_band/detail_band, which still span the full width and
overlay the logos -- short text there was never the problem).

Verified against the full harness size spread: a real game score like
'17-21' never needs ellipsis at any tested 2:1-or-tighter aspect ratio
(test_score_never_needs_ellipsis_for_a_short_score), and wide panels
(128x32/192x48/256x32-style) are provably unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 87ff97d006 feat(layout): fit_text_proportional gains an axis-specific scale override
self.scale (min(width_ratio, height_ratio)) is the right conservative
default for anything whose aspect ratio matters, but a caller whose
surrounding composition already scales along a single axis — e.g.
football-scoreboard's logo_slot = min(height, width // 2), which tracks
height alone — needs text sized the same way, or it reads as
under-scaled next to logos that grew on a panel that only got taller
(128x32 -> 128x64: self.scale stays 1.0 since width didn't grow, but
logos still double).

fit_text_proportional(..., scale=None) now accepts an explicit override;
None keeps the existing self.scale default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 4ab0c871c8 feat(layout): add fit_text_proportional — proportional sizing vs. always-maximize
fit_text always picks the largest ladder rung that fits its box. That's
right when an element owns dedicated space, but wrong when several
independently-fitted elements need to stay visually harmonious as the
panel grows: a score's box might have generous room while a neighboring
logo scales by a fixed geometry factor via px() — fit_text lets the score
balloon out of proportion (even overlapping the logo) even though its
individual pick is technically correct.

fit_text_proportional(text, box, base_size_px, ladder) instead targets
base_size_px * self.scale (the same scale factor px() already uses),
picking the nearest ladder rung at or below that target, still capped to
what fits the box, floored at the smallest rung when the target is below
every rung. Refactored the shared largest-that-fits/ellipsize walk into
_walk_ladder() so fit_text and fit_text_proportional don't duplicate it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 013a2663e5 feat(layout): add measure_font_crispness — verify a ladder rung isn't blurry
PIL antialiases TTF outlines by default; a 'pixel-style' font only
rasterizes without antialiasing at specific sizes (for PressStart2P:
exact multiples of its 8px design grid). A ladder rung at an unverified
size silently renders blurry on an LED panel — this exact bug shipped in
both text-display's and football-scoreboard's custom TTF ladders
(non-8-multiple PressStart2P sizes, and '5by7.regular'/'4x6-font' at
sizes that were never actually crisp).

measure_font_crispness(font, sample_text) renders the sample and reports
the fraction of ink-bbox pixels that are neither pure black nor pure
white. BDF fonts (real bitmaps) always score 0.0; TTF ladders should be
verified against this before shipping — see the new
TestFontFitting::test_ladder_arcade_is_crisp pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 af96c6ffd6 feat(plugins): adaptive-lib discoverability + advisory version compat warning
Discoverability: re-export the adaptive layout/image API from src.common
(the blessed-helpers package plugin authors already know) — canonical
paths stay src.adaptive_layout / src.adaptive_images so nothing breaks.
Document it in src/common/README.md and cross-link ADAPTIVE_LAYOUT.md
from the developer docs authors actually read (quick reference, API
reference, advanced dev, font manager, dev preview, plugin dev guide);
ADAPTIVE_LAYOUT.md gains adaptive-images, composite-layouts and
preserving-user-customization sections.

Compat: PluginLoader now logs one advisory warning (never raises) when a
plugin's manifest declares a min LEDMatrix version newer than the running
core, checking the min_ledmatrix_version / requires.* / versions[]
spellings found in the wild. Guarded against stale core version numbers.

src/__init__.py __version__ bumped 1.0.0 -> 3.1.0 to match the latest
release tag (v3.1.0) — it had never been updated and the compat check
needs a truthful number. NOTE: verify this matches the intended release
numbering before the next tag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 923611b836 feat(harness): scale-up fill check, config variants, multi-size dev gallery
Quality gates for adaptive layout:

- fill_metrics()/check_scale_up() in the safety harness: overflow catches
  content too big for a panel, but nothing caught content that stays tiny
  on panels >= 2x the plugin's declared design size. The check measures
  lit-content extents and warns (or fails, when a plugin opts into
  "fill_check": "strict" in test/harness.json) below 50% coverage on the
  doubled axis. Warn-only by default so no existing plugin breaks.

- harness.json "variants": extra runs with config overlays and their own
  golden dirs, so an opt-in mode (e.g. layout_mode: adaptive) is golden-
  tested beside the classic default. check_plugin.py loops base + variants
  and labels variant results mode@name.

- Dev preview server: GET /api/sizes (harness size sample), POST
  /api/render-matrix (render at up to 12 sizes in one call), size-preset
  dropdown, and an "All Sizes" side-by-side gallery in the preview UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 dabe7f05bc feat(layout): adaptive image fitting + composite region helpers
Add src/adaptive_images.py — the image counterpart to fit_text:
- fit_image(img, box, mode=contain|cover|fill_height|stretch,
  crop_to_ink, anchor, resample, upscale) promoting the proven plugin
  patterns (football's crop-to-ink fill-height logos, masters' cover
  crop + NEAREST flags, static-image's letterbox). Upscales by default —
  thumbnail()'s downscale-only behavior is why imagery stays tiny on
  big panels.
- draw_fitted_image() pastes aligned within a Region with alpha mask.
- One central Pillow>=9.1 RESAMPLE shim replacing ~15 plugin copies.

LayoutContext.fit_image() caches results per (identity, box size,
options) with a 64-entry LRU; id()-keyed entries pin the source image.
BasePlugin.draw_image() is the one-liner adoption path beside draw_fit.

Composites in adaptive_layout.py: Region.offset() (user x/y-offset
passthrough), scoreboard_regions() (the two-logos-plus-score card math
duplicated across six sports plugins, logo_slot = min(H, W//2)), and
media_row() (art-left/text-right).

Fix LogoHelper's size-blind cache key (stale sizes on panel change);
deprecation note on dead image_utils.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
ChuckandClaude Fable 5 21c0cfa22d feat(layout): adaptive layout & font scaling system for plugins
Add src/adaptive_layout.py — opt-in core helpers so plugins render
legibly on any panel size without hand-tuned per-display layouts:

- Region: integer rect algebra (bands/columns/weighted splits/centering)
  that partitions space so text bands can't overlap by construction
- Font ladders: ordered (family, size) steps known to render crisply
  (LADDER_GRID: X11 BDFs at native sizes; LADDER_ARCADE: PressStart2P at
  8px multiples) — fitting walks the ladder instead of scaling pixel
  fonts fractionally
- LayoutContext: breakpoint tiers, geometry scale vs. a declared design
  size, and cached fit_text/fit_lines/font_for_rows queries

Generalizes the three patterns proven in the field: f1-scoreboard's
scale factor, masters-tournament's tiers, baseball-scoreboard's font
fallback ladder.

Wiring: BasePlugin gains a lazy .layout property and draw_fit();
FontManager gains get_native_bdf_size() and a cache_generation counter;
manifest schema gains display.design_size and requires.display_size
max_width/max_height; 96x48 joins DEFAULT_TEST_SIZES; the bounds-check
harness records negative-coordinate draws; TextHelper's broken
measurement helpers are fixed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 09:00:37 -04:00
05e7c43b27 fix(plugins): replace dependency marker files with a real satisfaction check (#390)
* fix(plugins): replace dependency marker files with a real satisfaction check

The .dependencies_installed hash-marker system only tracked "was this exact
requirements.txt hashed before" — not whether the packages it names are
actually present. That made it fragile (a wiped venv, a manually removed
package, or a lost/corrupted marker forces a needless full pip reinstall or,
worse, a false skip) and produced dead weight for the ~10 plugins whose
requirements.txt is comment-only (they still paid a pip subprocess on first
boot before a marker existed).

Replace it with requirements_are_satisfied() in plugin_loader.py, which
checks each real requirement line against importlib.metadata directly, so
install_dependencies() only shells out to pip when something is actually
missing or version-mismatched. Drops the marker file entirely: removed all
marker read/write sites in plugin_loader.py and store_manager.py, the
now-pointless marker-cleanup step in the git-update path, the unused legacy
marker implementation in plugin_manager.py, and the already-stale
clear_dependency_markers.sh script.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(security): close path-injection gap in dependency-satisfaction checks

CodeQL flagged 2 new high-severity "uncontrolled data used in path
expression" alerts at the open() calls inside this PR's new
requirements_has_real_deps()/requirements_are_satisfied() -- both are
reachable from paths that were never run through the basename+trusted-base
sanitiser this codebase already uses elsewhere:

- PluginLoader.install_dependencies() only applied that sanitiser when its
  optional plugins_dir argument was actually passed; the "no plugins_dir"
  branch trusted plugin_dir_real directly. Made plugins_dir required (not
  Optional) so that branch can't exist, and added an explicit guard in
  load_plugin() so install_deps=True without a plugins_dir fails loudly
  instead of silently. Production's only real caller (PluginManager) always
  passes plugins_dir already; the harness/dev-server/render-plugin callers
  all use install_deps=False and are unaffected.

- StoreManager._install_dependencies() never sanitised plugin_path at all,
  and its call sites ultimately derive that path from a plugin's own
  manifest.json "id" field (install_plugin_from_url) -- a malicious plugin
  could otherwise point requirements_file outside plugins_dir. Applied the
  same os.path.basename()-based containment pattern PluginLoader already
  uses (and that CodeQL recognises as a real sanitiser).

Added test_install_dependencies_requires_plugins_dir and
test_install_dependencies_rejects_path_outside_plugins_dir to lock in the
actual security property, not just quiet the scanner. Verified: all 20
tests in test_plugin_loader.py pass, plus the PR's existing test plan
(test_plugin_system.py, test_store_manager_caches.py: 53 passed) and the
full CI plugin-safety suite (test_harness.py, test_visual_rendering.py,
test_plugin_matrix.py: 52 passed, 2 pre-existing skips) all still pass.

* fix(security): replace basename-only sanitiser with a trusted-enumeration check

The previous commit's os.path.basename() + os.path.join() pattern (which a
pre-existing code comment claimed CodeQL recognises as a sanitiser) did not
actually clear the alert -- the next CodeQL run still flagged the same 2
sink lines, plus a new one at the os.path.join() call itself. Taking a
substring of tainted data apparently isn't treated as a barrier by this
query, whatever the comment assumed.

Replaced it with find_trusted_subdir(): enumerate the trusted plugins_dir
via os.scandir() and only use a name that scandir itself produced, matched
by equality against the caller's requested name. The path is then built
from that enumerated entry, not from the caller's string -- a value
sourced from iterating a trusted, non-tainted directory carries no taint
regardless of what it happens to equal, which is a stronger and more
conventional allowlist-style barrier than string-stripping. Applied
identically in both PluginLoader.install_dependencies() and
StoreManager._install_dependencies(), sharing one implementation.

Re-verified: all 65 tests across test_plugin_loader.py (20, including the
2 new security regression tests), test_store_manager_caches.py (35),
test_plugin_system.py (10) pass, plus the full CI plugin-safety suite
(test_harness.py/test_visual_rendering.py/test_plugin_matrix.py: 52
passed, 2 pre-existing skips).

* fix(security): redact URL credentials from pip subprocess output before logging

CodeQL flagged 3 clear-text-logging-of-secrets alerts in
install_requirements_file() (src/common/permission_utils.py:353,360,371).
Pre-existing on main, unrelated to this PR's own diff, but now visible
since the path-injection alerts that previously took priority in the
annotation list are fixed.

The underlying risk is real: pip can echo a private index URL's embedded
basic-auth credentials (from a requirements.txt --index-url line or
PIP_INDEX_URL) back verbatim in its own stderr/stdout on failure, and this
function both logs that output directly and returns it to callers --
store_manager.py's _install_dependencies() logs result.stderr from this
same function too.

Added _redact_url_credentials(), applied immediately after each of the two
subprocess.run() calls (mutating result.stderr/stdout in place) rather
than patching each log call site individually. This closes the leak at
the source: every downstream use -- the three flagged log lines, the
"note" string embedded in the returned stdout, and store_manager.py's own
logging of the returned result -- gets the redacted text for free.

Verified the fixed-phrase "denied" check (`"a password is required" in
result.stderr`) is unaffected, since URL syntax and those phrases don't
overlap -- covered explicitly by
test_does_not_touch_denied_check_phrases. Added
test/test_permission_utils.py (6 tests) covering the redaction helper
directly and both subprocess.run() call sites (the sudo-wrapper branch,
which this repo's scripts/fix_perms/safe_pip_install.sh makes live, and
the no-wrapper fallback branch). All pass.

* fix(security): stop interpolating req_file/pip-output into log calls

The previous commit's redaction (mutating result.stderr/stdout right after
each subprocess.run()) didn't clear CodeQL's clear-text-logging alerts --
same lesson as the path-injection fix earlier in this PR: a static
analyzer can't tell "this value was already sanitised two lines up" from
"this is still the raw tainted value" just by looking at a single log
call in isolation, so it conservatively keeps flagging it regardless of
what the redaction function actually does.

Removed all dynamic interpolation (req_file, result.stderr) from the 3
flagged logger.warning() calls entirely, replacing them with fixed
messages plus (for the one that had it) result.returncode, which is a
plain int with no possible taint. The full redacted detail is still
available where it actually matters -- in the returned
CompletedProcess.stderr/stdout and the "note" text -- just not duplicated
into a log line a scanner has to reason about in isolation.

Re-verified: all 6 test_permission_utils.py tests still pass (they assert
on the returned result, not log call arguments), plus the full
test_plugin_loader.py/test_store_manager_caches.py/test_plugin_system.py
suite (71 passed, 1 pre-existing deselect, 4 subtests).

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 08:55:50 -04:00
ChuckandGitHub 2ffc57cf40 fix(plugin-harness): add no-op process_deferred_updates to test double (#391)
The safety harness's VisualTestDisplayManager (base of
BoundsCheckingDisplayManager) doesn't implement process_deferred_updates(),
which 5 first-party ledmatrix-plugins call unconditionally between
set_scrolling_state() and their scroll-position update: news, odds-ticker,
ledmatrix-leaderboard, stock-news, and ledmatrix-stocks. Any of them fails
the harness with AttributeError the moment it's touched (surfaced when
ledmatrix-plugins#177 had to add a local hasattr guard in ledmatrix-stocks
just to pass CI). Add the method as a no-op, mirroring the existing
"no-op for testing" pattern already used for set_scrolling_state, so these
plugins render under the harness without every touching PR needing its own
guard.
2026-07-11 08:55:03 -04:00
ChuckandGitHub aab0e9ade0 fix(plugin-manager): fix TypeError breaking every plugin's scheduled update (#392)
run_scheduled_updates()'s resource-monitor branch wrapped the update call
in a closure stored as a *class* attribute on a dynamically-built type
(type('obj', (object,), {'update': monitored_update})()). The descriptor
protocol turns a function found via class-attribute lookup into a bound
method on instance access, silently prepending the synthetic instance as
an implicit first argument -- but monitored_update() takes none, so every
call raised "monitored_update() takes 0 positional arguments but 1 was
given", was caught by run_scheduled_updates' try/except, and recorded as
an update failure.

self.resource_monitor is None by default and was dormant until PR #388
("activate dormant plugin health/metrics subsystem") wired it up in both
display_controller.py and web_interface/app.py -- meaning this bug went
live in every real deployment as of that merge (2026-07-09) despite the
buggy line itself dating back to 2025-12-27. In practice this means no
plugin's update() has succeeded since upgrading past #388: circuit
breakers cycle through half-open -> immediate failure -> reopened every
health-check interval forever, and all plugin data (scores, odds, prices,
etc.) goes stale from whatever was last fetched before the upgrade.
Confirmed live on a running instance: odds-ticker (and stock-news,
ledmatrix-stocks, baseball-scoreboard, ledmatrix-leaderboard, of-the-day)
failing this exact way every 5-minute circuit-breaker retry.

Fixed by using types.SimpleNamespace(update=monitored_update) instead of
a dynamic class: SimpleNamespace stores attributes on the instance
itself, so attribute lookup returns the plain function unchanged --
never routed through the class-attribute descriptor protocol that
injects an implicit self.

Added test_run_scheduled_updates_calls_update_with_resource_monitor to
test/test_plugin_system.py using a real PluginResourceMonitor (not a
mock of it), so the test exercises the actual descriptor-binding
behavior that caused this. Verified the test fails with the exact
reported error against the pre-fix code and passes against the fix.
2026-07-11 08:54:30 -04:00
978a03b42d Add settings tooltips and search to the web UI (#387)
* Add settings tooltips and search to the web UI

Help users quickly find settings and understand how each one works.

Tooltips: a new delegated controller (static/v3/js/tooltips.js) drives an
accessible (i) info tooltip that appears on hover, keyboard focus, and tap.
A shared `help_tip` Jinja macro (partials/_macros.html) emits the trigger;
the plugin config macro and the core settings partials now surface help
text through it. Per the design, the always-visible field help paragraphs
are folded into the tooltip to declutter the forms, and the hardware/display
settings carry authored detail (default, range, recommendation).

Search: a global header search box finds settings across every settings tab
— even ones not yet opened — via a lazy client-side index built by scanning
the same field markup (static/v3/js/settings-search.js). Selecting a result
switches tabs, waits for the field to load, then scrolls to and flashes it.
A per-tab filter box hides non-matching fields on the current tab.

Plugin settings get tooltips for free by reusing each field's schema
`description`; every settings field also gets a stable `setting-<tab>-<key>`
anchor id for search navigation.

Styling uses the existing --color-* theme vars so light/dark mode both work,
and honors prefers-reduced-motion. Adds Flask render smoke tests that assert
each settings partial ships tooltips, anchors, and a filter box.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Address Codacy static-analysis findings in settings JS

Refactor the two new modules to clear the flagged patterns without any
behavior change:

- Build the search dropdown with DOM nodes + textContent instead of
  innerHTML string concatenation, removing the XSS sinks and the manual
  escapeHtml helper it needed.
- Replace numeric index access (index[i], terms[j], opts[idx],
  currentResults[i]) with array iteration methods, NodeList.item(), and
  Array.prototype.at() to clear detect-object-injection.
- Use === via a shared termsMatch() helper, optional-catch binding, and
  drop a useless initial assignment.

Verified with ESLint (eslint:recommended + eslint-plugin-security) at zero
findings and re-ran the headless-Chromium behavior test (tooltip, per-tab
filter, global search navigation) — all green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Reduce complexity of revealAncestors in settings search

Extract isNodeHidden() and revealNode() helpers so revealAncestors drops
below the cyclomatic-complexity threshold. No behavior change; verified with
the headless-Chromium test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Resolve remaining Codacy findings in settings search

- Validate plugin ids against a strict allowlist (mirroring the server's
  _SAFE_PLUGIN_ID_RE) before they can appear in a fetch path, so the request
  URL is never built from unvalidated input (Codacy: user-controlled URL).
- Document that the fetched HTML is parsed into an inert document (scripts
  never run, never inserted into the live DOM) purely to read field text for
  the search index.
- Declare block-scoped locals with const instead of var where they were
  nested inside conditionals (Codacy: var not at function root).

No behavior change; re-verified with ESLint and the headless-Chromium test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Serve the settings search index from the server as JSON

Move index building off the client so there is no client-side HTML fetching
or DOM parsing (resolves Codacy's variable-fetch and DOMParser flags on the
read-only, same-origin index build).

- Add GET /v3/settings/search-index (pages_v3.py): renders the settings
  partials server-side and extracts each field's anchor id, key, label,
  tooltip, and section with a small stdlib HTMLParser, then caches the result
  keyed on the installed-plugin set. Parsing the rendered HTML keeps anchor
  ids identical to the live DOM, so the index cannot drift.
- settings-search.js: buildIndex() now does a single fetch of the literal
  endpoint + .json(); removed the per-partial fetch loop, DOMParser, scanDoc,
  CORE_TABS, and the plugin-id allowlist. Search, keyboard nav, navigation,
  and the per-tab filter are unchanged.

Net: fewer requests and no client-side HTML parsing. Verified with a new
endpoint test in test_web_settings_ui.py (12 pass) and the headless-Chromium
test (tooltip, filter, search navigate + flash all green).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Address CodeRabbit review on settings search/tooltips

- pages_v3: include Durations tab in the search index so
  setting-durations-* fields are actually indexed
- test_web_settings_ui: assert setting-durations-clock is present in
  the search-index endpoint response
- settings-search.js: on index fetch failure, reset buildPromise
  instead of caching an empty (truthy) index so search can retry
- settings-search.js: filterScope returns null (not document) when no
  tab container matches, and the caller guards, so the per-tab filter
  can't hide fields across unrelated tabs
- settings-search.js: refresh the stale header comment to describe the
  server-side JSON index flow
- app.css: cap #settings-search-results height with overflow-y so the
  dropdown scrolls instead of overflowing small screens

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Fix stuck search dropdown; add plugin-tab nested-settings filter

Global search:
- Close the results dropdown on input blur (guarded, with a short delay)
  so it reliably dismisses when focus leaves — previously it could linger
  because the only outside-close was a document click that Alpine/HTMX
  handlers can swallow.
- Clear the query text after navigating to a result so refocusing the box
  doesn't re-open stale results.
- Also dismiss on htmx:afterSwap (tab changes / navigation).

Per-tab filter (now on plugin tabs too):
- Render the shared settings_filter box in the plugin Configuration panel.
  It auto-wires: the delegated input handler and filterScope already target
  .plugin-config-tab.
- Teach applyTabFilter to reveal matches inside collapsed nested sections
  (render_nested_section defaults them shut), hide nested-section wrappers
  with no matches, and restore the original collapsed layout when cleared
  (only re-collapsing sections the filter itself opened).
- Count a visible nested-section as content for its parent heading so the
  heading isn't hidden while a subsection below still has matches.

Adds a plugin-config render test (filter box + nested anchors + tooltips).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Close search dropdown via capture-phase outside-click

The dropdown could stay open after clicking away because the only
outside-click close was a bubble-phase document listener. The v3 UI is
one Alpine app() component full of HTMX/Alpine/widget click handlers;
when a click lands inside an element that calls stopPropagation(), the
event never bubbles to document and the close never runs.

- Replace the bubble-phase document 'click' close with a capture-phase
  'pointerdown' listener scoped to #settings-search-wrap. Capture runs
  before any bubbling stopPropagation can swallow the event, so it always
  fires; pointerdown also covers touch on the Pi screen. Clicking a result
  stays inside the wrap, so selection is unaffected.
- Guard the debounced input handler so a delayed render can't re-open the
  box after focus has left (type-then-click-away race).

Keeps the existing blur / Escape / htmx:afterSwap closes as secondary paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Fix settings search dropdown not visually closing

.hidden has no effect in this app: app.css is a hand-picked utility
subset (no Tailwind build step) and never defines .hidden { display:
none }. openResults()/closeResults() only toggled the class, so the
dropdown stayed rendered (display: block) even once closeResults()
ran - confirmed via computed style in a headless browser. Set
style.display directly, matching the fallback already used by
revealNode()/collapseNode() elsewhere in this file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-09 09:22:24 -04:00
bd9f461f70 Add system diagnostics, power controls, and WiFi radio toggle to Tools tab (#389)
* Add system diagnostics, power controls, and WiFi radio toggle to Tools tab

Expands the web UI Tools tab with safe, purpose-built controls so users can
manage the Pi without SSHing in, instead of an arbitrary-command terminal
(the web UI has no auth and CSRF is disabled, so a shell would be unsafe).

- System Diagnostics card: renders the existing but previously-unused
  GET /api/v3/system/status endpoint (CPU, memory, temp, disk, uptime),
  with a manual refresh and a 10s poll.
- System Power section: reboot/shutdown buttons wired to the existing
  reboot_system / shutdown_system actions, behind a confirm step, with a
  dedicated powerAction() helper that treats the dropped connection as the
  expected "going offline" outcome rather than an error.
- Network Radio section: WiFi on/off toggle backed by new
  GET/POST /api/v3/wifi/radio endpoints and WiFiManager.set_wifi_radio() /
  get_wifi_radio_state(). Disabling WiFi is refused unless a wired
  connection is present (reusing the existing lockout guards), with an
  explicit force-off confirmation for advanced users.

No new privileged commands: uses nmcli radio wifi on|off (already
sudo-allowlisted) and the existing reboot/poweroff grants, so the
sudoers-alignment guard test stays green. Bluetooth toggle intentionally
omitted since the installer removes the BlueZ stack for LED timing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE

* Harden WiFi radio endpoint and diagnostics poll (review feedback)

Addresses code review feedback on the Tools tab additions:

- api_v3.py: parse the `enabled` POST field with the same string-aware
  coercion as `force`. A plain bool() cast turned {"enabled":"false"} into
  True (enabling instead of disabling) for any non-UI API caller.
- wifi_manager.set_wifi_radio() now returns a reason code alongside
  (success, message); the /wifi/radio error response includes it. The Tools
  UI only shows the force-off confirmation when reason == 'no_ethernet', so a
  genuine nmcli failure surfaces its real error instead of a misleading
  "no wired connection" prompt that would just retry into the same failure.
- tools.html: gate the 10s diagnostics poll on panel visibility
  (document.hidden / offsetParent), so switching to another tab stops the
  recurring /api/v3/system/status calls instead of churning the Pi off-screen.
  The initial load and manual Refresh remain unconditional.

Verified: {"enabled":"false"} now disables (refused w/ reason:no_ethernet),
{"enabled":"true"} enables; inline JS passes node --check; py_compile clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE

* Narrow WiFi-disable fallback except and add stack trace (review)

Addresses a CodeRabbit nitpick: the fallback handler in set_wifi_radio()'s
disable path caught bare Exception and logged without a traceback. Narrow it to
(OSError, subprocess.SubprocessError) — the errors subprocess.run realistically
raises — and log with exc_info=True for full context on the Pi. Anything
genuinely unexpected now propagates to the endpoint's outer handler (500),
matching the codebase's specific-exception convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-09 09:22:11 -04:00
3b93024993 feat: activate dormant plugin health/metrics subsystem and surface it in the web UI (#388)
* feat(plugin-system): activate dormant plugin health & metrics subsystem

PluginManager shipped a fully-built health tracker, resource monitor and
circuit breaker that were never instantiated (health_tracker/resource_monitor
were left as None), so the circuit breaker never engaged and the existing
health/metrics API routes always returned "not available".

- DisplayController now wires a PluginHealthTracker and PluginResourceMonitor
  onto the plugin manager, enabling the circuit breaker (a repeatedly-failing
  plugin's update() is skipped after consecutive failures, then retried after
  a cooldown) and per-plugin execution-time metrics. Both persist to the
  shared cache.
- load_plugin() now validates each plugin's config against its JSON schema in
  a strictly warn/degrade-only way: a violation logs a warning and flags the
  plugin degraded in the health tracker, but never changes whether the plugin
  loads or its pass/fail behaviour. Adds PluginHealthTracker.set_degraded(),
  which never touches the circuit breaker.
- ResourceMonitor CPU/memory sampling now reuses a cached psutil.Process and
  reads cpu_percent(interval=None), so monitoring no longer blocks ~100ms per
  call on the display loop's update path.
- Fix DiskCache.get() raising TypeError for max_age=None ("never expires"),
  which silently discarded persisted plugin health/metrics on read and thus
  broke cross-process and post-restart surfacing.
- Fix two dead PluginManager helpers that called non-existent tracker methods.

Tests: new test_resource_monitor, test_plugin_health,
test_plugin_manager_schema_soft; extended test_cache_manager and
test_display_controller.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq

* feat(web-ui): surface plugin health, metrics and load state

With the health/metrics subsystem now active in the display service, expose it
in the web UI (which runs as a separate process from the display loop):

- Wire a health tracker / resource monitor backed by the shared on-disk cache
  into the web process so /api/v3/plugins/health and /plugins/metrics read the
  data the display service persists.
- Build those route responses per installed plugin id (the tracker's in-memory
  view is empty in a fresh web process) so cross-process data is included.
- Add state + error_info to /plugins/installed entries so the UI can show why a
  plugin isn't running instead of just loaded:false.
- Add a "Plugin Health" panel to the Tools page (circuit status, avg/max update
  time, update count, last error) plus PluginAPI.getPluginMetrics().

Tests: route-level tests for the health/metrics endpoints in test_web_api.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq

* fix(plugin-metrics): refresh cross-process health/metrics reads; type hints

Addresses CodeRabbit review on #388:

- Major: the web process's health/resource trackers cached the first persisted
  read in an in-memory dict (and the CacheManager memory tier held max_age=None
  entries indefinitely), so a long-lived web process showed the first snapshot
  and never reflected the display service's later updates. Add an opt-in
  force_reload path (get_health_summary/get_health_state/_load_health_state and
  get_metrics_summary/get_metrics) that bypasses the in-memory copy and, via a
  new memory_ttl passthrough on CacheManager.get, the cache manager's memory
  tier — so each /plugins/health and /plugins/metrics poll reads fresh persisted
  state. Default behaviour (force_reload=False) is unchanged for the display
  process and existing callers.
- Minor: DiskCache.get type hint is now Optional[int] with the None ("never
  expires") semantics documented, matching MemoryCache.get.

Tests: new force_reload staleness cases in test_plugin_health and
test_resource_monitor.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-09 07:54:18 -04:00
85d321cf33 fix: plugin_loader retries with --ignore-installed on apt/pip RECORD conflicts (#386)
* fix: plugin_loader retries with --ignore-installed before assuming apt package satisfies pin

install_dependencies treated any "uninstall-no-record-file" pip failure as
"dependency satisfied" and wrote the success marker without ever attempting
--ignore-installed, unlike install_dependencies_apt.py and
safe_pip_install.sh (added in #385 for the Plugin Store/first-time-install
paths). A plugin pinning a newer version of a system-managed package (e.g.
requests) would silently keep running against whatever version apt shipped,
while the marker file claimed the pinned requirement was met.

Now retries the same install with --ignore-installed on that specific
failure so pip actually lays the pinned version down (shadowing the
system-managed copy) before falling back to the prior tolerant behavior if
the retry itself fails too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* chore: suppress Codacy finding on new retry subprocess.run call

Same generic Bandit/semgrep pattern-match on non-literal subprocess.run
argv flagged in #385's install_requirements_file, now on the new
--ignore-installed retry call added here: list-form argv (no shell=True),
sys.executable is this process's own interpreter, and requirements_file is
built internally by find_plugin_directory, never raw external input.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: tolerate a timed-out --ignore-installed retry, dedupe marker-write logic

CodeRabbit review caught a real inconsistency: if the --ignore-installed
retry itself timed out, subprocess.TimeoutExpired propagated to the outer
handler and returned False, failing plugin load — contradicting the
intended "tolerate this specific apt/pip conflict" behavior, where a mere
non-zero retry return code already returns True. Wraps the retry in its own
try/except so a timeout is logged and tolerated the same way as any other
retry failure.

Also extracts the marker-writing logic (open/write/chmod, ignoring OSError)
into _write_dependency_marker, since it was duplicated identically between
the direct-success path and the apt-conflict-retry-fallback path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: revert marker-write dedup helper, resolves CodeQL path-injection alert

CodeQL flagged _write_dependency_marker's open(marker_file, ...) as
"uncontrolled data used in path expression" (high severity) once the
marker-write logic was extracted into its own method. marker_file is
actually safe — it's built from safe_plugin_dir, which install_dependencies
sanitizes via os.path.basename() (CodeQL's own recognized py/path-injection
sanitizer, per the existing comment a few lines above) — but CodeQL's
interprocedural analysis doesn't carry that sanitized status across the new
method boundary, since the sanitizer call and the open() sink were no
longer in the same function.

This exact code produced zero CodeQL findings before the extraction (in two
duplicated inline blocks) and is unchanged in what data reaches it — only
its location moved. Reverting the extraction (keeping CodeRabbit's other,
independent timeout-handling fix) restores the previously-clean shape rather
than trying to convince the analyzer's cross-function taint tracking that a
refactor changed nothing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 11:19:37 -04:00
63a233f3ed fix: dependency installation gaps in Plugin Store and first-time install (#385)
* fix: install plugin dependencies through root-visible installer in Plugin Store

install_plugin/update_plugin (store_manager.py) installed requirements.txt
with a bare `pip3` off PATH, bypassing the root-visible installer added in
#380 for the "Reinstall Plugin Deps" button. Two bugs stacked: (1) `pip3`
can resolve to a different Python install than the one that actually runs
ledmatrix.service, and (2) even when it resolves correctly, ledmatrix-web
runs as a non-root user so the package lands in that user's local
site-packages, invisible to root-run ledmatrix.service. Either way the
install reports success and writes the .dependencies_installed hash marker,
so plugin_loader's own (correct) install-on-load path skips reinstalling —
leaving the dependency permanently missing until a user finds and clicks
the separate "Reinstall Plugin Deps" tool. This is why users kept hitting
"No module named 'astral'" for the weather plugin even after installing it
from the Store.

Extracts the sudo-wrapper-then-fallback install logic from api_v3.py's
_pip_install_requirements into src/common/permission_utils.py as
install_requirements_file, and routes store_manager.py's dependency
installation through it so the automatic Store install/update path now
matches the manual "Reinstall Plugin Deps" path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* chore: suppress Codacy false-positive on subprocess.run in install_requirements_file

Codacy's generic subprocess-security rule (Bandit B603 equivalent) flagged
the pip/sudo subprocess.run calls in install_requirements_file for lacking a
"static string argument" — the standard pattern-based flag for any
subprocess.run() call with a variable in its argv list. Both calls use
list-form argv (no shell=True, so no shell-injection surface), and the only
dynamic value is req_file, a Path built internally by callers rather than
raw external input; safe_pip_install.sh independently re-validates it before
installing anything as root. Suppresses with inline `# nosec B603` comments
matching this codebase's existing convention (see permission_utils.py's own
PROTECTED_SYSTEM_DIRECTORIES, display_manager.py, sync_manager.py, etc.).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: first-time install script fails on apt-managed requests package

web_interface/requirements.txt and requirements.txt both pin
requests>=2.33.0,<3.0.0, but Raspberry Pi OS ships an apt-managed
python3-requests with no pip RECORD file. Upgrading it via plain
`pip install` aborts with "uninstall-no-record-file" because pip refuses to
uninstall a package it has no record of, in place — which is exactly the
"Some web interface dependencies failed to install" warning first-time
install hits.

scripts/install_dependencies_apt.py and scripts/fix_perms/safe_pip_install.sh
already work around this with --ignore-installed (lets pip lay the new
version down in /usr/local, shadowing the apt copy, instead of trying to
remove it first). first_time_install.sh's own direct pip invocations —
the per-package requirements.txt loop, the web_interface/requirements.txt
install, and the requirements_web_v2.txt fallback — didn't have it. Adds
--ignore-installed to all three so first-time install no longer fails on
this well-known apt/pip conflict.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* chore: add nosemgrep to subprocess.run calls Codacy still flagged

The prior # nosec B603 comments suppressed Bandit's check but Codacy's
semgrep-based rule ("subprocess function 'run' without a static string")
kept flagging the same two lines as a critical security issue even after
that fix landed. install_dependencies_apt.py's _run() already needed both
tags together (# nosec B603 B607 ... # nosemgrep) for the identical
subprocess.run pattern, so apply the same double suppression here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: add --ignore-installed to install_requirements_file fallback path

CodeRabbit review caught this (confirming a gap already flagged in
conversation): the non-sudo fallback pip install in install_requirements_file
was missing --ignore-installed, unlike the sudo-wrapper branch and
safe_pip_install.sh. Without it, the same apt/pip RECORD-file conflict this
PR fixes elsewhere (first_time_install.sh, install_dependencies_apt.py) could
still hit installs that fall back to this path (e.g. a plugin's
requirements.txt on a host where safe_pip_install.sh isn't set up yet).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 09:59:12 -04:00
7a9d01342a Fix inconsistent Vegas scroll transition gaps caused by plugin-baked padding (#384)
* Strip plugin-baked scroll padding when capturing content for Vegas mode

Plugins that build their own ticker image via ScrollHelper.create_scrolling_image()
(or that manually pad both ends for a clean standalone loop) carry a solid-black
margin up to display_width wide on one or both edges. Vegas mode already adds its
own configurable gap around every item, so leaving that margin in place stacked an
extra, uncontrolled blank stretch on top of separator_width for whichever plugin
took the ScrollHelper-capture path — producing inconsistent transition gaps between
modules compared to plugins that provide content natively via get_vegas_content().

_get_scroll_helper_content() now detects and crops any such margin before handing
the image to the Vegas render pipeline, so every plugin's gap is governed solely by
vegas_scroll.separator_width regardless of which capture path produced its content.

* Address CodeRabbit nitpick: warn on double-edge padding crop, add unit tests

Logging a double-edge match at warning level (vs. info for a single edge)
makes it easy to spot an unexpected crop in the field, since two edges
matching at once is a much stronger signal of genuine baked-in padding than
one edge coinciding with real all-black content.

Also adds test/test_vegas_plugin_adapter.py covering _strip_scroll_padding's
branch logic: leading-only, trailing-only, both-edges, no-match, degenerate
all-black, missing/undersized display_width, and the info-vs-warning log level.

* Add type hints and docstring to test _solid helper (CodeRabbit nitpick)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 09:43:13 -04:00
9b2f02681d feat(web-ui): detect and surface Raspberry Pi under-voltage/throttling (#383)
* feat(web-ui): detect and surface Raspberry Pi under-voltage/throttling

Adds a vcgencmd get_throttled check to the system-status SSE stream and
surfaces it in the web UI:

- A header badge (next to CPU/Memory/Temp) that stays hidden when healthy,
  turns red when under-voltage/throttling is happening right now, and
  yellow if it happened earlier this session but has since cleared.
- A dismissible top banner (same pattern as the update-available banner)
  that appears while under-voltage/throttling is actively occurring, with
  guidance to check the power supply. Re-appears on a fresh occurrence
  even if a previous one was dismissed.
- A "Power Supply" card on the Overview tab alongside CPU/Memory/Temp/
  Display Status.

Motivated by a real device showing intermittent brightness flicker that
turned out to be ~1 under-voltage event every 30-90s (visible live via
`vcgencmd get_throttled` and dmesg's "Undervoltage detected!" messages) --
there was no way to see this from the web UI, only by SSHing in.

Returns None on non-Pi platforms (no vcgencmd on PATH), matching the
existing guard pattern used for the CPU temperature read.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web-ui): add power supply diagnostics detail to Tools tab

The header badge/banner/Overview card added in the previous commit only
show a collapsed "is it bad right now" signal. This adds a "Power Supply"
section to the Tools tab with the full 8-flag breakdown (under-voltage,
throttled, freq-capped, soft-temp-limit -- each split into "right now" vs
"occurred since boot") for actually troubleshooting a recurring issue,
plus a pointer to the README's power supply sizing guidance when something
is or was flagged.

Reuses the existing stats SSE stream (window.statsSource) rather than
adding a new endpoint -- the same payload already drives the header/
banner/Overview card, so this just listens for it too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web-ui): address review findings on power supply monitoring

- Overview card left its "--" placeholder forever on non-Pi platforms since
  the update only handled a truthy data.power. Now explicitly renders "Not
  available" with a neutral icon/color for that case.
- vcgencmd failures logged at debug, invisible in default remote log
  output. Bumped to warning to match the nearby systemctl failure logging.
- The banner/badge/card and the Tools summary line only looked at
  under_voltage_now/throttled_now (+ occurred), silently ignoring
  freq_capped_now/occurred and soft_temp_limit_now/occurred from
  _get_power_status() -- a Pi that's actively soft-thermal-limited or
  frequency-capped showed a green "OK" everywhere except the detailed
  flag table buried in Tools. All four surfaces now fold all four "now"/
  "occurred" flags into the same active/occurred state.
- The banner text was hardcoded to "Under-voltage detected..." even when
  the actual active condition was throttling/freq-capping/thermal limiting.
  Added _activePowerConditionLabels() (shared, non-module global scope) to
  build the banner/tooltip text from whichever flags are actually set.

Skipped: TTL-caching _get_power_status() to avoid "multiplying forks
across browser tabs" -- that premise doesn't hold against this codebase.
_StreamBroadcaster (its own docstring says as much) already runs exactly
one shared generator per tick regardless of client count, identical to
the uncached cpu_temp file-read two lines above it; there's nothing to
multiply.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web-ui): address second round of review findings on power monitoring

- updatePowerStatus's falsy-power branch only hid power-stat, leaving
  power-warning-banner visible with stale text if _get_power_status()
  fails transiently. Now hides the banner too and resets the dismissed
  flag, same as the "not active" branch.
- The Tools tab's status badge (and the pre-existing dirty/clean badge
  right next to it) build class names like bg-${color}-100/text-${color}-800
  at runtime. This project hand-rolls its own Tailwind-named utility
  classes in app.css rather than running a real Tailwind build, and the
  light-mode base rules for bg-red-100/bg-yellow-100/bg-green-100/
  text-red-800/text-yellow-800/text-green-800 were simply never defined --
  only some had dark-mode overrides, which are no-ops without a base rule
  in light mode. Added the missing light-mode bases plus the two missing
  dark-mode overrides (bg-yellow-100/bg-green-100), fixing both badges.
- The "occurred earlier" tooltip was a hardcoded string regardless of
  which flag(s) actually fired. Generalized _activePowerConditionLabels()
  to take a suffix ('_now' or '_occurred') and reused it for both tooltips.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* refactor(web-ui): move Power Supply status off the Overview tab

The Overview tab's "Power Supply" stat card duplicated what the Tools
tab's diagnostics section already shows (summary badge + full flag
breakdown), so drop the card and its now-dead JS rather than keep two
copies in sync. The header badge and warning banner (visible on every
page) are unaffected -- only the Overview-tab card is removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 09:35:55 -04:00
7a6bad29fe feat(display-controller): hot-reload plugin enable/disable without a restart (#374)
* feat(display-controller): hot-reload plugin enable/disable without a restart

Enabling or disabling a plugin in config previously required restarting the
display service: the plugin list and available_modes were built once at init
and the run loop never revisited them. (Per-plugin config *values* already
hot-reloaded; only the enabled set was restart-only.)

Now the controller reconciles its running plugins against the config's enabled
set whenever that set changes:

- The ConfigService watcher thread only sets a `_pending_plugin_reconcile`
  flag (via a cheap enabled-set diff). It never mutates loop state.
- The run loop applies the reconcile on its own thread (top of each
  iteration, deferred while on-demand is active), so loading/unloading and
  rebuilding available_modes can't race with rendering.
- `_reconcile_enabled_plugins` diffs desired vs running plugins, unloads the
  removed ones (cleanup + on_disable + config-unsubscribe via the new
  `_unregister_plugin`) and loads the added ones, then clamps the rotation
  index so the current mode stays valid.

The per-plugin registration done at startup is extracted into
`_register_loaded_plugin` and reused by the live-enable path so both build
identical state. Extracting it also fixes a latent late-binding bug: the
per-plugin config-change callbacks were closures over the loop variable, so
every plugin's callback targeted the last-loaded instance; each now binds its
own id/instance.

Adds test/test_display_controller_plugin_toggle.py covering live enable,
live disable, index clamping, no-op when unchanged, and the enabled-set diff.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(display-controller): don't exit on empty available_modes, guard rotation modulo

Hot-reload means available_modes can legitimately be empty at startup (no
plugins enabled yet) and become non-empty later via the web UI, or vice
versa mid-run. Fix four issues found reviewing this PR:

- run() exited the process entirely when available_modes was empty at
  startup instead of idling, permanently defeating the point of live
  enable/disable for anyone who starts with zero plugins enabled.
- The mode-rotation step divided by len(available_modes) unconditionally,
  raising ZeroDivisionError if the last enabled plugin is disabled between
  frames.
- _reconcile_enabled_plugins() called .get('enabled', False) on a config
  section without checking it was a dict first, raising AttributeError on
  a malformed config value.
- Minor: pop the config-change callback only after attempting to
  unsubscribe it, and log the exception in the config-read fallback
  instead of swallowing it silently.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(display-controller): address review findings on the hot-reload PR

- Idle-wait tick was a fixed 30s sleep, delaying pickup of a plugin
  enabled via the web UI while no modes were active. Shortened to ~1s so
  it's roughly as responsive as the per-frame check once modes exist.
- _unregister_plugin popped the config-change callback from
  _plugin_config_callbacks even when config_service.unsubscribe() raised,
  losing the only reference to it. Now only pops on a successful
  unsubscribe.
- _pending_plugin_reconcile was cleared before _reconcile_enabled_plugins()
  ran, so a retryable failure (e.g. plugin discovery erroring) silently
  dropped the enable/disable request. _reconcile_enabled_plugins() now
  returns True/False and the caller only clears the flag on True.
- Added a warning log for the malformed-config case (a plugin's config
  section present but not a dict) so it's actually visible, and updated
  the existing test to assert it via caplog.

Left the broad `except Exception` around config_service.unsubscribe() as
Exception -- the current implementation is a simple lock+dict/list op that
doesn't document or realistically raise a narrower type, so this is a
defensive catch-all, not user error handling.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: ChuckBuilds <charlesmynard@gmail.com>
2026-07-06 17:37:01 -04:00
ChuckandGitHub bea00448d3 update mlb logos (#382)
update mlb logos

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-07-06 08:55:43 -04:00
deaa3d7a98 fix: array-table boolean columns always saved as unchecked (#381)
Reported bug: countdown plugin always shows "No Active" even with a
countdown configured and enabled — because every save silently unchecked
it. Root cause: the boolean column's hidden-sentinel input (needed so
unchecked checkboxes still submit "false", since browsers omit unchecked
checkboxes entirely) was hardcoded to value="false" and shared the same
`name` as the checkbox, with no sync between them. Whichever of the two
same-named inputs the save request's form-collection happens to prefer,
the hidden's stale "false" could silently override an actually-checked
box on every save, independent of what the user set.

Fixed in both places this pattern is rendered:
- plugin_config.html: the initial server-rendered row for every array-table
  boolean column (affects countdown's per-countdown "enabled" and the
  custom-feeds widget's per-feed "enabled") — now sets the hidden's initial
  value from the actual data, and syncs it via onchange on every toggle.
- array-table.js: the client-side row renderer used when adding/rebuilding
  rows in the browser — same fix, keeping both inputs in sync at render
  time and via a change listener.

Verified other checkbox/hidden-input patterns in plugin_config.html and
confirmed they're unrelated/already-safe: the default single-checkbox
renderer (no hidden pair), checkbox-group (array-bracket names with its
own explicit sync), the array-table advanced-props modal editor (single
hidden per name, explicitly written on Save), the top-level plugin
enable/disable toggle (separate immediate hx-post), and the no-schema
fallback checkbox. custom-feeds.js's own client-side row renderer never
creates a hidden sentinel at all, so it isn't affected either.

Verified the fix's rendered output directly with an isolated Jinja render
of the exact snippet: hidden and checkbox now agree ("true"/checked or
"false"/unchecked) for both states, where before the hidden was always
"false" regardless of the actual value.


Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 08:05:06 -04:00
cbb8ec41e8 Fix: plugin/base requirements installed as web-user, invisible to root-run display service (#380)
* fix: install plugin/base requirements as root so ledmatrix.service can see them

ledmatrix-web.service runs as a non-root user, so "Reinstall plugin
requirements" installed packages into that user's ~/.local site-packages.
ledmatrix.service (the actual display, which loads and runs plugin code)
runs as root and can't see another user's user-site packages, so plugins
with dependencies not already present system-wide would silently fail at
runtime with ModuleNotFoundError even after a "successful" reinstall.
Reproduced and fixed live against a real device (weather plugin's astral
dependency, used for moon-phase data): confirmed the exact failure
("No module named 'astral'" on every almanac cycle) and confirmed it's
gone after this fix.

Adds scripts/fix_perms/safe_pip_install.sh, a root-owned wrapper (mirroring
the existing safe_plugin_rm.sh pattern) that validates the target is
requirements.txt at the project root or under plugin-repos/ or plugins/
before running pip install as root. configure_web_sudo.sh provisions a
narrowly-scoped sudoers rule for it. api_v3.py's install_base_requirements
and install_plugin_requirements actions now use it via `sudo -n`, falling
back to today's current-user-only install (with an explanatory note) if
the wrapper isn't set up yet, so existing installs don't regress.

Also uses --ignore-installed in the wrapper: root's site-packages often has
apt/dpkg-managed copies of common libraries (requests, etc.) with no pip
RECORD file, which pip refuses to upgrade in place and aborts the *entire*
requirements.txt install over — discovered this while testing the fix live,
since a plugin's other already-satisfied-for-the-web-user dependencies had
never actually been attempted as root before.

Also fixes a pre-existing bug in configure_web_sudo.sh where the
display_controller.py/start_display.sh/stop_display.sh sudoers entries used
PROJECT_DIR (scripts/install/, where this script lives) instead of
PROJECT_ROOT (where those files actually live) — visible as the script's
own "File access test" self-check failing. Verified fixed live.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix: invoke safe_pip_install.sh via explicit bash, matching sudoers rule

CodeRabbit caught this on review: the sudoers rule configure_web_sudo.sh
provisions is scoped to "$BASH_PATH $SAFE_PIP_INSTALL_PATH *" (matching
the existing safe_plugin_rm.sh precedent in
src/common/permission_utils.py), but _pip_install_requirements() called
`sudo -n <wrapper> <req_file>` directly, relying on the script's shebang
instead of an explicit bash prefix. sudo matches the literal command line,
so this never matched the allowlisted rule on an install with only the
specific sudoers entries this script provisions — it silently fell back
to the non-root install path every time, which is the exact bug this PR
set out to fix.

This wasn't caught by live testing on ledpi.local because that device
also has a broader, non-standard "NOPASSWD: ALL" grant which masked the
mismatch. Confirmed the fix is correct by reading sudo's documented
command-matching semantics and mirroring the already-proven-working
bash-prefix pattern from permission_utils.py's safe_plugin_rm.sh call
exactly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix: harden safe_pip_install.sh invocation against bash-path drift + address CodeRabbit nitpicks

CodeRabbit follow-up findings on the bash-prefix fix (7558aaab):

1. (Actionable) shutil.which('bash') at runtime could in principle resolve
   to a different absolute path than configure_web_sudo.sh's `command -v
   bash`, which is resolved once at setup time and baked into the static
   sudoers file as a literal string — sudo requires an exact match. Now
   tries /usr/bin/bash and /bin/bash (the standard Debian/Raspberry Pi OS
   locations, matching what the setup script virtually always produces)
   before falling back to this process's own PATH resolution, so a
   divergence in just one of them doesn't break the install.

2. (Nitpick) Any nonzero returncode was treated as "sudo denied", so a
   real pip failure (bad package, build error) would trigger a pointless
   duplicate non-root install attempt and a misleading error message.
   Now distinguishes "sudo -n rejected this exact command line" from
   "sudo ran it but the command itself failed" via sudo's own diagnostic
   text, and surfaces genuine failures immediately without retrying other
   bash candidates or falling back.

3. (Nitpick) Added structured logging for every fallback/failure path
   (wrapper missing, sudo denied, real install failure), previously only
   visible via the returned stdout note — needed for remote debugging on
   a headless Pi.

Verified: function-level smoke test confirms a real failure (this sandbox's
system python3 lacking pip) is now correctly classified as non-denial and
returned immediately without retrying candidates or double-installing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 08:04:49 -04:00
c6ce332d49 fix(web): restore missing brace in Tools tab HTMX-fallback path (#378)
The `else if (++tries > 100)` block added by #373 was missing its
closing `}`, leaving the setInterval arrow function syntactically
unclosed. This caused a JS parse error that silenced the entire
1400-line script block — including the EventSource setup — so the
connection-status indicator never left its default "Disconnected"
state for all users after updating.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 10:05:33 -04:00
ChuckandGitHub 8e5f66501a Add Claude Code GitHub Workflow (#377)
* "Claude PR Assistant workflow"

* "Claude Code Review workflow"
2026-06-29 14:30:41 -04:00
639e1c3a93 fix(web): repair news ticker custom-feeds save for JSON path (#376)
* feat(install): surface root cause of web dependency install failures

install_dependencies_apt.py previously reported only which packages
failed, not why - the actual apt/pip error was discarded (apt) or
could scroll out of the on_error log tail (pip), leaving "Step 7:
Install web interface dependencies (line 915)" as the only visible
detail.

Capture command output for each install attempt and print a compact
DEPENDENCY INSTALLATION FAILURES summary with the last lines of error
output per package. Also run the installer with `python3 -u` for
real-time, correctly-ordered logging, and widen the on_error tail from
50 to 100 lines so the summary isn't cut off.

* fix(web): repair news ticker custom-feeds save for JSON path

The JS dotToNested() helper converts indexed form fields like
feeds.custom_feeds.0.name into a dict {'0': {name:...}} rather than a
proper array. The form-data path already had fix_array_structures() to
convert those dicts back to arrays before schema validation, but the
JSON path (used by all web-UI saves) never ran that fix, so saving any
custom feed produced a schema validation error: "Expected type array,
got object".

Add _fix_json_arrays() immediately after schema loading on the JSON
path, mirroring the existing fix_array_structures() logic.

Also fix custom-feeds.js getValue() to omit the logo key entirely when
no logo is present instead of returning logo:null, which would fail
schema validation (logo expects type object).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(install,tools): address PR 376 review findings

- first_time_install.sh: add _clone_rpi_rgb() wrapper so retry() cleans
  up any partial rpi-rgb-led-matrix-master dir before each clone attempt
- first_time_install.sh: use apt-get -o DPkg::Lock::Timeout=180 so apt
  handles lock contention natively instead of relying solely on flock TOCTOU check
- install_dependencies_apt.py: pass DPkg::Lock::Timeout=180 to apt-get
  install to avoid failing when unattended-upgrades holds the lock
- install_dependencies_apt.py: add type annotations to all public helpers
- api_v3.py: fix install_plugin_requirements to read plugin_manager from
  api_v3 blueprint attribute instead of the always-None module variable
- tools.html: loadGitInfo() now checks r.ok before parsing JSON and
  surfaces d.status === 'error' with the server's message in the panel

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tools,api): address three additional review findings

- api_v3.py install_plugin_requirements: replace hardcoded plugin-repos
  fallback with config-driven resolution (plugin_system.plugins_directory),
  matching the pattern used elsewhere in the module
- api_v3.py _fix_json_arrays: recurse into converted and existing array
  elements when items.type is object, so nested numeric-keyed dicts inside
  array items are also normalized
- tools.html toolsAction: check r.ok before r.json() and recover
  gracefully from non-JSON error bodies (HTML 500 pages), consistent
  with the existing loadGitInfo guard

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 14:21:05 -04:00
6096a22c3d feat(web): add Tools tab and row address type display setting (#373)
* feat(web): add Tools tab and row address type setting

Adds a Tools/Utilities tab to the web interface with one-click
maintenance buttons that previously required SSH:
- Git status panel (branch, dirty state, recent commits)
- Pull latest (rebase) and force reset to origin/main
- Reinstall base requirements (pip, with output)
- Reinstall per-plugin requirements (pass/fail per plugin)
- Clear __pycache__ directories
- Quick-access restart for display and web services

Also exposes the hzeller row_address_type option (0–4) in the
Display settings tab. The backend already read this value from
config; the UI, API field list, and validation were missing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tools-tab): address code review findings

- Add _GIT = shutil.which('git') alongside _SUDO/_JOURNALCTL; return
  503 in force_git_reset and get_git_info if git is unavailable
- Check git branch/status returncodes in get_git_info(); return a clear
  500 error instead of silently treating a failed run as a clean repo
- Cap pip stdout+stderr at 50 KB via _truncate_output() helper to
  avoid OOM on verbose dependency resolution or build failures
- Scrub embedded HTTPS credentials from remote_url via
  _scrub_git_remote_url() using urllib.parse before returning to UI
- Fix clear_pycache to track and report failed deletions separately
  instead of counting them as successes (removed ignore_errors=True,
  wrapped in try/except OSError)

Skipped: plugin_manager-vs-api_v3.plugin_manager (api_v3 is the
Blueprint object; accessing .plugin_manager on it would fail — module-
level variable is the correct pattern used throughout this blueprint);
pages_v3 broad-except (identical to every other _load_*_partial in the
file); base.html HTMX fallback (loadTabContent handles all tabs
generically; named fallbacks only exist for tabs needing JS re-init);
tools.html auth (pre-existing architectural decision — reboot/shutdown
on the same endpoint are also unauthenticated).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tools-tab): resolve remaining PR review comments

- api_v3: use getattr(api_v3, 'plugin_manager', None) instead of the
  module-level plugin_manager (always None); app.py sets the blueprint
  attribute, not the module global, so the fallback to plugin-repos was
  always taken
- pages_v3: replace broad except Exception in _load_tools_partial with
  specific TemplateNotFound / OSError handlers and add [Pages V3][Tools]
  context prefix to log messages and error responses for easier Pi
  debugging
- base.html: add Tools tab branch to the HTMX-unavailable fallback block
  in loadTabContent so the tab loads gracefully via direct fetch if HTMX
  never initialises

Skipped: auth on execute_system_action — pre-existing app-wide design;
reboot/shutdown and all other system actions share the same exposure.
An app-level auth layer is the correct fix and is out of scope here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tools-tab): resolve second-pass review findings

- Wrap per-plugin subprocess.run in try/except TimeoutExpired/OSError so
  one plugin's failure appends a result entry and continues the loop
  rather than collapsing the whole batch into a 500
- Validate double_sided_copies divisibility against chain_length
  (horizontal axis) or parallel (vertical axis) after the range check;
  reads effective axis from the current request or stored config
- Exclude double_sided_fields from the generic key-merge loop so
  double_sided_enabled/copies/axis are never written as root-level keys
- Fix tools.html copy: "then restores the stash" removed — git_pull
  stashes changes but never pops them
- Check r.ok and d.status in loadGitInfo before building the panel;
  backend error messages now surface instead of silently showing a
  false-clean state

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(tools-tab): don't expose filesystem paths in OSError messages

CodeQL flagged str(exc) flowing into the JSON response for the
install_plugin_requirements action. Use exc.strerror instead, which
gives the OS error description ("No such file or directory",
"Permission denied") without the internal filesystem path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 12:19:54 -04:00
fefc2d44a2 feat(display): double-sided mode — mirror one screen across the panel chain (#375)
* feat(display): add double-sided mode to mirror one screen across the panel chain

Renders a plugin once at a logical (per-screen) size, then tiles the
rendered frame across the full physical chain so two (or more) panels show
identical content. A 128x32 chain configured with 2 copies drives two 64x32
screens; vertical axis splits parallel outputs instead of the chain.

Plugins size themselves from matrix.width/height, so a thin _LogicalMatrix
proxy reports the logical size while delegating every real operation
(CreateFrameCanvas, SwapOnVSync, brightness, Clear) to the physical matrix —
no plugin changes required. Duplication is a single PIL paste per copy in
update_display(), so render cost is unchanged.

Config: display.double_sided { enabled, copies, axis }. Invalid config
(non-divisible dimension, bad axis/copies) logs a warning and falls back to
single-screen rather than failing to light up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(web): expose double-sided display config in the settings UI

Adds a Double-Sided Display section to the Display settings page (enabled
checkbox, copies, horizontal/vertical axis) and wires the save handler to
persist it under display.double_sided. Validates copies (2-8) and axis,
returning 400 on bad input; an omitted checkbox is saved as disabled.
Like the other hardware fields, changes take effect after a display restart.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(display): add type hints and docstrings to double-sided proxy

Addresses CodeRabbit nits: sort _LogicalMatrix.__slots__ (Ruff RUF023),
annotate the proxy's __init__/properties/dunders and _resolve_double_sided's
return type, and add docstrings to the property/dunder methods.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 15:24:53 -04:00
d297dd6217 feat(display-controller): round-robin between simultaneous live-priority games (#372)
_check_live_priority() was stateless first-match-wins: it returned the
first plugin in registration order with live content, and the post-dwell
hold pinned the carousel to it, so when two games were live at once (e.g.
a baseball game and a soccer match) the second never showed until the
first ended.

Add _collect_live_modes() (all currently-live modes, deduped, in
registration order) and give _check_live_priority an 'advance' flag. The
main rotation calls it with advance=True, which returns the live mode
after the one currently shown -- using current_display_mode as the cursor
-- so each dwell advances to the next live game and they take turns. The
Vegas coordinator and the vegas-active check keep the default
non-advancing peek (advance=False), so they only report whether any game
is live without spinning the cursor. should_rotate and _apply_live_priority
are unchanged; a single live game still holds as before.

Adds regression tests to TestDisplayControllerLivePriority.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:07:21 -04:00
974d7ea57a fix(install): avoid apt-package uninstall failure during web dep install (#371)
* fix(install): avoid apt-package uninstall failure on web deps

On a fresh Pi install, requests is installed via apt (python3-requests),
which ships no pip RECORD file. When pip later installs
google-api-python-client, its dependency tree pulls a newer requests and
attempts to uninstall the apt copy, failing with "uninstall-no-record-file"
and aborting the whole install at step 7 (web interface dependencies).

Add --ignore-installed to install_via_pip so pip lays the new version down
in /usr/local (shadowing the apt copy) instead of trying to remove an
apt-managed package. This resolves the failure for any transitive
dependency pip needs to upgrade over an apt-installed package, not just
requests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(install): also pass --ignore-installed for local rgbmatrix install

Keeps the rgbmatrix pip install consistent with install_via_pip so it
won't fail trying to uninstall an apt-managed dependency either.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:06:56 -04:00
ab0cfd2362 fix(web): preserve dotted schema keys when saving plugin config (#370)
The plugin config form posts form-data with dot-notation paths
(e.g. "leagues.fifa.world.enabled"). _get_schema_property and
_set_nested_value split those paths on every dot, so a schema key that
itself contains a dot (soccer league keys like "fifa.world", "eng.1")
was mistaken for nested "fifa" -> "world" objects. Per-league edits
(enable, favorite_teams, nested booleans) were written to a fabricated
"leagues.fifa.world" branch while the real league object was never
updated, so saves silently dropped the change and produced a
byte-identical config.

Both helpers now greedily match the longest path segment that exists in
the schema (_get_schema_property) or the config being updated
(_set_nested_value), mirroring the frontend's dotted-key handling.

Adds regression tests covering schema lookup, value typing, and writes
under dotted league keys, plus a guard that plain nested paths still work.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:06:32 -04:00
d22d0a3754 fix(plugins): stop core updates from resurrecting uninstalled built-in plugins (#368)
* fix(plugins): stop core updates from resurrecting uninstalled built-in plugins

Built-in plugins (e.g. web-ui-info, starlark-apps) are committed into the
repo under plugin-repos/. When a user uninstalls one, a subsequent core
`git pull` update restores the committed files, so the plugin reappears on
every update. The update endpoint stashes the deletion and never pops it,
and `git pull` faithfully restores any committed file whose deletion was
never committed — so excluding plugin-repos/ from the stash can't fix this
(it would only make `git pull --rebase` fail on a dirty tree).

Add a persistent uninstall registry (config/uninstalled_plugins.json,
gitignored) that survives restarts, unlike the existing in-memory tombstone:

- Uninstall records the plugin id; install clears it.
- purge_uninstalled_plugins() re-removes any recorded plugin whose directory
  reappears on disk; called after a successful git-pull update and at web
  startup (covers manual `git pull` on the Pi too).
- The state reconciler also refuses to auto-repair a persistently
  uninstalled plugin.

Wires up mark_recently_uninstalled in the uninstall flow (previously only
referenced by tests) via the new persistent record.

Adds regression tests covering record/forget/purge lifecycle, persistence
across manager instances, and corrupt-registry tolerance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(plugins): validate uninstall-registry ids and lock registry writes

Address review feedback on the persistent uninstall registry:

- Critical: validate plugin ids on read/record and add a containment guard
  in purge_uninstalled_plugins. A corrupt or hand-edited registry entry of
  "" resolves to the plugins root, so purge could have deleted every plugin;
  traversal ids ("..", "../x") could target paths outside the root. Invalid
  ids are now dropped on read, refused on record, and never removed unless
  the path is a direct child of the plugins directory.
- Major: guard record/forget read-modify-write with a lock so concurrent
  install/uninstall requests can't lose updates.
- Minor: narrow the startup and post-update purge exception handlers from
  bare Exception to (OSError, RuntimeError).

Adds regression tests for empty-id, traversal-id, and invalid-record cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:18:28 -04:00
5beef0aa01 Improve first-time install error diagnostics and resilience (#369)
* fix(install): don't let outer ERR trap mask first_time_install.sh failures

set +e alone doesn't suppress bash's ERR trap, so any non-zero exit from
first_time_install.sh inside the one-shot installer immediately triggered
the outer on_error handler with a generic "Main installation, line 370"
message — before the script could report the real exit code or point to
logs/. Suspend the trap for that block so the existing if/else handling
runs instead.

* feat(install): surface root cause of web dependency install failures

install_dependencies_apt.py previously reported only which packages
failed, not why - the actual apt/pip error was discarded (apt) or
could scroll out of the on_error log tail (pip), leaving "Step 7:
Install web interface dependencies (line 915)" as the only visible
detail.

Capture command output for each install attempt and print a compact
DEPENDENCY INSTALLATION FAILURES summary with the last lines of error
output per package. Also run the installer with `python3 -u` for
real-time, correctly-ordered logging, and widen the on_error tail from
50 to 100 lines so the summary isn't cut off.

* feat(install): harden first-time install against common Pi failure modes

- wait_for_apt_lock: apt_update/apt_install now wait (up to 3min) for
  unattended-upgrades to release the dpkg lock instead of failing
  outright with "Command failed after 3 attempts" right after first boot.
- check_disk_space: new pre-flight check (Step 1) so a full SD card fails
  fast with a clear message instead of a cryptic mid-build error.
- Step 6: wrap rpi-rgb-led-matrix git clone/submodule operations in retry
  for resilience to transient network issues.
- Step 6: capture `pip install .` build output and print the last 50
  lines on failure, so the actual cmake/compiler error is visible instead
  of just "Failed to install rpi-rgb-led-matrix Python package".

* fix(install): bound subprocess output and dedupe apt update in dependency installer

Address coderabbitai review on PR #369:
- _run() now streams combined stdout/stderr to a temp file and returns
  only the last ERROR_TAIL_LINES lines, instead of buffering full
  output in memory (Codacy also flagged the previous capture_output
  call as a subprocess-without-static-string security issue; the new
  call is annotated as safe since cmd is built from hardcoded args).
- `apt update` now runs once in main() instead of once per package
  needing an apt fallback.

* fix(install): suppress remaining Codacy subprocess false-positive

Codacy's Semgrep-based check still flagged the cmd-built subprocess.run
call as "without a static string" even with the Bandit nosec applied.
Add a nosemgrep marker alongside it - cmd is always a hardcoded
apt/pip argument list, never user input.

* fix(install): correctly detect already-installed dateutil/websocket-client

Address remaining coderabbitai findings on PR #369:
- check_package_installed() did __import__(package_name) directly, but
  python-dateutil and websocket-client import as dateutil/websocket. Both
  always failed the "already installed" check and were reinstalled on
  every run. Add an IMPORT_NAME_MAP for the mismatched names.
- _run() still read the entire temp file into memory before slicing the
  tail. Stream it line-by-line into a deque(maxlen=ERROR_TAIL_LINES)
  instead so memory use stays bounded for very chatty commands.

---------

Co-authored-by: Chuck <chuck@example.com>
2026-06-11 18:12:35 -04:00
cf28a8c0d5 fix(display): restore early-continue guard for mid-loop mode changes (#367)
When the display loop breaks early because current_display_mode changed
(on-demand activation, live priority, etc.), it would fall through to the
"honour minimum duration" sleep for the *previous* mode — blocking for up
to that mode's full display_duration (default 30s) without polling
on-demand requests or re-checking the mode. New modes could sit unrendered
for up to 30s, or get clobbered by a queued stop request before ever
displaying.

This guard was added in #298 to fix #196 (live priority not interrupting
long display durations) and was accidentally dropped in #330 as collateral
damage of an unrelated time.monotonic() -> time.time() cleanup in the same
diff hunk. Restoring it fixes both the original #196 regression and a new
symptom found via the on-air MQTT plugin, where ON/OFF toggles could be
delayed by up to 30s or missed entirely depending on timing within the
previous mode's display cycle.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 10:16:30 -04:00
a06682981c fix(web): allow up to 24 panels in chain length config (#366)
Raises the Chain Length input's max from 8 to 24 to support longer
LED panel strings.

Co-authored-by: Chuck <chuck@example.com>
2026-06-09 21:31:07 -04:00
bc027c921d fix: check_plugin.py honors per-plugin test/harness.json (#365)
check_one() always compares the render against committed golden images, but
the CLI never loaded the plugin's test/harness.json — so the deterministic
settings the goldens were generated with (config, mock data, frozen time,
sizes) weren't applied. For any time/data-dependent plugin this means the CLI
(and the plugins-repo CI workflow that calls it) renders live data and the
golden drifts on every run, even with no real regression. The pytest matrix
path already reads harness.json via load_harness_spec; the CLI now does too.

- check_one loads load_harness_spec(plugin_dir) and layers it under explicit
  CLI flags: config = schema defaults < harness.json < --config; sizes =
  --sizes > LEDMATRIX_TEST_SIZES env > harness.json > default sample;
  mock_data/freeze_time/skip_update fall back to harness.json when not given
  on the CLI.
- parse_sizes returns None (not DEFAULT_TEST_SIZES) when --sizes is omitted,
  so the env/harness.json/default fallback chain in resolve_test_sizes applies.
- Regression tests: harness.json supplies render settings, and CLI flags
  override it. Use a temp fixture plugin so they run in core CI (no plugins).

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 12:52:33 -04:00
e0bd7088fa fix: make requirements-test.txt installable alongside requirements.txt (#364)
requirements.txt already pins pytest>=9.0.3,<10 (from #331), but
requirements-test.txt re-pinned pytest>=7.4,<9. The two ranges are
disjoint, so `pip install -r requirements.txt -r requirements-test.txt`
fails with ResolutionImpossible — breaking the core test workflow and the
plugins-repo safety workflow that installs both files.

pytest, pytest-cov, pytest-mock, and jsonschema are all already pinned
with major-version caps in requirements.txt, so drop them from
requirements-test.txt and keep only freezegun (the one test dep
requirements.txt doesn't provide).

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 10:32:05 -04:00
313e35a98f Add cross-size/cross-screen plugin safety harness (#361)
* feat(testing): add cross-size/cross-screen plugin safety harness

Render every plugin across all supported matrix sizes (64x32, 128x32,
128x64, 256x32) and every declared screen, failing on crashes, content
drawn past the panel edge, or visual drift vs committed golden images.

- BoundsCheckingDisplayManager: oversized-canvas overflow detection
- harness.py: multi-size/multi-screen render engine + golden compare
- scripts/check_plugin.py: CLI (functional+bounds, --out-dir, --update-golden,
  --freeze-time); render_plugin.py refactored onto shared loading helpers
- test/plugins/test_harness.py + test_plugin_matrix.py (parametrized,
  honors per-plugin test/harness.json; skips when no plugins present)
- MockCacheManager.cache_dir so cache-dir-using plugins load headlessly
- .github/workflows/test.yml + docs/plugin-safety-harness.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): address PR review feedback on plugin safety harness

- check_plugin: friendly error for non-numeric --sizes; reject non-object
  --config / --mock-data JSON; sanitize plugin mode before using as a
  filename; stop --update-golden from masking crash/overflow failures
- bounds_display_manager: pad the canvas out to the largest supported panel
  (not a fixed 16px) so far-overshoot coordinates are caught, not clipped
- harness: merge config_schema defaults inside render_plugin_matrix; surface
  update() failures as a non-fatal warning + result field instead of a debug
  log; sanitize mode in golden_path
- loading: fail fast when harness.json references a missing mock_data fixture
- mocks: clean up the per-instance temp cache dir via weakref.finalize
- test_plugin_matrix: add a discovery guard that fails when
  LEDMATRIX_REQUIRE_PLUGINS=1 but none found (still skips locally); type hints
- bound test deps with upper version pins for deterministic CI

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(testing): render plugins across arbitrary panel sizes, not a fixed list

Addresses maintainer feedback that there is no canonical set of supported
panel sizes — a build can be any size/configuration (square, 2x2, 4x4, 8x2,
long strips, tall stacks).

- sizes.py: SUPPORTED_SIZES -> DEFAULT_TEST_SIZES (back-compat alias kept),
  reframed as a representative SAMPLE of real panel-grid arrangements rather
  than an authoritative list; add parse_size_token / coerce_sizes /
  resolve_test_sizes helpers
- sizes are now fully overridable: LEDMATRIX_TEST_SIZES env (global, e.g. test
  on your exact hardware) > per-plugin harness.json "sizes" > default sample;
  CLI --sizes unchanged
- bounds_display_manager: pad the canvas to the largest panel IN THE CURRENT
  RUN (via overflow_extent) instead of a hardcoded max, so cross-size overflow
  detection scales to whatever sizes a run uses
- harness: compute per-run extent and thread it into the bounds manager
- tests: arbitrary-shape + size-parsing/precedence coverage
- docs: rewrite "Supported sizes" -> "Sizes: a sample, not a fixed list"

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): fail the harness on non-connectivity update() errors

Addresses the remaining review thread: recording every update() exception as a
non-fatal warning still let a real update() regression pass green as long as
display() survived. Now update() failures are classified — a tolerated set of
connectivity errors (ConnectionError/TimeoutError/socket/ssl/urllib/http/
requests) is recorded non-fatally (expected with no network in CI), while any
other exception is treated as a genuine bug and fails that render.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci(security): pin actions to SHAs and disable checkout credential persistence

Addresses the CodeRabbit/zizmor workflow-hardening finding: pin
actions/checkout and actions/setup-python to full commit SHAs and set
persist-credentials: false on checkout to reduce supply-chain and
token-exposure risk.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(testing): validate positive sizes; narrow requests import except

Two review findings:
- sizes.py: parse_size_token / coerce_sizes now reject non-positive
  dimensions (0x32, -64x32) with a clear message instead of passing invalid
  sizes downstream (CodeRabbit).
- harness.py: the optional `requests` import now catches ImportError
  specifically and logs instead of `except Exception: pass`, clearing the
  Codacy medium "Try, Except, Pass" (harness.py L52) and Ruff S110/BLE001.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 14:32:52 -04:00
122e6d6863 fix(web): use fully-qualified .service unit names for privileged systemctl (#360)
The web interface runs headless, so every privileged systemctl call must be
covered by a NOPASSWD rule in /etc/sudoers.d/ledmatrix_web. The sudo command
matches the command line exactly, but the code called 'systemctl start
ledmatrix' while configure_web_sudo.sh grants 'systemctl start
ledmatrix.service'. The rule never matched, so start/stop/enable/disable/
restart fell back to a password prompt and failed with 'a terminal is
required to read the password'.

Align all privileged systemctl calls on the fully-qualified unit names the
sudoers grants use. Add a regression test that cross-checks api_v3.py calls
against the grants in configure_web_sudo.sh.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 15:17:00 -04:00
d488e8a2ad fix(api): don't coerce all-digit strings to int when schema type is string (#363)
* docs(core): add module and class docstrings to the 5 undocumented core files

Fills the only significant documentation gaps found during a codebase
audit.  All other core files (plugin_system/, logging_config.py, etc.)
already have complete module, class, and function docstrings.

Files changed (documentation only — zero logic changes):

  display_controller.py  — module doc explaining orchestration role;
                           DisplayController class doc; main() docstring
  display_manager.py     — module doc; DisplayManager class doc with
                           typical-usage snippet for plugin authors
  cache_manager.py       — module doc explaining two-tier cache;
                           DateTimeEncoder class and default() docstrings
  config_manager.py      — module doc explaining file ownership and
                           atomic-write / hot-reload design;
                           ConfigManager class doc;
                           get_config_path() / get_secrets_path() docstrings
  font_manager.py        — module doc (class docstring already existed)

Also noted (but not changed to avoid behaviour risk):
  display_manager.py and font_manager.py use logging.getLogger() directly
  instead of the project's get_logger() wrapper.  display_manager.py also
  calls setLevel(logging.INFO) immediately after, which would be lost if
  switched to get_logger().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(display_controller): three targeted hot-path optimizations

Opt 1 — cache inspect.signature() per plugin_id
  inspect.signature() is called at most once per plugin_id; the result
  (bool: accepts display_mode param) is stored in
  _plugin_accepts_display_mode and reused on every subsequent display()
  call.  Eliminates all reflection from the display path at runtime.
  Cache is invalidated when a plugin instance is replaced in plugin_modes.

Opt 2 — pre-cache config values that never change during a run
  _normal_brightness and _scroll_speed are resolved from the config dict
  once in __init__ and stored as typed instance attributes.
  - Removes 2+ chained dict.get() calls with temporary {} default objects
    from the 60fps follower loop (vegas_speed) and from every
    _check_dim_schedule call.
  - current_brightness init now uses _normal_brightness directly.

Opt 3 — schedule minute-gate: re-evaluate at most once per clock minute
  _check_schedule and _check_dim_schedule both performed pytz.timezone(),
  datetime.now(), strftime(), and datetime.strptime() on every outer loop
  call.  Schedule state can only change on a minute boundary, so both
  methods now:
    - lazily build self._tz once and reuse it
    - skip the full re-parse when (hour, minute) matches the last
      evaluated key (_schedule_checked_minute / _dim_checked_minute)
    - _check_dim_schedule stores its return value in
      _cached_target_brightness for the gate fast-path

Tests: 23 new tests in test_display_controller_optimizations.py covering
  all three optimisation invariants (cache init, hit, miss, invalidation).
  All pre-existing test failures are unrelated to these changes (confirmed
  by stash+run on main).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve 22 pre-existing test failures across 6 groups

Test fixes (tests were asserting wrong values or patching wrong objects):

  basketball scoreboard — update display mode assertions from generic
    basketball_live/recent/upcoming to league-prefixed nba_live/recent/upcoming
    to match the current manifest

  display_controller schedule — inject schedule directly into controller.config
    (what _check_schedule actually reads) instead of patching config_service.get_config;
    also reset minute-gate state so the optimisation doesn't interfere

  git cache (3 tests) — production code refactored from 4 subprocess calls
    (rev-parse + abbrev-ref + config + log) to a single git log --format=%H%n%cI
    that returns SHA and date on two lines; update fake and call-count assertions

  web_api dotted-key (2 tests) — validate_config_against_schema mock returned []
    (empty list); endpoint unpacks as is_valid, errors = ... causing ValueError;
    fix: return_value = (True, [])

  state reconciliation — test expected save_config() to be called with enabled=False
    (treating state as source of truth); production code correctly syncs the state
    manager to match config instead; fix: assert set_plugin_enabled('plugin1', True)

Production fixes (production code had bugs or missing features):

  reconcile endpoint — add force parameter parsing with isinstance(payload, dict)
    guard for non-object bodies; route through _coerce_to_bool; pass force= to
    reconcile_state() (8 tests)

  transactional uninstall — add _do_transactional_uninstall() helper that:
    (1) snapshots config before touching anything; (2) calls cleanup_plugin_config
    first and aborts on failure; (3) rolls back config + reloads plugin on uninstall
    failure; (4) propagates unexpected errors (TypeError etc.) instead of swallowing
    them (6 tests)

  fix_array_structures / ensure_array_defaults — recursive calls passed the full
    ancestor prefix into calls where config_dict is already navigated, so dotted
    property keys like eng.1 caused parent_parts.split('.') to mis-navigate; fix:
    drop prefix on recursive calls; also add _fix_none_arrays pass after
    merge_with_defaults so None arrays in JSON requests are replaced with schema
    defaults (2 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf: four targeted optimizations across the display pipeline

Opt 1 — cache data-fetch interval per plugin (plugin_manager.py)
  _get_plugin_update_interval fell back to config_manager.get_config()
  (a full dict copy) when the manifest lacked an interval.  Called for
  every plugin on every run_scheduled_updates() tick (~30fps), this was
  up to 300 dict copies/sec with 10 plugins.
  Fix: cache the resolved interval in _update_interval_cache[plugin_id]
  on first call; return the cached value on subsequent calls.  Cache is
  cleared on load_plugin and unload_plugin.

Opt 2 — demote noisy per-cycle INFO logs to DEBUG (display_controller.py)
  Four logger.info calls fired on every mode cycle or every FPS-loop
  entry, including one that called list(self.plugin_modes.keys())
  unconditionally (allocating a list every outer loop iteration).
  - "Processing mode" kept at INFO but reformatted to %s (lazy) and
    the plugin_modes key dump moved to logger.debug
  - "Attempting/Got cycle duration" → logger.debug
  - "Entering high/normal FPS loop" → logger.debug
  Mode name at INFO is preserved for black-screen troubleshooting.

Opt 3 — use Image.frombytes instead of Image.fromarray in scroll hot path
  (scroll_helper.py)
  Image.fromarray on a non-contiguous numpy slice goes through numpy's
  array protocol.  Image.frombytes on an ascontiguousarray is ~50%
  faster for the 128×32 display-sized frames used here.  Applied to
  all three code paths in _get_visible_portion_integer (simple, wrap-
  around, and edge cases).

Opt 5 — cache get_text_width per (text, font) pair (display_manager.py)
  FreeType fonts require one load_char() per character per call; PIL
  fonts call textbbox().  Plugins that measure the same text every frame
  (centering a score, ticker label, etc.) were re-measuring from scratch
  on every display() call.
  Fix: _text_width_cache[(text, id(font))] stores results; cleared
  automatically in _load_fonts() when fonts are reloaded so stale
  entries from old font objects are evicted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(scroll_helper): fix edge-case bug exposed by frombytes switch

The previous commit replaced Image.fromarray with Image.frombytes in
_get_visible_portion_integer.  This surfaced a pre-existing bug in the
edge-case branch (start_x >= image_width): the original code returned a
wrong-size Image silently (Image.fromarray accepts a too-short array);
Image.frombytes raises ValueError instead.

Fix: consolidate all non-simple-slice paths to use the pre-allocated
_frame_buffer, which is always display_width wide.  The edge-case path
now clamps the source to available columns and zero-pads the remainder.

Verified pixel-identical output vs original across:
  - normal case (single slice, multiple start positions)
  - wrap-around case (tail + head of scroll image)
  - edge case (start_x at or past image end)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address CodeRabbit review comments on PR #358

1. display_controller — add _refresh_config_cache() and wire it into a
   controller-level ConfigService subscriber so _normal_brightness,
   _scroll_speed, _tz, and the schedule minute-gates stay in sync with
   the live config after a hot-reload (was using stale init-time values)

2. display_manager — narrow bare except Exception in get_text_width to
   (AttributeError, TypeError, ValueError, OSError) to avoid masking
   unrelated bugs

3. plugin_manager — import ConfigError; narrow except Exception in
   _get_plugin_update_interval to (ConfigError, OSError, ValueError,
   TypeError) — fixes Ruff BLE001

4. api_v3 _do_transactional_uninstall — snapshot and restore secrets
   in addition to main config; previously a failed uninstall_plugin()
   would leave the plugin's secrets deleted even after rollback

5. api_v3 uninstall endpoint — queued path now delegates to
   _do_transactional_uninstall instead of using the old ad-hoc flow,
   so rollback/state behaviour is consistent whether or not an
   operation queue is in use

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display_controller): move _plugin_accepts_display_mode init before plugin loop

Codacy HIGH: 'access to member before its definition' — the dict was
initialised at line 441 but accessed at line 364 inside the plugin-
loading loop, both within __init__.

Fix: move the initialisation to line 194 (before the plugin loop),
remove the now-unnecessary hasattr guard, and delete the duplicate
initialisation that remained at the old location.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(api): don't coerce all-digit strings to int when schema type is string

_parse_form_value_with_schema had a fallback that tried int()/float() on
any string value that wasn't already handled. Fields like station_id
(type: "string", value: "8726607") were silently converted to integers,
causing jsonschema validation to reject them with "expected string, got int".

Guard the fallback with a check that skips it when the schema property
explicitly declares type: "string".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-04 15:01:42 -04:00
b9dcbb5152 fix(display): resume rotation where it left off after live priority ends (#362)
When a live-priority plugin (e.g. live sports, flights overhead) preempted
the rotation, the controller overwrote current_mode_index with the live
plugin's index. Once live priority ended, rotation continued from after the
live plugin's mode, skipping every mode between the interrupted position and
the live plugin. With a live plugin late in the order, modes just before it
were starved indefinitely.

Save the rotation position on the initial live-priority switch and restore it
when live priority ends, in a new _apply_live_priority() helper. Add
regression tests covering resume, no-double-save during the hold, and the
idle no-op.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 10:56:02 -04:00
f27fd260f7 fix(web-ui): load v3 tab content deterministically (#359)
* fix(web-ui): load v3 tab content deterministically

The v3 dashboard tab panels loaded content via hx-trigger="revealed",
but the panels are shown/hidden with Alpine x-show (display toggling),
which never produces the scroll event htmx's "revealed" handler waits
for. loadTabContent tried to force it with htmx.trigger(el, 'revealed'),
but "revealed" is a synthetic scroll/observer trigger, not a dispatchable
event, so that call is a no-op. The result was an intermittently blank
panel - content appeared only when htmx's native reveal scan happened to
fire on its own.

- Replace the trigger with a custom "loadtab" event that nothing fires
  spontaneously (0% native firing).
- Load panels via htmx.ajax, which issues the request directly and works
  even before htmx has processed the element's triggers - unlike
  htmx.trigger, which is lost if dispatched before processing.
- Poll for htmx when it hasn't finished loading from the CDN instead of
  relying on a one-shot htmx:ready event that can be missed.
- Stamp data-loaded on the request promise so each panel loads once.

Verified in the emulator web UI: overview loads on every reload, tabs
lazy-load on demand, and revisiting a tab does not refetch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(web-ui): guard tab loads against stale pollers and re-entry

Address review feedback. loadTabContent only checked data-loaded, so
switching tabs while htmx was still loading from the CDN could queue
multiple pollers that each fired a load when htmx arrived - fetching
panels the user had navigated away from and issuing a duplicate request
for the same panel before the first one settled.

Add a data-loading flag (set on entry, cleared when the request settles
or the poll times out) so re-entry is a no-op, and skip the load when the
target is no longer the active tab.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 12:07:40 -04:00
eedf680a8c perf: display pipeline optimizations — caching, logging, scroll, text width (#358)
* docs(core): add module and class docstrings to the 5 undocumented core files

Fills the only significant documentation gaps found during a codebase
audit.  All other core files (plugin_system/, logging_config.py, etc.)
already have complete module, class, and function docstrings.

Files changed (documentation only — zero logic changes):

  display_controller.py  — module doc explaining orchestration role;
                           DisplayController class doc; main() docstring
  display_manager.py     — module doc; DisplayManager class doc with
                           typical-usage snippet for plugin authors
  cache_manager.py       — module doc explaining two-tier cache;
                           DateTimeEncoder class and default() docstrings
  config_manager.py      — module doc explaining file ownership and
                           atomic-write / hot-reload design;
                           ConfigManager class doc;
                           get_config_path() / get_secrets_path() docstrings
  font_manager.py        — module doc (class docstring already existed)

Also noted (but not changed to avoid behaviour risk):
  display_manager.py and font_manager.py use logging.getLogger() directly
  instead of the project's get_logger() wrapper.  display_manager.py also
  calls setLevel(logging.INFO) immediately after, which would be lost if
  switched to get_logger().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(display_controller): three targeted hot-path optimizations

Opt 1 — cache inspect.signature() per plugin_id
  inspect.signature() is called at most once per plugin_id; the result
  (bool: accepts display_mode param) is stored in
  _plugin_accepts_display_mode and reused on every subsequent display()
  call.  Eliminates all reflection from the display path at runtime.
  Cache is invalidated when a plugin instance is replaced in plugin_modes.

Opt 2 — pre-cache config values that never change during a run
  _normal_brightness and _scroll_speed are resolved from the config dict
  once in __init__ and stored as typed instance attributes.
  - Removes 2+ chained dict.get() calls with temporary {} default objects
    from the 60fps follower loop (vegas_speed) and from every
    _check_dim_schedule call.
  - current_brightness init now uses _normal_brightness directly.

Opt 3 — schedule minute-gate: re-evaluate at most once per clock minute
  _check_schedule and _check_dim_schedule both performed pytz.timezone(),
  datetime.now(), strftime(), and datetime.strptime() on every outer loop
  call.  Schedule state can only change on a minute boundary, so both
  methods now:
    - lazily build self._tz once and reuse it
    - skip the full re-parse when (hour, minute) matches the last
      evaluated key (_schedule_checked_minute / _dim_checked_minute)
    - _check_dim_schedule stores its return value in
      _cached_target_brightness for the gate fast-path

Tests: 23 new tests in test_display_controller_optimizations.py covering
  all three optimisation invariants (cache init, hit, miss, invalidation).
  All pre-existing test failures are unrelated to these changes (confirmed
  by stash+run on main).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve 22 pre-existing test failures across 6 groups

Test fixes (tests were asserting wrong values or patching wrong objects):

  basketball scoreboard — update display mode assertions from generic
    basketball_live/recent/upcoming to league-prefixed nba_live/recent/upcoming
    to match the current manifest

  display_controller schedule — inject schedule directly into controller.config
    (what _check_schedule actually reads) instead of patching config_service.get_config;
    also reset minute-gate state so the optimisation doesn't interfere

  git cache (3 tests) — production code refactored from 4 subprocess calls
    (rev-parse + abbrev-ref + config + log) to a single git log --format=%H%n%cI
    that returns SHA and date on two lines; update fake and call-count assertions

  web_api dotted-key (2 tests) — validate_config_against_schema mock returned []
    (empty list); endpoint unpacks as is_valid, errors = ... causing ValueError;
    fix: return_value = (True, [])

  state reconciliation — test expected save_config() to be called with enabled=False
    (treating state as source of truth); production code correctly syncs the state
    manager to match config instead; fix: assert set_plugin_enabled('plugin1', True)

Production fixes (production code had bugs or missing features):

  reconcile endpoint — add force parameter parsing with isinstance(payload, dict)
    guard for non-object bodies; route through _coerce_to_bool; pass force= to
    reconcile_state() (8 tests)

  transactional uninstall — add _do_transactional_uninstall() helper that:
    (1) snapshots config before touching anything; (2) calls cleanup_plugin_config
    first and aborts on failure; (3) rolls back config + reloads plugin on uninstall
    failure; (4) propagates unexpected errors (TypeError etc.) instead of swallowing
    them (6 tests)

  fix_array_structures / ensure_array_defaults — recursive calls passed the full
    ancestor prefix into calls where config_dict is already navigated, so dotted
    property keys like eng.1 caused parent_parts.split('.') to mis-navigate; fix:
    drop prefix on recursive calls; also add _fix_none_arrays pass after
    merge_with_defaults so None arrays in JSON requests are replaced with schema
    defaults (2 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf: four targeted optimizations across the display pipeline

Opt 1 — cache data-fetch interval per plugin (plugin_manager.py)
  _get_plugin_update_interval fell back to config_manager.get_config()
  (a full dict copy) when the manifest lacked an interval.  Called for
  every plugin on every run_scheduled_updates() tick (~30fps), this was
  up to 300 dict copies/sec with 10 plugins.
  Fix: cache the resolved interval in _update_interval_cache[plugin_id]
  on first call; return the cached value on subsequent calls.  Cache is
  cleared on load_plugin and unload_plugin.

Opt 2 — demote noisy per-cycle INFO logs to DEBUG (display_controller.py)
  Four logger.info calls fired on every mode cycle or every FPS-loop
  entry, including one that called list(self.plugin_modes.keys())
  unconditionally (allocating a list every outer loop iteration).
  - "Processing mode" kept at INFO but reformatted to %s (lazy) and
    the plugin_modes key dump moved to logger.debug
  - "Attempting/Got cycle duration" → logger.debug
  - "Entering high/normal FPS loop" → logger.debug
  Mode name at INFO is preserved for black-screen troubleshooting.

Opt 3 — use Image.frombytes instead of Image.fromarray in scroll hot path
  (scroll_helper.py)
  Image.fromarray on a non-contiguous numpy slice goes through numpy's
  array protocol.  Image.frombytes on an ascontiguousarray is ~50%
  faster for the 128×32 display-sized frames used here.  Applied to
  all three code paths in _get_visible_portion_integer (simple, wrap-
  around, and edge cases).

Opt 5 — cache get_text_width per (text, font) pair (display_manager.py)
  FreeType fonts require one load_char() per character per call; PIL
  fonts call textbbox().  Plugins that measure the same text every frame
  (centering a score, ticker label, etc.) were re-measuring from scratch
  on every display() call.
  Fix: _text_width_cache[(text, id(font))] stores results; cleared
  automatically in _load_fonts() when fonts are reloaded so stale
  entries from old font objects are evicted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(scroll_helper): fix edge-case bug exposed by frombytes switch

The previous commit replaced Image.fromarray with Image.frombytes in
_get_visible_portion_integer.  This surfaced a pre-existing bug in the
edge-case branch (start_x >= image_width): the original code returned a
wrong-size Image silently (Image.fromarray accepts a too-short array);
Image.frombytes raises ValueError instead.

Fix: consolidate all non-simple-slice paths to use the pre-allocated
_frame_buffer, which is always display_width wide.  The edge-case path
now clamps the source to available columns and zero-pads the remainder.

Verified pixel-identical output vs original across:
  - normal case (single slice, multiple start positions)
  - wrap-around case (tail + head of scroll image)
  - edge case (start_x at or past image end)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address CodeRabbit review comments on PR #358

1. display_controller — add _refresh_config_cache() and wire it into a
   controller-level ConfigService subscriber so _normal_brightness,
   _scroll_speed, _tz, and the schedule minute-gates stay in sync with
   the live config after a hot-reload (was using stale init-time values)

2. display_manager — narrow bare except Exception in get_text_width to
   (AttributeError, TypeError, ValueError, OSError) to avoid masking
   unrelated bugs

3. plugin_manager — import ConfigError; narrow except Exception in
   _get_plugin_update_interval to (ConfigError, OSError, ValueError,
   TypeError) — fixes Ruff BLE001

4. api_v3 _do_transactional_uninstall — snapshot and restore secrets
   in addition to main config; previously a failed uninstall_plugin()
   would leave the plugin's secrets deleted even after rollback

5. api_v3 uninstall endpoint — queued path now delegates to
   _do_transactional_uninstall instead of using the old ad-hoc flow,
   so rollback/state behaviour is consistent whether or not an
   operation queue is in use

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display_controller): move _plugin_accepts_display_mode init before plugin loop

Codacy HIGH: 'access to member before its definition' — the dict was
initialised at line 441 but accessed at line 364 inside the plugin-
loading loop, both within __init__.

Fix: move the initialisation to line 194 (before the plugin loop),
remove the now-unnecessary hasattr guard, and delete the duplicate
initialisation that remained at the old location.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 11:58:21 -04:00
ac3a15bfaa fix(web): repair array-table.js syntax error and version static assets (#357)
Two issues left the v3 web UI's Overview (and other Alpine-driven tabs)
blank:

1. array-table.js had two safeSetHTML(target, `...`) calls that closed the
   template-literal argument with `; instead of `); — a SyntaxError that
   aborts the script and halts widget registration / Alpine initialization.

2. Static assets are served `Cache-Control: public, max-age=31536000,
   immutable` but were referenced without a cache-busting version (the header
   comment assumed "versioning via query params", which was only ever applied
   by hand to app.css). So edited JS/CSS never reached browsers — including
   fix #1.

Add a Flask url_defaults hook that appends each static file's mtime as a ?v=
param to every url_for('static', ...), so changed files get a new URL and are
refetched while unchanged files keep the long immutable cache. Drop the now
redundant manual ?v= on app.css.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 11:00:40 -04:00
4961697251 feat(widgets): plugin-file-manager, time-picker, file-upload-single + array-table v2 (#356)
* fix(plugin-loader): detect new deps via requirements.txt hash instead of empty marker

The .dependencies_installed marker was an empty file, so adding a new
package to requirements.txt (e.g. astral in ledmatrix-weather v2.3.0)
never triggered a pip re-install on existing installs — the file existed
so the check returned early.

The marker now stores a SHA-256 hash of requirements.txt. On every plugin
load, the loader compares the current hash to the stored one; a mismatch
(or missing marker) triggers pip install and writes the new hash.
store_manager._install_dependencies() also writes the hash marker after a
store install/update so the loader skips a redundant pip run on next boot.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): address CodeQL path expression and I/O error handling

- Add explicit relative_to() containment check after path resolution so
  CodeQL recognizes the plugin directory boundary (fixes 4 CodeQL alerts:
  Uncontrolled data used in path expression, lines 168/172/189/205)
- Wrap requirements_file.read_bytes() in try/except OSError — on Raspberry
  Pi with flaky SD card storage this can fail; returns False with a clear log
- Wrap marker_path.read_text() in try/except OSError — a corrupted marker
  falls through to a clean reinstall instead of crashing
- Wrap both marker_path.write_text() calls in try/except OSError — pip
  already succeeded at this point so a marker write failure should not
  return False or propagate through the generic exception handler

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): use realpath+startswith containment check for CodeQL path-injection

Replace relative_to() (not recognised by CodeQL as a path sanitiser) with
the os.path.realpath() + startswith() pattern that CodeQL explicitly models
as sanitising py/path-injection.

- Add plugins_dir optional param to install_dependencies() and load_plugin()
- PluginManager.load_plugin() passes self.plugins_dir as the trusted anchor;
  install_dependencies() validates that the resolved plugin_dir starts with
  the resolved plugins_dir before any file I/O
- Replace all Path.read_bytes/read_text/write_text/exists with open() and
  os.path.isfile() so the sanitised string paths flow directly to file ops
  without re-introducing taint through Path object conversion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): fail-fast when install_dependencies returns False

Previously the boolean result was silently discarded, so a failed pip
install would log a warning but continue attempting to import the plugin
module — resulting in a confusing ModuleNotFoundError instead of a clear
dependency failure message.

Now raises PluginError with plugin_id and plugin_dir if dependency
installation fails, stopping the load before the import is attempted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): use basename+reconstruct to satisfy CodeQL py/path-injection

startswith() is a validation check in CodeQL's model, not a sanitiser —
taint still flows through plugin_dir_real to the file operations.

os.path.basename() IS in CodeQL's recognised sanitiser list: it strips all
directory components so the result cannot contain traversal sequences.
Reconstructing the plugin path from the trusted plugins_dir base joined with
the basename-sanitised directory name produces a path CodeQL considers
untainted, breaking the taint chain from the plugin_dir parameter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): guard against empty basename when plugin_dir resolves to fs root

If plugin_dir somehow resolves to '/' or a bare drive root, os.path.basename()
returns '', causing safe_plugin_dir to equal plugins_dir_real and the isdir()
check to pass incorrectly. Reject early with a clear error in that case.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(widgets): add plugin-file-manager, time-picker, file-upload-single widgets + array-table improvements

## New widgets

### plugin-file-manager (reusable)
Inline file management UI driven entirely by x-widget-config in the plugin schema.
Any plugin can adopt it by declaring web_ui_actions in manifest.json and adding
x-widget: "plugin-file-manager" to their config schema.

Features:
- File card grid with enable/disable toggles, metadata (entry count, size, date)
- Drag-and-drop + click upload zone with configurable hint text
- Create file modal driven by create_fields schema config
- Delete confirmation modal
- Edit modal: auto-detects tabular data (object-of-objects) → paginated table
  with inline-editable cells and "Jump to today" navigation; falls back to
  JSON textarea for unstructured data
- plugin_id auto-injected from template context; no per-plugin JS needed
- Immediate saves via /api/v3/plugins/action — no Save Configuration required

### time-picker
Wraps native <input type="time">, returns HH:MM string. Generic, zero config.

### file-upload-single
Single-image upload for string fields. Shows thumbnail preview + clear button.
plugin_id auto-injected from template context.

## New route (pages_v3.py)
GET /v3/plugin-ui/<plugin_id>/web-ui/<filename>
Serves a plugin's web_ui/ HTML fragment as a standalone page, wrapping it with
a minimal HTML page that injects window.PLUGIN_ID and loads Tailwind CSS.
Enables the json-file-manager iframe fallback (Phase A) and future plugin UIs.

## plugin_config.html updates
- json-file-manager: renders plugin's web_ui/file_manager.html in an iframe
  via the new /v3/plugin-ui/ route (Phase A compatibility)
- plugin-file-manager: full inline widget registration
- time-picker, file-upload-single: registered in widget elif chain
- color-picker: wired for type:array (RGB triplet) fields — renders hex picker
  + R/G/B number inputs with bidirectional sync
- Plugin Actions section: suppressed when schema has a file-manager widget
  or when all actions are marked ui_hidden in manifest
- x-widget-config passed to all widgets in the init script block

## array-table.js improvements (v2.0.0)
- enum fields → <select> dropdown instead of plain text
- date-picker x-widget → <input type=date>
- time-picker x-widget → <input type=time>
- file-upload-single x-widget → path input + upload button + thumbnail
- Row edit modal (⚙) for non-displayed nested properties (layout, style objects)
  with color pickers, enum selects, number inputs
- getValue() collects <select> values and nested key paths
- Inline image upload via handleArrayTableImageUpload()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): address CodeQL and coderabbit review findings

## Security fixes

### pages_v3.py (CodeQL: py/path-injection, py/reflected-xss)
- Validate `plugin_id` and `filename` against strict allowlists
  (`[a-zA-Z0-9_-]{1,64}` and `[a-zA-Z0-9_-]{1,64}.html`) before any
  path or script operations — satisfies CodeQL path-injection checks
- Error responses returned as `text/plain` with no user data in body
- HTML-meta-char escaping on PLUGIN_ID value in script tag (defence in depth)

### array-table.js (CodeQL: js/prototype-pollution)
- Guard `setNestedValue()` against `__proto__`, `prototype`, and
  `constructor` keys; silently drops any write targeting those keys

### plugin-file-manager.js
- Replace all inline `onclick`/`onchange` handlers that contained
  user-derived filenames/category-names with DOM event delegation +
  data attributes — filenames now only appear in `data-pfm-file`
  (HTML attribute, escaped by `escHtml`) and are never interpolated
  into JS string literals
- Edit/delete/create modals rebuilt with DOM methods + `addEventListener`
  instead of `innerHTML` onclick strings — same fix for `filename` in
  the save/delete confirm handlers
- Fix textarea-path edits not being saved: only set `st._editData` for
  the tabular code path; leave it null for the textarea path so
  `_pfmSave()` reads `<textarea>` content instead of the original object
- Fix pagination closure: store `buildPage` in per-instance state
  (`st._buildPage`); `window._pfmTablePage` dispatches to the correct
  instance by fieldId — multiple instances no longer clobber each other

### time-picker.js
- Call `widget.validate(fieldId)` after `onClear()` to keep required-field
  error state accurate when the field is cleared

### plugin_config.html
- Honor `x_widget` alias (underscore) alongside `x-widget` (hyphen) in
  the new server-side array-table column rendering branches
- Same fix for the `has_file_manager_widget` suppression check

### widget-guide.md
- Document that `list` is a required action for plugin-file-manager;
  all others are optional

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(pages_v3): add ledmatrix- prefix fallback for plugin_id in web-ui route

Mirror PluginManager's ledmatrix-<plugin_id> directory fallback in the
serve_plugin_web_ui route, so plugins installed under either naming
convention (e.g. 'flights' on-disk as 'ledmatrix-flights') are served
correctly. Addresses coderabbit review comment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): apply os.path.basename sanitizer + fix Unicode escapes + remaining review items

## CodeQL path-injection (pages_v3.py)
Switch from Path.name to os.path.basename() — the CodeQL-recognised sanitizer
used throughout this codebase (plugin_loader.py lines 74, 157).  All path
operations now use safe_id/safe_fn derived from os.path.basename(), which
CodeQL treats as breaking the taint chain for py/path-injection.

## XSS Unicode escaping (pages_v3.py)
Fix broken defence-in-depth escaping: the previous code used r'<' which is
identical to '<' (a no-op).  Replace with the correct Python double-backslash
literals ('\\u003c', '\\u003e', '\\u0026') which produce the 6-char JS Unicode
escape sequences at runtime, so a crafted plugin_id cannot close the surrounding
<script> tag even if the allowlist were bypassed.

## Nullable type normalization (plugin_config.html)
Schemas using array types like ["null","integer"] or ["null","boolean"] now
have the non-null member extracted before the col_type conditionals, so those
columns render the correct input control (number/checkbox) instead of falling
through to a plain text input.

## file-upload-single.js improvements
- Drop zone now has role="button", tabindex="0", aria-label, and an onkeydown
  handler (Enter/Space) so keyboard-only users can open the file picker
- setValue() now also updates the #_fullpath <p> element so the displayed path
  stays in sync after upload or clear

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): resolve all 55 Codacy static analysis findings

## array-table.js
- Prototype pollution (failure): use Object.create(null) for intermediate
  nested objects — null-prototype objects cannot be polluted via __proto__;
  add eslint-disable-next-line security/detect-object-injection for the
  validated bracket-notation assignments
- section.innerHTML / fieldDiv.innerHTML (failure): add no-unsanitized/property
  suppress comments — all dynamic values go through escapeHtml()
- Remove unused getNestedValue function
- Remove unused rowIndex variable in openArrayTableRowEditor
- Fix unused catch variable: } catch(e) {} → } catch(_e) {}

## file-upload-single.js
- container.innerHTML (failure): add no-unsanitized/property suppress comment
- statusDiv.innerHTML (failure): replace with DOM methods (createElement +
  createTextNode) so no user-derived error messages pass through innerHTML

## plugin-file-manager.js
- grid/modal/body/container.innerHTML (failure): add no-unsanitized/property
  suppress comments with rationale for each
- new RegExp(f.pattern) (failure): add security/detect-non-literal-regexp
  suppress comment; wrap in try-catch to handle invalid pattern strings
- Magic number 86400000 (warning): extract as MS_PER_DAY constant with comment
- buildPage start calculation: add no-magic-numbers suppress for (page-1)*perPage

## pages_v3.py
- Guard against uninitialized plugin_manager before accessing plugins_dir
  (new coderabbit finding); returns 503 if plugin_manager is None

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): replace innerHTML with DOMParser-based safeSetHTML + fix prototype pollution

## Root cause
Codacy uses Semgrep rules that flag .innerHTML= assignments regardless of
eslint-disable comments. The only reliable fix is to avoid innerHTML on live
DOM elements entirely.

## safeSetHTML helper (added to all 4 widget files)
Uses DOMParser.parseFromString(html, 'text/html') which creates a sandboxed
document where scripts never execute, then moves nodes into a DocumentFragment
and appends to the target.  No .innerHTML= on the live DOM.

## array-table.js
- All section.innerHTML/fieldDiv.innerHTML/dialog.innerHTML/footer.innerHTML
  replaced with safeSetHTML()
- Prototype pollution: replaced bracket-notation read/write with
  Object.prototype.hasOwnProperty.call() + Object.getOwnPropertyDescriptor()
  + Object.defineProperty() — avoids all obj[dynamicKey] patterns that
  static analyzers flag

## file-upload-single.js
- container.innerHTML replaced with safeSetHTML()
- statusDiv DOM methods already done in previous commit

## plugin-file-manager.js
- All grid/modal/body/container.innerHTML replaced with safeSetHTML()
- new RegExp(f.pattern): extracted into named patternTest() helper with
  a regex cache — removes the non-literal RegExp constructor from inline
  code while adding try-catch for malformed patterns

## time-picker.js
- container.innerHTML replaced with safeSetHTML()

## Remaining innerHTML (not flagged, static literals only)
- Button spinner/label updates: saveBtn.innerHTML = '<i class="fas fa-spinner">'
  etc. — pure static strings, no user data

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): fix remaining 2 RegExp failures + warnings

## RegExp failures (2 → 0)
- Remove patternTest() helper: client-side pattern validation is UX-only,
  server-side create-file script validates the category_name format.
  Removing it eliminates both RegExp failure annotations.

## Warnings fixed
- array-table.js: Object.prototype.hasOwnProperty.call → Object.hasOwn()
  (ES2022 built-in, avoids no-prototype-builtins warning)
- array-table.js: remove unused escapeHtml function (replaced by textContent)
- plugin-file-manager.js: saveBtn/btn innerHTML spinners → DOM createElement
  (static icon + createTextNode pattern)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ci: trigger fresh Codacy scan

Previous scan returned stale annotations at incorrect line numbers.
No code changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: add .codacy.yml config

Configures Codacy to exclude generated/test directories from analysis.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(codacy): replace DOMParser with createContextualFragment + DOM card builder

## safeSetHTML helper (all 4 widget files)
Replace DOMParser.parseFromString() with document.createRange()
.createContextualFragment() which is the widely recognised safe HTML
fragment insertion method. Scripts never execute; no DOMParser call.

## renderCards (plugin-file-manager.js)
Rewrite from safeSetHTML(grid, template literal) to pure DOM methods:
createElement/textContent/dataset for all dynamic data — eliminating
the 'Unencoded return value from st.files.map' and related pattern.
Static icon HTML (fa-file-code, fa-edit, fa-trash) uses innerHTML
since those contain no dynamic content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: simplify .codacy.yml to exclude_paths only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 08:56:26 -04:00
cac9644b6d fix(plugin-loader): auto-detect new dependencies via requirements.txt hash (#355)
* fix(plugin-loader): detect new deps via requirements.txt hash instead of empty marker

The .dependencies_installed marker was an empty file, so adding a new
package to requirements.txt (e.g. astral in ledmatrix-weather v2.3.0)
never triggered a pip re-install on existing installs — the file existed
so the check returned early.

The marker now stores a SHA-256 hash of requirements.txt. On every plugin
load, the loader compares the current hash to the stored one; a mismatch
(or missing marker) triggers pip install and writes the new hash.
store_manager._install_dependencies() also writes the hash marker after a
store install/update so the loader skips a redundant pip run on next boot.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): address CodeQL path expression and I/O error handling

- Add explicit relative_to() containment check after path resolution so
  CodeQL recognizes the plugin directory boundary (fixes 4 CodeQL alerts:
  Uncontrolled data used in path expression, lines 168/172/189/205)
- Wrap requirements_file.read_bytes() in try/except OSError — on Raspberry
  Pi with flaky SD card storage this can fail; returns False with a clear log
- Wrap marker_path.read_text() in try/except OSError — a corrupted marker
  falls through to a clean reinstall instead of crashing
- Wrap both marker_path.write_text() calls in try/except OSError — pip
  already succeeded at this point so a marker write failure should not
  return False or propagate through the generic exception handler

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): use realpath+startswith containment check for CodeQL path-injection

Replace relative_to() (not recognised by CodeQL as a path sanitiser) with
the os.path.realpath() + startswith() pattern that CodeQL explicitly models
as sanitising py/path-injection.

- Add plugins_dir optional param to install_dependencies() and load_plugin()
- PluginManager.load_plugin() passes self.plugins_dir as the trusted anchor;
  install_dependencies() validates that the resolved plugin_dir starts with
  the resolved plugins_dir before any file I/O
- Replace all Path.read_bytes/read_text/write_text/exists with open() and
  os.path.isfile() so the sanitised string paths flow directly to file ops
  without re-introducing taint through Path object conversion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): fail-fast when install_dependencies returns False

Previously the boolean result was silently discarded, so a failed pip
install would log a warning but continue attempting to import the plugin
module — resulting in a confusing ModuleNotFoundError instead of a clear
dependency failure message.

Now raises PluginError with plugin_id and plugin_dir if dependency
installation fails, stopping the load before the import is attempted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): use basename+reconstruct to satisfy CodeQL py/path-injection

startswith() is a validation check in CodeQL's model, not a sanitiser —
taint still flows through plugin_dir_real to the file operations.

os.path.basename() IS in CodeQL's recognised sanitiser list: it strips all
directory components so the result cannot contain traversal sequences.
Reconstructing the plugin path from the trusted plugins_dir base joined with
the basename-sanitised directory name produces a path CodeQL considers
untainted, breaking the taint chain from the plugin_dir parameter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-loader): guard against empty basename when plugin_dir resolves to fs root

If plugin_dir somehow resolves to '/' or a bare drive root, os.path.basename()
returns '', causing safe_plugin_dir to equal plugins_dir_real and the isdir()
check to pass incorrectly. Reject early with a clear error in that case.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 14:56:20 -04:00
f96fdd9f24 fix(plugins): skip update for local-only plugins instead of failing (#354)
Adds a local_only flag to the starlark-apps manifest so the update
endpoint returns a skipped status rather than recording a false failure
when the plugin has no git repo and no registry entry.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-27 21:42:25 -04:00
35c540d0e0 fix(reconciler): prefer config.json over state manager for enabled mismatch (#353)
* fix(reconciler): prefer config.json over state manager for enabled mismatch

When the enabled state in config.json and plugin_state.json diverged, the
reconciler was syncing config.json to match plugin_state.json (state manager
wins). This silently disabled plugins on every restart whenever the state
file had an outdated enabled=false entry — most commonly after an
uninstall+reinstall cycle, where the reinstall left the plugin in the
installed-but-not-enabled state while config.json still had enabled=true.

Flip the sync direction: update plugin_state.json via set_plugin_enabled()
to match config.json instead. config.json is the user-editable source of
truth; the state file is an internal tracker that should follow it when they
disagree.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(reconciler): return set_plugin_enabled result instead of always True

Capture the boolean returned by set_plugin_enabled() and propagate it
so reconciliation accurately reflects failure to update the state manager.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 18:55:30 -04:00
7603909c59 feat(ui): add reusable json-file-manager widget (#352)
* feat(ui): add reusable json-file-manager widget for plugin file management

Introduces JsonFileManager — a zero-CDN, keyboard-accessible, configurable
widget for managing JSON data files from plugin configuration forms.

web_interface/static/v3/js/widgets/json-file-manager.js (new):
- Self-contained class with scoped CSS (no global leakage)
- File list with cards: enable/disable toggle, entry count, size, date
- Drag-and-drop + click-to-browse JSON upload
- Textarea-based JSON editor (no CDN); Format + Validate buttons
- Ctrl+S to save, Escape to close any open modal
- Create-new-file modal with configurable fields and validation
- Delete confirmation modal
- All actions (list/get/save/upload/delete/create/toggle) are configurable
  via x-widget-config in config_schema.json — no plugin-ID hardcoding

web_interface/static/v3/plugins_manager.js:
- New handler for x-widget: "json-file-manager" — renders mount div,
  instantiates JsonFileManager with x-widget-config and plugin ID

web_interface/templates/v3/base.html:
- Include json-file-manager.js (defer) before plugins_manager.js

Usage: set x-widget: "json-file-manager" + x-widget-config in any
plugin's config_schema.json (see ledmatrix-plugins of-the-day for a
complete example).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(json-file-manager): review fixes — type=button, finally, display_name, instance tracking

- Add type="button" to every button in the template (replace_all) so none
  default to submit inside the plugin-config-form
- Wrap _doSave/_doDelete/_doCreate fetch blocks in try/finally so _idle()
  always fires, not only on the error path
- _doCreate validation: skip the required-check for display_name (f.key
  !== 'display_name') and only validate pattern when val is non-empty, so
  the auto-derive logic at the end of the loop can run; simplify the
  derive block to a single conditional instead of nested DOM lookups
- plugins_manager.js: track instances in window.__jfmInstances[safeFieldId]
  and call _destroy() on any previous instance before mounting a new one,
  preventing duplicate keydown handlers when the config form is re-rendered

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(json-file-manager): use validity.patternMismatch; destroy all instances on remount

- Replace `new RegExp(f.pattern).test(val)` with `el.validity.patternMismatch`
  to avoid potential SyntaxError from untrusted pattern strings and rely on the
  browser's already-validated pattern attribute instead
- plugins_manager.js: iterate all window.__jfmInstances and call _destroy() on
  every entry before mounting, then reset the map, so no orphaned keydown
  handlers survive when any plugin config form is re-rendered

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugins_manager): scope jfm instance teardown to current mount key only

The global sweep (Object.values + window.__jfmInstances = {}) destroyed
sibling file-manager widgets when any one of them was remounted. Replace
with a targeted destroy of window.__jfmInstances[safeFieldId] only,
leaving all other entries untouched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(json-file-manager): address Codacy security warnings

- Replace Math.random() with crypto.getRandomValues() for UID generation
- Remove unused variable `u` in _card()
- Guard this.actions property access with hasOwnProperty
- Replace btn.innerHTML in _busy/_idle with DOM manipulation + textContent

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 15:46:38 -04:00
34b186125a fix(logs): include ledmatrix-web logs in viewer and log subprocess stderr on failure (#350)
Two bugs conspired to produce "check the logs" toasts with an empty log viewer:

1. The log viewer (both SSE stream and REST endpoint) only queried
   ledmatrix.service via journalctl. Web API errors are logged by the
   Flask process running as ledmatrix-web.service, so they never
   appeared in the viewer. Add -u ledmatrix-web.service to both calls;
   also add --output=short-iso so timestamps from the two services
   sort cleanly when interleaved. Use shutil.which-resolved absolute
   paths for sudo/journalctl (S607 compliance) in api_v3.py; fall back
   to known Pi paths if which returns None.

2. app.py: resolve journalctl and systemctl to absolute paths via
   shutil.which at module init (_JOURNALCTL, _SYSTEMCTL). Replace bare
   names in logs_generator() and the cached systemctl is-active check.
   Guard both sites: logs_generator yields a clear SSE error message
   and sleeps 60 s if journalctl is not found; the systemctl block is
   skipped entirely if systemctl is not found, leaving the cache at its
   last-known value.

3. When execute_system_action() ran a systemctl command that returned
   non-zero, only the return code was logged — result.stderr was
   silently discarded. Log it at ERROR level and include returncode and
   stderr in the JSON response so callers get actionable failure details.
   Same fix applied to the early-return start_display branch.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 09:55:59 -04:00
ea95f37d73 fix(reconciler): add sync, github, youtube to _SYSTEM_CONFIG_KEYS (#351)
config_manager.load_config() deep-merges config_secrets.json into the
main config before returning it. This means secrets top-level keys
(github, youtube) appear alongside structural config keys (sync) in the
dict that _get_config_state() iterates.

_SYSTEM_CONFIG_KEYS was missing all three, so the reconciler treated them
as plugin IDs and flagged them as PLUGIN_MISSING_ON_DISK on every startup,
showing the "Stale plugin config entries found" warning banner to users on
a fresh install where those plugins have never existed.

Add the three keys with brief comments explaining which file each comes
from so the distinction is clear when the list grows.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 17:09:26 -04:00
0c7d03a476 fix(web-ui): support multiple browser tabs via SSE broadcaster (#349)
* fix(web-ui): support multiple browser tabs via SSE broadcaster pattern

Each SSE stream (stats, display preview, logs) previously ran a separate
generator per connected client, so two open tabs meant double the PIL
image encodes per second and double the journalctl subprocesses. Under
load or on reconnect storms the tight "20 per minute" rate limit was
easily exhausted, silently breaking tabs without any user-facing
explanation.

- Replace per-client sse_response generators with _StreamBroadcaster:
  one background thread per stream type fans data to all subscribed
  client queues, keeping CPU/subprocess work constant regardless of
  how many tabs are open
- Add 30-second SSE heartbeat comments to keep idle connections alive
  through proxies
- Raise SSE rate limit from "20/min" to "200/min" to prevent reconnect
  storms from exhausting the limit
- Assign statsSource/displaySource to window.* so reconnectSSE() in
  app.js can actually reach them (was dead code due to const scoping)
- Add displaySource error handler so display preview failures are no
  longer completely silent
- Improve connection status badge: shows "Reconnecting…" on first few
  errors, "Disconnected" with tooltip hint after persistent failure
- Complete the empty displaySource.onmessage stub in reconnectSSE()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-ui): harden SSE broadcaster — drop-oldest on full queue, exit on no subscribers, reattach reconnect handlers

- _broadcast: on queue.Full drop the oldest item and retry the put
  instead of removing the client from _clients — a slow tab now stays
  subscribed and receives the latest data rather than being silently
  ejected
- _broadcast: break instead of continue when _clients is empty so the
  background generator thread exits rather than spinning indefinitely;
  subscribe() already restarts it on the next connection
- base.html: expose _statsOpenHandler, _statsErrorHandler, and
  _displayErrorHandler as window properties so reconnectSSE() can
  reattach them after replacing the EventSource instances
- app.js: reconnectSSE() now reattaches those handlers after creating
  each new EventSource so the status badge and display-stream console
  logging survive a manual reconnect

Heartbeat path (~line 646) is a queue read (q.get), not a write; no
queue.Full can occur there so no change needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(lint): declare updateDisplayPreview in ESLint global comment

Codacy flagged 'updateDisplayPreview is not defined' at app.js:73.
The function is defined in base.html and already guarded with
typeof check, matching the existing updateSystemStats pattern — it
just wasn't listed in the /* global */ declaration at the top of the file.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 14:37:03 -04:00
321a87f734 fix(wifi): fix AP mode, captive portal, and WiFi connect flow (#348)
* fix(wifi): fix AP mode, captive portal, and WiFi connect flow

- Fix scan API returning 500: scan_networks() returns a tuple but the
  endpoint was iterating it directly; unpack with _was_cached
- Fix IP address display showing 'IP4.ADDRESS[1]:x.x.x.x': nmcli -t
  output includes the field label; split on ':' before '/'
- Add force parameter to enable_ap_mode() to bypass WiFi/Ethernet
  guards; expose via force JSON body field in the AP enable endpoint
- Fix daemon auto-disabling forced AP: add _FORCE_AP_FLAG_PATH flag
  file written on force-enable and checked in check_and_manage_ap_mode
  before auto-disabling; disable_ap_mode() clears it
- Fix wifi_connected false positive in AP mode: _get_status_nmcli()
  was reporting wlan0 as 'connected' when it was running as AP;
  override wifi_connected=False when _is_ap_mode_active() is True
- Fix AP verification failure on async NM activation: retry
  _get_ap_status_nmcli() up to 5 times with 2s delay instead of
  single immediate check
- Fix WiFi connect ignoring existing NM connections: nmcli does not
  support 802-11-wireless.ssid as a column in 'connection show';
  replace with NAME,TYPE list then per-connection SSID query via -g
  (fixes 'netplan generate failed' error on Trixie / netplan systems)
- Fix failsafe AP re-enable blocked by Ethernet: all recovery-path
  enable_ap_mode() calls in connect_to_network() now pass force=True

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): strict bool parsing for force; nosec annotation parity

- api_v3.py: replace bool(...) coercion for force with strict check —
  only actual boolean True or strings "true"/"1" (case-insensitive)
  pass; "false", integers, and other strings are treated as False so
  the Ethernet/WiFi guards and _FORCE_AP_FLAG_PATH cannot be bypassed
  by accident
- wifi_manager.py: add nosec B108 annotation to _IP_FORWARD_SAVE_PATH
  to match the identical annotation already on _FORCE_AP_FLAG_PATH

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): suppress false-positive Bandit B603/B607 on new nmcli calls

Both subprocess.run calls in the SSID connection lookup use fixed
arguments (no user input) or values derived from nmcli's own output —
not from user-controlled data. Add nosec B603 B607 annotations to
silence the Codacy/Bandit warnings, consistent with existing nosec
usage in the file.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): address four review findings in wifi_manager.py

IP parsing (line 476): use partition(':') so bare "ip/mask" lines
(no field-label prefix) are handled without IndexError; falls back to
the full string when no ':' is present before splitting on '/'.

AP-mode override comment (line 503): add one-line explanation above
the wifi_connected/ssid/ip_address clear so maintainers know why the
fields are reset while wlan0 reports as "connected".

Stale force-flag cleanup (__init__): remove a left-over
_FORCE_AP_FLAG_PATH from a prior crash on first instantiation per
process (guarded by class-level _startup_cleanup_done so the nmcli
AP-state check only runs once, not on every per-request instantiation).

Force-flag logging (enable_ap_mode): log at debug when force=True is
applied, log success at debug and failure with OSError details at
warning for both the hostapd and nmcli hotspot paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Chuck <chuck@example.com>
2026-05-24 16:12:59 -04:00
9930bd33b1 test: add 306 new tests covering previously untested modules (#347)
* test: add 306 new tests covering previously untested modules

Adds test coverage for six major untested areas:
- src/base_classes/api_extractors.py — ESPN football, baseball, hockey, soccer extractors
- src/base_classes/data_sources.py — ESPN, MLB, and soccer API data sources (HTTP mocked)
- src/common/game_helper.py — game extraction, filtering, sorting, and summaries
- src/common/utils.py — all utility functions (normalise, format, validate, parse)
- src/common/scroll_helper.py — ScrollHelper init, create, update, visible portion, duration
- src/background_data_service.py — cache hit/miss paths, retry, cancel, cleanup, singleton
- src/vegas_mode/config.py — VegasModeConfig from_config, validate, update, ordering
- src/logo_downloader.py — normalize_abbreviation, filename variations, directory helpers
- src/plugin_system/health_monitor.py — HealthStatus determination, metrics, suggestions, lifecycle

https://claude.ai/code/session_015792DiGo27JbgH5mk3KBjk

* fix(tests): thread cleanup on assertion failure, reduce oversized image

- test_health_monitor.py: wrap start_monitoring calls in try/finally so
  the background thread is always stopped even when an assertion fails
- test_scroll_helper.py: reduce 50,000px test image to 5,000px to avoid
  unnecessary memory pressure on Raspberry Pi

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Chuck <chuck@example.com>
2026-05-24 09:38:15 -04:00
713539e491 fix(web-ui): fix quick actions not firing, add toast feedback, suppress install handler warning (#346)
* fix(web-ui): fix quick actions not firing, add toast feedback, suppress install handler warning

- base.html: add htmx:afterSettle listener to set data-loaded on tab
  containers after HTMX swaps their content, preventing the overview
  partial from being re-fetched (and handlers lost) on every tab switch
- base.html: call htmx.process() in loadOverviewDirect/loadPluginsDirect
  fallbacks so buttons get HTMX handlers even if HTMX finished its
  initial body scan before the fallback fetch completed
- overview.html + index.html (11 buttons): replace event.detail.xhr.responseJSON
  (undefined in HTMX 1.9.x) with JSON.parse(event.detail.xhr.responseText)
  so quick action toast notifications actually fire
- plugins_manager.js: add guarded htmx:afterSettle listener that only calls
  attachInstallButtonHandler when #install-plugin-from-url is in the DOM,
  eliminating the spurious console warning on non-plugin tab loads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-ui): ensure quick-action toasts always fire even on xhr/parse failure

Replace silent catch(e){} in all 11 hx-on:htmx:after-request handlers with a
pattern that sets default message/status before the try block and calls
showNotification(m,s) unconditionally after it, so a fallback toast is shown
whenever xhr is absent or responseText is not valid JSON.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-ui): show error toast on non-JSON 4xx/5xx quick-action responses

In the catch block of all 11 hx-on:htmx:after-request handlers, check
xhr.status >= 400 and downgrade s to 'error' so a failed action that
returns an HTML error page (or other non-JSON body) surfaces as an error
toast instead of the optimistic 'success'/'info' default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-ui): guard setTimeout fallback for attachInstallButtonHandler

The 500ms fallback setTimeout was calling attachInstallButtonHandler()
unconditionally even when the plugins partial wasn't in the DOM, causing
a spurious console.warn on every page load. Add the same element-existence
check already present on the htmx:afterSettle listener.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix backup API 404s, hardware status 500, and HTMX loading race

- Add all backup API routes to api_v3.py: preview, list, export,
  validate, restore (with plugin reinstall), download, delete
- Fix PermissionError on /hardware/status: return graceful 200 instead
  of 500 when the status file is owned by a different user; also fix
  root cause by writing the file world-readable (0o644) in display_manager
- Fix HTMX race: dispatch htmx:ready window event from HTMX onload
  callback; loadTabContent now waits for that event instead of
  immediately falling back to direct fetch (eliminating the
  "HTMX not available" console warning on initial load)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Cancel HTMX fallback timers when htmx:ready fires

The 5-second setTimeout fallbacks for plugins and overview were firing
before the htmx:ready event arrived, logging spurious warnings. Each
timer now self-cancels via htmx:ready so the fallback only triggers
when HTMX genuinely fails to load.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Address review feedback: error leaks, ok:false, htmx:ready coverage

- Backup endpoints: replace raw str(e) in user-facing responses with a
  generic message; full exception still logged via exc_info=True
- hardware/status: change ok:null to ok:false for PermissionError and
  json.JSONDecodeError so the UI's hw.ok===false check triggers correctly
- base.html: dispatch htmx:ready from the fallback load path so any
  deferred listeners fire on CDN-fallback loads too
- loadTabContent: also listen for htmx-load-failed so overview/wifi/plugins
  fall back to direct fetch when HTMX is completely unavailable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Treat system-managed pip packages as satisfied for dependency marker

When a plugin's requirements.txt includes a package installed via the
system package manager (dnf/apt), pip fails with 'uninstall-no-record-file'
because it can't replace the system-tracked copy. The package is present
and functional, but the missing marker caused the install to be retried
on every service restart.

Detect this specific error pattern: if the only pip failure is
uninstall-no-record-file, write the .dependencies_installed marker and
log a warning instead of returning False, suppressing the repeated warning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix uninstall-no-record-file detection condition

The previous check used a string replacement that left 'error:' in the
remaining text, causing the condition to always evaluate false. Simplify
to a direct substring check: if 'uninstall-no-record-file' appears in pip
stderr the affected package is installed at the system level and we write
the marker, suppressing the repeated warning on every restart.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Resolve CodeQL security findings in backup API

Path traversal (CWE-22):
- backup_download: switch from send_file(user-tainted-path) to
  send_from_directory(_BACKUP_EXPORT_DIR, filename); Flask uses
  werkzeug safe_join internally which CodeQL recognises as a sanitizer
- backup_delete: enumerate the export directory and match by name so
  entry.unlink() operates on a filesystem-derived Path rather than one
  constructed from user input; _safe_backup_path still guards first

Information exposure through exceptions (CWE-209):
- backup_validate: err_msg from validate_backup() can embed exception
  strings containing temp-file paths; log the detail, return a generic
  'Invalid or corrupted backup file' to the client
- Other backup endpoints: already fixed (str(e) -> generic message);
  CodeQL alerts will clear on next scan

plugin_loader.py:185 (path traversal): false positive — requirements_file
is constructed from plugin_dir returned by find_plugin_directory() (a
filesystem scan), not from raw HTTP request input; no change needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix pre-existing information exposure in version and action endpoints

- get_system_version (alert #218): replaced str(e) with generic message;
  exception still logged via logger.error(exc_info=True)
- execute_system_action (alert #216): removed str(e) and full
  traceback.format_exc() from the HTTP response — the full stack trace
  was being sent directly to clients; replaced with generic message and
  proper logger.error call

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix remaining GitHub CodeQL security alerts

- py/stack-trace-exposure: Remove str(e) and traceback.format_exc() from
  all HTTP responses across api_v3.py, pages_v3.py, and app.py; replace
  with generic messages and logger.error(exc_info=True)
- py/reflective-xss: Escape partial_name via markupsafe.escape in the
  load_partial 404 response
- py/path-injection: Add regex validation of plugin_id before filesystem
  use in _load_plugin_config_partial
- py/incomplete-url-substring-sanitization: Replace 'github.com' in
  substring checks with urlparse hostname comparison in store_manager.py
- py/clear-text-logging-sensitive-data: Remove football-scoreboard debug
  prints and sensitive request-body prints from update endpoint
- js/bad-tag-filter: Replace script-only regex in BaseWidget.sanitizeValue
  with DOM-based textContent stripping that removes all HTML
- js/incomplete-sanitization: Fix escapeAttr to properly encode &, ", ',
  <, > using HTML entities instead of backslash escaping
- js/prototype-pollution-utility: Add __proto__/constructor/prototype
  key guards to deepMerge function in plugins_manager.js
- app.py error handlers: Always return generic messages; remove debug-mode
  branches that could expose tracebacks in production

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix three remaining CodeQL path-injection and info-exposure alerts

- plugin_loader.py: resolve plugin_dir with strict=True and validate
  marker_path with relative_to() before any filesystem writes, giving
  CodeQL the positive sanitization pattern it requires (py/path-injection)
- api_v3.py _safe_backup_path: replace substring negative checks with a
  strict positive regex (^[a-zA-Z0-9][a-zA-Z0-9._-]{0,200}\.zip$) that
  CodeQL recognises as sanitising the user-supplied filename
  (py/path-injection)
- api_v3.py backup_validate: whitelist known-safe manifest fields before
  returning JSON, preventing any exception strings captured inside
  validate_backup() from reaching the HTTP response (py/stack-trace-exposure)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Resolve 29 open CodeQL security alerts across 5 files

py/flask-debug (#214):
- debug_web_manual.py: read debug mode from LEDMATRIX_FLASK_DEBUG env var
  instead of hardcoded True

py/stack-trace-exposure (#216, #218):
- api_v3.py execute_system_action: remove subprocess stdout/stderr from
  HTTP responses; log via logger instead
- api_v3.py get_git_version: validate output matches safe ref format
  (^[a-zA-Z0-9._-]+$) before including in response
- api_v3.py: remove all remaining traceback.format_exc() dead variables
  and print() debug calls (replaced with logger.debug/warning)

py/reflective-xss (#207, #208, #209, #210, #211, #212):
- api_v3.py: remove plugin_id from all error/success response messages
  (uninstall, install, update, health, not-found responses)
- pages_v3.py load_partial: return static "Partial not found" message
  instead of echoing partial_name
- pages_v3.py _load_starlark_config_partial: add app_id regex validation,
  use static error messages instead of f-strings with app_id

py/path-injection (#187–#206):
- pages_v3.py _load_plugin_config_partial: resolve plugins_base and
  validate _plugin_dir with relative_to() before all file operations;
  same for assets metadata directory
- pages_v3.py _load_starlark_config_partial: resolve starlark_base and
  validate schema_file/config_file paths with relative_to()
- plugin_loader.py _find_plugin_directory: resolve plugins_dir and
  validate strategy-2 candidates with relative_to()
- plugin_loader.py install_dependencies: resolve plugin_dir first, then
  construct requirements_file and marker_path from resolved base
- plugin_loader.py load_module: resolve plugin_dir with strict=True and
  validate entry_file with relative_to() before exec_module

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix 15 remaining CodeQL path-injection and stack-trace-exposure alerts

Switch from resolve()+relative_to() to os.path.basename() reassignment,
which CodeQL recognizes as a path sanitizer that breaks the taint chain.
Also remove exception objects from backup_manager validate_backup return
strings to eliminate the stack-trace-exposure taint source.

Fixes alerts #227, #233, #234, #235, #237, #238, #239, #240, #241,
#242, #243, #244, #245, #246, #247.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix broken logger format string and leaked exception in config save error

- pages_v3.py: plain string was used instead of %-style substitution,
  so every manifest-read failure logged the literal "{plugin_id}"
- api_v3.py save_main_config: exception message was still leaking
  through the error response; replace with generic message (consistent
  with the rest of the CodeQL sweep in this PR)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 09:29:53 -04:00
327e87f735 fix(pi5): auto-detect Pi 5 and force rgbmatrix rebuild when rp1_rio missing (#341)
* fix(pi5): auto-detect Pi 5 and force rgbmatrix rebuild when rp1_rio missing

first_time_install.sh:
- Detect Pi 5 from /proc/device-tree/model at startup
- Step 6 skip logic now also checks hasattr(RGBMatrixOptions(), 'rp1_rio'):
  if the installed library lacks rp1_rio (built before Pi 5 support was added)
  the build is forced even when the module is already importable. This is the
  root cause of mmap errors to 0x3f000000 (Pi 3 bus) on Pi 5 hardware.
- After a successful Pi 5 build, verify rp1_rio is present and print a
  diagnostic with the submodule update command if it's still missing.

src/display_manager.py:
- rp1_rio warning now names the symptom (mmap to 0x3f000000) and gives the
  exact fix command so users can act immediately from the log.

README.md:
- Remove "Pi 5 is unsupported" — Pi 5 is fully supported since the library
  submodule includes rp1_pio and rp1_rio backends.
- Document the forced-rebuild command for users migrating from Pi 4.
- Fix gpio_slowdown guidance: Pi 5 PIO mode uses 1–2, not 5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(install): only append Pi 5 suffix in skip-build message when IS_PI5=1

${IS_PI5:+...} expands whenever IS_PI5 is set, including when it's "0".
Replace with an explicit equality check so the suffix only appears on
actual Pi 5 installs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 16:02:58 -04:00
b5426da2a7 fix(fonts): skip preview API call for BDF bitmap fonts (#345)
The font preview endpoint explicitly rejects .bdf files (glyph rendering
not implemented server-side), returning 400. The JS didn't know this and
called the endpoint for every selected font, causing a noisy 400 on load.

Guard added in updateFontPreview(): if the selected font ends in .bdf,
show "Preview not available for BDF bitmap fonts" and return early instead
of hitting the API.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 14:25:38 -04:00
302ab1da4f fix(plugin-config): handle missing type key in oneOf/anyOf schema fields (#344)
* fix(web-ui): dedup registry fetches, surface reconciliation warnings, add check-update endpoint

Story 1 — src/plugin_system/store_manager.py:
Add threading.Lock (_registry_fetch_lock) to fetch_registry(). The outer cache
check remains the hot path (no lock). When the cache is cold, only one thread
hits the network; concurrent callers block on the lock then get the result from
the warm cache (double-checked locking). Eliminates duplicate GitHub requests
on every page load when the 15-minute cache expires.

Story 2 — web_interface/app.py + api_v3.py + overview.html:
_run_startup_reconciliation() now writes /tmp/ledmatrix_reconciliation.json
(atomic tempfile+replace, mirrors hw_status pattern) so the result survives
the background thread. New GET /api/v3/plugins/reconciliation-status reads
that file. Overview page gains a dismissible yellow banner that shows stale
plugin_id values (e.g. sync, github, youtube) and tells the user to remove
them or reinstall from the Plugin Store. Banner is suppressed for the session
after dismiss using sessionStorage keyed on the plugin_id list.

Story 3 — web_interface/blueprints/api_v3.py:
Add GET /api/v3/system/check-update. Does git fetch origin main then compares
local HEAD vs origin/main to compute update_available, remote_sha, and
commits_behind. Result is cached for 5 minutes so it doesn't run git on every
page load. Falls back to {update_available: false} on any error. Eliminates
the 404 logged on every page load.

Story 4 (Pi 5 rgbmatrix rebuild) was already fixed in PR #341.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugin-config): handle missing `type` key in schema fields using oneOf/anyOf

Jinja2's `prop.type` on a dict without a `type` key returns an Undefined
object. Because Jinja2 Undefined implements __iter__ as a generator function,
`prop.type is iterable` evaluates True, then `prop.type[0]` calls
Undefined.__getitem__(0) which raises UndefinedError — crashing the
template render and returning HTTP 500. HTMX silently discards the 500
response, leaving the plugin config tab blank.

Fix: use `prop.get('type')` which returns None for missing keys instead of
Undefined. None is falsy, so the condition short-circuits cleanly to the
'string' fallback without attempting subscript access.

Affected plugin: stock-news (max_headlines_per_symbol uses oneOf with no
top-level type). Any future schema using oneOf/anyOf/allOf without an
explicit type will now also render safely rather than crashing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): harden check-update, reconciliation status endpoint, and temp-file write

api_v3.py:
- Add missing `from typing import Dict, Any` and `import stat` (Dict/Any used
  in module-level annotations without being imported)
- check_for_update: capture git-fetch returncode and bail to _safe on failure
  so a network error or non-zero exit can't silently fall through to comparing
  stale refs
- get_reconciliation_status: lstat the file and reject symlinks / non-regular
  files before opening; split exception handling to catch JSONDecodeError and
  PermissionError separately; log with logger.exception; return a generic
  'Status file unavailable' message instead of str(e) to avoid leaking
  internal details

overview.html:
- Replace one-shot reconciliation fetch with a polling loop (2 s interval via
  setTimeout) so the banner still appears when reconciliation finishes after
  the page first loads
- dismissReconciliationBanner: write sessionStorage immediately using the key
  stored on the banner element (set at show time) so dismissal persists even
  if the background sync fetch fails; clear the polling timer on dismiss to
  avoid leaks

app.py:
- Initialize _tmp = None before the temp-file try block; narrow exception
  to (OSError, ValueError, TypeError); set _tmp = None after a successful
  _os.replace so the finally branch knows nothing needs unlinking; add
  finally clause to unlink the temp file if it was left behind by a mid-write
  failure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: reconciliation status errors return graceful not-done instead of HTTP 500; log fetch stderr

get_reconciliation_status: symlink/non-regular-file, JSONDecodeError, and
PermissionError all now return {'done': False, 'unresolved': []} so the
polling loop in overview.html keeps retrying rather than stopping on a
transient error.

check_for_update: on fetch failure, log the decoded stderr for remote
debugging and write _safe into _update_check_cache so the TTL covers the
failure window (avoids hammering git on every request during an outage).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(bandit): replace hardcoded /tmp paths with tempfile.gettempdir() (B108)

Codacy/Bandit B108 flagged two hardcoded '/tmp/' string literals in app.py
(lines 737, 741). Replaced with _tempfile.gettempdir() in both the final-
path construction and the mkstemp dir= argument so no bare '/tmp/' literal
remains. Also updated the matching reader path in api_v3.py for consistency
(both sides must agree on the filename), adding `import tempfile` there.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 15:53:16 -04:00
ChuckandGitHub 9cd2bd14ce Update README.md (#342)
Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-05-19 20:47:34 -04:00
53ee184bc5 chore: remove march-madness from bundled plugin-repos (#340)
March Madness is now available in the ledmatrix-plugins monorepo store
(ChuckBuilds/ledmatrix-plugins/plugins/march-madness) and should be
installed via the Plugin Store like any other plugin.

Removing the bundled copy so new installs don't automatically include it.
Existing users keep their installed version until they choose to uninstall.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 20:00:41 -04:00
ChuckandGitHub e00d75bbb5 Disable schedule and update timezone and location (#338)
Updated schedule settings to disable all days and changed timezone and location.

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-05-19 18:57:09 -04:00
33f76b4895 feat(pi5): RP1 backend UI, gpio slowdown guidance, and hardware init error banner (#337)
* feat(pi5): expose RP1 backend selector, fix gpio defaults, surface init failures in web UI

- Add rp1_rio select (PIO/RIO) to Display Settings hardware config section;
  saved via /api/v3/config/main with 0-or-1 validation — previously the key
  existed in config.json but was not editable from the UI
- Update gpio_slowdown help text with per-model guidance (Pi 3: 3, Pi 4: 4,
  Pi 5: 4–5) and raise max from 5 → 10 to match full library range
- Fix gpio_slowdown Python fallback default from 2 → 3 (only affects edge case
  where the runtime config section is absent; explicit config values are unchanged)
- display_manager writes /tmp/led_matrix_hw_status.json at startup: ok/error;
  Display Settings page fetches it and shows a yellow warning banner when the
  matrix failed to initialize, including Pi 5 remediation steps
- Add GET /api/v3/hardware/status endpoint that reads the status file
- Improve fallback error log to include Pi 5 rebuild hint

Pi 3/4 users: rp1_rio=0 is set in config but silently ignored by the library
on non-RP1 hardware; all other changes are additive or tighten defaults only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(pi5): correct gpio_slowdown guidance — Pi 5 PIO default is 1, not 4-5

The upstream library defaults gpio_slowdown to 1 for Pi 5 (IsPi4() ? 2 : 1).
In PIO mode the value is a pixel-clock divisor, so 4-5 was unnecessarily
conservative advice. Updated help text and error log to reflect the actual
range (1-3 typical for Pi 5 PIO; inverted effect in RIO mode).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): atomic hw-status write, narrow bare excepts, urllib3 CVE floor

- display_manager: replace open()+bare-except with tempfile.mkstemp→fsync→
  chmod(0o600)→os.replace; adds symlink guard and logs errors via logger
  instead of swallowing them silently; pull json/tempfile to module imports
- display_manager cleanup(): narrow broad `except Exception: pass` to
  (OSError, RuntimeError, ValueError, MemoryError) with debug log
- api_v3 get_hardware_status(): catch json.JSONDecodeError and PermissionError
  explicitly; log full traceback server-side; return generic "Unable to read
  hardware status" to client instead of leaking str(e)
- march-madness/requirements.txt: bump urllib3 floor 2.2.2→2.6.3 (CVE fix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(template): apply |int filter to rp1_rio comparisons in display.html

Without |int, a string-typed value (e.g. from a hand-edited config.json)
causes both selected tests to fail and the select renders with no option
pre-selected. Matches the existing pattern used for multiplexing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 17:42:19 -04:00
c6b79e11d5 fix: Codacy round-2 — urllib3 CVEs, missed JS/Python issues (#336)
urllib3 CVEs (10 Trivy findings):
  plugin-repos/march-madness/requirements.txt: bump urllib3>=1.26.0 to
  >=2.2.2 to address CVE-2021-33503, CVE-2023-43804, CVE-2023-45803,
  CVE-2024-37891, and 2025-2026 decompression/redirect CVEs.

Missed code fixes from round-1:
  display_helper.py: remove unused draw=ImageDraw.Draw(img) — the method
  delegates to _draw_centered_text which creates its own draw context.
  custom-feeds.js:334: one bare removeCustomFeedRow(this) was missed by
  the earlier replace_all; changed to window.removeCustomFeedRow(this).
  app.js: add htmx to /* global */ declaration — htmx.ajax() is called
  at lines 146 and 172 but htmx was only declared in the extension files.
  timezone-selector.js:215: second unused catch (e) → catch {} missed
  when we fixed line 361 in round-1.

Bandit B110 annotations (3 new except/pass blocks from newer PRs):
  start.py: hostname -I IP parsing — non-critical startup info.
  display_controller.py: scroll_helper.get_portion_at — optional method.
  display_manager.py: canvas reset during cleanup — best-effort.

41 confirmed false positives suppressed via Codacy API:
  35x pyflakes in test/, plugin-repos/, scripts/ — not production code
  Flask 0.0.0.0, os.execvp, Bandit B603, vendor ESLint, already-fixed
  Biome noPrototypeBuiltins.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 18:04:21 -04:00
d941c91f24 fix(systemd): wait for network connectivity before starting services (#335)
Change After=network.target → After=network-online.target + Wants=network-
online.target in both service templates and install_web_service.sh.

network.target only guarantees NetworkManager has started — it does NOT
mean the device has an active internet connection. On boot the LED matrix
service was starting within seconds of the network interface appearing,
before WiFi association and DHCP completed, causing every first-update API
call to fail with "Network is unreachable" or DNS resolution errors.

network-online.target waits for a confirmed route before the service fires.
On Raspberry Pi OS this is provided by NetworkManager-wait-online. The
tradeoff is a few extra seconds at boot, acceptable for a display device.

Observed on devpi: service started at 14:48:03, all API calls (weather,
FlightRadar24, local ADS-B) failed at 14:48:07 with network errors, then
the service restarted cleanly at 14:50:40 once WiFi was established.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 15:47:35 -04:00
054ad78d7b chore(deps): update rpi-rgb-led-matrix to latest upstream for Pi 5 support (#334)
* chore(deps): update rpi-rgb-led-matrix to latest upstream for Pi 5 support

Configure submodule to track upstream master branch (branch = master in
.gitmodules) so future updates are a single 'git submodule update --remote'
rather than manual SHA management.

Update first_time_install.sh to use --remote flag so fresh installs always
pull the current upstream master, not the commit recorded at clone time.

Current upstream HEAD (8907235) brings:
- PR #1886: Raspberry Pi 5 support — new RP1 PIO and RIO backends. The
  library auto-detects Pi 5 hardware at runtime; no config change required
  for basic operation. adafruit-hat-pwm is confirmed supported on Pi 5.
- PR #1833: setup.py migrated from distutils → setuptools, fixing Python
  3.12+ build failure (Pi runs Python 3.13). Previous version could not
  build the bindings at all on current Pi OS.

Expose new rp1_rio option in display_manager.py and config.template.json:
  0 (default) = PIO mode — uses Pi 5 RP1 coprocessor, minimal CPU usage
  1 = RIO mode — Registered IO, faster throughput, higher CPU; note that
      gpio_slowdown has inverted effect in this mode

No API changes to RGBMatrix, RGBMatrixOptions, or FrameCanvas. Pi 4 and
earlier hardware is unaffected — rp1_rio is silently ignored on non-Pi-5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(deps): update rpi-rgb-led-matrix install for new scikit-build-core system

The library migrated from 'make build-python' + 'pip install bindings/python'
to a scikit-build-core + cmake build where the entire repo root is pip-
installable via 'pip install .'. Update first_time_install.sh accordingly:
- Remove the 'make build-python' step (target no longer exists)
- Install directly from the repo root instead of bindings/python
- Replace build deps: remove cython3/scons/python3-dev, add python-dev-is-python3

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: deterministic submodule install + guard rp1_rio for older rgbmatrix

first_time_install.sh: remove --remote from both git submodule update
calls so first-time installs check out the pinned commit recorded in the
repo rather than whatever upstream master happens to be at install time.
The branch = master config in .gitmodules reserves --remote for an
explicit maintainer upgrade (git submodule update --remote).

display_manager.py: guard rp1_rio assignment with hasattr() so setting
the option in config does not cause an AttributeError and silently fall
through to emulator mode when running against RGBMatrixEmulator or an
older rgbmatrix build that predates the Pi 5 property. Emit a warning
instead so the operator knows the value was ignored.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 14:17:00 -04:00
05b3fa56cb fix: Codacy security fixes, CVE dependency bumps, and code quality cleanup (#331)
* fix(deps): bump minimum versions to address CVEs

Pillow 10.4.0 → 12.2.0: CVE-2026-40192 (DoS via FITS decompression bomb),
CVE-2026-25990 (OOB write via PSD image), CVE-2026-42311/42308/42310

requests 2.32.0 → 2.33.0: CVE-2026-25645 (temp file security bypass),
CVE-2024-47081 (.netrc credentials leak)

werkzeug 3.0.0 → 3.1.6: CVE-2023-46136, CVE-2024-49766/49767,
CVE-2025-66221, CVE-2026-21860/27199 (DoS, path traversal, safe_join bypass)

Flask 3.0.0 → 3.1.3: CVE-2026-27205 (session data caching info disclosure)

spotipy 2.24.0 → 2.25.2: CVE-2025-27154, CVE-2025-66040

python-socketio 5.11.0 → 5.14.0: CVE-2025-61765

pytest 7.4.0 → 9.0.3: CVE-2025-71176 (insecure temp dir handling)

Updated in requirements.txt, web_interface/requirements.txt,
plugin-repos/starlark-apps/requirements.txt, and
plugin-repos/march-madness/requirements.txt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve Pylint errors in executor, data service, and odds call

Rename TimeoutError to PluginTimeoutError in plugin_executor.py to
avoid shadowing the built-in; no external callers affected.

Remove dead try/except in BackgroundDataService.shutdown: executor.shutdown()
never accepted a timeout kwarg so the try branch always raised TypeError.
Simplify to a direct shutdown(wait=wait) call.

Remove is_live kwarg from odds_manager.get_odds() call in sports.py;
BaseOddsManager.get_odds() has no such parameter. The live update interval
is already encoded in the update_interval_seconds argument passed alongside.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: MD5→SHA-256, shellcheck warnings, and broken doc links

config_service.py: replace MD5 with SHA-256 for config change detection;
same semantics (equality comparison), no stored hashes affected.

Shell scripts — shellcheck warnings:
- diagnose_web_interface.sh: remove useless cat (SC2002)
- dev_plugin_setup.sh: restructure A&&B||C into if/then (SC2015)
- fix_assets_permissions.sh: remove unused REAL_HOME block (SC2034)
- install_web_service.sh: remove unused USER_HOME assignment (SC2034)
- diagnose_web_ui.sh: remove unused SUDO assignments (SC2034)
- diagnose_plugin_permissions.sh: remove unused BLUE color var (SC2034)
- first_time_install.sh: remove unused CLEAR var, PACKAGE_NAME
  assignment, and replace loop variable with _ (SC2034)

docs/PLUGIN_ARCHITECTURE_SPEC.md: fix 10 broken TOC anchor links to
include section numbers matching the actual headings (MD051).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unused imports and bare exception aliases (pyflakes F401/F841)

Remove unused imports across 86 files in src/, web_interface/, test/,
and scripts/ using autoflake. No logic changes — only dead import
statements and unused names in from-imports are removed.

Also remove bare exception aliases where the variable is never
referenced in the handler body:
- src/cache/disk_cache.py: except (IOError, OSError, PermissionError) as e
- src/cache_manager.py: except (OSError, IOError, PermissionError) as perm_error
- src/plugin_system/resource_monitor.py: except Exception as e
- web_interface/app.py: except Exception as read_err

86 files changed, 205 lines removed, 18 pre-existing test failures unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unused local variable assignments (pyflakes F841)

Dead assignments removed across src/ and web_interface/:

- background_data_service: drop future= on fire-and-forget executor.submit
- base_classes/baseball: drop font= (all rendering uses self.fonts['time'])
- base_classes/hockey: drop status_short= (never referenced after assignment)
- common/cli: drop game_helper=/config_helper= bindings in import-test block;
  constructors called for instantiation-only validation
- common/display_helper: drop text_width= (x_position uses display_width
  directly); drop draw= in create_error_image (uses _draw_centered_text)
- config_manager: remove dead secrets_content loading block in migration path
  (comment already noted save_config_atomic handles secrets internally)
- display_manager: drop setup_start= (timing was never completed or read)
- font_manager: drop target_path= (catalog uses font_file_path directly);
  drop face=/font= bindings in validate_font (validation by construction —
  TypeError on failure is the signal, not the return value)
- font_test_manager: drop width=/height= (draw_text uses display_manager directly)
- plugin_system/state_reconciliation: drop manager= (only config/disk/state_mgr used)
- plugin_system/store_manager: drop result= on pip install subprocess.run
  (check=True raises on failure; stdout unused)
- web_interface/blueprints/pages_v3: drop main_config_path=""/secrets_config_path=""
  (render_template uses config_manager.get_*_path() inline)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(js): resolve ESLint no-undef warnings across 6 JS files

Three distinct patterns:

1. Vendor library globals — htmx is injected by <script> before these
   extension files load; ESLint lints files in isolation and doesn't know.
   Fix: add /* global htmx */ to htmx-sse.js and htmx-json-enc.js.

2. Cross-file globals — showNotification is defined as window.showNotification
   in app.js/notification.js but called bare in app.js and error_handler.js.
   ESLint doesn't connect window.X = Y with a bare call to X.
   Fix: add /* global showNotification */ to app.js and error_handler.js.

3. Forward-reference window.* functions — in array-table.js, checkbox-group.js,
   and custom-feeds.js, functions like removeArrayTableRow are called early
   inside event-handler closures but assigned to window.* later in the file.
   At runtime this works (the handler fires after the assignment), but ESLint
   sees the bare name at the call site.
   Fix: change bare calls to window.removeArrayTableRow(this) etc. so the
   reference is explicit and ESLint-safe.

Also guard the updateSystemStats call in app.js reconnectSSE: the function
is called but defined nowhere in the codebase. Guard with typeof check so
it won't throw ReferenceError if the reconnect path is hit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(js): resolve Biome lint warnings across 9 JS files

noUnusedVariables (catch bindings → optional catch syntax):
- app.js, file-upload.js, timezone-selector.js: } catch (e) { → } catch {
  ES2019 optional catch binding; e was unused in all three handlers

noUnusedVariables (dead assignments):
- app.js: remove const data= in display SSE stub (handler does nothing yet)
- api_client.js: remove const timeoutId= (setTimeout ID never used to cancel)
- custom-feeds.js: remove const oldIndex= (getAttribute result never read)
- schedule-picker.js: remove const compactMode= (never used in HTML build)
- select-dropdown.js: remove const icons= (icons not yet rendered in options)

noPrototypeBuiltins:
- day-selector.js: DAY_LABELS.hasOwnProperty(x) →
  Object.prototype.hasOwnProperty.call(DAY_LABELS, x)
  Safe form that works even on null-prototype objects

useIterableCallbackReturn:
- file-upload.js, notification.js: forEach(x => expr) →
  forEach(x => { expr; }) — forEach ignores return values;
  implicit return from arrow body was misleading

htmx-sse.js is a vendor extension file with old-style var/== patterns
that are correct for it; 18 Biome issues suppressed via Codacy API
rather than modifying the vendor source.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(security): escape user input in raw HTML responses in pages_v3.py

plugin_id comes directly from the URL path
(/partials/plugin-config/<plugin_id>) and was interpolated into an HTML
fragment without escaping. A crafted URL like
/partials/plugin-config/<script>alert(1)</script> would inject that
tag into the DOM via the HTMX partial response.

Fix: wrap all user-controlled values in markupsafe.escape() before
embedding in raw HTML strings. Affects the plugin-not-found 404
response and both error 500 responses in the plugin config partial.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address Bandit B108/B110 across production code

B110 (try/except/pass):
- display_controller.py: narrow 'except Exception' to 'except AttributeError'
  for get_offset_frame() — plugins not having this optional method is the
  expected case, not all exceptions
- config_manager.py: B110 already resolved by the earlier removal of the
  dead secrets-loading block (the except/pass was inside it)
- All other except/pass blocks in src/ and web_interface/ are intentional
  (last-resort recovery, best-effort fallbacks, non-critical startup probes).
  Annotated each with # nosec B110 and a brief inline reason so the decision
  is explicit for future reviewers.
- Test files and plugin-repos B110 suppressed via Codacy API (not prod code).

B108 (/tmp usage):
- permission_utils.py: /tmp listed to PREVENT permission changes on it — not
  used as a temp path. Annotated # nosec B108.
- display_manager.py: fixed snapshot path is intentional (web UI reads same
  path); path-check guard also annotated.
- wifi_manager.py: named /tmp files match the sudoers allowlist installed with
  the system (the paths are hard-coded in both places by design). Annotated
  all six open/cp references # nosec B108.
- scripts/render_plugin.py: dev script default overridable by user. Annotated.
- web_interface/app.py: reads the same fixed path written by display_manager.
  Annotated # nosec B108.
- Test files suppressed via Codacy API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address remaining Codacy security findings

Flask debug=True (real fix):
- web_interface/app.py: debug=True in __main__ block exposes the Werkzeug
  interactive debugger (arbitrary code execution). Changed to
  os.environ.get('FLASK_DEBUG', '0') == '1' — off by default, opt-in
  via environment variable for local development.

nosec annotations (accepted risk with documented rationale):
- disk_cache.py: os.chmod(0o660) is intentional — web UI and LED matrix
  service share a group, 660 gives group write while denying world access
  (B103 + Semgrep insecure-file-permissions suppressed in Codacy)
- wifi_manager.py: urlopen to hardcoded connectivity-check.ubuntu.com URL
  (B310 — no user input involved)
- font_manager.py: urlretrieve URL comes from user's own config file on
  their local device (B310)
- start_web_conditionally.py: os.execvp with both sys.executable and a
  fixed PROJECT_DIR-relative constant (B606)

Confirmed false positives suppressed via Codacy API (15 issues):
- SSRF (3x): client-side JS fetch — SSRF is server-side; browser fetch
  is CORS-restricted to same origin
- B105 (3x): test fixtures use dummy secrets by design; store_manager
  checks for the placeholder string, it is not itself a secret
- PMD numeric literal (2x): 10000000 is within Number.MAX_SAFE_INTEGER
- Prototype pollution (1x): read-only schema traversal, no writes
- no-unsanitized_method (1x): dynamic import() is CORS-restricted
- detect-unsafe-regex (1x): operates on server-controlled config values
- plugin-repos B103 (1x): vendor code chmod on executable
- Semgrep insecure-file-permissions (3x): same disk_cache 0o660 as above

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove unnecessary f prefix from f-strings without placeholders (F541)

Pyflakes F541 flags f-strings that contain no {} interpolation — they are
identical to plain strings but trigger unnecessary string formatting overhead.

Fixed in production code:
- src/base_classes/data_sources.py (2 debug log calls)
- src/logo_downloader.py (1 error log)
- src/plugin_system/store_manager.py (5 strings across 3 log calls)
- src/web_interface/validators.py (1 return value)
- src/wifi_manager.py (4 log/message strings)
- web_interface/start.py (1 print)

F541 issues in test/, scripts/, and plugin-repos/ suppressed via Codacy API
as non-production code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(dev): add Pillow compatibility smoke test script

Covers all Pillow APIs used in LEDMatrix — image creation, drawing,
font metrics, LANCZOS resampling, paste/alpha_composite, and PNG I/O.
Run after any Pillow version bump to catch regressions before deploy.

    python3 scripts/dev/test_pillow_compat.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve 8 new Codacy issues introduced by PR changes

shellcheck SC2034:
- first_time_install.sh: 'type' loop variable also unused in the wifi
  status loop (we previously fixed 'device' → '_' but left 'type').
  Changed to '_ _ state' since neither device nor type is referenced.

ESLint no-undef:
- app.js: typeof guards don't satisfy no-undef; added updateSystemStats
  to the /* global */ declaration alongside showNotification.

nosec annotation:
- web_interface/app.py: app.run(host='0.0.0.0') line changed when we
  fixed debug=True, giving it a new issue ID. Re-added # nosec B104.

pyflakes F401:
- scripts/dev/test_pillow_compat.py: ImageFilter was imported but never
  used in the smoke test. Removed from the import.

Codacy API suppressions (false positives on changed lines):
- disk_cache.py 0o660 chmod (2x): lines changed when # nosec B103 was
  added, producing new Semgrep issue IDs. Re-suppressed.
- pages_v3.py raw-html-concat: Semgrep does not recognise escape() as
  a sanitizer; the escape() call IS the correct fix.
- app.py flask 0.0.0.0: same line as B104 above; Semgrep rule also
  re-suppressed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address PR review findings

Fix (10 of 15 findings):

plugin-repos/march-madness/requirements.txt:
  Add urllib3>=1.26.0 — manager.py directly imports from urllib3; it was
  an undeclared transitive dependency via requests.

scripts/dev/dev_plugin_setup.sh:
  Restore subshell form (cd "$target_dir" && git pull --rebase) || true
  so the shell's working directory is not permanently changed after the
  if-cd block. Previous fix for SC2015 leaked cwd into the remainder of
  the script.

src/base_classes/sports.py:
  Narrow 'except Exception' to 'except RuntimeError as e' and log via
  self.logger.debug — Path.home() raises only RuntimeError for service
  users; other exceptions should not be silently swallowed.

src/config_service.py:
  Fix stale "MD5 checksum" in ConfigVersion.__init__ docstring (line 40);
  the implementation uses SHA-256 since the Codacy fix.

src/wifi_manager.py:
  Log the last-resort AP enable failure with exc_info=True instead of
  silently passing — failure here means the device may be unreachable.

web_interface/blueprints/pages_v3.py:
  Log the outer metadata pre-load exception at debug level instead of
  swallowing it silently; schema still loads fully below.

src/background_data_service.py:
  Remove unused 'timeout' parameter from shutdown() — executor.shutdown()
  does not accept timeout; update __del__ caller accordingly.

src/font_manager.py:
  Validate URL scheme before urlretrieve — reject non-http/https schemes
  (e.g. file://) to prevent reading local files from config-supplied URLs.

src/plugin_system/plugin_executor.py:
  Simplify redundant except tuple: (PluginTimeoutError, PluginError,
  Exception) → Exception, which already covers the others.

test/test_display_controller.py:
  Mark empty test_plugin_discovery_and_loading as @pytest.mark.skip with
  reason. Move duplicate 'from datetime import datetime' to module header
  and remove the stray mid-module copy.

Skip (5 of 15 findings, with reasons):
  - pytest 9.0.3 concerns: full suite already verified (467 pass, 18 pre-existing)
  - Pillow 12.2.0 API concerns: no deprecated APIs in codebase; tests + Pi smoke test pass
  - diagnose_web_ui.sh sudo validation: set -e already ensures fail-fast on any sudo failure
  - app.py request-logging except: must stay silent (recursive logging risk); annotated
  - app.py SSE file-read except: genuinely transient I/O; annotated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 10:19:55 -04:00
44d1a08db4 perf(plugins): dramatically speed up plugin manager tab load time (#333)
* fix(cache): check odds keys before generic live check in get_data_type_from_key

Cache keys like odds_espn_basketball_nba_<id>_live contain both 'odds'
and 'live'. The previous ordering matched the generic 'live' check first,
returning 'sports_live' (30 s TTL) instead of the correct 'odds_live'
(120 s TTL). This caused the ESPN odds API to be hit every 30 s per live
game, frequently triggering the 3-second per-request timeout and returning
no odds data.

Moving the 'odds' check above the generic 'live' block restores the
correct 120-second cache TTL for in-progress game odds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display): use single-quoted HTML attributes for JSON hidden inputs

Placing |tojson output (which contains double quotes) inside a
double-quoted HTML attribute broke the attribute — browsers closed
the attribute at the first inner quote, leaving JS with an empty or
truncated value. JSON.parse then failed silently, leaving excluded=[]
so all Vegas scroll plugins appeared checked (included) regardless of
the actual excluded_plugins config.

Switch to single-quoted HTML attributes so the JSON double quotes
are valid inside the attribute value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(plugins): dramatically speed up plugin manager tab load time

## Problem

The Plugins tab loaded slowly and inconsistently (5–30s depending on
cache state), with a blank spinner for the entire wait. Three root
causes:

1. **N+1 subprocess per installed plugin** — `_get_local_git_info` ran
   4 separate git subprocesses per plugin (rev-parse HEAD, abbrev-ref,
   config --get remote.origin.url, log --format=%cI). With 15 plugins
   that's 60 blocking subprocess spawns before the endpoint returned.

2. **Serial per-plugin loop** — the `/plugins/installed` endpoint
   processed each plugin sequentially: manifest read → git info →
   instance lookup → Vegas mode query, one plugin at a time.

3. **Serial JS loading** — the store search only started after installed
   plugins fully completed, so users waited for both round-trips back
   to back. No UI feedback during the wait.

## Changes

### Backend — src/plugin_system/store_manager.py
- Consolidate 4 git subprocesses → 1: branch read from `.git/HEAD`
  (file I/O, no subprocess), remote URL parsed from `.git/config`
  (file I/O, no subprocess), SHA + commit date fetched together in a
  single `git log -1 --format=%H%n%cI` call
- Existing signature-based cache already eliminates all subprocesses on
  warm hits; this change cuts cold-cache cost from 4 → 1 per plugin

### Backend — web_interface/blueprints/api_v3.py
- Wrap per-plugin work in a `_build_plugin_entry()` helper and execute
  it across a `ThreadPoolExecutor(max_workers=8)` so all plugins are
  processed in parallel instead of sequentially
- Fix double `get_plugin()` call per plugin (was called once for the
  enabled fallback and again for Vegas mode — now one shared call)

### Frontend — web_interface/static/v3/plugins_manager.js
- Fire `searchPluginStore()` and `loadInstalledPlugins()` simultaneously
  instead of waiting for installed to complete before starting the store
- After installed data arrives, call `applyStoreFiltersAndSort(true)` to
  refresh install/update/reinstall badges from already-cached store data
  (instant, no extra network call)

### Frontend — web_interface/templates/v3/partials/plugins.html
- Add responsive skeleton cards to the installed plugins section that
  match real card proportions (removed automatically when data renders)
- Replace the 5 featureless gray boxes in the store skeleton with 10
  structured skeleton cards matching the real card layout

## Measured improvement on Pi 4 (11 installed plugins, ledpi-ticker)

| Scenario | Before | After |
|---|---|---|
| Cold cache (first open) | ~8–15s | **0.9s** |
| Warm cache (git cache hit) | ~1–2s | **55ms** |
| UI feedback during load | blank spinner | skeleton cards |
| Store waits for installed | yes (serial) | no (parallel) |

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugins): harden git metadata parsing and plugin entry building

store_manager.py:
- Detect worktree/submodule .git files (gitdir: <path>) and resolve
  to the actual git directory before reading HEAD or config
- Wrap HEAD read_text in try/except OSError/NotADirectoryError so
  atypical repos return None instead of propagating exceptions
- Guard config url line split with '=' presence check to avoid
  IndexError on malformed lines

api_v3.py:
- Wrap _build_plugin_entry body in a try/except via a thin outer
  wrapper so a single plugin's failure doesn't 500 the whole endpoint;
  failed entries return None and are filtered by the existing [r for r
  in results if r is not None] step
- Narrow manifest except clause to FileNotFoundError, PermissionError,
  json.JSONDecodeError instead of bare Exception
- Validate manifest is a dict before calling plugin_info.update() and
  log a debug message when it isn't

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 18:09:33 -04:00
6a4644007d fix(display): Vegas excluded plugins always showing as checked (#332)
* fix(cache): check odds keys before generic live check in get_data_type_from_key

Cache keys like odds_espn_basketball_nba_<id>_live contain both 'odds'
and 'live'. The previous ordering matched the generic 'live' check first,
returning 'sports_live' (30 s TTL) instead of the correct 'odds_live'
(120 s TTL). This caused the ESPN odds API to be hit every 30 s per live
game, frequently triggering the 3-second per-request timeout and returning
no odds data.

Moving the 'odds' check above the generic 'live' block restores the
correct 120-second cache TTL for in-progress game odds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display): use single-quoted HTML attributes for JSON hidden inputs

Placing |tojson output (which contains double quotes) inside a
double-quoted HTML attribute broke the attribute — browsers closed
the attribute at the first inner quote, leaving JS with an empty or
truncated value. JSON.parse then failed silently, leaving excluded=[]
so all Vegas scroll plugins appeared checked (included) regardless of
the actual excluded_plugins config.

Switch to single-quoted HTML attributes so the JSON double quotes
are valid inside the attribute value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 18:09:16 -04:00
1c4d5c5271 feat(sync): multi-display wireless sync — extend scrolling across two LED matrices (#330)
* feat(sync): multi-display wireless sync — extend scrolling across two LED matrices

Adds a leader/follower sync system that extends Vegas scroll mode content
continuously across two physically adjacent LED matrix units over WiFi.

Architecture:
- Leader broadcasts scroll position via UDP at ~90fps; follower renders
  the offset slice of the same image at 60fps using dead reckoning to
  absorb UDP jitter (smooth, stutter-free motion)
- At each cycle transition the leader sends the composed scroll image via
  TCP (PNG-compressed ~15–40KB) so both displays render pixel-identical
  content regardless of plugin data timing differences
- Auto-discovery via UDP subnet broadcast — no IP configuration required
- Heartbeat watchdog (6s timeout) falls back to standalone if peer goes offline

Key files:
- src/common/sync_manager.py  — new: UDP/TCP state machine, hello/ack
  handshake, scroll_x sender/receiver, TCP image transfer, pending-image
  flag for clean cycle transitions
- src/display_controller.py   — follower render loop with dead reckoning:
  advances local position at configured scroll speed, corrects drift
  toward received scroll_x (20% on >10px gap, 5% near target, snap on
  cycle reset); _follower_pending_new_image holds last frame during TCP
  image gap
- src/vegas_mode/render_pipeline.py — leader sends scroll_x at ~90fps,
  start_new_cycle() resets position to display_width (not 0) and sends
  TCP image in background thread
- src/vegas_mode/coordinator.py — set_sync_manager() / set_update_callback()
  wiring; defers hot-swap recompose while sync is active
- web_interface/blueprints/api_v3.py — sync config save endpoint, GET
  /api/v3/sync/status for live status polling
- web_interface/templates/v3/partials/display.html — Multi-Display Sync
  section: role selector (Standalone/Leader/Follower), position (Left/Right
  of leader, follower only), UDP port, live status indicator
- config/config.template.json — sync block: role, port, follower_position

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address PR review findings

- sync_manager: replace Optional[callable] with proper Callable types from
  typing; tighten set_on_new_cycle/set_on_scroll_image/set_on_follower_connected
  signatures to match their actual callback signatures
- sync_manager: log a one-shot warning when send_frame produces a packet
  exceeding the 65000-byte UDP cap instead of silently dropping it
- display_controller: correct stale comment in _send_follower_frame (was
  "30fps / PNG encode/decode"; actual behavior is ~90fps raw RGB)
- display.html: guard setInterval with window.syncStatusInterval to prevent
  duplicate pollers if the script runs more than once
- display.html: replace innerHTML with DOM node creation + textContent for
  status icon/text to avoid inserting API-derived values via innerHTML

Skip: time.time() → monotonic and self.config staleness are pre-existing
issues not introduced by this PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address second round of PR review findings

- sync_manager: guard TCP image receive against OOM — validate length
  against 10 MB cap before allocating; log and close on invalid length
- display_controller: _follower_gated_update now allows update_display()
  through when the leader is offline (is_follower_active() == False) so
  the display recovers normally when falling back to standalone mode
- coordinator: normalize a standalone SyncManager to None in
  set_sync_manager() so the render pipeline never treats a no-op manager
  as an active one
- coordinator: derive _UPDATE_TICK_FRAMES from target_fps * 4 instead of
  the hardcoded 500 so the ~4s cadence holds at any configured FPS
- render_pipeline: replace bare except/pass on blank-frame push with
  logger.exception() so failures are visible in logs

Skip: config.template.json comments — JSON does not support inline comments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address third round of PR review findings

- sync_manager: use 'with socket.socket(...)' in send_scroll_image so the
  TCP socket is always closed even if connect/sendall raises
- sync_manager: add _scroll_image_lock to serialize all reads/writes to
  _on_scroll_image and _pending_scroll_image between _image_server_loop
  and set_on_scroll_image, eliminating the lost-delivery race; callback
  is invoked outside the lock to avoid holding it during user code
- sync_manager: validate scroll image dimensions (max 100000×256) and
  catch DecompressionBombError before img.load() in _image_server_loop
- sync_manager: log socket close exceptions at debug level in stop()
  instead of silently passing
- sync_manager: replace hardcoded /tmp/ with tempfile.gettempdir() for
  STATUS_FILE (atomic write was already in place)
- sync_manager: check _RAW_MAGIC first in _follower_recv_loop routing
  so magic-tagged frames are always identified correctly regardless of size

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): address fourth round of PR review findings

- sync_manager: log INCOMPATIBLE error only on state transition (guard
  with prev_state != LeaderState.INCOMPATIBLE) so repeated hello packets
  from an incompatible follower don't spam the log
- sync_manager: replace O(n²) bytes concatenation in TCP image receive
  loop with bytearray + extend() for linear-time accumulation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): suppress Codacy false positives

- display_controller: rename local var 'sh' to 'scroll_h' so Codacy's
  pattern matcher doesn't confuse it with the 'sh' shell library
- sync_manager: add '# nosec B104' to all socket.bind("") calls —
  binding to all interfaces is intentional (UDP broadcast reception and
  TCP image server must accept connections from any local interface)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sync): add nosec B104 to socket creation lines for Codacy

Codacy attributes the bind-to-all-interfaces finding to the socket.socket()
creation lines (140, 439) rather than the .bind() calls. Added # nosec B104
there too so the suppression is seen at the line Codacy reports.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 09:51:44 -04:00
dbb53da31d fix(vegas): eliminate plugin re-appearance at scroll cycle boundaries (#327)
* fix(vegas): eliminate plugin re-appearance at scroll cycle boundaries

The Vegas scroll image is wider than the display. scroll_helper marks a
cycle complete only after total_distance_scrolled >= total_scroll_width +
display_width, meaning it keeps scrolling for an extra display_width of
pixels after all content has exited left. During that extra travel the
scroll_position wraps back to ~0 and the first plugin re-enters from the
right - visible for ~2-3 seconds as a plugin partially displaying before
the next one starts.

render_pipeline.render_frame(): end the cycle the moment
total_distance_scrolled >= total_scroll_width (the natural wrap point),
before any second-pass content becomes visible. Push a blank frame
immediately on detection so hardware never shows a frozen content
snapshot while start_new_cycle() recomposes (~100 ms).

display_manager.py: add capture_mode() context manager. When active,
update_display() and the canvas clear in clear() skip the hardware
write, preventing plugins that call update_display() internally from
flashing on the matrix during off-screen content capture inside
start_new_cycle().

plugin_adapter.py: wrap all plugin.display() calls in
_capture_display_content() and _trigger_scroll_content_generation()
with capture_mode() so the fallback capture path never produces
hardware output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(vegas): tighten exception handling in clear() and blank-frame push

display_manager.clear(): replace bare except/pass on the three hardware
Clear() calls with (RuntimeError, OSError) and a logger.error() so
failures are visible in logs rather than silently swallowed.  Still
non-fatal — the PIL image buffer is already black before these calls,
so the next update_display() will push clean content regardless.

render_pipeline.render_frame(): replace broad except/pass in the
blank-frame push with (ImportError, ValueError, TypeError, MemoryError)
and a logger.error() that includes display dimensions for context.
update_display() already handles its own hardware errors internally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(vegas): catch OSError and RuntimeError in blank-frame push

Image.new() can raise OSError in some PIL environments and hardware
libraries may surface RuntimeError on I/O failures.  Add both to the
exception tuple alongside the existing ImportError/ValueError/TypeError/
MemoryError so no boundary failure escapes the local handler.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 15:51:38 -04:00
452afacd12 fix(news): custom RSS feed save fails with validation error when no logo (#329)
_set_missing_booleans_to_false was unconditionally creating an empty
dict for every nested-object sub-property when processing array items.
For the news plugin's custom_feeds, this produced logo:{} on every feed
item that had no logo uploaded. jsonschema then validated that empty
object against logo's required:["id","path"] constraint and failed.

Fix: skip recursion into a sub-object when it isn't already present in
the array item. There's no reason to create an optional object like
logo just to look for boolean fields inside it.

Also extend _filter_config_by_schema to recurse into array items when
the items schema has properties. Previously arrays were passed through
unchanged, so any stray field on a feed item (legacy data, migration
artifacts) would survive to validation where additionalProperties:false
would reject it.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 15:51:17 -04:00
3b45a75f75 fix: pixlet install false-failure, force render in web service, web UI perf (#328)
* fix: pixlet install false-failure, force render in web service, web UI perf

Fixes three user-reported issues:

1. Pixlet install reported failure even on success — `set -e` in
   download_pixlet.sh caused the script to exit non-zero when
   `((success_count++))` evaluated the post-increment old value (0),
   which bash treats as a failed command. Fixed with shell arithmetic
   assignment instead.

2. Force render returned 503 "plugin not loaded in web service" — the
   web service runs its own PluginManager with display_manager=None,
   so the starlark-apps plugin can never be loaded there. Added a
   standalone render path (_standalone_render_starlark_app) that calls
   pixlet directly from the web service, writing to cached_render.webp.
   The main-service plugin path is preserved and tried first.

3. Web UI sluggishness — the SSE /stream/stats generator was blocking
   a Flask thread for 1s per tick via psutil.cpu_percent(interval=1).
   Switched to non-blocking interval=None (primed at startup). Also
   cached the per-client ledmatrix systemctl check (15s TTL), raised
   the AP mode check TTL from 5s to 30s, and halved the display
   preview poll rate from 0.5s to 1s.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(starlark): surface config errors, deduplicate pixlet lookup, uniform response shape

- _standalone_render_starlark_app: split silent except into separate
  json.JSONDecodeError and OSError handlers that return (False, message)
  with the file path and parse error, so callers know when config.json
  is unreadable rather than silently rendering with empty config

- get_starlark_status: replace 14-line inline platform/shutil pixlet
  lookup with _find_pixlet_binary(), which also checks the user-
  configured starlark-apps.pixlet_path — the old code never consulted
  that setting so the status endpoint could wrongly report pixlet
  unavailable even when a custom path was configured

- render_starlark_app standalone branch: add frame_count: 0 to success
  and error responses so both branches return the same payload shape;
  0 is accurate since standalone render writes cached_render.webp but
  does not extract frames into memory

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(starlark): safe chmod fallback in pixlet lookup, semantic HTTP codes in standalone render

_find_pixlet_binary: bundled.chmod(0o755) could raise OSError (e.g.
file owned by root) and abort the entire resolution chain, skipping
the PATH fallback. Now checks os.access first; if chmod is needed,
wraps it in try/except OSError, logs a warning, and falls through to
shutil.which so PATH is always tried.

_standalone_render_starlark_app: expanded return type from
(bool, str) to (bool, int, str) so each failure mode carries a
semantic HTTP status code:
  app/star file missing  → 404
  invalid/unreadable config.json → 400
  pixlet binary missing  → 503
  pixlet non-zero exit   → 502
  subprocess timeout     → 504
  unexpected exception   → 500
  success                → 200
Call site updated to unpack (success, status_code, error) and forward
status_code directly in both success and error responses.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(starlark): validate config.json is a dict before iterating items

json.load succeeds on valid JSON that isn't an object (e.g. an array
or bare string), leaving app_config as a non-dict and causing an
AttributeError on the subsequent .items() call. Added isinstance check
immediately after json.load; non-dict values return (False, 400, ...)
with the actual type name in the message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(starlark): harden pixlet binary check and manifest shape validation

_find_pixlet_binary: switched bundled.exists() to bundled.is_file() so
a directory named pixlet-linux-arm64 (traversable, hence X_OK) is no
longer returned as a binary; re-check os.access after chmod succeeds
so we only return the path when executability is confirmed, with a
separate warning if it still isn't executable after chmod.

_standalone_render_starlark_app: added isinstance guards on the
manifest and apps values returned by _read_starlark_manifest(); a
manifest.json containing valid non-object JSON (e.g. an array) would
previously raise AttributeError on .get(); invalid shape now returns
(False, 400, ...) instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 14:14:48 -04:00
1a0f1c8015 fix: service control buttons and AP-mode SSH lockout post-install (#326)
* fix: service control buttons and AP-mode SSH lockout post-install

Two user-reported issues after fresh install:

1. All service buttons (Start/Stop/Restart Display, Restart Web Service)
   failed silently — only Reboot worked.

   Root cause: sudoers rules use `ledmatrix.service` (with suffix) but
   api_v3.py called `sudo systemctl start ledmatrix` (no suffix). sudo
   does exact string matching, so every service action was rejected with
   returncode=1. Also missing from sudoers: ledmatrix-web, journalctl,
   and is-active entries.

   Fix:
   - Add `.service` suffix to all 8 sudo systemctl call sites in
     api_v3.py (_ensure_display_service_running, _stop_display_service,
     and all execute_system_action branches).
   - Add timeout=15 to all subprocess.run calls in execute_system_action
     (previously could hang indefinitely).
   - Add missing sudoers rules to first_time_install.sh and
     configure_web_sudo.sh: ledmatrix-web.service start/stop/restart,
     is-active for both name forms, and journalctl -u/-t ledmatrix rules.

2. SSH and web UI became inaccessible after ~1 hour even though the
   display kept running.

   Root cause: wifi_monitor_daemon restarts NetworkManager after 5
   consecutive internet failures (~2.5 min). Each NM restart drops WiFi
   briefly. During that window check_and_manage_ap_mode() increments
   _disconnected_checks but the daemon never reset it after the restart.
   After 3 such NM-restart cycles, _disconnected_checks reached 3 and
   AP mode activated — changing the Pi from WiFi client to hotspot
   (192.168.4.1) and killing SSH on the old IP.

   Fix:
   - Reset wifi_manager._disconnected_checks = 0 in the daemon
     immediately after a successful NM restart so the brief drop it
     causes doesn't count toward AP-mode activation.
   - Increase _disconnected_checks_required from 3 to 6 (90s → 3min)
     as an additional buffer against transient network flaps.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* revert: restore AP-mode grace period to 90s (3 checks)

The counter reset after NM restart already fully prevents the SSH-lockout
cascade: _disconnected_checks can never accumulate across NM restarts
because it is reset to 0 before the next daemon iteration runs.

The 3→6 increase provided no additional fix for the described problem and
caused a UX regression: fresh Pi devices with no WiFi configured would
wait 3 minutes instead of 90 seconds for the LEDMatrix-Setup hotspot to
appear.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address five valid review findings; skip two

Fixed:
- march-madness/requirements.txt: Pillow>=10.3.0 (patches CVE-2024-28219;
  10.3.0 is the actual fix version — reviewer cited 12.2.0 but that risks
  breaking API changes without test coverage)
- wifi_monitor_daemon.py: add missing `import subprocess`; subprocess.run
  and CalledProcessError would NameError at runtime on the NM restart path
- wifi_manager.py: validate ap_idle_timeout_minutes before arithmetic —
  coerce to int, clamp 1–1440, fall back to 15 on bad config values
- wifi_manager.py: call _remove_nm_dnsmasq_captive_conf() on all three
  rollback paths in _enable_ap_mode_nmcli_hotspot() and in the top-level
  except block so stale dnsmasq drop-ins are never left behind
- api_v3.py: fix wrong_password prefix strip — removeprefix("wrong_password:")
  then lstrip() handles both "wrong_password: msg" and "wrong_password:msg"
- plugins_manager.js: add .catch() to loadInstalledPlugins().then() to
  surface failures instead of silently dropping unhandled rejections

Skipped:
- WiFiManager AP state persistence: architectural overhaul; _is_ap_mode_active()
  already derives from live system state, not in-memory variables
- Absolute subprocess paths in api_v3.py: paths vary by distro (/usr/bin vs
  /bin); web service has a normal PATH; sudoers already use resolved paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address five review findings (NM retry loop, start_display message, code quality)

- wifi_monitor_daemon: reset _consecutive_internet_failures = 0 in both
  NM-restart exception handlers; previously both left the counter at threshold,
  causing an immediate retry on the next iteration instead of waiting another
  full backoff period

- api_v3: fix start_display failure message — when mode is set and systemctl
  returns non-zero, message now includes the failure reason and a hint rather
  than always reporting success phrasing

- wifi_manager: move _redirect_backend from class variable to instance variable
  in __init__ alongside _ap_enabled_at; class-level default shadowed correctly
  in practice (single instance) but was misleading

- wifi_manager: narrow broad except Exception in _check_internet_connectivity
  to (subprocess.SubprocessError, OSError) for ping and OSError for HTTP
  (urllib.error.URLError is an OSError subclass in Python 3)

- wifi_manager: remove redundant local 'import re as _re' in _validate_ap_config;
  re is already imported at module level (line 37)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address five review findings (Pillow CVEs, daemon exception narrowing, timeout handling, plugin store)

- march-madness/requirements.txt: Pillow>=12.2.0 (patches CVE-2026-42308
  and CVE-2026-42310; previous floor of 10.3.0 was insufficient)

- wifi_monitor_daemon: narrow final except Exception to
  (subprocess.SubprocessError, OSError) so programming errors in the NM
  restart block are no longer silently swallowed

- api_v3/execute_system_action: add explicit subprocess.TimeoutExpired
  handler before the generic Exception catch; returns action-specific
  message with 'status','message','returncode','stdout','stderr' fields
  so the UI receives a precise, actionable payload instead of the generic
  'Failed to execute system action' string

- plugins_manager.js: move searchPluginStore into .finally() so the
  plugin store renders regardless of whether loadInstalledPlugins succeeds
  or fails; .catch() still logs the error

- first_time_install.sh: add safe_plugin_rm.sh NOPASSWD rule to the
  /tmp/ledmatrix_web_sudoers block; configure_web_sudo.sh had this rule
  but the standalone installer never granted it, leaving plugin removal
  broken after first-time install

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(api): resolve sudo/systemctl/reboot/poweroff paths at startup

Use shutil.which() with safe fallbacks for the four privileged binaries
instead of relying on bare names being resolved by the subprocess shell
search. Resolves paths once at module load rather than per-call.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 17:58:51 -04:00
b361866679 fix(security): escape user-controlled output in plugin action UI (#323)
* style: trim trailing whitespace and fix EOF in plugins_manager.js

Autofix from pre-commit hooks (trailing-whitespace, end-of-file-fixer).
No code logic changes — purely whitespace.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(security): escape user-controlled output in plugin action UI

The plugin action result handlers in executePluginAction() injected
data.message, data.output, and data.auth_url directly into innerHTML
template literals without escaping. A plugin's action handler returning
malicious content (e.g., from a third-party plugin or compromised
upstream) could execute arbitrary JavaScript in the web UI context.

Wrap user-controlled strings in escapeHtml() at all four sites:
- Step-2 (continuation) error path (message + output)
- OAuth flow auth_url (link href + display text, with http:// guard)
- Step-1 simple-success output
- Step-1 failure path (message + output)

The escapeHtml() helper is already defined in this file and used
elsewhere (validation errors, plugin store cards).

Co-Authored-By: 5ymb01 <5ymb01@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: 5ymb01 <5ymb01@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-06 10:12:31 -04:00
ceb4c4105f fix(wifi): reliable open AP with captive portal — tested on Trixie Pi (#320)
* fix(wifi): create truly open AP via nmcli connection add; add captive portal to nmcli path

nmcli device wifi hotspot always attaches a WPA2 PSK on Bookworm/Trixie
and silently ignores post-creation security modifications, causing users
to be prompted for an unknown password. Switch to nmcli connection add
with 802-11-wireless.mode ap and no security section — NM cannot auto-add
a password to a profile that has no 802-11-wireless-security block.

Also:
- Remove dead DEFAULT_AP_PASSWORD / ap_password config field (stored but
  never passed to hostapd or nmcli, causing user confusion)
- Add iptables port 80→5000 redirect to the nmcli AP path so captive portal
  auto-popup works on phones without hostapd (previously only worked on
  the hostapd path)
- Clean up iptables rules on disable for the nmcli path
- Improve LED message on AP enable: show SSID, "No password", and IP:port
  on both paths so users know exactly how to connect
- Fix systemd template: replace hardcoded /home/ledpi/LEDMatrix/ with
  __PROJECT_ROOT_DIR__ placeholder (install script already writes correct path)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): address Codacy review findings in AP mode implementation

- Validate ap_ssid/ap_channel from config before passing to subprocess
  (printable ASCII ≤32 chars; channel 1-14) to prevent command injection

- Fix INPUT iptables rule: PREROUTING redirects port 80→5000 so the INPUT
  chain sees dport=5000, not 80. Old INPUT rule on port 80 was a no-op.

- Refactor iptables setup/teardown into _setup_iptables_redirect() and
  _teardown_iptables_redirect() helpers, eliminating duplicate logic in
  the hostapd and nmcli paths

- Save/restore ip_forward state (via /tmp/ledmatrix_ip_forward_saved)
  instead of forcing it to 0 on cleanup, which could break VPNs or
  bridges already relying on forwarding

- nmcli path skips ip_forward management entirely: NM's ipv4.method=shared
  already manages it for the duration of the connection

- Fix _get_ap_status_nmcli() verification: new 'connection add type wifi'
  profiles have type '802-11-wireless', not 'hotspot', so verification was
  always returning False. Now also matches by our known connection name.

- Remove SSID-based connection deletion: deleting any profile whose SSID
  matched the AP SSID could destroy a user's saved home WiFi profile.
  Now only deletes by our application-managed profile names.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(plugins): fix async race in refreshPlugins; use cache TTL to gate re-swap metadata fetch

refreshPlugins() called searchPluginStore(true) and showNotification() immediately
after refreshInstalledPlugins() without awaiting the returned Promise, so
window.installedPlugins could still be stale when the store rendered its
Installed/Reinstall badges. Chain .then() so both run only after the fetch
completes.

In initializePlugins(), the re-swap path always passed fetchCommitInfo=false to
searchPluginStore, skipping GitHub metadata even when the 5-minute cache TTL had
expired. Add storeCacheExpired() helper and compute isReswapWarm = _reswap &&
!storeCacheExpired() so fresh metadata is fetched whenever the cache is cold,
regardless of whether the render is a first load or a tab re-swap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address three wifi_manager and one plugins_manager review findings

wifi_manager.py:
- _create_hostapd_config: use _validate_ap_config() for ssid/channel instead
  of raw self.config values; strip newlines from SSID to prevent config-file
  injection via the generated hostapd.conf
- _setup_iptables_redirect: check return codes of sysctl ip_forward enable and
  both iptables -A calls; on any failure log the error output, call
  _teardown_iptables_redirect() to restore state, and return False instead of
  silently succeeding
- _enable_ap_mode_nmcli_hotspot: on AP verification failure roll back fully —
  tear down iptables redirect, delete the LEDMatrix-Setup-AP connection profile,
  clear the LED message — before returning False

plugins_manager.js:
- initializePlugins: chain searchPluginStore(!isReswapWarm) inside
  loadInstalledPlugins().then() so window.installedPlugins is populated before
  the store renders Installed/Reinstall badges (same pattern applied to
  refreshPlugins() in the previous commit)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): use _find_command_path for iptables/sysctl; harden ip_forward save/restore

Add _find_command_path() helper that extends _check_command()'s sbin-aware lookup to
return the absolute binary path rather than a boolean. Use it in
_setup_iptables_redirect and _teardown_iptables_redirect so iptables and sysctl are
resolved via /sbin or /usr/sbin even when those directories are absent from PATH in
systemd service environments.

Also harden the ip_forward save/restore logic:
- Read ip_forward from /proc/sys/net/ipv4/ip_forward (no subprocess, no PATH
  dependency) instead of spawning sysctl -n
- Skip the sysctl -w ip_forward=1 write when the value is already "1" to avoid
  mutating state owned by another service (VPN, NM shared mode, bridge)
- Track save success via presence of the save file: if the /proc read or file write
  fails, leave the file absent so teardown knows not to restore
- In _teardown_iptables_redirect, only restore ip_forward when the save file exists;
  if absent, leave the current value untouched rather than forcing "0"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): check _setup_iptables_redirect return; fix hostapd LED SSID; teardown on exception

- Both AP startup paths (hostapd and nmcli) now check the bool returned by
  _setup_iptables_redirect() and treat False as a hard failure: the hostapd
  path stops hostapd/dnsmasq and returns an error tuple; the nmcli path brings
  down and deletes the LEDMatrix-Setup-AP profile and clears the LED message

- _enable_ap_mode_hostapd's LED message now calls _validate_ap_config() to get
  the same sanitized SSID that _create_hostapd_config() uses, so the displayed
  name always matches the AP actually broadcast by hostapd

- _setup_iptables_redirect's outer except block now calls
  _teardown_iptables_redirect() before returning False so partial iptables/
  ip_forward state is always cleaned up on unexpected exceptions; cleanup
  exceptions are caught and logged separately

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(wifi): add unit tests for AP mode — open network, iptables, LED, cleanup ordering

Six pytest unit tests covering the five review scenarios. All subprocess and
filesystem side-effects are mocked so the tests run without root, hardware, or
a Pi OS environment.

1. test_nmcli_ap_profile_has_no_security_params — asserts the nmcli connection
   add command has no key-mgmt / psk / WPA arguments and sets mode=ap.
2. test_iptables_nat_rules_added_on_ap_start — verifies _setup_iptables_redirect
   emits a PREROUTING REDIRECT 80→5000 rule and an INPUT ACCEPT rule for port
   5000 (not 80, which never hits INPUT after PREROUTING rewrites it).
3. test_iptables_rules_and_ip_forward_reverted_on_teardown — verifies the -D
   PREROUTING/-D INPUT calls and that sysctl restores the saved ip_forward value
   and removes the save file.
4. test_ip_forward_not_restored_when_save_file_absent — verifies teardown skips
   sysctl when the save file was never written, preventing blind ip_forward=0 on
   systems using ip_forward for VPNs or NM shared mode.
5. test_led_message_shows_ssid_no_password_and_url — asserts the LED message
   includes the SSID, 'No password', and the 192.168.4.1:5000 setup URL.
6. test_existing_ap_profiles_deleted_before_new_profile_created — asserts all
   known profile names are targeted for deletion before 'nmcli connection add'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(wifi): adopt adsb-feeder-image hotspot patterns — DNS spoofing, connectivity check, idle timeout, wrong-password UX, watchdog escalation

Inspired by the production-proven approach in dirkhh/adsb-feeder-image.

1. DNS spoofing for automatic captive-portal popup (Change 1 — Critical)
   Write /etc/NetworkManager/dnsmasq-shared.d/ledmatrix-captive.conf with
   address=/#/192.168.4.1 before nmcli connection up so NM's built-in
   dnsmasq (ipv4.method=shared) resolves every hostname to the AP IP.
   This triggers the OS captive-portal popup automatically on iOS / Android /
   Windows / macOS — no manual navigation to 192.168.4.1:5000/setup required.
   New helpers: _write_nm_dnsmasq_captive_conf / _remove_nm_dnsmasq_captive_conf.
   New constants: NM_DNSMASQ_SHARED_DIR / NM_DNSMASQ_SHARED_CONF.

2. Real internet connectivity check (Change 2 — High)
   Add _check_internet_connectivity() (ping 8.8.8.8 + HTTP fallback).
   check_and_manage_ap_mode() now considers a device "disconnected" when nmcli
   shows connected but no real internet reachability, matching adsb-feeder's
   multi-method gateway/DNS/HTTP test approach.

3. AP idle timeout (Change 3 — Medium)
   Track _ap_enabled_at timestamp in enable_ap_mode(). Add _has_ap_clients()
   using 'iw dev <iface> station dump'. check_and_manage_ap_mode() auto-disables
   AP after ap_idle_timeout_minutes (default 15) with no associated clients.

4. Wrong-password error feedback (Change 4 — Medium)
   _connect_nmcli() detects "Secrets were required" / "authentication rejected"
   in nmcli stderr and prefixes the message with "wrong_password: ".
   The /api/v3/wifi/connect route propagates error_type="wrong_password" in the
   JSON response. captive_setup.html shows "Incorrect password — try again"
   (keeping the form active) instead of the generic failure message.

5. Escalating watchdog NM restart (Change 5 — Low)
   wifi_monitor_daemon.py tracks _consecutive_internet_failures. After
   _nm_restart_threshold (5) consecutive checks where nmcli shows connected but
   internet is unreachable, restart NetworkManager as a recovery step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): restore safe AP-enable trigger; decouple internet check from AP logic

The previous commit introduced _check_internet_connectivity() into
check_and_manage_ap_mode(), which shared the same _disconnected_checks counter
that triggers AP enable. This created a false-positive risk: 90 seconds of
packet loss on working WiFi would enable AP mode and kick off the connection.

Fix: restore nmcli association state as the sole AP-enable trigger (original,
safe behaviour). The internet connectivity check is now used only in the daemon
watchdog for the NM-restart escalation — matching how adsb-feeder-image actually
structures the two concerns (initial setup detection vs. ongoing monitoring).

Also clarify daemon comment: the connectivity check runs once per cycle in the
watchdog block, not inside check_and_manage_ap_mode, so there is no double-call.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): remove PMF setting from open AP profile — breaks nmcli connection add on Trixie NM 1.52+

802-11-wireless-security.pmf is only valid within a security section that also
includes key-mgmt. Adding it to an open-network profile causes NM 1.52+ to
reject the connection add with 'key-mgmt: property is missing'. PMF has no
meaning for open APs (it only applies to WPA2/WPA3), so the setting is simply
removed rather than worked around.

Found by testing on devpi (Trixie, NM 1.52.1).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): add nftables fallback for port redirect; graceful degradation when neither available

Tested on devpi (Trixie, NM 1.52.1): iptables is not installed; nftables is.
The original code called _setup_iptables_redirect() and treated 'iptables not
found' as a hard failure, rolling back the entire AP setup.

Changes:
- _setup_iptables_redirect() now tries iptables first, then nftables as a
  fallback. When neither is available it logs a warning and returns True so
  the AP still comes up (DNS spoofing still triggers the captive portal popup;
  users land on port 5000 directly instead of being auto-redirected from 80).
- Split into _setup_iptables_redirect_iptables() and
  _setup_iptables_redirect_nftables() for clarity.
- Added _redirect_backend instance var ("iptables" | "nftables" | None) so
  _teardown_iptables_redirect() uses the same tool that setup used.
- nftables teardown: deletes the 'ledmatrix' table (clean, no leftover rules).
- iptables teardown: unchanged logic (ip_forward save/restore).
- Also removed the PMF workaround for Trixie: 802-11-wireless-security.pmf
  requires key-mgmt to also be set, breaking open-network creation on NM 1.52+.
  Open APs have no management frame protection by definition.
- Update teardown test to set _redirect_backend = "iptables" before calling it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wifi): public check_internet_connectivity(); absolute systemctl path; stricter mode assertion

wifi_manager.py:
- Add public check_internet_connectivity() wrapping the private method so the
  daemon does not reach into the private API

wifi_monitor_daemon.py:
- Call wifi_manager.check_internet_connectivity() instead of the private
  _check_internet_connectivity()
- Use /usr/bin/systemctl (absolute path) instead of bare "systemctl"
- Wrap NM restart in try/except with check=True; only reset
  _consecutive_internet_failures on success — on CalledProcessError or other
  exception, log the error and leave the counter unchanged so the next cycle
  retries

test/test_wifi_manager_ap.py:
- Replace loose `assert "ap" in add_calls[0]` (list-membership check that
  could be satisfied by any element equal to "ap") with an explicit key/value
  check: locate "802-11-wireless.mode" in the command list and assert the next
  element is exactly "ap"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 08:25:20 -04:00
ChuckandGitHub e9af18cdf1 Add Codacy badge to README (#322)
Added a Codacy badge to the README for code quality.

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-05-01 09:41:50 -04:00
260 changed files with 24000 additions and 6372 deletions
+7
View File
@@ -0,0 +1,7 @@
---
exclude_paths:
- "plugin-repos/**"
- "plugins/**"
- "assets/**"
- "test/**"
- "scripts/debug/**"
+44
View File
@@ -0,0 +1,44 @@
name: Claude Code Review
on:
pull_request:
types: [opened, synchronize, ready_for_review, reopened]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
jobs:
claude-review:
# Optional: Filter by PR author
# if: |
# github.event.pull_request.user.login == 'external-contributor' ||
# github.event.pull_request.user.login == 'new-developer' ||
# github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
plugin_marketplaces: 'https://github.com/anthropics/claude-code.git'
plugins: 'code-review@claude-code-plugins'
prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }}'
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
+50
View File
@@ -0,0 +1,50 @@
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
# prompt: 'Update the pull request description to include a summary of changes.'
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://code.claude.com/docs/en/cli-reference for available options
# claude_args: '--allowed-tools Bash(gh pr *)'
+33
View File
@@ -0,0 +1,33 @@
name: Tests
on:
pull_request:
push:
branches: [main]
jobs:
plugin-safety:
name: Plugin safety harness + unit tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: "3.12"
cache: pip
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt -r requirements-test.txt
pip install RGBMatrixEmulator
- name: Run harness + visual rendering tests
run: |
pytest --no-cov \
test/plugins/test_harness.py \
test/plugins/test_visual_rendering.py \
test/plugins/test_plugin_matrix.py
+1
View File
@@ -8,6 +8,7 @@ config/config_secrets.json
config/config.json
config/config.json.backup
config/wifi_config.json
config/uninstalled_plugins.json
credentials.json
token.pickle
+1
View File
@@ -1,3 +1,4 @@
[submodule "rpi-rgb-led-matrix-master"]
path = rpi-rgb-led-matrix-master
url = https://github.com/hzeller/rpi-rgb-led-matrix.git
branch = master
+14 -3
View File
@@ -1,4 +1,10 @@
# LEDMatrix
[![License](https://img.shields.io/badge/license-GPL--3.0-green)](LICENSE)
[![Discord](https://img.shields.io/badge/Discord-community-5865F2?logo=discord&logoColor=white)](https://discord.gg/RdrC37rEag)
[![GitHub Stars](https://img.shields.io/github/stars/ChuckBuilds/ledmatrix?style=flat&color=yellow)](https://github.com/ChuckBuilds/ledmatrix)
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/77fc9b446a5948e5b0aed7a7aaeb1bab)](https://app.codacy.com/gh/ChuckBuilds/LEDMatrix/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)
## Welcome to LEDMatrix!
Welcome to the LEDMatrix Project! This open-source project enables you to run an information-rich display on a Raspberry Pi connected to an LED RGB Matrix panel. Whether you want to see your calendar, weather forecasts, sports scores, stock prices, or any other information at a glance, LEDMatrix brings it all together.
@@ -126,10 +132,15 @@ The system supports live, recent, and upcoming game information for multiple spo
| This project can be finnicky! RGB LED Matrix displays are not built the same or to a high-quality standard. We have seen many displays arrive dead or partially working in our discord. Please purchase from a reputable vendor. |
### Raspberry Pi
- Raspberry Pi Zero's don't have enough processing power for this project and the Pi 5 is unsupported due to new GPIO output.
- **Raspberry Pi 3B or 4 (NOT RPi 5!)**
- Raspberry Pi Zero's don't have enough processing power for this project.
- **Raspberry Pi 3B, 4, or 5**
[Amazon Affiliate Link Raspberry Pi 4 4GB RAM](https://amzn.to/4dJixuX)
[Amazon Affiliate Link Raspberry Pi 4 8GB RAM](https://amzn.to/4qbqY7F)
- **Pi 5 users**: the installer automatically detects Pi 5 and builds the `rpi-rgb-led-matrix` library with RP1 support. If you previously installed on a Pi 4 and migrated the SD card, or if you see `mmap` errors in the logs, force a fresh library build:
```bash
sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh
```
- Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and set `gpio_slowdown` to `1` or `2`.
### RGB Matrix Bonnet / HAT
@@ -581,7 +592,7 @@ These settings control runtime behavior and GPIO timing:
- **Critical setting**: Must match your Raspberry Pi model for stability
- **Raspberry Pi 3**: Use 3
- **Raspberry Pi 4**: Use 4
- **Raspberry Pi 5**: Use 5 (or higher if needed)
- **Raspberry Pi 5**: Use 12 in PIO mode (`rp1_rio: 0`, the default); start with `1` and increase if you see flickering
- **Raspberry Pi Zero/1**: Use 1-2
- Incorrect values can cause display corruption, flickering, or system instability
- If you experience issues, try adjusting this value up or down by 1
Binary file not shown.

Before

Width:  |  Height:  |  Size: 48 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

After

Width:  |  Height:  |  Size: 109 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 105 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 101 KiB

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.8 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 103 KiB

After

Width:  |  Height:  |  Size: 126 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 111 KiB

After

Width:  |  Height:  |  Size: 140 KiB

+30 -19
View File
@@ -1,43 +1,43 @@
{
"web_display_autostart": true,
"schedule": {
"enabled": true,
"enabled": false,
"mode": "per-day",
"start_time": "07:00",
"end_time": "23:00",
"days": {
"monday": {
"enabled": true,
"enabled": false,
"start_time": "07:00",
"end_time": "23:00"
},
"tuesday": {
"enabled": true,
"enabled": false,
"start_time": "07:00",
"end_time": "23:00"
},
"wednesday": {
"enabled": true,
"enabled": false,
"start_time": "07:00",
"end_time": "23:00"
},
"thursday": {
"enabled": true,
"enabled": false,
"start_time": "07:00",
"end_time": "23:00"
},
"friday": {
"enabled": true,
"enabled": false,
"start_time": "07:00",
"end_time": "23:00"
},
"saturday": {
"enabled": true,
"enabled": false,
"start_time": "07:00",
"end_time": "23:00"
},
"sunday": {
"enabled": true,
"enabled": false,
"start_time": "07:00",
"end_time": "23:00"
}
@@ -51,46 +51,46 @@
"end_time": "07:00",
"days": {
"monday": {
"enabled": true,
"enabled": false,
"start_time": "20:00",
"end_time": "07:00"
},
"tuesday": {
"enabled": true,
"enabled": false,
"start_time": "20:00",
"end_time": "07:00"
},
"wednesday": {
"enabled": true,
"enabled": false,
"start_time": "20:00",
"end_time": "07:00"
},
"thursday": {
"enabled": true,
"enabled": false,
"start_time": "20:00",
"end_time": "07:00"
},
"friday": {
"enabled": true,
"enabled": false,
"start_time": "20:00",
"end_time": "07:00"
},
"saturday": {
"enabled": true,
"enabled": false,
"start_time": "20:00",
"end_time": "07:00"
},
"sunday": {
"enabled": true,
"enabled": false,
"start_time": "20:00",
"end_time": "07:00"
}
}
},
"timezone": "America/Chicago",
"timezone": "America/New_York",
"location": {
"city": "Dallas",
"state": "Texas",
"city": "Tampa",
"state": "Florida",
"country": "US"
},
"display": {
@@ -112,7 +112,13 @@
"limit_refresh_rate_hz": 100
},
"runtime": {
"gpio_slowdown": 3
"gpio_slowdown": 3,
"rp1_rio": 0
},
"double_sided": {
"enabled": false,
"copies": 2,
"axis": "horizontal"
},
"display_durations": {},
"use_short_date_format": true,
@@ -126,6 +132,11 @@
"buffer_ahead": 2
}
},
"sync": {
"role": "standalone",
"port": 5765,
"follower_position": "left"
},
"plugin_system": {
"plugins_directory": "plugin-repos",
"auto_discover": true,
+234
View File
@@ -0,0 +1,234 @@
# Adaptive Layout & Font Scaling
`src/adaptive_layout.py` lets a plugin render legibly on **any** panel size
(64x32, 128x32, 96x48, 128x64, 256x64, ...) without hand-tuned per-display
layouts. It is **opt-in**: nothing changes for plugins that don't use it.
It generalizes three patterns proven in the plugin ecosystem:
| Pattern | Origin | Core API |
|---|---|---|
| Geometry scale factor vs. a design size | f1-scoreboard | `ctx.px(base)` / `ctx.scale` |
| Breakpoint tiers | masters-tournament | `ctx.tier` / `ctx.by_tier({...})` |
| "Largest crisp font that fits" ladder | baseball-scoreboard | `ctx.fit_text(...)` and friends |
## Quick start
Every `BasePlugin` has a lazy `self.layout` (a `LayoutContext` for the
current logical display size, rebuilt automatically if the size changes)
and a one-liner `self.draw_fit(...)`:
```python
def display(self, force_clear=False):
from src.adaptive_layout import LADDER_ARCADE
b = self.layout.bounds.inset(1) # Region(0,0,W,H) minus 1px margin
rows = b.split_v(3, 1, 1, gap=1) # 3/5 for time, 1/5 each for the rest
self.draw_fit(self.time_str, rows[0], ladder=LADDER_ARCADE)
self.draw_fit(self.weekday, rows[1]) # default LADDER_GRID
self.draw_fit(self.date_str, rows[2])
self.display_manager.update_display()
```
On 128x64 the time renders at press_start 24px; on 64x32 it steps down to
8px. The rows partition the height, so bands can never overlap — no more
`y = height - 7` magic numbers.
## Region — rect algebra
`Region(x, y, w, h)` is a frozen dataclass. All carving clamps to
non-negative dimensions, so degenerate panels behave.
- Carving: `inset(dx, dy)`, `top_band(h)`, `bottom_band(h)`,
`middle(top_h, bottom_h)`, `left_col(w)`, `right_col(w)`,
`split_h(*weights, gap=0)`, `split_v(*weights, gap=0)`
- Placement: `align_xy(w, h, align, valign)`, `center_xy(w, h)`,
`contains(w, h)`, `.center`, `.right`, `.bottom`
Scoreboard-style layout:
```python
b = self.layout.bounds
status = b.top_band(self.layout.px(7))
detail = b.bottom_band(self.layout.px(7))
score_area = b.middle(status.h, detail.h)
away_slot, home_slot = b.left_col(b.h), b.right_col(b.h)
```
## Font ladders — discrete, never fractional
Pixel fonts (BDF, PressStart2P) only look right at native/integer sizes, so
fonts are never scaled continuously. A `FontLadder` is an ordered tuple of
`FontStep(family, size_px)` rungs, largest first; fitting walks down until
the measured text fits.
- `LADDER_GRID` (default): X11 BDFs at native sizes — 10x20 → 9x18 → 9x15 →
8x13 → 7x13 → 6x13 → 6x12 → 6x10 → 6x9 → 5x8 → 5x7 → 4x6 → tom-thumb.
Body text, labels, multi-row content.
- `LADDER_ARCADE`: PressStart2P at 32/24/16/8 (integer multiples of its 8px
grid). Headline text: clocks, scores.
Custom ladders are just tuples — e.g. to add your plugin's registered font
on top: `(FontStep("myplugin::digits", 16),) + LADDER_GRID`.
## LayoutContext
Built per (width, height); exposes facts and fit queries:
- `bounds`, `width`, `height`, `aspect`
- `tier` by height (`xs`≤16, `sm`≤32, `md`≤48, `lg`≤64, `xl`) and
`width_tier` (`narrow`≤64, `normal`≤128, `wide`≤256, `ultrawide`)
- `is_wide_short` — aspect ≥ 2.5 and height ≤ 32 (the classic 128x32 shape)
- `scale``min(w/design_w, h/design_h)` vs. your manifest's
`display.design_size` (default 128x32). **Geometry only** — gaps, icon
and logo sizes via `px(base, minimum, maximum)`; fonts use ladders.
- `by_tier({"sm": 10, "lg": 18})` — value for the nearest defined tier
at-or-below the panel's tier.
- `fit_text(text, box, ladder, ellipsis=True)``FitResult` — largest rung
that fits; ellipsizes as a last resort. Cached per (text, box, ladder).
- `fit_text_proportional(text, box, base_size_px, ladder, ellipsis=True, scale=None)`
rung closest to (not exceeding) `base_size_px * scale`, still capped to
what fits the box. Use this instead of `fit_text` when several
independently-fitted elements need to stay visually harmonious as the
panel grows — `fit_text` maximizes *each one* within its own region,
which can make one element (e.g. a score with a generous box) balloon
out of proportion to a neighbor that scales by geometry (e.g. logos
sized via `px()`), even though each individual pick is "correct" in
isolation. `base_size_px` is normally the element's existing classic/
fixed font size. `scale` defaults to `self.scale` (the conservative
min-of-both-axes factor `px()` uses); pass an axis-specific value when
the surrounding composition already scales that way — e.g. a scoreboard
whose logo slots track height alone (`min(height, width // 2)`) should
size its text by `height / design_height` too, or the text reads as
under-scaled next to bigger logos on a panel that only grew taller.
- `fit_lines(lines, box, ladder, spacing)` — every line fits the width and
the stack fits the height (measures the actual strings).
- `font_for_rows(rows, box_h, ladder)` — largest rung whose line height
fits `rows` rows.
`FitResult` carries the ready-to-use `font` (drops straight into
`display_manager.draw_text(font=...)`), the possibly-ellipsized `text`,
ink `width`/`height`, `baseline`, `y_offset`, `line_height`, and `fits`.
## Adaptive images
`src/adaptive_images.py` is the image counterpart to `fit_text`, exposed as
`self.layout.fit_image(...)` (cached per panel size) and the one-liner
`self.draw_image(...)`:
```python
# Team logo: trim its transparent padding, fill the slot height (the
# football/hockey pattern), cached across frames by a stable key
self.draw_image(logo, regs.away_slot, mode="fill_height",
crop_to_ink=True, cache_key=f"logo:{abbr}")
# Album art: cover-crop a square, faces kept by the top anchor
self.draw_image(art, row.art, mode="cover", anchor="top")
# Pixel flags / sprite icons: NEAREST keeps hard edges
from src.adaptive_images import RESAMPLE_NEAREST
self.draw_image(flag, box, resample=RESAMPLE_NEAREST)
```
Modes: `contain` (letterbox, default), `cover` (crop-to-fill),
`fill_height` (logo-style), `stretch`. Unlike PIL's `thumbnail()`
(downscale-only — why imagery stays tiny on big panels) fitting **upscales
by default**; pass `upscale=False` for the legacy behavior. Results are
cached per (image, box size, options) with a bounded LRU — always pass a
stable `cache_key` (e.g. `"logo:KC"`) for images you reload. The module
also exports the Pillow-compat `RESAMPLE_LANCZOS`/`RESAMPLE_NEAREST`
constants so plugins can drop their local shims.
## Composite layouts
Pre-carved Region arrangements for the layouts plugins keep rebuilding:
```python
from src.adaptive_layout import scoreboard_regions, media_row
regs = scoreboard_regions(self.layout.bounds, ctx=self.layout)
# regs.away_slot / home_slot — logo slots (logo_slot = min(H, W // 2),
# capped so a center reserve always exists —
# see below)
# regs.status_band — top band (replaces the magic y = 1)
# regs.score_area — center gap, plus a controlled bleed into
# each logo slot (replaces y = H//2 - 3)
# regs.detail_band — bottom band (replaces y = H - 7)
# regs.bottom_left / bottom_right — record/timeout corners
row = media_row(self.layout.bounds, ctx=self.layout) # art left, text right
```
Both work on the full panel or on a scroll-mode card Region. They return
Regions and never draw — compose them with `draw_fit`/`draw_image`.
**`scoreboard_regions`'s center reserve.** The raw `logo_slot = min(H, W//2)`
formula has a blind spot: at exactly 2:1 aspect ratio (width = 2×height —
two, four, or more square modules stacked into a taller panel, e.g.
96x48, 128x64, 256x128) the two logo slots mathematically claim the
*entire* width, leaving zero pixels for a center column no matter how
big the panel gets. Wide panels (the 128x32 design baseline, 192x48,
256x32) never hit this, since height is already the tighter constraint
there. Two parameters fix it without any plugin-side code:
`min_center_fraction`/`min_center_design_px` guarantee a real minimum
center reserve at any aspect ratio, and `score_bleed_fraction` lets the
score's *fit box* extend a controlled amount into each logo slot — the
same way a real broadcast scoreboard's numbers cross slightly into the
team marks flanking them — so a short score string never has to truncate
even on the tightest aspect ratios. All three have sane defaults; override
them per call if a plugin's card proportions genuinely differ.
## Preserving user customization
Adaptive layout supplies *defaults*; explicit user configuration wins:
- **User-set fonts win.** If the plugin's config has an explicit
`font`/`font_size` for an element, load it as before and skip the ladder —
fit only when the user hasn't overridden (see the football-scoreboard
`_resolve_element_fit` pattern).
- **Offsets apply on top.** `customization.layout.<element>.{x_offset,y_offset}`
style knobs translate the *computed* region as a final step:
`region.offset(user_dx, user_dy)`. `draw_image(..., offset=(dx, dy))`
does the same for images.
- **Colors pass through.** `draw_fit`/`draw_fitted_text` take explicit
`color=` params; adaptive mode never repaints semantic or user-chosen
colors.
## Manifest declaration
Declare the size your layout was authored against so `ctx.scale` means
something:
```json
"display": { "design_size": { "width": 128, "height": 32 } }
```
Also available under `requires.display_size`: `min_width`, `min_height`,
`max_width`, `max_height`.
## Performance notes (Pi)
Fit queries are cached, so cost is O(unique strings). For per-second text
(clocks, live scores), fit on a **shape placeholder** and reuse the font:
```python
fit = self.layout.fit_text("00:00", box, ladder=LADDER_ARCADE) # cached once
self.display_manager.draw_text(current_time, font=fit.font, ...)
```
## Testing across sizes
The harness already renders every plugin at a spread of sizes (now
including 96x48):
```bash
python scripts/check_plugin.py <plugin-dir> --sizes 64x32,128x32,96x48,128x64,256x64
python scripts/render_plugin.py <plugin-dir> --width 96 --height 48
```
`BoundsCheckingDisplayManager` flags right/bottom overflow and now records
mediated draw calls with negative coordinates in
`negative_coordinate_calls` (raw-PIL draws remain uncovered).
Reference migration: the **text-display** plugin's `font_mode: "auto"`.
+6
View File
@@ -2,6 +2,12 @@
Advanced patterns, examples, and best practices for developing LEDMatrix plugins.
> **Adaptive layout:** for plugins that should render legibly on any panel
> size (fonts that grow on big panels, layouts that degrade gracefully on
> small ones), use the adaptive layout system — `self.layout`, `draw_fit`,
> `draw_image`, `scoreboard_regions` — documented in
> [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md).
## Table of Contents
- [Using Weather Icons](#using-weather-icons)
+6
View File
@@ -48,6 +48,12 @@ display_manager.draw_text("Centered", centered=True) # Auto-center
width = display_manager.get_text_width("Text", font)
height = display_manager.get_font_height(font)
# Adaptive layout (recommended for multi-size support — text and images
# that scale to any panel; see docs/ADAPTIVE_LAYOUT.md)
rows = self.layout.bounds.inset(1).split_v(3, 1, gap=1)
self.draw_fit("12:34", rows[0]) # largest crisp font that fits
self.draw_image(logo, rows[1], mode="fill_height", crop_to_ink=True)
# Weather icons
display_manager.draw_weather_icon("rain", x=10, y=10, size=16)
+6
View File
@@ -6,6 +6,12 @@ Tools for rapid plugin development without deploying to the RPi.
Interactive web UI for tweaking plugin configs and seeing the rendered display in real time.
The size inputs have a preset dropdown with the harness's standard panel
sizes, and the **All Sizes** button renders the current config at every
harness size in a side-by-side gallery (`POST /api/render-matrix`) — the
quickest way to eyeball adaptive-layout behavior across panels
(see [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md)).
### Quick Start
```bash
+7
View File
@@ -1,5 +1,12 @@
# FontManager Usage Guide
> **Picking a size automatically:** if you want the *largest font that fits
> a given area* rather than a fixed size, use the adaptive layout system's
> font ladders, which resolve through this FontManager. `BasePlugin`
> subclasses get this as `self.layout.fit_text(...)`; other code can build
> a `LayoutContext(width, height, font_manager)` directly — see
> [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md).
## Overview
The enhanced FontManager provides comprehensive font management for the LEDMatrix application with support for:
+5
View File
@@ -2,6 +2,11 @@
Complete API reference for plugin developers. This document describes all methods and properties available to plugins through the Display Manager, Cache Manager, and Plugin Manager.
> **Adaptive layout:** every `BasePlugin` also exposes `self.layout`,
> `self.draw_fit(text, region)` and `self.draw_image(img, region, ...)`
> the recommended way to render text and images that scale to any panel
> size. See [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md).
## Table of Contents
- [BasePlugin](#baseplugin)
+10 -10
View File
@@ -34,16 +34,16 @@ This document outlines the transformation of the LEDMatrix project into a modula
## Table of Contents
1. [Current Architecture Analysis](#current-architecture-analysis)
2. [Plugin System Design](#plugin-system-design)
3. [Plugin Store & Discovery](#plugin-store--discovery)
4. [Web UI Transformation](#web-ui-transformation)
5. [Migration Strategy](#migration-strategy)
6. [Plugin Developer Guidelines](#plugin-developer-guidelines)
7. [Technical Implementation Details](#technical-implementation-details)
8. [Best Practices & Standards](#best-practices--standards)
9. [Security Considerations](#security-considerations)
10. [Implementation Roadmap](#implementation-roadmap)
1. [Current Architecture Analysis](#1-current-architecture-analysis)
2. [Plugin System Design](#2-plugin-system-design)
3. [Plugin Store & Discovery](#3-plugin-store--discovery)
4. [Web UI Transformation](#4-web-ui-transformation)
5. [Migration Strategy](#5-migration-strategy)
6. [Plugin Developer Guidelines](#6-plugin-developer-guidelines)
7. [Technical Implementation Details](#7-technical-implementation-details)
8. [Best Practices & Standards](#8-best-practices--standards)
9. [Security Considerations](#9-security-considerations)
10. [Implementation Roadmap](#10-implementation-roadmap)
---
+8
View File
@@ -2,6 +2,14 @@
This guide explains how to set up a development workflow for plugins that are maintained in separate Git repositories while still being able to test them within the LEDMatrix project.
> **Rendering guidance:** plugins should read the display size dynamically
> (`self.display_manager.matrix.width/height`) rather than hardcoding one
> panel. For plugins that want to *scale* their layout to any panel, the
> opt-in adaptive layout system ([ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md))
> provides the shared helpers — fonts, images, and composite layouts that
> scale. Existing plugins keep their classic rendering unless they adopt
> those APIs; nothing migrates automatically.
## Overview
When developing plugins in separate repositories, you need a way to:
+136
View File
@@ -0,0 +1,136 @@
# Plugin Safety Harness
Renders a plugin across **every declared screen (mode)** and **a spread of
matrix sizes**, and fails if any combination crashes, draws past the panel edge,
or — for plugins that ship golden images — drifts visually. The goal: change a
plugin without breaking a size or screen you didn't think to test.
## Sizes: a sample, not a fixed list
There is **no fixed set of supported panel sizes** — an RGB matrix build can be
any width/height and configuration (square, rectangle, 2×2, 4×4, 8×2, long
strips, tall stacks). Plugins are expected to read dimensions dynamically
(`self.display_manager.matrix.width/height`) and lay themselves out
accordingly, so a hardcoded coordinate or unscaled font shows up as a failure
here.
The harness therefore renders against a **representative sample** that spans the
axes of variation (`DEFAULT_TEST_SIZES` in `src/plugin_system/testing/sizes.py`),
not an authoritative list:
Each module is 64×32; entries are real panel-grid arrangements (cols × rows):
| Size | Grid | Why it's in the sample |
|---------|------|--------------------------------------------|
| 64×32 | 1×1 | single panel — tightest common rectangle |
| 128×32 | 2×1 | the baseline most plugins are tuned for |
| 64×64 | 1×2 | stacked — tall-narrow centering |
| 128×64 | 2×2 | block — icon scaling / vertical centering |
| 256×32 | 4×1 | long strip — wide horizontal layout |
| 128×96 | 2×3 | tall — vertical overflow |
| 256×128 | 4×4 | large block — both dimensions big at once |
**Override the sizes entirely** to test your actual hardware (or any shape):
```bash
# CLI — one-off:
python scripts/check_plugin.py --plugin clock-simple --sizes 8x16,64x64,256x32
# pytest — force every plugin onto your panel(s):
LEDMATRIX_TEST_SIZES="8x16,128x128" pytest test/plugins/test_plugin_matrix.py
# Per-plugin — declare the shapes a plugin targets in its test/harness.json:
# { "sizes": [[8, 16], [64, 64]] }
```
Precedence: `LEDMATRIX_TEST_SIZES` env (global) → per-plugin `harness.json`
`sizes` → the default sample. Bounds checking adapts to whatever sizes a run
uses — the backing canvas is padded out to the **largest** panel in the run, so
a coordinate meant for a big build is still caught when rendering a small one.
## Quick start
```bash
# Functional + bounds check across all sizes/screens:
python scripts/check_plugin.py --plugin clock-simple
# Every discovered plugin:
python scripts/check_plugin.py --all
# Dump PNGs to eyeball each size/screen:
python scripts/check_plugin.py --plugin ledmatrix-weather --out-dir /tmp/preview
```
Exit code is non-zero if any `(plugin, size, screen)` fails. Plugins are
discovered in `plugin-repos/` and `plugins/` (override with `--plugin-dir`).
## What it checks (Phase 1 — always on)
1. **Loads** and builds its mode list.
2. **Renders every screen** at every size without raising. `update()` may fail
(no network in CI) and is tolerated; a crash in `display()` is a failure —
`display()` must handle the no-data state.
3. **Bounds**: nothing is drawn past the right/bottom edge. Implemented by
`BoundsCheckingDisplayManager`, which backs the declared panel with an
oversized canvas and flags any pixels that land in the margin. (Left/top
overflow at negative coordinates and BDF text are not flagged — golden images
cover those.)
## Golden images (Phase 2 — opt-in per plugin)
A plugin opts in by committing reference PNGs and (usually) a small harness spec:
```
plugins/<id>/test/harness.json # how to render deterministically
plugins/<id>/test/fixtures/mock.json # optional cached data
plugins/<id>/test/golden/<WxH>/<mode>.png
```
`test/harness.json` keys (all optional):
```json
{
"config": { "timezone": "UTC" },
"mock_data": "fixtures/mock.json",
"freeze_time": "2025-08-01 15:25:00",
"skip_update": false,
"sizes": [[128, 32], [128, 64]]
}
```
Generate / refresh goldens after an intentional visual change, then review the
diff before committing:
```bash
python scripts/check_plugin.py --plugin clock-simple --update-golden \
--config '{"timezone":"UTC"}' --freeze-time "2025-08-01 15:25:00"
```
Comparison is exact by default (`compare_images` in `harness.py` accepts a
tolerance for known anti-aliasing noise). Determinism requires a pinned Pillow
and the bundled fonts — keep both stable when regenerating goldens.
## Tests & CI
- `test/plugins/test_harness.py` — unit tests for bounds detection, image
comparison, and mode enumeration (run anywhere).
- `test/plugins/test_plugin_matrix.py` — parametrized over discovered plugins ×
sizes × screens; honors each plugin's `test/harness.json` and goldens. Skips
when no plugins are present (e.g. a fresh core checkout); set
`LEDMATRIX_REQUIRE_PLUGINS=1` in a pipeline where plugins must be present to
turn an empty discovery into a hard failure instead. Point it at the monorepo
with `LEDMATRIX_PLUGINS_DIR=/path/to/ledmatrix-plugins/plugins`.
- `.github/workflows/test.yml` — runs the harness + visual tests on every PR.
The plugin monorepo has its own `Plugin Safety` workflow that runs this harness
against changed plugins on every PR.
## Developer workflow
1. Change the plugin on a branch.
2. `python scripts/check_plugin.py --plugin <id> --out-dir /tmp/preview` and
eyeball the PNGs.
3. Intentional visual change? `--update-golden`, review diffs, commit goldens.
4. (Monorepo) bump `manifest.json` version and let the pre-commit hook sync
`plugins.json`.
5. Push — CI re-runs the harness across all sizes and gates the PR.
+92
View File
@@ -10,6 +10,98 @@ The LEDMatrix Widget Registry system allows plugins to use reusable UI component
## Available Core Widgets
### Plugin File Manager Widget (`plugin-file-manager`)
Full inline file management UI for plugins that manage files via the `web_ui_actions` system. Renders a card grid, upload zone, create/delete modals, and an entry table editor — entirely inline, no iframe.
`plugin_id` is **automatically injected** from template context. File operations call `/api/v3/plugins/action` immediately on user action; no Save Configuration needed.
**Schema Configuration:**
```json
{
"file_manager": {
"type": "null",
"title": "Data Files",
"x-widget": "plugin-file-manager",
"x-widget-config": {
"actions": {
"list": "list-files",
"get": "get-file",
"save": "save-file",
"upload": "upload-file",
"delete": "delete-file",
"create": "create-file",
"toggle": "toggle-category"
},
"upload_hint": "JSON files with day numbers 1365 as keys",
"directory_label": "my_data/",
"create_fields": [
{ "key": "category_name", "label": "Category Name",
"placeholder": "e.g., my_words", "pattern": "^[a-z0-9_]+$",
"hint": "Lowercase letters, numbers, underscores" },
{ "key": "display_name", "label": "Display Name",
"placeholder": "e.g., My Words", "hint": "Optional" }
]
}
}
}
```
**`list` is required** — the widget calls it on render to populate the file grid; omitting it leaves the widget stuck in a loading state. All other actions are optional — omit any key to hide its UI element (e.g., no `create` = no New File button, no `toggle` = no enable/disable switch).
The edit view auto-detects whether file content is tabular (object-of-objects with uniform keys) and shows a paginated table editor with inline cells. Otherwise falls back to a JSON textarea.
**Used by:** of-the-day
---
### Time Picker Widget (`time-picker`)
Single time selection using the browser's native time input. Returns a string in `HH:MM` (24-hour) format. Generic — works in any plugin without configuration.
**Schema Configuration:**
```json
{
"target_time": {
"type": "string",
"x-widget": "time-picker",
"default": "00:00",
"x-options": {
"placeholder": "Select time",
"clearable": true
}
}
}
```
**Used by:** countdown
---
### File Upload Single Widget (`file-upload-single`)
Single-image upload for string fields. Uploads to the plugin's asset folder (`assets/plugins/<plugin_id>/uploads/`) and sets the string field value to the returned relative path. Shows a thumbnail preview and a clear button. The `plugin_id` is **automatically injected** from the template context — no need to specify it in the schema.
**Schema Configuration:**
```json
{
"image_path": {
"type": "string",
"x-widget": "file-upload-single",
"x-upload-config": {
"allowed_types": ["image/png", "image/jpeg", "image/bmp", "image/gif"],
"max_size_mb": 5
}
}
}
```
Note: Unlike `file-upload` (array-level), this widget is for a single `string` field. It is ideal for per-item images inside `array-table` rows.
**Used by:** countdown
---
### File Upload Widget (`file-upload`)
Upload and manage image files with drag-and-drop support, preview, delete, and scheduling.
+138 -40
View File
@@ -15,8 +15,8 @@ on_error() {
echo "✗ An error occurred during: $CURRENT_STEP (line $line_no, exit $exit_code)" >&2
if [ -n "${LOG_FILE:-}" ]; then
echo "See the log for details: $LOG_FILE" >&2
echo "-- Last 50 lines from log --" >&2
tail -n 50 "$LOG_FILE" >&2 || true
echo "-- Last 100 lines from log --" >&2
tail -n 100 "$LOG_FILE" >&2 || true
fi
echo "\nCommon fixes:" >&2
echo "- Ensure the Pi is online (try: ping -c1 8.8.8.8)." >&2
@@ -36,9 +36,17 @@ if [ -r /proc/device-tree/model ]; then
DEVICE_MODEL=$(tr -d '\0' </proc/device-tree/model)
echo "Detected device: $DEVICE_MODEL"
else
DEVICE_MODEL=""
echo "⚠ Could not detect Raspberry Pi model (continuing anyway)"
fi
# Detect Pi 5 for hardware-specific install decisions (RP1 library verification)
IS_PI5=0
if echo "${DEVICE_MODEL:-}" | grep -qi "Raspberry Pi 5"; then
IS_PI5=1
echo "Raspberry Pi 5 detected — will verify RP1 library support."
fi
# Check OS version - must be Raspberry Pi OS Lite (Trixie)
echo ""
echo "Checking operating system requirements..."
@@ -194,8 +202,33 @@ retry() {
done
}
apt_update() { retry apt update; }
apt_install() { retry apt install -y "$@"; }
# Wait for another apt/dpkg process (commonly unattended-upgrades running
# shortly after first boot) to release its lock before we try apt ourselves.
# Without this, apt_update/apt_install can fail outright in the first couple
# minutes after a fresh Pi OS boot with a generic "Command failed after 3
# attempts" error.
wait_for_apt_lock() {
command -v flock >/dev/null 2>&1 || return 0
local lock_file="/var/lib/dpkg/lock-frontend"
local max_wait=180
local waited=0
local printed=0
while ! flock -n "$lock_file" -c true 2>/dev/null; do
if [ "$printed" -eq 0 ]; then
echo "⚠ Waiting for another apt/dpkg process to finish (e.g. unattended-upgrades on first boot)..."
printed=1
fi
if [ "$waited" -ge "$max_wait" ]; then
echo "⚠ Still waiting after ${max_wait}s; proceeding anyway."
break
fi
sleep 5
waited=$((waited+5))
done
}
apt_update() { wait_for_apt_lock; retry apt-get -o DPkg::Lock::Timeout=180 update; }
apt_install() { wait_for_apt_lock; retry apt-get -o DPkg::Lock::Timeout=180 install -y "$@"; }
apt_remove() { apt-get remove -y "$@" || true; }
check_network() {
@@ -214,6 +247,22 @@ check_network() {
exit 1
}
check_disk_space() {
command -v df >/dev/null 2>&1 || return 0
local available_mb
available_mb=$(df -m "$PROJECT_ROOT_DIR" | awk 'NR==2{print $4}')
available_mb=${available_mb:-0}
if [ "$available_mb" -lt 500 ]; then
echo "✗ ERROR: Insufficient disk space: ${available_mb}MB available (need at least 500MB)"
echo " Free up space first, e.g.: sudo apt clean && sudo apt autoremove"
exit 1
elif [ "$available_mb" -lt 1024 ]; then
echo "⚠ Limited disk space: ${available_mb}MB available (recommend at least 1GB for the rpi-rgb-led-matrix build in Step 6)"
else
echo "✓ Disk space sufficient: ${available_mb}MB available"
fi
}
echo ""
echo "This script will perform the following steps:"
echo "1. Install system dependencies"
@@ -259,21 +308,20 @@ else
fi
echo ""
CLEAR='
'
CURRENT_STEP="Install system dependencies"
echo "Step 1: Installing system dependencies..."
echo "----------------------------------------"
# Ensure network is available before APT operations
# Pre-flight checks before APT operations
check_network
check_disk_space
# Update package list
apt_update
# Install required system packages
echo "Installing Python packages and dependencies..."
apt_install python3-pip python3-venv python3-dev python3-pil python3-pil.imagetk build-essential python3-setuptools python3-wheel cython3 scons cmake ninja-build
apt_install python3-pip python3-venv python-dev-is-python3 python3-pil python3-pil.imagetk build-essential python3-setuptools python3-wheel cmake ninja-build
# Install additional system dependencies that might be needed
echo "Installing additional system dependencies..."
@@ -671,8 +719,6 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
echo "[$PACKAGE_NUM/$TOTAL_PACKAGES] Installing: $line"
# Check if package is already installed (basic check - may not catch all cases)
PACKAGE_NAME=$(echo "$line" | sed -E 's/[<>=!].*$//' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
# Try installing with verbose output and timeout (if available)
# Use --no-cache-dir to avoid cache issues, --verbose for diagnostics
INSTALL_OUTPUT=$(mktemp)
@@ -680,7 +726,11 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
if command -v timeout >/dev/null 2>&1; then
# Use timeout if available (10 minutes = 600 seconds)
if timeout 600 python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
# --ignore-installed: apt-managed packages (e.g. python3-requests)
# ship no pip RECORD file, so upgrading them would otherwise abort
# with "uninstall-no-record-file"; this lays the new version down
# alongside instead of trying to uninstall the apt copy first.
if timeout 600 python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
INSTALL_SUCCESS=true
else
EXIT_CODE=$?
@@ -688,7 +738,7 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
echo "✗ Timeout (10 minutes) installing: $line"
echo " This package may require building from source, which can be slow on Raspberry Pi."
echo " You can try installing it manually later with:"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose '$line'"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose '$line'"
else
echo "✗ Failed to install: $line (exit code: $EXIT_CODE)"
fi
@@ -696,7 +746,7 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
else
# No timeout command available, install without timeout
echo " Note: timeout command not available, installation may take a while..."
if python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
if python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
INSTALL_SUCCESS=true
else
EXIT_CODE=$?
@@ -748,7 +798,7 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
echo " 1. Ensure you have enough disk space: df -h"
echo " 2. Check available memory: free -h"
echo " 3. Try installing failed packages individually with verbose output:"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose <package>"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose <package>"
echo " 4. For packages that build from source (like numpy), consider:"
echo " - Installing pre-built wheels: python3 -m pip install --only-binary :all: <package>"
echo " - Or installing via apt if available: sudo apt install python3-<package>"
@@ -770,7 +820,10 @@ echo ""
# Install web interface dependencies
echo "Installing web interface dependencies..."
if [ -f "$PROJECT_ROOT_DIR/web_interface/requirements.txt" ]; then
if python3 -m pip install --break-system-packages --prefer-binary -r "$PROJECT_ROOT_DIR/web_interface/requirements.txt"; then
# --ignore-installed: apt-managed packages (e.g. python3-requests) ship no
# pip RECORD file, so upgrading them to the version pinned here would
# otherwise abort the whole install with "uninstall-no-record-file".
if python3 -m pip install --break-system-packages --prefer-binary --ignore-installed -r "$PROJECT_ROOT_DIR/web_interface/requirements.txt"; then
echo "✓ Web interface dependencies installed"
# Create marker file to indicate dependencies are installed
touch "$PROJECT_ROOT_DIR/.web_deps_installed"
@@ -787,29 +840,54 @@ CURRENT_STEP="Build and install rpi-rgb-led-matrix"
echo "Step 6: Building and installing rpi-rgb-led-matrix..."
echo "-----------------------------------------------------"
# If already installed and not forcing rebuild, skip expensive build
# On Pi 5, also check that the installed library has rp1_rio support.
# A library built before Pi 5 support was added imports fine but maps to the
# Pi 3 peripheral bus address (0x3f000000) instead of the RP1 chip at runtime.
_HAS_RP1=0
if python3 -c 'from rgbmatrix import RGBMatrixOptions; assert hasattr(RGBMatrixOptions(), "rp1_rio")' >/dev/null 2>&1; then
_HAS_RP1=1
fi
_SKIP_BUILD=0
if python3 -c 'from rgbmatrix import RGBMatrix, RGBMatrixOptions' >/dev/null 2>&1 && [ "${RPI_RGB_FORCE_REBUILD:-0}" != "1" ]; then
echo "rgbmatrix Python package already available; skipping build (set RPI_RGB_FORCE_REBUILD=1 to force rebuild)."
if [ "$IS_PI5" = "1" ] && [ "$_HAS_RP1" = "0" ]; then
echo "⚠ Pi 5 detected: installed rgbmatrix lacks rp1_rio support (older build)."
echo " Forcing rebuild to get Pi 5 RP1 support..."
else
_SKIP_BUILD=1
fi
fi
if [ "$_SKIP_BUILD" = "1" ]; then
_skip_suffix=""
if [ "$IS_PI5" = "1" ]; then _skip_suffix=" with Pi 5 RP1 support"; fi
echo "rgbmatrix already installed${_skip_suffix}; skipping build (set RPI_RGB_FORCE_REBUILD=1 to force rebuild)."
else
# Ensure rpi-rgb-led-matrix submodule is initialized
# Wrapper used with retry(): removes any partial clone dir before each attempt
# so git clone doesn't fail with "destination path already exists".
_clone_rpi_rgb() {
rm -rf "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master"
git clone https://github.com/hzeller/rpi-rgb-led-matrix.git rpi-rgb-led-matrix-master
}
if [ ! -d "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" ]; then
echo "rpi-rgb-led-matrix-master not found. Initializing git submodule..."
cd "$PROJECT_ROOT_DIR"
# Try to initialize submodule if .gitmodules exists
if [ -f "$PROJECT_ROOT_DIR/.gitmodules" ] && grep -q "rpi-rgb-led-matrix" "$PROJECT_ROOT_DIR/.gitmodules"; then
echo "Initializing rpi-rgb-led-matrix submodule..."
if ! git submodule update --init --recursive rpi-rgb-led-matrix-master 2>&1; then
if ! retry git submodule update --init --recursive rpi-rgb-led-matrix-master; then
echo "⚠ Submodule init failed, cloning directly from GitHub..."
git clone https://github.com/hzeller/rpi-rgb-led-matrix.git rpi-rgb-led-matrix-master
retry _clone_rpi_rgb
fi
else
# Fallback: clone directly if submodule not configured
echo "Submodule not configured, cloning directly from GitHub..."
git clone https://github.com/hzeller/rpi-rgb-led-matrix.git rpi-rgb-led-matrix-master
retry _clone_rpi_rgb
fi
fi
# Build and install rpi-rgb-led-matrix Python bindings
if [ -d "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" ]; then
# Check if submodule is properly initialized (not empty)
@@ -818,30 +896,34 @@ else
cd "$PROJECT_ROOT_DIR"
rm -rf rpi-rgb-led-matrix-master
if [ -f "$PROJECT_ROOT_DIR/.gitmodules" ] && grep -q "rpi-rgb-led-matrix" "$PROJECT_ROOT_DIR/.gitmodules"; then
git submodule update --init --recursive rpi-rgb-led-matrix-master
retry git submodule update --init --recursive rpi-rgb-led-matrix-master
else
git clone https://github.com/hzeller/rpi-rgb-led-matrix.git rpi-rgb-led-matrix-master
retry _clone_rpi_rgb
fi
fi
pushd "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" >/dev/null
echo "Building rpi-rgb-led-matrix Python bindings..."
# Build the library first, then Python bindings
# The build-python target depends on the library being built
if ! make build-python; then
echo "✗ Failed to build rpi-rgb-led-matrix Python bindings"
echo " Make sure you have the required build tools installed:"
echo " sudo apt install -y build-essential python3-dev cython3 scons"
popd >/dev/null
exit 1
echo "Installing rpi-rgb-led-matrix Python package (scikit-build-core + cmake)..."
echo " Build deps required: python-dev-is-python3 cmake"
echo " This compiles C++ — may take 2-5 minutes on Pi 4/5..."
BUILD_OUTPUT=$(mktemp)
BUILD_SUCCESS=false
if python3 -m pip install --break-system-packages . > "$BUILD_OUTPUT" 2>&1; then
BUILD_SUCCESS=true
fi
cd bindings/python
echo "Installing rpi-rgb-led-matrix Python package via pip..."
if ! python3 -m pip install --break-system-packages .; then
cat "$BUILD_OUTPUT" >> "$LOG_FILE"
if [ "$BUILD_SUCCESS" != true ]; then
echo "✗ Failed to install rpi-rgb-led-matrix Python package"
echo " Ensure build tools are installed:"
echo " sudo apt install -y python-dev-is-python3 cmake build-essential"
echo ""
echo "-- Last 50 lines of build output --"
tail -n 50 "$BUILD_OUTPUT"
rm -f "$BUILD_OUTPUT"
popd >/dev/null
exit 1
fi
rm -f "$BUILD_OUTPUT"
popd >/dev/null
else
echo "✗ rpi-rgb-led-matrix-master directory not found at $PROJECT_ROOT_DIR"
@@ -863,6 +945,17 @@ except Exception as e:
PY
then
echo "✓ rpi-rgb-led-matrix installed and verified"
# Pi 5: confirm the freshly-built library has rp1_rio support
if [ "$IS_PI5" = "1" ]; then
if python3 -c 'from rgbmatrix import RGBMatrixOptions; assert hasattr(RGBMatrixOptions(), "rp1_rio")' >/dev/null 2>&1; then
echo "✓ Pi 5 RP1 (rp1_rio) support confirmed"
else
echo "⚠ rp1_rio not found after rebuild — the submodule may be an older version."
echo " Try updating the submodule and rebuilding:"
echo " git submodule update --remote rpi-rgb-led-matrix-master"
echo " sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh"
fi
fi
else
echo "✗ rpi-rgb-led-matrix import test failed"
exit 1
@@ -885,11 +978,15 @@ else
# Try to install dependencies using the smart installer if available
if [ -f "$PROJECT_ROOT_DIR/scripts/install_dependencies_apt.py" ]; then
echo "Using smart dependency installer..."
python3 "$PROJECT_ROOT_DIR/scripts/install_dependencies_apt.py"
# -u: unbuffered stdout/stderr so output is captured in $LOG_FILE in
# real time and in order relative to this script's own echo statements
python3 -u "$PROJECT_ROOT_DIR/scripts/install_dependencies_apt.py"
else
echo "Using pip to install dependencies..."
if [ -f "$PROJECT_ROOT_DIR/requirements_web_v2.txt" ]; then
python3 -m pip install --break-system-packages --prefer-binary -r requirements_web_v2.txt
# --ignore-installed: see the Step 5 web_interface/requirements.txt
# install above — same apt/pip RECORD-file conflict applies here.
python3 -m pip install --break-system-packages --prefer-binary --ignore-installed -r requirements_web_v2.txt
else
echo "⚠ requirements_web_v2.txt not found; skipping web dependency install"
fi
@@ -1110,6 +1207,7 @@ $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service
$ACTUAL_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT_DIR/display_controller.py
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/start_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/stop_display.sh
$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_plugin_rm.sh *
EOF
if [ -n "$JOURNALCTL_PATH" ]; then
cat >> /tmp/ledmatrix_web_sudoers << EOF
@@ -1478,7 +1576,7 @@ echo "WiFi Connection Status:"
if command -v nmcli >/dev/null 2>&1; then
WIFI_STATUS=$(nmcli -t -f DEVICE,TYPE,STATE device status 2>/dev/null | grep -i wifi || echo "")
if [ -n "$WIFI_STATUS" ]; then
echo "$WIFI_STATUS" | while IFS=':' read -r device type state; do
echo "$WIFI_STATUS" | while IFS=':' read -r _ _ state; do
if [ "$state" = "connected" ]; then
SSID=$(nmcli -t -f active,ssid device wifi 2>/dev/null | grep "^yes:" | cut -d: -f2 | head -1)
if [ -n "$SSID" ]; then
@@ -1,138 +0,0 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "March Madness Plugin Configuration",
"type": "object",
"properties": {
"enabled": {
"type": "boolean",
"default": false,
"description": "Enable the March Madness tournament display"
},
"leagues": {
"type": "object",
"title": "Tournament Leagues",
"description": "Which NCAA tournaments to display",
"properties": {
"ncaam": {
"type": "boolean",
"default": true,
"description": "Show NCAA Men's Tournament games"
},
"ncaaw": {
"type": "boolean",
"default": true,
"description": "Show NCAA Women's Tournament games"
}
},
"additionalProperties": false
},
"favorite_teams": {
"type": "array",
"title": "Favorite Teams",
"description": "Team abbreviations to highlight (e.g., DUKE, UNC). Leave empty to show all teams equally.",
"items": {
"type": "string"
},
"uniqueItems": true,
"default": []
},
"display_options": {
"type": "object",
"title": "Display Options",
"x-collapsed": true,
"properties": {
"show_seeds": {
"type": "boolean",
"default": true,
"description": "Show tournament seeds (1-16) next to team names"
},
"show_round_logos": {
"type": "boolean",
"default": true,
"description": "Show round logo separators between game groups"
},
"highlight_upsets": {
"type": "boolean",
"default": true,
"description": "Highlight upset winners (higher seed beating lower seed) in gold"
},
"show_bracket_progress": {
"type": "boolean",
"default": true,
"description": "Show which teams are still alive in each region"
},
"scroll_speed": {
"type": "number",
"default": 1.0,
"minimum": 0.5,
"maximum": 5.0,
"description": "Scroll speed (pixels per frame)"
},
"scroll_delay": {
"type": "number",
"default": 0.02,
"minimum": 0.001,
"maximum": 0.1,
"description": "Delay between scroll frames (seconds)"
},
"target_fps": {
"type": "integer",
"default": 120,
"minimum": 30,
"maximum": 200,
"description": "Target frames per second"
},
"loop": {
"type": "boolean",
"default": true,
"description": "Loop the scroll continuously"
},
"dynamic_duration": {
"type": "boolean",
"default": true,
"description": "Automatically adjust display duration based on content width"
},
"min_duration": {
"type": "integer",
"default": 30,
"minimum": 10,
"maximum": 300,
"description": "Minimum display duration in seconds"
},
"max_duration": {
"type": "integer",
"default": 300,
"minimum": 30,
"maximum": 600,
"description": "Maximum display duration in seconds"
}
},
"additionalProperties": false
},
"data_settings": {
"type": "object",
"title": "Data Settings",
"x-collapsed": true,
"properties": {
"update_interval": {
"type": "integer",
"default": 300,
"minimum": 60,
"maximum": 3600,
"description": "How often to refresh tournament data (seconds). Automatically shortens to 60s when live games are detected."
},
"request_timeout": {
"type": "integer",
"default": 30,
"minimum": 5,
"maximum": 60,
"description": "API request timeout in seconds"
}
},
"additionalProperties": false
}
},
"required": ["enabled"],
"additionalProperties": false,
"x-propertyOrder": ["enabled", "leagues", "favorite_teams", "display_options", "data_settings"]
}
-910
View File
@@ -1,910 +0,0 @@
"""March Madness Plugin — NCAA Tournament bracket tracker for LED Matrix.
Displays a horizontally-scrolling ticker of NCAA Tournament games grouped by
round, with seeds, round logos, live scores, and upset highlighting.
"""
import re
import threading
import time
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
import numpy as np
import pytz
import requests
from PIL import Image, ImageDraw, ImageFont
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from src.plugin_system.base_plugin import BasePlugin
try:
from src.common.scroll_helper import ScrollHelper
except ImportError:
ScrollHelper = None
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
SCOREBOARD_URLS = {
"ncaam": "https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/scoreboard",
"ncaaw": "https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/scoreboard",
}
ROUND_ORDER = {"NCG": 0, "F4": 1, "E8": 2, "S16": 3, "R32": 4, "R64": 5, "": 6}
ROUND_DISPLAY_NAMES = {
"NCG": "Championship",
"F4": "Final Four",
"E8": "Elite Eight",
"S16": "Sweet Sixteen",
"R32": "Round of 32",
"R64": "Round of 64",
}
ROUND_LOGO_FILES = {
"NCG": "CHAMPIONSHIP.png",
"F4": "FINAL_4.png",
"E8": "ELITE_8.png",
"S16": "SWEET_16.png",
"R32": "ROUND_32.png",
"R64": "ROUND_64.png",
}
REGION_ORDER = {"E": 0, "W": 1, "S": 2, "MW": 3, "": 4}
# Colors
COLOR_WHITE = (255, 255, 255)
COLOR_GOLD = (255, 215, 0)
COLOR_GRAY = (160, 160, 160)
COLOR_DIM = (100, 100, 100)
COLOR_RED = (255, 60, 60)
COLOR_GREEN = (60, 200, 60)
COLOR_BLACK = (0, 0, 0)
COLOR_DARK_BG = (20, 20, 20)
# ---------------------------------------------------------------------------
# Plugin Class
# ---------------------------------------------------------------------------
class MarchMadnessPlugin(BasePlugin):
"""NCAA March Madness tournament bracket tracker."""
def __init__(
self,
plugin_id: str,
config: Dict[str, Any],
display_manager: Any,
cache_manager: Any,
plugin_manager: Any,
):
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# Config
leagues_config = config.get("leagues", {})
self.show_ncaam: bool = leagues_config.get("ncaam", True)
self.show_ncaaw: bool = leagues_config.get("ncaaw", True)
self.favorite_teams: List[str] = [t.upper() for t in config.get("favorite_teams", [])]
display_options = config.get("display_options", {})
self.show_seeds: bool = display_options.get("show_seeds", True)
self.show_round_logos: bool = display_options.get("show_round_logos", True)
self.highlight_upsets: bool = display_options.get("highlight_upsets", True)
self.show_bracket_progress: bool = display_options.get("show_bracket_progress", True)
self.scroll_speed: float = display_options.get("scroll_speed", 1.0)
self.scroll_delay: float = display_options.get("scroll_delay", 0.02)
self.target_fps: int = display_options.get("target_fps", 120)
self.loop: bool = display_options.get("loop", True)
self.dynamic_duration_enabled: bool = display_options.get("dynamic_duration", True)
self.min_duration: int = display_options.get("min_duration", 30)
self.max_duration: int = display_options.get("max_duration", 300)
if self.min_duration > self.max_duration:
self.logger.warning(
f"min_duration ({self.min_duration}) > max_duration ({self.max_duration}); swapping values"
)
self.min_duration, self.max_duration = self.max_duration, self.min_duration
data_settings = config.get("data_settings", {})
self.update_interval: int = data_settings.get("update_interval", 300)
self.request_timeout: int = data_settings.get("request_timeout", 30)
# Scrolling flag for display controller
self.enable_scrolling = True
# State
self.games_data: List[Dict] = []
self.ticker_image: Optional[Image.Image] = None
self.last_update: float = 0
self.dynamic_duration: float = 60
self.total_scroll_width: int = 0
self._display_start_time: Optional[float] = None
self._end_reached_logged: bool = False
self._update_lock = threading.Lock()
self._has_live_games: bool = False
self._cached_dynamic_duration: Optional[float] = None
self._duration_cache_time: float = 0
# Display dimensions
self.display_width: int = self.display_manager.matrix.width
self.display_height: int = self.display_manager.matrix.height
# HTTP session with retry
self.session = requests.Session()
retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504])
self.session.mount("https://", HTTPAdapter(max_retries=retry))
self.headers = {"User-Agent": "LEDMatrix/2.0"}
# ScrollHelper
if ScrollHelper:
self.scroll_helper = ScrollHelper(self.display_width, self.display_height, logger=self.logger)
if hasattr(self.scroll_helper, "set_frame_based_scrolling"):
self.scroll_helper.set_frame_based_scrolling(True)
self.scroll_helper.set_scroll_speed(self.scroll_speed)
self.scroll_helper.set_scroll_delay(self.scroll_delay)
if hasattr(self.scroll_helper, "set_target_fps"):
self.scroll_helper.set_target_fps(self.target_fps)
self.scroll_helper.set_dynamic_duration_settings(
enabled=self.dynamic_duration_enabled,
min_duration=self.min_duration,
max_duration=self.max_duration,
buffer=0.1,
)
else:
self.scroll_helper = None
self.logger.warning("ScrollHelper not available")
# Fonts
self.fonts = self._load_fonts()
# Logos
self._round_logos: Dict[str, Image.Image] = {}
self._team_logo_cache: Dict[str, Optional[Image.Image]] = {}
self._march_madness_logo: Optional[Image.Image] = None
self._load_round_logos()
self.logger.info(
f"MarchMadnessPlugin initialized — NCAAM: {self.show_ncaam}, "
f"NCAAW: {self.show_ncaaw}, favorites: {self.favorite_teams}"
)
# ------------------------------------------------------------------
# Fonts
# ------------------------------------------------------------------
def _load_fonts(self) -> Dict[str, ImageFont.FreeTypeFont]:
fonts = {}
try:
fonts["score"] = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 10)
except IOError:
fonts["score"] = ImageFont.load_default()
try:
fonts["time"] = ImageFont.truetype("assets/fonts/PressStart2P-Regular.ttf", 8)
except IOError:
fonts["time"] = ImageFont.load_default()
try:
fonts["detail"] = ImageFont.truetype("assets/fonts/4x6-font.ttf", 6)
except IOError:
fonts["detail"] = ImageFont.load_default()
return fonts
# ------------------------------------------------------------------
# Logo loading
# ------------------------------------------------------------------
def _load_round_logos(self) -> None:
logo_dir = Path("assets/sports/ncaa_logos")
for round_key, filename in ROUND_LOGO_FILES.items():
path = logo_dir / filename
try:
img = Image.open(path).convert("RGBA")
# Resize to fit display height
target_h = self.display_height - 4
ratio = target_h / img.height
target_w = int(img.width * ratio)
self._round_logos[round_key] = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
except (OSError, ValueError) as e:
self.logger.warning(f"Could not load round logo {filename}: {e}")
except Exception:
self.logger.exception(f"Unexpected error loading round logo {filename}")
# March Madness logo
mm_path = logo_dir / "MARCH_MADNESS.png"
try:
img = Image.open(mm_path).convert("RGBA")
target_h = self.display_height - 4
ratio = target_h / img.height
target_w = int(img.width * ratio)
self._march_madness_logo = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
except (OSError, ValueError) as e:
self.logger.warning(f"Could not load March Madness logo: {e}")
except Exception:
self.logger.exception("Unexpected error loading March Madness logo")
def _get_team_logo(self, abbr: str) -> Optional[Image.Image]:
if abbr in self._team_logo_cache:
return self._team_logo_cache[abbr]
logo_dir = Path("assets/sports/ncaa_logos")
path = logo_dir / f"{abbr}.png"
try:
img = Image.open(path).convert("RGBA")
target_h = self.display_height - 6
ratio = target_h / img.height
target_w = int(img.width * ratio)
img = img.resize((target_w, target_h), Image.Resampling.LANCZOS)
self._team_logo_cache[abbr] = img
return img
except (FileNotFoundError, OSError, ValueError):
self._team_logo_cache[abbr] = None
return None
except Exception:
self.logger.exception(f"Unexpected error loading team logo for {abbr}")
self._team_logo_cache[abbr] = None
return None
# ------------------------------------------------------------------
# Data fetching
# ------------------------------------------------------------------
def _is_tournament_window(self) -> bool:
today = datetime.now(pytz.utc)
return (3, 10) <= (today.month, today.day) <= (4, 10)
def _fetch_tournament_data(self) -> List[Dict]:
"""Fetch tournament games from ESPN scoreboard API."""
all_games: List[Dict] = []
leagues = []
if self.show_ncaam:
leagues.append("ncaam")
if self.show_ncaaw:
leagues.append("ncaaw")
for league_key in leagues:
url = SCOREBOARD_URLS.get(league_key)
if not url:
continue
cache_key = f"march_madness_{league_key}_scoreboard"
cache_max_age = 60 if self._has_live_games else self.update_interval
cached = self.cache_manager.get(cache_key, max_age=cache_max_age)
if cached:
all_games.extend(cached)
continue
try:
# NCAA basketball scoreboard without dates param returns current games
params = {"limit": 1000, "groups": 100}
resp = self.session.get(url, params=params, headers=self.headers, timeout=self.request_timeout)
resp.raise_for_status()
data = resp.json()
events = data.get("events", [])
league_games = []
for event in events:
game = self._parse_event(event, league_key)
if game:
league_games.append(game)
self.cache_manager.set(cache_key, league_games)
self.logger.info(f"Fetched {len(league_games)} {league_key} tournament games")
all_games.extend(league_games)
except Exception:
self.logger.exception(f"Error fetching {league_key} tournament data")
return all_games
def _parse_event(self, event: Dict, league_key: str) -> Optional[Dict]:
"""Parse an ESPN event into a game dict."""
competitions = event.get("competitions", [])
if not competitions:
return None
comp = competitions[0]
# Confirm tournament game
comp_type = comp.get("type", {})
is_tournament = comp_type.get("abbreviation") == "TRNMNT"
notes = comp.get("notes", [])
headline = ""
if notes:
headline = notes[0].get("headline", "")
if not is_tournament and "Championship" in headline:
is_tournament = True
if not is_tournament:
return None
# Status
status = comp.get("status", {}).get("type", {})
state = status.get("state", "pre")
status_detail = status.get("shortDetail", "")
# Teams
competitors = comp.get("competitors", [])
home_team = next((c for c in competitors if c.get("homeAway") == "home"), None)
away_team = next((c for c in competitors if c.get("homeAway") == "away"), None)
if not home_team or not away_team:
return None
home_abbr = home_team.get("team", {}).get("abbreviation", "???")
away_abbr = away_team.get("team", {}).get("abbreviation", "???")
home_score = home_team.get("score", "0")
away_score = away_team.get("score", "0")
# Seeds
home_seed = home_team.get("curatedRank", {}).get("current", 0)
away_seed = away_team.get("curatedRank", {}).get("current", 0)
if home_seed >= 99:
home_seed = 0
if away_seed >= 99:
away_seed = 0
# Round and region
tournament_round = self._parse_round(headline)
tournament_region = self._parse_region(headline)
# Date/time
date_str = event.get("date", "")
start_time_utc = None
game_date = ""
game_time = ""
try:
if date_str.endswith("Z"):
date_str = date_str.replace("Z", "+00:00")
dt = datetime.fromisoformat(date_str)
if dt.tzinfo is None:
start_time_utc = dt.replace(tzinfo=pytz.UTC)
else:
start_time_utc = dt.astimezone(pytz.UTC)
local = start_time_utc.astimezone(pytz.timezone("US/Eastern"))
game_date = local.strftime("%-m/%-d")
game_time = local.strftime("%-I:%M%p").replace("AM", "am").replace("PM", "pm")
except (ValueError, AttributeError):
pass
# Period / clock for live games
period = 0
clock = ""
period_text = ""
is_halftime = False
if state == "in":
status_obj = comp.get("status", {})
period = status_obj.get("period", 0)
clock = status_obj.get("displayClock", "")
detail_lower = status_detail.lower()
uses_quarters = league_key == "ncaaw" or "quarter" in detail_lower or detail_lower.startswith("q")
if period <= (4 if uses_quarters else 2):
period_text = f"Q{period}" if uses_quarters else f"H{period}"
else:
ot_num = period - (4 if uses_quarters else 2)
period_text = f"OT{ot_num}" if ot_num > 1 else "OT"
if "halftime" in detail_lower:
is_halftime = True
elif state == "post":
period_text = status.get("shortDetail", "Final")
if "Final" not in period_text:
period_text = "Final"
# Determine winner and upset
is_final = state == "post"
is_upset = False
winner_side = ""
if is_final:
try:
h = int(float(home_score))
a = int(float(away_score))
if h > a:
winner_side = "home"
if home_seed > away_seed > 0:
is_upset = True
elif a > h:
winner_side = "away"
if away_seed > home_seed > 0:
is_upset = True
except (ValueError, TypeError):
pass
return {
"id": event.get("id", ""),
"league": league_key,
"home_abbr": home_abbr,
"away_abbr": away_abbr,
"home_score": str(home_score),
"away_score": str(away_score),
"home_seed": home_seed,
"away_seed": away_seed,
"tournament_round": tournament_round,
"tournament_region": tournament_region,
"state": state,
"is_final": is_final,
"is_live": state == "in",
"is_upcoming": state == "pre",
"is_halftime": is_halftime,
"period": period,
"period_text": period_text,
"clock": clock,
"status_detail": status_detail,
"game_date": game_date,
"game_time": game_time,
"start_time_utc": start_time_utc,
"is_upset": is_upset,
"winner_side": winner_side,
"headline": headline,
}
@staticmethod
def _parse_round(headline: str) -> str:
hl = headline.lower()
if "national championship" in hl:
return "NCG"
if "final four" in hl:
return "F4"
if "elite 8" in hl or "elite eight" in hl:
return "E8"
if "sweet 16" in hl or "sweet sixteen" in hl:
return "S16"
if "2nd round" in hl or "second round" in hl:
return "R32"
if "1st round" in hl or "first round" in hl:
return "R64"
return ""
@staticmethod
def _parse_region(headline: str) -> str:
if "East Region" in headline:
return "E"
if "West Region" in headline:
return "W"
if "South Region" in headline:
return "S"
if "Midwest Region" in headline:
return "MW"
m = re.search(r"Regional (\d+)", headline)
if m:
return f"R{m.group(1)}"
return ""
# ------------------------------------------------------------------
# Game processing
# ------------------------------------------------------------------
def _process_games(self, games: List[Dict]) -> Dict[str, List[Dict]]:
"""Group games by round, sorted by round significance then region/seed."""
grouped: Dict[str, List[Dict]] = {}
for game in games:
rnd = game.get("tournament_round", "")
grouped.setdefault(rnd, []).append(game)
# Sort each round's games by region then seed matchup
for rnd, round_games in grouped.items():
round_games.sort(
key=lambda g: (
REGION_ORDER.get(g.get("tournament_region", ""), 4),
min(g.get("away_seed", 99), g.get("home_seed", 99)),
)
)
return grouped
# ------------------------------------------------------------------
# Rendering
# ------------------------------------------------------------------
def _draw_text_with_outline(
self,
draw: ImageDraw.Draw,
text: str,
xy: tuple,
font: ImageFont.FreeTypeFont,
fill: tuple = COLOR_WHITE,
outline: tuple = COLOR_BLACK,
) -> None:
x, y = xy
for dx in (-1, 0, 1):
for dy in (-1, 0, 1):
if dx or dy:
draw.text((x + dx, y + dy), text, font=font, fill=outline)
draw.text((x, y), text, font=font, fill=fill)
def _create_round_separator(self, round_key: str) -> Image.Image:
"""Create a separator tile for a tournament round."""
height = self.display_height
name = ROUND_DISPLAY_NAMES.get(round_key, round_key)
font = self.fonts["time"]
# Measure text
tmp = Image.new("RGB", (1, 1))
tmp_draw = ImageDraw.Draw(tmp)
text_width = int(tmp_draw.textlength(name, font=font))
# Logo on each side
logo = self._round_logos.get(round_key, self._march_madness_logo)
logo_w = logo.width if logo else 0
padding = 6
total_w = padding + logo_w + padding + text_width + padding + logo_w + padding
total_w = max(total_w, 80)
img = Image.new("RGB", (total_w, height), COLOR_DARK_BG)
draw = ImageDraw.Draw(img)
# Draw logos
x = padding
if logo:
logo_y = (height - logo.height) // 2
img.paste(logo, (x, logo_y), logo)
x += logo_w + padding
# Draw round name
text_y = (height - 8) // 2 # 8px font
self._draw_text_with_outline(draw, name, (x, text_y), font, fill=COLOR_GOLD)
x += text_width + padding
if logo:
logo_y = (height - logo.height) // 2
img.paste(logo, (x, logo_y), logo)
return img
def _create_game_tile(self, game: Dict) -> Image.Image:
"""Create a single game tile for the scrolling ticker."""
height = self.display_height
font_score = self.fonts["score"]
font_time = self.fonts["time"]
font_detail = self.fonts["detail"]
# Load team logos
away_logo = self._get_team_logo(game["away_abbr"])
home_logo = self._get_team_logo(game["home_abbr"])
logo_w = 0
if away_logo:
logo_w = max(logo_w, away_logo.width)
if home_logo:
logo_w = max(logo_w, home_logo.width)
if logo_w == 0:
logo_w = 24
# Build text elements
away_seed_str = f"({game['away_seed']})" if self.show_seeds and game.get("away_seed", 0) > 0 else ""
home_seed_str = f"({game['home_seed']})" if self.show_seeds and game.get("home_seed", 0) > 0 else ""
away_text = f"{away_seed_str}{game['away_abbr']}"
home_text = f"{game['home_abbr']}{home_seed_str}"
# Measure text widths
tmp = Image.new("RGB", (1, 1))
tmp_draw = ImageDraw.Draw(tmp)
away_text_w = int(tmp_draw.textlength(away_text, font=font_detail))
home_text_w = int(tmp_draw.textlength(home_text, font=font_detail))
# Center content: status line
if game["is_live"]:
if game["is_halftime"]:
status_text = "Halftime"
else:
status_text = f"{game['period_text']} {game['clock']}".strip()
elif game["is_final"]:
status_text = game.get("period_text", "Final")
else:
status_text = f"{game['game_date']} {game['game_time']}".strip()
status_w = int(tmp_draw.textlength(status_text, font=font_time))
# Score line (for live/final)
score_text = ""
if game["is_live"] or game["is_final"]:
score_text = f"{game['away_score']}-{game['home_score']}"
score_w = int(tmp_draw.textlength(score_text, font=font_score)) if score_text else 0
# Calculate tile width
h_pad = 4
center_w = max(status_w, score_w, 40)
tile_w = h_pad + logo_w + h_pad + away_text_w + h_pad + center_w + h_pad + home_text_w + h_pad + logo_w + h_pad
img = Image.new("RGB", (tile_w, height), COLOR_BLACK)
draw = ImageDraw.Draw(img)
# Paste away logo
x = h_pad
if away_logo:
logo_y = (height - away_logo.height) // 2
img.paste(away_logo, (x, logo_y), away_logo)
x += logo_w + h_pad
# Away team text (seed + abbr)
is_fav_away = game["away_abbr"] in self.favorite_teams if self.favorite_teams else False
away_color = COLOR_GOLD if is_fav_away else COLOR_WHITE
if game["is_final"] and game["winner_side"] == "away" and self.highlight_upsets and game["is_upset"]:
away_color = COLOR_GOLD
team_text_y = (height - 6) // 2 - 5 # Upper half
self._draw_text_with_outline(draw, away_text, (x, team_text_y), font_detail, fill=away_color)
x += away_text_w + h_pad
# Center block
center_x = x
center_mid = center_x + center_w // 2
# Status text (top center of center block)
status_x = center_mid - status_w // 2
status_y = 2
status_color = COLOR_GREEN if game["is_live"] else COLOR_GRAY
self._draw_text_with_outline(draw, status_text, (status_x, status_y), font_time, fill=status_color)
# Score (bottom center of center block, for live/final)
if score_text:
score_x = center_mid - score_w // 2
score_y = height - 13
# Upset highlighting
if game["is_final"] and game["is_upset"] and self.highlight_upsets:
score_color = COLOR_GOLD
elif game["is_live"]:
score_color = COLOR_WHITE
else:
score_color = COLOR_WHITE
self._draw_text_with_outline(draw, score_text, (score_x, score_y), font_score, fill=score_color)
# Date for final games (below score)
if game["is_final"] and game.get("game_date"):
date_w = int(draw.textlength(game["game_date"], font=font_detail))
date_x = center_mid - date_w // 2
date_y = height - 6
self._draw_text_with_outline(draw, game["game_date"], (date_x, date_y), font_detail, fill=COLOR_DIM)
x = center_x + center_w + h_pad
# Home team text
is_fav_home = game["home_abbr"] in self.favorite_teams if self.favorite_teams else False
home_color = COLOR_GOLD if is_fav_home else COLOR_WHITE
if game["is_final"] and game["winner_side"] == "home" and self.highlight_upsets and game["is_upset"]:
home_color = COLOR_GOLD
self._draw_text_with_outline(draw, home_text, (x, team_text_y), font_detail, fill=home_color)
x += home_text_w + h_pad
# Paste home logo
if home_logo:
logo_y = (height - home_logo.height) // 2
img.paste(home_logo, (x, logo_y), home_logo)
return img
def _create_ticker_image(self) -> None:
"""Build the full scrolling ticker image from game tiles."""
if not self.games_data:
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
grouped = self._process_games(self.games_data)
content_items: List[Image.Image] = []
# Order rounds by significance (most important first)
sorted_rounds = sorted(grouped.keys(), key=lambda r: ROUND_ORDER.get(r, 6))
for rnd in sorted_rounds:
games = grouped[rnd]
if not games:
continue
# Add round separator
if self.show_round_logos and rnd:
separator = self._create_round_separator(rnd)
content_items.append(separator)
# Add game tiles
for game in games:
tile = self._create_game_tile(game)
content_items.append(tile)
if not content_items:
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
if not self.scroll_helper:
self.ticker_image = None
return
gap_width = 16
# Use ScrollHelper to create the scrolling image
self.ticker_image = self.scroll_helper.create_scrolling_image(
content_items=content_items,
item_gap=gap_width,
element_gap=0,
)
self.total_scroll_width = self.scroll_helper.total_scroll_width
self.dynamic_duration = self.scroll_helper.get_dynamic_duration()
self.logger.info(
f"Ticker image created: {self.ticker_image.width}px wide, "
f"{len(self.games_data)} games, dynamic_duration={self.dynamic_duration:.0f}s"
)
# ------------------------------------------------------------------
# Plugin lifecycle
# ------------------------------------------------------------------
def update(self) -> None:
"""Fetch and process tournament data."""
if not self.enabled:
return
current_time = time.time()
# Use shorter interval if live games detected
interval = 60 if self._has_live_games else self.update_interval
if current_time - self.last_update < interval:
return
with self._update_lock:
self.last_update = current_time
if not self._is_tournament_window():
self.logger.debug("Outside tournament window, skipping fetch")
self.games_data = []
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
return
try:
games = self._fetch_tournament_data()
self._has_live_games = any(g["is_live"] for g in games)
self.games_data = games
self._create_ticker_image()
self.logger.info(
f"Updated: {len(games)} games, "
f"live={self._has_live_games}"
)
except Exception as e:
self.logger.error(f"Update error: {e}", exc_info=True)
def display(self, force_clear: bool = False) -> None:
"""Render one scroll frame."""
if not self.enabled:
return
if force_clear or self._display_start_time is None:
self._display_start_time = time.time()
if self.scroll_helper:
self.scroll_helper.reset_scroll()
self._end_reached_logged = False
if not self.games_data or self.ticker_image is None:
self._display_fallback()
return
if not self.scroll_helper:
self._display_fallback()
return
try:
if self.loop or not self.scroll_helper.is_scroll_complete():
self.scroll_helper.update_scroll_position()
elif not self._end_reached_logged:
self.logger.info("Scroll complete")
self._end_reached_logged = True
visible = self.scroll_helper.get_visible_portion()
if visible is None:
self._display_fallback()
return
self.dynamic_duration = self.scroll_helper.get_dynamic_duration()
matrix_w = self.display_manager.matrix.width
matrix_h = self.display_manager.matrix.height
if not hasattr(self.display_manager, "image") or self.display_manager.image is None:
self.display_manager.image = Image.new("RGB", (matrix_w, matrix_h), COLOR_BLACK)
self.display_manager.image.paste(visible, (0, 0))
self.display_manager.update_display()
self.scroll_helper.log_frame_rate()
except Exception as e:
self.logger.error(f"Display error: {e}", exc_info=True)
self._display_fallback()
def _display_fallback(self) -> None:
w = self.display_manager.matrix.width
h = self.display_manager.matrix.height
img = Image.new("RGB", (w, h), COLOR_BLACK)
draw = ImageDraw.Draw(img)
if self._is_tournament_window():
text = "No games"
else:
text = "Off-season"
text_w = int(draw.textlength(text, font=self.fonts["time"]))
text_x = (w - text_w) // 2
text_y = (h - 8) // 2
draw.text((text_x, text_y), text, font=self.fonts["time"], fill=COLOR_GRAY)
# Show March Madness logo if available
if self._march_madness_logo:
logo_y = (h - self._march_madness_logo.height) // 2
img.paste(self._march_madness_logo, (2, logo_y), self._march_madness_logo)
self.display_manager.image = img
self.display_manager.update_display()
# ------------------------------------------------------------------
# Duration / cycle management
# ------------------------------------------------------------------
def get_display_duration(self) -> float:
current_time = time.time()
if self._cached_dynamic_duration is not None:
cache_age = current_time - self._duration_cache_time
if cache_age < 5.0:
return self._cached_dynamic_duration
self._cached_dynamic_duration = self.dynamic_duration
self._duration_cache_time = current_time
return self.dynamic_duration
def supports_dynamic_duration(self) -> bool:
if not self.enabled:
return False
return self.dynamic_duration_enabled
def is_cycle_complete(self) -> bool:
if not self.supports_dynamic_duration():
return True
if self._display_start_time is not None and self.dynamic_duration > 0:
elapsed = time.time() - self._display_start_time
if elapsed >= self.dynamic_duration:
return True
if not self.loop and self.scroll_helper and self.scroll_helper.is_scroll_complete():
return True
return False
def reset_cycle_state(self) -> None:
super().reset_cycle_state()
self._display_start_time = None
self._end_reached_logged = False
if self.scroll_helper:
self.scroll_helper.reset_scroll()
# ------------------------------------------------------------------
# Vegas mode
# ------------------------------------------------------------------
def get_vegas_content(self):
if not self.games_data:
return None
tiles = []
for game in self.games_data:
tiles.append(self._create_game_tile(game))
return tiles if tiles else None
def get_vegas_content_type(self) -> str:
return "multi"
# ------------------------------------------------------------------
# Info / cleanup
# ------------------------------------------------------------------
def get_info(self) -> Dict:
info = super().get_info()
info["total_games"] = len(self.games_data)
info["has_live_games"] = self._has_live_games
info["dynamic_duration"] = self.dynamic_duration
info["tournament_window"] = self._is_tournament_window()
return info
def cleanup(self) -> None:
self.games_data = []
self.ticker_image = None
if self.scroll_helper:
self.scroll_helper.clear_cache()
self._team_logo_cache.clear()
if self.session:
self.session.close()
self.session = None
super().cleanup()
-37
View File
@@ -1,37 +0,0 @@
{
"id": "march-madness",
"name": "March Madness",
"version": "1.0.0",
"description": "NCAA March Madness tournament bracket tracker with round branding, seeded matchups, live scores, and upset highlighting",
"author": "ChuckBuilds",
"category": "sports",
"tags": [
"ncaa",
"basketball",
"march-madness",
"tournament",
"bracket",
"scrolling"
],
"repo": "https://github.com/ChuckBuilds/ledmatrix-plugins",
"branch": "main",
"plugin_path": "plugins/march-madness",
"versions": [
{
"version": "1.0.0",
"ledmatrix_min": "2.0.0",
"released": "2026-02-16"
}
],
"stars": 0,
"downloads": 0,
"last_updated": "2026-02-16",
"verified": true,
"screenshot": "",
"display_modes": [
"march_madness"
],
"dependencies": {},
"entry_point": "manager.py",
"class_name": "MarchMadnessPlugin"
}
@@ -1,4 +0,0 @@
requests>=2.28.0
Pillow>=10.3.0
pytz>=2022.1
numpy>=1.24.0
+2 -1
View File
@@ -22,5 +22,6 @@
"Pillow>=10.0.0",
"PyYAML>=6.0",
"requests>=2.31.0"
]
],
"local_only": true
}
+2 -2
View File
@@ -1,3 +1,3 @@
Pillow>=10.4.0
Pillow>=12.2.0
PyYAML>=6.0.2
requests>=2.32.0
requests>=2.33.0
+9
View File
@@ -0,0 +1,9 @@
# Test-only dependencies for the plugin safety harness and pytest suite.
# Install alongside requirements.txt: pip install -r requirements.txt -r requirements-test.txt
#
# pytest, pytest-cov, pytest-mock, and jsonschema are already pinned (with
# major-version caps) in requirements.txt, so they are intentionally NOT
# repeated here — re-pinning pytest to <9 collided with requirements.txt's
# pytest>=9.0.3,<10 and made the two files impossible to install together.
# Only declare what requirements.txt doesn't already provide.
freezegun>=1.2,<2 # deterministic time for golden-image tests
+9 -6
View File
@@ -3,7 +3,7 @@
# Tested on Raspbian OS 12 (Bookworm) and 13 (Trixie)
# Image processing
Pillow>=10.4.0,<12.0.0
Pillow>=12.2.0,<13.0.0
numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
# Timezone handling
@@ -12,7 +12,7 @@ timezonefinder>=6.5.0,<7.0.0 # Updated for better performance and accuracy
geopy>=2.4.1,<3.0.0
# HTTP requests
requests>=2.32.0,<3.0.0
requests>=2.33.0,<3.0.0
# Google API integration
google-auth-oauthlib>=1.2.0,<2.0.0
@@ -23,10 +23,10 @@ google-api-python-client>=2.147.0,<3.0.0
freetype-py>=2.5.1,<3.0.0
# Spotify integration
spotipy>=2.24.0,<3.0.0
spotipy>=2.25.2,<3.0.0
# Flask web framework
Flask>=3.0.0,<4.0.0
Flask>=3.1.3,<4.0.0
# Text processing
unidecode>=1.3.8,<2.0.0
@@ -35,7 +35,7 @@ unidecode>=1.3.8,<2.0.0
icalevents>=0.1.27,<1.0.0
# WebSocket support
python-socketio>=5.11.0,<6.0.0
python-socketio>=5.14.0,<6.0.0
python-engineio>=4.9.0,<5.0.0
websockets>=12.0,<14.0
websocket-client>=1.8.0,<2.0.0
@@ -43,8 +43,11 @@ websocket-client>=1.8.0,<2.0.0
# JSON Schema validation
jsonschema>=4.20.0,<5.0.0
# Requirement specifier parsing (plugin dependency satisfaction checks)
packaging>=23.0,<27.0
# Testing dependencies
pytest>=7.4.0,<8.0.0
pytest>=9.0.3,<10.0.0
pytest-cov>=4.1.0,<5.0.0
pytest-mock>=3.11.0,<4.0.0
mypy>=1.5.0,<2.0.0
-1
View File
@@ -51,7 +51,6 @@ if debug_mode:
# Try to import the plugin system directly to get better error info
print("DEBUG: Attempting to import src.plugin_system...", flush=True)
from src.plugin_system import PluginManager
print("DEBUG: Plugin system import successful", flush=True)
except ImportError as e:
print(f"DEBUG: Plugin system import failed: {e}", flush=True)
+29
View File
@@ -90,11 +90,40 @@
"min_height": {
"type": "integer",
"minimum": 1
},
"max_width": {
"type": "integer",
"minimum": 1
},
"max_height": {
"type": "integer",
"minimum": 1
}
}
}
}
},
"display": {
"type": "object",
"properties": {
"design_size": {
"type": "object",
"properties": {
"width": {
"type": "integer",
"minimum": 8
},
"height": {
"type": "integer",
"minimum": 8
}
},
"required": ["width", "height"],
"description": "Panel size the plugin's layout was authored against; core derives the adaptive-layout scale factor from it. Defaults to 128x32 when omitted."
}
},
"description": "Display/layout hints for the adaptive layout system"
},
"config_schema": {
"type": "string",
"description": "Path to configuration schema file"
+1 -1
View File
@@ -9,7 +9,7 @@ and preventing validation errors.
import json
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional
from typing import Any, Dict, List
def get_default_for_field(prop: Dict[str, Any]) -> Any:
+1 -2
View File
@@ -9,9 +9,8 @@ Analyze all plugin config schemas to identify issues:
"""
import json
import os
from pathlib import Path
from typing import Dict, List, Set, Any
from typing import Dict, List, Any
import jsonschema
from jsonschema import Draft7Validator
+268
View File
@@ -0,0 +1,268 @@
#!/usr/bin/env python3
"""
Plugin safety checker.
Renders a plugin across every declared screen (mode) and every supported matrix
size, and fails if any screen crashes, overflows the panel, or (for plugins with
committed golden images) drifts visually.
Usage:
# Functional + bounds check across all sizes/modes:
python scripts/check_plugin.py --plugin clock-simple
# Every discovered plugin:
python scripts/check_plugin.py --all
# Dump PNGs for each size/mode so you can eyeball them:
python scripts/check_plugin.py --plugin ledmatrix-weather --out-dir /tmp/preview
# Refresh committed golden images after an intentional visual change:
python scripts/check_plugin.py --plugin clock-simple --update-golden \
--mock-data plugins/clock-simple/test/fixtures/mock.json
Exit code is non-zero if any (plugin, size, mode) fails.
"""
import argparse
import json
import os
import sys
from pathlib import Path
from typing import Dict, List, Optional
PROJECT_ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
os.environ['EMULATOR'] = 'true'
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_config_defaults, load_harness_spec, load_manifest,
)
from src.plugin_system.testing.harness import ( # noqa: E402
RenderResult, render_plugin_matrix, compare_to_goldens, write_goldens,
check_scale_up,
)
from src.plugin_system.testing.sizes import ( # noqa: E402
parse_size_token, resolve_test_sizes, safe_mode_filename, size_label,
)
logger = get_logger("[Check Plugin]")
DEFAULT_SEARCH_DIRS = [
str(PROJECT_ROOT / 'plugins'),
str(PROJECT_ROOT / 'plugin-repos'),
]
def discover_plugins(search_dirs: List[str]) -> List[str]:
"""All plugin ids found across the search dirs (dirs containing manifest.json)."""
found = []
for d in search_dirs:
base = Path(d)
if not base.exists():
continue
for child in sorted(base.iterdir()):
if (child / 'manifest.json').exists() and child.name not in found:
found.append(child.name)
return found
def parse_sizes(spec: Optional[str]):
if not spec:
return None
sizes = []
for token in spec.split(','):
if not token.strip():
continue
try:
sizes.append(parse_size_token(token))
except ValueError as exc:
raise SystemExit(str(exc)) from exc
return sizes
def check_one(plugin_id: str, search_dirs: List[str], sizes, mock_data: Dict,
config: Dict, run_update: bool, out_dir: Optional[Path],
update_golden: bool, golden_dir_override: Optional[Path],
freeze_time: Optional[str]) -> List[RenderResult]:
plugin_dir = find_plugin_dir(plugin_id, search_dirs)
if not plugin_dir:
logger.error("Plugin '%s' not found in: %s", plugin_id, search_dirs)
return [RenderResult(plugin_id, 0, 0, "<not-found>", error="plugin directory not found")]
# Per-plugin test/harness.json holds the deterministic settings the committed
# goldens were generated with (config, mock data, frozen time, sizes). Load
# them so the CLI/CI render reproduces the golden the same way the pytest
# matrix path does; explicit CLI flags still override the file.
spec = load_harness_spec(plugin_dir)
# config_schema defaults (real-install behavior), then harness.json config,
# then CLI --config — most specific wins.
full_config = {"enabled": True}
full_config.update(load_config_defaults(plugin_dir))
full_config.update(spec.get("config", {}))
full_config.update(config)
# Precedence: CLI flag > LEDMATRIX_TEST_SIZES env > harness.json > default.
effective_sizes = sizes if sizes else resolve_test_sizes(spec.get("sizes"))
# CLI value wins when provided, else fall back to the harness.json setting.
effective_mock_data = mock_data or spec.get("mock_data_contents", {})
effective_freeze = freeze_time or spec.get("freeze_time")
effective_run_update = run_update and not spec.get("skip_update", False)
# The plugin's declared design size drives the scale-up fill check
# (panels >= 2x the design size must not be left mostly empty).
declared = load_manifest(plugin_dir).get("display", {}).get("design_size", {})
design_size = (int(declared.get("width", 128)), int(declared.get("height", 32)))
fill_strict = spec.get("fill_check") == "strict"
# Every run: the base config, plus one per harness.json "variant" —
# a config overlay with its own golden dir (e.g. adaptive layout mode
# tested alongside the classic default).
runs = [(None, {}, golden_dir_override or (plugin_dir / 'test' / 'golden'))]
for variant in spec.get("variants", []):
name = variant.get("name") or "variant"
vdir = plugin_dir / variant.get("golden_dir", f"test/golden-{name}")
runs.append((name, variant.get("config", {}), vdir))
all_run_results: List[RenderResult] = []
for variant_name, overlay, golden_dir in runs:
run_config = {**full_config, **overlay}
results = render_plugin_matrix(
plugin_id=plugin_id, plugin_dir=plugin_dir, config=run_config,
mock_data=effective_mock_data, sizes=effective_sizes,
run_update=effective_run_update, freeze_time=effective_freeze,
)
if update_golden:
written = write_goldens(results, golden_dir)
logger.info("Wrote %d golden image(s) for %s%s to %s", written, plugin_id,
f" [{variant_name}]" if variant_name else "", golden_dir)
else:
compare_to_goldens(results, golden_dir)
check_scale_up(results, design_size=design_size, strict=fill_strict)
# Tag variant runs so the report and PNG dumps stay distinguishable.
if variant_name:
for r in results:
r.mode = f"{r.mode}@{variant_name}"
if out_dir:
for r in results:
if r.image is None:
continue
dest = out_dir / plugin_id / size_label(r.width, r.height)
dest.mkdir(parents=True, exist_ok=True)
r.image.save(dest / f"{safe_mode_filename(r.mode)}.png", format="PNG")
all_run_results.extend(results)
return all_run_results
def print_report(all_results: Dict[str, List[RenderResult]]) -> bool:
"""Print a per-plugin grid. Returns True if everything passed."""
everything_ok = True
for plugin_id, results in all_results.items():
print(f"\n=== {plugin_id} ===")
for r in results:
if r.ok:
status = "PASS"
detail = ""
if r.golden_checked:
detail = " (golden ✓)"
if r.update_error is not None:
detail += f" (update warn: {r.update_error})"
if r.fill_checked and r.fill_ok is None and r.fill_extent:
# warn-only underfill: big panel left mostly empty
ex, ey = r.fill_extent
detail += f" (fill warn: extent {ex:.0%}x{ey:.0%})"
else:
everything_ok = False
if r.error is not None:
status, detail = "FAIL", f" error={r.error}"
elif r.overflow is not None:
status, detail = "FAIL", f" overflow bbox={r.overflow}"
elif r.golden_ok is False:
status = "FAIL"
detail = f" golden drift: {r.golden_diff_pixels}px (max Δ={r.golden_max_delta})"
elif r.fill_ok is False:
ex, ey = r.fill_extent or (0.0, 0.0)
status = "FAIL"
detail = f" fill: extent {ex:.0%}x{ey:.0%} below required coverage"
else:
status, detail = "FAIL", ""
print(f" [{status}] {r.size_label:>7} {r.mode}{detail}")
print()
return everything_ok
def main() -> int:
parser = argparse.ArgumentParser(description="Check a plugin renders safely across sizes & screens")
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('--plugin', '-p', help='Plugin id to check')
group.add_argument('--all', action='store_true', help='Check every discovered plugin')
parser.add_argument('--plugin-dir', '-d', default=None, help='Directory to search for plugins')
parser.add_argument('--sizes', default=None, help='Comma-separated WxH list (default: all supported)')
parser.add_argument('--config', '-c', default='{}', help='Plugin config overrides as JSON')
parser.add_argument('--mock-data', '-m', default=None, help='Path to JSON file with mock cache data')
parser.add_argument('--out-dir', '-o', default=None, help='Also dump rendered PNGs here')
parser.add_argument('--skip-update', action='store_true', help='Skip calling update()')
parser.add_argument('--update-golden', action='store_true', help='Write/refresh golden images')
parser.add_argument('--golden-dir', default=None, help='Override golden dir (default: <plugin>/test/golden)')
parser.add_argument('--freeze-time', default=None,
help='Freeze wall clock, e.g. "2025-08-01 15:25:00" (for time-dependent plugins)')
args = parser.parse_args()
search_dirs = [args.plugin_dir] if args.plugin_dir else DEFAULT_SEARCH_DIRS
sizes = parse_sizes(args.sizes)
try:
config = json.loads(args.config)
except json.JSONDecodeError as e:
logger.error("Invalid --config JSON: %s", e)
return 2
if not isinstance(config, dict):
logger.error("--config must be a JSON object, got %s", type(config).__name__)
return 2
mock_data = {}
if args.mock_data:
mock_path = Path(args.mock_data)
if not mock_path.exists():
logger.error("Mock data file not found: %s", args.mock_data)
return 2
with open(mock_path) as f:
mock_data = json.load(f)
if not isinstance(mock_data, dict):
logger.error("--mock-data must be a JSON object (key -> cache value), got %s",
type(mock_data).__name__)
return 2
plugin_ids = discover_plugins(search_dirs) if args.all else [args.plugin]
if not plugin_ids:
logger.error("No plugins found in: %s", search_dirs)
return 2
out_dir = Path(args.out_dir) if args.out_dir else None
golden_dir_override = Path(args.golden_dir) if args.golden_dir else None
all_results: Dict[str, List[RenderResult]] = {}
for plugin_id in plugin_ids:
all_results[plugin_id] = check_one(
plugin_id=plugin_id, search_dirs=search_dirs, sizes=sizes,
mock_data=mock_data, config=config, run_update=not args.skip_update,
out_dir=out_dir, update_golden=args.update_golden,
golden_dir_override=golden_dir_override, freeze_time=args.freeze_time,
)
# When refreshing goldens we skip drift comparison, but a crash or overflow
# still means the plugin is broken — never let --update-golden mask that.
ok = print_report(all_results)
return 0 if ok else 1
if __name__ == '__main__':
sys.exit(main())
-29
View File
@@ -1,29 +0,0 @@
#!/bin/bash
# Clear all plugin dependency markers to force fresh dependency check
# Useful after updating plugins or troubleshooting dependency issues
echo "Clearing plugin dependency markers..."
# Check both possible cache locations
CACHE_DIRS=(
"/var/cache/ledmatrix"
"$HOME/.cache/ledmatrix"
)
for CACHE_DIR in "${CACHE_DIRS[@]}"; do
if [ -d "$CACHE_DIR" ]; then
echo "Checking $CACHE_DIR..."
marker_count=$(find "$CACHE_DIR" -name "plugin_*_deps_installed" 2>/dev/null | wc -l)
if [ "$marker_count" -gt 0 ]; then
echo "Found $marker_count dependency marker(s) in $CACHE_DIR"
find "$CACHE_DIR" -name "plugin_*_deps_installed" -delete
echo "Cleared $marker_count marker(s)"
else
echo "No dependency markers found in $CACHE_DIR"
fi
fi
done
echo "Done! Dependency markers cleared."
echo "Next startup will check and install dependencies as needed."
-2
View File
@@ -3,8 +3,6 @@
Check what imports are actually in the app.py file on the Pi
"""
import sys
import os
from pathlib import Path
# Read the app.py file and check the import lines
+3 -2
View File
@@ -67,8 +67,9 @@ def main():
print(" 📍 Will run on: http://0.0.0.0:5000")
print(" ⏹️ Press Ctrl+C to stop")
# Run the app (this should start the server)
app.run(host='0.0.0.0', port=5000, debug=True)
# Run the app (debug mode controlled by env var to satisfy security scanners)
_debug = os.environ.get('LEDMATRIX_FLASK_DEBUG', '0') == '1'
app.run(host='0.0.0.0', port=5000, debug=_debug)
except KeyboardInterrupt:
print("\n ⏹️ Server stopped by user")
+1 -1
View File
@@ -203,7 +203,7 @@ link_github_plugin() {
log_info "Repository already exists at $target_dir"
if [[ -d "$target_dir/.git" ]]; then
log_info "Updating repository..."
(cd "$target_dir" && git pull --rebase || true)
(cd "$target_dir" && git pull --rebase) || true
fi
else
# Clone the repository
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env python3
"""
Pillow compatibility smoke test.
Exercises the Pillow APIs used throughout LEDMatrix to verify a new
Pillow version doesn't break image rendering, font handling, or resize ops.
Run after upgrading Pillow:
python3 scripts/dev/test_pillow_compat.py
"""
import sys
def check(label, fn):
try:
result = fn()
print(f"{label}" + (f"{result}" if result is not None else ""))
return True
except Exception as e:
print(f"{label}{type(e).__name__}: {e}", file=sys.stderr)
return False
def main():
from PIL import Image, ImageDraw, ImageFont
import PIL
print(f"Pillow {PIL.__version__} on Python {sys.version.split()[0]}\n")
failures = 0
print("Image creation:")
failures += not check("Image.new RGB",
lambda: Image.new('RGB', (128, 32), (0, 0, 0)).size)
failures += not check("Image.new RGBA",
lambda: Image.new('RGBA', (64, 64), (255, 0, 0, 128)).size)
failures += not check("Image.new 1-bit",
lambda: Image.new('1', (16, 16)).size)
print("\nDraw operations:")
img = Image.new('RGB', (128, 32), (0, 0, 0))
draw = ImageDraw.Draw(img)
font = ImageFont.load_default()
failures += not check("draw.rectangle",
lambda: draw.rectangle([0, 0, 127, 31], outline=(255, 0, 0)))
failures += not check("draw.text",
lambda: draw.text((2, 2), "Hello", fill=(255, 255, 255), font=font))
failures += not check("draw.line",
lambda: draw.line([0, 0, 127, 31], fill=(0, 255, 0)))
print("\nFont metrics (used in text_helper, scroll_helper):")
failures += not check("draw.textlength",
lambda: f"{draw.textlength('Test', font=font):.1f}px")
failures += not check("draw.textbbox",
lambda: draw.textbbox((0, 0), "Test", font=font))
print("\nResampling (used in logo_helper, image_utils, sports base):")
logo = Image.new('RGBA', (200, 200), (255, 128, 0, 200))
failures += not check("Image.Resampling.LANCZOS exists",
lambda: str(Image.Resampling.LANCZOS))
failures += not check("thumbnail with LANCZOS",
lambda: (logo.thumbnail((64, 32), Image.Resampling.LANCZOS), logo.size)[1])
big = Image.new('RGB', (300, 300), (0, 128, 255))
failures += not check("resize with LANCZOS",
lambda: big.resize((128, 32), Image.Resampling.LANCZOS).size)
print("\nComposite / paste (used in display rendering):")
base = Image.new('RGB', (128, 32), (0, 0, 0))
overlay = Image.new('RGBA', (32, 32), (255, 0, 0, 128))
failures += not check("paste RGBA onto RGB",
lambda: (base.paste(overlay.convert('RGB'), (0, 0)), base.size)[1])
failures += not check("Image.alpha_composite",
lambda: Image.alpha_composite(
Image.new('RGBA', (32, 32)), overlay).size)
print("\nImage I/O:")
import io
buf = io.BytesIO()
img.save(buf, format='PNG')
buf.seek(0)
failures += not check("save/load PNG roundtrip",
lambda: Image.open(buf).size)
print()
if failures == 0:
print(f"All checks passed. Pillow {PIL.__version__} is compatible.")
return 0
else:
print(f"{failures} check(s) failed — review output above.", file=sys.stderr)
return 1
if __name__ == '__main__':
sys.exit(main())
-1
View File
@@ -15,7 +15,6 @@ Usage: python tools/validate_python.py <python_file>
import ast
import sys
import os
from pathlib import Path
def validate_file(filepath: str) -> bool:
"""Validate a Python file for common issues."""
+152 -72
View File
@@ -16,6 +16,7 @@ Opens at http://localhost:5001
import sys
import os
import json
import re
import time
import argparse
import logging
@@ -44,6 +45,10 @@ MAX_HEIGHT = 512
MIN_WIDTH = 1
MIN_HEIGHT = 1
# plugin_id arrives in request input and is used to build filesystem paths —
# allowlist it (same pattern the web UI's pages_v3 uses)
_SAFE_PLUGIN_ID_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
# --------------------------------------------------------------------------
# Plugin discovery
@@ -106,15 +111,30 @@ def discover_plugins() -> List[Dict[str, Any]]:
def find_plugin_dir(plugin_id: str) -> Optional[Path]:
"""Find a plugin directory by ID."""
"""Find a plugin directory by ID.
plugin_id comes from request input: it must pass an allowlist match,
and the resulting directory is normalized and required to live inside
one of the plugin search dirs, so a crafted id can never name a path
outside them.
"""
if not isinstance(plugin_id, str) or not _SAFE_PLUGIN_ID_RE.match(plugin_id):
return None
from src.plugin_system.plugin_loader import PluginLoader
loader = PluginLoader()
for search_dir in get_search_dirs():
if not search_dir.exists():
continue
result = loader.find_plugin_directory(plugin_id, search_dir)
if result:
return Path(result)
if not result:
continue
# Normalize WITHOUT following symlinks (dev plugins are often
# symlinked into plugins/) and require lexical containment in the
# search dir, so no id can ever name a path outside it.
result_abs = os.path.abspath(str(result))
root_abs = os.path.abspath(str(search_dir))
if os.path.commonpath([result_abs, root_abs]) == root_abs:
return Path(result_abs)
return None
@@ -176,6 +196,72 @@ def api_plugin_defaults(plugin_id):
return jsonify({'defaults': defaults})
def _render_once(plugin_id, plugin_dir, manifest, config, mock_data, width, height,
skip_update):
"""Render one plugin at one size. Returns the /api/render response dict.
Thin wrapper over the shared render service (also used by the web UI's
config-page preview); a fresh plugin instance per call, mirroring the
safety harness, so sizes never share state.
"""
from src.plugin_system.testing.render_service import render_plugin_once
return render_plugin_once(
plugin_id, plugin_dir, manifest=manifest, config=config,
mock_data=mock_data, width=width, height=height,
skip_update=skip_update)
def _trusted_plugin_dir(plugin_dir: Path) -> Optional[Path]:
"""Re-derive a plugin directory from the search dirs' own listings.
Path-injection barrier: unlike ``Path.iterdir()`` (which CodeQL doesn't
recognize as a taint-clearing enumeration), ``os.scandir()`` is. The
returned Path is built from a trusted root plus a name the filesystem
itself produced under that root via scandir request-derived strings
never enter its construction so a crafted plugin id can never make
downstream file access leave the plugin search dirs. Comparison is by
name, deliberately without symlink resolution (dev plugins are
commonly symlinked into plugins/).
"""
wanted_name = Path(os.path.normpath(str(plugin_dir))).name
for search_dir in get_search_dirs():
search_dir_str = str(search_dir)
try:
with os.scandir(search_dir_str) as entries:
for entry in entries:
if entry.name == wanted_name and entry.is_dir():
return Path(search_dir_str) / entry.name
except OSError:
continue
return None
def _parse_render_request(data):
"""Shared /api/render* request prep. Returns (plugin_dir, manifest, config,
mock_data, skip_update) or raises ValueError with a client message."""
plugin_id = data['plugin_id']
candidate_dir = find_plugin_dir(plugin_id)
# Never reuse `candidate_dir` past this point: it's built from
# request-derived input, and a variable reassigned only on some paths
# isn't a barrier CodeQL's flow analysis honors. `trusted_dir` is the
# sole name used below, always the scandir-sourced result.
trusted_dir = _trusted_plugin_dir(candidate_dir) if candidate_dir else None
if not trusted_dir:
raise LookupError(f'Plugin not found: {plugin_id}')
manifest_path = trusted_dir / 'manifest.json'
with open(manifest_path, 'r') as f:
manifest = json.load(f)
# Build config: schema defaults + user overrides
config = {'enabled': True}
config.update(load_config_defaults(trusted_dir))
config.update(data.get('config', {}))
return trusted_dir, manifest, config, data.get('mock_data', {}), data.get('skip_update', False)
@app.route('/api/render', methods=['POST'])
def api_render():
"""Render a plugin and return the display as base64 PNG."""
@@ -183,11 +269,6 @@ def api_render():
if not data or 'plugin_id' not in data:
return jsonify({'error': 'plugin_id is required'}), 400
plugin_id = data['plugin_id']
user_config = data.get('config', {})
mock_data = data.get('mock_data', {})
skip_update = data.get('skip_update', False)
try:
width = int(data.get('width', 128))
height = int(data.get('height', 32))
@@ -199,78 +280,77 @@ def api_render():
if not (MIN_HEIGHT <= height <= MAX_HEIGHT):
return jsonify({'error': f'height must be between {MIN_HEIGHT} and {MAX_HEIGHT}'}), 400
# Find plugin
plugin_dir = find_plugin_dir(plugin_id)
if not plugin_dir:
return jsonify({'error': f'Plugin not found: {plugin_id}'}), 404
# Load manifest
manifest_path = plugin_dir / 'manifest.json'
with open(manifest_path, 'r') as f:
manifest = json.load(f)
# Build config: schema defaults + user overrides
config_defaults = load_config_defaults(plugin_dir)
config = {'enabled': True}
config.update(config_defaults)
config.update(user_config)
# Create display manager and mocks
from src.plugin_system.testing import VisualTestDisplayManager, MockCacheManager, MockPluginManager
from src.plugin_system.plugin_loader import PluginLoader
display_manager = VisualTestDisplayManager(width=width, height=height)
cache_manager = MockCacheManager()
plugin_manager = MockPluginManager()
# Pre-populate cache with mock data
for key, value in mock_data.items():
cache_manager.set(key, value)
# Load plugin
loader = PluginLoader()
errors = []
warnings = []
try:
plugin_dir, manifest, config, mock_data, skip_update = _parse_render_request(data)
except LookupError:
return jsonify({'error': f"Plugin not found: {data['plugin_id']}"}), 404
except Exception:
# Bad manifest.json / schema / fixture — details go to the dev's
# console, not the HTTP response
app.logger.exception('render request preparation failed')
return jsonify({'error': 'Could not prepare render request; see server log'}), 400
try:
plugin_instance, module = loader.load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=plugin_dir,
config=config,
display_manager=display_manager,
cache_manager=cache_manager,
plugin_manager=plugin_manager,
install_deps=False,
)
except Exception as e:
return jsonify({'error': f'Failed to load plugin: {e}'}), 500
result = _render_once(data['plugin_id'], plugin_dir, manifest, config,
mock_data, width, height, skip_update)
except Exception:
app.logger.exception('plugin load failed during render')
return jsonify({'error': 'Failed to load plugin; see server log'}), 500
return jsonify(result)
start_time = time.time()
# Run update()
if not skip_update:
@app.route('/api/sizes')
def api_sizes():
"""The representative panel-size sample the safety harness renders at."""
from src.plugin_system.testing.sizes import DEFAULT_TEST_SIZES
return jsonify({'sizes': [list(s) for s in DEFAULT_TEST_SIZES]})
MAX_MATRIX_SIZES = 12
@app.route('/api/render-matrix', methods=['POST'])
def api_render_matrix():
"""Render a plugin at a list of sizes (default: the harness sample) so the
UI can show a side-by-side multi-resolution gallery."""
data = request.get_json()
if not data or 'plugin_id' not in data:
return jsonify({'error': 'plugin_id is required'}), 400
from src.plugin_system.testing.sizes import DEFAULT_TEST_SIZES
sizes = data.get('sizes') or [list(s) for s in DEFAULT_TEST_SIZES]
if len(sizes) > MAX_MATRIX_SIZES:
return jsonify({'error': f'at most {MAX_MATRIX_SIZES} sizes per request'}), 400
parsed_sizes = []
for pair in sizes:
try:
plugin_instance.update()
except Exception as e:
warnings.append(f"update() raised: {e}")
w, h = int(pair[0]), int(pair[1])
except (TypeError, ValueError, IndexError):
return jsonify({'error': f'invalid size entry {pair!r} (expected [w, h])'}), 400
if not (MIN_WIDTH <= w <= MAX_WIDTH and MIN_HEIGHT <= h <= MAX_HEIGHT):
return jsonify({'error': f'size {w}x{h} out of bounds'}), 400
parsed_sizes.append((w, h))
# Run display()
try:
plugin_instance.display(force_clear=True)
except Exception as e:
errors.append(f"display() raised: {e}")
plugin_dir, manifest, config, mock_data, skip_update = _parse_render_request(data)
except LookupError:
return jsonify({'error': f"Plugin not found: {data['plugin_id']}"}), 404
except Exception:
app.logger.exception('render request preparation failed')
return jsonify({'error': 'Could not prepare render request; see server log'}), 400
render_time_ms = round((time.time() - start_time) * 1000, 1)
return jsonify({
'image': f'data:image/png;base64,{display_manager.get_image_base64()}',
'width': width,
'height': height,
'render_time_ms': render_time_ms,
'errors': errors,
'warnings': warnings,
})
results = []
for w, h in parsed_sizes:
try:
results.append(_render_once(data['plugin_id'], plugin_dir, manifest,
config, mock_data, w, h, skip_update))
except Exception:
app.logger.exception('plugin load failed during %dx%d render', w, h)
results.append({'image': None, 'width': w, 'height': h,
'render_time_ms': 0,
'errors': ['Failed to load plugin; see server log'],
'warnings': []})
return jsonify({'results': results})
# --------------------------------------------------------------------------
-1
View File
@@ -13,7 +13,6 @@ echo ""
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Get the actual user
+1 -1
View File
@@ -41,7 +41,7 @@ if [ -f "$PROJECT_DIR/config/config.json" ]; then
echo -e "${GREEN}✓ Config file found${NC}"
# Check web_display_autostart setting
AUTOSTART=$(cat "$PROJECT_DIR/config/config.json" | grep -o '"web_display_autostart"[[:space:]]*:[[:space:]]*[a-z]*' | grep -o '[a-z]*$')
AUTOSTART=$(grep -o '"web_display_autostart"[[:space:]]*:[[:space:]]*[a-z]*' "$PROJECT_DIR/config/config.json" | grep -o '[a-z]*$')
if [ "$AUTOSTART" == "true" ]; then
echo -e "${GREEN}✓ web_display_autostart: true${NC}"
+1 -4
View File
@@ -16,11 +16,8 @@ YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Check if running as root or with sudo
if [ "$EUID" -ne 0 ]; then
if [ "$EUID" -ne 0 ]; then
echo -e "${YELLOW}Warning: Some checks require sudo. Running what we can...${NC}"
SUDO=""
else
SUDO=""
fi
PROJECT_DIR="${HOME}/LEDMatrix"
+1 -1
View File
@@ -118,7 +118,7 @@ total_count=${#ARCHITECTURES[@]}
for arch in "${!ARCHITECTURES[@]}"; do
if download_binary "$arch" "${ARCHITECTURES[$arch]}"; then
((success_count++))
success_count=$((success_count + 1))
fi
done
@@ -7,12 +7,6 @@ echo "Fixing LEDMatrix assets directory permissions..."
# Get the real user (not root when running with sudo)
REAL_USER=${SUDO_USER:-$USER}
# Resolve the home directory of the real user robustly
if command -v getent >/dev/null 2>&1; then
REAL_HOME=$(getent passwd "$REAL_USER" | cut -d: -f6)
else
REAL_HOME=$(eval echo ~"$REAL_USER")
fi
REAL_GROUP=$(id -gn "$REAL_USER")
# Get the project directory
+74
View File
@@ -0,0 +1,74 @@
#!/bin/bash
# safe_pip_install.sh — Install a requirements.txt as root after validating
# that the resolved path is the project's own requirements.txt or a plugin's
# requirements.txt under plugin-repos/ or plugins/.
#
# This script is intended to be called via sudo from the web interface, so
# that packages a plugin declares end up visible to ledmatrix.service (which
# runs as root) rather than only to whichever non-root user runs the web
# interface. Plugin code already runs as root once loaded, so installing its
# declared dependencies as root is not a new trust boundary.
#
# Usage: safe_pip_install.sh <requirements_txt_path>
set -euo pipefail
if [ $# -ne 1 ]; then
echo "Usage: $0 <requirements_txt_path>" >&2
exit 1
fi
TARGET="$1"
# Determine the project root (parent of scripts/fix_perms/)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Allowed locations (resolved, no trailing slash):
# - the project's own requirements.txt
# - any requirements.txt under plugin-repos/ or plugins/
ALLOWED_EXACT="$(realpath --canonicalize-missing "$PROJECT_ROOT/requirements.txt")"
ALLOWED_BASES=(
"$(realpath --canonicalize-missing "$PROJECT_ROOT/plugin-repos")"
"$(realpath --canonicalize-missing "$PROJECT_ROOT/plugins")"
)
# Resolve the target path (follow symlinks); works even if it doesn't exist.
RESOLVED_TARGET="$(realpath --canonicalize-missing "$TARGET")"
# Must be named requirements.txt — never install from an arbitrary file.
if [ "$(basename "$RESOLVED_TARGET")" != "requirements.txt" ]; then
echo "DENIED: $RESOLVED_TARGET is not a requirements.txt file" >&2
exit 2
fi
ALLOWED=false
if [ "$RESOLVED_TARGET" = "$ALLOWED_EXACT" ]; then
ALLOWED=true
else
for BASE in "${ALLOWED_BASES[@]}"; do
if [[ "$RESOLVED_TARGET" == "$BASE/"* ]]; then
ALLOWED=true
break
fi
done
fi
if [ "$ALLOWED" = false ]; then
echo "DENIED: $RESOLVED_TARGET is not an allowed requirements.txt location" >&2
echo "Allowed: $ALLOWED_EXACT, or any requirements.txt under: ${ALLOWED_BASES[*]}" >&2
exit 2
fi
if [ ! -f "$RESOLVED_TARGET" ]; then
echo "ERROR: $RESOLVED_TARGET does not exist" >&2
exit 3
fi
PYTHON_PATH="$(command -v python3)"
# --ignore-installed: root's site-packages often has apt/dpkg-managed copies
# of common libraries (requests, urllib3, ...) with no pip RECORD file, which
# pip refuses to uninstall in place ("Cannot uninstall: no RECORD file was
# found"). This tells pip to install the newer version alongside rather than
# aborting the whole requirements.txt install over one such conflict.
exec "$PYTHON_PATH" -m pip install --break-system-packages --ignore-installed -r "$RESOLVED_TARGET"
+28 -5
View File
@@ -33,6 +33,7 @@ POWEROFF_PATH=$(command -v poweroff) || true
BASH_PATH=$(command -v bash) || true
JOURNALCTL_PATH=$(command -v journalctl) || true
SAFE_RM_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_plugin_rm.sh"
SAFE_PIP_INSTALL_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh"
# Validate required commands (systemctl, bash, python3 are essential)
for CMD_NAME in SYSTEMCTL_PATH BASH_PATH PYTHON_PATH; do
@@ -48,11 +49,15 @@ if [ ${#MISSING_CMDS[@]} -gt 0 ]; then
exit 1
fi
# Validate helper script exists
# Validate helper scripts exist
if [ ! -f "$SAFE_RM_PATH" ]; then
echo "Error: Safe plugin removal helper not found: $SAFE_RM_PATH" >&2
exit 1
fi
if [ ! -f "$SAFE_PIP_INSTALL_PATH" ]; then
echo "Error: Safe pip install helper not found: $SAFE_PIP_INSTALL_PATH" >&2
exit 1
fi
echo "Command paths:"
echo " Python: $PYTHON_PATH"
@@ -62,6 +67,7 @@ echo " Poweroff: ${POWEROFF_PATH:-(not found, skipping)}"
echo " Bash: $BASH_PATH"
echo " Journalctl: ${JOURNALCTL_PATH:-(not found, skipping)}"
echo " Safe plugin rm: $SAFE_RM_PATH"
echo " Safe pip install: $SAFE_PIP_INSTALL_PATH"
# Create a temporary sudoers file
TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
@@ -101,13 +107,22 @@ TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
fi
# Required: python3, bash
echo "$WEB_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_DIR/display_controller.py"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_DIR/start_display.sh"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_DIR/stop_display.sh"
# NOTE: display_controller.py/start_display.sh/stop_display.sh live at the
# project root, not under scripts/install/ (where this script lives) —
# must use PROJECT_ROOT here, not PROJECT_DIR.
echo "$WEB_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT/display_controller.py"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/start_display.sh"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/stop_display.sh"
echo ""
echo "# Allow web user to remove plugin directories via vetted helper script"
echo "# The helper validates that the target path resolves inside plugin-repos/ or plugins/"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $SAFE_RM_PATH *"
echo ""
echo "# Allow web user to install a plugin's requirements.txt as root via vetted"
echo "# helper script, so packages are visible to root-run ledmatrix.service"
echo "# (not just the web interface's own user). The helper validates the target"
echo "# is requirements.txt at the project root or under plugin-repos/ or plugins/."
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $SAFE_PIP_INSTALL_PATH *"
} > "$TEMP_SUDOERS"
echo ""
@@ -126,6 +141,7 @@ echo "- Run display_controller.py directly"
echo "- Execute start_display.sh and stop_display.sh"
echo "- Reboot and shutdown the system"
echo "- Remove plugin directories (for update/uninstall when root-owned files block deletion)"
echo "- Install plugin/base requirements.txt as root (so ledmatrix.service can see them)"
echo ""
# Ask for confirmation
@@ -147,6 +163,13 @@ fi
if ! sudo chmod 755 "$SAFE_RM_PATH"; then
echo "Warning: Could not set permissions on $SAFE_RM_PATH"
fi
echo "Hardening safe_pip_install.sh ownership..."
if ! sudo chown root:root "$SAFE_PIP_INSTALL_PATH"; then
echo "Warning: Could not set ownership on $SAFE_PIP_INSTALL_PATH"
fi
if ! sudo chmod 755 "$SAFE_PIP_INSTALL_PATH"; then
echo "Warning: Could not set permissions on $SAFE_PIP_INSTALL_PATH"
fi
if sudo cp "$TEMP_SUDOERS" /etc/sudoers.d/ledmatrix_web; then
echo "Configuration applied successfully!"
@@ -160,7 +183,7 @@ if sudo cp "$TEMP_SUDOERS" /etc/sudoers.d/ledmatrix_web; then
echo "✗ systemctl status ledmatrix.service - Failed"
fi
if sudo -n test -f "$PROJECT_DIR/start_display.sh"; then
if sudo -n test -f "$PROJECT_ROOT/start_display.sh"; then
echo "✓ File access test - OK"
else
echo "✗ File access test - Failed"
+2 -4
View File
@@ -14,9 +14,6 @@ else
ACTUAL_USER=$(whoami)
fi
# Get the home directory of the actual user
USER_HOME=$(eval echo ~$ACTUAL_USER)
# Determine the Project Root Directory (parent of scripts/install/)
PROJECT_ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
@@ -34,7 +31,8 @@ echo "Generating service file with dynamic paths..."
WEB_SERVICE_FILE_CONTENT=$(cat <<EOF
[Unit]
Description=LED Matrix Web Interface Service
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
+8 -2
View File
@@ -340,9 +340,14 @@ main() {
echo ""
# Execute with proper error handling and non-interactive mode
# Temporarily disable errexit to capture exit code instead of exiting immediately
# Temporarily disable errexit AND the ERR trap to capture exit code instead of
# exiting immediately. `set +e` alone does not suppress the ERR trap, so without
# `trap '' ERR` a non-zero exit from first_time_install.sh would trigger on_error
# here with the generic "Main installation" message instead of the detailed
# if/else handling below.
set +e
trap '' ERR
# Check /tmp permissions - only fix if actually wrong (common in automated scenarios)
# When running manually, /tmp usually has correct permissions (1777)
TMP_PERMS=$(stat -c '%a' /tmp 2>/dev/null || echo "unknown")
@@ -370,6 +375,7 @@ main() {
sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 bash ./first_time_install.sh -y </dev/null
fi
INSTALL_EXIT_CODE=$?
trap 'on_error $LINENO' ERR # Re-enable ERR trap
set -e # Re-enable errexit
if [ $INSTALL_EXIT_CODE -eq 0 ]; then
+151 -84
View File
@@ -6,82 +6,143 @@ then falls back to pip with --break-system-packages
import subprocess
import sys
import tempfile
import warnings
from collections import deque
from pathlib import Path
from typing import List, Tuple
def install_via_apt(package_name):
"""Try to install a package via apt."""
try:
# Map pip package names to apt package names
apt_package_map = {
'flask': 'python3-flask',
'PIL': 'python3-pil',
'freetype': 'python3-freetype',
'psutil': 'python3-psutil',
'werkzeug': 'python3-werkzeug',
'numpy': 'python3-numpy',
'requests': 'python3-requests',
'python-dateutil': 'python3-dateutil',
'pytz': 'python3-tz',
'geopy': 'python3-geopy',
'unidecode': 'python3-unidecode',
'websockets': 'python3-websockets',
'websocket-client': 'python3-websocket-client'
}
apt_package = apt_package_map.get(package_name, f'python3-{package_name}')
print(f"Trying to install {apt_package} via apt...")
subprocess.check_call([
'sudo', 'apt', 'update'
], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
subprocess.check_call([
'sudo', 'apt', 'install', '-y', apt_package
], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
# How many trailing lines of a failed command's output to keep for the
# end-of-run failure summary. Keeps the root cause near the end of the log,
# which is where first_time_install.sh's error handler tails from.
ERROR_TAIL_LINES = 15
def _run(cmd: List[str]) -> Tuple[bool, str]:
"""Run a command, streaming combined stdout/stderr to a temp file.
Returns (success, output) instead of raising, so callers can report
*why* a command failed rather than just that it failed. `output` is
bounded to the last ERROR_TAIL_LINES lines so failures from very
chatty commands (e.g. pip build logs) don't get buffered in memory.
"""
with tempfile.TemporaryFile(mode='w+b') as f:
result = subprocess.run(cmd, stdout=f, stderr=subprocess.STDOUT) # nosec B603 B607 - hardcoded apt/pip args # nosemgrep
f.seek(0)
# Stream line-by-line so only the last ERROR_TAIL_LINES are ever held
# in memory, regardless of how much output the command produced.
tail = deque(
(line.decode('utf-8', errors='replace').rstrip('\n') for line in f),
maxlen=ERROR_TAIL_LINES,
)
return result.returncode == 0, '\n'.join(tail)
def install_via_apt(package_name: str) -> Tuple[bool, str]:
"""Try to install a package via apt. Returns (success, output)."""
# Map pip package names to apt package names
apt_package_map = {
'flask': 'python3-flask',
'PIL': 'python3-pil',
'freetype': 'python3-freetype',
'psutil': 'python3-psutil',
'werkzeug': 'python3-werkzeug',
'numpy': 'python3-numpy',
'requests': 'python3-requests',
'python-dateutil': 'python3-dateutil',
'pytz': 'python3-tz',
'geopy': 'python3-geopy',
'unidecode': 'python3-unidecode',
'websockets': 'python3-websockets',
'websocket-client': 'python3-websocket-client'
}
apt_package = apt_package_map.get(package_name, f'python3-{package_name}')
print(f"Trying to install {apt_package} via apt...")
success, output = _run(['sudo', 'apt-get', '-o', 'DPkg::Lock::Timeout=180', 'install', '-y', apt_package])
if success:
print(f"Successfully installed {apt_package} via apt")
return True
except subprocess.CalledProcessError:
print(f"Failed to install {package_name} via apt, will try pip")
return False
return True, ""
def install_via_pip(package_name):
print(f"Failed to install {apt_package} via apt, will try pip")
return False, output
def install_via_pip(package_name: str) -> Tuple[bool, str]:
"""Install a package via pip with --break-system-packages and --prefer-binary.
--break-system-packages allows pip to install into the system Python on
Debian/Ubuntu-based systems without a virtual environment.
--prefer-binary prefers pre-built wheels over source distributions to avoid
exhausting /tmp space during compilation.
"""
try:
print(f"Installing {package_name} via pip...")
subprocess.check_call([
sys.executable, '-m', 'pip', 'install', '--break-system-packages', '--prefer-binary', package_name
])
print(f"Successfully installed {package_name} via pip")
return True
except subprocess.CalledProcessError as e:
print(f"Failed to install {package_name} via pip: {e}")
return False
--ignore-installed stops pip from trying to *uninstall* packages that were
installed by apt (e.g. python3-requests). Those Debian packages ship no
pip RECORD file, so an uninstall attempt fails with "uninstall-no-record-file"
and aborts the whole install. With --ignore-installed, pip lays the new
version down in /usr/local where it shadows the apt copy instead of removing
it. This matters when a pip dependency (google-api-python-client pulls a
newer requests) needs to upgrade an apt-managed package.
def check_package_installed(package_name):
Returns (success, output).
"""
print(f"Installing {package_name} via pip...")
success, output = _run([
sys.executable, '-m', 'pip', 'install',
'--break-system-packages', '--prefer-binary', '--ignore-installed', package_name
])
if success:
print(f"Successfully installed {package_name} via pip")
return True, ""
print(f"Failed to install {package_name} via pip (see failure summary at end of log)")
return False, output
# Distribution (pip/apt) names whose importable module name differs.
IMPORT_NAME_MAP = {
'python-dateutil': 'dateutil',
'websocket-client': 'websocket',
}
def check_package_installed(package_name: str) -> bool:
"""Check if a package is already installed."""
import_name = IMPORT_NAME_MAP.get(package_name, package_name)
# Suppress deprecation warnings when checking if packages are installed
# (we're just checking, not using them)
with warnings.catch_warnings():
warnings.filterwarnings('ignore', category=DeprecationWarning)
try:
__import__(package_name)
__import__(import_name)
return True
except ImportError:
return False
def print_failure_summary(failed_packages: List[str], failure_details: dict) -> None:
print("\n" + "=" * 60)
print("DEPENDENCY INSTALLATION FAILURES - DETAILS")
print("=" * 60)
for package in failed_packages:
print(f"\nPackage: {package}")
print("-" * 40)
output = failure_details.get(package, "").strip()
if not output:
print(" (no output captured)")
continue
for line in output.splitlines()[-ERROR_TAIL_LINES:]:
print(f" {line}")
print("=" * 60)
def main():
"""Main installation function."""
print("Installing dependencies for LED Matrix Web Interface V2...")
print("Refreshing apt package index...")
_run(['sudo', 'apt', 'update']) # best-effort; individual installs surface their own errors
# List of required packages
required_packages = [
'flask',
@@ -98,19 +159,23 @@ def main():
'websockets',
'websocket-client'
]
failed_packages = []
failure_details = {}
for package in required_packages:
if check_package_installed(package):
print(f"{package} is already installed")
continue
# Try apt first, then pip
if not install_via_apt(package):
if not install_via_pip(package):
ok, apt_output = install_via_apt(package)
if not ok:
ok, pip_output = install_via_pip(package)
if not ok:
failed_packages.append(package)
failure_details[package] = pip_output or apt_output
# Install packages that don't have apt equivalents
special_packages = [
'timezonefinder>=6.5.0,<7.0.0',
@@ -122,47 +187,49 @@ def main():
'python-socketio>=5.11.0,<6.0.0',
'python-engineio>=4.9.0,<5.0.0'
]
for package in special_packages:
if not install_via_pip(package):
ok, pip_output = install_via_pip(package)
if not ok:
failed_packages.append(package)
failure_details[package] = pip_output
# Install rgbmatrix module from local source (optional - may already be installed in Step 6)
# Check if already installed first
if check_package_installed('rgbmatrix'):
print("rgbmatrix module already installed, skipping...")
else:
print("Installing rgbmatrix module from local source...")
try:
# Get project root (parent of scripts directory)
PROJECT_ROOT = Path(__file__).parent.parent
rgbmatrix_path = PROJECT_ROOT / 'rpi-rgb-led-matrix-master' / 'bindings' / 'python'
if rgbmatrix_path.exists():
# Check if the module has been built (look for setup.py)
setup_py = rgbmatrix_path / 'setup.py'
if setup_py.exists():
# Try installing - use regular install, not editable mode
# This is optional for web interface and should already be installed in Step 6
subprocess.check_call([
sys.executable, '-m', 'pip', 'install', '--break-system-packages', str(rgbmatrix_path)
], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
# Get project root (parent of scripts directory)
PROJECT_ROOT = Path(__file__).parent.parent
rgbmatrix_path = PROJECT_ROOT / 'rpi-rgb-led-matrix-master' / 'bindings' / 'python'
if rgbmatrix_path.exists():
# Check if the module has been built (look for setup.py)
setup_py = rgbmatrix_path / 'setup.py'
if setup_py.exists():
# Try installing - use regular install, not editable mode
# This is optional for web interface and should already be installed in Step 6
ok, output = _run([sys.executable, '-m', 'pip', 'install', '--break-system-packages', str(rgbmatrix_path)])
if ok:
print("rgbmatrix module installed successfully")
else:
print("Warning: rgbmatrix setup.py not found, module may need to be built first")
print(" This is normal if Step 6 hasn't completed yet.")
# Don't fail the whole installation - rgbmatrix is optional for web interface
# and should be installed in Step 6 of first_time_install.sh
print("Warning: Failed to install rgbmatrix module:")
for line in output.strip().splitlines()[-ERROR_TAIL_LINES:]:
print(f" {line}")
print(" This is normal if rgbmatrix hasn't been built yet (Step 6).")
print(" The web interface will work without it.")
else:
print("Warning: rgbmatrix source not found (this is normal if Step 6 hasn't run yet)")
except subprocess.CalledProcessError as e:
# Don't fail the whole installation - rgbmatrix is optional for web interface
# and should be installed in Step 6 of first_time_install.sh
print(f"Warning: Failed to install rgbmatrix module: {e}")
print(" This is normal if rgbmatrix hasn't been built yet (Step 6).")
print(" The web interface will work without it.")
# Don't add to failed_packages since it's optional
print("Warning: rgbmatrix setup.py not found, module may need to be built first")
print(" This is normal if Step 6 hasn't completed yet.")
else:
print("Warning: rgbmatrix source not found (this is normal if Step 6 hasn't run yet)")
if failed_packages:
print(f"\nFailed to install the following packages: {failed_packages}")
print("You may need to install them manually or check your system configuration.")
print_failure_summary(failed_packages, failure_details)
return False
else:
print("\nAll dependencies installed successfully!")
+4 -39
View File
@@ -17,7 +17,6 @@ import os
import json
import argparse
from pathlib import Path
from typing import Any, Dict, Optional, Sequence, Union
# Add project root to path
PROJECT_ROOT = Path(__file__).resolve().parent.parent
@@ -28,49 +27,15 @@ os.environ['EMULATOR'] = 'true'
# Import logger after path setup so src.logging_config is importable
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_manifest, load_config_defaults,
)
logger = get_logger("[Render Plugin]")
MIN_DIMENSION = 1
MAX_DIMENSION = 512
def find_plugin_dir(plugin_id: str, search_dirs: Sequence[Union[str, Path]]) -> Optional[Path]:
"""Find a plugin directory by searching multiple paths."""
from src.plugin_system.plugin_loader import PluginLoader
loader = PluginLoader()
for search_dir in search_dirs:
search_path = Path(search_dir)
if not search_path.exists():
continue
result = loader.find_plugin_directory(plugin_id, search_path)
if result:
return Path(result)
return None
def load_manifest(plugin_dir: Path) -> Dict[str, Any]:
"""Load and return manifest.json from plugin directory."""
manifest_path = plugin_dir / 'manifest.json'
if not manifest_path.exists():
raise FileNotFoundError(f"No manifest.json in {plugin_dir}")
with open(manifest_path, 'r') as f:
return json.load(f)
def load_config_defaults(plugin_dir: Path) -> Dict[str, Any]:
"""Extract default values from config_schema.json."""
schema_path = plugin_dir / 'config_schema.json'
if not schema_path.exists():
return {}
with open(schema_path, 'r') as f:
schema = json.load(f)
defaults: Dict[str, Any] = {}
for key, prop in schema.get('properties', {}).items():
if 'default' in prop:
defaults[key] = prop['default']
return defaults
def main() -> int:
"""Load a plugin, call update() + display(), and save the result as a PNG image."""
parser = argparse.ArgumentParser(description='Render a plugin display to a PNG image')
@@ -81,7 +46,7 @@ def main() -> int:
help='Plugin config as JSON string')
parser.add_argument('--mock-data', '-m', default=None,
help='Path to JSON file with mock cache data')
parser.add_argument('--output', '-o', default='/tmp/plugin_render.png',
parser.add_argument('--output', '-o', default='/tmp/plugin_render.png', # nosec B108 - dev script default; user can override
help='Output PNG path (default: /tmp/plugin_render.png)')
parser.add_argument('--width', type=int, default=128, help='Display width (default: 128)')
parser.add_argument('--height', type=int, default=32, help='Display height (default: 32)')
-5
View File
@@ -7,9 +7,7 @@ Supports both unittest and pytest.
"""
import sys
import os
import argparse
import subprocess
from pathlib import Path
from typing import Optional
@@ -198,17 +196,14 @@ def main():
if runner == 'auto':
# Try pytest first, fall back to unittest
try:
import pytest
runner = 'pytest'
except ImportError:
runner = 'unittest'
# Run tests
if runner == 'pytest':
import importlib.util
return run_pytest_tests(test_files, args.verbose, args.coverage)
else:
import importlib.util
return run_unittest_tests(test_files, args.verbose)
+125 -1
View File
@@ -209,6 +209,11 @@
onchange="onConfigChange()">
<span class="text-xs ml-2" style="color: var(--text-secondary);">px</span>
</div>
<select id="sizePreset" onchange="applySizePreset()"
class="w-full mt-2 px-2 py-1.5 rounded text-xs"
style="background: var(--bg-primary); color: var(--text-secondary); border: 1px solid var(--border-color);">
<option value="">Preset sizes…</option>
</select>
</div>
<!-- Config form -->
@@ -242,13 +247,18 @@
</div>
</details>
<!-- Render button -->
<!-- Render buttons -->
<div class="flex gap-2">
<button onclick="renderPlugin()" id="renderBtn"
class="flex-1 px-4 py-2.5 rounded-lg text-sm font-medium text-white"
style="background: var(--accent);">
Render
</button>
<button onclick="renderAllSizes()" id="renderAllBtn" title="Render at every harness test size"
class="px-4 py-2.5 rounded-lg text-sm font-medium"
style="background: var(--bg-tertiary); color: var(--text-primary); border: 1px solid var(--border-color);">
All Sizes
</button>
</div>
</div>
@@ -311,6 +321,15 @@
<div id="messagesPanel" class="panel p-3 hidden">
<div id="messagesList" class="text-xs font-mono space-y-1"></div>
</div>
<!-- Multi-size gallery -->
<div id="galleryPanel" class="panel p-4 hidden">
<div class="flex items-center justify-between mb-3">
<span class="text-xs font-medium" style="color: var(--text-secondary);">All Sizes</span>
<span class="text-xs" style="color: var(--text-secondary);" id="galleryStatus"></span>
</div>
<div id="galleryGrid" class="flex flex-wrap gap-4 items-start"></div>
</div>
</div>
</div>
@@ -340,8 +359,30 @@
opt.textContent = `${p.name} (${p.id})`;
select.appendChild(opt);
});
// Load harness size presets
try {
const sizesRes = await fetch('/api/sizes');
const sizesData = await sizesRes.json();
const preset = document.getElementById('sizePreset');
(sizesData.sizes || []).forEach(([w, h]) => {
const opt = document.createElement('option');
opt.value = `${w}x${h}`;
opt.textContent = `${w} x ${h}`;
preset.appendChild(opt);
});
} catch (e) { /* presets are a convenience; ignore */ }
});
function applySizePreset() {
const value = document.getElementById('sizePreset').value;
if (!value) return;
const [w, h] = value.split('x');
document.getElementById('displayWidth').value = w;
document.getElementById('displayHeight').value = h;
onConfigChange();
}
// ---------- Plugin selection ----------
async function onPluginChange() {
const pluginId = document.getElementById('pluginSelect').value;
@@ -485,6 +526,89 @@
}
}
// ---------- Multi-size gallery ----------
async function renderAllSizes() {
if (!currentPluginId) return;
const btn = document.getElementById('renderAllBtn');
const panel = document.getElementById('galleryPanel');
const grid = document.getElementById('galleryGrid');
const status = document.getElementById('galleryStatus');
btn.disabled = true;
btn.textContent = 'Rendering…';
panel.classList.remove('hidden');
grid.innerHTML = '';
status.textContent = 'Rendering at all harness sizes…';
const config = jsonEditor ? jsonEditor.getValue() : {};
config.enabled = true;
let mockData = {};
const mockInput = document.getElementById('mockDataInput').value.trim();
if (mockInput) {
try { mockData = JSON.parse(mockInput); }
catch (e) { showMessages([], [`Mock data JSON error: ${e.message}`]); }
}
try {
const res = await fetch('/api/render-matrix', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
plugin_id: currentPluginId,
config: config,
mock_data: mockData,
}),
});
const data = await res.json();
if (data.error) {
status.textContent = data.error;
return;
}
let failures = 0;
(data.results || []).forEach(r => {
const cell = document.createElement('div');
cell.style.cssText = 'display:flex;flex-direction:column;gap:4px;';
const failed = (r.errors || []).length > 0 || !r.image;
if (failed) failures++;
const label = document.createElement('span');
label.className = 'text-xs font-mono';
label.style.color = failed ? '#f87171' : 'var(--text-secondary)';
label.textContent = `${r.width}x${r.height} · ${r.render_time_ms}ms`;
cell.appendChild(label);
if (r.image) {
const img = document.createElement('img');
img.src = r.image;
// Small panels get 2x zoom so they stay legible in the grid
const zoom = r.height >= 128 ? 1 : 2;
img.style.cssText =
`image-rendering: pixelated; width:${r.width * zoom}px; ` +
`height:${r.height * zoom}px; ` +
`border:1px solid ${failed ? '#f87171' : 'var(--border-color)'};`;
cell.appendChild(img);
}
if (failed) {
const err = document.createElement('span');
err.className = 'text-xs font-mono';
err.style.color = '#f87171';
err.textContent = (r.errors || ['render failed']).join('; ');
cell.appendChild(err);
}
grid.appendChild(cell);
});
status.textContent = failures
? `${failures} size(s) failed`
: `${(data.results || []).length} sizes rendered`;
} catch (e) {
status.textContent = `Network error: ${e.message}`;
} finally {
btn.disabled = false;
btn.textContent = 'All Sizes';
}
}
// ---------- Zoom ----------
function updateZoom() {
const zoom = parseInt(document.getElementById('zoomSlider').value);
-2
View File
@@ -6,9 +6,7 @@ This script allows manual clearing of specific cache keys or all cache data.
import os
import sys
import json
import argparse
from pathlib import Path
# Add the src directory to the path so we can import our modules
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src'))
+1 -1
View File
@@ -111,7 +111,7 @@ def main():
# Ensure PYTHONPATH is set correctly if web_interface.py has relative imports to src
# The WorkingDirectory in systemd service should handle this for web_interface.py
print(f"Launching web interface v3: {sys.executable} {WEB_INTERFACE_SCRIPT}")
os.execvp(sys.executable, [sys.executable, WEB_INTERFACE_SCRIPT])
os.execvp(sys.executable, [sys.executable, WEB_INTERFACE_SCRIPT]) # nosec B606 - both args are fixed constants
except Exception as e:
print(f"Failed to exec web interface: {e}")
sys.exit(1) # Failed to start
+1 -1
View File
@@ -155,7 +155,7 @@ class WiFiMonitorDaemon:
logger.error(f"NetworkManager restart failed (rc={e.returncode}); "
"resetting failure counter to avoid tight retry loop")
self._consecutive_internet_failures = 0
except Exception as e:
except (subprocess.SubprocessError, OSError) as e:
logger.error(f"NetworkManager restart error: {e}; "
"resetting failure counter to avoid tight retry loop")
self._consecutive_internet_failures = 0
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "1.0.0"
__version__ = "3.1.0"
+174
View File
@@ -0,0 +1,174 @@
"""
Adaptive image fitting for plugins the image counterpart to
src/adaptive_layout.py's text fitting.
Promotes the proven in-field image patterns into one shared helper so
plugins stop hand-copying resize/cache code:
- "crop transparent padding, then fill the row height" (football/hockey
logo pattern) -> ``crop_to_ink=True, mode="fill_height"``
- "crop-to-fill with a top anchor for faces" (masters-tournament headshot
pattern) -> ``mode="cover", anchor="top"``
- "letterbox to fit, centered on a background" (static-image pattern)
-> ``mode="contain"``
- NEAREST for pixel art/flags vs LANCZOS for photos (masters flag pattern)
-> ``resample=RESAMPLE_NEAREST``
Unlike PIL's ``thumbnail()`` (downscale-only — the reason plugin imagery
stays tiny on big panels), ``fit_image`` upscales by default so content
genuinely adapts to larger displays; pass ``upscale=False`` for the old
behavior.
Use via ``LayoutContext.fit_image(...)`` (cached per panel size) or
``BasePlugin.draw_image(...)``; the module-level functions are the
uncached primitives.
"""
from dataclasses import dataclass
from typing import Any, Optional, Tuple
from PIL import Image
# The one Pillow >= 9.1 compat shim (replaces the per-plugin copies).
try:
RESAMPLE_LANCZOS = Image.Resampling.LANCZOS
RESAMPLE_NEAREST = Image.Resampling.NEAREST
except AttributeError: # Pillow < 9.1
RESAMPLE_LANCZOS = Image.LANCZOS
RESAMPLE_NEAREST = Image.NEAREST
FIT_MODES = ("contain", "cover", "fill_height", "stretch")
@dataclass(frozen=True)
class ImageFitResult:
"""A processed RGBA copy of a source image, sized for a target box."""
image: Image.Image
width: int
height: int
scale: float # scale applied vs the (possibly ink-cropped) source
mode: str
source_size: Tuple[int, int]
@property
def is_empty(self) -> bool:
return self.width <= 0 or self.height <= 0
_EMPTY_IMAGE = Image.new("RGBA", (1, 1), (0, 0, 0, 0))
def _empty_result(mode: str, source_size: Tuple[int, int]) -> ImageFitResult:
return ImageFitResult(_EMPTY_IMAGE, 0, 0, 0.0, mode, source_size)
def _box_dims(box: Any) -> Tuple[int, int]:
"""Accept a Region (duck-typed .w/.h) or a (w, h) tuple."""
if hasattr(box, "w") and hasattr(box, "h"):
return (int(box.w), int(box.h))
w, h = box
return (int(w), int(h))
def fit_image(img: Image.Image, box: Any, *, mode: str = "contain",
crop_to_ink: bool = False, anchor: str = "center",
resample: Any = None, upscale: bool = True) -> ImageFitResult:
"""Fit an image into a box, preserving crispness policy per content type.
Args:
img: Source PIL image (any mode; output is always RGBA).
box: Region or (w, h) target box.
mode: "contain" (letterbox), "cover" (crop-to-fill),
"fill_height" (height == box height, contain-capped by width),
"stretch" (exact resize).
crop_to_ink: Trim fully-transparent padding (getbbox) before fitting
logos shipped with generous padding otherwise render small.
anchor: For "cover" crops: "center" or "top" (keeps faces/tops).
resample: PIL resampling filter; defaults to RESAMPLE_LANCZOS.
Use RESAMPLE_NEAREST for pixel art, flags, and sprite icons.
upscale: Allow scaling above source size (default True the adaptive
point). False mimics the legacy thumbnail() behavior.
"""
if mode not in FIT_MODES:
raise ValueError(f"Unknown fit mode '{mode}' (expected one of {FIT_MODES})")
box_w, box_h = _box_dims(box)
if box_w <= 0 or box_h <= 0 or img.width <= 0 or img.height <= 0:
return _empty_result(mode, img.size)
resample = RESAMPLE_LANCZOS if resample is None else resample
work = img if img.mode == "RGBA" else img.convert("RGBA")
if crop_to_ink:
bbox = work.getbbox()
if bbox is None: # fully transparent
return _empty_result(mode, img.size)
work = work.crop(bbox)
src_w, src_h = work.size
if mode == "stretch":
out = work.resize((box_w, box_h), resample)
return ImageFitResult(out, box_w, box_h, box_w / src_w, mode, (src_w, src_h))
if mode == "cover":
scale = max(box_w / src_w, box_h / src_h)
if not upscale:
scale = min(scale, 1.0)
scaled_w = max(1, round(src_w * scale))
scaled_h = max(1, round(src_h * scale))
out = work.resize((scaled_w, scaled_h), resample)
# Crop the overhang down to the box (only when the scaled image is
# larger; with upscale=False it may be smaller and is left as-is).
crop_w, crop_h = min(box_w, scaled_w), min(box_h, scaled_h)
left = (scaled_w - crop_w) // 2
top = 0 if anchor == "top" else (scaled_h - crop_h) // 2
out = out.crop((left, top, left + crop_w, top + crop_h))
return ImageFitResult(out, out.width, out.height, scale, mode, (src_w, src_h))
# contain / fill_height share the "preserve aspect, no crop" path
if mode == "fill_height":
scale = box_h / src_h
# contain-cap: never exceed the box width (football's logo_slot rule)
scale = min(scale, box_w / src_w)
else: # contain
scale = min(box_w / src_w, box_h / src_h)
if not upscale:
scale = min(scale, 1.0)
out_w = max(1, round(src_w * scale))
out_h = max(1, round(src_h * scale))
if (out_w, out_h) == (src_w, src_h):
# No resize needed — but `work` may still BE the caller's original
# image (RGBA source, no ink crop). The result must always be an
# independent copy: LayoutContext caches ImageFitResults, and an
# aliased image would let later mutations of the source corrupt
# cached fits (or vice versa).
out = work.copy() if work is img else work
else:
out = work.resize((out_w, out_h), resample)
return ImageFitResult(out, out_w, out_h, scale, mode, (src_w, src_h))
def draw_fitted_image(display_manager: Any, ifit: ImageFitResult, box: Any, *,
align: str = "center", valign: str = "center",
offset: Tuple[int, int] = (0, 0)) -> Optional[Tuple[int, int]]:
"""Paste a fitted image aligned within a Region onto the display canvas.
Pastes with the image's own alpha mask. Returns the (x, y) actually used
so callers can position adjacent decorations, or None when nothing was
drawn (empty fit / no canvas).
"""
if ifit is None or ifit.is_empty:
return None
image = getattr(display_manager, "image", None)
if image is None:
return None
if hasattr(box, "align_xy"):
x, y = box.align_xy(ifit.width, ifit.height, align, valign)
else:
box_w, box_h = _box_dims(box)
x = (box_w - ifit.width) // 2
y = (box_h - ifit.height) // 2
x += int(offset[0])
y += int(offset[1])
image.paste(ifit.image, (x, y), ifit.image)
return (x, y)
+746
View File
@@ -0,0 +1,746 @@
"""
Adaptive layout and font scaling helpers for plugins.
Generalizes the three size-adaptation patterns proven in the plugin
ecosystem into small composable core helpers, so plugins render legibly on
any panel size (64x32, 128x32, 96x48, 128x64, 256x64, ...) without
hand-tuned per-display layouts:
- Region: integer rect algebra (bands, columns, weighted splits, centering).
Regions partition space, so text bands can't overlap by construction —
replacing the magic ``y = 1`` / ``y = height - 7`` offsets tuned for 128x32.
- Font ladders: ordered (family, size) steps known to render crisply.
Pixel fonts (BDF, PressStart2P) only look right at native/integer sizes,
so fonts are never scaled continuously fitting walks a ladder from the
largest rung down until the measured text fits the target box. This is
baseball-scoreboard's fallback-ladder pattern promoted to core.
- LayoutContext: per-(width, height) facts breakpoint tiers
(masters-tournament's pattern), a geometry scale factor vs. a declared
design size (f1-scoreboard's pattern), and cached fit-text queries.
Everything is opt-in: plugins get a context via ``self.layout`` on
BasePlugin (or construct one directly) and existing plugins are unaffected.
Fonts are resolved through FontManager's catalog (family names are
lowercased file stems from assets/fonts, e.g. "9x15", "tom-thumb", plus
aliases like "press_start"). FitResult.font is a plain PIL font or
freetype.Face, so it drops straight into DisplayManager.draw_text().
"""
import logging
from collections import OrderedDict
from dataclasses import dataclass
from typing import Any, Dict, List, Optional, Sequence, Tuple, Union
import freetype
logger = logging.getLogger(__name__)
# Height-based breakpoint tiers, smallest to largest. A 32px-tall panel is
# the ecosystem baseline ("sm"); 96x48 lands in "md"; 128x64 in "lg".
_HEIGHT_TIERS: Tuple[Tuple[str, int], ...] = (
("xs", 16), ("sm", 32), ("md", 48), ("lg", 64), ("xl", 10 ** 9),
)
TIER_ORDER: Tuple[str, ...] = tuple(name for name, _ in _HEIGHT_TIERS)
_WIDTH_TIERS: Tuple[Tuple[str, int], ...] = (
("narrow", 64), ("normal", 128), ("wide", 256), ("ultrawide", 10 ** 9),
)
WIDTH_TIER_ORDER: Tuple[str, ...] = tuple(name for name, _ in _WIDTH_TIERS)
# The panel size most existing plugins were authored against.
DEFAULT_DESIGN_SIZE: Tuple[int, int] = (128, 32)
@dataclass(frozen=True)
class Region:
"""An integer rectangle. Carving methods return sub-Regions clamped to
non-negative dimensions, so degenerate panels never produce negative
boxes a band request larger than the region simply consumes it all."""
x: int
y: int
w: int
h: int
def __post_init__(self):
object.__setattr__(self, "w", max(0, int(self.w)))
object.__setattr__(self, "h", max(0, int(self.h)))
object.__setattr__(self, "x", int(self.x))
object.__setattr__(self, "y", int(self.y))
@property
def right(self) -> int:
return self.x + self.w
@property
def bottom(self) -> int:
return self.y + self.h
@property
def center(self) -> Tuple[int, int]:
return (self.x + self.w // 2, self.y + self.h // 2)
# ---- carving -----------------------------------------------------
def inset(self, dx: int, dy: Optional[int] = None) -> "Region":
"""Shrink by dx horizontally and dy (default dx) vertically, each side."""
if dy is None:
dy = dx
return Region(self.x + dx, self.y + dy, self.w - 2 * dx, self.h - 2 * dy)
def offset(self, dx: int, dy: int) -> "Region":
"""Translate without resizing — the hook for user x/y-offset
customization: compute regions first, then apply the user's
configured offsets as a final translation."""
return Region(self.x + dx, self.y + dy, self.w, self.h)
def top_band(self, h: int) -> "Region":
return Region(self.x, self.y, self.w, min(h, self.h))
def bottom_band(self, h: int) -> "Region":
h = min(h, self.h)
return Region(self.x, self.bottom - h, self.w, h)
def middle(self, top_h: int = 0, bottom_h: int = 0) -> "Region":
"""What remains between a top band and a bottom band."""
return Region(self.x, self.y + top_h, self.w, self.h - top_h - bottom_h)
def left_col(self, w: int) -> "Region":
return Region(self.x, self.y, min(w, self.w), self.h)
def right_col(self, w: int) -> "Region":
w = min(w, self.w)
return Region(self.right - w, self.y, w, self.h)
def split_h(self, *weights: float, gap: int = 0) -> List["Region"]:
"""Side-by-side columns sized by weight; gaps between them."""
sizes = _weighted_sizes(self.w, weights, gap)
cols, cursor = [], self.x
for size in sizes:
cols.append(Region(cursor, self.y, size, self.h))
cursor += size + gap
return cols
def split_v(self, *weights: float, gap: int = 0) -> List["Region"]:
"""Stacked rows sized by weight; gaps between them."""
sizes = _weighted_sizes(self.h, weights, gap)
rows, cursor = [], self.y
for size in sizes:
rows.append(Region(self.x, cursor, self.w, size))
cursor += size + gap
return rows
# ---- placement ---------------------------------------------------
def align_xy(self, w: int, h: int, align: str = "center",
valign: str = "center") -> Tuple[int, int]:
"""Top-left position for a w x h box aligned within this region.
align: left|center|right; valign: top|center|bottom."""
if align == "left":
x = self.x
elif align == "right":
x = self.right - w
else:
x = self.x + (self.w - w) // 2
if valign == "top":
y = self.y
elif valign == "bottom":
y = self.bottom - h
else:
y = self.y + (self.h - h) // 2
return (x, y)
def center_xy(self, w: int, h: int) -> Tuple[int, int]:
return self.align_xy(w, h)
def contains(self, w: int, h: int) -> bool:
return w <= self.w and h <= self.h
def _weighted_sizes(total: int, weights: Sequence[float], gap: int) -> List[int]:
"""Integer sizes proportional to weights, remainder spread left-to-right."""
if not weights:
return []
usable = max(0, total - gap * (len(weights) - 1))
weight_sum = sum(weights) or 1
sizes = [int(usable * w / weight_sum) for w in weights]
remainder = usable - sum(sizes)
for i in range(remainder):
sizes[i % len(sizes)] += 1
return sizes
# ---------------------------------------------------------------------------
# Font ladders
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class FontStep:
"""One rung: a FontManager catalog family at a size it renders crisply."""
family: str
size_px: int
FontLadder = Tuple[FontStep, ...]
# X11 BDF bitmap fonts at their native pixel sizes, largest to smallest —
# baseball-scoreboard's fallback ladder extended upward. Same-height rungs
# are ordered widest first so width-constrained text steps to a narrower
# face before dropping a size.
LADDER_GRID: FontLadder = (
FontStep("10x20", 20),
FontStep("9x18", 18),
FontStep("9x15", 15),
FontStep("8x13", 13),
FontStep("7x13", 13),
FontStep("6x13", 13),
FontStep("6x12", 12),
FontStep("6x10", 10),
FontStep("6x9", 9),
FontStep("5x8", 8),
FontStep("5x7", 7),
FontStep("4x6", 6),
FontStep("tom-thumb", 6),
)
# PressStart2P at integer multiples of its 8px pixel grid only — fractional
# sizes blur a pixel font. For headline text (clocks, scores).
LADDER_ARCADE: FontLadder = (
FontStep("press_start", 32),
FontStep("press_start", 24),
FontStep("press_start", 16),
FontStep("press_start", 8),
)
LADDER_DEFAULT: FontLadder = LADDER_GRID
ELLIPSIS = ""
@dataclass(frozen=True)
class FitResult:
"""A fitted font plus the ink metrics of the (possibly ellipsized) text.
``y_offset`` is the gap between the y passed to draw_text() and where
ink actually starts; subtract it from the desired ink-top position when
drawing (draw_fitted_text does this for you).
"""
font: Any
family: str
size_px: int
text: str
width: int
height: int
baseline: int
y_offset: int
fits: bool
line_height: int = 0
def measure_ink(text: str, font: Any) -> Tuple[int, int, int, int]:
"""Measure the ink box of text: (width, height, baseline, y_offset).
y_offset is the distance from the y coordinate DisplayManager.draw_text()
is given to the top of the actual ink PIL draws TTF from the em-box
top and _draw_bdf_text derives the baseline from y + ascender, so both
leave a font-dependent gap that matters when centering in short bands.
"""
if isinstance(font, freetype.Face):
width = 0
ascender = font.size.ascender >> 6
ink_top, ink_bottom = None, None
for char in text:
font.load_char(char)
width += font.glyph.advance.x >> 6
rows = font.glyph.bitmap.rows
if rows:
top = ascender - font.glyph.bitmap_top
ink_top = top if ink_top is None else min(ink_top, top)
ink_bottom = top + rows if ink_bottom is None else max(ink_bottom, top + rows)
if ink_top is None:
ink_top, ink_bottom = 0, 0
return (width, ink_bottom - ink_top, ascender, ink_top)
bbox = font.getbbox(text)
return (bbox[2] - bbox[0], bbox[3] - bbox[1], -bbox[1], bbox[1])
def font_line_height(font: Any) -> int:
"""Recommended line spacing for a font (matches DisplayManager.get_font_height)."""
if isinstance(font, freetype.Face):
return font.size.height >> 6
ascent, descent = font.getmetrics()
return ascent + descent
def measure_font_crispness(font: Any, sample_text: str = "Ay0",
canvas_size: Tuple[int, int] = (250, 60)) -> float:
"""Fraction of the rendered sample's ink-bbox pixels that are neither
pure black nor pure white i.e. antialiased.
BDF (freetype.Face) glyphs are true bitmaps and always render at 0.0.
"Pixel-style" TTFs (PressStart2P, and similar fonts bundled for
plugins that draw through ImageDraw.text() and so can't take a BDF
face) are NOT automatically crisp at arbitrary sizes PIL antialiases
TTF outlines by default, and a pixel-grid font only lands on whole
pixels at specific sizes (for PressStart2P: exact multiples of 8).
Requesting an unverified size silently produces soft/blurry glyphs on
an LED panel, which reads as fuzzy compared to a true BDF rung.
Use this to vet any custom FontLadder rung that mixes TTF fonts before
shipping it see test_adaptive_layout.py::test_ladder_is_crisp for the
pattern. A rung should score 0.0 (or very close, to allow for the odd
diagonal stroke) before it belongs in a "crisp" ladder.
"""
if isinstance(font, freetype.Face):
return 0.0
from PIL import Image, ImageDraw
img = Image.new("L", canvas_size, 0)
ImageDraw.Draw(img).text((2, 2), sample_text, font=font, fill=255)
bbox = img.getbbox()
if bbox is None:
return 0.0
pixels = img.crop(bbox).tobytes()
pure = sum(1 for p in pixels if p == 0 or p == 255)
return (len(pixels) - pure) / len(pixels)
class LayoutContext:
"""Per-render-size layout facts and fit-text queries for one panel size.
Construct once per (width, height); BasePlugin.layout does this and
rebuilds automatically when the logical display size changes.
"""
def __init__(self, width: int, height: int, font_manager: Any,
design_size: Tuple[int, int] = DEFAULT_DESIGN_SIZE):
self.width = int(width)
self.height = int(height)
self.font_manager = font_manager
self.design_size = design_size
self.bounds = Region(0, 0, self.width, self.height)
self.aspect = self.width / max(1, self.height)
self.tier = _pick_tier(_HEIGHT_TIERS, self.height)
self.width_tier = _pick_tier(_WIDTH_TIERS, self.width)
self.is_wide_short = self.aspect >= 2.5 and self.height <= 32
design_w, design_h = design_size
# Geometry scale only (gaps, icon/logo sizes) — never applied to
# fonts, which step between crisp ladder rungs instead.
self.scale = min(self.width / max(1, design_w),
self.height / max(1, design_h))
# LRU-bounded: entries are small, but keys embed the fitted TEXT —
# a plugin fitting changing text (a live game clock, a ticker) on a
# 24/7 service would otherwise grow this without bound.
self._fit_cache: "OrderedDict[Any, FitResult]" = OrderedDict()
# LRU-bounded (images are big). Entries hold a strong reference to
# the source image when keyed by id() so the id can't be recycled
# out from under the cache.
self._image_cache: "OrderedDict[Any, Tuple[Any, Any]]" = OrderedDict()
_IMAGE_CACHE_MAX = 64
_FIT_CACHE_MAX = 512
def _fit_cache_get(self, key: Any) -> Optional["FitResult"]:
cached = self._fit_cache.get(key)
if cached is not None:
self._fit_cache.move_to_end(key)
return cached
def _fit_cache_put(self, key: Any, result: "FitResult") -> None:
self._fit_cache[key] = result
while len(self._fit_cache) > self._FIT_CACHE_MAX:
self._fit_cache.popitem(last=False)
# ---- the three adaptation patterns --------------------------------
def px(self, base: int, minimum: int = 1, maximum: Optional[int] = None) -> int:
"""Scale a design-size pixel measurement (f1's pattern): gaps,
icon sizes, logo slots. Clamped to [minimum, maximum]."""
value = max(minimum, round(base * self.scale))
if maximum is not None:
value = min(value, maximum)
return value
def by_tier(self, mapping: Dict[str, Any], default: Any = None) -> Any:
"""Pick the value for the nearest defined tier at-or-below the
panel's height tier (masters' pattern). Falls forward to the
smallest defined tier above, then to default.
by_tier({"sm": 10, "lg": 18}) -> 10 on 128x32, 18 on 128x64.
Keys may also use width tiers ("narrow", "wide", ...)."""
order = TIER_ORDER if any(k in TIER_ORDER for k in mapping) else WIDTH_TIER_ORDER
current = self.tier if order is TIER_ORDER else self.width_tier
idx = order.index(current)
for name in reversed(order[: idx + 1]):
if name in mapping:
return mapping[name]
for name in order[idx + 1:]:
if name in mapping:
return mapping[name]
return default
def fit_text(self, text: str, box: Union[Region, Tuple[int, int]],
ladder: FontLadder = LADDER_DEFAULT,
ellipsis: bool = True) -> FitResult:
"""Largest ladder rung whose rendered text fits the box (baseball's
pattern). If even the smallest rung is too wide, the text is
ellipsized to fit (unless ellipsis=False); fits=False only when no
acceptable rendering exists."""
box_w, box_h = _box_dims(box)
key = ("text", text, box_w, box_h, ladder, ellipsis)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
result = self._walk_ladder(text, ladder, box_w, box_h, ellipsis)
self._fit_cache_put(key, result)
return result
def fit_text_proportional(self, text: str, box: Union[Region, Tuple[int, int]],
base_size_px: int, ladder: FontLadder = LADDER_DEFAULT,
ellipsis: bool = True,
scale: Optional[float] = None) -> FitResult:
"""Ladder rung closest to (but not exceeding) ``base_size_px * scale``
that still fits the box proportional sizing instead of ``fit_text``'s
"always maximize" behavior.
Use this when several independently-fitted elements need to stay
visually harmonious as the panel grows (e.g. a scoreboard's score,
status, and detail text) ``fit_text`` maximizes each one within
its own region, which can make one element balloon out of
proportion to its neighbors (a huge score overlapping logos it fit
fine at the design size) even though every individual pick is
independently "correct". ``base_size_px`` is the size that element
renders at on the design size (``design_size``, typically 128x32)
commonly a plugin's existing classic/fixed font size for that
element.
``scale`` defaults to ``self.scale`` (the same conservative
min(width_ratio, height_ratio) factor ``px()`` uses safe for
content whose aspect ratio matters). Pass an explicit axis-specific
value when the surrounding composition already scales that way
e.g. a scoreboard whose logos scale with height alone
(``logo_slot = min(height, width // 2)``) should size its score
text by ``height / design_height`` too, or its text will look
under-scaled next to bigger logos on a panel that only grew taller.
Falls back to the smallest rung when even that exceeds the target
(a tiny scale factor), and to fit_text's ordinary smaller-rung
fallback when the closest-to-target rung doesn't actually fit the
box.
"""
box_w, box_h = _box_dims(box)
effective_scale = self.scale if scale is None else scale
key = ("text_prop", text, box_w, box_h, ladder, base_size_px, ellipsis, effective_scale)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
target = base_size_px * effective_scale
eligible = [step for step in ladder if step.size_px <= target]
candidates = eligible if eligible else (min(ladder, key=lambda s: s.size_px),)
result = self._walk_ladder(text, candidates, box_w, box_h, ellipsis)
self._fit_cache_put(key, result)
return result
def _walk_ladder(self, text: str, ladder: Sequence[FontStep],
box_w: int, box_h: int, ellipsis: bool) -> FitResult:
"""Shared by fit_text/fit_text_proportional: first ladder entry (in
the order given) whose rendered text fits, ellipsizing the last one
tried if none do."""
result = None
for step in ladder:
font = self.font_manager.get_font(step.family, step.size_px)
width, height, baseline, y_offset = measure_ink(text, font)
result = FitResult(font, step.family, step.size_px, text,
width, height, baseline, y_offset,
fits=(width <= box_w and height <= box_h),
line_height=font_line_height(font))
if result.fits:
break
if result is not None and not result.fits and ellipsis:
short = self.ellipsize(text, result.font, box_w)
width, height, baseline, y_offset = measure_ink(short, result.font)
result = FitResult(result.font, result.family, result.size_px,
short, width, height, baseline, y_offset,
fits=(width <= box_w and height <= box_h),
line_height=result.line_height)
return result
def fit_lines(self, lines: Sequence[str], box: Union[Region, Tuple[int, int]],
ladder: FontLadder = LADDER_DEFAULT,
spacing: int = 1) -> FitResult:
"""Largest rung where every line fits the box width and the stacked
lines (line_height + spacing apart) fit the box height. Measures the
actual strings, so a long line pushes the ladder down a rung a short
one wouldn't (baseball's multiline pattern). Text is the widest line."""
box_w, box_h = _box_dims(box)
key = ("lines", tuple(lines), box_w, box_h, ladder, spacing)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
rows = max(1, len(lines))
result = None
for step in ladder:
font = self.font_manager.get_font(step.family, step.size_px)
line_h = font_line_height(font)
widest, metrics = "", (0, 0, 0, 0)
for line in lines:
m = measure_ink(line, font)
if m[0] >= metrics[0]:
widest, metrics = line, m
total_h = rows * line_h + (rows - 1) * spacing
result = FitResult(font, step.family, step.size_px, widest,
metrics[0], metrics[1], metrics[2], metrics[3],
fits=(metrics[0] <= box_w and total_h <= box_h),
line_height=line_h)
if result.fits:
break
self._fit_cache_put(key, result)
return result
def font_for_rows(self, rows: int, box_h: int,
ladder: FontLadder = LADDER_GRID) -> FitResult:
"""Largest rung whose line height lets `rows` rows fit in box_h
(baseball's traditional-scoreboard pattern). Measures a digit/cap
sample rather than specific strings."""
key = ("rows", rows, box_h, ladder)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
sample = "0Ay"
result = None
for step in ladder:
font = self.font_manager.get_font(step.family, step.size_px)
line_h = font_line_height(font)
width, height, baseline, y_offset = measure_ink(sample, font)
result = FitResult(font, step.family, step.size_px, sample,
width, height, baseline, y_offset,
fits=(max(1, rows) * line_h <= box_h),
line_height=line_h)
if result.fits:
break
self._fit_cache_put(key, result)
return result
# ---- images ---------------------------------------------------------
def fit_image(self, img: Any, box: Union[Region, Tuple[int, int]], *,
mode: str = "contain", crop_to_ink: bool = False,
anchor: str = "center", resample: Any = None,
upscale: bool = True, cache_key: Any = None) -> Any:
"""Fit an image into a box (see src/adaptive_images.py for modes),
cached per (image, box size, options) for this panel size.
Prefer a stable ``cache_key`` (e.g. "logo:KC") for images that get
reloaded the default id()-based key is safe (the entry pins the
source image) but misses across reloads of the same content.
"""
from src.adaptive_images import fit_image as _fit_image
box_w, box_h = _box_dims(box)
resample_name = getattr(resample, "name", repr(resample)) if resample is not None else "default"
identity = cache_key if cache_key is not None else ("id", id(img))
key = ("image", identity, img.size, box_w, box_h, mode,
crop_to_ink, anchor, resample_name, upscale)
cached = self._image_cache.get(key)
if cached is not None:
self._image_cache.move_to_end(key)
return cached[0]
result = _fit_image(img, (box_w, box_h), mode=mode,
crop_to_ink=crop_to_ink, anchor=anchor,
resample=resample, upscale=upscale)
# Pin the source only for id()-keyed entries (see docstring).
self._image_cache[key] = (result, img if cache_key is None else None)
while len(self._image_cache) > self._IMAGE_CACHE_MAX:
self._image_cache.popitem(last=False)
return result
# ---- text utilities ------------------------------------------------
def ellipsize(self, text: str, font: Any, max_w: int) -> str:
"""Trim text to fit max_w, appending an ellipsis. Returns '' when
not even the ellipsis fits."""
if measure_ink(text, font)[0] <= max_w:
return text
for end in range(len(text) - 1, 0, -1):
candidate = text[:end].rstrip() + ELLIPSIS
if measure_ink(candidate, font)[0] <= max_w:
return candidate
return ELLIPSIS if measure_ink(ELLIPSIS, font)[0] <= max_w else ""
def measure(self, text: str, font: Any) -> Tuple[int, int, int]:
"""Ink (width, height, baseline) of text — see measure_ink."""
width, height, baseline, _ = measure_ink(text, font)
return (width, height, baseline)
def clear_cache(self) -> None:
"""Drop cached fit results (call after fonts are reloaded)."""
self._fit_cache.clear()
self._image_cache.clear()
def _pick_tier(tiers: Tuple[Tuple[str, int], ...], value: int) -> str:
for name, limit in tiers:
if value <= limit:
return name
return tiers[-1][0]
def _box_dims(box: Union[Region, Tuple[int, int]]) -> Tuple[int, int]:
if isinstance(box, Region):
return (box.w, box.h)
w, h = box
return (int(w), int(h))
def draw_fitted_text(display_manager: Any, fit: FitResult,
box: Union[Region, Tuple[int, int]],
color: Tuple[int, int, int] = (255, 255, 255),
align: str = "center", valign: str = "center") -> None:
"""Draw a FitResult's text aligned within a Region via
DisplayManager.draw_text(), compensating for the font's ink offset so
the ink (not the em box) is what gets aligned."""
region = box if isinstance(box, Region) else Region(0, 0, box[0], box[1])
x, y = region.align_xy(fit.width, fit.height, align, valign)
display_manager.draw_text(fit.text, x=x, y=y - fit.y_offset,
color=color, font=fit.font)
# ---------------------------------------------------------------------------
# Composite layouts — the region arrangements repeated across plugins,
# expressed as Region math so migrated plugins stop hand-copying coordinate
# formulas. Deliberately tiny: these return Regions, they don't draw.
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class ScoreboardRegions:
"""The two-logos-plus-center-score card shared by the sports plugins."""
bounds: Region
logo_slot: int # width of each logo slot: min(H, W // 2), center-reserved
away_slot: Region # left logo slot
home_slot: Region # right logo slot
center_col: Region # column between the slots (>= min_center_fraction of width)
status_band: Region # top band (replaces the magic y = 1)
score_area: Region # center_col's true width, between the bands (replaces y = H//2 - 3)
detail_band: Region # bottom band (replaces the magic y = H - 7)
bottom_left: Region # bottom corner: away records / timeouts
bottom_right: Region # bottom corner: home records / timeouts
def scoreboard_regions(bounds: Region, *, ctx: Optional["LayoutContext"] = None,
status_h: Optional[int] = None,
detail_h: Optional[int] = None,
min_center_fraction: float = 0.15,
min_center_design_px: int = 40,
score_bleed_fraction: float = 0.5) -> ScoreboardRegions:
"""Carve a game-card Region into the standard scoreboard arrangement.
Encodes the invariant duplicated across the sports plugins:
``logo_slot = min(height, width // 2)`` (capped at half the card so the
home slot never collapses), away logo centered in the left slot, home in
the right.
That formula alone has a blind spot: at exactly 2:1 aspect ratio
(width == 2 * height a very common shape, e.g. two, four, or more
square modules stacked into a taller panel) ``width // 2`` and
``height`` are equal, so the two logo slots claim the *entire* width
and leave zero pixels for a center column, no matter how large the
panel gets. It isn't a "small panel" problem: 96x48, 128x64, and
256x128 (all exactly 2:1) hit it identically, while wide panels like
the 128x32 design baseline or a 192x48/256x32 panel never do, because
height is already the tighter constraint there.
Two knobs fix it, both defaulted to values verified against the full
harness size spread (see test_adaptive_layout.py::TestScoreboardRegions):
- ``min_center_fraction`` / ``min_center_design_px`` reserve at least
``max(width * min_center_fraction, min_center_design_px * ctx.scale)``
for the center column, capping ``logo_slot`` further when needed. The
design-px term (scaled by the context's geometry factor, so it grows
on bigger panels like everything else in ``px()``) matters most on
small panels where a flat fraction alone reserves too little absolute
space for even a short score string. On wide panels the height
constraint already leaves more room than either reserves, so both are
a no-op there 128x32/192x48-style layouts are unaffected.
- ``score_bleed_fraction`` extends the score's own *fit box* (not the
logo slots themselves) an extra ``logo_slot * score_bleed_fraction``
into each side controlled, intentional overlap with the logo art,
the same way real broadcast scoreboards let a big score number's
edges cross into the team marks flanking it. Without this, on a
square-ish panel the center reserve alone can be too narrow for even
a modest score to render without truncating (`"17-21"` -> `"17-2…"`),
which is worse than a little overlap.
status_band and detail_band span the FULL card width and overlay the
logo slots matching the classic layouts, where short outlined status/
date text is drawn over the logos without issue; only score_area (the
one element whose size actively grows with the panel) uses the
narrower, bleed-adjusted box. Band heights default to the classic
128x32 values, scaled by the context's geometry factor when one is
provided. Works on a full panel or on a scroll-mode card Region.
"""
if status_h is None:
status_h = ctx.px(9, minimum=7) if ctx else 9
if detail_h is None:
detail_h = ctx.px(8, minimum=7) if ctx else 8
logo_slot = min(bounds.h, bounds.w // 2)
design_reserve = int(min_center_design_px * (ctx.scale if ctx else 1.0))
min_center_w = max(1, int(bounds.w * min_center_fraction), design_reserve)
max_logo_slot_by_center = max(1, (bounds.w - min_center_w) // 2)
logo_slot = min(logo_slot, max_logo_slot_by_center)
away_slot = bounds.left_col(logo_slot)
home_slot = bounds.right_col(logo_slot)
center_col = Region(bounds.x + logo_slot, bounds.y,
bounds.w - 2 * logo_slot, bounds.h)
status_band = bounds.top_band(status_h)
detail_band = bounds.bottom_band(detail_h)
middle = bounds.middle(status_band.h, detail_band.h)
# score_area is the true center gap's width plus a controlled bleed
# into each logo slot (see score_bleed_fraction above) -- narrower than
# the full card width status/detail get, since it's the one element
# whose size actively grows with the panel and needs its *fit box* to
# reflect real available space, but generous enough that a short score
# string never has to truncate on a square-ish panel.
bleed = int(logo_slot * score_bleed_fraction)
score_area = Region(center_col.x - bleed, middle.y,
center_col.w + 2 * bleed, middle.h)
bottom = bounds.bottom_band(detail_h)
return ScoreboardRegions(
bounds=bounds, logo_slot=logo_slot,
away_slot=away_slot, home_slot=home_slot, center_col=center_col,
status_band=status_band, score_area=score_area, detail_band=detail_band,
bottom_left=bottom.left_col(logo_slot),
bottom_right=bottom.right_col(logo_slot),
)
@dataclass(frozen=True)
class MediaRow:
"""Art/icon on the left, text column on the right (music's idiom)."""
art: Region
body: Region
def media_row(bounds: Region, *, ctx: Optional["LayoutContext"] = None,
square: bool = True, gap: Optional[int] = None) -> MediaRow:
"""Split a Region into an art slot and a body column.
With ``square=True`` the art slot is bounds.h wide (album-art style);
otherwise it takes the left half. The gap defaults to 2px scaled by the
context's geometry factor.
"""
if gap is None:
gap = ctx.px(2, minimum=1) if ctx else 2
art_w = bounds.h if square else bounds.w // 2
art_w = min(art_w, bounds.w)
art = bounds.left_col(art_w)
body = Region(bounds.x + art_w + gap, bounds.y,
bounds.w - art_w - gap, bounds.h)
return MediaRow(art=art, body=body)
-3
View File
@@ -7,10 +7,7 @@ where Recent/Upcoming managers consume data from the background service cache.
"""
import time
import logging
from typing import Dict, Optional, Any, Callable
from datetime import datetime
import pytz
class BackgroundCacheMixin:
+7 -22
View File
@@ -14,19 +14,15 @@ Key Features:
- Memory-efficient data storage
"""
import os
import time
import logging
import threading
import requests
from typing import Dict, Any, Optional, List, Callable, Union
from datetime import datetime, timedelta
from typing import Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
import json
import queue
from concurrent.futures import ThreadPoolExecutor, Future
import weakref
from concurrent.futures import ThreadPoolExecutor
from src.cache_manager import CacheManager
# Configure logging
logger = logging.getLogger(__name__)
@@ -227,7 +223,7 @@ class BackgroundDataService:
self.stats['cache_misses'] += 1
# Submit to executor
future = self.executor.submit(self._fetch_data_worker, request)
self.executor.submit(self._fetch_data_worker, request)
logger.info(f"Submitted background fetch request {request_id} for {sport} {year}")
return request_id
@@ -553,13 +549,12 @@ class BackgroundDataService:
if to_remove:
logger.info(f"Cleared {len(to_remove)} old completed requests")
def shutdown(self, wait: bool = True, timeout: int = 30):
def shutdown(self, wait: bool = True):
"""
Shutdown the background data service.
Args:
wait: Whether to wait for active requests to complete
timeout: Maximum time to wait for shutdown
"""
logger.info("Shutting down BackgroundDataService...")
@@ -570,24 +565,14 @@ class BackgroundDataService:
for request_id in list(self.active_requests.keys()):
self.cancel_request(request_id)
# Shutdown executor with compatibility for older Python versions
try:
# Try with timeout parameter (Python 3.9+)
self.executor.shutdown(wait=wait, timeout=timeout)
except TypeError:
# Fallback for older Python versions that don't support timeout
if wait and timeout:
# For older versions, we can't specify timeout, so just wait
self.executor.shutdown(wait=True)
else:
self.executor.shutdown(wait=wait)
self.executor.shutdown(wait=wait)
logger.info("BackgroundDataService shutdown complete")
def __del__(self):
"""Cleanup when service is destroyed."""
if not self._shutdown:
self.shutdown(wait=False, timeout=None)
self.shutdown(wait=False)
# Global service instance
_background_service: Optional[BackgroundDataService] = None
+4 -4
View File
@@ -410,8 +410,8 @@ def validate_backup(zip_path: Path) -> Tuple[bool, str, Dict[str, Any]]:
try:
manifest_raw = zf.read(MANIFEST_NAME).decode("utf-8")
manifest = json.loads(manifest_raw)
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as e:
return False, f"Invalid manifest.json: {e}", {}
except (OSError, UnicodeDecodeError, json.JSONDecodeError):
return False, "Invalid manifest.json", {}
if not isinstance(manifest, dict) or "schema_version" not in manifest:
return False, "Invalid manifest structure", {}
@@ -456,8 +456,8 @@ def validate_backup(zip_path: Path) -> Tuple[bool, str, Dict[str, Any]]:
return True, "", result_manifest
except zipfile.BadZipFile:
return False, "File is not a valid ZIP archive", {}
except OSError as e:
return False, f"Could not read backup: {e}", {}
except OSError:
return False, "Could not read backup", {}
# ---------------------------------------------------------------------------
+1 -3
View File
@@ -7,7 +7,7 @@ fields and data structures.
"""
from abc import ABC, abstractmethod
from typing import Dict, Any, Optional, List
from typing import Dict, Optional
import logging
from datetime import datetime
import pytz
@@ -21,12 +21,10 @@ class APIDataExtractor(ABC):
@abstractmethod
def extract_game_details(self, game_event: Dict) -> Optional[Dict]:
"""Extract common game details from raw API data."""
pass
@abstractmethod
def get_sport_specific_fields(self, game_event: Dict) -> Dict:
"""Extract sport-specific fields (downs, innings, periods, etc.)."""
pass
def _extract_common_details(self, game_event: Dict) -> tuple[Dict | None, Dict | None, Dict | None, Dict | None, Dict | None]:
"""Extract common game details that work across all sports."""
-1
View File
@@ -329,7 +329,6 @@ class Baseball(SportsCore):
return
series_summary = game.get("series_summary", "")
font = self.fonts.get('detail', ImageFont.load_default())
bbox = draw_overlay.textbbox((0, 0), series_summary, font=self.fonts['time'])
height = bbox[3] - bbox[1]
shots_y = (self.display_height - height) // 2
-2
View File
@@ -1,6 +1,4 @@
import logging
import time
from datetime import datetime, timezone
from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont
+4 -8
View File
@@ -6,11 +6,10 @@ to support different APIs and data providers.
"""
from abc import ABC, abstractmethod
from typing import Dict, Any, Optional, List
from typing import Dict, List
import requests
import logging
from datetime import datetime, timedelta
import time
from datetime import datetime
class DataSource(ABC):
"""Abstract base class for data sources."""
@@ -35,17 +34,14 @@ class DataSource(ABC):
@abstractmethod
def fetch_live_games(self, sport: str, league: str) -> List[Dict]:
"""Fetch live games for a sport/league."""
pass
@abstractmethod
def fetch_schedule(self, sport: str, league: str, date_range: tuple) -> List[Dict]:
"""Fetch schedule for a sport/league within date range."""
pass
@abstractmethod
def fetch_standings(self, sport: str, league: str) -> Dict:
"""Fetch standings for a sport/league."""
pass
def get_headers(self) -> Dict[str, str]:
"""Get headers for API requests."""
@@ -217,7 +213,7 @@ class MLBAPIDataSource(DataSource):
response.raise_for_status()
data = response.json()
self.logger.debug(f"Fetched standings from MLB API")
self.logger.debug("Fetched standings from MLB API")
return data
except Exception as e:
@@ -296,7 +292,7 @@ class SoccerAPIDataSource(DataSource):
response.raise_for_status()
data = response.json()
self.logger.debug(f"Fetched standings from soccer API")
self.logger.debug("Fetched standings from soccer API")
return data
except Exception as e:
+1 -3
View File
@@ -1,10 +1,8 @@
from typing import Dict, Any, Optional, List
from typing import Dict, Any, Optional
from src.display_manager import DisplayManager
from src.cache_manager import CacheManager
from datetime import datetime, timezone, timedelta
import logging
from PIL import Image, ImageDraw, ImageFont
import time
from src.base_classes.data_sources import ESPNDataSource
from src.base_classes.sports import SportsCore, SportsLive
-4
View File
@@ -1,6 +1,4 @@
import logging
import time
from datetime import datetime, timezone
from typing import Any, Dict, Optional
from PIL import Image, ImageDraw, ImageFont
@@ -79,8 +77,6 @@ class Hockey(SportsCore):
away_shots = round(home_team_saves / home_team_saves_per)
if away_team_saves_per > 0:
home_shots = round(away_team_saves / away_team_saves_per)
status_short = status["type"].get("shortDetail", "")
if situation and status["type"]["state"] == "in":
# Detect scoring events from status detail
# status_detail = status["type"].get("detail", "")
+3 -4
View File
@@ -5,7 +5,7 @@ import time
from abc import ABC, abstractmethod
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
from typing import Any, Dict, List, Optional
import pytz
import requests
@@ -172,8 +172,8 @@ class SportsCore(ABC):
try:
fallbacks.append(Path.home() / ".ledmatrix" / "logos" / self.sport_key)
except Exception:
pass
except RuntimeError as e:
self.logger.debug("Could not resolve home directory (expected for service users): %s", e)
fallbacks.append(Path(tempfile.gettempdir()) / "ledmatrix_logos" / self.sport_key)
@@ -416,7 +416,6 @@ class SportsCore(ABC):
league=self.league,
event_id=game['id'],
update_interval_seconds=update_interval,
is_live=is_live
)
if odds_data:
-3
View File
@@ -11,13 +11,10 @@ Follows LEDMatrix configuration management patterns:
- Maintainable: Changes to odds logic affect all plugins
"""
import time
import logging
import requests
import json
from datetime import datetime, timedelta, timezone
from typing import Dict, Any, Optional, List
import pytz
class BaseOddsManager:

Some files were not shown because too many files have changed in this diff Show More