Compare commits

...
Author SHA1 Message Date
Claude b67f9e4a2a fix(web): treat only 2xx as a successful display save
Review feedback on #422.

- showDisplaySaveResult tested `xhr.status >= 400` for failure, so a
  network error — which reports status 0 — was waved through as
  "Display settings saved". That's the same class of false-success bug
  this branch set out to fix. Test the 2xx range instead, and let a
  response body refine a successful verdict without overturning a
  failed one.
- Annotate the _copies_fits_hardware helper, matching the annotated
  helpers already in api_v3.py.
- Cover the vertical divisibility branch: chain_length 2 would divide
  evenly, so only parallel 3 can produce the rejection, which pins the
  branch to the right hardware dimension.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016aUvzTXpEYhNEWYQvFqQU9
2026-07-28 20:23:35 +00:00
Claude b7f5f8483a fix(web): don't validate double-sided settings when the feature is disabled
Saving anything on the Display tab failed with a 400 when double-sided
mode was off:

    Double-sided copies (2) must divide chain length (3) evenly

The Display form posts every field in one request, including
double_sided_copies (default 2) and double_sided_axis, whether or not
the Enabled checkbox is ticked. The server block was gated only on "is
any double-sided field present in the payload" — it wrote
ds_config['enabled'] but never read it. A user with chain_length: 3 and
the untouched default copies: 2 was locked out of saving any display
setting at all: brightness, GPIO slowdown, Vegas, sync.

Gate the checks on the enabled flag:

- Divisibility against chain_length/parallel is hardware-relational and
  only runs when the feature is on.
- Structural checks (copies parses as an int in 2..8, axis in the
  whitelist) still 400 when enabled; when disabled they drop the value
  and leave the stored one untouched rather than rejecting the save.

The runtime already gated correctly (_resolve_double_sided returns None
when disabled), so nothing there changes.

Also in the Display tab:

- Hide Copies / Split Axis until Enabled is ticked, mirroring the Vegas
  Scroll pattern. Hidden rather than disabled, so the fields keep
  submitting and the server still sees an 'off' state to persist.
- Fix the save toast: the form's handler read xhr.responseJSON, a jQuery
  property that doesn't exist on a native XMLHttpRequest, so it was
  always undefined and every save reported a green "Display settings
  saved" — even the 400s. Parse responseText and use the real status.

Tests: two existing double-sided tests asserted 200 on payloads that the
divisibility check (added later, in #373) turns into 400s; the first now
supplies matching hardware values and the second passes as written now
that a disabled save skips the check. Added coverage for the reported
regression, for bad values while disabled, and for the check still
firing when enabled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016aUvzTXpEYhNEWYQvFqQU9
2026-07-28 20:17:29 +00:00
3872a68ff7 Surface plugin update availability on the plugins page (#421)
* Surface plugin update availability on the plugins page

The plugin manager page already showed each installed plugin's version and
had an Update button, but nothing told users an update actually existed —
they had to guess. The store manager already compares the installed
manifest version against the registry's latest_version for its reinstall
decision; this surfaces that same signal in the UI.

- api_v3 /plugins/installed now returns `latest_version` (from the registry
  cache, no extra network call) and an `update_available` flag computed by a
  new semver-aware helper `_is_plugin_update_available`. A locally modified
  plugin whose version is ahead of the registry is not flagged.
- The installed-plugin card shows "vX.Y.Z available" next to the installed
  version, and the Update button becomes emphasized ("Update to vX.Y.Z" with
  a gentle pulse) when a newer version is published — mirroring the app's own
  update banner styling.
- Added tests for the helper and the endpoint fields.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EW8yDbsk8EceDpq6Hjgqj

* Catch InvalidVersion specifically in plugin update comparison

Address review feedback: the version comparison caught a blind `except
Exception` (ruff BLE001). Split the two failure modes and catch each
specifically — ImportError for a missing `packaging` (a core dependency)
and InvalidVersion for an unparseable version string — while preserving
the existing "surface the mismatch" fallback behavior. Added a test for
the unparseable-version path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EW8yDbsk8EceDpq6Hjgqj

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 08:06:36 -04:00
989162d28f fix(web): custom-feed logo upload uses the wrong request/response contract (#420)
Every custom-feed logo upload has been failing: handleCustomFeedLogoUpload
posts the file under field name "file" and reads the response from
data.data.files, but the backend endpoint it calls
(api_v3.upload_plugin_asset, /api/v3/plugins/assets/upload) requires the
field name "files" (checks 'files' not in request.files, 400s "No files
provided" otherwise) and returns the result in a top-level "uploaded_files"
key - there is no nested "data" wrapper in the response at all. Confirmed
by reading the endpoint directly, and cross-checked against
file-upload-single.js, a sibling widget that uses the correct contract
against the same endpoint.

- formData.append('file', file) -> formData.append('files', file)
- data.data.files / data.data.files[0] -> data.uploaded_files /
  data.uploaded_files[0]

No other call sites in this file used the stale contract (grepped for both
patterns after the fix - zero remaining). The response entries' 'path' and
'id' fields (both read further down in the same handler) are unaffected -
only the wrapper shape was wrong.

Found incidentally while re-verifying a CodeRabbit review on an unrelated
PR (#417) that had deleted a differently-named dead file
(custom-feeds-helpers.js) with the same bug; this widget (custom-feeds.js)
is the live code path and was never touched by that PR.

Validation: brace/paren balance check; explicit assertions that the old
field name and response shape no longer appear anywhere in the file. No
Python changed, no existing tests cover this endpoint's client flow.


Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 11:07:36 -04:00
cdf03fb107 Add skin system: user-installable visual overlays for sports scoreboards (#419)
* Add skin system: user-installable visual overlays for sports scoreboards

Skins restyle a scoreboard's live/recent/upcoming rendering while the
host plugin keeps doing data fetching, scheduling, caching, live
priority, and vegas mode — the anti-fork alternative for users who only
want a different layout.

- src/skin_system/: ScoreboardSkin API, SkinContext (canvas + adaptive
  layout + logo/font helpers), discovery/loading runtime with API major
  version gating and per-skin module namespacing
- src/base_classes/sports.py: _render_game() seam at the three
  _draw_scorebug_layout call sites; skin-first with built-in fallback,
  3-strikes session disable, slow-render warning; per-mode skin config
- scripts/validate_skin.py: headless multi-mode/multi-size validator
  with bundled per-sport fixtures (no hardware or network needed)
- skins/example-classic-baseball/: working reference skin
- Web UI: served-schema Visual Skin dropdown (validation never
  enum-restricted, so uninstalled skins can't invalidate configs) and
  GET /api/v3/skins
- Store: registry entries with type "skin" install to skins/
- docs/SKIN_SYSTEM.md (architecture), docs/CREATING_SKINS.md (author
  guide incl. Claude Code prompt), view-model contract locked by tests

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1

* Address review feedback on skin system

- skin_runtime: cache the entry module so the 2nd/3rd load of the same
  skin (live/recent/upcoming hosts) doesn't re-execute it with unbound
  sibling aliases; rebind cached sibling modules to their bare names
  around entry import and restore prior bindings after; include per-
  manifest mtimes in the discovery cache fingerprint so in-place skin
  updates are picked up
- sports.py: count render_skin_card exceptions toward the 3-strike
  session disable
- store_manager: validate skin ids (pattern + resolved-path containment
  in skins/), reject registry/manifest id mismatches, and stage+validate
  downloads in a temp sibling before replacing an existing skin
- schema_manager: leave the schema untouched when the configured skin
  value is a per-mode mapping (a string dropdown could overwrite it)
- validate_skin.py: reject non-positive sizes and non-object --options
  at parse time; support --output-dir outside the repo; type annotations
- example skin: validate accent_color once at load with logged fallback
- fixtures: pregame 0-0 scores in football/hockey upcoming fixtures
- api /skins: rely on the self-invalidating discovery cache instead of
  force_refresh
- docs: valid JSON manifest example, load_logo caching semantics spelled
  out, language ids on fenced blocks
- tests: view-model contract test now exercises the real extractor;
  regression test for repeated same-skin loads with sibling modules

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 11:07:08 -04:00
6a9d8014e5 Tag plugin logs structurally and surface the active plugin in System Logs (#418)
- get_logger() now returns a PluginLoggerAdapter when given a plugin_id,
  so every plugin log call is stamped with plugin_id automatically instead
  of only calls that explicitly passed extra={'plugin_id': ...}. This makes
  the "[Plugin: x]" prefix reliable in the journalctl-backed log stream.
- display_controller publishes the currently active mode/plugin to the
  shared cache whenever it changes, exposed via a new
  GET /api/v3/display/current-status endpoint.
- System Logs page: adds a "Now showing" banner backed by that endpoint, a
  plugin filter dropdown (populated from parsed log lines), a plugin badge
  per log entry, and fixes log parsing to handle the short-iso timestamp
  format journalctl actually returns (the old regex only matched syslog
  timestamps, so level/plugin extraction silently never ran).

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-18 10:59:36 -04:00
c90129285c Web UI: mobile navigation, guided onboarding, basic/advanced config tiering + performance (#417)
* chore(web): remove dead legacy client-side plugin-config generator (~2,300 lines)

Plugin config forms have been rendered server-side (plugin_config.html via
GET /partials/plugin-config/<id>) since the HTMX migration; the old
client-side generator survived as unreachable code. Verified dead by call
graph, not by naming: showPluginConfigModal and showGithubTokenInstructions
have zero callers anywhere in templates or JS, and everything removed here
is reachable only from those two roots.

Removed:
- plugins_manager.js: showPluginConfigModal, generatePluginConfigForm,
  generateFormFromSchema, generateFieldHtml, generateSimpleConfigForm,
  handlePluginConfigSubmit, the modal's JSON-editor view (initJsonEditor,
  switchPluginConfigView, syncFormToJson/JsonToForm, saveConfigFromJsonEditor,
  resetPluginConfigToDefaults, displayValidationErrors, closePluginConfigModal,
  savePluginConfiguration, currentPluginConfigState), their exclusive helpers
  (getSchemaPropertyType, escapeCssSelector, dotToNested, collectBooleanFields,
  normalizeFormDataForConfig, flattenConfig, loadCustomHtmlWidget), the
  orphaned-modal cleanup block, the modal's listener wiring, and the
  never-invoked showGithubTokenInstructions/closeInstructionsModal pair.
- plugins.html: the #plugin-config-modal markup those functions drove.
- base.html: the deprecated pluginConfigData() component and the
  window.PluginConfigHelpers shim (only ever called by pluginConfigData).

Deliberately kept, verified still live:
- renderArrayObjectItem, getSchemaProperty, escapeHtml/escapeAttribute
  (window-exposed for the top-level array-of-objects handlers the
  server-rendered form uses), toggleNestedSection, addKeyValuePair/
  addArrayObjectItem families, executePluginAction, and
  window.currentPluginConfig = null init (file-upload.js and
  executePluginAction read it, optional-chained).
- app()'s internal generateConfigForm/generateSimpleConfigForm methods in
  base.html: unreachable now but embedded in the live Alpine component;
  excising methods from a live object is deferred to keep this change
  zero-risk.

Validation: every deletion seam inspected line-by-line; Jinja parse of both
templates passes; repo-wide sweep confirms zero remaining references to any
deleted function or element id (deleted ranges contained no Jinja tags).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): mobile navigation drawer + responsive CSS gap fixes

Phones previously got the desktop layout squeezed: ~12 system tabs plus one
tab per installed plugin wrapped into many rows of small pill buttons, and
the header's settings-search and system-stats widgets were dropped entirely
(hidden below their breakpoints, never relocated).

- Off-canvas nav drawer below md: the existing nav markup (system tab row +
  #plugin-tabs-row, including dynamically injected plugin tabs) is wrapped in
  a #site-nav container that CSS repositions into a slide-in drawer on small
  screens. Same DOM nodes, same @click handlers, nothing duplicated. Tabs
  become full-width rows with 44px+ touch targets. A hamburger button
  (md:hidden) in the header and a backdrop toggle the new mobileNavOpen
  Alpine state (added to both app() definitions, mirroring activeTab).
  Clicking any tab, a search result, or the backdrop closes the drawer.
  At md+ hard CSS guards make all drawer styles inert - desktop renders
  exactly as before.
- Header widgets relocated, not hidden: placeHeaderWidgets() in app.js moves
  the #settings-search-wrap and #system-stats nodes (same elements, listeners
  intact - both are looked up by id from SSE/search code, so they must never
  be duplicated) into the drawer below md and back into the header above it,
  via a matchMedia listener.
- Fixed 13 breakpoint utility classes that templates referenced but app.css
  never defined (sm:block, sm:grid-cols-2, sm:text-sm, md:block, md:w-auto,
  lg:block, lg:flex, lg:w-64, xl:grid-cols-2/3, 2xl:grid-cols-2/3/4). This
  was a live bug: 'hidden sm:block' on the search box and 'hidden lg:flex'
  on the stats meant BOTH were invisible at every screen width. Audit method
  (repeatable): diff classes used in templates vs defined in app.css.
- Mobile modal sizing: one global rule caps .modal-content at 95vw/90vh with
  internal scroll below 640px - covers every modal without per-template
  changes.
- Horizontal-scroll affordance: pure-CSS edge-fade shadows on
  .overflow-x-auto containers (scrolling-shadows technique), plus larger
  in-table touch targets below md.

Validation: breakpoint used-vs-defined audit now returns zero gaps; Jinja
parse of base.html passes; all changes to desktop behavior are additive
(new utilities) or scoped inside max-width media queries.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): x-advanced schema flag groups plugin config fields under a collapsed Advanced Settings section

Plugin config pages show every schema property at equal visual priority,
which overwhelms first-time users. Plugin authors can now add
"x-advanced": true to any flat (non-object) property in config_schema.json
to move it into one collapsed "Advanced Settings (N)" section rendered after
the basic fields - progressive disclosure with zero loss of control.

Implementation: the main render loop in plugin_config.html splits ordered
properties into basic/advanced tiers; the advanced group reuses the exact
.nested-section/.nested-content/toggleSection() shell that nested object
sections already use, so the settings search's expand-on-match behavior
works on advanced fields with no JS changes. Object-type properties ignore
the flag (they already render as their own collapsible sections). No
backend change needed: jsonschema ignores unknown x-* keywords exactly as
it does for x-widget/x-propertyOrder.

Documented in docs/widget-guide.md alongside the other x-* extensions.

Validation (rendered with real Jinja, not just parsed):
- synthetic schema with 2 advanced fields: basic fields render before the
  section, advanced inside the collapsed shell, count badge correct,
  x-advanced on an object property correctly ignored
- schema without any x-advanced: output is identical to the pre-change
  template (whitespace-normalized diff against git HEAD's version)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Display Settings basic/advanced split + live total-resolution readout

The Hardware Configuration card showed ~17 fields at equal priority; a new
user only needs 7 of them to get a correctly-sized, correctly-colored image
(rows, cols, chain_length, parallel, brightness, hardware_mapping,
led_rgb_sequence). The other 10 (multiplexing, panel_type, row_address_type,
gpio_slowdown, rp1_rio, scan_mode, pwm_bits, pwm_dither_bits,
pwm_lsb_nanoseconds, limit_refresh_rate_hz) now live in a collapsed
"Advanced Hardware Settings" section using the same nested-section shell as
plugin config forms, so toggleSection() and settings-search auto-expand work
unchanged. led_rgb_sequence moved up beside brightness/hardware_mapping
(2-col grid became 3-col). No field was removed or renamed; the form still
posts the same names to /api/v3/config/main.

Also adds a live "Your display: W x H pixels" readout under the four sizing
fields (width = cols x chain_length, height = rows x parallel - the exact
math the chain-length tooltip describes in prose), recomputed client-side on
every input event, no round-trip.

Deviation from plan, deliberate: disable_hardware_pulsing / inverse_colors /
show_refresh_rate stay in their separate "Display Options" card rather than
moving across cards - relocating fields between form sections risks
regressions for no decluttering gain in the card users complained about.

Validation (real Jinja render): all 17 hardware fields present exactly once,
basic fields render before the advanced section and the 10 advanced fields
inside it, div count balanced (71/71), readout + recompute script present.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): plugin install auto-enables + persistent restart nudge

Getting a plugin onto the display used to take three disconnected manual
steps: install from the store, flip its enable toggle, then restart the
display service - with no in-UI hint that steps 2 and 3 were needed (only
docs/GETTING_STARTED.md mentions it).

- installPlugin() now enables the plugin immediately on successful install
  (owner-confirmed behavior change: always auto-enable, no opt-out; users
  who don't want it running toggle it off as before), then shows a
  persistent toast ("... restart the display to show it") with an inline
  "Restart Now" button wired to the existing restartDisplay() - the same
  function the three existing Restart Display buttons call.
- notification.js: show() accepts optional { actionLabel, onAction } to
  render one inline action button per toast. Callbacks are stored per
  notification id and cleaned up on dismiss; a new triggerAction() public
  method runs the callback and dismisses. The global showNotification()
  shorthand now forwards a full options object as its second argument
  (legacy type-string calls unchanged).

Scope note: applies to the plugin store's install path (window.installPlugin).
The custom-registry install path keeps its existing behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): dismissible Getting Started checklist on Overview

New users land on a dense multi-tab dashboard with no suggested order of
operations (the only guided flow is the WiFi captive portal). This adds a
non-gating checklist card at the top of Overview with five steps, each a
deep link that switches to the right tab (and closes the mobile nav drawer):

1. Set panel size            -> Display tab   (done: rows/cols/chain_length > 0)
2. Set timezone/location     -> General tab   (done: differs from template
                                               defaults America/New_York / Tampa)
3. Install a plugin          -> Plugins tab   (done: /api/v3/plugins/installed
                                               non-empty)
4. Enable a plugin           -> Plugins tab   (done: any installed plugin enabled)
5. Configure it              -> Plugins tab   (done: first enabled plugin has >=1
                                               saved value differing from its
                                               schema defaults)

Steps 1-2 are computed server-side in Jinja from main_config (already in the
partial's context); 3-5 client-side from existing endpoints. No new backend
state: dismissal persists in localStorage (mirroring the reconciliation
banner's sessionStorage pattern one section up); deep links use the same
_x_dataStack app-data access as settings-search.js. Disclosed heuristic
limit: values left at legitimate defaults (a user actually in Tampa) read
as "not done".

Validation: real Jinja render across 3 config variants confirms the
server-side done-flags flip correctly; div balance intact; /plugins/config
response shape (config dict directly in .data) verified against api_v3.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(display): drag-and-drop plugin rotation order for the primary display mode

The primary rotation's order was invisible and unconfigurable: modes are
registered in parallel-load COMPLETION order, so rotation order actually
varied between restarts. Only the niche Vegas Scroll mode had a working
order UI. This adds real, persisted ordering end to end:

Backend:
- config.template.json: new display.plugin_rotation_order (default [],
  fully backward compatible).
- display_controller.py: _apply_plugin_rotation_order() rebuilds
  available_modes grouped by plugin per the configured list (each plugin's
  modes keep their declared order; unlisted plugins follow in existing
  relative order; empty config = exact no-op). Applied at startup after
  parallel load and after live enable/disable reconcile (before the
  existing _resync_mode_index_after_change, which preserves the current
  mode). Mirrors vegas_mode get_ordered_plugins() semantics.
- api_v3.py save_main_config: accepts plugin_rotation_order as a JSON
  array (same parse/guard pattern as vegas_plugin_order).

Frontend:
- New shared widget static/v3/js/widgets/plugin-order-list.js: the Vegas
  section's drag-and-drop list factored out verbatim (native HTML5 drag
  events, saved-order-first rendering, hidden-input JSON sync),
  parameterized by container/order-input/optional exclude-checkbox/badge.
- display.html: Vegas section now calls the shared module; its ~130-line
  inline copy of the same logic is deleted.
- durations.html: new "Rotation Order" card above the durations grid using
  the same module, posting plugin_rotation_order with the existing form.

Deviation from plan, deliberate: durations stay as their own mode-keyed
grid rather than inline in the drag rows - verified display_durations keys
are MODE names (display_controller.py resolves duration per mode_key), not
plugin ids, and one plugin can own several modes, so the planned 1:1
inline pairing was wrong.

Validation: py_compile on both Python files; _apply_plugin_rotation_order
unit-tested standalone (configured order applied, empty-config no-op,
unknown ids skipped - 3/3); both templates render with balanced divs, the
hidden input carries the saved order, and the old inline implementation is
confirmed gone; config.template.json parses.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): serve the interface at / — /v3 kept as a legacy alias

The user-visible URL no longer carries the interface version: the pages
blueprint is now registered un-prefixed (primary) AND at /v3 (second
registration, name='pages_v3_legacy'), so:

- http://<device>/ serves the interface directly (the old @app.route('/')
  redirect is removed — the blueprint's own index takes its place)
- every existing /v3/... bookmark and all the hardcoded /v3/partials/...
  fetches in templates/JS keep working verbatim through the alias mount —
  zero template/JS churn, zero broken links
- url_for('pages_v3.*') resolves against the primary registration, so all
  server-side redirects (captive portal detection endpoints) now emit
  un-prefixed URLs
- the AP-mode captive-portal allowlist learned the un-prefixed page paths
  (/setup, /partials/, /settings/, /plugin-ui/) so setup-mode requests
  don't redirect-loop
- /api/v3 and the templates/v3, static/v3 directories are deliberately
  untouched (internal, invisible to users; owner-confirmed scope)

Validation: dual registration mechanics tested against real Flask (test
client): /, /v3, /v3/ redirect, partials and /setup reachable on both
mounts, url_for yields un-prefixed paths; py_compile passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): stream the preview PNG raw instead of PIL decode + re-encode

display_preview_generator() opened each changed snapshot with PIL and
re-encoded it to PNG just to base64 it — but /tmp/led_matrix_preview.png
already IS a PNG, written atomically by the display service (tmp file +
os.replace in display_manager.py), so a partially-written file can never be
observed. Read the bytes and base64 them directly: identical payload
(front-end consumes data:image/png;base64 — verified in base.html), one
full image decode+encode per frame less on the same Pi that's driving the
matrix. The existing mtime skip and viewer-marker throttling are unchanged
(they already covered the "skip unchanged frames" concern).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gzip response compression via flask-compress

The interface ships a ~5,000-line HTML shell and >20k lines of JS
uncompressed; on phone/WiFi that dominates load time. Flask-Compress
gzips/brotlis compressible responses transparently.

- Optional dependency, same graceful pattern as flask-limiter: missing
  package = uncompressed responses, no crash.
- SSE safety verified empirically against the real package (1.24): an
  actual streamed text/event-stream response comes back with no
  Content-Encoding while a large HTML response gzips — the display
  preview / stats / logs streams are unaffected.
- Added flask-compress>=1.14 to web_interface/requirements.txt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): gate verbose console logging behind the existing pluginDebug switch

plugins_manager.js, base.html's inline scripts, and app.js emitted 198
console.log calls in production - including per-interaction [DEBUG] dumps -
costing main-thread time and drowning real errors in noise.

- New window.debugLog() gate defined in base.html's first inline script
  (before any other script runs): forwards to console.log only when
  localStorage.pluginDebug === 'true' - the SAME switch plugins_manager.js
  already used for its _PLUGIN_DEBUG_EARLY logs, so existing debug workflow
  docs stay valid. Exposed as window.LEDMATRIX_DEBUG for other scripts.
- Mechanically rewrote console.log( -> debugLog( in plugins_manager.js
  (127), base.html (64), app.js (7). Verified no occurrences lived inside
  string literals before rewriting; console.error/console.warn untouched.
- app.js's no-Alpine showNotification fallback restored to console.info -
  it's a user-facing last resort, not debug output.

Both load paths are safe: the gate is the first inline <script> in <head>,
and every rewritten file loads deferred after it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): vendor CDN assets locally — LAN-speed loads, fully offline-capable

Font Awesome, CodeMirror, and htmx were fetched from cdnjs/unpkg on every
fresh page load, adding third-party round-trips on a device that often
lives on a local network (and htmx was local-only in AP mode, meaning two
different loading behaviors to reason about).

- Vendored pinned copies under static/v3/vendor/: Font Awesome 6.0.0
  (css/all.min.css + the 8 webfonts it references relatively) and
  CodeMirror 5.65.2 (core, javascript mode, closebrackets/matchbrackets
  addons, base + monokai css) - ~1.1 MB total, exact versions the CDN tags
  pinned.
- htmx + sse + json-enc extensions now load from the existing local copies
  (verified 1.9.10, matching the CDN pin) on EVERY network, not just AP
  mode; the pinned CDN copies remain as a one-shot rescue fallback,
  mirroring the pattern Alpine already used. The convoluted isAPMode
  source-flipping logic collapses away.
- Dropped the CDN preconnect/dns-prefetch hints (no longer on the critical
  path).
- Fixed a latent bug while relinking CodeMirror: the loader requested
  mode/json/json.min.js, which does not exist on cdnjs (HTTP 404 verified)
  - it 404'd on every JSON-editor open. JSON highlighting comes from the
  javascript mode; the phantom entry is removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* perf(web): extract 3,850 lines of inline JS from base.html to cacheable static files

base.html shipped ~4,200 lines of inline JavaScript inside the HTML
document, re-downloaded and re-parsed on every page load (gzip helps the
transfer, but inline scripts can never be browser-cached). The four
largest blocks - none containing any Jinja syntax, verified by scanning
every inline block for {{ }} / {% %} - now live as static files served
with the app's existing mtime-versioned immutable caching:

- js/htmx-config.js (246 lines): HTMX swap/script-execution config,
  toggleSection helpers
- js/app-early.js (346 lines): early helpers + the app() stub that must
  precede Alpine init
- js/app-shell.js (2,997 lines): SSE wiring + the full Alpine app()
  implementation and tab logic
- js/custom-feeds-helpers.js (262 lines): custom-feeds table helpers

Each replacement <script src> is CLASSIC (no defer/async) at the exact
position of the inline block it replaces - identical execution timing and
DOM visibility to inline scripts, so relative ordering with the deferred
scripts and with each other is unchanged. base.html drops from ~4,940 to
1,079 lines.

Validation: extraction proven lossless by programmatically reassembling
the four files back into the template and comparing against git HEAD -
byte-for-byte identical. Jinja parse passes; script open/close tags
balanced (53/53, after excluding a literal "<script>" inside an HTML
comment).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): size the live preview from the PNG's real dimensions, not config

The initial SSE render sized the preview image (and both overlay canvases)
from the server-reported config dimensions (cols x chain_length,
rows x parallel), while the scale slider's re-render path sized from
img.naturalWidth/naturalHeight. Whenever the snapshot PNG's actual size
disagrees with the config (stale config, display service not restarted
after a hardware change), the initial render stretched the image at a
fractional ratio - blurry despite image-rendering: pixelated - and
touching the scale slider "fixed" it. Reported live on the devpi test rig.

Both paths now size from the loaded image's natural dimensions inside
img.onload (which also removes a transient wrong-size flash between
src assignment and load). The meta label now reports the true snapshot
size. The preview card also gets overflow-x-auto so on narrow screens a
wide preview scrolls at its exact pixel-perfect size instead of being
squeezed into the viewport (fractional downscaling of pixel art also
reads as blur).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): fold Display Options into the advanced dropdown; Vegas above Double-Sided

Owner-requested layout refinement of the Display Settings tab:

- The "Display Options" card (disable_hardware_pulsing, inverse_colors,
  show_refresh_rate, use_short_date_format, Dynamic Duration) moves inside
  the collapsed advanced section, now titled "Advanced Hardware & Display
  Options (15)". Hidden form fields still submit with the form, and
  settings search still auto-expands the section on match, so nothing is
  lost - the tab just leads with the essentials.
- The "Vegas Scroll Mode" section moves above "Double-Sided Display".
  New section order: Hardware (+ advanced dropdown) > Vegas Scroll >
  Double-Sided > Multi-Display Sync.

Validation (real Jinja render): all 23 field names present exactly once,
divs balanced (70/70), the five Display Options fields render inside the
advanced section's bounds, and section markers confirm the new order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): Getting Started items are manually checkable; card auto-hides when complete

Two gaps reported from live testing on the devpi rig:

1. The timezone/location step never showed done for a user whose real
   timezone IS the shipped default (America/New_York) - the heuristic can
   only detect difference-from-default, not "user saved this". Clicking an
   item's checkbox now toggles it done manually (persisted per browser in
   localStorage), so any heuristic false-negative is one tap to clear.
   Clicking the item text still deep-links to its tab.
2. The card now hides itself automatically once every step is done
   (auto-detected or manually checked) - previously it stayed until the X
   was clicked.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): CI cleanup — declare debugLog global, fix entity-unescape order, drop v3 from UI branding

- Add debugLog to the /* global */ headers of the six JS files that call it
  (defined in base.html's first inline script) — resolves the wall of
  "'debugLog' is not defined" ESLint errors failing the Codacy check.
- Fix the two js/double-escaping CodeQL alerts in app-shell.js: the
  entity-unescape chains decoded &amp; before &lt;/&gt;, so a value
  containing a pre-escaped "&amp;lt;" wrongly double-decoded to "<".
  &amp; now decodes last (standard order). Pre-existing bug, made visible
  when the inline scripts moved into scannable .js files.
- Page title / header drop the "- v3" suffix, matching the de-versioned
  user-facing URL.

The remaining 7 CodeQL alerts are pre-existing patterns newly visible to
scanning (CodeQL doesn't see inline template JS): 4 github.com/htmx.org
URL-substring checks (the htmx ones match error-message text, not URLs —
false positives in context) and 1 innerHTML XSS-through-DOM in the GitHub
install flow. Triage/fix deferred to a focused follow-up rather than
expanding this PR.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): add the missing nav tab for the Rotation & Durations page

The durations partial (/partials/durations) has existed as a route with no
nav tab and no content panel referencing it - an orphaned page. That made
the new rotation-order UI unreachable through the interface (caught by the
owner testing on the rig; my endpoint-level tests fetched the partial by
URL and never noticed the missing entry point).

- New "Rotation" tab (fa-rotate icon, verified present in the vendored
  FA 6.0.0 css) between Display and Backup & Restore, wired exactly like
  the other tabs (#durations-content + hx-get + loadtab; loadTabContent()
  is fully generic, so no JS changes needed).
- Page heading updated from "Display Durations" to "Rotation & Durations"
  to match its content since the rotation-order card landed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): Rotation & Durations page lists every enabled plugin's screens

The durations grid looped over display.display_durations, which nothing has
ever populated (verified {} on a real production install) - so the page
rendered no duration fields at all. Worse, its inputs posted bare mode
names, which save_main_config's endswith('_duration') filter silently
dropped: the page was broken in both directions, unnoticed because it was
also unreachable (previous commit).

- pages_v3._load_durations_partial now builds one entry per display mode of
  every ENABLED plugin via plugin_manager.get_plugin_display_modes()
  (falling back to the plugin id), overlaid with saved values, defaulting
  to the display controller's 30s. Grouped per plugin, sorted by name.
  Saved keys not owned by any enabled plugin stay visible under "Other
  saved entries" instead of vanishing.
- durations.html renders the grouped inputs, named duration__<mode_key>
  (mode keys are arbitrary, so they can't use the *_duration suffix
  convention), with an explanatory empty state when no plugins are enabled.
- api_v3.save_main_config accepts the new duration__<mode> fields and
  writes them into display.display_durations under the bare mode key -
  exactly what the display controller reads
  (display_durations.get(mode_key, 30)).

Validation: py_compile both blueprints; Jinja render with 3 groups asserts
grouped inputs, saved-value overlay, stale-entry group, empty state, and
div balance.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): restart-pending banner, unsaved-changes guard, installable web app

Three usability improvements from live testing feedback:

- Restart-pending banner: every successful POST to /api/v3/config/main
  (display hardware, rotation/durations, general settings) now raises a
  persistent banner - "Configuration saved, restart the display to apply" -
  with a Restart Now button that posts restart_display_service directly.
  Backed by sessionStorage so it survives tab switches and reloads until
  restarted or dismissed. Plugin config saves are deliberately excluded:
  they apply live via the display process's config watcher.
- Unsaved-changes guard: plugin config panels are Alpine x-if templates,
  so navigating away destroys the panel and revisiting re-fetches it -
  edits were silently discarded. Forms now mark themselves dirty on input
  (cleared on successful submit), a capture-phase click handler confirms
  before a lossy tab switch, and beforeunload guards full page unloads.
  System tabs (x-show, persistent DOM) are exempt - no false prompts.
- Installable web app: manifest.json (standalone display, dark theme) +
  generated LED-grid icons (192/512 maskable + 180 apple-touch), linked
  from base.html. "Add to Home Screen" now yields an app-like fullscreen
  experience; no service worker, so zero behavioral risk.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* test(web): smoke tests + static-analysis audits for the web UI

Guardrails so this branch's fix classes can't regress silently:

- test_web_smoke.py (24 tests): boots the pages blueprint with the same
  dual registration app.py uses and asserts every page/partial returns 200
  with its load-bearing markers (nav wiring, getting-started card, advanced
  section, rotation order card, per-mode duration inputs), the /v3 legacy
  alias serves everything, all critical static assets (incl. vendored
  fontawesome/codemirror, PWA manifest/icons) are served, durations group
  per plugin with the leftover bucket, and the advanced-hardware section
  really contains the tuning fields. Would have caught this session's
  unreachable-durations-page and orphaned-tab bugs instantly.
- test_web_static_audit.py (3 tests): (1) every responsive utility class
  referenced in templates is actually defined in app.css - the
  silently-no-op class bug that left the header search box invisible at
  every width; (2) every url_for('static', ...) reference points to a real
  file; (3) any JS file calling the debugLog global declares it in a
  /* global */ header.

All 40 web tests pass (24 + 3 new, 13 existing) under pytest + Flask.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): floating live preview + per-plugin "Preview on display", drawer a11y

Preview-while-configuring:
- Floating mini preview (fixed, bottom-right) available on every tab except
  Overview, fed from the same SSE display stream by updateDisplayPreview -
  no new connections. Collapses to a round toggle button; open/closed state
  persists in localStorage; hides on Overview where the full preview lives.
- "Preview on display" button on every plugin config page header: runs that
  plugin on the real display for 60 seconds via the existing
  /display/on-demand/start API and opens the floating preview, closing the
  configure -> see-the-result loop.

Drawer/nav accessibility:
- aria-current="page" tracks the active tab (system + dynamic plugin tabs,
  matched via their Alpine @click expression), updated from the activeTab
  watcher so search deep-links and checklist navigation are covered too.
- Escape closes the mobile drawer and returns focus to the hamburger;
  opening the drawer moves focus to its first tab.

Validation: all 40 web tests pass; Jinja parse + div balance on both
touched templates.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): resolve Codacy findings — DOM building over innerHTML, Map callbacks, misc lint

Verified each of the 27 reported findings against current code; all fixed
except one rule class skipped with reason below.

- app-early.js: plugin tab buttons are now built with createElement/
  createTextNode instead of innerHTML template strings (icon class and name
  come from plugin manifests - semi-trusted input; the old code escaped the
  name but interpolated the icon class). Both construction sites. Also the
  forEach arrow no longer returns tab.remove()'s value.
- plugin-order-list.js: rows, empty state, and error state all built with
  DOM APIs - the file no longer contains innerHTML at all (the now-unneeded
  escapeHtml/escapeAttr helpers are removed); MODE_LABELS is a Map so the
  vegas-mode lookup can't hit prototype properties.
- notification.js: actionCallbacks is a Map (get/set/delete) instead of a
  plain object - resolves the object-injection-sink and dynamic-delete
  findings; triggerAction also type-checks the callback.
- htmx-config.js: unused catch binding dropped; var -> const in the
  afterSettle handler; the swapped-<script> re-execution reads/writes
  textContent instead of innerHTML; the diagnostic form payload uses a
  null-prototype object so a field named __proto__ can't pollute.
- custom-feeds-helpers.js DELETED (with its script tag): all three of its
  functions (addCustomFeedRow, removeCustomFeedRow,
  handleCustomFeedLogoUpload) are shadowed by the deferred
  widgets/custom-feeds.js window assignments, which always win at call time
  - the copies were dead even when they lived inline in base.html. This
  also resolves the unused-function and unused-variable findings there.

Skipped: 4x "Non-serializable expression must be wrapped with $(...)" in
app-early.js - that rule targets code crossing a browser-automation
serialization boundary (e.g. page.evaluate); these are ordinary arrow
functions in plain browser code with no such boundary.

Validation: all 40 web tests pass (incl. the static-asset reference audit,
which confirms no template still points at the deleted file); Jinja parse OK.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web): update flow installs changed Python dependencies automatically

The in-app updater (Update Now banner -> git_pull action) stashed, pulled,
and purged plugins - but never touched Python dependencies. Any release
adding a package (e.g. this branch's flask-compress, which lives in
web_interface/requirements.txt) silently required an SSH session and a
manual pip install that most users will never do.

- git_pull now records HEAD before pulling; after a successful pull it
  diffs old..new and, if requirements.txt or web_interface/requirements.txt
  changed, installs exactly those via _pip_install_requirements - the same
  vetted root-visible sudo path the Tools-tab buttons use (with its
  existing graceful fallback when the sudo wrapper isn't configured).
  Results are appended to the update toast; a failure points the user at
  the Tools-tab button instead of failing the whole update.
- install_base_requirements (Tools tab) now also installs
  web_interface/requirements.txt - previously it only covered the root
  file, so web-only dependencies were unreachable from the UI entirely.

No install happens when the pull was already-up-to-date or when no
requirements file changed, so routine updates stay fast.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): address CodeRabbit review — validation, a11y, perf, and privacy fixes

Verified each finding against current code. Fixed:

- api_v3: plugin_rotation_order is now strictly validated (JSON list of
  strings, 400 with a descriptive message otherwise) and popped from the
  payload before any further handling.
- display_controller: _apply_plugin_rotation_order defensively ignores a
  non-list value (keeps the existing rotation, logs a warning) and drops
  non-string entries; new logs carry the [DisplayController] prefix.
  Unit-tested both defensive paths.
- app.py: snapshot-read handler narrowed to OSError with debug logging;
  flask-compress ImportError now emits one structured warning with the
  install remedy.
- htmx-config: the response-error logger prints form FIELD NAMES only -
  values (API keys, passwords) never reach the console.
- plugin-order-list: saved order/exclusions normalized with Array.isArray
  (a saved "null" previously crashed .forEach); each row gained
  keyboard/touch-accessible move-up/move-down buttons (HTML5 drag events
  don't fire on most mobile browsers) that reorder and syncInputs()
  immediately alongside native drag.
- app-shell: window.installedPlugins setter always takes the new list
  (same-ID metadata/enabled updates were silently dropped); tab rebuild
  stays gated on ID changes. LED dot renderer reads the frame with ONE
  getImageData call instead of one per pixel (~9,200/frame at 192x48).
- plugins_manager: togglePlugin returns its request promise resolving the
  API outcome; the install flow now shows the "installed and enabled"
  toast (with Restart Now) only after enablement succeeds, and a warning
  without a restart offer when it fails.
- a11y: hamburger aria-label flips Open/Close with drawer state; both
  Advanced-section toggle buttons declare aria-controls/aria-expanded and
  the shared toggleSection() keeps aria-expanded in sync; move buttons
  have per-plugin aria-labels.
- Rotation/Vegas order-list bootstraps cap their retries (~5s) and show a
  reload hint instead of spinning forever; Alpine app-state lookups prefer
  [x-data="app()"] with a generic fallback.

Skipped, with reasons:
- executePluginAction arg order: caller (plugin_config.html) already
  passes (actionId, index, pluginId) matching the signature exactly.
- generateFieldHtml XSS, entity-unescape blocks, dotToNested pollution,
  and "app.loadInstalledPlugins" in app-shell: all inside the legacy
  client-side config cluster whose entry points are shadowed by
  plugins_manager.js / replaced by server-rendered forms (zero live
  callers, verified) - queued for wholesale deletion in the follow-up
  rather than patching dead code.
- custom-feeds-helpers.js findings (3): file was deleted in a prior commit.
- console.error/warn override removal and afterSwap script re-execution
  removal: deliberate pre-existing workarounds every partial's inline
  init currently depends on; reworking them safely needs isolated testing
  (follow-up), and the error suppression is already double-gated
  (insertBefore AND htmx match).
- "move durations bootstrap into a bundle": inline partial-scoped init is
  the established pattern for HTMX partials in this codebase.

Validation: all 40 web tests pass; py_compile on all touched Python; all
touched templates parse; rotation-order defensive paths unit-tested.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(web): fix remaining real Codacy findings (2 of 6)

- htmx-config.js: two more unused catch bindings dropped (optional catch
  binding), matching the earlier fix.
- app-early.js: second forEach arrow (the stub updatePluginTabs copy)
  braced so the callback no longer returns tab.remove()'s value.

The other 4 findings ("Non-serializable expression must be wrapped with
$(...)") are deliberately NOT "fixed": that rule belongs to a
browser-automation (WebdriverIO-style) lint context and is misfiring on
ordinary arrow-function constants. Converting them to function
declarations would look compliant but BREAK the code - all four arrows
intentionally capture the enclosing Alpine component's `this` for the
stub-to-full enhancement logic. The right remedy is disabling that
pattern for this repo in Codacy's Code Patterns settings (or dismissing
the four findings), not a code change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): floating preview shows a frame immediately on open + is resizable

The floating preview opened empty and stayed empty until the display next
CHANGED - the SSE stream only pushes frames on change, and the panel only
consumed frames while already open, so the connection's initial frame
(sent while the panel was closed) was dropped. Reported from mobile
testing as "the button doesn't work".

- updateDisplayPreview now caches the latest frame globally regardless of
  panel state; opening the panel populates the image from that cache
  instantly, then live frames take over.
- Resizable: a size button cycles 192/256/384/512px presets (persisted per
  browser; works on touch), and desktop additionally gets a native drag
  handle (CSS resize: both). The image is fluid within the panel; on
  phones the panel is capped to the viewport width. The size icon
  (fa-up-right-and-down-left-from-center) is verified present in the
  vendored FA 6.0.0.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): stop duration fields leaking into config root; isolate per-file dependency installs; fix test fixture leak

Verified each finding against current code.

- api_v3 save_main_config: both duration blocks (*_duration suffix fields
  and the newer duration__<mode> fields) only READ from `data`, never
  removed the keys. The generic "remaining keys" merge later in the same
  function has no skip-list entry for either pattern, so every duration
  field was ALSO written a second time as a bogus top-level config key
  (e.g. "clock_duration": 30 and "duration__mlb_live": 42 sitting at
  config root, alongside the correct nested
  display.display_durations.<key>). Confirmed by tracing the full
  function. Fixed by popping each handled key from `data` (same pattern
  already used for plugin_rotation_order) and validating strictly: a
  non-integer duration now returns 400 with a message naming the
  offending field/mode instead of silently logging and moving on (for the
  *_duration fields, which previously had zero validation at all).
- api_v3 dependency-install loops (git_pull's post-update sync and
  install_base_requirements): _pip_install_requirements can raise
  subprocess.TimeoutExpired or OSError (confirmed: install_requirements_file
  in permission_utils.py never catches either internally, despite its
  docstring's "never raises on non-zero exit" only covering return codes).
  Both loops previously let one file's exception either abort the whole
  try block (skipping the second requirements file entirely) or propagate
  uncaught. Each file's install is now in its own try/except, so a timeout
  or OSError on one file is recorded as a labeled failure and the loop
  continues to the next file.
- test_web_smoke.py: the `client` fixture mutated the module-level
  pages_v3 Blueprint singleton's config_manager/plugin_manager directly
  with no teardown - since pages_v3 is shared across the whole pytest
  process (test_web_settings_ui.py touches the same attributes), this
  fixture's mocks could leak into whichever test ran next. Now saves the
  originals, yields the client, and restores them in a finally block.

Validation: py_compile passes; all 40 web tests pass with the now-generator
fixture.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair garbled Advanced Hardware section description

An earlier sed-based text update concatenated the old and new copies of
this description instead of replacing one with the other, leaving a
duplicated sentence with the &mdash; entity broken into ".mdash;" (visible
as literal "mdash;" text on the page). Restored to one clean sentence.

Other findings from this review were already fixed in a prior commit
(installedPlugins setter) or are confirmed dead code with zero live
callers (executePluginAction/dotToNested/entity-unescape/generateFieldHtml,
all reachable only from the two unused savePluginConfig copies in
app-shell.js - grepped every template, no references) - same legacy
cluster flagged in earlier review passes, still queued for a dedicated
deletion follow-up rather than patched in place here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): remove redundant htmx:afterSwap script re-execution (was double-executing every partial's inline script)

Re-verified this CodeRabbit finding, previously deferred as "needs isolated
testing" - traced it to a confirmed, active bug rather than a style
concern:

htmx 1.9.10's own config defaults to allowScriptTags: true (confirmed in
the vendored htmx.min.js, which itself contains the same clone-and-reinsert
<script> mechanism internally). This means htmx ALREADY re-executes every
<script> tag in swapped content by default, exactly like a browser
navigating to a new page. The custom htmx:afterSwap listener in
htmx-config.js did the identical clone-and-reinsert a SECOND time on top of
htmx's own handling - so every inline <script> block in every HTMX-loaded
partial (overview, display, durations, plugin config, etc. - most partials
have one) executed twice per load.

Confirmed safe to delete outright, not just narrow: grepped every hand-written
JS file for a manual `dispatchEvent(... 'htmx:afterSwap' ...)` that might
have relied on this handler for a non-htmx code path (e.g. the direct-fetch
fallbacks like loadOverviewDirect) - none exists, so nothing depended on
this listener specifically; htmx's native handling covers every real
htmx-driven swap on its own.

Left in place, unchanged: the console.error/console.warn global override
a few lines up in the same file, which suppresses known-noisy
HTMX-timing-race messages. That one is a legitimate anti-pattern too
(broad substring matching can mask unrelated errors) but redesigning it
needs care to preserve real diagnostics while still hiding the specific
harmless races it targets - a scoped follow-up, not a same-day deletion
like this confirmed-duplicate handler.

Validation: all 27 fast web tests pass; JS brace/paren balance sanity
checked (no local Node/browser available in this sandbox to execute the
file directly - verify manually in-browser before merge).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore(display): add missing [DisplayController] prefix to the reconcile-complete log

Re-verifying the full CodeRabbit findings list against current code
surfaced one still-open item: the nitpick asked for the prefix on BOTH
rotation-related log lines, but only "Applied plugin rotation order" got
it in the earlier pass - "Plugin reconcile complete" was missed. No
message/argument/level change, matching the finding's own scope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web): repair dead /v3/logs link on the display hardware-error banner

The "Logs tab" link in the display-settings simulation-mode banner was a
real <a href> to /v3/logs, but no such route has ever existed (log content
is loaded client-side via activeTab, not a dedicated page route) - the link
404'd regardless of the /v3 prefix change. Switch it to the same
activeTab-switching pattern the real nav uses.

* fix(web): raise display Rows field max from 64 to 128

Cols already allowed up to 128; Rows was capped at 64, which rejects
valid larger panel configurations (e.g. 128-row tile chains). No
server-side schema enforces a rows max, so this was purely an
overly-strict HTML input attribute.

* fix(web): remove redundant htmx.org substring check flagged by CodeQL

CodeQL flags .includes('htmx.org') as "incomplete URL substring
sanitization" - a false positive here, since this string is only ever
matched against console.error/warn message text to decide whether to
suppress a known-harmless HTMX timing-race log line, not used for any
URL-trust/redirect decision. The check was also redundant: 'htmx' is
already a substring of 'htmx.org', so the plain .includes('htmx') check
right next to it already covers every case the removed check did.

* fix(web): restore htmx script re-execution timing that Alpine x-data depends on

Removing the custom htmx:afterSwap script-reexecution handler (in a prior
commit, as a "duplicate execution" cleanup) broke every partial whose Alpine
x-data component function is defined by an inline <script> in that same
partial (e.g. wifi.html's wifiSetup()) - confirmed live via browser console:
"Alpine Expression Error: wifiSetup is not defined" on every field in the
WiFi tab.

Root cause: htmx's own native script execution runs during its "settle"
phase (~20ms after swap, per htmx's own defaultSettleDelay), but Alpine's
MutationObserver evaluates x-data on newly-inserted elements synchronously,
right as the swap lands - before settle. So the inline <script> defining
wifiSetup() was still un-run when Alpine tried to call it, and Alpine does
not retry a failed x-data evaluation later once the function does become
defined.

Fix: re-execute swapped <script> tags ourselves on htmx:afterSwap (which
fires synchronously, before settle, beating Alpine's observer), and disable
htmx's own native script re-execution (htmx.config.allowScriptTags = false)
so the same script doesn't also run a second time during settle - restoring
correct timing without reintroducing the original double-execution bug.

Also in this commit:
- fix XSS: unescaped repoUrl in a title attribute in renderSavedRepositories
- replace .includes('github.com') substring checks with real URL hostname
  validation (CodeQL: incomplete URL substring sanitization)

* fix(web): wait for async plugin install to finish before auto-enabling it

Confirmed live: installing hockey-scoreboard logged "installation queued"
(success) immediately followed by "enabling it failed" with a 404 "Plugin
not found" from /api/v3/plugins/toggle.

/api/v3/plugins/install runs the actual clone + plugin-manager discovery
asynchronously via an operation queue when one is configured - the response
installPlugin() was checking only means the operation was queued, not that
the plugin is installed yet. Calling togglePlugin() right after that
response 404s because plugin_manager hasn't discovered the new plugin.

Fix: reuse the same operation-polling mechanism uninstallPlugin() already
has (generalized pollOperationStatus() to take onComplete/onFailed/onTimeout
callbacks instead of hardcoding uninstall behavior) so installPlugin() waits
for the operation to actually complete before enabling it. Falls back to
enabling immediately when no operation_id is returned (direct/synchronous
install path, no queue configured).

* fix(web): resolve remaining valid findings from latest review pass

- custom-feeds.js: fix asset-upload contract mismatch (field name "file" ->
  "files", response read from top-level "uploaded_files" not "data.files") -
  same bug already fixed in this file on a separate branch/PR (#420), which
  this branch never received since they're independent PRs off main
- custom-feeds.js: add aria-label to the two icon-only "remove feed" buttons
- custom-feeds.js: move file-input reset into .finally() so a failed upload
  doesn't leave the input stuck holding the file, blocking retry of the
  same file
- app-shell.js: fix executePluginAction(pluginId, actionId) parameter
  order/count mismatch vs. its callers' (actionId, index, pluginId) -
  currently masked by plugins_manager.js's correct version overwriting this
  one at load time (classic vs. deferred script order), but worth fixing
  outright since it's an isolated, self-contained reassignment (not inside
  the Alpine app() object literal) and removes a latent footgun
- overview.html: align Alpine-state resolution with settings-search.js's
  two-tier getAppData() (also check appEl.__x.$data, not just _x_dataStack)

Verified already addressed by earlier passes (no change needed):
plugin_rotation_order validation, DisplayController log prefixes, togglePlugin
returning its promise for install-flow chaining, installedPlugins setter
always updating state, mobile-nav aria-label, toggleSection aria-expanded
sync, PluginOrderList bounded init retries (both display.html and
durations.html), plugin-order-list.js Array.isArray validation, batched
getImageData in the LED-dot preview renderer, app.py exception narrowing/
logging, form-submission log redaction.

Confirmed dead code, skipped (unreachable - zero template/JS callers,
verified via full-repo grep): dotToNested prototype-pollution hardening,
generateFieldHtml HTML-injection hardening, and the HTML-entity-unescape
block in JSON parsing - all three live only inside app-shell.js's two
legacy savePluginConfig implementations (one Alpine-method, one standalone),
neither of which any template or script calls. The real, live plugin-config
path is server-rendered via GET /partials/plugin-config/<id>.

Explicitly NOT reverted: the htmx:afterSwap script-execution listener. An
earlier finding batch asked to remove it as "duplicate" htmx behavior; that
was tried and reverted this session after live testing on hardware proved
it broke every partial whose Alpine x-data depends on an inline <script>
in the same partial (confirmed: WiFi tab hard-failed with "wifiSetup is not
defined"). Removing it again would reintroduce that regression.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 20:32:07 -04:00
66f9950a30 chore(ci): add security-audit tooling scripts (workflow files pushed separately) (#414)
* chore(ci): add security-audit workflow and plugin security-proof scripts

- scripts/prove_security.py, audit_plugins.py, generate_report.py --
  automated checks (dangerous eval()/exec() calls, dependency scanning,
  report generation) for plugins.
- .github/workflows/security-audit.yml + bandit.yaml -- CI wiring for
  the above plus gitleaks secret scanning and bandit static analysis.
- .github/workflows/tests.yml -- pytest matrix across Python 3.10-3.12.

Also fixes two Codacy findings while these files are freshly landing:
- prove_security.py: dropped a pointless f-string prefix with no
  placeholders.
- security-audit.yml: pinned gitleaks/gitleaks-action to a full commit
  SHA (matching this repo's existing pinning convention in test.yml)
  instead of the floating v2 tag.

Split out of the original chore/dead-code-removal commit, which had
accidentally bundled this in alongside unrelated dead-code deletions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* chore: drop workflow files -- pushed separately (needs workflow OAuth scope)

* fix(security-tooling): address PR review findings across bandit.yaml, audit_plugins.py, generate_report.py, prove_security.py

bandit.yaml:
- Removed scripts/prove_security.py's file-level exclusion. Ran bandit
  directly to get ground truth: the real false positive is B105 (dict key
  "PASS" misread as password-like), not the eval/exec pattern the old
  comment claimed. Added a targeted # nosec B105 there, and found+fixed
  the identical pattern already present in generate_report.py.
- Left the repo-wide B607 skip as-is: confirmed via AST scan that properly
  narrowing it touches 100+ bare-name subprocess call sites across
  wifi_manager.py, store_manager.py, permission_utils.py, app.py, and
  start.py -- none of which are part of this PR. Out of proportion to fix
  here; flagged as a dedicated follow-up.

scripts/audit_plugins.py:
- SyntaxError/OSError while scanning a plugin file now report CRITICAL
  (blocking) instead of WARNING/INFO -- a file that couldn't be parsed or
  read was never actually checked for danger, so it must not silently
  pass the audit.
- --plugin <name> now tracks whether the requested plugin was found across
  all PLUGIN_BASE_DIRS and exits 1 with a clear error if not, instead of
  silently scanning zero plugins and reporting success.
- The AST visitor now tracks import aliases (import subprocess as sp;
  from builtins import eval as e) and resolves them before checking
  against dangerous APIs, closing a straightforward evasion of every
  PLUGIN-001 through PLUGIN-005 check. Verified against both aliased and
  unaliased evasion patterns.

scripts/generate_report.py:
- _md_table_row now escapes pipe characters and normalizes newlines in
  every cell, so scanner-controlled content (a matched secret, a bandit
  issue_text) can't corrupt the Markdown table structure.
- _load now distinguishes "artifact missing/malformed" from "valid empty
  result": each summarizer returns an availability flag, and main() now
  reports INCOMPLETE (not PASSED) with exit code 1 when any artifact is
  unavailable, instead of silently folding it in as 0 findings.
- Gitleaks suppression now uses exact-match placeholder values (pulled
  from the actual config_secrets.template.json) plus a template-path
  allowlist, replacing broad substring checks that could hide a real
  secret containing something like "example.com" as part of its value.

scripts/prove_security.py:
- T1b (dangerous plugin calls): a file that fails to parse/read now
  reports CRITICAL with the exception details instead of being silently
  swallowed by `except (SyntaxError, OSError): pass`.
- T6 (Docker hardening): base images must now be pinned to an @sha256
  digest; a specific tag like python:3.12 is mutable and is now correctly
  flagged as unpinned, not just missing tags or :latest.
- T2a (API surface): no config mechanism for enforcing local-only access
  exists in this codebase today (app.py hardcodes host='0.0.0.0'), so the
  "environment-aware" check as described isn't buildable without adding
  new config infrastructure -- out of scope here. Applied the achievable
  part: upgraded from INFO to WARNING, since enforcement can never
  currently be confirmed.
- T1a (zip-slip): replaced the whole-file substring check with an AST
  walk that finds every extract()/extractall() call and confirms an
  is_relative_to() guard + "Zip-slip detected" log precede it in the same
  function. Verified it still passes on the real store_manager.py (both
  the per-member and validate-then-bulk-extract call sites) and correctly
  flags a synthetic unguarded extractall().
- T3a (hardcoded secrets): violation details no longer include the
  matched credential text -- only file, line, pattern type, and a
  redacted SHA-256 fingerprint, so a real finding doesn't get published
  into CI logs/artifacts/PR comments with wider exposure than the
  original leak. Verified with a synthetic secret that no raw content
  reaches the output.

Validated: all four files compile; bandit scans all three scripts clean
(2 legitimate targeted suppressions, 0 unaddressed findings); each new/
changed code path exercised directly (alias evasion, unmatched --plugin,
missing/malformed/valid-empty artifacts, digest-pinning, zip-slip
guard/no-guard, secret redaction); full audit_plugins.py -> prove_security.py
-> generate_report.py pipeline run end-to-end producing a correct report.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(security-tooling): address follow-up review findings on PR #414

scripts/generate_report.py:
- Added an explicit `object` type annotation to _md_sanitize_cell's value
  parameter -- it deliberately accepts any stringifiable value (calls
  str(value) unconditionally), so `object` reflects its actual contract
  more accurately than leaving it untyped.

scripts/prove_security.py:
- Dockerfile FROM-line parsing: renamed the comprehension variable `l` to
  `line` (ambiguous single-letter name). More importantly, fixed a real
  false-positive: `FROM --platform=<platform> <image>` was reading the
  --platform= flag itself as the image token, so a properly digest-pinned
  image behind a platform flag was incorrectly reported as unpinned.
  Verified against platform+digest, platform+tag-only, and digest+AS-alias
  Dockerfiles.

scripts/audit_plugins.py:
- Consolidated visit_Call's dangerous-API detection: previously, alias
  resolution only covered ast.Name calls for eval/exec/compile and
  ast.Attribute calls for subprocess/os.system, missing from-imported
  subprocess/os functions called as bare names (from subprocess import
  run as prun; prun(cmd, shell=True) or from os import system as s;
  s(cmd)). Added _resolve_call_target() to resolve both call shapes to a
  single fully-qualified target, then run all five PLUGIN-00x checks
  against that one resolved value. Verified against 10 evasion
  combinations (from-import aliases, direct/attribute calls, aliased
  module imports) and confirmed zero false positives on benign os/
  subprocess usage without shell=True.

Validated: all three files compile, bandit scans clean (same 2 legitimate
suppressions as before, 0 new findings), audit_plugins.py/prove_security.py
re-run against the real repo with no regressions from the prior fix pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 14:33:17 -04:00
4abcd0e4f9 Fix PermissionError reading config_secrets.json in web interface (#416)
ledmatrix.service (main display) runs as root while ledmatrix-web.service
runs as the non-root install user (install_web_service.sh). Both
config_manager.py and config_manager_atomic.py only chmod'd
config_secrets.json to 0o640 without ever fixing its group, so a file
written by the root service ended up group-owned by root and unreadable
by the web user, crashing the settings page with a raw PermissionError.

Add ensure_shared_group_ownership() to chgrp secrets/config files (best
effort, root-only) to the project directory's owning group whenever they
are created or saved, and self-heal existing files on load. Also make
get_raw_file_content() tolerate an unreadable secrets file the same way
load_config() already does, degrading to empty secrets instead of a 500.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-15 14:25:58 -04:00
2a1c47fa76 chore: remove dead modules and unused dependencies (~1,180 LOC) (#412)
* chore: remove dead modules and unused dependencies (~1,180 LOC)

Deletions, each re-verified with a fresh repo-wide grep (core, web,
scripts, docs, plugin monorepo) immediately before removal:

Modules with zero live importers:
- src/background_cache_mixin.py + src/generic_cache_mixin.py (134+150
  LOC — referenced only by each other)
- src/font_test_manager.py (134 LOC)
- src/image_utils.py (22 LOC, self-documented deprecated)
- src/layout_manager.py (408 LOC — only its own test imported it) +
  test/test_layout_manager.py
- src/common/basketball_plugin_example.py (328 LOC sample)

requirements.txt entries with zero importers in core (pre-plugin-era
manager deps): icalevents, geopy, timezonefinder, unidecode. Plus the
google-auth trio (google-auth-oauthlib, google-auth-httplib2,
google-api-python-client): their only importer is the calendar PLUGIN,
which declares all three in its own requirements.txt (verified in the
monorepo and on an installed copy) — the plugin dependency installer
owns them. Existing venvs are unaffected (removal doesn't uninstall);
fresh installs get them when calendar is installed.

Two stale references cleaned (a comment in test_pillow_compat.py, a
directory listing in HOW_TO_RUN_TESTS.md). Full suite green except the
two documented pre-existing failures (circuit_breaker mock drift, fixed
in #400; clock-simple 64x32 overflow, pre-dates this series); all core
entry modules verified importing cleanly under the emulator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: remove content accidentally bundled into the dead-code-removal commit

The previous commit's git add/commit swept in a lot of unrelated,
unreviewed work alongside the intended dead-code deletions: a new Plugin
Composer web UI, new security-audit/CI tooling, a new march-madness
plugin, and 23 local-development-only symlinks under plugin-repos/ (per
scripts/setup_plugin_repos.py's own docstring, these are meant to be
generated locally, never committed -- .gitignore has no entry for them,
which is how they slipped in).

Removed here, split into their own PRs instead (except plugin-repos/*
symlinks and march-madness/ncaa_logos, which are dropped rather than
carried forward -- see PR discussion):
- web_interface/blueprints/composer.py + composer-app.js +
  composer-canvas.js + composer.html + manager.py.j2
- scripts/prove_security.py, audit_plugins.py, generate_report.py
- .github/workflows/security-audit.yml, .github/workflows/tests.yml,
  bandit.yaml
- All plugin-repos/* symlinks (local dev artifacts, not meant to be
  committed at all)
- plugin-repos/march-madness/* and the 4 new assets/sports/ncaa_logos/*
  PNGs it needed (left out of every split PR pending a decision on
  whether march-madness belongs in the core repo or the plugin monorepo)

This PR now contains only what its title describes: the dead-code
removal from the previous commit, untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 09:58:31 -04:00
9db1d2391a chore(assets): add 4 NCAA team logos needed by march-madness (COLGATE, LEHIGH, MICHIGAN, RUTGERS) (#415)
march-madness (already live in ledmatrix-plugins) loads team logos from
this shared assets/sports/ncaa_logos/<ABBR>.png cache at runtime
(manager.py:233) -- these 4 were missing.

Split out of PR #412 (chore/dead-code-removal), which had accidentally
bundled these in alongside unrelated dead-code deletions.


Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 09:57:52 -04:00
14a59c863c perf(wifi): one status fetch per monitor tick instead of three (#411)
* perf(wifi): one status fetch per monitor tick instead of three

The wifi monitor daemon fetched WiFi status + ethernet state before its
AP-mode check, the check internally fetched the same state again (with
retry), and the daemon fetched a third time afterwards — each fetch is
several nmcli subprocess forks, every 30s, forever, even on a perfectly
healthy link.

check_and_manage_ap_mode's decision logic is extracted to
_manage_ap_mode(status, ethernet, ap_active); the new
check_and_manage_ap_mode_with_state() runs the single (retrying) fetch
battery and returns (changed, status, ethernet, ap_active_after) — the
post-state is derivable because state only ever flips via one enable or
one disable. The original bool-returning method delegates, so existing
callers are untouched. The daemon's dead pre-fetch is removed and its
post-check reads use the returned state; the AP-enable retry semantics
(_get_wifi_status_with_retry) are preserved exactly. ~10-15 forks/30s
drops to ~4-6.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(wifi): add missing type hints on AP-mode state helpers, fix unused-var lint in tests

- check_and_manage_ap_mode_with_state now declares its
  Tuple[bool, WiFiStatus, bool, bool] return type instead of being untyped.
- _manage_ap_mode's status/ethernet_connected/ap_active parameters are now
  typed (WiFiStatus, bool, bool), matching its existing -> bool return hint.
- test_wifi_check_state.py: rename unused unpacked variables (status/
  ethernet/ap_after) to underscore-prefixed names at the three call sites
  that don't use them, leaving the used ones untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 11:22:37 -04:00
bff13129c4 perf(config): mtime-signature fast path for load_config (#410)
load_config re-read and re-parsed config.json, the secrets file, AND the
template (running the recursive migration diff) on every call — with
~30 call sites in web request handlers, some hit 2-3x per request.

Fast path: stat all three files (mtime_ns + size); when unchanged since
the last successful load, return the already-parsed self.config (same
aliasing semantics as before). The signature is taken AFTER load +
migration so a migration write-back doesn't retrigger, and both save
paths refresh it. Cross-process freshness is preserved by construction:
a save from the other process bumps the file mtime, so the next load
here re-reads — verified by a dedicated test. Same-second edits are
caught by mtime_ns plus a size check.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 08:49:43 -04:00
6499794c12 fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache (#409)
* fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache

ledmatrix-leaderboard's data_fetcher.py calls these two real-CacheManager
methods (src/cache_manager.py:313,817), but MockCacheManager had neither --
update() always hit an AttributeError, caught by a broad except and logged,
so the harness rendered an empty-but-green leaderboard on every test run
without ever exercising real standings data.

Both mocks delegate to the existing get()/set() -- a mock doesn't need the
real strategy's per-data-type max_age/market-hours timing, plugins under
test just need the methods to exist and round-trip whatever was cached.

* fix(testing): clear get_cached_data_with_strategy_calls in MockCacheManager.reset()

reset() cleared get_calls/set_calls/delete_calls but not the newer
get_cached_data_with_strategy_calls tracker, so a reused mock (e.g. across
test cases sharing a fixture) retained stale strategy-call records after
reset().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-14 08:33:08 -04:00
ChuckandGitHub 3d347a368a fix(testing): stop plugin enabled:false schema defaults from silently disabling harness tests (#408)
check_plugin.py, render_plugin.py, and the pytest plugin matrix each built
config as {"enabled": True} then merged in config_schema.json's defaults on
top, letting a plugin's own enabled:false default (a reasonable choice for
a seasonal/opt-in plugin -- 15 of 23 real plugins ship one) silently win.
Every harness/CI render of those plugins was testing "disabled, do
nothing" rather than real behavior.

Extract build_full_config() into testing/loading.py (already the shared
home for plugin-discovery/config-default logic) and use it from all three
call sites: schema defaults, then a forced enabled=True, then harness.json's
config, then the caller's explicit config -- so a test can still
deliberately disable a plugin on purpose, it just can't happen by accident
via the plugin's own shipped schema default anymore.
2026-07-14 08:21:35 -04:00
0aca40cf3a perf(plugins): run scheduled updates off the render thread (#407)
* perf(plugins): run scheduled updates off the render thread

plugin update() executed inline in the render loop — execute_update's
internal thread.join(timeout=30) blocked it, so one slow plugin HTTP
fetch froze scrolling for the whole fetch (up to 30s; DNS-retry storms
made this a regular occurrence on flaky networks).

Scheduling stays on the render thread and keeps every existing gate
(enabled, circuit breaker, can_execute, interval); due updates are now
enqueued to a single background worker (serialized — same one-at-a-time
execution as before, no thundering herd). RUNNING is set at enqueue so
can_execute blocks re-entry alongside the pending-set dedup.

Per-plugin locks make the old implicit update/display no-overlap
guarantee explicit: the worker holds the plugin's lock through its
update; the display side try-locks and, when the plugin is mid-update,
holds the last frame for that iteration — reported as success so a
mid-update skip never advances the rotation. Unlike before, the
guarantee now also holds across the post-timeout window (previously the
lingering update thread overlapped display()). Deadlock-free by
construction: the worker takes one lock; display never blocks.

Timeout semantics unchanged (lingering daemon thread documented).
Kill switch: plugin_system.synchronous_updates: true restores the
inline path.

8 new concurrency tests (non-blocking scheduler, overlap assertion
under a hammering display loop, lock release on failure/timeout paths,
dedup, kill switch); 4-min devpi soak clean (updates completing,
rotation advancing, no stuck RUNNING states).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(plugins): fix skipped-frame health/force_change tracking, config validation, and lock-lifetime gaps in async updates

Addresses PR #407 review findings:
- display_controller: only clear force_change / record health success
  when display() actually ran this frame, not when the frame was skipped
  because the plugin's lock was busy (a skip must preserve a pending
  mode-switch force_clear).
- plugin_manager: replace bool() coercion of synchronous_updates with
  explicit isinstance validation of plugin_system/synchronous_updates,
  failing safe to synchronous mode (with a logged reason) on malformed
  config instead of silently defaulting to async.
- plugin_manager: _update_worker_loop now acquires the plugin lock before
  looking up its instance and re-checks under the lock, so an unloaded
  plugin's lifecycle state is never resurrected to ENABLED.
- plugin_manager + display_controller: move lock ownership (and, for
  updates, RUNNING/pending lifecycle bookkeeping) into the actual update()/
  display() call itself rather than the timeout-wrapped caller, so the
  lock stays held for the real operation's duration even after
  PluginExecutor's own join(timeout) elapses and a lingering daemon thread
  keeps running in the background.
- DisplayController.cleanup() now stops the update worker before tearing
  down display/cache resources; stop_update_worker() logs when the join
  times out instead of failing silently.
- test_async_plugin_updates: rewrite test_unloaded_while_queued_is_harmless
  to exercise the public unload_plugin() lifecycle (via a deterministic
  blocker) instead of deleting pm.plugins directly, and add a regression
  test proving the plugin lock stays held through PluginExecutor's own
  timeout while the real update() call is still running.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 08:08:51 -04:00
9837315308 perf(display): dirty tracking in update_display + plugin FPS declaration (#406)
* perf(display): dirty tracking in update_display + plugin FPS declaration

update_display now skips SetImage+SwapOnVSync when the frame is
byte-identical to the last pushed one (adler32 digest) AND brightness
is unchanged — brightness is part of the digest, and set_brightness
additionally resets it, so a dim-schedule change can never be skipped.
clear() resets the digest (it writes to the matrix directly). Skipping
a swap is hardware-safe: the panel refreshes the current frame from the
driver's own thread; swaps only change content.

Kill switch: display.dirty_tracking: false restores always-push.

display_controller's high-FPS decision gains a precedence step: a
plugin exposing needs_high_fps is honored first (so static-image can
declare False for still PNGs and stop burning a 125fps loop on them);
static-image without the attribute keeps its historical forced
high-FPS (GIF back-compat); scrolling logic is otherwise unchanged.

Verified with 7 tests against the real DisplayManager on
RGBMatrixEmulator (identical-frame skip, pixel-change push, clear and
brightness invalidation, snapshot-through-skip, kill switch) plus the
202-test display/controller/vegas suites, and a clean devpi deploy.
Audit: every SetImage/SwapOnVSync/Clear/brightness call site is inside
display_manager — no external writer can bypass the digest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(display): serialize update_display, narrow brightness exception, log fixes

CodeRabbit review on #406, verified against current code:

- update_display() can genuinely be called from background threads (some
  sports base classes call it directly from inside update() for an
  immediate "live" refresh), not just the render loop — confirmed via the
  existing follower-mode gating wrapper in display_controller.py, which
  exists specifically because "background plugin threads" can reach it.
  Without a lock, two callers could both pass the digest check before
  either writes _last_pushed_digest back, causing a redundant push, or
  interleave the offscreen/current canvas swap. Added self._update_lock
  (RLock, in case of re-entrant callers) around the full method body so
  every call site is automatically covered — no caller changes needed.
  (No prior lock existed to reuse on DisplayManager; this adds one.)
- Narrowed the brightness-read exception handler to AttributeError,
  matching the established pattern in get_brightness()/set_brightness()
  — a getattr() with a default already swallows AttributeError, so the
  only case this guards is the property getter itself raising, and the
  established pattern treats that as an expected, specific failure mode
  rather than something to blanket-catch.
- FPS-check debug log now includes the plugin_id already in scope
  (previously only active_mode) and a "[DisplayController]" prefix for
  grep-ability, matching the sibling log two lines below it.
- test_display_dirty_tracking.py: dm fixture and test_config_flag_wires_through
  now reset the DisplayManager singleton on teardown, matching the pattern
  test_display_manager.py already uses elsewhere in the same file family.
- test_snapshot_still_written_on_skip previously only exercised the
  non-skip (push) path despite its name; now performs a second update that
  meets the skip conditions (identical frame) and asserts the snapshot is
  still written even though the panel push itself is skipped.

All 7 dirty-tracking tests pass, plus the full display_manager/
display_controller/vegas suite (140 passed). Full repo suite has only the
5 known pre-existing failures (double-sided config x2, state_reconciliation
x2, and test_circuit_breaker's conftest.py mock signature drift — the
latter fixed in #400, which this branch's base predates).

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 10:31:20 -04:00
c1fa5094be fix(store): plugin updates keep the old install until the new one succeeds (#405)
* fix(store): plugin updates keep the old install until the new one succeeds

Both reinstall paths in update_plugin — the monorepo-migration remote
switch AND the routine archive update every store user hits — deleted
the installed plugin directory BEFORE downloading its replacement. A
mid-update failure (bad network, registry error) permanently destroyed
the plugin. Seen in the field: a Pi with broken DNS lost 12 plugins in
one update pass during the monorepo migration.

New _reinstall_with_rollback: rename the old install aside (using the
'.standalone-backup-' name pattern plugin discovery already excludes),
run install_plugin, remove the aside on success — restore it on ANY
failure, clearing partial-download debris first. A stale aside from a
previous crash is cleared before starting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(store): serialize concurrent updates per plugin, check cleanup results

CodeRabbit review on #405 flagged two things in
_reinstall_with_rollback, both verified against current code:

- Real race: the web UI runs Flask with threaded=True and there's a
  single update route, so two overlapping requests for the same
  plugin_id (double-click, two tabs) can interleave. The loser could
  rename the winner's in-progress install aside mid-download, deleting
  its own rollback safety net — worse than the bug this function
  exists to fix. Added a lazy per-plugin_id lock dict (mirrors the
  plugin_manager per-plugin lock pattern) held for the whole function.
- _safe_remove_directory's return value was ignored at both call
  sites. Stale-aside cleanup failure now aborts cleanly instead of
  falling through to a rename that would fail anyway with a less
  useful error; post-success backup-removal failure now logs instead
  of failing silently (still returns True — the update itself
  succeeded, and the next update self-heals the leftover aside).

Left the third nitpick (test_stale_aside_from_previous_crash_is_cleared)
addressed by asserting the stale dir is actually gone and that
install_plugin was reached, rather than just the end-to-end result.

Added a concurrency regression test asserting install_plugin never
runs for the same plugin_id while another call is in flight.

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:32:37 -04:00
4d49b0f892 perf(display): snapshot mirror — viewer gating, digest skip, keepalive (#404)
The display service PNG-encoded its frame to /tmp/led_matrix_preview.png
at 5 fps, 24/7 — identical frames, no viewers, per-call imports and a
chmod every write. On the devpi baseline the display service idles at
~92% CPU; this was one of its biggest fixed costs.

- New pure policy (src/common/snapshot_policy.py, unit-tested off-Pi):
  WRITE changed frames at full rate only while a viewer is watching,
  at a 30s idle cadence otherwise; NEVER re-encode unchanged frames —
  bump mtime (os.utime) every 20s instead, keeping the health check's
  snapshot-age liveness proxy (60s threshold in api_v3) green. Cross-
  referencing comments guard the two constants.
- Viewer detection: the web SSE display broadcaster (which only runs
  while browsers are subscribed) touches /tmp/led_matrix_preview_viewer
  each loop; the display service stats it at most 1/s. On viewer
  arrival the write clock resets so the first frame lands within ~1s.
- Hoisted the per-call pathlib/permission_utils imports; directory
  permissions ensured once instead of every frame.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:28:30 -04:00
efe76d3add perf: hot-path micro fixes in the render loop (#403)
* perf: hot-path micro fixes in the render loop

- _check_wifi_status_message stat'd the status file on every render
  iteration (60+ fps) for a message whose lifetime is seconds; throttle
  the check to 1 Hz with a cached result.
- Demote the per-iteration "Display active, processing mode" INFO to
  DEBUG and convert the remaining eager f-string logs to lazy % args —
  the devpi baseline showed ~9 journald lines/sec, which is both noise
  and SD-card wear.
- Vegas cycle-end blank frame: hoist the inline PIL import and reuse a
  preallocated buffer instead of allocating per cycle wrap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix: initialise wifi-status throttle state in __init__

Codacy (pylint access-member-before-definition) on #403: the throttled
early-return read _wifi_status_last_result relying on the non-local
invariant that the first call always passes the throttle window and
assigns it. Correct at runtime, but fragile — initialise both throttle
fields in the constructor and drop the getattr fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:23:25 -04:00
273d9962d1 fix(cache): stop fsync-hammering the SD card on unchanged data (#402)
DiskCache.set wrote every key as mkstemp -> json.dump(indent=4) ->
flush+fsync -> replace -> chmod, on the persistent cache dir — dozens
of force-flushed SD writes per minute on an API-heavy install, mostly
rewriting identical data every plugin update cycle.

- Serialize once, compact (no indent): cache files are machine-read
  only; indenting multiplied the bytes written.
- Skip the disk when the payload for a key is unchanged (adler32 map,
  per-process); refresh the file mtime instead so records relying on
  mtime for TTL don't expire early. Self-heals if the file was removed
  externally (expiry cleanup).
- Drop the per-write fsync: os.replace already guarantees readers never
  see a torn file, and cache data is re-fetchable — the flush bought
  nothing but card wear.

API unchanged; DateTimeEncoder round-trip covered by tests.


Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 09:00:35 -04:00
9e3b5f366e fix(core): harden text-measurement caches; surface snapshot failures (#400)
* fix(core): harden text-measurement caches; surface snapshot failures

Deep-dive findings, all three latent on every 24/7 install:

- font_manager.metrics_cache and display_manager._text_width_cache were
  unbounded dicts keyed by (text, id(font)). Two problems: keys embed
  the measured TEXT, so ever-changing strings (a clock, a live score, a
  ticker) grow them without limit; and id()-keying without holding a
  reference means a garbage-collected font's id can be recycled by a
  DIFFERENT font, silently returning wrong widths/metrics (classic
  plugins create fonts per render, so this is reachable). Both caches
  are now LRU-bounded (1024) and pin the font in the entry so its id
  stays valid. metrics_cache also keyed on the text itself instead of
  hash(text), removing a collision path.

- _write_snapshot_if_due logged failures at DEBUG — invisible at the
  default level. The snapshot's mtime is the web UI's display mirror
  AND its hardware-liveness proxy, so a quiet failure freezes the
  mirror and makes health checks lie (seen in the field: a stale
  root-owned /tmp file froze it for a day). Failures now WARN, rate-
  limited to once per 5 minutes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* test: sync mock cache manager signature with CacheManager.get

test_circuit_breaker has been failing on main: plugin_health passes
memory_ttl= to cache_manager.get(), and the conftest mock's signature
was never updated — the same component/double drift class as the
monitored_update bug (#392).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 08:59:41 -04:00
6edd80d9f3 fix(schedule): stop stray 'days' data from overriding Global schedule (#399)
save_schedule_config never persisted the schedule's 'mode' field, and
_check_schedule inferred per-day vs global purely from whether a 'days'
dict was present for the current day. Config migration
(_merge_template_defaults) re-adds the template's 'schedule.days' (all
days disabled by default) whenever it's missing from the user's saved
config - which is exactly the case after saving Global mode, since that
save path intentionally pops 'days'. The result: a user on Global mode
would get their schedule silently reinterpreted as per-day, with today's
day disabled, blanking the display.

Persist 'mode' on save and have _check_schedule honor it explicitly
(mirroring how _check_dim_schedule already does), so a resurrected
'days' dict can't override an explicit Global selection. Falls back to
the old inference behavior only when no 'mode' is recorded.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 10:53:33 -04:00
1c7a0cef66 fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation (#398)
* fix(vegas): restore live plugin-update refresh dropped by sync refactor

Investigating a user report that Vegas scroll mode doesn't update scores
or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live
score change reached the ticker within a few seconds instead of waiting
for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed
plugin_last_update timestamps to detect which plugins got fresh data and
called coordinator.mark_plugin_updated() for each, and should_recompose()
checked has_pending_updates_for_visible_segments() to trigger an immediate
hot-swap.

PR #330 (May 14, multi-display wireless sync) refactored both call sites
while adding sync support and silently deleted this entire mechanism --
not just gated it behind the new sync-mode deferral it legitimately
needed, but removed it outright. The result: VegasModeCoordinator.
mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_
segments() have been fully implemented but never called from anywhere
since. Vegas mode's only remaining freshness sources are a 5s content
cache TTL (fine) and full recompose at cycle boundaries, which depending
on min/max_cycle_duration can be minutes away -- so live scores/status
can sit stale far longer than a user would expect from a "live" ticker.

Fix:
- Restored _tick_plugin_updates_for_vegas() in display_controller.py,
  wired as the Vegas coordinator's update callback in place of the plain
  _tick_plugin_updates(). Diffs plugin_last_update before/after the tick
  and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each
  plugin that actually got new data (rather than returning the list, since
  the callback interface no longer consumes a return value).
- Restored the has_pending_updates_for_visible_segments() check in
  render_pipeline.should_recompose(), positioned after (not instead of)
  the sync-mode early return PR #330 added, so standalone installations
  regain immediate refresh while synced leader/follower pairs correctly
  keep deferring hot-swaps to cycle boundaries as PR #330 intended.

Test plan:
- Added test_display_controller_vegas_tick.py and
  test_vegas_render_pipeline_recompose.py -- neither area had any prior
  test coverage, which is very likely why this regression went unnoticed
  for ~2.5 months.
- Verified both new test files fail against the pre-fix code (swapped in
  the current main versions of both files) with exactly the expected
  errors -- AttributeError for the deleted method, and the recompose
  assertion returning False instead of True -- then pass against the fix.
- Confirmed the sync-mode deferral this restoration must not break still
  holds: test_sync_active_defers_pending_updates_to_cycle_boundary.
- Full related suite (test_vegas_plugin_adapter, test_vegas_config,
  test_display_controller_plugin_toggle, test_display_controller_
  optimizations, test_plugin_system): 108 passed, 1 pre-existing failure
  unrelated to this change (test_circuit_breaker, stale mock signature).
- Full CI plugin-safety suite (test_harness, test_visual_rendering,
  test_plugin_matrix): 52 passed, 2 pre-existing skips.

* fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation

_tick_plugin_updates_for_vegas() snapshotted and later re-iterated
plugin_manager.plugin_last_update from the Vegas background update-tick
thread while the main render loop (or other callers) could mutate the
same dict concurrently — a real race (unprotected dict iteration/mutation
across threads), not just a style nit.

Move the snapshot/update/diff into a new locked
PluginManager.run_scheduled_updates_with_changes() so all reads and
mutations of plugin_last_update happen under one lock, and update
DisplayController to use it. The lock is only held around the dict
accesses, not the update pass itself, so slow plugin update() calls don't
serialize against other callers.

Also add a regression test covering that the Vegas coordinator is wired
to the Vegas-aware tick callback rather than the plain one.

Skipped as not worth the change:
- Narrowing the broad `except Exception` around
  vc.mark_plugin_updated(plugin_id) to specific types: it's a deliberate
  per-plugin isolation boundary (matches the same pattern used elsewhere
  in this file for plugin/coordinator calls) and there's no documented,
  stable set of exceptions that call can raise to narrow to.
- Adding an inactive-DisplaySyncManager test to
  test_vegas_render_pipeline_recompose.py: verified
  VegasModeCoordinator.set_sync_manager() already normalizes a
  SyncRole.STANDALONE manager to None before handing it to the render
  pipeline (src/vegas_mode/coordinator.py:152-156), so should_recompose()'s
  `is not None` check is correct in practice; the suggested case is
  already covered by that normalization.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-12 10:52:18 -04:00
ChuckandGitHub 6052a60d22 fix(vegas): restore live plugin-update refresh dropped by sync refactor (#395)
Investigating a user report that Vegas scroll mode doesn't update scores
or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live
score change reached the ticker within a few seconds instead of waiting
for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed
plugin_last_update timestamps to detect which plugins got fresh data and
called coordinator.mark_plugin_updated() for each, and should_recompose()
checked has_pending_updates_for_visible_segments() to trigger an immediate
hot-swap.

PR #330 (May 14, multi-display wireless sync) refactored both call sites
while adding sync support and silently deleted this entire mechanism --
not just gated it behind the new sync-mode deferral it legitimately
needed, but removed it outright. The result: VegasModeCoordinator.
mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_
segments() have been fully implemented but never called from anywhere
since. Vegas mode's only remaining freshness sources are a 5s content
cache TTL (fine) and full recompose at cycle boundaries, which depending
on min/max_cycle_duration can be minutes away -- so live scores/status
can sit stale far longer than a user would expect from a "live" ticker.

Fix:
- Restored _tick_plugin_updates_for_vegas() in display_controller.py,
  wired as the Vegas coordinator's update callback in place of the plain
  _tick_plugin_updates(). Diffs plugin_last_update before/after the tick
  and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each
  plugin that actually got new data (rather than returning the list, since
  the callback interface no longer consumes a return value).
- Restored the has_pending_updates_for_visible_segments() check in
  render_pipeline.should_recompose(), positioned after (not instead of)
  the sync-mode early return PR #330 added, so standalone installations
  regain immediate refresh while synced leader/follower pairs correctly
  keep deferring hot-swaps to cycle boundaries as PR #330 intended.

Test plan:
- Added test_display_controller_vegas_tick.py and
  test_vegas_render_pipeline_recompose.py -- neither area had any prior
  test coverage, which is very likely why this regression went unnoticed
  for ~2.5 months.
- Verified both new test files fail against the pre-fix code (swapped in
  the current main versions of both files) with exactly the expected
  errors -- AttributeError for the deleted method, and the recompose
  assertion returning False instead of True -- then pass against the fix.
- Confirmed the sync-mode deferral this restoration must not break still
  holds: test_sync_active_defers_pending_updates_to_cycle_boundary.
- Full related suite (test_vegas_plugin_adapter, test_vegas_config,
  test_display_controller_plugin_toggle, test_display_controller_
  optimizations, test_plugin_system): 108 passed, 1 pre-existing failure
  unrelated to this change (test_circuit_breaker, stale mock signature).
- Full CI plugin-safety suite (test_harness, test_visual_rendering,
  test_plugin_matrix): 52 passed, 2 pre-existing skips.
2026-07-12 10:40:34 -04:00
7f7f0d6464 feat: adaptive layout system — size-aware regions, crisp font ladders, image fitting (#393)
* feat(layout): adaptive layout & font scaling system for plugins

Add src/adaptive_layout.py — opt-in core helpers so plugins render
legibly on any panel size without hand-tuned per-display layouts:

- Region: integer rect algebra (bands/columns/weighted splits/centering)
  that partitions space so text bands can't overlap by construction
- Font ladders: ordered (family, size) steps known to render crisply
  (LADDER_GRID: X11 BDFs at native sizes; LADDER_ARCADE: PressStart2P at
  8px multiples) — fitting walks the ladder instead of scaling pixel
  fonts fractionally
- LayoutContext: breakpoint tiers, geometry scale vs. a declared design
  size, and cached fit_text/fit_lines/font_for_rows queries

Generalizes the three patterns proven in the field: f1-scoreboard's
scale factor, masters-tournament's tiers, baseball-scoreboard's font
fallback ladder.

Wiring: BasePlugin gains a lazy .layout property and draw_fit();
FontManager gains get_native_bdf_size() and a cache_generation counter;
manifest schema gains display.design_size and requires.display_size
max_width/max_height; 96x48 joins DEFAULT_TEST_SIZES; the bounds-check
harness records negative-coordinate draws; TextHelper's broken
measurement helpers are fixed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): adaptive image fitting + composite region helpers

Add src/adaptive_images.py — the image counterpart to fit_text:
- fit_image(img, box, mode=contain|cover|fill_height|stretch,
  crop_to_ink, anchor, resample, upscale) promoting the proven plugin
  patterns (football's crop-to-ink fill-height logos, masters' cover
  crop + NEAREST flags, static-image's letterbox). Upscales by default —
  thumbnail()'s downscale-only behavior is why imagery stays tiny on
  big panels.
- draw_fitted_image() pastes aligned within a Region with alpha mask.
- One central Pillow>=9.1 RESAMPLE shim replacing ~15 plugin copies.

LayoutContext.fit_image() caches results per (identity, box size,
options) with a 64-entry LRU; id()-keyed entries pin the source image.
BasePlugin.draw_image() is the one-liner adoption path beside draw_fit.

Composites in adaptive_layout.py: Region.offset() (user x/y-offset
passthrough), scoreboard_regions() (the two-logos-plus-score card math
duplicated across six sports plugins, logo_slot = min(H, W//2)), and
media_row() (art-left/text-right).

Fix LogoHelper's size-blind cache key (stale sizes on panel change);
deprecation note on dead image_utils.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(harness): scale-up fill check, config variants, multi-size dev gallery

Quality gates for adaptive layout:

- fill_metrics()/check_scale_up() in the safety harness: overflow catches
  content too big for a panel, but nothing caught content that stays tiny
  on panels >= 2x the plugin's declared design size. The check measures
  lit-content extents and warns (or fails, when a plugin opts into
  "fill_check": "strict" in test/harness.json) below 50% coverage on the
  doubled axis. Warn-only by default so no existing plugin breaks.

- harness.json "variants": extra runs with config overlays and their own
  golden dirs, so an opt-in mode (e.g. layout_mode: adaptive) is golden-
  tested beside the classic default. check_plugin.py loops base + variants
  and labels variant results mode@name.

- Dev preview server: GET /api/sizes (harness size sample), POST
  /api/render-matrix (render at up to 12 sizes in one call), size-preset
  dropdown, and an "All Sizes" side-by-side gallery in the preview UI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(plugins): adaptive-lib discoverability + advisory version compat warning

Discoverability: re-export the adaptive layout/image API from src.common
(the blessed-helpers package plugin authors already know) — canonical
paths stay src.adaptive_layout / src.adaptive_images so nothing breaks.
Document it in src/common/README.md and cross-link ADAPTIVE_LAYOUT.md
from the developer docs authors actually read (quick reference, API
reference, advanced dev, font manager, dev preview, plugin dev guide);
ADAPTIVE_LAYOUT.md gains adaptive-images, composite-layouts and
preserving-user-customization sections.

Compat: PluginLoader now logs one advisory warning (never raises) when a
plugin's manifest declares a min LEDMatrix version newer than the running
core, checking the min_ledmatrix_version / requires.* / versions[]
spellings found in the wild. Guarded against stale core version numbers.

src/__init__.py __version__ bumped 1.0.0 -> 3.1.0 to match the latest
release tag (v3.1.0) — it had never been updated and the compat check
needs a truthful number. NOTE: verify this matches the intended release
numbering before the next tag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add measure_font_crispness — verify a ladder rung isn't blurry

PIL antialiases TTF outlines by default; a 'pixel-style' font only
rasterizes without antialiasing at specific sizes (for PressStart2P:
exact multiples of its 8px design grid). A ladder rung at an unverified
size silently renders blurry on an LED panel — this exact bug shipped in
both text-display's and football-scoreboard's custom TTF ladders
(non-8-multiple PressStart2P sizes, and '5by7.regular'/'4x6-font' at
sizes that were never actually crisp).

measure_font_crispness(font, sample_text) renders the sample and reports
the fraction of ink-bbox pixels that are neither pure black nor pure
white. BDF fonts (real bitmaps) always score 0.0; TTF ladders should be
verified against this before shipping — see the new
TestFontFitting::test_ladder_arcade_is_crisp pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): add fit_text_proportional — proportional sizing vs. always-maximize

fit_text always picks the largest ladder rung that fits its box. That's
right when an element owns dedicated space, but wrong when several
independently-fitted elements need to stay visually harmonious as the
panel grows: a score's box might have generous room while a neighboring
logo scales by a fixed geometry factor via px() — fit_text lets the score
balloon out of proportion (even overlapping the logo) even though its
individual pick is technically correct.

fit_text_proportional(text, box, base_size_px, ladder) instead targets
base_size_px * self.scale (the same scale factor px() already uses),
picking the nearest ladder rung at or below that target, still capped to
what fits the box, floored at the smallest rung when the target is below
every rung. Refactored the shared largest-that-fits/ellipsize walk into
_walk_ladder() so fit_text and fit_text_proportional don't duplicate it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(layout): fit_text_proportional gains an axis-specific scale override

self.scale (min(width_ratio, height_ratio)) is the right conservative
default for anything whose aspect ratio matters, but a caller whose
surrounding composition already scales along a single axis — e.g.
football-scoreboard's logo_slot = min(height, width // 2), which tracks
height alone — needs text sized the same way, or it reads as
under-scaled next to logos that grew on a panel that only got taller
(128x32 -> 128x64: self.scale stays 1.0 since width didn't grow, but
logos still double).

fit_text_proportional(..., scale=None) now accepts an explicit override;
None keeps the existing self.scale default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(layout): scoreboard_regions reserves real center space at 2:1 aspect ratios

logo_slot = min(height, width // 2) has a blind spot: at exactly 2:1
aspect ratio (width == 2 * height -- a very common shape: two, four, or
more square modules stacked into a taller panel) width // 2 and height
are equal, so the two logo slots claim the ENTIRE width and leave zero
pixels for a center column, no matter how large the panel gets. Not a
'small panel' problem -- 96x48, 128x64, and 256x128 (all exactly 2:1) hit
it identically, while the 128x32 design baseline and panels like 192x48
or 256x32 never do, because height is already the tighter constraint
there.

Two new parameters fix it in the one shared helper every scoreboard-style
plugin composes through:

- min_center_fraction / min_center_design_px reserve at least
  max(width * fraction, design_px * ctx.scale) for the center column,
  capping logo_slot further when needed. The scaled design-px term
  matters on small panels where a flat fraction alone reserves too little
  absolute space.
- score_bleed_fraction extends the score's own fit box (not the logo
  slots themselves) a controlled amount into each side -- the same way
  real broadcast scoreboards let a big score number's edges cross into
  the team marks flanking it. Without this the reserve alone can still be
  too narrow for a short score to render without truncating.

score_area is now genuinely narrower than the full card width (previously
identical to status_band/detail_band, which still span the full width and
overlay the logos -- short text there was never the problem).

Verified against the full harness size spread: a real game score like
'17-21' never needs ellipsis at any tested 2:1-or-tighter aspect ratio
(test_score_never_needs_ellipsis_for_a_short_score), and wide panels
(128x32/192x48/256x32-style) are provably unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: document scoreboard_regions' center-reserve and score-bleed params

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: address CodeRabbit review on PR #393

- docs: scope the self.layout note to BasePlugin subclasses (others build
  a LayoutContext directly) and make explicit that adaptive layout is
  opt-in — classic rendering stays unless a plugin adopts the APIs.
- dev_server: broaden the render-request catch (a bad manifest.json now
  returns a clean 400 instead of an unhandled 500) and stop echoing raw
  exception text in the loader-failure responses — full tracebacks go to
  the dev server's console log instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): allowlist plugin_id before any path lookup

CodeQL (py/path-injection): plugin_id arrives in request input and flows
into filesystem paths via find_plugin_dir. Gate it with the same
^[a-zA-Z0-9_-]{1,64}$ allowlist the web UI's pages_v3 uses, at the
single choke point every route resolves through.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): lexical containment check on resolved plugin dirs

CodeQL doesn't recognize the interprocedural allowlist as a
path-injection barrier; add the canonical one — normalize (without
following symlinks, since dev plugins are commonly symlinked into
plugins/) and require the result to stay inside the search dir.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): inline normpath containment barrier before render

CodeQL doesn't credit the sanitization inside find_plugin_dir along
this flow; apply its documented barrier (normpath + startswith against
the allowed roots) inline in _parse_render_request, on the exact path
that reaches the render/load sinks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): derive plugin dir from trusted directory listings

CodeQL's barrier-guard recognition doesn't see a startswith check
inside an any() comprehension, so the normalize-and-prefix approach
still flagged. Break the taint outright instead: after lookup, re-derive
the directory by enumerating the search dirs (iterdir) and matching by
path equality — the Path used for all downstream file access is built
solely from trusted listings, never from request input.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

* fix(dev-server): use os.scandir for path-injection barrier, redact stack traces from render responses

CodeQL doesn't model Path.iterdir() as a taint-clearing enumeration the
way it does os.scandir() -- _trusted_plugin_dir's iterdir-based rebuild
still traced plugin_id through to the manifest.json open(). Switched to
scandir, matching the pattern already verified clean on PR #396.

Also stops surfacing raw exception text (update()/display() failures)
in the JSON render response -- logs full detail server-side via
exc_info instead, returning only the exception class name to the
client. And drops path values from three plugin_loader debug/error
logs that CodeQL flags as clear-text-logging of externally-influenced
data, keeping plugin_id (not flagged) for context.

* fix(dev-server): remove conditional-reassignment ambiguity in plugin_dir resolution

CodeQL's path-injection flow still traced through _parse_render_request
after the scandir fix -- the tainted find_plugin_dir() result and the
scandir-derived _trusted_plugin_dir() result shared the same variable
name (plugin_dir), reassigned only on the truthy branch. That merge
point apparently isn't treated as a barrier by the flow analysis, so it
kept tracing the pre-reassignment value through to the manifest open().

Split into two distinct names -- candidate_dir (tainted, used only to
call _trusted_plugin_dir) and trusted_dir (the only name used for any
downstream file access) -- so there's no reassigned variable for the
flow to walk through.

* fix: remove unused imports flagged by Codacy

Union in adaptive_images.py and field in adaptive_layout.py are both
imported but never used -- the last two Codacy findings on this PR,
matching the same fix already applied on PR #396.

* fix(layout): bound the fit cache; never alias the source image in fits

Two latent issues found in a self-review pass:

- LayoutContext._fit_cache was an unbounded dict (the image cache got an
  LRU cap, the text-fit cache didn't). Cache keys embed the fitted TEXT,
  so a plugin fitting changing strings — a live game clock, a ticker —
  on a 24/7 service grows it forever. Now LRU-bounded at 512 entries via
  the same pattern as the image cache.

- fit_image returned the caller's ORIGINAL image object when the source
  was already RGBA at target size (contain/fill_height, no ink crop).
  ImageFitResult is documented as an independent copy, and LayoutContext
  caches results — an aliased image lets later mutations of the source
  corrupt cached fits (or vice versa). Copy in that branch.

Both covered by new regression tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 10:38:52 -04:00
05e7c43b27 fix(plugins): replace dependency marker files with a real satisfaction check (#390)
* fix(plugins): replace dependency marker files with a real satisfaction check

The .dependencies_installed hash-marker system only tracked "was this exact
requirements.txt hashed before" — not whether the packages it names are
actually present. That made it fragile (a wiped venv, a manually removed
package, or a lost/corrupted marker forces a needless full pip reinstall or,
worse, a false skip) and produced dead weight for the ~10 plugins whose
requirements.txt is comment-only (they still paid a pip subprocess on first
boot before a marker existed).

Replace it with requirements_are_satisfied() in plugin_loader.py, which
checks each real requirement line against importlib.metadata directly, so
install_dependencies() only shells out to pip when something is actually
missing or version-mismatched. Drops the marker file entirely: removed all
marker read/write sites in plugin_loader.py and store_manager.py, the
now-pointless marker-cleanup step in the git-update path, the unused legacy
marker implementation in plugin_manager.py, and the already-stale
clear_dependency_markers.sh script.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(security): close path-injection gap in dependency-satisfaction checks

CodeQL flagged 2 new high-severity "uncontrolled data used in path
expression" alerts at the open() calls inside this PR's new
requirements_has_real_deps()/requirements_are_satisfied() -- both are
reachable from paths that were never run through the basename+trusted-base
sanitiser this codebase already uses elsewhere:

- PluginLoader.install_dependencies() only applied that sanitiser when its
  optional plugins_dir argument was actually passed; the "no plugins_dir"
  branch trusted plugin_dir_real directly. Made plugins_dir required (not
  Optional) so that branch can't exist, and added an explicit guard in
  load_plugin() so install_deps=True without a plugins_dir fails loudly
  instead of silently. Production's only real caller (PluginManager) always
  passes plugins_dir already; the harness/dev-server/render-plugin callers
  all use install_deps=False and are unaffected.

- StoreManager._install_dependencies() never sanitised plugin_path at all,
  and its call sites ultimately derive that path from a plugin's own
  manifest.json "id" field (install_plugin_from_url) -- a malicious plugin
  could otherwise point requirements_file outside plugins_dir. Applied the
  same os.path.basename()-based containment pattern PluginLoader already
  uses (and that CodeQL recognises as a real sanitiser).

Added test_install_dependencies_requires_plugins_dir and
test_install_dependencies_rejects_path_outside_plugins_dir to lock in the
actual security property, not just quiet the scanner. Verified: all 20
tests in test_plugin_loader.py pass, plus the PR's existing test plan
(test_plugin_system.py, test_store_manager_caches.py: 53 passed) and the
full CI plugin-safety suite (test_harness.py, test_visual_rendering.py,
test_plugin_matrix.py: 52 passed, 2 pre-existing skips) all still pass.

* fix(security): replace basename-only sanitiser with a trusted-enumeration check

The previous commit's os.path.basename() + os.path.join() pattern (which a
pre-existing code comment claimed CodeQL recognises as a sanitiser) did not
actually clear the alert -- the next CodeQL run still flagged the same 2
sink lines, plus a new one at the os.path.join() call itself. Taking a
substring of tainted data apparently isn't treated as a barrier by this
query, whatever the comment assumed.

Replaced it with find_trusted_subdir(): enumerate the trusted plugins_dir
via os.scandir() and only use a name that scandir itself produced, matched
by equality against the caller's requested name. The path is then built
from that enumerated entry, not from the caller's string -- a value
sourced from iterating a trusted, non-tainted directory carries no taint
regardless of what it happens to equal, which is a stronger and more
conventional allowlist-style barrier than string-stripping. Applied
identically in both PluginLoader.install_dependencies() and
StoreManager._install_dependencies(), sharing one implementation.

Re-verified: all 65 tests across test_plugin_loader.py (20, including the
2 new security regression tests), test_store_manager_caches.py (35),
test_plugin_system.py (10) pass, plus the full CI plugin-safety suite
(test_harness.py/test_visual_rendering.py/test_plugin_matrix.py: 52
passed, 2 pre-existing skips).

* fix(security): redact URL credentials from pip subprocess output before logging

CodeQL flagged 3 clear-text-logging-of-secrets alerts in
install_requirements_file() (src/common/permission_utils.py:353,360,371).
Pre-existing on main, unrelated to this PR's own diff, but now visible
since the path-injection alerts that previously took priority in the
annotation list are fixed.

The underlying risk is real: pip can echo a private index URL's embedded
basic-auth credentials (from a requirements.txt --index-url line or
PIP_INDEX_URL) back verbatim in its own stderr/stdout on failure, and this
function both logs that output directly and returns it to callers --
store_manager.py's _install_dependencies() logs result.stderr from this
same function too.

Added _redact_url_credentials(), applied immediately after each of the two
subprocess.run() calls (mutating result.stderr/stdout in place) rather
than patching each log call site individually. This closes the leak at
the source: every downstream use -- the three flagged log lines, the
"note" string embedded in the returned stdout, and store_manager.py's own
logging of the returned result -- gets the redacted text for free.

Verified the fixed-phrase "denied" check (`"a password is required" in
result.stderr`) is unaffected, since URL syntax and those phrases don't
overlap -- covered explicitly by
test_does_not_touch_denied_check_phrases. Added
test/test_permission_utils.py (6 tests) covering the redaction helper
directly and both subprocess.run() call sites (the sudo-wrapper branch,
which this repo's scripts/fix_perms/safe_pip_install.sh makes live, and
the no-wrapper fallback branch). All pass.

* fix(security): stop interpolating req_file/pip-output into log calls

The previous commit's redaction (mutating result.stderr/stdout right after
each subprocess.run()) didn't clear CodeQL's clear-text-logging alerts --
same lesson as the path-injection fix earlier in this PR: a static
analyzer can't tell "this value was already sanitised two lines up" from
"this is still the raw tainted value" just by looking at a single log
call in isolation, so it conservatively keeps flagging it regardless of
what the redaction function actually does.

Removed all dynamic interpolation (req_file, result.stderr) from the 3
flagged logger.warning() calls entirely, replacing them with fixed
messages plus (for the one that had it) result.returncode, which is a
plain int with no possible taint. The full redacted detail is still
available where it actually matters -- in the returned
CompletedProcess.stderr/stdout and the "note" text -- just not duplicated
into a log line a scanner has to reason about in isolation.

Re-verified: all 6 test_permission_utils.py tests still pass (they assert
on the returned result, not log call arguments), plus the full
test_plugin_loader.py/test_store_manager_caches.py/test_plugin_system.py
suite (71 passed, 1 pre-existing deselect, 4 subtests).

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 08:55:50 -04:00
ChuckandGitHub 2ffc57cf40 fix(plugin-harness): add no-op process_deferred_updates to test double (#391)
The safety harness's VisualTestDisplayManager (base of
BoundsCheckingDisplayManager) doesn't implement process_deferred_updates(),
which 5 first-party ledmatrix-plugins call unconditionally between
set_scrolling_state() and their scroll-position update: news, odds-ticker,
ledmatrix-leaderboard, stock-news, and ledmatrix-stocks. Any of them fails
the harness with AttributeError the moment it's touched (surfaced when
ledmatrix-plugins#177 had to add a local hasattr guard in ledmatrix-stocks
just to pass CI). Add the method as a no-op, mirroring the existing
"no-op for testing" pattern already used for set_scrolling_state, so these
plugins render under the harness without every touching PR needing its own
guard.
2026-07-11 08:55:03 -04:00
ChuckandGitHub aab0e9ade0 fix(plugin-manager): fix TypeError breaking every plugin's scheduled update (#392)
run_scheduled_updates()'s resource-monitor branch wrapped the update call
in a closure stored as a *class* attribute on a dynamically-built type
(type('obj', (object,), {'update': monitored_update})()). The descriptor
protocol turns a function found via class-attribute lookup into a bound
method on instance access, silently prepending the synthetic instance as
an implicit first argument -- but monitored_update() takes none, so every
call raised "monitored_update() takes 0 positional arguments but 1 was
given", was caught by run_scheduled_updates' try/except, and recorded as
an update failure.

self.resource_monitor is None by default and was dormant until PR #388
("activate dormant plugin health/metrics subsystem") wired it up in both
display_controller.py and web_interface/app.py -- meaning this bug went
live in every real deployment as of that merge (2026-07-09) despite the
buggy line itself dating back to 2025-12-27. In practice this means no
plugin's update() has succeeded since upgrading past #388: circuit
breakers cycle through half-open -> immediate failure -> reopened every
health-check interval forever, and all plugin data (scores, odds, prices,
etc.) goes stale from whatever was last fetched before the upgrade.
Confirmed live on a running instance: odds-ticker (and stock-news,
ledmatrix-stocks, baseball-scoreboard, ledmatrix-leaderboard, of-the-day)
failing this exact way every 5-minute circuit-breaker retry.

Fixed by using types.SimpleNamespace(update=monitored_update) instead of
a dynamic class: SimpleNamespace stores attributes on the instance
itself, so attribute lookup returns the plain function unchanged --
never routed through the class-attribute descriptor protocol that
injects an implicit self.

Added test_run_scheduled_updates_calls_update_with_resource_monitor to
test/test_plugin_system.py using a real PluginResourceMonitor (not a
mock of it), so the test exercises the actual descriptor-binding
behavior that caused this. Verified the test fails with the exact
reported error against the pre-fix code and passes against the fix.
2026-07-11 08:54:30 -04:00
978a03b42d Add settings tooltips and search to the web UI (#387)
* Add settings tooltips and search to the web UI

Help users quickly find settings and understand how each one works.

Tooltips: a new delegated controller (static/v3/js/tooltips.js) drives an
accessible (i) info tooltip that appears on hover, keyboard focus, and tap.
A shared `help_tip` Jinja macro (partials/_macros.html) emits the trigger;
the plugin config macro and the core settings partials now surface help
text through it. Per the design, the always-visible field help paragraphs
are folded into the tooltip to declutter the forms, and the hardware/display
settings carry authored detail (default, range, recommendation).

Search: a global header search box finds settings across every settings tab
— even ones not yet opened — via a lazy client-side index built by scanning
the same field markup (static/v3/js/settings-search.js). Selecting a result
switches tabs, waits for the field to load, then scrolls to and flashes it.
A per-tab filter box hides non-matching fields on the current tab.

Plugin settings get tooltips for free by reusing each field's schema
`description`; every settings field also gets a stable `setting-<tab>-<key>`
anchor id for search navigation.

Styling uses the existing --color-* theme vars so light/dark mode both work,
and honors prefers-reduced-motion. Adds Flask render smoke tests that assert
each settings partial ships tooltips, anchors, and a filter box.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Address Codacy static-analysis findings in settings JS

Refactor the two new modules to clear the flagged patterns without any
behavior change:

- Build the search dropdown with DOM nodes + textContent instead of
  innerHTML string concatenation, removing the XSS sinks and the manual
  escapeHtml helper it needed.
- Replace numeric index access (index[i], terms[j], opts[idx],
  currentResults[i]) with array iteration methods, NodeList.item(), and
  Array.prototype.at() to clear detect-object-injection.
- Use === via a shared termsMatch() helper, optional-catch binding, and
  drop a useless initial assignment.

Verified with ESLint (eslint:recommended + eslint-plugin-security) at zero
findings and re-ran the headless-Chromium behavior test (tooltip, per-tab
filter, global search navigation) — all green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Reduce complexity of revealAncestors in settings search

Extract isNodeHidden() and revealNode() helpers so revealAncestors drops
below the cyclomatic-complexity threshold. No behavior change; verified with
the headless-Chromium test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Resolve remaining Codacy findings in settings search

- Validate plugin ids against a strict allowlist (mirroring the server's
  _SAFE_PLUGIN_ID_RE) before they can appear in a fetch path, so the request
  URL is never built from unvalidated input (Codacy: user-controlled URL).
- Document that the fetched HTML is parsed into an inert document (scripts
  never run, never inserted into the live DOM) purely to read field text for
  the search index.
- Declare block-scoped locals with const instead of var where they were
  nested inside conditionals (Codacy: var not at function root).

No behavior change; re-verified with ESLint and the headless-Chromium test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Serve the settings search index from the server as JSON

Move index building off the client so there is no client-side HTML fetching
or DOM parsing (resolves Codacy's variable-fetch and DOMParser flags on the
read-only, same-origin index build).

- Add GET /v3/settings/search-index (pages_v3.py): renders the settings
  partials server-side and extracts each field's anchor id, key, label,
  tooltip, and section with a small stdlib HTMLParser, then caches the result
  keyed on the installed-plugin set. Parsing the rendered HTML keeps anchor
  ids identical to the live DOM, so the index cannot drift.
- settings-search.js: buildIndex() now does a single fetch of the literal
  endpoint + .json(); removed the per-partial fetch loop, DOMParser, scanDoc,
  CORE_TABS, and the plugin-id allowlist. Search, keyboard nav, navigation,
  and the per-tab filter are unchanged.

Net: fewer requests and no client-side HTML parsing. Verified with a new
endpoint test in test_web_settings_ui.py (12 pass) and the headless-Chromium
test (tooltip, filter, search navigate + flash all green).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Address CodeRabbit review on settings search/tooltips

- pages_v3: include Durations tab in the search index so
  setting-durations-* fields are actually indexed
- test_web_settings_ui: assert setting-durations-clock is present in
  the search-index endpoint response
- settings-search.js: on index fetch failure, reset buildPromise
  instead of caching an empty (truthy) index so search can retry
- settings-search.js: filterScope returns null (not document) when no
  tab container matches, and the caller guards, so the per-tab filter
  can't hide fields across unrelated tabs
- settings-search.js: refresh the stale header comment to describe the
  server-side JSON index flow
- app.css: cap #settings-search-results height with overflow-y so the
  dropdown scrolls instead of overflowing small screens

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Fix stuck search dropdown; add plugin-tab nested-settings filter

Global search:
- Close the results dropdown on input blur (guarded, with a short delay)
  so it reliably dismisses when focus leaves — previously it could linger
  because the only outside-close was a document click that Alpine/HTMX
  handlers can swallow.
- Clear the query text after navigating to a result so refocusing the box
  doesn't re-open stale results.
- Also dismiss on htmx:afterSwap (tab changes / navigation).

Per-tab filter (now on plugin tabs too):
- Render the shared settings_filter box in the plugin Configuration panel.
  It auto-wires: the delegated input handler and filterScope already target
  .plugin-config-tab.
- Teach applyTabFilter to reveal matches inside collapsed nested sections
  (render_nested_section defaults them shut), hide nested-section wrappers
  with no matches, and restore the original collapsed layout when cleared
  (only re-collapsing sections the filter itself opened).
- Count a visible nested-section as content for its parent heading so the
  heading isn't hidden while a subsection below still has matches.

Adds a plugin-config render test (filter box + nested anchors + tooltips).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Close search dropdown via capture-phase outside-click

The dropdown could stay open after clicking away because the only
outside-click close was a bubble-phase document listener. The v3 UI is
one Alpine app() component full of HTMX/Alpine/widget click handlers;
when a click lands inside an element that calls stopPropagation(), the
event never bubbles to document and the close never runs.

- Replace the bubble-phase document 'click' close with a capture-phase
  'pointerdown' listener scoped to #settings-search-wrap. Capture runs
  before any bubbling stopPropagation can swallow the event, so it always
  fires; pointerdown also covers touch on the Pi screen. Clicking a result
  stays inside the wrap, so selection is unaffected.
- Guard the debounced input handler so a delayed render can't re-open the
  box after focus has left (type-then-click-away race).

Keeps the existing blur / Escape / htmx:afterSwap closes as secondary paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz

* Fix settings search dropdown not visually closing

.hidden has no effect in this app: app.css is a hand-picked utility
subset (no Tailwind build step) and never defines .hidden { display:
none }. openResults()/closeResults() only toggled the class, so the
dropdown stayed rendered (display: block) even once closeResults()
ran - confirmed via computed style in a headless browser. Set
style.display directly, matching the fallback already used by
revealNode()/collapseNode() elsewhere in this file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-09 09:22:24 -04:00
bd9f461f70 Add system diagnostics, power controls, and WiFi radio toggle to Tools tab (#389)
* Add system diagnostics, power controls, and WiFi radio toggle to Tools tab

Expands the web UI Tools tab with safe, purpose-built controls so users can
manage the Pi without SSHing in, instead of an arbitrary-command terminal
(the web UI has no auth and CSRF is disabled, so a shell would be unsafe).

- System Diagnostics card: renders the existing but previously-unused
  GET /api/v3/system/status endpoint (CPU, memory, temp, disk, uptime),
  with a manual refresh and a 10s poll.
- System Power section: reboot/shutdown buttons wired to the existing
  reboot_system / shutdown_system actions, behind a confirm step, with a
  dedicated powerAction() helper that treats the dropped connection as the
  expected "going offline" outcome rather than an error.
- Network Radio section: WiFi on/off toggle backed by new
  GET/POST /api/v3/wifi/radio endpoints and WiFiManager.set_wifi_radio() /
  get_wifi_radio_state(). Disabling WiFi is refused unless a wired
  connection is present (reusing the existing lockout guards), with an
  explicit force-off confirmation for advanced users.

No new privileged commands: uses nmcli radio wifi on|off (already
sudo-allowlisted) and the existing reboot/poweroff grants, so the
sudoers-alignment guard test stays green. Bluetooth toggle intentionally
omitted since the installer removes the BlueZ stack for LED timing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE

* Harden WiFi radio endpoint and diagnostics poll (review feedback)

Addresses code review feedback on the Tools tab additions:

- api_v3.py: parse the `enabled` POST field with the same string-aware
  coercion as `force`. A plain bool() cast turned {"enabled":"false"} into
  True (enabling instead of disabling) for any non-UI API caller.
- wifi_manager.set_wifi_radio() now returns a reason code alongside
  (success, message); the /wifi/radio error response includes it. The Tools
  UI only shows the force-off confirmation when reason == 'no_ethernet', so a
  genuine nmcli failure surfaces its real error instead of a misleading
  "no wired connection" prompt that would just retry into the same failure.
- tools.html: gate the 10s diagnostics poll on panel visibility
  (document.hidden / offsetParent), so switching to another tab stops the
  recurring /api/v3/system/status calls instead of churning the Pi off-screen.
  The initial load and manual Refresh remain unconditional.

Verified: {"enabled":"false"} now disables (refused w/ reason:no_ethernet),
{"enabled":"true"} enables; inline JS passes node --check; py_compile clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE

* Narrow WiFi-disable fallback except and add stack trace (review)

Addresses a CodeRabbit nitpick: the fallback handler in set_wifi_radio()'s
disable path caught bare Exception and logged without a traceback. Narrow it to
(OSError, subprocess.SubprocessError) — the errors subprocess.run realistically
raises — and log with exc_info=True for full context on the Pi. Anything
genuinely unexpected now propagates to the endpoint's outer handler (500),
matching the codebase's specific-exception convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-09 09:22:11 -04:00
3b93024993 feat: activate dormant plugin health/metrics subsystem and surface it in the web UI (#388)
* feat(plugin-system): activate dormant plugin health & metrics subsystem

PluginManager shipped a fully-built health tracker, resource monitor and
circuit breaker that were never instantiated (health_tracker/resource_monitor
were left as None), so the circuit breaker never engaged and the existing
health/metrics API routes always returned "not available".

- DisplayController now wires a PluginHealthTracker and PluginResourceMonitor
  onto the plugin manager, enabling the circuit breaker (a repeatedly-failing
  plugin's update() is skipped after consecutive failures, then retried after
  a cooldown) and per-plugin execution-time metrics. Both persist to the
  shared cache.
- load_plugin() now validates each plugin's config against its JSON schema in
  a strictly warn/degrade-only way: a violation logs a warning and flags the
  plugin degraded in the health tracker, but never changes whether the plugin
  loads or its pass/fail behaviour. Adds PluginHealthTracker.set_degraded(),
  which never touches the circuit breaker.
- ResourceMonitor CPU/memory sampling now reuses a cached psutil.Process and
  reads cpu_percent(interval=None), so monitoring no longer blocks ~100ms per
  call on the display loop's update path.
- Fix DiskCache.get() raising TypeError for max_age=None ("never expires"),
  which silently discarded persisted plugin health/metrics on read and thus
  broke cross-process and post-restart surfacing.
- Fix two dead PluginManager helpers that called non-existent tracker methods.

Tests: new test_resource_monitor, test_plugin_health,
test_plugin_manager_schema_soft; extended test_cache_manager and
test_display_controller.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq

* feat(web-ui): surface plugin health, metrics and load state

With the health/metrics subsystem now active in the display service, expose it
in the web UI (which runs as a separate process from the display loop):

- Wire a health tracker / resource monitor backed by the shared on-disk cache
  into the web process so /api/v3/plugins/health and /plugins/metrics read the
  data the display service persists.
- Build those route responses per installed plugin id (the tracker's in-memory
  view is empty in a fresh web process) so cross-process data is included.
- Add state + error_info to /plugins/installed entries so the UI can show why a
  plugin isn't running instead of just loaded:false.
- Add a "Plugin Health" panel to the Tools page (circuit status, avg/max update
  time, update count, last error) plus PluginAPI.getPluginMetrics().

Tests: route-level tests for the health/metrics endpoints in test_web_api.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq

* fix(plugin-metrics): refresh cross-process health/metrics reads; type hints

Addresses CodeRabbit review on #388:

- Major: the web process's health/resource trackers cached the first persisted
  read in an in-memory dict (and the CacheManager memory tier held max_age=None
  entries indefinitely), so a long-lived web process showed the first snapshot
  and never reflected the display service's later updates. Add an opt-in
  force_reload path (get_health_summary/get_health_state/_load_health_state and
  get_metrics_summary/get_metrics) that bypasses the in-memory copy and, via a
  new memory_ttl passthrough on CacheManager.get, the cache manager's memory
  tier — so each /plugins/health and /plugins/metrics poll reads fresh persisted
  state. Default behaviour (force_reload=False) is unchanged for the display
  process and existing callers.
- Minor: DiskCache.get type hint is now Optional[int] with the None ("never
  expires") semantics documented, matching MemoryCache.get.

Tests: new force_reload staleness cases in test_plugin_health and
test_resource_monitor.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-09 07:54:18 -04:00
85d321cf33 fix: plugin_loader retries with --ignore-installed on apt/pip RECORD conflicts (#386)
* fix: plugin_loader retries with --ignore-installed before assuming apt package satisfies pin

install_dependencies treated any "uninstall-no-record-file" pip failure as
"dependency satisfied" and wrote the success marker without ever attempting
--ignore-installed, unlike install_dependencies_apt.py and
safe_pip_install.sh (added in #385 for the Plugin Store/first-time-install
paths). A plugin pinning a newer version of a system-managed package (e.g.
requests) would silently keep running against whatever version apt shipped,
while the marker file claimed the pinned requirement was met.

Now retries the same install with --ignore-installed on that specific
failure so pip actually lays the pinned version down (shadowing the
system-managed copy) before falling back to the prior tolerant behavior if
the retry itself fails too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* chore: suppress Codacy finding on new retry subprocess.run call

Same generic Bandit/semgrep pattern-match on non-literal subprocess.run
argv flagged in #385's install_requirements_file, now on the new
--ignore-installed retry call added here: list-form argv (no shell=True),
sys.executable is this process's own interpreter, and requirements_file is
built internally by find_plugin_directory, never raw external input.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: tolerate a timed-out --ignore-installed retry, dedupe marker-write logic

CodeRabbit review caught a real inconsistency: if the --ignore-installed
retry itself timed out, subprocess.TimeoutExpired propagated to the outer
handler and returned False, failing plugin load — contradicting the
intended "tolerate this specific apt/pip conflict" behavior, where a mere
non-zero retry return code already returns True. Wraps the retry in its own
try/except so a timeout is logged and tolerated the same way as any other
retry failure.

Also extracts the marker-writing logic (open/write/chmod, ignoring OSError)
into _write_dependency_marker, since it was duplicated identically between
the direct-success path and the apt-conflict-retry-fallback path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: revert marker-write dedup helper, resolves CodeQL path-injection alert

CodeQL flagged _write_dependency_marker's open(marker_file, ...) as
"uncontrolled data used in path expression" (high severity) once the
marker-write logic was extracted into its own method. marker_file is
actually safe — it's built from safe_plugin_dir, which install_dependencies
sanitizes via os.path.basename() (CodeQL's own recognized py/path-injection
sanitizer, per the existing comment a few lines above) — but CodeQL's
interprocedural analysis doesn't carry that sanitized status across the new
method boundary, since the sanitizer call and the open() sink were no
longer in the same function.

This exact code produced zero CodeQL findings before the extraction (in two
duplicated inline blocks) and is unchanged in what data reaches it — only
its location moved. Reverting the extraction (keeping CodeRabbit's other,
independent timeout-handling fix) restores the previously-clean shape rather
than trying to convince the analyzer's cross-function taint tracking that a
refactor changed nothing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 11:19:37 -04:00
63a233f3ed fix: dependency installation gaps in Plugin Store and first-time install (#385)
* fix: install plugin dependencies through root-visible installer in Plugin Store

install_plugin/update_plugin (store_manager.py) installed requirements.txt
with a bare `pip3` off PATH, bypassing the root-visible installer added in
#380 for the "Reinstall Plugin Deps" button. Two bugs stacked: (1) `pip3`
can resolve to a different Python install than the one that actually runs
ledmatrix.service, and (2) even when it resolves correctly, ledmatrix-web
runs as a non-root user so the package lands in that user's local
site-packages, invisible to root-run ledmatrix.service. Either way the
install reports success and writes the .dependencies_installed hash marker,
so plugin_loader's own (correct) install-on-load path skips reinstalling —
leaving the dependency permanently missing until a user finds and clicks
the separate "Reinstall Plugin Deps" tool. This is why users kept hitting
"No module named 'astral'" for the weather plugin even after installing it
from the Store.

Extracts the sudo-wrapper-then-fallback install logic from api_v3.py's
_pip_install_requirements into src/common/permission_utils.py as
install_requirements_file, and routes store_manager.py's dependency
installation through it so the automatic Store install/update path now
matches the manual "Reinstall Plugin Deps" path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* chore: suppress Codacy false-positive on subprocess.run in install_requirements_file

Codacy's generic subprocess-security rule (Bandit B603 equivalent) flagged
the pip/sudo subprocess.run calls in install_requirements_file for lacking a
"static string argument" — the standard pattern-based flag for any
subprocess.run() call with a variable in its argv list. Both calls use
list-form argv (no shell=True, so no shell-injection surface), and the only
dynamic value is req_file, a Path built internally by callers rather than
raw external input; safe_pip_install.sh independently re-validates it before
installing anything as root. Suppresses with inline `# nosec B603` comments
matching this codebase's existing convention (see permission_utils.py's own
PROTECTED_SYSTEM_DIRECTORIES, display_manager.py, sync_manager.py, etc.).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: first-time install script fails on apt-managed requests package

web_interface/requirements.txt and requirements.txt both pin
requests>=2.33.0,<3.0.0, but Raspberry Pi OS ships an apt-managed
python3-requests with no pip RECORD file. Upgrading it via plain
`pip install` aborts with "uninstall-no-record-file" because pip refuses to
uninstall a package it has no record of, in place — which is exactly the
"Some web interface dependencies failed to install" warning first-time
install hits.

scripts/install_dependencies_apt.py and scripts/fix_perms/safe_pip_install.sh
already work around this with --ignore-installed (lets pip lay the new
version down in /usr/local, shadowing the apt copy, instead of trying to
remove it first). first_time_install.sh's own direct pip invocations —
the per-package requirements.txt loop, the web_interface/requirements.txt
install, and the requirements_web_v2.txt fallback — didn't have it. Adds
--ignore-installed to all three so first-time install no longer fails on
this well-known apt/pip conflict.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* chore: add nosemgrep to subprocess.run calls Codacy still flagged

The prior # nosec B603 comments suppressed Bandit's check but Codacy's
semgrep-based rule ("subprocess function 'run' without a static string")
kept flagging the same two lines as a critical security issue even after
that fix landed. install_dependencies_apt.py's _run() already needed both
tags together (# nosec B603 B607 ... # nosemgrep) for the identical
subprocess.run pattern, so apply the same double suppression here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

* fix: add --ignore-installed to install_requirements_file fallback path

CodeRabbit review caught this (confirming a gap already flagged in
conversation): the non-sudo fallback pip install in install_requirements_file
was missing --ignore-installed, unlike the sudo-wrapper branch and
safe_pip_install.sh. Without it, the same apt/pip RECORD-file conflict this
PR fixes elsewhere (first_time_install.sh, install_dependencies_apt.py) could
still hit installs that fall back to this path (e.g. a plugin's
requirements.txt on a host where safe_pip_install.sh isn't set up yet).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 09:59:12 -04:00
7a9d01342a Fix inconsistent Vegas scroll transition gaps caused by plugin-baked padding (#384)
* Strip plugin-baked scroll padding when capturing content for Vegas mode

Plugins that build their own ticker image via ScrollHelper.create_scrolling_image()
(or that manually pad both ends for a clean standalone loop) carry a solid-black
margin up to display_width wide on one or both edges. Vegas mode already adds its
own configurable gap around every item, so leaving that margin in place stacked an
extra, uncontrolled blank stretch on top of separator_width for whichever plugin
took the ScrollHelper-capture path — producing inconsistent transition gaps between
modules compared to plugins that provide content natively via get_vegas_content().

_get_scroll_helper_content() now detects and crops any such margin before handing
the image to the Vegas render pipeline, so every plugin's gap is governed solely by
vegas_scroll.separator_width regardless of which capture path produced its content.

* Address CodeRabbit nitpick: warn on double-edge padding crop, add unit tests

Logging a double-edge match at warning level (vs. info for a single edge)
makes it easy to spot an unexpected crop in the field, since two edges
matching at once is a much stronger signal of genuine baked-in padding than
one edge coinciding with real all-black content.

Also adds test/test_vegas_plugin_adapter.py covering _strip_scroll_padding's
branch logic: leading-only, trailing-only, both-edges, no-match, degenerate
all-black, missing/undersized display_width, and the info-vs-warning log level.

* Add type hints and docstring to test _solid helper (CodeRabbit nitpick)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-08 09:43:13 -04:00
9b2f02681d feat(web-ui): detect and surface Raspberry Pi under-voltage/throttling (#383)
* feat(web-ui): detect and surface Raspberry Pi under-voltage/throttling

Adds a vcgencmd get_throttled check to the system-status SSE stream and
surfaces it in the web UI:

- A header badge (next to CPU/Memory/Temp) that stays hidden when healthy,
  turns red when under-voltage/throttling is happening right now, and
  yellow if it happened earlier this session but has since cleared.
- A dismissible top banner (same pattern as the update-available banner)
  that appears while under-voltage/throttling is actively occurring, with
  guidance to check the power supply. Re-appears on a fresh occurrence
  even if a previous one was dismissed.
- A "Power Supply" card on the Overview tab alongside CPU/Memory/Temp/
  Display Status.

Motivated by a real device showing intermittent brightness flicker that
turned out to be ~1 under-voltage event every 30-90s (visible live via
`vcgencmd get_throttled` and dmesg's "Undervoltage detected!" messages) --
there was no way to see this from the web UI, only by SSHing in.

Returns None on non-Pi platforms (no vcgencmd on PATH), matching the
existing guard pattern used for the CPU temperature read.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* feat(web-ui): add power supply diagnostics detail to Tools tab

The header badge/banner/Overview card added in the previous commit only
show a collapsed "is it bad right now" signal. This adds a "Power Supply"
section to the Tools tab with the full 8-flag breakdown (under-voltage,
throttled, freq-capped, soft-temp-limit -- each split into "right now" vs
"occurred since boot") for actually troubleshooting a recurring issue,
plus a pointer to the README's power supply sizing guidance when something
is or was flagged.

Reuses the existing stats SSE stream (window.statsSource) rather than
adding a new endpoint -- the same payload already drives the header/
banner/Overview card, so this just listens for it too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web-ui): address review findings on power supply monitoring

- Overview card left its "--" placeholder forever on non-Pi platforms since
  the update only handled a truthy data.power. Now explicitly renders "Not
  available" with a neutral icon/color for that case.
- vcgencmd failures logged at debug, invisible in default remote log
  output. Bumped to warning to match the nearby systemctl failure logging.
- The banner/badge/card and the Tools summary line only looked at
  under_voltage_now/throttled_now (+ occurred), silently ignoring
  freq_capped_now/occurred and soft_temp_limit_now/occurred from
  _get_power_status() -- a Pi that's actively soft-thermal-limited or
  frequency-capped showed a green "OK" everywhere except the detailed
  flag table buried in Tools. All four surfaces now fold all four "now"/
  "occurred" flags into the same active/occurred state.
- The banner text was hardcoded to "Under-voltage detected..." even when
  the actual active condition was throttling/freq-capping/thermal limiting.
  Added _activePowerConditionLabels() (shared, non-module global scope) to
  build the banner/tooltip text from whichever flags are actually set.

Skipped: TTL-caching _get_power_status() to avoid "multiplying forks
across browser tabs" -- that premise doesn't hold against this codebase.
_StreamBroadcaster (its own docstring says as much) already runs exactly
one shared generator per tick regardless of client count, identical to
the uncached cpu_temp file-read two lines above it; there's nothing to
multiply.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(web-ui): address second round of review findings on power monitoring

- updatePowerStatus's falsy-power branch only hid power-stat, leaving
  power-warning-banner visible with stale text if _get_power_status()
  fails transiently. Now hides the banner too and resets the dismissed
  flag, same as the "not active" branch.
- The Tools tab's status badge (and the pre-existing dirty/clean badge
  right next to it) build class names like bg-${color}-100/text-${color}-800
  at runtime. This project hand-rolls its own Tailwind-named utility
  classes in app.css rather than running a real Tailwind build, and the
  light-mode base rules for bg-red-100/bg-yellow-100/bg-green-100/
  text-red-800/text-yellow-800/text-green-800 were simply never defined --
  only some had dark-mode overrides, which are no-ops without a base rule
  in light mode. Added the missing light-mode bases plus the two missing
  dark-mode overrides (bg-yellow-100/bg-green-100), fixing both badges.
- The "occurred earlier" tooltip was a hardcoded string regardless of
  which flag(s) actually fired. Generalized _activePowerConditionLabels()
  to take a suffix ('_now' or '_occurred') and reused it for both tooltips.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* refactor(web-ui): move Power Supply status off the Overview tab

The Overview tab's "Power Supply" stat card duplicated what the Tools
tab's diagnostics section already shows (summary badge + full flag
breakdown), so drop the card and its now-dead JS rather than keep two
copies in sync. The header badge and warning banner (visible on every
page) are unaffected -- only the Overview-tab card is removed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 09:35:55 -04:00
7a6bad29fe feat(display-controller): hot-reload plugin enable/disable without a restart (#374)
* feat(display-controller): hot-reload plugin enable/disable without a restart

Enabling or disabling a plugin in config previously required restarting the
display service: the plugin list and available_modes were built once at init
and the run loop never revisited them. (Per-plugin config *values* already
hot-reloaded; only the enabled set was restart-only.)

Now the controller reconciles its running plugins against the config's enabled
set whenever that set changes:

- The ConfigService watcher thread only sets a `_pending_plugin_reconcile`
  flag (via a cheap enabled-set diff). It never mutates loop state.
- The run loop applies the reconcile on its own thread (top of each
  iteration, deferred while on-demand is active), so loading/unloading and
  rebuilding available_modes can't race with rendering.
- `_reconcile_enabled_plugins` diffs desired vs running plugins, unloads the
  removed ones (cleanup + on_disable + config-unsubscribe via the new
  `_unregister_plugin`) and loads the added ones, then clamps the rotation
  index so the current mode stays valid.

The per-plugin registration done at startup is extracted into
`_register_loaded_plugin` and reused by the live-enable path so both build
identical state. Extracting it also fixes a latent late-binding bug: the
per-plugin config-change callbacks were closures over the loop variable, so
every plugin's callback targeted the last-loaded instance; each now binds its
own id/instance.

Adds test/test_display_controller_plugin_toggle.py covering live enable,
live disable, index clamping, no-op when unchanged, and the enabled-set diff.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(display-controller): don't exit on empty available_modes, guard rotation modulo

Hot-reload means available_modes can legitimately be empty at startup (no
plugins enabled yet) and become non-empty later via the web UI, or vice
versa mid-run. Fix four issues found reviewing this PR:

- run() exited the process entirely when available_modes was empty at
  startup instead of idling, permanently defeating the point of live
  enable/disable for anyone who starts with zero plugins enabled.
- The mode-rotation step divided by len(available_modes) unconditionally,
  raising ZeroDivisionError if the last enabled plugin is disabled between
  frames.
- _reconcile_enabled_plugins() called .get('enabled', False) on a config
  section without checking it was a dict first, raising AttributeError on
  a malformed config value.
- Minor: pop the config-change callback only after attempting to
  unsubscribe it, and log the exception in the config-read fallback
  instead of swallowing it silently.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix(display-controller): address review findings on the hot-reload PR

- Idle-wait tick was a fixed 30s sleep, delaying pickup of a plugin
  enabled via the web UI while no modes were active. Shortened to ~1s so
  it's roughly as responsive as the per-frame check once modes exist.
- _unregister_plugin popped the config-change callback from
  _plugin_config_callbacks even when config_service.unsubscribe() raised,
  losing the only reference to it. Now only pops on a successful
  unsubscribe.
- _pending_plugin_reconcile was cleared before _reconcile_enabled_plugins()
  ran, so a retryable failure (e.g. plugin discovery erroring) silently
  dropped the enable/disable request. _reconcile_enabled_plugins() now
  returns True/False and the caller only clears the flag on True.
- Added a warning log for the malformed-config case (a plugin's config
  section present but not a dict) so it's actually visible, and updated
  the existing test to assert it via caplog.

Left the broad `except Exception` around config_service.unsubscribe() as
Exception -- the current implementation is a simple lock+dict/list op that
doesn't document or realistically raise a narrower type, so this is a
defensive catch-all, not user error handling.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: ChuckBuilds <charlesmynard@gmail.com>
2026-07-06 17:37:01 -04:00
ChuckandGitHub bea00448d3 update mlb logos (#382)
update mlb logos

Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
2026-07-06 08:55:43 -04:00
deaa3d7a98 fix: array-table boolean columns always saved as unchecked (#381)
Reported bug: countdown plugin always shows "No Active" even with a
countdown configured and enabled — because every save silently unchecked
it. Root cause: the boolean column's hidden-sentinel input (needed so
unchecked checkboxes still submit "false", since browsers omit unchecked
checkboxes entirely) was hardcoded to value="false" and shared the same
`name` as the checkbox, with no sync between them. Whichever of the two
same-named inputs the save request's form-collection happens to prefer,
the hidden's stale "false" could silently override an actually-checked
box on every save, independent of what the user set.

Fixed in both places this pattern is rendered:
- plugin_config.html: the initial server-rendered row for every array-table
  boolean column (affects countdown's per-countdown "enabled" and the
  custom-feeds widget's per-feed "enabled") — now sets the hidden's initial
  value from the actual data, and syncs it via onchange on every toggle.
- array-table.js: the client-side row renderer used when adding/rebuilding
  rows in the browser — same fix, keeping both inputs in sync at render
  time and via a change listener.

Verified other checkbox/hidden-input patterns in plugin_config.html and
confirmed they're unrelated/already-safe: the default single-checkbox
renderer (no hidden pair), checkbox-group (array-bracket names with its
own explicit sync), the array-table advanced-props modal editor (single
hidden per name, explicitly written on Save), the top-level plugin
enable/disable toggle (separate immediate hx-post), and the no-schema
fallback checkbox. custom-feeds.js's own client-side row renderer never
creates a hidden sentinel at all, so it isn't affected either.

Verified the fix's rendered output directly with an isolated Jinja render
of the exact snippet: hidden and checkbox now agree ("true"/checked or
"false"/unchecked) for both states, where before the hidden was always
"false" regardless of the actual value.


Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 08:05:06 -04:00
cbb8ec41e8 Fix: plugin/base requirements installed as web-user, invisible to root-run display service (#380)
* fix: install plugin/base requirements as root so ledmatrix.service can see them

ledmatrix-web.service runs as a non-root user, so "Reinstall plugin
requirements" installed packages into that user's ~/.local site-packages.
ledmatrix.service (the actual display, which loads and runs plugin code)
runs as root and can't see another user's user-site packages, so plugins
with dependencies not already present system-wide would silently fail at
runtime with ModuleNotFoundError even after a "successful" reinstall.
Reproduced and fixed live against a real device (weather plugin's astral
dependency, used for moon-phase data): confirmed the exact failure
("No module named 'astral'" on every almanac cycle) and confirmed it's
gone after this fix.

Adds scripts/fix_perms/safe_pip_install.sh, a root-owned wrapper (mirroring
the existing safe_plugin_rm.sh pattern) that validates the target is
requirements.txt at the project root or under plugin-repos/ or plugins/
before running pip install as root. configure_web_sudo.sh provisions a
narrowly-scoped sudoers rule for it. api_v3.py's install_base_requirements
and install_plugin_requirements actions now use it via `sudo -n`, falling
back to today's current-user-only install (with an explanatory note) if
the wrapper isn't set up yet, so existing installs don't regress.

Also uses --ignore-installed in the wrapper: root's site-packages often has
apt/dpkg-managed copies of common libraries (requests, etc.) with no pip
RECORD file, which pip refuses to upgrade in place and aborts the *entire*
requirements.txt install over — discovered this while testing the fix live,
since a plugin's other already-satisfied-for-the-web-user dependencies had
never actually been attempted as root before.

Also fixes a pre-existing bug in configure_web_sudo.sh where the
display_controller.py/start_display.sh/stop_display.sh sudoers entries used
PROJECT_DIR (scripts/install/, where this script lives) instead of
PROJECT_ROOT (where those files actually live) — visible as the script's
own "File access test" self-check failing. Verified fixed live.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix: invoke safe_pip_install.sh via explicit bash, matching sudoers rule

CodeRabbit caught this on review: the sudoers rule configure_web_sudo.sh
provisions is scoped to "$BASH_PATH $SAFE_PIP_INSTALL_PATH *" (matching
the existing safe_plugin_rm.sh precedent in
src/common/permission_utils.py), but _pip_install_requirements() called
`sudo -n <wrapper> <req_file>` directly, relying on the script's shebang
instead of an explicit bash prefix. sudo matches the literal command line,
so this never matched the allowlisted rule on an install with only the
specific sudoers entries this script provisions — it silently fell back
to the non-root install path every time, which is the exact bug this PR
set out to fix.

This wasn't caught by live testing on ledpi.local because that device
also has a broader, non-standard "NOPASSWD: ALL" grant which masked the
mismatch. Confirmed the fix is correct by reading sudo's documented
command-matching semantics and mirroring the already-proven-working
bash-prefix pattern from permission_utils.py's safe_plugin_rm.sh call
exactly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

* fix: harden safe_pip_install.sh invocation against bash-path drift + address CodeRabbit nitpicks

CodeRabbit follow-up findings on the bash-prefix fix (7558aaab):

1. (Actionable) shutil.which('bash') at runtime could in principle resolve
   to a different absolute path than configure_web_sudo.sh's `command -v
   bash`, which is resolved once at setup time and baked into the static
   sudoers file as a literal string — sudo requires an exact match. Now
   tries /usr/bin/bash and /bin/bash (the standard Debian/Raspberry Pi OS
   locations, matching what the setup script virtually always produces)
   before falling back to this process's own PATH resolution, so a
   divergence in just one of them doesn't break the install.

2. (Nitpick) Any nonzero returncode was treated as "sudo denied", so a
   real pip failure (bad package, build error) would trigger a pointless
   duplicate non-root install attempt and a misleading error message.
   Now distinguishes "sudo -n rejected this exact command line" from
   "sudo ran it but the command itself failed" via sudo's own diagnostic
   text, and surfaces genuine failures immediately without retrying other
   bash candidates or falling back.

3. (Nitpick) Added structured logging for every fallback/failure path
   (wrapper missing, sudo denied, real install failure), previously only
   visible via the returned stdout note — needed for remote debugging on
   a headless Pi.

Verified: function-level smoke test confirms a real failure (this sandbox's
system python3 lacking pip) is now correctly classified as non-denial and
returned immediately without retrying candidates or double-installing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-06 08:04:49 -04:00
c6ce332d49 fix(web): restore missing brace in Tools tab HTMX-fallback path (#378)
The `else if (++tries > 100)` block added by #373 was missing its
closing `}`, leaving the setInterval arrow function syntactically
unclosed. This caused a JS parse error that silenced the entire
1400-line script block — including the EventSource setup — so the
connection-status indicator never left its default "Disconnected"
state for all users after updating.

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-30 10:05:33 -04:00
210 changed files with 19877 additions and 9028 deletions
+1
View File
@@ -48,3 +48,4 @@ config/backups/
# Starlark apps runtime storage (installed .star files and cached renders)
/starlark-apps/
skin_renders/
+8
View File
@@ -31,6 +31,14 @@
- Plugin configs stored in `config/config.json`, NOT in plugin directories — safe across reinstalls
- Third-party plugins can use their own repo URL with empty `plugin_path`
## Skin System (visual overlays for sports scoreboards)
- Skins live in `skins/<skin-id>/` (skin.json + skin.py), NOT in plugin dirs — plugin reinstall deletes plugin dirs
- Core: `src/skin_system/` (ScoreboardSkin, SkinContext, runtime); hook: `SportsCore._render_game()` in `src/base_classes/sports.py`
- Skins render onto `ctx.canvas` only; fallback to built-in renderer on `False`/exception (3 strikes disables for session)
- View-model guaranteed keys are frozen (see `test/test_skin_system.py::TestViewModelContract`) — renaming keys in `_extract_game_details_common` or sport extractors breaks published skins
- Validate skins headlessly: `python scripts/validate_skin.py --skin <id>`; docs: `docs/SKIN_SYSTEM.md`, `docs/CREATING_SKINS.md`
- Skins are NOT monorepo plugins: no manifest bump / update_registry.py needed
## Common Pitfalls
- paho-mqtt 2.x needs `callback_api_version=mqtt.CallbackAPIVersion.VERSION1` for v1 compat
- BasePlugin uses `get_logger()` from `src.logging_config`, not standard `logging.getLogger()`
+10
View File
@@ -440,6 +440,16 @@ See the [Plugin Store documentation](https://github.com/ChuckBuilds/ledmatrix-pl
For plugin development, check out the [Hello World Plugin](https://github.com/ChuckBuilds/ledmatrix-hello-world) repository as a starter template.
### Visual Skins for Scoreboards
Want a different look for a sports scoreboard without forking the plugin?
**Skins** restyle the live/recent/upcoming screens while the plugin keeps
handling data, scheduling, caching, and vegas mode. Install one with
`git clone <skin repo> skins/<skin-id>`, select it in the plugin's config,
and you're done — see [docs/SKIN_SYSTEM.md](docs/SKIN_SYSTEM.md) (how it
works) and [docs/CREATING_SKINS.md](docs/CREATING_SKINS.md) (build your own,
including a ready-made Claude Code prompt).
2. **Built-in Managers Deprecated**: The built-in managers (hockey, football, stocks, etc.) are now deprecated and have been moved to the plugin system. **You must install replacement plugins from the Plugin Store** in the web interface instead. The plugin system provides the same functionality with better maintainability and extensibility.
</details>
Binary file not shown.

Before

Width:  |  Height:  |  Size: 48 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

After

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

After

Width:  |  Height:  |  Size: 109 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 90 KiB

After

Width:  |  Height:  |  Size: 105 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 42 KiB

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 102 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 101 KiB

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 9.8 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 103 KiB

After

Width:  |  Height:  |  Size: 126 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 93 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 111 KiB

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

+29
View File
@@ -0,0 +1,29 @@
# bandit.yaml — LEDMatrix bandit configuration
# https://bandit.readthedocs.io/en/latest/config.html
#
# Skips are justified by the specific codebase context documented below.
# Do not remove skips without updating the justification comment.
skips:
# B104: Binding to all interfaces (0.0.0.0)
# Intentional — the Flask server binds 0.0.0.0 for LAN access on a Raspberry Pi.
# This is not internet-facing and is documented in web_interface/app.py.
- B104
# B603: subprocess call without shell=True
# All subprocess.run() calls in this codebase use list arguments (confirmed by
# grep — zero uses of shell=True in src/ or web_interface/). List args prevent
# shell injection. See src/common/permission_utils.py for the primary usage.
- B603
# B607: Starting a process with a partial executable path
# The subprocess calls invoke system utilities (systemctl, sudo, git) by name.
# These are fixed-list invocations, not user-controlled, and rely on PATH.
- B607
exclude_dirs:
- tests
- test
- venv
- .venv
- rpi-rgb-led-matrix-master
+1
View File
@@ -121,6 +121,7 @@
"axis": "horizontal"
},
"display_durations": {},
"plugin_rotation_order": [],
"use_short_date_format": true,
"vegas_scroll": {
"enabled": false,
+234
View File
@@ -0,0 +1,234 @@
# Adaptive Layout & Font Scaling
`src/adaptive_layout.py` lets a plugin render legibly on **any** panel size
(64x32, 128x32, 96x48, 128x64, 256x64, ...) without hand-tuned per-display
layouts. It is **opt-in**: nothing changes for plugins that don't use it.
It generalizes three patterns proven in the plugin ecosystem:
| Pattern | Origin | Core API |
|---|---|---|
| Geometry scale factor vs. a design size | f1-scoreboard | `ctx.px(base)` / `ctx.scale` |
| Breakpoint tiers | masters-tournament | `ctx.tier` / `ctx.by_tier({...})` |
| "Largest crisp font that fits" ladder | baseball-scoreboard | `ctx.fit_text(...)` and friends |
## Quick start
Every `BasePlugin` has a lazy `self.layout` (a `LayoutContext` for the
current logical display size, rebuilt automatically if the size changes)
and a one-liner `self.draw_fit(...)`:
```python
def display(self, force_clear=False):
from src.adaptive_layout import LADDER_ARCADE
b = self.layout.bounds.inset(1) # Region(0,0,W,H) minus 1px margin
rows = b.split_v(3, 1, 1, gap=1) # 3/5 for time, 1/5 each for the rest
self.draw_fit(self.time_str, rows[0], ladder=LADDER_ARCADE)
self.draw_fit(self.weekday, rows[1]) # default LADDER_GRID
self.draw_fit(self.date_str, rows[2])
self.display_manager.update_display()
```
On 128x64 the time renders at press_start 24px; on 64x32 it steps down to
8px. The rows partition the height, so bands can never overlap — no more
`y = height - 7` magic numbers.
## Region — rect algebra
`Region(x, y, w, h)` is a frozen dataclass. All carving clamps to
non-negative dimensions, so degenerate panels behave.
- Carving: `inset(dx, dy)`, `top_band(h)`, `bottom_band(h)`,
`middle(top_h, bottom_h)`, `left_col(w)`, `right_col(w)`,
`split_h(*weights, gap=0)`, `split_v(*weights, gap=0)`
- Placement: `align_xy(w, h, align, valign)`, `center_xy(w, h)`,
`contains(w, h)`, `.center`, `.right`, `.bottom`
Scoreboard-style layout:
```python
b = self.layout.bounds
status = b.top_band(self.layout.px(7))
detail = b.bottom_band(self.layout.px(7))
score_area = b.middle(status.h, detail.h)
away_slot, home_slot = b.left_col(b.h), b.right_col(b.h)
```
## Font ladders — discrete, never fractional
Pixel fonts (BDF, PressStart2P) only look right at native/integer sizes, so
fonts are never scaled continuously. A `FontLadder` is an ordered tuple of
`FontStep(family, size_px)` rungs, largest first; fitting walks down until
the measured text fits.
- `LADDER_GRID` (default): X11 BDFs at native sizes — 10x20 → 9x18 → 9x15 →
8x13 → 7x13 → 6x13 → 6x12 → 6x10 → 6x9 → 5x8 → 5x7 → 4x6 → tom-thumb.
Body text, labels, multi-row content.
- `LADDER_ARCADE`: PressStart2P at 32/24/16/8 (integer multiples of its 8px
grid). Headline text: clocks, scores.
Custom ladders are just tuples — e.g. to add your plugin's registered font
on top: `(FontStep("myplugin::digits", 16),) + LADDER_GRID`.
## LayoutContext
Built per (width, height); exposes facts and fit queries:
- `bounds`, `width`, `height`, `aspect`
- `tier` by height (`xs`≤16, `sm`≤32, `md`≤48, `lg`≤64, `xl`) and
`width_tier` (`narrow`≤64, `normal`≤128, `wide`≤256, `ultrawide`)
- `is_wide_short` — aspect ≥ 2.5 and height ≤ 32 (the classic 128x32 shape)
- `scale``min(w/design_w, h/design_h)` vs. your manifest's
`display.design_size` (default 128x32). **Geometry only** — gaps, icon
and logo sizes via `px(base, minimum, maximum)`; fonts use ladders.
- `by_tier({"sm": 10, "lg": 18})` — value for the nearest defined tier
at-or-below the panel's tier.
- `fit_text(text, box, ladder, ellipsis=True)``FitResult` — largest rung
that fits; ellipsizes as a last resort. Cached per (text, box, ladder).
- `fit_text_proportional(text, box, base_size_px, ladder, ellipsis=True, scale=None)`
rung closest to (not exceeding) `base_size_px * scale`, still capped to
what fits the box. Use this instead of `fit_text` when several
independently-fitted elements need to stay visually harmonious as the
panel grows — `fit_text` maximizes *each one* within its own region,
which can make one element (e.g. a score with a generous box) balloon
out of proportion to a neighbor that scales by geometry (e.g. logos
sized via `px()`), even though each individual pick is "correct" in
isolation. `base_size_px` is normally the element's existing classic/
fixed font size. `scale` defaults to `self.scale` (the conservative
min-of-both-axes factor `px()` uses); pass an axis-specific value when
the surrounding composition already scales that way — e.g. a scoreboard
whose logo slots track height alone (`min(height, width // 2)`) should
size its text by `height / design_height` too, or the text reads as
under-scaled next to bigger logos on a panel that only grew taller.
- `fit_lines(lines, box, ladder, spacing)` — every line fits the width and
the stack fits the height (measures the actual strings).
- `font_for_rows(rows, box_h, ladder)` — largest rung whose line height
fits `rows` rows.
`FitResult` carries the ready-to-use `font` (drops straight into
`display_manager.draw_text(font=...)`), the possibly-ellipsized `text`,
ink `width`/`height`, `baseline`, `y_offset`, `line_height`, and `fits`.
## Adaptive images
`src/adaptive_images.py` is the image counterpart to `fit_text`, exposed as
`self.layout.fit_image(...)` (cached per panel size) and the one-liner
`self.draw_image(...)`:
```python
# Team logo: trim its transparent padding, fill the slot height (the
# football/hockey pattern), cached across frames by a stable key
self.draw_image(logo, regs.away_slot, mode="fill_height",
crop_to_ink=True, cache_key=f"logo:{abbr}")
# Album art: cover-crop a square, faces kept by the top anchor
self.draw_image(art, row.art, mode="cover", anchor="top")
# Pixel flags / sprite icons: NEAREST keeps hard edges
from src.adaptive_images import RESAMPLE_NEAREST
self.draw_image(flag, box, resample=RESAMPLE_NEAREST)
```
Modes: `contain` (letterbox, default), `cover` (crop-to-fill),
`fill_height` (logo-style), `stretch`. Unlike PIL's `thumbnail()`
(downscale-only — why imagery stays tiny on big panels) fitting **upscales
by default**; pass `upscale=False` for the legacy behavior. Results are
cached per (image, box size, options) with a bounded LRU — always pass a
stable `cache_key` (e.g. `"logo:KC"`) for images you reload. The module
also exports the Pillow-compat `RESAMPLE_LANCZOS`/`RESAMPLE_NEAREST`
constants so plugins can drop their local shims.
## Composite layouts
Pre-carved Region arrangements for the layouts plugins keep rebuilding:
```python
from src.adaptive_layout import scoreboard_regions, media_row
regs = scoreboard_regions(self.layout.bounds, ctx=self.layout)
# regs.away_slot / home_slot — logo slots (logo_slot = min(H, W // 2),
# capped so a center reserve always exists —
# see below)
# regs.status_band — top band (replaces the magic y = 1)
# regs.score_area — center gap, plus a controlled bleed into
# each logo slot (replaces y = H//2 - 3)
# regs.detail_band — bottom band (replaces y = H - 7)
# regs.bottom_left / bottom_right — record/timeout corners
row = media_row(self.layout.bounds, ctx=self.layout) # art left, text right
```
Both work on the full panel or on a scroll-mode card Region. They return
Regions and never draw — compose them with `draw_fit`/`draw_image`.
**`scoreboard_regions`'s center reserve.** The raw `logo_slot = min(H, W//2)`
formula has a blind spot: at exactly 2:1 aspect ratio (width = 2×height —
two, four, or more square modules stacked into a taller panel, e.g.
96x48, 128x64, 256x128) the two logo slots mathematically claim the
*entire* width, leaving zero pixels for a center column no matter how
big the panel gets. Wide panels (the 128x32 design baseline, 192x48,
256x32) never hit this, since height is already the tighter constraint
there. Two parameters fix it without any plugin-side code:
`min_center_fraction`/`min_center_design_px` guarantee a real minimum
center reserve at any aspect ratio, and `score_bleed_fraction` lets the
score's *fit box* extend a controlled amount into each logo slot — the
same way a real broadcast scoreboard's numbers cross slightly into the
team marks flanking them — so a short score string never has to truncate
even on the tightest aspect ratios. All three have sane defaults; override
them per call if a plugin's card proportions genuinely differ.
## Preserving user customization
Adaptive layout supplies *defaults*; explicit user configuration wins:
- **User-set fonts win.** If the plugin's config has an explicit
`font`/`font_size` for an element, load it as before and skip the ladder —
fit only when the user hasn't overridden (see the football-scoreboard
`_resolve_element_fit` pattern).
- **Offsets apply on top.** `customization.layout.<element>.{x_offset,y_offset}`
style knobs translate the *computed* region as a final step:
`region.offset(user_dx, user_dy)`. `draw_image(..., offset=(dx, dy))`
does the same for images.
- **Colors pass through.** `draw_fit`/`draw_fitted_text` take explicit
`color=` params; adaptive mode never repaints semantic or user-chosen
colors.
## Manifest declaration
Declare the size your layout was authored against so `ctx.scale` means
something:
```json
"display": { "design_size": { "width": 128, "height": 32 } }
```
Also available under `requires.display_size`: `min_width`, `min_height`,
`max_width`, `max_height`.
## Performance notes (Pi)
Fit queries are cached, so cost is O(unique strings). For per-second text
(clocks, live scores), fit on a **shape placeholder** and reuse the font:
```python
fit = self.layout.fit_text("00:00", box, ladder=LADDER_ARCADE) # cached once
self.display_manager.draw_text(current_time, font=fit.font, ...)
```
## Testing across sizes
The harness already renders every plugin at a spread of sizes (now
including 96x48):
```bash
python scripts/check_plugin.py <plugin-dir> --sizes 64x32,128x32,96x48,128x64,256x64
python scripts/render_plugin.py <plugin-dir> --width 96 --height 48
```
`BoundsCheckingDisplayManager` flags right/bottom overflow and now records
mediated draw calls with negative coordinates in
`negative_coordinate_calls` (raw-PIL draws remain uncovered).
Reference migration: the **text-display** plugin's `font_mode: "auto"`.
+6
View File
@@ -2,6 +2,12 @@
Advanced patterns, examples, and best practices for developing LEDMatrix plugins.
> **Adaptive layout:** for plugins that should render legibly on any panel
> size (fonts that grow on big panels, layouts that degrade gracefully on
> small ones), use the adaptive layout system — `self.layout`, `draw_fit`,
> `draw_image`, `scoreboard_regions` — documented in
> [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md).
## Table of Contents
- [Using Weather Icons](#using-weather-icons)
+242
View File
@@ -0,0 +1,242 @@
# Creating Skins
A skin restyles a sports scoreboard (live / recent / upcoming) without
forking the plugin: the plugin keeps fetching data, scheduling, caching, and
doing vegas mode; your skin only draws. Architecture background:
[SKIN_SYSTEM.md](SKIN_SYSTEM.md).
## Quick start
```bash
cp -r skins/example-classic-baseball skins/my-skin
# edit skins/my-skin/skin.json -> set id ("my-skin"), name, author, class_name
# edit skins/my-skin/skin.py -> rename the class, start restyling
python scripts/validate_skin.py --skin my-skin
```
The validator renders your skin against bundled fixture games at several
panel sizes with **no hardware, no network, no running service**, saves PNGs
(plus 4x previews) to `skin_renders/`, and fails loudly on errors. Iterate:
edit → validate → look at the PNGs.
To see it on your matrix, add to your plugin's section in `config/config.json`:
```json
"baseball-scoreboard": {
"skin": "my-skin",
"skin_options": { }
}
```
or pick it from the **Visual Skin** dropdown in the web UI (it appears once a
matching skin is installed). `"skin"` also accepts a per-mode mapping:
`{"live": "my-skin", "recent": "built-in"}`.
## The manifest (`skin.json`)
```json
{
"id": "my-skin",
"name": "My Skin",
"version": "1.0.0",
"author": "you",
"description": "What it looks like",
"skin_api_version": "1.0.0",
"targets": {
"sports": ["baseball"],
"sport_keys": ["mlb", "milb"],
"plugins": []
},
"entry_point": "skin.py",
"class_name": "MySkin",
"modes": ["live", "recent", "upcoming"],
"preview": "preview.png"
}
```
Field notes: `id` must equal the directory name; `skin_api_version`'s major
version must match the host's `SKIN_API_VERSION` or the skin is refused at
load; `targets` takes sport families (`sports`), exact sport keys
(`sport_keys`), and/or exact plugin ids (`plugins`) — any match applies.
## The renderer (`skin.py`)
```python
from src.skin_system.skin_base import ScoreboardSkin, SkinContext
class MySkin(ScoreboardSkin):
def render_live(self, ctx: SkinContext, game: dict) -> bool:
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
fit = ctx.layout.fit_text(score, ctx.layout.bounds)
ctx.draw_fit(fit, ctx.layout.bounds)
return True # True = "I drew it"; False = use the built-in layout
```
Implement only the modes you care about — anything else falls back to the
plugin's built-in rendering. Return `False` to decline a specific game (e.g.
a layout that only makes sense while a game is live).
### The rules (they keep your skin from breaking the display)
1. **Draw only onto `ctx.canvas`** (via the helpers or `ctx.draw`). Never
reassign `ctx.canvas`, never touch the display or call any update method.
2. **No I/O in render paths.** No network, no file loads per frame —
`render_live` runs every display pass, and a slow render stalls the whole
matrix (the host warns at >150 ms). Use `ctx.load_logo` (cached) and
`cache_key=` for images.
3. **Derive everything from `(ctx, game)`.** Skins must be stateless: the
live/recent/upcoming modes each get their own instance.
4. **Always `.get()` optional keys.** Only the guaranteed keys below are
promised to exist.
5. **Never hardcode pixel positions for the panel.** Use `ctx.width`/
`ctx.height`, `ctx.layout` regions and `fit_text` — your skin will be run
at sizes you didn't test (64x32, 128x64, vegas cards).
6. **No third-party dependencies.** Stdlib + PIL + what `ctx` provides.
A skin that raises 3 renders in a row is disabled until the service restarts
(the built-in layout takes over), so a bug is cosmetic — but check your logs.
## SkinContext reference
| Member | What it is |
|---|---|
| `ctx.canvas` / `ctx.draw` | Fresh RGB `PIL.Image` at display size + its `ImageDraw` (raw-PIL escape hatch) |
| `ctx.width`, `ctx.height` | Canvas size — the only size truth |
| `ctx.layout` | `LayoutContext` (see [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md)): `bounds`, `fit_text`, `fit_text_proportional`, `fit_image`, `px`, `by_tier` |
| `ctx.draw_fit(fit, box, color, align, valign)` | Draw a `fit_text` result aligned in a `Region` (handles BDF fonts) |
| `ctx.draw_text(text, x, y, color, font)` | Positioned text (handles BDF fonts) |
| `ctx.draw_image(img, box, mode, align, valign, cache_key)` | Fit + paste an image with alpha; no-ops on `None` |
| `ctx.load_logo("home" \| "away")` | Team logo as RGBA, or `None` (always handle `None`). Cached after first use; see note below |
| `ctx.draw_text_outlined(text, (x, y), font, fill, outline_color)` | The classic scorebug outlined text (TTF fonts only) |
| `ctx.fonts` | The host's font dict — keys `score`, `time`, `team`, `status`, `detail`, `rank` |
| `ctx.options` | Your user's `skin_options` from config |
| `ctx.sport`, `ctx.view_model_version`, `ctx.logger` | Context metadata + logger |
**A note on `ctx.load_logo` vs the no-I/O rule:** `load_logo` is the one
sanctioned exception. It goes through the host's logo cache — after the
first call per team it's a pure in-memory lookup. If a logo file is missing
on disk, the *first* call may download it, exactly like the built-in
renderer does for the same game (a skin is never worse than built-in here).
Always pass a stable `cache_key` when drawing it, never load image files
yourself in a render path, and always handle `None`.
The default layout idiom — carve regions, then fit text into them:
```python
from src.adaptive_layout import scoreboard_regions
regions = scoreboard_regions(ctx.layout.bounds, ctx=ctx.layout)
ctx.draw_image(ctx.load_logo("away"), regions.away_slot, cache_key=f"logo:{game.get('away_abbr')}")
ctx.draw_image(ctx.load_logo("home"), regions.home_slot, cache_key=f"logo:{game.get('home_abbr')}")
fit = ctx.layout.fit_text("3-5", regions.score_area)
ctx.draw_fit(fit, regions.score_area)
```
`Region` supports `split_h`/`split_v`/`inset`/`top_band`/`bottom_band`/
`left_col`/`right_col` for custom carves. Raw `ctx.draw.rectangle/polygon/
ellipse/...` is always available for custom marks (see the bases diamond in
the example skin).
## The game view model
Guaranteed for every sport (view model v1.0 — renaming these breaks skins and
is treated as a breaking change upstream):
| Key | Notes |
|---|---|
| `id` | Event id (string) |
| `status_text` | Display-ready status, e.g. `"Final"`, `"7:30 PM"`, `"Bot 7th"` |
| `is_live`, `is_final`, `is_upcoming`, `is_halftime` | Booleans |
| `game_date`, `game_time` | Pre-formatted local date/time strings |
| `start_time_utc` | UTC `datetime` |
| `home_abbr`, `away_abbr` | Team abbreviations (can be 25 chars — fit, don't assume) |
| `home_id`, `away_id` | Team ids |
| `home_score`, `away_score` | **Strings**, not ints |
| `home_record`, `away_record` | `"58-33"` or `""` (0-0 records are blanked) |
| `home_logo_path`, `away_logo_path` | Prefer `ctx.load_logo` over touching these |
Sport extras (present for that sport, still `.get()` defensively):
- **baseball**: `inning` (int), `inning_half` (`"top"`/`"bottom"`), `balls`,
`strikes`, `outs` (ints), `bases_occupied` (`[first, second, third]`
booleans), `series_summary` (str)
- **football**: `period`, `period_text`, `clock`, `home_timeouts`,
`away_timeouts`, `down_distance_text`, `down_distance_text_long`,
`is_redzone`, `possession`, `possession_indicator` (`"home"`/`"away"`),
`scoring_event`
- **basketball**: `period`, `period_text`, `clock`
- **hockey**: `period`, `period_text`, `clock`, `power_play`, `penalties`,
`home_shots`, `away_shots`
Optional everywhere (only when the user enabled the feature): `odds` (dict),
`series_summary`, rankings-related fields.
Fixture copies of these dicts live in `src/skin_system/fixtures/` — that's
exactly what the validator feeds your skin.
## Vegas mode
You get vegas support for free: vegas captures the normal display output,
which is already your skin's rendering. Optionally implement
`render_vegas_card(ctx, game)` to return a purpose-built card at
`ctx.width x ctx.height` (sizes vary — never assume 128x32).
## Building a skin with Claude Code
Skins are ideal Claude Code projects: small, isolated, and verifiable with
one command. Paste this to start:
> You are building a **display skin** for LEDMatrix — a visual overlay for a
> sports scoreboard on a small LED matrix (commonly 128x32 or 64x32 pixels).
> First read `docs/CREATING_SKINS.md` and the reference skin in
> `skins/example-classic-baseball/`.
>
> Rules:
> - Create/modify files ONLY under `skins/<my-skin-id>/`. Do NOT modify
> anything in `src/`, `scripts/`, the plugins, or any other skin.
> - Render only from the `game` dict and `ctx` helpers. No network calls, no
> per-frame file I/O, no new pip dependencies, no touching the display —
> draw onto `ctx.canvas` and return True.
> - Use `ctx.layout` regions and `fit_text` for positioning so the skin works
> at any panel size; use `.get()` for every optional game key.
> - After every change run
> `python scripts/validate_skin.py --skin <my-skin-id>` and LOOK at the
> PNGs it writes to `skin_renders/` (the `_x4.png` files are easiest to
> read). Iterate until it passes and looks right at both 128x32 and 64x32.
>
> What I want it to look like: <describe your layout — where logos, score,
> status go; colors; what shows during live vs upcoming vs final>
Tips that keep Claude (and you) out of trouble:
- One mode at a time: get `render_live` right before touching the others —
unimplemented modes automatically use the built-in look.
- Ask for edge-case renders: long team abbreviations, missing logos
(`ctx.load_logo` returning `None`), 0-0 records, extra innings/OT.
- If the render looks cramped at 64x32, ask Claude to use
`ctx.layout.by_tier(...)` to drop elements on small panels rather than
shrinking everything.
- Never let it "fix" a problem by editing `src/` — if the skin can't do
something within its directory, that's a feature request, not a workaround.
## Pre-publish checklist
- [ ] `python scripts/validate_skin.py --skin <id> --size 128x32 --size 64x32 --size 128x64` passes
- [ ] Looked at every PNG in `skin_renders/` — nothing clipped or overlapping
- [ ] Handles a missing logo (`None`) without crashing — temporarily point a
fixture's logo path at a nonexistent file to test
- [ ] Long abbreviations (`"TA&M"`, 45 chars) don't overflow
- [ ] No render warning above the time budget
- [ ] `skin.json`: `id` matches the directory, `version` set,
`skin_api_version` matches the host, targets correct
- [ ] `preview.png` added (grab your favorite `_x4` render)
- [ ] Tested on real hardware if you have it — a Pi is much slower than your
dev machine
Distribute by publishing the directory as a git repo (users
`git clone <repo> skins/<id>`), or submit it to the plugin registry as an
entry with `"type": "skin"` (see [SKIN_SYSTEM.md](SKIN_SYSTEM.md) §Distribution).
**Trust note:** a skin is Python running inside the display service — the
same trust level as a plugin. Review code before installing skins from
others.
+6
View File
@@ -48,6 +48,12 @@ display_manager.draw_text("Centered", centered=True) # Auto-center
width = display_manager.get_text_width("Text", font)
height = display_manager.get_font_height(font)
# Adaptive layout (recommended for multi-size support — text and images
# that scale to any panel; see docs/ADAPTIVE_LAYOUT.md)
rows = self.layout.bounds.inset(1).split_v(3, 1, gap=1)
self.draw_fit("12:34", rows[0]) # largest crisp font that fits
self.draw_image(logo, rows[1], mode="fill_height", crop_to_ink=True)
# Weather icons
display_manager.draw_weather_icon("rain", x=10, y=10, size=16)
+6
View File
@@ -6,6 +6,12 @@ Tools for rapid plugin development without deploying to the RPi.
Interactive web UI for tweaking plugin configs and seeing the rendered display in real time.
The size inputs have a preset dropdown with the harness's standard panel
sizes, and the **All Sizes** button renders the current config at every
harness size in a side-by-side gallery (`POST /api/render-matrix`) — the
quickest way to eyeball adaptive-layout behavior across panels
(see [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md)).
### Quick Start
```bash
+7
View File
@@ -1,5 +1,12 @@
# FontManager Usage Guide
> **Picking a size automatically:** if you want the *largest font that fits
> a given area* rather than a fixed size, use the adaptive layout system's
> font ladders, which resolve through this FontManager. `BasePlugin`
> subclasses get this as `self.layout.fit_text(...)`; other code can build
> a `LayoutContext(width, height, font_manager)` directly — see
> [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md).
## Overview
The enhanced FontManager provides comprehensive font management for the LEDMatrix application with support for:
-1
View File
@@ -248,7 +248,6 @@ test/
├── test_config_service.py # Config service tests
├── test_config_validation_edge_cases.py # Config edge cases
├── test_font_manager.py # Font manager tests
├── test_layout_manager.py # Layout manager tests
├── test_text_helper.py # Text helper tests
├── test_error_handling.py # Error handling tests
├── test_error_aggregator.py # Error aggregation tests
+5
View File
@@ -2,6 +2,11 @@
Complete API reference for plugin developers. This document describes all methods and properties available to plugins through the Display Manager, Cache Manager, and Plugin Manager.
> **Adaptive layout:** every `BasePlugin` also exposes `self.layout`,
> `self.draw_fit(text, region)` and `self.draw_image(img, region, ...)` —
> the recommended way to render text and images that scale to any panel
> size. See [ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md).
## Table of Contents
- [BasePlugin](#baseplugin)
+14
View File
@@ -2,6 +2,20 @@
This guide explains how to set up a development workflow for plugins that are maintained in separate Git repositories while still being able to test them within the LEDMatrix project.
> **Rendering guidance:** plugins should read the display size dynamically
> (`self.display_manager.matrix.width/height`) rather than hardcoding one
> panel. For plugins that want to *scale* their layout to any panel, the
> opt-in adaptive layout system ([ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md))
> provides the shared helpers — fonts, images, and composite layouts that
> scale. Existing plugins keep their classic rendering unless they adopt
> those APIs; nothing migrates automatically.
> **Just want a different look for an existing sports scoreboard?** You may
> not need a plugin at all — a **skin** restyles the live/recent/upcoming
> rendering while the plugin keeps handling data, scheduling, caching, and
> vegas mode, in ~100 lines of drawing code. See
> [CREATING_SKINS.md](CREATING_SKINS.md).
## Overview
When developing plugins in separate repositories, you need a way to:
+170
View File
@@ -0,0 +1,170 @@
# Skin System Architecture
Skins are user-installable **visual overlays** for the sports scoreboards.
A skin replaces only the *look* of a scoreboard — the host plugin keeps doing
data fetching, scheduling, caching, dedup, live-priority takeover, and vegas
mode. If you only want to **build** a skin, read
[CREATING_SKINS.md](CREATING_SKINS.md); this document explains how the system
works and why it is shaped this way.
## Why skins instead of forks
Before skins, changing a scoreboard's layout meant forking the whole plugin
(e.g. the community MLB scoreboard fork). The fork gets the new look but loses
everything the maintained plugin keeps earning: duration/scheduling behavior,
vegas mode support, caching and background-fetch improvements, bug fixes. It
also silently drifts: every upstream improvement now has to be re-ported by
hand.
A skin inverts that trade. The plugin remains stock and keeps updating through
the store; the skin is ~100 lines of pure rendering code that receives the
plugin's already-fetched data each frame. Uninstalling the skin (or the skin
crashing) simply restores the built-in look.
```text
(unchanged) (the skin seam)
ESPN API ──► update() ──► game view model ──► _render_game() ──► display
fetching (a dict) │ │
caching │ └─ built-in
scheduling └─ skin.render_<mode>(ctx, game)
live priority draws onto ctx.canvas
```
## The render funnel
Every sports scoreboard (baseball, football, basketball, hockey — anything
built on `src/base_classes/sports.py`) renders through exactly one seam:
`SportsCore._render_game(game, force_clear)`.
1. The mode class's `display()` (live, `SportsUpcoming`, `SportsRecent`)
picks `self.current_game` and calls `_render_game`.
2. `_render_game` lazily loads the configured skin (once, on first render —
a broken skin can never block plugin startup).
3. If a skin is active, the host builds a `SkinContext` — a fresh black
canvas at the current display size plus layout/font/logo helpers — and
calls the skin's `render_live` / `render_recent` / `render_upcoming`
with a **copy** of the game dict.
4. If the skin returns `True`, the canvas is composited onto the display.
If it returns `False`, isn't implemented for that mode, or raises, the
built-in `_draw_scorebug_layout` runs instead.
Key properties that fall out of this design:
- **Per-mode fallback.** A skin that only implements `render_live` gets the
stock recent/upcoming screens for free.
- **Three strikes.** A skin that raises 3 times in a row is disabled for the
rest of the session (one loud error log per failure); the display never
goes dark. Restarting the service re-arms it.
- **Copies, not references.** Skins receive a shallow copy of the game dict,
so a buggy skin cannot corrupt the plugin's scheduling state.
- **Vegas mode works untouched.** Vegas capture falls back to grabbing the
regular `display()` output, which is already skin-rendered. Skins can
additionally implement `render_vegas_card` for purpose-built scroll cards,
and hosts can call `SportsCore.render_skin_card(game, size)` to use it.
- **Hot-loop caution.** `render_live` runs every display-loop pass during a
live game. The host logs a warning when a skin render exceeds 150 ms, and
`scripts/validate_skin.py` enforces a budget at development time — but
Python cannot forcibly time-out a stuck render, so a skin that blocks
(network I/O, giant image ops) stalls the display. This is why the rules
in CREATING_SKINS.md ban I/O in render paths.
## The view model contract
The `game` dict a skin receives is the plugin's already-extracted view model
(`SportsCore._extract_game_details_common` plus per-sport extras from
`src/base_classes/{baseball,basketball,football,hockey}.py`).
- **Guaranteed keys (view model v1.0)** — always present for every sport:
`id`, `game_time`, `game_date`, `start_time_utc` (a UTC `datetime`),
`status_text`, `is_live`, `is_final`, `is_upcoming`, `is_halftime`,
`home_abbr`/`away_abbr`, `home_id`/`away_id`, `home_score`/`away_score`
(**strings**), `home_logo_path`/`away_logo_path`, `home_record`/`away_record`.
- **Sport extras** — documented per sport in CREATING_SKINS.md (e.g. baseball
adds `inning`, `inning_half`, `balls`, `strikes`, `outs`, `bases_occupied`).
- **Optional keys** (`odds`, rankings, `series_summary`, …) are present only
when the feature is enabled — skins must always use `.get()`.
Versioning policy: additive changes bump the minor version
(`VIEW_MODEL_VERSION` in `src/skin_system/skin_base.py`, surfaced to skins as
`ctx.view_model_version`); renaming or removing a guaranteed key requires a
major bump plus a compat shim. `test/test_skin_system.py::TestViewModelContract`
fails CI if a guaranteed key disappears from the extractor.
Separately, `SKIN_API_VERSION` versions the Python API (`ScoreboardSkin`,
`SkinContext`). The loader refuses a skin whose manifest declares a different
major version and falls back to the built-in renderer with a clear
"skin needs an update" log line.
## Package layout and lifecycle
```text
skins/<skin-id>/
skin.json # manifest (required)
skin.py # ScoreboardSkin subclass (required)
preview.png # optional, shown by the web UI
assets/ # optional skin-local images
helpers.py ... # optional extra modules (namespaced per skin at import)
```
Skins live in the central `skins/` directory — deliberately **not** inside the
plugin's directory, because plugin reinstall/update deletes the whole plugin
directory and a skin must survive that. One skin can also target several
plugins (mlb + milb).
Lifecycle: discovered lazily on first render → manifest validated → API major
version gated → module imported under a namespaced `sys.modules` key (two
skins can both ship a `helpers.py`, same scheme plugins use) → instantiated
with `(manifest, options)`. Every failure logs and falls back to built-in.
Skins should be **stateless**: the live, recent, and upcoming mode classes
each hold their own skin instance, so derive everything from `(ctx, game)`.
## Selection and configuration
Inside the plugin's own config section in `config/config.json`:
```json
"baseball-scoreboard": {
"skin": "retro-baseball",
"skin_options": { "accent_color": [255, 80, 0] }
}
```
`"skin"` is either one id for all modes or a per-mode mapping
(`{"live": "retro-baseball", "recent": "built-in"}`). Absent, empty, or
`"built-in"` means the stock renderer. Because this rides the plugin's config
section, it persists across plugin reinstalls like every other setting.
The web UI shows a **Visual Skin** dropdown for plugins that have matching
skins installed: `SchemaManager.inject_skin_selector` adds an enum to the
*served* schema only. Validation never sees the enum — so a config that
references an uninstalled skin stays valid (rendering just falls back), and
the currently-configured value is always kept selectable. `GET /api/v3/skins`
lists installed skins (optionally filtered by `?plugin_id=`).
## Distribution
- **Manual:** `git clone <skin repo> skins/<skin-id>` — that's the whole
install. No manifest bumps, no `update_registry.py`; skins are not monorepo
plugins.
- **Store:** registry entries with `"type": "skin"` install through the same
`plugins.json` pipeline; `PluginStoreManager` routes them to `skins/`,
validates `skin.json` (including the API major version) instead of
`manifest.json`, and never installs dependencies — skins are render-only
(stdlib + PIL + the provided context, no third-party packages in v1).
## Trust model
A skin is Python executing inside the display service — **exactly the same
trust level as a plugin**, even though "skin" sounds cosmetic. Only install
skins from sources you'd be willing to install a plugin from.
## v2 directions (not in v1)
- A generic `BasePlugin` opt-in (`render_with_skin()`) so non-sports plugins
(weather, music) can offer skinnable layouts; `skin_runtime` is already
sports-agnostic in anticipation.
- Store UI: preview gallery, one-click install from the skin browser.
- An update path for git-cloned skins (today: re-clone or store reinstall).
- Animation support in skins (today the API is one frame per render call;
stateful tricks work but are at-your-own-risk).
+34
View File
@@ -206,6 +206,40 @@ To use an existing widget in your plugin's `config_schema.json`, simply add the
The widget will be automatically rendered when the plugin configuration form is loaded.
## Marking Fields as Advanced (`x-advanced`)
Add `"x-advanced": true` to any top-level, non-object property to move it out
of the main form and into a single collapsed **Advanced Settings** section at
the bottom of the plugin's configuration page:
```json
{
"properties": {
"city": {
"type": "string",
"title": "City"
},
"request_timeout": {
"type": "integer",
"default": 10,
"description": "HTTP timeout in seconds",
"x-advanced": true
}
}
}
```
Guidelines:
- Use it for fine-tuning knobs most users never touch (timeouts, retry
behavior, cache TTLs, styling overrides). Anything a first-time user must
set to get the plugin working should stay basic.
- Nothing is hidden permanently — the section expands on click, and the
settings search finds and auto-expands advanced fields like any others.
- The flag is ignored on `object`-type properties (they already render as
their own collapsible sections) and is safely ignored by older cores, so
adding it never breaks compatibility.
## Creating Custom Widgets
### Step 1: Create Widget File
+15 -6
View File
@@ -726,7 +726,11 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
if command -v timeout >/dev/null 2>&1; then
# Use timeout if available (10 minutes = 600 seconds)
if timeout 600 python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
# --ignore-installed: apt-managed packages (e.g. python3-requests)
# ship no pip RECORD file, so upgrading them would otherwise abort
# with "uninstall-no-record-file"; this lays the new version down
# alongside instead of trying to uninstall the apt copy first.
if timeout 600 python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
INSTALL_SUCCESS=true
else
EXIT_CODE=$?
@@ -734,7 +738,7 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
echo "✗ Timeout (10 minutes) installing: $line"
echo " This package may require building from source, which can be slow on Raspberry Pi."
echo " You can try installing it manually later with:"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose '$line'"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose '$line'"
else
echo "✗ Failed to install: $line (exit code: $EXIT_CODE)"
fi
@@ -742,7 +746,7 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
else
# No timeout command available, install without timeout
echo " Note: timeout command not available, installation may take a while..."
if python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
if python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose "$line" > "$INSTALL_OUTPUT" 2>&1; then
INSTALL_SUCCESS=true
else
EXIT_CODE=$?
@@ -794,7 +798,7 @@ if [ -f "$PROJECT_ROOT_DIR/requirements.txt" ]; then
echo " 1. Ensure you have enough disk space: df -h"
echo " 2. Check available memory: free -h"
echo " 3. Try installing failed packages individually with verbose output:"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --verbose <package>"
echo " python3 -m pip install --break-system-packages --no-cache-dir --prefer-binary --ignore-installed --verbose <package>"
echo " 4. For packages that build from source (like numpy), consider:"
echo " - Installing pre-built wheels: python3 -m pip install --only-binary :all: <package>"
echo " - Or installing via apt if available: sudo apt install python3-<package>"
@@ -816,7 +820,10 @@ echo ""
# Install web interface dependencies
echo "Installing web interface dependencies..."
if [ -f "$PROJECT_ROOT_DIR/web_interface/requirements.txt" ]; then
if python3 -m pip install --break-system-packages --prefer-binary -r "$PROJECT_ROOT_DIR/web_interface/requirements.txt"; then
# --ignore-installed: apt-managed packages (e.g. python3-requests) ship no
# pip RECORD file, so upgrading them to the version pinned here would
# otherwise abort the whole install with "uninstall-no-record-file".
if python3 -m pip install --break-system-packages --prefer-binary --ignore-installed -r "$PROJECT_ROOT_DIR/web_interface/requirements.txt"; then
echo "✓ Web interface dependencies installed"
# Create marker file to indicate dependencies are installed
touch "$PROJECT_ROOT_DIR/.web_deps_installed"
@@ -977,7 +984,9 @@ else
else
echo "Using pip to install dependencies..."
if [ -f "$PROJECT_ROOT_DIR/requirements_web_v2.txt" ]; then
python3 -m pip install --break-system-packages --prefer-binary -r requirements_web_v2.txt
# --ignore-installed: see the Step 5 web_interface/requirements.txt
# install above — same apt/pip RECORD-file conflict applies here.
python3 -m pip install --break-system-packages --prefer-binary --ignore-installed -r requirements_web_v2.txt
else
echo "⚠ requirements_web_v2.txt not found; skipping web dependency install"
fi
+3 -7
View File
@@ -8,16 +8,11 @@ numpy>=1.24.0 # For fast array operations in ScrollHelper (compatible with 2.x)
# Timezone handling
pytz>=2024.2,<2025.0 # Updated for latest timezone data
timezonefinder>=6.5.0,<7.0.0 # Updated for better performance and accuracy
geopy>=2.4.1,<3.0.0
# HTTP requests
requests>=2.33.0,<3.0.0
# Google API integration
google-auth-oauthlib>=1.2.0,<2.0.0
google-auth-httplib2>=0.2.0,<1.0.0
google-api-python-client>=2.147.0,<3.0.0
# Font rendering
freetype-py>=2.5.1,<3.0.0
@@ -29,10 +24,8 @@ spotipy>=2.25.2,<3.0.0
Flask>=3.1.3,<4.0.0
# Text processing
unidecode>=1.3.8,<2.0.0
# Calendar integration
icalevents>=0.1.27,<1.0.0
# WebSocket support
python-socketio>=5.14.0,<6.0.0
@@ -43,6 +36,9 @@ websocket-client>=1.8.0,<2.0.0
# JSON Schema validation
jsonschema>=4.20.0,<5.0.0
# Requirement specifier parsing (plugin dependency satisfaction checks)
packaging>=23.0,<27.0
# Testing dependencies
pytest>=9.0.3,<10.0.0
pytest-cov>=4.1.0,<5.0.0
+29
View File
@@ -90,11 +90,40 @@
"min_height": {
"type": "integer",
"minimum": 1
},
"max_width": {
"type": "integer",
"minimum": 1
},
"max_height": {
"type": "integer",
"minimum": 1
}
}
}
}
},
"display": {
"type": "object",
"properties": {
"design_size": {
"type": "object",
"properties": {
"width": {
"type": "integer",
"minimum": 8
},
"height": {
"type": "integer",
"minimum": 8
}
},
"required": ["width", "height"],
"description": "Panel size the plugin's layout was authored against; core derives the adaptive-layout scale factor from it. Defaults to 128x32 when omitted."
}
},
"description": "Display/layout hints for the adaptive layout system"
},
"config_schema": {
"type": "string",
"description": "Path to configuration schema file"
+344
View File
@@ -0,0 +1,344 @@
#!/usr/bin/env python3
"""
LEDMatrix Plugin Security Auditor
Performs AST-based security analysis of all Python files in plugin directories.
Designed to run in CI — exits non-zero on CRITICAL findings only.
Usage:
python scripts/audit_plugins.py
python scripts/audit_plugins.py --verbose
python scripts/audit_plugins.py --plugin hello-world
python scripts/audit_plugins.py --output results.json
"""
import ast
import argparse
import json
import sys
from dataclasses import dataclass, asdict
from pathlib import Path
from datetime import datetime, timezone
PROJECT_ROOT = Path(__file__).resolve().parent.parent
PLUGIN_BASE_DIRS = [
PROJECT_ROOT / "plugins",
PROJECT_ROOT / "plugin-repos",
]
# ─────────────────────────────────────────────────────────────────────────────
# Finding dataclass
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class Finding:
plugin_id: str
file: str
line: int
severity: str # CRITICAL | WARNING | INFO
rule: str
message: str
def to_dict(self) -> dict:
return asdict(self)
# ─────────────────────────────────────────────────────────────────────────────
# AST visitor
# ─────────────────────────────────────────────────────────────────────────────
class _PluginVisitor(ast.NodeVisitor):
"""Collect security findings from a single plugin Python file."""
def __init__(self, filepath: Path, plugin_id: str):
self.filepath = filepath
self.plugin_id = plugin_id
self.findings: list[Finding] = []
# Local name -> real dotted path, so aliased imports and from-imports
# of dangerous APIs (import subprocess as sp; from builtins import
# eval as e) are still recognized in visit_Call below.
self._aliases: dict[str, str] = {}
def _add(self, node: ast.AST, severity: str, rule: str, message: str) -> None:
self.findings.append(Finding(
plugin_id=self.plugin_id,
file=str(self.filepath.relative_to(PROJECT_ROOT)),
line=getattr(node, "lineno", 0),
severity=severity,
rule=rule,
message=message,
))
def _resolve(self, local_name: str) -> str:
"""Resolve a local name through recorded import aliases to its real
dotted path (e.g. "sp" -> "subprocess"); unresolved names pass through
unchanged."""
return self._aliases.get(local_name, local_name)
def _resolve_call_target(self, func: ast.expr) -> str | None:
"""Resolve a Call's func node to a fully-qualified dotted target,
covering a direct name (bare builtin, aliased import, or
from-import: from builtins import eval as e; from subprocess
import run; from os import system as s) and module-attribute
access (subprocess.run, sp.run, os.system, o.system) uniformly.
Returns None for call shapes this doesn't attempt to resolve."""
if isinstance(func, ast.Name):
return self._resolve(func.id)
if isinstance(func, ast.Attribute) and isinstance(func.value, ast.Name):
base = self._resolve(func.value.id)
return f"{base}.{func.attr}"
return None
def visit_Call(self, node: ast.Call) -> None:
target = self._resolve_call_target(node.func)
if target is None:
self.generic_visit(node)
return
leaf = target.rsplit(".", 1)[-1]
# eval() / exec() / compile() — arbitrary code execution, whether a
# bare call, an aliased import, or a from-import
# (from builtins import eval as e; e(...))
if leaf == "eval":
self._add(node, "CRITICAL", "PLUGIN-001",
"eval() call — arbitrary code execution risk")
elif leaf == "exec":
self._add(node, "CRITICAL", "PLUGIN-002",
"exec() call — arbitrary code execution risk")
elif leaf == "compile":
self._add(node, "WARNING", "PLUGIN-003",
"compile() call — dynamic code compilation")
# subprocess.*(shell=True), whether subprocess.run(...), sp.run(...),
# or a from-import (from subprocess import run; run(..., shell=True))
if target in {
"subprocess.run", "subprocess.call", "subprocess.Popen",
"subprocess.check_call", "subprocess.check_output",
}:
for kw in node.keywords:
if (kw.arg == "shell" and
isinstance(kw.value, ast.Constant) and
kw.value.value is True):
self._add(node, "WARNING", "PLUGIN-004",
f"subprocess.{leaf}(shell=True) — "
f"shell injection risk if args include user input")
# os.system(), whether os.system(...), o.system(...), or a
# from-import (from os import system as s; s(...))
if target == "os.system":
self._add(node, "WARNING", "PLUGIN-005",
"os.system() call — prefer subprocess with list args")
self.generic_visit(node)
def visit_Import(self, node: ast.Import) -> None:
for alias in node.names:
if alias.asname:
local, real = alias.asname, alias.name
else:
# `import os.path` binds the top-level name `os`, not `os.path`
local = real = alias.name.split(".")[0]
self._aliases[local] = real
self._check_import(node, alias.name)
self.generic_visit(node)
def visit_ImportFrom(self, node: ast.ImportFrom) -> None:
if node.module:
for alias in node.names:
local = alias.asname or alias.name
self._aliases[local] = f"{node.module}.{alias.name}"
self._check_import(node, node.module)
self.generic_visit(node)
def _check_import(self, node: ast.AST, module_name: str) -> None:
dangerous = {
"ctypes": ("WARNING", "PLUGIN-010", "ctypes import — native code execution"),
"cffi": ("WARNING", "PLUGIN-011", "cffi import — native code execution"),
"pickle": ("WARNING", "PLUGIN-012",
"pickle import — deserialization can execute arbitrary code"),
"marshal": ("WARNING", "PLUGIN-013",
"marshal import — deserialization risk"),
}
for mod, (severity, rule, msg) in dangerous.items():
if module_name == mod or module_name.startswith(mod + "."):
self._add(node, severity, rule, msg)
# ─────────────────────────────────────────────────────────────────────────────
# Per-plugin audit
# ─────────────────────────────────────────────────────────────────────────────
def audit_plugin(plugin_dir: Path) -> list[Finding]:
"""Audit a single plugin directory. Returns all findings."""
findings: list[Finding] = []
plugin_id = plugin_dir.name
# Check for required files
for required_file, rule, msg in [
("manifest.json", "PLUGIN-020",
"manifest.json missing — plugin may be incomplete"),
("config_schema.json", "PLUGIN-021",
"config_schema.json missing — no input validation schema declared"),
]:
if not (plugin_dir / required_file).exists():
findings.append(Finding(
plugin_id=plugin_id,
file=str((plugin_dir / required_file).relative_to(PROJECT_ROOT)),
line=0,
severity="WARNING",
rule=rule,
message=msg,
))
# AST analysis of all Python files
for py_file in sorted(plugin_dir.rglob("*.py")):
try:
source = py_file.read_text(encoding="utf-8")
tree = ast.parse(source, filename=str(py_file))
visitor = _PluginVisitor(py_file, plugin_id)
visitor.visit(tree)
findings.extend(visitor.findings)
except SyntaxError as exc:
# A file the visitor can't even parse is a file we can't verify
# is safe -- this must block the audit, not just warn.
findings.append(Finding(
plugin_id=plugin_id,
file=str(py_file.relative_to(PROJECT_ROOT)),
line=getattr(exc, "lineno", 0) or 0,
severity="CRITICAL",
rule="PLUGIN-030",
message=f"Python syntax error — cannot be parsed: {exc}",
))
except OSError as exc:
# Same reasoning as SyntaxError: an unreadable file was never
# actually scanned, so it must block rather than pass silently.
findings.append(Finding(
plugin_id=plugin_id,
file=str(py_file.relative_to(PROJECT_ROOT)),
line=0,
severity="CRITICAL",
rule="PLUGIN-031",
message=f"Could not read file: {exc}",
))
return findings
# ─────────────────────────────────────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="LEDMatrix plugin security auditor",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--plugin", "-p", default=None,
help="Audit a specific plugin ID only")
parser.add_argument("--output", "-o", default=None,
help="Write JSON results to this file")
parser.add_argument("--verbose", "-v", action="store_true",
help="Show all findings, not just summary")
args = parser.parse_args()
print("=" * 60)
print("LEDMatrix Plugin Security Audit")
print(f"Project root: {PROJECT_ROOT}")
print("=" * 60)
all_findings: list[Finding] = []
plugins_scanned = 0
plugin_found = args.plugin is None
for base_dir in PLUGIN_BASE_DIRS:
if not base_dir.exists():
if args.verbose:
print(f" ⏭️ Skipping {base_dir.name}/ (directory not found)")
continue
base_label = base_dir.relative_to(PROJECT_ROOT)
print(f"\n Scanning {base_label}/")
for plugin_dir in sorted(base_dir.iterdir()):
if not plugin_dir.is_dir():
continue
if plugin_dir.name.startswith((".", "_")):
continue
if args.plugin and plugin_dir.name != args.plugin:
continue
if args.plugin:
plugin_found = True
findings = audit_plugin(plugin_dir)
all_findings.extend(findings)
plugins_scanned += 1
critical = [f for f in findings if f.severity == "CRITICAL"]
warnings = [f for f in findings if f.severity == "WARNING"]
if critical:
icon, label = "🚨", "CRITICAL"
elif warnings:
icon, label = "⚠️ ", "WARN "
else:
icon, label = "", "PASS "
print(f" {icon} [{label}] {plugin_dir.name}"
f"{len(critical)} critical, {len(warnings)} warnings")
if args.verbose:
for f in findings:
severity_icon = {"CRITICAL": "🚨", "WARNING": "⚠️ ", "INFO": ""}.get(
f.severity, " "
)
print(f" {severity_icon} {f.rule} {f.file}:{f.line}{f.message}")
if args.plugin and not plugin_found:
print(f"\n 🚨 Plugin '{args.plugin}' not found in any of "
f"{[str(d.relative_to(PROJECT_ROOT)) for d in PLUGIN_BASE_DIRS]}"
f"nothing was audited")
return 1
# Summary
critical_findings = [f for f in all_findings if f.severity == "CRITICAL"]
warning_findings = [f for f in all_findings if f.severity == "WARNING"]
print(f"\n{'=' * 60}")
print(f" Plugins scanned : {plugins_scanned}")
print(f" CRITICAL : {len(critical_findings)}")
print(f" WARNING : {len(warning_findings)}")
if critical_findings:
print("\n 🚨 CRITICAL findings:")
for f in critical_findings:
print(f" {f.plugin_id} | {Path(f.file).name}:{f.line} | {f.message}")
# Write JSON output
if args.output:
output_data = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"plugins_scanned": plugins_scanned,
"summary": {
"critical": len(critical_findings),
"warnings": len(warning_findings),
},
"findings": [f.to_dict() for f in all_findings],
}
Path(args.output).write_text(
json.dumps(output_data, indent=2), encoding="utf-8"
)
print(f"\n Results written to: {args.output}")
if critical_findings:
print("\n 🚨 Blocking — CRITICAL issues must be resolved")
return 1
print("\n ✅ No critical issues found")
return 0
if __name__ == "__main__":
sys.exit(main())
+61 -26
View File
@@ -37,10 +37,11 @@ os.environ['EMULATOR'] = 'true'
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_config_defaults, load_harness_spec,
build_full_config, find_plugin_dir, load_harness_spec, load_manifest,
)
from src.plugin_system.testing.harness import ( # noqa: E402
RenderResult, render_plugin_matrix, compare_to_goldens, write_goldens,
check_scale_up,
)
from src.plugin_system.testing.sizes import ( # noqa: E402
parse_size_token, resolve_test_sizes, safe_mode_filename, size_label,
@@ -96,12 +97,11 @@ def check_one(plugin_id: str, search_dirs: List[str], sizes, mock_data: Dict,
# matrix path does; explicit CLI flags still override the file.
spec = load_harness_spec(plugin_dir)
# config_schema defaults (real-install behavior), then harness.json config,
# then CLI --config — most specific wins.
full_config = {"enabled": True}
full_config.update(load_config_defaults(plugin_dir))
full_config.update(spec.get("config", {}))
full_config.update(config)
# config_schema defaults (real-install behavior, with enabled forced True
# so a plugin's own enabled:false default can't accidentally disable
# testing), then harness.json config, then CLI --config — most specific
# wins.
full_config = build_full_config(plugin_dir, spec, config)
# Precedence: CLI flag > LEDMATRIX_TEST_SIZES env > harness.json > default.
effective_sizes = sizes if sizes else resolve_test_sizes(spec.get("sizes"))
@@ -110,28 +110,55 @@ def check_one(plugin_id: str, search_dirs: List[str], sizes, mock_data: Dict,
effective_freeze = freeze_time or spec.get("freeze_time")
effective_run_update = run_update and not spec.get("skip_update", False)
results = render_plugin_matrix(
plugin_id=plugin_id, plugin_dir=plugin_dir, config=full_config,
mock_data=effective_mock_data, sizes=effective_sizes,
run_update=effective_run_update, freeze_time=effective_freeze,
)
# The plugin's declared design size drives the scale-up fill check
# (panels >= 2x the design size must not be left mostly empty).
declared = load_manifest(plugin_dir).get("display", {}).get("design_size", {})
design_size = (int(declared.get("width", 128)), int(declared.get("height", 32)))
fill_strict = spec.get("fill_check") == "strict"
golden_dir = golden_dir_override or (plugin_dir / 'test' / 'golden')
if update_golden:
written = write_goldens(results, golden_dir)
logger.info("Wrote %d golden image(s) for %s to %s", written, plugin_id, golden_dir)
else:
compare_to_goldens(results, golden_dir)
# Every run: the base config, plus one per harness.json "variant" —
# a config overlay with its own golden dir (e.g. adaptive layout mode
# tested alongside the classic default).
runs = [(None, {}, golden_dir_override or (plugin_dir / 'test' / 'golden'))]
for variant in spec.get("variants", []):
name = variant.get("name") or "variant"
vdir = plugin_dir / variant.get("golden_dir", f"test/golden-{name}")
runs.append((name, variant.get("config", {}), vdir))
if out_dir:
for r in results:
if r.image is None:
continue
dest = out_dir / plugin_id / size_label(r.width, r.height)
dest.mkdir(parents=True, exist_ok=True)
r.image.save(dest / f"{safe_mode_filename(r.mode)}.png", format="PNG")
all_run_results: List[RenderResult] = []
for variant_name, overlay, golden_dir in runs:
run_config = {**full_config, **overlay}
results = render_plugin_matrix(
plugin_id=plugin_id, plugin_dir=plugin_dir, config=run_config,
mock_data=effective_mock_data, sizes=effective_sizes,
run_update=effective_run_update, freeze_time=effective_freeze,
)
return results
if update_golden:
written = write_goldens(results, golden_dir)
logger.info("Wrote %d golden image(s) for %s%s to %s", written, plugin_id,
f" [{variant_name}]" if variant_name else "", golden_dir)
else:
compare_to_goldens(results, golden_dir)
check_scale_up(results, design_size=design_size, strict=fill_strict)
# Tag variant runs so the report and PNG dumps stay distinguishable.
if variant_name:
for r in results:
r.mode = f"{r.mode}@{variant_name}"
if out_dir:
for r in results:
if r.image is None:
continue
dest = out_dir / plugin_id / size_label(r.width, r.height)
dest.mkdir(parents=True, exist_ok=True)
r.image.save(dest / f"{safe_mode_filename(r.mode)}.png", format="PNG")
all_run_results.extend(results)
return all_run_results
def print_report(all_results: Dict[str, List[RenderResult]]) -> bool:
@@ -147,6 +174,10 @@ def print_report(all_results: Dict[str, List[RenderResult]]) -> bool:
detail = " (golden ✓)"
if r.update_error is not None:
detail += f" (update warn: {r.update_error})"
if r.fill_checked and r.fill_ok is None and r.fill_extent:
# warn-only underfill: big panel left mostly empty
ex, ey = r.fill_extent
detail += f" (fill warn: extent {ex:.0%}x{ey:.0%})"
else:
everything_ok = False
if r.error is not None:
@@ -156,6 +187,10 @@ def print_report(all_results: Dict[str, List[RenderResult]]) -> bool:
elif r.golden_ok is False:
status = "FAIL"
detail = f" golden drift: {r.golden_diff_pixels}px (max Δ={r.golden_max_delta})"
elif r.fill_ok is False:
ex, ey = r.fill_extent or (0.0, 0.0)
status = "FAIL"
detail = f" fill: extent {ex:.0%}x{ey:.0%} below required coverage"
else:
status, detail = "FAIL", ""
print(f" [{status}] {r.size_label:>7} {r.mode}{detail}")
-29
View File
@@ -1,29 +0,0 @@
#!/bin/bash
# Clear all plugin dependency markers to force fresh dependency check
# Useful after updating plugins or troubleshooting dependency issues
echo "Clearing plugin dependency markers..."
# Check both possible cache locations
CACHE_DIRS=(
"/var/cache/ledmatrix"
"$HOME/.cache/ledmatrix"
)
for CACHE_DIR in "${CACHE_DIRS[@]}"; do
if [ -d "$CACHE_DIR" ]; then
echo "Checking $CACHE_DIR..."
marker_count=$(find "$CACHE_DIR" -name "plugin_*_deps_installed" 2>/dev/null | wc -l)
if [ "$marker_count" -gt 0 ]; then
echo "Found $marker_count dependency marker(s) in $CACHE_DIR"
find "$CACHE_DIR" -name "plugin_*_deps_installed" -delete
echo "Cleared $marker_count marker(s)"
else
echo "No dependency markers found in $CACHE_DIR"
fi
fi
done
echo "Done! Dependency markers cleared."
echo "Next startup will check and install dependencies as needed."
+1 -1
View File
@@ -55,7 +55,7 @@ def main():
failures += not check("draw.textbbox",
lambda: draw.textbbox((0, 0), "Test", font=font))
print("\nResampling (used in logo_helper, image_utils, sports base):")
print("\nResampling (used in logo_helper, sports base):")
logo = Image.new('RGBA', (200, 200), (255, 128, 0, 200))
failures += not check("Image.Resampling.LANCZOS exists",
lambda: str(Image.Resampling.LANCZOS))
+198 -72
View File
@@ -16,6 +16,7 @@ Opens at http://localhost:5001
import sys
import os
import json
import re
import time
import argparse
import logging
@@ -44,6 +45,10 @@ MAX_HEIGHT = 512
MIN_WIDTH = 1
MIN_HEIGHT = 1
# plugin_id arrives in request input and is used to build filesystem paths —
# allowlist it (same pattern the web UI's pages_v3 uses)
_SAFE_PLUGIN_ID_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
# --------------------------------------------------------------------------
# Plugin discovery
@@ -106,15 +111,30 @@ def discover_plugins() -> List[Dict[str, Any]]:
def find_plugin_dir(plugin_id: str) -> Optional[Path]:
"""Find a plugin directory by ID."""
"""Find a plugin directory by ID.
plugin_id comes from request input: it must pass an allowlist match,
and the resulting directory is normalized and required to live inside
one of the plugin search dirs, so a crafted id can never name a path
outside them.
"""
if not isinstance(plugin_id, str) or not _SAFE_PLUGIN_ID_RE.match(plugin_id):
return None
from src.plugin_system.plugin_loader import PluginLoader
loader = PluginLoader()
for search_dir in get_search_dirs():
if not search_dir.exists():
continue
result = loader.find_plugin_directory(plugin_id, search_dir)
if result:
return Path(result)
if not result:
continue
# Normalize WITHOUT following symlinks (dev plugins are often
# symlinked into plugins/) and require lexical containment in the
# search dir, so no id can ever name a path outside it.
result_abs = os.path.abspath(str(result))
root_abs = os.path.abspath(str(search_dir))
if os.path.commonpath([result_abs, root_abs]) == root_abs:
return Path(result_abs)
return None
@@ -176,6 +196,118 @@ def api_plugin_defaults(plugin_id):
return jsonify({'defaults': defaults})
def _render_once(plugin_id, plugin_dir, manifest, config, mock_data, width, height,
skip_update):
"""Render one plugin at one size. Returns the /api/render response dict.
A fresh plugin instance per call, mirroring the safety harness, so sizes
never share state.
"""
from src.plugin_system.testing import VisualTestDisplayManager, MockCacheManager, MockPluginManager
from src.plugin_system.plugin_loader import PluginLoader
display_manager = VisualTestDisplayManager(width=width, height=height)
cache_manager = MockCacheManager()
plugin_manager = MockPluginManager()
# Pre-populate cache with mock data
for key, value in mock_data.items():
cache_manager.set(key, value)
loader = PluginLoader()
errors = []
warnings = []
plugin_instance, _module = loader.load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=plugin_dir,
config=config,
display_manager=display_manager,
cache_manager=cache_manager,
plugin_manager=plugin_manager,
install_deps=False,
)
start_time = time.time()
# Run update()
if not skip_update:
try:
plugin_instance.update()
except Exception as e:
logger.warning("update() raised for plugin %s", plugin_id, exc_info=True)
warnings.append(f"update() raised: {type(e).__name__} — see server log")
# Run display()
try:
plugin_instance.display(force_clear=True)
except Exception as e:
logger.warning("display() raised for plugin %s", plugin_id, exc_info=True)
errors.append(f"display() raised: {type(e).__name__} — see server log")
render_time_ms = round((time.time() - start_time) * 1000, 1)
return {
'image': f'data:image/png;base64,{display_manager.get_image_base64()}',
'width': width,
'height': height,
'render_time_ms': render_time_ms,
'errors': errors,
'warnings': warnings,
}
def _trusted_plugin_dir(plugin_dir: Path) -> Optional[Path]:
"""Re-derive a plugin directory from the search dirs' own listings.
Path-injection barrier: unlike ``Path.iterdir()`` (which CodeQL doesn't
recognize as a taint-clearing enumeration), ``os.scandir()`` is. The
returned Path is built from a trusted root plus a name the filesystem
itself produced under that root via scandir — request-derived strings
never enter its construction — so a crafted plugin id can never make
downstream file access leave the plugin search dirs. Comparison is by
name, deliberately without symlink resolution (dev plugins are
commonly symlinked into plugins/).
"""
wanted_name = Path(os.path.normpath(str(plugin_dir))).name
for search_dir in get_search_dirs():
search_dir_str = str(search_dir)
try:
with os.scandir(search_dir_str) as entries:
for entry in entries:
if entry.name == wanted_name and entry.is_dir():
return Path(search_dir_str) / entry.name
except OSError:
continue
return None
def _parse_render_request(data):
"""Shared /api/render* request prep. Returns (plugin_dir, manifest, config,
mock_data, skip_update) or raises ValueError with a client message."""
plugin_id = data['plugin_id']
candidate_dir = find_plugin_dir(plugin_id)
# Never reuse `candidate_dir` past this point: it's built from
# request-derived input, and a variable reassigned only on some paths
# isn't a barrier CodeQL's flow analysis honors. `trusted_dir` is the
# sole name used below, always the scandir-sourced result.
trusted_dir = _trusted_plugin_dir(candidate_dir) if candidate_dir else None
if not trusted_dir:
raise LookupError(f'Plugin not found: {plugin_id}')
manifest_path = trusted_dir / 'manifest.json'
with open(manifest_path, 'r') as f:
manifest = json.load(f)
# Build config: schema defaults + user overrides
config = {'enabled': True}
config.update(load_config_defaults(trusted_dir))
config.update(data.get('config', {}))
return trusted_dir, manifest, config, data.get('mock_data', {}), data.get('skip_update', False)
@app.route('/api/render', methods=['POST'])
def api_render():
"""Render a plugin and return the display as base64 PNG."""
@@ -183,11 +315,6 @@ def api_render():
if not data or 'plugin_id' not in data:
return jsonify({'error': 'plugin_id is required'}), 400
plugin_id = data['plugin_id']
user_config = data.get('config', {})
mock_data = data.get('mock_data', {})
skip_update = data.get('skip_update', False)
try:
width = int(data.get('width', 128))
height = int(data.get('height', 32))
@@ -199,78 +326,77 @@ def api_render():
if not (MIN_HEIGHT <= height <= MAX_HEIGHT):
return jsonify({'error': f'height must be between {MIN_HEIGHT} and {MAX_HEIGHT}'}), 400
# Find plugin
plugin_dir = find_plugin_dir(plugin_id)
if not plugin_dir:
return jsonify({'error': f'Plugin not found: {plugin_id}'}), 404
# Load manifest
manifest_path = plugin_dir / 'manifest.json'
with open(manifest_path, 'r') as f:
manifest = json.load(f)
# Build config: schema defaults + user overrides
config_defaults = load_config_defaults(plugin_dir)
config = {'enabled': True}
config.update(config_defaults)
config.update(user_config)
# Create display manager and mocks
from src.plugin_system.testing import VisualTestDisplayManager, MockCacheManager, MockPluginManager
from src.plugin_system.plugin_loader import PluginLoader
display_manager = VisualTestDisplayManager(width=width, height=height)
cache_manager = MockCacheManager()
plugin_manager = MockPluginManager()
# Pre-populate cache with mock data
for key, value in mock_data.items():
cache_manager.set(key, value)
# Load plugin
loader = PluginLoader()
errors = []
warnings = []
try:
plugin_dir, manifest, config, mock_data, skip_update = _parse_render_request(data)
except LookupError:
return jsonify({'error': f"Plugin not found: {data['plugin_id']}"}), 404
except Exception:
# Bad manifest.json / schema / fixture — details go to the dev's
# console, not the HTTP response
app.logger.exception('render request preparation failed')
return jsonify({'error': 'Could not prepare render request; see server log'}), 400
try:
plugin_instance, module = loader.load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=plugin_dir,
config=config,
display_manager=display_manager,
cache_manager=cache_manager,
plugin_manager=plugin_manager,
install_deps=False,
)
except Exception as e:
return jsonify({'error': f'Failed to load plugin: {e}'}), 500
result = _render_once(data['plugin_id'], plugin_dir, manifest, config,
mock_data, width, height, skip_update)
except Exception:
app.logger.exception('plugin load failed during render')
return jsonify({'error': 'Failed to load plugin; see server log'}), 500
return jsonify(result)
start_time = time.time()
# Run update()
if not skip_update:
@app.route('/api/sizes')
def api_sizes():
"""The representative panel-size sample the safety harness renders at."""
from src.plugin_system.testing.sizes import DEFAULT_TEST_SIZES
return jsonify({'sizes': [list(s) for s in DEFAULT_TEST_SIZES]})
MAX_MATRIX_SIZES = 12
@app.route('/api/render-matrix', methods=['POST'])
def api_render_matrix():
"""Render a plugin at a list of sizes (default: the harness sample) so the
UI can show a side-by-side multi-resolution gallery."""
data = request.get_json()
if not data or 'plugin_id' not in data:
return jsonify({'error': 'plugin_id is required'}), 400
from src.plugin_system.testing.sizes import DEFAULT_TEST_SIZES
sizes = data.get('sizes') or [list(s) for s in DEFAULT_TEST_SIZES]
if len(sizes) > MAX_MATRIX_SIZES:
return jsonify({'error': f'at most {MAX_MATRIX_SIZES} sizes per request'}), 400
parsed_sizes = []
for pair in sizes:
try:
plugin_instance.update()
except Exception as e:
warnings.append(f"update() raised: {e}")
w, h = int(pair[0]), int(pair[1])
except (TypeError, ValueError, IndexError):
return jsonify({'error': f'invalid size entry {pair!r} (expected [w, h])'}), 400
if not (MIN_WIDTH <= w <= MAX_WIDTH and MIN_HEIGHT <= h <= MAX_HEIGHT):
return jsonify({'error': f'size {w}x{h} out of bounds'}), 400
parsed_sizes.append((w, h))
# Run display()
try:
plugin_instance.display(force_clear=True)
except Exception as e:
errors.append(f"display() raised: {e}")
plugin_dir, manifest, config, mock_data, skip_update = _parse_render_request(data)
except LookupError:
return jsonify({'error': f"Plugin not found: {data['plugin_id']}"}), 404
except Exception:
app.logger.exception('render request preparation failed')
return jsonify({'error': 'Could not prepare render request; see server log'}), 400
render_time_ms = round((time.time() - start_time) * 1000, 1)
return jsonify({
'image': f'data:image/png;base64,{display_manager.get_image_base64()}',
'width': width,
'height': height,
'render_time_ms': render_time_ms,
'errors': errors,
'warnings': warnings,
})
results = []
for w, h in parsed_sizes:
try:
results.append(_render_once(data['plugin_id'], plugin_dir, manifest,
config, mock_data, w, h, skip_update))
except Exception:
app.logger.exception('plugin load failed during %dx%d render', w, h)
results.append({'image': None, 'width': w, 'height': h,
'render_time_ms': 0,
'errors': ['Failed to load plugin; see server log'],
'warnings': []})
return jsonify({'results': results})
# --------------------------------------------------------------------------
+74
View File
@@ -0,0 +1,74 @@
#!/bin/bash
# safe_pip_install.sh — Install a requirements.txt as root after validating
# that the resolved path is the project's own requirements.txt or a plugin's
# requirements.txt under plugin-repos/ or plugins/.
#
# This script is intended to be called via sudo from the web interface, so
# that packages a plugin declares end up visible to ledmatrix.service (which
# runs as root) rather than only to whichever non-root user runs the web
# interface. Plugin code already runs as root once loaded, so installing its
# declared dependencies as root is not a new trust boundary.
#
# Usage: safe_pip_install.sh <requirements_txt_path>
set -euo pipefail
if [ $# -ne 1 ]; then
echo "Usage: $0 <requirements_txt_path>" >&2
exit 1
fi
TARGET="$1"
# Determine the project root (parent of scripts/fix_perms/)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Allowed locations (resolved, no trailing slash):
# - the project's own requirements.txt
# - any requirements.txt under plugin-repos/ or plugins/
ALLOWED_EXACT="$(realpath --canonicalize-missing "$PROJECT_ROOT/requirements.txt")"
ALLOWED_BASES=(
"$(realpath --canonicalize-missing "$PROJECT_ROOT/plugin-repos")"
"$(realpath --canonicalize-missing "$PROJECT_ROOT/plugins")"
)
# Resolve the target path (follow symlinks); works even if it doesn't exist.
RESOLVED_TARGET="$(realpath --canonicalize-missing "$TARGET")"
# Must be named requirements.txt — never install from an arbitrary file.
if [ "$(basename "$RESOLVED_TARGET")" != "requirements.txt" ]; then
echo "DENIED: $RESOLVED_TARGET is not a requirements.txt file" >&2
exit 2
fi
ALLOWED=false
if [ "$RESOLVED_TARGET" = "$ALLOWED_EXACT" ]; then
ALLOWED=true
else
for BASE in "${ALLOWED_BASES[@]}"; do
if [[ "$RESOLVED_TARGET" == "$BASE/"* ]]; then
ALLOWED=true
break
fi
done
fi
if [ "$ALLOWED" = false ]; then
echo "DENIED: $RESOLVED_TARGET is not an allowed requirements.txt location" >&2
echo "Allowed: $ALLOWED_EXACT, or any requirements.txt under: ${ALLOWED_BASES[*]}" >&2
exit 2
fi
if [ ! -f "$RESOLVED_TARGET" ]; then
echo "ERROR: $RESOLVED_TARGET does not exist" >&2
exit 3
fi
PYTHON_PATH="$(command -v python3)"
# --ignore-installed: root's site-packages often has apt/dpkg-managed copies
# of common libraries (requests, urllib3, ...) with no pip RECORD file, which
# pip refuses to uninstall in place ("Cannot uninstall: no RECORD file was
# found"). This tells pip to install the newer version alongside rather than
# aborting the whole requirements.txt install over one such conflict.
exec "$PYTHON_PATH" -m pip install --break-system-packages --ignore-installed -r "$RESOLVED_TARGET"
+356
View File
@@ -0,0 +1,356 @@
#!/usr/bin/env python3
"""
Security Report Generator
Aggregates JSON output from all CI security audit jobs into a single
Markdown report suitable for PR comments and artifact storage.
Expected artifact layout (from actions/download-artifact@v4):
<artifact-dir>/
sast-results/
bandit-results.json
semgrep-results.json
dependency-audit-results/
pip-audit-results.json
safety-results.json
secrets-scan-results/
gitleaks-results.json
security-proofs-results/
security-proofs-results.json
plugin-audit-results/
plugin-audit-results.json
Usage:
python scripts/generate_report.py --artifact-dir audit-artifacts/ --output report.md
python scripts/generate_report.py --artifact-dir audit-artifacts/ --output report.md --verbose
"""
import argparse
import json
import sys
from pathlib import Path
from datetime import datetime, timezone
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# Gitleaks matches exactly equal to one of these (not a substring match -- a
# real secret that merely contains one of these words as part of its actual
# value must still be reported) are known template placeholders.
_GITLEAKS_SUPPRESS_EXACT_VALUES = {
"YOUR_YOUTUBE_API_KEY",
"YOUR_YOUTUBE_CHANNEL_ID",
"YOUR_GITHUB_PERSONAL_ACCESS_TOKEN",
}
# Findings in these files are suppressed regardless of value -- they are
# template/example files that are expected to only ever contain placeholders.
_GITLEAKS_SUPPRESS_PATHS = [
"config_secrets.template.json",
"config.template.json",
]
# ─────────────────────────────────────────────────────────────────────────────
# Helpers
# ─────────────────────────────────────────────────────────────────────────────
def _load(path: Path) -> tuple[dict | list | None, str | None]:
"""Load a JSON artifact file.
Returns (data, error): error is None on success (data is whatever was
parsed, which may legitimately be an empty list/dict for a clean scan);
otherwise error is a human-readable reason the artifact is unavailable,
distinguishing "missing/malformed artifact" from "valid empty result" so
callers don't silently treat a broken CI job as a clean pass.
"""
if not path.exists():
return None, f"artifact not found: {path}"
try:
return json.loads(path.read_text(encoding="utf-8")), None
except (json.JSONDecodeError, OSError) as exc:
return None, f"could not read/parse {path}: {exc}"
def _md_sanitize_cell(value: object) -> str:
"""Escape/normalize a value so scanner-controlled content (a matched
secret, a bandit issue_text, a file path) can't alter the Markdown
table's structure: pipes would add bogus columns, newlines would break
out of the row (or forge a fake header/separator line)."""
text = str(value)
text = text.replace("\\", "\\\\").replace("|", "\\|")
text = text.replace("\r\n", " ").replace("\n", " ").replace("\r", " ")
return text
def _md_table_row(*cells: str) -> str:
return "| " + " | ".join(_md_sanitize_cell(c) for c in cells) + " |"
# ─────────────────────────────────────────────────────────────────────────────
# Per-tool summarizers
# Returns: (markdown_lines: list[str], critical_count: int, available: bool)
# `available=False` means the artifact was missing or malformed -- distinct
# from a valid scan that simply found nothing -- so the caller can report
# INCOMPLETE instead of silently counting it as a clean pass.
# ─────────────────────────────────────────────────────────────────────────────
def _summarize_bandit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "sast-results" / "bandit-results.json")
if error:
return [f"_bandit results unavailable: {error}_"], 0, False
results = data.get("results", [])
high = [r for r in results if r.get("issue_severity") == "HIGH"]
medium = [r for r in results if r.get("issue_severity") == "MEDIUM"]
low = [r for r in results if r.get("issue_severity") == "LOW"]
lines = [
f"**Bandit**: {len(high)} HIGH · {len(medium)} MEDIUM · {len(low)} LOW"
]
if high:
lines += [
"",
"| Severity | File | Line | Issue |",
"| --- | --- | --- | --- |",
]
for r in high[:10]:
fname = Path(r.get("filename", "")).name
lines.append(_md_table_row(
"HIGH", f"`{fname}`",
str(r.get("line_number", "?")),
r.get("issue_text", "")
))
if len(high) > 10:
lines.append(f"_… and {len(high) - 10} more HIGH findings_")
return lines, len(high), True
def _summarize_pip_audit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "dependency-audit-results" / "pip-audit-results.json")
if error:
return [f"_pip-audit results unavailable: {error}_"], 0, False
# pip-audit JSON format: {"dependencies": [{"name": ..., "vulns": [...]}]}
vulns: list[dict] = []
for dep in data.get("dependencies", []):
for v in dep.get("vulns", []):
vulns.append({"package": dep.get("name", "?"), **v})
lines = [f"**pip-audit**: {len(vulns)} vulnerabilities found"]
if vulns:
lines += ["", "| Package | ID | Fix |", "| --- | --- | --- |"]
for v in vulns[:10]:
fix = v.get("fix_versions", ["none"])
fix_str = ", ".join(fix) if fix else "none"
lines.append(_md_table_row(
v.get("package", "?"),
v.get("id", "?"),
fix_str,
))
# Treat known vulnerabilities as warnings, not critical (they may be unavoidable)
return lines, 0, True
def _summarize_gitleaks(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "secrets-scan-results" / "gitleaks-results.json")
if error:
return [f"_gitleaks results unavailable: {error}_"], 0, False
if not isinstance(data, list):
data = []
real_findings = []
suppressed = 0
for finding in data:
secret_val = str(finding.get("Secret", "") or finding.get("Match", ""))
file_name = Path(finding.get("File", "")).name
if (secret_val in _GITLEAKS_SUPPRESS_EXACT_VALUES
or file_name in _GITLEAKS_SUPPRESS_PATHS):
suppressed += 1
else:
real_findings.append(finding)
lines = [
f"**Gitleaks**: {len(real_findings)} finding(s) "
f"({suppressed} suppressed as template placeholders)"
]
if real_findings:
lines += ["", "| Rule | File | Line | Description |", "| --- | --- | --- | --- |"]
for f in real_findings[:10]:
fname = Path(f.get("File", "")).name
lines.append(_md_table_row(
f.get("RuleID", "?"),
f"`{fname}`",
str(f.get("StartLine", "?")),
f.get("Description", ""),
))
critical = len(real_findings) # any real secret is critical
return lines, critical, True
def _summarize_security_proofs(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "security-proofs-results" / "security-proofs-results.json")
if error:
return [f"_security proofs results unavailable: {error}_"], 0, False
if not isinstance(data, list):
data = []
critical = [r for r in data if r.get("severity") == "CRITICAL"]
warnings = [r for r in data if r.get("severity") == "WARNING"]
passed = [r for r in data if r.get("severity") == "PASS"]
skipped = [r for r in data if r.get("severity") == "SKIP"]
lines = [
f"**Security Proofs**: "
f"{len(passed)} PASS · {len(warnings)} WARN · "
f"{len(critical)} CRITICAL · {len(skipped)} SKIP",
"",
]
_icon = {"PASS": "", "INFO": "", "WARNING": "⚠️", # nosec B105 - severity labels, not credentials
"CRITICAL": "🚨", "SKIP": "⏭️"}
for r in data:
icon = _icon.get(r.get("severity", ""), "")
lines.append(
f"- {icon} **{r.get('test_id', '?')}**: {r.get('message', '')}"
)
if r.get("details") and r.get("severity") in ("CRITICAL", "WARNING"):
lines.append(f" - _{r['details']}_")
return lines, len(critical), True
def _summarize_plugin_audit(artifact_dir: Path) -> tuple[list[str], int, bool]:
data, error = _load(artifact_dir / "plugin-audit-results" / "plugin-audit-results.json")
if error:
return [f"_plugin audit results unavailable: {error}_"], 0, False
summary = data.get("summary", {})
findings = data.get("findings", [])
critical_findings = [f for f in findings if f.get("severity") == "CRITICAL"]
warning_findings = [f for f in findings if f.get("severity") == "WARNING"]
lines = [
f"**Plugin Audit**: {data.get('plugins_scanned', '?')} plugins scanned — "
f"{summary.get('critical', 0)} CRITICAL · {summary.get('warnings', 0)} WARNINGS"
]
if critical_findings:
lines += ["", "| Plugin | File | Line | Rule | Message |",
"| --- | --- | --- | --- | --- |"]
for f in critical_findings[:10]:
fname = Path(f.get("file", "")).name
lines.append(_md_table_row(
f.get("plugin_id", "?"),
f"`{fname}`",
str(f.get("line", "?")),
f.get("rule", "?"),
f.get("message", ""),
))
if warning_findings and not critical_findings:
lines.append(f"\n_{len(warning_findings)} warning(s) found — see artifact for details_")
return lines, summary.get("critical", 0), True
# ─────────────────────────────────────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="Generate consolidated security audit report",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--artifact-dir", required=True,
help="Directory containing downloaded CI artifacts")
parser.add_argument("--output", "-o", required=True,
help="Output Markdown file path")
parser.add_argument("--verbose", "-v", action="store_true")
args = parser.parse_args()
artifact_dir = Path(args.artifact_dir)
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
bandit_lines, bandit_crit, bandit_ok = _summarize_bandit(artifact_dir)
pip_audit_lines, pip_audit_crit, pip_audit_ok = _summarize_pip_audit(artifact_dir)
gitleaks_lines, gitleaks_crit, gitleaks_ok = _summarize_gitleaks(artifact_dir)
proofs_lines, proofs_crit, proofs_ok = _summarize_security_proofs(artifact_dir)
plugins_lines, plugins_crit, plugins_ok = _summarize_plugin_audit(artifact_dir)
unavailable_tools = [
name for name, ok in [
("bandit", bandit_ok), ("pip-audit", pip_audit_ok),
("gitleaks", gitleaks_ok), ("security-proofs", proofs_ok),
("plugin-audit", plugins_ok),
] if not ok
]
total_critical = bandit_crit + pip_audit_crit + gitleaks_crit + proofs_crit + plugins_crit
if unavailable_tools:
# A missing/malformed artifact means that tool's checks never
# actually ran -- this must not be reported as a clean PASS just
# because the *artifacts that did load* found nothing.
overall = "INCOMPLETE ⚠️"
elif total_critical > 0:
overall = "ACTION REQUIRED 🚨"
else:
overall = "PASSED ✅"
def section(title: str, lines: list[str]) -> str:
return f"### {title}\n\n" + "\n".join(lines) + "\n"
incomplete_note = (
f"\n_⚠️ Incomplete: results unavailable for {', '.join(unavailable_tools)} "
f"— see the corresponding section(s) below for details_\n"
if unavailable_tools else ""
)
report = f"""## 🔒 Security Audit — {overall}
_Generated: {timestamp}_
{incomplete_note}
| Critical | High/Warn | Overall |
| :---: | :---: | :---: |
| {'🚨 ' + str(total_critical) if total_critical else '✅ 0'} | ⚠️ see below | {overall} |
---
{section('SAST — Bandit', bandit_lines)}
{section('Dependencies — pip-audit', pip_audit_lines)}
{section('Secrets — Gitleaks', gitleaks_lines)}
{section('LEDMatrix Security Proofs', proofs_lines)}
{section('Plugin Security Audit', plugins_lines)}
---
_Total critical findings: **{total_critical}**_
"""
output_path = Path(args.output)
output_path.write_text(report, encoding="utf-8")
if args.verbose:
print(f" Report written to: {output_path}")
print(f" Status: {overall}")
print(f" Critical findings: {total_critical}")
print(f" bandit={bandit_crit} pip-audit={pip_audit_crit} "
f"gitleaks={gitleaks_crit} proofs={proofs_crit} plugins={plugins_crit}")
if unavailable_tools:
print(f" Unavailable: {', '.join(unavailable_tools)}")
if unavailable_tools:
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+28 -5
View File
@@ -33,6 +33,7 @@ POWEROFF_PATH=$(command -v poweroff) || true
BASH_PATH=$(command -v bash) || true
JOURNALCTL_PATH=$(command -v journalctl) || true
SAFE_RM_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_plugin_rm.sh"
SAFE_PIP_INSTALL_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh"
# Validate required commands (systemctl, bash, python3 are essential)
for CMD_NAME in SYSTEMCTL_PATH BASH_PATH PYTHON_PATH; do
@@ -48,11 +49,15 @@ if [ ${#MISSING_CMDS[@]} -gt 0 ]; then
exit 1
fi
# Validate helper script exists
# Validate helper scripts exist
if [ ! -f "$SAFE_RM_PATH" ]; then
echo "Error: Safe plugin removal helper not found: $SAFE_RM_PATH" >&2
exit 1
fi
if [ ! -f "$SAFE_PIP_INSTALL_PATH" ]; then
echo "Error: Safe pip install helper not found: $SAFE_PIP_INSTALL_PATH" >&2
exit 1
fi
echo "Command paths:"
echo " Python: $PYTHON_PATH"
@@ -62,6 +67,7 @@ echo " Poweroff: ${POWEROFF_PATH:-(not found, skipping)}"
echo " Bash: $BASH_PATH"
echo " Journalctl: ${JOURNALCTL_PATH:-(not found, skipping)}"
echo " Safe plugin rm: $SAFE_RM_PATH"
echo " Safe pip install: $SAFE_PIP_INSTALL_PATH"
# Create a temporary sudoers file
TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
@@ -101,13 +107,22 @@ TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
fi
# Required: python3, bash
echo "$WEB_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_DIR/display_controller.py"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_DIR/start_display.sh"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_DIR/stop_display.sh"
# NOTE: display_controller.py/start_display.sh/stop_display.sh live at the
# project root, not under scripts/install/ (where this script lives) —
# must use PROJECT_ROOT here, not PROJECT_DIR.
echo "$WEB_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT/display_controller.py"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/start_display.sh"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/stop_display.sh"
echo ""
echo "# Allow web user to remove plugin directories via vetted helper script"
echo "# The helper validates that the target path resolves inside plugin-repos/ or plugins/"
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $SAFE_RM_PATH *"
echo ""
echo "# Allow web user to install a plugin's requirements.txt as root via vetted"
echo "# helper script, so packages are visible to root-run ledmatrix.service"
echo "# (not just the web interface's own user). The helper validates the target"
echo "# is requirements.txt at the project root or under plugin-repos/ or plugins/."
echo "$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $SAFE_PIP_INSTALL_PATH *"
} > "$TEMP_SUDOERS"
echo ""
@@ -126,6 +141,7 @@ echo "- Run display_controller.py directly"
echo "- Execute start_display.sh and stop_display.sh"
echo "- Reboot and shutdown the system"
echo "- Remove plugin directories (for update/uninstall when root-owned files block deletion)"
echo "- Install plugin/base requirements.txt as root (so ledmatrix.service can see them)"
echo ""
# Ask for confirmation
@@ -147,6 +163,13 @@ fi
if ! sudo chmod 755 "$SAFE_RM_PATH"; then
echo "Warning: Could not set permissions on $SAFE_RM_PATH"
fi
echo "Hardening safe_pip_install.sh ownership..."
if ! sudo chown root:root "$SAFE_PIP_INSTALL_PATH"; then
echo "Warning: Could not set ownership on $SAFE_PIP_INSTALL_PATH"
fi
if ! sudo chmod 755 "$SAFE_PIP_INSTALL_PATH"; then
echo "Warning: Could not set permissions on $SAFE_PIP_INSTALL_PATH"
fi
if sudo cp "$TEMP_SUDOERS" /etc/sudoers.d/ledmatrix_web; then
echo "Configuration applied successfully!"
@@ -160,7 +183,7 @@ if sudo cp "$TEMP_SUDOERS" /etc/sudoers.d/ledmatrix_web; then
echo "✗ systemctl status ledmatrix.service - Failed"
fi
if sudo -n test -f "$PROJECT_DIR/start_display.sh"; then
if sudo -n test -f "$PROJECT_ROOT/start_display.sh"; then
echo "✓ File access test - OK"
else
echo "✗ File access test - Failed"
+593
View File
@@ -0,0 +1,593 @@
#!/usr/bin/env python3
"""
LEDMatrix Security Proof Tests
Automated proofs that run in CI to verify security properties hold on every
commit. Inspired by the Huntarr security review approach of using standard
tooling to confirm specific vulnerability classes are absent.
Usage:
python scripts/prove_security.py
python scripts/prove_security.py --verbose
python scripts/prove_security.py --output results.json
Exit code: 1 only if CRITICAL findings are detected. Warnings are reported
but do not block CI.
"""
import ast
import argparse
import hashlib
import json
import re
import sys
from dataclasses import dataclass, asdict
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# ─────────────────────────────────────────────────────────────────────────────
# Result dataclass
# ─────────────────────────────────────────────────────────────────────────────
@dataclass
class TestResult:
test_id: str
severity: str # PASS | INFO | WARNING | CRITICAL | SKIP
message: str
details: str = ""
def to_dict(self) -> dict:
return asdict(self)
@property
def icon(self) -> str:
return {
"PASS": "", # nosec B105 - severity label, not a credential
"INFO": "",
"WARNING": "⚠️ ",
"CRITICAL": "🚨",
"SKIP": "⏭️ ",
}.get(self.severity, "")
# ─────────────────────────────────────────────────────────────────────────────
# T1: Plugin Loading / Zip Slip
# ─────────────────────────────────────────────────────────────────────────────
def test_t1a_zip_slip_protection() -> TestResult:
"""
Verify that zip-slip protection actually guards zip extraction in
store_manager.py.
A whole-file substring check for "is_relative_to"/"Zip-slip detected"
would pass even if the guard existed somewhere unrelated, or covered
only one of several extract()/extractall() call sites. Instead, this
walks the AST: for every extract()/extractall() call, it confirms an
is_relative_to() check (and the "Zip-slip detected" log) appears
earlier in that same enclosing function -- validate-then-bulk-extract
(validate every member, then call extractall() only after all passed)
counts as protecting the call, since it covers the same member list.
"""
store_manager = PROJECT_ROOT / "src" / "plugin_system" / "store_manager.py"
if not store_manager.exists():
return TestResult("T1a", "CRITICAL",
"store_manager.py not found",
f"Expected at {store_manager}")
content = store_manager.read_text(encoding="utf-8")
try:
tree = ast.parse(content, filename=str(store_manager))
except SyntaxError as exc:
return TestResult("T1a", "CRITICAL",
"store_manager.py could not be parsed",
str(exc))
extraction_sites = 0
unprotected: list[str] = []
for func in ast.walk(tree):
if not isinstance(func, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
extract_calls = [
node for node in ast.walk(func)
if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute)
and node.func.attr in ("extract", "extractall")
]
if not extract_calls:
continue
extraction_sites += len(extract_calls)
guard_lines = [
n.lineno for n in ast.walk(func)
if isinstance(n, ast.Attribute) and n.attr == "is_relative_to"
]
has_zip_slip_log = any(
isinstance(n, ast.Constant) and isinstance(n.value, str)
and "Zip-slip detected" in n.value
for n in ast.walk(func)
)
for call in extract_calls:
guarded = has_zip_slip_log and any(g < call.lineno for g in guard_lines)
if not guarded:
unprotected.append(
f"{func.name}() line {call.lineno}: {call.func.attr}() call not "
f"clearly preceded by an is_relative_to() guard + Zip-slip log "
f"in the same function"
)
if extraction_sites == 0:
return TestResult("T1a", "WARNING",
"No zipfile extract()/extractall() calls found in store_manager.py",
"Verify plugin installation no longer extracts zip archives, "
"or that this check still targets the right file")
if unprotected:
return TestResult("T1a", "CRITICAL",
f"{len(unprotected)} of {extraction_sites} zip extraction "
f"call(s) not clearly guarded",
"; ".join(unprotected))
return TestResult("T1a", "PASS",
"Zip-slip protection verified",
f"All {extraction_sites} extract()/extractall() call(s) in "
f"store_manager.py are preceded by an is_relative_to() guard "
f"with a Zip-slip log in the same function")
def test_t1b_dangerous_plugin_calls() -> list[TestResult]:
"""
Scan plugin directories for dangerous function calls (eval, exec).
These represent arbitrary code execution risks in plugin code.
"""
results = []
plugin_dirs = [
PROJECT_ROOT / "plugins",
PROJECT_ROOT / "plugin-repos",
]
violations: list[str] = []
files_scanned = 0
scan_errors: list[str] = []
for base in plugin_dirs:
if not base.exists():
continue
for plugin_dir in sorted(base.iterdir()):
if not plugin_dir.is_dir() or plugin_dir.name.startswith(('.', '_')):
continue
for py_file in plugin_dir.rglob("*.py"):
files_scanned += 1
try:
source = py_file.read_text(encoding="utf-8")
tree = ast.parse(source, filename=str(py_file))
for node in ast.walk(tree):
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
if node.func.id in ("eval", "exec"):
rel = py_file.relative_to(PROJECT_ROOT)
violations.append(
f"{rel}:{node.lineno}{node.func.id}() call")
except (SyntaxError, OSError) as exc:
# A file we couldn't parse/read was never actually
# scanned for eval()/exec() -- that must block this
# test, not silently pass as if it were clean.
rel = py_file.relative_to(PROJECT_ROOT)
scan_errors.append(f"{rel}{type(exc).__name__}: {exc}")
if scan_errors:
results.append(TestResult(
"T1b", "CRITICAL",
f"{len(scan_errors)} plugin file(s) could not be scanned for eval()/exec()",
"; ".join(scan_errors[:10])
))
if violations:
results.append(TestResult(
"T1b", "CRITICAL",
f"Dangerous function calls found in plugins ({len(violations)} instance(s))",
"; ".join(violations[:10])
))
elif not scan_errors:
results.append(TestResult(
"T1b", "PASS",
"No eval()/exec() calls found in plugins",
f"{files_scanned} plugin Python files scanned"
))
return results
# ─────────────────────────────────────────────────────────────────────────────
# T2: API Surface Inventory
# ─────────────────────────────────────────────────────────────────────────────
def test_t2a_api_surface_inventory() -> TestResult:
"""
Document the API surface area.
This app intentionally has no authentication (local-only Raspberry Pi
design, documented in web_interface/app.py). This test produces an
inventory for audit purposes and warns only if the design-intent comment
is removed from app.py (which would indicate someone deleted the rationale
without adding auth, rather than a deliberate undocumented change).
"""
api_file = PROJECT_ROOT / "web_interface" / "blueprints" / "api_v3.py"
app_file = PROJECT_ROOT / "web_interface" / "app.py"
if not api_file.exists():
return TestResult("T2a", "WARNING", "api_v3.py not found", str(api_file))
api_content = api_file.read_text(encoding="utf-8")
routes = re.findall(r"@api_v3\.route\('([^']+)'", api_content)
csrf_documented = False
if app_file.exists():
app_content = app_file.read_text(encoding="utf-8")
csrf_documented = "CSRF protection disabled for local-only" in app_content
summary = (
f"{len(routes)} API routes in api_v3.py. "
f"No auth decorators (intentional local-only design). "
f"CSRF disabled: {'YES — design intent documented in app.py' if csrf_documented else 'YES — but design intent comment NOT found in app.py'}. "
f"Rate limiting: 1000/min."
)
if not csrf_documented:
return TestResult(
"T2a", "WARNING",
"CSRF is disabled but the design-intent comment is missing from app.py",
"Add the rationale comment back, or add proper CSRF protection if "
"the app is now internet-facing"
)
# There is currently no config mechanism that actually enforces the
# local-only boundary the design-intent comment describes -- app.py
# hardcodes host='0.0.0.0' unconditionally, so nothing here can confirm
# this deployment is in fact LAN-only. Reporting this as mere INFO
# understates that: an unauthenticated, CSRF-disabled API surface is a
# real risk the moment this ever runs somewhere other than a home LAN,
# documented rationale or not.
return TestResult(
"T2a", "WARNING",
"API surface has no auth and CSRF disabled; enforcement of the "
"documented local-only boundary cannot be confirmed",
summary
)
# ─────────────────────────────────────────────────────────────────────────────
# T3: Secrets & Credential Handling
# ─────────────────────────────────────────────────────────────────────────────
# Patterns that suggest real credentials (must be >8 chars, not placeholders)
_SECRET_PATTERNS = [
(r'(?i)password\s*=\s*["\'](?!none|empty|placeholder|example|test|default|""|'')[^"\']{8,}["\']', "WARNING", "password"),
(r'(?i)api[_-]?key\s*=\s*["\'](?!none|empty|placeholder|YOUR_|example|test)[^"\']{16,}["\']', "WARNING", "api_key"),
(r'(?i)secret\s*=\s*["\'](?!none|empty|placeholder|YOUR_|example|test)[^"\']{16,}["\']', "WARNING", "secret"),
# Real GitHub token pattern
(r'ghp_[a-zA-Z0-9]{36}', "CRITICAL", "github_token"),
# Generic long bearer tokens
(r'Bearer\s+[a-zA-Z0-9\-_\.]{32,}', "WARNING", "bearer_token"),
]
_TEMPLATE_SKIP_STRINGS = [
"YOUR_", "PLACEHOLDER", "_HERE", "example.com", "config_secrets.template",
"prove_security", # this file itself
]
_SCAN_DIRS = ["src", "web_interface", "scripts"]
def test_t3a_hardcoded_secrets() -> TestResult:
"""Scan source code for hardcoded credentials."""
violations: list[str] = []
for dir_name in _SCAN_DIRS:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
# Skip test files and this script
if "test" in str(py_file).lower() or "prove_security" in str(py_file):
continue
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
for pattern, severity, pattern_type in _SECRET_PATTERNS:
for match in re.finditer(pattern, content):
line_content = match.group(0)
# Skip lines containing template placeholder strings.
# line_content is only used for this in-memory check --
# it must never be stored or included in output below.
if any(skip in line_content for skip in _TEMPLATE_SKIP_STRINGS):
continue
rel = py_file.relative_to(PROJECT_ROOT)
line_no = content[: match.start()].count("\n") + 1
# Redacted fingerprint lets the same finding be recognized
# across scans without ever reporting the matched
# credential itself (which would otherwise get published
# into CI logs, JSON artifacts, and PR comments -- wider
# exposure than the original leak).
fingerprint = hashlib.sha256(line_content.encode()).hexdigest()[:12]
violations.append(
f"[{severity}] {rel}:{line_no}{pattern_type} "
f"(fingerprint {fingerprint})"
)
critical_violations = [v for v in violations if "[CRITICAL]" in v]
if critical_violations:
return TestResult(
"T3a", "CRITICAL",
f"Hardcoded secrets found ({len(critical_violations)} critical)",
"; ".join(critical_violations[:5])
)
if violations:
return TestResult(
"T3a", "WARNING",
f"Potential hardcoded secrets found ({len(violations)} instance(s))",
"; ".join(violations[:5])
)
return TestResult("T3a", "PASS", "No hardcoded secrets detected",
f"Scanned {', '.join(_SCAN_DIRS)}")
def test_t3b_plaintext_password_storage() -> TestResult:
"""
Check for user account password storage without hashing.
The LEDMatrix app has no user account system, so this should produce INFO.
It would only CRITICAL if someone added user auth and stored passwords without hashing.
We require all three of: a password *variable assignment or DB operation*,
a clear storage call (INSERT / db commit / ORM save), and no hashing lib present
— to avoid false positives from files that contain 'password' for WiFi handling
and '.save()' for image/file saving in unrelated functions.
"""
hashing_libs = ["bcrypt", "argon2", "pbkdf2", "scrypt",
"generate_password_hash", "hashpw", "make_password"]
# Patterns that indicate password being stored in a database / ORM context.
# Must be specific enough to avoid matching set.add(), file.save(), etc.
db_storage_patterns = ["INSERT INTO", "db.session", "session.add(", "session.commit(", "orm.save"]
password_storage_found = False
for dir_name in _SCAN_DIRS:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
# Require DB/ORM context specifically — not just any .save() call
if ("password" in content.lower() and
any(store in content for store in db_storage_patterns) and
not any(h in content for h in hashing_libs)):
password_storage_found = True
if password_storage_found:
return TestResult(
"T3b", "CRITICAL",
"Potential plaintext password storage in database/ORM detected",
"Found password + database storage operations without a recognized hashing library"
)
return TestResult("T3b", "INFO",
"No plaintext password storage detected",
"App has no user account system — expected result")
# ─────────────────────────────────────────────────────────────────────────────
# T4: Path Traversal
# ─────────────────────────────────────────────────────────────────────────────
def test_t4a_path_traversal() -> TestResult:
"""
Verify static file serving uses send_from_directory (safe) rather than
open() with user-supplied paths. Also checks for extractall() calls that
lack the is_relative_to() guard.
"""
issues: list[str] = []
app_file = PROJECT_ROOT / "web_interface" / "app.py"
if app_file.exists():
content = app_file.read_text(encoding="utf-8")
# The file-serve route should use send_from_directory or commonpath
if "send_from_directory" not in content and "commonpath" not in content:
issues.append("app.py: file-serve routes may not use send_from_directory/commonpath")
# Check all extractall() calls have a preceding is_relative_to guard
for py_file in (PROJECT_ROOT / "src").rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
if "extractall(" in content and "is_relative_to" not in content:
rel = py_file.relative_to(PROJECT_ROOT)
issues.append(f"{rel}: extractall() without is_relative_to() guard")
if issues:
return TestResult(
"T4a", "WARNING",
f"Potential path traversal patterns found ({len(issues)})",
"; ".join(issues)
)
return TestResult("T4a", "PASS",
"Path traversal mitigations verified",
"send_from_directory/commonpath used for file serving; "
"extractall() calls have is_relative_to() guards")
# ─────────────────────────────────────────────────────────────────────────────
# T5: Auth Bypass Patterns
# ─────────────────────────────────────────────────────────────────────────────
def test_t5a_auth_bypass_patterns() -> TestResult:
"""
Look for broken auth bypass patterns — not the intentional no-auth design
(T2a covers that), but patterns that suggest auth was INTENDED to exist
but has an exploitable bypass: broad substring matching, debug-mode skips,
or if-True conditions.
"""
bypass_signals = [
(r'if\s+True\s*:', "if True: bypass"),
(r'if\s+debug\s*:', "debug-mode auth skip"),
(r'request\.path\s+in\s+', "substring path matching in auth (Huntarr pattern)"),
(r'EXEMPT_ROUTES\s*=', "exempt routes list"),
]
findings: list[str] = []
for dir_name in ["src", "web_interface"]:
scan_dir = PROJECT_ROOT / dir_name
if not scan_dir.exists():
continue
for py_file in scan_dir.rglob("*.py"):
try:
content = py_file.read_text(encoding="utf-8")
except OSError:
continue
for pattern, label in bypass_signals:
if re.search(pattern, content):
# Only flag if the file also contains auth-related terms
if any(auth in content.lower() for auth in
["auth", "login", "authenticate", "token", "permission"]):
rel = py_file.relative_to(PROJECT_ROOT)
findings.append(f"{rel}: {label}")
if findings:
return TestResult(
"T5a", "WARNING",
f"Potential auth bypass patterns found ({len(findings)})",
"; ".join(findings[:5])
)
return TestResult("T5a", "PASS",
"No auth bypass patterns detected",
"Checked src/ and web_interface/ for bypass signals")
# ─────────────────────────────────────────────────────────────────────────────
# T6: Docker / Container Hardening
# ─────────────────────────────────────────────────────────────────────────────
def test_t6_docker_hardening() -> TestResult:
"""Container security — skipped if no Dockerfile exists."""
dockerfile = PROJECT_ROOT / "Dockerfile"
if not dockerfile.exists():
return TestResult("T6", "SKIP",
"No Dockerfile found — container security scan not applicable",
"If Docker support is added in future, enable hadolint/trivy scanning "
"in .github/workflows/security-audit.yml")
content = dockerfile.read_text(encoding="utf-8")
issues: list[str] = []
# Check for non-root USER directive
user_lines = [l for l in content.splitlines() if l.strip().startswith("USER")]
if not user_lines or user_lines[-1].strip() == "USER root":
issues.append("Container runs as root — use USER directive to drop privileges")
# Check for pinned base image tags. A tag (even a specific version, not
# just :latest) is mutable -- the same tag can point to a different
# image later. Only a @sha256 digest is truly immutable/reproducible.
from_lines = [line for line in content.splitlines() if line.strip().startswith("FROM")]
for from_line in from_lines:
parts = from_line.split()
# FROM [--platform=<platform>] <image> [AS <name>] -- skip an
# optional --platform= flag so it's never mistaken for the image
# token itself (which would falsely report it as unpinned).
image_parts = [p for p in parts[1:] if not p.startswith("--platform=")]
if image_parts:
image = image_parts[0]
if "@sha256:" not in image:
issues.append(f"Base image not pinned to a digest: {image}")
if issues:
return TestResult("T6", "WARNING",
f"Dockerfile hardening issues ({len(issues)})",
"; ".join(issues))
return TestResult("T6", "PASS", "Dockerfile hardening checks passed", "")
# ─────────────────────────────────────────────────────────────────────────────
# Runner
# ─────────────────────────────────────────────────────────────────────────────
def main() -> int:
parser = argparse.ArgumentParser(
description="LEDMatrix security proof tests",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--output", "-o", default=None,
help="Write JSON results to this file")
parser.add_argument("--verbose", "-v", action="store_true",
help="Show details for each check")
args = parser.parse_args()
print("=" * 60)
print("LEDMatrix Security Proof Tests")
print(f"Project root: {PROJECT_ROOT}")
print("=" * 60)
all_results: list[TestResult] = []
# Run all test groups
all_results.append(test_t1a_zip_slip_protection())
all_results.extend(test_t1b_dangerous_plugin_calls())
all_results.append(test_t2a_api_surface_inventory())
all_results.append(test_t3a_hardcoded_secrets())
all_results.append(test_t3b_plaintext_password_storage())
all_results.append(test_t4a_path_traversal())
all_results.append(test_t5a_auth_bypass_patterns())
all_results.append(test_t6_docker_hardening())
# Print results
print()
for r in all_results:
line = f" {r.icon} [{r.severity:<8}] {r.test_id}: {r.message}"
print(line)
if args.verbose and r.details:
print(f" {r.details}")
# Tally
critical = [r for r in all_results if r.severity == "CRITICAL"]
warnings = [r for r in all_results if r.severity == "WARNING"]
passed = [r for r in all_results if r.severity == "PASS"]
skipped = [r for r in all_results if r.severity == "SKIP"]
print()
print(f" Results: {len(passed)} PASS {len(warnings)} WARN "
f"{len(critical)} CRITICAL {len(skipped)} SKIP")
# Write JSON output
if args.output:
output_data = [r.to_dict() for r in all_results]
Path(args.output).write_text(
json.dumps(output_data, indent=2), encoding="utf-8"
)
print(f" Results written to: {args.output}")
if critical:
print(f"\n 🚨 {len(critical)} CRITICAL issue(s) found — blocking")
return 1
if warnings:
print(f"\n ⚠️ {len(warnings)} warning(s) found — non-blocking")
print("\n ✅ All checks passed (warnings are non-blocking)")
return 0
if __name__ == "__main__":
sys.exit(main())
+2 -5
View File
@@ -28,7 +28,7 @@ os.environ['EMULATOR'] = 'true'
# Import logger after path setup so src.logging_config is importable
from src.logging_config import get_logger # noqa: E402
from src.plugin_system.testing.loading import ( # noqa: E402
find_plugin_dir, load_manifest, load_config_defaults,
build_full_config, find_plugin_dir, load_manifest,
)
logger = get_logger("[Render Plugin]")
@@ -83,16 +83,13 @@ def main() -> int:
manifest = load_manifest(Path(plugin_dir))
# Parse config: start with schema defaults, then apply overrides
config_defaults = load_config_defaults(Path(plugin_dir))
try:
user_config = json.loads(args.config)
except json.JSONDecodeError as e:
logger.error("Invalid JSON config: %s", e)
return 1
config = {'enabled': True}
config.update(config_defaults)
config.update(user_config)
config = build_full_config(Path(plugin_dir), cli_config=user_config)
# Load mock data if provided
mock_data = {}
+125 -1
View File
@@ -209,6 +209,11 @@
onchange="onConfigChange()">
<span class="text-xs ml-2" style="color: var(--text-secondary);">px</span>
</div>
<select id="sizePreset" onchange="applySizePreset()"
class="w-full mt-2 px-2 py-1.5 rounded text-xs"
style="background: var(--bg-primary); color: var(--text-secondary); border: 1px solid var(--border-color);">
<option value="">Preset sizes…</option>
</select>
</div>
<!-- Config form -->
@@ -242,13 +247,18 @@
</div>
</details>
<!-- Render button -->
<!-- Render buttons -->
<div class="flex gap-2">
<button onclick="renderPlugin()" id="renderBtn"
class="flex-1 px-4 py-2.5 rounded-lg text-sm font-medium text-white"
style="background: var(--accent);">
Render
</button>
<button onclick="renderAllSizes()" id="renderAllBtn" title="Render at every harness test size"
class="px-4 py-2.5 rounded-lg text-sm font-medium"
style="background: var(--bg-tertiary); color: var(--text-primary); border: 1px solid var(--border-color);">
All Sizes
</button>
</div>
</div>
@@ -311,6 +321,15 @@
<div id="messagesPanel" class="panel p-3 hidden">
<div id="messagesList" class="text-xs font-mono space-y-1"></div>
</div>
<!-- Multi-size gallery -->
<div id="galleryPanel" class="panel p-4 hidden">
<div class="flex items-center justify-between mb-3">
<span class="text-xs font-medium" style="color: var(--text-secondary);">All Sizes</span>
<span class="text-xs" style="color: var(--text-secondary);" id="galleryStatus"></span>
</div>
<div id="galleryGrid" class="flex flex-wrap gap-4 items-start"></div>
</div>
</div>
</div>
@@ -340,8 +359,30 @@
opt.textContent = `${p.name} (${p.id})`;
select.appendChild(opt);
});
// Load harness size presets
try {
const sizesRes = await fetch('/api/sizes');
const sizesData = await sizesRes.json();
const preset = document.getElementById('sizePreset');
(sizesData.sizes || []).forEach(([w, h]) => {
const opt = document.createElement('option');
opt.value = `${w}x${h}`;
opt.textContent = `${w} x ${h}`;
preset.appendChild(opt);
});
} catch (e) { /* presets are a convenience; ignore */ }
});
function applySizePreset() {
const value = document.getElementById('sizePreset').value;
if (!value) return;
const [w, h] = value.split('x');
document.getElementById('displayWidth').value = w;
document.getElementById('displayHeight').value = h;
onConfigChange();
}
// ---------- Plugin selection ----------
async function onPluginChange() {
const pluginId = document.getElementById('pluginSelect').value;
@@ -485,6 +526,89 @@
}
}
// ---------- Multi-size gallery ----------
async function renderAllSizes() {
if (!currentPluginId) return;
const btn = document.getElementById('renderAllBtn');
const panel = document.getElementById('galleryPanel');
const grid = document.getElementById('galleryGrid');
const status = document.getElementById('galleryStatus');
btn.disabled = true;
btn.textContent = 'Rendering…';
panel.classList.remove('hidden');
grid.innerHTML = '';
status.textContent = 'Rendering at all harness sizes…';
const config = jsonEditor ? jsonEditor.getValue() : {};
config.enabled = true;
let mockData = {};
const mockInput = document.getElementById('mockDataInput').value.trim();
if (mockInput) {
try { mockData = JSON.parse(mockInput); }
catch (e) { showMessages([], [`Mock data JSON error: ${e.message}`]); }
}
try {
const res = await fetch('/api/render-matrix', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
plugin_id: currentPluginId,
config: config,
mock_data: mockData,
}),
});
const data = await res.json();
if (data.error) {
status.textContent = data.error;
return;
}
let failures = 0;
(data.results || []).forEach(r => {
const cell = document.createElement('div');
cell.style.cssText = 'display:flex;flex-direction:column;gap:4px;';
const failed = (r.errors || []).length > 0 || !r.image;
if (failed) failures++;
const label = document.createElement('span');
label.className = 'text-xs font-mono';
label.style.color = failed ? '#f87171' : 'var(--text-secondary)';
label.textContent = `${r.width}x${r.height} · ${r.render_time_ms}ms`;
cell.appendChild(label);
if (r.image) {
const img = document.createElement('img');
img.src = r.image;
// Small panels get 2x zoom so they stay legible in the grid
const zoom = r.height >= 128 ? 1 : 2;
img.style.cssText =
`image-rendering: pixelated; width:${r.width * zoom}px; ` +
`height:${r.height * zoom}px; ` +
`border:1px solid ${failed ? '#f87171' : 'var(--border-color)'};`;
cell.appendChild(img);
}
if (failed) {
const err = document.createElement('span');
err.className = 'text-xs font-mono';
err.style.color = '#f87171';
err.textContent = (r.errors || ['render failed']).join('; ');
cell.appendChild(err);
}
grid.appendChild(cell);
});
status.textContent = failures
? `${failures} size(s) failed`
: `${(data.results || []).length} sizes rendered`;
} catch (e) {
status.textContent = `Network error: ${e.message}`;
} finally {
btn.disabled = false;
btn.textContent = 'All Sizes';
}
}
// ---------- Zoom ----------
function updateZoom() {
const zoom = parseInt(document.getElementById('zoomSlider').value);
+12 -16
View File
@@ -78,21 +78,17 @@ class WiFiMonitorDaemon:
while self.running:
try:
# Get current status before checking
status = self.wifi_manager.get_wifi_status()
ethernet_connected = self.wifi_manager._is_ethernet_connected()
# Check WiFi status and manage AP mode
state_changed = self.wifi_manager.check_and_manage_ap_mode()
# Get updated status after check
updated_status = self.wifi_manager.get_wifi_status()
updated_ethernet = self.wifi_manager._is_ethernet_connected()
# One combined check that also returns the state it observed —
# the previous flow fetched status before AND after the check
# on top of the check's own internal fetch, each one several
# nmcli subprocess forks, every 30s, forever.
(state_changed, updated_status, updated_ethernet,
ap_active) = self.wifi_manager.check_and_manage_ap_mode_with_state()
current_state = {
'connected': updated_status.connected,
'ethernet_connected': updated_ethernet,
'ap_active': updated_status.ap_mode_active,
'ap_active': ap_active,
'ssid': updated_status.ssid
}
@@ -109,7 +105,7 @@ class WiFiMonitorDaemon:
else:
logger.debug("Ethernet not connected")
if updated_status.ap_mode_active:
if ap_active:
logger.info(f"AP mode ACTIVE - SSID: {ap_ssid} (IP: 192.168.4.1)")
else:
logger.debug("AP mode inactive")
@@ -123,16 +119,16 @@ class WiFiMonitorDaemon:
# Log periodic status (less verbose)
if updated_status.connected:
logger.debug(f"Status check: WiFi={updated_status.ssid} ({updated_status.signal}%), "
f"Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
f"Ethernet={updated_ethernet}, AP={ap_active}")
else:
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={updated_status.ap_mode_active}")
logger.debug(f"Status check: WiFi=disconnected, Ethernet={updated_ethernet}, AP={ap_active}")
# Escalating recovery: if nmcli reports connected but actual internet
# is unreachable for several consecutive checks, restart NetworkManager.
# This is done HERE (not inside check_and_manage_ap_mode) to keep the
# AP-enable trigger clean and avoid false-positive AP enables from
# transient packet loss on otherwise working WiFi.
if updated_status.connected and not updated_status.ap_mode_active:
if updated_status.connected and not ap_active:
if not self.wifi_manager.check_internet_connectivity():
self._consecutive_internet_failures += 1
logger.warning(
+248
View File
@@ -0,0 +1,248 @@
#!/usr/bin/env python3
"""
Headless skin validator render a skin against bundled fixture games at
multiple panel sizes without hardware, a network, or a running service.
python scripts/validate_skin.py --skin my-skin
python scripts/validate_skin.py --skin my-skin --sport baseball \
--size 128x32 --size 64x32 --output-dir /tmp/skin_renders
For each (mode x size) it checks: the manifest loads and its API version
matches, the render raises no exception, the canvas isn't blank, and the
render finishes inside a time budget (warn the live renderer runs every
display-loop pass, and a Pi is far slower than your dev machine). PNGs are
saved (native plus 4x nearest-neighbor previews) so you can eyeball the
result. Exit code is non-zero when any check fails.
"""
import argparse
import json
import logging
import sys
import time
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(PROJECT_ROOT))
from PIL import Image, ImageDraw, ImageFont # noqa: E402
FIXTURES_DIR = PROJECT_ROOT / "src" / "skin_system" / "fixtures"
MODES = ("live", "recent", "upcoming")
SPORTS = ("baseball", "basketball", "football", "hockey")
RENDER_BUDGET_S = 0.100
class FixtureHost:
"""Stands in for a SportsCore instance: fonts, logger, logo loading,
outlined text everything build_context needs, no network."""
def __init__(self, sport: str, skin_options: dict) -> None:
self.sport = sport
self.sport_key = sport
self.skin_options = skin_options
self.logger = logging.getLogger(f"validate_skin.{sport}")
self.fonts = self._load_fonts()
self._logo_cache = {}
self.display_manager = None # build_context is always given a size
def _load_fonts(self) -> dict:
"""Load the SportsCore font set (TTF, with PIL default fallback)."""
fonts = {}
try:
press = str(PROJECT_ROOT / "assets/fonts/PressStart2P-Regular.ttf")
small = str(PROJECT_ROOT / "assets/fonts/4x6-font.ttf")
fonts['score'] = ImageFont.truetype(press, 10)
fonts['time'] = ImageFont.truetype(press, 8)
fonts['team'] = ImageFont.truetype(press, 8)
fonts['status'] = ImageFont.truetype(small, 6)
fonts['detail'] = ImageFont.truetype(small, 6)
fonts['rank'] = ImageFont.truetype(press, 10)
except IOError:
default = ImageFont.load_default()
for key in ('score', 'time', 'team', 'status', 'detail', 'rank'):
fonts[key] = default
return fonts
def _load_and_resize_logo(self, team_id: str, team_abbrev: str,
logo_path, logo_url) -> "Image.Image | None":
"""Load a fixture logo from disk (no downloads), cached per team."""
if team_abbrev in self._logo_cache:
return self._logo_cache[team_abbrev]
path = Path(logo_path)
if not path.is_absolute():
path = PROJECT_ROOT / path
if not path.exists():
return None
logo = Image.open(path).convert('RGBA')
self._logo_cache[team_abbrev] = logo
return logo
def _draw_text_with_outline(self, draw: "ImageDraw.ImageDraw", text: str,
position: tuple, font,
fill: tuple = (255, 255, 255),
outline_color: tuple = (0, 0, 0)) -> None:
"""Classic outlined scorebug text, same as SportsCore's helper."""
x, y = position
for dx, dy in [(-1, -1), (-1, 0), (-1, 1), (0, -1), (0, 1),
(1, -1), (1, 0), (1, 1)]:
draw.text((x + dx, y + dy), text, font=font, fill=outline_color)
draw.text((x, y), text, font=font, fill=fill)
def load_fixture(sport: str, mode: str) -> dict:
with open(FIXTURES_DIR / f"{sport}_{mode}.json", encoding="utf-8") as f:
game = json.load(f)
# Real view models carry start_time_utc as a UTC datetime, not a string.
if isinstance(game.get("start_time_utc"), str):
from datetime import datetime
game["start_time_utc"] = datetime.fromisoformat(game["start_time_utc"])
return game
def parse_size(value: str) -> "tuple[int, int]":
try:
w_text, h_text = value.lower().split("x")
w, h = int(w_text), int(h_text)
except ValueError as exc:
raise argparse.ArgumentTypeError(f"size must look like 128x32, got {value!r}") from exc
if w <= 0 or h <= 0:
raise argparse.ArgumentTypeError(f"size dimensions must be positive, got {value!r}")
return w, h
def parse_options(value: str) -> dict:
try:
options = json.loads(value)
except json.JSONDecodeError as exc:
raise argparse.ArgumentTypeError(f"options must be valid JSON: {exc.msg}") from exc
if not isinstance(options, dict):
raise argparse.ArgumentTypeError("options must be a JSON object")
return options
def display_path(path: Path) -> str:
"""Repo-relative when inside the repo, absolute otherwise (--output-dir
may point anywhere, e.g. /tmp/skin_renders)."""
try:
return str(path.relative_to(PROJECT_ROOT))
except ValueError:
return str(path)
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--skin", required=True, help="skin id (directory name under skins/)")
parser.add_argument("--sport", choices=SPORTS,
help="fixture sport (default: first sport the skin targets, else baseball)")
parser.add_argument("--size", action="append", type=parse_size, dest="sizes",
metavar="WxH", help="panel size to render at (repeatable; default 128x32 and 64x32)")
parser.add_argument("--output-dir", type=Path,
default=PROJECT_ROOT / "skin_renders",
help="where rendered PNGs are written")
parser.add_argument("--options", type=parse_options, default={},
help="skin_options JSON to pass the skin")
args = parser.parse_args()
sizes = args.sizes or [(128, 32), (64, 32)]
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
from src.skin_system import skin_runtime
from src.skin_system.skin_base import SKIN_API_VERSION
skins = skin_runtime.discover_skins()
manifest = skins.get(args.skin)
if manifest is None:
print(f"FAIL: skin '{args.skin}' not found under {skin_runtime.get_skins_directory()}")
if skins:
print(f" installed skins: {', '.join(sorted(skins))}")
return 1
sport = args.sport
if sport is None:
declared = skin_runtime.skin_targets(manifest)[0]
sport = next((s for s in declared if s in SPORTS), "baseball")
skin = skin_runtime.load_skin(args.skin, sport=sport, sport_key=sport,
options=args.options)
if skin is None:
print(f"FAIL: skin '{args.skin}' did not load "
f"(see log above; host API is {SKIN_API_VERSION})")
return 1
host = FixtureHost(sport, args.options)
args.output_dir.mkdir(parents=True, exist_ok=True)
failures = 0
rendered = 0
for mode in MODES:
game = load_fixture(sport, mode)
render = getattr(skin, f"render_{mode}")
for width, height in sizes:
label = f"{mode}@{width}x{height}"
try:
# Warm-up render absorbs one-time font/image loads, second
# render is the one timed against the budget.
ctx = skin_runtime.build_context(host, game, size=(width, height))
handled = render(ctx, dict(game))
if handled:
ctx = skin_runtime.build_context(host, game, size=(width, height))
started = time.monotonic()
handled = render(ctx, dict(game))
elapsed = time.monotonic() - started
else:
elapsed = 0.0
except Exception as e:
print(f"FAIL {label}: render raised {type(e).__name__}: {e}")
import traceback
traceback.print_exc()
failures += 1
continue
if not handled:
print(f"skip {label}: render_{mode} returned False (built-in renderer would be used)")
continue
if ctx.canvas.size != (width, height):
print(f"FAIL {label}: canvas was replaced/resized to {ctx.canvas.size} — draw onto ctx.canvas, never reassign it")
failures += 1
continue
if ctx.canvas.convert("L").getbbox() is None:
print(f"FAIL {label}: canvas is blank — render returned True but drew nothing")
failures += 1
continue
if elapsed > RENDER_BUDGET_S:
print(f"WARN {label}: render took {elapsed * 1000:.0f}ms "
f"(budget {RENDER_BUDGET_S * 1000:.0f}ms; a Pi is much slower than this machine)")
out = args.output_dir / f"{args.skin}_{sport}_{mode}_{width}x{height}.png"
ctx.canvas.save(out)
preview = ctx.canvas.resize((width * 4, height * 4), Image.NEAREST)
preview.save(out.with_name(out.stem + "_x4.png"))
print(f"ok {label}: {elapsed * 1000:.0f}ms -> {display_path(out)}")
rendered += 1
# Vegas card, once per mode at the first size (optional API)
try:
width, height = sizes[0]
ctx = skin_runtime.build_context(host, game, size=(width, height))
card = skin.render_vegas_card(ctx, dict(game))
if card is not None:
out = args.output_dir / f"{args.skin}_{sport}_{mode}_vegas.png"
card.save(out)
print(f"ok {mode} vegas card -> {display_path(out)}")
except Exception as e:
print(f"FAIL {mode} vegas card: {type(e).__name__}: {e}")
failures += 1
if rendered == 0 and failures == 0:
print(f"FAIL: skin '{args.skin}' rendered nothing — no render_<mode> returned True")
return 1
print(f"\n{'FAILED' if failures else 'PASSED'}: {rendered} renders, {failures} failures "
f"(PNGs in {args.output_dir})")
return 1 if failures else 0
if __name__ == "__main__":
sys.exit(main())
+23
View File
@@ -0,0 +1,23 @@
# skins/
User-installable **visual skins** for the sports scoreboards. Each
subdirectory is one skin:
```text
skins/<skin-id>/
skin.json # manifest
skin.py # renderer (a ScoreboardSkin subclass)
preview.png # optional
```
- Install a skin: `git clone <skin repo> skins/<skin-id>` (or via the Plugin
Store for registry entries with `"type": "skin"`).
- Select it: set `"skin": "<skin-id>"` in the plugin's section of
`config/config.json`, or use the web UI's Visual Skin dropdown.
- Build one: start from `example-classic-baseball/` and read
[docs/CREATING_SKINS.md](../docs/CREATING_SKINS.md). Validate with
`python scripts/validate_skin.py --skin <skin-id>`.
Skins survive plugin reinstalls/updates (that's why they live here and not in
the plugin's directory). A skin is Python at the same trust level as a
plugin — review before installing.
Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

+25
View File
@@ -0,0 +1,25 @@
{
"id": "example-classic-baseball",
"name": "Example: Classic Baseball",
"version": "1.0.0",
"author": "LEDMatrix",
"description": "Reference skin: a restyled baseball scorebug demonstrating the skin API. Copy this directory to start your own skin.",
"skin_api_version": "1.0.0",
"targets": {
"sports": [
"baseball"
],
"sport_keys": [
"mlb",
"milb"
]
},
"entry_point": "skin.py",
"class_name": "ClassicBaseballSkin",
"modes": [
"live",
"recent",
"upcoming"
],
"preview": "preview.png"
}
+131
View File
@@ -0,0 +1,131 @@
"""
Example: Classic Baseball the reference skin.
Shows the whole skin API surface on purpose: adaptive regions
(scoreboard_regions), fitted text (ctx.layout.fit_text + ctx.draw_fit),
logos (ctx.load_logo + ctx.draw_image), raw PIL (ctx.draw for the bases
diamond), and per-user options (ctx.options). Everything is derived from
ctx and the game dict a skin holds no state, does no I/O, and never
touches the display.
Copy this directory to skins/<your-skin-id>/, rename the class and the
manifest fields, and run:
python scripts/validate_skin.py --skin <your-skin-id>
"""
from src.adaptive_layout import LADDER_GRID, scoreboard_regions
from src.skin_system.skin_base import ScoreboardSkin, SkinContext
DEFAULT_ACCENT = (255, 200, 0)
class ClassicBaseballSkin(ScoreboardSkin):
"""Reference baseball skin: classic scorebug with bases/outs/count."""
def __init__(self, manifest: dict, options: dict):
super().__init__(manifest, options)
# Validate user options once at load time (fail fast, fall back
# gracefully) rather than surprising every render.
accent = self.options.get("accent_color", DEFAULT_ACCENT)
if (isinstance(accent, (list, tuple)) and len(accent) == 3
and all(isinstance(c, int) and 0 <= c <= 255 for c in accent)):
self._accent_color = tuple(accent)
else:
import logging
logging.getLogger(__name__).error(
"accent_color must be three 0-255 integers, got %r; using default", accent)
self._accent_color = DEFAULT_ACCENT
# -- shared pieces ----------------------------------------------------
def _accent(self, ctx: SkinContext) -> tuple:
"""Users can recolor the skin from config via skin_options."""
return self._accent_color
def _draw_card(self, ctx: SkinContext, game: dict, status: str,
center_lines: list, detail: str) -> None:
"""The common card: logos left/right, status on top, the given
center content, detail along the bottom."""
regions = scoreboard_regions(ctx.layout.bounds, ctx=ctx.layout)
ctx.draw_image(ctx.load_logo("away"), regions.away_slot,
cache_key=f"logo:{game.get('away_abbr')}")
ctx.draw_image(ctx.load_logo("home"), regions.home_slot,
cache_key=f"logo:{game.get('home_abbr')}")
if status:
fit = ctx.layout.fit_text(status, regions.status_band, LADDER_GRID)
ctx.draw_fit(fit, regions.status_band, color=self._accent(ctx))
if center_lines:
rows = regions.score_area.split_v(*[1] * len(center_lines))
for line, row in zip(center_lines, rows):
if line:
fit = ctx.layout.fit_text(line, row, LADDER_GRID)
ctx.draw_fit(fit, row)
if detail:
fit = ctx.layout.fit_text(detail, regions.detail_band, LADDER_GRID)
ctx.draw_fit(fit, regions.detail_band, color=(160, 160, 160))
def _draw_bases_and_outs(self, ctx: SkinContext, game: dict) -> None:
"""Raw-PIL escape hatch: a bases diamond + out dots in the bottom
band, sized from the layout scale so it works on any panel."""
size = ctx.layout.px(3, minimum=2) # half-diagonal of one base
gap = ctx.layout.px(1)
cx = ctx.width // 2
cy = ctx.height - (size * 2) - 1
bases = game.get("bases_occupied") or [False, False, False]
# (dx, dy) per base: first (right), second (top), third (left)
offsets = [(size + gap, 0), (0, -(size + gap)), (-(size + gap), 0)]
for occupied, (dx, dy) in zip(bases, offsets):
x, y = cx + dx, cy + dy
diamond = [(x, y - size), (x + size, y), (x, y + size), (x - size, y)]
if occupied:
ctx.draw.polygon(diamond, fill=self._accent(ctx))
else:
ctx.draw.polygon(diamond, outline=(110, 110, 110))
outs = min(int(game.get("outs") or 0), 3)
r = max(1, size - 1)
for i in range(3):
x = cx + (i - 1) * (2 * r + 2 * gap)
y = ctx.height - r - 1
dot = [x - r, y - r, x + r, y + r]
if i < outs:
ctx.draw.ellipse(dot, fill=(255, 255, 255))
else:
ctx.draw.ellipse(dot, outline=(110, 110, 110))
# -- the three modes --------------------------------------------------
def render_live(self, ctx: SkinContext, game: dict) -> bool:
half = "" if game.get("inning_half") == "top" else ""
inning = game.get("inning") or ""
status = f"{half}{inning}" if inning else game.get("status_text", "")
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
count = f"{game.get('balls', 0)}-{game.get('strikes', 0)}"
self._draw_card(ctx, game, status, [score], "")
self._draw_bases_and_outs(ctx, game)
# Ball-strike count in the top-left corner, over the away logo.
fit = ctx.layout.fit_text(count, (ctx.width // 4, ctx.layout.px(8, minimum=6)), LADDER_GRID)
ctx.draw_fit(fit, ctx.layout.bounds.top_band(fit.height + 1).left_col(fit.width + 2),
color=(200, 200, 200))
return True
def render_recent(self, ctx: SkinContext, game: dict) -> bool:
score = f"{game.get('away_score', '0')}-{game.get('home_score', '0')}"
self._draw_card(ctx, game, game.get("status_text", "Final"),
[score], game.get("series_summary", ""))
return True
def render_upcoming(self, ctx: SkinContext, game: dict) -> bool:
matchup = f"{game.get('away_abbr', '')}@{game.get('home_abbr', '')}"
self._draw_card(ctx, game, game.get("game_date", ""),
[matchup, game.get("game_time", "")],
f"{game.get('away_record', '')} {game.get('home_record', '')}".strip())
return True
+1 -1
View File
@@ -4,5 +4,5 @@ LEDMatrix Display System
Core source package for the LED Matrix Display project.
"""
__version__ = "1.0.0"
__version__ = "3.1.0"
+174
View File
@@ -0,0 +1,174 @@
"""
Adaptive image fitting for plugins the image counterpart to
src/adaptive_layout.py's text fitting.
Promotes the proven in-field image patterns into one shared helper so
plugins stop hand-copying resize/cache code:
- "crop transparent padding, then fill the row height" (football/hockey
logo pattern) -> ``crop_to_ink=True, mode="fill_height"``
- "crop-to-fill with a top anchor for faces" (masters-tournament headshot
pattern) -> ``mode="cover", anchor="top"``
- "letterbox to fit, centered on a background" (static-image pattern)
-> ``mode="contain"``
- NEAREST for pixel art/flags vs LANCZOS for photos (masters flag pattern)
-> ``resample=RESAMPLE_NEAREST``
Unlike PIL's ``thumbnail()`` (downscale-only — the reason plugin imagery
stays tiny on big panels), ``fit_image`` upscales by default so content
genuinely adapts to larger displays; pass ``upscale=False`` for the old
behavior.
Use via ``LayoutContext.fit_image(...)`` (cached per panel size) or
``BasePlugin.draw_image(...)``; the module-level functions are the
uncached primitives.
"""
from dataclasses import dataclass
from typing import Any, Optional, Tuple
from PIL import Image
# The one Pillow >= 9.1 compat shim (replaces the per-plugin copies).
try:
RESAMPLE_LANCZOS = Image.Resampling.LANCZOS
RESAMPLE_NEAREST = Image.Resampling.NEAREST
except AttributeError: # Pillow < 9.1
RESAMPLE_LANCZOS = Image.LANCZOS
RESAMPLE_NEAREST = Image.NEAREST
FIT_MODES = ("contain", "cover", "fill_height", "stretch")
@dataclass(frozen=True)
class ImageFitResult:
"""A processed RGBA copy of a source image, sized for a target box."""
image: Image.Image
width: int
height: int
scale: float # scale applied vs the (possibly ink-cropped) source
mode: str
source_size: Tuple[int, int]
@property
def is_empty(self) -> bool:
return self.width <= 0 or self.height <= 0
_EMPTY_IMAGE = Image.new("RGBA", (1, 1), (0, 0, 0, 0))
def _empty_result(mode: str, source_size: Tuple[int, int]) -> ImageFitResult:
return ImageFitResult(_EMPTY_IMAGE, 0, 0, 0.0, mode, source_size)
def _box_dims(box: Any) -> Tuple[int, int]:
"""Accept a Region (duck-typed .w/.h) or a (w, h) tuple."""
if hasattr(box, "w") and hasattr(box, "h"):
return (int(box.w), int(box.h))
w, h = box
return (int(w), int(h))
def fit_image(img: Image.Image, box: Any, *, mode: str = "contain",
crop_to_ink: bool = False, anchor: str = "center",
resample: Any = None, upscale: bool = True) -> ImageFitResult:
"""Fit an image into a box, preserving crispness policy per content type.
Args:
img: Source PIL image (any mode; output is always RGBA).
box: Region or (w, h) target box.
mode: "contain" (letterbox), "cover" (crop-to-fill),
"fill_height" (height == box height, contain-capped by width),
"stretch" (exact resize).
crop_to_ink: Trim fully-transparent padding (getbbox) before fitting
logos shipped with generous padding otherwise render small.
anchor: For "cover" crops: "center" or "top" (keeps faces/tops).
resample: PIL resampling filter; defaults to RESAMPLE_LANCZOS.
Use RESAMPLE_NEAREST for pixel art, flags, and sprite icons.
upscale: Allow scaling above source size (default True the adaptive
point). False mimics the legacy thumbnail() behavior.
"""
if mode not in FIT_MODES:
raise ValueError(f"Unknown fit mode '{mode}' (expected one of {FIT_MODES})")
box_w, box_h = _box_dims(box)
if box_w <= 0 or box_h <= 0 or img.width <= 0 or img.height <= 0:
return _empty_result(mode, img.size)
resample = RESAMPLE_LANCZOS if resample is None else resample
work = img if img.mode == "RGBA" else img.convert("RGBA")
if crop_to_ink:
bbox = work.getbbox()
if bbox is None: # fully transparent
return _empty_result(mode, img.size)
work = work.crop(bbox)
src_w, src_h = work.size
if mode == "stretch":
out = work.resize((box_w, box_h), resample)
return ImageFitResult(out, box_w, box_h, box_w / src_w, mode, (src_w, src_h))
if mode == "cover":
scale = max(box_w / src_w, box_h / src_h)
if not upscale:
scale = min(scale, 1.0)
scaled_w = max(1, round(src_w * scale))
scaled_h = max(1, round(src_h * scale))
out = work.resize((scaled_w, scaled_h), resample)
# Crop the overhang down to the box (only when the scaled image is
# larger; with upscale=False it may be smaller and is left as-is).
crop_w, crop_h = min(box_w, scaled_w), min(box_h, scaled_h)
left = (scaled_w - crop_w) // 2
top = 0 if anchor == "top" else (scaled_h - crop_h) // 2
out = out.crop((left, top, left + crop_w, top + crop_h))
return ImageFitResult(out, out.width, out.height, scale, mode, (src_w, src_h))
# contain / fill_height share the "preserve aspect, no crop" path
if mode == "fill_height":
scale = box_h / src_h
# contain-cap: never exceed the box width (football's logo_slot rule)
scale = min(scale, box_w / src_w)
else: # contain
scale = min(box_w / src_w, box_h / src_h)
if not upscale:
scale = min(scale, 1.0)
out_w = max(1, round(src_w * scale))
out_h = max(1, round(src_h * scale))
if (out_w, out_h) == (src_w, src_h):
# No resize needed — but `work` may still BE the caller's original
# image (RGBA source, no ink crop). The result must always be an
# independent copy: LayoutContext caches ImageFitResults, and an
# aliased image would let later mutations of the source corrupt
# cached fits (or vice versa).
out = work.copy() if work is img else work
else:
out = work.resize((out_w, out_h), resample)
return ImageFitResult(out, out_w, out_h, scale, mode, (src_w, src_h))
def draw_fitted_image(display_manager: Any, ifit: ImageFitResult, box: Any, *,
align: str = "center", valign: str = "center",
offset: Tuple[int, int] = (0, 0)) -> Optional[Tuple[int, int]]:
"""Paste a fitted image aligned within a Region onto the display canvas.
Pastes with the image's own alpha mask. Returns the (x, y) actually used
so callers can position adjacent decorations, or None when nothing was
drawn (empty fit / no canvas).
"""
if ifit is None or ifit.is_empty:
return None
image = getattr(display_manager, "image", None)
if image is None:
return None
if hasattr(box, "align_xy"):
x, y = box.align_xy(ifit.width, ifit.height, align, valign)
else:
box_w, box_h = _box_dims(box)
x = (box_w - ifit.width) // 2
y = (box_h - ifit.height) // 2
x += int(offset[0])
y += int(offset[1])
image.paste(ifit.image, (x, y), ifit.image)
return (x, y)
+746
View File
@@ -0,0 +1,746 @@
"""
Adaptive layout and font scaling helpers for plugins.
Generalizes the three size-adaptation patterns proven in the plugin
ecosystem into small composable core helpers, so plugins render legibly on
any panel size (64x32, 128x32, 96x48, 128x64, 256x64, ...) without
hand-tuned per-display layouts:
- Region: integer rect algebra (bands, columns, weighted splits, centering).
Regions partition space, so text bands can't overlap by construction —
replacing the magic ``y = 1`` / ``y = height - 7`` offsets tuned for 128x32.
- Font ladders: ordered (family, size) steps known to render crisply.
Pixel fonts (BDF, PressStart2P) only look right at native/integer sizes,
so fonts are never scaled continuously fitting walks a ladder from the
largest rung down until the measured text fits the target box. This is
baseball-scoreboard's fallback-ladder pattern promoted to core.
- LayoutContext: per-(width, height) facts breakpoint tiers
(masters-tournament's pattern), a geometry scale factor vs. a declared
design size (f1-scoreboard's pattern), and cached fit-text queries.
Everything is opt-in: plugins get a context via ``self.layout`` on
BasePlugin (or construct one directly) and existing plugins are unaffected.
Fonts are resolved through FontManager's catalog (family names are
lowercased file stems from assets/fonts, e.g. "9x15", "tom-thumb", plus
aliases like "press_start"). FitResult.font is a plain PIL font or
freetype.Face, so it drops straight into DisplayManager.draw_text().
"""
import logging
from collections import OrderedDict
from dataclasses import dataclass
from typing import Any, Dict, List, Optional, Sequence, Tuple, Union
import freetype
logger = logging.getLogger(__name__)
# Height-based breakpoint tiers, smallest to largest. A 32px-tall panel is
# the ecosystem baseline ("sm"); 96x48 lands in "md"; 128x64 in "lg".
_HEIGHT_TIERS: Tuple[Tuple[str, int], ...] = (
("xs", 16), ("sm", 32), ("md", 48), ("lg", 64), ("xl", 10 ** 9),
)
TIER_ORDER: Tuple[str, ...] = tuple(name for name, _ in _HEIGHT_TIERS)
_WIDTH_TIERS: Tuple[Tuple[str, int], ...] = (
("narrow", 64), ("normal", 128), ("wide", 256), ("ultrawide", 10 ** 9),
)
WIDTH_TIER_ORDER: Tuple[str, ...] = tuple(name for name, _ in _WIDTH_TIERS)
# The panel size most existing plugins were authored against.
DEFAULT_DESIGN_SIZE: Tuple[int, int] = (128, 32)
@dataclass(frozen=True)
class Region:
"""An integer rectangle. Carving methods return sub-Regions clamped to
non-negative dimensions, so degenerate panels never produce negative
boxes a band request larger than the region simply consumes it all."""
x: int
y: int
w: int
h: int
def __post_init__(self):
object.__setattr__(self, "w", max(0, int(self.w)))
object.__setattr__(self, "h", max(0, int(self.h)))
object.__setattr__(self, "x", int(self.x))
object.__setattr__(self, "y", int(self.y))
@property
def right(self) -> int:
return self.x + self.w
@property
def bottom(self) -> int:
return self.y + self.h
@property
def center(self) -> Tuple[int, int]:
return (self.x + self.w // 2, self.y + self.h // 2)
# ---- carving -----------------------------------------------------
def inset(self, dx: int, dy: Optional[int] = None) -> "Region":
"""Shrink by dx horizontally and dy (default dx) vertically, each side."""
if dy is None:
dy = dx
return Region(self.x + dx, self.y + dy, self.w - 2 * dx, self.h - 2 * dy)
def offset(self, dx: int, dy: int) -> "Region":
"""Translate without resizing — the hook for user x/y-offset
customization: compute regions first, then apply the user's
configured offsets as a final translation."""
return Region(self.x + dx, self.y + dy, self.w, self.h)
def top_band(self, h: int) -> "Region":
return Region(self.x, self.y, self.w, min(h, self.h))
def bottom_band(self, h: int) -> "Region":
h = min(h, self.h)
return Region(self.x, self.bottom - h, self.w, h)
def middle(self, top_h: int = 0, bottom_h: int = 0) -> "Region":
"""What remains between a top band and a bottom band."""
return Region(self.x, self.y + top_h, self.w, self.h - top_h - bottom_h)
def left_col(self, w: int) -> "Region":
return Region(self.x, self.y, min(w, self.w), self.h)
def right_col(self, w: int) -> "Region":
w = min(w, self.w)
return Region(self.right - w, self.y, w, self.h)
def split_h(self, *weights: float, gap: int = 0) -> List["Region"]:
"""Side-by-side columns sized by weight; gaps between them."""
sizes = _weighted_sizes(self.w, weights, gap)
cols, cursor = [], self.x
for size in sizes:
cols.append(Region(cursor, self.y, size, self.h))
cursor += size + gap
return cols
def split_v(self, *weights: float, gap: int = 0) -> List["Region"]:
"""Stacked rows sized by weight; gaps between them."""
sizes = _weighted_sizes(self.h, weights, gap)
rows, cursor = [], self.y
for size in sizes:
rows.append(Region(self.x, cursor, self.w, size))
cursor += size + gap
return rows
# ---- placement ---------------------------------------------------
def align_xy(self, w: int, h: int, align: str = "center",
valign: str = "center") -> Tuple[int, int]:
"""Top-left position for a w x h box aligned within this region.
align: left|center|right; valign: top|center|bottom."""
if align == "left":
x = self.x
elif align == "right":
x = self.right - w
else:
x = self.x + (self.w - w) // 2
if valign == "top":
y = self.y
elif valign == "bottom":
y = self.bottom - h
else:
y = self.y + (self.h - h) // 2
return (x, y)
def center_xy(self, w: int, h: int) -> Tuple[int, int]:
return self.align_xy(w, h)
def contains(self, w: int, h: int) -> bool:
return w <= self.w and h <= self.h
def _weighted_sizes(total: int, weights: Sequence[float], gap: int) -> List[int]:
"""Integer sizes proportional to weights, remainder spread left-to-right."""
if not weights:
return []
usable = max(0, total - gap * (len(weights) - 1))
weight_sum = sum(weights) or 1
sizes = [int(usable * w / weight_sum) for w in weights]
remainder = usable - sum(sizes)
for i in range(remainder):
sizes[i % len(sizes)] += 1
return sizes
# ---------------------------------------------------------------------------
# Font ladders
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class FontStep:
"""One rung: a FontManager catalog family at a size it renders crisply."""
family: str
size_px: int
FontLadder = Tuple[FontStep, ...]
# X11 BDF bitmap fonts at their native pixel sizes, largest to smallest —
# baseball-scoreboard's fallback ladder extended upward. Same-height rungs
# are ordered widest first so width-constrained text steps to a narrower
# face before dropping a size.
LADDER_GRID: FontLadder = (
FontStep("10x20", 20),
FontStep("9x18", 18),
FontStep("9x15", 15),
FontStep("8x13", 13),
FontStep("7x13", 13),
FontStep("6x13", 13),
FontStep("6x12", 12),
FontStep("6x10", 10),
FontStep("6x9", 9),
FontStep("5x8", 8),
FontStep("5x7", 7),
FontStep("4x6", 6),
FontStep("tom-thumb", 6),
)
# PressStart2P at integer multiples of its 8px pixel grid only — fractional
# sizes blur a pixel font. For headline text (clocks, scores).
LADDER_ARCADE: FontLadder = (
FontStep("press_start", 32),
FontStep("press_start", 24),
FontStep("press_start", 16),
FontStep("press_start", 8),
)
LADDER_DEFAULT: FontLadder = LADDER_GRID
ELLIPSIS = ""
@dataclass(frozen=True)
class FitResult:
"""A fitted font plus the ink metrics of the (possibly ellipsized) text.
``y_offset`` is the gap between the y passed to draw_text() and where
ink actually starts; subtract it from the desired ink-top position when
drawing (draw_fitted_text does this for you).
"""
font: Any
family: str
size_px: int
text: str
width: int
height: int
baseline: int
y_offset: int
fits: bool
line_height: int = 0
def measure_ink(text: str, font: Any) -> Tuple[int, int, int, int]:
"""Measure the ink box of text: (width, height, baseline, y_offset).
y_offset is the distance from the y coordinate DisplayManager.draw_text()
is given to the top of the actual ink PIL draws TTF from the em-box
top and _draw_bdf_text derives the baseline from y + ascender, so both
leave a font-dependent gap that matters when centering in short bands.
"""
if isinstance(font, freetype.Face):
width = 0
ascender = font.size.ascender >> 6
ink_top, ink_bottom = None, None
for char in text:
font.load_char(char)
width += font.glyph.advance.x >> 6
rows = font.glyph.bitmap.rows
if rows:
top = ascender - font.glyph.bitmap_top
ink_top = top if ink_top is None else min(ink_top, top)
ink_bottom = top + rows if ink_bottom is None else max(ink_bottom, top + rows)
if ink_top is None:
ink_top, ink_bottom = 0, 0
return (width, ink_bottom - ink_top, ascender, ink_top)
bbox = font.getbbox(text)
return (bbox[2] - bbox[0], bbox[3] - bbox[1], -bbox[1], bbox[1])
def font_line_height(font: Any) -> int:
"""Recommended line spacing for a font (matches DisplayManager.get_font_height)."""
if isinstance(font, freetype.Face):
return font.size.height >> 6
ascent, descent = font.getmetrics()
return ascent + descent
def measure_font_crispness(font: Any, sample_text: str = "Ay0",
canvas_size: Tuple[int, int] = (250, 60)) -> float:
"""Fraction of the rendered sample's ink-bbox pixels that are neither
pure black nor pure white i.e. antialiased.
BDF (freetype.Face) glyphs are true bitmaps and always render at 0.0.
"Pixel-style" TTFs (PressStart2P, and similar fonts bundled for
plugins that draw through ImageDraw.text() and so can't take a BDF
face) are NOT automatically crisp at arbitrary sizes PIL antialiases
TTF outlines by default, and a pixel-grid font only lands on whole
pixels at specific sizes (for PressStart2P: exact multiples of 8).
Requesting an unverified size silently produces soft/blurry glyphs on
an LED panel, which reads as fuzzy compared to a true BDF rung.
Use this to vet any custom FontLadder rung that mixes TTF fonts before
shipping it see test_adaptive_layout.py::test_ladder_is_crisp for the
pattern. A rung should score 0.0 (or very close, to allow for the odd
diagonal stroke) before it belongs in a "crisp" ladder.
"""
if isinstance(font, freetype.Face):
return 0.0
from PIL import Image, ImageDraw
img = Image.new("L", canvas_size, 0)
ImageDraw.Draw(img).text((2, 2), sample_text, font=font, fill=255)
bbox = img.getbbox()
if bbox is None:
return 0.0
pixels = img.crop(bbox).tobytes()
pure = sum(1 for p in pixels if p == 0 or p == 255)
return (len(pixels) - pure) / len(pixels)
class LayoutContext:
"""Per-render-size layout facts and fit-text queries for one panel size.
Construct once per (width, height); BasePlugin.layout does this and
rebuilds automatically when the logical display size changes.
"""
def __init__(self, width: int, height: int, font_manager: Any,
design_size: Tuple[int, int] = DEFAULT_DESIGN_SIZE):
self.width = int(width)
self.height = int(height)
self.font_manager = font_manager
self.design_size = design_size
self.bounds = Region(0, 0, self.width, self.height)
self.aspect = self.width / max(1, self.height)
self.tier = _pick_tier(_HEIGHT_TIERS, self.height)
self.width_tier = _pick_tier(_WIDTH_TIERS, self.width)
self.is_wide_short = self.aspect >= 2.5 and self.height <= 32
design_w, design_h = design_size
# Geometry scale only (gaps, icon/logo sizes) — never applied to
# fonts, which step between crisp ladder rungs instead.
self.scale = min(self.width / max(1, design_w),
self.height / max(1, design_h))
# LRU-bounded: entries are small, but keys embed the fitted TEXT —
# a plugin fitting changing text (a live game clock, a ticker) on a
# 24/7 service would otherwise grow this without bound.
self._fit_cache: "OrderedDict[Any, FitResult]" = OrderedDict()
# LRU-bounded (images are big). Entries hold a strong reference to
# the source image when keyed by id() so the id can't be recycled
# out from under the cache.
self._image_cache: "OrderedDict[Any, Tuple[Any, Any]]" = OrderedDict()
_IMAGE_CACHE_MAX = 64
_FIT_CACHE_MAX = 512
def _fit_cache_get(self, key: Any) -> Optional["FitResult"]:
cached = self._fit_cache.get(key)
if cached is not None:
self._fit_cache.move_to_end(key)
return cached
def _fit_cache_put(self, key: Any, result: "FitResult") -> None:
self._fit_cache[key] = result
while len(self._fit_cache) > self._FIT_CACHE_MAX:
self._fit_cache.popitem(last=False)
# ---- the three adaptation patterns --------------------------------
def px(self, base: int, minimum: int = 1, maximum: Optional[int] = None) -> int:
"""Scale a design-size pixel measurement (f1's pattern): gaps,
icon sizes, logo slots. Clamped to [minimum, maximum]."""
value = max(minimum, round(base * self.scale))
if maximum is not None:
value = min(value, maximum)
return value
def by_tier(self, mapping: Dict[str, Any], default: Any = None) -> Any:
"""Pick the value for the nearest defined tier at-or-below the
panel's height tier (masters' pattern). Falls forward to the
smallest defined tier above, then to default.
by_tier({"sm": 10, "lg": 18}) -> 10 on 128x32, 18 on 128x64.
Keys may also use width tiers ("narrow", "wide", ...)."""
order = TIER_ORDER if any(k in TIER_ORDER for k in mapping) else WIDTH_TIER_ORDER
current = self.tier if order is TIER_ORDER else self.width_tier
idx = order.index(current)
for name in reversed(order[: idx + 1]):
if name in mapping:
return mapping[name]
for name in order[idx + 1:]:
if name in mapping:
return mapping[name]
return default
def fit_text(self, text: str, box: Union[Region, Tuple[int, int]],
ladder: FontLadder = LADDER_DEFAULT,
ellipsis: bool = True) -> FitResult:
"""Largest ladder rung whose rendered text fits the box (baseball's
pattern). If even the smallest rung is too wide, the text is
ellipsized to fit (unless ellipsis=False); fits=False only when no
acceptable rendering exists."""
box_w, box_h = _box_dims(box)
key = ("text", text, box_w, box_h, ladder, ellipsis)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
result = self._walk_ladder(text, ladder, box_w, box_h, ellipsis)
self._fit_cache_put(key, result)
return result
def fit_text_proportional(self, text: str, box: Union[Region, Tuple[int, int]],
base_size_px: int, ladder: FontLadder = LADDER_DEFAULT,
ellipsis: bool = True,
scale: Optional[float] = None) -> FitResult:
"""Ladder rung closest to (but not exceeding) ``base_size_px * scale``
that still fits the box proportional sizing instead of ``fit_text``'s
"always maximize" behavior.
Use this when several independently-fitted elements need to stay
visually harmonious as the panel grows (e.g. a scoreboard's score,
status, and detail text) ``fit_text`` maximizes each one within
its own region, which can make one element balloon out of
proportion to its neighbors (a huge score overlapping logos it fit
fine at the design size) even though every individual pick is
independently "correct". ``base_size_px`` is the size that element
renders at on the design size (``design_size``, typically 128x32)
commonly a plugin's existing classic/fixed font size for that
element.
``scale`` defaults to ``self.scale`` (the same conservative
min(width_ratio, height_ratio) factor ``px()`` uses safe for
content whose aspect ratio matters). Pass an explicit axis-specific
value when the surrounding composition already scales that way
e.g. a scoreboard whose logos scale with height alone
(``logo_slot = min(height, width // 2)``) should size its score
text by ``height / design_height`` too, or its text will look
under-scaled next to bigger logos on a panel that only grew taller.
Falls back to the smallest rung when even that exceeds the target
(a tiny scale factor), and to fit_text's ordinary smaller-rung
fallback when the closest-to-target rung doesn't actually fit the
box.
"""
box_w, box_h = _box_dims(box)
effective_scale = self.scale if scale is None else scale
key = ("text_prop", text, box_w, box_h, ladder, base_size_px, ellipsis, effective_scale)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
target = base_size_px * effective_scale
eligible = [step for step in ladder if step.size_px <= target]
candidates = eligible if eligible else (min(ladder, key=lambda s: s.size_px),)
result = self._walk_ladder(text, candidates, box_w, box_h, ellipsis)
self._fit_cache_put(key, result)
return result
def _walk_ladder(self, text: str, ladder: Sequence[FontStep],
box_w: int, box_h: int, ellipsis: bool) -> FitResult:
"""Shared by fit_text/fit_text_proportional: first ladder entry (in
the order given) whose rendered text fits, ellipsizing the last one
tried if none do."""
result = None
for step in ladder:
font = self.font_manager.get_font(step.family, step.size_px)
width, height, baseline, y_offset = measure_ink(text, font)
result = FitResult(font, step.family, step.size_px, text,
width, height, baseline, y_offset,
fits=(width <= box_w and height <= box_h),
line_height=font_line_height(font))
if result.fits:
break
if result is not None and not result.fits and ellipsis:
short = self.ellipsize(text, result.font, box_w)
width, height, baseline, y_offset = measure_ink(short, result.font)
result = FitResult(result.font, result.family, result.size_px,
short, width, height, baseline, y_offset,
fits=(width <= box_w and height <= box_h),
line_height=result.line_height)
return result
def fit_lines(self, lines: Sequence[str], box: Union[Region, Tuple[int, int]],
ladder: FontLadder = LADDER_DEFAULT,
spacing: int = 1) -> FitResult:
"""Largest rung where every line fits the box width and the stacked
lines (line_height + spacing apart) fit the box height. Measures the
actual strings, so a long line pushes the ladder down a rung a short
one wouldn't (baseball's multiline pattern). Text is the widest line."""
box_w, box_h = _box_dims(box)
key = ("lines", tuple(lines), box_w, box_h, ladder, spacing)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
rows = max(1, len(lines))
result = None
for step in ladder:
font = self.font_manager.get_font(step.family, step.size_px)
line_h = font_line_height(font)
widest, metrics = "", (0, 0, 0, 0)
for line in lines:
m = measure_ink(line, font)
if m[0] >= metrics[0]:
widest, metrics = line, m
total_h = rows * line_h + (rows - 1) * spacing
result = FitResult(font, step.family, step.size_px, widest,
metrics[0], metrics[1], metrics[2], metrics[3],
fits=(metrics[0] <= box_w and total_h <= box_h),
line_height=line_h)
if result.fits:
break
self._fit_cache_put(key, result)
return result
def font_for_rows(self, rows: int, box_h: int,
ladder: FontLadder = LADDER_GRID) -> FitResult:
"""Largest rung whose line height lets `rows` rows fit in box_h
(baseball's traditional-scoreboard pattern). Measures a digit/cap
sample rather than specific strings."""
key = ("rows", rows, box_h, ladder)
cached = self._fit_cache_get(key)
if cached is not None:
return cached
sample = "0Ay"
result = None
for step in ladder:
font = self.font_manager.get_font(step.family, step.size_px)
line_h = font_line_height(font)
width, height, baseline, y_offset = measure_ink(sample, font)
result = FitResult(font, step.family, step.size_px, sample,
width, height, baseline, y_offset,
fits=(max(1, rows) * line_h <= box_h),
line_height=line_h)
if result.fits:
break
self._fit_cache_put(key, result)
return result
# ---- images ---------------------------------------------------------
def fit_image(self, img: Any, box: Union[Region, Tuple[int, int]], *,
mode: str = "contain", crop_to_ink: bool = False,
anchor: str = "center", resample: Any = None,
upscale: bool = True, cache_key: Any = None) -> Any:
"""Fit an image into a box (see src/adaptive_images.py for modes),
cached per (image, box size, options) for this panel size.
Prefer a stable ``cache_key`` (e.g. "logo:KC") for images that get
reloaded the default id()-based key is safe (the entry pins the
source image) but misses across reloads of the same content.
"""
from src.adaptive_images import fit_image as _fit_image
box_w, box_h = _box_dims(box)
resample_name = getattr(resample, "name", repr(resample)) if resample is not None else "default"
identity = cache_key if cache_key is not None else ("id", id(img))
key = ("image", identity, img.size, box_w, box_h, mode,
crop_to_ink, anchor, resample_name, upscale)
cached = self._image_cache.get(key)
if cached is not None:
self._image_cache.move_to_end(key)
return cached[0]
result = _fit_image(img, (box_w, box_h), mode=mode,
crop_to_ink=crop_to_ink, anchor=anchor,
resample=resample, upscale=upscale)
# Pin the source only for id()-keyed entries (see docstring).
self._image_cache[key] = (result, img if cache_key is None else None)
while len(self._image_cache) > self._IMAGE_CACHE_MAX:
self._image_cache.popitem(last=False)
return result
# ---- text utilities ------------------------------------------------
def ellipsize(self, text: str, font: Any, max_w: int) -> str:
"""Trim text to fit max_w, appending an ellipsis. Returns '' when
not even the ellipsis fits."""
if measure_ink(text, font)[0] <= max_w:
return text
for end in range(len(text) - 1, 0, -1):
candidate = text[:end].rstrip() + ELLIPSIS
if measure_ink(candidate, font)[0] <= max_w:
return candidate
return ELLIPSIS if measure_ink(ELLIPSIS, font)[0] <= max_w else ""
def measure(self, text: str, font: Any) -> Tuple[int, int, int]:
"""Ink (width, height, baseline) of text — see measure_ink."""
width, height, baseline, _ = measure_ink(text, font)
return (width, height, baseline)
def clear_cache(self) -> None:
"""Drop cached fit results (call after fonts are reloaded)."""
self._fit_cache.clear()
self._image_cache.clear()
def _pick_tier(tiers: Tuple[Tuple[str, int], ...], value: int) -> str:
for name, limit in tiers:
if value <= limit:
return name
return tiers[-1][0]
def _box_dims(box: Union[Region, Tuple[int, int]]) -> Tuple[int, int]:
if isinstance(box, Region):
return (box.w, box.h)
w, h = box
return (int(w), int(h))
def draw_fitted_text(display_manager: Any, fit: FitResult,
box: Union[Region, Tuple[int, int]],
color: Tuple[int, int, int] = (255, 255, 255),
align: str = "center", valign: str = "center") -> None:
"""Draw a FitResult's text aligned within a Region via
DisplayManager.draw_text(), compensating for the font's ink offset so
the ink (not the em box) is what gets aligned."""
region = box if isinstance(box, Region) else Region(0, 0, box[0], box[1])
x, y = region.align_xy(fit.width, fit.height, align, valign)
display_manager.draw_text(fit.text, x=x, y=y - fit.y_offset,
color=color, font=fit.font)
# ---------------------------------------------------------------------------
# Composite layouts — the region arrangements repeated across plugins,
# expressed as Region math so migrated plugins stop hand-copying coordinate
# formulas. Deliberately tiny: these return Regions, they don't draw.
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class ScoreboardRegions:
"""The two-logos-plus-center-score card shared by the sports plugins."""
bounds: Region
logo_slot: int # width of each logo slot: min(H, W // 2), center-reserved
away_slot: Region # left logo slot
home_slot: Region # right logo slot
center_col: Region # column between the slots (>= min_center_fraction of width)
status_band: Region # top band (replaces the magic y = 1)
score_area: Region # center_col's true width, between the bands (replaces y = H//2 - 3)
detail_band: Region # bottom band (replaces the magic y = H - 7)
bottom_left: Region # bottom corner: away records / timeouts
bottom_right: Region # bottom corner: home records / timeouts
def scoreboard_regions(bounds: Region, *, ctx: Optional["LayoutContext"] = None,
status_h: Optional[int] = None,
detail_h: Optional[int] = None,
min_center_fraction: float = 0.15,
min_center_design_px: int = 40,
score_bleed_fraction: float = 0.5) -> ScoreboardRegions:
"""Carve a game-card Region into the standard scoreboard arrangement.
Encodes the invariant duplicated across the sports plugins:
``logo_slot = min(height, width // 2)`` (capped at half the card so the
home slot never collapses), away logo centered in the left slot, home in
the right.
That formula alone has a blind spot: at exactly 2:1 aspect ratio
(width == 2 * height a very common shape, e.g. two, four, or more
square modules stacked into a taller panel) ``width // 2`` and
``height`` are equal, so the two logo slots claim the *entire* width
and leave zero pixels for a center column, no matter how large the
panel gets. It isn't a "small panel" problem: 96x48, 128x64, and
256x128 (all exactly 2:1) hit it identically, while wide panels like
the 128x32 design baseline or a 192x48/256x32 panel never do, because
height is already the tighter constraint there.
Two knobs fix it, both defaulted to values verified against the full
harness size spread (see test_adaptive_layout.py::TestScoreboardRegions):
- ``min_center_fraction`` / ``min_center_design_px`` reserve at least
``max(width * min_center_fraction, min_center_design_px * ctx.scale)``
for the center column, capping ``logo_slot`` further when needed. The
design-px term (scaled by the context's geometry factor, so it grows
on bigger panels like everything else in ``px()``) matters most on
small panels where a flat fraction alone reserves too little absolute
space for even a short score string. On wide panels the height
constraint already leaves more room than either reserves, so both are
a no-op there 128x32/192x48-style layouts are unaffected.
- ``score_bleed_fraction`` extends the score's own *fit box* (not the
logo slots themselves) an extra ``logo_slot * score_bleed_fraction``
into each side controlled, intentional overlap with the logo art,
the same way real broadcast scoreboards let a big score number's
edges cross into the team marks flanking it. Without this, on a
square-ish panel the center reserve alone can be too narrow for even
a modest score to render without truncating (`"17-21"` -> `"17-2…"`),
which is worse than a little overlap.
status_band and detail_band span the FULL card width and overlay the
logo slots matching the classic layouts, where short outlined status/
date text is drawn over the logos without issue; only score_area (the
one element whose size actively grows with the panel) uses the
narrower, bleed-adjusted box. Band heights default to the classic
128x32 values, scaled by the context's geometry factor when one is
provided. Works on a full panel or on a scroll-mode card Region.
"""
if status_h is None:
status_h = ctx.px(9, minimum=7) if ctx else 9
if detail_h is None:
detail_h = ctx.px(8, minimum=7) if ctx else 8
logo_slot = min(bounds.h, bounds.w // 2)
design_reserve = int(min_center_design_px * (ctx.scale if ctx else 1.0))
min_center_w = max(1, int(bounds.w * min_center_fraction), design_reserve)
max_logo_slot_by_center = max(1, (bounds.w - min_center_w) // 2)
logo_slot = min(logo_slot, max_logo_slot_by_center)
away_slot = bounds.left_col(logo_slot)
home_slot = bounds.right_col(logo_slot)
center_col = Region(bounds.x + logo_slot, bounds.y,
bounds.w - 2 * logo_slot, bounds.h)
status_band = bounds.top_band(status_h)
detail_band = bounds.bottom_band(detail_h)
middle = bounds.middle(status_band.h, detail_band.h)
# score_area is the true center gap's width plus a controlled bleed
# into each logo slot (see score_bleed_fraction above) -- narrower than
# the full card width status/detail get, since it's the one element
# whose size actively grows with the panel and needs its *fit box* to
# reflect real available space, but generous enough that a short score
# string never has to truncate on a square-ish panel.
bleed = int(logo_slot * score_bleed_fraction)
score_area = Region(center_col.x - bleed, middle.y,
center_col.w + 2 * bleed, middle.h)
bottom = bounds.bottom_band(detail_h)
return ScoreboardRegions(
bounds=bounds, logo_slot=logo_slot,
away_slot=away_slot, home_slot=home_slot, center_col=center_col,
status_band=status_band, score_area=score_area, detail_band=detail_band,
bottom_left=bottom.left_col(logo_slot),
bottom_right=bottom.right_col(logo_slot),
)
@dataclass(frozen=True)
class MediaRow:
"""Art/icon on the left, text column on the right (music's idiom)."""
art: Region
body: Region
def media_row(bounds: Region, *, ctx: Optional["LayoutContext"] = None,
square: bool = True, gap: Optional[int] = None) -> MediaRow:
"""Split a Region into an art slot and a body column.
With ``square=True`` the art slot is bounds.h wide (album-art style);
otherwise it takes the left half. The gap defaults to 2px scaled by the
context's geometry factor.
"""
if gap is None:
gap = ctx.px(2, minimum=1) if ctx else 2
art_w = bounds.h if square else bounds.w // 2
art_w = min(art_w, bounds.w)
art = bounds.left_col(art_w)
body = Region(bounds.x + art_w + gap, bounds.y,
bounds.w - art_w - gap, bounds.h)
return MediaRow(art=art, body=body)
-134
View File
@@ -1,134 +0,0 @@
"""
Background Cache Mixin for Sports Managers
This mixin provides common caching functionality to eliminate code duplication
across all sports managers. It implements the background service cache pattern
where Recent/Upcoming managers consume data from the background service cache.
"""
import time
from typing import Dict, Optional, Any, Callable
class BackgroundCacheMixin:
"""
Mixin class that provides background service cache functionality to sports managers.
This mixin eliminates code duplication by providing a common implementation
for the background service cache pattern used across all sports managers.
Note: For non-sports managers (weather, stocks, news, etc.), use
GenericCacheMixin instead. See src/generic_cache_mixin.py for details.
"""
def _fetch_data_with_background_cache(self,
sport_key: str,
api_fetch_method: Callable,
live_manager_class: type = None) -> Optional[Dict]:
"""
Common logic for fetching data with background service cache support.
This method implements the background service cache pattern:
1. Live managers always fetch fresh data
2. Recent/Upcoming managers try background cache first
3. Fallback to direct API call if background data unavailable
Args:
sport_key: Sport identifier (e.g., 'nba', 'nfl', 'ncaa_fb')
api_fetch_method: Method to call for direct API fetch
live_manager_class: Class to check if this is a live manager
Returns:
Cached or fresh data from API
"""
start_time = time.time()
cache_hit = False
cache_source = None
try:
# For Live managers, always fetch fresh data
if live_manager_class and isinstance(self, live_manager_class):
self.logger.info(f"[{sport_key.upper()}] Live manager - fetching fresh data")
result = api_fetch_method(use_cache=False)
cache_source = "live_fresh"
else:
# For Recent/Upcoming managers, try background service cache first
cache_key = self.cache_manager.generate_sport_cache_key(sport_key)
# Check if background service has fresh data
if self.cache_manager.is_background_data_available(cache_key, sport_key):
cached_data = self.cache_manager.get_background_cached_data(cache_key, sport_key)
if cached_data:
self.logger.info(f"[{sport_key.upper()}] Using background service cache for {cache_key}")
result = cached_data
cache_hit = True
cache_source = "background_cache"
else:
self.logger.warning(f"[{sport_key.upper()}] Background cache check passed but no data returned for {cache_key}")
result = None
cache_source = "background_miss"
else:
self.logger.info(f"[{sport_key.upper()}] Background data not available for {cache_key}")
result = None
cache_source = "background_unavailable"
# Fallback to direct API call if background data not available
if result is None:
self.logger.info(f"[{sport_key.upper()}] Fetching directly from API for {cache_key}")
result = api_fetch_method(use_cache=True)
cache_source = "api_fallback"
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
# Log performance metrics
self._log_fetch_performance(sport_key, duration, cache_hit, cache_source)
return result
except Exception as e:
duration = time.time() - start_time
self.logger.error(f"[{sport_key.upper()}] Error in background cache fetch after {duration:.2f}s: {e}")
self.cache_manager.record_fetch_time(duration)
raise
def _log_fetch_performance(self, sport_key: str, duration: float, cache_hit: bool, cache_source: str):
"""
Log detailed performance metrics for fetch operations.
Args:
sport_key: Sport identifier
duration: Fetch operation duration in seconds
cache_hit: Whether this was a cache hit
cache_source: Source of the data (background_cache, api_fallback, etc.)
"""
# Log basic performance info
self.logger.info(f"[{sport_key.upper()}] Fetch completed in {duration:.2f}s "
f"(cache_hit={cache_hit}, source={cache_source})")
# Log detailed metrics every 10 operations
if hasattr(self, '_fetch_count'):
self._fetch_count += 1
else:
self._fetch_count = 1
if self._fetch_count % 10 == 0:
metrics = self.cache_manager.get_cache_metrics()
self.logger.info(f"[{sport_key.upper()}] Cache Performance Summary - "
f"Hit Rate: {metrics['cache_hit_rate']:.2%}, "
f"Background Hit Rate: {metrics['background_hit_rate']:.2%}, "
f"API Calls Saved: {metrics['api_calls_saved']}")
def get_cache_performance_summary(self) -> Dict[str, Any]:
"""
Get cache performance summary for this manager.
Returns:
Dictionary containing cache performance metrics
"""
return self.cache_manager.get_cache_metrics()
def log_cache_performance(self):
"""Log current cache performance metrics."""
self.cache_manager.log_cache_metrics()
+110 -3
View File
@@ -29,6 +29,10 @@ except ImportError:
class SportsCore(ABC):
# Which ScoreboardSkin render method this class's display path maps to.
# SportsLive inherits the default; SportsUpcoming/SportsRecent override.
SKIN_MODE = "live"
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
self.logger = logger
self.config = config
@@ -99,6 +103,17 @@ class SportsCore(ABC):
self.last_update = 0
self.current_game = None
self.fonts = self._load_fonts()
# Optional visual skin (see docs/SKIN_SYSTEM.md). "skin" is either a
# skin id applied to all modes, or a per-mode mapping like
# {"live": "retro", "recent": "built-in"}. Loaded lazily on first
# render so a broken skin can never block startup.
self._skin_config = self.mode_config.get("skin")
self.skin_options = self.mode_config.get("skin_options", {}) or {}
self._skin = None
self._skin_load_attempted = False
self._skin_failures = 0
self._skin_slow_renders = 0
# Initialize dynamic team resolver and resolve favorite teams
self.dynamic_resolver = DynamicTeamResolver()
@@ -205,6 +220,95 @@ class SportsCore(ABC):
self.logger.error(f"Error in base _draw_scorebug_layout: {e}", exc_info=True)
def _resolve_skin_id(self) -> Optional[str]:
"""The skin id configured for this instance's mode, or None for the
built-in renderer. Accepts a plain id (all modes) or a per-mode
mapping ({"live": "retro-baseball", "recent": "built-in"})."""
skin_id = self._skin_config
if isinstance(skin_id, dict):
skin_id = skin_id.get(self.SKIN_MODE)
if not skin_id or not isinstance(skin_id, str) or skin_id == "built-in":
return None
return skin_id
def _get_skin(self):
"""Lazily load the configured skin once. Returns None (built-in
renderer) when no skin is configured or loading failed."""
if not self._skin_load_attempted:
self._skin_load_attempted = True
skin_id = self._resolve_skin_id()
if skin_id:
try:
from src.skin_system import skin_runtime
self._skin = skin_runtime.load_skin(
skin_id, sport=self.sport, sport_key=self.sport_key,
options=self.skin_options)
except Exception as e:
self.logger.error(f"Failed to load skin '{skin_id}': {e}", exc_info=True)
self._skin = None
return self._skin
def _render_game(self, game: Dict, force_clear: bool = False) -> None:
"""Render one game: try the configured skin first, fall back to the
built-in _draw_scorebug_layout. A skin that raises 3 times in a row
is disabled for the rest of the session."""
skin = self._get_skin()
if skin is not None and self._skin_failures < 3:
try:
from src.skin_system import skin_runtime
ctx = skin_runtime.build_context(self, game)
render = getattr(skin, f"render_{self.SKIN_MODE}")
started = time.monotonic()
handled = render(ctx, dict(game))
elapsed = time.monotonic() - started
if elapsed > 0.15 and self._skin_slow_renders < 5:
self._skin_slow_renders += 1
self.logger.warning(
f"Skin '{self._resolve_skin_id()}' took {elapsed * 1000:.0f}ms to "
f"render {self.SKIN_MODE} — slow renders stall the whole display loop")
if handled:
self._skin_failures = 0
self.display_manager.image.paste(ctx.canvas, (0, 0))
self.display_manager.update_display()
return
except Exception:
self._skin_failures += 1
outcome = ("disabling skin for this session" if self._skin_failures >= 3
else "falling back to built-in renderer")
self.logger.error(
f"Skin '{self._resolve_skin_id()}' failed rendering {self.SKIN_MODE} "
f"({self._skin_failures}/3); {outcome}", exc_info=True)
self._draw_scorebug_layout(game, force_clear)
def render_skin_card(self, game: Dict, size: tuple) -> Optional[Image.Image]:
"""Render one game as a standalone card via the configured skin —
for vegas mode and previews. Tries render_vegas_card at the given
size, then the mode renderer on a card-sized canvas. Returns None
when no skin is active or the skin declined, so callers can use
their default rendering."""
skin = self._get_skin()
if skin is None or self._skin_failures >= 3:
return None
try:
from src.skin_system import skin_runtime
ctx = skin_runtime.build_context(self, game, size=size)
card = skin.render_vegas_card(ctx, dict(game))
if card is not None:
return card
ctx = skin_runtime.build_context(self, game, size=size)
render = getattr(skin, f"render_{self.SKIN_MODE}")
if render(ctx, dict(game)):
return ctx.canvas
except Exception:
# Card failures count toward the same 3-strike session disable
# as display failures — a skin broken for vegas shouldn't get
# to throw on every scroll tick forever.
self._skin_failures += 1
self.logger.error(
f"Skin '{self._resolve_skin_id()}' card render failed "
f"({self._skin_failures}/3)", exc_info=True)
return None
def display(self, force_clear: bool = False) -> bool:
"""Common display method for all NCAA FB managers""" # Updated docstring
if not self.is_enabled: # Check if module is enabled
@@ -229,7 +333,7 @@ class SportsCore(ABC):
return False
try:
self._draw_scorebug_layout(self.current_game, force_clear)
self._render_game(self.current_game, force_clear)
# display_manager.update_display() should be called within subclass draw methods
# or after calling display() in the main loop. Let's keep it out of the base display.
return True
@@ -646,6 +750,8 @@ class SportsCore(ABC):
pass
class SportsUpcoming(SportsCore):
SKIN_MODE = "upcoming"
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
self.upcoming_games = [] # Store all fetched upcoming games initially
@@ -973,7 +1079,7 @@ class SportsUpcoming(SportsCore):
self.logger.debug(f"Switched to game index {self.current_game_index}")
if self.current_game:
self._draw_scorebug_layout(self.current_game, force_clear)
self._render_game(self.current_game, force_clear)
return True
# update_display() is called within _draw_scorebug_layout for upcoming
return False
@@ -984,6 +1090,7 @@ class SportsUpcoming(SportsCore):
class SportsRecent(SportsCore):
SKIN_MODE = "recent"
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager, cache_manager: CacheManager, logger: logging.Logger, sport_key: str):
super().__init__(config, display_manager, cache_manager, logger, sport_key)
@@ -1274,7 +1381,7 @@ class SportsRecent(SportsCore):
self.logger.debug(f"Switched to game index {self.current_game_index}")
if self.current_game:
self._draw_scorebug_layout(self.current_game, force_clear)
self._render_game(self.current_game, force_clear)
return True
# update_display() is called within _draw_scorebug_layout for recent
return False
+60 -14
View File
@@ -10,6 +10,7 @@ import time
import tempfile
import logging
import threading
import zlib
from typing import Dict, Any, Optional, Protocol
from datetime import datetime
@@ -53,6 +54,11 @@ class DiskCache:
self.cache_dir = cache_dir
self.logger = logger or logging.getLogger(__name__)
self._lock = threading.Lock()
# key -> adler32 of the last payload successfully written to the
# primary cache path; lets set() skip rewriting identical data
# (per-process only — worst case another process rewrites, never
# a missed write). Guarded by _lock.
self._write_digests: Dict[str, int] = {}
def get_cache_path(self, key: str) -> Optional[str]:
"""
@@ -68,14 +74,15 @@ class DiskCache:
return None
return os.path.join(self.cache_dir, f"{key}.json")
def get(self, key: str, max_age: int = 300) -> Optional[Dict[str, Any]]:
def get(self, key: str, max_age: Optional[int] = 300) -> Optional[Dict[str, Any]]:
"""
Get data from disk cache.
Args:
key: Cache key
max_age: Maximum age in seconds
max_age: Maximum age in seconds; None disables age-based expiry
(the record never counts as stale). Mirrors MemoryCache.get.
Returns:
Cached data or None if not found or expired
"""
@@ -105,7 +112,13 @@ class DiskCache:
record_ts = None
now = time.time()
if record_ts is None or (now - record_ts) <= max_age:
# max_age=None means "never expires" (mirrors MemoryCache and the
# cache_manager docstring). Guard it explicitly — otherwise the
# comparison below raises TypeError and the record is treated as a
# miss, which silently breaks callers that persist long-lived state
# via get(key, max_age=None) (e.g. plugin health/metrics that must
# survive restarts and be read cross-process).
if record_ts is None or max_age is None or (now - record_ts) <= max_age:
return record
else:
# Stale on disk; keep file for potential diagnostics but treat as miss
@@ -148,10 +161,35 @@ class DiskCache:
cache_path = self.get_cache_path(key)
if not cache_path:
return
# Serialize once, compact (no indent): the payload is reused by every
# write path below, and cache files are machine-read only — indenting
# them just multiplied the bytes written to the SD card.
try:
payload = json.dumps(data, cls=DateTimeEncoder)
except (TypeError, ValueError) as e:
self.logger.warning("Cache data for key '%s' not serializable: %s", key, e)
return
digest = zlib.adler32(payload.encode('utf-8'))
try:
# Atomic write to avoid partial/corrupt files
with self._lock:
# Skip the disk entirely when this exact payload was already
# written for this key (plugins re-save unchanged API data
# every update cycle — each write is real SD-card wear).
# Refresh the file mtime so records that rely on it for TTL
# (no embedded 'timestamp') don't expire early; a metadata
# touch is journal-cheap compared to rewriting the data.
if self._write_digests.get(key) == digest:
try:
os.utime(cache_path, None)
return
except OSError:
# File vanished or perms changed — fall through and write
self._write_digests.pop(key, None)
tmp_dir = os.path.dirname(cache_path)
# Try to create temp file in cache directory first
# If that fails due to permissions, fall back to direct write
@@ -174,13 +212,17 @@ class DiskCache:
fd = None
if tmp_path and fd is not None:
# Use atomic write with temp file
# Atomic write with temp file. No fsync: os.replace
# already guarantees readers never see a torn file,
# and cache data is re-fetchable — forcing a disk
# flush per write was the single biggest SD-card
# wear source (dozens of fsyncs/min on API-heavy
# installs) for data that can be re-downloaded.
try:
with os.fdopen(fd, 'w', encoding='utf-8') as tmp_file:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
tmp_file.flush()
os.fsync(tmp_file.fileno())
tmp_file.write(payload)
os.replace(tmp_path, cache_path)
self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -196,9 +238,8 @@ class DiskCache:
# Fallback: direct write (not atomic, but better than failing)
try:
with open(cache_path, 'w', encoding='utf-8') as cache_file:
json.dump(data, cache_file, indent=4, cls=DateTimeEncoder)
cache_file.flush()
os.fsync(cache_file.fileno())
cache_file.write(payload)
self._write_digests[key] = digest
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(cache_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -222,9 +263,12 @@ class DiskCache:
pass
if os.path.isdir(fallback_dir) and os.access(fallback_dir, os.W_OK):
# NOTE: no digest record here — the fallback file
# is a different path, so future sets must keep
# retrying the primary location.
fallback_path = os.path.join(fallback_dir, os.path.basename(cache_path))
with open(fallback_path, 'w', encoding='utf-8') as tmp_file:
json.dump(data, tmp_file, indent=4, cls=DateTimeEncoder)
tmp_file.write(payload)
# Set proper permissions: 660 (rw-rw----) for group-readable cache files
try:
os.chmod(fallback_path, 0o660) # nosec B103 - intentional; web UI and service share a group
@@ -265,6 +309,7 @@ class DiskCache:
with self._lock:
if key:
self._write_digests.pop(key, None)
cache_path = self.get_cache_path(key)
if cache_path and os.path.exists(cache_path):
try:
@@ -273,6 +318,7 @@ class DiskCache:
self.logger.warning("Could not remove cache file %s: %s", cache_path, e)
else:
# Clear all cache files
self._write_digests.clear()
if os.path.exists(self.cache_dir):
for filename in os.listdir(self.cache_dir):
if filename.endswith('.json'):
+13 -3
View File
@@ -574,9 +574,19 @@ class CacheManager:
}
return self.save_cache(data_type, cache_data)
def get(self, key: str, max_age: int = 300) -> Optional[Dict[str, Any]]:
"""Get data from cache if it exists and is not stale."""
cached_data = self.get_cached_data(key, max_age)
def get(self, key: str, max_age: Optional[int] = 300,
memory_ttl: Optional[int] = None) -> Optional[Dict[str, Any]]:
"""Get data from cache if it exists and is not stale.
Args:
key: Cache key
max_age: Max age (seconds) for the on-disk entry; None never expires.
memory_ttl: Max age (seconds) for the in-memory entry. Pass 0 to
bypass the memory tier and force a fresh read from disk used by
cross-process readers that must observe another process's latest
write rather than a stale first snapshot. Defaults to max_age.
"""
cached_data = self.get_cached_data(key, max_age, memory_ttl=memory_ttl)
if cached_data and 'data' in cached_data:
return cached_data['data']
return cached_data
+22
View File
@@ -2,6 +2,28 @@
This directory contains reusable utilities and helpers for LEDMatrix plugins and core modules.
## Adaptive Layout & Images (`src/adaptive_layout.py`, `src/adaptive_images.py`)
The recommended way to lay out plugins that render legibly on **any** panel
size (64x32 through 256x128+) without hand-tuned coordinates. Re-exported
from `src.common` for convenience; canonical import paths are
`src.adaptive_layout` / `src.adaptive_images`.
```python
# Every BasePlugin already has self.layout and the draw helpers:
regs = scoreboard_regions(self.layout.bounds, ctx=self.layout)
self.draw_image(away_logo, regs.away_slot, mode="fill_height",
crop_to_ink=True, cache_key=f"logo:{abbr}")
self.draw_fit(score_text, regs.score_area) # largest crisp font that fits
self.draw_fit(status, regs.status_band)
```
Key pieces: `Region` (rect algebra: bands/columns/splits/offset),
font ladders (`LADDER_GRID`, `LADDER_ARCADE` — discrete crisp sizes, never
fractional scaling), `LayoutContext` (`fit_text`, `fit_image`, `by_tier`,
`px`), and composite carvers `scoreboard_regions()` / `media_row()`.
Full guide: [docs/ADAPTIVE_LAYOUT.md](../../docs/ADAPTIVE_LAYOUT.md).
## Error Handling (`error_handler.py`)
Common error handling patterns and utilities:
+43
View File
@@ -26,6 +26,31 @@ from src.common.scroll_helper import ScrollHelper
from src.common.logo_helper import LogoHelper
from src.common.text_helper import TextHelper
# Adaptive layout & images (canonical homes: src.adaptive_layout /
# src.adaptive_images — re-exported here so plugin authors find them in the
# blessed-helpers package). See docs/ADAPTIVE_LAYOUT.md.
from src.adaptive_layout import (
Region,
LayoutContext,
FontStep,
FontLadder,
LADDER_GRID,
LADDER_ARCADE,
FitResult,
draw_fitted_text,
ScoreboardRegions,
scoreboard_regions,
MediaRow,
media_row,
)
from src.adaptive_images import (
ImageFitResult,
fit_image,
draw_fitted_image,
RESAMPLE_LANCZOS,
RESAMPLE_NEAREST,
)
__all__ = [
'handle_file_operation',
'handle_json_operation',
@@ -37,4 +62,22 @@ __all__ = [
'ScrollHelper',
'LogoHelper',
'TextHelper',
# adaptive layout & images
'Region',
'LayoutContext',
'FontStep',
'FontLadder',
'LADDER_GRID',
'LADDER_ARCADE',
'FitResult',
'draw_fitted_text',
'ScoreboardRegions',
'scoreboard_regions',
'MediaRow',
'media_row',
'ImageFitResult',
'fit_image',
'draw_fitted_image',
'RESAMPLE_LANCZOS',
'RESAMPLE_NEAREST',
]
-328
View File
@@ -1,328 +0,0 @@
"""
Example: Basketball Plugin using LEDMatrix Common Helpers
This example shows how to refactor the basketball plugin to use the
ledmatrix-common package for cleaner, more maintainable code.
"""
from pathlib import Path
from typing import Any, Dict, List, Optional
# Import common helpers
from src.common import (
LogoHelper, TextHelper, APIHelper, DisplayHelper,
GameHelper, ConfigHelper
)
from src.plugin_system.base_plugin import BasePlugin
class BasketballPluginManager(BasePlugin):
"""
Basketball scoreboard plugin using LEDMatrix Common helpers.
This version is much cleaner and more maintainable than the original
because it delegates common functionality to the shared helpers.
"""
def __init__(
self,
plugin_id: str,
config: Dict[str, Any],
display_manager,
cache_manager,
plugin_manager
):
"""Initialize the basketball plugin with common helpers."""
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
# Get display dimensions
self.display_width = display_manager.matrix.width
self.display_height = display_manager.matrix.height
# Initialize common helpers
self._init_helpers()
# Load configuration
self._load_config()
# State tracking
self.current_games = []
self.current_game = None
# Log initialization
enabled_leagues = [k for k, v in self.league_configs.items() if v['enabled']]
self.logger.info(f"Basketball plugin initialized with leagues: {enabled_leagues}")
def _init_helpers(self):
"""Initialize all common helpers."""
# Logo helper for team logos
self.logo_helper = LogoHelper(
display_width=self.display_width,
display_height=self.display_height,
logger=self.logger
)
# Text helper for rendering
self.text_helper = TextHelper(logger=self.logger)
self.fonts = self.text_helper.load_fonts()
# API helper for ESPN data
self.api_helper = APIHelper(
cache_manager=self.cache_manager,
logger=self.logger
)
# Display helper for layouts
self.display_helper = DisplayHelper(
display_width=self.display_width,
display_height=self.display_height,
logger=self.logger
)
# Game helper for data processing
self.game_helper = GameHelper(
timezone_str=self.config.get('timezone', 'UTC'),
logger=self.logger
)
# Config helper for configuration management
self.config_helper = ConfigHelper(logger=self.logger)
def _load_config(self):
"""Load and validate configuration."""
# Get basketball-specific config
basketball_config = self.config_helper.get_sports_config(self.config, 'basketball')
# Build league configurations
self.league_configs = {
'nba': {
'enabled': basketball_config.get('nba_enabled', True),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/nba/scoreboard',
'logo_dir': Path('assets/sports/nba_logos'),
'favorite_teams': basketball_config.get('nba_favorite_teams', []),
'display_modes': {
'nba_live': basketball_config.get('nba_display_modes_live', True),
'nba_recent': basketball_config.get('nba_display_modes_recent', True),
'nba_upcoming': basketball_config.get('nba_display_modes_upcoming', True),
},
},
'wnba': {
'enabled': basketball_config.get('wnba_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/wnba/scoreboard',
'logo_dir': Path('assets/sports/wnba_logos'),
'favorite_teams': basketball_config.get('wnba_favorite_teams', []),
'display_modes': {
'wnba_live': basketball_config.get('wnba_display_modes_live', True),
'wnba_recent': basketball_config.get('wnba_display_modes_recent', True),
'wnba_upcoming': basketball_config.get('wnba_display_modes_upcoming', True),
},
},
'ncaam': {
'enabled': basketball_config.get('ncaam_basketball_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/mens-college-basketball/scoreboard',
'logo_dir': Path('assets/sports/ncaa_logos'),
'favorite_teams': basketball_config.get('ncaam_basketball_favorite_teams', []),
'display_modes': {
'ncaam_basketball_live': basketball_config.get('ncaam_basketball_display_modes_live', True),
'ncaam_basketball_recent': basketball_config.get('ncaam_basketball_display_modes_recent', True),
'ncaam_basketball_upcoming': basketball_config.get('ncaam_basketball_display_modes_upcoming', True),
},
},
'ncaaw': {
'enabled': basketball_config.get('ncaaw_basketball_enabled', False),
'url': 'https://site.api.espn.com/apis/site/v2/sports/basketball/womens-college-basketball/scoreboard',
'logo_dir': Path('assets/sports/ncaa_logos'),
'favorite_teams': basketball_config.get('ncaaw_basketball_favorite_teams', []),
'display_modes': {
'ncaaw_basketball_live': basketball_config.get('ncaaw_basketball_display_modes_live', True),
'ncaaw_basketball_recent': basketball_config.get('ncaaw_basketball_display_modes_recent', True),
'ncaaw_basketball_upcoming': basketball_config.get('ncaaw_basketball_display_modes_upcoming', True),
},
},
}
def update(self) -> None:
"""Update game data for all enabled leagues."""
try:
all_games = []
for league_key, league_config in self.league_configs.items():
if not league_config['enabled']:
continue
games = self._fetch_league_games(league_key, league_config)
for game in games:
game['league_key'] = league_key
game['league_config'] = league_config
all_games.extend(games)
self.current_games = all_games
self.logger.debug(f"Updated basketball data: {len(all_games)} total games")
except Exception as e:
self.logger.error(f"Error updating basketball data: {e}", exc_info=True)
def _fetch_league_games(self, league_key: str, league_config: Dict) -> List[Dict]:
"""Fetch games for a specific league using API helper."""
try:
# Use API helper to fetch ESPN data with caching
data = self.api_helper.fetch_espn_scoreboard(
sport='basketball',
league=league_key,
cache_key=f"basketball_{league_key}",
cache_ttl=300 # 5 minutes cache
)
if not data:
return []
# Use game helper to process events
events = data.get('events', [])
games = self.game_helper.process_games(events, sport='basketball')
# Add logo paths to games
for game in games:
logo_dir = league_config['logo_dir']
game['home_logo_path'] = logo_dir / f"{game['home_abbr']}.png"
game['away_logo_path'] = logo_dir / f"{game['away_abbr']}.png"
return games
except Exception as e:
self.logger.error(f"Error fetching {league_key} games: {e}", exc_info=True)
return []
def display(self, force_clear: bool = False, display_mode: str = None) -> None:
"""Display basketball games using display helper."""
try:
mode = display_mode or self._determine_display_mode()
if not mode:
self._display_no_games()
return
# Filter games for mode
filtered_games = self._filter_games_for_mode(mode)
if not filtered_games:
self._display_no_games()
return
# Display first game
self.current_game = filtered_games[0]
self._draw_scorebug_layout(self.current_game, force_clear)
except Exception as e:
self.logger.error(f"Error displaying game: {e}", exc_info=True)
def _determine_display_mode(self) -> Optional[str]:
"""Determine display mode based on available games."""
# Priority: live > recent > upcoming
for game in self.current_games:
if game.get('is_live'):
return f"{game['league_key']}_live"
for game in self.current_games:
if game.get('is_final'):
return f"{game['league_key']}_recent"
for game in self.current_games:
if game.get('is_upcoming'):
return f"{game['league_key']}_upcoming"
return None
def _filter_games_for_mode(self, mode: str) -> List[Dict]:
"""Filter games based on display mode."""
filtered = []
for game in self.current_games:
league_config = game.get('league_config', {})
display_modes = league_config.get('display_modes', {})
if mode in display_modes and display_modes[mode]:
if 'live' in mode and game.get('is_live'):
filtered.append(game)
elif 'recent' in mode and game.get('is_final'):
filtered.append(game)
elif 'upcoming' in mode and game.get('is_upcoming'):
filtered.append(game)
return filtered[:5]
def _draw_scorebug_layout(self, game: Dict, force_clear: bool = False) -> None:
"""Draw the basketball scorebug layout using display helper."""
try:
# Load logos using logo helper
home_logo = self.logo_helper.load_logo(
game['home_abbr'],
game['home_logo_path']
)
away_logo = self.logo_helper.load_logo(
game['away_abbr'],
game['away_logo_path']
)
if not home_logo or not away_logo:
self.logger.error("Failed to load logos")
self._display_error("Logo Error")
return
# Use display helper to create scorebug layout
final_img = self.display_helper.draw_scorebug_layout(
game_data=game,
fonts=self.fonts,
home_logo=home_logo,
away_logo=away_logo
)
# Display the image
self.display_manager.image.paste(final_img, (0, 0))
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error drawing scorebug: {e}", exc_info=True)
def _display_no_games(self) -> None:
"""Display 'no games' message using display helper."""
try:
img = self.display_helper.draw_no_data_message("No Games")
self.display_manager.image = img.copy()
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error displaying no games: {e}", exc_info=True)
def _display_error(self, message: str) -> None:
"""Display error message using display helper."""
try:
img = self.display_helper.draw_error_message(message)
self.display_manager.image = img.copy()
self.display_manager.update_display()
except Exception as e:
self.logger.error(f"Error displaying error message: {e}", exc_info=True)
def get_display_duration(self) -> float:
"""Get display duration."""
return self.config.get('display_duration', 15)
def cleanup(self) -> None:
"""Cleanup resources."""
self.current_games = []
self.logger.info("Basketball plugin cleaned up")
# Example usage and benefits:
"""
Benefits of using LEDMatrix Common helpers:
1. **Cleaner Code**: The plugin is much shorter and more readable
2. **Reusable Components**: Common functionality is shared across plugins
3. **Better Testing**: Each helper can be tested independently
4. **Easier Maintenance**: Bug fixes in helpers benefit all plugins
5. **Consistent Behavior**: All plugins use the same underlying logic
6. **Reduced Dependencies**: Plugins don't need to import LEDMatrix core
7. **Better Error Handling**: Centralized error handling in helpers
8. **Configuration Management**: Consistent config handling across plugins
The original basketball plugin was 326 lines. This version is much cleaner
and delegates most functionality to the common helpers, making it easier to
maintain and extend.
"""
+13 -2
View File
@@ -72,9 +72,20 @@ class LogoHelper:
Returns:
PIL Image object or None if loading fails
Note: for new adaptive-layout code prefer ``BasePlugin.draw_image``
/ ``LayoutContext.fit_image`` (src/adaptive_images.py) for the
fitting step LogoHelper remains useful for its download and
placeholder logic.
"""
# Check cache first
cache_key = f"{team_abbr}_{logo_path}"
# Resolve the effective target size BEFORE the cache lookup so the
# key is size-qualified — a panel-size change must not return a
# logo resized for the old dimensions.
if max_width is None:
max_width = int(self.display_width * 1.5)
if max_height is None:
max_height = int(self.display_height * 1.5)
cache_key = f"{team_abbr}_{logo_path}_{max_width}x{max_height}"
if cache_key in self._logo_cache:
self.logger.debug(f"Using cached logo for {team_abbr}")
# Update LRU order (move to end)
+191
View File
@@ -8,13 +8,34 @@ files that need to be accessible by both root service and web user.
import os
import logging
import re
import shutil as _shutil
import subprocess
import sys
from pathlib import Path
from typing import Optional
logger = logging.getLogger(__name__)
# Matches the credentials portion of a "scheme://user:pass@host" URL, so pip's
# own error output can be logged/displayed without echoing back a private
# index URL's embedded basic-auth secret verbatim (e.g. from a
# requirements.txt --index-url line or the PIP_INDEX_URL env var).
_URL_CREDENTIALS_RE = re.compile(r'://[^/\s@:]+:[^/\s@]+@')
def _redact_url_credentials(text: Optional[str]) -> str:
"""Replace embedded user:pass@ URL credentials in text with a placeholder.
Safe to call on any subprocess output destined for logs: it only ever
shortens/replaces the credential substring, never changes the presence
or absence of the specific fixed phrases callers check for
(e.g. "a password is required"), so it can't affect control flow.
"""
if not text:
return text or ""
return _URL_CREDENTIALS_RE.sub('://***:***@', text)
# System directories that should never have their permissions modified
# These directories have special system-level permissions that must be preserved
PROTECTED_SYSTEM_DIRECTORIES = { # nosec B108 - these are checked to PREVENT permission changes, not to use as temp paths
@@ -125,6 +146,60 @@ def ensure_file_permissions(path: Path, mode: int = 0o644) -> None:
raise
_shared_group_gid_cache: Optional[int] = None
def get_shared_group_gid() -> Optional[int]:
"""
Return the gid that should own config/secrets files shared between the
root-run ``ledmatrix.service`` (main display) and the non-root user that
``ledmatrix-web.service`` runs as (see install_web_service.sh, which sets
``User=$SUDO_USER``).
Resolved once from the project root directory's current group (normally
the login user's group from the initial ``git clone``), since that user
is stable across reinstalls unlike any single file's ownership.
Returns:
The gid, or None if it cannot be determined.
"""
global _shared_group_gid_cache
if _shared_group_gid_cache is not None:
return _shared_group_gid_cache
try:
project_root = Path(__file__).resolve().parent.parent.parent
_shared_group_gid_cache = project_root.stat().st_gid
return _shared_group_gid_cache
except OSError:
return None
def ensure_shared_group_ownership(path: Path) -> None:
"""
Best-effort chgrp of ``path`` to the shared group (see
:func:`get_shared_group_gid`) when running as root.
Only root can change a file's group to one the calling process isn't a
member of, which is exactly the case that causes the web interface
(running as a non-root user) to get ``PermissionError`` reading files
the root-run display service just wrote with a 0o640/2775 mode: the mode
is group-readable, but without this the group is root's, not the web
user's. Silently does nothing if not running as root or on any error —
this is a hardening step, not a required one.
"""
if os.geteuid() != 0:
return
gid = get_shared_group_gid()
if gid is None:
return
try:
if path.exists() and path.stat().st_gid != gid:
os.chown(path, -1, gid)
logger.debug(f"Set shared group ownership (gid {gid}) on {path}")
except OSError as e:
logger.debug(f"Could not set shared group ownership on {path}: {e}")
def get_config_file_mode(file_path: Path) -> int:
"""
Return appropriate permission mode for config files.
@@ -287,3 +362,119 @@ def sudo_remove_directory(path: Path, allowed_bases: Optional[list] = None) -> b
logger.error(f"Unexpected error during sudo helper for {path}: {e}")
return False
def install_requirements_file(req_file: Path, timeout: int = 300) -> subprocess.CompletedProcess:
"""
Install a requirements.txt file for a plugin (or the project itself).
Prefers the vetted sudo wrapper (scripts/fix_perms/safe_pip_install.sh) so
packages end up visible to root-run ledmatrix.service, not just to
whichever non-root user happens to run the calling process (e.g. the web
interface). Falls back to installing with the calling process's own
interpreter if the wrapper isn't set up yet (the admin hasn't run
scripts/install/configure_web_sudo.sh), so dependency installation still
does *something* useful rather than hard-failing.
Always installs with the interpreter that will actually run the code
(``sys.executable`` in the fallback path, the wrapper's ``python3`` in the
sudo path) rather than a bare ``pip``/``pip3`` off PATH, which can
silently resolve to a different Python installation (e.g. system Python
vs. a virtualenv) than the one importing the package at runtime.
Args:
req_file: Path to a requirements.txt file
timeout: Subprocess timeout in seconds
Returns:
subprocess.CompletedProcess from the pip (or wrapper) invocation.
Never raises on a non-zero exit; callers should check ``returncode``.
``stdout`` is prefixed with an explanatory note when the root wrapper
was unavailable and the fallback path was used.
"""
project_root = Path(__file__).resolve().parent.parent.parent
wrapper = project_root / "scripts" / "fix_perms" / "safe_pip_install.sh"
if wrapper.exists():
# See sudo_remove_directory / configure_web_sudo.sh for why bash must
# be invoked with an explicit, known path rather than relying on the
# wrapper's shebang: sudoers matches the exact command line.
bash_candidates = []
for candidate in ("/usr/bin/bash", "/bin/bash", _shutil.which("bash")):
if candidate and candidate not in bash_candidates:
bash_candidates.append(candidate)
result = None
for bash_path in bash_candidates:
# bash_path and wrapper are fixed, known-good paths, and
# safe_pip_install.sh independently re-validates req_file is an
# allowed requirements.txt before installing anything as root.
result = subprocess.run( # nosec B603 - no shell invoked (list-form argv) # nosemgrep
["sudo", "-n", bash_path, str(wrapper), str(req_file)],
capture_output=True, text=True, timeout=timeout, cwd=str(project_root)
)
# Redact immediately: pip can echo a private index URL's embedded
# basic-auth credentials back in its own error/progress output
# (e.g. from a requirements.txt --index-url line). Doesn't affect
# the fixed-phrase "denied" check below -- those phrases never
# overlap with URL syntax.
result.stderr = _redact_url_credentials(result.stderr)
result.stdout = _redact_url_credentials(result.stdout)
if result.returncode == 0:
return result
# Distinguish "sudo rejected this exact command line" (worth
# trying the next bash candidate) from "sudo ran it but pip
# itself failed" (a real error — stop and surface it).
denied = any(
phrase in result.stderr
for phrase in ("a password is required", "is not allowed to run", "no tty present")
)
if not denied:
# Deliberately don't interpolate req_file or the pip output here:
# this log line is scanner-visible, and a static analyzer can't
# tell "already redacted above" from "still raw" just by looking
# at this call in isolation. The full (redacted) text is still
# available to callers via the returned CompletedProcess.
logger.warning(
"Root pip install failed (rc=%s); see the returned "
"CompletedProcess.stderr for details.",
result.returncode,
)
return result
# Same reasoning as above: no req_file / pip-output interpolation in
# this log line, only in the returned note/CompletedProcess.
logger.warning(
"Root pip install wrapper denied via sudo for all candidates; "
"falling back to user-level install. See the returned "
"CompletedProcess.stderr for details."
)
note = (
f"[Root install unavailable ({(result.stderr.strip() if result else 'sudo denied') or 'sudo denied'}); "
"installed for the current process's user only. Packages may not be "
"visible to ledmatrix.service if it runs as a different user — "
"run scripts/install/configure_web_sudo.sh to fix this.]\n"
)
else:
logger.warning(
"safe_pip_install.sh not found; falling back to user-level install."
)
note = (
"[safe_pip_install.sh not found; installed for the current process's "
"user only. Run scripts/install/configure_web_sudo.sh to enable "
"root installs visible to ledmatrix.service.]\n"
)
# sys.executable is this process's own interpreter (not
# attacker-influenced), and req_file is a Path built internally by callers
# (store_manager.py plugin paths, PROJECT_ROOT/requirements.txt), never
# raw external/user input. --ignore-installed matches safe_pip_install.sh:
# apt-managed packages (e.g. python3-requests) ship no pip RECORD file, so
# upgrading them would otherwise abort with "uninstall-no-record-file".
result = subprocess.run( # nosec B603 - no shell invoked (list-form argv) # nosemgrep
[sys.executable, "-m", "pip", "install", "--break-system-packages", "--ignore-installed", "-r", str(req_file)],
capture_output=True, text=True, timeout=timeout, cwd=str(project_root)
)
result.stderr = _redact_url_credentials(result.stderr)
result.stdout = note + _redact_url_credentials(result.stdout)
return result
+68
View File
@@ -0,0 +1,68 @@
"""Snapshot write policy for the display preview mirror.
The display service mirrors frames to /tmp/led_matrix_preview.png, which
serves two consumers with different needs:
- The web UI's live preview (SSE reader in web_interface/app.py) wants
fresh frames but only while a browser is actually watching.
- The health check (web_interface/blueprints/api_v3.py, hardware status)
uses the file's AGE as a liveness proxy: age >= 60s reads as degraded.
PNG-encoding every frame at 5 fps forever identical frames, no viewers
was one of the biggest fixed CPU costs on the Pi. This module is the pure
decision logic (extracted so it's unit-testable off-Pi; display_manager
imports rgbmatrix unconditionally and can't be):
WRITE encode + atomically replace the snapshot file
TOUCH os.utime only: keeps the health-check mtime fresh and lets
the SSE reader (mtime-gated) resend at a low rate, without
paying for a PNG encode of an unchanged frame
SKIP do nothing
Policy:
- With a fresh viewer marker: changed frames write at up to 1/VIEWER_INTERVAL.
- Without viewers: changed frames still write at 1/IDLE_INTERVAL so the
preview page shows something recent on open.
- Unchanged frames are never re-encoded; the mtime is touched every
TOUCH_INTERVAL so the health check (60s threshold) never degrades.
If any constant here changes, re-check the health threshold in
api_v3.py (get_hardware_status) TOUCH_INTERVAL must stay well under it.
"""
from enum import Enum
# Snapshot cadence with a browser preview open (seconds).
VIEWER_INTERVAL = 0.2
# Snapshot cadence with no viewers — cheap freshness for page-open (seconds).
IDLE_INTERVAL = 30.0
# Max age of the last write/touch before bumping mtime for the health
# check. MUST stay well under api_v3's 60s degraded threshold.
TOUCH_INTERVAL = 20.0
# A viewer marker older than this no longer counts as a live viewer.
VIEWER_MARKER_FRESH_SEC = 5.0
class SnapshotAction(Enum):
WRITE = "write"
TOUCH = "touch"
SKIP = "skip"
def decide(now: float, last_write_ts: float, last_touch_ts: float,
viewer_fresh: bool, frame_changed: bool) -> SnapshotAction:
"""Decide what to do with the current frame.
Args:
now: current monotonic-ish timestamp (same clock as the ts args)
last_write_ts: when a frame was last actually encoded+written
last_touch_ts: when the file mtime was last bumped (write or touch)
viewer_fresh: a browser preview is currently watching
frame_changed: the frame differs from the last WRITTEN frame
"""
interval = VIEWER_INTERVAL if viewer_fresh else IDLE_INTERVAL
if frame_changed and (now - last_write_ts) >= interval:
return SnapshotAction.WRITE
if (now - max(last_write_ts, last_touch_ts)) >= TOUCH_INTERVAL:
return SnapshotAction.TOUCH
return SnapshotAction.SKIP
+7 -9
View File
@@ -11,6 +11,9 @@ from typing import Dict, List, Optional, Tuple, Union
from PIL import Image, ImageDraw, ImageFont
# Shared throwaway draw surface for measuring text without a target canvas.
_measure_draw = ImageDraw.Draw(Image.new("RGB", (1, 1)))
class TextHelper:
"""
@@ -112,10 +115,10 @@ class TextHelper:
Width in pixels
"""
try:
return draw.textlength(text, font=font)
return int(_measure_draw.textlength(text, font=font))
except AttributeError:
# Fallback for older PIL versions
bbox = draw.textbbox((0, 0), text, font=font)
bbox = _measure_draw.textbbox((0, 0), text, font=font)
return bbox[2] - bbox[0]
def get_text_height(self, text: str, font: ImageFont.ImageFont) -> int:
@@ -129,13 +132,8 @@ class TextHelper:
Returns:
Height in pixels
"""
try:
bbox = draw.textbbox((0, 0), text, font=font)
return bbox[3] - bbox[1]
except AttributeError:
# Fallback for older PIL versions
bbox = draw.textbbox((0, 0), text, font=font)
return bbox[3] - bbox[1]
bbox = _measure_draw.textbbox((0, 0), text, font=font)
return bbox[3] - bbox[1]
def get_text_dimensions(self, text: str, font: ImageFont.ImageFont) -> Tuple[int, int]:
"""
+71 -5
View File
@@ -38,6 +38,7 @@ from src.config_manager_atomic import (
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
ensure_shared_group_ownership,
get_config_file_mode,
get_config_dir_mode
)
@@ -56,6 +57,13 @@ class ConfigManager:
self.secrets_path: str = secrets_path or "config/config_secrets.json"
self.template_path: str = "config/config.template.json"
self.config: Dict[str, Any] = {}
# (mtime_ns, size) signature of (config, secrets, template) at the
# last successful load. load_config() skips the full re-read (3 file
# parses + recursive template migration) when nothing changed —
# ~30 web request handlers call it, some 2-3x per request. Cross-
# process freshness is preserved: another process's save bumps the
# mtime, so the next load here re-reads.
self._loaded_sig: Optional[tuple] = None
self.logger: logging.Logger = get_logger(__name__)
# Initialize atomic config manager
@@ -122,6 +130,14 @@ class ConfigManager:
# Update in-memory config if save was successful
if result.status == SaveResultStatus.SUCCESS:
self.config = new_config_data
# In-memory config now matches what was just written; refresh
# the load signature so the fast path stays valid. NOTE: the
# in-memory copy includes merged secrets; the on-disk file has
# them stripped — the fast path returning self.config preserves
# exactly the pre-cache behavior (load-after-save also returned
# the secret-merged self.config only after re-reading secrets;
# here secrets file is unchanged, so contents are equivalent).
self._loaded_sig = self._files_signature()
self.logger.info(f"Configuration successfully saved atomically to {os.path.abspath(self.config_path)}")
elif result.status == SaveResultStatus.ROLLED_BACK:
# Reload config from file after rollback
@@ -179,13 +195,36 @@ class ConfigManager:
atomic_mgr = self._get_atomic_manager()
return atomic_mgr.validate_config_file(config_path)
def _files_signature(self) -> tuple:
"""(mtime_ns, size) of config/secrets/template, None for missing —
cheap staleness probe (3 stats) for the load_config fast path."""
sig = []
for path in (self.config_path, self.secrets_path, self.template_path):
try:
st = os.stat(path)
sig.append((st.st_mtime_ns, st.st_size))
except OSError:
sig.append(None)
return tuple(sig)
def load_config(self) -> Dict[str, Any]:
"""Load configuration from JSON files."""
"""Load configuration from JSON files.
Fast path: when config.json, config_secrets.json and the template
are all unchanged since the last successful load (mtime_ns + size),
the already-parsed self.config is returned without touching the
files same aliasing semantics as the full path, which also
returns self.config.
"""
try:
current_sig = self._files_signature()
if self.config and self._loaded_sig == current_sig:
return self.config
# Check if config file exists, if not create from template
if not os.path.exists(self.config_path):
self._create_config_from_template()
# Load main config
self.logger.info(f"Attempting to load config from: {os.path.abspath(self.config_path)}")
with open(self.config_path, 'r') as f:
@@ -196,6 +235,11 @@ class ConfigManager:
# Load and merge secrets if they exist (be permissive on errors)
if os.path.exists(self.secrets_path):
# Self-heal stale group ownership (e.g. the root-run display
# service wrote this file before the web user was granted
# group access) before every load attempt; no-op unless
# running as root and the group is already wrong.
ensure_shared_group_ownership(Path(self.secrets_path))
try:
with open(self.secrets_path, 'r') as f:
secrets = json.load(f)
@@ -205,7 +249,10 @@ class ConfigManager:
self.logger.warning(f"Secrets file not readable ({self.secrets_path}): {e}. Continuing without secrets.")
except (json.JSONDecodeError, OSError) as e:
self.logger.warning(f"Error reading secrets file ({self.secrets_path}): {e}. Continuing without secrets.")
# Signature taken AFTER load + migration (migration may write the
# config back), so it reflects exactly what was read/written.
self._loaded_sig = self._files_signature()
return self.config
except FileNotFoundError as e:
@@ -264,7 +311,8 @@ class ConfigManager:
json.dump(config_to_write, f, indent=4)
# Update the in-memory config to the new state (which includes secrets for runtime)
self.config = new_config_data
self.config = new_config_data
self._loaded_sig = self._files_signature()
self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
if secrets_content:
self.logger.info("Secret values were preserved in memory and not written to the main config file.")
@@ -321,6 +369,7 @@ class ConfigManager:
# Set proper file permissions after creation
config_path_obj = Path(self.config_path)
ensure_file_permissions(config_path_obj, get_config_file_mode(config_path_obj))
ensure_shared_group_ownership(config_path_obj)
self.logger.info(f"Created config.json from template at {os.path.abspath(self.config_path)}")
@@ -433,6 +482,11 @@ class ConfigManager:
self.logger.error(error_msg)
raise ConfigError(error_msg, config_path=path_to_load)
if file_type == "secrets":
# Best-effort self-heal: no-op unless running as root and the
# group is stale (see load_config for why this can happen).
ensure_shared_group_ownership(Path(path_to_load))
try:
with open(path_to_load, 'r') as f:
return json.load(f)
@@ -440,7 +494,18 @@ class ConfigManager:
error_msg = f"Error parsing {file_type} configuration file: {path_to_load}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
except (IOError, OSError, PermissionError) as e:
except PermissionError as e:
if file_type == "secrets":
# Match load_config()'s tolerance: a secrets file the web
# process can't read (e.g. written 0640 by the root-run
# display service before the group was fixed up) shouldn't
# 500 the settings page — degrade to "no secrets" instead.
self.logger.warning(f"Secrets file not readable ({path_to_load}): {e}. Returning empty secrets.")
return {}
error_msg = f"Error loading {file_type} configuration file {path_to_load}: {str(e)}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
except (IOError, OSError) as e:
error_msg = f"Error loading {file_type} configuration file {path_to_load}: {str(e)}"
self.logger.error(error_msg, exc_info=True)
raise ConfigError(error_msg, config_path=path_to_load) from e
@@ -497,6 +562,7 @@ class ConfigManager:
# Ensure final file has correct permissions
try:
ensure_file_permissions(path_obj, file_mode)
ensure_shared_group_ownership(path_obj)
except OSError as perm_error:
# If we can't set permissions but file was written, log warning but don't fail
self.logger.warning(
+8
View File
@@ -17,6 +17,7 @@ from enum import Enum
from src.exceptions import ConfigError
from src.logging_config import get_logger
from src.common.permission_utils import ensure_shared_group_ownership
class SaveResultStatus(Enum):
@@ -410,6 +411,13 @@ class AtomicConfigManager:
# This is important because temp files may have different permissions
# and we need root service to be able to read config.json
os.chmod(destination, target_mode)
# Also fix group ownership when this save is running as root
# (the display service): 0o640 alone only helps the non-root web
# user read a root-written secrets file if its group already
# matches the web user's group, which isn't guaranteed. See
# permission_utils.ensure_shared_group_ownership for why.
ensure_shared_group_ownership(destination)
except Exception as e:
raise ConfigError(f"Error during atomic move: {e}") from e
+525 -92
View File
@@ -23,8 +23,11 @@ Entry point: :func:`main` — instantiates :class:`DisplayController` and calls
import time
import os
import json
import threading
import types
from contextlib import contextmanager
from pathlib import Path
from typing import Dict, Any, List, Optional
from typing import Dict, Any, List, Optional, Callable
from datetime import datetime
from concurrent.futures import ThreadPoolExecutor, as_completed # pylint: disable=no-name-in-module
import pytz
@@ -163,6 +166,13 @@ class DisplayController:
self.plugin_modes = {} # mode -> plugin_instance mapping for plugin-first dispatch
self.mode_to_plugin_id: Dict[str, str] = {}
self.plugin_display_modes: Dict[str, List[str]] = {}
# Per-plugin config-change callbacks, kept so we can unsubscribe a
# plugin when it is disabled live.
self._plugin_config_callbacks: Dict[str, Callable] = {}
# Set by the config-watcher thread when the enabled-plugin set changes;
# the main run loop reconciles (loads/unloads) on its own thread so
# mutating available_modes never races with rendering.
self._pending_plugin_reconcile = False
self.on_demand_active = False
self.on_demand_mode: Optional[str] = None
self.on_demand_modes: List[str] = [] # All modes for the on-demand plugin
@@ -192,6 +202,10 @@ class DisplayController:
self.wifi_status_file = WIFI_STATUS_FILE
self.wifi_status_active = False
self.wifi_status_expires_at: Optional[float] = None
# _check_wifi_status_message throttle state (checked at frame rate,
# stat'd at most once per second)
self._wifi_status_check_ts = 0.0
self._wifi_status_last_result: Optional[Dict[str, Any]] = None
# Plugin display() signature cache — must be initialised before the plugin
# loading loop below so the .pop() invalidation at load time is always safe.
@@ -223,7 +237,24 @@ class DisplayController:
cache_manager=self.cache_manager,
font_manager=self.font_manager
)
# Activate the plugin health/metrics subsystem. PluginManager leaves
# health_tracker/resource_monitor as None by default; wiring real
# instances here turns on the circuit breaker (a repeatedly-failing
# plugin's update() is skipped after consecutive failures, then
# retried after a cooldown) and per-plugin execution-time metrics.
# Both persist to the shared cache so the web UI can surface them.
# Done before discovery/loading so load-time schema warnings have a
# tracker to record against.
try:
from src.plugin_system.plugin_health import PluginHealthTracker
from src.plugin_system.resource_monitor import PluginResourceMonitor
self.plugin_manager.health_tracker = PluginHealthTracker(self.cache_manager)
self.plugin_manager.resource_monitor = PluginResourceMonitor(self.cache_manager)
logger.info("Plugin health tracking and resource monitoring enabled")
except Exception as e:
logger.warning("Could not enable plugin health/resource monitoring: %s", e)
# Validate plugins after plugin manager is created
try:
from src.startup_validator import StartupValidator
@@ -331,47 +362,10 @@ class DisplayController:
logger.info("✓ Loaded plugin %s in %.3f seconds (%d/%d)",
plugin_id, result['load_time'], loaded_count, enabled_count)
# Get plugin instance and manifest
plugin_instance = self.plugin_manager.get_plugin(plugin_id)
manifest = self.plugin_manager.plugin_manifests.get(plugin_id, {})
# Prefer plugin's modes attribute if available (dynamic based on enabled leagues)
# Fall back to manifest display_modes if plugin doesn't provide modes
if plugin_instance and hasattr(plugin_instance, 'modes') and plugin_instance.modes:
display_modes = list(plugin_instance.modes)
logger.debug("Using plugin.modes for %s: %s", plugin_id, display_modes)
else:
display_modes = manifest.get('display_modes', [plugin_id])
logger.debug("Using manifest display_modes for %s: %s", plugin_id, display_modes)
if isinstance(display_modes, list) and display_modes:
self.plugin_display_modes[plugin_id] = list(display_modes)
else:
display_modes = [plugin_id]
self.plugin_display_modes[plugin_id] = list(display_modes)
# Subscribe plugin to config changes for hot-reload
if hasattr(self, 'config_service') and hasattr(plugin_instance, 'on_config_change'):
def config_change_callback(old_config: Dict[str, Any], new_config: Dict[str, Any]) -> None:
"""Callback for plugin config changes."""
try:
plugin_instance.on_config_change(new_config)
logger.debug("Plugin %s notified of config change", plugin_id)
except Exception as e:
logger.error("Error in plugin %s config change handler: %s", plugin_id, e, exc_info=True)
self.config_service.subscribe(config_change_callback, plugin_id=plugin_id)
logger.debug("Subscribed plugin %s to config changes", plugin_id)
# Add plugin modes to available modes
for mode in display_modes:
self.available_modes.append(mode)
self.plugin_modes[mode] = plugin_instance
self.mode_to_plugin_id[mode] = plugin_id
logger.debug(" Added mode: %s", mode)
# Invalidate signature cache so the new instance is re-inspected
self._plugin_accepts_display_mode.pop(plugin_id, None)
# Register the loaded plugin's modes, config subscription
# and dispatch maps (shared with live enable hot-reload).
self._register_loaded_plugin(plugin_id)
# Show progress
progress_pct = int((loaded_count / enabled_count) * 100)
elapsed = time.time() - plugin_time
@@ -387,6 +381,10 @@ class DisplayController:
logger.debug("%d plugin(s) disabled in config", disabled_count)
logger.info("Plugin system initialized in %.3f seconds", time.time() - plugin_time)
# Parallel loading appends modes in load-completion order, which
# varies between restarts; apply the user's configured rotation
# order (no-op when not configured).
self._apply_plugin_rotation_order()
logger.info("Total available modes: %d", len(self.available_modes))
logger.info("Available modes: %s", self.available_modes)
@@ -447,6 +445,10 @@ class DisplayController:
# when the user saves settings via the web UI.
def _controller_config_change(old_config: Dict[str, Any], new_config: Dict[str, Any]) -> None:
self._refresh_config_cache(new_config)
# If a plugin was enabled/disabled, flag a reconcile for the main
# loop to apply (loading/unloading off the watcher thread is unsafe).
if self._enabled_set_changed(old_config, new_config):
self._pending_plugin_reconcile = True
self.config_service.subscribe(_controller_config_change)
@@ -511,7 +513,10 @@ class DisplayController:
# Run plugin updates inside the Vegas loop so the inter-iteration
# gap is <1 ms (nothing left for _tick_plugin_updates() to do).
self.vegas_coordinator.set_update_callback(self._tick_plugin_updates)
# Use the Vegas-aware variant so plugins that got fresh data are
# hot-swapped into the scroll promptly instead of waiting for the
# next full cycle.
self.vegas_coordinator.set_update_callback(self._tick_plugin_updates_for_vegas)
# Wire multi-display sync into Vegas render pipeline
follower_pos = self.config.get("sync", {}).get("follower_position", "left")
@@ -630,18 +635,28 @@ class DisplayController:
current_day = current_time.strftime('%A').lower() # e.g. 'monday'
current_time_only = current_time.time()
# Check if per-day schedule is configured
days_config = schedule_config.get('days')
# Determine which schedule to use
# Determine which schedule to use. Respect an explicit 'mode' field
# (like the dim schedule does) so a stray/legacy 'days' dict left over
# from config migration or a prior per-day setup can't silently
# override a user's Global schedule selection.
mode = schedule_config.get('mode')
mode_normalized = mode.replace('_', '-') if mode else None
use_per_day = False
if days_config:
# Check if days dict is not empty and contains current day
if days_config and current_day in days_config:
if mode_normalized == 'global':
use_per_day = False
elif mode_normalized == 'per-day':
use_per_day = bool(days_config and current_day in days_config)
elif days_config:
# No explicit mode recorded (legacy config) - fall back to
# inferring from presence of a 'days' dict for the current day.
if current_day in days_config:
use_per_day = True
elif days_config:
# Days dict exists but doesn't have current day - fall back to global
else:
logger.debug("Per-day schedule exists but %s not configured, using global schedule", current_day)
if use_per_day:
@@ -837,6 +852,42 @@ class DisplayController:
if hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_failure(plugin_id, exc)
def _tick_plugin_updates_for_vegas(self) -> None:
"""Run scheduled plugin updates and tell Vegas mode which plugins
actually got fresh data, so it can hot-swap them into the scroll
without waiting for a full cycle to complete.
Used as the Vegas coordinator's update callback instead of the plain
_tick_plugin_updates() so that a live score change is reflected in
the ticker within a few seconds rather than at the next cycle
boundary (which, depending on min/max_cycle_duration, can be
minutes away). Restores wiring that PR #299 added and PR #330's
sync-mode refactor inadvertently dropped: coordinator.mark_plugin_updated()
has been unreachable dead code since.
Delegates the before/after plugin_last_update snapshot to
PluginManager.run_scheduled_updates_with_changes() so the snapshot,
update pass, and diff are lock-protected against this callback's own
background update-tick thread racing the main render loop.
"""
if not self.plugin_manager or not hasattr(self.plugin_manager, "run_scheduled_updates_with_changes"):
self._tick_plugin_updates()
return
updated = self.plugin_manager.run_scheduled_updates_with_changes()
vc = getattr(self, "vegas_coordinator", None)
if vc is None:
return
if updated:
logger.info("Vegas update tick: %d plugin(s) updated: %s", len(updated), updated)
for plugin_id in updated:
try:
vc.mark_plugin_updated(plugin_id)
except Exception: # pylint: disable=broad-except
logger.exception("Error marking plugin %s updated for Vegas", plugin_id)
def _tick_plugin_updates(self):
"""Run scheduled plugin updates if the plugin manager supports them."""
if not self.plugin_manager:
@@ -848,6 +899,30 @@ class DisplayController:
except Exception: # pylint: disable=broad-except
logger.exception("Error running scheduled plugin updates")
@contextmanager
def _display_lock_or_skip(self, plugin_id):
"""Try-lock guard keeping a plugin's display() off its in-flight update().
Yields True when display may run (lock held, released on exit) or
when no lock support exists (older plugin manager). Yields False when
the plugin's update() is currently executing on the background
worker the caller should treat the frame as displayed (the panel
holds the last pushed frame) rather than as a plugin failure, so a
mid-update skip never advances the rotation.
"""
pm = self.plugin_manager
if not pm or not hasattr(pm, 'get_plugin_lock') or not plugin_id:
yield True
return
lock = pm.get_plugin_lock(plugin_id)
if not lock.acquire(blocking=False):
yield False
return
try:
yield True
finally:
lock.release()
_FOLLOWER_SEND_INTERVAL = 1.0 / 90 # raw bytes are cheap; 90fps > follower render rate
def _follower_rebuild_scroll_image(self) -> None:
@@ -1062,6 +1137,29 @@ class DisplayController:
remaining = self.on_demand_expires_at - time.time()
return max(0.0, remaining)
def _publish_current_mode_state(self) -> None:
"""Publish the currently active display mode/plugin to cache for the web UI."""
try:
state = {
'mode': self.current_display_mode,
'plugin_id': self.mode_to_plugin_id.get(self.current_display_mode),
'mode_index': self.current_mode_index,
'total_modes': len(self.available_modes),
'on_demand_active': self.on_demand_active,
'is_display_active': self.is_display_active,
'last_updated': time.time(),
}
self.cache_manager.set('display_current_state', state)
self._last_published_mode = self.current_display_mode
except (OSError, RuntimeError, ValueError, TypeError) as err:
logger.error("Failed to publish current display state: %s", err, exc_info=True)
def _publish_current_mode_state_if_changed(self) -> None:
"""Publish current mode state only when it actually changed, to avoid
writing to the shared cache on every render tick."""
if self.current_display_mode != getattr(self, '_last_published_mode', None):
self._publish_current_mode_state()
def _publish_on_demand_state(self) -> None:
"""Publish current on-demand state to cache for external consumers."""
try:
@@ -1571,17 +1669,38 @@ class DisplayController:
def run(self):
"""Run the display controller, switching between displays."""
if not self.available_modes:
logger.warning("No display modes are enabled. Exiting.")
self.display_manager.cleanup()
return
logger.warning(
"No display modes are enabled at startup; idling until a "
"plugin is enabled via the web UI."
)
try:
# Initialize with cached data for fast startup - let background updates refresh naturally
logger.info("Starting display with cached data (fast startup mode)")
self.current_display_mode = self.available_modes[self.current_mode_index] if self.available_modes else 'none'
logger.info(f"Initial mode set to: {self.current_display_mode} (index: {self.current_mode_index}, total modes: {len(self.available_modes)})")
self._publish_current_mode_state()
while True:
# Apply plugin enable/disable edits saved via the web UI. The
# config-watcher thread only sets the flag; loading/unloading and
# rebuilding available_modes happens here on the render thread so
# it can't race with rendering. Deferred while on-demand is active
# (the flag stays set) so we don't fight its temporary-enable.
if self._pending_plugin_reconcile and not self.on_demand_active:
# Only clear the flag on success -- a retryable failure
# (e.g. discovery) leaves it set so the request isn't lost.
if self._reconcile_enabled_plugins():
self._pending_plugin_reconcile = False
if not self.available_modes:
# Nothing to render yet. Re-check _pending_plugin_reconcile
# every ~1s (rather than a long sleep) so enabling a plugin
# via the web UI is picked up about as promptly as it would
# be once modes exist and the loop is iterating per-frame.
self._sleep_with_plugin_updates(1)
continue
# Handle on-demand commands before rendering
self._poll_on_demand_requests()
self._check_on_demand_expiration()
@@ -1621,10 +1740,12 @@ class DisplayController:
logger.debug(f"Error clearing display when inactive: {e}")
logger.info(f"Display not active (is_display_active={self.is_display_active}), sleeping...")
self._publish_current_mode_state()
self._sleep_with_plugin_updates(60)
continue
logger.info(f"Display active, processing mode: {self.current_display_mode}")
self._publish_current_mode_state_if_changed()
logger.debug("Display active, processing mode: %s", self.current_display_mode)
# Plugins update on their own schedules - no forced sync updates needed
# Each plugin has its own update_interval and background services
@@ -1792,7 +1913,7 @@ class DisplayController:
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
should_skip = self.plugin_manager.health_tracker.should_skip_plugin(plugin_id)
if should_skip:
logger.info(f"Skipping plugin {plugin_id} due to circuit breaker (mode: {active_mode})")
logger.info("Skipping plugin %s due to circuit breaker (mode: %s)", plugin_id, active_mode)
display_result = False
# Skip to next mode - let existing logic handle it
manager_to_display = None
@@ -1815,6 +1936,7 @@ class DisplayController:
plugin_id = getattr(manager_to_display, 'plugin_id', active_mode)
try:
logger.debug(f"Calling display() for {active_mode} with force_clear={self.force_change}")
can_display = False
if hasattr(manager_to_display, 'display'):
# Opt #1: look up (or compute once) whether display() accepts display_mode
_cache_key = plugin_id
@@ -1825,14 +1947,64 @@ class DisplayController:
)
_accepts_display_mode = self._plugin_accepts_display_mode[_cache_key]
# Use PluginExecutor for safe execution with timeout
if self.plugin_manager and hasattr(self.plugin_manager, 'plugin_executor'):
result = self.plugin_manager.plugin_executor.execute_display(
manager_to_display,
plugin_id,
force_clear=self.force_change,
display_mode=active_mode if _accepts_display_mode else None
)
pm = self.plugin_manager
display_lock = None
can_display = True
if pm and hasattr(pm, 'get_plugin_lock'):
display_lock = pm.get_plugin_lock(plugin_id)
can_display = display_lock.acquire(blocking=False)
if not can_display:
# update() in flight on the worker — hold
# the last frame; not a plugin failure
result = True
elif pm and hasattr(pm, 'plugin_executor'):
# PluginExecutor's own thread.join(timeout) can
# return before the real display() call
# finishes (a lingering daemon thread keeps
# running it) -- so the lock is released from
# inside the wrapped call itself, whichever
# thread actually finishes it, rather than
# here when this dispatch merely returns.
release_guard = threading.Lock()
released = {'done': False}
def _release_display_lock():
with release_guard:
if released['done']:
return
released['done'] = True
if display_lock is not None:
display_lock.release()
if _accepts_display_mode:
def _display_target(display_mode=None, force_clear=False):
try:
return manager_to_display.display(
display_mode=display_mode, force_clear=force_clear)
finally:
_release_display_lock()
else:
def _display_target(force_clear=False):
try:
return manager_to_display.display(force_clear=force_clear)
finally:
_release_display_lock()
try:
result = self.plugin_manager.plugin_executor.execute_display(
types.SimpleNamespace(display=_display_target),
plugin_id,
force_clear=self.force_change,
display_mode=active_mode if _accepts_display_mode else None
)
except Exception: # pragma: no cover - defensive;
# execute_display catches everything
# internally, but guarantee the lock is
# never leaked if something unexpected
# slips through.
_release_display_lock()
raise
# execute_display returns bool, convert to expected format
if result:
result = True # Success
@@ -1840,23 +2012,31 @@ class DisplayController:
result = False # Failed
else:
# Fallback to direct call if executor not available
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=self.force_change)
else:
result = manager_to_display.display(force_clear=self.force_change)
try:
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=self.force_change)
else:
result = manager_to_display.display(force_clear=self.force_change)
finally:
if display_lock is not None:
display_lock.release()
logger.debug(f"display() returned: {result} (type: {type(result)})")
# Check if display() returned a boolean (new behavior)
if isinstance(result, bool):
display_result = result
if not display_result:
logger.info(f"Plugin {plugin_id} display() returned False for mode {active_mode}")
logger.info("Plugin %s display() returned False for mode %s", plugin_id, active_mode)
# Record success if display completed without exception
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_success(plugin_id)
self.force_change = False
# Record success only when display() actually ran this
# frame -- a skipped frame (lock busy) held the last
# frame, not a real success, and must not clear
# force_change or the pending mode-switch clear will
# be lost when display() finally does run.
if can_display:
if self.plugin_manager and hasattr(self.plugin_manager, 'health_tracker') and self.plugin_manager.health_tracker:
self.plugin_manager.health_tracker.record_success(plugin_id)
self.force_change = False
except Exception as exc: # pylint: disable=broad-except
logger.exception("Error displaying %s", self.current_display_mode)
# Record failure
@@ -2061,10 +2241,23 @@ class DisplayController:
# For plugins, call display multiple times to allow game rotation
if manager_to_display and hasattr(manager_to_display, 'display'):
# Check if plugin needs high FPS (like stock ticker)
# Always enable high-FPS for static-image plugin (for GIF animation support)
# High-FPS decision, in precedence order:
# 1. A plugin that declares needs_high_fps knows best
# (e.g. static-image sets it False for still PNGs,
# True for animated GIFs).
# 2. Back-compat: older static-image versions without
# the attribute keep the historical forced high-FPS
# (GIF support).
# 3. Otherwise scrolling plugins get high FPS.
plugin_id = getattr(manager_to_display, 'plugin_id', None)
if plugin_id == 'static-image':
declared = getattr(manager_to_display, 'needs_high_fps', None)
if declared is not None:
needs_high_fps = bool(declared)
logger.debug(
"[DisplayController] FPS check for %s (plugin=%s) - "
"plugin declares needs_high_fps=%s",
active_mode, plugin_id, needs_high_fps)
elif plugin_id == 'static-image':
needs_high_fps = True
logger.debug("FPS check - static-image plugin: forcing high-FPS mode for GIF support")
else:
@@ -2128,11 +2321,16 @@ class DisplayController:
while True:
try:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
with self._display_lock_or_skip(plugin_id) as can_display:
if can_display:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
else:
# update() in flight — hold the last frame
result = True
if isinstance(result, bool) and not result:
logger.debug("Display returned False, breaking early")
break
@@ -2192,11 +2390,16 @@ class DisplayController:
break
try:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
with self._display_lock_or_skip(plugin_id) as can_display:
if can_display:
# Pass display_mode to maintain sticky manager state
if _accepts_display_mode:
result = manager_to_display.display(display_mode=active_mode, force_clear=False)
else:
result = manager_to_display.display(force_clear=False)
else:
# update() in flight — hold the last frame
result = True
if isinstance(result, bool) and not result:
# For dynamic duration plugins, don't exit on False - keep looping
# until cycle is complete or max duration is reached
@@ -2319,7 +2522,7 @@ class DisplayController:
except Exception as e:
logger.warning("Error checking live priority for %s: %s", active_mode, e)
if should_rotate:
if should_rotate and self.available_modes:
self.current_mode_index = (self.current_mode_index + 1) % len(self.available_modes)
self.current_display_mode = self.available_modes[self.current_mode_index]
self.last_mode_change = time.time()
@@ -2343,6 +2546,16 @@ class DisplayController:
Returns None on any error or if message is expired/invalid.
"""
try:
# Throttle the existence stat to ~1 Hz: this runs on every render
# iteration (60+ fps), and the file usually doesn't exist — the
# status message's lifetime is measured in seconds anyway.
# Both attributes are initialised in __init__.
now = time.time()
if (now - self._wifi_status_check_ts) < 1.0:
return self._wifi_status_last_result
self._wifi_status_check_ts = now
self._wifi_status_last_result = None
# Check if file exists
if not self.wifi_status_file or not self.wifi_status_file.exists():
return None
@@ -2393,13 +2606,14 @@ class DisplayController:
pass
return None
# Message is valid and not expired
return {
# Message is valid and not expired — cache for the throttle window
self._wifi_status_last_result = {
'message': message,
'timestamp': timestamp,
'duration': duration,
'expires_at': expires_at
}
return self._wifi_status_last_result
except Exception as e:
# Catch-all for any unexpected errors - log but don't break the display
@@ -2507,6 +2721,217 @@ class DisplayController:
self.wifi_status_active = False
self.wifi_status_expires_at = None
def _register_loaded_plugin(self, plugin_id: str) -> List[str]:
"""Register an already-loaded plugin's display modes, config-change
subscription and dispatch maps with the controller.
Shared by startup loading and live enable hot-reload so both paths
build identical controller state. Returns the registered modes.
"""
plugin_instance = self.plugin_manager.get_plugin(plugin_id)
manifest = self.plugin_manager.plugin_manifests.get(plugin_id, {})
# Prefer the plugin's dynamic modes attribute (e.g. based on enabled
# leagues), else fall back to manifest display_modes, else the id.
if plugin_instance is not None and getattr(plugin_instance, 'modes', None):
display_modes = list(plugin_instance.modes)
logger.debug("Using plugin.modes for %s: %s", plugin_id, display_modes)
else:
display_modes = manifest.get('display_modes', [plugin_id])
logger.debug("Using manifest display_modes for %s: %s", plugin_id, display_modes)
if not (isinstance(display_modes, list) and display_modes):
display_modes = [plugin_id]
self.plugin_display_modes[plugin_id] = list(display_modes)
# Subscribe to config changes for per-plugin hot-reload. Bind plugin_id
# and instance as defaults so each plugin's callback targets its own
# instance (avoids late-binding when registering many plugins), and
# remember the callback so we can unsubscribe on disable.
if hasattr(self, 'config_service') and hasattr(plugin_instance, 'on_config_change'):
def config_change_callback(old_config: Dict[str, Any], new_config: Dict[str, Any],
_pid: str = plugin_id, _plugin: Any = plugin_instance) -> None:
"""Callback for plugin config changes."""
try:
_plugin.on_config_change(new_config)
logger.debug("Plugin %s notified of config change", _pid)
except Exception as e:
logger.error("Error in plugin %s config change handler: %s", _pid, e, exc_info=True)
self.config_service.subscribe(config_change_callback, plugin_id=plugin_id)
self._plugin_config_callbacks[plugin_id] = config_change_callback
logger.debug("Subscribed plugin %s to config changes", plugin_id)
# Add modes to the dispatch maps.
for mode in display_modes:
if mode not in self.available_modes:
self.available_modes.append(mode)
self.plugin_modes[mode] = plugin_instance
self.mode_to_plugin_id[mode] = plugin_id
logger.debug(" Added mode: %s", mode)
# Invalidate signature cache so the new instance is re-inspected.
self._plugin_accepts_display_mode.pop(plugin_id, None)
return display_modes
def _unregister_plugin(self, plugin_id: str) -> None:
"""Remove a plugin's modes, config subscription and instance, then
unload it. Used by live disable hot-reload."""
modes = self.plugin_display_modes.pop(plugin_id, [])
for mode in modes:
if mode in self.available_modes:
self.available_modes.remove(mode)
self.plugin_modes.pop(mode, None)
self.mode_to_plugin_id.pop(mode, None)
# Unsubscribe the plugin's config-change callback. Pop only on a
# successful unsubscribe -- if it raises, keep our reference so a
# later retry (or at least cleanup) still has the real callback
# instead of a lost one.
callback = self._plugin_config_callbacks.get(plugin_id)
if callback is not None and hasattr(self, 'config_service'):
try:
self.config_service.unsubscribe(callback, plugin_id=plugin_id)
except Exception as e:
logger.debug("Error unsubscribing plugin %s from config changes: %s", plugin_id, e)
else:
self._plugin_config_callbacks.pop(plugin_id, None)
else:
self._plugin_config_callbacks.pop(plugin_id, None)
self._plugin_accepts_display_mode.pop(plugin_id, None)
# Tear down the instance (cleanup + on_disable + module unload).
try:
self.plugin_manager.unload_plugin(plugin_id)
except Exception as e:
logger.error("Error unloading plugin %s: %s", plugin_id, e, exc_info=True)
logger.info("Disabled plugin %s live (removed modes: %s)", plugin_id, modes)
def _enabled_set_changed(self, old_config: Dict[str, Any], new_config: Dict[str, Any]) -> bool:
"""True if any top-level section's ``enabled`` flag differs between two
configs. A cheap watcher-thread check that gates the full reconcile.
Non-plugin sections (e.g. schedule) may match too; the reconcile
no-ops for anything that isn't a discovered plugin."""
def enabled_map(cfg: Dict[str, Any]) -> Dict[str, bool]:
return {
key: bool(value.get('enabled', False))
for key, value in cfg.items()
if isinstance(value, dict)
}
return enabled_map(old_config) != enabled_map(new_config)
def _reconcile_enabled_plugins(self) -> bool:
"""Load/unload plugins so the running set matches the enabled set in
config. Runs on the main display thread (never the config-watcher
thread) so mutating available_modes is race-free against rendering.
Returns True if reconciliation completed (including a no-op), or
False on a retryable failure -- the caller keeps the pending-reconcile
flag set in that case so the request isn't silently dropped."""
if self.plugin_manager is None:
return True
try:
config = self.config_service.get_config()
except Exception as e:
logger.warning("Plugin reconcile: falling back to cached config: %s", e)
config = self.config
try:
discovered = set(self.plugin_manager.discover_plugins())
except Exception as e:
logger.error("Plugin reconcile: discovery failed: %s", e, exc_info=True)
return False
for p in discovered:
if p in config and not isinstance(config.get(p), dict):
logger.warning(
"Plugin reconcile: config for %s is a %s, not a dict; treating as disabled",
p, type(config.get(p)).__name__
)
desired = {
p for p in discovered
if isinstance(config.get(p), dict) and config.get(p, {}).get('enabled', False)
}
current = set(self.plugin_display_modes.keys())
to_add = desired - current
to_remove = current - desired
if not to_add and not to_remove:
return True
previous_mode = self.current_display_mode
for plugin_id in to_remove:
self._unregister_plugin(plugin_id)
for plugin_id in to_add:
try:
if self.plugin_manager.load_plugin(plugin_id):
modes = self._register_loaded_plugin(plugin_id)
logger.info("Enabled plugin %s live (modes: %s)", plugin_id, modes)
else:
logger.warning("Plugin reconcile: failed to load %s", plugin_id)
except Exception as e:
logger.error("Plugin reconcile: error enabling %s: %s", plugin_id, e, exc_info=True)
# Newly enabled plugins were appended at the end; put them in the
# configured rotation slot before resyncing the index.
self._apply_plugin_rotation_order()
self._resync_mode_index_after_change(previous_mode)
logger.info("[DisplayController] Plugin reconcile complete: +%s -%s (%d modes)",
sorted(to_add), sorted(to_remove), len(self.available_modes))
return True
def _apply_plugin_rotation_order(self) -> None:
"""Reorder available_modes to follow display.plugin_rotation_order.
The configured value is a list of plugin ids; their modes rotate in
that order (each plugin's own modes keep their declared order), with
any enabled-but-unlisted plugins appended afterwards in their current
relative order. An empty/missing list leaves available_modes exactly
as built (today's behavior). Mirrors vegas_mode/config.py's
get_ordered_plugins() semantics for the primary rotation.
"""
configured = (self.config.get("display", {}) or {}).get("plugin_rotation_order", []) or []
# Defensive: hand-edited or migrated configs may hold a non-list or
# non-string entries; keep the existing rotation rather than applying
# a garbage order.
if not isinstance(configured, list):
logger.warning("[DisplayController] Ignoring invalid plugin_rotation_order (not a list): %r",
type(configured).__name__)
return
configured = [p for p in configured if isinstance(p, str)]
if not configured or not self.available_modes:
return
ordered_ids = [p for p in configured if p in self.plugin_display_modes]
new_modes: List[str] = []
for plugin_id in ordered_ids:
for mode in self.plugin_display_modes[plugin_id]:
if mode in self.available_modes and mode not in new_modes:
new_modes.append(mode)
# Unlisted plugins' modes (and any mode not attributable to a plugin)
# follow in their existing relative order.
for mode in self.available_modes:
if mode not in new_modes:
new_modes.append(mode)
if new_modes != self.available_modes:
self.available_modes = new_modes
logger.info("[DisplayController] Applied plugin rotation order %s -> modes: %s",
configured, self.available_modes)
def _resync_mode_index_after_change(self, previous_mode: Optional[str]) -> None:
"""Clamp rotation state after available_modes changed. Stays on the
previous mode if it survived, otherwise restarts cleanly within range."""
if not self.available_modes:
self.current_mode_index = 0
self.current_display_mode = None
return
if previous_mode in self.available_modes:
self.current_mode_index = self.available_modes.index(previous_mode)
else:
self.current_mode_index %= len(self.available_modes)
self.current_display_mode = self.available_modes[self.current_mode_index]
def _refresh_config_cache(self, new_config: Dict[str, Any]) -> None:
"""Refresh all config-derived caches when a hot-reload fires.
@@ -2533,6 +2958,14 @@ class DisplayController:
def cleanup(self):
"""Clean up resources."""
# Stop the async update worker first so no in-flight update() call
# is still touching display/cache-backed resources while they're
# torn down below.
if self.plugin_manager and hasattr(self.plugin_manager, 'stop_update_worker'):
try:
self.plugin_manager.stop_update_worker()
except Exception as e:
logger.warning("Error stopping plugin update worker: %s", e)
# Shutdown config service if it exists
if hasattr(self, 'config_service'):
try:
+184 -51
View File
@@ -31,13 +31,25 @@ if os.getenv("EMULATOR", "false") == "true":
else:
from rgbmatrix import RGBMatrix, RGBMatrixOptions
from contextlib import contextmanager
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
import threading
import time
from typing import Dict, Any, List, Optional
from collections import OrderedDict
from typing import Dict, Any, List, Optional, Tuple
import logging
import math
import zlib
import freetype
from src.common import snapshot_policy
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
get_assets_dir_mode,
get_assets_file_mode,
)
# Get logger without configuring
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO) # Set to INFO level
@@ -180,14 +192,45 @@ class DisplayManager:
# the logical image is blitted to the matrix unchanged.
self._double_sided = None # dict {copies, axis, logical_width, logical_height} or None
self._physical_image = None # full-chain buffer reused each frame when tiling
# Text-width measurement cache: (text, id(font)) -> pixel_width
# Text-width measurement cache: (text, id(font)) -> (width, font_ref)
# Avoids re-measuring the same string+font on every display() call.
# LRU-bounded: keys embed the TEXT, so changing strings (a clock, a
# live score) would otherwise grow it forever on a 24/7 service.
# Entries hold a strong reference to the font so its id() can't be
# recycled by a different font object — an id-keyed cache without
# the reference can return the WRONG width after garbage collection.
# Cleared on _load_fonts() so stale entries don't survive a font reload.
self._text_width_cache: Dict[tuple, int] = {}
# Snapshot settings for web preview integration (service writes, web reads)
self._text_width_cache: "OrderedDict[tuple, Tuple[int, Any]]" = OrderedDict()
self._TEXT_WIDTH_CACHE_MAX = 1024
# Snapshot mirror for web preview + health check (service writes, web
# reads). Cadence/skip decisions live in src/common/snapshot_policy.py:
# full rate only while the web SSE broadcaster keeps the viewer marker
# fresh; unchanged frames are never re-encoded, only mtime-touched.
self._snapshot_path = "/tmp/led_matrix_preview.png" # nosec B108 - fixed path intentional; web UI reads same path
self._snapshot_min_interval_sec = 0.2 # max ~5 fps
self._viewer_marker_path = "/tmp/led_matrix_preview_viewer" # nosec B108 - touched by web SSE broadcaster
self._last_snapshot_ts = 0.0
self._last_snapshot_touch_ts = 0.0
self._last_snapshot_digest: Optional[int] = None
self._snapshot_dir_prepared = False
self._viewer_check_ts = 0.0
self._viewer_fresh = False
self._viewer_was_fresh = False
# Snapshot failures are logged as warnings, rate-limited so a
# persistent failure (e.g. an unwritable file) can't spam the log —
# but is never silent: the snapshot's mtime doubles as the web UI's
# hardware-liveness signal, so a quiet failure makes health checks lie.
self._snapshot_fail_log_ts = 0.0
# Dirty tracking: (image digest, brightness) of the last frame pushed
# to the panel; update_display() skips identical pushes. Kill switch:
# display.dirty_tracking: false.
self._dirty_tracking_enabled = bool(
self.config.get('display', {}).get('dirty_tracking', True))
self._last_pushed_digest = None
# Serializes update_display(): plugins can call it directly from
# background threads (see docstring on update_display), not just the
# render loop. RLock in case a caller within the critical section
# ever re-enters (e.g. via a nested draw callback).
self._update_lock = threading.RLock()
# Scrolling state tracking for graceful updates
self._scrolling_state = {
@@ -418,6 +461,10 @@ class DisplayManager:
try:
# RGBMatrix accepts brightness as a property
self.matrix.brightness = brightness
# Brightness applies on the next swap — force a re-push even if
# the image itself is unchanged (belt-and-braces: brightness is
# also part of the dirty-tracking digest when readable).
self._last_pushed_digest = None
logger.info(f"[BRIGHTNESS] Display brightness set to {brightness}%")
return True
except AttributeError as e:
@@ -509,33 +556,70 @@ class DisplayManager:
return phys
def update_display(self):
"""Update the display using double buffering with proper sync."""
"""Update the display using double buffering with proper sync.
Skips the panel push entirely when the frame is byte-identical to
the last pushed one (same image digest AND same brightness) static
content re-rendered every second, and 125 fps loops between actual
scroll steps, otherwise re-walk the full framebuffer for nothing.
The panel keeps refreshing the current frame from its own thread,
so skipping a swap never blanks or freezes the hardware.
Correctness hinges on invalidation: clear() resets the digest (it
writes to the matrix directly), and brightness is PART of the digest
so a dim-schedule change is never skipped. Disable via config
``display.dirty_tracking: false`` if a redraw issue is ever suspected.
Serialized via ``_update_lock``: plugins can call this directly from
background threads (e.g. sports base classes push an immediate
"live" refresh from inside update()), so without a lock two callers
could both pass the digest check before either writes it back,
double-pushing a frame, or interleave the offscreen/current canvas
swap below. The lock is scoped to this method, so callers never
need to know about it.
"""
try:
if self.matrix is None:
# Fallback mode - no actual hardware to update
logger.debug("Update display called in fallback mode (no hardware)")
# Still write a snapshot so the web UI can preview
with self._update_lock:
if self.matrix is None:
# Fallback mode - no actual hardware to update
logger.debug("Update display called in fallback mode (no hardware)")
# Still write a snapshot so the web UI can preview
self._write_snapshot_if_due()
return
if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
digest = None
if self._dirty_tracking_enabled:
try:
brightness = getattr(self.matrix, 'brightness', None)
except AttributeError:
brightness = None
digest = (zlib.adler32(self.image.tobytes()), brightness)
if digest == self._last_pushed_digest:
# Nothing changed since the last push — the panel is
# already showing exactly this frame.
self._write_snapshot_if_due()
return
# Copy the current image to the offscreen canvas. In double-sided
# mode the logical screen is first tiled across the full chain.
if self._double_sided is not None:
self.offscreen_canvas.SetImage(self._composite_double_sided())
else:
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
# Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
self._last_pushed_digest = digest
# Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due()
return
if self._capture_mode_active:
return # Skip hardware write — content is being captured off-screen
# Copy the current image to the offscreen canvas. In double-sided
# mode the logical screen is first tiled across the full chain.
if self._double_sided is not None:
self.offscreen_canvas.SetImage(self._composite_double_sided())
else:
self.offscreen_canvas.SetImage(self.image)
# Swap buffers immediately
self.matrix.SwapOnVSync(self.offscreen_canvas)
# Swap our canvas references
self.offscreen_canvas, self.current_canvas = self.current_canvas, self.offscreen_canvas
# Write a snapshot for the web preview (throttled)
self._write_snapshot_if_due()
except Exception as e:
logger.error(f"Error updating display: {e}")
@@ -569,6 +653,9 @@ class DisplayManager:
# Clear both canvases and the underlying matrix to ensure no artifacts.
# Failures are non-fatal — the image buffer is already black above, so
# the next update_display() call will push clean content regardless.
# The matrix content no longer matches the last pushed digest,
# so dirty tracking must not skip the next push.
self._last_pushed_digest = None
try:
self.offscreen_canvas.Clear()
except (RuntimeError, OSError) as e:
@@ -699,12 +786,15 @@ class DisplayManager:
Results are cached by (text, font identity) so plugins that measure
the same string every frame (e.g. to centre a score) pay only one
measurement per unique (text, font) pair.
measurement per unique (text, font) pair. The entry keeps the font
alive so its id() can't be recycled, and the cache is LRU-bounded so
ever-changing text (clocks, tickers) can't grow it without limit.
"""
cache_key = (text, id(font))
cached = self._text_width_cache.get(cache_key)
if cached is not None:
return cached
self._text_width_cache.move_to_end(cache_key)
return cached[0]
try:
if isinstance(font, freetype.Face):
@@ -719,7 +809,9 @@ class DisplayManager:
logger.error("Error getting text width: %s", e)
return 0
self._text_width_cache[cache_key] = width
self._text_width_cache[cache_key] = (width, font)
while len(self._text_width_cache) > self._TEXT_WIDTH_CACHE_MAX:
self._text_width_cache.popitem(last=False)
return width
def get_font_height(self, font):
@@ -1128,27 +1220,56 @@ class DisplayManager:
'deferred_update_ttl': self._scrolling_state['deferred_update_ttl']
}
def _viewer_is_fresh(self, now: float) -> bool:
"""True when a browser preview is watching (marker file touched by
the web SSE broadcaster). The marker is stat'd at most once per
second at 125 fps loops a per-call stat would be pure overhead."""
if (now - self._viewer_check_ts) >= 1.0:
self._viewer_check_ts = now
try:
marker_age = now - os.stat(self._viewer_marker_path).st_mtime
self._viewer_fresh = marker_age < snapshot_policy.VIEWER_MARKER_FRESH_SEC
except OSError:
self._viewer_fresh = False
return self._viewer_fresh
def _write_snapshot_if_due(self) -> None:
"""Write the current image to a PNG snapshot file at a limited frequency."""
"""Mirror the current frame to the preview snapshot when the policy
says it's worth it — see src/common/snapshot_policy.py. Unchanged
frames are never re-encoded; without viewers the cadence drops to
the idle keepalive."""
try:
now = time.time()
if (now - self._last_snapshot_ts) < self._snapshot_min_interval_sec:
viewer_fresh = self._viewer_is_fresh(now)
if viewer_fresh and not self._viewer_was_fresh:
# A preview just opened: let the next changed frame through
# immediately instead of waiting out the idle interval.
self._last_snapshot_ts = 0.0
self._viewer_was_fresh = viewer_fresh
digest = zlib.adler32(self.image.tobytes())
action = snapshot_policy.decide(
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
viewer_fresh, digest != self._last_snapshot_digest)
if action is snapshot_policy.SnapshotAction.SKIP:
return
# Ensure directory exists with proper permissions
from pathlib import Path
from src.common.permission_utils import (
ensure_directory_permissions,
ensure_file_permissions,
get_assets_dir_mode,
get_assets_file_mode
)
if action is snapshot_policy.SnapshotAction.TOUCH:
# mtime bump only: keeps the health check (snapshot age)
# green without paying for a PNG encode of an unchanged frame
os.utime(self._snapshot_path, None)
self._last_snapshot_touch_ts = now
return
# WRITE: ensure directory permissions once, not per frame
snapshot_path_obj = Path(self._snapshot_path)
# Only ensure permissions on non-system directories
# Never modify /tmp permissions - it has special system permissions (1777)
# that must not be changed or it breaks apt and other system tools
parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
if not self._snapshot_dir_prepared:
# Never modify /tmp permissions - it has special system
# permissions (1777) that must not be changed or it breaks
# apt and other system tools
parent_dir = snapshot_path_obj.parent
if parent_dir and str(parent_dir) != '/tmp': # nosec B108 - guard to skip /tmp for permission ops
ensure_directory_permissions(parent_dir, get_assets_dir_mode())
self._snapshot_dir_prepared = True
# Write atomically: temp then replace
tmp_path = f"{self._snapshot_path}.tmp"
self.image.save(tmp_path, format='PNG')
@@ -1163,6 +1284,18 @@ class DisplayManager:
except Exception:
pass
self._last_snapshot_ts = now
self._last_snapshot_touch_ts = now
self._last_snapshot_digest = digest
except Exception as e:
# Snapshot failures should never break display; log at debug to avoid noise
logger.debug(f"Snapshot write skipped: {e}")
# Snapshot failures must never break display — but they must not
# be silent either: the snapshot's mtime is the web UI's display
# mirror AND its hardware-liveness proxy, so a quietly failing
# write freezes the mirror and makes health checks lie (seen in
# the field: a stale root-owned /tmp file froze it for a day).
# Warn at most once per 5 minutes to avoid log spam.
if (now - self._snapshot_fail_log_ts) > 300:
self._snapshot_fail_log_ts = now
logger.warning("Snapshot write failing (web preview/health "
"mirror is stale): %s", e)
else:
logger.debug(f"Snapshot write skipped: {e}")
+66 -5
View File
@@ -35,6 +35,7 @@ import urllib.request
import zipfile
import tempfile
import time
from collections import OrderedDict
from pathlib import Path
from PIL import ImageFont
from typing import Dict, Tuple, Optional, Union, Any, List
@@ -58,7 +59,13 @@ class FontManager:
# Font discovery and catalog
self.font_catalog: Dict[str, str] = {} # family_name -> file_path
self.font_cache: Dict[str, Union[ImageFont.FreeTypeFont, freetype.Face]] = {} # (family, size) -> font
self.metrics_cache: Dict[str, Tuple[int, int, int]] = {} # (text, font_id) -> (width, height, baseline)
# (text, id(font)) -> ((width, height, baseline), font_ref).
# LRU-bounded — keys embed the measured TEXT, so changing strings
# (clocks, live scores) would otherwise grow it forever. Entries
# keep the font alive so its id() can't be recycled by a different
# font object (which would silently return wrong metrics).
self.metrics_cache: "OrderedDict[Any, Tuple[Tuple[int, int, int], Any]]" = OrderedDict()
self._METRICS_CACHE_MAX = 1024
# Plugin font management
self.plugin_fonts: Dict[str, Dict[str, Any]] = {} # plugin_id -> font_manifest
@@ -103,6 +110,10 @@ class FontManager:
# Font overrides storage (for manual overrides)
self.font_overrides_file = "config/font_overrides.json"
self.font_overrides: Dict[str, Dict[str, Any]] = {}
# Bumped whenever cached font objects are invalidated, so holders of
# derived caches (e.g. adaptive-layout fit results) know to rebuild.
self.cache_generation = 0
self._initialize_fonts()
@@ -112,6 +123,7 @@ class FontManager:
self.fonts_config = new_config.get("fonts", {})
self.font_cache.clear() # Clear cache to force reload
self.metrics_cache.clear() # Clear metrics cache
self.cache_generation += 1
self._initialize_fonts()
logger.info("FontManager configuration reloaded successfully")
@@ -482,6 +494,14 @@ class FontManager:
def _load_bdf_font(self, font_path: str, size_px: int) -> freetype.Face:
"""Load a BDF font using FreeType."""
try:
native_size = self._read_bdf_native_size(font_path)
if native_size is not None and native_size != size_px:
# BDF is a fixed-strike bitmap format: FreeType renders the
# native size no matter what set_char_size asks for.
logger.debug(
"BDF font %s requested at %spx but renders at its native "
"%spx", font_path, size_px, native_size
)
face = freetype.Face(font_path)
# Set character size (width, height) in 1/64th of points
face.set_char_size(size_px * 64, size_px * 64, 72, 72)
@@ -490,6 +510,41 @@ class FontManager:
logger.error(f"Error loading BDF font {font_path}: {e}")
raise
def get_native_bdf_size(self, family: str) -> Optional[int]:
"""The one true pixel size of a BDF family in the catalog, or None
for scalable (TTF) families / unknown families."""
font_path = self.font_catalog.get(family)
if not font_path or not font_path.endswith('.bdf'):
return None
return self._read_bdf_native_size(font_path)
@staticmethod
def _read_bdf_native_size(bdf_path: str) -> Optional[int]:
"""Read a BDF file's own header to find its one true pixel size.
Prefers the PIXEL_SIZE property, which states the real pixel height
directly; falls back to the SIZE line's point-size only if PIXEL_SIZE
is absent, since point-size only equals pixel height at exactly
100dpi several bundled fonts (e.g. 6x13.bdf, 5x8.bdf) are defined
at 75dpi, where the two values genuinely differ."""
size_line_value = None
try:
with open(bdf_path, "r", encoding="ascii", errors="ignore") as f:
for line in f:
if line.startswith("PIXEL_SIZE"):
parts = line.split()
if len(parts) >= 2:
return int(float(parts[1]))
elif line.startswith("SIZE") and size_line_value is None:
# Format: "SIZE <point_size> <xres> <yres>"
parts = line.split()
if len(parts) >= 2:
size_line_value = int(float(parts[1]))
elif line.startswith("STARTCHAR"):
break
except (OSError, ValueError):
return None
return size_line_value
def _get_fallback_font(self) -> ImageFont.ImageFont:
"""Get a fallback font when loading fails."""
return ImageFont.load_default()
@@ -507,10 +562,14 @@ class FontManager:
Returns:
Tuple of (width, height, baseline_offset)
"""
cache_key = f"{hash(text)}_{id(font)}"
# Key on the text itself (hash(text) could collide) + font identity;
# the entry below keeps the font referenced so the id stays valid.
cache_key = (text, id(font))
if cache_key in self.metrics_cache:
return self.metrics_cache[cache_key]
cached = self.metrics_cache.get(cache_key)
if cached is not None:
self.metrics_cache.move_to_end(cache_key)
return cached[0]
try:
if isinstance(font, freetype.Face):
@@ -547,7 +606,9 @@ class FontManager:
baseline = 10
result = (width, height, baseline)
self.metrics_cache[cache_key] = result
self.metrics_cache[cache_key] = (result, font)
while len(self.metrics_cache) > self._METRICS_CACHE_MAX:
self.metrics_cache.popitem(last=False)
return result
def get_font_height(self, font: Union[ImageFont.FreeTypeFont, freetype.Face]) -> int:
-135
View File
@@ -1,135 +0,0 @@
import os
import freetype
from PIL import ImageDraw, ImageFont
import logging
from typing import Dict, Any
from src.display_manager import DisplayManager
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class FontTestManager:
"""Manager for testing fonts with easy BDF/TTF switching."""
def __init__(self, config: Dict[str, Any], display_manager: DisplayManager):
self.display_manager = display_manager
self.config = config
self.logger = logging.getLogger('FontTest')
# FONT CONFIGURATION - EASY SWITCHING
# Set to 'bdf' or 'ttf' to switch font types
self.font_type = 'bdf' # Change this to 'ttf' to use TTF font
# Font configurations
self.font_configs = {
'bdf': {
'path': "assets/fonts/cozette.bdf",
'display_name': "Cozette BTF",
'description': "BTF font Test"
},
'ttf': {
'path': "assets/fonts/5by7.regular.ttf",
'display_name': "5by7 TTF",
'description': "TTF font test"
}
}
# Get current font configuration
self.current_config = self.font_configs[self.font_type]
self.font_path = self.current_config['path']
# Verify font exists
if not os.path.exists(self.font_path):
self.logger.error(f"Font file not found: {self.font_path}")
raise FileNotFoundError(f"Font file not found: {self.font_path}")
# Load the font based on type
if self.font_type == 'bdf':
self._load_bdf_font()
else:
self._load_ttf_font()
self.logger.info(f"Initialized FontTestManager with {self.current_config['description']}")
def _load_bdf_font(self):
"""Load BDF font using freetype."""
try:
self.face = freetype.Face(self.font_path)
self.logger.info(f"Successfully loaded BDF font from {self.font_path}")
except Exception as e:
self.logger.error(f"Failed to load BDF font: {e}")
raise
def _load_ttf_font(self):
"""Load TTF font using PIL."""
try:
self.font = ImageFont.truetype(self.font_path, 8) # Size 8 for 5x7 font
self.logger.info(f"Successfully loaded TTF font from {self.font_path}")
except Exception as e:
self.logger.error(f"Failed to load TTF font: {e}")
raise
def update(self):
"""No update needed for static display."""
def display(self, force_clear: bool = False):
"""Display the font with sample text."""
try:
# Clear the display
self.display_manager.clear()
# Draw font name at the top
self.display_manager.draw_text(self.current_config['display_name'], y=2, color=(255, 255, 255))
# Draw sample text
draw = ImageDraw.Draw(self.display_manager.image)
sample_text = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
# Calculate starting position
x = 10 # Start 10 pixels from the left
y = 10 # Start 10 pixels from the top
# Draw text based on font type
if self.font_type == 'bdf':
self._draw_bdf_text(draw, sample_text, x, y)
else:
self._draw_ttf_text(draw, sample_text, x, y)
# Update the display once
self.display_manager.update_display()
# Log that display is complete
self.logger.info("Font test display complete.")
except Exception as e:
self.logger.error(f"Error displaying font test: {e}", exc_info=True)
def _draw_bdf_text(self, draw, text, x, y):
"""Draw text using BDF font."""
for char in text:
# Load the glyph
self.face.load_char(char)
bitmap = self.face.glyph.bitmap
# Draw the glyph
for i in range(bitmap.rows):
for j in range(bitmap.width):
try:
# Get the byte containing the pixel
byte_index = i * bitmap.pitch + (j // 8)
if byte_index < len(bitmap.buffer):
byte = bitmap.buffer[byte_index]
# Check if the specific bit is set
if byte & (1 << (7 - (j % 8))):
draw.point((x + j, y + i), fill=(255, 255, 255))
except IndexError:
self.logger.warning(f"Index out of range for char '{char}' at position ({i}, {j})")
continue
# Move to next character position
x += self.face.glyph.advance.x >> 6
def _draw_ttf_text(self, draw, text, x, y):
"""Draw text using TTF font."""
draw.text((x, y), text, font=self.font, fill=(255, 255, 255))
-150
View File
@@ -1,150 +0,0 @@
"""
Generic Cache Mixin for Any Manager
This mixin provides caching functionality that can be used by any manager
that needs to cache data, not just sports managers. It's a more general
version of BackgroundCacheMixin that works for weather, stocks, news, etc.
"""
import time
from typing import Dict, Optional, Any, Callable
class GenericCacheMixin:
"""
Generic mixin class that provides caching functionality to any manager.
This mixin can be used by weather, stock, news, or any other manager
that needs to cache data with performance monitoring.
Note: For sports managers that need background service cache integration,
use BackgroundCacheMixin instead. See src/background_cache_mixin.py for details.
"""
def _fetch_data_with_cache(self,
cache_key: str,
api_fetch_method: Callable,
cache_ttl: int = 300,
force_refresh: bool = False) -> Optional[Dict]:
"""
Generic caching pattern for any manager.
Args:
cache_key: Unique cache key for this data
api_fetch_method: Method to call for fresh data
cache_ttl: Time-to-live in seconds (default: 5 minutes)
force_refresh: Skip cache and fetch fresh data
Returns:
Cached or fresh data from API
"""
start_time = time.time()
cache_hit = False
cache_source = None
try:
# Check cache first (unless forcing refresh)
if not force_refresh:
cached_data = self.cache_manager.get_cached_data(cache_key, cache_ttl)
if cached_data:
self.logger.info(f"Using cached data for {cache_key}")
cache_hit = True
cache_source = "cache"
self.cache_manager.record_cache_hit('regular')
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
self._log_fetch_performance(cache_key, duration, cache_hit, cache_source)
return cached_data
# Fetch fresh data
self.logger.info(f"Fetching fresh data for {cache_key}")
result = api_fetch_method()
cache_source = "api_fresh"
# Store in cache if we got data
if result:
self.cache_manager.save_cache(cache_key, result)
self.cache_manager.record_cache_miss('regular')
else:
self.logger.warning(f"No data returned for {cache_key}")
# Record performance metrics
duration = time.time() - start_time
self.cache_manager.record_fetch_time(duration)
# Log performance
self._log_fetch_performance(cache_key, duration, cache_hit, cache_source)
return result
except Exception as e:
duration = time.time() - start_time
self.logger.error(f"Error fetching data for {cache_key} after {duration:.2f}s: {e}")
self.cache_manager.record_fetch_time(duration)
raise
def _log_fetch_performance(self, cache_key: str, duration: float, cache_hit: bool, cache_source: str):
"""
Log detailed performance metrics for fetch operations.
Args:
cache_key: Cache key that was accessed
duration: Fetch operation duration in seconds
cache_hit: Whether this was a cache hit
cache_source: Source of the data (cache, api_fresh, etc.)
"""
# Log basic performance info
self.logger.info(f"Fetch completed for {cache_key} in {duration:.2f}s "
f"(cache_hit={cache_hit}, source={cache_source})")
# Log detailed metrics every 10 operations
if hasattr(self, '_fetch_count'):
self._fetch_count += 1
else:
self._fetch_count = 1
if self._fetch_count % 10 == 0:
metrics = self.cache_manager.get_cache_metrics()
self.logger.info(f"Cache Performance Summary - "
f"Hit Rate: {metrics['cache_hit_rate']:.2%}, "
f"API Calls Saved: {metrics['api_calls_saved']}, "
f"Avg Fetch Time: {metrics['average_fetch_time']:.2f}s")
def get_cache_performance_summary(self) -> Dict[str, Any]:
"""
Get cache performance summary for this manager.
Returns:
Dictionary containing cache performance metrics
"""
return self.cache_manager.get_cache_metrics()
def log_cache_performance(self):
"""Log current cache performance metrics."""
self.cache_manager.log_cache_metrics()
def clear_cache_for_key(self, cache_key: str):
"""Clear cache for a specific key."""
self.cache_manager.clear_cache(cache_key)
self.logger.info(f"Cleared cache for {cache_key}")
def get_cache_info(self, cache_key: str) -> Dict[str, Any]:
"""
Get information about a cached item.
Args:
cache_key: Cache key to check
Returns:
Dictionary with cache information
"""
# This would need to be implemented in CacheManager
# For now, just return basic info
return {
'key': cache_key,
'exists': self.cache_manager.get_cached_data(cache_key, 0) is not None,
'ttl': 'unknown' # Would need to be implemented
}
-16
View File
@@ -1,16 +0,0 @@
import logging
from PIL import Image
logger = logging.getLogger(__name__)
def scale_to_max_dimensions(img, max_width, max_height):
h_to_w_ratio = img.height / img.width
w_to_h_ratio = img.width / img.height
if img.height > max_height:
img = img.resize((int(max_height * w_to_h_ratio), max_height), Image.Resampling.LANCZOS)
if img.width > max_width:
img = img.resize((max_width, int(max_width * h_to_w_ratio)), Image.Resampling.LANCZOS)
return img
-409
View File
@@ -1,409 +0,0 @@
"""
Layout Manager for LED Matrix Display
Handles custom layouts, element positioning, and display composition.
"""
import json
import os
import logging
from typing import Dict, List, Any
from datetime import datetime
logger = logging.getLogger(__name__)
class LayoutManager:
def __init__(self, display_manager=None, config_path="config/custom_layouts.json"):
self.display_manager = display_manager
self.config_path = config_path
self.layouts = self.load_layouts()
self.current_layout = None
def load_layouts(self) -> Dict[str, Any]:
"""Load saved layouts from file."""
try:
if os.path.exists(self.config_path):
with open(self.config_path, 'r') as f:
return json.load(f)
return {}
except Exception as e:
logger.error(f"Error loading layouts: {e}")
return {}
def save_layouts(self) -> bool:
"""Save layouts to file."""
try:
from pathlib import Path
from src.common.permission_utils import (
ensure_directory_permissions,
get_config_dir_mode
)
config_path_obj = Path(self.config_path)
ensure_directory_permissions(config_path_obj.parent, get_config_dir_mode())
with open(self.config_path, 'w') as f:
json.dump(self.layouts, f, indent=2)
return True
except Exception as e:
logger.error(f"Error saving layouts: {e}")
return False
def create_layout(self, name: str, elements: List[Dict], description: str = "") -> bool:
"""Create a new layout."""
try:
self.layouts[name] = {
'elements': elements,
'description': description,
'created': datetime.now().isoformat(),
'modified': datetime.now().isoformat()
}
return self.save_layouts()
except Exception as e:
logger.error(f"Error creating layout '{name}': {e}")
return False
def update_layout(self, name: str, elements: List[Dict], description: str = None) -> bool:
"""Update an existing layout."""
try:
if name not in self.layouts:
return False
self.layouts[name]['elements'] = elements
self.layouts[name]['modified'] = datetime.now().isoformat()
if description is not None:
self.layouts[name]['description'] = description
return self.save_layouts()
except Exception as e:
logger.error(f"Error updating layout '{name}': {e}")
return False
def delete_layout(self, name: str) -> bool:
"""Delete a layout."""
try:
if name in self.layouts:
del self.layouts[name]
return self.save_layouts()
return False
except Exception as e:
logger.error(f"Error deleting layout '{name}': {e}")
return False
def get_layout(self, name: str) -> Dict[str, Any]:
"""Get a specific layout."""
return self.layouts.get(name, {})
def list_layouts(self) -> List[str]:
"""Get list of all layout names."""
return list(self.layouts.keys())
def set_current_layout(self, name: str) -> bool:
"""Set the current active layout."""
if name in self.layouts:
self.current_layout = name
return True
return False
def render_layout(self, layout_name: str = None, data_context: Dict = None) -> bool:
"""Render a layout to the display."""
if not self.display_manager:
logger.error("No display manager available")
return False
layout_name = layout_name or self.current_layout
if not layout_name or layout_name not in self.layouts:
logger.error(f"Layout '{layout_name}' not found")
return False
try:
# Clear the display
self.display_manager.clear()
# Get layout elements
elements = self.layouts[layout_name]['elements']
# Render each element
for element in elements:
self.render_element(element, data_context or {})
# Update the display
self.display_manager.update_display()
return True
except Exception as e:
logger.error(f"Error rendering layout '{layout_name}': {e}")
return False
def render_element(self, element: Dict, data_context: Dict) -> None:
"""Render a single element."""
element_type = element.get('type')
x = element.get('x', 0)
y = element.get('y', 0)
properties = element.get('properties', {})
try:
if element_type == 'text':
self._render_text_element(x, y, properties, data_context)
elif element_type == 'weather_icon':
self._render_weather_icon_element(x, y, properties, data_context)
elif element_type == 'rectangle':
self._render_rectangle_element(x, y, properties)
elif element_type == 'line':
self._render_line_element(x, y, properties)
elif element_type == 'clock':
self._render_clock_element(x, y, properties)
elif element_type == 'data_text':
self._render_data_text_element(x, y, properties, data_context)
else:
logger.warning(f"Unknown element type: {element_type}")
except Exception as e:
logger.error(f"Error rendering element {element_type}: {e}")
def _render_text_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a text element."""
text = properties.get('text', 'Sample Text')
color = tuple(properties.get('color', [255, 255, 255]))
font_size = properties.get('font_size', 'normal')
# Support template variables in text
text = self._process_template_text(text, data_context)
# Select font
if font_size == 'small':
font = self.display_manager.small_font
elif font_size == 'large':
font = self.display_manager.regular_font
else:
font = self.display_manager.regular_font
self.display_manager.draw_text(text, x, y, color, font=font)
def _render_weather_icon_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a weather icon element."""
condition = properties.get('condition', 'sunny')
size = properties.get('size', 16)
# Use weather data from context if available
if 'weather' in data_context and 'condition' in data_context['weather']:
condition = data_context['weather']['condition'].lower()
self.display_manager.draw_weather_icon(condition, x, y, size)
def _render_rectangle_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a rectangle element."""
width = properties.get('width', 10)
height = properties.get('height', 10)
color = tuple(properties.get('color', [255, 255, 255]))
filled = properties.get('filled', False)
if filled:
self.display_manager.draw.rectangle(
[x, y, x + width, y + height],
fill=color
)
else:
self.display_manager.draw.rectangle(
[x, y, x + width, y + height],
outline=color
)
def _render_line_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a line element."""
x2 = properties.get('x2', x + 10)
y2 = properties.get('y2', y)
color = tuple(properties.get('color', [255, 255, 255]))
width = properties.get('width', 1)
self.display_manager.draw.line([x, y, x2, y2], fill=color, width=width)
def _render_clock_element(self, x: int, y: int, properties: Dict) -> None:
"""Render a clock element."""
format_str = properties.get('format', '%H:%M')
color = tuple(properties.get('color', [255, 255, 255]))
current_time = datetime.now().strftime(format_str)
self.display_manager.draw_text(current_time, x, y, color)
def _render_data_text_element(self, x: int, y: int, properties: Dict, data_context: Dict) -> None:
"""Render a data-driven text element."""
data_key = properties.get('data_key', '')
format_str = properties.get('format', '{value}')
color = tuple(properties.get('color', [255, 255, 255]))
default_value = properties.get('default', 'N/A')
# Extract data from context
value = self._get_nested_value(data_context, data_key, default_value)
# Format the text
try:
text = format_str.format(value=value)
except (ValueError, TypeError, KeyError, IndexError):
text = str(value)
self.display_manager.draw_text(text, x, y, color)
def _process_template_text(self, text: str, data_context: Dict) -> str:
"""Process template variables in text."""
try:
# Simple template processing - replace {key} with values from context
for key, value in data_context.items():
placeholder = f"{{{key}}}"
if placeholder in text:
text = text.replace(placeholder, str(value))
return text
except Exception as e:
logger.error(f"Error processing template text: {e}")
return text
def _get_nested_value(self, data: Dict, key: str, default=None):
"""Get a nested value from a dictionary using dot notation."""
try:
keys = key.split('.')
value = data
for k in keys:
value = value[k]
return value
except (KeyError, TypeError):
return default
def create_preset_layouts(self) -> None:
"""Create some preset layouts for common use cases."""
# Basic clock layout
clock_layout = [
{
'type': 'clock',
'x': 10,
'y': 10,
'properties': {
'format': '%H:%M',
'color': [255, 255, 255]
}
},
{
'type': 'clock',
'x': 10,
'y': 20,
'properties': {
'format': '%m/%d',
'color': [100, 100, 255]
}
}
]
self.create_layout('basic_clock', clock_layout, 'Simple clock with date')
# Weather layout
weather_layout = [
{
'type': 'weather_icon',
'x': 5,
'y': 5,
'properties': {
'condition': 'sunny',
'size': 20
}
},
{
'type': 'data_text',
'x': 30,
'y': 8,
'properties': {
'data_key': 'weather.temperature',
'format': '{value}°',
'color': [255, 200, 0],
'default': '--°'
}
},
{
'type': 'data_text',
'x': 30,
'y': 18,
'properties': {
'data_key': 'weather.condition',
'format': '{value}',
'color': [200, 200, 200],
'default': 'Unknown'
}
}
]
self.create_layout('weather_display', weather_layout, 'Weather icon with temperature and condition')
# Mixed dashboard layout
dashboard_layout = [
{
'type': 'clock',
'x': 2,
'y': 2,
'properties': {
'format': '%H:%M',
'color': [255, 255, 255]
}
},
{
'type': 'weather_icon',
'x': 50,
'y': 2,
'properties': {
'size': 16
}
},
{
'type': 'data_text',
'x': 70,
'y': 5,
'properties': {
'data_key': 'weather.temperature',
'format': '{value}°',
'color': [255, 200, 0],
'default': '--°'
}
},
{
'type': 'line',
'x': 0,
'y': 15,
'properties': {
'x2': 128,
'y2': 15,
'color': [100, 100, 100]
}
},
{
'type': 'data_text',
'x': 2,
'y': 18,
'properties': {
'data_key': 'stocks.AAPL.price',
'format': 'AAPL: ${value}',
'color': [0, 255, 0],
'default': 'AAPL: N/A'
}
}
]
self.create_layout('dashboard', dashboard_layout, 'Mixed dashboard with clock, weather, and stocks')
logger.info("Created preset layouts")
def get_layout_preview(self, layout_name: str) -> Dict[str, Any]:
"""Get a preview representation of a layout."""
if layout_name not in self.layouts:
return {}
layout = self.layouts[layout_name]
elements = layout['elements']
# Create a simple preview representation
preview = {
'name': layout_name,
'description': layout.get('description', ''),
'element_count': len(elements),
'elements': []
}
for element in elements:
preview['elements'].append({
'type': element.get('type'),
'position': f"({element.get('x', 0)}, {element.get('y', 0)})",
'properties': list(element.get('properties', {}).keys())
})
return preview
+26 -8
View File
@@ -139,23 +139,41 @@ def setup_logging(
sys.stderr.write(f"Warning: Could not set up file logging to {log_file}: {e}\n")
def get_logger(name: str, plugin_id: Optional[str] = None) -> logging.Logger:
class PluginLoggerAdapter(logging.LoggerAdapter):
"""LoggerAdapter that stamps every record with its plugin_id.
A plain `logging.Logger` attribute (the old approach) is never copied
onto individual `LogRecord`s, so `ContextualFormatter`/`StructuredFormatter`
only ever saw `plugin_id` on calls that explicitly passed
`extra={'plugin_id': ...}` (i.e. `log_with_context`). This adapter injects
it into `extra` on every call, so `self.logger.info(...)` in plugin code
is tagged automatically.
"""
def process(self, msg, kwargs):
extra = dict(kwargs.get('extra') or {})
extra.setdefault('plugin_id', self.extra.get('plugin_id'))
kwargs['extra'] = extra
return msg, kwargs
def get_logger(name: str, plugin_id: Optional[str] = None):
"""
Get a logger with consistent configuration.
Args:
name: Logger name (typically __name__)
plugin_id: Optional plugin ID for automatic context
Returns:
Configured logger instance
Configured logger instance (or a PluginLoggerAdapter when plugin_id
is given, which supports the same .debug/.info/.warning/.error API)
"""
logger = logging.getLogger(name)
# Add plugin_id as attribute for formatters
if plugin_id:
logger.plugin_id = plugin_id
return PluginLoggerAdapter(logger, {'plugin_id': plugin_id})
return logger
+143 -1
View File
@@ -15,6 +15,19 @@ import logging
from src.logging_config import get_logger
_shared_fallback_font_manager: Optional[Any] = None
def _fallback_font_manager() -> Any:
"""Shared FontManager for environments (unit tests, mocks) where the
plugin manager doesn't carry one. Scans assets/fonts like the real one."""
global _shared_fallback_font_manager
if _shared_fallback_font_manager is None:
from src.font_manager import FontManager
_shared_fallback_font_manager = FontManager({})
return _shared_fallback_font_manager
class VegasDisplayMode(Enum):
"""
Display mode for Vegas scroll integration.
@@ -73,7 +86,9 @@ class BasePlugin(ABC):
self.display_manager: Any = display_manager
self.cache_manager: Any = cache_manager
self.plugin_manager: Any = plugin_manager
self.logger: logging.Logger = get_logger(f"plugin.{plugin_id}", plugin_id=plugin_id)
# get_logger returns a PluginLoggerAdapter here (plugin_id given), which
# stamps every record with plugin_id so it survives into formatted output.
self.logger = get_logger(f"plugin.{plugin_id}", plugin_id=plugin_id)
self.enabled: bool = config.get("enabled", True)
self.logger.info("Initialized plugin: %s", plugin_id)
@@ -130,6 +145,133 @@ class BasePlugin(ABC):
"""
raise NotImplementedError("Plugins must implement display()")
# -------------------------------------------------------------------------
# Adaptive layout support (opt-in)
# -------------------------------------------------------------------------
@property
def layout(self) -> Any:
"""
LayoutContext for the current logical display size.
Lazily built and rebuilt automatically when the display size changes
(e.g. Vegas segment widths, double-sided logical screens). Provides
Region carving (self.layout.bounds), breakpoint tiers, a geometry
scale factor vs. the manifest's display.design_size, and fit-text
queries against font ladders. See src/adaptive_layout.py.
Example:
rows = self.layout.bounds.inset(1).split_v(3, 1, gap=1)
self.draw_fit(big_text, rows[0], ladder=LADDER_ARCADE)
self.draw_fit(small_text, rows[1])
"""
from src.adaptive_layout import LayoutContext
width = getattr(self.display_manager, "width", None)
height = getattr(self.display_manager, "height", None)
if not width or not height:
matrix = getattr(self.display_manager, "matrix", None)
width = getattr(matrix, "width", 128)
height = getattr(matrix, "height", 32)
font_manager = self._get_font_manager()
generation = getattr(font_manager, "cache_generation", 0)
cached = getattr(self, "_layout_context", None)
if (cached is not None
and (cached.width, cached.height) == (width, height)
and getattr(self, "_layout_font_generation", None) == generation):
return cached
context = LayoutContext(
width, height, font_manager,
design_size=self._get_design_size(),
)
self._layout_context = context
self._layout_font_generation = generation
return context
def draw_fit(self, text: str, box: Any,
color: tuple = (255, 255, 255),
ladder: Optional[Any] = None,
align: str = "center", valign: str = "center") -> Any:
"""
Fit text to a Region with the largest crisp font that fits, then draw
it aligned within that region via the display manager.
Args:
text: Text to display (ellipsized if even the smallest rung is too wide)
box: Region (or (w, h) tuple anchored at 0,0) to fit and align within
color: RGB color tuple
ladder: FontLadder to walk (default LADDER_GRID; use LADDER_ARCADE
for headline text like clocks and scores)
align/valign: alignment of the text ink within the box
Returns:
FitResult (font, family, size_px, text, ink metrics, fits flag)
"""
from src.adaptive_layout import LADDER_DEFAULT, draw_fitted_text
fit = self.layout.fit_text(text, box, ladder=ladder or LADDER_DEFAULT)
draw_fitted_text(self.display_manager, fit, box,
color=color, align=align, valign=valign)
return fit
def draw_image(self, img: Any, box: Any, *,
mode: str = "contain", align: str = "center",
valign: str = "center", crop_to_ink: bool = False,
anchor: str = "center", resample: Optional[Any] = None,
cache_key: Optional[Any] = None,
offset: tuple = (0, 0)) -> Any:
"""
Fit an image into a Region and paste it aligned within that region
onto the display canvas the image counterpart to draw_fit().
Args:
img: Source PIL image (logos, art, icons)
box: Region (or (w, h) tuple) to fit and align within
mode: "contain" (letterbox), "cover" (crop-to-fill),
"fill_height" (logo-style), "stretch"
crop_to_ink: Trim transparent padding before fitting
anchor: "center" or "top" for cover crops
resample: PIL filter; default LANCZOS. Use RESAMPLE_NEAREST
(from src.adaptive_images) for pixel art/flags
cache_key: Stable identity (e.g. "logo:KC") for cross-reload
caching; defaults to the image object's identity
offset: Final (dx, dy) translation the hook for user
x/y-offset customization
Returns:
ImageFitResult (processed image + dimensions + scale)
"""
from src.adaptive_images import draw_fitted_image
ifit = self.layout.fit_image(img, box, mode=mode,
crop_to_ink=crop_to_ink, anchor=anchor,
resample=resample, cache_key=cache_key)
draw_fitted_image(self.display_manager, ifit, box,
align=align, valign=valign, offset=offset)
return ifit
def _get_font_manager(self) -> Any:
"""The shared FontManager, or a module-level fallback when running
under mocks/harnesses that don't provide one."""
font_manager = getattr(self.plugin_manager, "font_manager", None)
if font_manager is not None and hasattr(font_manager, "get_font"):
return font_manager
return _fallback_font_manager()
def _get_design_size(self) -> tuple:
"""Panel size this plugin's layout was authored against, from the
manifest's optional display.design_size (defaults to 128x32)."""
from src.adaptive_layout import DEFAULT_DESIGN_SIZE
if self.plugin_manager and hasattr(self.plugin_manager, "plugin_manifests"):
manifest = self.plugin_manager.plugin_manifests.get(self.plugin_id, {})
declared = manifest.get("display", {}).get("design_size", {})
width, height = declared.get("width"), declared.get("height")
if width and height:
return (int(width), int(height))
return DEFAULT_DESIGN_SIZE
def get_display_duration(self) -> float:
"""
Get the display duration for this plugin instance.
+53 -11
View File
@@ -52,11 +52,18 @@ class PluginHealthTracker:
"""Get cache key for plugin health data."""
return f"plugin_health:{plugin_id}"
def _load_health_state(self, plugin_id: str) -> Dict[str, Any]:
"""Load health state from cache or return defaults."""
def _load_health_state(self, plugin_id: str, force_reload: bool = False) -> Dict[str, Any]:
"""Load health state from cache or return defaults.
``force_reload=True`` bypasses the cache manager's in-memory tier so a
read-only consumer (e.g. the web process) observes the writer process's
latest persisted state instead of a stale first snapshot.
"""
cache_key = self._get_health_key(plugin_id)
cached = self.cache_manager.get(cache_key, max_age=None)
cached = self.cache_manager.get(
cache_key, max_age=None, memory_ttl=0 if force_reload else None
)
if cached:
return cached
@@ -79,10 +86,17 @@ class PluginHealthTracker:
self.cache_manager.set(cache_key, state) # Persist indefinitely
self._health_state[plugin_id] = state
def get_health_state(self, plugin_id: str) -> Dict[str, Any]:
"""Get current health state for a plugin."""
if plugin_id not in self._health_state:
self._health_state[plugin_id] = self._load_health_state(plugin_id)
def get_health_state(self, plugin_id: str, force_reload: bool = False) -> Dict[str, Any]:
"""Get current health state for a plugin.
``force_reload=True`` re-reads the persisted state from the cache,
bypassing the in-memory copy needed by cross-process readers that
would otherwise be pinned to the first snapshot they loaded.
"""
if force_reload or plugin_id not in self._health_state:
self._health_state[plugin_id] = self._load_health_state(
plugin_id, force_reload=force_reload
)
return self._health_state[plugin_id]
def record_success(self, plugin_id: str) -> None:
@@ -139,6 +153,28 @@ class PluginHealthTracker:
self._save_health_state(plugin_id, state)
def set_degraded(self, plugin_id: str, reason: Optional[str]) -> None:
"""Flag (or clear) a plugin as degraded without touching the circuit breaker.
Used for non-fatal issues e.g. a config that no longer satisfies the
plugin's schema — that should be surfaced to the user but must NOT cause
the plugin to be skipped or counted as a runtime failure. Passing
``reason=None`` clears the flag. The write is skipped when nothing
actually changes, so calling this on every load is cheap.
Args:
plugin_id: Plugin identifier
reason: Human-readable reason string, or None to clear the flag
"""
state = self.get_health_state(plugin_id)
new_degraded = bool(reason)
new_reason = reason if reason else None
if state.get('degraded', False) == new_degraded and state.get('degraded_reason') == new_reason:
return # No change — avoid a redundant cache write
state['degraded'] = new_degraded
state['degraded_reason'] = new_reason
self._save_health_state(plugin_id, state)
def should_skip_plugin(self, plugin_id: str) -> bool:
"""
Check if plugin should be skipped due to circuit breaker.
@@ -181,9 +217,13 @@ class PluginHealthTracker:
return False
def get_health_summary(self, plugin_id: str) -> Dict[str, Any]:
"""Get health summary for a plugin."""
state = self.get_health_state(plugin_id)
def get_health_summary(self, plugin_id: str, force_reload: bool = False) -> Dict[str, Any]:
"""Get health summary for a plugin.
``force_reload=True`` refreshes from the persisted cache first so
cross-process readers reflect the writer's latest state.
"""
state = self.get_health_state(plugin_id, force_reload=force_reload)
total_calls = state.get('total_successes', 0) + state.get('total_failures', 0)
success_rate = 0.0
@@ -201,6 +241,8 @@ class PluginHealthTracker:
'last_failure_time': state.get('last_failure_time'),
'last_error': state.get('last_error'),
'is_healthy': state.get('circuit_state') == CircuitState.CLOSED.value,
'degraded': state.get('degraded', False),
'degraded_reason': state.get('degraded_reason'),
'circuit_opened_time': state.get('circuit_opened_time'),
'half_open_start_time': state.get('half_open_start_time')
}
+233 -76
View File
@@ -5,10 +5,10 @@ Handles plugin module imports, dependency installation, and class instantiation.
Extracted from PluginManager to improve separation of concerns.
"""
import hashlib
import json
import importlib
import importlib.metadata
import importlib.util
import json
import os
import sys
import subprocess
@@ -17,12 +17,101 @@ from pathlib import Path
from typing import Dict, Any, Optional, Tuple, Type
import logging
from packaging.requirements import InvalidRequirement, Requirement
from src.exceptions import PluginError
from src.logging_config import get_logger
from src.common.permission_utils import (
ensure_file_permissions,
get_plugin_file_mode
)
def requirements_has_real_deps(requirements_file: str) -> bool:
"""
Check whether a requirements.txt actually specifies anything to install.
Plugins that ship all their dependencies with LEDMatrix core often keep a
requirements.txt where every line is commented out, for documentation
purposes only. Running pip against such a file still pays the full
subprocess/resolver cost for zero effect, so callers should skip the
install step entirely when this returns False.
"""
try:
with open(requirements_file, 'r', encoding='utf-8') as fh:
for line in fh:
line = line.strip()
if line and not line.startswith('#'):
return True
except OSError:
# Let the caller's own file handling report the error.
return True
return False
def requirements_are_satisfied(requirements_file: str) -> bool:
"""
Check whether every real requirement line in requirements.txt is already
satisfied by packages installed in the current interpreter.
This replaces marker-file tracking with a direct fact check, so it's
immune to stale/missing/corrupted markers: it looks at what's actually
importable right now rather than trusting a hash comparison from a
previous run. Anything ambiguous (pip options, unparseable lines,
extras, unresolvable versions) conservatively returns False so the
caller falls through to running pip this check only ever saves work,
never masks a real install.
"""
try:
with open(requirements_file, 'r', encoding='utf-8') as fh:
lines = fh.readlines()
except OSError:
return False
for raw_line in lines:
line = raw_line.strip()
if not line or line.startswith('#'):
continue
if line.startswith('-'):
return False # pip option (-r, --index-url, ...), can't verify
try:
req = Requirement(line)
except InvalidRequirement:
return False
if req.extras:
return False # verifying extras' sub-dependencies isn't worth it here
if req.marker is not None and not req.marker.evaluate():
continue # not applicable on this platform/interpreter
try:
installed_version = importlib.metadata.version(req.name)
except importlib.metadata.PackageNotFoundError:
return False
if req.specifier and not req.specifier.contains(installed_version, prereleases=True):
return False
return True
def find_trusted_subdir(trusted_dir: str, name: str) -> Optional[str]:
"""Return `name` if it names an actual subdirectory of trusted_dir, else None.
Used as a containment check for a directory name derived from untrusted
input (a manifest-declared plugin id, an externally-supplied plugin
path): the returned value always comes from enumerating trusted_dir
itself via os.scandir(), so a caller that builds a path by joining
trusted_dir with this return value is joining against a name the
filesystem produced under a trusted root -- not the caller's original
string, which could otherwise smuggle a traversal sequence through.
"""
try:
with os.scandir(trusted_dir) as entries:
for entry in entries:
if entry.name == name and entry.is_dir():
return entry.name
except OSError:
pass
return None
class PluginLoader:
@@ -132,14 +221,14 @@ class PluginLoader:
except (json.JSONDecodeError, Exception) as e:
self.logger.debug("Skipping %s due to manifest error: %s", item.name, e)
continue
return None
def install_dependencies(
self,
plugin_dir: Path,
plugin_id: str,
plugins_dir: Optional[Path] = None,
plugins_dir: Path,
timeout: int = 300
) -> bool:
"""
@@ -148,7 +237,12 @@ class PluginLoader:
Args:
plugin_dir: Plugin directory path
plugin_id: Plugin identifier
plugins_dir: Trusted base plugins directory for path containment check
plugins_dir: Trusted base plugins directory for path containment check.
Required (not optional) so every caller reconstructs the plugin
path through the sanitiser below rather than trusting plugin_dir
directly -- CodeQL's path-injection query (and a malicious
manifest/plugin_id in practice) can't tell a legitimate
plugin_dir from one crafted to traverse outside plugins_dir.
timeout: Installation timeout in seconds
Returns:
@@ -160,59 +254,42 @@ class PluginLoader:
# Resolve to a canonical absolute path (normalises .. and symlinks)
plugin_dir_real = os.path.realpath(str(plugin_dir))
plugins_dir_real = os.path.realpath(str(plugins_dir))
requested_name = os.path.basename(plugin_dir_real)
if plugins_dir is not None:
# Reconstruct the plugin path from a trusted base + a sanitised
# directory name. os.path.basename() is CodeQL's recognised
# py/path-injection sanitiser: it strips all directory components
# so the result cannot contain traversal sequences. Joining it
# with the resolved, trusted plugins_dir produces a path that
# CodeQL considers untainted.
plugins_dir_real = os.path.realpath(str(plugins_dir))
safe_dir_name = os.path.basename(plugin_dir_real)
if not safe_dir_name:
self.logger.error("Could not determine plugin directory name for %s", plugin_id)
return False
safe_plugin_dir = os.path.join(plugins_dir_real, safe_dir_name)
if not os.path.isdir(safe_plugin_dir):
self.logger.error(
"Plugin directory for %s not found inside plugins dir", plugin_id
)
return False
else:
safe_plugin_dir = plugin_dir_real
if not os.path.isdir(safe_plugin_dir):
self.logger.error("Plugin directory does not exist: %s", plugin_dir)
return False
# Match the requested directory against an entry actually enumerated
# from the trusted plugins_dir, and build the path from that entry --
# not from requested_name. A name that came out of os.scandir() on a
# trusted root carries no taint regardless of what the caller asked
# for, so this is a real containment guarantee (an allowlist check
# against a trusted source), not a string-sanitisation of untrusted
# input that a static analyzer has to trust blindly.
matched_name = find_trusted_subdir(plugins_dir_real, requested_name)
if matched_name is None:
self.logger.error(
"Plugin directory for %s not found inside plugins dir", plugin_id
)
return False
safe_plugin_dir = os.path.join(plugins_dir_real, matched_name)
requirements_file = os.path.join(safe_plugin_dir, "requirements.txt")
marker_file = os.path.join(safe_plugin_dir, ".dependencies_installed")
if not os.path.isfile(requirements_file):
return True # No dependencies needed
try:
with open(requirements_file, 'rb') as fh:
current_hash = hashlib.sha256(fh.read()).hexdigest()
except OSError as e:
self.logger.error("Failed to read requirements.txt for %s: %s", plugin_id, e)
return False
if not requirements_has_real_deps(requirements_file):
self.logger.debug(
"requirements.txt for %s has no real dependencies (comments/blank only), skipping pip",
plugin_id
)
return True
# Skip if requirements.txt hasn't changed since last install
if os.path.isfile(marker_file):
try:
with open(marker_file, 'r', encoding='utf-8') as fh:
stored_hash = fh.read().strip()
except OSError as e:
self.logger.warning(
"Could not read dependency marker for %s (%s), will reinstall dependencies",
plugin_id, e
)
else:
if stored_hash == current_hash:
self.logger.debug("Dependencies already installed for %s (requirements unchanged)", plugin_id)
return True
self.logger.info("Requirements changed for %s, reinstalling dependencies", plugin_id)
if requirements_are_satisfied(requirements_file):
self.logger.debug(
"Dependencies for %s already satisfied in current environment, skipping pip",
plugin_id
)
return True
try:
self.logger.info("Installing dependencies for plugin %s...", plugin_id)
@@ -225,32 +302,54 @@ class PluginLoader:
)
if result.returncode == 0:
try:
with open(marker_file, 'w', encoding='utf-8') as fh:
fh.write(current_hash)
ensure_file_permissions(Path(marker_file), get_plugin_file_mode())
except OSError as marker_err:
self.logger.debug("Could not write dependency marker for %s: %s", plugin_id, marker_err)
self.logger.info("Dependencies installed successfully for %s", plugin_id)
return True
else:
stderr = result.stderr or ""
# uninstall-no-record-file means the package is already present at the
# system level (e.g. installed via dnf/apt without a pip RECORD file).
# pip can't replace it, but it IS installed — write the marker so we
# don't retry on every restart.
# uninstall-no-record-file means a system-managed copy of a package
# (e.g. apt's python3-requests, which ships no pip RECORD file) is in
# the way of the version this requirements.txt pins. Retry with
# --ignore-installed so pip lays the pinned version down alongside
# the system copy instead of trying to replace it — matching the
# retry already used by install_dependencies_apt.py / safe_pip_install.sh.
# Without this retry, the plugin would silently keep running against
# whatever version the system happened to ship.
if "uninstall-no-record-file" in stderr:
self.logger.warning(
"Dependencies for %s include system-managed packages (no pip RECORD). "
"Assuming they are satisfied: %s",
"Dependencies for %s conflict with a system-managed package "
"(no pip RECORD); retrying with --ignore-installed: %s",
plugin_id, stderr.strip()
)
# Wrapped in its own try/except so a retry timeout is
# tolerated the same way as a retry failure, instead of
# propagating to the outer handler and returning False
# (which would contradict the "assume satisfied" fallback
# below).
try:
with open(marker_file, 'w', encoding='utf-8') as fh:
fh.write(current_hash)
ensure_file_permissions(Path(marker_file), get_plugin_file_mode())
except OSError as marker_err:
self.logger.debug("Could not write dependency marker for %s: %s", plugin_id, marker_err)
# sys.executable is this process's own interpreter (not
# attacker-influenced), and requirements_file is a path
# built internally by find_plugin_directory, never raw
# external input.
retry_result = subprocess.run( # nosec B603 - no shell invoked (list-form argv) # nosemgrep
[sys.executable, "-m", "pip", "install", "--break-system-packages",
"--ignore-installed", "-r", requirements_file],
capture_output=True,
text=True,
timeout=timeout,
check=False
)
if retry_result.returncode != 0:
self.logger.warning(
"Retry with --ignore-installed also failed for %s; assuming the "
"system-managed version satisfies the requirement: %s",
plugin_id, (retry_result.stderr or "").strip()
)
except subprocess.TimeoutExpired:
self.logger.warning(
"Retry with --ignore-installed timed out for %s; assuming the "
"system-managed version satisfies the requirement",
plugin_id
)
return True
self.logger.warning(
"Dependency installation returned non-zero exit code for %s: %s",
@@ -338,8 +437,7 @@ class PluginLoader:
if not Path(existing_file).resolve().is_relative_to(resolved_dir):
evicted[mod_name] = sys.modules.pop(mod_name)
self.logger.debug(
"Evicted stale module '%s' (from %s) before loading plugin in %s",
mod_name, existing_file, plugin_dir,
"Evicted stale bare-name module '%s' before loading plugin", mod_name,
)
except (ValueError, TypeError):
continue
@@ -452,7 +550,7 @@ class PluginLoader:
plugin_dir_str = str(plugin_dir)
if plugin_dir_str not in sys.path:
sys.path.insert(0, plugin_dir_str)
self.logger.debug("Added plugin directory to sys.path: %s", plugin_dir_str)
self.logger.debug("Added plugin %s's directory to sys.path", plugin_id)
# Import the plugin module
module_name = f"plugin_{plugin_id.replace('-', '_')}"
@@ -464,8 +562,8 @@ class PluginLoader:
spec = importlib.util.spec_from_file_location(module_name, entry_file)
if spec is None or spec.loader is None:
self.logger.error("Could not create module spec for plugin %s", plugin_id)
error_msg = f"Could not create module spec for {entry_file}"
self.logger.error(error_msg)
raise PluginError(error_msg, plugin_id=plugin_id, context={'entry_file': str(entry_file)})
module = importlib.util.module_from_spec(spec)
@@ -584,6 +682,55 @@ class PluginLoader:
self.logger.error(error_msg, exc_info=True)
raise PluginError(error_msg, plugin_id=plugin_id) from e
@staticmethod
def _parse_semver(value: Any) -> Optional[Tuple[int, int, int]]:
"""Parse 'X.Y.Z' (extra parts/suffixes ignored) into a comparable
3-tuple, or None when unparseable."""
if not isinstance(value, str):
return None
parts = value.strip().lstrip('v').split('.')
try:
nums = [int(''.join(ch for ch in p if ch.isdigit()) or 0) for p in parts[:3]]
except ValueError:
return None
while len(nums) < 3:
nums.append(0)
return tuple(nums) # type: ignore[return-value]
def _warn_if_incompatible(self, plugin_id: str, manifest: Dict[str, Any]) -> None:
"""Log one warning when a plugin declares a minimum LEDMatrix version
newer than the running core. Advisory only never raises so a
plugin that guards optional features with try/except keeps working.
"""
declared = (
manifest.get('min_ledmatrix_version')
or manifest.get('requires', {}).get('min_ledmatrix_version')
)
if not declared:
versions = manifest.get('versions') or []
if versions and isinstance(versions[0], dict):
declared = (versions[0].get('ledmatrix_min_version')
or versions[0].get('ledmatrix_min'))
needed = self._parse_semver(declared)
if needed is None:
return
from src import __version__ as core_version
current = self._parse_semver(core_version)
# Anti-spam guard: if the core's own version number is stale (below
# the ecosystem floor every shipped plugin declares), comparing would
# warn on nearly everything — skip with a debug note instead.
if current is None or current < (2, 0, 0):
self.logger.debug(
"Skipping version compatibility check for %s: core __version__ "
"(%s) is below the ecosystem floor", plugin_id, core_version)
return
if needed > current:
self.logger.warning(
"Plugin %s declares min LEDMatrix version %s but this core is %s"
"features it relies on may be missing; update the core or expect "
"degraded fallbacks", plugin_id, declared, core_version)
def load_plugin(
self,
plugin_id: str,
@@ -616,8 +763,18 @@ class PluginLoader:
Raises:
PluginError: If loading fails
"""
self._warn_if_incompatible(plugin_id, manifest)
# Install dependencies if needed
if install_deps:
if plugins_dir is None:
raise PluginError(
f"plugins_dir is required to install dependencies for plugin {plugin_id} "
"(needed for path containment; pass install_deps=False if the caller "
"doesn't have a trusted plugins directory to supply)",
plugin_id=plugin_id,
context={'plugin_dir': str(plugin_dir)},
)
if not self.install_dependencies(plugin_dir, plugin_id, plugins_dir=plugins_dir):
raise PluginError(
f"Dependency installation failed for plugin {plugin_id} in {plugin_dir}",
+334 -123
View File
@@ -8,12 +8,13 @@ API Version: 1.0.0
"""
import json
import queue
import sys
import subprocess
import time
import threading
import types
from pathlib import Path
from typing import Dict, List, Optional, Any
from typing import Dict, List, Optional, Any, Tuple
import logging
from src.exceptions import PluginError, ConfigError
from src.logging_config import get_logger
@@ -76,6 +77,12 @@ class PluginManager:
# concurrent mutation (background reconciliation) and reads (requests).
self._discovery_lock = threading.RLock()
# Lock protecting plugin_last_update from concurrent mutation/iteration.
# It's written from run_scheduled_updates()/update_all_plugins() (main
# loop) and read/diffed by run_scheduled_updates_with_changes(), which
# Vegas mode calls from its own background update-tick thread.
self._plugin_last_update_lock = threading.RLock()
# Active plugins
self.plugins: Dict[str, Any] = {}
self.plugin_manifests: Dict[str, Dict[str, Any]] = {}
@@ -91,6 +98,50 @@ class PluginManager:
# Health tracking (optional, set by display_controller if available)
self.health_tracker = None
self.resource_monitor = None
# --- Asynchronous plugin updates -------------------------------
# update() used to run inline in the render loop (execute_update's
# internal thread.join(timeout=30) blocked it), so one slow plugin
# HTTP fetch froze scrolling for the whole fetch. Scheduling still
# happens on the render thread (run_scheduled_updates), but
# execution moves to this single background worker. Per-plugin
# locks keep a plugin's update() and display() mutually exclusive —
# today's implicit guarantee, now explicit (and, unlike today,
# also held across the post-timeout window).
# Kill switch: plugin_system.synchronous_updates: true restores the
# inline path.
self._update_queue: "queue.Queue[Optional[Tuple[str, float]]]" = queue.Queue()
self._pending_updates: set = set()
self._pending_lock = threading.Lock()
self._plugin_locks: Dict[str, threading.Lock] = {}
self._plugin_locks_guard = threading.Lock()
self._update_worker: Optional[threading.Thread] = None
self._synchronous_updates = False
if self.config_manager is not None:
try:
cfg = self.config_manager.get_config() or {}
except (OSError, ValueError) as exc:
self.logger.warning(
"Could not load config to check plugin_system.synchronous_updates "
"(%s: %s); defaulting to synchronous updates", type(exc).__name__, exc)
self._synchronous_updates = True
else:
plugin_system_cfg = cfg.get('plugin_system', {})
if not isinstance(plugin_system_cfg, dict):
self.logger.warning(
"config plugin_system must be a mapping, got %s; "
"defaulting to synchronous updates",
type(plugin_system_cfg).__name__)
self._synchronous_updates = True
else:
sync_value = plugin_system_cfg.get('synchronous_updates', False)
if not isinstance(sync_value, bool):
self.logger.warning(
"config plugin_system.synchronous_updates must be a boolean, "
"got %r; defaulting to synchronous updates", sync_value)
self._synchronous_updates = True
else:
self._synchronous_updates = sync_value
# Ensure plugins directory exists with proper permissions
try:
@@ -177,90 +228,6 @@ class PluginManager:
return plugin_ids
def _get_dependency_marker_path(self, plugin_id: str) -> Path:
"""Get path to dependency installation marker file."""
plugin_dir = self.plugins_dir / plugin_id
if not plugin_dir.exists():
# Try with ledmatrix- prefix
plugin_dir = self.plugins_dir / f"ledmatrix-{plugin_id}"
return plugin_dir / ".dependencies_installed"
def _check_dependencies_installed(self, plugin_id: str) -> bool:
"""Check if dependencies are already installed for a plugin."""
marker_path = self._get_dependency_marker_path(plugin_id)
return marker_path.exists()
def _mark_dependencies_installed(self, plugin_id: str) -> None:
"""Mark dependencies as installed for a plugin."""
marker_path = self._get_dependency_marker_path(plugin_id)
try:
marker_path.touch()
# Set proper file permissions after creating marker
from src.common.permission_utils import (
ensure_file_permissions,
get_plugin_file_mode
)
ensure_file_permissions(marker_path, get_plugin_file_mode())
except (OSError, PermissionError) as e:
self.logger.warning("Could not create dependency marker for %s: %s", plugin_id, e)
def _remove_dependency_marker(self, plugin_id: str) -> None:
"""Remove dependency installation marker."""
marker_path = self._get_dependency_marker_path(plugin_id)
try:
if marker_path.exists():
marker_path.unlink()
except (OSError, PermissionError) as e:
self.logger.warning("Could not remove dependency marker for %s: %s", plugin_id, e)
def _install_plugin_dependencies(self, requirements_file: Path) -> bool:
"""
Install plugin dependencies from requirements.txt.
Args:
requirements_file: Path to requirements.txt
Returns:
True if installation succeeded or not needed, False on error
"""
try:
self.logger.info("Installing dependencies from %s", requirements_file)
result = subprocess.run(
[sys.executable, "-m", "pip", "install", "--break-system-packages", "--no-cache-dir", "-r", str(requirements_file)],
capture_output=True,
text=True,
timeout=300,
check=False
)
if result.returncode == 0:
self.logger.info("Dependencies installed successfully")
return True
else:
self.logger.warning("Dependency installation returned non-zero exit code: %s", result.stderr)
return False
except subprocess.TimeoutExpired:
self.logger.error("Dependency installation timed out")
return False
except FileNotFoundError as e:
self.logger.warning("Command not found: %s. Skipping dependency installation", e)
return True
except (BrokenPipeError, OSError) as e:
# Handle broken pipe errors (errno 32) which can occur during pip downloads
# Often caused by network interruptions or output buffer issues
if isinstance(e, OSError) and e.errno == 32:
self.logger.error(
"Broken pipe error during dependency installation. "
"This usually indicates a network interruption or pip output buffer issue. "
"Try installing again or check your network connection."
)
else:
self.logger.error("OS error during dependency installation: %s", e)
return False
except Exception as e:
self.logger.error("Unexpected error installing dependencies: %s", e, exc_info=True)
return True
def load_plugin(self, plugin_id: str) -> bool:
"""
Load a plugin by ID.
@@ -390,10 +357,19 @@ class PluginManager:
self.logger.error("Error validating plugin %s config: %s", plugin_id, e, exc_info=True)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
return False
# Schema validation (warn/degrade only — never blocks loading).
# A config that violates the plugin's JSON schema is surfaced to the
# user (log warning + degraded flag in the health tracker) but the
# plugin still loads exactly as it does today. This deliberately does
# NOT change load_plugin()'s pass/fail behaviour for any plugin that
# loads under the current code.
self._validate_config_schema_soft(plugin_id, config)
# Store plugin instance
self.plugins[plugin_id] = plugin_instance
self.plugin_last_update[plugin_id] = 0.0
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = 0.0
# Invalidate cached interval so next tick re-derives it for this plugin
self._update_interval_cache.pop(plugin_id, None)
@@ -419,6 +395,59 @@ class PluginManager:
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
return False
def _validate_config_schema_soft(self, plugin_id: str, config: Dict[str, Any]) -> None:
"""Validate a plugin's config against its JSON schema — warn/degrade only.
On a schema violation this logs a warning and marks the plugin degraded
in the health tracker (when one is wired), so the problem is visible in
the web UI. It never raises, never changes plugin state, and never
affects whether the plugin loads. ``config`` here has already been
merged with schema defaults by the caller, so fields that ship a default
never appear "missing" only genuinely user-supplied required fields
(e.g. an API key) can trip the required-field check.
"""
try:
schema = self.schema_manager.load_schema(plugin_id)
except Exception as e: # pragma: no cover - defensive
self.logger.debug("Could not load schema for %s: %s", plugin_id, e)
return
if not schema:
# No schema shipped — nothing to validate. Clear any stale flag.
self._set_degraded_safe(plugin_id, None)
return
try:
is_valid, errors = self.schema_manager.validate_config_against_schema(
config, schema, plugin_id
)
except Exception as e: # pragma: no cover - defensive
# Validation machinery itself failed — do not penalise the plugin.
self.logger.debug("Schema validation raised for %s: %s", plugin_id, e)
return
if is_valid or not errors:
self._set_degraded_safe(plugin_id, None)
return
summary = "; ".join(errors[:5])
if len(errors) > 5:
summary += f" (+{len(errors) - 5} more)"
self.logger.warning(
"Plugin %s config does not match its schema (loading anyway): %s",
plugin_id, summary,
)
self._set_degraded_safe(plugin_id, f"Config schema: {summary}")
def _set_degraded_safe(self, plugin_id: str, reason: Optional[str]) -> None:
"""Best-effort ``health_tracker.set_degraded`` that never raises."""
if not self.health_tracker:
return
try:
self.health_tracker.set_degraded(plugin_id, reason)
except Exception as e: # pragma: no cover - defensive
self.logger.debug("Could not set degraded flag for %s: %s", plugin_id, e)
def unload_plugin(self, plugin_id: str) -> bool:
"""
Unload a plugin by ID.
@@ -452,7 +481,8 @@ class PluginManager:
# Remove from active plugins
del self.plugins[plugin_id]
self.plugin_last_update.pop(plugin_id, None)
with self._plugin_last_update_lock:
self.plugin_last_update.pop(plugin_id, None)
self._update_interval_cache.pop(plugin_id, None)
# Remove main module from sys.modules if present
@@ -721,7 +751,8 @@ class PluginManager:
'recoverable': True,
}
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
self.plugin_last_update[plugin_id] = failure_time
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = failure_time
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info, error=err)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, err)
@@ -754,39 +785,218 @@ class PluginManager:
if interval is None:
continue
last_update = self.plugin_last_update.get(plugin_id, 0.0)
with self._plugin_last_update_lock:
last_update = self.plugin_last_update.get(plugin_id, 0.0)
if last_update == 0.0 or (current_time - last_update) >= interval:
# Update state to RUNNING
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
try:
# Use PluginExecutor for safe execution
success = False
if self.resource_monitor:
# If resource monitor exists, wrap the call
def monitored_update():
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
success = self.plugin_executor.execute_update(
type('obj', (object,), {'update': monitored_update})(),
plugin_id
)
else:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
self.plugin_last_update[plugin_id] = current_time
self.state_manager.record_update(plugin_id)
# Update state back to ENABLED
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
# Record success
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
if self._synchronous_updates:
# Kill-switch path: the original inline execution
# (blocks the caller until update() completes/times out)
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
self._execute_update_now(plugin_id, plugin_instance, current_time)
else:
self._enqueue_update(plugin_id, current_time)
def get_plugin_lock(self, plugin_id: str) -> threading.Lock:
"""Per-plugin lock keeping update() and display() mutually exclusive.
The update worker holds it for the duration of a plugin's update();
the display side acquires it non-blocking and skips that frame's
display() call when the plugin is mid-update.
"""
with self._plugin_locks_guard:
lock = self._plugin_locks.get(plugin_id)
if lock is None:
lock = threading.Lock()
self._plugin_locks[plugin_id] = lock
return lock
def _enqueue_update(self, plugin_id: str, scheduled_time: float) -> None:
"""Queue a due update for the background worker (dedup while pending)."""
with self._pending_lock:
if plugin_id in self._pending_updates:
return
self._pending_updates.add(plugin_id)
# RUNNING is set at enqueue time so can_execute() blocks re-entry and
# the web UI shows the truthful state while the item waits its turn.
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
self._ensure_update_worker()
self._update_queue.put((plugin_id, scheduled_time))
def _ensure_update_worker(self) -> None:
if self._update_worker is not None and self._update_worker.is_alive():
return
self._update_worker = threading.Thread(
target=self._update_worker_loop, name='plugin-update-worker',
daemon=True)
self._update_worker.start()
def _update_worker_loop(self) -> None:
"""Single worker: dispatches queued updates off the render thread
(matching the old inline behavior no thundering herd of
concurrent fetches).
The plugin's lock is acquired here, before its instance is looked
up, and the instance is re-fetched under the lock a concurrent
unload_plugin() can't leave this loop about to run update() on an
instance that's already been torn down. The lock — and RUNNING/
pending lifecycle state is released by the update itself once the
real update() call genuinely finishes (see _execute_update_now),
which can be after this dispatch returns if PluginExecutor's own
timeout elapses first.
"""
while True:
item = self._update_queue.get()
if item is None: # shutdown sentinel
return
plugin_id, scheduled_time = item
lock = self.get_plugin_lock(plugin_id)
lock.acquire()
plugin_instance = self.plugins.get(plugin_id)
if plugin_instance is None: # unloaded while queued; its
# lifecycle state was already cleared by unload_plugin —
# leave it alone rather than resurrecting it to ENABLED
lock.release()
with self._pending_lock:
self._pending_updates.discard(plugin_id)
continue
try:
self._execute_update_now(plugin_id, plugin_instance,
scheduled_time, lock=lock)
except Exception: # pylint: disable=broad-except
# _execute_update_now guarantees the lock/pending bookkeeping
# is released via its own _finish() before returning or
# raising; this is a last-resort log only.
self.logger.exception("update worker: unexpected error for %s",
plugin_id)
def stop_update_worker(self, timeout: float = 5.0) -> None:
"""Signal the worker to exit (used by cleanup; thread is a daemon)."""
if self._update_worker is not None and self._update_worker.is_alive():
self._update_queue.put(None)
self._update_worker.join(timeout=timeout)
if self._update_worker.is_alive():
self.logger.warning(
"Update worker did not stop within %.1fs; it is a daemon "
"thread and will be abandoned on shutdown", timeout)
def _execute_update_now(self, plugin_id: str, plugin_instance: Any,
scheduled_time: float,
lock: Optional[threading.Lock] = None) -> None:
"""Execute a plugin's update() via PluginExecutor, then bookkeep.
Caller is responsible for having set RUNNING state.
On the synchronous path (``lock=None``) this is the original,
unchanged inline behavior. On the async worker path, PluginExecutor's
internal thread.join(timeout) blocks only the calling thread -- on
timeout the lingering daemon update-thread keeps running the real
plugin.update() call unkillable in the background. So that the
plugin's lock (and its RUNNING/pending lifecycle state) stays held
for that real duration rather than just this bounded wait, ownership
of both is carried by the wrapped update callable itself, released
from whichever thread actually finishes it -- see _finish() below.
"""
finish_guard = threading.Lock()
finished = {'done': False}
def _finish(success: bool, exc: Optional[Exception] = None) -> None:
with finish_guard:
if finished['done']:
return
finished['done'] = True
try:
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = scheduled_time
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id, exc=exc)
finally:
if lock is not None:
lock.release()
with self._pending_lock:
self._pending_updates.discard(plugin_id)
if lock is None:
# Synchronous / no-lock path: unchanged behavior.
try:
if self.resource_monitor:
def monitored_update():
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
# SimpleNamespace stores `update` as an *instance*
# attribute, so attribute lookup returns the plain
# function object as-is. A dynamically-built class
# (`type(..., {'update': monitored_update})`) instead
# stores it as a *class* attribute, which the
# descriptor protocol turns into a bound method on
# access -- silently prepending the instance as an
# implicit first argument to a function that takes
# none, raising "monitored_update() takes 0
# positional arguments but 1 was given" on every call.
success = self.plugin_executor.execute_update(
types.SimpleNamespace(update=monitored_update),
plugin_id
)
else:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
_finish(success)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
_finish(False, exc=exc)
return
# Async worker path: the real update() call -- through the resource
# monitor, if configured -- owns finishing the lock/lifecycle
# bookkeeping, from whichever thread actually runs it to completion.
def _target_update() -> None:
try:
if self.resource_monitor:
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
else:
plugin_instance.update()
except Exception as exc:
_finish(False, exc=exc)
raise
else:
_finish(True)
try:
self.plugin_executor.execute_update(
types.SimpleNamespace(update=_target_update), plugin_id)
except Exception as exc: # pragma: no cover - defensive; execute_update
# catches everything internally, but guarantee _finish still
# runs (releasing the lock) if something unexpected slips through.
self.logger.exception("Unexpected error dispatching update for %s: %s", plugin_id, exc)
_finish(False, exc=exc)
def run_scheduled_updates_with_changes(self, current_time: Optional[float] = None) -> List[str]:
"""
Like run_scheduled_updates(), but also returns the plugin_ids whose
plugin_last_update timestamp actually advanced during this call.
The before/after snapshots and the update pass itself are each
individually lock-protected against concurrent plugin_last_update
mutation (Vegas mode calls this from its own background
update-tick thread, racing the main render loop's plugin updates),
so callers get an atomic "who got fresh data" answer without
reaching into plugin_last_update themselves. The lock is not held
across the update pass so slow/blocking plugin update() calls don't
serialize against other plugin_last_update readers.
"""
with self._plugin_last_update_lock:
old_times = dict(self.plugin_last_update)
self.run_scheduled_updates(current_time)
with self._plugin_last_update_lock:
return [
plugin_id for plugin_id, new_time in self.plugin_last_update.items()
if new_time > old_times.get(plugin_id, 0.0)
]
def update_all_plugins(self) -> None:
"""
@@ -810,7 +1020,8 @@ class PluginManager:
try:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
self.plugin_last_update[plugin_id] = time.time()
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = time.time()
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else:
@@ -836,7 +1047,7 @@ class PluginManager:
# Get health tracker metrics if available
if self.health_tracker:
health_info = self.health_tracker.get_plugin_health(plugin_id)
health_info = self.health_tracker.get_health_summary(plugin_id)
plugin_metrics['health'] = health_info
else:
plugin_metrics['health'] = {'status': 'unknown'}
@@ -861,7 +1072,7 @@ class PluginManager:
# Get resource monitor metrics if available
if self.resource_monitor:
resource_info = self.resource_monitor.get_plugin_metrics(plugin_id)
resource_info = self.resource_monitor.get_metrics_summary(plugin_id)
plugin_metrics['resources'] = resource_info
else:
plugin_metrics['resources'] = {'status': 'unknown'}
+50 -20
View File
@@ -71,17 +71,32 @@ class PluginResourceMonitor:
self.cache_manager = cache_manager
self.enable_monitoring = enable_monitoring and PSUTIL_AVAILABLE
self.logger = logging.getLogger(__name__)
# Resource metrics per plugin
self._metrics: Dict[str, ResourceMetrics] = {}
self._limits: Dict[str, ResourceLimits] = {}
# Thread-local storage for execution tracking
self._local = threading.local()
# Lock for thread-safe access
self._lock = threading.Lock()
# Cache a single psutil.Process handle. Reusing the same handle is what
# lets cpu_percent() be read non-blocking (interval=None): psutil returns
# the utilisation since the *previous* call on that same object. Creating
# a fresh Process() per call would force interval-based sampling that
# blocks the caller — unacceptable on the display loop's update path.
self._process = None
if self.enable_monitoring:
try:
self._process = psutil.Process()
# Prime cpu_percent so the first real measurement returns a
# meaningful delta instead of 0.0.
self._process.cpu_percent(interval=None)
except Exception: # pragma: no cover - psutil edge cases
self._process = None
if not PSUTIL_AVAILABLE and enable_monitoring:
self.logger.warning(
"psutil not available - resource monitoring will be limited to execution time only"
@@ -95,13 +110,21 @@ class PluginResourceMonitor:
"""Get cache key for plugin limits."""
return f"plugin_limits:{plugin_id}"
def get_metrics(self, plugin_id: str) -> ResourceMetrics:
"""Get current metrics for a plugin."""
def get_metrics(self, plugin_id: str, force_reload: bool = False) -> ResourceMetrics:
"""Get current metrics for a plugin.
``force_reload=True`` bypasses both the in-memory copy and the cache
manager's memory tier so a read-only consumer (e.g. the web process)
sees the writer process's latest persisted metrics rather than a stale
first snapshot.
"""
with self._lock:
if plugin_id not in self._metrics:
if force_reload or plugin_id not in self._metrics:
# Try to load from cache
cache_key = self._get_metrics_key(plugin_id)
cached = self.cache_manager.get(cache_key, max_age=None)
cached = self.cache_manager.get(
cache_key, max_age=None, memory_ttl=0 if force_reload else None
)
if cached:
metrics = ResourceMetrics(**cached)
else:
@@ -137,21 +160,24 @@ class PluginResourceMonitor:
def _get_process_memory_mb(self) -> float:
"""Get current process memory usage in MB."""
if not self.enable_monitoring:
if not self.enable_monitoring or self._process is None:
return 0.0
try:
process = psutil.Process()
return process.memory_info().rss / 1024 / 1024
return self._process.memory_info().rss / 1024 / 1024
except Exception:
return 0.0
def _get_process_cpu_percent(self, interval: float = 0.1) -> float:
"""Get current process CPU usage percentage."""
if not self.enable_monitoring:
def _get_process_cpu_percent(self) -> float:
"""Get current process CPU usage percentage (non-blocking).
Reads cpu_percent(interval=None) against the cached process handle, so
it returns immediately with the utilisation observed since the previous
call rather than blocking to sample a fresh interval.
"""
if not self.enable_monitoring or self._process is None:
return 0.0
try:
process = psutil.Process()
return process.cpu_percent(interval=interval)
return self._process.cpu_percent(interval=None)
except Exception:
return 0.0
@@ -281,9 +307,13 @@ class PluginResourceMonitor:
self.logger.error(error_msg)
raise ResourceLimitExceeded(error_msg)
def get_metrics_summary(self, plugin_id: str) -> Dict[str, Any]:
"""Get metrics summary for a plugin."""
metrics = self.get_metrics(plugin_id)
def get_metrics_summary(self, plugin_id: str, force_reload: bool = False) -> Dict[str, Any]:
"""Get metrics summary for a plugin.
``force_reload=True`` refreshes from the persisted cache first so
cross-process readers reflect the writer's latest metrics.
"""
metrics = self.get_metrics(plugin_id, force_reload=force_reload)
limits = self.get_limits(plugin_id)
avg_execution_time = 0.0
+60
View File
@@ -284,6 +284,19 @@ class SchemaManager:
"type": "boolean",
"default": False,
"description": "Enable live priority takeover when plugin has live content"
},
# Skin selection (docs/SKIN_SYSTEM.md). Deliberately NOT an
# enum here: validation must keep passing when a configured
# skin gets uninstalled (rendering falls back to built-in).
# The install-dependent enum is injected only at serve time
# (inject_skin_selector) for the web UI dropdown.
"skin": {
"type": ["string", "object", "null"],
"description": "Visual skin id, or a per-mode mapping like {\"live\": \"my-skin\"}"
},
"skin_options": {
"type": "object",
"description": "Options passed through to the selected skin"
}
}
@@ -354,6 +367,53 @@ class SchemaManager:
self.logger.error(error_msg)
return False, [error_msg]
def inject_skin_selector(self, schema: Dict[str, Any], plugin_id: str,
current_value: Any = None) -> Dict[str, Any]:
"""Return a copy of a plugin's schema with a "skin" dropdown added
when installed skins target this plugin (docs/SKIN_SYSTEM.md).
Serve-time only validation never sees this enum, so a config
referencing an uninstalled skin stays valid (rendering falls back
to the built-in layout). The currently-configured value is always
included in the enum for the same reason: the dropdown must be able
to display a selection whose skin was removed.
"""
# A per-mode mapping ({"live": ..., "recent": ...}) can't be edited
# through a string dropdown — injecting one would let the form save
# a string over the mapping. Leave the schema alone; per-mode users
# edit via the raw JSON config editor.
if isinstance(current_value, dict):
return schema
try:
from src.skin_system import skin_runtime
matching = skin_runtime.skins_for_plugin(plugin_id)
except Exception as e:
self.logger.debug(f"Skin discovery failed for {plugin_id}: {e}")
return schema
choices = sorted(matching.keys())
if isinstance(current_value, str) and current_value and \
current_value != "built-in" and current_value not in choices:
choices.append(current_value)
if not choices:
return schema
enhanced = copy.deepcopy(schema)
enhanced.setdefault("properties", {})
if "skin" not in enhanced["properties"]:
names = {sid: (matching.get(sid, {}).get("name") or sid) for sid in choices}
enhanced["properties"]["skin"] = {
"type": "string",
"title": "Visual Skin",
"description": "Replace this scoreboard's look with an installed skin "
"(data, scheduling, and vegas mode are unaffected)",
"enum": ["built-in", *choices],
"enumNames": ["Built-in", *(names[sid] for sid in choices)],
"default": "built-in"
}
return enhanced
def _format_validation_error(self, error: ValidationError, plugin_id: Optional[str] = None) -> str:
"""
Format a validation error into a readable message.

Some files were not shown because too many files have changed in this diff Show More