mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-08-01 16:58:06 +00:00
aba04755cfc79730a3f0e4d3a1d6baf0169c2399
1441
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ChuckBuildsandClaude
|
d14381980a |
Stop Vegas mode showing last night's games as if they were live
A game that was live in the evening was still being drawn as live the next morning. Two faults combined to freeze plugin visuals indefinitely. PR #291 added a call to plugin_adapter.invalidate_plugin_scroll_cache() so a plugin's own cached scroll image would be rebuilt from fresh data. That method was never implemented. hot_swap_content() wraps the call in a broad except, so every hot swap has raised AttributeError and been swallowed silently ever since — which is why the visuals it was meant to keep fresh never were. Continuous scrolling then removed the only path that reached it at all: should_recompose() and hot_swap_content() are called from the non-continuous branch of run_frame(), and continuous_scroll defaults to True. So on a default install the pending-update flags were set by the update tick, never consumed, and grew without bound. Together these froze content completely, because refetching is not enough on its own: the sports plugins' get_vegas_content() regenerates only "if the cache is empty", so take_next_group() kept receiving the same picture however often it asked. Fixed by: - Implementing invalidate_plugin_scroll_cache(). It covers both layouts — a helper directly on the plugin (stocks, news, odds-ticker) and one owned by a scroll-display manager (the sports scoreboards, which is the shape that produced this bug) — and clears cached_image and cached_array together, since the array is the image's numpy mirror. - Adding StreamManager.invalidate_pending_updates() and calling it from the continuous branch. It only drops the caches; the plugin recomposes when it next comes round in the rotation. process_updates() is wrong here: it refetches synchronously and merges into the active buffer that continuous mode bypasses, and hot_swap_content() rebuilds and repositions the whole strip, which is the freeze-and-jump this mode exists to avoid. Tests assert the fix rather than the implementation: 14 of the 17 new tests fail without it. Includes the wiring itself, since the regression was a call that was simply absent, and a check that the scroll position is untouched so this cannot regress into the swap's visible jump. All Vegas suites pass (355 tests). test_display_controller_vegas_tick.py still cannot be collected off-device for want of rgbmatrix, identically with and without this change. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
a413849891 |
Add overflow handling: keep ordered content whole instead of rotating a window
The width budget split any oversized plugin by advancing a window each cycle.
That is right for interchangeable items — news headlines, odds, stock prices —
but wrong for ordered content: a league table showed ranks 1-6, then resumed at
7 two rotations later, which reads as out of order and out of context. Nobody
needs rank 23 in a ticker; they need the top of the table, every time.
overflow_mode chooses between them:
rotate — advance a window each cycle so everything is seen eventually
(unchanged default)
truncate — always show the start and drop the rest, keeping ordered content
coherent. Records no window state, so every pass starts at the top.
Per-plugin vegas_overflow overrides the global setting, since one install has
both kinds of plugin. Also adds per-plugin vegas_max_width_screens, so content
that must stay whole can be given more room — or uncapped with 0 — without
lifting the cap on every ticker.
Applied on the test rig: f1-scoreboard and ledmatrix-leaderboard set to
truncate, and baseball given 4.5 screens because it was showing 8 of 9 games
when the whole slate needed only a little more room. Verified: F1 now reports
"the first 10 of 116 ... the rest are not shown", baseball has dropped out of
the budget log entirely, and stocks, odds-ticker and stock-news still rotate.
Also corrects the crop log, which claimed "window advances next cycle"
unconditionally and so misreported truncated crops. A test now pins the
behaviour behind the message: truncate must leave no offset recorded.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
|
||
|
ChuckBuildsandClaude
|
7b6b7cd153 |
Sub-pixel scrolling: motion at the frame rate, not the pixel rate
With integer positioning the number of distinct frames per second equals the scroll speed in px/s, however fast the loop renders. Measured at 50px/s and 78.7fps, 36% of frames were byte-identical: the extra frames cost work and bought no motion, and what was left was 50 discrete 1px steps a second. Two things were wrong with the pre-existing sub-pixel support. get_visible_portion never consulted sub_pixel_scrolling — it always took the integer path, so the flag and _get_visible_portion_subpixel were dead code. And that implementation needed scipy.ndimage.shift, which is not installed on the target devices (HAS_SCIPY is False there), so it would not have interpolated even if reached. Verified both: positions 1000.0 and 1000.5 produced identical frames either way. Blending is now wired up and implemented with numpy. Two details make it affordable: slice cached_array directly instead of building two PIL images only to convert them straight back (the naive version measured 15x the integer path), and use fixed-point uint16 multiply-add rather than float32, which suits the Pi's cores and gives finer weighting than the panel can resolve. Result 0.939ms against 0.237ms — 0.70ms added per frame, a 1065fps ceiling. Measured on hardware: 81.2 fps with blending on, against 78.7 with it off, so no cost within noise — and every frame is now a distinct position rather than one in three being a repeat. The trade is a slight horizontal softening of text, since each frame blends two positions. Set smooth_scroll false for maximum crispness. Also benchmarked and cleared as non-issues: extending the strip costs 9.4ms on an 11,000px strip and trimming 2.5ms, both under one frame at this rate. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
932e89ccb8 |
Pace the Vegas frame loop adaptively: 31.5 -> 78.7 fps
The loop slept a fixed frame_interval on top of however long the frame took, so at a measured 31.6ms per frame a flat 8ms of that was pure idle — a quarter of the budget spent not rendering. It now sleeps only the remainder of the budget. Measured on hardware: 31.5 fps to 78.7 fps sustained, with CPU going *down* from 150% to 127%. Scroll speed is unchanged at 49.9px/s against a configured 50, because motion is derived from elapsed time rather than frame count — this buys smoothness, not speed. Worth recording what the bottleneck was not: the per-frame render path measures 0.34ms in total (0.18ms for the numpy slice, 0.17ms for the dirty-tracking digest), which is a theoretical 2900 fps. Optimising any of that would have been wasted effort. The frame was idle, not busy. Also nices the prefetch thread. Its work is PIL and numpy that releases the GIL, so the scheduler can act on the priority, and without it the prefetch competes for the same cores as the render loop. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
94032e2101 |
Vegas mode: one continuous strip instead of swapping cycles
A cycle used to be a discrete strip that got replaced: motion stopped, every pixel was substituted at once, and the next group started with the viewport already full. That is the freeze, the flash and the jump. The strip is now extended rather than replaced. ScrollHelper gains append_content(), which adds items on the right without touching scroll_position or total_distance_scrolled, so motion continues and the next group simply arrives from the right. Because completion is measured against total_scroll_width, extending also defers completion — there is no longer a cycle boundary to see. drop_scrolled_prefix() reclaims what has gone past, keeping the strip bounded however long Vegas runs (observed 5,000-11,000px against an unbounded strip otherwise). It shifts total_distance_scrolled and total_scroll_width together so the completion arithmetic is unchanged, and refuses to run while the viewport is wrapping: wrapping reads the head of the strip into the right of the frame, so trimming the head there would visibly change the picture. A test caught that. Groups are prepared off the render thread. The constraint is that the canvas and the matrix proxy are process-wide mutable state, so narrowing or capturing through them from another thread would corrupt the frame the render loop is pushing. get_content() therefore takes offscreen_only: the background thread uses only paths that avoid the canvas, and anything needing it is marked and picked up on the render thread. That puts the expensive work (native renders of leaderboard and baseball cards, seconds each) in the background and leaves the cheap work (display capture, 40-600ms) in the foreground. DisplayManager's capture flag is now thread-local. As a shared flag, a background capture would have suppressed the render loop's own frame pushes for its duration, freezing the panel precisely when the point was to avoid a freeze. Canvas-bound plugins are drained one at a time rather than as a batch: six at once held the render thread for 1.75s. Drains are also spaced by two seconds while the lookahead is healthy, since taking them back to back turns one long stall into a run of short ones. When the strip is genuinely running short the throttle is ignored, because content matters more than smoothness there. Measured on hardware: zero cycle-complete swaps, drains landing 2-4s apart, lookahead holding at 1,200-3,500px, no errors. Set continuous_scroll false to restore the swap behaviour; the old path is intact. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
554fb858af |
Hold capture_mode for every plugin render, not just narrowed ones
The native content path only entered capture_mode when it was also narrowing the canvas, so at full width — which is every plugin without a vegas_width_pct override, i.e. most of them — a plugin calling update_display() while building its Vegas content wrote straight to the hardware. That is a visible flash mid-scroll, and it lines up with the flash reported at cycle transitions, when several plugins are fetched back to back. Suppression is now unconditional; the narrowing context stays separate because it is already a no-op at full width. Both contexts are reached through helpers that degrade to nullcontext when the display manager lacks them. That matters more than it looks: the adapter's handlers are deliberately broad, so an AttributeError from a missing context does not surface as an error — it surfaces as the plugin contributing nothing. Making the call unconditional without this turned 44 tests red for exactly that reason, all of them reporting lost content rather than the real cause. The test double now provides capture_mode and render_size too, so tests exercise the real contexts instead of silently taking the degraded path. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
a1c528a091 |
Only cut oversized segments at real gaps between items
The width-budget crop snapped to the nearest blank column, and in rendered text the gap between two characters is a single column. So a cut routinely landed inside a word: the cycle showed "Wednesda" and the orphaned "y" turned up as a lone floating letter in the next cycle, positioned after whatever plugin happened to precede it. Measured on the clock-simple segment to confirm: its blank runs are [1, 1, 1, 1, 1, 8, 8] — five single-column letter gaps, every one of which find_blank_cut would happily have chosen. Cuts now only land in a run of at least min_cut_gap blank columns (default 6), which excludes letter spacing while still finding the gaps plugins put between items (the stocks ticker uses 32px, baseball 48px). Where no boundary falls inside the budget the cut waits for the next one and overruns, because splitting an item is worse than a slightly long segment. Continuous content is treated differently on purpose: an image with no internal gaps is a map or a chart, where any column is as good as another, so it is still cut to the budget exactly. The gap rule protects discrete items; letting a solid image escape the cap in its name would be wrong. blank_runs() is vectorised — 48ms for a 17,000px strip, against seconds for a per-column Python loop. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
105c6df019 |
Fix IndexError in find_blank_cut when the cut lands on the image edge
A cut position after the last column is legitimate — _crop_to_budget asks for min(start + budget, img.width), which equals the width whenever the remaining strip is shorter than the budget. find_blank_cut clamped target to width but then walked leftwards starting at target itself, so ink[width] raised IndexError. Caught on hardware: it killed the ledmatrix-stocks fetch, and because _fetch_plugin_content catches broadly that surfaced as the plugin silently contributing nothing for the cycle. Only reachable on the second or later pass of the rotating window over a single oversized image, which is why the existing tests missed it — they all exercised the first pass, where start is 0 and start + budget is comfortably inside the image. Added TestRotationAcrossMultipleCycles, which walks the window round several times and asserts content is never lost, plus direct coverage of find_blank_cut at and beyond the image edge. Both bounds now stop at width - 1 so neither direction can index past the end. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
62919a13e3 |
Vegas mode: end cycles before the wrap, keep the width budget honest
Three fixes, the first a regression from lead_in_width defaulting to 0. get_visible_portion wraps: once scroll_position + display_width passes the end of the strip it fills the right of the frame from the *head* of the same strip. So the final display_width of travel showed the cycle's first plugin re-entering on the right while its last plugin exited on the left, and the recompose that followed replaced both at once. On a 512px panel at 50px/s that was 10.2s of two plugins on screen at once, ending in a hard cut — reported as the ticker "switching mid-scroll" from F1 to news. That used to be invisible because the strip began with a full display_width of blank, so the wrapped-in region was black. Removing that blank (it was 10s of dead panel per cycle) exposed the wrap. Cycles now end one display width earlier, before any wrapped content appears, clamped for strips no wider than the display so they don't complete instantly and spin the recompose loop. Verified on hardware: a 3936px strip now completes at 68.5s, exactly (3936 - 512) / 50. Second, auto_trim=False also skipped the width budget, which is an unrelated concern — turning off margin cropping should not let one plugin hold the panel for minutes. Seen in the field: the F1 scoreboard contributed 116 images and 14,848px untouched, giving a 33,821px cycle (11 minutes of content). The budget now applies regardless of trimming; with it restored that cycle is 6,362px. Third, the budget accounted for row gaps using the flat intra_plugin_gap while the compositor had moved to measured separation, so it under-counted by up to (min_content_separation - intra_plugin_gap) per row and a many-row plugin overran its cap. Both now use the same separation_gap() rule, and a test asserts the composed block fits the budget end to end rather than trusting the two paths to agree. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
616d21c6d3 |
Vegas mode: render plugins narrower, space rows by measured separation
Trimming reclaims blank margins but cannot compact a layout that genuinely spans the display — a five-column forecast, a progress bar drawn at 100% width, a stat block with the panel's whole width between its elements. Those need the plugin to make different layout decisions, which means telling it the screen is narrower while it renders. DisplayManager.render_size() presents a smaller logical canvas for the duration of a Vegas content fetch, reusing the same _LogicalMatrix indirection double-sided mode already relies on so plugins see a consistent size from every accessor. Plugins that size themselves from matrix.width need no changes at all; one that wants to be explicit can read the new BasePlugin.get_vegas_render_width(). Width is a percentage so a single setting travels across panel sizes: vegas_scroll.render_width_pct globally, or vegas_width_pct in an individual plugin's config. Measured on a 512x64 panel with real data: ledmatrix-weather 1536px -> 576px (forecast becomes narrow cards) youtube-stats 353px -> 199px (2% blank left, so genuinely compact) geochron 453px -> 153px (ink density rises to 100%) ledmatrix-flights 950px -> 740px The youtube-stats figure is the clearest evidence the layout itself changed rather than being cropped: at full width the content had to be trimmed from 512px to 353px, whereas at 40% it arrives with almost no blank to reclaim. Row spacing is now measured rather than added. A flat gap gets it wrong in both directions at once — content drawn flush to its own edges ends up nearly touching (reported for recent sports scores, which sat 8px apart), while content already carrying wide margins gets pushed even further out. separation_gap() measures the blank each pair already has and adds only the shortfall, up to min_content_separation (default 24). intra_plugin_gap stays as a floor applied regardless. Two tests shipped in the previous commit encoded the old flat-gap arithmetic and are updated to the measured semantics, including one renamed to reflect that zero intra_plugin_gap alone no longer butts rows together. Also fixes a real bug found while testing: the harness display manager had no render_size(), and because the adapter catches broadly that surfaced as "no content" rather than an error, silently dropping five plugins. Added the context to VisualTestDisplayManager for parity, and _render_at() now degrades to a no-op on any display manager lacking it, so a third-party or older harness loses the narrowing rather than the content. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
d69dfbbaee |
Align Vegas API bounds with validate(), fix audit config plumbing
Both from review feedback on #423. The web API's accepted ranges disagreed with VegasModeConfig.validate(), which is what actually gates Vegas starting: scroll_speed 1-100 -> 1-200 (a slider value of 150 returned 400) separator_width 0-500 -> 0-128 target_fps 1-200 -> 30-200 buffer_ahead 1-20 -> 1-5 The three loose ones were the dangerous direction: the value saved with a 200, then VegasModeCoordinator.start() failed validation with only a log line, so the ticker silently never ran. The UI already matched validate() in all four cases, so the API was the odd one out. test_vegas_api_bounds_match_validate parses the numeric_fields map out of api_v3 and asserts every bound against validate(), plus that validate() accepts both endpoints and rejects just outside them, so these cannot drift apart again. That test immediately caught a missing upper bound on min_plugin_width, now added — unbounded it would drop every segment and leave a blank ticker. Separately, vegas_audit.py constructed PluginAdapter without the config, so it fell back to VegasModeConfig() defaults and would report trimming and width-budget behaviour that differed from the user's config.json. It now passes the loaded config exactly as the coordinator does. This is the same class of drift the explicit lead_gap and grouping arguments already guard against. Output is unchanged on a rig whose config matches the defaults. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
ChuckBuildsandClaude
|
8d57a748a7 |
Vegas mode: reclaim dead space and pace the rotation
On a wide panel Vegas mode spent much of its time showing black. At 50px/s on a 512px display, one display width of blank is 10.2 seconds, which makes several long-standing behaviours expensive: - ScrollHelper prepended a full display width of black as an "initial gap", charged once per cycle — 10.2s of black at the start of every rotation. - Plugins without get_vegas_content() are captured off a full-display canvas, so their blank margins entered the ticker too. Measured: of-the-day drew 35px of "No Data" on a 512px canvas (92% blank), youtube-stats 142px of content with 185px of black either side. Only the scroll_helper path had any trimming. - Cycle transitions deliberately pushed a blank frame and then recomposed synchronously: 84ms at best, 4.8s at worst, every millisecond of it black. - buffer_ahead doubled as the cycle size, so a 21-plugin install showed 3 plugins per cycle and took ~7 cycles to come around. - separator_width was applied between every image rather than at plugin boundaries, so a per-row ticker like the F1 scoreboard (116 images, which it renders 4px apart internally) got a 32px chasm between each row — and the width budget didn't count those gaps, so the plugin quietly occupied far more of the panel than intended. Changes: - src/vegas_mode/geometry.py: numpy column-ink primitives shared by the trimmer and the audit tool, so the number reported is the number acted on. A Python per-column loop over a 17,000px strip is far too slow for the render path. - PluginAdapter trims every content path, not just scroll_helper. Only outer edges are cropped: interior blank columns are the plugin's own layout (logo left, score right) and closing them would corrupt the design. A plugin on a non-black background is inherently unaffected. - ScrollHelper.create_scrolling_image takes an explicit lead_gap, still defaulting to display_width so the many standalone-ticker callers are unchanged. Vegas passes lead_in_width (default 0). - Cycle end holds the last rendered frame instead of blanking, turning the recompose into a brief freeze rather than the panel switching off. - plugins_per_cycle (default 6) is split from buffer_ahead, which goes back to being only a prefetch low-water mark. - max_plugin_width_ratio (default 3x display width) caps one plugin's share of a cycle. Overflow is deferred, not discarded: a rotation offset advances each fetch so later rows appear on subsequent cycles. Single oversized images are cropped at a blank column so the cut misses glyphs. - Composition groups images by plugin: rows are joined by intra_plugin_gap (default 8) and separator_width applies only between plugins. The width budget now counts those gaps. - Plugin data updates no longer run on the Vegas render path. All new settings are user-configurable in Display -> Vegas Scroll, including min/max cycle duration and dynamic duration, which previously existed in code but were reachable only by hand-editing config.json. Measured with scripts/dev/vegas_audit.py on a 512x64 panel: mean ink coverage 42.7% -> 69.4% fully blank 5.9% -> 0% reads as empty 13.6% -> 0% worst blank stretch 4.8s -> 0s full rotation 414s -> 123s plugins per cycle 3 -> 6 Note the metric choice: a "fully blank" scan (>=95% black viewport) reported only 0.4% and badly understated the problem, because two full-width segments with mid-canvas content never fully blank the viewport — they hold it at ~28%. window_coverage_stats grades every viewport position by how much ink it carries, which is what tracks perceived dead time. Known remaining: cycle transitions still freeze ~3.5s while the next cycle is fetched. Fixing that needs background prefetch, which is deferred because the fallback-capture path mutates the shared display_manager.image and racing it against the render loop risks torn frames. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ |
||
|
|
cdf03fb107 |
Add skin system: user-installable visual overlays for sports scoreboards (#419)
* Add skin system: user-installable visual overlays for sports scoreboards Skins restyle a scoreboard's live/recent/upcoming rendering while the host plugin keeps doing data fetching, scheduling, caching, live priority, and vegas mode — the anti-fork alternative for users who only want a different layout. - src/skin_system/: ScoreboardSkin API, SkinContext (canvas + adaptive layout + logo/font helpers), discovery/loading runtime with API major version gating and per-skin module namespacing - src/base_classes/sports.py: _render_game() seam at the three _draw_scorebug_layout call sites; skin-first with built-in fallback, 3-strikes session disable, slow-render warning; per-mode skin config - scripts/validate_skin.py: headless multi-mode/multi-size validator with bundled per-sport fixtures (no hardware or network needed) - skins/example-classic-baseball/: working reference skin - Web UI: served-schema Visual Skin dropdown (validation never enum-restricted, so uninstalled skins can't invalidate configs) and GET /api/v3/skins - Store: registry entries with type "skin" install to skins/ - docs/SKIN_SYSTEM.md (architecture), docs/CREATING_SKINS.md (author guide incl. Claude Code prompt), view-model contract locked by tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1 * Address review feedback on skin system - skin_runtime: cache the entry module so the 2nd/3rd load of the same skin (live/recent/upcoming hosts) doesn't re-execute it with unbound sibling aliases; rebind cached sibling modules to their bare names around entry import and restore prior bindings after; include per- manifest mtimes in the discovery cache fingerprint so in-place skin updates are picked up - sports.py: count render_skin_card exceptions toward the 3-strike session disable - store_manager: validate skin ids (pattern + resolved-path containment in skins/), reject registry/manifest id mismatches, and stage+validate downloads in a temp sibling before replacing an existing skin - schema_manager: leave the schema untouched when the configured skin value is a per-mode mapping (a string dropdown could overwrite it) - validate_skin.py: reject non-positive sizes and non-object --options at parse time; support --output-dir outside the repo; type annotations - example skin: validate accent_color once at load with logged fallback - fixtures: pregame 0-0 scores in football/hockey upcoming fixtures - api /skins: rely on the self-invalidating discovery cache instead of force_refresh - docs: valid JSON manifest example, load_logo caching semantics spelled out, language ids on fenced blocks - tests: view-model contract test now exercises the real extractor; regression test for repeated same-skin loads with sibling modules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1 --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
6a9d8014e5 |
Tag plugin logs structurally and surface the active plugin in System Logs (#418)
- get_logger() now returns a PluginLoggerAdapter when given a plugin_id,
so every plugin log call is stamped with plugin_id automatically instead
of only calls that explicitly passed extra={'plugin_id': ...}. This makes
the "[Plugin: x]" prefix reliable in the journalctl-backed log stream.
- display_controller publishes the currently active mode/plugin to the
shared cache whenever it changes, exposed via a new
GET /api/v3/display/current-status endpoint.
- System Logs page: adds a "Now showing" banner backed by that endpoint, a
plugin filter dropdown (populated from parsed log lines), a plugin badge
per log entry, and fixes log parsing to handle the short-iso timestamp
format journalctl actually returns (the old regex only matched syslog
timestamps, so level/plugin extraction silently never ran).
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
c90129285c |
Web UI: mobile navigation, guided onboarding, basic/advanced config tiering + performance (#417)
* chore(web): remove dead legacy client-side plugin-config generator (~2,300 lines) Plugin config forms have been rendered server-side (plugin_config.html via GET /partials/plugin-config/<id>) since the HTMX migration; the old client-side generator survived as unreachable code. Verified dead by call graph, not by naming: showPluginConfigModal and showGithubTokenInstructions have zero callers anywhere in templates or JS, and everything removed here is reachable only from those two roots. Removed: - plugins_manager.js: showPluginConfigModal, generatePluginConfigForm, generateFormFromSchema, generateFieldHtml, generateSimpleConfigForm, handlePluginConfigSubmit, the modal's JSON-editor view (initJsonEditor, switchPluginConfigView, syncFormToJson/JsonToForm, saveConfigFromJsonEditor, resetPluginConfigToDefaults, displayValidationErrors, closePluginConfigModal, savePluginConfiguration, currentPluginConfigState), their exclusive helpers (getSchemaPropertyType, escapeCssSelector, dotToNested, collectBooleanFields, normalizeFormDataForConfig, flattenConfig, loadCustomHtmlWidget), the orphaned-modal cleanup block, the modal's listener wiring, and the never-invoked showGithubTokenInstructions/closeInstructionsModal pair. - plugins.html: the #plugin-config-modal markup those functions drove. - base.html: the deprecated pluginConfigData() component and the window.PluginConfigHelpers shim (only ever called by pluginConfigData). Deliberately kept, verified still live: - renderArrayObjectItem, getSchemaProperty, escapeHtml/escapeAttribute (window-exposed for the top-level array-of-objects handlers the server-rendered form uses), toggleNestedSection, addKeyValuePair/ addArrayObjectItem families, executePluginAction, and window.currentPluginConfig = null init (file-upload.js and executePluginAction read it, optional-chained). - app()'s internal generateConfigForm/generateSimpleConfigForm methods in base.html: unreachable now but embedded in the live Alpine component; excising methods from a live object is deferred to keep this change zero-risk. Validation: every deletion seam inspected line-by-line; Jinja parse of both templates passes; repo-wide sweep confirms zero remaining references to any deleted function or element id (deleted ranges contained no Jinja tags). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): mobile navigation drawer + responsive CSS gap fixes Phones previously got the desktop layout squeezed: ~12 system tabs plus one tab per installed plugin wrapped into many rows of small pill buttons, and the header's settings-search and system-stats widgets were dropped entirely (hidden below their breakpoints, never relocated). - Off-canvas nav drawer below md: the existing nav markup (system tab row + #plugin-tabs-row, including dynamically injected plugin tabs) is wrapped in a #site-nav container that CSS repositions into a slide-in drawer on small screens. Same DOM nodes, same @click handlers, nothing duplicated. Tabs become full-width rows with 44px+ touch targets. A hamburger button (md:hidden) in the header and a backdrop toggle the new mobileNavOpen Alpine state (added to both app() definitions, mirroring activeTab). Clicking any tab, a search result, or the backdrop closes the drawer. At md+ hard CSS guards make all drawer styles inert - desktop renders exactly as before. - Header widgets relocated, not hidden: placeHeaderWidgets() in app.js moves the #settings-search-wrap and #system-stats nodes (same elements, listeners intact - both are looked up by id from SSE/search code, so they must never be duplicated) into the drawer below md and back into the header above it, via a matchMedia listener. - Fixed 13 breakpoint utility classes that templates referenced but app.css never defined (sm:block, sm:grid-cols-2, sm:text-sm, md:block, md:w-auto, lg:block, lg:flex, lg:w-64, xl:grid-cols-2/3, 2xl:grid-cols-2/3/4). This was a live bug: 'hidden sm:block' on the search box and 'hidden lg:flex' on the stats meant BOTH were invisible at every screen width. Audit method (repeatable): diff classes used in templates vs defined in app.css. - Mobile modal sizing: one global rule caps .modal-content at 95vw/90vh with internal scroll below 640px - covers every modal without per-template changes. - Horizontal-scroll affordance: pure-CSS edge-fade shadows on .overflow-x-auto containers (scrolling-shadows technique), plus larger in-table touch targets below md. Validation: breakpoint used-vs-defined audit now returns zero gaps; Jinja parse of base.html passes; all changes to desktop behavior are additive (new utilities) or scoped inside max-width media queries. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): x-advanced schema flag groups plugin config fields under a collapsed Advanced Settings section Plugin config pages show every schema property at equal visual priority, which overwhelms first-time users. Plugin authors can now add "x-advanced": true to any flat (non-object) property in config_schema.json to move it into one collapsed "Advanced Settings (N)" section rendered after the basic fields - progressive disclosure with zero loss of control. Implementation: the main render loop in plugin_config.html splits ordered properties into basic/advanced tiers; the advanced group reuses the exact .nested-section/.nested-content/toggleSection() shell that nested object sections already use, so the settings search's expand-on-match behavior works on advanced fields with no JS changes. Object-type properties ignore the flag (they already render as their own collapsible sections). No backend change needed: jsonschema ignores unknown x-* keywords exactly as it does for x-widget/x-propertyOrder. Documented in docs/widget-guide.md alongside the other x-* extensions. Validation (rendered with real Jinja, not just parsed): - synthetic schema with 2 advanced fields: basic fields render before the section, advanced inside the collapsed shell, count badge correct, x-advanced on an object property correctly ignored - schema without any x-advanced: output is identical to the pre-change template (whitespace-normalized diff against git HEAD's version) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): Display Settings basic/advanced split + live total-resolution readout The Hardware Configuration card showed ~17 fields at equal priority; a new user only needs 7 of them to get a correctly-sized, correctly-colored image (rows, cols, chain_length, parallel, brightness, hardware_mapping, led_rgb_sequence). The other 10 (multiplexing, panel_type, row_address_type, gpio_slowdown, rp1_rio, scan_mode, pwm_bits, pwm_dither_bits, pwm_lsb_nanoseconds, limit_refresh_rate_hz) now live in a collapsed "Advanced Hardware Settings" section using the same nested-section shell as plugin config forms, so toggleSection() and settings-search auto-expand work unchanged. led_rgb_sequence moved up beside brightness/hardware_mapping (2-col grid became 3-col). No field was removed or renamed; the form still posts the same names to /api/v3/config/main. Also adds a live "Your display: W x H pixels" readout under the four sizing fields (width = cols x chain_length, height = rows x parallel - the exact math the chain-length tooltip describes in prose), recomputed client-side on every input event, no round-trip. Deviation from plan, deliberate: disable_hardware_pulsing / inverse_colors / show_refresh_rate stay in their separate "Display Options" card rather than moving across cards - relocating fields between form sections risks regressions for no decluttering gain in the card users complained about. Validation (real Jinja render): all 17 hardware fields present exactly once, basic fields render before the advanced section and the 10 advanced fields inside it, div count balanced (71/71), readout + recompute script present. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): plugin install auto-enables + persistent restart nudge Getting a plugin onto the display used to take three disconnected manual steps: install from the store, flip its enable toggle, then restart the display service - with no in-UI hint that steps 2 and 3 were needed (only docs/GETTING_STARTED.md mentions it). - installPlugin() now enables the plugin immediately on successful install (owner-confirmed behavior change: always auto-enable, no opt-out; users who don't want it running toggle it off as before), then shows a persistent toast ("... restart the display to show it") with an inline "Restart Now" button wired to the existing restartDisplay() - the same function the three existing Restart Display buttons call. - notification.js: show() accepts optional { actionLabel, onAction } to render one inline action button per toast. Callbacks are stored per notification id and cleaned up on dismiss; a new triggerAction() public method runs the callback and dismisses. The global showNotification() shorthand now forwards a full options object as its second argument (legacy type-string calls unchanged). Scope note: applies to the plugin store's install path (window.installPlugin). The custom-registry install path keeps its existing behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): dismissible Getting Started checklist on Overview New users land on a dense multi-tab dashboard with no suggested order of operations (the only guided flow is the WiFi captive portal). This adds a non-gating checklist card at the top of Overview with five steps, each a deep link that switches to the right tab (and closes the mobile nav drawer): 1. Set panel size -> Display tab (done: rows/cols/chain_length > 0) 2. Set timezone/location -> General tab (done: differs from template defaults America/New_York / Tampa) 3. Install a plugin -> Plugins tab (done: /api/v3/plugins/installed non-empty) 4. Enable a plugin -> Plugins tab (done: any installed plugin enabled) 5. Configure it -> Plugins tab (done: first enabled plugin has >=1 saved value differing from its schema defaults) Steps 1-2 are computed server-side in Jinja from main_config (already in the partial's context); 3-5 client-side from existing endpoints. No new backend state: dismissal persists in localStorage (mirroring the reconciliation banner's sessionStorage pattern one section up); deep links use the same _x_dataStack app-data access as settings-search.js. Disclosed heuristic limit: values left at legitimate defaults (a user actually in Tampa) read as "not done". Validation: real Jinja render across 3 config variants confirms the server-side done-flags flip correctly; div balance intact; /plugins/config response shape (config dict directly in .data) verified against api_v3.py. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(display): drag-and-drop plugin rotation order for the primary display mode The primary rotation's order was invisible and unconfigurable: modes are registered in parallel-load COMPLETION order, so rotation order actually varied between restarts. Only the niche Vegas Scroll mode had a working order UI. This adds real, persisted ordering end to end: Backend: - config.template.json: new display.plugin_rotation_order (default [], fully backward compatible). - display_controller.py: _apply_plugin_rotation_order() rebuilds available_modes grouped by plugin per the configured list (each plugin's modes keep their declared order; unlisted plugins follow in existing relative order; empty config = exact no-op). Applied at startup after parallel load and after live enable/disable reconcile (before the existing _resync_mode_index_after_change, which preserves the current mode). Mirrors vegas_mode get_ordered_plugins() semantics. - api_v3.py save_main_config: accepts plugin_rotation_order as a JSON array (same parse/guard pattern as vegas_plugin_order). Frontend: - New shared widget static/v3/js/widgets/plugin-order-list.js: the Vegas section's drag-and-drop list factored out verbatim (native HTML5 drag events, saved-order-first rendering, hidden-input JSON sync), parameterized by container/order-input/optional exclude-checkbox/badge. - display.html: Vegas section now calls the shared module; its ~130-line inline copy of the same logic is deleted. - durations.html: new "Rotation Order" card above the durations grid using the same module, posting plugin_rotation_order with the existing form. Deviation from plan, deliberate: durations stay as their own mode-keyed grid rather than inline in the drag rows - verified display_durations keys are MODE names (display_controller.py resolves duration per mode_key), not plugin ids, and one plugin can own several modes, so the planned 1:1 inline pairing was wrong. Validation: py_compile on both Python files; _apply_plugin_rotation_order unit-tested standalone (configured order applied, empty-config no-op, unknown ids skipped - 3/3); both templates render with balanced divs, the hidden input carries the saved order, and the old inline implementation is confirmed gone; config.template.json parses. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): serve the interface at / — /v3 kept as a legacy alias The user-visible URL no longer carries the interface version: the pages blueprint is now registered un-prefixed (primary) AND at /v3 (second registration, name='pages_v3_legacy'), so: - http://<device>/ serves the interface directly (the old @app.route('/') redirect is removed — the blueprint's own index takes its place) - every existing /v3/... bookmark and all the hardcoded /v3/partials/... fetches in templates/JS keep working verbatim through the alias mount — zero template/JS churn, zero broken links - url_for('pages_v3.*') resolves against the primary registration, so all server-side redirects (captive portal detection endpoints) now emit un-prefixed URLs - the AP-mode captive-portal allowlist learned the un-prefixed page paths (/setup, /partials/, /settings/, /plugin-ui/) so setup-mode requests don't redirect-loop - /api/v3 and the templates/v3, static/v3 directories are deliberately untouched (internal, invisible to users; owner-confirmed scope) Validation: dual registration mechanics tested against real Flask (test client): /, /v3, /v3/ redirect, partials and /setup reachable on both mounts, url_for yields un-prefixed paths; py_compile passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): stream the preview PNG raw instead of PIL decode + re-encode display_preview_generator() opened each changed snapshot with PIL and re-encoded it to PNG just to base64 it — but /tmp/led_matrix_preview.png already IS a PNG, written atomically by the display service (tmp file + os.replace in display_manager.py), so a partially-written file can never be observed. Read the bytes and base64 them directly: identical payload (front-end consumes data:image/png;base64 — verified in base.html), one full image decode+encode per frame less on the same Pi that's driving the matrix. The existing mtime skip and viewer-marker throttling are unchanged (they already covered the "skip unchanged frames" concern). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): gzip response compression via flask-compress The interface ships a ~5,000-line HTML shell and >20k lines of JS uncompressed; on phone/WiFi that dominates load time. Flask-Compress gzips/brotlis compressible responses transparently. - Optional dependency, same graceful pattern as flask-limiter: missing package = uncompressed responses, no crash. - SSE safety verified empirically against the real package (1.24): an actual streamed text/event-stream response comes back with no Content-Encoding while a large HTML response gzips — the display preview / stats / logs streams are unaffected. - Added flask-compress>=1.14 to web_interface/requirements.txt. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): gate verbose console logging behind the existing pluginDebug switch plugins_manager.js, base.html's inline scripts, and app.js emitted 198 console.log calls in production - including per-interaction [DEBUG] dumps - costing main-thread time and drowning real errors in noise. - New window.debugLog() gate defined in base.html's first inline script (before any other script runs): forwards to console.log only when localStorage.pluginDebug === 'true' - the SAME switch plugins_manager.js already used for its _PLUGIN_DEBUG_EARLY logs, so existing debug workflow docs stay valid. Exposed as window.LEDMATRIX_DEBUG for other scripts. - Mechanically rewrote console.log( -> debugLog( in plugins_manager.js (127), base.html (64), app.js (7). Verified no occurrences lived inside string literals before rewriting; console.error/console.warn untouched. - app.js's no-Alpine showNotification fallback restored to console.info - it's a user-facing last resort, not debug output. Both load paths are safe: the gate is the first inline <script> in <head>, and every rewritten file loads deferred after it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): vendor CDN assets locally — LAN-speed loads, fully offline-capable Font Awesome, CodeMirror, and htmx were fetched from cdnjs/unpkg on every fresh page load, adding third-party round-trips on a device that often lives on a local network (and htmx was local-only in AP mode, meaning two different loading behaviors to reason about). - Vendored pinned copies under static/v3/vendor/: Font Awesome 6.0.0 (css/all.min.css + the 8 webfonts it references relatively) and CodeMirror 5.65.2 (core, javascript mode, closebrackets/matchbrackets addons, base + monokai css) - ~1.1 MB total, exact versions the CDN tags pinned. - htmx + sse + json-enc extensions now load from the existing local copies (verified 1.9.10, matching the CDN pin) on EVERY network, not just AP mode; the pinned CDN copies remain as a one-shot rescue fallback, mirroring the pattern Alpine already used. The convoluted isAPMode source-flipping logic collapses away. - Dropped the CDN preconnect/dns-prefetch hints (no longer on the critical path). - Fixed a latent bug while relinking CodeMirror: the loader requested mode/json/json.min.js, which does not exist on cdnjs (HTTP 404 verified) - it 404'd on every JSON-editor open. JSON highlighting comes from the javascript mode; the phantom entry is removed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): extract 3,850 lines of inline JS from base.html to cacheable static files base.html shipped ~4,200 lines of inline JavaScript inside the HTML document, re-downloaded and re-parsed on every page load (gzip helps the transfer, but inline scripts can never be browser-cached). The four largest blocks - none containing any Jinja syntax, verified by scanning every inline block for {{ }} / {% %} - now live as static files served with the app's existing mtime-versioned immutable caching: - js/htmx-config.js (246 lines): HTMX swap/script-execution config, toggleSection helpers - js/app-early.js (346 lines): early helpers + the app() stub that must precede Alpine init - js/app-shell.js (2,997 lines): SSE wiring + the full Alpine app() implementation and tab logic - js/custom-feeds-helpers.js (262 lines): custom-feeds table helpers Each replacement <script src> is CLASSIC (no defer/async) at the exact position of the inline block it replaces - identical execution timing and DOM visibility to inline scripts, so relative ordering with the deferred scripts and with each other is unchanged. base.html drops from ~4,940 to 1,079 lines. Validation: extraction proven lossless by programmatically reassembling the four files back into the template and comparing against git HEAD - byte-for-byte identical. Jinja parse passes; script open/close tags balanced (53/53, after excluding a literal "<script>" inside an HTML comment). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): size the live preview from the PNG's real dimensions, not config The initial SSE render sized the preview image (and both overlay canvases) from the server-reported config dimensions (cols x chain_length, rows x parallel), while the scale slider's re-render path sized from img.naturalWidth/naturalHeight. Whenever the snapshot PNG's actual size disagrees with the config (stale config, display service not restarted after a hardware change), the initial render stretched the image at a fractional ratio - blurry despite image-rendering: pixelated - and touching the scale slider "fixed" it. Reported live on the devpi test rig. Both paths now size from the loaded image's natural dimensions inside img.onload (which also removes a transient wrong-size flash between src assignment and load). The meta label now reports the true snapshot size. The preview card also gets overflow-x-auto so on narrow screens a wide preview scrolls at its exact pixel-perfect size instead of being squeezed into the viewport (fractional downscaling of pixel art also reads as blur). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): fold Display Options into the advanced dropdown; Vegas above Double-Sided Owner-requested layout refinement of the Display Settings tab: - The "Display Options" card (disable_hardware_pulsing, inverse_colors, show_refresh_rate, use_short_date_format, Dynamic Duration) moves inside the collapsed advanced section, now titled "Advanced Hardware & Display Options (15)". Hidden form fields still submit with the form, and settings search still auto-expands the section on match, so nothing is lost - the tab just leads with the essentials. - The "Vegas Scroll Mode" section moves above "Double-Sided Display". New section order: Hardware (+ advanced dropdown) > Vegas Scroll > Double-Sided > Multi-Display Sync. Validation (real Jinja render): all 23 field names present exactly once, divs balanced (70/70), the five Display Options fields render inside the advanced section's bounds, and section markers confirm the new order. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): Getting Started items are manually checkable; card auto-hides when complete Two gaps reported from live testing on the devpi rig: 1. The timezone/location step never showed done for a user whose real timezone IS the shipped default (America/New_York) - the heuristic can only detect difference-from-default, not "user saved this". Clicking an item's checkbox now toggles it done manually (persisted per browser in localStorage), so any heuristic false-negative is one tap to clear. Clicking the item text still deep-links to its tab. 2. The card now hides itself automatically once every step is done (auto-detected or manually checked) - previously it stayed until the X was clicked. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * chore(web): CI cleanup — declare debugLog global, fix entity-unescape order, drop v3 from UI branding - Add debugLog to the /* global */ headers of the six JS files that call it (defined in base.html's first inline script) — resolves the wall of "'debugLog' is not defined" ESLint errors failing the Codacy check. - Fix the two js/double-escaping CodeQL alerts in app-shell.js: the entity-unescape chains decoded & before </>, so a value containing a pre-escaped "&lt;" wrongly double-decoded to "<". & now decodes last (standard order). Pre-existing bug, made visible when the inline scripts moved into scannable .js files. - Page title / header drop the "- v3" suffix, matching the de-versioned user-facing URL. The remaining 7 CodeQL alerts are pre-existing patterns newly visible to scanning (CodeQL doesn't see inline template JS): 4 github.com/htmx.org URL-substring checks (the htmx ones match error-message text, not URLs — false positives in context) and 1 innerHTML XSS-through-DOM in the GitHub install flow. Triage/fix deferred to a focused follow-up rather than expanding this PR. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): add the missing nav tab for the Rotation & Durations page The durations partial (/partials/durations) has existed as a route with no nav tab and no content panel referencing it - an orphaned page. That made the new rotation-order UI unreachable through the interface (caught by the owner testing on the rig; my endpoint-level tests fetched the partial by URL and never noticed the missing entry point). - New "Rotation" tab (fa-rotate icon, verified present in the vendored FA 6.0.0 css) between Display and Backup & Restore, wired exactly like the other tabs (#durations-content + hx-get + loadtab; loadTabContent() is fully generic, so no JS changes needed). - Page heading updated from "Display Durations" to "Rotation & Durations" to match its content since the rotation-order card landed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): Rotation & Durations page lists every enabled plugin's screens The durations grid looped over display.display_durations, which nothing has ever populated (verified {} on a real production install) - so the page rendered no duration fields at all. Worse, its inputs posted bare mode names, which save_main_config's endswith('_duration') filter silently dropped: the page was broken in both directions, unnoticed because it was also unreachable (previous commit). - pages_v3._load_durations_partial now builds one entry per display mode of every ENABLED plugin via plugin_manager.get_plugin_display_modes() (falling back to the plugin id), overlaid with saved values, defaulting to the display controller's 30s. Grouped per plugin, sorted by name. Saved keys not owned by any enabled plugin stay visible under "Other saved entries" instead of vanishing. - durations.html renders the grouped inputs, named duration__<mode_key> (mode keys are arbitrary, so they can't use the *_duration suffix convention), with an explanatory empty state when no plugins are enabled. - api_v3.save_main_config accepts the new duration__<mode> fields and writes them into display.display_durations under the bare mode key - exactly what the display controller reads (display_durations.get(mode_key, 30)). Validation: py_compile both blueprints; Jinja render with 3 groups asserts grouped inputs, saved-value overlay, stale-entry group, empty state, and div balance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): restart-pending banner, unsaved-changes guard, installable web app Three usability improvements from live testing feedback: - Restart-pending banner: every successful POST to /api/v3/config/main (display hardware, rotation/durations, general settings) now raises a persistent banner - "Configuration saved, restart the display to apply" - with a Restart Now button that posts restart_display_service directly. Backed by sessionStorage so it survives tab switches and reloads until restarted or dismissed. Plugin config saves are deliberately excluded: they apply live via the display process's config watcher. - Unsaved-changes guard: plugin config panels are Alpine x-if templates, so navigating away destroys the panel and revisiting re-fetches it - edits were silently discarded. Forms now mark themselves dirty on input (cleared on successful submit), a capture-phase click handler confirms before a lossy tab switch, and beforeunload guards full page unloads. System tabs (x-show, persistent DOM) are exempt - no false prompts. - Installable web app: manifest.json (standalone display, dark theme) + generated LED-grid icons (192/512 maskable + 180 apple-touch), linked from base.html. "Add to Home Screen" now yields an app-like fullscreen experience; no service worker, so zero behavioral risk. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * test(web): smoke tests + static-analysis audits for the web UI Guardrails so this branch's fix classes can't regress silently: - test_web_smoke.py (24 tests): boots the pages blueprint with the same dual registration app.py uses and asserts every page/partial returns 200 with its load-bearing markers (nav wiring, getting-started card, advanced section, rotation order card, per-mode duration inputs), the /v3 legacy alias serves everything, all critical static assets (incl. vendored fontawesome/codemirror, PWA manifest/icons) are served, durations group per plugin with the leftover bucket, and the advanced-hardware section really contains the tuning fields. Would have caught this session's unreachable-durations-page and orphaned-tab bugs instantly. - test_web_static_audit.py (3 tests): (1) every responsive utility class referenced in templates is actually defined in app.css - the silently-no-op class bug that left the header search box invisible at every width; (2) every url_for('static', ...) reference points to a real file; (3) any JS file calling the debugLog global declares it in a /* global */ header. All 40 web tests pass (24 + 3 new, 13 existing) under pytest + Flask. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): floating live preview + per-plugin "Preview on display", drawer a11y Preview-while-configuring: - Floating mini preview (fixed, bottom-right) available on every tab except Overview, fed from the same SSE display stream by updateDisplayPreview - no new connections. Collapses to a round toggle button; open/closed state persists in localStorage; hides on Overview where the full preview lives. - "Preview on display" button on every plugin config page header: runs that plugin on the real display for 60 seconds via the existing /display/on-demand/start API and opens the floating preview, closing the configure -> see-the-result loop. Drawer/nav accessibility: - aria-current="page" tracks the active tab (system + dynamic plugin tabs, matched via their Alpine @click expression), updated from the activeTab watcher so search deep-links and checklist navigation are covered too. - Escape closes the mobile drawer and returns focus to the hamburger; opening the drawer moves focus to its first tab. Validation: all 40 web tests pass; Jinja parse + div balance on both touched templates. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): resolve Codacy findings — DOM building over innerHTML, Map callbacks, misc lint Verified each of the 27 reported findings against current code; all fixed except one rule class skipped with reason below. - app-early.js: plugin tab buttons are now built with createElement/ createTextNode instead of innerHTML template strings (icon class and name come from plugin manifests - semi-trusted input; the old code escaped the name but interpolated the icon class). Both construction sites. Also the forEach arrow no longer returns tab.remove()'s value. - plugin-order-list.js: rows, empty state, and error state all built with DOM APIs - the file no longer contains innerHTML at all (the now-unneeded escapeHtml/escapeAttr helpers are removed); MODE_LABELS is a Map so the vegas-mode lookup can't hit prototype properties. - notification.js: actionCallbacks is a Map (get/set/delete) instead of a plain object - resolves the object-injection-sink and dynamic-delete findings; triggerAction also type-checks the callback. - htmx-config.js: unused catch binding dropped; var -> const in the afterSettle handler; the swapped-<script> re-execution reads/writes textContent instead of innerHTML; the diagnostic form payload uses a null-prototype object so a field named __proto__ can't pollute. - custom-feeds-helpers.js DELETED (with its script tag): all three of its functions (addCustomFeedRow, removeCustomFeedRow, handleCustomFeedLogoUpload) are shadowed by the deferred widgets/custom-feeds.js window assignments, which always win at call time - the copies were dead even when they lived inline in base.html. This also resolves the unused-function and unused-variable findings there. Skipped: 4x "Non-serializable expression must be wrapped with $(...)" in app-early.js - that rule targets code crossing a browser-automation serialization boundary (e.g. page.evaluate); these are ordinary arrow functions in plain browser code with no such boundary. Validation: all 40 web tests pass (incl. the static-asset reference audit, which confirms no template still points at the deleted file); Jinja parse OK. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): update flow installs changed Python dependencies automatically The in-app updater (Update Now banner -> git_pull action) stashed, pulled, and purged plugins - but never touched Python dependencies. Any release adding a package (e.g. this branch's flask-compress, which lives in web_interface/requirements.txt) silently required an SSH session and a manual pip install that most users will never do. - git_pull now records HEAD before pulling; after a successful pull it diffs old..new and, if requirements.txt or web_interface/requirements.txt changed, installs exactly those via _pip_install_requirements - the same vetted root-visible sudo path the Tools-tab buttons use (with its existing graceful fallback when the sudo wrapper isn't configured). Results are appended to the update toast; a failure points the user at the Tools-tab button instead of failing the whole update. - install_base_requirements (Tools tab) now also installs web_interface/requirements.txt - previously it only covered the root file, so web-only dependencies were unreachable from the UI entirely. No install happens when the pull was already-up-to-date or when no requirements file changed, so routine updates stay fast. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): address CodeRabbit review — validation, a11y, perf, and privacy fixes Verified each finding against current code. Fixed: - api_v3: plugin_rotation_order is now strictly validated (JSON list of strings, 400 with a descriptive message otherwise) and popped from the payload before any further handling. - display_controller: _apply_plugin_rotation_order defensively ignores a non-list value (keeps the existing rotation, logs a warning) and drops non-string entries; new logs carry the [DisplayController] prefix. Unit-tested both defensive paths. - app.py: snapshot-read handler narrowed to OSError with debug logging; flask-compress ImportError now emits one structured warning with the install remedy. - htmx-config: the response-error logger prints form FIELD NAMES only - values (API keys, passwords) never reach the console. - plugin-order-list: saved order/exclusions normalized with Array.isArray (a saved "null" previously crashed .forEach); each row gained keyboard/touch-accessible move-up/move-down buttons (HTML5 drag events don't fire on most mobile browsers) that reorder and syncInputs() immediately alongside native drag. - app-shell: window.installedPlugins setter always takes the new list (same-ID metadata/enabled updates were silently dropped); tab rebuild stays gated on ID changes. LED dot renderer reads the frame with ONE getImageData call instead of one per pixel (~9,200/frame at 192x48). - plugins_manager: togglePlugin returns its request promise resolving the API outcome; the install flow now shows the "installed and enabled" toast (with Restart Now) only after enablement succeeds, and a warning without a restart offer when it fails. - a11y: hamburger aria-label flips Open/Close with drawer state; both Advanced-section toggle buttons declare aria-controls/aria-expanded and the shared toggleSection() keeps aria-expanded in sync; move buttons have per-plugin aria-labels. - Rotation/Vegas order-list bootstraps cap their retries (~5s) and show a reload hint instead of spinning forever; Alpine app-state lookups prefer [x-data="app()"] with a generic fallback. Skipped, with reasons: - executePluginAction arg order: caller (plugin_config.html) already passes (actionId, index, pluginId) matching the signature exactly. - generateFieldHtml XSS, entity-unescape blocks, dotToNested pollution, and "app.loadInstalledPlugins" in app-shell: all inside the legacy client-side config cluster whose entry points are shadowed by plugins_manager.js / replaced by server-rendered forms (zero live callers, verified) - queued for wholesale deletion in the follow-up rather than patching dead code. - custom-feeds-helpers.js findings (3): file was deleted in a prior commit. - console.error/warn override removal and afterSwap script re-execution removal: deliberate pre-existing workarounds every partial's inline init currently depends on; reworking them safely needs isolated testing (follow-up), and the error suppression is already double-gated (insertBefore AND htmx match). - "move durations bootstrap into a bundle": inline partial-scoped init is the established pattern for HTMX partials in this codebase. Validation: all 40 web tests pass; py_compile on all touched Python; all touched templates parse; rotation-order defensive paths unit-tested. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * chore(web): fix remaining real Codacy findings (2 of 6) - htmx-config.js: two more unused catch bindings dropped (optional catch binding), matching the earlier fix. - app-early.js: second forEach arrow (the stub updatePluginTabs copy) braced so the callback no longer returns tab.remove()'s value. The other 4 findings ("Non-serializable expression must be wrapped with $(...)") are deliberately NOT "fixed": that rule belongs to a browser-automation (WebdriverIO-style) lint context and is misfiring on ordinary arrow-function constants. Converting them to function declarations would look compliant but BREAK the code - all four arrows intentionally capture the enclosing Alpine component's `this` for the stub-to-full enhancement logic. The right remedy is disabling that pattern for this repo in Codacy's Code Patterns settings (or dismissing the four findings), not a code change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): floating preview shows a frame immediately on open + is resizable The floating preview opened empty and stayed empty until the display next CHANGED - the SSE stream only pushes frames on change, and the panel only consumed frames while already open, so the connection's initial frame (sent while the panel was closed) was dropped. Reported from mobile testing as "the button doesn't work". - updateDisplayPreview now caches the latest frame globally regardless of panel state; opening the panel populates the image from that cache instantly, then live frames take over. - Resizable: a size button cycles 192/256/384/512px presets (persisted per browser; works on touch), and desktop additionally gets a native drag handle (CSS resize: both). The image is fluid within the panel; on phones the panel is capped to the viewport width. The size icon (fa-up-right-and-down-left-from-center) is verified present in the vendored FA 6.0.0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): stop duration fields leaking into config root; isolate per-file dependency installs; fix test fixture leak Verified each finding against current code. - api_v3 save_main_config: both duration blocks (*_duration suffix fields and the newer duration__<mode> fields) only READ from `data`, never removed the keys. The generic "remaining keys" merge later in the same function has no skip-list entry for either pattern, so every duration field was ALSO written a second time as a bogus top-level config key (e.g. "clock_duration": 30 and "duration__mlb_live": 42 sitting at config root, alongside the correct nested display.display_durations.<key>). Confirmed by tracing the full function. Fixed by popping each handled key from `data` (same pattern already used for plugin_rotation_order) and validating strictly: a non-integer duration now returns 400 with a message naming the offending field/mode instead of silently logging and moving on (for the *_duration fields, which previously had zero validation at all). - api_v3 dependency-install loops (git_pull's post-update sync and install_base_requirements): _pip_install_requirements can raise subprocess.TimeoutExpired or OSError (confirmed: install_requirements_file in permission_utils.py never catches either internally, despite its docstring's "never raises on non-zero exit" only covering return codes). Both loops previously let one file's exception either abort the whole try block (skipping the second requirements file entirely) or propagate uncaught. Each file's install is now in its own try/except, so a timeout or OSError on one file is recorded as a labeled failure and the loop continues to the next file. - test_web_smoke.py: the `client` fixture mutated the module-level pages_v3 Blueprint singleton's config_manager/plugin_manager directly with no teardown - since pages_v3 is shared across the whole pytest process (test_web_settings_ui.py touches the same attributes), this fixture's mocks could leak into whichever test ran next. Now saves the originals, yields the client, and restores them in a finally block. Validation: py_compile passes; all 40 web tests pass with the now-generator fixture. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): repair garbled Advanced Hardware section description An earlier sed-based text update concatenated the old and new copies of this description instead of replacing one with the other, leaving a duplicated sentence with the — entity broken into ".mdash;" (visible as literal "mdash;" text on the page). Restored to one clean sentence. Other findings from this review were already fixed in a prior commit (installedPlugins setter) or are confirmed dead code with zero live callers (executePluginAction/dotToNested/entity-unescape/generateFieldHtml, all reachable only from the two unused savePluginConfig copies in app-shell.js - grepped every template, no references) - same legacy cluster flagged in earlier review passes, still queued for a dedicated deletion follow-up rather than patched in place here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): remove redundant htmx:afterSwap script re-execution (was double-executing every partial's inline script) Re-verified this CodeRabbit finding, previously deferred as "needs isolated testing" - traced it to a confirmed, active bug rather than a style concern: htmx 1.9.10's own config defaults to allowScriptTags: true (confirmed in the vendored htmx.min.js, which itself contains the same clone-and-reinsert <script> mechanism internally). This means htmx ALREADY re-executes every <script> tag in swapped content by default, exactly like a browser navigating to a new page. The custom htmx:afterSwap listener in htmx-config.js did the identical clone-and-reinsert a SECOND time on top of htmx's own handling - so every inline <script> block in every HTMX-loaded partial (overview, display, durations, plugin config, etc. - most partials have one) executed twice per load. Confirmed safe to delete outright, not just narrow: grepped every hand-written JS file for a manual `dispatchEvent(... 'htmx:afterSwap' ...)` that might have relied on this handler for a non-htmx code path (e.g. the direct-fetch fallbacks like loadOverviewDirect) - none exists, so nothing depended on this listener specifically; htmx's native handling covers every real htmx-driven swap on its own. Left in place, unchanged: the console.error/console.warn global override a few lines up in the same file, which suppresses known-noisy HTMX-timing-race messages. That one is a legitimate anti-pattern too (broad substring matching can mask unrelated errors) but redesigning it needs care to preserve real diagnostics while still hiding the specific harmless races it targets - a scoped follow-up, not a same-day deletion like this confirmed-duplicate handler. Validation: all 27 fast web tests pass; JS brace/paren balance sanity checked (no local Node/browser available in this sandbox to execute the file directly - verify manually in-browser before merge). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * chore(display): add missing [DisplayController] prefix to the reconcile-complete log Re-verifying the full CodeRabbit findings list against current code surfaced one still-open item: the nitpick asked for the prefix on BOTH rotation-related log lines, but only "Applied plugin rotation order" got it in the earlier pass - "Plugin reconcile complete" was missed. No message/argument/level change, matching the finding's own scope. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): repair dead /v3/logs link on the display hardware-error banner The "Logs tab" link in the display-settings simulation-mode banner was a real <a href> to /v3/logs, but no such route has ever existed (log content is loaded client-side via activeTab, not a dedicated page route) - the link 404'd regardless of the /v3 prefix change. Switch it to the same activeTab-switching pattern the real nav uses. * fix(web): raise display Rows field max from 64 to 128 Cols already allowed up to 128; Rows was capped at 64, which rejects valid larger panel configurations (e.g. 128-row tile chains). No server-side schema enforces a rows max, so this was purely an overly-strict HTML input attribute. * fix(web): remove redundant htmx.org substring check flagged by CodeQL CodeQL flags .includes('htmx.org') as "incomplete URL substring sanitization" - a false positive here, since this string is only ever matched against console.error/warn message text to decide whether to suppress a known-harmless HTMX timing-race log line, not used for any URL-trust/redirect decision. The check was also redundant: 'htmx' is already a substring of 'htmx.org', so the plain .includes('htmx') check right next to it already covers every case the removed check did. * fix(web): restore htmx script re-execution timing that Alpine x-data depends on Removing the custom htmx:afterSwap script-reexecution handler (in a prior commit, as a "duplicate execution" cleanup) broke every partial whose Alpine x-data component function is defined by an inline <script> in that same partial (e.g. wifi.html's wifiSetup()) - confirmed live via browser console: "Alpine Expression Error: wifiSetup is not defined" on every field in the WiFi tab. Root cause: htmx's own native script execution runs during its "settle" phase (~20ms after swap, per htmx's own defaultSettleDelay), but Alpine's MutationObserver evaluates x-data on newly-inserted elements synchronously, right as the swap lands - before settle. So the inline <script> defining wifiSetup() was still un-run when Alpine tried to call it, and Alpine does not retry a failed x-data evaluation later once the function does become defined. Fix: re-execute swapped <script> tags ourselves on htmx:afterSwap (which fires synchronously, before settle, beating Alpine's observer), and disable htmx's own native script re-execution (htmx.config.allowScriptTags = false) so the same script doesn't also run a second time during settle - restoring correct timing without reintroducing the original double-execution bug. Also in this commit: - fix XSS: unescaped repoUrl in a title attribute in renderSavedRepositories - replace .includes('github.com') substring checks with real URL hostname validation (CodeQL: incomplete URL substring sanitization) * fix(web): wait for async plugin install to finish before auto-enabling it Confirmed live: installing hockey-scoreboard logged "installation queued" (success) immediately followed by "enabling it failed" with a 404 "Plugin not found" from /api/v3/plugins/toggle. /api/v3/plugins/install runs the actual clone + plugin-manager discovery asynchronously via an operation queue when one is configured - the response installPlugin() was checking only means the operation was queued, not that the plugin is installed yet. Calling togglePlugin() right after that response 404s because plugin_manager hasn't discovered the new plugin. Fix: reuse the same operation-polling mechanism uninstallPlugin() already has (generalized pollOperationStatus() to take onComplete/onFailed/onTimeout callbacks instead of hardcoding uninstall behavior) so installPlugin() waits for the operation to actually complete before enabling it. Falls back to enabling immediately when no operation_id is returned (direct/synchronous install path, no queue configured). * fix(web): resolve remaining valid findings from latest review pass - custom-feeds.js: fix asset-upload contract mismatch (field name "file" -> "files", response read from top-level "uploaded_files" not "data.files") - same bug already fixed in this file on a separate branch/PR (#420), which this branch never received since they're independent PRs off main - custom-feeds.js: add aria-label to the two icon-only "remove feed" buttons - custom-feeds.js: move file-input reset into .finally() so a failed upload doesn't leave the input stuck holding the file, blocking retry of the same file - app-shell.js: fix executePluginAction(pluginId, actionId) parameter order/count mismatch vs. its callers' (actionId, index, pluginId) - currently masked by plugins_manager.js's correct version overwriting this one at load time (classic vs. deferred script order), but worth fixing outright since it's an isolated, self-contained reassignment (not inside the Alpine app() object literal) and removes a latent footgun - overview.html: align Alpine-state resolution with settings-search.js's two-tier getAppData() (also check appEl.__x.$data, not just _x_dataStack) Verified already addressed by earlier passes (no change needed): plugin_rotation_order validation, DisplayController log prefixes, togglePlugin returning its promise for install-flow chaining, installedPlugins setter always updating state, mobile-nav aria-label, toggleSection aria-expanded sync, PluginOrderList bounded init retries (both display.html and durations.html), plugin-order-list.js Array.isArray validation, batched getImageData in the LED-dot preview renderer, app.py exception narrowing/ logging, form-submission log redaction. Confirmed dead code, skipped (unreachable - zero template/JS callers, verified via full-repo grep): dotToNested prototype-pollution hardening, generateFieldHtml HTML-injection hardening, and the HTML-entity-unescape block in JSON parsing - all three live only inside app-shell.js's two legacy savePluginConfig implementations (one Alpine-method, one standalone), neither of which any template or script calls. The real, live plugin-config path is server-rendered via GET /partials/plugin-config/<id>. Explicitly NOT reverted: the htmx:afterSwap script-execution listener. An earlier finding batch asked to remove it as "duplicate" htmx behavior; that was tried and reverted this session after live testing on hardware proved it broke every partial whose Alpine x-data depends on an inline <script> in the same partial (confirmed: WiFi tab hard-failed with "wifiSetup is not defined"). Removing it again would reintroduce that regression. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
4abcd0e4f9 |
Fix PermissionError reading config_secrets.json in web interface (#416)
ledmatrix.service (main display) runs as root while ledmatrix-web.service runs as the non-root install user (install_web_service.sh). Both config_manager.py and config_manager_atomic.py only chmod'd config_secrets.json to 0o640 without ever fixing its group, so a file written by the root service ended up group-owned by root and unreadable by the web user, crashing the settings page with a raw PermissionError. Add ensure_shared_group_ownership() to chgrp secrets/config files (best effort, root-only) to the project directory's owning group whenever they are created or saved, and self-heal existing files on load. Also make get_raw_file_content() tolerate an unreadable secrets file the same way load_config() already does, degrading to empty secrets instead of a 500. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
2a1c47fa76 |
chore: remove dead modules and unused dependencies (~1,180 LOC) (#412)
* chore: remove dead modules and unused dependencies (~1,180 LOC) Deletions, each re-verified with a fresh repo-wide grep (core, web, scripts, docs, plugin monorepo) immediately before removal: Modules with zero live importers: - src/background_cache_mixin.py + src/generic_cache_mixin.py (134+150 LOC — referenced only by each other) - src/font_test_manager.py (134 LOC) - src/image_utils.py (22 LOC, self-documented deprecated) - src/layout_manager.py (408 LOC — only its own test imported it) + test/test_layout_manager.py - src/common/basketball_plugin_example.py (328 LOC sample) requirements.txt entries with zero importers in core (pre-plugin-era manager deps): icalevents, geopy, timezonefinder, unidecode. Plus the google-auth trio (google-auth-oauthlib, google-auth-httplib2, google-api-python-client): their only importer is the calendar PLUGIN, which declares all three in its own requirements.txt (verified in the monorepo and on an installed copy) — the plugin dependency installer owns them. Existing venvs are unaffected (removal doesn't uninstall); fresh installs get them when calendar is installed. Two stale references cleaned (a comment in test_pillow_compat.py, a directory listing in HOW_TO_RUN_TESTS.md). Full suite green except the two documented pre-existing failures (circuit_breaker mock drift, fixed in #400; clock-simple 64x32 overflow, pre-dates this series); all core entry modules verified importing cleanly under the emulator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix: remove content accidentally bundled into the dead-code-removal commit The previous commit's git add/commit swept in a lot of unrelated, unreviewed work alongside the intended dead-code deletions: a new Plugin Composer web UI, new security-audit/CI tooling, a new march-madness plugin, and 23 local-development-only symlinks under plugin-repos/ (per scripts/setup_plugin_repos.py's own docstring, these are meant to be generated locally, never committed -- .gitignore has no entry for them, which is how they slipped in). Removed here, split into their own PRs instead (except plugin-repos/* symlinks and march-madness/ncaa_logos, which are dropped rather than carried forward -- see PR discussion): - web_interface/blueprints/composer.py + composer-app.js + composer-canvas.js + composer.html + manager.py.j2 - scripts/prove_security.py, audit_plugins.py, generate_report.py - .github/workflows/security-audit.yml, .github/workflows/tests.yml, bandit.yaml - All plugin-repos/* symlinks (local dev artifacts, not meant to be committed at all) - plugin-repos/march-madness/* and the 4 new assets/sports/ncaa_logos/* PNGs it needed (left out of every split PR pending a decision on whether march-madness belongs in the core repo or the plugin monorepo) This PR now contains only what its title describes: the dead-code removal from the previous commit, untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
14a59c863c |
perf(wifi): one status fetch per monitor tick instead of three (#411)
* perf(wifi): one status fetch per monitor tick instead of three The wifi monitor daemon fetched WiFi status + ethernet state before its AP-mode check, the check internally fetched the same state again (with retry), and the daemon fetched a third time afterwards — each fetch is several nmcli subprocess forks, every 30s, forever, even on a perfectly healthy link. check_and_manage_ap_mode's decision logic is extracted to _manage_ap_mode(status, ethernet, ap_active); the new check_and_manage_ap_mode_with_state() runs the single (retrying) fetch battery and returns (changed, status, ethernet, ap_active_after) — the post-state is derivable because state only ever flips via one enable or one disable. The original bool-returning method delegates, so existing callers are untouched. The daemon's dead pre-fetch is removed and its post-check reads use the returned state; the AP-enable retry semantics (_get_wifi_status_with_retry) are preserved exactly. ~10-15 forks/30s drops to ~4-6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(wifi): add missing type hints on AP-mode state helpers, fix unused-var lint in tests - check_and_manage_ap_mode_with_state now declares its Tuple[bool, WiFiStatus, bool, bool] return type instead of being untyped. - _manage_ap_mode's status/ethernet_connected/ap_active parameters are now typed (WiFiStatus, bool, bool), matching its existing -> bool return hint. - test_wifi_check_state.py: rename unused unpacked variables (status/ ethernet/ap_after) to underscore-prefixed names at the three call sites that don't use them, leaving the used ones untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
bff13129c4 |
perf(config): mtime-signature fast path for load_config (#410)
load_config re-read and re-parsed config.json, the secrets file, AND the template (running the recursive migration diff) on every call — with ~30 call sites in web request handlers, some hit 2-3x per request. Fast path: stat all three files (mtime_ns + size); when unchanged since the last successful load, return the already-parsed self.config (same aliasing semantics as before). The signature is taken AFTER load + migration so a migration write-back doesn't retrigger, and both save paths refresh it. Cross-process freshness is preserved by construction: a save from the other process bumps the file mtime, so the next load here re-reads — verified by a dedicated test. Same-second edits are caught by mtime_ns plus a size check. Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6499794c12 |
fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache (#409)
* fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache ledmatrix-leaderboard's data_fetcher.py calls these two real-CacheManager methods (src/cache_manager.py:313,817), but MockCacheManager had neither -- update() always hit an AttributeError, caught by a broad except and logged, so the harness rendered an empty-but-green leaderboard on every test run without ever exercising real standings data. Both mocks delegate to the existing get()/set() -- a mock doesn't need the real strategy's per-data-type max_age/market-hours timing, plugins under test just need the methods to exist and round-trip whatever was cached. * fix(testing): clear get_cached_data_with_strategy_calls in MockCacheManager.reset() reset() cleared get_calls/set_calls/delete_calls but not the newer get_cached_data_with_strategy_calls tracker, so a reused mock (e.g. across test cases sharing a fixture) retained stale strategy-call records after reset(). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
ChuckandGitHub
|
3d347a368a |
fix(testing): stop plugin enabled:false schema defaults from silently disabling harness tests (#408)
check_plugin.py, render_plugin.py, and the pytest plugin matrix each built
config as {"enabled": True} then merged in config_schema.json's defaults on
top, letting a plugin's own enabled:false default (a reasonable choice for
a seasonal/opt-in plugin -- 15 of 23 real plugins ship one) silently win.
Every harness/CI render of those plugins was testing "disabled, do
nothing" rather than real behavior.
Extract build_full_config() into testing/loading.py (already the shared
home for plugin-discovery/config-default logic) and use it from all three
call sites: schema defaults, then a forced enabled=True, then harness.json's
config, then the caller's explicit config -- so a test can still
deliberately disable a plugin on purpose, it just can't happen by accident
via the plugin's own shipped schema default anymore.
|
||
|
|
0aca40cf3a |
perf(plugins): run scheduled updates off the render thread (#407)
* perf(plugins): run scheduled updates off the render thread plugin update() executed inline in the render loop — execute_update's internal thread.join(timeout=30) blocked it, so one slow plugin HTTP fetch froze scrolling for the whole fetch (up to 30s; DNS-retry storms made this a regular occurrence on flaky networks). Scheduling stays on the render thread and keeps every existing gate (enabled, circuit breaker, can_execute, interval); due updates are now enqueued to a single background worker (serialized — same one-at-a-time execution as before, no thundering herd). RUNNING is set at enqueue so can_execute blocks re-entry alongside the pending-set dedup. Per-plugin locks make the old implicit update/display no-overlap guarantee explicit: the worker holds the plugin's lock through its update; the display side try-locks and, when the plugin is mid-update, holds the last frame for that iteration — reported as success so a mid-update skip never advances the rotation. Unlike before, the guarantee now also holds across the post-timeout window (previously the lingering update thread overlapped display()). Deadlock-free by construction: the worker takes one lock; display never blocks. Timeout semantics unchanged (lingering daemon thread documented). Kill switch: plugin_system.synchronous_updates: true restores the inline path. 8 new concurrency tests (non-blocking scheduler, overlap assertion under a hammering display loop, lock release on failure/timeout paths, dedup, kill switch); 4-min devpi soak clean (updates completing, rotation advancing, no stuck RUNNING states). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(plugins): fix skipped-frame health/force_change tracking, config validation, and lock-lifetime gaps in async updates Addresses PR #407 review findings: - display_controller: only clear force_change / record health success when display() actually ran this frame, not when the frame was skipped because the plugin's lock was busy (a skip must preserve a pending mode-switch force_clear). - plugin_manager: replace bool() coercion of synchronous_updates with explicit isinstance validation of plugin_system/synchronous_updates, failing safe to synchronous mode (with a logged reason) on malformed config instead of silently defaulting to async. - plugin_manager: _update_worker_loop now acquires the plugin lock before looking up its instance and re-checks under the lock, so an unloaded plugin's lifecycle state is never resurrected to ENABLED. - plugin_manager + display_controller: move lock ownership (and, for updates, RUNNING/pending lifecycle bookkeeping) into the actual update()/ display() call itself rather than the timeout-wrapped caller, so the lock stays held for the real operation's duration even after PluginExecutor's own join(timeout) elapses and a lingering daemon thread keeps running in the background. - DisplayController.cleanup() now stops the update worker before tearing down display/cache resources; stop_update_worker() logs when the join times out instead of failing silently. - test_async_plugin_updates: rewrite test_unloaded_while_queued_is_harmless to exercise the public unload_plugin() lifecycle (via a deterministic blocker) instead of deleting pm.plugins directly, and add a regression test proving the plugin lock stays held through PluginExecutor's own timeout while the real update() call is still running. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9837315308 |
perf(display): dirty tracking in update_display + plugin FPS declaration (#406)
* perf(display): dirty tracking in update_display + plugin FPS declaration update_display now skips SetImage+SwapOnVSync when the frame is byte-identical to the last pushed one (adler32 digest) AND brightness is unchanged — brightness is part of the digest, and set_brightness additionally resets it, so a dim-schedule change can never be skipped. clear() resets the digest (it writes to the matrix directly). Skipping a swap is hardware-safe: the panel refreshes the current frame from the driver's own thread; swaps only change content. Kill switch: display.dirty_tracking: false restores always-push. display_controller's high-FPS decision gains a precedence step: a plugin exposing needs_high_fps is honored first (so static-image can declare False for still PNGs and stop burning a 125fps loop on them); static-image without the attribute keeps its historical forced high-FPS (GIF back-compat); scrolling logic is otherwise unchanged. Verified with 7 tests against the real DisplayManager on RGBMatrixEmulator (identical-frame skip, pixel-change push, clear and brightness invalidation, snapshot-through-skip, kill switch) plus the 202-test display/controller/vegas suites, and a clean devpi deploy. Audit: every SetImage/SwapOnVSync/Clear/brightness call site is inside display_manager — no external writer can bypass the digest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(display): serialize update_display, narrow brightness exception, log fixes CodeRabbit review on #406, verified against current code: - update_display() can genuinely be called from background threads (some sports base classes call it directly from inside update() for an immediate "live" refresh), not just the render loop — confirmed via the existing follower-mode gating wrapper in display_controller.py, which exists specifically because "background plugin threads" can reach it. Without a lock, two callers could both pass the digest check before either writes _last_pushed_digest back, causing a redundant push, or interleave the offscreen/current canvas swap. Added self._update_lock (RLock, in case of re-entrant callers) around the full method body so every call site is automatically covered — no caller changes needed. (No prior lock existed to reuse on DisplayManager; this adds one.) - Narrowed the brightness-read exception handler to AttributeError, matching the established pattern in get_brightness()/set_brightness() — a getattr() with a default already swallows AttributeError, so the only case this guards is the property getter itself raising, and the established pattern treats that as an expected, specific failure mode rather than something to blanket-catch. - FPS-check debug log now includes the plugin_id already in scope (previously only active_mode) and a "[DisplayController]" prefix for grep-ability, matching the sibling log two lines below it. - test_display_dirty_tracking.py: dm fixture and test_config_flag_wires_through now reset the DisplayManager singleton on teardown, matching the pattern test_display_manager.py already uses elsewhere in the same file family. - test_snapshot_still_written_on_skip previously only exercised the non-skip (push) path despite its name; now performs a second update that meets the skip conditions (identical frame) and asserts the snapshot is still written even though the panel push itself is skipped. All 7 dirty-tracking tests pass, plus the full display_manager/ display_controller/vegas suite (140 passed). Full repo suite has only the 5 known pre-existing failures (double-sided config x2, state_reconciliation x2, and test_circuit_breaker's conftest.py mock signature drift — the latter fixed in #400, which this branch's base predates). --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c1fa5094be |
fix(store): plugin updates keep the old install until the new one succeeds (#405)
* fix(store): plugin updates keep the old install until the new one succeeds Both reinstall paths in update_plugin — the monorepo-migration remote switch AND the routine archive update every store user hits — deleted the installed plugin directory BEFORE downloading its replacement. A mid-update failure (bad network, registry error) permanently destroyed the plugin. Seen in the field: a Pi with broken DNS lost 12 plugins in one update pass during the monorepo migration. New _reinstall_with_rollback: rename the old install aside (using the '.standalone-backup-' name pattern plugin discovery already excludes), run install_plugin, remove the aside on success — restore it on ANY failure, clearing partial-download debris first. A stale aside from a previous crash is cleared before starting. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(store): serialize concurrent updates per plugin, check cleanup results CodeRabbit review on #405 flagged two things in _reinstall_with_rollback, both verified against current code: - Real race: the web UI runs Flask with threaded=True and there's a single update route, so two overlapping requests for the same plugin_id (double-click, two tabs) can interleave. The loser could rename the winner's in-progress install aside mid-download, deleting its own rollback safety net — worse than the bug this function exists to fix. Added a lazy per-plugin_id lock dict (mirrors the plugin_manager per-plugin lock pattern) held for the whole function. - _safe_remove_directory's return value was ignored at both call sites. Stale-aside cleanup failure now aborts cleanly instead of falling through to a rename that would fail anyway with a less useful error; post-success backup-removal failure now logs instead of failing silently (still returns True — the update itself succeeded, and the next update self-heals the leftover aside). Left the third nitpick (test_stale_aside_from_previous_crash_is_cleared) addressed by asserting the stale dir is actually gone and that install_plugin was reached, rather than just the end-to-end result. Added a concurrency regression test asserting install_plugin never runs for the same plugin_id while another call is in flight. --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
4d49b0f892 |
perf(display): snapshot mirror — viewer gating, digest skip, keepalive (#404)
The display service PNG-encoded its frame to /tmp/led_matrix_preview.png at 5 fps, 24/7 — identical frames, no viewers, per-call imports and a chmod every write. On the devpi baseline the display service idles at ~92% CPU; this was one of its biggest fixed costs. - New pure policy (src/common/snapshot_policy.py, unit-tested off-Pi): WRITE changed frames at full rate only while a viewer is watching, at a 30s idle cadence otherwise; NEVER re-encode unchanged frames — bump mtime (os.utime) every 20s instead, keeping the health check's snapshot-age liveness proxy (60s threshold in api_v3) green. Cross- referencing comments guard the two constants. - Viewer detection: the web SSE display broadcaster (which only runs while browsers are subscribed) touches /tmp/led_matrix_preview_viewer each loop; the display service stats it at most 1/s. On viewer arrival the write clock resets so the first frame lands within ~1s. - Hoisted the per-call pathlib/permission_utils imports; directory permissions ensured once instead of every frame. Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
efe76d3add |
perf: hot-path micro fixes in the render loop (#403)
* perf: hot-path micro fixes in the render loop - _check_wifi_status_message stat'd the status file on every render iteration (60+ fps) for a message whose lifetime is seconds; throttle the check to 1 Hz with a cached result. - Demote the per-iteration "Display active, processing mode" INFO to DEBUG and convert the remaining eager f-string logs to lazy % args — the devpi baseline showed ~9 journald lines/sec, which is both noise and SD-card wear. - Vegas cycle-end blank frame: hoist the inline PIL import and reuse a preallocated buffer instead of allocating per cycle wrap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix: initialise wifi-status throttle state in __init__ Codacy (pylint access-member-before-definition) on #403: the throttled early-return read _wifi_status_last_result relying on the non-local invariant that the first call always passes the throttle window and assigns it. Correct at runtime, but fragile — initialise both throttle fields in the constructor and drop the getattr fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
273d9962d1 |
fix(cache): stop fsync-hammering the SD card on unchanged data (#402)
DiskCache.set wrote every key as mkstemp -> json.dump(indent=4) -> flush+fsync -> replace -> chmod, on the persistent cache dir — dozens of force-flushed SD writes per minute on an API-heavy install, mostly rewriting identical data every plugin update cycle. - Serialize once, compact (no indent): cache files are machine-read only; indenting multiplied the bytes written. - Skip the disk when the payload for a key is unchanged (adler32 map, per-process); refresh the file mtime instead so records relying on mtime for TTL don't expire early. Self-heals if the file was removed externally (expiry cleanup). - Drop the per-write fsync: os.replace already guarantees readers never see a torn file, and cache data is re-fetchable — the flush bought nothing but card wear. API unchanged; DateTimeEncoder round-trip covered by tests. Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9e3b5f366e |
fix(core): harden text-measurement caches; surface snapshot failures (#400)
* fix(core): harden text-measurement caches; surface snapshot failures Deep-dive findings, all three latent on every 24/7 install: - font_manager.metrics_cache and display_manager._text_width_cache were unbounded dicts keyed by (text, id(font)). Two problems: keys embed the measured TEXT, so ever-changing strings (a clock, a live score, a ticker) grow them without limit; and id()-keying without holding a reference means a garbage-collected font's id can be recycled by a DIFFERENT font, silently returning wrong widths/metrics (classic plugins create fonts per render, so this is reachable). Both caches are now LRU-bounded (1024) and pin the font in the entry so its id stays valid. metrics_cache also keyed on the text itself instead of hash(text), removing a collision path. - _write_snapshot_if_due logged failures at DEBUG — invisible at the default level. The snapshot's mtime is the web UI's display mirror AND its hardware-liveness proxy, so a quiet failure freezes the mirror and makes health checks lie (seen in the field: a stale root-owned /tmp file froze it for a day). Failures now WARN, rate- limited to once per 5 minutes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * test: sync mock cache manager signature with CacheManager.get test_circuit_breaker has been failing on main: plugin_health passes memory_ttl= to cache_manager.get(), and the conftest mock's signature was never updated — the same component/double drift class as the monitored_update bug (#392). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6edd80d9f3 |
fix(schedule): stop stray 'days' data from overriding Global schedule (#399)
save_schedule_config never persisted the schedule's 'mode' field, and _check_schedule inferred per-day vs global purely from whether a 'days' dict was present for the current day. Config migration (_merge_template_defaults) re-adds the template's 'schedule.days' (all days disabled by default) whenever it's missing from the user's saved config - which is exactly the case after saving Global mode, since that save path intentionally pops 'days'. The result: a user on Global mode would get their schedule silently reinterpreted as per-day, with today's day disabled, blanking the display. Persist 'mode' on save and have _check_schedule honor it explicitly (mirroring how _check_dim_schedule already does), so a resurrected 'days' dict can't override an explicit Global selection. Falls back to the old inference behavior only when no 'mode' is recorded. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
1c7a0cef66 |
fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation (#398)
* fix(vegas): restore live plugin-update refresh dropped by sync refactor Investigating a user report that Vegas scroll mode doesn't update scores or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live score change reached the ticker within a few seconds instead of waiting for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed plugin_last_update timestamps to detect which plugins got fresh data and called coordinator.mark_plugin_updated() for each, and should_recompose() checked has_pending_updates_for_visible_segments() to trigger an immediate hot-swap. PR #330 (May 14, multi-display wireless sync) refactored both call sites while adding sync support and silently deleted this entire mechanism -- not just gated it behind the new sync-mode deferral it legitimately needed, but removed it outright. The result: VegasModeCoordinator. mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_ segments() have been fully implemented but never called from anywhere since. Vegas mode's only remaining freshness sources are a 5s content cache TTL (fine) and full recompose at cycle boundaries, which depending on min/max_cycle_duration can be minutes away -- so live scores/status can sit stale far longer than a user would expect from a "live" ticker. Fix: - Restored _tick_plugin_updates_for_vegas() in display_controller.py, wired as the Vegas coordinator's update callback in place of the plain _tick_plugin_updates(). Diffs plugin_last_update before/after the tick and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each plugin that actually got new data (rather than returning the list, since the callback interface no longer consumes a return value). - Restored the has_pending_updates_for_visible_segments() check in render_pipeline.should_recompose(), positioned after (not instead of) the sync-mode early return PR #330 added, so standalone installations regain immediate refresh while synced leader/follower pairs correctly keep deferring hot-swaps to cycle boundaries as PR #330 intended. Test plan: - Added test_display_controller_vegas_tick.py and test_vegas_render_pipeline_recompose.py -- neither area had any prior test coverage, which is very likely why this regression went unnoticed for ~2.5 months. - Verified both new test files fail against the pre-fix code (swapped in the current main versions of both files) with exactly the expected errors -- AttributeError for the deleted method, and the recompose assertion returning False instead of True -- then pass against the fix. - Confirmed the sync-mode deferral this restoration must not break still holds: test_sync_active_defers_pending_updates_to_cycle_boundary. - Full related suite (test_vegas_plugin_adapter, test_vegas_config, test_display_controller_plugin_toggle, test_display_controller_ optimizations, test_plugin_system): 108 passed, 1 pre-existing failure unrelated to this change (test_circuit_breaker, stale mock signature). - Full CI plugin-safety suite (test_harness, test_visual_rendering, test_plugin_matrix): 52 passed, 2 pre-existing skips. * fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation _tick_plugin_updates_for_vegas() snapshotted and later re-iterated plugin_manager.plugin_last_update from the Vegas background update-tick thread while the main render loop (or other callers) could mutate the same dict concurrently — a real race (unprotected dict iteration/mutation across threads), not just a style nit. Move the snapshot/update/diff into a new locked PluginManager.run_scheduled_updates_with_changes() so all reads and mutations of plugin_last_update happen under one lock, and update DisplayController to use it. The lock is only held around the dict accesses, not the update pass itself, so slow plugin update() calls don't serialize against other callers. Also add a regression test covering that the Vegas coordinator is wired to the Vegas-aware tick callback rather than the plain one. Skipped as not worth the change: - Narrowing the broad `except Exception` around vc.mark_plugin_updated(plugin_id) to specific types: it's a deliberate per-plugin isolation boundary (matches the same pattern used elsewhere in this file for plugin/coordinator calls) and there's no documented, stable set of exceptions that call can raise to narrow to. - Adding an inactive-DisplaySyncManager test to test_vegas_render_pipeline_recompose.py: verified VegasModeCoordinator.set_sync_manager() already normalizes a SyncRole.STANDALONE manager to None before handing it to the render pipeline (src/vegas_mode/coordinator.py:152-156), so should_recompose()'s `is not None` check is correct in practice; the suggested case is already covered by that normalization. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
ChuckandGitHub
|
6052a60d22 |
fix(vegas): restore live plugin-update refresh dropped by sync refactor (#395)
Investigating a user report that Vegas scroll mode doesn't update scores or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live score change reached the ticker within a few seconds instead of waiting for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed plugin_last_update timestamps to detect which plugins got fresh data and called coordinator.mark_plugin_updated() for each, and should_recompose() checked has_pending_updates_for_visible_segments() to trigger an immediate hot-swap. PR #330 (May 14, multi-display wireless sync) refactored both call sites while adding sync support and silently deleted this entire mechanism -- not just gated it behind the new sync-mode deferral it legitimately needed, but removed it outright. The result: VegasModeCoordinator. mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_ segments() have been fully implemented but never called from anywhere since. Vegas mode's only remaining freshness sources are a 5s content cache TTL (fine) and full recompose at cycle boundaries, which depending on min/max_cycle_duration can be minutes away -- so live scores/status can sit stale far longer than a user would expect from a "live" ticker. Fix: - Restored _tick_plugin_updates_for_vegas() in display_controller.py, wired as the Vegas coordinator's update callback in place of the plain _tick_plugin_updates(). Diffs plugin_last_update before/after the tick and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each plugin that actually got new data (rather than returning the list, since the callback interface no longer consumes a return value). - Restored the has_pending_updates_for_visible_segments() check in render_pipeline.should_recompose(), positioned after (not instead of) the sync-mode early return PR #330 added, so standalone installations regain immediate refresh while synced leader/follower pairs correctly keep deferring hot-swaps to cycle boundaries as PR #330 intended. Test plan: - Added test_display_controller_vegas_tick.py and test_vegas_render_pipeline_recompose.py -- neither area had any prior test coverage, which is very likely why this regression went unnoticed for ~2.5 months. - Verified both new test files fail against the pre-fix code (swapped in the current main versions of both files) with exactly the expected errors -- AttributeError for the deleted method, and the recompose assertion returning False instead of True -- then pass against the fix. - Confirmed the sync-mode deferral this restoration must not break still holds: test_sync_active_defers_pending_updates_to_cycle_boundary. - Full related suite (test_vegas_plugin_adapter, test_vegas_config, test_display_controller_plugin_toggle, test_display_controller_ optimizations, test_plugin_system): 108 passed, 1 pre-existing failure unrelated to this change (test_circuit_breaker, stale mock signature). - Full CI plugin-safety suite (test_harness, test_visual_rendering, test_plugin_matrix): 52 passed, 2 pre-existing skips. |
||
|
|
7f7f0d6464 |
feat: adaptive layout system — size-aware regions, crisp font ladders, image fitting (#393)
* feat(layout): adaptive layout & font scaling system for plugins Add src/adaptive_layout.py — opt-in core helpers so plugins render legibly on any panel size without hand-tuned per-display layouts: - Region: integer rect algebra (bands/columns/weighted splits/centering) that partitions space so text bands can't overlap by construction - Font ladders: ordered (family, size) steps known to render crisply (LADDER_GRID: X11 BDFs at native sizes; LADDER_ARCADE: PressStart2P at 8px multiples) — fitting walks the ladder instead of scaling pixel fonts fractionally - LayoutContext: breakpoint tiers, geometry scale vs. a declared design size, and cached fit_text/fit_lines/font_for_rows queries Generalizes the three patterns proven in the field: f1-scoreboard's scale factor, masters-tournament's tiers, baseball-scoreboard's font fallback ladder. Wiring: BasePlugin gains a lazy .layout property and draw_fit(); FontManager gains get_native_bdf_size() and a cache_generation counter; manifest schema gains display.design_size and requires.display_size max_width/max_height; 96x48 joins DEFAULT_TEST_SIZES; the bounds-check harness records negative-coordinate draws; TextHelper's broken measurement helpers are fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): adaptive image fitting + composite region helpers Add src/adaptive_images.py — the image counterpart to fit_text: - fit_image(img, box, mode=contain|cover|fill_height|stretch, crop_to_ink, anchor, resample, upscale) promoting the proven plugin patterns (football's crop-to-ink fill-height logos, masters' cover crop + NEAREST flags, static-image's letterbox). Upscales by default — thumbnail()'s downscale-only behavior is why imagery stays tiny on big panels. - draw_fitted_image() pastes aligned within a Region with alpha mask. - One central Pillow>=9.1 RESAMPLE shim replacing ~15 plugin copies. LayoutContext.fit_image() caches results per (identity, box size, options) with a 64-entry LRU; id()-keyed entries pin the source image. BasePlugin.draw_image() is the one-liner adoption path beside draw_fit. Composites in adaptive_layout.py: Region.offset() (user x/y-offset passthrough), scoreboard_regions() (the two-logos-plus-score card math duplicated across six sports plugins, logo_slot = min(H, W//2)), and media_row() (art-left/text-right). Fix LogoHelper's size-blind cache key (stale sizes on panel change); deprecation note on dead image_utils.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(harness): scale-up fill check, config variants, multi-size dev gallery Quality gates for adaptive layout: - fill_metrics()/check_scale_up() in the safety harness: overflow catches content too big for a panel, but nothing caught content that stays tiny on panels >= 2x the plugin's declared design size. The check measures lit-content extents and warns (or fails, when a plugin opts into "fill_check": "strict" in test/harness.json) below 50% coverage on the doubled axis. Warn-only by default so no existing plugin breaks. - harness.json "variants": extra runs with config overlays and their own golden dirs, so an opt-in mode (e.g. layout_mode: adaptive) is golden- tested beside the classic default. check_plugin.py loops base + variants and labels variant results mode@name. - Dev preview server: GET /api/sizes (harness size sample), POST /api/render-matrix (render at up to 12 sizes in one call), size-preset dropdown, and an "All Sizes" side-by-side gallery in the preview UI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(plugins): adaptive-lib discoverability + advisory version compat warning Discoverability: re-export the adaptive layout/image API from src.common (the blessed-helpers package plugin authors already know) — canonical paths stay src.adaptive_layout / src.adaptive_images so nothing breaks. Document it in src/common/README.md and cross-link ADAPTIVE_LAYOUT.md from the developer docs authors actually read (quick reference, API reference, advanced dev, font manager, dev preview, plugin dev guide); ADAPTIVE_LAYOUT.md gains adaptive-images, composite-layouts and preserving-user-customization sections. Compat: PluginLoader now logs one advisory warning (never raises) when a plugin's manifest declares a min LEDMatrix version newer than the running core, checking the min_ledmatrix_version / requires.* / versions[] spellings found in the wild. Guarded against stale core version numbers. src/__init__.py __version__ bumped 1.0.0 -> 3.1.0 to match the latest release tag (v3.1.0) — it had never been updated and the compat check needs a truthful number. NOTE: verify this matches the intended release numbering before the next tag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): add measure_font_crispness — verify a ladder rung isn't blurry PIL antialiases TTF outlines by default; a 'pixel-style' font only rasterizes without antialiasing at specific sizes (for PressStart2P: exact multiples of its 8px design grid). A ladder rung at an unverified size silently renders blurry on an LED panel — this exact bug shipped in both text-display's and football-scoreboard's custom TTF ladders (non-8-multiple PressStart2P sizes, and '5by7.regular'/'4x6-font' at sizes that were never actually crisp). measure_font_crispness(font, sample_text) renders the sample and reports the fraction of ink-bbox pixels that are neither pure black nor pure white. BDF fonts (real bitmaps) always score 0.0; TTF ladders should be verified against this before shipping — see the new TestFontFitting::test_ladder_arcade_is_crisp pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): add fit_text_proportional — proportional sizing vs. always-maximize fit_text always picks the largest ladder rung that fits its box. That's right when an element owns dedicated space, but wrong when several independently-fitted elements need to stay visually harmonious as the panel grows: a score's box might have generous room while a neighboring logo scales by a fixed geometry factor via px() — fit_text lets the score balloon out of proportion (even overlapping the logo) even though its individual pick is technically correct. fit_text_proportional(text, box, base_size_px, ladder) instead targets base_size_px * self.scale (the same scale factor px() already uses), picking the nearest ladder rung at or below that target, still capped to what fits the box, floored at the smallest rung when the target is below every rung. Refactored the shared largest-that-fits/ellipsize walk into _walk_ladder() so fit_text and fit_text_proportional don't duplicate it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): fit_text_proportional gains an axis-specific scale override self.scale (min(width_ratio, height_ratio)) is the right conservative default for anything whose aspect ratio matters, but a caller whose surrounding composition already scales along a single axis — e.g. football-scoreboard's logo_slot = min(height, width // 2), which tracks height alone — needs text sized the same way, or it reads as under-scaled next to logos that grew on a panel that only got taller (128x32 -> 128x64: self.scale stays 1.0 since width didn't grow, but logos still double). fit_text_proportional(..., scale=None) now accepts an explicit override; None keeps the existing self.scale default. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(layout): scoreboard_regions reserves real center space at 2:1 aspect ratios logo_slot = min(height, width // 2) has a blind spot: at exactly 2:1 aspect ratio (width == 2 * height -- a very common shape: two, four, or more square modules stacked into a taller panel) width // 2 and height are equal, so the two logo slots claim the ENTIRE width and leave zero pixels for a center column, no matter how large the panel gets. Not a 'small panel' problem -- 96x48, 128x64, and 256x128 (all exactly 2:1) hit it identically, while the 128x32 design baseline and panels like 192x48 or 256x32 never do, because height is already the tighter constraint there. Two new parameters fix it in the one shared helper every scoreboard-style plugin composes through: - min_center_fraction / min_center_design_px reserve at least max(width * fraction, design_px * ctx.scale) for the center column, capping logo_slot further when needed. The scaled design-px term matters on small panels where a flat fraction alone reserves too little absolute space. - score_bleed_fraction extends the score's own fit box (not the logo slots themselves) a controlled amount into each side -- the same way real broadcast scoreboards let a big score number's edges cross into the team marks flanking it. Without this the reserve alone can still be too narrow for a short score to render without truncating. score_area is now genuinely narrower than the full card width (previously identical to status_band/detail_band, which still span the full width and overlay the logos -- short text there was never the problem). Verified against the full harness size spread: a real game score like '17-21' never needs ellipsis at any tested 2:1-or-tighter aspect ratio (test_score_never_needs_ellipsis_for_a_short_score), and wide panels (128x32/192x48/256x32-style) are provably unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: document scoreboard_regions' center-reserve and score-bleed params Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address CodeRabbit review on PR #393 - docs: scope the self.layout note to BasePlugin subclasses (others build a LayoutContext directly) and make explicit that adaptive layout is opt-in — classic rendering stays unless a plugin adopts the APIs. - dev_server: broaden the render-request catch (a bad manifest.json now returns a clean 400 instead of an unhandled 500) and stop echoing raw exception text in the loader-failure responses — full tracebacks go to the dev server's console log instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): allowlist plugin_id before any path lookup CodeQL (py/path-injection): plugin_id arrives in request input and flows into filesystem paths via find_plugin_dir. Gate it with the same ^[a-zA-Z0-9_-]{1,64}$ allowlist the web UI's pages_v3 uses, at the single choke point every route resolves through. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): lexical containment check on resolved plugin dirs CodeQL doesn't recognize the interprocedural allowlist as a path-injection barrier; add the canonical one — normalize (without following symlinks, since dev plugins are commonly symlinked into plugins/) and require the result to stay inside the search dir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): inline normpath containment barrier before render CodeQL doesn't credit the sanitization inside find_plugin_dir along this flow; apply its documented barrier (normpath + startswith against the allowed roots) inline in _parse_render_request, on the exact path that reaches the render/load sinks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): derive plugin dir from trusted directory listings CodeQL's barrier-guard recognition doesn't see a startswith check inside an any() comprehension, so the normalize-and-prefix approach still flagged. Break the taint outright instead: after lookup, re-derive the directory by enumerating the search dirs (iterdir) and matching by path equality — the Path used for all downstream file access is built solely from trusted listings, never from request input. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): use os.scandir for path-injection barrier, redact stack traces from render responses CodeQL doesn't model Path.iterdir() as a taint-clearing enumeration the way it does os.scandir() -- _trusted_plugin_dir's iterdir-based rebuild still traced plugin_id through to the manifest.json open(). Switched to scandir, matching the pattern already verified clean on PR #396. Also stops surfacing raw exception text (update()/display() failures) in the JSON render response -- logs full detail server-side via exc_info instead, returning only the exception class name to the client. And drops path values from three plugin_loader debug/error logs that CodeQL flags as clear-text-logging of externally-influenced data, keeping plugin_id (not flagged) for context. * fix(dev-server): remove conditional-reassignment ambiguity in plugin_dir resolution CodeQL's path-injection flow still traced through _parse_render_request after the scandir fix -- the tainted find_plugin_dir() result and the scandir-derived _trusted_plugin_dir() result shared the same variable name (plugin_dir), reassigned only on the truthy branch. That merge point apparently isn't treated as a barrier by the flow analysis, so it kept tracing the pre-reassignment value through to the manifest open(). Split into two distinct names -- candidate_dir (tainted, used only to call _trusted_plugin_dir) and trusted_dir (the only name used for any downstream file access) -- so there's no reassigned variable for the flow to walk through. * fix: remove unused imports flagged by Codacy Union in adaptive_images.py and field in adaptive_layout.py are both imported but never used -- the last two Codacy findings on this PR, matching the same fix already applied on PR #396. * fix(layout): bound the fit cache; never alias the source image in fits Two latent issues found in a self-review pass: - LayoutContext._fit_cache was an unbounded dict (the image cache got an LRU cap, the text-fit cache didn't). Cache keys embed the fitted TEXT, so a plugin fitting changing strings — a live game clock, a ticker — on a 24/7 service grows it forever. Now LRU-bounded at 512 entries via the same pattern as the image cache. - fit_image returned the caller's ORIGINAL image object when the source was already RGBA at target size (contain/fill_height, no ink crop). ImageFitResult is documented as an independent copy, and LayoutContext caches results — an aliased image lets later mutations of the source corrupt cached fits (or vice versa). Copy in that branch. Both covered by new regression tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
05e7c43b27 |
fix(plugins): replace dependency marker files with a real satisfaction check (#390)
* fix(plugins): replace dependency marker files with a real satisfaction check The .dependencies_installed hash-marker system only tracked "was this exact requirements.txt hashed before" — not whether the packages it names are actually present. That made it fragile (a wiped venv, a manually removed package, or a lost/corrupted marker forces a needless full pip reinstall or, worse, a false skip) and produced dead weight for the ~10 plugins whose requirements.txt is comment-only (they still paid a pip subprocess on first boot before a marker existed). Replace it with requirements_are_satisfied() in plugin_loader.py, which checks each real requirement line against importlib.metadata directly, so install_dependencies() only shells out to pip when something is actually missing or version-mismatched. Drops the marker file entirely: removed all marker read/write sites in plugin_loader.py and store_manager.py, the now-pointless marker-cleanup step in the git-update path, the unused legacy marker implementation in plugin_manager.py, and the already-stale clear_dependency_markers.sh script. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(security): close path-injection gap in dependency-satisfaction checks CodeQL flagged 2 new high-severity "uncontrolled data used in path expression" alerts at the open() calls inside this PR's new requirements_has_real_deps()/requirements_are_satisfied() -- both are reachable from paths that were never run through the basename+trusted-base sanitiser this codebase already uses elsewhere: - PluginLoader.install_dependencies() only applied that sanitiser when its optional plugins_dir argument was actually passed; the "no plugins_dir" branch trusted plugin_dir_real directly. Made plugins_dir required (not Optional) so that branch can't exist, and added an explicit guard in load_plugin() so install_deps=True without a plugins_dir fails loudly instead of silently. Production's only real caller (PluginManager) always passes plugins_dir already; the harness/dev-server/render-plugin callers all use install_deps=False and are unaffected. - StoreManager._install_dependencies() never sanitised plugin_path at all, and its call sites ultimately derive that path from a plugin's own manifest.json "id" field (install_plugin_from_url) -- a malicious plugin could otherwise point requirements_file outside plugins_dir. Applied the same os.path.basename()-based containment pattern PluginLoader already uses (and that CodeQL recognises as a real sanitiser). Added test_install_dependencies_requires_plugins_dir and test_install_dependencies_rejects_path_outside_plugins_dir to lock in the actual security property, not just quiet the scanner. Verified: all 20 tests in test_plugin_loader.py pass, plus the PR's existing test plan (test_plugin_system.py, test_store_manager_caches.py: 53 passed) and the full CI plugin-safety suite (test_harness.py, test_visual_rendering.py, test_plugin_matrix.py: 52 passed, 2 pre-existing skips) all still pass. * fix(security): replace basename-only sanitiser with a trusted-enumeration check The previous commit's os.path.basename() + os.path.join() pattern (which a pre-existing code comment claimed CodeQL recognises as a sanitiser) did not actually clear the alert -- the next CodeQL run still flagged the same 2 sink lines, plus a new one at the os.path.join() call itself. Taking a substring of tainted data apparently isn't treated as a barrier by this query, whatever the comment assumed. Replaced it with find_trusted_subdir(): enumerate the trusted plugins_dir via os.scandir() and only use a name that scandir itself produced, matched by equality against the caller's requested name. The path is then built from that enumerated entry, not from the caller's string -- a value sourced from iterating a trusted, non-tainted directory carries no taint regardless of what it happens to equal, which is a stronger and more conventional allowlist-style barrier than string-stripping. Applied identically in both PluginLoader.install_dependencies() and StoreManager._install_dependencies(), sharing one implementation. Re-verified: all 65 tests across test_plugin_loader.py (20, including the 2 new security regression tests), test_store_manager_caches.py (35), test_plugin_system.py (10) pass, plus the full CI plugin-safety suite (test_harness.py/test_visual_rendering.py/test_plugin_matrix.py: 52 passed, 2 pre-existing skips). * fix(security): redact URL credentials from pip subprocess output before logging CodeQL flagged 3 clear-text-logging-of-secrets alerts in install_requirements_file() (src/common/permission_utils.py:353,360,371). Pre-existing on main, unrelated to this PR's own diff, but now visible since the path-injection alerts that previously took priority in the annotation list are fixed. The underlying risk is real: pip can echo a private index URL's embedded basic-auth credentials (from a requirements.txt --index-url line or PIP_INDEX_URL) back verbatim in its own stderr/stdout on failure, and this function both logs that output directly and returns it to callers -- store_manager.py's _install_dependencies() logs result.stderr from this same function too. Added _redact_url_credentials(), applied immediately after each of the two subprocess.run() calls (mutating result.stderr/stdout in place) rather than patching each log call site individually. This closes the leak at the source: every downstream use -- the three flagged log lines, the "note" string embedded in the returned stdout, and store_manager.py's own logging of the returned result -- gets the redacted text for free. Verified the fixed-phrase "denied" check (`"a password is required" in result.stderr`) is unaffected, since URL syntax and those phrases don't overlap -- covered explicitly by test_does_not_touch_denied_check_phrases. Added test/test_permission_utils.py (6 tests) covering the redaction helper directly and both subprocess.run() call sites (the sudo-wrapper branch, which this repo's scripts/fix_perms/safe_pip_install.sh makes live, and the no-wrapper fallback branch). All pass. * fix(security): stop interpolating req_file/pip-output into log calls The previous commit's redaction (mutating result.stderr/stdout right after each subprocess.run()) didn't clear CodeQL's clear-text-logging alerts -- same lesson as the path-injection fix earlier in this PR: a static analyzer can't tell "this value was already sanitised two lines up" from "this is still the raw tainted value" just by looking at a single log call in isolation, so it conservatively keeps flagging it regardless of what the redaction function actually does. Removed all dynamic interpolation (req_file, result.stderr) from the 3 flagged logger.warning() calls entirely, replacing them with fixed messages plus (for the one that had it) result.returncode, which is a plain int with no possible taint. The full redacted detail is still available where it actually matters -- in the returned CompletedProcess.stderr/stdout and the "note" text -- just not duplicated into a log line a scanner has to reason about in isolation. Re-verified: all 6 test_permission_utils.py tests still pass (they assert on the returned result, not log call arguments), plus the full test_plugin_loader.py/test_store_manager_caches.py/test_plugin_system.py suite (71 passed, 1 pre-existing deselect, 4 subtests). --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
ChuckandGitHub
|
2ffc57cf40 |
fix(plugin-harness): add no-op process_deferred_updates to test double (#391)
The safety harness's VisualTestDisplayManager (base of BoundsCheckingDisplayManager) doesn't implement process_deferred_updates(), which 5 first-party ledmatrix-plugins call unconditionally between set_scrolling_state() and their scroll-position update: news, odds-ticker, ledmatrix-leaderboard, stock-news, and ledmatrix-stocks. Any of them fails the harness with AttributeError the moment it's touched (surfaced when ledmatrix-plugins#177 had to add a local hasattr guard in ledmatrix-stocks just to pass CI). Add the method as a no-op, mirroring the existing "no-op for testing" pattern already used for set_scrolling_state, so these plugins render under the harness without every touching PR needing its own guard. |
||
|
ChuckandGitHub
|
aab0e9ade0 |
fix(plugin-manager): fix TypeError breaking every plugin's scheduled update (#392)
run_scheduled_updates()'s resource-monitor branch wrapped the update call
in a closure stored as a *class* attribute on a dynamically-built type
(type('obj', (object,), {'update': monitored_update})()). The descriptor
protocol turns a function found via class-attribute lookup into a bound
method on instance access, silently prepending the synthetic instance as
an implicit first argument -- but monitored_update() takes none, so every
call raised "monitored_update() takes 0 positional arguments but 1 was
given", was caught by run_scheduled_updates' try/except, and recorded as
an update failure.
self.resource_monitor is None by default and was dormant until PR #388
("activate dormant plugin health/metrics subsystem") wired it up in both
display_controller.py and web_interface/app.py -- meaning this bug went
live in every real deployment as of that merge (2026-07-09) despite the
buggy line itself dating back to 2025-12-27. In practice this means no
plugin's update() has succeeded since upgrading past #388: circuit
breakers cycle through half-open -> immediate failure -> reopened every
health-check interval forever, and all plugin data (scores, odds, prices,
etc.) goes stale from whatever was last fetched before the upgrade.
Confirmed live on a running instance: odds-ticker (and stock-news,
ledmatrix-stocks, baseball-scoreboard, ledmatrix-leaderboard, of-the-day)
failing this exact way every 5-minute circuit-breaker retry.
Fixed by using types.SimpleNamespace(update=monitored_update) instead of
a dynamic class: SimpleNamespace stores attributes on the instance
itself, so attribute lookup returns the plain function unchanged --
never routed through the class-attribute descriptor protocol that
injects an implicit self.
Added test_run_scheduled_updates_calls_update_with_resource_monitor to
test/test_plugin_system.py using a real PluginResourceMonitor (not a
mock of it), so the test exercises the actual descriptor-binding
behavior that caused this. Verified the test fails with the exact
reported error against the pre-fix code and passes against the fix.
|
||
|
|
bd9f461f70 |
Add system diagnostics, power controls, and WiFi radio toggle to Tools tab (#389)
* Add system diagnostics, power controls, and WiFi radio toggle to Tools tab Expands the web UI Tools tab with safe, purpose-built controls so users can manage the Pi without SSHing in, instead of an arbitrary-command terminal (the web UI has no auth and CSRF is disabled, so a shell would be unsafe). - System Diagnostics card: renders the existing but previously-unused GET /api/v3/system/status endpoint (CPU, memory, temp, disk, uptime), with a manual refresh and a 10s poll. - System Power section: reboot/shutdown buttons wired to the existing reboot_system / shutdown_system actions, behind a confirm step, with a dedicated powerAction() helper that treats the dropped connection as the expected "going offline" outcome rather than an error. - Network Radio section: WiFi on/off toggle backed by new GET/POST /api/v3/wifi/radio endpoints and WiFiManager.set_wifi_radio() / get_wifi_radio_state(). Disabling WiFi is refused unless a wired connection is present (reusing the existing lockout guards), with an explicit force-off confirmation for advanced users. No new privileged commands: uses nmcli radio wifi on|off (already sudo-allowlisted) and the existing reboot/poweroff grants, so the sudoers-alignment guard test stays green. Bluetooth toggle intentionally omitted since the installer removes the BlueZ stack for LED timing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE * Harden WiFi radio endpoint and diagnostics poll (review feedback) Addresses code review feedback on the Tools tab additions: - api_v3.py: parse the `enabled` POST field with the same string-aware coercion as `force`. A plain bool() cast turned {"enabled":"false"} into True (enabling instead of disabling) for any non-UI API caller. - wifi_manager.set_wifi_radio() now returns a reason code alongside (success, message); the /wifi/radio error response includes it. The Tools UI only shows the force-off confirmation when reason == 'no_ethernet', so a genuine nmcli failure surfaces its real error instead of a misleading "no wired connection" prompt that would just retry into the same failure. - tools.html: gate the 10s diagnostics poll on panel visibility (document.hidden / offsetParent), so switching to another tab stops the recurring /api/v3/system/status calls instead of churning the Pi off-screen. The initial load and manual Refresh remain unconditional. Verified: {"enabled":"false"} now disables (refused w/ reason:no_ethernet), {"enabled":"true"} enables; inline JS passes node --check; py_compile clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE * Narrow WiFi-disable fallback except and add stack trace (review) Addresses a CodeRabbit nitpick: the fallback handler in set_wifi_radio()'s disable path caught bare Exception and logged without a traceback. Narrow it to (OSError, subprocess.SubprocessError) — the errors subprocess.run realistically raises — and log with exc_info=True for full context on the Pi. Anything genuinely unexpected now propagates to the endpoint's outer handler (500), matching the codebase's specific-exception convention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
3b93024993 |
feat: activate dormant plugin health/metrics subsystem and surface it in the web UI (#388)
* feat(plugin-system): activate dormant plugin health & metrics subsystem
PluginManager shipped a fully-built health tracker, resource monitor and
circuit breaker that were never instantiated (health_tracker/resource_monitor
were left as None), so the circuit breaker never engaged and the existing
health/metrics API routes always returned "not available".
- DisplayController now wires a PluginHealthTracker and PluginResourceMonitor
onto the plugin manager, enabling the circuit breaker (a repeatedly-failing
plugin's update() is skipped after consecutive failures, then retried after
a cooldown) and per-plugin execution-time metrics. Both persist to the
shared cache.
- load_plugin() now validates each plugin's config against its JSON schema in
a strictly warn/degrade-only way: a violation logs a warning and flags the
plugin degraded in the health tracker, but never changes whether the plugin
loads or its pass/fail behaviour. Adds PluginHealthTracker.set_degraded(),
which never touches the circuit breaker.
- ResourceMonitor CPU/memory sampling now reuses a cached psutil.Process and
reads cpu_percent(interval=None), so monitoring no longer blocks ~100ms per
call on the display loop's update path.
- Fix DiskCache.get() raising TypeError for max_age=None ("never expires"),
which silently discarded persisted plugin health/metrics on read and thus
broke cross-process and post-restart surfacing.
- Fix two dead PluginManager helpers that called non-existent tracker methods.
Tests: new test_resource_monitor, test_plugin_health,
test_plugin_manager_schema_soft; extended test_cache_manager and
test_display_controller.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq
* feat(web-ui): surface plugin health, metrics and load state
With the health/metrics subsystem now active in the display service, expose it
in the web UI (which runs as a separate process from the display loop):
- Wire a health tracker / resource monitor backed by the shared on-disk cache
into the web process so /api/v3/plugins/health and /plugins/metrics read the
data the display service persists.
- Build those route responses per installed plugin id (the tracker's in-memory
view is empty in a fresh web process) so cross-process data is included.
- Add state + error_info to /plugins/installed entries so the UI can show why a
plugin isn't running instead of just loaded:false.
- Add a "Plugin Health" panel to the Tools page (circuit status, avg/max update
time, update count, last error) plus PluginAPI.getPluginMetrics().
Tests: route-level tests for the health/metrics endpoints in test_web_api.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq
* fix(plugin-metrics): refresh cross-process health/metrics reads; type hints
Addresses CodeRabbit review on #388:
- Major: the web process's health/resource trackers cached the first persisted
read in an in-memory dict (and the CacheManager memory tier held max_age=None
entries indefinitely), so a long-lived web process showed the first snapshot
and never reflected the display service's later updates. Add an opt-in
force_reload path (get_health_summary/get_health_state/_load_health_state and
get_metrics_summary/get_metrics) that bypasses the in-memory copy and, via a
new memory_ttl passthrough on CacheManager.get, the cache manager's memory
tier — so each /plugins/health and /plugins/metrics poll reads fresh persisted
state. Default behaviour (force_reload=False) is unchanged for the display
process and existing callers.
- Minor: DiskCache.get type hint is now Optional[int] with the None ("never
expires") semantics documented, matching MemoryCache.get.
Tests: new force_reload staleness cases in test_plugin_health and
test_resource_monitor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
85d321cf33 |
fix: plugin_loader retries with --ignore-installed on apt/pip RECORD conflicts (#386)
* fix: plugin_loader retries with --ignore-installed before assuming apt package satisfies pin install_dependencies treated any "uninstall-no-record-file" pip failure as "dependency satisfied" and wrote the success marker without ever attempting --ignore-installed, unlike install_dependencies_apt.py and safe_pip_install.sh (added in #385 for the Plugin Store/first-time-install paths). A plugin pinning a newer version of a system-managed package (e.g. requests) would silently keep running against whatever version apt shipped, while the marker file claimed the pinned requirement was met. Now retries the same install with --ignore-installed on that specific failure so pip actually lays the pinned version down (shadowing the system-managed copy) before falling back to the prior tolerant behavior if the retry itself fails too. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * chore: suppress Codacy finding on new retry subprocess.run call Same generic Bandit/semgrep pattern-match on non-literal subprocess.run argv flagged in #385's install_requirements_file, now on the new --ignore-installed retry call added here: list-form argv (no shell=True), sys.executable is this process's own interpreter, and requirements_file is built internally by find_plugin_directory, never raw external input. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: tolerate a timed-out --ignore-installed retry, dedupe marker-write logic CodeRabbit review caught a real inconsistency: if the --ignore-installed retry itself timed out, subprocess.TimeoutExpired propagated to the outer handler and returned False, failing plugin load — contradicting the intended "tolerate this specific apt/pip conflict" behavior, where a mere non-zero retry return code already returns True. Wraps the retry in its own try/except so a timeout is logged and tolerated the same way as any other retry failure. Also extracts the marker-writing logic (open/write/chmod, ignoring OSError) into _write_dependency_marker, since it was duplicated identically between the direct-success path and the apt-conflict-retry-fallback path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: revert marker-write dedup helper, resolves CodeQL path-injection alert CodeQL flagged _write_dependency_marker's open(marker_file, ...) as "uncontrolled data used in path expression" (high severity) once the marker-write logic was extracted into its own method. marker_file is actually safe — it's built from safe_plugin_dir, which install_dependencies sanitizes via os.path.basename() (CodeQL's own recognized py/path-injection sanitizer, per the existing comment a few lines above) — but CodeQL's interprocedural analysis doesn't carry that sanitized status across the new method boundary, since the sanitizer call and the open() sink were no longer in the same function. This exact code produced zero CodeQL findings before the extraction (in two duplicated inline blocks) and is unchanged in what data reaches it — only its location moved. Reverting the extraction (keeping CodeRabbit's other, independent timeout-handling fix) restores the previously-clean shape rather than trying to convince the analyzer's cross-function taint tracking that a refactor changed nothing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
63a233f3ed |
fix: dependency installation gaps in Plugin Store and first-time install (#385)
* fix: install plugin dependencies through root-visible installer in Plugin Store install_plugin/update_plugin (store_manager.py) installed requirements.txt with a bare `pip3` off PATH, bypassing the root-visible installer added in #380 for the "Reinstall Plugin Deps" button. Two bugs stacked: (1) `pip3` can resolve to a different Python install than the one that actually runs ledmatrix.service, and (2) even when it resolves correctly, ledmatrix-web runs as a non-root user so the package lands in that user's local site-packages, invisible to root-run ledmatrix.service. Either way the install reports success and writes the .dependencies_installed hash marker, so plugin_loader's own (correct) install-on-load path skips reinstalling — leaving the dependency permanently missing until a user finds and clicks the separate "Reinstall Plugin Deps" tool. This is why users kept hitting "No module named 'astral'" for the weather plugin even after installing it from the Store. Extracts the sudo-wrapper-then-fallback install logic from api_v3.py's _pip_install_requirements into src/common/permission_utils.py as install_requirements_file, and routes store_manager.py's dependency installation through it so the automatic Store install/update path now matches the manual "Reinstall Plugin Deps" path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * chore: suppress Codacy false-positive on subprocess.run in install_requirements_file Codacy's generic subprocess-security rule (Bandit B603 equivalent) flagged the pip/sudo subprocess.run calls in install_requirements_file for lacking a "static string argument" — the standard pattern-based flag for any subprocess.run() call with a variable in its argv list. Both calls use list-form argv (no shell=True, so no shell-injection surface), and the only dynamic value is req_file, a Path built internally by callers rather than raw external input; safe_pip_install.sh independently re-validates it before installing anything as root. Suppresses with inline `# nosec B603` comments matching this codebase's existing convention (see permission_utils.py's own PROTECTED_SYSTEM_DIRECTORIES, display_manager.py, sync_manager.py, etc.). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: first-time install script fails on apt-managed requests package web_interface/requirements.txt and requirements.txt both pin requests>=2.33.0,<3.0.0, but Raspberry Pi OS ships an apt-managed python3-requests with no pip RECORD file. Upgrading it via plain `pip install` aborts with "uninstall-no-record-file" because pip refuses to uninstall a package it has no record of, in place — which is exactly the "Some web interface dependencies failed to install" warning first-time install hits. scripts/install_dependencies_apt.py and scripts/fix_perms/safe_pip_install.sh already work around this with --ignore-installed (lets pip lay the new version down in /usr/local, shadowing the apt copy, instead of trying to remove it first). first_time_install.sh's own direct pip invocations — the per-package requirements.txt loop, the web_interface/requirements.txt install, and the requirements_web_v2.txt fallback — didn't have it. Adds --ignore-installed to all three so first-time install no longer fails on this well-known apt/pip conflict. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * chore: add nosemgrep to subprocess.run calls Codacy still flagged The prior # nosec B603 comments suppressed Bandit's check but Codacy's semgrep-based rule ("subprocess function 'run' without a static string") kept flagging the same two lines as a critical security issue even after that fix landed. install_dependencies_apt.py's _run() already needed both tags together (# nosec B603 B607 ... # nosemgrep) for the identical subprocess.run pattern, so apply the same double suppression here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: add --ignore-installed to install_requirements_file fallback path CodeRabbit review caught this (confirming a gap already flagged in conversation): the non-sudo fallback pip install in install_requirements_file was missing --ignore-installed, unlike the sudo-wrapper branch and safe_pip_install.sh. Without it, the same apt/pip RECORD-file conflict this PR fixes elsewhere (first_time_install.sh, install_dependencies_apt.py) could still hit installs that fall back to this path (e.g. a plugin's requirements.txt on a host where safe_pip_install.sh isn't set up yet). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
7a9d01342a |
Fix inconsistent Vegas scroll transition gaps caused by plugin-baked padding (#384)
* Strip plugin-baked scroll padding when capturing content for Vegas mode Plugins that build their own ticker image via ScrollHelper.create_scrolling_image() (or that manually pad both ends for a clean standalone loop) carry a solid-black margin up to display_width wide on one or both edges. Vegas mode already adds its own configurable gap around every item, so leaving that margin in place stacked an extra, uncontrolled blank stretch on top of separator_width for whichever plugin took the ScrollHelper-capture path — producing inconsistent transition gaps between modules compared to plugins that provide content natively via get_vegas_content(). _get_scroll_helper_content() now detects and crops any such margin before handing the image to the Vegas render pipeline, so every plugin's gap is governed solely by vegas_scroll.separator_width regardless of which capture path produced its content. * Address CodeRabbit nitpick: warn on double-edge padding crop, add unit tests Logging a double-edge match at warning level (vs. info for a single edge) makes it easy to spot an unexpected crop in the field, since two edges matching at once is a much stronger signal of genuine baked-in padding than one edge coinciding with real all-black content. Also adds test/test_vegas_plugin_adapter.py covering _strip_scroll_padding's branch logic: leading-only, trailing-only, both-edges, no-match, degenerate all-black, missing/undersized display_width, and the info-vs-warning log level. * Add type hints and docstring to test _solid helper (CodeRabbit nitpick) --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
7a6bad29fe |
feat(display-controller): hot-reload plugin enable/disable without a restart (#374)
* feat(display-controller): hot-reload plugin enable/disable without a restart Enabling or disabling a plugin in config previously required restarting the display service: the plugin list and available_modes were built once at init and the run loop never revisited them. (Per-plugin config *values* already hot-reloaded; only the enabled set was restart-only.) Now the controller reconciles its running plugins against the config's enabled set whenever that set changes: - The ConfigService watcher thread only sets a `_pending_plugin_reconcile` flag (via a cheap enabled-set diff). It never mutates loop state. - The run loop applies the reconcile on its own thread (top of each iteration, deferred while on-demand is active), so loading/unloading and rebuilding available_modes can't race with rendering. - `_reconcile_enabled_plugins` diffs desired vs running plugins, unloads the removed ones (cleanup + on_disable + config-unsubscribe via the new `_unregister_plugin`) and loads the added ones, then clamps the rotation index so the current mode stays valid. The per-plugin registration done at startup is extracted into `_register_loaded_plugin` and reused by the live-enable path so both build identical state. Extracting it also fixes a latent late-binding bug: the per-plugin config-change callbacks were closures over the loop variable, so every plugin's callback targeted the last-loaded instance; each now binds its own id/instance. Adds test/test_display_controller_plugin_toggle.py covering live enable, live disable, index clamping, no-op when unchanged, and the enabled-set diff. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(display-controller): don't exit on empty available_modes, guard rotation modulo Hot-reload means available_modes can legitimately be empty at startup (no plugins enabled yet) and become non-empty later via the web UI, or vice versa mid-run. Fix four issues found reviewing this PR: - run() exited the process entirely when available_modes was empty at startup instead of idling, permanently defeating the point of live enable/disable for anyone who starts with zero plugins enabled. - The mode-rotation step divided by len(available_modes) unconditionally, raising ZeroDivisionError if the last enabled plugin is disabled between frames. - _reconcile_enabled_plugins() called .get('enabled', False) on a config section without checking it was a dict first, raising AttributeError on a malformed config value. - Minor: pop the config-change callback only after attempting to unsubscribe it, and log the exception in the config-read fallback instead of swallowing it silently. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(display-controller): address review findings on the hot-reload PR - Idle-wait tick was a fixed 30s sleep, delaying pickup of a plugin enabled via the web UI while no modes were active. Shortened to ~1s so it's roughly as responsive as the per-frame check once modes exist. - _unregister_plugin popped the config-change callback from _plugin_config_callbacks even when config_service.unsubscribe() raised, losing the only reference to it. Now only pops on a successful unsubscribe. - _pending_plugin_reconcile was cleared before _reconcile_enabled_plugins() ran, so a retryable failure (e.g. plugin discovery erroring) silently dropped the enable/disable request. _reconcile_enabled_plugins() now returns True/False and the caller only clears the flag on True. - Added a warning log for the malformed-config case (a plugin's config section present but not a dict) so it's actually visible, and updated the existing test to assert it via caplog. Left the broad `except Exception` around config_service.unsubscribe() as Exception -- the current implementation is a simple lock+dict/list op that doesn't document or realistically raise a narrower type, so this is a defensive catch-all, not user error handling. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: ChuckBuilds <charlesmynard@gmail.com> |
||
|
|
fefc2d44a2 |
feat(display): double-sided mode — mirror one screen across the panel chain (#375)
* feat(display): add double-sided mode to mirror one screen across the panel chain
Renders a plugin once at a logical (per-screen) size, then tiles the
rendered frame across the full physical chain so two (or more) panels show
identical content. A 128x32 chain configured with 2 copies drives two 64x32
screens; vertical axis splits parallel outputs instead of the chain.
Plugins size themselves from matrix.width/height, so a thin _LogicalMatrix
proxy reports the logical size while delegating every real operation
(CreateFrameCanvas, SwapOnVSync, brightness, Clear) to the physical matrix —
no plugin changes required. Duplication is a single PIL paste per copy in
update_display(), so render cost is unchanged.
Config: display.double_sided { enabled, copies, axis }. Invalid config
(non-divisible dimension, bad axis/copies) logs a warning and falls back to
single-screen rather than failing to light up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(web): expose double-sided display config in the settings UI
Adds a Double-Sided Display section to the Display settings page (enabled
checkbox, copies, horizontal/vertical axis) and wires the save handler to
persist it under display.double_sided. Validates copies (2-8) and axis,
returning 400 on bad input; an omitted checkbox is saved as disabled.
Like the other hardware fields, changes take effect after a display restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(display): add type hints and docstrings to double-sided proxy
Addresses CodeRabbit nits: sort _LogicalMatrix.__slots__ (Ruff RUF023),
annotate the proxy's __init__/properties/dunders and _resolve_double_sided's
return type, and add docstrings to the property/dunder methods.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
d297dd6217 |
feat(display-controller): round-robin between simultaneous live-priority games (#372)
_check_live_priority() was stateless first-match-wins: it returned the first plugin in registration order with live content, and the post-dwell hold pinned the carousel to it, so when two games were live at once (e.g. a baseball game and a soccer match) the second never showed until the first ended. Add _collect_live_modes() (all currently-live modes, deduped, in registration order) and give _check_live_priority an 'advance' flag. The main rotation calls it with advance=True, which returns the live mode after the one currently shown -- using current_display_mode as the cursor -- so each dwell advances to the next live game and they take turns. The Vegas coordinator and the vegas-active check keep the default non-advancing peek (advance=False), so they only report whether any game is live without spinning the cursor. should_rotate and _apply_live_priority are unchanged; a single live game still holds as before. Adds regression tests to TestDisplayControllerLivePriority. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
d22d0a3754 |
fix(plugins): stop core updates from resurrecting uninstalled built-in plugins (#368)
* fix(plugins): stop core updates from resurrecting uninstalled built-in plugins Built-in plugins (e.g. web-ui-info, starlark-apps) are committed into the repo under plugin-repos/. When a user uninstalls one, a subsequent core `git pull` update restores the committed files, so the plugin reappears on every update. The update endpoint stashes the deletion and never pops it, and `git pull` faithfully restores any committed file whose deletion was never committed — so excluding plugin-repos/ from the stash can't fix this (it would only make `git pull --rebase` fail on a dirty tree). Add a persistent uninstall registry (config/uninstalled_plugins.json, gitignored) that survives restarts, unlike the existing in-memory tombstone: - Uninstall records the plugin id; install clears it. - purge_uninstalled_plugins() re-removes any recorded plugin whose directory reappears on disk; called after a successful git-pull update and at web startup (covers manual `git pull` on the Pi too). - The state reconciler also refuses to auto-repair a persistently uninstalled plugin. Wires up mark_recently_uninstalled in the uninstall flow (previously only referenced by tests) via the new persistent record. Adds regression tests covering record/forget/purge lifecycle, persistence across manager instances, and corrupt-registry tolerance. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(plugins): validate uninstall-registry ids and lock registry writes Address review feedback on the persistent uninstall registry: - Critical: validate plugin ids on read/record and add a containment guard in purge_uninstalled_plugins. A corrupt or hand-edited registry entry of "" resolves to the plugins root, so purge could have deleted every plugin; traversal ids ("..", "../x") could target paths outside the root. Invalid ids are now dropped on read, refused on record, and never removed unless the path is a direct child of the plugins directory. - Major: guard record/forget read-modify-write with a lock so concurrent install/uninstall requests can't lose updates. - Minor: narrow the startup and post-update purge exception handlers from bare Exception to (OSError, RuntimeError). Adds regression tests for empty-id, traversal-id, and invalid-record cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
cf28a8c0d5 |
fix(display): restore early-continue guard for mid-loop mode changes (#367)
When the display loop breaks early because current_display_mode changed (on-demand activation, live priority, etc.), it would fall through to the "honour minimum duration" sleep for the *previous* mode — blocking for up to that mode's full display_duration (default 30s) without polling on-demand requests or re-checking the mode. New modes could sit unrendered for up to 30s, or get clobbered by a queued stop request before ever displaying. This guard was added in #298 to fix #196 (live priority not interrupting long display durations) and was accidentally dropped in #330 as collateral damage of an unrelated time.monotonic() -> time.time() cleanup in the same diff hunk. Restoring it fixes both the original #196 regression and a new symptom found via the on-air MQTT plugin, where ON/OFF toggles could be delayed by up to 30s or missed entirely depending on timing within the previous mode's display cycle. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
313e35a98f |
Add cross-size/cross-screen plugin safety harness (#361)
* feat(testing): add cross-size/cross-screen plugin safety harness Render every plugin across all supported matrix sizes (64x32, 128x32, 128x64, 256x32) and every declared screen, failing on crashes, content drawn past the panel edge, or visual drift vs committed golden images. - BoundsCheckingDisplayManager: oversized-canvas overflow detection - harness.py: multi-size/multi-screen render engine + golden compare - scripts/check_plugin.py: CLI (functional+bounds, --out-dir, --update-golden, --freeze-time); render_plugin.py refactored onto shared loading helpers - test/plugins/test_harness.py + test_plugin_matrix.py (parametrized, honors per-plugin test/harness.json; skips when no plugins present) - MockCacheManager.cache_dir so cache-dir-using plugins load headlessly - .github/workflows/test.yml + docs/plugin-safety-harness.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(testing): address PR review feedback on plugin safety harness - check_plugin: friendly error for non-numeric --sizes; reject non-object --config / --mock-data JSON; sanitize plugin mode before using as a filename; stop --update-golden from masking crash/overflow failures - bounds_display_manager: pad the canvas out to the largest supported panel (not a fixed 16px) so far-overshoot coordinates are caught, not clipped - harness: merge config_schema defaults inside render_plugin_matrix; surface update() failures as a non-fatal warning + result field instead of a debug log; sanitize mode in golden_path - loading: fail fast when harness.json references a missing mock_data fixture - mocks: clean up the per-instance temp cache dir via weakref.finalize - test_plugin_matrix: add a discovery guard that fails when LEDMATRIX_REQUIRE_PLUGINS=1 but none found (still skips locally); type hints - bound test deps with upper version pins for deterministic CI Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(testing): render plugins across arbitrary panel sizes, not a fixed list Addresses maintainer feedback that there is no canonical set of supported panel sizes — a build can be any size/configuration (square, 2x2, 4x4, 8x2, long strips, tall stacks). - sizes.py: SUPPORTED_SIZES -> DEFAULT_TEST_SIZES (back-compat alias kept), reframed as a representative SAMPLE of real panel-grid arrangements rather than an authoritative list; add parse_size_token / coerce_sizes / resolve_test_sizes helpers - sizes are now fully overridable: LEDMATRIX_TEST_SIZES env (global, e.g. test on your exact hardware) > per-plugin harness.json "sizes" > default sample; CLI --sizes unchanged - bounds_display_manager: pad the canvas to the largest panel IN THE CURRENT RUN (via overflow_extent) instead of a hardcoded max, so cross-size overflow detection scales to whatever sizes a run uses - harness: compute per-run extent and thread it into the bounds manager - tests: arbitrary-shape + size-parsing/precedence coverage - docs: rewrite "Supported sizes" -> "Sizes: a sample, not a fixed list" Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(testing): fail the harness on non-connectivity update() errors Addresses the remaining review thread: recording every update() exception as a non-fatal warning still let a real update() regression pass green as long as display() survived. Now update() failures are classified — a tolerated set of connectivity errors (ConnectionError/TimeoutError/socket/ssl/urllib/http/ requests) is recorded non-fatally (expected with no network in CI), while any other exception is treated as a genuine bug and fails that render. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(security): pin actions to SHAs and disable checkout credential persistence Addresses the CodeRabbit/zizmor workflow-hardening finding: pin actions/checkout and actions/setup-python to full commit SHAs and set persist-credentials: false on checkout to reduce supply-chain and token-exposure risk. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(testing): validate positive sizes; narrow requests import except Two review findings: - sizes.py: parse_size_token / coerce_sizes now reject non-positive dimensions (0x32, -64x32) with a clear message instead of passing invalid sizes downstream (CodeRabbit). - harness.py: the optional `requests` import now catches ImportError specifically and logs instead of `except Exception: pass`, clearing the Codacy medium "Try, Except, Pass" (harness.py L52) and Ruff S110/BLE001. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
b9dcbb5152 |
fix(display): resume rotation where it left off after live priority ends (#362)
When a live-priority plugin (e.g. live sports, flights overhead) preempted the rotation, the controller overwrote current_mode_index with the live plugin's index. Once live priority ended, rotation continued from after the live plugin's mode, skipping every mode between the interrupted position and the live plugin. With a live plugin late in the order, modes just before it were starved indefinitely. Save the rotation position on the initial live-priority switch and restore it when live priority ends, in a new _apply_live_priority() helper. Add regression tests covering resume, no-double-save during the hold, and the idle no-op. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
eedf680a8c |
perf: display pipeline optimizations — caching, logging, scroll, text width (#358)
* docs(core): add module and class docstrings to the 5 undocumented core files
Fills the only significant documentation gaps found during a codebase
audit. All other core files (plugin_system/, logging_config.py, etc.)
already have complete module, class, and function docstrings.
Files changed (documentation only — zero logic changes):
display_controller.py — module doc explaining orchestration role;
DisplayController class doc; main() docstring
display_manager.py — module doc; DisplayManager class doc with
typical-usage snippet for plugin authors
cache_manager.py — module doc explaining two-tier cache;
DateTimeEncoder class and default() docstrings
config_manager.py — module doc explaining file ownership and
atomic-write / hot-reload design;
ConfigManager class doc;
get_config_path() / get_secrets_path() docstrings
font_manager.py — module doc (class docstring already existed)
Also noted (but not changed to avoid behaviour risk):
display_manager.py and font_manager.py use logging.getLogger() directly
instead of the project's get_logger() wrapper. display_manager.py also
calls setLevel(logging.INFO) immediately after, which would be lost if
switched to get_logger().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* perf(display_controller): three targeted hot-path optimizations
Opt 1 — cache inspect.signature() per plugin_id
inspect.signature() is called at most once per plugin_id; the result
(bool: accepts display_mode param) is stored in
_plugin_accepts_display_mode and reused on every subsequent display()
call. Eliminates all reflection from the display path at runtime.
Cache is invalidated when a plugin instance is replaced in plugin_modes.
Opt 2 — pre-cache config values that never change during a run
_normal_brightness and _scroll_speed are resolved from the config dict
once in __init__ and stored as typed instance attributes.
- Removes 2+ chained dict.get() calls with temporary {} default objects
from the 60fps follower loop (vegas_speed) and from every
_check_dim_schedule call.
- current_brightness init now uses _normal_brightness directly.
Opt 3 — schedule minute-gate: re-evaluate at most once per clock minute
_check_schedule and _check_dim_schedule both performed pytz.timezone(),
datetime.now(), strftime(), and datetime.strptime() on every outer loop
call. Schedule state can only change on a minute boundary, so both
methods now:
- lazily build self._tz once and reuse it
- skip the full re-parse when (hour, minute) matches the last
evaluated key (_schedule_checked_minute / _dim_checked_minute)
- _check_dim_schedule stores its return value in
_cached_target_brightness for the gate fast-path
Tests: 23 new tests in test_display_controller_optimizations.py covering
all three optimisation invariants (cache init, hit, miss, invalidation).
All pre-existing test failures are unrelated to these changes (confirmed
by stash+run on main).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: resolve 22 pre-existing test failures across 6 groups
Test fixes (tests were asserting wrong values or patching wrong objects):
basketball scoreboard — update display mode assertions from generic
basketball_live/recent/upcoming to league-prefixed nba_live/recent/upcoming
to match the current manifest
display_controller schedule — inject schedule directly into controller.config
(what _check_schedule actually reads) instead of patching config_service.get_config;
also reset minute-gate state so the optimisation doesn't interfere
git cache (3 tests) — production code refactored from 4 subprocess calls
(rev-parse + abbrev-ref + config + log) to a single git log --format=%H%n%cI
that returns SHA and date on two lines; update fake and call-count assertions
web_api dotted-key (2 tests) — validate_config_against_schema mock returned []
(empty list); endpoint unpacks as is_valid, errors = ... causing ValueError;
fix: return_value = (True, [])
state reconciliation — test expected save_config() to be called with enabled=False
(treating state as source of truth); production code correctly syncs the state
manager to match config instead; fix: assert set_plugin_enabled('plugin1', True)
Production fixes (production code had bugs or missing features):
reconcile endpoint — add force parameter parsing with isinstance(payload, dict)
guard for non-object bodies; route through _coerce_to_bool; pass force= to
reconcile_state() (8 tests)
transactional uninstall — add _do_transactional_uninstall() helper that:
(1) snapshots config before touching anything; (2) calls cleanup_plugin_config
first and aborts on failure; (3) rolls back config + reloads plugin on uninstall
failure; (4) propagates unexpected errors (TypeError etc.) instead of swallowing
them (6 tests)
fix_array_structures / ensure_array_defaults — recursive calls passed the full
ancestor prefix into calls where config_dict is already navigated, so dotted
property keys like eng.1 caused parent_parts.split('.') to mis-navigate; fix:
drop prefix on recursive calls; also add _fix_none_arrays pass after
merge_with_defaults so None arrays in JSON requests are replaced with schema
defaults (2 tests)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* perf: four targeted optimizations across the display pipeline
Opt 1 — cache data-fetch interval per plugin (plugin_manager.py)
_get_plugin_update_interval fell back to config_manager.get_config()
(a full dict copy) when the manifest lacked an interval. Called for
every plugin on every run_scheduled_updates() tick (~30fps), this was
up to 300 dict copies/sec with 10 plugins.
Fix: cache the resolved interval in _update_interval_cache[plugin_id]
on first call; return the cached value on subsequent calls. Cache is
cleared on load_plugin and unload_plugin.
Opt 2 — demote noisy per-cycle INFO logs to DEBUG (display_controller.py)
Four logger.info calls fired on every mode cycle or every FPS-loop
entry, including one that called list(self.plugin_modes.keys())
unconditionally (allocating a list every outer loop iteration).
- "Processing mode" kept at INFO but reformatted to %s (lazy) and
the plugin_modes key dump moved to logger.debug
- "Attempting/Got cycle duration" → logger.debug
- "Entering high/normal FPS loop" → logger.debug
Mode name at INFO is preserved for black-screen troubleshooting.
Opt 3 — use Image.frombytes instead of Image.fromarray in scroll hot path
(scroll_helper.py)
Image.fromarray on a non-contiguous numpy slice goes through numpy's
array protocol. Image.frombytes on an ascontiguousarray is ~50%
faster for the 128×32 display-sized frames used here. Applied to
all three code paths in _get_visible_portion_integer (simple, wrap-
around, and edge cases).
Opt 5 — cache get_text_width per (text, font) pair (display_manager.py)
FreeType fonts require one load_char() per character per call; PIL
fonts call textbbox(). Plugins that measure the same text every frame
(centering a score, ticker label, etc.) were re-measuring from scratch
on every display() call.
Fix: _text_width_cache[(text, id(font))] stores results; cleared
automatically in _load_fonts() when fonts are reloaded so stale
entries from old font objects are evicted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(scroll_helper): fix edge-case bug exposed by frombytes switch
The previous commit replaced Image.fromarray with Image.frombytes in
_get_visible_portion_integer. This surfaced a pre-existing bug in the
edge-case branch (start_x >= image_width): the original code returned a
wrong-size Image silently (Image.fromarray accepts a too-short array);
Image.frombytes raises ValueError instead.
Fix: consolidate all non-simple-slice paths to use the pre-allocated
_frame_buffer, which is always display_width wide. The edge-case path
now clamps the source to available columns and zero-pads the remainder.
Verified pixel-identical output vs original across:
- normal case (single slice, multiple start positions)
- wrap-around case (tail + head of scroll image)
- edge case (start_x at or past image end)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address CodeRabbit review comments on PR #358
1. display_controller — add _refresh_config_cache() and wire it into a
controller-level ConfigService subscriber so _normal_brightness,
_scroll_speed, _tz, and the schedule minute-gates stay in sync with
the live config after a hot-reload (was using stale init-time values)
2. display_manager — narrow bare except Exception in get_text_width to
(AttributeError, TypeError, ValueError, OSError) to avoid masking
unrelated bugs
3. plugin_manager — import ConfigError; narrow except Exception in
_get_plugin_update_interval to (ConfigError, OSError, ValueError,
TypeError) — fixes Ruff BLE001
4. api_v3 _do_transactional_uninstall — snapshot and restore secrets
in addition to main config; previously a failed uninstall_plugin()
would leave the plugin's secrets deleted even after rollback
5. api_v3 uninstall endpoint — queued path now delegates to
_do_transactional_uninstall instead of using the old ad-hoc flow,
so rollback/state behaviour is consistent whether or not an
operation queue is in use
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(display_controller): move _plugin_accepts_display_mode init before plugin loop
Codacy HIGH: 'access to member before its definition' — the dict was
initialised at line 441 but accessed at line 364 inside the plugin-
loading loop, both within __init__.
Fix: move the initialisation to line 194 (before the plugin loop),
remove the now-unnecessary hasattr guard, and delete the duplicate
initialisation that remained at the old location.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
|
cac9644b6d |
fix(plugin-loader): auto-detect new dependencies via requirements.txt hash (#355)
* fix(plugin-loader): detect new deps via requirements.txt hash instead of empty marker The .dependencies_installed marker was an empty file, so adding a new package to requirements.txt (e.g. astral in ledmatrix-weather v2.3.0) never triggered a pip re-install on existing installs — the file existed so the check returned early. The marker now stores a SHA-256 hash of requirements.txt. On every plugin load, the loader compares the current hash to the stored one; a mismatch (or missing marker) triggers pip install and writes the new hash. store_manager._install_dependencies() also writes the hash marker after a store install/update so the loader skips a redundant pip run on next boot. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(plugin-loader): address CodeQL path expression and I/O error handling - Add explicit relative_to() containment check after path resolution so CodeQL recognizes the plugin directory boundary (fixes 4 CodeQL alerts: Uncontrolled data used in path expression, lines 168/172/189/205) - Wrap requirements_file.read_bytes() in try/except OSError — on Raspberry Pi with flaky SD card storage this can fail; returns False with a clear log - Wrap marker_path.read_text() in try/except OSError — a corrupted marker falls through to a clean reinstall instead of crashing - Wrap both marker_path.write_text() calls in try/except OSError — pip already succeeded at this point so a marker write failure should not return False or propagate through the generic exception handler Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(plugin-loader): use realpath+startswith containment check for CodeQL path-injection Replace relative_to() (not recognised by CodeQL as a path sanitiser) with the os.path.realpath() + startswith() pattern that CodeQL explicitly models as sanitising py/path-injection. - Add plugins_dir optional param to install_dependencies() and load_plugin() - PluginManager.load_plugin() passes self.plugins_dir as the trusted anchor; install_dependencies() validates that the resolved plugin_dir starts with the resolved plugins_dir before any file I/O - Replace all Path.read_bytes/read_text/write_text/exists with open() and os.path.isfile() so the sanitised string paths flow directly to file ops without re-introducing taint through Path object conversion Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(plugin-loader): fail-fast when install_dependencies returns False Previously the boolean result was silently discarded, so a failed pip install would log a warning but continue attempting to import the plugin module — resulting in a confusing ModuleNotFoundError instead of a clear dependency failure message. Now raises PluginError with plugin_id and plugin_dir if dependency installation fails, stopping the load before the import is attempted. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(plugin-loader): use basename+reconstruct to satisfy CodeQL py/path-injection startswith() is a validation check in CodeQL's model, not a sanitiser — taint still flows through plugin_dir_real to the file operations. os.path.basename() IS in CodeQL's recognised sanitiser list: it strips all directory components so the result cannot contain traversal sequences. Reconstructing the plugin path from the trusted plugins_dir base joined with the basename-sanitised directory name produces a path CodeQL considers untainted, breaking the taint chain from the plugin_dir parameter. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(plugin-loader): guard against empty basename when plugin_dir resolves to fs root If plugin_dir somehow resolves to '/' or a bare drive root, os.path.basename() returns '', causing safe_plugin_dir to equal plugins_dir_real and the isdir() check to pass incorrectly. Reject early with a clear error in that case. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
35c540d0e0 |
fix(reconciler): prefer config.json over state manager for enabled mismatch (#353)
* fix(reconciler): prefer config.json over state manager for enabled mismatch When the enabled state in config.json and plugin_state.json diverged, the reconciler was syncing config.json to match plugin_state.json (state manager wins). This silently disabled plugins on every restart whenever the state file had an outdated enabled=false entry — most commonly after an uninstall+reinstall cycle, where the reinstall left the plugin in the installed-but-not-enabled state while config.json still had enabled=true. Flip the sync direction: update plugin_state.json via set_plugin_enabled() to match config.json instead. config.json is the user-editable source of truth; the state file is an internal tracker that should follow it when they disagree. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(reconciler): return set_plugin_enabled result instead of always True Capture the boolean returned by set_plugin_enabled() and propagate it so reconciliation accurately reflects failure to update the state manager. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |