mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-08-01 16:58:06 +00:00
2486bdb249cfac97ed85e6af5ebcffb76b0b17e5
1873
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Claude
|
2486bdb249 |
fix(sports): stop dropping hockey and baseball events on optional feed keys
Both bugs were pinned AS-IS by the B0 characterization suite so this
phase could change them knowingly. Both fixes are adoptions of code the
corresponding plugins already ship, not new inventions.
Hockey: the extractor read competitor["statistics"] unguarded, so a
competitor arriving without that array raised KeyError inside the
generator and the WHOLE event was discarded -- valid scores and status
included. Shot/save counts now default to 0, which is already what the
suite expects for an empty statistics array.
Baseball: for live games the extractor read game_event["status"], the
event TOP-LEVEL status, to get the inning. Real ESPN events duplicate
status there, but MiLB events (synthesized from the MLB Stats API into
an ESPN-like shape) populate only the competition-level one, so the
lookup raised a bare KeyError and dropped the event. It now reads the
competition-level status that _extract_game_details_common has already
validated, so it cannot be missing at that point.
The two characterization tests that pinned the old behaviour are
rewritten to assert the fix rather than deleted, so the suite still
documents the edge case -- and still totals 94.
CHANGELOG records these plus the live-clock change from
|
||
|
Claude
|
aaabc614cc |
feat(sports): promote the nine universal methods into the base classes
Phase B1b of docs/SPORTS_UNIFICATION.md. Every method here is present in
all nine bundled plugin sports.py copies and absent from core, so this is
reuse of code the fleet already agreed on — not new behavior. The
promotions are inert until B5: the plugins' own overrides still run.
SportsCore: cleanup, _get_layout_offset, _load_custom_font_from_element_config
SportsUpcoming: _select_games_for_display
SportsRecent: _get_zero_clock_duration, _clear_zero_clock_tracking,
_select_recent_games_for_display
SportsLive: _is_game_really_over, _detect_stale_games
Where the copies disagreed, the canonical form was chosen on evidence and
the genuine per-sport differences became seams rather than branches:
- _favorite_key(game, side) -- NRL matches favorites on team id because its
abbreviations are ambiguous (NEW is both Newcastle Knights and New
Zealand Warriors). Default is the abbreviation; NRL overrides. Core never
learns the string nrl.
- FINAL_PERIOD / CLOCK_COUNTS_DOWN -- hockey ends in P3, and soccer/afl/nrl
clocks count UP, so 0:00 means kickoff, not expiry.
- _config_schema_path() / _font_root() -- plugin-supplied locations, never
derived from this module's __file__.
BEHAVIOR CHANGE (baseball, ufc): the rejected variant coerced a missing or
non-str clock to the literal 0:00 and then declared the game over at
period >= 4. MLB has no game clock and period is the inning, so live games
were being evicted from the 5th inning onward; UFC likewise. The promoted
variant skips the clock check when the clock is unusable -- it fails safe
(keeps showing the game) instead of failing destructive.
Also fixes a regression from the package move in
|
||
|
Claude
|
e591ceca48 |
refactor(sports): convert sports.py into a package (pure move)
Phase B1a of docs/SPORTS_UNIFICATION.md. src/base_classes/sports.py becomes a package so the upcoming capability modules have a home and diffs show their blast radius: sports/__init__.py re-exports the public API sports/core.py SportsCore sports/modes.py SportsUpcoming / SportsRecent / SportsLive No logic change: the 1515 class-body lines are byte-identical to the original (verified by concatenating the two modules and diffing against HEAD). Only module docstrings and the redistributed import blocks are new. MRO and __abstractmethods__ are unchanged, and every existing import site — including 'from src.base_classes.sports import SportsCore' in the sport subclasses, the skin tests, and the characterization suite — resolves through the package __init__. One test edit was required: the characterization suite monkeypatched 'src.base_classes.sports.get_background_service', which is no longer a module attribute on a package. Retargeted to 'src.base_classes.sports.core.get_background_service' — the module whose globals SportsCore.__init__ actually resolves, so the patch is effective exactly as before. No test logic or assertion changed. Also adds docs/SPORTS_UNIFICATION.md: the architecture for the whole B1-B5 sequence — how upgradability (guarded imports, capability probing, frozen view-model keys, the sunset rule), reusability (promote only what all nine copies share), and modularity (capabilities as opt-in mixins rather than config branches, variants as named strategies, sport-unique code as declared override points) are kept as three separate mechanisms. Verified: characterization + skin 94 passed; the 10-file unit suite 338 passed; test/plugins 60 passed — all identical to pre-change counts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4 |
||
|
Claude
|
64e8f87f86 |
ci: restrict the test workflow's GITHUB_TOKEN to contents:read
CodeQL flagged the new unit-tests job for running with the default unrestricted token; the pre-existing job had the same exposure. Both jobs only check out the repo and run pytest, so a workflow-level contents:read is sufficient. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4 |
||
|
Claude
|
1a0792225b |
test: characterization suite for src/base_classes/sports.py ahead of unification
Pins current behavior before the planned merge of the nine drifted plugin copies back into this ancestor: the _extract_game_details_common key contract per sport (reusing GUARANTEED_KEYS from the skin tests), update() flows for upcoming/recent/live against cache-seeded fixtures under frozen time, rendering smoke per mode class, and guard rails on the skin-system seam. Five surprising behaviors are pinned AS-IS and flagged in comments so the merge changes them knowingly or not at all: is_upcoming also matching status.type.name; hockey dropping events whose competitors lack 'statistics'; baseball reading the event-level status for innings; no past-date filter in upcoming; and favorites-only mode with an empty favorites list showing nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4 |
||
|
Claude
|
5fe9c07f80 |
feat: ship src/element_style — the per-element style resolver plugins already expect
Three plugins (of-the-day, ledmatrix-music, football-scoreboard) import src.element_style behind guarded try/except with classic fallbacks, but the module never existed in core, so the richer per-element styling UI those code paths implement has been dormant. This lands it: - ElementStyleResolver.style() resolves per-element font/size/color with the key semantic the consumers encode: a config value counts as user-forced only when it differs from the schema default (the web UI bakes defaults into config.json on save), and untouched configs resolve to exactly the caller's classic values — byte-identical rendering, proven by of-the-day's committed goldens passing unchanged. - defaults_from_schema_file parses both declaration forms (the compact x-style-elements map and hand-written customization blocks). - expand_style_elements() expands x-style-elements into full config blocks; schema_manager.load_schema() applies it (guarded, no-op for schemas without the declaration) so the config form and defaults merging see the expanded UI. - Fonts resolve cwd-independently with (path, size) caching; .bdf loads via freetype like FontManager; nothing in the module raises out of style(). Verified: 31 new unit tests; of-the-day's previously-skipped 9-test spec suite now runs and passes; football's resolver tests pass (27); music's 38 plugin tests pass; schema-manager suites pass (43). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4 |
||
|
Claude
|
dc659940ab |
ci: enroll the core unit suites in a dedicated job
The existing workflow ran only the three plugin-harness suites; the skin-system, font-manager, data-source, extractor, scroll-helper, adaptive-layout, and loader-compat suites (266 tests) existed but never ran in CI, so a refactor of src/base_classes or src/common could regress them silently. Also enrolls the new sports characterization and element-style suites landing in this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4 |
||
|
Claude
|
3f4e915af7 |
docs: seed CHANGELOG.md with the module-availability release discipline
The plugins monorepo's sunset rule ('delete a bundled fallback copy only
when the manifest floors on the first core release shipping the module')
needs core module additions recorded against version numbers. Seeds the
changelog at 3.1.0 and documents the discipline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4
|
||
|
Claude
|
1baebd2d09 |
fix(fonts): resolve asset paths against the install root, not the cwd
FontManager built its catalog from cwd-relative paths ('assets/fonts'),
so any process started outside the install root — the plugin safety
harness on CI being the recurring case — found no fonts and silently
degraded every plugin to PIL's default face. Several plugins grew
per-plugin workarounds for exactly this (countdown, text-display,
tide-display in the plugins monorepo).
Catalog population now falls back to the install root derived from this
module's location when the cwd-relative path is missing; behavior when
running from the install root is unchanged. Verified: resolve_font
returns the real FreeType face from a foreign cwd, and the full unit
suites (266 tests) pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FgbA8SMutQQpXkMG8LMmC4
|
||
|
|
5b45f35888 |
Vegas mode: reclaim dead space and pace the rotation (#423)
* Vegas mode: reclaim dead space and pace the rotation On a wide panel Vegas mode spent much of its time showing black. At 50px/s on a 512px display, one display width of blank is 10.2 seconds, which makes several long-standing behaviours expensive: - ScrollHelper prepended a full display width of black as an "initial gap", charged once per cycle — 10.2s of black at the start of every rotation. - Plugins without get_vegas_content() are captured off a full-display canvas, so their blank margins entered the ticker too. Measured: of-the-day drew 35px of "No Data" on a 512px canvas (92% blank), youtube-stats 142px of content with 185px of black either side. Only the scroll_helper path had any trimming. - Cycle transitions deliberately pushed a blank frame and then recomposed synchronously: 84ms at best, 4.8s at worst, every millisecond of it black. - buffer_ahead doubled as the cycle size, so a 21-plugin install showed 3 plugins per cycle and took ~7 cycles to come around. - separator_width was applied between every image rather than at plugin boundaries, so a per-row ticker like the F1 scoreboard (116 images, which it renders 4px apart internally) got a 32px chasm between each row — and the width budget didn't count those gaps, so the plugin quietly occupied far more of the panel than intended. Changes: - src/vegas_mode/geometry.py: numpy column-ink primitives shared by the trimmer and the audit tool, so the number reported is the number acted on. A Python per-column loop over a 17,000px strip is far too slow for the render path. - PluginAdapter trims every content path, not just scroll_helper. Only outer edges are cropped: interior blank columns are the plugin's own layout (logo left, score right) and closing them would corrupt the design. A plugin on a non-black background is inherently unaffected. - ScrollHelper.create_scrolling_image takes an explicit lead_gap, still defaulting to display_width so the many standalone-ticker callers are unchanged. Vegas passes lead_in_width (default 0). - Cycle end holds the last rendered frame instead of blanking, turning the recompose into a brief freeze rather than the panel switching off. - plugins_per_cycle (default 6) is split from buffer_ahead, which goes back to being only a prefetch low-water mark. - max_plugin_width_ratio (default 3x display width) caps one plugin's share of a cycle. Overflow is deferred, not discarded: a rotation offset advances each fetch so later rows appear on subsequent cycles. Single oversized images are cropped at a blank column so the cut misses glyphs. - Composition groups images by plugin: rows are joined by intra_plugin_gap (default 8) and separator_width applies only between plugins. The width budget now counts those gaps. - Plugin data updates no longer run on the Vegas render path. All new settings are user-configurable in Display -> Vegas Scroll, including min/max cycle duration and dynamic duration, which previously existed in code but were reachable only by hand-editing config.json. Measured with scripts/dev/vegas_audit.py on a 512x64 panel: mean ink coverage 42.7% -> 69.4% fully blank 5.9% -> 0% reads as empty 13.6% -> 0% worst blank stretch 4.8s -> 0s full rotation 414s -> 123s plugins per cycle 3 -> 6 Note the metric choice: a "fully blank" scan (>=95% black viewport) reported only 0.4% and badly understated the problem, because two full-width segments with mid-canvas content never fully blank the viewport — they hold it at ~28%. window_coverage_stats grades every viewport position by how much ink it carries, which is what tracks perceived dead time. Known remaining: cycle transitions still freeze ~3.5s while the next cycle is fetched. Fixing that needs background prefetch, which is deferred because the fallback-capture path mutates the shared display_manager.image and racing it against the render loop risks torn frames. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Drop unused Optional import from the vegas audit script Flagged by Codacy (F401). Any, Dict and List are all still used. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Align Vegas API bounds with validate(), fix audit config plumbing Both from review feedback on #423. The web API's accepted ranges disagreed with VegasModeConfig.validate(), which is what actually gates Vegas starting: scroll_speed 1-100 -> 1-200 (a slider value of 150 returned 400) separator_width 0-500 -> 0-128 target_fps 1-200 -> 30-200 buffer_ahead 1-20 -> 1-5 The three loose ones were the dangerous direction: the value saved with a 200, then VegasModeCoordinator.start() failed validation with only a log line, so the ticker silently never ran. The UI already matched validate() in all four cases, so the API was the odd one out. test_vegas_api_bounds_match_validate parses the numeric_fields map out of api_v3 and asserts every bound against validate(), plus that validate() accepts both endpoints and rejects just outside them, so these cannot drift apart again. That test immediately caught a missing upper bound on min_plugin_width, now added — unbounded it would drop every segment and leave a blank ticker. Separately, vegas_audit.py constructed PluginAdapter without the config, so it fell back to VegasModeConfig() defaults and would report trimming and width-budget behaviour that differed from the user's config.json. It now passes the loaded config exactly as the coordinator does. This is the same class of drift the explicit lead_gap and grouping arguments already guard against. Output is unchanged on a rig whose config matches the defaults. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Vegas mode: render plugins narrower, space rows by measured separation Trimming reclaims blank margins but cannot compact a layout that genuinely spans the display — a five-column forecast, a progress bar drawn at 100% width, a stat block with the panel's whole width between its elements. Those need the plugin to make different layout decisions, which means telling it the screen is narrower while it renders. DisplayManager.render_size() presents a smaller logical canvas for the duration of a Vegas content fetch, reusing the same _LogicalMatrix indirection double-sided mode already relies on so plugins see a consistent size from every accessor. Plugins that size themselves from matrix.width need no changes at all; one that wants to be explicit can read the new BasePlugin.get_vegas_render_width(). Width is a percentage so a single setting travels across panel sizes: vegas_scroll.render_width_pct globally, or vegas_width_pct in an individual plugin's config. Measured on a 512x64 panel with real data: ledmatrix-weather 1536px -> 576px (forecast becomes narrow cards) youtube-stats 353px -> 199px (2% blank left, so genuinely compact) geochron 453px -> 153px (ink density rises to 100%) ledmatrix-flights 950px -> 740px The youtube-stats figure is the clearest evidence the layout itself changed rather than being cropped: at full width the content had to be trimmed from 512px to 353px, whereas at 40% it arrives with almost no blank to reclaim. Row spacing is now measured rather than added. A flat gap gets it wrong in both directions at once — content drawn flush to its own edges ends up nearly touching (reported for recent sports scores, which sat 8px apart), while content already carrying wide margins gets pushed even further out. separation_gap() measures the blank each pair already has and adds only the shortfall, up to min_content_separation (default 24). intra_plugin_gap stays as a floor applied regardless. Two tests shipped in the previous commit encoded the old flat-gap arithmetic and are updated to the measured semantics, including one renamed to reflect that zero intra_plugin_gap alone no longer butts rows together. Also fixes a real bug found while testing: the harness display manager had no render_size(), and because the adapter catches broadly that surfaced as "no content" rather than an error, silently dropping five plugins. Added the context to VisualTestDisplayManager for parity, and _render_at() now degrades to a no-op on any display manager lacking it, so a third-party or older harness loses the narrowing rather than the content. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Vegas mode: end cycles before the wrap, keep the width budget honest Three fixes, the first a regression from lead_in_width defaulting to 0. get_visible_portion wraps: once scroll_position + display_width passes the end of the strip it fills the right of the frame from the *head* of the same strip. So the final display_width of travel showed the cycle's first plugin re-entering on the right while its last plugin exited on the left, and the recompose that followed replaced both at once. On a 512px panel at 50px/s that was 10.2s of two plugins on screen at once, ending in a hard cut — reported as the ticker "switching mid-scroll" from F1 to news. That used to be invisible because the strip began with a full display_width of blank, so the wrapped-in region was black. Removing that blank (it was 10s of dead panel per cycle) exposed the wrap. Cycles now end one display width earlier, before any wrapped content appears, clamped for strips no wider than the display so they don't complete instantly and spin the recompose loop. Verified on hardware: a 3936px strip now completes at 68.5s, exactly (3936 - 512) / 50. Second, auto_trim=False also skipped the width budget, which is an unrelated concern — turning off margin cropping should not let one plugin hold the panel for minutes. Seen in the field: the F1 scoreboard contributed 116 images and 14,848px untouched, giving a 33,821px cycle (11 minutes of content). The budget now applies regardless of trimming; with it restored that cycle is 6,362px. Third, the budget accounted for row gaps using the flat intra_plugin_gap while the compositor had moved to measured separation, so it under-counted by up to (min_content_separation - intra_plugin_gap) per row and a many-row plugin overran its cap. Both now use the same separation_gap() rule, and a test asserts the composed block fits the budget end to end rather than trusting the two paths to agree. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Fix IndexError in find_blank_cut when the cut lands on the image edge A cut position after the last column is legitimate — _crop_to_budget asks for min(start + budget, img.width), which equals the width whenever the remaining strip is shorter than the budget. find_blank_cut clamped target to width but then walked leftwards starting at target itself, so ink[width] raised IndexError. Caught on hardware: it killed the ledmatrix-stocks fetch, and because _fetch_plugin_content catches broadly that surfaced as the plugin silently contributing nothing for the cycle. Only reachable on the second or later pass of the rotating window over a single oversized image, which is why the existing tests missed it — they all exercised the first pass, where start is 0 and start + budget is comfortably inside the image. Added TestRotationAcrossMultipleCycles, which walks the window round several times and asserts content is never lost, plus direct coverage of find_blank_cut at and beyond the image edge. Both bounds now stop at width - 1 so neither direction can index past the end. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Only cut oversized segments at real gaps between items The width-budget crop snapped to the nearest blank column, and in rendered text the gap between two characters is a single column. So a cut routinely landed inside a word: the cycle showed "Wednesda" and the orphaned "y" turned up as a lone floating letter in the next cycle, positioned after whatever plugin happened to precede it. Measured on the clock-simple segment to confirm: its blank runs are [1, 1, 1, 1, 1, 8, 8] — five single-column letter gaps, every one of which find_blank_cut would happily have chosen. Cuts now only land in a run of at least min_cut_gap blank columns (default 6), which excludes letter spacing while still finding the gaps plugins put between items (the stocks ticker uses 32px, baseball 48px). Where no boundary falls inside the budget the cut waits for the next one and overruns, because splitting an item is worse than a slightly long segment. Continuous content is treated differently on purpose: an image with no internal gaps is a map or a chart, where any column is as good as another, so it is still cut to the budget exactly. The gap rule protects discrete items; letting a solid image escape the cap in its name would be wrong. blank_runs() is vectorised — 48ms for a 17,000px strip, against seconds for a per-column Python loop. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Hold capture_mode for every plugin render, not just narrowed ones The native content path only entered capture_mode when it was also narrowing the canvas, so at full width — which is every plugin without a vegas_width_pct override, i.e. most of them — a plugin calling update_display() while building its Vegas content wrote straight to the hardware. That is a visible flash mid-scroll, and it lines up with the flash reported at cycle transitions, when several plugins are fetched back to back. Suppression is now unconditional; the narrowing context stays separate because it is already a no-op at full width. Both contexts are reached through helpers that degrade to nullcontext when the display manager lacks them. That matters more than it looks: the adapter's handlers are deliberately broad, so an AttributeError from a missing context does not surface as an error — it surfaces as the plugin contributing nothing. Making the call unconditional without this turned 44 tests red for exactly that reason, all of them reporting lost content rather than the real cause. The test double now provides capture_mode and render_size too, so tests exercise the real contexts instead of silently taking the degraded path. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Vegas mode: one continuous strip instead of swapping cycles A cycle used to be a discrete strip that got replaced: motion stopped, every pixel was substituted at once, and the next group started with the viewport already full. That is the freeze, the flash and the jump. The strip is now extended rather than replaced. ScrollHelper gains append_content(), which adds items on the right without touching scroll_position or total_distance_scrolled, so motion continues and the next group simply arrives from the right. Because completion is measured against total_scroll_width, extending also defers completion — there is no longer a cycle boundary to see. drop_scrolled_prefix() reclaims what has gone past, keeping the strip bounded however long Vegas runs (observed 5,000-11,000px against an unbounded strip otherwise). It shifts total_distance_scrolled and total_scroll_width together so the completion arithmetic is unchanged, and refuses to run while the viewport is wrapping: wrapping reads the head of the strip into the right of the frame, so trimming the head there would visibly change the picture. A test caught that. Groups are prepared off the render thread. The constraint is that the canvas and the matrix proxy are process-wide mutable state, so narrowing or capturing through them from another thread would corrupt the frame the render loop is pushing. get_content() therefore takes offscreen_only: the background thread uses only paths that avoid the canvas, and anything needing it is marked and picked up on the render thread. That puts the expensive work (native renders of leaderboard and baseball cards, seconds each) in the background and leaves the cheap work (display capture, 40-600ms) in the foreground. DisplayManager's capture flag is now thread-local. As a shared flag, a background capture would have suppressed the render loop's own frame pushes for its duration, freezing the panel precisely when the point was to avoid a freeze. Canvas-bound plugins are drained one at a time rather than as a batch: six at once held the render thread for 1.75s. Drains are also spaced by two seconds while the lookahead is healthy, since taking them back to back turns one long stall into a run of short ones. When the strip is genuinely running short the throttle is ignored, because content matters more than smoothness there. Measured on hardware: zero cycle-complete swaps, drains landing 2-4s apart, lookahead holding at 1,200-3,500px, no errors. Set continuous_scroll false to restore the swap behaviour; the old path is intact. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Pace the Vegas frame loop adaptively: 31.5 -> 78.7 fps The loop slept a fixed frame_interval on top of however long the frame took, so at a measured 31.6ms per frame a flat 8ms of that was pure idle — a quarter of the budget spent not rendering. It now sleeps only the remainder of the budget. Measured on hardware: 31.5 fps to 78.7 fps sustained, with CPU going *down* from 150% to 127%. Scroll speed is unchanged at 49.9px/s against a configured 50, because motion is derived from elapsed time rather than frame count — this buys smoothness, not speed. Worth recording what the bottleneck was not: the per-frame render path measures 0.34ms in total (0.18ms for the numpy slice, 0.17ms for the dirty-tracking digest), which is a theoretical 2900 fps. Optimising any of that would have been wasted effort. The frame was idle, not busy. Also nices the prefetch thread. Its work is PIL and numpy that releases the GIL, so the scheduler can act on the priority, and without it the prefetch competes for the same cores as the render loop. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Sub-pixel scrolling: motion at the frame rate, not the pixel rate With integer positioning the number of distinct frames per second equals the scroll speed in px/s, however fast the loop renders. Measured at 50px/s and 78.7fps, 36% of frames were byte-identical: the extra frames cost work and bought no motion, and what was left was 50 discrete 1px steps a second. Two things were wrong with the pre-existing sub-pixel support. get_visible_portion never consulted sub_pixel_scrolling — it always took the integer path, so the flag and _get_visible_portion_subpixel were dead code. And that implementation needed scipy.ndimage.shift, which is not installed on the target devices (HAS_SCIPY is False there), so it would not have interpolated even if reached. Verified both: positions 1000.0 and 1000.5 produced identical frames either way. Blending is now wired up and implemented with numpy. Two details make it affordable: slice cached_array directly instead of building two PIL images only to convert them straight back (the naive version measured 15x the integer path), and use fixed-point uint16 multiply-add rather than float32, which suits the Pi's cores and gives finer weighting than the panel can resolve. Result 0.939ms against 0.237ms — 0.70ms added per frame, a 1065fps ceiling. Measured on hardware: 81.2 fps with blending on, against 78.7 with it off, so no cost within noise — and every frame is now a distinct position rather than one in three being a repeat. The trade is a slight horizontal softening of text, since each frame blends two positions. Set smooth_scroll false for maximum crispness. Also benchmarked and cleared as non-issues: extending the strip costs 9.4ms on an 11,000px strip and trimming 2.5ms, both under one frame at this rate. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Add overflow handling: keep ordered content whole instead of rotating a window The width budget split any oversized plugin by advancing a window each cycle. That is right for interchangeable items — news headlines, odds, stock prices — but wrong for ordered content: a league table showed ranks 1-6, then resumed at 7 two rotations later, which reads as out of order and out of context. Nobody needs rank 23 in a ticker; they need the top of the table, every time. overflow_mode chooses between them: rotate — advance a window each cycle so everything is seen eventually (unchanged default) truncate — always show the start and drop the rest, keeping ordered content coherent. Records no window state, so every pass starts at the top. Per-plugin vegas_overflow overrides the global setting, since one install has both kinds of plugin. Also adds per-plugin vegas_max_width_screens, so content that must stay whole can be given more room — or uncapped with 0 — without lifting the cap on every ticker. Applied on the test rig: f1-scoreboard and ledmatrix-leaderboard set to truncate, and baseball given 4.5 screens because it was showing 8 of 9 games when the whole slate needed only a little more room. Verified: F1 now reports "the first 10 of 116 ... the rest are not shown", baseball has dropped out of the budget log entirely, and stocks, odds-ticker and stock-news still rotate. Also corrects the crop log, which claimed "window advances next cycle" unconditionally and so misreported truncated crops. A test now pins the behaviour behind the message: truncate must leave no offset recorded. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Stop Vegas mode showing last night's games as if they were live A game that was live in the evening was still being drawn as live the next morning. Two faults combined to freeze plugin visuals indefinitely. PR #291 added a call to plugin_adapter.invalidate_plugin_scroll_cache() so a plugin's own cached scroll image would be rebuilt from fresh data. That method was never implemented. hot_swap_content() wraps the call in a broad except, so every hot swap has raised AttributeError and been swallowed silently ever since — which is why the visuals it was meant to keep fresh never were. Continuous scrolling then removed the only path that reached it at all: should_recompose() and hot_swap_content() are called from the non-continuous branch of run_frame(), and continuous_scroll defaults to True. So on a default install the pending-update flags were set by the update tick, never consumed, and grew without bound. Together these froze content completely, because refetching is not enough on its own: the sports plugins' get_vegas_content() regenerates only "if the cache is empty", so take_next_group() kept receiving the same picture however often it asked. Fixed by: - Implementing invalidate_plugin_scroll_cache(). It covers both layouts — a helper directly on the plugin (stocks, news, odds-ticker) and one owned by a scroll-display manager (the sports scoreboards, which is the shape that produced this bug) — and clears cached_image and cached_array together, since the array is the image's numpy mirror. - Adding StreamManager.invalidate_pending_updates() and calling it from the continuous branch. It only drops the caches; the plugin recomposes when it next comes round in the rotation. process_updates() is wrong here: it refetches synchronously and merges into the active buffer that continuous mode bypasses, and hot_swap_content() rebuilds and repositions the whole strip, which is the freeze-and-jump this mode exists to avoid. Tests assert the fix rather than the implementation: 14 of the 17 new tests fail without it. Includes the wiring itself, since the regression was a call that was simply absent, and a check that the scroll position is untouched so this cannot regress into the swap's visible jump. All Vegas suites pass (355 tests). test_display_controller_vegas_tick.py still cannot be collected off-device for want of rgbmatrix, identically with and without this change. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Fix two CodeRabbit-flagged test assertions in vegas density tests test_prepared_group_is_used_without_refetching had a tautological final assertion; now checks stream.calls directly. test_no_partial_letter_at_either_edge required both crop edges to be blank, but the left edge here is always the crop's start position with no lead-in gap in word_strip, so it legitimately carries ink — only the right edge is an actual cut and needs the check. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
e2acbfb566 |
fix(web): don't validate double-sided settings when the feature is disabled (#422)
* fix(web): don't validate double-sided settings when the feature is disabled
Saving anything on the Display tab failed with a 400 when double-sided
mode was off:
Double-sided copies (2) must divide chain length (3) evenly
The Display form posts every field in one request, including
double_sided_copies (default 2) and double_sided_axis, whether or not
the Enabled checkbox is ticked. The server block was gated only on "is
any double-sided field present in the payload" — it wrote
ds_config['enabled'] but never read it. A user with chain_length: 3 and
the untouched default copies: 2 was locked out of saving any display
setting at all: brightness, GPIO slowdown, Vegas, sync.
Gate the checks on the enabled flag:
- Divisibility against chain_length/parallel is hardware-relational and
only runs when the feature is on.
- Structural checks (copies parses as an int in 2..8, axis in the
whitelist) still 400 when enabled; when disabled they drop the value
and leave the stored one untouched rather than rejecting the save.
The runtime already gated correctly (_resolve_double_sided returns None
when disabled), so nothing there changes.
Also in the Display tab:
- Hide Copies / Split Axis until Enabled is ticked, mirroring the Vegas
Scroll pattern. Hidden rather than disabled, so the fields keep
submitting and the server still sees an 'off' state to persist.
- Fix the save toast: the form's handler read xhr.responseJSON, a jQuery
property that doesn't exist on a native XMLHttpRequest, so it was
always undefined and every save reported a green "Display settings
saved" — even the 400s. Parse responseText and use the real status.
Tests: two existing double-sided tests asserted 200 on payloads that the
divisibility check (added later, in #373) turns into 400s; the first now
supplies matching hardware values and the second passes as written now
that a disabled save skips the check. Added coverage for the reported
regression, for bad values while disabled, and for the check still
firing when enabled.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016aUvzTXpEYhNEWYQvFqQU9
* fix(web): treat only 2xx as a successful display save
Review feedback on #422.
- showDisplaySaveResult tested `xhr.status >= 400` for failure, so a
network error — which reports status 0 — was waved through as
"Display settings saved". That's the same class of false-success bug
this branch set out to fix. Test the 2xx range instead, and let a
response body refine a successful verdict without overturning a
failed one.
- Annotate the _copies_fits_hardware helper, matching the annotated
helpers already in api_v3.py.
- Cover the vertical divisibility branch: chain_length 2 would divide
evenly, so only parallel 3 can produce the rejection, which pins the
branch to the right hardware dimension.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016aUvzTXpEYhNEWYQvFqQU9
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
3872a68ff7 |
Surface plugin update availability on the plugins page (#421)
* Surface plugin update availability on the plugins page
The plugin manager page already showed each installed plugin's version and
had an Update button, but nothing told users an update actually existed —
they had to guess. The store manager already compares the installed
manifest version against the registry's latest_version for its reinstall
decision; this surfaces that same signal in the UI.
- api_v3 /plugins/installed now returns `latest_version` (from the registry
cache, no extra network call) and an `update_available` flag computed by a
new semver-aware helper `_is_plugin_update_available`. A locally modified
plugin whose version is ahead of the registry is not flagged.
- The installed-plugin card shows "vX.Y.Z available" next to the installed
version, and the Update button becomes emphasized ("Update to vX.Y.Z" with
a gentle pulse) when a newer version is published — mirroring the app's own
update banner styling.
- Added tests for the helper and the endpoint fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EW8yDbsk8EceDpq6Hjgqj
* Catch InvalidVersion specifically in plugin update comparison
Address review feedback: the version comparison caught a blind `except
Exception` (ruff BLE001). Split the two failure modes and catch each
specifically — ImportError for a missing `packaging` (a core dependency)
and InvalidVersion for an unparseable version string — while preserving
the existing "surface the mismatch" fallback behavior. Added a test for
the unparseable-version path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012EW8yDbsk8EceDpq6Hjgqj
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
989162d28f |
fix(web): custom-feed logo upload uses the wrong request/response contract (#420)
Every custom-feed logo upload has been failing: handleCustomFeedLogoUpload
posts the file under field name "file" and reads the response from
data.data.files, but the backend endpoint it calls
(api_v3.upload_plugin_asset, /api/v3/plugins/assets/upload) requires the
field name "files" (checks 'files' not in request.files, 400s "No files
provided" otherwise) and returns the result in a top-level "uploaded_files"
key - there is no nested "data" wrapper in the response at all. Confirmed
by reading the endpoint directly, and cross-checked against
file-upload-single.js, a sibling widget that uses the correct contract
against the same endpoint.
- formData.append('file', file) -> formData.append('files', file)
- data.data.files / data.data.files[0] -> data.uploaded_files /
data.uploaded_files[0]
No other call sites in this file used the stale contract (grepped for both
patterns after the fix - zero remaining). The response entries' 'path' and
'id' fields (both read further down in the same handler) are unaffected -
only the wrapper shape was wrong.
Found incidentally while re-verifying a CodeRabbit review on an unrelated
PR (#417) that had deleted a differently-named dead file
(custom-feeds-helpers.js) with the same bug; this widget (custom-feeds.js)
is the live code path and was never touched by that PR.
Validation: brace/paren balance check; explicit assertions that the old
field name and response shape no longer appear anywhere in the file. No
Python changed, no existing tests cover this endpoint's client flow.
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
|
cdf03fb107 |
Add skin system: user-installable visual overlays for sports scoreboards (#419)
* Add skin system: user-installable visual overlays for sports scoreboards Skins restyle a scoreboard's live/recent/upcoming rendering while the host plugin keeps doing data fetching, scheduling, caching, live priority, and vegas mode — the anti-fork alternative for users who only want a different layout. - src/skin_system/: ScoreboardSkin API, SkinContext (canvas + adaptive layout + logo/font helpers), discovery/loading runtime with API major version gating and per-skin module namespacing - src/base_classes/sports.py: _render_game() seam at the three _draw_scorebug_layout call sites; skin-first with built-in fallback, 3-strikes session disable, slow-render warning; per-mode skin config - scripts/validate_skin.py: headless multi-mode/multi-size validator with bundled per-sport fixtures (no hardware or network needed) - skins/example-classic-baseball/: working reference skin - Web UI: served-schema Visual Skin dropdown (validation never enum-restricted, so uninstalled skins can't invalidate configs) and GET /api/v3/skins - Store: registry entries with type "skin" install to skins/ - docs/SKIN_SYSTEM.md (architecture), docs/CREATING_SKINS.md (author guide incl. Claude Code prompt), view-model contract locked by tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1 * Address review feedback on skin system - skin_runtime: cache the entry module so the 2nd/3rd load of the same skin (live/recent/upcoming hosts) doesn't re-execute it with unbound sibling aliases; rebind cached sibling modules to their bare names around entry import and restore prior bindings after; include per- manifest mtimes in the discovery cache fingerprint so in-place skin updates are picked up - sports.py: count render_skin_card exceptions toward the 3-strike session disable - store_manager: validate skin ids (pattern + resolved-path containment in skins/), reject registry/manifest id mismatches, and stage+validate downloads in a temp sibling before replacing an existing skin - schema_manager: leave the schema untouched when the configured skin value is a per-mode mapping (a string dropdown could overwrite it) - validate_skin.py: reject non-positive sizes and non-object --options at parse time; support --output-dir outside the repo; type annotations - example skin: validate accent_color once at load with logged fallback - fixtures: pregame 0-0 scores in football/hockey upcoming fixtures - api /skins: rely on the self-invalidating discovery cache instead of force_refresh - docs: valid JSON manifest example, load_logo caching semantics spelled out, language ids on fenced blocks - tests: view-model contract test now exercises the real extractor; regression test for repeated same-skin loads with sibling modules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LrCusPasy1qeUN5anK3aA1 --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
6a9d8014e5 |
Tag plugin logs structurally and surface the active plugin in System Logs (#418)
- get_logger() now returns a PluginLoggerAdapter when given a plugin_id,
so every plugin log call is stamped with plugin_id automatically instead
of only calls that explicitly passed extra={'plugin_id': ...}. This makes
the "[Plugin: x]" prefix reliable in the journalctl-backed log stream.
- display_controller publishes the currently active mode/plugin to the
shared cache whenever it changes, exposed via a new
GET /api/v3/display/current-status endpoint.
- System Logs page: adds a "Now showing" banner backed by that endpoint, a
plugin filter dropdown (populated from parsed log lines), a plugin badge
per log entry, and fixes log parsing to handle the short-iso timestamp
format journalctl actually returns (the old regex only matched syslog
timestamps, so level/plugin extraction silently never ran).
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
c90129285c |
Web UI: mobile navigation, guided onboarding, basic/advanced config tiering + performance (#417)
* chore(web): remove dead legacy client-side plugin-config generator (~2,300 lines) Plugin config forms have been rendered server-side (plugin_config.html via GET /partials/plugin-config/<id>) since the HTMX migration; the old client-side generator survived as unreachable code. Verified dead by call graph, not by naming: showPluginConfigModal and showGithubTokenInstructions have zero callers anywhere in templates or JS, and everything removed here is reachable only from those two roots. Removed: - plugins_manager.js: showPluginConfigModal, generatePluginConfigForm, generateFormFromSchema, generateFieldHtml, generateSimpleConfigForm, handlePluginConfigSubmit, the modal's JSON-editor view (initJsonEditor, switchPluginConfigView, syncFormToJson/JsonToForm, saveConfigFromJsonEditor, resetPluginConfigToDefaults, displayValidationErrors, closePluginConfigModal, savePluginConfiguration, currentPluginConfigState), their exclusive helpers (getSchemaPropertyType, escapeCssSelector, dotToNested, collectBooleanFields, normalizeFormDataForConfig, flattenConfig, loadCustomHtmlWidget), the orphaned-modal cleanup block, the modal's listener wiring, and the never-invoked showGithubTokenInstructions/closeInstructionsModal pair. - plugins.html: the #plugin-config-modal markup those functions drove. - base.html: the deprecated pluginConfigData() component and the window.PluginConfigHelpers shim (only ever called by pluginConfigData). Deliberately kept, verified still live: - renderArrayObjectItem, getSchemaProperty, escapeHtml/escapeAttribute (window-exposed for the top-level array-of-objects handlers the server-rendered form uses), toggleNestedSection, addKeyValuePair/ addArrayObjectItem families, executePluginAction, and window.currentPluginConfig = null init (file-upload.js and executePluginAction read it, optional-chained). - app()'s internal generateConfigForm/generateSimpleConfigForm methods in base.html: unreachable now but embedded in the live Alpine component; excising methods from a live object is deferred to keep this change zero-risk. Validation: every deletion seam inspected line-by-line; Jinja parse of both templates passes; repo-wide sweep confirms zero remaining references to any deleted function or element id (deleted ranges contained no Jinja tags). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): mobile navigation drawer + responsive CSS gap fixes Phones previously got the desktop layout squeezed: ~12 system tabs plus one tab per installed plugin wrapped into many rows of small pill buttons, and the header's settings-search and system-stats widgets were dropped entirely (hidden below their breakpoints, never relocated). - Off-canvas nav drawer below md: the existing nav markup (system tab row + #plugin-tabs-row, including dynamically injected plugin tabs) is wrapped in a #site-nav container that CSS repositions into a slide-in drawer on small screens. Same DOM nodes, same @click handlers, nothing duplicated. Tabs become full-width rows with 44px+ touch targets. A hamburger button (md:hidden) in the header and a backdrop toggle the new mobileNavOpen Alpine state (added to both app() definitions, mirroring activeTab). Clicking any tab, a search result, or the backdrop closes the drawer. At md+ hard CSS guards make all drawer styles inert - desktop renders exactly as before. - Header widgets relocated, not hidden: placeHeaderWidgets() in app.js moves the #settings-search-wrap and #system-stats nodes (same elements, listeners intact - both are looked up by id from SSE/search code, so they must never be duplicated) into the drawer below md and back into the header above it, via a matchMedia listener. - Fixed 13 breakpoint utility classes that templates referenced but app.css never defined (sm:block, sm:grid-cols-2, sm:text-sm, md:block, md:w-auto, lg:block, lg:flex, lg:w-64, xl:grid-cols-2/3, 2xl:grid-cols-2/3/4). This was a live bug: 'hidden sm:block' on the search box and 'hidden lg:flex' on the stats meant BOTH were invisible at every screen width. Audit method (repeatable): diff classes used in templates vs defined in app.css. - Mobile modal sizing: one global rule caps .modal-content at 95vw/90vh with internal scroll below 640px - covers every modal without per-template changes. - Horizontal-scroll affordance: pure-CSS edge-fade shadows on .overflow-x-auto containers (scrolling-shadows technique), plus larger in-table touch targets below md. Validation: breakpoint used-vs-defined audit now returns zero gaps; Jinja parse of base.html passes; all changes to desktop behavior are additive (new utilities) or scoped inside max-width media queries. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): x-advanced schema flag groups plugin config fields under a collapsed Advanced Settings section Plugin config pages show every schema property at equal visual priority, which overwhelms first-time users. Plugin authors can now add "x-advanced": true to any flat (non-object) property in config_schema.json to move it into one collapsed "Advanced Settings (N)" section rendered after the basic fields - progressive disclosure with zero loss of control. Implementation: the main render loop in plugin_config.html splits ordered properties into basic/advanced tiers; the advanced group reuses the exact .nested-section/.nested-content/toggleSection() shell that nested object sections already use, so the settings search's expand-on-match behavior works on advanced fields with no JS changes. Object-type properties ignore the flag (they already render as their own collapsible sections). No backend change needed: jsonschema ignores unknown x-* keywords exactly as it does for x-widget/x-propertyOrder. Documented in docs/widget-guide.md alongside the other x-* extensions. Validation (rendered with real Jinja, not just parsed): - synthetic schema with 2 advanced fields: basic fields render before the section, advanced inside the collapsed shell, count badge correct, x-advanced on an object property correctly ignored - schema without any x-advanced: output is identical to the pre-change template (whitespace-normalized diff against git HEAD's version) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): Display Settings basic/advanced split + live total-resolution readout The Hardware Configuration card showed ~17 fields at equal priority; a new user only needs 7 of them to get a correctly-sized, correctly-colored image (rows, cols, chain_length, parallel, brightness, hardware_mapping, led_rgb_sequence). The other 10 (multiplexing, panel_type, row_address_type, gpio_slowdown, rp1_rio, scan_mode, pwm_bits, pwm_dither_bits, pwm_lsb_nanoseconds, limit_refresh_rate_hz) now live in a collapsed "Advanced Hardware Settings" section using the same nested-section shell as plugin config forms, so toggleSection() and settings-search auto-expand work unchanged. led_rgb_sequence moved up beside brightness/hardware_mapping (2-col grid became 3-col). No field was removed or renamed; the form still posts the same names to /api/v3/config/main. Also adds a live "Your display: W x H pixels" readout under the four sizing fields (width = cols x chain_length, height = rows x parallel - the exact math the chain-length tooltip describes in prose), recomputed client-side on every input event, no round-trip. Deviation from plan, deliberate: disable_hardware_pulsing / inverse_colors / show_refresh_rate stay in their separate "Display Options" card rather than moving across cards - relocating fields between form sections risks regressions for no decluttering gain in the card users complained about. Validation (real Jinja render): all 17 hardware fields present exactly once, basic fields render before the advanced section and the 10 advanced fields inside it, div count balanced (71/71), readout + recompute script present. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): plugin install auto-enables + persistent restart nudge Getting a plugin onto the display used to take three disconnected manual steps: install from the store, flip its enable toggle, then restart the display service - with no in-UI hint that steps 2 and 3 were needed (only docs/GETTING_STARTED.md mentions it). - installPlugin() now enables the plugin immediately on successful install (owner-confirmed behavior change: always auto-enable, no opt-out; users who don't want it running toggle it off as before), then shows a persistent toast ("... restart the display to show it") with an inline "Restart Now" button wired to the existing restartDisplay() - the same function the three existing Restart Display buttons call. - notification.js: show() accepts optional { actionLabel, onAction } to render one inline action button per toast. Callbacks are stored per notification id and cleaned up on dismiss; a new triggerAction() public method runs the callback and dismisses. The global showNotification() shorthand now forwards a full options object as its second argument (legacy type-string calls unchanged). Scope note: applies to the plugin store's install path (window.installPlugin). The custom-registry install path keeps its existing behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): dismissible Getting Started checklist on Overview New users land on a dense multi-tab dashboard with no suggested order of operations (the only guided flow is the WiFi captive portal). This adds a non-gating checklist card at the top of Overview with five steps, each a deep link that switches to the right tab (and closes the mobile nav drawer): 1. Set panel size -> Display tab (done: rows/cols/chain_length > 0) 2. Set timezone/location -> General tab (done: differs from template defaults America/New_York / Tampa) 3. Install a plugin -> Plugins tab (done: /api/v3/plugins/installed non-empty) 4. Enable a plugin -> Plugins tab (done: any installed plugin enabled) 5. Configure it -> Plugins tab (done: first enabled plugin has >=1 saved value differing from its schema defaults) Steps 1-2 are computed server-side in Jinja from main_config (already in the partial's context); 3-5 client-side from existing endpoints. No new backend state: dismissal persists in localStorage (mirroring the reconciliation banner's sessionStorage pattern one section up); deep links use the same _x_dataStack app-data access as settings-search.js. Disclosed heuristic limit: values left at legitimate defaults (a user actually in Tampa) read as "not done". Validation: real Jinja render across 3 config variants confirms the server-side done-flags flip correctly; div balance intact; /plugins/config response shape (config dict directly in .data) verified against api_v3.py. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(display): drag-and-drop plugin rotation order for the primary display mode The primary rotation's order was invisible and unconfigurable: modes are registered in parallel-load COMPLETION order, so rotation order actually varied between restarts. Only the niche Vegas Scroll mode had a working order UI. This adds real, persisted ordering end to end: Backend: - config.template.json: new display.plugin_rotation_order (default [], fully backward compatible). - display_controller.py: _apply_plugin_rotation_order() rebuilds available_modes grouped by plugin per the configured list (each plugin's modes keep their declared order; unlisted plugins follow in existing relative order; empty config = exact no-op). Applied at startup after parallel load and after live enable/disable reconcile (before the existing _resync_mode_index_after_change, which preserves the current mode). Mirrors vegas_mode get_ordered_plugins() semantics. - api_v3.py save_main_config: accepts plugin_rotation_order as a JSON array (same parse/guard pattern as vegas_plugin_order). Frontend: - New shared widget static/v3/js/widgets/plugin-order-list.js: the Vegas section's drag-and-drop list factored out verbatim (native HTML5 drag events, saved-order-first rendering, hidden-input JSON sync), parameterized by container/order-input/optional exclude-checkbox/badge. - display.html: Vegas section now calls the shared module; its ~130-line inline copy of the same logic is deleted. - durations.html: new "Rotation Order" card above the durations grid using the same module, posting plugin_rotation_order with the existing form. Deviation from plan, deliberate: durations stay as their own mode-keyed grid rather than inline in the drag rows - verified display_durations keys are MODE names (display_controller.py resolves duration per mode_key), not plugin ids, and one plugin can own several modes, so the planned 1:1 inline pairing was wrong. Validation: py_compile on both Python files; _apply_plugin_rotation_order unit-tested standalone (configured order applied, empty-config no-op, unknown ids skipped - 3/3); both templates render with balanced divs, the hidden input carries the saved order, and the old inline implementation is confirmed gone; config.template.json parses. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): serve the interface at / — /v3 kept as a legacy alias The user-visible URL no longer carries the interface version: the pages blueprint is now registered un-prefixed (primary) AND at /v3 (second registration, name='pages_v3_legacy'), so: - http://<device>/ serves the interface directly (the old @app.route('/') redirect is removed — the blueprint's own index takes its place) - every existing /v3/... bookmark and all the hardcoded /v3/partials/... fetches in templates/JS keep working verbatim through the alias mount — zero template/JS churn, zero broken links - url_for('pages_v3.*') resolves against the primary registration, so all server-side redirects (captive portal detection endpoints) now emit un-prefixed URLs - the AP-mode captive-portal allowlist learned the un-prefixed page paths (/setup, /partials/, /settings/, /plugin-ui/) so setup-mode requests don't redirect-loop - /api/v3 and the templates/v3, static/v3 directories are deliberately untouched (internal, invisible to users; owner-confirmed scope) Validation: dual registration mechanics tested against real Flask (test client): /, /v3, /v3/ redirect, partials and /setup reachable on both mounts, url_for yields un-prefixed paths; py_compile passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): stream the preview PNG raw instead of PIL decode + re-encode display_preview_generator() opened each changed snapshot with PIL and re-encoded it to PNG just to base64 it — but /tmp/led_matrix_preview.png already IS a PNG, written atomically by the display service (tmp file + os.replace in display_manager.py), so a partially-written file can never be observed. Read the bytes and base64 them directly: identical payload (front-end consumes data:image/png;base64 — verified in base.html), one full image decode+encode per frame less on the same Pi that's driving the matrix. The existing mtime skip and viewer-marker throttling are unchanged (they already covered the "skip unchanged frames" concern). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): gzip response compression via flask-compress The interface ships a ~5,000-line HTML shell and >20k lines of JS uncompressed; on phone/WiFi that dominates load time. Flask-Compress gzips/brotlis compressible responses transparently. - Optional dependency, same graceful pattern as flask-limiter: missing package = uncompressed responses, no crash. - SSE safety verified empirically against the real package (1.24): an actual streamed text/event-stream response comes back with no Content-Encoding while a large HTML response gzips — the display preview / stats / logs streams are unaffected. - Added flask-compress>=1.14 to web_interface/requirements.txt. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): gate verbose console logging behind the existing pluginDebug switch plugins_manager.js, base.html's inline scripts, and app.js emitted 198 console.log calls in production - including per-interaction [DEBUG] dumps - costing main-thread time and drowning real errors in noise. - New window.debugLog() gate defined in base.html's first inline script (before any other script runs): forwards to console.log only when localStorage.pluginDebug === 'true' - the SAME switch plugins_manager.js already used for its _PLUGIN_DEBUG_EARLY logs, so existing debug workflow docs stay valid. Exposed as window.LEDMATRIX_DEBUG for other scripts. - Mechanically rewrote console.log( -> debugLog( in plugins_manager.js (127), base.html (64), app.js (7). Verified no occurrences lived inside string literals before rewriting; console.error/console.warn untouched. - app.js's no-Alpine showNotification fallback restored to console.info - it's a user-facing last resort, not debug output. Both load paths are safe: the gate is the first inline <script> in <head>, and every rewritten file loads deferred after it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): vendor CDN assets locally — LAN-speed loads, fully offline-capable Font Awesome, CodeMirror, and htmx were fetched from cdnjs/unpkg on every fresh page load, adding third-party round-trips on a device that often lives on a local network (and htmx was local-only in AP mode, meaning two different loading behaviors to reason about). - Vendored pinned copies under static/v3/vendor/: Font Awesome 6.0.0 (css/all.min.css + the 8 webfonts it references relatively) and CodeMirror 5.65.2 (core, javascript mode, closebrackets/matchbrackets addons, base + monokai css) - ~1.1 MB total, exact versions the CDN tags pinned. - htmx + sse + json-enc extensions now load from the existing local copies (verified 1.9.10, matching the CDN pin) on EVERY network, not just AP mode; the pinned CDN copies remain as a one-shot rescue fallback, mirroring the pattern Alpine already used. The convoluted isAPMode source-flipping logic collapses away. - Dropped the CDN preconnect/dns-prefetch hints (no longer on the critical path). - Fixed a latent bug while relinking CodeMirror: the loader requested mode/json/json.min.js, which does not exist on cdnjs (HTTP 404 verified) - it 404'd on every JSON-editor open. JSON highlighting comes from the javascript mode; the phantom entry is removed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * perf(web): extract 3,850 lines of inline JS from base.html to cacheable static files base.html shipped ~4,200 lines of inline JavaScript inside the HTML document, re-downloaded and re-parsed on every page load (gzip helps the transfer, but inline scripts can never be browser-cached). The four largest blocks - none containing any Jinja syntax, verified by scanning every inline block for {{ }} / {% %} - now live as static files served with the app's existing mtime-versioned immutable caching: - js/htmx-config.js (246 lines): HTMX swap/script-execution config, toggleSection helpers - js/app-early.js (346 lines): early helpers + the app() stub that must precede Alpine init - js/app-shell.js (2,997 lines): SSE wiring + the full Alpine app() implementation and tab logic - js/custom-feeds-helpers.js (262 lines): custom-feeds table helpers Each replacement <script src> is CLASSIC (no defer/async) at the exact position of the inline block it replaces - identical execution timing and DOM visibility to inline scripts, so relative ordering with the deferred scripts and with each other is unchanged. base.html drops from ~4,940 to 1,079 lines. Validation: extraction proven lossless by programmatically reassembling the four files back into the template and comparing against git HEAD - byte-for-byte identical. Jinja parse passes; script open/close tags balanced (53/53, after excluding a literal "<script>" inside an HTML comment). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): size the live preview from the PNG's real dimensions, not config The initial SSE render sized the preview image (and both overlay canvases) from the server-reported config dimensions (cols x chain_length, rows x parallel), while the scale slider's re-render path sized from img.naturalWidth/naturalHeight. Whenever the snapshot PNG's actual size disagrees with the config (stale config, display service not restarted after a hardware change), the initial render stretched the image at a fractional ratio - blurry despite image-rendering: pixelated - and touching the scale slider "fixed" it. Reported live on the devpi test rig. Both paths now size from the loaded image's natural dimensions inside img.onload (which also removes a transient wrong-size flash between src assignment and load). The meta label now reports the true snapshot size. The preview card also gets overflow-x-auto so on narrow screens a wide preview scrolls at its exact pixel-perfect size instead of being squeezed into the viewport (fractional downscaling of pixel art also reads as blur). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): fold Display Options into the advanced dropdown; Vegas above Double-Sided Owner-requested layout refinement of the Display Settings tab: - The "Display Options" card (disable_hardware_pulsing, inverse_colors, show_refresh_rate, use_short_date_format, Dynamic Duration) moves inside the collapsed advanced section, now titled "Advanced Hardware & Display Options (15)". Hidden form fields still submit with the form, and settings search still auto-expands the section on match, so nothing is lost - the tab just leads with the essentials. - The "Vegas Scroll Mode" section moves above "Double-Sided Display". New section order: Hardware (+ advanced dropdown) > Vegas Scroll > Double-Sided > Multi-Display Sync. Validation (real Jinja render): all 23 field names present exactly once, divs balanced (70/70), the five Display Options fields render inside the advanced section's bounds, and section markers confirm the new order. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): Getting Started items are manually checkable; card auto-hides when complete Two gaps reported from live testing on the devpi rig: 1. The timezone/location step never showed done for a user whose real timezone IS the shipped default (America/New_York) - the heuristic can only detect difference-from-default, not "user saved this". Clicking an item's checkbox now toggles it done manually (persisted per browser in localStorage), so any heuristic false-negative is one tap to clear. Clicking the item text still deep-links to its tab. 2. The card now hides itself automatically once every step is done (auto-detected or manually checked) - previously it stayed until the X was clicked. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * chore(web): CI cleanup — declare debugLog global, fix entity-unescape order, drop v3 from UI branding - Add debugLog to the /* global */ headers of the six JS files that call it (defined in base.html's first inline script) — resolves the wall of "'debugLog' is not defined" ESLint errors failing the Codacy check. - Fix the two js/double-escaping CodeQL alerts in app-shell.js: the entity-unescape chains decoded & before </>, so a value containing a pre-escaped "&lt;" wrongly double-decoded to "<". & now decodes last (standard order). Pre-existing bug, made visible when the inline scripts moved into scannable .js files. - Page title / header drop the "- v3" suffix, matching the de-versioned user-facing URL. The remaining 7 CodeQL alerts are pre-existing patterns newly visible to scanning (CodeQL doesn't see inline template JS): 4 github.com/htmx.org URL-substring checks (the htmx ones match error-message text, not URLs — false positives in context) and 1 innerHTML XSS-through-DOM in the GitHub install flow. Triage/fix deferred to a focused follow-up rather than expanding this PR. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): add the missing nav tab for the Rotation & Durations page The durations partial (/partials/durations) has existed as a route with no nav tab and no content panel referencing it - an orphaned page. That made the new rotation-order UI unreachable through the interface (caught by the owner testing on the rig; my endpoint-level tests fetched the partial by URL and never noticed the missing entry point). - New "Rotation" tab (fa-rotate icon, verified present in the vendored FA 6.0.0 css) between Display and Backup & Restore, wired exactly like the other tabs (#durations-content + hx-get + loadtab; loadTabContent() is fully generic, so no JS changes needed). - Page heading updated from "Display Durations" to "Rotation & Durations" to match its content since the rotation-order card landed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): Rotation & Durations page lists every enabled plugin's screens The durations grid looped over display.display_durations, which nothing has ever populated (verified {} on a real production install) - so the page rendered no duration fields at all. Worse, its inputs posted bare mode names, which save_main_config's endswith('_duration') filter silently dropped: the page was broken in both directions, unnoticed because it was also unreachable (previous commit). - pages_v3._load_durations_partial now builds one entry per display mode of every ENABLED plugin via plugin_manager.get_plugin_display_modes() (falling back to the plugin id), overlaid with saved values, defaulting to the display controller's 30s. Grouped per plugin, sorted by name. Saved keys not owned by any enabled plugin stay visible under "Other saved entries" instead of vanishing. - durations.html renders the grouped inputs, named duration__<mode_key> (mode keys are arbitrary, so they can't use the *_duration suffix convention), with an explanatory empty state when no plugins are enabled. - api_v3.save_main_config accepts the new duration__<mode> fields and writes them into display.display_durations under the bare mode key - exactly what the display controller reads (display_durations.get(mode_key, 30)). Validation: py_compile both blueprints; Jinja render with 3 groups asserts grouped inputs, saved-value overlay, stale-entry group, empty state, and div balance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): restart-pending banner, unsaved-changes guard, installable web app Three usability improvements from live testing feedback: - Restart-pending banner: every successful POST to /api/v3/config/main (display hardware, rotation/durations, general settings) now raises a persistent banner - "Configuration saved, restart the display to apply" - with a Restart Now button that posts restart_display_service directly. Backed by sessionStorage so it survives tab switches and reloads until restarted or dismissed. Plugin config saves are deliberately excluded: they apply live via the display process's config watcher. - Unsaved-changes guard: plugin config panels are Alpine x-if templates, so navigating away destroys the panel and revisiting re-fetches it - edits were silently discarded. Forms now mark themselves dirty on input (cleared on successful submit), a capture-phase click handler confirms before a lossy tab switch, and beforeunload guards full page unloads. System tabs (x-show, persistent DOM) are exempt - no false prompts. - Installable web app: manifest.json (standalone display, dark theme) + generated LED-grid icons (192/512 maskable + 180 apple-touch), linked from base.html. "Add to Home Screen" now yields an app-like fullscreen experience; no service worker, so zero behavioral risk. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * test(web): smoke tests + static-analysis audits for the web UI Guardrails so this branch's fix classes can't regress silently: - test_web_smoke.py (24 tests): boots the pages blueprint with the same dual registration app.py uses and asserts every page/partial returns 200 with its load-bearing markers (nav wiring, getting-started card, advanced section, rotation order card, per-mode duration inputs), the /v3 legacy alias serves everything, all critical static assets (incl. vendored fontawesome/codemirror, PWA manifest/icons) are served, durations group per plugin with the leftover bucket, and the advanced-hardware section really contains the tuning fields. Would have caught this session's unreachable-durations-page and orphaned-tab bugs instantly. - test_web_static_audit.py (3 tests): (1) every responsive utility class referenced in templates is actually defined in app.css - the silently-no-op class bug that left the header search box invisible at every width; (2) every url_for('static', ...) reference points to a real file; (3) any JS file calling the debugLog global declares it in a /* global */ header. All 40 web tests pass (24 + 3 new, 13 existing) under pytest + Flask. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): floating live preview + per-plugin "Preview on display", drawer a11y Preview-while-configuring: - Floating mini preview (fixed, bottom-right) available on every tab except Overview, fed from the same SSE display stream by updateDisplayPreview - no new connections. Collapses to a round toggle button; open/closed state persists in localStorage; hides on Overview where the full preview lives. - "Preview on display" button on every plugin config page header: runs that plugin on the real display for 60 seconds via the existing /display/on-demand/start API and opens the floating preview, closing the configure -> see-the-result loop. Drawer/nav accessibility: - aria-current="page" tracks the active tab (system + dynamic plugin tabs, matched via their Alpine @click expression), updated from the activeTab watcher so search deep-links and checklist navigation are covered too. - Escape closes the mobile drawer and returns focus to the hamburger; opening the drawer moves focus to its first tab. Validation: all 40 web tests pass; Jinja parse + div balance on both touched templates. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): resolve Codacy findings — DOM building over innerHTML, Map callbacks, misc lint Verified each of the 27 reported findings against current code; all fixed except one rule class skipped with reason below. - app-early.js: plugin tab buttons are now built with createElement/ createTextNode instead of innerHTML template strings (icon class and name come from plugin manifests - semi-trusted input; the old code escaped the name but interpolated the icon class). Both construction sites. Also the forEach arrow no longer returns tab.remove()'s value. - plugin-order-list.js: rows, empty state, and error state all built with DOM APIs - the file no longer contains innerHTML at all (the now-unneeded escapeHtml/escapeAttr helpers are removed); MODE_LABELS is a Map so the vegas-mode lookup can't hit prototype properties. - notification.js: actionCallbacks is a Map (get/set/delete) instead of a plain object - resolves the object-injection-sink and dynamic-delete findings; triggerAction also type-checks the callback. - htmx-config.js: unused catch binding dropped; var -> const in the afterSettle handler; the swapped-<script> re-execution reads/writes textContent instead of innerHTML; the diagnostic form payload uses a null-prototype object so a field named __proto__ can't pollute. - custom-feeds-helpers.js DELETED (with its script tag): all three of its functions (addCustomFeedRow, removeCustomFeedRow, handleCustomFeedLogoUpload) are shadowed by the deferred widgets/custom-feeds.js window assignments, which always win at call time - the copies were dead even when they lived inline in base.html. This also resolves the unused-function and unused-variable findings there. Skipped: 4x "Non-serializable expression must be wrapped with $(...)" in app-early.js - that rule targets code crossing a browser-automation serialization boundary (e.g. page.evaluate); these are ordinary arrow functions in plain browser code with no such boundary. Validation: all 40 web tests pass (incl. the static-asset reference audit, which confirms no template still points at the deleted file); Jinja parse OK. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web): update flow installs changed Python dependencies automatically The in-app updater (Update Now banner -> git_pull action) stashed, pulled, and purged plugins - but never touched Python dependencies. Any release adding a package (e.g. this branch's flask-compress, which lives in web_interface/requirements.txt) silently required an SSH session and a manual pip install that most users will never do. - git_pull now records HEAD before pulling; after a successful pull it diffs old..new and, if requirements.txt or web_interface/requirements.txt changed, installs exactly those via _pip_install_requirements - the same vetted root-visible sudo path the Tools-tab buttons use (with its existing graceful fallback when the sudo wrapper isn't configured). Results are appended to the update toast; a failure points the user at the Tools-tab button instead of failing the whole update. - install_base_requirements (Tools tab) now also installs web_interface/requirements.txt - previously it only covered the root file, so web-only dependencies were unreachable from the UI entirely. No install happens when the pull was already-up-to-date or when no requirements file changed, so routine updates stay fast. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): address CodeRabbit review — validation, a11y, perf, and privacy fixes Verified each finding against current code. Fixed: - api_v3: plugin_rotation_order is now strictly validated (JSON list of strings, 400 with a descriptive message otherwise) and popped from the payload before any further handling. - display_controller: _apply_plugin_rotation_order defensively ignores a non-list value (keeps the existing rotation, logs a warning) and drops non-string entries; new logs carry the [DisplayController] prefix. Unit-tested both defensive paths. - app.py: snapshot-read handler narrowed to OSError with debug logging; flask-compress ImportError now emits one structured warning with the install remedy. - htmx-config: the response-error logger prints form FIELD NAMES only - values (API keys, passwords) never reach the console. - plugin-order-list: saved order/exclusions normalized with Array.isArray (a saved "null" previously crashed .forEach); each row gained keyboard/touch-accessible move-up/move-down buttons (HTML5 drag events don't fire on most mobile browsers) that reorder and syncInputs() immediately alongside native drag. - app-shell: window.installedPlugins setter always takes the new list (same-ID metadata/enabled updates were silently dropped); tab rebuild stays gated on ID changes. LED dot renderer reads the frame with ONE getImageData call instead of one per pixel (~9,200/frame at 192x48). - plugins_manager: togglePlugin returns its request promise resolving the API outcome; the install flow now shows the "installed and enabled" toast (with Restart Now) only after enablement succeeds, and a warning without a restart offer when it fails. - a11y: hamburger aria-label flips Open/Close with drawer state; both Advanced-section toggle buttons declare aria-controls/aria-expanded and the shared toggleSection() keeps aria-expanded in sync; move buttons have per-plugin aria-labels. - Rotation/Vegas order-list bootstraps cap their retries (~5s) and show a reload hint instead of spinning forever; Alpine app-state lookups prefer [x-data="app()"] with a generic fallback. Skipped, with reasons: - executePluginAction arg order: caller (plugin_config.html) already passes (actionId, index, pluginId) matching the signature exactly. - generateFieldHtml XSS, entity-unescape blocks, dotToNested pollution, and "app.loadInstalledPlugins" in app-shell: all inside the legacy client-side config cluster whose entry points are shadowed by plugins_manager.js / replaced by server-rendered forms (zero live callers, verified) - queued for wholesale deletion in the follow-up rather than patching dead code. - custom-feeds-helpers.js findings (3): file was deleted in a prior commit. - console.error/warn override removal and afterSwap script re-execution removal: deliberate pre-existing workarounds every partial's inline init currently depends on; reworking them safely needs isolated testing (follow-up), and the error suppression is already double-gated (insertBefore AND htmx match). - "move durations bootstrap into a bundle": inline partial-scoped init is the established pattern for HTMX partials in this codebase. Validation: all 40 web tests pass; py_compile on all touched Python; all touched templates parse; rotation-order defensive paths unit-tested. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * chore(web): fix remaining real Codacy findings (2 of 6) - htmx-config.js: two more unused catch bindings dropped (optional catch binding), matching the earlier fix. - app-early.js: second forEach arrow (the stub updatePluginTabs copy) braced so the callback no longer returns tab.remove()'s value. The other 4 findings ("Non-serializable expression must be wrapped with $(...)") are deliberately NOT "fixed": that rule belongs to a browser-automation (WebdriverIO-style) lint context and is misfiring on ordinary arrow-function constants. Converting them to function declarations would look compliant but BREAK the code - all four arrows intentionally capture the enclosing Alpine component's `this` for the stub-to-full enhancement logic. The right remedy is disabling that pattern for this repo in Codacy's Code Patterns settings (or dismissing the four findings), not a code change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): floating preview shows a frame immediately on open + is resizable The floating preview opened empty and stayed empty until the display next CHANGED - the SSE stream only pushes frames on change, and the panel only consumed frames while already open, so the connection's initial frame (sent while the panel was closed) was dropped. Reported from mobile testing as "the button doesn't work". - updateDisplayPreview now caches the latest frame globally regardless of panel state; opening the panel populates the image from that cache instantly, then live frames take over. - Resizable: a size button cycles 192/256/384/512px presets (persisted per browser; works on touch), and desktop additionally gets a native drag handle (CSS resize: both). The image is fluid within the panel; on phones the panel is capped to the viewport width. The size icon (fa-up-right-and-down-left-from-center) is verified present in the vendored FA 6.0.0. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): stop duration fields leaking into config root; isolate per-file dependency installs; fix test fixture leak Verified each finding against current code. - api_v3 save_main_config: both duration blocks (*_duration suffix fields and the newer duration__<mode> fields) only READ from `data`, never removed the keys. The generic "remaining keys" merge later in the same function has no skip-list entry for either pattern, so every duration field was ALSO written a second time as a bogus top-level config key (e.g. "clock_duration": 30 and "duration__mlb_live": 42 sitting at config root, alongside the correct nested display.display_durations.<key>). Confirmed by tracing the full function. Fixed by popping each handled key from `data` (same pattern already used for plugin_rotation_order) and validating strictly: a non-integer duration now returns 400 with a message naming the offending field/mode instead of silently logging and moving on (for the *_duration fields, which previously had zero validation at all). - api_v3 dependency-install loops (git_pull's post-update sync and install_base_requirements): _pip_install_requirements can raise subprocess.TimeoutExpired or OSError (confirmed: install_requirements_file in permission_utils.py never catches either internally, despite its docstring's "never raises on non-zero exit" only covering return codes). Both loops previously let one file's exception either abort the whole try block (skipping the second requirements file entirely) or propagate uncaught. Each file's install is now in its own try/except, so a timeout or OSError on one file is recorded as a labeled failure and the loop continues to the next file. - test_web_smoke.py: the `client` fixture mutated the module-level pages_v3 Blueprint singleton's config_manager/plugin_manager directly with no teardown - since pages_v3 is shared across the whole pytest process (test_web_settings_ui.py touches the same attributes), this fixture's mocks could leak into whichever test ran next. Now saves the originals, yields the client, and restores them in a finally block. Validation: py_compile passes; all 40 web tests pass with the now-generator fixture. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): repair garbled Advanced Hardware section description An earlier sed-based text update concatenated the old and new copies of this description instead of replacing one with the other, leaving a duplicated sentence with the — entity broken into ".mdash;" (visible as literal "mdash;" text on the page). Restored to one clean sentence. Other findings from this review were already fixed in a prior commit (installedPlugins setter) or are confirmed dead code with zero live callers (executePluginAction/dotToNested/entity-unescape/generateFieldHtml, all reachable only from the two unused savePluginConfig copies in app-shell.js - grepped every template, no references) - same legacy cluster flagged in earlier review passes, still queued for a dedicated deletion follow-up rather than patched in place here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): remove redundant htmx:afterSwap script re-execution (was double-executing every partial's inline script) Re-verified this CodeRabbit finding, previously deferred as "needs isolated testing" - traced it to a confirmed, active bug rather than a style concern: htmx 1.9.10's own config defaults to allowScriptTags: true (confirmed in the vendored htmx.min.js, which itself contains the same clone-and-reinsert <script> mechanism internally). This means htmx ALREADY re-executes every <script> tag in swapped content by default, exactly like a browser navigating to a new page. The custom htmx:afterSwap listener in htmx-config.js did the identical clone-and-reinsert a SECOND time on top of htmx's own handling - so every inline <script> block in every HTMX-loaded partial (overview, display, durations, plugin config, etc. - most partials have one) executed twice per load. Confirmed safe to delete outright, not just narrow: grepped every hand-written JS file for a manual `dispatchEvent(... 'htmx:afterSwap' ...)` that might have relied on this handler for a non-htmx code path (e.g. the direct-fetch fallbacks like loadOverviewDirect) - none exists, so nothing depended on this listener specifically; htmx's native handling covers every real htmx-driven swap on its own. Left in place, unchanged: the console.error/console.warn global override a few lines up in the same file, which suppresses known-noisy HTMX-timing-race messages. That one is a legitimate anti-pattern too (broad substring matching can mask unrelated errors) but redesigning it needs care to preserve real diagnostics while still hiding the specific harmless races it targets - a scoped follow-up, not a same-day deletion like this confirmed-duplicate handler. Validation: all 27 fast web tests pass; JS brace/paren balance sanity checked (no local Node/browser available in this sandbox to execute the file directly - verify manually in-browser before merge). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * chore(display): add missing [DisplayController] prefix to the reconcile-complete log Re-verifying the full CodeRabbit findings list against current code surfaced one still-open item: the nitpick asked for the prefix on BOTH rotation-related log lines, but only "Applied plugin rotation order" got it in the earlier pass - "Plugin reconcile complete" was missed. No message/argument/level change, matching the finding's own scope. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web): repair dead /v3/logs link on the display hardware-error banner The "Logs tab" link in the display-settings simulation-mode banner was a real <a href> to /v3/logs, but no such route has ever existed (log content is loaded client-side via activeTab, not a dedicated page route) - the link 404'd regardless of the /v3 prefix change. Switch it to the same activeTab-switching pattern the real nav uses. * fix(web): raise display Rows field max from 64 to 128 Cols already allowed up to 128; Rows was capped at 64, which rejects valid larger panel configurations (e.g. 128-row tile chains). No server-side schema enforces a rows max, so this was purely an overly-strict HTML input attribute. * fix(web): remove redundant htmx.org substring check flagged by CodeQL CodeQL flags .includes('htmx.org') as "incomplete URL substring sanitization" - a false positive here, since this string is only ever matched against console.error/warn message text to decide whether to suppress a known-harmless HTMX timing-race log line, not used for any URL-trust/redirect decision. The check was also redundant: 'htmx' is already a substring of 'htmx.org', so the plain .includes('htmx') check right next to it already covers every case the removed check did. * fix(web): restore htmx script re-execution timing that Alpine x-data depends on Removing the custom htmx:afterSwap script-reexecution handler (in a prior commit, as a "duplicate execution" cleanup) broke every partial whose Alpine x-data component function is defined by an inline <script> in that same partial (e.g. wifi.html's wifiSetup()) - confirmed live via browser console: "Alpine Expression Error: wifiSetup is not defined" on every field in the WiFi tab. Root cause: htmx's own native script execution runs during its "settle" phase (~20ms after swap, per htmx's own defaultSettleDelay), but Alpine's MutationObserver evaluates x-data on newly-inserted elements synchronously, right as the swap lands - before settle. So the inline <script> defining wifiSetup() was still un-run when Alpine tried to call it, and Alpine does not retry a failed x-data evaluation later once the function does become defined. Fix: re-execute swapped <script> tags ourselves on htmx:afterSwap (which fires synchronously, before settle, beating Alpine's observer), and disable htmx's own native script re-execution (htmx.config.allowScriptTags = false) so the same script doesn't also run a second time during settle - restoring correct timing without reintroducing the original double-execution bug. Also in this commit: - fix XSS: unescaped repoUrl in a title attribute in renderSavedRepositories - replace .includes('github.com') substring checks with real URL hostname validation (CodeQL: incomplete URL substring sanitization) * fix(web): wait for async plugin install to finish before auto-enabling it Confirmed live: installing hockey-scoreboard logged "installation queued" (success) immediately followed by "enabling it failed" with a 404 "Plugin not found" from /api/v3/plugins/toggle. /api/v3/plugins/install runs the actual clone + plugin-manager discovery asynchronously via an operation queue when one is configured - the response installPlugin() was checking only means the operation was queued, not that the plugin is installed yet. Calling togglePlugin() right after that response 404s because plugin_manager hasn't discovered the new plugin. Fix: reuse the same operation-polling mechanism uninstallPlugin() already has (generalized pollOperationStatus() to take onComplete/onFailed/onTimeout callbacks instead of hardcoding uninstall behavior) so installPlugin() waits for the operation to actually complete before enabling it. Falls back to enabling immediately when no operation_id is returned (direct/synchronous install path, no queue configured). * fix(web): resolve remaining valid findings from latest review pass - custom-feeds.js: fix asset-upload contract mismatch (field name "file" -> "files", response read from top-level "uploaded_files" not "data.files") - same bug already fixed in this file on a separate branch/PR (#420), which this branch never received since they're independent PRs off main - custom-feeds.js: add aria-label to the two icon-only "remove feed" buttons - custom-feeds.js: move file-input reset into .finally() so a failed upload doesn't leave the input stuck holding the file, blocking retry of the same file - app-shell.js: fix executePluginAction(pluginId, actionId) parameter order/count mismatch vs. its callers' (actionId, index, pluginId) - currently masked by plugins_manager.js's correct version overwriting this one at load time (classic vs. deferred script order), but worth fixing outright since it's an isolated, self-contained reassignment (not inside the Alpine app() object literal) and removes a latent footgun - overview.html: align Alpine-state resolution with settings-search.js's two-tier getAppData() (also check appEl.__x.$data, not just _x_dataStack) Verified already addressed by earlier passes (no change needed): plugin_rotation_order validation, DisplayController log prefixes, togglePlugin returning its promise for install-flow chaining, installedPlugins setter always updating state, mobile-nav aria-label, toggleSection aria-expanded sync, PluginOrderList bounded init retries (both display.html and durations.html), plugin-order-list.js Array.isArray validation, batched getImageData in the LED-dot preview renderer, app.py exception narrowing/ logging, form-submission log redaction. Confirmed dead code, skipped (unreachable - zero template/JS callers, verified via full-repo grep): dotToNested prototype-pollution hardening, generateFieldHtml HTML-injection hardening, and the HTML-entity-unescape block in JSON parsing - all three live only inside app-shell.js's two legacy savePluginConfig implementations (one Alpine-method, one standalone), neither of which any template or script calls. The real, live plugin-config path is server-rendered via GET /partials/plugin-config/<id>. Explicitly NOT reverted: the htmx:afterSwap script-execution listener. An earlier finding batch asked to remove it as "duplicate" htmx behavior; that was tried and reverted this session after live testing on hardware proved it broke every partial whose Alpine x-data depends on an inline <script> in the same partial (confirmed: WiFi tab hard-failed with "wifiSetup is not defined"). Removing it again would reintroduce that regression. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
66f9950a30 |
chore(ci): add security-audit tooling scripts (workflow files pushed separately) (#414)
* chore(ci): add security-audit workflow and plugin security-proof scripts - scripts/prove_security.py, audit_plugins.py, generate_report.py -- automated checks (dangerous eval()/exec() calls, dependency scanning, report generation) for plugins. - .github/workflows/security-audit.yml + bandit.yaml -- CI wiring for the above plus gitleaks secret scanning and bandit static analysis. - .github/workflows/tests.yml -- pytest matrix across Python 3.10-3.12. Also fixes two Codacy findings while these files are freshly landing: - prove_security.py: dropped a pointless f-string prefix with no placeholders. - security-audit.yml: pinned gitleaks/gitleaks-action to a full commit SHA (matching this repo's existing pinning convention in test.yml) instead of the floating v2 tag. Split out of the original chore/dead-code-removal commit, which had accidentally bundled this in alongside unrelated dead-code deletions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * chore: drop workflow files -- pushed separately (needs workflow OAuth scope) * fix(security-tooling): address PR review findings across bandit.yaml, audit_plugins.py, generate_report.py, prove_security.py bandit.yaml: - Removed scripts/prove_security.py's file-level exclusion. Ran bandit directly to get ground truth: the real false positive is B105 (dict key "PASS" misread as password-like), not the eval/exec pattern the old comment claimed. Added a targeted # nosec B105 there, and found+fixed the identical pattern already present in generate_report.py. - Left the repo-wide B607 skip as-is: confirmed via AST scan that properly narrowing it touches 100+ bare-name subprocess call sites across wifi_manager.py, store_manager.py, permission_utils.py, app.py, and start.py -- none of which are part of this PR. Out of proportion to fix here; flagged as a dedicated follow-up. scripts/audit_plugins.py: - SyntaxError/OSError while scanning a plugin file now report CRITICAL (blocking) instead of WARNING/INFO -- a file that couldn't be parsed or read was never actually checked for danger, so it must not silently pass the audit. - --plugin <name> now tracks whether the requested plugin was found across all PLUGIN_BASE_DIRS and exits 1 with a clear error if not, instead of silently scanning zero plugins and reporting success. - The AST visitor now tracks import aliases (import subprocess as sp; from builtins import eval as e) and resolves them before checking against dangerous APIs, closing a straightforward evasion of every PLUGIN-001 through PLUGIN-005 check. Verified against both aliased and unaliased evasion patterns. scripts/generate_report.py: - _md_table_row now escapes pipe characters and normalizes newlines in every cell, so scanner-controlled content (a matched secret, a bandit issue_text) can't corrupt the Markdown table structure. - _load now distinguishes "artifact missing/malformed" from "valid empty result": each summarizer returns an availability flag, and main() now reports INCOMPLETE (not PASSED) with exit code 1 when any artifact is unavailable, instead of silently folding it in as 0 findings. - Gitleaks suppression now uses exact-match placeholder values (pulled from the actual config_secrets.template.json) plus a template-path allowlist, replacing broad substring checks that could hide a real secret containing something like "example.com" as part of its value. scripts/prove_security.py: - T1b (dangerous plugin calls): a file that fails to parse/read now reports CRITICAL with the exception details instead of being silently swallowed by `except (SyntaxError, OSError): pass`. - T6 (Docker hardening): base images must now be pinned to an @sha256 digest; a specific tag like python:3.12 is mutable and is now correctly flagged as unpinned, not just missing tags or :latest. - T2a (API surface): no config mechanism for enforcing local-only access exists in this codebase today (app.py hardcodes host='0.0.0.0'), so the "environment-aware" check as described isn't buildable without adding new config infrastructure -- out of scope here. Applied the achievable part: upgraded from INFO to WARNING, since enforcement can never currently be confirmed. - T1a (zip-slip): replaced the whole-file substring check with an AST walk that finds every extract()/extractall() call and confirms an is_relative_to() guard + "Zip-slip detected" log precede it in the same function. Verified it still passes on the real store_manager.py (both the per-member and validate-then-bulk-extract call sites) and correctly flags a synthetic unguarded extractall(). - T3a (hardcoded secrets): violation details no longer include the matched credential text -- only file, line, pattern type, and a redacted SHA-256 fingerprint, so a real finding doesn't get published into CI logs/artifacts/PR comments with wider exposure than the original leak. Verified with a synthetic secret that no raw content reaches the output. Validated: all four files compile; bandit scans all three scripts clean (2 legitimate targeted suppressions, 0 unaddressed findings); each new/ changed code path exercised directly (alias evasion, unmatched --plugin, missing/malformed/valid-empty artifacts, digest-pinning, zip-slip guard/no-guard, secret redaction); full audit_plugins.py -> prove_security.py -> generate_report.py pipeline run end-to-end producing a correct report. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(security-tooling): address follow-up review findings on PR #414 scripts/generate_report.py: - Added an explicit `object` type annotation to _md_sanitize_cell's value parameter -- it deliberately accepts any stringifiable value (calls str(value) unconditionally), so `object` reflects its actual contract more accurately than leaving it untyped. scripts/prove_security.py: - Dockerfile FROM-line parsing: renamed the comprehension variable `l` to `line` (ambiguous single-letter name). More importantly, fixed a real false-positive: `FROM --platform=<platform> <image>` was reading the --platform= flag itself as the image token, so a properly digest-pinned image behind a platform flag was incorrectly reported as unpinned. Verified against platform+digest, platform+tag-only, and digest+AS-alias Dockerfiles. scripts/audit_plugins.py: - Consolidated visit_Call's dangerous-API detection: previously, alias resolution only covered ast.Name calls for eval/exec/compile and ast.Attribute calls for subprocess/os.system, missing from-imported subprocess/os functions called as bare names (from subprocess import run as prun; prun(cmd, shell=True) or from os import system as s; s(cmd)). Added _resolve_call_target() to resolve both call shapes to a single fully-qualified target, then run all five PLUGIN-00x checks against that one resolved value. Verified against 10 evasion combinations (from-import aliases, direct/attribute calls, aliased module imports) and confirmed zero false positives on benign os/ subprocess usage without shell=True. Validated: all three files compile, bandit scans clean (same 2 legitimate suppressions as before, 0 new findings), audit_plugins.py/prove_security.py re-run against the real repo with no regressions from the prior fix pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
4abcd0e4f9 |
Fix PermissionError reading config_secrets.json in web interface (#416)
ledmatrix.service (main display) runs as root while ledmatrix-web.service runs as the non-root install user (install_web_service.sh). Both config_manager.py and config_manager_atomic.py only chmod'd config_secrets.json to 0o640 without ever fixing its group, so a file written by the root service ended up group-owned by root and unreadable by the web user, crashing the settings page with a raw PermissionError. Add ensure_shared_group_ownership() to chgrp secrets/config files (best effort, root-only) to the project directory's owning group whenever they are created or saved, and self-heal existing files on load. Also make get_raw_file_content() tolerate an unreadable secrets file the same way load_config() already does, degrading to empty secrets instead of a 500. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
2a1c47fa76 |
chore: remove dead modules and unused dependencies (~1,180 LOC) (#412)
* chore: remove dead modules and unused dependencies (~1,180 LOC) Deletions, each re-verified with a fresh repo-wide grep (core, web, scripts, docs, plugin monorepo) immediately before removal: Modules with zero live importers: - src/background_cache_mixin.py + src/generic_cache_mixin.py (134+150 LOC — referenced only by each other) - src/font_test_manager.py (134 LOC) - src/image_utils.py (22 LOC, self-documented deprecated) - src/layout_manager.py (408 LOC — only its own test imported it) + test/test_layout_manager.py - src/common/basketball_plugin_example.py (328 LOC sample) requirements.txt entries with zero importers in core (pre-plugin-era manager deps): icalevents, geopy, timezonefinder, unidecode. Plus the google-auth trio (google-auth-oauthlib, google-auth-httplib2, google-api-python-client): their only importer is the calendar PLUGIN, which declares all three in its own requirements.txt (verified in the monorepo and on an installed copy) — the plugin dependency installer owns them. Existing venvs are unaffected (removal doesn't uninstall); fresh installs get them when calendar is installed. Two stale references cleaned (a comment in test_pillow_compat.py, a directory listing in HOW_TO_RUN_TESTS.md). Full suite green except the two documented pre-existing failures (circuit_breaker mock drift, fixed in #400; clock-simple 64x32 overflow, pre-dates this series); all core entry modules verified importing cleanly under the emulator. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix: remove content accidentally bundled into the dead-code-removal commit The previous commit's git add/commit swept in a lot of unrelated, unreviewed work alongside the intended dead-code deletions: a new Plugin Composer web UI, new security-audit/CI tooling, a new march-madness plugin, and 23 local-development-only symlinks under plugin-repos/ (per scripts/setup_plugin_repos.py's own docstring, these are meant to be generated locally, never committed -- .gitignore has no entry for them, which is how they slipped in). Removed here, split into their own PRs instead (except plugin-repos/* symlinks and march-madness/ncaa_logos, which are dropped rather than carried forward -- see PR discussion): - web_interface/blueprints/composer.py + composer-app.js + composer-canvas.js + composer.html + manager.py.j2 - scripts/prove_security.py, audit_plugins.py, generate_report.py - .github/workflows/security-audit.yml, .github/workflows/tests.yml, bandit.yaml - All plugin-repos/* symlinks (local dev artifacts, not meant to be committed at all) - plugin-repos/march-madness/* and the 4 new assets/sports/ncaa_logos/* PNGs it needed (left out of every split PR pending a decision on whether march-madness belongs in the core repo or the plugin monorepo) This PR now contains only what its title describes: the dead-code removal from the previous commit, untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9db1d2391a |
chore(assets): add 4 NCAA team logos needed by march-madness (COLGATE, LEHIGH, MICHIGAN, RUTGERS) (#415)
march-madness (already live in ledmatrix-plugins) loads team logos from this shared assets/sports/ncaa_logos/<ABBR>.png cache at runtime (manager.py:233) -- these 4 were missing. Split out of PR #412 (chore/dead-code-removal), which had accidentally bundled these in alongside unrelated dead-code deletions. Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
14a59c863c |
perf(wifi): one status fetch per monitor tick instead of three (#411)
* perf(wifi): one status fetch per monitor tick instead of three The wifi monitor daemon fetched WiFi status + ethernet state before its AP-mode check, the check internally fetched the same state again (with retry), and the daemon fetched a third time afterwards — each fetch is several nmcli subprocess forks, every 30s, forever, even on a perfectly healthy link. check_and_manage_ap_mode's decision logic is extracted to _manage_ap_mode(status, ethernet, ap_active); the new check_and_manage_ap_mode_with_state() runs the single (retrying) fetch battery and returns (changed, status, ethernet, ap_active_after) — the post-state is derivable because state only ever flips via one enable or one disable. The original bool-returning method delegates, so existing callers are untouched. The daemon's dead pre-fetch is removed and its post-check reads use the returned state; the AP-enable retry semantics (_get_wifi_status_with_retry) are preserved exactly. ~10-15 forks/30s drops to ~4-6. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(wifi): add missing type hints on AP-mode state helpers, fix unused-var lint in tests - check_and_manage_ap_mode_with_state now declares its Tuple[bool, WiFiStatus, bool, bool] return type instead of being untyped. - _manage_ap_mode's status/ethernet_connected/ap_active parameters are now typed (WiFiStatus, bool, bool), matching its existing -> bool return hint. - test_wifi_check_state.py: rename unused unpacked variables (status/ ethernet/ap_after) to underscore-prefixed names at the three call sites that don't use them, leaving the used ones untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
bff13129c4 |
perf(config): mtime-signature fast path for load_config (#410)
load_config re-read and re-parsed config.json, the secrets file, AND the template (running the recursive migration diff) on every call — with ~30 call sites in web request handlers, some hit 2-3x per request. Fast path: stat all three files (mtime_ns + size); when unchanged since the last successful load, return the already-parsed self.config (same aliasing semantics as before). The signature is taken AFTER load + migration so a migration write-back doesn't retrigger, and both save paths refresh it. Cross-process freshness is preserved by construction: a save from the other process bumps the file mtime, so the next load here re-reads — verified by a dedicated test. Same-second edits are caught by mtime_ns plus a size check. Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6499794c12 |
fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache (#409)
* fix(testing): add MockCacheManager.get_cached_data_with_strategy/save_cache ledmatrix-leaderboard's data_fetcher.py calls these two real-CacheManager methods (src/cache_manager.py:313,817), but MockCacheManager had neither -- update() always hit an AttributeError, caught by a broad except and logged, so the harness rendered an empty-but-green leaderboard on every test run without ever exercising real standings data. Both mocks delegate to the existing get()/set() -- a mock doesn't need the real strategy's per-data-type max_age/market-hours timing, plugins under test just need the methods to exist and round-trip whatever was cached. * fix(testing): clear get_cached_data_with_strategy_calls in MockCacheManager.reset() reset() cleared get_calls/set_calls/delete_calls but not the newer get_cached_data_with_strategy_calls tracker, so a reused mock (e.g. across test cases sharing a fixture) retained stale strategy-call records after reset(). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
ChuckandGitHub
|
3d347a368a |
fix(testing): stop plugin enabled:false schema defaults from silently disabling harness tests (#408)
check_plugin.py, render_plugin.py, and the pytest plugin matrix each built
config as {"enabled": True} then merged in config_schema.json's defaults on
top, letting a plugin's own enabled:false default (a reasonable choice for
a seasonal/opt-in plugin -- 15 of 23 real plugins ship one) silently win.
Every harness/CI render of those plugins was testing "disabled, do
nothing" rather than real behavior.
Extract build_full_config() into testing/loading.py (already the shared
home for plugin-discovery/config-default logic) and use it from all three
call sites: schema defaults, then a forced enabled=True, then harness.json's
config, then the caller's explicit config -- so a test can still
deliberately disable a plugin on purpose, it just can't happen by accident
via the plugin's own shipped schema default anymore.
|
||
|
|
0aca40cf3a |
perf(plugins): run scheduled updates off the render thread (#407)
* perf(plugins): run scheduled updates off the render thread plugin update() executed inline in the render loop — execute_update's internal thread.join(timeout=30) blocked it, so one slow plugin HTTP fetch froze scrolling for the whole fetch (up to 30s; DNS-retry storms made this a regular occurrence on flaky networks). Scheduling stays on the render thread and keeps every existing gate (enabled, circuit breaker, can_execute, interval); due updates are now enqueued to a single background worker (serialized — same one-at-a-time execution as before, no thundering herd). RUNNING is set at enqueue so can_execute blocks re-entry alongside the pending-set dedup. Per-plugin locks make the old implicit update/display no-overlap guarantee explicit: the worker holds the plugin's lock through its update; the display side try-locks and, when the plugin is mid-update, holds the last frame for that iteration — reported as success so a mid-update skip never advances the rotation. Unlike before, the guarantee now also holds across the post-timeout window (previously the lingering update thread overlapped display()). Deadlock-free by construction: the worker takes one lock; display never blocks. Timeout semantics unchanged (lingering daemon thread documented). Kill switch: plugin_system.synchronous_updates: true restores the inline path. 8 new concurrency tests (non-blocking scheduler, overlap assertion under a hammering display loop, lock release on failure/timeout paths, dedup, kill switch); 4-min devpi soak clean (updates completing, rotation advancing, no stuck RUNNING states). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(plugins): fix skipped-frame health/force_change tracking, config validation, and lock-lifetime gaps in async updates Addresses PR #407 review findings: - display_controller: only clear force_change / record health success when display() actually ran this frame, not when the frame was skipped because the plugin's lock was busy (a skip must preserve a pending mode-switch force_clear). - plugin_manager: replace bool() coercion of synchronous_updates with explicit isinstance validation of plugin_system/synchronous_updates, failing safe to synchronous mode (with a logged reason) on malformed config instead of silently defaulting to async. - plugin_manager: _update_worker_loop now acquires the plugin lock before looking up its instance and re-checks under the lock, so an unloaded plugin's lifecycle state is never resurrected to ENABLED. - plugin_manager + display_controller: move lock ownership (and, for updates, RUNNING/pending lifecycle bookkeeping) into the actual update()/ display() call itself rather than the timeout-wrapped caller, so the lock stays held for the real operation's duration even after PluginExecutor's own join(timeout) elapses and a lingering daemon thread keeps running in the background. - DisplayController.cleanup() now stops the update worker before tearing down display/cache resources; stop_update_worker() logs when the join times out instead of failing silently. - test_async_plugin_updates: rewrite test_unloaded_while_queued_is_harmless to exercise the public unload_plugin() lifecycle (via a deterministic blocker) instead of deleting pm.plugins directly, and add a regression test proving the plugin lock stays held through PluginExecutor's own timeout while the real update() call is still running. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9837315308 |
perf(display): dirty tracking in update_display + plugin FPS declaration (#406)
* perf(display): dirty tracking in update_display + plugin FPS declaration update_display now skips SetImage+SwapOnVSync when the frame is byte-identical to the last pushed one (adler32 digest) AND brightness is unchanged — brightness is part of the digest, and set_brightness additionally resets it, so a dim-schedule change can never be skipped. clear() resets the digest (it writes to the matrix directly). Skipping a swap is hardware-safe: the panel refreshes the current frame from the driver's own thread; swaps only change content. Kill switch: display.dirty_tracking: false restores always-push. display_controller's high-FPS decision gains a precedence step: a plugin exposing needs_high_fps is honored first (so static-image can declare False for still PNGs and stop burning a 125fps loop on them); static-image without the attribute keeps its historical forced high-FPS (GIF back-compat); scrolling logic is otherwise unchanged. Verified with 7 tests against the real DisplayManager on RGBMatrixEmulator (identical-frame skip, pixel-change push, clear and brightness invalidation, snapshot-through-skip, kill switch) plus the 202-test display/controller/vegas suites, and a clean devpi deploy. Audit: every SetImage/SwapOnVSync/Clear/brightness call site is inside display_manager — no external writer can bypass the digest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(display): serialize update_display, narrow brightness exception, log fixes CodeRabbit review on #406, verified against current code: - update_display() can genuinely be called from background threads (some sports base classes call it directly from inside update() for an immediate "live" refresh), not just the render loop — confirmed via the existing follower-mode gating wrapper in display_controller.py, which exists specifically because "background plugin threads" can reach it. Without a lock, two callers could both pass the digest check before either writes _last_pushed_digest back, causing a redundant push, or interleave the offscreen/current canvas swap. Added self._update_lock (RLock, in case of re-entrant callers) around the full method body so every call site is automatically covered — no caller changes needed. (No prior lock existed to reuse on DisplayManager; this adds one.) - Narrowed the brightness-read exception handler to AttributeError, matching the established pattern in get_brightness()/set_brightness() — a getattr() with a default already swallows AttributeError, so the only case this guards is the property getter itself raising, and the established pattern treats that as an expected, specific failure mode rather than something to blanket-catch. - FPS-check debug log now includes the plugin_id already in scope (previously only active_mode) and a "[DisplayController]" prefix for grep-ability, matching the sibling log two lines below it. - test_display_dirty_tracking.py: dm fixture and test_config_flag_wires_through now reset the DisplayManager singleton on teardown, matching the pattern test_display_manager.py already uses elsewhere in the same file family. - test_snapshot_still_written_on_skip previously only exercised the non-skip (push) path despite its name; now performs a second update that meets the skip conditions (identical frame) and asserts the snapshot is still written even though the panel push itself is skipped. All 7 dirty-tracking tests pass, plus the full display_manager/ display_controller/vegas suite (140 passed). Full repo suite has only the 5 known pre-existing failures (double-sided config x2, state_reconciliation x2, and test_circuit_breaker's conftest.py mock signature drift — the latter fixed in #400, which this branch's base predates). --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c1fa5094be |
fix(store): plugin updates keep the old install until the new one succeeds (#405)
* fix(store): plugin updates keep the old install until the new one succeeds Both reinstall paths in update_plugin — the monorepo-migration remote switch AND the routine archive update every store user hits — deleted the installed plugin directory BEFORE downloading its replacement. A mid-update failure (bad network, registry error) permanently destroyed the plugin. Seen in the field: a Pi with broken DNS lost 12 plugins in one update pass during the monorepo migration. New _reinstall_with_rollback: rename the old install aside (using the '.standalone-backup-' name pattern plugin discovery already excludes), run install_plugin, remove the aside on success — restore it on ANY failure, clearing partial-download debris first. A stale aside from a previous crash is cleared before starting. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(store): serialize concurrent updates per plugin, check cleanup results CodeRabbit review on #405 flagged two things in _reinstall_with_rollback, both verified against current code: - Real race: the web UI runs Flask with threaded=True and there's a single update route, so two overlapping requests for the same plugin_id (double-click, two tabs) can interleave. The loser could rename the winner's in-progress install aside mid-download, deleting its own rollback safety net — worse than the bug this function exists to fix. Added a lazy per-plugin_id lock dict (mirrors the plugin_manager per-plugin lock pattern) held for the whole function. - _safe_remove_directory's return value was ignored at both call sites. Stale-aside cleanup failure now aborts cleanly instead of falling through to a rename that would fail anyway with a less useful error; post-success backup-removal failure now logs instead of failing silently (still returns True — the update itself succeeded, and the next update self-heals the leftover aside). Left the third nitpick (test_stale_aside_from_previous_crash_is_cleared) addressed by asserting the stale dir is actually gone and that install_plugin was reached, rather than just the end-to-end result. Added a concurrency regression test asserting install_plugin never runs for the same plugin_id while another call is in flight. --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
4d49b0f892 |
perf(display): snapshot mirror — viewer gating, digest skip, keepalive (#404)
The display service PNG-encoded its frame to /tmp/led_matrix_preview.png at 5 fps, 24/7 — identical frames, no viewers, per-call imports and a chmod every write. On the devpi baseline the display service idles at ~92% CPU; this was one of its biggest fixed costs. - New pure policy (src/common/snapshot_policy.py, unit-tested off-Pi): WRITE changed frames at full rate only while a viewer is watching, at a 30s idle cadence otherwise; NEVER re-encode unchanged frames — bump mtime (os.utime) every 20s instead, keeping the health check's snapshot-age liveness proxy (60s threshold in api_v3) green. Cross- referencing comments guard the two constants. - Viewer detection: the web SSE display broadcaster (which only runs while browsers are subscribed) touches /tmp/led_matrix_preview_viewer each loop; the display service stats it at most 1/s. On viewer arrival the write clock resets so the first frame lands within ~1s. - Hoisted the per-call pathlib/permission_utils imports; directory permissions ensured once instead of every frame. Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
efe76d3add |
perf: hot-path micro fixes in the render loop (#403)
* perf: hot-path micro fixes in the render loop - _check_wifi_status_message stat'd the status file on every render iteration (60+ fps) for a message whose lifetime is seconds; throttle the check to 1 Hz with a cached result. - Demote the per-iteration "Display active, processing mode" INFO to DEBUG and convert the remaining eager f-string logs to lazy % args — the devpi baseline showed ~9 journald lines/sec, which is both noise and SD-card wear. - Vegas cycle-end blank frame: hoist the inline PIL import and reuse a preallocated buffer instead of allocating per cycle wrap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix: initialise wifi-status throttle state in __init__ Codacy (pylint access-member-before-definition) on #403: the throttled early-return read _wifi_status_last_result relying on the non-local invariant that the first call always passes the throttle window and assigns it. Correct at runtime, but fragile — initialise both throttle fields in the constructor and drop the getattr fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
273d9962d1 |
fix(cache): stop fsync-hammering the SD card on unchanged data (#402)
DiskCache.set wrote every key as mkstemp -> json.dump(indent=4) -> flush+fsync -> replace -> chmod, on the persistent cache dir — dozens of force-flushed SD writes per minute on an API-heavy install, mostly rewriting identical data every plugin update cycle. - Serialize once, compact (no indent): cache files are machine-read only; indenting multiplied the bytes written. - Skip the disk when the payload for a key is unchanged (adler32 map, per-process); refresh the file mtime instead so records relying on mtime for TTL don't expire early. Self-heals if the file was removed externally (expiry cleanup). - Drop the per-write fsync: os.replace already guarantees readers never see a torn file, and cache data is re-fetchable — the flush bought nothing but card wear. API unchanged; DateTimeEncoder round-trip covered by tests. Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9e3b5f366e |
fix(core): harden text-measurement caches; surface snapshot failures (#400)
* fix(core): harden text-measurement caches; surface snapshot failures Deep-dive findings, all three latent on every 24/7 install: - font_manager.metrics_cache and display_manager._text_width_cache were unbounded dicts keyed by (text, id(font)). Two problems: keys embed the measured TEXT, so ever-changing strings (a clock, a live score, a ticker) grow them without limit; and id()-keying without holding a reference means a garbage-collected font's id can be recycled by a DIFFERENT font, silently returning wrong widths/metrics (classic plugins create fonts per render, so this is reachable). Both caches are now LRU-bounded (1024) and pin the font in the entry so its id stays valid. metrics_cache also keyed on the text itself instead of hash(text), removing a collision path. - _write_snapshot_if_due logged failures at DEBUG — invisible at the default level. The snapshot's mtime is the web UI's display mirror AND its hardware-liveness proxy, so a quiet failure freezes the mirror and makes health checks lie (seen in the field: a stale root-owned /tmp file froze it for a day). Failures now WARN, rate- limited to once per 5 minutes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * test: sync mock cache manager signature with CacheManager.get test_circuit_breaker has been failing on main: plugin_health passes memory_ttl= to cache_manager.get(), and the conftest mock's signature was never updated — the same component/double drift class as the monitored_update bug (#392). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6edd80d9f3 |
fix(schedule): stop stray 'days' data from overriding Global schedule (#399)
save_schedule_config never persisted the schedule's 'mode' field, and _check_schedule inferred per-day vs global purely from whether a 'days' dict was present for the current day. Config migration (_merge_template_defaults) re-adds the template's 'schedule.days' (all days disabled by default) whenever it's missing from the user's saved config - which is exactly the case after saving Global mode, since that save path intentionally pops 'days'. The result: a user on Global mode would get their schedule silently reinterpreted as per-day, with today's day disabled, blanking the display. Persist 'mode' on save and have _check_schedule honor it explicitly (mirroring how _check_dim_schedule already does), so a resurrected 'days' dict can't override an explicit Global selection. Falls back to the old inference behavior only when no 'mode' is recorded. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
1c7a0cef66 |
fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation (#398)
* fix(vegas): restore live plugin-update refresh dropped by sync refactor Investigating a user report that Vegas scroll mode doesn't update scores or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live score change reached the ticker within a few seconds instead of waiting for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed plugin_last_update timestamps to detect which plugins got fresh data and called coordinator.mark_plugin_updated() for each, and should_recompose() checked has_pending_updates_for_visible_segments() to trigger an immediate hot-swap. PR #330 (May 14, multi-display wireless sync) refactored both call sites while adding sync support and silently deleted this entire mechanism -- not just gated it behind the new sync-mode deferral it legitimately needed, but removed it outright. The result: VegasModeCoordinator. mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_ segments() have been fully implemented but never called from anywhere since. Vegas mode's only remaining freshness sources are a 5s content cache TTL (fine) and full recompose at cycle boundaries, which depending on min/max_cycle_duration can be minutes away -- so live scores/status can sit stale far longer than a user would expect from a "live" ticker. Fix: - Restored _tick_plugin_updates_for_vegas() in display_controller.py, wired as the Vegas coordinator's update callback in place of the plain _tick_plugin_updates(). Diffs plugin_last_update before/after the tick and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each plugin that actually got new data (rather than returning the list, since the callback interface no longer consumes a return value). - Restored the has_pending_updates_for_visible_segments() check in render_pipeline.should_recompose(), positioned after (not instead of) the sync-mode early return PR #330 added, so standalone installations regain immediate refresh while synced leader/follower pairs correctly keep deferring hot-swaps to cycle boundaries as PR #330 intended. Test plan: - Added test_display_controller_vegas_tick.py and test_vegas_render_pipeline_recompose.py -- neither area had any prior test coverage, which is very likely why this regression went unnoticed for ~2.5 months. - Verified both new test files fail against the pre-fix code (swapped in the current main versions of both files) with exactly the expected errors -- AttributeError for the deleted method, and the recompose assertion returning False instead of True -- then pass against the fix. - Confirmed the sync-mode deferral this restoration must not break still holds: test_sync_active_defers_pending_updates_to_cycle_boundary. - Full related suite (test_vegas_plugin_adapter, test_vegas_config, test_display_controller_plugin_toggle, test_display_controller_ optimizations, test_plugin_system): 108 passed, 1 pre-existing failure unrelated to this change (test_circuit_breaker, stale mock signature). - Full CI plugin-safety suite (test_harness, test_visual_rendering, test_plugin_matrix): 52 passed, 2 pre-existing skips. * fix(vegas): lock plugin_last_update snapshot/diff against concurrent mutation _tick_plugin_updates_for_vegas() snapshotted and later re-iterated plugin_manager.plugin_last_update from the Vegas background update-tick thread while the main render loop (or other callers) could mutate the same dict concurrently — a real race (unprotected dict iteration/mutation across threads), not just a style nit. Move the snapshot/update/diff into a new locked PluginManager.run_scheduled_updates_with_changes() so all reads and mutations of plugin_last_update happen under one lock, and update DisplayController to use it. The lock is only held around the dict accesses, not the update pass itself, so slow plugin update() calls don't serialize against other callers. Also add a regression test covering that the Vegas coordinator is wired to the Vegas-aware tick callback rather than the plain one. Skipped as not worth the change: - Narrowing the broad `except Exception` around vc.mark_plugin_updated(plugin_id) to specific types: it's a deliberate per-plugin isolation boundary (matches the same pattern used elsewhere in this file for plugin/coordinator calls) and there's no documented, stable set of exceptions that call can raise to narrow to. - Adding an inactive-DisplaySyncManager test to test_vegas_render_pipeline_recompose.py: verified VegasModeCoordinator.set_sync_manager() already normalizes a SyncRole.STANDALONE manager to None before handing it to the render pipeline (src/vegas_mode/coordinator.py:152-156), so should_recompose()'s `is not None` check is correct in practice; the suggested case is already covered by that normalization. --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
ChuckandGitHub
|
6052a60d22 |
fix(vegas): restore live plugin-update refresh dropped by sync refactor (#395)
Investigating a user report that Vegas scroll mode doesn't update scores or game status. Root cause: PR #299 (Mar 28) added a mechanism so a live score change reached the ticker within a few seconds instead of waiting for a full scroll cycle -- _tick_plugin_updates_for_vegas() diffed plugin_last_update timestamps to detect which plugins got fresh data and called coordinator.mark_plugin_updated() for each, and should_recompose() checked has_pending_updates_for_visible_segments() to trigger an immediate hot-swap. PR #330 (May 14, multi-display wireless sync) refactored both call sites while adding sync support and silently deleted this entire mechanism -- not just gated it behind the new sync-mode deferral it legitimately needed, but removed it outright. The result: VegasModeCoordinator. mark_plugin_updated() and StreamManager.has_pending_updates_for_visible_ segments() have been fully implemented but never called from anywhere since. Vegas mode's only remaining freshness sources are a 5s content cache TTL (fine) and full recompose at cycle boundaries, which depending on min/max_cycle_duration can be minutes away -- so live scores/status can sit stale far longer than a user would expect from a "live" ticker. Fix: - Restored _tick_plugin_updates_for_vegas() in display_controller.py, wired as the Vegas coordinator's update callback in place of the plain _tick_plugin_updates(). Diffs plugin_last_update before/after the tick and calls vegas_coordinator.mark_plugin_updated(plugin_id) for each plugin that actually got new data (rather than returning the list, since the callback interface no longer consumes a return value). - Restored the has_pending_updates_for_visible_segments() check in render_pipeline.should_recompose(), positioned after (not instead of) the sync-mode early return PR #330 added, so standalone installations regain immediate refresh while synced leader/follower pairs correctly keep deferring hot-swaps to cycle boundaries as PR #330 intended. Test plan: - Added test_display_controller_vegas_tick.py and test_vegas_render_pipeline_recompose.py -- neither area had any prior test coverage, which is very likely why this regression went unnoticed for ~2.5 months. - Verified both new test files fail against the pre-fix code (swapped in the current main versions of both files) with exactly the expected errors -- AttributeError for the deleted method, and the recompose assertion returning False instead of True -- then pass against the fix. - Confirmed the sync-mode deferral this restoration must not break still holds: test_sync_active_defers_pending_updates_to_cycle_boundary. - Full related suite (test_vegas_plugin_adapter, test_vegas_config, test_display_controller_plugin_toggle, test_display_controller_ optimizations, test_plugin_system): 108 passed, 1 pre-existing failure unrelated to this change (test_circuit_breaker, stale mock signature). - Full CI plugin-safety suite (test_harness, test_visual_rendering, test_plugin_matrix): 52 passed, 2 pre-existing skips. |
||
|
|
7f7f0d6464 |
feat: adaptive layout system — size-aware regions, crisp font ladders, image fitting (#393)
* feat(layout): adaptive layout & font scaling system for plugins Add src/adaptive_layout.py — opt-in core helpers so plugins render legibly on any panel size without hand-tuned per-display layouts: - Region: integer rect algebra (bands/columns/weighted splits/centering) that partitions space so text bands can't overlap by construction - Font ladders: ordered (family, size) steps known to render crisply (LADDER_GRID: X11 BDFs at native sizes; LADDER_ARCADE: PressStart2P at 8px multiples) — fitting walks the ladder instead of scaling pixel fonts fractionally - LayoutContext: breakpoint tiers, geometry scale vs. a declared design size, and cached fit_text/fit_lines/font_for_rows queries Generalizes the three patterns proven in the field: f1-scoreboard's scale factor, masters-tournament's tiers, baseball-scoreboard's font fallback ladder. Wiring: BasePlugin gains a lazy .layout property and draw_fit(); FontManager gains get_native_bdf_size() and a cache_generation counter; manifest schema gains display.design_size and requires.display_size max_width/max_height; 96x48 joins DEFAULT_TEST_SIZES; the bounds-check harness records negative-coordinate draws; TextHelper's broken measurement helpers are fixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): adaptive image fitting + composite region helpers Add src/adaptive_images.py — the image counterpart to fit_text: - fit_image(img, box, mode=contain|cover|fill_height|stretch, crop_to_ink, anchor, resample, upscale) promoting the proven plugin patterns (football's crop-to-ink fill-height logos, masters' cover crop + NEAREST flags, static-image's letterbox). Upscales by default — thumbnail()'s downscale-only behavior is why imagery stays tiny on big panels. - draw_fitted_image() pastes aligned within a Region with alpha mask. - One central Pillow>=9.1 RESAMPLE shim replacing ~15 plugin copies. LayoutContext.fit_image() caches results per (identity, box size, options) with a 64-entry LRU; id()-keyed entries pin the source image. BasePlugin.draw_image() is the one-liner adoption path beside draw_fit. Composites in adaptive_layout.py: Region.offset() (user x/y-offset passthrough), scoreboard_regions() (the two-logos-plus-score card math duplicated across six sports plugins, logo_slot = min(H, W//2)), and media_row() (art-left/text-right). Fix LogoHelper's size-blind cache key (stale sizes on panel change); deprecation note on dead image_utils.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(harness): scale-up fill check, config variants, multi-size dev gallery Quality gates for adaptive layout: - fill_metrics()/check_scale_up() in the safety harness: overflow catches content too big for a panel, but nothing caught content that stays tiny on panels >= 2x the plugin's declared design size. The check measures lit-content extents and warns (or fails, when a plugin opts into "fill_check": "strict" in test/harness.json) below 50% coverage on the doubled axis. Warn-only by default so no existing plugin breaks. - harness.json "variants": extra runs with config overlays and their own golden dirs, so an opt-in mode (e.g. layout_mode: adaptive) is golden- tested beside the classic default. check_plugin.py loops base + variants and labels variant results mode@name. - Dev preview server: GET /api/sizes (harness size sample), POST /api/render-matrix (render at up to 12 sizes in one call), size-preset dropdown, and an "All Sizes" side-by-side gallery in the preview UI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(plugins): adaptive-lib discoverability + advisory version compat warning Discoverability: re-export the adaptive layout/image API from src.common (the blessed-helpers package plugin authors already know) — canonical paths stay src.adaptive_layout / src.adaptive_images so nothing breaks. Document it in src/common/README.md and cross-link ADAPTIVE_LAYOUT.md from the developer docs authors actually read (quick reference, API reference, advanced dev, font manager, dev preview, plugin dev guide); ADAPTIVE_LAYOUT.md gains adaptive-images, composite-layouts and preserving-user-customization sections. Compat: PluginLoader now logs one advisory warning (never raises) when a plugin's manifest declares a min LEDMatrix version newer than the running core, checking the min_ledmatrix_version / requires.* / versions[] spellings found in the wild. Guarded against stale core version numbers. src/__init__.py __version__ bumped 1.0.0 -> 3.1.0 to match the latest release tag (v3.1.0) — it had never been updated and the compat check needs a truthful number. NOTE: verify this matches the intended release numbering before the next tag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): add measure_font_crispness — verify a ladder rung isn't blurry PIL antialiases TTF outlines by default; a 'pixel-style' font only rasterizes without antialiasing at specific sizes (for PressStart2P: exact multiples of its 8px design grid). A ladder rung at an unverified size silently renders blurry on an LED panel — this exact bug shipped in both text-display's and football-scoreboard's custom TTF ladders (non-8-multiple PressStart2P sizes, and '5by7.regular'/'4x6-font' at sizes that were never actually crisp). measure_font_crispness(font, sample_text) renders the sample and reports the fraction of ink-bbox pixels that are neither pure black nor pure white. BDF fonts (real bitmaps) always score 0.0; TTF ladders should be verified against this before shipping — see the new TestFontFitting::test_ladder_arcade_is_crisp pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): add fit_text_proportional — proportional sizing vs. always-maximize fit_text always picks the largest ladder rung that fits its box. That's right when an element owns dedicated space, but wrong when several independently-fitted elements need to stay visually harmonious as the panel grows: a score's box might have generous room while a neighboring logo scales by a fixed geometry factor via px() — fit_text lets the score balloon out of proportion (even overlapping the logo) even though its individual pick is technically correct. fit_text_proportional(text, box, base_size_px, ladder) instead targets base_size_px * self.scale (the same scale factor px() already uses), picking the nearest ladder rung at or below that target, still capped to what fits the box, floored at the smallest rung when the target is below every rung. Refactored the shared largest-that-fits/ellipsize walk into _walk_ladder() so fit_text and fit_text_proportional don't duplicate it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(layout): fit_text_proportional gains an axis-specific scale override self.scale (min(width_ratio, height_ratio)) is the right conservative default for anything whose aspect ratio matters, but a caller whose surrounding composition already scales along a single axis — e.g. football-scoreboard's logo_slot = min(height, width // 2), which tracks height alone — needs text sized the same way, or it reads as under-scaled next to logos that grew on a panel that only got taller (128x32 -> 128x64: self.scale stays 1.0 since width didn't grow, but logos still double). fit_text_proportional(..., scale=None) now accepts an explicit override; None keeps the existing self.scale default. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(layout): scoreboard_regions reserves real center space at 2:1 aspect ratios logo_slot = min(height, width // 2) has a blind spot: at exactly 2:1 aspect ratio (width == 2 * height -- a very common shape: two, four, or more square modules stacked into a taller panel) width // 2 and height are equal, so the two logo slots claim the ENTIRE width and leave zero pixels for a center column, no matter how large the panel gets. Not a 'small panel' problem -- 96x48, 128x64, and 256x128 (all exactly 2:1) hit it identically, while the 128x32 design baseline and panels like 192x48 or 256x32 never do, because height is already the tighter constraint there. Two new parameters fix it in the one shared helper every scoreboard-style plugin composes through: - min_center_fraction / min_center_design_px reserve at least max(width * fraction, design_px * ctx.scale) for the center column, capping logo_slot further when needed. The scaled design-px term matters on small panels where a flat fraction alone reserves too little absolute space. - score_bleed_fraction extends the score's own fit box (not the logo slots themselves) a controlled amount into each side -- the same way real broadcast scoreboards let a big score number's edges cross into the team marks flanking it. Without this the reserve alone can still be too narrow for a short score to render without truncating. score_area is now genuinely narrower than the full card width (previously identical to status_band/detail_band, which still span the full width and overlay the logos -- short text there was never the problem). Verified against the full harness size spread: a real game score like '17-21' never needs ellipsis at any tested 2:1-or-tighter aspect ratio (test_score_never_needs_ellipsis_for_a_short_score), and wide panels (128x32/192x48/256x32-style) are provably unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: document scoreboard_regions' center-reserve and score-bleed params Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address CodeRabbit review on PR #393 - docs: scope the self.layout note to BasePlugin subclasses (others build a LayoutContext directly) and make explicit that adaptive layout is opt-in — classic rendering stays unless a plugin adopts the APIs. - dev_server: broaden the render-request catch (a bad manifest.json now returns a clean 400 instead of an unhandled 500) and stop echoing raw exception text in the loader-failure responses — full tracebacks go to the dev server's console log instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): allowlist plugin_id before any path lookup CodeQL (py/path-injection): plugin_id arrives in request input and flows into filesystem paths via find_plugin_dir. Gate it with the same ^[a-zA-Z0-9_-]{1,64}$ allowlist the web UI's pages_v3 uses, at the single choke point every route resolves through. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): lexical containment check on resolved plugin dirs CodeQL doesn't recognize the interprocedural allowlist as a path-injection barrier; add the canonical one — normalize (without following symlinks, since dev plugins are commonly symlinked into plugins/) and require the result to stay inside the search dir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): inline normpath containment barrier before render CodeQL doesn't credit the sanitization inside find_plugin_dir along this flow; apply its documented barrier (normpath + startswith against the allowed roots) inline in _parse_render_request, on the exact path that reaches the render/load sinks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): derive plugin dir from trusted directory listings CodeQL's barrier-guard recognition doesn't see a startswith check inside an any() comprehension, so the normalize-and-prefix approach still flagged. Break the taint outright instead: after lookup, re-derive the directory by enumerating the search dirs (iterdir) and matching by path equality — the Path used for all downstream file access is built solely from trusted listings, never from request input. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam * fix(dev-server): use os.scandir for path-injection barrier, redact stack traces from render responses CodeQL doesn't model Path.iterdir() as a taint-clearing enumeration the way it does os.scandir() -- _trusted_plugin_dir's iterdir-based rebuild still traced plugin_id through to the manifest.json open(). Switched to scandir, matching the pattern already verified clean on PR #396. Also stops surfacing raw exception text (update()/display() failures) in the JSON render response -- logs full detail server-side via exc_info instead, returning only the exception class name to the client. And drops path values from three plugin_loader debug/error logs that CodeQL flags as clear-text-logging of externally-influenced data, keeping plugin_id (not flagged) for context. * fix(dev-server): remove conditional-reassignment ambiguity in plugin_dir resolution CodeQL's path-injection flow still traced through _parse_render_request after the scandir fix -- the tainted find_plugin_dir() result and the scandir-derived _trusted_plugin_dir() result shared the same variable name (plugin_dir), reassigned only on the truthy branch. That merge point apparently isn't treated as a barrier by the flow analysis, so it kept tracing the pre-reassignment value through to the manifest open(). Split into two distinct names -- candidate_dir (tainted, used only to call _trusted_plugin_dir) and trusted_dir (the only name used for any downstream file access) -- so there's no reassigned variable for the flow to walk through. * fix: remove unused imports flagged by Codacy Union in adaptive_images.py and field in adaptive_layout.py are both imported but never used -- the last two Codacy findings on this PR, matching the same fix already applied on PR #396. * fix(layout): bound the fit cache; never alias the source image in fits Two latent issues found in a self-review pass: - LayoutContext._fit_cache was an unbounded dict (the image cache got an LRU cap, the text-fit cache didn't). Cache keys embed the fitted TEXT, so a plugin fitting changing strings — a live game clock, a ticker — on a 24/7 service grows it forever. Now LRU-bounded at 512 entries via the same pattern as the image cache. - fit_image returned the caller's ORIGINAL image object when the source was already RGBA at target size (contain/fill_height, no ink crop). ImageFitResult is documented as an independent copy, and LayoutContext caches results — an aliased image lets later mutations of the source corrupt cached fits (or vice versa). Copy in that branch. Both covered by new regression tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FqzC1nzTWL4kaqgMaQZFam --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
05e7c43b27 |
fix(plugins): replace dependency marker files with a real satisfaction check (#390)
* fix(plugins): replace dependency marker files with a real satisfaction check The .dependencies_installed hash-marker system only tracked "was this exact requirements.txt hashed before" — not whether the packages it names are actually present. That made it fragile (a wiped venv, a manually removed package, or a lost/corrupted marker forces a needless full pip reinstall or, worse, a false skip) and produced dead weight for the ~10 plugins whose requirements.txt is comment-only (they still paid a pip subprocess on first boot before a marker existed). Replace it with requirements_are_satisfied() in plugin_loader.py, which checks each real requirement line against importlib.metadata directly, so install_dependencies() only shells out to pip when something is actually missing or version-mismatched. Drops the marker file entirely: removed all marker read/write sites in plugin_loader.py and store_manager.py, the now-pointless marker-cleanup step in the git-update path, the unused legacy marker implementation in plugin_manager.py, and the already-stale clear_dependency_markers.sh script. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(security): close path-injection gap in dependency-satisfaction checks CodeQL flagged 2 new high-severity "uncontrolled data used in path expression" alerts at the open() calls inside this PR's new requirements_has_real_deps()/requirements_are_satisfied() -- both are reachable from paths that were never run through the basename+trusted-base sanitiser this codebase already uses elsewhere: - PluginLoader.install_dependencies() only applied that sanitiser when its optional plugins_dir argument was actually passed; the "no plugins_dir" branch trusted plugin_dir_real directly. Made plugins_dir required (not Optional) so that branch can't exist, and added an explicit guard in load_plugin() so install_deps=True without a plugins_dir fails loudly instead of silently. Production's only real caller (PluginManager) always passes plugins_dir already; the harness/dev-server/render-plugin callers all use install_deps=False and are unaffected. - StoreManager._install_dependencies() never sanitised plugin_path at all, and its call sites ultimately derive that path from a plugin's own manifest.json "id" field (install_plugin_from_url) -- a malicious plugin could otherwise point requirements_file outside plugins_dir. Applied the same os.path.basename()-based containment pattern PluginLoader already uses (and that CodeQL recognises as a real sanitiser). Added test_install_dependencies_requires_plugins_dir and test_install_dependencies_rejects_path_outside_plugins_dir to lock in the actual security property, not just quiet the scanner. Verified: all 20 tests in test_plugin_loader.py pass, plus the PR's existing test plan (test_plugin_system.py, test_store_manager_caches.py: 53 passed) and the full CI plugin-safety suite (test_harness.py, test_visual_rendering.py, test_plugin_matrix.py: 52 passed, 2 pre-existing skips) all still pass. * fix(security): replace basename-only sanitiser with a trusted-enumeration check The previous commit's os.path.basename() + os.path.join() pattern (which a pre-existing code comment claimed CodeQL recognises as a sanitiser) did not actually clear the alert -- the next CodeQL run still flagged the same 2 sink lines, plus a new one at the os.path.join() call itself. Taking a substring of tainted data apparently isn't treated as a barrier by this query, whatever the comment assumed. Replaced it with find_trusted_subdir(): enumerate the trusted plugins_dir via os.scandir() and only use a name that scandir itself produced, matched by equality against the caller's requested name. The path is then built from that enumerated entry, not from the caller's string -- a value sourced from iterating a trusted, non-tainted directory carries no taint regardless of what it happens to equal, which is a stronger and more conventional allowlist-style barrier than string-stripping. Applied identically in both PluginLoader.install_dependencies() and StoreManager._install_dependencies(), sharing one implementation. Re-verified: all 65 tests across test_plugin_loader.py (20, including the 2 new security regression tests), test_store_manager_caches.py (35), test_plugin_system.py (10) pass, plus the full CI plugin-safety suite (test_harness.py/test_visual_rendering.py/test_plugin_matrix.py: 52 passed, 2 pre-existing skips). * fix(security): redact URL credentials from pip subprocess output before logging CodeQL flagged 3 clear-text-logging-of-secrets alerts in install_requirements_file() (src/common/permission_utils.py:353,360,371). Pre-existing on main, unrelated to this PR's own diff, but now visible since the path-injection alerts that previously took priority in the annotation list are fixed. The underlying risk is real: pip can echo a private index URL's embedded basic-auth credentials (from a requirements.txt --index-url line or PIP_INDEX_URL) back verbatim in its own stderr/stdout on failure, and this function both logs that output directly and returns it to callers -- store_manager.py's _install_dependencies() logs result.stderr from this same function too. Added _redact_url_credentials(), applied immediately after each of the two subprocess.run() calls (mutating result.stderr/stdout in place) rather than patching each log call site individually. This closes the leak at the source: every downstream use -- the three flagged log lines, the "note" string embedded in the returned stdout, and store_manager.py's own logging of the returned result -- gets the redacted text for free. Verified the fixed-phrase "denied" check (`"a password is required" in result.stderr`) is unaffected, since URL syntax and those phrases don't overlap -- covered explicitly by test_does_not_touch_denied_check_phrases. Added test/test_permission_utils.py (6 tests) covering the redaction helper directly and both subprocess.run() call sites (the sudo-wrapper branch, which this repo's scripts/fix_perms/safe_pip_install.sh makes live, and the no-wrapper fallback branch). All pass. * fix(security): stop interpolating req_file/pip-output into log calls The previous commit's redaction (mutating result.stderr/stdout right after each subprocess.run()) didn't clear CodeQL's clear-text-logging alerts -- same lesson as the path-injection fix earlier in this PR: a static analyzer can't tell "this value was already sanitised two lines up" from "this is still the raw tainted value" just by looking at a single log call in isolation, so it conservatively keeps flagging it regardless of what the redaction function actually does. Removed all dynamic interpolation (req_file, result.stderr) from the 3 flagged logger.warning() calls entirely, replacing them with fixed messages plus (for the one that had it) result.returncode, which is a plain int with no possible taint. The full redacted detail is still available where it actually matters -- in the returned CompletedProcess.stderr/stdout and the "note" text -- just not duplicated into a log line a scanner has to reason about in isolation. Re-verified: all 6 test_permission_utils.py tests still pass (they assert on the returned result, not log call arguments), plus the full test_plugin_loader.py/test_store_manager_caches.py/test_plugin_system.py suite (71 passed, 1 pre-existing deselect, 4 subtests). --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
ChuckandGitHub
|
2ffc57cf40 |
fix(plugin-harness): add no-op process_deferred_updates to test double (#391)
The safety harness's VisualTestDisplayManager (base of BoundsCheckingDisplayManager) doesn't implement process_deferred_updates(), which 5 first-party ledmatrix-plugins call unconditionally between set_scrolling_state() and their scroll-position update: news, odds-ticker, ledmatrix-leaderboard, stock-news, and ledmatrix-stocks. Any of them fails the harness with AttributeError the moment it's touched (surfaced when ledmatrix-plugins#177 had to add a local hasattr guard in ledmatrix-stocks just to pass CI). Add the method as a no-op, mirroring the existing "no-op for testing" pattern already used for set_scrolling_state, so these plugins render under the harness without every touching PR needing its own guard. |
||
|
ChuckandGitHub
|
aab0e9ade0 |
fix(plugin-manager): fix TypeError breaking every plugin's scheduled update (#392)
run_scheduled_updates()'s resource-monitor branch wrapped the update call
in a closure stored as a *class* attribute on a dynamically-built type
(type('obj', (object,), {'update': monitored_update})()). The descriptor
protocol turns a function found via class-attribute lookup into a bound
method on instance access, silently prepending the synthetic instance as
an implicit first argument -- but monitored_update() takes none, so every
call raised "monitored_update() takes 0 positional arguments but 1 was
given", was caught by run_scheduled_updates' try/except, and recorded as
an update failure.
self.resource_monitor is None by default and was dormant until PR #388
("activate dormant plugin health/metrics subsystem") wired it up in both
display_controller.py and web_interface/app.py -- meaning this bug went
live in every real deployment as of that merge (2026-07-09) despite the
buggy line itself dating back to 2025-12-27. In practice this means no
plugin's update() has succeeded since upgrading past #388: circuit
breakers cycle through half-open -> immediate failure -> reopened every
health-check interval forever, and all plugin data (scores, odds, prices,
etc.) goes stale from whatever was last fetched before the upgrade.
Confirmed live on a running instance: odds-ticker (and stock-news,
ledmatrix-stocks, baseball-scoreboard, ledmatrix-leaderboard, of-the-day)
failing this exact way every 5-minute circuit-breaker retry.
Fixed by using types.SimpleNamespace(update=monitored_update) instead of
a dynamic class: SimpleNamespace stores attributes on the instance
itself, so attribute lookup returns the plain function unchanged --
never routed through the class-attribute descriptor protocol that
injects an implicit self.
Added test_run_scheduled_updates_calls_update_with_resource_monitor to
test/test_plugin_system.py using a real PluginResourceMonitor (not a
mock of it), so the test exercises the actual descriptor-binding
behavior that caused this. Verified the test fails with the exact
reported error against the pre-fix code and passes against the fix.
|
||
|
|
978a03b42d |
Add settings tooltips and search to the web UI (#387)
* Add settings tooltips and search to the web UI Help users quickly find settings and understand how each one works. Tooltips: a new delegated controller (static/v3/js/tooltips.js) drives an accessible (i) info tooltip that appears on hover, keyboard focus, and tap. A shared `help_tip` Jinja macro (partials/_macros.html) emits the trigger; the plugin config macro and the core settings partials now surface help text through it. Per the design, the always-visible field help paragraphs are folded into the tooltip to declutter the forms, and the hardware/display settings carry authored detail (default, range, recommendation). Search: a global header search box finds settings across every settings tab — even ones not yet opened — via a lazy client-side index built by scanning the same field markup (static/v3/js/settings-search.js). Selecting a result switches tabs, waits for the field to load, then scrolls to and flashes it. A per-tab filter box hides non-matching fields on the current tab. Plugin settings get tooltips for free by reusing each field's schema `description`; every settings field also gets a stable `setting-<tab>-<key>` anchor id for search navigation. Styling uses the existing --color-* theme vars so light/dark mode both work, and honors prefers-reduced-motion. Adds Flask render smoke tests that assert each settings partial ships tooltips, anchors, and a filter box. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Address Codacy static-analysis findings in settings JS Refactor the two new modules to clear the flagged patterns without any behavior change: - Build the search dropdown with DOM nodes + textContent instead of innerHTML string concatenation, removing the XSS sinks and the manual escapeHtml helper it needed. - Replace numeric index access (index[i], terms[j], opts[idx], currentResults[i]) with array iteration methods, NodeList.item(), and Array.prototype.at() to clear detect-object-injection. - Use === via a shared termsMatch() helper, optional-catch binding, and drop a useless initial assignment. Verified with ESLint (eslint:recommended + eslint-plugin-security) at zero findings and re-ran the headless-Chromium behavior test (tooltip, per-tab filter, global search navigation) — all green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Reduce complexity of revealAncestors in settings search Extract isNodeHidden() and revealNode() helpers so revealAncestors drops below the cyclomatic-complexity threshold. No behavior change; verified with the headless-Chromium test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Resolve remaining Codacy findings in settings search - Validate plugin ids against a strict allowlist (mirroring the server's _SAFE_PLUGIN_ID_RE) before they can appear in a fetch path, so the request URL is never built from unvalidated input (Codacy: user-controlled URL). - Document that the fetched HTML is parsed into an inert document (scripts never run, never inserted into the live DOM) purely to read field text for the search index. - Declare block-scoped locals with const instead of var where they were nested inside conditionals (Codacy: var not at function root). No behavior change; re-verified with ESLint and the headless-Chromium test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Serve the settings search index from the server as JSON Move index building off the client so there is no client-side HTML fetching or DOM parsing (resolves Codacy's variable-fetch and DOMParser flags on the read-only, same-origin index build). - Add GET /v3/settings/search-index (pages_v3.py): renders the settings partials server-side and extracts each field's anchor id, key, label, tooltip, and section with a small stdlib HTMLParser, then caches the result keyed on the installed-plugin set. Parsing the rendered HTML keeps anchor ids identical to the live DOM, so the index cannot drift. - settings-search.js: buildIndex() now does a single fetch of the literal endpoint + .json(); removed the per-partial fetch loop, DOMParser, scanDoc, CORE_TABS, and the plugin-id allowlist. Search, keyboard nav, navigation, and the per-tab filter are unchanged. Net: fewer requests and no client-side HTML parsing. Verified with a new endpoint test in test_web_settings_ui.py (12 pass) and the headless-Chromium test (tooltip, filter, search navigate + flash all green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Address CodeRabbit review on settings search/tooltips - pages_v3: include Durations tab in the search index so setting-durations-* fields are actually indexed - test_web_settings_ui: assert setting-durations-clock is present in the search-index endpoint response - settings-search.js: on index fetch failure, reset buildPromise instead of caching an empty (truthy) index so search can retry - settings-search.js: filterScope returns null (not document) when no tab container matches, and the caller guards, so the per-tab filter can't hide fields across unrelated tabs - settings-search.js: refresh the stale header comment to describe the server-side JSON index flow - app.css: cap #settings-search-results height with overflow-y so the dropdown scrolls instead of overflowing small screens Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Fix stuck search dropdown; add plugin-tab nested-settings filter Global search: - Close the results dropdown on input blur (guarded, with a short delay) so it reliably dismisses when focus leaves — previously it could linger because the only outside-close was a document click that Alpine/HTMX handlers can swallow. - Clear the query text after navigating to a result so refocusing the box doesn't re-open stale results. - Also dismiss on htmx:afterSwap (tab changes / navigation). Per-tab filter (now on plugin tabs too): - Render the shared settings_filter box in the plugin Configuration panel. It auto-wires: the delegated input handler and filterScope already target .plugin-config-tab. - Teach applyTabFilter to reveal matches inside collapsed nested sections (render_nested_section defaults them shut), hide nested-section wrappers with no matches, and restore the original collapsed layout when cleared (only re-collapsing sections the filter itself opened). - Count a visible nested-section as content for its parent heading so the heading isn't hidden while a subsection below still has matches. Adds a plugin-config render test (filter box + nested anchors + tooltips). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Close search dropdown via capture-phase outside-click The dropdown could stay open after clicking away because the only outside-click close was a bubble-phase document listener. The v3 UI is one Alpine app() component full of HTMX/Alpine/widget click handlers; when a click lands inside an element that calls stopPropagation(), the event never bubbles to document and the close never runs. - Replace the bubble-phase document 'click' close with a capture-phase 'pointerdown' listener scoped to #settings-search-wrap. Capture runs before any bubbling stopPropagation can swallow the event, so it always fires; pointerdown also covers touch on the Pi screen. Clicking a result stays inside the wrap, so selection is unaffected. - Guard the debounced input handler so a delayed render can't re-open the box after focus has left (type-then-click-away race). Keeps the existing blur / Escape / htmx:afterSwap closes as secondary paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014gZxznuxw8L92FUMBN3Nqz * Fix settings search dropdown not visually closing .hidden has no effect in this app: app.css is a hand-picked utility subset (no Tailwind build step) and never defines .hidden { display: none }. openResults()/closeResults() only toggled the class, so the dropdown stayed rendered (display: block) even once closeResults() ran - confirmed via computed style in a headless browser. Set style.display directly, matching the fallback already used by revealNode()/collapseNode() elsewhere in this file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
bd9f461f70 |
Add system diagnostics, power controls, and WiFi radio toggle to Tools tab (#389)
* Add system diagnostics, power controls, and WiFi radio toggle to Tools tab Expands the web UI Tools tab with safe, purpose-built controls so users can manage the Pi without SSHing in, instead of an arbitrary-command terminal (the web UI has no auth and CSRF is disabled, so a shell would be unsafe). - System Diagnostics card: renders the existing but previously-unused GET /api/v3/system/status endpoint (CPU, memory, temp, disk, uptime), with a manual refresh and a 10s poll. - System Power section: reboot/shutdown buttons wired to the existing reboot_system / shutdown_system actions, behind a confirm step, with a dedicated powerAction() helper that treats the dropped connection as the expected "going offline" outcome rather than an error. - Network Radio section: WiFi on/off toggle backed by new GET/POST /api/v3/wifi/radio endpoints and WiFiManager.set_wifi_radio() / get_wifi_radio_state(). Disabling WiFi is refused unless a wired connection is present (reusing the existing lockout guards), with an explicit force-off confirmation for advanced users. No new privileged commands: uses nmcli radio wifi on|off (already sudo-allowlisted) and the existing reboot/poweroff grants, so the sudoers-alignment guard test stays green. Bluetooth toggle intentionally omitted since the installer removes the BlueZ stack for LED timing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE * Harden WiFi radio endpoint and diagnostics poll (review feedback) Addresses code review feedback on the Tools tab additions: - api_v3.py: parse the `enabled` POST field with the same string-aware coercion as `force`. A plain bool() cast turned {"enabled":"false"} into True (enabling instead of disabling) for any non-UI API caller. - wifi_manager.set_wifi_radio() now returns a reason code alongside (success, message); the /wifi/radio error response includes it. The Tools UI only shows the force-off confirmation when reason == 'no_ethernet', so a genuine nmcli failure surfaces its real error instead of a misleading "no wired connection" prompt that would just retry into the same failure. - tools.html: gate the 10s diagnostics poll on panel visibility (document.hidden / offsetParent), so switching to another tab stops the recurring /api/v3/system/status calls instead of churning the Pi off-screen. The initial load and manual Refresh remain unconditional. Verified: {"enabled":"false"} now disables (refused w/ reason:no_ethernet), {"enabled":"true"} enables; inline JS passes node --check; py_compile clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE * Narrow WiFi-disable fallback except and add stack trace (review) Addresses a CodeRabbit nitpick: the fallback handler in set_wifi_radio()'s disable path caught bare Exception and logged without a traceback. Narrow it to (OSError, subprocess.SubprocessError) — the errors subprocess.run realistically raises — and log with exc_info=True for full context on the Pi. Anything genuinely unexpected now propagates to the endpoint's outer handler (500), matching the codebase's specific-exception convention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019X3he87Ggr1qt8y7mnFeuE --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
3b93024993 |
feat: activate dormant plugin health/metrics subsystem and surface it in the web UI (#388)
* feat(plugin-system): activate dormant plugin health & metrics subsystem
PluginManager shipped a fully-built health tracker, resource monitor and
circuit breaker that were never instantiated (health_tracker/resource_monitor
were left as None), so the circuit breaker never engaged and the existing
health/metrics API routes always returned "not available".
- DisplayController now wires a PluginHealthTracker and PluginResourceMonitor
onto the plugin manager, enabling the circuit breaker (a repeatedly-failing
plugin's update() is skipped after consecutive failures, then retried after
a cooldown) and per-plugin execution-time metrics. Both persist to the
shared cache.
- load_plugin() now validates each plugin's config against its JSON schema in
a strictly warn/degrade-only way: a violation logs a warning and flags the
plugin degraded in the health tracker, but never changes whether the plugin
loads or its pass/fail behaviour. Adds PluginHealthTracker.set_degraded(),
which never touches the circuit breaker.
- ResourceMonitor CPU/memory sampling now reuses a cached psutil.Process and
reads cpu_percent(interval=None), so monitoring no longer blocks ~100ms per
call on the display loop's update path.
- Fix DiskCache.get() raising TypeError for max_age=None ("never expires"),
which silently discarded persisted plugin health/metrics on read and thus
broke cross-process and post-restart surfacing.
- Fix two dead PluginManager helpers that called non-existent tracker methods.
Tests: new test_resource_monitor, test_plugin_health,
test_plugin_manager_schema_soft; extended test_cache_manager and
test_display_controller.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq
* feat(web-ui): surface plugin health, metrics and load state
With the health/metrics subsystem now active in the display service, expose it
in the web UI (which runs as a separate process from the display loop):
- Wire a health tracker / resource monitor backed by the shared on-disk cache
into the web process so /api/v3/plugins/health and /plugins/metrics read the
data the display service persists.
- Build those route responses per installed plugin id (the tracker's in-memory
view is empty in a fresh web process) so cross-process data is included.
- Add state + error_info to /plugins/installed entries so the UI can show why a
plugin isn't running instead of just loaded:false.
- Add a "Plugin Health" panel to the Tools page (circuit status, avg/max update
time, update count, last error) plus PluginAPI.getPluginMetrics().
Tests: route-level tests for the health/metrics endpoints in test_web_api.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq
* fix(plugin-metrics): refresh cross-process health/metrics reads; type hints
Addresses CodeRabbit review on #388:
- Major: the web process's health/resource trackers cached the first persisted
read in an in-memory dict (and the CacheManager memory tier held max_age=None
entries indefinitely), so a long-lived web process showed the first snapshot
and never reflected the display service's later updates. Add an opt-in
force_reload path (get_health_summary/get_health_state/_load_health_state and
get_metrics_summary/get_metrics) that bypasses the in-memory copy and, via a
new memory_ttl passthrough on CacheManager.get, the cache manager's memory
tier — so each /plugins/health and /plugins/metrics poll reads fresh persisted
state. Default behaviour (force_reload=False) is unchanged for the display
process and existing callers.
- Minor: DiskCache.get type hint is now Optional[int] with the None ("never
expires") semantics documented, matching MemoryCache.get.
Tests: new force_reload staleness cases in test_plugin_health and
test_resource_monitor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvTav268UXv44ub9K11LYq
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
||
|
|
85d321cf33 |
fix: plugin_loader retries with --ignore-installed on apt/pip RECORD conflicts (#386)
* fix: plugin_loader retries with --ignore-installed before assuming apt package satisfies pin install_dependencies treated any "uninstall-no-record-file" pip failure as "dependency satisfied" and wrote the success marker without ever attempting --ignore-installed, unlike install_dependencies_apt.py and safe_pip_install.sh (added in #385 for the Plugin Store/first-time-install paths). A plugin pinning a newer version of a system-managed package (e.g. requests) would silently keep running against whatever version apt shipped, while the marker file claimed the pinned requirement was met. Now retries the same install with --ignore-installed on that specific failure so pip actually lays the pinned version down (shadowing the system-managed copy) before falling back to the prior tolerant behavior if the retry itself fails too. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * chore: suppress Codacy finding on new retry subprocess.run call Same generic Bandit/semgrep pattern-match on non-literal subprocess.run argv flagged in #385's install_requirements_file, now on the new --ignore-installed retry call added here: list-form argv (no shell=True), sys.executable is this process's own interpreter, and requirements_file is built internally by find_plugin_directory, never raw external input. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: tolerate a timed-out --ignore-installed retry, dedupe marker-write logic CodeRabbit review caught a real inconsistency: if the --ignore-installed retry itself timed out, subprocess.TimeoutExpired propagated to the outer handler and returned False, failing plugin load — contradicting the intended "tolerate this specific apt/pip conflict" behavior, where a mere non-zero retry return code already returns True. Wraps the retry in its own try/except so a timeout is logged and tolerated the same way as any other retry failure. Also extracts the marker-writing logic (open/write/chmod, ignoring OSError) into _write_dependency_marker, since it was duplicated identically between the direct-success path and the apt-conflict-retry-fallback path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: revert marker-write dedup helper, resolves CodeQL path-injection alert CodeQL flagged _write_dependency_marker's open(marker_file, ...) as "uncontrolled data used in path expression" (high severity) once the marker-write logic was extracted into its own method. marker_file is actually safe — it's built from safe_plugin_dir, which install_dependencies sanitizes via os.path.basename() (CodeQL's own recognized py/path-injection sanitizer, per the existing comment a few lines above) — but CodeQL's interprocedural analysis doesn't carry that sanitized status across the new method boundary, since the sanitizer call and the open() sink were no longer in the same function. This exact code produced zero CodeQL findings before the extraction (in two duplicated inline blocks) and is unchanged in what data reaches it — only its location moved. Reverting the extraction (keeping CodeRabbit's other, independent timeout-handling fix) restores the previously-clean shape rather than trying to convince the analyzer's cross-function taint tracking that a refactor changed nothing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
63a233f3ed |
fix: dependency installation gaps in Plugin Store and first-time install (#385)
* fix: install plugin dependencies through root-visible installer in Plugin Store install_plugin/update_plugin (store_manager.py) installed requirements.txt with a bare `pip3` off PATH, bypassing the root-visible installer added in #380 for the "Reinstall Plugin Deps" button. Two bugs stacked: (1) `pip3` can resolve to a different Python install than the one that actually runs ledmatrix.service, and (2) even when it resolves correctly, ledmatrix-web runs as a non-root user so the package lands in that user's local site-packages, invisible to root-run ledmatrix.service. Either way the install reports success and writes the .dependencies_installed hash marker, so plugin_loader's own (correct) install-on-load path skips reinstalling — leaving the dependency permanently missing until a user finds and clicks the separate "Reinstall Plugin Deps" tool. This is why users kept hitting "No module named 'astral'" for the weather plugin even after installing it from the Store. Extracts the sudo-wrapper-then-fallback install logic from api_v3.py's _pip_install_requirements into src/common/permission_utils.py as install_requirements_file, and routes store_manager.py's dependency installation through it so the automatic Store install/update path now matches the manual "Reinstall Plugin Deps" path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * chore: suppress Codacy false-positive on subprocess.run in install_requirements_file Codacy's generic subprocess-security rule (Bandit B603 equivalent) flagged the pip/sudo subprocess.run calls in install_requirements_file for lacking a "static string argument" — the standard pattern-based flag for any subprocess.run() call with a variable in its argv list. Both calls use list-form argv (no shell=True, so no shell-injection surface), and the only dynamic value is req_file, a Path built internally by callers rather than raw external input; safe_pip_install.sh independently re-validates it before installing anything as root. Suppresses with inline `# nosec B603` comments matching this codebase's existing convention (see permission_utils.py's own PROTECTED_SYSTEM_DIRECTORIES, display_manager.py, sync_manager.py, etc.). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: first-time install script fails on apt-managed requests package web_interface/requirements.txt and requirements.txt both pin requests>=2.33.0,<3.0.0, but Raspberry Pi OS ships an apt-managed python3-requests with no pip RECORD file. Upgrading it via plain `pip install` aborts with "uninstall-no-record-file" because pip refuses to uninstall a package it has no record of, in place — which is exactly the "Some web interface dependencies failed to install" warning first-time install hits. scripts/install_dependencies_apt.py and scripts/fix_perms/safe_pip_install.sh already work around this with --ignore-installed (lets pip lay the new version down in /usr/local, shadowing the apt copy, instead of trying to remove it first). first_time_install.sh's own direct pip invocations — the per-package requirements.txt loop, the web_interface/requirements.txt install, and the requirements_web_v2.txt fallback — didn't have it. Adds --ignore-installed to all three so first-time install no longer fails on this well-known apt/pip conflict. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * chore: add nosemgrep to subprocess.run calls Codacy still flagged The prior # nosec B603 comments suppressed Bandit's check but Codacy's semgrep-based rule ("subprocess function 'run' without a static string") kept flagging the same two lines as a critical security issue even after that fix landed. install_dependencies_apt.py's _run() already needed both tags together (# nosec B603 B607 ... # nosemgrep) for the identical subprocess.run pattern, so apply the same double suppression here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx * fix: add --ignore-installed to install_requirements_file fallback path CodeRabbit review caught this (confirming a gap already flagged in conversation): the non-sudo fallback pip install in install_requirements_file was missing --ignore-installed, unlike the sudo-wrapper branch and safe_pip_install.sh. Without it, the same apt/pip RECORD-file conflict this PR fixes elsewhere (first_time_install.sh, install_dependencies_apt.py) could still hit installs that fall back to this path (e.g. a plugin's requirements.txt on a host where safe_pip_install.sh isn't set up yet). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X1NnDduw53kTe67i5zWwYx --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
7a9d01342a |
Fix inconsistent Vegas scroll transition gaps caused by plugin-baked padding (#384)
* Strip plugin-baked scroll padding when capturing content for Vegas mode Plugins that build their own ticker image via ScrollHelper.create_scrolling_image() (or that manually pad both ends for a clean standalone loop) carry a solid-black margin up to display_width wide on one or both edges. Vegas mode already adds its own configurable gap around every item, so leaving that margin in place stacked an extra, uncontrolled blank stretch on top of separator_width for whichever plugin took the ScrollHelper-capture path — producing inconsistent transition gaps between modules compared to plugins that provide content natively via get_vegas_content(). _get_scroll_helper_content() now detects and crops any such margin before handing the image to the Vegas render pipeline, so every plugin's gap is governed solely by vegas_scroll.separator_width regardless of which capture path produced its content. * Address CodeRabbit nitpick: warn on double-edge padding crop, add unit tests Logging a double-edge match at warning level (vs. info for a single edge) makes it easy to spot an unexpected crop in the field, since two edges matching at once is a much stronger signal of genuine baked-in padding than one edge coinciding with real all-black content. Also adds test/test_vegas_plugin_adapter.py covering _strip_scroll_padding's branch logic: leading-only, trailing-only, both-edges, no-match, degenerate all-black, missing/undersized display_width, and the info-vs-warning log level. * Add type hints and docstring to test _solid helper (CodeRabbit nitpick) --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
9b2f02681d |
feat(web-ui): detect and surface Raspberry Pi under-voltage/throttling (#383)
* feat(web-ui): detect and surface Raspberry Pi under-voltage/throttling Adds a vcgencmd get_throttled check to the system-status SSE stream and surfaces it in the web UI: - A header badge (next to CPU/Memory/Temp) that stays hidden when healthy, turns red when under-voltage/throttling is happening right now, and yellow if it happened earlier this session but has since cleared. - A dismissible top banner (same pattern as the update-available banner) that appears while under-voltage/throttling is actively occurring, with guidance to check the power supply. Re-appears on a fresh occurrence even if a previous one was dismissed. - A "Power Supply" card on the Overview tab alongside CPU/Memory/Temp/ Display Status. Motivated by a real device showing intermittent brightness flicker that turned out to be ~1 under-voltage event every 30-90s (visible live via `vcgencmd get_throttled` and dmesg's "Undervoltage detected!" messages) -- there was no way to see this from the web UI, only by SSHing in. Returns None on non-Pi platforms (no vcgencmd on PATH), matching the existing guard pattern used for the CPU temperature read. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * feat(web-ui): add power supply diagnostics detail to Tools tab The header badge/banner/Overview card added in the previous commit only show a collapsed "is it bad right now" signal. This adds a "Power Supply" section to the Tools tab with the full 8-flag breakdown (under-voltage, throttled, freq-capped, soft-temp-limit -- each split into "right now" vs "occurred since boot") for actually troubleshooting a recurring issue, plus a pointer to the README's power supply sizing guidance when something is or was flagged. Reuses the existing stats SSE stream (window.statsSource) rather than adding a new endpoint -- the same payload already drives the header/ banner/Overview card, so this just listens for it too. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web-ui): address review findings on power supply monitoring - Overview card left its "--" placeholder forever on non-Pi platforms since the update only handled a truthy data.power. Now explicitly renders "Not available" with a neutral icon/color for that case. - vcgencmd failures logged at debug, invisible in default remote log output. Bumped to warning to match the nearby systemctl failure logging. - The banner/badge/card and the Tools summary line only looked at under_voltage_now/throttled_now (+ occurred), silently ignoring freq_capped_now/occurred and soft_temp_limit_now/occurred from _get_power_status() -- a Pi that's actively soft-thermal-limited or frequency-capped showed a green "OK" everywhere except the detailed flag table buried in Tools. All four surfaces now fold all four "now"/ "occurred" flags into the same active/occurred state. - The banner text was hardcoded to "Under-voltage detected..." even when the actual active condition was throttling/freq-capping/thermal limiting. Added _activePowerConditionLabels() (shared, non-module global scope) to build the banner/tooltip text from whichever flags are actually set. Skipped: TTL-caching _get_power_status() to avoid "multiplying forks across browser tabs" -- that premise doesn't hold against this codebase. _StreamBroadcaster (its own docstring says as much) already runs exactly one shared generator per tick regardless of client count, identical to the uncached cpu_temp file-read two lines above it; there's nothing to multiply. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(web-ui): address second round of review findings on power monitoring - updatePowerStatus's falsy-power branch only hid power-stat, leaving power-warning-banner visible with stale text if _get_power_status() fails transiently. Now hides the banner too and resets the dismissed flag, same as the "not active" branch. - The Tools tab's status badge (and the pre-existing dirty/clean badge right next to it) build class names like bg-${color}-100/text-${color}-800 at runtime. This project hand-rolls its own Tailwind-named utility classes in app.css rather than running a real Tailwind build, and the light-mode base rules for bg-red-100/bg-yellow-100/bg-green-100/ text-red-800/text-yellow-800/text-green-800 were simply never defined -- only some had dark-mode overrides, which are no-ops without a base rule in light mode. Added the missing light-mode bases plus the two missing dark-mode overrides (bg-yellow-100/bg-green-100), fixing both badges. - The "occurred earlier" tooltip was a hardcoded string regardless of which flag(s) actually fired. Generalized _activePowerConditionLabels() to take a suffix ('_now' or '_occurred') and reused it for both tooltips. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * refactor(web-ui): move Power Supply status off the Overview tab The Overview tab's "Power Supply" stat card duplicated what the Tools tab's diagnostics section already shows (summary badge + full flag breakdown), so drop the card and its now-dead JS rather than keep two copies in sync. The header badge and warning banner (visible on every page) are unaffected -- only the Overview-tab card is removed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
7a6bad29fe |
feat(display-controller): hot-reload plugin enable/disable without a restart (#374)
* feat(display-controller): hot-reload plugin enable/disable without a restart Enabling or disabling a plugin in config previously required restarting the display service: the plugin list and available_modes were built once at init and the run loop never revisited them. (Per-plugin config *values* already hot-reloaded; only the enabled set was restart-only.) Now the controller reconciles its running plugins against the config's enabled set whenever that set changes: - The ConfigService watcher thread only sets a `_pending_plugin_reconcile` flag (via a cheap enabled-set diff). It never mutates loop state. - The run loop applies the reconcile on its own thread (top of each iteration, deferred while on-demand is active), so loading/unloading and rebuilding available_modes can't race with rendering. - `_reconcile_enabled_plugins` diffs desired vs running plugins, unloads the removed ones (cleanup + on_disable + config-unsubscribe via the new `_unregister_plugin`) and loads the added ones, then clamps the rotation index so the current mode stays valid. The per-plugin registration done at startup is extracted into `_register_loaded_plugin` and reused by the live-enable path so both build identical state. Extracting it also fixes a latent late-binding bug: the per-plugin config-change callbacks were closures over the loop variable, so every plugin's callback targeted the last-loaded instance; each now binds its own id/instance. Adds test/test_display_controller_plugin_toggle.py covering live enable, live disable, index clamping, no-op when unchanged, and the enabled-set diff. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(display-controller): don't exit on empty available_modes, guard rotation modulo Hot-reload means available_modes can legitimately be empty at startup (no plugins enabled yet) and become non-empty later via the web UI, or vice versa mid-run. Fix four issues found reviewing this PR: - run() exited the process entirely when available_modes was empty at startup instead of idling, permanently defeating the point of live enable/disable for anyone who starts with zero plugins enabled. - The mode-rotation step divided by len(available_modes) unconditionally, raising ZeroDivisionError if the last enabled plugin is disabled between frames. - _reconcile_enabled_plugins() called .get('enabled', False) on a config section without checking it was a dict first, raising AttributeError on a malformed config value. - Minor: pop the config-change callback only after attempting to unsubscribe it, and log the exception in the config-read fallback instead of swallowing it silently. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * fix(display-controller): address review findings on the hot-reload PR - Idle-wait tick was a fixed 30s sleep, delaying pickup of a plugin enabled via the web UI while no modes were active. Shortened to ~1s so it's roughly as responsive as the per-frame check once modes exist. - _unregister_plugin popped the config-change callback from _plugin_config_callbacks even when config_service.unsubscribe() raised, losing the only reference to it. Now only pops on a successful unsubscribe. - _pending_plugin_reconcile was cleared before _reconcile_enabled_plugins() ran, so a retryable failure (e.g. plugin discovery erroring) silently dropped the enable/disable request. _reconcile_enabled_plugins() now returns True/False and the caller only clears the flag on True. - Added a warning log for the malformed-config case (a plugin's config section present but not a dict) so it's actually visible, and updated the existing test to assert it via caplog. Left the broad `except Exception` around config_service.unsubscribe() as Exception -- the current implementation is a simple lock+dict/list op that doesn't document or realistically raise a narrower type, so this is a defensive catch-all, not user error handling. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: ChuckBuilds <charlesmynard@gmail.com> |
||
|
ChuckandGitHub
|
bea00448d3 |
update mlb logos (#382)
update mlb logos Signed-off-by: Chuck <33324927+ChuckBuilds@users.noreply.github.com> |
||
|
|
deaa3d7a98 |
fix: array-table boolean columns always saved as unchecked (#381)
Reported bug: countdown plugin always shows "No Active" even with a
countdown configured and enabled — because every save silently unchecked
it. Root cause: the boolean column's hidden-sentinel input (needed so
unchecked checkboxes still submit "false", since browsers omit unchecked
checkboxes entirely) was hardcoded to value="false" and shared the same
`name` as the checkbox, with no sync between them. Whichever of the two
same-named inputs the save request's form-collection happens to prefer,
the hidden's stale "false" could silently override an actually-checked
box on every save, independent of what the user set.
Fixed in both places this pattern is rendered:
- plugin_config.html: the initial server-rendered row for every array-table
boolean column (affects countdown's per-countdown "enabled" and the
custom-feeds widget's per-feed "enabled") — now sets the hidden's initial
value from the actual data, and syncs it via onchange on every toggle.
- array-table.js: the client-side row renderer used when adding/rebuilding
rows in the browser — same fix, keeping both inputs in sync at render
time and via a change listener.
Verified other checkbox/hidden-input patterns in plugin_config.html and
confirmed they're unrelated/already-safe: the default single-checkbox
renderer (no hidden pair), checkbox-group (array-bracket names with its
own explicit sync), the array-table advanced-props modal editor (single
hidden per name, explicitly written on Save), the top-level plugin
enable/disable toggle (separate immediate hx-post), and the no-schema
fallback checkbox. custom-feeds.js's own client-side row renderer never
creates a hidden sentinel at all, so it isn't affected either.
Verified the fix's rendered output directly with an isolated Jinja render
of the exact snippet: hidden and checkbox now agree ("true"/checked or
"false"/unchecked) for both states, where before the hidden was always
"false" regardless of the actual value.
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
|
cbb8ec41e8 |
Fix: plugin/base requirements installed as web-user, invisible to root-run display service (#380)
* fix: install plugin/base requirements as root so ledmatrix.service can see them
ledmatrix-web.service runs as a non-root user, so "Reinstall plugin
requirements" installed packages into that user's ~/.local site-packages.
ledmatrix.service (the actual display, which loads and runs plugin code)
runs as root and can't see another user's user-site packages, so plugins
with dependencies not already present system-wide would silently fail at
runtime with ModuleNotFoundError even after a "successful" reinstall.
Reproduced and fixed live against a real device (weather plugin's astral
dependency, used for moon-phase data): confirmed the exact failure
("No module named 'astral'" on every almanac cycle) and confirmed it's
gone after this fix.
Adds scripts/fix_perms/safe_pip_install.sh, a root-owned wrapper (mirroring
the existing safe_plugin_rm.sh pattern) that validates the target is
requirements.txt at the project root or under plugin-repos/ or plugins/
before running pip install as root. configure_web_sudo.sh provisions a
narrowly-scoped sudoers rule for it. api_v3.py's install_base_requirements
and install_plugin_requirements actions now use it via `sudo -n`, falling
back to today's current-user-only install (with an explanatory note) if
the wrapper isn't set up yet, so existing installs don't regress.
Also uses --ignore-installed in the wrapper: root's site-packages often has
apt/dpkg-managed copies of common libraries (requests, etc.) with no pip
RECORD file, which pip refuses to upgrade in place and aborts the *entire*
requirements.txt install over — discovered this while testing the fix live,
since a plugin's other already-satisfied-for-the-web-user dependencies had
never actually been attempted as root before.
Also fixes a pre-existing bug in configure_web_sudo.sh where the
display_controller.py/start_display.sh/stop_display.sh sudoers entries used
PROJECT_DIR (scripts/install/, where this script lives) instead of
PROJECT_ROOT (where those files actually live) — visible as the script's
own "File access test" self-check failing. Verified fixed live.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
* fix: invoke safe_pip_install.sh via explicit bash, matching sudoers rule
CodeRabbit caught this on review: the sudoers rule configure_web_sudo.sh
provisions is scoped to "$BASH_PATH $SAFE_PIP_INSTALL_PATH *" (matching
the existing safe_plugin_rm.sh precedent in
src/common/permission_utils.py), but _pip_install_requirements() called
`sudo -n <wrapper> <req_file>` directly, relying on the script's shebang
instead of an explicit bash prefix. sudo matches the literal command line,
so this never matched the allowlisted rule on an install with only the
specific sudoers entries this script provisions — it silently fell back
to the non-root install path every time, which is the exact bug this PR
set out to fix.
This wasn't caught by live testing on ledpi.local because that device
also has a broader, non-standard "NOPASSWD: ALL" grant which masked the
mismatch. Confirmed the fix is correct by reading sudo's documented
command-matching semantics and mirroring the already-proven-working
bash-prefix pattern from permission_utils.py's safe_plugin_rm.sh call
exactly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ
* fix: harden safe_pip_install.sh invocation against bash-path drift + address CodeRabbit nitpicks
CodeRabbit follow-up findings on the bash-prefix fix (
|
||
|
|
c6ce332d49 |
fix(web): restore missing brace in Tools tab HTMX-fallback path (#378)
The `else if (++tries > 100)` block added by #373 was missing its closing `}`, leaving the setInterval arrow function syntactically unclosed. This caused a JS parse error that silenced the entire 1400-line script block — including the EventSource setup — so the connection-status indicator never left its default "Disconnected" state for all users after updating. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |