mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 14:25:08 +00:00
3ea1fd42df2a1713acb0ad04d3e6234fec3957fd
21
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ChuckandClaude Opus 5.5
|
064b9c9912 |
fix(display): a non-numeric plugin duration no longer stops the display; narrow scroll strips no longer raise (#739)
* fix(display): a plugin duration that is not a number no longer stops the display DisplayController._get_display_duration returned whatever the plugin's get_display_duration() gave back. clock-simple, calendar and countdown return their display_duration setting straight from config.json, so a value saved as "20" or null reached _resolve_durations as a string or None, and its `<= 0` check raised a TypeError. Nothing in the loop caught it: run()'s outer handler logged "Unexpected error in display controller" and cleanup() ended the service when that plugin's screen came up, and systemd restarted it into the same crash. The plugin's answer is now read as seconds: a finite number or a numeric string is used (as BasePlugin.get_display_duration already accepts), a number at or below zero still goes to _resolve_durations' 15 s rule, and anything else -- None, a non-numeric string, a bool, NaN, infinity, or a get_display_duration() that raises -- gets the 30 s a mode without a plugin gets. The warning is logged once per plugin, not at every screen. Tests: test/test_display_duration_not_a_number.py, including the real run() on the run-loop harness, which returned at t=30 before the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(scroll): a strip narrower than the panel no longer raises on every frame ScrollHelper._get_visible_portion_integer handled a frame that runs off the end of the strip by copying the strip's tail and then the rest of the frame from its head, which assumed the head was at least that wide. For a strip narrower than the panel that raised "could not broadcast input array" at every position, so get_visible_portion() never returned a frame and the caller logged a traceback each frame. Vegas composes such a strip (lead_in_width defaults to 0) when its content is narrower than the chain. A wrapping frame is now taken column by column modulo the strip's width (np.take, mode='wrap', into the reused frame buffer): the tail then the head, as before, and a narrow strip repeated across the panel. The same path takes a position before the start of the strip, whose [-n:m] slice was empty and made frombytes raise; the integer and sub-pixel fast paths now leave a negative start to it. A zero-width strip is still a black frame. Tests: test/test_scroll_helper_narrow_strip.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
ed0753c1ca |
perf: cheap per-frame and per-fetch savings (#725)
Six small savings with no behaviour change: the odds fetch no longer pretty-prints every response for a debug line; the scroll integer-slice path drops a redundant full-frame np.ascontiguousarray; ledmatrix-web.service gets MALLOC_ARENA_MAX=2 like the display unit; core ESPN responses are parsed via response_json (orjson when installed); and the scroll frame stats go to INFO only for degraded windows plus a 5-minute heartbeat. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
795834811f |
perf(scroll): extend and trim the Vegas strip in place (#701)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements update in place while they scroll One background worker (src/vegas_mode/live_worker.py) redraws a plugin's live elements when its data epoch moves on (update listener) or on their refresh_hz, nearest the screen first, and hands changed pixels lock-free to the render thread, which copies them into the strip between frames (RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most four patches or two screens of bytes a frame, no drawing or locks there. The worker takes over group prefetch once a live element is placed, runs inside the render gate, and is supervised. Update tick 1s while live elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md describes what was built and why SegmentStrip was not needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(sports): live Vegas cards for the scoreboards (shared layer) One live element per game, drawn only when what the card shows changes, so a score changes on a card already crossing the panel. The shared part, so each scoreboard adopts it in a few lines: - src/common/sports_vegas.py: game_key, game_fingerprint (the whole game dict, frozen: no drawn field can be missed), dedupe_games, VegasCardCache, StickyOdds (odds a live poll left out stay drawn), finished_games / with_finished_games (a game that just went final keeps its card, after its league's live games; one a heuristic only judged over keeps its live state, so a tied end of regulation never shows FINAL early). - SportsScrollDisplay.make_vegas_renderer() is the override point; build_vegas_elements() and SportsScrollDisplayManager .get_vegas_elements_for() do the rest. A card's version includes its teams' ranks, which the renderer draws from the rankings cache. - SportsLiveSharedMixin._record_finished_game() / finished_games_snapshot(): held for FINISHED_GAME_TTL after it leaves the live list. A sport that does not implement make_vegas_renderer keeps its ordinary Vegas content, so no scoreboard changes until it opts in. scripts/render_plugin.py --vegas renders a plugin's Vegas block as the ticker lays it out, and --timeline stacks it at successive moments as the ticker would update it in place; the join is now render_pipeline.join_plugin_rows(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): keep live games in the ticker by default display.vegas_scroll.live_in_ticker now defaults to true: through a live game the marquee keeps running and the live scoreboard takes extra turns in it -- its cards updating in place while they scroll -- instead of the ticker giving way to the full-screen scoreboard. The new default would reach nobody on its own: every existing config holds an explicit false copied from the template (there was no control for it), and the template merge only adds missing keys. ConfigManager therefore turns a stored false on once, with a backup, and records live_in_ticker_migrated so a false chosen afterwards stays. The marker is never in the template. A "Keep live games in the ticker" checkbox under Vegas mode sets it. Tests that pin the full-screen takeover now say live_in_ticker=false. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(dev): preview a plugin's Vegas strip in the dev server The dev server's View selector gains "Vegas strip (live elements)" and "Vegas strip (plain Vegas content)": the plugin's block of the Vegas ticker, laid out by the ticker's own code (render_vegas_strip, as render_plugin.py --vegas uses), with its live elements listed. /api/render takes "vegas": "live" | "plain"; the display view is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): extend and trim the Vegas strip in place Every strip extension rebuilt the whole strip (np.concatenate: 2-2.6 ms for a 10-14k px strip at 512x64 on a Pi 4) and every trim copied what was left (1.2-1.8 ms), on the render thread. With ~4 ms of slack per refresh, every extension frame on hdpi missed its refresh (5/5 in each soak run). The strip now lives in a buffer with spare room; cached_array is a view of its live columns. An append writes only the new columns (~0.2 ms), a trim only moves the view's start, and the one full copy happens when the buffer is reallocated (STRIP_SPARE_FACTOR 3: about once every two strip-lengths scrolled). A cached_array set from outside -- the multi-display follower's read-only one, create_scrolling_image's -- is never written through, and a new strip lets the old buffer go. last_copy_bytes says what was copied, and the Vegas frame-timing attribution reports that instead of the whole strip. test_scroll_helper_in_place.py: the buffer is reused and only new columns copied, trims copy nothing, reallocation when the room runs out, outside arrays untouched, and random appends/trims/patches/scrolling checked frame by frame against the old copying strip (mutation-checked). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(sports): a default _determine_game_type on SportsScrollDisplay render_vegas_card looked the method up with getattr and a None default, which static analysis (Codacy) reports as calling something that may not be callable. The base class now has the default -- the card type from the game's state -- and the plugins that define their own override it as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(scroll): no assert in _extended_strip An assert vanishes under python -O (Codacy); a real check says the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: review follow-ups on the shared live-card layer - The reused Vegas renderer always gets the current rankings, empty included, so ranks cleared since are not kept drawn. - render_plugin.py: --timeline refuses --no-live (a timeline shows live elements changing), --timeline/--no-live need --vegas, and the Vegas paths create the output's directory like the display path does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(vegas): lay a group's blocks out before the extension, off the render thread #701 cut the strip copy, but the frame after an extension was still late: the render thread also joined each plugin's rows (separation_gap measures every pair) and pasted the blocks into an addition image, ~37 ms on hdpi (Pi 4, 512x64) against ~3.75 ms of slack. The thread that fetched the group now does that as each member arrives: RenderPipeline.prepare_group_member joins the rows and turns the block into pixels (the prefetch thread and the live worker, under the render gate). extend_scroll_content takes those blocks, and ScrollHelper.append_content writes items -- images or RGB arrays -- straight into the strip's spare room, blanking only the gaps. A member that was not prepared (an inline fetch) is joined at the extension as before; the strip is identical either way. On hdpi the extension's render-thread work goes from 37.5 ms to 3.2 ms p50 in place (6.9 ms when the buffer is reallocated). The helper's per-append INFO line, a duplicate of the pipeline's, is now DEBUG. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
f4bda50710 |
feat(vegas): live elements update in place while they scroll (#697)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements update in place while they scroll One background worker (src/vegas_mode/live_worker.py) redraws a plugin's live elements when its data epoch moves on (update listener) or on their refresh_hz, nearest the screen first, and hands changed pixels lock-free to the render thread, which copies them into the strip between frames (RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most four patches or two screens of bytes a frame, no drawing or locks there. The worker takes over group prefetch once a live element is placed, runs inside the render gate, and is supervised. Update tick 1s while live elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md describes what was built and why SegmentStrip was not needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
596809acc3 |
perf(scroll): build the strip's PIL image only when something reads it (#695)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
7eb7a58d0c |
fix: web UI and src.common bugs (wifi wrong-password, plugin icon, starlark toggle, API caching, scroll/logo/font helpers) (#646)
- wifi: keep the "wrong_password:" prefix through the restore/AP fallback so the UI's incorrect-password prompt fires again. - /plugins/installed returns the manifest's icon (string only). - /starlark/apps/<id>/toggle coerces `enabled` and delegates to _toggle_starlark_app (disk before memory, no KeyError, "false" is false). - /api/v3/ JSON GETs are sent Cache-Control: no-store; non-JSON keeps 5s. - ScrollHelper.set_scrolling_image converts non-RGB input (alpha onto black); create/set_scrolling_image reset last_update_time like reset_scroll. - LogoHelper backs off a failed download per path for MISSING_LOGO_RECHECK_SECONDS; cleared on invalidate/clear_cache. - refresh_placeholder_timestamp saves atomically. - FontManager.clear_cache / _clear_plugin_font_cache bump cache_generation. - Odds manager: per-game logs to DEBUG; JSON decode error caught before RequestException (same cooldown). - element_style mangled continuations; startup validator skips null plugin blocks and reuses the controller's discovery. - src/common/README lists frame_timing, json_body, render_gate. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
b9416ef803 |
fix(display): Vegas teardown and 240s default; remove dead Vegas buffer code (#637)
* fix(display): tear down Vegas mode on controller cleanup DisplayController.cleanup() never called VegasModeCoordinator.cleanup(), so the Vegas teardown (stop, pipeline/stream reset, adapter cache drop) was unreachable. Call it before the display manager is cleaned up, and skip it when Vegas was never created. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(vegas): default max_cycle_duration to the documented 240s The template, the web UI help, CONFIG_REFERENCE and the controller all say 240, but the code defaulted to 600 in two places, so a config without the key ran Vegas iterations 2.5x longer than documented. from_config now falls back to the dataclass field defaults instead of repeating each one, so the two copies can no longer drift, and the controller's follower scroll-speed default reads VegasModeConfig's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(display): let run.py -d show display_manager's DEBUG output display_manager pinned its logger to INFO at import, overriding the root level, so debug mode never showed its DEBUG lines. Use get_logger() from src.logging_config like the rest of the core and leave the level to the logging setup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(display): run each startup validation check once StartupValidator.validate_all() ran twice at boot, before and after the plugin manager was created, so every config, cache, display and systemd-unit warning was logged twice. The second pass now runs only the plugin checks. Drop the commented-out raise_on_errors line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(vegas): one INFO line per plugin-list refresh StreamManager logged "=" * 60 banners and a line per plugin (INCLUDED, SKIPPED, FETCHING CONTENT, SEGMENT CREATED) at INFO on every refresh and fetch, i.e. at each cycle start and every 30s. Log one INFO summary of the rotation per refresh and move the per-plugin detail, the weighting breakdown and "no content this cycle" to DEBUG (the adapter still warns when every content path fails). Also drop the check/cross marks from the controller's log messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(vegas): drop the per-iteration static-mode plugin scan run_iteration() rebuilt _static_mode_plugins on every iteration, asking every plugin for its display mode and logging the set at INFO, but nothing ever read it: static pauses are triggered by _check_static_plugin_trigger() from the next segment. Delete it, the coordinator's get_ordered_plugins() that only it used, and the write-only _static_pause_plugin / _static_pause_start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(vegas): remove the staging buffer that was never filled StreamManager and RenderPipeline carried a double-buffer design that nothing used: _staging_buffer was only ever cleared or swapped, so swap_buffers() never did anything and should_recompose()'s staging_count > 0 branch was dead, and _active_scroll_image, _staging_scroll_image, _is_rendering, _last_frame_time and _frame_interval were written but never read. Delete the machinery and rewrite the docstrings around what actually carries updates: _pending_updates, consumed by process_updates() in swap mode and invalidate_pending_updates() in continuous mode. should_recompose() no longer builds a buffer-status dict every frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(display): tidy the display controller without changing behaviour - Import VegasModeCoordinator locally instead of through module globals (there is no circular import to avoid). - Drop hasattr() checks on attributes PluginManager.__init__ always sets (plugin_executor, plugin_last_update, get_plugin_lock, run_scheduled_updates*, stop_update_worker) and the dead "older manager" fallbacks; keep the health_tracker None checks, now via _health_tracker(). - Extract _display_once() for the per-frame display call both render loops copied, _advance_on_demand() for the two on-demand rotations, _reset_on_demand_fields() for the error and clear paths, and _timezone() / _in_window() for the two schedule checks. - Remove always-true conditions and the unreachable non-plugin else branch in run(), and read _was_display_active / _last_published_mode / vegas_coordinator directly now that __init__ declares them. - Declare the follower render state in __init__, name its tuning constants, add _follower_sign(), and share the 90/s sync send interval with the render pipeline (SYNC_SEND_INTERVAL). - Delete history narration and the "Opt #N" labels, fix the comment that called _scroll_speed constant (hot reload updates it), and drop a startup timing log that measured nothing. - render_pipeline / plugin_adapter: read display_manager.width/height as the properties they are, drop an empty TYPE_CHECKING block, an aliased threading import and a duplicated `if result and self.sync_manager:`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(display): trim dead code from display_manager - Add _new_canvas() for the image/draw/fontmode="1" setup that was copied six times. - Call resolve_double_sided() and compose_pixel_mapper_config() directly instead of through a module alias and a passthrough method, and replace the comment that said the passthrough read class attributes. - Delete the unused _initialized flag and _ORIENTATION_ROTATE_DEGREES alias (no core or monorepo reader; tests stop resetting the flag), the test pattern's unreachable no-matrix branch (it only runs once the matrix exists), `del old_image # help GC` (a no-op on a local), a duplicated early return in process_deferred_updates, and stale comments. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(vegas): remove unread fields and test-only helpers, fix docstrings - ContentSegment: drop total_width, fetched_at, is_stale, image_count and is_static, none of which is read. - StreamManager: drop _current_index (never advanced) and the test-only get_all_content_for_composition() and has_pending_updates(); VegasModeConfig: drop the test-only is_plugin_included(). - geometry.find_blank_cut() has had no production caller since the crop moved to item boundaries; delete it and its tests. - PluginAdapter: the _finalize docstring described separator_width between every image, and _crop_to_budget's said cuts snap to the nearest blank column; both now describe what the code does. - Coordinator: the static-pause interrupt log no longer blames follower mode for every interrupt, and set_update_callback names the callback the controller actually wires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(scroll): correct ScrollHelper comments and drop dead branches - Four comments said the strip always starts with display_width of blank; it does only when lead_gap is None (Vegas passes its own). - Delete the "Width calculation mismatch" warning: the image is created at the calculated width, so the two can never differ. - Remove the two scroll_delay <= 0 fallbacks (which disagreed with each other): set_scroll_delay clamps it to at least 0.001 and nothing in core or the plugin monorepo assigns it directly. - Trim the scipy history from the blend docstring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(run): drop a redundant comment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(display): log set_scrolling_state only when it changes Vegas and scrolling plugins set the scrolling state every frame, so once display_manager's DEBUG output became visible in debug mode it printed "Scrolling state set to: True" about 120 times a second. Log only when the value differs from the previous one; the state, activity timestamp and frame hold still update on every call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): display-vegas Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5
|
116abb0daa |
fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595)
* fix(sports): share the ESPN rejected-range memo with the background service BackgroundDataService always sent a season range first and, on a 400, fell back to chunks without recording the rejection, so every background season fetch spent a doomed request and live scoreboards learned nothing from it (or it from them). The worker now consults and sets the same 6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight to month/day chunks, and if every chunk fails the range is asked once for a real error without re-spending the chunks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): keep plugin asset and action routes inside their directories POST /plugins/assets/upload, GET /plugins/assets/list and POST /plugins/assets/delete joined the request's plugin_id onto assets/plugins unchecked, so '../../config' created, wrote, listed and deleted outside it. #561 guarded only the route that serves the files. All three now go through path_safety.resolve_under and answer 400 for anything but a plain name, and delete only unlinks a metadata path that resolves into that plugin's uploads directory. PluginManager.get_plugin_directory refuses ids that are not one plain path segment, so /plugins/action (which runs a manifest script from the returned directory) and every other caller get the guard; the action route also rejects such ids up front, covering its no-manager fallback. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): report a no-op plugin update as already up to date update_plugin() returns True both for a real update and for "nothing to do" (a ZIP-installed monorepo plugin already at the registry version, a bundled plugin). With no git commit to compare, POST /plugins/update called every such success "updated successfully", so Check & Update All counted most official plugins as updated on every run. The route now reads what changed off the plugin itself (commit, else manifest version, else last_updated) and returns data.update_status (updated / up_to_date / local_only). The update-all toast is summarised by PluginInstallManager.summarizeUpdateResults from that status, falling back to the message for older servers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(sports): scoreboard scroll speed no longer follows target_fps sports_scroll computed the crisp speed ladder against the global target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since frame-locked presentation (#545) the helper steps a fixed number of whole pixels per presented frame and the panel presents at its real refresh, so the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a 50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s. The ladder now uses the display manager's refresh_hz, then display.hardware.limit_refresh_rate_hz, then the default. target_fps is not consulted. Docstrings now say scroll_delay is ignored for pacing (no behaviour change there) and describe the fixed-step model. Tests: replace the tests that pinned target_fps as the ladder refresh and described time-based stepping; assert speed independence from target_fps (unit and end-to-end presented px/s against the real helper), that the fixed per-frame step is applied, and that scroll_delay does not change speed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): escape registry and upload values in plugin manager inline handlers The store, saved-repository and custom-registry buttons built onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone, so a custom registry entry whose id contained ' closed the attribute and added its own handler. One helper, jsStringAttr(), now HTML-escapes the JSON literal for every one of those handlers, and the store View button opens only http(s) repo links. The live window.updateImageList (plugins_manager.js loads last, so its copy wins over the file-upload widget's) wrote the uploaded file's original name, path and ids into markup raw; they are escaped now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note plugin asset, action and inline handler guards Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): let the root pip wrapper install web_interface/requirements.txt Update Code, the automatic update's health check and Install Base Requirements install web_interface/requirements.txt through safe_pip_install.sh, which only allowed the root requirements.txt. The first commit changing that file would fail its dependency install, and the automatic updater rolls back any update whose dependencies did not install -- on every device, for every newer commit. The wrapper now lists both core requirement files. Only their folders are resolved, so a requirements.txt symlinked out of the project is compared by its target and refused (previously the root file's own symlink target was what got allowed). The updater's file list is a named constant, and a test runs the real wrapper (pip stubbed) on every file Update Code and the rollback install. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): do not retry plugin requests that got an HTTP answer PluginAPI.request wrapped everything that was not a structured error as NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a JSON error without error_code included. Check & Update All retries NETWORK_ERROR, so those updates were re-sent five more times with backoff, contrary to the #587 contract that an HTTP error response is the server's answer. NETWORK_ERROR now means only that fetch() rejected. Any HTTP response without an error_code, or with a body that is not JSON, is API_ERROR with the HTTP status attached. Tested against the shipped api_client.js. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scroll): restart the stats window when an idle gap is dropped by size #582 dropped an idle gap from the frame stats two ways: the reset_scroll() sentinel, which also restarts the 5s window timer, and a size guard for scrollers that never call reset_scroll(), which did not. On that path the first real frame after the gap found the boundary overdue and logged a stats line for a one-frame window. Both paths now share one seeding helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): leave plugins alone when update_core's own rollback fails update_core returns rollback_failed directly when a partial pull or an update whose health check never started cannot be rolled back. run() only held plugins back for 'verifying', so those devices still got new plugin versions and a display restart on top of a core in an unknown state -- the opposite of what the health-check path does, and of the 3.4.0 changelog (plugins are left alone if the rollback fails). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(api): make the REST reference match the api_v3 package Every documented request body, query parameter and response shape was re-checked against the handlers in web_interface/blueprints/api_v3/. Fixes calls that failed as documented (repo_url, action_id/params, files/image_id, font_file+font_family, ?font=, cache key, auto_enable_ap_mode, plugin limit keys), removes the font-override endpoints dropped in #566, corrects response shapes (plugins/config, plugins/schema, health, metrics, operation history, github-status, fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds the 26 routes it omitted (backup, system auto-update/git, wifi radio, starlark editor, MQTT bridge, status endpoints, skins). Documents the merge semantics of partial JSON saves to /config/main and /plugins/config and the dim-schedule POST accepting GET's days shape, which land in the same change set. Replaces app.py line numbers and the removed api_v3.py path with file and function names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): remove the General-tab plugin system toggles that did nothing plugin_system.auto_discover, auto_load_enabled and development_mode had General-tab toggles whose help tips promised dormant plugins and verbose logging, but nothing reads them: every enabled plugin is discovered and loaded regardless. Remove the three toggles. The keys stay tolerated in stored configs. The save handler now stores a flag only when a client sends it; treating a missing key as an unchecked box would otherwise rewrite all three to false on every General-tab save, which still posts plugins_directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(scroll): remove dead code left by #523/#570 - Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read them since the numpy blend replaced the scipy path. - Drop ScrollHelper._last_integer_position and frame_time_target, which were written but never read. - Keep target_fps and set_target_fps() but document them as informational: nothing paces off them, yet ledmatrix-elections' test_scroll_pacing.py reads helper.target_fps back and third-party plugins may call the setter. - Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to throttle", set_sub_pixel_scrolling's "default: True", and set_frame_based_scrolling's claim that it steps. The plugins monorepo was grepped for every removed name; none is used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI FONT_MANAGER.md told plugins to read display_manager.font_manager, which does not exist, so a plugin following it failed to load with AttributeError. The shared FontManager lives on the PluginManager and BasePlugin._get_font_manager() returns it (with a fallback for harnesses). Also removes the Fonts-tab override workflow and element-override panels that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls /plugins/store/search does not exist (404) and the list endpoint reads query, not q. The registry guide's curl examples omitted the JSON Content-Type, so the handlers saw an empty body and answered 400. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(config): use the shared core-key list in the last three private copies StartupValidator warned "Plugin 'auto_update' is enabled but not found" on every display start with auto-update or a dim schedule on; the reserved plugin-id check missed auto_update, sync, location and the rest; and ConfigManager's (uncalled) orphan cleanup would have deleted display, schedule and auto_update. All three now read src/core_config_keys.py, which also gains CORE_SECRETS_KEYS for the github/youtube secrets sections. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): partial JSON saves to /config/main change only what they send A JSON body with one field reset every checkbox in the sections it touched: the MQTT bridge's brightness slider turned off disable_hardware_pulsing, inverse_colors, show_refresh_rate and use_short_date_format, and a timezone-only save turned off web-UI autostart and weekly auto-updates. Missing-means-unchecked now applies only to form posts: form-encoded bodies and the v3 forms, which mark themselves with a hidden __form_section input. Also on the config routes: - vegas_min/max_cycle_duration no longer match the generic *_duration rule, so they stop landing in display_durations and a blank one no longer rejects the whole Display save; - saving from the Raw JSON editor calls start_setup_if_needed like the General form, so enabling auto-update there finishes its setup; - the schedule and dim-schedule POSTs accept the per-day days.<day> shape their GETs return, as well as the flat form keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): install plugin dependencies from the configured plugins directory install_plugin_dependencies.sh scanned only plugins/, but the Plugin Store installs into plugin_system.plugins_directory (default plugin-repos), so the documented "Recommended" fix found 0 plugins on every store install. It now reads plugins_directory from config/config.json (relative to the project root or absolute, default plugin-repos) and also scans plugins/ for dev symlinks, installing a plugin reached through both only once. With set -e alone, `pip ... | tee` took tee's exit status, so a failed pip install was reported as success; set -o pipefail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: replace stale API names, line numbers and the api_v3.py path - ADVANCED_FEATURES: StreamManager methods that exist (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the real on-demand status envelope ({status, data: {state, service}}) - app.py:199 / :144 / :607-619 line citations and web_interface/blueprints/api_v3.py (now a package) replaced with file and function names in ADVANCED_FEATURES, CONFIG_DEBUGGING, PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE, PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README - CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use /config/raw/main to replace the file; describe where validation runs - TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints usage) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): verify the web interface that actually ships, on port 5000 verify_installation.sh failed every healthy install: it required the long-removed web_interface_v2.py and looked for a listener on port 5001, while the web interface binds 5000 (web_interface/start.py). It now checks the files ledmatrix-web.service runs (start_web_conditionally.py, web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the same 5001 port in its listen check, HTTP probe and printed URLs. Port matches are anchored so :50001 no longer counts as :5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(plugins): one display-size contract: display_manager.width/height CLAUDE.md (#580) says to read display_manager.width/height because matrix is None when hardware init fails; the development guide, the safety-harness doc and two DisplayManager docstrings still recommended matrix.width/height. The bundled starlark-apps plugin read matrix.width unguarded, so its magnify recommendation and frame scaling raised in fallback mode (e.g. after the Pi 5 hardware refusal). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(install): make install_service.sh --help print usage instead of installing install_service.sh parsed no arguments, so `sudo ./scripts/install/ install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md) rewrote ledmatrix.service, ledmatrix-web.service and both update-verify units and enabled/started them. It now handles -h/--help (usage, exit 0, no changes) and rejects any other argument with exit 2 before doing anything. Running it with no arguments, as first_time_install.sh does, is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scroll): describe the fixed-step model and document frame_hold Since #545 a crisp speed from scroll_config.configure() makes the helper advance a fixed whole-pixel step per presented frame with no clock, and the display manager's frame hold is part of the speed. The docs still described the removed wall-clock model: - scroll_config's module and configure() docstrings said speed is applied in time-based mode and that omitting the hold "falls back to fractional pixels"; omitting it actually runs the scroll frame_hold times too fast. - SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both modes, and read a 20 ms stats median as missed refreshes although that is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step, the hold-dependent healthy median, that target_fps plays no part, and that a hand-added scroll_pixels_per_second loses to a schema-default pair. - PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling) without frame_hold; it now documents the parameter (core 3.4.0) with a configure() + set_scrolling_state example. - update_scroll_position/set_scroll_speed and set_scrolling_state docstrings say the same. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark target_fps legacy; describe what Vegas scroll_delay does - General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after the sports_scroll fix nothing in core scrolling reads it. The field and its API validation stay so saved configs and plugins that read global_config['target_fps'] keep working. CONFIG_REFERENCE says the same. - Vegas frame_based_scrolling/scroll_delay were described as frame-count stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and still advances by elapsed time, so the applied speed is clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The config comments, render_pipeline comment and CONFIG_REFERENCE rows now say so. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(deps): describe how plugin dependencies are really installed The guides said the web service runs as root, that installs pick --user from os.geteuid(), and quoted a warning and a PluginManager._install_plugin_dependencies() method that don't exist. The web unit runs as the installing user; store installs go through install_requirements_file() and sudo safe_pip_install.sh (root), with a user-level fallback that says so, and load-time installs run in the display service's own (root) interpreter. Manual paths now use the configured plugins directory (plugin-repos/ by default) instead of plugins/, which store installs no longer use, and install_plugin_dependencies.sh is described as scanning that directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): count local changes one way for the preflight and the pull The automatic update's preflight ignored mode-only changes and anything whose status line contained plugins/ or plugin-repos/, then promised "Automatic updates will not stash your changes". perform_core_update used plain git status (modes count) and ignored only 'plugins/', then ran 'git stash push -- :!plugins', which nothing ever pops. So an edit to a bundled plugin under plugin-repos/, or the installer's chmods on tracked scripts, passed the preflight and was stashed away for good. - auto_update.local_changes() is the one predicate both use: core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/ excluded by leading folder rather than substring (a core file under web_interface/static/v3/js/plugins/ now counts). - Update Code's explicit stash leaves out both plugin folders; the pull's --autostash carries their edits and mode changes across and reapplies them. - The automatic updater calls perform_core_update(stash_local_changes= False), which refuses instead of stashing edits that appeared after the preflight; update_core reports that as 'blocked'. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): diagnostics follow the web autostart default and api_v3 package #556 made a missing web_display_autostart mean "start" (only an explicit false/off keeps the web interface down), but the diagnostics still said otherwise: diagnose_web_ui.sh reported a missing key as "defaults to false", diagnose_web_interface.sh said the web interface "will not start unless this is set to true" and recommended enabling it, and debug_web_manual.py printed False. Troubleshooting a down web UI pointed users at a non-cause. Both shell scripts now evaluate the setting with the launcher's own autostart_enabled() (inline fallback if it cannot be imported) and report on / off / not set (on) / unparseable config; debug_web_manual.py uses the same function. They also check web_interface/blueprints/api_v3/ __init__.py: api_v3.py became a package in #553, so every healthy checkout was reported as missing a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(install): what install_service.sh installs; verify script port; no sudo for --help install_service.sh installs and starts ledmatrix, ledmatrix-web and the update-verify units, not only ledmatrix.service (systemd/README.md, README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help' as a harmless check; it now shows --help without sudo and warns what a real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh checks the web interface on port 5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note update-all, plugin system settings and script fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): size the preview after orientation and pixel mappers display_geometry.physical_size claimed to give DisplayManager's answer but only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured after the library's pixel mappers, so a Rotate:90 / orientation 90 chain previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for 128x64. Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown names leave it alone), and move the orientation composition here so DisplayManager and the preview share it. The module docstring no longer claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH. Audit finding F18. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): refuse settings the rgbmatrix library aborts on, on every board The library answers several settings with a NULL matrix or abort() rather than an error, so the display service crash-looped (Restart=on-failure) instead of reaching fallback mode: rows above 64, chain_length above 255 (uint8_t binding setter, documented as "no upper limit"), a misspelled hardware_mapping, and parallel 2-3 on a single-output mapping, reachable from the Display form on the default adafruit-hat(-pwm) mapping. #586 only guarded the Pi 5 subset. - src/matrix_support.py holds the rules for every board (Options::Validate ranges, binding integer types, mapping names and outputs from lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the API's numeric ranges. - DisplayManager checks them before building options and raises MatrixSettingsRefused, so a hand-edited config falls back with a logged, reported reason. Emulator mode only warns. - The config API refuses them with a 400 naming the setting; combinations are checked against stored values but reported only when the request sets a field involved. - The hardware status file gains "cause" (settings/library/forced). The fallback log and Display banner give the Pi 5 rebuild hint only for a library failure instead of rebuild + gpio_slowdown advice for every failure; one Pi 5 slowdown recommendation (1-3, start at 1). - The Display form offers classic/classic-pi1 and orientation 90/270 and renders any other stored mapping selected with a warning, so an unrelated save no longer rewrites them; the API accepts 90/270. Audit findings F03, F16, F19, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(display): library limits, template defaults and Pi 5 slowdown - rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs, classic/classic-pi1 mappings and orientation 90/270 documented. - Defaults are the config.template.json values: config migration adds missing keys from the template, so the listed "code defaults" never applied. - One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode, starting at 1. - Troubleshooting describes the refused-settings fallback, and CHANGELOG corrects the Unreleased "no upper limit" entry. Audit findings F19, F20, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py opens the panel with the service's options --measure and --demo built RGBMatrixOptions from a private copy of the display service's builder that had drifted: gpio_slowdown came from display.hardware (default 2) instead of display.runtime (default 3), and rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors, pixel_mapper_config and orientation were skipped, with different defaults (hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown was measured -- or garbled -- in a setup the service never drives. The option filling in DisplayManager._setup_matrix moves, unchanged, into DisplayManager.apply_matrix_options(options, config), which _setup_matrix calls and the script reuses (overriding only limit_refresh_rate_hz for --measure). The script now loads the whole config rather than the hardware block. Tests pin the script's options to the service's attribute for attribute. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py recommends keys the resolver honours The ladder ended by telling users to set display_options.scroll_pixels_per_second. scroll_config ranks that key below the scroll_speed + scroll_delay pair, deliberately, and several plugin schemas default the pair into config, so the advised key was silently ignored (a schema-default 1/0.02 pair plus an advised 66 still resolved to 50 px/s). The advice is now the pair that selects the crisp speed exactly (pixels_per_frame every frame_hold/refresh seconds), explains that the pair outranks scroll_pixels_per_second, and gives the scoreboards' per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the printed pair over a schema-default pair and check it lands on the advertised speed and hold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula - SPORTS_UNIFICATION.md still presented honouring global target_fps as sports_scroll's added behaviour and its one user-visible gain; note that it was withdrawn because it had become a speed multiplier. - ADVANCED_FEATURES.md gave Vegas scrolling as (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s by elapsed time, through a 0.1-5 px per scroll_delay clamp when frame_based_scrolling is on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): scroll model fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dev): link-github links plugins from the ledmatrix-plugins monorepo link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git, and those per-plugin repositories no longer exist: official plugins are directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the monorepo once into the dev directory, finds plugins/<name>, plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and links it under its manifest id. With an explicit repo URL it still links a single-repository plugin as before. dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a fork), plus plugins_repo and plugins_branch; github_pattern, which was documented but never read, is dropped and warned about. Ships dev_plugins.json.example and git-ignores dev_plugins.json, both of which the guide promised. Reading JSON falls back to python3 when jq is missing (get_plugin_id silently returned nothing without jq). update/status/list find the git checkout above a monorepo plugin directory (its .git is not in the plugin dir), and update pulls a shared checkout once. status no longer exits 1 when nothing is broken. Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration, workflow, store integration, hello-world link, submission), and the nonexistent scripts/git-hooks/pre-push-plugin-version and scripts/bump_plugin_version.py replaced with the real rule: bump the manifest version and run update_registry.py. scripts/dev/README.md and CLAUDE.md updated to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scripts): monorepo workspace layout; fix_perms and install READMEs MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin; setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the workspace file opens LEDMatrix plus ../ledmatrix-plugins. scripts/fix_perms/README.md listed cache directories fix_cache_permissions.sh never touches and a 'ledmatrix' service user that doesn't exist (also in scripts/install/README.md); adds safe_pip_install.sh. install/README: install_service.sh installs the web and update-verify units too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): keep the rollback's pip retries inside the unit time limit The health check reinstalled the previous requirements by trying the next bash path after any failure, including a 600 s pip timeout. Two files, two paths: up to 40 minutes of pip alone, while systemd stops ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the rollback half-way and leaving the update 'verifying' until the web UI calls it lost. - Like permission_utils.install_requirements_file, only a sudo refusal moves on to the next bash; a pip that ran and failed or timed out is not repeated. The refusal wording is one list (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only verifier and pinned equal by a test. - All reinstalls in one rollback share a 600 s budget. - WORST_CASE_SECONDS adds up every timeout on the longest path (27.5 min); a test holds it under the unit's TimeoutStartSec and that under the web UI's VERIFY_LOST_SECONDS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools Plugin config was prepared differently depending on how it arrived: - JSON POST /plugins/config built a partial body on schema defaults, so {"enabled": true} reset every other setting of the plugin. It now merges onto the stored section first, as the form path already did. - Legacy-boolean normalization (#588) ran only at load: GET /plugins/config returned the raw boolean, posting it back failed validation, and hot reload handed plugins the raw section (a legacy dynamic_duration: true came back as a boolean). schema_manager.prepare_plugin_config (normalize, then defaults) is now used by PluginManager.load_plugin, both save paths, GET, the save notifications and DisplayController's hot-reload callback. - The JSON save's filter kept only enabled/display_duration/live_priority and dropped a submitted skin, skin_options or vegas_* tuning key. There is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES, used by validation and by the save filter; PluginManager's CORE_OWNED_CONFIG_KEYS is its vegas subset. - Plugin sections posted to /config/main were stored verbatim, including values /plugins/config rejects. They now go through the same preparation (_prepare_plugin_config_for_save, extracted from save_plugin_config), and a failing section rejects the whole save before anything is written. - dev_server read only top-level defaults and let a schema enabled:false win; build_full_config shallow-merged overrides, dropping sibling defaults; the harness extracted defaults differently from the device. loading.build_config now uses the device's extraction and preparation, and dev_server, check_plugin, render_plugin and the harness all use it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt-bridge): brightness changes apply live and touch nothing else The display service's hot reload applies a saved brightness within a few seconds, and /config/main no longer resets other display settings on a brightness-only JSON body. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): automatic update hardening Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI It described web_interface_v2.py and index_v2.html (both gone), client-side form generation, one POST per field with {key, value}, and 'no nested objects'. The v3 UI renders plugin forms server-side from the schema (pages_v3 partial + plugin_config.html macros, nested sections and x-widgets), posts the whole form once, and save_plugin_config() merges onto the stored section, validates, splits x-secret fields and notifies the plugin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt): brightness saves apply via hot reload and leave other settings alone The bridge README said brightness is applied on the display's next restart; the display controller's config hot reload applies it within seconds. It also now states that the bridge's partial JSON save changes only brightness (the /config/main merge fix in this change set). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): don't log pip's output from the health check's reinstall pip can echo a private index URL with embedded credentials; permission_utils redacts it, the stdlib-only verifier cannot, so it logs the exit code only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark the plugin_system toggles as unused legacy keys auto_discover, auto_load_enabled and development_mode are read by nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and the REST reference listed them as live settings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): docs and developer tools group Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): legacy plugin-system toggles no longer count as a General save auto_discover, auto_load_enabled and development_mode have left the General form, so a post carrying only one of them is not a general-settings save and must not treat web_display_autostart and auto_update as unchecked. The plugin_system block itself is left as on main for the branch that reworks it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): config-save and plugin-config preparation fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(claude): re-check matrix_support.py rules when the library submodule is bumped Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address Codacy findings on the core audit PR - plugin_manager.prepare_plugin_config: when the fallback legacy-boolean pass also fails, log a warning instead of a bare except/pass. - api_client.js: request() refuses any endpoint that is not a plain path under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace, control characters) with INVALID_ENDPOINT before calling fetch(), and plugin ids are URL-encoded wherever they are put into a URL (also in the app-shell batch load). - test_update_all.js: pins both against the shipped client. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): check endpoint control characters without a control-character regex Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags the \x00-\x1f range in checkEndpoint's regex. Test the char codes instead; the endpoints refused are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(auto-update): make the seed script executable on disk, not only in the index On Linux Repo.publish() commits with -a, which recorded scripts/run.sh as 100644 upstream because the seed file was never chmod +x. The pull then brought in the same mode the installer chmod had made locally, so installer_chmod saw no mode change left to check. The updater was fine: with the upstream commit at 100755 the --autostash carries the device's chmod across. Verified under Linux (WSL, git 2.43): the old helper fails exactly as CI did, the fixed one passes all 63 tests in the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5
|
d01da3bd9f |
fix(scroll): stop timing the idle gap between scrolls as a frame (#582)
ScrollHelper.last_frame_time was set once in __init__ and thereafter only
at the end of log_frame_rate(). Nothing re-armed it when a scroll began, so
the first frame of every scroll was timed against the last frame of the
*previous* one and the whole idle period between them was recorded as a
single frame.
Measured over 3 hours on a 256x64 Pi 4, that produced 31 windows reading
Scroll frame stats - 0.0 fps over 1 frames | median 136776.02ms
p95 136776.02ms max 136776.02ms min 136776.02ms | stalls 0 (0.0%)
and -- worse, because it is not obviously wrong -- put the same gap in the
max field of otherwise healthy windows, where the worst values were 537s
and 604s. It also counted as one stall per scroll start: at ~500 frames to
a window that is ~0.2%, against measured stall rates of 0.07-0.16%. The
stall rate is the number used to judge whether a scroll change worked, and
it was the same order of magnitude as its own artefact.
The first frame of a scroll has no predecessor, so it has no frame time.
last_frame_time is now None until one is rendered, and reset_scroll() puts
it back -- the same treatment last_update_time already gets three lines
above, for the same reason. reset_scroll() alone is not enough, because the
scrollers actually emitting these lines never call it, so a sample at or
past the 5s log interval is dropped as well: nothing that renders a scroll
takes that long over one frame. Seeding also restarts the window timer, or
the boundary is already overdue when the second frame arrives and every
scroll opens by reporting a window of exactly one frame. A window whose
samples were all dropped now logs nothing rather than reporting the gap.
docs/SCROLL_PERFORMANCE.md documented the diagnostic in terms of a
"Frame time: N ms" line that
|
||
|
ChuckandClaude Opus 5
|
d51f7ada14 |
chore(scroll): drop the dead sub-pixel path, and two dev-tooling papercuts (#570)
Three independent changes, none of which alter runtime behaviour. 1. Remove ScrollHelper._get_visible_portion_subpixel and _interpolate_subpixel (162 lines). get_visible_portion dispatches only to _blend_visible_portion, so _get_visible_portion_subpixel had no caller, and _interpolate_subpixel was reachable only from inside it -- a closed island. _blend_visible_portion's own docstring already records that the scipy path it replaced was dead; the replacement landed but the corpse stayed. 2. scripts/check_plugin.py: also search ../ledmatrix-plugins/plugins. The scoreboards live in the sibling checkout, so --all silently skipped every one of them and only --plugin-dir reached them. 3. .gitignore: ignore config/.config_secrets.json.tmp.*, which the suite leaves behind several of per run. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5
|
a29c84208e |
fix(scroll): advance whole pixels per frame, not per wall-clock second (#545)
* fix(scroll): advance whole pixels per frame, not per wall-clock second Smooth motion is not a frame-rate property, and measuring it as one is why this survived three rounds of fixes. odds-ticker's frame timing is excellent -- 100.0 fps, 10.00ms median, 0% stalls, worst in-scroll frame 19.95ms -- and it still visibly stuttered. What the eye judges is whether the strip advances the same number of whole pixels on every presented frame. update_scroll_position derived position from scroll_speed * delta_time and get_visible_portion truncated it with int(), so jitter in delta_time decided which side of a pixel boundary the position landed on. The live windows show why that matters: a rock-steady 100.0 fps whose individual frames still range 5.6ms to 15.2ms, which at 100 px/s is 0.57px to 1.44px of movement. Run the measured frame times through the real helper and 5.8% of frames advance 0 or 2 pixels instead of 1 -- about six hitches a second. A frame that moves nothing followed by one that jumps two is exactly what micro-stutter looks like. It is worst at a crisp speed, which is the part that stings: at 100 px/s on a 100Hz panel the accumulator sits exactly on integer boundaries, so sub-millisecond jitter flips it either way and the motion beats at around 50Hz. Snapping to the crisp ladder fixes the average and the wall clock then throws away the per-frame uniformity the ladder was bought for. So when scroll_config snaps to a crisp speed it now also puts the helper in fixed-step mode: each presented frame advances exactly pixels_per_frame and no clock is consulted. 100% of frames move by the same amount, whatever the jitter. This is only correct because SwapOnVSync blocks until the panel has taken the frame, which makes the frame count a truer clock than time.time(). Before the swap was locked to vsync it would have run at whatever speed the loop spun at. Related: frame-based mode used to step discretely and was converted to elapsed-time accumulation earlier in this series, because its threshold comparison flipped on jitter. That was right for the code as it stood -- but it treated the symptom, replacing a broken discrete step with a smooth-looking accumulator instead of asking why a wall clock was involved at all. Non-crisp speeds keep pacing off time, and set_scroll_speed() clears the fixed step so a legacy caller changing speed is not silently ignored. Trade-off worth naming: speed is now tied to the presentation rate rather than to real time. If the loop cannot keep up with the panel the scroll runs slow rather than jumping to catch up. That is the better failure -- uniform motion at a slightly wrong speed beats correct average speed with a hitch six times a second -- and a loop that cannot hit the resolved rate is a measurement problem for the crisp ladder, not something to paper over with uneven steps. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(scroll): make the time-based pin actually pin something Review caught that test_time_based_stepping_is_what_it_replaces could pass against perfectly uniform motion, and it was right. update_scroll_position sets last_update_time on its way through, so the very first call sees a delta_time of zero and moves nothing in time-based mode. _advances counted that synthetic frame, which put a guaranteed zero in every histogram -- enough on its own to satisfy "uneven > 0". The test asserting the defect exists would have passed after the defect was gone. The first call is now primed and discarded, and the assertion is a proportion rather than "more than zero": against these frame times the old path misses roughly one frame in twenty, so 1% is well below the real rate and far above anything a stray frame could produce. Re-measured with the artefact removed, the numbers in the PR description are unchanged: 5.85% of frames uneven before (114 zero-advance and 120 double frames in 4000), 0.00% after. Also fills in the docstrings the review flagged: everything in the new test file, plus three pre-existing one-liners in scroll_config that the diff touched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5
|
d12323e7f1 |
perf(scroll): pace frames to the panel — 44→100 fps, stalls 14% → 0.02% (#523)
* perf(scroll): pace frames to the panel, not to a fixed sleep Scrolling ran at 44-46 fps on a 2x128x64 chain and 14-17% of frames took 41-53ms, which reads as judder. Four independent causes, each measured on the hardware; details and the diagnostic recipe are in docs/SCROLL_PERFORMANCE.md. The high-FPS loop slept a flat 8ms after every render. display() has already blocked on the panel's vsync by then, so that sleep was added to a wait that had happened: ~4ms of render plus 8ms put each iteration at ~12ms against a 10ms refresh grid, so every swap missed a refresh and the loop settled at 50fps while asking for 125 -- with no headroom, so a further 14% of frames slipped again. It now sleeps only the remainder, with a 1ms floor so plugin threads still get the GIL. ScrollHelper stepped position on a wall clock at 1/scroll_delay steps per second. Plugins set scroll_delay to the frame period, so that comparison sat exactly on its own threshold: a frame arriving a hair early moved zero pixels and rendered an identical frame, dirty-tracking skipped the swap, it returned in ~2ms, and the beat repeated. No scroll_delay value tunes that out -- a shorter delay trades stalled frames for periodic double-steps. Both modes now accumulate elapsed time at the same configured speed, so position stays proportional to real time. Sub-pixel blending goes back to off by default. It renders a half-step by mixing two adjacent columns, which on a coarse panel showing pixel-font text alternates crisp and smeared frames and reads as shimmer -- visibly worse than integer stepping on the hardware. Vegas mode still opts in. disk_cache uses orjson when importable, falling back to the stdlib. Encoding a ~1MB record drops from 14.8ms to 5.4ms end-to-end, and that work holds the GIL while a marquee is on screen. display_manager also checksummed the whole framebuffer twice per frame (dirty tracking, then the preview snapshot); the snapshot now takes the checksum the caller already computed. New src/common/scroll_config.py resolves scroll settings in one place. Five ticker plugins each hand-rolled this and disagreed: odds-ticker ranked the deprecated scroll_pixels_per_second above the documented scroll_speed/delay pair, and because that key carries a schema default the documented settings were dead for every user (ChuckBuilds/ledmatrix-plugins#408), while ledmatrix-leaderboard read the same key only as a fallback. The resolver also warns when a speed will not advance a whole number of pixels per refresh, which is the property that actually determines whether a scroll looks smooth. scripts/build_rgbmatrix_nogil.sh rebuilds the rgbmatrix binding so it releases the GIL. Upstream declares SwapOnVSync without nogil, unlike SetPixel/Clear/Fill beside it, so the render thread held the GIL for the whole vsync wait and starved background threads into long uninterruptible bursts. The script patches, builds and self-verifies into a scratch tree; --install backs up the original and rolls back if the service does not come back healthy. Measured after: 100 fps locked, no stalls observed, render thread down from 51% to 19% of one core. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): keep the panel swap locked to vsync while scrolling Dirty tracking skipped SwapOnVSync for byte-identical frames. That is the right call for static content, but SwapOnVSync is also what paces the render loop, so skipping it skips the wait for the panel: a duplicate frame returns in ~8ms instead of ~10ms on a 100Hz panel, advances the strip only 0.8px instead of 1.0px, and so makes the next frame more likely to repeat as well. The effect sustains itself once it starts. Measured over 20 minutes on a 2x128x64 chain, both scrollers configured identically at 100 px/s: leaderboard 10ms x35, 11ms x3 (clean) odds-ticker 10ms x26, 8ms x7, 15ms x5 (~20% duplicates mid-scroll) The duplicates were not end-of-cycle idling -- 38% of fast frames fell within 90s of a scroll completion against 35% of normal frames, a null result. The trigger is per-frame work: odds does more of it, and more variably, so it is first to land a frame that advances less than a whole pixel. Pushing an identical frame costs one canvas copy. Falling out of vsync lock costs smooth motion. Static content is untouched, because is_currently_scrolling() expires on its own inactivity threshold -- covered by test_stale_scrolling_state_stops_forcing_pushes so a plugin that stops scrolling without saying so cannot pin the panel into always-push. Also de-flakes test_snapshot_still_written_on_skip, which asserted a strict mtime increase between two writes that can land in the same filesystem tick; it failed about two runs in three on Windows regardless of the code under test. The file is now backdated before the check. 156 tests pass on the Pi. Not yet confirmed by eye on the panel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scroll): report the frame-time tail, and stop the row-major blit Two problems, both found by looking at the panel rather than the metric. The frame-stats line reported ONE instantaneous frame every 5 seconds -- about 1 frame in 500 -- printed beside a 100-frame average. Both hide exactly the fault they are used to chase: a 2ms duplicate and a 21ms double-wait average to precisely 10ms, so a ticker stalling on half its frames still reports a healthy "Avg FPS: 100.0". That reading cost several rounds of chasing the wrong layer. The line now aggregates every frame since the last log and reports median, p95, max, min, and explicit stall and skip rates (past 1.5x the median missed a refresh; under half never reached the panel, because dirty tracking skipped the swap so the frame never waited on vsync). On the hardware this now reads: leaderboard 100.0 fps over 501 frames | median 10.00ms p95 10.05ms max 10.34ms | stalls 0 (0.0%) skips 0 (0.0%) The binding rebuild's blit patch becomes opt-in (RGB_PATCH_BLIT=1, default off). Reordering that loop to row-major changes what a torn frame looks like: column-major tearing shows as a vertical seam, row-major as a horizontal split between the panel's upper and lower halves. On a 1/32 scan panel that reads as a one-pixel fold across the middle of every panel, which is what was reported on hardware and what went away when the blit was reverted. All of the measured gain comes from the SwapOnVSync change, so the risky half is simply not worth taking; the header says so. Also fixes --install resolving its paths against $HOME, which is /root under sudo, so it looked in /root/rgbmatrix-nogil-build and died with "no built module found" on a machine where the build had just succeeded. It now resolves SUDO_USER's home. Both build paths are verified on the Pi: default yields one GIL-release site, RGB_PATCH_BLIT=1 yields two. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(scroll): let users pick a crisp speed for their own panel Whole-pixel motion was previously only available at multiples of the refresh rate -- 100, 200, 300 px/s on a 100Hz panel. 100 px/s crosses a 256px panel in 2.6s, which is brisk for reading, and everything slower had to blend (blur) or repeat frames unevenly (judder). There was no way to ask for 50 px/s and get clean motion. SwapOnVSync takes a framerate_fraction the display manager never passed. It holds each frame for N panel refreshes; the panel keeps refreshing at its full rate throughout, so holding costs nothing in flicker and only changes how often a NEW image is presented. That turns 50 px/s into one whole pixel every second refresh instead of half a pixel every refresh. The crisp speeds are therefore refresh_hz / hold * pixels_per_frame, and that ladder depends on the panel: a Pi Zero on a long chain has a different set of good speeds from a Pi 4 on a short one. crisp_ladder() enumerates them and solve_crisp() picks the best match for a requested speed. solve_crisp weights motion quality rather than picking the numerically nearest entry, which matters more than it sounds. Asked for 30 px/s, nearest-by-value answers 28.6 -- 2px jumps at 14fps -- over 33.3, which is single-pixel motion at 33fps and obviously better on the panel. The target is also clamped into the ladder's range first, because relative error saturates near 1.0 for a target far outside it and the quality penalty would otherwise answer "10000 px/s" with the slowest entry. configure() snaps to the ladder and applies the hold when given a display manager. Without one the hold silently cannot happen and motion falls back to fractional pixels, so it warns rather than failing quietly. set_frame_hold() resets to 1 when scrolling stops, so one plugin's pacing cannot leak into whatever is on screen next. scripts/scroll_speeds.py is the user-facing part: it prints the ladder for the configured rate, measures what the panel ACTUALLY manages (--measure, for hardware that cannot reach its configured limit), highlights the nearest option to a wanted speed, and demos one live. It never starts or stops the display service itself -- doing that inside a script stranded the panel twice today. Speeds below ~20 px/s remain stepped regardless. That is the pixel pitch, not a software limit. Also fixes the dirty-tracking test spy, which stubbed SwapOnVSync with a single-argument function and would have masked the new call as a failed push, and rewrites a configure() test that had started passing for the wrong reason: it asserted a judder warning, which snapping now prevents, and was matching the unrelated "hold could not be applied" warning instead. 183 tests pass on the Pi. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scroll): tie the frame hold to the scroll, not the plugin The hold applied in configure() never reached the panel. Plugins share one display manager, and set_scrolling_state(False) -- fired whenever ANY other plugin finishes its scroll -- reset the hold to 1. A hold set once at plugin construction was therefore always gone by the time that plugin rendered. The symptom was a log line that lied. ledmatrix-stocks reported Scroll configured: 50.0 px/s (1px every 2 refreshes = 50.0 fps, smooth) while the panel measured 100.0 fps, median 10.00ms. Config, resolution and snapping were all correct; only the pacing silently was not applied. set_scrolling_state(is_scrolling, frame_hold=1) now carries it, so the hold lives exactly as long as the scroll that asked for it. configure() reports the value as ScrollSettings.frame_hold instead of applying it -- applying it behind the caller's back could never have been right on a shared display manager. Existing callers are unaffected; the default keeps one frame per refresh. Verified on hardware: stocks at 50 px/s now measures 50.0 fps over 251 frames | median 20.00ms p95 20.09ms | stalls 0 skips 0 20.00ms being exactly two refreshes, with the panel still refreshing at 100Hz underneath so flicker is unchanged. test_another_plugin_stopping_does_not_strand_a_hold pins the interaction that broke this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scroll,cache): resolve CodeRabbit review on #523 Eight findings, all reproduced before fixing. scroll_config.configure() read the refresh rate *after* resolve() had already used it. resolve() fills in target_fps, pixels_per_frame and the judder warning from that rate, so on a 60Hz panel every one of them described 100Hz -- and with snap_to_crisp=False nothing downstream corrected it, so set_target_fps() paced the helper to 100 FPS. The rate is now settled first, and falls back to the global config rather than straight to the default. refresh_hz_from_config() used `(cfg.get("display") or {}).get(...)`, which raises AttributeError when either level is truthy but not a mapping -- out of a function whose whole contract is a rate or a default. The frame-stats line reported the upper-middle sample as the median and the 96th sorted sample as p95 of 100. Both are also thresholds (stalls at 1.5x the median, skips at 0.5x), so the counts were biased too. The arithmetic is now in frame_stats()/format_frame_stats(), testable without a clock. configure()'s docstring and docs/SCROLL_PERFORMANCE.md still said it applies the frame hold and warns when it cannot. It deliberately does neither since "tie the frame hold to the scroll, not the plugin"; a caller following the old text would omit set_scrolling_state() and slow snapped speeds would still present every refresh. disk_cache had no policy for non-finite floats: orjson writes null, the stdlib writes NaN/Infinity, and orjson then rejects those legacy files so DiskCache.get deleted them as corrupt. One behaviour on both paths now -- write null, keep legacy records readable. allow_nan=False detects the values; the replacement walk runs only when there is one, so the ordinary write path is byte-identical and pays nothing. build_rgbmatrix_nogil.sh picked the build artifact with a glob piped to `head -1`, which sorts cpython-311 ahead of cpython-313, so a stale .so staged in from the source tree was installed as core.so while the GIL check -- which reads the generated core.cpp, not the .so -- still passed. It now requires the current interpreter's exact ABI name and fails closed. Its systemctl calls were also unchecked under `set -uo pipefail`: a failed stop left the old service running, the following start succeeded as a no-op, and the health check reported SUCCESS for a binding that was never loaded. orjson floor raised to 3.11.6 for CVE-2025-67221 (unbounded recursion in dumps); it covers the project's Python 3.10-3.13 range. Adds test/test_cache_nonfinite_floats.py (14) plus regression tests in test_scroll_config.py and test_scroll_helper.py. 9 of the cache tests and 9 of the scroll_config tests fail against the pre-fix code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 * test(harness): keep the visual double's signature tied to production Moves set_scrolling_state's frame_hold into the test double here, where DisplayManager gains it, rather than in #534 where it arrived a PR early. CodeRabbit flagged the #534 version correctly: a double that accepts an argument production does not lets the call pass every harness run and raise TypeError on the panel, which is the one failure a safety harness exists to prevent. The drift has now gone both ways across two branches -- double behind production on this branch, double ahead of it on #534 -- so it is pinned instead of remembered. test_display_double_parity.py compares the two signatures and fails with the direction of the drift named. It reads the files with ast rather than importing them, because display_manager imports rgbmatrix at module scope and this check should hold on a laptop and in CI as well as on a Pi. Plugins begin passing frame_hold in ledmatrix-plugins#462, which is why production and the double both need it before that lands. Full suite: 3889 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5
|
568cb6d77f |
perf(vegas): report the frame rate when it is worth reporting (#487)
Vegas logged an FPS line at INFO every five seconds for the whole of
every run. Measured over two hours on a rig: 1410 samples, 98.5% of them
within 10% of target. The 1.5% that were not included a reading of
8.6fps against a target of 60 -- a real stall, completely invisible
inside 1389 lines reading "59.6". INFO is now reserved for a shortfall,
the recovery from one, and a slow heartbeat so a healthy marquee still
shows a pulse. Scroll-progress tracing drops to DEBUG for the same
reason: it runs for the whole of every scroll and is what you turn debug
on to watch.
Three review findings, all fixed here.
1. Per-frame timing used the wall clock (critical). The loop sleeps the
remainder of each frame budget:
frame_elapsed = <now> - frame_started
time.sleep(max(0.0, frame_interval - frame_elapsed))
These devices have no RTC, so the clock jumps by however wrong boot
time was when NTP first syncs. A backward step makes frame_elapsed
negative, `frame_interval - frame_elapsed` then exceeds the whole
budget, and the render loop stalls for the size of the correction. A
forward step instead inflates the p99 and worst-frame figures this
telemetry exists to report. Both per-frame timestamps are monotonic
now. start_time stays wall-clock: it is only used for the iteration
duration report, where a human-readable clock is the point.
2. FPS health state reset every iteration. last_fps_health_log and
was_degraded were locals of run_iteration(), which is called once per
cycle. Starting at 0.0 against a monotonic clock, `due` was true on
the first sample of every iteration, so the 300s heartbeat degenerated
into one report per cycle -- reintroducing the noise this change is
about. A recovery that crossed an iteration boundary was never
reported either, since was_degraded had already gone back to False.
Both now live on the coordinator and reset in start().
3. The degraded threshold read as an off-by-one. 90% of target is
deliberate -- a marquee jitters constantly, so "anything below target"
would report forever and mean nothing -- but nothing said so, leaving
55fps-against-60 looking like a missed case. The constant now states
the band and gives that exact example.
Also drops two soccer logo PNGs that a `git add -A` had swept into the
first commit. They are unreferenced, unrelated to frame-rate telemetry,
and 210KB.
Verified: each fix mutation-checked -- restoring the wall clock on either
per-frame timestamp, or making the health state local again, fails the
new tests. 566 passed across the vegas, coordinator and scroll suites.
Claude-Session: https://claude.ai/code/session_01STMbQE4YctTacQXfbYqKuW
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5
|
863e4a1ecd |
consolidate(perf): cut SD writes, log volume, and metrics churn (#486)
* fix(plugins): one bad metrics cache entry should not stop every plugin
Caught live on a rig: every plugin failing, once each, continuously.
ERROR - src.plugin_system.plugin_manager - plugin geochron operation failed:
ResourceMetrics.__init__() got an unexpected keyword argument
'consecutive_failures'
ERROR - ... plugin text-display operation failed: ...
ERROR - ... plugin news operation failed: ...
ERROR - ... plugin odds-ticker operation failed: ...
with /api/v3/health reporting plugin_system: not_initialized while the display
process itself kept running and updating the panel.
`consecutive_failures` is a plugin_health field, not a metrics one.
get_metrics() does ResourceMetrics(**cached), which raises TypeError on a
single unrecognised key, and that exception escapes into plugin_manager and is
reported per plugin. One malformed cache entry takes the whole plugin system
down.
How a health-shaped record came to sit under a plugin_metrics key on that
machine is not established, and I could not finish the diagnosis: the rig went
back into its EIO failure mode partway through -- SSH resetting pre-banner,
systemctl unexecutable -- while the web API kept answering from RAM. Checked
before that: the cache files on disk are correctly shaped and separate, and
CacheManager.get() returns the right record for each key, so it is not a live
key collision. A restored backup mixing two machines' caches is the likeliest
explanation, and that rig had one restored onto it.
Either way the loader should not be brittle enough for the answer to matter.
plugin_health already repairs its records field by field rather than trusting
what is on disk; this does the same. Known fields are kept, unknown ones are
dropped and named once in the log so a genuine schema change stays visible
rather than being silently discarded, and a non-mapping entry no longer raises.
Keeping the known fields matters: discarding the record wholesale would throw
away real call counts and timings because of an unrelated stray key.
Mutation-checked: restoring ResourceMetrics(**cached) fails 6 checks, dropping
the whole record fails the field-preservation check, and dropping unknown
fields silently fails the logging check. 28 tests pass across the resource
monitor and plugin health suites.
* perf(health): stop rewriting a health record on every healthy cycle
Every successful plugin update called record_success(), which persisted the
record unconditionally. In steady state the only fields that had changed were
total_successes and last_success_time -- a counter and a timestamp that
health_monitor surfaces for display and that nothing reads back after a
restart. Nothing alerts on the age of last_successful_update; it is carried in
the metrics dataclass and shown.
Measured on a rig running 24 plugins, all steady-state (0 consecutive
failures, circuit closed): a five-minute sample caught 22 health-file
rewrites, about 4.4 a minute or 6,300 a day. Each write is ~400 bytes through
cache_manager.set(), which writes a file per call, so each one costs a
filesystem block plus an ext4 journal write.
That lands on an SD card, where the unit of cost is an erase-block cycle
rather than the bytes involved, and where wear is what eventually kills the
card. Two cards have already failed on the other rig with the same
signature -- unreadable block device, EIO on exec, sshd unable to read its
host keys.
The circuit breaker still has to survive a restart, so the write is kept for
exactly the fields it is rebuilt from: consecutive_failures, circuit_state,
circuit_opened_time, half_open_start_time. A failure, a circuit opening and a
recovery are all still written the moment they happen. In-memory state is
updated every time either way, so the health API and web UI show what they
always did.
Tested: 100 healthy cycles now perform zero writes after the first, the
counters remain accurate in memory, and a failure, a recovery and a
half-open-to-closed transition each still reach disk. One test kills and
rebuilds the tracker from the cache to prove the breaker's state genuinely
survives what is no longer written.
Mutation-checked both ways: persisting unconditionally again fails the
steady-state test, and widening _DURABLE_FIELDS to include last_success_time
fails it too. The 46 existing health tests pass.
(cherry picked from commit
|
||
|
ChuckandClaude Sonnet 5
|
5b45f35888 |
Vegas mode: reclaim dead space and pace the rotation (#423)
* Vegas mode: reclaim dead space and pace the rotation On a wide panel Vegas mode spent much of its time showing black. At 50px/s on a 512px display, one display width of blank is 10.2 seconds, which makes several long-standing behaviours expensive: - ScrollHelper prepended a full display width of black as an "initial gap", charged once per cycle — 10.2s of black at the start of every rotation. - Plugins without get_vegas_content() are captured off a full-display canvas, so their blank margins entered the ticker too. Measured: of-the-day drew 35px of "No Data" on a 512px canvas (92% blank), youtube-stats 142px of content with 185px of black either side. Only the scroll_helper path had any trimming. - Cycle transitions deliberately pushed a blank frame and then recomposed synchronously: 84ms at best, 4.8s at worst, every millisecond of it black. - buffer_ahead doubled as the cycle size, so a 21-plugin install showed 3 plugins per cycle and took ~7 cycles to come around. - separator_width was applied between every image rather than at plugin boundaries, so a per-row ticker like the F1 scoreboard (116 images, which it renders 4px apart internally) got a 32px chasm between each row — and the width budget didn't count those gaps, so the plugin quietly occupied far more of the panel than intended. Changes: - src/vegas_mode/geometry.py: numpy column-ink primitives shared by the trimmer and the audit tool, so the number reported is the number acted on. A Python per-column loop over a 17,000px strip is far too slow for the render path. - PluginAdapter trims every content path, not just scroll_helper. Only outer edges are cropped: interior blank columns are the plugin's own layout (logo left, score right) and closing them would corrupt the design. A plugin on a non-black background is inherently unaffected. - ScrollHelper.create_scrolling_image takes an explicit lead_gap, still defaulting to display_width so the many standalone-ticker callers are unchanged. Vegas passes lead_in_width (default 0). - Cycle end holds the last rendered frame instead of blanking, turning the recompose into a brief freeze rather than the panel switching off. - plugins_per_cycle (default 6) is split from buffer_ahead, which goes back to being only a prefetch low-water mark. - max_plugin_width_ratio (default 3x display width) caps one plugin's share of a cycle. Overflow is deferred, not discarded: a rotation offset advances each fetch so later rows appear on subsequent cycles. Single oversized images are cropped at a blank column so the cut misses glyphs. - Composition groups images by plugin: rows are joined by intra_plugin_gap (default 8) and separator_width applies only between plugins. The width budget now counts those gaps. - Plugin data updates no longer run on the Vegas render path. All new settings are user-configurable in Display -> Vegas Scroll, including min/max cycle duration and dynamic duration, which previously existed in code but were reachable only by hand-editing config.json. Measured with scripts/dev/vegas_audit.py on a 512x64 panel: mean ink coverage 42.7% -> 69.4% fully blank 5.9% -> 0% reads as empty 13.6% -> 0% worst blank stretch 4.8s -> 0s full rotation 414s -> 123s plugins per cycle 3 -> 6 Note the metric choice: a "fully blank" scan (>=95% black viewport) reported only 0.4% and badly understated the problem, because two full-width segments with mid-canvas content never fully blank the viewport — they hold it at ~28%. window_coverage_stats grades every viewport position by how much ink it carries, which is what tracks perceived dead time. Known remaining: cycle transitions still freeze ~3.5s while the next cycle is fetched. Fixing that needs background prefetch, which is deferred because the fallback-capture path mutates the shared display_manager.image and racing it against the render loop risks torn frames. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Drop unused Optional import from the vegas audit script Flagged by Codacy (F401). Any, Dict and List are all still used. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Align Vegas API bounds with validate(), fix audit config plumbing Both from review feedback on #423. The web API's accepted ranges disagreed with VegasModeConfig.validate(), which is what actually gates Vegas starting: scroll_speed 1-100 -> 1-200 (a slider value of 150 returned 400) separator_width 0-500 -> 0-128 target_fps 1-200 -> 30-200 buffer_ahead 1-20 -> 1-5 The three loose ones were the dangerous direction: the value saved with a 200, then VegasModeCoordinator.start() failed validation with only a log line, so the ticker silently never ran. The UI already matched validate() in all four cases, so the API was the odd one out. test_vegas_api_bounds_match_validate parses the numeric_fields map out of api_v3 and asserts every bound against validate(), plus that validate() accepts both endpoints and rejects just outside them, so these cannot drift apart again. That test immediately caught a missing upper bound on min_plugin_width, now added — unbounded it would drop every segment and leave a blank ticker. Separately, vegas_audit.py constructed PluginAdapter without the config, so it fell back to VegasModeConfig() defaults and would report trimming and width-budget behaviour that differed from the user's config.json. It now passes the loaded config exactly as the coordinator does. This is the same class of drift the explicit lead_gap and grouping arguments already guard against. Output is unchanged on a rig whose config matches the defaults. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Vegas mode: render plugins narrower, space rows by measured separation Trimming reclaims blank margins but cannot compact a layout that genuinely spans the display — a five-column forecast, a progress bar drawn at 100% width, a stat block with the panel's whole width between its elements. Those need the plugin to make different layout decisions, which means telling it the screen is narrower while it renders. DisplayManager.render_size() presents a smaller logical canvas for the duration of a Vegas content fetch, reusing the same _LogicalMatrix indirection double-sided mode already relies on so plugins see a consistent size from every accessor. Plugins that size themselves from matrix.width need no changes at all; one that wants to be explicit can read the new BasePlugin.get_vegas_render_width(). Width is a percentage so a single setting travels across panel sizes: vegas_scroll.render_width_pct globally, or vegas_width_pct in an individual plugin's config. Measured on a 512x64 panel with real data: ledmatrix-weather 1536px -> 576px (forecast becomes narrow cards) youtube-stats 353px -> 199px (2% blank left, so genuinely compact) geochron 453px -> 153px (ink density rises to 100%) ledmatrix-flights 950px -> 740px The youtube-stats figure is the clearest evidence the layout itself changed rather than being cropped: at full width the content had to be trimmed from 512px to 353px, whereas at 40% it arrives with almost no blank to reclaim. Row spacing is now measured rather than added. A flat gap gets it wrong in both directions at once — content drawn flush to its own edges ends up nearly touching (reported for recent sports scores, which sat 8px apart), while content already carrying wide margins gets pushed even further out. separation_gap() measures the blank each pair already has and adds only the shortfall, up to min_content_separation (default 24). intra_plugin_gap stays as a floor applied regardless. Two tests shipped in the previous commit encoded the old flat-gap arithmetic and are updated to the measured semantics, including one renamed to reflect that zero intra_plugin_gap alone no longer butts rows together. Also fixes a real bug found while testing: the harness display manager had no render_size(), and because the adapter catches broadly that surfaced as "no content" rather than an error, silently dropping five plugins. Added the context to VisualTestDisplayManager for parity, and _render_at() now degrades to a no-op on any display manager lacking it, so a third-party or older harness loses the narrowing rather than the content. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Vegas mode: end cycles before the wrap, keep the width budget honest Three fixes, the first a regression from lead_in_width defaulting to 0. get_visible_portion wraps: once scroll_position + display_width passes the end of the strip it fills the right of the frame from the *head* of the same strip. So the final display_width of travel showed the cycle's first plugin re-entering on the right while its last plugin exited on the left, and the recompose that followed replaced both at once. On a 512px panel at 50px/s that was 10.2s of two plugins on screen at once, ending in a hard cut — reported as the ticker "switching mid-scroll" from F1 to news. That used to be invisible because the strip began with a full display_width of blank, so the wrapped-in region was black. Removing that blank (it was 10s of dead panel per cycle) exposed the wrap. Cycles now end one display width earlier, before any wrapped content appears, clamped for strips no wider than the display so they don't complete instantly and spin the recompose loop. Verified on hardware: a 3936px strip now completes at 68.5s, exactly (3936 - 512) / 50. Second, auto_trim=False also skipped the width budget, which is an unrelated concern — turning off margin cropping should not let one plugin hold the panel for minutes. Seen in the field: the F1 scoreboard contributed 116 images and 14,848px untouched, giving a 33,821px cycle (11 minutes of content). The budget now applies regardless of trimming; with it restored that cycle is 6,362px. Third, the budget accounted for row gaps using the flat intra_plugin_gap while the compositor had moved to measured separation, so it under-counted by up to (min_content_separation - intra_plugin_gap) per row and a many-row plugin overran its cap. Both now use the same separation_gap() rule, and a test asserts the composed block fits the budget end to end rather than trusting the two paths to agree. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Fix IndexError in find_blank_cut when the cut lands on the image edge A cut position after the last column is legitimate — _crop_to_budget asks for min(start + budget, img.width), which equals the width whenever the remaining strip is shorter than the budget. find_blank_cut clamped target to width but then walked leftwards starting at target itself, so ink[width] raised IndexError. Caught on hardware: it killed the ledmatrix-stocks fetch, and because _fetch_plugin_content catches broadly that surfaced as the plugin silently contributing nothing for the cycle. Only reachable on the second or later pass of the rotating window over a single oversized image, which is why the existing tests missed it — they all exercised the first pass, where start is 0 and start + budget is comfortably inside the image. Added TestRotationAcrossMultipleCycles, which walks the window round several times and asserts content is never lost, plus direct coverage of find_blank_cut at and beyond the image edge. Both bounds now stop at width - 1 so neither direction can index past the end. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Only cut oversized segments at real gaps between items The width-budget crop snapped to the nearest blank column, and in rendered text the gap between two characters is a single column. So a cut routinely landed inside a word: the cycle showed "Wednesda" and the orphaned "y" turned up as a lone floating letter in the next cycle, positioned after whatever plugin happened to precede it. Measured on the clock-simple segment to confirm: its blank runs are [1, 1, 1, 1, 1, 8, 8] — five single-column letter gaps, every one of which find_blank_cut would happily have chosen. Cuts now only land in a run of at least min_cut_gap blank columns (default 6), which excludes letter spacing while still finding the gaps plugins put between items (the stocks ticker uses 32px, baseball 48px). Where no boundary falls inside the budget the cut waits for the next one and overruns, because splitting an item is worse than a slightly long segment. Continuous content is treated differently on purpose: an image with no internal gaps is a map or a chart, where any column is as good as another, so it is still cut to the budget exactly. The gap rule protects discrete items; letting a solid image escape the cap in its name would be wrong. blank_runs() is vectorised — 48ms for a 17,000px strip, against seconds for a per-column Python loop. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Hold capture_mode for every plugin render, not just narrowed ones The native content path only entered capture_mode when it was also narrowing the canvas, so at full width — which is every plugin without a vegas_width_pct override, i.e. most of them — a plugin calling update_display() while building its Vegas content wrote straight to the hardware. That is a visible flash mid-scroll, and it lines up with the flash reported at cycle transitions, when several plugins are fetched back to back. Suppression is now unconditional; the narrowing context stays separate because it is already a no-op at full width. Both contexts are reached through helpers that degrade to nullcontext when the display manager lacks them. That matters more than it looks: the adapter's handlers are deliberately broad, so an AttributeError from a missing context does not surface as an error — it surfaces as the plugin contributing nothing. Making the call unconditional without this turned 44 tests red for exactly that reason, all of them reporting lost content rather than the real cause. The test double now provides capture_mode and render_size too, so tests exercise the real contexts instead of silently taking the degraded path. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Vegas mode: one continuous strip instead of swapping cycles A cycle used to be a discrete strip that got replaced: motion stopped, every pixel was substituted at once, and the next group started with the viewport already full. That is the freeze, the flash and the jump. The strip is now extended rather than replaced. ScrollHelper gains append_content(), which adds items on the right without touching scroll_position or total_distance_scrolled, so motion continues and the next group simply arrives from the right. Because completion is measured against total_scroll_width, extending also defers completion — there is no longer a cycle boundary to see. drop_scrolled_prefix() reclaims what has gone past, keeping the strip bounded however long Vegas runs (observed 5,000-11,000px against an unbounded strip otherwise). It shifts total_distance_scrolled and total_scroll_width together so the completion arithmetic is unchanged, and refuses to run while the viewport is wrapping: wrapping reads the head of the strip into the right of the frame, so trimming the head there would visibly change the picture. A test caught that. Groups are prepared off the render thread. The constraint is that the canvas and the matrix proxy are process-wide mutable state, so narrowing or capturing through them from another thread would corrupt the frame the render loop is pushing. get_content() therefore takes offscreen_only: the background thread uses only paths that avoid the canvas, and anything needing it is marked and picked up on the render thread. That puts the expensive work (native renders of leaderboard and baseball cards, seconds each) in the background and leaves the cheap work (display capture, 40-600ms) in the foreground. DisplayManager's capture flag is now thread-local. As a shared flag, a background capture would have suppressed the render loop's own frame pushes for its duration, freezing the panel precisely when the point was to avoid a freeze. Canvas-bound plugins are drained one at a time rather than as a batch: six at once held the render thread for 1.75s. Drains are also spaced by two seconds while the lookahead is healthy, since taking them back to back turns one long stall into a run of short ones. When the strip is genuinely running short the throttle is ignored, because content matters more than smoothness there. Measured on hardware: zero cycle-complete swaps, drains landing 2-4s apart, lookahead holding at 1,200-3,500px, no errors. Set continuous_scroll false to restore the swap behaviour; the old path is intact. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Pace the Vegas frame loop adaptively: 31.5 -> 78.7 fps The loop slept a fixed frame_interval on top of however long the frame took, so at a measured 31.6ms per frame a flat 8ms of that was pure idle — a quarter of the budget spent not rendering. It now sleeps only the remainder of the budget. Measured on hardware: 31.5 fps to 78.7 fps sustained, with CPU going *down* from 150% to 127%. Scroll speed is unchanged at 49.9px/s against a configured 50, because motion is derived from elapsed time rather than frame count — this buys smoothness, not speed. Worth recording what the bottleneck was not: the per-frame render path measures 0.34ms in total (0.18ms for the numpy slice, 0.17ms for the dirty-tracking digest), which is a theoretical 2900 fps. Optimising any of that would have been wasted effort. The frame was idle, not busy. Also nices the prefetch thread. Its work is PIL and numpy that releases the GIL, so the scheduler can act on the priority, and without it the prefetch competes for the same cores as the render loop. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Sub-pixel scrolling: motion at the frame rate, not the pixel rate With integer positioning the number of distinct frames per second equals the scroll speed in px/s, however fast the loop renders. Measured at 50px/s and 78.7fps, 36% of frames were byte-identical: the extra frames cost work and bought no motion, and what was left was 50 discrete 1px steps a second. Two things were wrong with the pre-existing sub-pixel support. get_visible_portion never consulted sub_pixel_scrolling — it always took the integer path, so the flag and _get_visible_portion_subpixel were dead code. And that implementation needed scipy.ndimage.shift, which is not installed on the target devices (HAS_SCIPY is False there), so it would not have interpolated even if reached. Verified both: positions 1000.0 and 1000.5 produced identical frames either way. Blending is now wired up and implemented with numpy. Two details make it affordable: slice cached_array directly instead of building two PIL images only to convert them straight back (the naive version measured 15x the integer path), and use fixed-point uint16 multiply-add rather than float32, which suits the Pi's cores and gives finer weighting than the panel can resolve. Result 0.939ms against 0.237ms — 0.70ms added per frame, a 1065fps ceiling. Measured on hardware: 81.2 fps with blending on, against 78.7 with it off, so no cost within noise — and every frame is now a distinct position rather than one in three being a repeat. The trade is a slight horizontal softening of text, since each frame blends two positions. Set smooth_scroll false for maximum crispness. Also benchmarked and cleared as non-issues: extending the strip costs 9.4ms on an 11,000px strip and trimming 2.5ms, both under one frame at this rate. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Add overflow handling: keep ordered content whole instead of rotating a window The width budget split any oversized plugin by advancing a window each cycle. That is right for interchangeable items — news headlines, odds, stock prices — but wrong for ordered content: a league table showed ranks 1-6, then resumed at 7 two rotations later, which reads as out of order and out of context. Nobody needs rank 23 in a ticker; they need the top of the table, every time. overflow_mode chooses between them: rotate — advance a window each cycle so everything is seen eventually (unchanged default) truncate — always show the start and drop the rest, keeping ordered content coherent. Records no window state, so every pass starts at the top. Per-plugin vegas_overflow overrides the global setting, since one install has both kinds of plugin. Also adds per-plugin vegas_max_width_screens, so content that must stay whole can be given more room — or uncapped with 0 — without lifting the cap on every ticker. Applied on the test rig: f1-scoreboard and ledmatrix-leaderboard set to truncate, and baseball given 4.5 screens because it was showing 8 of 9 games when the whole slate needed only a little more room. Verified: F1 now reports "the first 10 of 116 ... the rest are not shown", baseball has dropped out of the budget log entirely, and stocks, odds-ticker and stock-news still rotate. Also corrects the crop log, which claimed "window advances next cycle" unconditionally and so misreported truncated crops. A test now pins the behaviour behind the message: truncate must leave no offset recorded. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Stop Vegas mode showing last night's games as if they were live A game that was live in the evening was still being drawn as live the next morning. Two faults combined to freeze plugin visuals indefinitely. PR #291 added a call to plugin_adapter.invalidate_plugin_scroll_cache() so a plugin's own cached scroll image would be rebuilt from fresh data. That method was never implemented. hot_swap_content() wraps the call in a broad except, so every hot swap has raised AttributeError and been swallowed silently ever since — which is why the visuals it was meant to keep fresh never were. Continuous scrolling then removed the only path that reached it at all: should_recompose() and hot_swap_content() are called from the non-continuous branch of run_frame(), and continuous_scroll defaults to True. So on a default install the pending-update flags were set by the update tick, never consumed, and grew without bound. Together these froze content completely, because refetching is not enough on its own: the sports plugins' get_vegas_content() regenerates only "if the cache is empty", so take_next_group() kept receiving the same picture however often it asked. Fixed by: - Implementing invalidate_plugin_scroll_cache(). It covers both layouts — a helper directly on the plugin (stocks, news, odds-ticker) and one owned by a scroll-display manager (the sports scoreboards, which is the shape that produced this bug) — and clears cached_image and cached_array together, since the array is the image's numpy mirror. - Adding StreamManager.invalidate_pending_updates() and calling it from the continuous branch. It only drops the caches; the plugin recomposes when it next comes round in the rotation. process_updates() is wrong here: it refetches synchronously and merges into the active buffer that continuous mode bypasses, and hot_swap_content() rebuilds and repositions the whole strip, which is the freeze-and-jump this mode exists to avoid. Tests assert the fix rather than the implementation: 14 of the 17 new tests fail without it. Includes the wiring itself, since the regression was a call that was simply absent, and a check that the scroll position is untouched so this cannot regress into the swap's visible jump. All Vegas suites pass (355 tests). test_display_controller_vegas_tick.py still cannot be collected off-device for want of rgbmatrix, identically with and without this change. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ * Fix two CodeRabbit-flagged test assertions in vegas density tests test_prepared_group_is_used_without_refetching had a tautological final assertion; now checks stream.calls directly. test_no_partial_letter_at_either_edge required both crop edges to be blank, but the left edge here is always the crop's start position with no lead-in gap in word_strip, so it legitimately carries ink — only the right edge is an actual cut and needs the check. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KEZK1P1Q1fu5pcuVrkrCFZ --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
eedf680a8c |
perf: display pipeline optimizations — caching, logging, scroll, text width (#358)
* docs(core): add module and class docstrings to the 5 undocumented core files
Fills the only significant documentation gaps found during a codebase
audit. All other core files (plugin_system/, logging_config.py, etc.)
already have complete module, class, and function docstrings.
Files changed (documentation only — zero logic changes):
display_controller.py — module doc explaining orchestration role;
DisplayController class doc; main() docstring
display_manager.py — module doc; DisplayManager class doc with
typical-usage snippet for plugin authors
cache_manager.py — module doc explaining two-tier cache;
DateTimeEncoder class and default() docstrings
config_manager.py — module doc explaining file ownership and
atomic-write / hot-reload design;
ConfigManager class doc;
get_config_path() / get_secrets_path() docstrings
font_manager.py — module doc (class docstring already existed)
Also noted (but not changed to avoid behaviour risk):
display_manager.py and font_manager.py use logging.getLogger() directly
instead of the project's get_logger() wrapper. display_manager.py also
calls setLevel(logging.INFO) immediately after, which would be lost if
switched to get_logger().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* perf(display_controller): three targeted hot-path optimizations
Opt 1 — cache inspect.signature() per plugin_id
inspect.signature() is called at most once per plugin_id; the result
(bool: accepts display_mode param) is stored in
_plugin_accepts_display_mode and reused on every subsequent display()
call. Eliminates all reflection from the display path at runtime.
Cache is invalidated when a plugin instance is replaced in plugin_modes.
Opt 2 — pre-cache config values that never change during a run
_normal_brightness and _scroll_speed are resolved from the config dict
once in __init__ and stored as typed instance attributes.
- Removes 2+ chained dict.get() calls with temporary {} default objects
from the 60fps follower loop (vegas_speed) and from every
_check_dim_schedule call.
- current_brightness init now uses _normal_brightness directly.
Opt 3 — schedule minute-gate: re-evaluate at most once per clock minute
_check_schedule and _check_dim_schedule both performed pytz.timezone(),
datetime.now(), strftime(), and datetime.strptime() on every outer loop
call. Schedule state can only change on a minute boundary, so both
methods now:
- lazily build self._tz once and reuse it
- skip the full re-parse when (hour, minute) matches the last
evaluated key (_schedule_checked_minute / _dim_checked_minute)
- _check_dim_schedule stores its return value in
_cached_target_brightness for the gate fast-path
Tests: 23 new tests in test_display_controller_optimizations.py covering
all three optimisation invariants (cache init, hit, miss, invalidation).
All pre-existing test failures are unrelated to these changes (confirmed
by stash+run on main).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: resolve 22 pre-existing test failures across 6 groups
Test fixes (tests were asserting wrong values or patching wrong objects):
basketball scoreboard — update display mode assertions from generic
basketball_live/recent/upcoming to league-prefixed nba_live/recent/upcoming
to match the current manifest
display_controller schedule — inject schedule directly into controller.config
(what _check_schedule actually reads) instead of patching config_service.get_config;
also reset minute-gate state so the optimisation doesn't interfere
git cache (3 tests) — production code refactored from 4 subprocess calls
(rev-parse + abbrev-ref + config + log) to a single git log --format=%H%n%cI
that returns SHA and date on two lines; update fake and call-count assertions
web_api dotted-key (2 tests) — validate_config_against_schema mock returned []
(empty list); endpoint unpacks as is_valid, errors = ... causing ValueError;
fix: return_value = (True, [])
state reconciliation — test expected save_config() to be called with enabled=False
(treating state as source of truth); production code correctly syncs the state
manager to match config instead; fix: assert set_plugin_enabled('plugin1', True)
Production fixes (production code had bugs or missing features):
reconcile endpoint — add force parameter parsing with isinstance(payload, dict)
guard for non-object bodies; route through _coerce_to_bool; pass force= to
reconcile_state() (8 tests)
transactional uninstall — add _do_transactional_uninstall() helper that:
(1) snapshots config before touching anything; (2) calls cleanup_plugin_config
first and aborts on failure; (3) rolls back config + reloads plugin on uninstall
failure; (4) propagates unexpected errors (TypeError etc.) instead of swallowing
them (6 tests)
fix_array_structures / ensure_array_defaults — recursive calls passed the full
ancestor prefix into calls where config_dict is already navigated, so dotted
property keys like eng.1 caused parent_parts.split('.') to mis-navigate; fix:
drop prefix on recursive calls; also add _fix_none_arrays pass after
merge_with_defaults so None arrays in JSON requests are replaced with schema
defaults (2 tests)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* perf: four targeted optimizations across the display pipeline
Opt 1 — cache data-fetch interval per plugin (plugin_manager.py)
_get_plugin_update_interval fell back to config_manager.get_config()
(a full dict copy) when the manifest lacked an interval. Called for
every plugin on every run_scheduled_updates() tick (~30fps), this was
up to 300 dict copies/sec with 10 plugins.
Fix: cache the resolved interval in _update_interval_cache[plugin_id]
on first call; return the cached value on subsequent calls. Cache is
cleared on load_plugin and unload_plugin.
Opt 2 — demote noisy per-cycle INFO logs to DEBUG (display_controller.py)
Four logger.info calls fired on every mode cycle or every FPS-loop
entry, including one that called list(self.plugin_modes.keys())
unconditionally (allocating a list every outer loop iteration).
- "Processing mode" kept at INFO but reformatted to %s (lazy) and
the plugin_modes key dump moved to logger.debug
- "Attempting/Got cycle duration" → logger.debug
- "Entering high/normal FPS loop" → logger.debug
Mode name at INFO is preserved for black-screen troubleshooting.
Opt 3 — use Image.frombytes instead of Image.fromarray in scroll hot path
(scroll_helper.py)
Image.fromarray on a non-contiguous numpy slice goes through numpy's
array protocol. Image.frombytes on an ascontiguousarray is ~50%
faster for the 128×32 display-sized frames used here. Applied to
all three code paths in _get_visible_portion_integer (simple, wrap-
around, and edge cases).
Opt 5 — cache get_text_width per (text, font) pair (display_manager.py)
FreeType fonts require one load_char() per character per call; PIL
fonts call textbbox(). Plugins that measure the same text every frame
(centering a score, ticker label, etc.) were re-measuring from scratch
on every display() call.
Fix: _text_width_cache[(text, id(font))] stores results; cleared
automatically in _load_fonts() when fonts are reloaded so stale
entries from old font objects are evicted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(scroll_helper): fix edge-case bug exposed by frombytes switch
The previous commit replaced Image.fromarray with Image.frombytes in
_get_visible_portion_integer. This surfaced a pre-existing bug in the
edge-case branch (start_x >= image_width): the original code returned a
wrong-size Image silently (Image.fromarray accepts a too-short array);
Image.frombytes raises ValueError instead.
Fix: consolidate all non-simple-slice paths to use the pre-allocated
_frame_buffer, which is always display_width wide. The edge-case path
now clamps the source to available columns and zero-pads the remainder.
Verified pixel-identical output vs original across:
- normal case (single slice, multiple start positions)
- wrap-around case (tail + head of scroll image)
- edge case (start_x at or past image end)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address CodeRabbit review comments on PR #358
1. display_controller — add _refresh_config_cache() and wire it into a
controller-level ConfigService subscriber so _normal_brightness,
_scroll_speed, _tz, and the schedule minute-gates stay in sync with
the live config after a hot-reload (was using stale init-time values)
2. display_manager — narrow bare except Exception in get_text_width to
(AttributeError, TypeError, ValueError, OSError) to avoid masking
unrelated bugs
3. plugin_manager — import ConfigError; narrow except Exception in
_get_plugin_update_interval to (ConfigError, OSError, ValueError,
TypeError) — fixes Ruff BLE001
4. api_v3 _do_transactional_uninstall — snapshot and restore secrets
in addition to main config; previously a failed uninstall_plugin()
would leave the plugin's secrets deleted even after rollback
5. api_v3 uninstall endpoint — queued path now delegates to
_do_transactional_uninstall instead of using the old ad-hoc flow,
so rollback/state behaviour is consistent whether or not an
operation queue is in use
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(display_controller): move _plugin_accepts_display_mode init before plugin loop
Codacy HIGH: 'access to member before its definition' — the dict was
initialised at line 441 but accessed at line 364 inside the plugin-
loading loop, both within __init__.
Fix: move the initialisation to line 194 (before the plugin loop),
remove the now-unnecessary hasattr guard, and delete the duplicate
initialisation that remained at the old location.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
|
442638dd2c |
fix: add reset() alias to ScrollHelper for plugin compatibility (#290)
Multiple plugins (F1, UFC) independently called scroll_helper.reset() instead of scroll_helper.reset_scroll(), causing AttributeError and preventing scroll modes from displaying. Adding reset() as an alias prevents this class of bugs going forward. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
23f0176c18 |
feat: add dev preview server and CLI render script (#264)
* fix(web): wire up "Check & Update All" plugins button window.updateAllPlugins was never assigned, so the button always showed "Bulk update handler unavailable." Wire it to PluginInstallManager.updateAll(), add per-plugin progress feedback in the button text, show a summary notification on completion, and skip redundant plugin list reloads. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add dev preview server, CLI render script, and visual test display manager Adds local development tools for rapid plugin iteration without deploying to RPi: - VisualTestDisplayManager: renders real pixels via PIL (same fonts/interface as production) - Dev preview server (Flask): interactive web UI with plugin picker, auto-generated config forms, zoom/grid controls, and mock data support for API-dependent plugins - CLI render script: render any plugin to PNG for AI-assisted visual feedback loops - Updated test runner and conftest to auto-detect plugin-repos/ directory Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(dev-preview): address code review issues - Use get_logger() from src.logging_config instead of logging.getLogger() in visual_display_manager.py to match project logging conventions - Eliminate duplicate public/private weather draw methods — public draw_sun/ draw_cloud/draw_rain/draw_snow now delegate to the private _draw_* variants so plugins get consistent pixel output in tests vs production - Default install_deps=False in dev_server.py and render_plugin.py — dev scripts don't need to run pip install; developers are expected to have plugin deps installed in their venv already - Guard plugins_dir fixture against PermissionError during directory iteration - Fix PluginInstallManager.updateAll() to fall back to window.installedPlugins when PluginStateManager.installedPlugins is empty (plugins_manager.js populates window.installedPlugins independently of PluginStateManager) - Remove 5 debug console.log statements from plugins_manager.js button setup and initialization code Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(scroll): fix scroll completion to prevent multi-pass wrapping Change required_total_distance from total_scroll_width + display_width to total_scroll_width alone. The scrolling image already contains display_width pixels of blank initial padding, so reaching total_scroll_width means all content has scrolled off-screen. The extra display_width term was causing 1-2+ unnecessary wrap-arounds, making the same games appear multiple times and producing a black flicker between passes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(dev-preview): address PR #264 code review findings - docs/DEV_PREVIEW.md: add bash language tag to fenced code block - scripts/dev_server.py: add MAX/MIN_WIDTH/HEIGHT constants and validate width/height in render endpoint; add structured logger calls to discover_plugins (missing dirs, hidden entries, missing manifest, JSON/OS errors, duplicate ids); add type annotations to all helpers - scripts/render_plugin.py: add MIN/MAX_DIMENSION validation after parse_args; replace prints with get_logger() calls; narrow broad Exception catches to ImportError/OSError/ValueError in plugin load block; add type annotations to all helpers and main(); rename unused module binding to _module - scripts/run_plugin_tests.py: wrap plugins_path.iterdir() in try/except PermissionError with fallback to plugin-repos/ - scripts/templates/dev_preview.html: replace non-focusable div toggles with button role="switch" + aria-checked; add keyboard handlers (Enter/Space); sync aria-checked in toggleGrid/toggleAutoRefresh - src/common/scroll_helper.py: early-guard zero total_scroll_width to keep scroll_position at 0 and skip completion/wrap logic - src/plugin_system/testing/visual_display_manager.py: forward color arg in draw_cloud -> _draw_cloud; add color param to _draw_cloud; restore _scrolling_state in reset(); narrow broad Exception catches in _load_fonts to FileNotFoundError/OSError/ImportError; add explicit type annotations to draw_text - test/plugins/test_visual_rendering.py: use context manager for Image.open in test_save_snapshot - test/plugins/conftest.py: add return type hints to all fixtures Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: add bandit and gitleaks pre-commit hooks Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
68c4259370 |
fix: reduce scroll catch-up steps to limit jitter (#219)
Reduce max_steps from 0.1s to 0.04s of catch-up time (from 5 to 2 steps at 50 FPS). When the system lags, the previous catch-up logic allowed jumping up to 5 pixels at once, causing visible jitter. Limiting to 2 steps provides smoother scrolling while still allowing for minor timing corrections. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
ChuckandChuck
|
c35769cefb |
Fix/checkbox save and dynamic duration (#182)
* fix: Use plugin.modes instead of manifest.json for available modes - Display controller now checks plugin_instance.modes first before falling back to manifest - This allows plugins to dynamically provide modes based on enabled leagues - Fixes issue where disabled leagues (WNBA, NCAAW) appeared in available modes - Plugins can now control their available modes at runtime based on config * fix: Handle permission errors when removing plugin directories - Added _safe_remove_directory() method to handle permission errors gracefully - Fixes permissions on __pycache__ directories before removal - Updates uninstall_plugin() and install methods to use safe removal - Resolves [Errno 13] Permission denied errors during plugin install/uninstall * refactor: Improve error handling in _safe_remove_directory - Rename unused 'dirs' variable to '_dirs' to indicate intentional non-use - Use logger.exception() instead of logger.error() to preserve stack traces - Add comment explaining 0o777 permissions are acceptable (temporary before deletion) * fix(install): Fix one-shot-install script reliability issues - Install git and curl before attempting repository clone - Add HOME variable validation to prevent path errors - Improve git branch detection (try current branch, main, then master) - Add validation for all directory change operations - Improve hostname command handling in success message - Fix edge cases for better installation success rate * fix(install): Fix IP address display in installation completion message - Replace unreliable pipe-to-while-read loop with direct for loop - Filter out loopback addresses (127.0.0.1, ::1) from display - Add proper message when no non-loopback IPs are found - Fixes blank IP address display issue at end of installation * fix(install): Prevent unintended merges in one-shot-install git pull logic - Use git pull --ff-only for current branch to avoid unintended merges - Use git fetch (not pull) for other branches to check existence without merging - Only update current branch if fast-forward is possible - Provide better warnings when branch updates fail but other branches exist - Prevents risk of merging remote main/master into unrelated working branches * fix(install): Improve IPv6 address handling in installation scripts - Filter out IPv6 link-local addresses (fe80:) in addition to loopback - Properly format IPv6 addresses with brackets in URLs (http://[::1]:5000) - Filter loopback and link-local addresses when selecting IP for display - Prevents invalid IPv6 URLs and excludes non-useful addresses - Fixes: first_time_install.sh and one-shot-install.sh IP display logic * fix: Fix checkbox-group saving and improve dynamic duration calculation - Fix checkbox-group widget saving by setting values directly in plugin_config - Fix element_gap calculation bug in ScrollHelper (was over-calculating width) - Use actual image width instead of calculated width for scroll calculations - Add comprehensive INFO-level logging for dynamic duration troubleshooting - Enhanced scroll completion logging with position and percentage details This fixes issues where checkbox-group values weren't saving correctly and improves dynamic duration calculation accuracy for scrolling content. --------- Co-authored-by: Chuck <chuck@example.com> |
||
|
ChuckandChuck
|
7d71656cf1 |
Plugins (#145)
Chaotic mega-merge into main. THINGS WILL PROBABLY BE BROKEN
* chore: Update soccer-scoreboard submodule to merged commit
- Update submodule reference to include manifest.json v2 registry format
- Version updated to 1.0.1
* refactor: Remove test_mode and logo_dir config reading from base SportsCore
- Remove test_mode initialization and usage
- Remove logo_dir reading from mode_config
- Use LogoDownloader defaults directly for logo directories
* chore: Update plugin submodules after removing global properties
- Update basketball-scoreboard submodule (removed global test_mode, live_priority, dynamic_duration, logo_dir)
- Update soccer-scoreboard submodule (removed global test_mode, live_priority, dynamic_duration, logo_dir)
* feat(calendar): Add credentials.json file upload via web interface
- Add API endpoint /api/v3/plugins/calendar/upload-credentials for file upload
- Validate JSON format and Google OAuth structure
- Save file to plugin directory with secure permissions (0o600)
- Backup existing credentials.json before overwriting
- Add file upload widget support for string fields in config forms
- Add frontend handler handleCredentialsUpload() for single file uploads
- Update .gitignore to allow calendar submodule
- Update calendar submodule reference
* fix(web): Improve spacing for nested configuration sections
- Add dynamic margin based on nesting depth (mb-6 for deeply nested sections)
- Increase padding in nested content areas (py-3 to py-4)
- Add extra spacing after nested sections to prevent overlap
- Enhance CSS spacing for nested sections (1.5rem for nested, 2rem for deeply nested)
- Add padding-bottom to expanded nested content to prevent cutoff
- Fixes issue where game_limits and other nested settings were hidden under next section header
* chore(plugins): Update sports scoreboard plugins with live update interval fix
- Updated hockey-scoreboard, football-scoreboard, basketball-scoreboard, and soccer-scoreboard submodules
- All plugins now fix the interval selection bug that caused live games to update every 5 minutes instead of 30 seconds
- Ensures all live games update at the configured live_update_interval (30s) for timely score updates
* fix: Initialize test_mode in SportsLive and fix config migration
- Add test_mode initialization in SportsLive.__init__() to prevent AttributeError
- Remove invalid new_secrets parameter from save_config_atomic() call in config migration
- Fixes errors: 'NBALiveManager' object has no attribute 'test_mode'
- Fixes errors: ConfigManager.save_config_atomic() got unexpected keyword argument 'new_secrets'
* chore: Update submodules with test_mode initialization fixes
- Update basketball-scoreboard submodule
- Update soccer-scoreboard submodule
* fix(plugins): Auto-stash local changes before plugin updates
- Automatically stash uncommitted changes before git pull during plugin updates
- Prevents update failures when plugins have local modifications
- Improves error messages for git update failures
- Matches behavior of main LEDMatrix update process
* fix(basketball-scoreboard): Update submodule with timeout fix
- Updated basketball-scoreboard plugin to fix update() timeout issue
- Plugin now uses fire-and-forget odds fetching for upcoming games
- Prevents 30-second timeout when processing many upcoming games
Also fixed permission issue on devpi:
- Changed /var/cache/ledmatrix/display_on_demand_state.json permissions
from 600 to 660 to allow web service (devpi user) to read the file
* fix(cache): Ensure cache files use 660 permissions for group access
- Updated setup_cache.sh to set file permissions to 660 (not 775)
- Updated first_time_install.sh to properly set cache file permissions
- Modified DiskCache to set 660 permissions when creating cache files
- Ensures display_on_demand_state.json and other cache files are readable
by web service (devpi user) which is in ledmatrix group
This fixes permission issues where cache files were created with 600
permissions, preventing the web service from reading them. Now files
are created with 660 (rw-rw----) allowing group read access.
* fix(soccer-scoreboard): Update submodule with manifest fix
- Updated soccer-scoreboard plugin submodule
- Added missing entry_point and class_name to manifest.json
- Fixes plugin loading error: 'No class_name in manifest'
Also fixed cache file permissions on devpi server:
- Changed display_on_demand_state.json from 600 to 660 permissions
- Allows web service (devpi user) to read cache files
* fix(display): Remove update_display() calls from clear() to prevent black flash
Previously, display_manager.clear() was calling update_display() twice,
which immediately showed a black screen on the hardware before new
content could be drawn. This caused visible black flashes when switching
between modes, especially when plugins switch from general modes (e.g.,
football_upcoming) to specific sub-modes (e.g., nfl_upcoming).
Now clear() only prepares the buffer without updating the hardware.
Callers can decide when to update the display, allowing smooth transitions
from clear → draw → update_display() without intermediate black flashes.
Places that intentionally show a cleared screen (error cases) already
explicitly call update_display() after clear(), so backward compatibility
is maintained.
* fix(scroll): Prevent wrap-around before cycle completion in dynamic duration
- Check scroll completion BEFORE allowing wrap-around
- Clamp scroll_position when complete to prevent visual loop
- Only wrap-around if cycle is not complete yet
- Fixes issue where stocks plugin showed first stock again at end
- Completion logged only once to avoid spam
- Ensures smooth transition to next mode without visual repeat
* fix(on-demand): Ensure on-demand buttons work and display service runs correctly
- Add early stub functions for on-demand modal to ensure availability when Alpine.js initializes
- Increase on-demand request cache max_age from 5min to 1hr to prevent premature expiration
- Fixes issue where on-demand buttons were not functional due to timing issues
- Ensures display service properly picks up on-demand requests when started
* test: Add comprehensive test coverage (30%+)
- Add 100+ new tests across core components
- Add tests for LayoutManager (27 tests)
- Add tests for PluginLoader (14 tests)
- Add tests for SchemaManager (20 tests)
- Add tests for MemoryCache and DiskCache (24 tests)
- Add tests for TextHelper (9 tests)
- Expand error handling tests (7 new tests)
- Improve coverage from 25.63% to 30.26%
- All 237 tests passing
Test files added:
- test/test_layout_manager.py
- test/test_plugin_loader.py
- test/test_schema_manager.py
- test/test_text_helper.py
- test/test_config_service.py
- test/test_display_controller.py
- test/test_display_manager.py
- test/test_error_handling.py
- test/test_font_manager.py
- test/test_plugin_system.py
Updated:
- pytest.ini: Enable coverage reporting with 30% threshold
- test/conftest.py: Enhanced fixtures for better test isolation
- test/test_cache_manager.py: Expanded cache component tests
- test/test_config_manager.py: Additional config tests
Documentation:
- HOW_TO_RUN_TESTS.md: Guide for running and understanding tests
* test(web): Add comprehensive API endpoint tests
- Add 30 new tests for Flask API endpoints in test/test_web_api.py
- Cover config, system, display, plugins, fonts, and error handling APIs
- Increase test coverage from 30.26% to 30.87%
- All 267 tests passing
Tests cover:
- Config API: GET/POST main config, schedule, secrets
- System API: Status, version, system actions
- Display API: Current display, on-demand start/stop
- Plugins API: Installed plugins, health, config, operations, state
- Fonts API: Catalog, tokens, overrides
- Error handling: Invalid JSON, missing fields, 404s
* test(plugins): Add comprehensive integration tests for all plugins
- Add base test class for plugin integration tests
- Create integration tests for all 6 plugins:
- basketball-scoreboard (11 tests)
- calendar (10 tests)
- clock-simple (11 tests)
- odds-ticker (9 tests)
- soccer-scoreboard (11 tests)
- text-display (12 tests)
- Total: 64 new plugin integration tests
- Increase test coverage from 30.87% to 33.38%
- All 331 tests passing
Tests verify:
- Plugin loading and instantiation
- Required methods (update, display)
- Manifest validation
- Display modes
- Config schema validation
- Graceful handling of missing API credentials
Uses hybrid approach: integration tests in main repo,
plugin-specific unit tests remain in plugin submodules.
* Add mqtt-notifications plugin as submodule
* fix(sports): Respect games_to_show settings for favorite teams
- Fix upcoming games to show N games per team (not just 1)
- Fix recent games to show N games per team (not just 1)
- Add duplicate removal for games involving multiple favorite teams
- Match behavior of basketball-scoreboard plugin
- Affects NFL, NHL, and other sports using base_classes/sports.py
* chore: Remove debug instrumentation logs
- Remove temporary debug logging added during fix verification
- Fix confirmed working by user
* debug: Add instrumentation to debug configuration header visibility issue
* fix: Resolve nested section content sliding under next header
- Remove overflow-hidden from nested-section to allow proper document flow
- Add proper z-index and positioning to prevent overlap
- Add margin-top to nested sections for better spacing
- Remove debug instrumentation that was causing ERR_BLOCKED_BY_CLIENT errors
* fix: Prevent unnecessary plugin tab redraws
- Add check to only update tabs when plugin list actually changes
- Increase debounce timeout to batch rapid changes
- Compare plugin IDs before updating to avoid redundant redraws
- Fix setter to check for actual changes before triggering updates
* fix: Prevent form-groups from sliding out of view when nested sections expand
- Increase margin-bottom on nested-sections for better spacing
- Add clear: both to nested-sections to ensure proper document flow
- Change overflow to visible when expanded to allow natural flow
- Add margin-bottom to expanded content
- Add spacing rules for form-groups that follow nested sections
- Add clear spacer div after nested sections
* fix: Reduce excessive debug logging in generateConfigForm
- Only log once per plugin instead of on every function call
- Prevents log spam when Alpine.js re-renders the form multiple times
- Reduces console noise from 10+ logs per plugin to 1 log per plugin
* fix: Prevent nested section content from sliding out of view when expanded
- Remove overflow-hidden from nested-section in base.html (was causing clipping)
- Add scrollIntoView to scroll expanded sections into view within modal
- Set nested-section overflow to visible to prevent content clipping
- Add min-height to nested-content to ensure proper rendering
- Wait for animation to complete before scrolling into view
* fix: Prevent form-groups from overlapping and appearing outside view
- Change nested-section overflow to hidden by default, visible when expanded
- Add :has() selector to allow overflow when content is expanded
- Ensure form-groups after nested sections have proper spacing and positioning
- Add clear: both and width: 100% to prevent overlap
- Use !important for margin-top to ensure spacing is applied
- Ensure form-groups are in normal document flow with float: none
* fix: Use JavaScript to toggle overflow instead of :has() selector
- :has() selector may not be supported in all browsers
- Use JavaScript to set overflow: visible when expanded, hidden when collapsed
- This ensures better browser compatibility while maintaining functionality
* fix: Make parent sections expand when nested sections expand
- Add updateParentNestedContentHeight() helper to recursively update parent heights
- When a nested section expands, recalculate all parent nested-content max-heights
- Ensures parent sections (like NFL) expand to accommodate expanded child sections
- Updates parent heights both on expand and collapse for proper animation
* refactor: Simplify parent section expansion using CSS max-height: none
- Remove complex recursive parent height update function
- Use CSS max-height: none when expanded to allow natural expansion
- Parent sections automatically expand because nested-content has no height constraint
- Simpler and more maintainable solution
* refactor: Remove complex recursive parent height update function
- CSS max-height: none already handles parent expansion automatically
- No need for JavaScript to manually update parent heights
- Much simpler and cleaner solution
* debug: Add instrumentation to debug auto-collapse issue
- Add logging to track toggle calls and state changes
- Add guard to prevent multiple simultaneous toggles
- Pass event object to prevent bubbling
- Improve state detection logic
- Add return false to onclick handlers
* chore: Remove debug instrumentation from toggleNestedSection
- Remove all debug logging code
- Keep functional fixes: event handling, toggle guard, improved state detection
- Code is now clean and production-ready
* fix(web): Add browser refresh note to plugin fetch errors
* refactor(text-display): Update submodule to use ScrollHelper
* fix(text-display): Fix scrolling display issue - update position in display()
* feat(text-display): Add scroll_loop option and improve scroll speed control
* debug: Add instrumentation to track plugin enabled state changes
Added debug logging to investigate why plugins appear to disable themselves:
- Track enabled state during plugin load (before/after schema merge)
- Track enabled state during plugin reload
- Track enabled state preservation during config save
- Track state reconciliation fixes
- Track enabled state updates in on_config_change
This will help identify which code path is causing plugins to disable.
* debug: Fix debug log path to work on Pi
Changed hardcoded log path to use dynamic project root detection:
- Uses LEDMATRIX_ROOT env var if set
- Falls back to detecting project root by looking for config directory
- Creates .cursor directory if it doesn't exist
- Falls back to /tmp/ledmatrix_debug.log if all else fails
- Added better error handling with logger fallback
* Remove debug instrumentation for plugin enabled state tracking
Removed all debug logging that was added to track plugin enabled state changes.
The instrumentation has been removed as requested.
* Reorganize documentation and cleanup test files
- Move documentation files to docs/ directory
- Remove obsolete test files
- Update .gitignore and README
* feat(text-display): Switch to frame-based scrolling with high FPS support
* fix(text-display): Add backward compatibility for ScrollHelper sub-pixel scrolling
* feat(scroll_helper): Add sub-pixel scrolling support for smooth movement
- Add sub-pixel interpolation using scipy (if available) or numpy fallback
- Add set_sub_pixel_scrolling() method to enable/disable feature
- Implement _get_visible_portion_subpixel() for fractional pixel positioning
- Implement _interpolate_subpixel() for linear interpolation
- Prevents pixel skipping at slow scroll speeds
- Maintains backward compatibility with integer pixel path
* fix(scroll_helper): Reset last_update_time in reset_scroll() to prevent jump-ahead
- Reset last_update_time when resetting scroll position
- Prevents large delta_time on next update after reset
- Fixes issue where scroll would immediately complete again after reset
- Ensures smooth scrolling continuation after loop reset
* fix(scroll_helper): Fix numpy broadcasting error in sub-pixel interpolation
- Add output_width parameter to _interpolate_subpixel() for variable widths
- Fix wrap-around case to use correct widths for interpolation
- Handle edge cases where source array is smaller than expected
- Prevent 'could not broadcast input array' errors in sub-pixel scrolling
- Ensure proper width matching in all interpolation paths
* feat(scroll): Add frame-based scrolling mode for smooth LED matrix movement
- Add frame_based_scrolling flag to ScrollHelper
- When enabled, moves fixed pixels per step, throttled by scroll_delay
- Eliminates time-based jitter by ignoring frame timing variations
- Provides stock-ticker-like smooth, predictable scrolling
- Update text-display plugin to use frame-based mode
This addresses stuttering issues where time-based scrolling caused
visual jitter due to frame timing variations in the main display loop.
* fix(scroll): Fix NumPy broadcasting errors in sub-pixel wrap-around
- Ensure _interpolate_subpixel always returns exactly requested width
- Handle cases where scipy.ndimage.shift produces smaller arrays
- Add padding logic for wrap-around cases when arrays are smaller than expected
- Prevents 'could not broadcast input array' errors during scrolling
* refactor(scroll): Remove sub-pixel interpolation, use high FPS integer scrolling
- Disable sub-pixel scrolling by default in ScrollHelper
- Simplify get_visible_portion to always use integer pixel positioning
- Restore frame-based scrolling logic for smooth high FPS movement
- Use high frame rate (like stock ticker) for smoothness instead of interpolation
- Reduces complexity and eliminates broadcasting errors
* fix(scroll): Prevent large pixel jumps in frame-based scrolling
- Initialize last_step_time properly to prevent huge initial jumps
- Clamp scroll_speed to max 5 pixels/frame in frame-based mode
- Prevents 60-pixel jumps when scroll_speed is misconfigured
- Simplified step calculation to avoid lag catch-up jumps
* fix(text-display): Align config schema and add validation
- Update submodule reference
- Adds warning and logging for scroll_speed config issues
* fix(scroll): Simplify frame-based scrolling to match stock ticker behavior
- Remove throttling logic from frame-based scrolling
- Move pixels every call (DisplayController's loop timing controls rate)
- Add enable_scrolling attribute to text-display plugin for high-FPS treatment
- Matches stock ticker: simple, predictable movement every frame
- Eliminates jitter from timing mismatches between DisplayController and ScrollHelper
* fix(scroll): Restore scroll_delay throttling in frame-based mode
- Restore time-based throttling using scroll_delay
- Move pixels only when scroll_delay has passed
- Handle lag catch-up with reasonable caps to prevent huge jumps
- Preserve fractional timing for smooth operation
- Now scroll_delay actually controls the scroll speed as intended
* feat(text-display): Add FPS counter logging
- Update submodule reference
- Adds FPS tracking and logging every 5 seconds
* fix(text-display): Add display-width buffer so text scrolls completely off
- Update submodule reference
- Adds end buffer to ensure text exits viewport before looping
* fix: Prevent premature game switching in SportsLive
- Set last_game_switch when games load even if current_game already exists
- Set last_game_switch when same games update but it's still 0
- Add guard to prevent switching check when last_game_switch is 0
- Fixes issue where first game shows for only ~2 seconds before switching
- Also fixes random screen flickering when games change prematurely
* feat(plugins): Add branch selection support for plugin installation
- Add optional branch parameter to install_plugin() and install_from_url() in store_manager
- Update API endpoints to accept and pass branch parameter
- Update frontend JavaScript to support branch selection in install calls
- Maintain backward compatibility - branch parameter is optional everywhere
- Falls back to default branch logic if specified branch doesn't exist
* feat(plugins): Add UI for branch selection in plugin installation
- Add branch input field in 'Install Single Plugin' section
- Add global branch input for store installations
- Update JavaScript to read branch from input fields
- Branch input applies to all store installations when specified
* feat(plugins): Change branch selection to be per-plugin instead of global
- Remove global store branch input field
- Add individual branch input field to each plugin card in store
- Add branch input to custom registry plugin cards
- Each plugin can now have its own branch specified independently
* debug: Add logging to _should_exit_dynamic
* feat(display_controller): Add universal get_cycle_duration support for all plugins
UNIVERSAL FEATURE: Any plugin can now implement get_cycle_duration() to dynamically
calculate the total time needed to show all content for a mode.
New method:
- _plugin_cycle_duration(plugin, display_mode): Queries plugin for calculated duration
Integration:
- Display controller calls plugin.get_cycle_duration(display_mode)
- Uses returned duration as target (respecting max cap)
- Falls back to cap if not provided
Benefits:
- Football plugin: Show all games (3 games × 15s = 45s total)
- Basketball plugin: Could implement same logic
- Hockey/Baseball/any sport: Universal support
- Stock ticker: Could calculate based on number of stocks
- Weather: Could calculate based on forecast days
Example plugin implementation:
Result: Plugins control their own display duration based on actual content,
creating a smooth user experience where all content is shown before switching.
* debug: Add logging to cycle duration call
* debug: Change loop exit logs to INFO level
* fix: Change cycle duration logs to INFO level
* fix: Don't exit loop on False for dynamic duration plugins
For plugins with dynamic duration enabled, keep the display loop running
even when display() returns False. This allows games to continue rotating
within the calculated duration.
The loop will only exit when:
- Cycle is complete (plugin reports all content shown)
- Max duration is reached
- Mode is changed externally
* fix(schedule): Improve display scheduling functionality
- Add GET endpoint for schedule configuration retrieval
- Fix mode switching to clean up old config keys (days/start_time/end_time)
- Improve error handling with consistent error_response() usage
- Enhance display controller schedule checking with better edge case handling
- Add validation for time formats and ensure at least one day enabled in per-day mode
- Add debug logging for schedule state changes
Fixes issues where schedule mode switching left stale config causing incorrect behavior.
* fix(install): Add cmake and ninja-build to system dependencies
Resolves h3 package build failure during first-time installation.
The h3 package (dependency of timezonefinder) requires CMake and
Ninja to build from source. Adding these build tools ensures
successful installation of all Python dependencies.
* fix: Pass display_mode in ALL loop calls to maintain sticky manager
CRITICAL FIX: Display controller was only passing display_mode on first call,
causing plugins to fall back to internal mode cycling and bypass sticky
manager logic.
Now consistently passes display_mode=active_mode on every display() call in
both high-FPS and normal loops. This ensures plugins maintain mode context
and sticky manager state throughout the entire display duration.
* feat(install): Add OS check for Raspberry Pi OS Lite (Trixie)
- Verify OS is Raspberry Pi OS (raspbian/debian)
- Require Debian 13 (Trixie) specifically
- Check for Lite version (no desktop environment)
- Exit with clear error message if requirements not met
- Provide instructions for obtaining correct OS version
* fix(web-ui): Add missing notification handlers to quick action buttons
- Added hx-on:htmx:after-request handlers to all quick action buttons in overview.html
- Added hx-ext='json-enc' for proper JSON encoding
- Added missing notification handler for reboot button in index.html
- Users will now see toast notifications when actions complete or fail
* fix(display): Ensure consistent display mode handling in all plugin calls
- Updated display controller to consistently pass display_mode in all plugin display() calls.
- This change maintains the sticky manager state and ensures plugins retain their mode context throughout the display duration.
- Addresses issues with mode cycling and improves overall display reliability.
* fix(display): Enhance display mode persistence across plugin updates
- Updated display controller to ensure display_mode is consistently maintained during plugin updates.
- This change prevents unintended mode resets and improves the reliability of display transitions.
- Addresses issues with mode persistence, ensuring a smoother user experience across all plugins.
* feat: Add Olympics countdown plugin as submodule
- Add olympics-countdown plugin submodule
- Update .gitignore to allow olympics-countdown plugin
- Plugin automatically determines next Olympics and counts down to opening/closing ceremonies
* feat(web-ui): Add checkbox-group widget support for multi-select arrays
- Add checkbox-group widget rendering in plugins_manager.js
- Update form processing to handle checkbox groups with [] naming
- Support for friendly labels via x-options in config schemas
- Update odds-ticker submodule with checkbox-group implementation
* fix(plugins): Preserve enabled state when saving plugin config from main config endpoint
When saving plugin configuration through save_main_config endpoint, the enabled
field was not preserved if missing from the form data. This caused plugins to
be automatically disabled when users saved their configuration from the plugin
manager tab.
This fix adds the same enabled state preservation logic that exists in
save_plugin_config endpoint, ensuring consistent behavior across both endpoints.
The enabled state is preserved from current config, plugin instance, or defaults
to True to prevent unexpected disabling of plugins.
* fix(git): Resolve git status timeout and exclude plugins from base project updates
- Add --untracked-files=no flag to git status for faster execution
- Increase timeout from 5s to 30s for git status operations
- Add timeout exception handling for git status and stash operations
- Filter out plugins directory from git status checks (plugins are separate repos)
- Exclude plugins from stash operations using :!plugins pathspec
- Apply same fixes to plugin store manager update operations
* feat(plugins): Add granular scroll speed control to odds-ticker and leaderboard plugins
- Add display object to both plugins' config schemas with scroll_speed and scroll_delay
- Enable frame-based scrolling mode for precise FPS control (100 FPS for leaderboard)
- Add set_scroll_speed() and set_scroll_delay() methods to both plugins
- Maintain backward compatibility with scroll_pixels_per_second config
- Leaderboard plugin now explicitly sets target_fps to 100 for high-performance scrolling
* fix(scroll): Correct dynamic duration calculation for frame-based scrolling
- Fix calculate_dynamic_duration() to properly handle frame-based scrolling mode
- Convert scroll_speed from pixels/frame to pixels/second when in frame-based mode
- Prevents incorrect duration calculations (e.g., 2609s instead of 52s)
- Affects all plugins using ScrollHelper: odds-ticker, leaderboard, stocks, text-display
- Add debug logging to show scroll mode and effective speed
* Remove version logic from plugin system, use git commits instead
- Remove version parameter from install_plugin() method
- Rename fetch_latest_versions to fetch_commit_info throughout codebase
- Remove version fields from plugins.json registry (versions, latest_version, download_url_template)
- Remove version logging from plugin manager
- Update web UI to use fetch_commit_info parameter
- Update .gitignore to ignore all plugin folders (remove whitelist exceptions)
- Remove plugin directories from git index (plugins now installed via plugin store only)
Plugins now always install latest commit from default branch. Version fields
replaced with git commit SHA and commit dates. System uses git-based approach
for all plugin metadata.
* feat(plugins): Normalize all plugins as git submodules
- Convert all 18 plugins to git submodules for uniform management
- Add submodules for: baseball-scoreboard, christmas-countdown, football-scoreboard, hockey-scoreboard, ledmatrix-flights, ledmatrix-leaderboard, ledmatrix-music, ledmatrix-stocks, ledmatrix-weather, static-image
- Re-initialize mqtt-notifications as proper submodule
- Update .gitignore to allow all plugin submodules
- Add normalize_plugin_submodules.sh script for future plugin management
All plugins with GitHub repositories are now managed as git submodules,
ensuring consistent version control and easier updates.
* refactor(repository): Reorganize scripts and files into organized directory structure
- Move installation scripts to scripts/install/ (except first_time_install.sh)
- Move development scripts to scripts/dev/
- Move utility scripts to scripts/utils/
- Move systemd service files to systemd/
- Keep first_time_install.sh, start_display.sh, stop_display.sh in root
- Update all path references in scripts, documentation, and service files
- Add README.md files to new directories explaining their purpose
- Remove empty tools/ directory (contents moved to scripts/dev/)
- Add .gitkeep to data/ directory
* fix(scripts): Fix PROJECT_DIR path in start_web_conditionally.py after move to scripts/utils/
* fix(scripts): Fix PROJECT_DIR/PROJECT_ROOT path resolution in moved scripts
- Fix wifi_monitor_daemon.py to use project root instead of scripts/utils/
- Fix shell scripts in scripts/ to correctly resolve project root (go up one more level)
- Fix scripts in scripts/fix_perms/ to correctly resolve project root
- Update diagnose_web_interface.sh to reference moved start_web_conditionally.py path
All scripts now correctly determine project root after reorganization.
* fix(install): Update first_time_install.sh to detect and update service files with old paths
- Check for old paths in service files and reinstall if needed
- Always reinstall main service (install_service.sh is idempotent)
- This ensures existing installations get updated paths after reorganization
* fix(install): Update install_service.sh message to indicate it updates existing services
* fix(wifi): Enable WiFi scan to work when AP mode is active
- Temporarily disable AP mode during network scanning
- Automatically re-enable AP mode after scan completes
- Add proper error handling with try/finally to ensure AP mode restoration
- Add user notification when AP mode is temporarily disabled
- Improve error messages for common scanning failures
- Add timing delays for interface mode switching
* fix(wifi): Fix network parsing to handle frequency with 'MHz' suffix
- Strip 'MHz' suffix from frequency field before float conversion
- Add better error logging for parsing failures
- Fixes issue where all networks were silently skipped due to ValueError
* debug(wifi): Add console logging and Alpine.js reactivity fixes for network display
- Add console.log statements to debug network scanning
- Add x-effect to force Alpine.js reactivity updates
- Add unique keys to x-for template
- Add debug display showing network count
- Improve error handling and user feedback
* fix(wifi): Manually update select options instead of using Alpine.js x-for
- Replace Alpine.js x-for template with manual DOM manipulation
- Add updateSelectOptions() method to directly update select dropdown
- This fixes issue where networks weren't appearing in dropdown
- Alpine.js x-for inside select elements can be unreliable
* feat(web-ui): Add patternProperties support for dynamic key-value pairs
- Add UI support for patternProperties objects (custom_feeds, feed_logo_map)
- Implement key-value pair editor with add/remove functionality
- Add JavaScript functions for managing dynamic key-value pairs
- Update form submission to handle patternProperties JSON data
- Enable easy configuration of feed_logo_map in web UI
* chore: Update ledmatrix-news submodule to latest commit
* fix(plugins): Handle arrays of objects in config normalization
Fix configuration validation failure for static-image plugin by adding
recursive normalization support for arrays of objects. The normalize_config_values
function now properly handles arrays containing objects (like image_config.images)
by recursively normalizing each object in the array using the items schema properties.
This resolves the 'configuration validation failed' error when saving static
image plugin configuration with multiple images.
* fix(plugins): Handle union types in config normalization and form generation
Fix configuration validation for fields with union types like ['integer', 'null'].
The normalization function now properly handles:
- Union types in top-level fields (e.g., random_seed: ['integer', 'null'])
- Union types in array items
- Empty string to None conversion for nullable fields
- Form generation and submission for union types
This resolves validation errors when saving plugin configs with nullable
integer/number fields (e.g., rotation_settings.random_seed in static-image plugin).
Also improves UX by:
- Adding placeholder text for nullable fields explaining empty = use default
- Properly handling empty values in form submission for union types
* fix(plugins): Improve union type normalization with better edge case handling
Enhanced normalization for union types like ['integer', 'null']:
- Better handling of whitespace in string values
- More robust empty string to None conversion
- Fallback to None when conversion fails and null is allowed
- Added debug logging for troubleshooting normalization issues
- Improved handling of nested object fields with union types
This should resolve remaining validation errors for nullable integer/number
fields in nested objects (e.g., rotation_settings.random_seed).
* chore: Add ledmatrix-news plugin to .gitignore exceptions
* Fix web interface service script path in install_service.sh
- Updated ExecStart path from start_web_conditionally.py to scripts/utils/start_web_conditionally.py
- Updated diagnose_web_ui.sh to check for correct script path
- Fixes issue where web UI service fails to start due to incorrect script path
* Fix nested configuration section headers not expanding
Fixed toggleNestedSection function to properly calculate scrollHeight when
expanding nested configuration sections. The issue occurred when sections
started with display:none - the scrollHeight was being measured before the
browser had a chance to lay out the element, resulting in a value of 0.
Changes:
- Added setTimeout to delay scrollHeight measurement until after layout
- Added overflow handling during animations to prevent content jumping
- Added fallback for edge cases where scrollHeight might still be 0
- Set maxHeight to 'none' after expansion completes for natural growth
- Updated function in both base.html and plugins_manager.js
This fix applies to all plugins with nested configuration sections, including:
- Hockey/Football/Basketball/Baseball/Soccer scoreboards (customization, global sections)
- All plugins with transition, display, and other nested configuration objects
Fixes configuration header expansion issues across all plugins.
* Fix syntax error in first_time_install.sh step 8.5
Added missing 'fi' statement to close the if block in the WiFi monitor
service installation section. This resolves the 'unexpected end of file'
error that occurred at line 1385 during step 8.5.
* Fix WiFi UI: Display correct SSID and accurate signal strength
- Fix WiFi network selection dropdown not showing available networks
- Replace manual DOM manipulation with Alpine.js x-for directive
- Add fallback watcher to ensure select updates reactively
- Fix WiFi status display showing netplan connection name instead of SSID
- Query actual SSID from device properties (802-11-wireless.ssid)
- Add fallback methods to get SSID from active WiFi connection list
- Improve signal strength accuracy
- Get signal directly from device properties (WIFI.SIGNAL)
- Add multiple fallback methods for robust signal retrieval
- Ensure signal percentage is accurate and up-to-date
* Improve WiFi connection UI and error handling
- Fix connect button disabled condition to check both selectedSSID and manualSSID
- Improve error handling to display actual server error messages from 400 responses
- Add step-by-step labels (Step 1, Step 2, Step 3) to clarify connection workflow
- Add visual feedback showing selected network in blue highlight box
- Improve password field labeling with helpful instructions
- Add auto-clear logic between dropdown and manual SSID entry
- Enhance backend validation with better error messages and logging
- Trim SSID whitespace before processing to prevent validation errors
* Add WiFi disconnect functionality for AP mode testing
- Add disconnect_from_network() method to WiFiManager
- Disconnects from current WiFi network using nmcli
- Automatically triggers AP mode check if auto_enable_ap_mode is enabled
- Returns success/error status with descriptive messages
- Add /api/v3/wifi/disconnect API endpoint
- POST endpoint to disconnect from current WiFi network
- Includes proper error handling and logging
- Add disconnect button to WiFi status section
- Only visible when connected to a network
- Red styling to indicate disconnection action
- Shows 'Disconnecting...' state during operation
- Automatically refreshes status after disconnect
- Integrates with AP mode auto-enable functionality
- When disconnected, automatically enables AP mode if configured
- Perfect for testing captive portal and AP mode features
* Add explicit handling for broken pipe errors during plugin dependency installation
- Catch BrokenPipeError and OSError (errno 32) explicitly in all dependency installation methods
- Add clear error messages explaining network interruption or buffer overflow causes
- Improves error handling in store_manager, plugin_loader, and plugin_manager
- Helps diagnose 'Errno 32 Broken Pipe' errors during pip install operations
* Add WiFi permissions configuration script and integrate into first-time install
- Create configure_wifi_permissions.sh script
- Configures passwordless sudo for nmcli commands
- Configures PolicyKit rules for NetworkManager control
- Fixes 'Not Authorized to control Networking' error
- Allows web interface to connect/disconnect WiFi without password prompts
- Integrate WiFi permissions configuration into first_time_install.sh
- Added as Step 10.1 after passwordless sudo configuration
- Runs automatically during first-time installation
- Ensures WiFi management works out of the box
- Resolves authorization errors when connecting/disconnecting WiFi networks
- NetworkManager requires both sudo and PolicyKit permissions
- Script configures both automatically for seamless WiFi management
* Add WiFi status LED message display integration
- Integrate WiFi status messages from wifi_manager into display_controller
- WiFi status messages interrupt normal rotation (but respect on-demand)
- Priority: on-demand > wifi-status > live-priority > normal rotation
- Safe implementation with comprehensive error handling
- Automatic cleanup of expired/corrupted status files
- Word-wrapping for long messages (max 2 lines)
- Centered text display with small font
- Non-intrusive: all errors are caught and logged, never crash controller
* Fix display loop issues: reduce log spam and handle missing plugins
- Change _should_exit_dynamic logging from INFO to DEBUG to reduce log spam
in tight loops (every 8ms) that was causing high CPU usage
- Fix display loop not running when manager_to_display is None
- Add explicit check to set display_result=False when no plugin manager found
- Fix logic bug where manager_to_display was overwritten after circuit breaker skip
- Ensure proper mode rotation when plugins have no content or aren't found
* Add debug logging to diagnose display loop stuck issue
* Change debug logs to INFO level to diagnose display loop stuck
* Add schedule activation logging and ensure display is blanked when inactive
- Add clear INFO-level log message when schedule makes display inactive
- Track previous display state to detect schedule transitions
- Clear display when schedule makes it inactive to ensure blank screen
(prevents showing initialization screen when schedule kicks in)
- Initialize _was_display_active state tracking in __init__
* Fix indentation errors in schedule state tracking
* Add rotation between hostname and IP address every 10 seconds
- Added _get_local_ip() method to detect device IP address
- Implemented automatic rotation between hostname and IP every 10 seconds
- Enhanced logging to include both hostname and IP in initialization
- Updated get_info() to expose device_ip and current_display_mode
* Add WiFi connection failsafe system
- Save original connection before attempting new connection
- Automatically restore original connection if new connection fails
- Enable AP mode as last resort if restoration fails
- Enhanced connection verification with multiple attempts
- Verify correct SSID (not just 'connected' status)
- Better error handling and exception recovery
- Prevents Pi from becoming unresponsive on connection failure
- Always ensures device remains accessible via original WiFi or AP mode
* feat(web): Improve web UI startup speed and fix cache permissions
- Defer plugin discovery until first API request (removed from startup)
- Add lazy loading to operation queue, state manager, and operation history
- Defer health monitor initialization until first request
- Fix cache directory permission issue:
- Add systemd CacheDirectory feature for automatic cache dir creation
- Add manual cache directory creation in install script as fallback
- Improve cache manager logging (reduce alarming warnings)
- Fix syntax errors in wifi_manager.py (unclosed try blocks)
These changes significantly improve web UI startup time, especially with many
plugins installed, while maintaining full backward compatibility.
* feat(plugins): Improve GitHub token pop-up UX and combine warning/settings
- Fix visibility toggle to handle inline styles properly
- Remove redundant inline styles from HTML elements
- Combine warning banner and settings panel into unified component
- Add loading states to save/load token buttons
- Improve error handling with better user feedback
- Add token format validation (ghp_ or github_pat_ prefix)
- Auto-refresh GitHub auth status after saving token
- Hide warning banner when settings panel opens
- Clear input field after successful save for security
This creates a smoother UX flow where clicking 'Configure Token'
transitions from warning directly to configuration form.
* fix(wifi): Prevent WiFi radio disabling during AP mode disable
- Make NetworkManager restart conditional (only for hostapd mode)
- Add enhanced WiFi radio enable with retry and verification logic
- Add connectivity safety check before NetworkManager restart
- Ensure WiFi radio enabled after all AP mode disable operations
- Fix indentation bug in dnsmasq backup restoration logic
- Add pre-connection WiFi radio check for safety
Fixes issue where WiFi radio was being disabled when disabling AP mode,
especially when connected via Ethernet, making it impossible to enable
WiFi from the web UI.
* fix(plugin-templates): Fix unreachable fallback to expired cache in update() method
The exception handler in update() checked the cached variable, which would
always be None or falsy at that point. If fresh cached data existed, the
method returned early. If cached data was expired, it was filtered out by
max_age constraint. The fix retrieves cached data again in the exception
handler with a very large max_age (1 year) to effectively bypass expiration
check and allow fallback to expired data when fetch fails.
* fix(plugin-templates): Resolve plugin_id mismatch in test template setUp method
* feat(plugins): Standardize manifest version fields schema
- Consolidate version fields to use consistent naming:
- compatible_versions: array of semver ranges (required)
- min_ledmatrix_version: string (optional)
- max_ledmatrix_version: string (optional)
- versions[].ledmatrix_min_version: renamed from ledmatrix_min
- Add manifest schema validation (schema/manifest_schema.json)
- Update store_manager to validate version fields and schema
- Update template and all documentation examples to use standardized fields
- Add deprecation warnings for ledmatrix_version and ledmatrix_min fields
* fix(templates): Update plugin README template script path to correct location
* docs(plugin): Resolve conflicting version management guidance in .cursorrules
* chore(.gitignore): Consolidate plugin exclusion patterns
Remove unnecessary !plugins/*/.git pattern and consolidate duplicate
negations by keeping only trailing-slash directory exclusions.
* docs: Add language specifiers to code blocks in STATIC_IMAGE_MULTI_UPLOAD_PLAN.md
* fix(templates): Remove api_key from config.json example in plugin README template
Remove api_key field from config.json example to prevent credential leakage.
API keys should only be stored in config_secrets.json. Added clarifying note
about proper credential storage.
* docs(README): Add plugin installation and migration information
- Add plugin installation instructions via web interface and GitHub URL
- Add plugin migration guide for users upgrading from old managers
- Improve plugin documentation for new users
* docs(readme): Update donation links and add Discord acknowledgment
* docs: Add comprehensive API references and consolidate documentation
- Add API_REFERENCE.md with complete REST API documentation (50+ endpoints)
- Add PLUGIN_API_REFERENCE.md documenting Display Manager, Cache Manager, and Plugin Manager APIs
- Add ADVANCED_PLUGIN_DEVELOPMENT.md with advanced patterns and examples
- Add DEVELOPER_QUICK_REFERENCE.md for quick developer reference
- Consolidate plugin configuration docs into single PLUGIN_CONFIGURATION_GUIDE.md
- Archive completed implementation summaries to docs/archive/
- Enhance PLUGIN_DEVELOPMENT_GUIDE.md with API links and 3rd party submission guidelines
- Update docs/README.md with new API reference sections
- Update root README.md with documentation links
* fix(install): Fix IP detection and network diagnostics after fresh install
- Fix web-ui-info plugin IP detection to handle no internet, AP mode, and network state changes
- Replace socket-based detection with robust interface scanning using hostname -I and ip addr
- Add AP mode detection returning 192.168.4.1 when AP mode is active
- Add periodic IP refresh every 30 seconds to handle network state changes
- Improve network diagnostics in first_time_install.sh showing actual IPs, WiFi status, and AP mode
- Add WiFi connection check in WiFi monitor installation with warnings
- Enhance web service startup logging to show accessible IP addresses
- Update README with network troubleshooting section and fix port references (5001->5000)
Fixes issue where display showed incorrect IP (127.0.11:5000) and users couldn't access web UI after fresh install.
* chore: Add GitHub sponsor button configuration
* fix(wifi): Fix aggressive AP mode enabling and improve WiFi detection
Critical fixes:
- Change auto_enable_ap_mode default from True to False (manual enable only)
- Fixes issue where Pi would disconnect from network after code updates
- Matches documented behavior (was incorrectly defaulting to True in code)
Improvements:
- Add grace period: require 3 consecutive disconnected checks (90s) before enabling AP mode
- Prevents AP mode from enabling on transient network hiccups
- Improve WiFi status detection with retry logic and better nmcli parsing
- Enhanced logging for debugging WiFi connection issues
- Better handling of WiFi device detection (works with any wlan device)
This prevents the WiFi monitor from aggressively enabling AP mode and
disconnecting the Pi from the network when there are brief network issues
or during system initialization.
* fix(wifi): Revert auto_enable_ap_mode default to True with grace period protection
Change default back to True for auto_enable_ap_mode while keeping the grace
period protection that prevents interrupting valid WiFi connections.
- Default auto_enable_ap_mode back to True (useful for setup scenarios)
- Grace period (3 consecutive checks = 90s) prevents false positives
- Improved WiFi detection with retry logic ensures accurate status
- AP mode will auto-enable when truly disconnected, but won't interrupt
valid connections due to transient detection issues
* fix(news): Update submodule reference for manifest fix
Update ledmatrix-news submodule to include the fixed manifest.json with
required entry_point and class_name fields.
* fix(news): Update submodule reference with validate_config addition
Update ledmatrix-news submodule to include validate_config method for
proper configuration validation.
* feat: Add of-the-day plugin as git submodule
- Add ledmatrix-of-the-day plugin as git submodule
- Rename submodule path from plugins/of-the-day to plugins/ledmatrix-of-the-day to match repository naming convention
- Update .gitignore to allow ledmatrix-of-the-day submodule
- Plugin includes fixes for display rendering and web UI configuration support
* fix(wifi): Make AP mode open network and fix WiFi page loading in AP mode
AP Mode Changes:
- Remove password requirement from AP mode (open network for easier setup)
- Update hostapd config to create open network (no WPA/WPA2)
- Update nmcli hotspot to create open network (no password parameter)
WiFi Page Loading Fixes:
- Download local copies of HTMX and Alpine.js libraries
- Auto-detect AP mode (192.168.4.x) and use local JS files instead of CDN
- Auto-open WiFi tab when accessing via AP mode IP
- Add fallback loading if HTMX fails to load
- Ensures WiFi setup page works in AP mode without internet access
This fixes the issue where the WiFi page wouldn't load on iPhone when
accessing via AP mode (192.168.4.1:5000) because CDN resources couldn't
be fetched without internet connectivity.
* feat(wifi): Add explicit network switching support with clean disconnection
WiFi Manager Improvements:
- Explicitly disconnect from current network before connecting to a new one
- Add skip_ap_check parameter to disconnect_from_network() to prevent AP mode
from activating during network switches
- Check if already connected to target network to avoid unnecessary work
- Improved logging for network switching operations
Web UI Improvements:
- Detect and display network switching status in UI
- Show 'Switching from [old] to [new]...' message when switching networks
- Enhanced status reloading after connection (multiple checks at 2s, 5s, 10s)
- Better user feedback during network transitions
This ensures clean network switching without AP mode interruptions and
provides clear feedback to users when changing WiFi networks.
* fix(web-ui): Add fallback content loading when HTMX fails to load
Problem:
- After recent updates, web UI showed navigation and CPU status but main
content tabs never loaded
- Content tabs depend on HTMX's 'revealed' trigger to load
- If HTMX failed to load or initialize, content would never appear
Solutions:
- Enhanced HTMX loading verification with timeout checks
- Added fallback direct fetch for overview tab if HTMX fails
- Added automatic tab content loading when tabs change
- Added loadTabContent() method to manually trigger content loading
- Added global 'htmx-load-failed' event for error handling
- Automatic retry after 5 seconds if HTMX isn't available
- Better error messages and console logging for debugging
This ensures the web UI loads content even if HTMX has issues,
providing graceful degradation and better user experience.
* feat(web-ui): Add support for plugin custom HTML widgets and static file serving
- Add x-widget: custom-html support in config schema generation
- Add loadCustomHtmlWidget() function to load HTML from plugin directories
- Add /api/v3/plugins/<plugin_id>/static/<file_path> endpoint for serving plugin static files
- Enhance execute_plugin_action() to pass params via stdin as JSON for scripts
- Add JSON output parsing for script action responses
These changes enable plugins to provide custom UI components while keeping
all functionality plugin-scoped. Used by of-the-day plugin for file management.
* fix(web-ui): Resolve Alpine.js initialization errors
- Prevent Alpine.js from auto-initializing before app() function is defined
- Add deferLoadingAlpine to ensure proper initialization order
- Make app() function globally available via window.app
- Fix 'app is not defined' and 'activeTab is not defined' errors
- Remove duplicate Alpine.start() calls that caused double initialization warnings
* fix(web-ui): Fix IndentationError in api_v3.py OAuth flow
- Fix indentation in if action_def.get('oauth_flow') block
- Properly indent try/except block and all nested code
- Resolves IndentationError that prevented web interface from starting
* fix(web-ui): Fix SyntaxError in api_v3.py else block
- Fix indentation of OAuth flow code inside else block
- Properly indent else block for simple script execution
- Resolves SyntaxError at line 3458 that prevented web interface from starting
* fix(web-ui): Restructure OAuth flow check to fix SyntaxError
- Move OAuth flow check before script execution in else block
- Remove unreachable code that was causing syntax error
- OAuth check now happens first, then falls back to script execution
- Resolves SyntaxError at line 3458
* fix(web-ui): Define app() function in head for Alpine.js initialization
- Define minimal app() function in head before Alpine.js loads
- Ensures app() is available when Alpine initializes
- Full implementation in body enhances/replaces the stub
- Fixes 'app is not defined' and 'activeTab is not defined' errors
* fix(web-ui): Ensure plugin tabs load when full app() implementation is available
- Update stub init() to detect and use full implementation when available
- Ensure full implementation properly replaces stub methods
- Call init() after merging to load plugins and set up watchers
- Fixes issue where installed plugins weren't showing in navigation bar
* fix(web-ui): Prevent 'Cannot redefine property' error for installedPlugins
- Check if window.installedPlugins property already exists before defining
- Make property configurable to allow redefinition if needed
- Add _initialized flag to prevent multiple init() calls
- Fixes TypeError when stub tries to enhance with full implementation
* fix(web-ui): Fix variable redeclaration errors in logs tab
- Replace let/const declarations with window properties to avoid redeclaration
- Use window._logsEventSource, window._allLogs, etc. to persist across HTMX reloads
- Clean up existing event source before reinitializing
- Remove and re-add event listeners to prevent duplicates
- Fixes 'Identifier has already been declared' error when accessing logs tab multiple times
* feat(web-ui): Add support for additionalProperties object rendering
- Add handler for objects with additionalProperties containing object schemas
- Render dynamic category controls with enable/disable toggles
- Display category metadata (display name, data file path)
- Used by of-the-day plugin for category management
* fix(wifi): Ensure AP mode hotspot is always open (no password)
Problem:
- LEDMatrix-Setup WiFi AP was still asking for password despite code changes
- Existing hotspot connections with passwords weren't being fully cleaned up
- NetworkManager might reuse old connection profiles with passwords
Solutions:
- More thorough cleanup: Delete all hotspot-related connections, not just known names
- Verification: Check if hotspot has password after creation
- Automatic fix: Remove password and restart connection if security is detected
- Better logging: Log when password is detected and removed
This ensures the AP mode hotspot is always open for easy setup access,
even if there were previously saved connections with passwords.
* fix(wifi): Improve network switching reliability and device state handling
Problem:
- Pi failing to switch WiFi networks via web UI
- Connection attempts happening before device is ready
- Disconnect not fully completing before new connection attempt
- Connection name lookup issues when SSID doesn't match connection name
Solutions:
- Improved disconnect logic: Disconnect specific connection first, then device
- Device state verification: Wait for device to be ready (disconnected/unavailable) before connecting
- Better connection lookup: Search by SSID, not just connection name
- Increased wait times: 2 seconds for disconnect to complete
- State checking before activating existing connections
- Enhanced error handling and logging throughout
This ensures network switching works reliably by properly managing device
state transitions and using correct connection identifiers.
* debug(web-ui): Add debug logging for custom HTML widget loading
- Add console logging to track widget generation
- Improve error messages with missing configuration details
- Help diagnose why file manager widget may not be appearing
* fix(web-ui): Fix [object Object] display in categories field
- Add type checking to ensure category values are strings before rendering
- Safely extract data_file and display_name properties
- Prevent object coercion issues in category display
* perf(web-ui): Optimize plugin loading in navigation bar
- Reduce stub init timeout from 100ms to 10ms for faster enhancement
- Change full implementation merge from 50ms setTimeout to requestAnimationFrame
- Add direct plugin loading in stub while waiting for full implementation
- Skip plugin reload in full implementation if already loaded by stub
- Significantly improves plugin tab loading speed in navigation bar
* feat(web-ui): Adapt file-upload widget for JSON files in of-the-day plugin
- Add specialized JSON upload/delete endpoints for of-the-day plugin
- Modify file-upload widget to support JSON files (file_type: json)
- Render JSON files with file-code icon instead of image preview
- Show entry count for JSON files
- Store files in plugins/ledmatrix-of-the-day/of_the_day/ directory
- Automatically update categories config when files are uploaded/deleted
- Populate uploaded_files array from categories on form load
- Remove custom HTML widget, use standard file-upload widget instead
* fix(web-ui): Add working updatePluginTabs to stub for immediate plugin tab rendering
- Stub's updatePluginTabs was empty, preventing tabs from showing
- Add basic implementation that creates plugin tabs in navigation bar
- Ensures plugin tabs appear immediately when plugins load, even before full implementation merges
- Fixes issue where plugin navigation bar wasn't working
* feat(api): Populate uploaded_files and categories from disk for of-the-day plugin
- Scan of_the_day directory for existing JSON files when loading config
- Populate uploaded_files array from files on disk
- Populate categories from files on disk if not in config
- Categories default to disabled, user can enable them
- Ensures existing JSON files (word_of_the_day.json, slovenian_word_of_the_day.json) appear in UI
* fix(api): Improve category merging logic for of-the-day plugin
- Preserve existing category enabled state when merging with files from disk
- Ensure all JSON files from disk appear in categories section
- Categories from files default to disabled, preserving user choices
- Properly merge existing config with scanned files
* fix(wifi): More aggressive password removal for AP mode hotspot
Problem:
- LEDMatrix-Setup network still asking for password despite previous fixes
- NetworkManager may add default security settings to hotspots
- Existing connections with passwords may not be fully cleaned up
Solutions:
- Always remove ALL security settings after creating hotspot (not just when detected)
- Remove multiple security settings: key-mgmt, psk, wep-key, auth-alg
- Verify security was removed and recreate connection if verification fails
- Improved cleanup: Delete connections by SSID match, not just by name
- Disconnect connections before deleting them
- Always restart connection after removing security to apply changes
- Better logging for debugging
This ensures the AP mode hotspot is always open, even if NetworkManager
tries to add default security settings.
* perf(web): Optimize web interface performance and fix JavaScript errors
- Add resource hints (preconnect, dns-prefetch) for CDN resources to reduce DNS lookup delays
- Fix duplicate response parsing bug in loadPluginConfig that was parsing JSON twice
- Replace direct fetch() calls with PluginAPI.getInstalledPlugins() to leverage caching and throttling
- Fix Alpine.js function availability issues with defensive checks and $nextTick
- Enhance request deduplication with debug logging and statistics
- Add response caching headers for static assets and API responses
- Add performance monitoring utilities with detailed metrics
Fixes console errors for loadPluginConfig and generateConfigForm not being defined.
Reduces duplicate API calls to /api/v3/plugins/installed endpoint.
Improves initial page load time with resource hints and optimized JavaScript loading.
* perf(web-ui): optimize CSS for Raspberry Pi performance
- Remove backdrop-filter blur from modal-backdrop
- Remove box-shadow transitions (use transform/opacity only)
- Remove button ::before pseudo-element animation
- Simplify skeleton loader (gradient to opacity pulse)
- Optimize transition utility (specific properties, not 'all')
- Improve color contrast for WCAG AA compliance
- Add CSS containment to cards, plugin-cards, modals
- Remove unused CSS classes (duration-300, divider, divider-light)
- Remove duplicate spacing utility classes
All animations now GPU-accelerated (transform/opacity only).
Optimized for low-powered Raspberry Pi devices.
* fix(web): Resolve ReferenceError for getInstalledPluginsSafe in v3 stub initialization
Move getInstalledPluginsSafe() function definition before the app() stub code that uses it. The function was previously defined at line 3756 but was being called at line 849 during Alpine.js initialization, causing a ReferenceError when loadInstalledPluginsDirectly() attempted to load plugins before the full implementation was ready.
* fix(web): Resolve TypeError for installedPlugins.map in plugin loading
Fix PluginAPI.getInstalledPlugins() to properly extract plugins array from API response structure. The API returns {status: 'success', data: {plugins: [...]}}, but the method was returning response.data (the object) instead of response.data.plugins (the array).
Changes:
- api_client.js: Extract plugins array from response.data.plugins
- plugins_manager.js: Add defensive array checks and handle array return value correctly
- base.html: Add defensive check in getInstalledPluginsSafe() to ensure plugins is always an array
This prevents 'installedPlugins.map is not a function' errors when loading plugins.
* style(web-ui): Enhance navigation bar styling for better readability
- Improve contrast: Change inactive tab text from gray-500 to gray-700
- Add gradient background and thicker border for active tabs
- Enhance hover states with background highlights
- Add smooth transitions using GPU-accelerated properties
- Update all navigation buttons (system tabs and plugin tabs)
- Add updatePluginTabStates() method for dynamic tab state management
All changes are CSS-only with zero performance overhead.
* fix(web-ui): Optimize plugin loading and reduce initialization errors
- Make generateConfigForm accessible to inline Alpine components via parent scope
- Consolidate plugin initialization to prevent duplicate API calls
- Fix script execution from HTMX-loaded content by extracting scripts before DOM insertion
- Add request deduplication to loadInstalledPlugins() to prevent concurrent requests
- Improve Alpine component initialization with proper guards and fallbacks
This eliminates 'generateConfigForm is not defined' errors and reduces plugin
API calls from 3-4 duplicate calls to 1 per page load, significantly improving
page load performance.
* fix(web-ui): Add guard check for generateConfigForm to prevent Alpine errors
Add typeof check in x-show to prevent Alpine from evaluating generateConfigForm
before the component methods are fully initialized. This eliminates the
'generateConfigForm is not defined' error that was occurring during component
initialization.
* fix(web-ui): Fix try-catch block structure in script execution code
Correct the nesting of try-catch block inside the if statement for script execution.
The catch block was incorrectly placed after the else clause, causing a syntax error.
* fix(web-ui): Escape quotes in querySelector to avoid HTML attribute conflicts
Change double quotes to single quotes in the CSS selector to prevent conflicts
with HTML attribute parsing when the x-data expression is embedded.
* style(web): Improve button text readability in Quick Actions section
* fix(web): Resolve Alpine.js expression errors in plugin configuration component
- Capture plugin from parent scope into component data to fix parsing errors
- Update all plugin references to use this.plugin in component methods
- Fix x-init to properly call loadPluginConfig method
- Resolves 'Uncaught ReferenceError' for isOnDemandLoading, onDemandLastUpdated, and other component properties
* fix(web): Fix remaining Alpine.js scope issues in plugin configuration
- Use this.generateConfigForm in typeof checks and method calls
- Fix form submission to use this.plugin.id
- Use $root. prefix for parent scope function calls (refreshPlugin, updatePlugin, etc.)
- Fix confirm dialog string interpolation
- Ensures all component methods and properties are properly scoped
* fix(web): Add this. prefix to all Alpine.js component property references
- Fix all template expressions to use this. prefix for component properties
- Update isOnDemandLoading, onDemandLastUpdated, onDemandRefreshing references
- Update onDemandStatusClass, onDemandStatusText, onDemandServiceClass, onDemandServiceText
- Update disableRunButton, canStopOnDemand, showEnableHint, loading references
- Ensures Alpine.js can properly resolve all component getters and properties
* fix(web): Resolve Alpine.js expression errors in plugin configuration
- Move complex x-data object to pluginConfigData() function for better parsing
- Fix all template expressions to use this.plugin instead of plugin
- Add this. prefix to all method calls in event handlers
- Fix duplicate x-on:click attribute on uninstall button
- Add proper loading state management in loadPluginConfig method
This resolves the 'Invalid or unexpected token' and 'Uncaught ReferenceError'
errors in the browser console.
* fix(web): Fix plugin undefined errors in Alpine.js plugin configuration
- Change x-data initialization to capture plugin from loop scope first
- Use Object.assign in x-init to merge pluginConfigData properties
- Add safety check in pluginConfigData function for undefined plugins
- Ensure plugin is available before accessing properties in expressions
This resolves the 'Cannot read properties of undefined' errors by ensuring
the plugin object is properly captured from the x-for loop scope before
any template expressions try to access it.
* style(web): Make Quick Actions button text styling consistent
- Update Start Display, Stop Display, and Reboot System buttons
- Change from text-sm font-medium to text-base font-semibold
- All Quick Actions buttons now have consistent bold, larger text
- Matches the styling of Update Code, Restart Display Service, and Restart Web Service buttons
* fix(wifi): Properly handle AP mode disable during WiFi connection
- Check return value of disable_ap_mode() before proceeding with connection
- Add verification loop to ensure AP mode is actually disabled
- Increase wait time to 5 seconds for NetworkManager restart stabilization
- Return clear error messages if AP mode cannot be disabled
- Prevents connection failures when switching networks from web UI or AP mode
This fixes the issue where WiFi network switching would fail silently when
AP mode disable failed, leaving the system in an inconsistent state.
* fix(web): Handle API response errors in plugin configuration loading
- Add null/undefined checks before accessing API response status
- Set fallback defaults when API responses don't have status 'success'
- Add error handling for batch API requests with fallback to individual requests
- Add .catch() handlers to individual fetch calls to prevent unhandled rejections
- Add console warnings to help debug API response failures
- Fix applies to both main loadPluginConfig and PluginConfigHelpers.loadPluginConfig
This fixes the issue where plugin configuration sections would get stuck
showing the loading animation when API responses failed or returned error status.
* fix(web): Fix Alpine.js reactivity for plugin config by using direct x-data
Changed from Object.assign pattern to direct x-data assignment to ensure
Alpine.js properly tracks reactive properties. The previous approach used
Object.assign to merge properties into the component after initialization,
which caused Alpine to not detect changes to config/schema properties.
The fix uses pluginConfigData(plugin) directly as x-data, ensuring all
properties including config, schema, loading, etc. are reactive from
component initialization.
* fix(web): Ensure plugin variable is captured in x-data scope
Use spread operator to merge pluginConfigData properties while explicitly
capturing the plugin variable from outer x-for scope. This fixes undefined
plugin errors when Alpine evaluates the component data.
* fix(web): Use $data for Alpine.js reactivity when merging plugin config
Use Object.assign with Alpine's $data reactive proxy instead of this to
ensure added properties are properly reactive. This fixes the issue where
plugin variable scoping from x-for wasn't accessible in x-data expressions.
* fix(web): Remove incorrect 'this.' prefix in Alpine.js template expressions
Alpine.js template expressions (x-show, x-html, x-text, x-on) use the
component data as the implicit context, so 'this.' prefix is incorrect.
In template expressions, 'this' refers to the DOM element, not the
component data.
Changes:
- Replace 'this.plugin.' with 'plugin.' in all template expressions (19 instances)
- Replace 'this.loading' with 'loading' in x-show directives
- Replace 'this.generateConfigForm' with 'generateConfigForm' in x-show/x-html
- Replace 'this.savePluginConfig' with 'savePluginConfig' in x-on:submit
- Replace 'this.config/schema/webUiActions' with direct property access
- Use '$data.loadPluginConfig' in x-init for explicit method call
Note: 'this.' is still correct inside JavaScript method definitions within
pluginConfigData() function since those run with proper object context.
* fix(web): Prevent infinite recursion in plugin config methods
Add 'parent !== this' check to loadPluginConfig, generateConfigForm, and
savePluginConfig methods in pluginConfigData to prevent infinite recursion
when the component tries to delegate to a parent that resolves to itself.
This fixes the 'Maximum call stack size exceeded' error that occurred when
the nested Alpine component's $root reference resolved to a component that
had the same delegating methods via Object.assign.
* fix(web): Resolve infinite recursion in plugin config by calling $root directly
The previous implementation had delegating methods (generateConfigForm,
savePluginConfig) in pluginConfigData that tried to call parent.method(),
but the parent detection via getParentApp() was causing circular calls
because multiple components had the same methods.
Changes:
- Template now calls $root.generateConfigForm() and $root.savePluginConfig()
directly instead of going through nested component delegation
- Removed delegating generateConfigForm and savePluginConfig from pluginConfigData
- Removed getParentApp() helper that was enabling the circular calls
- Simplified loadPluginConfig to use PluginConfigHelpers directly
This fixes the 'Maximum call stack size exceeded' error when rendering
plugin configuration forms.
* fix(web): Use window.PluginConfigHelpers instead of $root for plugin config
The $root magic variable in Alpine.js doesn't correctly reference the
app() component's data scope from nested x-data contexts. This causes
generateConfigForm and savePluginConfig to be undefined.
Changed to use window.PluginConfigHelpers which has explicit logic to
find and use the app component's methods.
* fix(web): Use direct x-data initialization for plugin config reactivity
Changed from Object.assign($data, pluginConfigData(plugin)) to
x-data="pluginConfigData(plugin)" to ensure Alpine.js properly
tracks reactivity for all plugin config properties. This fixes
the issue where all plugin tabs were showing the same config.
* refactor(web): Implement server-side plugin config rendering with HTMX
Major architectural improvement to plugin configuration management:
- Add server-side Jinja2 template for plugin config forms
(web_interface/templates/v3/partials/plugin_config.html)
- Add Flask route to serve plugin config partials on-demand
- Replace complex client-side form generation with HTMX lazy loading
- Add Alpine.js store for centralized plugin state management
- Mark old pluginConfigData and PluginConfigHelpers as deprecated
Benefits:
- Lazy loading: configs only load when tab is accessed
- Server-side rendering: reduces client-side complexity
- Better performance: especially on Raspberry Pi
- Cleaner code: Jinja2 macros replace JS string templates
- More maintainable: form logic in one place (server)
The old client-side code is preserved for backwards compatibility
but is no longer used by the main plugin configuration UI.
* fix(web): Trigger HTMX manually after Alpine renders plugin tabs
HTMX processes attributes at page load time, before Alpine.js
renders dynamic content. Changed from :hx-get attribute to
x-init with htmx.ajax() to properly trigger the request after
the element is rendered.
* fix(web): Remove duplicate 'enabled' toggle from plugin config form
The 'enabled' field was appearing twice in plugin configuration:
1. Header toggle (quick action, uses HTMX)
2. Configuration form (from schema, requires save)
Now only the header toggle is shown, avoiding user confusion.
The 'enabled' key is explicitly skipped when rendering schema properties.
* perf(web): Optimize plugin manager with request caching and init guards
Major performance improvements to plugins_manager.js:
1. Request Deduplication & Caching
- Added pluginLoadCache with 3-second TTL
- Subsequent calls return cached data instead of making API requests
- In-flight request deduplication prevents parallel duplicate fetches
- Added refreshInstalledPlugins() for explicit force-refresh
2. Initialization Guards
- Added pluginsInitialized flag to prevent multiple initializePlugins() calls
- Added _eventDelegationSetup guard on container to prevent duplicate listeners
- Added _listenerSetup guards on search/category inputs
3. Debug Logging Control
- Added PLUGIN_DEBUG flag (localStorage.setItem('pluginDebug', 'true'))
- Most console.log calls now use pluginLog() which only logs when debug enabled
- Reduces console noise from ~150 logs to ~10 in production
Expected improvements:
- API calls reduced from 6+ to 2 on page load
- Event listeners no longer duplicated
- Cleaner console output
- Faster perceived performance
* fix(web): Handle missing search elements in searchPluginStore
The searchPluginStore function was failing silently when called before
the plugin-search and plugin-category elements existed in the DOM.
This caused the plugin store to never load.
Now safely checks if elements exist before accessing their values.
* fix(web): Ensure plugin store loads via pluginManager.searchPluginStore
- Exposed searchPluginStore on window.pluginManager for easier access
- Updated base.html to fallback to pluginManager.searchPluginStore
- Added logging when loading plugin store
* fix(web): Expose searchPluginStore from inside the IIFE
The function was defined inside the IIFE but only exposed after the IIFE
ended, where the function was out of scope. Now exposed immediately after
definition inside the IIFE.
* fix(web): Add cache-busting version to plugins_manager.js URL
Static JS files were being aggressively cached, preventing updates
from being loaded by browsers.
* fix(web): Fix pluginLog reference error outside IIFE
pluginLog is defined inside the IIFE, so use _PLUGIN_DEBUG_EARLY and
console.log directly for code outside the IIFE.
* chore(web): Update plugins_manager.js cache version
* fix(web): Defer plugin store render when grid not ready
Instead of showing an error when plugin-store-grid doesn't exist,
store plugins in window.__pendingStorePlugins for later rendering
when the tab loads (consistent with how installed plugins work).
* chore: Bump JS cache version
* fix(web): Restore enabledBool variable in plugin render
Variable was removed during debug logging optimization but was still
being used in the template string for toggle switch rendering.
* fix(ui): Add header and improve categories section rendering
- Add proper header (h4) to categories section with label
- Add debug logging to diagnose categories field rendering
- Improve additionalProperties condition check readability
* fix(ui): Improve additionalProperties condition check
- Explicitly exclude objects with properties to avoid conflicts
- Ensure categories section is properly detected and rendered
- Categories should show as header with toggles, not text box
* fix(web-ui): Fix JSON parsing errors and default value loading for plugin configs
- Fix JSON parsing errors when saving file upload fields by properly unescaping HTML entities
- Merge config with schema defaults when loading plugin config so form shows default values
- Improve default value handling in form generation for nested objects and arrays
- Add better error handling for malformed JSON in file upload fields
* fix(plugins): Return plugins array from getInstalledPlugins() instead of data object
Fixed PluginAPI.getInstalledPlugins() to return response.data.plugins (array)
instead of response.data (object). This was preventing window.installedPlugins
from being set correctly, which caused plugin configuration tabs to not appear
and prevented users from saving plugin configurations via the web UI.
The fix ensures that:
- window.installedPlugins is properly populated with plugin array
- Plugin tabs are created automatically on page load
- Configuration forms and save buttons are rendered correctly
- Save functionality works as expected
* fix(api): Support form data submission for plugin config saves
The HTMX form submissions use application/x-www-form-urlencoded format
instead of JSON. This update allows the /api/v3/plugins/config POST
endpoint to accept both formats:
- JSON: plugin_id and config in request body (existing behavior)
- Form data: plugin_id from query string, config fields from form
Added _parse_form_value helper to properly convert form strings to
appropriate Python types (bool, int, float, JSON arrays/objects).
* debug: Add form data logging to diagnose config save issue
* fix(web): Re-discover plugins before loading config partial
The plugin config partial was returning 'not found' for plugins
because the plugin manifests weren't loaded. The installed plugins
API was working because it calls discover_plugins() first.
Changes:
- Add discover_plugins() call in _load_plugin_config_partial when
plugin info is not found on first try
- Remove debug logging from form data handling
* fix(web): Comprehensive plugin config save improvements
SWEEPING FIX for plugin configuration saving issues:
1. Form data now MERGES with existing config instead of replacing
- Partial form submissions (missing fields) no longer wipe out
existing config values
- Fixes plugins with complex schemas (football, clock, etc.)
2. Improved nested value handling with _set_nested_value helper
- Correctly handles deeply nested structures like customization
- Properly merges when intermediate objects already exist
3. Better JSON parsing for arrays
- RGB color arrays like [255, 0, 0] now parse correctly
- Parse JSON before trying number conversion
4. Bump cache version to force JS reload
* fix(web): Add early stubs for updatePlugin and uninstallPlugin
Ensures these functions are available immediately when the page loads,
even before the full IIFE executes. Provides immediate user feedback
and makes API calls directly.
This fixes the 'Update button does not work' issue by ensuring the
function is always defined and callable.
* fix(web): Support form data in toggle endpoint
The toggle endpoint now accepts both JSON and HTMX form submissions.
Also updated the plugin config template to send the enabled state
via hx-vals when the checkbox changes.
Fixes: 415 Unsupported Media Type error when toggling plugins
* fix(web): Prevent config duplication when toggling plugins
Changed handleToggleResponse to update UI in place instead of
refreshing the entire config partial, which was causing duplication.
Also improved refreshPluginConfig with proper container targeting
and concurrent refresh prevention (though it's no longer needed
for toggles since we update in place).
* fix(api): Schema-aware form value parsing for plugin configs
Major fix for plugin config saving issues:
1. Load schema BEFORE processing form data to enable type-aware parsing
2. New _parse_form_value_with_schema() function that:
- Converts comma-separated strings to arrays when schema says 'array'
- Parses JSON strings for arrays/objects
- Handles empty strings for arrays (returns [] instead of None)
- Uses schema to determine correct number types
3. Post-processing to ensure None arrays get converted to empty arrays
4. Proper handling of nested object fields
Fixes validation errors:
- 'category_order': Expected type array, got str
- 'categories': Expected type object, got str
- 'uploaded_files': Expected type array, got NoneType
- RGB color arrays: Expected type array, got str
* fix(web): Make plugin config handlers idempotent and remove scripts from HTMX partials
CRITICAL FIX for script redeclaration errors:
1. Removed all <script> tags from plugin_config.html partial
- Scripts were being re-executed on every HTMX swap
- Caused 'Identifier already declared' errors
2. Moved all handler functions to base.html with idempotent initialization
- Added window.__pluginConfigHandlersInitialized guard
- Functions only initialized once, even if script runs multiple times
- All state stored on window object (e.g., window.pluginConfigRefreshInProgress)
3. Enhanced error logging:
- Client-side: Logs form payload, response status, and parsed error details
- Server-side: Logs raw form data and parsed config on validation failures
4. Functions moved to window scope:
- toggleSection
- handleConfigSave (with detailed error logging)
- handleToggleResponse (updates UI in place, no refresh)
- handlePluginUpdate
- refreshPluginConfig (with duplicate prevention)
- runPluginOnDemand
- stopOnDemand
- executePluginAction
This ensures HTMX-swapped fragments only contain HTML, and all
scripts run once in the base layout.
* fix(api): Filter config to only schema-defined fields before validation
When merging with existing_config, fields not in the plugin's schema
(like high_performance_transitions, transition, dynamic_duration)
were being preserved, causing validation failures when
additionalProperties is false.
Add _filter_config_by_schema() function to recursively filter config
to only include fields defined in the schema before validation.
This fixes validation errors like:
- 'Additional properties are not allowed (high_performance_transitions, transition were unexpected)'
* fix(web): Improve update plugin error handling and support form data
1. Enhanced updatePlugin JavaScript function:
- Validates pluginId before sending request
- Checks response.ok before parsing JSON
- Better error logging with request/response details
- Handles both successful and error responses properly
2. Update endpoint now supports both JSON and form data:
- Similar to config endpoint, accepts plugin_id from query string or form
- Better error messages and debug logging
3. Prevent duplicate function definitions:
- Second updatePlugin definition checks if improved version exists
- Both definitions now have consistent error handling
Fixes: 400 BAD REQUEST 'Request body must be valid JSON' error
* fix(web): Show correct 'update' message instead of 'save' for plugin updates
The handlePluginUpdate function now:
1. Checks actual HTTP status code (not just event.detail.successful)
2. Parses JSON response to get server's actual message
3. Replaces 'save' with 'update' if message incorrectly says 'save'
Fixes: Update button showing 'saved successfully' instead of
'updated successfully'
* fix(web): Execute plugin updates immediately instead of queuing
Plugin updates are now executed directly (synchronously) instead of
being queued for async processing. This provides immediate feedback
to users about whether the update succeeded or failed.
Updates are fast git pull operations, so they don't need async
processing. The operation queue is reserved for longer operations
like install/uninstall.
Fixes: Update button not actually updating plugins (operations were
queued but users didn't see results)
* fix(web): Ensure toggleSection function is always available for collapsible headers
Moved toggleSection outside the initialization guard block so it's
always defined, even if the plugin config handlers have already been
initialized. This ensures collapsible sections in plugin config forms
work correctly.
Added debug logging to help diagnose if sections/icons aren't found.
Fixes: Collapsible headers in plugin config schema not collapsing
* fix(web): Improve toggleSection to explicitly show/hide collapsible content
Changed from classList.toggle() to explicit add/remove of 'hidden' class
based on current state. This ensures the content visibility is properly
controlled when collapsing/expanding sections.
Added better error checking and state detection for more reliable
collapsible section behavior.
* fix(web): Load plugin tabs on page load instead of waiting for plugin manager tab click
The stub's loadInstalledPlugins was an empty function, so plugin tabs
weren't loading until the plugin manager tab was clicked. Now the stub
implementation:
1. Tries to use global window.loadInstalledPlugins if available
2. Falls back to window.pluginManager.loadInstalledPlugins
3. Finally falls back to direct loading via loadInstalledPluginsDirectly
4. Always updates tabs after loading plugins
This ensures plugin navigation tabs are available immediately on page load.
Fixes: Plugin tabs only loading after clicking plugin manager tab
* fix(web): Ensure plugin navigation tabs load on any page regardless of active tab
Multiple improvements to ensure plugin tabs are always visible:
1. Stub's loadInstalledPluginsDirectly now waits for DOM to be ready
before updating tabs, using requestAnimationFrame for proper timing
2. Stub's init() now has a retry mechanism that periodically checks
if plugins have been loaded by plugins_manager.js and updates tabs
accordingly (checks for 2 seconds)
3. Full implementation's init() now properly handles async plugin loading
and ensures tabs are updated after loading completes, checking
window.installedPlugins first before attempting to load
4. Both stub and full implementation ensure tabs update using $nextTick
to wait for Alpine.js rendering cycle
This ensures plugin navigation tabs are visible immediately when the
page loads, regardless of whether the user is on overview, plugin manager,
or any other tab.
Fixes: Plugin tabs only appearing after clicking plugin manager tab
* fix(web): Fix restart display button not working
The initPluginsPage function was returning early before event listeners
were set up, making all the event listener code unreachable. Moved the
return statement to after all event listeners are attached.
This fixes the restart display button and all other buttons in the
plugin manager (refresh plugins, update all, search, etc.) that depend
on event listeners being set up.
Fixes: Restart Display button not working in plugin manager
* fix(web-ui): Improve categories field rendering for of-the-day plugin
- Add more explicit condition checking for additionalProperties objects
- Add debug logging specifically for categories field
- Add fallback handler for objects that don't match special cases (render as JSON textarea)
- Ensure categories section displays correctly with toggle cards instead of plain text
* fix(install): Prevent following broken symlinks during file ownership setup
- Add -P flag to find commands to prevent following symlinks when traversing
- Add -h flag to chown to operate on symlinks themselves rather than targets
- Exclude scripts/dev/plugins directory which contains development symlinks
- Fixes error when chown tries to dereference broken symlinks with extra LEDMatrix in path
* fix(scroll): Ensure scroll completes fully before switching displays
- Add display_width to total scroll distance calculation
- Scroll now continues until content is completely off screen
- Update scroll completion check to use total_scroll_width + display_width
- Prevents scroll from being cut off mid-way when switching to next display
* fix(install): Remove unsupported -P flag from find commands
- Remove -P flag which is not supported on all find versions
- Keep -h flag on chown to operate on symlinks themselves
- Change to {} \; syntax for better error handling
- Add error suppression to continue on broken symlinks
- Exclude scripts/dev/plugins directory to prevent traversal into broken symlinks
* docs(wifi): Add trailing newline to WiFi AP failover setup guide
* fix(web): Suppress non-critical socket errors and fix WiFi permissions script
- Add error filtering in web interface to suppress harmless client disconnection errors
- Downgrade 'No route to host' and broken pipe errors from ERROR to DEBUG level
- Fix WiFi permissions script to use mktemp instead of manual temp file creation
- Add cleanup trap to ensure temp files are removed on script exit
- Resolves permission denied errors when creating temp files during installation
* fix(web): Ensure plugin navigation tabs load on any page by dispatching events
The issue was that when plugins_manager.js loaded and called
loadInstalledPlugins(), it would set window.installedPlugins but the
Alpine.js component wouldn't know to update its tabs unless the plugin
manager tab was clicked.
Changes:
1. loadInstalledPlugins() now always dispatches a 'pluginsUpdated' event
when it sets window.installedPlugins, not just when plugin IDs change
2. renderInstalledPlugins() also dispatches the event and always updates
window.installedPlugins for reactivity
3. Cached plugin data also dispatches the event when returned
The Alpine component already listens for the 'pluginsUpdated' event in
its init() method, so tabs will now update immediately when plugins are
loaded, regardless of which tab is active.
Fixes: Plugin navigation tabs only loading after clicking plugin manager tab
* fix(web): Improve input field contrast in plugin configuration forms
Changed input backgrounds from bg-gray-800 to bg-gray-900 (darker) to
ensure high contrast with white text. Added placeholder:text-gray-400
for better placeholder text visibility.
Updated in both server-side template (plugin_config.html) and client-side
form generation (plugins_manager.js):
- Number inputs
- Text inputs
- Array inputs (comma-separated)
- Select dropdowns
- Textareas (JSON objects)
- Fallback inputs without schema
This ensures all form inputs have high contrast white text on dark
background, making them clearly visible and readable.
Fixes: White text on white background in plugin config inputs
* fix(web): Change plugin config input text from white to black
Changed all input fields in plugin configuration forms to use black text
on white background instead of white text on dark background for better
readability and standard form appearance.
Updated:
- Input backgrounds: bg-gray-900 -> bg-white
- Text color: text-white -> text-black
- Placeholder color: text-gray-400 -> text-gray-500
Applied to both server-side template and client-side form generation
for all input types (number, text, select, textarea).
* fix(web): Ensure toggleSection function is available for plugin config collapsible sections
Moved toggleSection function definition to an early script block so it's
available immediately when HTMX loads plugin configuration content. The
function was previously defined later in the page which could cause it
to not be accessible when inline onclick handlers try to call it.
The function toggles the 'hidden' class on collapsible section content
divs and rotates the chevron icon between right (collapsed) and down
(expanded) states.
Fixes: Plugin configuration section headers not collapsing/expanding
* fix(web): Fix collapsible section toggle to properly hide/show content
Updated toggleSection function to explicitly set display style in addition
to toggling the hidden class. This ensures the content is properly hidden
even if CSS specificity or other styles might interfere with just the
hidden class.
The function now:
- Checks both the hidden class and computed display style
- Explicitly sets display: '' when showing and display: 'none' when hiding
- Rotates chevron icon between right (collapsed) and down (expanded)
This ensures collapsible sections in plugin configuration forms properly
hide and show their content when the header is clicked.
Fixes: Collapsible section headers rotate chevron but don't hide content
* fix(web): Fix collapsible section toggle to work on first click
Simplified the toggle logic to rely primarily on the 'hidden' class check
rather than mixing it with computed display styles. When hiding, we now
remove any inline display style to let Tailwind's 'hidden' class properly
control the display property.
This ensures sections respond correctly on the first click, whether they're
starting in a collapsed or expanded state.
Fixes: Sections requiring 2 clicks to collapse
* fix(web): Ensure collapsible sections start collapsed by default
Added explicit display: none style to nested content divs in plugin config
template to ensure they start collapsed. The hidden class should handle this,
but adding the inline style ensures sections are definitely collapsed on
initial page load.
Sections now:
- Start collapsed (hidden) with chevron pointing right
- Expand when clicked (chevron points down)
- Collapse when clicked again (chevron points right)
This ensures a consistent collapsed initial state across all plugin
configuration sections.
* fix(web): Fix collapsible section toggle to properly collapse on second click
Fixed the toggle logic to explicitly set display: block when showing and
display: none when hiding, rather than clearing the display style. This
ensures the section state is properly tracked and the toggle works correctly
on both expand and collapse clicks.
The function now:
- When hidden: removes hidden class, sets display: block, chevron down
- When visible: adds hidden class, sets display: none, chevron right
This fixes the issue where sections would expand but not collapse again.
Fixes: Sections not collapsing on second click
* feat(web): Ensure plugin navigation tabs load automatically on any page
Implemented comprehensive solution to ensure plugin navigation tabs load
automatically without requiring a visit to the plugin manager page:
1. Global event listener for 'pluginsUpdated' - works even if Alpine isn't
ready yet, updates tabs directly when plugins_manager.js loads plugins
2. Enhanced stub's loadInstalledPluginsDirectly():
- Sets window.installedPlugins after loading
- Dispatches 'pluginsUpdated' event for global listener
- Adds console logging for debugging
3. Event listener in stub's init() method:
- Listens for 'pluginsUpdated' events
- Updates component state and tabs when events fire
4. Fallback timer:
- If plugins_manager.js hasn't loaded after 2 seconds, fetches
plugins directly via API
- Ensures tabs appear even if plugins_manager.js fails
5. Improved checkAndUpdateTabs():
- Better logging
- Fallback to direct fetch after timeout
6. Enhanced logging throughout plugin loading flow for debugging
This ensures plugin tabs are visible immediately on page load, regardless
of which tab is active or when plugins_manager.js loads.
Fixes: Plugin navigation tabs only loading after visiting plugin manager
* fix(web): Improve plugin tabs update logging and ensure immediate execution
Enhanced logging in updatePluginTabs() and _doUpdatePluginTabs() to help
debug why tabs aren't appearing. Changed debounce behavior to execute
immediately on first call to ensure tabs appear quickly.
Added detailed console logging with [FULL] prefix to track:
- When updatePluginTabs() is called
- When _doUpdatePluginTabs() executes
- DOM element availability
- Tab creation process
- Final tab count
This will help identify if tabs are being created but not visible, or if
the update function isn't being called at all.
Fixes: Plugin tabs loading but not visible in navigation bar
* fix(web): Prevent duplicate plugin tab updates and clearing
Added debouncing and duplicate prevention to stub's updatePluginTabs() to
prevent tabs from being cleared and re-added multiple times. Also checks
if tabs already match before clearing them.
Changes:
1. Debounce stub's updatePluginTabs() with 100ms delay
2. Check if existing tabs match current plugin list before clearing
3. Global event listener only triggers full implementation's updatePluginTabs
4. Stub's event listener only works in stub mode (before enhancement)
This prevents the issue where tabs were being cleared and re-added
multiple times in rapid succession, which could leave tabs empty.
Fixes: Plugin tabs being cleared and not re-added properly
* fix(web): Fix plugin tabs not rendering when plugins are loaded
Fixed _doUpdatePluginTabs() to properly use component's installedPlugins
instead of checking window.installedPlugins first. Also fixed the 'unchanged'
check to not skip when both lists are empty (first load scenario).
Changes:
1. Check component's installedPlugins first (most up-to-date)
2. Only skip update if plugins exist AND match (don't skip empty lists)
3. Retry if no plugins found (in case they're still loading)
4. Ensure window.installedPlugins is set when loading directly
5. Better logging to show which plugin source is being used
This ensures tabs are rendered when plugins are loaded, even on first page load.
Fixes: Plugin tabs not being drawn despite plugins being loaded
* fix(config): Fix array field parsing and validation for plugin config forms
- Added logic to detect and combine indexed array fields (text_color.0, text_color.1, etc.)
- Fixed array fields incorrectly stored as dicts with numeric keys
- Improved handling of comma-separated array values from form submissions
- Ensures array fields meet minItems requirements before validation
- Resolves 400 BAD REQUEST errors when saving plugin config with RGB color arrays
* fix(config): Improve array field handling and secrets error handling
- Use schema defaults when array fields don't meet minItems requirement
- Add debug logging for array field parsing
- Improve error handling for secrets file writes
- Fix arrays stored as dicts with numeric keys conversion
- Better handling of incomplete array values from form submissions
* fix(config): Convert array elements to correct types (numbers not strings)
- Fix array element type conversion when converting dicts to arrays
- Ensure RGB color arrays have integer elements, not strings
- Apply type conversion for both nested and top-level array fields
- Fixes validation errors: 'Expected type number, got str'
* fix(config): Fix array fields showing 'none' when value is null
- Handle None/null values in array field templates properly
- Use schema defaults when array values are None/null
- Fix applies to both Jinja2 template and JavaScript form generation
- Resolves issue where stock ticker plugin shows 'none' instead of default values
* fix(config): Add novalidate to plugin config form to prevent HTML5 validation blocking saves
- Prevents browser HTML5 validation from blocking form submission
- Allows custom validation logic to handle form data properly
- Fixes issue where save button appears unclickable due to invalid form controls
- Resolves problems with plugins like clock-simple that have nested/array fields
* feat(config): Add helpful form validation with detailed error messages
- Keep HTML5 validation enabled (removed novalidate) to prevent broken configs
- Add validatePluginConfigForm function that shows which fields fail and why
- Automatically expands collapsed sections containing invalid fields
- Focuses first invalid field and scrolls to it
- Shows user-friendly error messages with field names and specific issues
- Prevents form submission until all fields are valid
* fix(schema): Remove core properties from required array during validation
- Core properties (enabled, display_duration, live_priority) are system-managed
- SchemaManager now removes them from required array after injection
- Added default values for core properties (enabled=True, display_duration=15, live_priority=False)
- Updated generate_default_config() to ensure live_priority has default
- Resolves 186 validation issues, reducing to 3 non-blocking warnings (98.4% reduction)
- All 19 of 20 plugins now pass validation without errors
Documentation:
- Created docs/PLUGIN_CONFIG_CORE_PROPERTIES.md explaining core property handling
- Updated existing docs to reflect core property behavior
- Removed temporary audit files and scripts
* fix(ui): Improve button text contrast on white backgrounds
- Changed Screenshot button text from text-gray-700 to text-gray-900
- Added global CSS rule to ensure all buttons with white backgrounds use dark text (text-gray-900) for better readability
- Fixes contrast issues where light text on light backgrounds was illegible
* fix(ui): Add explicit text color to form-control inputs
- Added color: #111827 to .form-control class to ensure dark text on white backgrounds
- Fixes issue where input fields had white text on white background after button contrast fix
- Ensures all form inputs are readable with proper contrast
* docs: Update impact explanation and plugin config documentation
* docs: Improve documentation and fix template inconsistencies
- Add migration guide for script path reorganization (scripts moved to scripts/install/ and scripts/fix_perms/)
- Add breaking changes section to README with migration guidance
- Fix config template: set plugins_directory to 'plugins' to match actual plugin locations
- Fix test template: replace Jinja2 placeholders with plain text to match other templates
- Fix markdown linting: add language identifiers to code blocks (python, text, javascript)
- Update permission guide: document setgid bit (0o2775) for directory modes
- Fix example JSON: pin dependency versions and fix compatible_versions range
- Improve readability: reduce repetition in IMPACT_EXPLANATION.md
* feat(web): Make v3 interface production-ready for local deployment
- Phase 2: Real Service Integration
- Replace sample data with real psutil system monitoring (CPU, memory, disk, temp, uptime)
- Integrate display controller to read from /tmp/led_matrix_preview.png snapshot
- Scan assets/fonts directory and extract font metadata with freetype
- Phase 1: Security & Input Validation
- Add input validation module with URL, file upload, and config sanitization
- Add optional CSRF protection (gracefully degrades if flask-wtf missing)
- Add rate limiting (lenient for local use, prevents accidental abuse)
- Add file upload validation to font upload endpoint
- Phase 3: Error Handling
- Add global error handlers for 404, 500, and unhandled exceptions
- All endpoints have comprehensive try/except blocks
- Phase 4: Monitoring & Observability
- Add structured logging with JSON format support
- Add request logging middleware (tracks method, path, status, duration, IP)
- Add /api/v3/health endpoint with service status checks
- Phase 5: Performance & Caching
- Add in-memory caching system (separate module to avoid circular imports)
- Cache font catalog (5 minute TTL)
- Cache system status (10 second TTL)
- Invalidate cache on config changes
- All changes are non-blocking with graceful error handling
- Optional dependencies (flask-wtf, flask-limiter) degrade gracefully
- All imports protected with try/except blocks
- Verified compilation and import tests pass
* docs: Fix caching pattern logic flaw and merge conflict resolution plan
- Fix Basic Caching Pattern: Replace broken stale cache fallback with correct pattern
- Re-fetch cache with large max_age (31536000) in except block instead of checking already-falsy cached variable
- Fixes both instances in ADVANCED_PLUGIN_DEVELOPMENT.md
- Matches correct pattern from manager.py.template
- Fix MERGE_CONFLICT_RESOLUTION_PLAN.md merge direction
- Correct Step 1 to checkout main and merge plugins into it (not vice versa)
- Update commit message to reflect 'Merge plugins into main' direction
- Fixes workflow to match documented plugins → main merge
---------
Co-authored-by: Chuck <chuck@example.com>
|