mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 06:15:09 +00:00
41192b95883a68bec1ae84d14af657dae7d3fab0
234
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ChuckandClaude Opus 5.5
|
05deb1ee7d |
feat(install): support Raspberry Pi OS Bookworm (Python 3.11) alongside Trixie (3.13) (#689)
The installer and scripts/check_system_compatibility.sh share one set of OS rules (scripts/install/lib_os.sh): Bookworm (Debian 12, Python 3.11) and Trixie (Debian 13, Python 3.13) are supported, python3 older than 3.11 stops the install before anything changes, and dhcpcd gets a warning with directions. setcap targets /usr/bin/python3, the apt fallback honours the requirement floors, and the desktop check no longer misreads under pipefail. CI runs the unit and plugin-safety suites on 3.11 and 3.13 (tooling jobs on 3.13); mypy targets 3.11. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
f841fa36b6 |
feat(install): updates refresh systemd units; new installs run the newest release (#729)
Updates that move HEAD now install changed systemd units through a root-owned helper (/usr/local/sbin/ledmatrix-refresh-units, two literal sudo lines), with a backup restored on rollback; a refresh that fails part-way puts the old units back. Devices without the new sudo rule keep updating and are told to re-run the installer once. The one-shot installer now checks out the newest vX.Y.Z release (LEDMATRIX_CHANNEL=beta keeps main) and never moves an existing checkout backwards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
515248b34e |
feat(ipc): control socket stage 3 - a state stream replaces polled cache keys (#735)
Adds state.get / state.subscribe to the display's control socket (StateHub in src/ipc/server.py). The web interface holds one subscription per process (web_interface/display_state.py) and reads current-status, on-demand status, plugin runtime and /health's display_loop from it, falling back to the cache keys and heartbeat file. While the socket serves readers, display_current_state and plugin_runtime_snapshot are written less often (about 1.5 instead of 5 cache writes a minute for 15 s screens). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
a7b3f33952 |
feat(web): Rotation, Operation History, Config Editor and Backup & Restore become ES-module pages (stage 2) (#727)
Rotation, Operation History, Config Editor and Backup & Restore become ES-module pages (stage 2): no inline scripts or onclick in the four partials, delegated data-action listeners, reads cancelled on swap-out, old globals kept as deprecated aliases through window.LEDMatrix, and four new DOM suites. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
c14002edc3 |
fix(status): runtime status agrees with the heartbeat; current-status republishes on wake (#726)
The runtime status snapshot now agrees with the display heartbeat, and current-status is republished when the display wakes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
746dcfcadb |
feat(ipc): control socket stage 2 - wake the render thread, brightness.set, plugin.reload (#720)
Control socket stage 2: the render thread wakes for queued commands (static screens ~1 ms, Vegas within one frame), brightness.set, and plugin.reload after a store update, with mailbox/restart fallbacks. Rig checks listed in the PR body are still to run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
f72d69c2b0 |
perf(display): skip preview work nobody reads (lazy checksum, 1 Hz snapshot writer) (#717)
Mid-scroll, update_display() no longer checksums every frame: it asks is_currently_scrolling() once per frame and reuses the answer, hashes only when dirty tracking can skip a static frame, and the preview snapshot asks its policy first and hashes only when a write or touch could follow (decide() is monotone, pinned by a property test). With the preview open the snapshot is written at most once a second (VIEWER_INTERVAL 1.0 s, was 0.2 s; the SSE stream re-read it once a second, so four encodes in five went unread); the stream now polls its mtime every 0.25 s (VIEWER_POLL_INTERVAL), so the preview stays about as fresh. The PNG is written at compress_level=1. --preview soaks are not comparable across this change. Merged with #716: _scan_segments takes the frame's one scrolling answer and #716's static-handover pass-through, as soaked on ledpi (A B B A, 20 min each: main 0.118% / 0.113% late, with #716 and this 0.107% / 0.104%). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
4be53d048b |
feat(fetch): shared fetch service, stage 1 (pooling, merging, host budgets, counters) (#702)
Core's own HTTP fetch paths (APIHelper, fetch_espn_scoreboard and its date chunks, BackgroundDataService, BaseOddsManager.get_odds) go through one service in src/common/fetch_service.py: shared connection pools per retry policy, merged identical in-flight GETs, per-host token-bucket budgets (fetch_service.rate_limits), and per-plugin request counters published to GET /api/v3/plugins/fetch-stats. Return values, exceptions, cache keys, TTLs and retry policies are unchanged. Core-internal in this release; plugins should not import it directly yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Sonnet 5.5
|
16b566e14f |
feat(scroll): show which scroll speeds are smooth on this panel (#710)
* feat(scroll): show which scroll speeds are smooth on this panel The Vegas Scroll Speed slider now says what the panel will do with the chosen speed and offers the nearest smooth ones to click. Backed by scroll_config.speed_advice() and GET /api/v3/config/scroll-speed-advice, which uses the refresh the display measured rather than the cap. Also stops the default 50 px/s snapping to a stepped 48 px/s (2px every 5 refreshes, 24fps) on a 120Hz panel: the low-fps penalty in solve_crisp() now loses to 60 or 40 px/s. 100Hz panels are unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(scroll): hint threw before its timer variables existed; count 25-30fps as stepped The Vegas speed hint called refreshScrollSpeedHint() before the let declarations it uses, so it never rendered (found on ledpi). And the solver's low-fps penalty stopped at 25fps, which let a measured 125.7Hz panel keep a 25.1fps 2px-every-5-refreshes scroll. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * test: add the scroll-speed-advice route to the /api/v3 URL map snapshot Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
695ff92009 |
feat(ipc): display control socket, stage 1 - on-demand with acks (#706)
The display serves a control socket (/run/ledmatrix/control.sock) carrying versioned JSON commands, one per line, each answered. Stage 1 covers on-demand start, stop and status; commands are queued on the socket thread and applied on the render thread through the mailbox's own handler, and the web interface falls back to the file mailbox when the socket is unavailable. Protocol and security model: docs/IPC_CONTROL_SOCKET.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
c6701ac00d |
feat(web): ES-module page lifecycle and one schema field model (stage 1) (#703)
Adds a native ES-module layer to the web UI (core/boot, registry, api, facade; window.LEDMatrix as the one global), a page lifecycle that the Cache tab is converted to as the reference, text/javascript serving and revalidation for unversioned module requests, and src/plugin_system/field_model.py with a parity test against the render_field macro. Also: the cache page toggles its grey 'Not configured' style instead of only adding it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
7f06cc9c3b |
feat(vegas): keep live games in the ticker by default (#699)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements update in place while they scroll One background worker (src/vegas_mode/live_worker.py) redraws a plugin's live elements when its data epoch moves on (update listener) or on their refresh_hz, nearest the screen first, and hands changed pixels lock-free to the render thread, which copies them into the strip between frames (RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most four patches or two screens of bytes a frame, no drawing or locks there. The worker takes over group prefetch once a live element is placed, runs inside the render gate, and is supervised. Update tick 1s while live elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md describes what was built and why SegmentStrip was not needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(sports): live Vegas cards for the scoreboards (shared layer) One live element per game, drawn only when what the card shows changes, so a score changes on a card already crossing the panel. The shared part, so each scoreboard adopts it in a few lines: - src/common/sports_vegas.py: game_key, game_fingerprint (the whole game dict, frozen: no drawn field can be missed), dedupe_games, VegasCardCache, StickyOdds (odds a live poll left out stay drawn), finished_games / with_finished_games (a game that just went final keeps its card, after its league's live games; one a heuristic only judged over keeps its live state, so a tied end of regulation never shows FINAL early). - SportsScrollDisplay.make_vegas_renderer() is the override point; build_vegas_elements() and SportsScrollDisplayManager .get_vegas_elements_for() do the rest. A card's version includes its teams' ranks, which the renderer draws from the rankings cache. - SportsLiveSharedMixin._record_finished_game() / finished_games_snapshot(): held for FINISHED_GAME_TTL after it leaves the live list. A sport that does not implement make_vegas_renderer keeps its ordinary Vegas content, so no scoreboard changes until it opts in. scripts/render_plugin.py --vegas renders a plugin's Vegas block as the ticker lays it out, and --timeline stacks it at successive moments as the ticker would update it in place; the join is now render_pipeline.join_plugin_rows(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): keep live games in the ticker by default display.vegas_scroll.live_in_ticker now defaults to true: through a live game the marquee keeps running and the live scoreboard takes extra turns in it -- its cards updating in place while they scroll -- instead of the ticker giving way to the full-screen scoreboard. The new default would reach nobody on its own: every existing config holds an explicit false copied from the template (there was no control for it), and the template merge only adds missing keys. ConfigManager therefore turns a stored false on once, with a backup, and records live_in_ticker_migrated so a false chosen afterwards stays. The marker is never in the template. A "Keep live games in the ticker" checkbox under Vegas mode sets it. Tests that pin the full-screen takeover now say live_in_ticker=false. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(sports): a default _determine_game_type on SportsScrollDisplay render_vegas_card looked the method up with getattr and a None default, which static analysis (Codacy) reports as calling something that may not be callable. The base class now has the default -- the card type from the game's state -- and the plugins that define their own override it as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: review follow-ups on the shared live-card layer - The reused Vegas renderer always gets the current rankings, empty included, so ranks cleared since are not kept drawn. - render_plugin.py: --timeline refuses --no-live (a timeline shows live elements changing), --timeline/--no-live need --vegas, and the Vegas paths create the output's directory like the display path does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
f4bda50710 |
feat(vegas): live elements update in place while they scroll (#697)
* perf(timing): say which render-thread work a late frame followed The soak already says how often a moving frame reached the panel late, but not what the render thread was doing just before it. Vegas does two kinds of work there between frames -- building its strip (compose, extend) and, with live elements, patching changed pixels into it -- and deciding whether either is affordable needs their own numbers. - FrameTimingRecorder.note_op(kind, nbytes) tags the next presented frame. Totals gain op_frames, late_op_frames, op_freezes and op_bytes per kind; aggregate() still takes frames without ops. The file schema is unchanged. - Vegas tags compose and every strip extension (with the bytes it copied). - frame_soak prints an "after work" table: frames, late %, freezes and MB moved per kind, only when something tagged its work. - render_bench gains --strip-screens (Vegas-sized strips), --patch-bytes / --patch-every / --patch-where (in-place column writes, as a live element update does) and --extend-every-screens / --extend-width (append + trim on a fixed cadence that holds the strip's width). No runtime behaviour changes: this is the measurement gate for live Vegas elements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): note the frame-op attribution and bench modes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(scroll): build the strip's PIL image only when something reads it Every Vegas strip extension rebuilt ScrollHelper.cached_image from cached_array in full, twice (append, then trim), on the render thread: Image.fromarray is 1.7ms for an 8,000px strip and 3.8ms for 20,000px on a Pi 4 (measured on ledpi), about two thirds of an extension's render-thread cost. Nothing on the frame path reads the image's pixels; every frame is cut from the array. cached_image is now a property. append_content and drop_scrolled_prefix defer it; the first read builds it from the array it started with and keeps it only if the strip has not changed meanwhile, so a sync push racing an extension cannot leave a stale image cached. Assigning cached_image stores exactly what was assigned, as before. has_strip() says whether there is a strip without building its image; the helper's frame path, Vegas and the adapter's scroll-cache invalidation use it. The strip is also no longer held in memory twice. In Vegas the image is now built only by a multi-display sync push. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements -- a plugin API for content that changes while it scrolls Vegas bakes each plugin's pictures into one strip, so a card already on its way across the panel keeps what it showed when it was drawn. This adds the API and bookkeeping for content that can be updated in place; the worker that redraws and swaps it follows separately. No shipped plugin implements the hook yet, so nothing changes for users. Plugin API (core 3.8.0), all no-ops by default: - BasePlugin.get_vegas_elements() -> [VegasElement(key, image, version, live, refresh_hz)]: named, fixed-width pieces of Vegas content. - BasePlugin.redraw_vegas_element(key, width, height, at): a lock-free redraw for content that changes with time. - BasePlugin.notify_vegas_data_changed(): data that lands outside update(). - src/plugin_system/vegas_elements.py (VegasElement, re-exported from base_plugin). Core: - PluginAdapter asks a plugin that implements the hook for elements on the background fetch only (under its lock, on its own canvas); every other path keeps get_vegas_content(). Live elements are pinned (padded with content_padding, never trimmed), tagged with their key, digest and data epoch in Image.info so the existing cache and group plumbing carry them unchanged, and untagged if a width budget crops them. - RenderPipeline records where each live element lands (ElementRecord), in absolute strip columns a trim does not move; the block-start arithmetic is shared with the STATIC markers. - PluginManager update listeners (add/remove_update_listener, notify_data_changed): told the moment update() completes, not at the next ~4s Vegas poll. The coordinator uses one to move each plugin's data epoch on. - vegas_scroll.live_refresh (kill switch), live_max_hz, live_min_interval, live_lead_screens; per-plugin core-owned vegas_live. Live elements are off under multi-display sync, in swap mode and with offscreen_prefetch off. - scripts/check_plugin.py checks the element contract (src/plugin_system/testing/vegas.py); test/fixtures/plugins/vegas-live-stub is a working example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(vegas): live elements update in place while they scroll One background worker (src/vegas_mode/live_worker.py) redraws a plugin's live elements when its data epoch moves on (update listener) or on their refresh_hz, nearest the screen first, and hands changed pixels lock-free to the render thread, which copies them into the strip between frames (RenderPipeline.apply_live_patches, ScrollHelper.patch_columns): at most four patches or two screens of bytes a frame, no drawing or locks there. The worker takes over group prefetch once a live element is placed, runs inside the render gate, and is supervised. Update tick 1s while live elements exist. Web UI switch for live_refresh. OFFSCREEN_RENDERING.md describes what was built and why SegmentStrip was not needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
7804ea8f69 |
feat(update): stable/beta update channel; stable follows release tags (#684)
Adds auto_update.channel: stable follows the newest vX.Y.Z release tag (detached HEAD; pre-releases and other tags ignored), beta follows main as before. Nothing ever moves a device backwards: a checkout newer than the newest release keeps following main (or stays put when detached) until a release contains its commit. Legacy configs migrate to stable when they reach a release. Update Code, the weekly updater's preflight, and the verifier's rollback (back to old_ref: branch or detached release) all honour the channel. General tab Update Channel select, GET/POST /api/v3/system/update-channel, release-aware Overview banner and Tools git panel. New installs default to stable. Rig fix (ledpi): /system/check-update reports update_available: false when the channel's action is none (a detached HEAD newer than the newest release), matching Update Code; the Tools panel no longer calls every detached HEAD "a release". Merged with main through #687 (heartbeat verifier, #683 login, #688 plugin_catalog, #685 Tailwind build). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
64c7289593 |
feat(display): systemd watchdog and heartbeat for a frozen render loop (#687)
If the render loop gets stuck inside a plugin's display(), ledmatrix.service stays active and the panel stays frozen. This adds a way to detect that. - src/display_watchdog.py (standard library only) sends sd_notify over $NOTIFY_SOCKET and writes /run/ledmatrix/display-heartbeat.json. Only the render thread counts: beats from other threads are ignored. - ledmatrix.service: WatchdogSec=120, NotifyAccess=main, RuntimeDirectory=ledmatrix (0755), RestartSteps=4 and RestartMaxDelaySec=2min. It stays Type=simple. run.py widens the watchdog to 15 min for start-up, and load_plugin() does the same on the render thread. The loop arms after its first frame. - /api/v3/health adds checks.display_loop: running, stalled (no heartbeat for over 60s, which makes the status degraded) or not_reported. With web login on, a caller who is not logged in still gets only healthy/degraded, and a stall degrades that answer. - The update verifier requires a fresh heartbeat from the restarted display when the display it replaced was writing one. A frozen panel is rolled back. - Existing installs get the systemd watchdog only after install_service.sh is re-run. The heartbeat works right away. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
b09434a418 |
refactor(plugins): the display publishes plugin runtime state; retire plugin_state.json (#690)
Stage 2 of the web plugin catalog, after #688. - The display publishes a plugin runtime snapshot (plugin_runtime.py) to the shared cache: per plugin loaded, lifecycle state, a short redacted error summary, the version it loaded and when, plus published_at / stale_after / running. Written on change (throttled to 10 s; the RUNNING/ENABLED flip of an ordinary update is not a change) and once a minute otherwise; cleanup() publishes running: false. - The web reads it back and restores loaded / state / error_info in /api/v3/plugins/installed (plus loaded_version, loaded_at and data.runtime). Only a live snapshot counts; stale, stopped or missing answers null and says which. - data/plugin_state.json is retired: every reader and writer moved to config + disk (desired) or the snapshot (observed). Nothing in it was non-derivable, so nothing is migrated and an existing file is left unread. The web-side PluginStateManager (state_manager.py) is removed; the display's plugin_state.PluginStateManager is the only state machine. - StateReconciliation compares config + disk with the snapshot, reporting enabled-but-not-loaded and older-version-loaded as no_action findings. - Backups list installed manifests with enabled from config.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
7ab6fb1aff |
refactor(web): read plugins through a PluginCatalog; only the display runs them (#688)
The web process built its own PluginManager and loaded plugins into itself: store installs and updates loaded or reloaded a web-side copy, and config saves and enable/disable called on_config_change, on_enable and on_disable on it. None of that reached the panel, and /plugins/installed reported runtime state from those copies. - Add PluginCatalog (src/plugin_system/plugin_catalog.py): manifests, directories, display modes, installed version, schema and config reads, with no way to run a plugin. app.py and both blueprints use it; the plugin_manager blueprint attribute is gone. - Remove every lifecycle call from the web routes. Config changes already reach the display through ConfigService (on_config_change) and the enabled-set reconcile. - Health and metrics readers move to api_v3.health_tracker / resource_monitor. /plugins/installed reports loaded/state/error_info as null (the display does not publish them) and enabled by the display's rule. - Store install, update and uninstall answer restart_required when the running display will not pick the change up by itself (display_restart_required). The restart banner follows the flag via window.noteRestartRequired instead of the /config/main URL heuristic; /config/main now sends restart_required: true. - The one remaining in-process import of plugin code (Starlark helper modules, oauth_flow action scripts) goes through _import_plugin_code_in_web_process() until a web-entry contract. - /plugins/installed reports vegas_participation (from #682) from the user's setting or the manifest, with vegas_participation_source; when only the plugin's code decides it, null with source 'runtime', since the web process no longer has plugin instances to ask. - Check & Update All keeps its restart flags when the final list refresh fails, and asks for a restart when an enabled plugin's first request got no answer and the re-sent one found it up to date. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
ba6eccb489 |
build(web): generate the UI's Tailwind CSS with the pinned standalone CLI (#685)
Replaces the hand-written Tailwind subset in app.css with a real, purged Tailwind build: scripts/build_css.py runs the pinned, SHA-256-checked standalone Tailwind CLI (no Node), the generated tailwind.css and plugin-frame.css are committed, and CI fails when they are stale. The Pi never builds anything. The login page (#683) now links tailwind.css too, and the load-order test covers every template that links app.css. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
5ea0d511dc |
feat(store): read ledmatrix_min_version, aliases and commit from the registry (#686)
The store reads three optional registry fields: ledmatrix_min_version (an incompatible install/update is refused before any download, with a "Needs LEDMatrix X+" card badge), aliases (update/uninstall/reinstall by registry id find a plugin installed under its manifest id, with registry proof only), and commit (shown and linked on the store card). An older plugins.json behaves as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
1c928b2033 |
feat(vegas): one declared participation per plugin (scroll | pause | exclude) (#682)
A plugin takes part in Vegas mode in one declared way: 'scroll', 'pause' or 'exclude', resolved from the user's vegas_participation setting, the manifest field, then the legacy hooks, so no plugin changes behaviour. The stream manager decides inclusion and pauses through it; the installed plugins API and the Vegas plugin-order list report it. Deprecates get_supported_vegas_modes, get_vegas_segment_width and vegas_panel_count for removal in 3.9.0, and regenerates docs/DEPRECATIONS_3.8.md to include them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
15c61def67 |
fix(web): make the SSE streams' 200/min rate limit actually apply (#691)
app.py called limiter.limit("200 per minute")(stream_x) after the routes
were registered and discarded the result. flask-limiter 3.x enforces a
decorated limit in the wrapper limit() returns, and marks the original
function so the before_request middleware skips it, so the streams had
no limit at all -- not even the 1000/min default. Register the wrapper
as the view instead.
The new test (skipped without flask-limiter) reconnects to each stream
201 times and expects the last to get a 429; it fails on the old code.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5.5
|
e3c85cece6 |
feat(web): optional web login and API tokens, off by default (stacked on #674) (#683)
Optional web login, off by default: a device that sets no password behaves exactly as before. Set under General > Security; then every page and API route needs a session login or an API token (Authorization: Bearer). Loopback, the Wi-Fi setup flow in AP mode, static files, captive-portal probes and a reduced /api/v3/health stay open. Secrets live in the web_auth section of config_secrets.json and no API returns them. scripts/reset_web_password.py turns login off. Stacked on #674. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
ba38a83c2c |
fix(web): refuse cross-site state-changing requests (Origin/Referer check) (#674)
The web interface refuses state-changing requests (POST/PUT/PATCH/DELETE) whose Origin (or, without one, Referer) is not the host they were sent to, or is null: 403 CROSS_SITE_REQUEST (web_interface/origin_guard.py). Any website a LAN user visited could otherwise make their browser POST a plain form to the Pi. /api/v3/system/action also refuses form-encoded and text/plain bodies (415) unless sent by HTMX. Clients that send no Origin or Referer (curl, requests, Home Assistant, the MQTT bridge) are unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
c3a7a110c4 |
fix(display): on-demand loads a disabled plugin live instead of failing (#678)
* fix(web): on-demand no longer restarts a running display service POST /display/on-demand/start treated start_service (default true, sent by "Preview on display", the on-demand dialog and the MQTT bridge) as "restart": with the service running it ran systemctl stop, slept 1.5s and started it again. Every request cold-started the display process -- every plugin reloaded, panel blank -- to deliver a request the running process already reads from the cache mailbox every ON_DEMAND_POLL_INTERVAL (0.25s), including mid-dwell, mid-screen and mid-Vegas. The restart bought nothing: startup only restores a session the display saved itself (display_on_demand_config), so the new request arrived through the same mailbox either way. start_service now means "start it if it is not running". The stop route coerces stop_service to a boolean so "false" no longer stops the service. test_api_v3_on_demand_restart.py pinned the old restart path; it now pins the replacement. Docs updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(display): on-demand loads a disabled plugin live instead of failing The display process only loads enabled plugins, so an on-demand request for a disabled one -- "Preview on display" offers it on every config page, with a note that the plugin will be enabled for the preview -- failed with invalid-mode. Nothing enabled it short of a restart, and the on-demand route no longer restarts the service. _activate_on_demand now loads an installed-but-not-running plugin through the live-enable path (load_plugin + _register_loaded_plugin), with a new load_plugin(force_enabled=True) so the instance runs enabled while config.json keeps saying disabled. The plugin is tracked in _on_demand_loaded_plugins, and the main loop unloads it through _unregister_plugin once on-demand moves off it (stop, expiry, another request, or a failed request that ends the session) -- right after its own poll, where no display() is on the stack. A failed load publishes status error with load-failed. A plugin enabled during the session stays loaded. A session restored after a restart uses the same tracking instead of setting enabled in the config dict config_manager caches, so its plugin is unloaded when the session ends rather than staying loaded until the next restart. Ending a session no longer resumes the rotation onto a plugin that is about to be unloaded, which a restored session did. Also: a stop sent while on-demand is inactive clears a failed request's error, instead of /display/on-demand/status reporting status: error until the state aged out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
c4c46d3ba7 |
fix(web): on-demand no longer restarts a running display service (#676)
POST /display/on-demand/start treated start_service (default true, sent by "Preview on display", the on-demand dialog and the MQTT bridge) as "restart": with the service running it ran systemctl stop, slept 1.5s and started it again. Every request cold-started the display process -- every plugin reloaded, panel blank -- to deliver a request the running process already reads from the cache mailbox every ON_DEMAND_POLL_INTERVAL (0.25s), including mid-dwell, mid-screen and mid-Vegas. The restart bought nothing: startup only restores a session the display saved itself (display_on_demand_config), so the new request arrived through the same mailbox either way. start_service now means "start it if it is not running". The stop route coerces stop_service to a boolean so "false" no longer stops the service. test_api_v3_on_demand_restart.py pinned the old restart path; it now pins the replacement. Docs updated. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
09103a8a7d |
refactor(web): split api_v3/plugins.py by area (#658)
* refactor(web): split api_v3/plugins.py by area
web_interface/blueprints/api_v3/plugins.py (3,285 lines) becomes:
- plugins.py: installed list, enable/disable, plugin actions
- plugin_store.py: install, update, uninstall, store, saved repositories
- plugin_config.py: config get/save, schema, reset
- plugin_assets.py: asset uploads and plugin static files
- plugin_health.py: health, metrics, limits
- plugin_operations.py: operation history, state reconciliation
- plugin_calendar.py: calendar credentials and auth
Pure move: all 44 functions and 38 route decorators are byte-identical
(checked with ast), URLs and endpoint names are unchanged (url-map test).
Each module imports only what it uses. Tests and config.py that reached
into plugins.py for moved names now import from the new module.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): keep exception text out of calendar responses; annotate moved code
The split made scanners report existing findings in the moved code as new:
- CodeQL: the calendar auth and calendar-list routes returned exception
text (redacted, but still derived from the exception). Both now log the
exception and return a fixed message pointing at the log.
- MD5 in the asset upload only makes a filename unique: usedforsecurity=False.
- pickle reads/writes the calendar plugin's own OAuth token (as before):
annotated. Token-status labels and a log line naming the secrets path are
false positives: annotated with the repo's nosec/nosemgrep convention.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): name uploaded assets with SHA-256 instead of MD5
The hash only makes an uploaded image's filename unique. Codacy flags MD5
even with usedforsecurity=False, and SHA-256 does the job as well; existing
files keep their names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): keep the redacted exception detail in calendar errors
Reverts the calendar part of
|
||
|
ChuckandClaude Opus 5.5
|
6cfcf2e384 |
fix(web): plugin dir resolver in routes, nmcli AP detection, daemon config reload, upload safety, BDF preview (#655)
* fix(web): plugin dir resolver in routes, nmcli AP detection, daemon config reload, upload safety - Route plugin lookups (installed list, update, recorded version, config form, web UI pages) through the plugin manager's resolver so plugins in ledmatrix-<id> directories work. - Captive-portal detection also sees the nmcli fallback AP (cached). - WiFi monitor daemon re-reads wifi_config.json when its mtime changes. - Drop the AP check in disconnect_from_network that could never fire. - LED status file per WiFiManager; config path falls back to this checkout. - BDF font preview via src.common.bdf_font. - Asset uploads validate every file before saving; metadata and calendar credentials written atomically; no absolute path in the response; asset delete answers 400 for a missing body. - Coerce string booleans in plugin toggle, on-demand start and AP force. - SSE broadcaster clears its thread handle before exiting. - start.py log filter handles every exc_info form. - Cleanups: unused plugins/fonts partial work, duplicate backup catch-alls, raw-config error helper, update-route tidy, redundant imports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): request BDF font previews now that the server renders them The Fonts tab skipped the preview request for .bdf files because the server used to refuse them; /fonts/preview now draws BDF with the shared loader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): take the update route's plugin directory from a directory listing CodeQL flagged the path built from the request's plugin_id (the id was already validated with safe_path_component, which CodeQL doesn't model; the same flow on main is alerts 738/739). The directory is now the entry of plugins_dir matched by name, so nothing built from user input reaches the filesystem; an id with nothing installed goes to the store manager, which reports it not found as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): read the blueprint's plugin_manager defensively in _plugin_directory _get_plugin_version now goes through _plugin_directory, which read api_v3.plugin_manager directly; the attribute exists only once the app sets it, so test_path_traversal_guards::test_a_real_manifest_is_read failed when run on its own (order-dependent in the full suite). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
c00bf5e8e6 |
fix(plugin-system): unload/update race, failed-load cleanup, limits validation, schema lookup, install rollback (#653)
* fix(plugin-system): unload/update race, failed-load module cleanup, limits validation, schema lookup, install rollback, op-queue dedupe - unload_plugin takes the per-plugin lock (5s bounded) before cleanup(), and an update() that finishes after its plugin was unloaded no longer sets the state back to ENABLED. - A load that fails after import drops plugin_<id> and its submodules and forgets its manager fonts, so a fixed plugin reloads new code. - Resource limits are validated as non-negative numbers: 400 at POST /plugins/limits, bad cached records ignored with one warning. Route docstrings note health/metrics reset and limits only change the web process's view. - SchemaManager.get_schema_path resolves each search dir via resolve_plugin_dir (manifest id, ledmatrix-<id>) before the literal paths; plugins/ still before plugin-repos/. Misses cached 30s and logged once at DEBUG. - install_from_url sets an existing copy aside and restores it if the move fails, under the per-plugin reinstall lock. - Operation queue refuses a second pending op for a plugin and trims _operations with history. - get_vegas_render_width reads display_manager.width first. - get_logger in store/schema/health/resource/saved_repositories; UTF-8 reads in store_manager and state_manager. - Docs: update_interval precedence (manifest over config) stated where users are told to set it in config. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): build the limits 400 message from the field name, not an exception CodeQL flagged str(e) flowing into the response. invalid_limit_field() returns the offending field without raising, and limits_from_dict uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
0e9e2cabba |
fix(web): widget cache-busting, dead frontend code, and dependency pins (#656)
- Plugin-supplied widgets load as /static/plugin-widgets/...js?v=<plugin version>, so an update isn't hidden behind the year-long immutable cache. - Fire-and-forget loadInstalledPlugins() calls catch the rejection it has already reported, so the global handler no longer adds a second toast. - Timezone picker renders again when the General partial is re-injected. - Remove dead code: executePluginAction's six plugin-id fallbacks and [DEBUG] logging, window.currentPluginConfig and every read of it, the file-upload JSON delete branch, unused PluginAPI / PluginInstallManager / PluginStateManager helpers, loadPluginWidgetsFromManifest, the stale install_manager.js and LEDVisibility fallbacks, error_handler.js's global escapeHtml, 13 unused CSS rules, and stale comments/no-op returns. - pytz < 2027, psutil < 7 in requirements-test.txt, pytest-cov < 8. - Pin anthropics/claude-code-action to the commit v1 resolves to. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
65d82580bc |
test(starlark): cover the review fixes #535 shipped without tests (#650)
Six Starlark fixes are on main via #535 and #537, but a follow-up commit carrying tests for half of them was pushed to fix/starlark-pixlet-install six minutes after #535 merged, so those tests never landed. This ports them onto the api_v3 package split: - a failed toggle write answers 500, and a loaded app is not flipped in memory when the manifest write fails - each manifest writer gets its own temp file; concurrent writes leave readable JSON; no temp files are left behind - a failed dynamic import of tronbyte_repository / pixlet_renderer does not stay cached in sys.modules - a failed save_config() leaves config and timing untouched and does not re-render; a successful save still applies It also logs when the timing update to the manifest is not persisted. _update_manifest_safe answers False rather than raising, so the existing except branch never saw that failure. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
76f5d8a336 |
fix(web): seven web UI bugs, and remove dead plugins_manager.js helpers (#647)
- Operation History: the plugin filter lists the installed plugin ids
instead of one option, "plugins" (Object.keys of {plugins: [...]}).
- Ctrl/Cmd+S submits the active tab's first visible form with
requestSubmit() (validation and onsubmit guards run) instead of a bare
Event on the first form in the document; skipped inside a modal dialog
and on tabs without a form.
- Overview "Check Updates" confirms like "Update Code", takes its button
explicitly (no implicit global event) and shows the server's message.
Both, and the Tools tab git pull, raise the restart-pending banner on
restart_required.
- Tools: toolsAction and diagnostics show the server's error message;
only a non-JSON body falls back to HTTP <status>.
- Installed list after uninstall: PluginAPI writes clear the throttler's
GET cache, a forced loadInstalledPlugins clears it too, and the
post-uninstall reload goes through refreshInstalledPlugins().
- Plugin widgets load from /static/plugin-widgets/ only (the other two
paths have no route).
- Raw JSON editor escapes the parse error; slider escapes value/min/max/step.
- Removed the unreferenced array-of-objects and key-value helpers from
plugins_manager.js, the textarea auto-resize and Ctrl+R handlers in
app.js, and a redundant ?v= on the plugins_manager.js script tag.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5.5
|
7eb7a58d0c |
fix: web UI and src.common bugs (wifi wrong-password, plugin icon, starlark toggle, API caching, scroll/logo/font helpers) (#646)
- wifi: keep the "wrong_password:" prefix through the restore/AP fallback so the UI's incorrect-password prompt fires again. - /plugins/installed returns the manifest's icon (string only). - /starlark/apps/<id>/toggle coerces `enabled` and delegates to _toggle_starlark_app (disk before memory, no KeyError, "false" is false). - /api/v3/ JSON GETs are sent Cache-Control: no-store; non-JSON keeps 5s. - ScrollHelper.set_scrolling_image converts non-RGB input (alpha onto black); create/set_scrolling_image reset last_update_time like reset_scroll. - LogoHelper backs off a failed download per path for MISSING_LOGO_RECHECK_SECONDS; cleared on invalidate/clear_cache. - refresh_placeholder_timestamp saves atomically. - FontManager.clear_cache / _clear_plugin_font_cache bump cache_generation. - Odds manager: per-game logs to DEBUG; JSON decode error caught before RequestException (same cooldown). - element_style mangled continuations; startup validator skips null plugin blocks and reuses the controller's discovery. - src/common/README lists frame_timing, json_body, render_gate. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
bcef1957a9 |
fix(security): refuse unsafe plugin ids, keep secrets private, validate request bodies (#643)
* fix(security): refuse unsafe plugin ids, keep secrets private, validate bodies - install_from_url and the registry install's manifest rename refuse a plugin id that is not a single safe name (no ../ out of plugins_dir). - Uninstall and config reset refuse core config sections and ids with path parts; uninstall of a plugin whose directory is gone still works. - separate_secrets checks a field's own x-secret marker before recursing, so object/array secrets no longer land in config.json. - Backup restore creates missing secrets/wifi/ytm files with mode 640; export skips non-object manifests and no longer collides on same-second exports. - SYSTEM_FONTS includes every bundled font from BUNDLED_FONTS. - Raw config/secrets saves and validate_request_json require a JSON object. - A blank max_dynamic_duration_seconds keeps the stored value; other values are validated to 30-1800 instead of raising a 500. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(security): validate the id before install_plugin moves anything; claim backup names atomically - install_plugin set aside plugins_dir / plugin_id before any id check, so "../x" moved a directory outside the plugins dir (the rollback moved it back, but only if the install path got that far) - two exports finishing in the same second could both see a free name and the later os.replace destroyed the first archive; the name is now claimed with O_EXCL before the archive is swapped in Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5
|
c4927e82a3 |
fix(config): normalize nullable arrays and objects instead of refusing them (#642)
* fix(config): normalize nullable arrays and objects instead of refusing them
`element_style._nullable` widens every `customization.modes.<mode>` override
with 'null' so a blank means "inherit the base", which turns a colour declared
"array" into ["array", "null"]. `normalize_config_values` only knew how to
convert null/integer/number/boolean out of a union, so a valid [0, 249, 0]
matched nothing and logged
Could not normalize field customization.modes.upcoming.odds_text.text_color:
value=[0, 249, 0], type=<class 'list'>, schema_type=['array', 'null']
The warning was the harmless half. It `continue`d past the single-type handling
below, where `prop_type == 'array'` coerces items, so a nullable array never had
its items normalized while a plain one did. A form posts numbers as strings, so
["0", "249", "0"] survived to the validator and was rejected with "Expected type
integer, got str" -- setting a per-mode colour in the web UI failed outright.
Every per-mode override of a structural or string type was exposed, across all
eight scoreboard plugins, not only colours.
Re-enter the single-type handling with the matched member rather than bailing,
accept a string that matches, and warn only on a genuine mismatch so the
diagnostic still reaches the validator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ET8e5weDrb5Ju5QTKLU7zh
* fix(config): convert only integral numbers for integer array items
Review catch on the previous commit. Routing nullable arrays into the shared
item handling made its integer coercion reachable for them, and that coercion
called int(v) on any number: a client sending [2.5, 249, 0] for an RGB array
got 2 stored and a 200 back, so a wrong value was silently corrected into a
valid-looking one rather than refused.
Convert only genuinely integral values, at both the union-item and the plain
'array' item branch so the two cannot drift. A whole float -- 2.0, which is all
JSON can express for an integer -- still converts. This also settles an
inconsistency that predates the change: int('2.5') raises, so the string form
was always preserved and rejected while the numeric form was truncated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ET8e5weDrb5Ju5QTKLU7zh
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5.5
|
7f9c73e9aa |
fix(vegas): smooth Vegas scroll pacing -- whole pixels per refresh, measured refresh, off-thread preview writes (#628)
Vegas scrolls a whole number of pixels per panel refresh, locked to SwapOnVSync, against the refresh the panel really holds (measured from swap gaps), instead of blending sub-pixel positions against the refresh cap. The web preview PNG is encoded off the render thread while scrolling, with writes ordered and retried. On hdpi, late frames fell from 6.3% to 0.7%. See docs/SCROLL_PERFORMANCE.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
f6afbdbb15 |
fix(web): Cache/Logs error mix-up, store errors, tab fallbacks; remove ~2.5k lines of dead JS (#639)
* fix(web): keep Cache and Logs helpers out of each other's way Both partials declared top-level showError and escapeHtml. Their scripts run at global scope after every HTMX swap, so whichever tab was opened last owned window.showError, and a Cache failure after visiting Logs rendered into the Logs panel (and the other way round). Each script is now an IIFE; Cache still exports deleteCacheFile for its row buttons. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): make the HTMX-failure fallbacks for tab panels actually run - The "HTMX never loaded" fallback read appElement.__x.$data, which is Alpine 2. The page ships Alpine 3, so the check was always false and the Overview never loaded without HTMX. It now reads Alpine.$data(). - The Overview and WiFi panels used hx-on::htmx:response-error, which htmx expands to "htmx:htmx:response-error", an event that never fires. - loadTabContent sent requests with <body> as the source, so htmx fired its events on <body> and no panel's hx-on handler ran at all. The panel is now the source. htmx also resolves its promise on a 4xx/5xx, and the panel was stamped data-loaded anyway, leaving a skeleton that never retried; it is now stamped only when no responseError fired. loadPluginsDirect, loadOverviewDirect and loadWifiDirect are merged into one window.loadPartialDirect(id, url), which also runs the partial's inline scripts before Alpine sees the markup (as htmx-config.js does on htmx:afterSwap). The ~10 s "htmx never arrived" path in loadTabContent uses it for every tab instead of four hard-coded ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): store and registry failures no longer wipe the Plugin Manager showError replaced the whole #plugins-content with an error message, so one failed store search, custom-registry install or saved-repository call took the installed list, the store and every control with it, with no way back short of reloading the tab. Those failures are now error notifications. The full-panel message is kept only for a first load of the installed list that failed (nothing to show yet); a failed refresh of an already-rendered list is a notification too. showSuccess's fallback branch, which wrote the message into innerHTML unescaped, is gone: showNotification always exists. The "Please try refreshing your browser" hint tested for the text "Failed to Fetch", which no browser produces (Chrome says "Failed to fetch", Firefox "NetworkError..."), so it never appeared. It now keys on the failure itself: a TypeError from fetch(), or PluginAPI's NETWORK_ERROR wrapper around one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): escape plugin action and install output on every path executePluginAction escaped data.message and data.output when an action failed but put data.message straight into innerHTML when it succeeded, and set the OAuth step-2 button's innerHTML from the manifest's step2_button_text. Plugin actions run plugin code, so that is plugin- or server-controlled markup in the page. Both paths now escape, and the button label is set with textContent. The same pattern sat in the install-from-GitHub-URL status lines (plugin_id, the server's message, and error.message, which can echo a repository URL) and the custom-registry load error; those are escaped too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): file-upload widget owns the image list and schedule editor plugins_manager.js loads after the widget bundle, so its older copies of deleteUploadedFile, updateImageList, hideUploadProgress, formatDate, openImageSchedule, toggleImageScheduleEnabled, updateImageSchedule{Mode, Time,Day} and updateCheckboxGroupData replaced the widget's. They are deleted; the widget files are the only definitions. Before switching over, the two sets were diffed and fixed so nothing regresses: - The old copy labelled the schedule/delete buttons for screen readers and lazy-loaded thumbnails; the widget now does both. - The schedule button did nothing on a card rendered by plugin_config.html whenever the image id is a UUID (every upload): the template turns "-" into "_" in the editor's id, and neither JS copy did. Both now use the template's rule. - The widget's "keep the open editor open" copied the editor's innerHTML into the new list. That dropped its event listeners and showed the old values, so after the first change the editor looked live but ignored input. A schedule edit now saves to the hidden input and updates the card's summary in place without re-rendering the list; a list re-render (upload, delete) rebuilds an open editor from the data. Editor controls are routed by one delegated change listener, so there are no per-element listeners to lose. - The old deleteUploadedFile had a JSON branch that removed a #file_<id> element and skipped the re-render. No template or script renders such an element, and JSON uploads are listed through updateImageList like images, so re-rendering (the widget's behaviour) is the consistent one; the branch was not carried over. - The template always renders the summary line (".image-schedule-summary", "Always shown" when unscheduled) so an edit has a line to update. The inline-handler test evaluated plugins_manager.js's updateImageList; test_file_upload_widget.js now covers the widget's list and editor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): delete the unused handleCredentialsUpload Its last caller went when plugin_config.html switched credential uploads to the file-upload widget's handleSingleFileSelect. Nothing in the web UI, the tests or the plugin monorepo references it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): delete dead and shadowed front-end code Nothing calls any of these (checked across web_interface/, test/ and the ledmatrix-plugins monorepo, including hx-*/x-*/onclick attributes): - app-shell.js: the Alpine methods refreshPlugins (it called a nonexistent this.searchPluginStore), loadPluginConfig, savePluginConfig, getSchemaPropertyType, escapeCssSelector, formatCommitInfo and formatDateInfo, and the top-level copies of savePluginConfig, getSchemaPropertyType, escapeCssSelector, formatCommitInfo, formatDateInfo and togglePluginFromTab. Plugin config forms save through hx-post in plugin_config.html. - window.reconnectSSE (app-shell.js); window.updateArrayTableAddButtonState (array-table.js). - toggleNestedSection, defined twice (app-shell.js and plugins_manager.js) and called from nowhere. - plugins_manager.js: the window.initializePlugins wrapper around an IIFE-local origInit that was always undefined, and __pluginDomReady, which was written but never read. - display.html's fixInvalidNumberInputs fallback: app-shell.js defines it before any partial loads. - base.html's window.loadCodeMirror and the two CodeMirror stylesheet preloads, and the .CodeMirror rules in plugins.html. The raw JSON editor is a plain textarea. Also deleted: app-shell.js definitions that a later script always replaced, so they never ran: executePluginAction (plugins_manager.js assigns its own), uninstallPlugin and its pollUninstallOperation (plugins_manager.js), and updateAllPlugins (install_manager.js). vendor/codemirror stays: test/test_web_smoke.py still requests codemirror.min.js as a sample static asset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): call showNotification without checking it exists app-shell.js defines window.showNotification (a stand-in that queues until the notification widget loads) and base.html runs it, deferred, before every other script that notifies: app.js, the utilities, the widget bundle, plugins_manager.js, and all partials, which HTMX loads after the page. The 81 `typeof showNotification === 'function'` / `!== 'undefined'` checks, the `window.showNotification || console.log` and `|| alert` fallbacks, and their else branches (alert(), console output, and schedule.html's own hand-built toast) could never take the fallback path. They are removed, as is fonts.html's second copy of the queueing stand-in. The stand-in in app-shell.js keeps its guard (it must not replace the widget's implementation if load order ever changes), and BaseWidget's public notify()/getNotificationFunction() keep their shape for widgets that plugins ship. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): one HTML escaper, window.LEDEscape About 30 files each carried their own escapeHtml / escapeAttr / escHtml / _esc / escapeJs. They disagreed: several (notification.js, display.html's escapeHtml, operation_history.html, the app() stub) did not escape quotes, google-calendar-picker.js and tools.html's escHtml left ' alone, and some turned 0 into ''. Most were fine only because the quote-safe widget copies were preferred at runtime. window.LEDEscape now lives at the top of app-early.js, a blocking script in <head>, so it exists before any other script runs: html(v) & < > " ' as entities, null/undefined as '' attr(v) the same, for call sites that want to say "attribute" jsStringAttr(v) a JS string literal safe inside an inline handler Every former copy is now a one-line name for it (kept so call sites do not change), widgets included, with no fallback. plugins_manager.js loses its four escapeJs wrappers (callers use jsStringAttr), the duplicate escapeAttr and escapeHtml inside renderInstalledCards and renderCustomRegistryPlugins, and the window.escapeHtml / window.escapeAttribute exports, which nothing read. addArrayObjectItem's fallback markup (with a sixth hand-written escape chain) is gone too: window.renderArrayObjectItem is defined earlier in the same file, so the fallback could not run. The unused escapeHtml methods on the Alpine app (app-early.js stub and app-shell.js) are deleted. test_html_escaping.js now runs LEDEscape and every remaining name for it, and fails if a hand-rolled escaper reappears anywhere in web_interface/. Suites that evaluate slices of plugins_manager.js or widget files load LEDEscape from app-early.js through test/js/led_escape.js. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): stop htmx re-running partial scripts after every tab load htmx-config.js runs each swapped-in <script> itself on htmx:afterSwap and meant to turn htmx's own script handling off with htmx.config.allowScriptTags = false. It did that once, while setting up, but base.html loads htmx with a dynamic <script>, so htmx was not defined yet and the setting never applied. On every tab load htmx then tried to run each script again in its settle phase, found it already replaced (no parent node) and threw "Cannot read properties of null (reading 'insertBefore')" into the console, which also skipped the rest of that swap's settle tasks. The setting is now applied in the afterSwap handler, which always runs after htmx exists and before htmx settles the same swap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): show "--" for a system stat the server could not read The stats stream and /system/status now send null for a metric they cannot read (cpu_temp off a Pi, for one) instead of 0. updateSystemStats built the header and Overview text as value + unit, so a null showed as "null°C". CPU, memory and temperature, in the header and on the Overview, now render "--" plus the unit for null or a missing field -- the same placeholder the page starts with, and what tools.html already shows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): one Alpine accessor and one plugin-list signal window.getApp() (app-early.js) returns the root <body x-data="app()"> component through Alpine's public Alpine.$data, or null before Alpine has initialised it. It replaces the private el._x_dataStack[0] reads in app.js, app-early.js, app-shell.js, settings-search.js, overview.html and plugins_manager.js, the three local getAppComponent/appData/getAppData copies, and the Alpine 2 el.__x.$data fallbacks, which Alpine 3 never provides. Publishing the installed-plugin list: one load set window.installedPlugins and dispatched pluginsUpdated twice (loadInstalledPlugins, then renderInstalledPlugins), then wrote into the Alpine component through _x_dataStack[0] and called its updatePluginTabs() directly, and app-early.js's global listener set window.installedPlugins a third time and called updatePluginTabs() again. Now renderInstalledPlugins is the one publisher: it sets window.installedPlugins and dispatches pluginsUpdated once, and the full app()'s listener (app-shell.js) is the receiver. The app-early.js listener only builds the tab row while the app is not the full implementation yet. The "grid not loaded yet" case is a normal state (Plugin Manager tab not opened), so it logs through pluginLog instead of console.warn. updatePluginTabs had a "Debounce" comment and clearTimeout over a timer nothing ever set, and two identical branches; it now just calls _doUpdatePluginTabs (app-early.js detects the full implementation by that name in its source, which the new comment says). app()'s unused baseComponent lookup is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): reload the plugin list after installs and failed toggles Several callers refreshed the installed list with if (typeof loadInstalledPlugins === 'function') loadInstalledPlugins(); else if (typeof window.loadInstalledPlugins === 'function') ... but loadInstalledPlugins is local to the plugin-manager IIFE and window.loadInstalledPlugins is never defined, so from outside that IIFE both tests were false and nothing reloaded: - A failed plugin toggle left the switch drawn in the new state while the data said the old one. It now re-renders from the reverted data. The optimistic in-place edit also has to forget the grid's last-rendered markup, or setGridHtmlIfChanged sees identical HTML and skips the revert. A successful toggle still keeps the switch (and focus) as drawn. - Installing from a GitHub URL (the early handleGitHubPluginInstall), installing or uploading a Starlark app, and toggling a Starlark app on its config tab never refreshed the list, so the new app had no tab or Installed badge until the page was reloaded. They now force a reload through window.pluginManager.loadInstalledPlugins(true), and the Starlark grid redraws when that finishes instead of after a fixed 500 ms. - The Starlark uninstall inside the IIFE reloaded from the 3 s cache, which could still hold the app; it now forces a reload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): route debug output through debugLog base.html defines window.debugLog, gated on localStorage.pluginDebug. plugins_manager.js read the same key twice more into its own flags (_PLUGIN_DEBUG_EARLY, and PLUGIN_DEBUG behind a pluginLog() wrapper), and api_client.js's RequestThrottler had a separate `debug` property with a setDebug() that nothing called. All of it now goes through debugLog. The "functions defined" dumps with their ✓ lines, and two per-plugin "enabled=" loops that ran on every render, are dropped; "[PLUGINS STUB]" labels on code that has not been a stub for a long time read "[PLUGINS]". Ungated console.log calls that announced normal events on every page load or action (settings search and tooltips registering, every toast repeated to the console, the schedule pickers initialising, widget registry unregister/clear) go through debugLog too. What remains on console.log is the widget registry's on-demand LEDMatrixWidgets.debug() dump and BaseWidget.notify's no-notifier fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): drop waits and guards that could never fire - handlePluginAction polled up to 10 x 50 ms for window.togglePlugin, configurePlugin, updatePlugin and uninstallPlugin before calling them. All four are defined when the scripts load, before any card can be clicked, so the poll always succeeded at once; it now calls them. The long thinking-aloud comment over the toggle state is replaced by two lines on why the stored state, not the checkbox, decides. - initializePlugins checked typeof on setupGitHubInstallHandlers and applyStoreFiltersAndSort, function declarations in the same IIFE, and wrapped window.checkGitHubAuthStatus(), which returns a promise with its own .catch, in try/catch. - searchPluginStore wrapped each "#store-count" update (a getElementById and an innerHTML assignment) in try/catch four times; one setStoreCount() helper does it. The store's post-render re-attach of the GitHub token handler dropped its try/catch and existence checks for the same reason. - The load-time fallback outside the IIFE tested typeof initializePluginPageWhenReady, which is IIFE-local and so always undefined there; it calls window.initPluginsPage directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): delete two unused plugin-manager helpers stopOnDemand (IIFE-local; the page's stop button calls window.stopOnDemand from app-shell.js) and debounce had no callers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): document the plugin-config handlers templates call validatePluginConfigForm, handleConfigSave, handleToggleResponse, handlePluginUpdate and refreshPluginConfig each get a JSDoc naming the attribute in partials/plugin_config.html that calls it and what the return value means (only validatePluginConfigForm's matters: false cancels the submit). - The `if (!window.__pluginConfigHandlersInitialized)` wrapper is gone: app-shell.js runs once per page, so it was never false. The block is dedented one level; `git diff -w` shows the real change. - The three handlers read xhr.responseJSON first. XMLHttpRequest has no such property (it is jQuery's), so that branch never ran; one xhrJson(xhr) helper parses responseText for all of them, with the same fallbacks as before. - runPluginOnDemand and stopOnDemand checked that plugins_manager.js's openOnDemandModal/requestOnDemandStop exist; plugins_manager.js is on every page, so they call them. - fixInvalidNumberInputs had a stray "Notification helper function" comment on top of its own; a leftover "section toggle ... duplicate definition removed" note is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): one toast per save, and a failed durations save says so app.js's global htmx:afterRequest listener showed the server's message for every htmx request, and every form and button that posts through htmx (plugin config save/toggle/update, Display, Durations, General, Schedule, Dim schedule, the Overview actions) also reports its own result from hx-on after-request. Each save showed two toasts. The global listener now stays quiet for a request whose element, or its form, has its own after-request handler. That exposed the Rotation & Durations form's handler, which read xhr.responseJSON: XMLHttpRequest has no such property, so it always said "Durations saved" in green, even when the save failed (the global toast had been the only place the error showed). display.html already had a correct version (2xx only counts as saved; the server's message wins; its status may refine success but never overturn failure). That is now window.showSaveResult(xhr, savedText, failedText) in app.js, used by the Display, Durations and General forms; General's inline copy of the same logic is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(web): file headers and comments that say what the code does now - plugins_manager.js, app-shell.js, app.js and app-early.js open with a header: what the file owns, how base.html loads it and in what order relative to the others, and the globals it defines. app-early.js's app() stub also says why it exists and that, with app-shell.js now loaded before Alpine, it does not run in practice. - base.html's note on plugins_manager.js said it must load last to win over same-named functions in app.js/app-shell.js; there are none left, so it now gives the real reason (it uses everything loaded before it). - Change-narration and "already defined at the top, no need to redefine" notes are gone or rewritten as present-tense reasons; comments that were wrong are fixed ("Toggle password visibility" over the function that opens the token panel, "Insert before the closing </nav>" over an appendChild, "(from v2)", the export note that still listed escapeHtml). About forty comments that restated the line below them are removed, and a second window.currentPluginConfig = null outside the IIFE is dropped (the IIFE sets it). - The file-upload, checkbox-group and custom-feeds widgets' render() stubs say plainly that the widget is rendered server-side, instead of "for now" / "placeholder for future client-side rendering". test_plugin_action_delegation.js sliced the source up to one of the removed notes; it now ends the slice at the next section header. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): keep the escapeHtml/escapeAttribute globals for plugin pages 6da77363 removed window.escapeHtml and window.escapeAttribute because nothing in core or the plugin monorepo read them. Plugin web UIs served through serve_plugin_web_ui and third-party plugin pages may still call them, so they come back as aliases of window.LEDEscape.html and .attr, defined in app-early.js before any other script runs. test_html_escaping.js checks the aliases exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): web-frontend Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): encode the image thumbnail path; match script tags case-insensitively CodeQL flagged the upload widget building an <img> src from a stored path, and the escaper test extracting inline scripts with a case-sensitive regex. Each path segment is now URL-encoded (still a same-origin path, and correct for names with spaces or * fix(web): clear Codacy findings in the escaper, app shell and upload widget - LEDEscape looks entities up in a Map instead of indexing an object. - showNotification is declared as a global for app-shell.js. - openImageSchedule checks the index is a non-negative integer and reads the image with Array.prototype.at. - The schedule editor calls escapeHtml directly and documents why its innerHTML template is safe: every value is escaped or constrained. The remaining rule hits are suppressed on that line with the reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): build the image schedule editor with DOM calls Codacy does not honour inline suppressions, and the editor's innerHTML template kept tripping its XSS rules even though every value was escaped. The editor is now built with a small element helper (createElement and setAttribute), so no value is ever parsed as HTML, and the file's own escapeHtml goes away. Also for Codacy: - LEDEscape.attr is its own function rather than a second name for html. - The tab loader records a failed load on the panel (data-load-failed) from a named handler, instead of a closure over a local flag. The fake DOM in test_file_upload_widget.js gains append/replaceChildren, its hostile-id check now asserts the id arrives as attribute data with no innerHTML anywhere in the editor, and test_html_escaping.js drops the file-upload.js escaper it no longer has. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): schedule editor helpers as plain functions Codacy's lint flags arrow functions held in local constants and a forEach callback that returns a value. The editor's pieces are now named function declarations (displayStyle, scheduleModeOption, scheduleRangeTime, scheduleDayTime, scheduleDayRow) taking what they need as arguments, and the element helper loops with for...of. htmx is declared as a global in app-shell.js. Output is unchanged; test_file_upload_widget.js passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
3a81f38f09 |
fix(web): uniqueItems saves, /health count, Vegas order wipe; one list-repair helper (#638)
* fix(web): drop repeats from uniqueItems lists before validating a plugin save dedup_unique_arrays lost its only caller in #330, so submitting a value a uniqueItems list already holds (a stock symbol saved once and posted again) failed the whole save with a validation error. _prepare_plugin_config_for_save runs it again just before validation, which covers both POST /plugins/config and plugin sections posted to /config/main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): /health counts the discovered plugins and logs the checks it fails The plugin check counted plugin_manager.get_available_plugins(), which PluginManager does not have, behind a hasattr guard that made plugin_count 0 on every device. It now counts the discovered manifests, discovering first when nothing has been scanned yet. The config, plugin and hardware checks answered "see logs for details" without logging anything. Each now logs a warning with the traceback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): store refresh no longer claims a commit-metadata refresh POST /plugins/store/refresh read fetch_commit_info (or fetch_latest_versions) only to append "(with refreshed commit metadata from GitHub)" to its message. It never fetched any: the route re-downloads the registry and nothing else. search_plugins takes the flag, but it reads commit info through its cache, so passing it on would not refresh anything either. The flag is ignored now and the message says what happened. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): refuse a malformed Vegas plugin order instead of clearing it A vegas_plugin_order or vegas_excluded_plugins value that was not JSON, or not a list, was stored as [] and the save answered 200, so a bad value wiped the saved order or exclusions. Both now answer 400 and save nothing, the way plugin_rotation_order already did; the three share one parser. A list that holds anything but plugin-id strings is refused as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): per-plugin health and metrics read the display service's latest GET /plugins/health/<id> and /plugins/metrics/<id> called get_health_summary and get_metrics_summary without force_reload, so they answered with whatever the web process read first and kept in memory, while the display service kept writing newer state. They now pass force_reload=True, as the list routes do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): plugin config reset saves through the shared atomic save POST /plugins/config/reset called config_manager.save_config directly, so it took no backup, and a failed write escaped as an unhandled exception. It then handed on_config_change the raw stored section, not the prepared config a loaded plugin runs with. It now saves through _save_config_atomic with a backup, answers CONFIG_SAVE_FAILED when that fails, and notifies with _prepared_plugin_config, as POST /plugins/config does. POST /plugins/toggle carried its own copy of _save_config_atomic's save_config_atomic-or-save_config fallback; it calls the shared helper now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): one reading and one "unavailable" for each system metric system_metrics.collect_system_metrics() promised None for a metric it could not read, but returned cpu_temp as 0 off a Pi, and the whole no-psutil fallback as zeros. GET /system/status measured the same numbers a second time with its own code, and answered None there. Now both come from collect_system_metrics(), and "unavailable" is None everywhere. /system/status keeps its 0.1s CPU sample and its 10s cache, and gains nothing it did not already send. Two differences: without psutil it answers 200 with null metrics instead of 503, and a disk it cannot stat is null instead of a 500. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(web): /display/current sends the snapshot as-is and logs a failed read GET /display/current PIL-decoded the preview snapshot and re-encoded it before base64-ing it, spending CPU on the Pi to send the same picture, and dropped any failure with `except Exception: pass`. The /stream/display SSE stream already passed the PNG's bytes straight through. Both now read through web_interface/display_preview.py and answer with the same payload. A missing snapshot is still a null image; any other read failure is logged as a warning. /health reads the snapshot path from the same module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): one helper puts a submitted plugin config's lists back The plugin-config save turned position-keyed dicts ({"0": ..., "1": ...}) back into lists in five copies: four in the form path's fix_array_structures (whose prefix branches never ran, since no caller passed one), and _fix_json_arrays on the JSON path. It then force-fixed the news plugin's feeds.custom_feeds by name, in case the generic pass had missed it. src/web_interface/config_arrays.coerce_array_shapes now does it for both paths, custom_feeds included. ensure_array_defaults duplicated _fix_none_arrays and is gone. In the same function: the union-type re-checks that the null handling above them made unreachable, the "(temporary)" random_seed debug log, and a commented-out log line are removed. A failed validation is logged once as a warning, not four ERROR lines and a WARNING. Element types are left to normalize_config_values, which already converted them for both paths. One difference: the form path no longer adds an empty {} for a nested object the post left out that has no defaults. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): import at module top and log through the module logger The web_interface.cache imports in config.py and fonts.py were wrapped in `except ImportError` fallbacks. It is an in-repo module that imports nothing from the project, so it cannot fail to import; it is imported once at module top, as system.py now does. cache.py's docstring said blueprints import it lazily "to avoid circular imports"; it now says why that is unnecessary. Five logging.error calls in the dim-schedule GET and three logging.warning calls in plugins.py went to the root logger; they use the module logger. Function-local re-imports of json, os, shutil, logging and Path, all already imported by the module, are gone. The `import os` inside two except blocks of save_plugin_config also made os a local name for the whole function. execute_plugin_action's step-1 handler gets a comment saying why it stays: it looks like a copy of the blueprint handler, but without it a TimeoutExpired from the plugin's script would reach the route's own `except subprocess.TimeoutExpired` and be answered as a 408. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): app.py loses dead CSRF and reconciliation state, comments fixed - csrf was always None, so `if csrf: csrf.exempt(...)` never ran, and its note that the api_v3 blueprint "is exempted above" named an exemption that does not exist. Both are gone; the reason there is no CSRF protection stays, shortened. - The SSE rate-limit comment called the default "tight" at 20 per minute. The default is 1000 per minute and the streams' 200 is the tighter one; the comment now says so. The limits are unchanged. - _reconciliation_done was written and never read. The docstring that explains why reconciliation runs once keeps its reason, in the present tense. - Removed: a dangling "import cache functions" comment with no import under it, a "security check ... within project_root" label on an existence check, the "(simplified version)" narration, and the note that no redirect route is needed. The preview loop's sleep comment no longer mentions a PIL encode that the loop does not do. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(web): api_v3 comments name the package __init__, not a _common module Every route module's docstring said the shared blueprint comes "from ._common", a module the package split never created; they name the package __init__. The PROJECT_ROOT comment described the path from _common.py; it now describes this package and keeps the incident it guards against. The "(corrected) in this commit" note in resolve_pull_command and the /health comment the split's mechanical time -> _pkg.time rewrite garbled ("Stamp the start _pkg.time") read correctly again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): drop hasattr checks for attributes PluginManager always has PluginManager.__init__ sets health_tracker and resource_monitor (to None until they are configured), so the seven hasattr(api_v3.plugin_manager, ...) guards in the health, metrics and limits routes were always true. The falsy checks that do the work stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): pages_v3 dispatches partials from a dict with one error handler load_partial chose a loader through a fourteen-branch if/elif, and thirteen of the loaders then wrapped themselves in the same try/except, logging "Error loading partial" without saying which. The route now looks the name up in _PARTIAL_LOADERS and has the one handler, which logs the partial's name. The loaders just render. _load_tools_partial keeps its own messages. The search index's _partial_html already catches a loader that raises. serve_plugin_web_ui repeated _plugin_dir_for inline (containment plus the ledmatrix- prefix fallback); it calls it now. Also removed: the unused markupsafe.escape import, function-local json/Path re-imports, and unused exception bindings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): remove unused imports, locals and a try that cannot fail - get_error_aggregator was imported by the api_v3 package and used by no one; seven names config.py imported, and Path in misc.py and logging in plugins.py, likewise. - branch_info in install_plugin was built and never logged; test_config in /health was bound and never read (the load_config call is the check). - An f-string with no placeholders in the asset upload route. - _installed_plugin_ids wrapped list(manifests.keys()) in try/except; _discovered_plugin_manifests always returns a dict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): start.py logs its startup lines and drops unreachable branches The startup banner went to stdout with print(); it goes through a logger now, which the app import has already configured, so it reaches the journal with a level and timestamp like every other line. The "no addresses" branch is gone: get_local_ips() always returns at least "localhost". The except around app.run re-raised "only if it's not a client disconnection error" from inside the branch that had just established it was one, so that raise could not run. It is one check now, on a named tuple of the errnos, which the werkzeug log filter uses too. The comment on threaded=True counts three SSE endpoints, which is how many there are. Trailing whitespace is stripped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): save_main_config names its General fields once The General tab's field names were listed twice, once to detect a General form post and again, with four more, to keep the remaining-keys merge from storing them as top-level keys. GENERAL_FIELDS and _MAPPED_TOP_LEVEL_FIELDS hold them now, and the four per-section skip checks are one set. The comment on that merge said plugin configs are handled "here too", and "(including plugin keys)". Plugin sections are handled and removed from the body before it runs; the comment says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(web): plugin directories come from the plugin manager only Six lookups fell back to PROJECT_ROOT/plugins/<id> when there was no plugin manager: GET /plugins/config's of-the-day data, POST /plugins/action, the plugin static-file route, the calendar credentials upload and the calendar OAuth routes. The loader never scans plugins/ (PluginManager.discover_plugins reads only the configured directory, plugin-repos by default), so what they found there was a plugin that never runs. _plugin_directory() asks the manager and answers None without one, which each route already reports as "not found". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): web-backend Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
7b90759252 |
fix: /errors stack traces, Wi-Fi disconnect and save, plugin fonts, API cache TTL (#636)
* fix(errors): record the exception's own stack trace record_error() called traceback.format_exc(), which only sees an exception while its except block is running. plugin_executor records exceptions caught on a worker thread after that block has ended, so every trace on /errors read "NoneType: None". The trace is now built from the exception's __traceback__. The executor's log call had the same problem with exc_info=True and now passes the exception. record_error() also merged LEDMatrixError context into the caller's dict in place; it now works on a copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(wifi): point at configure_wifi_permissions.sh instead of a sudoers list The module docstring told users to grant NOPASSWD sudo on iptables and ip. configure_wifi_permissions.sh refuses those grants on purpose: a wildcard rule for either runs an arbitrary program as root. Point at the script and say why it leaves them out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(wifi): disconnect finds the saved profile by SSID disconnect_from_network() asked `nmcli -f NAME,802-11-wireless.ssid connection show` for the profile to take down, but nmcli rejects that column for `connection show`, so the lookup always failed and only the device was disconnected. The per-profile lookup _connect_nmcli() already used is now _find_profile_for_ssid(), and both callers share it. It also splits terse output on the last colon and unescapes "\:", so a profile name containing a colon is found. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(wifi): write wifi_config.json atomically and report a failed save _save_config() opened the file for writing in place and swallowed any error, so a wifi_config.json left owned by root made the web toggle for auto-enabling AP mode report success while nothing was saved, and a crash mid-write could truncate the file. It now uses atomic_write_json, which also keeps the file's owner and shared group when root saves it, and returns False on failure. POST /wifi/ap/auto-enable answers 500 in that case. The file is now written with indent=4, like the other config files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(fonts): resolve plugin:// fonts in the plugin's own directory FontManager looked for a plugin's bundled fonts under Path("plugins") / plugin_id: relative to the process cwd, and not the default install directory (plugin-repos/), so a manifest's plugin:// fonts never loaded. register_plugin_fonts() takes an optional plugin_dir, and PluginManager passes the directory it loaded the plugin from. Callers that omit it get a lookup in the configured plugin_system.plugins_directory, then plugins/, resolved against the install root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(api-helper): cache responses for the requested cache_ttl APIHelper.get(cache_ttl=...) and set_cache(ttl=...) dropped the ttl on the claim that CacheManager does not support one, but CacheManager.set() takes a ttl, stores it with the entry, and both cache tiers honour it over a reader's max_age. Without it every response expired after the 300-second default read age, whatever the plugin asked for. The ttl is now passed through, and the cache read passes cache_ttl as max_age for entries written without one. The class docstring describes what the helper actually does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(style): one scale range for the schema, element_scale and LogoHelper The generated Scale field allowed 0.1 to 10, element_style's reader capped at 10 with no floor, and LogoHelper accepted 0.05 to 8 and reset anything else to 1.0. A logo scale of 9, which the form accepts, drew at the shipped size. MIN_ELEMENT_SCALE / MAX_ELEMENT_SCALE (0.1, 10.0) in src.element_style are now the schema bounds and the clamp every reader applies through coerce_scale(): a positive number outside the range is clamped, and anything that is not a finite positive number means the default. That also stops element_scale() passing NaN through, since min(nan, 10.0) is nan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(logos): placeholder lands at the requested path; empty logos list download_missing_logo() wrote its fallback placeholder to <normalize_abbreviation(abbr)>.png in the logo directory rather than to the logo_path the caller passed, so it could return True while nothing existed where the plugin looks (e.g. "TA&M.png" vs "TAANDM.png"). create_placeholder_logo() takes an optional filepath, and download_missing_logo passes the requested one. download_missing_logo_for_team() only caught KeyError, so a team whose "logos" list is empty raised IndexError; it now treats KeyError, IndexError and TypeError as "no logo URL". The placeholder is drawn with PLACEHOLDER_SIZE / PLACEHOLDER_BG, the constants is_placeholder_logo() recognises it by, instead of repeated literals. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(fonts): resolve bundled font paths against the install root TextHelper's default font_dir, the logo placeholder's font and FontManager's font_overrides.json were all relative to the process cwd, so a process started anywhere but the install root (the plugin safety harness, a manual run, a unit without WorkingDirectory) drew with PIL's default face and read no overrides. They now go through font_layout.resolve_asset_path; the overrides file sits in the install root's config/. The resolver docstrings described an order the code does not follow: resolve_asset_path never consults the cwd, and sports_shared's _resolve_font_path tries the cwd first. Both docstrings now say what the code does, and _resolve_font_path calls resolve_asset_path instead of probing FontManager for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(sync): the web UI reads the sync status file the display writes sync_manager writes its status to tempfile.gettempdir(), but GET /api/v3/sync/status read a hardcoded /tmp/led_matrix_sync_status.json and defaulted the port to a literal 5765. Wherever TMPDIR is set (or on any non-/tmp host) the page only ever showed "starting". The endpoint now uses sync_manager.STATUS_FILE and SYNC_PORT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(http): the rankings resolver sends the project's User-Agent DynamicTeamResolver fetched ESPN rankings with a bare requests.get, so it sent python-requests' default User-Agent, which ESPN rejects; the AP_TOP_N favourites then resolved to nothing. It now sends DEFAULT_HTTP_HEADERS. BaseOddsManager carried its own copy of the User-Agent string and now uses the same shared headers (which also adds Accept-Language). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(backup): record the core release and read the configured plugin dir The manifest's ledmatrix_version came from a VERSION file that does not exist, then from .git/HEAD: a 12-character sha, or "ref: refs/he" when the branch's ref was packed. It is now src.__version__. list_installed_plugins() scanned a hardcoded plugin-repos/, so on an install whose plugin_system.plugins_directory points elsewhere, plugins missing from plugin_state.json were left out of the backup. It now reads the configured directory from config/config.json, defaulting to plugin-repos. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(startup): report a missing display section once A config without a display section produced three errors for the one problem ("Missing required configuration key: display", "Display configuration is missing or empty" and "Display configuration is missing"), and an empty one produced two. _validate_config now reports it once, as a missing key or an empty section, and _validate_display_config leaves it to that. The module docstring said the validator fails fast; nothing in the display service calls raise_on_errors(), so it now says the errors are reported and startup continues. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(wifi): share the copied blocks and name the AP constants - _parse_nmcli_wifi_list() is the one parser behind _scan_nmcli and _scan_nmcli_cached. - _verify_connected(), _wait_for_device_idle(), _failsafe_ap() and _mark_forced() replace blocks that were pasted two or three times in the connect and enable-AP paths. The device-idle wait now checks before its first one-second sleep instead of after it. - _check_command() calls _find_command_path() instead of repeating it. - AP_IP, PORTAL_PORT, AP_PROFILE_NAME and AP_PROFILE_NAMES name values that were spelled out 14, 12, 8 and 2 times; the two deletion loops now walk the same tuple. The iwconfig status path compares the AP address exactly: startswith() also skipped 192.168.4.10-19. - Dropped a second WIFI.SIGNAL query that repeated the first, a no-op "if ssid: continue", the try/except around _connect_wpa_supplicant's constant return, and a second save of a scan scan_networks already saves. - _ensure_wifi_radio_enabled's docstring says it returns True when the radio state cannot be read at all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(config): drop dead branches and history comments in ConfigManager - The module docstring pointed plugin authors at update_plugin_config(), which does not exist; it now names save_config_atomic() and save_raw_file_content(). - load_config's FileNotFoundError handler tested the message for "config_secrets.json", but a missing secrets file is handled where it is read, so only config.json reaches it; the check is gone. - save_raw_file_content's `file_type == "main" or "secrets"` guard was always true (anything else raised earlier). - get_raw_file_content('secrets') already returns {} for a missing file, so the os.path.exists() in front of two calls to it is gone. - Comments that narrated earlier behaviour are rewritten as what the code does now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(background-data): present-tense comments, drop unused API - Comments that told the history of each fix (what "used to" happen, "the old per-delivery release") now state the invariant the code keeps. - get_statistics() no longer reports a constant 'queue_size': 0, and the uncalled clear_completed_requests() is gone (_cleanup_completed_requests does that job on every completion). Neither is referenced in core, the web UI or the plugin monorepo. shutdown_background_service() has no production caller either, but it is the only way to tear down the get_background_service() singleton, which the tests rely on, so it stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(odds): drop the unread cache_ttl and merge the odds_data branches BaseOddsManager loaded base_odds_manager.cache_ttl from config and never used it: cached odds live for the update interval (get_odds' ttl=interval). No core or monorepo code reads the attribute, so it is gone along with its log line. The two consecutive `if odds_data:` blocks are one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(backup): one table for the single-file sections config, secrets, wifi and ytm_auth were each spelled out in create, preview, validate and restore. _SINGLE_FILE_SECTIONS lists them once, with the RestoreOptions flag that restores each, and all four walk it. Restore error messages keep their wording ("Failed to restore <file name>"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(fonts): drop FontManager's write-only state and duplicate logs - fonts_config, font_metadata and font_dependencies were written and never read; the performance_stats keys font_load_times, render_times, total_renders and the per-call "resolve" timings (_record_performance_metric) likewise. get_performance_stats() reads only the counters that remain. Nothing in core or the plugin monorepo references any of them. - A failed BDF load was logged twice, by _load_bdf_font and again by get_font; get_font's line is the one kept. - Removed "NEW:" and commented-out cozette entries, the "Copy font to assets/fonts" comment on code that copies nothing, and local imports of names the module already imports. The deprecated add_font() now resolves assets/fonts against the install root. The @deprecated methods stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(text-helper): cache loaded fonts; drop the pre-textlength fallback TextHelper declared _font_cache, cleared it and reported its size, but never stored anything in it. load_fonts() now keeps each (file, size) it loads there, so clear_font_cache() and get_font_cache_stats() mean what they say and repeated load_fonts() calls reuse the fonts. get_text_width() no longer catches AttributeError for Pillow releases without ImageDraw.textlength; requirements.txt pins Pillow>=12.2. The class docstring describes what the helper does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(common): fix wrong docstrings in api_helper, permission_utils, snapshot_policy - permission_utils called 0o2775 "sticky bit"; the 2 is setgid, which is what makes new files take the directory's group. - snapshot_policy pointed at web_interface/blueprints/api_v3.py, which is a package now; the health check is in api_v3/misc.py. - APIHelper.clear_cache() lost a history note and a fallback to a clear() method that neither CacheManager nor the testing MockCacheManager has. The session headers are built from DEFAULT_HTTP_HEADERS instead of a copy of them, and the module docstring says what the module offers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(sports): present-tense comments in the shared scoreboard renderers - sports_scroll and sports_game_renderer comments that referred to "this PR", "the old flat 128px card" or what the renderer "previously" did now describe the current behaviour and its reason. - The block explaining why non-finite settings are rejected sat above _score_reserve_width; it describes _center_gap_width and now lives in it. - unshare_element_fonts wrapped its import of font_layout.load_truetype in an `except ImportError` that cannot fire inside core; the import stays at call time so tests can spy on the pinned loader. - sports_card docstrings that told the history of a fix say what the code does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(sports-shared): drop dead code, name the ESPN limit - _get_weeks_data asked for limit=1000, which fetch_espn_scoreboard clamps to ESPN_MAX_LIMIT anyway; it now names that constant. Its unused `immediate_events = []` is gone. - _get_season_schedule_dates() returned ("", "") and has no caller in core or the plugin monorepo. - _should_log keeps its warning_type parameter (part of the inherited signature, though nothing in core or the monorepo calls it) and its docstring says the cooldown is shared across types. - An unused ImageFont import is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(sync): one follower-mode switch, shared panel defaults - The class docstring said the leader sends PNG frames. Frames go over UDP as raw RGB; PNG is only the Vegas scroll image sent over TCP. It now describes both paths. - _enter_follower_mode() replaces the two copies of "note the leader, switch from standalone to follower, log, write status" in the frame and scroll-position handlers. - The rows/cols fallbacks use DEFAULT_ROWS / DEFAULT_COLS from src.display_geometry, as chain_length already did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(style): drop _layout_axis, name the layout group title - ElementStyleResolver._layout_axis() had no caller in core or the plugin monorepo. - _element_block_from_spec checked spec['size'] was a dict again after size_spec already had; it reads size_spec. - The "Layout Offsets" title written into three generated schema blocks is _LAYOUT_TITLE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(logo-helper): say what the placeholder draws; name the 1.5 box factor - _create_placeholder_logo's docstring said it draws the team abbreviation; it draws an outlined grey box and nothing else. The docstring says so, and the "in a real implementation you'd want text" comments are gone. - The 1.5 x panel default logo box, written out six times, is DEFAULT_LOGO_BOX_FACTOR. - ImageDraw is imported with Image at the top of the module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(logos): drop dead code and a duplicate regex in logo_downloader - _SAFE_LEAGUE_CODE_RE was the same pattern as _SAFE_LEAGUE_RE; both checks use the one. - get_logo_filename_variations reassigned the TA&M case to the list it already had; the function returns the two names directly. - _get_team_name_variations() had no caller in core or the plugin monorepo. - fetch_single_team's docstring was copied from fetch_teams_data; a log message read "for{team_id}". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: drop the Pillow<9.1 resample shim and a catch-and-reraise - adaptive_images fell back to Image.LANCZOS/NEAREST for Pillow < 9.1; requirements.txt pins Pillow>=12.2. RESAMPLE_LANCZOS and RESAMPLE_NEAREST keep their names (src.common re-exports them). - CacheManager.save_cache caught CacheError only to re-raise it; the disk write is now called directly, with the same result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(api-helper): stop the real CacheManager's cleanup thread The cache-lifetime tests built a CacheManager and left its cleanup thread's class-wide claim on the directory in place, which broke test_cache_cleanup_thread_ownership when it ran later in the session. The fixture now stops the thread on teardown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): core-common Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
b11bcfa204 |
fix(plugins): store and plugin-manager bugs; tidy src/plugin_system (#635)
* fix(store): don't read a ZIP-installed plugin's remote from the LEDMatrix repo update_plugin looked up remote.origin.url with `git -C <plugin> config --local` for plugins that are not git checkouts. Under plugin-repos/ git walks up to the enclosing LEDMatrix repository, so the lookup returned LEDMatrix's own URL and a plugin missing from the registry was "reinstalled" from the LEDMatrix repo. Only ask git when the plugin directory has its own .git, the test _get_local_git_info already uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(schema): report each missing required field once, by name validate_config_against_schema ran its own required-fields loop after Draft7Validator.iter_errors, which already yields one `required` error per missing field, so every missing top-level field was listed twice. The validator's copy also printed the schema's whole `required` list ("Missing required property '['api_key', 'city']'") instead of the field. Drop the loop and take the field name from the error itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(store): stop mangling repository URLs that contain ".git" install_from_url and fetch_registry_from_url cleaned URLs with `rstrip('/').replace('.git', '')`, which removes ".git" anywhere: https://github.com/user/my.github.io became .../myhub.io, so installing or browsing that repository asked GitHub for one that does not exist. Add src/plugin_system/repo_urls.py with one anchored normalize_repo_url(), same_repo() for comparisons, github_owner_repo() and github_api_headers(), and use them for the five copies of the owner/repo parsing and GitHub headers in the store and for saved repositories. GitHub URLs are now recognised by urlparse().hostname everywhere: _get_latest_commit_info used a substring test, and _install_from_monorepo_api parsed any host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(store): install a repository whose only branch is not main/master _install_via_git returned None both when every clone failed and when the last-resort clone of the repository's default branch succeeded. _install_plugin_impl papered over it with `and not plugin_path.exists()`; install_from_url did not, so a repository whose only branch is e.g. `develop` was cloned, then treated as a failure, then "downloaded" from main/master archives that do not exist. After a default-branch clone, return the branch the clone checked out (read from .git/HEAD), so None means failure and nothing else, and give both callers the same `branch_used is None` fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plugins): judge the memory limit on each call's own growth monitor_call stores `metrics.memory_mb = max(previous, growth)`, and _check_limits compared that high-water mark with max_memory_mb. It never decreases, so once one update() grew the process past the limit every later call raised ResourceLimitExceeded and the circuit breaker kept reopening. Pass the call's own RSS growth to _check_limits; keep the high-water mark for reporting and document what it measures. Remove ResourceMetrics.update_average_execution_time: nothing called it, and it overwrote the running total with the average. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plugins): reload_plugin re-reads the manifest from the discovered directory reload_plugin read `plugins_dir / plugin_id / "manifest.json"`, ignoring the discovery map and the plugin_dirs rules. For a plugin whose directory name differs from its manifest id the path did not exist, the re-read was skipped without a word, and the reload kept the stale manifest. Resolve the directory with find_plugin_directory, as load_plugin does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plugins): drop the always-null last_display from plugin state info PluginStateManager reported `last_display` from `_last_display`, which nothing ever wrote, so it was null for every plugin. Recording it in PluginExecutor.execute_display would not help: get_state_info's only reader is the web process, whose PluginManager never calls display(). Remove the field, its dict and get_last_display() (no caller in core, the web UI or the plugin monorepo). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(store): share the rollback and requirements helpers, drop dead code - install_plugin and _reinstall_with_rollback set aside, discard and restore the old copy through _set_aside/_discard_backup/_restore_backup instead of two copies of the same blocks. - The loader and the store run the same pre-pip checks through contained_plugin_dir() and requirements_to_install() in plugin_loader. They still invoke pip differently (sys.executable -m pip vs. the sudo wrapper). `except (BrokenPipeError, OSError)` + `isinstance(e, OSError)` becomes `except OSError` checking errno.EPIPE. - load_module never returns None, so load_plugin's check is gone and the docstring says what it raises. - Remove the always-true JSONSCHEMA_AVAILABLE, the inline re-imports of re and permission_utils, the fake status_result object nobody reads, hasattr(git_error, 'cmd'), a redundant "merge conflict" test and `import traceback` (exc_info=True does it). - Correct comments: install_from_url names the directory for the caller's id when given (not always the manifest id), _get_local_git_info saves one git subprocess (not four), _enrich calls two helpers, search_plugins documents all its arguments, _find_plugin_path states its behaviour instead of a TODO, and history narration is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(plugins): tidy base_plugin, correct plugin_manager/state comments - base_plugin: drop the unused `import logging`; get_display_duration runs the instance value and the config value through one _positive_seconds() helper instead of two copies of the coercion; the 'static'/'none'/fallback branches of get_vegas_display_mode, which all returned FIXED_SEGMENT, are one; fix the mis-indented validate_config example; say that get_supported_vegas_modes/get_vegas_segment_width are not consulted by core (kept, plugins override them). - schema_manager: import expand_style_elements normally rather than swallowing an ImportError of a core module. - plugin_manager: the plugins directory is the configured one (plugin-repos/ by default), not plugins/; get_config() returns the live dict, not a copy, so the interval cache comments say what it saves. - state_manager: config_version and the file version are not used to detect corruption; say what they are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(plugins): stop writing data/plugin_operations.json PluginOperationQueue wrote its finished-operation history to data/plugin_operations.json after every operation, and read it back only into its own in-memory list, which only get_operation_history() exposes -- and nothing calls that. The operation-history endpoint reads OperationHistory (data/operation_history.json). No code in src/, web_interface/, scripts/ or test/ reads the file. Drop the history_file/lazy_load parameters and the load/save code; the bounded in-memory history stays. web_interface/app.py and the integration test stop passing the removed arguments. An existing data/plugin_operations.json is left in place (data/* is gitignored). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): plugin-system Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
3967a6cffc |
fix(security): re-harden root sudo helpers; installer fixes; ARCHITECTURE and PERMISSIONS docs (#640)
* docs: add ARCHITECTURE and PERMISSIONS guides ARCHITECTURE.md maps the processes, the state the display and web services share through the cache, the display loop, the plugin system, the web UI and the update path, with links into the code and a where-to-start table. PERMISSIONS.md lists who owns what after install, both sudoers files (and why iptables is not granted), the polkit rule, and which scripts/fix_perms script to run as which user. Both are linked from the docs index, along with the MQTT bridge README and src/common/README.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: correct stale setup, service and troubleshooting claims - README: quick actions run systemctl on ledmatrix.service (run.py), not display_controller.py; use_short_date_format has no effect; the installer uses system pip with --break-system-packages, not a venv. - CONFIG_DEBUGGING: LEDMATRIX_DEBUG must be "true"; logs are in journald. - GETTING_STARTED, WEB_INTERFACE_GUIDE, TROUBLESHOOTING: enabling a plugin, plugin settings, brightness and Vegas settings apply without a restart; matrix hardware settings still need one. - TROUBLESHOOTING: install dependencies with sudo so the root service sees them; point permission problems at PERMISSIONS.md instead of a project-wide chown. - ADVANCED_FEATURES: real BackgroundDataService stats keys; Vegas hooks return VegasDisplayMode and None falls back to capture; cache files are 0660; fix_web_permissions.sh runs as the web user and does not touch sudoers. - STARLARK_APPS_GUIDE: only the linux-arm64 pixlet binary is downloaded. - HOW_TO_RUN_TESTS: test class examples that exist. - CLAUDE.md: PluginStoreManager, plugin_dirs.py, monorepo installs via the Trees API with ZIP fallback, requirements.txt is optional. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: mark deprecated plugin APIs and state manifest fields once Methods @deprecated("3.7.0") (the set pinned in test_deprecation.py) were shown as current API in the quick reference, API reference, advanced guide, development guide and FONT_MANAGER. Each is now marked deprecated with its replacement. FONT_MANAGER is rewritten around the current API; the override editor is gone and override methods are deprecated. Required manifest fields were stated three different ways. The API reference now has one section: the 7 schema-required fields, the 4 the store refuses without, class_name for the loader, and the 8 to set. The other guides link to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: document every src/common module and every widget - src/common/README.md covered 7 of 17 modules. It now has a table of all of them (purpose, whether plugins import it, release to floor on), a short entry each, and logging advice that matches the code. - SPORTS_UNIFICATION listed two shared modules and called sports_helpers the first; it now lists all six. - The widgets README lists all 28 registered widgets plus the support files, and absorbs the parts that only docs/widget-guide.md had (x-options.labels, x-advanced, x-display hidden, plugin-file-manager). docs/widget-guide.md is now a pointer to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(security): fix_web_permissions.sh re-hardens the root sudo helpers The script chowns the whole project to the web user. That included scripts/fix_perms/safe_plugin_rm.sh and safe_pip_install.sh -- the two helpers /etc/sudoers.d/ledmatrix_web lets the web user run as root -- so running it turned both into a root shell for whoever can edit them. It also re-grouped config_secrets.json away from ledmatrix. After the chown it now does what first_time_install.sh's Steps 11 and 11.1 do: helpers back to root:root 755, and config_secrets.json back to the web unit's User=:ledmatrix 640. Each step is non-fatal and prints the manual command if it fails. Also fixes what the script and its docs claimed: it never configured sudoers, its closing hint pointed at ./configure_web_sudo.sh (wrong path), and the README and ADVANCED_FEATURES.md said to run it with sudo, which it refuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(security): validate and harden every sudoers drop-in the scripts write configure_wifi_permissions.sh copied its rules into /etc/sudoers.d/ledmatrix_wifi without `visudo -c`. A malformed drop-in makes sudo refuse every command for every user, which on a headless Pi leaves no way back in. It now checks first and leaves the installed file alone when the rules do not parse, as the other two writers do. (It already used mktemp, so that part of the review did not apply.) It also grants the two literal commands wifi_manager.py runs for NetworkManager's shared-mode dnsmasq drop-in -- `cp /tmp/ledmatrix-nm-dnsmasq.conf .../dnsmasq-shared.d/ledmatrix-captive.conf` and `rm -f` of that file. The directory's mkdir was granted, the file was not. Both are pinned in test_sudo_allowlist_covers_calls.py. configure_web_sudo.sh wrote its rules to /tmp/ledmatrix_web_sudoers_$$, a predictable name in a world-writable directory; it now uses mktemp with an EXIT trap, as first_time_install.sh does. It sets mode 440 on the installed file instead of leaving the temp file's mode, and finds visudo in /usr/sbin when that is not on the user's PATH, which skipped the check silently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(install): escape the project path in the DNS-fix and MQTT unit renderers install_dns_fix.sh and install_mqtt_bridge.sh substituted __PROJECT_ROOT_DIR__ with the raw path, while the other three renderers go through sed_escape_replacement from lib_systemd_render.sh. A checkout under a path containing `&`, `\` or `|` rendered a corrupted unit from these two only. Both now source the helper and use it, and a test checks that every placeholder substitution in scripts/install uses an escaped value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(install): stop the installer scripts reporting things that are not true - first_time_install.sh printed "Password: ledmatrix123" for the setup access point. wifi_manager creates it as an open network ("No password" on the panel), so it now says so. - Step 10.1 printed "✓ WiFi management permissions configured" straight after its own failure message; install_wifi_monitor.sh printed "✓ Package installation completed" after a failed apt install. The tick now only follows success. - Step 7 printed "Web dependencies already installed ... in Step 5" in the one branch that runs because Step 5 did not install them, then created .web_deps_installed on that basis. It now warns and leaves the marker off so the next run retries, as the comment below it intends. - check_system_compatibility.sh called Debian 12 Bookworm "full compatibility confirmed" while first_time_install.sh refuses anything but Debian 13. Bookworm, older Debian and non-Debian systems are now errors. Its counters used ((X++)), which under `set -e` exits the script at the first warning or error (the expression is 0), so the check never reached its summary on any system with one. - configure_web_sudo.sh and configure_wifi_permissions.sh finished by testing `sudo -n test -f ...` and `sudo -n nmcli device status`, neither of which is granted, so they always reported a failure. They now ask `sudo -n -l` about commands the new rules do grant, which checks the rule without running anything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(install): print the completion summary before rebooting With -y -- and so for every one-shot `curl | bash` install, which always passes -y -- first_time_install.sh ran `reboot` about 180 lines before its "Installation Complete / Web UI Access" summary. reboot returns at once, so the summary printed while the Pi was going down and the SSH session usually dropped before the web UI address could be read. The reboot block moves, unchanged, to the very end of the script. The interactive prompt now also follows the summary. Because the summary now runs before the -y reboot, its one command that could fail under `set -Eeuo pipefail` (the SSID lookup, when nmcli reports a connected device but no active network line) gets `|| true`; a missing SSID was already handled as "SSID unknown". one-shot-install.sh prints its "Next steps" after the installer returns, by which time the reboot is under way, so it now says so, and README's Quick Install mentions the automatic reboot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(scripts): correct wrong comments and messages, drop dead code No behaviour change except the output text noted below. - 2775 is setgid, not the sticky bit (first_time_install.sh Step 3.1, fix_plugin_permissions.sh), and root needs no "PWM hardware access" to plugin files. - The 777 comments in first_time_install.sh Step 3's fallback and fix_assets_permissions.sh said root needs it to write. Root ignores mode bits; the comments now say what 777 actually opens. The 777 itself is unchanged. - apt_remove ends in `|| true`, so Step 12's "Some packages could not be removed" branch could never run; it is gone and the helper stays non-fatal. - detect_web_service_user's comment named Step 8 for the web unit (install_service.sh installs it in Step 7.5) and now says which branch actually runs. - Step 5 described an "already installed" check that does not exist; the ACTUAL_USER comment described the re-exec backwards. - on_error printed a literal "\n" before "Common fixes:". - Dead code: one-shot-install.sh's uncalled fix_tmp_permissions, LEDMATRIX_ELEVATED=1 (never read) on the sudo re-exec, and configure_web_sudo.sh's unused PYTHON_PATH, which also made a missing python3 fatal for rules that never mention it. - start_display.sh / stop_display.sh said "for user: <you>"; the service runs as root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(fix_perms): fix_cache_permissions.sh uses setup_cache.sh's model There were two models for /var/cache/ledmatrix. setup_cache.sh (the installer's Step 2) and install_web_service.sh share it through the ledmatrix group: root:ledmatrix, 2775, files 660, which is also what DiskCache relies on to give files the directory's group. fix_cache_permissions.sh instead made it 777 and re-grouped it to the invoking user's group, undoing that. It now runs setup_cache.sh for /var/cache/ledmatrix and keeps its own handling of ~/.ledmatrix_cache. Dropped: /var/cache/ledmatrix/ placeholder_logos (nothing reads it) and the checks against the `daemon` user (no service runs as daemon). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: pin actions/checkout in the Claude workflows, drop template comments claude.yml and claude-code-review.yml used actions/checkout@v4 while test.yml and release-version-check.yml pin the v4.2.2 commit SHA; they now pin the same SHA. The commented-out starter-template settings (prompt, claude_args, paths, author filter) are removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(scripts): index every script and list removal candidates New scripts/README.md gives one line per top-level script and scripts directory, marked keep, dev-only or diagnostic, and lists the eight scripts nothing in the repo refers to as candidates for removal (kept for now). The install, utils and dev READMEs now list the files they were missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: tighten two checks that mutation testing showed were too loose - The wifi sudoers check matched `visudo -c -f "$TEMP_SUDOERS"` in the error report too, so replacing the check with `if false` still passed. It now requires the command as the condition. - The summary test never had the setup access point up, so reinstating the bogus "Password: ledmatrix123" line went unnoticed. A case with hostapd active now checks the AP is described as open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(permissions): describe the repaired fix_perms scripts and new WiFi grants Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): docs-scripts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
4e61d7248a |
refactor(web): one error-response path for api_v3 (#624)
* refactor(web): answer unhandled api_v3 errors from one blueprint handler
Fifty-three api_v3 routes ended in a copy of the same catch-all: log the
traceback, return {status, "An error occurred; see logs for details",
details: describe_exception(e)} with a 500. They are replaced by one
errorhandler on the api_v3 blueprint that returns exactly that body.
It lives on the blueprint rather than falling through to app.py's global
handler because the two answers differ: the global one adds
error_code: UNKNOWN_ERROR, and api_client.js sends a body with an
error_code to the error modal and one without to a plain toast. A
blueprint handler also gives tests that mount api_v3 on a bare Flask app
the same answer the real app gives.
Only handlers that were byte-for-byte that shape were removed (matched on
the AST, and each rewritten function re-parsed and compared). Handlers
with their own message, extra keys, operation-history records or cleanup
stay, as does execute_plugin_action's step-1 handler, which sits inside
an `except subprocess.TimeoutExpired` arm that would otherwise turn a
plugin's timeout into a 408.
HTTPExceptions raised inside a route go back as themselves in the global
handler's 4xx shape. Where a removed catch-all used to swallow one (only
delete_plugin_asset's non-silent get_json() is reachable), a malformed
request now gets its 415/400 instead of a 500.
Most of the diff is re-indentation from unwrapping the try blocks;
`git diff -w` shows the real change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): plugin action errors name the real failure, not UnboundLocalError
execute_plugin_action bound a local `logger` in its JSON-parsing arm,
which made `logger` local to the whole function. Every other
`logger.error` in it then raised UnboundLocalError, so a failing OAuth
step-1 script was reported as "UnboundLocalError: cannot access local
variable 'logger'" -- from the step-1 handler, and before the previous
commit from the route's outer catch-all too. Use the module logger.
Found by comparing every api_v3 route's forced-failure response before
and after the catch-all consolidation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(web): drop the error category and exception-name code guessing
WebInterfaceError derived an ErrorCategory from every error code and put
it in each structured error body as `error_category`. Nothing reads it:
not the web UI (static/ and templates/), not the tests beyond the ones
pinning the mapping itself, and not any plugin in ledmatrix-plugins. The
enum, the inference table and the JSON key go.
from_exception() could also guess an error code from the exception's
class name ("Config" -> CONFIG_LOAD_FAILED, and so on). Every caller
passes a code, so the guess never ran; error_code is now required.
suggested_fixes stays: the error dialog in static/v3/js/utils/
error_handler.js lists them.
The REST reference loses error_category and says what an unanticipated
exception in an /api/v3 route answers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(web): one call for the from_exception error responses
Nine plugin routes built a structured error by hand:
from src.web_interface.errors import WebInterfaceError
error = WebInterfaceError.from_exception(e, ErrorCode.X)
return error_response(error.error_code, error.message,
details=error.details, context=error.context,
status_code=500)
That is now exception_error_response(e, ErrorCode.X) in api_helpers, so
error_response() is the only structured-error entry point the routes
use. The three operation-history routes never passed the context, and
with_context=False keeps their bodies exactly as they were; a test
compares the helper against the hand-written pair for both forms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): one api_v3 error-response path
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5.5
|
ece416c4e5 |
refactor(plugins): one plugin-directory resolver (#623)
* refactor(plugins): one resolver for plugin id -> directory Five places mapped a plugin id to its directory, each with its own rules and each re-reading manifests per lookup: PluginManager discovery and get_plugin_directory, PluginLoader.find_plugin_directory, PluginStoreManager._find_plugin_path / list_installed_plugins, and state_reconciliation.disk_plugin_ids. They disagreed on backup dirs, on whether the manifest id or the directory name is the id, on duplicate ids and on path safety. src/plugin_system/plugin_dirs.py now holds the rules once: PluginDirectoryIndex scans one directory and reads each manifest once; resolve_plugin_dir() searches directories in order. What legitimately differs per caller is an explicit argument: search dirs (discovery and the loader: configured dir only; the store: configured then sibling plugins/), ledmatrix- prefix (not for the store), case folding (loader only), manifest pass (not for get_plugin_directory, whose discovery map already holds it). Behaviour changes, all for layouts installs do not produce: - a directory whose manifest declares the id beats one merely named for it (discovery already worked this way; the loader and store now agree) - the store searches the configured dir completely before plugins/ - backup and hidden dirs are skipped everywhere (the loader's case and manifest scans and list_installed_plugins used to return them) - duplicate ids resolve deterministically (exact name, then ledmatrix-<id>, then by name) with a one-time warning; discovery no longer lists the id twice - disk_plugin_ids / list_installed_plugins report manifest ids, falling back to the directory name; auto-update looks the directory up - ids that are not one plain path segment resolve to nothing in every caller (the loader used to truncate them, the store to join them) The .standalone-backup- marker is one constant, BACKUP_MARKER, used by store_manager's rename-aside names and every lookup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(changelog): one plugin-directory resolver Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
13bbb537f3 |
refactor(web): one logging setup and one TTL cache for the web process (#621)
* refactor(web): use src.logging_config in the web process; routine requests to DEBUG
The web interface had its own logging setup (web_interface/logging_config.py)
that replaced the root handlers with a plain stdout formatter. The web
service's journal lines therefore never carried a syslog priority, so
`journalctl -p err -u ledmatrix-web` returned nothing while errors were
logged, and the line shape differed from the display's (the log viewer's
prefix stripping only matched the display format). It also ran after the
module-level managers were built, so their INFO lines at import (including
"Re-removed N uninstalled plugin(s)") were dropped.
app.py now calls src.logging_config.setup_logging() first thing, the same as
run.py: journald priorities under systemd, LEDMATRIX_DEBUG honoured,
LEDMATRIX_JSON_LOGGING still selects JSON.
Per-request logging moves to web_interface/request_logging.py. Every request
used to be logged at INFO, so the UI's polling filled the journal
("GET /api/v3/errors/summary - 200" every minute per tab). Now a successful
GET/HEAD/OPTIONS is DEBUG, a successful write is INFO, 4xx WARNING, 5xx
ERROR. Durations use perf_counter and print to 0.1ms.
The duplicate module is deleted; nothing else imported it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(web): one thread-safe TTL cache for the web process
web_interface/cache.py becomes a small TTLCache class (lock-guarded,
monotonic clock) with the existing get_cached/set_cached/delete_cached/
invalidate_cache helpers kept on top of a shared instance, so the api_v3
callers are unchanged.
Bugs fixed:
- set_cached(ttl_seconds=...) ignored its TTL; only the reader's value
counted and get_cached defaulted to 60s. An entry now expires after the TTL
it was stored with; a reader's ttl_seconds can only shorten that. Both
current callers pass the same value on both sides (fonts_catalog 300s,
system_status 10s), so their observable TTLs are unchanged.
- get_cached deleted expired keys without a lock; two threads reading the
same expired key could raise KeyError (reproduced), which the endpoints
turned into a 500.
app.py's two hand-rolled systemctl caches (_ap_mode_cache, 30s, and
_ledmatrix_service_cache, 15s) now share one helper over a private
TTLCache, with the same TTLs. The AP-mode check used to retry on every
request after a failure (and log an ERROR each time); a failure now keeps the
last known answer for the TTL, as the display-service check already did. With
no systemctl at all (a dev machine) it answers False without forking.
Left alone as not TTL memoisation: the gzip cache (size-bounded, keyed by URL
and version), the settings search index (keyed by installed-plugin set), the
widget bundle (keyed by file fingerprint) and CacheManager (cross-process).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(changelog): web logging and TTL cache
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): only ask systemctl about known units
Codacy flagged the systemctl argv built from a variable. The unit now has
to be one of two literals, and anything else raises.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(web): response_time_ms reads the same clock request_logging stamps
request_logging now stamps request.start_time from perf_counter, but
success_response still subtracted it from time.time(), so metadata
reported ~1.8e12 ms. Found testing on ledpi.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5.5
|
c1ce0b7b04 |
fix(web): two api_v3 paths called names that no longer exist (#625)
The Pixlet editor stop route restarts the display after a SIGKILL with _run_systemctl_command, which starlark.py never imported (since #554). The Starlark device-location resolver fell back to _ensure_cache_manager, which #609 deleted; the resolver already accepts no cache manager. Both raised NameError on the rare path that reaches them. pyflakes finds no other undefined names in src/ or web_interface/. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
f3894916a9 |
feat(web): show which plugins use each font; warn before deleting one (#619)
* feat(web): show which plugins use each font, warn before deleting one
The Fonts tab lists font files from the web process's own scan, and the
plugins that register fonts run in the display process, so the tab had no
way to say whether a font was in use before deleting it.
The display service now publishes {catalog key: [plugin ids]} to the
shared cache (font_usage_snapshot, src/font_usage.py), built from the
loaded plugins' FontManager.register_manager_font() registrations. A
daemon thread checks every 10 s and writes only when the usage changed
(plus a daily refresh so cache cleanup cannot expire it); it never raises.
Families, aliases (press_start, four_by_six, ...) and paths are resolved
through FontManager's catalog to the file stem the Fonts tab keys rows by;
fonts outside assets/fonts are left out. Unloading a plugin drops its
registrations (new FontManager.forget_manager_fonts).
GET /api/v3/fonts/catalog merges used_by into each row per request (the
5-minute scan cache is copied, never edited): a list of plugin ids, or
null when the display service has not reported. The tab shows a Used by
column ("unknown" / "-" / ids, rendered as text) and deleting an in-use
font names the plugins in the confirmation, from a fresh read. The server
still refuses only system fonts. Catalog fetches bypass the browser's
5-second API cache, which otherwise served the pre-delete list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix: call forget_manager_fonts through a hasattr check pylint can follow
getattr(..., None) then callable() is fine at runtime, but pylint's E1102
("not callable") can't see through it, and Codacy fails the check on it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5.5
|
9a1f94f793 |
fix(starlark): blank app locations use the device location, not San Francisco (#617)
* fix(starlark): blank app locations use the device location, not San Francisco A Starlark (Tidbyt) app whose Location field is blank rendered at its author's hard-coded DEFAULT_LOCATION -- usually San Francisco -- even with the device city set under General settings. A user in Charlotte, NC got San Francisco weather and radar with nothing in config.json to explain it. src/device_location.py fills unset location fields at render time (display plugin and the web standalone render): the device city is geocoded once via Open-Meteo, preferring a match in the configured state/country, and cached permanently. A saved location always wins; if the lookup fails the field is dropped so the app uses its own default, and the failure is not retried for 30 minutes. Also fixes the config form: clearing a location omitted the key, and the save merges, so the old value could never be removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(starlark): say what happens when the device location can't be used A blank app Location only renders at the device's city when one is set and the Open-Meteo lookup finds it. With no city, no match, or the geocoder unreachable (retried after 30 minutes), the app gets no location and keeps its author's default. The guide, the config page hint, CONFIG_REFERENCE and the CHANGELOG entry now say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
61e462c635 |
refactor: remove the skin system and the unused src/base_classes package (#615)
* refactor: remove the skin system Skins never rendered with the current scoreboard plugins: the only hook was SportsCore._render_game in src/base_classes, which no plugin builds on, so the UI and store already treated them as unsupported. The owner decided on 2026-09-23 to remove them outright. Removed src/skin_system/ (runtime, base class, fixtures), skins/, scripts/validate_skin.py and their tests; the store's "type": "skin" installer, uninstaller and hide/refuse filters (the official registry lists no skins); SchemaManager.inject_skin_selector; and GET /api/v3/skins. Stored skin/skin_options config values are handled in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(config): drop retired skin/skin_options keys instead of validating them A config.json written while the skin system existed can carry skin and skin_options in any plugin section, and most plugin schemas set additionalProperties: false. They are no longer core plugin properties; RETIRED_PLUGIN_KEYS in schema_manager lists them and drop_retired_plugin_keys removes them (unless the plugin's own schema declares the name) in prepare_plugin_config, which loading, hot reload, GET /plugins/config and both web saves already share, and in validate_config_against_schema for callers that validate a raw section. POST /plugins/config and /config/main also drop them from the stored section they merge into, so they leave config.json on the next save. Tests cover the load path (real PluginManager.load_plugin: no schema warning, not degraded), raw and prepared validation, validate_all_plugin_configs, and the JSON, form and /config/main saves. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: remove the unused src/base_classes package No scoreboard plugin builds on src.base_classes: the nine monorepo scoreboards ship their own sports.py and share code through src/common (docs/SPORTS_UNIFICATION.md), and none of the third-party registry plugins imports it. The one import anywhere, baseball-scoreboard's rankings_manager.py, is a lazy import of ESPNDataSource in a class nothing instantiates. Removed the package and the eight test files that only tested it (test_api_extractors, test_data_sources, test_sports_base_characterization, test_sports_capabilities, test_sports_core_promotions, test_sports_logo_cache_bounded, test_sports_modes_promotions, test_sports_odds_fanout). test_common_is_hardware_free no longer lists src.base_classes as a forbidden import, and comments in sports_helpers.py and base_odds_manager.py stop pointing at it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: drop the skin system and src/base_classes from the docs Deletes docs/SKIN_SYSTEM.md and docs/CREATING_SKINS.md and every link to them (docs/README.md, README.md, PLUGIN_DEVELOPMENT_GUIDE.md, the /skins section of REST_API_REFERENCE.md), the skin section of CLAUDE.md and the term in PRODUCT.md. SPORTS_UNIFICATION.md now says src/base_classes was removed and shared code lives in src/common, in the Layering section and the view-model-contract rule. Other docs stop pointing at the removed package. CHANGELOG records both removals under Unreleased. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(store): hide and refuse registry entries that aren't plugins The skin filters went with the skin system, but a custom registry can still list "type": "skin" entries, and installing one as a plugin would unpack it into the plugins directory. PluginStoreManager.is_plugin_entry() (a missing type means plugin) now hides non-plugin entries from the store and custom-registry listings, and install refuses them, in the route with a clear 400 and in _install_plugin_impl for any other caller. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5
|
4fe3cdd906 |
fix(starlark): stop the root display service locking the web UI out of starlark-apps (#604)
* fix(starlark): stop the root display service locking the web UI out
Reported after a fresh install: installing an app from the Starlark tab
failed with "install failed: Failed to install from repository", and so did
uploading a .star file and installing from a GitHub directory. The reporter
found the cause only by reading service logs, and fixed it with
sudo chown -R ledpi:ledpi /home/ledpi/LEDMatrix/starlark-apps
starlark-apps is gitignored, so it is never checked out -- it is created
lazily by whichever process reaches it first. Those processes run as
different users. systemd/ledmatrix.service is User=root and constructs this
plugin at startup, which is where _get_apps_directory() is called from;
systemd/ledmatrix-web.service runs as the login user and is what actually
installs apps.
The documented first step is to install pixlet and reboot, so on a fresh
machine the display service usually wins that race and mkdir() leaves the
directory root-owned. The web process then fails in _install_star_file() on
app_dir.mkdir(), which catches nothing, so PermissionError reaches the
route's outer `except Exception` and becomes the generic message the user
saw. All three install paths write to the same directory, which is why all
three failed.
The web user cannot repair this -- chown needs root. So root does it, on
every startup, which also heals machines already broken by this without the
owner having to find the chown themselves. It is a no-op when not root, when
the platform has no POSIX ownership, and when the checkout genuinely belongs
to root; a chown that fails warns rather than killing startup.
Also made the failure legible if the handover is ever prevented: a
PermissionError now names the directory, the automatic repair, and the
manual chown, instead of a message that names neither path nor cause.
Verified by mutation: dropping the handover call, chowning a genuinely
root-owned checkout, and letting a non-root process chown each fail their
own test. 121 starlark tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9
* fix(starlark): address the review on the ownership repair
Findings from the automated review of #604.
Symlinks (CWE-59, the serious one). A root chown that follows links is a
privilege-escalation primitive: anyone able to write in starlark-apps could
point a link at a root-owned file and have the repair hand it over. Entries
are now read with os.lstat, symlinks are skipped outright, and the chown
passes follow_symlinks=False. Descendants are processed before the directory
itself, so the container does not change hands while its contents are still
being walked.
install_app() caught PermissionError in its broad handler and returned
False, which both routes report as a generic install failure -- the exact
shape of the bug this PR exists to fix, since the caller could not tell
"this app is broken" from "this process cannot write here". PermissionError
is now re-raised; every other failure still returns False.
The test fixtures skipped on bare Exception, which would have turned a
syntax error or NameError in the plugin into a green run. They now skip only
for a named absent dependency and re-raise anything else.
Also fixed the _Stat stub that failed in CI but passed locally: it carried
only st_uid/st_gid, and pathlib reads st_mode while walking. It now wraps
the real stat result and overrides ownership alone.
NOT taken: the CodeQL "information exposure through an exception" finding on
the hint response. Dropping `details` would contradict this package's
documented rule -- "if it returns 5xx, it says why" -- which
test_no_api_v3_handler_discards_its_exception enforces with an allowance
that may shrink and never grow. The Starlark routes are the ones that policy
was written for: they answered 500 with no detail for three releases.
describe_exception already redacts credentials and truncates. Keeping the
detail is the deliberate trade-off, so the finding is declined rather than
silently worked around.
Verified on hdpi with the updated code: a symlink to /etc/shadow planted in
starlark-apps was skipped while the directory was handed back, and
/etc/shadow stayed root:shadow.
Mutation-checked all three behaviours. The symlink test was vacuous on the
first attempt -- the link already had the target owner, so it was skipped
for the wrong reason and the mutation passed. It now forces the link to look
like it needs handing over, and fails when the check is removed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
ChuckandClaude Opus 5.5
|
4a1fd7464a |
fix(errors): serve /api/v3/errors/* from the display service; add a Plugin errors panel (#614)
* fix(errors): serve /api/v3/errors/* from the display service's aggregator The error aggregator is a per-process singleton and only the display service runs plugins, so only its aggregator records anything. The routes read the web process's own, empty one and always reported no errors. The display service now publishes a bounded snapshot of its aggregator to the shared cache (plugin_error_snapshot) from a daemon thread: at most once every 10 s and only when something changed, never raising into the caller. The routes read it and keep their response shapes, adding snapshot_available, generated_at and clear_pending; exception text has credentials redacted. POST /errors/clear writes a clear request (plugin_error_clear_request) that the display applies on its next 5 s tick via the new clear_before(), which keeps errors recorded after the cutoff and rebuilds the counts. Until the snapshot acknowledges the request, reads hide everything before the cutoff, so a snapshot written just before the click cannot bring errors back. Adds "all": true; cleared_count is null when only the display can know it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(web): show plugin errors in the Logs tab A compact panel under the log viewer: per-plugin error counts, repeating errors (type, count, affected plugins, a sample message, last seen) and a Clear button, with empty states for "no errors" and "display service hasn't reported yet". Polls every 15 s while the tab is active; all text goes through escapeHtml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: describe where plugin error reports come from and how clear works Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(errors): redact the published snapshot before clipping it Keeping only a traceback's tail (or clipping a message) could cut an `api_key=` marker off while keeping the secret after it, and the web side's redaction would then have nothing to match. The display now redacts every free-text field of the snapshot first. The patterns move to a Flask-free src/redaction.py so the display service can use them; redact_text in the web error handler uses the same function, unchanged in behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ChuckandClaude Opus 5.5
|
f813ea2117 |
fix(web): define project_root when plugins_directory is absolute (#616)
project_root was only assigned in the relative-path branch, so an absolute plugin_system.plugins_directory made web_interface/app.py raise NameError at import (first use: the SchemaManager construction). Define it before the if/else; plugins_dir resolution is unchanged. Adds a regression test that imports the real module in a fresh interpreter with an absolute and a relative plugins_directory. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |