20 Commits

Include renames

Author	SHA1	Message	Date
ChuckandClaude Opus 5.5	9a1f94f793	fix(starlark): blank app locations use the device location, not San Francisco (#617) ... * fix(starlark): blank app locations use the device location, not San Francisco A Starlark (Tidbyt) app whose Location field is blank rendered at its author's hard-coded DEFAULT_LOCATION -- usually San Francisco -- even with the device city set under General settings. A user in Charlotte, NC got San Francisco weather and radar with nothing in config.json to explain it. src/device_location.py fills unset location fields at render time (display plugin and the web standalone render): the device city is geocoded once via Open-Meteo, preferring a match in the configured state/country, and cached permanently. A saved location always wins; if the lookup fails the field is dropped so the app uses its own default, and the failure is not retried for 30 minutes. Also fixes the config form: clearing a location omitted the key, and the save merges, so the old value could never be removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(starlark): say what happens when the device location can't be used A blank app Location only renders at the device's city when one is set and the Open-Meteo lookup finds it. With no city, no match, or the geocoder unreachable (retried after 30 minutes), the app gets no location and keeps its author's default. The guide, the config page hint, CONFIG_REFERENCE and the CHANGELOG entry now say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-23 16:48:02 -04:00
ChuckandClaude Opus 5	4fe3cdd906	fix(starlark): stop the root display service locking the web UI out of starlark-apps (#604) ... * fix(starlark): stop the root display service locking the web UI out Reported after a fresh install: installing an app from the Starlark tab failed with "install failed: Failed to install from repository", and so did uploading a .star file and installing from a GitHub directory. The reporter found the cause only by reading service logs, and fixed it with sudo chown -R ledpi:ledpi /home/ledpi/LEDMatrix/starlark-apps starlark-apps is gitignored, so it is never checked out -- it is created lazily by whichever process reaches it first. Those processes run as different users. systemd/ledmatrix.service is User=root and constructs this plugin at startup, which is where _get_apps_directory() is called from; systemd/ledmatrix-web.service runs as the login user and is what actually installs apps. The documented first step is to install pixlet and reboot, so on a fresh machine the display service usually wins that race and mkdir() leaves the directory root-owned. The web process then fails in _install_star_file() on app_dir.mkdir(), which catches nothing, so PermissionError reaches the route's outer `except Exception` and becomes the generic message the user saw. All three install paths write to the same directory, which is why all three failed. The web user cannot repair this -- chown needs root. So root does it, on every startup, which also heals machines already broken by this without the owner having to find the chown themselves. It is a no-op when not root, when the platform has no POSIX ownership, and when the checkout genuinely belongs to root; a chown that fails warns rather than killing startup. Also made the failure legible if the handover is ever prevented: a PermissionError now names the directory, the automatic repair, and the manual chown, instead of a message that names neither path nor cause. Verified by mutation: dropping the handover call, chowning a genuinely root-owned checkout, and letting a non-root process chown each fail their own test. 121 starlark tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 * fix(starlark): address the review on the ownership repair Findings from the automated review of #604. Symlinks (CWE-59, the serious one). A root chown that follows links is a privilege-escalation primitive: anyone able to write in starlark-apps could point a link at a root-owned file and have the repair hand it over. Entries are now read with os.lstat, symlinks are skipped outright, and the chown passes follow_symlinks=False. Descendants are processed before the directory itself, so the container does not change hands while its contents are still being walked. install_app() caught PermissionError in its broad handler and returned False, which both routes report as a generic install failure -- the exact shape of the bug this PR exists to fix, since the caller could not tell "this app is broken" from "this process cannot write here". PermissionError is now re-raised; every other failure still returns False. The test fixtures skipped on bare Exception, which would have turned a syntax error or NameError in the plugin into a green run. They now skip only for a named absent dependency and re-raise anything else. Also fixed the _Stat stub that failed in CI but passed locally: it carried only st_uid/st_gid, and pathlib reads st_mode while walking. It now wraps the real stat result and overrides ownership alone. NOT taken: the CodeQL "information exposure through an exception" finding on the hint response. Dropping `details` would contradict this package's documented rule -- "if it returns 5xx, it says why" -- which test_no_api_v3_handler_discards_its_exception enforces with an allowance that may shrink and never grow. The Starlark routes are the ones that policy was written for: they answered 500 with no detail for three releases. describe_exception already redacts credentials and truncates. Keeping the detail is the deliberate trade-off, so the finding is declined rather than silently worked around. Verified on hdpi with the updated code: a symlink to /etc/shadow planted in starlark-apps was skipped while the directory was handed back, and /etc/shadow stayed root:shadow. Mutation-checked all three behaviours. The symlink test was vacuous on the first attempt -- the link already had the target owner, so it was skipped for the wrong reason and the mutation passed. It now forces the link to look like it needs handing over, and fails when the check is removed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-23 16:33:12 -04:00
ChuckandClaude Opus 5	116abb0daa	fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595) ... * fix(sports): share the ESPN rejected-range memo with the background service BackgroundDataService always sent a season range first and, on a 400, fell back to chunks without recording the rejection, so every background season fetch spent a doomed request and live scoreboards learned nothing from it (or it from them). The worker now consults and sets the same 6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight to month/day chunks, and if every chunk fails the range is asked once for a real error without re-spending the chunks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): keep plugin asset and action routes inside their directories POST /plugins/assets/upload, GET /plugins/assets/list and POST /plugins/assets/delete joined the request's plugin_id onto assets/plugins unchecked, so '../../config' created, wrote, listed and deleted outside it. #561 guarded only the route that serves the files. All three now go through path_safety.resolve_under and answer 400 for anything but a plain name, and delete only unlinks a metadata path that resolves into that plugin's uploads directory. PluginManager.get_plugin_directory refuses ids that are not one plain path segment, so /plugins/action (which runs a manifest script from the returned directory) and every other caller get the guard; the action route also rejects such ids up front, covering its no-manager fallback. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): report a no-op plugin update as already up to date update_plugin() returns True both for a real update and for "nothing to do" (a ZIP-installed monorepo plugin already at the registry version, a bundled plugin). With no git commit to compare, POST /plugins/update called every such success "updated successfully", so Check & Update All counted most official plugins as updated on every run. The route now reads what changed off the plugin itself (commit, else manifest version, else last_updated) and returns data.update_status (updated / up_to_date / local_only). The update-all toast is summarised by PluginInstallManager.summarizeUpdateResults from that status, falling back to the message for older servers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(sports): scoreboard scroll speed no longer follows target_fps sports_scroll computed the crisp speed ladder against the global target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since frame-locked presentation (#545) the helper steps a fixed number of whole pixels per presented frame and the panel presents at its real refresh, so the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a 50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s. The ladder now uses the display manager's refresh_hz, then display.hardware.limit_refresh_rate_hz, then the default. target_fps is not consulted. Docstrings now say scroll_delay is ignored for pacing (no behaviour change there) and describe the fixed-step model. Tests: replace the tests that pinned target_fps as the ladder refresh and described time-based stepping; assert speed independence from target_fps (unit and end-to-end presented px/s against the real helper), that the fixed per-frame step is applied, and that scroll_delay does not change speed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): escape registry and upload values in plugin manager inline handlers The store, saved-repository and custom-registry buttons built onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone, so a custom registry entry whose id contained ' closed the attribute and added its own handler. One helper, jsStringAttr(), now HTML-escapes the JSON literal for every one of those handlers, and the store View button opens only http(s) repo links. The live window.updateImageList (plugins_manager.js loads last, so its copy wins over the file-upload widget's) wrote the uploaded file's original name, path and ids into markup raw; they are escaped now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note plugin asset, action and inline handler guards Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): let the root pip wrapper install web_interface/requirements.txt Update Code, the automatic update's health check and Install Base Requirements install web_interface/requirements.txt through safe_pip_install.sh, which only allowed the root requirements.txt. The first commit changing that file would fail its dependency install, and the automatic updater rolls back any update whose dependencies did not install -- on every device, for every newer commit. The wrapper now lists both core requirement files. Only their folders are resolved, so a requirements.txt symlinked out of the project is compared by its target and refused (previously the root file's own symlink target was what got allowed). The updater's file list is a named constant, and a test runs the real wrapper (pip stubbed) on every file Update Code and the rollback install. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): do not retry plugin requests that got an HTTP answer PluginAPI.request wrapped everything that was not a structured error as NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a JSON error without error_code included. Check & Update All retries NETWORK_ERROR, so those updates were re-sent five more times with backoff, contrary to the #587 contract that an HTTP error response is the server's answer. NETWORK_ERROR now means only that fetch() rejected. Any HTTP response without an error_code, or with a body that is not JSON, is API_ERROR with the HTTP status attached. Tested against the shipped api_client.js. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scroll): restart the stats window when an idle gap is dropped by size #582 dropped an idle gap from the frame stats two ways: the reset_scroll() sentinel, which also restarts the 5s window timer, and a size guard for scrollers that never call reset_scroll(), which did not. On that path the first real frame after the gap found the boundary overdue and logged a stats line for a one-frame window. Both paths now share one seeding helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): leave plugins alone when update_core's own rollback fails update_core returns rollback_failed directly when a partial pull or an update whose health check never started cannot be rolled back. run() only held plugins back for 'verifying', so those devices still got new plugin versions and a display restart on top of a core in an unknown state -- the opposite of what the health-check path does, and of the 3.4.0 changelog (plugins are left alone if the rollback fails). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(api): make the REST reference match the api_v3 package Every documented request body, query parameter and response shape was re-checked against the handlers in web_interface/blueprints/api_v3/. Fixes calls that failed as documented (repo_url, action_id/params, files/image_id, font_file+font_family, ?font=, cache key, auto_enable_ap_mode, plugin limit keys), removes the font-override endpoints dropped in #566, corrects response shapes (plugins/config, plugins/schema, health, metrics, operation history, github-status, fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds the 26 routes it omitted (backup, system auto-update/git, wifi radio, starlark editor, MQTT bridge, status endpoints, skins). Documents the merge semantics of partial JSON saves to /config/main and /plugins/config and the dim-schedule POST accepting GET's days shape, which land in the same change set. Replaces app.py line numbers and the removed api_v3.py path with file and function names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): remove the General-tab plugin system toggles that did nothing plugin_system.auto_discover, auto_load_enabled and development_mode had General-tab toggles whose help tips promised dormant plugins and verbose logging, but nothing reads them: every enabled plugin is discovered and loaded regardless. Remove the three toggles. The keys stay tolerated in stored configs. The save handler now stores a flag only when a client sends it; treating a missing key as an unchecked box would otherwise rewrite all three to false on every General-tab save, which still posts plugins_directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(scroll): remove dead code left by #523/#570 - Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read them since the numpy blend replaced the scipy path. - Drop ScrollHelper._last_integer_position and frame_time_target, which were written but never read. - Keep target_fps and set_target_fps() but document them as informational: nothing paces off them, yet ledmatrix-elections' test_scroll_pacing.py reads helper.target_fps back and third-party plugins may call the setter. - Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to throttle", set_sub_pixel_scrolling's "default: True", and set_frame_based_scrolling's claim that it steps. The plugins monorepo was grepped for every removed name; none is used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI FONT_MANAGER.md told plugins to read display_manager.font_manager, which does not exist, so a plugin following it failed to load with AttributeError. The shared FontManager lives on the PluginManager and BasePlugin._get_font_manager() returns it (with a fallback for harnesses). Also removes the Fonts-tab override workflow and element-override panels that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls /plugins/store/search does not exist (404) and the list endpoint reads query, not q. The registry guide's curl examples omitted the JSON Content-Type, so the handlers saw an empty body and answered 400. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(config): use the shared core-key list in the last three private copies StartupValidator warned "Plugin 'auto_update' is enabled but not found" on every display start with auto-update or a dim schedule on; the reserved plugin-id check missed auto_update, sync, location and the rest; and ConfigManager's (uncalled) orphan cleanup would have deleted display, schedule and auto_update. All three now read src/core_config_keys.py, which also gains CORE_SECRETS_KEYS for the github/youtube secrets sections. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): partial JSON saves to /config/main change only what they send A JSON body with one field reset every checkbox in the sections it touched: the MQTT bridge's brightness slider turned off disable_hardware_pulsing, inverse_colors, show_refresh_rate and use_short_date_format, and a timezone-only save turned off web-UI autostart and weekly auto-updates. Missing-means-unchecked now applies only to form posts: form-encoded bodies and the v3 forms, which mark themselves with a hidden __form_section input. Also on the config routes: - vegas_min/max_cycle_duration no longer match the generic *_duration rule, so they stop landing in display_durations and a blank one no longer rejects the whole Display save; - saving from the Raw JSON editor calls start_setup_if_needed like the General form, so enabling auto-update there finishes its setup; - the schedule and dim-schedule POSTs accept the per-day days.<day> shape their GETs return, as well as the flat form keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): install plugin dependencies from the configured plugins directory install_plugin_dependencies.sh scanned only plugins/, but the Plugin Store installs into plugin_system.plugins_directory (default plugin-repos), so the documented "Recommended" fix found 0 plugins on every store install. It now reads plugins_directory from config/config.json (relative to the project root or absolute, default plugin-repos) and also scans plugins/ for dev symlinks, installing a plugin reached through both only once. With set -e alone, `pip ... | tee` took tee's exit status, so a failed pip install was reported as success; set -o pipefail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: replace stale API names, line numbers and the api_v3.py path - ADVANCED_FEATURES: StreamManager methods that exist (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the real on-demand status envelope ({status, data: {state, service}}) - app.py:199 / :144 / :607-619 line citations and web_interface/blueprints/api_v3.py (now a package) replaced with file and function names in ADVANCED_FEATURES, CONFIG_DEBUGGING, PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE, PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README - CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use /config/raw/main to replace the file; describe where validation runs - TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints usage) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): verify the web interface that actually ships, on port 5000 verify_installation.sh failed every healthy install: it required the long-removed web_interface_v2.py and looked for a listener on port 5001, while the web interface binds 5000 (web_interface/start.py). It now checks the files ledmatrix-web.service runs (start_web_conditionally.py, web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the same 5001 port in its listen check, HTTP probe and printed URLs. Port matches are anchored so :50001 no longer counts as :5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(plugins): one display-size contract: display_manager.width/height CLAUDE.md (#580) says to read display_manager.width/height because matrix is None when hardware init fails; the development guide, the safety-harness doc and two DisplayManager docstrings still recommended matrix.width/height. The bundled starlark-apps plugin read matrix.width unguarded, so its magnify recommendation and frame scaling raised in fallback mode (e.g. after the Pi 5 hardware refusal). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(install): make install_service.sh --help print usage instead of installing install_service.sh parsed no arguments, so `sudo ./scripts/install/ install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md) rewrote ledmatrix.service, ledmatrix-web.service and both update-verify units and enabled/started them. It now handles -h/--help (usage, exit 0, no changes) and rejects any other argument with exit 2 before doing anything. Running it with no arguments, as first_time_install.sh does, is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scroll): describe the fixed-step model and document frame_hold Since #545 a crisp speed from scroll_config.configure() makes the helper advance a fixed whole-pixel step per presented frame with no clock, and the display manager's frame hold is part of the speed. The docs still described the removed wall-clock model: - scroll_config's module and configure() docstrings said speed is applied in time-based mode and that omitting the hold "falls back to fractional pixels"; omitting it actually runs the scroll frame_hold times too fast. - SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both modes, and read a 20 ms stats median as missed refreshes although that is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step, the hold-dependent healthy median, that target_fps plays no part, and that a hand-added scroll_pixels_per_second loses to a schema-default pair. - PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling) without frame_hold; it now documents the parameter (core 3.4.0) with a configure() + set_scrolling_state example. - update_scroll_position/set_scroll_speed and set_scrolling_state docstrings say the same. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark target_fps legacy; describe what Vegas scroll_delay does - General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after the sports_scroll fix nothing in core scrolling reads it. The field and its API validation stay so saved configs and plugins that read global_config['target_fps'] keep working. CONFIG_REFERENCE says the same. - Vegas frame_based_scrolling/scroll_delay were described as frame-count stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and still advances by elapsed time, so the applied speed is clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The config comments, render_pipeline comment and CONFIG_REFERENCE rows now say so. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(deps): describe how plugin dependencies are really installed The guides said the web service runs as root, that installs pick --user from os.geteuid(), and quoted a warning and a PluginManager._install_plugin_dependencies() method that don't exist. The web unit runs as the installing user; store installs go through install_requirements_file() and sudo safe_pip_install.sh (root), with a user-level fallback that says so, and load-time installs run in the display service's own (root) interpreter. Manual paths now use the configured plugins directory (plugin-repos/ by default) instead of plugins/, which store installs no longer use, and install_plugin_dependencies.sh is described as scanning that directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): count local changes one way for the preflight and the pull The automatic update's preflight ignored mode-only changes and anything whose status line contained plugins/ or plugin-repos/, then promised "Automatic updates will not stash your changes". perform_core_update used plain git status (modes count) and ignored only 'plugins/', then ran 'git stash push -- :!plugins', which nothing ever pops. So an edit to a bundled plugin under plugin-repos/, or the installer's chmods on tracked scripts, passed the preflight and was stashed away for good. - auto_update.local_changes() is the one predicate both use: core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/ excluded by leading folder rather than substring (a core file under web_interface/static/v3/js/plugins/ now counts). - Update Code's explicit stash leaves out both plugin folders; the pull's --autostash carries their edits and mode changes across and reapplies them. - The automatic updater calls perform_core_update(stash_local_changes= False), which refuses instead of stashing edits that appeared after the preflight; update_core reports that as 'blocked'. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): diagnostics follow the web autostart default and api_v3 package #556 made a missing web_display_autostart mean "start" (only an explicit false/off keeps the web interface down), but the diagnostics still said otherwise: diagnose_web_ui.sh reported a missing key as "defaults to false", diagnose_web_interface.sh said the web interface "will not start unless this is set to true" and recommended enabling it, and debug_web_manual.py printed False. Troubleshooting a down web UI pointed users at a non-cause. Both shell scripts now evaluate the setting with the launcher's own autostart_enabled() (inline fallback if it cannot be imported) and report on / off / not set (on) / unparseable config; debug_web_manual.py uses the same function. They also check web_interface/blueprints/api_v3/ __init__.py: api_v3.py became a package in #553, so every healthy checkout was reported as missing a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(install): what install_service.sh installs; verify script port; no sudo for --help install_service.sh installs and starts ledmatrix, ledmatrix-web and the update-verify units, not only ledmatrix.service (systemd/README.md, README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help' as a harmless check; it now shows --help without sudo and warns what a real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh checks the web interface on port 5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note update-all, plugin system settings and script fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): size the preview after orientation and pixel mappers display_geometry.physical_size claimed to give DisplayManager's answer but only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured after the library's pixel mappers, so a Rotate:90 / orientation 90 chain previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for 128x64. Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown names leave it alone), and move the orientation composition here so DisplayManager and the preview share it. The module docstring no longer claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH. Audit finding F18. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): refuse settings the rgbmatrix library aborts on, on every board The library answers several settings with a NULL matrix or abort() rather than an error, so the display service crash-looped (Restart=on-failure) instead of reaching fallback mode: rows above 64, chain_length above 255 (uint8_t binding setter, documented as "no upper limit"), a misspelled hardware_mapping, and parallel 2-3 on a single-output mapping, reachable from the Display form on the default adafruit-hat(-pwm) mapping. #586 only guarded the Pi 5 subset. - src/matrix_support.py holds the rules for every board (Options::Validate ranges, binding integer types, mapping names and outputs from lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the API's numeric ranges. - DisplayManager checks them before building options and raises MatrixSettingsRefused, so a hand-edited config falls back with a logged, reported reason. Emulator mode only warns. - The config API refuses them with a 400 naming the setting; combinations are checked against stored values but reported only when the request sets a field involved. - The hardware status file gains "cause" (settings/library/forced). The fallback log and Display banner give the Pi 5 rebuild hint only for a library failure instead of rebuild + gpio_slowdown advice for every failure; one Pi 5 slowdown recommendation (1-3, start at 1). - The Display form offers classic/classic-pi1 and orientation 90/270 and renders any other stored mapping selected with a warning, so an unrelated save no longer rewrites them; the API accepts 90/270. Audit findings F03, F16, F19, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(display): library limits, template defaults and Pi 5 slowdown - rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs, classic/classic-pi1 mappings and orientation 90/270 documented. - Defaults are the config.template.json values: config migration adds missing keys from the template, so the listed "code defaults" never applied. - One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode, starting at 1. - Troubleshooting describes the refused-settings fallback, and CHANGELOG corrects the Unreleased "no upper limit" entry. Audit findings F19, F20, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py opens the panel with the service's options --measure and --demo built RGBMatrixOptions from a private copy of the display service's builder that had drifted: gpio_slowdown came from display.hardware (default 2) instead of display.runtime (default 3), and rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors, pixel_mapper_config and orientation were skipped, with different defaults (hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown was measured -- or garbled -- in a setup the service never drives. The option filling in DisplayManager._setup_matrix moves, unchanged, into DisplayManager.apply_matrix_options(options, config), which _setup_matrix calls and the script reuses (overriding only limit_refresh_rate_hz for --measure). The script now loads the whole config rather than the hardware block. Tests pin the script's options to the service's attribute for attribute. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py recommends keys the resolver honours The ladder ended by telling users to set display_options.scroll_pixels_per_second. scroll_config ranks that key below the scroll_speed + scroll_delay pair, deliberately, and several plugin schemas default the pair into config, so the advised key was silently ignored (a schema-default 1/0.02 pair plus an advised 66 still resolved to 50 px/s). The advice is now the pair that selects the crisp speed exactly (pixels_per_frame every frame_hold/refresh seconds), explains that the pair outranks scroll_pixels_per_second, and gives the scoreboards' per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the printed pair over a schema-default pair and check it lands on the advertised speed and hold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula - SPORTS_UNIFICATION.md still presented honouring global target_fps as sports_scroll's added behaviour and its one user-visible gain; note that it was withdrawn because it had become a speed multiplier. - ADVANCED_FEATURES.md gave Vegas scrolling as (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s by elapsed time, through a 0.1-5 px per scroll_delay clamp when frame_based_scrolling is on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): scroll model fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dev): link-github links plugins from the ledmatrix-plugins monorepo link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git, and those per-plugin repositories no longer exist: official plugins are directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the monorepo once into the dev directory, finds plugins/<name>, plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and links it under its manifest id. With an explicit repo URL it still links a single-repository plugin as before. dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a fork), plus plugins_repo and plugins_branch; github_pattern, which was documented but never read, is dropped and warned about. Ships dev_plugins.json.example and git-ignores dev_plugins.json, both of which the guide promised. Reading JSON falls back to python3 when jq is missing (get_plugin_id silently returned nothing without jq). update/status/list find the git checkout above a monorepo plugin directory (its .git is not in the plugin dir), and update pulls a shared checkout once. status no longer exits 1 when nothing is broken. Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration, workflow, store integration, hello-world link, submission), and the nonexistent scripts/git-hooks/pre-push-plugin-version and scripts/bump_plugin_version.py replaced with the real rule: bump the manifest version and run update_registry.py. scripts/dev/README.md and CLAUDE.md updated to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scripts): monorepo workspace layout; fix_perms and install READMEs MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin; setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the workspace file opens LEDMatrix plus ../ledmatrix-plugins. scripts/fix_perms/README.md listed cache directories fix_cache_permissions.sh never touches and a 'ledmatrix' service user that doesn't exist (also in scripts/install/README.md); adds safe_pip_install.sh. install/README: install_service.sh installs the web and update-verify units too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): keep the rollback's pip retries inside the unit time limit The health check reinstalled the previous requirements by trying the next bash path after any failure, including a 600 s pip timeout. Two files, two paths: up to 40 minutes of pip alone, while systemd stops ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the rollback half-way and leaving the update 'verifying' until the web UI calls it lost. - Like permission_utils.install_requirements_file, only a sudo refusal moves on to the next bash; a pip that ran and failed or timed out is not repeated. The refusal wording is one list (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only verifier and pinned equal by a test. - All reinstalls in one rollback share a 600 s budget. - WORST_CASE_SECONDS adds up every timeout on the longest path (27.5 min); a test holds it under the unit's TimeoutStartSec and that under the web UI's VERIFY_LOST_SECONDS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools Plugin config was prepared differently depending on how it arrived: - JSON POST /plugins/config built a partial body on schema defaults, so {"enabled": true} reset every other setting of the plugin. It now merges onto the stored section first, as the form path already did. - Legacy-boolean normalization (#588) ran only at load: GET /plugins/config returned the raw boolean, posting it back failed validation, and hot reload handed plugins the raw section (a legacy dynamic_duration: true came back as a boolean). schema_manager.prepare_plugin_config (normalize, then defaults) is now used by PluginManager.load_plugin, both save paths, GET, the save notifications and DisplayController's hot-reload callback. - The JSON save's filter kept only enabled/display_duration/live_priority and dropped a submitted skin, skin_options or vegas_* tuning key. There is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES, used by validation and by the save filter; PluginManager's CORE_OWNED_CONFIG_KEYS is its vegas subset. - Plugin sections posted to /config/main were stored verbatim, including values /plugins/config rejects. They now go through the same preparation (_prepare_plugin_config_for_save, extracted from save_plugin_config), and a failing section rejects the whole save before anything is written. - dev_server read only top-level defaults and let a schema enabled:false win; build_full_config shallow-merged overrides, dropping sibling defaults; the harness extracted defaults differently from the device. loading.build_config now uses the device's extraction and preparation, and dev_server, check_plugin, render_plugin and the harness all use it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt-bridge): brightness changes apply live and touch nothing else The display service's hot reload applies a saved brightness within a few seconds, and /config/main no longer resets other display settings on a brightness-only JSON body. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): automatic update hardening Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI It described web_interface_v2.py and index_v2.html (both gone), client-side form generation, one POST per field with {key, value}, and 'no nested objects'. The v3 UI renders plugin forms server-side from the schema (pages_v3 partial + plugin_config.html macros, nested sections and x-widgets), posts the whole form once, and save_plugin_config() merges onto the stored section, validates, splits x-secret fields and notifies the plugin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt): brightness saves apply via hot reload and leave other settings alone The bridge README said brightness is applied on the display's next restart; the display controller's config hot reload applies it within seconds. It also now states that the bridge's partial JSON save changes only brightness (the /config/main merge fix in this change set). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): don't log pip's output from the health check's reinstall pip can echo a private index URL with embedded credentials; permission_utils redacts it, the stdlib-only verifier cannot, so it logs the exit code only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark the plugin_system toggles as unused legacy keys auto_discover, auto_load_enabled and development_mode are read by nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and the REST reference listed them as live settings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): docs and developer tools group Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): legacy plugin-system toggles no longer count as a General save auto_discover, auto_load_enabled and development_mode have left the General form, so a post carrying only one of them is not a general-settings save and must not treat web_display_autostart and auto_update as unchecked. The plugin_system block itself is left as on main for the branch that reworks it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): config-save and plugin-config preparation fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(claude): re-check matrix_support.py rules when the library submodule is bumped Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address Codacy findings on the core audit PR - plugin_manager.prepare_plugin_config: when the fallback legacy-boolean pass also fails, log a warning instead of a bare except/pass. - api_client.js: request() refuses any endpoint that is not a plain path under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace, control characters) with INVALID_ENDPOINT before calling fetch(), and plugin ids are URL-encoded wherever they are put into a URL (also in the app-shell batch load). - test_update_all.js: pins both against the shipped client. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): check endpoint control characters without a control-character regex Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags the \x00-\x1f range in checkEndpoint's regex. Test the char codes instead; the endpoints refused are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(auto-update): make the seed script executable on disk, not only in the index On Linux Repo.publish() commits with -a, which recorded scripts/run.sh as 100644 upstream because the seed file was never chmod +x. The pull then brought in the same mode the installer chmod had made locally, so installer_chmod saw no mode change left to check. The updater was fine: with the upstream commit at 100755 the --autostash carries the device's chmod across. Verified under Linux (WSL, git 2.43): the old helper fails exactly as CI did, the fixed one passes all 63 tests in the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-17 16:37:29 -04:00
ant456	a3d505384d	Add render_width/render_height support to Starlark Apps (#552)	2026-09-11 11:22:52 -04:00
ChuckandClaude Opus 5	bdb9a94033	refactor(api-v3): split the 10,469-line blueprint into a package (#553) ... * refactor(api-v3): split the 10,469-line blueprint into a package web_interface/blueprints/api_v3.py held 111 routes, 56 helpers and 181 functions in one module -- 9% of the core by line count and three times the next largest file. It becomes a package of nine route modules grouped by path segment, plus __init__.py for the shared imports, constants, Blueprint and helpers. Every route module decorates the SAME api_v3 Blueprint object, so endpoint names stay api_v3.<function>, the URL map is unchanged and app.py is untouched. Verified: 111 routes before, 111 after, byte-identical rules, endpoints and methods, and every endpoint still on the one blueprint. plugins 3,867 config 1,178 starlark 692 system 619 fonts 452 misc 398 wifi 361 display 326 backup 212 __init__ 1,787 (imports, constants, Blueprint, 56 helpers) Two things the URL-map check could not catch, both found by running the suite: 1. PROJECT_ROOT = Path(__file__).parent.parent.parent. Moving the code one directory deeper made that resolve to web_interface/ instead of the project root. Nothing failed at import; it surfaced as ~110 tests failing with 404s and "installation script not found", because every path built from it was one level too shallow. Now parents[3], and test_api_v3_url_map.py asserts PROJECT_ROOT/run.py exists so the next move cannot repeat it. 2. Module-attribute patching. Tests do monkeypatch.setattr(api_v3_module, "_BACKUP_EXPORT_DIR", ...) and a route module that binds such a name by value never sees the patch. The shared code therefore stays in __init__.py rather than moving to a _common submodule -- it has to live on the module the tests patch -- and the eleven names tests patch are read back through the package (_pkg.X) instead of bound by value. Those eleven were found by AST-scanning every setattr in the test tree, not by guessing; "time" is among them, used to drive a fake clock through the second-resolution credential-backup filenames. Test changes are confined to what genuinely moved: patch targets that now name the owning route module, imports of helpers, and six tests that scan the api_v3 source as a file and now read the package directory. Full suite: 4,278 passed, 68 skipped, 0 failed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 * fix(api-v3): address CodeRabbit findings from the blueprint-split review Fixes to the api_v3 package split (PR #553), one per finding verified against the actual code: - __init__.py: _redact_credentials only blanked scalar values under a credential-named key; a bare list of secrets under such a key (e.g. tokens: ["a", "b"]) passed through untouched, since the list branch recursed with no memory that its key looked like a credential. Nested dicts still walk normally (a documented, tested behaviour -- a container like secrets: {api_key: ..., note: ...} is a section name, not a value to blank outright), but any value reached under a credential-shaped key is now actually blanked. - __init__.py: the OAuth helper script's raw stderr/stdout went to logger.error unredacted (CWE-532) right next to a comment claiming this was deliberate; the HTTP response already used the existing redact_text helper. Routed the log line through the same helper. - __init__.py / starlark.py: the standalone Starlark manifest fallback (used when the plugin instance isn't loaded) read-modified-wrote manifest.json with no lock, unlike StarlarkAppsPlugin._update_manifest_safe (plugin-repos/starlark-apps/manager.py), which already holds an flock for the same file when the plugin is loaded. Added _starlark_manifest_lock, mirroring that pattern, and wrapped every standalone read-modify-write call site in it. The app-config update route also wrote config.json and the manifest as two separate, non-transactional writes (a second, distinct finding at the same call site); config.json is now rolled back if the manifest write that follows it fails. - backup.py: restore options used bare bool() on values from the request, so {"restore_secrets": "false"} restored secrets anyway (bool("false") is True). Switched to the existing _coerce_to_bool helper already used for this exact purpose elsewhere in the package. - config.py: an automated import-rewrite mangled four user-facing validation strings and their neighbouring comments -- "Invalid start time" had become "Invalid start _pkg.time" (and likewise for "end time") in both the schedule and dim-schedule per-day validation paths. - display.py: `import _pkg.time as time_module` -- _pkg is a local alias for the package, not a real importable module, so this raised ModuleNotFoundError whenever a caller restarted an already-running display service via /display/on-demand/start, after the on-demand request was already written to cache. Fixed to `import time`. Audited the rest of the package for the same `_pkg.<module>` import mistake; every other `_pkg.` reference is a legitimate attribute read-through (`_pkg.time.time()`, `_pkg._get_starlark_plugin()`, ...), not a broken import statement. - fonts.py: validate_file_upload's max_size_mb parameter is silently unused by that helper (it only checks filename/extension) -- the font upload route saved arbitrarily large files as a result. Added the same seek-and-check pattern already used for the sibling .star upload. - wifi.py: two ad hoc, inconsistent bool coercions. POST /wifi/ap/auto-enable used bare bool(), so a JSON string "false" enabled it. POST /wifi/radio's enabled/force parsing recognized real bool and some strings but not int 1/0 (1 is True is False in Python). Factored one small _parse_bool_ish helper local to this file and used it at all three sites. Not changed: the "unknown/misspelled restore option keys default to True" half of the backup.py finding -- the file's own comment documents that a missing key deliberately means "restore everything," matching the already-existing JSON-parse-failure guard a few lines above it; only the bool-coercion defect was a real bug. Added or extended regression tests for every fix, following each area's existing test conventions. Full suite: 4328 passed, 62 skipped, 2 failed on both this branch and origin/main (missing tzdata package breaks two timezone-alias tests in test_onboarding_checklist.py, unrelated to this change) -- no new failures. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3bPMESe2TfrGvbs1ef9c5 * fix(api-v3): reject unknown restore option keys CodeRabbit's review of the blueprint split (#553) asked that POST /backup/restore reject option keys outside RestoreOptions' known set. The follow-up commit fixed the bool("false")-is-True bug with _coerce_to_bool but never added the key check: a typo'd or renamed key (e.g. "restoreSecrets") is silently ignored by opts_dict.get(key, True), so the flag stays at its True default and secrets get restored despite the caller's request saying otherwise -- with no indication anything was wrong. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vmcwf5vMgYqdt8bJTZtiwb * fix(api-v3): address CodeRabbit findings on the blueprint split - _redact_credentials: blank scalar descendants of objects reached through a credential-owned list (e.g. tokens: [{"value": "secret"}]) regardless of field name -- the existing name-based walk only protected direct dict values under a credential key, not list items. - wifi.py: reject enabled/force/auto_enable_ap_mode values _parse_bool_ish can't recognize (400) instead of silently treating them as False, which could disable Wi-Fi or the radio itself. - Starlark manifest locking: lock a stable manifest.json.lock sidecar instead of manifest.json itself, in both the standalone route path (_starlark_manifest_lock) and the plugin path (StarlarkAppsPlugin._save_manifest / _update_manifest_safe). manifest.json is replaced by an atomic rename on every write, which swaps in a fresh inode; a lock held on the old inode does not exclude a second locker that opens the path afresh right after the rename and gets the new inode, so two writers could race despite each holding "a lock". A sidecar that no write ever touches always resolves to the same inode for every locker. Skipped as stale: the "serialize the complete manifest read-modify-write" finding at api_v3/__init__.py -- every standalone handler that calls _write_starlark_manifest is already wrapped in _starlark_manifest_lock() on this branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(api-v3): re-check reconciliation findings by the reconciler's own rules Both CodeRabbit findings on the merge commit, verified against the code first. Major, plugins.py: the stale-findings filter derived its own notion of "in config" and "on disk", and both were looser than the reconciliation module's. set(load_config()) also contains system keys, the secrets-file keys load_config() merges in, and non-dict values; and any directory holding a manifest.json counted as installed even when that manifest does not parse. Either looseness clears a finding that is still true -- and a secrets key read as a plugin is the precise bug the filter exists to stop reporting, so reintroducing that asymmetry while re-checking was the wrong way round. The two extractions now live in state_reconciliation.py as config_plugin_ids() and disk_plugin_ids(), with ignored_config_keys() and secrets_top_level_keys() alongside. _get_config_state() and _get_disk_state() use them too, so there is one definition rather than two that can drift. _get_disk_state() re-reads each manifest for version/name after taking membership from the shared extractor; that costs one extra small read per plugin on a path that runs once per boot. Minor, the new test: the fixture assigned api_v3.config_manager and api_v3.plugin_manager directly. Those live on a module-level blueprint singleton, so the mocks leaked into every later test that imports api_v3 -- pointing at a tmp_path already deleted. Both now go through monkeypatch.setattr, which restores them. This is the same pollution class that made an earlier test in this session break seven unrelated ones, so it is worth getting right. Five cases added for the parity itself: a secrets key, a system key and a non-dict value must not clear an "installed but missing from config" finding, and neither an unparseable manifest nor a .standalone-backup- directory may count as installed. All five fail against the looser version. Linux CI on the preceding commit: Core unit tests, plugin harness, CodeQL and CodeRabbit all pass. Codacy reads action_required on every commit of this branch including the first, so it is pre-existing and not from this work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-11 10:07:38 -04:00
ChuckandClaude Opus 5	26769ee37f	fix(starlark): the store authenticated with a key nothing writes (#541) ... #535 restored the thirteen routes, so the store stopped answering 404 -- and still would not load. Confirmed against a running device before anything was changed: /repository/browse answers 200 with 1000 apps in 27s, so the routes are fine. Two things underneath them are not. **The store never used the token the user configured.** The three repository routes read `github_token` off config.json. Nothing writes that key -- it is not in config.template.json, no setting offers it, and it appears nowhere else in the codebase. The configured token goes to config_secrets.json as `github.api_token`, which PluginStoreManager loads and every other GitHub caller uses. So the store could never be authenticated: 60 requests/hour, on the same per-IP budget 48 installed plugins spend on update checks, while the 5000 the user had already configured sat unused. On the device, /plugins/store/github-status reported authenticated with a limit of 5000 at the same moment /starlark/repository/browse reported 60, with 18 left. The store going blank was that 60 running out. **Every failure looked identical.** list_all_apps_cached turned any listing failure -- rate limit, DNS, timeout, non-200 -- into an empty app list, and the route sent that out as `status: success`, so a rate limit and an empty repository drew the same blank grid with no error anywhere. It now returns the reason, the route answers 502 with it, and a failure is no longer cached as an empty repository for two hours. The guard for a bad response was itself a crash: _make_request catches `(json.JSONDecodeError, ValueError)` but `json` was never imported, so evaluating the tuple raises NameError and the guard written for exactly this case never ran. Reachable whenever something on the path answers with HTML -- a captive portal, a proxy page, a DNS-hijacking router. Seventeen handlers answered 5xx with no detail at all. test_no_api_v3_handler_discards_its_exception is meant to prevent that across api_v3, but it matched one exact message string, and all thirteen Starlark routes wrote their own wording. The guard now keys on the shape that matters: if it returns 5xx, it says why. The 15 pre-existing non-Starlark functions are listed as a set that may shrink, never grow. **The listing was capped at 1000 and did not say so.** The contents API truncates a directory silently; tronbyt/apps has 1075 app directories, so the store showed a truncated repository and looked complete doing it. Now listed via the git trees API, which reports `truncated`, with the contents API kept as a fallback. Not addressed: the 27-second cold load -- 1075 manifests fetched five at a time behind skeleton placeholders -- which is probably the largest part of what "does not load" feels like, and wants its own change. 25 new tests. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-08 20:04:27 -04:00
ChuckandClaude Opus 5	e23f1f45d3	feat(starlark,on-demand): the third-party fixes worth taking, plus a Home Assistant MQTT bridge (#538) ... * feat(starlark,on-demand): the third-party fixes worth taking, plus an MQTT bridge Analysis of ant456/ledmatrix-fixes-repo, a third-party collection of patches and services built while running this project on Starlark apps under MQTT control. Its patches are whole-file copies taken against an older tree, so applying them as written would revert #523's frame pacing, #534's display() bool returns and the GitHub token masking in plugins_manager.js. Three of its claimed fixes are already in main, and its api_v3 Starlark routes are #535's. What follows is the rest -- verified against current code, and reimplemented where the patch's approach did not hold up. **On-demand display.** `pinned` reached the controller from the API, was stored on it and republished in the status payload, but never narrowed the rotation -- a pinned request still cycled every mode its plugin owns. Right for a sports plugin, whose modes are views of one subject; wrong for a plugin whose modes are unrelated, which is every Starlark app. Now honoured, and it survives a restart. Restarting while on-demand was active loaded *only* the on-demand plugin, so normal rotation had nothing to return to for the life of the process -- and a restart mid-session is routine, since that is how an update is applied. The panel came back cycling one plugin's modes with no way out but clearing the cache by hand. Every enabled plugin loads now; on-demand still resumes on its saved mode. Stop requests are exempt from the duplicate guards on purpose, so that a second click stops a mode a race left running -- which means consuming the mailbox is the only thing that ends one. It was never consumed, so the same stop was re-read and re-processed on every poll, forever. Both paths now share one compare-before-delete helper. **Starlark rendering.** `extract_schema` parsed the source with a regex, which can only see option lists written out literally: an app whose dropdown is filled from a live API call inside `get_schema()` came back empty, and the config form offered nothing to pick. Now runs `pixlet schema`, which executes the app, and falls back to the parser when Pixlet is absent, too old for the subcommand, or the app fails to run. The third-party patch replaced the parser outright and hardcoded /usr/local/bin/pixlet; this keeps the fallback and the binary search. A `|` in a config value was dropped by a shell-metacharacter filter, though the command is a list with no shell involved -- and apps do use it as a separator inside one value. The key went missing silently and the app rendered its own "not configured" screen with nothing to say why. And a 0-byte render was reported as success: Pixlet exits 0 and writes nothing when an app has no content, which read downstream as a working app drawing a black panel. **Starlark display.** `display()` ignored the mode it was called with, so a specific app could not be addressed. It now accepts `display_mode` -- which is the whole mechanism, since the controller inspects the signature before passing it. Found while there: `_select_next_app` ran only while `current_app` was unset, so with several apps installed the first was picked once and shown forever while the rest were rendered on schedule and never displayed. And `enable_scrolling` was missing, so multi-frame apps were called once per rotation slot and never advanced past frame one. **GET /api/v3/display/modes.** Every mode that can be requested on-demand, with the plugin that owns it. Nothing exposed this, so anything driving the display from outside the web UI read each plugin's manifest.json off disk and reimplemented PluginManager's fallbacks. It also triggers discovery, which is otherwise lazy and normally happens because a person opened the dashboard. **integrations/mqtt_bridge.** Home Assistant control over MQTT Discovery: a mode select, a stop button, power, brightness. Rewritten against the API rather than the filesystem, so it needs no read access to config.json and cannot drift from the web UI. paho-mqtt 2.x VERSION2, TLS, an availability topic that is also the last will, and secrets from the environment. **Two opt-in extras.** A DNS single-request unit, for glibc's parallel A/AAAA lookup stalling ~5s per name on routers that answer only the A query -- which makes any plugin calling an external API slow and Starlark apps, which have a render timeout, fail outright. And a Pixlet config editor: a script you run and Ctrl+C rather than the third-party version's always-on unauthenticated Flask service, since it stops the display for the length of a session. Neither is installed by default. Long Starlark app names now wrap instead of overflowing their card. 115 new tests across 5 files. Also unblocked test_starlark_display_contract.py, which was silently skipping wherever fcntl is absent. Whole suite: no new failures against main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mqtt_bridge): the five issues Codacy flagged on this branch All in the new bridge, all real: * requests floor was 2.31.0, which carries CVE-2024-35195, CVE-2024-47081 and CVE-2026-25645. Raised to >=2.33.0,<3.0.0, which is what the project's own requirements.txt already pins. * `import time` was never used. * `"mqtt_password": None` in DEFAULTS read as a hardcoded credential. It is the "no password configured" default; marked nosec B105, the convention used elsewhere in the repo. Also dropped an unused `build_app` from the display-modes test imports. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: the review findings on this PR Nine of CodeRabbit's ten, plus the CodeQL alert. The tenth is wrong and is answered below. **One bad config section blanked the whole mode list.** `/display/modes` read `full_config.get(plugin_id, {}).get('enabled')`, so a non-dict under a plugin id -- a shape DisplayController already guards, so it happens -- raised AttributeError mid-loop and answered 500 with no modes at all. Every MQTT bridge entity is built from that list. Now skipped with a warning. **The DNS scripts reported success they had not earned.** Three separate paths: `resolvconf -u` failing was swallowed by `|| true`; the systemd-resolved branch exited 0 without applying anything, so the oneshot unit recorded success while the workaround was inactive; and the installer's `|| echo` turned a failed start into "installation complete." with exit 0. All three now fail loudly. `single-request` is a glibc resolv.conf option with no resolved.conf equivalent, so on those hosts the honest answer is that it cannot be applied. A NetworkManager-generated resolv.conf is regenerated on connection changes, not only at boot, and the unit is oneshot with RemainAfterExit -- so the option can vanish mid-boot with nothing to put it back. Now detected and stated plainly rather than implied to be permanent. **`Before=` does not order a manual restart.** It only orders units already in the same transaction, so `systemctl restart ledmatrix` could bypass the fix. install_dns_fix.sh now writes a ledmatrix.service drop-in with Wants= and After=. Wants=, not Requires=: a DNS workaround failing should not stop the display. **The Pixlet editor's `--lan` is gone.** `pixlet serve` has no authentication, and a printed warning is not access control. Loopback only, with the SSH port-forward in the header where the flag used to be documented -- SSH does the authenticating and nothing is left listening. **The MQTT example config now defaults to TLS** on 8883. The installer copies it verbatim, and without TLS the broker password and every command cross the network in cleartext. A plaintext broker is still supported and documented, and the bridge warns once at startup when a password is configured without TLS. **Not taken: "the upstream Pixlet CLI has no `schema` subcommand."** Upstream tidbyt/pixlet has none, but `scripts/download_pixlet.sh` installs `tronbyt/pixlet`, whose `cmd/schema.go` is `schema [PATH]` -> JSON on stdout, built on `runtime.NewAppletFromPath`, so it does execute `get_schema()`. That is exactly what extract_schema_via_pixlet calls. A binary without the subcommand exits non-zero and falls back to the source parser, which is already covered by a test. **CodeQL stack-trace exposure: not taken either.** I removed `details` first and that broke test_web_error_detail.py::test_no_api_v3_handler_discards_its_exception, which enforces `describe_exception` across all ~75 handlers -- written because a device with failing storage answered "see logs for details" from the log viewer itself. describe_exception redacts credentials; the trade-off is the project's and is already made. Restored, with the reasoning in a comment. 11 new tests. Whole suite: no new failures against main, 4127 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-08 08:34:52 -04:00
ChuckandClaude Opus 5	a0d3e64099	fix: ten defects found validating the whole plugin fleet on hardware (#534) ... * fix(core): register tom_thumb, accept frame_hold in the test double, wire api_v3's managers Three independent fixes found while validating every plugin on a 256x64 rig. FontManager never registered tom_thumb even though assets/fonts/tom-thumb.bdf ships with the core, so every plugin offering it logged "Font family 'tom_thumb' not found" (16 warnings per countdown render) and had to carry a private loader to use a bundled font. Closes #524. VisualTestDisplayManager.set_scrolling_state() lacked the frame_hold parameter that DisplayManager gained, so any plugin passing it died with TypeError at render time and failed every size. Nine plugins now make that call; ledmatrix-stocks and ledmatrix-leaderboard were failing outright and the other seven only passed because their scroll path was unreachable without data. Closes #525. api_v3 declared module-level config_manager/plugin_manager = None that nothing ever assigned -- app.py sets the blueprint attributes, which the other 150+ call sites use. Three sites read the decoys, so /health reported the config unreadable and the plugin system uninitialised (making "degraded" permanent and unreachable-by-design) and /display/current fell back to a hardcoded 128x64 on every rig. The decoys are removed rather than assigned, so a bare name is now a NameError at test time instead of a silent None. The same function's first-call uptime was computed from two separate clock reads and came out negative. Closes #529. Verified on the rig: both previously-failing plugins render, the tom_thumb warnings are gone, /health reports "healthy" with all three checks passing, and /display/current reports the real 256x64. * fix(core): unique snapshot temp name, honour on-demand requests, skip empty starlark The preview snapshot wrote through a fixed "<snapshot>.tmp". /tmp is world-writable and sticky, and the display service runs as a different user from the tooling, so a leftover temp owned by anyone else became unopenable even by root -- fs.protected_regular refuses O_CREAT on a foreign file in a sticky directory. The preview and the health check's liveness proxy then froze until someone deleted the file by hand; on the test rig that meant 23 hours of a healthy display reporting "hardware: stale". Now uses tempfile.mkstemp with cleanup on failure, matching the hardware-status write a few hundred lines above. Closes #528. _poll_on_demand_requests read its mailbox with max_age=3600, and get() defaults the in-memory TTL to max_age -- so the first request was pinned in memory for an hour and every later poll returned that stale copy. No second on-demand request was honoured until the service restarted, while the API kept returning 200. get() already documents memory_ttl=0 for exactly this cross-process case. The consumed request is also now deleted: leaving it on disk meant a restart replayed the previous request, activated it, and ignored the one the caller had just made. Closes #530. starlark-apps returned None from display() when it has no app to show, which is the state of every install without Pixlet and of a fresh one before any app is added. The controller only skips on a boolean False, so that held a black panel for the full display_duration instead of rotating on. Closes #456 (core side). Verified on the rig: two consecutive on-demand requests with no restart between them are both activated, where the second was previously dropped in silence. * perf(harness): share one cache across a plugin's renders _instantiate built a fresh MockCacheManager for every (size, mode), and that mock is a per-instance in-memory dict, so each render was a cold start. A plugin that fetches per game or per player re-fetched everything N times over -- baseball-scoreboard at one size took 840s for nine renders where the arithmetic said ~72s, and at eight sizes it exceeded a 900s timeout. The second and later renders also never exercised the cache-hit path, which is what a running rig executes almost all of the time, so a caching regression could not be caught here. The cache is now built once per render_plugin_matrix call and threaded down. The display manager stays per-render -- the bounds checking depends on that -- so only fetched data is shared. Measured on the rig, same render counts and same goldens: tide-display 2s -> 1s (32 renders) cricket-scoreboard 10s -> 3s (24 renders) No pass/fail change across tide-display, cricket-scoreboard, clock-simple, geochron, christmas-countdown, of-the-day, web-ui-info and incoming-packages. Closes #533. * fix(scripts): run standalone plugin tests instead of collecting nothing run_plugin_tests.py discovered every plugin test file and handed the lot to pytest. Most plugin tests are standalone scripts -- module-level main() plus an `if __name__ == "__main__"` guard, signalling through an exit code -- and pytest collects zero items from those. The run printed how many files it had *found*, then "no tests ran", and exited without executing any of them. On a rig with all 44 first-party plugins that is 151 of 248 files. Files are now classified and each kind runs under the right runner: pytest for real test modules, subprocess for scripts, honouring the 0 pass / 2 skip / 1 fail convention ledmatrix-plugins' own runner established (a script that wants a tty or an LED matrix is a skip, not a regression). Before: $ python3 scripts/run_plugin_tests.py -p countdown -d ~/LEDMatrix/plugin-repos Found 1 test file(s) collected 0 items no tests ran in 0.31s rc=0 After: Found 1 test file(s) -- 0 collectable, 1 standalone script(s) 1 passed, 0 skipped, 0 failed (scripts) rc=0 Verified across three shapes: countdown (1 script), jellyfin-now-playing and pomodoro-timer (pytest only, 16 and 42 tests), and ledmatrix-flights (11 files split 4 collectable / 7 scripts, all seven of which had never run). Closes #532. Running the flights scripts for the first time also surfaced four genuinely failing tests there, hidden by the mirror-image bug in the plugins repo's own runner -- filed as ChuckBuilds/ledmatrix-plugins#464 and #465. * fix(harness): give an empty-looking mode a few frames before warning about it check_plugin's "drew nothing but display() returned X" warning fired on a single frame, rendered with force_clear=True, under a frozen clock. All three defeat a scrolling plugin, whose first frame is legitimately its blank scroll-in buffer. Across 44 first-party plugins, 60 of 76 warnings were false -- the rate at which people stop reading a warning, which matters because the true positives are real: a mode that draws nothing and does not return False holds a blank panel for its whole display duration. An apparently-empty frame is now re-driven for up to 48 more frames with force_clear=False (force_clear means "reset the scroll", so repeating it would redraw frame 1 for ever) and with the clock advancing -- freezegun's factory where time is frozen, a real sleep where it is not, since scroll position is usually a function of elapsed time. The first frame that draws content replaces the result. The clock is moved back afterwards. It is shared by every render in the matrix, so time borrowed by the probe leaked into later modes and drifted their goldens -- f1_upcoming picked up 5 spurious drifts before this was restored. Measured on the rig: empty warns check before after f1-scoreboard 42 0 48 PASS / 0 FAIL, goldens intact ledmatrix-elections 16 0 16 PASS / 0 FAIL on-air 8 8 true positive, kept nfl-draft 8 8 true positive, kept clock-simple/geochron/ 0 0 unchanged christmas-countdown 58 false positives gone, both true positives kept, no golden regressions. Cost is confined to modes that really are blank: plugins that draw immediately are unchanged (clock-simple and tide-display still 2s), while on-air -- eight deliberately blank modes -- goes to 21s. Closes #527. * fix(harness): load nested schema defaults, and merge caller config at leaf level load_config_defaults read only top-level properties. An object property carries its defaults on its children, not on itself, so everything nested was dropped -- 2,386 defaults across 37 of 44 plugins, soccer-scoreboard alone losing 539 of 565. render_plugin_matrix's comment says the plugin then "behaves like a real install", which for most of the fleet it did not. _defaults_from_properties now recurses. merge_config deep-merges the caller's config onto the result so an override lands at the leaf: a shallow merge would let -c '{"nhl": {"enabled": true}}' replace the whole nhl subtree and discard every other nhl default, which is the same class of bug being fixed here. Measured before/after across all 49 installed plugins on the rig: **no render changed** -- identical PASS/FAIL counts, byte-identical output, goldens intact. Plugins already fall back to the same values internally via config.get(key, default), so supplying them explicitly agrees with what they were doing. The defaults really are arriving now: ufc-scoreboard 9 -> 87 defaults ledmatrix-flights 51 -> 95 masters-tournament 10 -> 51 cricket-scoreboard 22 -> 50 tide-display 12 -> 18 and hockey-scoreboard, which used to load nhl.enabled=None, now gets nhl.enabled=True with its full display_modes block. Caveat worth carrying: the eight plugins with the most nested config (soccer, baseball, basketball, hockey, lacrosse, football, afl, nrl -- 1,634 of the 2,386 dropped defaults, 68%) could not be measured. They import src.common.sports_shared, which the test rig's core branch predates, so they fail to load there identically before and after. Re-run this comparison against a core that has that module before trusting the "nothing changed" result for them; those are exactly the plugins whose renders should change most. Closes #531. * refactor: narrow the exception handlers this branch introduced Codacy flagged the new code; it passes on other recent PRs, so the finding is mine. Four of the five broad `except Exception` clauses I added were catching far more than they needed to, which is the same shape as several bugs this branch fixes -- hello-world's TypeError sat invisible for exactly this reason. freezer() / move_to() / tick() -> (AttributeError, TypeError, ValueError) cache_manager.delete() -> (OSError, AttributeError, KeyError) The fifth stays broad and now says why: it wraps a call into a plugin's own display(), which can raise anything, and the first frame has already rendered -- so a failure there must not turn a good result into an error. Verified against a checkout of main: f1-scoreboard 48 PASS / 0 FAIL with 0 empty warnings, on-air keeps its 8 true positives, clock-simple 8 PASS. geochron shows 7 golden drifts both before and after this branch, so it is not from these changes -- its committed goldens predate #521's 1-bit text rendering. * fix: resolve CodeRabbit review and Codacy findings on #534 CodeRabbit raised six; all six were real. The test double had drifted ahead of production. VisualTestDisplayManager accepted set_scrolling_state(frame_hold=...) while DisplayManager did not, so such a call passed every harness run and would raise TypeError on the panel -- the one failure a safety harness exists to prevent. frame_hold belongs to the change that adds it to DisplayManager (#523), so it moves there and the double matches main again. The harness swallowed exceptions from re-rendered frames. _settle_loop re-renders a mode that came back blank, to give a scroll time to draw; returning silently on a crash meant a mode that renders one good frame and then explodes was reported as passing. Recorded on result.error now, keeping the captured frame so the failure stays inspectable. starlark-apps display() returned True after _display_frame() failed, so the controller held a dead frame for the whole display_duration instead of rotating on. _display_frame now returns bool on all three paths. run_plugin_tests.py used env.setdefault for PYTHONPATH and LEDMATRIX_CORE, so an inherited value won and the subprocess imported a different core than the one under test -- ledmatrix-plugins#467 exactly. Prepends PROJECT_ROOT and sets LEDMATRIX_CORE unconditionally. The on-demand mailbox is polled after every frame, ~125x/second on a scrolling mode, and the read is deliberately uncached, so it was that many disk reads per second to find nothing. Floored at 250ms, which is imperceptible for a web-UI click. Consuming it also deleted whatever was present rather than what had just been processed, so a request posted while the previous one was in flight was thrown away and never ran; the delete is now keyed by request_id. That narrows the window rather than closing it -- a true atomic claim needs a primitive the cache layer does not offer, and the code says so rather than implying otherwise. Codacy's 2 criticals were bandit B404/B603 on the subprocess call added to run_plugin_tests.py. Fixed interpreter, argument list, no shell; annotated with the repo's existing nosec convention. Bandit is clean on the file. Adds test/test_on_demand_mailbox.py (8), test_starlark_display_contract.py (4) and two settle cases in test_harness_empty_claimed.py. 4, 4 and 2 of those fail against the pre-fix code. Full suite: 3961 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 * chore: satisfy Codacy's subprocess checks on the new test runner Codacy runs Bandit and Opengrep (its Semgrep fork). The new subprocess.run in scripts/run_plugin_tests.py trips three patterns, on two different lines: Bandit B404 on the import, B603 on the call Opengrep dangerous-subprocess-use-audit on the run( line dangerous-subprocess-use-tainted-env-args on the argv line A nosemgrep applies only to its own line, so the call line and the argv line each need one; a single comment on the call covered neither rule fully. Suppression is the right answer here rather than a rewrite: the interpreter is sys.executable, the arguments are a list, and no shell is involved, so there is nothing to word-split or expand. Matches the pair the rest of the repo already uses for this shape -- permission_utils.py, plugin_loader.py, install_dependencies_apt.py. Codacy: 0 new issues, up to standards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 * chore: leave visual_display_manager untouched so #523 can merge The only change this branch made to that file was a docstring, and it collided with #523's rewrite of the same method -- so #534 and #523 each merged cleanly against main but conflicted with each other. Reverted to main's text; #523 owns this method and adds frame_hold to it. The note the docstring carried ('frame_hold arrives in #523') would have been stale the moment #523 landed anyway. The parity test in #523 is what actually keeps the two signatures honest. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 * chore: add the Ruff suppression nosec/nosemgrep do not cover Ruff reports S603 on the same call Bandit and Opengrep do, and none of the three suppressions covers the others. Confirmed the precondition first: path comes from discover_plugin_tests(), which globs test files inside the repo, and the call is a fixed interpreter with a list argv and no shell. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014RRtqXDCnvnY6EQwhT5CV9 --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-07 13:37:38 -04:00
3 people	f96fdd9f24	fix(plugins): skip update for local-only plugins instead of failing (#354) ... Adds a local_only flag to the starlark-apps manifest so the update endpoint returns a skipped status rather than recording a false failure when the plugin has no git repo and no registry entry. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-27 21:42:25 -04:00
3 people	53ee184bc5	chore: remove march-madness from bundled plugin-repos (#340) ... March Madness is now available in the ledmatrix-plugins monorepo store (ChuckBuilds/ledmatrix-plugins/plugins/march-madness) and should be installed via the Plugin Store like any other plugin. Removing the bundled copy so new installs don't automatically include it. Existing users keep their installed version until they choose to uninstall. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-19 20:00:41 -04:00
3 people	33f76b4895	feat(pi5): RP1 backend UI, gpio slowdown guidance, and hardware init error banner (#337) ... * feat(pi5): expose RP1 backend selector, fix gpio defaults, surface init failures in web UI - Add rp1_rio select (PIO/RIO) to Display Settings hardware config section; saved via /api/v3/config/main with 0-or-1 validation — previously the key existed in config.json but was not editable from the UI - Update gpio_slowdown help text with per-model guidance (Pi 3: 3, Pi 4: 4, Pi 5: 4–5) and raise max from 5 → 10 to match full library range - Fix gpio_slowdown Python fallback default from 2 → 3 (only affects edge case where the runtime config section is absent; explicit config values are unchanged) - display_manager writes /tmp/led_matrix_hw_status.json at startup: ok/error; Display Settings page fetches it and shows a yellow warning banner when the matrix failed to initialize, including Pi 5 remediation steps - Add GET /api/v3/hardware/status endpoint that reads the status file - Improve fallback error log to include Pi 5 rebuild hint Pi 3/4 users: rp1_rio=0 is set in config but silently ignored by the library on non-RP1 hardware; all other changes are additive or tighten defaults only. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(pi5): correct gpio_slowdown guidance — Pi 5 PIO default is 1, not 4-5 The upstream library defaults gpio_slowdown to 1 for Pi 5 (IsPi4() ? 2 : 1). In PIO mode the value is a pixel-clock divisor, so 4-5 was unnecessarily conservative advice. Updated help text and error log to reflect the actual range (1-3 typical for Pi 5 PIO; inverted effect in RIO mode). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(security): atomic hw-status write, narrow bare excepts, urllib3 CVE floor - display_manager: replace open()+bare-except with tempfile.mkstemp→fsync→ chmod(0o600)→os.replace; adds symlink guard and logs errors via logger instead of swallowing them silently; pull json/tempfile to module imports - display_manager cleanup(): narrow broad `except Exception: pass` to (OSError, RuntimeError, ValueError, MemoryError) with debug log - api_v3 get_hardware_status(): catch json.JSONDecodeError and PermissionError explicitly; log full traceback server-side; return generic "Unable to read hardware status" to client instead of leaking str(e) - march-madness/requirements.txt: bump urllib3 floor 2.2.2→2.6.3 (CVE fix) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(template): apply |int filter to rp1_rio comparisons in display.html Without |int, a string-typed value (e.g. from a hand-edited config.json) causes both selected tests to fail and the select renders with no option pre-selected. Matches the existing pattern used for multiplexing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-18 17:42:19 -04:00
3 people	c6b79e11d5	fix: Codacy round-2 — urllib3 CVEs, missed JS/Python issues (#336) ... urllib3 CVEs (10 Trivy findings): plugin-repos/march-madness/requirements.txt: bump urllib3>=1.26.0 to >=2.2.2 to address CVE-2021-33503, CVE-2023-43804, CVE-2023-45803, CVE-2024-37891, and 2025-2026 decompression/redirect CVEs. Missed code fixes from round-1: display_helper.py: remove unused draw=ImageDraw.Draw(img) — the method delegates to _draw_centered_text which creates its own draw context. custom-feeds.js:334: one bare removeCustomFeedRow(this) was missed by the earlier replace_all; changed to window.removeCustomFeedRow(this). app.js: add htmx to /* global */ declaration — htmx.ajax() is called at lines 146 and 172 but htmx was only declared in the extension files. timezone-selector.js:215: second unused catch (e) → catch {} missed when we fixed line 361 in round-1. Bandit B110 annotations (3 new except/pass blocks from newer PRs): start.py: hostname -I IP parsing — non-critical startup info. display_controller.py: scroll_helper.get_portion_at — optional method. display_manager.py: canvas reset during cleanup — best-effort. 41 confirmed false positives suppressed via Codacy API: 35x pyflakes in test/, plugin-repos/, scripts/ — not production code Flask 0.0.0.0, os.execvp, Bandit B603, vendor ESLint, already-fixed Biome noPrototypeBuiltins. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-15 18:04:21 -04:00
3 people	05b3fa56cb	fix: Codacy security fixes, CVE dependency bumps, and code quality cleanup (#331) ... * fix(deps): bump minimum versions to address CVEs Pillow 10.4.0 → 12.2.0: CVE-2026-40192 (DoS via FITS decompression bomb), CVE-2026-25990 (OOB write via PSD image), CVE-2026-42311/42308/42310 requests 2.32.0 → 2.33.0: CVE-2026-25645 (temp file security bypass), CVE-2024-47081 (.netrc credentials leak) werkzeug 3.0.0 → 3.1.6: CVE-2023-46136, CVE-2024-49766/49767, CVE-2025-66221, CVE-2026-21860/27199 (DoS, path traversal, safe_join bypass) Flask 3.0.0 → 3.1.3: CVE-2026-27205 (session data caching info disclosure) spotipy 2.24.0 → 2.25.2: CVE-2025-27154, CVE-2025-66040 python-socketio 5.11.0 → 5.14.0: CVE-2025-61765 pytest 7.4.0 → 9.0.3: CVE-2025-71176 (insecure temp dir handling) Updated in requirements.txt, web_interface/requirements.txt, plugin-repos/starlark-apps/requirements.txt, and plugin-repos/march-madness/requirements.txt. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: resolve Pylint errors in executor, data service, and odds call Rename TimeoutError to PluginTimeoutError in plugin_executor.py to avoid shadowing the built-in; no external callers affected. Remove dead try/except in BackgroundDataService.shutdown: executor.shutdown() never accepted a timeout kwarg so the try branch always raised TypeError. Simplify to a direct shutdown(wait=wait) call. Remove is_live kwarg from odds_manager.get_odds() call in sports.py; BaseOddsManager.get_odds() has no such parameter. The live update interval is already encoded in the update_interval_seconds argument passed alongside. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: MD5→SHA-256, shellcheck warnings, and broken doc links config_service.py: replace MD5 with SHA-256 for config change detection; same semantics (equality comparison), no stored hashes affected. Shell scripts — shellcheck warnings: - diagnose_web_interface.sh: remove useless cat (SC2002) - dev_plugin_setup.sh: restructure A&&B||C into if/then (SC2015) - fix_assets_permissions.sh: remove unused REAL_HOME block (SC2034) - install_web_service.sh: remove unused USER_HOME assignment (SC2034) - diagnose_web_ui.sh: remove unused SUDO assignments (SC2034) - diagnose_plugin_permissions.sh: remove unused BLUE color var (SC2034) - first_time_install.sh: remove unused CLEAR var, PACKAGE_NAME assignment, and replace loop variable with _ (SC2034) docs/PLUGIN_ARCHITECTURE_SPEC.md: fix 10 broken TOC anchor links to include section numbers matching the actual headings (MD051). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unused imports and bare exception aliases (pyflakes F401/F841) Remove unused imports across 86 files in src/, web_interface/, test/, and scripts/ using autoflake. No logic changes — only dead import statements and unused names in from-imports are removed. Also remove bare exception aliases where the variable is never referenced in the handler body: - src/cache/disk_cache.py: except (IOError, OSError, PermissionError) as e - src/cache_manager.py: except (OSError, IOError, PermissionError) as perm_error - src/plugin_system/resource_monitor.py: except Exception as e - web_interface/app.py: except Exception as read_err 86 files changed, 205 lines removed, 18 pre-existing test failures unchanged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unused local variable assignments (pyflakes F841) Dead assignments removed across src/ and web_interface/: - background_data_service: drop future= on fire-and-forget executor.submit - base_classes/baseball: drop font= (all rendering uses self.fonts['time']) - base_classes/hockey: drop status_short= (never referenced after assignment) - common/cli: drop game_helper=/config_helper= bindings in import-test block; constructors called for instantiation-only validation - common/display_helper: drop text_width= (x_position uses display_width directly); drop draw= in create_error_image (uses _draw_centered_text) - config_manager: remove dead secrets_content loading block in migration path (comment already noted save_config_atomic handles secrets internally) - display_manager: drop setup_start= (timing was never completed or read) - font_manager: drop target_path= (catalog uses font_file_path directly); drop face=/font= bindings in validate_font (validation by construction — TypeError on failure is the signal, not the return value) - font_test_manager: drop width=/height= (draw_text uses display_manager directly) - plugin_system/state_reconciliation: drop manager= (only config/disk/state_mgr used) - plugin_system/store_manager: drop result= on pip install subprocess.run (check=True raises on failure; stdout unused) - web_interface/blueprints/pages_v3: drop main_config_path=""/secrets_config_path="" (render_template uses config_manager.get_*_path() inline) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(js): resolve ESLint no-undef warnings across 6 JS files Three distinct patterns: 1. Vendor library globals — htmx is injected by <script> before these extension files load; ESLint lints files in isolation and doesn't know. Fix: add /* global htmx */ to htmx-sse.js and htmx-json-enc.js. 2. Cross-file globals — showNotification is defined as window.showNotification in app.js/notification.js but called bare in app.js and error_handler.js. ESLint doesn't connect window.X = Y with a bare call to X. Fix: add /* global showNotification */ to app.js and error_handler.js. 3. Forward-reference window.* functions — in array-table.js, checkbox-group.js, and custom-feeds.js, functions like removeArrayTableRow are called early inside event-handler closures but assigned to window.* later in the file. At runtime this works (the handler fires after the assignment), but ESLint sees the bare name at the call site. Fix: change bare calls to window.removeArrayTableRow(this) etc. so the reference is explicit and ESLint-safe. Also guard the updateSystemStats call in app.js reconnectSSE: the function is called but defined nowhere in the codebase. Guard with typeof check so it won't throw ReferenceError if the reconnect path is hit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(js): resolve Biome lint warnings across 9 JS files noUnusedVariables (catch bindings → optional catch syntax): - app.js, file-upload.js, timezone-selector.js: } catch (e) { → } catch { ES2019 optional catch binding; e was unused in all three handlers noUnusedVariables (dead assignments): - app.js: remove const data= in display SSE stub (handler does nothing yet) - api_client.js: remove const timeoutId= (setTimeout ID never used to cancel) - custom-feeds.js: remove const oldIndex= (getAttribute result never read) - schedule-picker.js: remove const compactMode= (never used in HTML build) - select-dropdown.js: remove const icons= (icons not yet rendered in options) noPrototypeBuiltins: - day-selector.js: DAY_LABELS.hasOwnProperty(x) → Object.prototype.hasOwnProperty.call(DAY_LABELS, x) Safe form that works even on null-prototype objects useIterableCallbackReturn: - file-upload.js, notification.js: forEach(x => expr) → forEach(x => { expr; }) — forEach ignores return values; implicit return from arrow body was misleading htmx-sse.js is a vendor extension file with old-style var/== patterns that are correct for it; 18 Biome issues suppressed via Codacy API rather than modifying the vendor source. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(security): escape user input in raw HTML responses in pages_v3.py plugin_id comes directly from the URL path (/partials/plugin-config/<plugin_id>) and was interpolated into an HTML fragment without escaping. A crafted URL like /partials/plugin-config/<script>alert(1)</script> would inject that tag into the DOM via the HTMX partial response. Fix: wrap all user-controlled values in markupsafe.escape() before embedding in raw HTML strings. Affects the plugin-not-found 404 response and both error 500 responses in the plugin config partial. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address Bandit B108/B110 across production code B110 (try/except/pass): - display_controller.py: narrow 'except Exception' to 'except AttributeError' for get_offset_frame() — plugins not having this optional method is the expected case, not all exceptions - config_manager.py: B110 already resolved by the earlier removal of the dead secrets-loading block (the except/pass was inside it) - All other except/pass blocks in src/ and web_interface/ are intentional (last-resort recovery, best-effort fallbacks, non-critical startup probes). Annotated each with # nosec B110 and a brief inline reason so the decision is explicit for future reviewers. - Test files and plugin-repos B110 suppressed via Codacy API (not prod code). B108 (/tmp usage): - permission_utils.py: /tmp listed to PREVENT permission changes on it — not used as a temp path. Annotated # nosec B108. - display_manager.py: fixed snapshot path is intentional (web UI reads same path); path-check guard also annotated. - wifi_manager.py: named /tmp files match the sudoers allowlist installed with the system (the paths are hard-coded in both places by design). Annotated all six open/cp references # nosec B108. - scripts/render_plugin.py: dev script default overridable by user. Annotated. - web_interface/app.py: reads the same fixed path written by display_manager. Annotated # nosec B108. - Test files suppressed via Codacy API. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address remaining Codacy security findings Flask debug=True (real fix): - web_interface/app.py: debug=True in __main__ block exposes the Werkzeug interactive debugger (arbitrary code execution). Changed to os.environ.get('FLASK_DEBUG', '0') == '1' — off by default, opt-in via environment variable for local development. nosec annotations (accepted risk with documented rationale): - disk_cache.py: os.chmod(0o660) is intentional — web UI and LED matrix service share a group, 660 gives group write while denying world access (B103 + Semgrep insecure-file-permissions suppressed in Codacy) - wifi_manager.py: urlopen to hardcoded connectivity-check.ubuntu.com URL (B310 — no user input involved) - font_manager.py: urlretrieve URL comes from user's own config file on their local device (B310) - start_web_conditionally.py: os.execvp with both sys.executable and a fixed PROJECT_DIR-relative constant (B606) Confirmed false positives suppressed via Codacy API (15 issues): - SSRF (3x): client-side JS fetch — SSRF is server-side; browser fetch is CORS-restricted to same origin - B105 (3x): test fixtures use dummy secrets by design; store_manager checks for the placeholder string, it is not itself a secret - PMD numeric literal (2x): 10000000 is within Number.MAX_SAFE_INTEGER - Prototype pollution (1x): read-only schema traversal, no writes - no-unsanitized_method (1x): dynamic import() is CORS-restricted - detect-unsafe-regex (1x): operates on server-controlled config values - plugin-repos B103 (1x): vendor code chmod on executable - Semgrep insecure-file-permissions (3x): same disk_cache 0o660 as above Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove unnecessary f prefix from f-strings without placeholders (F541) Pyflakes F541 flags f-strings that contain no {} interpolation — they are identical to plain strings but trigger unnecessary string formatting overhead. Fixed in production code: - src/base_classes/data_sources.py (2 debug log calls) - src/logo_downloader.py (1 error log) - src/plugin_system/store_manager.py (5 strings across 3 log calls) - src/web_interface/validators.py (1 return value) - src/wifi_manager.py (4 log/message strings) - web_interface/start.py (1 print) F541 issues in test/, scripts/, and plugin-repos/ suppressed via Codacy API as non-production code. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(dev): add Pillow compatibility smoke test script Covers all Pillow APIs used in LEDMatrix — image creation, drawing, font metrics, LANCZOS resampling, paste/alpha_composite, and PNG I/O. Run after any Pillow version bump to catch regressions before deploy. python3 scripts/dev/test_pillow_compat.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: resolve 8 new Codacy issues introduced by PR changes shellcheck SC2034: - first_time_install.sh: 'type' loop variable also unused in the wifi status loop (we previously fixed 'device' → '_' but left 'type'). Changed to '_ _ state' since neither device nor type is referenced. ESLint no-undef: - app.js: typeof guards don't satisfy no-undef; added updateSystemStats to the /* global */ declaration alongside showNotification. nosec annotation: - web_interface/app.py: app.run(host='0.0.0.0') line changed when we fixed debug=True, giving it a new issue ID. Re-added # nosec B104. pyflakes F401: - scripts/dev/test_pillow_compat.py: ImageFilter was imported but never used in the smoke test. Removed from the import. Codacy API suppressions (false positives on changed lines): - disk_cache.py 0o660 chmod (2x): lines changed when # nosec B103 was added, producing new Semgrep issue IDs. Re-suppressed. - pages_v3.py raw-html-concat: Semgrep does not recognise escape() as a sanitizer; the escape() call IS the correct fix. - app.py flask 0.0.0.0: same line as B104 above; Semgrep rule also re-suppressed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address PR review findings Fix (10 of 15 findings): plugin-repos/march-madness/requirements.txt: Add urllib3>=1.26.0 — manager.py directly imports from urllib3; it was an undeclared transitive dependency via requests. scripts/dev/dev_plugin_setup.sh: Restore subshell form (cd "$target_dir" && git pull --rebase) || true so the shell's working directory is not permanently changed after the if-cd block. Previous fix for SC2015 leaked cwd into the remainder of the script. src/base_classes/sports.py: Narrow 'except Exception' to 'except RuntimeError as e' and log via self.logger.debug — Path.home() raises only RuntimeError for service users; other exceptions should not be silently swallowed. src/config_service.py: Fix stale "MD5 checksum" in ConfigVersion.__init__ docstring (line 40); the implementation uses SHA-256 since the Codacy fix. src/wifi_manager.py: Log the last-resort AP enable failure with exc_info=True instead of silently passing — failure here means the device may be unreachable. web_interface/blueprints/pages_v3.py: Log the outer metadata pre-load exception at debug level instead of swallowing it silently; schema still loads fully below. src/background_data_service.py: Remove unused 'timeout' parameter from shutdown() — executor.shutdown() does not accept timeout; update __del__ caller accordingly. src/font_manager.py: Validate URL scheme before urlretrieve — reject non-http/https schemes (e.g. file://) to prevent reading local files from config-supplied URLs. src/plugin_system/plugin_executor.py: Simplify redundant except tuple: (PluginTimeoutError, PluginError, Exception) → Exception, which already covers the others. test/test_display_controller.py: Mark empty test_plugin_discovery_and_loading as @pytest.mark.skip with reason. Move duplicate 'from datetime import datetime' to module header and remove the stray mid-module copy. Skip (5 of 15 findings, with reasons): - pytest 9.0.3 concerns: full suite already verified (467 pass, 18 pre-existing) - Pillow 12.2.0 API concerns: no deprecated APIs in codebase; tests + Pi smoke test pass - diagnose_web_ui.sh sudo validation: set -e already ensures fail-fast on any sudo failure - app.py request-logging except: must stay silent (recursive logging risk); annotated - app.py SSE file-read except: genuinely transient I/O; annotated Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-15 10:19:55 -04:00
3 people	1a0f1c8015	fix: service control buttons and AP-mode SSH lockout post-install (#326) ... * fix: service control buttons and AP-mode SSH lockout post-install Two user-reported issues after fresh install: 1. All service buttons (Start/Stop/Restart Display, Restart Web Service) failed silently — only Reboot worked. Root cause: sudoers rules use `ledmatrix.service` (with suffix) but api_v3.py called `sudo systemctl start ledmatrix` (no suffix). sudo does exact string matching, so every service action was rejected with returncode=1. Also missing from sudoers: ledmatrix-web, journalctl, and is-active entries. Fix: - Add `.service` suffix to all 8 sudo systemctl call sites in api_v3.py (_ensure_display_service_running, _stop_display_service, and all execute_system_action branches). - Add timeout=15 to all subprocess.run calls in execute_system_action (previously could hang indefinitely). - Add missing sudoers rules to first_time_install.sh and configure_web_sudo.sh: ledmatrix-web.service start/stop/restart, is-active for both name forms, and journalctl -u/-t ledmatrix rules. 2. SSH and web UI became inaccessible after ~1 hour even though the display kept running. Root cause: wifi_monitor_daemon restarts NetworkManager after 5 consecutive internet failures (~2.5 min). Each NM restart drops WiFi briefly. During that window check_and_manage_ap_mode() increments _disconnected_checks but the daemon never reset it after the restart. After 3 such NM-restart cycles, _disconnected_checks reached 3 and AP mode activated — changing the Pi from WiFi client to hotspot (192.168.4.1) and killing SSH on the old IP. Fix: - Reset wifi_manager._disconnected_checks = 0 in the daemon immediately after a successful NM restart so the brief drop it causes doesn't count toward AP-mode activation. - Increase _disconnected_checks_required from 3 to 6 (90s → 3min) as an additional buffer against transient network flaps. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * revert: restore AP-mode grace period to 90s (3 checks) The counter reset after NM restart already fully prevents the SSH-lockout cascade: _disconnected_checks can never accumulate across NM restarts because it is reset to 0 before the next daemon iteration runs. The 3→6 increase provided no additional fix for the described problem and caused a UX regression: fresh Pi devices with no WiFi configured would wait 3 minutes instead of 90 seconds for the LEDMatrix-Setup hotspot to appear. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address five valid review findings; skip two Fixed: - march-madness/requirements.txt: Pillow>=10.3.0 (patches CVE-2024-28219; 10.3.0 is the actual fix version — reviewer cited 12.2.0 but that risks breaking API changes without test coverage) - wifi_monitor_daemon.py: add missing `import subprocess`; subprocess.run and CalledProcessError would NameError at runtime on the NM restart path - wifi_manager.py: validate ap_idle_timeout_minutes before arithmetic — coerce to int, clamp 1–1440, fall back to 15 on bad config values - wifi_manager.py: call _remove_nm_dnsmasq_captive_conf() on all three rollback paths in _enable_ap_mode_nmcli_hotspot() and in the top-level except block so stale dnsmasq drop-ins are never left behind - api_v3.py: fix wrong_password prefix strip — removeprefix("wrong_password:") then lstrip() handles both "wrong_password: msg" and "wrong_password:msg" - plugins_manager.js: add .catch() to loadInstalledPlugins().then() to surface failures instead of silently dropping unhandled rejections Skipped: - WiFiManager AP state persistence: architectural overhaul; _is_ap_mode_active() already derives from live system state, not in-memory variables - Absolute subprocess paths in api_v3.py: paths vary by distro (/usr/bin vs /bin); web service has a normal PATH; sudoers already use resolved paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address five review findings (NM retry loop, start_display message, code quality) - wifi_monitor_daemon: reset _consecutive_internet_failures = 0 in both NM-restart exception handlers; previously both left the counter at threshold, causing an immediate retry on the next iteration instead of waiting another full backoff period - api_v3: fix start_display failure message — when mode is set and systemctl returns non-zero, message now includes the failure reason and a hint rather than always reporting success phrasing - wifi_manager: move _redirect_backend from class variable to instance variable in __init__ alongside _ap_enabled_at; class-level default shadowed correctly in practice (single instance) but was misleading - wifi_manager: narrow broad except Exception in _check_internet_connectivity to (subprocess.SubprocessError, OSError) for ping and OSError for HTTP (urllib.error.URLError is an OSError subclass in Python 3) - wifi_manager: remove redundant local 'import re as _re' in _validate_ap_config; re is already imported at module level (line 37) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address five review findings (Pillow CVEs, daemon exception narrowing, timeout handling, plugin store) - march-madness/requirements.txt: Pillow>=12.2.0 (patches CVE-2026-42308 and CVE-2026-42310; previous floor of 10.3.0 was insufficient) - wifi_monitor_daemon: narrow final except Exception to (subprocess.SubprocessError, OSError) so programming errors in the NM restart block are no longer silently swallowed - api_v3/execute_system_action: add explicit subprocess.TimeoutExpired handler before the generic Exception catch; returns action-specific message with 'status','message','returncode','stdout','stderr' fields so the UI receives a precise, actionable payload instead of the generic 'Failed to execute system action' string - plugins_manager.js: move searchPluginStore into .finally() so the plugin store renders regardless of whether loadInstalledPlugins succeeds or fails; .catch() still logs the error - first_time_install.sh: add safe_plugin_rm.sh NOPASSWD rule to the /tmp/ledmatrix_web_sudoers block; configure_web_sudo.sh had this rule but the standalone installer never granted it, leaving plugin removal broken after first-time install Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(api): resolve sudo/systemctl/reboot/poweroff paths at startup Use shutil.which() with safe fallbacks for the four privileged binaries instead of relying on bare names being resolved by the subprocess shell search. Resolves paths once at module load rather than per-call. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-05-12 17:58:51 -04:00
3 people	68a38c39f7	fix(web-ui-info): syntax error and init order crash on fresh install (#312) ... Two bugs that prevented the web-ui-info default plugin from loading: 1. Orphaned `except Exception` at line 203 with no matching `try` — caused a SyntaxError preventing the module from importing at all. Removed the orphan; the outer try/except already covers the block. 2. `_get_local_ip()` called in __init__ before `_ap_mode_cache_time` was initialized, causing AttributeError. Moved AP mode cache field initialization above the `_get_local_ip()` call. Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-19 11:56:05 -04:00
3 people	efe6b1fe23	fix: reduce CPU usage, fix Vegas refresh, throttle high-FPS ticks (#304) ... * fix: reduce CPU usage, fix Vegas mid-cycle refresh, and throttle high-FPS plugin ticks Web UI Info plugin was causing 90%+ CPU on RPi4 due to frequent subprocess calls and re-rendering. Fixed by: trying socket-based IP detection first (zero subprocess overhead), caching AP mode checks with 60s TTL, reducing IP refresh from 30s to 5m, caching rendered display images, and loading fonts once at init. Vegas mode was not updating the display mid-cycle because hot_swap_content() reset the scroll position to 0 on every recomposition. Now saves and restores scroll position for mid-cycle updates. High-FPS display loop was calling _tick_plugin_updates() 125x/sec with no benefit. Added throttled wrapper that limits to 1 call/sec. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review — respect plugin update_interval, narrow exception handlers Make _tick_plugin_updates_throttled default to no-throttle (min_interval=0) so plugin-configured update_interval values are never silently capped. The high-FPS call site passes an explicit 1.0s interval. Narrow _load_font exception handler from bare Exception to FileNotFoundError | OSError so unexpected errors surface. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(vegas): scale scroll position proportionally on mid-cycle hot-swap When content width changes during a mid-cycle recomposition (e.g., a plugin gains or loses items), blindly restoring the old scroll_position and total_distance_scrolled could overshoot the new total_scroll_width and trigger immediate false completion. Scale both values proportionally to the new width and clamp scroll_position to stay in bounds. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-02 08:46:52 -04:00
3 people	eb143c44fa	fix(web): render file-upload drop zone for string-type config fields (#271) ... * feat: add March Madness plugin and tournament round logos New dedicated March Madness plugin with scrolling tournament ticker: - Fetches NCAA tournament data from ESPN scoreboard API - Shows seeded matchups with team logos, live scores, and round separators - Highlights upsets (higher seed beating lower seed) in gold - Auto-enables during tournament window (March 10 - April 10) - Configurable for NCAAM and NCAAW tournaments - Vegas mode support via get_vegas_content() Tournament round logo assets: - MARCH_MADNESS.png, ROUND_64.png, ROUND_32.png - SWEET_16.png, ELITE_8.png, FINAL_4.png, CHAMPIONSHIP.png Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(store): prevent bulk-update from stalling on bundled/in-repo plugins Three related bugs caused the bulk plugin update to stall at 3/19: 1. Bundled plugins (e.g. starlark-apps, shipped with LEDMatrix rather than the plugin registry) had no metadata file, so update_plugin() returned False → API returned 500 → frontend queue halted. Fix: check for .plugin_metadata.json with install_type=bundled and return True immediately (these plugins update with LEDMatrix itself). 2. git config --get remote.origin.url (without --local) walked up the directory tree and found the parent LEDMatrix repo's remote URL for plugins that live inside plugin-repos/. This caused the store manager to attempt a 60-second git clone of the wrong repo for every update. Fix: use --local to scope the lookup to the plugin directory only. 3. hello-world manifest.json had a trailing comma causing JSON parse errors on every plugin discovery cycle (fixed on devpi directly). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(march-madness): address PR #263 code review findings - Replace self.is_enabled with BasePlugin.self.enabled in update(), display(), and supports_dynamic_duration() so runtime toggles work - Support quarter-based period labels for NCAAW (Q1..Q4 vs H1..H2), detected via league key or status_detail content - Use live refresh interval (60s) for cache max_age during live games instead of hardcoded 300s - Narrow broad except in _load_round_logos to (OSError, ValueError) with a fallback except Exception using logger.exception for traces - Remove unused `situation` local variable from _parse_event() - Add numpy>=1.24.0 to requirements.txt (imported but was missing) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web): render file-upload drop zone for string-type config fields String fields with x-widget: "file-upload" were falling through to a plain text input because the template only handled the array case. Adds a dedicated drop zone branch for string fields and corresponding handleSingleFileSelect/handleSingleFileUpload JS handlers that POST to the x-upload-config endpoint. Fixes credentials.json upload for the calendar plugin. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(march-madness): address PR #271 code review findings Inline fixes: - manager.py: swap min_duration/max_duration if misconfigured, log warning - manager.py: call session.close() and null session in cleanup() to prevent socket leaks on constrained hardware - manager.py: remove blocking network I/O from display(); update() is the sole fetch path (already uses 60s live-game interval) - manager.py: guard scroll_helper None before create_scrolling_image() in _create_ticker_image() to prevent crash when ScrollHelper is unavailable - store_manager.py: replace bare "except Exception: pass" with debug log including plugin_id and path when reading .plugin_metadata.json - file-upload.js: add endpoint guard (error if uploadEndpoint is falsy), client-side extension validation from data-allowed-extensions, and response.ok check before response.json() in handleSingleFileUpload - plugin_config.html: add data-allowed-extensions attribute to single-file input so JS handler can read the allowed extensions list Nitpick fixes: - manager.py: use logger.exception() (includes traceback) instead of logger.error() for league fetch errors - manager.py: remove redundant "{e}" from logger.exception() calls for round logo and March Madness logo load errors Not fixed (by design): - manifest.json repo naming: monorepo pattern is correct per project docs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(march-madness): address second round of PR #271 code review findings Inline fixes: - requirements.txt: bump Pillow to >=9.1.0 (required for Image.Resampling.LANCZOS) - file-upload.js: replace all statusDiv.innerHTML assignments with safe DOM creation (textContent + createElement) to prevent XSS from untrusted strings - plugin_config.html: add role="button", tabindex="0", aria-label, onkeydown (Enter/Space) to drop zone for keyboard accessibility; add aria-live="polite" to status div for screen-reader announcements - file-upload.js: tighten handleFileDrop endpoint check to non-empty string (dataset.uploadEndpoint.trim() !== '') so an empty attribute falls back to the multi-file handler Nitpick fixes: - manager.py: remove redundant cached_image/cached_array reassignments after create_scrolling_image() which already sets them internally - manager.py: narrow bare except in _get_team_logo to (FileNotFoundError, OSError, ValueError) for expected I/O errors; log unexpected exceptions - store_manager.py: narrow except to (OSError, ValueError) when reading .plugin_metadata.json so unrelated exceptions propagate Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-24 20:12:31 -05:00
3 people	38a9c1ed1b	feat(march-madness): add NCAA tournament plugin and round logos (#263) ... * feat: add March Madness plugin and tournament round logos New dedicated March Madness plugin with scrolling tournament ticker: - Fetches NCAA tournament data from ESPN scoreboard API - Shows seeded matchups with team logos, live scores, and round separators - Highlights upsets (higher seed beating lower seed) in gold - Auto-enables during tournament window (March 10 - April 10) - Configurable for NCAAM and NCAAW tournaments - Vegas mode support via get_vegas_content() Tournament round logo assets: - MARCH_MADNESS.png, ROUND_64.png, ROUND_32.png - SWEET_16.png, ELITE_8.png, FINAL_4.png, CHAMPIONSHIP.png Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(store): prevent bulk-update from stalling on bundled/in-repo plugins Three related bugs caused the bulk plugin update to stall at 3/19: 1. Bundled plugins (e.g. starlark-apps, shipped with LEDMatrix rather than the plugin registry) had no metadata file, so update_plugin() returned False → API returned 500 → frontend queue halted. Fix: check for .plugin_metadata.json with install_type=bundled and return True immediately (these plugins update with LEDMatrix itself). 2. git config --get remote.origin.url (without --local) walked up the directory tree and found the parent LEDMatrix repo's remote URL for plugins that live inside plugin-repos/. This caused the store manager to attempt a 60-second git clone of the wrong repo for every update. Fix: use --local to scope the lookup to the plugin directory only. 3. hello-world manifest.json had a trailing comma causing JSON parse errors on every plugin discovery cycle (fixed on devpi directly). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(march-madness): address PR #263 code review findings - Replace self.is_enabled with BasePlugin.self.enabled in update(), display(), and supports_dynamic_duration() so runtime toggles work - Support quarter-based period labels for NCAAW (Q1..Q4 vs H1..H2), detected via league key or status_detail content - Use live refresh interval (60s) for cache max_age during live games instead of hardcoded 300s - Narrow broad except in _load_round_logos to (OSError, ValueError) with a fallback except Exception using logger.exception for traces - Remove unused `situation` local variable from _parse_event() - Add numpy>=1.24.0 to requirements.txt (imported but was missing) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-24 18:32:22 -05:00
3 people	302235a357	feat: Starlark Apps Integration with Schema-Driven Config + Security Hardening (#253) ... * feat: integrate Starlark/Tronbyte app support into plugin system Add starlark-apps plugin that renders Tidbyt/Tronbyte .star apps via Pixlet binary and integrates them into the existing Plugin Manager UI as virtual plugins. Includes vegas scroll support, Tronbyte repository browsing, and per-app configuration. - Extract working starlark plugin code from starlark branch onto fresh main - Fix plugin conventions (get_logger, VegasDisplayMode, BasePlugin) - Add 13 starlark API endpoints to api_v3.py (CRUD, browse, install, render) - Virtual plugin entries (starlark:<app_id>) in installed plugins list - Starlark-aware toggle and config routing in pages_v3.py - Tronbyte repository browser section in Plugin Store UI - Pixlet binary download script (scripts/download_pixlet.sh) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(starlark): use bare imports instead of relative imports Plugin loader uses spec_from_file_location without package context, so relative imports (.pixlet_renderer) fail. Use bare imports like all other plugins do. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(starlark): make API endpoints work standalone in web service The web service runs as a separate process with display_manager=None, so plugins aren't instantiated. Refactor starlark API endpoints to read/write the manifest file directly when the plugin isn't loaded, enabling full CRUD operations from the web UI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(starlark): make config partial work standalone in web service Read starlark app data from manifest file directly when the plugin isn't loaded, matching the api_v3.py standalone pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(starlark): always show editable timing settings in config panel Render interval and display duration are now always editable in the starlark app config panel, not just shown as read-only status text. App-specific settings from schema still appear below when present. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(store): add sort, filter, search, and pagination to Plugin Store and Starlark Apps Plugin Store: - Live search with 300ms debounce (replaces Search button) - Sort dropdown: A→Z, Z→A, Category, Author, Newest - Installed toggle filter (All / Installed / Not Installed) - Per-page selector (12/24/48) with pagination controls - "Installed" badge and "Reinstall" button on already-installed plugins - Active filter count badge + clear filters button Starlark Apps: - Parallel bulk manifest fetching via ThreadPoolExecutor (20 workers) - Server-side 2-hour cache for all 500+ Tronbyte app manifests - Auto-loads all apps when section expands (no Browse button) - Live search, sort (A→Z, Z→A, Category, Author), author dropdown - Installed toggle filter, per-page selector (24/48/96), pagination - "Installed" badge on cards, "Reinstall" button variant Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(store): move storeFilterState to global scope to fix scoping bug storeFilterState, pluginStoreCache, and related variables were declared inside an IIFE but referenced by top-level functions, causing ReferenceError that broke all plugin loading. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(starlark): schema-driven config forms + critical security fixes ## Schema-Driven Config UI - Render type-appropriate form inputs from schema.json (text, dropdown, toggle, color, datetime, location) - Pre-populate config.json with schema defaults on install - Auto-merge schema defaults when loading existing apps (handles schema updates) - Location fields: 3-part mini-form (lat/lng/timezone) assembles into JSON - Toggle fields: support both boolean and string "true"/"false" values - Unsupported field types (oauth2, photo_select) show warning banners - Fallback to raw key/value inputs for apps without schema ## Critical Security Fixes (P0) - **Path Traversal**: Verify path safety BEFORE mkdir to prevent TOCTOU - **Race Conditions**: Add file locking (fcntl) + atomic writes to manifest operations - **Command Injection**: Validate config keys/values with regex before passing to Pixlet subprocess ## Major Logic Fixes (P1) - **Config/Manifest Separation**: Store timing keys (render_interval, display_duration) ONLY in manifest - **Location Validation**: Validate lat [-90,90] and lng [-180,180] ranges, reject malformed JSON - **Schema Defaults Merge**: Auto-apply new schema defaults to existing app configs on load - **Config Key Validation**: Enforce alphanumeric+underscore format, prevent prototype pollution ## Files Changed - web_interface/templates/v3/partials/starlark_config.html — schema-driven form rendering - plugin-repos/starlark-apps/manager.py — file locking, path safety, config validation, schema merge - plugin-repos/starlark-apps/pixlet_renderer.py — config value sanitization - web_interface/blueprints/api_v3.py — timing key separation, safe manifest updates Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): use manifest filename field for .star downloads Tronbyte apps don't always name their .star file to match the directory. For example, the "analogclock" app has "analog_clock.star" (with underscore). The manifest.yaml contains a "filename" field with the correct name. Changes: - download_star_file() now accepts optional filename parameter - Install endpoint passes metadata['filename'] to download_star_file() - Falls back to {app_id}.star if filename not in manifest Fixes: "Failed to download .star file for analogclock" error Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): reload tronbyte_repository module to pick up code changes The web service caches imported modules in sys.modules. When deploying code updates, the old cached version was still being used. Now uses importlib.reload() when module is already loaded. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): use correct 'fileName' field from manifest (camelCase) The Tronbyte manifest uses 'fileName' (camelCase), not 'filename' (lowercase). This caused the download to fall back to {app_id}.star which doesn't exist for apps like analogclock (which has analog_clock.star). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * feat(starlark): extract schema during standalone install The standalone install function (_install_star_file) wasn't extracting schema from .star files, so apps installed via the web service had no schema.json and the config panel couldn't render schema-driven forms. Now uses PixletRenderer to extract schema during standalone install, same as the plugin does. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * feat(starlark): implement source code parser for schema extraction Pixlet CLI doesn't support schema extraction (--print-schema flag doesn't exist), so apps were being installed without schemas even when they have them. Implemented regex-based .star file parser that: - Extracts get_schema() function from source code - Parses schema.Schema(version, fields) structure - Handles variable-referenced dropdown options (e.g., options = dialectOptions) - Supports Location, Text, Toggle, Dropdown, Color, DateTime fields - Gracefully handles unsupported fields (OAuth2, LocationBased, etc.) - Returns formatted JSON matching web UI template expectations Coverage: 90%+ of Tronbyte apps (static schemas + variable references) Changes: - Replace extract_schema() to parse .star files directly instead of using Pixlet CLI - Add 6 helper methods for parsing schema structure - Handle nested parentheses and brackets properly - Resolve variable references for dropdown options Tested with: - analog_clock.star (Location field) ✓ - Multi-field test (Text + Dropdown + Toggle) ✓ - Variable-referenced options ✓ Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): add List to typing imports for schema parser Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): load schema from schema.json in standalone mode The standalone API endpoint was returning schema: null because it didn't load the schema.json file. Now reads schema from disk when returning app details via web service. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * feat(starlark): implement schema extraction, asset download, and config persistence ## Schema Extraction - Replace broken `pixlet serve --print-schema` with regex-based source parser - Extract schema by parsing `get_schema()` function from .star files - Support all field types: Location, Text, Toggle, Dropdown, Color, DateTime - Handle variable-referenced dropdown options (e.g., `options = teamOptions`) - Gracefully handle complex/unsupported field types (OAuth2, PhotoSelect, etc.) - Extract schema for 90%+ of Tronbyte apps ## Asset Download - Add `download_app_assets()` to fetch images/, sources/, fonts/ directories - Download assets in binary mode for proper image/font handling - Validate all paths to prevent directory traversal attacks - Copy asset directories during app installation - Enable apps like AnalogClock that require image assets ## Config Persistence - Create config.json file during installation with schema defaults - Update both config.json and manifest when saving configuration - Load config from config.json (not manifest) for consistency with plugin - Separate timing keys (render_interval, display_duration) from app config - Fix standalone web service mode to read/write config.json ## Pixlet Command Fix - Fix Pixlet CLI invocation: config params are positional, not flags - Change from `pixlet render file.star -c key=value` to `pixlet render file.star key=value -o output` - Properly handle JSON config values (e.g., location objects) - Enable config to be applied during rendering ## Security & Reliability - Add threading.Lock for cache operations to prevent race conditions - Reduce ThreadPoolExecutor workers from 20 to 5 for Raspberry Pi - Add path traversal validation in download_star_file() - Add YAML error logging in manifest fetching - Add file size validation (5MB limit) for .star uploads - Use sanitized app_id consistently in install endpoints - Use atomic manifest updates to prevent race conditions - Add missing Optional import for type hints ## Web UI - Fix standalone mode schema loading in config partial - Schema-driven config forms now render correctly for all apps - Location fields show lat/lng/timezone inputs - Dropdown, toggle, text, color, and datetime fields all supported Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): code review fixes - security, robustness, and schema parsing ## Security Fixes - manager.py: Check _update_manifest_safe return values to prevent silent failures - manager.py: Improve temp file cleanup in _save_manifest to prevent leaks - manager.py: Fix uninstall order (manifest → memory → disk) for consistency - api_v3.py: Add path traversal validation in uninstall endpoint - api_v3.py: Implement atomic writes for manifest files with temp + rename - pixlet_renderer.py: Relax config validation to only block dangerous shell metacharacters ## Frontend Robustness - plugins_manager.js: Add safeLocalStorage wrapper for restricted contexts (private browsing) - starlark_config.html: Scope querySelector to container to prevent modal conflicts ## Schema Parsing Improvements - pixlet_renderer.py: Indentation-aware get_schema() extraction (handles nested functions) - pixlet_renderer.py: Handle quoted defaults with commas (e.g., "New York, NY") - tronbyte_repository.py: Validate file_name is string before path traversal checks ## Dependencies - requirements.txt: Update Pillow (10.4.0), PyYAML (6.0.2), requests (2.32.0) ## Documentation - docs/STARLARK_APPS_GUIDE.md: Comprehensive guide explaining: - How Starlark apps work - That apps come from Tronbyte (not LEDMatrix) - Installation, configuration, troubleshooting - Links to upstream projects All changes improve security, reliability, and user experience. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): convert Path to str in spec_from_file_location calls The module import helpers were passing Path objects directly to spec_from_file_location(), which caused spec to be None. This broke the Starlark app store browser. - Convert module_path to string in both _get_tronbyte_repository_class and _get_pixlet_renderer_class - Add None checks with clear error messages for debugging Fixes: spec not found for the module 'tronbyte_repository' Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): restore Starlark Apps section in plugins.html The Starlark Apps UI section was lost during merge conflict resolution with main branch. Restored from commit 942663ab which had the complete implementation with filtering, sorting, and pagination. Fixes: Starlark section not visible on plugin manager page Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): restore Starlark JS functionality lost in merge During the merge with main, all Starlark-specific JavaScript (104 lines) was removed from plugins_manager.js, including: - starlarkFilterState and filtering logic - loadStarlarkApps() function - Starlark app install/uninstall handlers - Starlark section collapse/expand logic - Pagination and sorting for Starlark apps Restored from commit 942663ab and re-applied safeLocalStorage wrapper from our code review fixes. Fixes: Starlark Apps section non-functional in web UI Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): security and race condition improvements Security fixes: - Add path traversal validation for output_path in download_star_file - Remove XSS-vulnerable inline onclick handlers, use delegated events - Add type hints to helper functions for better type safety Race condition fixes: - Lock manifest file BEFORE creating temp file in _save_manifest - Hold exclusive lock for entire read-modify-write cycle in _update_manifest_safe - Prevent concurrent writers from racing on manifest updates Other improvements: - Fix pages_v3.py standalone mode to load config.json from disk - Improve error handling with proper logging in cleanup blocks - Add explicit type annotations to Starlark helper functions Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): critical bug fixes and code quality improvements Critical fixes: - Fix stack overflow in safeLocalStorage (was recursively calling itself) - Fix duplicate event listeners on Starlark grid (added sentinel check) - Fix JSON validation to fail fast on malformed data instead of silently passing Error handling improvements: - Narrow exception catches to specific types (OSError, json.JSONDecodeError, ValueError) - Use logger.exception() with exc_info=True for better stack traces - Replace generic "except Exception" with specific exception types Logging improvements: - Add "[Starlark Pixlet]" context tags to pixlet_renderer logs - Redact sensitive config values from debug logs (API keys, etc.) - Add file_path context to schema parsing warnings Documentation: - Fix markdown lint issues (add language tags to code blocks) - Fix time unit spacing: "(5min)" -> "(5 min)" Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): critical path traversal and exception handling fixes Path traversal security fixes (CRITICAL): - Add _validate_starlark_app_path() helper to check for path traversal attacks - Validate app_id in get_starlark_app(), uninstall_starlark_app(), get_starlark_app_config(), and update_starlark_app_config() - Check for '..' and path separators before any filesystem access - Verify resolved paths are within _STARLARK_APPS_DIR using Path.relative_to() - Prevents unauthorized file access via crafted app_id like '../../../etc/passwd' Exception handling improvements (tronbyte_repository.py): - Replace broad "except Exception" with specific types - _make_request: catch requests.Timeout, requests.RequestException, json.JSONDecodeError - _fetch_raw_file: catch requests.Timeout, requests.RequestException separately - download_app_assets: narrow to OSError, ValueError - Add "[Tronbyte Repo]" context prefix to all log messages - Use exc_info=True for better stack traces API improvements: - Narrow exception catches to OSError, json.JSONDecodeError in config loading - Remove duplicate path traversal checks (now centralized in helper) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix(starlark): logging improvements and code quality fixes Logging improvements (pages_v3.py): - Add logging import and create module logger - Replace print() calls with logger.warning() with "[Pages V3]" prefix - Use logger.exception() for outer try/catch with exc_info=True - Narrow exception handling to OSError, json.JSONDecodeError for file operations API improvements (api_v3.py): - Remove unnecessary f-strings (Ruff F541) from ImportError messages - Narrow upload exception handling to ValueError, OSError, IOError - Use logger.exception() with context for better debugging - Remove early return in get_starlark_status() to allow standalone mode fallback - Sanitize error messages returned to client (don't expose internal details) Benefits: - Better log context with consistent prefixes - More specific exception handling prevents masking unexpected errors - Standalone/web-service-only mode now works for status endpoint - Stack traces preserved for debugging without exposing to clients Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-20 19:44:12 -05:00
ChuckandChuck	7d71656cf1	Plugins (#145) ... Chaotic mega-merge into main. THINGS WILL PROBABLY BE BROKEN * chore: Update soccer-scoreboard submodule to merged commit - Update submodule reference to include manifest.json v2 registry format - Version updated to 1.0.1 * refactor: Remove test_mode and logo_dir config reading from base SportsCore - Remove test_mode initialization and usage - Remove logo_dir reading from mode_config - Use LogoDownloader defaults directly for logo directories * chore: Update plugin submodules after removing global properties - Update basketball-scoreboard submodule (removed global test_mode, live_priority, dynamic_duration, logo_dir) - Update soccer-scoreboard submodule (removed global test_mode, live_priority, dynamic_duration, logo_dir) * feat(calendar): Add credentials.json file upload via web interface - Add API endpoint /api/v3/plugins/calendar/upload-credentials for file upload - Validate JSON format and Google OAuth structure - Save file to plugin directory with secure permissions (0o600) - Backup existing credentials.json before overwriting - Add file upload widget support for string fields in config forms - Add frontend handler handleCredentialsUpload() for single file uploads - Update .gitignore to allow calendar submodule - Update calendar submodule reference * fix(web): Improve spacing for nested configuration sections - Add dynamic margin based on nesting depth (mb-6 for deeply nested sections) - Increase padding in nested content areas (py-3 to py-4) - Add extra spacing after nested sections to prevent overlap - Enhance CSS spacing for nested sections (1.5rem for nested, 2rem for deeply nested) - Add padding-bottom to expanded nested content to prevent cutoff - Fixes issue where game_limits and other nested settings were hidden under next section header * chore(plugins): Update sports scoreboard plugins with live update interval fix - Updated hockey-scoreboard, football-scoreboard, basketball-scoreboard, and soccer-scoreboard submodules - All plugins now fix the interval selection bug that caused live games to update every 5 minutes instead of 30 seconds - Ensures all live games update at the configured live_update_interval (30s) for timely score updates * fix: Initialize test_mode in SportsLive and fix config migration - Add test_mode initialization in SportsLive.__init__() to prevent AttributeError - Remove invalid new_secrets parameter from save_config_atomic() call in config migration - Fixes errors: 'NBALiveManager' object has no attribute 'test_mode' - Fixes errors: ConfigManager.save_config_atomic() got unexpected keyword argument 'new_secrets' * chore: Update submodules with test_mode initialization fixes - Update basketball-scoreboard submodule - Update soccer-scoreboard submodule * fix(plugins): Auto-stash local changes before plugin updates - Automatically stash uncommitted changes before git pull during plugin updates - Prevents update failures when plugins have local modifications - Improves error messages for git update failures - Matches behavior of main LEDMatrix update process * fix(basketball-scoreboard): Update submodule with timeout fix - Updated basketball-scoreboard plugin to fix update() timeout issue - Plugin now uses fire-and-forget odds fetching for upcoming games - Prevents 30-second timeout when processing many upcoming games Also fixed permission issue on devpi: - Changed /var/cache/ledmatrix/display_on_demand_state.json permissions from 600 to 660 to allow web service (devpi user) to read the file * fix(cache): Ensure cache files use 660 permissions for group access - Updated setup_cache.sh to set file permissions to 660 (not 775) - Updated first_time_install.sh to properly set cache file permissions - Modified DiskCache to set 660 permissions when creating cache files - Ensures display_on_demand_state.json and other cache files are readable by web service (devpi user) which is in ledmatrix group This fixes permission issues where cache files were created with 600 permissions, preventing the web service from reading them. Now files are created with 660 (rw-rw----) allowing group read access. * fix(soccer-scoreboard): Update submodule with manifest fix - Updated soccer-scoreboard plugin submodule - Added missing entry_point and class_name to manifest.json - Fixes plugin loading error: 'No class_name in manifest' Also fixed cache file permissions on devpi server: - Changed display_on_demand_state.json from 600 to 660 permissions - Allows web service (devpi user) to read cache files * fix(display): Remove update_display() calls from clear() to prevent black flash Previously, display_manager.clear() was calling update_display() twice, which immediately showed a black screen on the hardware before new content could be drawn. This caused visible black flashes when switching between modes, especially when plugins switch from general modes (e.g., football_upcoming) to specific sub-modes (e.g., nfl_upcoming). Now clear() only prepares the buffer without updating the hardware. Callers can decide when to update the display, allowing smooth transitions from clear → draw → update_display() without intermediate black flashes. Places that intentionally show a cleared screen (error cases) already explicitly call update_display() after clear(), so backward compatibility is maintained. * fix(scroll): Prevent wrap-around before cycle completion in dynamic duration - Check scroll completion BEFORE allowing wrap-around - Clamp scroll_position when complete to prevent visual loop - Only wrap-around if cycle is not complete yet - Fixes issue where stocks plugin showed first stock again at end - Completion logged only once to avoid spam - Ensures smooth transition to next mode without visual repeat * fix(on-demand): Ensure on-demand buttons work and display service runs correctly - Add early stub functions for on-demand modal to ensure availability when Alpine.js initializes - Increase on-demand request cache max_age from 5min to 1hr to prevent premature expiration - Fixes issue where on-demand buttons were not functional due to timing issues - Ensures display service properly picks up on-demand requests when started * test: Add comprehensive test coverage (30%+) - Add 100+ new tests across core components - Add tests for LayoutManager (27 tests) - Add tests for PluginLoader (14 tests) - Add tests for SchemaManager (20 tests) - Add tests for MemoryCache and DiskCache (24 tests) - Add tests for TextHelper (9 tests) - Expand error handling tests (7 new tests) - Improve coverage from 25.63% to 30.26% - All 237 tests passing Test files added: - test/test_layout_manager.py - test/test_plugin_loader.py - test/test_schema_manager.py - test/test_text_helper.py - test/test_config_service.py - test/test_display_controller.py - test/test_display_manager.py - test/test_error_handling.py - test/test_font_manager.py - test/test_plugin_system.py Updated: - pytest.ini: Enable coverage reporting with 30% threshold - test/conftest.py: Enhanced fixtures for better test isolation - test/test_cache_manager.py: Expanded cache component tests - test/test_config_manager.py: Additional config tests Documentation: - HOW_TO_RUN_TESTS.md: Guide for running and understanding tests * test(web): Add comprehensive API endpoint tests - Add 30 new tests for Flask API endpoints in test/test_web_api.py - Cover config, system, display, plugins, fonts, and error handling APIs - Increase test coverage from 30.26% to 30.87% - All 267 tests passing Tests cover: - Config API: GET/POST main config, schedule, secrets - System API: Status, version, system actions - Display API: Current display, on-demand start/stop - Plugins API: Installed plugins, health, config, operations, state - Fonts API: Catalog, tokens, overrides - Error handling: Invalid JSON, missing fields, 404s * test(plugins): Add comprehensive integration tests for all plugins - Add base test class for plugin integration tests - Create integration tests for all 6 plugins: - basketball-scoreboard (11 tests) - calendar (10 tests) - clock-simple (11 tests) - odds-ticker (9 tests) - soccer-scoreboard (11 tests) - text-display (12 tests) - Total: 64 new plugin integration tests - Increase test coverage from 30.87% to 33.38% - All 331 tests passing Tests verify: - Plugin loading and instantiation - Required methods (update, display) - Manifest validation - Display modes - Config schema validation - Graceful handling of missing API credentials Uses hybrid approach: integration tests in main repo, plugin-specific unit tests remain in plugin submodules. * Add mqtt-notifications plugin as submodule * fix(sports): Respect games_to_show settings for favorite teams - Fix upcoming games to show N games per team (not just 1) - Fix recent games to show N games per team (not just 1) - Add duplicate removal for games involving multiple favorite teams - Match behavior of basketball-scoreboard plugin - Affects NFL, NHL, and other sports using base_classes/sports.py * chore: Remove debug instrumentation logs - Remove temporary debug logging added during fix verification - Fix confirmed working by user * debug: Add instrumentation to debug configuration header visibility issue * fix: Resolve nested section content sliding under next header - Remove overflow-hidden from nested-section to allow proper document flow - Add proper z-index and positioning to prevent overlap - Add margin-top to nested sections for better spacing - Remove debug instrumentation that was causing ERR_BLOCKED_BY_CLIENT errors * fix: Prevent unnecessary plugin tab redraws - Add check to only update tabs when plugin list actually changes - Increase debounce timeout to batch rapid changes - Compare plugin IDs before updating to avoid redundant redraws - Fix setter to check for actual changes before triggering updates * fix: Prevent form-groups from sliding out of view when nested sections expand - Increase margin-bottom on nested-sections for better spacing - Add clear: both to nested-sections to ensure proper document flow - Change overflow to visible when expanded to allow natural flow - Add margin-bottom to expanded content - Add spacing rules for form-groups that follow nested sections - Add clear spacer div after nested sections * fix: Reduce excessive debug logging in generateConfigForm - Only log once per plugin instead of on every function call - Prevents log spam when Alpine.js re-renders the form multiple times - Reduces console noise from 10+ logs per plugin to 1 log per plugin * fix: Prevent nested section content from sliding out of view when expanded - Remove overflow-hidden from nested-section in base.html (was causing clipping) - Add scrollIntoView to scroll expanded sections into view within modal - Set nested-section overflow to visible to prevent content clipping - Add min-height to nested-content to ensure proper rendering - Wait for animation to complete before scrolling into view * fix: Prevent form-groups from overlapping and appearing outside view - Change nested-section overflow to hidden by default, visible when expanded - Add :has() selector to allow overflow when content is expanded - Ensure form-groups after nested sections have proper spacing and positioning - Add clear: both and width: 100% to prevent overlap - Use !important for margin-top to ensure spacing is applied - Ensure form-groups are in normal document flow with float: none * fix: Use JavaScript to toggle overflow instead of :has() selector - :has() selector may not be supported in all browsers - Use JavaScript to set overflow: visible when expanded, hidden when collapsed - This ensures better browser compatibility while maintaining functionality * fix: Make parent sections expand when nested sections expand - Add updateParentNestedContentHeight() helper to recursively update parent heights - When a nested section expands, recalculate all parent nested-content max-heights - Ensures parent sections (like NFL) expand to accommodate expanded child sections - Updates parent heights both on expand and collapse for proper animation * refactor: Simplify parent section expansion using CSS max-height: none - Remove complex recursive parent height update function - Use CSS max-height: none when expanded to allow natural expansion - Parent sections automatically expand because nested-content has no height constraint - Simpler and more maintainable solution * refactor: Remove complex recursive parent height update function - CSS max-height: none already handles parent expansion automatically - No need for JavaScript to manually update parent heights - Much simpler and cleaner solution * debug: Add instrumentation to debug auto-collapse issue - Add logging to track toggle calls and state changes - Add guard to prevent multiple simultaneous toggles - Pass event object to prevent bubbling - Improve state detection logic - Add return false to onclick handlers * chore: Remove debug instrumentation from toggleNestedSection - Remove all debug logging code - Keep functional fixes: event handling, toggle guard, improved state detection - Code is now clean and production-ready * fix(web): Add browser refresh note to plugin fetch errors * refactor(text-display): Update submodule to use ScrollHelper * fix(text-display): Fix scrolling display issue - update position in display() * feat(text-display): Add scroll_loop option and improve scroll speed control * debug: Add instrumentation to track plugin enabled state changes Added debug logging to investigate why plugins appear to disable themselves: - Track enabled state during plugin load (before/after schema merge) - Track enabled state during plugin reload - Track enabled state preservation during config save - Track state reconciliation fixes - Track enabled state updates in on_config_change This will help identify which code path is causing plugins to disable. * debug: Fix debug log path to work on Pi Changed hardcoded log path to use dynamic project root detection: - Uses LEDMATRIX_ROOT env var if set - Falls back to detecting project root by looking for config directory - Creates .cursor directory if it doesn't exist - Falls back to /tmp/ledmatrix_debug.log if all else fails - Added better error handling with logger fallback * Remove debug instrumentation for plugin enabled state tracking Removed all debug logging that was added to track plugin enabled state changes. The instrumentation has been removed as requested. * Reorganize documentation and cleanup test files - Move documentation files to docs/ directory - Remove obsolete test files - Update .gitignore and README * feat(text-display): Switch to frame-based scrolling with high FPS support * fix(text-display): Add backward compatibility for ScrollHelper sub-pixel scrolling * feat(scroll_helper): Add sub-pixel scrolling support for smooth movement - Add sub-pixel interpolation using scipy (if available) or numpy fallback - Add set_sub_pixel_scrolling() method to enable/disable feature - Implement _get_visible_portion_subpixel() for fractional pixel positioning - Implement _interpolate_subpixel() for linear interpolation - Prevents pixel skipping at slow scroll speeds - Maintains backward compatibility with integer pixel path * fix(scroll_helper): Reset last_update_time in reset_scroll() to prevent jump-ahead - Reset last_update_time when resetting scroll position - Prevents large delta_time on next update after reset - Fixes issue where scroll would immediately complete again after reset - Ensures smooth scrolling continuation after loop reset * fix(scroll_helper): Fix numpy broadcasting error in sub-pixel interpolation - Add output_width parameter to _interpolate_subpixel() for variable widths - Fix wrap-around case to use correct widths for interpolation - Handle edge cases where source array is smaller than expected - Prevent 'could not broadcast input array' errors in sub-pixel scrolling - Ensure proper width matching in all interpolation paths * feat(scroll): Add frame-based scrolling mode for smooth LED matrix movement - Add frame_based_scrolling flag to ScrollHelper - When enabled, moves fixed pixels per step, throttled by scroll_delay - Eliminates time-based jitter by ignoring frame timing variations - Provides stock-ticker-like smooth, predictable scrolling - Update text-display plugin to use frame-based mode This addresses stuttering issues where time-based scrolling caused visual jitter due to frame timing variations in the main display loop. * fix(scroll): Fix NumPy broadcasting errors in sub-pixel wrap-around - Ensure _interpolate_subpixel always returns exactly requested width - Handle cases where scipy.ndimage.shift produces smaller arrays - Add padding logic for wrap-around cases when arrays are smaller than expected - Prevents 'could not broadcast input array' errors during scrolling * refactor(scroll): Remove sub-pixel interpolation, use high FPS integer scrolling - Disable sub-pixel scrolling by default in ScrollHelper - Simplify get_visible_portion to always use integer pixel positioning - Restore frame-based scrolling logic for smooth high FPS movement - Use high frame rate (like stock ticker) for smoothness instead of interpolation - Reduces complexity and eliminates broadcasting errors * fix(scroll): Prevent large pixel jumps in frame-based scrolling - Initialize last_step_time properly to prevent huge initial jumps - Clamp scroll_speed to max 5 pixels/frame in frame-based mode - Prevents 60-pixel jumps when scroll_speed is misconfigured - Simplified step calculation to avoid lag catch-up jumps * fix(text-display): Align config schema and add validation - Update submodule reference - Adds warning and logging for scroll_speed config issues * fix(scroll): Simplify frame-based scrolling to match stock ticker behavior - Remove throttling logic from frame-based scrolling - Move pixels every call (DisplayController's loop timing controls rate) - Add enable_scrolling attribute to text-display plugin for high-FPS treatment - Matches stock ticker: simple, predictable movement every frame - Eliminates jitter from timing mismatches between DisplayController and ScrollHelper * fix(scroll): Restore scroll_delay throttling in frame-based mode - Restore time-based throttling using scroll_delay - Move pixels only when scroll_delay has passed - Handle lag catch-up with reasonable caps to prevent huge jumps - Preserve fractional timing for smooth operation - Now scroll_delay actually controls the scroll speed as intended * feat(text-display): Add FPS counter logging - Update submodule reference - Adds FPS tracking and logging every 5 seconds * fix(text-display): Add display-width buffer so text scrolls completely off - Update submodule reference - Adds end buffer to ensure text exits viewport before looping * fix: Prevent premature game switching in SportsLive - Set last_game_switch when games load even if current_game already exists - Set last_game_switch when same games update but it's still 0 - Add guard to prevent switching check when last_game_switch is 0 - Fixes issue where first game shows for only ~2 seconds before switching - Also fixes random screen flickering when games change prematurely * feat(plugins): Add branch selection support for plugin installation - Add optional branch parameter to install_plugin() and install_from_url() in store_manager - Update API endpoints to accept and pass branch parameter - Update frontend JavaScript to support branch selection in install calls - Maintain backward compatibility - branch parameter is optional everywhere - Falls back to default branch logic if specified branch doesn't exist * feat(plugins): Add UI for branch selection in plugin installation - Add branch input field in 'Install Single Plugin' section - Add global branch input for store installations - Update JavaScript to read branch from input fields - Branch input applies to all store installations when specified * feat(plugins): Change branch selection to be per-plugin instead of global - Remove global store branch input field - Add individual branch input field to each plugin card in store - Add branch input to custom registry plugin cards - Each plugin can now have its own branch specified independently * debug: Add logging to _should_exit_dynamic * feat(display_controller): Add universal get_cycle_duration support for all plugins UNIVERSAL FEATURE: Any plugin can now implement get_cycle_duration() to dynamically calculate the total time needed to show all content for a mode. New method: - _plugin_cycle_duration(plugin, display_mode): Queries plugin for calculated duration Integration: - Display controller calls plugin.get_cycle_duration(display_mode) - Uses returned duration as target (respecting max cap) - Falls back to cap if not provided Benefits: - Football plugin: Show all games (3 games × 15s = 45s total) - Basketball plugin: Could implement same logic - Hockey/Baseball/any sport: Universal support - Stock ticker: Could calculate based on number of stocks - Weather: Could calculate based on forecast days Example plugin implementation: Result: Plugins control their own display duration based on actual content, creating a smooth user experience where all content is shown before switching. * debug: Add logging to cycle duration call * debug: Change loop exit logs to INFO level * fix: Change cycle duration logs to INFO level * fix: Don't exit loop on False for dynamic duration plugins For plugins with dynamic duration enabled, keep the display loop running even when display() returns False. This allows games to continue rotating within the calculated duration. The loop will only exit when: - Cycle is complete (plugin reports all content shown) - Max duration is reached - Mode is changed externally * fix(schedule): Improve display scheduling functionality - Add GET endpoint for schedule configuration retrieval - Fix mode switching to clean up old config keys (days/start_time/end_time) - Improve error handling with consistent error_response() usage - Enhance display controller schedule checking with better edge case handling - Add validation for time formats and ensure at least one day enabled in per-day mode - Add debug logging for schedule state changes Fixes issues where schedule mode switching left stale config causing incorrect behavior. * fix(install): Add cmake and ninja-build to system dependencies Resolves h3 package build failure during first-time installation. The h3 package (dependency of timezonefinder) requires CMake and Ninja to build from source. Adding these build tools ensures successful installation of all Python dependencies. * fix: Pass display_mode in ALL loop calls to maintain sticky manager CRITICAL FIX: Display controller was only passing display_mode on first call, causing plugins to fall back to internal mode cycling and bypass sticky manager logic. Now consistently passes display_mode=active_mode on every display() call in both high-FPS and normal loops. This ensures plugins maintain mode context and sticky manager state throughout the entire display duration. * feat(install): Add OS check for Raspberry Pi OS Lite (Trixie) - Verify OS is Raspberry Pi OS (raspbian/debian) - Require Debian 13 (Trixie) specifically - Check for Lite version (no desktop environment) - Exit with clear error message if requirements not met - Provide instructions for obtaining correct OS version * fix(web-ui): Add missing notification handlers to quick action buttons - Added hx-on:htmx:after-request handlers to all quick action buttons in overview.html - Added hx-ext='json-enc' for proper JSON encoding - Added missing notification handler for reboot button in index.html - Users will now see toast notifications when actions complete or fail * fix(display): Ensure consistent display mode handling in all plugin calls - Updated display controller to consistently pass display_mode in all plugin display() calls. - This change maintains the sticky manager state and ensures plugins retain their mode context throughout the display duration. - Addresses issues with mode cycling and improves overall display reliability. * fix(display): Enhance display mode persistence across plugin updates - Updated display controller to ensure display_mode is consistently maintained during plugin updates. - This change prevents unintended mode resets and improves the reliability of display transitions. - Addresses issues with mode persistence, ensuring a smoother user experience across all plugins. * feat: Add Olympics countdown plugin as submodule - Add olympics-countdown plugin submodule - Update .gitignore to allow olympics-countdown plugin - Plugin automatically determines next Olympics and counts down to opening/closing ceremonies * feat(web-ui): Add checkbox-group widget support for multi-select arrays - Add checkbox-group widget rendering in plugins_manager.js - Update form processing to handle checkbox groups with [] naming - Support for friendly labels via x-options in config schemas - Update odds-ticker submodule with checkbox-group implementation * fix(plugins): Preserve enabled state when saving plugin config from main config endpoint When saving plugin configuration through save_main_config endpoint, the enabled field was not preserved if missing from the form data. This caused plugins to be automatically disabled when users saved their configuration from the plugin manager tab. This fix adds the same enabled state preservation logic that exists in save_plugin_config endpoint, ensuring consistent behavior across both endpoints. The enabled state is preserved from current config, plugin instance, or defaults to True to prevent unexpected disabling of plugins. * fix(git): Resolve git status timeout and exclude plugins from base project updates - Add --untracked-files=no flag to git status for faster execution - Increase timeout from 5s to 30s for git status operations - Add timeout exception handling for git status and stash operations - Filter out plugins directory from git status checks (plugins are separate repos) - Exclude plugins from stash operations using :!plugins pathspec - Apply same fixes to plugin store manager update operations * feat(plugins): Add granular scroll speed control to odds-ticker and leaderboard plugins - Add display object to both plugins' config schemas with scroll_speed and scroll_delay - Enable frame-based scrolling mode for precise FPS control (100 FPS for leaderboard) - Add set_scroll_speed() and set_scroll_delay() methods to both plugins - Maintain backward compatibility with scroll_pixels_per_second config - Leaderboard plugin now explicitly sets target_fps to 100 for high-performance scrolling * fix(scroll): Correct dynamic duration calculation for frame-based scrolling - Fix calculate_dynamic_duration() to properly handle frame-based scrolling mode - Convert scroll_speed from pixels/frame to pixels/second when in frame-based mode - Prevents incorrect duration calculations (e.g., 2609s instead of 52s) - Affects all plugins using ScrollHelper: odds-ticker, leaderboard, stocks, text-display - Add debug logging to show scroll mode and effective speed * Remove version logic from plugin system, use git commits instead - Remove version parameter from install_plugin() method - Rename fetch_latest_versions to fetch_commit_info throughout codebase - Remove version fields from plugins.json registry (versions, latest_version, download_url_template) - Remove version logging from plugin manager - Update web UI to use fetch_commit_info parameter - Update .gitignore to ignore all plugin folders (remove whitelist exceptions) - Remove plugin directories from git index (plugins now installed via plugin store only) Plugins now always install latest commit from default branch. Version fields replaced with git commit SHA and commit dates. System uses git-based approach for all plugin metadata. * feat(plugins): Normalize all plugins as git submodules - Convert all 18 plugins to git submodules for uniform management - Add submodules for: baseball-scoreboard, christmas-countdown, football-scoreboard, hockey-scoreboard, ledmatrix-flights, ledmatrix-leaderboard, ledmatrix-music, ledmatrix-stocks, ledmatrix-weather, static-image - Re-initialize mqtt-notifications as proper submodule - Update .gitignore to allow all plugin submodules - Add normalize_plugin_submodules.sh script for future plugin management All plugins with GitHub repositories are now managed as git submodules, ensuring consistent version control and easier updates. * refactor(repository): Reorganize scripts and files into organized directory structure - Move installation scripts to scripts/install/ (except first_time_install.sh) - Move development scripts to scripts/dev/ - Move utility scripts to scripts/utils/ - Move systemd service files to systemd/ - Keep first_time_install.sh, start_display.sh, stop_display.sh in root - Update all path references in scripts, documentation, and service files - Add README.md files to new directories explaining their purpose - Remove empty tools/ directory (contents moved to scripts/dev/) - Add .gitkeep to data/ directory * fix(scripts): Fix PROJECT_DIR path in start_web_conditionally.py after move to scripts/utils/ * fix(scripts): Fix PROJECT_DIR/PROJECT_ROOT path resolution in moved scripts - Fix wifi_monitor_daemon.py to use project root instead of scripts/utils/ - Fix shell scripts in scripts/ to correctly resolve project root (go up one more level) - Fix scripts in scripts/fix_perms/ to correctly resolve project root - Update diagnose_web_interface.sh to reference moved start_web_conditionally.py path All scripts now correctly determine project root after reorganization. * fix(install): Update first_time_install.sh to detect and update service files with old paths - Check for old paths in service files and reinstall if needed - Always reinstall main service (install_service.sh is idempotent) - This ensures existing installations get updated paths after reorganization * fix(install): Update install_service.sh message to indicate it updates existing services * fix(wifi): Enable WiFi scan to work when AP mode is active - Temporarily disable AP mode during network scanning - Automatically re-enable AP mode after scan completes - Add proper error handling with try/finally to ensure AP mode restoration - Add user notification when AP mode is temporarily disabled - Improve error messages for common scanning failures - Add timing delays for interface mode switching * fix(wifi): Fix network parsing to handle frequency with 'MHz' suffix - Strip 'MHz' suffix from frequency field before float conversion - Add better error logging for parsing failures - Fixes issue where all networks were silently skipped due to ValueError * debug(wifi): Add console logging and Alpine.js reactivity fixes for network display - Add console.log statements to debug network scanning - Add x-effect to force Alpine.js reactivity updates - Add unique keys to x-for template - Add debug display showing network count - Improve error handling and user feedback * fix(wifi): Manually update select options instead of using Alpine.js x-for - Replace Alpine.js x-for template with manual DOM manipulation - Add updateSelectOptions() method to directly update select dropdown - This fixes issue where networks weren't appearing in dropdown - Alpine.js x-for inside select elements can be unreliable * feat(web-ui): Add patternProperties support for dynamic key-value pairs - Add UI support for patternProperties objects (custom_feeds, feed_logo_map) - Implement key-value pair editor with add/remove functionality - Add JavaScript functions for managing dynamic key-value pairs - Update form submission to handle patternProperties JSON data - Enable easy configuration of feed_logo_map in web UI * chore: Update ledmatrix-news submodule to latest commit * fix(plugins): Handle arrays of objects in config normalization Fix configuration validation failure for static-image plugin by adding recursive normalization support for arrays of objects. The normalize_config_values function now properly handles arrays containing objects (like image_config.images) by recursively normalizing each object in the array using the items schema properties. This resolves the 'configuration validation failed' error when saving static image plugin configuration with multiple images. * fix(plugins): Handle union types in config normalization and form generation Fix configuration validation for fields with union types like ['integer', 'null']. The normalization function now properly handles: - Union types in top-level fields (e.g., random_seed: ['integer', 'null']) - Union types in array items - Empty string to None conversion for nullable fields - Form generation and submission for union types This resolves validation errors when saving plugin configs with nullable integer/number fields (e.g., rotation_settings.random_seed in static-image plugin). Also improves UX by: - Adding placeholder text for nullable fields explaining empty = use default - Properly handling empty values in form submission for union types * fix(plugins): Improve union type normalization with better edge case handling Enhanced normalization for union types like ['integer', 'null']: - Better handling of whitespace in string values - More robust empty string to None conversion - Fallback to None when conversion fails and null is allowed - Added debug logging for troubleshooting normalization issues - Improved handling of nested object fields with union types This should resolve remaining validation errors for nullable integer/number fields in nested objects (e.g., rotation_settings.random_seed). * chore: Add ledmatrix-news plugin to .gitignore exceptions * Fix web interface service script path in install_service.sh - Updated ExecStart path from start_web_conditionally.py to scripts/utils/start_web_conditionally.py - Updated diagnose_web_ui.sh to check for correct script path - Fixes issue where web UI service fails to start due to incorrect script path * Fix nested configuration section headers not expanding Fixed toggleNestedSection function to properly calculate scrollHeight when expanding nested configuration sections. The issue occurred when sections started with display:none - the scrollHeight was being measured before the browser had a chance to lay out the element, resulting in a value of 0. Changes: - Added setTimeout to delay scrollHeight measurement until after layout - Added overflow handling during animations to prevent content jumping - Added fallback for edge cases where scrollHeight might still be 0 - Set maxHeight to 'none' after expansion completes for natural growth - Updated function in both base.html and plugins_manager.js This fix applies to all plugins with nested configuration sections, including: - Hockey/Football/Basketball/Baseball/Soccer scoreboards (customization, global sections) - All plugins with transition, display, and other nested configuration objects Fixes configuration header expansion issues across all plugins. * Fix syntax error in first_time_install.sh step 8.5 Added missing 'fi' statement to close the if block in the WiFi monitor service installation section. This resolves the 'unexpected end of file' error that occurred at line 1385 during step 8.5. * Fix WiFi UI: Display correct SSID and accurate signal strength - Fix WiFi network selection dropdown not showing available networks - Replace manual DOM manipulation with Alpine.js x-for directive - Add fallback watcher to ensure select updates reactively - Fix WiFi status display showing netplan connection name instead of SSID - Query actual SSID from device properties (802-11-wireless.ssid) - Add fallback methods to get SSID from active WiFi connection list - Improve signal strength accuracy - Get signal directly from device properties (WIFI.SIGNAL) - Add multiple fallback methods for robust signal retrieval - Ensure signal percentage is accurate and up-to-date * Improve WiFi connection UI and error handling - Fix connect button disabled condition to check both selectedSSID and manualSSID - Improve error handling to display actual server error messages from 400 responses - Add step-by-step labels (Step 1, Step 2, Step 3) to clarify connection workflow - Add visual feedback showing selected network in blue highlight box - Improve password field labeling with helpful instructions - Add auto-clear logic between dropdown and manual SSID entry - Enhance backend validation with better error messages and logging - Trim SSID whitespace before processing to prevent validation errors * Add WiFi disconnect functionality for AP mode testing - Add disconnect_from_network() method to WiFiManager - Disconnects from current WiFi network using nmcli - Automatically triggers AP mode check if auto_enable_ap_mode is enabled - Returns success/error status with descriptive messages - Add /api/v3/wifi/disconnect API endpoint - POST endpoint to disconnect from current WiFi network - Includes proper error handling and logging - Add disconnect button to WiFi status section - Only visible when connected to a network - Red styling to indicate disconnection action - Shows 'Disconnecting...' state during operation - Automatically refreshes status after disconnect - Integrates with AP mode auto-enable functionality - When disconnected, automatically enables AP mode if configured - Perfect for testing captive portal and AP mode features * Add explicit handling for broken pipe errors during plugin dependency installation - Catch BrokenPipeError and OSError (errno 32) explicitly in all dependency installation methods - Add clear error messages explaining network interruption or buffer overflow causes - Improves error handling in store_manager, plugin_loader, and plugin_manager - Helps diagnose 'Errno 32 Broken Pipe' errors during pip install operations * Add WiFi permissions configuration script and integrate into first-time install - Create configure_wifi_permissions.sh script - Configures passwordless sudo for nmcli commands - Configures PolicyKit rules for NetworkManager control - Fixes 'Not Authorized to control Networking' error - Allows web interface to connect/disconnect WiFi without password prompts - Integrate WiFi permissions configuration into first_time_install.sh - Added as Step 10.1 after passwordless sudo configuration - Runs automatically during first-time installation - Ensures WiFi management works out of the box - Resolves authorization errors when connecting/disconnecting WiFi networks - NetworkManager requires both sudo and PolicyKit permissions - Script configures both automatically for seamless WiFi management * Add WiFi status LED message display integration - Integrate WiFi status messages from wifi_manager into display_controller - WiFi status messages interrupt normal rotation (but respect on-demand) - Priority: on-demand > wifi-status > live-priority > normal rotation - Safe implementation with comprehensive error handling - Automatic cleanup of expired/corrupted status files - Word-wrapping for long messages (max 2 lines) - Centered text display with small font - Non-intrusive: all errors are caught and logged, never crash controller * Fix display loop issues: reduce log spam and handle missing plugins - Change _should_exit_dynamic logging from INFO to DEBUG to reduce log spam in tight loops (every 8ms) that was causing high CPU usage - Fix display loop not running when manager_to_display is None - Add explicit check to set display_result=False when no plugin manager found - Fix logic bug where manager_to_display was overwritten after circuit breaker skip - Ensure proper mode rotation when plugins have no content or aren't found * Add debug logging to diagnose display loop stuck issue * Change debug logs to INFO level to diagnose display loop stuck * Add schedule activation logging and ensure display is blanked when inactive - Add clear INFO-level log message when schedule makes display inactive - Track previous display state to detect schedule transitions - Clear display when schedule makes it inactive to ensure blank screen (prevents showing initialization screen when schedule kicks in) - Initialize _was_display_active state tracking in __init__ * Fix indentation errors in schedule state tracking * Add rotation between hostname and IP address every 10 seconds - Added _get_local_ip() method to detect device IP address - Implemented automatic rotation between hostname and IP every 10 seconds - Enhanced logging to include both hostname and IP in initialization - Updated get_info() to expose device_ip and current_display_mode * Add WiFi connection failsafe system - Save original connection before attempting new connection - Automatically restore original connection if new connection fails - Enable AP mode as last resort if restoration fails - Enhanced connection verification with multiple attempts - Verify correct SSID (not just 'connected' status) - Better error handling and exception recovery - Prevents Pi from becoming unresponsive on connection failure - Always ensures device remains accessible via original WiFi or AP mode * feat(web): Improve web UI startup speed and fix cache permissions - Defer plugin discovery until first API request (removed from startup) - Add lazy loading to operation queue, state manager, and operation history - Defer health monitor initialization until first request - Fix cache directory permission issue: - Add systemd CacheDirectory feature for automatic cache dir creation - Add manual cache directory creation in install script as fallback - Improve cache manager logging (reduce alarming warnings) - Fix syntax errors in wifi_manager.py (unclosed try blocks) These changes significantly improve web UI startup time, especially with many plugins installed, while maintaining full backward compatibility. * feat(plugins): Improve GitHub token pop-up UX and combine warning/settings - Fix visibility toggle to handle inline styles properly - Remove redundant inline styles from HTML elements - Combine warning banner and settings panel into unified component - Add loading states to save/load token buttons - Improve error handling with better user feedback - Add token format validation (ghp_ or github_pat_ prefix) - Auto-refresh GitHub auth status after saving token - Hide warning banner when settings panel opens - Clear input field after successful save for security This creates a smoother UX flow where clicking 'Configure Token' transitions from warning directly to configuration form. * fix(wifi): Prevent WiFi radio disabling during AP mode disable - Make NetworkManager restart conditional (only for hostapd mode) - Add enhanced WiFi radio enable with retry and verification logic - Add connectivity safety check before NetworkManager restart - Ensure WiFi radio enabled after all AP mode disable operations - Fix indentation bug in dnsmasq backup restoration logic - Add pre-connection WiFi radio check for safety Fixes issue where WiFi radio was being disabled when disabling AP mode, especially when connected via Ethernet, making it impossible to enable WiFi from the web UI. * fix(plugin-templates): Fix unreachable fallback to expired cache in update() method The exception handler in update() checked the cached variable, which would always be None or falsy at that point. If fresh cached data existed, the method returned early. If cached data was expired, it was filtered out by max_age constraint. The fix retrieves cached data again in the exception handler with a very large max_age (1 year) to effectively bypass expiration check and allow fallback to expired data when fetch fails. * fix(plugin-templates): Resolve plugin_id mismatch in test template setUp method * feat(plugins): Standardize manifest version fields schema - Consolidate version fields to use consistent naming: - compatible_versions: array of semver ranges (required) - min_ledmatrix_version: string (optional) - max_ledmatrix_version: string (optional) - versions[].ledmatrix_min_version: renamed from ledmatrix_min - Add manifest schema validation (schema/manifest_schema.json) - Update store_manager to validate version fields and schema - Update template and all documentation examples to use standardized fields - Add deprecation warnings for ledmatrix_version and ledmatrix_min fields * fix(templates): Update plugin README template script path to correct location * docs(plugin): Resolve conflicting version management guidance in .cursorrules * chore(.gitignore): Consolidate plugin exclusion patterns Remove unnecessary !plugins/*/.git pattern and consolidate duplicate negations by keeping only trailing-slash directory exclusions. * docs: Add language specifiers to code blocks in STATIC_IMAGE_MULTI_UPLOAD_PLAN.md * fix(templates): Remove api_key from config.json example in plugin README template Remove api_key field from config.json example to prevent credential leakage. API keys should only be stored in config_secrets.json. Added clarifying note about proper credential storage. * docs(README): Add plugin installation and migration information - Add plugin installation instructions via web interface and GitHub URL - Add plugin migration guide for users upgrading from old managers - Improve plugin documentation for new users * docs(readme): Update donation links and add Discord acknowledgment * docs: Add comprehensive API references and consolidate documentation - Add API_REFERENCE.md with complete REST API documentation (50+ endpoints) - Add PLUGIN_API_REFERENCE.md documenting Display Manager, Cache Manager, and Plugin Manager APIs - Add ADVANCED_PLUGIN_DEVELOPMENT.md with advanced patterns and examples - Add DEVELOPER_QUICK_REFERENCE.md for quick developer reference - Consolidate plugin configuration docs into single PLUGIN_CONFIGURATION_GUIDE.md - Archive completed implementation summaries to docs/archive/ - Enhance PLUGIN_DEVELOPMENT_GUIDE.md with API links and 3rd party submission guidelines - Update docs/README.md with new API reference sections - Update root README.md with documentation links * fix(install): Fix IP detection and network diagnostics after fresh install - Fix web-ui-info plugin IP detection to handle no internet, AP mode, and network state changes - Replace socket-based detection with robust interface scanning using hostname -I and ip addr - Add AP mode detection returning 192.168.4.1 when AP mode is active - Add periodic IP refresh every 30 seconds to handle network state changes - Improve network diagnostics in first_time_install.sh showing actual IPs, WiFi status, and AP mode - Add WiFi connection check in WiFi monitor installation with warnings - Enhance web service startup logging to show accessible IP addresses - Update README with network troubleshooting section and fix port references (5001->5000) Fixes issue where display showed incorrect IP (127.0.11:5000) and users couldn't access web UI after fresh install. * chore: Add GitHub sponsor button configuration * fix(wifi): Fix aggressive AP mode enabling and improve WiFi detection Critical fixes: - Change auto_enable_ap_mode default from True to False (manual enable only) - Fixes issue where Pi would disconnect from network after code updates - Matches documented behavior (was incorrectly defaulting to True in code) Improvements: - Add grace period: require 3 consecutive disconnected checks (90s) before enabling AP mode - Prevents AP mode from enabling on transient network hiccups - Improve WiFi status detection with retry logic and better nmcli parsing - Enhanced logging for debugging WiFi connection issues - Better handling of WiFi device detection (works with any wlan device) This prevents the WiFi monitor from aggressively enabling AP mode and disconnecting the Pi from the network when there are brief network issues or during system initialization. * fix(wifi): Revert auto_enable_ap_mode default to True with grace period protection Change default back to True for auto_enable_ap_mode while keeping the grace period protection that prevents interrupting valid WiFi connections. - Default auto_enable_ap_mode back to True (useful for setup scenarios) - Grace period (3 consecutive checks = 90s) prevents false positives - Improved WiFi detection with retry logic ensures accurate status - AP mode will auto-enable when truly disconnected, but won't interrupt valid connections due to transient detection issues * fix(news): Update submodule reference for manifest fix Update ledmatrix-news submodule to include the fixed manifest.json with required entry_point and class_name fields. * fix(news): Update submodule reference with validate_config addition Update ledmatrix-news submodule to include validate_config method for proper configuration validation. * feat: Add of-the-day plugin as git submodule - Add ledmatrix-of-the-day plugin as git submodule - Rename submodule path from plugins/of-the-day to plugins/ledmatrix-of-the-day to match repository naming convention - Update .gitignore to allow ledmatrix-of-the-day submodule - Plugin includes fixes for display rendering and web UI configuration support * fix(wifi): Make AP mode open network and fix WiFi page loading in AP mode AP Mode Changes: - Remove password requirement from AP mode (open network for easier setup) - Update hostapd config to create open network (no WPA/WPA2) - Update nmcli hotspot to create open network (no password parameter) WiFi Page Loading Fixes: - Download local copies of HTMX and Alpine.js libraries - Auto-detect AP mode (192.168.4.x) and use local JS files instead of CDN - Auto-open WiFi tab when accessing via AP mode IP - Add fallback loading if HTMX fails to load - Ensures WiFi setup page works in AP mode without internet access This fixes the issue where the WiFi page wouldn't load on iPhone when accessing via AP mode (192.168.4.1:5000) because CDN resources couldn't be fetched without internet connectivity. * feat(wifi): Add explicit network switching support with clean disconnection WiFi Manager Improvements: - Explicitly disconnect from current network before connecting to a new one - Add skip_ap_check parameter to disconnect_from_network() to prevent AP mode from activating during network switches - Check if already connected to target network to avoid unnecessary work - Improved logging for network switching operations Web UI Improvements: - Detect and display network switching status in UI - Show 'Switching from [old] to [new]...' message when switching networks - Enhanced status reloading after connection (multiple checks at 2s, 5s, 10s) - Better user feedback during network transitions This ensures clean network switching without AP mode interruptions and provides clear feedback to users when changing WiFi networks. * fix(web-ui): Add fallback content loading when HTMX fails to load Problem: - After recent updates, web UI showed navigation and CPU status but main content tabs never loaded - Content tabs depend on HTMX's 'revealed' trigger to load - If HTMX failed to load or initialize, content would never appear Solutions: - Enhanced HTMX loading verification with timeout checks - Added fallback direct fetch for overview tab if HTMX fails - Added automatic tab content loading when tabs change - Added loadTabContent() method to manually trigger content loading - Added global 'htmx-load-failed' event for error handling - Automatic retry after 5 seconds if HTMX isn't available - Better error messages and console logging for debugging This ensures the web UI loads content even if HTMX has issues, providing graceful degradation and better user experience. * feat(web-ui): Add support for plugin custom HTML widgets and static file serving - Add x-widget: custom-html support in config schema generation - Add loadCustomHtmlWidget() function to load HTML from plugin directories - Add /api/v3/plugins/<plugin_id>/static/<file_path> endpoint for serving plugin static files - Enhance execute_plugin_action() to pass params via stdin as JSON for scripts - Add JSON output parsing for script action responses These changes enable plugins to provide custom UI components while keeping all functionality plugin-scoped. Used by of-the-day plugin for file management. * fix(web-ui): Resolve Alpine.js initialization errors - Prevent Alpine.js from auto-initializing before app() function is defined - Add deferLoadingAlpine to ensure proper initialization order - Make app() function globally available via window.app - Fix 'app is not defined' and 'activeTab is not defined' errors - Remove duplicate Alpine.start() calls that caused double initialization warnings * fix(web-ui): Fix IndentationError in api_v3.py OAuth flow - Fix indentation in if action_def.get('oauth_flow') block - Properly indent try/except block and all nested code - Resolves IndentationError that prevented web interface from starting * fix(web-ui): Fix SyntaxError in api_v3.py else block - Fix indentation of OAuth flow code inside else block - Properly indent else block for simple script execution - Resolves SyntaxError at line 3458 that prevented web interface from starting * fix(web-ui): Restructure OAuth flow check to fix SyntaxError - Move OAuth flow check before script execution in else block - Remove unreachable code that was causing syntax error - OAuth check now happens first, then falls back to script execution - Resolves SyntaxError at line 3458 * fix(web-ui): Define app() function in head for Alpine.js initialization - Define minimal app() function in head before Alpine.js loads - Ensures app() is available when Alpine initializes - Full implementation in body enhances/replaces the stub - Fixes 'app is not defined' and 'activeTab is not defined' errors * fix(web-ui): Ensure plugin tabs load when full app() implementation is available - Update stub init() to detect and use full implementation when available - Ensure full implementation properly replaces stub methods - Call init() after merging to load plugins and set up watchers - Fixes issue where installed plugins weren't showing in navigation bar * fix(web-ui): Prevent 'Cannot redefine property' error for installedPlugins - Check if window.installedPlugins property already exists before defining - Make property configurable to allow redefinition if needed - Add _initialized flag to prevent multiple init() calls - Fixes TypeError when stub tries to enhance with full implementation * fix(web-ui): Fix variable redeclaration errors in logs tab - Replace let/const declarations with window properties to avoid redeclaration - Use window._logsEventSource, window._allLogs, etc. to persist across HTMX reloads - Clean up existing event source before reinitializing - Remove and re-add event listeners to prevent duplicates - Fixes 'Identifier has already been declared' error when accessing logs tab multiple times * feat(web-ui): Add support for additionalProperties object rendering - Add handler for objects with additionalProperties containing object schemas - Render dynamic category controls with enable/disable toggles - Display category metadata (display name, data file path) - Used by of-the-day plugin for category management * fix(wifi): Ensure AP mode hotspot is always open (no password) Problem: - LEDMatrix-Setup WiFi AP was still asking for password despite code changes - Existing hotspot connections with passwords weren't being fully cleaned up - NetworkManager might reuse old connection profiles with passwords Solutions: - More thorough cleanup: Delete all hotspot-related connections, not just known names - Verification: Check if hotspot has password after creation - Automatic fix: Remove password and restart connection if security is detected - Better logging: Log when password is detected and removed This ensures the AP mode hotspot is always open for easy setup access, even if there were previously saved connections with passwords. * fix(wifi): Improve network switching reliability and device state handling Problem: - Pi failing to switch WiFi networks via web UI - Connection attempts happening before device is ready - Disconnect not fully completing before new connection attempt - Connection name lookup issues when SSID doesn't match connection name Solutions: - Improved disconnect logic: Disconnect specific connection first, then device - Device state verification: Wait for device to be ready (disconnected/unavailable) before connecting - Better connection lookup: Search by SSID, not just connection name - Increased wait times: 2 seconds for disconnect to complete - State checking before activating existing connections - Enhanced error handling and logging throughout This ensures network switching works reliably by properly managing device state transitions and using correct connection identifiers. * debug(web-ui): Add debug logging for custom HTML widget loading - Add console logging to track widget generation - Improve error messages with missing configuration details - Help diagnose why file manager widget may not be appearing * fix(web-ui): Fix [object Object] display in categories field - Add type checking to ensure category values are strings before rendering - Safely extract data_file and display_name properties - Prevent object coercion issues in category display * perf(web-ui): Optimize plugin loading in navigation bar - Reduce stub init timeout from 100ms to 10ms for faster enhancement - Change full implementation merge from 50ms setTimeout to requestAnimationFrame - Add direct plugin loading in stub while waiting for full implementation - Skip plugin reload in full implementation if already loaded by stub - Significantly improves plugin tab loading speed in navigation bar * feat(web-ui): Adapt file-upload widget for JSON files in of-the-day plugin - Add specialized JSON upload/delete endpoints for of-the-day plugin - Modify file-upload widget to support JSON files (file_type: json) - Render JSON files with file-code icon instead of image preview - Show entry count for JSON files - Store files in plugins/ledmatrix-of-the-day/of_the_day/ directory - Automatically update categories config when files are uploaded/deleted - Populate uploaded_files array from categories on form load - Remove custom HTML widget, use standard file-upload widget instead * fix(web-ui): Add working updatePluginTabs to stub for immediate plugin tab rendering - Stub's updatePluginTabs was empty, preventing tabs from showing - Add basic implementation that creates plugin tabs in navigation bar - Ensures plugin tabs appear immediately when plugins load, even before full implementation merges - Fixes issue where plugin navigation bar wasn't working * feat(api): Populate uploaded_files and categories from disk for of-the-day plugin - Scan of_the_day directory for existing JSON files when loading config - Populate uploaded_files array from files on disk - Populate categories from files on disk if not in config - Categories default to disabled, user can enable them - Ensures existing JSON files (word_of_the_day.json, slovenian_word_of_the_day.json) appear in UI * fix(api): Improve category merging logic for of-the-day plugin - Preserve existing category enabled state when merging with files from disk - Ensure all JSON files from disk appear in categories section - Categories from files default to disabled, preserving user choices - Properly merge existing config with scanned files * fix(wifi): More aggressive password removal for AP mode hotspot Problem: - LEDMatrix-Setup network still asking for password despite previous fixes - NetworkManager may add default security settings to hotspots - Existing connections with passwords may not be fully cleaned up Solutions: - Always remove ALL security settings after creating hotspot (not just when detected) - Remove multiple security settings: key-mgmt, psk, wep-key, auth-alg - Verify security was removed and recreate connection if verification fails - Improved cleanup: Delete connections by SSID match, not just by name - Disconnect connections before deleting them - Always restart connection after removing security to apply changes - Better logging for debugging This ensures the AP mode hotspot is always open, even if NetworkManager tries to add default security settings. * perf(web): Optimize web interface performance and fix JavaScript errors - Add resource hints (preconnect, dns-prefetch) for CDN resources to reduce DNS lookup delays - Fix duplicate response parsing bug in loadPluginConfig that was parsing JSON twice - Replace direct fetch() calls with PluginAPI.getInstalledPlugins() to leverage caching and throttling - Fix Alpine.js function availability issues with defensive checks and $nextTick - Enhance request deduplication with debug logging and statistics - Add response caching headers for static assets and API responses - Add performance monitoring utilities with detailed metrics Fixes console errors for loadPluginConfig and generateConfigForm not being defined. Reduces duplicate API calls to /api/v3/plugins/installed endpoint. Improves initial page load time with resource hints and optimized JavaScript loading. * perf(web-ui): optimize CSS for Raspberry Pi performance - Remove backdrop-filter blur from modal-backdrop - Remove box-shadow transitions (use transform/opacity only) - Remove button ::before pseudo-element animation - Simplify skeleton loader (gradient to opacity pulse) - Optimize transition utility (specific properties, not 'all') - Improve color contrast for WCAG AA compliance - Add CSS containment to cards, plugin-cards, modals - Remove unused CSS classes (duration-300, divider, divider-light) - Remove duplicate spacing utility classes All animations now GPU-accelerated (transform/opacity only). Optimized for low-powered Raspberry Pi devices. * fix(web): Resolve ReferenceError for getInstalledPluginsSafe in v3 stub initialization Move getInstalledPluginsSafe() function definition before the app() stub code that uses it. The function was previously defined at line 3756 but was being called at line 849 during Alpine.js initialization, causing a ReferenceError when loadInstalledPluginsDirectly() attempted to load plugins before the full implementation was ready. * fix(web): Resolve TypeError for installedPlugins.map in plugin loading Fix PluginAPI.getInstalledPlugins() to properly extract plugins array from API response structure. The API returns {status: 'success', data: {plugins: [...]}}, but the method was returning response.data (the object) instead of response.data.plugins (the array). Changes: - api_client.js: Extract plugins array from response.data.plugins - plugins_manager.js: Add defensive array checks and handle array return value correctly - base.html: Add defensive check in getInstalledPluginsSafe() to ensure plugins is always an array This prevents 'installedPlugins.map is not a function' errors when loading plugins. * style(web-ui): Enhance navigation bar styling for better readability - Improve contrast: Change inactive tab text from gray-500 to gray-700 - Add gradient background and thicker border for active tabs - Enhance hover states with background highlights - Add smooth transitions using GPU-accelerated properties - Update all navigation buttons (system tabs and plugin tabs) - Add updatePluginTabStates() method for dynamic tab state management All changes are CSS-only with zero performance overhead. * fix(web-ui): Optimize plugin loading and reduce initialization errors - Make generateConfigForm accessible to inline Alpine components via parent scope - Consolidate plugin initialization to prevent duplicate API calls - Fix script execution from HTMX-loaded content by extracting scripts before DOM insertion - Add request deduplication to loadInstalledPlugins() to prevent concurrent requests - Improve Alpine component initialization with proper guards and fallbacks This eliminates 'generateConfigForm is not defined' errors and reduces plugin API calls from 3-4 duplicate calls to 1 per page load, significantly improving page load performance. * fix(web-ui): Add guard check for generateConfigForm to prevent Alpine errors Add typeof check in x-show to prevent Alpine from evaluating generateConfigForm before the component methods are fully initialized. This eliminates the 'generateConfigForm is not defined' error that was occurring during component initialization. * fix(web-ui): Fix try-catch block structure in script execution code Correct the nesting of try-catch block inside the if statement for script execution. The catch block was incorrectly placed after the else clause, causing a syntax error. * fix(web-ui): Escape quotes in querySelector to avoid HTML attribute conflicts Change double quotes to single quotes in the CSS selector to prevent conflicts with HTML attribute parsing when the x-data expression is embedded. * style(web): Improve button text readability in Quick Actions section * fix(web): Resolve Alpine.js expression errors in plugin configuration component - Capture plugin from parent scope into component data to fix parsing errors - Update all plugin references to use this.plugin in component methods - Fix x-init to properly call loadPluginConfig method - Resolves 'Uncaught ReferenceError' for isOnDemandLoading, onDemandLastUpdated, and other component properties * fix(web): Fix remaining Alpine.js scope issues in plugin configuration - Use this.generateConfigForm in typeof checks and method calls - Fix form submission to use this.plugin.id - Use $root. prefix for parent scope function calls (refreshPlugin, updatePlugin, etc.) - Fix confirm dialog string interpolation - Ensures all component methods and properties are properly scoped * fix(web): Add this. prefix to all Alpine.js component property references - Fix all template expressions to use this. prefix for component properties - Update isOnDemandLoading, onDemandLastUpdated, onDemandRefreshing references - Update onDemandStatusClass, onDemandStatusText, onDemandServiceClass, onDemandServiceText - Update disableRunButton, canStopOnDemand, showEnableHint, loading references - Ensures Alpine.js can properly resolve all component getters and properties * fix(web): Resolve Alpine.js expression errors in plugin configuration - Move complex x-data object to pluginConfigData() function for better parsing - Fix all template expressions to use this.plugin instead of plugin - Add this. prefix to all method calls in event handlers - Fix duplicate x-on:click attribute on uninstall button - Add proper loading state management in loadPluginConfig method This resolves the 'Invalid or unexpected token' and 'Uncaught ReferenceError' errors in the browser console. * fix(web): Fix plugin undefined errors in Alpine.js plugin configuration - Change x-data initialization to capture plugin from loop scope first - Use Object.assign in x-init to merge pluginConfigData properties - Add safety check in pluginConfigData function for undefined plugins - Ensure plugin is available before accessing properties in expressions This resolves the 'Cannot read properties of undefined' errors by ensuring the plugin object is properly captured from the x-for loop scope before any template expressions try to access it. * style(web): Make Quick Actions button text styling consistent - Update Start Display, Stop Display, and Reboot System buttons - Change from text-sm font-medium to text-base font-semibold - All Quick Actions buttons now have consistent bold, larger text - Matches the styling of Update Code, Restart Display Service, and Restart Web Service buttons * fix(wifi): Properly handle AP mode disable during WiFi connection - Check return value of disable_ap_mode() before proceeding with connection - Add verification loop to ensure AP mode is actually disabled - Increase wait time to 5 seconds for NetworkManager restart stabilization - Return clear error messages if AP mode cannot be disabled - Prevents connection failures when switching networks from web UI or AP mode This fixes the issue where WiFi network switching would fail silently when AP mode disable failed, leaving the system in an inconsistent state. * fix(web): Handle API response errors in plugin configuration loading - Add null/undefined checks before accessing API response status - Set fallback defaults when API responses don't have status 'success' - Add error handling for batch API requests with fallback to individual requests - Add .catch() handlers to individual fetch calls to prevent unhandled rejections - Add console warnings to help debug API response failures - Fix applies to both main loadPluginConfig and PluginConfigHelpers.loadPluginConfig This fixes the issue where plugin configuration sections would get stuck showing the loading animation when API responses failed or returned error status. * fix(web): Fix Alpine.js reactivity for plugin config by using direct x-data Changed from Object.assign pattern to direct x-data assignment to ensure Alpine.js properly tracks reactive properties. The previous approach used Object.assign to merge properties into the component after initialization, which caused Alpine to not detect changes to config/schema properties. The fix uses pluginConfigData(plugin) directly as x-data, ensuring all properties including config, schema, loading, etc. are reactive from component initialization. * fix(web): Ensure plugin variable is captured in x-data scope Use spread operator to merge pluginConfigData properties while explicitly capturing the plugin variable from outer x-for scope. This fixes undefined plugin errors when Alpine evaluates the component data. * fix(web): Use $data for Alpine.js reactivity when merging plugin config Use Object.assign with Alpine's $data reactive proxy instead of this to ensure added properties are properly reactive. This fixes the issue where plugin variable scoping from x-for wasn't accessible in x-data expressions. * fix(web): Remove incorrect 'this.' prefix in Alpine.js template expressions Alpine.js template expressions (x-show, x-html, x-text, x-on) use the component data as the implicit context, so 'this.' prefix is incorrect. In template expressions, 'this' refers to the DOM element, not the component data. Changes: - Replace 'this.plugin.' with 'plugin.' in all template expressions (19 instances) - Replace 'this.loading' with 'loading' in x-show directives - Replace 'this.generateConfigForm' with 'generateConfigForm' in x-show/x-html - Replace 'this.savePluginConfig' with 'savePluginConfig' in x-on:submit - Replace 'this.config/schema/webUiActions' with direct property access - Use '$data.loadPluginConfig' in x-init for explicit method call Note: 'this.' is still correct inside JavaScript method definitions within pluginConfigData() function since those run with proper object context. * fix(web): Prevent infinite recursion in plugin config methods Add 'parent !== this' check to loadPluginConfig, generateConfigForm, and savePluginConfig methods in pluginConfigData to prevent infinite recursion when the component tries to delegate to a parent that resolves to itself. This fixes the 'Maximum call stack size exceeded' error that occurred when the nested Alpine component's $root reference resolved to a component that had the same delegating methods via Object.assign. * fix(web): Resolve infinite recursion in plugin config by calling $root directly The previous implementation had delegating methods (generateConfigForm, savePluginConfig) in pluginConfigData that tried to call parent.method(), but the parent detection via getParentApp() was causing circular calls because multiple components had the same methods. Changes: - Template now calls $root.generateConfigForm() and $root.savePluginConfig() directly instead of going through nested component delegation - Removed delegating generateConfigForm and savePluginConfig from pluginConfigData - Removed getParentApp() helper that was enabling the circular calls - Simplified loadPluginConfig to use PluginConfigHelpers directly This fixes the 'Maximum call stack size exceeded' error when rendering plugin configuration forms. * fix(web): Use window.PluginConfigHelpers instead of $root for plugin config The $root magic variable in Alpine.js doesn't correctly reference the app() component's data scope from nested x-data contexts. This causes generateConfigForm and savePluginConfig to be undefined. Changed to use window.PluginConfigHelpers which has explicit logic to find and use the app component's methods. * fix(web): Use direct x-data initialization for plugin config reactivity Changed from Object.assign($data, pluginConfigData(plugin)) to x-data="pluginConfigData(plugin)" to ensure Alpine.js properly tracks reactivity for all plugin config properties. This fixes the issue where all plugin tabs were showing the same config. * refactor(web): Implement server-side plugin config rendering with HTMX Major architectural improvement to plugin configuration management: - Add server-side Jinja2 template for plugin config forms (web_interface/templates/v3/partials/plugin_config.html) - Add Flask route to serve plugin config partials on-demand - Replace complex client-side form generation with HTMX lazy loading - Add Alpine.js store for centralized plugin state management - Mark old pluginConfigData and PluginConfigHelpers as deprecated Benefits: - Lazy loading: configs only load when tab is accessed - Server-side rendering: reduces client-side complexity - Better performance: especially on Raspberry Pi - Cleaner code: Jinja2 macros replace JS string templates - More maintainable: form logic in one place (server) The old client-side code is preserved for backwards compatibility but is no longer used by the main plugin configuration UI. * fix(web): Trigger HTMX manually after Alpine renders plugin tabs HTMX processes attributes at page load time, before Alpine.js renders dynamic content. Changed from :hx-get attribute to x-init with htmx.ajax() to properly trigger the request after the element is rendered. * fix(web): Remove duplicate 'enabled' toggle from plugin config form The 'enabled' field was appearing twice in plugin configuration: 1. Header toggle (quick action, uses HTMX) 2. Configuration form (from schema, requires save) Now only the header toggle is shown, avoiding user confusion. The 'enabled' key is explicitly skipped when rendering schema properties. * perf(web): Optimize plugin manager with request caching and init guards Major performance improvements to plugins_manager.js: 1. Request Deduplication & Caching - Added pluginLoadCache with 3-second TTL - Subsequent calls return cached data instead of making API requests - In-flight request deduplication prevents parallel duplicate fetches - Added refreshInstalledPlugins() for explicit force-refresh 2. Initialization Guards - Added pluginsInitialized flag to prevent multiple initializePlugins() calls - Added _eventDelegationSetup guard on container to prevent duplicate listeners - Added _listenerSetup guards on search/category inputs 3. Debug Logging Control - Added PLUGIN_DEBUG flag (localStorage.setItem('pluginDebug', 'true')) - Most console.log calls now use pluginLog() which only logs when debug enabled - Reduces console noise from ~150 logs to ~10 in production Expected improvements: - API calls reduced from 6+ to 2 on page load - Event listeners no longer duplicated - Cleaner console output - Faster perceived performance * fix(web): Handle missing search elements in searchPluginStore The searchPluginStore function was failing silently when called before the plugin-search and plugin-category elements existed in the DOM. This caused the plugin store to never load. Now safely checks if elements exist before accessing their values. * fix(web): Ensure plugin store loads via pluginManager.searchPluginStore - Exposed searchPluginStore on window.pluginManager for easier access - Updated base.html to fallback to pluginManager.searchPluginStore - Added logging when loading plugin store * fix(web): Expose searchPluginStore from inside the IIFE The function was defined inside the IIFE but only exposed after the IIFE ended, where the function was out of scope. Now exposed immediately after definition inside the IIFE. * fix(web): Add cache-busting version to plugins_manager.js URL Static JS files were being aggressively cached, preventing updates from being loaded by browsers. * fix(web): Fix pluginLog reference error outside IIFE pluginLog is defined inside the IIFE, so use _PLUGIN_DEBUG_EARLY and console.log directly for code outside the IIFE. * chore(web): Update plugins_manager.js cache version * fix(web): Defer plugin store render when grid not ready Instead of showing an error when plugin-store-grid doesn't exist, store plugins in window.__pendingStorePlugins for later rendering when the tab loads (consistent with how installed plugins work). * chore: Bump JS cache version * fix(web): Restore enabledBool variable in plugin render Variable was removed during debug logging optimization but was still being used in the template string for toggle switch rendering. * fix(ui): Add header and improve categories section rendering - Add proper header (h4) to categories section with label - Add debug logging to diagnose categories field rendering - Improve additionalProperties condition check readability * fix(ui): Improve additionalProperties condition check - Explicitly exclude objects with properties to avoid conflicts - Ensure categories section is properly detected and rendered - Categories should show as header with toggles, not text box * fix(web-ui): Fix JSON parsing errors and default value loading for plugin configs - Fix JSON parsing errors when saving file upload fields by properly unescaping HTML entities - Merge config with schema defaults when loading plugin config so form shows default values - Improve default value handling in form generation for nested objects and arrays - Add better error handling for malformed JSON in file upload fields * fix(plugins): Return plugins array from getInstalledPlugins() instead of data object Fixed PluginAPI.getInstalledPlugins() to return response.data.plugins (array) instead of response.data (object). This was preventing window.installedPlugins from being set correctly, which caused plugin configuration tabs to not appear and prevented users from saving plugin configurations via the web UI. The fix ensures that: - window.installedPlugins is properly populated with plugin array - Plugin tabs are created automatically on page load - Configuration forms and save buttons are rendered correctly - Save functionality works as expected * fix(api): Support form data submission for plugin config saves The HTMX form submissions use application/x-www-form-urlencoded format instead of JSON. This update allows the /api/v3/plugins/config POST endpoint to accept both formats: - JSON: plugin_id and config in request body (existing behavior) - Form data: plugin_id from query string, config fields from form Added _parse_form_value helper to properly convert form strings to appropriate Python types (bool, int, float, JSON arrays/objects). * debug: Add form data logging to diagnose config save issue * fix(web): Re-discover plugins before loading config partial The plugin config partial was returning 'not found' for plugins because the plugin manifests weren't loaded. The installed plugins API was working because it calls discover_plugins() first. Changes: - Add discover_plugins() call in _load_plugin_config_partial when plugin info is not found on first try - Remove debug logging from form data handling * fix(web): Comprehensive plugin config save improvements SWEEPING FIX for plugin configuration saving issues: 1. Form data now MERGES with existing config instead of replacing - Partial form submissions (missing fields) no longer wipe out existing config values - Fixes plugins with complex schemas (football, clock, etc.) 2. Improved nested value handling with _set_nested_value helper - Correctly handles deeply nested structures like customization - Properly merges when intermediate objects already exist 3. Better JSON parsing for arrays - RGB color arrays like [255, 0, 0] now parse correctly - Parse JSON before trying number conversion 4. Bump cache version to force JS reload * fix(web): Add early stubs for updatePlugin and uninstallPlugin Ensures these functions are available immediately when the page loads, even before the full IIFE executes. Provides immediate user feedback and makes API calls directly. This fixes the 'Update button does not work' issue by ensuring the function is always defined and callable. * fix(web): Support form data in toggle endpoint The toggle endpoint now accepts both JSON and HTMX form submissions. Also updated the plugin config template to send the enabled state via hx-vals when the checkbox changes. Fixes: 415 Unsupported Media Type error when toggling plugins * fix(web): Prevent config duplication when toggling plugins Changed handleToggleResponse to update UI in place instead of refreshing the entire config partial, which was causing duplication. Also improved refreshPluginConfig with proper container targeting and concurrent refresh prevention (though it's no longer needed for toggles since we update in place). * fix(api): Schema-aware form value parsing for plugin configs Major fix for plugin config saving issues: 1. Load schema BEFORE processing form data to enable type-aware parsing 2. New _parse_form_value_with_schema() function that: - Converts comma-separated strings to arrays when schema says 'array' - Parses JSON strings for arrays/objects - Handles empty strings for arrays (returns [] instead of None) - Uses schema to determine correct number types 3. Post-processing to ensure None arrays get converted to empty arrays 4. Proper handling of nested object fields Fixes validation errors: - 'category_order': Expected type array, got str - 'categories': Expected type object, got str - 'uploaded_files': Expected type array, got NoneType - RGB color arrays: Expected type array, got str * fix(web): Make plugin config handlers idempotent and remove scripts from HTMX partials CRITICAL FIX for script redeclaration errors: 1. Removed all <script> tags from plugin_config.html partial - Scripts were being re-executed on every HTMX swap - Caused 'Identifier already declared' errors 2. Moved all handler functions to base.html with idempotent initialization - Added window.__pluginConfigHandlersInitialized guard - Functions only initialized once, even if script runs multiple times - All state stored on window object (e.g., window.pluginConfigRefreshInProgress) 3. Enhanced error logging: - Client-side: Logs form payload, response status, and parsed error details - Server-side: Logs raw form data and parsed config on validation failures 4. Functions moved to window scope: - toggleSection - handleConfigSave (with detailed error logging) - handleToggleResponse (updates UI in place, no refresh) - handlePluginUpdate - refreshPluginConfig (with duplicate prevention) - runPluginOnDemand - stopOnDemand - executePluginAction This ensures HTMX-swapped fragments only contain HTML, and all scripts run once in the base layout. * fix(api): Filter config to only schema-defined fields before validation When merging with existing_config, fields not in the plugin's schema (like high_performance_transitions, transition, dynamic_duration) were being preserved, causing validation failures when additionalProperties is false. Add _filter_config_by_schema() function to recursively filter config to only include fields defined in the schema before validation. This fixes validation errors like: - 'Additional properties are not allowed (high_performance_transitions, transition were unexpected)' * fix(web): Improve update plugin error handling and support form data 1. Enhanced updatePlugin JavaScript function: - Validates pluginId before sending request - Checks response.ok before parsing JSON - Better error logging with request/response details - Handles both successful and error responses properly 2. Update endpoint now supports both JSON and form data: - Similar to config endpoint, accepts plugin_id from query string or form - Better error messages and debug logging 3. Prevent duplicate function definitions: - Second updatePlugin definition checks if improved version exists - Both definitions now have consistent error handling Fixes: 400 BAD REQUEST 'Request body must be valid JSON' error * fix(web): Show correct 'update' message instead of 'save' for plugin updates The handlePluginUpdate function now: 1. Checks actual HTTP status code (not just event.detail.successful) 2. Parses JSON response to get server's actual message 3. Replaces 'save' with 'update' if message incorrectly says 'save' Fixes: Update button showing 'saved successfully' instead of 'updated successfully' * fix(web): Execute plugin updates immediately instead of queuing Plugin updates are now executed directly (synchronously) instead of being queued for async processing. This provides immediate feedback to users about whether the update succeeded or failed. Updates are fast git pull operations, so they don't need async processing. The operation queue is reserved for longer operations like install/uninstall. Fixes: Update button not actually updating plugins (operations were queued but users didn't see results) * fix(web): Ensure toggleSection function is always available for collapsible headers Moved toggleSection outside the initialization guard block so it's always defined, even if the plugin config handlers have already been initialized. This ensures collapsible sections in plugin config forms work correctly. Added debug logging to help diagnose if sections/icons aren't found. Fixes: Collapsible headers in plugin config schema not collapsing * fix(web): Improve toggleSection to explicitly show/hide collapsible content Changed from classList.toggle() to explicit add/remove of 'hidden' class based on current state. This ensures the content visibility is properly controlled when collapsing/expanding sections. Added better error checking and state detection for more reliable collapsible section behavior. * fix(web): Load plugin tabs on page load instead of waiting for plugin manager tab click The stub's loadInstalledPlugins was an empty function, so plugin tabs weren't loading until the plugin manager tab was clicked. Now the stub implementation: 1. Tries to use global window.loadInstalledPlugins if available 2. Falls back to window.pluginManager.loadInstalledPlugins 3. Finally falls back to direct loading via loadInstalledPluginsDirectly 4. Always updates tabs after loading plugins This ensures plugin navigation tabs are available immediately on page load. Fixes: Plugin tabs only loading after clicking plugin manager tab * fix(web): Ensure plugin navigation tabs load on any page regardless of active tab Multiple improvements to ensure plugin tabs are always visible: 1. Stub's loadInstalledPluginsDirectly now waits for DOM to be ready before updating tabs, using requestAnimationFrame for proper timing 2. Stub's init() now has a retry mechanism that periodically checks if plugins have been loaded by plugins_manager.js and updates tabs accordingly (checks for 2 seconds) 3. Full implementation's init() now properly handles async plugin loading and ensures tabs are updated after loading completes, checking window.installedPlugins first before attempting to load 4. Both stub and full implementation ensure tabs update using $nextTick to wait for Alpine.js rendering cycle This ensures plugin navigation tabs are visible immediately when the page loads, regardless of whether the user is on overview, plugin manager, or any other tab. Fixes: Plugin tabs only appearing after clicking plugin manager tab * fix(web): Fix restart display button not working The initPluginsPage function was returning early before event listeners were set up, making all the event listener code unreachable. Moved the return statement to after all event listeners are attached. This fixes the restart display button and all other buttons in the plugin manager (refresh plugins, update all, search, etc.) that depend on event listeners being set up. Fixes: Restart Display button not working in plugin manager * fix(web-ui): Improve categories field rendering for of-the-day plugin - Add more explicit condition checking for additionalProperties objects - Add debug logging specifically for categories field - Add fallback handler for objects that don't match special cases (render as JSON textarea) - Ensure categories section displays correctly with toggle cards instead of plain text * fix(install): Prevent following broken symlinks during file ownership setup - Add -P flag to find commands to prevent following symlinks when traversing - Add -h flag to chown to operate on symlinks themselves rather than targets - Exclude scripts/dev/plugins directory which contains development symlinks - Fixes error when chown tries to dereference broken symlinks with extra LEDMatrix in path * fix(scroll): Ensure scroll completes fully before switching displays - Add display_width to total scroll distance calculation - Scroll now continues until content is completely off screen - Update scroll completion check to use total_scroll_width + display_width - Prevents scroll from being cut off mid-way when switching to next display * fix(install): Remove unsupported -P flag from find commands - Remove -P flag which is not supported on all find versions - Keep -h flag on chown to operate on symlinks themselves - Change to {} \; syntax for better error handling - Add error suppression to continue on broken symlinks - Exclude scripts/dev/plugins directory to prevent traversal into broken symlinks * docs(wifi): Add trailing newline to WiFi AP failover setup guide * fix(web): Suppress non-critical socket errors and fix WiFi permissions script - Add error filtering in web interface to suppress harmless client disconnection errors - Downgrade 'No route to host' and broken pipe errors from ERROR to DEBUG level - Fix WiFi permissions script to use mktemp instead of manual temp file creation - Add cleanup trap to ensure temp files are removed on script exit - Resolves permission denied errors when creating temp files during installation * fix(web): Ensure plugin navigation tabs load on any page by dispatching events The issue was that when plugins_manager.js loaded and called loadInstalledPlugins(), it would set window.installedPlugins but the Alpine.js component wouldn't know to update its tabs unless the plugin manager tab was clicked. Changes: 1. loadInstalledPlugins() now always dispatches a 'pluginsUpdated' event when it sets window.installedPlugins, not just when plugin IDs change 2. renderInstalledPlugins() also dispatches the event and always updates window.installedPlugins for reactivity 3. Cached plugin data also dispatches the event when returned The Alpine component already listens for the 'pluginsUpdated' event in its init() method, so tabs will now update immediately when plugins are loaded, regardless of which tab is active. Fixes: Plugin navigation tabs only loading after clicking plugin manager tab * fix(web): Improve input field contrast in plugin configuration forms Changed input backgrounds from bg-gray-800 to bg-gray-900 (darker) to ensure high contrast with white text. Added placeholder:text-gray-400 for better placeholder text visibility. Updated in both server-side template (plugin_config.html) and client-side form generation (plugins_manager.js): - Number inputs - Text inputs - Array inputs (comma-separated) - Select dropdowns - Textareas (JSON objects) - Fallback inputs without schema This ensures all form inputs have high contrast white text on dark background, making them clearly visible and readable. Fixes: White text on white background in plugin config inputs * fix(web): Change plugin config input text from white to black Changed all input fields in plugin configuration forms to use black text on white background instead of white text on dark background for better readability and standard form appearance. Updated: - Input backgrounds: bg-gray-900 -> bg-white - Text color: text-white -> text-black - Placeholder color: text-gray-400 -> text-gray-500 Applied to both server-side template and client-side form generation for all input types (number, text, select, textarea). * fix(web): Ensure toggleSection function is available for plugin config collapsible sections Moved toggleSection function definition to an early script block so it's available immediately when HTMX loads plugin configuration content. The function was previously defined later in the page which could cause it to not be accessible when inline onclick handlers try to call it. The function toggles the 'hidden' class on collapsible section content divs and rotates the chevron icon between right (collapsed) and down (expanded) states. Fixes: Plugin configuration section headers not collapsing/expanding * fix(web): Fix collapsible section toggle to properly hide/show content Updated toggleSection function to explicitly set display style in addition to toggling the hidden class. This ensures the content is properly hidden even if CSS specificity or other styles might interfere with just the hidden class. The function now: - Checks both the hidden class and computed display style - Explicitly sets display: '' when showing and display: 'none' when hiding - Rotates chevron icon between right (collapsed) and down (expanded) This ensures collapsible sections in plugin configuration forms properly hide and show their content when the header is clicked. Fixes: Collapsible section headers rotate chevron but don't hide content * fix(web): Fix collapsible section toggle to work on first click Simplified the toggle logic to rely primarily on the 'hidden' class check rather than mixing it with computed display styles. When hiding, we now remove any inline display style to let Tailwind's 'hidden' class properly control the display property. This ensures sections respond correctly on the first click, whether they're starting in a collapsed or expanded state. Fixes: Sections requiring 2 clicks to collapse * fix(web): Ensure collapsible sections start collapsed by default Added explicit display: none style to nested content divs in plugin config template to ensure they start collapsed. The hidden class should handle this, but adding the inline style ensures sections are definitely collapsed on initial page load. Sections now: - Start collapsed (hidden) with chevron pointing right - Expand when clicked (chevron points down) - Collapse when clicked again (chevron points right) This ensures a consistent collapsed initial state across all plugin configuration sections. * fix(web): Fix collapsible section toggle to properly collapse on second click Fixed the toggle logic to explicitly set display: block when showing and display: none when hiding, rather than clearing the display style. This ensures the section state is properly tracked and the toggle works correctly on both expand and collapse clicks. The function now: - When hidden: removes hidden class, sets display: block, chevron down - When visible: adds hidden class, sets display: none, chevron right This fixes the issue where sections would expand but not collapse again. Fixes: Sections not collapsing on second click * feat(web): Ensure plugin navigation tabs load automatically on any page Implemented comprehensive solution to ensure plugin navigation tabs load automatically without requiring a visit to the plugin manager page: 1. Global event listener for 'pluginsUpdated' - works even if Alpine isn't ready yet, updates tabs directly when plugins_manager.js loads plugins 2. Enhanced stub's loadInstalledPluginsDirectly(): - Sets window.installedPlugins after loading - Dispatches 'pluginsUpdated' event for global listener - Adds console logging for debugging 3. Event listener in stub's init() method: - Listens for 'pluginsUpdated' events - Updates component state and tabs when events fire 4. Fallback timer: - If plugins_manager.js hasn't loaded after 2 seconds, fetches plugins directly via API - Ensures tabs appear even if plugins_manager.js fails 5. Improved checkAndUpdateTabs(): - Better logging - Fallback to direct fetch after timeout 6. Enhanced logging throughout plugin loading flow for debugging This ensures plugin tabs are visible immediately on page load, regardless of which tab is active or when plugins_manager.js loads. Fixes: Plugin navigation tabs only loading after visiting plugin manager * fix(web): Improve plugin tabs update logging and ensure immediate execution Enhanced logging in updatePluginTabs() and _doUpdatePluginTabs() to help debug why tabs aren't appearing. Changed debounce behavior to execute immediately on first call to ensure tabs appear quickly. Added detailed console logging with [FULL] prefix to track: - When updatePluginTabs() is called - When _doUpdatePluginTabs() executes - DOM element availability - Tab creation process - Final tab count This will help identify if tabs are being created but not visible, or if the update function isn't being called at all. Fixes: Plugin tabs loading but not visible in navigation bar * fix(web): Prevent duplicate plugin tab updates and clearing Added debouncing and duplicate prevention to stub's updatePluginTabs() to prevent tabs from being cleared and re-added multiple times. Also checks if tabs already match before clearing them. Changes: 1. Debounce stub's updatePluginTabs() with 100ms delay 2. Check if existing tabs match current plugin list before clearing 3. Global event listener only triggers full implementation's updatePluginTabs 4. Stub's event listener only works in stub mode (before enhancement) This prevents the issue where tabs were being cleared and re-added multiple times in rapid succession, which could leave tabs empty. Fixes: Plugin tabs being cleared and not re-added properly * fix(web): Fix plugin tabs not rendering when plugins are loaded Fixed _doUpdatePluginTabs() to properly use component's installedPlugins instead of checking window.installedPlugins first. Also fixed the 'unchanged' check to not skip when both lists are empty (first load scenario). Changes: 1. Check component's installedPlugins first (most up-to-date) 2. Only skip update if plugins exist AND match (don't skip empty lists) 3. Retry if no plugins found (in case they're still loading) 4. Ensure window.installedPlugins is set when loading directly 5. Better logging to show which plugin source is being used This ensures tabs are rendered when plugins are loaded, even on first page load. Fixes: Plugin tabs not being drawn despite plugins being loaded * fix(config): Fix array field parsing and validation for plugin config forms - Added logic to detect and combine indexed array fields (text_color.0, text_color.1, etc.) - Fixed array fields incorrectly stored as dicts with numeric keys - Improved handling of comma-separated array values from form submissions - Ensures array fields meet minItems requirements before validation - Resolves 400 BAD REQUEST errors when saving plugin config with RGB color arrays * fix(config): Improve array field handling and secrets error handling - Use schema defaults when array fields don't meet minItems requirement - Add debug logging for array field parsing - Improve error handling for secrets file writes - Fix arrays stored as dicts with numeric keys conversion - Better handling of incomplete array values from form submissions * fix(config): Convert array elements to correct types (numbers not strings) - Fix array element type conversion when converting dicts to arrays - Ensure RGB color arrays have integer elements, not strings - Apply type conversion for both nested and top-level array fields - Fixes validation errors: 'Expected type number, got str' * fix(config): Fix array fields showing 'none' when value is null - Handle None/null values in array field templates properly - Use schema defaults when array values are None/null - Fix applies to both Jinja2 template and JavaScript form generation - Resolves issue where stock ticker plugin shows 'none' instead of default values * fix(config): Add novalidate to plugin config form to prevent HTML5 validation blocking saves - Prevents browser HTML5 validation from blocking form submission - Allows custom validation logic to handle form data properly - Fixes issue where save button appears unclickable due to invalid form controls - Resolves problems with plugins like clock-simple that have nested/array fields * feat(config): Add helpful form validation with detailed error messages - Keep HTML5 validation enabled (removed novalidate) to prevent broken configs - Add validatePluginConfigForm function that shows which fields fail and why - Automatically expands collapsed sections containing invalid fields - Focuses first invalid field and scrolls to it - Shows user-friendly error messages with field names and specific issues - Prevents form submission until all fields are valid * fix(schema): Remove core properties from required array during validation - Core properties (enabled, display_duration, live_priority) are system-managed - SchemaManager now removes them from required array after injection - Added default values for core properties (enabled=True, display_duration=15, live_priority=False) - Updated generate_default_config() to ensure live_priority has default - Resolves 186 validation issues, reducing to 3 non-blocking warnings (98.4% reduction) - All 19 of 20 plugins now pass validation without errors Documentation: - Created docs/PLUGIN_CONFIG_CORE_PROPERTIES.md explaining core property handling - Updated existing docs to reflect core property behavior - Removed temporary audit files and scripts * fix(ui): Improve button text contrast on white backgrounds - Changed Screenshot button text from text-gray-700 to text-gray-900 - Added global CSS rule to ensure all buttons with white backgrounds use dark text (text-gray-900) for better readability - Fixes contrast issues where light text on light backgrounds was illegible * fix(ui): Add explicit text color to form-control inputs - Added color: #111827 to .form-control class to ensure dark text on white backgrounds - Fixes issue where input fields had white text on white background after button contrast fix - Ensures all form inputs are readable with proper contrast * docs: Update impact explanation and plugin config documentation * docs: Improve documentation and fix template inconsistencies - Add migration guide for script path reorganization (scripts moved to scripts/install/ and scripts/fix_perms/) - Add breaking changes section to README with migration guidance - Fix config template: set plugins_directory to 'plugins' to match actual plugin locations - Fix test template: replace Jinja2 placeholders with plain text to match other templates - Fix markdown linting: add language identifiers to code blocks (python, text, javascript) - Update permission guide: document setgid bit (0o2775) for directory modes - Fix example JSON: pin dependency versions and fix compatible_versions range - Improve readability: reduce repetition in IMPACT_EXPLANATION.md * feat(web): Make v3 interface production-ready for local deployment - Phase 2: Real Service Integration - Replace sample data with real psutil system monitoring (CPU, memory, disk, temp, uptime) - Integrate display controller to read from /tmp/led_matrix_preview.png snapshot - Scan assets/fonts directory and extract font metadata with freetype - Phase 1: Security & Input Validation - Add input validation module with URL, file upload, and config sanitization - Add optional CSRF protection (gracefully degrades if flask-wtf missing) - Add rate limiting (lenient for local use, prevents accidental abuse) - Add file upload validation to font upload endpoint - Phase 3: Error Handling - Add global error handlers for 404, 500, and unhandled exceptions - All endpoints have comprehensive try/except blocks - Phase 4: Monitoring & Observability - Add structured logging with JSON format support - Add request logging middleware (tracks method, path, status, duration, IP) - Add /api/v3/health endpoint with service status checks - Phase 5: Performance & Caching - Add in-memory caching system (separate module to avoid circular imports) - Cache font catalog (5 minute TTL) - Cache system status (10 second TTL) - Invalidate cache on config changes - All changes are non-blocking with graceful error handling - Optional dependencies (flask-wtf, flask-limiter) degrade gracefully - All imports protected with try/except blocks - Verified compilation and import tests pass * docs: Fix caching pattern logic flaw and merge conflict resolution plan - Fix Basic Caching Pattern: Replace broken stale cache fallback with correct pattern - Re-fetch cache with large max_age (31536000) in except block instead of checking already-falsy cached variable - Fixes both instances in ADVANCED_PLUGIN_DEVELOPMENT.md - Matches correct pattern from manager.py.template - Fix MERGE_CONFLICT_RESOLUTION_PLAN.md merge direction - Correct Step 1 to checkout main and merge plugins into it (not vice versa) - Update commit message to reflect 'Merge plugins into main' direction - Fixes workflow to match documented plugins → main merge --------- Co-authored-by: Chuck <chuck@example.com>	2025-12-27 14:15:49 -05:00