mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 14:25:08 +00:00
fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595)
* fix(sports): share the ESPN rejected-range memo with the background service BackgroundDataService always sent a season range first and, on a 400, fell back to chunks without recording the rejection, so every background season fetch spent a doomed request and live scoreboards learned nothing from it (or it from them). The worker now consults and sets the same 6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight to month/day chunks, and if every chunk fails the range is asked once for a real error without re-spending the chunks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): keep plugin asset and action routes inside their directories POST /plugins/assets/upload, GET /plugins/assets/list and POST /plugins/assets/delete joined the request's plugin_id onto assets/plugins unchecked, so '../../config' created, wrote, listed and deleted outside it. #561 guarded only the route that serves the files. All three now go through path_safety.resolve_under and answer 400 for anything but a plain name, and delete only unlinks a metadata path that resolves into that plugin's uploads directory. PluginManager.get_plugin_directory refuses ids that are not one plain path segment, so /plugins/action (which runs a manifest script from the returned directory) and every other caller get the guard; the action route also rejects such ids up front, covering its no-manager fallback. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): report a no-op plugin update as already up to date update_plugin() returns True both for a real update and for "nothing to do" (a ZIP-installed monorepo plugin already at the registry version, a bundled plugin). With no git commit to compare, POST /plugins/update called every such success "updated successfully", so Check & Update All counted most official plugins as updated on every run. The route now reads what changed off the plugin itself (commit, else manifest version, else last_updated) and returns data.update_status (updated / up_to_date / local_only). The update-all toast is summarised by PluginInstallManager.summarizeUpdateResults from that status, falling back to the message for older servers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(sports): scoreboard scroll speed no longer follows target_fps sports_scroll computed the crisp speed ladder against the global target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since frame-locked presentation (#545) the helper steps a fixed number of whole pixels per presented frame and the panel presents at its real refresh, so the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a 50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s. The ladder now uses the display manager's refresh_hz, then display.hardware.limit_refresh_rate_hz, then the default. target_fps is not consulted. Docstrings now say scroll_delay is ignored for pacing (no behaviour change there) and describe the fixed-step model. Tests: replace the tests that pinned target_fps as the ladder refresh and described time-based stepping; assert speed independence from target_fps (unit and end-to-end presented px/s against the real helper), that the fixed per-frame step is applied, and that scroll_delay does not change speed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): escape registry and upload values in plugin manager inline handlers The store, saved-repository and custom-registry buttons built onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone, so a custom registry entry whose id contained ' closed the attribute and added its own handler. One helper, jsStringAttr(), now HTML-escapes the JSON literal for every one of those handlers, and the store View button opens only http(s) repo links. The live window.updateImageList (plugins_manager.js loads last, so its copy wins over the file-upload widget's) wrote the uploaded file's original name, path and ids into markup raw; they are escaped now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note plugin asset, action and inline handler guards Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): let the root pip wrapper install web_interface/requirements.txt Update Code, the automatic update's health check and Install Base Requirements install web_interface/requirements.txt through safe_pip_install.sh, which only allowed the root requirements.txt. The first commit changing that file would fail its dependency install, and the automatic updater rolls back any update whose dependencies did not install -- on every device, for every newer commit. The wrapper now lists both core requirement files. Only their folders are resolved, so a requirements.txt symlinked out of the project is compared by its target and refused (previously the root file's own symlink target was what got allowed). The updater's file list is a named constant, and a test runs the real wrapper (pip stubbed) on every file Update Code and the rollback install. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): do not retry plugin requests that got an HTTP answer PluginAPI.request wrapped everything that was not a structured error as NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a JSON error without error_code included. Check & Update All retries NETWORK_ERROR, so those updates were re-sent five more times with backoff, contrary to the #587 contract that an HTTP error response is the server's answer. NETWORK_ERROR now means only that fetch() rejected. Any HTTP response without an error_code, or with a body that is not JSON, is API_ERROR with the HTTP status attached. Tested against the shipped api_client.js. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scroll): restart the stats window when an idle gap is dropped by size #582 dropped an idle gap from the frame stats two ways: the reset_scroll() sentinel, which also restarts the 5s window timer, and a size guard for scrollers that never call reset_scroll(), which did not. On that path the first real frame after the gap found the boundary overdue and logged a stats line for a one-frame window. Both paths now share one seeding helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): leave plugins alone when update_core's own rollback fails update_core returns rollback_failed directly when a partial pull or an update whose health check never started cannot be rolled back. run() only held plugins back for 'verifying', so those devices still got new plugin versions and a display restart on top of a core in an unknown state -- the opposite of what the health-check path does, and of the 3.4.0 changelog (plugins are left alone if the rollback fails). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(api): make the REST reference match the api_v3 package Every documented request body, query parameter and response shape was re-checked against the handlers in web_interface/blueprints/api_v3/. Fixes calls that failed as documented (repo_url, action_id/params, files/image_id, font_file+font_family, ?font=, cache key, auto_enable_ap_mode, plugin limit keys), removes the font-override endpoints dropped in #566, corrects response shapes (plugins/config, plugins/schema, health, metrics, operation history, github-status, fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds the 26 routes it omitted (backup, system auto-update/git, wifi radio, starlark editor, MQTT bridge, status endpoints, skins). Documents the merge semantics of partial JSON saves to /config/main and /plugins/config and the dim-schedule POST accepting GET's days shape, which land in the same change set. Replaces app.py line numbers and the removed api_v3.py path with file and function names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): remove the General-tab plugin system toggles that did nothing plugin_system.auto_discover, auto_load_enabled and development_mode had General-tab toggles whose help tips promised dormant plugins and verbose logging, but nothing reads them: every enabled plugin is discovered and loaded regardless. Remove the three toggles. The keys stay tolerated in stored configs. The save handler now stores a flag only when a client sends it; treating a missing key as an unchecked box would otherwise rewrite all three to false on every General-tab save, which still posts plugins_directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(scroll): remove dead code left by #523/#570 - Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read them since the numpy blend replaced the scipy path. - Drop ScrollHelper._last_integer_position and frame_time_target, which were written but never read. - Keep target_fps and set_target_fps() but document them as informational: nothing paces off them, yet ledmatrix-elections' test_scroll_pacing.py reads helper.target_fps back and third-party plugins may call the setter. - Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to throttle", set_sub_pixel_scrolling's "default: True", and set_frame_based_scrolling's claim that it steps. The plugins monorepo was grepped for every removed name; none is used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI FONT_MANAGER.md told plugins to read display_manager.font_manager, which does not exist, so a plugin following it failed to load with AttributeError. The shared FontManager lives on the PluginManager and BasePlugin._get_font_manager() returns it (with a fallback for harnesses). Also removes the Fonts-tab override workflow and element-override panels that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls /plugins/store/search does not exist (404) and the list endpoint reads query, not q. The registry guide's curl examples omitted the JSON Content-Type, so the handlers saw an empty body and answered 400. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(config): use the shared core-key list in the last three private copies StartupValidator warned "Plugin 'auto_update' is enabled but not found" on every display start with auto-update or a dim schedule on; the reserved plugin-id check missed auto_update, sync, location and the rest; and ConfigManager's (uncalled) orphan cleanup would have deleted display, schedule and auto_update. All three now read src/core_config_keys.py, which also gains CORE_SECRETS_KEYS for the github/youtube secrets sections. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): partial JSON saves to /config/main change only what they send A JSON body with one field reset every checkbox in the sections it touched: the MQTT bridge's brightness slider turned off disable_hardware_pulsing, inverse_colors, show_refresh_rate and use_short_date_format, and a timezone-only save turned off web-UI autostart and weekly auto-updates. Missing-means-unchecked now applies only to form posts: form-encoded bodies and the v3 forms, which mark themselves with a hidden __form_section input. Also on the config routes: - vegas_min/max_cycle_duration no longer match the generic *_duration rule, so they stop landing in display_durations and a blank one no longer rejects the whole Display save; - saving from the Raw JSON editor calls start_setup_if_needed like the General form, so enabling auto-update there finishes its setup; - the schedule and dim-schedule POSTs accept the per-day days.<day> shape their GETs return, as well as the flat form keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): install plugin dependencies from the configured plugins directory install_plugin_dependencies.sh scanned only plugins/, but the Plugin Store installs into plugin_system.plugins_directory (default plugin-repos), so the documented "Recommended" fix found 0 plugins on every store install. It now reads plugins_directory from config/config.json (relative to the project root or absolute, default plugin-repos) and also scans plugins/ for dev symlinks, installing a plugin reached through both only once. With set -e alone, `pip ... | tee` took tee's exit status, so a failed pip install was reported as success; set -o pipefail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: replace stale API names, line numbers and the api_v3.py path - ADVANCED_FEATURES: StreamManager methods that exist (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the real on-demand status envelope ({status, data: {state, service}}) - app.py:199 / :144 / :607-619 line citations and web_interface/blueprints/api_v3.py (now a package) replaced with file and function names in ADVANCED_FEATURES, CONFIG_DEBUGGING, PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE, PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README - CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use /config/raw/main to replace the file; describe where validation runs - TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints usage) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): verify the web interface that actually ships, on port 5000 verify_installation.sh failed every healthy install: it required the long-removed web_interface_v2.py and looked for a listener on port 5001, while the web interface binds 5000 (web_interface/start.py). It now checks the files ledmatrix-web.service runs (start_web_conditionally.py, web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the same 5001 port in its listen check, HTTP probe and printed URLs. Port matches are anchored so :50001 no longer counts as :5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(plugins): one display-size contract: display_manager.width/height CLAUDE.md (#580) says to read display_manager.width/height because matrix is None when hardware init fails; the development guide, the safety-harness doc and two DisplayManager docstrings still recommended matrix.width/height. The bundled starlark-apps plugin read matrix.width unguarded, so its magnify recommendation and frame scaling raised in fallback mode (e.g. after the Pi 5 hardware refusal). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(install): make install_service.sh --help print usage instead of installing install_service.sh parsed no arguments, so `sudo ./scripts/install/ install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md) rewrote ledmatrix.service, ledmatrix-web.service and both update-verify units and enabled/started them. It now handles -h/--help (usage, exit 0, no changes) and rejects any other argument with exit 2 before doing anything. Running it with no arguments, as first_time_install.sh does, is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scroll): describe the fixed-step model and document frame_hold Since #545 a crisp speed from scroll_config.configure() makes the helper advance a fixed whole-pixel step per presented frame with no clock, and the display manager's frame hold is part of the speed. The docs still described the removed wall-clock model: - scroll_config's module and configure() docstrings said speed is applied in time-based mode and that omitting the hold "falls back to fractional pixels"; omitting it actually runs the scroll frame_hold times too fast. - SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both modes, and read a 20 ms stats median as missed refreshes although that is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step, the hold-dependent healthy median, that target_fps plays no part, and that a hand-added scroll_pixels_per_second loses to a schema-default pair. - PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling) without frame_hold; it now documents the parameter (core 3.4.0) with a configure() + set_scrolling_state example. - update_scroll_position/set_scroll_speed and set_scrolling_state docstrings say the same. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark target_fps legacy; describe what Vegas scroll_delay does - General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after the sports_scroll fix nothing in core scrolling reads it. The field and its API validation stay so saved configs and plugins that read global_config['target_fps'] keep working. CONFIG_REFERENCE says the same. - Vegas frame_based_scrolling/scroll_delay were described as frame-count stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and still advances by elapsed time, so the applied speed is clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The config comments, render_pipeline comment and CONFIG_REFERENCE rows now say so. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(deps): describe how plugin dependencies are really installed The guides said the web service runs as root, that installs pick --user from os.geteuid(), and quoted a warning and a PluginManager._install_plugin_dependencies() method that don't exist. The web unit runs as the installing user; store installs go through install_requirements_file() and sudo safe_pip_install.sh (root), with a user-level fallback that says so, and load-time installs run in the display service's own (root) interpreter. Manual paths now use the configured plugins directory (plugin-repos/ by default) instead of plugins/, which store installs no longer use, and install_plugin_dependencies.sh is described as scanning that directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): count local changes one way for the preflight and the pull The automatic update's preflight ignored mode-only changes and anything whose status line contained plugins/ or plugin-repos/, then promised "Automatic updates will not stash your changes". perform_core_update used plain git status (modes count) and ignored only 'plugins/', then ran 'git stash push -- :!plugins', which nothing ever pops. So an edit to a bundled plugin under plugin-repos/, or the installer's chmods on tracked scripts, passed the preflight and was stashed away for good. - auto_update.local_changes() is the one predicate both use: core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/ excluded by leading folder rather than substring (a core file under web_interface/static/v3/js/plugins/ now counts). - Update Code's explicit stash leaves out both plugin folders; the pull's --autostash carries their edits and mode changes across and reapplies them. - The automatic updater calls perform_core_update(stash_local_changes= False), which refuses instead of stashing edits that appeared after the preflight; update_core reports that as 'blocked'. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): diagnostics follow the web autostart default and api_v3 package #556 made a missing web_display_autostart mean "start" (only an explicit false/off keeps the web interface down), but the diagnostics still said otherwise: diagnose_web_ui.sh reported a missing key as "defaults to false", diagnose_web_interface.sh said the web interface "will not start unless this is set to true" and recommended enabling it, and debug_web_manual.py printed False. Troubleshooting a down web UI pointed users at a non-cause. Both shell scripts now evaluate the setting with the launcher's own autostart_enabled() (inline fallback if it cannot be imported) and report on / off / not set (on) / unparseable config; debug_web_manual.py uses the same function. They also check web_interface/blueprints/api_v3/ __init__.py: api_v3.py became a package in #553, so every healthy checkout was reported as missing a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(install): what install_service.sh installs; verify script port; no sudo for --help install_service.sh installs and starts ledmatrix, ledmatrix-web and the update-verify units, not only ledmatrix.service (systemd/README.md, README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help' as a harmless check; it now shows --help without sudo and warns what a real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh checks the web interface on port 5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note update-all, plugin system settings and script fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): size the preview after orientation and pixel mappers display_geometry.physical_size claimed to give DisplayManager's answer but only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured after the library's pixel mappers, so a Rotate:90 / orientation 90 chain previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for 128x64. Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown names leave it alone), and move the orientation composition here so DisplayManager and the preview share it. The module docstring no longer claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH. Audit finding F18. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): refuse settings the rgbmatrix library aborts on, on every board The library answers several settings with a NULL matrix or abort() rather than an error, so the display service crash-looped (Restart=on-failure) instead of reaching fallback mode: rows above 64, chain_length above 255 (uint8_t binding setter, documented as "no upper limit"), a misspelled hardware_mapping, and parallel 2-3 on a single-output mapping, reachable from the Display form on the default adafruit-hat(-pwm) mapping. #586 only guarded the Pi 5 subset. - src/matrix_support.py holds the rules for every board (Options::Validate ranges, binding integer types, mapping names and outputs from lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the API's numeric ranges. - DisplayManager checks them before building options and raises MatrixSettingsRefused, so a hand-edited config falls back with a logged, reported reason. Emulator mode only warns. - The config API refuses them with a 400 naming the setting; combinations are checked against stored values but reported only when the request sets a field involved. - The hardware status file gains "cause" (settings/library/forced). The fallback log and Display banner give the Pi 5 rebuild hint only for a library failure instead of rebuild + gpio_slowdown advice for every failure; one Pi 5 slowdown recommendation (1-3, start at 1). - The Display form offers classic/classic-pi1 and orientation 90/270 and renders any other stored mapping selected with a warning, so an unrelated save no longer rewrites them; the API accepts 90/270. Audit findings F03, F16, F19, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(display): library limits, template defaults and Pi 5 slowdown - rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs, classic/classic-pi1 mappings and orientation 90/270 documented. - Defaults are the config.template.json values: config migration adds missing keys from the template, so the listed "code defaults" never applied. - One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode, starting at 1. - Troubleshooting describes the refused-settings fallback, and CHANGELOG corrects the Unreleased "no upper limit" entry. Audit findings F19, F20, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py opens the panel with the service's options --measure and --demo built RGBMatrixOptions from a private copy of the display service's builder that had drifted: gpio_slowdown came from display.hardware (default 2) instead of display.runtime (default 3), and rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors, pixel_mapper_config and orientation were skipped, with different defaults (hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown was measured -- or garbled -- in a setup the service never drives. The option filling in DisplayManager._setup_matrix moves, unchanged, into DisplayManager.apply_matrix_options(options, config), which _setup_matrix calls and the script reuses (overriding only limit_refresh_rate_hz for --measure). The script now loads the whole config rather than the hardware block. Tests pin the script's options to the service's attribute for attribute. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py recommends keys the resolver honours The ladder ended by telling users to set display_options.scroll_pixels_per_second. scroll_config ranks that key below the scroll_speed + scroll_delay pair, deliberately, and several plugin schemas default the pair into config, so the advised key was silently ignored (a schema-default 1/0.02 pair plus an advised 66 still resolved to 50 px/s). The advice is now the pair that selects the crisp speed exactly (pixels_per_frame every frame_hold/refresh seconds), explains that the pair outranks scroll_pixels_per_second, and gives the scoreboards' per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the printed pair over a schema-default pair and check it lands on the advertised speed and hold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula - SPORTS_UNIFICATION.md still presented honouring global target_fps as sports_scroll's added behaviour and its one user-visible gain; note that it was withdrawn because it had become a speed multiplier. - ADVANCED_FEATURES.md gave Vegas scrolling as (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s by elapsed time, through a 0.1-5 px per scroll_delay clamp when frame_based_scrolling is on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): scroll model fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dev): link-github links plugins from the ledmatrix-plugins monorepo link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git, and those per-plugin repositories no longer exist: official plugins are directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the monorepo once into the dev directory, finds plugins/<name>, plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and links it under its manifest id. With an explicit repo URL it still links a single-repository plugin as before. dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a fork), plus plugins_repo and plugins_branch; github_pattern, which was documented but never read, is dropped and warned about. Ships dev_plugins.json.example and git-ignores dev_plugins.json, both of which the guide promised. Reading JSON falls back to python3 when jq is missing (get_plugin_id silently returned nothing without jq). update/status/list find the git checkout above a monorepo plugin directory (its .git is not in the plugin dir), and update pulls a shared checkout once. status no longer exits 1 when nothing is broken. Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration, workflow, store integration, hello-world link, submission), and the nonexistent scripts/git-hooks/pre-push-plugin-version and scripts/bump_plugin_version.py replaced with the real rule: bump the manifest version and run update_registry.py. scripts/dev/README.md and CLAUDE.md updated to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scripts): monorepo workspace layout; fix_perms and install READMEs MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin; setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the workspace file opens LEDMatrix plus ../ledmatrix-plugins. scripts/fix_perms/README.md listed cache directories fix_cache_permissions.sh never touches and a 'ledmatrix' service user that doesn't exist (also in scripts/install/README.md); adds safe_pip_install.sh. install/README: install_service.sh installs the web and update-verify units too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): keep the rollback's pip retries inside the unit time limit The health check reinstalled the previous requirements by trying the next bash path after any failure, including a 600 s pip timeout. Two files, two paths: up to 40 minutes of pip alone, while systemd stops ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the rollback half-way and leaving the update 'verifying' until the web UI calls it lost. - Like permission_utils.install_requirements_file, only a sudo refusal moves on to the next bash; a pip that ran and failed or timed out is not repeated. The refusal wording is one list (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only verifier and pinned equal by a test. - All reinstalls in one rollback share a 600 s budget. - WORST_CASE_SECONDS adds up every timeout on the longest path (27.5 min); a test holds it under the unit's TimeoutStartSec and that under the web UI's VERIFY_LOST_SECONDS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools Plugin config was prepared differently depending on how it arrived: - JSON POST /plugins/config built a partial body on schema defaults, so {"enabled": true} reset every other setting of the plugin. It now merges onto the stored section first, as the form path already did. - Legacy-boolean normalization (#588) ran only at load: GET /plugins/config returned the raw boolean, posting it back failed validation, and hot reload handed plugins the raw section (a legacy dynamic_duration: true came back as a boolean). schema_manager.prepare_plugin_config (normalize, then defaults) is now used by PluginManager.load_plugin, both save paths, GET, the save notifications and DisplayController's hot-reload callback. - The JSON save's filter kept only enabled/display_duration/live_priority and dropped a submitted skin, skin_options or vegas_* tuning key. There is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES, used by validation and by the save filter; PluginManager's CORE_OWNED_CONFIG_KEYS is its vegas subset. - Plugin sections posted to /config/main were stored verbatim, including values /plugins/config rejects. They now go through the same preparation (_prepare_plugin_config_for_save, extracted from save_plugin_config), and a failing section rejects the whole save before anything is written. - dev_server read only top-level defaults and let a schema enabled:false win; build_full_config shallow-merged overrides, dropping sibling defaults; the harness extracted defaults differently from the device. loading.build_config now uses the device's extraction and preparation, and dev_server, check_plugin, render_plugin and the harness all use it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt-bridge): brightness changes apply live and touch nothing else The display service's hot reload applies a saved brightness within a few seconds, and /config/main no longer resets other display settings on a brightness-only JSON body. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): automatic update hardening Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI It described web_interface_v2.py and index_v2.html (both gone), client-side form generation, one POST per field with {key, value}, and 'no nested objects'. The v3 UI renders plugin forms server-side from the schema (pages_v3 partial + plugin_config.html macros, nested sections and x-widgets), posts the whole form once, and save_plugin_config() merges onto the stored section, validates, splits x-secret fields and notifies the plugin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt): brightness saves apply via hot reload and leave other settings alone The bridge README said brightness is applied on the display's next restart; the display controller's config hot reload applies it within seconds. It also now states that the bridge's partial JSON save changes only brightness (the /config/main merge fix in this change set). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): don't log pip's output from the health check's reinstall pip can echo a private index URL with embedded credentials; permission_utils redacts it, the stdlib-only verifier cannot, so it logs the exit code only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark the plugin_system toggles as unused legacy keys auto_discover, auto_load_enabled and development_mode are read by nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and the REST reference listed them as live settings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): docs and developer tools group Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): legacy plugin-system toggles no longer count as a General save auto_discover, auto_load_enabled and development_mode have left the General form, so a post carrying only one of them is not a general-settings save and must not treat web_display_autostart and auto_update as unchecked. The plugin_system block itself is left as on main for the branch that reworks it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): config-save and plugin-config preparation fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(claude): re-check matrix_support.py rules when the library submodule is bumped Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address Codacy findings on the core audit PR - plugin_manager.prepare_plugin_config: when the fallback legacy-boolean pass also fails, log a warning instead of a bare except/pass. - api_client.js: request() refuses any endpoint that is not a plain path under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace, control characters) with INVALID_ENDPOINT before calling fetch(), and plugin ids are URL-encoded wherever they are put into a URL (also in the app-shell batch load). - test_update_all.js: pins both against the shipped client. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): check endpoint control characters without a control-character regex Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags the \x00-\x1f range in checkEndpoint's regex. Test the char codes instead; the endpoints refused are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(auto-update): make the seed script executable on disk, not only in the index On Linux Repo.publish() commits with -a, which recorded scripts/run.sh as 100644 upstream because the seed file was never chmod +x. The pull then brought in the same mode the installer chmod had made locally, so installer_chmod saw no mode change left to check. The updater was fine: with the upstream commit at 100755 the --autostash carries the device's chmod across. Verified under Linux (WSL, git 2.43): the old helper fails exactly as CI did, the fixed one passes all 63 tests in the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
@@ -25,7 +25,14 @@ from enum import Enum
|
||||
import queue
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from src.cache_manager import CacheManager
|
||||
from src.common.espn_dates import clamp_espn_limit, fetch_espn_date_chunks
|
||||
from src.common.espn_dates import (
|
||||
RANGE_RETRY_SECONDS,
|
||||
_note_range_rejected,
|
||||
_ranges_known_rejected,
|
||||
clamp_espn_limit,
|
||||
fetch_espn_date_chunks,
|
||||
parse_espn_date_range,
|
||||
)
|
||||
# Configure logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -350,19 +357,38 @@ class BackgroundDataService:
|
||||
|
||||
logger.info(f"Starting background fetch for {request.sport} {request.year}")
|
||||
|
||||
# Perform HTTP request with retry logic
|
||||
response = self._make_request_with_retry(request)
|
||||
|
||||
# ESPN stopped accepting dates=YYYYMMDD-YYYYMMDD on 2026-09-15 and
|
||||
# answers 400 for every sport. Re-ask in months and days rather
|
||||
# than let a whole season fail. See src/common/espn_dates.py.
|
||||
if response.status_code == 400:
|
||||
# The "ranges are rejected" memo is shared with
|
||||
# fetch_espn_scoreboard(): once either path has seen a range
|
||||
# rejected, the other skips the doomed range request too.
|
||||
is_range = parse_espn_date_range(request.params.get("dates")) is not None
|
||||
data = None
|
||||
chunks_tried = False
|
||||
if is_range and _ranges_known_rejected():
|
||||
data = self._fetch_in_date_chunks(request)
|
||||
if data is None:
|
||||
# Every chunk failed: ask for the range itself below so the
|
||||
# failure carries a real HTTP error, without re-spending chunks.
|
||||
chunks_tried = data is None
|
||||
|
||||
if data is None:
|
||||
# Perform HTTP request with retry logic
|
||||
response = self._make_request_with_retry(request)
|
||||
if is_range and response.status_code == 400 and not chunks_tried:
|
||||
_note_range_rejected()
|
||||
logger.warning(
|
||||
"ESPN rejected the date range %s (400); fetching it as "
|
||||
"month/day chunks, and fetching ranges that way for the "
|
||||
"next %d hours",
|
||||
request.params.get("dates"), RANGE_RETRY_SECONDS // 3600,
|
||||
)
|
||||
data = self._fetch_in_date_chunks(request)
|
||||
if data is None:
|
||||
response.raise_for_status()
|
||||
else:
|
||||
response.raise_for_status()
|
||||
else:
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
data = response.json()
|
||||
|
||||
# Validate data structure
|
||||
if not isinstance(data, dict):
|
||||
|
||||
@@ -373,6 +373,14 @@ def sudo_remove_directory(path: Path, allowed_bases: Optional[list] = None) -> b
|
||||
return False
|
||||
|
||||
|
||||
#: What sudo prints when it refuses a command line outright (not in sudoers for
|
||||
#: that exact argv, or it wants a password). Only then is another bash path
|
||||
#: worth trying: after pip itself ran, a retry just repeats the failure. The
|
||||
#: automatic update's health check keeps its own copy of this list
|
||||
#: (scripts/utils/auto_update_verify.py runs without importing src/).
|
||||
SUDO_REFUSAL_PHRASES = ("a password is required", "is not allowed to run", "no tty present")
|
||||
|
||||
|
||||
def install_requirements_file(req_file: Path, timeout: int = 300) -> subprocess.CompletedProcess:
|
||||
"""
|
||||
Install a requirements.txt file for a plugin (or the project itself).
|
||||
@@ -434,10 +442,7 @@ def install_requirements_file(req_file: Path, timeout: int = 300) -> subprocess.
|
||||
# Distinguish "sudo rejected this exact command line" (worth
|
||||
# trying the next bash candidate) from "sudo ran it but pip
|
||||
# itself failed" (a real error — stop and surface it).
|
||||
denied = any(
|
||||
phrase in result.stderr
|
||||
for phrase in ("a password is required", "is not allowed to run", "no tty present")
|
||||
)
|
||||
denied = any(phrase in result.stderr for phrase in SUDO_REFUSAL_PHRASES)
|
||||
if not denied:
|
||||
# Deliberately don't interpolate req_file or the pip output here:
|
||||
# this log line is scanner-visible, and a static analyzer can't
|
||||
|
||||
+33
-13
@@ -22,12 +22,27 @@ shimmer) or repeat frames (which reads as judder). :func:`resolve` warns when
|
||||
the requested speed will not divide evenly, because that is a real display
|
||||
artefact and not a rounding detail.
|
||||
|
||||
Speed is always expressed to the helper as pixels per second and applied in
|
||||
time-based mode. Frame-based stepping gates motion on a wall clock at
|
||||
``1/scroll_delay`` steps per second; plugins set ``scroll_delay`` to the frame
|
||||
period, which puts that comparison exactly on its own threshold and makes the
|
||||
step count flip on sub-millisecond jitter. Accumulating elapsed time keeps
|
||||
position proportional to real time instead.
|
||||
How the speed is applied
|
||||
------------------------
|
||||
:func:`configure` snaps the requested speed to a :class:`CrispSpeed` -- a whole
|
||||
number of pixels per presented frame, with each frame held for ``frame_hold``
|
||||
panel refreshes -- and puts the helper in fixed-step mode
|
||||
(``ScrollHelper.set_pixels_per_frame``). In that mode the helper consults no
|
||||
clock: every ``update_scroll_position()`` call moves exactly that many pixels.
|
||||
``SwapOnVSync`` blocks until the panel has taken each frame, so the frame count
|
||||
is the clock, and the speed on the panel is::
|
||||
|
||||
px/s = refresh_hz / frame_hold * pixels_per_frame
|
||||
|
||||
That makes the hold part of the speed. The caller must pass
|
||||
``settings.frame_hold`` to ``display_manager.set_scrolling_state(True, ...)``;
|
||||
a caller that omits it is presented every refresh and scrolls ``frame_hold``
|
||||
times too fast. The refresh is the panel's (``display_manager.refresh_hz``,
|
||||
i.e. ``display.hardware.limit_refresh_rate_hz``) -- never the global
|
||||
``target_fps``, which no longer paces anything.
|
||||
|
||||
With ``snap_to_crisp=False`` there is no fixed step: the helper is set to
|
||||
px/s and advances by elapsed time, and the hold is 1.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -330,17 +345,18 @@ def configure(
|
||||
) -> ScrollSettings:
|
||||
"""Resolve the config and apply it to ``scroll_helper``.
|
||||
|
||||
Applied in time-based mode: see the module docstring for why frame-based
|
||||
stepping is not used. ``hasattr`` guards keep this usable against older
|
||||
ScrollHelper builds that a plugin may be running on.
|
||||
Frame-based mode is switched off, the (snapped) speed is set, and with
|
||||
``snap_to_crisp`` the helper steps a fixed whole-pixel amount per presented
|
||||
frame -- see the module docstring. ``hasattr`` guards keep this usable
|
||||
against older ScrollHelper builds that a plugin may be running on.
|
||||
|
||||
:param display_manager: consulted for the panel's refresh rate only (it can
|
||||
see display.hardware; a plugin cannot). The frame hold is NOT applied
|
||||
here -- see the note in the body. The caller must pass
|
||||
``settings.frame_hold`` to ``display_manager.set_scrolling_state(True,
|
||||
...)`` when it starts scrolling, or a sub-refresh speed still presents
|
||||
a new frame every refresh and the motion falls back to fractional
|
||||
pixels.
|
||||
...)`` when it scrolls. The helper moves its fixed step on every call,
|
||||
so without the hold each step is presented every refresh and the
|
||||
scroll runs ``frame_hold`` times faster than the resolved speed.
|
||||
:param snap_to_crisp: move the requested speed to the nearest speed the
|
||||
panel can show in whole pixels. On by default because a speed that does
|
||||
not divide evenly has no good rendering, only a choice of artefacts.
|
||||
@@ -357,7 +373,8 @@ def configure(
|
||||
# refresh to fill in target_fps, pixels_per_frame and the judder warning,
|
||||
# so deriving it afterwards described a 100Hz panel to everyone running at
|
||||
# 60 -- and with snap_to_crisp=False nothing downstream corrected it, so
|
||||
# set_target_fps() paced the helper to 100 FPS on a 60Hz panel.
|
||||
# the settings and the helper's (informational) target_fps said 100 FPS on
|
||||
# a 60Hz panel.
|
||||
hz = _coerce(refresh_hz)
|
||||
if hz is None and display_manager is not None:
|
||||
hz = _coerce(getattr(display_manager, "refresh_hz", None))
|
||||
@@ -399,6 +416,9 @@ def configure(
|
||||
if hasattr(scroll_helper, "set_pixels_per_frame"):
|
||||
scroll_helper.set_pixels_per_frame(
|
||||
choice.pixels_per_frame if choice else None)
|
||||
# Informational only: nothing in the helper paces off target_fps. It is
|
||||
# still recorded because plugins read it back (ledmatrix-elections'
|
||||
# test_scroll_pacing.py asserts it equals the crisp presentation rate).
|
||||
if choice and hasattr(scroll_helper, "set_target_fps"):
|
||||
scroll_helper.set_target_fps(choice.frames_per_second)
|
||||
elif settings.target_fps and hasattr(scroll_helper, "set_target_fps"):
|
||||
|
||||
+60
-44
@@ -22,13 +22,6 @@ from typing import Optional, Dict, Any
|
||||
from PIL import Image
|
||||
import numpy as np
|
||||
|
||||
# Try to import scipy for sub-pixel interpolation, fallback to simpler method if not available
|
||||
try:
|
||||
from scipy.ndimage import shift
|
||||
HAS_SCIPY = True
|
||||
except ImportError:
|
||||
HAS_SCIPY = False
|
||||
|
||||
|
||||
# How often the frame-stats line is emitted, and therefore also the ceiling
|
||||
# on a believable frame time: a scroll that renders at all cannot take this
|
||||
@@ -138,21 +131,22 @@ class ScrollHelper:
|
||||
# blend (blur) or repeat frames (judder); blending is the worse of the
|
||||
# two here. Vegas mode still opts in via set_sub_pixel_scrolling().
|
||||
self.sub_pixel_scrolling = False
|
||||
self._last_integer_position = 0 # Cache for integer position to avoid repeated calculations
|
||||
|
||||
|
||||
# Frame-based scrolling settings
|
||||
self.frame_based_scrolling = False
|
||||
#: Whole pixels to advance per presented frame, or None to pace
|
||||
#: off elapsed time. See set_pixels_per_frame.
|
||||
self.fixed_pixels_per_frame = None # If True, use scroll_delay to throttle and move scroll_speed pixels
|
||||
self.last_step_time = 0.0 # Track last step time for frame-based throttling
|
||||
|
||||
self.fixed_pixels_per_frame = None
|
||||
self.last_step_time = 0.0 # Time of the last position update
|
||||
|
||||
# Time tracking for scroll updates
|
||||
self.last_update_time: Optional[float] = None
|
||||
|
||||
# High FPS settings
|
||||
self.target_fps = 120 # Target 120 FPS for smooth scrolling
|
||||
self.frame_time_target = 1.0 / self.target_fps
|
||||
|
||||
#: Informational only: the presentation rate scroll_config chose
|
||||
#: (panel refresh / frame hold). Nothing paces off it -- the helper
|
||||
#: steps per call and SwapOnVSync paces the calls. Kept because
|
||||
#: plugins and their tests read it back.
|
||||
self.target_fps = 120
|
||||
|
||||
# Dynamic duration settings
|
||||
self.dynamic_duration_enabled = True
|
||||
@@ -288,7 +282,14 @@ class ScrollHelper:
|
||||
|
||||
def update_scroll_position(self) -> None:
|
||||
"""
|
||||
Update scroll position with high FPS control and handle wrap-around.
|
||||
Advance the scroll by one presented frame and handle wrap-around.
|
||||
|
||||
With a fixed per-frame step (set_pixels_per_frame, which
|
||||
scroll_config.configure sets for a crisp speed) every call moves
|
||||
exactly that many pixels and no clock is read; the caller's
|
||||
vsync-blocking swap, held for ``frame_hold`` refreshes, sets the rate.
|
||||
Otherwise the position advances by elapsed time at the configured
|
||||
speed.
|
||||
"""
|
||||
if not self.cached_image:
|
||||
return
|
||||
@@ -461,10 +462,9 @@ class ScrollHelper:
|
||||
Linear blend between the frames at ``start_x`` and ``start_x + 1``.
|
||||
|
||||
Implemented with numpy rather than scipy.ndimage.shift: scipy is not
|
||||
installed on the target devices (HAS_SCIPY is False there), which is why
|
||||
the pre-existing sub-pixel path was dead code — get_visible_portion never
|
||||
consulted the flag, and the scipy fallback would not have interpolated
|
||||
anyway.
|
||||
installed on the target devices, and the old scipy-based sub-pixel path
|
||||
was dead code -- get_visible_portion never consulted the flag. The scipy
|
||||
import was removed with it; installing scipy has no effect.
|
||||
|
||||
Args:
|
||||
start_x: Left column of the earlier of the two frames
|
||||
@@ -830,11 +830,13 @@ class ScrollHelper:
|
||||
|
||||
def set_scroll_speed(self, speed: float) -> None:
|
||||
"""
|
||||
Set the scroll speed.
|
||||
|
||||
In time-based mode: pixels per second (typically 10-200)
|
||||
In frame-based mode: pixels per frame (typically 0.5-5 for smooth scrolling)
|
||||
|
||||
Set the scroll speed, and leave fixed-step mode.
|
||||
|
||||
In time-based mode: pixels per second (clamped to 1-500).
|
||||
In frame-based mode: pixels per ``scroll_delay`` seconds (clamped to
|
||||
0.1-5), still applied by elapsed time as scroll_speed / scroll_delay
|
||||
px/s.
|
||||
|
||||
Args:
|
||||
speed: Scroll speed (interpretation depends on frame_based_scrolling mode)
|
||||
"""
|
||||
@@ -896,14 +898,19 @@ class ScrollHelper:
|
||||
|
||||
def set_target_fps(self, fps: float) -> None:
|
||||
"""
|
||||
Set the target frames per second for scrolling.
|
||||
|
||||
Record the presentation rate, for diagnostics only.
|
||||
|
||||
Nothing paces off this value: with a fixed per-frame step the helper
|
||||
advances once per call, and without one it advances by elapsed time.
|
||||
The rate frames are shown at is the panel refresh divided by the frame
|
||||
hold passed to ``display_manager.set_scrolling_state``. scroll_config
|
||||
sets it to that rate so it can be read back.
|
||||
|
||||
Args:
|
||||
fps: Target FPS (typically 30-200, default 120)
|
||||
fps: Frames per second (clamped to 30-200)
|
||||
"""
|
||||
self.target_fps = max(30.0, min(200.0, fps))
|
||||
self.frame_time_target = 1.0 / self.target_fps
|
||||
self.logger.debug(f"Target FPS set to: {self.target_fps} FPS (frame_time_target: {self.frame_time_target:.4f}s)")
|
||||
self.logger.debug("Target FPS recorded: %s FPS (informational)", self.target_fps)
|
||||
|
||||
def set_sub_pixel_scrolling(self, enabled: bool) -> None:
|
||||
"""
|
||||
@@ -914,7 +921,7 @@ class ScrollHelper:
|
||||
When disabled, uses integer pixel positioning (faster but may skip pixels).
|
||||
|
||||
Args:
|
||||
enabled: True to enable sub-pixel scrolling (default: True)
|
||||
enabled: True to enable sub-pixel scrolling (default: False)
|
||||
"""
|
||||
self.sub_pixel_scrolling = enabled
|
||||
self.logger.debug(f"Sub-pixel scrolling {'enabled' if enabled else 'disabled'}")
|
||||
@@ -923,10 +930,12 @@ class ScrollHelper:
|
||||
"""
|
||||
Enable or disable frame-based scrolling.
|
||||
|
||||
When enabled, update_scroll_position() respects scroll_delay and moves
|
||||
scroll_speed pixels per step. This provides a "stepped" look similar to
|
||||
traditional tickers and can be visually smoother on LED matrices.
|
||||
|
||||
This does not step. When enabled, ``scroll_speed`` is read as pixels
|
||||
per ``scroll_delay`` seconds (set_scroll_speed clamps it to 0.1-5), and
|
||||
update_scroll_position() still advances by elapsed time at
|
||||
``scroll_speed / scroll_delay`` px/s. A fixed per-frame step set by
|
||||
set_pixels_per_frame() takes precedence over both modes.
|
||||
|
||||
Args:
|
||||
enabled: True to enable frame-based scrolling (default: False)
|
||||
"""
|
||||
@@ -985,11 +994,7 @@ class ScrollHelper:
|
||||
# the real stall rates being measured, so the number could not be
|
||||
# trusted at all. Seed the clock and take no sample.
|
||||
if self.last_frame_time is None:
|
||||
self.last_frame_time = current_time
|
||||
# Restart the window with the scroll. Otherwise the boundary is
|
||||
# already long overdue when the second frame arrives, and the new
|
||||
# scroll opens by reporting a window of exactly one frame.
|
||||
self.last_fps_log_time = current_time
|
||||
self._restart_stats_window(current_time)
|
||||
return
|
||||
|
||||
# Calculate instantaneous frame time
|
||||
@@ -998,9 +1003,10 @@ class ScrollHelper:
|
||||
# A caller that scrolls without ever calling reset_scroll() never arms
|
||||
# the sentinel above, so catch the same gap by its size. Nothing that
|
||||
# renders a scroll produces a frame longer than the log interval; a
|
||||
# sample that large is an idle period, not a frame.
|
||||
# sample that large is an idle period, not a frame. It starts a new
|
||||
# scroll exactly as the sentinel does, window timer included.
|
||||
if frame_time >= FPS_LOG_INTERVAL:
|
||||
self.last_frame_time = current_time
|
||||
self._restart_stats_window(current_time)
|
||||
return
|
||||
|
||||
self.frame_times.append(frame_time)
|
||||
@@ -1034,7 +1040,17 @@ class ScrollHelper:
|
||||
|
||||
self.last_frame_time = current_time
|
||||
self.frame_count += 1
|
||||
|
||||
|
||||
def _restart_stats_window(self, current_time: float) -> None:
|
||||
"""Seed the frame clock at the start of a scroll, taking no sample.
|
||||
|
||||
The window timer restarts with it. Otherwise the 5s boundary is
|
||||
already long overdue when the next frame arrives, and the new scroll
|
||||
opens by reporting a window of exactly one frame.
|
||||
"""
|
||||
self.last_frame_time = current_time
|
||||
self.last_fps_log_time = current_time
|
||||
|
||||
def clear_cache(self) -> None:
|
||||
"""
|
||||
Clear the cached scrolling image.
|
||||
|
||||
+47
-41
@@ -18,10 +18,15 @@ Promoting the content layer would be exactly the mistake
|
||||
``docs/SPORTS_UNIFICATION.md`` warns against — merging on the intuition that
|
||||
same-named methods are the same method. Same name, different job.
|
||||
|
||||
The one behavior this module adds over the plugin copies is native support for
|
||||
``global_config['target_fps']``: the bundled copies hardcode ~100 FPS via
|
||||
``scroll_delay=0.01`` and never consult the global smooth-scrolling target. A
|
||||
plugin inheriting from here gets it for free.
|
||||
What this module adds over the plugin copies is pacing through
|
||||
:mod:`src.common.scroll_config`, the resolver every other scroller uses: the
|
||||
configured px/s is snapped to a whole-pixel speed for the panel's refresh, the
|
||||
helper steps a fixed number of pixels per presented frame, and each drawn frame
|
||||
publishes the frame hold to the display manager. Speed depends only on
|
||||
``scroll_speed`` and the panel refresh
|
||||
(``display.hardware.limit_refresh_rate_hz``) -- not on the global
|
||||
``target_fps``, and not on ``scroll_delay``, which is kept in the settings for
|
||||
compatibility only.
|
||||
|
||||
Usage::
|
||||
|
||||
@@ -94,9 +99,9 @@ class SportsScrollDisplay:
|
||||
:param display_manager: the core display manager
|
||||
:param config: the plugin's configuration
|
||||
:param custom_logger: the plugin's logger, so scroll lines are attributed
|
||||
:param global_config: the LEDMatrix global config — the source of
|
||||
``target_fps``. Optional so an older caller that does not pass it
|
||||
keeps working at the config-derived pacing.
|
||||
:param global_config: the LEDMatrix global config, consulted for the
|
||||
panel refresh when the display manager cannot report one.
|
||||
Optional so an older caller that does not pass it keeps working.
|
||||
"""
|
||||
self.display_manager = display_manager
|
||||
self.config = config
|
||||
@@ -196,21 +201,6 @@ class SportsScrollDisplay:
|
||||
return {**settings, **override}
|
||||
return settings
|
||||
|
||||
def _resolve_target_fps(self) -> Optional[float]:
|
||||
"""The global smooth-scrolling FPS target, or None to keep config pacing.
|
||||
|
||||
Coerced before use: a malformed value in the global config must degrade
|
||||
to the existing ``scroll_delay`` pacing, never raise on a display path.
|
||||
"""
|
||||
raw = self.global_config.get("target_fps") or self.global_config.get(
|
||||
"scroll_target_fps"
|
||||
)
|
||||
try:
|
||||
return float(raw) if raw is not None else None
|
||||
except (TypeError, ValueError):
|
||||
self.logger.debug("Ignoring unusable target_fps: %r", raw)
|
||||
return None
|
||||
|
||||
def _coerce_float(self, value: Any, default: float) -> float:
|
||||
"""A usable float from config, or ``default``.
|
||||
|
||||
@@ -246,8 +236,8 @@ class SportsScrollDisplay:
|
||||
# settings dict: the two read the same key names with different
|
||||
# meanings, and the collision is a factor of 1/scroll_delay.
|
||||
#
|
||||
# sports_scroll: scroll_speed is px/SECOND; scroll_delay is only the
|
||||
# frame period used to reach px/frame.
|
||||
# sports_scroll: scroll_speed is px/SECOND; scroll_delay is ignored
|
||||
# for pacing (see _resolve_pixels_per_second).
|
||||
# scroll_config: scroll_speed is px per STEP, so px/s = speed/delay.
|
||||
#
|
||||
# Passing {"scroll_speed": 50.0, "scroll_delay": 0.01} straight through
|
||||
@@ -276,8 +266,13 @@ class SportsScrollDisplay:
|
||||
def _resolve_pixels_per_second(self, settings: Dict[str, Any]) -> float:
|
||||
"""This module's config shape, expressed as plain pixels per second.
|
||||
|
||||
``scroll_speed`` is already px/s here. ``scroll_delay`` only matters
|
||||
when a caller supplied px/frame instead, which the 0 case covers.
|
||||
``scroll_speed`` is already px/s here, and it is the whole answer.
|
||||
``scroll_delay`` is kept in the settings for compatibility but is
|
||||
ignored for pacing: the frame rate is the panel refresh divided by the
|
||||
frame hold scroll_config picks, so no positive delay changes the speed.
|
||||
The one exception is a delay of exactly 0 (below every scoreboard
|
||||
schema's minimum), which is read as ``scroll_speed`` being px per
|
||||
frame at ``ASSUMED_FPS_WHEN_UNPACED``.
|
||||
"""
|
||||
scroll_speed = self._coerce_float(settings.get("scroll_speed"), 50.0)
|
||||
scroll_delay = self._coerce_float(settings.get("scroll_delay"), 0.01)
|
||||
@@ -285,19 +280,29 @@ class SportsScrollDisplay:
|
||||
return scroll_speed * ASSUMED_FPS_WHEN_UNPACED
|
||||
return scroll_speed
|
||||
|
||||
def _resolve_refresh_hz(self) -> Optional[float]:
|
||||
def _resolve_refresh_hz(self) -> float:
|
||||
"""The panel refresh the crisp ladder should be computed against.
|
||||
|
||||
Prefers the configured hardware refresh. Falls back to the global
|
||||
``target_fps``/``scroll_target_fps`` this module has always honoured:
|
||||
under the old model that key *was* the rate frames were presented at,
|
||||
so it is the faithful translation for anyone who set it. Returning
|
||||
None lets scroll_config apply its own default.
|
||||
This has to be the rate frames are actually presented at, because the
|
||||
helper advances a fixed number of whole pixels per presented frame and
|
||||
the display manager holds each frame for ``frame_hold`` refreshes. A
|
||||
ladder built for any other rate plays back at the wrong speed: built
|
||||
for 60 Hz and shown on a 100 Hz panel, it runs 100/60 too fast.
|
||||
|
||||
So it is the display manager's ``refresh_hz`` (what the panel is
|
||||
driven at), then ``display.hardware.limit_refresh_rate_hz`` from the
|
||||
global config, then scroll_config's default. The global ``target_fps``
|
||||
is deliberately NOT consulted: before frame-locked presentation it was
|
||||
the rate frames were shown at, but now honouring it only turned the
|
||||
General tab's "Scroll Frame Rate" into a scoreboard speed multiplier.
|
||||
"""
|
||||
hardware = scroll_config.refresh_hz_from_config(self.global_config)
|
||||
if hardware and hardware != scroll_config.DEFAULT_REFRESH_HZ:
|
||||
return hardware
|
||||
return self._resolve_target_fps() or hardware or None
|
||||
reported = getattr(self.display_manager, "refresh_hz", None)
|
||||
# A real number only: a stub or a mock answering for anything must not
|
||||
# become the refresh rate.
|
||||
if (isinstance(reported, (int, float)) and not isinstance(reported, bool)
|
||||
and reported > 0):
|
||||
return float(reported)
|
||||
return scroll_config.refresh_hz_from_config(self.global_config)
|
||||
|
||||
def _scroll_frame_hold(self) -> int:
|
||||
"""Refreshes to hold each frame for, from the resolved settings."""
|
||||
@@ -327,11 +332,12 @@ class SportsScrollDisplay:
|
||||
return False
|
||||
|
||||
# Tell the core the panel is scrolling, and for how many
|
||||
# refreshes to hold each frame. Without this the frame hold is
|
||||
# never applied -- so a speed the ladder made crisp still presents
|
||||
# a new frame every refresh and judders -- and, because deferred
|
||||
# updates only run while nothing is scrolling, core would run
|
||||
# blocking work in the middle of this scroll.
|
||||
# refreshes to hold each frame. The helper advances a fixed number
|
||||
# of whole pixels per presented frame, so without the hold a new
|
||||
# frame is presented every refresh and the scroll runs frame_hold
|
||||
# times too fast. And because deferred updates only run while
|
||||
# nothing is scrolling, core would otherwise run blocking work in
|
||||
# the middle of this scroll.
|
||||
if hasattr(self.display_manager, "set_scrolling_state"):
|
||||
self.display_manager.set_scrolling_state(
|
||||
True, frame_hold=self._scroll_frame_hold())
|
||||
|
||||
@@ -29,6 +29,7 @@ import os
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional, List
|
||||
from src.core_config_keys import CORE_CONFIG_KEYS, CORE_SECRETS_KEYS
|
||||
from src.exceptions import ConfigError
|
||||
from src.logging_config import get_logger
|
||||
from src.config_manager_atomic import (
|
||||
@@ -756,12 +757,13 @@ class ConfigManager:
|
||||
|
||||
valid_set = set(valid_plugin_ids)
|
||||
|
||||
# Find orphaned plugins in main config
|
||||
main_plugins = set(main_config.keys())
|
||||
# Find orphaned plugins in main config. Core sections (display,
|
||||
# schedule, auto_update, ...) are not plugins and never orphans.
|
||||
main_plugins = set(main_config.keys()) - CORE_CONFIG_KEYS
|
||||
orphaned_main = main_plugins - valid_set
|
||||
|
||||
# Find orphaned plugins in secrets config
|
||||
secrets_plugins = set(secrets_config.keys())
|
||||
secrets_plugins = set(secrets_config.keys()) - CORE_CONFIG_KEYS - CORE_SECRETS_KEYS
|
||||
orphaned_secrets = secrets_plugins - valid_set
|
||||
|
||||
all_orphaned = orphaned_main | orphaned_secrets
|
||||
@@ -815,8 +817,8 @@ class ConfigManager:
|
||||
if not isinstance(plugin_config, dict):
|
||||
continue
|
||||
|
||||
# Skip non-plugin config sections
|
||||
if plugin_id in ['display', 'schedule', 'timezone', 'plugin_system']:
|
||||
# Skip core config sections
|
||||
if plugin_id in CORE_CONFIG_KEYS:
|
||||
continue
|
||||
|
||||
schema = plugin_schema_manager.load_schema(plugin_id, use_cache=True)
|
||||
|
||||
@@ -41,3 +41,13 @@ CORE_CONFIG_KEYS = frozenset({
|
||||
'vegas_excluded_plugins',
|
||||
'vegas_scroll_enabled',
|
||||
})
|
||||
|
||||
#: Top-level keys of ``config_secrets.json`` that belong to the core rather than
|
||||
#: to a plugin: the GitHub token the Plugin Store reads, and the historical
|
||||
#: ``youtube`` section. Plugin secrets are namespaced by plugin id, so anything
|
||||
#: deciding whether a secrets section is a plugin's needs this as well as
|
||||
#: ``CORE_CONFIG_KEYS``.
|
||||
CORE_SECRETS_KEYS = frozenset({
|
||||
'github',
|
||||
'youtube',
|
||||
})
|
||||
|
||||
@@ -2976,6 +2976,14 @@ class DisplayController:
|
||||
_pid: str = plugin_id, _plugin: Any = plugin_instance) -> None:
|
||||
"""Callback for plugin config changes."""
|
||||
try:
|
||||
# ConfigService hands over the raw config.json section.
|
||||
# Prepare it as loading did (legacy booleans read as
|
||||
# objects, schema defaults filled in), so the plugin gets
|
||||
# the same shape it was constructed with.
|
||||
prepare = getattr(self.plugin_manager, 'prepare_plugin_config', None)
|
||||
prepared = prepare(_pid, new_config) if callable(prepare) else None
|
||||
if isinstance(prepared, dict):
|
||||
new_config = prepared
|
||||
_plugin.on_config_change(new_config)
|
||||
logger.debug("Plugin %s notified of config change", _pid)
|
||||
except Exception as e:
|
||||
|
||||
+142
-8
@@ -1,17 +1,25 @@
|
||||
"""Display size from config: the one computation every caller shares.
|
||||
"""Display size from config: the one computation the size readers share.
|
||||
|
||||
``DisplayManager`` sizes its canvas from ``display.hardware`` plus
|
||||
``display.double_sided``. The web preview, the Starlark magnify default and
|
||||
the multi-display sync handshake used to re-derive that size themselves,
|
||||
each with its own defaults (``chain_length`` fell back to 2 in one place and
|
||||
1 in three others) and none of them applying double-sided mode. They now all
|
||||
call this module.
|
||||
``display.double_sided``. The web preview endpoints (``/display/current``, the
|
||||
SSE fallback), the Starlark magnify default and ``scripts/dev/vegas_audit.py``
|
||||
used to re-derive that size themselves, each with its own defaults
|
||||
(``chain_length`` fell back to 2 in one place and 1 in others) and none of
|
||||
them applying double-sided mode. They now call this module. (The multi-display
|
||||
sync handshake doesn't compute a size; it shares only
|
||||
``DEFAULT_CHAIN_LENGTH``.)
|
||||
|
||||
The size includes what the rgbmatrix library's pixel mappers do to it --
|
||||
``orientation`` and ``pixel_mapper_config`` (``Rotate:90`` swaps the axes,
|
||||
``U-mapper`` folds the chain) -- because ``RGBMatrix.width``/``height``, which
|
||||
``DisplayManager`` reports on hardware, are measured after them.
|
||||
|
||||
Kept free of hardware imports on purpose: the web interface imports it, and
|
||||
``display_manager`` pulls in ``rgbmatrix``.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import re
|
||||
from typing import Any, Dict, Mapping, Optional, Tuple
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -22,6 +30,10 @@ DEFAULT_COLS = 64
|
||||
DEFAULT_CHAIN_LENGTH = 2
|
||||
DEFAULT_PARALLEL = 1
|
||||
|
||||
#: ``display.hardware.orientation`` -> degrees of the ``Rotate`` mapper
|
||||
#: DisplayManager appends to ``pixel_mapper_config`` (None: no mapper).
|
||||
ORIENTATION_ROTATE_DEGREES = {'normal': None, '90': 90, '180': 180, '270': 270}
|
||||
|
||||
|
||||
def _display(config: Optional[Mapping[str, Any]]) -> Mapping[str, Any]:
|
||||
# A hand-edited config.json can hold anything here; treat a non-mapping
|
||||
@@ -35,10 +47,128 @@ def _hardware(config: Optional[Mapping[str, Any]]) -> Mapping[str, Any]:
|
||||
return hw if isinstance(hw, Mapping) else {}
|
||||
|
||||
|
||||
def compose_pixel_mapper_config(hardware: Mapping[str, Any]) -> str:
|
||||
"""The ``pixel_mapper_config`` DisplayManager hands the library.
|
||||
|
||||
``pixel_mapper_config`` stays a free-form advanced field (e.g. "U-mapper"
|
||||
for chain layouts); ``orientation`` is the user-facing mounting rotation,
|
||||
appended as a trailing ``Rotate:<deg>`` mapper rather than overwriting it.
|
||||
"""
|
||||
base = hardware.get('pixel_mapper_config') or ''
|
||||
base = base.strip() if isinstance(base, str) else ''
|
||||
degrees = ORIENTATION_ROTATE_DEGREES.get(hardware.get('orientation', 'normal'))
|
||||
if degrees is None:
|
||||
return base
|
||||
rotate = f'Rotate:{degrees}'
|
||||
return f'{base};{rotate}' if base else rotate
|
||||
|
||||
|
||||
def _c_div(a: int, b: int) -> int:
|
||||
"""C integer division (truncates toward zero)."""
|
||||
q = abs(a) // abs(b)
|
||||
return q if (a >= 0) == (b >= 0) else -q
|
||||
|
||||
|
||||
def _c_strtol(text: str) -> Tuple[int, str]:
|
||||
"""strtol(text, &end, 10): the parsed value (0 if none) and the rest."""
|
||||
match = re.match(r'\s*([+-]?\d+)', text)
|
||||
if not match:
|
||||
return 0, text
|
||||
return int(match.group(1)), text[match.end():]
|
||||
|
||||
|
||||
def _remap_size(param: Optional[str], width: int, height: int,
|
||||
chain: int, parallel: int) -> Optional[Tuple[int, int]]:
|
||||
"""RemapMapper::SetParameters and GetSizeMapping (lib/pixel-mapper.cc)."""
|
||||
if not param:
|
||||
return None
|
||||
new_w, rest = _c_strtol(param)
|
||||
if not rest.startswith(','):
|
||||
return None
|
||||
new_h, rest = _c_strtol(rest[1:])
|
||||
if not rest.startswith('|'):
|
||||
return None
|
||||
rest = rest[1:]
|
||||
tiles = []
|
||||
while rest:
|
||||
x, rest = _c_strtol(rest)
|
||||
if not rest.startswith(','):
|
||||
return None
|
||||
y, rest = _c_strtol(rest[1:])
|
||||
if not rest or rest[0].lower() not in 'neswx':
|
||||
return None
|
||||
tiles.append((x, y, rest[0].lower()))
|
||||
rest = rest[1:]
|
||||
if rest.startswith('|'):
|
||||
rest = rest[1:]
|
||||
elif rest:
|
||||
return None
|
||||
if len(tiles) != chain * parallel:
|
||||
return None
|
||||
panel_w, panel_h = _c_div(width, chain), _c_div(height, parallel)
|
||||
for x, y, kind in tiles:
|
||||
if kind == 'x':
|
||||
continue
|
||||
# MapTile::MapToVisible of the panel's (0, 0) and far corner.
|
||||
x0, y0, x1, y1 = {
|
||||
'n': (x, y, x + panel_w - 1, y + panel_h - 1),
|
||||
'w': (x, y + panel_w - 1, x + panel_h - 1, y),
|
||||
's': (x + panel_w - 1, y + panel_h - 1, x, y),
|
||||
'e': (x + panel_h - 1, y, x, y + panel_w - 1),
|
||||
}[kind]
|
||||
if x1 < 0 or x0 >= new_w or y1 < 0 or y0 >= new_h:
|
||||
return None
|
||||
return new_w, new_h
|
||||
|
||||
|
||||
def apply_pixel_mappers(width: int, height: int, mapper_config: str,
|
||||
chain: int, parallel: int) -> Tuple[int, int]:
|
||||
"""The canvas size after the library applies ``mapper_config``.
|
||||
|
||||
Mirrors ``RGBMatrix::Impl::ApplyNamedPixelMappers`` and each built-in
|
||||
mapper's ``SetParameters``/``GetSizeMapping`` in the pinned
|
||||
``lib/pixel-mapper.cc``: mappers apply left to right, and one the library
|
||||
doesn't know or can't configure is skipped and leaves the size alone.
|
||||
``multiplexing`` isn't modelled: its mappers give back the configured
|
||||
size for the panel sizes they are made for.
|
||||
"""
|
||||
for entry in (mapper_config or '').split(';'):
|
||||
name, colon, param = entry.partition(':')
|
||||
name = name.lower()
|
||||
param = param if colon else None
|
||||
if name == 'rotate':
|
||||
if not param:
|
||||
continue
|
||||
angle, rest = _c_strtol(param)
|
||||
if rest or angle % 90:
|
||||
continue
|
||||
if angle % 180:
|
||||
width, height = height, width
|
||||
elif name == 'u-mapper':
|
||||
if chain < 2 or chain % 2 or height % parallel:
|
||||
continue
|
||||
width, height = _c_div(width, 64) * 32, 2 * height
|
||||
elif name == 'v-mapper':
|
||||
width, height = (_c_div(width * parallel, chain),
|
||||
_c_div(height * chain, parallel))
|
||||
elif name == 'stacktorow':
|
||||
if param and any(c not in 'ZzFf, ' for c in param):
|
||||
continue
|
||||
width, height = width * parallel, _c_div(height, parallel)
|
||||
elif name == 'remap':
|
||||
size = _remap_size(param, width, height, chain, parallel)
|
||||
if size is not None:
|
||||
width, height = size
|
||||
# "mirror" keeps the size; the library skips names it doesn't know.
|
||||
return width, height
|
||||
|
||||
|
||||
def physical_size(config: Optional[Mapping[str, Any]]) -> Tuple[int, int]:
|
||||
"""Width and height of the whole panel chain, in pixels.
|
||||
|
||||
``cols * chain_length`` by ``rows * parallel``. Raises ``ValueError`` or
|
||||
``cols * chain_length`` by ``rows * parallel``, then through the pixel
|
||||
mappers ``orientation`` and ``pixel_mapper_config`` set up -- what
|
||||
``RGBMatrix.width``/``height`` report. Raises ``ValueError`` or
|
||||
``TypeError`` on a non-numeric value, as ``DisplayManager`` does; callers
|
||||
decide their own fallback.
|
||||
|
||||
@@ -54,7 +184,11 @@ def physical_size(config: Optional[Mapping[str, Any]]) -> Tuple[int, int]:
|
||||
parallel = int(hw.get('parallel', DEFAULT_PARALLEL))
|
||||
except OverflowError as e:
|
||||
raise ValueError(f"display.hardware size is not finite: {e}") from e
|
||||
return max(1, cols * chain_length), max(1, rows * parallel)
|
||||
width, height = max(1, cols * chain_length), max(1, rows * parallel)
|
||||
if chain_length >= 1 and parallel >= 1:
|
||||
width, height = apply_pixel_mappers(
|
||||
width, height, compose_pixel_mapper_config(hw), chain_length, parallel)
|
||||
return max(1, width), max(1, height)
|
||||
|
||||
|
||||
def resolve_double_sided(physical_width: int, physical_height: int,
|
||||
|
||||
+138
-81
@@ -37,9 +37,11 @@ from PIL import Image, ImageDraw, ImageFont
|
||||
from src.common.font_layout import crisp_size, load_truetype, resolve_asset_path
|
||||
from src.display_geometry import (
|
||||
DEFAULT_CHAIN_LENGTH, DEFAULT_COLS, DEFAULT_PARALLEL, DEFAULT_ROWS,
|
||||
physical_size, resolve_double_sided,
|
||||
ORIENTATION_ROTATE_DEGREES, compose_pixel_mapper_config, physical_size,
|
||||
resolve_double_sided,
|
||||
)
|
||||
from src.pi5_matrix_support import is_raspberry_pi_5, pi5_unsupported_settings
|
||||
from src.matrix_support import MatrixSettingsRefused, library_refusals, refusal_message
|
||||
from src.pi5_matrix_support import is_raspberry_pi_5
|
||||
import threading
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
@@ -91,9 +93,11 @@ class _LogicalMatrix:
|
||||
"""Proxy that reports a logical (per-screen) size for a physical matrix.
|
||||
|
||||
In double-sided mode the physical panel chain shows N identical copies of a
|
||||
smaller logical screen. Plugins size themselves from ``matrix.width`` /
|
||||
``matrix.height`` (the documented convention, used at 30+ call sites), so
|
||||
this proxy reports the logical dimensions while delegating every real
|
||||
smaller logical screen. Plugins size themselves from
|
||||
``display_manager.width`` / ``height`` (the documented convention), which
|
||||
defer to ``matrix.width`` / ``matrix.height`` -- and many older plugins read
|
||||
``matrix.width`` directly -- so this proxy reports the logical dimensions
|
||||
while delegating every real
|
||||
operation — ``CreateFrameCanvas``, ``SwapOnVSync``, ``brightness``,
|
||||
``Clear`` and so on — to the underlying physical matrix. The duplication
|
||||
itself happens once per frame in :meth:`DisplayManager.update_display`.
|
||||
@@ -247,28 +251,41 @@ class DisplayManager:
|
||||
# Calendar manager is now initialized by DisplayController
|
||||
|
||||
# Orientation setting -> rpi-rgb-led-matrix "Rotate:<deg>" pixel-mapper suffix.
|
||||
# "normal" needs no suffix since 0 degrees is the identity transform.
|
||||
_ORIENTATION_ROTATE_DEGREES = {'normal': None, '90': 90, '180': 180, '270': 270}
|
||||
_ORIENTATION_ROTATE_DEGREES = ORIENTATION_ROTATE_DEGREES
|
||||
|
||||
def _build_pixel_mapper_config(self, hardware_config: dict) -> str:
|
||||
"""Compose the raw pixel_mapper_config string with the orientation setting.
|
||||
"""Compose pixel_mapper_config with the orientation setting.
|
||||
|
||||
`pixel_mapper_config` stays available as a free-form advanced field (e.g.
|
||||
for "U-mapper" chain layouts); `orientation` is the user-facing dropdown
|
||||
for physical mounting (e.g. panels mounted upside down) and is appended as
|
||||
a "Rotate:<deg>" mapper rather than overwriting any existing config.
|
||||
See :func:`src.display_geometry.compose_pixel_mapper_config`, which the
|
||||
web preview shares so it sizes the canvas the same way.
|
||||
"""
|
||||
base_mapper = (hardware_config.get('pixel_mapper_config') or '').strip()
|
||||
orientation = hardware_config.get('orientation', 'normal')
|
||||
degrees = self._ORIENTATION_ROTATE_DEGREES.get(orientation)
|
||||
if degrees is None:
|
||||
return base_mapper
|
||||
rotate_mapper = f'Rotate:{degrees}'
|
||||
return f'{base_mapper};{rotate_mapper}' if base_mapper else rotate_mapper
|
||||
return compose_pixel_mapper_config(hardware_config)
|
||||
|
||||
@staticmethod
|
||||
def _fallback_advice(cause: str, error: Exception) -> str:
|
||||
"""What to do about a failed matrix init, for the log.
|
||||
|
||||
Only a library failure gets the rebuild hint: advice about the build
|
||||
or GPIO timing sends someone whose settings were refused the wrong way.
|
||||
"""
|
||||
if cause == "settings":
|
||||
return (f"{error} Change these in the web interface's Display tab "
|
||||
"(or display.hardware / display.runtime in config.json) "
|
||||
"and restart the display service.")
|
||||
if cause == "forced":
|
||||
return f"Error: {error}."
|
||||
advice = (f"Error: {error}. If the rgbmatrix library printed a message "
|
||||
"just before this, it names the problem.")
|
||||
if is_raspberry_pi_5():
|
||||
advice += (" On a Raspberry Pi 5, an mmap error means the library was "
|
||||
"built without Pi 5 support: sudo RPI_RGB_FORCE_REBUILD=1 "
|
||||
"./first_time_install.sh")
|
||||
return advice
|
||||
|
||||
def _setup_matrix(self):
|
||||
"""Initialize the RGB matrix with configuration settings."""
|
||||
_init_error_str = None
|
||||
_init_cause = None
|
||||
try:
|
||||
# Allow callers (e.g., web UI) to force non-hardware fallback mode
|
||||
if getattr(self, '_force_fallback', False):
|
||||
@@ -278,63 +295,25 @@ class DisplayManager:
|
||||
# Hardware configuration
|
||||
hardware_config = self.config.get('display', {}).get('hardware', {})
|
||||
runtime_config = self.config.get('display', {}).get('runtime', {})
|
||||
|
||||
# The library has no error path for many settings it can't use:
|
||||
# it returns no matrix (which the binding doesn't check, so the
|
||||
# process crashes on its next call) or calls abort(), and systemd
|
||||
# restarts the service into the same crash. Refuse those first so
|
||||
# they become a logged, reported fallback (src/matrix_support.py).
|
||||
refused = refusal_message(library_refusals(
|
||||
hardware_config, runtime_config, pi5=is_raspberry_pi_5()))
|
||||
if refused:
|
||||
if os.getenv("EMULATOR", "false") != "true":
|
||||
raise MatrixSettingsRefused(refused)
|
||||
logger.warning("Emulator mode: continuing, but on a real panel the display would not start. %s", refused)
|
||||
|
||||
# Basic hardware settings
|
||||
options.rows = hardware_config.get('rows', DEFAULT_ROWS)
|
||||
options.cols = hardware_config.get('cols', DEFAULT_COLS)
|
||||
options.chain_length = hardware_config.get('chain_length', DEFAULT_CHAIN_LENGTH)
|
||||
options.parallel = hardware_config.get('parallel', DEFAULT_PARALLEL)
|
||||
options.hardware_mapping = hardware_config.get('hardware_mapping', 'adafruit-hat-pwm')
|
||||
|
||||
# Performance and stability settings
|
||||
options.brightness = hardware_config.get('brightness', 90)
|
||||
options.pwm_bits = hardware_config.get('pwm_bits', 10)
|
||||
options.pwm_lsb_nanoseconds = hardware_config.get('pwm_lsb_nanoseconds', 150)
|
||||
options.led_rgb_sequence = hardware_config.get('led_rgb_sequence', 'RGB')
|
||||
options.pixel_mapper_config = self._build_pixel_mapper_config(hardware_config)
|
||||
options.row_address_type = hardware_config.get('row_address_type', 0)
|
||||
options.multiplexing = hardware_config.get('multiplexing', 0)
|
||||
options.panel_type = hardware_config.get('panel_type', '')
|
||||
options.disable_hardware_pulsing = hardware_config.get('disable_hardware_pulsing', False)
|
||||
options.show_refresh_rate = hardware_config.get('show_refresh_rate', False)
|
||||
options.limit_refresh_rate_hz = hardware_config.get('limit_refresh_rate_hz', 90)
|
||||
options.gpio_slowdown = runtime_config.get('gpio_slowdown', 3)
|
||||
|
||||
# Disable internal privilege dropping - we manage this via systemd or remain root
|
||||
# This prevents the library from dropping to 'daemon' user which breaks file permissions
|
||||
options.drop_privileges = False
|
||||
|
||||
# Additional settings from config
|
||||
if 'scan_mode' in hardware_config:
|
||||
options.scan_mode = hardware_config.get('scan_mode')
|
||||
if 'pwm_dither_bits' in hardware_config:
|
||||
options.pwm_dither_bits = hardware_config.get('pwm_dither_bits')
|
||||
if 'inverse_colors' in hardware_config:
|
||||
options.inverse_colors = hardware_config.get('inverse_colors')
|
||||
# Pi 5 only: 0=PIO/RP1 coprocessor (default, less CPU),
|
||||
# 1=RIO/Registered IO (faster; gpio_slowdown effect is inverted in this mode)
|
||||
if 'rp1_rio' in runtime_config:
|
||||
if hasattr(options, 'rp1_rio'):
|
||||
options.rp1_rio = runtime_config.get('rp1_rio')
|
||||
else:
|
||||
logger.warning(
|
||||
"rp1_rio is set in config but the installed rgbmatrix library does "
|
||||
"not support it — the library was likely built without Pi 5 RP1 "
|
||||
"support (mmap to 0x3f000000 instead of RP1 chip). "
|
||||
"Fix: sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh"
|
||||
)
|
||||
# Every option comes from display.hardware / display.runtime, in
|
||||
# one place that scripts/scroll_speeds.py shares.
|
||||
self.apply_matrix_options(options, self.config)
|
||||
|
||||
logger.info(f"Initializing RGB Matrix with settings: rows={options.rows}, cols={options.cols}, chain_length={options.chain_length}, parallel={options.parallel}, hardware_mapping={options.hardware_mapping}")
|
||||
|
||||
# On a Pi 5 the library hands back no matrix for settings its RP1
|
||||
# path can't drive, and the binding doesn't check -- the process
|
||||
# would crash on its next call instead of reaching the fallback
|
||||
# below. Raise first so it is a logged, reported init failure.
|
||||
if os.getenv("EMULATOR", "false") != "true" and is_raspberry_pi_5():
|
||||
unsupported = pi5_unsupported_settings(hardware_config)
|
||||
if unsupported:
|
||||
raise RuntimeError(unsupported)
|
||||
|
||||
# Initialize the matrix
|
||||
self.matrix = RGBMatrix(options=options)
|
||||
logger.info("RGB Matrix initialized successfully")
|
||||
@@ -379,7 +358,12 @@ class DisplayManager:
|
||||
|
||||
except Exception as e:
|
||||
_init_error_str = str(e)
|
||||
logger.error(f"Failed to initialize RGB Matrix: {e}", exc_info=True)
|
||||
if isinstance(e, MatrixSettingsRefused):
|
||||
_init_cause = "settings"
|
||||
logger.error("Failed to initialize RGB Matrix: %s", e)
|
||||
else:
|
||||
_init_cause = "forced" if getattr(self, '_force_fallback', False) else "library"
|
||||
logger.error(f"Failed to initialize RGB Matrix: {e}", exc_info=True)
|
||||
# Create a fallback image for web preview using configured dimensions when available
|
||||
self.matrix = None
|
||||
try:
|
||||
@@ -406,15 +390,18 @@ class DisplayManager:
|
||||
# Best-effort; ignore drawing errors in fallback
|
||||
pass
|
||||
logger.error(
|
||||
f"Matrix initialization failed — running in fallback/simulation mode "
|
||||
f"(size {fallback_width}x{fallback_height}). Error: {e}. "
|
||||
"On Raspberry Pi 5: ensure rpi-rgb-led-matrix was built from the latest "
|
||||
"submodule (re-run first_time_install.sh). gpio_slowdown of 2–3 is typical for Pi 5 PIO mode."
|
||||
)
|
||||
"Matrix initialization failed — running in fallback/simulation mode "
|
||||
"(size %dx%d). %s",
|
||||
fallback_width, fallback_height, self._fallback_advice(_init_cause, e))
|
||||
# Do not raise here; allow fallback mode so web preview and non-hardware environments work
|
||||
|
||||
# Write hardware status file so the web UI can surface init failures
|
||||
_hw_status = {"ok": self.matrix is not None, "error": _init_error_str}
|
||||
# cause: None when ok; "settings" when LEDMatrix refused the config
|
||||
# (fix the named settings), "library" when the library itself failed,
|
||||
# "forced" for a caller-requested fallback. The Display tab keys its
|
||||
# advice on it.
|
||||
_hw_status = {"ok": self.matrix is not None, "error": _init_error_str,
|
||||
"cause": None if self.matrix is not None else _init_cause}
|
||||
_status_path = "/tmp/led_matrix_hw_status.json" # nosec B108
|
||||
try:
|
||||
if os.path.islink(_status_path):
|
||||
@@ -682,8 +669,10 @@ class DisplayManager:
|
||||
def render_size(self, width: int, height: Optional[int] = None):
|
||||
"""Temporarily present a smaller logical canvas to plugins.
|
||||
|
||||
Plugins lay out against ``display_manager.matrix.width`` (and the
|
||||
``width``/``height`` properties, which defer to it), so the only way to
|
||||
Plugins lay out against the ``display_manager.width``/``height``
|
||||
properties (which defer to ``matrix.width`` when hardware is present,
|
||||
and to the canvas when it is not; some older plugins read
|
||||
``matrix.width`` directly), so the only way to
|
||||
get a *narrower layout* rather than a cropped one is to tell the plugin
|
||||
the screen is narrower while it renders. Trimming after the fact cannot
|
||||
fix a forecast spread across five columns or a progress bar drawn at
|
||||
@@ -1370,6 +1359,68 @@ class DisplayManager:
|
||||
|
||||
return dt.strftime(f"%b %-d{suffix}")
|
||||
|
||||
@classmethod
|
||||
def apply_matrix_options(cls, options, config: Dict[str, Any]):
|
||||
"""Fill ``options`` (an ``RGBMatrixOptions``) from the LEDMatrix config.
|
||||
|
||||
This is exactly what the display service drives the panel with, so a
|
||||
tool that opens the matrix itself (``scripts/scroll_speeds.py``) gets
|
||||
the same panel -- same runtime ``gpio_slowdown``, ``rp1_rio``,
|
||||
``panel_type``, orientation and defaults -- rather than a private copy
|
||||
that drifts. Does not open the matrix. Returns ``options``.
|
||||
"""
|
||||
display = config.get('display', {}) if isinstance(config, dict) else {}
|
||||
hardware_config = display.get('hardware', {})
|
||||
runtime_config = display.get('runtime', {})
|
||||
|
||||
# Basic hardware settings
|
||||
options.rows = hardware_config.get('rows', DEFAULT_ROWS)
|
||||
options.cols = hardware_config.get('cols', DEFAULT_COLS)
|
||||
options.chain_length = hardware_config.get('chain_length', DEFAULT_CHAIN_LENGTH)
|
||||
options.parallel = hardware_config.get('parallel', DEFAULT_PARALLEL)
|
||||
options.hardware_mapping = hardware_config.get('hardware_mapping', 'adafruit-hat-pwm')
|
||||
|
||||
# Performance and stability settings
|
||||
options.brightness = hardware_config.get('brightness', 90)
|
||||
options.pwm_bits = hardware_config.get('pwm_bits', 10)
|
||||
options.pwm_lsb_nanoseconds = hardware_config.get('pwm_lsb_nanoseconds', 150)
|
||||
options.led_rgb_sequence = hardware_config.get('led_rgb_sequence', 'RGB')
|
||||
# _build_pixel_mapper_config reads only class attributes, so the class
|
||||
# stands in for an instance here.
|
||||
options.pixel_mapper_config = cls._build_pixel_mapper_config(cls, hardware_config)
|
||||
options.row_address_type = hardware_config.get('row_address_type', 0)
|
||||
options.multiplexing = hardware_config.get('multiplexing', 0)
|
||||
options.panel_type = hardware_config.get('panel_type', '')
|
||||
options.disable_hardware_pulsing = hardware_config.get('disable_hardware_pulsing', False)
|
||||
options.show_refresh_rate = hardware_config.get('show_refresh_rate', False)
|
||||
options.limit_refresh_rate_hz = hardware_config.get('limit_refresh_rate_hz', 90)
|
||||
options.gpio_slowdown = runtime_config.get('gpio_slowdown', 3)
|
||||
|
||||
# Disable internal privilege dropping - we manage this via systemd or remain root
|
||||
# This prevents the library from dropping to 'daemon' user which breaks file permissions
|
||||
options.drop_privileges = False
|
||||
|
||||
# Additional settings from config
|
||||
if 'scan_mode' in hardware_config:
|
||||
options.scan_mode = hardware_config.get('scan_mode')
|
||||
if 'pwm_dither_bits' in hardware_config:
|
||||
options.pwm_dither_bits = hardware_config.get('pwm_dither_bits')
|
||||
if 'inverse_colors' in hardware_config:
|
||||
options.inverse_colors = hardware_config.get('inverse_colors')
|
||||
# Pi 5 only: 0=PIO/RP1 coprocessor (default, less CPU),
|
||||
# 1=RIO/Registered IO (faster; gpio_slowdown effect is inverted in this mode)
|
||||
if 'rp1_rio' in runtime_config:
|
||||
if hasattr(options, 'rp1_rio'):
|
||||
options.rp1_rio = runtime_config.get('rp1_rio')
|
||||
else:
|
||||
logger.warning(
|
||||
"rp1_rio is set in config but the installed rgbmatrix library does "
|
||||
"not support it — the library was likely built without Pi 5 RP1 "
|
||||
"support (mmap to 0x3f000000 instead of RP1 chip). "
|
||||
"Fix: sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh"
|
||||
)
|
||||
return options
|
||||
|
||||
@property
|
||||
def refresh_hz(self) -> float:
|
||||
"""The panel's refresh rate in Hz, from the hardware config.
|
||||
@@ -1415,6 +1466,12 @@ class DisplayManager:
|
||||
pixel every second refresh, which is how a scroll runs at half the
|
||||
refresh rate without fractional pixel positions.
|
||||
|
||||
The hold is part of the scroll's speed. A ScrollHelper configured by
|
||||
``scroll_config.configure()`` advances a fixed whole-pixel step per
|
||||
presented frame and reads no clock, so pass the returned
|
||||
``settings.frame_hold`` here: a scroll that leaves it at 1 is
|
||||
presented every refresh and runs ``frame_hold`` times too fast.
|
||||
|
||||
The hold is set here rather than once at plugin construction because
|
||||
it must not outlive the scroll that asked for it: plugins share one
|
||||
display manager, so a hold left set by whoever scrolled last would
|
||||
|
||||
@@ -0,0 +1,205 @@
|
||||
"""Display settings the pinned rgbmatrix library refuses, on every board.
|
||||
|
||||
The library does not raise on a setting it can't use. For most it returns no
|
||||
matrix -- ``RGBMatrix::CreateFromOptions()`` gives back NULL when
|
||||
``Options::Validate()`` (``lib/options-initialize.cc``) or its GPIO slowdown
|
||||
check (``lib/led-matrix.cc``) fails -- and the Python binding stores that
|
||||
pointer without checking, so the display process crashes on its first call
|
||||
into the matrix. For two it calls ``abort()``: an unknown hardware mapping
|
||||
name, and more parallel chains than the mapping has outputs
|
||||
(``Framebuffer::InitHardwareMapping()`` and the ``Framebuffer`` constructor in
|
||||
``lib/framebuffer.cc``). Either way systemd restarts the service into the same
|
||||
crash, and no fallback or hardware status is ever reported.
|
||||
|
||||
``DisplayManager`` runs :func:`library_refusals` before creating the matrix,
|
||||
turning those crashes into a logged error and the usual fallback mode. The
|
||||
config API uses the same rules to refuse the settings up front, and
|
||||
:data:`INT_SETTING_LIMITS` is the one place the numeric ranges live.
|
||||
|
||||
Pi 5-only limits come from :mod:`src.pi5_matrix_support` and are added when
|
||||
``pi5=True``.
|
||||
|
||||
**Re-check this when the submodule is bumped** (pinned at 1ee4f76): the
|
||||
ranges in ``Options::Validate()``, the mapping table in
|
||||
``lib/hardware-mapping.c`` and the setter types in
|
||||
``bindings/python/rgbmatrix/core.pyx``. A stale rule here blocks settings a
|
||||
new library accepts; a missing one lets the display service crash-loop.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, Mapping, NamedTuple, Optional, Tuple
|
||||
|
||||
from src.pi5_matrix_support import pi5_unsupported_settings
|
||||
|
||||
#: The binding's setters for rows, chain_length, parallel and the other small
|
||||
#: integers are declared ``uint8_t`` (``core.pyx``), cols ``uint32_t``, and
|
||||
#: limit_refresh_rate_hz lands in a C ``int``; a larger value raises
|
||||
#: ``OverflowError`` before the library sees it.
|
||||
UINT8_MAX = 255
|
||||
UINT32_MAX = 2 ** 32 - 1
|
||||
INT32_MAX = 2 ** 31 - 1
|
||||
|
||||
#: field -> (config section, lowest, highest, must be even). The library's own
|
||||
#: ranges where it has one, otherwise the binding's integer type.
|
||||
INT_SETTING_LIMITS: Dict[str, Tuple[str, int, int, bool]] = {
|
||||
'rows': ('hardware', 8, 64, True),
|
||||
'cols': ('hardware', 16, UINT32_MAX, False),
|
||||
'chain_length': ('hardware', 1, UINT8_MAX, False),
|
||||
# 3 is the most any mapping in the default build has (see MAPPING_OUTPUTS).
|
||||
'parallel': ('hardware', 1, 3, False),
|
||||
'brightness': ('hardware', 1, 100, False),
|
||||
'scan_mode': ('hardware', 0, 1, False),
|
||||
'pwm_bits': ('hardware', 1, 11, False),
|
||||
'pwm_dither_bits': ('hardware', 0, 2, False),
|
||||
'pwm_lsb_nanoseconds': ('hardware', 50, 3000, False),
|
||||
# 0 = no cap; the library has no upper bound.
|
||||
'limit_refresh_rate_hz': ('hardware', 0, INT32_MAX, False),
|
||||
'row_address_type': ('hardware', 0, 5, False),
|
||||
# 22 registered multiplexers (lib/multiplex-mappers.cc).
|
||||
'multiplexing': ('hardware', 0, 22, False),
|
||||
'gpio_slowdown': ('runtime', 0, 10, False),
|
||||
'rp1_rio': ('runtime', 0, 1, False),
|
||||
}
|
||||
|
||||
#: Hardware mapping name -> parallel chains it has outputs for, as
|
||||
#: ``lib/hardware-mapping.c`` defines them. ``compute-module`` exists only when
|
||||
#: the library is built with ENABLE_WIDE_GPIO_COMPUTE_MODULE, which the pinned
|
||||
#: Makefile leaves commented out and the installer does not set, so the library
|
||||
#: LEDMatrix installs aborts on it like any other unknown name.
|
||||
MAPPING_OUTPUTS: Dict[str, int] = {
|
||||
'regular': 3,
|
||||
'adafruit-hat': 1,
|
||||
'adafruit-hat-pwm': 1,
|
||||
'regular-pi1': 1,
|
||||
'classic': 3,
|
||||
'classic-pi1': 1,
|
||||
}
|
||||
|
||||
#: What DisplayManager passes when a key is missing from display.hardware /
|
||||
#: display.runtime. Config migration normally fills these from
|
||||
#: config/config.template.json first, so they rarely apply.
|
||||
DISPLAY_MANAGER_DEFAULTS: Dict[str, Any] = {
|
||||
'rows': 32, 'cols': 64, 'chain_length': 2, 'parallel': 1,
|
||||
'hardware_mapping': 'adafruit-hat-pwm', 'brightness': 90, 'pwm_bits': 10,
|
||||
'pwm_lsb_nanoseconds': 150, 'led_rgb_sequence': 'RGB',
|
||||
'row_address_type': 0, 'multiplexing': 0, 'limit_refresh_rate_hz': 90,
|
||||
'gpio_slowdown': 3,
|
||||
}
|
||||
|
||||
|
||||
class Refusal(NamedTuple):
|
||||
"""One reason the library can't start with a config.
|
||||
|
||||
``fields`` are the settings involved; the config API reports a refusal
|
||||
only when the request sets one of them, so a problem already stored
|
||||
doesn't block an unrelated save.
|
||||
"""
|
||||
fields: Tuple[str, ...]
|
||||
message: str
|
||||
#: A Raspberry Pi 5-only limit, whose message is already a full sentence.
|
||||
pi5: bool = False
|
||||
|
||||
|
||||
class MatrixSettingsRefused(RuntimeError):
|
||||
"""Raised by DisplayManager instead of handing the library such a config."""
|
||||
|
||||
|
||||
def _as_int(value: Any) -> Optional[int]:
|
||||
"""The integer the binding would receive, or None if it isn't one.
|
||||
|
||||
A value that isn't a number is left to the binding, which raises a
|
||||
``TypeError`` DisplayManager already turns into fallback mode.
|
||||
"""
|
||||
try:
|
||||
return int(value)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
return None
|
||||
|
||||
|
||||
def _setting(hardware: Mapping[str, Any], runtime: Mapping[str, Any], field: str) -> Any:
|
||||
section = runtime if INT_SETTING_LIMITS.get(field, ('hardware',))[0] == 'runtime' else hardware
|
||||
if field in section:
|
||||
return section[field]
|
||||
return DISPLAY_MANAGER_DEFAULTS.get(field)
|
||||
|
||||
|
||||
def describe_range(low: int, high: int, even: bool = False) -> str:
|
||||
kind = "an even integer" if even else "an integer"
|
||||
if high >= INT32_MAX:
|
||||
return f"{kind} of at least {low}"
|
||||
return f"{kind} from {low} to {high}"
|
||||
|
||||
|
||||
def library_refusals(hardware: Optional[Mapping[str, Any]],
|
||||
runtime: Optional[Mapping[str, Any]] = None,
|
||||
pi5: bool = False) -> List[Refusal]:
|
||||
"""Every reason the pinned library would refuse (NULL, abort or overflow)
|
||||
to start with this ``display.hardware`` / ``display.runtime`` config.
|
||||
|
||||
Missing keys take DisplayManager's defaults. ``pi5`` adds the Raspberry
|
||||
Pi 5 limits from :func:`pi5_unsupported_settings`.
|
||||
"""
|
||||
hardware = hardware if isinstance(hardware, Mapping) else {}
|
||||
runtime = runtime if isinstance(runtime, Mapping) else {}
|
||||
refusals: List[Refusal] = []
|
||||
|
||||
for field, (section, low, high, even) in INT_SETTING_LIMITS.items():
|
||||
# DisplayManager sets these only when present, and scan_mode /
|
||||
# pwm_dither_bits / rp1_rio have no default of its own.
|
||||
source = runtime if section == 'runtime' else hardware
|
||||
if field not in source and field not in DISPLAY_MANAGER_DEFAULTS:
|
||||
continue
|
||||
value = _as_int(_setting(hardware, runtime, field))
|
||||
if value is None:
|
||||
continue
|
||||
if not low <= value <= high or (even and value % 2):
|
||||
refusals.append(Refusal(
|
||||
(field,), f"{field} {value} (must be {describe_range(low, high, even)})"))
|
||||
|
||||
mapping = _setting(hardware, runtime, 'hardware_mapping')
|
||||
outputs = None
|
||||
if not isinstance(mapping, str):
|
||||
refusals.append(Refusal(
|
||||
('hardware_mapping',), f"hardware mapping {mapping!r} (must be a mapping name)"))
|
||||
else:
|
||||
# The library matches names case-insensitively and reads an empty
|
||||
# name as "regular".
|
||||
outputs = MAPPING_OUTPUTS.get((mapping or 'regular').lower())
|
||||
if outputs is None:
|
||||
refusals.append(Refusal(
|
||||
('hardware_mapping',),
|
||||
f'hardware mapping "{mapping}" (the installed library has '
|
||||
+ ", ".join(MAPPING_OUTPUTS) + ")"))
|
||||
|
||||
parallel = _as_int(_setting(hardware, runtime, 'parallel'))
|
||||
if outputs is not None and parallel is not None and 1 <= parallel <= 3 and parallel > outputs:
|
||||
refusals.append(Refusal(
|
||||
('parallel', 'hardware_mapping'),
|
||||
f'parallel {parallel} with hardware mapping "{mapping}", which has '
|
||||
f'{outputs} output{"s" if outputs != 1 else ""}'))
|
||||
|
||||
sequence = _setting(hardware, runtime, 'led_rgb_sequence')
|
||||
if isinstance(sequence, str) and not (
|
||||
len(sequence) == 3 and set(sequence.upper()) == {'R', 'G', 'B'}):
|
||||
refusals.append(Refusal(
|
||||
('led_rgb_sequence',),
|
||||
f'LED RGB sequence "{sequence}" (must be R, G and B in some order)'))
|
||||
|
||||
if pi5:
|
||||
unsupported = pi5_unsupported_settings(hardware)
|
||||
if unsupported:
|
||||
refusals.append(Refusal(
|
||||
('row_address_type', 'parallel', 'hardware_mapping'), unsupported, pi5=True))
|
||||
|
||||
return refusals
|
||||
|
||||
|
||||
def refusal_message(refusals: List[Refusal]) -> Optional[str]:
|
||||
"""One sentence naming every refusal, or None when there are none."""
|
||||
general = [r.message for r in refusals if not r.pi5]
|
||||
pi5 = [r.message for r in refusals if r.pi5]
|
||||
parts = []
|
||||
if general:
|
||||
parts.append("The installed rgbmatrix library can't start with these "
|
||||
"display settings: " + "; ".join(general) + ".")
|
||||
parts.extend(pi5)
|
||||
return " ".join(parts) or None
|
||||
@@ -22,7 +22,10 @@ from src.logging_config import get_logger
|
||||
from src.plugin_system.plugin_loader import PluginLoader
|
||||
from src.plugin_system.plugin_executor import PluginExecutor
|
||||
from src.plugin_system.plugin_state import PluginStateManager, PluginState
|
||||
from src.plugin_system.schema_manager import SchemaManager, normalize_legacy_booleans
|
||||
from src.plugin_system.schema_manager import (
|
||||
CORE_VEGAS_TUNING_KEYS, SchemaManager, normalize_legacy_booleans,
|
||||
)
|
||||
from src.common.path_safety import safe_path_component
|
||||
from src.common.permission_utils import (
|
||||
ensure_directory_permissions,
|
||||
get_plugin_dir_mode
|
||||
@@ -369,32 +372,11 @@ class PluginManager:
|
||||
f"The schema may be invalid. Please verify the schema file at: {schema_path}"
|
||||
)
|
||||
|
||||
# A plugin that turned an on/off boolean into an {enabled, ...}
|
||||
# object still finds the boolean in config.json until the user saves
|
||||
# its settings form, which carries it over (plugin_config.html).
|
||||
# Read it the same way here, before the defaults fill in the rest
|
||||
# of the object and before schema validation, so the plugin doesn't
|
||||
# start with a schema warning and a degraded flag. In memory only:
|
||||
# config.json is written by saves, never by loading a plugin.
|
||||
if schema:
|
||||
upgraded: List[str] = []
|
||||
config = normalize_legacy_booleans(config, schema, upgraded)
|
||||
if upgraded:
|
||||
self.logger.info(
|
||||
"Plugin %s: reading legacy boolean setting %s as "
|
||||
"{\"enabled\": ...}; saving the plugin's settings "
|
||||
"stores the new shape",
|
||||
plugin_id, ", ".join(upgraded),
|
||||
)
|
||||
|
||||
# Merge config with schema defaults to ensure all defaults are applied
|
||||
try:
|
||||
defaults = self.schema_manager.generate_default_config(plugin_id, use_cache=True)
|
||||
config = self.schema_manager.merge_with_defaults(config, defaults)
|
||||
self.logger.debug(f"Merged config with schema defaults for {plugin_id}")
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Could not apply schema defaults for {plugin_id}: {e}")
|
||||
# Continue with original config if defaults can't be applied
|
||||
# Legacy booleans read as objects, then schema defaults: the same
|
||||
# preparation saves, GET /plugins/config and hot reload apply
|
||||
# (prepare_plugin_config). In memory only: config.json is written
|
||||
# by saves, never by loading a plugin.
|
||||
config = self.prepare_plugin_config(plugin_id, config, schema=schema)
|
||||
|
||||
# Use PluginLoader to load plugin
|
||||
plugin_instance, module = self.plugin_loader.load_plugin(
|
||||
@@ -489,11 +471,57 @@ class PluginManager:
|
||||
#:
|
||||
#: Read by: ``vegas_mode/plugin_adapter.py`` (``vegas_width_pct``,
|
||||
#: ``vegas_overflow``) and ``base_plugin.py`` (``vegas_max_width_screens``).
|
||||
CORE_OWNED_CONFIG_KEYS = frozenset({
|
||||
'vegas_width_pct',
|
||||
'vegas_overflow',
|
||||
'vegas_max_width_screens',
|
||||
})
|
||||
#:
|
||||
#: The list itself lives with the other core-owned per-plugin properties in
|
||||
#: ``schema_manager.CORE_PLUGIN_PROPERTIES``, which the web save path also
|
||||
#: uses to keep these keys.
|
||||
CORE_OWNED_CONFIG_KEYS = CORE_VEGAS_TUNING_KEYS
|
||||
|
||||
def prepare_plugin_config(self, plugin_id: str, config: Any,
|
||||
schema: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
|
||||
"""The config a plugin runs with, built from its raw config.json section.
|
||||
|
||||
A plugin that turned an on/off boolean into an ``{enabled, ...}``
|
||||
object still finds the boolean in config.json until its settings are
|
||||
next saved; it is read as the object, and schema defaults fill in the
|
||||
rest (``SchemaManager.prepare_plugin_config``). Used when loading a
|
||||
plugin and on hot reload (DisplayController), so ``on_config_change``
|
||||
receives the same shape the plugin was constructed with.
|
||||
|
||||
Never raises: on failure the legacy-boolean pass alone is applied, or
|
||||
failing that the section is returned as it was.
|
||||
"""
|
||||
if schema is None:
|
||||
try:
|
||||
schema = self.schema_manager.load_schema(plugin_id)
|
||||
except Exception as e:
|
||||
self.logger.debug("Could not load schema for %s: %s", plugin_id, e)
|
||||
schema = None
|
||||
upgraded: List[str] = []
|
||||
try:
|
||||
prepared = self.schema_manager.prepare_plugin_config(
|
||||
plugin_id, config, schema=schema, changed_paths=upgraded)
|
||||
self.logger.debug("Merged config with schema defaults for %s", plugin_id)
|
||||
except Exception as e:
|
||||
self.logger.warning("Could not apply schema defaults for %s: %s", plugin_id, e)
|
||||
# Continue without defaults if they can't be applied
|
||||
upgraded = []
|
||||
prepared = config if isinstance(config, dict) else {}
|
||||
if schema:
|
||||
try:
|
||||
prepared = normalize_legacy_booleans(prepared, schema, upgraded)
|
||||
except Exception as legacy_error:
|
||||
self.logger.warning(
|
||||
"Could not read legacy boolean settings for %s: %s",
|
||||
plugin_id, legacy_error)
|
||||
if upgraded:
|
||||
self.logger.info(
|
||||
"Plugin %s: reading legacy boolean setting %s as "
|
||||
"{\"enabled\": ...}; saving the plugin's settings "
|
||||
"stores the new shape",
|
||||
plugin_id, ", ".join(upgraded),
|
||||
)
|
||||
return prepared
|
||||
|
||||
def _strip_core_owned_keys(self, config: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""A shallow copy of ``config`` without the core's own tuning keys.
|
||||
@@ -743,11 +771,20 @@ class PluginManager:
|
||||
|
||||
Returns:
|
||||
Directory path as string or None if not found
|
||||
|
||||
``plugin_id`` often comes straight from a request, so anything that is
|
||||
not one plain path segment (``..``, ``a/b``, an absolute path) is
|
||||
refused instead of being joined onto ``plugins_dir``. The join is not
|
||||
resolved further: dev plugins are symlinks into ``plugins_dir``.
|
||||
"""
|
||||
with self._discovery_lock:
|
||||
if hasattr(self, 'plugin_directories') and plugin_id in self.plugin_directories:
|
||||
return str(self.plugin_directories[plugin_id])
|
||||
|
||||
|
||||
plugin_id = safe_path_component(plugin_id)
|
||||
if plugin_id is None:
|
||||
return None
|
||||
|
||||
plugin_dir = self.plugins_dir / plugin_id
|
||||
if plugin_dir.exists():
|
||||
return str(plugin_dir)
|
||||
|
||||
+268
-188
@@ -13,6 +13,8 @@ from typing import Any, Dict, List, Optional, Tuple
|
||||
import jsonschema
|
||||
from jsonschema import Draft7Validator, ValidationError
|
||||
|
||||
from src.core_config_keys import CORE_CONFIG_KEYS
|
||||
|
||||
|
||||
def _renders_as_object(prop: Dict[str, Any]) -> bool:
|
||||
"""``field_type == 'object'`` as ``plugin_config.html`` computes it.
|
||||
@@ -93,6 +95,222 @@ def normalize_legacy_booleans(config: Any, schema: Any,
|
||||
return result
|
||||
|
||||
|
||||
#: Per-plugin settings the **core** owns: it reads them out of each plugin's
|
||||
#: config section, so they are allowed in every plugin's config whether or not
|
||||
#: the plugin's schema declares them. The one list for validation, for the web
|
||||
#: save filter and for the load-time checks -- a private copy is how JSON saves
|
||||
#: came to drop ``skin`` and the ``vegas_*`` keys while the validator accepted
|
||||
#: them.
|
||||
#:
|
||||
#: Values are the schema used when the plugin does not declare the property.
|
||||
CORE_PLUGIN_PROPERTIES: Dict[str, Dict[str, Any]] = {
|
||||
# Defaults match BasePlugin behavior: enabled=True, display_duration=15,
|
||||
# live_priority=False.
|
||||
"enabled": {
|
||||
"type": "boolean",
|
||||
"default": True,
|
||||
"description": "Enable or disable this plugin"
|
||||
},
|
||||
"display_duration": {
|
||||
"type": "number",
|
||||
"default": 15,
|
||||
"minimum": 1,
|
||||
"maximum": 300,
|
||||
"description": "How long to display this plugin in seconds"
|
||||
},
|
||||
"live_priority": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Enable live priority takeover when plugin has live content"
|
||||
},
|
||||
# Skin selection (docs/SKIN_SYSTEM.md). Deliberately NOT an enum here:
|
||||
# validation must keep passing when a configured skin gets uninstalled
|
||||
# (rendering falls back to built-in). The install-dependent enum is
|
||||
# injected only at serve time (inject_skin_selector) for the web UI
|
||||
# dropdown.
|
||||
"skin": {
|
||||
"type": ["string", "object", "null"],
|
||||
"description": "Visual skin id, or a per-mode mapping like {\"live\": \"my-skin\"}"
|
||||
},
|
||||
"skin_options": {
|
||||
"type": "object",
|
||||
"description": "Options passed through to the selected skin"
|
||||
},
|
||||
# Vegas tuning read by vegas_mode/plugin_adapter.py and base_plugin.py.
|
||||
# Left untyped: the adapter validates them itself and ignores a bad
|
||||
# value with a log line, so a stored one must never block a save.
|
||||
"vegas_width_pct": {
|
||||
"description": "Vegas mode: width of this plugin's card, as a percentage of the panel"
|
||||
},
|
||||
"vegas_overflow": {
|
||||
"description": "Vegas mode: 'rotate' or 'truncate' when this plugin's content overflows"
|
||||
},
|
||||
"vegas_max_width_screens": {
|
||||
"description": "Vegas mode: widest this plugin's card may be, in screens"
|
||||
},
|
||||
}
|
||||
|
||||
#: The keys of CORE_PLUGIN_PROPERTIES that are Vegas tuning rather than plugin
|
||||
#: state. PluginManager strips these before its soft validation (see
|
||||
#: PluginManager.CORE_OWNED_CONFIG_KEYS).
|
||||
CORE_VEGAS_TUNING_KEYS = frozenset({
|
||||
'vegas_width_pct', 'vegas_overflow', 'vegas_max_width_screens',
|
||||
})
|
||||
|
||||
|
||||
def with_core_plugin_properties(schema: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""A deep copy of a plugin schema with CORE_PLUGIN_PROPERTIES allowed.
|
||||
|
||||
Properties the plugin declares itself are left as declared. Core
|
||||
properties are removed from ``required``: they are system-managed.
|
||||
"""
|
||||
enhanced = copy.deepcopy(schema) if isinstance(schema, dict) else {}
|
||||
properties = enhanced.setdefault("properties", {})
|
||||
for name, definition in CORE_PLUGIN_PROPERTIES.items():
|
||||
if name not in properties:
|
||||
properties[name] = copy.deepcopy(definition)
|
||||
if "required" in enhanced:
|
||||
enhanced["required"] = [field for field in enhanced["required"]
|
||||
if field not in CORE_PLUGIN_PROPERTIES]
|
||||
return enhanced
|
||||
|
||||
|
||||
def extract_schema_defaults(schema: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Default values of a JSON Schema's properties, recursively.
|
||||
|
||||
A property's own ``default`` wins; otherwise a nested object contributes
|
||||
its children's defaults, and an array contributes ``[]`` (or a one-item
|
||||
list of its ``items`` default). This is what a device runs with, so the
|
||||
dev tools use it too (src/plugin_system/testing/loading.py).
|
||||
"""
|
||||
defaults: Dict[str, Any] = {}
|
||||
properties = schema.get('properties', {}) if isinstance(schema, dict) else {}
|
||||
if not isinstance(properties, dict):
|
||||
return defaults
|
||||
|
||||
for key, prop_schema in properties.items():
|
||||
if not isinstance(prop_schema, dict):
|
||||
continue
|
||||
# If property has a default, use it
|
||||
if 'default' in prop_schema:
|
||||
defaults[key] = prop_schema['default']
|
||||
continue
|
||||
|
||||
# Handle nested objects
|
||||
if prop_schema.get('type') == 'object' and 'properties' in prop_schema:
|
||||
nested_defaults = extract_schema_defaults(prop_schema)
|
||||
if nested_defaults:
|
||||
defaults[key] = nested_defaults
|
||||
|
||||
# Handle arrays with object items
|
||||
elif prop_schema.get('type') == 'array' and 'items' in prop_schema:
|
||||
items_schema = prop_schema['items']
|
||||
if items_schema.get('type') == 'object' and 'properties' in items_schema:
|
||||
# For arrays of objects, use empty array as default
|
||||
# Individual objects will use their defaults when created
|
||||
defaults[key] = []
|
||||
elif 'default' in items_schema:
|
||||
# Array with default item value
|
||||
defaults[key] = [items_schema['default']]
|
||||
else:
|
||||
# Empty array as default
|
||||
defaults[key] = []
|
||||
|
||||
# For other types without defaults, don't add to defaults dict
|
||||
# This allows plugins to handle missing values as needed
|
||||
|
||||
return defaults
|
||||
|
||||
|
||||
def plugin_config_defaults(schema: Optional[Dict[str, Any]]) -> Dict[str, Any]:
|
||||
"""Every default a plugin's config gets: the schema's plus the core ones.
|
||||
|
||||
A plugin with no schema gets the minimal ``enabled: False,
|
||||
display_duration: 15``. Device location is not applied here; that needs a
|
||||
config manager (SchemaManager.generate_default_config).
|
||||
"""
|
||||
if not schema:
|
||||
return {
|
||||
'enabled': False,
|
||||
'display_duration': 15
|
||||
}
|
||||
|
||||
defaults = extract_schema_defaults(schema)
|
||||
|
||||
# Ensure core properties have defaults (they may not be in the schema)
|
||||
# These match BasePlugin behavior
|
||||
for name in ('enabled', 'display_duration', 'live_priority'):
|
||||
if name not in defaults:
|
||||
defaults[name] = CORE_PLUGIN_PROPERTIES[name]['default']
|
||||
return defaults
|
||||
|
||||
|
||||
def merge_config_defaults(config: Dict[str, Any], defaults: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Merge configuration with defaults, preserving user values.
|
||||
|
||||
Also replaces None values with defaults so a config never starts with
|
||||
None where a default exists. Neither argument is mutated.
|
||||
"""
|
||||
merged = copy.deepcopy(defaults)
|
||||
|
||||
def deep_merge(target: Dict[str, Any], source: Dict[str, Any], default_dict: Dict[str, Any]) -> None:
|
||||
"""Recursively merge source into target, replacing None with defaults."""
|
||||
for key, value in source.items():
|
||||
default_value = default_dict.get(key)
|
||||
|
||||
if key in target and isinstance(target[key], dict) and isinstance(value, dict):
|
||||
# Both are dicts, recursively merge
|
||||
if isinstance(default_value, dict):
|
||||
deep_merge(target[key], value, default_value)
|
||||
else:
|
||||
deep_merge(target[key], value, {})
|
||||
elif value is None and default_value is not None:
|
||||
# Value is None and we have a default, use the default
|
||||
target[key] = copy.deepcopy(default_value) if isinstance(default_value, (dict, list)) else default_value
|
||||
else:
|
||||
# Normal merge: user value takes precedence (copy if dict/list)
|
||||
if isinstance(value, (dict, list)):
|
||||
target[key] = copy.deepcopy(value)
|
||||
else:
|
||||
target[key] = value
|
||||
|
||||
deep_merge(merged, config, defaults)
|
||||
|
||||
# Final pass: replace any remaining None values at any level with defaults
|
||||
def replace_none_with_defaults(target: Dict[str, Any], default_dict: Dict[str, Any]) -> None:
|
||||
"""Recursively replace None values with defaults."""
|
||||
for key in list(target.keys()):
|
||||
value = target[key]
|
||||
default_value = default_dict.get(key)
|
||||
|
||||
if value is None and default_value is not None:
|
||||
# Replace None with default
|
||||
target[key] = copy.deepcopy(default_value) if isinstance(default_value, (dict, list)) else default_value
|
||||
elif isinstance(value, dict) and isinstance(default_value, dict):
|
||||
# Recursively process nested dicts
|
||||
replace_none_with_defaults(value, default_value)
|
||||
|
||||
replace_none_with_defaults(merged, defaults)
|
||||
return merged
|
||||
|
||||
|
||||
def prepare_plugin_config(config: Any, schema: Optional[Dict[str, Any]],
|
||||
defaults: Dict[str, Any],
|
||||
changed_paths: Optional[List[str]] = None) -> Dict[str, Any]:
|
||||
"""The config a plugin runs with, from its stored (or submitted) section.
|
||||
|
||||
Legacy booleans are read as ``{"enabled": ...}`` objects
|
||||
(normalize_legacy_booleans), then schema defaults fill in whatever is
|
||||
missing. Loading a plugin, both config saves, GET /plugins/config, hot
|
||||
reload and the dev tools all go through this, so a plugin sees the same
|
||||
shape however its config reached it.
|
||||
"""
|
||||
config = config if isinstance(config, dict) else {}
|
||||
if schema:
|
||||
config = normalize_legacy_booleans(config, schema, changed_paths)
|
||||
return merge_config_defaults(config, defaults)
|
||||
|
||||
|
||||
class SchemaManager:
|
||||
"""
|
||||
Manages plugin configuration schemas with caching and validation.
|
||||
@@ -262,57 +480,12 @@ class SchemaManager:
|
||||
def extract_defaults_from_schema(self, schema: Dict[str, Any], prefix: str = '') -> Dict[str, Any]:
|
||||
"""
|
||||
Recursively extract default values from a JSON Schema.
|
||||
|
||||
Handles nested objects, arrays, and all schema types.
|
||||
|
||||
Args:
|
||||
schema: JSON Schema dictionary
|
||||
prefix: Optional prefix for logging/debugging
|
||||
|
||||
Returns:
|
||||
Dictionary of default values
|
||||
|
||||
See :func:`extract_schema_defaults`; ``prefix`` is accepted for
|
||||
compatibility and unused.
|
||||
"""
|
||||
defaults = {}
|
||||
|
||||
# Handle schema with properties
|
||||
properties = schema.get('properties', {})
|
||||
if not properties:
|
||||
return defaults
|
||||
|
||||
for key, prop_schema in properties.items():
|
||||
field_path = f"{prefix}.{key}" if prefix else key
|
||||
|
||||
# If property has a default, use it
|
||||
if 'default' in prop_schema:
|
||||
defaults[key] = prop_schema['default']
|
||||
self.logger.debug(f"Found default for {field_path}: {prop_schema['default']}")
|
||||
continue
|
||||
|
||||
# Handle nested objects
|
||||
if prop_schema.get('type') == 'object' and 'properties' in prop_schema:
|
||||
nested_defaults = self.extract_defaults_from_schema(prop_schema, field_path)
|
||||
if nested_defaults:
|
||||
defaults[key] = nested_defaults
|
||||
|
||||
# Handle arrays with object items
|
||||
elif prop_schema.get('type') == 'array' and 'items' in prop_schema:
|
||||
items_schema = prop_schema['items']
|
||||
if items_schema.get('type') == 'object' and 'properties' in items_schema:
|
||||
# For arrays of objects, use empty array as default
|
||||
# Individual objects will use their defaults when created
|
||||
defaults[key] = []
|
||||
elif 'default' in items_schema:
|
||||
# Array with default item value
|
||||
defaults[key] = [items_schema['default']]
|
||||
else:
|
||||
# Empty array as default
|
||||
defaults[key] = []
|
||||
|
||||
# For other types without defaults, don't add to defaults dict
|
||||
# This allows plugins to handle missing values as needed
|
||||
|
||||
return defaults
|
||||
|
||||
return extract_schema_defaults(schema)
|
||||
|
||||
def get_device_location(self) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Return the device-wide ``location`` block from config.json, or None.
|
||||
@@ -391,31 +564,39 @@ class SchemaManager:
|
||||
schema = self.load_schema(plugin_id, use_cache=use_cache)
|
||||
if not schema:
|
||||
# Return minimal defaults if no schema
|
||||
return {
|
||||
'enabled': False,
|
||||
'display_duration': 15
|
||||
}
|
||||
|
||||
# Extract defaults from schema
|
||||
defaults = self.extract_defaults_from_schema(schema)
|
||||
|
||||
# Ensure core properties have defaults (they may not be in the schema)
|
||||
# These match BasePlugin behavior
|
||||
if 'enabled' not in defaults:
|
||||
defaults['enabled'] = schema.get('properties', {}).get('enabled', {}).get('default', True)
|
||||
|
||||
if 'display_duration' not in defaults:
|
||||
defaults['display_duration'] = schema.get('properties', {}).get('display_duration', {}).get('default', 15)
|
||||
|
||||
if 'live_priority' not in defaults:
|
||||
defaults['live_priority'] = schema.get('properties', {}).get('live_priority', {}).get('default', False)
|
||||
|
||||
return plugin_config_defaults(None)
|
||||
|
||||
# Schema defaults plus the core properties' (they may not be in the
|
||||
# schema)
|
||||
defaults = plugin_config_defaults(schema)
|
||||
|
||||
# Cache the defaults *before* the device location is layered on, so a
|
||||
# later change to the device location is picked up by the next call.
|
||||
self._defaults_cache[plugin_id] = defaults.copy()
|
||||
|
||||
return self.apply_device_location(defaults)
|
||||
|
||||
def prepare_plugin_config(self, plugin_id: str, config: Any,
|
||||
schema: Optional[Dict[str, Any]] = None,
|
||||
changed_paths: Optional[List[str]] = None) -> Dict[str, Any]:
|
||||
"""
|
||||
The config a plugin runs with: see :func:`prepare_plugin_config`.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
config: The plugin's stored or submitted config section
|
||||
schema: The plugin's schema, when the caller already has it
|
||||
changed_paths: Receives the dotted path of each legacy boolean
|
||||
read as an object
|
||||
|
||||
Returns:
|
||||
A new dict; ``config`` is not mutated
|
||||
"""
|
||||
if schema is None:
|
||||
schema = self.load_schema(plugin_id, use_cache=True)
|
||||
defaults = self.generate_default_config(plugin_id, use_cache=True)
|
||||
return prepare_plugin_config(config, schema, defaults, changed_paths)
|
||||
|
||||
def validate_config_against_schema(self, config: Dict[str, Any], schema: Dict[str, Any],
|
||||
plugin_id: Optional[str] = None) -> Tuple[bool, List[str]]:
|
||||
"""
|
||||
@@ -436,80 +617,18 @@ class SchemaManager:
|
||||
errors = []
|
||||
|
||||
try:
|
||||
# Core plugin properties that should always be allowed
|
||||
# These are handled by the base plugin system and should not cause validation failures
|
||||
# Defaults match BasePlugin behavior: enabled=True, display_duration=15, live_priority=False
|
||||
core_properties = {
|
||||
"enabled": {
|
||||
"type": "boolean",
|
||||
"default": True,
|
||||
"description": "Enable or disable this plugin"
|
||||
},
|
||||
"display_duration": {
|
||||
"type": "number",
|
||||
"default": 15,
|
||||
"minimum": 1,
|
||||
"maximum": 300,
|
||||
"description": "How long to display this plugin in seconds"
|
||||
},
|
||||
"live_priority": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Enable live priority takeover when plugin has live content"
|
||||
},
|
||||
# Skin selection (docs/SKIN_SYSTEM.md). Deliberately NOT an
|
||||
# enum here: validation must keep passing when a configured
|
||||
# skin gets uninstalled (rendering falls back to built-in).
|
||||
# The install-dependent enum is injected only at serve time
|
||||
# (inject_skin_selector) for the web UI dropdown.
|
||||
"skin": {
|
||||
"type": ["string", "object", "null"],
|
||||
"description": "Visual skin id, or a per-mode mapping like {\"live\": \"my-skin\"}"
|
||||
},
|
||||
"skin_options": {
|
||||
"type": "object",
|
||||
"description": "Options passed through to the selected skin"
|
||||
}
|
||||
}
|
||||
|
||||
# Create a deep copy of the schema to modify (to avoid mutating the original)
|
||||
enhanced_schema = copy.deepcopy(schema)
|
||||
if "properties" not in enhanced_schema:
|
||||
enhanced_schema["properties"] = {}
|
||||
|
||||
# Inject core properties if they're not already defined in the schema
|
||||
# This ensures core properties are always allowed even if not in the plugin's schema
|
||||
properties_added = []
|
||||
for prop_name, prop_def in core_properties.items():
|
||||
if prop_name not in enhanced_schema["properties"]:
|
||||
enhanced_schema["properties"][prop_name] = copy.deepcopy(prop_def)
|
||||
properties_added.append(prop_name)
|
||||
|
||||
# Log if we added any core properties (for debugging)
|
||||
if properties_added and plugin_id:
|
||||
# Core plugin properties (CORE_PLUGIN_PROPERTIES) are handled by
|
||||
# the base plugin system and should not cause validation failures:
|
||||
# they are allowed even when the plugin's schema doesn't declare
|
||||
# them, and never required.
|
||||
enhanced_schema = with_core_plugin_properties(schema)
|
||||
if plugin_id:
|
||||
declared = schema.get("properties", {}) if isinstance(schema, dict) else {}
|
||||
self.logger.debug(
|
||||
f"Injected core properties into schema for {plugin_id}: {properties_added}"
|
||||
"Injected core properties into schema for %s: %s", plugin_id,
|
||||
[name for name in CORE_PLUGIN_PROPERTIES if name not in declared]
|
||||
)
|
||||
|
||||
# Remove core properties from required array (they're system-managed)
|
||||
# Core properties should be allowed but not required for validation
|
||||
if "required" in enhanced_schema:
|
||||
core_prop_names = list(core_properties.keys())
|
||||
removed_from_required = [
|
||||
field for field in enhanced_schema["required"]
|
||||
if field in core_prop_names
|
||||
]
|
||||
enhanced_schema["required"] = [
|
||||
field for field in enhanced_schema["required"]
|
||||
if field not in core_prop_names
|
||||
]
|
||||
|
||||
# Log if we removed any core properties from required (for debugging)
|
||||
if removed_from_required and plugin_id:
|
||||
self.logger.debug(
|
||||
f"Removed core properties from required array for {plugin_id}: {removed_from_required}"
|
||||
)
|
||||
|
||||
|
||||
# Create validator with enhanced schema
|
||||
validator = Draft7Validator(enhanced_schema)
|
||||
|
||||
@@ -626,55 +745,15 @@ class SchemaManager:
|
||||
"""
|
||||
Merge configuration with defaults, preserving user values.
|
||||
Also replaces None values with defaults to ensure config never has None from the start.
|
||||
|
||||
|
||||
Args:
|
||||
config: User configuration
|
||||
defaults: Default values from schema
|
||||
|
||||
|
||||
Returns:
|
||||
Merged configuration with defaults applied where missing or None
|
||||
"""
|
||||
merged = copy.deepcopy(defaults)
|
||||
|
||||
def deep_merge(target: Dict[str, Any], source: Dict[str, Any], default_dict: Dict[str, Any]) -> None:
|
||||
"""Recursively merge source into target, replacing None with defaults."""
|
||||
for key, value in source.items():
|
||||
default_value = default_dict.get(key)
|
||||
|
||||
if key in target and isinstance(target[key], dict) and isinstance(value, dict):
|
||||
# Both are dicts, recursively merge
|
||||
if isinstance(default_value, dict):
|
||||
deep_merge(target[key], value, default_value)
|
||||
else:
|
||||
deep_merge(target[key], value, {})
|
||||
elif value is None and default_value is not None:
|
||||
# Value is None and we have a default, use the default
|
||||
target[key] = copy.deepcopy(default_value) if isinstance(default_value, (dict, list)) else default_value
|
||||
else:
|
||||
# Normal merge: user value takes precedence (copy if dict/list)
|
||||
if isinstance(value, (dict, list)):
|
||||
target[key] = copy.deepcopy(value)
|
||||
else:
|
||||
target[key] = value
|
||||
|
||||
deep_merge(merged, config, defaults)
|
||||
|
||||
# Final pass: replace any remaining None values at any level with defaults
|
||||
def replace_none_with_defaults(target: Dict[str, Any], default_dict: Dict[str, Any]) -> None:
|
||||
"""Recursively replace None values with defaults."""
|
||||
for key in list(target.keys()):
|
||||
value = target[key]
|
||||
default_value = default_dict.get(key)
|
||||
|
||||
if value is None and default_value is not None:
|
||||
# Replace None with default
|
||||
target[key] = copy.deepcopy(default_value) if isinstance(default_value, (dict, list)) else default_value
|
||||
elif isinstance(value, dict) and isinstance(default_value, dict):
|
||||
# Recursively process nested dicts
|
||||
replace_none_with_defaults(value, default_value)
|
||||
|
||||
replace_none_with_defaults(merged, defaults)
|
||||
return merged
|
||||
return merge_config_defaults(config, defaults)
|
||||
|
||||
def detect_config_key_collisions(
|
||||
self,
|
||||
@@ -698,10 +777,11 @@ class SchemaManager:
|
||||
"""
|
||||
collisions = []
|
||||
|
||||
# Reserved top-level config keys that plugins should not use as IDs
|
||||
reserved_keys = {
|
||||
'display', 'schedule', 'timezone', 'plugin_system',
|
||||
'display_modes', 'system', 'hardware', 'debug',
|
||||
# Reserved top-level config keys that plugins should not use as IDs:
|
||||
# every core section (src/core_config_keys.py), plus a few names that
|
||||
# read as core even though no current section uses them.
|
||||
reserved_keys = set(CORE_CONFIG_KEYS) | {
|
||||
'display_modes', 'hardware', 'debug',
|
||||
'log_level', 'emulator', 'web_interface'
|
||||
}
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ from PIL import Image, ImageChops
|
||||
|
||||
from src.logging_config import get_logger
|
||||
from .bounds_display_manager import BoundsCheckingDisplayManager
|
||||
from .loading import load_config_defaults, load_manifest, merge_config
|
||||
from .loading import build_config, load_manifest
|
||||
from .sizes import DEFAULT_TEST_SIZES, safe_mode_filename, size_label
|
||||
|
||||
logger = get_logger("[Plugin Harness]")
|
||||
@@ -305,9 +305,7 @@ def render_plugin_matrix(
|
||||
manifest = load_manifest(plugin_dir)
|
||||
# Start from config_schema.json defaults so the plugin behaves like a real
|
||||
# install; explicit caller config still wins over a schema default.
|
||||
config = merge_config(
|
||||
merge_config({"enabled": True}, load_config_defaults(plugin_dir)),
|
||||
config or {})
|
||||
config = build_config(plugin_dir, config)
|
||||
sizes = sizes or DEFAULT_TEST_SIZES
|
||||
results: List[RenderResult] = []
|
||||
|
||||
|
||||
@@ -42,29 +42,6 @@ def load_manifest(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def _defaults_from_properties(properties: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Defaults for one `properties` block, recursing into nested objects.
|
||||
|
||||
An object property carries its defaults on its children, not on itself, so
|
||||
reading only the top level dropped everything nested. That is most of the
|
||||
fleet: config organised by league, or under customization/display_options,
|
||||
lost 2,386 defaults across 37 of 44 plugins -- soccer-scoreboard alone lost
|
||||
539 of 565 -- and the harness rendered them with a config no install would
|
||||
ever have.
|
||||
"""
|
||||
defaults: Dict[str, Any] = {}
|
||||
for key, prop in (properties or {}).items():
|
||||
if not isinstance(prop, dict):
|
||||
continue
|
||||
if prop.get('type') == 'object' and isinstance(prop.get('properties'), dict):
|
||||
nested = _defaults_from_properties(prop['properties'])
|
||||
if nested:
|
||||
defaults[key] = nested
|
||||
elif 'default' in prop:
|
||||
defaults[key] = prop['default']
|
||||
return defaults
|
||||
|
||||
|
||||
def merge_config(base: Dict[str, Any], override: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Deep-merge override onto base, without dropping sibling defaults.
|
||||
|
||||
@@ -81,14 +58,45 @@ def merge_config(base: Dict[str, Any], override: Dict[str, Any]) -> Dict[str, An
|
||||
return merged
|
||||
|
||||
|
||||
def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
|
||||
"""Extract default values from a plugin's config_schema.json (empty if none)."""
|
||||
def load_schema(plugin_dir: Union[str, Path]) -> Optional[Dict[str, Any]]:
|
||||
"""A plugin's config_schema.json, or None when it has none."""
|
||||
schema_path = Path(plugin_dir) / 'config_schema.json'
|
||||
if not schema_path.exists():
|
||||
return {}
|
||||
return None
|
||||
with open(schema_path, 'r', encoding='utf-8') as f:
|
||||
schema = json.load(f)
|
||||
return _defaults_from_properties(schema.get('properties', {}))
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def load_config_defaults(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
|
||||
"""Default values from a plugin's config_schema.json (empty if none).
|
||||
|
||||
The device's own extraction (schema_manager.extract_schema_defaults), so a
|
||||
harness run starts from the config an install would have: nested objects
|
||||
contribute their children's defaults (organised-by-league configs lost
|
||||
thousands of them when only the top level was read), and arrays without a
|
||||
default start as [].
|
||||
"""
|
||||
from src.plugin_system.schema_manager import extract_schema_defaults
|
||||
schema = load_schema(plugin_dir)
|
||||
return extract_schema_defaults(schema) if schema else {}
|
||||
|
||||
|
||||
def build_config(plugin_dir: Union[str, Path],
|
||||
overrides: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
|
||||
"""The config a device would give this plugin, with ``overrides`` applied.
|
||||
|
||||
Starts from a forced ``enabled: True``, deep-merges ``overrides`` onto it
|
||||
(merge_config), then prepares the result exactly as the device does when it
|
||||
loads a plugin: legacy booleans read as objects, and schema plus core
|
||||
defaults filled in (schema_manager.prepare_plugin_config). Used by the
|
||||
harness, check_plugin, render_plugin and the dev preview server.
|
||||
"""
|
||||
from src.plugin_system.schema_manager import (
|
||||
plugin_config_defaults, prepare_plugin_config,
|
||||
)
|
||||
schema = load_schema(plugin_dir)
|
||||
requested = merge_config({"enabled": True}, overrides or {})
|
||||
return prepare_plugin_config(requested, schema, plugin_config_defaults(schema))
|
||||
|
||||
|
||||
def load_harness_spec(plugin_dir: Union[str, Path]) -> Dict[str, Any]:
|
||||
@@ -143,16 +151,14 @@ def build_full_config(
|
||||
|
||||
Merge order: config_schema.json defaults, then a forced ``enabled: True``,
|
||||
then harness.json's config overlay, then the caller's explicit config --
|
||||
most specific wins. `enabled` is re-asserted *after* the schema defaults
|
||||
so a plugin that reasonably ships `enabled: false` (e.g. a seasonal or
|
||||
opt-in plugin) can't silently make every harness run test "disabled, do
|
||||
nothing" by accident -- callers that genuinely want to test the disabled
|
||||
path can still do so via `cli_config={"enabled": False}`.
|
||||
most specific wins, and each layer deep-merges (a nested override such as
|
||||
``{"nhl": {"enabled": true}}`` keeps the other nhl defaults). `enabled` is
|
||||
asserted over the schema defaults so a plugin that reasonably ships
|
||||
`enabled: false` (e.g. a seasonal or opt-in plugin) can't silently make
|
||||
every harness run test "disabled, do nothing" by accident -- callers that
|
||||
genuinely want to test the disabled path can still do so via
|
||||
`cli_config={"enabled": False}`. See build_config.
|
||||
"""
|
||||
spec = spec or {}
|
||||
config: Dict[str, Any] = {}
|
||||
config.update(load_config_defaults(plugin_dir))
|
||||
config["enabled"] = True
|
||||
config.update(spec.get("config", {}))
|
||||
config.update(cli_config or {})
|
||||
return config
|
||||
overrides = merge_config(spec.get("config", {}) or {}, cli_config or {})
|
||||
return build_config(plugin_dir, overrides)
|
||||
|
||||
@@ -8,6 +8,7 @@ Fails fast with clear error messages to prevent runtime issues.
|
||||
import os
|
||||
from typing import Any, List, Optional, Tuple
|
||||
from pathlib import Path
|
||||
from src.core_config_keys import CORE_CONFIG_KEYS
|
||||
from src.exceptions import ConfigError, PluginError, CacheError
|
||||
from src.logging_config import get_logger
|
||||
|
||||
@@ -275,8 +276,9 @@ class StartupValidator:
|
||||
|
||||
# Check for enabled plugins that don't exist
|
||||
for plugin_id, plugin_config in config.items():
|
||||
# Skip non-plugin config sections
|
||||
if plugin_id in ['display', 'schedule', 'timezone', 'plugin_system']:
|
||||
# Skip core sections: auto_update and dim_schedule have an
|
||||
# 'enabled' key too, and are not plugins that went missing.
|
||||
if plugin_id in CORE_CONFIG_KEYS:
|
||||
continue
|
||||
|
||||
if not isinstance(plugin_config, dict):
|
||||
|
||||
@@ -155,9 +155,14 @@ class VegasModeConfig:
|
||||
target_fps: int = 125 # Target frame rate
|
||||
buffer_ahead: int = 2 # Number of plugins to buffer ahead
|
||||
|
||||
# Scroll behavior
|
||||
# Scroll behavior. Neither key steps the scroll or sets a frame rate:
|
||||
# motion is always by elapsed time at scroll_speed px/s. With
|
||||
# frame_based_scrolling the speed is first converted to px per
|
||||
# scroll_delay and clamped to 0.1-5 (ScrollHelper.set_scroll_speed), so
|
||||
# the speed actually applied is clamp(scroll_speed * scroll_delay, 0.1, 5)
|
||||
# / scroll_delay -- at the 0.02 default, speeds under 5 px/s run at 5.
|
||||
frame_based_scrolling: bool = True
|
||||
scroll_delay: float = 0.02 # 50 FPS effective scroll updates
|
||||
scroll_delay: float = 0.02 # only feeds the clamp above; not a frame period
|
||||
|
||||
# Dynamic duration
|
||||
dynamic_duration_enabled: bool = True
|
||||
|
||||
@@ -127,12 +127,14 @@ class RenderPipeline:
|
||||
self.scroll_helper.set_sub_pixel_scrolling(self.config.smooth_scroll)
|
||||
|
||||
# Config scroll_speed is always pixels per second, but ScrollHelper
|
||||
# interprets it differently based on frame_based_scrolling mode:
|
||||
# - Frame-based: pixels per frame step
|
||||
# - Time-based: pixels per second
|
||||
# takes it in different units depending on frame_based_scrolling:
|
||||
# - Frame-based: pixels per scroll_delay seconds (clamped to 0.1-5)
|
||||
# - Time-based: pixels per second (clamped to 1-500)
|
||||
# Both modes then advance by elapsed time; frame-based mode does not
|
||||
# step. So frame-based with scroll_delay only adds the clamp: the
|
||||
# applied speed is clamp(scroll_speed * scroll_delay, 0.1, 5) /
|
||||
# scroll_delay px/s.
|
||||
if self.config.frame_based_scrolling:
|
||||
# Convert pixels/second to pixels/frame
|
||||
# pixels_per_frame = pixels_per_second * seconds_per_frame
|
||||
pixels_per_frame = self.config.scroll_speed * self.config.scroll_delay
|
||||
self.scroll_helper.set_scroll_speed(pixels_per_frame)
|
||||
else:
|
||||
|
||||
Reference in New Issue
Block a user