mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 06:15:09 +00:00
* fix(sports): share the ESPN rejected-range memo with the background service BackgroundDataService always sent a season range first and, on a 400, fell back to chunks without recording the rejection, so every background season fetch spent a doomed request and live scoreboards learned nothing from it (or it from them). The worker now consults and sets the same 6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight to month/day chunks, and if every chunk fails the range is asked once for a real error without re-spending the chunks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): keep plugin asset and action routes inside their directories POST /plugins/assets/upload, GET /plugins/assets/list and POST /plugins/assets/delete joined the request's plugin_id onto assets/plugins unchecked, so '../../config' created, wrote, listed and deleted outside it. #561 guarded only the route that serves the files. All three now go through path_safety.resolve_under and answer 400 for anything but a plain name, and delete only unlinks a metadata path that resolves into that plugin's uploads directory. PluginManager.get_plugin_directory refuses ids that are not one plain path segment, so /plugins/action (which runs a manifest script from the returned directory) and every other caller get the guard; the action route also rejects such ids up front, covering its no-manager fallback. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): report a no-op plugin update as already up to date update_plugin() returns True both for a real update and for "nothing to do" (a ZIP-installed monorepo plugin already at the registry version, a bundled plugin). With no git commit to compare, POST /plugins/update called every such success "updated successfully", so Check & Update All counted most official plugins as updated on every run. The route now reads what changed off the plugin itself (commit, else manifest version, else last_updated) and returns data.update_status (updated / up_to_date / local_only). The update-all toast is summarised by PluginInstallManager.summarizeUpdateResults from that status, falling back to the message for older servers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(sports): scoreboard scroll speed no longer follows target_fps sports_scroll computed the crisp speed ladder against the global target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since frame-locked presentation (#545) the helper steps a fixed number of whole pixels per presented frame and the panel presents at its real refresh, so the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a 50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s. The ladder now uses the display manager's refresh_hz, then display.hardware.limit_refresh_rate_hz, then the default. target_fps is not consulted. Docstrings now say scroll_delay is ignored for pacing (no behaviour change there) and describe the fixed-step model. Tests: replace the tests that pinned target_fps as the ladder refresh and described time-based stepping; assert speed independence from target_fps (unit and end-to-end presented px/s against the real helper), that the fixed per-frame step is applied, and that scroll_delay does not change speed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): escape registry and upload values in plugin manager inline handlers The store, saved-repository and custom-registry buttons built onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone, so a custom registry entry whose id contained ' closed the attribute and added its own handler. One helper, jsStringAttr(), now HTML-escapes the JSON literal for every one of those handlers, and the store View button opens only http(s) repo links. The live window.updateImageList (plugins_manager.js loads last, so its copy wins over the file-upload widget's) wrote the uploaded file's original name, path and ids into markup raw; they are escaped now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note plugin asset, action and inline handler guards Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): let the root pip wrapper install web_interface/requirements.txt Update Code, the automatic update's health check and Install Base Requirements install web_interface/requirements.txt through safe_pip_install.sh, which only allowed the root requirements.txt. The first commit changing that file would fail its dependency install, and the automatic updater rolls back any update whose dependencies did not install -- on every device, for every newer commit. The wrapper now lists both core requirement files. Only their folders are resolved, so a requirements.txt symlinked out of the project is compared by its target and refused (previously the root file's own symlink target was what got allowed). The updater's file list is a named constant, and a test runs the real wrapper (pip stubbed) on every file Update Code and the rollback install. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): do not retry plugin requests that got an HTTP answer PluginAPI.request wrapped everything that was not a structured error as NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a JSON error without error_code included. Check & Update All retries NETWORK_ERROR, so those updates were re-sent five more times with backoff, contrary to the #587 contract that an HTTP error response is the server's answer. NETWORK_ERROR now means only that fetch() rejected. Any HTTP response without an error_code, or with a body that is not JSON, is API_ERROR with the HTTP status attached. Tested against the shipped api_client.js. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scroll): restart the stats window when an idle gap is dropped by size #582 dropped an idle gap from the frame stats two ways: the reset_scroll() sentinel, which also restarts the 5s window timer, and a size guard for scrollers that never call reset_scroll(), which did not. On that path the first real frame after the gap found the boundary overdue and logged a stats line for a one-frame window. Both paths now share one seeding helper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): leave plugins alone when update_core's own rollback fails update_core returns rollback_failed directly when a partial pull or an update whose health check never started cannot be rolled back. run() only held plugins back for 'verifying', so those devices still got new plugin versions and a display restart on top of a core in an unknown state -- the opposite of what the health-check path does, and of the 3.4.0 changelog (plugins are left alone if the rollback fails). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(api): make the REST reference match the api_v3 package Every documented request body, query parameter and response shape was re-checked against the handlers in web_interface/blueprints/api_v3/. Fixes calls that failed as documented (repo_url, action_id/params, files/image_id, font_file+font_family, ?font=, cache key, auto_enable_ap_mode, plugin limit keys), removes the font-override endpoints dropped in #566, corrects response shapes (plugins/config, plugins/schema, health, metrics, operation history, github-status, fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds the 26 routes it omitted (backup, system auto-update/git, wifi radio, starlark editor, MQTT bridge, status endpoints, skins). Documents the merge semantics of partial JSON saves to /config/main and /plugins/config and the dim-schedule POST accepting GET's days shape, which land in the same change set. Replaces app.py line numbers and the removed api_v3.py path with file and function names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): remove the General-tab plugin system toggles that did nothing plugin_system.auto_discover, auto_load_enabled and development_mode had General-tab toggles whose help tips promised dormant plugins and verbose logging, but nothing reads them: every enabled plugin is discovered and loaded regardless. Remove the three toggles. The keys stay tolerated in stored configs. The save handler now stores a flag only when a client sends it; treating a missing key as an unchecked box would otherwise rewrite all three to false on every General-tab save, which still posts plugins_directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(scroll): remove dead code left by #523/#570 - Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read them since the numpy blend replaced the scipy path. - Drop ScrollHelper._last_integer_position and frame_time_target, which were written but never read. - Keep target_fps and set_target_fps() but document them as informational: nothing paces off them, yet ledmatrix-elections' test_scroll_pacing.py reads helper.target_fps back and third-party plugins may call the setter. - Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to throttle", set_sub_pixel_scrolling's "default: True", and set_frame_based_scrolling's claim that it steps. The plugins monorepo was grepped for every removed name; none is used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI FONT_MANAGER.md told plugins to read display_manager.font_manager, which does not exist, so a plugin following it failed to load with AttributeError. The shared FontManager lives on the PluginManager and BasePlugin._get_font_manager() returns it (with a fallback for harnesses). Also removes the Fonts-tab override workflow and element-override panels that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls /plugins/store/search does not exist (404) and the list endpoint reads query, not q. The registry guide's curl examples omitted the JSON Content-Type, so the handlers saw an empty body and answered 400. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(config): use the shared core-key list in the last three private copies StartupValidator warned "Plugin 'auto_update' is enabled but not found" on every display start with auto-update or a dim schedule on; the reserved plugin-id check missed auto_update, sync, location and the rest; and ConfigManager's (uncalled) orphan cleanup would have deleted display, schedule and auto_update. All three now read src/core_config_keys.py, which also gains CORE_SECRETS_KEYS for the github/youtube secrets sections. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): partial JSON saves to /config/main change only what they send A JSON body with one field reset every checkbox in the sections it touched: the MQTT bridge's brightness slider turned off disable_hardware_pulsing, inverse_colors, show_refresh_rate and use_short_date_format, and a timezone-only save turned off web-UI autostart and weekly auto-updates. Missing-means-unchecked now applies only to form posts: form-encoded bodies and the v3 forms, which mark themselves with a hidden __form_section input. Also on the config routes: - vegas_min/max_cycle_duration no longer match the generic *_duration rule, so they stop landing in display_durations and a blank one no longer rejects the whole Display save; - saving from the Raw JSON editor calls start_setup_if_needed like the General form, so enabling auto-update there finishes its setup; - the schedule and dim-schedule POSTs accept the per-day days.<day> shape their GETs return, as well as the flat form keys. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): install plugin dependencies from the configured plugins directory install_plugin_dependencies.sh scanned only plugins/, but the Plugin Store installs into plugin_system.plugins_directory (default plugin-repos), so the documented "Recommended" fix found 0 plugins on every store install. It now reads plugins_directory from config/config.json (relative to the project root or absolute, default plugin-repos) and also scans plugins/ for dev symlinks, installing a plugin reached through both only once. With set -e alone, `pip ... | tee` took tee's exit status, so a failed pip install was reported as success; set -o pipefail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: replace stale API names, line numbers and the api_v3.py path - ADVANCED_FEATURES: StreamManager methods that exist (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the real on-demand status envelope ({status, data: {state, service}}) - app.py:199 / :144 / :607-619 line citations and web_interface/blueprints/api_v3.py (now a package) replaced with file and function names in ADVANCED_FEATURES, CONFIG_DEBUGGING, PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE, PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README - CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use /config/raw/main to replace the file; describe where validation runs - TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints usage) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): verify the web interface that actually ships, on port 5000 verify_installation.sh failed every healthy install: it required the long-removed web_interface_v2.py and looked for a listener on port 5001, while the web interface binds 5000 (web_interface/start.py). It now checks the files ledmatrix-web.service runs (start_web_conditionally.py, web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the same 5001 port in its listen check, HTTP probe and printed URLs. Port matches are anchored so :50001 no longer counts as :5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(plugins): one display-size contract: display_manager.width/height CLAUDE.md (#580) says to read display_manager.width/height because matrix is None when hardware init fails; the development guide, the safety-harness doc and two DisplayManager docstrings still recommended matrix.width/height. The bundled starlark-apps plugin read matrix.width unguarded, so its magnify recommendation and frame scaling raised in fallback mode (e.g. after the Pi 5 hardware refusal). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(install): make install_service.sh --help print usage instead of installing install_service.sh parsed no arguments, so `sudo ./scripts/install/ install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md) rewrote ledmatrix.service, ledmatrix-web.service and both update-verify units and enabled/started them. It now handles -h/--help (usage, exit 0, no changes) and rejects any other argument with exit 2 before doing anything. Running it with no arguments, as first_time_install.sh does, is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scroll): describe the fixed-step model and document frame_hold Since #545 a crisp speed from scroll_config.configure() makes the helper advance a fixed whole-pixel step per presented frame with no clock, and the display manager's frame hold is part of the speed. The docs still described the removed wall-clock model: - scroll_config's module and configure() docstrings said speed is applied in time-based mode and that omitting the hold "falls back to fractional pixels"; omitting it actually runs the scroll frame_hold times too fast. - SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both modes, and read a 20 ms stats median as missed refreshes although that is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step, the hold-dependent healthy median, that target_fps plays no part, and that a hand-added scroll_pixels_per_second loses to a schema-default pair. - PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling) without frame_hold; it now documents the parameter (core 3.4.0) with a configure() + set_scrolling_state example. - update_scroll_position/set_scroll_speed and set_scrolling_state docstrings say the same. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark target_fps legacy; describe what Vegas scroll_delay does - General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after the sports_scroll fix nothing in core scrolling reads it. The field and its API validation stay so saved configs and plugins that read global_config['target_fps'] keep working. CONFIG_REFERENCE says the same. - Vegas frame_based_scrolling/scroll_delay were described as frame-count stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and still advances by elapsed time, so the applied speed is clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The config comments, render_pipeline comment and CONFIG_REFERENCE rows now say so. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(deps): describe how plugin dependencies are really installed The guides said the web service runs as root, that installs pick --user from os.geteuid(), and quoted a warning and a PluginManager._install_plugin_dependencies() method that don't exist. The web unit runs as the installing user; store installs go through install_requirements_file() and sudo safe_pip_install.sh (root), with a user-level fallback that says so, and load-time installs run in the display service's own (root) interpreter. Manual paths now use the configured plugins directory (plugin-repos/ by default) instead of plugins/, which store installs no longer use, and install_plugin_dependencies.sh is described as scanning that directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): count local changes one way for the preflight and the pull The automatic update's preflight ignored mode-only changes and anything whose status line contained plugins/ or plugin-repos/, then promised "Automatic updates will not stash your changes". perform_core_update used plain git status (modes count) and ignored only 'plugins/', then ran 'git stash push -- :!plugins', which nothing ever pops. So an edit to a bundled plugin under plugin-repos/, or the installer's chmods on tracked scripts, passed the preflight and was stashed away for good. - auto_update.local_changes() is the one predicate both use: core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/ excluded by leading folder rather than substring (a core file under web_interface/static/v3/js/plugins/ now counts). - Update Code's explicit stash leaves out both plugin folders; the pull's --autostash carries their edits and mode changes across and reapplies them. - The automatic updater calls perform_core_update(stash_local_changes= False), which refuses instead of stashing edits that appeared after the preflight; update_core reports that as 'blocked'. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): diagnostics follow the web autostart default and api_v3 package #556 made a missing web_display_autostart mean "start" (only an explicit false/off keeps the web interface down), but the diagnostics still said otherwise: diagnose_web_ui.sh reported a missing key as "defaults to false", diagnose_web_interface.sh said the web interface "will not start unless this is set to true" and recommended enabling it, and debug_web_manual.py printed False. Troubleshooting a down web UI pointed users at a non-cause. Both shell scripts now evaluate the setting with the launcher's own autostart_enabled() (inline fallback if it cannot be imported) and report on / off / not set (on) / unparseable config; debug_web_manual.py uses the same function. They also check web_interface/blueprints/api_v3/ __init__.py: api_v3.py became a package in #553, so every healthy checkout was reported as missing a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(install): what install_service.sh installs; verify script port; no sudo for --help install_service.sh installs and starts ledmatrix, ledmatrix-web and the update-verify units, not only ledmatrix.service (systemd/README.md, README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help' as a harmless check; it now shows --help without sudo and warns what a real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh checks the web interface on port 5000. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): note update-all, plugin system settings and script fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): size the preview after orientation and pixel mappers display_geometry.physical_size claimed to give DisplayManager's answer but only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured after the library's pixel mappers, so a Rotate:90 / orientation 90 chain previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for 128x64. Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown names leave it alone), and move the orientation composition here so DisplayManager and the preview share it. The module docstring no longer claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH. Audit finding F18. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(display): refuse settings the rgbmatrix library aborts on, on every board The library answers several settings with a NULL matrix or abort() rather than an error, so the display service crash-looped (Restart=on-failure) instead of reaching fallback mode: rows above 64, chain_length above 255 (uint8_t binding setter, documented as "no upper limit"), a misspelled hardware_mapping, and parallel 2-3 on a single-output mapping, reachable from the Display form on the default adafruit-hat(-pwm) mapping. #586 only guarded the Pi 5 subset. - src/matrix_support.py holds the rules for every board (Options::Validate ranges, binding integer types, mapping names and outputs from lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the API's numeric ranges. - DisplayManager checks them before building options and raises MatrixSettingsRefused, so a hand-edited config falls back with a logged, reported reason. Emulator mode only warns. - The config API refuses them with a 400 naming the setting; combinations are checked against stored values but reported only when the request sets a field involved. - The hardware status file gains "cause" (settings/library/forced). The fallback log and Display banner give the Pi 5 rebuild hint only for a library failure instead of rebuild + gpio_slowdown advice for every failure; one Pi 5 slowdown recommendation (1-3, start at 1). - The Display form offers classic/classic-pi1 and orientation 90/270 and renders any other stored mapping selected with a warning, so an unrelated save no longer rewrites them; the API accepts 90/270. Audit findings F03, F16, F19, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(display): library limits, template defaults and Pi 5 slowdown - rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs, classic/classic-pi1 mappings and orientation 90/270 documented. - Defaults are the config.template.json values: config migration adds missing keys from the template, so the listed "code defaults" never applied. - One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode, starting at 1. - Troubleshooting describes the refused-settings fallback, and CHANGELOG corrects the Unreleased "no upper limit" entry. Audit findings F19, F20, F21. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py opens the panel with the service's options --measure and --demo built RGBMatrixOptions from a private copy of the display service's builder that had drifted: gpio_slowdown came from display.hardware (default 2) instead of display.runtime (default 3), and rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors, pixel_mapper_config and orientation were skipped, with different defaults (hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown was measured -- or garbled -- in a setup the service never drives. The option filling in DisplayManager._setup_matrix moves, unchanged, into DisplayManager.apply_matrix_options(options, config), which _setup_matrix calls and the script reuses (overriding only limit_refresh_rate_hz for --measure). The script now loads the whole config rather than the hardware block. Tests pin the script's options to the service's attribute for attribute. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scripts): scroll_speeds.py recommends keys the resolver honours The ladder ended by telling users to set display_options.scroll_pixels_per_second. scroll_config ranks that key below the scroll_speed + scroll_delay pair, deliberately, and several plugin schemas default the pair into config, so the advised key was silently ignored (a schema-default 1/0.02 pair plus an advised 66 still resolved to 50 px/s). The advice is now the pair that selects the crisp speed exactly (pixels_per_frame every frame_hold/refresh seconds), explains that the pair outranks scroll_pixels_per_second, and gives the scoreboards' per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the printed pair over a schema-default pair and check it lands on the advertised speed and hold. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula - SPORTS_UNIFICATION.md still presented honouring global target_fps as sports_scroll's added behaviour and its one user-visible gain; note that it was withdrawn because it had become a speed multiplier. - ADVANCED_FEATURES.md gave Vegas scrolling as (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s by elapsed time, through a 0.1-5 px per scroll_delay clamp when frame_based_scrolling is on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): scroll model fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(dev): link-github links plugins from the ledmatrix-plugins monorepo link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git, and those per-plugin repositories no longer exist: official plugins are directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the monorepo once into the dev directory, finds plugins/<name>, plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and links it under its manifest id. With an explicit repo URL it still links a single-repository plugin as before. dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a fork), plus plugins_repo and plugins_branch; github_pattern, which was documented but never read, is dropped and warned about. Ships dev_plugins.json.example and git-ignores dev_plugins.json, both of which the guide promised. Reading JSON falls back to python3 when jq is missing (get_plugin_id silently returned nothing without jq). update/status/list find the git checkout above a monorepo plugin directory (its .git is not in the plugin dir), and update pulls a shared checkout once. status no longer exits 1 when nothing is broken. Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration, workflow, store integration, hello-world link, submission), and the nonexistent scripts/git-hooks/pre-push-plugin-version and scripts/bump_plugin_version.py replaced with the real rule: bump the manifest version and run update_registry.py. scripts/dev/README.md and CLAUDE.md updated to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(scripts): monorepo workspace layout; fix_perms and install READMEs MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin; setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the workspace file opens LEDMatrix plus ../ledmatrix-plugins. scripts/fix_perms/README.md listed cache directories fix_cache_permissions.sh never touches and a 'ledmatrix' service user that doesn't exist (also in scripts/install/README.md); adds safe_pip_install.sh. install/README: install_service.sh installs the web and update-verify units too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): keep the rollback's pip retries inside the unit time limit The health check reinstalled the previous requirements by trying the next bash path after any failure, including a 600 s pip timeout. Two files, two paths: up to 40 minutes of pip alone, while systemd stops ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the rollback half-way and leaving the update 'verifying' until the web UI calls it lost. - Like permission_utils.install_requirements_file, only a sudo refusal moves on to the next bash; a pip that ran and failed or timed out is not repeated. The refusal wording is one list (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only verifier and pinned equal by a test. - All reinstalls in one rollback share a 600 s budget. - WORST_CASE_SECONDS adds up every timeout on the longest path (27.5 min); a test holds it under the unit's TimeoutStartSec and that under the web UI's VERIFY_LOST_SECONDS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools Plugin config was prepared differently depending on how it arrived: - JSON POST /plugins/config built a partial body on schema defaults, so {"enabled": true} reset every other setting of the plugin. It now merges onto the stored section first, as the form path already did. - Legacy-boolean normalization (#588) ran only at load: GET /plugins/config returned the raw boolean, posting it back failed validation, and hot reload handed plugins the raw section (a legacy dynamic_duration: true came back as a boolean). schema_manager.prepare_plugin_config (normalize, then defaults) is now used by PluginManager.load_plugin, both save paths, GET, the save notifications and DisplayController's hot-reload callback. - The JSON save's filter kept only enabled/display_duration/live_priority and dropped a submitted skin, skin_options or vegas_* tuning key. There is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES, used by validation and by the save filter; PluginManager's CORE_OWNED_CONFIG_KEYS is its vegas subset. - Plugin sections posted to /config/main were stored verbatim, including values /plugins/config rejects. They now go through the same preparation (_prepare_plugin_config_for_save, extracted from save_plugin_config), and a failing section rejects the whole save before anything is written. - dev_server read only top-level defaults and let a schema enabled:false win; build_full_config shallow-merged overrides, dropping sibling defaults; the harness extracted defaults differently from the device. loading.build_config now uses the device's extraction and preparation, and dev_server, check_plugin, render_plugin and the harness all use it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt-bridge): brightness changes apply live and touch nothing else The display service's hot reload applies a saved brightness within a few seconds, and /config/main no longer resets other display settings on a brightness-only JSON body. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): automatic update hardening Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI It described web_interface_v2.py and index_v2.html (both gone), client-side form generation, one POST per field with {key, value}, and 'no nested objects'. The v3 UI renders plugin forms server-side from the schema (pages_v3 partial + plugin_config.html macros, nested sections and x-widgets), posts the whole form once, and save_plugin_config() merges onto the stored section, validates, splits x-secret fields and notifies the plugin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(mqtt): brightness saves apply via hot reload and leave other settings alone The bridge README said brightness is applied on the display's next restart; the display controller's config hot reload applies it within seconds. It also now states that the bridge's partial JSON save changes only brightness (the /config/main merge fix in this change set). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(update): don't log pip's output from the health check's reinstall pip can echo a private index URL with embedded credentials; permission_utils redacts it, the stdlib-only verifier cannot, so it logs the exit code only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(config): mark the plugin_system toggles as unused legacy keys auto_discover, auto_load_enabled and development_mode are read by nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and the REST reference listed them as live settings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): docs and developer tools group Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): legacy plugin-system toggles no longer count as a General save auto_discover, auto_load_enabled and development_mode have left the General form, so a post carrying only one of them is not a general-settings save and must not treat web_display_autostart and auto_update as unchecked. The plugin_system block itself is left as on main for the branch that reworks it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(changelog): config-save and plugin-config preparation fixes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(claude): re-check matrix_support.py rules when the library submodule is bumped Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address Codacy findings on the core audit PR - plugin_manager.prepare_plugin_config: when the fallback legacy-boolean pass also fails, log a warning instead of a bare except/pass. - api_client.js: request() refuses any endpoint that is not a plain path under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace, control characters) with INVALID_ENDPOINT before calling fetch(), and plugin ids are URL-encoded wherever they are put into a URL (also in the app-shell batch load). - test_update_all.js: pins both against the shipped client. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(web): check endpoint control characters without a control-character regex Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags the \x00-\x1f range in checkEndpoint's regex. Test the char codes instead; the endpoints refused are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(auto-update): make the seed script executable on disk, not only in the index On Linux Repo.publish() commits with -a, which recorded scripts/run.sh as 100644 upstream because the seed file was never chmod +x. The pull then brought in the same mode the installer chmod had made locally, so installer_chmod saw no mode change left to check. The updater was fine: with the upstream commit at 100755 the --autostash carries the device's chmod across. Verified under Linux (WSL, git 2.43): the old helper fails exactly as CI did, the fixed one passes all 63 tests in the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
548 lines
32 KiB
Markdown
548 lines
32 KiB
Markdown
# Sports Code Unification — Architecture
|
||
|
||
How the nine sports scoreboard plugins converge onto shared core code **without**
|
||
becoming nine clients of a god class.
|
||
|
||
## The problem
|
||
|
||
Nine plugins (`afl`, `baseball`, `basketball`, `football`, `hockey`, `lacrosse`,
|
||
`nrl`, `soccer`, `ufc`) each ship a ~3,000-line `sports.py` descended from this
|
||
repo's `src/base_classes/sports.py`. They have drifted into three lineages, and
|
||
only 28 of the 66 methods appearing across them are present in all nine. One
|
||
logical fix (the UTC start-time bug) cost 75 files.
|
||
|
||
Merging everything into one base class would fix the duplication and create a
|
||
worse problem: a single 2,500-line class that all nine plugins inherit, where any
|
||
change has a nine-plugin blast radius and per-sport behavior survives only as
|
||
`if self.sport == "hockey"` branches.
|
||
|
||
## Three properties, three mechanisms
|
||
|
||
These are independent concerns. Conflating them is what produces god classes.
|
||
|
||
### Upgradability — a plugin keeps working across core versions
|
||
|
||
| Rule | Mechanism |
|
||
|---|---|
|
||
| Plugin loads on a core that predates a module | Guarded import with a bundled fallback (`try: from src.X import Y / except ModuleNotFoundError: from y import Y`) |
|
||
| Plugin loads on a core that predates a *method* | Capability probing — `hasattr(SportsCore, "_detect_stale_games")` — never a version comparison. The loader's compat check is advisory-only (it logs and continues), so probing is the real protection. |
|
||
| Core changes never break a plugin's rendering | The **view-model contract**: `_extract_game_details_common` returns a dict whose `GUARANTEED_KEYS` are frozen by `test/test_skin_system.py::TestViewModelContract`. Keys may be added, never renamed or removed. |
|
||
| A plugin can drop its bundled copy safely | The **sunset rule**: its manifest must floor `ledmatrix_min_version` at the first core release shipping the module (recorded in `CHANGELOG.md`) — *necessary but not sufficient*. The store enforces that floor on every registry-managed install and on both supported update paths (sideloading via `install_from_url` is not gated), but a floor cannot reach a user who never updates, so the copy also waits for the B6 gate below. |
|
||
|
||
The core API is **additive-only**. A method the plugins call is never removed or
|
||
given a new required parameter; new behavior arrives as new methods with
|
||
defaults, or as capabilities they opt into.
|
||
|
||
### Reusability — write once, nine plugins benefit
|
||
|
||
Only code that is **identical in intent across all nine** moves into the base
|
||
class. That set is small and knowable — it is exactly the methods present in every
|
||
copy today (phase B1 below). Everything else stays where it is until it earns
|
||
promotion.
|
||
|
||
### Modularity — a change to one feature cannot reach a plugin that doesn't use it
|
||
|
||
This is the property the naive merge destroys, and it is enforced structurally:
|
||
|
||
1. **Capabilities are separate modules composed by inheritance, not config
|
||
branches inside the base class.** Hockey has no celebrations, so
|
||
`HockeyLive` does not inherit `CelebrationMixin` — the celebration code is not
|
||
merely disabled for hockey, it is *not in hockey's MRO at all*. No shared
|
||
state, no dead branches, no risk. Contrast with
|
||
`if self.celebrations_enabled:` inside `SportsLive`, where a bug in
|
||
celebration code can still crash a plugin that never wanted the feature.
|
||
|
||
2. **Variant behavior is a strategy object chosen by name, not a branch.**
|
||
Live rotation exists in three dialects across the lineages; core ships all
|
||
three behind `rotation_strategy: "swrr" | "weighted" | "simple"` and a plugin
|
||
may register its own. Core never learns sport names.
|
||
|
||
3. **Sport-specific behavior is a documented override point.** The base class
|
||
declares the seam; the plugin fills it. Basketball's tournament-round parsing
|
||
and baseball's BDF sizing stay in their plugins forever — they are not
|
||
candidates for promotion, and core must never grow a branch for them.
|
||
|
||
4. **Files bound the blast radius.** Capabilities live in their own modules so a
|
||
diff shows at a glance which plugins a change can reach.
|
||
|
||
## Layering
|
||
|
||
```
|
||
src/base_classes/sports/
|
||
__init__.py re-exports the public API (import path unchanged)
|
||
core.py SportsCore — fetch, cache, config, logos, fonts, odds,
|
||
view-model extraction, the skin seam
|
||
modes.py SportsUpcoming / SportsRecent / SportsLive
|
||
capabilities/
|
||
celebrations.py CelebrationMixin (opt-in: 4 of 9 plugins)
|
||
rotation.py RotationStrategy + registry
|
||
|
||
src/common/
|
||
sports_scroll.py SportsScrollDisplay / …Manager — scroll orchestration
|
||
(content building stays in the plugins)
|
||
sports_helpers.py clamp/logo/rotation free functions + SportsHelpersMixin
|
||
(unreleased) — the helpers byte-identical in the
|
||
plugins' sports.py, and the _favorite_key seam
|
||
```
|
||
|
||
`from src.base_classes.sports import SportsCore` keeps working — the package
|
||
`__init__` re-exports, so the conversion is invisible to every existing importer.
|
||
|
||
### Converging on `src/common`
|
||
|
||
The scoreboards do not build on `src/base_classes`; their own `sports.py` copies
|
||
have moved past it. So shared code now lands in hardware-free `src/common`
|
||
modules taken from the plugin copies, each a **new module** rather than growth
|
||
on an existing one: a plugin that deletes a method copy and relies on an older
|
||
module having gained it fails at runtime with an `AttributeError`, while a
|
||
missing module fails at load, where the version checks can see it.
|
||
`sports_helpers.py` is the first (it holds `_favorite_key`, the override point
|
||
listed below, for later phases); its parity test compares every body against
|
||
the plugin copies when `LEDMATRIX_PLUGINS` points at a checkout, and
|
||
`test/test_common_is_hardware_free.py` keeps `src/common` free of
|
||
`rgbmatrix`, `src.base_classes` and `src.plugin_system`. How a plugin adopts a
|
||
module and drops its copy is documented in the plugins repo's
|
||
`docs/plugin-development/08-shared-sports-code.md`.
|
||
|
||
## Override points (the plugin-facing seam)
|
||
|
||
The base class calls these; plugins implement or override them. This table is the
|
||
contract — additions require a default implementation, removals require a
|
||
deprecation cycle.
|
||
|
||
| Hook | Purpose | Default |
|
||
|---|---|---|
|
||
| `_fetch_data()` | Sport's schedule source | abstract |
|
||
| `_extract_game_details(event)` | Sport-specific view-model fields on top of the common ones | delegates to `_extract_game_details_common` |
|
||
| `_draw_scorebug_layout(game, force_clear)` | Sport's card rendering | base layout |
|
||
| `_custom_scorebug_layout(game, draw)` | Per-sport overlay on the base layout | no-op |
|
||
| `render_skin_card(game, size)` | Skin-system entry point | built-in fallback |
|
||
| `score_phrase(points, team_abbr)` | Celebration wording (`"GOOOOAAALLL!"` vs `"TOUCHDOWN!"`). `points` is the score delta, which sports with variable-value scores use to name the play | `"<abbr> SCORES!"` — only consulted when `CelebrationMixin` is present |
|
||
| `win_phrase(team_abbr)` | Win-celebration wording | `"<abbr> WINS!"` — mixin only |
|
||
| `_favorite_key(game, side)` | Which view-model field identifies a team for favorites matching | `game["<side>_abbr"]` |
|
||
| `_config_schema_path()` | Plugin's `config_schema.json` — returning it routes `_get_layout_offset` through the `src.element_style` resolver (and gives it the defaults to compare against) | `None`, i.e. the classic inline `customization.layout` read |
|
||
| `_font_root()` | Directory to resolve `assets/fonts` against | core install root |
|
||
|
||
Two class attributes serve the same purpose for values that are per-sport
|
||
constants rather than behavior:
|
||
|
||
| Attribute | Meaning | Default |
|
||
|---|---|---|
|
||
| `FINAL_PERIOD` | Period at/after which a zero clock can mean "over" | `4` (hockey overrides to `3`) |
|
||
| `CLOCK_COUNTS_DOWN` | Whether `0:00` means "expired" | `True` (soccer/afl/nrl override to `False` — their clocks count up, so `0:00` is kickoff) |
|
||
| `COALESCE_SCORING_SEQUENCE` | Fold score increments arriving during an active celebration into that one celebration | `False` (football overrides to `True` — a touchdown lands as +6, then +1 for the extra point) |
|
||
|
||
### Why these are seams and not branches
|
||
|
||
`_favorite_key` exists because NRL abbreviations are **not unique** — "NEW" is both
|
||
Newcastle Knights and New Zealand Warriors, "CAN" both Canberra and Canterbury —
|
||
so NRL matches favorites on team ID. Flattening every plugin to abbreviations
|
||
would silently select the wrong club for NRL users. The base declares the seam,
|
||
NRL fills it, and core never learns the string `"nrl"`.
|
||
|
||
`CLOCK_COUNTS_DOWN` exists for the same reason in the opposite direction: a
|
||
soccer clock reading `0:00` means the match has not kicked off, so running the
|
||
clock-expiry branch there would evict live games.
|
||
|
||
`COALESCE_SCORING_SEQUENCE` is the third of the same kind. In football one
|
||
scoring play arrives as two score updates, so the follow-up must be folded into
|
||
the first celebration; in soccer two increments a few seconds apart are two real
|
||
goals, and folding them would swallow one. Neither default is "right" — which is
|
||
precisely why it is a declared per-sport constant rather than a hidden
|
||
assumption baked into the shared body.
|
||
|
||
## Capabilities
|
||
|
||
```
|
||
capabilities/
|
||
celebrations.py CelebrationMixin opt-in: afl, nrl, soccer, football
|
||
rotation.py RotationStrategy + registry
|
||
```
|
||
|
||
**`CelebrationMixin`** merges the two dialects the lineages grew
|
||
(`_check_for_goal`/`celebrate_opponent_goals` vs
|
||
`_check_for_score`/`celebrate_opponent_scores`). Their bodies were identical
|
||
apart from three things, each now a seam: wording (`score_phrase`), follow-up
|
||
suppression (`COALESCE_SCORING_SEQUENCE`), and team identity (`_favorite_key`,
|
||
so NRL matches on id). Both config spellings are read, so a plugin adopting the
|
||
mixin keeps working with the keys already in its published schema.
|
||
|
||
Mix it in **before** the mode class — `class SoccerLive(CelebrationMixin,
|
||
SportsLive)` — so the celebration `display()` runs first and falls through to
|
||
the scorebug via `super()`.
|
||
|
||
**Rotation strategies.** The three "dialects" turned out to be one algorithm
|
||
(Smooth Weighted Round-Robin) in two shapes: an incremental picker holding state
|
||
across calls (afl/nrl/soccer) and a precomputed per-cycle list
|
||
(football/baseball/basketball, and hockey with a different loop shape). They
|
||
agree within a cycle and differ only at the boundary — the incremental form has
|
||
no restart seam — so core ships both rather than declaring a winner:
|
||
|
||
```python
|
||
self.rotation = get_rotation_strategy("swrr", weight_for=self._live_weight)
|
||
```
|
||
|
||
`weight_for` is supplied by the host, so the *favorites* policy stays with the
|
||
plugin and `rotation.py` never learns what a favorite is. An unknown strategy
|
||
name degrades to `simple` rather than raising: the name comes from user config,
|
||
and a typo should cost the boost, not the scoreboard. When a plugin needs an
|
||
ordering that core does not ship, it calls `register_rotation_strategy` to add
|
||
its own — rather than core growing a branch for it.
|
||
|
||
`test_sports_capabilities.py` checks each strategy against a **verbatim
|
||
transcription** of the plugin code it replaces, over every live-game shape up to
|
||
four games. That differential is what B5 deletes the bundled copies on the
|
||
strength of.
|
||
|
||
## Scroll display — where the promotion line falls
|
||
|
||
`src/common/sports_scroll.py` is deliberately *not* a superset of the ten
|
||
`scroll_display.py` copies. A method-level comparison of the eight that share a
|
||
shape (f1 and ufc are genuine forks) found a sharp split:
|
||
|
||
| Layer | Evidence | Outcome |
|
||
|---|---|---|
|
||
| Orchestration — `get_all_vegas_content_items`, `clear_all`, `get_scroll_info`, `get_dynamic_duration`, `is_complete`, `display_frame` | identical to 96–100% similar across all eight | **promoted** |
|
||
| Settings — `_get_scroll_settings` | one algorithm; the copies differ *only* in which league keys they walk | **promoted**, with the ladder as data (`SCROLL_LEAGUE_KEYS`) |
|
||
| Content — `prepare_scroll_content`, `_load_separator_icons` | 8 distinct bodies across 8 plugins (145 lines, 53% similar at worst); icons 6% | **override point, permanently** |
|
||
|
||
Same name, different job: `prepare_scroll_content` draws *this sport's* game
|
||
card. Merging the eight bodies would be the exact mistake the promotion rule
|
||
exists to prevent, so the base class raises `NotImplementedError` rather than
|
||
rendering something plausible — a base that rendered *something* would let a
|
||
plugin ship a silently blank scroll.
|
||
|
||
The one behavior the upstreamed version adds is native
|
||
`global_config['target_fps']` support. The bundled copies hardcode ~100 FPS via
|
||
`scroll_delay = 0.01` and never consult the global smooth-scrolling target;
|
||
Part A threaded it through each copy by hand, and this makes that threading
|
||
legacy compatibility rather than the mechanism.
|
||
|
||
> **Superseded.** Once presentation became frame-locked (#545) the helper
|
||
> steps a fixed whole-pixel amount per presented frame and the panel presents
|
||
> at its own refresh, so honouring `target_fps` only turned it into a speed
|
||
> multiplier (60 doubled a scoreboard's speed, 200 halved it). `sports_scroll`
|
||
> no longer reads it: the crisp-speed ladder uses the panel refresh
|
||
> (`display_manager.refresh_hz`), and speed comes from
|
||
> `scroll_settings.scroll_speed` alone. See `docs/SCROLL_PERFORMANCE.md`.
|
||
|
||
## Phases
|
||
|
||
B0–B3 are merged and shipping in core 3.2.0. Everything that remains is
|
||
**rollout**, and it splits into three phases with very different risk profiles.
|
||
The original plan folded the last two together; they are separated here because
|
||
one of them cannot break a user on an old core and the other can.
|
||
|
||
| Phase | Scope | Status | Gate |
|
||
|---|---|---|---|
|
||
| **B0** | Characterization tests, CI unit job, `element_style`, font cwd fix, CHANGELOG discipline | ✅ | — |
|
||
| **B1** | Promote the nine universal methods; convert `sports.py` → package | ✅ | Characterization suite green; no behavior change intended |
|
||
| **B2** | `CelebrationMixin` + rotation strategies as opt-in capabilities | ✅ | Non-adopters have zero new code in their MRO; strategies checked against verbatim plugin transcriptions |
|
||
| **B3** | Upstream the scroll **orchestration** layer as `src/common/sports_scroll.py`, reading `global_config['target_fps']` natively | ✅ | Content building stays per-sport |
|
||
| **B4** | Ship 3.2.0 *and* make version reporting trustworthy | ✅ | Released 2026-08-03; tag, release and `src.__version__` agree; compatibility gate merged (#428, #431, #433) |
|
||
| **B5** | Adoption — guarded core imports, all eight. **Bundled copies stay.** | ✅ | All eight adopted; harness byte-identical; see the B5 retrospective below — four shipped broken and were repaired in plugins #251 |
|
||
| **B6** | Sunset — delete the bundled copies | ✅ | Ran 2026-09-01, all eight. Floors at 3.2.0; the store refuses on all three routes in (#431/#433, #508, #510). See "B6 — what actually happened" |
|
||
|
||
### B4 — what "ship 3.2.0" actually requires
|
||
|
||
Cutting the tag is the small part. The version *number* has to become something
|
||
a floor can be trusted against, and today it is not:
|
||
|
||
- **The tag and `src.__version__` have never agreed.** `v3.1.0` was tagged
|
||
2026-05-31; `__version__` only became `"3.1.0"` on 2026-07-12 (`7f7f0d64`).
|
||
The v3.1.0 release therefore reports `__version__ = "1.0.0"`.
|
||
- **Which silences the compatibility warning entirely for that population.**
|
||
`PluginLoader._warn_if_incompatible` skips the check when the parsed core
|
||
version is below `(2, 0, 0)` — an anti-spam guard that, given the above,
|
||
matches exactly the users most likely to be behind.
|
||
- **Nothing enforces a floor anyway.** The check is advisory (it logs and
|
||
continues), and neither `StoreManager.install_plugin` nor
|
||
`StoreManager.update_plugin` compares the core version at all — `update_plugin`
|
||
compares the plugin's manifest version against the registry's
|
||
`latest_version` and nothing else.
|
||
|
||
*Fixed, in two parts.* `install_plugin` gained the gate in #431/#433, which
|
||
covers every registry-managed install and, through `_reinstall_with_rollback`,
|
||
the update path that re-downloads.
|
||
`update_plugin`'s git branch pulls in place and re-downloads nothing, so it
|
||
stayed ungated until `_gate_pulled_commit` closed it — checked after the pull
|
||
(the registry carries no floor field, so the incoming floor is unknowable
|
||
before it) and undone with `git reset --hard` to the pre-pull commit. That
|
||
route is rare in practice, since monorepo plugins install as archives; it was
|
||
closed because the sunset rule in the plugins repo's
|
||
`08-shared-sports-code.md` states as **condition 3** that the core enforces
|
||
the floor "at install/update time", and B6 rests on that being true rather
|
||
than merely written down. `install_from_url` — sideloading a plugin from a
|
||
URL — is still ungated.
|
||
|
||
So B4 is: tag and release 3.2.0; make the tag, the release, and `__version__`
|
||
agree, and keep them agreeing; reconsider the `< 2.0.0` skip; migrate manifests
|
||
from `ledmatrix_min` to `ledmatrix_min_version`; and add the install/update
|
||
compatibility gate that B6 depends on.
|
||
|
||
#### Two fields express compatibility, and the gate only reads one
|
||
|
||
`compatible_versions` is the canonical contract: `schema/manifest_schema.json`
|
||
**requires** it, all 42 published manifests carry it, and it holds semver
|
||
*ranges* — `[">=2.0.0"]` in 41 of them, `[">=1.0.0"]` in `7-segment-clock`.
|
||
`ledmatrix_min_version` is the optional per-release floor inside `versions[]`.
|
||
|
||
The gate as merged reads only the floor. Today that is harmless: no manifest
|
||
uses an upper bound, and the two fields agree everywhere except
|
||
`7-segment-clock` (`>=1.0.0` against a `2.0.0` floor). But the fields *can*
|
||
disagree, and the range syntax the schema already permits includes upper bounds
|
||
— a plugin declaring `["2.0.0 - 2.9.9"]` means "not compatible with 3.x" and
|
||
the gate would install it on 3.2.0 regardless.
|
||
|
||
**Before B6, the gate must evaluate `compatible_versions` as well**, and the
|
||
manifest migration must reconcile the two fields rather than only renaming the
|
||
floor. Deciding which wins when they disagree is part of that work; the safe
|
||
default is the more restrictive.
|
||
|
||
(The schema also deprecates a top-level `ledmatrix_version` in favour of
|
||
`compatible_versions`. No manifest still carries it, so there is nothing to
|
||
migrate there.)
|
||
|
||
### B5 — the *fallback* is safe by construction; the modern path is not
|
||
|
||
The heading matters, because the unqualified version of this claim is false and
|
||
this document proves it two sections down: four of the eight adopted plugins
|
||
shipped with scroll mode broken on a 3.2.0 core. What is safe by construction is
|
||
narrower than "adoption".
|
||
|
||
A plugin adopting core imports keeps its bundled copy and reaches it through the
|
||
guarded import (see the Upgradability table above). On a core that doesn't ship
|
||
the module the plugin falls back and behaves exactly as it does today. That
|
||
fallback compatibility — and only that — is safe by construction. On a core that
|
||
*does* ship the module, correctness is not automatic — object-level and scroll-mode validation
|
||
(building both classes and comparing, per the retrospective below) is required
|
||
to prove full behavior. There is no version of this step that breaks a user *on
|
||
an old core*, which is why it does not wait for B6's gate.
|
||
|
||
The hockey scroll-display pilot is **already validated**: adopted against a core
|
||
carrying 3.2.0, `scroll_display.py` went from 691 to 289 lines and all 16 harness
|
||
renders (8 sizes × 2 screens) came out byte-for-byte identical to the
|
||
pre-adoption run. That byte-comparison is the acceptance gate for every
|
||
adoption. The recipe and its two gotchas are in the plugins repo's
|
||
`docs/plugin-development/08-shared-sports-code.md`.
|
||
|
||
### B6 — why the sunset needs more than a version floor
|
||
|
||
Deleting a bundled copy removes the fallback, so the guarded import becomes a
|
||
hard dependency. On a core without the module the plugin raises
|
||
`ModuleNotFoundError` at load; `PluginManager.load_plugin` catches it, records
|
||
`PluginState.ERROR`, logs one line, and continues. Nothing crashes — the user
|
||
simply loses that scoreboard, with no visible explanation.
|
||
|
||
Verified against a `v3.1.0` worktree: `src/common/sports_scroll.py`,
|
||
`src/element_style.py` and the `src/base_classes/sports/` package are all absent
|
||
there, and the import fails with `exc.name == 'src.common.sports_scroll'`. Guard
|
||
sets must name that exact dotted path — `{"src"}` alone does not match it.
|
||
|
||
Combined with the B4 findings, a plugin that deletes its copy today reaches an
|
||
un-updated user through a normal store update, fails to load, and warns nobody.
|
||
**B6 therefore waits for B4's compatibility gate to have shipped and to have
|
||
been in users' hands long enough that the population running a core without it
|
||
is small.** The bundled copies cost disk space; deleting them early costs
|
||
scoreboards, silently. That trade is not close.
|
||
|
||
Before the first sunset, add a **compatibility regression test**. **Built:**
|
||
core `test/test_sports_sunset_matrix.py` (#505). It has to
|
||
cover four cases, not one — B5's safety claim and B6's failure mode are
|
||
different propositions and only the second is obvious:
|
||
|
||
| | bundled copy present | bundled copy removed |
|
||
|---|---|---|
|
||
| **pinned old core** | **loads** — this is B5's whole guarantee, that the guarded import falls back | `PluginState.ERROR`, and the recorded error names the exact missing module |
|
||
| **current core** | loads, using core code | loads, using core code |
|
||
|
||
The top-left cell is the one worth writing first: nothing in the suite currently
|
||
proves that an adopted plugin still works on a core that predates the module,
|
||
which is the entire basis for saying B5 is safe to run ahead of the gate.
|
||
|
||
Assert the old-core/removed-copy case as `PluginState.ERROR` **plus the missing
|
||
module path**, not as an uncaught exception. `PluginManager.load_plugin` catches
|
||
`ModuleNotFoundError`, so nothing propagates — a test expecting a raise would
|
||
pass for the wrong reason on a core where the module is merely broken rather
|
||
than absent. "Fails loudly" is aspirational, not what the code does today: it
|
||
fails into `ERROR` state with one log line, which is precisely why B6 needs the
|
||
gate rather than trusting the failure to be noticed.
|
||
|
||
The same suite should exercise the install/update gate, since it is the other
|
||
half of the guarantee.
|
||
|
||
### B6 — what actually happened
|
||
|
||
**Ran 2026-09-01, across all eight scoreboards.** Held from 2026-08-05 to
|
||
2026-09-01 on the argument below, which is kept because the reasoning applies to
|
||
the next module, not because it is still in force.
|
||
|
||
**The hold, and why it lifted.** The stated gate was evidence of 3.2.0 uptake —
|
||
"a few months of it being the default download, or store-side install data".
|
||
That evidence never arrived and could not: the core updates by
|
||
`git pull --rebase`, so release-asset counts cannot measure uptake, and no
|
||
store-side telemetry exists. What changed instead is that the *risk* the gate
|
||
protected against was closed directly. The store now refuses a plugin whose
|
||
floor exceeds the running core on **all three** routes in:
|
||
|
||
| route | gated by |
|
||
|---|---|
|
||
| `install_plugin` — every path that re-downloads, `_reinstall_with_rollback` included | #431, #433 |
|
||
| `update_plugin`'s git branch — pulls in place, re-downloads nothing | #508 |
|
||
| `install_from_url` — sideloading | #510 |
|
||
|
||
With all three closed a pre-3.2.0 user cannot receive a sunset plugin at all;
|
||
they keep the version they already run. The population the hold existed to
|
||
protect is protected by refusal rather than by a bundled copy — which is what
|
||
the copy was standing in for.
|
||
|
||
**What shipped.** Eight plugins, ~5,800 lines of frozen fallback deleted. Each:
|
||
copy removed, guarded import collapsed to a plain one, floor raised to 3.2.0,
|
||
`test_core_fallback.py` rewritten as `test_core_scroll.py` asserting the sunset
|
||
rather than the fallback. `scripts/check_scroll_adoption.py` gained
|
||
`sunset_violations` and a `SUNSET_PLUGINS` set naming all eight, so a
|
||
resurrected copy or a returned guard fails CI.
|
||
|
||
Delivered as plugins #346 (hockey, later folded into #351), #349 (football),
|
||
#350 (baseball), #351 (the remaining six).
|
||
|
||
**Two things found by doing it, both worth carrying forward:**
|
||
|
||
- **Only one fallback held orchestration logic the core lacked.** baseball's
|
||
`_configure_scroll_helper` reinterpreted `scroll_speed` as pixels-per-*frame*
|
||
when `speed × delay` fell outside the 0.1–5.0 window — measured, 10–20×
|
||
faster than configured for a speed between 1.0 and 5.0. Standardised onto the
|
||
core's behaviour (honour the documented px/sec, clamp) rather than preserved.
|
||
Every other difference across the eight was a docstring, an unreachable
|
||
`scroll_helper is None` guard, or an equivalent diagnostic.
|
||
- **Two tests had been leaning on the guard without anyone knowing.**
|
||
`soccer/test_live_screens.py` stubbed `src` in a way that shadowed the core,
|
||
so its guarded import fell back and the test had been exercising the *frozen
|
||
copy* rather than the shipping class since B5. Before sunsetting anything else
|
||
that carries a guarded core import, grep for tests that stub `src`.
|
||
|
||
**The floor-raising traps still apply** to any future sunset: four plugins
|
||
declare their floor top-level, where editing `versions[0]` is a silent no-op,
|
||
and the floor has three live spellings (`min_ledmatrix_version`,
|
||
`requires.min_ledmatrix_version`, `versions[].ledmatrix_min_version`, plus
|
||
deprecated `ledmatrix_min`). See
|
||
`src/plugin_system/compatibility.py:declared_min_version` for the resolution
|
||
order any floor-raising tool must reproduce — and note the name is **inverted**
|
||
between the top level and `versions[]`.
|
||
|
||
**Still not adopted, deliberately:** `data_sources.py`, `game_renderer.py` and
|
||
`base_odds_manager.py`. The standing decision held them until B6 closed; it now
|
||
has, so they can be reconsidered — with B5's lesson applied, which is to build
|
||
the object and diff rendered output rather than trust a static check.
|
||
|
||
### B5 retrospective — what the adoption actually cost
|
||
|
||
Recorded because it is the evidence behind the two decisions above, and because
|
||
"the adoption went fine" is not what happened.
|
||
|
||
**Four of the eight shipped with scroll mode broken** on a 3.2.0 core, and were
|
||
repaired in plugins-repo #251. The restructure lifted the content methods
|
||
verbatim but left the state they read off `self` behind: separator-icon
|
||
constants (hockey, basketball, lacrosse) and the game-renderer cache (afl).
|
||
hockey/basketball/lacrosse could not construct the scroll display at all; afl
|
||
raised inside `prepare_scroll_content`, which the core base *catches*, so its
|
||
only symptom was scroll mode silently drawing nothing.
|
||
|
||
Three things are worth carrying forward:
|
||
|
||
- **The bundled fallback did not protect anyone from this.** The break was on
|
||
the modern path, which the fallback never touches. Carrying the second copy
|
||
bought nothing against the actual defect while creating the divergence that
|
||
produced it. That is an argument *for* B6, not against it.
|
||
- **Every gate was green.** The safety harness renders the scoreboard screens,
|
||
not scroll mode; `test_core_fallback.py` checked that methods existed and that
|
||
their *globals* resolved, and `self.NHL_SEPARATOR_ICON` is an attribute read,
|
||
invisible to an AST scan for `Name` loads. The fix was to stop reasoning about
|
||
source and **build the object**: construct both classes on both paths, compare
|
||
the separator icons they end up with, and assert the adopted class ends up
|
||
with every instance attribute the bundled one sets.
|
||
- **Test what the change touches, not what is convenient to render.** Scroll
|
||
mode had no coverage because the harness could not reach it. A comparison
|
||
harness that renders the same games through both paths and diffs the pixels
|
||
needs no per-sport knowledge of the right answer, only that adopting core code
|
||
did not change it.
|
||
|
||
**The ledger.** Before adoption, eight duplicated copies totalled 5,685 lines.
|
||
After adoption plus the frozen legacy copies it was 10,610; removing the dead
|
||
inline duplication (plugins #252) brought it to roughly 8,620. B6 would take it
|
||
to about 3,300 including the shared core module — some 2,400 fewer than before
|
||
this project started. **Until B6 runs, the adoption is net negative on disk**,
|
||
and its one delivered user-visible gain was that adopted plugins honoured the
|
||
global `target_fps` instead of hardcoding ~100 FPS (since withdrawn: see the
|
||
note under the B3 design above).
|
||
|
||
### Decision: stop adopting further modules until B6 closes
|
||
|
||
`data_sources.py` (9 copies), `game_renderer.py` (8) and `base_odds_manager.py`
|
||
are the obvious next candidates. **Do not adopt them yet.** Each adoption adds
|
||
carrying cost — a second copy to keep in step — against a payoff that is
|
||
contingent on B6, and B6 is gated on an installed base we cannot currently
|
||
measure. Consolidate what is already committed; revisit when B6 does.
|
||
|
||
## What's next
|
||
|
||
Steps 1–5 of the original plan are **done**: 3.2.0 is tagged and published with
|
||
a version number CI now asserts (#428), the compatibility gate is in
|
||
`install_plugin` and reads `compatible_versions` as well as the floor
|
||
(#431, #433), the newest manifest entry is required to use `ledmatrix_min_version`
|
||
(plugins #244), and all eight plugins have adopted the scroll orchestration
|
||
(plugins #245–#249, repaired in #251, tidied in #252).
|
||
|
||
What actually remains, smallest first:
|
||
|
||
1. **Soak the adoptions on hardware.** football and hockey have been run on a
|
||
live rig through real games; baseball was watched through one earlier. The
|
||
rest are proven by harness, unit tests and pixel comparison. Out-of-season
|
||
sports cannot be soaked until their season starts. When you do, **check the
|
||
rig's `*_display_mode` first** — a board in `switch` mode will happily load a
|
||
sunset plugin and tell you nothing about the scroll code the sunset changed.
|
||
2. **Cut 3.3.0.** Not required by B6 — its floors are 3.2.0, which is released —
|
||
but `calendar` 1.2.3 floors at 3.3.0 for the device-authorization endpoints
|
||
that landed after 3.2.0, so it is un-installable until the release exists.
|
||
3. **Reconsider the held modules** (`data_sources.py`, `game_renderer.py`,
|
||
`base_odds_manager.py`) now that the sunset has closed. `game_renderer.py` is
|
||
the largest single duplication left: ~11,500 lines across eight plugins, with
|
||
~36,500 more in the eight `sports.py`. Note that core already ships
|
||
`src/base_classes/sports/` (~143KB, promoted in B1/B2) that **no plugin
|
||
imports** — check whether it has drifted before treating it as the target.
|
||
|
||
## How to keep this project healthy
|
||
|
||
Lessons this migration paid for, worth applying beyond it:
|
||
|
||
- **A version number is a promise; keep it in one place.** Three different
|
||
answers to "what version am I on" (tag, release, `__version__`) is what made
|
||
the floor untrustworthy. Assert their agreement mechanically.
|
||
- **Advisory checks protect nobody.** If a rule matters, enforce it where the
|
||
action happens — the install path, not a log line the user will never read.
|
||
If it doesn't matter enough to enforce, don't write the rule.
|
||
- **Prefer failures that are loud and early.** A plugin that dies at load with
|
||
one journal line is indistinguishable, to a user, from a plugin that was never
|
||
installed. Surface plugin health in the UI.
|
||
- **Keep the two repos' rules in sync deliberately.** The sunset rule lives in
|
||
both this file and the plugins repo's
|
||
`docs/plugin-development/08-shared-sports-code.md`. When one changes, change
|
||
the other in the same PR — drift between them is how a contributor ends up
|
||
following a rule that was superseded.
|
||
- **Measure before and after, on real hardware.** Byte-identical harness renders
|
||
and a device soak caught what unit tests could not. Reserve "it should be
|
||
fine" for things you have actually looked at.
|
||
|
||
## Rules for contributors
|
||
|
||
- **Promote on evidence, not intuition.** A method moves to core when every copy
|
||
has it and they agree on intent. Otherwise it stays in the plugins.
|
||
- **Never add a sport name to core.** If core needs to know which sport it is,
|
||
the design is wrong — add an override point instead.
|
||
- **A capability that is not opted into must not execute.** If you find yourself
|
||
writing `if self.<capability>_enabled` inside a base class, it belongs in a
|
||
mixin.
|
||
- **Touch the view-model keys only additively.** Published skins depend on them.
|
||
- **Every promotion lands with the characterization suite green**, and every
|
||
pilot adoption lands with that plugin's harness and golden suites green.
|