Files
LEDMatrix/docs/SPORTS_UNIFICATION.md
T
ChuckandClaude Opus 5 116abb0daa fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595)
* fix(sports): share the ESPN rejected-range memo with the background service

BackgroundDataService always sent a season range first and, on a 400,
fell back to chunks without recording the rejection, so every background
season fetch spent a doomed request and live scoreboards learned nothing
from it (or it from them). The worker now consults and sets the same
6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight
to month/day chunks, and if every chunk fails the range is asked once for
a real error without re-spending the chunks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): keep plugin asset and action routes inside their directories

POST /plugins/assets/upload, GET /plugins/assets/list and POST
/plugins/assets/delete joined the request's plugin_id onto assets/plugins
unchecked, so '../../config' created, wrote, listed and deleted outside
it. #561 guarded only the route that serves the files. All three now go
through path_safety.resolve_under and answer 400 for anything but a
plain name, and delete only unlinks a metadata path that resolves into
that plugin's uploads directory.

PluginManager.get_plugin_directory refuses ids that are not one plain
path segment, so /plugins/action (which runs a manifest script from the
returned directory) and every other caller get the guard; the action
route also rejects such ids up front, covering its no-manager fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): report a no-op plugin update as already up to date

update_plugin() returns True both for a real update and for "nothing to
do" (a ZIP-installed monorepo plugin already at the registry version, a
bundled plugin). With no git commit to compare, POST /plugins/update
called every such success "updated successfully", so Check & Update All
counted most official plugins as updated on every run.

The route now reads what changed off the plugin itself (commit, else
manifest version, else last_updated) and returns data.update_status
(updated / up_to_date / local_only). The update-all toast is summarised
by PluginInstallManager.summarizeUpdateResults from that status, falling
back to the message for older servers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(sports): scoreboard scroll speed no longer follows target_fps

sports_scroll computed the crisp speed ladder against the global
target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since
frame-locked presentation (#545) the helper steps a fixed number of whole
pixels per presented frame and the panel presents at its real refresh, so
the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a
50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s.

The ladder now uses the display manager's refresh_hz, then
display.hardware.limit_refresh_rate_hz, then the default. target_fps is
not consulted. Docstrings now say scroll_delay is ignored for pacing (no
behaviour change there) and describe the fixed-step model.

Tests: replace the tests that pinned target_fps as the ladder refresh and
described time-based stepping; assert speed independence from target_fps
(unit and end-to-end presented px/s against the real helper), that the
fixed per-frame step is applied, and that scroll_delay does not change
speed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): escape registry and upload values in plugin manager inline handlers

The store, saved-repository and custom-registry buttons built
onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone,
so a custom registry entry whose id contained ' closed the attribute and
added its own handler. One helper, jsStringAttr(), now HTML-escapes the
JSON literal for every one of those handlers, and the store View button
opens only http(s) repo links.

The live window.updateImageList (plugins_manager.js loads last, so its
copy wins over the file-upload widget's) wrote the uploaded file's
original name, path and ids into markup raw; they are escaped now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note plugin asset, action and inline handler guards

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): let the root pip wrapper install web_interface/requirements.txt

Update Code, the automatic update's health check and Install Base
Requirements install web_interface/requirements.txt through
safe_pip_install.sh, which only allowed the root requirements.txt. The
first commit changing that file would fail its dependency install, and
the automatic updater rolls back any update whose dependencies did not
install -- on every device, for every newer commit.

The wrapper now lists both core requirement files. Only their folders
are resolved, so a requirements.txt symlinked out of the project is
compared by its target and refused (previously the root file's own
symlink target was what got allowed). The updater's file list is a
named constant, and a test runs the real wrapper (pip stubbed) on
every file Update Code and the rollback install.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): do not retry plugin requests that got an HTTP answer

PluginAPI.request wrapped everything that was not a structured error as
NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a
JSON error without error_code included. Check & Update All retries
NETWORK_ERROR, so those updates were re-sent five more times with
backoff, contrary to the #587 contract that an HTTP error response is
the server's answer.

NETWORK_ERROR now means only that fetch() rejected. Any HTTP response
without an error_code, or with a body that is not JSON, is API_ERROR
with the HTTP status attached. Tested against the shipped api_client.js.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): restart the stats window when an idle gap is dropped by size

#582 dropped an idle gap from the frame stats two ways: the reset_scroll()
sentinel, which also restarts the 5s window timer, and a size guard for
scrollers that never call reset_scroll(), which did not. On that path the
first real frame after the gap found the boundary overdue and logged a
stats line for a one-frame window. Both paths now share one seeding helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): leave plugins alone when update_core's own rollback fails

update_core returns rollback_failed directly when a partial pull or an
update whose health check never started cannot be rolled back. run()
only held plugins back for 'verifying', so those devices still got new
plugin versions and a display restart on top of a core in an unknown
state -- the opposite of what the health-check path does, and of the
3.4.0 changelog (plugins are left alone if the rollback fails).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(api): make the REST reference match the api_v3 package

Every documented request body, query parameter and response shape was
re-checked against the handlers in web_interface/blueprints/api_v3/.
Fixes calls that failed as documented (repo_url, action_id/params,
files/image_id, font_file+font_family, ?font=, cache key,
auto_enable_ap_mode, plugin limit keys), removes the font-override
endpoints dropped in #566, corrects response shapes (plugins/config,
plugins/schema, health, metrics, operation history, github-status,
fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds
the 26 routes it omitted (backup, system auto-update/git, wifi radio,
starlark editor, MQTT bridge, status endpoints, skins).

Documents the merge semantics of partial JSON saves to /config/main and
/plugins/config and the dim-schedule POST accepting GET's days shape,
which land in the same change set. Replaces app.py line numbers and the
removed api_v3.py path with file and function names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): remove the General-tab plugin system toggles that did nothing

plugin_system.auto_discover, auto_load_enabled and development_mode had
General-tab toggles whose help tips promised dormant plugins and verbose
logging, but nothing reads them: every enabled plugin is discovered and
loaded regardless. Remove the three toggles.

The keys stay tolerated in stored configs. The save handler now stores
a flag only when a client sends it; treating a missing key as an
unchecked box would otherwise rewrite all three to false on every
General-tab save, which still posts plugins_directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(scroll): remove dead code left by #523/#570

- Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read
  them since the numpy blend replaced the scipy path.
- Drop ScrollHelper._last_integer_position and frame_time_target, which
  were written but never read.
- Keep target_fps and set_target_fps() but document them as
  informational: nothing paces off them, yet ledmatrix-elections'
  test_scroll_pacing.py reads helper.target_fps back and third-party
  plugins may call the setter.
- Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to
  throttle", set_sub_pixel_scrolling's "default: True", and
  set_frame_based_scrolling's claim that it steps.

The plugins monorepo was grepped for every removed name; none is used.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI

FONT_MANAGER.md told plugins to read display_manager.font_manager, which
does not exist, so a plugin following it failed to load with
AttributeError. The shared FontManager lives on the PluginManager and
BasePlugin._get_font_manager() returns it (with a fallback for harnesses).

Also removes the Fonts-tab override workflow and element-override panels
that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls

/plugins/store/search does not exist (404) and the list endpoint reads
query, not q. The registry guide's curl examples omitted the JSON
Content-Type, so the handlers saw an empty body and answered 400.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(config): use the shared core-key list in the last three private copies

StartupValidator warned "Plugin 'auto_update' is enabled but not found" on
every display start with auto-update or a dim schedule on; the reserved
plugin-id check missed auto_update, sync, location and the rest; and
ConfigManager's (uncalled) orphan cleanup would have deleted display,
schedule and auto_update. All three now read src/core_config_keys.py, which
also gains CORE_SECRETS_KEYS for the github/youtube secrets sections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): partial JSON saves to /config/main change only what they send

A JSON body with one field reset every checkbox in the sections it touched:
the MQTT bridge's brightness slider turned off disable_hardware_pulsing,
inverse_colors, show_refresh_rate and use_short_date_format, and a
timezone-only save turned off web-UI autostart and weekly auto-updates.
Missing-means-unchecked now applies only to form posts: form-encoded bodies
and the v3 forms, which mark themselves with a hidden __form_section input.

Also on the config routes:
- vegas_min/max_cycle_duration no longer match the generic *_duration rule,
  so they stop landing in display_durations and a blank one no longer
  rejects the whole Display save;
- saving from the Raw JSON editor calls start_setup_if_needed like the
  General form, so enabling auto-update there finishes its setup;
- the schedule and dim-schedule POSTs accept the per-day days.<day> shape
  their GETs return, as well as the flat form keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): install plugin dependencies from the configured plugins directory

install_plugin_dependencies.sh scanned only plugins/, but the Plugin
Store installs into plugin_system.plugins_directory (default
plugin-repos), so the documented "Recommended" fix found 0 plugins on
every store install. It now reads plugins_directory from
config/config.json (relative to the project root or absolute, default
plugin-repos) and also scans plugins/ for dev symlinks, installing a
plugin reached through both only once.

With set -e alone, `pip ... | tee` took tee's exit status, so a failed
pip install was reported as success; set -o pipefail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: replace stale API names, line numbers and the api_v3.py path

- ADVANCED_FEATURES: StreamManager methods that exist
  (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the
  real on-demand status envelope ({status, data: {state, service}})
- app.py:199 / :144 / :607-619 line citations and
  web_interface/blueprints/api_v3.py (now a package) replaced with file and
  function names in ADVANCED_FEATURES, CONFIG_DEBUGGING,
  PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE,
  PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README
- CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use
  /config/raw/main to replace the file; describe where validation runs
- TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints
  usage)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): verify the web interface that actually ships, on port 5000

verify_installation.sh failed every healthy install: it required the
long-removed web_interface_v2.py and looked for a listener on port 5001,
while the web interface binds 5000 (web_interface/start.py). It now
checks the files ledmatrix-web.service runs (start_web_conditionally.py,
web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the
same 5001 port in its listen check, HTTP probe and printed URLs.

Port matches are anchored so :50001 no longer counts as :5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): one display-size contract: display_manager.width/height

CLAUDE.md (#580) says to read display_manager.width/height because
matrix is None when hardware init fails; the development guide, the
safety-harness doc and two DisplayManager docstrings still recommended
matrix.width/height. The bundled starlark-apps plugin read matrix.width
unguarded, so its magnify recommendation and frame scaling raised in
fallback mode (e.g. after the Pi 5 hardware refusal).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(install): make install_service.sh --help print usage instead of installing

install_service.sh parsed no arguments, so `sudo ./scripts/install/
install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md)
rewrote ledmatrix.service, ledmatrix-web.service and both update-verify
units and enabled/started them. It now handles -h/--help (usage, exit 0,
no changes) and rejects any other argument with exit 2 before doing
anything. Running it with no arguments, as first_time_install.sh does,
is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scroll): describe the fixed-step model and document frame_hold

Since #545 a crisp speed from scroll_config.configure() makes the helper
advance a fixed whole-pixel step per presented frame with no clock, and the
display manager's frame hold is part of the speed. The docs still described
the removed wall-clock model:

- scroll_config's module and configure() docstrings said speed is applied
  in time-based mode and that omitting the hold "falls back to fractional
  pixels"; omitting it actually runs the scroll frame_hold times too fast.
- SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both
  modes, and read a 20 ms stats median as missed refreshes although that
  is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step,
  the hold-dependent healthy median, that target_fps plays no part, and
  that a hand-added scroll_pixels_per_second loses to a schema-default pair.
- PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling)
  without frame_hold; it now documents the parameter (core 3.4.0) with a
  configure() + set_scrolling_state example.
- update_scroll_position/set_scroll_speed and set_scrolling_state
  docstrings say the same.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark target_fps legacy; describe what Vegas scroll_delay does

- General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after
  the sports_scroll fix nothing in core scrolling reads it. The field and
  its API validation stay so saved configs and plugins that read
  global_config['target_fps'] keep working. CONFIG_REFERENCE says the same.
- Vegas frame_based_scrolling/scroll_delay were described as frame-count
  stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based
  mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and
  still advances by elapsed time, so the applied speed is
  clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The
  config comments, render_pipeline comment and CONFIG_REFERENCE rows now
  say so. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(deps): describe how plugin dependencies are really installed

The guides said the web service runs as root, that installs pick --user
from os.geteuid(), and quoted a warning and a
PluginManager._install_plugin_dependencies() method that don't exist. The
web unit runs as the installing user; store installs go through
install_requirements_file() and sudo safe_pip_install.sh (root), with a
user-level fallback that says so, and load-time installs run in the
display service's own (root) interpreter.

Manual paths now use the configured plugins directory (plugin-repos/ by
default) instead of plugins/, which store installs no longer use, and
install_plugin_dependencies.sh is described as scanning that directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): count local changes one way for the preflight and the pull

The automatic update's preflight ignored mode-only changes and anything
whose status line contained plugins/ or plugin-repos/, then promised
"Automatic updates will not stash your changes". perform_core_update
used plain git status (modes count) and ignored only 'plugins/', then
ran 'git stash push -- :!plugins', which nothing ever pops. So an edit
to a bundled plugin under plugin-repos/, or the installer's chmods on
tracked scripts, passed the preflight and was stashed away for good.

- auto_update.local_changes() is the one predicate both use:
  core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/
  excluded by leading folder rather than substring (a core file under
  web_interface/static/v3/js/plugins/ now counts).
- Update Code's explicit stash leaves out both plugin folders; the
  pull's --autostash carries their edits and mode changes across and
  reapplies them.
- The automatic updater calls perform_core_update(stash_local_changes=
  False), which refuses instead of stashing edits that appeared after
  the preflight; update_core reports that as 'blocked'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): diagnostics follow the web autostart default and api_v3 package

#556 made a missing web_display_autostart mean "start" (only an explicit
false/off keeps the web interface down), but the diagnostics still said
otherwise: diagnose_web_ui.sh reported a missing key as "defaults to
false", diagnose_web_interface.sh said the web interface "will not start
unless this is set to true" and recommended enabling it, and
debug_web_manual.py printed False. Troubleshooting a down web UI pointed
users at a non-cause.

Both shell scripts now evaluate the setting with the launcher's own
autostart_enabled() (inline fallback if it cannot be imported) and report
on / off / not set (on) / unparseable config; debug_web_manual.py uses
the same function. They also check web_interface/blueprints/api_v3/
__init__.py: api_v3.py became a package in #553, so every healthy
checkout was reported as missing a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(install): what install_service.sh installs; verify script port; no sudo for --help

install_service.sh installs and starts ledmatrix, ledmatrix-web and the
update-verify units, not only ledmatrix.service (systemd/README.md,
README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help'
as a harmless check; it now shows --help without sudo and warns what a
real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh
checks the web interface on port 5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note update-all, plugin system settings and script fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): size the preview after orientation and pixel mappers

display_geometry.physical_size claimed to give DisplayManager's answer but
only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured
after the library's pixel mappers, so a Rotate:90 / orientation 90 chain
previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for
128x64.

Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc
does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown
names leave it alone), and move the orientation composition here so
DisplayManager and the preview share it. The module docstring no longer
claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH.

Audit finding F18.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): refuse settings the rgbmatrix library aborts on, on every board

The library answers several settings with a NULL matrix or abort() rather
than an error, so the display service crash-looped (Restart=on-failure)
instead of reaching fallback mode: rows above 64, chain_length above 255
(uint8_t binding setter, documented as "no upper limit"), a misspelled
hardware_mapping, and parallel 2-3 on a single-output mapping, reachable
from the Display form on the default adafruit-hat(-pwm) mapping. #586 only
guarded the Pi 5 subset.

- src/matrix_support.py holds the rules for every board (Options::Validate
  ranges, binding integer types, mapping names and outputs from
  lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the
  API's numeric ranges.
- DisplayManager checks them before building options and raises
  MatrixSettingsRefused, so a hand-edited config falls back with a logged,
  reported reason. Emulator mode only warns.
- The config API refuses them with a 400 naming the setting; combinations
  are checked against stored values but reported only when the request
  sets a field involved.
- The hardware status file gains "cause" (settings/library/forced). The
  fallback log and Display banner give the Pi 5 rebuild hint only for a
  library failure instead of rebuild + gpio_slowdown advice for every
  failure; one Pi 5 slowdown recommendation (1-3, start at 1).
- The Display form offers classic/classic-pi1 and orientation 90/270 and
  renders any other stored mapping selected with a warning, so an
  unrelated save no longer rewrites them; the API accepts 90/270.

Audit findings F03, F16, F19, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(display): library limits, template defaults and Pi 5 slowdown

- rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs,
  classic/classic-pi1 mappings and orientation 90/270 documented.
- Defaults are the config.template.json values: config migration adds
  missing keys from the template, so the listed "code defaults" never
  applied.
- One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode,
  starting at 1.
- Troubleshooting describes the refused-settings fallback, and CHANGELOG
  corrects the Unreleased "no upper limit" entry.

Audit findings F19, F20, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py opens the panel with the service's options

--measure and --demo built RGBMatrixOptions from a private copy of the
display service's builder that had drifted: gpio_slowdown came from
display.hardware (default 2) instead of display.runtime (default 3), and
rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors,
pixel_mapper_config and orientation were skipped, with different defaults
(hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown
was measured -- or garbled -- in a setup the service never drives.

The option filling in DisplayManager._setup_matrix moves, unchanged, into
DisplayManager.apply_matrix_options(options, config), which _setup_matrix
calls and the script reuses (overriding only limit_refresh_rate_hz for
--measure). The script now loads the whole config rather than the hardware
block. Tests pin the script's options to the service's attribute for
attribute.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py recommends keys the resolver honours

The ladder ended by telling users to set
display_options.scroll_pixels_per_second. scroll_config ranks that key
below the scroll_speed + scroll_delay pair, deliberately, and several
plugin schemas default the pair into config, so the advised key was
silently ignored (a schema-default 1/0.02 pair plus an advised 66 still
resolved to 50 px/s).

The advice is now the pair that selects the crisp speed exactly
(pixels_per_frame every frame_hold/refresh seconds), explains that the
pair outranks scroll_pixels_per_second, and gives the scoreboards'
per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the
printed pair over a schema-default pair and check it lands on the
advertised speed and hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula

- SPORTS_UNIFICATION.md still presented honouring global target_fps as
  sports_scroll's added behaviour and its one user-visible gain; note that
  it was withdrawn because it had become a speed multiplier.
- ADVANCED_FEATURES.md gave Vegas scrolling as
  (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s
  by elapsed time, through a 0.1-5 px per scroll_delay clamp when
  frame_based_scrolling is on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): scroll model fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(dev): link-github links plugins from the ledmatrix-plugins monorepo

link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git,
and those per-plugin repositories no longer exist: official plugins are
directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the
monorepo once into the dev directory, finds plugins/<name>,
plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and
links it under its manifest id. With an explicit repo URL it still links a
single-repository plugin as before.

dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a
fork), plus plugins_repo and plugins_branch; github_pattern, which was
documented but never read, is dropped and warned about. Ships
dev_plugins.json.example and git-ignores dev_plugins.json, both of which
the guide promised. Reading JSON falls back to python3 when jq is missing
(get_plugin_id silently returned nothing without jq).

update/status/list find the git checkout above a monorepo plugin
directory (its .git is not in the plugin dir), and update pulls a shared
checkout once. status no longer exits 1 when nothing is broken.

Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration,
workflow, store integration, hello-world link, submission), and the
nonexistent scripts/git-hooks/pre-push-plugin-version and
scripts/bump_plugin_version.py replaced with the real rule: bump the
manifest version and run update_registry.py. scripts/dev/README.md and
CLAUDE.md updated to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scripts): monorepo workspace layout; fix_perms and install READMEs

MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin;
setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into
plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the
workspace file opens LEDMatrix plus ../ledmatrix-plugins.

scripts/fix_perms/README.md listed cache directories
fix_cache_permissions.sh never touches and a 'ledmatrix' service user
that doesn't exist (also in scripts/install/README.md); adds
safe_pip_install.sh. install/README: install_service.sh installs the web
and update-verify units too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): keep the rollback's pip retries inside the unit time limit

The health check reinstalled the previous requirements by trying the
next bash path after any failure, including a 600 s pip timeout. Two
files, two paths: up to 40 minutes of pip alone, while systemd stops
ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the
rollback half-way and leaving the update 'verifying' until the web UI
calls it lost.

- Like permission_utils.install_requirements_file, only a sudo refusal
  moves on to the next bash; a pip that ran and failed or timed out is
  not repeated. The refusal wording is one list
  (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only
  verifier and pinned equal by a test.
- All reinstalls in one rollback share a 600 s budget.
- WORST_CASE_SECONDS adds up every timeout on the longest path (27.5
  min); a test holds it under the unit's TimeoutStartSec and that under
  the web UI's VERIFY_LOST_SECONDS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools

Plugin config was prepared differently depending on how it arrived:

- JSON POST /plugins/config built a partial body on schema defaults, so
  {"enabled": true} reset every other setting of the plugin. It now merges
  onto the stored section first, as the form path already did.
- Legacy-boolean normalization (#588) ran only at load: GET /plugins/config
  returned the raw boolean, posting it back failed validation, and hot
  reload handed plugins the raw section (a legacy dynamic_duration: true
  came back as a boolean). schema_manager.prepare_plugin_config (normalize,
  then defaults) is now used by PluginManager.load_plugin, both save paths,
  GET, the save notifications and DisplayController's hot-reload callback.
- The JSON save's filter kept only enabled/display_duration/live_priority
  and dropped a submitted skin, skin_options or vegas_* tuning key. There
  is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES,
  used by validation and by the save filter; PluginManager's
  CORE_OWNED_CONFIG_KEYS is its vegas subset.
- Plugin sections posted to /config/main were stored verbatim, including
  values /plugins/config rejects. They now go through the same preparation
  (_prepare_plugin_config_for_save, extracted from save_plugin_config), and
  a failing section rejects the whole save before anything is written.
- dev_server read only top-level defaults and let a schema enabled:false
  win; build_full_config shallow-merged overrides, dropping sibling
  defaults; the harness extracted defaults differently from the device.
  loading.build_config now uses the device's extraction and preparation,
  and dev_server, check_plugin, render_plugin and the harness all use it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt-bridge): brightness changes apply live and touch nothing else

The display service's hot reload applies a saved brightness within a few
seconds, and /config/main no longer resets other display settings on a
brightness-only JSON body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): automatic update hardening

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI

It described web_interface_v2.py and index_v2.html (both gone), client-side
form generation, one POST per field with {key, value}, and 'no nested
objects'. The v3 UI renders plugin forms server-side from the schema
(pages_v3 partial + plugin_config.html macros, nested sections and
x-widgets), posts the whole form once, and save_plugin_config() merges onto
the stored section, validates, splits x-secret fields and notifies the
plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt): brightness saves apply via hot reload and leave other settings alone

The bridge README said brightness is applied on the display's next
restart; the display controller's config hot reload applies it within
seconds. It also now states that the bridge's partial JSON save changes
only brightness (the /config/main merge fix in this change set).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): don't log pip's output from the health check's reinstall

pip can echo a private index URL with embedded credentials;
permission_utils redacts it, the stdlib-only verifier cannot, so it
logs the exit code only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark the plugin_system toggles as unused legacy keys

auto_discover, auto_load_enabled and development_mode are read by
nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE
said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and
the REST reference listed them as live settings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): docs and developer tools group

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): legacy plugin-system toggles no longer count as a General save

auto_discover, auto_load_enabled and development_mode have left the General
form, so a post carrying only one of them is not a general-settings save and
must not treat web_display_autostart and auto_update as unchecked. The
plugin_system block itself is left as on main for the branch that reworks it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): config-save and plugin-config preparation fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(claude): re-check matrix_support.py rules when the library submodule is bumped

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address Codacy findings on the core audit PR

- plugin_manager.prepare_plugin_config: when the fallback legacy-boolean
  pass also fails, log a warning instead of a bare except/pass.
- api_client.js: request() refuses any endpoint that is not a plain path
  under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace,
  control characters) with INVALID_ENDPOINT before calling fetch(), and
  plugin ids are URL-encoded wherever they are put into a URL (also in the
  app-shell batch load).
- test_update_all.js: pins both against the shipped client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): check endpoint control characters without a control-character regex

Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags
the \x00-\x1f range in checkEndpoint's regex. Test the char codes
instead; the endpoints refused are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(auto-update): make the seed script executable on disk, not only in the index

On Linux Repo.publish() commits with -a, which recorded scripts/run.sh
as 100644 upstream because the seed file was never chmod +x. The pull
then brought in the same mode the installer chmod had made locally, so
installer_chmod saw no mode change left to check. The updater was fine:
with the upstream commit at 100755 the --autostash carries the device's
chmod across. Verified under Linux (WSL, git 2.43): the old helper fails
exactly as CI did, the fixed one passes all 63 tests in the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 16:37:29 -04:00

32 KiB
Raw Blame History

Sports Code Unification — Architecture

How the nine sports scoreboard plugins converge onto shared core code without becoming nine clients of a god class.

The problem

Nine plugins (afl, baseball, basketball, football, hockey, lacrosse, nrl, soccer, ufc) each ship a ~3,000-line sports.py descended from this repo's src/base_classes/sports.py. They have drifted into three lineages, and only 28 of the 66 methods appearing across them are present in all nine. One logical fix (the UTC start-time bug) cost 75 files.

Merging everything into one base class would fix the duplication and create a worse problem: a single 2,500-line class that all nine plugins inherit, where any change has a nine-plugin blast radius and per-sport behavior survives only as if self.sport == "hockey" branches.

Three properties, three mechanisms

These are independent concerns. Conflating them is what produces god classes.

Upgradability — a plugin keeps working across core versions

Rule Mechanism
Plugin loads on a core that predates a module Guarded import with a bundled fallback (try: from src.X import Y / except ModuleNotFoundError: from y import Y)
Plugin loads on a core that predates a method Capability probing — hasattr(SportsCore, "_detect_stale_games") — never a version comparison. The loader's compat check is advisory-only (it logs and continues), so probing is the real protection.
Core changes never break a plugin's rendering The view-model contract: _extract_game_details_common returns a dict whose GUARANTEED_KEYS are frozen by test/test_skin_system.py::TestViewModelContract. Keys may be added, never renamed or removed.
A plugin can drop its bundled copy safely The sunset rule: its manifest must floor ledmatrix_min_version at the first core release shipping the module (recorded in CHANGELOG.md) — necessary but not sufficient. The store enforces that floor on every registry-managed install and on both supported update paths (sideloading via install_from_url is not gated), but a floor cannot reach a user who never updates, so the copy also waits for the B6 gate below.

The core API is additive-only. A method the plugins call is never removed or given a new required parameter; new behavior arrives as new methods with defaults, or as capabilities they opt into.

Reusability — write once, nine plugins benefit

Only code that is identical in intent across all nine moves into the base class. That set is small and knowable — it is exactly the methods present in every copy today (phase B1 below). Everything else stays where it is until it earns promotion.

Modularity — a change to one feature cannot reach a plugin that doesn't use it

This is the property the naive merge destroys, and it is enforced structurally:

  1. Capabilities are separate modules composed by inheritance, not config branches inside the base class. Hockey has no celebrations, so HockeyLive does not inherit CelebrationMixin — the celebration code is not merely disabled for hockey, it is not in hockey's MRO at all. No shared state, no dead branches, no risk. Contrast with if self.celebrations_enabled: inside SportsLive, where a bug in celebration code can still crash a plugin that never wanted the feature.

  2. Variant behavior is a strategy object chosen by name, not a branch. Live rotation exists in three dialects across the lineages; core ships all three behind rotation_strategy: "swrr" | "weighted" | "simple" and a plugin may register its own. Core never learns sport names.

  3. Sport-specific behavior is a documented override point. The base class declares the seam; the plugin fills it. Basketball's tournament-round parsing and baseball's BDF sizing stay in their plugins forever — they are not candidates for promotion, and core must never grow a branch for them.

  4. Files bound the blast radius. Capabilities live in their own modules so a diff shows at a glance which plugins a change can reach.

Layering

src/base_classes/sports/
  __init__.py            re-exports the public API (import path unchanged)
  core.py                SportsCore — fetch, cache, config, logos, fonts, odds,
                         view-model extraction, the skin seam
  modes.py               SportsUpcoming / SportsRecent / SportsLive
  capabilities/
    celebrations.py      CelebrationMixin        (opt-in: 4 of 9 plugins)
    rotation.py          RotationStrategy + registry

src/common/
  sports_scroll.py       SportsScrollDisplay / …Manager — scroll orchestration
                         (content building stays in the plugins)
  sports_helpers.py      clamp/logo/rotation free functions + SportsHelpersMixin
                         (unreleased) — the helpers byte-identical in the
                         plugins' sports.py, and the _favorite_key seam

from src.base_classes.sports import SportsCore keeps working — the package __init__ re-exports, so the conversion is invisible to every existing importer.

Converging on src/common

The scoreboards do not build on src/base_classes; their own sports.py copies have moved past it. So shared code now lands in hardware-free src/common modules taken from the plugin copies, each a new module rather than growth on an existing one: a plugin that deletes a method copy and relies on an older module having gained it fails at runtime with an AttributeError, while a missing module fails at load, where the version checks can see it. sports_helpers.py is the first (it holds _favorite_key, the override point listed below, for later phases); its parity test compares every body against the plugin copies when LEDMATRIX_PLUGINS points at a checkout, and test/test_common_is_hardware_free.py keeps src/common free of rgbmatrix, src.base_classes and src.plugin_system. How a plugin adopts a module and drops its copy is documented in the plugins repo's docs/plugin-development/08-shared-sports-code.md.

Override points (the plugin-facing seam)

The base class calls these; plugins implement or override them. This table is the contract — additions require a default implementation, removals require a deprecation cycle.

Hook Purpose Default
_fetch_data() Sport's schedule source abstract
_extract_game_details(event) Sport-specific view-model fields on top of the common ones delegates to _extract_game_details_common
_draw_scorebug_layout(game, force_clear) Sport's card rendering base layout
_custom_scorebug_layout(game, draw) Per-sport overlay on the base layout no-op
render_skin_card(game, size) Skin-system entry point built-in fallback
score_phrase(points, team_abbr) Celebration wording ("GOOOOAAALLL!" vs "TOUCHDOWN!"). points is the score delta, which sports with variable-value scores use to name the play "<abbr> SCORES!" — only consulted when CelebrationMixin is present
win_phrase(team_abbr) Win-celebration wording "<abbr> WINS!" — mixin only
_favorite_key(game, side) Which view-model field identifies a team for favorites matching game["<side>_abbr"]
_config_schema_path() Plugin's config_schema.json — returning it routes _get_layout_offset through the src.element_style resolver (and gives it the defaults to compare against) None, i.e. the classic inline customization.layout read
_font_root() Directory to resolve assets/fonts against core install root

Two class attributes serve the same purpose for values that are per-sport constants rather than behavior:

Attribute Meaning Default
FINAL_PERIOD Period at/after which a zero clock can mean "over" 4 (hockey overrides to 3)
CLOCK_COUNTS_DOWN Whether 0:00 means "expired" True (soccer/afl/nrl override to False — their clocks count up, so 0:00 is kickoff)
COALESCE_SCORING_SEQUENCE Fold score increments arriving during an active celebration into that one celebration False (football overrides to True — a touchdown lands as +6, then +1 for the extra point)

Why these are seams and not branches

_favorite_key exists because NRL abbreviations are not unique — "NEW" is both Newcastle Knights and New Zealand Warriors, "CAN" both Canberra and Canterbury — so NRL matches favorites on team ID. Flattening every plugin to abbreviations would silently select the wrong club for NRL users. The base declares the seam, NRL fills it, and core never learns the string "nrl".

CLOCK_COUNTS_DOWN exists for the same reason in the opposite direction: a soccer clock reading 0:00 means the match has not kicked off, so running the clock-expiry branch there would evict live games.

COALESCE_SCORING_SEQUENCE is the third of the same kind. In football one scoring play arrives as two score updates, so the follow-up must be folded into the first celebration; in soccer two increments a few seconds apart are two real goals, and folding them would swallow one. Neither default is "right" — which is precisely why it is a declared per-sport constant rather than a hidden assumption baked into the shared body.

Capabilities

capabilities/
  celebrations.py   CelebrationMixin        opt-in: afl, nrl, soccer, football
  rotation.py       RotationStrategy + registry

CelebrationMixin merges the two dialects the lineages grew (_check_for_goal/celebrate_opponent_goals vs _check_for_score/celebrate_opponent_scores). Their bodies were identical apart from three things, each now a seam: wording (score_phrase), follow-up suppression (COALESCE_SCORING_SEQUENCE), and team identity (_favorite_key, so NRL matches on id). Both config spellings are read, so a plugin adopting the mixin keeps working with the keys already in its published schema.

Mix it in before the mode class — class SoccerLive(CelebrationMixin, SportsLive) — so the celebration display() runs first and falls through to the scorebug via super().

Rotation strategies. The three "dialects" turned out to be one algorithm (Smooth Weighted Round-Robin) in two shapes: an incremental picker holding state across calls (afl/nrl/soccer) and a precomputed per-cycle list (football/baseball/basketball, and hockey with a different loop shape). They agree within a cycle and differ only at the boundary — the incremental form has no restart seam — so core ships both rather than declaring a winner:

self.rotation = get_rotation_strategy("swrr", weight_for=self._live_weight)

weight_for is supplied by the host, so the favorites policy stays with the plugin and rotation.py never learns what a favorite is. An unknown strategy name degrades to simple rather than raising: the name comes from user config, and a typo should cost the boost, not the scoreboard. When a plugin needs an ordering that core does not ship, it calls register_rotation_strategy to add its own — rather than core growing a branch for it.

test_sports_capabilities.py checks each strategy against a verbatim transcription of the plugin code it replaces, over every live-game shape up to four games. That differential is what B5 deletes the bundled copies on the strength of.

Scroll display — where the promotion line falls

src/common/sports_scroll.py is deliberately not a superset of the ten scroll_display.py copies. A method-level comparison of the eight that share a shape (f1 and ufc are genuine forks) found a sharp split:

Layer Evidence Outcome
Orchestration — get_all_vegas_content_items, clear_all, get_scroll_info, get_dynamic_duration, is_complete, display_frame identical to 96–100% similar across all eight promoted
Settings — _get_scroll_settings one algorithm; the copies differ only in which league keys they walk promoted, with the ladder as data (SCROLL_LEAGUE_KEYS)
Content — prepare_scroll_content, _load_separator_icons 8 distinct bodies across 8 plugins (145 lines, 53% similar at worst); icons 6% override point, permanently

Same name, different job: prepare_scroll_content draws this sport's game card. Merging the eight bodies would be the exact mistake the promotion rule exists to prevent, so the base class raises NotImplementedError rather than rendering something plausible — a base that rendered something would let a plugin ship a silently blank scroll.

The one behavior the upstreamed version adds is native global_config['target_fps'] support. The bundled copies hardcode ~100 FPS via scroll_delay = 0.01 and never consult the global smooth-scrolling target; Part A threaded it through each copy by hand, and this makes that threading legacy compatibility rather than the mechanism.

Superseded. Once presentation became frame-locked (#545) the helper steps a fixed whole-pixel amount per presented frame and the panel presents at its own refresh, so honouring target_fps only turned it into a speed multiplier (60 doubled a scoreboard's speed, 200 halved it). sports_scroll no longer reads it: the crisp-speed ladder uses the panel refresh (display_manager.refresh_hz), and speed comes from scroll_settings.scroll_speed alone. See docs/SCROLL_PERFORMANCE.md.

Phases

B0–B3 are merged and shipping in core 3.2.0. Everything that remains is rollout, and it splits into three phases with very different risk profiles. The original plan folded the last two together; they are separated here because one of them cannot break a user on an old core and the other can.

Phase Scope Status Gate
B0 Characterization tests, CI unit job, element_style, font cwd fix, CHANGELOG discipline ✅ —
B1 Promote the nine universal methods; convert sports.py → package ✅ Characterization suite green; no behavior change intended
B2 CelebrationMixin + rotation strategies as opt-in capabilities ✅ Non-adopters have zero new code in their MRO; strategies checked against verbatim plugin transcriptions
B3 Upstream the scroll orchestration layer as src/common/sports_scroll.py, reading global_config['target_fps'] natively ✅ Content building stays per-sport
B4 Ship 3.2.0 and make version reporting trustworthy ✅ Released 2026-08-03; tag, release and src.__version__ agree; compatibility gate merged (#428, #431, #433)
B5 Adoption — guarded core imports, all eight. Bundled copies stay. ✅ All eight adopted; harness byte-identical; see the B5 retrospective below — four shipped broken and were repaired in plugins #251
B6 Sunset — delete the bundled copies ✅ Ran 2026-09-01, all eight. Floors at 3.2.0; the store refuses on all three routes in (#431/#433, #508, #510). See "B6 — what actually happened"

B4 — what "ship 3.2.0" actually requires

Cutting the tag is the small part. The version number has to become something a floor can be trusted against, and today it is not:

  • The tag and src.__version__ have never agreed. v3.1.0 was tagged 2026-05-31; __version__ only became "3.1.0" on 2026-07-12 (7f7f0d64). The v3.1.0 release therefore reports __version__ = "1.0.0".

  • Which silences the compatibility warning entirely for that population. PluginLoader._warn_if_incompatible skips the check when the parsed core version is below (2, 0, 0) — an anti-spam guard that, given the above, matches exactly the users most likely to be behind.

  • Nothing enforces a floor anyway. The check is advisory (it logs and continues), and neither StoreManager.install_plugin nor StoreManager.update_plugin compares the core version at all — update_plugin compares the plugin's manifest version against the registry's latest_version and nothing else.

    Fixed, in two parts. install_plugin gained the gate in #431/#433, which covers every registry-managed install and, through _reinstall_with_rollback, the update path that re-downloads. update_plugin's git branch pulls in place and re-downloads nothing, so it stayed ungated until _gate_pulled_commit closed it — checked after the pull (the registry carries no floor field, so the incoming floor is unknowable before it) and undone with git reset --hard to the pre-pull commit. That route is rare in practice, since monorepo plugins install as archives; it was closed because the sunset rule in the plugins repo's 08-shared-sports-code.md states as condition 3 that the core enforces the floor "at install/update time", and B6 rests on that being true rather than merely written down. install_from_url — sideloading a plugin from a URL — is still ungated.

So B4 is: tag and release 3.2.0; make the tag, the release, and __version__ agree, and keep them agreeing; reconsider the < 2.0.0 skip; migrate manifests from ledmatrix_min to ledmatrix_min_version; and add the install/update compatibility gate that B6 depends on.

Two fields express compatibility, and the gate only reads one

compatible_versions is the canonical contract: schema/manifest_schema.json requires it, all 42 published manifests carry it, and it holds semver ranges — [">=2.0.0"] in 41 of them, [">=1.0.0"] in 7-segment-clock. ledmatrix_min_version is the optional per-release floor inside versions[].

The gate as merged reads only the floor. Today that is harmless: no manifest uses an upper bound, and the two fields agree everywhere except 7-segment-clock (>=1.0.0 against a 2.0.0 floor). But the fields can disagree, and the range syntax the schema already permits includes upper bounds — a plugin declaring ["2.0.0 - 2.9.9"] means "not compatible with 3.x" and the gate would install it on 3.2.0 regardless.

Before B6, the gate must evaluate compatible_versions as well, and the manifest migration must reconcile the two fields rather than only renaming the floor. Deciding which wins when they disagree is part of that work; the safe default is the more restrictive.

(The schema also deprecates a top-level ledmatrix_version in favour of compatible_versions. No manifest still carries it, so there is nothing to migrate there.)

B5 — the fallback is safe by construction; the modern path is not

The heading matters, because the unqualified version of this claim is false and this document proves it two sections down: four of the eight adopted plugins shipped with scroll mode broken on a 3.2.0 core. What is safe by construction is narrower than "adoption".

A plugin adopting core imports keeps its bundled copy and reaches it through the guarded import (see the Upgradability table above). On a core that doesn't ship the module the plugin falls back and behaves exactly as it does today. That fallback compatibility — and only that — is safe by construction. On a core that does ship the module, correctness is not automatic — object-level and scroll-mode validation (building both classes and comparing, per the retrospective below) is required to prove full behavior. There is no version of this step that breaks a user on an old core, which is why it does not wait for B6's gate.

The hockey scroll-display pilot is already validated: adopted against a core carrying 3.2.0, scroll_display.py went from 691 to 289 lines and all 16 harness renders (8 sizes × 2 screens) came out byte-for-byte identical to the pre-adoption run. That byte-comparison is the acceptance gate for every adoption. The recipe and its two gotchas are in the plugins repo's docs/plugin-development/08-shared-sports-code.md.

B6 — why the sunset needs more than a version floor

Deleting a bundled copy removes the fallback, so the guarded import becomes a hard dependency. On a core without the module the plugin raises ModuleNotFoundError at load; PluginManager.load_plugin catches it, records PluginState.ERROR, logs one line, and continues. Nothing crashes — the user simply loses that scoreboard, with no visible explanation.

Verified against a v3.1.0 worktree: src/common/sports_scroll.py, src/element_style.py and the src/base_classes/sports/ package are all absent there, and the import fails with exc.name == 'src.common.sports_scroll'. Guard sets must name that exact dotted path — {"src"} alone does not match it.

Combined with the B4 findings, a plugin that deletes its copy today reaches an un-updated user through a normal store update, fails to load, and warns nobody. B6 therefore waits for B4's compatibility gate to have shipped and to have been in users' hands long enough that the population running a core without it is small. The bundled copies cost disk space; deleting them early costs scoreboards, silently. That trade is not close.

Before the first sunset, add a compatibility regression test. Built: core test/test_sports_sunset_matrix.py (#505). It has to cover four cases, not one — B5's safety claim and B6's failure mode are different propositions and only the second is obvious:

bundled copy present bundled copy removed
pinned old core loads — this is B5's whole guarantee, that the guarded import falls back PluginState.ERROR, and the recorded error names the exact missing module
current core loads, using core code loads, using core code

The top-left cell is the one worth writing first: nothing in the suite currently proves that an adopted plugin still works on a core that predates the module, which is the entire basis for saying B5 is safe to run ahead of the gate.

Assert the old-core/removed-copy case as PluginState.ERROR plus the missing module path, not as an uncaught exception. PluginManager.load_plugin catches ModuleNotFoundError, so nothing propagates — a test expecting a raise would pass for the wrong reason on a core where the module is merely broken rather than absent. "Fails loudly" is aspirational, not what the code does today: it fails into ERROR state with one log line, which is precisely why B6 needs the gate rather than trusting the failure to be noticed.

The same suite should exercise the install/update gate, since it is the other half of the guarantee.

B6 — what actually happened

Ran 2026-09-01, across all eight scoreboards. Held from 2026-08-05 to 2026-09-01 on the argument below, which is kept because the reasoning applies to the next module, not because it is still in force.

The hold, and why it lifted. The stated gate was evidence of 3.2.0 uptake — "a few months of it being the default download, or store-side install data". That evidence never arrived and could not: the core updates by git pull --rebase, so release-asset counts cannot measure uptake, and no store-side telemetry exists. What changed instead is that the risk the gate protected against was closed directly. The store now refuses a plugin whose floor exceeds the running core on all three routes in:

route gated by
install_plugin — every path that re-downloads, _reinstall_with_rollback included #431, #433
update_plugin's git branch — pulls in place, re-downloads nothing #508
install_from_url — sideloading #510

With all three closed a pre-3.2.0 user cannot receive a sunset plugin at all; they keep the version they already run. The population the hold existed to protect is protected by refusal rather than by a bundled copy — which is what the copy was standing in for.

What shipped. Eight plugins, ~5,800 lines of frozen fallback deleted. Each: copy removed, guarded import collapsed to a plain one, floor raised to 3.2.0, test_core_fallback.py rewritten as test_core_scroll.py asserting the sunset rather than the fallback. scripts/check_scroll_adoption.py gained sunset_violations and a SUNSET_PLUGINS set naming all eight, so a resurrected copy or a returned guard fails CI.

Delivered as plugins #346 (hockey, later folded into #351), #349 (football), #350 (baseball), #351 (the remaining six).

Two things found by doing it, both worth carrying forward:

  • Only one fallback held orchestration logic the core lacked. baseball's _configure_scroll_helper reinterpreted scroll_speed as pixels-per-frame when speed × delay fell outside the 0.1–5.0 window — measured, 10–20× faster than configured for a speed between 1.0 and 5.0. Standardised onto the core's behaviour (honour the documented px/sec, clamp) rather than preserved. Every other difference across the eight was a docstring, an unreachable scroll_helper is None guard, or an equivalent diagnostic.
  • Two tests had been leaning on the guard without anyone knowing. soccer/test_live_screens.py stubbed src in a way that shadowed the core, so its guarded import fell back and the test had been exercising the frozen copy rather than the shipping class since B5. Before sunsetting anything else that carries a guarded core import, grep for tests that stub src.

The floor-raising traps still apply to any future sunset: four plugins declare their floor top-level, where editing versions[0] is a silent no-op, and the floor has three live spellings (min_ledmatrix_version, requires.min_ledmatrix_version, versions[].ledmatrix_min_version, plus deprecated ledmatrix_min). See src/plugin_system/compatibility.py:declared_min_version for the resolution order any floor-raising tool must reproduce — and note the name is inverted between the top level and versions[].

Still not adopted, deliberately: data_sources.py, game_renderer.py and base_odds_manager.py. The standing decision held them until B6 closed; it now has, so they can be reconsidered — with B5's lesson applied, which is to build the object and diff rendered output rather than trust a static check.

B5 retrospective — what the adoption actually cost

Recorded because it is the evidence behind the two decisions above, and because "the adoption went fine" is not what happened.

Four of the eight shipped with scroll mode broken on a 3.2.0 core, and were repaired in plugins-repo #251. The restructure lifted the content methods verbatim but left the state they read off self behind: separator-icon constants (hockey, basketball, lacrosse) and the game-renderer cache (afl). hockey/basketball/lacrosse could not construct the scroll display at all; afl raised inside prepare_scroll_content, which the core base catches, so its only symptom was scroll mode silently drawing nothing.

Three things are worth carrying forward:

  • The bundled fallback did not protect anyone from this. The break was on the modern path, which the fallback never touches. Carrying the second copy bought nothing against the actual defect while creating the divergence that produced it. That is an argument for B6, not against it.
  • Every gate was green. The safety harness renders the scoreboard screens, not scroll mode; test_core_fallback.py checked that methods existed and that their globals resolved, and self.NHL_SEPARATOR_ICON is an attribute read, invisible to an AST scan for Name loads. The fix was to stop reasoning about source and build the object: construct both classes on both paths, compare the separator icons they end up with, and assert the adopted class ends up with every instance attribute the bundled one sets.
  • Test what the change touches, not what is convenient to render. Scroll mode had no coverage because the harness could not reach it. A comparison harness that renders the same games through both paths and diffs the pixels needs no per-sport knowledge of the right answer, only that adopting core code did not change it.

The ledger. Before adoption, eight duplicated copies totalled 5,685 lines. After adoption plus the frozen legacy copies it was 10,610; removing the dead inline duplication (plugins #252) brought it to roughly 8,620. B6 would take it to about 3,300 including the shared core module — some 2,400 fewer than before this project started. Until B6 runs, the adoption is net negative on disk, and its one delivered user-visible gain was that adopted plugins honoured the global target_fps instead of hardcoding ~100 FPS (since withdrawn: see the note under the B3 design above).

Decision: stop adopting further modules until B6 closes

data_sources.py (9 copies), game_renderer.py (8) and base_odds_manager.py are the obvious next candidates. Do not adopt them yet. Each adoption adds carrying cost — a second copy to keep in step — against a payoff that is contingent on B6, and B6 is gated on an installed base we cannot currently measure. Consolidate what is already committed; revisit when B6 does.

What's next

Steps 1–5 of the original plan are done: 3.2.0 is tagged and published with a version number CI now asserts (#428), the compatibility gate is in install_plugin and reads compatible_versions as well as the floor (#431, #433), the newest manifest entry is required to use ledmatrix_min_version (plugins #244), and all eight plugins have adopted the scroll orchestration (plugins #245–#249, repaired in #251, tidied in #252).

What actually remains, smallest first:

  1. Soak the adoptions on hardware. football and hockey have been run on a live rig through real games; baseball was watched through one earlier. The rest are proven by harness, unit tests and pixel comparison. Out-of-season sports cannot be soaked until their season starts. When you do, check the rig's *_display_mode first — a board in switch mode will happily load a sunset plugin and tell you nothing about the scroll code the sunset changed.
  2. Cut 3.3.0. Not required by B6 — its floors are 3.2.0, which is released — but calendar 1.2.3 floors at 3.3.0 for the device-authorization endpoints that landed after 3.2.0, so it is un-installable until the release exists.
  3. Reconsider the held modules (data_sources.py, game_renderer.py, base_odds_manager.py) now that the sunset has closed. game_renderer.py is the largest single duplication left: ~11,500 lines across eight plugins, with ~36,500 more in the eight sports.py. Note that core already ships src/base_classes/sports/ (~143KB, promoted in B1/B2) that no plugin imports — check whether it has drifted before treating it as the target.

How to keep this project healthy

Lessons this migration paid for, worth applying beyond it:

  • A version number is a promise; keep it in one place. Three different answers to "what version am I on" (tag, release, __version__) is what made the floor untrustworthy. Assert their agreement mechanically.
  • Advisory checks protect nobody. If a rule matters, enforce it where the action happens — the install path, not a log line the user will never read. If it doesn't matter enough to enforce, don't write the rule.
  • Prefer failures that are loud and early. A plugin that dies at load with one journal line is indistinguishable, to a user, from a plugin that was never installed. Surface plugin health in the UI.
  • Keep the two repos' rules in sync deliberately. The sunset rule lives in both this file and the plugins repo's docs/plugin-development/08-shared-sports-code.md. When one changes, change the other in the same PR — drift between them is how a contributor ends up following a rule that was superseded.
  • Measure before and after, on real hardware. Byte-identical harness renders and a device soak caught what unit tests could not. Reserve "it should be fine" for things you have actually looked at.

Rules for contributors

  • Promote on evidence, not intuition. A method moves to core when every copy has it and they agree on intent. Otherwise it stays in the plugins.
  • Never add a sport name to core. If core needs to know which sport it is, the design is wrong — add an override point instead.
  • A capability that is not opted into must not execute. If you find yourself writing if self.<capability>_enabled inside a base class, it belongs in a mixin.
  • Touch the view-model keys only additively. Published skins depend on them.
  • Every promotion lands with the characterization suite green, and every pilot adoption lands with that plugin's harness and golden suites green.