fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595)

* fix(sports): share the ESPN rejected-range memo with the background service

BackgroundDataService always sent a season range first and, on a 400,
fell back to chunks without recording the rejection, so every background
season fetch spent a doomed request and live scoreboards learned nothing
from it (or it from them). The worker now consults and sets the same
6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight
to month/day chunks, and if every chunk fails the range is asked once for
a real error without re-spending the chunks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): keep plugin asset and action routes inside their directories

POST /plugins/assets/upload, GET /plugins/assets/list and POST
/plugins/assets/delete joined the request's plugin_id onto assets/plugins
unchecked, so '../../config' created, wrote, listed and deleted outside
it. #561 guarded only the route that serves the files. All three now go
through path_safety.resolve_under and answer 400 for anything but a
plain name, and delete only unlinks a metadata path that resolves into
that plugin's uploads directory.

PluginManager.get_plugin_directory refuses ids that are not one plain
path segment, so /plugins/action (which runs a manifest script from the
returned directory) and every other caller get the guard; the action
route also rejects such ids up front, covering its no-manager fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): report a no-op plugin update as already up to date

update_plugin() returns True both for a real update and for "nothing to
do" (a ZIP-installed monorepo plugin already at the registry version, a
bundled plugin). With no git commit to compare, POST /plugins/update
called every such success "updated successfully", so Check & Update All
counted most official plugins as updated on every run.

The route now reads what changed off the plugin itself (commit, else
manifest version, else last_updated) and returns data.update_status
(updated / up_to_date / local_only). The update-all toast is summarised
by PluginInstallManager.summarizeUpdateResults from that status, falling
back to the message for older servers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(sports): scoreboard scroll speed no longer follows target_fps

sports_scroll computed the crisp speed ladder against the global
target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since
frame-locked presentation (#545) the helper steps a fixed number of whole
pixels per presented frame and the panel presents at its real refresh, so
the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a
50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s.

The ladder now uses the display manager's refresh_hz, then
display.hardware.limit_refresh_rate_hz, then the default. target_fps is
not consulted. Docstrings now say scroll_delay is ignored for pacing (no
behaviour change there) and describe the fixed-step model.

Tests: replace the tests that pinned target_fps as the ladder refresh and
described time-based stepping; assert speed independence from target_fps
(unit and end-to-end presented px/s against the real helper), that the
fixed per-frame step is applied, and that scroll_delay does not change
speed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): escape registry and upload values in plugin manager inline handlers

The store, saved-repository and custom-registry buttons built
onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone,
so a custom registry entry whose id contained ' closed the attribute and
added its own handler. One helper, jsStringAttr(), now HTML-escapes the
JSON literal for every one of those handlers, and the store View button
opens only http(s) repo links.

The live window.updateImageList (plugins_manager.js loads last, so its
copy wins over the file-upload widget's) wrote the uploaded file's
original name, path and ids into markup raw; they are escaped now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note plugin asset, action and inline handler guards

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): let the root pip wrapper install web_interface/requirements.txt

Update Code, the automatic update's health check and Install Base
Requirements install web_interface/requirements.txt through
safe_pip_install.sh, which only allowed the root requirements.txt. The
first commit changing that file would fail its dependency install, and
the automatic updater rolls back any update whose dependencies did not
install -- on every device, for every newer commit.

The wrapper now lists both core requirement files. Only their folders
are resolved, so a requirements.txt symlinked out of the project is
compared by its target and refused (previously the root file's own
symlink target was what got allowed). The updater's file list is a
named constant, and a test runs the real wrapper (pip stubbed) on
every file Update Code and the rollback install.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): do not retry plugin requests that got an HTTP answer

PluginAPI.request wrapped everything that was not a structured error as
NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a
JSON error without error_code included. Check & Update All retries
NETWORK_ERROR, so those updates were re-sent five more times with
backoff, contrary to the #587 contract that an HTTP error response is
the server's answer.

NETWORK_ERROR now means only that fetch() rejected. Any HTTP response
without an error_code, or with a body that is not JSON, is API_ERROR
with the HTTP status attached. Tested against the shipped api_client.js.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): restart the stats window when an idle gap is dropped by size

#582 dropped an idle gap from the frame stats two ways: the reset_scroll()
sentinel, which also restarts the 5s window timer, and a size guard for
scrollers that never call reset_scroll(), which did not. On that path the
first real frame after the gap found the boundary overdue and logged a
stats line for a one-frame window. Both paths now share one seeding helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): leave plugins alone when update_core's own rollback fails

update_core returns rollback_failed directly when a partial pull or an
update whose health check never started cannot be rolled back. run()
only held plugins back for 'verifying', so those devices still got new
plugin versions and a display restart on top of a core in an unknown
state -- the opposite of what the health-check path does, and of the
3.4.0 changelog (plugins are left alone if the rollback fails).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(api): make the REST reference match the api_v3 package

Every documented request body, query parameter and response shape was
re-checked against the handlers in web_interface/blueprints/api_v3/.
Fixes calls that failed as documented (repo_url, action_id/params,
files/image_id, font_file+font_family, ?font=, cache key,
auto_enable_ap_mode, plugin limit keys), removes the font-override
endpoints dropped in #566, corrects response shapes (plugins/config,
plugins/schema, health, metrics, operation history, github-status,
fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds
the 26 routes it omitted (backup, system auto-update/git, wifi radio,
starlark editor, MQTT bridge, status endpoints, skins).

Documents the merge semantics of partial JSON saves to /config/main and
/plugins/config and the dim-schedule POST accepting GET's days shape,
which land in the same change set. Replaces app.py line numbers and the
removed api_v3.py path with file and function names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): remove the General-tab plugin system toggles that did nothing

plugin_system.auto_discover, auto_load_enabled and development_mode had
General-tab toggles whose help tips promised dormant plugins and verbose
logging, but nothing reads them: every enabled plugin is discovered and
loaded regardless. Remove the three toggles.

The keys stay tolerated in stored configs. The save handler now stores
a flag only when a client sends it; treating a missing key as an
unchecked box would otherwise rewrite all three to false on every
General-tab save, which still posts plugins_directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(scroll): remove dead code left by #523/#570

- Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read
  them since the numpy blend replaced the scipy path.
- Drop ScrollHelper._last_integer_position and frame_time_target, which
  were written but never read.
- Keep target_fps and set_target_fps() but document them as
  informational: nothing paces off them, yet ledmatrix-elections'
  test_scroll_pacing.py reads helper.target_fps back and third-party
  plugins may call the setter.
- Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to
  throttle", set_sub_pixel_scrolling's "default: True", and
  set_frame_based_scrolling's claim that it steps.

The plugins monorepo was grepped for every removed name; none is used.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI

FONT_MANAGER.md told plugins to read display_manager.font_manager, which
does not exist, so a plugin following it failed to load with
AttributeError. The shared FontManager lives on the PluginManager and
BasePlugin._get_font_manager() returns it (with a fallback for harnesses).

Also removes the Fonts-tab override workflow and element-override panels
that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls

/plugins/store/search does not exist (404) and the list endpoint reads
query, not q. The registry guide's curl examples omitted the JSON
Content-Type, so the handlers saw an empty body and answered 400.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(config): use the shared core-key list in the last three private copies

StartupValidator warned "Plugin 'auto_update' is enabled but not found" on
every display start with auto-update or a dim schedule on; the reserved
plugin-id check missed auto_update, sync, location and the rest; and
ConfigManager's (uncalled) orphan cleanup would have deleted display,
schedule and auto_update. All three now read src/core_config_keys.py, which
also gains CORE_SECRETS_KEYS for the github/youtube secrets sections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): partial JSON saves to /config/main change only what they send

A JSON body with one field reset every checkbox in the sections it touched:
the MQTT bridge's brightness slider turned off disable_hardware_pulsing,
inverse_colors, show_refresh_rate and use_short_date_format, and a
timezone-only save turned off web-UI autostart and weekly auto-updates.
Missing-means-unchecked now applies only to form posts: form-encoded bodies
and the v3 forms, which mark themselves with a hidden __form_section input.

Also on the config routes:
- vegas_min/max_cycle_duration no longer match the generic *_duration rule,
  so they stop landing in display_durations and a blank one no longer
  rejects the whole Display save;
- saving from the Raw JSON editor calls start_setup_if_needed like the
  General form, so enabling auto-update there finishes its setup;
- the schedule and dim-schedule POSTs accept the per-day days.<day> shape
  their GETs return, as well as the flat form keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): install plugin dependencies from the configured plugins directory

install_plugin_dependencies.sh scanned only plugins/, but the Plugin
Store installs into plugin_system.plugins_directory (default
plugin-repos), so the documented "Recommended" fix found 0 plugins on
every store install. It now reads plugins_directory from
config/config.json (relative to the project root or absolute, default
plugin-repos) and also scans plugins/ for dev symlinks, installing a
plugin reached through both only once.

With set -e alone, `pip ... | tee` took tee's exit status, so a failed
pip install was reported as success; set -o pipefail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: replace stale API names, line numbers and the api_v3.py path

- ADVANCED_FEATURES: StreamManager methods that exist
  (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the
  real on-demand status envelope ({status, data: {state, service}})
- app.py:199 / :144 / :607-619 line citations and
  web_interface/blueprints/api_v3.py (now a package) replaced with file and
  function names in ADVANCED_FEATURES, CONFIG_DEBUGGING,
  PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE,
  PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README
- CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use
  /config/raw/main to replace the file; describe where validation runs
- TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints
  usage)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): verify the web interface that actually ships, on port 5000

verify_installation.sh failed every healthy install: it required the
long-removed web_interface_v2.py and looked for a listener on port 5001,
while the web interface binds 5000 (web_interface/start.py). It now
checks the files ledmatrix-web.service runs (start_web_conditionally.py,
web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the
same 5001 port in its listen check, HTTP probe and printed URLs.

Port matches are anchored so :50001 no longer counts as :5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): one display-size contract: display_manager.width/height

CLAUDE.md (#580) says to read display_manager.width/height because
matrix is None when hardware init fails; the development guide, the
safety-harness doc and two DisplayManager docstrings still recommended
matrix.width/height. The bundled starlark-apps plugin read matrix.width
unguarded, so its magnify recommendation and frame scaling raised in
fallback mode (e.g. after the Pi 5 hardware refusal).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(install): make install_service.sh --help print usage instead of installing

install_service.sh parsed no arguments, so `sudo ./scripts/install/
install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md)
rewrote ledmatrix.service, ledmatrix-web.service and both update-verify
units and enabled/started them. It now handles -h/--help (usage, exit 0,
no changes) and rejects any other argument with exit 2 before doing
anything. Running it with no arguments, as first_time_install.sh does,
is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scroll): describe the fixed-step model and document frame_hold

Since #545 a crisp speed from scroll_config.configure() makes the helper
advance a fixed whole-pixel step per presented frame with no clock, and the
display manager's frame hold is part of the speed. The docs still described
the removed wall-clock model:

- scroll_config's module and configure() docstrings said speed is applied
  in time-based mode and that omitting the hold "falls back to fractional
  pixels"; omitting it actually runs the scroll frame_hold times too fast.
- SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both
  modes, and read a 20 ms stats median as missed refreshes although that
  is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step,
  the hold-dependent healthy median, that target_fps plays no part, and
  that a hand-added scroll_pixels_per_second loses to a schema-default pair.
- PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling)
  without frame_hold; it now documents the parameter (core 3.4.0) with a
  configure() + set_scrolling_state example.
- update_scroll_position/set_scroll_speed and set_scrolling_state
  docstrings say the same.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark target_fps legacy; describe what Vegas scroll_delay does

- General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after
  the sports_scroll fix nothing in core scrolling reads it. The field and
  its API validation stay so saved configs and plugins that read
  global_config['target_fps'] keep working. CONFIG_REFERENCE says the same.
- Vegas frame_based_scrolling/scroll_delay were described as frame-count
  stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based
  mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and
  still advances by elapsed time, so the applied speed is
  clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The
  config comments, render_pipeline comment and CONFIG_REFERENCE rows now
  say so. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(deps): describe how plugin dependencies are really installed

The guides said the web service runs as root, that installs pick --user
from os.geteuid(), and quoted a warning and a
PluginManager._install_plugin_dependencies() method that don't exist. The
web unit runs as the installing user; store installs go through
install_requirements_file() and sudo safe_pip_install.sh (root), with a
user-level fallback that says so, and load-time installs run in the
display service's own (root) interpreter.

Manual paths now use the configured plugins directory (plugin-repos/ by
default) instead of plugins/, which store installs no longer use, and
install_plugin_dependencies.sh is described as scanning that directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): count local changes one way for the preflight and the pull

The automatic update's preflight ignored mode-only changes and anything
whose status line contained plugins/ or plugin-repos/, then promised
"Automatic updates will not stash your changes". perform_core_update
used plain git status (modes count) and ignored only 'plugins/', then
ran 'git stash push -- :!plugins', which nothing ever pops. So an edit
to a bundled plugin under plugin-repos/, or the installer's chmods on
tracked scripts, passed the preflight and was stashed away for good.

- auto_update.local_changes() is the one predicate both use:
  core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/
  excluded by leading folder rather than substring (a core file under
  web_interface/static/v3/js/plugins/ now counts).
- Update Code's explicit stash leaves out both plugin folders; the
  pull's --autostash carries their edits and mode changes across and
  reapplies them.
- The automatic updater calls perform_core_update(stash_local_changes=
  False), which refuses instead of stashing edits that appeared after
  the preflight; update_core reports that as 'blocked'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): diagnostics follow the web autostart default and api_v3 package

#556 made a missing web_display_autostart mean "start" (only an explicit
false/off keeps the web interface down), but the diagnostics still said
otherwise: diagnose_web_ui.sh reported a missing key as "defaults to
false", diagnose_web_interface.sh said the web interface "will not start
unless this is set to true" and recommended enabling it, and
debug_web_manual.py printed False. Troubleshooting a down web UI pointed
users at a non-cause.

Both shell scripts now evaluate the setting with the launcher's own
autostart_enabled() (inline fallback if it cannot be imported) and report
on / off / not set (on) / unparseable config; debug_web_manual.py uses
the same function. They also check web_interface/blueprints/api_v3/
__init__.py: api_v3.py became a package in #553, so every healthy
checkout was reported as missing a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(install): what install_service.sh installs; verify script port; no sudo for --help

install_service.sh installs and starts ledmatrix, ledmatrix-web and the
update-verify units, not only ledmatrix.service (systemd/README.md,
README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help'
as a harmless check; it now shows --help without sudo and warns what a
real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh
checks the web interface on port 5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note update-all, plugin system settings and script fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): size the preview after orientation and pixel mappers

display_geometry.physical_size claimed to give DisplayManager's answer but
only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured
after the library's pixel mappers, so a Rotate:90 / orientation 90 chain
previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for
128x64.

Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc
does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown
names leave it alone), and move the orientation composition here so
DisplayManager and the preview share it. The module docstring no longer
claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH.

Audit finding F18.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): refuse settings the rgbmatrix library aborts on, on every board

The library answers several settings with a NULL matrix or abort() rather
than an error, so the display service crash-looped (Restart=on-failure)
instead of reaching fallback mode: rows above 64, chain_length above 255
(uint8_t binding setter, documented as "no upper limit"), a misspelled
hardware_mapping, and parallel 2-3 on a single-output mapping, reachable
from the Display form on the default adafruit-hat(-pwm) mapping. #586 only
guarded the Pi 5 subset.

- src/matrix_support.py holds the rules for every board (Options::Validate
  ranges, binding integer types, mapping names and outputs from
  lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the
  API's numeric ranges.
- DisplayManager checks them before building options and raises
  MatrixSettingsRefused, so a hand-edited config falls back with a logged,
  reported reason. Emulator mode only warns.
- The config API refuses them with a 400 naming the setting; combinations
  are checked against stored values but reported only when the request
  sets a field involved.
- The hardware status file gains "cause" (settings/library/forced). The
  fallback log and Display banner give the Pi 5 rebuild hint only for a
  library failure instead of rebuild + gpio_slowdown advice for every
  failure; one Pi 5 slowdown recommendation (1-3, start at 1).
- The Display form offers classic/classic-pi1 and orientation 90/270 and
  renders any other stored mapping selected with a warning, so an
  unrelated save no longer rewrites them; the API accepts 90/270.

Audit findings F03, F16, F19, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(display): library limits, template defaults and Pi 5 slowdown

- rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs,
  classic/classic-pi1 mappings and orientation 90/270 documented.
- Defaults are the config.template.json values: config migration adds
  missing keys from the template, so the listed "code defaults" never
  applied.
- One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode,
  starting at 1.
- Troubleshooting describes the refused-settings fallback, and CHANGELOG
  corrects the Unreleased "no upper limit" entry.

Audit findings F19, F20, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py opens the panel with the service's options

--measure and --demo built RGBMatrixOptions from a private copy of the
display service's builder that had drifted: gpio_slowdown came from
display.hardware (default 2) instead of display.runtime (default 3), and
rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors,
pixel_mapper_config and orientation were skipped, with different defaults
(hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown
was measured -- or garbled -- in a setup the service never drives.

The option filling in DisplayManager._setup_matrix moves, unchanged, into
DisplayManager.apply_matrix_options(options, config), which _setup_matrix
calls and the script reuses (overriding only limit_refresh_rate_hz for
--measure). The script now loads the whole config rather than the hardware
block. Tests pin the script's options to the service's attribute for
attribute.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py recommends keys the resolver honours

The ladder ended by telling users to set
display_options.scroll_pixels_per_second. scroll_config ranks that key
below the scroll_speed + scroll_delay pair, deliberately, and several
plugin schemas default the pair into config, so the advised key was
silently ignored (a schema-default 1/0.02 pair plus an advised 66 still
resolved to 50 px/s).

The advice is now the pair that selects the crisp speed exactly
(pixels_per_frame every frame_hold/refresh seconds), explains that the
pair outranks scroll_pixels_per_second, and gives the scoreboards'
per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the
printed pair over a schema-default pair and check it lands on the
advertised speed and hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula

- SPORTS_UNIFICATION.md still presented honouring global target_fps as
  sports_scroll's added behaviour and its one user-visible gain; note that
  it was withdrawn because it had become a speed multiplier.
- ADVANCED_FEATURES.md gave Vegas scrolling as
  (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s
  by elapsed time, through a 0.1-5 px per scroll_delay clamp when
  frame_based_scrolling is on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): scroll model fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(dev): link-github links plugins from the ledmatrix-plugins monorepo

link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git,
and those per-plugin repositories no longer exist: official plugins are
directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the
monorepo once into the dev directory, finds plugins/<name>,
plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and
links it under its manifest id. With an explicit repo URL it still links a
single-repository plugin as before.

dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a
fork), plus plugins_repo and plugins_branch; github_pattern, which was
documented but never read, is dropped and warned about. Ships
dev_plugins.json.example and git-ignores dev_plugins.json, both of which
the guide promised. Reading JSON falls back to python3 when jq is missing
(get_plugin_id silently returned nothing without jq).

update/status/list find the git checkout above a monorepo plugin
directory (its .git is not in the plugin dir), and update pulls a shared
checkout once. status no longer exits 1 when nothing is broken.

Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration,
workflow, store integration, hello-world link, submission), and the
nonexistent scripts/git-hooks/pre-push-plugin-version and
scripts/bump_plugin_version.py replaced with the real rule: bump the
manifest version and run update_registry.py. scripts/dev/README.md and
CLAUDE.md updated to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scripts): monorepo workspace layout; fix_perms and install READMEs

MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin;
setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into
plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the
workspace file opens LEDMatrix plus ../ledmatrix-plugins.

scripts/fix_perms/README.md listed cache directories
fix_cache_permissions.sh never touches and a 'ledmatrix' service user
that doesn't exist (also in scripts/install/README.md); adds
safe_pip_install.sh. install/README: install_service.sh installs the web
and update-verify units too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): keep the rollback's pip retries inside the unit time limit

The health check reinstalled the previous requirements by trying the
next bash path after any failure, including a 600 s pip timeout. Two
files, two paths: up to 40 minutes of pip alone, while systemd stops
ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the
rollback half-way and leaving the update 'verifying' until the web UI
calls it lost.

- Like permission_utils.install_requirements_file, only a sudo refusal
  moves on to the next bash; a pip that ran and failed or timed out is
  not repeated. The refusal wording is one list
  (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only
  verifier and pinned equal by a test.
- All reinstalls in one rollback share a 600 s budget.
- WORST_CASE_SECONDS adds up every timeout on the longest path (27.5
  min); a test holds it under the unit's TimeoutStartSec and that under
  the web UI's VERIFY_LOST_SECONDS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools

Plugin config was prepared differently depending on how it arrived:

- JSON POST /plugins/config built a partial body on schema defaults, so
  {"enabled": true} reset every other setting of the plugin. It now merges
  onto the stored section first, as the form path already did.
- Legacy-boolean normalization (#588) ran only at load: GET /plugins/config
  returned the raw boolean, posting it back failed validation, and hot
  reload handed plugins the raw section (a legacy dynamic_duration: true
  came back as a boolean). schema_manager.prepare_plugin_config (normalize,
  then defaults) is now used by PluginManager.load_plugin, both save paths,
  GET, the save notifications and DisplayController's hot-reload callback.
- The JSON save's filter kept only enabled/display_duration/live_priority
  and dropped a submitted skin, skin_options or vegas_* tuning key. There
  is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES,
  used by validation and by the save filter; PluginManager's
  CORE_OWNED_CONFIG_KEYS is its vegas subset.
- Plugin sections posted to /config/main were stored verbatim, including
  values /plugins/config rejects. They now go through the same preparation
  (_prepare_plugin_config_for_save, extracted from save_plugin_config), and
  a failing section rejects the whole save before anything is written.
- dev_server read only top-level defaults and let a schema enabled:false
  win; build_full_config shallow-merged overrides, dropping sibling
  defaults; the harness extracted defaults differently from the device.
  loading.build_config now uses the device's extraction and preparation,
  and dev_server, check_plugin, render_plugin and the harness all use it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt-bridge): brightness changes apply live and touch nothing else

The display service's hot reload applies a saved brightness within a few
seconds, and /config/main no longer resets other display settings on a
brightness-only JSON body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): automatic update hardening

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI

It described web_interface_v2.py and index_v2.html (both gone), client-side
form generation, one POST per field with {key, value}, and 'no nested
objects'. The v3 UI renders plugin forms server-side from the schema
(pages_v3 partial + plugin_config.html macros, nested sections and
x-widgets), posts the whole form once, and save_plugin_config() merges onto
the stored section, validates, splits x-secret fields and notifies the
plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt): brightness saves apply via hot reload and leave other settings alone

The bridge README said brightness is applied on the display's next
restart; the display controller's config hot reload applies it within
seconds. It also now states that the bridge's partial JSON save changes
only brightness (the /config/main merge fix in this change set).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): don't log pip's output from the health check's reinstall

pip can echo a private index URL with embedded credentials;
permission_utils redacts it, the stdlib-only verifier cannot, so it
logs the exit code only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark the plugin_system toggles as unused legacy keys

auto_discover, auto_load_enabled and development_mode are read by
nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE
said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and
the REST reference listed them as live settings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): docs and developer tools group

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): legacy plugin-system toggles no longer count as a General save

auto_discover, auto_load_enabled and development_mode have left the General
form, so a post carrying only one of them is not a general-settings save and
must not treat web_display_autostart and auto_update as unchecked. The
plugin_system block itself is left as on main for the branch that reworks it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): config-save and plugin-config preparation fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(claude): re-check matrix_support.py rules when the library submodule is bumped

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address Codacy findings on the core audit PR

- plugin_manager.prepare_plugin_config: when the fallback legacy-boolean
  pass also fails, log a warning instead of a bare except/pass.
- api_client.js: request() refuses any endpoint that is not a plain path
  under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace,
  control characters) with INVALID_ENDPOINT before calling fetch(), and
  plugin ids are URL-encoded wherever they are put into a URL (also in the
  app-shell batch load).
- test_update_all.js: pins both against the shipped client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): check endpoint control characters without a control-character regex

Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags
the \x00-\x1f range in checkEndpoint's regex. Test the char codes
instead; the endpoints refused are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(auto-update): make the seed script executable on disk, not only in the index

On Linux Repo.publish() commits with -a, which recorded scripts/run.sh
as 100644 upstream because the seed file was never chmod +x. The pull
then brought in the same mode the installer chmod had made locally, so
installer_chmod saw no mode change left to check. The updater was fine:
with the upstream commit at 100755 the --autostash carries the device's
chmod across. Verified under Linux (WSL, git 2.43): the old helper fails
exactly as CI did, the fixed one passes all 63 tests in the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

scripts/debug/debug_web_manual.py

@@ -54,8 +54,13 @@ def main():
         config = config_manager.load_config()
         print("   ✅ Config loaded")
 
-        autostart = config.get('web_display_autostart', False)
-        print(f"   🔧 web_display_autostart: {autostart}")
+        # Same rule ledmatrix-web.service applies: only an explicit false/off
+        # keeps the web interface down; a missing key means on.
+        sys.path.insert(0, str(project_root / 'scripts' / 'utils'))
+        from start_web_conditionally import autostart_enabled
+        raw = config.get('web_display_autostart', '(not set, defaults to on)')
+        state = 'starts' if autostart_enabled(config) else 'will NOT start'
+        print(f"   🔧 web_display_autostart: {raw} (web interface {state})")
     except Exception as e:
         print(f"   ❌ Config check failed: {e}")
         traceback.print_exc()

scripts/dev/README.md

@@ -12,9 +12,19 @@ This directory contains scripts and utilities for development and testing.
 
 ### Plugin Development Setup
 ```bash
+# Official plugin: clones ChuckBuilds/ledmatrix-plugins (once) and links
+# its plugins/<plugin-name> into plugins/
 ./scripts/dev/dev_plugin_setup.sh link-github <plugin-name>
+
+# Plugin with its own repository
+./scripts/dev/dev_plugin_setup.sh link-github <plugin-name> <repo-url>
 ```
 
+Set `plugin_system.plugins_directory` to `plugins` so the loader finds the
+links. To use a fork or another clone location, copy
+`dev_plugins.json.example` to `dev_plugins.json`. Details:
+[docs/PLUGIN_DEVELOPMENT_GUIDE.md](../../docs/PLUGIN_DEVELOPMENT_GUIDE.md).
+
 ### Running Emulator
 ```bash
 ./scripts/dev/run_emulator.sh

scripts/dev/dev_plugin_setup.sh

@@ -10,8 +10,14 @@ PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
 PLUGINS_DIR="$PROJECT_ROOT/plugins"
 CONFIG_FILE="$PROJECT_ROOT/dev_plugins.json"
 DEFAULT_DEV_DIR="$HOME/.ledmatrix-dev-plugins"
-GITHUB_USER="ChuckBuilds"
-GITHUB_PATTERN="ledmatrix-"
+# Official plugins live in one monorepo: <github_user>/<plugins_repo>, one
+# directory per plugin under plugins/. Both can be overridden in
+# dev_plugins.json (e.g. to work from a fork).
+DEFAULT_GITHUB_USER="ChuckBuilds"
+DEFAULT_PLUGINS_REPO="ledmatrix-plugins"
+GITHUB_USER="$DEFAULT_GITHUB_USER"
+PLUGINS_REPO="$DEFAULT_PLUGINS_REPO"
+PLUGINS_BRANCH=""
 
 # Colors for output
 RED='\033[0;31m'
@@ -37,18 +43,55 @@ log_error() {
     echo -e "${RED}[ERROR]${NC} $1"
 }
 
+# Print a top-level string field of a JSON file, or nothing if it is absent.
+# Uses jq when installed, else python3.
+json_field() {
+    local file="$1"
+    local key="$2"
+    if command -v jq >/dev/null 2>&1; then
+        jq -r --arg k "$key" '.[$k] // empty | select(type == "string")' "$file" 2>/dev/null || true
+    elif command -v python3 >/dev/null 2>&1; then
+        python3 - "$file" "$key" <<'PY' 2>/dev/null || true
+import json, sys
+try:
+    with open(sys.argv[1], encoding="utf-8") as f:
+        value = json.load(f).get(sys.argv[2])
+except Exception:
+    value = None
+if isinstance(value, str):
+    print(value)
+PY
+    fi
+}
+
 # Load configuration file
 load_config() {
+    DEV_PLUGINS_DIR="$DEFAULT_DEV_DIR"
     if [[ -f "$CONFIG_FILE" ]]; then
-        DEV_PLUGINS_DIR=$(jq -r '.dev_plugins_dir // "'"$DEFAULT_DEV_DIR"'"' "$CONFIG_FILE" 2>/dev/null || echo "$DEFAULT_DEV_DIR")
-        # Expand ~ in path
-        DEV_PLUGINS_DIR="${DEV_PLUGINS_DIR/#\~/$HOME}"
-    else
-        DEV_PLUGINS_DIR="$DEFAULT_DEV_DIR"
+        local value
+        value=$(json_field "$CONFIG_FILE" dev_plugins_dir)
+        [[ -n "$value" ]] && DEV_PLUGINS_DIR="$value"
+        value=$(json_field "$CONFIG_FILE" github_user)
+        [[ -n "$value" ]] && GITHUB_USER="$value"
+        value=$(json_field "$CONFIG_FILE" plugins_repo)
+        [[ -n "$value" ]] && PLUGINS_REPO="$value"
+        value=$(json_field "$CONFIG_FILE" plugins_branch)
+        [[ -n "$value" ]] && PLUGINS_BRANCH="$value"
+        if [[ -n "$(json_field "$CONFIG_FILE" github_pattern)" ]]; then
+            log_warn "dev_plugins.json: github_pattern is no longer used (official plugins are in the $PLUGINS_REPO monorepo)"
+        fi
     fi
+    # Expand ~ in path
+    DEV_PLUGINS_DIR="${DEV_PLUGINS_DIR/#\~/$HOME}"
     mkdir -p "$DEV_PLUGINS_DIR"
 }
 
+# Top level of the git checkout containing a path, or nothing.
+# A monorepo plugin is a subdirectory, so its .git is not in the plugin dir.
+git_root_of() {
+    git -C "$1" rev-parse --show-toplevel 2>/dev/null || true
+}
+
 # Validate plugin structure
 validate_plugin() {
     local plugin_path="$1"
@@ -63,7 +106,7 @@ validate_plugin() {
 get_plugin_id() {
     local plugin_path="$1"
     if [[ -f "$plugin_path/manifest.json" ]]; then
-        jq -r '.id // empty' "$plugin_path/manifest.json" 2>/dev/null || echo ""
+        json_field "$plugin_path/manifest.json" id
     fi
 }
 
@@ -176,50 +219,104 @@ clone_from_github() {
     return 0
 }
 
+# Clone a repository into DEV_PLUGINS_DIR, or update the existing clone.
+# Prints the clone's path on stdout (log output goes to stderr).
+ensure_clone() {
+    local repo_url="$1"
+    local branch="${2:-}"
+    local repo_name
+    repo_name=$(basename "$repo_url" .git)
+    local target_dir="$DEV_PLUGINS_DIR/$repo_name"
+
+    if [[ -d "$target_dir" ]]; then
+        log_info "Repository already exists at $target_dir" >&2
+        if [[ -d "$target_dir/.git" ]]; then
+            log_info "Updating repository..." >&2
+            (cd "$target_dir" && git pull --rebase) >&2 || true
+        fi
+    else
+        if ! clone_from_github "$repo_url" "$target_dir" "$branch" >&2; then
+            return 1
+        fi
+    fi
+    echo "$target_dir"
+}
+
+# Find a plugin's directory inside a monorepo clone: plugins/<name>,
+# plugins/ledmatrix-<name>, or the directory whose manifest id is <name>.
+find_monorepo_plugin() {
+    local repo_dir="$1"
+    local name="$2"
+    local candidate
+    for candidate in "$repo_dir/plugins/$name" "$repo_dir/plugins/ledmatrix-$name"; do
+        if [[ -f "$candidate/manifest.json" ]]; then
+            echo "$candidate"
+            return 0
+        fi
+    done
+    for candidate in "$repo_dir"/plugins/*/; do
+        candidate="${candidate%/}"
+        [[ -f "$candidate/manifest.json" ]] || continue
+        if [[ "$(get_plugin_id "$candidate")" == "$name" ]]; then
+            echo "$candidate"
+            return 0
+        fi
+    done
+    return 1
+}
+
 # Link plugin from GitHub
 link_github_plugin() {
-    local plugin_name="$1"
+    local plugin_name="${1:-}"
     local repo_url="${2:-}"
-    
+
     if [[ -z "$plugin_name" ]]; then
         log_error "Usage: $0 link-github <plugin-name> [repo-url]"
         exit 1
     fi
-    
+
     load_config
-    
-    # Construct repo URL if not provided
-    if [[ -z "$repo_url" ]]; then
-        repo_url="https://github.com/${GITHUB_USER}/${GITHUB_PATTERN}${plugin_name}.git"
-        log_info "Using default GitHub URL: $repo_url"
-    fi
-    
-    # Determine target directory name from URL
-    local repo_name=$(basename "$repo_url" .git)
-    local target_dir="$DEV_PLUGINS_DIR/$repo_name"
-    
-    # Check if already cloned
-    if [[ -d "$target_dir" ]]; then
-        log_info "Repository already exists at $target_dir"
-        if [[ -d "$target_dir/.git" ]]; then
-            log_info "Updating repository..."
-            (cd "$target_dir" && git pull --rebase) || true
-        fi
-    else
-        # Clone the repository
-        if ! clone_from_github "$repo_url" "$target_dir"; then
+
+    if [[ -n "$repo_url" ]]; then
+        # A plugin with its own repository (e.g. a third-party plugin): the
+        # repository root is the plugin.
+        local target_dir
+        if ! target_dir=$(ensure_clone "$repo_url"); then
             exit 1
         fi
+        if ! validate_plugin "$target_dir"; then
+            log_error "Cloned repository does not appear to be a valid plugin"
+            exit 1
+        fi
+        link_plugin "$plugin_name" "$target_dir"
+        return
     fi
-    
-    # Validate plugin structure
-    if ! validate_plugin "$target_dir"; then
-        log_error "Cloned repository does not appear to be a valid plugin"
+
+    # Official plugins: clone the monorepo once, link plugins/<dir> from it.
+    repo_url="https://github.com/${GITHUB_USER}/${PLUGINS_REPO}.git"
+    log_info "Using plugin monorepo: $repo_url"
+    local repo_dir
+    if ! repo_dir=$(ensure_clone "$repo_url" "$PLUGINS_BRANCH"); then
         exit 1
     fi
-    
-    # Link the plugin
-    link_plugin "$plugin_name" "$target_dir"
+
+    local plugin_dir
+    if ! plugin_dir=$(find_monorepo_plugin "$repo_dir" "$plugin_name"); then
+        log_error "No plugin named '$plugin_name' in $repo_dir/plugins"
+        log_info "Plugins are the directory names under $repo_dir/plugins, or their manifest ids"
+        exit 1
+    fi
+
+    # Link under the manifest id: that is the name the plugin loader and
+    # config.json use, and it can differ from the directory name
+    # (plugins/ledmatrix-music has id ledmatrix-music, not music).
+    local link_name
+    link_name=$(get_plugin_id "$plugin_dir")
+    [[ -n "$link_name" ]] || link_name=$(basename "$plugin_dir")
+    if [[ "$link_name" != "$plugin_name" ]]; then
+        log_info "Linking as '$link_name' (the plugin's manifest id)"
+    fi
+    link_plugin "$link_name" "$plugin_dir"
 }
 
 # Unlink a plugin
@@ -274,7 +371,7 @@ list_plugins() {
             echo "  → $target"
             
             # Check git status if it's a git repo
-            if [[ -d "$target/.git" ]]; then
+            if [[ -n "$(git_root_of "$target")" ]]; then
                 local branch=$(cd "$target" && git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "unknown")
                 local status=$(cd "$target" && git status --porcelain 2>/dev/null | head -1)
                 if [[ -n "$status" ]]; then
@@ -327,7 +424,7 @@ check_status() {
             echo -e "${GREEN}✓${NC} ${BLUE}$plugin_name${NC}"
             echo "  Path: $target"
             
-            if [[ -d "$target/.git" ]]; then
+            if [[ -n "$(git_root_of "$target")" ]]; then
                 local branch=$(cd "$target" && git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "unknown")
                 local remote=$(cd "$target" && git remote get-url origin 2>/dev/null || echo "no remote")
                 local commits_behind=$(cd "$target" && git rev-list --count HEAD..@{upstream} 2>/dev/null || echo "0")
@@ -360,9 +457,13 @@ check_status() {
     done
     
     echo "Summary:"
-    echo "  ${GREEN}Clean: $clean_count${NC}"
-    echo "  ${YELLOW}Needs attention: $dirty_count${NC}"
-    [[ $broken_count -gt 0 ]] && echo -e "  ${RED}Broken: $broken_count${NC}"
+    echo -e "  ${GREEN}Clean: $clean_count${NC}"
+    echo -e "  ${YELLOW}Needs attention: $dirty_count${NC}"
+    # An if, not `[[ ]] &&`: as the function's last command a false test made
+    # `status` exit 1 whenever nothing was broken.
+    if [[ $broken_count -gt 0 ]]; then
+        echo -e "  ${RED}Broken: $broken_count${NC}"
+    fi
 }
 
 # Update plugin(s)
@@ -384,39 +485,46 @@ update_plugins() {
         fi
         
         local target=$(get_symlink_target "$plugin_name")
-        
-        if [[ ! -d "$target/.git" ]]; then
+        local root
+        root=$(git_root_of "$target")
+
+        if [[ -z "$root" ]]; then
             log_error "Plugin repository is not a git repository: $target"
             exit 1
         fi
-        
-        log_info "Updating $plugin_name from $target"
-        (cd "$target" && git pull --rebase)
+
+        log_info "Updating $plugin_name from $root"
+        (cd "$root" && git pull --rebase)
         log_success "Updated $plugin_name"
     else
-        # Update all linked plugins
+        # Update all linked plugins. Plugins linked from the monorepo share one
+        # checkout, which is pulled once.
         log_info "Updating all linked plugins..."
         local updated=0
         local failed=0
-        
+        local pulled_roots=" "
+
         for item in "$PLUGINS_DIR"/*; do
             [[ -e "$item" ]] || continue
             [[ -d "$item" ]] || continue
-            
+
             local name=$(basename "$item")
             [[ "$name" =~ ^\.|^_ ]] && continue
-            
+
             if is_symlink "$item"; then
                 local target=$(get_symlink_target "$name")
-                if [[ -d "$target/.git" ]]; then
-                    log_info "Updating $name..."
-                    if (cd "$target" && git pull --rebase); then
-                        log_success "Updated $name"
-                        updated=$((updated + 1))
-                    else
-                        log_error "Failed to update $name"
-                        failed=$((failed + 1))
-                    fi
+                local root
+                root=$(git_root_of "$target")
+                [[ -n "$root" ]] || continue
+                [[ "$pulled_roots" == *" $root "* ]] && continue
+                pulled_roots="$pulled_roots$root "
+                log_info "Updating $root (for $name)..."
+                if (cd "$root" && git pull --rebase); then
+                    log_success "Updated $root"
+                    updated=$((updated + 1))
+                else
+                    log_error "Failed to update $root"
+                    failed=$((failed + 1))
                 fi
             fi
         done
@@ -439,7 +547,12 @@ Commands:
     
   link-github <plugin-name> [repo-url]
       Clone and link a plugin from GitHub
-      If repo-url is not provided, uses: https://github.com/${GITHUB_USER}/${GITHUB_PATTERN}<plugin-name>.git
+      Without repo-url: clones (or updates) the official plugin monorepo,
+      https://github.com/${DEFAULT_GITHUB_USER}/${DEFAULT_PLUGINS_REPO}.git, and links its
+      plugins/<plugin-name> (or plugins/ledmatrix-<plugin-name>) under the
+      plugin's manifest id
+      With repo-url: clones a plugin that has its own repository and links
+      the repository root
     
   unlink <plugin-name>
       Remove symlink for a plugin (preserves repository)
@@ -458,25 +571,30 @@ Commands:
       Show this help message
 
 Examples:
-  # Link a local plugin
-  $0 link music ../ledmatrix-music
-  
-  # Link from GitHub (auto-detects URL)
-  $0 link-github music
-  
-  # Link from GitHub with custom URL
-  $0 link-github stocks https://github.com/ChuckBuilds/ledmatrix-stocks.git
-  
+  # Link an official plugin from the monorepo
+  $0 link-github football-scoreboard
+
+  # Link a plugin from a local monorepo checkout
+  $0 link hello-world ../ledmatrix-plugins/plugins/hello-world
+
+  # Link a third-party plugin from its own repository
+  $0 link-github my-plugin https://github.com/OtherUser/ledmatrix-my-plugin.git
+
   # Check status
   $0 status
-  
+
   # Update all plugins
   $0 update
 
 Configuration:
-  Create dev_plugins.json in project root to customize:
+  Copy dev_plugins.json.example to dev_plugins.json (git-ignored) to customize:
   - dev_plugins_dir: Where to clone GitHub repos (default: ~/.ledmatrix-dev-plugins)
-  - plugins: Plugin definitions (optional, for auto-discovery)
+  - github_user:     Owner of the plugin monorepo, e.g. your fork (default: ${DEFAULT_GITHUB_USER})
+  - plugins_repo:    Name of the plugin monorepo (default: ${DEFAULT_PLUGINS_REPO})
+  - plugins_branch:  Branch to clone the monorepo at (default: its default branch)
+
+  Symlinks are created in plugins/. Set plugin_system.plugins_directory to
+  "plugins" in config/config.json so the plugin loader discovers them.
 
 EOF
 }

scripts/dev_server.py

@@ -142,17 +142,18 @@ def find_plugin_dir(plugin_id: str) -> Optional[Path]:
 
 
 def load_config_defaults(plugin_dir: 'str | Path') -> Dict[str, Any]:
-    """Extract default values from config_schema.json."""
+    """Extract default values from config_schema.json.
+
+    The same extraction a device and the plugin harness use
+    (src/plugin_system/testing/loading.py), nested defaults included.
+    """
+    from src.plugin_system.testing.loading import (
+        load_config_defaults as _load_config_defaults,
+    )
     schema_path = resolve_under(plugin_dir, 'config_schema.json')
     if schema_path is None or not schema_path.exists():
         return {}
-    with open(schema_path, 'r') as f:
-        schema = json.load(f)
-    defaults: Dict[str, Any] = {}
-    for key, prop in schema.get('properties', {}).items():
-        if 'default' in prop:
-            defaults[key] = prop['default']
-    return defaults
+    return _load_config_defaults(schema_path.parent)
 
 
 # --------------------------------------------------------------------------
@@ -303,10 +304,13 @@ def _parse_render_request(data):
     with open(manifest_path, 'r') as f:
         manifest = json.load(f)
 
-    # Build config: schema defaults + user overrides
-    config = {'enabled': True}
-    config.update(load_config_defaults(trusted_dir))
-    config.update(data.get('config', {}))
+    # Build config the way a device would: schema defaults under a forced
+    # enabled, with the user's overrides deep-merged on top
+    from src.plugin_system.testing.loading import build_config
+    overrides = data.get('config') or {}
+    if not isinstance(overrides, dict):
+        raise ValueError('config must be a JSON object')
+    config = build_config(trusted_dir, overrides)
 
     return trusted_dir, manifest, config, data.get('mock_data', {}), data.get('skip_update', False)
 

scripts/diagnose_web_interface.sh

@@ -20,6 +20,38 @@ PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
 
 cd "$PROJECT_DIR"
 
+# Report web_display_autostart the way scripts/utils/start_web_conditionally.py
+# (what ledmatrix-web.service runs) decides it: only an explicit false/off keeps
+# the web interface down; a missing key or an unreadable config starts it.
+# Prints "on <value>", "off <value>", "default" (key not set) or "unreadable".
+web_autostart_state() {
+    (cd "$1" && python3 - 2>/dev/null <<'PY'
+import json, os, sys
+sys.path.insert(0, os.path.join(os.getcwd(), "scripts", "utils"))
+try:
+    from start_web_conditionally import autostart_enabled
+except Exception:
+    def autostart_enabled(config):
+        value = config.get("web_display_autostart", True)
+        if isinstance(value, str):
+            return value.strip().lower() not in ("off", "false", "no", "0")
+        return bool(value)
+try:
+    with open(os.path.join("config", "config.json"), encoding="utf-8") as f:
+        config = json.load(f)
+except Exception:
+    config = None
+if not isinstance(config, dict):
+    print("unreadable")
+elif "web_display_autostart" not in config:
+    print("default")
+else:
+    raw = json.dumps(config["web_display_autostart"])
+    print(("on " if autostart_enabled(config) else "off ") + raw)
+PY
+    ) || echo "unknown"
+}
+
 echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
 echo -e "${BLUE}1. SERVICE STATUS${NC}"
 echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
@@ -41,14 +73,26 @@ if [ -f "$PROJECT_DIR/config/config.json" ]; then
     echo -e "${GREEN}✓ Config file found${NC}"
     
     # Check web_display_autostart setting
-    AUTOSTART=$(grep -o '"web_display_autostart"[[:space:]]*:[[:space:]]*[a-z]*' "$PROJECT_DIR/config/config.json" | grep -o '[a-z]*$')
-    
-    if [ "$AUTOSTART" == "true" ]; then
-        echo -e "${GREEN}✓ web_display_autostart: true${NC}"
-    else
-        echo -e "${YELLOW}⚠ web_display_autostart: ${AUTOSTART:-not set}${NC}"
-        echo -e "${YELLOW}  Web interface will not start unless this is set to true${NC}"
-    fi
+    AUTOSTART=$(web_autostart_state "$PROJECT_DIR")
+
+    case "$AUTOSTART" in
+        on\ *)
+            echo -e "${GREEN}✓ web_display_autostart: ${AUTOSTART#on }${NC}"
+            ;;
+        off\ *)
+            echo -e "${YELLOW}⚠ web_display_autostart: ${AUTOSTART#off }${NC}"
+            echo -e "${YELLOW}  Web interface will not start with this value${NC}"
+            ;;
+        default)
+            echo -e "${GREEN}✓ web_display_autostart: not set (defaults to on)${NC}"
+            ;;
+        unreadable)
+            echo -e "${YELLOW}⚠ config.json could not be parsed (the web interface still starts so it can be repaired)${NC}"
+            ;;
+        *)
+            echo -e "${YELLOW}⚠ web_display_autostart: could not be evaluated (python3 unavailable?)${NC}"
+            ;;
+    esac
 else
     echo -e "${RED}✗ Config file not found at: $PROJECT_DIR/config/config.json${NC}"
 fi
@@ -63,7 +107,7 @@ declare -a REQUIRED_FILES=(
     "web_interface/app.py"
     "web_interface/start.py"
     "web_interface/requirements.txt"
-    "web_interface/blueprints/api_v3.py"
+    "web_interface/blueprints/api_v3/__init__.py"
     "web_interface/blueprints/pages_v3.py"
     "scripts/utils/start_web_conditionally.py"
 )
@@ -134,8 +178,8 @@ if ! sudo systemctl is-active --quiet ledmatrix-web; then
     echo "   sudo systemctl start ledmatrix-web"
 fi
 
-if [ "$AUTOSTART" != "true" ]; then
-    echo -e "${YELLOW}→ Enable web_display_autostart in config/config.json${NC}"
+if [ "${AUTOSTART%% *}" = "off" ]; then
+    echo -e "${YELLOW}→ Set web_display_autostart to true in config/config.json (or remove it; missing means on)${NC}"
 fi
 
 if [ "$ALL_FILES_OK" = false ]; then

scripts/diagnose_web_ui.sh

@@ -22,6 +22,38 @@ fi
 
 PROJECT_DIR="${HOME}/LEDMatrix"
 
+# Report web_display_autostart the way scripts/utils/start_web_conditionally.py
+# (what ledmatrix-web.service runs) decides it: only an explicit false/off keeps
+# the web interface down; a missing key or an unreadable config starts it.
+# Prints "on <value>", "off <value>", "default" (key not set) or "unreadable".
+web_autostart_state() {
+    (cd "$1" && python3 - 2>/dev/null <<'PY'
+import json, os, sys
+sys.path.insert(0, os.path.join(os.getcwd(), "scripts", "utils"))
+try:
+    from start_web_conditionally import autostart_enabled
+except Exception:
+    def autostart_enabled(config):
+        value = config.get("web_display_autostart", True)
+        if isinstance(value, str):
+            return value.strip().lower() not in ("off", "false", "no", "0")
+        return bool(value)
+try:
+    with open(os.path.join("config", "config.json"), encoding="utf-8") as f:
+        config = json.load(f)
+except Exception:
+    config = None
+if not isinstance(config, dict):
+    print("unreadable")
+elif "web_display_autostart" not in config:
+    print("default")
+else:
+    raw = json.dumps(config["web_display_autostart"])
+    print(("on " if autostart_enabled(config) else "off ") + raw)
+PY
+    ) || echo "unknown"
+}
+
 echo "1. Checking service status..."
 echo "------------------------------"
 if systemctl is-active --quiet ledmatrix-web 2>/dev/null || sudo systemctl is-active --quiet ledmatrix-web 2>/dev/null; then
@@ -47,16 +79,25 @@ echo "3. Checking configuration file..."
 echo "------------------------------"
 if [ -f "${PROJECT_DIR}/config/config.json" ]; then
     echo -e "${GREEN}✓ Config file exists${NC}"
-    AUTOSTART=$(grep -o '"web_display_autostart":\s*\(true\|false\)' "${PROJECT_DIR}/config/config.json" | grep -o '\(true\|false\)' || echo "not found")
-    if [ "$AUTOSTART" = "true" ]; then
-        echo -e "${GREEN}✓ web_display_autostart is set to TRUE${NC}"
-    elif [ "$AUTOSTART" = "false" ]; then
-        echo -e "${RED}✗ web_display_autostart is set to FALSE (web UI won't start!)${NC}"
-        echo "   Fix: Edit config.json and set 'web_display_autostart': true"
-    else
-        echo -e "${YELLOW}⚠ web_display_autostart setting not found (defaults to false)${NC}"
-        echo "   Fix: Add 'web_display_autostart': true to config.json"
-    fi
+    AUTOSTART=$(web_autostart_state "$PROJECT_DIR")
+    case "$AUTOSTART" in
+        on\ *)
+            echo -e "${GREEN}✓ web_display_autostart is ${AUTOSTART#on } (web UI starts)${NC}"
+            ;;
+        off\ *)
+            echo -e "${RED}✗ web_display_autostart is ${AUTOSTART#off } (web UI won't start!)${NC}"
+            echo "   Fix: Edit config.json and set 'web_display_autostart': true"
+            ;;
+        default)
+            echo -e "${GREEN}✓ web_display_autostart is not set (defaults to on; web UI starts)${NC}"
+            ;;
+        unreadable)
+            echo -e "${YELLOW}⚠ config.json could not be parsed (the web UI still starts so it can be repaired)${NC}"
+            ;;
+        *)
+            echo -e "${YELLOW}⚠ Could not evaluate web_display_autostart (python3 unavailable?)${NC}"
+            ;;
+    esac
 else
     echo -e "${RED}✗ Config file NOT FOUND at ${PROJECT_DIR}/config/config.json${NC}"
 fi
@@ -82,7 +123,7 @@ FILES_TO_CHECK=(
     "web_interface/start.py"
     "web_interface/app.py"
     "web_interface/requirements.txt"
-    "web_interface/blueprints/api_v3.py"
+    "web_interface/blueprints/api_v3/__init__.py"
     "web_interface/blueprints/pages_v3.py"
 )
 
@@ -175,7 +216,7 @@ echo "Diagnostic Summary"
 echo "=========================================="
 echo ""
 echo "Most common issues:"
-echo "  1. web_display_autostart is false or missing in config.json"
+echo "  1. web_display_autostart is set to false in config.json (a missing key means on)"
 echo "  2. Service not enabled or not started"
 echo "  3. Missing dependencies (Flask, etc.)"
 echo "  4. Import errors in web_interface/app.py"

scripts/fix_perms/README.md

@@ -7,8 +7,10 @@ install as the wrong user, after a manual file copy that didn't preserve
 ownership, or after a permissions-related error from the display or
 web service.
 
-Most of these scripts require `sudo` since they touch directories
-owned by the `ledmatrix` service user or by `root`.
+Most of these scripts require `sudo` since they touch directories owned
+by `root` (the display service's user) or by the user you installed
+LEDMatrix as (the web service's user). There is no dedicated `ledmatrix`
+system user.
 
 ## Scripts
 
@@ -16,11 +18,12 @@ owned by the `ledmatrix` service user or by `root`.
   permissions on the `assets/` tree so plugins can download and cache
   team logos, fonts, and other static content.
 
-- **`fix_cache_permissions.sh`** — Fixes permissions on every cache
-  directory the project may use (`/var/cache/ledmatrix/`,
-  `~/.cache/ledmatrix/`, `/opt/ledmatrix/cache/`, project-local
-  `cache/`). Also creates placeholder logo subdirectories used by the
-  sports plugins.
+- **`fix_cache_permissions.sh`** — Creates (if missing) and fixes
+  permissions on `/var/cache/ledmatrix/` and `~/.ledmatrix_cache/` of the
+  user running `sudo`, and creates
+  `/var/cache/ledmatrix/placeholder_logos/` for the sports plugins. It does
+  not touch the cache manager's other fallbacks (`/opt/ledmatrix/cache`,
+  `$TMPDIR/ledmatrix_cache`).
 
 - **`fix_plugin_permissions.sh`** — Fixes ownership on the plugins
   directory so both the root display service and the web service user
@@ -31,6 +34,11 @@ owned by the `ledmatrix` service user or by `root`.
   systemd journal access, and the sudoers entries the web interface
   needs to control the display service.
 
+- **`safe_pip_install.sh`** — Installs a `requirements.txt` as root
+  after checking it is the project's own or one under `plugin-repos/` or
+  `plugins/`. Used by the web interface (via sudo) to install plugin
+  dependencies where `ledmatrix.service` can import them.
+
 - **`safe_plugin_rm.sh`** — Validates that a plugin removal path is
   inside an allowed base directory before deleting it. Used by the web
   interface (via sudo) when a user clicks **Uninstall** on a plugin —

scripts/fix_perms/safe_pip_install.sh

@@ -1,6 +1,7 @@
 #!/bin/bash
 # safe_pip_install.sh — Install a requirements.txt as root after validating
-# that the resolved path is the project's own requirements.txt or a plugin's
+# that the resolved path is one of the project's own requirements files
+# (requirements.txt, web_interface/requirements.txt) or a plugin's
 # requirements.txt under plugin-repos/ or plugins/.
 #
 # This script is intended to be called via sudo from the web interface, so
@@ -25,9 +26,17 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
 
 # Allowed locations (resolved, no trailing slash):
-#   - the project's own requirements.txt
+#   - the project's own requirements files. Update Code, the automatic
+#     update's health check and Install Base Requirements install both, so
+#     one missing here is refused on every device -- and the automatic
+#     updater rolls back any update that changes it.
+#     Only their folders are resolved: resolving the files too would follow
+#     a requirements.txt symlinked out of the project and allow its target.
 #   - any requirements.txt under plugin-repos/ or plugins/
-ALLOWED_EXACT="$(realpath --canonicalize-missing "$PROJECT_ROOT/requirements.txt")"
+ALLOWED_EXACT=(
+    "$(realpath --canonicalize-missing "$PROJECT_ROOT")/requirements.txt"
+    "$(realpath --canonicalize-missing "$PROJECT_ROOT/web_interface")/requirements.txt"
+)
 ALLOWED_BASES=(
     "$(realpath --canonicalize-missing "$PROJECT_ROOT/plugin-repos")"
     "$(realpath --canonicalize-missing "$PROJECT_ROOT/plugins")"
@@ -43,9 +52,13 @@ if [ "$(basename "$RESOLVED_TARGET")" != "requirements.txt" ]; then
 fi
 
 ALLOWED=false
-if [ "$RESOLVED_TARGET" = "$ALLOWED_EXACT" ]; then
-    ALLOWED=true
-else
+for EXACT in "${ALLOWED_EXACT[@]}"; do
+    if [ "$RESOLVED_TARGET" = "$EXACT" ]; then
+        ALLOWED=true
+        break
+    fi
+done
+if [ "$ALLOWED" = false ]; then
     for BASE in "${ALLOWED_BASES[@]}"; do
         if [[ "$RESOLVED_TARGET" == "$BASE/"* ]]; then
             ALLOWED=true
@@ -56,7 +69,7 @@ fi
 
 if [ "$ALLOWED" = false ]; then
     echo "DENIED: $RESOLVED_TARGET is not an allowed requirements.txt location" >&2
-    echo "Allowed: $ALLOWED_EXACT, or any requirements.txt under: ${ALLOWED_BASES[*]}" >&2
+    echo "Allowed: ${ALLOWED_EXACT[*]}, or any requirements.txt under: ${ALLOWED_BASES[*]}" >&2
     exit 2
 fi
 

scripts/install/README.md

@@ -7,14 +7,18 @@ This directory contains scripts for installing and configuring the LEDMatrix sys
 - **`one-shot-install.sh`** - Single-command installer; clones the
   repo, checks prerequisites, then runs `first_time_install.sh`.
   Invoked via `curl ... | bash` from the project root README.
-- **`install_service.sh`** - Installs the main LED Matrix display service (systemd)
-- **`install_web_service.sh`** - Installs the web interface service (systemd)
+- **`install_service.sh`** - Installs, enables and starts the display
+  service (`ledmatrix.service`), the web interface service
+  (`ledmatrix-web.service`) and the update-verify units (systemd)
+- **`install_web_service.sh`** - Installs only the web interface service
+  and the update-verify units (systemd)
 - **`install_wifi_monitor.sh`** - Installs the WiFi monitor daemon service
 - **`setup_cache.sh`** - Sets up persistent cache directory with proper permissions
 - **`configure_web_sudo.sh`** - Configures passwordless sudo access for web interface actions
-- **`configure_wifi_permissions.sh`** - Grants the `ledmatrix` user
-  the WiFi management permissions needed by the web interface and
-  the WiFi monitor service
+- **`configure_wifi_permissions.sh`** - Grants the web interface's user
+  (the user who runs the script, i.e. the one you installed LEDMatrix as;
+  there is no `ledmatrix` system user) the passwordless `nmcli` and related
+  WiFi permissions the web interface needs
 - **`migrate_config.sh`** - Migrates configuration files to new formats (if needed)
 - **`debug_install.sh`** - Diagnostic helper used when an install
   fails; collects environment info and recent logs

scripts/install/install_service.sh

@@ -3,6 +3,41 @@
 # Exit on error
 set -e
 
+usage() {
+    cat <<'USAGE'
+Usage: sudo ./scripts/install/install_service.sh [-h|--help]
+
+Installs (or reinstalls) the LEDMatrix systemd units from the templates in
+systemd/, then enables and starts them:
+  - ledmatrix.service                  main display (runs as root)
+  - ledmatrix-web.service              web interface (runs as the invoking user)
+  - ledmatrix-update-verify.service    automatic-update health check
+  - ledmatrix-update-verify.path
+
+Existing unit files in /etc/systemd/system are overwritten. The script takes
+no other options; run it with no arguments to install.
+
+Options:
+  -h, --help    Show this help and exit without changing anything.
+USAGE
+}
+
+# Parse arguments before touching anything: this script rewrites and restarts
+# services, so an unrecognised option must not fall through to a full install.
+for arg in "$@"; do
+    case "$arg" in
+        -h|--help)
+            usage
+            exit 0
+            ;;
+        *)
+            echo "ERROR: unknown option: $arg" >&2
+            usage >&2
+            exit 2
+            ;;
+    esac
+done
+
 # Get the actual user who invoked sudo
 if [ -n "$SUDO_USER" ]; then
     ACTUAL_USER="$SUDO_USER"

scripts/install_plugin_dependencies.sh

@@ -3,6 +3,8 @@
 # Use this if automatic dependency installation fails
 
 set -e
+# A failed pip must fail the `pip ... | tee` pipeline below, not be hidden by tee.
+set -o pipefail
 
 # Colors for output
 RED='\033[0;31m'
@@ -28,15 +30,50 @@ echo ""
 # Get the directory where this script is located
 SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 LEDMATRIX_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
-PLUGINS_DIR="$LEDMATRIX_DIR/plugins"
+CONFIG_FILE="$LEDMATRIX_DIR/config/config.json"
+
+# The Plugin Store installs into plugin_system.plugins_directory from
+# config.json (default plugin-repos), resolved against the project root like
+# the display and web services do. plugins/ is also scanned: it holds the
+# symlinks scripts/dev/dev_plugin_setup.sh creates.
+CONFIGURED_DIR="plugin-repos"
+if [ -f "$CONFIG_FILE" ] && command -v python3 >/dev/null 2>&1; then
+    CONFIGURED_DIR="$(LEDMATRIX_CONFIG_FILE="$CONFIG_FILE" python3 -c '
+import json, os
+try:
+    with open(os.environ["LEDMATRIX_CONFIG_FILE"], encoding="utf-8") as f:
+        value = (json.load(f).get("plugin_system") or {}).get("plugins_directory")
+except Exception:
+    value = None
+print(value if isinstance(value, str) and value.strip() else "plugin-repos")
+' 2>/dev/null)" || CONFIGURED_DIR="plugin-repos"
+    [ -n "$CONFIGURED_DIR" ] || CONFIGURED_DIR="plugin-repos"
+fi
+case "$CONFIGURED_DIR" in
+    /*) PLUGINS_DIR="$CONFIGURED_DIR" ;;
+    *)  PLUGINS_DIR="$LEDMATRIX_DIR/$CONFIGURED_DIR" ;;
+esac
+DEV_PLUGINS_DIR="$LEDMATRIX_DIR/plugins"
 
 echo "LEDMatrix directory: $LEDMATRIX_DIR"
-echo "Plugins directory: $PLUGINS_DIR"
+echo "Plugins directory: $PLUGINS_DIR (plugin_system.plugins_directory)"
+
+SCAN_DIRS=()
+PLUGINS_DIR_REAL=""
+if [ -d "$PLUGINS_DIR" ]; then
+    SCAN_DIRS+=("$PLUGINS_DIR")
+    PLUGINS_DIR_REAL="$(cd "$PLUGINS_DIR" && pwd -P)"
+fi
+if [ -d "$DEV_PLUGINS_DIR" ] && [ "$(cd "$DEV_PLUGINS_DIR" && pwd -P)" != "$PLUGINS_DIR_REAL" ]; then
+    echo "Also scanning dev plugins: $DEV_PLUGINS_DIR"
+    SCAN_DIRS+=("$DEV_PLUGINS_DIR")
+fi
 echo ""
 
-# Check if plugins directory exists
-if [ ! -d "$PLUGINS_DIR" ]; then
+# Check if a plugins directory exists
+if [ ${#SCAN_DIRS[@]} -eq 0 ]; then
     echo -e "${RED}Error: Plugins directory not found at $PLUGINS_DIR${NC}"
+    echo "Install a plugin from the Plugin Store first, or check plugin_system.plugins_directory in $CONFIG_FILE"
     exit 1
 fi
 
@@ -47,12 +84,21 @@ echo ""
 PLUGINS_FOUND=0
 PLUGINS_INSTALLED=0
 PLUGINS_FAILED=0
+SEEN_PLUGIN_PATHS=" "
 
-for plugin_dir in "$PLUGINS_DIR"/*/ ; do
+for scan_dir in "${SCAN_DIRS[@]}"; do
+for plugin_dir in "$scan_dir"/*/ ; do
     if [ -d "$plugin_dir" ]; then
         plugin_name=$(basename "$plugin_dir")
         requirements_file="$plugin_dir/requirements.txt"
-        
+
+        # A dev symlink can point at a plugin already scanned; install it once.
+        real_plugin_dir="$(cd "$plugin_dir" && pwd -P)"
+        case "$SEEN_PLUGIN_PATHS" in
+            *" $real_plugin_dir "*) continue ;;
+        esac
+        SEEN_PLUGIN_PATHS="$SEEN_PLUGIN_PATHS$real_plugin_dir "
+
         if [ -f "$requirements_file" ]; then
             PLUGINS_FOUND=$((PLUGINS_FOUND + 1))
             echo -e "${GREEN}Found plugin: ${plugin_name}${NC}"
@@ -79,6 +125,7 @@ for plugin_dir in "$PLUGINS_DIR"/*/ ; do
         fi
     fi
 done
+done
 
 # Summary
 echo ""

scripts/scroll_speeds.py

@@ -47,52 +47,47 @@ from src.common import scroll_config  # noqa: E402
 CONFIG = Path(__file__).resolve().parent.parent / "config" / "config.json"
 
 
-def load_hardware():
+def load_config():
+    """The whole config.json, or {} when it is missing or unreadable."""
     try:
         with open(CONFIG, encoding="utf-8") as handle:
-            return (json.load(handle).get("display") or {}).get("hardware") or {}
+            config = json.load(handle)
     except (OSError, ValueError):
         return {}
+    return config if isinstance(config, dict) else {}
 
 
-def build_options(hardware, refresh_override=None):
-    from rgbmatrix import RGBMatrixOptions
-
-    o = RGBMatrixOptions()
-    from src.display_geometry import (
-        DEFAULT_CHAIN_LENGTH, DEFAULT_COLS, DEFAULT_PARALLEL, DEFAULT_ROWS,
-    )
-    o.rows = int(hardware.get("rows", DEFAULT_ROWS))
-    o.cols = int(hardware.get("cols", DEFAULT_COLS))
-    o.chain_length = int(hardware.get("chain_length", DEFAULT_CHAIN_LENGTH))
-    o.parallel = int(hardware.get("parallel", DEFAULT_PARALLEL))
-    o.brightness = int(hardware.get("brightness", 80))
-    o.hardware_mapping = hardware.get("hardware_mapping", "regular")
-    o.pwm_bits = int(hardware.get("pwm_bits", 11))
-    o.pwm_dither_bits = int(hardware.get("pwm_dither_bits", 0))
-    o.pwm_lsb_nanoseconds = int(hardware.get("pwm_lsb_nanoseconds", 130))
-    o.led_rgb_sequence = hardware.get("led_rgb_sequence", "RGB")
-    o.scan_mode = int(hardware.get("scan_mode", 0))
-    o.row_address_type = int(hardware.get("row_address_type", 0))
-    o.multiplexing = int(hardware.get("multiplexing", 0))
-    o.gpio_slowdown = int(hardware.get("gpio_slowdown", 2))
-    o.limit_refresh_rate_hz = (
-        int(refresh_override) if refresh_override is not None
-        else int(hardware.get("limit_refresh_rate_hz", 0))
-    )
-    return o
+def hardware_of(config):
+    return (config.get("display") or {}).get("hardware") or {}
 
 
-def open_matrix(hardware, refresh_override=None):
+def build_options(config, refresh_override=None):
+    """The matrix options the display service would use for this config.
+
+    Built by DisplayManager.apply_matrix_options, not a copy of it, so the
+    measurement and the demo drive the panel exactly as the service does
+    (runtime gpio_slowdown, rp1_rio, panel_type, orientation, defaults).
+    ``refresh_override`` replaces limit_refresh_rate_hz; 0 means uncapped.
+    """
+    from src.display_manager import DisplayManager, RGBMatrixOptions
+
+    options = DisplayManager.apply_matrix_options(RGBMatrixOptions(), config)
+    if refresh_override is not None:
+        options.limit_refresh_rate_hz = int(refresh_override)
+    return options
+
+
+def open_matrix(config, refresh_override=None):
     """Construct the matrix, or explain why it will not open."""
     if os.geteuid() != 0:
         sys.exit("this needs root for GPIO access - rerun with sudo")
     try:
-        from rgbmatrix import RGBMatrix
-    except ImportError:
-        sys.exit("rgbmatrix is not installed on this machine")
+        from src.display_manager import RGBMatrix
+    except ImportError as exc:
+        sys.exit("could not load the display stack ({}); is rgbmatrix "
+                 "installed on this machine?".format(exc))
     try:
-        return RGBMatrix(options=build_options(hardware, refresh_override))
+        return RGBMatrix(options=build_options(config, refresh_override))
     except Exception as exc:  # pragma: no cover - hardware dependent
         sys.exit(
             "could not open the panel ({}).\n"
@@ -101,7 +96,7 @@ def open_matrix(hardware, refresh_override=None):
         )
 
 
-def measure_refresh(hardware, seconds=6.0):
+def measure_refresh(config, seconds=6.0):
     """Actual refresh rate, by running uncapped and timing the swaps.
 
     SwapOnVSync blocks until the panel's next refresh, so an unthrottled loop
@@ -109,7 +104,7 @@ def measure_refresh(hardware, seconds=6.0):
     chain will really give you, as opposed to whatever limit_refresh_rate_hz
     optimistically asks for.
     """
-    matrix = open_matrix(hardware, refresh_override=0)
+    matrix = open_matrix(config, refresh_override=0)
     canvas = matrix.CreateFrameCanvas()
     canvas = matrix.SwapOnVSync(canvas)  # discard the first, it includes setup
     frames = 0
@@ -122,17 +117,17 @@ def measure_refresh(hardware, seconds=6.0):
     return measured
 
 
-def demo(hardware, target, seconds):
+def demo(config, target, seconds):
     """Scroll text at the crisp speed nearest `target`."""
     from PIL import Image, ImageDraw, ImageFont
 
     from src.common.font_layout import load_truetype
 
-    hz = float(hardware.get("limit_refresh_rate_hz") or scroll_config.DEFAULT_REFRESH_HZ)
+    hz = scroll_config.refresh_hz_from_config(config)
     choice = scroll_config.solve_crisp(target, hz)
     print("asked for {:.0f} px/s -> {}".format(target, choice.describe()))
 
-    matrix = open_matrix(hardware)
+    matrix = open_matrix(config)
     canvas = matrix.CreateFrameCanvas()
     W, H = canvas.width, canvas.height
 
@@ -195,9 +190,38 @@ def print_ladder(hz, highlight=None):
         ) else ""
         print("  " + entry.describe() + mark)
     print("")
-    print("Set one in config.json as pixels per second, e.g.")
-    print('  "display_options": {{"scroll_pixels_per_second": {:.0f}}}'.format(
-        scroll_config.solve_crisp(highlight if highlight else hz / 2, hz).pixels_per_second))
+    print_config_advice(scroll_config.solve_crisp(highlight if highlight else hz / 2, hz))
+
+
+def config_advice(choice):
+    """The config that selects ``choice``, in the keys the resolver honours.
+
+    Tickers take a ``scroll_speed`` (px per step) + ``scroll_delay`` (seconds)
+    pair, and scroll_config ranks that pair ABOVE ``scroll_pixels_per_second``
+    -- deliberately, because some plugins give the flat key a schema default.
+    Many schemas default the pair too, so a flat key added by hand is usually
+    ignored. Advise the pair: pixels_per_frame every frame_hold/refresh
+    seconds is exactly the crisp speed.
+    """
+    pair = {
+        "scroll_speed": choice.pixels_per_frame,
+        "scroll_delay": round(choice.frame_hold / choice.refresh_hz, 6),
+    }
+    scoreboard = {"scroll_speed": round(choice.pixels_per_second, 2)}
+    return pair, scoreboard
+
+
+def print_config_advice(choice):
+    pair, scoreboard = config_advice(choice)
+    print("To use {:.1f} px/s, set it where the plugin keeps its scroll speed.".format(
+        choice.pixels_per_second))
+    print("Tickers take a scroll_speed (px per step) + scroll_delay (seconds) pair:")
+    print('  "display_options": {}'.format(json.dumps(pair)))
+    print("(some plugins keep the pair at the top level or under \"display\").")
+    print("The pair outranks scroll_pixels_per_second, which is ignored whenever the")
+    print("pair is present -- and schema defaults usually put it there.")
+    print("Sports scoreboards take pixels per second per league instead:")
+    print('  "scroll_settings": {}'.format(json.dumps(scoreboard)))
 
 
 def main():
@@ -214,15 +238,15 @@ def main():
                     help="highlight the entry nearest this speed")
     args = ap.parse_args()
 
-    hardware = load_hardware()
-    configured = float(hardware.get("limit_refresh_rate_hz") or 0)
+    config = load_config()
+    configured = float(hardware_of(config).get("limit_refresh_rate_hz") or 0)
 
     if args.demo is not None:
-        demo(hardware, args.demo, args.seconds)
+        demo(config, args.demo, args.seconds)
         return
 
     if args.measure:
-        measured = measure_refresh(hardware)
+        measured = measure_refresh(config)
         print("measured panel refresh: {:.1f}Hz".format(measured))
         if configured:
             print("configured limit_refresh_rate_hz: {:.0f}".format(configured))

scripts/utils/auto_update_verify.py

@@ -42,10 +42,30 @@ HEALTH_TIMEOUT_SECONDS = 180
 #: loop look healthy between attempts, so a single "is-active" proves nothing.
 STABLE_SECONDS = 45
 POLL_SECONDS = 5
+WEB_CHECK_TIMEOUT_SECONDS = 5
+SYSTEMCTL_QUERY_TIMEOUT_SECONDS = 10
+RESTART_TIMEOUT_SECONDS = 90
+GIT_TIMEOUT_SECONDS = 60
+GIT_RESET_TIMEOUT_SECONDS = 120
 PIP_TIMEOUT_SECONDS = 600
+#: All of a rollback's dependency reinstalls together. A pip that times out
+#: or fails is not retried: systemd stops this unit at TimeoutStartSec, and a
+#: rollback killed half-way leaves the update reported as still verifying.
+PIP_BUDGET_SECONDS = 600
 #: sudoers matches the exact command line, so bash is named by path, the same
-#: candidates src/common/permission_utils.install_requirements_file tries.
+#: candidates src/common/permission_utils.install_requirements_file tries...
 BASH_CANDIDATES = ('/usr/bin/bash', '/bin/bash')
+#: ...and, like it, moves to the next one only when sudo refused the command
+#: line (permission_utils.SUDO_REFUSAL_PHRASES), never after pip itself ran.
+SUDO_REFUSAL_PHRASES = ('a password is required', 'is not allowed to run', 'no tty present')
+
+#: The longest one health check can take: restart and wait, roll back
+#: (diff, reset, reinstalls), restart and wait again. A wait's last poll can
+#: start just before its deadline and run every query to its timeout.
+_WAIT_WORST_SECONDS = (HEALTH_TIMEOUT_SECONDS + STABLE_SECONDS + WEB_CHECK_TIMEOUT_SECONDS
+                       + 2 * SYSTEMCTL_QUERY_TIMEOUT_SECONDS + POLL_SECONDS)
+WORST_CASE_SECONDS = (2 * (2 * RESTART_TIMEOUT_SECONDS + _WAIT_WORST_SECONDS)
+                      + GIT_TIMEOUT_SECONDS + GIT_RESET_TIMEOUT_SECONDS + PIP_BUDGET_SECONDS)
 
 #: What a command that could not run at all reports: its callers only read
 #: these three fields, the same ones a completed subprocess has.
@@ -83,7 +103,7 @@ def write_pending(path, data):
 
 def _web_responds(url=WEB_HEALTH_URL):
     try:
-        with urllib.request.urlopen(url, timeout=5) as resp:  # nosec B310 - fixed loopback URL
+        with urllib.request.urlopen(url, timeout=WEB_CHECK_TIMEOUT_SECONDS) as resp:  # nosec B310 - fixed loopback URL
             return resp.status == 200
     except (urllib.error.URLError, OSError, ValueError):
         return False
@@ -104,7 +124,7 @@ class Verifier:
         self.web_responds = web_responds
         self.log = log or (lambda msg: print(f'[auto-update-verify] {msg}', flush=True))
 
-    def _run(self, args, timeout=60):
+    def _run(self, args, timeout=GIT_TIMEOUT_SECONDS):
         try:
             return self.run(args, cwd=str(self.project_root), capture_output=True,
                             text=True, timeout=timeout)
@@ -114,15 +134,17 @@ class Verifier:
     # -- services ---------------------------------------------------------
 
     def service_active(self, unit):
-        return self._run(['systemctl', 'is-active', unit], timeout=10).stdout.strip() == 'active'
+        return self._run(['systemctl', 'is-active', unit],
+                         timeout=SYSTEMCTL_QUERY_TIMEOUT_SECONDS).stdout.strip() == 'active'
 
     def restart_count(self, unit):
         out = self._run(['systemctl', 'show', '-p', 'NRestarts', '--value', unit],
-                        timeout=10).stdout.strip()
+                        timeout=SYSTEMCTL_QUERY_TIMEOUT_SECONDS).stdout.strip()
         return int(out) if out.isdigit() else None
 
     def restart(self, unit):
-        result = self._run(['sudo', '-n', 'systemctl', 'restart', f'{unit}.service'], timeout=90)
+        result = self._run(['sudo', '-n', 'systemctl', 'restart', f'{unit}.service'],
+                           timeout=RESTART_TIMEOUT_SECONDS)
         if result.returncode != 0:
             self.log(f'restarting {unit} failed: {(result.stderr or "").strip()}')
         return result.returncode == 0
@@ -173,16 +195,28 @@ class Verifier:
         changed = set(result.stdout.split()) if result.returncode == 0 else set(REQUIREMENT_FILES)
         return [rel for rel in REQUIREMENT_FILES if rel in changed]
 
-    def install_requirements(self, rel):
+    def install_requirements(self, rel, deadline=None):
+        """Install one requirements file through the root wrapper, by ``deadline``."""
         wrapper = self.project_root / 'scripts' / 'fix_perms' / 'safe_pip_install.sh'
         req = self.project_root / rel
         if not req.exists():
             return True
         for bash in BASH_CANDIDATES:
-            result = self._run(['sudo', '-n', bash, str(wrapper), str(req)],
-                               timeout=PIP_TIMEOUT_SECONDS)
+            timeout = PIP_TIMEOUT_SECONDS
+            if deadline is not None:
+                timeout = min(timeout, deadline - self.clock())
+                if timeout <= 0:
+                    self.log(f'no time left to reinstall {rel}')
+                    return False
+            result = self._run(['sudo', '-n', bash, str(wrapper), str(req)], timeout=timeout)
             if result.returncode == 0:
                 return True
+            # Only a refused command line is worth the next candidate. A pip
+            # that ran and failed, or timed out, would just do it again.
+            if not any(phrase in (result.stderr or '') for phrase in SUDO_REFUSAL_PHRASES):
+                # Not pip's output: it can echo an index URL's credentials.
+                self.log(f'reinstalling {rel} failed (exit {result.returncode})')
+                return False
         return False
 
     def rollback(self, pending):
@@ -191,13 +225,17 @@ class Verifier:
         if not old:
             return False, 'the commit to roll back to is unknown'
         requirements = self.changed_requirements(old, new) if new else list(REQUIREMENT_FILES)
-        # Safe to --hard: the updater refuses to run with local edits to
-        # tracked files, so the only thing this discards is the update.
-        result = self._run(['git', 'reset', '--hard', old], timeout=120)
+        # --hard: the updater refuses to run with local edits to tracked core
+        # files (web_interface/auto_update.local_changes), so outside the
+        # plugin folders the only thing this discards is the update. Edits
+        # under plugins/ and plugin-repos/, which that check leaves to the
+        # pull's --autostash, are reset along with it.
+        result = self._run(['git', 'reset', '--hard', old], timeout=GIT_RESET_TIMEOUT_SECONDS)
         if result.returncode != 0:
             return False, (f'"git reset --hard {old}" failed: '
                            f'{(result.stderr or result.stdout or "").strip()}')
-        failed = [rel for rel in requirements if not self.install_requirements(rel)]
+        deadline = self.clock() + PIP_BUDGET_SECONDS
+        failed = [rel for rel in requirements if not self.install_requirements(rel, deadline)]
         if failed:
             return True, ('reinstalling the previous dependencies from ' + ', '.join(failed)
                           + ' failed; run Install Base Requirements from the Tools tab')

scripts/verify_installation.sh

@@ -121,19 +121,24 @@ fi
 echo ""
 
 # 5. Check web interface
+# ledmatrix-web.service runs scripts/utils/start_web_conditionally.py, which
+# starts web_interface/start.py; the app binds port 5000 (web_interface/start.py).
 echo "=== Web Interface ==="
-if [ -f "$PROJECT_ROOT/web_interface_v2.py" ]; then
-    check_pass "web_interface_v2.py exists"
-else
-    check_fail "web_interface_v2.py is missing"
-fi
+WEB_PORT=5000
+for web_file in scripts/utils/start_web_conditionally.py web_interface/start.py web_interface/app.py; do
+    if [ -f "$PROJECT_ROOT/$web_file" ]; then
+        check_pass "$web_file exists"
+    else
+        check_fail "$web_file is missing"
+    fi
+done
 
 # Check if web service is listening
 if systemctl is-active --quiet ledmatrix-web.service 2>/dev/null; then
-    if netstat -tuln 2>/dev/null | grep -q ":5001" || ss -tuln 2>/dev/null | grep -q ":5001"; then
-        check_pass "Web interface is listening on port 5001"
+    if netstat -tuln 2>/dev/null | grep -qE ":${WEB_PORT}([^0-9]|$)" || ss -tuln 2>/dev/null | grep -qE ":${WEB_PORT}([^0-9]|$)"; then
+        check_pass "Web interface is listening on port $WEB_PORT"
     else
-        check_warn "Web service is running but port 5001 may not be listening"
+        check_warn "Web service is running but port $WEB_PORT may not be listening"
     fi
 else
     check_warn "Web service is not running (cannot check port)"
@@ -204,7 +209,7 @@ if [ "$ALL_PASSED" = true ]; then
     echo -e "${GREEN}Installation verification PASSED${NC}"
     echo ""
     echo "Next steps:"
-    echo "1. Access the web interface at: http://$(hostname -I | awk '{print $1}'):5001"
+    echo "1. Access the web interface at: http://$(hostname -I | awk '{print $1}'):$WEB_PORT"
     echo "2. Check service status: sudo systemctl status ledmatrix.service"
     echo "3. View logs: journalctl -u ledmatrix.service -f"
     exit 0

scripts/verify_web_ui.sh

@@ -8,6 +8,10 @@ echo "Web UI Verification"
 echo "=========================================="
 echo ""
 
+# The web interface binds port 5000 (web_interface/start.py).
+WEB_PORT=5000
+PORT_PATTERN=":${WEB_PORT}([^0-9]|$)"
+
 # Colors
 GREEN='\033[0;32m'
 RED='\033[0;31m'
@@ -32,25 +36,25 @@ else
 fi
 echo ""
 
-# 2. Check if port 5001 is listening
-echo "2. Checking if port 5001 is listening..."
+# 2. Check if port $WEB_PORT is listening
+echo "2. Checking if port $WEB_PORT is listening..."
 if command -v ss >/dev/null 2>&1; then
-    if ss -tuln 2>/dev/null | grep -q ":5001"; then
-        echo -e "${GREEN}✓${NC} Port 5001 is listening"
+    if ss -tuln 2>/dev/null | grep -qE "$PORT_PATTERN"; then
+        echo -e "${GREEN}✓${NC} Port $WEB_PORT is listening"
         echo ""
-        echo "Active connections on port 5001:"
-        ss -tuln | grep ":5001"
+        echo "Active connections on port $WEB_PORT:"
+        ss -tuln | grep -E "$PORT_PATTERN"
     else
-        echo -e "${RED}✗${NC} Port 5001 is NOT listening"
+        echo -e "${RED}✗${NC} Port $WEB_PORT is NOT listening"
     fi
 elif command -v netstat >/dev/null 2>&1; then
-    if netstat -tuln 2>/dev/null | grep -q ":5001"; then
-        echo -e "${GREEN}✓${NC} Port 5001 is listening"
+    if netstat -tuln 2>/dev/null | grep -qE "$PORT_PATTERN"; then
+        echo -e "${GREEN}✓${NC} Port $WEB_PORT is listening"
         echo ""
-        echo "Active connections on port 5001:"
-        netstat -tuln | grep ":5001"
+        echo "Active connections on port $WEB_PORT:"
+        netstat -tuln | grep -E "$PORT_PATTERN"
     else
-        echo -e "${RED}✗${NC} Port 5001 is NOT listening"
+        echo -e "${RED}✗${NC} Port $WEB_PORT is NOT listening"
     fi
 else
     echo -e "${YELLOW}⚠${NC} Cannot check port (ss/netstat not available)"
@@ -59,15 +63,15 @@ echo ""
 
 # 3. Test HTTP connection
 echo "3. Testing HTTP connection..."
-if curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:5001 > /dev/null 2>&1; then
-    HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:5001 2>/dev/null)
+if curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:$WEB_PORT > /dev/null 2>&1; then
+    HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:$WEB_PORT 2>/dev/null)
     if [ "$HTTP_CODE" = "200" ] || [ "$HTTP_CODE" = "302" ] || [ "$HTTP_CODE" = "301" ]; then
         echo -e "${GREEN}✓${NC} Web interface is responding (HTTP $HTTP_CODE)"
     else
         echo -e "${YELLOW}⚠${NC} Web interface responded with HTTP $HTTP_CODE"
     fi
 else
-    echo -e "${RED}✗${NC} Cannot connect to web interface on port 5001"
+    echo -e "${RED}✗${NC} Cannot connect to web interface on port $WEB_PORT"
 fi
 echo ""
 
@@ -79,7 +83,7 @@ if [ -n "$IP_ADDRESSES" ]; then
     echo ""
     echo "Access web interface at:"
     for ip in $IP_ADDRESSES; do
-        echo "  http://$ip:5001"
+        echo "  http://$ip:$WEB_PORT"
     done
 else
     echo -e "${YELLOW}⚠${NC} Could not determine IP address"
@@ -118,12 +122,12 @@ if systemctl is-active --quiet ledmatrix-web.service 2>/dev/null; then
     SERVICE_RUNNING=true
 fi
 
-if (ss -tuln 2>/dev/null | grep -q ":5001") || (netstat -tuln 2>/dev/null | grep -q ":5001"); then
+if (ss -tuln 2>/dev/null | grep -qE "$PORT_PATTERN") || (netstat -tuln 2>/dev/null | grep -qE "$PORT_PATTERN"); then
     PORT_LISTENING=true
 fi
 
-if curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:5001 > /dev/null 2>&1; then
-    HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:5001 2>/dev/null)
+if curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:$WEB_PORT > /dev/null 2>&1; then
+    HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 http://localhost:$WEB_PORT 2>/dev/null)
     if [ "$HTTP_CODE" = "200" ] || [ "$HTTP_CODE" = "302" ] || [ "$HTTP_CODE" = "301" ]; then
         HTTP_RESPONDING=true
     fi
@@ -134,7 +138,7 @@ if [ "$SERVICE_RUNNING" = true ] && [ "$PORT_LISTENING" = true ] && [ "$HTTP_RES
     echo ""
     echo "You can access it at:"
     for ip in $IP_ADDRESSES; do
-        echo "  http://$ip:5001"
+        echo "  http://$ip:$WEB_PORT"
     done
     exit 0
 elif [ "$SERVICE_RUNNING" = false ]; then
@@ -145,7 +149,7 @@ elif [ "$SERVICE_RUNNING" = false ]; then
     echo "  sudo systemctl enable ledmatrix-web.service  # to start on boot"
     exit 1
 elif [ "$PORT_LISTENING" = false ]; then
-    echo -e "${RED}✗ Service is running but port 5001 is not listening${NC}"
+    echo -e "${RED}✗ Service is running but port $WEB_PORT is not listening${NC}"
     echo ""
     echo "Check logs for errors:"
     echo "  sudo journalctl -u ledmatrix-web.service -f"