rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595)

Browse Source 
* fix(sports): share the ESPN rejected-range memo with the background service

BackgroundDataService always sent a season range first and, on a 400,
fell back to chunks without recording the rejection, so every background
season fetch spent a doomed request and live scoreboards learned nothing
from it (or it from them). The worker now consults and sets the same
6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight
to month/day chunks, and if every chunk fails the range is asked once for
a real error without re-spending the chunks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): keep plugin asset and action routes inside their directories

POST /plugins/assets/upload, GET /plugins/assets/list and POST
/plugins/assets/delete joined the request's plugin_id onto assets/plugins
unchecked, so '../../config' created, wrote, listed and deleted outside
it. #561 guarded only the route that serves the files. All three now go
through path_safety.resolve_under and answer 400 for anything but a
plain name, and delete only unlinks a metadata path that resolves into
that plugin's uploads directory.

PluginManager.get_plugin_directory refuses ids that are not one plain
path segment, so /plugins/action (which runs a manifest script from the
returned directory) and every other caller get the guard; the action
route also rejects such ids up front, covering its no-manager fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): report a no-op plugin update as already up to date

update_plugin() returns True both for a real update and for "nothing to
do" (a ZIP-installed monorepo plugin already at the registry version, a
bundled plugin). With no git commit to compare, POST /plugins/update
called every such success "updated successfully", so Check & Update All
counted most official plugins as updated on every run.

The route now reads what changed off the plugin itself (commit, else
manifest version, else last_updated) and returns data.update_status
(updated / up_to_date / local_only). The update-all toast is summarised
by PluginInstallManager.summarizeUpdateResults from that status, falling
back to the message for older servers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(sports): scoreboard scroll speed no longer follows target_fps

sports_scroll computed the crisp speed ladder against the global
target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since
frame-locked presentation (#545) the helper steps a fixed number of whole
pixels per presented frame and the panel presents at its real refresh, so
the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a
50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s.

The ladder now uses the display manager's refresh_hz, then
display.hardware.limit_refresh_rate_hz, then the default. target_fps is
not consulted. Docstrings now say scroll_delay is ignored for pacing (no
behaviour change there) and describe the fixed-step model.

Tests: replace the tests that pinned target_fps as the ladder refresh and
described time-based stepping; assert speed independence from target_fps
(unit and end-to-end presented px/s against the real helper), that the
fixed per-frame step is applied, and that scroll_delay does not change
speed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): escape registry and upload values in plugin manager inline handlers

The store, saved-repository and custom-registry buttons built
onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone,
so a custom registry entry whose id contained ' closed the attribute and
added its own handler. One helper, jsStringAttr(), now HTML-escapes the
JSON literal for every one of those handlers, and the store View button
opens only http(s) repo links.

The live window.updateImageList (plugins_manager.js loads last, so its
copy wins over the file-upload widget's) wrote the uploaded file's
original name, path and ids into markup raw; they are escaped now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note plugin asset, action and inline handler guards

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): let the root pip wrapper install web_interface/requirements.txt

Update Code, the automatic update's health check and Install Base
Requirements install web_interface/requirements.txt through
safe_pip_install.sh, which only allowed the root requirements.txt. The
first commit changing that file would fail its dependency install, and
the automatic updater rolls back any update whose dependencies did not
install -- on every device, for every newer commit.

The wrapper now lists both core requirement files. Only their folders
are resolved, so a requirements.txt symlinked out of the project is
compared by its target and refused (previously the root file's own
symlink target was what got allowed). The updater's file list is a
named constant, and a test runs the real wrapper (pip stubbed) on
every file Update Code and the rollback install.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): do not retry plugin requests that got an HTTP answer

PluginAPI.request wrapped everything that was not a structured error as
NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a
JSON error without error_code included. Check & Update All retries
NETWORK_ERROR, so those updates were re-sent five more times with
backoff, contrary to the #587 contract that an HTTP error response is
the server's answer.

NETWORK_ERROR now means only that fetch() rejected. Any HTTP response
without an error_code, or with a body that is not JSON, is API_ERROR
with the HTTP status attached. Tested against the shipped api_client.js.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): restart the stats window when an idle gap is dropped by size

#582 dropped an idle gap from the frame stats two ways: the reset_scroll()
sentinel, which also restarts the 5s window timer, and a size guard for
scrollers that never call reset_scroll(), which did not. On that path the
first real frame after the gap found the boundary overdue and logged a
stats line for a one-frame window. Both paths now share one seeding helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): leave plugins alone when update_core's own rollback fails

update_core returns rollback_failed directly when a partial pull or an
update whose health check never started cannot be rolled back. run()
only held plugins back for 'verifying', so those devices still got new
plugin versions and a display restart on top of a core in an unknown
state -- the opposite of what the health-check path does, and of the
3.4.0 changelog (plugins are left alone if the rollback fails).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(api): make the REST reference match the api_v3 package

Every documented request body, query parameter and response shape was
re-checked against the handlers in web_interface/blueprints/api_v3/.
Fixes calls that failed as documented (repo_url, action_id/params,
files/image_id, font_file+font_family, ?font=, cache key,
auto_enable_ap_mode, plugin limit keys), removes the font-override
endpoints dropped in #566, corrects response shapes (plugins/config,
plugins/schema, health, metrics, operation history, github-status,
fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds
the 26 routes it omitted (backup, system auto-update/git, wifi radio,
starlark editor, MQTT bridge, status endpoints, skins).

Documents the merge semantics of partial JSON saves to /config/main and
/plugins/config and the dim-schedule POST accepting GET's days shape,
which land in the same change set. Replaces app.py line numbers and the
removed api_v3.py path with file and function names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): remove the General-tab plugin system toggles that did nothing

plugin_system.auto_discover, auto_load_enabled and development_mode had
General-tab toggles whose help tips promised dormant plugins and verbose
logging, but nothing reads them: every enabled plugin is discovered and
loaded regardless. Remove the three toggles.

The keys stay tolerated in stored configs. The save handler now stores
a flag only when a client sends it; treating a missing key as an
unchecked box would otherwise rewrite all three to false on every
General-tab save, which still posts plugins_directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(scroll): remove dead code left by #523/#570

- Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read
  them since the numpy blend replaced the scipy path.
- Drop ScrollHelper._last_integer_position and frame_time_target, which
  were written but never read.
- Keep target_fps and set_target_fps() but document them as
  informational: nothing paces off them, yet ledmatrix-elections'
  test_scroll_pacing.py reads helper.target_fps back and third-party
  plugins may call the setter.
- Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to
  throttle", set_sub_pixel_scrolling's "default: True", and
  set_frame_based_scrolling's claim that it steps.

The plugins monorepo was grepped for every removed name; none is used.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI

FONT_MANAGER.md told plugins to read display_manager.font_manager, which
does not exist, so a plugin following it failed to load with
AttributeError. The shared FontManager lives on the PluginManager and
BasePlugin._get_font_manager() returns it (with a fallback for harnesses).

Also removes the Fonts-tab override workflow and element-override panels
that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls

/plugins/store/search does not exist (404) and the list endpoint reads
query, not q. The registry guide's curl examples omitted the JSON
Content-Type, so the handlers saw an empty body and answered 400.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(config): use the shared core-key list in the last three private copies

StartupValidator warned "Plugin 'auto_update' is enabled but not found" on
every display start with auto-update or a dim schedule on; the reserved
plugin-id check missed auto_update, sync, location and the rest; and
ConfigManager's (uncalled) orphan cleanup would have deleted display,
schedule and auto_update. All three now read src/core_config_keys.py, which
also gains CORE_SECRETS_KEYS for the github/youtube secrets sections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): partial JSON saves to /config/main change only what they send

A JSON body with one field reset every checkbox in the sections it touched:
the MQTT bridge's brightness slider turned off disable_hardware_pulsing,
inverse_colors, show_refresh_rate and use_short_date_format, and a
timezone-only save turned off web-UI autostart and weekly auto-updates.
Missing-means-unchecked now applies only to form posts: form-encoded bodies
and the v3 forms, which mark themselves with a hidden __form_section input.

Also on the config routes:
- vegas_min/max_cycle_duration no longer match the generic *_duration rule,
  so they stop landing in display_durations and a blank one no longer
  rejects the whole Display save;
- saving from the Raw JSON editor calls start_setup_if_needed like the
  General form, so enabling auto-update there finishes its setup;
- the schedule and dim-schedule POSTs accept the per-day days.<day> shape
  their GETs return, as well as the flat form keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): install plugin dependencies from the configured plugins directory

install_plugin_dependencies.sh scanned only plugins/, but the Plugin
Store installs into plugin_system.plugins_directory (default
plugin-repos), so the documented "Recommended" fix found 0 plugins on
every store install. It now reads plugins_directory from
config/config.json (relative to the project root or absolute, default
plugin-repos) and also scans plugins/ for dev symlinks, installing a
plugin reached through both only once.

With set -e alone, `pip ... | tee` took tee's exit status, so a failed
pip install was reported as success; set -o pipefail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: replace stale API names, line numbers and the api_v3.py path

- ADVANCED_FEATURES: StreamManager methods that exist
  (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the
  real on-demand status envelope ({status, data: {state, service}})
- app.py:199 / :144 / :607-619 line citations and
  web_interface/blueprints/api_v3.py (now a package) replaced with file and
  function names in ADVANCED_FEATURES, CONFIG_DEBUGGING,
  PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE,
  PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README
- CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use
  /config/raw/main to replace the file; describe where validation runs
- TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints
  usage)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): verify the web interface that actually ships, on port 5000

verify_installation.sh failed every healthy install: it required the
long-removed web_interface_v2.py and looked for a listener on port 5001,
while the web interface binds 5000 (web_interface/start.py). It now
checks the files ledmatrix-web.service runs (start_web_conditionally.py,
web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the
same 5001 port in its listen check, HTTP probe and printed URLs.

Port matches are anchored so :50001 no longer counts as :5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): one display-size contract: display_manager.width/height

CLAUDE.md (#580) says to read display_manager.width/height because
matrix is None when hardware init fails; the development guide, the
safety-harness doc and two DisplayManager docstrings still recommended
matrix.width/height. The bundled starlark-apps plugin read matrix.width
unguarded, so its magnify recommendation and frame scaling raised in
fallback mode (e.g. after the Pi 5 hardware refusal).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(install): make install_service.sh --help print usage instead of installing

install_service.sh parsed no arguments, so `sudo ./scripts/install/
install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md)
rewrote ledmatrix.service, ledmatrix-web.service and both update-verify
units and enabled/started them. It now handles -h/--help (usage, exit 0,
no changes) and rejects any other argument with exit 2 before doing
anything. Running it with no arguments, as first_time_install.sh does,
is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scroll): describe the fixed-step model and document frame_hold

Since #545 a crisp speed from scroll_config.configure() makes the helper
advance a fixed whole-pixel step per presented frame with no clock, and the
display manager's frame hold is part of the speed. The docs still described
the removed wall-clock model:

- scroll_config's module and configure() docstrings said speed is applied
  in time-based mode and that omitting the hold "falls back to fractional
  pixels"; omitting it actually runs the scroll frame_hold times too fast.
- SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both
  modes, and read a 20 ms stats median as missed refreshes although that
  is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step,
  the hold-dependent healthy median, that target_fps plays no part, and
  that a hand-added scroll_pixels_per_second loses to a schema-default pair.
- PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling)
  without frame_hold; it now documents the parameter (core 3.4.0) with a
  configure() + set_scrolling_state example.
- update_scroll_position/set_scroll_speed and set_scrolling_state
  docstrings say the same.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark target_fps legacy; describe what Vegas scroll_delay does

- General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after
  the sports_scroll fix nothing in core scrolling reads it. The field and
  its API validation stay so saved configs and plugins that read
  global_config['target_fps'] keep working. CONFIG_REFERENCE says the same.
- Vegas frame_based_scrolling/scroll_delay were described as frame-count
  stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based
  mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and
  still advances by elapsed time, so the applied speed is
  clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The
  config comments, render_pipeline comment and CONFIG_REFERENCE rows now
  say so. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(deps): describe how plugin dependencies are really installed

The guides said the web service runs as root, that installs pick --user
from os.geteuid(), and quoted a warning and a
PluginManager._install_plugin_dependencies() method that don't exist. The
web unit runs as the installing user; store installs go through
install_requirements_file() and sudo safe_pip_install.sh (root), with a
user-level fallback that says so, and load-time installs run in the
display service's own (root) interpreter.

Manual paths now use the configured plugins directory (plugin-repos/ by
default) instead of plugins/, which store installs no longer use, and
install_plugin_dependencies.sh is described as scanning that directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): count local changes one way for the preflight and the pull

The automatic update's preflight ignored mode-only changes and anything
whose status line contained plugins/ or plugin-repos/, then promised
"Automatic updates will not stash your changes". perform_core_update
used plain git status (modes count) and ignored only 'plugins/', then
ran 'git stash push -- :!plugins', which nothing ever pops. So an edit
to a bundled plugin under plugin-repos/, or the installer's chmods on
tracked scripts, passed the preflight and was stashed away for good.

- auto_update.local_changes() is the one predicate both use:
  core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/
  excluded by leading folder rather than substring (a core file under
  web_interface/static/v3/js/plugins/ now counts).
- Update Code's explicit stash leaves out both plugin folders; the
  pull's --autostash carries their edits and mode changes across and
  reapplies them.
- The automatic updater calls perform_core_update(stash_local_changes=
  False), which refuses instead of stashing edits that appeared after
  the preflight; update_core reports that as 'blocked'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): diagnostics follow the web autostart default and api_v3 package

#556 made a missing web_display_autostart mean "start" (only an explicit
false/off keeps the web interface down), but the diagnostics still said
otherwise: diagnose_web_ui.sh reported a missing key as "defaults to
false", diagnose_web_interface.sh said the web interface "will not start
unless this is set to true" and recommended enabling it, and
debug_web_manual.py printed False. Troubleshooting a down web UI pointed
users at a non-cause.

Both shell scripts now evaluate the setting with the launcher's own
autostart_enabled() (inline fallback if it cannot be imported) and report
on / off / not set (on) / unparseable config; debug_web_manual.py uses
the same function. They also check web_interface/blueprints/api_v3/
__init__.py: api_v3.py became a package in #553, so every healthy
checkout was reported as missing a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(install): what install_service.sh installs; verify script port; no sudo for --help

install_service.sh installs and starts ledmatrix, ledmatrix-web and the
update-verify units, not only ledmatrix.service (systemd/README.md,
README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help'
as a harmless check; it now shows --help without sudo and warns what a
real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh
checks the web interface on port 5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note update-all, plugin system settings and script fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): size the preview after orientation and pixel mappers

display_geometry.physical_size claimed to give DisplayManager's answer but
only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured
after the library's pixel mappers, so a Rotate:90 / orientation 90 chain
previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for
128x64.

Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc
does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown
names leave it alone), and move the orientation composition here so
DisplayManager and the preview share it. The module docstring no longer
claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH.

Audit finding F18.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): refuse settings the rgbmatrix library aborts on, on every board

The library answers several settings with a NULL matrix or abort() rather
than an error, so the display service crash-looped (Restart=on-failure)
instead of reaching fallback mode: rows above 64, chain_length above 255
(uint8_t binding setter, documented as "no upper limit"), a misspelled
hardware_mapping, and parallel 2-3 on a single-output mapping, reachable
from the Display form on the default adafruit-hat(-pwm) mapping. #586 only
guarded the Pi 5 subset.

- src/matrix_support.py holds the rules for every board (Options::Validate
  ranges, binding integer types, mapping names and outputs from
  lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the
  API's numeric ranges.
- DisplayManager checks them before building options and raises
  MatrixSettingsRefused, so a hand-edited config falls back with a logged,
  reported reason. Emulator mode only warns.
- The config API refuses them with a 400 naming the setting; combinations
  are checked against stored values but reported only when the request
  sets a field involved.
- The hardware status file gains "cause" (settings/library/forced). The
  fallback log and Display banner give the Pi 5 rebuild hint only for a
  library failure instead of rebuild + gpio_slowdown advice for every
  failure; one Pi 5 slowdown recommendation (1-3, start at 1).
- The Display form offers classic/classic-pi1 and orientation 90/270 and
  renders any other stored mapping selected with a warning, so an
  unrelated save no longer rewrites them; the API accepts 90/270.

Audit findings F03, F16, F19, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(display): library limits, template defaults and Pi 5 slowdown

- rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs,
  classic/classic-pi1 mappings and orientation 90/270 documented.
- Defaults are the config.template.json values: config migration adds
  missing keys from the template, so the listed "code defaults" never
  applied.
- One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode,
  starting at 1.
- Troubleshooting describes the refused-settings fallback, and CHANGELOG
  corrects the Unreleased "no upper limit" entry.

Audit findings F19, F20, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py opens the panel with the service's options

--measure and --demo built RGBMatrixOptions from a private copy of the
display service's builder that had drifted: gpio_slowdown came from
display.hardware (default 2) instead of display.runtime (default 3), and
rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors,
pixel_mapper_config and orientation were skipped, with different defaults
(hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown
was measured -- or garbled -- in a setup the service never drives.

The option filling in DisplayManager._setup_matrix moves, unchanged, into
DisplayManager.apply_matrix_options(options, config), which _setup_matrix
calls and the script reuses (overriding only limit_refresh_rate_hz for
--measure). The script now loads the whole config rather than the hardware
block. Tests pin the script's options to the service's attribute for
attribute.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py recommends keys the resolver honours

The ladder ended by telling users to set
display_options.scroll_pixels_per_second. scroll_config ranks that key
below the scroll_speed + scroll_delay pair, deliberately, and several
plugin schemas default the pair into config, so the advised key was
silently ignored (a schema-default 1/0.02 pair plus an advised 66 still
resolved to 50 px/s).

The advice is now the pair that selects the crisp speed exactly
(pixels_per_frame every frame_hold/refresh seconds), explains that the
pair outranks scroll_pixels_per_second, and gives the scoreboards'
per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the
printed pair over a schema-default pair and check it lands on the
advertised speed and hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula

- SPORTS_UNIFICATION.md still presented honouring global target_fps as
  sports_scroll's added behaviour and its one user-visible gain; note that
  it was withdrawn because it had become a speed multiplier.
- ADVANCED_FEATURES.md gave Vegas scrolling as
  (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s
  by elapsed time, through a 0.1-5 px per scroll_delay clamp when
  frame_based_scrolling is on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): scroll model fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(dev): link-github links plugins from the ledmatrix-plugins monorepo

link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git,
and those per-plugin repositories no longer exist: official plugins are
directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the
monorepo once into the dev directory, finds plugins/<name>,
plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and
links it under its manifest id. With an explicit repo URL it still links a
single-repository plugin as before.

dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a
fork), plus plugins_repo and plugins_branch; github_pattern, which was
documented but never read, is dropped and warned about. Ships
dev_plugins.json.example and git-ignores dev_plugins.json, both of which
the guide promised. Reading JSON falls back to python3 when jq is missing
(get_plugin_id silently returned nothing without jq).

update/status/list find the git checkout above a monorepo plugin
directory (its .git is not in the plugin dir), and update pulls a shared
checkout once. status no longer exits 1 when nothing is broken.

Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration,
workflow, store integration, hello-world link, submission), and the
nonexistent scripts/git-hooks/pre-push-plugin-version and
scripts/bump_plugin_version.py replaced with the real rule: bump the
manifest version and run update_registry.py. scripts/dev/README.md and
CLAUDE.md updated to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scripts): monorepo workspace layout; fix_perms and install READMEs

MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin;
setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into
plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the
workspace file opens LEDMatrix plus ../ledmatrix-plugins.

scripts/fix_perms/README.md listed cache directories
fix_cache_permissions.sh never touches and a 'ledmatrix' service user
that doesn't exist (also in scripts/install/README.md); adds
safe_pip_install.sh. install/README: install_service.sh installs the web
and update-verify units too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): keep the rollback's pip retries inside the unit time limit

The health check reinstalled the previous requirements by trying the
next bash path after any failure, including a 600 s pip timeout. Two
files, two paths: up to 40 minutes of pip alone, while systemd stops
ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the
rollback half-way and leaving the update 'verifying' until the web UI
calls it lost.

- Like permission_utils.install_requirements_file, only a sudo refusal
  moves on to the next bash; a pip that ran and failed or timed out is
  not repeated. The refusal wording is one list
  (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only
  verifier and pinned equal by a test.
- All reinstalls in one rollback share a 600 s budget.
- WORST_CASE_SECONDS adds up every timeout on the longest path (27.5
  min); a test holds it under the unit's TimeoutStartSec and that under
  the web UI's VERIFY_LOST_SECONDS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools

Plugin config was prepared differently depending on how it arrived:

- JSON POST /plugins/config built a partial body on schema defaults, so
  {"enabled": true} reset every other setting of the plugin. It now merges
  onto the stored section first, as the form path already did.
- Legacy-boolean normalization (#588) ran only at load: GET /plugins/config
  returned the raw boolean, posting it back failed validation, and hot
  reload handed plugins the raw section (a legacy dynamic_duration: true
  came back as a boolean). schema_manager.prepare_plugin_config (normalize,
  then defaults) is now used by PluginManager.load_plugin, both save paths,
  GET, the save notifications and DisplayController's hot-reload callback.
- The JSON save's filter kept only enabled/display_duration/live_priority
  and dropped a submitted skin, skin_options or vegas_* tuning key. There
  is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES,
  used by validation and by the save filter; PluginManager's
  CORE_OWNED_CONFIG_KEYS is its vegas subset.
- Plugin sections posted to /config/main were stored verbatim, including
  values /plugins/config rejects. They now go through the same preparation
  (_prepare_plugin_config_for_save, extracted from save_plugin_config), and
  a failing section rejects the whole save before anything is written.
- dev_server read only top-level defaults and let a schema enabled:false
  win; build_full_config shallow-merged overrides, dropping sibling
  defaults; the harness extracted defaults differently from the device.
  loading.build_config now uses the device's extraction and preparation,
  and dev_server, check_plugin, render_plugin and the harness all use it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt-bridge): brightness changes apply live and touch nothing else

The display service's hot reload applies a saved brightness within a few
seconds, and /config/main no longer resets other display settings on a
brightness-only JSON body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): automatic update hardening

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI

It described web_interface_v2.py and index_v2.html (both gone), client-side
form generation, one POST per field with {key, value}, and 'no nested
objects'. The v3 UI renders plugin forms server-side from the schema
(pages_v3 partial + plugin_config.html macros, nested sections and
x-widgets), posts the whole form once, and save_plugin_config() merges onto
the stored section, validates, splits x-secret fields and notifies the
plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt): brightness saves apply via hot reload and leave other settings alone

The bridge README said brightness is applied on the display's next
restart; the display controller's config hot reload applies it within
seconds. It also now states that the bridge's partial JSON save changes
only brightness (the /config/main merge fix in this change set).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): don't log pip's output from the health check's reinstall

pip can echo a private index URL with embedded credentials;
permission_utils redacts it, the stdlib-only verifier cannot, so it
logs the exit code only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark the plugin_system toggles as unused legacy keys

auto_discover, auto_load_enabled and development_mode are read by
nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE
said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and
the REST reference listed them as live settings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): docs and developer tools group

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): legacy plugin-system toggles no longer count as a General save

auto_discover, auto_load_enabled and development_mode have left the General
form, so a post carrying only one of them is not a general-settings save and
must not treat web_display_autostart and auto_update as unchecked. The
plugin_system block itself is left as on main for the branch that reworks it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): config-save and plugin-config preparation fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(claude): re-check matrix_support.py rules when the library submodule is bumped

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address Codacy findings on the core audit PR

- plugin_manager.prepare_plugin_config: when the fallback legacy-boolean
  pass also fails, log a warning instead of a bare except/pass.
- api_client.js: request() refuses any endpoint that is not a plain path
  under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace,
  control characters) with INVALID_ENDPOINT before calling fetch(), and
  plugin ids are URL-encoded wherever they are put into a URL (also in the
  app-shell batch load).
- test_update_all.js: pins both against the shipped client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): check endpoint control characters without a control-character regex

Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags
the \x00-\x1f range in checkEndpoint's regex. Test the char codes
instead; the endpoints refused are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(auto-update): make the seed script executable on disk, not only in the index

On Linux Repo.publish() commits with -a, which recorded scripts/run.sh
as 100644 upstream because the seed file was never chmod +x. The pull
then brought in the same mode the installer chmod had made locally, so
installer_chmod saw no mode change left to check. The updater was fine:
with the upstream commit at 100755 the --autostash carries the device's
chmod across. Verified under Linux (WSL, git 2.43): the old helper fails
exactly as CI did, the fixed one passes all 63 tests in the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-17 16:37:29 -04:00
committed by GitHub
co-authored by Claude Opus 5
parent 7e5967e160
commit 116abb0daa
101 changed files with 7244 additions and 2904 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ADVANCED_FEATURES.md
+36 -14
View File
@@ -377,9 +377,16 @@ Vegas mode consists of four core components working together to provide smooth 1
5. Compose into continuous stream with separators
**Key Methods:**
- `get_stream_content()` - Returns current stream content as PIL Image
- `advance_stream(pixels)` - Advances stream by N pixels
- `refresh_stream()` - Regenerates stream from current plugins
- `get_next_segment()` - Returns the next buffered `ContentSegment` (or `None`)
- `take_next_group(count=None, offscreen_only=False)` - Hands over the next
slice of the rotation as `(plugin_id, images)` groups
- `get_grouped_content_for_composition()` - Buffered images grouped by plugin
- `mark_plugin_updated(plugin_id)` / `process_updates()` - Refresh one
plugin's segment in place when its data changes
- `refresh()` - Re-read the plugin list and config
- `advance_cycle()` - Clear the active buffer when a scroll cycle completes
(`src/vegas_mode/stream_manager.py`)
#### 3. PluginAdapter
@@ -433,10 +440,14 @@ Vegas mode consists of four core components working together to provide smooth 1
- **Frame Rate Control:** Precise timing to maintain 125 FPS
- **Pre-rendered Content:** Plugins pre-render during update()
**Scroll Speed Calculation:**
**Scroll Speed Calculation:** motion is by elapsed time; `target_fps` paces
the render loop, not the speed.
```python
pixels_per_frame = (scroll_speed / target_fps)
scroll_position += pixels_per_frame * elapsed_time
# frame_based_scrolling: false
scroll_position += scroll_speed * elapsed_time # scroll_speed in px/s
# frame_based_scrolling: true (the default) -- not stepping, just a clamp
applied = clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay
scroll_position += applied * elapsed_time
```
#### Component Interactions
@@ -552,7 +563,8 @@ time when something is active.
### REST API Reference
The API is mounted at `/api/v3` (`web_interface/app.py:199`).
The API is mounted at `/api/v3` (the `api_v3` blueprint, registered in
`web_interface/app.py`). Full details: [REST_API_REFERENCE.md](REST_API_REFERENCE.md#display-control).
#### Start On-Demand Display
@@ -608,20 +620,30 @@ curl http://localhost:5000/api/v3/display/on-demand/status
# Response:
{
"active": true,
"plugin_id": "weather",
"mode": "weather",
"remaining": 25.5,
"pinned": false,
"status": "active"
"status": "success",
"data": {
"state": {
"active": true,
"plugin_id": "weather",
"mode": "weather",
"duration": 30,
"pinned": false,
"status": "running",
"last_updated": 1234567890.1
},
"service": {"active": true, "returncode": 0, "stdout": "active", "stderr": ""}
}
}
```
When nothing is running on demand, `data.state` is
`{"active": false, "status": "idle", "last_updated": null}`.
> There is no public Python on-demand API. The display controller's
> on-demand machinery is internal — drive it through the REST endpoints
> above (or the web UI buttons). The API handlers
> (`start_on_demand_display()` / `stop_on_demand_display()` in
> `web_interface/blueprints/api_v3.py`) write a request into the cache
> `web_interface/blueprints/api_v3/display.py`) write a request into the cache
> manager under the `display_on_demand_request` key, which
> `DisplayController._poll_on_demand_requests()`
> (`src/display_controller.py`) picks up. A separate
docs/CONFIG_DEBUGGING.md
+14 -5
View File
@@ -250,14 +250,21 @@ WARNING - Plugin ID 'Football-Scoreboard' may conflict with 'football-scoreboard
## Checking Configuration via API
The API blueprint mounts at `/api/v3` (`web_interface/app.py:144`).
The API blueprint (`web_interface/blueprints/api_v3/`) is registered at
`/api/v3` in `web_interface/app.py`.
```bash
# Get full main config (includes all plugin sections)
# Get full main config (includes all plugin sections; credential-named
# fields are blanked in the response)
curl http://localhost:5000/api/v3/config/main
# Save updated main config
# Change some settings: only the keys you send are changed
curl -X POST http://localhost:5000/api/v3/config/main \
-H "Content-Type: application/json" \
-d '{"timezone": "America/Chicago", "brightness": 80}'
# Replace config.json wholesale (advanced)
curl -X POST http://localhost:5000/api/v3/config/raw/main \
-H "Content-Type: application/json" \
-d @new-config.json
@@ -269,8 +276,10 @@ curl "http://localhost:5000/api/v3/plugins/config?plugin_id=football-scoreboard"
```
> There is no dedicated `/config/plugin/<id>` or `/config/validate`
> endpoint — config validation runs server-side automatically when you
> POST to `/config/main` or `/plugins/config`. See
> endpoint. `POST /plugins/config` validates against the plugin's schema
> and rejects an invalid config with `400`; `POST /config/main` checks the
> individual fields it knows (display hardware values, durations, Vegas
> and sync settings). See
> [REST_API_REFERENCE.md](REST_API_REFERENCE.md) for the full list.
## Backup and Recovery
docs/CONFIG_REFERENCE.md
+26 -23
View File
@@ -17,7 +17,7 @@ tooling against it.
|---|---|---|---|
| `web_display_autostart` | bool, `true` | Whether the web interface service starts with the system | `scripts/utils/start_web_conditionally.py` |
| `timezone` | string, `"America/New_York"` | IANA timezone for schedules and displays | `ConfigManager.get_timezone()` |
| `target_fps` | int, `100` | Frame-rate ceiling for plugin rendering | `src/plugin_system/base_plugin.py`, `src/common/sports_scroll.py` |
| `target_fps` | int, `100` | Legacy "Scroll Frame Rate". Core scrolling no longer reads it: scroll frames are presented at `display.hardware.limit_refresh_rate_hz` divided by each scroll's frame hold, and speed comes from each plugin's scroll settings. Still exposed to plugins via `BasePlugin.global_config` | `src/plugin_system/base_plugin.py` |
| `location` | object | `city` / `state` / `country`. Supplies the **default** for a plugin's own `location_city` / `location_state` / `location_country` setting, so weather, radar and friends follow this device without being configured twice. A value saved on the plugin itself still overrides it. | `SchemaManager.apply_device_location()`, then plugins via merged config |
## `schedule` — display on/off hours
@@ -47,38 +47,43 @@ saved via `POST /api/v3/config/dim-schedule`). The display returns to
## `display.hardware` — matrix panel hardware
All keys map to the corresponding `rpi-rgb-led-matrix` options and are read
in `DisplayManager` (`src/display_manager.py`, ~lines 270–295).
in `DisplayManager._setup_matrix` (`src/display_manager.py`). Defaults are the
`config/config.template.json` values: `ConfigManager` adds any key missing from
`config.json` from the template on load, so `DisplayManager`'s own fallbacks
don't apply on a normal install.
The ranges are what the pinned rgbmatrix library and its Python binding accept
(`src/matrix_support.py`). The config API refuses anything else; a value
hand-edited into `config.json` makes the display log the setting and run in
fallback mode instead of starting the matrix.
| Key | Type / default |
|---|---|
| `rows` / `cols` | int, `32` / `64` — rows: even, at least 8, no upper limit here (the current rgbmatrix library rejects more than 64); cols: at least 16, no upper limit |
| `chain_length` | int, `2` — at least 1, no upper limit |
| `parallel` | int, `1` — 1–3 |
| `rows` / `cols` | int, `32` / `64` — rows: even, 8–64; cols: at least 16 |
| `chain_length` | int, `2` — 1–255 (the Python binding stores it in one byte) |
| `parallel` | int, `1` — 1–3, and no more than `hardware_mapping` has outputs (`regular`, `classic`: 3; the others: 1) |
| `brightness` | int, `90` — 1–100 |
| `hardware_mapping` | string, `"adafruit-hat"` (code default `"adafruit-hat-pwm"`) |
| `hardware_mapping` | string, `"adafruit-hat"` — `"adafruit-hat-pwm"`, `"adafruit-hat"`, `"regular"`, `"regular-pi1"`, `"classic"` or `"classic-pi1"` (case-insensitive; `compute-module` isn't in the installed build). A Pi 5 doesn't support `"classic-pi1"` |
| `scan_mode` | int, `0` — `0` progressive, `1` interlaced |
| `pwm_bits` | int, `9` (code default 10) — 1–11 |
| `pwm_bits` | int, `9` — 1–11 |
| `pwm_dither_bits` | int, `1` — 0–2 |
| `pwm_lsb_nanoseconds` | int, `130` (code default 150) — 50–3000 |
| `pwm_lsb_nanoseconds` | int, `130` — 50–3000 |
| `disable_hardware_pulsing` | bool, `false` — `true` times brightness pulses in software (less exact); hardware pulsing needs the OE line on GPIO 18 and the Pi's onboard sound driver off |
| `inverse_colors` | bool, `false` |
| `show_refresh_rate` | bool, `false` — prints the refresh rate to stdout; draws nothing on the panel |
| `led_rgb_sequence` | string, `"RGB"` — `"RGB"`, `"RBG"`, `"GRB"`, `"GBR"`, `"BRG"` or `"BGR"` |
| `limit_refresh_rate_hz` | int, `100` (code default 90) — `0` = no cap; scroll timing assumes 100 Hz when `0` |
| `pixel_mapper_config` | string, `""` — e.g. `"U-mapper"` / `"Rotate:90"` |
| `orientation` | string, `"normal"` — `"180"` rotates the rendered image 180° for panels physically mounted upside down (e.g. to move the Pi/wiring to a more convenient side); composed onto `pixel_mapper_config` as a trailing `Rotate:180` mapper, so it stays independent of any custom `pixel_mapper_config` value |
| `limit_refresh_rate_hz` | int, `100` — `0` = no cap; scroll timing assumes 100 Hz when `0` |
| `pixel_mapper_config` | string, `""` — e.g. `"U-mapper"` / `"Rotate:90"`; mappers that rotate or fold the chain change the display size plugins and the web preview see |
| `orientation` | string, `"normal"` — `"180"` rotates the rendered image 180° for panels physically mounted upside down (e.g. to move the Pi/wiring to a more convenient side); `"90"` / `"270"` for a panel on its side, swapping width and height; composed onto `pixel_mapper_config` as a trailing `Rotate:<degrees>` mapper, so it stays independent of any custom `pixel_mapper_config` value |
| `row_address_type` | int, `0` — non-standard panel row addressing: `1` AB, `2` direct row select, `3` ABC, `4` ABC shift + DE direct, `5` SM5368 / B707 row shift register (e.g. Waveshare 96x48 V2, with `led_rgb_sequence` `"BGR"`). On a Pi 5 the library supports only `0` and `2`, and LEDMatrix enforces that (`src/pi5_matrix_support.py`) |
| `multiplexing` | int, `0` — 0–22, pixel wiring scheme for outdoor/specialty panels (names listed in the README) |
| `panel_type` | string, `""` — set to `"FM6126A"` or `"FM6127"` for panels needing init; FM6124 / FM6124D / FM6124DJ panels need none, so leave it `""` |
Where "code default" differs from the template value, the code default only
applies if the key is missing entirely from your config.
## `display.runtime`
| Key | Type / default | Meaning |
|---|---|---|
| `gpio_slowdown` | int, `3` | GPIO timing slowdown for faster Pis (0–10). Panels on `row_address_type` `5` (SM5368 row drivers) can need 6–8 on a Pi 4 — lower values make rows jump |
| `gpio_slowdown` | int, `3` | GPIO timing slowdown for faster Pis (0–10). On a Pi 5 in PIO mode start at `1` (`0` acts as `1`) and raise it if the image flickers or shows garbage. Panels on `row_address_type` `5` (SM5368 row drivers) can need 6–8 on a Pi 4 — lower values make rows jump |
| `rp1_rio` | int, `0` | Pi 5 only: `0` = PIO (less CPU), `1` = RIO (higher refresh; `gpio_slowdown` effect inverted). Applied only if the installed matrix library supports it |
## `display.double_sided`
@@ -134,8 +139,8 @@ Read by `src/vegas_mode/config.py` (`VegasScrollConfig.from_config`). See
| `dynamic_duration_enabled` | bool, `true` |
| `min_cycle_duration` | int, `60` |
| `max_cycle_duration` | int, `240` |
| `frame_based_scrolling` | bool, `true` — frame-count-based scroll stepping |
| `scroll_delay` | float, `0.02` — seconds between scroll updates (~50 FPS) |
| `frame_based_scrolling` | bool, `true` — does not step or set a frame rate; motion is by elapsed time either way. When `true`, `scroll_speed` passes through a clamp of 0.1–5 px per `scroll_delay` (see next row) |
| `scroll_delay` | float, `0.02` — not a frame period. Only used with `frame_based_scrolling`: the applied speed is `clamp(scroll_speed × scroll_delay, 0.1, 5) / scroll_delay` px/s, so at `0.02` speeds under 5 px/s run at 5, and at `0.001` nothing runs slower than 100 px/s |
| `live_in_ticker` | bool, `false` — keep scrolling during live games instead of handing the display to a full-screen scoreboard |
| `live_weight` | int, `3` (1–10) — slots per cycle for a plugin with live content |
| `favorite_live_weight` | int, `5` (1–10) — slots per cycle when a plugin reports a favorite team is live |
@@ -152,14 +157,12 @@ Read by `src/common/sync_manager.py` and `src/display_controller.py`.
## `plugin_system`
Read by the plugin loader/manager (`src/plugin_system/`).
| Key | Type / default | Meaning |
|---|---|---|
| `plugins_directory` | string, `"plugin-repos"` | Where the Plugin Store installs plugins |
| `auto_discover` | bool, `true` | Scan the plugins directory at startup |
| `auto_load_enabled` | bool, `true` | Load discovered plugins automatically |
| `development_mode` | bool, `false` | Development conveniences in the web UI (editable under General settings) |
| `plugins_directory` | string, `"plugin-repos"` | Where the Plugin Store installs plugins and the only directory the plugin loader scans. Read by `PluginManager` and `PluginStoreManager` (`src/plugin_system/`); editable under General settings |
| `auto_discover` | bool, `true` | **Unused.** Legacy key, read by nothing. Plugins are always discovered, and every plugin with `enabled: true` is loaded. Not shown in the web UI; may be left in or removed from config.json |
| `auto_load_enabled` | bool, `true` | **Unused.** Legacy key, read by nothing (see `auto_discover`). To keep a plugin installed but dormant, set its own `enabled` to `false` |
| `development_mode` | bool, `false` | **Unused.** Legacy key, read by nothing |
## Plugin config blocks
docs/FONT_MANAGER.md
+38 -33
View File
@@ -12,10 +12,28 @@
The enhanced FontManager provides comprehensive font management for the LEDMatrix application with support for:
- Manager font registration and detection
- Plugin font management
- Manual font overrides via web interface
- Programmatic per-element font overrides
- Performance monitoring and caching
- Dynamic font discovery
## Getting the FontManager
There is one shared FontManager per display process. The display controller
creates it and hands it to the `PluginManager`, so a plugin reaches it
through its `plugin_manager`:
```python
class MyPlugin(BasePlugin):
def __init__(self, plugin_id, config, display_manager, cache_manager, plugin_manager):
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
self.font_manager = self._get_font_manager()
```
`BasePlugin._get_font_manager()` returns `plugin_manager.font_manager`, or a
standalone FontManager when none is available (test harnesses, mocks).
`DisplayManager` has **no** `font_manager` attribute —
`display_manager.font_manager` raises `AttributeError`.
## Architecture
### Manager-Centric Design
@@ -40,8 +58,9 @@ Manager requests font → Check manual overrides → Apply manager choice → Ca
from src.font_manager import FontManager
class MyManager:
def __init__(self, config, display_manager, cache_manager):
self.font_manager = display_manager.font_manager # Access shared FontManager
def __init__(self, config, display_manager, cache_manager, plugin_manager):
self.display_manager = display_manager
self.font_manager = plugin_manager.font_manager # Shared FontManager
self.manager_id = "my_manager"
def display(self):
@@ -80,8 +99,9 @@ class MyManager:
```python
class AdvancedManager:
def __init__(self, config, display_manager, cache_manager):
self.font_manager = display_manager.font_manager
def __init__(self, config, display_manager, cache_manager, plugin_manager):
self.display_manager = display_manager
self.font_manager = plugin_manager.font_manager
self.manager_id = "advanced_manager"
# Define your font specifications
@@ -152,19 +172,13 @@ font = self.font_manager.resolve_font(
> URIs documented below are resolved relative to the plugin's
> install directory.
>
> The **Fonts** tab in the web UI that lists detected
> manager-registered fonts is still a **placeholder
> implementation** — fonts that managers register through
> `register_manager_font()` do not yet appear there. The
> programmatic per-element override workflow described in
> [Manual Font Overrides](#manual-font-overrides) below
> (`set_override()` / `remove_override()` / the
> `config/font_overrides.json` store) **does** work today and is
> the supported way to override a font for an element until the
> Fonts tab is wired up. If you can't wait and need a workaround
> right now, you can also just load the font directly with PIL
> (or `freetype-py` for BDF) inside your plugin's `manager.py`
> and skip the override system entirely.
> The web UI's **Fonts** tab lists, uploads, previews and deletes the
> font files in `assets/fonts/`. It does not show fonts registered
> through `register_manager_font()` and has no override editor (the
> override panels and `/api/v3/fonts/overrides` endpoints were removed).
> The programmatic override workflow in
> [Manual Font Overrides](#manual-font-overrides) below still works.
> Let users pick fonts through your plugin's own config schema.
### Plugin Font Registration
@@ -200,10 +214,10 @@ In your plugin's `manifest.json`:
### Using Plugin Fonts
```python
class PluginManager:
def __init__(self, config, display_manager, cache_manager, plugin_id):
self.font_manager = display_manager.font_manager
self.plugin_id = plugin_id
class MyPlugin(BasePlugin):
def __init__(self, plugin_id, config, display_manager, cache_manager, plugin_manager):
super().__init__(plugin_id, config, display_manager, cache_manager, plugin_manager)
self.font_manager = self._get_font_manager()
def display(self):
# Use plugin font (automatically namespaced)
@@ -219,17 +233,8 @@ class PluginManager:
## Manual Font Overrides
Users can override any font through the web interface:
1. Navigate to **Fonts** tab
2. View **Detected Manager Fonts** to see what's currently in use
3. In **Element Overrides** section:
- Select the element (e.g., "nfl.live.score")
- Choose a different font family
- Choose a different size
- Click **Add Override**
Overrides are stored in `config/font_overrides.json` and persist across restarts.
Overrides are set in code (there is no web UI or REST endpoint for them).
They are stored in `config/font_overrides.json` and persist across restarts.
### Programmatic Overrides
docs/MIGRATION_GUIDE.md
+5 -2
View File
@@ -59,9 +59,12 @@ sudo ./scripts/install/install_service.sh
After updating your scripts, verify they still work:
```bash
# Test installation scripts (if needed)
# Check the installation scripts are at their new paths
ls scripts/install/*.sh
sudo ./scripts/install/install_service.sh --help
./scripts/install/install_service.sh --help # prints usage only
# Note: running install_service.sh for real (with sudo, no --help)
# reinstalls, enables and restarts ledmatrix.service, ledmatrix-web.service
# and the update-verify units.
# Test permission scripts
ls scripts/fix_perms/*.sh
docs/MULTI_ROOT_WORKSPACE_SETUP.md
+81 -96
View File
@@ -1,169 +1,154 @@
# Multi-Root Workspace Setup Guide
This document explains how the LEDMatrix project uses a multi-root workspace to manage plugins as separate Git repositories.
This document explains how to work on LEDMatrix and the official plugins side
by side, with one editor workspace and the plugins loaded straight from your
plugin checkout.
## Overview
The LEDMatrix project has been migrated from a git submodule implementation to a **multi-root workspace** implementation for managing plugins. This allows:
Official plugins live in a single repository,
[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins), with one
directory per plugin under `plugins/`. There are no separate per-plugin
repositories. For development you clone that monorepo **next to** LEDMatrix
and symlink its plugin directories into LEDMatrix's `plugin-repos/`, which is
where the plugin loader looks by default.
- ✅ Plugins to exist as independent Git repositories
- ✅ Updates to plugins without modifying the LEDMatrix project
- ✅ Easy development workflow with all repos in one workspace
- ✅ Plugin system discovers plugins via symlinks in `plugin-repos/`
- ✅ Plugin code stays in the monorepo checkout, with its own git history
- ✅ LEDMatrix discovers the plugins through symlinks in `plugin-repos/`
- ✅ `LEDMatrix.code-workspace` opens both repositories in VS Code/Cursor
## Directory Structure
```text
/home/chuck/Github/
├── LEDMatrix/ # Main project
│ ├── plugin-repos/ # Symlinks to actual repos (managed automatically)
│ │ ├── ledmatrix-clock-simple -> ../../ledmatrix-clock-simple
│ │ ├── ledmatrix-weather -> ../../ledmatrix-weather
~/Github/
├── LEDMatrix/ # Main project
│ ├── plugin-repos/ # Plugin directory the loader scans
│ │ ├── starlark-apps/ # Bundled with LEDMatrix (tracked in git)
│ │ ├── web-ui-info/ # Bundled with LEDMatrix (tracked in git)
│ │ ├── clock-simple -> ../../ledmatrix-plugins/plugins/clock-simple
│ │ ├── ledmatrix-weather -> ../../ledmatrix-plugins/plugins/ledmatrix-weather
│ │ └── ...
│ ├── LEDMatrix.code-workspace # Multi-root workspace configuration
│ ├── LEDMatrix.code-workspace # Opens LEDMatrix and ../ledmatrix-plugins
│ └── ...
├── ledmatrix-clock-simple/ # Plugin repository (actual git repo)
├── ledmatrix-weather/ # Plugin repository (actual git repo)
├── ledmatrix-football-scoreboard/ # Plugin repository (actual git repo)
└── ... # Other plugin repos
└── ledmatrix-plugins/ # Plugin monorepo (git repo)
├── plugins/
│ ├── clock-simple/
│ ├── ledmatrix-weather/
│ └── ...
├── plugins.json # Store registry
└── update_registry.py
```
## How It Works
### 1. Plugin Repositories
### 1. The plugin monorepo
All plugin repositories are cloned to `/home/chuck/Github/` (parent directory of LEDMatrix) as regular Git repositories:
Clone ledmatrix-plugins into the same parent directory as LEDMatrix (the
scripts below look for `../ledmatrix-plugins` relative to the LEDMatrix
root):
- `ledmatrix-clock-simple/`
- `ledmatrix-weather/`
- `ledmatrix-football-scoreboard/`
- etc.
```bash
cd ~/Github
git clone https://github.com/ChuckBuilds/ledmatrix-plugins.git
```
### 2. Symlinks in plugin-repos/
The `LEDMatrix/plugin-repos/` directory contains symlinks pointing to the actual repositories in the parent directory. This allows the plugin system to discover plugins without modifying the project structure.
`scripts/setup_plugin_repos.py` creates one symlink per plugin in
`LEDMatrix/plugin-repos/`, named after the plugin's manifest `id` and pointing
at `../ledmatrix-plugins/plugins/<dir>`.
### 3. Multi-Root Workspace
### 3. Multi-root workspace
The `LEDMatrix.code-workspace` file configures VS Code/Cursor to open all plugin repositories as separate workspace roots, allowing easy development across all repos.
`LEDMatrix.code-workspace` has two roots: LEDMatrix itself and
`../ledmatrix-plugins`.
## Setup Scripts
### Initial Setup
If you already have plugin repositories cloned, use the setup script:
```bash
cd /home/chuck/Github/LEDMatrix
cd ~/Github/LEDMatrix
python3 scripts/setup_plugin_repos.py
```
This script:
- Reads the workspace configuration
- Creates symlinks in `plugin-repos/` pointing to actual repos
- Verifies all links are created correctly
- Reads each `manifest.json` under `../ledmatrix-plugins/plugins/`
- Creates `plugin-repos/<id>` symlinks (relative) to those directories
- Leaves correct links alone, replaces links that point elsewhere, and skips
(does not overwrite) a real directory of the same name — for example a
plugin you installed from the Plugin Store. Remove that directory first if
you want the linked copy.
### Updating Plugins
To update all plugin repositories:
```bash
cd /home/chuck/Github/LEDMatrix
cd ~/Github/LEDMatrix
python3 scripts/update_plugin_repos.py
```
This script:
- Finds all plugins in the workspace
- Runs `git pull` on each repository
- Reports which plugins were updated
This runs `git pull` in `../ledmatrix-plugins` and prints the result. The
symlinks pick up the new code; restart the display to load it.
## Configuration
The plugin system is configured in `config/config.json`:
The loader reads plugins from `plugin_system.plugins_directory` in
`config/config.json`. The default is already right for this setup:
```json
{
"plugin_system": {
"plugins_directory": "plugin-repos",
"auto_discover": true,
"auto_load_enabled": true
"plugins_directory": "plugin-repos"
}
}
```
The `plugins_directory` points to `plugin-repos/`, which contains symlinks to the actual repositories.
## Workflow
### Daily Development
1. **Open Workspace**: Open `LEDMatrix.code-workspace` in VS Code/Cursor
2. **All Repos Available**: All plugin repos appear as separate folders in the workspace
3. **Edit Plugins**: Edit plugin code directly in their repositories
4. **Update Plugins**: Run `update_plugin_repos.py` to pull latest changes
2. **Edit Plugins**: Edit code under `ledmatrix-plugins/plugins/<plugin>/`
3. **Test**: `python3 run.py -e` (emulator) or
`python3 scripts/check_plugin.py --plugin <id>` from LEDMatrix
4. **Ship**: Bump `version` in the plugin's `manifest.json`, run
`python update_registry.py` in ledmatrix-plugins, commit there
### Adding New Plugins
1. **Clone Repository**: Clone the new plugin repo to `/home/chuck/Github/`
2. **Add to Workspace**: Add the plugin folder to `LEDMatrix.code-workspace`
3. **Create Symlink**: Run `setup_plugin_repos.py` to create the symlink
### Updating Individual Plugins
Since plugins are regular Git repositories, you can update them individually:
```bash
cd /home/chuck/Github/ledmatrix-weather
git pull origin master
```
Or update all at once:
```bash
cd /home/chuck/Github/LEDMatrix
python3 scripts/update_plugin_repos.py
```
## Benefits
1. **No Submodule Hassle**: No need to update `.gitmodules` or run `git submodule update`
2. **Independent Updates**: Update plugins independently without touching LEDMatrix
3. **Clean Separation**: Each plugin is a separate repository with its own history
4. **Easy Development**: Multi-root workspace makes it easy to work across repos
5. **Automatic Discovery**: Plugin system automatically discovers plugins via symlinks
1. Create `plugins/<your-plugin-id>/` in the monorepo checkout
2. Run `python3 scripts/setup_plugin_repos.py` in LEDMatrix to link it
## Troubleshooting
### Symlinks Not Working
If plugins aren't being discovered:
### Plugins not discovered
```bash
cd /home/chuck/Github/LEDMatrix
python3 scripts/setup_plugin_repos.py
cd ~/Github/LEDMatrix
ls -la plugin-repos/ # links present and not broken?
python3 scripts/setup_plugin_repos.py # recreate them
```
This will recreate all symlinks.
Also check that `plugin_system.plugins_directory` is `plugin-repos`.
### Missing Plugins
### "Monorepo plugins directory not found"
If a plugin is in the workspace but not found:
`setup_plugin_repos.py` expects the monorepo at `../ledmatrix-plugins`. Clone
it there (or symlink it there).
1. Check if the repo exists in `/home/chuck/Github/`
2. Check if the symlink exists in `plugin-repos/`
3. Run `setup_plugin_repos.py` to recreate symlinks
### Plugin updates not showing
### Plugin Updates Not Showing
If changes to plugins aren't appearing:
1. Verify the symlink points to the correct directory: `ls -la plugin-repos/ledmatrix-weather`
2. Check that you're editing in the actual repo, not a copy
3. Restart the LEDMatrix service if running
1. Verify the link target: `ls -la plugin-repos/<id>`
2. Check that you're editing the monorepo checkout, not a store-installed copy
3. Restart the LEDMatrix service (or `run.py`)
## Notes
- The `plugin-repos/` directory is tracked in git, but only contains symlinks
- Actual plugin code lives in `/home/chuck/Github/ledmatrix-*/`
- Each plugin repo can be updated independently via `git pull`
- The LEDMatrix project doesn't need to be updated when plugins change
- `plugin-repos/` is tracked in git only for the bundled plugins
(`starlark-apps`, `web-ui-info`). The symlinks you create are untracked
files; don't commit them.
- For linking a single plugin into `plugins/` instead (without a sibling
checkout), see `scripts/dev/dev_plugin_setup.sh` in the
[Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md).
- When changing a plugin in the monorepo, bump its manifest `version` and run
`python update_registry.py`, or users won't receive the update.
docs/PLUGIN_API_REFERENCE.md
+46 -7
View File
@@ -514,21 +514,59 @@ self.display_manager.draw_text_with_icons(
For plugins that implement scrolling content, use these methods to coordinate with the display system.
#### `set_scrolling_state(is_scrolling: bool) -> None`
#### `set_scrolling_state(is_scrolling: bool, frame_hold: int = 1) -> None`
Mark the display as scrolling or not scrolling. Call when scrolling starts/stops.
Mark the display as scrolling or not scrolling, and set this scroll's frame
pacing. Call it when a scroll starts (calling it on every scroll frame is fine)
and with `False` when it stops.
**Parameters**:
- `is_scrolling` (bool): True if currently scrolling, False otherwise
- `frame_hold` (int, default 1): how many panel refreshes each pushed frame is
held for (clamped to 1-255; ignored when `is_scrolling` is False, which
resets it to 1). Pass the `frame_hold` of the settings
`src.common.scroll_config.configure()` returned. Added in core 3.4.0.
**Why `frame_hold` matters**: `scroll_config.configure()` snaps the speed to
one the panel can show in whole pixels and sets the `ScrollHelper` to advance a
fixed number of pixels on every presented frame -- no clock is consulted. The
panel presents frames at its refresh rate divided by the hold, so the hold is
part of the speed. Omit it and a 50 px/s scroll (1px every 2nd refresh on a
100 Hz panel) runs at 100 px/s. The hold is not applied by `configure()`
because it must not outlive the scroll: plugins share one display manager.
**Example**:
```python
from src.common import scroll_config
from src.common.scroll_helper import ScrollHelper
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.scroll_helper = ScrollHelper(
self.display_manager.width, self.display_manager.height, self.logger)
# ...later, hand it content with self.scroll_helper.set_scrolling_image(img)
self.scroll_settings = scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
display_manager=self.display_manager,
plugin_logger=self.logger,
)
def display(self, force_clear=False):
self.display_manager.set_scrolling_state(True)
# Scroll content...
self.display_manager.set_scrolling_state(False)
self.display_manager.set_scrolling_state(
True, frame_hold=self.scroll_settings.frame_hold)
self.scroll_helper.update_scroll_position()
self.display_manager.image = self.scroll_helper.get_visible_portion()
self.display_manager.update_display()
if self.scroll_helper.is_scroll_complete():
self.display_manager.set_scrolling_state(False)
```
Don't pace the loop with `time.sleep()`: `update_display()` blocks on the
panel's vsync, which is what paces a scroll. See `docs/SCROLL_PERFORMANCE.md`
for choosing a speed.
#### `is_currently_scrolling() -> bool`
Check if the display is currently in a scrolling state.
@@ -998,9 +1036,10 @@ if "weather" in enabled_plugins:
self.display_manager.update_display()
```
3. **Handle scrolling state**: If your plugin scrolls, use scrolling state methods
3. **Handle scrolling state**: If your plugin scrolls, use scrolling state methods,
passing the frame hold `scroll_config.configure()` returned
```python
self.display_manager.set_scrolling_state(True)
self.display_manager.set_scrolling_state(True, frame_hold=settings.frame_hold)
# Scroll content...
self.display_manager.set_scrolling_state(False)
```
docs/PLUGIN_ARCHITECTURE_SPEC.md
+2 -1
View File
@@ -8,7 +8,8 @@
> - Code paths reference `web_interface_v2.py`; the current web UI is
> `web_interface/app.py` with v3 Blueprint-based templates.
> - The example Flask routes use `/api/plugins/*`; the real API
> blueprint is mounted at `/api/v3` (`web_interface/app.py:199`).
> blueprint (`web_interface/blueprints/api_v3/`) is mounted at `/api/v3`
> in `web_interface/app.py`.
> - The default plugin location is `plugin-repos/` (configurable via
> `plugin_system.plugins_directory`), not `./plugins/`.
> - Example imports use `src/plugin_system/base_classes/*_plugin.py`;
docs/PLUGIN_CONFIGURATION_GUIDE.md
+4 -6
View File
@@ -67,9 +67,7 @@ The main configuration file (`config/config.json`) now contains only essential s
"time_format": "%I:%M %p"
},
"plugin_system": {
"plugins_directory": "plugin-repos",
"auto_discover": true,
"auto_load_enabled": true
"plugins_directory": "plugin-repos"
}
}
```
@@ -93,9 +91,9 @@ The main configuration file (`config/config.json`) now contains only essential s
#### 4. Plugin System
- **plugin_system**: Plugin system configuration
- **plugins_directory**: Directory where plugins are stored
- **auto_discover**: Automatically discover plugins
- **auto_load_enabled**: Automatically load enabled plugins
- **plugins_directory**: Directory where plugins are stored (the only one the loader scans)
- `auto_discover`, `auto_load_enabled`, `development_mode` may still appear in
older configs; nothing reads them (see [CONFIG_REFERENCE.md](CONFIG_REFERENCE.md#plugin_system))
## Plugin Configuration
docs/PLUGIN_CONFIGURATION_TABS.md
+2 -1
View File
@@ -6,7 +6,8 @@
> in the "Implementation Details" section below still reference the
> pre-v3 file layout (`web_interface_v2.py`, `templates/index_v2.html`).
> The current implementation lives in `web_interface/app.py`,
> `web_interface/blueprints/api_v3.py`, and `web_interface/templates/v3/`.
> `web_interface/blueprints/api_v3/` (plugin config handlers in
> `plugins.py`), and `web_interface/templates/v3/`.
> The user-facing description (Overview, Features, Form Generation
> Process) is still accurate.
docs/PLUGIN_CONFIG_ARCHITECTURE.md
+134 -382
View File
@@ -11,427 +11,179 @@
### Component Overview
```
┌─────────────────────────────────────────────────────────────────┐
│ Web Browser │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Tab Navigation Bar │ │
│ │ [Overview] [General] ... [Plugins] [Plugin X] [Plugin Y]│ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────┐ ┌──────────────────────────────────┐ │
│ │ Plugins Tab │ │ Plugin X Configuration Tab │ │
│ │ │ │ │ │
│ │ • Install │ │ Form Generated from Schema: │ │
│ │ • Update │ │ • Boolean → Toggle │ │
│ │ • Uninstall │ │ • Number → Number Input │ │
│ │ • Enable │ │ • String → Text Input │ │
│ │ • [Configure]──────→ • Array → Comma Input │ │
│ │ │ │ • Enum → Dropdown │ │
│ └─────────────────┘ │ │ │
│ │ [Save] [Back] [Reset] │ │
│ └──────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ Web browser (templates/v3/base.html, Alpine.js + HTMX) │
│ │
│ Second nav row: one tab per installed plugin │
│ Clicking a tab: GET /v3/partials/plugin-config/<plugin_id> │
│ → server-rendered form swapped into the tab │
│ │
│ Save: hx-post="/api/v3/plugins/config?plugin_id=<id>" (form data) │
└──────────────────────────────────────────────────────────────────┘
│
│ HTTP API
▼
┌─────────────────────────────────────────────────────────────────┐
│ Flask Backend │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ /api/v3/plugins/installed │ │
│ │ • Discover plugins in plugins/ directory │ │
│ │ • Load manifest.json for each plugin │ │
│ │ • Load config_schema.json if exists │ │
│ │ • Load current config from config.json │ │
│ │ • Return combined data to frontend │ │
│ └───────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ /api/v3/plugins/config │ │
│ │ • Receive key-value pair │ │
│ │ • Update config.json │ │
│ │ • Return success/error │ │
│ └───────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ Flask (web_interface/app.py) │
│ │
│ pages_v3 blueprint (blueprints/pages_v3.py) │
│ _load_plugin_config_partial(plugin_id) │
│ • SchemaManager.load_schema() → config_schema.json │
│ • config.json section for the plugin │
│ • masks x-secret fields │
│ • renders partials/plugin_config.html (render_field macros) │
│ │
│ api_v3 blueprint (blueprints/api_v3/plugins.py) │
│ save_plugin_config() POST /api/v3/plugins/config │
│ get_plugin_config() GET /api/v3/plugins/config │
│ get_plugin_schema() GET /api/v3/plugins/schema │
│ reset_plugin_config() POST /api/v3/plugins/config/reset │
└──────────────────────────────────────────────────────────────────┘
│
│ File System
▼
┌─────────────────────────────────────────────────────────────────┐
│ File System │
│ │
│ plugins/ │
│ ├── hello-world/ │
│ │ ├── manifest.json ───┐ │
│ │ ├── config_schema.json ─┼─→ Defines UI structure │
│ │ ├── manager.py │ │
│ │ └── requirements.txt │ │
│ └── clock-simple/ │ │
│ ├── manifest.json │ │
│ └── config_schema.json ──┘ │
│ │
│ config/ │
│ └── config.json ────────────→ Stores configuration values │
│ { │
│ "hello-world": { │
│ "enabled": true, │
│ "message": "Hello!", │
│ ... │
│ } │
│ } │
└─────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────┐
│ Files │
│ plugin-repos/<id>/config_schema.json JSON Schema (Draft-7) │
│ config/config.json { "<id>": { ... } } │
│ config/config_secrets.json { "<id>": { secrets } } │
└──────────────────────────────────────────────────────────────────┘
```
The plugins directory is `plugin_system.plugins_directory` in
`config/config.json` (default `plugin-repos/`). Plugin configuration lives in
`config/config.json`, not in the plugin directory, so it survives reinstalls.
## Data Flow
### 1. Page Load Sequence
### 1. Rendering a plugin's tab
```
User Opens Web Interface
│
▼
DOMContentLoaded Event
│
▼
refreshPlugins()
│
▼
GET /api/v3/plugins/installed
│
├─→ For each plugin directory:
│ ├─→ Read manifest.json
│ ├─→ Read config_schema.json (if exists)
│ └─→ Read config from config.json
│
▼
Return JSON Array:
[{
id: "hello-world",
name: "Hello World",
config: { enabled: true, message: "Hello!" },
config_schema_data: {
properties: {
enabled: { type: "boolean", ... },
message: { type: "string", ... }
}
}
}, ...]
│
▼
generatePluginTabs(plugins)
│
├─→ For each plugin:
│ ├─→ Create tab button
│ ├─→ Create tab content div
│ └─→ generatePluginConfigForm(plugin)
│ │
│ ├─→ Read schema properties
│ ├─→ Get current config values
│ └─→ Generate HTML form inputs
│
▼
Tabs Rendered in UI
User opens the plugin's tab
│
▼
GET /v3/partials/plugin-config/<plugin_id> (pages_v3)
│
├─→ Load schema (SchemaManager, no cache)
├─→ Load config.json[<plugin_id>]
├─→ Mask "x-secret" values (fails closed if the schema is unusable)
└─→ render partials/plugin_config.html
│
└─→ render_field() per property, recursively:
boolean → toggle, number/integer → input or slider,
string → input / textarea / select (enum),
array → list or table widget,
object → collapsible nested section,
"x-widget" → a registered widget
(static/v3/js/widgets/, or one the plugin ships)
```
### 2. Configuration Save Sequence
Nested objects are supported: a nested field is posted with a dotted name
(e.g. `transition.type`).
### 2. Saving
```
User Modifies Form
│
▼
User Clicks "Save"
│
▼
savePluginConfiguration(pluginId)
│
├─→ Get form data
├─→ For each field:
│ ├─→ Get schema type
│ ├─→ Convert value to correct type
│ │ • boolean: checkbox.checked
│ │ • integer: parseInt()
│ │ • number: parseFloat()
│ │ • array: split(',')
│ │ • string: as-is
│ │
│ └─→ POST /api/v3/plugins/config
│ {
│ plugin_id: "hello-world",
│ key: "message",
│ value: "Hello, World!"
│ }
│
▼
Backend Updates config.json
│
▼
Return Success
│
▼
Show Notification
│
▼
Refresh Plugins
User clicks Save
│
▼
validatePluginConfigForm() (client-side checks)
│
▼
POST /api/v3/plugins/config?plugin_id=<id> (form data, all fields of the form)
│
▼
save_plugin_config() (api_v3/plugins.py)
├─→ Start from the stored config.json[<id>]
├─→ Apply form fields: dotted names → nested keys, "[]" checkbox
│ groups → lists, values coerced to the schema's types
├─→ Merge schema defaults for keys that are still missing
├─→ Validate against the schema (plus core per-plugin properties);
│ invalid → 400 with the validation errors, nothing saved
├─→ Split "x-secret" fields out; masked/blank secrets are dropped so
│ an untouched secret keeps its stored value
├─→ Deep-merge regular fields into config.json[<id>] (atomic save)
├─→ Merge secrets into config_secrets.json[<id>]
└─→ Call the loaded plugin's on_config_change() (and
on_enable/on_disable if "enabled" changed)
│
▼
One response for the whole form → notification in the UI
```
## Class and Function Hierarchy
The display service picks up the new config through its config hot reload
(ConfigService) without a restart.
### Frontend (JavaScript)
JSON clients can post `{"plugin_id": ..., "config": {...}}` instead; the keys
sent are merged onto the stored config the same way. See
[REST_API_REFERENCE.md](REST_API_REFERENCE.md#save-plugin-configuration).
```
Window Load
└── DOMContentLoaded
└── refreshPlugins()
├── fetch('/api/v3/plugins/installed')
├── renderInstalledPlugins(plugins)
└── generatePluginTabs(plugins)
└── For each plugin:
├── Create tab button
├── Create tab content
└── generatePluginConfigForm(plugin)
├── Read config_schema_data
├── Read current config
└── Generate form HTML
├── Boolean → Toggle switch
├── Number → Number input
├── String → Text input
├── Array → Comma-separated input
└── Enum → Select dropdown
### 3. Reset
User Interactions
├── configurePlugin(pluginId)
│ └── showTab(`plugin-${pluginId}`)
│
├── savePluginConfiguration(pluginId)
│ ├── Process form data
│ ├── Convert types per schema
│ └── For each field:
│ └── POST /api/v3/plugins/config
│
└── resetPluginConfig(pluginId)
├── Get schema defaults
└── For each field:
└── POST /api/v3/plugins/config
```
### Backend (Python)
```
Flask Routes
├── /api/v3/plugins/installed (GET)
│ └── api_plugins_installed()
│ ├── PluginManager.discover_plugins()
│ ├── For each plugin:
│ │ ├── PluginManager.get_plugin_info()
│ │ ├── Load config_schema.json
│ │ └── Load config from config.json
│ └── Return JSON response
│
└── /api/v3/plugins/config (POST)
└── api_plugin_config()
├── Parse request JSON
├── Load current config
├── Update config[plugin_id][key] = value
└── Save config.json
```
## File Structure
```
LEDMatrix/
│
├── web_interface_v2.py
│ └── Flask backend with plugin API endpoints
│
├── templates/
│ └── index_v2.html
│ └── Frontend with dynamic tab generation
│
├── config/
│ └── config.json
│ └── Stores all plugin configurations
│
├── plugins/
│ ├── hello-world/
│ │ ├── manifest.json ← Plugin metadata
│ │ ├── config_schema.json ← UI schema definition
│ │ ├── manager.py ← Plugin logic
│ │ └── requirements.txt
│ │
│ └── clock-simple/
│ ├── manifest.json
│ ├── config_schema.json
│ └── manager.py
│
└── docs/
├── PLUGIN_CONFIGURATION_TABS.md ← Full documentation
├── PLUGIN_CONFIG_TABS_SUMMARY.md ← Implementation summary
├── PLUGIN_CONFIG_QUICK_START.md ← Quick start guide
└── PLUGIN_CONFIG_ARCHITECTURE.md ← This file
```
`POST /api/v3/plugins/config/reset` replaces the plugin's section with the
schema defaults (keeping secrets unless `preserve_secrets` is false).
## Key Design Decisions
### 1. Dynamic Tab Generation
### 1. Server-side rendered forms
**Why**: Plugins are installed/uninstalled dynamically
**How**: JavaScript creates/removes tab elements on plugin list refresh
**Benefit**: No server-side template rendering needed
**Why**: One renderer for every plugin, no per-plugin frontend code
**How**: Jinja macros in `partials/plugin_config.html` walk the schema
**Benefit**: The settings search index is built from the same rendered HTML
(`/v3/settings/search-index`)
### 2. JSON Schema as Source of Truth
### 2. JSON Schema as source of truth
**Why**: Standard, well-documented, validation-ready
**How**: Frontend interprets schema to generate forms
**Benefit**: Plugin developers use familiar format
**Why**: Standard, well-documented, validation-ready
**How**: The same schema drives the form, the defaults and server-side validation
**Benefit**: Plugin developers use a familiar format
### 3. Individual Config Updates
### 3. Whole-form saves that merge
**Why**: Simplifies backend API
**How**: Each field saved separately via `/api/v3/plugins/config`
**Benefit**: Atomic updates, easier error handling
**Why**: A partial form (or a field the form doesn't show) must not wipe
stored values
**How**: The handler starts from the stored section and merges what was posted
**Benefit**: One request per save, atomic write
### 4. Type Conversion in Frontend
### 4. Secrets kept out of config.json
**Why**: HTML forms only return strings
**How**: JavaScript converts based on schema type before sending
**Benefit**: Backend receives correctly-typed values
### 5. No Nested Objects
**Why**: Keeps UI simple
**How**: Only flat property structures supported
**Benefit**: Easy form generation, clear to users
**Why**: `config.json` is shown in the raw editor and returned by the API
**How**: `"x-secret": true` fields go to `config_secrets.json`, which is
deep-merged back into the plugin's config at load time
**Benefit**: Plugins read secrets with plain `config.get(...)`
## Extension Points
### Adding New Input Types
### Custom input widgets
Location: `generatePluginConfigForm()` in `index_v2.html`
Set `"x-widget": "<name>"` on a property. Core widgets are in
`web_interface/static/v3/js/widgets/` (see its README); a plugin can ship its
own widget script, served from `/static/plugin-widgets/<plugin_id>/<name>.js`.
See [widget-guide.md](widget-guide.md).
```javascript
if (type === 'your-new-type') {
formHTML += `
<!-- Your custom input HTML -->
`;
}
```
### Custom actions
### Custom Validation
Buttons that run plugin scripts are declared in the manifest's
`web_ui_actions`. See [PLUGIN_WEB_UI_ACTIONS.md](PLUGIN_WEB_UI_ACTIONS.md).
Location: `savePluginConfiguration()` in `index_v2.html`
### Reacting to changes
```javascript
// Add validation before sending
if (!validateCustomConstraint(value, propSchema)) {
throw new Error('Validation failed');
}
```
Implement `on_config_change(new_config)` in the plugin (see
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md)).
### Backend Hook
## Where to Look
Location: `api_plugin_config()` in `web_interface_v2.py`
```python
# Add custom logic before saving
if plugin_id == 'special-plugin':
value = transform_value(value)
```
## Performance Considerations
### Frontend
- **Tab Generation**: O(n) where n = number of plugins (typically < 20)
- **Form Generation**: O(m) where m = number of config properties (typically < 10)
- **Memory**: Each plugin tab ~5KB HTML
- **Total Impact**: Negligible for typical use cases
### Backend
- **Schema Loading**: Cached after first load
- **Config Updates**: Single file write (atomic)
- **API Calls**: One per config field on save (sequential)
- **Optimization**: Could batch updates in single API call
## Security Considerations
1. **Input Validation**: Schema constraints enforced client-side (UX) and should be enforced server-side
2. **Path Traversal**: Plugin paths validated against known plugin directory
3. **XSS**: All user inputs escaped before rendering in HTML
4. **CSRF**: Flask CSRF tokens should be used in production
5. **File Permissions**: config.json requires write access
| Concern | File |
|---------|------|
| Tab partial loader | `web_interface/blueprints/pages_v3.py` (`_load_plugin_config_partial`) |
| Form template and field macros | `web_interface/templates/v3/partials/plugin_config.html` |
| Save / get / schema / reset handlers | `web_interface/blueprints/api_v3/plugins.py` |
| Schema loading, defaults, validation | `src/plugin_system/schema_manager.py` |
| Secret masking and splitting | `src/web_interface/secret_helpers.py` |
| Widgets | `web_interface/static/v3/js/widgets/` |
## Error Handling
### Frontend
- Network errors: Show notification, don't crash
- Schema errors: Graceful fallback to no config tab
- Type errors: Log to console, continue processing other fields
### Backend
- Invalid plugin_id: 400 Bad Request
- Schema not found: Return null, frontend handles gracefully
- Config save error: 500 Internal Server Error with message
## Testing Strategy
### Unit Tests
- `generatePluginConfigForm()` for each schema type
- Type conversion logic in `savePluginConfiguration()`
- Backend schema loading logic
### Integration Tests
- Full save flow: form → API → config.json
- Tab generation from API response
- Reset to defaults
### E2E Tests
- Install plugin → verify tab appears
- Configure plugin → verify config saved
- Uninstall plugin → verify tab removed
## Monitoring
### Frontend Metrics
- Time to generate tabs
- Form submission success rate
- User interactions (configure, save, reset)
### Backend Metrics
- API response times
- Config update success rate
- Schema loading errors
### User Feedback
- Are users finding the configuration interface?
- Are validation errors clear?
- Are default values sensible?
## Future Roadmap
### Phase 2: Enhanced Validation
- Real-time validation feedback
- Custom error messages
- Dependent field validation
### Phase 3: Advanced Inputs
- Color pickers for RGB arrays
- File upload for assets
- Rich text editor for descriptions
### Phase 4: Configuration Management
- Export/import configurations
- Configuration presets
- Version history/rollback
### Phase 5: Developer Tools
- Schema editor in web UI
- Live preview while editing schema
- Validation tester
- Unknown plugin or unreadable schema: the partial renders an error message
- Validation failure: `400` with `details` and `context.validation_errors`;
the form shows them and nothing is saved
- Save failure: `500` with an error message; config.json is written
atomically, so a failed save leaves the previous file intact
docs/PLUGIN_DEPENDENCY_GUIDE.md
+110 -184
View File
@@ -2,234 +2,160 @@
## Overview
The LEDMatrix system has smart dependency installation that adapts based on who is running it. This guide explains how it works and potential pitfalls.
A plugin lists its Python packages in its `requirements.txt`. LEDMatrix
installs them for you when a plugin is installed, updated or loaded. This
guide explains where they end up and what to do when a plugin can't import a
package.
## How It Works
The rule to remember: **packages must be importable by `ledmatrix.service`,
which runs as root.** Anything installed only into another user's
`~/.local/` is invisible to it.
### Execution Context Detection
## Who Runs What
The plugin manager checks if it's running as root:
```python
running_as_root = os.geteuid() == 0
| Service | Runs as | Set by |
|---------|---------|--------|
| `ledmatrix.service` (display) | `root` | `systemd/ledmatrix.service` |
| `ledmatrix-web.service` (web UI) | the user who ran the installer (e.g. `ledpi`) | `User=__USER__` in `systemd/ledmatrix-web.service`, filled in by `scripts/install/install_service.sh` |
## How Dependencies Get Installed
### 1. Installing or updating a plugin from the web UI
The web interface is not root, so it installs through a narrow sudo helper:
1. `PluginStoreManager._install_dependencies()`
(`src/plugin_system/store_manager.py`) calls
`install_requirements_file()` (`src/common/permission_utils.py`).
2. That runs `sudo -n bash scripts/fix_perms/safe_pip_install.sh <plugin>/requirements.txt`.
The helper checks the path is the project's own `requirements.txt` or a
`requirements.txt` under `plugin-repos/` or `plugins/`, then runs
`python3 -m pip install --break-system-packages --ignore-installed -r ...`
**as root**, so the display service can import the packages.
3. The sudoers rule that allows this is written by the installer
(`first_time_install.sh`) or by `scripts/install/configure_web_sudo.sh`.
If sudo refuses (the rule isn't installed), `install_requirements_file()`
falls back to installing with the web process's own interpreter, as the web
user, and prefixes the pip output with a note like:
```
[Root install unavailable (...); installed for the current process's user only.
Packages may not be visible to ledmatrix.service if it runs as a different
user — run scripts/install/configure_web_sudo.sh to fix this.]
```
Based on this, it chooses the appropriate installation method:
Fix it by running `./scripts/install/configure_web_sudo.sh` as the web
user (not with `sudo`; it asks for your password itself), then
reinstall the plugin (or use the manual install below).
| Running As | Installation Method | Location | Accessible To |
|------------|-------------------|----------|---------------|
| **root** (systemd service) | System-wide (`--break-system-packages`) | `/usr/local/lib/python3.X/dist-packages/` | All users |
| **ledpi** or other user | User-specific (`--user`) | `~/.local/lib/python3.X/site-packages/` | Only that user |
The **Reinstall Plugin Deps** button on the web UI's Tools tab goes
through the same helper for every installed plugin.
### 2. Loading a plugin
When a plugin loads, `PluginLoader.install_dependencies()`
(`src/plugin_system/plugin_loader.py`) checks its `requirements.txt`. If the
requirements are already satisfied it does nothing; otherwise it runs
`python3 -m pip install --break-system-packages -r requirements.txt` with the
interpreter of the process doing the loading (retrying with
`--ignore-installed` when a system package without a pip RECORD file is in
the way).
In `ledmatrix.service` that process is root, so restarting the display
service installs anything missing system-wide:
```bash
sudo systemctl restart ledmatrix
```
If you run `python3 run.py` by hand as a normal user instead, pip cannot
write to the system site-packages and installs into your `~/.local/`. That
works for your manual run but not for the service.
## Common Scenarios
### ✅ Scenario 1: Normal Production Use (Recommended)
### Installing plugins from the web UI (recommended)
**What:** Services running via systemd
Use the **Plugin Manager** tab. Dependencies are installed as root through
the sudo helper and the display service can use them.
### Running the display manually for debugging
```bash
sudo systemctl start ledmatrix
sudo systemctl start ledmatrix-web
cd ~/LEDMatrix
sudo python3 run.py # same user as the service
```
- **Runs as:** root (configured in .service files)
- **Installs to:** System-wide
- **Result:** ✅ Works perfectly, all dependencies accessible
Running as your own user works for plugins whose packages are already
installed system-wide, but any *missing* package lands in `~/.local/`.
### ✅ Scenario 2: Web Interface Plugin Installation
### A plugin works when run manually but fails in the service
**What:** Installing/enabling plugins via web interface at `http://pi-ip:5000`
Its packages were installed for your user only. Install them as root (see
below) and restart the service.
- **Web service runs as:** root (ledmatrix-web.service)
- **Installs to:** System-wide
- **Result:** ✅ Works perfectly, systemd service can access them
## Manual Installation
### ✅ Scenario 3: Manual Testing as ledpi (Read-only)
**What:** Running display manually as ledpi to test/debug
### All plugins
```bash
# As ledpi user
cd /home/ledpi/LEDMatrix
python3 run.py
```
- **Runs as:** ledpi
- **Can import:** ✅ System-wide packages (installed by root)
- **Result:** ✅ Works! Can use existing plugins with root-installed dependencies
### ⚠️ Scenario 4: Manual Plugin Installation as ledpi (Problematic)
**What:** Enabling a NEW plugin and running manually as ledpi
```bash
# As ledpi user
cd /home/ledpi/LEDMatrix
# Edit config to enable new plugin
nano config/config.json
# Run display - will try to install new plugin dependencies
python3 run.py
```
**What Happens:**
1. Plugin manager runs as `ledpi`
2. Installs dependencies with `--user` flag
3. Dependencies go to `~/.local/lib/python3.X/site-packages/`
4. ⚠️ **Warning logged:** "Installing plugin dependencies for current user (not root)"
**Problem:**
- When systemd service restarts (as root), it **can't see** `~/.local/` packages
- Plugin will fail to load for the systemd service
**Solution:**
After testing, restart the service to install dependencies system-wide:
```bash
sudo ~/LEDMatrix/scripts/install_plugin_dependencies.sh
sudo systemctl restart ledmatrix
```
## Best Practices
The script installs every `requirements.txt` found in the plugins directory
configured by `plugin_system.plugins_directory` in `config/config.json`
(default `plugin-repos/`). Run it with `sudo` so the packages are installed
system-wide.
### For Production/Normal Use
### One plugin
1. **Always use the web interface** to install/enable plugins
2. **Or restart the systemd service** after config changes:
```bash
sudo systemctl restart ledmatrix
```
### For Development/Testing
1. **Read existing plugins:** Safe to run as `ledpi` - can import system packages
2. **Test new plugins:** Use sudo or restart service to install dependencies:
```bash
# Option 1: Run as root
sudo python3 run.py
# Option 2: Install deps manually
sudo pip3 install --break-system-packages -r plugins/my-plugin/requirements.txt
python3 run.py
# Option 3: Let service install them
sudo systemctl restart ledmatrix
```
## Warning Messages
### If you see this warning:
```
Installing plugin dependencies for current user (not root).
These will NOT be accessible to the systemd service.
For production use, install plugins via the web interface or restart the ledmatrix service.
```
**What it means:**
- You're running as a non-root user
- Dependencies were installed to your user directory only
- The systemd service won't be able to use this plugin
**What to do:**
```bash
# Restart the service to install dependencies system-wide
cd ~/LEDMatrix/plugin-repos/PLUGIN-NAME # or your configured plugins directory
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
sudo systemctl restart ledmatrix
```
`--no-cache-dir` avoids errors about `/root/.cache/pip` not being writable.
## Troubleshooting
### Plugin works when I run manually but fails in systemd service
**Cause:** Dependencies installed to user directory (`~/.local/`) instead of system-wide
**Fix:**
```bash
# Check where package is installed
pip3 list -v | grep <package-name>
# If it shows ~/.local/, reinstall system-wide:
sudo pip3 install --break-system-packages <package-name>
# Or just restart the service:
sudo systemctl restart ledmatrix
```
### Permission denied when installing dependencies
**If you see errors like:**
```
ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/root/.local'
WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable
```
**Quick Fix - Use the Helper Script:**
```bash
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
sudo systemctl restart ledmatrix
```
Use one of the manual installs above (they pass `--no-cache-dir`).
**Manual Fix:**
```bash
# Install dependencies with --no-cache-dir to avoid cache permission issues
cd /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
sudo systemctl restart ledmatrix
```
**For more detailed troubleshooting, see:** [Plugin Dependency Troubleshooting Guide](PLUGIN_DEPENDENCY_TROUBLESHOOTING.md)
## Architecture Summary
```
┌─────────────────────────────────────────────────────────────┐
│ LEDMatrix Services │
├─────────────────────────────────────────────────────────────┤
│ │
│ ledmatrix.service (User=root) │
│ ledmatrix-web.service (User=root) │
│ ├── Install dependencies system-wide │
│ └── Accessible to all users │
│ │
├─────────────────────────────────────────────────────────────┤
│ │
│ Manual execution as ledpi │
│ ├── Can READ system-wide packages ✅ │
│ ├── WRITES go to ~/.local/ ⚠️ │
│ └── Not accessible to root service │
│ │
└─────────────────────────────────────────────────────────────┘
```
## Recommendations
1. **For end users:** Always use the web interface for plugin management
2. **For developers:** Be aware of the user context when testing
3. **For plugin authors:** Test with `sudo systemctl restart ledmatrix` to ensure dependencies install correctly
4. **For CI/CD:** Always run installation as root or use the service
## Helper Scripts
### Install Plugin Dependencies Script
Located at: `scripts/install_plugin_dependencies.sh`
This script automatically finds and installs dependencies for all plugins:
### Checking where a package is installed
```bash
# Run as root (recommended for production)
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
# How the service sees it
sudo python3 -c "import package_name; print(package_name.__file__)"
# Make executable if needed
chmod +x /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
# A path under /home/<user>/.local/ means it was installed for that user only
python3 -m pip show -f package_name
```
Features:
- Auto-detects all plugins with requirements.txt
- Uses correct installation method (system-wide vs user)
- Bypasses pip cache to avoid permission issues
- Provides detailed logging and error messages
For more, see the [Plugin Dependency Troubleshooting Guide](PLUGIN_DEPENDENCY_TROUBLESHOOTING.md).
## For Plugin Authors
1. Keep `requirements.txt` minimal and pin only what you need.
2. Test that it installs the way the Pi will install it:
```bash
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
```
3. Note any `apt` packages your plugin needs in its README.
## Files to Reference
- Service configs: `ledmatrix.service`, `ledmatrix-web.service`
- Plugin manager: `src/plugin_system/plugin_manager.py`
- Installation script: `first_time_install.sh`
- Dependency installer: `scripts/install_plugin_dependencies.sh`
- Troubleshooting guide: `PLUGIN_DEPENDENCY_TROUBLESHOOTING.md`
- Service units: `systemd/ledmatrix.service`, `systemd/ledmatrix-web.service`
- Store installs: `src/plugin_system/store_manager.py` (`_install_dependencies`)
- Root install helper: `src/common/permission_utils.py` (`install_requirements_file`), `scripts/fix_perms/safe_pip_install.sh`
- Load-time installs: `src/plugin_system/plugin_loader.py` (`install_dependencies`)
- Sudo rules: `scripts/install/configure_web_sudo.sh`
- Manual installer: `scripts/install_plugin_dependencies.sh`
docs/PLUGIN_DEPENDENCY_TROUBLESHOOTING.md
+77 -82
View File
@@ -1,6 +1,7 @@
# Plugin Dependency Installation Troubleshooting
This guide helps resolve issues with automatic plugin dependency installation in the LEDMatrix system.
This guide helps resolve problems installing a plugin's Python packages. For
how installation works, see the [Plugin Dependency Guide](PLUGIN_DEPENDENCY_GUIDE.md).
## Common Error Symptoms
@@ -10,109 +11,118 @@ ERROR: Could not install packages due to an OSError: [Errno 13] Permission denie
WARNING: The directory '/root/.cache/pip' or its parent directory is not owned or is not writable
```
### Context Mismatch
### Installed for the wrong user
The pip output shown after a web-UI install starts with:
```
WARNING: Installing plugin dependencies for current user (not root).
These will NOT be accessible to the systemd service.
[Root install unavailable (...); installed for the current process's user only.
Packages may not be visible to ledmatrix.service if it runs as a different
user — run scripts/install/configure_web_sudo.sh to fix this.]
```
### Plugin fails to load with `ModuleNotFoundError`
The display service can't see a package the plugin needs.
## Root Cause
Plugin dependencies must be installed in a context accessible to the LEDMatrix systemd service, which runs as root. Permission errors typically occur when:
Plugin packages must be importable by `ledmatrix.service`, which runs as
root. The web interface (`ledmatrix-web.service`) runs as the user who
installed LEDMatrix, so it installs through a sudo helper
(`scripts/fix_perms/safe_pip_install.sh`). Problems usually come from:
1. The pip cache directory has incorrect permissions
2. The process tries to install to user directories without proper permissions
3. Environment variables (like HOME) are not set correctly for the service context
1. The sudoers rule for that helper missing, so the web UI installed the
packages for its own user only
2. Running `python3 run.py` by hand as a normal user, which installs missing
packages into `~/.local/`
3. pip's cache directory not being writable for root
## Solutions
### Solution 1: Use the Manual Installation Script (Recommended)
We provide a helper script that handles dependency installation correctly:
### Solution 1: Restore the sudo rule, then reinstall
```bash
# Run as root to install system-wide (for production)
sudo /home/ledpi/LEDMatrix/scripts/install_plugin_dependencies.sh
cd ~/LEDMatrix
./scripts/install/configure_web_sudo.sh # as the web user, not with sudo
```
# After installation, restart the service
Then reinstall the plugin from the **Plugin Manager** tab, or click
**Reinstall Plugin Deps** on the **Tools** tab.
### Solution 2: Install every plugin's dependencies from the terminal
```bash
sudo ~/LEDMatrix/scripts/install_plugin_dependencies.sh
sudo systemctl restart ledmatrix
```
This script:
- Detects all plugins with requirements.txt files
- Installs dependencies with correct permissions
- Uses `--no-cache-dir` to avoid cache permission issues
- Provides detailed logging for troubleshooting
The script finds each `requirements.txt` in the plugins directory set by
`plugin_system.plugins_directory` in `config/config.json` (default
`plugin-repos/`), installs with `--no-cache-dir`, and reports what it found.
### Solution 2: Manual Installation per Plugin
If you need to install dependencies for a specific plugin:
### Solution 3: Install one plugin's dependencies
```bash
# Navigate to the plugin directory
cd /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME
# Your configured plugins directory; plugin-repos/ by default
cd ~/LEDMatrix/plugin-repos/PLUGIN-NAME
# Install as root (system-wide)
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
# Restart the service
sudo systemctl restart ledmatrix
```
### Solution 3: Fix Cache Directory Permissions
### Solution 4: Let the display service install them
If you specifically have cache permission issues:
When a plugin loads, the display service installs any missing requirements
itself, as root:
```bash
sudo systemctl restart ledmatrix
sudo journalctl -u ledmatrix -f # watch for "Installing dependencies for plugin ..."
```
### Solution 5: Fix pip cache permissions
```bash
# Option A: Skip the cache (recommended)
sudo pip3 install --no-cache-dir --break-system-packages -r requirements.txt
sudo python3 -m pip install --no-cache-dir --break-system-packages -r requirements.txt
# Option B: Fix cache permissions (if needed)
# Option B: Fix cache permissions
sudo mkdir -p /root/.cache/pip
sudo chown -R root:root /root/.cache
sudo chmod -R 755 /root/.cache
```
### Solution 4: Install via Web Interface
The web interface handles dependency installation correctly in the service context:
1. Access the web interface (`http://ledpi:5000` or `http://your-pi-ip:5000`)
2. Open the **Plugin Manager** tab (use the **Plugin Store** section to
find the plugin, or **Install from GitHub**)
3. Install the plugin through the web UI
4. The system automatically handles dependency installation in the
service context (which has the right permissions)
## Prevention
### For Plugin Developers
When creating plugins with dependencies:
1. **Keep requirements minimal**: Only include essential packages
2. **Test installation**: Verify your requirements.txt works with:
2. **Test installation** the way the Pi does it:
```bash
sudo pip3 install --break-system-packages --no-cache-dir -r requirements.txt
sudo python3 -m pip install --break-system-packages --no-cache-dir -r requirements.txt
```
3. **Document dependencies**: Note any system packages needed (via apt)
### For Users
1. **Use web interface**: Install plugins via the web UI when possible
2. **Install as root**: When using SSH/terminal, use sudo for plugin installations
3. **Restart service**: After manual installations, restart the ledmatrix service
1. **Use the web interface** to install plugins
2. **Use sudo** for installs from SSH/terminal
3. **Restart the service** after manual installations
## Technical Details
### How Dependency Installation Works
### Where installs happen
The `PluginManager._install_plugin_dependencies()` method:
1. Detects if running as root using `os.geteuid() == 0`
2. If root: Uses system-wide installation with `--break-system-packages --no-cache-dir`
3. If not root: Uses user installation with `--user --break-system-packages --no-cache-dir`
4. The `--no-cache-dir` flag prevents cache-related permission issues
- **Web UI install/update:** `PluginStoreManager._install_dependencies()`
→ `install_requirements_file()` in `src/common/permission_utils.py`, which
runs `sudo -n bash scripts/fix_perms/safe_pip_install.sh <requirements.txt>`.
The helper only accepts the project's `requirements.txt` or one under
`plugin-repos/` or `plugins/`, and runs
`pip install --break-system-packages --ignore-installed` as root. If sudo
refuses, it falls back to a pip install as the web user and says so.
- **Plugin load:** `PluginLoader.install_dependencies()` in
`src/plugin_system/plugin_loader.py` skips satisfied requirements and
otherwise runs `pip install --break-system-packages` with the loading
process's interpreter — root in `ledmatrix.service`.
### Why `--break-system-packages`?
@@ -120,26 +130,20 @@ Debian 12+ (Bookworm) and Raspberry Pi OS based on it implement PEP 668, which p
### Service Context
The ledmatrix.service runs as:
- **User**: root
- **WorkingDirectory**: /home/ledpi/LEDMatrix
- **Python**: /usr/bin/python3
- `ledmatrix.service` runs as **root** with `/usr/bin/python3`
- `ledmatrix-web.service` runs as **the installing user**
Dependencies must be installed in root's Python environment or system-wide to be accessible.
Dependencies must be installed system-wide (as root) to be visible to the
display service.
## Checking Installation
Verify dependencies are installed correctly:
```bash
# Check as root (how the service sees it)
sudo python3 -c "import package_name"
sudo python3 -c "import package_name; print(package_name.__file__)"
# List installed packages
pip3 list
# Check specific package
pip3 show package_name
# A path under /home/<user>/.local/ means a user-only install
python3 -m pip show -f package_name
```
## Getting Help
@@ -151,19 +155,11 @@ If you continue to experience issues:
sudo journalctl -u ledmatrix -f
```
2. Check pip logs (created by manual script):
2. Verify the plugin manifest and requirements (default plugins directory
shown):
```bash
cat /tmp/pip_install_*.log
```
3. Verify plugin manifest is correct:
```bash
cat /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME/manifest.json
```
4. Check plugin requirements:
```bash
cat /home/ledpi/LEDMatrix/plugins/PLUGIN-NAME/requirements.txt
cat ~/LEDMatrix/plugin-repos/PLUGIN-NAME/manifest.json
cat ~/LEDMatrix/plugin-repos/PLUGIN-NAME/requirements.txt
```
## Related Documentation
@@ -171,4 +167,3 @@ If you continue to experience issues:
- [Plugin Dependency Guide](PLUGIN_DEPENDENCY_GUIDE.md)
- [Plugin Development Guide](PLUGIN_DEVELOPMENT_GUIDE.md)
- [Troubleshooting](TROUBLESHOOTING.md)
docs/PLUGIN_DEVELOPMENT_GUIDE.md
+109 -88
View File
@@ -3,8 +3,10 @@
This guide explains how to set up a development workflow for plugins that are maintained in separate Git repositories while still being able to test them within the LEDMatrix project.
> **Rendering guidance:** plugins should read the display size dynamically
> (`self.display_manager.matrix.width/height`) rather than hardcoding one
> panel. For plugins that want to *scale* their layout to any panel, the
> (`self.display_manager.width/height`) rather than hardcoding one
> panel. Don't read `display_manager.matrix.width/height`: `matrix` is
> `None` when hardware init fails, while the `width`/`height` properties
> fall back to the canvas size. For plugins that want to *scale* their layout to any panel, the
> opt-in adaptive layout system ([ADAPTIVE_LAYOUT.md](ADAPTIVE_LAYOUT.md))
> provides the shared helpers — fonts, images, and composite layouts that
> scale. Existing plugins keep their classic rendering unless they adopt
@@ -43,28 +45,45 @@ The solution uses **symbolic links** to connect plugin repositories to the `plug
## Quick Start
### 1. Link a Plugin from GitHub
Official plugins all live in one repository,
[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins), with
one directory per plugin under `plugins/` (there are no per-plugin
`ledmatrix-<name>` repositories). The helper script links a plugin directory
from a checkout of that monorepo into LEDMatrix's `plugins/` directory.
The easiest way to link a plugin that's already on GitHub:
### 1. Link an Official Plugin
```bash
./scripts/dev/dev_plugin_setup.sh link-github music
./scripts/dev/dev_plugin_setup.sh link-github football-scoreboard
```
This will:
- Clone `https://github.com/ChuckBuilds/ledmatrix-music.git` to `~/.ledmatrix-dev-plugins/ledmatrix-music`
- Create a symbolic link from `plugins/music` to the cloned repository
- Validate that the plugin has a proper `manifest.json`
- Clone `https://github.com/ChuckBuilds/ledmatrix-plugins.git` to
`~/.ledmatrix-dev-plugins/ledmatrix-plugins` (or `git pull` it if it is
already there)
- Find `plugins/football-scoreboard` in it (also accepted:
`plugins/ledmatrix-<name>`, or a plugin whose manifest `id` is the name)
- Validate that it has a `manifest.json`
- Create a symbolic link named after the plugin's manifest id, e.g.
`plugins/football-scoreboard` → `~/.ledmatrix-dev-plugins/ledmatrix-plugins/plugins/football-scoreboard`
### 2. Link a Local Plugin Repository
`link-github music` finds the monorepo's `plugins/ledmatrix-music` directory
and links it into LEDMatrix as `plugins/ledmatrix-music`, because
`ledmatrix-music` is that plugin's manifest id.
If you already have a plugin repository cloned locally:
To work from your fork of the monorepo, set `github_user` in
`dev_plugins.json` (see [Configuration](#configuration)).
### 2. Link a Local Plugin Directory
If you already have the monorepo (or a third-party plugin repository) cloned
locally:
```bash
./scripts/dev/dev_plugin_setup.sh link music ../ledmatrix-music
./scripts/dev/dev_plugin_setup.sh link hello-world ../ledmatrix-plugins/plugins/hello-world
```
This creates a symlink from `plugins/music` to your local repository path.
This creates a symlink from `plugins/hello-world` to that directory.
### 3. Check Status
@@ -77,13 +96,17 @@ See which plugins are linked and their git status:
### 4. Work on Your Plugin
```bash
cd plugins/music # Actually editing the linked repository
# Make your changes
cd plugins/football-scoreboard # Actually editing the monorepo checkout
# Make your changes, then bump "version" in manifest.json
git add .
git commit -m "feat: add new feature"
git push origin main
git commit -m "feat(football-scoreboard): add new feature"
git push # to your fork, then open a PR against ledmatrix-plugins
```
In the monorepo, every plugin change must bump `version` in the plugin's
`manifest.json` and run `python update_registry.py`, or users won't receive
the update.
### 5. Update Plugins
Pull latest changes from remote:
@@ -116,7 +139,7 @@ Links a local plugin repository to the plugins directory.
**Example:**
```bash
./scripts/dev/dev_plugin_setup.sh link football-scoreboard ../ledmatrix-football-scoreboard
./scripts/dev/dev_plugin_setup.sh link football-scoreboard ../ledmatrix-plugins/plugins/football-scoreboard
```
**Notes:**
@@ -129,23 +152,25 @@ Links a local plugin repository to the plugins directory.
Clones a plugin from GitHub and links it.
**Arguments:**
- `plugin-name`: The name of the plugin (will be the directory name in `plugins/`)
- `repo-url`: (Optional) Full GitHub repository URL. If omitted, constructs from pattern: `https://github.com/ChuckBuilds/ledmatrix-<plugin-name>.git`
- `plugin-name`: Without `repo-url`, the plugin to link from the monorepo: a
directory under `plugins/` (`<name>` or `ledmatrix-<name>`) or a manifest
id. The link is named after the plugin's manifest id. With `repo-url`, the
name of the link in `plugins/`.
- `repo-url`: (Optional) A plugin that has its own repository (e.g. a
third-party plugin). The repository root is linked.
**Examples:**
```bash
# Auto-construct URL from plugin name
./scripts/dev/dev_plugin_setup.sh link-github music
# Official plugin, from the ledmatrix-plugins monorepo
./scripts/dev/dev_plugin_setup.sh link-github stocks
# Use explicit URL
./scripts/dev/dev_plugin_setup.sh link-github stocks https://github.com/ChuckBuilds/ledmatrix-stocks.git
# Link from a different GitHub user
# Third-party plugin with its own repository
./scripts/dev/dev_plugin_setup.sh link-github custom-plugin https://github.com/OtherUser/custom-plugin.git
```
**Notes:**
- Repositories are cloned to `~/.ledmatrix-dev-plugins/` by default (configurable)
- The monorepo is cloned once and shared by every plugin you link from it
- If the repository already exists, it will be updated with `git pull` instead of re-cloning
- The cloned repository is preserved when you unlink the plugin
@@ -217,30 +242,28 @@ Updates plugin(s) by running `git pull` in their repositories.
### Custom Development Directory
By default, GitHub repositories are cloned to `~/.ledmatrix-dev-plugins/`. You can customize this by creating a `dev_plugins.json` file:
By default, GitHub repositories are cloned to `~/.ledmatrix-dev-plugins/`
and official plugins come from `ChuckBuilds/ledmatrix-plugins`. To change
either, copy `dev_plugins.json.example` (in the LEDMatrix root) to
`dev_plugins.json` and edit it. `dev_plugins.json` is git-ignored.
```json
{
"dev_plugins_dir": "/path/to/your/dev/plugins",
"github_user": "ChuckBuilds",
"github_pattern": "ledmatrix-",
"plugins": {
"music": {
"source": "github",
"url": "https://github.com/ChuckBuilds/ledmatrix-music.git",
"branch": "main"
}
}
"dev_plugins_dir": "~/.ledmatrix-dev-plugins",
"github_user": "your-github-user",
"plugins_repo": "ledmatrix-plugins",
"plugins_branch": "main"
}
```
**Configuration options:**
**Configuration options** (all optional):
- `dev_plugins_dir`: Where to clone GitHub repositories (default: `~/.ledmatrix-dev-plugins`)
- `github_user`: Default GitHub username for auto-constructing URLs
- `github_pattern`: Pattern for repository names (default: `ledmatrix-`)
- `plugins`: Plugin definitions (optional, for future auto-discovery features)
- `github_user`: Owner of the plugin monorepo that `link-github <name>` clones — set it to use your fork (default: `ChuckBuilds`)
- `plugins_repo`: Name of that monorepo (default: `ledmatrix-plugins`)
- `plugins_branch`: Branch to clone it at (default: the repository's default branch). Only applies when the clone is first made.
**Note:** Copy `dev_plugins.json.example` to `dev_plugins.json` and customize it. The `dev_plugins.json` file is git-ignored.
`github_pattern` from older versions of this guide is no longer used (the
script warns if it is set).
## Development Workflow
@@ -248,43 +271,46 @@ By default, GitHub repositories are cloned to `~/.ledmatrix-dev-plugins/`. You c
1. **Link your plugin for development:**
```bash
./scripts/dev/dev_plugin_setup.sh link-github music
./scripts/dev/dev_plugin_setup.sh link-github clock-simple
```
2. **Test in LEDMatrix:**
```bash
# Run LEDMatrix with your plugin
python run.py
# Run LEDMatrix with your plugin (emulator shown)
python3 run.py -e
```
3. **Make changes:**
```bash
cd plugins/music
cd plugins/clock-simple
# Edit files...
# Test changes...
```
4. **Commit to plugin repository:**
4. **Commit to the plugin repository:**
```bash
cd plugins/music # This is actually your repo
cd plugins/clock-simple # This is inside your monorepo checkout
# bump "version" in manifest.json, then from the monorepo root:
# python update_registry.py
git add .
git commit -m "feat: add new feature"
git push origin main
git commit -m "feat(clock-simple): add new feature"
git push
```
5. **Update from remote (if needed):**
```bash
./scripts/dev/dev_plugin_setup.sh update music
./scripts/dev/dev_plugin_setup.sh update clock-simple
```
6. **When done developing:**
```bash
./scripts/dev/dev_plugin_setup.sh unlink music
./scripts/dev/dev_plugin_setup.sh unlink clock-simple
```
### Working with Multiple Plugins
You can have multiple plugins linked simultaneously:
You can have multiple plugins linked simultaneously. Plugins linked from the
monorepo share one checkout:
```bash
./scripts/dev/dev_plugin_setup.sh link-github music
@@ -294,7 +320,7 @@ You can have multiple plugins linked simultaneously:
# Check status of all
./scripts/dev/dev_plugin_setup.sh status
# Update all at once
# Update all at once (the shared monorepo checkout is pulled once)
./scripts/dev/dev_plugin_setup.sh update
```
@@ -409,7 +435,7 @@ If you have conflicts when updating:
1. **Manually resolve in the plugin repository:**
```bash
cd ~/.ledmatrix-dev-plugins/ledmatrix-music
cd ~/.ledmatrix-dev-plugins/ledmatrix-plugins
git pull
# Resolve conflicts...
git add .
@@ -470,18 +496,19 @@ You can mix local and GitHub plugins:
The development workflow is separate from the plugin store installation:
- **Plugin Store:** Installs plugins to `plugins/` as regular directories
- **Development Setup:** Links plugin repositories as symlinks
- **Plugin Store:** Installs plugins as regular directories in the configured
plugins directory (`plugin-repos/` by default)
- **Development Setup:** Links plugin directories as symlinks in `plugins/`
If you install a plugin via the store, you can still link it for development:
The plugin loader scans only one directory, so while developing set
`plugin_system.plugins_directory` to `plugins` (see the note at the top of
this guide). If `plugins/` already holds a regular directory of the same
name, `link`/`link-github` offers to rename it to
`<name>.backup.<timestamp>` before linking.
```bash
# Store installs to plugins/music (regular directory)
# Link for development (will prompt to replace)
./scripts/dev/dev_plugin_setup.sh link-github music
```
When you unlink, the directory is removed. If you want to switch back to the store version, re-install it via the plugin store.
`unlink` removes only the symlink. To switch back to the store version, set
`plugins_directory` back to `plugin-repos` (or reinstall the plugin from the
store).
## API Reference
@@ -525,7 +552,7 @@ Want to create and share your own plugin? Here's everything you need to know.
- [Advanced Plugin Development](ADVANCED_PLUGIN_DEVELOPMENT.md) - Patterns and examples
2. **Start with a template**:
- Use the [Hello World plugin](https://github.com/ChuckBuilds/ledmatrix-hello-world) as a starting point
- Use the [Hello World plugin](https://github.com/ChuckBuilds/ledmatrix-plugins/tree/main/plugins/hello-world) as a starting point
- Or fork an existing plugin and modify it
3. **Follow the plugin structure**:
@@ -589,24 +616,16 @@ Your plugin must:
### Versioning Best Practices
- **Use semantic versioning**: `MAJOR.MINOR.PATCH` (e.g., `1.2.3`)
- **GitHub as source of truth**: the plugin store resolves versions in this
order: GitHub Releases → GitHub Tags → manifest from branch → git commit hash
- **Automatic version bumping**: install the self-contained pre-push hook in
your plugin repo and patch versions bump themselves on push (a git tag
`v{version}` is created and `manifest.json` staged automatically):
```bash
# From your plugin repository directory
cp /path/to/LEDMatrix/scripts/git-hooks/pre-push-plugin-version .git/hooks/pre-push
chmod +x .git/hooks/pre-push
```
Set `SKIP_TAG=1` in the environment to skip auto-tagging for one push.
- **Manual versioning**: only needed for major/minor bumps, CI pipelines that
bypass hooks, or forks without the hook — use
`scripts/bump_plugin_version.py`.
- **Registry stores no versions**: `plugins.json` holds only metadata (name,
description, repo URL).
- **Bump `version` in `manifest.json` by hand** for every change you ship.
There is no automatic version-bump hook or bump script.
- **Official (monorepo) plugins**: after bumping the manifest, run
`python update_registry.py` in the `ledmatrix-plugins` checkout. It copies
each manifest's version into `plugins.json` as `latest_version`, which is
what the store compares installed versions against. Without it, users
won't be offered the update.
- **Plugins in their own repository**: still bump the manifest `version`,
so users can see which version they run; tagging releases (`v1.2.3`) to
match is a good habit.
### Submitting to Official Registry
@@ -618,12 +637,14 @@ To have your plugin added to the official plugin store:
- Follows best practices
- Tested on Raspberry Pi hardware
2. **Create GitHub repository**:
- Repository name: `ledmatrix-<plugin-name>`
- Public repository
- Proper README.md with installation instructions
2. **Choose where it lives** (see `SUBMISSION.md` in
[ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins)):
- **In the monorepo (preferred):** fork ledmatrix-plugins, add
`plugins/<your-plugin-id>/`, and open a pull request
- **In your own public repository** (conventionally
`ledmatrix-<plugin-name>`), with a README that covers installation
3. **Contact maintainers**:
3. **Contact maintainers** (own-repository plugins):
- Open a GitHub issue in the [ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) repository
- Or reach out on Discord: https://discord.gg/uW36dVAtcT
- Include: Repository URL, plugin description, why it's useful
docs/PLUGIN_QUICK_REFERENCE.md
+2 -1
View File
@@ -127,7 +127,8 @@ git push origin v1.0.0
### REST API
The API is mounted at `/api/v3` (`web_interface/app.py:199`).
The API is mounted at `/api/v3` (the `api_v3` blueprint in
`web_interface/blueprints/api_v3/`, registered in `web_interface/app.py`).
```bash
# Install plugin from the registry
docs/PLUGIN_REGISTRY_SETUP_GUIDE.md
+3
View File
@@ -103,6 +103,7 @@ All plugins can be installed through the LEDMatrix web interface:
Or via API:
```bash
curl -X POST http://your-pi-ip:5000/api/v3/plugins/install \
-H "Content-Type: application/json" \
-d '{"plugin_id": "clock-simple"}'
```
@@ -153,6 +154,7 @@ Before submitting, ensure your plugin:
```bash
# Install via URL on your Pi
curl -X POST http://your-pi:5000/api/v3/plugins/install-from-url \
-H "Content-Type: application/json" \
-d '{"repo_url": "https://github.com/you/ledmatrix-your-plugin"}'
```
@@ -312,6 +314,7 @@ git push
# 2. Review using VERIFICATION.md checklist
# 3. Test installation:
curl -X POST http://pi:5000/api/v3/plugins/install-from-url \
-H "Content-Type: application/json" \
-d '{"repo_url": "https://github.com/contributor/plugin"}'
# 4. If approved, merge PR
docs/PLUGIN_STORE_GUIDE.md
+4 -5
View File
@@ -131,13 +131,13 @@ else:
**Via REST API:**
```bash
# Search by query
curl "http://your-pi-ip:5000/api/v3/plugins/store/search?q=hockey"
curl "http://your-pi-ip:5000/api/v3/plugins/store/list?query=hockey"
# Filter by category
curl "http://your-pi-ip:5000/api/v3/plugins/store/search?category=sports"
curl "http://your-pi-ip:5000/api/v3/plugins/store/list?category=sports"
# Filter by tags
curl "http://your-pi-ip:5000/api/v3/plugins/store/search?tags=nhl&tags=hockey"
curl "http://your-pi-ip:5000/api/v3/plugins/store/list?tags=nhl&tags=hockey"
```
**Via Python:**
@@ -351,8 +351,7 @@ All API endpoints return JSON with this structure:
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/v3/plugins/store/list` | List all plugins in store |
| GET | `/api/v3/plugins/store/search` | Search for plugins |
| GET | `/api/v3/plugins/store/list` | List plugins in store; `?query=`, `?category=`, `?tags=` search and filter |
| GET | `/api/v3/plugins/installed` | List installed plugins |
| POST | `/api/v3/plugins/install` | Install from registry |
| POST | `/api/v3/plugins/install-from-url` | Install from GitHub URL |
docs/REST_API_REFERENCE.md
+832 -380
View File
File diff suppressed because it is too large Load Diff
docs/SCROLL_PERFORMANCE.md
+46 -12
View File
@@ -83,8 +83,9 @@ second refresh, instead of half a pixel every refresh (which has no good
rendering, only a choice between blur and judder).
`scroll_config.configure()` snaps the requested speed to the nearest entry on
the ladder and reports the hold that speed needs. It does **not** apply the
hold: the hold belongs to a scroll, not to a plugin's lifetime, and plugins
the ladder, sets the helper to advance that entry's whole-pixel step on every
presented frame (`ScrollHelper.set_pixels_per_frame`), and reports the hold
that speed needs. It does **not** apply the hold: the hold belongs to a scroll, not to a plugin's lifetime, and plugins
share one display manager -- one set at construction is reset the moment any
other plugin finishes scrolling. Apply it yourself when the scroll starts:
@@ -102,10 +103,18 @@ self.display_manager.set_scrolling_state(True, frame_hold=settings.frame_hold)
Passing `display_manager` only lets `configure` read the true refresh rate from
`display.hardware`, which a plugin config cannot see. Skipping the
`set_scrolling_state` call is the mistake that matters: the speed still
resolves, but the panel keeps presenting a new frame every refresh, so a slow
snapped speed falls back to fractional pixels. Pass `snap_to_crisp=False` to
keep an exact requested speed and accept the artefacts.
`set_scrolling_state(True, frame_hold=...)` call is the mistake that matters.
The helper consults no clock in this mode -- it moves the fixed step once per
`update_scroll_position()` call, and `SwapOnVSync` is what paces those calls --
so without the hold the panel presents a new frame every refresh and the scroll
runs `frame_hold` times too fast: 50 px/s (hold 2) plays at 100 px/s.
Pass `snap_to_crisp=False` to keep an exact requested speed and accept the
artefacts. The helper then paces off elapsed time instead of stepping, and the
hold is 1.
The General tab's `target_fps` ("Scroll Frame Rate") plays no part in any of
this: frames are presented at the panel refresh divided by the hold.
Speeds slower than about 20 px/s are stepped no matter what, because a 1-pixel
advance at 20 fps is simply a coarse increment. That is the pixel pitch, not a
@@ -123,9 +132,12 @@ settings = scroll_config.configure(
self.scroll_helper,
plugin_config=self.config,
global_config=self.global_config,
refresh_hz=scroll_config.refresh_hz_from_config(self.global_config),
display_manager=self.display_manager,
plugin_logger=self.logger,
)
# each frame of a scroll (or at least when it starts):
self.display_manager.set_scrolling_state(True, frame_hold=settings.frame_hold)
```
It resolves every config shape in one place, applies the speed, and returns
@@ -154,6 +166,14 @@ it is always present and always wins, so the documented settings become
unreachable. That is a real, shipped bug — see
[ledmatrix-plugins#408](https://github.com/ChuckBuilds/ledmatrix-plugins/issues/408).
The flip side: a `scroll_pixels_per_second` you add by hand is ignored whenever
the plugin's config also carries the pair, which it does whenever the pair has
a schema default. Set the speed through the pair instead.
The sports scoreboards (`src.common.sports_scroll`) are the exception to all of
the above: they read `scroll_settings.scroll_speed` per league as px/s directly,
and their `scroll_delay` is kept for compatibility but ignored for pacing.
If you are writing a plugin: do not give a deprecated key a schema default.
## What was actually wrong
@@ -198,8 +218,16 @@ zero pixels and rendered an identical frame, which dirty-tracking skipped, so
it returned in ~2 ms and the beat repeated. No `scroll_delay` value tunes this
out — a shorter delay just trades stalled frames for periodic double-steps.
`ScrollHelper` now accumulates elapsed time in both modes at the same
configured speed, so position stays proportional to real time.
A crisp speed configured through `scroll_config` no longer consults a clock at
all. Once `SwapOnVSync` blocks until the panel has taken the frame, the frame
count is a truer clock than `time.time()`, so the helper advances a fixed whole
number of pixels per presented frame (`set_pixels_per_frame`) and the display
manager holds each frame for `frame_hold` refreshes. Every frame moves the eye
by the same amount.
The time-based path remains only for callers that set a speed directly or pass
`snap_to_crisp=False`. There, frame-based mode no longer steps either: it
advances by elapsed time at `scroll_speed / scroll_delay` px/s.
## Diagnosing a juddery scroller
@@ -222,11 +250,17 @@ p95 10.11ms max 12.03ms min 7.98ms | stalls 0 (0.0%) skips 0 (0.0%)
Reading it, on a 100 Hz panel:
A healthy median is the refresh period times the scroll's frame hold: 10 ms
for a hold of 1 (100 px/s), **20 ms for 50 px/s** (hold 2), 30 ms for 33.3 px/s.
A 20 ms median on a 50 px/s scroll is the hold doing its job, not missed
refreshes. The `Scroll configured:` log line gives the hold (`1px every 2
refreshes`).
| you see | it means |
|---|---|
| median 10 ms, p95 within ~0.5 ms of it | healthy — locked to the panel |
| p95 or max at 20/30/50 ms | frames missing refreshes — per-frame work is overrunning, or a background thread is holding the GIL |
| non-zero **skips**, or a median *below* 10 ms | **duplicate frames** — the swap was skipped because the image did not change, so the frame never waited on vsync. The scroller is advancing less than one pixel per frame. |
| median = refresh period × hold, p95 within ~0.5 ms of it | healthy — locked to the panel |
| p95 or max a whole refresh period or more above that median | frames missing refreshes — per-frame work is overrunning, or a background thread is holding the GIL |
| non-zero **skips**, or a median *below* the expected one | **duplicate frames** — the swap was skipped because the image did not change, so the frame never waited on vsync. The scroller is advancing less than one pixel per frame, which a crisp fixed-step scroll never does; look for a plugin pacing off time or not passing the hold. |
| non-zero **stalls** | frames past 1.5× the median, which is the measure of judder that survives averaging |
`stalls` and `skips` are both counted against that window's own median, so they
docs/SPORTS_UNIFICATION.md
+11 -2
View File
@@ -218,6 +218,14 @@ The one behavior the upstreamed version adds is native
Part A threaded it through each copy by hand, and this makes that threading
legacy compatibility rather than the mechanism.
> **Superseded.** Once presentation became frame-locked (#545) the helper
> steps a fixed whole-pixel amount per presented frame and the panel presents
> at its own refresh, so honouring `target_fps` only turned it into a speed
> multiplier (60 doubled a scoreboard's speed, 200 halved it). `sports_scroll`
> no longer reads it: the crisp-speed ladder uses the panel refresh
> (`display_manager.refresh_hz`), and speed comes from
> `scroll_settings.scroll_speed` alone. See `docs/SCROLL_PERFORMANCE.md`.
## Phases
B0–B3 are merged and shipping in core 3.2.0. Everything that remains is
@@ -464,8 +472,9 @@ After adoption plus the frozen legacy copies it was 10,610; removing the dead
inline duplication (plugins #252) brought it to roughly 8,620. B6 would take it
to about 3,300 including the shared core module — some 2,400 fewer than before
this project started. **Until B6 runs, the adoption is net negative on disk**,
and its one delivered user-visible gain is that adopted plugins honour the
global `target_fps` instead of hardcoding ~100 FPS.
and its one delivered user-visible gain was that adopted plugins honoured the
global `target_fps` instead of hardcoding ~100 FPS (since withdrawn: see the
note under the B3 design above).
### Decision: stop adopting further modules until B6 closes
docs/SSH_UNAVAILABLE_AFTER_INSTALL.md
+3 -1
View File
@@ -158,9 +158,11 @@ This script will check:
- Python dependencies
- Configuration files
- File permissions
- Web interface availability
- Web interface availability (`ledmatrix-web` listening on port 5000)
- Network connectivity
Once it passes, the web interface is at `http://<pi-ip>:5000`.
## Quick Reference Commands
```bash
docs/TROUBLESHOOTING.md
+2 -2
View File
@@ -273,7 +273,7 @@ sudo systemctl cat ledmatrix-web | grep User
1. **Verify file structure:**
```bash
ls -l web_interface/app.py
ls -l web_interface/blueprints/api_v3.py
ls -ld web_interface/blueprints/api_v3/
ls -l web_interface/blueprints/pages_v3.py
```
@@ -531,7 +531,7 @@ sudo systemctl cat ledmatrix-web | grep User
```bash
# Clear the cache with the helper script
sudo python3 scripts/utils/clear_cache.py
sudo python3 scripts/utils/clear_cache.py --clear-all
# Or remove files manually from the cache dir in use, e.g.:
sudo rm -rf /var/cache/ledmatrix/*
docs/WEB_INTERFACE_GUIDE.md
+10 -9
View File
@@ -102,7 +102,9 @@ Configure basic system settings:
check are shown under the toggle, and anything other than success raises a
banner on **Overview**.
- *Checks first:* the code update is skipped, with the reason shown, if
tracked files were edited locally, the checkout has local commits, a
tracked files were edited locally (permission-only changes and edits under
`plugins/` or `plugin-repos/` don't count; the pull carries those across
and puts them back), the checkout has local commits, a
rebase/merge is in progress, the branch has no upstream, less than 300 MB
is free, or the newest version already failed once. A failed fetch is
retried the next day.
@@ -206,11 +208,11 @@ Manage fonts for your display:
- See font previews
- Check font sizes and styles
**Font Overrides:**
- Overrides are set per display *element* (e.g. a specific score or
clock text element), not per plugin
- Override default font choices for individual elements
- Preview font changes
**Font Preview:**
- Render sample text in any TTF/OTF font at a chosen size
Fonts used by a plugin are chosen in that plugin's own settings tab; the
Fonts tab has no per-element override editor.
**Delete Fonts:**
- Remove unused fonts
@@ -327,9 +329,8 @@ The web interface is built on a REST API that you can access programmatically:
http://your-pi-ip:5000/api/v3
```
The API blueprint mounts at `/api/v3` (see
`web_interface/app.py:199`). All endpoints below are relative to that
base.
The API blueprint (`web_interface/blueprints/api_v3/`) is registered at
`/api/v3` in `web_interface/app.py`.
**Common Endpoints:**
- `GET /api/v3/config/main` — Get main configuration
docs/plugin-safety-harness.md
+2 -1
View File
@@ -10,7 +10,8 @@ plugin without breaking a size or screen you didn't think to test.
There is **no fixed set of supported panel sizes** — an RGB matrix build can be
any width/height and configuration (square, rectangle, 2×2, 4×4, 8×2, long
strips, tall stacks). Plugins are expected to read dimensions dynamically
(`self.display_manager.matrix.width/height`) and lay themselves out
(`self.display_manager.width/height` — not `matrix.width/height`, since
`matrix` is `None` when hardware init fails) and lay themselves out
accordingly, so a hardcoded coordinate or unscaled font shows up as a failure
here.