rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: September 16 core audit — partial saves, asset path safety, auto-update, display settings the library refuses, scroll speed (#595)

Browse Source 
* fix(sports): share the ESPN rejected-range memo with the background service

BackgroundDataService always sent a season range first and, on a 400,
fell back to chunks without recording the rejection, so every background
season fetch spent a doomed request and live scoreboards learned nothing
from it (or it from them). The worker now consults and sets the same
6-hour memo fetch_espn_scoreboard() uses: a known rejection goes straight
to month/day chunks, and if every chunk fails the range is asked once for
a real error without re-spending the chunks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): keep plugin asset and action routes inside their directories

POST /plugins/assets/upload, GET /plugins/assets/list and POST
/plugins/assets/delete joined the request's plugin_id onto assets/plugins
unchecked, so '../../config' created, wrote, listed and deleted outside
it. #561 guarded only the route that serves the files. All three now go
through path_safety.resolve_under and answer 400 for anything but a
plain name, and delete only unlinks a metadata path that resolves into
that plugin's uploads directory.

PluginManager.get_plugin_directory refuses ids that are not one plain
path segment, so /plugins/action (which runs a manifest script from the
returned directory) and every other caller get the guard; the action
route also rejects such ids up front, covering its no-manager fallback.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): report a no-op plugin update as already up to date

update_plugin() returns True both for a real update and for "nothing to
do" (a ZIP-installed monorepo plugin already at the registry version, a
bundled plugin). With no git commit to compare, POST /plugins/update
called every such success "updated successfully", so Check & Update All
counted most official plugins as updated on every run.

The route now reads what changed off the plugin itself (commit, else
manifest version, else last_updated) and returns data.update_status
(updated / up_to_date / local_only). The update-all toast is summarised
by PluginInstallManager.summarizeUpdateResults from that status, falling
back to the message for older servers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(sports): scoreboard scroll speed no longer follows target_fps

sports_scroll computed the crisp speed ladder against the global
target_fps whenever limit_refresh_rate_hz was the 100 Hz default. Since
frame-locked presentation (#545) the helper steps a fixed number of whole
pixels per presented frame and the panel presents at its real refresh, so
the General tab's "Scroll Frame Rate" became a speed multiplier: 60 ran a
50 px/s scoreboard at 100 px/s, 200 ran it at 25 px/s.

The ladder now uses the display manager's refresh_hz, then
display.hardware.limit_refresh_rate_hz, then the default. target_fps is
not consulted. Docstrings now say scroll_delay is ignored for pacing (no
behaviour change there) and describe the fixed-step model.

Tests: replace the tests that pinned target_fps as the ladder refresh and
described time-based stepping; assert speed independence from target_fps
(unit and end-to-end presented px/s against the real helper), that the
fixed per-frame step is applied, and that scroll_delay does not change
speed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): escape registry and upload values in plugin manager inline handlers

The store, saved-repository and custom-registry buttons built
onclick='...(${JSON.stringify(id)})...'. JSON.stringify leaves ' alone,
so a custom registry entry whose id contained ' closed the attribute and
added its own handler. One helper, jsStringAttr(), now HTML-escapes the
JSON literal for every one of those handlers, and the store View button
opens only http(s) repo links.

The live window.updateImageList (plugins_manager.js loads last, so its
copy wins over the file-upload widget's) wrote the uploaded file's
original name, path and ids into markup raw; they are escaped now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note plugin asset, action and inline handler guards

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): let the root pip wrapper install web_interface/requirements.txt

Update Code, the automatic update's health check and Install Base
Requirements install web_interface/requirements.txt through
safe_pip_install.sh, which only allowed the root requirements.txt. The
first commit changing that file would fail its dependency install, and
the automatic updater rolls back any update whose dependencies did not
install -- on every device, for every newer commit.

The wrapper now lists both core requirement files. Only their folders
are resolved, so a requirements.txt symlinked out of the project is
compared by its target and refused (previously the root file's own
symlink target was what got allowed). The updater's file list is a
named constant, and a test runs the real wrapper (pip stubbed) on
every file Update Code and the rollback install.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): do not retry plugin requests that got an HTTP answer

PluginAPI.request wrapped everything that was not a structured error as
NETWORK_ERROR: a proxy's 502 HTML page (response.json() throws) and a
JSON error without error_code included. Check & Update All retries
NETWORK_ERROR, so those updates were re-sent five more times with
backoff, contrary to the #587 contract that an HTTP error response is
the server's answer.

NETWORK_ERROR now means only that fetch() rejected. Any HTTP response
without an error_code, or with a body that is not JSON, is API_ERROR
with the HTTP status attached. Tested against the shipped api_client.js.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scroll): restart the stats window when an idle gap is dropped by size

#582 dropped an idle gap from the frame stats two ways: the reset_scroll()
sentinel, which also restarts the 5s window timer, and a size guard for
scrollers that never call reset_scroll(), which did not. On that path the
first real frame after the gap found the boundary overdue and logged a
stats line for a one-frame window. Both paths now share one seeding helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): leave plugins alone when update_core's own rollback fails

update_core returns rollback_failed directly when a partial pull or an
update whose health check never started cannot be rolled back. run()
only held plugins back for 'verifying', so those devices still got new
plugin versions and a display restart on top of a core in an unknown
state -- the opposite of what the health-check path does, and of the
3.4.0 changelog (plugins are left alone if the rollback fails).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(api): make the REST reference match the api_v3 package

Every documented request body, query parameter and response shape was
re-checked against the handlers in web_interface/blueprints/api_v3/.
Fixes calls that failed as documented (repo_url, action_id/params,
files/image_id, font_file+font_family, ?font=, cache key,
auto_enable_ap_mode, plugin limit keys), removes the font-override
endpoints dropped in #566, corrects response shapes (plugins/config,
plugins/schema, health, metrics, operation history, github-status,
fonts/catalog, cache/list, logs, wifi, on-demand, SSE streams), and adds
the 26 routes it omitted (backup, system auto-update/git, wifi radio,
starlark editor, MQTT bridge, status endpoints, skins).

Documents the merge semantics of partial JSON saves to /config/main and
/plugins/config and the dim-schedule POST accepting GET's days shape,
which land in the same change set. Replaces app.py line numbers and the
removed api_v3.py path with file and function names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): remove the General-tab plugin system toggles that did nothing

plugin_system.auto_discover, auto_load_enabled and development_mode had
General-tab toggles whose help tips promised dormant plugins and verbose
logging, but nothing reads them: every enabled plugin is discovered and
loaded regardless. Remove the three toggles.

The keys stay tolerated in stored configs. The save handler now stores
a flag only when a client sends it; treating a missing key as an
unchecked box would otherwise rewrite all three to false on every
General-tab save, which still posts plugins_directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(scroll): remove dead code left by #523/#570

- Drop the optional scipy.ndimage import and HAS_SCIPY; nothing read
  them since the numpy blend replaced the scipy path.
- Drop ScrollHelper._last_integer_position and frame_time_target, which
  were written but never read.
- Keep target_fps and set_target_fps() but document them as
  informational: nothing paces off them, yet ledmatrix-elections'
  test_scroll_pacing.py reads helper.target_fps back and third-party
  plugins may call the setter.
- Fix stale comments: fixed_pixels_per_frame's "use scroll_delay to
  throttle", set_sub_pixel_scrolling's "default: True", and
  set_frame_based_scrolling's claim that it steps.

The plugins monorepo was grepped for every removed name; none is used.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(fonts): point plugins at plugin_manager.font_manager; drop removed overrides UI

FONT_MANAGER.md told plugins to read display_manager.font_manager, which
does not exist, so a plugin following it failed to load with
AttributeError. The shared FontManager lives on the PluginManager and
BasePlugin._get_font_manager() returns it (with a fallback for harnesses).

Also removes the Fonts-tab override workflow and element-override panels
that #566 deleted, from FONT_MANAGER.md and WEB_INTERFACE_GUIDE.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(store): search via /plugins/store/list?query=; send Content-Type on registry curls

/plugins/store/search does not exist (404) and the list endpoint reads
query, not q. The registry guide's curl examples omitted the JSON
Content-Type, so the handlers saw an empty body and answered 400.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(config): use the shared core-key list in the last three private copies

StartupValidator warned "Plugin 'auto_update' is enabled but not found" on
every display start with auto-update or a dim schedule on; the reserved
plugin-id check missed auto_update, sync, location and the rest; and
ConfigManager's (uncalled) orphan cleanup would have deleted display,
schedule and auto_update. All three now read src/core_config_keys.py, which
also gains CORE_SECRETS_KEYS for the github/youtube secrets sections.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): partial JSON saves to /config/main change only what they send

A JSON body with one field reset every checkbox in the sections it touched:
the MQTT bridge's brightness slider turned off disable_hardware_pulsing,
inverse_colors, show_refresh_rate and use_short_date_format, and a
timezone-only save turned off web-UI autostart and weekly auto-updates.
Missing-means-unchecked now applies only to form posts: form-encoded bodies
and the v3 forms, which mark themselves with a hidden __form_section input.

Also on the config routes:
- vegas_min/max_cycle_duration no longer match the generic *_duration rule,
  so they stop landing in display_durations and a blank one no longer
  rejects the whole Display save;
- saving from the Raw JSON editor calls start_setup_if_needed like the
  General form, so enabling auto-update there finishes its setup;
- the schedule and dim-schedule POSTs accept the per-day days.<day> shape
  their GETs return, as well as the flat form keys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): install plugin dependencies from the configured plugins directory

install_plugin_dependencies.sh scanned only plugins/, but the Plugin
Store installs into plugin_system.plugins_directory (default
plugin-repos), so the documented "Recommended" fix found 0 plugins on
every store install. It now reads plugins_directory from
config/config.json (relative to the project root or absolute, default
plugin-repos) and also scans plugins/ for dev symlinks, installing a
plugin reached through both only once.

With set -e alone, `pip ... | tee` took tee's exit status, so a failed
pip install was reported as success; set -o pipefail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: replace stale API names, line numbers and the api_v3.py path

- ADVANCED_FEATURES: StreamManager methods that exist
  (get_next_segment, take_next_group, refresh, advance_cycle, ...), and the
  real on-demand status envelope ({status, data: {state, service}})
- app.py:199 / :144 / :607-619 line citations and
  web_interface/blueprints/api_v3.py (now a package) replaced with file and
  function names in ADVANCED_FEATURES, CONFIG_DEBUGGING,
  PLUGIN_ARCHITECTURE_SPEC, PLUGIN_QUICK_REFERENCE,
  PLUGIN_CONFIGURATION_TABS, TROUBLESHOOTING and web_interface/README
- CONFIG_DEBUGGING: partial /config/main saves change only sent keys; use
  /config/raw/main to replace the file; describe where validation runs
- TROUBLESHOOTING: clear_cache.py needs --clear-all (no args only prints
  usage)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): verify the web interface that actually ships, on port 5000

verify_installation.sh failed every healthy install: it required the
long-removed web_interface_v2.py and looked for a listener on port 5001,
while the web interface binds 5000 (web_interface/start.py). It now
checks the files ledmatrix-web.service runs (start_web_conditionally.py,
web_interface/start.py, app.py) and port 5000. verify_web_ui.sh had the
same 5001 port in its listen check, HTTP probe and printed URLs.

Port matches are anchored so :50001 no longer counts as :5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(plugins): one display-size contract: display_manager.width/height

CLAUDE.md (#580) says to read display_manager.width/height because
matrix is None when hardware init fails; the development guide, the
safety-harness doc and two DisplayManager docstrings still recommended
matrix.width/height. The bundled starlark-apps plugin read matrix.width
unguarded, so its magnify recommendation and frame scaling raised in
fallback mode (e.g. after the Pi 5 hardware refusal).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(install): make install_service.sh --help print usage instead of installing

install_service.sh parsed no arguments, so `sudo ./scripts/install/
install_service.sh --help` (presented as harmless in MIGRATION_GUIDE.md)
rewrote ledmatrix.service, ledmatrix-web.service and both update-verify
units and enabled/started them. It now handles -h/--help (usage, exit 0,
no changes) and rejects any other argument with exit 2 before doing
anything. Running it with no arguments, as first_time_install.sh does,
is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scroll): describe the fixed-step model and document frame_hold

Since #545 a crisp speed from scroll_config.configure() makes the helper
advance a fixed whole-pixel step per presented frame with no clock, and the
display manager's frame hold is part of the speed. The docs still described
the removed wall-clock model:

- scroll_config's module and configure() docstrings said speed is applied
  in time-based mode and that omitting the hold "falls back to fractional
  pixels"; omitting it actually runs the scroll frame_hold times too fast.
- SCROLL_PERFORMANCE.md said ScrollHelper accumulates elapsed time in both
  modes, and read a 20 ms stats median as missed refreshes although that
  is a healthy 50 px/s (hold 2) scroll. It now explains the fixed step,
  the hold-dependent healthy median, that target_fps plays no part, and
  that a hand-added scroll_pixels_per_second loses to a schema-default pair.
- PLUGIN_API_REFERENCE.md documented set_scrolling_state(is_scrolling)
  without frame_hold; it now documents the parameter (core 3.4.0) with a
  configure() + set_scrolling_state example.
- update_scroll_position/set_scroll_speed and set_scrolling_state
  docstrings say the same.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark target_fps legacy; describe what Vegas scroll_delay does

- General tab "Scroll Frame Rate" (target_fps) is labelled legacy: after
  the sports_scroll fix nothing in core scrolling reads it. The field and
  its API validation stay so saved configs and plugins that read
  global_config['target_fps'] keep working. CONFIG_REFERENCE says the same.
- Vegas frame_based_scrolling/scroll_delay were described as frame-count
  stepping at ~50 FPS. Neither steps nor sets a frame rate: frame-based
  mode converts the speed to px per scroll_delay, clamps it to 0.1-5, and
  still advances by elapsed time, so the applied speed is
  clamp(scroll_speed * scroll_delay, 0.1, 5) / scroll_delay px/s. The
  config comments, render_pipeline comment and CONFIG_REFERENCE rows now
  say so. No behaviour change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(deps): describe how plugin dependencies are really installed

The guides said the web service runs as root, that installs pick --user
from os.geteuid(), and quoted a warning and a
PluginManager._install_plugin_dependencies() method that don't exist. The
web unit runs as the installing user; store installs go through
install_requirements_file() and sudo safe_pip_install.sh (root), with a
user-level fallback that says so, and load-time installs run in the
display service's own (root) interpreter.

Manual paths now use the configured plugins directory (plugin-repos/ by
default) instead of plugins/, which store installs no longer use, and
install_plugin_dependencies.sh is described as scanning that directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): count local changes one way for the preflight and the pull

The automatic update's preflight ignored mode-only changes and anything
whose status line contained plugins/ or plugin-repos/, then promised
"Automatic updates will not stash your changes". perform_core_update
used plain git status (modes count) and ignored only 'plugins/', then
ran 'git stash push -- :!plugins', which nothing ever pops. So an edit
to a bundled plugin under plugin-repos/, or the installer's chmods on
tracked scripts, passed the preflight and was stashed away for good.

- auto_update.local_changes() is the one predicate both use:
  core.fileMode=false, porcelain -z, and plugins/ and plugin-repos/
  excluded by leading folder rather than substring (a core file under
  web_interface/static/v3/js/plugins/ now counts).
- Update Code's explicit stash leaves out both plugin folders; the
  pull's --autostash carries their edits and mode changes across and
  reapplies them.
- The automatic updater calls perform_core_update(stash_local_changes=
  False), which refuses instead of stashing edits that appeared after
  the preflight; update_core reports that as 'blocked'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): diagnostics follow the web autostart default and api_v3 package

#556 made a missing web_display_autostart mean "start" (only an explicit
false/off keeps the web interface down), but the diagnostics still said
otherwise: diagnose_web_ui.sh reported a missing key as "defaults to
false", diagnose_web_interface.sh said the web interface "will not start
unless this is set to true" and recommended enabling it, and
debug_web_manual.py printed False. Troubleshooting a down web UI pointed
users at a non-cause.

Both shell scripts now evaluate the setting with the launcher's own
autostart_enabled() (inline fallback if it cannot be imported) and report
on / off / not set (on) / unparseable config; debug_web_manual.py uses
the same function. They also check web_interface/blueprints/api_v3/
__init__.py: api_v3.py became a package in #553, so every healthy
checkout was reported as missing a file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(install): what install_service.sh installs; verify script port; no sudo for --help

install_service.sh installs and starts ledmatrix, ledmatrix-web and the
update-verify units, not only ledmatrix.service (systemd/README.md,
README.md). MIGRATION_GUIDE presented 'sudo install_service.sh --help'
as a harmless check; it now shows --help without sudo and warns what a
real run does. SSH_UNAVAILABLE_AFTER_INSTALL: verify_installation.sh
checks the web interface on port 5000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): note update-all, plugin system settings and script fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): size the preview after orientation and pixel mappers

display_geometry.physical_size claimed to give DisplayManager's answer but
only computed cols*chain x rows*parallel. RGBMatrix.width/height are measured
after the library's pixel mappers, so a Rotate:90 / orientation 90 chain
previewed 128x32 for a 32x128 panel and a U-mapper chain of four 256x32 for
128x64.

Model the built-in mappers' size effect as the pinned lib/pixel-mapper.cc
does (Rotate, U-mapper, V-mapper, StackToRow, Remap; Mirror and unknown
names leave it alone), and move the orientation composition here so
DisplayManager and the preview share it. The module docstring no longer
claims the sync handshake uses it; that imports only DEFAULT_CHAIN_LENGTH.

Audit finding F18.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(display): refuse settings the rgbmatrix library aborts on, on every board

The library answers several settings with a NULL matrix or abort() rather
than an error, so the display service crash-looped (Restart=on-failure)
instead of reaching fallback mode: rows above 64, chain_length above 255
(uint8_t binding setter, documented as "no upper limit"), a misspelled
hardware_mapping, and parallel 2-3 on a single-output mapping, reachable
from the Display form on the default adafruit-hat(-pwm) mapping. #586 only
guarded the Pi 5 subset.

- src/matrix_support.py holds the rules for every board (Options::Validate
  ranges, binding integer types, mapping names and outputs from
  lib/hardware-mapping.c) plus the Pi 5 ones, and is the one source of the
  API's numeric ranges.
- DisplayManager checks them before building options and raises
  MatrixSettingsRefused, so a hand-edited config falls back with a logged,
  reported reason. Emulator mode only warns.
- The config API refuses them with a 400 naming the setting; combinations
  are checked against stored values but reported only when the request
  sets a field involved.
- The hardware status file gains "cause" (settings/library/forced). The
  fallback log and Display banner give the Pi 5 rebuild hint only for a
  library failure instead of rebuild + gpio_slowdown advice for every
  failure; one Pi 5 slowdown recommendation (1-3, start at 1).
- The Display form offers classic/classic-pi1 and orientation 90/270 and
  renders any other stored mapping selected with a warning, so an
  unrelated save no longer rewrites them; the API accepts 90/270.

Audit findings F03, F16, F19, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(display): library limits, template defaults and Pi 5 slowdown

- rows 8-64, chain_length 1-255, parallel limited by the mapping's outputs,
  classic/classic-pi1 mappings and orientation 90/270 documented.
- Defaults are the config.template.json values: config migration adds
  missing keys from the template, so the listed "code defaults" never
  applied.
- One Raspberry Pi 5 gpio_slowdown recommendation: 1-3 in PIO mode,
  starting at 1.
- Troubleshooting describes the refused-settings fallback, and CHANGELOG
  corrects the Unreleased "no upper limit" entry.

Audit findings F19, F20, F21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py opens the panel with the service's options

--measure and --demo built RGBMatrixOptions from a private copy of the
display service's builder that had drifted: gpio_slowdown came from
display.hardware (default 2) instead of display.runtime (default 3), and
rp1_rio, panel_type, disable_hardware_pulsing, inverse_colors,
pixel_mapper_config and orientation were skipped, with different defaults
(hardware_mapping "regular", pwm_bits 11). A panel needing a high slowdown
was measured -- or garbled -- in a setup the service never drives.

The option filling in DisplayManager._setup_matrix moves, unchanged, into
DisplayManager.apply_matrix_options(options, config), which _setup_matrix
calls and the script reuses (overriding only limit_refresh_rate_hz for
--measure). The script now loads the whole config rather than the hardware
block. Tests pin the script's options to the service's attribute for
attribute.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(scripts): scroll_speeds.py recommends keys the resolver honours

The ladder ended by telling users to set
display_options.scroll_pixels_per_second. scroll_config ranks that key
below the scroll_speed + scroll_delay pair, deliberately, and several
plugin schemas default the pair into config, so the advised key was
silently ignored (a schema-default 1/0.02 pair plus an advised 66 still
resolved to 50 px/s).

The advice is now the pair that selects the crisp speed exactly
(pixels_per_frame every frame_hold/refresh seconds), explains that the
pair outranks scroll_pixels_per_second, and gives the scoreboards'
per-league scroll_settings.scroll_speed (px/s) form. Tests resolve the
printed pair over a schema-default pair and check it lands on the
advertised speed and hold.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: withdraw the target_fps claim for sports_scroll; fix the Vegas speed formula

- SPORTS_UNIFICATION.md still presented honouring global target_fps as
  sports_scroll's added behaviour and its one user-visible gain; note that
  it was withdrawn because it had become a speed multiplier.
- ADVANCED_FEATURES.md gave Vegas scrolling as
  (scroll_speed / target_fps) * elapsed; the real rule is scroll_speed px/s
  by elapsed time, through a 0.1-5 px per scroll_delay clamp when
  frame_based_scrolling is on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): scroll model fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(dev): link-github links plugins from the ledmatrix-plugins monorepo

link-github <name> cloned https://github.com/ChuckBuilds/ledmatrix-<name>.git,
and those per-plugin repositories no longer exist: official plugins are
directories in the ledmatrix-plugins monorepo. It now clones (or pulls) the
monorepo once into the dev directory, finds plugins/<name>,
plugins/ledmatrix-<name> or the plugin whose manifest id is <name>, and
links it under its manifest id. With an explicit repo URL it still links a
single-repository plugin as before.

dev_plugins.json: github_user is honoured again (monorepo owner, e.g. a
fork), plus plugins_repo and plugins_branch; github_pattern, which was
documented but never read, is dropped and warned about. Ships
dev_plugins.json.example and git-ignores dev_plugins.json, both of which
the guide promised. Reading JSON falls back to python3 when jq is missing
(get_plugin_id silently returned nothing without jq).

update/status/list find the git checkout above a monorepo plugin
directory (its .git is not in the plugin dir), and update pulls a shared
checkout once. status no longer exits 1 when nothing is broken.

Docs: PLUGIN_DEVELOPMENT_GUIDE (quick start, link-github, configuration,
workflow, store integration, hello-world link, submission), and the
nonexistent scripts/git-hooks/pre-push-plugin-version and
scripts/bump_plugin_version.py replaced with the real rule: bump the
manifest version and run update_registry.py. scripts/dev/README.md and
CLAUDE.md updated to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(scripts): monorepo workspace layout; fix_perms and install READMEs

MULTI_ROOT_WORKSPACE_SETUP described one sibling repository per plugin;
setup_plugin_repos.py links ../ledmatrix-plugins/plugins/* into
plugin-repos/ and update_plugin_repos.py pulls only the monorepo, and the
workspace file opens LEDMatrix plus ../ledmatrix-plugins.

scripts/fix_perms/README.md listed cache directories
fix_cache_permissions.sh never touches and a 'ledmatrix' service user
that doesn't exist (also in scripts/install/README.md); adds
safe_pip_install.sh. install/README: install_service.sh installs the web
and update-verify units too.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): keep the rollback's pip retries inside the unit time limit

The health check reinstalled the previous requirements by trying the
next bash path after any failure, including a 600 s pip timeout. Two
files, two paths: up to 40 minutes of pip alone, while systemd stops
ledmatrix-update-verify.service at TimeoutStartSec=30min -- killing the
rollback half-way and leaving the update 'verifying' until the web UI
calls it lost.

- Like permission_utils.install_requirements_file, only a sudo refusal
  moves on to the next bash; a pip that ran and failed or timed out is
  not repeated. The refusal wording is one list
  (permission_utils.SUDO_REFUSAL_PHRASES), mirrored in the stdlib-only
  verifier and pinned equal by a test.
- All reinstalls in one rollback share a 600 s budget.
- WORST_CASE_SECONDS adds up every timeout on the longest path (27.5
  min); a test holds it under the unit's TimeoutStartSec and that under
  the web UI's VERIFY_LOST_SECONDS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(plugins): prepare plugin configs one way for load, saves, GET, hot reload and dev tools

Plugin config was prepared differently depending on how it arrived:

- JSON POST /plugins/config built a partial body on schema defaults, so
  {"enabled": true} reset every other setting of the plugin. It now merges
  onto the stored section first, as the form path already did.
- Legacy-boolean normalization (#588) ran only at load: GET /plugins/config
  returned the raw boolean, posting it back failed validation, and hot
  reload handed plugins the raw section (a legacy dynamic_duration: true
  came back as a boolean). schema_manager.prepare_plugin_config (normalize,
  then defaults) is now used by PluginManager.load_plugin, both save paths,
  GET, the save notifications and DisplayController's hot-reload callback.
- The JSON save's filter kept only enabled/display_duration/live_priority
  and dropped a submitted skin, skin_options or vegas_* tuning key. There
  is now one core-owned per-plugin list, schema_manager.CORE_PLUGIN_PROPERTIES,
  used by validation and by the save filter; PluginManager's
  CORE_OWNED_CONFIG_KEYS is its vegas subset.
- Plugin sections posted to /config/main were stored verbatim, including
  values /plugins/config rejects. They now go through the same preparation
  (_prepare_plugin_config_for_save, extracted from save_plugin_config), and
  a failing section rejects the whole save before anything is written.
- dev_server read only top-level defaults and let a schema enabled:false
  win; build_full_config shallow-merged overrides, dropping sibling
  defaults; the harness extracted defaults differently from the device.
  loading.build_config now uses the device's extraction and preparation,
  and dev_server, check_plugin, render_plugin and the harness all use it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt-bridge): brightness changes apply live and touch nothing else

The display service's hot reload applies a saved brightness within a few
seconds, and /config/main no longer resets other display settings on a
brightness-only JSON body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): automatic update hardening

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): rewrite PLUGIN_CONFIG_ARCHITECTURE for the v3 web UI

It described web_interface_v2.py and index_v2.html (both gone), client-side
form generation, one POST per field with {key, value}, and 'no nested
objects'. The v3 UI renders plugin forms server-side from the schema
(pages_v3 partial + plugin_config.html macros, nested sections and
x-widgets), posts the whole form once, and save_plugin_config() merges onto
the stored section, validates, splits x-secret fields and notifies the
plugin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(mqtt): brightness saves apply via hot reload and leave other settings alone

The bridge README said brightness is applied on the display's next
restart; the display controller's config hot reload applies it within
seconds. It also now states that the bridge's partial JSON save changes
only brightness (the /config/main merge fix in this change set).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(update): don't log pip's output from the health check's reinstall

pip can echo a private index URL with embedded credentials;
permission_utils redacts it, the stdlib-only verifier cannot, so it
logs the exit code only.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(config): mark the plugin_system toggles as unused legacy keys

auto_discover, auto_load_enabled and development_mode are read by
nothing and leave the General tab in this change set (F40). CONFIG_REFERENCE
said they were read by the plugin loader; PLUGIN_CONFIGURATION_GUIDE and
the REST reference listed them as live settings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): docs and developer tools group

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): legacy plugin-system toggles no longer count as a General save

auto_discover, auto_load_enabled and development_mode have left the General
form, so a post carrying only one of them is not a general-settings save and
must not treat web_display_autostart and auto_update as unchecked. The
plugin_system block itself is left as on main for the branch that reworks it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): config-save and plugin-config preparation fixes

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(claude): re-check matrix_support.py rules when the library submodule is bumped

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix: address Codacy findings on the core audit PR

- plugin_manager.prepare_plugin_config: when the fallback legacy-boolean
  pass also fails, log a warning instead of a bare except/pass.
- api_client.js: request() refuses any endpoint that is not a plain path
  under /api/v3 ("//host", backslashes, ".." or "." segments, whitespace,
  control characters) with INVALID_ENDPOINT before calling fetch(), and
  plugin ids are URL-encoded wherever they are put into a URL (also in the
  app-shell batch load).
- test_update_all.js: pins both against the shipped client.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(web): check endpoint control characters without a control-character regex

Codacy (ESLint no-control-regex, Biome noControlCharactersInRegex) flags
the \x00-\x1f range in checkEndpoint's regex. Test the char codes
instead; the endpoints refused are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(auto-update): make the seed script executable on disk, not only in the index

On Linux Repo.publish() commits with -a, which recorded scripts/run.sh
as 100644 upstream because the seed file was never chmod +x. The pull
then brought in the same mode the installer chmod had made locally, so
installer_chmod saw no mode change left to check. The updater was fine:
with the upstream commit at 100755 the --autostash carries the device's
chmod across. Verified under Linux (WSL, git 2.43): the old helper fails
exactly as CI did, the fixed one passes all 63 tests in the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-17 16:37:29 -04:00
committed by GitHub
co-authored by Claude Opus 5
parent 7e5967e160
commit 116abb0daa
101 changed files with 7244 additions and 2904 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/js/README.md
+2 -1
View File
@@ -35,11 +35,12 @@ nothing is listening, so it stays useful in a bare checkout.
| Suite | Needs a server | Covers |
|---|---|---|
| `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
| `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), and re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
| `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin, never re-sends one that got any HTTP answer (the real `api_client.js` classifies a proxy 502 or a JSON error without `error_code` as `API_ERROR`), and counts a no-op update as already up to date in the summary. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
| `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
| `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
| `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
| `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
| `unit/test_inline_handler_escaping.js` | no | The store, saved-repository and custom-registry inline `onclick` handlers and the live `window.updateImageList` from `plugins_manager.js`: a registry id, URL or uploaded file name carrying `'`, `"` or entities adds no attributes and reaches the handler intact, and the store's View button opens only http(s) links |
| `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
| `dom/test_no_double_fetch.js` | yes | Loads the **whole** `plugins_manager.js` and counts requests: typing in the store search must filter the cached list, not refetch `/api/v3/plugins/store/list` |
test/js/run_all.js
+1 -1
View File
@@ -18,7 +18,7 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
'unit/test_style_editor_layout_leaf_columns.js',
'unit/test_style_editor_layout_leaf_collision.js',
'unit/test_update_all.js'];
'unit/test_update_all.js', 'unit/test_inline_handler_escaping.js'];
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
'dom/test_tools_sections.js'];
test/js/unit/test_inline_handler_escaping.js
+215
View File
@@ -0,0 +1,215 @@
// Registry- and upload-supplied strings must not escape the inline handlers
// plugins_manager.js builds for them.
//
// The store, saved-repository and custom-registry renderers wrote
//
// <button onclick='... installFromCustomRegistry(${JSON.stringify(id)}) ...'>
//
// JSON.stringify makes a valid JS string but leaves `'` alone, so an entry id
// of x' onmouseover='alert(1) closed the single-quoted attribute and added a
// handler of its own. The live window.updateImageList (plugins_manager.js loads
// last, so its copy beats the file-upload widget's) put the uploaded file's
// original name into the markup unescaped.
//
// Each case renders with the shipped function, parses the tag the way a browser
// does (quoted attribute values, entities decoded), and checks two things: no
// attribute appeared that the template did not write, and the decoded handler
// runs and passes the hostile value through intact as data.
const fs = require('fs');
const path = require('path');
const SRC = fs.readFileSync(
path.resolve(__dirname, '../../../web_interface/static/v3/plugins_manager.js'), 'utf8');
let pass = 0, fail = 0;
const ok = (label, cond, extra) => cond
? (pass++, console.log(' ok ' + label))
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra).slice(0, 400) : '')));
function extract(opener) {
const start = SRC.indexOf(opener);
if (start < 0) { console.error('FAIL: cannot find ' + JSON.stringify(opener)); process.exit(1); }
let depth = 0;
for (let j = SRC.indexOf('{', start); j < SRC.length; j++) {
if (SRC[j] === '{') depth++;
else if (SRC[j] === '}' && --depth === 0) return SRC.slice(start, j + 1);
}
console.error('FAIL: unbalanced braces after ' + opener); process.exit(1);
}
// ── DOM shim ───────────────────────────────────────────────────────────────
class FakeEl {
constructor() { this.innerHTML = ''; this.value = ''; this.textContent = ''; }
}
class TextEl {
set textContent(v) { this._t = String(v == null ? '' : v); }
get innerHTML() {
return (this._t || '').replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
}
const els = {};
global.document = {
getElementById: id => (els[id] ||= new FakeEl()),
createElement: () => new TextEl(),
};
global.window = global;
global.pluginLog = () => {};
global.isStorePluginInstalled = () => false;
global.isNewPlugin = () => false;
global.formatDate = () => '';
global.setGridHtmlIfChanged = (container, html) => { container.innerHTML = html; };
// eslint-disable-next-line no-eval
eval([
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
'function renderPluginStore(plugins) {', 'function renderSavedRepositories(repositories) {',
'function renderCustomRegistryPlugins(plugins, registryUrl) {',
].map(extract).join('\n') + '\nglobal.jsStringAttr = jsStringAttr; global.escapeHtml = escapeHtml;'
+ '\nglobal.renderPluginStore = renderPluginStore; global.renderSavedRepositories = renderSavedRepositories;'
+ '\nglobal.renderCustomRegistryPlugins = renderCustomRegistryPlugins; global.escapeAttribute = escapeAttribute;');
// eslint-disable-next-line no-eval
eval(extract('window.updateImageList = function(fieldId, images) {'));
// ── minimal HTML start-tag tokenizer ───────────────────────────────────────
function decodeEntities(s) {
return s.replace(/&(#x[0-9a-f]+|#\d+|quot|amp|lt|gt|apos);/gi, (m, e) => {
const k = e.toLowerCase();
if (k === 'quot') return '"';
if (k === 'amp') return '&';
if (k === 'lt') return '<';
if (k === 'gt') return '>';
if (k === 'apos') return "'";
return String.fromCodePoint(k[1] === 'x' ? parseInt(k.slice(2), 16) : parseInt(k.slice(1), 10));
});
}
function tags(html, name) {
const out = [];
let i = 0;
while ((i = html.indexOf('<' + name, i)) >= 0) {
let j = i + name.length + 1;
const attrs = [];
for (;;) {
while (/\s/.test(html[j])) j++;
if (html[j] === '>' || j >= html.length) break;
if (html[j] === '/') { j++; continue; }
let n = '';
while (j < html.length && !/[\s=>]/.test(html[j])) n += html[j++];
let v = '';
while (/\s/.test(html[j])) j++;
if (html[j] === '=') {
j++;
while (/\s/.test(html[j])) j++;
const q = html[j];
if (q === '"' || q === "'") {
const end = html.indexOf(q, j + 1);
v = html.slice(j + 1, end); j = end + 1;
} else {
while (j < html.length && !/[\s>]/.test(html[j])) v += html[j++];
}
}
attrs.push([n.toLowerCase(), decodeEntities(v)]);
}
out.push(attrs);
i = j;
}
return out;
}
const names = attrs => attrs.map(a => a[0]);
const attr = (attrs, n) => (attrs.find(a => a[0] === n) || [])[1];
// Run a decoded inline handler with window/document stubs; return the calls.
function runHandler(code) {
const calls = [];
const win = new Proxy({}, {
get: (_, prop) => (...args) => { calls.push([prop, ...args]); },
has: () => true,
});
const doc = { getElementById: () => ({ value: '' }) };
// eslint-disable-next-line no-new-func
new Function('window', 'document', 'console', code)(win, doc, { error() {} });
return calls;
}
const SQ = "x' onmouseover='alert(1)";
const DQ = 'x" onmouseover="alert(1)';
const AMP = 'x&#39; onmouseover=&#39;alert(1)';
console.log('\n1. custom registry install button');
for (const hostile of [SQ, DQ, AMP]) {
renderCustomRegistryPlugins([{ id: hostile, name: 'n', plugin_path: hostile }], hostile);
const buttons = tags(els['custom-registry-grid'].innerHTML, 'button');
const b = buttons.find(a => attr(a, 'onclick'));
ok(`${JSON.stringify(hostile)}: only onclick and class attributes`,
b && names(b).join(',') === 'onclick,class', b && names(b));
let calls = [];
try { calls = runHandler(attr(b, 'onclick')); } catch (e) { calls = [['threw', String(e)]]; }
ok(`${JSON.stringify(hostile)}: handler passes id, url and path intact`,
calls.length === 1 && calls[0][0] === 'installFromCustomRegistry'
&& calls[0][1] === hostile && calls[0][2] === hostile && calls[0][3] === hostile, calls);
}
console.log('\n2. store install and view buttons');
for (const hostile of [SQ, DQ, AMP]) {
renderPluginStore([{ id: hostile, name: 'n', repo: 'https://github.com/o/r', plugin_path: 'p' }]);
const buttons = tags(els['plugin-store-grid'].innerHTML, 'button');
ok(`${JSON.stringify(hostile)}: two buttons, no extra attributes`,
buttons.length === 2 && buttons.every(b => names(b).every(n => ['onclick', 'class', 'disabled'].includes(n))),
buttons.map(names));
let calls = [];
try { calls = runHandler(attr(buttons[0], 'onclick')); } catch (e) { calls = [['threw', String(e)]]; }
ok(`${JSON.stringify(hostile)}: install handler gets the id intact`,
calls.length === 1 && calls[0][0] === 'installPlugin' && calls[0][1] === hostile, calls);
}
console.log('\n3. store view button only opens http(s) links');
renderPluginStore([{ id: 'a', name: 'n', repo: 'https://github.com/o/r', plugin_path: 'plugins/a' }]);
{
const view = tags(els['plugin-store-grid'].innerHTML, 'button')[1];
const calls = runHandler(attr(view, 'onclick'));
ok('https repo opens the plugin tree',
calls.length === 1 && calls[0][0] === 'open' && calls[0][1] === 'https://github.com/o/r/tree/main/plugins/a', calls);
}
renderPluginStore([{ id: 'a', name: 'n', repo: 'javascript:alert(document.domain)//' }]);
{
const view = tags(els['plugin-store-grid'].innerHTML, 'button')[1];
const calls = runHandler(attr(view, 'onclick'));
ok('javascript: repo opens nothing', calls.length === 0, calls);
ok('javascript: repo button is disabled', names(view).includes('disabled'), names(view));
}
console.log('\n4. saved repository remove button');
for (const hostile of [SQ, DQ, AMP]) {
renderSavedRepositories([{ url: hostile, name: hostile }]);
const b = tags(els['saved-repositories-list'].innerHTML, 'button')[0];
ok(`${JSON.stringify(hostile)}: no extra attributes`,
b && names(b).join(',') === 'onclick,class,title,aria-label', b && names(b));
let calls = [];
try { calls = runHandler(attr(b, 'onclick')); } catch (e) { calls = [['threw', String(e)]]; }
ok(`${JSON.stringify(hostile)}: handler gets the url intact`,
calls.length === 1 && calls[0][0] === 'removeSavedRepository' && calls[0][1] === hostile, calls);
}
console.log('\n5. live window.updateImageList escapes the uploaded file name');
window.getUploadConfig = () => ({ plugin_id: SQ });
window.currentPluginConfig = null;
{
const name = '<img src=x onerror=alert(1)>' + DQ + '.png';
window.updateImageList('f', [{ id: SQ, path: 'assets/x".png', filename: DQ, original_filename: name, size: 1 }]);
const html = els['f_image_list'].innerHTML;
ok('no raw markup from original_filename', !html.includes('<img src=x onerror'), html);
const imgs = tags(html, 'img');
ok('one <img>, only the template attributes',
imgs.length === 1 && names(imgs[0]).join(',') === 'src,alt,loading,decoding,class,onerror', imgs.map(names));
ok('alt carries the stored filename as text', attr(imgs[0], 'alt') === DQ, imgs[0]);
const buttons = tags(html, 'button');
ok('two buttons, no extra attributes',
buttons.length === 2 && buttons.every(b => names(b).join(',') === 'type,onclick,class,title,aria-label'),
buttons.map(names));
const del = runHandler(attr(buttons[1], 'onclick'));
ok('delete handler gets field, image and plugin ids intact',
del.length === 1 && del[0][0] === 'deleteUploadedImage' && del[0][1] === 'f' && del[0][2] === SQ && del[0][3] === SQ, del);
}
console.log(`\n${pass} passed, ${fail} failed\n`);
process.exit(fail ? 1 : 0);
test/js/unit/test_update_all.js
+83
View File
@@ -146,6 +146,89 @@ const noSleep = { sleep: async () => {} };
ok('...and is reported as a failure', results.find(x => x.pluginId === 'static-image').success === false);
}
console.log('\nthe real api_client.js: only a missing HTTP answer is retried');
{
// The fake PluginAPI above decides error_code itself. This runs the shipped
// client so its classification is what gets tested: a proxy's 502 page or
// a JSON 500 without error_code used to come back as NETWORK_ERROR and be
// re-sent five more times.
const PluginAPI = require(path.join(V3, 'js/plugins/api_client.js'));
const run = async (makeFetch) => {
let requests = 0, sleeps = 0;
global.fetch = async () => { requests++; return makeFetch(requests); };
global.window = { PluginAPI, installedPlugins: [{ id: 'stock-news' }] };
const results = await Manager.updateAll(null, { sleep: async () => { sleeps++; }, retryDelaysMs: [1, 1, 1] });
return { requests, sleeps, result: results[0] };
};
const httpAnswer = (status, json) => ({ ok: status < 400, status, json });
let r = await run(() => httpAnswer(502, async () => { throw new SyntaxError('Unexpected token <'); }));
ok('a 502 with an HTML body is sent once', r.requests === 1 && r.sleeps === 0, r);
ok('...and is an API_ERROR carrying the status, not NETWORK_ERROR',
r.result.success === false && r.result.error.error_code === 'API_ERROR' && r.result.error.status === 502, r.result.error);
r = await run(() => httpAnswer(500, async () => ({ status: 'error', message: 'boom' })));
ok('a JSON 500 without error_code is sent once', r.requests === 1 && r.sleeps === 0, r);
ok('...and keeps the server message', r.result.error.message === 'boom', r.result.error);
r = await run(() => httpAnswer(500, async () => ({ status: 'error', error_code: 'PLUGIN_UPDATE_FAILED', message: 'x' })));
ok('a structured error is passed through unchanged',
r.requests === 1 && r.result.error.error_code === 'PLUGIN_UPDATE_FAILED', r.result.error);
r = await run((n) => {
if (n === 1) throw new TypeError('Failed to fetch');
return httpAnswer(200, async () => ({ status: 'success', message: 'ok', data: { update_status: 'updated' } }));
});
ok('a fetch() that rejects is NETWORK_ERROR and re-sent', r.requests === 2 && r.sleeps === 1 && r.result.success, r);
r = await run(() => httpAnswer(200, async () => { throw new SyntaxError('Unexpected end of JSON input'); }));
ok('an unreadable 200 is not retried either', r.requests === 1 && r.result.error.error_code === 'API_ERROR', r);
// Every endpoint is one of the client's own API paths; one that could
// leave baseURL never reaches fetch(), and plugin ids are encoded.
const urls = [];
global.fetch = async (url) => { urls.push(url); return httpAnswer(200, async () => ({ status: 'success' })); };
const refusal = async (endpoint) => {
try { await PluginAPI.request(endpoint, 'POST'); return null; } catch (e) { return e.error_code; }
};
const bad = ['//evil.example/x', '/plugins/../../x', '/a\b', '/a b', 'plugins', null];
const codes = [];
for (const endpoint of bad) codes.push(await refusal(endpoint));
ok('an endpoint that could leave the API path is refused before fetch()',
codes.every(c => c === 'INVALID_ENDPOINT') && urls.length === 0, { codes, urls });
await PluginAPI.resetPluginConfig('a/../b&x=1');
ok('a plugin id is encoded into the URL, not spliced into it',
urls[0] === '/api/v3/plugins/config/reset?plugin_id=a%2F..%2Fb%26x%3D1', urls);
delete global.fetch;
}
console.log('\nsummary: a no-op update is not counted as updated');
{
const answer = (update_status, message) => ({ success: true, result: { status: 'success', message, data: { update_status } } });
const results = [
answer('updated', 'Plugin a updated to version 2.8.0'),
// ZIP-installed monorepo plugin already at the registry version: the
// route used to call this "updated successfully".
answer('up_to_date', 'Plugin stock-news already up to date (version 2.8.0)'),
answer('up_to_date', 'Plugin clock already up to date (commit abcdef1)'),
answer('local_only', 'Plugin mine is managed locally and does not receive registry updates'),
{ success: false, error: { error_code: 'PLUGIN_UPDATE_FAILED' } },
];
const s = Manager.summarizeUpdateResults(results);
ok('counts come from update_status',
s.updated === 1 && s.upToDate === 2 && s.localOnly === 1 && s.failed === 1, s);
ok('toast text names each outcome',
s.text === '1 updated, 2 already up to date, 1 managed locally, 1 failed', s.text);
ok('a failure alongside an update is a warning', s.type === 'warning', s.type);
ok('an older server that only says so in the message is still up to date',
Manager.updateOutcome({ success: true, result: { message: 'Plugin x already up to date (commit 1234567)' } }) === 'up_to_date');
ok('a success without a status or telltale message counts as updated',
Manager.updateOutcome({ success: true, result: { message: 'Plugin x updated successfully' } }) === 'updated');
const allNoop = Manager.summarizeUpdateResults([answer('up_to_date', ''), answer('up_to_date', '')]);
ok('nothing to do is a success toast with no "updated"',
allNoop.type === 'success' && allNoop.text === '2 already up to date', allNoop);
}
console.log(`\n${pass} passed, ${fail} failed`);
process.exit(fail ? 1 : 0);
})().catch(e => { console.error(e); process.exit(1); });
test/test_api_v3_display_hardware.py
+118 -10
View File
@@ -1,9 +1,10 @@
"""Display hardware settings accept what the rgbmatrix library accepts.
Held to the ranges in the pinned library (RGBMatrix::Options::Validate in
lib/options-initialize.cc, the gpio_slowdown check in lib/led-matrix.cc), with
one deliberate exception: rows has no upper bound here, although the library
currently rejects more than 64 per panel. Two ways this used to go wrong:
Held to what the pinned library and its Python binding accept
(src/matrix_support.py: RGBMatrix::Options::Validate in
lib/options-initialize.cc, the gpio_slowdown check in lib/led-matrix.cc, the
mapping table in lib/hardware-mapping.c and the binding's uint8_t setters).
Ways this used to go wrong:
- The Display form capped cols at 128, chain_length at 24 and
pwm_lsb_nanoseconds at 500, and its submit handler (fixInvalidNumberInputs)
@@ -11,6 +12,10 @@ currently rejects more than 64 per panel. Two ways this used to go wrong:
a long chain silently saved as the wrong size.
- The API checked none of these, so a value the library rejects (odd rows,
parallel 4, pwm_dither_bits 3) saved, and the matrix then refused to start.
- After that, rows above 64, chain_length above 255, a misspelled hardware
mapping and parallel 2-3 on a single-output HAT mapping still saved. The
library answers those with no matrix or abort(), not an error, so the
display service crash-looped instead of falling back.
Row address type 5 is the SM5368 / B707 row shift register the Waveshare 96x48
V2 needs (Waveshare's own "96X48_1_24_SM5368" panel type in their library fork
@@ -102,10 +107,9 @@ def test_waveshare_96x48_v2_settings_all_save(api_v3_client, saved, as_strings):
@pytest.mark.parametrize('field,value', [
('rows', 8), ('rows', 64), ('rows', 96), ('rows', 128),
('rows', 8), ('rows', 64),
('cols', 16), ('cols', 192), ('cols', 512),
('chain_length', 1), ('chain_length', 32),
('parallel', 3),
('chain_length', 1), ('chain_length', 32), ('chain_length', 255),
('row_address_type', 0), ('row_address_type', 5), ('row_address_type', 5.0),
('multiplexing', 0), ('multiplexing', 22),
('gpio_slowdown', 0), ('gpio_slowdown', 10),
@@ -123,9 +127,9 @@ def test_values_in_range_are_saved(api_v3_client, saved, field, value):
@pytest.mark.parametrize('field,value', [
('rows', 6), ('rows', 47), ('rows', 97), ('rows', '48.5'),
('rows', 6), ('rows', 47), ('rows', 66), ('rows', 96), ('rows', 128), ('rows', '48.5'),
('cols', 15), ('cols', 96.5), ('cols', True), ('cols', 'wide'),
('chain_length', 0),
('chain_length', 0), ('chain_length', 256), ('chain_length', 300),
('parallel', 0), ('parallel', 4),
('row_address_type', -1), ('row_address_type', 6),
('row_address_type', True), ('row_address_type', 5.5),
@@ -146,6 +150,69 @@ def test_values_out_of_range_are_refused(api_v3_client, saved, field, value):
assert 'config' not in saved
@pytest.mark.parametrize('body', [
{'hardware_mapping': 'regular', 'parallel': 3},
{'hardware_mapping': 'classic', 'parallel': 2},
{'hardware_mapping': 'Regular', 'parallel': 3},
{'hardware_mapping': 'classic-pi1'},
{'hardware_mapping': 'adafruit-hat', 'parallel': 1},
])
def test_mappings_the_library_has_are_saved(api_v3_client, saved, body):
response = _post(api_v3_client, body)
assert response.status_code == 200, response.get_data(as_text=True)[:200]
for field, value in body.items():
assert saved['config']['display']['hardware'][field] == value
@pytest.mark.parametrize('body,named', [
# Framebuffer() abort()s: the HAT mappings define one output.
({'hardware_mapping': 'adafruit-hat-pwm', 'parallel': 2}, 'parallel 2'),
({'hardware_mapping': 'adafruit-hat', 'parallel': 3}, 'parallel 3'),
({'hardware_mapping': 'regular-pi1', 'parallel': 2}, 'parallel 2'),
# InitHardwareMapping() abort()s on a name it doesn't have; compute-module
# isn't compiled into the default build.
({'hardware_mapping': 'adafruit-hat-pwn'}, 'adafruit-hat-pwn'),
({'hardware_mapping': 'compute-module'}, 'compute-module'),
({'hardware_mapping': 5}, 'hardware mapping'),
])
def test_combinations_the_library_aborts_on_are_refused(api_v3_client, saved, body, named):
response = _post(api_v3_client, body)
assert response.status_code == 400
assert named in response.get_json()['message']
assert 'config' not in saved
def test_parallel_is_checked_against_the_stored_mapping(api_v3_client, api_v3_module, saved):
api_v3_module.api_v3.config_manager.load_config.return_value = {
'display': {'hardware': {'hardware_mapping': 'adafruit-hat'}}}
response = _post(api_v3_client, {'parallel': 2})
assert response.status_code == 400
assert 'adafruit-hat' in response.get_json()['message']
assert 'config' not in saved
def test_stored_refusal_does_not_block_unrelated_saves(api_v3_client, api_v3_module, saved):
api_v3_module.api_v3.config_manager.load_config.return_value = {
'display': {'hardware': {'hardware_mapping': 'adafruit-hat', 'parallel': 2}}}
response = _post(api_v3_client, {'brightness': 70})
assert response.status_code == 200, response.get_data(as_text=True)[:200]
def test_a_request_value_is_checked_not_the_stored_one(api_v3_client, api_v3_module, saved):
"""Fixing a stored bad value in the same save as a mapping change must work."""
api_v3_module.api_v3.config_manager.load_config.return_value = {
'display': {'hardware': {'rows': 128, 'parallel': 2}}}
response = _post(api_v3_client, {'rows': 64, 'hardware_mapping': 'regular'})
assert response.status_code == 200, response.get_data(as_text=True)[:200]
@pytest.mark.parametrize('orientation', ['normal', '90', '180', '270'])
def test_every_orientation_display_manager_applies_is_saved(api_v3_client, saved, orientation):
response = _post(api_v3_client, {'orientation': orientation})
assert response.status_code == 200, response.get_data(as_text=True)[:200]
assert saved['config']['display']['hardware']['orientation'] == orientation
@pytest.fixture
def display_page(monkeypatch):
"""Render the Display settings partial for a given config."""
@@ -231,6 +298,38 @@ def test_waveshare_96x48_v2_config_renders_back_unchanged(display_page):
assert _attr(_input_tag(body, input_id), 'value') == str(WAVESHARE_96X48_V2[input_id]), input_id
@pytest.mark.parametrize('hardware,select_id,expected', [
({'hardware_mapping': 'classic'}, 'hardware_mapping', 'classic'),
({'hardware_mapping': 'classic-pi1'}, 'hardware_mapping', 'classic-pi1'),
({'hardware_mapping': 'adafruit-hat'}, 'hardware_mapping', 'adafruit-hat'),
({'hardware_mapping': 'Regular'}, 'hardware_mapping', 'regular'),
({'hardware_mapping': ''}, 'hardware_mapping', 'regular'),
({'orientation': '90'}, 'orientation', '90'),
({'orientation': '270'}, 'orientation', '270'),
])
def test_stored_mapping_and_orientation_render_back_unchanged(display_page, hardware, select_id, expected):
"""With nothing selected the browser posts the first option, so one unrelated
Display save turned classic into adafruit-hat-pwm and 90 degrees into normal."""
body = display_page(_config_with(hardware=hardware))
assert _selected_option(body, select_id) == [expected]
assert "saved hardware mapping" not in body
def test_missing_mapping_renders_display_managers_default(display_page):
config = _config_with()
del config['display']['hardware']['hardware_mapping']
assert _selected_option(display_page(config), 'hardware_mapping') == ['adafruit-hat-pwm']
@pytest.mark.parametrize('stored', ['compute-module', 'adafruit-hat-pwn'])
def test_unusable_stored_mapping_renders_selected_with_a_warning(display_page, stored):
"""Kept selected rather than silently swapped for the first option; the API
refuses it on save, and the warning says why."""
body = display_page(_config_with(hardware={'hardware_mapping': stored}))
assert _selected_option(body, 'hardware_mapping') == [stored]
assert f'Your saved hardware mapping ("{stored}")' in body
@pytest.mark.parametrize('field,section', [
('gpio_slowdown', 'runtime'), ('pwm_dither_bits', 'hardware'),
('limit_refresh_rate_hz', 'hardware'),
@@ -248,7 +347,7 @@ def test_a_stored_zero_renders_as_zero(display_page, field, section):
@pytest.mark.parametrize('body', [
{'row_address_type': 5}, {'row_address_type': '1'},
{'hardware_mapping': 'compute-module'},
{'hardware_mapping': 'classic-pi1'},
])
def test_pi5_refuses_what_its_library_cannot_drive(api_v3_client, saved, board, body):
board(PI5_MODEL)
@@ -296,6 +395,15 @@ def test_pi5_form_offers_only_supported_row_address_types(display_page, board):
assert "can't be used on this Raspberry Pi 5" not in body
def test_pi5_form_offers_only_mappings_it_can_drive(display_page, board):
board(PI5_MODEL)
body = display_page(_config_with())
assert 'classic-pi1' not in _option_values(body, 'hardware_mapping')
body = display_page(_config_with(hardware={'hardware_mapping': 'classic-pi1'}))
assert _selected_option(body, 'hardware_mapping') == ['classic-pi1']
assert 'can use on a Raspberry Pi 5' in body
def test_other_boards_offer_every_row_address_type(display_page):
body = display_page(_config_with())
assert _option_values(body, 'row_address_type') == ['0', '1', '2', '3', '4', '5']
test/test_api_v3_lazy_plugin_discovery.py
+5 -1
View File
@@ -31,6 +31,7 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
from src.config_manager import ConfigManager # noqa: E402
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
from src.plugin_system.schema_manager import SchemaManager # noqa: E402
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
PLUGIN_ID = 'ledmatrix-stocks'
@@ -142,7 +143,10 @@ def test_toggle_finds_the_plugin(api_v3_client, api_v3_module, fresh_web_process
def test_main_config_save_keeps_a_plugin_secret_out_of_config_json(
api_v3_client, api_v3_module, fresh_web_process, tmp_path):
api_v3_client, api_v3_module, fresh_web_process, plugins_dir, tmp_path):
# /config/main validates a plugin section like POST /plugins/config does,
# so it needs a real schema manager rather than the helper's mock.
api_v3_module.api_v3.schema_manager = SchemaManager(plugins_dir=plugins_dir)
config_file = tmp_path / 'config.json'
config_file.write_text('{}')
secrets_file = tmp_path / 'config_secrets.json'
test/test_api_v3_partial_main_save.py
+288
View File
@@ -0,0 +1,288 @@
"""POST /config/main: a partial JSON body changes only what it sends.
The settings forms post every field, and a browser leaves an unchecked box out,
so for a form a missing checkbox means False. JSON API clients send only what
they change. Treating their missing keys as unchecked meant:
- the MQTT bridge's Home Assistant brightness slider (``{"brightness": N}``)
turned off disable_hardware_pulsing, inverse_colors, show_refresh_rate and
use_short_date_format on every change;
- the documented timezone/location update turned off web_display_autostart and
weekly automatic updates.
The v3 forms post JSON as well (htmx json-enc), so they mark themselves with a
hidden ``__form_section`` input; these tests pin both halves of that contract.
Also here: the Vegas cycle-time fields no longer land in display_durations
(and a blank one no longer 400s the Display save), the Raw JSON editor starts
auto-update setup like the General form does, and both schedule POSTs accept
the per-day shape their GETs return.
"""
import copy
import json
import re
from pathlib import Path
import pytest
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401
REPO = Path(__file__).resolve().parent.parent
PARTIALS = REPO / 'web_interface' / 'templates' / 'v3' / 'partials'
STORED = {
'web_display_autostart': True,
'auto_update': {'enabled': True},
'timezone': 'America/Chicago',
'plugin_system': {'auto_discover': True, 'auto_load_enabled': True,
'development_mode': True},
'display': {
'hardware': {'rows': 32, 'cols': 64, 'chain_length': 2, 'brightness': 90,
'disable_hardware_pulsing': True, 'inverse_colors': True,
'show_refresh_rate': True},
'runtime': {'gpio_slowdown': 4},
'use_short_date_format': True,
'double_sided': {'enabled': True, 'copies': 2, 'axis': 'horizontal'},
'vegas_scroll': {'enabled': True, 'auto_trim': True,
'dynamic_duration_enabled': True,
'continuous_scroll': True, 'smooth_scroll': True},
},
}
@pytest.fixture
def saved(api_v3_module, monkeypatch):
captured = {}
api_v3_module.api_v3.config_manager.load_config.side_effect = \
lambda *a, **k: copy.deepcopy(STORED)
def fake_save(_manager, config, **_kwargs):
captured['config'] = config
return True, ''
monkeypatch.setattr(api_v3_module, '_save_config_atomic', fake_save)
from web_interface import auto_update
monkeypatch.setattr(auto_update, 'start_setup_if_needed', lambda *a, **k: None)
return captured
def _post_json(client, body):
return client.post('/api/v3/config/main', data=json.dumps(body),
content_type='application/json')
class TestJsonPartialSaves:
def test_mqtt_bridge_brightness_changes_only_brightness(self, api_v3_client, saved):
# integrations/mqtt_bridge/ledmatrix_mqtt_bridge.py set_brightness
resp = _post_json(api_v3_client, {'brightness': 40})
assert resp.status_code == 200, resp.get_json()
display = saved['config']['display']
assert display['hardware']['brightness'] == 40
assert display['hardware']['disable_hardware_pulsing'] is True
assert display['hardware']['inverse_colors'] is True
assert display['hardware']['show_refresh_rate'] is True
assert display['use_short_date_format'] is True
def test_timezone_only_keeps_autostart_and_auto_update(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'timezone': 'UTC'})
assert resp.status_code == 200, resp.get_json()
config = saved['config']
assert config['timezone'] == 'UTC'
assert config['web_display_autostart'] is True
assert config['auto_update'] == {'enabled': True}
def test_location_only_keeps_autostart_and_auto_update(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'city': 'Paris', 'country': 'FR'})
assert resp.status_code == 200, resp.get_json()
config = saved['config']
assert config['location'] == {'city': 'Paris', 'country': 'FR'}
assert config['web_display_autostart'] is True
assert config['auto_update'] == {'enabled': True}
@pytest.mark.parametrize('key', ['auto_discover', 'auto_load_enabled', 'development_mode'])
def test_a_legacy_plugin_system_toggle_is_not_a_general_save(self, api_v3_client, saved, key):
# These left the General form; a client still sending one must not
# have the general-settings checkboxes treated as unchecked.
resp = api_v3_client.post('/api/v3/config/main', data={key: 'on'},
content_type='application/x-www-form-urlencoded')
assert resp.status_code == 200, resp.get_json()
assert saved['config']['web_display_autostart'] is True
assert saved['config']['auto_update'] == {'enabled': True}
def test_vegas_speed_only_keeps_vegas_toggles(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'vegas_scroll_speed': 80})
assert resp.status_code == 200, resp.get_json()
vegas = saved['config']['display']['vegas_scroll']
assert vegas['scroll_speed'] == 80
for key in ('enabled', 'auto_trim', 'dynamic_duration_enabled',
'continuous_scroll', 'smooth_scroll'):
assert vegas[key] is True, key
def test_double_sided_axis_only_keeps_enabled(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'double_sided_axis': 'horizontal'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['double_sided']['enabled'] is True
def test_json_can_still_turn_a_checkbox_off(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'inverse_colors': False, 'auto_update_enabled': False})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['inverse_colors'] is False
assert saved['config']['display']['hardware']['disable_hardware_pulsing'] is True
assert saved['config']['auto_update'] == {'enabled': False}
def test_json_with_charset_is_still_json(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data=json.dumps({'brightness': 41}),
content_type='application/json; charset=utf-8')
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['brightness'] == 41
assert saved['config']['display']['hardware']['inverse_colors'] is True
def test_a_non_object_body_is_refused(self, api_v3_client, saved):
assert _post_json(api_v3_client, [1, 2]).status_code == 400
class TestFormSavesStillUncheck:
"""Unchecking a box in the UI must still save false."""
def test_marked_json_form_post_unchecks_missing_display_boxes(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'__form_section': 'display', 'brightness': '40',
'vegas_scroll_speed': '50'})
assert resp.status_code == 200, resp.get_json()
display = saved['config']['display']
for key in ('disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate'):
assert display['hardware'][key] is False, key
assert display['use_short_date_format'] is False
assert display['vegas_scroll']['enabled'] is False
assert '__form_section' not in saved['config']
def test_marked_json_form_post_keeps_checked_boxes(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'__form_section': 'display', 'brightness': '40',
'inverse_colors': 'on'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['inverse_colors'] is True
assert saved['config']['display']['hardware']['show_refresh_rate'] is False
def test_marked_general_form_unchecks_autostart_and_auto_update(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'__form_section': 'general', 'timezone': 'UTC'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['web_display_autostart'] is False
assert saved['config']['auto_update'] == {'enabled': False}
def test_form_encoded_post_unchecks_missing_boxes(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data={'brightness': '40'},
content_type='application/x-www-form-urlencoded')
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['hardware']['inverse_colors'] is False
@pytest.mark.parametrize('partial', ['general.html', 'display.html', 'durations.html'])
def test_every_config_main_form_carries_the_marker(self, partial):
html = (PARTIALS / partial).read_text(encoding='utf-8')
form = re.search(r'<form hx-post="/api/v3/config/main".*?</form>', html, re.S)
assert form, f'{partial} no longer posts to /config/main'
assert re.search(r'<input type="hidden" name="__form_section" value="\w+">',
form.group(0)), f'{partial} form lost its __form_section marker'
class TestVegasCycleDurations:
def test_cycle_durations_are_not_display_durations(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data={
'vegas_scroll_enabled': 'on', 'vegas_min_cycle_duration': '90',
'vegas_max_cycle_duration': '300'})
assert resp.status_code == 200, resp.get_json()
display = saved['config']['display']
assert display['vegas_scroll']['min_cycle_duration'] == 90
assert display['vegas_scroll']['max_cycle_duration'] == 300
durations = display.get('display_durations', {})
assert 'vegas_min_cycle_duration' not in durations
assert 'vegas_max_cycle_duration' not in durations
assert 'vegas_min_cycle_duration' not in saved['config']
def test_blank_cycle_duration_does_not_reject_the_save(self, api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data={
'vegas_scroll_enabled': 'on', 'vegas_scroll_speed': '60',
'vegas_min_cycle_duration': ''})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['vegas_scroll']['scroll_speed'] == 60
def test_real_display_durations_still_save(self, api_v3_client, saved):
resp = _post_json(api_v3_client, {'clock_duration': '45'})
assert resp.status_code == 200, resp.get_json()
assert saved['config']['display']['display_durations']['clock_duration'] == 45
class TestRawSaveStartsAutoUpdateSetup:
@pytest.fixture
def raw_env(self, api_v3_module, monkeypatch):
cm = api_v3_module.api_v3.config_manager
cm.get_raw_file_content.return_value = {'timezone': 'UTC', 'auto_update': {'enabled': False}}
calls = []
from web_interface import auto_update
monkeypatch.setattr(auto_update, 'start_setup_if_needed',
lambda was, config: calls.append((was, config)) or 'Setup note.')
return cm, calls
def test_enabling_from_raw_json_calls_setup(self, api_v3_client, raw_env):
cm, calls = raw_env
body = {'timezone': 'UTC', 'auto_update': {'enabled': True}}
resp = api_v3_client.post('/api/v3/config/raw/main', json=body)
assert resp.status_code == 200, resp.get_json()
cm.save_raw_file_content.assert_called_once_with('main', body)
assert calls == [(False, body)]
assert 'Setup note.' in resp.get_json()['message']
def test_previously_enabled_is_passed_through(self, api_v3_client, raw_env):
cm, calls = raw_env
cm.get_raw_file_content.return_value = {'auto_update': {'enabled': True}}
body = {'auto_update': {'enabled': True}}
assert api_v3_client.post('/api/v3/config/raw/main', json=body).status_code == 200
assert calls == [(True, body)]
class TestSchedulesAcceptTheirGetShape:
PER_DAY = {
'enabled': True, 'mode': 'per-day', 'dim_brightness': 20,
'days': {
'monday': {'enabled': True, 'start_time': '21:15', 'end_time': '06:45'},
'tuesday': {'enabled': False},
'wednesday': {'enabled': True, 'start_time': '22:00', 'end_time': '05:30'},
'thursday': {'enabled': True, 'start_time': '20:00', 'end_time': '07:00'},
'friday': {'enabled': True, 'start_time': '23:00', 'end_time': '08:00'},
'saturday': {'enabled': True, 'start_time': '23:30', 'end_time': '09:00'},
'sunday': {'enabled': True, 'start_time': '21:00', 'end_time': '07:00'},
},
}
@pytest.fixture
def store(self, api_v3_module, monkeypatch):
state = {'config': {}}
api_v3_module.api_v3.config_manager.load_config.side_effect = \
lambda *a, **k: copy.deepcopy(state['config'])
def fake_save(_manager, config, **_kwargs):
state['config'] = copy.deepcopy(config)
return True, ''
monkeypatch.setattr(api_v3_module, '_save_config_atomic', fake_save)
return state
@pytest.mark.parametrize('route,section,extra', [
('/api/v3/config/dim-schedule', 'dim_schedule', {'dim_brightness': 20}),
('/api/v3/config/schedule', 'schedule', {}),
])
def test_get_output_posts_back_unchanged(self, api_v3_client, store, route, section, extra):
body = {k: v for k, v in self.PER_DAY.items() if k != 'dim_brightness'}
body.update(extra)
store['config'] = {section: copy.deepcopy(body)}
read = api_v3_client.get(route).get_json()['data']
resp = api_v3_client.post(route, json=read)
assert resp.status_code == 200, resp.get_json()
assert store['config'][section]['days'] == body['days']
def test_flat_form_keys_still_work(self, api_v3_client, store):
body = {'enabled': True, 'mode': 'per-day', 'dim_brightness': 25,
'monday_enabled': 'on', 'monday_start': '19:00', 'monday_end': '06:00'}
resp = api_v3_client.post('/api/v3/config/dim-schedule', json=body)
assert resp.status_code == 200, resp.get_json()
assert store['config']['dim_schedule']['days']['monday'] == {
'enabled': True, 'start_time': '19:00', 'end_time': '06:00'}
test/test_auto_update.py
+175 -4
View File
@@ -72,7 +72,7 @@ class Repo:
def real_pull(device, **extra):
def core_update():
def core_update(**kwargs):
before = git(device, 'rev-parse', 'HEAD')
r = subprocess.run(['git', 'pull', '-q', '--rebase', '--autostash'],
cwd=str(device), capture_output=True, text=True)
@@ -339,6 +339,145 @@ class TestPreflightRefuses:
assert repo.head() == old and h.sudo == []
# -- what counts as a local change ---------------------------------------------
def _device_with_plugin_and_script(tmp_path):
"""A clone carrying a bundled plugin and an executable script, as LEDMatrix does."""
repo = Repo(tmp_path)
for rel, text in (('plugin-repos/web-ui-info/manager.py', 'x = 1\n'),
('web_interface/static/v3/js/plugins/store.js', 'var a;\n'),
('scripts/run.sh', 'echo hi\n')):
(repo.seed / rel).parent.mkdir(parents=True, exist_ok=True)
(repo.seed / rel).write_text(text)
# Executable on disk too, not only in the index: Repo.publish() commits
# with -a, which on Linux would otherwise record run.sh as 100644 upstream
# and hide the device's own mode change behind the pull.
import os
os.chmod(repo.seed / 'scripts' / 'run.sh', 0o755)
git(repo.seed, 'add', '.')
git(repo.seed, 'update-index', '--chmod=+x', 'scripts/run.sh')
git(repo.seed, 'commit', '-qm', 'layout')
git(repo.seed, 'push', '-q', 'origin', 'main')
git(repo.device, 'pull', '-q')
return repo
def _chmod_like_the_installer(device):
"""first_time_install.sh sets tracked 100755 files to 644: a mode-only change.
Windows has no exec bit, so there core.fileMode=true alone shows it."""
import os
git(device, 'config', 'core.fileMode', 'true')
os.chmod(device / 'scripts' / 'run.sh', 0o644)
assert 'mode change 100755 => 100644 scripts/run.sh' in git(device, 'diff', '--summary')
@needs_git
class TestLocalChangesAreCountedOnce:
"""The preflight promises the pull will not stash. That holds only if both
count local changes the same way (auto_update.local_changes)."""
def test_mode_changes_and_plugin_folders_do_not_count(self, tmp_path):
repo = _device_with_plugin_and_script(tmp_path)
_chmod_like_the_installer(repo.device)
(repo.device / 'plugin-repos/web-ui-info/manager.py').write_text('x = 2 # store update\n')
assert au.local_changes(repo.device) == []
def test_a_core_folder_named_plugins_still_counts(self, tmp_path):
"""The old filter matched 'plugins/' anywhere in the line."""
repo = _device_with_plugin_and_script(tmp_path)
(repo.device / 'web_interface/static/v3/js/plugins/store.js').write_text('var mine;\n')
assert au.local_changes(repo.device) == ['web_interface/static/v3/js/plugins/store.js']
def test_a_staged_rename_is_reported_by_its_new_name(self, tmp_path):
repo = Repo(tmp_path)
git(repo.device, 'mv', 'app.py', 'main.py')
assert au.local_changes(repo.device) == ['main.py']
def test_the_preflight_refuses_a_core_edit_under_a_plugins_folder(self, tmp_path):
repo = _device_with_plugin_and_script(tmp_path)
old = repo.head()
repo.publish()
(repo.device / 'web_interface/static/v3/js/plugins/store.js').write_text('var mine;\n')
result = Harness(tmp_path, repo).updater.run()
assert result['core_outcome'] == 'blocked'
assert 'store.js' in result['core_message']
assert repo.head() == old
@pytest.fixture
def core_update(self, monkeypatch):
"""The real perform_core_update, pointed at a test clone."""
from web_interface.blueprints import api_v3 as pkg
from web_interface.blueprints.api_v3 import system
def point_at(device):
monkeypatch.setattr(system, 'PROJECT_ROOT', device)
monkeypatch.setattr(pkg.api_v3, 'plugin_store_manager', None, raising=False)
monkeypatch.setattr(system, '_pip_install_requirements',
lambda *a, **k: pytest.fail('no requirements changed'))
return system.perform_core_update
return point_at
@pytest.mark.parametrize('change', [
'bundled_plugin_edit',
pytest.param('installer_chmod', marks=pytest.mark.skipif(
sys.platform == 'win32',
reason='no exec bit: the reset inside --autostash cannot clear a mode change, '
'so git will not reapply it (on Linux it reapplies cleanly)')),
])
def test_an_update_that_passed_the_preflight_leaves_no_stash(self, tmp_path, core_update, change):
"""Found by audit: a bundled-plugin edit or the installer's chmods
passed the preflight, then perform_core_update stashed them for good.
Now --autostash carries them across the pull and puts them back."""
repo = _device_with_plugin_and_script(tmp_path)
new = repo.publish()
if change == 'installer_chmod':
_chmod_like_the_installer(repo.device)
else:
(repo.device / 'plugin-repos/web-ui-info/manager.py').write_text('x = 2 # store update\n')
h = Harness(tmp_path, repo, core_update=core_update(repo.device))
assert h.updater.preflight({})[0] == 'ready'
result = h.updater.run()
assert result['core_outcome'] == 'verifying', result['core_message']
assert repo.head() == new
assert git(repo.device, 'stash', 'list') == ''
if change == 'installer_chmod':
assert 'mode change 100755 => 100644 scripts/run.sh' in git(repo.device, 'diff', '--summary')
else:
assert 'store update' in (repo.device / 'plugin-repos/web-ui-info/manager.py').read_text()
def test_edits_made_after_the_preflight_are_refused_not_stashed(self, tmp_path, core_update):
repo = Repo(tmp_path)
old = repo.head()
repo.publish()
perform = core_update(repo.device)
def edit_then_update(**kwargs):
(repo.device / 'app.py').write_text('mine\n')
return perform(**kwargs)
result = Harness(tmp_path, repo, core_update=edit_then_update).updater.run()
assert result['core_outcome'] == 'blocked'
assert 'app.py' in result['core_message'] and 'will not stash' in result['core_message']
assert repo.head() == old
assert git(repo.device, 'stash', 'list') == ''
assert (repo.device / 'app.py').read_text() == 'mine\n'
def test_update_code_still_stashes_core_edits_but_not_plugin_folders(self, tmp_path, core_update):
repo = _device_with_plugin_and_script(tmp_path)
new = repo.publish()
(repo.device / 'scripts/run.sh').write_text('echo mine\n')
(repo.device / 'plugin-repos/web-ui-info/manager.py').write_text('x = 2 # store update\n')
payload = core_update(repo.device)()
assert payload['status'] == 'success', payload['message']
assert 'stashed' in payload['message']
assert repo.head() == new
assert 'LEDMatrix auto-stash before update' in git(repo.device, 'stash', 'list')
stashed = git(repo.device, 'stash', 'show', '--name-only', 'stash@{0}')
assert stashed.split() == ['scripts/run.sh']
assert 'store update' in (repo.device / 'plugin-repos/web-ui-info/manager.py').read_text()
# -- the update and its hand-off to the health check ---------------------------
@needs_git
@@ -395,7 +534,7 @@ class TestUpdateIsVerified:
old = repo.head()
repo.publish()
h = Harness(tmp_path, repo,
core_update=lambda: {'status': 'error', 'message': 'Update failed: network'})
core_update=lambda **kw: {'status': 'error', 'message': 'Update failed: network'})
result = h.updater.run()
assert result['core_outcome'] == 'error'
assert repo.head() == old
@@ -407,13 +546,43 @@ class TestUpdateIsVerified:
repo.publish()
pull = real_pull(repo.device)
def half_failed():
def half_failed(**kwargs):
pull()
return {'status': 'error', 'message': 'Update failed: interrupted'}
result = Harness(tmp_path, repo, core_update=half_failed).updater.run()
assert repo.head() == old
assert 'rolled back' in result['core_message']
@pytest.mark.parametrize('how', ['partial_pull', 'check_never_started'])
def test_a_failed_rollback_leaves_plugins_and_the_display_alone(self, tmp_path, how):
"""update_core returns rollback_failed itself on two paths. The device
is then in an unknown state, exactly as when the health check reports
rollback_failed, so no plugin updates and no restart may follow."""
repo = Repo(tmp_path)
repo.publish()
store = FakeStore(tmp_path / 'plugins', {'clock': '1.0.0'}, bump={'clock'})
pull = real_pull(repo.device)
def half_failed(**kwargs):
pull()
return {'status': 'error', 'message': 'Update failed: interrupted'}
h = Harness(tmp_path, repo, store=store, pickup=how != 'check_never_started',
core_update=half_failed if how == 'partial_pull' else None)
real_run = h.updater.run_command
def reset_fails(args, **kwargs):
if args[:3] == ['git', 'reset', '--hard']:
return subprocess.CompletedProcess(args, 1, stdout='', stderr='index.lock exists')
return real_run(args, **kwargs)
h.updater.run_command = reset_fails
result = h.updater.run()
assert result['core_outcome'] == 'rollback_failed'
assert store.updated_calls == [], "plugins must not pile onto a core in an unknown state"
assert h.restarts == []
assert h.state['plugins_pending'] is False and result['plugins_deferred'] is False
assert h.state['alert']
# -- reporting the health check's outcome --------------------------------------
@@ -668,7 +837,9 @@ class TestSettingsSave:
def test_unchecked_toggle_on_general_save_disables(self, api_client):
client, cm, setup_calls = api_client
resp = client.post('/api/v3/config/main', json={'timezone': 'UTC'})
# The General form marks its posts; an unchecked box is then absent.
resp = client.post('/api/v3/config/main',
json={'__form_section': 'general', 'timezone': 'UTC'})
assert resp.status_code == 200
assert self._saved(cm)['auto_update'] == {'enabled': False}
assert setup_calls == [True]
test/test_auto_update_verify.py
+76
View File
@@ -69,6 +69,8 @@ class FakeHost:
self.running_head = None # not restarted yet: still the old, healthy code
self.restarts = [] # (unit, commit it was restarted onto)
self.pip_installs = []
self.pip_timeouts = []
self.pip = None # optional (args, host) -> result, or raises, instead of pip_ok
self.now = 0.0
self.nrestarts = 0
@@ -90,6 +92,9 @@ class FakeHost:
return done(args, f'{self.nrestarts}\n')
if args[0] == 'sudo' and any(a.endswith('safe_pip_install.sh') for a in args):
self.pip_installs.append(args[-1])
self.pip_timeouts.append(kwargs.get('timeout'))
if self.pip:
return self.pip(args, self)
return done(args, rc=0 if self.pip_ok else 1)
if args[:4] == ['sudo', '-n', 'systemctl', 'restart']:
if self.restart_failures:
@@ -171,6 +176,77 @@ def test_a_failed_dependency_reinstall_is_reported(tmp_path):
assert 'Install Base Requirements' in result['detail']
def _rollback_with_pip(tmp_path, pip, web_requirements_too=False):
"""Roll back an update that changed the requirements, with pip faked by ``pip``."""
repo, old, new = updated_repo(tmp_path, new_requirements=True)
if web_requirements_too:
(repo / 'web_interface').mkdir()
(repo / 'web_interface' / 'requirements.txt').write_text('flask\n')
git(repo, 'add', '.')
git(repo, 'commit', '-qm', 'web reqs')
(repo / 'web_interface' / 'requirements.txt').write_text('flask\nnew\n')
(repo / 'requirements.txt').write_text('requests\nnewer\n')
git(repo, 'commit', '-qam', 'bump both')
old, new = git(repo, 'rev-parse', 'HEAD~1'), git(repo, 'rev-parse', 'HEAD')
host = FakeHost(repo, new)
host.pip = pip
ok, detail = host.verifier().rollback({'old_head': old, 'new_head': new})
return ok, detail, host
def test_a_failed_pip_is_not_run_again_with_the_other_bash(tmp_path):
"""Retrying after pip itself ran only repeats it, and every repeat can
take PIP_TIMEOUT_SECONDS of the unit's time limit."""
ok, detail, host = _rollback_with_pip(
tmp_path, lambda args, h: subprocess.CompletedProcess(args, 1, '', 'ERROR: No matching distribution'))
assert ok and 'requirements.txt' in detail
assert len(host.pip_installs) == 1
def test_a_pip_timeout_is_not_retried(tmp_path):
def slow(args, h):
h.now += h.pip_timeouts[-1]
raise subprocess.TimeoutExpired(args, h.pip_timeouts[-1])
ok, detail, host = _rollback_with_pip(tmp_path, slow)
assert ok and 'Install Base Requirements' in detail
assert len(host.pip_installs) == 1
def test_a_sudo_refusal_tries_the_next_bash(tmp_path):
def refused_once(args, h):
if len(h.pip_installs) == 1:
return subprocess.CompletedProcess(args, 1, '', 'sudo: a password is required')
return done(args)
ok, detail, host = _rollback_with_pip(tmp_path, refused_once)
assert ok and detail == ''
assert len(host.pip_installs) == 2
def test_reinstalls_share_one_time_budget(tmp_path):
def hangs(args, h):
h.now += h.pip_timeouts[-1]
raise subprocess.TimeoutExpired(args, h.pip_timeouts[-1])
ok, detail, host = _rollback_with_pip(tmp_path, hangs, web_requirements_too=True)
assert ok and 'requirements.txt' in detail and 'web_interface/requirements.txt' in detail
assert sum(host.pip_timeouts) <= av.PIP_BUDGET_SECONDS
assert len(host.pip_installs) == 1, "the first file used the whole budget"
def test_the_worst_case_fits_the_unit_time_limit():
"""systemd kills the check at TimeoutStartSec, mid-rollback, and the update
then sits in "verifying" until the web interface calls it lost."""
from web_interface import auto_update as au
service = (ROOT / 'systemd' / 'ledmatrix-update-verify.service').read_text(encoding='utf-8')
minutes = int(re.search(r'^TimeoutStartSec=(\d+)min$', service, re.M).group(1))
assert av.WORST_CASE_SECONDS < minutes * 60
assert minutes * 60 < au.VERIFY_LOST_SECONDS, "the web UI must not call a running check lost"
def test_sudo_refusal_wording_matches_permission_utils():
from src.common import permission_utils
assert set(av.SUDO_REFUSAL_PHRASES) == set(permission_utils.SUDO_REFUSAL_PHRASES)
def test_still_broken_after_rolling_back_is_rollback_failed(tmp_path):
code, result, host, head, old, new = check(tmp_path, 'always')
assert code == 1 and result['status'] == 'rollback_failed'
test/test_background_data_service_espn_ranges.py
+51 -1
View File
@@ -13,7 +13,12 @@ from unittest.mock import MagicMock, patch
import pytest
from src.background_data_service import BackgroundDataService
from src.common.espn_dates import ESPN_MAX_LIMIT, parse_espn_date_range
from src.common import espn_dates
from src.common.espn_dates import (
ESPN_MAX_LIMIT,
fetch_espn_scoreboard,
parse_espn_date_range,
)
URL = "https://site.api.espn.com/apis/site/v2/sports/football/nfl/scoreboard"
@@ -50,6 +55,12 @@ class RangeRejectingSession:
return FakeResponse(200, {"events": self.events_by_chunk.get(dates, [])})
@pytest.fixture(autouse=True)
def ranges_not_yet_rejected(monkeypatch):
# The "ranges are rejected" memo is process-wide; every test starts clean.
monkeypatch.setattr(espn_dates, "_ranges_rejected_until", 0.0)
@pytest.fixture
def cache():
manager = MagicMock()
@@ -161,3 +172,42 @@ def test_a_400_on_a_single_day_is_still_a_failure(service, cache):
assert not result.success
assert len(session.calls) == 1
cache.set.assert_not_called()
def test_a_rejection_seen_by_the_service_is_remembered_for_scoreboards(service):
submit_and_wait(service, RangeRejectingSession({"202609": [{"id": "a"}]}),
"20260901-20260930")
# A live scoreboard asking for a range next must not spend a doomed 400.
session = RangeRejectingSession({"202609": [{"id": "a"}]})
data = fetch_espn_scoreboard(session, URL, params={"dates": "20260901-20260930"})
assert [call["dates"] for call in session.calls] == ["202609"]
assert [event["id"] for event in data["events"]] == ["a"]
def test_a_rejection_seen_by_a_scoreboard_skips_the_range_in_the_service(service, cache):
fetch_espn_scoreboard(RangeRejectingSession(), URL,
params={"dates": "20260901-20260930"})
session = RangeRejectingSession({"202609": [{"id": "a"}]})
result = submit_and_wait(service, session, "20260901-20261001")
assert result.success, result.error
assert [call["dates"] for call in session.calls] == ["202609", "20261001"]
def test_known_rejection_with_every_chunk_failing_asks_the_range_once(service, cache):
espn_dates._note_range_rejected()
session = RangeRejectingSession(fail_chunks={"202609"})
result = submit_and_wait(service, session, "20260901-20260930")
assert not result.success
# Chunks once, then the range for a real error -- not the chunks again.
assert [call["dates"] for call in session.calls] == ["202609", "20260901-20260930"]
cache.set.assert_not_called()
def test_ranges_are_tried_again_once_the_memo_expires(service, monkeypatch):
monkeypatch.setattr(espn_dates, "_ranges_rejected_until", time.monotonic() - 1)
session = RangeRejectingSession({"202609": [{"id": "a"}]})
submit_and_wait(service, session, "20260901-20260930")
assert [call["dates"] for call in session.calls] == ["20260901-20260930", "202609"]
test/test_core_config_key_adopters.py
+108
View File
@@ -0,0 +1,108 @@
"""Every "is this top-level key a plugin?" decision uses src/core_config_keys.py.
#589 moved plugin-state reconciliation onto the shared list, but three private
copies stayed behind and did not know about auto_update, dim_schedule, sync,
location, target_fps, ...:
- StartupValidator warned "Plugin 'auto_update' is enabled but not found in
plugins directory" on every display start with auto-update or a dim
schedule switched on;
- SchemaManager.detect_config_key_collisions let a plugin id collide with
those sections silently;
- ConfigManager.cleanup_orphaned_plugin_configs deleted display, schedule,
auto_update, ... as orphans, and validate_all_plugin_configs validated core
sections as plugins.
"""
import json
import re
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from src.config_manager import ConfigManager
from src.core_config_keys import CORE_CONFIG_KEYS, CORE_SECRETS_KEYS
from src.plugin_system.schema_manager import SchemaManager
from src.startup_validator import StartupValidator
REPO = Path(__file__).resolve().parent.parent
class TestStartupValidator:
def test_enabled_core_sections_are_not_missing_plugins(self):
config = {
'display': {'hardware': {}},
'auto_update': {'enabled': True},
'dim_schedule': {'enabled': True},
'schedule': {'enabled': True},
'gone-plugin': {'enabled': True},
}
cm = MagicMock()
cm.get_config.return_value = config
pm = MagicMock()
pm.discover_plugins.return_value = []
validator = StartupValidator(cm, plugin_manager=pm, cache_manager=MagicMock())
validator.warnings = []
validator.errors = []
validator._validate_plugins()
missing = [w for w in validator.warnings if 'not found in plugins directory' in w]
assert missing == ["Plugin 'gone-plugin' is enabled but not found in plugins directory"]
class TestReservedPluginIds:
@pytest.mark.parametrize('key', sorted(CORE_CONFIG_KEYS))
def test_every_core_key_is_reserved(self, key):
collisions = SchemaManager().detect_config_key_collisions([key])
assert [c['type'] for c in collisions] == ['reserved_key_collision']
def test_an_ordinary_plugin_id_is_not(self):
assert SchemaManager().detect_config_key_collisions(['clock-simple']) == []
class TestConfigManagerHelpers:
@pytest.fixture
def manager(self, tmp_path):
config = {key: {'enabled': True} for key in ('display', 'schedule', 'auto_update',
'dim_schedule', 'sync', 'location')}
config.update({'timezone': 'UTC', 'installed': {'enabled': True},
'orphan': {'enabled': True}})
secrets = {'github': {'token': 't'}, 'youtube': {'api_key': 'k'},
'installed': {'api_key': 'a'}, 'orphan': {'api_key': 'o'}}
(tmp_path / 'config.json').write_text(json.dumps(config))
(tmp_path / 'secrets.json').write_text(json.dumps(secrets))
manager = ConfigManager(config_path=str(tmp_path / 'config.json'),
secrets_path=str(tmp_path / 'secrets.json'))
manager.template_path = str(tmp_path / 'no-template.json')
return manager
def test_cleanup_removes_only_real_orphans(self, manager, tmp_path):
removed = manager.cleanup_orphaned_plugin_configs(['installed'])
assert removed == ['orphan']
config = json.loads((tmp_path / 'config.json').read_text())
assert {'display', 'schedule', 'auto_update', 'dim_schedule', 'sync',
'location', 'timezone', 'installed'} == set(config)
secrets = json.loads((tmp_path / 'secrets.json').read_text())
assert set(secrets) == {'github', 'youtube', 'installed'}
def test_validate_all_skips_core_sections(self, manager):
schema_manager = MagicMock()
schema_manager.load_schema.return_value = None
results = manager.validate_all_plugin_configs(schema_manager)
assert set(results) == {'installed', 'orphan'}
def test_core_secrets_keys_are_not_core_config_keys():
assert not (CORE_SECRETS_KEYS & CORE_CONFIG_KEYS)
@pytest.mark.parametrize('relpath', [
'src/startup_validator.py',
'src/config_manager.py',
'src/plugin_system/schema_manager.py',
])
def test_no_private_copy_of_the_list_remains(relpath):
"""The old copies all started with this literal; a new copy likely would too."""
source = (REPO / relpath).read_text(encoding='utf-8')
assert not re.search(r"""\[\s*['"]display['"]\s*,\s*['"]schedule['"]""", source), \
f'{relpath} has a private core-key list again; use src/core_config_keys.py'
test/test_display_geometry.py
+51
View File
@@ -142,3 +142,54 @@ def test_preview_callers_do_not_rederive_the_size():
assert "get('chain_length', 1)" not in source, rel
assert 'get("chain_length", 1)' not in source, rel
assert 'cols * chain_length' not in source, rel
# --- Pixel mappers ----------------------------------------------------------
# RGBMatrix.width/height are measured after the library's pixel mappers
# (lib/pixel-mapper.cc), so the preview has to apply them too. This used to
# report 128x32 for a Rotate:90 chain the panel drew at 32x128.
@pytest.mark.parametrize('hardware,expected', [
({'pixel_mapper_config': 'Rotate:90'}, (32, 128)),
({'pixel_mapper_config': 'Rotate:270'}, (32, 128)),
({'pixel_mapper_config': 'Rotate:-90'}, (32, 128)),
({'pixel_mapper_config': 'Rotate:180'}, (128, 32)),
({'orientation': '90'}, (32, 128)),
({'orientation': '270'}, (32, 128)),
({'orientation': '180'}, (128, 32)),
# Two quarter turns cancel out.
({'pixel_mapper_config': 'Rotate:90', 'orientation': '270'}, (128, 32)),
# U-mapper folds a chain of 4 into two rows: (256 / 64) * 32 by 2 * 32.
({'chain_length': 4, 'pixel_mapper_config': 'U-mapper'}, (128, 64)),
({'chain_length': 4, 'pixel_mapper_config': 'u-mapper;Rotate:90'}, (64, 128)),
# U-mapper needs an even chain of at least 2, or the library skips it.
({'chain_length': 3, 'pixel_mapper_config': 'U-mapper'}, (192, 32)),
({'cols': 32, 'chain_length': 4, 'pixel_mapper_config': 'V-mapper'}, (32, 128)),
({'chain_length': 1, 'parallel': 2, 'pixel_mapper_config': 'StackToRow:Z'}, (128, 32)),
({'pixel_mapper_config': 'Remap:64,64|0,0n|0,32n'}, (64, 64)),
# A Remap panel entirely outside the visible area: the library skips it.
({'pixel_mapper_config': 'Remap:64,64|0,0n|0,99n'}, (128, 32)),
({'pixel_mapper_config': 'Mirror:H'}, (128, 32)),
# Unknown or unusable mappers are skipped by the library.
({'pixel_mapper_config': 'Bogus;Rotate:45;Rotate:ninety'}, (128, 32)),
])
def test_pixel_mappers_change_the_size_as_the_library_does(hardware, expected):
hw = {'rows': 32, 'cols': 64, 'chain_length': 2, 'parallel': 1}
hw.update(hardware)
assert physical_size(_config(hw)) == expected
assert logical_size(_config(hw)) == expected
def test_double_sided_splits_the_mapped_size():
cfg = _config({'rows': 32, 'cols': 64, 'chain_length': 2, 'parallel': 1, 'orientation': '90'},
{'enabled': True, 'copies': 2, 'axis': 'vertical'})
assert logical_size(cfg) == (32, 64)
def test_display_manager_composes_the_mapper_config_it_sizes_from():
from src.display_geometry import compose_pixel_mapper_config
assert compose_pixel_mapper_config({}) == ''
assert compose_pixel_mapper_config({'orientation': '90'}) == 'Rotate:90'
assert compose_pixel_mapper_config(
{'pixel_mapper_config': ' U-mapper ', 'orientation': '180'}) == 'U-mapper;Rotate:180'
assert compose_pixel_mapper_config({'orientation': 'sideways'}) == ''
test/test_display_manager.py
+99 -5
View File
@@ -265,6 +265,14 @@ class TestDisplayManagerOrientation:
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == ''
@pytest.mark.parametrize('orientation', ['90', '270'])
def test_sideways_orientation_appends_rotate_mapper(self, mock_rgb_matrix, orientation):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(orientation=orientation), suppress_test_pattern=True)
options = mock_rgb_matrix['options_class'].return_value
assert options.pixel_mapper_config == f'Rotate:{orientation}'
def test_orientation_180_appends_rotate_mapper(self, mock_rgb_matrix):
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
@@ -281,10 +289,11 @@ class TestDisplayManagerOrientation:
assert options.pixel_mapper_config == 'U-mapper;Rotate:180'
class TestDisplayManagerPi5Guard:
"""On a Pi 5 the library returns no matrix for settings its RP1 path can't
drive, and the binding doesn't check, so the process would crash on its next
call. DisplayManager has to refuse before creating the matrix and fall back."""
class TestDisplayManagerLibraryGuard:
"""For many settings it can't use the library returns no matrix (which the
binding doesn't check, so the process crashes on its next call) or calls
abort() -- on every board, with more on a Pi 5. DisplayManager has to refuse
before creating the matrix and fall back, reporting why."""
def _config(self, **hardware_overrides):
config = {
@@ -327,10 +336,95 @@ class TestDisplayManagerPi5Guard:
mock_rgb_matrix['matrix_class'].assert_called_once()
assert dm.matrix is not None
def test_other_boards_are_left_to_the_library(self, mock_rgb_matrix, board):
def test_pi5_only_limits_do_not_apply_to_other_boards(self, mock_rgb_matrix, board):
board('Raspberry Pi 4 Model B Rev 1.5')
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
dm = DisplayManager(self._config(row_address_type=5), suppress_test_pattern=True)
mock_rgb_matrix['matrix_class'].assert_called_once()
assert dm.matrix is not None
@pytest.fixture
def hw_status(self, tmp_path, monkeypatch):
"""What DisplayManager writes to /tmp/led_matrix_hw_status.json."""
import json as _json
import tempfile as _tempfile
from src import display_manager as dm_module
written = {}
real_replace = os.replace
real_mkstemp = _tempfile.mkstemp
def fake_mkstemp(dir=None, prefix=None):
return real_mkstemp(dir=str(tmp_path), prefix=prefix)
def fake_replace(src, dst):
if str(dst).endswith('led_matrix_hw_status.json'):
with open(src) as f:
written.update(_json.load(f))
os.remove(src)
else:
real_replace(src, dst)
monkeypatch.setattr(dm_module.tempfile, 'mkstemp', fake_mkstemp)
monkeypatch.setattr(dm_module.os, 'replace', fake_replace)
monkeypatch.setattr(dm_module.os.path, 'islink', lambda _p: False)
return written
@pytest.mark.parametrize('overrides,named', [
({'hardware_mapping': 'adafruit-hat-pwm', 'parallel': 2}, 'parallel 2'),
({'hardware_mapping': 'adafruit-hat', 'parallel': 3}, 'parallel 3'),
({'hardware_mapping': 'adafruit-hat-pwn'}, 'adafruit-hat-pwn'),
({'rows': 128}, 'rows 128'),
({'chain_length': 300}, 'chain_length 300'),
])
def test_hand_edited_setting_the_library_refuses_falls_back_on_any_board(
self, mock_rgb_matrix, board, hw_status, overrides, named):
board('Raspberry Pi 4 Model B Rev 1.5')
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
dm = DisplayManager(self._config(**overrides), suppress_test_pattern=True)
mock_rgb_matrix['matrix_class'].assert_not_called()
assert dm.matrix is None
assert hw_status['ok'] is False
assert hw_status['cause'] == 'settings'
assert named in hw_status['error']
def test_library_failure_is_reported_as_the_library(self, mock_rgb_matrix, board, hw_status):
board('Raspberry Pi 4 Model B Rev 1.5')
mock_rgb_matrix['matrix_class'].side_effect = RuntimeError('boom')
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
dm = DisplayManager(self._config(), suppress_test_pattern=True)
assert dm.matrix is None
assert hw_status == {'ok': False, 'error': 'boom', 'cause': 'library'}
def test_success_reports_no_cause(self, mock_rgb_matrix, board, hw_status):
board('Raspberry Pi 4 Model B Rev 1.5')
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'false'}):
DisplayManager(self._config(), suppress_test_pattern=True)
assert hw_status == {'ok': True, 'error': None, 'cause': None}
def test_emulator_warns_but_still_starts(self, mock_rgb_matrix, board, caplog):
board('Raspberry Pi 4 Model B Rev 1.5')
DisplayManager._instance = None
with patch.dict('os.environ', {'EMULATOR': 'true'}):
dm = DisplayManager(self._config(rows=128), suppress_test_pattern=True)
mock_rgb_matrix['matrix_class'].assert_called_once()
assert dm.matrix is not None
assert 'rows 128' in caplog.text
def test_refused_settings_advice_does_not_send_users_to_rebuild(self, board):
"""The Pi 5 rebuild / GPIO slowdown hint used to follow every failure,
including settings LEDMatrix itself refused."""
from src.matrix_support import MatrixSettingsRefused
board('Raspberry Pi 5 Model B Rev 1.0')
advice = DisplayManager._fallback_advice(
'settings', MatrixSettingsRefused('row address type 5'))
assert 'Display tab' in advice
assert 'first_time_install' not in advice and 'slowdown' not in advice
library = DisplayManager._fallback_advice('library', RuntimeError('mmap failed'))
assert 'RPI_RGB_FORCE_REBUILD=1' in library
board('Raspberry Pi 4 Model B Rev 1.5')
assert 'RPI_RGB_FORCE_REBUILD' not in DisplayManager._fallback_advice(
'library', RuntimeError('boom'))
test/test_matrix_support.py
+125
View File
@@ -0,0 +1,125 @@
"""The library-refusal rules in src/matrix_support.py mirror the pinned library.
rpi-rgb-led-matrix-master at 1ee4f76: RGBMatrix::Options::Validate in
lib/options-initialize.cc, the runtime checks in RGBMatrix::CreateFromOptions
(lib/led-matrix.cc), the mapping table in lib/hardware-mapping.c with the
abort()s in lib/framebuffer.cc, and the setter types in
bindings/python/rgbmatrix/core.pyx. When the submodule is bumped and those
change, these are the cases to revisit.
"""
import json
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).parent.parent))
from src import matrix_support as ms # noqa: E402
REPO_ROOT = Path(__file__).resolve().parents[1]
def _messages(hardware, runtime=None, pi5=False):
return [r.message for r in ms.library_refusals(hardware, runtime, pi5=pi5)]
def test_the_config_template_starts():
template = json.loads((REPO_ROOT / 'config' / 'config.template.json').read_text(encoding='utf-8'))
display = template['display']
assert ms.library_refusals(display['hardware'], display['runtime']) == []
assert ms.library_refusals(display['hardware'], display['runtime'], pi5=True) == []
def test_display_manager_defaults_start():
assert ms.library_refusals({}) == []
@pytest.mark.parametrize('hardware', [
{'rows': 8}, {'rows': 64}, {'cols': 16}, {'cols': 1024},
{'chain_length': 1}, {'chain_length': 255},
{'hardware_mapping': 'regular', 'parallel': 3},
{'hardware_mapping': 'classic', 'parallel': 3},
{'hardware_mapping': 'REGULAR', 'parallel': 2},
{'hardware_mapping': '', 'parallel': 3}, # empty reads as "regular"
{'hardware_mapping': 'classic-pi1'},
{'hardware_mapping': 'regular-pi1', 'parallel': 1},
{'pwm_bits': 11}, {'pwm_dither_bits': 2}, {'pwm_lsb_nanoseconds': 3000},
{'row_address_type': 5}, {'multiplexing': 22}, {'scan_mode': 1},
{'brightness': 1}, {'limit_refresh_rate_hz': 0},
{'led_rgb_sequence': 'bgr'}, {'led_rgb_sequence': 'GBR'},
# Not a number: left to the binding, which raises a TypeError.
{'rows': 'thirty-two'},
])
def test_what_the_library_starts_with(hardware):
assert ms.library_refusals(hardware) == []
@pytest.mark.parametrize('hardware,runtime,named', [
({'rows': 66}, None, 'rows 66'),
({'rows': 128}, None, 'rows 128'),
({'rows': 31}, None, 'rows 31'),
({'cols': 8}, None, 'cols 8'),
({'chain_length': 0}, None, 'chain_length 0'),
({'chain_length': 256}, None, 'chain_length 256'), # uint8_t setter
({'hardware_mapping': 'regular', 'parallel': 4}, None, 'parallel 4'),
({'hardware_mapping': 'adafruit-hat-pwm', 'parallel': 2}, None, 'which has 1 output'),
({'hardware_mapping': 'adafruit-hat', 'parallel': 3}, None, 'which has 1 output'),
({'hardware_mapping': 'regular-pi1', 'parallel': 2}, None, 'which has 1 output'),
({'hardware_mapping': 'classic-pi1', 'parallel': 2}, None, 'which has 1 output'),
({'parallel': 2}, None, 'adafruit-hat-pwm'), # DisplayManager's default mapping
({'hardware_mapping': 'adafruit-hat-pwn'}, None, '"adafruit-hat-pwn"'),
({'hardware_mapping': 'compute-module'}, None, '"compute-module"'),
({'hardware_mapping': None}, None, 'hardware mapping None'),
({'brightness': 0}, None, 'brightness 0'),
({'pwm_bits': 12}, None, 'pwm_bits 12'),
({'pwm_dither_bits': 3}, None, 'pwm_dither_bits 3'),
({'pwm_lsb_nanoseconds': 49}, None, 'pwm_lsb_nanoseconds 49'),
({'row_address_type': 6}, None, 'row_address_type 6'),
({'multiplexing': 23}, None, 'multiplexing 23'),
({'scan_mode': 2}, None, 'scan_mode 2'),
({'led_rgb_sequence': 'RGBW'}, None, 'LED RGB sequence'),
({'led_rgb_sequence': 'RRB'}, None, 'LED RGB sequence'),
({}, {'gpio_slowdown': 11}, 'gpio_slowdown 11'),
({}, {'gpio_slowdown': -1}, 'gpio_slowdown -1'),
({}, {'rp1_rio': 2}, 'rp1_rio 2'),
])
def test_what_it_refuses(hardware, runtime, named):
messages = _messages(hardware, runtime)
assert any(named in m for m in messages), messages
def test_pi5_limits_apply_only_on_a_pi5():
assert ms.library_refusals({'row_address_type': 5}) == []
refusals = ms.library_refusals({'row_address_type': 5}, pi5=True)
assert len(refusals) == 1 and refusals[0].pi5
assert set(refusals[0].fields) == {'row_address_type', 'parallel', 'hardware_mapping'}
def test_refusal_names_the_fields_involved():
(refusal,) = ms.library_refusals({'hardware_mapping': 'adafruit-hat', 'parallel': 2})
assert set(refusal.fields) == {'parallel', 'hardware_mapping'}
(refusal,) = ms.library_refusals({'rows': 128})
assert refusal.fields == ('rows',)
def test_message_names_every_problem():
message = ms.refusal_message(ms.library_refusals(
{'rows': 128, 'row_address_type': 5}, {'gpio_slowdown': 11}, pi5=True))
assert message.startswith("The installed rgbmatrix library can't start")
assert 'rows 128' in message and 'gpio_slowdown 11' in message
assert 'Raspberry Pi 5' in message and 'row address type 5' in message
assert ms.refusal_message([]) is None
def test_non_mapping_sections_are_treated_as_empty():
assert ms.library_refusals('oops', ['a']) == []
def test_display_manager_defaults_match_display_manager():
"""The guard fills missing keys the way DisplayManager._setup_matrix does."""
source = (REPO_ROOT / 'src' / 'display_manager.py').read_text(encoding='utf-8')
for field, value in ms.DISPLAY_MANAGER_DEFAULTS.items():
if field in ('rows', 'cols', 'chain_length', 'parallel'):
continue # read through display_geometry's DEFAULT_* constants
assert f"get('{field}', {value!r})" in source, field
test/test_path_traversal_guards.py
+199
View File
@@ -315,3 +315,202 @@ class TestPluginVersionLookup:
assert module._get_plugin_version("..") == ""
finally:
module.api_v3.plugin_store_manager = original
PNG = b"\x89PNG\r\n\x1a\n" + b"0" * 20
class TestPluginAssetRoutes:
"""/api/v3/plugins/assets/upload, /list and /delete
All three joined a request-supplied plugin_id straight onto
assets/plugins, so '../../config' created an uploads directory under
config/, wrote images and .metadata.json there, listed it, and deleted
whatever file a metadata entry's path named. #561 fixed only the route
that serves the uploaded files.
"""
@pytest.fixture
def project(self, tmp_path, api_v3_module, monkeypatch):
import web_interface.blueprints.api_v3.plugins as plugins_module
monkeypatch.setattr(plugins_module, "PROJECT_ROOT", tmp_path)
(tmp_path / "config").mkdir()
secrets = tmp_path / "config" / "config_secrets.json"
secrets.write_text('{"api_key":"hunter2"}', encoding="utf-8")
return tmp_path, secrets
def _upload(self, client, plugin_id):
import io
return client.post(
"/api/v3/plugins/assets/upload",
data={"plugin_id": plugin_id, "files": (io.BytesIO(PNG), "x.png")},
content_type="multipart/form-data",
)
def _tree(self, root):
return sorted(str(p.relative_to(root)) for p in root.rglob("*"))
def test_an_ordinary_upload_list_and_delete_still_work(self, api_v3_client, project):
root, _ = project
response = self._upload(api_v3_client, "static-image")
assert response.status_code == 200, response.get_json()
uploaded = response.get_json()["uploaded_files"][0]
assert uploaded["path"].startswith("assets/plugins/static-image/uploads/")
stored = root / uploaded["path"]
assert stored.exists()
listed = api_v3_client.get(
"/api/v3/plugins/assets/list", query_string={"plugin_id": "static-image"}
)
assert listed.status_code == 200
assert [a["id"] for a in listed.get_json()["data"]["assets"]] == [uploaded["id"]]
deleted = api_v3_client.post(
"/api/v3/plugins/assets/delete",
json={"plugin_id": "static-image", "image_id": uploaded["id"]},
)
assert deleted.status_code == 200
assert not stored.exists()
@pytest.mark.parametrize("plugin_id", [
"../../config", "..", "a/b", "/abs", "x" + BACKSLASH + "y", "C:",
])
def test_a_traversing_upload_writes_nothing(self, api_v3_client, project, plugin_id):
root, _ = project
before = self._tree(root)
response = self._upload(api_v3_client, plugin_id)
assert response.status_code == 400
assert self._tree(root) == before
def test_a_traversing_list_reads_nothing(self, api_v3_client, project):
root, _ = project
outside = root / "config" / "uploads"
outside.mkdir()
(outside / ".metadata.json").write_text(
json.dumps({"i": {"id": "i", "path": "leaked"}}), encoding="utf-8"
)
response = api_v3_client.get(
"/api/v3/plugins/assets/list", query_string={"plugin_id": "../../config"}
)
assert response.status_code == 400
assert b"leaked" not in response.data
def test_a_traversing_delete_deletes_nothing(self, api_v3_client, project):
root, secrets = project
outside = root / "config" / "uploads"
outside.mkdir()
(outside / ".metadata.json").write_text(
json.dumps({"i": {"id": "i", "path": "config/config_secrets.json"}}),
encoding="utf-8",
)
response = api_v3_client.post(
"/api/v3/plugins/assets/delete",
json={"plugin_id": "../../config", "image_id": "i"},
)
assert response.status_code == 400
assert secrets.exists(), "file outside assets/plugins was deleted"
@pytest.mark.parametrize("stored_path", [
"config/config_secrets.json",
"assets/plugins/static-image/uploads/../../../../config/config_secrets.json",
"assets/plugins/other/uploads/x.png",
None,
])
def test_a_metadata_path_outside_the_uploads_is_never_unlinked(
self, api_v3_client, project, stored_path
):
root, secrets = project
uploads = root / "assets" / "plugins" / "static-image" / "uploads"
uploads.mkdir(parents=True)
other = root / "assets" / "plugins" / "other" / "uploads"
other.mkdir(parents=True)
(other / "x.png").write_bytes(PNG)
(uploads / ".metadata.json").write_text(
json.dumps({"i": {"id": "i", "path": stored_path}}), encoding="utf-8"
)
response = api_v3_client.post(
"/api/v3/plugins/assets/delete",
json={"plugin_id": "static-image", "image_id": "i"},
)
assert response.status_code == 200
assert secrets.exists(), "file named by tampered metadata was deleted"
assert (other / "x.png").exists(), "another plugin's upload was deleted"
# The bad entry is still dropped, so the UI can get rid of it.
assert json.loads((uploads / ".metadata.json").read_text(encoding="utf-8")) == {}
class TestPluginActionDirectory:
"""POST /api/v3/plugins/action runs a script named by the manifest in
get_plugin_directory(plugin_id), and plugin_id is the request body's.
get_plugin_directory joined it onto plugins_dir unchecked, so
'../elsewhere' ran a script from any directory holding a manifest.
"""
@pytest.fixture
def tree(self, tmp_path):
import threading
from src.plugin_system.plugin_manager import PluginManager
plugins = tmp_path / "plugin-repos"
(plugins / "demo").mkdir(parents=True)
(plugins / "ledmatrix-legacy").mkdir()
outside = tmp_path / "elsewhere"
outside.mkdir()
(outside / "manifest.json").write_text(json.dumps({
"web_ui_actions": [{"id": "go", "type": "script", "script": "s.py"}],
}), encoding="utf-8")
marker = tmp_path / "PWNED"
(outside / "s.py").write_text(
"open(%r, 'w').write('ran')\n" % str(marker), encoding="utf-8"
)
manager = MagicMock()
manager.plugins_dir = plugins
manager._discovery_lock = threading.Lock()
manager.plugin_directories = {}
manager.get_plugin_directory = (
lambda pid: PluginManager.get_plugin_directory(manager, pid)
)
return manager, plugins, marker
def test_real_plugin_directories_are_still_found(self, tree):
manager, plugins, _ = tree
assert manager.get_plugin_directory("demo") == str(plugins / "demo")
assert manager.get_plugin_directory("legacy") == str(plugins / "ledmatrix-legacy")
assert manager.get_plugin_directory("nope") is None
def test_a_discovered_plugin_is_returned_from_the_registry(self, tree, tmp_path):
manager, _, _ = tree
manager.plugin_directories = {"linked": tmp_path / "somewhere"}
assert manager.get_plugin_directory("linked") == str(tmp_path / "somewhere")
@pytest.mark.parametrize("plugin_id", [
"..", "../elsewhere", "a/b", "/abs", "x" + BACKSLASH + "..", "",
])
def test_get_plugin_directory_refuses_anything_but_a_plain_name(self, tree, plugin_id):
manager, _, _ = tree
assert manager.get_plugin_directory(plugin_id) is None
def test_the_action_endpoint_runs_nothing_outside_the_plugins_dir(
self, api_v3_client, api_v3_module, tree
):
manager, _, marker = tree
api_v3_module.api_v3.plugin_manager = manager
response = api_v3_client.post(
"/api/v3/plugins/action",
json={"plugin_id": "../elsewhere", "action_id": "go", "params": {}},
)
assert response.status_code == 400
assert not marker.exists(), "script outside the plugins directory ran"
def test_the_fallback_without_a_plugin_manager_is_guarded_too(
self, api_v3_client, api_v3_module
):
api_v3_module.api_v3.plugin_manager = None
response = api_v3_client.post(
"/api/v3/plugins/action",
json={"plugin_id": "../elsewhere", "action_id": "go", "params": {}},
)
assert response.status_code == 400
test/test_plugin_config_preparation.py
+205
View File
@@ -0,0 +1,205 @@
"""One preparation for a plugin's config, wherever the config comes from.
A plugin's stored section becomes the config it runs with through
schema_manager.prepare_plugin_config: legacy booleans (#588) read as
``{"enabled": ...}`` objects, then schema and core defaults filled in. Loading
did that; hot reload handed plugins the raw section instead (a legacy
``dynamic_duration: true`` came back as a boolean, turning dynamic duration
off), and the dev tools each built configs their own way:
- dev_server read only top-level defaults and let a schema ``enabled: false``
override its forced ``enabled: True``;
- check_plugin/render_plugin (build_full_config) merged overrides with
dict.update, so ``{"nhl": {"enabled": true}}`` dropped every other nhl
default;
- the harness and the device disagreed on object and array defaults.
The web saves and GET are covered in
test/web_interface/test_plugin_config_json_saves.py.
"""
import importlib.util
import json
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from src.plugin_system.plugin_manager import PluginManager
from src.plugin_system.schema_manager import (
CORE_PLUGIN_PROPERTIES, SchemaManager, extract_schema_defaults,
)
from src.plugin_system.testing import loading
REPO = Path(__file__).resolve().parent.parent
SCHEMA = {
"type": "object",
"properties": {
"enabled": {"type": "boolean", "default": False},
"global": {
"type": "object",
"properties": {
"dynamic_duration": {
"type": "object",
"properties": {
"enabled": {"type": "boolean", "default": True},
"max_duration_seconds": {"type": "integer", "default": 300},
},
},
},
},
"nhl": {
"type": "object",
"properties": {
"enabled": {"type": "boolean", "default": False},
"favorite_teams": {"type": "array", "default": ["TB"]},
"show_records": {"type": "boolean", "default": True},
},
},
"colors": {
"type": "object",
"default": {"text": [255, 255, 255]},
"properties": {"text": {"type": "array", "default": [1, 2, 3]}},
},
"feeds": {"type": "array", "items": {"type": "string"}},
},
}
@pytest.fixture
def plugin_dir(tmp_path):
pdir = tmp_path / "plugins" / "demo"
pdir.mkdir(parents=True)
(pdir / "config_schema.json").write_text(json.dumps(SCHEMA))
(pdir / "manifest.json").write_text(json.dumps({"id": "demo"}))
return pdir
@pytest.fixture
def plugin_manager(plugin_dir, tmp_path):
manager = PluginManager.__new__(PluginManager) # skip the heavy constructor
manager.logger = MagicMock()
manager.schema_manager = SchemaManager(plugins_dir=plugin_dir.parent, project_root=tmp_path)
return manager
class TestPluginManagerPreparation:
def test_legacy_boolean_and_defaults(self, plugin_manager):
prepared = plugin_manager.prepare_plugin_config(
"demo", {"enabled": True, "global": {"dynamic_duration": True}})
assert prepared["global"]["dynamic_duration"] == {
"enabled": True, "max_duration_seconds": 300}
assert prepared["nhl"]["favorite_teams"] == ["TB"]
assert prepared["display_duration"] == 15
def test_does_not_mutate_the_raw_section(self, plugin_manager):
raw = {"global": {"dynamic_duration": True}}
plugin_manager.prepare_plugin_config("demo", raw)
assert raw == {"global": {"dynamic_duration": True}}
def test_never_raises(self, plugin_manager):
plugin_manager.schema_manager = MagicMock()
plugin_manager.schema_manager.load_schema.return_value = SCHEMA
plugin_manager.schema_manager.prepare_plugin_config.side_effect = RuntimeError("boom")
prepared = plugin_manager.prepare_plugin_config("demo", {"global": {"dynamic_duration": False}})
assert prepared["global"]["dynamic_duration"] == {"enabled": False}
class TestHotReload:
def test_on_config_change_gets_the_prepared_config(self, test_display_controller, plugin_manager):
controller = test_display_controller
plugin = MagicMock()
plugin.modes = ["demo"]
pm = controller.plugin_manager
pm.discover_plugins.return_value = ["demo"]
pm.load_plugin.return_value = True
pm.plugin_manifests = {}
pm.get_plugin.side_effect = lambda pid: plugin if pid == "demo" else None
pm.prepare_plugin_config.side_effect = plugin_manager.prepare_plugin_config
controller.config_service.get_config = lambda: {"demo": {"enabled": True}}
controller._reconcile_enabled_plugins()
callback = controller._plugin_config_callbacks["demo"]
callback({"enabled": True}, {"enabled": True, "global": {"dynamic_duration": True}})
new_config = plugin.on_config_change.call_args[0][0]
assert new_config["global"]["dynamic_duration"] == {
"enabled": True, "max_duration_seconds": 300}
assert new_config["nhl"]["show_records"] is True
def test_raw_section_still_delivered_without_a_preparer(self, test_display_controller):
controller = test_display_controller
plugin = MagicMock()
plugin.modes = ["demo"]
pm = controller.plugin_manager
pm.discover_plugins.return_value = ["demo"]
pm.load_plugin.return_value = True
pm.plugin_manifests = {}
pm.get_plugin.side_effect = lambda pid: plugin if pid == "demo" else None
pm.prepare_plugin_config.return_value = None
controller.config_service.get_config = lambda: {"demo": {"enabled": True}}
controller._reconcile_enabled_plugins()
raw = {"enabled": False}
controller._plugin_config_callbacks["demo"]({}, raw)
plugin.on_config_change.assert_called_once_with(raw)
class TestDevToolsMatchTheDevice:
def test_harness_defaults_are_the_device_defaults(self, plugin_dir):
assert loading.load_config_defaults(plugin_dir) == extract_schema_defaults(SCHEMA)
defaults = loading.load_config_defaults(plugin_dir)
# An object's own default wins, as on a device; arrays start empty
assert defaults["colors"] == {"text": [255, 255, 255]}
assert defaults["feeds"] == []
def test_nested_override_keeps_sibling_defaults(self, plugin_dir):
config = loading.build_full_config(plugin_dir, cli_config={"nhl": {"enabled": True}})
assert config["nhl"] == {"enabled": True, "favorite_teams": ["TB"], "show_records": True}
def test_spec_and_cli_overrides_both_deep_merge(self, plugin_dir):
config = loading.build_full_config(
plugin_dir, spec={"config": {"nhl": {"show_records": False}}},
cli_config={"nhl": {"enabled": True}})
assert config["nhl"] == {"enabled": True, "favorite_teams": ["TB"], "show_records": False}
def test_build_config_matches_the_device_load(self, plugin_dir, plugin_manager):
overrides = {"enabled": True, "global": {"dynamic_duration": False}}
assert loading.build_config(plugin_dir, overrides) == \
plugin_manager.prepare_plugin_config("demo", overrides)
def test_core_defaults_are_present(self, plugin_dir):
config = loading.build_full_config(plugin_dir)
assert config["enabled"] is True
assert config["display_duration"] == CORE_PLUGIN_PROPERTIES["display_duration"]["default"]
assert config["live_priority"] is False
def test_render_plugin_matrix_config(self, plugin_dir, monkeypatch):
from src.plugin_system.testing import harness
seen = {}
def fake_render_size(plugin_id, manifest, pdir, config, *args, **kwargs):
seen["config"] = config
return []
monkeypatch.setattr(harness, "_render_size", fake_render_size)
harness.render_plugin_matrix("demo", plugin_dir, config={"nhl": {"enabled": True}},
sizes=[(64, 32)], run_update=False)
assert seen["config"]["enabled"] is True, "a schema enabled:false must not win"
assert seen["config"]["nhl"]["favorite_teams"] == ["TB"]
def test_dev_server_render_config(self, plugin_dir, monkeypatch):
spec = importlib.util.spec_from_file_location("dev_server_under_test",
REPO / "scripts" / "dev_server.py")
dev_server = importlib.util.module_from_spec(spec)
spec.loader.exec_module(dev_server)
monkeypatch.setattr(dev_server, "find_plugin_dir", lambda pid: plugin_dir)
monkeypatch.setattr(dev_server, "_trusted_plugin_dir", lambda d: plugin_dir)
_, _, config, _, _ = dev_server._parse_render_request(
{"plugin_id": "demo", "config": {"nhl": {"enabled": True}}})
assert config["enabled"] is True, "a schema enabled:false must not win"
assert config["nhl"] == {"enabled": True, "favorite_teams": ["TB"], "show_records": True}
assert config["global"]["dynamic_duration"]["max_duration_seconds"] == 300
assert dev_server.load_config_defaults(plugin_dir) == extract_schema_defaults(SCHEMA)
test/test_plugin_system_legacy_flags.py
+70
View File
@@ -0,0 +1,70 @@
"""plugin_system.auto_discover / auto_load_enabled / development_mode.
The General tab offered three toggles for these, with help tips promising
"plugins installed but dormant" and "verbose logging". Nothing in src/,
web_interface/ or scripts/ reads them: every enabled plugin is discovered and
loaded regardless. The toggles are gone; the keys stay tolerated in stored
configs, and saving the General tab must not rewrite them.
"""
import copy
import json
import sys
from pathlib import Path
import pytest
PROJECT_ROOT = Path(__file__).parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
from test.test_web_settings_ui import REALISTIC_CONFIG, client # noqa: F401,E402
LEGACY_FLAGS = ('auto_discover', 'auto_load_enabled', 'development_mode')
def test_the_general_tab_no_longer_offers_the_toggles(client):
body = client.get('/v3/partials/general').get_data(as_text=True)
for flag in LEGACY_FLAGS:
assert f'name="{flag}"' not in body, flag
assert f'setting-general-{flag}' not in body, flag
# The setting that does work stays.
assert 'name="plugins_directory"' in body
@pytest.fixture
def saved(api_v3_module, monkeypatch):
captured = {}
stored = copy.deepcopy(REALISTIC_CONFIG)
stored['plugin_system'].update(
auto_discover=True, auto_load_enabled=True, development_mode=True)
api_v3_module.api_v3.config_manager.load_config.return_value = stored
def fake_save(_manager, config, **_kwargs):
captured['config'] = config
return True, ''
monkeypatch.setattr(api_v3_module, '_save_config_atomic', fake_save)
return captured
def test_saving_the_general_form_leaves_stored_flags_alone(api_v3_client, saved):
# What the General form posts now: no checkbox fields for the flags.
resp = api_v3_client.post('/api/v3/config/main', data={
'timezone': 'America/Chicago', 'city': 'Dallas', 'state': 'Texas',
'country': 'US', 'plugins_directory': 'plugin-repos',
})
assert resp.status_code == 200, resp.get_json()
plugin_system = saved['config']['plugin_system']
# Missing used to mean "unchecked" and saved all three as false.
assert all(plugin_system[flag] is True for flag in LEGACY_FLAGS), plugin_system
assert plugin_system['plugins_directory'] == 'plugin-repos'
def test_an_api_client_can_still_store_a_flag(api_v3_client, saved):
resp = api_v3_client.post('/api/v3/config/main', data=json.dumps({
'timezone': 'America/Chicago', 'development_mode': False,
}), content_type='application/json')
assert resp.status_code == 200, resp.get_json()
plugin_system = saved['config']['plugin_system']
assert plugin_system['development_mode'] is False
assert plugin_system['auto_discover'] is True
test/test_safe_pip_install_allowlist.py
+128
View File
@@ -0,0 +1,128 @@
"""scripts/fix_perms/safe_pip_install.sh must accept what the updaters install.
Update Code and the automatic update's health check install the core's own
requirement files through that root wrapper, and the wrapper refuses any path
it does not list. It used to list only requirements.txt, so an update that
changed web_interface/requirements.txt failed its dependency install -- and
the automatic updater rolls back any update whose dependencies did not
install, on every device, every week.
The real script runs here with only its final pip command swapped for an
echo, so the path checks under test are the shipped ones.
"""
import importlib.util
import os
import shutil
import subprocess
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
WRAPPER_REL = Path('scripts') / 'fix_perms' / 'safe_pip_install.sh'
PIP_LINE = 'exec "$PYTHON_PATH" -m pip install'
def _bash():
"""A bash whose realpath understands --canonicalize-missing, or None."""
candidates = [shutil.which('bash')]
if os.name == 'nt':
candidates.insert(0, r'C:\Program Files\Git\bin\bash.exe')
for bash in candidates:
if not bash or not os.path.exists(bash):
continue
try:
probe = subprocess.run([bash, '-c', 'realpath --canonicalize-missing /no/such/x'],
capture_output=True, text=True, timeout=30)
except (OSError, subprocess.SubprocessError):
continue
if probe.returncode == 0:
return bash
return None
BASH = _bash()
pytestmark = pytest.mark.skipif(BASH is None, reason='no bash with GNU realpath')
@pytest.fixture
def project(tmp_path):
"""A project tree holding the real wrapper, with pip stubbed out."""
root = tmp_path / 'LEDMatrix'
wrapper = root / WRAPPER_REL
wrapper.parent.mkdir(parents=True)
text = (ROOT / WRAPPER_REL).read_text(encoding='utf-8').replace('\r\n', '\n')
lines = text.split('\n')
pip = [i for i, line in enumerate(lines) if line.startswith(PIP_LINE)]
assert len(pip) == 1, 'the wrapper no longer ends in the pip install this test stubs'
lines[pip[0]] = 'echo "WOULD INSTALL $RESOLVED_TARGET"; exit 0'
wrapper.write_text('\n'.join(lines), encoding='utf-8', newline='\n')
for rel in ('requirements.txt', 'web_interface/requirements.txt',
'plugin-repos/clock/requirements.txt', 'src/requirements.txt',
'web_interface/extra/requirements.txt'):
(root / rel).parent.mkdir(parents=True, exist_ok=True)
(root / rel).write_text('requests\n', encoding='utf-8')
return root
def run_wrapper(root, rel):
"""Run the wrapper on ``rel``, with the path spelled the way bash sees the tree."""
return subprocess.run(
[BASH, '-c', 'cd "$1" && bash scripts/fix_perms/safe_pip_install.sh "$PWD/$2"',
'_', str(root), rel],
capture_output=True, text=True, timeout=60)
def _core_requirement_files():
from web_interface.blueprints.api_v3 import system
spec = importlib.util.spec_from_file_location(
'auto_update_verify_for_allowlist', ROOT / 'scripts' / 'utils' / 'auto_update_verify.py')
verifier = importlib.util.module_from_spec(spec)
spec.loader.exec_module(verifier)
assert set(verifier.REQUIREMENT_FILES) == set(system.CORE_REQUIREMENT_FILES), (
'the rollback must reinstall the same files Update Code installs')
return system.CORE_REQUIREMENT_FILES
@pytest.mark.parametrize('rel', _core_requirement_files())
def test_every_core_requirement_file_the_updaters_install_is_allowed(project, rel):
result = run_wrapper(project, rel)
assert result.returncode == 0, result.stderr
assert 'WOULD INSTALL' in result.stdout
def test_plugin_requirements_are_still_allowed(project):
assert run_wrapper(project, 'plugin-repos/clock/requirements.txt').returncode == 0
@pytest.mark.parametrize('rel', [
'src/requirements.txt', # repo-owned folder, but not listed
'web_interface/extra/requirements.txt', # below an allowed file's folder
'web_interface/requirements.txt.bak', # not a requirements.txt
])
def test_other_paths_are_still_refused(project, rel):
if rel.endswith('.bak'):
(project / rel).write_text('requests\n', encoding='utf-8')
result = run_wrapper(project, rel)
assert result.returncode == 2, result.stdout + result.stderr
assert 'DENIED' in result.stderr
@pytest.mark.parametrize('rel', ['requirements.txt', 'web_interface/requirements.txt'])
def test_a_core_requirement_file_symlinked_out_of_the_project_is_refused(project, tmp_path, rel):
"""Only the allowed files' folders are resolved, so the link's target is
compared, and refused, rather than allowed as the file itself."""
outside = tmp_path / 'elsewhere' / 'requirements.txt'
outside.parent.mkdir()
outside.write_text('evil\n', encoding='utf-8')
link = project / rel
link.unlink()
try:
link.symlink_to(outside)
except (OSError, NotImplementedError):
pytest.skip('symlinks unavailable')
result = run_wrapper(project, rel)
assert result.returncode == 2, result.stdout + result.stderr
test/test_scroll_helper.py
+13
View File
@@ -460,6 +460,19 @@ class TestIdleGapIsNotAFrame:
assert not info.called
assert len(helper._window) == 1
def test_a_dropped_gap_restarts_the_window_timer_too(self, helper):
"""The size-guard path is the one scrollers that never call
reset_scroll() take, so it must restart the window like the sentinel
does, or their next scroll opens with a one-frame stats line."""
helper.log_frame_rate() # seeds
helper.last_frame_time = time.time() - 137.0
helper.last_fps_log_time = 0.0 # boundary long overdue
with patch.object(helper.logger, "info") as info:
helper.log_frame_rate() # the gap, dropped
helper.log_frame_rate() # first real frame
assert not info.called, "a one-frame window was reported"
assert len(helper._window) == 1
def test_a_normal_frame_still_counts(self, helper):
helper.last_frame_time = time.time() - 0.010
helper.log_frame_rate()
test/test_scroll_speeds_script.py
+144
View File
@@ -0,0 +1,144 @@
"""scripts/scroll_speeds.py must drive the panel the way the service does.
--measure and --demo open the matrix themselves. They used to build the
options from a private copy of DisplayManager's builder that had drifted: it
read gpio_slowdown from display.hardware (the service reads display.runtime),
and skipped rp1_rio, panel_type, orientation and more. On a panel that needs a
high slowdown that measured -- or garbled -- a panel the service never drives.
"""
import importlib.util
import os
from pathlib import Path
from unittest.mock import patch
import pytest
os.environ.setdefault("EMULATOR", "true")
import src.display_manager as display_manager # noqa: E402
from src.display_manager import DisplayManager # noqa: E402
SCRIPT = Path(__file__).resolve().parents[1] / "scripts" / "scroll_speeds.py"
@pytest.fixture(scope="module")
def script():
spec = importlib.util.spec_from_file_location("scroll_speeds_script", SCRIPT)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class _Options:
"""Records what is set; declares rp1_rio like a Pi 5-capable binding."""
rp1_rio = None
CONFIG = {
"display": {
"hardware": {
"rows": 64, "cols": 64, "chain_length": 2, "parallel": 1,
"hardware_mapping": "adafruit-hat-pwm",
"brightness": 70, "pwm_bits": 9, "pwm_lsb_nanoseconds": 130,
"row_address_type": 5, "panel_type": "FM6126A",
"limit_refresh_rate_hz": 120, "orientation": "180",
"pixel_mapper_config": "U-mapper",
"gpio_slowdown": 2, # a stale key in the wrong block
},
"runtime": {"gpio_slowdown": 7, "rp1_rio": 1},
}
}
def _script_options(script, config, **kwargs):
with patch.object(display_manager, "RGBMatrixOptions", _Options):
return script.build_options(config, **kwargs)
def test_runtime_settings_are_used(script):
options = _script_options(script, CONFIG)
assert options.gpio_slowdown == 7
assert options.rp1_rio == 1
assert options.panel_type == "FM6126A"
assert options.row_address_type == 5
assert options.pixel_mapper_config == "U-mapper;Rotate:180"
assert options.limit_refresh_rate_hz == 120
def test_the_options_match_the_display_service(script):
"""Same config in, same options out, attribute for attribute."""
service = DisplayManager.apply_matrix_options(_Options(), CONFIG)
assert vars(_script_options(script, CONFIG)) == vars(service)
def test_measure_runs_uncapped(script):
assert _script_options(script, CONFIG, refresh_override=0).limit_refresh_rate_hz == 0
def test_an_empty_config_gets_the_service_defaults(script):
options = _script_options(script, {})
assert vars(options) == vars(DisplayManager.apply_matrix_options(_Options(), {}))
assert options.gpio_slowdown == 3
def test_the_service_uses_the_shared_builder(test_config):
"""DisplayManager._setup_matrix fills its options through the same call."""
with patch.object(display_manager, "RGBMatrix"), \
patch.object(display_manager, "RGBMatrixOptions", _Options), \
patch.object(display_manager, "freetype"), \
patch.object(DisplayManager, "apply_matrix_options",
wraps=DisplayManager.apply_matrix_options) as shared, \
patch.dict(os.environ, {"EMULATOR": "false"}):
DisplayManager._instance = None
try:
DisplayManager(test_config)
finally:
DisplayManager._instance = None
shared.assert_called_once()
class TestSpeedAdvice:
"""The advice must name keys the resolver actually honours."""
def _advice(self, script, capsys, hz, want):
script.print_ladder(hz, want)
return capsys.readouterr().out
def test_advice_is_the_speed_delay_pair(self, script, capsys):
out = self._advice(script, capsys, 100.0, 60)
assert '"scroll_speed": 2' in out
assert '"scroll_delay": 0.03' in out
@pytest.mark.parametrize("hz,want", [(100.0, 60), (100.0, 50), (120.0, 45),
(60.0, 30), (100.0, None)])
def test_the_advised_pair_resolves_to_the_advised_speed(self, script, capsys,
hz, want):
"""Apply the printed pair over a schema-default pair, as a saved config
would carry it, and the resolver must land on the advertised speed."""
import json
import re
from src.common import scroll_config
out = self._advice(script, capsys, hz, want)
block = re.search(r'"display_options": (\{[^}]*\})', out)
assert block, out
advised = json.loads(block.group(1))
config = {"display_options": {"scroll_speed": 1.0, "scroll_delay": 0.02,
"scroll_pixels_per_second": 999,
**advised}}
expected = scroll_config.solve_crisp(want if want else hz / 2, hz)
settings = scroll_config.configure(
_Helper(), plugin_config=config, refresh_hz=hz)
assert settings.pixels_per_second == pytest.approx(expected.pixels_per_second)
assert settings.frame_hold == expected.frame_hold
def test_the_deprecated_key_is_not_recommended(self, script, capsys):
out = self._advice(script, capsys, 100.0, 60)
assert '"scroll_pixels_per_second":' not in out
class _Helper:
def set_scroll_speed(self, speed):
pass
test/test_sports_scroll.py
+123 -45
View File
@@ -6,8 +6,11 @@ is shared, the content layer is not. Two things are worth asserting beyond
* ``prepare_scroll_content`` must stay an override point — a base class that
quietly rendered *something* would let a plugin ship a blank scroll.
* ``target_fps`` must actually reach the helper. That is the whole reason this
module exists upstream; the bundled plugin copies hardcode ~100 FPS.
* Speed must depend only on ``scroll_speed`` and the panel refresh. The helper
steps a fixed number of whole pixels per presented frame and the panel
presents at its refresh divided by the frame hold, so a ladder computed for
any other rate -- the global ``target_fps`` used to be one -- plays back at
the wrong speed.
"""
import logging
@@ -221,12 +224,14 @@ class TestConfigureScrollHelper:
bare = SportsScrollDisplay.__new__(SportsScrollDisplay)
assert SportsScrollDisplay._scroll_frame_hold(bare) == 1
def test_stepping_is_time_based(self, build):
"""Frame-based mode gated motion on a wall clock at 1/scroll_delay
steps, with scroll_delay set to the frame period -- putting the
comparison exactly on its own threshold, so it flipped on sub-
millisecond jitter."""
build().scroll_helper.set_frame_based_scrolling.assert_called_once_with(False)
def test_helper_steps_whole_pixels_per_presented_frame(self, build):
"""No wall clock: the helper advances the ladder's whole-pixel step on
every presented frame, and the frame hold sets how often that is."""
display = build()
helper = display.scroll_helper
helper.set_frame_based_scrolling.assert_called_once_with(False)
helper.set_pixels_per_frame.assert_called_once_with(
display._scroll_settings.crisp.pixels_per_frame)
def test_dynamic_duration_settings_are_applied(self, build):
display = build({"nhl": {"scroll_settings": {
@@ -243,31 +248,65 @@ class TestConfigureScrollHelper:
applied = display.scroll_helper.set_scroll_speed.call_args[0][0]
assert applied == pytest.approx(100.0, abs=1.0)
def test_global_target_fps_sets_the_refresh_the_ladder_uses(self, build):
"""Under the old model this key was the rate frames were presented at,
so it is the faithful translation of it into a panel refresh."""
display = build(global_config={"target_fps": 60})
assert display._resolve_refresh_hz() == 60.0
@pytest.mark.parametrize("global_config", [
{},
{"target_fps": 60},
{"target_fps": 100},
{"target_fps": 120},
{"target_fps": 200},
{"scroll_target_fps": 90},
{"target_fps": 120, "scroll_target_fps": 90},
])
def test_target_fps_does_not_change_scroll_speed(self, build, global_config):
"""The General tab's "Scroll Frame Rate" is not a speed control.
def test_legacy_key_is_honored(self, build):
display = build(global_config={"scroll_target_fps": 90})
assert display._resolve_refresh_hz() == 90.0
It used to set the refresh the ladder was computed against, while the
panel kept presenting at its real 100Hz: 60 doubled every scoreboard's
speed and 200 halved it."""
display = build(global_config=global_config)
assert display._resolve_refresh_hz() == 100.0
assert display._scroll_settings.pixels_per_second == pytest.approx(50.0)
assert display._scroll_settings.frame_hold == 2
assert display._scroll_settings.crisp.pixels_per_frame == 1
def test_modern_key_wins_over_legacy(self, build):
display = build(global_config={"target_fps": 120, "scroll_target_fps": 90})
assert display._resolve_refresh_hz() == 120.0
def test_configured_hardware_refresh_wins_over_the_fps_target(self, build):
def test_configured_hardware_refresh_sets_the_ladder(self, build):
display = build(global_config={
"target_fps": 60,
"display": {"hardware": {"limit_refresh_rate_hz": 120}}})
assert display._resolve_refresh_hz() == 120.0
assert display._scroll_settings.crisp.refresh_hz == 120.0
def test_display_manager_refresh_wins_over_global_config(self, build,
display_manager):
"""The display manager reports the rate the panel is driven at."""
display_manager.refresh_hz = 60.0
display = build(global_config={
"target_fps": 200,
"display": {"hardware": {"limit_refresh_rate_hz": 120}}})
assert display._resolve_refresh_hz() == 60.0
@pytest.mark.parametrize("reported", [True, 0, -5, "100", None])
def test_unusable_display_manager_refresh_falls_back(self, build,
display_manager,
reported):
display_manager.refresh_hz = reported
display = build(global_config={
"display": {"hardware": {"limit_refresh_rate_hz": 120}}})
assert display._resolve_refresh_hz() == 120.0
@pytest.mark.parametrize("bad", ["fast", None, {}, [], "", 0])
def test_unusable_target_fps_degrades_instead_of_raising(self, build, bad):
"""A malformed global config must cost the FPS target, not the display."""
def test_unusable_target_fps_is_harmless(self, build, bad):
"""A malformed global config must not cost the display."""
display = build(global_config={"target_fps": bad})
assert display._scroll_settings.pixels_per_second > 0
assert display._scroll_settings.pixels_per_second == pytest.approx(50.0)
@pytest.mark.parametrize("delay", [0.001, 0.01, 0.05, 0.1])
def test_scroll_delay_is_ignored_for_pacing(self, build, delay):
"""Kept in the settings for compatibility; it does not change speed."""
display = build({"nhl": {"scroll_settings": {
"scroll_speed": 50.0, "scroll_delay": delay}}})
assert display._scroll_settings.pixels_per_second == pytest.approx(50.0)
assert display._scroll_settings.frame_hold == 2
@pytest.mark.parametrize("bad", [None, "fast", {}, []])
def test_unusable_scroll_speed_degrades_instead_of_crashing(self, build, bad):
@@ -475,7 +514,9 @@ class TestLifecycle:
class TestManager:
@pytest.fixture
def manager(self, display_manager):
return _Manager(display_manager, {}, LOGGER, global_config={"target_fps": 120})
return _Manager(display_manager, {}, LOGGER, global_config={
"target_fps": 120,
"display": {"hardware": {"limit_refresh_rate_hz": 120}}})
def test_displays_are_created_lazily_and_reused(self, manager):
first = manager.get_scroll_display("live")
@@ -487,15 +528,13 @@ class TestManager:
"recent")
def test_global_config_is_threaded_to_children(self, manager):
"""A missed hand-off here is exactly how the plugin copies ended up
never honoring target_fps."""
"""The child needs the global config to find the panel refresh when
the display manager cannot report one."""
child = manager.get_scroll_display("live")
assert child.global_config == {"target_fps": 120}
# The target is now the refresh the crisp ladder is computed against,
# so what proves the hand-off is that the child reasons about 120Hz --
# not the presentation rate, which the chosen hold divides down.
assert child.global_config["target_fps"] == 120
# What proves the hand-off is that the child reasons about the 120Hz
# hardware refresh from that config (target_fps plays no part).
assert child._resolve_refresh_hz() == 120.0
child.scroll_helper.set_target_fps.assert_called_once()
def test_prepare_sets_the_active_type(self, manager):
assert manager.prepare_and_display([{"id": "g1"}], "live", ["nhl"]) is True
@@ -619,15 +658,17 @@ class TestAgainstTheRealScrollHelper:
def test_configuration_lands_on_the_real_helper(self, real):
display = real.get_scroll_display("live")
helper = display.scroll_helper
# 500 px/s on a 120Hz panel snaps to a whole number of pixels per
# refresh, and the helper is driven in px/s in time-based mode.
# 500 px/s on the default 100Hz panel is a whole number of pixels per
# refresh, and the helper steps that many pixels per presented frame.
assert helper.frame_based_scrolling is False
assert helper.scroll_speed == pytest.approx(
display._scroll_settings.pixels_per_second)
assert helper.scroll_speed == pytest.approx(500.0, abs=25.0)
# target_fps is the presentation rate the chosen hold produces.
assert helper.fixed_pixels_per_frame == (
display._scroll_settings.crisp.pixels_per_frame)
# target_fps is informational: the presentation rate the hold gives.
assert helper.target_fps == pytest.approx(
120.0 / display._scroll_settings.frame_hold, abs=1.0)
100.0 / display._scroll_settings.frame_hold, abs=1.0)
def test_a_strip_is_built_and_scrolls_to_completion(self, real):
import time
@@ -638,21 +679,58 @@ class TestAgainstTheRealScrollHelper:
# 3 cards of 100px + gaps, so the strip is wider than the 128px panel.
assert display.scroll_helper.cached_image.width > 128
# Frame-based scrolling is wall-clock gated on scroll_delay, so the
# loop has to actually pass time rather than spin.
# The helper steps a fixed whole-pixel amount per presented frame and
# consults no clock, so the scroll completes in a bounded number of
# frames however fast this loop spins.
deadline = time.time() + 20
while not real.is_complete() and time.time() < deadline:
frames = 0
while not real.is_complete() and time.time() < deadline and frames < 10000:
real.display_frame()
time.sleep(0.0012)
frames += 1
# Asserted separately so a host too slow to sustain the frame rate
# reports a timeout rather than looking like a scrolling defect.
assert time.time() < deadline, (
"scroll did not finish within 20s — the host may be too slow to "
"sustain the configured frame rate")
assert time.time() < deadline, "scroll did not finish within 20s"
assert real.is_complete() is True
assert display.scroll_helper.scroll_position > 128
def test_dynamic_duration_is_a_real_number(self, real):
real.prepare_and_display([{"id": "a"}], "live", ["nhl"])
assert real.get_scroll_display("live").get_dynamic_duration() > 0
@pytest.mark.parametrize("target_fps", [None, 60, 100, 200])
def test_presented_speed_is_independent_of_target_fps(self, monkeypatch,
target_fps):
"""End to end: the px/s the panel actually shows.
The panel is simulated the way SwapOnVSync paces it: each drawn frame
occupies ``frame_hold`` refreshes of a 100Hz panel. Ten seconds of
refreshes must move the strip 500px at the default 50 px/s, whatever
the General tab's target_fps says."""
from src.common.scroll_helper import ScrollHelper
monkeypatch.setattr("src.common.sports_scroll.ScrollHelper", ScrollHelper)
class _Panel:
matrix = None
width, height = 128, 32
refresh_hz = 100.0
hold = 1
image = None
def set_scrolling_state(self, scrolling, frame_hold=1):
self.hold = frame_hold
def update_display(self):
pass
global_config = {"display": {"hardware": {"limit_refresh_rate_hz": 100}}}
if target_fps is not None:
global_config["target_fps"] = target_fps
panel = _Panel()
display = _RealDisplay(panel, {}, LOGGER, global_config=global_config)
display.scroll_helper.set_scrolling_image(Image.new("RGB", (5000, 32)))
refreshes = 0
while refreshes < 1000: # ten seconds at 100Hz
assert display.display_scroll_frame() is True
refreshes += panel.hold
moved = display.scroll_helper.total_distance_scrolled
assert moved / 10.0 == pytest.approx(50.0, abs=1.0)
test/web_interface/test_plugin_config_json_saves.py
+258
View File
@@ -0,0 +1,258 @@
"""JSON plugin-config saves store what the device would load, and only what was sent.
Runs a real ConfigManager and SchemaManager over tmp_path, like
test_api_v3_secret_roundtrip.py, so assertions are on config.json itself.
- POST /plugins/config with a partial ``config`` reset every unsent setting to
its schema default: the JSON path built on defaults, not the stored section.
- Legacy booleans (#588) were normalized only at load, so posting back what
GET /plugins/config returned failed validation.
- The JSON save's filter kept only enabled/display_duration/live_priority, so
a submitted skin, skin_options or vegas_* tuning key was silently dropped.
- Plugin sections posted to /config/main skipped all of that and were stored
verbatim, including values /plugins/config rejects.
"""
import json
from unittest.mock import MagicMock
import pytest
from flask import Flask
from src.config_manager import ConfigManager
from src.plugin_system.schema_manager import SchemaManager
from web_interface.blueprints.api_v3 import api_v3
PLUGIN_ID = "demo"
SCHEMA = {
"type": "object",
"additionalProperties": False,
"properties": {
"enabled": {"type": "boolean", "default": True},
"city": {"type": "string", "default": "Dallas"},
"update_interval": {"type": "integer", "default": 300, "minimum": 30},
"api_key": {"type": "string", "x-secret": True, "default": ""},
"display": {
"type": "object",
"additionalProperties": False,
"properties": {
"brightness": {"type": "integer", "default": 50},
"show_icons": {"type": "boolean", "default": True},
},
},
"global": {
"type": "object",
"additionalProperties": False,
"properties": {
"dynamic_duration": {
"type": "object",
"additionalProperties": False,
"properties": {
"enabled": {"type": "boolean", "default": True},
"min_duration_seconds": {"type": "integer", "default": 30},
},
},
},
},
},
}
STORED = {
"enabled": True,
"city": "Paris",
"update_interval": 600,
"display": {"brightness": 80, "show_icons": False},
"global": {"dynamic_duration": {"enabled": False, "min_duration_seconds": 45}},
"vegas_width_pct": 60,
"skin": "neon",
}
_ATTRS = ('config_manager', 'plugin_manager', 'plugin_store_manager',
'plugin_state_manager', 'saved_repositories_manager', 'schema_manager',
'operation_queue', 'operation_history', 'cache_manager')
@pytest.fixture
def env(tmp_path):
config_file = tmp_path / "config.json"
secrets_file = tmp_path / "config_secrets.json"
plugins_dir = tmp_path / "plugins"
plugin_dir = plugins_dir / PLUGIN_ID
plugin_dir.mkdir(parents=True)
(plugin_dir / "config_schema.json").write_text(json.dumps(SCHEMA))
(plugin_dir / "manifest.json").write_text(json.dumps({"id": PLUGIN_ID}))
sentinel = object()
originals = {name: getattr(api_v3, name, sentinel) for name in _ATTRS}
config_manager = ConfigManager(config_path=str(config_file),
secrets_path=str(secrets_file))
config_manager.template_path = str(tmp_path / "no-template.json")
plugin_manager = MagicMock()
plugin_manager.plugin_manifests = {PLUGIN_ID: {"id": PLUGIN_ID}}
plugin_manager.plugins_dir = plugins_dir
plugin_manager.get_plugin.return_value = None
for name in _ATTRS:
setattr(api_v3, name, MagicMock())
api_v3.config_manager = config_manager
api_v3.schema_manager = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path)
api_v3.plugin_manager = plugin_manager
api_v3.operation_queue = None
app = Flask(__name__)
app.config["TESTING"] = True
app.register_blueprint(api_v3, url_prefix="/api/v3")
class Env:
client = app.test_client()
@staticmethod
def store(section):
config_file.write_text(json.dumps({"timezone": "UTC", PLUGIN_ID: section}))
@staticmethod
def main():
return json.loads(config_file.read_text())
@staticmethod
def stored():
return json.loads(config_file.read_text())[PLUGIN_ID]
@staticmethod
def secrets():
return json.loads(secrets_file.read_text()) if secrets_file.exists() else {}
@staticmethod
def save(config):
return app.test_client().post("/api/v3/plugins/config",
json={"plugin_id": PLUGIN_ID, "config": config})
Env.store(json.loads(json.dumps(STORED)))
yield Env
for name, original in originals.items():
if original is sentinel:
if hasattr(api_v3, name):
delattr(api_v3, name)
else:
setattr(api_v3, name, original)
class TestPartialJsonSave:
def test_unsent_settings_keep_their_values(self, env):
resp = env.save({"enabled": True})
assert resp.status_code == 200, resp.get_json()
stored = env.stored()
assert stored["city"] == "Paris"
assert stored["update_interval"] == 600
assert stored["display"] == {"brightness": 80, "show_icons": False}
assert stored["global"]["dynamic_duration"] == {"enabled": False, "min_duration_seconds": 45}
def test_a_nested_partial_keeps_its_siblings(self, env):
resp = env.save({"display": {"brightness": 20}})
assert resp.status_code == 200, resp.get_json()
assert env.stored()["display"] == {"brightness": 20, "show_icons": False}
def test_a_sent_setting_still_changes(self, env):
assert env.save({"city": "Lyon", "enabled": False}).status_code == 200
stored = env.stored()
assert stored["city"] == "Lyon" and stored["enabled"] is False
def test_an_invalid_sent_value_is_still_rejected(self, env):
resp = env.save({"update_interval": 5})
assert resp.status_code == 400
assert env.stored()["update_interval"] == 600
def test_config_must_be_an_object(self, env):
assert env.save(["city"]).status_code == 400
class TestCoreOwnedKeysSurviveTheFilter:
def test_submitted_core_keys_are_stored(self, env):
env.store({"enabled": True, "city": "Paris"})
resp = env.save({
"vegas_width_pct": 50, "vegas_overflow": "truncate",
"vegas_max_width_screens": 2, "skin": "retro",
"skin_options": {"accent": "#ff0000"}, "live_priority": True,
})
assert resp.status_code == 200, resp.get_json()
stored = env.stored()
assert stored["vegas_width_pct"] == 50
assert stored["vegas_overflow"] == "truncate"
assert stored["vegas_max_width_screens"] == 2
assert stored["skin"] == "retro"
assert stored["skin_options"] == {"accent": "#ff0000"}
assert stored["live_priority"] is True
def test_stored_core_keys_survive_an_unrelated_save(self, env):
assert env.save({"city": "Nice"}).status_code == 200
stored = env.stored()
assert stored["vegas_width_pct"] == 60 and stored["skin"] == "neon"
def test_a_non_core_unknown_key_is_still_filtered(self, env):
assert env.save({"not_in_schema": 1}).status_code == 200
assert "not_in_schema" not in env.stored()
class TestLegacyBooleans:
LEGACY = {"enabled": True, "city": "Paris", "global": {"dynamic_duration": True}}
def test_get_returns_the_object_shape(self, env):
env.store(dict(self.LEGACY))
data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
assert data["global"]["dynamic_duration"] == {"enabled": True, "min_duration_seconds": 30}
def test_get_output_posts_back(self, env):
env.store(dict(self.LEGACY))
data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
resp = env.save(data)
assert resp.status_code == 200, resp.get_json()
assert env.stored()["global"]["dynamic_duration"] == {"enabled": True, "min_duration_seconds": 30}
def test_an_unrelated_json_save_upgrades_the_stored_boolean(self, env):
env.store(dict(self.LEGACY))
resp = env.save({"city": "Lyon"})
assert resp.status_code == 200, resp.get_json()
assert env.stored()["global"]["dynamic_duration"] == {"enabled": True, "min_duration_seconds": 30}
def test_a_posted_legacy_boolean_is_read_as_the_object(self, env):
resp = env.save({"global": {"dynamic_duration": False}})
assert resp.status_code == 200, resp.get_json()
assert env.stored()["global"]["dynamic_duration"]["enabled"] is False
class TestPluginSectionsInConfigMain:
def _post(self, env, body):
return env.client.post("/api/v3/config/main", json=body)
def test_a_value_plugins_config_rejects_is_rejected_here_too(self, env):
resp = self._post(env, {PLUGIN_ID: {"update_interval": "abc"}})
assert resp.status_code == 400
assert env.stored()["update_interval"] == 600
assert env.save({"update_interval": "abc"}).status_code == 400
def test_nothing_is_saved_when_a_plugin_section_fails(self, env):
resp = self._post(env, {"timezone": "Europe/Paris", PLUGIN_ID: {"update_interval": 1}})
assert resp.status_code == 400
assert env.main()["timezone"] == "UTC"
assert env.stored()["update_interval"] == 600
def test_partial_section_merges_and_keeps_core_keys(self, env):
resp = self._post(env, {PLUGIN_ID: {"city": "Lyon", "vegas_overflow": "rotate"}})
assert resp.status_code == 200, resp.get_json()
stored = env.stored()
assert stored["city"] == "Lyon"
assert stored["display"] == {"brightness": 80, "show_icons": False}
assert stored["vegas_overflow"] == "rotate"
assert stored["vegas_width_pct"] == 60 and stored["skin"] == "neon"
def test_secrets_still_go_to_the_secrets_file(self, env):
resp = self._post(env, {PLUGIN_ID: {"api_key": "s3cret"}})
assert resp.status_code == 200, resp.get_json()
assert "api_key" not in env.stored() or env.stored()["api_key"] in ("", None)
assert env.secrets()[PLUGIN_ID]["api_key"] == "s3cret"
def test_a_non_object_section_is_rejected(self, env):
assert self._post(env, {PLUGIN_ID: "on"}).status_code == 400
test/web_interface/test_update_all_plugins.py
+61
View File
@@ -80,6 +80,67 @@ class TestUpdateRouteRejectsStarlarkIds:
assert [c.args[0] for c in store.update_plugin.call_args_list] == ['stock-news', 'starlark-apps']
class TestUpdateRouteReportsNoOps:
"""update_plugin() answers True when there was nothing to do.
A ZIP-installed monorepo plugin (no .git) already at the registry version
is the common case for official plugins. The route used to call that
"updated successfully", so Check & Update All counted it as updated.
"""
def _install(self, tmp_path, pid, version, last_updated='2026-09-01'):
(tmp_path / pid).mkdir()
(tmp_path / pid / 'manifest.json').write_text(
'{"id": "%s", "version": "%s", "last_updated": "%s"}' % (pid, version, last_updated),
encoding='utf-8')
def test_a_zip_plugin_already_at_the_registry_version_is_up_to_date(self, client, store, tmp_path):
self._install(tmp_path, 'stock-news', '2.8.0')
body = client.post('/api/v3/plugins/update', json={'plugin_id': 'stock-news'}).get_json()
assert body['status'] == 'success'
assert body['data']['update_status'] == 'up_to_date'
assert 'already up to date' in body['message'], body
assert 'updated successfully' not in body['message']
def test_a_zip_plugin_reinstalled_at_a_new_version_is_updated(self, client, store, tmp_path):
self._install(tmp_path, 'stock-news', '2.6.2')
def reinstall(pid):
(tmp_path / pid / 'manifest.json').write_text(
'{"id": "%s", "version": "2.8.0", "last_updated": "2026-09-01"}' % pid,
encoding='utf-8')
return True
store.update_plugin.side_effect = reinstall
body = client.post('/api/v3/plugins/update', json={'plugin_id': 'stock-news'}).get_json()
assert body['data']['update_status'] == 'updated'
assert '2.8.0' in body['message'], body
def test_a_git_plugin_whose_commit_is_unchanged_is_up_to_date(self, client, store, tmp_path):
self._install(tmp_path, 'clock', '1.0.0')
store._get_local_git_info.return_value = {'sha': 'abcdef1234567', 'branch': 'main'}
body = client.post('/api/v3/plugins/update', json={'plugin_id': 'clock'}).get_json()
assert body['data']['update_status'] == 'up_to_date'
def test_a_git_plugin_that_moved_is_updated(self, client, store, tmp_path):
self._install(tmp_path, 'clock', '1.0.0')
store._get_local_git_info.side_effect = [
{'sha': 'aaaaaaa000', 'branch': 'main'}, # before
{'sha': 'aaaaaaa000', 'branch': 'main'}, # is-git check
{'sha': 'bbbbbbb111', 'branch': 'main'}, # after
]
body = client.post('/api/v3/plugins/update', json={'plugin_id': 'clock'}).get_json()
assert body['data']['update_status'] == 'updated', body
def test_a_local_only_plugin_says_so(self, client, store, tmp_path):
(tmp_path / 'mine').mkdir()
(tmp_path / 'mine' / 'manifest.json').write_text(
'{"id": "mine", "version": "0.1.0", "local_only": true}', encoding='utf-8')
body = client.post('/api/v3/plugins/update', json={'plugin_id': 'mine'}).get_json()
assert body['data']['update_status'] == 'local_only'
store.update_plugin.assert_not_called()
class TestInstalledListContract:
"""What update-all filters on is what /plugins/installed publishes."""