mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-08 16:16:36 +00:00
Compare commits
21
Commits
v3.6.1
...
7804ea8f69
| Author | SHA1 | Date | |
|---|---|---|---|
 ChuckandClaude Opus 5.5
|
7804ea8f69 | ||
|
ChuckandClaude Opus 5.5
|
64c7289593 | ||
|
ChuckandClaude Opus 5.5
|
b09434a418 | ||
|
ChuckandClaude Opus 5.5
|
7ab6fb1aff | ||
|
ChuckandClaude Opus 5.5
|
ba6eccb489 | ||
|
ChuckandClaude Opus 5.5
|
5ea0d511dc | ||
|
ChuckandClaude Opus 5.5
|
1c928b2033 | ||
|
ChuckandClaude Opus 5.5
|
b2df0fda1b | ||
|
ChuckandClaude Opus 5.5
|
15c61def67 | ||
|
ChuckandClaude Opus 5.5
|
c8a0ddcf7b | ||
|
ChuckandClaude Opus 5.5
|
9fe23af432 | ||
|
ChuckandClaude Opus 5.5
|
c0d97e4867 | ||
|
ChuckandClaude Opus 5.5
|
e3c85cece6 | ||
|
ChuckandClaude Opus 5.5
|
ba38a83c2c | ||
|
ChuckandClaude Opus 5.5
|
c3a7a110c4 | ||
|
ChuckandClaude Opus 5.5
|
6047eb5e4e | ||
|
ChuckandClaude Opus 5.5
|
c4c46d3ba7 | ||
|
ChuckandClaude Opus 5.5
|
da9a999102 | ||
|
ChuckandClaude Opus 5.5
|
1e4c890d59 | ||
|
ChuckandClaude Opus 5.5
|
7f96075076 | ||
|
ChuckandClaude Opus 5.5
|
439013b18c |
@@ -6,3 +6,7 @@
|
||||
# and systemd rejects CRLF unit files.
|
||||
*.sh text eol=lf
|
||||
*.service text eol=lf
|
||||
|
||||
# Generated by scripts/build_css.py; collapsed in diffs, not hand-edited.
|
||||
web_interface/static/v3/tailwind.css linguist-generated=true
|
||||
web_interface/static/v3/plugin-frame.css linguist-generated=true
|
||||
|
||||
@@ -113,6 +113,25 @@ jobs:
|
||||
REQUIRE_DOM: "1"
|
||||
run: node test/js/run_all.js
|
||||
|
||||
css-build:
|
||||
name: Tailwind CSS is up to date
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
# Downloads the pinned standalone Tailwind CLI (SHA-256 checked; no
|
||||
# Node), rebuilds static/v3/tailwind.css and plugin-frame.css from the
|
||||
# templates and JS, and fails if the committed files differ. Fix a
|
||||
# failure by running `python3 scripts/build_css.py` and committing.
|
||||
- name: Check the committed CSS matches a fresh build
|
||||
run: python scripts/build_css.py --check
|
||||
|
||||
type-check:
|
||||
name: Type check (mypy ratchet)
|
||||
runs-on: ubuntu-latest
|
||||
@@ -140,3 +159,39 @@ jobs:
|
||||
# them, or if a listed file is missing. See CONTRIBUTING.md.
|
||||
- name: Run mypy on the ratchet list
|
||||
run: python scripts/check_types.py
|
||||
|
||||
sports-drift-report:
|
||||
name: Sports drift report (report only)
|
||||
runs-on: ubuntu-latest
|
||||
# A progress measure for docs/SPORTS_UNIFICATION.md, never a gate: the
|
||||
# monorepo's own check_sports_drift.py is the gate. The step summary shows
|
||||
# how many bodies each scoreboard method family still has.
|
||||
continue-on-error: true
|
||||
steps:
|
||||
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Check out ledmatrix-plugins (main)
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||
with:
|
||||
repository: ChuckBuilds/ledmatrix-plugins
|
||||
path: ledmatrix-plugins
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
# Stdlib only; exits 0 whatever it finds.
|
||||
- name: Report method-family drift across the nine scoreboards
|
||||
run: |
|
||||
python scripts/sports_drift_report.py --plugins ledmatrix-plugins \
|
||||
--markdown --json sports-drift.json >> "$GITHUB_STEP_SUMMARY"
|
||||
python scripts/sports_drift_report.py --plugins ledmatrix-plugins
|
||||
|
||||
- name: Upload the full report
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: sports-drift-report
|
||||
path: sports-drift.json
|
||||
|
||||
+418
-1
@@ -19,6 +19,422 @@ accepts both, but the store flags the old spelling as deprecated
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Update channels
|
||||
|
||||
- Devices no longer pick up every merge to `main`. A new setting,
|
||||
`auto_update.channel`, picks what Update Code and the weekly automatic
|
||||
update install: `stable` follows the newest release tag (`vX.Y.Z` by
|
||||
semantic version; pre-releases and other tags are ignored) and checks it
|
||||
out with a detached HEAD, and `beta` follows `main` as every device did
|
||||
before. New installs default to `stable` (config template and installer).
|
||||
- Nobody is moved backwards. A device running code newer than the newest
|
||||
release, which is any device that pulled `main` since that release, keeps
|
||||
following `main` until a release contains its commit, then moves to it and
|
||||
follows releases. A config written before channels existed behaves the
|
||||
same way and is saved as `stable` when that move happens. Switching from
|
||||
beta to stable says so instead of installing an older version.
|
||||
- Switch channels on the General tab (Update Channel, under Automatic
|
||||
Updates) or with `GET`/`POST /api/v3/system/update-channel`. The Overview
|
||||
update banner compares release tags on stable ("LEDMatrix v3.8.0 is
|
||||
available") rather than commits on `main`. A detached checkout newer
|
||||
than the newest release gets no banner: Update Code leaves it where it
|
||||
is until a release includes it.
|
||||
- A move between `main` and a release tag carries local edits across as the
|
||||
pull's `--autostash` does, and the automatic update's health check rolls
|
||||
it back to where HEAD was: the branch, or the detached release.
|
||||
|
||||
### Frozen-panel detection
|
||||
|
||||
A render loop stuck inside a plugin's `display()` left `ledmatrix.service`
|
||||
"active" with the panel frozen, and nothing noticed: `/api/v3/health` judged
|
||||
the display by the preview PNG's age, and the automatic update's health check
|
||||
passed "service active plus one HTTP 200".
|
||||
|
||||
- **systemd watchdog.** `ledmatrix.service` now has `WatchdogSec=120` and
|
||||
`NotifyAccess=main` (still `Type=simple`). The render thread itself pings
|
||||
systemd over `$NOTIFY_SOCKET` (`src/display_watchdog.py`, standard library
|
||||
only), so a stuck render thread stops the pings even while the update
|
||||
worker and Vegas's tick thread carry on. systemd then kills the display with
|
||||
SIGABRT -- faulthandler writes every thread's stack to the journal, which
|
||||
names the plugin -- and restarts it. The process widens the limit to 15
|
||||
minutes while it starts and while it loads a plugin enabled from the web UI
|
||||
(either can run pip), and sends `READY=1` and narrows it back after its
|
||||
first frame.
|
||||
- **Heartbeat.** The render loop writes `/run/ledmatrix/display-heartbeat.json`
|
||||
every 5 seconds (`RuntimeDirectory=ledmatrix`; tmpfs, so no SD-card
|
||||
writes). `/api/v3/health` reports it as `checks.display_loop`: `running`,
|
||||
`stalled` (older than 60s; the overall status turns `degraded`) or
|
||||
`not_reported` when there is no heartbeat (dev server, emulator, Windows),
|
||||
which leaves the verdict to the older checks as before.
|
||||
- **Update health check.** When the display wrote a heartbeat before an
|
||||
automatic update, the restarted display must keep one fresh (30s) for the
|
||||
update to pass; a frozen panel is rolled back. Code that never wrote one is
|
||||
checked as before. The check runs as the copy taken before the update, so
|
||||
this takes effect from the update after the one that installs it.
|
||||
- **Crash loops back off.** `RestartSteps=4` and `RestartMaxDelaySec=2min`
|
||||
stretch the delay between automatic restarts from 10s to two minutes, instead
|
||||
of retrying every 10s forever. systemd before 254 (Bookworm) ignores the two
|
||||
lines with a warning. A start limit was ruled out: once tripped it leaves the
|
||||
panel dark and refuses the web UI's Start button and the update rollback.
|
||||
- **Existing installs** keep their old unit until `sudo
|
||||
./scripts/install/install_service.sh` is re-run (an update never rewrites
|
||||
units; the startup validator warns about the drift). Until then there is no
|
||||
watchdog, but the display creates `/run/ledmatrix` itself, so the heartbeat,
|
||||
the health check and the update check work straight away.
|
||||
|
||||
### Security
|
||||
|
||||
- The web interface refuses state-changing requests (`POST`, `PUT`, `PATCH`,
|
||||
`DELETE`) sent by another website's page. Any site a LAN user visited could
|
||||
make their browser submit a plain HTML form to `http://<pi>:5000` -- CORS
|
||||
does not stop such a request, only hides its answer -- and
|
||||
`/api/v3/system/action` accepted form bodies, so that page could reboot or
|
||||
power off the Pi, pull code, or reach any other mutating route. A request
|
||||
whose `Origin` (or, without one, `Referer`) is not the host it was sent to,
|
||||
or is `null`, now gets 403 `CROSS_SITE_REQUEST`
|
||||
(`web_interface/origin_guard.py`). `/api/v3/system/action` also refuses a
|
||||
form-encoded or `text/plain` body (415) unless it carries HTMX's
|
||||
`HX-Request` header; every caller in the interface already sends JSON.
|
||||
- **Behaviour change for API scripts:** clients that send no `Origin` or
|
||||
`Referer` -- curl, Python `requests`, Home Assistant, the MQTT bridge --
|
||||
are unaffected. A browser page served from a *different* origin (a
|
||||
dashboard or userscript on another host) can no longer call the mutating
|
||||
API; call it server-side instead. Anyone posting a form body to
|
||||
`system/action` must switch to JSON. Behind a reverse proxy, forward the
|
||||
original `Host`, port included (`proxy_set_header Host $http_host;`;
|
||||
nginx's `$host` drops the port); `X-Forwarded-Host` is not trusted. A
|
||||
TLS-terminating proxy needs nothing more: a portless `Host` matches an
|
||||
`https://` page.
|
||||
|
||||
### Optional web login
|
||||
|
||||
- The web interface can require a password, **off by default**: a device that
|
||||
does not set one behaves exactly as before. Set it under **General >
|
||||
Security**; from then on every page and API route needs a login (a session
|
||||
cookie, 30 days, kept across restarts) or an API token. Unauthenticated page
|
||||
loads go to the new `/login` page, HTMX requests get `HX-Redirect` to it,
|
||||
and API calls get `401` JSON (`AUTH_REQUIRED` / `INVALID_TOKEN`). Wrong
|
||||
passwords are rate-limited per address (5 a minute, 30 an hour, through the
|
||||
existing flask-limiter). Log out from the header. Changing the password
|
||||
signs every other browser out. (`web_interface/auth.py`)
|
||||
- **API tokens** for Home Assistant, scripts and the MQTT bridge: create,
|
||||
list and revoke them in the same section, send them as
|
||||
`Authorization: Bearer <token>`. A token is shown once; only its SHA-256 is
|
||||
stored. Tokens cannot change login settings. The MQTT bridge takes one as
|
||||
`ledmatrix_api_token` (or `LEDMATRIX_MQTT_LEDMATRIX_API_TOKEN`, or the
|
||||
Tools tab); it needs one only when it runs on another machine.
|
||||
- Always open, login or not: requests from the Pi itself (loopback, without
|
||||
proxy headers), the Wi-Fi setup flow (`/setup` and the Wi-Fi status, scan
|
||||
and connect routes) while the Pi is in access-point mode, static files, the
|
||||
captive-portal probe URLs, and `/api/v3/health`, which then answers only
|
||||
`{"status": "healthy" | "degraded"}` to a caller that is not logged in.
|
||||
- The password hash (werkzeug), the token hashes and the cookie-signing key
|
||||
live in the `web_auth` section of `config/config_secrets.json`. No API
|
||||
returns them: `GET /api/v3/config/main`, `GET /api/v3/config/secrets` and
|
||||
the raw JSON editor leave the section out, the raw secrets save keeps the
|
||||
stored one, a `/config/main` save drops a `web_auth` key, and orphaned-plugin
|
||||
cleanup no longer treats it as a plugin (`CORE_SECRETS_KEYS`).
|
||||
- **Lost password:** `sudo python3 scripts/reset_web_password.py` on the Pi
|
||||
turns login off (`--revoke-tokens` also deletes the tokens), or open the
|
||||
interface from the Pi itself.
|
||||
- New routes: `/login`, `/logout`, `GET /api/v3/auth/status`,
|
||||
`POST /api/v3/auth/password`, `POST /api/v3/auth/disable`,
|
||||
`GET|POST /api/v3/auth/tokens`, `DELETE /api/v3/auth/tokens/<id>`.
|
||||
|
||||
### Vegas participation
|
||||
|
||||
A plugin now takes part in Vegas mode in one declared way: `'scroll'` (its
|
||||
content scrolls by), `'pause'` (the scroll stops for its turn and its
|
||||
`display()` draws it full screen) or `'exclude'`. No plugin changes
|
||||
behaviour: one that declares nothing gets exactly what the old hooks gave
|
||||
it, checked against every official plugin.
|
||||
|
||||
- `BasePlugin.get_vegas_participation()` resolves, in order: the user's
|
||||
`vegas_participation` config value, the manifest's `vegas_participation`,
|
||||
then the legacy hooks (`get_vegas_display_mode()` returning `STATIC` →
|
||||
pause, else `get_vegas_content_type()` returning `'none'` → exclude, else
|
||||
scroll). `resolve_vegas_participation()` in `src.plugin_system.base_plugin`
|
||||
is what the core calls; the user's setting wins even over a plugin that
|
||||
overrides the method.
|
||||
- The Vegas stream manager decides inclusion and pauses through it, and
|
||||
`PluginAdapter.get_content_type()` is removed (core-internal, now unused).
|
||||
Swap mode no longer drops a plugin's segment for a cycle when its
|
||||
`get_vegas_display_mode()` raises something other than
|
||||
`AttributeError`/`TypeError`: like every other decision point it now
|
||||
treats that as "not paused".
|
||||
- `vegas_participation` is a core-owned per-plugin property (an enum with no
|
||||
default) and a manifest field in `schema/manifest_schema.json`.
|
||||
- `GET /api/v3/plugins/installed` reports each plugin's
|
||||
`vegas_participation`, and the Vegas plugin-order list badges it (Scroll /
|
||||
Pause / Excluded) instead of the old Scroll / Fixed / Static.
|
||||
- `src.deprecation.warn_deprecated()` warns once per process for what
|
||||
`@deprecated` cannot decorate, such as a config key.
|
||||
|
||||
Deprecated, removed in 3.9.0 (each logs a warning on first use). Vegas never
|
||||
read any of them:
|
||||
|
||||
- `BasePlugin.get_supported_vegas_modes()` and
|
||||
`BasePlugin.get_vegas_segment_width()`.
|
||||
- The `vegas_panel_count` per-plugin setting (warns once per plugin that sets
|
||||
it).
|
||||
- The SCROLL / FIXED_SEGMENT distinction (`vegas_mode` `"scroll"` vs
|
||||
`"fixed"`): both always scrolled. Documented only; no warning, because
|
||||
official plugins' schemas still offer `"fixed"`.
|
||||
|
||||
### Plugin store
|
||||
|
||||
- The store reads three optional registry fields that ledmatrix-plugins'
|
||||
`update_registry.py` now publishes (ChuckBuilds/ledmatrix-plugins#579). An
|
||||
older `plugins.json` without them behaves as before.
|
||||
- `ledmatrix_min_version`: an install or update this core cannot run is
|
||||
refused before anything is downloaded, pulled or moved aside, and the web
|
||||
UI says why ("requires LEDMatrix X or newer…", HTTP 409) instead of "check
|
||||
logs for details". The store card shows a "Needs LEDMatrix X+" badge. The
|
||||
check on the downloaded manifest stays as the fallback (older registries,
|
||||
an explicitly requested other branch, `compatible_versions`).
|
||||
- `aliases`: the entry's other ids. Update, uninstall and reinstall by the
|
||||
registry id now find a plugin installed under its manifest id
|
||||
(`weather` → `ledmatrix-weather/`; likewise leaderboard, music, stocks).
|
||||
Only registry proof counts: the entry's `aliases` or its `plugin_path`
|
||||
name, or a folder whose manifest declares one of those ids. A
|
||||
`ledmatrix-<id>/` folder with no such proof is never replaced or removed;
|
||||
uninstall and update report "not installed" and log the folder's path.
|
||||
Install and update fetch the registry first when such a folder exists
|
||||
and none is loaded; uninstall stays offline.
|
||||
- `commit`: the monorepo commit that introduced the listed version, shown
|
||||
on the store card and linked to the plugin's source at that commit.
|
||||
Informational only; installs still come from the branch head.
|
||||
|
||||
### Changes
|
||||
|
||||
- The web interface no longer loads or runs plugins (web plugin catalog,
|
||||
stage 1). It built its own `PluginManager` and loaded plugins into the web
|
||||
process: store installs and updates loaded or reloaded a web-side copy, and
|
||||
config saves and enable/disable called `on_config_change`, `on_enable` and
|
||||
`on_disable` on it. None of that reached the panel. The web process now
|
||||
reads plugins as files through the new `PluginCatalog`
|
||||
(`src/plugin_system/plugin_catalog.py`); only the display runs them, and
|
||||
config changes reach them through its config watcher, as they already did.
|
||||
- A plugin update, an install of a plugin that is already enabled, or an
|
||||
uninstall that keeps an enabled plugin's config now answers
|
||||
`restart_required: true` and shows the restart banner, because the
|
||||
running display keeps the code it loaded until it restarts. Before, the
|
||||
update looked applied and the panel kept the old version.
|
||||
- The restart banner follows `restart_required` in any response
|
||||
(`POST /api/v3/config/main` sends it) rather than the URL that was
|
||||
called.
|
||||
- `/api/v3/plugins/installed` reports `loaded`, `state` and `error_info`
|
||||
as `null`: the display does not publish them, and the old values
|
||||
described web-side copies. `enabled` follows the display's rule, so a
|
||||
plugin whose config has no `enabled` flag shows as disabled (it never
|
||||
ran). `vegas_mode` is the configured value only.
|
||||
- `vegas_participation` there is the user's setting, else the manifest's
|
||||
declaration, with a new `vegas_participation_source` (`config` or
|
||||
`manifest`). When only the plugin's code decides it (a
|
||||
`get_vegas_participation()` override or the legacy Vegas hooks) it is
|
||||
`null` with source `runtime`: the display derives it, and the web no
|
||||
longer asks a web-side plugin instance.
|
||||
- Starlark routes always use their on-disk path. The one place the web
|
||||
process still imports plugin code -- the Starlark helper modules and an
|
||||
`oauth_flow` action script -- is `_import_plugin_code_in_web_process()`,
|
||||
until a plugin web-entry contract replaces it.
|
||||
- The display publishes its plugin runtime state, and the web interface
|
||||
reads it (web plugin catalog, stage 2). A new snapshot in the shared cache
|
||||
(`plugin_runtime_snapshot`, `src/plugin_system/plugin_runtime.py`) lists,
|
||||
per plugin, whether the display has it loaded, its lifecycle state, a
|
||||
short redacted summary of its last error, the version it loaded and when.
|
||||
It is written when something changes (at most every 10 s; an ordinary
|
||||
plugin update is not a change) and otherwise once a minute, carries its
|
||||
publish time, and says `running: false` when the display stops.
|
||||
- `/api/v3/plugins/installed` fills `loaded`, `state` and `error_info`
|
||||
again, from that snapshot, and adds `loaded_version` and `loaded_at`.
|
||||
Only a live snapshot counts: when the display is stopped, has not
|
||||
published, or has not refreshed for 3 minutes, those fields are `null`
|
||||
and the new `data.runtime.status` says `stopped`, `unknown` or `stale`.
|
||||
- `data/plugin_state.json` is retired: nothing reads or writes it. It held
|
||||
copies of config.json's enabled flags and the manifests' versions, plus
|
||||
install timestamps only `GET /api/v3/plugins/state` returned, so nothing
|
||||
in it is migrated; an existing file is left in place and can be deleted.
|
||||
The web-side `PluginStateManager` (`src/plugin_system/state_manager.py`)
|
||||
that wrote it is removed; the display's state machine in
|
||||
`plugin_state.py` is now the only `PluginStateManager`.
|
||||
- `GET /api/v3/plugins/state` is built per request from config.json, the
|
||||
plugins on disk and the display's snapshot (`installed`, `in_config`,
|
||||
`enabled`, `version`, `status`, the runtime fields, and `installed_at` /
|
||||
`last_updated` from the operation history), with a top-level `runtime`.
|
||||
It no longer returns `config_version` or `metadata`.
|
||||
- State reconciliation compares desired state (config.json plus disk) with
|
||||
the display's snapshot. New findings -- enabled but not loaded (with the
|
||||
load error), and loaded at an older version than is installed -- are
|
||||
reported with `fix_action: no_action`; the unresolved-issues banner is
|
||||
unchanged. `StateReconciliation` takes `config_manager`, `plugins_dir`,
|
||||
`store_manager` and `runtime_source` as keywords.
|
||||
- Backups list the installed plugins from disk, with `enabled` from
|
||||
config.json, instead of merging in `plugin_state.json`. A plugin that
|
||||
only that file still named (not installed, not configured) is no longer
|
||||
listed. Restores are unchanged.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Reinstalling a plugin by its registry id when it is installed under its
|
||||
manifest id (`weather` in `ledmatrix-weather/`) no longer deletes it when
|
||||
the install then fails. The safety copy was taken of `weather/`, which did
|
||||
not exist, and the real install was removed to make room for the download,
|
||||
so a refusal by the compatibility gate left no plugin at all. Uninstalling
|
||||
by the registry id reported success and removed nothing; updating by it
|
||||
said "not installed". All three now find the install.
|
||||
|
||||
- On-demand no longer restarts a running display. `POST
|
||||
/display/on-demand/start` treated `start_service` (on by default, and what
|
||||
"Preview on display", the on-demand dialog and the MQTT bridge all send) as
|
||||
"restart": it stopped the service, waited 1.5s and started it again, so
|
||||
every request reloaded every plugin and left the panel blank for seconds.
|
||||
The running display already reads the request within a quarter of a second,
|
||||
mid-screen and mid-Vegas included, so the route now only starts the service
|
||||
when it is not running. `POST /display/on-demand/stop` reads
|
||||
`stop_service` as a boolean, so `"false"` no longer stops the service.
|
||||
- On-demand works for a disabled plugin. The display only loads enabled
|
||||
plugins, so "Preview on display" on a disabled plugin's config page (which
|
||||
says the plugin will be enabled for the preview) failed with
|
||||
`invalid-mode`. The display now loads the plugin live for the session,
|
||||
without writing `enabled` to `config.json`, and unloads it when on-demand
|
||||
is stopped, expires or moves to another plugin. A plugin that fails to
|
||||
load reports on-demand status `error` with `load-failed`. A session
|
||||
restored after a restart unloads its disabled plugin the same way; it used
|
||||
to stay loaded until the next restart.
|
||||
- A stop request now clears an on-demand error. After a failed request,
|
||||
`/display/on-demand/status` kept reporting `status: error` for up to two
|
||||
minutes even after a stop.
|
||||
- One hung plugin no longer stops every plugin from updating. The single
|
||||
update worker waited on each plugin's lock with no time limit, and the
|
||||
render thread holds that lock while it runs the plugin's display(); a
|
||||
display() that never returned (or a first frame still running after the
|
||||
executor's 30s timeout) parked the worker for good, so scores, weather and
|
||||
clocks all froze while the panel kept scrolling. The worker now waits at
|
||||
most 5s (the bound `unload_plugin()` already uses) and skips that update;
|
||||
the other plugins keep updating. The skip is logged (at most once a minute
|
||||
per plugin) and counted in plugin health as a busy skip (`busy_skip_count`,
|
||||
`last_busy_skip`), but it is not a failure and never opens the circuit
|
||||
breaker: Vegas mode holds a plugin's lock for its whole content render,
|
||||
which on a slow Pi can outlast 5s, and a healthy plugin must not be pulled
|
||||
from rotation for that.
|
||||
- display() calls are timed on every frame. One taking 2s or more is logged
|
||||
(at most once a minute per plugin) and counted in plugin health
|
||||
(`slow_call_count`, `last_slow_call`); one that runs past the executor's
|
||||
timeout counts as a hang (`hang_count`, `last_hang`) and as a failure to
|
||||
the circuit breaker. A first frame that times out is no longer recorded as
|
||||
a success, and an update() still running after its timeout is recorded as
|
||||
a hang instead of leaving the plugin silently stuck. Only these real hangs
|
||||
count toward the breaker.
|
||||
- A plugin's `on_config_change()` no longer runs while its update() is
|
||||
running on the worker thread. It now runs under the plugin's lock; if the
|
||||
lock stays busy past the same 5s bound the change is handed to the update
|
||||
worker, which applies the latest one as soon as the lock frees, and before
|
||||
the plugin's next update() at the latest. The plugin API is unchanged.
|
||||
|
||||
### Tooling
|
||||
|
||||
- `scripts/sports_drift_report.py`: for a ledmatrix-plugins checkout, counts
|
||||
how many different bodies each method family has across the nine
|
||||
scoreboards' `sports.py`, `manager.py` and `game_renderer.py`, lists the
|
||||
families still identical everywhere and those with one outlier, and with
|
||||
`--family ... --diff` shows the variants. It is the progress measure for
|
||||
the reconcile-then-promote roadmap in `docs/SPORTS_UNIFICATION.md`, which
|
||||
this release rewrites. CI runs it against the monorepo's main as a
|
||||
report-only job ("Sports drift report"; never fails the build).
|
||||
|
||||
### Deprecations
|
||||
|
||||
- The 35 plugin-facing methods deprecated in 3.5.0 are now removed in 3.8.0,
|
||||
not 3.7.0: 3.7.0 shipped with all of them still in place, still warning
|
||||
"will be removed in LEDMatrix 3.7.0". The warning, the docs and
|
||||
`test/test_deprecation.py` now say 3.8.0. Nothing is removed yet.
|
||||
- New `scripts/plugin_api_usage.py` lists every `@deprecated` core method and
|
||||
scans core, the plugin monorepo and the registry's third-party plugins for
|
||||
calls and overrides, telling real uses from unrelated methods of the same
|
||||
name. Its output is `docs/DEPRECATIONS_3.8.md` (linked from
|
||||
`docs/PLUGIN_API_REFERENCE.md#deprecated-apis`): 34 of the 35 are unused;
|
||||
`CacheManager.get_memory_cache_stats` is still called by core's own
|
||||
`log_memory_cache_stats()`, so it stays until that call migrates.
|
||||
- `test/test_deprecation.py` fails while any `@deprecated` marker names a
|
||||
release at or below `src.__version__`, so a release can no longer ship
|
||||
warning about a removal it has already passed.
|
||||
|
||||
### Web UI styling: a real Tailwind build
|
||||
|
||||
- The web UI's utility classes now come from a generated
|
||||
`static/v3/tailwind.css` (Tailwind v3.4.19 standalone CLI, no Node)
|
||||
instead of ~500 hand-written rules in `app.css`. The CSS is built on a
|
||||
dev machine with `python3 scripts/build_css.py` and committed; the Pi
|
||||
never builds anything. CI's new "Tailwind CSS is up to date" job rebuilds
|
||||
it and fails when the committed file is stale. `app.css` keeps the theme
|
||||
tokens, components and dark theme, and loads after `tailwind.css`. The
|
||||
values `app.css` had customised (darker gray text, emerald/amber button
|
||||
fills, token shadows, font line-heights, keyboard-only focus rings) are
|
||||
kept in `web_interface/tailwind/tailwind.config.js`.
|
||||
- Border utilities now draw. `border-b`, `border-t` and `divide-y` set only
|
||||
a width, and nothing gave them a style, so the tab-row underlines and
|
||||
section dividers the markup asks for never showed. They do now.
|
||||
- `2xl:` classes now apply (the hand-written `.2xl\:…` selectors were
|
||||
invalid CSS): at 1536px and wider the plugin grids show five columns and
|
||||
the page gutters widen, as the markup intended.
|
||||
- Classes the hand-written file never defined now work, e.g. the teal
|
||||
"configure" badge in Operation History, the button of a purple
|
||||
`web_ui_actions` card (it had white text on no background), the
|
||||
toggle-switch knob offsets, the slider accent colours and the password
|
||||
strength colours.
|
||||
- A scrollable container with its own background (the live preview stage,
|
||||
command output in Tools) keeps it. The scroll-hint rule's `background`
|
||||
shorthand wiped it, so the preview stage rendered white instead of dark.
|
||||
- Plugin `web_ui/` pages no longer load Tailwind from a CDN, which failed
|
||||
in AP mode with no internet. They get a local `static/v3/plugin-frame.css`
|
||||
with the v2 palette they were written against.
|
||||
|
||||
## 3.7.0
|
||||
|
||||
Sports consolidation stage 3 (#672). No behaviour change: nothing in core
|
||||
uses these yet, and the scoreboards adopt them when they floor on 3.7.0.
|
||||
|
||||
### New modules
|
||||
|
||||
A plugin may import these via `src.*` (floor on 3.7.0). All three hold code
|
||||
the scoreboard plugins carry as identical copies, moved without behaviour
|
||||
change under the plugins' own method names; each docstring lists what the
|
||||
host class must provide. The plugins delete their copies when they floor on
|
||||
3.7.0.
|
||||
|
||||
- `src/common/sports_celebration.py` — `SportsCelebrationMixin`, the
|
||||
score/win celebration takeover drawn by afl, football, hockey, nrl and
|
||||
soccer (`_draw_celebration_layout` and the palette, backdrop, scenery,
|
||||
confetti and crest steps behind it), plus its colour helpers as free
|
||||
functions: `logo_palette`, `lift_color`, `cap_luminance`, `mix_color`,
|
||||
`scale_color`, `dim_rgba`, `rgb_luminance`, `rgb_saturation`,
|
||||
`color_distance`. Only the drawing: when to celebrate, the phrase and the
|
||||
scenery stay in each plugin.
|
||||
- `src/common/sports_fetch.py` — `SportsFetchMixin`, four `SportsCore`
|
||||
methods identical in all nine scoreboards: `_fetch_season_directly`,
|
||||
`_background_fetches_espn_ranges`, `_needs_previous_day` and
|
||||
`_wants_live_odds` (with `_LOOKBACK_CUTOFF_HOUR` and
|
||||
`_LIVE_ODDS_LOOKAHEAD`).
|
||||
- `src/common/sports_card_wrappers.py` — `SportsCardWrappersMixin`, the
|
||||
seventeen `sports_card` delegations the eight scoreboard game renderers
|
||||
carry (`_vs_text`, `_element_color`, `_format_game_date`, ...): the methods
|
||||
`SportsGameRendererMixin` expects its host to provide.
|
||||
|
||||
## 3.6.2
|
||||
|
||||
A fix to `src.common.favorite_team_check` (#670).
|
||||
|
||||
### Fixes
|
||||
|
||||
- The favourite-team check no longer says the Europa League season has
|
||||
finished between matchdays. Its scoreboard keeps showing the last matchday,
|
||||
and its calendar is a "list" of rounds rather than match days, so neither
|
||||
3.6.1 rule applied. When every event is past, a round in a list calendar
|
||||
that has not started yet (outside an offseason phase) now draws no
|
||||
conclusion. PLL, the World Cup and AFL, whose seasons are over, are still
|
||||
reported as finished: no round of theirs is still to start. (#670)
|
||||
|
||||
## 3.6.1
|
||||
|
||||
A fix to `src.common.favorite_team_check` (#667). Plugins that drop their
|
||||
@@ -121,7 +537,8 @@ New names in existing modules (a plugin using these must floor on 3.5.0):
|
||||
- `FontManager.register_plugin_fonts()` takes an optional `plugin_dir`, and
|
||||
`FontManager.forget_manager_fonts()` is new (see Fonts).
|
||||
|
||||
Deprecated, removed in 3.7.0 (each logs a warning on first use; see
|
||||
Deprecated for removal in 3.7.0, later moved to 3.8.0 (each logs a warning
|
||||
on first use; see
|
||||
`docs/PLUGIN_API_REFERENCE.md#deprecated-apis` for replacements). Nothing in
|
||||
core, the monorepo or the registry's third-party plugins calls them:
|
||||
|
||||
|
||||
@@ -46,6 +46,7 @@
|
||||
- Store manager (`PluginStoreManager` in `src/plugin_system/store_manager.py`) handles install/update/uninstall
|
||||
- Monorepo plugins are installed without a `.git` directory: GitHub Trees API + raw downloads, falling back to ZIP extraction
|
||||
- Update detection for monorepo plugins uses version comparison (manifest version vs registry latest_version)
|
||||
- Optional registry entry fields (`store_registry.py`): `ledmatrix_min_version` refuses an incompatible install/update before the download (the post-download manifest gate stays as the fallback); `aliases` are the entry's other ids (manifest id `ledmatrix-weather` for `weather`), used with the `plugin_path` name by update/uninstall/reinstall to find the install — only this registry proof counts, never a bare `ledmatrix-<id>` folder (owner decision, #686; such a folder is only logged); `commit` is informational. An older plugins.json has none of them
|
||||
- Plugin configs stored in `config/config.json`, NOT in plugin directories — safe across reinstalls
|
||||
- Third-party plugins can use their own repo URL with empty `plugin_path`
|
||||
|
||||
|
||||
+5
-1
@@ -71,7 +71,11 @@ integration tests.
|
||||
annotation-only where you can -- widen a hint rather than delete a
|
||||
defensive runtime check mypy calls unreachable. HTML/JS in
|
||||
`web_interface/` follows the patterns already in `templates/v3/`
|
||||
and `static/v3/`.
|
||||
and `static/v3/`. If you change a template or a static JS file,
|
||||
run `python3 scripts/build_css.py` and commit the regenerated
|
||||
`static/v3/tailwind.css` with it -- CI fails when the committed CSS
|
||||
is out of date. It needs no Node; see
|
||||
[`web_interface/README.md`](web_interface/README.md#styling-tailwind-css).
|
||||
5. **Update documentation** alongside code changes. If you add a
|
||||
config key, document it in the relevant `*.md` file (or, for
|
||||
plugins, in `config_schema.json` so the form is auto-generated).
|
||||
|
||||
+25
-2
@@ -61,8 +61,31 @@ Out of scope (please report upstream):
|
||||
LEDMatrix is designed for trusted local networks. Several limitations
|
||||
are intentional rather than vulnerabilities:
|
||||
|
||||
- **No web UI authentication.** The web interface assumes the network
|
||||
it's running on is trusted. Don't expose port 5000 to the internet.
|
||||
- **Web UI authentication is optional and off by default.** Out of the
|
||||
box the web interface assumes the network it's running on is trusted.
|
||||
Setting a password under **General > Security** makes every page and
|
||||
API route require a login or an API token (`Authorization: Bearer`),
|
||||
with wrong passwords rate-limited per address
|
||||
(`web_interface/auth.py`). Deliberately left open even then: requests
|
||||
from the Pi itself (loopback without proxy headers; a reverse proxy on
|
||||
the Pi must add `X-Forwarded-For`, or every request it relays counts as
|
||||
local), the Wi-Fi setup flow while the Pi is in access-point mode,
|
||||
static files, and a status-only `/api/v3/health`. The password is a
|
||||
werkzeug hash and tokens are stored as SHA-256, in
|
||||
`config/config_secrets.json`, which no API returns. There is no TLS:
|
||||
over plain HTTP the password and tokens cross the LAN in the clear, so
|
||||
still don't expose port 5000 to the internet; put a TLS reverse proxy
|
||||
or a VPN in front for remote access. Anyone with shell access to the Pi
|
||||
can turn login off (`scripts/reset_web_password.py`), which is the
|
||||
documented recovery path.
|
||||
"Trusted network" does not mean "trusted websites", though: any page
|
||||
a LAN user opens could make their browser POST to the Pi. So the
|
||||
interface refuses a `POST`/`PUT`/`PATCH`/`DELETE` whose `Origin` (or
|
||||
`Referer`) header names another site (`web_interface/origin_guard.py`),
|
||||
and `/api/v3/system/action` only accepts JSON or HTMX requests. Tools
|
||||
that send neither header (curl, Home Assistant, the MQTT bridge) are
|
||||
unaffected. Not covered: DNS rebinding, and anyone who can reach the
|
||||
port directly.
|
||||
- **Plugins run unsandboxed.** Installed plugins execute in the same
|
||||
Python process as the display loop with full file-system and
|
||||
network access. Review plugin code (especially third-party plugins
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
{
|
||||
"web_display_autostart": true,
|
||||
"auto_update": {
|
||||
"enabled": false
|
||||
"enabled": false,
|
||||
"channel": "stable"
|
||||
},
|
||||
"schedule": {
|
||||
"enabled": false,
|
||||
|
||||
+70
-68
@@ -10,22 +10,27 @@ This guide covers advanced LEDMatrix features for users and developers, includin
|
||||
|
||||
Vegas scroll mode displays content from multiple plugins in a continuous horizontal scroll, similar to news tickers seen in Las Vegas casinos. Plugins contribute content segments that flow across the display in a seamless ticker-style presentation.
|
||||
|
||||
### Display Modes
|
||||
### How a Plugin Takes Part
|
||||
|
||||
**SCROLL (Continuous Scrolling):**
|
||||
- Content scrolls continuously left
|
||||
- Smooth, fluid motion
|
||||
- Best for news-ticker style displays
|
||||
Each plugin has a *Vegas participation*:
|
||||
|
||||
**FIXED_SEGMENT (Fixed-Width Block):**
|
||||
- Plugin gets fixed-width block on display
|
||||
- Content doesn't scroll out of its segment
|
||||
- Multiple plugins can share the display simultaneously
|
||||
**`scroll` (the default):**
|
||||
- The plugin's content scrolls by with everyone else's
|
||||
- Best for news-ticker style content: scores, headlines, prices, the time
|
||||
|
||||
**STATIC (Scroll Pauses):**
|
||||
- Scrolling pauses when content is fully visible
|
||||
- Displays for specified duration, then resumes scrolling
|
||||
- Best for content that needs to be fully read
|
||||
**`pause`:**
|
||||
- The scroll stops when the plugin's turn comes round
|
||||
- The plugin draws the whole panel for its display duration, then the
|
||||
scroll resumes
|
||||
- Best for content that needs to be read in full, or alerts
|
||||
|
||||
**`exclude`:**
|
||||
- The plugin is left out of Vegas mode
|
||||
|
||||
A plugin declares its default; set `vegas_participation` in a plugin's
|
||||
config to override it (see [Per-Plugin Configuration](#per-plugin-configuration)).
|
||||
Older documentation also describes a *fixed segment* mode; Vegas never
|
||||
implemented one, and it has always behaved exactly like `scroll`.
|
||||
|
||||
### Configuration
|
||||
|
||||
@@ -164,8 +169,7 @@ Override Vegas behavior for specific plugins:
|
||||
{
|
||||
"my_plugin": {
|
||||
"enabled": true,
|
||||
"vegas_mode": "scroll",
|
||||
"vegas_panel_count": 2,
|
||||
"vegas_participation": "pause",
|
||||
"display_duration": 10
|
||||
}
|
||||
}
|
||||
@@ -175,19 +179,30 @@ Override Vegas behavior for specific plugins:
|
||||
|
||||
| Setting | Values | Description |
|
||||
|---------|--------|-------------|
|
||||
| `vegas_mode` | `scroll`, `fixed`, `static` | Display mode for this plugin |
|
||||
| `vegas_panel_count` | any positive integer | Width in panels (1 panel = display width) |
|
||||
| `display_duration` | seconds | Pause duration for STATIC mode |
|
||||
| `vegas_participation` | `scroll`, `pause`, `exclude` | How this plugin takes part: its content scrolls by, the scroll pauses for its turn and shows it full screen, or it is left out. Unset uses the plugin's own default |
|
||||
| `display_duration` | seconds | How long a `pause` plugin holds the screen |
|
||||
| `vegas_width_pct` | 10–100 | Width of this plugin's card, as a percentage of the panel |
|
||||
| `vegas_overflow` | `rotate`, `truncate` | What to do when its content is wider than its allowance |
|
||||
| `vegas_max_width_screens` | number of screens | The widest its card may be |
|
||||
|
||||
Plugins may also set `vegas_overflow` and `vegas_max_width_screens` in
|
||||
their config section to control how oversized content is handled (see
|
||||
`PluginManager` in `src/plugin_system/plugin_manager.py`).
|
||||
These are core-owned settings (see
|
||||
[PLUGIN_CONFIG_CORE_PROPERTIES.md](PLUGIN_CONFIG_CORE_PROPERTIES.md)): every
|
||||
plugin accepts them whether or not its own schema lists them. Set them in
|
||||
the plugin's section of config.json, in the web UI's **Config Editor**
|
||||
tab.
|
||||
|
||||
Some plugins also offer a `vegas_mode` setting of their own (`scroll`,
|
||||
`fixed` or `static`). It still works — `static` pauses, the other two scroll
|
||||
— but `vegas_participation` takes precedence, and `fixed` has never done
|
||||
anything different from `scroll`. The old `vegas_panel_count` setting never
|
||||
had an effect and is deprecated (removed in 3.9.0).
|
||||
|
||||
### Plugin Integration (Developer Guide)
|
||||
|
||||
All of these have defaults in
|
||||
[`BasePlugin`](../src/plugin_system/base_plugin.py); override only what you
|
||||
need.
|
||||
need. The reference is
|
||||
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#vegas-scroll-hooks).
|
||||
|
||||
**1. Implement Content Method:**
|
||||
|
||||
@@ -203,43 +218,41 @@ If it returns `None` (the default), Vegas falls back to the plugin's
|
||||
(`PluginAdapter.get_content()` in
|
||||
[`src/vegas_mode/plugin_adapter.py`](../src/vegas_mode/plugin_adapter.py)).
|
||||
|
||||
**2. Specify Content Type:**
|
||||
**2. Declare how the plugin takes part:**
|
||||
|
||||
```python
|
||||
def get_vegas_content_type(self):
|
||||
# 'multi' | 'static' | 'none' -- default is 'static'
|
||||
return 'multi'
|
||||
Most plugins need nothing: the default is `scroll`. A plugin that should
|
||||
pause the scroll, or stay out of Vegas, says so in `manifest.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"vegas_participation": "pause"
|
||||
}
|
||||
```
|
||||
|
||||
`'none'` excludes the plugin from Vegas mode.
|
||||
|
||||
**3. Optionally Specify Display Mode:**
|
||||
|
||||
These return `VegasDisplayMode` members, not strings:
|
||||
The user's own `vegas_participation` setting overrides the manifest. When
|
||||
the answer depends on state, override the method instead:
|
||||
|
||||
```python
|
||||
from src.plugin_system.base_plugin import VegasDisplayMode
|
||||
|
||||
def get_vegas_display_mode(self):
|
||||
return VegasDisplayMode.SCROLL
|
||||
|
||||
def get_supported_vegas_modes(self):
|
||||
return [VegasDisplayMode.SCROLL, VegasDisplayMode.STATIC]
|
||||
def get_vegas_participation(self):
|
||||
# 'scroll' | 'pause' | 'exclude'
|
||||
return 'pause' if self._alert_is_live() else 'scroll'
|
||||
```
|
||||
|
||||
`VegasDisplayMode` has `SCROLL` (`"scroll"`), `FIXED_SEGMENT` (`"fixed"`) and
|
||||
`STATIC` (`"static"`). The default `get_vegas_display_mode()` uses the
|
||||
plugin's `vegas_mode` config value if set, otherwise maps the content type
|
||||
(`multi` to `SCROLL`, anything else to `FIXED_SEGMENT`).
|
||||
A plugin written for an older core that declares nothing keeps its
|
||||
behaviour: `get_vegas_display_mode()` returning `VegasDisplayMode.STATIC`
|
||||
pauses, `get_vegas_content_type()` returning `'none'` excludes, and
|
||||
everything else scrolls. `get_supported_vegas_modes()`,
|
||||
`get_vegas_segment_width()` and the SCROLL / FIXED_SEGMENT distinction are
|
||||
deprecated (removed in 3.9.0): Vegas never read them.
|
||||
|
||||
### Content Rendering Guidelines
|
||||
|
||||
**Image Dimensions:**
|
||||
- **Height:** Must match display height (typically 32 pixels)
|
||||
- **Width:** Varies by mode:
|
||||
- SCROLL: Any width (recommended 64-512 pixels)
|
||||
- FIXED_SEGMENT: `panel_count * display_width`
|
||||
- STATIC: Any width, optimized for readability
|
||||
- **Width:** Any width for `scroll` (recommended 64-512 pixels);
|
||||
`get_vegas_render_width()` is the width Vegas would like, and it narrows
|
||||
`display_manager` to match while it asks. A `pause` plugin draws the
|
||||
whole panel in `display()`.
|
||||
|
||||
**Color Mode:**
|
||||
- Use RGB color mode
|
||||
@@ -289,17 +302,10 @@ class WeatherPlugin(BasePlugin):
|
||||
def get_vegas_content(self):
|
||||
"""Return cached Vegas image"""
|
||||
return self.vegas_image
|
||||
|
||||
def get_vegas_content_type(self):
|
||||
return 'multi'
|
||||
|
||||
def get_vegas_display_mode(self):
|
||||
return 'scroll'
|
||||
|
||||
def get_supported_vegas_modes(self):
|
||||
return ['scroll', 'static']
|
||||
```
|
||||
|
||||
It scrolls, the default participation, so it declares nothing else.
|
||||
|
||||
### System Architecture
|
||||
|
||||
Vegas mode consists of four core components working together to provide smooth 125 FPS continuous scrolling:
|
||||
@@ -382,7 +388,8 @@ Vegas mode consists of four core components working together to provide smooth 1
|
||||
|
||||
**Responsibilities:**
|
||||
- Convert plugin content to scrollable images
|
||||
- Handle different Vegas display modes (SCROLL, FIXED, STATIC)
|
||||
- Fetch the content of `scroll` plugins (a `pause` plugin is drawn by
|
||||
its own `display()` when the scroll pauses; see StreamManager)
|
||||
- Manage fallback for plugins without Vegas support
|
||||
- Cache plugin content for performance
|
||||
|
||||
@@ -391,21 +398,16 @@ Vegas mode consists of four core components working together to provide smooth 1
|
||||
- Calls `get_vegas_content()` if available
|
||||
- Falls back to `display()` method if not
|
||||
|
||||
2. **Handle display mode:**
|
||||
- SCROLL: Returns image as-is for continuous scrolling
|
||||
- FIXED_SEGMENT: Creates fixed-width block (panel_count * display_width)
|
||||
- STATIC: Marks content for pause-when-visible behavior
|
||||
|
||||
3. **Content type handling:**
|
||||
- `multi`: Multiple segments (list of images)
|
||||
- `static`: Single static image
|
||||
- `none`: Skip this plugin in current cycle
|
||||
2. **Participation** is decided by the StreamManager, not here
|
||||
(`resolve_vegas_participation()` in
|
||||
[`base_plugin.py`](../src/plugin_system/base_plugin.py)): `exclude`
|
||||
plugins never reach the adapter, and `pause` plugins are not fetched.
|
||||
|
||||
**Fallback Behavior:**
|
||||
- If plugin doesn't implement Vegas methods:
|
||||
- Calls plugin's `display()` method
|
||||
- Captures rendered display as static image
|
||||
- Treats as fixed segment
|
||||
- Scrolls it by as one block
|
||||
- Ensures all plugins work in Vegas mode without explicit support
|
||||
|
||||
#### 4. RenderPipeline
|
||||
@@ -508,7 +510,7 @@ All components use thread-safe patterns:
|
||||
If a plugin doesn't implement Vegas methods:
|
||||
- System calls the plugin's `display()` method
|
||||
- Captures the rendered display as a static image
|
||||
- Treats it as a fixed segment
|
||||
- Scrolls it by as one block
|
||||
|
||||
This ensures all plugins work in Vegas mode, even without explicit support.
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ Advanced patterns, examples, and best practices for developing LEDMatrix plugins
|
||||
|
||||
The Display Manager's icon methods — `draw_weather_icon()`, `draw_sun()`,
|
||||
`draw_cloud()`, `draw_rain()`, `draw_snow()` and `draw_text_with_icons()` —
|
||||
are deprecated, removed in 3.7.0. Draw your own icons instead: render them
|
||||
are deprecated, removed in 3.8.0. Draw your own icons instead: render them
|
||||
onto a PIL image and paste it onto `self.display_manager.image`, or ship
|
||||
icon images with the plugin. The weather plugin's `WeatherIcons` class is an
|
||||
example. See [Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis).
|
||||
@@ -194,7 +194,7 @@ def update(self):
|
||||
sport_key = "nhl"
|
||||
cache_key = f"{self.plugin_id}_{sport_key}_games"
|
||||
|
||||
# get_background_cached_data() is deprecated, removed in 3.7.0 — use get()
|
||||
# get_background_cached_data() is deprecated, removed in 3.8.0 — use get()
|
||||
cached = self.cache_manager.get(cache_key, max_age=60)
|
||||
|
||||
if cached:
|
||||
@@ -596,7 +596,7 @@ def update(self):
|
||||
|
||||
```python
|
||||
def update(self):
|
||||
# get_enabled_plugins() is deprecated, removed in 3.7.0 — check the
|
||||
# get_enabled_plugins() is deprecated, removed in 3.8.0 — check the
|
||||
# instance's `enabled` flag instead
|
||||
weather_plugin = self.plugin_manager.get_plugin("weather")
|
||||
if weather_plugin is not None and weather_plugin.enabled:
|
||||
|
||||
+183
-11
@@ -48,12 +48,123 @@ each other. They share three things:
|
||||
| Error clear | cache `plugin_error_clear_request` | web | display |
|
||||
| Font usage | cache `font_usage_snapshot` | display: `FontUsagePublisher` ([`src/font_usage.py`](../src/font_usage.py)) | web: Fonts tab |
|
||||
| Plugin health | cache `plugin_health:<id>` | display (web writes on reset) | web: `/api/v3/plugins/health` |
|
||||
| Plugin runtime (loaded, state, last error, version) | cache `plugin_runtime_snapshot` | display: `PluginRuntimePublisher` ([`src/plugin_system/plugin_runtime.py`](../src/plugin_system/plugin_runtime.py)) | web: `read_plugin_runtime()` for `/api/v3/plugins/installed`, `/plugins/state`, reconciliation |
|
||||
| Preview frame | `/tmp/led_matrix_preview.png` | display: `DisplayManager`, gated by [`snapshot_policy`](../src/common/snapshot_policy.py) | web: display SSE stream, `/api/v3/health` (file age) |
|
||||
| Preview viewer marker | `/tmp/led_matrix_preview_viewer` | web, while a preview is open | display: writes full-rate snapshots only while it is fresh |
|
||||
| Hardware init status | `/tmp/led_matrix_hw_status.json` | display | web: `/api/v3/hardware/status` |
|
||||
| Render-loop heartbeat | `/run/ledmatrix/display-heartbeat.json` (tmpfs) | display: the render thread, via [`display_watchdog`](../src/display_watchdog.py) | web: `/api/v3/health` (`checks.display_loop`); the update health check |
|
||||
|
||||
The on-demand start route also restarts `ledmatrix.service` by default so the
|
||||
request takes effect straight away.
|
||||
The on-demand start route starts `ledmatrix.service` when it is not running
|
||||
(`start_service`, on by default) but never restarts a running one: the display
|
||||
reads the mailbox every `ON_DEMAND_POLL_INTERVAL` (0.25s), from its dwell
|
||||
sleep, its render loops and Vegas's interrupt check as well as the main loop.
|
||||
|
||||
### Web and display processes: who runs plugins
|
||||
|
||||
Only the display process imports plugin code, instantiates plugins and calls
|
||||
their lifecycle hooks (`update`, `display`, `on_config_change`, `on_enable`,
|
||||
`on_disable`). The web process is metadata-only: it reads plugins as files
|
||||
through `PluginCatalog`
|
||||
([`src/plugin_system/plugin_catalog.py`](../src/plugin_system/plugin_catalog.py))
|
||||
-- manifests, config schemas (through `SchemaManager`), each plugin's
|
||||
section of `config.json`, and installed versions. The catalog keeps the
|
||||
read-only method names of `PluginManager` and has nothing that can run a
|
||||
plugin (no `load_plugin`, `get_plugin` or `plugins`).
|
||||
|
||||
How a web-side change reaches the running plugins:
|
||||
|
||||
| Change | How the display picks it up |
|
||||
|---|---|
|
||||
| Plugin settings saved, config reset | `ConfigService` sees the new `config.json` and calls the plugin's `on_config_change` with the prepared section |
|
||||
| Plugin enabled or disabled | `ConfigService` → `_controller_config_change` flags a reconcile; `_reconcile_enabled_plugins` loads it (fresh from disk) or unloads it on the render thread |
|
||||
| Plugin uninstalled (config removed) | the removed section flips its `enabled` flag, and the reconcile unloads it |
|
||||
| Plugin installed, not enabled | nothing to do until it is enabled, which loads it |
|
||||
| Plugin installed while already enabled, updated while enabled, or uninstalled with its config kept | **not picked up**: the display keeps running what it loaded. The route answers `restart_required: true` and the UI shows its restart banner |
|
||||
|
||||
`display_restart_required()` in `plugin_catalog.py` holds that last rule;
|
||||
routes return it as `restart_required` (with the banner's wording in
|
||||
`restart_message`), and `window.noteRestartRequired()` in
|
||||
`static/v3/app.js` raises the banner for any response that carries it,
|
||||
`POST /api/v3/config/main` included.
|
||||
|
||||
Runtime state shown in the UI comes from what the display publishes to the
|
||||
shared cache: health and metrics (`/api/v3/plugins/health`,
|
||||
`/plugins/metrics`), errors (`/api/v3/errors/*`), the current mode, and the
|
||||
plugin runtime snapshot described below. `enabled` is read from
|
||||
`config.json` by the display's rule (a missing flag is disabled).
|
||||
|
||||
Plugin code still runs in the web process in one place,
|
||||
`_import_plugin_code_in_web_process()` in
|
||||
[`api_v3/__init__.py`](../web_interface/blueprints/api_v3/__init__.py): the
|
||||
Starlark routes import the starlark-apps plugin's `tronbyte_repository` and
|
||||
`pixlet_renderer` helper modules (never the plugin class), and a web-UI
|
||||
action with `oauth_flow` imports its script for `get_auth_url()`. Every
|
||||
other web-UI action runs its script as a subprocess. A later, explicit
|
||||
**plugin web-entry contract** -- a declared entry point for plugin web code
|
||||
-- replaces that function.
|
||||
|
||||
Next stages: a **control socket** from the web process to the display
|
||||
(reload one plugin, ask for its state) in place of `restart_required` and
|
||||
the cache-key mailboxes, and the plugin web-entry contract above.
|
||||
|
||||
### Plugin state: desired, observed, and who owns it
|
||||
|
||||
There is one plugin state machine, and the display owns it:
|
||||
`PluginStateManager` in
|
||||
[`plugin_state.py`](../src/plugin_system/plugin_state.py) (unloaded →
|
||||
loaded → enabled ⇄ running, error, disabled), held by the display's
|
||||
`PluginManager`. It also records, per loaded plugin, the manifest version it
|
||||
loaded and when. Nothing else keeps plugin state:
|
||||
|
||||
| Question | Answered by |
|
||||
|---|---|
|
||||
| Is it installed, at which version? | the plugins directory (`manifest.json`) |
|
||||
| Should it run? | `config.json` (`<id>.enabled`, missing = disabled) |
|
||||
| Has the user uninstalled it for good? | the store's uninstalled-plugins record |
|
||||
| Is the display running it, at which version, and why not? | the display's runtime snapshot |
|
||||
|
||||
**The runtime snapshot.** `PluginRuntimePublisher`
|
||||
([`plugin_runtime.py`](../src/plugin_system/plugin_runtime.py)), started by
|
||||
`DisplayController` right after it creates the `PluginManager`, writes the
|
||||
cache key `plugin_runtime_snapshot`: per plugin `loaded`, `state`, `error`
|
||||
(type, a redacted message of at most 200 characters, when, recoverable),
|
||||
`version` and `loaded_at`, plus `published_at`, `stale_after` and `running`.
|
||||
The cache is on disk, usually the SD card, so it writes when something a
|
||||
reader sees changes -- throttled to once per 10 s -- and otherwise once a
|
||||
minute as a heartbeat. RUNNING, which every `update()` passes through, is
|
||||
published as ENABLED, so plugin updates alone never cause a write.
|
||||
`cleanup()` publishes `running: false`.
|
||||
|
||||
**Reading it.** `read_plugin_runtime()` judges the snapshot before anyone
|
||||
uses it: `live` (fresh, from a running display), `stale` (older than
|
||||
`stale_after`, 3 minutes: a hung or crashed display), `stopped` or
|
||||
`unknown` (none, unreadable, or another schema). Only a live view reports
|
||||
per-plugin facts; every other status answers `null` for them, so stale
|
||||
truth cannot leak into a response. `/api/v3/plugins/installed` returns
|
||||
`loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` per
|
||||
plugin and `data.runtime` (`status`, `published_at`, `age_seconds`);
|
||||
`/api/v3/plugins/state` returns the same beside the desired state.
|
||||
|
||||
**Reconciliation**
|
||||
([`state_reconciliation.py`](../src/plugin_system/state_reconciliation.py))
|
||||
compares desired state (config + disk) with observed state (the snapshot).
|
||||
It fixes desired-state gaps -- a plugin on disk with no config section gets
|
||||
`{"enabled": false}`, a configured plugin missing from disk is reinstalled
|
||||
unless the user uninstalled it -- and only reports observed-state gaps
|
||||
(enabled but not loaded, loaded at an older version): the display loads and
|
||||
unloads by config on its own, and a version gap needs a restart.
|
||||
|
||||
**`data/plugin_state.json` is retired.** The web process used to keep a
|
||||
second `PluginStateManager` (`state_manager.py`) persisted to that file:
|
||||
per plugin an enabled flag copied from config, a version copied from the
|
||||
manifest (when set at all), a status derived from those, and install/update
|
||||
timestamps. Reconciliation mostly synced it back to config and backups
|
||||
merged it into their plugin list. Every field is derivable (the timestamps
|
||||
from the operation history), so nothing is migrated: no code reads or
|
||||
writes the file, and a copy left on a device is inert and safe to delete.
|
||||
The two classes shared a name but not a concern -- a persisted install
|
||||
record versus the live lifecycle -- so they were not merged; the persisted
|
||||
one had nothing left to hold and was removed.
|
||||
|
||||
## Display loop
|
||||
|
||||
@@ -82,7 +193,11 @@ then normal rotation.
|
||||
- **On-demand.** A request from the web interface pins one plugin (or mode)
|
||||
for a duration. `_activate_on_demand()` / `_clear_on_demand()`; the
|
||||
session is saved under `display_on_demand_config` so it survives a
|
||||
restart. It also keeps the display on during scheduled off hours.
|
||||
restart. It also keeps the display on during scheduled off hours. A
|
||||
request for a plugin that is disabled in config loads it live
|
||||
(`_load_plugin_for_on_demand()`, `load_plugin(force_enabled=True)`)
|
||||
without writing `config.json`; the main loop unloads it once on-demand
|
||||
moves off it (`_release_on_demand_plugins()`).
|
||||
- **Live priority.** `_check_live_priority()` looks for a plugin whose
|
||||
`has_live_priority()` and `has_live_content()` are both true and switches
|
||||
to it, rotating between several live games.
|
||||
@@ -99,7 +214,8 @@ then normal rotation.
|
||||
changes. The controller refreshes its cached settings; enabling or
|
||||
disabling a plugin queues `_reconcile_enabled_plugins()`, which loads or
|
||||
unloads it on the display thread; each plugin gets `on_config_change()`
|
||||
for its own section. Set `LEDMATRIX_HOT_RELOAD=false` to turn this off.
|
||||
for its own section, under its plugin lock
|
||||
(`PluginManager.apply_config_change()`). Set `LEDMATRIX_HOT_RELOAD=false` to turn this off.
|
||||
Matrix hardware settings are only read at start-up.
|
||||
- **Vegas mode.** [`src/vegas_mode/`](../src/vegas_mode/): the display loop
|
||||
calls `VegasModeCoordinator.run_iteration()`
|
||||
@@ -113,6 +229,43 @@ then normal rotation.
|
||||
`sync.role`: a leader sends a follower its share of each frame over UDP
|
||||
(port 5765).
|
||||
|
||||
### Liveness
|
||||
|
||||
A render thread stuck inside a plugin leaves the service "active" and the
|
||||
panel frozen, so liveness is reported by the render thread itself
|
||||
([`src/display_watchdog.py`](../src/display_watchdog.py), standard library
|
||||
only). `beat()` from any other thread is ignored: the update worker, Vegas's
|
||||
tick thread and the prefetcher keep running while the render thread is stuck,
|
||||
and must not vouch for it.
|
||||
|
||||
- **Check-in points.** The top of `run()`'s loop (`loop_pass()`), every
|
||||
dwell second (`_sleep_with_plugin_updates`), every frame of the per-screen
|
||||
loops (`_display_once`), every frame of Vegas's own loop and static pause
|
||||
(`coordinator.run_iteration`), each plugin fetched for a Vegas cycle
|
||||
(`StreamManager._fetch_plugin_content`), each update on the
|
||||
`synchronous_updates` path, and every frame pushed
|
||||
(`DisplayManager.update_display` -> `note_frame()`). Beats are
|
||||
rate-limited to one ping and one heartbeat write every 5 s.
|
||||
- **systemd watchdog.** `ledmatrix.service` is `Type=simple` with
|
||||
`WatchdogSec=120` and `NotifyAccess=main`. `run.py` sends
|
||||
`WATCHDOG_USEC` = 15 minutes before importing anything heavy (start-up loads
|
||||
plugins and runs the 20 s update budget, and the watchdog clock starts with
|
||||
the process). After the first frame -- or the first full pass, when there is
|
||||
nothing to draw -- the loop sends `READY=1`, restores the unit's 120 s and
|
||||
pings. `PluginManager.load_plugin()` on the render thread (a plugin enabled
|
||||
from the web UI, or loaded for on-demand) gets 15 minutes again, since it
|
||||
can run pip. A missed deadline is a SIGABRT; faulthandler, enabled on
|
||||
arming, dumps every thread's stack to the journal.
|
||||
- **Heartbeat.** `/run/ledmatrix/display-heartbeat.json`
|
||||
(`{"pid", "mono", "wall"}`; `RuntimeDirectory=ledmatrix`, 0755, file 0644 so
|
||||
the web user can read it). Readers compare `mono` with their own
|
||||
`time.monotonic()` -- CLOCK_MONOTONIC is shared by every process and does not
|
||||
jump when NTP first sets an RTC-less Pi's clock. `/api/v3/health` calls it
|
||||
`stalled` past 60 s; no file is `not_reported` and changes nothing. A clean
|
||||
stop removes it. Without `RuntimeDirectory=` (an older unit) the display,
|
||||
as root, creates the directory itself; off Linux, or without root, there
|
||||
is no heartbeat.
|
||||
|
||||
## Plugin system
|
||||
|
||||
[`src/plugin_system/`](../src/plugin_system/):
|
||||
@@ -121,7 +274,8 @@ then normal rotation.
|
||||
|---|---|
|
||||
| Base class plugins implement | [`base_plugin.py`](../src/plugin_system/base_plugin.py) (`BasePlugin`, `VegasDisplayMode`) |
|
||||
| Finding a plugin's directory | [`plugin_dirs.py`](../src/plugin_system/plugin_dirs.py): manifest `id` first, then directory `<id>` or `ledmatrix-<id>` |
|
||||
| Discovery, load, unload, scheduled updates | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
|
||||
| Discovery, load, unload, scheduled updates (display process) | [`plugin_manager.py`](../src/plugin_system/plugin_manager.py) (`PluginManager`) |
|
||||
| Manifest, schema, config and version reads (web process) | [`plugin_catalog.py`](../src/plugin_system/plugin_catalog.py) (`PluginCatalog`; see [who runs plugins](#web-and-display-processes-who-runs-plugins)) |
|
||||
| Import and instantiate | [`plugin_loader.py`](../src/plugin_system/plugin_loader.py) (`PluginLoader.load_plugin()`: dependencies, module, class) |
|
||||
| Timeouts | [`plugin_executor.py`](../src/plugin_system/plugin_executor.py) (`PluginExecutor`, 30 s default; a timed-out thread is abandoned, not killed) |
|
||||
| Circuit breaker | [`plugin_health.py`](../src/plugin_system/plugin_health.py) (`PluginHealthTracker`: 3 consecutive failures open the circuit for 300 s) |
|
||||
@@ -148,8 +302,9 @@ everything else through `_reinstall_with_rollback()`.
|
||||
## Web interface
|
||||
|
||||
- **App.** [`web_interface/app.py`](../web_interface/app.py) builds the
|
||||
Flask `app` at import time, creates the managers, and registers two
|
||||
blueprints. `web_interface/start.py` runs it on port 5000.
|
||||
Flask `app` at import time, creates the managers -- a `PluginCatalog`,
|
||||
never a `PluginManager` -- and registers two blueprints.
|
||||
`web_interface/start.py` runs it on port 5000.
|
||||
- **Pages.** [`blueprints/pages_v3.py`](../web_interface/blueprints/pages_v3.py)
|
||||
serves the shell `templates/v3/base.html` at `/` and each tab as a
|
||||
partial at `/partials/<name>` (templates in
|
||||
@@ -181,8 +336,19 @@ everything else through `_reinstall_with_rollback()`.
|
||||
- **Update Code** on the Overview tab and the automatic updater both call
|
||||
`perform_core_update()` in
|
||||
[`api_v3/system.py`](../web_interface/blueprints/api_v3/system.py):
|
||||
`git pull --rebase`, reinstall changed requirement files, report whether a
|
||||
restart is needed.
|
||||
fetch branches and tags, move the checkout for the update channel, reinstall
|
||||
changed requirement files, report whether a restart is needed.
|
||||
- **Update channels** (`auto_update.channel`):
|
||||
[`web_interface/update_channel.py`](../web_interface/update_channel.py)
|
||||
decides the move. `stable` checks out the newest `vX.Y.Z` tag (detached
|
||||
HEAD) when it contains the current commit; `beta` is
|
||||
`git pull --rebase --autostash` on the current branch, and leaves a
|
||||
detached release for `main` first. A stable device newer than the newest
|
||||
release keeps pulling `main` until a release contains its commit, so no
|
||||
update ever moves backwards; a config without the key is written as
|
||||
`stable` once the device reaches a release. Checkouts carry uncommitted
|
||||
edits across with `git stash create`/`apply`, and keep them in the stash
|
||||
list if they no longer apply.
|
||||
- **Automatic updates** (`auto_update.enabled`, off by default):
|
||||
`AutoUpdater` in [`web_interface/auto_update.py`](../web_interface/auto_update.py)
|
||||
runs in the web process, checks every 30 minutes, and updates at most
|
||||
@@ -193,7 +359,11 @@ everything else through `_reinstall_with_rollback()`.
|
||||
`ledmatrix-update-verify.path`, which runs the verifier as a separate unit
|
||||
(so restarting the web service does not kill it). The verifier restarts
|
||||
both services, waits for the web API to answer and the display service to
|
||||
stay up, and on failure resets to the previous commit and restarts again.
|
||||
stay up -- and, when the display wrote a heartbeat before the update, to
|
||||
keep one fresh from the restarted process (see Liveness) -- and on failure
|
||||
returns to where HEAD was (the branch, or detached on the previous
|
||||
release; `old_ref` in the pending file), resets to the previous commit
|
||||
and restarts again.
|
||||
Plugin updates run only after a verified core update. State is in
|
||||
`data/auto_update_state.json` and `data/auto_update_pending.json`.
|
||||
- **Startup validator.** `StartupValidator`
|
||||
@@ -201,7 +371,9 @@ everything else through `_reinstall_with_rollback()`.
|
||||
`DisplayController.__init__`: config and cache directory first, then
|
||||
enabled plugins once the plugin manager exists. It also warns when an
|
||||
installed systemd unit differs from its template in `systemd/`. Results
|
||||
are logged; startup continues either way.
|
||||
are logged; startup continues either way. Nothing rewrites installed units
|
||||
on update: a unit change such as the watchdog reaches an existing install
|
||||
only when `install_service.sh` is re-run.
|
||||
|
||||
## Where to start reading
|
||||
|
||||
|
||||
@@ -17,6 +17,7 @@ tooling against it.
|
||||
|---|---|---|---|
|
||||
| `web_display_autostart` | bool, `true` | Whether the web interface service starts with the system | `scripts/utils/start_web_conditionally.py` |
|
||||
| `auto_update.enabled` | bool, `false` | Weekly automatic updates: LEDMatrix code first (health-checked, rolled back on failure), then installed plugins. Toggle in the General tab or install with `first_time_install.sh --enable-auto-update` | `web_interface/auto_update.py`, `src/auto_update_setup.py` (`is_enabled()`) |
|
||||
| `auto_update.channel` | `"stable"` or `"beta"`, `"stable"` (template) | What Update Code and the weekly update install. `stable`: the newest `vX.Y.Z` release tag (pre-releases ignored), checked out with a detached HEAD. `beta`: `main`. Never moves a device backwards: one newer than the newest release keeps following `main` until a release contains its commit. Missing (configs from before channels) behaves like `stable` and is saved as `stable` once the device is on a release. General tab, Update Channel | `web_interface/update_channel.py` (`resolve()`) |
|
||||
| `timezone` | string, `"America/New_York"` | IANA timezone for schedules and displays | `ConfigManager.get_timezone()` |
|
||||
| `target_fps` | int, `100` | Legacy "Scroll Frame Rate". Core scrolling no longer reads it: scroll frames are presented at `display.hardware.limit_refresh_rate_hz` divided by each scroll's frame hold, and speed comes from each plugin's scroll settings. Still exposed to plugins via `BasePlugin.global_config` | `src/plugin_system/base_plugin.py` |
|
||||
| `location` | object | `city` / `state` / `country`. Supplies the **default** for a plugin's own `location_city` / `location_state` / `location_country` setting, so weather, radar and friends follow this device without being configured twice. A value saved on the plugin itself still overrides it. Starlark (Tidbyt) apps get the same treatment: a `Location` field left blank on the app renders at this city (geocoded once via Open-Meteo, coordinates cached permanently) instead of the app author's default, which is usually San Francisco. If the city can't be looked up (no match, or the geocoder is unreachable; retried after 30 minutes), the app keeps its own default. | `SchemaManager.apply_device_location()`, then plugins via merged config; `src/device_location.py` for Starlark apps |
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
# Deprecated plugin APIs: usage scan
|
||||
|
||||
Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it (see [How to re-run](#how-to-re-run)).
|
||||
|
||||
- Scanned: 2026-09-30, core 3.7.0
|
||||
- Monorepo: [ChuckBuilds/ledmatrix-plugins](https://github.com/ChuckBuilds/ledmatrix-plugins) (main @ 4327c2e4), 46 plugins
|
||||
- Third-party plugins: 8 with their own repo in `plugins.json` (f1-live, gif-player, pga-tour-leaderboard, plex-marquee, ledmatrix-dresden-departures, tidbyt-baseball-scoreboard, sleeper-fantasy, ledmatrix-nascar)
|
||||
|
||||
**37 deprecated methods: 36 unused, 1 still used, 0 need review.**
|
||||
|
||||
Counted per plugin: a *call* is `<receiver>.method` on an object named like the owner (`cache_manager`, `display_manager`, `font_manager`, `plugin_manager`), or on `self`/`super()` in a subclass; an *override* is `def method` in a subclass of the owner. *Review* hits are `.method` on a receiver whose type the scan cannot tell. *Internal* hits sit inside another deprecated core method and go with it. *Unrelated* hits are a different class's own method with the same name (a name collision), and never block removal; neither do hits in test files.
|
||||
|
||||
| Method | Removal | Core | Plugins (calls / overrides) | Name collisions & tests | Verdict |
|
||||
|---|---|---|---|---|---|
|
||||
| `CacheManager.has_data_changed` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.update_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.setup_persistent_cache` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_sport_live_interval` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_background_cached_data` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.is_background_data_available` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_cache_hit` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_cache_miss` | 3.8.0 | core (1 internal) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.record_fetch_time` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.log_cache_metrics` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `CacheManager.get_memory_cache_stats` | 3.8.0 | core tests (3 test calls) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_sun` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_cloud` | 3.8.0 | core (2 internals) | — | ledmatrix-weather (1 unrelated) | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_rain` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_snow` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_weather_icon` | 3.8.0 | core (1 internal) | — | ledmatrix-weather (5 unrelateds) | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.draw_text_with_icons` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `DisplayManager.get_scrolling_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_manager_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_detected_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.unregister_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_plugin_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.set_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.remove_override` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_overrides` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_available_fonts` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_size_tokens` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_performance_stats` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.get_font_catalog` | 3.8.0 | core tests (1 test call) | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.add_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.remove_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `FontManager.validate_font` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
| `BasePlugin.get_supported_vegas_modes` | 3.9.0 | core tests (2 test reviews) | blackjack (1 call, 1 override); calendar (1 override); olympics (1 override) | — | still used by blackjack, calendar, olympics — keep or migrate first |
|
||||
| `BasePlugin.get_vegas_segment_width` | 3.9.0 | core tests (1 test review) | — | — | unused — safe to remove in 3.9.0 |
|
||||
| `PluginManager.get_enabled_plugins` | 3.8.0 | — | — | — | unused — safe to remove in 3.8.0 |
|
||||
|
||||
## Unused — safe to remove (36)
|
||||
|
||||
`CacheManager.has_data_changed`, `CacheManager.update_cache`, `CacheManager.setup_persistent_cache`, `CacheManager.get_sport_live_interval`, `CacheManager.get_sport_key_from_cache_key`, `CacheManager.get_background_cached_data`, `CacheManager.is_background_data_available`, `CacheManager.record_cache_hit`, `CacheManager.record_cache_miss`, `CacheManager.record_fetch_time`, `CacheManager.get_cache_metrics`, `CacheManager.log_cache_metrics`, `CacheManager.get_memory_cache_stats`, `DisplayManager.draw_sun`, `DisplayManager.draw_cloud`, `DisplayManager.draw_rain`, `DisplayManager.draw_snow`, `DisplayManager.draw_weather_icon`, `DisplayManager.draw_text_with_icons`, `DisplayManager.get_scrolling_stats`, `FontManager.get_manager_fonts`, `FontManager.get_detected_fonts`, `FontManager.unregister_plugin_fonts`, `FontManager.get_plugin_fonts`, `FontManager.set_override`, `FontManager.remove_override`, `FontManager.get_overrides`, `FontManager.get_available_fonts`, `FontManager.get_size_tokens`, `FontManager.get_performance_stats`, `FontManager.get_font_catalog`, `FontManager.add_font`, `FontManager.remove_font`, `FontManager.validate_font`, `BasePlugin.get_vegas_segment_width`, `PluginManager.get_enabled_plugins`
|
||||
|
||||
## Still used — keep or migrate first (1)
|
||||
|
||||
`BasePlugin.get_supported_vegas_modes`
|
||||
|
||||
## Every hit
|
||||
|
||||
File paths are relative to the plugin's directory (core: the repo root).
|
||||
|
||||
| Method | Where | File:line | Kind | Code |
|
||||
|---|---|---|---|---|
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:28 | unrelated | `def get_sport_live_interval(self, sport_key: str) -> int:` |
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache/cache_strategy.py:60 | unrelated | `live_interval = self.get_sport_live_interval(sport_key)` |
|
||||
| `CacheManager.get_sport_live_interval` | core | src/cache_manager.py:785 | unrelated | `return self._strategy_component.get_sport_live_interval(sport_key)` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache/cache_strategy.py:214 | unrelated | `def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:806 | unrelated | `return self._strategy_component.get_sport_key_from_cache_key(key)` |
|
||||
| `CacheManager.get_sport_key_from_cache_key` | core | src/cache_manager.py:816 | unrelated | `sport_key = self._strategy_component.get_sport_key_from_cache_key(key)` |
|
||||
| `CacheManager.record_cache_hit` | core | src/cache_manager.py:869 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_hit('background')` |
|
||||
| `CacheManager.record_cache_miss` | core | src/cache_manager.py:876 | internal (in `CacheManager.get_background_cached_data`) | `self.record_cache_miss('background')` |
|
||||
| `CacheManager.record_fetch_time` | core | src/cache/cache_metrics.py:67 | unrelated | `def record_fetch_time(self, duration: float) -> None:` |
|
||||
| `CacheManager.record_fetch_time` | core | src/cache_manager.py:922 | unrelated | `self._metrics_component.record_fetch_time(duration)` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:43 | test call | `stats = cm.get_memory_cache_stats()` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:63 | test call | `assert cm.get_memory_cache_stats()["last_cleanup"] >= before` |
|
||||
| `CacheManager.get_memory_cache_stats` | core tests | test/test_cache_manager_memory_tier.py:68 | test call | `stats = cm.get_memory_cache_stats()` |
|
||||
| `DisplayManager.draw_sun` | core | src/plugin_system/testing/visual_display_manager.py:417 | unrelated | `def draw_sun(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1356 | internal (in `DisplayManager.draw_rain`) | `self.draw_cloud(x, y, size)` |
|
||||
| `DisplayManager.draw_cloud` | core | src/display_manager.py:1371 | internal (in `DisplayManager.draw_snow`) | `self.draw_cloud(x, y, size)` |
|
||||
| `DisplayManager.draw_cloud` | core | src/plugin_system/testing/visual_display_manager.py:421 | unrelated | `def draw_cloud(self, x: int, y: int, size: int = 16, color: Tuple[int, int, int] = (200, 200, 200)):` |
|
||||
| `DisplayManager.draw_cloud` | ledmatrix-weather | weather_icons.py:184 | unrelated | `def draw_cloud(draw: ImageDraw, x: int, y: int, size: int = 16, color: tuple = (200, 200, 200)):` |
|
||||
| `DisplayManager.draw_rain` | core | src/plugin_system/testing/visual_display_manager.py:425 | unrelated | `def draw_rain(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_snow` | core | src/plugin_system/testing/visual_display_manager.py:429 | unrelated | `def draw_snow(self, x: int, y: int, size: int = 16):` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/display_manager.py:1515 | internal (in `DisplayManager.draw_text_with_icons`) | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:510 | unrelated | `def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:` |
|
||||
| `DisplayManager.draw_weather_icon` | core | src/plugin_system/testing/visual_display_manager.py:533 | unrelated | `self.draw_weather_icon(icon_type, icon_x, icon_y)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:76 | unrelated | `def draw_weather_icon(image, icon_code, x, y, size):` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1265 | unrelated | `WeatherIcons.draw_weather_icon(img, icon_code, icon_x, icon_y,` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1544 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | manager.py:1635 | unrelated | `WeatherIcons.draw_weather_icon(img, forecast['icon'], icon_x, icon_y, icon_size)` |
|
||||
| `DisplayManager.draw_weather_icon` | ledmatrix-weather | weather_icons.py:168 | unrelated | `def draw_weather_icon(image: Image.Image, icon_code: str, x: int, y: int, size: int = DEFAULT_SIZE):` |
|
||||
| `DisplayManager.draw_text_with_icons` | core | src/plugin_system/testing/visual_display_manager.py:526 | unrelated | `def draw_text_with_icons(self, text: str, icons: List[tuple] = None,` |
|
||||
| `FontManager.get_font_catalog` | core tests | test/test_deprecation.py:229 | test call | `assert fm.get_font_catalog() == fm.font_catalog` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:356 | test review | `assert plugin.get_supported_vegas_modes() == [` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | core tests | test/test_vegas_participation.py:358 | test review | `assert plugin.get_supported_vegas_modes()` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:732 | override | `def get_supported_vegas_modes(self):` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | blackjack | manager.py:695 | call | `if mode in self.get_supported_vegas_modes():` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | calendar | manager.py:875 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
|
||||
| `BasePlugin.get_supported_vegas_modes` | olympics | manager.py:624 | override | `def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:` |
|
||||
| `BasePlugin.get_vegas_segment_width` | core tests | test/test_vegas_participation.py:359 | test review | `assert plugin.get_vegas_segment_width() == 2` |
|
||||
|
||||
## Sources scanned
|
||||
|
||||
| Source | Group | Python files | Hits |
|
||||
|---|---|---|---|
|
||||
| core | core | 164 | 20 |
|
||||
| core tests | core-tests | 323 | 17 |
|
||||
| 7-segment-clock | monorepo | 3 | 0 |
|
||||
| afl-scoreboard | monorepo | 34 | 0 |
|
||||
| baseball-scoreboard | monorepo | 60 | 0 |
|
||||
| basketball-scoreboard | monorepo | 48 | 0 |
|
||||
| birdnet-go | monorepo | 2 | 0 |
|
||||
| blackjack | monorepo | 7 | 2 |
|
||||
| calendar | monorepo | 5 | 1 |
|
||||
| christmas-countdown | monorepo | 3 | 0 |
|
||||
| clock-simple | monorepo | 2 | 0 |
|
||||
| countdown | monorepo | 5 | 0 |
|
||||
| cricket-scoreboard | monorepo | 8 | 0 |
|
||||
| f1-scoreboard | monorepo | 15 | 0 |
|
||||
| fantasy-blitz | monorepo | 13 | 0 |
|
||||
| football-scoreboard | monorepo | 73 | 0 |
|
||||
| geochron | monorepo | 10 | 0 |
|
||||
| hello-world | monorepo | 2 | 0 |
|
||||
| hockey-scoreboard | monorepo | 51 | 0 |
|
||||
| incoming-packages | monorepo | 8 | 0 |
|
||||
| jellyfin-now-playing | monorepo | 4 | 0 |
|
||||
| lacrosse-scoreboard | monorepo | 39 | 0 |
|
||||
| ledmatrix-elections | monorepo | 12 | 0 |
|
||||
| ledmatrix-flights | monorepo | 45 | 0 |
|
||||
| ledmatrix-leaderboard | monorepo | 9 | 0 |
|
||||
| ledmatrix-music | monorepo | 11 | 0 |
|
||||
| ledmatrix-stocks | monorepo | 7 | 0 |
|
||||
| ledmatrix-weather | monorepo | 14 | 6 |
|
||||
| march-madness | monorepo | 4 | 0 |
|
||||
| masters-tournament | monorepo | 10 | 0 |
|
||||
| mqtt-notifications | monorepo | 4 | 0 |
|
||||
| news | monorepo | 6 | 0 |
|
||||
| nfl-draft | monorepo | 3 | 0 |
|
||||
| nfl-stat-leaders | monorepo | 8 | 0 |
|
||||
| nrl-scoreboard | monorepo | 29 | 0 |
|
||||
| odds-ticker | monorepo | 9 | 0 |
|
||||
| of-the-day | monorepo | 14 | 0 |
|
||||
| olympics | monorepo | 16 | 1 |
|
||||
| on-air | monorepo | 2 | 0 |
|
||||
| pomodoro-timer | monorepo | 3 | 0 |
|
||||
| soccer-scoreboard | monorepo | 46 | 0 |
|
||||
| static-image | monorepo | 3 | 0 |
|
||||
| stock-news | monorepo | 3 | 0 |
|
||||
| text-display | monorepo | 4 | 0 |
|
||||
| tide-display | monorepo | 3 | 0 |
|
||||
| ufc-scoreboard | monorepo | 34 | 0 |
|
||||
| web-ui-info | monorepo | 2 | 0 |
|
||||
| youtube-stats | monorepo | 5 | 0 |
|
||||
| f1-live | third-party | 10 | 0 |
|
||||
| gif-player | third-party | 1 | 0 |
|
||||
| pga-tour-leaderboard | third-party | 2 | 0 |
|
||||
| plex-marquee | third-party | 1 | 0 |
|
||||
| ledmatrix-dresden-departures | third-party | 1 | 0 |
|
||||
| tidbyt-baseball-scoreboard | third-party | 2 | 0 |
|
||||
| sleeper-fantasy | third-party | 1 | 0 |
|
||||
| ledmatrix-nascar | third-party | 1 | 0 |
|
||||
|
||||
## How to re-run
|
||||
|
||||
```bash
|
||||
# Clones the monorepo and each third-party plugin (depth 1) into a temp cache:
|
||||
python3 scripts/plugin_api_usage.py --output docs/DEPRECATIONS_3.8.md
|
||||
# Or scan a local monorepo checkout (read only) instead of cloning it:
|
||||
python3 scripts/plugin_api_usage.py --monorepo ../ledmatrix-plugins
|
||||
```
|
||||
|
||||
Before removing a method in its release, re-run the scan against the current monorepo and registry: a plugin added since this file was generated may have started calling it. Remove only methods the fresh scan reports unused; move the rest to a later release (the test in `test/test_deprecation.py` fails while a marker names a release at or below `src.__version__`).
|
||||
@@ -54,7 +54,7 @@ rows = self.layout.bounds.inset(1).split_v(3, 1, gap=1)
|
||||
self.draw_fit("12:34", rows[0]) # largest crisp font that fits
|
||||
self.draw_image(logo, rows[1], mode="fill_height", crop_to_ink=True)
|
||||
|
||||
# Weather icons: draw_weather_icon() is deprecated, removed in 3.7.0 —
|
||||
# Weather icons: draw_weather_icon() is deprecated, removed in 3.8.0 —
|
||||
# draw your own icons (the weather plugin ships WeatherIcons)
|
||||
|
||||
# Scrolling state
|
||||
@@ -78,7 +78,7 @@ strategy = cache_manager.get_cache_strategy("weather")
|
||||
```
|
||||
|
||||
`get_background_cached_data()` (use `get()`) and `get_sport_live_interval()`
|
||||
are deprecated, removed in 3.7.0. See
|
||||
are deprecated, removed in 3.8.0. See
|
||||
[Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis).
|
||||
|
||||
## Plugin Manager Quick Methods
|
||||
@@ -87,7 +87,7 @@ are deprecated, removed in 3.7.0. See
|
||||
# Get plugins
|
||||
plugin = plugin_manager.get_plugin("plugin-id")
|
||||
all_plugins = plugin_manager.get_all_plugins()
|
||||
# get_enabled_plugins() is deprecated, removed in 3.7.0 — check `enabled`
|
||||
# get_enabled_plugins() is deprecated, removed in 3.8.0 — check `enabled`
|
||||
# on the entries in plugin_manager.plugins
|
||||
|
||||
# Get info
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
BDF fonts in `assets/fonts/`, registers fonts that plugins ship, and records
|
||||
which plugin uses which font so the web UI can show it.
|
||||
|
||||
Several methods are deprecated and will be removed in LEDMatrix 3.7.0; they
|
||||
Several methods are deprecated and will be removed in LEDMatrix 3.8.0; they
|
||||
log a warning on first call. They are listed in
|
||||
[Deprecated methods](#deprecated-methods) below, and the full set is pinned in
|
||||
[`test/test_deprecation.py`](../test/test_deprecation.py).
|
||||
@@ -209,7 +209,7 @@ Current methods:
|
||||
|
||||
### Deprecated methods
|
||||
|
||||
Removed in 3.7.0. Each logs a warning on first call.
|
||||
Removed in 3.8.0. Each logs a warning on first call.
|
||||
|
||||
| Method | Use instead |
|
||||
|---|---|
|
||||
|
||||
@@ -240,6 +240,22 @@ The fastest way to verify a plugin works without waiting for the rotation:
|
||||
- Install community plugins straight from a GitHub URL via
|
||||
**Install from GitHub** on the same tab.
|
||||
|
||||
### Keep LEDMatrix Up to Date
|
||||
|
||||
- **Update Code** on the **Overview** tab installs the newest version, and a
|
||||
banner at the top of the page says when one is available.
|
||||
- **General → Automatic Updates** does it once a week, overnight, with a
|
||||
health check that undoes an update that breaks the device.
|
||||
- **General → Update Channel** picks which version that is. **Stable** (the
|
||||
default) installs releases, which have been tested and have release
|
||||
notes. **Beta** installs the newest code as soon as it is written, before
|
||||
it is released: fixes arrive sooner, and so do new problems.
|
||||
- Switching to Stable never installs an older version than the one you
|
||||
have. If your device is already newer than the latest release (which is
|
||||
normal if it was set up or updated from the newest code), it keeps
|
||||
getting the newest code until the next release includes it, then follows
|
||||
releases from there. The General tab says when this is the case.
|
||||
|
||||
### Enable Advanced Features
|
||||
|
||||
**Vegas Scroll Mode:**
|
||||
|
||||
+123
-28
@@ -149,15 +149,27 @@ Clean up resources when plugin is unloaded. Override to close connections, stop
|
||||
|
||||
#### `on_config_change(new_config: Dict[str, Any]) -> None`
|
||||
|
||||
Called after plugin configuration is updated via web API.
|
||||
Called after the plugin's section of `config.json` changes -- a save in the
|
||||
web UI, say. Every lifecycle hook runs in the display process, which is the
|
||||
only process that runs plugins: the web interface writes `config.json`, and
|
||||
the display's config watcher calls this with the prepared section. See
|
||||
[ARCHITECTURE.md](ARCHITECTURE.md#web-and-display-processes-who-runs-plugins).
|
||||
|
||||
In the display service it runs on the config watcher thread while holding
|
||||
the plugin's lock, so it never overlaps your `update()` or `display()`. If
|
||||
the plugin stays busy for more than 5 seconds, the change is applied later
|
||||
from the update thread: as soon as the plugin is free, and before its next
|
||||
`update()` at the latest.
|
||||
|
||||
#### `on_enable() -> None`
|
||||
|
||||
Called when plugin is enabled.
|
||||
Called when the display loads the plugin enabled: at startup, or when it is
|
||||
switched on in the web UI.
|
||||
|
||||
#### `on_disable() -> None`
|
||||
|
||||
Called when plugin is disabled.
|
||||
Called when the display unloads the plugin, e.g. when it is switched off in
|
||||
the web UI.
|
||||
|
||||
#### `get_update_interval() -> Optional[float]`
|
||||
|
||||
@@ -308,6 +320,58 @@ rotating one at a time. Plugins control how their content appears via
|
||||
these hooks. See [ADVANCED_FEATURES.md](ADVANCED_FEATURES.md) for the user
|
||||
side of Vegas mode.
|
||||
|
||||
#### Vegas participation
|
||||
|
||||
Each plugin takes part in Vegas mode in one of three ways:
|
||||
|
||||
| Participation | What Vegas does |
|
||||
|---|---|
|
||||
| `'scroll'` | The plugin's content (`get_vegas_content()`) scrolls by with everything else |
|
||||
| `'pause'` | The scroll stops when the plugin's turn comes round; its `display()` draws it full screen for `get_display_duration()` seconds, then the scroll resumes |
|
||||
| `'exclude'` | The plugin is left out of Vegas mode |
|
||||
|
||||
Declare the plugin's default in `manifest.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "my-alerts",
|
||||
"vegas_participation": "pause"
|
||||
}
|
||||
```
|
||||
|
||||
The user can override it per plugin with `vegas_participation` in that
|
||||
plugin's config section (it is one of the core-owned properties, see
|
||||
[PLUGIN_CONFIG_CORE_PROPERTIES.md](PLUGIN_CONFIG_CORE_PROPERTIES.md)).
|
||||
Vegas resolves it in this order:
|
||||
|
||||
1. the user's `vegas_participation` config value;
|
||||
2. the plugin's `get_vegas_participation()` — the default implementation
|
||||
reads the manifest's `vegas_participation`, then derives a value from
|
||||
the legacy hooks below;
|
||||
3. derived from the legacy hooks: `get_vegas_display_mode()` returning
|
||||
`VegasDisplayMode.STATIC` → `'pause'`; otherwise
|
||||
`get_vegas_content_type()` returning `'none'` → `'exclude'`; everything
|
||||
else → `'scroll'`.
|
||||
|
||||
Step 3 is exactly what Vegas did before participation existed, so a plugin
|
||||
that declares nothing behaves as it always has. Manifest
|
||||
`vegas_participation` is new in core 3.8.0; older cores ignore it and use
|
||||
the legacy hooks.
|
||||
|
||||
#### `get_vegas_participation() -> str`
|
||||
|
||||
Returns `'scroll'`, `'pause'` or `'exclude'`. Override it only when the
|
||||
answer depends on state — pause only while an alert is live, exclude while
|
||||
there is nothing to show; for a fixed answer use the manifest. Vegas applies
|
||||
the user's config value before calling an override, so an override does not
|
||||
need to check it. A value that is not one of the three is ignored with a log
|
||||
line and the legacy hooks decide.
|
||||
|
||||
```python
|
||||
def get_vegas_participation(self):
|
||||
return 'pause' if self._alert_is_live() else 'scroll'
|
||||
```
|
||||
|
||||
#### `get_vegas_content() -> Optional[PIL.Image | List[PIL.Image] | None]`
|
||||
|
||||
Return content to inject into the scroll. Multi-item plugins (sports,
|
||||
@@ -315,26 +379,40 @@ odds, news) should return a *list* of PIL Images so each item scrolls
|
||||
independently. Static plugins (clock, weather) can return a single image.
|
||||
Returning `None` falls back to capturing whatever `display()` produces.
|
||||
|
||||
#### `get_vegas_content_type() -> str`
|
||||
#### `get_vegas_render_width() -> int`
|
||||
|
||||
`'multi'`, `'static'`, or `'none'`. Affects how Vegas mode treats the
|
||||
plugin. Default `'static'`.
|
||||
The width Vegas wants this plugin's content to occupy, from the plugin's
|
||||
`vegas_width_pct` config value or the global
|
||||
`display.vegas_scroll.render_width_pct`. Vegas also narrows
|
||||
`display_manager` while it asks for content, so a plugin that sizes itself
|
||||
from `display_manager.width` does not need to read this.
|
||||
|
||||
#### `get_vegas_display_mode() -> VegasDisplayMode`
|
||||
#### Legacy: `get_vegas_content_type()` and `get_vegas_display_mode()`
|
||||
|
||||
Returns one of `VegasDisplayMode.SCROLL`, `FIXED_SEGMENT`, or `STATIC`.
|
||||
Read from `config["vegas_mode"]` or override directly.
|
||||
Superseded by participation, and still read to derive it when neither the
|
||||
user nor the manifest declares one (step 3 above). Only two answers ever
|
||||
mattered: `get_vegas_content_type()` returning `'none'`, and
|
||||
`get_vegas_display_mode()` returning `VegasDisplayMode.STATIC`.
|
||||
|
||||
#### `get_supported_vegas_modes() -> List[VegasDisplayMode]`
|
||||
- `get_vegas_content_type()` returns `'multi'`, `'static'` or `'none'`
|
||||
(default `'static'`).
|
||||
- `get_vegas_display_mode()` returns a `VegasDisplayMode` member (not a
|
||||
string — the string `'static'` never paused anything). The default reads
|
||||
the plugin's `vegas_mode` config value (`"scroll"`, `"fixed"` or
|
||||
`"static"`), else maps content type `'multi'` to `SCROLL` and anything
|
||||
else to `FIXED_SEGMENT`.
|
||||
|
||||
The set of Vegas modes this plugin can render. Used by the UI to populate
|
||||
the mode selector for this plugin.
|
||||
`SCROLL` and `FIXED_SEGMENT` (and `vegas_mode` `"scroll"` and `"fixed"`)
|
||||
have always behaved identically: both scroll. The distinction is deprecated
|
||||
and goes away in 3.9.0 — see [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
#### `get_vegas_segment_width() -> Optional[int]`
|
||||
#### Deprecated: `get_supported_vegas_modes()` and `get_vegas_segment_width()`
|
||||
|
||||
For `FIXED_SEGMENT` plugins, the number of *panels* the segment
|
||||
occupies in the scroll (pixel width = panels × `single_panel_width`,
|
||||
from `display.hardware.cols`). `None` uses the default of 1 panel.
|
||||
Never read by core, and removed in 3.9.0: calling the `BasePlugin`
|
||||
implementation logs a deprecation warning. A plugin's own override keeps
|
||||
working for the plugin itself. `get_vegas_segment_width()` read the
|
||||
`vegas_panel_count` config value, which has never affected Vegas — a card's
|
||||
width comes from `get_vegas_content()` and `vegas_width_pct`.
|
||||
|
||||
> The full source for `BasePlugin` lives in
|
||||
> `src/plugin_system/base_plugin.py`. If a method here disagrees with the
|
||||
@@ -479,7 +557,7 @@ This is the canonical way to render arbitrary images.
|
||||
|
||||
### Weather Icons (deprecated)
|
||||
|
||||
> Deprecated, removed in 3.7.0 — draw your own icons (the weather plugin
|
||||
> Deprecated, removed in 3.8.0 — draw your own icons (the weather plugin
|
||||
> ships `WeatherIcons`). See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
- `draw_weather_icon(condition, x, y, size=16)` — icon for a condition
|
||||
@@ -581,7 +659,7 @@ Process any deferred updates if not currently scrolling. Called automatically by
|
||||
|
||||
#### `get_scrolling_stats() -> dict`
|
||||
|
||||
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
|
||||
> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
Get current scrolling statistics for debugging.
|
||||
|
||||
@@ -724,7 +802,7 @@ data = self.cache_manager.get_with_auto_strategy("nhl_live_scores")
|
||||
|
||||
#### `get_background_cached_data(key: str, sport_key: Optional[str] = None) -> Optional[Dict[str, Any]]`
|
||||
|
||||
> Deprecated, removed in 3.7.0 — use `get()`. See [Deprecated APIs](#deprecated-apis).
|
||||
> Deprecated, removed in 3.8.0 — use `get()`. See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
Get background service cached data with sport-specific intervals.
|
||||
|
||||
@@ -763,7 +841,7 @@ max_age = strategy['max_age'] # Get configured max age
|
||||
|
||||
#### `get_sport_live_interval(sport_key: str) -> int`
|
||||
|
||||
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
|
||||
> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
Get the live_update_interval for a specific sport from config.
|
||||
|
||||
@@ -789,7 +867,7 @@ Extract data type from cache key to determine appropriate cache strategy.
|
||||
|
||||
#### `get_sport_key_from_cache_key(key: str) -> Optional[str]`
|
||||
|
||||
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
|
||||
> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
Extract sport key from cache key for sport-specific strategies.
|
||||
|
||||
@@ -839,7 +917,7 @@ for file_info in files:
|
||||
|
||||
#### `get_cache_metrics() -> Dict[str, Any]`
|
||||
|
||||
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
|
||||
> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
Get cache performance metrics.
|
||||
|
||||
@@ -853,7 +931,7 @@ self.logger.info(f"Cache hit rate: {metrics['cache_hit_rate']:.2%}")
|
||||
|
||||
#### `get_memory_cache_stats() -> Dict[str, Any]`
|
||||
|
||||
> Deprecated, removed in 3.7.0. See [Deprecated APIs](#deprecated-apis).
|
||||
> Deprecated, removed in 3.8.0. See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
Get memory cache statistics.
|
||||
|
||||
@@ -899,7 +977,7 @@ for plugin_id, plugin in all_plugins.items():
|
||||
|
||||
#### `get_enabled_plugins() -> List[str]`
|
||||
|
||||
> Deprecated, removed in 3.7.0 — check `enabled` on the instances in `plugin_manager.plugins`. See [Deprecated APIs](#deprecated-apis).
|
||||
> Deprecated, removed in 3.8.0 — check `enabled` on the instances in `plugin_manager.plugins`. See [Deprecated APIs](#deprecated-apis).
|
||||
|
||||
Get list of enabled plugin IDs.
|
||||
|
||||
@@ -1070,10 +1148,13 @@ if weather is not None and weather.enabled:
|
||||
|
||||
## Deprecated APIs
|
||||
|
||||
These still work in 3.6 but log a warning the first time they are called
|
||||
(`journalctl -u ledmatrix` shows which one), and are **removed in 3.7.0**.
|
||||
Nothing in core, the official plugins or the third-party plugins in the
|
||||
registry calls them.
|
||||
These still work but log a warning the first time they are called
|
||||
(`journalctl -u ledmatrix` shows which one), and are **removed in 3.8.0**
|
||||
(first announced for 3.7.0, which shipped with them still in place).
|
||||
[DEPRECATIONS_3.8.md](DEPRECATIONS_3.8.md) is the usage scan behind that
|
||||
decision: which of these the official plugins, the registry's third-party
|
||||
plugins and core still call or override. Only methods that scan reports unused
|
||||
are removed in 3.8.0; the rest stay until their callers migrate.
|
||||
|
||||
| Object | Methods | Instead |
|
||||
|---|---|---|
|
||||
@@ -1086,3 +1167,17 @@ registry calls them.
|
||||
| `font_manager` | `set_override`, `remove_override`, `get_overrides`, `add_font`, `remove_font`, `validate_font`, `get_size_tokens`, `get_performance_stats`, `get_manager_fonts`, `get_detected_fonts`, `get_plugin_fonts`, `unregister_plugin_fonts` | no replacement |
|
||||
| `plugin_manager` | `get_enabled_plugins` | check `enabled` on the entries in `plugin_manager.plugins` |
|
||||
|
||||
### Removed in 3.9.0
|
||||
|
||||
The Vegas APIs that described a fixed-width segment, which Vegas never
|
||||
implemented. Vegas participation (`'scroll'`, `'pause'`, `'exclude'`, see
|
||||
[Vegas scroll hooks](#vegas-scroll-hooks)) replaces them. Calling one of the
|
||||
methods, or setting `vegas_panel_count`, logs a warning once per process.
|
||||
No official plugin calls them; calendar, olympics and blackjack override
|
||||
`get_supported_vegas_modes()`, which keeps working for the plugin itself.
|
||||
|
||||
| What | Instead |
|
||||
|---|---|
|
||||
| `BasePlugin.get_supported_vegas_modes()` | declare `vegas_participation` in the manifest |
|
||||
| `BasePlugin.get_vegas_segment_width()` and the `vegas_panel_count` config key | nothing: a card's width comes from `get_vegas_content()` and `vegas_width_pct` |
|
||||
| `VegasDisplayMode.SCROLL` vs `FIXED_SEGMENT` (`vegas_mode` `"scroll"` vs `"fixed"`) | `'scroll'` participation; the two always behaved the same |
|
||||
|
||||
@@ -33,6 +33,20 @@ is `CORE_PLUGIN_PROPERTIES` in `src/plugin_system/schema_manager.py`):
|
||||
- Read by `src/vegas_mode/plugin_adapter.py` and `BasePlugin`, which
|
||||
validate the values themselves and ignore a bad one with a log line
|
||||
|
||||
5. **`vegas_participation`** (string enum: `"scroll"`, `"pause"`,
|
||||
`"exclude"`; no default)
|
||||
- Description: how this plugin takes part in Vegas mode — its content
|
||||
scrolls by, the scroll pauses for its turn and shows it full screen, or
|
||||
it is left out
|
||||
- Overrides the plugin's own default (its manifest's
|
||||
`vegas_participation`, else what its legacy Vegas hooks say); unset
|
||||
means "use the plugin's default"
|
||||
- Deliberately has no default: one would be written into every plugin's
|
||||
config and override what each plugin declares
|
||||
- Read by `resolve_vegas_participation()` in
|
||||
`src/plugin_system/base_plugin.py`; see
|
||||
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#vegas-participation)
|
||||
|
||||
`skin` and `skin_options` were core properties until the skin system was
|
||||
removed. A plugin config saved with them still loads and saves; the keys are
|
||||
dropped on the next save (see `RETIRED_PLUGIN_KEYS` in `schema_manager.py`).
|
||||
|
||||
@@ -520,14 +520,14 @@ When developing plugins, you'll need to use the APIs provided by the LEDMatrix s
|
||||
`display_manager.image` (a PIL Image) and call `update_display()`;
|
||||
there is no `draw_image()` helper method.
|
||||
- `draw_weather_icon()`, `draw_sun()`, `draw_cloud()` - Weather icons
|
||||
(deprecated, removed in 3.7.0 — draw your own icons)
|
||||
(deprecated, removed in 3.8.0 — draw your own icons)
|
||||
- `get_text_width()`, `get_font_height()` - Text utilities
|
||||
- `set_scrolling_state()`, `defer_update()` - Scrolling state management
|
||||
|
||||
**Cache Manager** (`self.cache_manager`):
|
||||
- `get()`, `set()`, `delete()` - Basic caching
|
||||
- `get_cached_data_with_strategy()` - Advanced caching with strategies
|
||||
- `get_background_cached_data()` - deprecated, removed in 3.7.0 — use `get()`
|
||||
- `get_background_cached_data()` - deprecated, removed in 3.8.0 — use `get()`
|
||||
|
||||
**Plugin Manager** (`self.plugin_manager`):
|
||||
- `get_plugin()`, `get_all_plugins()` - Access other plugins
|
||||
@@ -535,7 +535,7 @@ When developing plugins, you'll need to use the APIs provided by the LEDMatrix s
|
||||
|
||||
See [PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md) for complete
|
||||
documentation, and its [Deprecated APIs](PLUGIN_API_REFERENCE.md#deprecated-apis)
|
||||
table for everything removed in 3.7.0.
|
||||
table for everything removed in 3.8.0.
|
||||
|
||||
## 3rd Party Plugin Development
|
||||
|
||||
|
||||
+314
-29
@@ -18,6 +18,39 @@ top level instead of under `data` (install-from-url, registry-from-url, the
|
||||
auth endpoints, upload endpoints, `system/git-info`, `system/check-update`),
|
||||
the entry below says so.
|
||||
|
||||
**Cross-site requests are refused.** A `POST`, `PUT`, `PATCH` or `DELETE`
|
||||
carrying an `Origin` header (or, without one, a `Referer`) that is not the
|
||||
host the request was sent to gets `403` with `"error_code":
|
||||
"CROSS_SITE_REQUEST"`; so does `Origin: null`. This stops other websites from
|
||||
driving the Pi through a LAN user's browser. Scripts, curl, Home Assistant and
|
||||
the MQTT bridge send neither header and are unaffected. A browser page on
|
||||
another origin (a dashboard you host elsewhere, say) can no longer call the
|
||||
API; call it server-side instead. Behind a reverse proxy, pass the original
|
||||
`Host` through, port included (nginx: `proxy_set_header Host $http_host;`;
|
||||
`$host` drops the port) -- `X-Forwarded-Host` is not read.
|
||||
|
||||
**Authentication (optional, off by default).** With no web password set,
|
||||
nothing below needs credentials. Once one is set (General > Security, or
|
||||
[`POST /auth/password`](#web-login-and-api-tokens)), every route needs a login
|
||||
session or an API token:
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer lmx_..." http://your-pi-ip:5000/api/v3/display/current
|
||||
```
|
||||
|
||||
Without either, an API route answers `401` with
|
||||
`{"status": "error", "error_code": "AUTH_REQUIRED", "message": ...}`, a
|
||||
`WWW-Authenticate: Bearer realm="LEDMatrix"` header and the login page's URL
|
||||
in `X-LEDMatrix-Login`; an unknown or revoked token gets `"error_code":
|
||||
"INVALID_TOKEN"`. Browser page loads are redirected to `/login` instead, and
|
||||
HTMX requests get `401` with `HX-Redirect: /login?...`. Never asked for
|
||||
credentials: requests from the Pi itself (loopback, with no `X-Forwarded-For`,
|
||||
`X-Real-IP`, `Forwarded` or `X-Forwarded-Host` header), `/static/*`, the
|
||||
captive-portal probe URLs, `/login`, `/api/v3/health` (status only, see
|
||||
[Health Check](#health-check)), and -- only while the Pi is in access-point
|
||||
mode -- `/setup`, `GET /wifi/status`, `GET /wifi/scan` and
|
||||
`POST /wifi/connect`.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Configuration](#configuration)
|
||||
@@ -35,6 +68,7 @@ the entry below says so.
|
||||
- [Health and Status](#health-and-status)
|
||||
- [Schedule (dim/power)](#schedule-dimpower)
|
||||
- [Integrations](#integrations)
|
||||
- [Web login and API tokens](#web-login-and-api-tokens)
|
||||
- [Plugin-specific endpoints](#plugin-specific-endpoints)
|
||||
- [Starlark Apps](#starlark-apps)
|
||||
|
||||
@@ -120,10 +154,17 @@ there an unchecked checkbox — which the browser omits — is saved as
|
||||
```json
|
||||
{
|
||||
"status": "success",
|
||||
"message": "Configuration saved successfully"
|
||||
"message": "Configuration saved successfully",
|
||||
"restart_required": true
|
||||
}
|
||||
```
|
||||
|
||||
`restart_required` is always true here: display hardware, rotation,
|
||||
durations and general settings take effect when the display restarts, and
|
||||
the web UI shows its restart banner on the flag. (Plugin sections saved
|
||||
through this route reach the running plugin live, like
|
||||
`POST /plugins/config`.)
|
||||
|
||||
Invalid values (e.g. an out-of-range `target_fps`, a hardware option the
|
||||
Raspberry Pi 5 driver cannot use) are rejected with `400` and nothing is
|
||||
saved.
|
||||
@@ -213,7 +254,10 @@ times `07:00`-`23:00`. At least one day must be enabled.
|
||||
|
||||
Retrieve `config/config_secrets.json` with every set value replaced by eight
|
||||
bullet characters (`"••••••••"`). Empty values and `YOUR_*` placeholders
|
||||
are returned as-is, so a client can tell "set" from "not set".
|
||||
are returned as-is, so a client can tell "set" from "not set". The
|
||||
`web_auth` section (the web login's password hash, API-token hashes and
|
||||
cookie key) is left out entirely; [Get Main Configuration](#get-main-configuration)
|
||||
leaves it out too.
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
@@ -238,7 +282,8 @@ Replace `config/config.json` with the JSON body (advanced use only).
|
||||
Save the secrets file (advanced use only). Masked values (`"••••••••"`)
|
||||
and blank strings in the body are dropped, and the rest is merged onto the
|
||||
stored secrets, so posting back the GET response unchanged changes nothing.
|
||||
A secret cannot be cleared by blanking it here.
|
||||
A secret cannot be cleared by blanking it here. A `web_auth` key in the body
|
||||
is ignored; the stored login settings are kept.
|
||||
|
||||
---
|
||||
|
||||
@@ -390,7 +435,7 @@ Request a specific plugin to display on-demand.
|
||||
- `mode` (string, optional): Display mode name (plugin_id inferred if not provided)
|
||||
- `duration` (number, optional): Duration in seconds (0 = until stopped)
|
||||
- `pinned` (boolean, optional): Pin display (pause rotation)
|
||||
- `start_service` (boolean, optional): (Re)start the display service so it picks the request up (default: true)
|
||||
- `start_service` (boolean, optional): Start the display service if it is not running (default: true). A running service is never restarted: it picks the request up within about a quarter of a second. When false and the service is stopped, the route returns 400.
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
@@ -465,21 +510,65 @@ List all installed plugins with their status and metadata.
|
||||
"enabled": true,
|
||||
"verified": true,
|
||||
"loaded": true,
|
||||
"state": "loaded",
|
||||
"state": "enabled",
|
||||
"error_info": null,
|
||||
"loaded_version": "1.2.3",
|
||||
"loaded_at": 1790000000.0,
|
||||
"last_updated": "2025-01-15T10:30:00Z",
|
||||
"last_commit": "abc1234",
|
||||
"last_commit_message": "feat: Add live game updates",
|
||||
"branch": "main",
|
||||
"web_ui_actions": [],
|
||||
"vegas_mode": null,
|
||||
"vegas_content_type": null
|
||||
"vegas_content_type": null,
|
||||
"vegas_participation": "scroll",
|
||||
"vegas_participation_source": "manifest"
|
||||
}
|
||||
]
|
||||
],
|
||||
"runtime": {
|
||||
"status": "live",
|
||||
"published_at": 1790000030.0,
|
||||
"age_seconds": 12.4,
|
||||
"stale_after": 180.0
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Metadata comes from each plugin's files on disk; `enabled` is the plugin's
|
||||
`enabled` flag in `config.json` (missing means disabled, as the display
|
||||
reads it). `vegas_mode` is the plugin's configured `vegas_mode`, or `null`.
|
||||
|
||||
`loaded`, `state`, `error_info`, `loaded_version` and `loaded_at` come from
|
||||
the runtime snapshot the display publishes (the web process runs no plugin
|
||||
code). `state` is the display's lifecycle state (`loaded` while loading,
|
||||
`enabled`, `disabled`, `error`, `unloaded`); `error_info` is `null` or
|
||||
`{"type", "message", "at", "recoverable"}`, with the message redacted and at
|
||||
most 200 characters (the full error is at `/errors/*`). `loaded_version` is
|
||||
the version the display loaded, which differs from `version` after an update
|
||||
until the display restarts. A plugin a live snapshot does not list is
|
||||
`loaded: false`, `state: "unloaded"`.
|
||||
|
||||
`runtime.status` says whether to believe them: `live` (fresh snapshot from
|
||||
a running display), `stale` (not refreshed within `stale_after` seconds: the
|
||||
display is hung or died), `stopped` (the display shut down) or `unknown`
|
||||
(nothing published yet). Unless it is `live`, every one of those fields is
|
||||
`null`. Health and metrics are at [`/plugins/health`](#get-plugin-health)
|
||||
and `/plugins/metrics`.
|
||||
|
||||
`vegas_participation` is what Vegas mode does with the plugin: `"scroll"`,
|
||||
`"pause"` or `"exclude"` (see
|
||||
[PLUGIN_API_REFERENCE.md](PLUGIN_API_REFERENCE.md#vegas-participation)),
|
||||
and `vegas_participation_source` says where it came from. The web reads it
|
||||
the way the display resolves it, as far as files can tell: the user's own
|
||||
`vegas_participation` setting (`"config"`), else the manifest's declared
|
||||
`vegas_participation` (`"manifest"`). Past those the display derives it from
|
||||
the plugin's code -- a `get_vegas_participation()` override or the legacy
|
||||
Vegas hooks -- which the web process never runs, so `vegas_participation`
|
||||
is `null` and the source is `"runtime"`. A plugin that overrides
|
||||
`get_vegas_participation()` decides at run time and can differ from its
|
||||
manifest's declaration. `vegas_content_type` is always `null`.
|
||||
|
||||
### Get Plugin Configuration
|
||||
|
||||
**GET** `/api/v3/plugins/config?plugin_id=<plugin_id>`
|
||||
@@ -639,7 +728,19 @@ Install a plugin from the plugin store.
|
||||
```
|
||||
|
||||
When the operation queue is unavailable the install runs synchronously and
|
||||
the response has only a `message`.
|
||||
the response has only a `message` and the restart fields below.
|
||||
|
||||
The finished operation's `result` (from `/plugins/operation/<operation_id>`)
|
||||
carries `restart_required`: true when the plugin is already enabled in
|
||||
`config.json`, because the running display does not load newly installed
|
||||
files by itself; `restart_message` then holds the restart banner's wording.
|
||||
A plugin that is not enabled needs no restart: enabling it loads it.
|
||||
|
||||
A plugin whose registry entry (or downloaded manifest) needs a newer
|
||||
LEDMatrix is refused: the synchronous install answers `409` with a message
|
||||
such as `Failed to install plugin x: X requires LEDMatrix 3.8.0 or newer…`,
|
||||
and a queued one fails with that message. Nothing already installed is
|
||||
changed.
|
||||
|
||||
### Uninstall Plugin
|
||||
|
||||
@@ -664,6 +765,11 @@ Remove an installed plugin.
|
||||
}
|
||||
```
|
||||
|
||||
The finished operation's `result` carries `restart_required`. Removing the
|
||||
plugin's config (the default) lets the display unload it by itself, so it is
|
||||
false; with `preserve_config: true` an enabled plugin keeps running until
|
||||
the display restarts, and it is true.
|
||||
|
||||
### Update Plugin
|
||||
|
||||
**POST** `/api/v3/plugins/update`
|
||||
@@ -684,11 +790,21 @@ Update a plugin to the latest version. Runs synchronously.
|
||||
"message": "Plugin football-scoreboard updated ...",
|
||||
"data": {
|
||||
"last_updated": "2025-01-15T10:30:00Z",
|
||||
"commit": "abc1234..."
|
||||
}
|
||||
"commit": "abc1234...",
|
||||
"update_status": "updated"
|
||||
},
|
||||
"restart_required": true,
|
||||
"restart_message": "Plugin updated — restart the display to run the new version"
|
||||
}
|
||||
```
|
||||
|
||||
`update_status` is `updated`, `up_to_date` or `local_only`.
|
||||
`restart_required` is true when the plugin changed and is enabled: the
|
||||
running display keeps the code it loaded until it restarts.
|
||||
|
||||
An update this core cannot run answers `409` with `Plugin update refused:`
|
||||
and the reason; the installed version is left as it was.
|
||||
|
||||
### Install Plugin from URL
|
||||
|
||||
**POST** `/api/v3/plugins/install-from-url`
|
||||
@@ -717,10 +833,13 @@ Install a plugin directly from a GitHub repository URL. Runs synchronously.
|
||||
"message": "Plugin my-plugin installed successfully",
|
||||
"plugin_id": "my-plugin",
|
||||
"name": "My Plugin",
|
||||
"branch": "main"
|
||||
"branch": "main",
|
||||
"restart_required": false
|
||||
}
|
||||
```
|
||||
|
||||
`restart_required` follows the same rule as `/plugins/install`.
|
||||
|
||||
### Load Registry from URL
|
||||
|
||||
**POST** `/api/v3/plugins/registry-from-url`
|
||||
@@ -892,8 +1011,11 @@ copy, not the display service's in-memory state.
|
||||
|
||||
**GET** `/api/v3/plugins/state`
|
||||
|
||||
Get the state manager's record for every plugin, keyed by plugin id. Pass
|
||||
`?plugin_id=<id>` for one plugin (`data` is then that record).
|
||||
Every plugin that is installed or configured, keyed by plugin id: desired
|
||||
state from `config.json` and the plugins directory, observed state from the
|
||||
display's runtime snapshot. Built per request; there is no state file.
|
||||
Pass `?plugin_id=<id>` for one plugin (`data` is then that record; 404 if
|
||||
it is neither installed nor configured).
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
@@ -902,23 +1024,41 @@ Get the state manager's record for every plugin, keyed by plugin id. Pass
|
||||
"data": {
|
||||
"football-scoreboard": {
|
||||
"plugin_id": "football-scoreboard",
|
||||
"status": "loaded",
|
||||
"status": "enabled",
|
||||
"installed": true,
|
||||
"in_config": true,
|
||||
"enabled": true,
|
||||
"version": "1.2.3",
|
||||
"loaded": true,
|
||||
"state": "enabled",
|
||||
"error_info": null,
|
||||
"loaded_version": "1.2.3",
|
||||
"loaded_at": 1790000000.0,
|
||||
"installed_at": "2025-01-15T10:30:00",
|
||||
"last_updated": "2025-01-15T10:30:00",
|
||||
"config_version": 1,
|
||||
"metadata": {}
|
||||
"last_updated": "2025-01-15T10:30:00"
|
||||
}
|
||||
}
|
||||
},
|
||||
"runtime": {"status": "live", "published_at": 1790000030.0, "age_seconds": 12.4, "stale_after": 180.0}
|
||||
}
|
||||
```
|
||||
|
||||
`status` is `enabled` / `disabled` for an installed plugin, `unknown` for
|
||||
one that is configured but not installed, and `error` when the display
|
||||
reports its state as `error`. `installed_at` and `last_updated` are the
|
||||
newest successful install, and install or update, in the operation history
|
||||
(`null` when it has none). The runtime fields follow the same rule as
|
||||
[`/plugins/installed`](#get-installed-plugins): `null` unless
|
||||
`runtime.status` is `live`.
|
||||
|
||||
### Reconcile Plugin State
|
||||
|
||||
**POST** `/api/v3/plugins/state/reconcile`
|
||||
|
||||
Reconcile plugin state across config, disk and the state manager.
|
||||
Reconcile desired state (`config.json` plus the plugins on disk) with the
|
||||
display's runtime snapshot. Desired-state gaps are fixed (a plugin on disk
|
||||
with no config section is added disabled); observed-state gaps -- enabled
|
||||
but not loaded, loaded at an older version than is installed -- are
|
||||
reported with `fix_action: "no_action"`.
|
||||
|
||||
**Request Body** (optional):
|
||||
```json
|
||||
@@ -1189,13 +1329,22 @@ searches.
|
||||
"version": "1.2.3",
|
||||
"branch": "main",
|
||||
"default_branch": "main",
|
||||
"plugin_path": "plugins/football-scoreboard"
|
||||
"plugin_path": "plugins/football-scoreboard",
|
||||
"commit": "843588025a81197056f8d96779ccb2be19337ab8",
|
||||
"ledmatrix_min_version": "3.7.0",
|
||||
"aliases": [],
|
||||
"incompatible_reason": null
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`commit` (the monorepo commit that introduced `version`),
|
||||
`ledmatrix_min_version` and `aliases` come from the registry entry and are
|
||||
`null` / `[]` when an older registry lacks them. `incompatible_reason` is the
|
||||
message an install would be refused with on this core, or `null`.
|
||||
|
||||
### Get GitHub Status
|
||||
|
||||
**GET** `/api/v3/plugins/store/github-status`
|
||||
@@ -1336,20 +1485,59 @@ Get LEDMatrix repository version.
|
||||
|
||||
**GET** `/api/v3/system/check-update`
|
||||
|
||||
Whether `origin/main` has commits the checkout lacks. Cached briefly.
|
||||
Fields at the top level (no envelope):
|
||||
Whether newer code is available on this device's update channel. On
|
||||
`stable` that is a newer release tag than the checkout (`target_version`
|
||||
names it); on `beta`, and on `stable` while it waits on a branch for a
|
||||
release that contains the current commit, it is commits on `origin/main`
|
||||
the checkout lacks. A detached checkout newer than the newest release is
|
||||
never offered an update: Update Code leaves it where it is until a release
|
||||
includes it, and `channel_message` says so in the General tab's words.
|
||||
Cached briefly. Fields at the top level (no envelope):
|
||||
|
||||
```json
|
||||
{
|
||||
"update_available": true,
|
||||
"remote_sha": "abc123...",
|
||||
"commits_behind": 3
|
||||
"commits_behind": 3,
|
||||
"target_version": "v3.8.0",
|
||||
"channel": "stable",
|
||||
"configured_channel": "stable",
|
||||
"waiting": false,
|
||||
"newest_release": "v3.8.0",
|
||||
"current_release": null,
|
||||
"channel_message": "Stable: release v3.8.0 is available."
|
||||
}
|
||||
```
|
||||
|
||||
When git cannot run the check, the response also carries
|
||||
`"check_failed": true` and an `error` explaining why.
|
||||
|
||||
### Update Channel
|
||||
|
||||
**GET** `/api/v3/system/update-channel`
|
||||
|
||||
The update channel and what the next Update Code or weekly update would do
|
||||
(in `data`): `configured` (`"stable"`, `"beta"` or `null` for a config from
|
||||
before channels), `channel` (the one in effect), `waiting` (stable, but the
|
||||
device is newer than the newest release, so it follows `main` for now),
|
||||
`action` (`none`, `checkout_tag`, `pull` or `switch_to_beta`),
|
||||
`newest_release`, `current_release`, `branch` (`""` when on a release tag),
|
||||
`message`. Reads local refs; `?fetch=1` fetches from origin first.
|
||||
|
||||
**POST** `/api/v3/system/update-channel`
|
||||
|
||||
```json
|
||||
{
|
||||
"channel": "beta"
|
||||
}
|
||||
```
|
||||
|
||||
Saves `auto_update.channel`. The next update applies it; switching to
|
||||
`stable` never installs an older version than the one running, and the
|
||||
`message` says when the device keeps following `main` until a newer release.
|
||||
400 for anything but `stable` or `beta`. The General tab form also accepts
|
||||
`auto_update_channel` on `POST /api/v3/config/main`.
|
||||
|
||||
### Automatic Update Status
|
||||
|
||||
**GET** `/api/v3/system/auto-update`
|
||||
@@ -1375,7 +1563,9 @@ Hide the current automatic-update alert until a new one replaces it.
|
||||
|
||||
Branch, dirty state, recent commits and remote for the Tools tab. Fields at
|
||||
the top level: `branch`, `dirty`, `status`, `recent_commits`, `remote_url`
|
||||
(credentials scrubbed), `upstream`, `can_pull`.
|
||||
(credentials scrubbed), `upstream`, `can_pull`, and for the update channel
|
||||
`detached`, `version` (`git describe`), `current_release` (the release tag
|
||||
HEAD is exactly on, else `null`) and `channel_message` (detached only).
|
||||
|
||||
### Git Branches
|
||||
|
||||
@@ -1388,7 +1578,10 @@ Fetches `origin` and lists branches to switch to: `current`, `upstream`,
|
||||
|
||||
**POST** `/api/v3/system/action`
|
||||
|
||||
Execute system-level actions. JSON or form data.
|
||||
Execute system-level actions. Send JSON (`Content-Type: application/json`).
|
||||
A form-encoded or `text/plain` body is accepted only with an `HX-Request`
|
||||
header (HTMX sends it; a cross-site HTML form cannot) and is otherwise
|
||||
refused with `415`.
|
||||
|
||||
**Request Body**:
|
||||
```json
|
||||
@@ -1949,6 +2142,18 @@ Health of the web interface, display service, config file, plugin system and
|
||||
display snapshot. `data.status` is `healthy` or `degraded`, with
|
||||
`data.services` and `data.checks`.
|
||||
|
||||
`data.checks.display_loop` is the display's render-loop heartbeat: `running`
|
||||
(with `heartbeat_age_seconds`), `stalled` (no heartbeat for 60s: the panel is
|
||||
frozen even if the service is active; the status turns `degraded`), or
|
||||
`not_reported` when the display writes none (not started yet, the dev server,
|
||||
Windows), which does not affect the status.
|
||||
|
||||
Open even when the web login is on, for uptime monitors; a caller that is not
|
||||
logged in (and has no token) then gets only `{"status": "success", "data":
|
||||
{"status": "healthy" | "degraded"}}`. A stalled render loop still shows there
|
||||
as `degraded`; the `checks` detail is only for logged-in callers, tokens and
|
||||
requests from the Pi itself.
|
||||
|
||||
### Hardware Status
|
||||
|
||||
**GET** `/api/v3/hardware/status`
|
||||
@@ -2014,20 +2219,100 @@ enabled.
|
||||
|
||||
Home Assistant MQTT bridge service state and settings: `data.service`,
|
||||
`data.config_exists`, `data.config_path`, `data.config` (password
|
||||
omitted), `data.password_set`, `data.env_override_prefix`.
|
||||
omitted), `data.password_set`, `data.api_token_set`, `data.env_override_prefix`.
|
||||
|
||||
**PUT** `/api/v3/integrations/mqtt-bridge/config`
|
||||
|
||||
Write `integrations/mqtt_bridge/bridge_config.json`. Only the keys you send
|
||||
change. The password is write-only: omit `mqtt_password` to keep it, send a
|
||||
value to replace it, or send `"clear_password": true`. A password with
|
||||
value to replace it, or send `"clear_password": true`. The web-login API token
|
||||
the bridge sends (`ledmatrix_api_token`, needed only when login is on and the
|
||||
bridge runs on another machine) is write-only the same way, cleared with
|
||||
`"clear_api_token": true`. A password with
|
||||
`mqtt_tls` off is refused unless `allow_insecure_mqtt` is true. Returns
|
||||
`data.password_set` and `data.restart_required` (the bridge must be
|
||||
restarted to pick up changes). See
|
||||
`data.password_set`, `data.api_token_set` and `data.restart_required` (the
|
||||
bridge must be restarted to pick up changes). See
|
||||
[integrations/mqtt_bridge/README.md](../integrations/mqtt_bridge/README.md).
|
||||
|
||||
---
|
||||
|
||||
## Web login and API tokens
|
||||
|
||||
The optional password and API tokens (`web_auth` in
|
||||
`config/config_secrets.json`, `web_interface/auth.py`). No route here returns
|
||||
the password hash, a token hash or the cookie key. A request authenticated by
|
||||
an API token gets `403` `TOKEN_NOT_ALLOWED` from every route in this section:
|
||||
tokens are for integrations, not for changing who can log in. Wrong current
|
||||
passwords (`403` `WRONG_PASSWORD`) count against the same per-address limit as
|
||||
the login page: 5 a minute, 30 an hour, then `429`.
|
||||
|
||||
Lost password: run `sudo python3 scripts/reset_web_password.py` on the Pi.
|
||||
|
||||
### Login status
|
||||
|
||||
**GET** `/api/v3/auth/status`
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "success",
|
||||
"data": {
|
||||
"enabled": true,
|
||||
"signed_in": true,
|
||||
"access": "session",
|
||||
"min_password_length": 8,
|
||||
"tokens": [
|
||||
{"id": "3f9c1a2b4d5e6f70", "name": "Home Assistant", "prefix": "lmx_Ab3d",
|
||||
"created_at": "2026-09-29T20:14:03+00:00"}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`access` is how this request got in: `open` (login off), `session`,
|
||||
`localhost`, `ap-setup` or `token`.
|
||||
|
||||
### Set or change the password
|
||||
|
||||
**POST** `/api/v3/auth/password`
|
||||
|
||||
Body: `{"new_password": "...", "current_password": "..."}`.
|
||||
`current_password` is required once login is on. At least 8 characters, no
|
||||
leading or trailing space (`400` `WEAK_PASSWORD`). Setting the first password
|
||||
turns login on. Every existing login session ends; the caller's own browser
|
||||
is signed in again with the answer.
|
||||
|
||||
### Turn login off
|
||||
|
||||
**POST** `/api/v3/auth/disable`
|
||||
|
||||
Body: `{"current_password": "..."}`. Removes the password; API tokens are
|
||||
kept (and are needed again if login is turned back on).
|
||||
|
||||
### API tokens
|
||||
|
||||
**GET** `/api/v3/auth/tokens` — `data.tokens`, as in the status answer.
|
||||
|
||||
**POST** `/api/v3/auth/tokens` — body `{"name": "Home Assistant"}` (1-60
|
||||
characters). Answers `201` with `data.token`, the token itself (`lmx_` plus 43
|
||||
characters), and `data.record`. **The token is never shown again**; only its
|
||||
SHA-256 is stored. At most 50 tokens.
|
||||
|
||||
**DELETE** `/api/v3/auth/tokens/<id>` — revoke; it stops working on the next
|
||||
request. `404` for an unknown id.
|
||||
|
||||
Send a token as `Authorization: Bearer <token>`.
|
||||
|
||||
### Login page
|
||||
|
||||
`GET /login` shows the login form (and redirects home when login is off or
|
||||
this browser is already signed in); `POST /login` with a form field `password`
|
||||
(and optional `next`, a path on this server) signs in and redirects to `next`,
|
||||
or answers `401` with the form again. `POST /logout` ends the session. Both
|
||||
are outside `/api/v3` and go through the cross-site check like every other
|
||||
`POST`.
|
||||
|
||||
---
|
||||
|
||||
## Plugin-specific endpoints
|
||||
|
||||
A handful of endpoints belong to individual plugins. The music plugin's
|
||||
|
||||
+284
-52
@@ -35,10 +35,11 @@ defaults, or as capabilities they opt into.
|
||||
|
||||
### Reusability — write once, nine plugins benefit
|
||||
|
||||
Only code that is **identical in intent across all nine** moves into the base
|
||||
class. That set is small and knowable — it is exactly the methods present in every
|
||||
copy today (phase B1 below). Everything else stays where it is until it earns
|
||||
promotion.
|
||||
Only code that is **identical across every plugin that carries it** moves into
|
||||
core. Stages 0–3 moved the copies that already were; what is left has drifted,
|
||||
and earns promotion by being reconciled first — made identical in all nine
|
||||
plugins, one method family per release, with every visible difference decided
|
||||
rather than averaged away. See [Roadmap](#roadmap).
|
||||
|
||||
### Modularity — a change to one feature cannot reach a plugin that doesn't use it
|
||||
|
||||
@@ -83,6 +84,9 @@ more. Shared sports code lives in `src/common`:
|
||||
| `espn_dates.py` | 3.5.0 | ESPN date-range and `limit` workarounds |
|
||||
| `favorite_team_check.py` | 3.6.0 | `FavoriteTeamCheck` — logs why a favourite team code shows nothing |
|
||||
| `sports_timezone.py` | 3.6.0 | Which timezone start times are drawn in (`resolve_timezone_name`) |
|
||||
| `sports_celebration.py` | 3.7.0 | `SportsCelebrationMixin` — draws the score/win takeover; colour helpers |
|
||||
| `sports_fetch.py` | 3.7.0 | `SportsFetchMixin` — season fetch, live lookback and live-odds decisions |
|
||||
| `sports_card_wrappers.py` | 3.7.0 | `SportsCardWrappersMixin` — the game renderer's `sports_card` delegations |
|
||||
|
||||
Each is described in [src/common/README.md](../src/common/README.md).
|
||||
|
||||
@@ -94,9 +98,10 @@ modules taken from the plugin copies, each a **new module** rather than growth
|
||||
on an existing one: a plugin that deletes a method copy and relies on an older
|
||||
module having gained it fails at runtime with an `AttributeError`, while a
|
||||
missing module fails at load, where the version checks can see it.
|
||||
`sports_helpers.py` is the newest (it holds `_favorite_key`, the override point
|
||||
listed below, for later phases); its parity test compares every body against
|
||||
the plugin copies when `LEDMATRIX_PLUGINS` points at a checkout, and
|
||||
`sports_helpers.py` holds `_favorite_key`, the override point listed below.
|
||||
Each promoted module has a parity test that compares its bodies against the
|
||||
plugin copies when `LEDMATRIX_PLUGINS` points at a checkout
|
||||
(`test_sports_helpers.py`, `test_sports_stage3_parity.py`), and
|
||||
`test/test_common_is_hardware_free.py` keeps `src/common` free of
|
||||
`rgbmatrix`, `src.display_manager` and `src.plugin_system`. How a plugin adopts a
|
||||
module and drops its copy is documented in the plugins repo's
|
||||
@@ -168,6 +173,15 @@ Mix it in **before** the mode class — `class SoccerLive(CelebrationMixin,
|
||||
SportsLive)` — so the celebration `display()` runs first and falls through to
|
||||
the scorebug via `super()`.
|
||||
|
||||
What shipped is narrower. `src/common/sports_celebration.py`
|
||||
(`SportsCelebrationMixin`) holds only the drawing, which is identical in the
|
||||
five scoreboards that celebrate (afl, football, hockey, nrl, soccer — hockey
|
||||
grew celebrations after this was written). Arming a celebration stays in each
|
||||
plugin: the trigger bodies differ (nrl matches favourites by team id, football
|
||||
folds a touchdown's extra point into one celebration and picks scenery by
|
||||
points), and so does `display()`. The seams above were not needed to move the
|
||||
drawing, so none was added.
|
||||
|
||||
**Rotation strategies.** The three "dialects" turned out to be one algorithm
|
||||
(Smooth Weighted Round-Robin) in two shapes: an incremental picker holding state
|
||||
across calls (afl/nrl/soccer) and a precomputed per-cycle list
|
||||
@@ -223,12 +237,249 @@ legacy compatibility rather than the mechanism.
|
||||
> (`display_manager.refresh_hz`), and speed comes from
|
||||
> `scroll_settings.scroll_speed` alone. See `docs/SCROLL_PERFORMANCE.md`.
|
||||
|
||||
## Phases
|
||||
## Roadmap
|
||||
|
||||
B0–B3 are merged and shipping in core 3.2.0. Everything that remains is
|
||||
**rollout**, and it splits into three phases with very different risk profiles.
|
||||
The original plan folded the last two together; they are separated here because
|
||||
one of them cannot break a user on an old core and the other can.
|
||||
### Done: stages 0–3
|
||||
|
||||
The second project, after the B phases below: move what the nine `sports.py`
|
||||
copies (and their support files) carried byte-identically into `src/common`,
|
||||
one new module per stage, and delete the copies once the plugins floor on the
|
||||
release that ships it.
|
||||
|
||||
| Stage | Core | Plugins (ledmatrix-plugins) | What moved |
|
||||
|---|---|---|---|
|
||||
| 0 | none needed; found #662 (odds `no_odds` marker) and #663 (a reloaded plugin's dir goes first on `sys.path`) | #562 | Deleted the bundled copies nothing could reach (`base_odds_manager`, `logo_downloader`, three unused data sources, ~4.2k lines); three UFC fixes |
|
||||
| 1 | 3.5.0: `sports_helpers` (#583), `espn_dates`, `json_body` | #563 (1a, the eight team scoreboards), #564 (1b, ufc); floor 3.5.0 | The identical helpers and ESPN date-range handling; ufc also adopted the `sports_shared` mixins |
|
||||
| 2 | 3.6.0: `favorite_team_check`, `sports_timezone`; fixes in 3.6.1 (#667) and 3.6.2 (#670) | #565 (guarded adoption), #567 (f1), #570 (sunset, floor 3.6.1), #571 (soccer, 3.6.2) | The favourite-team check (seven copies) and the timezone resolver (ten); each plugin keeps a thin timezone binding |
|
||||
| 3 | 3.7.0 (#672): `sports_celebration`, `sports_fetch`, `sports_card_wrappers` | #572 (goldens first), #574 (floor 3.7.0, copies deleted) | Celebration drawing (five plugins), four fetch methods (nine), seventeen card delegations (eight renderers) |
|
||||
|
||||
Stage 3 was re-checked independently when this roadmap was written: #574's
|
||||
parent and #574 itself, rendered through the core harness against core 3.7.0,
|
||||
gave pixel-identical output for all 399 frames (192 harness screens across the
|
||||
nine plugins at the eight default sizes, 72 scroll/Vegas cards, 135
|
||||
celebration frames), with a parent-vs-parent rerun as the determinism control.
|
||||
|
||||
### Why the method changes
|
||||
|
||||
Byte-identical promotion has nearly run dry. Measured on ledmatrix-plugins
|
||||
`4327c2e` (2026-09-29, after stage 3) with `scripts/sports_drift_report.py`:
|
||||
|
||||
| File | Method families | In all nine | Method lines | Identical copies beyond the first | Drifted families |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| `sports.py` | 94 | 30 | 30,976 | 1,479 lines | 19 |
|
||||
| `manager.py` | 114 | 36 | 27,137 | 3,198 lines | 38 |
|
||||
| `game_renderer.py` (8 plugins) | 89 | — | 6,830 | 546 lines | 11 |
|
||||
|
||||
"Drifted" means in at least seven plugins with at least three different
|
||||
bodies. Everything still identical adds up to about 5,200 duplicated lines;
|
||||
the rest of the ~65,000 method lines is drifted, one outlier away from
|
||||
identical, or unique to one plugin. Drifted code cannot move unchanged, so
|
||||
consolidation stalls unless the copies are made identical first.
|
||||
`manager.py`, the largest copy of all and the layer the display controller and
|
||||
Vegas talk to, was in no plan before this one.
|
||||
|
||||
### The method: reconcile, then promote
|
||||
|
||||
**Owner decision (2026-09-29):** each release, pick one drifted method family,
|
||||
make all nine copies identical, then promote it to core. A *family* here is a
|
||||
set of methods that share state and ship together (the rankings methods, the
|
||||
game-over check); the report measures each method in it. The procedure:
|
||||
|
||||
1. **Measure.** `python scripts/sports_drift_report.py --family sports.py::<name> --diff`
|
||||
lists which plugins share each body and diffs every variant against the
|
||||
most common one. Put the grouping in the PR.
|
||||
2. **Classify every difference**, and say which class in the PR:
|
||||
- *A fix one copy has and the others lack* (a lock, a guard, a correct
|
||||
season year). Port it. It is a behaviour change, so it gets a CHANGELOG
|
||||
line in each plugin.
|
||||
- *A per-sport fact* (hockey ends in period 3; a soccer clock counts up).
|
||||
Make it a declared class constant or override point with a default, as
|
||||
`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`, `COALESCE_SCORING_SEQUENCE` and
|
||||
`_favorite_key` are, and add it to the tables above. Never a sport-name
|
||||
branch: core must not learn sport names.
|
||||
- *A product difference*: anything a user can see (which games show, a
|
||||
colour, a date, a badge, how long a screen stays). The owner picks the
|
||||
behaviour before the code changes; the decision goes in the PR and in a
|
||||
test that pins it (as `test/test_sports_twins.py` pins the twins).
|
||||
- *Noise*: comments, log wording, dead branches. Pick one.
|
||||
3. **Pin the output first.** Before touching the family, its output must be
|
||||
covered: the harness goldens (`test/golden`), the scroll cards
|
||||
(`golden-cards`) and celebrations (`golden-celebration`) for drawing
|
||||
families; for logic families, a table-driven test over the nine plugins'
|
||||
fixture games. Missing coverage lands in its own PR first, as #572 did for
|
||||
stage 3.
|
||||
4. **Reconcile in the plugins** (a monorepo PR). The report must show one
|
||||
variant per class for the family. Render every touched plugin before and
|
||||
after through the harness and diff pixels, not hashes. Every differing
|
||||
frame must match a recorded product decision; any other difference is a
|
||||
bug. Bump each plugin's `version`, add a `versions[]` entry and a CHANGELOG
|
||||
entry, and run `update_registry.py`.
|
||||
5. **Promote in core**: a new `src/common` module per family (a new module, not
|
||||
growth on an old one, for the reason under Converging on `src/common`), a
|
||||
parity test against the plugin copies, and a CHANGELOG module entry naming
|
||||
the release that ships it.
|
||||
6. **Adopt** once that release is out: each plugin floors on it, inherits the
|
||||
mixin, deletes its copy, gains a sunset guard (like the monorepo's
|
||||
`scripts/test_stage3_mixin_copies.py`), and is pixel-diffed again; the
|
||||
expected difference is zero.
|
||||
7. **Re-measure** and update the numbers here.
|
||||
|
||||
A family is only reconciled when *all nine* agree. Leaving one plugin behind
|
||||
recreates the drift the report exists to measure.
|
||||
|
||||
Soaks: pixel diffs prove the drawing, not the timing. A family that changes
|
||||
when data arrives or which games are live (5, 7, 9, 13 and 14 below) needs a
|
||||
live-game soak on a rig, and out-of-season sports wait for their season.
|
||||
Before a soak, check the rig's `*_display_mode`: a board in `switch` mode tells
|
||||
you nothing about the scroll path.
|
||||
|
||||
### Order
|
||||
|
||||
One family per release, in this order. Variant counts are from the report
|
||||
above (per method: distinct bodies across the plugins that carry it, counted
|
||||
per class role). Stage 4 needs no reconciliation and can ride along with any
|
||||
release.
|
||||
|
||||
| # | Family | Methods (variants) | Why here |
|
||||
|---|---|---|---|
|
||||
| 4 | Identical sweep | `manager.py`: `_dispatch_switch_refresh`, `_favorite_team_is_live`, `get_vegas_priority_weight`, `_game_involves`, `_favorite_scan_targets`, `_favorite_scan_games`, `_get_total_games_for_manager` (all nine, 1); the live-scroll helpers `_preserving_scroll_position`, `_refresh_live_scroll_managers`, `_live_scroll_managers`, `_note_live_scroll_built`, `_live_scroll_needs_rebuild`, `_live_scroll_fields` (eight, 1). `sports.py`: `_card_option`, `_filtered_or_all`, `_effective_live_duration`, `_recent_date_text` (eight, 1). 58 identical families in all | Nothing to decide; brings `manager.py` into core as a `SportsPluginHostMixin`. `_resolve_font_path` (identical in nine `sports.py` and eight renderers) is replaced by core's `font_layout.resolve_asset_path` rather than promoted |
|
||||
| 5 | Game-over check | `SportsLive._is_game_really_over` (5) | Pure logic, no pixels; its seams (`FINAL_PERIOD`, `CLOCK_COUNTS_DOWN`) were designed in B1. The pilot for the procedure |
|
||||
| 6 | Favourite matching | `_is_favorite_game` (7 across three classes), `_select_games_for_display` (2: nrl), `_select_recent_games_for_display` (3) | Everything that asks "is this a favourite" goes through the 3.5.0 `_favorite_key` seam |
|
||||
| 7 | Other-games rotation | `_by_importance`, `_other_games_window`, `_advance_other_games_if_due` (2 each: football), `_rotate_other_games_on_display` (2: ufc) | One outlier each; football carries two fixes the other eight lack |
|
||||
| 8 | Rankings | `_fetch_team_rankings` (3), `_choose_poll` (3), `_load_division_team_ids`, `_passes_other_filters`, `_best_rank`, `_is_ranked_game` (2 each: football) | Needs 7; the rank badge and the "ranked only" filter read it |
|
||||
| 9 | Live fetch and odds | `_fetch_todays_games` (5), `_fetch_odds` (3), `_attach_odds_to_rotated_games` (3) | The prerequisite for one shared ESPN poller across plugins |
|
||||
| 10 | View model | `_extract_game_details_common` (9 of 9) | Every renderer reads it; its keys are additive-only, so reconcile to the superset and leave sport extras in `_extract_game_details` |
|
||||
| 11 | Switch scorebug | `_load_fonts` (4), `_load_custom_font_from_element_config` (6), `_get_layout_offset` (5), `_fit_score_font` (2: football), `_load_and_resize_logo` (9), `_draw_dynamic_odds` (9), then `_draw_scorebug_layout` (20: Upcoming 9, Recent 8, Live 2, ufc's Core 1) | Needs 10 and the twin decisions below. Several releases: fonts and offsets, logos, odds, then one mode's layout per release |
|
||||
| 12 | Scroll/Vegas card | `game_renderer.py`: `render_game_card` (6), `_load_and_resize_logo` (8), `_load_custom_font`, `preload_logos`, `__init__` (7 each), `_draw_records_or_rankings` (6), `_load_fonts`, `_draw_dynamic_odds`, `_draw_live_game_status`, `_draw_recent_game_status`, `_get_team_display_text` (5 each), `_draw_text_with_outline` (2: football) | Same decisions as 11; sequence after the scroll-performance work on pre-rendered strips lands |
|
||||
| 13 | Mode lifecycle | `sports.py`: `update` (23), `__init__` (16), `display` (13), `_advance_live_game_if_due` (6) | Where per-sport behaviour lives; last in `sports.py`. Each difference becomes a seam or a strategy chosen by name (live rotation already has three) |
|
||||
| 14 | `manager.py` host | Nine different bodies in nine plugins: `__init__`, `display`, `update`, `has_live_content`, `get_live_modes`, `get_vegas_content`, `on_config_change`, `_initialize_managers`, `_get_available_modes`, `_get_current_manager`, `_adapt_config_for_manager`. Dynamic duration: `_evaluate_dynamic_cycle_completion` (9), `_record_dynamic_progress` (8), `get_cycle_duration` (8), `get_dynamic_duration_cap` (6), `supports_dynamic_duration`, `is_cycle_complete` (5 each), `reset_cycle_state` (4). Scroll: `_display_scroll_mode` (8), `_ensure_scroll_content_for_vegas` (8), `_collect_games_for_scroll` (7), `_should_use_scroll_mode`, `_has_any_scroll_mode` (6 each) | See below |
|
||||
|
||||
**`manager.py`.** Reconciling it body by body would take a release per
|
||||
method. The plan is a host class in core, `SportsScoreboardPlugin(BasePlugin)`,
|
||||
that takes the plugin's leagues as data (key, label, ESPN path, Live, Recent
|
||||
and Upcoming classes: basketball's `manager.py` already describes its leagues
|
||||
as such a table) and a typed mode key instead of the mode-name string parsing
|
||||
(`endswith('_live')`, `split('_')`) every copy repeats. Order within it:
|
||||
stage 4's identical helpers first; then dynamic duration, then live priority (`has_live_content`,
|
||||
`get_live_modes`, `has_live_priority`), then Vegas content, then mode
|
||||
resolution, then the lifecycle methods. Pilot the whole host on nrl or afl,
|
||||
the smallest copies (about 1,850 lines each), with a frame soak and a
|
||||
live-game soak before a second plugin moves. `get_vegas_content` is also being
|
||||
changed by the scroll-performance work: coordinate before touching it.
|
||||
|
||||
**Held:** `data_sources.py` (nine copies; soccer's matches core's) and
|
||||
`dynamic_team_resolver.py` (eight true forks, a different constructor from
|
||||
core's). Effort on data fetching is better spent on the shared poller that
|
||||
family 9 prepares.
|
||||
|
||||
### Product decisions each family needs
|
||||
|
||||
Owner calls to make before (or while) reconciling. Items marked *verify* are
|
||||
suspected behaviour that needs a payload or a rig to confirm first.
|
||||
|
||||
- **5, game-over check.** Which rule each sport gets: the clock never ends a
|
||||
game in afl, nrl and soccer (`CLOCK_COUNTS_DOWN = False`); hockey ends at
|
||||
0:00 from period 3, basketball, football and lacrosse from period 4.
|
||||
baseball and ufc share a copy that reads a missing clock as "0:00": dormant
|
||||
in baseball (its games carry no `period`), and not triggered by ufc's round
|
||||
breaks either. ESPN sends a break as `STATUS_END_OF_ROUND` with displayClock
|
||||
`-`, not `0:00` (verified against recorded payloads; ledmatrix-plugins#580
|
||||
pins it). Whatever rule ufc gets must not read `-` as `0:00`. Decide ufc's
|
||||
rule: no clock rule (ESPN's `STATUS_FINAL` is the only end signal it needs;
|
||||
this also closes a ~1 s window at the horn when the ticking clock reads
|
||||
`0:00`), or its own final period.
|
||||
- **6, favourite matching.** NRL keeps matching favourites by team id
|
||||
(abbreviations collide: NEW, CAN), through `_favorite_key` rather than its
|
||||
own copies of the selection methods. Six plugins log the recent-games
|
||||
selection at INFO; baseball, football and ufc do not.
|
||||
- **7, other-games rotation.** football advances the rotation window under
|
||||
`_games_lock` (update() and display() both advance it; interleaved, a
|
||||
window of games is skipped) and fixes a favourites-only pool that recomposed
|
||||
the list on every frame. Port both. ufc does not attach odds to fights
|
||||
rotated in: decide whether rotated fights show odds.
|
||||
- **8, rankings.** (a) afl, basketball, nrl and soccer turn a *standings*
|
||||
payload into ranks (a pro league's standings position becomes the rank
|
||||
badge); baseball, hockey, lacrosse, ufc and football do not. Which is
|
||||
right is visible on every pro-league card with "show ranking" on.
|
||||
(b) football also keys ranks by team id, so two schools sharing an
|
||||
abbreviation across divisions cannot be confused: adopt for all.
|
||||
(c) football asks for the division roster of the *season* year (July
|
||||
onward is this year's season), which is right for football and wrong for
|
||||
college basketball, hockey and lacrosse, whose ESPN season is the year it
|
||||
ends: a per-sport seam, not football's constant. (d) baseball's
|
||||
`_choose_poll` calls `dynamic_team_resolver.choose_top_division_poll`, the
|
||||
others inline it: one home, in core.
|
||||
- **9, live fetch and odds.** (a) afl, basketball, nrl and soccer cache the
|
||||
live scoreboard for 30 s under `<sport_key>_scoreboard_current`; the others
|
||||
do not (the harness fixtures seed that key, so the change shows up there).
|
||||
(b) basketball fetches college games with no `dates` parameter, citing a
|
||||
404 (*verify* now that `espn_dates` handles ranges). (c) odds are fetched
|
||||
three ways: blocking (hockey, lacrosse), a thread waited on for 1.5–2 s
|
||||
(six plugins), or fire-and-forget for upcoming games (basketball). This
|
||||
sets how long `update()` takes and when an odds line appears. (d) nrl
|
||||
guards on a missing odds manager; port it.
|
||||
- **10, view model.** Per key, whether every sport emits it. Additive only:
|
||||
no key is renamed or removed.
|
||||
- **11 and 12, the scorebug and the card.** The pinned divergences in
|
||||
`test/test_sports_twins.py`, where switch mode and scroll mode draw the
|
||||
same game differently:
|
||||
- weekday timezone: the card reads only `config["timezone"]` and falls back
|
||||
to UTC, so a board with only the global zone labels an evening kickoff
|
||||
with the next day. **Decided 2026-09-24: use the plugin's timezone
|
||||
(fix); not yet implemented;**
|
||||
- an out-of-range start time: the scorebug drops the weekday, the card
|
||||
raises;
|
||||
- favourite result on a nested payload, which score wins when flat and
|
||||
nested disagree, and where the favourites come from (the manager's list
|
||||
vs the game's stamped list plus config);
|
||||
- the element vocabulary (`team_text` vs `team_name`; rank and odds in one
|
||||
map, not the other), and its consequences: a `team_name` colour reaching
|
||||
one team face and not the other, and the odds face shared with the score
|
||||
face in scroll mode only;
|
||||
- per-mode colour overrides, which apply in switch mode only;
|
||||
- by design, kept unless the owner says otherwise: the date format
|
||||
(`switch_date_format` "numeric" vs the card's "abbrev") and the upcoming
|
||||
centre (`switch_upcoming_center` "date_time" vs "vs"), both with an
|
||||
"inherit" opt-in; and the two schema-font caches (per class vs per path).
|
||||
|
||||
Also: football's `_fit_score_font` swaps to the narrow score face at any
|
||||
panel height when the score overflows, where the other seven keep the
|
||||
design face at or below the design height (a 64x32 board shows the
|
||||
difference); and whether switch mode and the card become one renderer drawn
|
||||
at two sizes.
|
||||
- **13, mode lifecycle.** The live-rotation dialect per sport (incremental
|
||||
SWRR in afl, nrl and soccer; a precomputed schedule elsewhere); which sports
|
||||
arm celebrations and on what (stays in each plugin, as in stage 3).
|
||||
- **14, `manager.py`.** Dynamic-duration semantics (what completes a cycle,
|
||||
the floor and cap per mode), what counts as live content for live priority
|
||||
(favourites only or any live game), and the order of Vegas content.
|
||||
|
||||
### Measuring progress
|
||||
|
||||
`scripts/sports_drift_report.py` prints the numbers above for any
|
||||
ledmatrix-plugins checkout (`--plugins <path>` or `LEDMATRIX_PLUGINS`). CI runs
|
||||
it on every push and PR against the monorepo's main (the "Sports drift report"
|
||||
job in `.github/workflows/test.yml`): report only, never failing, with the
|
||||
tables in the job summary and the full JSON as an artifact. The monorepo's
|
||||
`scripts/check_sports_drift.py` is the gate: it fails when a function that
|
||||
agrees across the plugins starts to differ. A stage is done when its family
|
||||
shows one variant per class here and its copies are gone.
|
||||
|
||||
```
|
||||
python scripts/sports_drift_report.py --plugins ../ledmatrix-plugins
|
||||
python scripts/sports_drift_report.py --family sports.py::_is_game_really_over --diff
|
||||
```
|
||||
|
||||
## Phases B0–B6 (history)
|
||||
|
||||
The first project: it moved the scroll orchestration into core and proved the
|
||||
upgrade path (floors, the store's compatibility gate, the sunset). All seven
|
||||
phases are done. They are kept because the reasoning in B4–B6 is what every
|
||||
later stage relies on; the plan from here is [Roadmap](#roadmap).
|
||||
|
||||
B0–B3 shipped in core 3.2.0. The rollout after them split into three phases
|
||||
with very different risk profiles, because one of them cannot break a user on
|
||||
an old core and the other can.
|
||||
|
||||
| Phase | Scope | Status | Gate |
|
||||
|---|---|---|---|
|
||||
@@ -263,8 +514,12 @@ a floor can be trusted against, and today it is not:
|
||||
the update path that re-downloads.
|
||||
`update_plugin`'s git branch pulls in place and re-downloads nothing, so it
|
||||
stayed ungated until `_gate_pulled_commit` closed it — checked after the pull
|
||||
(the registry carries no floor field, so the incoming floor is unknowable
|
||||
before it) and undone with `git reset --hard` to the pre-pull commit. That
|
||||
(the registry then carried no floor field, so the incoming floor was
|
||||
unknowable before it) and undone with `git reset --hard` to the pre-pull
|
||||
commit. The registry now publishes `ledmatrix_min_version`, and install and
|
||||
update refuse on it before downloading or pulling; both post-download gates
|
||||
remain as the fallback for older registries, other branches and
|
||||
`compatible_versions`. That
|
||||
route is rare in practice, since monorepo plugins install as archives; it was
|
||||
closed because the sunset rule in the plugins repo's
|
||||
`08-shared-sports-code.md` states as **condition 3** that the core enforces
|
||||
@@ -427,10 +682,13 @@ deprecated `ledmatrix_min`). See
|
||||
order any floor-raising tool must reproduce — and note the name is **inverted**
|
||||
between the top level and `versions[]`.
|
||||
|
||||
**Still not adopted, deliberately:** `data_sources.py`, `game_renderer.py` and
|
||||
`base_odds_manager.py`. The standing decision held them until B6 closed; it now
|
||||
has, so they can be reconsidered — with B5's lesson applied, which is to build
|
||||
the object and diff rendered output rather than trust a static check.
|
||||
**The modules held back then** (`data_sources.py`, `game_renderer.py`,
|
||||
`base_odds_manager.py`) have since gone different ways: the eight team
|
||||
scoreboards import core's `base_odds_manager` (ufc keeps an MMA fork), the
|
||||
game renderers inherit core's `SportsCardWrappersMixin` (3.7.0) but keep their
|
||||
drawing, and `data_sources.py` is still copied. Their status is under
|
||||
[Roadmap](#roadmap). B5's lesson applies to all of them: build the object and
|
||||
diff rendered output rather than trust a static check.
|
||||
|
||||
### B5 retrospective — what the adoption actually cost
|
||||
|
||||
@@ -473,40 +731,12 @@ and its one delivered user-visible gain was that adopted plugins honoured the
|
||||
global `target_fps` instead of hardcoding ~100 FPS (since withdrawn: see the
|
||||
note under the B3 design above).
|
||||
|
||||
### Decision: stop adopting further modules until B6 closes
|
||||
### Decision: stop adopting further modules until B6 closes (lifted)
|
||||
|
||||
`data_sources.py` (9 copies), `game_renderer.py` (8) and `base_odds_manager.py`
|
||||
are the obvious next candidates. **Do not adopt them yet.** Each adoption adds
|
||||
carrying cost — a second copy to keep in step — against a payoff that is
|
||||
contingent on B6, and B6 is gated on an installed base we cannot currently
|
||||
measure. Consolidate what is already committed; revisit when B6 does.
|
||||
|
||||
## What's next
|
||||
|
||||
Steps 1–5 of the original plan are **done**: 3.2.0 is tagged and published with
|
||||
a version number CI now asserts (#428), the compatibility gate is in
|
||||
`install_plugin` and reads `compatible_versions` as well as the floor
|
||||
(#431, #433), the newest manifest entry is required to use `ledmatrix_min_version`
|
||||
(plugins #244), and all eight plugins have adopted the scroll orchestration
|
||||
(plugins #245–#249, repaired in #251, tidied in #252).
|
||||
|
||||
What actually remains, smallest first:
|
||||
|
||||
1. **Soak the adoptions on hardware.** football and hockey have been run on a
|
||||
live rig through real games; baseball was watched through one earlier. The
|
||||
rest are proven by harness, unit tests and pixel comparison. Out-of-season
|
||||
sports cannot be soaked until their season starts. When you do, **check the
|
||||
rig's `*_display_mode` first** — a board in `switch` mode will happily load a
|
||||
sunset plugin and tell you nothing about the scroll code the sunset changed.
|
||||
2. **Cut 3.3.0.** Not required by B6 — its floors are 3.2.0, which is released —
|
||||
but `calendar` 1.2.3 floors at 3.3.0 for the device-authorization endpoints
|
||||
that landed after 3.2.0, so it is un-installable until the release exists.
|
||||
3. **Reconsider the held modules** (`data_sources.py`, `game_renderer.py`,
|
||||
`base_odds_manager.py`) now that the sunset has closed. `game_renderer.py` is
|
||||
the largest single duplication left: ~11,500 lines across eight plugins, with
|
||||
~36,500 more in the eight `sports.py`. The `src/base_classes/sports/`
|
||||
package promoted in B1/B2 was never imported by a plugin and has been
|
||||
removed, so the plugin copies are the only starting point.
|
||||
Held from B5 until B6 ran on 2026-09-01: each adoption added a second copy to
|
||||
keep in step against a payoff that depended on the sunset. Once the store
|
||||
refused a too-new plugin on every route, adopting and sunsetting in one stage
|
||||
became safe, and stages 0–3 under [Roadmap](#roadmap) did exactly that.
|
||||
|
||||
## How to keep this project healthy
|
||||
|
||||
@@ -533,7 +763,9 @@ Lessons this migration paid for, worth applying beyond it:
|
||||
## Rules for contributors
|
||||
|
||||
- **Promote on evidence, not intuition.** A method moves to core when every copy
|
||||
has it and they agree on intent. Otherwise it stays in the plugins.
|
||||
that has it is identical. Drifted copies are reconciled first, one family
|
||||
per release, with each visible difference an owner decision (see
|
||||
[Roadmap](#roadmap)); until then they stay in the plugins.
|
||||
- **Never add a sport name to core.** If core needs to know which sport it is,
|
||||
the design is wrong — add an override point instead.
|
||||
- **A capability that is not opted into must not execute.** If you find yourself
|
||||
|
||||
@@ -295,6 +295,42 @@ sudo systemctl cat ledmatrix-web | grep User
|
||||
|
||||
---
|
||||
|
||||
#### Issue: Updates and the update channel
|
||||
|
||||
**Symptoms:**
|
||||
- The General tab says "Stable: this device runs code newer than the newest
|
||||
release ... keeps following main"
|
||||
- Tools shows a version such as `v3.8.0` instead of a branch name, or `git
|
||||
status` over SSH says `HEAD detached at v3.8.0`
|
||||
- Update Code says "already up to date" while GitHub's `main` has newer commits
|
||||
|
||||
**Explanation:** these are the Stable update channel working as intended
|
||||
(`auto_update.channel`, General → Update Channel). Stable installs the
|
||||
newest release tag, which git checks out without a branch ("detached
|
||||
HEAD"); that is normal and every update path handles it. Stable never
|
||||
installs an older version than the one running, so a device that is ahead of
|
||||
the newest release keeps following `main` until a release includes its
|
||||
commit, then switches to releases on its own.
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Want the newest code instead?** Set Update Channel to **Beta** and click
|
||||
Update Code. The device leaves the release for `main` and pulls it.
|
||||
Or from SSH:
|
||||
```bash
|
||||
curl -X POST http://localhost:5000/api/v3/system/update-channel \
|
||||
-H 'Content-Type: application/json' -d '{"channel": "beta"}'
|
||||
```
|
||||
2. **See what the next update will do:**
|
||||
```bash
|
||||
curl 'http://localhost:5000/api/v3/system/update-channel?fetch=1'
|
||||
```
|
||||
3. **Local changes after a channel switch:** edits that no longer fit the new
|
||||
version are kept in the git stash rather than lost; `git stash list`
|
||||
shows them as "LEDMatrix autostash before update".
|
||||
|
||||
---
|
||||
|
||||
### WiFi & AP Mode Issues
|
||||
|
||||
#### AP Mode Not Activating
|
||||
@@ -516,6 +552,64 @@ sudo systemctl cat ledmatrix-web | grep User
|
||||
python3 scripts/check_plugin.py --plugin plugin-id
|
||||
```
|
||||
|
||||
#### Panel Frozen, or the Display Restarts Every Few Minutes
|
||||
|
||||
**Symptoms:**
|
||||
- The panel stops changing while `systemctl status ledmatrix` says `active`
|
||||
- The display restarts on its own, a couple of minutes after it froze
|
||||
- `/api/v3/health` shows `checks.display_loop.status` as `stalled`
|
||||
|
||||
The display's render loop checks in with systemd every few seconds
|
||||
(`WatchdogSec=120` in `ledmatrix.service`) and writes a heartbeat to
|
||||
`/run/ledmatrix/display-heartbeat.json`. When the loop gets stuck -- almost
|
||||
always inside one plugin's `display()` -- the check-ins stop, and after two
|
||||
minutes systemd kills and restarts the display. The kill dumps every thread's
|
||||
stack into the log, so it says which plugin was stuck.
|
||||
|
||||
**Solutions:**
|
||||
|
||||
1. **Find the stuck plugin.** Look for the watchdog kill and the stack dump
|
||||
after it. The render loop is the thread whose stack runs through
|
||||
`display_controller.py` in `run` (usually the `Current thread` block);
|
||||
the first `plugin-repos/...` file in it is the plugin:
|
||||
```bash
|
||||
sudo journalctl -u ledmatrix --since "1 hour ago" | grep -A40 "Watchdog timeout"
|
||||
```
|
||||
|
||||
2. **Check the heartbeat by hand.** Its age should stay under about ten
|
||||
seconds while the display runs:
|
||||
```bash
|
||||
cat /run/ledmatrix/display-heartbeat.json
|
||||
curl -s http://localhost:5000/api/v3/health | python3 -m json.tool | grep -A3 display_loop
|
||||
```
|
||||
`not_reported` means the display writes no heartbeat: it has not drawn
|
||||
its first frame yet, or it runs an older version.
|
||||
|
||||
3. **Disable the plugin** in the web UI and report it to its author with the
|
||||
stack dump. Restarts that repeat back off from 10 seconds to two minutes
|
||||
apart, so a plugin that hangs on every start does not restart the display
|
||||
hundreds of times an hour.
|
||||
|
||||
4. **Is the watchdog installed?** Installs from before it keep their old unit
|
||||
until the installer is re-run (a startup warning says the unit differs
|
||||
from its template):
|
||||
```bash
|
||||
systemctl show -p WatchdogUSec ledmatrix # 2min once running; 0 = not installed
|
||||
sudo ./scripts/install/install_service.sh
|
||||
```
|
||||
`WatchdogUSec` reads `15min` for the first minutes after a start: that is
|
||||
the start-up allowance, narrowed to two minutes once the first frame is on
|
||||
the panel.
|
||||
|
||||
5. **A plugin that legitimately blocks longer** than two minutes (it should
|
||||
not; `display()` runs on the render thread) can be given more time with a
|
||||
drop-in, `sudo systemctl edit ledmatrix`:
|
||||
```ini
|
||||
[Service]
|
||||
WatchdogSec=300
|
||||
```
|
||||
`WatchdogSec=0` turns the watchdog off.
|
||||
|
||||
#### Stale Cache Data
|
||||
|
||||
**Symptoms:**
|
||||
@@ -952,6 +1046,11 @@ git reset --hard HEAD~1
|
||||
# Or rollback to specific commit
|
||||
git reset --hard <commit-hash>
|
||||
|
||||
# On the Stable update channel HEAD is a release tag, not a branch:
|
||||
# go back to an earlier release instead (the next update moves forward again)
|
||||
git tag --list 'v*' --sort=-v:refname | head
|
||||
git checkout --detach v3.7.0
|
||||
|
||||
# Restart all services
|
||||
sudo systemctl restart ledmatrix
|
||||
sudo systemctl restart ledmatrix-web
|
||||
|
||||
@@ -78,7 +78,9 @@ The Overview tab provides at-a-glance information and quick actions:
|
||||
- **Start Display** / **Stop Display** — control the display service
|
||||
- **Restart Display Service** — apply configuration changes
|
||||
- **Restart Web Service** — restart the web UI itself
|
||||
- **Update Code** — `git pull` the latest version (stashes local changes)
|
||||
- **Update Code** — update to the newest version on the update channel (the
|
||||
newest release on Stable, the newest code on `main` on Beta; stashes local
|
||||
changes). The channel is set on the General tab.
|
||||
- **Reboot System** / **Shutdown System** — confirm-gated power controls
|
||||
|
||||
**Display Preview:**
|
||||
@@ -90,6 +92,11 @@ The Overview tab provides at-a-glance information and quick actions:
|
||||
|
||||
Configure basic system settings:
|
||||
|
||||
- **Automatic Updates** — weekly updates with a health check and rollback
|
||||
- **Update Channel** — **Stable** (default) installs releases; **Beta**
|
||||
installs the newest code on `main` before it is released. Switching to
|
||||
Stable never installs an older version: a device ahead of the newest
|
||||
release keeps following `main` until a release includes it
|
||||
- **Timezone** — used by all time/date displays
|
||||
- **Location** — city/state/country for weather and other location-aware
|
||||
plugins
|
||||
@@ -130,6 +137,34 @@ Configure basic system settings:
|
||||
Click **Save** to write changes to `config/config.json`. Most changes
|
||||
require a display service restart from **Overview**.
|
||||
|
||||
Below the settings, the **Security** section (its own buttons, not the Save
|
||||
button) controls the optional login:
|
||||
|
||||
- **Web interface password** — off by default. Setting one turns login on:
|
||||
browsers on your network then see a login page, and stay logged in for 30
|
||||
days (across restarts). The browser you set it from stays logged in.
|
||||
Changing the password logs every other browser out. **Turn login off**
|
||||
needs the current password. A **Log out** button appears in the header
|
||||
while you are logged in. Five wrong passwords in a minute (or 30 in an
|
||||
hour) from one address make it wait.
|
||||
- **API tokens** — for Home Assistant, scripts, or the MQTT bridge on another
|
||||
machine. Give it a name, click **Create token**, and copy the token right
|
||||
away: it is shown once. Revoke it here when it is no longer needed.
|
||||
- Never asked for a password: a browser on the Pi itself, and the Wi-Fi setup
|
||||
page while the Pi is in access-point mode (so you can always get it back on
|
||||
a network).
|
||||
|
||||
**Forgot the password?** SSH into the Pi and run:
|
||||
|
||||
```bash
|
||||
sudo python3 ~/LEDMatrix/scripts/reset_web_password.py
|
||||
```
|
||||
|
||||
(use the folder LEDMatrix is installed in). Login is off again right away,
|
||||
no restart needed, and you can set a new password. API tokens are kept; add
|
||||
`--revoke-tokens` to delete them too. Alternatively, open
|
||||
`http://localhost:5000` in a browser on the Pi itself.
|
||||
|
||||
### Display Tab
|
||||
|
||||
Configure your LED matrix hardware:
|
||||
@@ -346,6 +381,14 @@ The API blueprint (`web_interface/blueprints/api_v3/`) is registered at
|
||||
- `POST /api/v3/plugins/install` — Install a plugin from the store
|
||||
- `POST /api/v3/plugins/install-from-url` — Install a plugin from a GitHub URL
|
||||
|
||||
If the optional login is on, send an API token (General > Security):
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer lmx_..." http://your-pi-ip:5000/api/v3/display/current
|
||||
```
|
||||
|
||||
Scripts running on the Pi itself need no token.
|
||||
|
||||
**Note:** See [REST_API_REFERENCE.md](REST_API_REFERENCE.md) for complete API documentation.
|
||||
|
||||
---
|
||||
@@ -408,9 +451,27 @@ The API blueprint (`web_interface/blueprints/api_v3/`) is registered at
|
||||
## Security Considerations
|
||||
|
||||
**Network Access:**
|
||||
- The interface is accessible to anyone on your local network
|
||||
- No authentication is currently implemented
|
||||
- Recommended for trusted networks only
|
||||
- By default the interface is accessible to anyone on your local network
|
||||
- An optional password (General > Security) makes every page and API call
|
||||
need a login or an API token; see [General Tab](#general-tab). Requests
|
||||
from the Pi itself and the Wi-Fi setup flow in access-point mode stay open,
|
||||
and `/api/v3/health` answers only its overall status without a login
|
||||
- The interface speaks plain HTTP, so the password and tokens cross your
|
||||
network unencrypted: still recommended for trusted networks only
|
||||
- Behind a reverse proxy **on the Pi**, make it send `X-Forwarded-For`
|
||||
(nginx: `proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;`).
|
||||
Without it every proxied request looks like it comes from the Pi itself,
|
||||
which is never asked to log in
|
||||
|
||||
**Other websites:**
|
||||
- A web page you open elsewhere could otherwise make your browser send
|
||||
commands to the Pi (reboot, update, config changes). The interface refuses
|
||||
any change request whose `Origin`/`Referer` header names a different site
|
||||
(403 `CROSS_SITE_REQUEST`), so use the interface from its own address.
|
||||
- Scripts, curl, Home Assistant and the MQTT bridge send no such header and
|
||||
keep working. Behind a reverse proxy, forward the original `Host` header
|
||||
with its port (nginx: `proxy_set_header Host $http_host;` -- `$host`
|
||||
drops the port).
|
||||
|
||||
**Best Practices:**
|
||||
1. Run on a private network (not exposed to internet)
|
||||
@@ -428,7 +489,9 @@ The web interface uses modern web technologies:
|
||||
|
||||
- **Backend:** Flask with Blueprint-based modular design
|
||||
- **Frontend:** HTMX for dynamic content, Alpine.js for reactive components
|
||||
- **Styling:** Tailwind CSS for responsive design
|
||||
- **Styling:** Tailwind CSS utilities, generated at development time and
|
||||
committed (the Pi never builds CSS; see
|
||||
[`web_interface/README.md`](../web_interface/README.md#styling-tailwind-css))
|
||||
- **Real-Time:** Server-Sent Events (SSE) for live updates
|
||||
|
||||
### File Locations
|
||||
|
||||
@@ -835,6 +835,10 @@ if [ ! -f "$PROJECT_ROOT_DIR/config/config.json" ]; then
|
||||
cat > "$PROJECT_ROOT_DIR/config/config.json" <<'EOF'
|
||||
{
|
||||
"web_display_autostart": true,
|
||||
"auto_update": {
|
||||
"enabled": false,
|
||||
"channel": "stable"
|
||||
},
|
||||
"timezone": "America/Chicago",
|
||||
"display": {
|
||||
"hardware": {
|
||||
|
||||
@@ -96,6 +96,13 @@ environment as `LEDMATRIX_MQTT_<KEY>` (`LEDMATRIX_MQTT_MQTT_PASSWORD`, say),
|
||||
which keeps a broker password out of a file on disk — put it in a systemd
|
||||
drop-in with `Environment=` or `EnvironmentFile=` instead.
|
||||
|
||||
**Web login.** If the web interface's optional login is on (General >
|
||||
Security), a bridge running on the Pi itself still needs nothing: requests from
|
||||
the Pi are never asked to log in. A bridge on another machine needs an API
|
||||
token: create one under General > Security and set `"ledmatrix_api_token"`
|
||||
(or `LEDMATRIX_MQTT_LEDMATRIX_API_TOKEN`, or the token field in the Tools tab's
|
||||
bridge settings). It is sent as `Authorization: Bearer <token>`.
|
||||
|
||||
Set `mqtt_tls: true` for a broker with TLS. `mqtt_tls_insecure` skips
|
||||
certificate verification and exists only for a self-signed broker on a
|
||||
trusted LAN; it logs a warning when used.
|
||||
|
||||
@@ -66,6 +66,10 @@ DEFAULTS = {
|
||||
"mqtt_tls": False,
|
||||
"mqtt_tls_insecure": False,
|
||||
"ledmatrix_api_base": "http://localhost:5000",
|
||||
# Only needed when the web interface's optional login is on AND the bridge
|
||||
# reaches it from another machine: requests from the Pi itself never need
|
||||
# one. Create it under General > Security; sent as a Bearer token.
|
||||
"ledmatrix_api_token": None,
|
||||
"request_timeout": 15,
|
||||
"on_demand_duration": None,
|
||||
"log_level": "INFO",
|
||||
@@ -125,10 +129,13 @@ class LEDMatrixClient:
|
||||
"""
|
||||
|
||||
def __init__(self, api_base: str, timeout: int = 15,
|
||||
session: Optional[requests.Session] = None):
|
||||
session: Optional[requests.Session] = None,
|
||||
api_token: Optional[str] = None):
|
||||
self.api_base = api_base.rstrip("/")
|
||||
self.timeout = timeout
|
||||
self.session = session or requests.Session()
|
||||
if api_token:
|
||||
self.session.headers["Authorization"] = f"Bearer {api_token}"
|
||||
|
||||
def _call(self, method: str, path: str, **kwargs) -> Dict[str, Any]:
|
||||
url = f"{self.api_base}/api/v3{path}"
|
||||
@@ -403,7 +410,8 @@ class Bridge:
|
||||
self.status_topic = f"{self.command_topic}/status"
|
||||
self.state_topic = f"{self.command_topic}/state"
|
||||
self.availability_topic = f"{self.command_topic}/availability"
|
||||
self.client = LEDMatrixClient(config["ledmatrix_api_base"], config["request_timeout"])
|
||||
self.client = LEDMatrixClient(config["ledmatrix_api_base"], config["request_timeout"],
|
||||
api_token=config.get("ledmatrix_api_token") or None)
|
||||
self.handler = CommandHandler(self.client, config.get("on_demand_duration"))
|
||||
self._stop = threading.Event()
|
||||
self._mqtt = None
|
||||
|
||||
@@ -32,6 +32,9 @@ src/common/render_gate.py
|
||||
src/common/scroll_config.py
|
||||
src/common/snapshot_policy.py
|
||||
src/common/sports_card.py
|
||||
src/common/sports_card_wrappers.py
|
||||
src/common/sports_celebration.py
|
||||
src/common/sports_fetch.py
|
||||
src/common/sports_scroll.py
|
||||
src/common/sports_timezone.py
|
||||
src/config_service.py
|
||||
@@ -51,10 +54,12 @@ src/plugin_system/compatibility.py
|
||||
src/plugin_system/operation_history.py
|
||||
src/plugin_system/operation_queue.py
|
||||
src/plugin_system/operation_types.py
|
||||
src/plugin_system/plugin_catalog.py
|
||||
src/plugin_system/plugin_dirs.py
|
||||
src/plugin_system/plugin_executor.py
|
||||
src/plugin_system/plugin_health.py
|
||||
src/plugin_system/plugin_loader.py
|
||||
src/plugin_system/plugin_runtime.py
|
||||
src/plugin_system/plugin_state.py
|
||||
src/plugin_system/repo_urls.py
|
||||
src/plugin_system/resource_monitor.py
|
||||
|
||||
@@ -14,6 +14,14 @@ project_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
if project_dir not in sys.path:
|
||||
sys.path.insert(0, project_dir)
|
||||
|
||||
# Under systemd the watchdog clock is already running, and start-up (plugin
|
||||
# loads, initial updates) takes far longer than the render loop's limit. Widen
|
||||
# it before anything slow is imported; the render loop narrows it again once
|
||||
# its first frame is on the panel. A no-op outside systemd. Standard library
|
||||
# only -- see src/display_watchdog.py.
|
||||
from src import display_watchdog
|
||||
display_watchdog.watchdog.begin_startup()
|
||||
|
||||
# Parse command-line arguments BEFORE any imports
|
||||
parser = argparse.ArgumentParser(description='LEDMatrix Display Controller')
|
||||
parser.add_argument('-e', '--emulator', action='store_true',
|
||||
|
||||
@@ -149,6 +149,11 @@
|
||||
},
|
||||
"description": "Array of display mode names this plugin provides"
|
||||
},
|
||||
"vegas_participation": {
|
||||
"type": "string",
|
||||
"enum": ["scroll", "pause", "exclude"],
|
||||
"description": "How this plugin takes part in Vegas mode by default: 'scroll' (its content scrolls by), 'pause' (the scroll stops for its turn and display() draws it full screen) or 'exclude' (left out). A user's per-plugin vegas_participation setting overrides it. Omit it to derive the participation from get_vegas_display_mode() / get_vegas_content_type(). Cores before 3.8.0 ignore it."
|
||||
},
|
||||
"api_requirements": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
|
||||
@@ -34,11 +34,14 @@ display; **diagnostic** — run by hand on a Pi when something is wrong.
|
||||
| `frame_soak.py` | diagnostic | Soaks a running display and reports how often frames reached the panel late (docs/SCROLL_PERFORMANCE.md) |
|
||||
| `install_dependencies_apt.py` | keep | Dependency installer that tries apt packages first, then pip (installer Step 7, plugin loader) |
|
||||
| `install_plugin_dependencies.sh` | diagnostic | Installs plugin requirements by hand when the automatic install fails |
|
||||
| `plugin_api_usage.py` | dev-only | Scans core, the plugin monorepo and the registry's third-party plugins for callers of every `@deprecated` core method; its output is [docs/DEPRECATIONS_3.8.md](../docs/DEPRECATIONS_3.8.md) |
|
||||
| `prove_security.py` | keep | Security property checks run by pre-commit |
|
||||
| `render_bench.py` | diagnostic | Benchmarks the render loop against the panel's real refresh rate on a synthetic strip |
|
||||
| `render_plugin.py` | dev-only | Runs a plugin's `update()` + `display()` and saves the frame as a PNG |
|
||||
| `reset_web_password.py` | keep | Turns the optional web login off when the password is lost (`sudo python3 scripts/reset_web_password.py`; docs/WEB_INTERFACE_GUIDE.md) |
|
||||
| `run_plugin_tests.py` | dev-only | Discovers and runs plugin test suites |
|
||||
| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
|
||||
| `sports_drift_report.py` | keep | Counts the different bodies of each method across the nine scoreboards in a `ledmatrix-plugins` checkout (report-only CI job; docs/SPORTS_UNIFICATION.md) |
|
||||
| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
|
||||
| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
|
||||
| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
|
||||
|
||||
@@ -0,0 +1,239 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the web UI's Tailwind CSS with the pinned standalone Tailwind CLI.
|
||||
|
||||
The generated files are committed, so the Pi never builds anything. Run this
|
||||
on a dev machine (or let CI run it) after changing a template, a static JS
|
||||
file, or anything under ``web_interface/tailwind/``:
|
||||
|
||||
python3 scripts/build_css.py # rebuild the committed CSS
|
||||
python3 scripts/build_css.py --check # exit 1 if the committed CSS is stale
|
||||
|
||||
No Node or npm: the script downloads Tailwind's standalone CLI (a single
|
||||
executable) for this OS and CPU from the Tailwind GitHub release, checks it
|
||||
against the SHA-256 pinned below, and caches it outside the repo
|
||||
(``$LEDMATRIX_TAILWIND_CACHE``, else the per-user cache directory).
|
||||
|
||||
Outputs (see ``BUILDS``):
|
||||
|
||||
- ``web_interface/static/v3/tailwind.css``: the utilities the templates and
|
||||
static JS use. Linked before ``app.css`` in ``base.html``.
|
||||
- ``web_interface/static/v3/plugin-frame.css``: preflight plus a broad set of
|
||||
common utilities, for plugin ``web_ui/`` fragments served in an iframe.
|
||||
Their markup lives in plugin repos, so it can't be scanned; the safelist in
|
||||
``plugin-frame.config.js`` stands in for it.
|
||||
|
||||
To move to a new Tailwind v3 release, change ``TAILWIND_VERSION`` and every
|
||||
hash in ``TAILWIND_ASSETS`` (the release's ``sha256sums.txt``, or the digests
|
||||
from ``gh api repos/tailwindlabs/tailwindcss/releases/tags/<tag>``), rebuild,
|
||||
and review the diff of the generated CSS.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
TAILWIND_DIR = PROJECT_ROOT / "web_interface" / "tailwind"
|
||||
STATIC_V3 = PROJECT_ROOT / "web_interface" / "static" / "v3"
|
||||
|
||||
TAILWIND_VERSION = "3.4.19"
|
||||
|
||||
# asset name -> SHA-256, from the v3.4.19 release.
|
||||
TAILWIND_ASSETS = {
|
||||
"tailwindcss-linux-arm64": "e5b2d27694daa80cc52ec29553ba2c6bd43d86bd51a9d633ed24058b9c05a676",
|
||||
"tailwindcss-linux-armv7": "e3610b109a64720295e1c00a18dd2d6d79d3cddc618219aa0830de97a55429a4",
|
||||
"tailwindcss-linux-x64": "4af3198c015616ea7d6617974ec3d70d987ecc00c1ca8463b0a30fd65cc7c06e",
|
||||
"tailwindcss-macos-arm64": "7fdeb00818b6214a337383063282b2361ecb08bbc08f8c8a7ba97ee1e2eaa4fe",
|
||||
"tailwindcss-macos-x64": "a597f407e0f1f03535731f5b42f1576a8152cb5fffc2f38e754722bc0c280045",
|
||||
"tailwindcss-windows-arm64.exe": "f2b6b999747aa0ae31999d59db117b1ba1e4e15e17675d7108e30aac4b680686",
|
||||
"tailwindcss-windows-x64.exe": "a15158c4c5e0e7a75f7229bfe4986fe7710d2edc468b6f96c8981f78ab211347",
|
||||
}
|
||||
|
||||
DOWNLOAD_URL = (
|
||||
"https://github.com/tailwindlabs/tailwindcss/releases/download/v{version}/{asset}"
|
||||
)
|
||||
|
||||
# (input CSS, config, output) -- all relative to the project root.
|
||||
BUILDS = (
|
||||
(
|
||||
"web_interface/tailwind/app.input.css",
|
||||
"web_interface/tailwind/tailwind.config.js",
|
||||
"web_interface/static/v3/tailwind.css",
|
||||
),
|
||||
(
|
||||
"web_interface/tailwind/plugin-frame.input.css",
|
||||
"web_interface/tailwind/plugin-frame.config.js",
|
||||
"web_interface/static/v3/plugin-frame.css",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def asset_name() -> str:
|
||||
"""The release asset for this OS and CPU."""
|
||||
system = platform.system()
|
||||
machine = platform.machine().lower()
|
||||
if machine in ("x86_64", "amd64"):
|
||||
arch = "x64"
|
||||
elif machine in ("aarch64", "arm64"):
|
||||
arch = "arm64"
|
||||
elif machine.startswith("armv7") or machine == "armv8l":
|
||||
arch = "armv7"
|
||||
else:
|
||||
raise SystemExit(f"No standalone Tailwind CLI for CPU {machine!r}.")
|
||||
|
||||
if system == "Linux":
|
||||
name = f"tailwindcss-linux-{arch}"
|
||||
elif system == "Darwin":
|
||||
name = f"tailwindcss-macos-{arch}"
|
||||
elif system == "Windows":
|
||||
name = f"tailwindcss-windows-{arch}.exe"
|
||||
else:
|
||||
raise SystemExit(f"No standalone Tailwind CLI for {system!r}.")
|
||||
if name not in TAILWIND_ASSETS:
|
||||
raise SystemExit(f"No standalone Tailwind CLI for {system} {machine}.")
|
||||
return name
|
||||
|
||||
|
||||
def cache_dir() -> Path:
|
||||
override = os.environ.get("LEDMATRIX_TAILWIND_CACHE")
|
||||
if override:
|
||||
return Path(override)
|
||||
if platform.system() == "Windows":
|
||||
base = Path(os.environ.get("LOCALAPPDATA", Path.home() / "AppData" / "Local"))
|
||||
elif platform.system() == "Darwin":
|
||||
base = Path.home() / "Library" / "Caches"
|
||||
else:
|
||||
base = Path(os.environ.get("XDG_CACHE_HOME", Path.home() / ".cache"))
|
||||
return base / "ledmatrix" / "tailwindcss"
|
||||
|
||||
|
||||
def sha256_of(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with open(path, "rb") as fh:
|
||||
for chunk in iter(lambda: fh.read(1 << 20), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def ensure_cli() -> Path:
|
||||
"""Path to the verified CLI, downloading it on first use."""
|
||||
name = asset_name()
|
||||
expected = TAILWIND_ASSETS[name]
|
||||
target = cache_dir() / f"v{TAILWIND_VERSION}" / name
|
||||
|
||||
if target.is_file():
|
||||
if sha256_of(target) == expected:
|
||||
return target
|
||||
print(f"Cached {target} fails its SHA-256 check; downloading it again.")
|
||||
target.unlink()
|
||||
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
url = DOWNLOAD_URL.format(version=TAILWIND_VERSION, asset=name)
|
||||
if not url.startswith("https://"):
|
||||
raise SystemExit(f"Refusing to download the Tailwind CLI over a non-https URL: {url}")
|
||||
print(f"Downloading Tailwind CLI v{TAILWIND_VERSION} ({name})...")
|
||||
fd, tmp_name = tempfile.mkstemp(dir=target.parent, prefix=".download-")
|
||||
tmp = Path(tmp_name)
|
||||
try:
|
||||
with os.fdopen(fd, "wb") as out, urllib.request.urlopen(url, timeout=120) as resp: # nosec B310 - https only, checked above
|
||||
shutil.copyfileobj(resp, out)
|
||||
actual = sha256_of(tmp)
|
||||
if actual != expected:
|
||||
raise SystemExit(
|
||||
f"SHA-256 mismatch for {url}\n expected {expected}\n got {actual}"
|
||||
)
|
||||
tmp.chmod(tmp.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
|
||||
os.replace(tmp, target)
|
||||
finally:
|
||||
if tmp.exists():
|
||||
tmp.unlink()
|
||||
return target
|
||||
|
||||
|
||||
def run_build(
|
||||
cli: Path, input_css: str, config: str, output: Path, work_dir: Path
|
||||
) -> None:
|
||||
# The minifier's rule merging depends on the input file's line endings,
|
||||
# so a Windows checkout (core.autocrlf, CRLF) would build different bytes
|
||||
# than CI's Linux one and --check would fail. Feed the CLI an LF copy.
|
||||
# (Content files' line endings don't matter; @import isn't used, so the
|
||||
# copy's location doesn't either.)
|
||||
lf_input = work_dir / (Path(input_css).name)
|
||||
lf_input.write_bytes(
|
||||
(PROJECT_ROOT / input_css).read_bytes().replace(b"\r\n", b"\n")
|
||||
)
|
||||
cmd = [
|
||||
str(cli),
|
||||
"--input", str(lf_input),
|
||||
"--config", str(PROJECT_ROOT / config),
|
||||
"--output", str(output),
|
||||
"--minify",
|
||||
]
|
||||
# NODE_ENV=production and no browserslist lookup keep the output the
|
||||
# same on every machine.
|
||||
env = dict(os.environ, NODE_ENV="production", BROWSERSLIST_IGNORE_OLD_DATA="1")
|
||||
# The CLI path is computed here (cache dir + pinned asset name) and the
|
||||
# binary was SHA-256-verified by ensure_cli(); env is os.environ plus two
|
||||
# fixed values.
|
||||
result = subprocess.run(cmd, cwd=PROJECT_ROOT, env=env, capture_output=True, text=True) # nosec B603 - list-form argv, no shell # nosemgrep
|
||||
if result.returncode != 0:
|
||||
sys.stderr.write(result.stdout + result.stderr)
|
||||
raise SystemExit(f"Tailwind build failed for {input_css}")
|
||||
# The CLI writes without a trailing newline; add one so the committed
|
||||
# file is a well-formed text file and editors leave it alone.
|
||||
text = output.read_text(encoding="utf-8").replace("\r\n", "\n")
|
||||
if not text.endswith("\n"):
|
||||
text += "\n"
|
||||
output.write_bytes(text.encode("utf-8"))
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
|
||||
parser.add_argument(
|
||||
"--check",
|
||||
action="store_true",
|
||||
help="build to a temp dir and fail if the committed CSS differs",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
cli = ensure_cli()
|
||||
stale = []
|
||||
with tempfile.TemporaryDirectory(prefix="ledmatrix-css-") as tmp:
|
||||
for input_css, config, output in BUILDS:
|
||||
committed = PROJECT_ROOT / output
|
||||
built = Path(tmp) / Path(output).name if args.check else committed
|
||||
run_build(cli, input_css, config, built, Path(tmp))
|
||||
if args.check:
|
||||
old = (
|
||||
committed.read_bytes().replace(b"\r\n", b"\n")
|
||||
if committed.is_file()
|
||||
else None
|
||||
)
|
||||
if old != built.read_bytes():
|
||||
stale.append(output)
|
||||
else:
|
||||
print(f"Wrote {output} ({committed.stat().st_size:,} bytes)")
|
||||
|
||||
if stale:
|
||||
print(
|
||||
"The committed CSS is out of date: " + ", ".join(stale) + "\n"
|
||||
"Run `python3 scripts/build_css.py` and commit the result."
|
||||
)
|
||||
return 1
|
||||
if args.check:
|
||||
print("Committed CSS is up to date.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,778 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Who still calls or overrides the core methods marked ``@deprecated``?
|
||||
|
||||
A deprecated plugin-facing method may only be removed once nothing uses it,
|
||||
and plugins live in other repositories. This script answers the question for
|
||||
every method ``src/deprecation.py``'s decorator marks in core:
|
||||
|
||||
1. it lists the markers by parsing ``src/`` (so the list can never drift from
|
||||
the code);
|
||||
2. it scans, with the ``ast`` module, core itself (``src/``,
|
||||
``web_interface/``, ``scripts/``, the top-level ``*.py``; ``test/``
|
||||
separately), the official monorepo's ``plugins/`` directory, and every
|
||||
third-party plugin the monorepo's ``plugins.json`` lists with its own repo
|
||||
URL (shallow-cloned read-only into a cache directory);
|
||||
3. it reports, per method and per plugin, the calls and overrides it found,
|
||||
and a verdict: unused (safe to remove in the marker's release), still used
|
||||
(keep or migrate those plugins first), or needs review.
|
||||
|
||||
Matching is by method name, so it has to separate real uses from unrelated
|
||||
methods that happen to share the name (the weather plugin's own ``draw_sun``,
|
||||
say). Each hit is classified by what it is attached to:
|
||||
|
||||
* **call** -- ``<receiver>.name`` where the receiver is named like the owning
|
||||
object (``self.cache_manager``, ``display_manager``, ``plugin_manager`` ...,
|
||||
or a local alias assigned from one), or ``self``/``super()`` inside a class
|
||||
that subclasses the owner. Attribute references that are not called
|
||||
(``callback=cm.get_cache_metrics``) count too.
|
||||
* **override** -- ``def name`` in a class that subclasses the owner.
|
||||
* **review** -- ``<receiver>.name`` where the receiver says nothing about its
|
||||
type, or ``getattr(obj, "name")``. Possibly a real use; read the listed line.
|
||||
* **unrelated** -- ``self.name`` inside a class that defines ``name`` itself
|
||||
and does not subclass the owner, ``Klass.name`` where the same tree defines
|
||||
``Klass.name``, or ``def name`` in such a class: a name collision, not a use.
|
||||
* **internal** -- a hit inside the body of another deprecated core method
|
||||
(``draw_rain`` calling ``draw_cloud``): it keeps the method only as long as
|
||||
that caller is kept.
|
||||
|
||||
Only calls and overrides make a method "still used"; review hits make it
|
||||
"needs review"; hits in test files are listed but never block removal (a test
|
||||
that mocks a method does not need it to exist).
|
||||
|
||||
python3 scripts/plugin_api_usage.py # clone everything, print Markdown
|
||||
python3 scripts/plugin_api_usage.py --monorepo ../ledmatrix-plugins
|
||||
python3 scripts/plugin_api_usage.py --output docs/DEPRECATIONS_3.8.md
|
||||
python3 scripts/plugin_api_usage.py --format json
|
||||
|
||||
Nothing is ever written to the repositories it scans: the monorepo path is only
|
||||
read, and clones live in ``--cache-dir``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
import sys
|
||||
import tempfile
|
||||
from collections import defaultdict
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Dict, Iterable, Iterator, List, Optional, Set, Tuple
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
MONOREPO_URL = "https://github.com/ChuckBuilds/ledmatrix-plugins"
|
||||
MONOREPO_SLUG = "chuckbuilds/ledmatrix-plugins"
|
||||
|
||||
#: Receiver names that mean "this is the owning core object". Compared against
|
||||
#: the last name in the receiver (``self.plugin_manager.cache_manager`` ->
|
||||
#: ``cache_manager``), lower-cased with leading underscores stripped.
|
||||
OWNER_RECEIVERS: Dict[str, Tuple[str, ...]] = {
|
||||
"CacheManager": ("cache_manager", "cache_mgr", "cachemanager", "cache", "cm"),
|
||||
"DisplayManager": ("display_manager", "display_mgr", "displaymanager", "display", "dm"),
|
||||
"FontManager": ("font_manager", "font_mgr", "fontmanager", "fonts", "fm"),
|
||||
"PluginManager": ("plugin_manager", "plugin_mgr", "pluginmanager", "pm"),
|
||||
}
|
||||
|
||||
#: Directories never scanned (vendored environments, VCS metadata, caches).
|
||||
SKIP_DIRS = {".git", "__pycache__", "node_modules", ".venv", "venv", "env",
|
||||
"site-packages", ".tox", ".mypy_cache", ".pytest_cache"}
|
||||
|
||||
CORE_DIRS = ("src", "web_interface", "scripts")
|
||||
CORE_TEST_DIRS = ("test",)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Markers
|
||||
|
||||
|
||||
@dataclass
|
||||
class Marker:
|
||||
owner: str # class name, e.g. "CacheManager"
|
||||
method: str
|
||||
removal: str
|
||||
alternative: Optional[str]
|
||||
module: str # e.g. "src.cache_manager"
|
||||
line: int
|
||||
|
||||
@property
|
||||
def key(self) -> str:
|
||||
return f"{self.owner}.{self.method}"
|
||||
|
||||
|
||||
def _decorator_name(node: ast.expr) -> Optional[str]:
|
||||
target = node.func if isinstance(node, ast.Call) else node
|
||||
if isinstance(target, ast.Name):
|
||||
return target.id
|
||||
if isinstance(target, ast.Attribute):
|
||||
return target.attr
|
||||
return None
|
||||
|
||||
|
||||
def find_markers(core_root: Path) -> List[Marker]:
|
||||
"""Every ``@deprecated(...)`` method under ``core_root/src``."""
|
||||
markers: List[Marker] = []
|
||||
for path in sorted((core_root / "src").rglob("*.py")):
|
||||
if path.name == "deprecation.py":
|
||||
continue
|
||||
tree = _parse(path)
|
||||
if tree is None:
|
||||
continue
|
||||
module = ".".join(path.relative_to(core_root).with_suffix("").parts)
|
||||
for cls in (n for n in ast.walk(tree) if isinstance(n, ast.ClassDef)):
|
||||
for fn in cls.body:
|
||||
if not isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
continue
|
||||
for dec in fn.decorator_list:
|
||||
if _decorator_name(dec) != "deprecated" or not isinstance(dec, ast.Call):
|
||||
continue
|
||||
args = [a.value if isinstance(a, ast.Constant) else None for a in dec.args]
|
||||
kw = {k.arg: k.value.value for k in dec.keywords
|
||||
if isinstance(k.value, ast.Constant)}
|
||||
removal = args[0] if args else kw.get("removal")
|
||||
alternative = args[1] if len(args) > 1 else kw.get("alternative")
|
||||
markers.append(Marker(cls.name, fn.name, str(removal), alternative,
|
||||
module, fn.lineno))
|
||||
return markers
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Scanning
|
||||
|
||||
|
||||
@dataclass
|
||||
class Hit:
|
||||
kind: str # call | override | review | unrelated | internal
|
||||
path: str
|
||||
line: int
|
||||
code: str
|
||||
test: bool
|
||||
via: Optional[str] = None # internal: the deprecated core method it sits in
|
||||
|
||||
|
||||
@dataclass
|
||||
class Source:
|
||||
"""One plugin (or core) tree to scan."""
|
||||
name: str
|
||||
group: str # core | core-tests | monorepo | third-party
|
||||
root: Optional[Path]
|
||||
error: Optional[str] = None
|
||||
hits: Dict[str, List[Hit]] = field(default_factory=lambda: defaultdict(list))
|
||||
files: int = 0 # Python files scanned
|
||||
|
||||
|
||||
def _parse(path: Path) -> Optional[ast.AST]:
|
||||
try:
|
||||
return ast.parse(path.read_text(encoding="utf-8", errors="replace"), str(path))
|
||||
except (SyntaxError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _iter_py(root: Path) -> Iterator[Path]:
|
||||
for dirpath, dirnames, filenames in os.walk(root):
|
||||
dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
|
||||
for name in filenames:
|
||||
if name.endswith(".py"):
|
||||
yield Path(dirpath) / name
|
||||
|
||||
|
||||
def _is_test_path(rel: Path) -> bool:
|
||||
parts = [p.lower() for p in rel.parts]
|
||||
return (any(p in ("test", "tests") for p in parts[:-1])
|
||||
or parts[-1].startswith("test_") or parts[-1].endswith("_test.py")
|
||||
or parts[-1] == "conftest.py")
|
||||
|
||||
|
||||
def _terminal(node: ast.expr) -> Optional[str]:
|
||||
"""The last name in a receiver expression, or None if it has none."""
|
||||
if isinstance(node, ast.Name):
|
||||
return node.id
|
||||
if isinstance(node, ast.Attribute):
|
||||
return node.attr
|
||||
if isinstance(node, ast.Call):
|
||||
return _terminal(node.func)
|
||||
if isinstance(node, ast.Subscript):
|
||||
return _terminal(node.value)
|
||||
return None
|
||||
|
||||
|
||||
def _norm(name: Optional[str]) -> str:
|
||||
return (name or "").lstrip("_").lower()
|
||||
|
||||
|
||||
def _base_names(cls: ast.ClassDef) -> List[str]:
|
||||
return [t for t in (_terminal(b) for b in cls.bases) if t]
|
||||
|
||||
|
||||
class _Scanner(ast.NodeVisitor):
|
||||
"""Collect hits for every marked method name in one file."""
|
||||
|
||||
def __init__(self, markers: Dict[str, List[Marker]], lines: List[str],
|
||||
rel: str, test: bool, core_modules: Dict[str, str], module: Optional[str],
|
||||
local_definers: Dict[str, Set[str]], built: Dict[str, str]):
|
||||
self.markers = markers # method name -> markers with that name
|
||||
self.local_definers = local_definers # method name -> this tree's own classes/modules defining it
|
||||
self.built = built # ``x``/``self.x`` -> class it was built from in this file
|
||||
self.lines = lines
|
||||
self.rel = rel
|
||||
self.test = test
|
||||
self.core_modules = core_modules # owner class -> defining module (core only)
|
||||
self.module = module # this file's module when scanning core
|
||||
self.classes: List[ast.ClassDef] = []
|
||||
self.scope: List[ast.AST] = [] # enclosing classes and functions
|
||||
self.aliases: List[Dict[str, str]] = [{}] # local name -> owner class
|
||||
self.out: Dict[str, List[Hit]] = defaultdict(list)
|
||||
|
||||
# -- helpers
|
||||
def _code(self, node: ast.AST) -> str:
|
||||
line = self.lines[node.lineno - 1] if 0 < node.lineno <= len(self.lines) else ""
|
||||
return line.strip()[:160]
|
||||
|
||||
def _add(self, marker: Marker, kind: str, node: ast.AST) -> None:
|
||||
via = self._inside_deprecated()
|
||||
if via and kind != "unrelated":
|
||||
# Only reached through another deprecated method: goes when that does.
|
||||
kind = "internal"
|
||||
self.out[marker.key].append(Hit(kind, self.rel, node.lineno, self._code(node),
|
||||
self.test, via if kind == "internal" else None))
|
||||
|
||||
def _inside_deprecated(self) -> Optional[str]:
|
||||
"""``Owner.method`` when this node sits in a deprecated core method's body."""
|
||||
for i in range(len(self.scope) - 2, -1, -1):
|
||||
cls, fn = self.scope[i], self.scope[i + 1]
|
||||
if isinstance(cls, ast.ClassDef):
|
||||
if isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
for m in self.markers.get(fn.name, ()):
|
||||
if self._is_owner_class(cls, m.owner):
|
||||
return m.key
|
||||
return None
|
||||
return None
|
||||
|
||||
def _owner_for_receiver(self, name: Optional[str]) -> Optional[str]:
|
||||
n = _norm(name)
|
||||
for scope in reversed(self.aliases):
|
||||
if name in scope:
|
||||
return scope[name]
|
||||
for owner, receivers in OWNER_RECEIVERS.items():
|
||||
if n in receivers:
|
||||
return owner
|
||||
return None
|
||||
|
||||
def _is_owner_class(self, cls: ast.ClassDef, owner: str) -> bool:
|
||||
"""True for the real core class (only when scanning its own module)."""
|
||||
return (self.module is not None and cls.name == owner
|
||||
and self.core_modules.get(owner) == self.module)
|
||||
|
||||
def _subclasses(self, cls: ast.ClassDef, owner: str) -> bool:
|
||||
return owner in _base_names(cls)
|
||||
|
||||
def _class_defines(self, cls: ast.ClassDef, name: str) -> bool:
|
||||
return any(isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef)) and n.name == name
|
||||
for n in cls.body)
|
||||
|
||||
# -- scopes
|
||||
def visit_ClassDef(self, node: ast.ClassDef) -> None:
|
||||
for fn in node.body:
|
||||
if isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)) and fn.name in self.markers:
|
||||
for m in self.markers[fn.name]:
|
||||
if self._is_owner_class(node, m.owner):
|
||||
continue # the definition itself
|
||||
kind = "override" if self._subclasses(node, m.owner) else "unrelated"
|
||||
self._add(m, kind, fn)
|
||||
self.classes.append(node)
|
||||
self.scope.append(node)
|
||||
self.generic_visit(node)
|
||||
self.scope.pop()
|
||||
self.classes.pop()
|
||||
|
||||
def _visit_function(self, node) -> None:
|
||||
self.aliases.append({})
|
||||
self.scope.append(node)
|
||||
self.generic_visit(node)
|
||||
self.scope.pop()
|
||||
self.aliases.pop()
|
||||
|
||||
visit_FunctionDef = _visit_function
|
||||
visit_AsyncFunctionDef = _visit_function
|
||||
|
||||
def visit_Assign(self, node: ast.Assign) -> None:
|
||||
# ``dm = self.display_manager`` makes ``dm.draw_sun()`` a call.
|
||||
owner = self._owner_for_receiver(_terminal(node.value))
|
||||
for target in node.targets:
|
||||
if isinstance(target, ast.Name) and owner:
|
||||
self.aliases[-1][target.id] = owner
|
||||
self.generic_visit(node)
|
||||
|
||||
# -- uses
|
||||
def visit_Attribute(self, node: ast.Attribute) -> None:
|
||||
if node.attr in self.markers:
|
||||
for m in self.markers[node.attr]:
|
||||
self._add(m, self._classify(node, m), node)
|
||||
self.generic_visit(node)
|
||||
|
||||
def _classify(self, node: ast.Attribute, m: Marker) -> str:
|
||||
recv = node.value
|
||||
cls = self.classes[-1] if self.classes else None
|
||||
is_self = isinstance(recv, ast.Name) and recv.id in ("self", "cls")
|
||||
is_super = (isinstance(recv, ast.Call) and isinstance(recv.func, ast.Name)
|
||||
and recv.func.id == "super")
|
||||
if is_self or is_super:
|
||||
if cls is not None and (self._is_owner_class(cls, m.owner) or self._subclasses(cls, m.owner)):
|
||||
return "call"
|
||||
if cls is not None and self._class_defines(cls, m.method):
|
||||
return "unrelated"
|
||||
return "review"
|
||||
name = _terminal(recv)
|
||||
if self._owner_for_receiver(name) == m.owner:
|
||||
return "call"
|
||||
definers = self.local_definers.get(m.method, ())
|
||||
if name in definers or self.built.get(name or "") in definers:
|
||||
# e.g. the weather plugin's WeatherIcons.draw_sun, or
|
||||
# self._strategy_component = CacheStrategy(); ...get_sport_live_interval()
|
||||
return "unrelated"
|
||||
return "review"
|
||||
|
||||
def visit_Call(self, node: ast.Call) -> None:
|
||||
func = node.func
|
||||
if (isinstance(func, ast.Name) and func.id in ("getattr", "hasattr", "setattr", "delattr")
|
||||
and len(node.args) >= 2 and isinstance(node.args[1], ast.Constant)
|
||||
and node.args[1].value in self.markers):
|
||||
for m in self.markers[node.args[1].value]:
|
||||
owner = self._owner_for_receiver(_terminal(node.args[0]))
|
||||
self._add(m, "call" if owner == m.owner else "review", node)
|
||||
self.generic_visit(node)
|
||||
|
||||
|
||||
def _built_from(tree: ast.AST) -> Dict[str, str]:
|
||||
"""``{name: Class}`` for every ``name = Class(...)`` / ``self.name = Class(...)``."""
|
||||
built: Dict[str, str] = {}
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Assign) and isinstance(node.value, ast.Call):
|
||||
cls = _terminal(node.value.func)
|
||||
for target in node.targets:
|
||||
name = _terminal(target) if isinstance(target, (ast.Name, ast.Attribute)) else None
|
||||
if name and cls:
|
||||
built[name] = cls
|
||||
return built
|
||||
|
||||
|
||||
def scan_tree(source: Source, roots: Iterable[Path], base: Path, markers: List[Marker],
|
||||
core: bool, test_override: Optional[bool] = None,
|
||||
definer_roots: Iterable[Path] = ()) -> None:
|
||||
by_name: Dict[str, List[Marker]] = defaultdict(list)
|
||||
for m in markers:
|
||||
by_name[m.method].append(m)
|
||||
core_modules = {m.owner: m.module for m in markers}
|
||||
owners = {m.owner for m in markers}
|
||||
files: List[Tuple[Path, str, Optional[ast.AST]]] = []
|
||||
for root in roots:
|
||||
if not root.exists():
|
||||
continue
|
||||
for path in ([root] if root.is_file() else sorted(_iter_py(root))):
|
||||
source.files += 1
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
if any(name in text for name in by_name):
|
||||
files.append((path, text, _parse(path)))
|
||||
|
||||
# Classes (and modules) in this tree with their own method of a marked
|
||||
# name, so ``WeatherIcons.draw_sun()`` is recognised as theirs.
|
||||
local_definers: Dict[str, Set[str]] = defaultdict(set)
|
||||
definer_files = list(files)
|
||||
for root in definer_roots:
|
||||
for path in (sorted(_iter_py(root)) if root.is_dir() else ()):
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
if any(name in text for name in by_name):
|
||||
definer_files.append((path, text, _parse(path)))
|
||||
for path, _, tree in definer_files:
|
||||
for node in (tree.body if tree is not None else ()):
|
||||
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name in by_name:
|
||||
local_definers[node.name].add(path.stem)
|
||||
for cls in (n for n in ast.walk(tree) if isinstance(n, ast.ClassDef)) if tree else ():
|
||||
if cls.name in owners and core:
|
||||
continue
|
||||
if owners & set(_base_names(cls)):
|
||||
continue
|
||||
for fn in cls.body:
|
||||
if isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)) and fn.name in by_name:
|
||||
local_definers[fn.name].add(cls.name)
|
||||
|
||||
for path, text, tree in files:
|
||||
rel = path.relative_to(base)
|
||||
test = _is_test_path(rel) if test_override is None else test_override
|
||||
if tree is None:
|
||||
# Unparseable (Python 2, a template ...): fall back to text, as review.
|
||||
for no, line in enumerate(text.splitlines(), 1):
|
||||
for name in by_name:
|
||||
if re.search(rf"{re.escape(name)}", line):
|
||||
for m in by_name[name]:
|
||||
source.hits[m.key].append(
|
||||
Hit("review", rel.as_posix(), no, line.strip()[:160], test))
|
||||
continue
|
||||
module = ".".join(rel.with_suffix("").parts) if core else None
|
||||
scanner = _Scanner(by_name, text.splitlines(), rel.as_posix(), test,
|
||||
core_modules, module, local_definers, _built_from(tree))
|
||||
scanner.visit(tree)
|
||||
for key, hits in scanner.out.items():
|
||||
source.hits[key].extend(hits)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Fetching plugin trees (read-only)
|
||||
|
||||
|
||||
def _git(*args: str, cwd: Optional[Path] = None) -> subprocess.CompletedProcess:
|
||||
# Never stop to ask for credentials: a deleted or private plugin repo
|
||||
# should be reported as not scanned, not hang the scan.
|
||||
env = {**os.environ, "GIT_TERMINAL_PROMPT": "0"}
|
||||
return subprocess.run( # nosec B603 B607 - list-form git argv, no shell; URLs follow "--" # nosemgrep
|
||||
["git", *args], cwd=cwd, capture_output=True, text=True,
|
||||
encoding="utf-8", errors="replace", timeout=300, env=env)
|
||||
|
||||
|
||||
def _rmtree(path: Path) -> None:
|
||||
"""Delete a clone; git marks pack files read-only, which Windows refuses to delete."""
|
||||
import shutil
|
||||
import stat
|
||||
|
||||
def retry(func, target, _exc):
|
||||
os.chmod(target, stat.S_IWRITE)
|
||||
func(target)
|
||||
|
||||
if sys.version_info >= (3, 12):
|
||||
shutil.rmtree(path, onexc=retry)
|
||||
else:
|
||||
shutil.rmtree(path, onerror=retry)
|
||||
|
||||
|
||||
def shallow_clone(url: str, branch: Optional[str], dest: Path, reuse: bool) -> Optional[str]:
|
||||
"""Clone ``url`` into ``dest`` (depth 1), replacing any earlier clone.
|
||||
|
||||
Returns an error string, or None on success.
|
||||
"""
|
||||
if reuse and (dest / ".git").exists():
|
||||
return None
|
||||
if dest.exists():
|
||||
_rmtree(dest)
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
args = ["-c", "core.longpaths=true", "clone", "--quiet", "--depth", "1"]
|
||||
if branch:
|
||||
args += ["--branch", branch]
|
||||
# "--" ends option parsing: a registry URL starting with "-" (for example
|
||||
# "--upload-pack=...") is then only ever a repository argument.
|
||||
result = _git(*args, "--", url, str(dest))
|
||||
if result.returncode != 0 and branch:
|
||||
result = _git("-c", "core.longpaths=true", "clone", "--quiet", "--depth", "1",
|
||||
"--", url, str(dest))
|
||||
if result.returncode != 0:
|
||||
lines = (result.stderr or result.stdout).strip().splitlines()
|
||||
return lines[-1] if lines else "git clone failed"
|
||||
return None
|
||||
|
||||
|
||||
def _head(path: Path, branch: bool = True) -> str:
|
||||
"""Commit (and branch) of a checkout, via read-only git calls."""
|
||||
rev = _git("--no-optional-locks", "rev-parse", "--short=8", "HEAD", cwd=path)
|
||||
if rev.returncode != 0:
|
||||
return "unknown revision"
|
||||
if not branch:
|
||||
return rev.stdout.strip()
|
||||
ref = _git("--no-optional-locks", "rev-parse", "--abbrev-ref", "HEAD", cwd=path)
|
||||
return f"{ref.stdout.strip()} @ {rev.stdout.strip()}"
|
||||
|
||||
|
||||
def _plugin_id(plugin_dir: Path) -> str:
|
||||
try:
|
||||
return json.loads((plugin_dir / "manifest.json").read_text(encoding="utf-8"))["id"]
|
||||
except (OSError, ValueError, KeyError, TypeError):
|
||||
return plugin_dir.name
|
||||
|
||||
|
||||
def _is_monorepo(url: str) -> bool:
|
||||
return MONOREPO_SLUG in url.lower().rstrip("/").removesuffix(".git")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Report
|
||||
|
||||
|
||||
def verdicts(markers: List[Marker], sources: List[Source]) -> Dict[str, Tuple[str, str]]:
|
||||
"""``{Owner.method: (status, text)}`` across every source.
|
||||
|
||||
An *internal* hit (a call from inside another deprecated method) keeps a
|
||||
method only while that caller is itself kept, so statuses are resolved
|
||||
until they stop changing.
|
||||
"""
|
||||
failed = [s.name for s in sources if s.error]
|
||||
status: Dict[str, Tuple[str, str]] = {}
|
||||
for _ in range(len(markers) + 1):
|
||||
changed = False
|
||||
for m in markers:
|
||||
used, review = [], []
|
||||
for s in sources:
|
||||
live = [h for h in s.hits.get(m.key, []) if not h.test]
|
||||
if any(h.kind in ("call", "override") for h in live) or any(
|
||||
h.kind == "internal" and status.get(h.via, ("",))[0] == "used"
|
||||
for h in live):
|
||||
used.append(s.name)
|
||||
elif any(h.kind == "review" for h in live) or any(
|
||||
h.kind == "internal" and status.get(h.via, ("",))[0] == "review"
|
||||
for h in live):
|
||||
review.append(s.name)
|
||||
if used:
|
||||
new = ("used", f"still used by {', '.join(used)} — keep or migrate first")
|
||||
elif review:
|
||||
new = ("review", f"needs review: possible use in {', '.join(review)}")
|
||||
elif failed:
|
||||
new = ("unknown", f"not proven unused: {len(failed)} plugin(s) could not be scanned")
|
||||
else:
|
||||
new = ("unused", f"unused — safe to remove in {m.removal}")
|
||||
if status.get(m.key) != new:
|
||||
status[m.key] = new
|
||||
changed = True
|
||||
if not changed:
|
||||
break
|
||||
return status
|
||||
|
||||
|
||||
def _counts(hits: List[Hit]) -> Dict[str, int]:
|
||||
c: Dict[str, int] = defaultdict(int)
|
||||
for h in hits:
|
||||
c[("test " if h.test else "") + h.kind] += 1
|
||||
return c
|
||||
|
||||
|
||||
def _usage_cell(marker: Marker, sources: List[Source], kinds: Tuple[str, ...]) -> str:
|
||||
parts = []
|
||||
for s in sources:
|
||||
c = _counts(s.hits.get(marker.key, []))
|
||||
bits = [f"{c[k]} {k}{'s' if c[k] != 1 else ''}" for k in kinds if c[k]]
|
||||
if bits:
|
||||
parts.append(f"{s.name} ({', '.join(bits)})")
|
||||
return "; ".join(parts) or "—"
|
||||
|
||||
|
||||
def render_markdown(markers: List[Marker], sources: List[Source], meta: Dict[str, str]) -> str:
|
||||
out: List[str] = []
|
||||
w = out.append
|
||||
w("# Deprecated plugin APIs: usage scan")
|
||||
w("")
|
||||
w("Generated by `scripts/plugin_api_usage.py` — do not edit by hand; re-run it "
|
||||
"(see [How to re-run](#how-to-re-run)).")
|
||||
w("")
|
||||
w(f"- Scanned: {meta['date']}, core {meta['core_version']}")
|
||||
w(f"- Monorepo: {meta['monorepo']}")
|
||||
w(f"- Third-party plugins: {meta['third_party']}")
|
||||
failed = [s for s in sources if s.error]
|
||||
if failed:
|
||||
w("- **Not scanned:** " + "; ".join(f"{s.name} ({s.error})" for s in failed))
|
||||
w("")
|
||||
status = verdicts(markers, sources)
|
||||
tally: Dict[str, int] = defaultdict(int)
|
||||
for st, _ in status.values():
|
||||
tally[st] += 1
|
||||
w(f"**{len(markers)} deprecated methods: {tally['unused']} unused, "
|
||||
f"{tally['used']} still used, {tally['review']} need review"
|
||||
+ (f", {tally['unknown']} not proven" if tally["unknown"] else "") + ".**")
|
||||
w("")
|
||||
w("Counted per plugin: a *call* is `<receiver>.method` on an object named like "
|
||||
"the owner (`cache_manager`, `display_manager`, `font_manager`, `plugin_manager`), "
|
||||
"or on `self`/`super()` in a subclass; an *override* is `def method` in a subclass "
|
||||
"of the owner. *Review* hits are `.method` on a receiver whose type the scan cannot "
|
||||
"tell. *Internal* hits sit inside another deprecated core method and go with it. "
|
||||
"*Unrelated* hits are a different class's own method with the same name "
|
||||
"(a name collision), and never block removal; neither do hits in test files.")
|
||||
w("")
|
||||
w("| Method | Removal | Core | Plugins (calls / overrides) | Name collisions & tests | Verdict |")
|
||||
w("|---|---|---|---|---|---|")
|
||||
core_sources = [s for s in sources if s.group in ("core", "core-tests")]
|
||||
plugin_sources = [s for s in sources if s.group not in ("core", "core-tests")]
|
||||
for m in markers:
|
||||
core = _usage_cell(m, core_sources, ("call", "override", "review", "internal",
|
||||
"test call", "test override", "test review",
|
||||
"test internal"))
|
||||
plugins = _usage_cell(m, plugin_sources, ("call", "override", "review"))
|
||||
other = _usage_cell(m, plugin_sources, ("unrelated", "test call", "test override",
|
||||
"test review", "test unrelated"))
|
||||
w(f"| `{m.key}` | {m.removal} | {core} | {plugins} | {other} | {status[m.key][1]} |")
|
||||
w("")
|
||||
|
||||
groups = [("unused", "Unused — safe to remove"), ("used", "Still used — keep or migrate first"),
|
||||
("review", "Needs review"), ("unknown", "Not proven unused")]
|
||||
for st, title in groups:
|
||||
names = [m.key for m in markers if status[m.key][0] == st]
|
||||
if names:
|
||||
w(f"## {title} ({len(names)})")
|
||||
w("")
|
||||
w(", ".join(f"`{n}`" for n in names))
|
||||
w("")
|
||||
|
||||
detail = [(m, s, h) for m in markers for s in sources
|
||||
for h in s.hits.get(m.key, []) if h.kind != "unrelated" or not h.test]
|
||||
if detail:
|
||||
w("## Every hit")
|
||||
w("")
|
||||
w("File paths are relative to the plugin's directory (core: the repo root).")
|
||||
w("")
|
||||
w("| Method | Where | File:line | Kind | Code |")
|
||||
w("|---|---|---|---|---|")
|
||||
for m, s, h in detail:
|
||||
kind = ("test " if h.test else "") + h.kind
|
||||
if h.via:
|
||||
kind += f" (in `{h.via}`)"
|
||||
code = h.code.replace("|", "\\|").replace("`", "'")
|
||||
w(f"| `{m.key}` | {s.name} | {h.path}:{h.line} | {kind} | `{code}` |")
|
||||
w("")
|
||||
|
||||
w("## Sources scanned")
|
||||
w("")
|
||||
w("| Source | Group | Python files | Hits |")
|
||||
w("|---|---|---|---|")
|
||||
for s in sources:
|
||||
n = sum(len(v) for v in s.hits.values())
|
||||
files = f"not scanned: {s.error}" if s.error else str(s.files)
|
||||
w(f"| {s.name} | {s.group} | {files} | {n} |")
|
||||
w("")
|
||||
|
||||
w("## How to re-run")
|
||||
w("")
|
||||
w("```bash")
|
||||
w("# Clones the monorepo and each third-party plugin (depth 1) into a temp cache:")
|
||||
w("python3 scripts/plugin_api_usage.py --output docs/DEPRECATIONS_3.8.md")
|
||||
w("# Or scan a local monorepo checkout (read only) instead of cloning it:")
|
||||
w("python3 scripts/plugin_api_usage.py --monorepo ../ledmatrix-plugins")
|
||||
w("```")
|
||||
w("")
|
||||
w("Before removing a method in its release, re-run the scan against the current "
|
||||
"monorepo and registry: a plugin added since this file was generated may have "
|
||||
"started calling it. Remove only methods the fresh scan reports unused; move "
|
||||
"the rest to a later release (the test in `test/test_deprecation.py` fails "
|
||||
"while a marker names a release at or below `src.__version__`).")
|
||||
w("")
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def render_json(markers: List[Marker], sources: List[Source], meta: Dict[str, str]) -> str:
|
||||
data = {"meta": meta, "sources": [{"name": s.name, "group": s.group, "error": s.error}
|
||||
for s in sources], "methods": []}
|
||||
status = verdicts(markers, sources)
|
||||
for m in markers:
|
||||
st, text = status[m.key]
|
||||
data["methods"].append({
|
||||
"method": m.key, "module": m.module, "removal": m.removal,
|
||||
"alternative": m.alternative, "status": st, "verdict": text,
|
||||
"hits": [{"source": s.name, **h.__dict__} for s in sources
|
||||
for h in s.hits.get(m.key, [])],
|
||||
})
|
||||
return json.dumps(data, indent=2)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
|
||||
def main(argv: Optional[List[str]] = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
|
||||
parser.add_argument("--monorepo", type=Path,
|
||||
help="local ledmatrix-plugins checkout to scan (read only); "
|
||||
"default: shallow-clone its main branch")
|
||||
parser.add_argument("--registry", type=Path,
|
||||
help="plugins.json to read third-party plugins from "
|
||||
"(default: the monorepo's)")
|
||||
parser.add_argument("--cache-dir", type=Path,
|
||||
default=Path(tempfile.gettempdir()) / "ledmatrix-plugin-api-usage",
|
||||
help="where clones go (default: %(default)s)")
|
||||
parser.add_argument("--reuse-cache", action="store_true",
|
||||
help="scan clones already in --cache-dir instead of re-cloning "
|
||||
"(offline re-runs; the report may then be stale)")
|
||||
parser.add_argument("--no-third-party", action="store_true",
|
||||
help="skip third-party plugins (the report then cannot prove anything unused)")
|
||||
parser.add_argument("--format", choices=("md", "json"), default="md")
|
||||
parser.add_argument("--output", type=Path, help="write the report here instead of stdout")
|
||||
args = parser.parse_args(argv)
|
||||
if hasattr(sys.stdout, "reconfigure"):
|
||||
sys.stdout.reconfigure(encoding="utf-8")
|
||||
|
||||
markers = find_markers(REPO_ROOT)
|
||||
if not markers:
|
||||
print("No @deprecated markers found in src/.", file=sys.stderr)
|
||||
return 0
|
||||
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
try:
|
||||
from src import __version__ as core_version
|
||||
except Exception: # noqa: BLE001 -- reporting only
|
||||
core_version = "unknown"
|
||||
|
||||
sources: List[Source] = []
|
||||
core = Source("core", "core", REPO_ROOT)
|
||||
scan_tree(core, [REPO_ROOT / d for d in CORE_DIRS] + sorted(REPO_ROOT.glob("*.py")),
|
||||
REPO_ROOT, markers, core=True)
|
||||
core_tests = Source("core tests", "core-tests", REPO_ROOT)
|
||||
scan_tree(core_tests, [REPO_ROOT / d for d in CORE_TEST_DIRS], REPO_ROOT, markers,
|
||||
core=True, test_override=True,
|
||||
definer_roots=[REPO_ROOT / d for d in CORE_DIRS])
|
||||
sources += [core, core_tests]
|
||||
|
||||
# Monorepo
|
||||
if args.monorepo:
|
||||
mono = args.monorepo.resolve()
|
||||
mono_desc = f"local checkout `{mono.name}` ({_head(mono)})"
|
||||
else:
|
||||
mono = args.cache_dir / "ledmatrix-plugins"
|
||||
err = shallow_clone(MONOREPO_URL, "main", mono, args.reuse_cache)
|
||||
if err:
|
||||
print(f"Could not clone the monorepo: {err}", file=sys.stderr)
|
||||
return 1
|
||||
mono_desc = f"[ChuckBuilds/ledmatrix-plugins]({MONOREPO_URL}) ({_head(mono)})"
|
||||
plugins_dir = mono / "plugins"
|
||||
mono_dirs = sorted(p for p in plugins_dir.iterdir() if p.is_dir()) if plugins_dir.is_dir() else []
|
||||
for d in mono_dirs:
|
||||
s = Source(_plugin_id(d), "monorepo", d)
|
||||
scan_tree(s, [d], d, markers, core=False)
|
||||
sources.append(s)
|
||||
mono_desc += f", {len(mono_dirs)} plugins"
|
||||
|
||||
# Third-party plugins from the registry
|
||||
registry = args.registry or (mono / "plugins.json")
|
||||
third: List[dict] = []
|
||||
try:
|
||||
reg = json.loads(registry.read_text(encoding="utf-8"))
|
||||
entries = reg["plugins"] if isinstance(reg, dict) else reg
|
||||
third = [e for e in entries if e.get("repo") and not _is_monorepo(e["repo"])]
|
||||
except (OSError, ValueError, KeyError) as exc:
|
||||
print(f"Could not read {registry}: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
if args.no_third_party:
|
||||
tp_desc = "skipped (--no-third-party)"
|
||||
else:
|
||||
for e in third:
|
||||
dest = args.cache_dir / "third-party" / re.sub(r"[^\w.-]", "_", e["id"])
|
||||
err = shallow_clone(e["repo"], e.get("branch") or None, dest, args.reuse_cache)
|
||||
root = dest / e["plugin_path"] if e.get("plugin_path") else dest
|
||||
s = Source(e["id"], "third-party", root, error=err)
|
||||
if not err:
|
||||
scan_tree(s, [root], root, markers, core=False)
|
||||
sources.append(s)
|
||||
tp_desc = (f"{len(third)} with their own repo in `plugins.json` "
|
||||
f"({', '.join(e['id'] for e in third)})")
|
||||
|
||||
meta = {
|
||||
"date": datetime.now(timezone.utc).strftime("%Y-%m-%d"),
|
||||
"core_version": core_version,
|
||||
"core_rev": _head(REPO_ROOT, branch=False),
|
||||
"monorepo": mono_desc,
|
||||
"third_party": tp_desc,
|
||||
}
|
||||
report = (render_json if args.format == "json" else render_markdown)(markers, sources, meta)
|
||||
if args.output:
|
||||
args.output.write_text(report, encoding="utf-8", newline="\n")
|
||||
print(f"Wrote {args.output}", file=sys.stderr)
|
||||
else:
|
||||
sys.stdout.write(report)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,104 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Turn the web interface's optional login off, for when the password is lost.
|
||||
|
||||
Removes the password (and the key that signs login cookies) from the
|
||||
``web_auth`` section of ``config/config_secrets.json``. The interface is then
|
||||
open again, as it is before a password is ever set, and a new password can be
|
||||
set under General > Security. API tokens are kept unless ``--revoke-tokens``
|
||||
is given. Nothing else in the secrets file is touched, and the web service
|
||||
does not need a restart: it notices the change on the next request.
|
||||
|
||||
Run it on the Pi, from any directory:
|
||||
|
||||
sudo python3 ~/LEDMatrix/scripts/reset_web_password.py
|
||||
|
||||
``sudo`` because the secrets file is not readable by every user. The file
|
||||
keeps its owner and permissions.
|
||||
|
||||
Another way in without the password: open the interface from the Pi itself
|
||||
(http://localhost:5000). Requests from the Pi are never asked to log in.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from src.config_manager_atomic import atomic_write_json # noqa: E402
|
||||
|
||||
SECTION = 'web_auth' # web_interface/auth.py; not imported to keep Flask out
|
||||
LOGIN_KEYS = ('password_hash', 'session_secret', 'password_set_at')
|
||||
|
||||
|
||||
def reset(settings_file: Path, revoke_tokens: bool = False) -> str:
|
||||
"""Clear the login from ``settings_file`` (config_secrets.json).
|
||||
|
||||
Returns what was done. The message names the file and counts tokens; it
|
||||
never includes anything read from the file.
|
||||
"""
|
||||
if not settings_file.exists():
|
||||
return f'{settings_file} does not exist, so no password is set. Nothing to do.'
|
||||
with open(settings_file, 'r', encoding='utf-8') as fh:
|
||||
data = json.load(fh)
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError(f'{settings_file} does not hold a JSON object')
|
||||
|
||||
section = data.get(SECTION)
|
||||
if not isinstance(section, dict):
|
||||
return 'No web login password is set. Nothing to do.'
|
||||
|
||||
had_password = bool(section.get('password_hash'))
|
||||
token_count = len(section.get('tokens') or [])
|
||||
for key in LOGIN_KEYS:
|
||||
section.pop(key, None)
|
||||
if revoke_tokens:
|
||||
section.pop('tokens', None)
|
||||
if section:
|
||||
data[SECTION] = section
|
||||
else:
|
||||
data.pop(SECTION, None)
|
||||
|
||||
if not had_password and not (revoke_tokens and token_count):
|
||||
return 'No web login password is set. Nothing to do.'
|
||||
atomic_write_json(settings_file, data)
|
||||
|
||||
done = []
|
||||
if had_password:
|
||||
done.append('Web login is off: the interface opens without a password. '
|
||||
'Set a new one under General > Security.')
|
||||
if revoke_tokens and token_count:
|
||||
done.append(f'Revoked {token_count} API token(s).')
|
||||
elif token_count:
|
||||
done.append(f'{token_count} API token(s) kept (use --revoke-tokens to remove them).')
|
||||
return ' '.join(done)
|
||||
|
||||
|
||||
def main(argv=None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Turn the LEDMatrix web login off (lost password recovery).')
|
||||
parser.add_argument('--secrets', dest='settings_file', type=Path,
|
||||
default=PROJECT_ROOT / 'config' / 'config_secrets.json',
|
||||
help='secrets file (default: config/config_secrets.json '
|
||||
'in this LEDMatrix checkout)')
|
||||
parser.add_argument('--revoke-tokens', action='store_true',
|
||||
help='also delete every API token')
|
||||
args = parser.parse_args(argv)
|
||||
settings_file = args.settings_file
|
||||
try:
|
||||
outcome = reset(settings_file, revoke_tokens=args.revoke_tokens)
|
||||
except PermissionError:
|
||||
print(f'Permission denied reading or writing {settings_file}. Run it with sudo.',
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
except (OSError, ValueError) as err:
|
||||
print(f'Could not reset the web login: {err}', file=sys.stderr)
|
||||
return 1
|
||||
print(outcome)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,484 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Report how far apart the nine scoreboards' copies of each method are.
|
||||
|
||||
The sports consolidation (docs/SPORTS_UNIFICATION.md) moves shared code from
|
||||
the scoreboard plugins into ``src/common``. Byte-identical copies have mostly
|
||||
been moved; what is left has drifted, and is promoted one *method family* at a
|
||||
time by first making every copy identical ("reconcile, then promote"). This
|
||||
report is the progress measure for that: for every method in the tracked
|
||||
files it counts the copies and the distinct bodies among them, so a stage can
|
||||
say "``_is_game_really_over``: 5 variants -> 1" instead of remembering it.
|
||||
|
||||
It reads a ledmatrix-plugins checkout and never fails a build: it is a report,
|
||||
not a gate. The monorepo's own ``scripts/check_sports_drift.py`` is the gate
|
||||
(it fails when a function that agrees across the plugins starts to differ).
|
||||
|
||||
Definitions
|
||||
-----------
|
||||
family
|
||||
One method name in one tracked file, across every class that defines it
|
||||
and every plugin. ``sports.py::update`` covers ``SportsLive.update``,
|
||||
``SportsRecent.update`` and ``SportsUpcoming.update`` in all nine plugins.
|
||||
Module-level functions are families too.
|
||||
copies
|
||||
How many definitions the family has (plugin x class).
|
||||
plugins
|
||||
How many of the nine plugins define it at least once.
|
||||
variants
|
||||
Distinct bodies among the copies, compared as ASTs with docstrings,
|
||||
comments, formatting, decorators and annotations ignored. A family is
|
||||
reconciled when every class in it is down to one variant.
|
||||
per-class variants
|
||||
The same count within one class role (``SportsLive.update`` across the
|
||||
plugins). Class names are folded the way the plugins name them
|
||||
(``SoccerScoreboardPlugin`` and ``UFCScoreboardPlugin`` are both
|
||||
``SScoreboardPlugin``), so manager.py lines up across sports.
|
||||
folded
|
||||
Variants left after sport and league names are folded to a placeholder
|
||||
(``self.nfl_live`` == ``self.nhl_live``, ``"NFL"`` == ``"NHL"``). The gap
|
||||
between ``variants`` and ``folded`` is drift that is only naming.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python scripts/sports_drift_report.py --plugins ../ledmatrix-plugins
|
||||
python scripts/sports_drift_report.py --markdown # for a CI summary
|
||||
python scripts/sports_drift_report.py --json out.json # machine-readable
|
||||
python scripts/sports_drift_report.py --family sports.py::update
|
||||
|
||||
``--plugins`` defaults to ``$LEDMATRIX_PLUGINS`` (a checkout root or its
|
||||
``plugins/`` directory, the same variable the core parity tests read). With no
|
||||
checkout it says so and exits 0.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import collections
|
||||
import difflib
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Dict, Iterable, List, Optional, Tuple
|
||||
|
||||
#: The nine scoreboards the consolidation covers, by directory prefix.
|
||||
SPORTS = ("afl", "baseball", "basketball", "football", "hockey", "lacrosse",
|
||||
"nrl", "soccer", "ufc")
|
||||
|
||||
#: Files every scoreboard carries a copy of. sports.py and game_renderer.py
|
||||
#: are the consolidation's subject; manager.py (the BasePlugin host, the
|
||||
#: largest copy of all) joined the plan with the reconcile-then-promote
|
||||
#: method. ufc has no game_renderer.py (it draws fights in fight_renderer.py).
|
||||
DEFAULT_FILES = ("sports.py", "manager.py", "game_renderer.py")
|
||||
|
||||
#: A family is "drifted" when it is widespread and has several bodies. The
|
||||
#: defaults match the review that introduced this report (at least 7 plugins,
|
||||
#: at least 3 variants).
|
||||
DEFAULT_MIN_PLUGINS = 7
|
||||
DEFAULT_MIN_VARIANTS = 3
|
||||
|
||||
#: Sport, league and competition names that legitimately differ between the
|
||||
#: plugins. Only used for the ``folded`` column.
|
||||
SPORT_TOKENS = (
|
||||
"afl", "nrl", "baseball", "basketball", "football", "hockey", "soccer",
|
||||
"lacrosse", "ufc", "mma", "mlb", "milb", "nhl", "nfl", "nba", "wnba",
|
||||
"ncaa", "ncaafb", "ncaam", "ncaaw", "ncaa_fb", "ncaa_baseball",
|
||||
"ncaa_basketball", "ncaam_hockey", "ncaaw_hockey", "ncaam_lacrosse",
|
||||
"ncaaw_lacrosse", "ncaam_basketball", "ncaaw_basketball", "epl",
|
||||
"uefa", "mls", "laliga", "bundesliga", "seriea", "ligue1",
|
||||
)
|
||||
_TOKEN_RE = re.compile(
|
||||
r"(?<![A-Za-z0-9])(" + "|".join(sorted(SPORT_TOKENS, key=len, reverse=True))
|
||||
+ r")(?![A-Za-z0-9])", re.IGNORECASE)
|
||||
_TOKEN_SET = {t.lower() for t in SPORT_TOKENS}
|
||||
_CAMEL_RE = re.compile(r"[A-Z]+(?![a-z])|[A-Z][a-z0-9]*|[a-z0-9]+|_")
|
||||
|
||||
|
||||
def fold(name: str) -> str:
|
||||
"""Replace sport and league names in an identifier or string with ``S``.
|
||||
|
||||
Both spellings the plugins use: snake_case (``nfl_live`` -> ``S_live``)
|
||||
and CamelCase (``UFCScoreboardPlugin`` -> ``SScoreboardPlugin``).
|
||||
"""
|
||||
name = _TOKEN_RE.sub("S", name)
|
||||
# sub, not findall + join: characters between words (spaces, dots,
|
||||
# braces in a log string) must survive, or distinct text folds together.
|
||||
return _CAMEL_RE.sub(
|
||||
lambda m: "S" if m.group(0).lower() in _TOKEN_SET else m.group(0), name)
|
||||
|
||||
|
||||
def _strip_docstring(body: List[ast.stmt]) -> List[ast.stmt]:
|
||||
if (body and isinstance(body[0], ast.Expr)
|
||||
and isinstance(body[0].value, ast.Constant)
|
||||
and isinstance(body[0].value.value, str)):
|
||||
return body[1:] or [ast.Pass()]
|
||||
return body
|
||||
|
||||
|
||||
class _Canonical(ast.NodeTransformer):
|
||||
"""Drop what is not behaviour: docstrings, decorators, annotations."""
|
||||
|
||||
def _func(self, node):
|
||||
self.generic_visit(node)
|
||||
node.body = _strip_docstring(node.body)
|
||||
node.decorator_list = []
|
||||
node.returns = None
|
||||
return node
|
||||
|
||||
visit_FunctionDef = _func
|
||||
visit_AsyncFunctionDef = _func
|
||||
|
||||
def visit_ClassDef(self, node):
|
||||
self.generic_visit(node)
|
||||
node.body = _strip_docstring(node.body)
|
||||
return node
|
||||
|
||||
def visit_arg(self, node):
|
||||
node.annotation = None
|
||||
return node
|
||||
|
||||
|
||||
class _Folded(_Canonical):
|
||||
"""Canonical, plus sport names folded out of identifiers and strings."""
|
||||
|
||||
def visit_Name(self, node):
|
||||
node.id = fold(node.id)
|
||||
return node
|
||||
|
||||
def visit_Attribute(self, node):
|
||||
self.generic_visit(node)
|
||||
node.attr = fold(node.attr)
|
||||
return node
|
||||
|
||||
def visit_arg(self, node):
|
||||
node = super().visit_arg(node)
|
||||
node.arg = fold(node.arg)
|
||||
return node
|
||||
|
||||
def visit_keyword(self, node):
|
||||
self.generic_visit(node)
|
||||
if node.arg:
|
||||
node.arg = fold(node.arg)
|
||||
return node
|
||||
|
||||
def visit_Constant(self, node):
|
||||
if isinstance(node.value, str):
|
||||
node.value = fold(node.value)
|
||||
return node
|
||||
|
||||
def _func(self, node):
|
||||
node = super()._func(node)
|
||||
node.name = fold(node.name)
|
||||
return node
|
||||
|
||||
visit_FunctionDef = _func
|
||||
visit_AsyncFunctionDef = _func
|
||||
|
||||
|
||||
def _digest(node: ast.AST, transformer: ast.NodeTransformer) -> str:
|
||||
# Re-parse a copy so the transformers never mutate the tree being walked.
|
||||
clone = ast.parse(ast.unparse(node)).body[0]
|
||||
clone = transformer.visit(clone)
|
||||
# The function's own name is the family key, not part of its body.
|
||||
if isinstance(clone, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
clone.name = "_"
|
||||
return hashlib.sha256(ast.dump(clone).encode()).hexdigest()[:12]
|
||||
|
||||
|
||||
class Copy:
|
||||
"""One definition of a method (or module-level function) in one plugin."""
|
||||
|
||||
__slots__ = ("plugin", "cls", "name", "lines", "exact", "folded", "source")
|
||||
|
||||
def __init__(self, plugin, cls, name, lines, exact, folded, source=""):
|
||||
self.plugin = plugin
|
||||
self.cls = cls
|
||||
self.name = name
|
||||
self.lines = lines
|
||||
self.exact = exact
|
||||
self.folded = folded
|
||||
self.source = source
|
||||
|
||||
|
||||
def collect_file(path: Path, plugin: str) -> List[Copy]:
|
||||
"""Every top-level function and class method in one file."""
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
try:
|
||||
tree = ast.parse(text)
|
||||
except SyntaxError as exc:
|
||||
print(f" ! {path}: {exc}", file=sys.stderr)
|
||||
return []
|
||||
out = []
|
||||
|
||||
def add(node, cls):
|
||||
out.append(Copy(plugin, cls, node.name,
|
||||
node.end_lineno - node.lineno + 1,
|
||||
_digest(node, _Canonical()), _digest(node, _Folded()),
|
||||
ast.get_source_segment(text, node) or ""))
|
||||
|
||||
for node in tree.body:
|
||||
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
add(node, "<module>")
|
||||
elif isinstance(node, ast.ClassDef):
|
||||
for child in node.body:
|
||||
if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
add(child, fold(node.name))
|
||||
return out
|
||||
|
||||
|
||||
def resolve_plugins_dir(raw: Optional[str]) -> Optional[Path]:
|
||||
"""A checkout root or its plugins/ directory; None when there is neither."""
|
||||
if not raw:
|
||||
return None
|
||||
root = Path(raw)
|
||||
if (root / "plugins").is_dir():
|
||||
root = root / "plugins"
|
||||
if not any((root / f"{s}-scoreboard").is_dir() for s in SPORTS):
|
||||
return None
|
||||
return root
|
||||
|
||||
|
||||
def build(plugins_dir: Path, files: Iterable[str]) -> Dict[Tuple[str, str], List[Copy]]:
|
||||
"""{(file, method name): [Copy, ...]} across the nine scoreboards."""
|
||||
families: Dict[Tuple[str, str], List[Copy]] = collections.defaultdict(list)
|
||||
for fname in files:
|
||||
for sport in SPORTS:
|
||||
path = plugins_dir / f"{sport}-scoreboard" / fname
|
||||
if path.is_file():
|
||||
for copy in collect_file(path, sport):
|
||||
families[(fname, copy.name)].append(copy)
|
||||
return families
|
||||
|
||||
|
||||
def summarise(key: Tuple[str, str], copies: List[Copy]) -> dict:
|
||||
"""The numbers for one family."""
|
||||
per_class = collections.defaultdict(list)
|
||||
for c in copies:
|
||||
per_class[c.cls].append(c)
|
||||
classes = []
|
||||
for cls, members in sorted(per_class.items()):
|
||||
groups = collections.defaultdict(list)
|
||||
for c in members:
|
||||
groups[c.exact].append(c.plugin)
|
||||
classes.append({
|
||||
"class": cls,
|
||||
"copies": len(members),
|
||||
"variants": len(groups),
|
||||
"folded": len({c.folded for c in members}),
|
||||
"groups": sorted((sorted(p) for p in groups.values()),
|
||||
key=lambda g: (-len(g), g)),
|
||||
})
|
||||
total_lines = sum(c.lines for c in copies)
|
||||
# What promotion would remove: every copy but one per class role.
|
||||
one_each = sum(max(c.lines for c in members) for members in per_class.values())
|
||||
return {
|
||||
"file": key[0],
|
||||
"family": key[1],
|
||||
"plugins": len({c.plugin for c in copies}),
|
||||
"copies": len(copies),
|
||||
"variants": len({(c.cls, c.exact) for c in copies}),
|
||||
"folded": len({(c.cls, c.folded) for c in copies}),
|
||||
"worst_class_variants": max(k["variants"] for k in classes),
|
||||
"lines": total_lines,
|
||||
"duplicated_lines": total_lines - one_each,
|
||||
"classes": classes,
|
||||
}
|
||||
|
||||
|
||||
def report(families, min_plugins: int, min_variants: int) -> dict:
|
||||
rows = [summarise(k, v) for k, v in families.items()]
|
||||
by_file = collections.defaultdict(list)
|
||||
for r in rows:
|
||||
by_file[r["file"]].append(r)
|
||||
files = {}
|
||||
for fname, frows in sorted(by_file.items()):
|
||||
files[fname] = {
|
||||
"families": len(frows),
|
||||
"in_all_plugins": sum(1 for r in frows if r["plugins"] == len(SPORTS)),
|
||||
"lines": sum(r["lines"] for r in frows),
|
||||
"identical_duplicated_lines": sum(
|
||||
r["duplicated_lines"] for r in frows if r["worst_class_variants"] == 1),
|
||||
}
|
||||
drifted = sorted(
|
||||
(r for r in rows
|
||||
if r["plugins"] >= min_plugins and r["variants"] >= min_variants),
|
||||
key=lambda r: (-r["variants"], -r["lines"], r["file"], r["family"]))
|
||||
identical = sorted(
|
||||
(r for r in rows if r["copies"] >= 2 and r["worst_class_variants"] == 1),
|
||||
key=lambda r: (-r["duplicated_lines"], r["file"], r["family"]))
|
||||
# One body shared by every plugin but one: the cheapest reconciliations.
|
||||
# "One" across the whole family: a class role whose odd one out is a
|
||||
# different plugin from another role's is two outliers, not one.
|
||||
one_outlier = sorted(
|
||||
(r for r in rows
|
||||
if r["plugins"] >= min_plugins and r["worst_class_variants"] == 2
|
||||
and all(len(k["groups"]) < 2 or len(k["groups"][1]) == 1
|
||||
for k in r["classes"])
|
||||
and len(_minorities(r)) == 1),
|
||||
key=lambda r: (-r["duplicated_lines"], r["file"], r["family"]))
|
||||
return {"files": files, "drifted": drifted, "identical": identical,
|
||||
"one_outlier": one_outlier,
|
||||
"rows": rows, "thresholds": {"min_plugins": min_plugins,
|
||||
"min_variants": min_variants}}
|
||||
|
||||
|
||||
def _minorities(r) -> set:
|
||||
"""Every plugin in a minority body, across the family's class roles."""
|
||||
return {p for k in r["classes"] for g in k["groups"][1:] for p in g}
|
||||
|
||||
|
||||
def _outlier(r) -> str:
|
||||
"""The plugin whose body differs, for a one-outlier family."""
|
||||
return ", ".join(sorted(_minorities(r)))
|
||||
|
||||
|
||||
def _text(rep, top_identical: int) -> str:
|
||||
out = []
|
||||
out.append("Per file (all methods and module functions):")
|
||||
for fname, f in rep["files"].items():
|
||||
out.append(f" {fname:<18} {f['families']:>4} families, "
|
||||
f"{f['in_all_plugins']:>3} in all {len(SPORTS)} plugins, "
|
||||
f"{f['lines']:>6} lines; identical copies beyond the first: "
|
||||
f"{f['identical_duplicated_lines']} lines")
|
||||
t = rep["thresholds"]
|
||||
out.append("")
|
||||
out.append(f"Drifted families (in >= {t['min_plugins']} plugins, "
|
||||
f">= {t['min_variants']} variants): {len(rep['drifted'])}")
|
||||
out.append(f" {'file::family':<58} {'plug':>4} {'copies':>6} {'var':>4} "
|
||||
f"{'fold':>4} {'worst':>5} {'lines':>6}")
|
||||
for r in rep["drifted"]:
|
||||
name = f"{r['file']}::{r['family']}"
|
||||
out.append(f" {name:<58} {r['plugins']:>4} {r['copies']:>6} "
|
||||
f"{r['variants']:>4} {r['folded']:>4} "
|
||||
f"{r['worst_class_variants']:>5} {r['lines']:>6}")
|
||||
out.append("")
|
||||
out.append(f"One outlier (in >= {t['min_plugins']} plugins, every plugin but "
|
||||
f"one agrees): {len(rep['one_outlier'])}")
|
||||
for r in rep["one_outlier"]:
|
||||
name = f"{r['file']}::{r['family']}"
|
||||
out.append(f" {name:<58} {r['plugins']:>4} plugins, differs in "
|
||||
f"{_outlier(r)}; {r['lines']} lines")
|
||||
out.append("")
|
||||
out.append(f"Identical in every copy (promote as-is), top {top_identical} "
|
||||
f"by duplicated lines, of {len(rep['identical'])}:")
|
||||
for r in rep["identical"][:top_identical]:
|
||||
name = f"{r['file']}::{r['family']}"
|
||||
out.append(f" {name:<58} {r['plugins']:>4} plugins "
|
||||
f"{r['duplicated_lines']:>5} duplicated lines")
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def _markdown(rep, top_identical: int, source: str) -> str:
|
||||
t = rep["thresholds"]
|
||||
out = ["## Sports drift report", "",
|
||||
f"Scoreboard copies read from `{source}`. Report only: this never fails "
|
||||
"the build. See docs/SPORTS_UNIFICATION.md.", "",
|
||||
"| File | Families | In all 9 | Lines | Identical duplicated lines |",
|
||||
"|---|---:|---:|---:|---:|"]
|
||||
for fname, f in rep["files"].items():
|
||||
out.append(f"| `{fname}` | {f['families']} | {f['in_all_plugins']} | "
|
||||
f"{f['lines']} | {f['identical_duplicated_lines']} |")
|
||||
out += ["", f"### Drifted families (in >= {t['min_plugins']} plugins, "
|
||||
f">= {t['min_variants']} variants): {len(rep['drifted'])}", "",
|
||||
"| Family | Plugins | Copies | Variants | Folded | Worst class | Lines |",
|
||||
"|---|---:|---:|---:|---:|---:|---:|"]
|
||||
for r in rep["drifted"]:
|
||||
out.append(f"| `{r['file']}::{r['family']}` | {r['plugins']} | {r['copies']} | "
|
||||
f"{r['variants']} | {r['folded']} | {r['worst_class_variants']} | "
|
||||
f"{r['lines']} |")
|
||||
out += ["", f"### One outlier (every plugin but one agrees): "
|
||||
f"{len(rep['one_outlier'])}", "",
|
||||
"| Family | Plugins | Differs in | Lines |", "|---|---:|---|---:|"]
|
||||
for r in rep["one_outlier"]:
|
||||
out.append(f"| `{r['file']}::{r['family']}` | {r['plugins']} | "
|
||||
f"{_outlier(r)} | {r['lines']} |")
|
||||
out += ["", f"### Identical in every copy: {len(rep['identical'])} "
|
||||
f"(top {top_identical} by duplicated lines)", "",
|
||||
"| Family | Plugins | Duplicated lines |", "|---|---:|---:|"]
|
||||
for r in rep["identical"][:top_identical]:
|
||||
out.append(f"| `{r['file']}::{r['family']}` | {r['plugins']} | "
|
||||
f"{r['duplicated_lines']} |")
|
||||
return "\n".join(out) + "\n"
|
||||
|
||||
|
||||
def _family_detail(rep, families, wanted: str, show_diff: bool) -> str:
|
||||
"""Which plugins share each body of one family; optionally the diffs.
|
||||
|
||||
The diff is against the body most plugins share (the first group), which
|
||||
is where a reconciliation usually starts.
|
||||
"""
|
||||
fname, _, family = wanted.partition("::")
|
||||
for r in rep["rows"]:
|
||||
if r["file"] == fname and r["family"] == family:
|
||||
out = [f"{wanted}: {r['plugins']} plugins, {r['copies']} copies, "
|
||||
f"{r['variants']} variants ({r['folded']} after folding sport "
|
||||
f"names), {r['lines']} lines"]
|
||||
copies = families[(fname, family)]
|
||||
for k in r["classes"]:
|
||||
out.append(f" {k['class']}: {k['variants']} variant(s) "
|
||||
f"({k['folded']} folded)")
|
||||
for g in k["groups"]:
|
||||
out.append(f" {', '.join(g)}")
|
||||
if not show_diff or len(k["groups"]) < 2:
|
||||
continue
|
||||
by_plugin = {c.plugin: c for c in copies if c.cls == k["class"]}
|
||||
base = by_plugin[k["groups"][0][0]]
|
||||
for g in k["groups"][1:]:
|
||||
other = by_plugin[g[0]]
|
||||
out.extend(difflib.unified_diff(
|
||||
base.source.splitlines(), other.source.splitlines(),
|
||||
f"{base.plugin}-scoreboard/{fname}",
|
||||
f"{other.plugin}-scoreboard/{fname}", lineterm="", n=2))
|
||||
return "\n".join(out)
|
||||
return f"{wanted}: no such family"
|
||||
|
||||
|
||||
def main(argv: Optional[List[str]] = None) -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
|
||||
ap.add_argument("--plugins", default=os.environ.get("LEDMATRIX_PLUGINS"),
|
||||
help="ledmatrix-plugins checkout (default: $LEDMATRIX_PLUGINS)")
|
||||
ap.add_argument("--files", default=",".join(DEFAULT_FILES),
|
||||
help="comma-separated files to compare (default: %(default)s)")
|
||||
ap.add_argument("--min-plugins", type=int, default=DEFAULT_MIN_PLUGINS)
|
||||
ap.add_argument("--min-variants", type=int, default=DEFAULT_MIN_VARIANTS)
|
||||
ap.add_argument("--top-identical", type=int, default=15)
|
||||
ap.add_argument("--markdown", action="store_true",
|
||||
help="print a Markdown summary (for $GITHUB_STEP_SUMMARY)")
|
||||
ap.add_argument("--json", metavar="PATH",
|
||||
help="also write the full report as JSON")
|
||||
ap.add_argument("--family", action="append", default=[],
|
||||
help="show which plugins share each body, e.g. sports.py::update")
|
||||
ap.add_argument("--diff", action="store_true",
|
||||
help="with --family, also diff each variant against the most common one")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
plugins_dir = resolve_plugins_dir(args.plugins)
|
||||
if plugins_dir is None:
|
||||
msg = ("No ledmatrix-plugins checkout: pass --plugins or set "
|
||||
"LEDMATRIX_PLUGINS. Nothing to report.")
|
||||
print(f"_{msg}_\n" if args.markdown else msg)
|
||||
return 0
|
||||
|
||||
files = [f.strip() for f in args.files.split(",") if f.strip()]
|
||||
families = build(plugins_dir, files)
|
||||
rep = report(families, args.min_plugins, args.min_variants)
|
||||
|
||||
if args.json:
|
||||
with open(args.json, "w", encoding="utf-8") as fh:
|
||||
json.dump(rep, fh, indent=2)
|
||||
fh.write("\n")
|
||||
if args.markdown:
|
||||
print(_markdown(rep, args.top_identical, str(plugins_dir)), end="")
|
||||
else:
|
||||
print(_text(rep, args.top_identical))
|
||||
for wanted in args.family:
|
||||
print()
|
||||
print(_family_detail(rep, families, wanted, args.diff))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -14,12 +14,20 @@ the same reason: the rollback cannot depend on packages the update changed.
|
||||
The updater leaves data/auto_update_pending.json:
|
||||
|
||||
{"status": "pending", "old_head": ..., "new_head": ...,
|
||||
"old_ref": "main" | "" (detached) | absent (older updaters),
|
||||
"display_was_active": bool, "dependency_failures": [...], "created_at": ...}
|
||||
|
||||
This moves its status to "verifying" and then to one of "success",
|
||||
"rolled_back" or "rollback_failed", with "reason" and "detail" saying why.
|
||||
The web interface reports that outcome and raises a banner for anything but
|
||||
success.
|
||||
|
||||
"The display service is active" does not mean the panel is drawing: a render
|
||||
loop stuck inside a plugin leaves the service active and the panel frozen.
|
||||
Where the display writes a heartbeat (/run/ledmatrix, see
|
||||
src/display_watchdog.py), the display also has to keep it fresh, from the
|
||||
restarted process, to count as healthy. Where it never wrote one -- the code
|
||||
being updated predates it -- the check is what it always was.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
@@ -36,6 +44,14 @@ from pathlib import Path
|
||||
PENDING_NAME = 'auto_update_pending.json'
|
||||
REQUIREMENT_FILES = ('requirements.txt', 'web_interface/requirements.txt')
|
||||
WEB_HEALTH_URL = 'http://127.0.0.1:5000/api/v3/system/version'
|
||||
#: Written by the display's render loop every few seconds. A copy of
|
||||
#: src/display_watchdog.HEARTBEAT_PATH, not an import: this file runs as a
|
||||
#: copy made before the update and must not depend on the code it checks.
|
||||
HEARTBEAT_PATH = '/run/ledmatrix/display-heartbeat.json'
|
||||
#: How old the heartbeat may be. Well under STABLE_SECONDS: a display that
|
||||
#: draws its first frame and then freezes must go stale inside the window it
|
||||
#: has to stay healthy for, or the check would pass it.
|
||||
HEARTBEAT_FRESH_SECONDS = 30
|
||||
#: How long the services get to come up after a restart...
|
||||
HEALTH_TIMEOUT_SECONDS = 180
|
||||
#: ...and how long they must then stay up. Restart=on-failure makes a crash
|
||||
@@ -113,16 +129,35 @@ def _short(sha):
|
||||
return (sha or 'unknown')[:7]
|
||||
|
||||
|
||||
def _read_heartbeat(path=HEARTBEAT_PATH):
|
||||
"""The display's heartbeat, or None when there is none (or it is unreadable)."""
|
||||
try:
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return data if isinstance(data, dict) else None
|
||||
|
||||
|
||||
class Verifier:
|
||||
def __init__(self, project_root, run=subprocess.run, sleep=time.sleep,
|
||||
clock=time.monotonic, web_responds=_web_responds, log=None):
|
||||
clock=time.monotonic, web_responds=_web_responds, log=None,
|
||||
read_heartbeat=_read_heartbeat):
|
||||
self.project_root = Path(project_root)
|
||||
self.pending_file = pending_path(project_root)
|
||||
self.run = run
|
||||
self.sleep = sleep
|
||||
# Monotonic, and compared with the heartbeat's own monotonic stamp:
|
||||
# CLOCK_MONOTONIC is one clock for every process on the machine.
|
||||
self.clock = clock
|
||||
self.web_responds = web_responds
|
||||
self.log = log or (lambda msg: print(f'[auto-update-verify] {msg}', flush=True))
|
||||
self.read_heartbeat = read_heartbeat
|
||||
#: Whether the display was writing a heartbeat before the update.
|
||||
self.expect_heartbeat = False
|
||||
#: When the display was last restarted; an older heartbeat is the
|
||||
#: previous process's, not proof the new one draws.
|
||||
self.display_restarted_at = None
|
||||
|
||||
def _run(self, args, timeout=GIT_TIMEOUT_SECONDS):
|
||||
try:
|
||||
@@ -154,18 +189,32 @@ class Verifier:
|
||||
ok = True
|
||||
# A display the user had stopped stays stopped.
|
||||
if display:
|
||||
self.display_restarted_at = self.clock()
|
||||
ok = self.restart('ledmatrix') and ok
|
||||
return self.restart('ledmatrix-web') and ok
|
||||
|
||||
def display_drawing(self):
|
||||
"""True while the restarted display keeps its heartbeat fresh."""
|
||||
data = self.read_heartbeat()
|
||||
mono = data.get('mono') if data else None
|
||||
if not isinstance(mono, (int, float)) or isinstance(mono, bool):
|
||||
return False
|
||||
if self.display_restarted_at is not None and mono < self.display_restarted_at:
|
||||
return False # still the process from before the restart
|
||||
return self.clock() - mono <= HEARTBEAT_FRESH_SECONDS
|
||||
|
||||
def wait_healthy(self, display):
|
||||
"""None once the services are up and stay up, else what went wrong."""
|
||||
deadline = self.clock() + HEALTH_TIMEOUT_SECONDS + STABLE_SECONDS
|
||||
healthy_since = baseline = None
|
||||
web = disp = False
|
||||
active = drawing = True
|
||||
count_known = True
|
||||
while self.clock() < deadline:
|
||||
web = self.web_responds()
|
||||
disp = self.service_active('ledmatrix') if display else True
|
||||
active = self.service_active('ledmatrix') if display else True
|
||||
drawing = self.display_drawing() if (display and self.expect_heartbeat) else True
|
||||
disp = active and drawing
|
||||
restarts = self.restart_count('ledmatrix') if display else None
|
||||
# Without a restart count a crash loop looks healthy between
|
||||
# attempts, so an unreadable count never counts as stable.
|
||||
@@ -181,8 +230,11 @@ class Verifier:
|
||||
problems = []
|
||||
if not web:
|
||||
problems.append('the web interface did not respond')
|
||||
if not disp:
|
||||
if not active:
|
||||
problems.append('the display service did not stay running')
|
||||
elif not drawing:
|
||||
problems.append('the display service is running but its panel is not '
|
||||
'being drawn (no fresh heartbeat)')
|
||||
if web and disp and not count_known:
|
||||
problems.append("the display service's restart count could not be read")
|
||||
return '; '.join(problems) or 'the display service kept restarting'
|
||||
@@ -225,6 +277,20 @@ class Verifier:
|
||||
if not old:
|
||||
return False, 'the commit to roll back to is unknown'
|
||||
requirements = self.changed_requirements(old, new) if new else list(REQUIREMENT_FILES)
|
||||
# An update may have moved HEAD between main and a detached release
|
||||
# tag (the stable/beta channels). Go back to where HEAD was -- the
|
||||
# branch, or detached -- before resetting, or resetting would drag
|
||||
# the wrong ref: main onto a release commit, or leave a device that
|
||||
# was following main stuck on a detached one. No old_ref (an older
|
||||
# updater wrote this file) means HEAD never moved between refs.
|
||||
old_ref = pending.get('old_ref')
|
||||
if old_ref is not None:
|
||||
move = (['git', 'checkout', '--quiet', '--force', old_ref] if old_ref
|
||||
else ['git', 'checkout', '--quiet', '--force', '--detach', old])
|
||||
result = self._run(move, timeout=GIT_RESET_TIMEOUT_SECONDS)
|
||||
if result.returncode != 0:
|
||||
return False, (f'"{" ".join(move)}" failed: '
|
||||
f'{(result.stderr or result.stdout or "").strip()}')
|
||||
# --hard: the updater refuses to run with local edits to tracked core
|
||||
# files (web_interface/auto_update.local_changes), so outside the
|
||||
# plugin folders the only thing this discards is the update. Edits
|
||||
@@ -258,6 +324,10 @@ class Verifier:
|
||||
write_pending(self.pending_file, pending)
|
||||
|
||||
display = bool(pending.get('display_was_active'))
|
||||
# Read before anything restarts: the display still running is the
|
||||
# pre-update code, and whether it writes a heartbeat decides whether
|
||||
# the updated one must.
|
||||
self.expect_heartbeat = display and self.read_heartbeat() is not None
|
||||
dependency_failures = pending.get('dependency_failures') or []
|
||||
if dependency_failures:
|
||||
# Never restart onto code whose packages did not install.
|
||||
|
||||
+1
-1
@@ -4,5 +4,5 @@ LEDMatrix Display System
|
||||
Core source package for the LED Matrix Display project.
|
||||
"""
|
||||
|
||||
__version__ = "3.6.1"
|
||||
__version__ = "3.7.0"
|
||||
|
||||
|
||||
+32
-35
@@ -88,7 +88,6 @@ _WIFI_REL = Path("config/wifi_config.json")
|
||||
_YTM_REL = Path("config/ytm_auth.json")
|
||||
_FONTS_REL = Path("assets/fonts")
|
||||
_PLUGIN_UPLOADS_REL = Path("assets/plugins")
|
||||
_STATE_REL = Path("data/plugin_state.json")
|
||||
|
||||
#: The sections that are one file each: (section name, path, the
|
||||
#: RestoreOptions flag that restores it). create, preview, validate and
|
||||
@@ -179,20 +178,27 @@ def _build_manifest(contents: List[str]) -> Dict[str, Any]:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _plugins_directory(project_root: Path) -> Path:
|
||||
"""The plugin install directory: ``plugin_system.plugins_directory`` from
|
||||
config/config.json (relative to ``project_root`` unless absolute), or
|
||||
``plugin-repos`` when the config does not say or cannot be read."""
|
||||
configured: Any = None
|
||||
def _read_config(project_root: Path) -> Dict[str, Any]:
|
||||
"""config/config.json as a dict; empty when missing or unreadable."""
|
||||
try:
|
||||
with (project_root / _CONFIG_REL).open("r", encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
if isinstance(config, dict):
|
||||
plugin_system = config.get("plugin_system")
|
||||
if isinstance(plugin_system, dict):
|
||||
configured = plugin_system.get("plugins_directory")
|
||||
except (OSError, json.JSONDecodeError):
|
||||
pass
|
||||
return {}
|
||||
return config if isinstance(config, dict) else {}
|
||||
|
||||
|
||||
def _plugins_directory(project_root: Path,
|
||||
config: Optional[Dict[str, Any]] = None) -> Path:
|
||||
"""The plugin install directory: ``plugin_system.plugins_directory`` from
|
||||
config/config.json (relative to ``project_root`` unless absolute), or
|
||||
``plugin-repos`` when the config does not say or cannot be read."""
|
||||
if config is None:
|
||||
config = _read_config(project_root)
|
||||
configured: Any = None
|
||||
plugin_system = config.get("plugin_system")
|
||||
if isinstance(plugin_system, dict):
|
||||
configured = plugin_system.get("plugins_directory")
|
||||
if not isinstance(configured, str) or not configured.strip():
|
||||
configured = "plugin-repos"
|
||||
path = Path(configured)
|
||||
@@ -202,33 +208,23 @@ def _plugins_directory(project_root: Path) -> Path:
|
||||
def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Return a list of currently-installed plugins suitable for the backup
|
||||
manifest. Each entry has ``plugin_id`` and ``version``.
|
||||
manifest. Each entry has ``plugin_id``, ``version`` and ``enabled``.
|
||||
|
||||
Reads ``data/plugin_state.json`` if present, then adds any plugin it
|
||||
does not list from the ``manifest.json`` files in the configured plugin
|
||||
directory (see :func:`_plugins_directory`).
|
||||
The plugins are the ``manifest.json`` files in the configured plugin
|
||||
directory (see :func:`_plugins_directory`), with the manifest's version;
|
||||
``enabled`` is config.json's flag by the display's rule (a missing flag
|
||||
is disabled). A restore reinstalls every listed plugin and takes enabled
|
||||
state from the restored config.json, so ``enabled`` is informational.
|
||||
|
||||
``data/plugin_state.json`` is not read: it only ever repeated config's
|
||||
enabled flags and the manifests' versions, and is retired (nothing
|
||||
writes it any more). An old backup that listed a plugin only from that
|
||||
file still restores it, since restore reads ``plugins.json`` as written.
|
||||
"""
|
||||
plugins: Dict[str, Dict[str, Any]] = {}
|
||||
config = _read_config(project_root)
|
||||
|
||||
state_file = project_root / _STATE_REL
|
||||
if state_file.exists():
|
||||
try:
|
||||
with state_file.open("r", encoding="utf-8") as f:
|
||||
state = json.load(f)
|
||||
raw_plugins = state.get("states", {}) if isinstance(state, dict) else {}
|
||||
if isinstance(raw_plugins, dict):
|
||||
for plugin_id, info in raw_plugins.items():
|
||||
if not isinstance(info, dict):
|
||||
continue
|
||||
plugins[plugin_id] = {
|
||||
"plugin_id": plugin_id,
|
||||
"version": info.get("version") or "",
|
||||
"enabled": bool(info.get("enabled", True)),
|
||||
}
|
||||
except (OSError, json.JSONDecodeError) as e:
|
||||
logger.warning("Could not read plugin_state.json: %s", e)
|
||||
|
||||
plugins_root = _plugins_directory(project_root)
|
||||
plugins_root = _plugins_directory(project_root, config)
|
||||
if plugins_root.exists():
|
||||
for entry in sorted(plugins_root.iterdir()):
|
||||
if not entry.is_dir():
|
||||
@@ -247,10 +243,11 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
|
||||
continue
|
||||
plugin_id = data.get("id") or entry.name
|
||||
if plugin_id not in plugins:
|
||||
section = config.get(plugin_id)
|
||||
plugins[plugin_id] = {
|
||||
"plugin_id": plugin_id,
|
||||
"version": data.get("version", ""),
|
||||
"enabled": True,
|
||||
"enabled": isinstance(section, dict) and bool(section.get("enabled", False)),
|
||||
}
|
||||
|
||||
return sorted(plugins.values(), key=lambda p: p["plugin_id"])
|
||||
|
||||
+16
-14
@@ -408,7 +408,7 @@ class CacheManager:
|
||||
"""Get the cache directory path."""
|
||||
return self.cache_dir
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def has_data_changed(self, data_type: str, new_data: Dict[str, Any]) -> bool:
|
||||
"""Check if data has changed from cached version."""
|
||||
cached_data = self.load_cache(data_type)
|
||||
@@ -514,7 +514,7 @@ class CacheManager:
|
||||
"""Check if the US stock market is currently open."""
|
||||
return self._strategy_component.is_market_open()
|
||||
|
||||
@deprecated("3.7.0", "use set()")
|
||||
@deprecated("3.8.0", "use set()")
|
||||
def update_cache(self, data_type: str, data: Dict[str, Any]) -> bool:
|
||||
"""Update cache with new data."""
|
||||
cache_data = {
|
||||
@@ -564,7 +564,7 @@ class CacheManager:
|
||||
cache_data['data'] = data
|
||||
self.save_cache(key, cache_data)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def setup_persistent_cache(self) -> bool:
|
||||
"""
|
||||
Set up a persistent cache directory with proper permissions.
|
||||
@@ -776,7 +776,7 @@ class CacheManager:
|
||||
else:
|
||||
self.logger.info("Disk cache cleanup thread stopped successfully")
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_sport_live_interval(self, sport_key: str) -> int:
|
||||
"""
|
||||
Get the live_update_interval for a specific sport from config.
|
||||
@@ -798,7 +798,7 @@ class CacheManager:
|
||||
"""
|
||||
return self._strategy_component.get_data_type_from_key(key)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_sport_key_from_cache_key(self, key: str) -> Optional[str]:
|
||||
"""
|
||||
Extract sport key from cache key to determine appropriate live_update_interval.
|
||||
@@ -838,7 +838,7 @@ class CacheManager:
|
||||
data_type = self.get_data_type_from_key(key)
|
||||
return self.get_cached_data_with_strategy(key, data_type)
|
||||
|
||||
@deprecated("3.7.0", "use get()")
|
||||
@deprecated("3.8.0", "use get()")
|
||||
def get_background_cached_data(self, key: str, sport_key: Optional[str] = None) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Get data from background service cache with appropriate strategy.
|
||||
@@ -876,7 +876,7 @@ class CacheManager:
|
||||
self.record_cache_miss('background')
|
||||
return None
|
||||
|
||||
@deprecated("3.7.0", "use get()")
|
||||
@deprecated("3.8.0", "use get()")
|
||||
def is_background_data_available(self, key: str, sport_key: Optional[str] = None) -> bool:
|
||||
"""
|
||||
Check if background service has fresh data available.
|
||||
@@ -906,32 +906,32 @@ class CacheManager:
|
||||
date_str = datetime.now(pytz.utc).strftime('%Y%m%d')
|
||||
return f"{sport}_{date_str}"
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def record_cache_hit(self, cache_type: str = 'regular') -> None:
|
||||
"""Record a cache hit for performance monitoring."""
|
||||
self._metrics_component.record_hit(cache_type)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def record_cache_miss(self, cache_type: str = 'regular') -> None:
|
||||
"""Record a cache miss for performance monitoring."""
|
||||
self._metrics_component.record_miss(cache_type)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def record_fetch_time(self, duration: float) -> None:
|
||||
"""Record fetch operation duration for performance monitoring."""
|
||||
self._metrics_component.record_fetch_time(duration)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_cache_metrics(self) -> Dict[str, Any]:
|
||||
"""Get current cache performance metrics."""
|
||||
return self._metrics_component.get_metrics()
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def log_cache_metrics(self) -> None:
|
||||
"""Log current cache performance metrics."""
|
||||
self._metrics_component.log_metrics()
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_memory_cache_stats(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get statistics about the memory cache.
|
||||
@@ -943,7 +943,9 @@ class CacheManager:
|
||||
|
||||
def log_memory_cache_stats(self) -> None:
|
||||
"""Log current memory cache statistics."""
|
||||
stats = self.get_memory_cache_stats()
|
||||
# Not get_memory_cache_stats(): that is deprecated, and core must not
|
||||
# trip its own deprecation warning every time memory logging runs.
|
||||
stats = self._memory_cache_component.get_stats()
|
||||
self.logger.info(f"Memory Cache - Size: {stats['size']}/{stats['max_size']} "
|
||||
f"({stats['usage_percent']:.1f}%), "
|
||||
f"Last cleanup: {time.time() - stats['last_cleanup']:.1f}s ago")
|
||||
+31
-1
@@ -38,6 +38,9 @@ Rules for the package:
|
||||
| [`scroll_helper`](#scroll_helper) | Pre-rendered horizontal scrolling | Yes | — |
|
||||
| [`snapshot_policy`](#snapshot_policy) | When to write the web preview frame | No, core-internal | n/a |
|
||||
| [`sports_card`](#sports_card) | Scoreboard card settings, colours, fonts, dates | Yes (scoreboards) | 3.3.0 |
|
||||
| [`sports_card_wrappers`](#sports_card_wrappers) | The game renderer's `sports_card` delegations | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_celebration`](#sports_celebration) | Draw a scoreboard's score/win celebration | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_fetch`](#sports_fetch) | Scoreboard season fetch, lookback and live-odds decisions | Yes (scoreboards) | 3.7.0 |
|
||||
| [`sports_game_renderer`](#sports_game_renderer) | Scoreboard scroll/Vegas card geometry | Yes (scoreboards) | 3.3.0 |
|
||||
| [`sports_helpers`](#sports_helpers) | Small helpers every scoreboard `sports.py` copies | Yes (scoreboards) | 3.5.0 |
|
||||
| [`sports_scroll`](#sports_scroll) | Scoreboard scroll-display orchestration | Yes (scoreboards) | 3.2.0 |
|
||||
@@ -46,7 +49,7 @@ Rules for the package:
|
||||
| [`sync_manager`](#sync_manager) | Leader/follower sync between two displays | No, core-internal | n/a |
|
||||
| [`text_helper`](#text_helper) | Outlined text, wrapping, measurement | Yes | — |
|
||||
|
||||
The four `sports_*` mixin and card modules hold code the scoreboard plugins
|
||||
The `sports_*` mixin and card modules hold code the scoreboard plugins
|
||||
used to carry as identical copies. Each module docstring lists what a host
|
||||
class must provide. The plan behind them is in
|
||||
[docs/SPORTS_UNIFICATION.md](../../docs/SPORTS_UNIFICATION.md).
|
||||
@@ -216,6 +219,33 @@ dates (`format_game_date()`, `format_game_time()`, `card_tzinfo()`) and font
|
||||
sizes (`schema_font_size()`, `resolve_font_size()`). A plugin keeps its own
|
||||
method and delegates the body.
|
||||
|
||||
### sports_card_wrappers
|
||||
|
||||
[`sports_card_wrappers.py`](sports_card_wrappers.py).
|
||||
`SportsCardWrappersMixin`: the one-line methods a scoreboard's game renderer
|
||||
uses to call `sports_card` with its own `config` and `logger`
|
||||
(`_vs_text()`, `_element_color()`, `_format_game_date()`, ... seventeen in
|
||||
all), under their existing names. They are what `sports_game_renderer`'s
|
||||
mixin expects its host to provide. No `__init__` and no state.
|
||||
|
||||
### sports_celebration
|
||||
|
||||
[`sports_celebration.py`](sports_celebration.py). `SportsCelebrationMixin`
|
||||
draws the full-screen takeover a scoreboard shows when a team scores or wins
|
||||
(`_draw_celebration_layout(celebration)`): a backdrop in the scoring team's
|
||||
colours read off its crest, scenery, confetti, the headline and the score.
|
||||
The colour helpers are free functions (`logo_palette()`, `lift_color()`,
|
||||
`mix_color()`, ...). Deciding *when* to celebrate stays in the plugin, which
|
||||
builds the celebration dict the docstring describes.
|
||||
|
||||
### sports_fetch
|
||||
|
||||
[`sports_fetch.py`](sports_fetch.py). `SportsFetchMixin`: the `SportsCore`
|
||||
methods that decide which requests a scoreboard makes --
|
||||
`_fetch_season_directly()` (a season, in chunks ESPN accepts),
|
||||
`_background_fetches_espn_ranges()`, `_needs_previous_day()` (the live
|
||||
lookback) and `_wants_live_odds()` (odds only for games near the screen).
|
||||
|
||||
### sports_game_renderer
|
||||
|
||||
[`sports_game_renderer.py`](sports_game_renderer.py).
|
||||
|
||||
@@ -218,6 +218,8 @@ class FavoriteTeamCheck:
|
||||
return None # Nothing published either way; draw no conclusion.
|
||||
if cls._moved_to_later_phase(payload):
|
||||
return None # e.g. postseason under way; see the method.
|
||||
if cls._later_round_scheduled(payload, now):
|
||||
return None # e.g. Europa League between matchdays.
|
||||
return ("the season has finished and the next one's fixtures are "
|
||||
"not published yet")
|
||||
|
||||
@@ -259,6 +261,44 @@ class FavoriteTeamCheck:
|
||||
known = [t for t in event_types if isinstance(t, int)]
|
||||
return bool(known) and all(t < league_type for t in known)
|
||||
|
||||
@classmethod
|
||||
def _later_round_scheduled(cls, payload, now: datetime) -> bool:
|
||||
"""
|
||||
Whether a "list" calendar has a round that has not started yet.
|
||||
|
||||
Competitions with a list calendar (the UEFA club competitions, the
|
||||
World Cup, AFL, NFL) give each phase its rounds as ``entries`` with
|
||||
start and end dates. Between matchdays the Europa League scoreboard
|
||||
keeps showing the last one: on 2026-09-29 every event was from 17
|
||||
September, the next matchday was only days away, and the rounds from
|
||||
the knockout play-offs to the final were all still to come. A round
|
||||
that starts later means the season is not over, even though the
|
||||
date of the next fixture is not known.
|
||||
|
||||
Only a round's *start* counts. End dates are padded well past the
|
||||
last game -- the World Cup's final round ran to 1 August for a 19 July
|
||||
final -- so a future end date is also true of a finished season.
|
||||
Rounds in an offseason phase (the college football All-Star week)
|
||||
are not games for the favourites and do not count either.
|
||||
"""
|
||||
league = (payload.get('leagues') or [{}])[0] or {}
|
||||
for phase in league.get('calendar') or []:
|
||||
if not isinstance(phase, dict) or cls._is_offseason(phase.get('label')):
|
||||
continue
|
||||
for entry in phase.get('entries') or []:
|
||||
if not isinstance(entry, dict) or cls._is_offseason(entry.get('label')):
|
||||
continue
|
||||
start = cls._parse_date(entry.get('startDate'))
|
||||
if start and start > now:
|
||||
return True
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _is_offseason(label) -> bool:
|
||||
"""'Off Season', 'Offseason', 'Off-season' ..."""
|
||||
return isinstance(label, str) and 'offseason' in re.sub(
|
||||
r'[^a-z]', '', label.lower())
|
||||
|
||||
@staticmethod
|
||||
def _parse_date(raw) -> Optional[datetime]:
|
||||
if not raw or not isinstance(raw, str):
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
"""The ``sports_card`` delegations every scoreboard's game renderer carries.
|
||||
|
||||
After the card helpers moved to ``sports_card`` (3.3.0), each of the eight
|
||||
scoreboards with a ``game_renderer.py`` -- afl, baseball, basketball,
|
||||
football, hockey, lacrosse, nrl and soccer -- kept one-line methods that
|
||||
forward to them with its own ``config`` and ``logger``. Seventeen are
|
||||
identical in all eight (executable AST, docstrings stripped) or in all but
|
||||
football, and were copied here from ledmatrix-plugins ``30455671``
|
||||
(origin/main, 2026-09-29) under their existing names. Football's own
|
||||
``_format_game_date`` and ``_upcoming_center_mode`` (they follow the
|
||||
switch-mode settings when it draws the full-screen scorebug) stay in football
|
||||
and override these.
|
||||
|
||||
``_schema_font_size`` and ``_resolve_font_size`` look the same in every copy
|
||||
but are not moved: they read ``_SCHEMA_PATH``, a module global that is each
|
||||
plugin's own ``config_schema.json``.
|
||||
|
||||
These are the methods ``SportsGameRendererMixin`` (``sports_game_renderer``)
|
||||
lists among what its host must provide, so a renderer that inherits both no
|
||||
longer has to write them. Like that mixin this has no ``__init__`` and no
|
||||
state. It is a separate module rather than more methods there for the reason
|
||||
``sports_helpers`` gives: a missing module fails at load, where the version
|
||||
checks see it; a missing method fails mid-render.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_card_wrappers.py`` fails if a read is added
|
||||
without being listed here.
|
||||
|
||||
- ``config`` and ``logger``.
|
||||
- ``fonts``, read with ``getattr`` -- ``_font_color``.
|
||||
- ``_FONT_NAME_ALIASES`` and ``_FONT_PIXEL_GRID`` class attributes --
|
||||
``_crisp_size``, which passes them to ``sports_card.crisp_size`` so a
|
||||
renderer that declares extra faces keeps them.
|
||||
|
||||
Add it as a base of the plugin's renderer, e.g.
|
||||
``class GameRenderer(SportsCardWrappersMixin, SportsGameRendererMixin)``.
|
||||
The two define no name in common; a method on the plugin's own class still
|
||||
wins over either.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Any, ClassVar, Dict, Optional, Tuple
|
||||
|
||||
from src.common import sports_card as _card
|
||||
|
||||
|
||||
class SportsCardWrappersMixin:
|
||||
"""The game renderer's ``sports_card`` delegations. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only: these create no
|
||||
# attributes, so the host's own values are what the methods read.
|
||||
config: Dict[str, Any]
|
||||
logger: logging.Logger
|
||||
_FONT_NAME_ALIASES: ClassVar[Dict[str, str]]
|
||||
_FONT_PIXEL_GRID: ClassVar[Dict[str, Any]]
|
||||
|
||||
# ---- fonts ---------------------------------------------------------
|
||||
|
||||
@classmethod
|
||||
def _crisp_size(cls, font_file, desired):
|
||||
"""``sports_card.crisp_size`` with this renderer's font tables."""
|
||||
return _card.crisp_size(font_file, desired,
|
||||
cls._FONT_NAME_ALIASES, cls._FONT_PIXEL_GRID)
|
||||
|
||||
def _unshare_element_fonts(self, fonts):
|
||||
"""``sports_card.unshare_element_fonts``."""
|
||||
return _card.unshare_element_fonts(self.logger, fonts)
|
||||
|
||||
def _font_color(self, font, default: Tuple[int, int, int] = (255, 255, 255)):
|
||||
"""``sports_card.font_color`` for one of ``self.fonts``."""
|
||||
return _card.font_color(self.config, getattr(self, "fonts", None), font, default)
|
||||
|
||||
# ---- colours and favourites ---------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _coerce_rgb(value, fallback):
|
||||
"""``sports_card.coerce_rgb``."""
|
||||
return _card.coerce_rgb(value, fallback)
|
||||
|
||||
@staticmethod
|
||||
def _side_is_favorite(game: Dict[str, Any], side: str, favorites: set) -> bool:
|
||||
"""``sports_card.side_is_favorite``."""
|
||||
return _card.side_is_favorite(game, side, favorites)
|
||||
|
||||
@staticmethod
|
||||
def _side_score(game: Dict[str, Any], side: str) -> Optional[int]:
|
||||
"""``sports_card.side_score``."""
|
||||
return _card.side_score(game, side)
|
||||
|
||||
def _favorite_result(self, game: Dict[str, Any]) -> Optional[str]:
|
||||
"""``sports_card.favorite_result``."""
|
||||
return _card.favorite_result(self.config, game)
|
||||
|
||||
def _score_color_for(self, game: Dict[str, Any], game_type: str, default=None):
|
||||
"""``sports_card.score_color_for``."""
|
||||
return _card.score_color_for(self.config, self.logger, game, game_type, default)
|
||||
|
||||
def _recent_score_color(self, game: Dict[str, Any], default):
|
||||
"""``sports_card.recent_score_color``."""
|
||||
return _card.recent_score_color(self.config, self.logger, game, default)
|
||||
|
||||
def _element_color(self, element: str, default: Tuple[int, int, int] = (255, 255, 255)):
|
||||
"""``sports_card.element_color``."""
|
||||
return _card.element_color(self.config, element, default)
|
||||
|
||||
# ---- card options, dates and times --------------------------------
|
||||
|
||||
def _scroll_card_option(self, key: str, default: Any = None) -> Any:
|
||||
"""``sports_card.scroll_card_option``."""
|
||||
return _card.scroll_card_option(self.config, key, default)
|
||||
|
||||
def _upcoming_center_mode(self) -> str:
|
||||
"""``sports_card.upcoming_center_mode``."""
|
||||
return _card.upcoming_center_mode(self.config)
|
||||
|
||||
def _vs_text(self) -> str:
|
||||
"""``sports_card.vs_text``."""
|
||||
return _card.vs_text(self.config)
|
||||
|
||||
def _format_game_date(self, date_text: str, game: Optional[Dict] = None) -> str:
|
||||
"""``sports_card.format_game_date``."""
|
||||
return _card.format_game_date(self.config, self.logger, date_text, game)
|
||||
|
||||
def _weekday_for(self, game: Optional[Dict]) -> str:
|
||||
"""``sports_card.weekday_for``."""
|
||||
return _card.weekday_for(self.config, self.logger, game)
|
||||
|
||||
def _card_tzinfo(self):
|
||||
"""``sports_card.card_tzinfo``."""
|
||||
return _card.card_tzinfo(self.config, self.logger)
|
||||
|
||||
def _format_game_time(self, time_text: str) -> str:
|
||||
"""``sports_card.format_game_time``."""
|
||||
return _card.format_game_time(self.config, time_text)
|
||||
@@ -0,0 +1,780 @@
|
||||
"""How the scoreboards draw a score or win celebration.
|
||||
|
||||
Five scoreboards -- afl, football, hockey, nrl and soccer -- take over the
|
||||
panel when a team scores or wins: a backdrop in the scoring team's colours
|
||||
read off its crest, scenery for the kind of score, confetti, the headline and
|
||||
the score with the scoring side's digits breathing. The drawing is identical
|
||||
in all five ``sports.py`` copies (executable AST, docstrings stripped), and
|
||||
so are the colour helpers it uses; they were copied here from
|
||||
ledmatrix-plugins ``30455671`` (origin/main, 2026-09-29).
|
||||
|
||||
Only the drawing moved. What *arms* a celebration stays in each plugin,
|
||||
because it differs: which scores count (``_check_for_goal`` /
|
||||
``_check_for_score``, and nrl matches favourites by team id), the phrase and
|
||||
the scenery (``_start_celebration``), and when a win fires
|
||||
(``_check_for_win``). So does ``display()``, which decides whether the
|
||||
takeover or the scorebug is on screen. A plugin hands this mixin a
|
||||
celebration dict and it draws it.
|
||||
|
||||
The colour helpers are public free functions here (``logo_palette``,
|
||||
``lift_color``, ``mix_color``, ...); in the plugins they were the same
|
||||
functions with a leading underscore.
|
||||
|
||||
THE CELEBRATION DICT
|
||||
--------------------
|
||||
Built by the plugin's ``_start_celebration``. Read here: ``game`` (a
|
||||
view-model dict; ``<side>_id``, ``<side>_abbr``, ``<side>_logo_path`` and
|
||||
``<side>_logo_url`` for the crests, ``id`` for the confetti seed),
|
||||
``scored_side`` (``"away"`` or ``"home"``), ``away_score``, ``home_score``,
|
||||
``phrase``, ``started_at`` (a ``time.time()`` value) and ``motif``
|
||||
(``"score"``, ``"kick"``, ``"touchdown"``, ``"net"`` or ``"win"``; anything
|
||||
else draws the ``"score"`` diagonals). The drawing caches what it derives in
|
||||
the same dict, under ``_palette``, ``_backdrop``, ``_confetti`` and
|
||||
``_crests``, so each is worked out once per celebration.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_celebration.py`` fails if a read is added without
|
||||
being listed here. All five scoreboards' ``SportsLive`` provide them.
|
||||
|
||||
- ``display_manager`` -- ``image`` is replaced with the frame, then
|
||||
``update_display()``; ``clear()`` on ``force_clear``. Its ``matrix``
|
||||
width and height are used when it has a matrix, else ``display_width`` /
|
||||
``display_height``.
|
||||
- ``fonts`` -- ``"time"`` and ``"status"`` for the headline (the first that
|
||||
fits), ``"score"`` for the score.
|
||||
- ``logger``.
|
||||
- ``_load_and_resize_logo(team_id, abbr, logo_path, logo_url)`` -- a crest
|
||||
as an RGBA image, or ``None``.
|
||||
- ``_draw_text_with_outline(draw, text, position, font, fill=...)`` -- on
|
||||
``SportsCoreSharedMixin``.
|
||||
- ``celebration_duration``, ``celebration_team_colors`` and
|
||||
``celebration_confetti``, read with ``getattr`` (defaults 8, on, on).
|
||||
|
||||
Mix it in ahead of the mode classes, e.g.
|
||||
``class SportsLive(SportsCelebrationMixin, SportsLiveSharedMixin,
|
||||
SportsCore)``. It defines nothing any of them define, so the order only
|
||||
matters for a plugin that keeps its own copy of one of these methods: a
|
||||
method on the plugin's class always wins over the mixin's.
|
||||
"""
|
||||
|
||||
import colorsys
|
||||
import logging
|
||||
import math
|
||||
import random
|
||||
import time
|
||||
from typing import Any, Callable, ClassVar, Dict, List, Optional, Sequence, Tuple
|
||||
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
#: A colour as the helpers return it: three 0-255 channels.
|
||||
Color = Tuple[int, ...]
|
||||
#: ``deep``, ``glow``, ``headline`` and ``accent``; see ``logo_palette``.
|
||||
Palette = Dict[str, Color]
|
||||
#: One confetti flake: column, start height, fall speed, sway phase, size
|
||||
#: in pixels, colour.
|
||||
Flake = Tuple[float, float, float, float, int, Color]
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Colour helpers for the score/win celebration
|
||||
#
|
||||
# Module level rather than methods: they are pure, which is what makes the
|
||||
# palette testable without standing up a live manager, and they are shared by
|
||||
# the takeover's backdrop, confetti and text.
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
#: The crest is sampled at this resolution. Big enough that a secondary
|
||||
#: colour survives (a helmet stripe, a trim), small enough that the whole
|
||||
#: sample is ~1600 pixels of pure-Python work, once per team.
|
||||
_PALETTE_SAMPLE_PX = 40
|
||||
#: Above this, a colour carries team identity; below it, it is a grey.
|
||||
_PALETTE_VIVID_SATURATION = 0.22
|
||||
#: Ignore pixels this dark -- crest outlines, drop shadows, anti-aliasing.
|
||||
_PALETTE_MIN_CHANNEL = 24
|
||||
#: How far apart two bins must be to count as a second, different colour.
|
||||
_PALETTE_DISTINCT_DISTANCE = 90.0
|
||||
#: Never bleed a lifted colour below this saturation; past it a hue stops
|
||||
#: being the team's colour and starts being a pastel.
|
||||
_PALETTE_MIN_SATURATION = 0.42
|
||||
#: Lift a headline colour until it is at least this luminous. Chosen so
|
||||
#: midnight navy reaches a blue that reads at 6px on a panel without
|
||||
#: becoming a different colour.
|
||||
_PALETTE_HEADLINE_LUMINANCE = 112.0
|
||||
#: A crest colour this luminous already reads on a panel, so it is preferred
|
||||
#: over a darker one that would have to be lifted to get there. Lifting is a
|
||||
#: compromise -- Green Bay's dark green only reaches legibility as a teal --
|
||||
#: and most teams whose primary is dark carry a bright second colour that is
|
||||
#: just as much theirs. This is what picks the Packers' gold over that teal.
|
||||
_PALETTE_LEGIBLE_LUMINANCE = 90.0
|
||||
#: ...but only from a colour the crest actually means. The pixels where a
|
||||
#: bright edge is anti-aliased into a dark fill are luminous too, and there is
|
||||
#: always a band of them: Kansas City's white-on-red outline leaves a pink at
|
||||
#: luminance 90 that would otherwise be preferred over the red itself. A blend
|
||||
#: is a mix, so it is markedly less saturated than either colour it sits
|
||||
#: between -- that pink is 0.48 where the red is 0.96 and the Packers' gold,
|
||||
#: which this must keep, is 0.89.
|
||||
_PALETTE_LEGIBLE_SATURATION = 0.65
|
||||
#: And it has to be a band of the crest, not a speck of one.
|
||||
_PALETTE_LEGIBLE_AREA = 0.02
|
||||
#: Cap the backdrop's luminance so the headline stays legible over it,
|
||||
#: and the scenery's so it stays behind the headline. Both are luminance and
|
||||
#: not HSV value on purpose: a silver crest -- the Raiders, or the grey
|
||||
#: placeholder a failed logo download leaves behind -- has a value of ~0.95,
|
||||
#: and capping that at 0.34 still yields a light grey card that white text
|
||||
#: then vanishes into. Scaling the channels down is also hue-exact, which is
|
||||
#: what lets this be the plain arithmetic that lifting a colour cannot be.
|
||||
_PALETTE_BACKDROP_LUMINANCE = 34.0
|
||||
_PALETTE_SCENERY_LUMINANCE = 70.0
|
||||
|
||||
|
||||
def rgb_luminance(color: Sequence[float]) -> float:
|
||||
"""Rec. 709 relative luminance, 0-255."""
|
||||
return 0.2126 * color[0] + 0.7152 * color[1] + 0.0722 * color[2]
|
||||
|
||||
|
||||
def rgb_saturation(color: Sequence[float]) -> float:
|
||||
"""HSV saturation, 0-1."""
|
||||
high = max(color)
|
||||
return (high - min(color)) / high if high else 0.0
|
||||
|
||||
|
||||
def color_distance(a: Sequence[float], b: Sequence[float]) -> float:
|
||||
"""Euclidean distance between two colours in RGB."""
|
||||
return math.sqrt(sum((x - y) ** 2 for x, y in zip(a, b)))
|
||||
|
||||
|
||||
def mix_color(a: Sequence[float], b: Sequence[float], t: float) -> Color:
|
||||
"""Blend ``a`` towards ``b``; t=0 is all a, t=1 is all b."""
|
||||
t = min(max(t, 0.0), 1.0)
|
||||
return tuple(int(round(a[i] + (b[i] - a[i]) * t)) for i in range(3))
|
||||
|
||||
|
||||
def scale_color(color: Sequence[float], factor: float) -> Color:
|
||||
"""Scale a colour's brightness, clamped to the panel's range."""
|
||||
return tuple(min(255, max(0, int(round(c * factor)))) for c in color)
|
||||
|
||||
|
||||
def lift_color(color: Sequence[float], min_luminance: float = _PALETTE_HEADLINE_LUMINANCE,
|
||||
cap_saturation: float = 0.92) -> Color:
|
||||
"""Raise a colour's brightness until it reads on a panel, keeping its hue.
|
||||
|
||||
Scaling the channels directly is what the obvious version of this does,
|
||||
and it shifts hue badly on exactly the colours that need lifting: it turns
|
||||
Baltimore's navy-purple into magenta. Working in HSV and raising only the
|
||||
value leaves the hue where the team put it.
|
||||
"""
|
||||
if rgb_luminance(color) >= min_luminance:
|
||||
return tuple(int(c) for c in color)
|
||||
hue, saturation, value = colorsys.rgb_to_hsv(*[c / 255.0 for c in color])
|
||||
if saturation < 0.12:
|
||||
# A grey or a silver has no hue to preserve; just make it bright.
|
||||
lifted = colorsys.hsv_to_rgb(hue, saturation, max(value, 0.85))
|
||||
return tuple(int(round(c * 255)) for c in lifted)
|
||||
saturation = min(saturation, cap_saturation)
|
||||
|
||||
def _rgb(s: float, v: float) -> Color:
|
||||
return tuple(int(round(c * 255)) for c in colorsys.hsv_to_rgb(hue, s, v))
|
||||
|
||||
out = _rgb(saturation, value)
|
||||
while value < 1.0 and rgb_luminance(out) < min_luminance:
|
||||
value = min(1.0, value + 0.05)
|
||||
out = _rgb(saturation, value)
|
||||
# Blue carries almost no luminance -- pure blue sits at 18 of 255 -- so a
|
||||
# navy or a deep purple runs out of value long before it is legible.
|
||||
# Bleeding saturation out of it is the only way up, and it keeps the hue
|
||||
# (Baltimore stays purple, just a lighter one) where giving up would
|
||||
# leave the headline unreadable. Floored so it never washes out to white.
|
||||
while saturation > _PALETTE_MIN_SATURATION and rgb_luminance(out) < min_luminance:
|
||||
saturation = max(_PALETTE_MIN_SATURATION, saturation - 0.05)
|
||||
out = _rgb(saturation, value)
|
||||
return out
|
||||
|
||||
|
||||
def cap_luminance(color: Sequence[float], max_luminance: float) -> Color:
|
||||
"""Darken a colour until it is no brighter than ``max_luminance``.
|
||||
|
||||
A straight channel scale, which is exactly hue-preserving on the way down
|
||||
-- unlike lifting, where clamping at 255 is what bends the hue.
|
||||
"""
|
||||
luminance = rgb_luminance(color)
|
||||
if luminance <= max_luminance or luminance <= 0:
|
||||
return tuple(int(c) for c in color)
|
||||
return scale_color(color, max_luminance / luminance)
|
||||
|
||||
|
||||
def dim_rgba(image: Image.Image, factor: float) -> Image.Image:
|
||||
"""Scale an RGBA image's colour channels, leaving its alpha alone.
|
||||
|
||||
ImageEnhance.Brightness would scale the alpha band too, which fades the
|
||||
crest out instead of dimming it and leaves its anti-aliased edge looking
|
||||
chewed against the backdrop.
|
||||
"""
|
||||
red, green, blue, alpha = image.split()
|
||||
lut = [min(255, int(i * factor)) for i in range(256)]
|
||||
return Image.merge(
|
||||
"RGBA", (red.point(lut), green.point(lut), blue.point(lut), alpha)
|
||||
)
|
||||
|
||||
|
||||
_Buckets = Dict[Tuple[int, int, int], List[int]]
|
||||
|
||||
|
||||
def _palette_buckets(logo: Image.Image) -> Tuple[_Buckets, _Buckets]:
|
||||
"""Bucket a crest's opaque pixels into coarse colour bins.
|
||||
|
||||
Returns ``(vivid, neutral)``; each maps a 3-bit-per-channel key to
|
||||
``[r_sum, g_sum, b_sum, count]``. Neutral holds the greys, silvers and
|
||||
whites that carry no identity on their own but are all a monochrome crest
|
||||
-- the Raiders' silver on black -- has to offer.
|
||||
"""
|
||||
sample = logo.convert("RGBA")
|
||||
sample.thumbnail((_PALETTE_SAMPLE_PX, _PALETTE_SAMPLE_PX), Image.Resampling.BOX)
|
||||
vivid: _Buckets = {}
|
||||
neutral: _Buckets = {}
|
||||
# tobytes() rather than getdata(): same pixels, no per-pixel Python
|
||||
# object, and getdata() is deprecated from Pillow 14.
|
||||
raw = sample.tobytes()
|
||||
for i in range(0, len(raw) - 3, 4):
|
||||
red, green, blue, alpha = raw[i], raw[i + 1], raw[i + 2], raw[i + 3]
|
||||
if alpha < 160:
|
||||
continue
|
||||
high, low = max(red, green, blue), min(red, green, blue)
|
||||
if high < _PALETTE_MIN_CHANNEL:
|
||||
continue
|
||||
target = vivid if (high - low) / high >= _PALETTE_VIVID_SATURATION else neutral
|
||||
acc = target.setdefault((red >> 5, green >> 5, blue >> 5), [0, 0, 0, 0])
|
||||
acc[0] += red
|
||||
acc[1] += green
|
||||
acc[2] += blue
|
||||
acc[3] += 1
|
||||
return vivid, neutral
|
||||
|
||||
|
||||
def _bucket_mean(acc: List[int]) -> Color:
|
||||
count = acc[3]
|
||||
return (acc[0] // count, acc[1] // count, acc[2] // count)
|
||||
|
||||
|
||||
def _bucket_headline_score(acc: List[int]) -> float:
|
||||
"""How well a colour bin would serve as 6px of text on a panel.
|
||||
|
||||
Area alone picks the biggest block of colour, which on a lot of crests is
|
||||
a dark navy fill -- correct as a backdrop, invisible as text. Weighting
|
||||
area by saturation and by luminance picks the colour the team is loud in:
|
||||
Chicago's orange over its navy, Baltimore's gold over its purple.
|
||||
"""
|
||||
color = _bucket_mean(acc)
|
||||
return (
|
||||
acc[3]
|
||||
* (0.30 + 0.70 * rgb_saturation(color))
|
||||
* (0.20 + 0.80 * min(1.0, rgb_luminance(color) / 120.0))
|
||||
)
|
||||
|
||||
|
||||
def logo_palette(logo: Image.Image) -> Optional[Palette]:
|
||||
"""Pick a celebration palette out of a team crest, or None.
|
||||
|
||||
Two rankings, because a crest's largest colour and its most legible one
|
||||
are usually not the same and the takeover needs both:
|
||||
|
||||
* ``deep`` -- the largest vivid area, darkened into the background wash.
|
||||
This is what the team reads as at a glance: Chicago navy, Dallas navy,
|
||||
Baltimore purple.
|
||||
* ``headline`` -- the vivid area that best survives being shrunk to text,
|
||||
then lifted until it is legible: Chicago orange, Baltimore gold.
|
||||
* ``accent`` -- the next vivid colour far enough away from the headline to
|
||||
be told apart, for confetti. Falls back to the headline.
|
||||
|
||||
A crest with no vivid pixels at all falls back to its brightest neutral,
|
||||
which for the Raiders' silver-on-black is exactly the right answer.
|
||||
"""
|
||||
try:
|
||||
vivid, neutral = _palette_buckets(logo)
|
||||
except Exception: # noqa: BLE001 - a crest is never worth the takeover
|
||||
return None
|
||||
|
||||
pool = list(vivid.values())
|
||||
if not pool and neutral:
|
||||
pool = [
|
||||
max(
|
||||
neutral.values(),
|
||||
key=lambda acc: acc[3]
|
||||
* (0.2 + 0.8 * min(1.0, rgb_luminance(_bucket_mean(acc)) / 160.0)),
|
||||
)
|
||||
]
|
||||
if not pool:
|
||||
return None
|
||||
|
||||
deep_base = _bucket_mean(max(pool, key=lambda acc: acc[3]))
|
||||
ranked = sorted(pool, key=_bucket_headline_score, reverse=True)
|
||||
headline_base = _bucket_mean(ranked[0])
|
||||
vivid_pixels = sum(acc[3] for acc in pool)
|
||||
for acc in ranked:
|
||||
candidate = _bucket_mean(acc)
|
||||
if (
|
||||
rgb_luminance(candidate) >= _PALETTE_LEGIBLE_LUMINANCE
|
||||
and rgb_saturation(candidate) >= _PALETTE_LEGIBLE_SATURATION
|
||||
and acc[3] >= max(3, vivid_pixels * _PALETTE_LEGIBLE_AREA)
|
||||
):
|
||||
headline_base = candidate
|
||||
break
|
||||
headline = lift_color(headline_base)
|
||||
|
||||
accent = headline
|
||||
for acc in ranked[1:]:
|
||||
candidate = _bucket_mean(acc)
|
||||
if color_distance(candidate, headline_base) > _PALETTE_DISTINCT_DISTANCE:
|
||||
accent = lift_color(candidate)
|
||||
break
|
||||
|
||||
deep = cap_luminance(deep_base, _PALETTE_BACKDROP_LUMINANCE)
|
||||
return {
|
||||
"deep": deep,
|
||||
# Scenery is the backdrop carried a little way towards the headline:
|
||||
# tied to the team's colours, and guaranteed to be visible even when
|
||||
# the backdrop is nearly black.
|
||||
"glow": cap_luminance(
|
||||
mix_color(deep, headline, 0.22), _PALETTE_SCENERY_LUMINANCE
|
||||
),
|
||||
"headline": headline,
|
||||
"accent": accent,
|
||||
}
|
||||
|
||||
|
||||
class SportsCelebrationMixin:
|
||||
"""Draws a score/win celebration takeover. See the module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only: these create no
|
||||
# attributes, so the host's own values are what the methods read.
|
||||
display_manager: Any
|
||||
display_width: int
|
||||
display_height: int
|
||||
fonts: Dict[str, Any]
|
||||
logger: logging.Logger
|
||||
_load_and_resize_logo: Callable[..., Optional[Image.Image]]
|
||||
_draw_text_with_outline: Callable[..., None]
|
||||
|
||||
def _fit_font(self, draw, text: str, max_width: int, fonts: list):
|
||||
"""Return the first font whose rendered ``text`` fits ``max_width``,
|
||||
falling back to the last (smallest) font."""
|
||||
for font in fonts:
|
||||
if draw.textlength(text, font=font) <= max_width - 2:
|
||||
return font
|
||||
return fonts[-1]
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Celebration palette
|
||||
#
|
||||
# The takeover is drawn in the scoring team's own colours, taken from the
|
||||
# pixels of its crest.
|
||||
#
|
||||
# ESPN does serve team.color / team.alternateColor, but only inside
|
||||
# _extract_game_details_common -- a function each scoreboard lineage
|
||||
# keeps its own copy of -- so reading it there would drag every one of
|
||||
# them into a celebration change. The crest is already downloaded,
|
||||
# decoded and sitting in the logo cache by the time a celebration draws,
|
||||
# so the colours come from it instead: no extra request, no per-league
|
||||
# colour table to maintain, and it works for any team ESPN can name --
|
||||
# including the FCS opponents no table would list.
|
||||
#
|
||||
# Where a crest's colour differs from the club's published one it
|
||||
# tends to differ usefully: a published primary is often a near-black
|
||||
# navy, or an actual #000000, where what the crest carries is the colour
|
||||
# that reads on an LED panel. Measured across all 32 clubs in
|
||||
# football-scoreboard.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
#: Used when the crest yields nothing (no logo on disk yet, or the grey
|
||||
#: placeholder a failed download leaves) or team colours are switched
|
||||
#: off -- the navy and amber the celebration wore before it had a palette.
|
||||
_DEFAULT_CELEBRATION_PALETTE: ClassVar[Palette] = {
|
||||
"deep": (10, 10, 40),
|
||||
"glow": (30, 30, 86),
|
||||
"headline": (255, 208, 56),
|
||||
"accent": (255, 255, 255),
|
||||
}
|
||||
|
||||
def _celebration_palette(self, celebration: Dict) -> Palette:
|
||||
"""The scoring team's colours, derived once per celebration."""
|
||||
cached: Optional[Palette] = celebration.get("_palette")
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
palette = dict(self._DEFAULT_CELEBRATION_PALETTE)
|
||||
if getattr(self, "celebration_team_colors", True):
|
||||
try:
|
||||
game = celebration["game"]
|
||||
side = celebration.get("scored_side") or "home"
|
||||
logo = self._load_and_resize_logo(
|
||||
game.get("%s_id" % side),
|
||||
game.get("%s_abbr" % side),
|
||||
game.get("%s_logo_path" % side),
|
||||
game.get("%s_logo_url" % side),
|
||||
)
|
||||
derived = logo_palette(logo) if logo is not None else None
|
||||
if derived:
|
||||
palette = derived
|
||||
except Exception as e: # noqa: BLE001 - never lose a takeover to a crest
|
||||
self.logger.debug(f"Celebration palette fell back to the default: {e}")
|
||||
|
||||
celebration["_palette"] = palette
|
||||
return palette
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Celebration choreography
|
||||
#
|
||||
# Every frame is a finished card. The beats below shift the emphasis --
|
||||
# an opening colour hit, confetti, a breathing score -- but none of them
|
||||
# leaves the panel mid-wipe, because on a switch-mode board the core
|
||||
# drives this plugin at 1 FPS (display_controller reserves its high-FPS
|
||||
# loop for plugins that scroll or declare needs_high_fps), so any single
|
||||
# frame may be the only one a viewer ever sees of it.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
#: Fraction of the celebration spent on the opening colour hit.
|
||||
_CELEBRATION_IMPACT: ClassVar[float] = 0.11
|
||||
#: Fraction of it after which the takeover eases back down.
|
||||
_CELEBRATION_SETTLE: ClassVar[float] = 0.80
|
||||
#: Seconds per breath of the scoring side's digits. Deliberately a
|
||||
#: continuous sine rather than an on/off toggle: the 4 Hz flash this
|
||||
#: replaced was sampled once a second on a switch-mode board, which
|
||||
#: aliases into a colour that changes at random. A ramp degrades into a
|
||||
#: slow glow instead, and still reads as a pulse at 125 FPS.
|
||||
_CELEBRATION_BREATH_SECONDS: ClassVar[float] = 1.7
|
||||
|
||||
def _celebration_backdrop(
|
||||
self,
|
||||
celebration: Dict,
|
||||
width: int,
|
||||
height: int,
|
||||
palette: Palette,
|
||||
) -> Image.Image:
|
||||
"""The static half of the takeover: a team-colour gradient with the
|
||||
scenery for this kind of score painted into it.
|
||||
|
||||
Built once per celebration per panel size and copied per frame, so the
|
||||
per-pixel work never lands on the render path.
|
||||
"""
|
||||
cached: Optional[Tuple[Tuple[int, int], Image.Image]] = celebration.get("_backdrop")
|
||||
if cached is not None and cached[0] == (width, height):
|
||||
return cached[1]
|
||||
|
||||
# One column, then stretched: filling the panel pixel by pixel would
|
||||
# be `width` times the work for the same image.
|
||||
column = Image.new("RGB", (1, max(height, 1)))
|
||||
pixels: Any = column.load()
|
||||
for y in range(height):
|
||||
k = y / max(height - 1, 1)
|
||||
pixels[0, y] = mix_color(palette["deep"], (0, 0, 0), 0.18 + 0.82 * k)
|
||||
backdrop = column.resize((width, height)).convert("RGBA")
|
||||
|
||||
try:
|
||||
self._draw_celebration_motif(
|
||||
ImageDraw.Draw(backdrop),
|
||||
celebration.get("motif") or "score",
|
||||
width,
|
||||
height,
|
||||
palette,
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 - scenery is never worth a blank panel
|
||||
self.logger.debug(f"Celebration motif skipped: {e}")
|
||||
|
||||
celebration["_backdrop"] = ((width, height), backdrop)
|
||||
return backdrop
|
||||
|
||||
def _draw_celebration_motif(
|
||||
self,
|
||||
draw,
|
||||
motif: str,
|
||||
width: int,
|
||||
height: int,
|
||||
palette: Palette,
|
||||
) -> None:
|
||||
"""Paint the scenery for one kind of score, dim enough to stay behind
|
||||
the headline and the score instead of competing with them."""
|
||||
glow = palette["glow"]
|
||||
if motif == "kick":
|
||||
# The uprights a field goal or an extra point went through,
|
||||
# spread wide enough to frame the score rather than sit beside it.
|
||||
half = max(8, min(width // 3, height))
|
||||
mid = width // 2
|
||||
crossbar = int(height * 0.60)
|
||||
draw.line([(mid - half, int(height * 0.08)), (mid - half, crossbar)], fill=glow)
|
||||
draw.line([(mid + half, int(height * 0.08)), (mid + half, crossbar)], fill=glow)
|
||||
draw.line([(mid - half, crossbar), (mid + half, crossbar)], fill=glow)
|
||||
draw.line([(mid, crossbar), (mid, height - 1)], fill=glow)
|
||||
elif motif == "touchdown":
|
||||
# The goal line, with its hash marks.
|
||||
line_y = int(height * 0.36)
|
||||
draw.line([(0, line_y), (width, line_y)], fill=glow)
|
||||
for x in range(3, width, 9):
|
||||
draw.line([(x, line_y - 2), (x, line_y + 2)], fill=glow)
|
||||
elif motif == "net":
|
||||
# The goal a puck just went into: frame, posts and mesh, sized to
|
||||
# frame the score the way the uprights do.
|
||||
half = max(7, min(width // 4, height))
|
||||
mid = width // 2
|
||||
top = int(height * 0.34)
|
||||
draw.rectangle([(mid - half, top), (mid + half, height - 1)], outline=glow)
|
||||
step = max(3, (half * 2) // 6)
|
||||
for x in range(mid - half + step, mid + half, step):
|
||||
draw.line([(x, top + 1), (x, height - 2)], fill=glow)
|
||||
for y in range(top + step, height - 1, step):
|
||||
draw.line([(mid - half + 1, y), (mid + half - 1, y)], fill=glow)
|
||||
elif motif == "win":
|
||||
# A sunburst behind the winner.
|
||||
cx, cy = width // 2, height // 2
|
||||
reach = max(width, height)
|
||||
for i in range(10):
|
||||
angle = (math.pi * 2 * i / 10) + math.pi / 20
|
||||
draw.line(
|
||||
[
|
||||
(cx, cy),
|
||||
(cx + math.cos(angle) * reach, cy + math.sin(angle) * reach),
|
||||
],
|
||||
fill=glow,
|
||||
)
|
||||
else:
|
||||
for x in range(-height, width + height, 11):
|
||||
draw.line([(x, height), (x + height, 0)], fill=glow)
|
||||
|
||||
def _celebration_confetti(
|
||||
self,
|
||||
celebration: Dict,
|
||||
width: int,
|
||||
height: int,
|
||||
palette: Palette,
|
||||
) -> List[Flake]:
|
||||
"""Seed the confetti once per celebration.
|
||||
|
||||
Seeded from the game rather than the clock, so the same score always
|
||||
produces the same fall -- which is what lets a golden screen lock the
|
||||
effect down instead of having to tolerate it.
|
||||
"""
|
||||
cached: Optional[Tuple[Tuple[int, int], List[Flake]]] = celebration.get("_confetti")
|
||||
if cached is not None and cached[0] == (width, height):
|
||||
return cached[1]
|
||||
|
||||
# Sparse on purpose. At one flake per 170 square pixels a 128x32
|
||||
# panel carried 24 single-pixel specks over the headline and the
|
||||
# score, which reads as a dead-pixel problem rather than as confetti.
|
||||
count = max(6, min(22, (width * height) // 260))
|
||||
seed = "%s/%s" % (
|
||||
(celebration.get("game") or {}).get("id", "?"),
|
||||
celebration.get("phrase", ""),
|
||||
)
|
||||
rng = random.Random(seed) # nosec B311 - confetti, not security
|
||||
# Team colours, plus a pale tint of the headline rather than a flat
|
||||
# white, so the fall still belongs to the team that scored.
|
||||
colors = [
|
||||
palette["headline"],
|
||||
palette["accent"],
|
||||
mix_color(palette["headline"], (255, 255, 255), 0.55),
|
||||
]
|
||||
flakes = [
|
||||
(
|
||||
float(rng.randrange(max(width, 1))), # column
|
||||
rng.uniform(0.0, float(height)), # start height
|
||||
rng.uniform(0.40, 1.15), # fall speed
|
||||
rng.uniform(0.0, math.pi * 2), # sway phase
|
||||
2 if rng.random() < 0.6 else 1, # size in pixels
|
||||
colors[rng.randrange(len(colors))],
|
||||
)
|
||||
for _ in range(count)
|
||||
]
|
||||
celebration["_confetti"] = ((width, height), flakes)
|
||||
return flakes
|
||||
|
||||
def _draw_celebration_confetti(
|
||||
self,
|
||||
draw,
|
||||
celebration: Dict,
|
||||
width: int,
|
||||
height: int,
|
||||
palette: Palette,
|
||||
elapsed: float,
|
||||
progress: float,
|
||||
) -> None:
|
||||
"""Draw the confetti for this instant, thinning it out as the
|
||||
celebration eases back towards the scorebug."""
|
||||
flakes = self._celebration_confetti(celebration, width, height, palette)
|
||||
fade = 1.0
|
||||
if progress > self._CELEBRATION_SETTLE:
|
||||
fade = max(
|
||||
0.0,
|
||||
1.0
|
||||
- (progress - self._CELEBRATION_SETTLE)
|
||||
/ (1.0 - self._CELEBRATION_SETTLE),
|
||||
)
|
||||
if fade <= 0.02:
|
||||
return
|
||||
alpha = int(235 * fade)
|
||||
for column, start, speed, phase, size, color in flakes:
|
||||
y = (start + speed * elapsed * height * 0.42) % (height + 4) - 2
|
||||
x = column + math.sin(elapsed * 2.1 + phase) * 2.4
|
||||
draw.rectangle(
|
||||
[(int(x), int(y)), (int(x) + size - 1, int(y) + size - 1)],
|
||||
fill=tuple(color) + (alpha,),
|
||||
)
|
||||
|
||||
def _celebration_crests(
|
||||
self, celebration: Dict, height: int
|
||||
) -> Dict[str, Optional[Image.Image]]:
|
||||
"""The two crests for the takeover, with the side that did not score
|
||||
dimmed so the scoring team reads at a glance."""
|
||||
cached: Optional[Tuple[int, Dict[str, Optional[Image.Image]]]] = celebration.get("_crests")
|
||||
if cached is not None and cached[0] == height:
|
||||
return cached[1]
|
||||
|
||||
game = celebration["game"]
|
||||
scored = celebration.get("scored_side")
|
||||
crests: Dict[str, Optional[Image.Image]] = {}
|
||||
for side in ("away", "home"):
|
||||
logo = None
|
||||
try:
|
||||
logo = self._load_and_resize_logo(
|
||||
game.get("%s_id" % side),
|
||||
game.get("%s_abbr" % side),
|
||||
game.get("%s_logo_path" % side),
|
||||
game.get("%s_logo_url" % side),
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 - a crest is never worth the panel
|
||||
self.logger.debug(f"Celebration logo load failed: {e}")
|
||||
if logo is not None and side != scored:
|
||||
logo = dim_rgba(logo, 0.40)
|
||||
crests[side] = logo
|
||||
|
||||
celebration["_crests"] = (height, crests)
|
||||
return crests
|
||||
|
||||
def _draw_celebration_layout(self, celebration: Dict, force_clear: bool = False) -> None:
|
||||
"""Render the full-screen goal/win takeover."""
|
||||
if force_clear:
|
||||
self.display_manager.clear()
|
||||
|
||||
display_width = (
|
||||
self.display_manager.matrix.width
|
||||
if hasattr(self.display_manager, "matrix") and self.display_manager.matrix
|
||||
else self.display_width
|
||||
)
|
||||
display_height = (
|
||||
self.display_manager.matrix.height
|
||||
if hasattr(self.display_manager, "matrix") and self.display_manager.matrix
|
||||
else self.display_height
|
||||
)
|
||||
|
||||
elapsed = max(0.0, time.time() - celebration["started_at"])
|
||||
# getattr throughout the render path: the golden-screen tests build a
|
||||
# live manager through __new__ and set only what they draw with, and a
|
||||
# celebration must never be lost to a missing knob.
|
||||
duration = max(float(getattr(self, "celebration_duration", 8) or 8), 0.5)
|
||||
progress = min(elapsed / duration, 1.0)
|
||||
palette = self._celebration_palette(celebration)
|
||||
|
||||
main_img = self._celebration_backdrop(
|
||||
celebration, display_width, display_height, palette
|
||||
).copy()
|
||||
|
||||
# Crests at the edges, bleeding off as the scorebug's do.
|
||||
crests = self._celebration_crests(celebration, display_height)
|
||||
center_y = display_height // 2
|
||||
home_logo, away_logo = crests.get("home"), crests.get("away")
|
||||
if home_logo is not None:
|
||||
main_img.paste(
|
||||
home_logo,
|
||||
(display_width - home_logo.width + 2, center_y - home_logo.height // 2),
|
||||
home_logo,
|
||||
)
|
||||
if away_logo is not None:
|
||||
main_img.paste(away_logo, (-2, center_y - away_logo.height // 2), away_logo)
|
||||
|
||||
# The opening hit: the team's headline colour washes the panel and
|
||||
# decays out of it. Held below opaque so the outlined text drawn on
|
||||
# top still reads in whichever frame happens to catch it.
|
||||
impact = max(0.0, 1.0 - progress / self._CELEBRATION_IMPACT)
|
||||
if impact > 0.0:
|
||||
# Scaled by how colourful the team is. A saturated crest gets the
|
||||
# full hit; a silver one -- the Raiders, or the grey placeholder a
|
||||
# failed logo download leaves -- would otherwise wash the whole
|
||||
# panel out to the same flat grey as its own headline colour.
|
||||
punch = 0.45 + 0.55 * rgb_saturation(palette["headline"])
|
||||
alpha = int(140 * punch * (impact ** 1.5))
|
||||
if alpha > 0:
|
||||
main_img = Image.alpha_composite(
|
||||
main_img,
|
||||
Image.new(
|
||||
"RGBA",
|
||||
(display_width, display_height),
|
||||
tuple(palette["headline"]) + (alpha,),
|
||||
),
|
||||
)
|
||||
|
||||
overlay = Image.new("RGBA", (display_width, display_height), (0, 0, 0, 0))
|
||||
draw = ImageDraw.Draw(overlay)
|
||||
|
||||
if getattr(self, "celebration_confetti", True):
|
||||
try:
|
||||
self._draw_celebration_confetti(
|
||||
draw,
|
||||
celebration,
|
||||
display_width,
|
||||
display_height,
|
||||
palette,
|
||||
elapsed,
|
||||
progress,
|
||||
)
|
||||
except Exception as e: # noqa: BLE001
|
||||
self.logger.debug(f"Celebration confetti skipped: {e}")
|
||||
|
||||
# Headline across the top, shrunk to fit the panel width, struck
|
||||
# white on the opening hit and settling into the team's colour.
|
||||
phrase = celebration["phrase"]
|
||||
phrase_font = self._fit_font(
|
||||
draw, phrase, display_width, [self.fonts["time"], self.fonts["status"]]
|
||||
)
|
||||
phrase_width = draw.textlength(phrase, font=phrase_font)
|
||||
# Eased in by colour rather than by position. Sliding it down into
|
||||
# place put the first frame at y=-3 with its top row cut off, and on a
|
||||
# 1 FPS board that clipped frame can be the only one anyone sees.
|
||||
self._draw_text_with_outline(
|
||||
draw,
|
||||
phrase,
|
||||
((display_width - phrase_width) // 2, 1),
|
||||
phrase_font,
|
||||
fill=mix_color(palette["headline"], (255, 255, 255), impact),
|
||||
)
|
||||
|
||||
# Score centred low, the scoring side's digits breathing in the team's
|
||||
# headline colour so the change reads at a glance.
|
||||
away_text = str(celebration["away_score"])
|
||||
home_text = str(celebration["home_score"])
|
||||
score_font = self.fonts["score"]
|
||||
segments = [
|
||||
(away_text, celebration["scored_side"] == "away"),
|
||||
("-", False),
|
||||
(home_text, celebration["scored_side"] == "home"),
|
||||
]
|
||||
total_width = sum(draw.textlength(seg, font=score_font) for seg, _ in segments)
|
||||
breath = 0.72 + 0.28 * (
|
||||
0.5
|
||||
+ 0.5 * math.sin(2 * math.pi * elapsed / self._CELEBRATION_BREATH_SECONDS)
|
||||
)
|
||||
highlight = scale_color(palette["headline"], breath)
|
||||
x = (display_width - total_width) // 2
|
||||
# display_height - 14 was sized for the old fixed 8px score. #338
|
||||
# scales the score with the panel (16px at 48 and 64 tall), which put
|
||||
# the bottom of the digits off the panel. Lift it by the measured ink
|
||||
# (+1 for the outline stroke) only when it would clip, so panels where
|
||||
# it always fitted render exactly as before.
|
||||
score_text = "".join(seg for seg, _ in segments)
|
||||
ink_bottom = draw.textbbox((0, 0), score_text, font=score_font)[3]
|
||||
y = min(display_height - 14, display_height - ink_bottom - 2)
|
||||
for seg, is_highlight in segments:
|
||||
color = highlight if is_highlight else (216, 216, 216)
|
||||
self._draw_text_with_outline(draw, seg, (int(x), y), score_font, fill=color)
|
||||
x += draw.textlength(seg, font=score_font)
|
||||
|
||||
main_img = Image.alpha_composite(main_img, overlay).convert("RGB")
|
||||
self.display_manager.image = main_img
|
||||
self.display_manager.update_display()
|
||||
@@ -0,0 +1,188 @@
|
||||
"""Which requests a scoreboard makes: season fetches, the lookback, live odds.
|
||||
|
||||
Four ``SportsCore`` methods are identical (executable AST, docstrings
|
||||
stripped) in all nine scoreboards' ``sports.py`` -- afl, baseball,
|
||||
basketball, football, hockey, lacrosse, nrl, soccer and ufc -- and were
|
||||
copied here from ledmatrix-plugins ``30455671`` (origin/main, 2026-09-29)
|
||||
under their existing names:
|
||||
|
||||
- ``_background_fetches_espn_ranges`` -- whether the core's background
|
||||
service can fetch an ESPN date range, or the plugin must;
|
||||
- ``_fetch_season_directly`` -- fetch and cache a season in chunks ESPN
|
||||
accepts, on the calling thread;
|
||||
- ``_needs_previous_day`` (with ``_LOOKBACK_CUTOFF_HOUR``) -- whether the
|
||||
live fetch still has to ask for yesterday;
|
||||
- ``_wants_live_odds`` (with ``_LIVE_ODDS_LOOKAHEAD``) -- whether a live
|
||||
game is close enough to the screen to be worth an odds request.
|
||||
|
||||
Three other ``SportsCore`` methods are as identical and stay in the plugins,
|
||||
for the reasons ``sports_shared`` gives: ``_get_timezone`` binds each
|
||||
plugin's own ``resolve_timezone`` shim, and ``_extract_game_details`` /
|
||||
``_fetch_data`` are the abstract sport-specific contract. So does
|
||||
``SportsUpcoming.__init__``: the mixins in ``src/common`` hold no
|
||||
constructor, so the plugins' constructor signature stays theirs.
|
||||
|
||||
A new module rather than more methods on ``sports_shared``, for the reason
|
||||
``sports_helpers`` gives: a missing module fails at load, where the version
|
||||
checks see it; a missing method fails mid-update.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_fetch.py`` fails if a read is added without being
|
||||
listed here.
|
||||
|
||||
- ``session``, ``headers``, ``cache_manager`` and ``logger`` --
|
||||
``_fetch_season_directly``.
|
||||
- ``_games_lock`` -- ``_wants_live_odds``, which also reads ``live_games``,
|
||||
``current_game_index`` and ``_rotation_schedule`` with ``getattr``
|
||||
(only ``SportsLive`` has them).
|
||||
- ``live_games``, read with ``getattr`` -- ``_needs_previous_day``.
|
||||
- ``background_service``, read with ``getattr`` --
|
||||
``_background_fetches_espn_ranges``.
|
||||
|
||||
Add it as a base of the plugin's ``SportsCore``, e.g.
|
||||
``class SportsCore(SportsFetchMixin, SportsCoreSharedMixin,
|
||||
SportsHelpersMixin, ABC)``. It defines nothing those define; a method or
|
||||
constant on the plugin's own class still wins over the mixin's.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Any, ClassVar, Dict, Optional
|
||||
|
||||
from src.common.espn_dates import ESPN_MAX_LIMIT, fetch_espn_scoreboard
|
||||
|
||||
|
||||
class SportsFetchMixin:
|
||||
"""Season fetch, lookback and live-odds decisions. See module docstring."""
|
||||
|
||||
# The host contract, declared for type checking only: these create no
|
||||
# attributes, so the host's own values are what the methods read.
|
||||
session: Any
|
||||
headers: Dict[str, str]
|
||||
cache_manager: Any
|
||||
logger: logging.Logger
|
||||
_games_lock: threading.RLock
|
||||
|
||||
#: How many games past the one on screen keep their odds warm. One is
|
||||
#: enough for the line to be ready when the rotation advances; more just
|
||||
#: re-creates the whole-slate fetch this replaced.
|
||||
_LIVE_ODDS_LOOKAHEAD: ClassVar[int] = 1
|
||||
|
||||
def _wants_live_odds(self, game: Dict) -> bool:
|
||||
"""Whether a live game is near enough the front of the rotation to be
|
||||
worth an odds request.
|
||||
|
||||
Odds used to be fetched for *every* live game in the league on every
|
||||
update. The renderer only ever draws ``current_game``, and a full
|
||||
rotation of a big slate takes minutes while ``live_odds_update_interval``
|
||||
is 60s -- so all but one of those requests expired before the game they
|
||||
belonged to came round.
|
||||
|
||||
Measured 2026-09-19 over a full college-football slate: 11,978 odds
|
||||
requests in 13h on one rig, 54% of all its ESPN traffic, across only
|
||||
~140 distinct games. The eager loop also cost up to 2s of ``update()``
|
||||
per live game, because ``_fetch_odds`` waits on its worker thread.
|
||||
|
||||
Mirrors the narrowing already applied to the upcoming path and to
|
||||
``_attach_odds_to_rotated_games``: only games about to be on screen are
|
||||
asked about. ``get_odds`` still caches per game, so a game re-entering
|
||||
the window inside its TTL costs a cache lookup, not a request.
|
||||
|
||||
The rotation state read here is the previous cycle's -- the new list is
|
||||
still being built -- which is exactly the question being asked: is this
|
||||
game at or near the position currently on the panel?
|
||||
"""
|
||||
# Read defensively: this predicate lives on SportsCore so it sits
|
||||
# beside _fetch_odds, but live_games/_rotation_schedule belong to
|
||||
# SportsLive, which is the only caller.
|
||||
with self._games_lock:
|
||||
games = list(getattr(self, "live_games", ()) or ())
|
||||
index = getattr(self, "current_game_index", 0)
|
||||
schedule = list(getattr(self, "_rotation_schedule", ()) or ())
|
||||
if not games:
|
||||
# Cold start: nothing is on screen yet, so let the games seen on
|
||||
# this first pass through rather than render a blank line for a
|
||||
# whole cycle. Bounded -- the next pass has a rotation to narrow by.
|
||||
return True
|
||||
order = schedule or [g.get("id") for g in games]
|
||||
if not order:
|
||||
return True
|
||||
start = index if 0 <= index < len(order) else 0
|
||||
wanted = {
|
||||
order[(start + offset) % len(order)]
|
||||
for offset in range(self._LIVE_ODDS_LOOKAHEAD + 1)
|
||||
}
|
||||
return game.get("id") in wanted
|
||||
|
||||
#: Hour of the Eastern day past which last night's games are assumed over.
|
||||
#:
|
||||
#: The live fetch asks ESPN for a two-day window so a game that started
|
||||
#: yesterday and is still running is not lost. ESPN rejects date *ranges*,
|
||||
#: so that window is split into one request per day -- doubling every live
|
||||
#: poll. Measured 2026-09-19: 1,858 requests per rig spent on yesterday's
|
||||
#: date, which after breakfast holds nothing but final games.
|
||||
#:
|
||||
#: No sport on these boards runs six hours past midnight, and one that
|
||||
#: somehow did is still covered: a game already being tracked keeps its own
|
||||
#: day in the window regardless of the hour.
|
||||
_LOOKBACK_CUTOFF_HOUR: ClassVar[int] = 6
|
||||
|
||||
def _needs_previous_day(self, now: datetime) -> bool:
|
||||
"""Whether the previous Eastern day can still hold a live game."""
|
||||
if now.hour < self._LOOKBACK_CUTOFF_HOUR:
|
||||
return True
|
||||
previous = (now - timedelta(days=1)).strftime("%Y%m%d")
|
||||
for game in (getattr(self, "live_games", None) or []):
|
||||
start: Any = game.get("start_time_utc") if hasattr(game, "get") else None
|
||||
try:
|
||||
if start.astimezone(now.tzinfo).strftime("%Y%m%d") == previous:
|
||||
return True
|
||||
except (AttributeError, ValueError, OSError, OverflowError):
|
||||
continue
|
||||
return False
|
||||
|
||||
def _background_fetches_espn_ranges(self) -> bool:
|
||||
"""Can the core's background service fetch an ESPN date range?
|
||||
|
||||
Cores from before the 2026-09-15 fix send a season range to ESPN as-is,
|
||||
which now answers 400 for every sport. On those cores the managers fetch
|
||||
the season themselves with _fetch_season_directly instead.
|
||||
"""
|
||||
service = getattr(self, "background_service", None)
|
||||
return bool(getattr(service, "handles_espn_date_ranges", False))
|
||||
|
||||
def _fetch_season_directly(
|
||||
self,
|
||||
url: str,
|
||||
datestring: str,
|
||||
cache_key: str,
|
||||
label: str,
|
||||
ttl: Optional[int] = None,
|
||||
) -> Optional[Dict]:
|
||||
"""Fetch a season schedule on this thread, in chunks ESPN accepts, and cache it.
|
||||
|
||||
``label`` names the schedule in log lines, e.g. ``"2026 season"``.
|
||||
"""
|
||||
try:
|
||||
data = fetch_espn_scoreboard(
|
||||
self.session,
|
||||
url,
|
||||
params={"dates": datestring, "limit": ESPN_MAX_LIMIT},
|
||||
headers=self.headers,
|
||||
timeout=30,
|
||||
logger=self.logger,
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Failed to fetch {label} schedule: {e}")
|
||||
return None
|
||||
if ttl is None:
|
||||
self.cache_manager.set(cache_key, data)
|
||||
else:
|
||||
self.cache_manager.set(cache_key, data, ttl=ttl)
|
||||
self.logger.info(
|
||||
f"Fetched {label} schedule: {len(data.get('events', []))} events"
|
||||
)
|
||||
return data
|
||||
@@ -43,11 +43,14 @@ CORE_CONFIG_KEYS = frozenset({
|
||||
})
|
||||
|
||||
#: Top-level keys of ``config_secrets.json`` that belong to the core rather than
|
||||
#: to a plugin: the GitHub token the Plugin Store reads, and the historical
|
||||
#: ``youtube`` section. Plugin secrets are namespaced by plugin id, so anything
|
||||
#: deciding whether a secrets section is a plugin's needs this as well as
|
||||
#: ``CORE_CONFIG_KEYS``.
|
||||
#: to a plugin: the GitHub token the Plugin Store reads, the historical
|
||||
#: ``youtube`` section, and ``web_auth`` (the optional web login's password
|
||||
#: hash and API-token hashes, web_interface/auth.py) -- which orphan-plugin
|
||||
#: cleanup would otherwise delete, logging everyone out. Plugin secrets are
|
||||
#: namespaced by plugin id, so anything deciding whether a secrets section is a
|
||||
#: plugin's needs this as well as ``CORE_CONFIG_KEYS``.
|
||||
CORE_SECRETS_KEYS = frozenset({
|
||||
'github',
|
||||
'youtube',
|
||||
'web_auth',
|
||||
})
|
||||
|
||||
@@ -6,6 +6,14 @@ still be called by a plugin nobody has checked. Such methods get
|
||||
process logs a warning naming the method and the release that removes it
|
||||
(visible in ``journalctl -u ledmatrix``), and emits a DeprecationWarning for
|
||||
tooling.
|
||||
|
||||
Before a release removes anything, ``scripts/plugin_api_usage.py`` scans core,
|
||||
the official plugin monorepo and the registry's third-party plugins for callers
|
||||
and overriders of every marked method. Its latest output is
|
||||
``docs/DEPRECATIONS_3.8.md``; remove only what it reports unused, and move the
|
||||
rest to a later release. ``test/test_deprecation.py`` fails while any marker
|
||||
names a release at or below ``src.__version__``, so a release cannot ship with
|
||||
a removal date it has already passed.
|
||||
"""
|
||||
|
||||
import functools
|
||||
@@ -45,3 +53,27 @@ def deprecated(removal: str, alternative: Optional[str] = None) -> Callable[[F],
|
||||
return wrapper # type: ignore[return-value]
|
||||
|
||||
return decorate
|
||||
|
||||
|
||||
def warn_deprecated(what: str, removal: str, alternative: Optional[str] = None,
|
||||
once_key: Optional[str] = None) -> bool:
|
||||
"""Warn that ``what`` will be removed in ``removal``, once per process.
|
||||
|
||||
For what ``@deprecated`` cannot decorate: a config key, a manifest field,
|
||||
a value a hook returns. Same message, log line and DeprecationWarning as
|
||||
the decorator. ``once_key`` (default: ``what``) is what "once" counts
|
||||
against, so one deprecated key can warn once for each plugin that sets it.
|
||||
|
||||
Returns whether this call warned.
|
||||
"""
|
||||
message = f"{what} is deprecated and will be removed in LEDMatrix {removal}"
|
||||
if alternative:
|
||||
message += f"; {alternative}"
|
||||
key = once_key or what
|
||||
with _warned_lock:
|
||||
if key in _warned:
|
||||
return False
|
||||
_warned.add(key)
|
||||
logger.warning(message)
|
||||
warnings.warn(message, DeprecationWarning, stacklevel=2)
|
||||
return True
|
||||
|
||||
+279
-28
@@ -29,11 +29,12 @@ import threading
|
||||
import types
|
||||
from collections import deque
|
||||
from contextlib import contextmanager
|
||||
from typing import Dict, Any, List, Optional, Callable, Tuple
|
||||
from typing import Dict, Any, List, Optional, Callable, Set, Tuple
|
||||
from datetime import datetime
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed # pylint: disable=no-name-in-module
|
||||
import pytz
|
||||
|
||||
from src import display_watchdog
|
||||
from src.display_manager import DisplayManager
|
||||
from src.config_manager import ConfigManager
|
||||
from src.config_service import ConfigService
|
||||
@@ -218,6 +219,7 @@ class DisplayController:
|
||||
# Initialize Plugin System
|
||||
plugin_time = time.time()
|
||||
self.plugin_manager = None
|
||||
self._plugin_runtime_publisher = None
|
||||
self.plugin_modes = {} # mode -> plugin_instance mapping for plugin-first dispatch
|
||||
self.mode_to_plugin_id: Dict[str, str] = {}
|
||||
self.plugin_display_modes: Dict[str, List[str]] = {}
|
||||
@@ -266,6 +268,10 @@ class DisplayController:
|
||||
self.on_demand_last_error: Optional[str] = None
|
||||
self.on_demand_last_event: Optional[str] = None
|
||||
self.on_demand_schedule_override = False
|
||||
# Plugins that are disabled in config and loaded only because an
|
||||
# on-demand request named them. The main loop unloads each one once
|
||||
# on-demand has moved off it (_release_on_demand_plugins).
|
||||
self._on_demand_loaded_plugins: Set[str] = set()
|
||||
self.rotation_resume_index: Optional[int] = None
|
||||
# Saved rotation position when a live-priority plugin preempts the
|
||||
# rotation, so it resumes where it left off (not after the live plugin)
|
||||
@@ -318,6 +324,13 @@ class DisplayController:
|
||||
font_manager=self.font_manager
|
||||
)
|
||||
|
||||
# The web UI's loaded / state / error_info for each plugin read
|
||||
# what this publishes. Started before loading, so the loads that
|
||||
# follow are published as they land.
|
||||
from src.plugin_system.plugin_runtime import start_plugin_runtime_publisher
|
||||
self._plugin_runtime_publisher = start_plugin_runtime_publisher(
|
||||
self.cache_manager, self.plugin_manager.state_manager)
|
||||
|
||||
# Activate the plugin health/metrics subsystem. PluginManager leaves
|
||||
# health_tracker/resource_monitor as None by default; wiring real
|
||||
# instances here turns on the circuit breaker (a repeatedly-failing
|
||||
@@ -369,7 +382,11 @@ class DisplayController:
|
||||
"""Load a single plugin and return result."""
|
||||
plugin_load_start = time.time()
|
||||
try:
|
||||
if self.plugin_manager.load_plugin(plugin_id):
|
||||
if plugin_id in self._on_demand_loaded_plugins:
|
||||
loaded = self.plugin_manager.load_plugin(plugin_id, force_enabled=True)
|
||||
else:
|
||||
loaded = self.plugin_manager.load_plugin(plugin_id)
|
||||
if loaded:
|
||||
plugin_load_time = time.time() - plugin_load_start
|
||||
return {
|
||||
'success': True,
|
||||
@@ -444,6 +461,12 @@ class DisplayController:
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.exception("Plugin system initialization failed")
|
||||
self.plugin_manager = None
|
||||
# Its state machine no longer describes what runs; let the last
|
||||
# snapshot go stale (readers then say unknown) rather than keep
|
||||
# refreshing it.
|
||||
if self._plugin_runtime_publisher is not None:
|
||||
self._plugin_runtime_publisher.stop(publish_stopped=False)
|
||||
self._plugin_runtime_publisher = None
|
||||
|
||||
# The web UI's Fonts tab ("Used by") reads what this publishes.
|
||||
from src.font_usage import start_font_usage_publisher
|
||||
@@ -1021,17 +1044,30 @@ class DisplayController:
|
||||
accepts_display_mode: Whether display() takes ``display_mode``.
|
||||
force_clear: Passed through to display().
|
||||
|
||||
Each call is timed (two monotonic reads) and handed to
|
||||
PluginManager.note_display_duration, which logs and records slow
|
||||
calls and counts one that ran past the executor's timeout as a hang.
|
||||
|
||||
Returns:
|
||||
display()'s result, or True when the frame was skipped because
|
||||
the plugin's update() holds its lock (the panel keeps the last
|
||||
frame; that is not a failure).
|
||||
"""
|
||||
with self._display_lock_or_skip(getattr(plugin, 'plugin_id', None)) as can_display:
|
||||
# Every frame of both per-screen render loops comes through here.
|
||||
display_watchdog.watchdog.beat()
|
||||
plugin_id = getattr(plugin, 'plugin_id', None)
|
||||
with self._display_lock_or_skip(plugin_id) as can_display:
|
||||
if not can_display:
|
||||
return True
|
||||
if accepts_display_mode:
|
||||
return plugin.display(display_mode=mode, force_clear=force_clear)
|
||||
return plugin.display(force_clear=force_clear)
|
||||
started = time.monotonic()
|
||||
try:
|
||||
if accepts_display_mode:
|
||||
return plugin.display(display_mode=mode, force_clear=force_clear)
|
||||
return plugin.display(force_clear=force_clear)
|
||||
finally:
|
||||
note = getattr(self.plugin_manager, 'note_display_duration', None)
|
||||
if note is not None and plugin_id:
|
||||
note(plugin_id, time.monotonic() - started)
|
||||
|
||||
def _health_tracker(self):
|
||||
"""The plugin circuit breaker, or None when it is not enabled."""
|
||||
@@ -1115,6 +1151,9 @@ class DisplayController:
|
||||
|
||||
sleep_time = min(tick_interval, remaining)
|
||||
time.sleep(sleep_time)
|
||||
# A dwell can be a minute long (sixty seconds while scheduled
|
||||
# off); the watchdog must hear from this thread throughout.
|
||||
display_watchdog.watchdog.beat()
|
||||
self._tick_plugin_updates()
|
||||
self._service_pending_changes()
|
||||
if (self.current_display_mode != mode
|
||||
@@ -1475,8 +1514,13 @@ class DisplayController:
|
||||
On-demand still resumes on its saved mode; this only widens what gets
|
||||
loaded, so normal rotation has somewhere to return to when it ends.
|
||||
A plugin that is disabled in config but named by the on-demand request
|
||||
is still enabled and added, since otherwise the mode being resumed
|
||||
would have nothing behind it.
|
||||
is still loaded, since otherwise the mode being resumed would have
|
||||
nothing behind it. It is tracked as loaded for on-demand only, the
|
||||
same as one loaded live by _activate_on_demand, so it is unloaded
|
||||
when the session ends instead of staying loaded until the next
|
||||
restart. Its config section is not touched: setting ``enabled`` in
|
||||
self.config wrote into the dict config_manager caches and returns to
|
||||
every later load_config() in this process.
|
||||
"""
|
||||
enabled_plugins = [p for p in discovered_plugins
|
||||
if self.config.get(p, {}).get('enabled', False)]
|
||||
@@ -1491,11 +1535,10 @@ class DisplayController:
|
||||
logger.warning("Falling back to normal mode (all enabled plugins)")
|
||||
return enabled_plugins
|
||||
|
||||
if not self.config.get(on_demand_plugin_id, {}).get('enabled', False):
|
||||
logger.info("Temporarily enabling plugin '%s' for on-demand mode", on_demand_plugin_id)
|
||||
self.config.setdefault(on_demand_plugin_id, {})['enabled'] = True
|
||||
if on_demand_plugin_id not in enabled_plugins:
|
||||
enabled_plugins.append(on_demand_plugin_id)
|
||||
if on_demand_plugin_id not in enabled_plugins:
|
||||
logger.info("Loading disabled plugin '%s' for on-demand mode only", on_demand_plugin_id)
|
||||
self._on_demand_loaded_plugins.add(on_demand_plugin_id)
|
||||
enabled_plugins.append(on_demand_plugin_id)
|
||||
|
||||
# Restore on-demand state from the cached request so it resumes.
|
||||
self.on_demand_active = True
|
||||
@@ -1591,6 +1634,11 @@ class DisplayController:
|
||||
logger.debug("Stop request %s received but on-demand is not active", request_id)
|
||||
# Still update request_id to acknowledge the request
|
||||
self.on_demand_request_id = request_id
|
||||
if self.on_demand_status == 'error':
|
||||
# A failed request left status 'error' published, and
|
||||
# without this the status route kept reporting it until
|
||||
# the state aged out (120s) or another request came in.
|
||||
self._clear_on_demand(reason='requested-stop')
|
||||
# Stop requests are deliberately exempt from the request_id/
|
||||
# processed_id guards above, so that a second click stops a mode
|
||||
# that a race left running. Consuming the mailbox is therefore the
|
||||
@@ -1757,10 +1805,136 @@ class DisplayController:
|
||||
plugin_id, ordered_modes, self.on_demand_mode_index,
|
||||
ordered_modes[self.on_demand_mode_index] if ordered_modes else 'N/A')
|
||||
|
||||
def _load_plugin_for_on_demand(self, plugin_id: str) -> bool:
|
||||
"""Load an installed plugin that isn't running so on-demand can show it.
|
||||
|
||||
This process only loads the plugins enabled in config, so a request
|
||||
for a disabled one -- the config page's "Preview on display" button
|
||||
offers it on every plugin -- failed with "invalid-mode" while the UI
|
||||
said the plugin would be enabled for the session. Nothing did that
|
||||
short of a restart, and restarts no longer happen on a request.
|
||||
|
||||
Loads through the same path as a live enable (load_plugin, then
|
||||
_register_loaded_plugin), with force_enabled so the instance runs
|
||||
enabled while config.json keeps saying disabled. The plugin is
|
||||
recorded in _on_demand_loaded_plugins, and the main loop unloads it
|
||||
once on-demand moves off it (_release_on_demand_plugins).
|
||||
|
||||
Returns False after publishing an error when the load fails. A
|
||||
plugin that isn't installed returns True without loading anything:
|
||||
the mode checks that follow report it as they always have.
|
||||
"""
|
||||
if self.plugin_manager is None:
|
||||
return True
|
||||
try:
|
||||
known = self.plugin_manager.discovered_plugin_ids()
|
||||
except AttributeError:
|
||||
known = set(getattr(self.plugin_manager, 'plugin_manifests', ()) or ())
|
||||
if plugin_id not in known:
|
||||
# Installed after this process scanned: the web process checked
|
||||
# its own, fresher list before posting the request.
|
||||
try:
|
||||
known = set(self.plugin_manager.discover_plugins())
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.exception("On-demand: plugin discovery failed")
|
||||
known = set()
|
||||
if plugin_id not in known:
|
||||
return True
|
||||
|
||||
logger.info("On-demand: loading disabled plugin '%s' for this session only", plugin_id)
|
||||
self._on_demand_loaded_plugins.add(plugin_id)
|
||||
try:
|
||||
loaded = self.plugin_manager.load_plugin(plugin_id, force_enabled=True)
|
||||
if loaded:
|
||||
modes = self._register_loaded_plugin(plugin_id)
|
||||
logger.info("On-demand: loaded plugin '%s' (modes: %s)", plugin_id, modes)
|
||||
except Exception: # pylint: disable=broad-except
|
||||
logger.exception("On-demand: error loading plugin '%s'", plugin_id)
|
||||
loaded = False
|
||||
if not loaded:
|
||||
# Stays in _on_demand_loaded_plugins so the main loop removes
|
||||
# whatever part of it did get registered.
|
||||
logger.error("On-demand: could not load plugin '%s'", plugin_id)
|
||||
self._set_on_demand_error("load-failed")
|
||||
return False
|
||||
return True
|
||||
|
||||
def _release_on_demand_plugins(self) -> None:
|
||||
"""Unload plugins loaded only for on-demand that it has moved off.
|
||||
|
||||
Runs from the main loop, right after its own on-demand poll, not
|
||||
where on-demand ends: a stop, an expiry or the next request is often
|
||||
read from inside a render loop or a dwell sleep, where the plugin
|
||||
being released may still be on the stack mid-display(). Unloading
|
||||
goes through _unregister_plugin, as a live disable does, and nothing
|
||||
is written to config.json.
|
||||
|
||||
A plugin the user enabled in the meantime stays loaded and takes its
|
||||
place in the rotation, which is what the reconcile that the enable
|
||||
queued would have done.
|
||||
"""
|
||||
if self.plugin_manager is None: # plugin system failed after startup restore
|
||||
self._on_demand_loaded_plugins.clear()
|
||||
return
|
||||
keep = self.on_demand_plugin_id if self.on_demand_active else None
|
||||
releasable = [p for p in self._on_demand_loaded_plugins if p != keep]
|
||||
if not releasable:
|
||||
return
|
||||
try:
|
||||
config = self.config_service.get_config()
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
logger.warning("On-demand release: falling back to cached config: %s", e)
|
||||
config = self.config
|
||||
previous_mode = self.current_display_mode
|
||||
for plugin_id in releasable:
|
||||
self._on_demand_loaded_plugins.discard(plugin_id)
|
||||
section = config.get(plugin_id)
|
||||
if isinstance(section, dict) and section.get('enabled', False):
|
||||
logger.info("On-demand: keeping plugin '%s' loaded; it was enabled "
|
||||
"while on-demand showed it", plugin_id)
|
||||
continue
|
||||
if (plugin_id in self.plugin_display_modes
|
||||
or self.plugin_manager.get_plugin(plugin_id) is not None):
|
||||
logger.info("On-demand: unloading plugin '%s'; it is disabled in config",
|
||||
plugin_id)
|
||||
self._unregister_plugin(plugin_id)
|
||||
if not self.on_demand_active:
|
||||
# Only outside a session: rotation_resume_index points into
|
||||
# available_modes until the session ends.
|
||||
self._apply_plugin_rotation_order()
|
||||
self._resync_mode_index_after_change(previous_mode)
|
||||
if self.current_display_mode != previous_mode:
|
||||
self.force_change = True
|
||||
|
||||
def _rotation_index_outside_on_demand(self, start: int) -> Optional[int]:
|
||||
"""First index from `start` (wrapping) whose mode is not owned by a
|
||||
plugin loaded only for on-demand, or None if every mode is.
|
||||
|
||||
Ending a session must not resume the rotation onto the plugin that
|
||||
is about to be unloaded. A live load appends that plugin's modes
|
||||
after the saved resume index, but a session restored after a
|
||||
restart has no saved index and its plugin was ordered in with the
|
||||
rest -- the rotation resumed onto it, and a stop read during its own
|
||||
screen changed nothing on the panel until that screen ended.
|
||||
"""
|
||||
if not self._on_demand_loaded_plugins:
|
||||
return start
|
||||
on_demand_only = {mode for plugin_id in self._on_demand_loaded_plugins
|
||||
for mode in self.plugin_display_modes.get(plugin_id, [])}
|
||||
count = len(self.available_modes)
|
||||
for step in range(count):
|
||||
index = (start + step) % count
|
||||
if self.available_modes[index] not in on_demand_only:
|
||||
return index
|
||||
return None
|
||||
|
||||
def _activate_on_demand(self, request: Dict[str, Any]) -> None:
|
||||
"""Activate on-demand mode for a specific plugin display."""
|
||||
plugin_id = request.get('plugin_id')
|
||||
mode = request.get('mode')
|
||||
if (plugin_id and plugin_id not in self.plugin_display_modes
|
||||
and not self._load_plugin_for_on_demand(plugin_id)):
|
||||
return
|
||||
resolved_mode = self._resolve_mode_for_plugin(plugin_id, mode)
|
||||
|
||||
if not resolved_mode:
|
||||
@@ -1866,6 +2040,15 @@ class DisplayController:
|
||||
self.on_demand_last_event = 'stop-request-ignored' # Already idle
|
||||
self._publish_on_demand_state()
|
||||
return
|
||||
if not self.on_demand_active and self.on_demand_status == 'error':
|
||||
# _set_on_demand_error already ended any session and dropped
|
||||
# rotation_resume_index; the full clear below would only move
|
||||
# the rotation and force a redraw. Just drop the error.
|
||||
self.on_demand_status = 'idle'
|
||||
self.on_demand_last_error = None
|
||||
self.on_demand_last_event = reason or 'cleared'
|
||||
self._publish_on_demand_state()
|
||||
return
|
||||
|
||||
self._reset_on_demand_fields()
|
||||
self.on_demand_status = 'idle'
|
||||
@@ -1875,17 +2058,27 @@ class DisplayController:
|
||||
# Clear on-demand configuration from cache
|
||||
self.cache_manager.clear_cache('display_on_demand_config')
|
||||
|
||||
if self.rotation_resume_index is not None and self.available_modes:
|
||||
self.current_mode_index = self.rotation_resume_index % len(self.available_modes)
|
||||
self.current_display_mode = self.available_modes[self.current_mode_index]
|
||||
logger.info("Resuming rotation from saved index %d: mode '%s'",
|
||||
self.rotation_resume_index, self.current_display_mode)
|
||||
elif self.available_modes:
|
||||
# Default to first mode if no resume index
|
||||
self.current_mode_index = self.current_mode_index % len(self.available_modes)
|
||||
self.current_display_mode = self.available_modes[self.current_mode_index]
|
||||
logger.info("Resuming rotation to mode '%s' (index %d)",
|
||||
self.current_display_mode, self.current_mode_index)
|
||||
if self.available_modes:
|
||||
saved = self.rotation_resume_index
|
||||
# Default to the current index if no resume index
|
||||
start = saved if saved is not None else self.current_mode_index
|
||||
index = self._rotation_index_outside_on_demand(start % len(self.available_modes))
|
||||
if index is None:
|
||||
# Every mode belongs to a plugin loaded only for on-demand,
|
||||
# which the main loop is about to unload; it then idles.
|
||||
self.current_mode_index = 0
|
||||
self.current_display_mode = None
|
||||
logger.info("No enabled mode to resume rotation to")
|
||||
elif saved is not None:
|
||||
self.current_mode_index = index
|
||||
self.current_display_mode = self.available_modes[index]
|
||||
logger.info("Resuming rotation from saved index %d: mode '%s'",
|
||||
saved, self.current_display_mode)
|
||||
else:
|
||||
self.current_mode_index = index
|
||||
self.current_display_mode = self.available_modes[index]
|
||||
logger.info("Resuming rotation to mode '%s' (index %d)",
|
||||
self.current_display_mode, self.current_mode_index)
|
||||
else:
|
||||
logger.warning("No available modes to resume rotation to")
|
||||
|
||||
@@ -2051,6 +2244,11 @@ class DisplayController:
|
||||
"plugin is enabled via the web UI."
|
||||
)
|
||||
|
||||
# This thread is the one the systemd watchdog and the heartbeat
|
||||
# vouch for: beats from any other thread are ignored, so a render
|
||||
# thread stuck inside a plugin stops them.
|
||||
display_watchdog.watchdog.bind_render_thread()
|
||||
|
||||
try:
|
||||
# Initialize with cached data for fast startup - let background updates refresh naturally
|
||||
logger.info("Starting display with cached data (fast startup mode)")
|
||||
@@ -2059,6 +2257,11 @@ class DisplayController:
|
||||
self._publish_current_mode_state()
|
||||
|
||||
while True:
|
||||
# Arms the watchdog after the first frame -- or after the
|
||||
# first full pass, when there is nothing to draw -- and pings
|
||||
# it from then on.
|
||||
display_watchdog.watchdog.loop_pass()
|
||||
|
||||
# Apply plugin enable/disable edits saved via the web UI. The
|
||||
# config-watcher thread only sets the flag; loading/unloading and
|
||||
# rebuilding available_modes happens here on the render thread so
|
||||
@@ -2082,6 +2285,14 @@ class DisplayController:
|
||||
# Handle on-demand commands before rendering
|
||||
self._poll_on_demand_requests()
|
||||
self._check_on_demand_expiration()
|
||||
# Unload plugins loaded only to show them on-demand once it
|
||||
# has moved off them. Here, where no display() is on the
|
||||
# stack; one ended from inside a screen is caught here on
|
||||
# the next pass.
|
||||
if self._on_demand_loaded_plugins:
|
||||
self._release_on_demand_plugins()
|
||||
if not self.available_modes:
|
||||
continue # it was all there was; idle as above
|
||||
self._tick_plugin_updates()
|
||||
|
||||
# Clean up expired WiFi status messages
|
||||
@@ -2328,6 +2539,7 @@ class DisplayController:
|
||||
pm = self.plugin_manager
|
||||
display_lock = pm.get_plugin_lock(plugin_id) if pm else None
|
||||
can_display = display_lock is None or display_lock.acquire(blocking=False)
|
||||
display_hung = False
|
||||
|
||||
if display_lock is None:
|
||||
# Only when plugin loading failed part-way.
|
||||
@@ -2347,7 +2559,7 @@ class DisplayController:
|
||||
# thread actually finishes it, rather than
|
||||
# here when this dispatch merely returns.
|
||||
release_guard = threading.Lock()
|
||||
released = {'done': False}
|
||||
released = {'done': False, 'started': False}
|
||||
|
||||
def _release_display_lock():
|
||||
with release_guard:
|
||||
@@ -2358,6 +2570,7 @@ class DisplayController:
|
||||
|
||||
if _accepts_display_mode:
|
||||
def _display_target(display_mode=None, force_clear=False):
|
||||
released['started'] = True
|
||||
try:
|
||||
return manager_to_display.display(
|
||||
display_mode=display_mode, force_clear=force_clear)
|
||||
@@ -2365,11 +2578,13 @@ class DisplayController:
|
||||
_release_display_lock()
|
||||
else:
|
||||
def _display_target(force_clear=False):
|
||||
released['started'] = True
|
||||
try:
|
||||
return manager_to_display.display(force_clear=force_clear)
|
||||
finally:
|
||||
_release_display_lock()
|
||||
|
||||
dispatch_start = time.monotonic()
|
||||
try:
|
||||
result = pm.plugin_executor.execute_display(
|
||||
types.SimpleNamespace(display=_display_target),
|
||||
@@ -2392,6 +2607,18 @@ class DisplayController:
|
||||
_release_display_lock()
|
||||
raise
|
||||
|
||||
dispatch_seconds = time.monotonic() - dispatch_start
|
||||
if released['started'] and not released['done']:
|
||||
# The executor gave up waiting and display()
|
||||
# is still running on its thread, holding
|
||||
# the lock. A hang, not a success: recorded
|
||||
# so repeats open the circuit breaker, and
|
||||
# the update worker's bounded wait skips it.
|
||||
display_hung = True
|
||||
pm.record_display_hang(plugin_id, dispatch_seconds)
|
||||
else:
|
||||
pm.note_display_duration(plugin_id, dispatch_seconds)
|
||||
|
||||
logger.debug(f"display() returned: {result} (type: {type(result)})")
|
||||
if isinstance(result, bool):
|
||||
display_result = result
|
||||
@@ -2405,7 +2632,7 @@ class DisplayController:
|
||||
# be lost when display() finally does run.
|
||||
if can_display:
|
||||
health_tracker = self._health_tracker()
|
||||
if health_tracker is not None:
|
||||
if health_tracker is not None and not display_hung:
|
||||
health_tracker.record_success(plugin_id)
|
||||
self.force_change = False
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
@@ -3099,8 +3326,23 @@ class DisplayController:
|
||||
prepared = prepare(_pid, new_config) if callable(prepare) else None
|
||||
if isinstance(prepared, dict):
|
||||
new_config = prepared
|
||||
_plugin.on_config_change(new_config)
|
||||
logger.debug("Plugin %s notified of config change", _pid)
|
||||
if _pid in self._on_demand_loaded_plugins:
|
||||
# Saved while on-demand shows it: config.json still
|
||||
# says disabled, and on_config_change would switch
|
||||
# the instance off mid-session.
|
||||
new_config = {**new_config, 'enabled': True}
|
||||
# Runs on ConfigService's watcher thread. Under the
|
||||
# plugin's lock, so it cannot interleave with update()
|
||||
# on the worker or display() on the render thread; a
|
||||
# lock held past the bound defers it to the worker.
|
||||
apply = getattr(self.plugin_manager, 'apply_config_change', None)
|
||||
if callable(apply):
|
||||
applied = apply(_pid, new_config, plugin_instance=_plugin)
|
||||
else:
|
||||
_plugin.on_config_change(new_config)
|
||||
applied = True
|
||||
logger.debug("Plugin %s notified of config change%s", _pid,
|
||||
"" if applied else " (deferred: plugin busy)")
|
||||
except Exception as e:
|
||||
logger.error("Error in plugin %s config change handler: %s", _pid, e, exc_info=True)
|
||||
|
||||
@@ -3399,6 +3641,9 @@ class DisplayController:
|
||||
|
||||
def cleanup(self):
|
||||
"""Clean up resources."""
|
||||
# First: a clean stop is not a hang, and a heartbeat left behind
|
||||
# would read as a frozen panel to the web interface.
|
||||
display_watchdog.watchdog.stopping()
|
||||
# Stop the async update worker first so no in-flight update() call
|
||||
# is still touching display/cache-backed resources while they're
|
||||
# torn down below.
|
||||
@@ -3429,6 +3674,12 @@ class DisplayController:
|
||||
logger.warning("Error shutting down config service: %s", e)
|
||||
if getattr(self, '_font_usage_publisher', None) is not None:
|
||||
self._font_usage_publisher.stop()
|
||||
# Publishes "stopped", so the web UI stops reporting what was loaded.
|
||||
if getattr(self, '_plugin_runtime_publisher', None) is not None:
|
||||
try:
|
||||
self._plugin_runtime_publisher.stop()
|
||||
except Exception as e:
|
||||
logger.warning("Error stopping the plugin runtime publisher: %s", e)
|
||||
logger.info("Cleaning up display controller...")
|
||||
if hasattr(self, 'display_manager'):
|
||||
self.display_manager.cleanup()
|
||||
|
||||
+11
-7
@@ -57,6 +57,7 @@ import zlib
|
||||
import freetype
|
||||
|
||||
from src.common import snapshot_policy
|
||||
from src import display_watchdog
|
||||
from src.common.frame_timing import FrameTimingRecorder
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -928,6 +929,9 @@ class DisplayManager:
|
||||
# the fallback branch, so captured content never reaches the
|
||||
# web preview either.
|
||||
return
|
||||
# The render loop's watchdog arms on the first frame to reach
|
||||
# the panel (or the emulator/fallback path standing in for it).
|
||||
display_watchdog.note_frame()
|
||||
with self._update_lock:
|
||||
if self.matrix is None:
|
||||
# Fallback mode - no actual hardware to update
|
||||
@@ -1320,7 +1324,7 @@ class DisplayManager:
|
||||
except Exception as e:
|
||||
logger.error(f"Error drawing text: {e}", exc_info=True)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def draw_sun(self, x: int, y: int, size: int = 16):
|
||||
"""Draw a sun icon using yellow circles and lines."""
|
||||
center = (x + size//2, y + size//2)
|
||||
@@ -1341,7 +1345,7 @@ class DisplayManager:
|
||||
end_y = center[1] + ((radius + ray_length) * math.sin(rad))
|
||||
self.draw.line([start_x, start_y, end_x, end_y], fill=(255, 255, 0), width=2)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def draw_cloud(self, x: int, y: int, size: int = 16, color=(200, 200, 200)):
|
||||
"""Draw a cloud icon."""
|
||||
# Draw multiple circles to form a cloud shape
|
||||
@@ -1349,7 +1353,7 @@ class DisplayManager:
|
||||
self.draw.ellipse([x+size//2, y+size//3, x+size//2+size//2, y+size//3+size//2], fill=color)
|
||||
self.draw.ellipse([x+size//3, y+size//6, x+size//3+size//2, y+size//6+size//2], fill=color)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def draw_rain(self, x: int, y: int, size: int = 16):
|
||||
"""Draw rain icon with cloud and droplets."""
|
||||
# Draw cloud
|
||||
@@ -1364,7 +1368,7 @@ class DisplayManager:
|
||||
self.draw.line([drop_x, drop_y, drop_x, drop_y+drop_size],
|
||||
fill=drop_color, width=2)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def draw_snow(self, x: int, y: int, size: int = 16):
|
||||
"""Draw snow icon with cloud and snowflakes."""
|
||||
# Draw cloud
|
||||
@@ -1485,7 +1489,7 @@ class DisplayManager:
|
||||
]
|
||||
self.draw.polygon(bolt_points, fill=bolt_color)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def draw_weather_icon(self, condition: str, x: int, y: int, size: int = 16) -> None:
|
||||
"""Draw a weather icon based on the condition."""
|
||||
if condition.lower() in ['clear', 'sunny']:
|
||||
@@ -1502,7 +1506,7 @@ class DisplayManager:
|
||||
self._draw_sun(x, y, size)
|
||||
# Note: No update_display() here - let the caller handle the update
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def draw_text_with_icons(self, text: str, icons: List[tuple] = None, x: int = None, y: int = None,
|
||||
color: tuple = (255, 255, 255)):
|
||||
"""Draw text with weather icons at specified positions."""
|
||||
@@ -1828,7 +1832,7 @@ class DisplayManager:
|
||||
if removed_count > 0:
|
||||
logger.debug(f"Cleaned up {removed_count} expired deferred updates")
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_scrolling_stats(self) -> dict:
|
||||
"""Get current scrolling statistics for debugging."""
|
||||
return {
|
||||
|
||||
@@ -0,0 +1,414 @@
|
||||
"""Render-loop liveness: systemd watchdog pings and a heartbeat file.
|
||||
|
||||
A panel can freeze while ``ledmatrix.service`` stays "active": a plugin's
|
||||
``display()`` that never returns, a deadlock, a stuck hardware swap. Nothing
|
||||
outside the process could tell, so nothing restarted it. This module lets the
|
||||
render loop prove it is still going round, in two ways:
|
||||
|
||||
* **systemd watchdog.** The unit sets ``WatchdogSec=`` and
|
||||
``NotifyAccess=main``; this sends ``WATCHDOG=1`` over ``$NOTIFY_SOCKET``.
|
||||
When the pings stop, systemd kills the process (SIGABRT, so faulthandler
|
||||
prints every thread's stack to the journal first) and ``Restart=`` brings it
|
||||
back.
|
||||
* **Heartbeat file**, ``/run/ledmatrix/display-heartbeat.json``, for the web
|
||||
interface's ``/api/v3/health`` and the automatic update's health check.
|
||||
``/run`` is tmpfs, so the writes never reach the SD card.
|
||||
|
||||
Both are driven only from the render thread -- ``beat()`` from any other
|
||||
thread is ignored -- so a render thread stuck inside a plugin stops them even
|
||||
while every other thread carries on. The unit's ``WatchdogSec=`` is the
|
||||
steady-state limit; start-up (plugin loads, the 20s initial update budget,
|
||||
dependency installs) is far longer and happens before the render loop exists,
|
||||
so ``begin_startup()`` widens the limit for it and the render loop narrows it
|
||||
back, sends ``READY=1`` and starts pinging once its first frame is on the
|
||||
panel. See docs/ARCHITECTURE.md ("Liveness") for the unit settings.
|
||||
|
||||
Standard library only, and no import of the rest of ``src``: ``run.py`` loads
|
||||
this before anything heavy so the start-up allowance is in place long before
|
||||
the unit's own ``WatchdogSec`` could expire. Without ``$NOTIFY_SOCKET`` (dev
|
||||
server, emulator, Windows, an older unit) every call is a cheap no-op, and the
|
||||
heartbeat is written only where ``/run/ledmatrix`` exists or can be created.
|
||||
"""
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import socket
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Callable, Dict, Iterator, Mapping, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Where the display writes its heartbeat. ``RuntimeDirectory=ledmatrix`` in
|
||||
#: the unit creates the directory; a display running under an older unit
|
||||
#: creates it itself (it runs as root). The web interface, which is not root,
|
||||
#: only reads it: the directory is 0755 and the file 0644.
|
||||
HEARTBEAT_DIR = '/run/ledmatrix'
|
||||
HEARTBEAT_NAME = 'display-heartbeat.json'
|
||||
HEARTBEAT_PATH = HEARTBEAT_DIR + '/' + HEARTBEAT_NAME
|
||||
|
||||
#: How often the render loop pings systemd and rewrites the heartbeat. Beats
|
||||
#: come many times a second; this is the rate limit on the side effects.
|
||||
BEAT_INTERVAL_SECONDS = 5.0
|
||||
|
||||
#: A heartbeat older than this means the render loop has stopped. Above the
|
||||
#: longest gap a healthy loop has (the executor's 30s display() timeout), so a
|
||||
#: slow plugin does not read as a frozen panel.
|
||||
HEARTBEAT_STALE_SECONDS = 60.0
|
||||
|
||||
#: The watchdog limit while the process starts, before the render loop runs.
|
||||
#: Start-up loads every plugin (pip included, when a dependency is missing:
|
||||
#: up to 300s a try), then spends up to 20s on initial updates. A hang in
|
||||
#: there is still caught, just later.
|
||||
STARTUP_ALLOWANCE_SECONDS = 15 * 60
|
||||
|
||||
#: The watchdog limit while the render thread loads a plugin that was just
|
||||
#: enabled from the web UI: loading can run pip, on this thread.
|
||||
PLUGIN_LOAD_ALLOWANCE_SECONDS = 15 * 60
|
||||
|
||||
|
||||
# -- sd_notify -------------------------------------------------------------
|
||||
|
||||
def notify(message: str, environ: Optional[Mapping[str, str]] = None,
|
||||
socket_factory: Optional[Callable[..., Any]] = None) -> bool:
|
||||
"""Send ``message`` to systemd over ``$NOTIFY_SOCKET``; True if it was sent.
|
||||
|
||||
The same protocol as libsystemd's ``sd_notify()``: one datagram of
|
||||
newline-separated ``KEY=VALUE`` lines to an AF_UNIX socket. An address
|
||||
starting with ``@`` is in the abstract namespace (a leading NUL byte).
|
||||
Never raises: a missing socket or a failed send is simply False, so a
|
||||
display run outside systemd behaves exactly as before.
|
||||
"""
|
||||
env = os.environ if environ is None else environ
|
||||
address = env.get('NOTIFY_SOCKET') or ''
|
||||
if address.startswith('@'):
|
||||
address = '\0' + address[1:]
|
||||
elif not address.startswith('/'):
|
||||
# Unset, or a vsock: address (systemd 253+, VMs only).
|
||||
return False
|
||||
family = getattr(socket, 'AF_UNIX', None)
|
||||
if family is None:
|
||||
return False
|
||||
factory = socket_factory or socket.socket
|
||||
try:
|
||||
sock = factory(family, socket.SOCK_DGRAM | getattr(socket, 'SOCK_CLOEXEC', 0))
|
||||
try:
|
||||
sock.connect(address)
|
||||
sock.sendall(message.encode('utf-8'))
|
||||
finally:
|
||||
sock.close()
|
||||
return True
|
||||
except OSError as e:
|
||||
logger.debug("sd_notify(%r) failed: %s", message, e)
|
||||
return False
|
||||
|
||||
|
||||
def watchdog_usec(environ: Optional[Mapping[str, str]] = None) -> Optional[int]:
|
||||
"""The unit's ``WatchdogSec`` in microseconds, or None when it has none.
|
||||
|
||||
systemd passes it as ``$WATCHDOG_USEC``, with ``$WATCHDOG_PID`` naming the
|
||||
process it is meant for (a child that inherited the environment must not
|
||||
think the watchdog is its own).
|
||||
"""
|
||||
env = os.environ if environ is None else environ
|
||||
pid = env.get('WATCHDOG_PID')
|
||||
if pid and pid != str(os.getpid()):
|
||||
return None
|
||||
try:
|
||||
usec = int(env.get('WATCHDOG_USEC', ''))
|
||||
except ValueError:
|
||||
return None
|
||||
return usec if usec > 0 else None
|
||||
|
||||
|
||||
# -- heartbeat reading (web interface) ---------------------------------------
|
||||
|
||||
def read_heartbeat(path: str = HEARTBEAT_PATH) -> Optional[Dict[str, Any]]:
|
||||
"""The heartbeat the display last wrote, or None when there is none.
|
||||
|
||||
None covers a display that does not write one -- dev server, emulator,
|
||||
Windows, a display that has not drawn its first frame yet -- as well as an
|
||||
unreadable file, so callers fall back to whatever they did before.
|
||||
"""
|
||||
try:
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return data if isinstance(data, dict) else None
|
||||
|
||||
|
||||
def heartbeat_age(data: Mapping[str, Any], now_mono: Optional[float] = None,
|
||||
now_wall: Optional[float] = None) -> Optional[float]:
|
||||
"""Seconds since the heartbeat in ``data`` was written, or None if it has no time.
|
||||
|
||||
Measured on the monotonic clock when it can be: on Linux that is
|
||||
CLOCK_MONOTONIC, shared by every process, and it does not jump when NTP
|
||||
first corrects the clock of a Pi with no RTC. /run is emptied at boot, so
|
||||
a heartbeat always comes from this boot. Falls back to the wall clock.
|
||||
"""
|
||||
now_mono = time.monotonic() if now_mono is None else now_mono
|
||||
now_wall = time.time() if now_wall is None else now_wall
|
||||
mono = data.get('mono')
|
||||
if isinstance(mono, (int, float)) and not isinstance(mono, bool):
|
||||
age = now_mono - mono
|
||||
if age >= -1.0: # a clock this far behind is not the same clock
|
||||
return max(age, 0.0)
|
||||
wall = data.get('wall')
|
||||
if isinstance(wall, (int, float)) and not isinstance(wall, bool):
|
||||
return max(now_wall - wall, 0.0)
|
||||
return None
|
||||
|
||||
|
||||
# -- the render loop's side ----------------------------------------------------
|
||||
|
||||
_DEFAULT_DIR = object()
|
||||
|
||||
|
||||
class RenderWatchdog:
|
||||
"""Pings systemd and writes the heartbeat, from the render thread only.
|
||||
|
||||
Lifecycle: ``begin_startup()`` as the process starts, ``bind_render_thread()``
|
||||
when ``DisplayController.run()`` starts, then ``note_frame()`` for every
|
||||
frame pushed to the panel and ``beat()`` / ``loop_pass()`` from every place
|
||||
the render loop reliably comes back to. The first beat after the first
|
||||
frame (or after the loop's first full pass, when there is nothing to draw)
|
||||
arms it: ``READY=1``, the unit's own ``WatchdogSec``, and the heartbeat.
|
||||
"""
|
||||
|
||||
def __init__(self, environ: Optional[Mapping[str, str]] = None,
|
||||
send: Optional[Callable[[str], bool]] = None,
|
||||
clock: Callable[[], float] = time.monotonic,
|
||||
wall_clock: Callable[[], float] = time.time,
|
||||
heartbeat_dir: Any = _DEFAULT_DIR,
|
||||
enable_faulthandler: bool = True):
|
||||
env = dict(os.environ if environ is None else environ)
|
||||
self._send = send or (lambda message: notify(message, env))
|
||||
self._clock = clock
|
||||
self._wall_clock = wall_clock
|
||||
self._usec = watchdog_usec(env)
|
||||
if heartbeat_dir is _DEFAULT_DIR:
|
||||
# /run exists only on Linux; elsewhere (Windows dev) there is no
|
||||
# heartbeat rather than a C:\run folder.
|
||||
heartbeat_dir = HEARTBEAT_DIR if os.name == 'posix' else None
|
||||
self._heartbeat_dir: Optional[str] = heartbeat_dir
|
||||
# None until the first write; False for good if that one failed
|
||||
# (nowhere to write: not root, no /run); True once one landed.
|
||||
self._heartbeat_ok: Optional[bool] = None
|
||||
self._heartbeat_warned = False
|
||||
self._enable_faulthandler = enable_faulthandler
|
||||
self._render_thread: Optional[int] = None
|
||||
self._frame_pushed = False
|
||||
self._passes = 0
|
||||
self._armed = False
|
||||
self._last_beat: Optional[float] = None
|
||||
self._extend_depth = 0
|
||||
interval = BEAT_INTERVAL_SECONDS
|
||||
if self._usec:
|
||||
# systemd's advice is to ping at half the limit; a third leaves
|
||||
# room for one late beat even if someone sets a very short one.
|
||||
interval = min(interval, self._usec / 1e6 / 3)
|
||||
self._interval = interval
|
||||
|
||||
@property
|
||||
def armed(self) -> bool:
|
||||
return self._armed
|
||||
|
||||
def _on_render_thread(self) -> bool:
|
||||
return self._render_thread is not None and threading.get_ident() == self._render_thread
|
||||
|
||||
def begin_startup(self) -> None:
|
||||
"""Widen the watchdog to cover start-up. Call as early as possible.
|
||||
|
||||
systemd starts the watchdog clock when a Type=simple service starts,
|
||||
and start-up routinely takes longer than the render loop's limit.
|
||||
Only widens: an operator who set a longer ``WatchdogSec`` keeps it.
|
||||
"""
|
||||
if not self._usec:
|
||||
return
|
||||
allowance = max(self._usec, int(STARTUP_ALLOWANCE_SECONDS * 1e6))
|
||||
self._send(f'WATCHDOG_USEC={allowance}\nSTATUS=Starting: loading plugins')
|
||||
|
||||
def bind_render_thread(self) -> None:
|
||||
"""Mark the calling thread as the render thread; beats from others are ignored."""
|
||||
self._render_thread = threading.get_ident()
|
||||
self._frame_pushed = False
|
||||
self._passes = 0
|
||||
|
||||
def note_frame(self) -> None:
|
||||
"""A frame was pushed to the panel (DisplayManager.update_display).
|
||||
|
||||
Any thread may push the first one -- the first dispatch of a screen
|
||||
runs on PluginExecutor's thread -- so this only records it; the
|
||||
render thread's next beat arms the watchdog.
|
||||
"""
|
||||
if self._render_thread is None:
|
||||
return # start-up screens, before the render loop exists
|
||||
self._frame_pushed = True
|
||||
if self._on_render_thread():
|
||||
self.beat()
|
||||
|
||||
def loop_pass(self) -> None:
|
||||
"""The top of the render loop's ``while True``.
|
||||
|
||||
A second arrival here means a whole pass finished. That counts as the
|
||||
first frame when there was nothing to draw (no plugins enabled, every
|
||||
screen empty): the loop is plainly alive, and a watchdog that never
|
||||
armed would leave a later hang uncaught.
|
||||
"""
|
||||
if not self._on_render_thread():
|
||||
return
|
||||
self._passes += 1
|
||||
if self._passes > 1:
|
||||
self._frame_pushed = True
|
||||
self.beat()
|
||||
|
||||
def beat(self) -> None:
|
||||
"""The render loop is still going round. Cheap; call it freely."""
|
||||
if not self._on_render_thread():
|
||||
return
|
||||
if not self._armed:
|
||||
if not self._frame_pushed:
|
||||
return
|
||||
self._arm()
|
||||
return
|
||||
now = self._clock()
|
||||
if self._last_beat is not None and now - self._last_beat < self._interval:
|
||||
return
|
||||
self._last_beat = now
|
||||
if self._usec:
|
||||
self._send('WATCHDOG=1')
|
||||
self._write_heartbeat(now)
|
||||
|
||||
def _arm(self) -> None:
|
||||
self._armed = True
|
||||
self._last_beat = self._clock()
|
||||
if self._usec:
|
||||
# Back from the start-up allowance to the unit's own limit.
|
||||
self._send(f'READY=1\nWATCHDOG_USEC={self._usec}\nWATCHDOG=1\nSTATUS=Rendering')
|
||||
self._install_faulthandler()
|
||||
logger.info("systemd watchdog armed: the render loop must check in every %.0fs",
|
||||
self._usec / 1e6)
|
||||
else:
|
||||
self._send('READY=1\nSTATUS=Rendering')
|
||||
self._write_heartbeat(self._last_beat)
|
||||
|
||||
def _install_faulthandler(self) -> None:
|
||||
"""Dump every thread's stack when the watchdog's SIGABRT arrives.
|
||||
|
||||
That trace, in the journal, is what says which plugin the render
|
||||
thread was stuck in.
|
||||
"""
|
||||
if not self._enable_faulthandler:
|
||||
return
|
||||
try:
|
||||
import faulthandler
|
||||
import sys
|
||||
if not faulthandler.is_enabled() and sys.stderr is not None:
|
||||
faulthandler.enable(all_threads=True)
|
||||
except (ImportError, RuntimeError, ValueError, OSError, AttributeError) as e:
|
||||
logger.debug("faulthandler not enabled: %s", e)
|
||||
|
||||
@contextlib.contextmanager
|
||||
def extended(self, seconds: float, reason: str = '') -> Iterator[None]:
|
||||
"""Allow the render thread ``seconds`` for one blocking job.
|
||||
|
||||
For the few legitimate jobs that can outlast the watchdog, such as
|
||||
loading a newly enabled plugin, which can run pip on this thread.
|
||||
Nests; the unit's limit comes back when the outermost one ends.
|
||||
"""
|
||||
if not (self._armed and self._usec and self._on_render_thread()):
|
||||
yield
|
||||
return
|
||||
usec = max(self._usec, int(seconds * 1e6))
|
||||
if self._extend_depth == 0:
|
||||
self._send(f'WATCHDOG_USEC={usec}\nWATCHDOG=1'
|
||||
+ (f'\nSTATUS=Busy: {reason}' if reason else ''))
|
||||
self._extend_depth += 1
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
self._extend_depth -= 1
|
||||
if self._extend_depth == 0:
|
||||
self._send(f'WATCHDOG_USEC={self._usec}\nWATCHDOG=1\nSTATUS=Rendering')
|
||||
self._last_beat = self._clock()
|
||||
self._write_heartbeat(self._last_beat)
|
||||
|
||||
def stopping(self) -> None:
|
||||
"""Clean shutdown: tell systemd, and take the heartbeat down with us.
|
||||
|
||||
A heartbeat left behind by a stopped display would read as a frozen
|
||||
one to the web interface.
|
||||
"""
|
||||
if self._usec or self._armed:
|
||||
self._send('STOPPING=1')
|
||||
path = self._heartbeat_path()
|
||||
if path and self._heartbeat_ok:
|
||||
try:
|
||||
os.unlink(path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# -- heartbeat file ----------------------------------------------------
|
||||
|
||||
def _heartbeat_path(self) -> Optional[str]:
|
||||
if not self._heartbeat_dir:
|
||||
return None
|
||||
return os.path.join(self._heartbeat_dir, HEARTBEAT_NAME)
|
||||
|
||||
def _write_heartbeat(self, now_mono: float) -> None:
|
||||
path = self._heartbeat_path()
|
||||
if path is None or self._heartbeat_ok is False:
|
||||
return
|
||||
directory = self._heartbeat_dir
|
||||
try:
|
||||
if not os.path.isdir(directory):
|
||||
# An install whose unit predates RuntimeDirectory=: the
|
||||
# display runs as root and can make it. Anyone else cannot,
|
||||
# and gets no heartbeat -- which readers treat as "unknown".
|
||||
os.makedirs(directory, mode=0o755, exist_ok=True)
|
||||
payload = json.dumps({'pid': os.getpid(), 'mono': now_mono,
|
||||
'wall': self._wall_clock()})
|
||||
fd, tmp = tempfile.mkstemp(dir=directory, prefix='.heartbeat-')
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
f.write(payload)
|
||||
os.chmod(tmp, 0o644)
|
||||
os.replace(tmp, path)
|
||||
except BaseException:
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
if self._heartbeat_ok is None:
|
||||
logger.info("Writing the display heartbeat to %s", path)
|
||||
self._heartbeat_ok = True
|
||||
except OSError as e:
|
||||
if self._heartbeat_ok is None:
|
||||
logger.info("Not writing a display heartbeat (%s: %s); health checks "
|
||||
"fall back to their older signals", directory, e)
|
||||
self._heartbeat_ok = False
|
||||
elif not self._heartbeat_warned:
|
||||
# It worked before, so keep trying, but say so only once.
|
||||
logger.warning("Could not update the display heartbeat: %s", e)
|
||||
self._heartbeat_warned = True
|
||||
|
||||
|
||||
#: The process-wide instance: one display process, one render loop.
|
||||
watchdog = RenderWatchdog()
|
||||
|
||||
|
||||
def beat() -> None:
|
||||
"""Module-level shortcut so the Vegas loop and the plugin manager need no reference."""
|
||||
watchdog.beat()
|
||||
|
||||
|
||||
def note_frame() -> None:
|
||||
watchdog.note_frame()
|
||||
|
||||
|
||||
def extended(seconds: float, reason: str = ''):
|
||||
return watchdog.extended(seconds, reason)
|
||||
+14
-14
@@ -187,7 +187,7 @@ class FontManager:
|
||||
if removed:
|
||||
self.manager_fonts_version += 1
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_manager_fonts(self, manager_id: Optional[str] = None) -> Dict[str, Any]:
|
||||
"""
|
||||
Get registered fonts for a specific manager or all managers.
|
||||
@@ -202,7 +202,7 @@ class FontManager:
|
||||
return self.manager_fonts.get(manager_id, {})
|
||||
return self.manager_fonts.copy()
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_detected_fonts(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get all detected font usage across managers."""
|
||||
return self.detected_fonts.copy()
|
||||
@@ -433,7 +433,7 @@ class FontManager:
|
||||
search_dirs = [Path(resolve_asset_path(configured)), Path(resolve_asset_path("plugins"))]
|
||||
return resolve_plugin_dir(plugin_id, search_dirs, prefix=True)
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def unregister_plugin_fonts(self, plugin_id: str) -> bool:
|
||||
"""Unregister all fonts for a plugin."""
|
||||
try:
|
||||
@@ -471,7 +471,7 @@ class FontManager:
|
||||
# Font objects someone may hold were dropped; see cache_generation.
|
||||
self.cache_generation += 1
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_plugin_fonts(self, plugin_id: str) -> List[str]:
|
||||
"""Get list of font families registered by a plugin."""
|
||||
if plugin_id in self.plugin_font_catalogs:
|
||||
@@ -670,7 +670,7 @@ class FontManager:
|
||||
|
||||
# ==================== Override Management ====================
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def set_override(self, element_key: str, family: str = None, size_px: int = None):
|
||||
"""Set font override for a specific element."""
|
||||
if element_key not in self.font_overrides:
|
||||
@@ -690,7 +690,7 @@ class FontManager:
|
||||
self.clear_cache()
|
||||
logger.info(f"Font override set for {element_key}: {self.font_overrides.get(element_key, {})}")
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def remove_override(self, element_key: str):
|
||||
"""Remove font override for a specific element."""
|
||||
if element_key in self.font_overrides:
|
||||
@@ -699,7 +699,7 @@ class FontManager:
|
||||
self.clear_cache()
|
||||
logger.info(f"Font override removed for {element_key}")
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_overrides(self) -> Dict[str, Dict[str, str]]:
|
||||
"""Get current font overrides."""
|
||||
return self.font_overrides.copy()
|
||||
@@ -787,17 +787,17 @@ class FontManager:
|
||||
self.cache_generation += 1
|
||||
logger.info("Font cache cleared")
|
||||
|
||||
@deprecated("3.7.0", "read font_catalog")
|
||||
@deprecated("3.8.0", "read font_catalog")
|
||||
def get_available_fonts(self) -> Dict[str, str]:
|
||||
"""Get dictionary of available font families and their paths."""
|
||||
return self.font_catalog.copy()
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_size_tokens(self) -> Dict[str, int]:
|
||||
"""Get available size tokens."""
|
||||
return self.size_tokens.copy()
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def get_performance_stats(self) -> Dict[str, Any]:
|
||||
"""Get performance statistics."""
|
||||
uptime = time.time() - self.performance_stats["start_time"]
|
||||
@@ -819,12 +819,12 @@ class FontManager:
|
||||
"detected_fonts": len(self.detected_fonts)
|
||||
}
|
||||
|
||||
@deprecated("3.7.0", "read font_catalog")
|
||||
@deprecated("3.8.0", "read font_catalog")
|
||||
def get_font_catalog(self) -> Dict[str, str]:
|
||||
"""Get the current font catalog."""
|
||||
return self.font_catalog.copy()
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def add_font(self, font_file_path: str, family_name: str) -> bool:
|
||||
"""Add ``font_file_path`` to the catalog as ``family_name``. The file
|
||||
stays where it is; only assets/fonts is created if it is missing."""
|
||||
@@ -852,7 +852,7 @@ class FontManager:
|
||||
logger.error(f"Error adding font {family_name}: {e}")
|
||||
return False
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def remove_font(self, family_name: str) -> bool:
|
||||
"""Remove a font from the catalog."""
|
||||
try:
|
||||
@@ -880,7 +880,7 @@ class FontManager:
|
||||
logger.error(f"Error removing font {family_name}: {e}")
|
||||
return False
|
||||
|
||||
@deprecated("3.7.0")
|
||||
@deprecated("3.8.0")
|
||||
def validate_font(self, font_path: str) -> Dict[str, Any]:
|
||||
"""Validate a font file."""
|
||||
try:
|
||||
|
||||
@@ -13,6 +13,7 @@ from enum import Enum
|
||||
from typing import Dict, Any, Optional, List
|
||||
import os
|
||||
import sys
|
||||
from src.deprecation import deprecated, warn_deprecated
|
||||
from src.logging_config import get_logger
|
||||
|
||||
|
||||
@@ -65,27 +66,178 @@ def _fallback_font_manager() -> Any:
|
||||
|
||||
class VegasDisplayMode(Enum):
|
||||
"""
|
||||
Display mode for Vegas scroll integration.
|
||||
Legacy display mode for Vegas scroll integration.
|
||||
|
||||
Determines how a plugin's content behaves within the continuous scroll:
|
||||
Superseded by :meth:`BasePlugin.get_vegas_participation`. Vegas still
|
||||
reads a plugin's :meth:`BasePlugin.get_vegas_display_mode` to derive its
|
||||
participation when nothing declares one, and only STATIC matters there:
|
||||
|
||||
- SCROLL: Content scrolls continuously within the stream.
|
||||
Best for multi-item plugins like sports scores, odds tickers, news feeds.
|
||||
Plugin provides multiple frames via get_vegas_content().
|
||||
|
||||
- FIXED_SEGMENT: Content is a fixed-width block that scrolls BY with
|
||||
the rest of the content. Best for static info like clock, weather.
|
||||
Plugin provides a single image sized to vegas_panel_count panels.
|
||||
|
||||
- STATIC: Scroll pauses, plugin displays for its duration, then scroll
|
||||
resumes. Best for important alerts or detailed views that need attention.
|
||||
Plugin uses standard display() method during the pause.
|
||||
- STATIC: the scroll pauses for the plugin's turn and its display() draws
|
||||
it full screen -- participation ``'pause'``.
|
||||
- SCROLL and FIXED_SEGMENT: the plugin's content joins the scroll --
|
||||
participation ``'scroll'``. Vegas has never told the two apart: a card's
|
||||
width comes from get_vegas_content() and ``vegas_width_pct``, not from
|
||||
the mode. The distinction is deprecated and goes away in LEDMatrix 3.9.0.
|
||||
"""
|
||||
SCROLL = "scroll"
|
||||
FIXED_SEGMENT = "fixed"
|
||||
STATIC = "static"
|
||||
|
||||
|
||||
#: How a plugin takes part in Vegas mode (BasePlugin.get_vegas_participation):
|
||||
#:
|
||||
#: - ``'scroll'``: its content joins the scrolling strip.
|
||||
#: - ``'pause'``: the scroll stops when the plugin's turn comes round, and its
|
||||
#: display() draws it full screen for its display duration.
|
||||
#: - ``'exclude'``: it is left out of Vegas mode.
|
||||
VEGAS_PARTICIPATION_VALUES = ('scroll', 'pause', 'exclude')
|
||||
|
||||
#: The release that removes get_supported_vegas_modes(),
|
||||
#: get_vegas_segment_width(), the ``vegas_panel_count`` setting and the
|
||||
#: SCROLL / FIXED_SEGMENT distinction. The two @deprecated markers below
|
||||
#: spell it as a literal, because tools that read markers statically (the
|
||||
#: deprecation tests, the plugin API usage scan) cannot follow a name.
|
||||
VEGAS_LEGACY_REMOVAL = "3.9.0"
|
||||
|
||||
_vegas_logger = get_logger(__name__)
|
||||
_vegas_warned: set = set()
|
||||
|
||||
|
||||
def _vegas_warn_once(key: Any, message: str, *args: Any) -> None:
|
||||
"""Log a warning about a plugin's Vegas settings once per process.
|
||||
|
||||
Participation is resolved at every rotation refresh, so a bad value would
|
||||
otherwise log on every one of them.
|
||||
"""
|
||||
if key in _vegas_warned:
|
||||
return
|
||||
_vegas_warned.add(key)
|
||||
_vegas_logger.warning(message, *args)
|
||||
|
||||
|
||||
def vegas_participation_value(value: Any) -> Optional[str]:
|
||||
"""``value`` as one of VEGAS_PARTICIPATION_VALUES, or None if it is not one.
|
||||
|
||||
Case and surrounding whitespace are ignored; anything that is not a string
|
||||
(None, a MagicMock standing in for a plugin in a test) is not a value.
|
||||
"""
|
||||
if isinstance(value, str):
|
||||
value = value.strip().lower()
|
||||
if value in VEGAS_PARTICIPATION_VALUES:
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
def configured_vegas_participation(plugin_id: str, config: Any) -> Optional[str]:
|
||||
"""The user's ``vegas_participation`` setting in a plugin's config, if valid.
|
||||
|
||||
An unset or empty value is no setting. Anything else that is not a
|
||||
participation is logged once and ignored, so the plugin keeps its own.
|
||||
"""
|
||||
if not isinstance(config, dict):
|
||||
return None
|
||||
raw = config.get('vegas_participation')
|
||||
if raw is None or (isinstance(raw, str) and not raw.strip()):
|
||||
return None
|
||||
value = vegas_participation_value(raw)
|
||||
if value is None:
|
||||
_vegas_warn_once(
|
||||
('config', plugin_id, repr(raw)),
|
||||
"[%s] Invalid vegas_participation %r, expected one of %s; ignoring it",
|
||||
plugin_id, raw, ', '.join(VEGAS_PARTICIPATION_VALUES))
|
||||
return value
|
||||
|
||||
|
||||
def legacy_vegas_participation(plugin: Any) -> str:
|
||||
"""The participation a plugin's pre-3.8 Vegas hooks describe.
|
||||
|
||||
Exactly what Vegas decided from them before participation existed:
|
||||
|
||||
1. get_vegas_display_mode() returning ``VegasDisplayMode.STATIC`` pauses,
|
||||
whatever the content type -- a STATIC plugin whose content type is
|
||||
``'none'`` was still kept in the rotation to pause it.
|
||||
2. Otherwise get_vegas_content_type() returning ``'none'`` excludes.
|
||||
3. Everything else scrolls. SCROLL and FIXED_SEGMENT were never told
|
||||
apart, and neither were content types ``'multi'``, ``'static'`` or any
|
||||
other string.
|
||||
|
||||
Only the enum member counts as STATIC (a plugin returning the string
|
||||
``'static'`` never paused), and a hook that raises or is missing counts as
|
||||
not STATIC and as content type ``'static'``.
|
||||
"""
|
||||
display_mode = None
|
||||
get_mode = getattr(plugin, 'get_vegas_display_mode', None)
|
||||
if get_mode is not None:
|
||||
try:
|
||||
display_mode = get_mode()
|
||||
except Exception:
|
||||
_vegas_logger.debug("get_vegas_display_mode() failed on %s; not pausing",
|
||||
type(plugin).__name__, exc_info=True)
|
||||
if display_mode == VegasDisplayMode.STATIC:
|
||||
return 'pause'
|
||||
|
||||
content_type = 'static'
|
||||
get_type = getattr(plugin, 'get_vegas_content_type', None)
|
||||
if get_type is not None:
|
||||
try:
|
||||
content_type = get_type()
|
||||
except Exception:
|
||||
_vegas_logger.debug("get_vegas_content_type() failed on %s; treating as 'static'",
|
||||
type(plugin).__name__, exc_info=True)
|
||||
if content_type == 'none':
|
||||
return 'exclude'
|
||||
return 'scroll'
|
||||
|
||||
|
||||
def resolve_vegas_participation(plugin: Any, plugin_id: Optional[str] = None) -> str:
|
||||
"""How Vegas mode treats ``plugin``: ``'scroll'``, ``'pause'`` or ``'exclude'``.
|
||||
|
||||
What the core calls, rather than the plugin's own
|
||||
get_vegas_participation(), so the user's setting wins even over a plugin
|
||||
that overrides that method, and so a plugin that is not a BasePlugin (or a
|
||||
test double) still gets the legacy derivation:
|
||||
|
||||
1. the user's ``vegas_participation`` in the plugin's config;
|
||||
2. the plugin's get_vegas_participation(), when it returns a valid value
|
||||
(BasePlugin's reads the manifest's ``vegas_participation``, then
|
||||
derives one from the legacy hooks);
|
||||
3. legacy_vegas_participation().
|
||||
|
||||
Also where the deprecated ``vegas_panel_count`` setting is reported, once
|
||||
per plugin. Never raises.
|
||||
"""
|
||||
pid = plugin_id or getattr(plugin, 'plugin_id', None) or type(plugin).__name__
|
||||
config = getattr(plugin, 'config', None)
|
||||
if isinstance(config, dict) and 'vegas_panel_count' in config:
|
||||
warn_deprecated(
|
||||
f"The vegas_panel_count setting (plugin '{pid}')", VEGAS_LEGACY_REMOVAL,
|
||||
"it has no effect -- use vegas_width_pct to size the plugin's card",
|
||||
once_key=f"vegas_panel_count:{pid}")
|
||||
|
||||
configured = configured_vegas_participation(pid, config)
|
||||
if configured is not None:
|
||||
return configured
|
||||
|
||||
getter = getattr(plugin, 'get_vegas_participation', None)
|
||||
if callable(getter):
|
||||
try:
|
||||
declared = getter()
|
||||
except Exception:
|
||||
_vegas_logger.exception("[%s] get_vegas_participation() failed; "
|
||||
"using its legacy Vegas hooks", pid)
|
||||
declared = None
|
||||
value = vegas_participation_value(declared)
|
||||
if value is not None:
|
||||
return value
|
||||
if isinstance(declared, str):
|
||||
_vegas_warn_once(
|
||||
('declared', pid, declared),
|
||||
"[%s] get_vegas_participation() returned %r, expected one of %s; "
|
||||
"using its legacy Vegas hooks",
|
||||
pid, declared, ', '.join(VEGAS_PARTICIPATION_VALUES))
|
||||
return legacy_vegas_participation(plugin)
|
||||
|
||||
|
||||
class BasePlugin(ABC):
|
||||
"""
|
||||
Base class that all plugins must inherit from.
|
||||
@@ -834,41 +986,97 @@ class BasePlugin(ABC):
|
||||
"""
|
||||
return None
|
||||
|
||||
def get_vegas_participation(self) -> str:
|
||||
"""
|
||||
How this plugin takes part in Vegas mode: ``'scroll'``, ``'pause'`` or
|
||||
``'exclude'``.
|
||||
|
||||
- ``'scroll'``: the plugin's content (get_vegas_content()) joins the
|
||||
scrolling strip.
|
||||
- ``'pause'``: the scroll stops when the plugin's turn comes round, and
|
||||
its display() draws it full screen for get_display_duration().
|
||||
- ``'exclude'``: the plugin is left out of Vegas mode.
|
||||
|
||||
Resolved in this order:
|
||||
|
||||
1. the user's ``vegas_participation`` setting in this plugin's config
|
||||
(the web UI's per-plugin override);
|
||||
2. ``vegas_participation`` in the plugin's manifest.json -- the way a
|
||||
plugin declares its own default;
|
||||
3. derived from the legacy hooks, so a plugin written before this
|
||||
method existed keeps the behaviour it had: get_vegas_display_mode()
|
||||
returning ``VegasDisplayMode.STATIC`` pauses, otherwise
|
||||
get_vegas_content_type() returning ``'none'`` excludes, and
|
||||
everything else scrolls.
|
||||
|
||||
Declare a fixed participation in the manifest rather than overriding
|
||||
this. Override it only when the answer depends on state -- pause only
|
||||
while an alert is live, exclude while there is nothing to show. Vegas
|
||||
applies the user's setting before calling an override, so an override
|
||||
need not check it.
|
||||
|
||||
Returns:
|
||||
One of VEGAS_PARTICIPATION_VALUES.
|
||||
|
||||
Example:
|
||||
def get_vegas_participation(self):
|
||||
return 'pause' if self._alert_is_live() else 'scroll'
|
||||
"""
|
||||
configured = configured_vegas_participation(self.plugin_id, self.config)
|
||||
if configured is not None:
|
||||
return configured
|
||||
manifest_default = self._manifest_vegas_participation()
|
||||
if manifest_default is not None:
|
||||
return manifest_default
|
||||
return legacy_vegas_participation(self)
|
||||
|
||||
def _manifest_vegas_participation(self) -> Optional[str]:
|
||||
"""``vegas_participation`` from this plugin's manifest, if valid."""
|
||||
manifests = getattr(self.plugin_manager, 'plugin_manifests', None)
|
||||
manifest = manifests.get(self.plugin_id) if isinstance(manifests, dict) else None
|
||||
if not isinstance(manifest, dict) or manifest.get('vegas_participation') is None:
|
||||
return None
|
||||
raw = manifest['vegas_participation']
|
||||
value = vegas_participation_value(raw)
|
||||
if value is None:
|
||||
_vegas_warn_once(
|
||||
('manifest', self.plugin_id, repr(raw)),
|
||||
"[%s] manifest vegas_participation %r is not one of %s; ignoring it",
|
||||
self.plugin_id, raw, ', '.join(VEGAS_PARTICIPATION_VALUES))
|
||||
return value
|
||||
|
||||
def get_vegas_content_type(self) -> str:
|
||||
"""
|
||||
Indicate the type of content this plugin provides for Vegas scroll.
|
||||
Legacy: the type of content this plugin provides for Vegas scroll.
|
||||
|
||||
Override this to specify how Vegas mode should treat this plugin's content.
|
||||
Superseded by get_vegas_participation(). Vegas reads it only to derive
|
||||
a participation when neither the user nor the manifest declares one,
|
||||
and only ``'none'`` matters there: it excludes the plugin (unless
|
||||
get_vegas_display_mode() says STATIC). Every other value scrolls.
|
||||
|
||||
Returns:
|
||||
'multi' - Plugin has multiple scrollable items (sports, odds, news)
|
||||
'static' - Plugin is a static block (clock, weather, music)
|
||||
'none' - Plugin should not appear in Vegas scroll mode
|
||||
|
||||
Example:
|
||||
def get_vegas_content_type(self):
|
||||
return 'multi' # We have multiple games to scroll
|
||||
"""
|
||||
return 'static'
|
||||
|
||||
def get_vegas_display_mode(self) -> VegasDisplayMode:
|
||||
"""
|
||||
Get the display mode for Vegas scroll integration.
|
||||
Legacy: the display mode for Vegas scroll integration.
|
||||
|
||||
This method determines how the plugin's content behaves within Vegas mode:
|
||||
- SCROLL: Content scrolls continuously (multi-item plugins)
|
||||
- FIXED_SEGMENT: Fixed block that scrolls by (clock, weather)
|
||||
- STATIC: Pause scroll to display (alerts, detailed views)
|
||||
Superseded by get_vegas_participation(). Vegas reads it only to derive
|
||||
a participation when neither the user nor the manifest declares one,
|
||||
and only STATIC matters there: it pauses the scroll for the plugin's
|
||||
turn. SCROLL and FIXED_SEGMENT both scroll -- Vegas has never told them
|
||||
apart, and the distinction is deprecated (removed in 3.9.0).
|
||||
|
||||
Override to change default behavior. By default, reads from config
|
||||
or maps legacy get_vegas_content_type() for backward compatibility.
|
||||
Reads the plugin's ``vegas_mode`` config value, else maps
|
||||
get_vegas_content_type() ('multi' to SCROLL, anything else to
|
||||
FIXED_SEGMENT).
|
||||
|
||||
Returns:
|
||||
VegasDisplayMode enum value
|
||||
|
||||
Example:
|
||||
def get_vegas_display_mode(self):
|
||||
return VegasDisplayMode.SCROLL
|
||||
"""
|
||||
# Check for explicit config setting first
|
||||
config_mode = self.config.get("vegas_mode")
|
||||
@@ -888,13 +1096,16 @@ class BasePlugin(ABC):
|
||||
return VegasDisplayMode.SCROLL
|
||||
return VegasDisplayMode.FIXED_SEGMENT
|
||||
|
||||
@deprecated("3.9.0",
|
||||
"nothing reads it -- declare vegas_participation in the manifest instead")
|
||||
def get_supported_vegas_modes(self) -> List[VegasDisplayMode]:
|
||||
"""
|
||||
Return list of Vegas display modes this plugin supports.
|
||||
Deprecated: the Vegas display modes this plugin supports.
|
||||
|
||||
Not currently consulted by core: neither Vegas mode nor the web UI
|
||||
calls it. It is kept, and plugins override it, as the declared set of
|
||||
modes a future mode picker would offer.
|
||||
Never consulted by core -- neither Vegas mode nor the web UI calls it
|
||||
-- and removed in LEDMatrix 3.9.0. A plugin's own override keeps
|
||||
working for the plugin itself; calling this base implementation logs a
|
||||
deprecation warning.
|
||||
|
||||
By default:
|
||||
- 'multi' content type plugins support SCROLL and FIXED_SEGMENT
|
||||
@@ -903,11 +1114,6 @@ class BasePlugin(ABC):
|
||||
|
||||
Returns:
|
||||
List of VegasDisplayMode values this plugin can use
|
||||
|
||||
Example:
|
||||
def get_supported_vegas_modes(self):
|
||||
# This plugin only makes sense as a scrolling ticker
|
||||
return [VegasDisplayMode.SCROLL]
|
||||
"""
|
||||
content_type = self.get_vegas_content_type()
|
||||
|
||||
@@ -918,30 +1124,21 @@ class BasePlugin(ABC):
|
||||
else: # 'static'
|
||||
return [VegasDisplayMode.FIXED_SEGMENT, VegasDisplayMode.STATIC]
|
||||
|
||||
@deprecated("3.9.0",
|
||||
"nothing reads it -- Vegas sizes a card from vegas_width_pct "
|
||||
"(see get_vegas_render_width())")
|
||||
def get_vegas_segment_width(self) -> Optional[int]:
|
||||
"""
|
||||
Get the preferred width for this plugin in Vegas FIXED_SEGMENT mode.
|
||||
Deprecated: the number of panels this plugin wanted as a FIXED_SEGMENT.
|
||||
|
||||
Not currently consulted by core: Vegas mode sizes a card from the
|
||||
``vegas_width_pct`` / ``vegas_scroll.render_width_pct`` settings
|
||||
(see get_vegas_render_width()). Kept because plugins override it.
|
||||
|
||||
Returns the number of panels this plugin should occupy when displayed
|
||||
as a fixed segment. The actual pixel width is calculated as:
|
||||
width = panels * single_panel_width
|
||||
|
||||
Where single_panel_width comes from display.hardware.cols in config.
|
||||
|
||||
Override to provide dynamic sizing based on content.
|
||||
Returns None to use the default (1 panel).
|
||||
Never consulted by core: Vegas sizes a card from the
|
||||
``vegas_width_pct`` / ``vegas_scroll.render_width_pct`` settings (see
|
||||
get_vegas_render_width()). Removed, with the ``vegas_panel_count``
|
||||
setting it reads, in LEDMatrix 3.9.0.
|
||||
|
||||
Returns:
|
||||
Number of panels, or None for default (1 panel)
|
||||
|
||||
Example:
|
||||
def get_vegas_segment_width(self):
|
||||
# Clock needs 2 panels to show time clearly
|
||||
return 2
|
||||
``vegas_panel_count`` from config when it is a positive integer,
|
||||
else None
|
||||
"""
|
||||
raw_value = self.config.get("vegas_panel_count", None)
|
||||
if raw_value is None:
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
"""
|
||||
Plugin catalog: what the web process knows about installed plugins.
|
||||
|
||||
The web interface and the display run as two processes. Only the display
|
||||
imports plugin code and runs it; the web process reads plugins as files --
|
||||
manifest, config schema, the plugin's section of config.json, the installed
|
||||
version -- and never imports a plugin module, instantiates a plugin class or
|
||||
calls a plugin lifecycle hook. This class is that read side.
|
||||
|
||||
It keeps the method names of the read-only part of :class:`PluginManager`
|
||||
(``discover_plugins``, ``plugin_manifests``, ``get_plugin_info``,
|
||||
``get_plugin_directory``, ``get_plugin_display_modes``,
|
||||
``find_plugin_for_mode``), so code that only ever read through a manager
|
||||
reads through a catalog unchanged. It has nothing that runs a plugin: no
|
||||
``load_plugin``, ``get_plugin`` or ``plugins``.
|
||||
|
||||
Runtime state -- whether the display has a plugin loaded, its health, its
|
||||
errors -- is not here either. The display process publishes what it knows to
|
||||
the shared cache (health and resource metrics, the current mode, the error
|
||||
aggregator snapshot), and the web routes read those publications. What the
|
||||
display does not publish (which plugins it has loaded, its plugin state
|
||||
machine) the web cannot know, and reports as unknown.
|
||||
|
||||
See docs/ARCHITECTURE.md ("Web and display processes").
|
||||
"""
|
||||
|
||||
import json
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Union, cast
|
||||
|
||||
from src.common.permission_utils import (
|
||||
ensure_directory_permissions, get_plugin_dir_mode,
|
||||
)
|
||||
from src.logging_config import get_logger
|
||||
from src.plugin_system.plugin_dirs import (
|
||||
ManifestStatus, PluginDirectoryIndex, resolve_plugin_dir,
|
||||
)
|
||||
|
||||
PathLike = Union[str, Path]
|
||||
|
||||
|
||||
class PluginCatalog:
|
||||
"""Manifests, schemas, config and versions of the installed plugins.
|
||||
|
||||
Discovery is explicit and cheap to repeat: :meth:`discover_plugins`
|
||||
rescans the plugins directory and replaces the manifest map, so an
|
||||
uninstalled plugin disappears and a new one appears.
|
||||
"""
|
||||
|
||||
def __init__(self, plugins_dir: PathLike, config_manager: Optional[Any] = None,
|
||||
schema_manager: Optional[Any] = None) -> None:
|
||||
self.plugins_dir: Path = Path(plugins_dir)
|
||||
self.config_manager = config_manager
|
||||
self.schema_manager = schema_manager
|
||||
self.logger = get_logger(__name__)
|
||||
|
||||
# Guards plugin_manifests/plugin_directories: request threads read
|
||||
# them while another request (or startup reconciliation) rescans.
|
||||
self._lock = threading.RLock()
|
||||
self.plugin_manifests: Dict[str, Dict[str, Any]] = {}
|
||||
self.plugin_directories: Dict[str, Path] = {}
|
||||
self._skip_reported: set = set()
|
||||
|
||||
# The Plugin Store installs into this directory, so it has to exist.
|
||||
# The display service logs its own error if it cannot use it; the web
|
||||
# interface stays up either way.
|
||||
try:
|
||||
ensure_directory_permissions(self.plugins_dir, get_plugin_dir_mode())
|
||||
except OSError as exc:
|
||||
self.logger.warning("Could not create plugins directory %s: %s",
|
||||
self.plugins_dir, exc)
|
||||
|
||||
# -- discovery --------------------------------------------------------
|
||||
|
||||
def discover_plugins(self) -> List[str]:
|
||||
"""Rescan the plugins directory; return the discovered plugin ids.
|
||||
|
||||
The rules for what counts as a plugin and which directory wins for a
|
||||
duplicated id are :class:`PluginDirectoryIndex`'s, the same ones the
|
||||
display process loads by. Only the configured directory is scanned.
|
||||
"""
|
||||
index = PluginDirectoryIndex.scan(self.plugins_dir)
|
||||
if index.error is not None:
|
||||
self.logger.error("Error scanning plugins directory %s: %s",
|
||||
self.plugins_dir, index.error)
|
||||
for entry in index.entries:
|
||||
if entry.status in (ManifestStatus.UNREADABLE, ManifestStatus.NOT_OBJECT,
|
||||
ManifestStatus.NO_ID):
|
||||
# The display logs these at load time; once per process is
|
||||
# enough here, since discovery runs on page loads.
|
||||
if entry.name not in self._skip_reported:
|
||||
self._skip_reported.add(entry.name)
|
||||
self.logger.info("Not listing %s: its manifest.json is unusable (%s)",
|
||||
entry.name, entry.status)
|
||||
|
||||
plugins = index.plugins()
|
||||
manifests = {pid: entry.manifest for pid, entry in plugins.items()}
|
||||
directories = {pid: entry.path for pid, entry in plugins.items()}
|
||||
with self._lock:
|
||||
self.plugin_manifests.clear()
|
||||
self.plugin_manifests.update(manifests)
|
||||
self.plugin_directories.clear()
|
||||
self.plugin_directories.update(directories)
|
||||
return list(plugins)
|
||||
|
||||
def discovered_plugin_ids(self) -> set:
|
||||
"""Snapshot of the discovered ids, taken under the lock."""
|
||||
with self._lock:
|
||||
return set(self.plugin_manifests)
|
||||
|
||||
# -- manifests --------------------------------------------------------
|
||||
|
||||
def get_manifest(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""A copy of the manifest discovery read for ``plugin_id``, or None."""
|
||||
with self._lock:
|
||||
manifest = self.plugin_manifests.get(plugin_id)
|
||||
return dict(manifest) if manifest else None
|
||||
|
||||
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""The plugin's manifest, as a new dict -- metadata only.
|
||||
|
||||
Unlike ``PluginManager.get_plugin_info`` there are no ``loaded``,
|
||||
``runtime_info`` or ``state`` keys: those described plugin instances
|
||||
in this process, which no longer exist.
|
||||
"""
|
||||
return self.get_manifest(plugin_id)
|
||||
|
||||
def get_all_plugin_info(self) -> List[Dict[str, Any]]:
|
||||
""":meth:`get_plugin_info` for every discovered plugin."""
|
||||
with self._lock:
|
||||
ids = list(self.plugin_manifests)
|
||||
return [info for info in (self.get_plugin_info(pid) for pid in ids) if info]
|
||||
|
||||
def read_manifest(self, plugin_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""The manifest as it is on disk now, not as discovery last saw it.
|
||||
|
||||
For reads that must reflect a change made since the last scan -- the
|
||||
version just after an update, say. None when the plugin has no
|
||||
directory or its manifest is missing, unreadable or not an object.
|
||||
"""
|
||||
plugin_dir = self.get_plugin_directory(plugin_id)
|
||||
if plugin_dir is None:
|
||||
return None
|
||||
try:
|
||||
with open(Path(plugin_dir) / 'manifest.json', 'r', encoding='utf-8') as f:
|
||||
manifest = json.load(f)
|
||||
except (OSError, ValueError) as exc:
|
||||
self.logger.debug("Could not read manifest for %s: %s", plugin_id, exc)
|
||||
return None
|
||||
return manifest if isinstance(manifest, dict) else None
|
||||
|
||||
def get_installed_version(self, plugin_id: str) -> str:
|
||||
"""The installed version from the on-disk manifest, or ''."""
|
||||
manifest = self.read_manifest(plugin_id) or {}
|
||||
version = manifest.get('version', '')
|
||||
return version if isinstance(version, str) else str(version)
|
||||
|
||||
def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
|
||||
"""Where ``plugin_id`` is installed, or None.
|
||||
|
||||
Same rules as ``PluginManager.get_plugin_directory``: the discovered
|
||||
directory, else ``<id>`` then ``ledmatrix-<id>`` by name within the
|
||||
plugins directory. An id that is not one plain path segment is
|
||||
refused rather than joined onto the plugins directory.
|
||||
"""
|
||||
with self._lock:
|
||||
if plugin_id in self.plugin_directories:
|
||||
return str(self.plugin_directories[plugin_id])
|
||||
plugin_dir = resolve_plugin_dir(
|
||||
plugin_id, [self.plugins_dir], prefix=True, case_insensitive=False,
|
||||
by_manifest=False)
|
||||
return str(plugin_dir) if plugin_dir is not None else None
|
||||
|
||||
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
|
||||
"""The manifest's ``display_modes``, or [].
|
||||
|
||||
What the display actually rotates can differ: a plugin may compute
|
||||
its modes at run time (``plugin.modes``). This is the declared list.
|
||||
"""
|
||||
with self._lock:
|
||||
manifest = self.plugin_manifests.get(plugin_id)
|
||||
modes = (manifest or {}).get('display_modes', [])
|
||||
return list(modes) if isinstance(modes, list) else []
|
||||
|
||||
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
|
||||
"""The plugin whose manifest declares ``mode`` (case-insensitive)."""
|
||||
wanted = mode.strip().lower()
|
||||
with self._lock:
|
||||
manifests = dict(self.plugin_manifests)
|
||||
for plugin_id, manifest in manifests.items():
|
||||
modes = manifest.get('display_modes')
|
||||
if isinstance(modes, list) and any(
|
||||
isinstance(m, str) and m.lower() == wanted for m in modes):
|
||||
return plugin_id
|
||||
return None
|
||||
|
||||
# -- schema and config ------------------------------------------------
|
||||
|
||||
def get_schema(self, plugin_id: str, use_cache: bool = True) -> Optional[Dict[str, Any]]:
|
||||
"""The plugin's config schema through SchemaManager, or None."""
|
||||
if self.schema_manager is None:
|
||||
return None
|
||||
schema = self.schema_manager.load_schema(plugin_id, use_cache=use_cache)
|
||||
return cast(Optional[Dict[str, Any]], schema)
|
||||
|
||||
def get_config(self, plugin_id: str) -> Dict[str, Any]:
|
||||
"""The plugin's section of config.json (secrets merged), or {}."""
|
||||
if self.config_manager is None:
|
||||
return {}
|
||||
section = (self.config_manager.load_config() or {}).get(plugin_id)
|
||||
return section if isinstance(section, dict) else {}
|
||||
|
||||
def is_enabled(self, plugin_id: str) -> bool:
|
||||
"""Whether config.json enables the plugin, by the display's rule.
|
||||
|
||||
The display loads a plugin only when its section says
|
||||
``"enabled": true``; a missing flag or section means disabled
|
||||
(``DisplayController._reconcile_enabled_plugins``).
|
||||
"""
|
||||
return bool(self.get_config(plugin_id).get('enabled', False))
|
||||
|
||||
|
||||
def display_restart_required(action: str, plugin_enabled: bool, *,
|
||||
changed: bool = True,
|
||||
preserve_config: bool = False) -> bool:
|
||||
"""Whether a store operation needs a display restart to reach the panel.
|
||||
|
||||
The display loads and unloads plugins live only through its config
|
||||
watcher: when a plugin's ``enabled`` flag changes it reconciles the
|
||||
running set (``DisplayController._reconcile_enabled_plugins``), and
|
||||
loading reads the plugin fresh from disk. Nothing makes it reload a
|
||||
plugin it is already running, and nothing tells it about files changing
|
||||
under a plugin whose flag did not move. So:
|
||||
|
||||
- ``install``: a plugin that is not enabled needs nothing -- enabling it
|
||||
later loads it. One already enabled in config (a reinstall, or a
|
||||
config carried over) is not picked up until a restart.
|
||||
- ``update``: the display keeps running the code it loaded until it
|
||||
restarts, if it runs the plugin at all -- only when it is enabled.
|
||||
``changed=False`` (already up to date) needs nothing.
|
||||
- ``uninstall``: removing the plugin's config section flips its enabled
|
||||
flag, and the reconcile unloads it. With ``preserve_config`` the flag
|
||||
stays, and an enabled plugin keeps running until a restart.
|
||||
|
||||
``plugin_enabled`` is the config flag as it was before the operation.
|
||||
"""
|
||||
if not plugin_enabled:
|
||||
return False
|
||||
if action == 'install':
|
||||
return True
|
||||
if action == 'update':
|
||||
return changed
|
||||
if action == 'uninstall':
|
||||
return preserve_config
|
||||
raise ValueError(f"unknown store action: {action!r}")
|
||||
@@ -19,8 +19,25 @@ class PluginTimeoutError(Exception):
|
||||
"""Raised when a plugin operation times out."""
|
||||
|
||||
|
||||
class PluginBusyError(PluginTimeoutError):
|
||||
"""A plugin's lock stayed held past its bound.
|
||||
|
||||
Not raised; recorded. The lock is held by the plugin's own display(),
|
||||
update(), on_config_change() or a Vegas content render -- slow, or hung
|
||||
-- so the caller skipped the plugin rather than wait on it. Report-only:
|
||||
it is kept as the plugin's state error info and counted as a busy skip in
|
||||
health, never as a failure, so it cannot open the circuit breaker.
|
||||
"""
|
||||
|
||||
|
||||
class PluginExecutor:
|
||||
"""Handles plugin execution with timeout and error isolation."""
|
||||
|
||||
#: A display() call at least this long is logged and counted as slow.
|
||||
#: A frame is milliseconds; two seconds is a plugin doing I/O in display().
|
||||
SLOW_DISPLAY_SECONDS = 2.0
|
||||
#: An update() call at least this long is logged as slow.
|
||||
SLOW_UPDATE_SECONDS = 5.0
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
@@ -117,15 +134,15 @@ class PluginExecutor:
|
||||
True if update succeeded, False otherwise
|
||||
"""
|
||||
try:
|
||||
start_time = time.time()
|
||||
start_time = time.monotonic()
|
||||
self.execute_with_timeout(
|
||||
lambda: plugin.update(),
|
||||
timeout=timeout,
|
||||
plugin_id=plugin_id
|
||||
)
|
||||
duration = time.time() - start_time
|
||||
duration = time.monotonic() - start_time
|
||||
|
||||
if duration > 5.0: # Warn if update takes more than 5 seconds
|
||||
if duration > self.SLOW_UPDATE_SECONDS:
|
||||
self.logger.warning(
|
||||
"Plugin %s update() took %.2fs (consider optimizing)",
|
||||
plugin_id,
|
||||
@@ -175,7 +192,7 @@ class PluginExecutor:
|
||||
True if display succeeded, False otherwise
|
||||
"""
|
||||
try:
|
||||
start_time = time.time()
|
||||
start_time = time.monotonic()
|
||||
|
||||
# Does display() take a display_mode keyword? The caller usually
|
||||
# knows and caches the answer, so prefer what it passed.
|
||||
@@ -206,9 +223,9 @@ class PluginExecutor:
|
||||
plugin_id=plugin_id
|
||||
)
|
||||
|
||||
duration = time.time() - start_time
|
||||
duration = time.monotonic() - start_time
|
||||
|
||||
if duration > 2.0: # Warn if display takes more than 2 seconds
|
||||
if duration > self.SLOW_DISPLAY_SECONDS:
|
||||
self.logger.warning(
|
||||
"Plugin %s display() took %.2fs (consider optimizing)",
|
||||
plugin_id,
|
||||
|
||||
@@ -254,6 +254,104 @@ class PluginHealthTracker:
|
||||
|
||||
self._save_health_state(plugin_id, state)
|
||||
|
||||
def record_hang(self, plugin_id: str, operation: str, seconds: float,
|
||||
error: Optional[Exception] = None) -> None:
|
||||
"""Record a display() or update() call that ran past its limit.
|
||||
|
||||
Counts as a failure, so the ordinary circuit breaker handles a plugin
|
||||
that keeps hanging: after ``failure_threshold`` in a row it is skipped
|
||||
by both the update scheduler and the display rotation until the
|
||||
cooldown ends. The hang itself is kept alongside (``hang_count``,
|
||||
``last_hang``) so the health API can tell "hung" from "raised".
|
||||
|
||||
Not for an update skipped because the plugin's lock stayed held: the
|
||||
holder may be a healthy but long render (Vegas prefetch). That is
|
||||
:meth:`record_busy_skip`, which never touches the breaker.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
operation: What hung: ``"display"`` or ``"update"``.
|
||||
seconds: How long it had been running when this was recorded.
|
||||
error: The error to store as ``last_error``; one is built from
|
||||
the other arguments when omitted.
|
||||
"""
|
||||
state = self.get_health_state(plugin_id)
|
||||
count = state.get('hang_count')
|
||||
state['hang_count'] = (count if isinstance(count, int) and not isinstance(count, bool)
|
||||
else 0) + 1
|
||||
state['last_hang'] = {
|
||||
'operation': operation,
|
||||
'seconds': round(float(seconds), 3),
|
||||
'time': time.time(),
|
||||
}
|
||||
if error is None:
|
||||
error = TimeoutError(f"{operation} still running after {seconds:.1f}s")
|
||||
# record_failure saves the record, hang fields included.
|
||||
self.record_failure(plugin_id, error)
|
||||
|
||||
#: Minimum seconds between persisting a plugin's slow-call or busy-skip
|
||||
#: counters. The in-memory record is updated every time; a plugin that is
|
||||
#: slow on every frame must not become an SD-card write per frame.
|
||||
SLOW_CALL_PERSIST_INTERVAL = 60.0
|
||||
|
||||
def record_slow_call(self, plugin_id: str, operation: str, seconds: float) -> None:
|
||||
"""Note a call that finished, but slowly. Reporting only.
|
||||
|
||||
Unlike :meth:`record_hang` this never touches the circuit breaker: a
|
||||
slow display() still drew its frame.
|
||||
"""
|
||||
state = self.get_health_state(plugin_id)
|
||||
count = state.get('slow_call_count')
|
||||
state['slow_call_count'] = (count if isinstance(count, int) and not isinstance(count, bool)
|
||||
else 0) + 1
|
||||
now = time.time()
|
||||
state['last_slow_call'] = {
|
||||
'operation': operation,
|
||||
'seconds': round(float(seconds), 3),
|
||||
'time': now,
|
||||
}
|
||||
self._save_reporting_throttled('slow', plugin_id, state, now)
|
||||
|
||||
def record_busy_skip(self, plugin_id: str, operation: str, seconds: float) -> None:
|
||||
"""Note a call skipped because the plugin's lock stayed held. Reporting only.
|
||||
|
||||
The update worker gives up on a plugin's lock after
|
||||
``PluginManager.PLUGIN_LOCK_TIMEOUT``. Whatever held it may be healthy
|
||||
-- Vegas prefetch holds the lock for a plugin's whole content render,
|
||||
which on a slow Pi can take longer than that -- so like
|
||||
:meth:`record_slow_call` this never touches the circuit breaker, the
|
||||
failure streak or ``last_error``. A real hang is recorded by
|
||||
:meth:`record_hang` where it is measured.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
operation: What was skipped, e.g. ``"update lock wait"``.
|
||||
seconds: How long the lock was waited on.
|
||||
"""
|
||||
state = self.get_health_state(plugin_id)
|
||||
count = state.get('busy_skip_count')
|
||||
state['busy_skip_count'] = (count if isinstance(count, int) and not isinstance(count, bool)
|
||||
else 0) + 1
|
||||
now = time.time()
|
||||
state['last_busy_skip'] = {
|
||||
'operation': operation,
|
||||
'seconds': round(float(seconds), 3),
|
||||
'time': now,
|
||||
}
|
||||
self._save_reporting_throttled('busy', plugin_id, state, now)
|
||||
|
||||
def _save_reporting_throttled(self, kind: str, plugin_id: str,
|
||||
state: Dict[str, Any], now: float) -> None:
|
||||
"""Persist a reporting-only change at most once per
|
||||
SLOW_CALL_PERSIST_INTERVAL per plugin and ``kind``. The first one is
|
||||
saved at once, so the web process (which reads the persisted record)
|
||||
sees it; repeats in between stay in memory until the next save."""
|
||||
saved_at = self.__dict__.setdefault('_reporting_saved_at', {})
|
||||
last = saved_at.get((kind, plugin_id))
|
||||
if last is None or now - last >= self.SLOW_CALL_PERSIST_INTERVAL:
|
||||
saved_at[(kind, plugin_id)] = now
|
||||
self._save_health_state(plugin_id, state)
|
||||
|
||||
def set_degraded(self, plugin_id: str, reason: Optional[str]) -> None:
|
||||
"""Flag (or clear) a plugin as degraded without touching the circuit breaker.
|
||||
|
||||
@@ -345,7 +443,13 @@ class PluginHealthTracker:
|
||||
'degraded': state.get('degraded', False),
|
||||
'degraded_reason': state.get('degraded_reason'),
|
||||
'circuit_opened_time': state.get('circuit_opened_time'),
|
||||
'half_open_start_time': state.get('half_open_start_time')
|
||||
'half_open_start_time': state.get('half_open_start_time'),
|
||||
'hang_count': state.get('hang_count', 0),
|
||||
'last_hang': state.get('last_hang'),
|
||||
'slow_call_count': state.get('slow_call_count', 0),
|
||||
'last_slow_call': state.get('last_slow_call'),
|
||||
'busy_skip_count': state.get('busy_skip_count', 0),
|
||||
'last_busy_skip': state.get('last_busy_skip'),
|
||||
}
|
||||
|
||||
def get_all_health_summaries(self) -> Dict[str, Dict[str, Any]]:
|
||||
|
||||
@@ -16,12 +16,15 @@ import time
|
||||
import threading
|
||||
import types
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Any, Tuple
|
||||
from typing import Dict, List, NamedTuple, Optional, Any, Tuple, Union
|
||||
import logging
|
||||
from src import display_watchdog
|
||||
from src.exceptions import PluginError, ConfigError
|
||||
from src.logging_config import get_logger
|
||||
from src.plugin_system.plugin_loader import PluginLoader
|
||||
from src.plugin_system.plugin_executor import PluginExecutor
|
||||
from src.plugin_system.plugin_executor import (
|
||||
PluginBusyError, PluginExecutor, PluginTimeoutError,
|
||||
)
|
||||
from src.plugin_system.plugin_state import PluginStateManager, PluginState
|
||||
from src.plugin_system.schema_manager import (
|
||||
CORE_VEGAS_TUNING_KEYS, SchemaManager, normalize_legacy_booleans,
|
||||
@@ -36,6 +39,16 @@ from src.common.permission_utils import (
|
||||
)
|
||||
|
||||
|
||||
class _DeferredConfigChange(NamedTuple):
|
||||
"""Update-queue item: apply the config change parked for ``plugin_id``.
|
||||
|
||||
Queued by apply_config_change() when the plugin's lock was busy; the
|
||||
change itself waits in ``PluginManager._deferred_config_changes`` so only
|
||||
the latest one is ever applied.
|
||||
"""
|
||||
plugin_id: str
|
||||
|
||||
|
||||
class PluginManager:
|
||||
"""
|
||||
Manages plugin discovery, loading, and lifecycle.
|
||||
@@ -56,6 +69,19 @@ class PluginManager:
|
||||
# How long unload_plugin() waits for an in-flight update() to finish
|
||||
# before tearing the instance down anyway.
|
||||
UNLOAD_LOCK_TIMEOUT = 5.0
|
||||
|
||||
# How long the update worker and apply_config_change() wait for a
|
||||
# plugin's lock -- the same bound unload already uses for the same lock.
|
||||
# A display() frame holds it for milliseconds, so this only runs out when
|
||||
# the holder is hung or pathologically slow. The worker then skips that
|
||||
# plugin (recorded as a hang, so repeats open its circuit breaker)
|
||||
# instead of stalling every other plugin's update behind it.
|
||||
PLUGIN_LOCK_TIMEOUT = UNLOAD_LOCK_TIMEOUT
|
||||
|
||||
# Minimum seconds between repeats of the same hang/slow-call warning for
|
||||
# one plugin. A hung plugin is re-detected every interval; a slow
|
||||
# display() can be re-detected every frame.
|
||||
HANG_LOG_INTERVAL = 60.0
|
||||
|
||||
def __init__(self, plugins_dir: str = "plugins",
|
||||
config_manager: Optional[Any] = None,
|
||||
@@ -122,7 +148,33 @@ class PluginManager:
|
||||
# post-timeout window.
|
||||
# Kill switch: plugin_system.synchronous_updates: true restores the
|
||||
# inline path.
|
||||
self._update_queue: "queue.Queue[Optional[Tuple[str, float]]]" = queue.Queue()
|
||||
#
|
||||
# Which thread runs each plugin hook, and what it holds:
|
||||
# __init__, on_enable the loading thread (main thread at startup,
|
||||
# the render thread on a live enable).
|
||||
# update() plugin-update-worker, under the plugin lock,
|
||||
# via PluginExecutor (whose daemon thread runs
|
||||
# the call; if it outlives the executor's
|
||||
# timeout it keeps the lock until it returns).
|
||||
# Exceptions: the startup pass
|
||||
# (DisplayController._run_initial_updates, main
|
||||
# thread, before the display loop starts) and
|
||||
# the synchronous_updates kill switch (render
|
||||
# thread) run it without the lock.
|
||||
# display() the render thread, under a try-lock: a busy
|
||||
# lock skips the frame. The first frame of a
|
||||
# screen goes through PluginExecutor. Vegas
|
||||
# mode's adapter and coordinator take the lock
|
||||
# with a bounded wait.
|
||||
# on_config_change() ConfigService's watcher thread, under the
|
||||
# plugin lock via apply_config_change(); if the
|
||||
# lock stays busy it is deferred to the update
|
||||
# worker, which applies it under the lock.
|
||||
# cleanup(), on_disable() whoever calls unload_plugin(), under the
|
||||
# lock with UNLOAD_LOCK_TIMEOUT.
|
||||
# No wait on a plugin lock is unbounded, so one hung plugin can only
|
||||
# cost the worker PLUGIN_LOCK_TIMEOUT per attempt.
|
||||
self._update_queue: "queue.Queue[Union[None, Tuple[str, float], _DeferredConfigChange]]" = queue.Queue()
|
||||
self._pending_updates: set = set()
|
||||
self._pending_lock = threading.Lock()
|
||||
# Serializes the "is this plugin eligible?" -> "claim it (RUNNING)"
|
||||
@@ -145,6 +197,12 @@ class PluginManager:
|
||||
# run_scheduled_updates_with_changes().
|
||||
self._completed_updates: set = set()
|
||||
self._completed_updates_lock = threading.Lock()
|
||||
# Config changes that found the plugin's lock busy, latest per plugin,
|
||||
# with the instance they were meant for. See apply_config_change().
|
||||
self._deferred_config_changes: Dict[str, Tuple[Any, Dict[str, Any]]] = {}
|
||||
self._deferred_config_lock = threading.Lock()
|
||||
# key -> (monotonic time last logged, repeats suppressed since)
|
||||
self._rate_limited_warnings: Dict[str, Tuple[float, int]] = {}
|
||||
self._synchronous_updates = False
|
||||
if self.config_manager is not None:
|
||||
try:
|
||||
@@ -296,7 +354,20 @@ class PluginManager:
|
||||
|
||||
return plugin_ids
|
||||
|
||||
def load_plugin(self, plugin_id: str) -> bool:
|
||||
def load_plugin(self, plugin_id: str, force_enabled: bool = False) -> bool:
|
||||
"""Load a plugin by ID; see _load_plugin.
|
||||
|
||||
Loading can install the plugin's dependencies with pip -- minutes,
|
||||
not seconds. When that happens on the display's render thread (a
|
||||
plugin enabled from the web UI, or loaded for on-demand), its
|
||||
systemd watchdog gets a longer limit for the duration. Start-up
|
||||
loads, on a thread pool, are covered by the start-up allowance.
|
||||
"""
|
||||
with display_watchdog.extended(display_watchdog.PLUGIN_LOAD_ALLOWANCE_SECONDS,
|
||||
f'loading plugin {plugin_id}'):
|
||||
return self._load_plugin(plugin_id, force_enabled)
|
||||
|
||||
def _load_plugin(self, plugin_id: str, force_enabled: bool = False) -> bool:
|
||||
"""
|
||||
Load a plugin by ID.
|
||||
|
||||
@@ -310,6 +381,10 @@ class PluginManager:
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
force_enabled: Run the plugin enabled even though config.json has
|
||||
it disabled. On-demand uses this to show a disabled plugin
|
||||
(DisplayController._load_plugin_for_on_demand). Only the
|
||||
instance's config says enabled; config.json is not written.
|
||||
|
||||
Returns:
|
||||
True if loaded successfully, False otherwise
|
||||
@@ -376,6 +451,12 @@ class PluginManager:
|
||||
# (prepare_plugin_config). In memory only: config.json is written
|
||||
# by saves, never by loading a plugin.
|
||||
config = self.prepare_plugin_config(plugin_id, config, schema=schema)
|
||||
if force_enabled:
|
||||
# A copy: prepare_plugin_config can hand back the section from
|
||||
# config_manager's cached config, and setting the flag there
|
||||
# would read as enabled to everything else in this process.
|
||||
config = dict(config)
|
||||
config['enabled'] = True
|
||||
|
||||
# Use PluginLoader to load plugin
|
||||
plugin_instance, _module = self.plugin_loader.load_plugin(
|
||||
@@ -455,7 +536,13 @@ class PluginManager:
|
||||
raise
|
||||
else:
|
||||
self.state_manager.set_state(plugin_id, PluginState.DISABLED)
|
||||
|
||||
|
||||
# The version this instance runs, for the runtime snapshot the
|
||||
# web UI reads: the manifest on disk can move on after an update.
|
||||
version = manifest.get('version')
|
||||
self.state_manager.record_loaded(
|
||||
plugin_id, version if isinstance(version, str) else None)
|
||||
|
||||
self.logger.info("Loaded plugin: %s", plugin_id)
|
||||
|
||||
return True
|
||||
@@ -502,7 +589,8 @@ class PluginManager:
|
||||
#: prefix rule would silently stop validating it.
|
||||
#:
|
||||
#: Read by: ``vegas_mode/plugin_adapter.py`` (``vegas_width_pct``,
|
||||
#: ``vegas_overflow``) and ``base_plugin.py`` (``vegas_max_width_screens``).
|
||||
#: ``vegas_overflow``) and ``base_plugin.py`` (``vegas_max_width_screens``,
|
||||
#: ``vegas_participation``).
|
||||
#:
|
||||
#: The list itself lives with the other core-owned per-plugin properties in
|
||||
#: ``schema_manager.CORE_PLUGIN_PROPERTIES``, which the web save path also
|
||||
@@ -676,6 +764,8 @@ class PluginManager:
|
||||
|
||||
# Remove from active plugins
|
||||
del self.plugins[plugin_id]
|
||||
with self._deferred_config_lock:
|
||||
self._deferred_config_changes.pop(plugin_id, None)
|
||||
with self._plugin_last_update_lock:
|
||||
self.plugin_last_update.pop(plugin_id, None)
|
||||
self._update_interval_cache.pop(plugin_id, None)
|
||||
@@ -704,6 +794,9 @@ class PluginManager:
|
||||
except Exception as e:
|
||||
self.logger.error("Error unloading plugin %s: %s", plugin_id, e, exc_info=True)
|
||||
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
|
||||
if plugin_id not in self.plugins:
|
||||
# Failed after the instance was dropped: it is not loaded.
|
||||
self.state_manager.record_unloaded(plugin_id)
|
||||
return False
|
||||
|
||||
def reload_plugin(self, plugin_id: str) -> bool:
|
||||
@@ -775,7 +868,7 @@ class PluginManager:
|
||||
"""
|
||||
return self.plugins.copy()
|
||||
|
||||
@deprecated("3.7.0", "check each plugin's enabled flag in plugins")
|
||||
@deprecated("3.8.0", "check each plugin's enabled flag in plugins")
|
||||
def get_enabled_plugins(self) -> List[str]:
|
||||
"""
|
||||
Get list of enabled plugin IDs.
|
||||
@@ -1012,6 +1105,8 @@ class PluginManager:
|
||||
self,
|
||||
plugin_id: str,
|
||||
exc: Optional[Exception] = None,
|
||||
log: bool = True,
|
||||
count_failure: bool = True,
|
||||
) -> None:
|
||||
"""Apply the standard failure-recovery path for a plugin update.
|
||||
|
||||
@@ -1025,6 +1120,11 @@ class PluginManager:
|
||||
exc: The exception that caused the failure, if any. When None a
|
||||
synthetic ExecutionFailure exception is constructed from the
|
||||
timeout/executor-error path.
|
||||
log: Log the generic failure line. Callers that already logged
|
||||
something more specific (rate-limited) pass False.
|
||||
count_failure: Record the failure in plugin health, where it
|
||||
counts toward the circuit breaker. A busy skip passes False:
|
||||
it records itself as a busy skip, reporting only.
|
||||
"""
|
||||
failure_time = time.time()
|
||||
if exc is not None:
|
||||
@@ -1040,13 +1140,91 @@ class PluginManager:
|
||||
'timestamp': failure_time,
|
||||
'recoverable': True,
|
||||
}
|
||||
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
|
||||
if log:
|
||||
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
|
||||
with self._plugin_last_update_lock:
|
||||
self.plugin_last_update[plugin_id] = failure_time
|
||||
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info)
|
||||
if self.health_tracker:
|
||||
if count_failure and self.health_tracker:
|
||||
self.health_tracker.record_failure(plugin_id, err)
|
||||
|
||||
def _warn_rate_limited(self, key: str, message: str, *args: Any) -> None:
|
||||
"""Log a warning at most once per HANG_LOG_INTERVAL for ``key``.
|
||||
|
||||
Repeats in between are counted and the count is appended to the next
|
||||
one that is logged, so the journal shows the problem continuing
|
||||
without a line per frame or per scheduler tick.
|
||||
"""
|
||||
# setdefault: tests build bare managers with PluginManager.__new__.
|
||||
seen = self.__dict__.setdefault('_rate_limited_warnings', {})
|
||||
now = time.monotonic()
|
||||
last, suppressed = seen.get(key, (None, 0))
|
||||
if last is not None and now - last < self.HANG_LOG_INTERVAL:
|
||||
seen[key] = (last, suppressed + 1)
|
||||
return
|
||||
seen[key] = (now, 0)
|
||||
if suppressed:
|
||||
message += " (%d more since the last warning)"
|
||||
args = args + (suppressed,)
|
||||
self.logger.warning(message, *args)
|
||||
|
||||
def _record_hang(self, plugin_id: str, operation: str, seconds: float,
|
||||
err: Exception) -> None:
|
||||
"""Record a hang in plugin health: a failure to the circuit breaker.
|
||||
|
||||
PluginHealthTracker.record_hang also counts the hang separately. Never
|
||||
raises: this runs on the update worker and the render thread.
|
||||
"""
|
||||
tracker = self.health_tracker
|
||||
if tracker is None:
|
||||
return
|
||||
try:
|
||||
tracker.record_hang(plugin_id, operation, seconds, err)
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
self.logger.debug("Could not record hang for %s: %s", plugin_id, e)
|
||||
|
||||
def note_display_duration(self, plugin_id: str, seconds: float) -> None:
|
||||
"""Account for one display() call that took ``seconds``.
|
||||
|
||||
Called by the render loop for every frame, so the common case is one
|
||||
comparison. At or above PluginExecutor.SLOW_DISPLAY_SECONDS the call
|
||||
is logged (rate-limited) and counted as slow in plugin health; at or
|
||||
above the executor's timeout -- the limit the first frame of a screen
|
||||
is already held to -- it is recorded as a hang, which the circuit
|
||||
breaker counts as a failure.
|
||||
"""
|
||||
if seconds < PluginExecutor.SLOW_DISPLAY_SECONDS:
|
||||
return
|
||||
if seconds >= self.plugin_executor.default_timeout:
|
||||
self.record_display_hang(plugin_id, seconds)
|
||||
return
|
||||
self._warn_rate_limited(
|
||||
"slow-display:" + plugin_id,
|
||||
"Plugin %s display() took %.2fs; a frame should take milliseconds "
|
||||
"(is it fetching or loading files in display()?)", plugin_id, seconds)
|
||||
tracker = self.health_tracker
|
||||
record_slow = getattr(tracker, 'record_slow_call', None) if tracker is not None else None
|
||||
if callable(record_slow):
|
||||
try:
|
||||
record_slow(plugin_id, 'display', seconds)
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
self.logger.debug("Could not record slow display for %s: %s", plugin_id, e)
|
||||
|
||||
def record_display_hang(self, plugin_id: str, seconds: float) -> None:
|
||||
"""Record a display() call that ran ``seconds``, past its limit.
|
||||
|
||||
Either it has since returned (note_display_duration) or it is still
|
||||
running on the executor's lingering thread, holding the plugin's lock
|
||||
(the render loop's first-frame dispatch).
|
||||
"""
|
||||
self._warn_rate_limited(
|
||||
"hung-display:" + plugin_id,
|
||||
"Plugin %s display() ran for at least %.1fs (limit %.0fs); recorded "
|
||||
"as a hang -- repeated hangs open its circuit breaker",
|
||||
plugin_id, seconds, self.plugin_executor.default_timeout)
|
||||
self._record_hang(plugin_id, 'display', seconds, PluginTimeoutError(
|
||||
f"Plugin {plugin_id} display() ran for at least {seconds:.1f}s"))
|
||||
|
||||
def run_scheduled_updates(self, current_time: Optional[float] = None) -> None:
|
||||
"""
|
||||
Trigger plugin updates based on their defined update intervals.
|
||||
@@ -1080,6 +1258,9 @@ class PluginManager:
|
||||
# Kill-switch path: the original inline execution
|
||||
# (blocks the caller until update() completes/times out)
|
||||
self._execute_update_now(plugin_id, plugin_instance, current_time)
|
||||
# Up to the executor's 30s each, one after another on the
|
||||
# render thread: check in with its watchdog between them.
|
||||
display_watchdog.beat()
|
||||
else:
|
||||
self._enqueue_update(plugin_id, current_time)
|
||||
|
||||
@@ -1132,11 +1313,13 @@ class PluginManager:
|
||||
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
|
||||
|
||||
def get_plugin_lock(self, plugin_id: str) -> threading.Lock:
|
||||
"""Per-plugin lock keeping update() and display() mutually exclusive.
|
||||
"""Per-plugin lock keeping update(), display() and on_config_change()
|
||||
mutually exclusive.
|
||||
|
||||
The update worker holds it for the duration of a plugin's update();
|
||||
the display side acquires it non-blocking and skips that frame's
|
||||
display() call when the plugin is mid-update.
|
||||
display() call when the plugin is mid-update. Every other waiter uses
|
||||
a bounded acquire (see the thread notes in __init__).
|
||||
"""
|
||||
with self._plugin_locks_guard:
|
||||
lock = self._plugin_locks.get(plugin_id)
|
||||
@@ -1202,14 +1385,28 @@ class PluginManager:
|
||||
real update() call genuinely finishes (see _execute_update_now),
|
||||
which can be after this dispatch returns if PluginExecutor's own
|
||||
timeout elapses first.
|
||||
|
||||
The lock wait is bounded by PLUGIN_LOCK_TIMEOUT. Whatever holds it
|
||||
past that -- a hung display() on the render thread, a lingering
|
||||
executor thread, or a long but healthy Vegas content render -- costs
|
||||
this worker that long once per attempt, and the plugin's update is
|
||||
skipped and reported as a busy skip (_skip_busy_update), which never
|
||||
counts toward the circuit breaker; the other plugins' queued updates
|
||||
carry on.
|
||||
"""
|
||||
while True:
|
||||
item = self._update_queue.get()
|
||||
if item is None: # shutdown sentinel
|
||||
return
|
||||
if isinstance(item, _DeferredConfigChange):
|
||||
self._apply_deferred_config_change(item.plugin_id)
|
||||
continue
|
||||
plugin_id, scheduled_time = item
|
||||
lock = self.get_plugin_lock(plugin_id)
|
||||
lock.acquire()
|
||||
wait_start = time.monotonic()
|
||||
if not lock.acquire(timeout=self.PLUGIN_LOCK_TIMEOUT):
|
||||
self._skip_busy_update(plugin_id, time.monotonic() - wait_start)
|
||||
continue
|
||||
plugin_instance = self.plugins.get(plugin_id)
|
||||
if plugin_instance is None: # unloaded while queued; its
|
||||
# lifecycle state was already cleared by unload_plugin —
|
||||
@@ -1218,6 +1415,9 @@ class PluginManager:
|
||||
with self._pending_lock:
|
||||
self._pending_updates.discard(plugin_id)
|
||||
continue
|
||||
# A config change that found the lock busy goes in first, so
|
||||
# this update() runs against the settings the user saved.
|
||||
self._apply_deferred_config_locked(plugin_id, plugin_instance)
|
||||
try:
|
||||
self._execute_update_now(plugin_id, plugin_instance,
|
||||
scheduled_time, lock=lock)
|
||||
@@ -1228,6 +1428,142 @@ class PluginManager:
|
||||
self.logger.exception("update worker: unexpected error for %s",
|
||||
plugin_id)
|
||||
|
||||
def _skip_busy_update(self, plugin_id: str, waited: float) -> None:
|
||||
"""Give up on a queued update whose plugin lock stayed held.
|
||||
|
||||
Same bookkeeping as a failed update() -- pending slot dropped before
|
||||
the state returns to ENABLED with PluginBusyError error info,
|
||||
last-update stamped so the retry waits a full interval -- but
|
||||
report-only in health: counted as a busy skip (``busy_skip_count`` /
|
||||
``last_busy_skip``), never as a failure or a hang. The lock holder
|
||||
may be perfectly healthy: Vegas prefetch holds a plugin's lock for its
|
||||
whole content render, which on a slow Pi can outlast
|
||||
PLUGIN_LOCK_TIMEOUT, and counting that would pull a healthy plugin
|
||||
from rotation. Real hangs -- display() or update() past the executor
|
||||
timeout -- are recorded where they are measured and still open the
|
||||
breaker.
|
||||
"""
|
||||
with self._pending_lock:
|
||||
self._pending_updates.discard(plugin_id)
|
||||
if plugin_id not in self.plugins:
|
||||
# Unloaded while we waited: its lifecycle state is already
|
||||
# cleared; recording anything would resurrect it as ENABLED.
|
||||
return
|
||||
self._warn_rate_limited(
|
||||
"busy-update:" + plugin_id,
|
||||
"Plugin %s update skipped: its lock was still held after %.1fs "
|
||||
"(a display(), Vegas render or update() of it is still running); "
|
||||
"retrying next interval, not counted as a failure", plugin_id, waited)
|
||||
self._record_update_failure(
|
||||
plugin_id,
|
||||
exc=PluginBusyError(
|
||||
f"Plugin {plugin_id} busy: its lock was held for over {waited:.1f}s "
|
||||
"by a slow or hung display()/update(); update skipped"),
|
||||
log=False,
|
||||
count_failure=False)
|
||||
tracker = self.health_tracker
|
||||
record_busy = getattr(tracker, 'record_busy_skip', None) if tracker is not None else None
|
||||
if callable(record_busy):
|
||||
try:
|
||||
record_busy(plugin_id, 'update lock wait', waited)
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
self.logger.debug("Could not record busy skip for %s: %s", plugin_id, e)
|
||||
|
||||
def apply_config_change(self, plugin_id: str, new_config: Dict[str, Any],
|
||||
plugin_instance: Optional[Any] = None) -> bool:
|
||||
"""Call ``on_config_change(new_config)`` without racing update()/display().
|
||||
|
||||
Runs on the calling thread -- ConfigService's watcher, for the display
|
||||
service -- holding the plugin's lock, waited on for at most
|
||||
PLUGIN_LOCK_TIMEOUT. If the lock is still busy (an update() mid-fetch
|
||||
can outlast that) the change is parked and handed to the update
|
||||
worker, which applies it under the same lock once it is free, and at
|
||||
the latest just before the plugin's next update(). A later change for
|
||||
the same plugin replaces a parked one.
|
||||
|
||||
Exceptions from on_config_change propagate on the immediate path,
|
||||
as they did when the caller invoked it directly.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier.
|
||||
new_config: The prepared config to hand the plugin.
|
||||
plugin_instance: The instance to notify; defaults to the loaded one.
|
||||
|
||||
Returns:
|
||||
True if on_config_change ran now, False if it was deferred or there
|
||||
is no loaded plugin to notify.
|
||||
"""
|
||||
if plugin_instance is None:
|
||||
plugin_instance = self.plugins.get(plugin_id)
|
||||
if plugin_instance is None or not hasattr(plugin_instance, 'on_config_change'):
|
||||
return False
|
||||
lock = self.get_plugin_lock(plugin_id)
|
||||
if lock.acquire(timeout=self.PLUGIN_LOCK_TIMEOUT):
|
||||
try:
|
||||
with self._deferred_config_lock:
|
||||
# This change supersedes any older one still parked.
|
||||
self._deferred_config_changes.pop(plugin_id, None)
|
||||
plugin_instance.on_config_change(new_config)
|
||||
finally:
|
||||
lock.release()
|
||||
return True
|
||||
|
||||
with self._deferred_config_lock:
|
||||
self._deferred_config_changes[plugin_id] = (plugin_instance, new_config)
|
||||
self._warn_rate_limited(
|
||||
"busy-config:" + plugin_id,
|
||||
"Plugin %s is busy (lock held for over %.1fs); its config change "
|
||||
"will be applied by the update worker once it is free",
|
||||
plugin_id, self.PLUGIN_LOCK_TIMEOUT)
|
||||
try:
|
||||
self._ensure_update_worker()
|
||||
self._update_queue.put(_DeferredConfigChange(plugin_id))
|
||||
except Exception as exc: # pylint: disable=broad-except
|
||||
# No worker (thread start refused): still parked, so the next
|
||||
# update() of this plugin applies it.
|
||||
self.logger.error(
|
||||
"Could not queue the config change for plugin %s (%s: %s); it "
|
||||
"will be applied before its next update()",
|
||||
plugin_id, type(exc).__name__, exc)
|
||||
return False
|
||||
|
||||
def _apply_deferred_config_change(self, plugin_id: str) -> None:
|
||||
"""Worker side of a parked config change: take the lock, apply it."""
|
||||
with self._deferred_config_lock:
|
||||
if plugin_id not in self._deferred_config_changes:
|
||||
return # applied or superseded meanwhile
|
||||
lock = self.get_plugin_lock(plugin_id)
|
||||
wait_start = time.monotonic()
|
||||
if not lock.acquire(timeout=self.PLUGIN_LOCK_TIMEOUT):
|
||||
self._warn_rate_limited(
|
||||
"busy-config:" + plugin_id,
|
||||
"Plugin %s still busy after %.1fs; its config change stays "
|
||||
"parked until its next update()",
|
||||
plugin_id, time.monotonic() - wait_start)
|
||||
return
|
||||
try:
|
||||
self._apply_deferred_config_locked(plugin_id, self.plugins.get(plugin_id))
|
||||
finally:
|
||||
lock.release()
|
||||
|
||||
def _apply_deferred_config_locked(self, plugin_id: str,
|
||||
current_instance: Optional[Any]) -> None:
|
||||
"""Apply the parked config change for plugin_id; caller holds its lock."""
|
||||
with self._deferred_config_lock:
|
||||
entry = self._deferred_config_changes.pop(plugin_id, None)
|
||||
if entry is None:
|
||||
return
|
||||
instance, new_config = entry
|
||||
if current_instance is None or instance is not current_instance:
|
||||
# Unloaded, or reloaded as a new instance built from the current
|
||||
# config: nothing left to tell.
|
||||
return
|
||||
try:
|
||||
instance.on_config_change(new_config)
|
||||
self.logger.info("Applied deferred config change for plugin %s", plugin_id)
|
||||
except Exception: # pylint: disable=broad-except
|
||||
self.logger.exception("Error in plugin %s config change handler", plugin_id)
|
||||
|
||||
def stop_update_worker(self, timeout: float = 5.0) -> None:
|
||||
"""Signal the worker to exit (used by cleanup; thread is a daemon)."""
|
||||
if self._update_worker is not None and self._update_worker.is_alive():
|
||||
@@ -1338,14 +1674,29 @@ class PluginManager:
|
||||
else:
|
||||
_finish(True)
|
||||
|
||||
started = time.monotonic()
|
||||
try:
|
||||
self.plugin_executor.execute_update(
|
||||
success = self.plugin_executor.execute_update(
|
||||
types.SimpleNamespace(update=_target_update), plugin_id)
|
||||
except Exception as exc: # pragma: no cover - defensive; execute_update
|
||||
# catches everything internally, but guarantee _finish still
|
||||
# runs (releasing the lock) if something unexpected slips through.
|
||||
self.logger.exception("Unexpected error dispatching update for %s: %s", plugin_id, exc)
|
||||
_finish(False, exc=exc)
|
||||
return
|
||||
if not success and not finished['done']:
|
||||
# The executor stopped waiting but update() is still running: it
|
||||
# keeps the lock and the RUNNING state until it returns (then
|
||||
# _finish records the outcome). Say so now, rather than leave the
|
||||
# plugin silently stuck; record_success on a late return clears it.
|
||||
elapsed = time.monotonic() - started
|
||||
self._warn_rate_limited(
|
||||
"hung-update:" + plugin_id,
|
||||
"Plugin %s update() still running after %.1fs; it keeps its "
|
||||
"lock until it returns, and is not rescheduled until then",
|
||||
plugin_id, elapsed)
|
||||
self._record_hang(plugin_id, 'update', elapsed, PluginTimeoutError(
|
||||
f"Plugin {plugin_id} update() still running after {elapsed:.1f}s"))
|
||||
|
||||
def run_scheduled_updates_with_changes(self, current_time: Optional[float] = None) -> List[str]:
|
||||
"""
|
||||
|
||||
@@ -0,0 +1,377 @@
|
||||
"""The display's plugin runtime snapshot, shared with the web interface.
|
||||
|
||||
Only the display process runs plugins, so only it knows which ones it has
|
||||
loaded, where each is in its lifecycle (``plugin_state.PluginStateManager``),
|
||||
why one failed and which version it is running. It publishes that to the
|
||||
shared cache directory -- the channel, and the file permissions, that the
|
||||
error snapshot, plugin health and ``display_current_state`` already use --
|
||||
and the web interface reads it back for ``/api/v3/plugins/installed``,
|
||||
``/api/v3/plugins/state`` and state reconciliation.
|
||||
|
||||
PLUGIN_RUNTIME_KEY written by the display service only
|
||||
|
||||
Writes. The cache lives on disk, usually the SD card, so the snapshot is
|
||||
written when something a reader would see changes, at most once every
|
||||
``MIN_INTERVAL`` seconds, and otherwise once every ``REFRESH_INTERVAL``
|
||||
seconds as a heartbeat. An ordinary plugin update is not a change: the
|
||||
RUNNING state it passes through is published as ENABLED
|
||||
(``plugin_state.published_state``). A display with nothing changing writes
|
||||
this one small file once a minute.
|
||||
|
||||
Staleness. Every snapshot carries ``published_at`` (wall clock) and
|
||||
``stale_after``. A reader treats a snapshot older than that as unknown, not
|
||||
as the truth: a display that died without cleaning up leaves its last
|
||||
snapshot behind. A display that stops cleanly publishes ``running: false``
|
||||
on the way out, so readers see "stopped" at once rather than after the
|
||||
stale window. Nothing on the reading side reports a runtime fact from a
|
||||
snapshot that is not live.
|
||||
"""
|
||||
|
||||
import math
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Callable, Dict, Optional
|
||||
|
||||
from src.logging_config import get_logger
|
||||
from src.redaction import redact_credentials
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
PLUGIN_RUNTIME_KEY = "plugin_runtime_snapshot"
|
||||
SNAPSHOT_SCHEMA = 1
|
||||
|
||||
#: Shortest gap, in seconds, between two change-driven writes. Startup loads
|
||||
#: every plugin in a burst, and a plugin failing each update cycle changes its
|
||||
#: error info each time; either is written at most this often.
|
||||
MIN_INTERVAL = 10.0
|
||||
|
||||
#: An unchanged snapshot is rewritten this often so readers can tell a quiet
|
||||
#: display from a dead one.
|
||||
REFRESH_INTERVAL = 60.0
|
||||
|
||||
#: How often the publisher thread looks for changes: an in-memory comparison.
|
||||
TICK_INTERVAL = 5.0
|
||||
|
||||
#: A snapshot older than this is stale: three missed refreshes.
|
||||
STALE_AFTER = 3 * REFRESH_INTERVAL
|
||||
|
||||
#: Bounds on a published ``stale_after``, so a corrupt value can make a
|
||||
#: reader neither trust a dead display for hours nor distrust a live one.
|
||||
_STALE_AFTER_MIN = 30.0
|
||||
_STALE_AFTER_MAX = 3600.0
|
||||
|
||||
_ERROR_MESSAGE_CHARS = 200
|
||||
_ERROR_TYPE_CHARS = 80
|
||||
_ID_CHARS = 100
|
||||
_VERSION_CHARS = 40
|
||||
|
||||
#: Reader statuses. Only LIVE carries runtime facts.
|
||||
LIVE = "live"
|
||||
STALE = "stale"
|
||||
STOPPED = "stopped"
|
||||
UNKNOWN = "unknown"
|
||||
|
||||
|
||||
def _clip(value: Any, limit: int) -> str:
|
||||
text = value if isinstance(value, str) else str(value)
|
||||
return text if len(text) <= limit else text[:limit - 3] + "..."
|
||||
|
||||
|
||||
def _epoch(value: Any) -> Optional[float]:
|
||||
"""Seconds since the epoch for a float or a datetime; None otherwise."""
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
if isinstance(value, (int, float)):
|
||||
number = float(value)
|
||||
return number if math.isfinite(number) else None
|
||||
timestamp = getattr(value, "timestamp", None)
|
||||
if callable(timestamp):
|
||||
try:
|
||||
number = float(timestamp())
|
||||
except (TypeError, ValueError, OverflowError, OSError):
|
||||
return None
|
||||
return number if math.isfinite(number) else None
|
||||
return None
|
||||
|
||||
|
||||
def summarize_error(error_info: Optional[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
|
||||
"""A short, redacted summary of the state machine's error info.
|
||||
|
||||
``message`` is redacted before it is clipped: clipping first could cut a
|
||||
``token=`` marker off and keep the secret after it. No stack trace: the
|
||||
full error, with its trace, is in the error snapshot (/api/v3/errors).
|
||||
"""
|
||||
if not isinstance(error_info, dict):
|
||||
return None
|
||||
message = error_info.get("error")
|
||||
error_type = error_info.get("error_type")
|
||||
return {
|
||||
"type": _clip(error_type, _ERROR_TYPE_CHARS) if error_type else None,
|
||||
"message": _clip(redact_credentials(message if isinstance(message, str)
|
||||
else str(message or "")),
|
||||
_ERROR_MESSAGE_CHARS),
|
||||
"at": _epoch(error_info.get("timestamp")),
|
||||
"recoverable": bool(error_info.get("recoverable", False)),
|
||||
}
|
||||
|
||||
|
||||
def build_runtime_snapshot(state_manager: Any, *, started_at: float,
|
||||
now: Optional[float] = None,
|
||||
running: bool = True) -> Dict[str, Any]:
|
||||
"""The snapshot for ``state_manager`` (a plugin_state.PluginStateManager).
|
||||
|
||||
A stopped snapshot (``running=False``) lists no plugins: nothing is
|
||||
loaded once the display has gone.
|
||||
"""
|
||||
plugins: Dict[str, Dict[str, Any]] = {}
|
||||
if running:
|
||||
for plugin_id, record in state_manager.runtime_records().items():
|
||||
version = record.get("version")
|
||||
plugins[_clip(plugin_id, _ID_CHARS)] = {
|
||||
"loaded": bool(record.get("loaded")),
|
||||
"state": record.get("state"),
|
||||
"error": summarize_error(record.get("error_info")),
|
||||
"version": _clip(version, _VERSION_CHARS) if version else None,
|
||||
"loaded_at": _epoch(record.get("loaded_at")),
|
||||
}
|
||||
return {
|
||||
"schema": SNAPSHOT_SCHEMA,
|
||||
"running": running,
|
||||
"published_at": time.time() if now is None else now,
|
||||
"started_at": started_at,
|
||||
"refresh_interval": REFRESH_INTERVAL,
|
||||
"stale_after": STALE_AFTER,
|
||||
"pid": os.getpid(),
|
||||
"plugins": plugins,
|
||||
}
|
||||
|
||||
|
||||
class PluginRuntimePublisher:
|
||||
"""Publishes the display's plugin state machine to the shared cache.
|
||||
|
||||
Runs in the display service only. tick() is the whole job; start() calls
|
||||
it from a daemon thread every TICK_INTERVAL seconds. Nothing here raises:
|
||||
a failed write is logged at debug and retried on a later tick, at the
|
||||
throttled rate.
|
||||
"""
|
||||
|
||||
def __init__(self, cache_manager: Any, state_manager: Any,
|
||||
min_interval: float = MIN_INTERVAL,
|
||||
refresh_interval: float = REFRESH_INTERVAL,
|
||||
clock: Callable[[], float] = time.monotonic,
|
||||
wall_clock: Callable[[], float] = time.time) -> None:
|
||||
self.cache_manager = cache_manager
|
||||
self.state_manager = state_manager
|
||||
self.min_interval = min_interval
|
||||
self.refresh_interval = refresh_interval
|
||||
self._clock = clock
|
||||
self._wall_clock = wall_clock
|
||||
self.started_at = wall_clock()
|
||||
# None forces a first publish, which replaces whatever a previous run
|
||||
# of the service left behind.
|
||||
self._published_change: Optional[int] = None
|
||||
self._last_attempt: Optional[float] = None
|
||||
self._tick_lock = threading.Lock()
|
||||
self._stop = threading.Event()
|
||||
self._thread: Optional[threading.Thread] = None
|
||||
|
||||
def _write(self, running: bool) -> None:
|
||||
snapshot = build_runtime_snapshot(self.state_manager, started_at=self.started_at,
|
||||
now=self._wall_clock(), running=running)
|
||||
self.cache_manager.set(PLUGIN_RUNTIME_KEY, snapshot)
|
||||
|
||||
def tick(self) -> bool:
|
||||
"""Publish if something changed (throttled) or the refresh is due.
|
||||
True if a snapshot was written."""
|
||||
with self._tick_lock:
|
||||
try:
|
||||
change = self.state_manager.change_count
|
||||
now = self._clock()
|
||||
since = None if self._last_attempt is None else now - self._last_attempt
|
||||
if since is not None:
|
||||
if change == self._published_change:
|
||||
if since < self.refresh_interval:
|
||||
return False
|
||||
elif since < self.min_interval:
|
||||
return False
|
||||
# Stamp the attempt before writing: a cache that keeps failing
|
||||
# is retried at the throttled rate, not on every tick.
|
||||
self._last_attempt = now
|
||||
self._write(running=True)
|
||||
self._published_change = change
|
||||
return True
|
||||
except Exception as err: # never let reporting break the display
|
||||
logger.debug("Could not publish the plugin runtime snapshot: %s",
|
||||
err, exc_info=True)
|
||||
return False
|
||||
|
||||
def start(self, interval: float = TICK_INTERVAL) -> None:
|
||||
"""Tick from a daemon thread until stop(). A no-op while running."""
|
||||
if self._thread is not None and self._thread.is_alive():
|
||||
return
|
||||
self._stop.clear()
|
||||
|
||||
def run() -> None:
|
||||
self.tick()
|
||||
while not self._stop.wait(interval):
|
||||
self.tick()
|
||||
|
||||
self._thread = threading.Thread(target=run, name="plugin-runtime-publisher",
|
||||
daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def stop(self, publish_stopped: bool = True) -> None:
|
||||
"""Stop ticking and, by default, publish ``running: false`` so readers
|
||||
see the display as stopped now rather than after the stale window."""
|
||||
self._stop.set()
|
||||
if self._thread is not None:
|
||||
self._thread.join(timeout=2)
|
||||
self._thread = None
|
||||
if publish_stopped:
|
||||
with self._tick_lock:
|
||||
try:
|
||||
self._write(running=False)
|
||||
except Exception as err:
|
||||
logger.debug("Could not publish the stopped plugin runtime snapshot: %s",
|
||||
err, exc_info=True)
|
||||
|
||||
|
||||
def start_plugin_runtime_publisher(cache_manager: Any,
|
||||
state_manager: Any) -> Optional[PluginRuntimePublisher]:
|
||||
"""Start publishing the display's plugin runtime state. Display service
|
||||
only: whichever process calls it becomes the source readers trust.
|
||||
Never raises."""
|
||||
try:
|
||||
publisher = PluginRuntimePublisher(cache_manager, state_manager)
|
||||
publisher.start()
|
||||
return publisher
|
||||
except Exception as err:
|
||||
logger.warning("Plugin runtime reporting to the web interface is unavailable: %s", err)
|
||||
return None
|
||||
|
||||
|
||||
# --- Reading side (web interface) -------------------------------------------
|
||||
|
||||
#: What a reader reports for a plugin when it does not know.
|
||||
_UNKNOWN_PLUGIN: Dict[str, Any] = {
|
||||
"loaded": None,
|
||||
"state": None,
|
||||
"error_info": None,
|
||||
"loaded_version": None,
|
||||
"loaded_at": None,
|
||||
}
|
||||
|
||||
#: A plugin a live snapshot does not list: the display has not loaded it
|
||||
#: (never enabled, or unloaded since), which is what its state machine
|
||||
#: reports for an id it has no record of.
|
||||
_NOT_LOADED_PLUGIN: Dict[str, Any] = {
|
||||
"loaded": False,
|
||||
"state": "unloaded",
|
||||
"error_info": None,
|
||||
"loaded_version": None,
|
||||
"loaded_at": None,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PluginRuntimeView:
|
||||
"""What a reader may say about the display's plugins right now.
|
||||
|
||||
``status``: ``live`` (a fresh snapshot from a running display),
|
||||
``stale`` (the last snapshot is older than its ``stale_after``: the
|
||||
display is hung or died without cleaning up), ``stopped`` (the display
|
||||
said so on its way out) or ``unknown`` (no readable snapshot). Only a
|
||||
live view reports per-plugin facts; every other status answers None for
|
||||
them, so a caller cannot pass stale truth on by accident.
|
||||
"""
|
||||
|
||||
status: str
|
||||
published_at: Optional[float] = None
|
||||
age_seconds: Optional[float] = None
|
||||
stale_after: float = STALE_AFTER
|
||||
plugins: Dict[str, Dict[str, Any]] = field(default_factory=dict)
|
||||
|
||||
@property
|
||||
def live(self) -> bool:
|
||||
return self.status == LIVE
|
||||
|
||||
def plugin(self, plugin_id: str) -> Dict[str, Any]:
|
||||
"""``loaded``, ``state``, ``error_info``, ``loaded_version`` and
|
||||
``loaded_at`` for one plugin; all None unless the view is live."""
|
||||
if not self.live:
|
||||
return dict(_UNKNOWN_PLUGIN)
|
||||
record = self.plugins.get(plugin_id)
|
||||
if not isinstance(record, dict):
|
||||
return dict(_NOT_LOADED_PLUGIN)
|
||||
error = record.get("error")
|
||||
return {
|
||||
"loaded": bool(record.get("loaded")),
|
||||
"state": record.get("state") if isinstance(record.get("state"), str) else None,
|
||||
"error_info": dict(error) if isinstance(error, dict) else None,
|
||||
"loaded_version": record.get("version"),
|
||||
"loaded_at": record.get("loaded_at"),
|
||||
}
|
||||
|
||||
def describe(self) -> Dict[str, Any]:
|
||||
"""The view's own status, for a response to carry beside the facts."""
|
||||
return {
|
||||
"status": self.status,
|
||||
"published_at": self.published_at,
|
||||
"age_seconds": None if self.age_seconds is None else round(self.age_seconds, 1),
|
||||
"stale_after": self.stale_after,
|
||||
}
|
||||
|
||||
|
||||
def _stale_after_of(snapshot: Dict[str, Any]) -> float:
|
||||
value = snapshot.get("stale_after")
|
||||
if isinstance(value, bool) or not isinstance(value, (int, float)):
|
||||
return STALE_AFTER
|
||||
number = float(value)
|
||||
if not math.isfinite(number):
|
||||
return STALE_AFTER
|
||||
return min(max(number, _STALE_AFTER_MIN), _STALE_AFTER_MAX)
|
||||
|
||||
|
||||
def view_from_snapshot(snapshot: Any, now: Optional[float] = None) -> PluginRuntimeView:
|
||||
"""Judge a snapshot read from the cache; never raises."""
|
||||
if not isinstance(snapshot, dict) or snapshot.get("schema") != SNAPSHOT_SCHEMA:
|
||||
return PluginRuntimeView(status=UNKNOWN)
|
||||
published_at = _epoch(snapshot.get("published_at"))
|
||||
if published_at is None:
|
||||
return PluginRuntimeView(status=UNKNOWN)
|
||||
stale_after = _stale_after_of(snapshot)
|
||||
age = (time.time() if now is None else now) - published_at
|
||||
if snapshot.get("running") is not True:
|
||||
return PluginRuntimeView(status=STOPPED, published_at=published_at,
|
||||
age_seconds=max(age, 0.0), stale_after=stale_after)
|
||||
# A snapshot from the future is trusted a little: the Pi has no RTC and
|
||||
# its clock steps when NTP syncs. Far in the future, it cannot be dated.
|
||||
if age > stale_after or age < -stale_after:
|
||||
return PluginRuntimeView(status=STALE, published_at=published_at,
|
||||
age_seconds=age, stale_after=stale_after)
|
||||
plugins = snapshot.get("plugins")
|
||||
return PluginRuntimeView(
|
||||
status=LIVE, published_at=published_at, age_seconds=max(age, 0.0),
|
||||
stale_after=stale_after,
|
||||
plugins={k: v for k, v in plugins.items() if isinstance(v, dict)}
|
||||
if isinstance(plugins, dict) else {},
|
||||
)
|
||||
|
||||
|
||||
def read_plugin_runtime(cache_manager: Any, now: Optional[float] = None) -> PluginRuntimeView:
|
||||
"""The display's latest snapshot, judged for staleness. Never raises; a
|
||||
missing cache manager or an unreadable snapshot is ``unknown``.
|
||||
|
||||
memory_ttl=0: the key is written by the other process, so only the file
|
||||
is current.
|
||||
"""
|
||||
if cache_manager is None:
|
||||
return PluginRuntimeView(status=UNKNOWN)
|
||||
try:
|
||||
snapshot = cache_manager.get(PLUGIN_RUNTIME_KEY, max_age=None, memory_ttl=0)
|
||||
except Exception as err:
|
||||
logger.debug("Could not read the plugin runtime snapshot: %s", err, exc_info=True)
|
||||
return PluginRuntimeView(status=UNKNOWN)
|
||||
return view_from_snapshot(snapshot, now=now)
|
||||
@@ -1,11 +1,14 @@
|
||||
"""
|
||||
Plugin State Management
|
||||
|
||||
Manages plugin state machine (loaded → enabled → running → error)
|
||||
with state transitions and queries.
|
||||
The display process's plugin state machine (loaded → enabled → running →
|
||||
error), with state transitions and queries. It is the only record of plugin
|
||||
lifecycle state: the web process runs no plugins, and reads this state as the
|
||||
snapshot ``plugin_runtime.PluginRuntimePublisher`` publishes from it.
|
||||
"""
|
||||
|
||||
import threading
|
||||
import time
|
||||
from enum import Enum
|
||||
from typing import Optional, Dict, Any
|
||||
from datetime import datetime
|
||||
@@ -24,8 +27,27 @@ class PluginState(Enum):
|
||||
DISABLED = "disabled" # Plugin is disabled in config
|
||||
|
||||
|
||||
def published_state(state: PluginState) -> PluginState:
|
||||
"""The state as readers outside the scheduler see it.
|
||||
|
||||
RUNNING is the scheduler's claim on a plugin for one update() call: every
|
||||
update flips ENABLED -> RUNNING -> ENABLED. Published as is, that would be
|
||||
a change -- and a cache write to the SD card -- on every plugin update, and
|
||||
a reader would see a plugin blink between two states that mean the same
|
||||
thing to it (loaded and taking part). Readers get ENABLED for both.
|
||||
"""
|
||||
return PluginState.ENABLED if state == PluginState.RUNNING else state
|
||||
|
||||
|
||||
class PluginStateManager:
|
||||
"""Manages plugin state transitions and queries."""
|
||||
"""Manages plugin state transitions and queries.
|
||||
|
||||
Owned by the display process's PluginManager. ``change_count`` moves
|
||||
whenever something a reader of the published snapshot would see changes
|
||||
(published state, error info, the loaded record) and stays put across the
|
||||
RUNNING/ENABLED flip of an ordinary update, so a publisher can tell
|
||||
"nothing new" without diffing.
|
||||
"""
|
||||
|
||||
def __init__(self, logger: Optional[logging.Logger] = None) -> None:
|
||||
"""
|
||||
@@ -41,6 +63,20 @@ class PluginStateManager:
|
||||
self._state_transition_counts: Dict[str, int] = {}
|
||||
self._error_info: Dict[str, Dict[str, Any]] = {}
|
||||
self._last_update: Dict[str, datetime] = {}
|
||||
# What load_plugin() registered: {'version', 'loaded_at'} per plugin
|
||||
# whose instance is live. Cleared with the rest of its state on unload.
|
||||
self._loaded: Dict[str, Dict[str, Any]] = {}
|
||||
self._change_count = 0
|
||||
|
||||
@property
|
||||
def change_count(self) -> int:
|
||||
"""Moves on every change a published snapshot would show."""
|
||||
with self._lock:
|
||||
return self._change_count
|
||||
|
||||
def _note_change(self) -> None:
|
||||
"""Count a reader-visible change. Callers must already hold ``_lock``."""
|
||||
self._change_count += 1
|
||||
|
||||
def _record_transition(self, plugin_id: str) -> None:
|
||||
"""Count a state transition. Callers must already hold ``_lock``."""
|
||||
@@ -63,9 +99,12 @@ class PluginStateManager:
|
||||
error: Optional error if transitioning to ERROR state
|
||||
"""
|
||||
with self._lock:
|
||||
known = plugin_id in self._states
|
||||
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
|
||||
self._states[plugin_id] = state
|
||||
self._record_transition(plugin_id)
|
||||
if not known or published_state(old_state) != published_state(state):
|
||||
self._note_change()
|
||||
|
||||
# Store error info if transitioning to ERROR state
|
||||
if state == PluginState.ERROR and error:
|
||||
@@ -74,9 +113,11 @@ class PluginStateManager:
|
||||
'error_type': type(error).__name__,
|
||||
'timestamp': datetime.now()
|
||||
}
|
||||
self._note_change()
|
||||
elif state != PluginState.ERROR:
|
||||
# Clear error info when leaving ERROR state
|
||||
self._error_info.pop(plugin_id, None)
|
||||
if self._error_info.pop(plugin_id, None) is not None:
|
||||
self._note_change()
|
||||
|
||||
self.logger.debug(
|
||||
"Plugin %s state transition: %s → %s",
|
||||
@@ -147,6 +188,7 @@ class PluginStateManager:
|
||||
self._states[plugin_id] = state
|
||||
self._record_transition(plugin_id)
|
||||
self._error_info[plugin_id] = dict(error_info)
|
||||
self._note_change()
|
||||
|
||||
self.logger.debug(
|
||||
"Plugin %s state transition: %s → %s (recoverable error stored)",
|
||||
@@ -173,6 +215,52 @@ class PluginStateManager:
|
||||
info = self._error_info.get(plugin_id)
|
||||
return dict(info) if info is not None else None
|
||||
|
||||
def record_loaded(self, plugin_id: str, version: Optional[str],
|
||||
loaded_at: Optional[float] = None) -> None:
|
||||
"""Record that ``plugin_id``'s instance is live, and which version.
|
||||
|
||||
Called by PluginManager.load_plugin() once the instance is registered;
|
||||
clear_state() (unload) forgets it. ``version`` is the manifest's at
|
||||
load time, which is what the display keeps running until it reloads
|
||||
the plugin -- the version on disk can move on after a store update.
|
||||
"""
|
||||
with self._lock:
|
||||
self._loaded[plugin_id] = {
|
||||
'version': version,
|
||||
'loaded_at': time.time() if loaded_at is None else loaded_at,
|
||||
}
|
||||
self._note_change()
|
||||
|
||||
def record_unloaded(self, plugin_id: str) -> None:
|
||||
"""Forget the loaded record alone, keeping state and error info: for
|
||||
an unload that failed after the instance was already dropped."""
|
||||
with self._lock:
|
||||
if self._loaded.pop(plugin_id, None) is not None:
|
||||
self._note_change()
|
||||
|
||||
def runtime_records(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Every known plugin's reader-visible state, taken in one critical
|
||||
section so a concurrent load or unload is seen whole or not at all.
|
||||
|
||||
Per plugin: ``state`` (published_state()'s value), ``loaded``,
|
||||
``version`` and ``loaded_at`` (None unless loaded) and ``error_info``
|
||||
(a copy, or None).
|
||||
"""
|
||||
with self._lock:
|
||||
records: Dict[str, Dict[str, Any]] = {}
|
||||
for plugin_id in set(self._states) | set(self._loaded):
|
||||
loaded = self._loaded.get(plugin_id)
|
||||
info = self._error_info.get(plugin_id)
|
||||
records[plugin_id] = {
|
||||
'state': published_state(
|
||||
self._states.get(plugin_id, PluginState.UNLOADED)).value,
|
||||
'loaded': loaded is not None,
|
||||
'version': loaded['version'] if loaded else None,
|
||||
'loaded_at': loaded['loaded_at'] if loaded else None,
|
||||
'error_info': dict(info) if info is not None else None,
|
||||
}
|
||||
return records
|
||||
|
||||
def record_update(self, plugin_id: str) -> None:
|
||||
"""Record that plugin update() was called."""
|
||||
self._last_update[plugin_id] = datetime.now()
|
||||
@@ -221,8 +309,13 @@ class PluginStateManager:
|
||||
state.
|
||||
"""
|
||||
with self._lock:
|
||||
had = (plugin_id in self._states or plugin_id in self._loaded
|
||||
or plugin_id in self._error_info)
|
||||
self._states.pop(plugin_id, None)
|
||||
self._state_transition_counts.pop(plugin_id, None)
|
||||
self._error_info.pop(plugin_id, None)
|
||||
self._last_update.pop(plugin_id, None)
|
||||
self._loaded.pop(plugin_id, None)
|
||||
if had:
|
||||
self._note_change()
|
||||
|
||||
|
||||
@@ -127,8 +127,9 @@ CORE_PLUGIN_PROPERTIES: Dict[str, Dict[str, Any]] = {
|
||||
"description": "Enable live priority takeover when plugin has live content"
|
||||
},
|
||||
# Vegas tuning read by vegas_mode/plugin_adapter.py and base_plugin.py.
|
||||
# Left untyped: the adapter validates them itself and ignores a bad
|
||||
# value with a log line, so a stored one must never block a save.
|
||||
# These three are left untyped: the adapter validates them itself and
|
||||
# ignores a bad value with a log line, so a stored one must never block a
|
||||
# save.
|
||||
"vegas_width_pct": {
|
||||
"description": "Vegas mode: width of this plugin's card, as a percentage of the panel"
|
||||
},
|
||||
@@ -138,6 +139,22 @@ CORE_PLUGIN_PROPERTIES: Dict[str, Dict[str, Any]] = {
|
||||
"vegas_max_width_screens": {
|
||||
"description": "Vegas mode: widest this plugin's card may be, in screens"
|
||||
},
|
||||
# Read by resolve_vegas_participation / BasePlugin.get_vegas_participation.
|
||||
# An enum with no default: a default would be written into every plugin's
|
||||
# config and override the participation the plugin itself declares.
|
||||
"vegas_participation": {
|
||||
"type": "string",
|
||||
"enum": ["scroll", "pause", "exclude"],
|
||||
"title": "Vegas participation",
|
||||
"description": (
|
||||
"Vegas mode: how this plugin takes part in the scrolling ticker. "
|
||||
"'scroll' = its content scrolls by with everything else; "
|
||||
"'pause' = the ticker stops for this plugin's turn and shows it "
|
||||
"full screen for its display duration; "
|
||||
"'exclude' = leave it out of Vegas mode. "
|
||||
"Leave unset to use the plugin's own default."
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
#: The keys of CORE_PLUGIN_PROPERTIES that are Vegas tuning rather than plugin
|
||||
@@ -145,6 +162,7 @@ CORE_PLUGIN_PROPERTIES: Dict[str, Dict[str, Any]] = {
|
||||
#: PluginManager.CORE_OWNED_CONFIG_KEYS).
|
||||
CORE_VEGAS_TUNING_KEYS = frozenset({
|
||||
'vegas_width_pct', 'vegas_overflow', 'vegas_max_width_screens',
|
||||
'vegas_participation',
|
||||
})
|
||||
|
||||
|
||||
|
||||
@@ -1,343 +0,0 @@
|
||||
"""
|
||||
Centralized plugin state management.
|
||||
|
||||
Provides a single source of truth for plugin state (installed, enabled, version, etc.)
|
||||
with persistence.
|
||||
"""
|
||||
|
||||
import json
|
||||
import threading
|
||||
from typing import Dict, Any, Optional
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
from dataclasses import dataclass, asdict
|
||||
from enum import Enum
|
||||
|
||||
from src.config_manager_atomic import atomic_write_text
|
||||
from src.logging_config import get_logger
|
||||
|
||||
|
||||
class PluginStateStatus(Enum):
|
||||
"""Status of a plugin."""
|
||||
INSTALLED = "installed"
|
||||
ENABLED = "enabled"
|
||||
DISABLED = "disabled"
|
||||
ERROR = "error"
|
||||
UNKNOWN = "unknown"
|
||||
|
||||
|
||||
@dataclass
|
||||
class PluginState:
|
||||
"""Represents the state of a plugin."""
|
||||
plugin_id: str
|
||||
status: PluginStateStatus
|
||||
enabled: bool
|
||||
version: Optional[str] = None
|
||||
installed_at: Optional[datetime] = None
|
||||
last_updated: Optional[datetime] = None
|
||||
# Bumped on every update_plugin_state(). Nothing reads it; it stays so
|
||||
# plugin_state.json keeps the shape older releases load with cls(**data).
|
||||
config_version: int = 1
|
||||
metadata: Dict[str, Any] = None
|
||||
|
||||
def __post_init__(self):
|
||||
if self.metadata is None:
|
||||
self.metadata = {}
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert state to dictionary for serialization."""
|
||||
result = asdict(self)
|
||||
# Convert enum to string
|
||||
result['status'] = self.status.value
|
||||
# Convert datetime to ISO string
|
||||
if result.get('installed_at'):
|
||||
result['installed_at'] = self.installed_at.isoformat()
|
||||
if result.get('last_updated'):
|
||||
result['last_updated'] = self.last_updated.isoformat()
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: Dict[str, Any]) -> 'PluginState':
|
||||
"""Create state from dictionary."""
|
||||
# Parse enum
|
||||
if isinstance(data.get('status'), str):
|
||||
data['status'] = PluginStateStatus(data['status'])
|
||||
|
||||
# Parse datetime
|
||||
if data.get('installed_at') and isinstance(data['installed_at'], str):
|
||||
data['installed_at'] = datetime.fromisoformat(data['installed_at'])
|
||||
if data.get('last_updated') and isinstance(data['last_updated'], str):
|
||||
data['last_updated'] = datetime.fromisoformat(data['last_updated'])
|
||||
|
||||
return cls(**data)
|
||||
|
||||
|
||||
class PluginStateManager:
|
||||
"""
|
||||
Centralized plugin state manager.
|
||||
|
||||
Provides:
|
||||
- Single source of truth for plugin state
|
||||
- State persistence
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
state_file: Optional[str] = None,
|
||||
auto_save: bool = True,
|
||||
lazy_load: bool = False
|
||||
):
|
||||
"""
|
||||
Initialize state manager.
|
||||
|
||||
Args:
|
||||
state_file: Path to file for persisting state
|
||||
auto_save: Whether to automatically save state on changes
|
||||
lazy_load: If True, defer loading state file until first access
|
||||
"""
|
||||
self.logger = get_logger(__name__)
|
||||
self.state_file = Path(state_file) if state_file else None
|
||||
self.auto_save = auto_save
|
||||
self._lazy_load = lazy_load
|
||||
self._state_loaded = False
|
||||
|
||||
# State storage
|
||||
self._states: Dict[str, PluginState] = {}
|
||||
# The file's top-level "version", written back as read. Nothing
|
||||
# checks it yet; it is there for a future format change to branch on.
|
||||
self._state_version = 1
|
||||
|
||||
# Threading
|
||||
self._lock = threading.RLock()
|
||||
|
||||
# Load state from file if it exists (unless lazy loading)
|
||||
if not self._lazy_load and self.state_file and self.state_file.exists():
|
||||
self._load_state()
|
||||
self._state_loaded = True
|
||||
|
||||
def _ensure_loaded(self) -> None:
|
||||
"""Ensure state is loaded (for lazy loading)."""
|
||||
if not self._state_loaded and self.state_file and self.state_file.exists():
|
||||
self._load_state()
|
||||
self._state_loaded = True
|
||||
|
||||
def get_plugin_state(self, plugin_id: str) -> Optional[PluginState]:
|
||||
"""
|
||||
Get state for a plugin.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
|
||||
Returns:
|
||||
PluginState if found, None otherwise
|
||||
"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return self._states.get(plugin_id)
|
||||
|
||||
def get_all_states(self) -> Dict[str, PluginState]:
|
||||
"""
|
||||
Get all plugin states.
|
||||
|
||||
Returns:
|
||||
Dictionary mapping plugin_id to PluginState
|
||||
"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
return self._states.copy()
|
||||
|
||||
def update_plugin_state(
|
||||
self,
|
||||
plugin_id: str,
|
||||
updates: Dict[str, Any]
|
||||
) -> bool:
|
||||
"""
|
||||
Update plugin state.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
updates: Dictionary of state updates
|
||||
|
||||
Returns:
|
||||
True if update successful
|
||||
"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
# Get current state or create new
|
||||
current_state = self._states.get(plugin_id)
|
||||
if not current_state:
|
||||
current_state = PluginState(
|
||||
plugin_id=plugin_id,
|
||||
status=PluginStateStatus.UNKNOWN,
|
||||
enabled=False
|
||||
)
|
||||
|
||||
# Apply updates
|
||||
if 'status' in updates:
|
||||
if isinstance(updates['status'], str):
|
||||
current_state.status = PluginStateStatus(updates['status'])
|
||||
else:
|
||||
current_state.status = updates['status']
|
||||
|
||||
if 'enabled' in updates:
|
||||
current_state.enabled = bool(updates['enabled'])
|
||||
|
||||
if 'version' in updates:
|
||||
current_state.version = updates['version']
|
||||
|
||||
if 'installed_at' in updates:
|
||||
current_state.installed_at = updates['installed_at']
|
||||
|
||||
if 'last_updated' in updates:
|
||||
current_state.last_updated = updates['last_updated']
|
||||
else:
|
||||
current_state.last_updated = datetime.now()
|
||||
|
||||
if 'metadata' in updates:
|
||||
if current_state.metadata is None:
|
||||
current_state.metadata = {}
|
||||
current_state.metadata.update(updates['metadata'])
|
||||
|
||||
current_state.config_version += 1
|
||||
|
||||
# Store updated state
|
||||
self._states[plugin_id] = current_state
|
||||
|
||||
# Auto-save if enabled
|
||||
if self.auto_save:
|
||||
self._save_state()
|
||||
|
||||
return True
|
||||
|
||||
def set_plugin_enabled(self, plugin_id: str, enabled: bool) -> bool:
|
||||
"""
|
||||
Set plugin enabled/disabled state.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
enabled: Whether plugin is enabled
|
||||
|
||||
Returns:
|
||||
True if update successful
|
||||
"""
|
||||
status = PluginStateStatus.ENABLED if enabled else PluginStateStatus.DISABLED
|
||||
return self.update_plugin_state(
|
||||
plugin_id,
|
||||
{
|
||||
'enabled': enabled,
|
||||
'status': status
|
||||
}
|
||||
)
|
||||
|
||||
def set_plugin_installed(
|
||||
self,
|
||||
plugin_id: str,
|
||||
version: Optional[str] = None,
|
||||
installed_at: Optional[datetime] = None
|
||||
) -> bool:
|
||||
"""
|
||||
Mark plugin as installed.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
version: Plugin version
|
||||
installed_at: Installation timestamp
|
||||
|
||||
Returns:
|
||||
True if update successful
|
||||
"""
|
||||
return self.update_plugin_state(
|
||||
plugin_id,
|
||||
{
|
||||
'status': PluginStateStatus.INSTALLED,
|
||||
'version': version,
|
||||
'installed_at': installed_at or datetime.now()
|
||||
}
|
||||
)
|
||||
|
||||
def remove_plugin_state(self, plugin_id: str) -> bool:
|
||||
"""
|
||||
Remove plugin state (e.g., after uninstall).
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
|
||||
Returns:
|
||||
True if removal successful
|
||||
"""
|
||||
self._ensure_loaded()
|
||||
with self._lock:
|
||||
if plugin_id in self._states:
|
||||
del self._states[plugin_id]
|
||||
|
||||
# Auto-save if enabled
|
||||
if self.auto_save:
|
||||
self._save_state()
|
||||
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def _save_state(self) -> None:
|
||||
"""Save state to file."""
|
||||
if not self.state_file:
|
||||
return
|
||||
|
||||
try:
|
||||
# The write stays under the lock and goes through a temp file:
|
||||
# Flask serves requests on threads, and two saves racing on a
|
||||
# plain open('w') could interleave or leave a truncated file
|
||||
# that _load_state then drops wholesale.
|
||||
with self._lock:
|
||||
# Convert states to dicts
|
||||
states_data = {
|
||||
plugin_id: state.to_dict()
|
||||
for plugin_id, state in self._states.items()
|
||||
}
|
||||
|
||||
state_data = {
|
||||
'version': self._state_version,
|
||||
'states': states_data,
|
||||
'last_updated': datetime.now().isoformat()
|
||||
}
|
||||
|
||||
# Ensure directory exists with proper permissions
|
||||
from src.common.permission_utils import (
|
||||
ensure_directory_permissions,
|
||||
get_config_dir_mode
|
||||
)
|
||||
ensure_directory_permissions(self.state_file.parent, get_config_dir_mode())
|
||||
|
||||
# Write to file
|
||||
atomic_write_text(self.state_file, json.dumps(state_data, indent=2))
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error saving plugin state: {e}", exc_info=True)
|
||||
|
||||
def _load_state(self) -> None:
|
||||
"""Load state from file."""
|
||||
if not self.state_file or not self.state_file.exists():
|
||||
return
|
||||
|
||||
try:
|
||||
with open(self.state_file, 'r', encoding='utf-8') as f:
|
||||
state_data = json.load(f)
|
||||
|
||||
with self._lock:
|
||||
# Load state version
|
||||
self._state_version = state_data.get('version', 1)
|
||||
|
||||
# Load states
|
||||
states_data = state_data.get('states', {})
|
||||
for plugin_id, state_dict in states_data.items():
|
||||
try:
|
||||
self._states[plugin_id] = PluginState.from_dict(state_dict)
|
||||
except Exception as e:
|
||||
self.logger.warning(
|
||||
f"Error loading state for plugin {plugin_id}: {e}"
|
||||
)
|
||||
|
||||
self.logger.info(f"Loaded {len(self._states)} plugin states from file")
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error loading plugin state: {e}", exc_info=True)
|
||||
@@ -1,22 +1,34 @@
|
||||
"""
|
||||
State reconciliation system.
|
||||
|
||||
Detects and fixes inconsistencies between:
|
||||
- Config file state
|
||||
- Plugin manager state
|
||||
- Disk state (installed plugins)
|
||||
- State manager state
|
||||
Compares what the user wants with what is there and what runs:
|
||||
|
||||
- desired: config.json (which plugins are configured, and enabled) plus the
|
||||
plugins directory on disk (which are installed, at which version);
|
||||
- observed: the runtime snapshot the display publishes
|
||||
(src/plugin_system/plugin_runtime.py) -- which plugins it has loaded, at
|
||||
which version, and why one failed. Only a live snapshot is compared; a
|
||||
stale, stopped or missing one is unknown and yields no findings.
|
||||
|
||||
Desired-state gaps (on disk but not in config, in config but not on disk)
|
||||
are fixed here. Observed-state gaps (enabled but not loaded, loaded at an
|
||||
older version) are reported, never "fixed": the display reconciles its own
|
||||
loaded set against config, and a version gap needs a display restart.
|
||||
|
||||
There is no third, persisted record any more. ``data/plugin_state.json``
|
||||
held a copy of config's enabled flags and the disk's versions, and this
|
||||
module mostly synced it back to config; it is no longer read or written.
|
||||
"""
|
||||
|
||||
import json
|
||||
from typing import Dict, Any, List, Set, cast
|
||||
from typing import Any, Callable, Dict, List, Optional, Set
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
from pathlib import Path
|
||||
|
||||
from src.core_config_keys import CORE_CONFIG_KEYS
|
||||
from src.core_config_keys import CORE_CONFIG_KEYS, CORE_SECRETS_KEYS
|
||||
from src.plugin_system.plugin_dirs import PluginDirectoryIndex
|
||||
from src.plugin_system.state_manager import PluginStateManager
|
||||
from src.plugin_system.plugin_runtime import PluginRuntimeView, UNKNOWN
|
||||
from src.logging_config import get_logger
|
||||
|
||||
|
||||
@@ -160,36 +172,40 @@ def still_unresolved(entries: List[Dict[str, Any]],
|
||||
return live
|
||||
|
||||
|
||||
RuntimeSource = Callable[[], PluginRuntimeView]
|
||||
|
||||
|
||||
class StateReconciliation:
|
||||
"""
|
||||
State reconciliation system.
|
||||
|
||||
Compares state from multiple sources and detects/fixes inconsistencies.
|
||||
|
||||
Compares desired state (config + disk) with observed state (the
|
||||
display's runtime snapshot) and fixes what can safely be fixed.
|
||||
"""
|
||||
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
state_manager: PluginStateManager,
|
||||
*,
|
||||
config_manager,
|
||||
plugin_manager,
|
||||
plugins_dir: Path,
|
||||
store_manager=None
|
||||
store_manager=None,
|
||||
runtime_source: Optional[RuntimeSource] = None,
|
||||
):
|
||||
"""
|
||||
Initialize reconciliation system.
|
||||
|
||||
Args:
|
||||
state_manager: PluginStateManager instance
|
||||
config_manager: ConfigManager instance
|
||||
plugin_manager: PluginManager instance
|
||||
plugins_dir: Path to plugins directory
|
||||
store_manager: Optional PluginStoreManager for auto-repair
|
||||
runtime_source: Returns the display's runtime snapshot as a
|
||||
PluginRuntimeView (plugin_runtime.read_plugin_runtime bound to
|
||||
a cache manager). None: observed state is unknown.
|
||||
"""
|
||||
self.state_manager = state_manager
|
||||
self.config_manager = config_manager
|
||||
self.plugin_manager = plugin_manager
|
||||
self.plugins_dir = Path(plugins_dir)
|
||||
self.store_manager = store_manager
|
||||
self.runtime_source = runtime_source
|
||||
self.logger = get_logger(__name__)
|
||||
|
||||
# Plugin IDs that failed auto-repair and should NOT be retried this
|
||||
@@ -230,30 +246,28 @@ class StateReconciliation:
|
||||
manual_fix_required = []
|
||||
|
||||
try:
|
||||
# Get state from all sources
|
||||
# Desired: config + disk. Observed: the display's snapshot.
|
||||
config_state = self._get_config_state()
|
||||
disk_state = self._get_disk_state()
|
||||
manager_state = self._get_manager_state()
|
||||
state_manager_state = self._get_state_manager_state()
|
||||
|
||||
# Find all unique plugin IDs
|
||||
observed = self._get_observed_state()
|
||||
|
||||
# Plugins the display reports but neither config nor disk knows
|
||||
# (removed while it still runs them) are not a finding of their
|
||||
# own: the display unloads them when their section goes.
|
||||
all_plugin_ids: Set[str] = set()
|
||||
all_plugin_ids.update(config_state.keys())
|
||||
all_plugin_ids.update(disk_state.keys())
|
||||
all_plugin_ids.update(manager_state.keys())
|
||||
all_plugin_ids.update(state_manager_state.keys())
|
||||
|
||||
|
||||
# Check each plugin for inconsistencies
|
||||
for plugin_id in all_plugin_ids:
|
||||
plugin_inconsistencies = self._check_plugin_consistency(
|
||||
plugin_id,
|
||||
config_state,
|
||||
disk_state,
|
||||
manager_state,
|
||||
state_manager_state
|
||||
observed,
|
||||
)
|
||||
inconsistencies.extend(plugin_inconsistencies)
|
||||
|
||||
|
||||
# Attempt to fix auto-fixable inconsistencies
|
||||
for inconsistency in inconsistencies:
|
||||
if inconsistency.can_auto_fix and inconsistency.fix_action == FixAction.AUTO_FIX:
|
||||
@@ -293,11 +307,11 @@ class StateReconciliation:
|
||||
# Top-level config keys that are NOT plugins. The core keys come from the
|
||||
# shared list in src/core_config_keys.py -- a private copy here missed
|
||||
# #581's 'auto_update' and reported it as a plugin missing from disk.
|
||||
# 'github'/'youtube' are the historical secrets-file keys. The secrets file
|
||||
# CORE_SECRETS_KEYS are the core's own secrets-file keys. The secrets file
|
||||
# itself is read at run time too (ignored_config_keys): load_config() merges
|
||||
# it in, and naming its keys one by one let a 'data' key become a phantom
|
||||
# plugin permanently reported as "in config but not on disk".
|
||||
_SYSTEM_CONFIG_KEYS = CORE_CONFIG_KEYS | frozenset({'github', 'youtube'})
|
||||
_SYSTEM_CONFIG_KEYS = CORE_CONFIG_KEYS | CORE_SECRETS_KEYS
|
||||
|
||||
def _get_config_state(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get plugin state from config file."""
|
||||
@@ -309,8 +323,9 @@ class StateReconciliation:
|
||||
for plugin_id in config_plugin_ids(config, ignored):
|
||||
plugin_config = config[plugin_id]
|
||||
state[plugin_id] = {
|
||||
'enabled': plugin_config.get('enabled', True),
|
||||
'version': plugin_config.get('version'),
|
||||
# The display's rule: it runs a plugin only when its
|
||||
# section says "enabled": true.
|
||||
'enabled': bool(plugin_config.get('enabled', False)),
|
||||
'exists_in_config': True
|
||||
}
|
||||
except Exception as e:
|
||||
@@ -339,45 +354,51 @@ class StateReconciliation:
|
||||
self.logger.warning(f"Error reading disk state: {e}")
|
||||
return state
|
||||
|
||||
def _get_manager_state(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get plugin state from plugin manager."""
|
||||
state = {}
|
||||
def _get_observed_state(self) -> PluginRuntimeView:
|
||||
"""The display's runtime snapshot; unknown when there is no source or
|
||||
it cannot be read. Only a live view is compared."""
|
||||
if self.runtime_source is None:
|
||||
return PluginRuntimeView(status=UNKNOWN)
|
||||
try:
|
||||
if self.plugin_manager:
|
||||
# Get discovered plugins
|
||||
if hasattr(self.plugin_manager, 'plugin_manifests'):
|
||||
for plugin_id in self.plugin_manager.plugin_manifests.keys():
|
||||
state[plugin_id] = {
|
||||
'exists_in_manager': True,
|
||||
'loaded': plugin_id in getattr(self.plugin_manager, 'plugins', {})
|
||||
}
|
||||
return self.runtime_source()
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Error reading manager state: {e}")
|
||||
return state
|
||||
|
||||
def _get_state_manager_state(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get plugin state from state manager."""
|
||||
state = {}
|
||||
try:
|
||||
all_states = self.state_manager.get_all_states()
|
||||
for plugin_id, plugin_state in all_states.items():
|
||||
state[plugin_id] = {
|
||||
'enabled': plugin_state.enabled,
|
||||
'status': plugin_state.status.value,
|
||||
'version': plugin_state.version,
|
||||
'exists_in_state_manager': True
|
||||
}
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Error reading state manager state: {e}")
|
||||
return state
|
||||
|
||||
self.logger.warning(f"Error reading the display's runtime state: {e}")
|
||||
return PluginRuntimeView(status=UNKNOWN)
|
||||
|
||||
def plugin_states(self) -> Dict[str, Dict[str, Any]]:
|
||||
"""Desired and observed state for every plugin config or disk knows.
|
||||
|
||||
Per plugin: ``installed`` and ``version`` (disk), ``in_config`` and
|
||||
``enabled`` (config, by the display's rule), and the display's
|
||||
``loaded`` / ``state`` / ``error_info`` / ``loaded_version`` /
|
||||
``loaded_at`` (None unless its snapshot is live). What
|
||||
/api/v3/plugins/state serves, in place of plugin_state.json.
|
||||
"""
|
||||
config_state = self._get_config_state()
|
||||
disk_state = self._get_disk_state()
|
||||
observed = self._get_observed_state()
|
||||
states: Dict[str, Dict[str, Any]] = {}
|
||||
for plugin_id in sorted(set(config_state) | set(disk_state)):
|
||||
if plugin_id in CORE_CONFIG_KEYS:
|
||||
continue
|
||||
config = config_state.get(plugin_id, {})
|
||||
disk = disk_state.get(plugin_id, {})
|
||||
states[plugin_id] = {
|
||||
'plugin_id': plugin_id,
|
||||
'installed': bool(disk.get('exists_on_disk')),
|
||||
'version': disk.get('version'),
|
||||
'in_config': bool(config.get('exists_in_config')),
|
||||
'enabled': bool(config.get('enabled', False)),
|
||||
**observed.plugin(plugin_id),
|
||||
}
|
||||
return states
|
||||
|
||||
def _check_plugin_consistency(
|
||||
self,
|
||||
plugin_id: str,
|
||||
config_state: Dict[str, Dict[str, Any]],
|
||||
disk_state: Dict[str, Dict[str, Any]],
|
||||
manager_state: Dict[str, Dict[str, Any]],
|
||||
state_manager_state: Dict[str, Dict[str, Any]]
|
||||
observed: PluginRuntimeView,
|
||||
) -> List[Inconsistency]:
|
||||
"""Check consistency for a single plugin."""
|
||||
inconsistencies: List[Inconsistency] = []
|
||||
@@ -397,7 +418,6 @@ class StateReconciliation:
|
||||
|
||||
config = config_state.get(plugin_id, {})
|
||||
disk = disk_state.get(plugin_id, {})
|
||||
state_mgr = state_manager_state.get(plugin_id, {})
|
||||
|
||||
# Check: Plugin exists on disk but not in config
|
||||
if disk.get('exists_on_disk') and not config.get('exists_in_config'):
|
||||
@@ -442,21 +462,47 @@ class StateReconciliation:
|
||||
can_auto_fix=can_repair
|
||||
))
|
||||
|
||||
# Check: Enabled state mismatch
|
||||
config_enabled = config.get('enabled', False)
|
||||
state_mgr_enabled = state_mgr.get('enabled')
|
||||
# Observed checks: only against a live snapshot, and only for a plugin
|
||||
# that is both configured and installed (the checks above cover the
|
||||
# rest). Reported, never fixed here: the display loads and unloads by
|
||||
# config on its own, so a gap is either transient (it is catching up)
|
||||
# or something only the user can act on (a failed load, a restart).
|
||||
if (observed.live and config.get('exists_in_config')
|
||||
and disk.get('exists_on_disk')):
|
||||
runtime = observed.plugin(plugin_id)
|
||||
config_enabled = bool(config.get('enabled', False))
|
||||
loaded = bool(runtime.get('loaded'))
|
||||
if config_enabled != loaded:
|
||||
error = runtime.get('error_info') or {}
|
||||
why = f" ({error.get('message')})" if error.get('message') else ""
|
||||
inconsistencies.append(Inconsistency(
|
||||
plugin_id=plugin_id,
|
||||
inconsistency_type=InconsistencyType.PLUGIN_ENABLED_MISMATCH,
|
||||
description=(
|
||||
f"Plugin {plugin_id} is {'enabled' if config_enabled else 'disabled'} "
|
||||
f"in config but the display has it "
|
||||
f"{'loaded' if loaded else 'not loaded'} "
|
||||
f"(state {runtime.get('state')}){why}"),
|
||||
fix_action=FixAction.NO_ACTION,
|
||||
current_state={'loaded': loaded, 'state': runtime.get('state')},
|
||||
expected_state={'loaded': config_enabled},
|
||||
can_auto_fix=False
|
||||
))
|
||||
loaded_version = runtime.get('loaded_version')
|
||||
disk_version = disk.get('version')
|
||||
if loaded and loaded_version and disk_version and loaded_version != disk_version:
|
||||
inconsistencies.append(Inconsistency(
|
||||
plugin_id=plugin_id,
|
||||
inconsistency_type=InconsistencyType.PLUGIN_VERSION_MISMATCH,
|
||||
description=(
|
||||
f"Plugin {plugin_id} {disk_version} is installed but the display "
|
||||
f"is running {loaded_version}; restart the display to run it"),
|
||||
fix_action=FixAction.NO_ACTION,
|
||||
current_state={'version': loaded_version},
|
||||
expected_state={'version': disk_version},
|
||||
can_auto_fix=False
|
||||
))
|
||||
|
||||
if state_mgr_enabled is not None and config_enabled != state_mgr_enabled:
|
||||
inconsistencies.append(Inconsistency(
|
||||
plugin_id=plugin_id,
|
||||
inconsistency_type=InconsistencyType.PLUGIN_ENABLED_MISMATCH,
|
||||
description=f"Plugin {plugin_id} enabled state mismatch: config={config_enabled}, state_manager={state_mgr_enabled}",
|
||||
fix_action=FixAction.AUTO_FIX,
|
||||
current_state={'enabled': state_mgr_enabled},
|
||||
expected_state={'enabled': config_enabled},
|
||||
can_auto_fix=True
|
||||
))
|
||||
|
||||
return inconsistencies
|
||||
|
||||
def _fix_inconsistency(self, inconsistency: Inconsistency) -> bool:
|
||||
@@ -490,26 +536,6 @@ class StateReconciliation:
|
||||
|
||||
elif inconsistency.inconsistency_type == InconsistencyType.PLUGIN_MISSING_ON_DISK:
|
||||
return self._auto_repair_missing_plugin(inconsistency.plugin_id)
|
||||
|
||||
elif inconsistency.inconsistency_type == InconsistencyType.PLUGIN_ENABLED_MISMATCH:
|
||||
# config.json is the user-editable source of truth for enabled state.
|
||||
# Bring the state manager in sync with config rather than the reverse,
|
||||
# so that manual config edits (or the state left behind after an
|
||||
# uninstall+reinstall cycle) don't silently override the user's intent.
|
||||
# Always set for this type (see _check_plugin_consistency).
|
||||
config_enabled = cast(bool, inconsistency.expected_state.get('enabled'))
|
||||
success = self.state_manager.set_plugin_enabled(inconsistency.plugin_id, config_enabled)
|
||||
if success:
|
||||
self.logger.info(
|
||||
f"Fixed: Synced state manager enabled={config_enabled} for "
|
||||
f"{inconsistency.plugin_id} to match config"
|
||||
)
|
||||
else:
|
||||
self.logger.warning(
|
||||
f"Failed to sync state manager enabled={config_enabled} for "
|
||||
f"{inconsistency.plugin_id}"
|
||||
)
|
||||
return success
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Error fixing inconsistency: {e}", exc_info=True)
|
||||
|
||||
@@ -63,7 +63,14 @@ class _InstallMixin:
|
||||
return False
|
||||
|
||||
with self._get_reinstall_lock(plugin_id):
|
||||
plugin_path = self.plugins_dir / plugin_id
|
||||
# The copy to protect is wherever this plugin is installed, not
|
||||
# necessarily plugins_dir/<id>: asked for the registry id
|
||||
# `weather`, the install lives in `ledmatrix-weather/`, the
|
||||
# manifest's id. Backing up only `weather/` protected nothing,
|
||||
# and _install_plugin_impl then deleted `ledmatrix-weather/` to
|
||||
# make room for the download -- so a refusal after that point
|
||||
# (the post-download compatibility gate) left no plugin at all.
|
||||
plugin_path = self._existing_install(plugin_id) or self.plugins_dir / plugin_id
|
||||
if not plugin_path.exists():
|
||||
return self._install_plugin_impl(plugin_id, branch)
|
||||
|
||||
@@ -91,6 +98,26 @@ class _InstallMixin:
|
||||
self._restore_backup(plugin_id, plugin_path, backup_path, "Install")
|
||||
return False
|
||||
|
||||
def _existing_install(self, plugin_id: str) -> Optional[Path]:
|
||||
"""The installed copy of ``plugin_id`` in plugins_dir, by the id or an
|
||||
alias the registry proves (`_installed_id_candidates`).
|
||||
|
||||
When nothing matches but a ``ledmatrix-<id>`` folder exists and no
|
||||
registry is loaded yet, the registry is fetched first -- the install
|
||||
fetches it anyway -- because without it that folder can be neither
|
||||
protected nor trusted: the download may be renamed onto it.
|
||||
"""
|
||||
dirs = [self.plugins_dir]
|
||||
found = self._resolve_installed(plugin_id, dirs)
|
||||
if (found is None and not getattr(self, 'registry_cache', None)
|
||||
and self._unproven_prefix_folder(plugin_id, dirs) is not None):
|
||||
try:
|
||||
self.fetch_registry()
|
||||
except Exception as e: # noqa: BLE001 - proceed as before without proof
|
||||
self.logger.debug("Registry fetch before installing %s failed: %s", plugin_id, e)
|
||||
found = self._resolve_installed(plugin_id, dirs)
|
||||
return found
|
||||
|
||||
def _set_aside(self, plugin_path: Path, backup_path: Path) -> Optional[str]:
|
||||
"""Rename an installed plugin to ``backup_path`` so a failed
|
||||
(re)install can put it back.
|
||||
@@ -164,6 +191,16 @@ class _InstallMixin:
|
||||
self.logger.error(f"Plugin {plugin_id} missing repository URL")
|
||||
return False
|
||||
|
||||
# The registry's floor describes the release on the entry's branch.
|
||||
# Checked here, before anything is removed or downloaded; the gate on
|
||||
# the downloaded manifest below stays as the fallback (older
|
||||
# registries, compatible_versions ranges). A different branch asked
|
||||
# for by name is a different release, so only the fallback applies.
|
||||
registry_branch = plugin_info.get('branch') or plugin_info.get('default_branch')
|
||||
if (not branch or not registry_branch or branch == registry_branch) and \
|
||||
self._refuse_if_registry_incompatible(plugin_id, plugin_info, "install"):
|
||||
return False
|
||||
|
||||
plugin_subpath = plugin_info.get('plugin_path')
|
||||
# If branch is provided, prioritize it; otherwise use default logic
|
||||
branch_candidates = self._distinct_sequence([
|
||||
@@ -280,9 +317,11 @@ class _InstallMixin:
|
||||
return False
|
||||
|
||||
# Refuse a plugin that needs a newer core than this one. The
|
||||
# registry carries no compatibility field, so the floor is only
|
||||
# knowable once the files are down — checking here, before
|
||||
# dependency installation, is the earliest possible point.
|
||||
# registry's `ledmatrix_min_version` already refused the
|
||||
# common case before the download (above); this is the
|
||||
# fallback for a registry without it, a branch other than the
|
||||
# registry's, and `compatible_versions`, which only the
|
||||
# manifest carries. Before dependency installation, still.
|
||||
#
|
||||
# Refusing costs the user nothing: on an update this returns
|
||||
# False and _reinstall_with_rollback restores the version they
|
||||
@@ -299,6 +338,7 @@ class _InstallMixin:
|
||||
if not compatible:
|
||||
self.logger.error(
|
||||
"Refusing to install %s: %s", plugin_id, reason)
|
||||
self._note_refusal(requested_id, reason)
|
||||
self._safe_remove_directory(plugin_path)
|
||||
return False
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ from src.plugin_system.plugin_dirs import (
|
||||
PluginDirectoryIndex, resolve_plugin_dir, store_search_dirs,
|
||||
)
|
||||
from src.plugin_system.store_install import _InstallMixin
|
||||
from src.plugin_system.store_registry import _RegistryMixin
|
||||
from src.plugin_system.store_registry import _RegistryMixin, prefix_hint
|
||||
from src.plugin_system.store_update import _UpdateMixin
|
||||
|
||||
|
||||
@@ -407,13 +407,20 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
|
||||
alone reported such a plugin as not installed, so update_plugin()
|
||||
silently did nothing.
|
||||
|
||||
No ``ledmatrix-`` prefix and no case folding here, unlike the loader:
|
||||
a store operation may delete what this returns, so it only accepts a
|
||||
directory that names the id exactly or declares it. So a registry id
|
||||
such as `stocks` does not resolve to an installed `ledmatrix-stocks/`
|
||||
declaring `ledmatrix-stocks` (the monorepo's leaderboard, music,
|
||||
stocks and weather); callers pass the installed id, and
|
||||
update_plugin() maps it back to the registry id itself.
|
||||
When nothing answers to the id itself, the ids the registry proves
|
||||
are the same plugin are tried the same way
|
||||
(`_installed_id_candidates`): the entry's own id, its ``aliases`` and
|
||||
its ``plugin_path`` name. So the registry id `stocks` finds an
|
||||
installed `ledmatrix-stocks/` declaring `ledmatrix-stocks` (the
|
||||
monorepo's leaderboard, music, stocks and weather), and uninstalling
|
||||
by the registry id no longer reports success while leaving the
|
||||
plugin on disk.
|
||||
|
||||
Never ``ledmatrix-<id>`` without that proof -- no registry loaded, or
|
||||
an entry that doesn't name it: a store operation may delete or
|
||||
replace what this returns, and an unrelated plugin can own that
|
||||
folder. Such a folder is only logged, so a person can act on it.
|
||||
Still no case folding.
|
||||
|
||||
Args:
|
||||
plugin_id: Plugin identifier
|
||||
@@ -421,9 +428,55 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
|
||||
Returns:
|
||||
Path to plugin directory if found, None otherwise
|
||||
"""
|
||||
return resolve_plugin_dir(
|
||||
plugin_id, self._candidate_plugin_dirs(), prefix=False,
|
||||
case_insensitive=False)
|
||||
return self._find_with_proof(plugin_id, fetch=False)
|
||||
|
||||
def _find_with_proof(self, plugin_id: str, fetch: bool) -> Optional[Path]:
|
||||
"""`_find_plugin_path`; with ``fetch``, a ``ledmatrix-<id>`` folder
|
||||
found while no registry is loaded makes it fetch the registry and
|
||||
look again, since only the registry can prove the folder is this
|
||||
plugin. Uninstall passes False (it must work offline); update, which
|
||||
needs the network anyway, passes True."""
|
||||
search_dirs = self._candidate_plugin_dirs()
|
||||
found = self._resolve_installed(plugin_id, search_dirs)
|
||||
if found is not None:
|
||||
return found
|
||||
folder = self._unproven_prefix_folder(plugin_id, search_dirs)
|
||||
if folder is not None and fetch and not getattr(self, 'registry_cache', None):
|
||||
try:
|
||||
self.fetch_registry()
|
||||
except Exception as e: # noqa: BLE001 - fall through to "not found"
|
||||
self.logger.debug("Registry fetch while looking for %s failed: %s", plugin_id, e)
|
||||
found = self._resolve_installed(plugin_id, search_dirs)
|
||||
if found is not None:
|
||||
return found
|
||||
if folder is not None:
|
||||
self.logger.warning(
|
||||
"Plugin %s not found. %s may be it, but nothing in the plugin "
|
||||
"registry says so (no alias), so the store leaves it alone; "
|
||||
"if it is this plugin, manage it as %s.",
|
||||
plugin_id, folder, prefix_hint(plugin_id))
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _unproven_prefix_folder(plugin_id: str, search_dirs: List[Path]) -> Optional[Path]:
|
||||
"""A ``ledmatrix-<id>`` folder, which the store names but won't touch."""
|
||||
hint = prefix_hint(plugin_id)
|
||||
if hint is None:
|
||||
return None
|
||||
return resolve_plugin_dir(hint, search_dirs, prefix=False, by_manifest=False)
|
||||
|
||||
def _resolve_installed(self, plugin_id: str, search_dirs: List[Path]) -> Optional[Path]:
|
||||
"""The first of ``plugin_id``'s candidate ids found in ``search_dirs``.
|
||||
|
||||
The id itself is looked for in every directory before any alias is,
|
||||
so an exact install anywhere beats an alias in the configured one.
|
||||
"""
|
||||
for candidate in self._installed_id_candidates(plugin_id):
|
||||
found = resolve_plugin_dir(
|
||||
candidate, search_dirs, prefix=False, case_insensitive=False)
|
||||
if found is not None:
|
||||
return found
|
||||
return None
|
||||
|
||||
def _candidate_plugin_dirs(self) -> List[Path]:
|
||||
"""Directories that may hold installed plugins, configured one first."""
|
||||
|
||||
@@ -13,11 +13,62 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List, Dict, Optional, Any
|
||||
from jsonschema import Draft7Validator, ValidationError
|
||||
from src.plugin_system.plugin_dirs import PLUGIN_DIR_PREFIX
|
||||
from src.plugin_system.repo_urls import (
|
||||
github_api_headers, github_owner_repo, normalize_repo_url,
|
||||
)
|
||||
|
||||
|
||||
# Registry entry fields the plugin monorepo's update_registry.py added after
|
||||
# 3.7.0. All optional: an older plugins.json has none of them, and every
|
||||
# reader here treats a missing or malformed one as "not stated".
|
||||
#
|
||||
# - ``ledmatrix_min_version``: the floor the plugin's manifest declares, so an
|
||||
# incompatible install or update is refused before the download
|
||||
# (`registry_incompatibility`). The post-download gate stays as the fallback.
|
||||
# - ``aliases``: other ids the plugin goes by (the manifest id when it differs
|
||||
# from the registry id, e.g. ``ledmatrix-weather`` for ``weather``). With
|
||||
# ``plugin_path``'s name, the only proof the store accepts that a folder
|
||||
# under another name is this plugin (`alternate_ids`).
|
||||
# - ``commit``: the monorepo commit that introduced ``latest_version``.
|
||||
# Informational only -- installs still come from the branch head.
|
||||
|
||||
|
||||
def declared_aliases(entry: Dict[str, Any]) -> Optional[List[str]]:
|
||||
"""The entry's ``aliases``, or None when it carries no such list."""
|
||||
aliases = entry.get('aliases')
|
||||
if not isinstance(aliases, list):
|
||||
return None
|
||||
own = entry.get('id')
|
||||
return [a for a in aliases if isinstance(a, str) and a and a != own]
|
||||
|
||||
|
||||
def alternate_ids(entry: Dict[str, Any]) -> List[str]:
|
||||
"""Ids other than the registry id that the registry *proves* an installed
|
||||
copy may carry: the entry's ``aliases``, then its ``plugin_path``
|
||||
directory name (all an older registry has).
|
||||
|
||||
Never ``ledmatrix-<id>`` on its own say-so. Store operations delete and
|
||||
replace what these ids resolve to, and an unrelated plugin can live in a
|
||||
folder of that name (owner decision on #686). A guess is only a hint:
|
||||
see `prefix_hint`.
|
||||
"""
|
||||
own = entry.get('id')
|
||||
ids: List[str] = list(declared_aliases(entry) or [])
|
||||
path = entry.get('plugin_path')
|
||||
if isinstance(path, str) and path.strip('/'):
|
||||
ids.append(path.rstrip('/').rsplit('/', 1)[-1])
|
||||
return [g for i, g in enumerate(ids) if g and g != own and g not in ids[:i]]
|
||||
|
||||
|
||||
def prefix_hint(plugin_id: Any) -> Optional[str]:
|
||||
"""``ledmatrix-<id>``: the legacy folder name worth *mentioning* when
|
||||
``plugin_id`` is not found -- never one to act on without registry proof."""
|
||||
if isinstance(plugin_id, str) and plugin_id and not plugin_id.startswith(PLUGIN_DIR_PREFIX):
|
||||
return PLUGIN_DIR_PREFIX + plugin_id
|
||||
return None
|
||||
|
||||
|
||||
class _RegistryMixin:
|
||||
"""PluginStoreManager methods: see the module docstring."""
|
||||
|
||||
@@ -821,19 +872,117 @@ class _RegistryMixin:
|
||||
Matching ``plugin_path`` fixes it without renaming any published id,
|
||||
which would orphan ``plugin_state.json`` entries keyed on the old ones.
|
||||
Exact id always wins, so an entry whose *path* happens to collide with
|
||||
another entry's id cannot shadow it.
|
||||
another entry's id cannot shadow it. An entry's ``aliases`` (registries
|
||||
from after 3.7.0) come next, then ``plugin_path``, which is what an
|
||||
older registry has to go on.
|
||||
"""
|
||||
if not plugin_id:
|
||||
return None
|
||||
exact = next((p for p in plugins if p.get('id') == plugin_id), None)
|
||||
if exact is not None:
|
||||
return exact
|
||||
for entry in plugins:
|
||||
if plugin_id in (declared_aliases(entry) or ()):
|
||||
return entry
|
||||
for entry in plugins:
|
||||
path = (entry.get('plugin_path') or '').rstrip('/')
|
||||
if path and path.rsplit('/', 1)[-1] == plugin_id:
|
||||
return entry
|
||||
return None
|
||||
|
||||
def registry_incompatibility(self, plugin_id: str,
|
||||
entry: Optional[Dict[str, Any]] = None) -> Optional[str]:
|
||||
"""Why the registry says this core cannot run the plugin's latest
|
||||
release, or None when it says nothing against it.
|
||||
|
||||
Reads the entry's ``ledmatrix_min_version`` and asks
|
||||
``compatibility.check`` -- the same function, and so the same wording
|
||||
and the same leniency (an untrustworthy or unparseable core version
|
||||
allows), as the gate that runs on the downloaded manifest. That gate
|
||||
stays: it also sees ``compatible_versions``, and an older registry
|
||||
without the field says nothing here.
|
||||
|
||||
``entry`` defaults to the registry entry for ``plugin_id``. Any
|
||||
failure to read the registry answers None: the pre-check exists to
|
||||
refuse early on evidence, never to block on a guess.
|
||||
"""
|
||||
if entry is None:
|
||||
try:
|
||||
entry = self.get_registry_info(plugin_id)
|
||||
except Exception as e: # noqa: BLE001 - never block an install on this
|
||||
self.logger.debug("Registry lookup for %s failed: %s", plugin_id, e)
|
||||
return None
|
||||
if not isinstance(entry, dict):
|
||||
return None
|
||||
floor = entry.get('ledmatrix_min_version')
|
||||
if not isinstance(floor, str) or not floor.strip():
|
||||
return None
|
||||
from src.plugin_system import compatibility
|
||||
compatible, reason = compatibility.check(
|
||||
{'id': entry.get('id') or plugin_id, 'name': entry.get('name'),
|
||||
'min_ledmatrix_version': floor.strip()},
|
||||
compatibility.current_core_version())
|
||||
return None if compatible else reason
|
||||
|
||||
def _refuse_if_registry_incompatible(self, plugin_id: str, entry: Optional[Dict[str, Any]],
|
||||
action: str, record_as: Optional[str] = None) -> bool:
|
||||
"""Log and record a registry-based refusal; True when refused.
|
||||
|
||||
``record_as`` is the id the caller will ask `pop_refusal` about (the
|
||||
id it was handed, which may be an alias of ``plugin_id``).
|
||||
"""
|
||||
reason = self.registry_incompatibility(plugin_id, entry)
|
||||
if reason is None:
|
||||
return False
|
||||
self.logger.error("Refusing to %s %s before downloading it: %s",
|
||||
action, plugin_id, reason)
|
||||
self._note_refusal(record_as or plugin_id, reason)
|
||||
return True
|
||||
|
||||
def _note_refusal(self, plugin_id: str, reason: str) -> None:
|
||||
"""Remember why an install or update of ``plugin_id`` was refused, so
|
||||
the web route can say so instead of "check logs for details"."""
|
||||
refusals = self.__dict__.setdefault('_refusals', {})
|
||||
refusals[plugin_id] = reason
|
||||
|
||||
def pop_refusal(self, *plugin_ids: str) -> Optional[str]:
|
||||
"""The compatibility refusal recorded for any of ``plugin_ids`` since
|
||||
the last call, clearing them all; None when there was none."""
|
||||
refusals = self.__dict__.get('_refusals') or {}
|
||||
found = None
|
||||
for plugin_id in plugin_ids:
|
||||
reason = refusals.pop(plugin_id, None)
|
||||
if found is None and reason:
|
||||
found = reason
|
||||
return found
|
||||
|
||||
def _installed_id_candidates(self, plugin_id: str) -> List[str]:
|
||||
"""``plugin_id`` and the other ids the registry proves its installed
|
||||
copy may carry.
|
||||
|
||||
From the registry already in memory -- no fetch, because uninstall
|
||||
and the update lookup must work offline. With an entry: its id and
|
||||
`alternate_ids` (``aliases``, ``plugin_path`` name). Without one (no
|
||||
registry loaded yet, or a plugin that isn't in it): the id alone.
|
||||
A folder whose manifest declares one of these ids is found by the
|
||||
resolver's manifest pass whatever it is called.
|
||||
"""
|
||||
ids: List[str] = [plugin_id]
|
||||
cache = getattr(self, 'registry_cache', None)
|
||||
plugins = cache.get('plugins') if isinstance(cache, dict) else None
|
||||
entry = None
|
||||
if isinstance(plugins, list) and isinstance(plugin_id, str):
|
||||
entry = self._match_registry_entry(
|
||||
[p for p in plugins if isinstance(p, dict)], plugin_id)
|
||||
if entry is not None:
|
||||
ids.append(entry.get('id'))
|
||||
ids.extend(alternate_ids(entry))
|
||||
unique: List[str] = []
|
||||
for candidate in ids:
|
||||
if isinstance(candidate, str) and candidate and candidate not in unique:
|
||||
unique.append(candidate)
|
||||
return unique
|
||||
|
||||
def get_registry_info(self, plugin_id: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get plugin information from the registry cache only (no GitHub API calls).
|
||||
|
||||
@@ -195,10 +195,12 @@ class _UpdateMixin:
|
||||
surfaces as one line in the journal and a scoreboard that silently
|
||||
stopped appearing.
|
||||
|
||||
Checked after the pull rather than before it, for the same reason
|
||||
``_install_plugin_impl`` checks after the download: the registry
|
||||
carries no compatibility field, so the incoming floor is only knowable
|
||||
once the new commit is on disk.
|
||||
The registry's ``ledmatrix_min_version`` refuses most of these before
|
||||
the pull (``update_plugin``). This is the fallback, for the same cases
|
||||
``_install_plugin_impl``'s post-download gate covers: a registry
|
||||
without the field, a checkout on another branch than the registry's,
|
||||
and ``compatible_versions`` -- all only knowable once the new commit
|
||||
is on disk.
|
||||
|
||||
Undone with ``git reset --hard`` rather than by removing the directory.
|
||||
This is a live checkout, the previous commit is still in the object
|
||||
@@ -233,6 +235,7 @@ class _UpdateMixin:
|
||||
return True
|
||||
|
||||
self.logger.error("Refusing the update to %s: %s", plugin_id, reason)
|
||||
self._note_refusal(plugin_id, reason)
|
||||
|
||||
if not previous_sha:
|
||||
self.logger.error(
|
||||
@@ -310,8 +313,10 @@ class _UpdateMixin:
|
||||
"""
|
||||
Update a plugin to the latest commit on its upstream branch.
|
||||
"""
|
||||
plugin_path = self._find_plugin_path(plugin_id)
|
||||
|
||||
# fetch=True: an update needs the registry anyway, and only it can
|
||||
# prove a ledmatrix-<id>/ folder is this plugin.
|
||||
plugin_path = self._find_with_proof(plugin_id, fetch=True)
|
||||
|
||||
if plugin_path is None or not plugin_path.exists():
|
||||
self.logger.error(f"Plugin not installed: {plugin_id}")
|
||||
return False
|
||||
@@ -368,6 +373,11 @@ class _UpdateMixin:
|
||||
f"Plugin {resolved_id} git remote ({local_remote}) differs from registry ({registry_repo}). "
|
||||
f"Reinstalling from registry to migrate to new source."
|
||||
)
|
||||
# Before the old copy is moved aside: the reinstall
|
||||
# would only refuse after a download and a restore.
|
||||
if self._refuse_if_registry_incompatible(
|
||||
resolved_id, plugin_info_remote, "update", record_as=plugin_id):
|
||||
return False
|
||||
return self._reinstall_with_rollback(resolved_id, plugin_path)
|
||||
|
||||
# Check if already up to date
|
||||
@@ -375,6 +385,14 @@ class _UpdateMixin:
|
||||
self.logger.info(f"Plugin {plugin_id} already matches remote commit {remote_sha[:7]}")
|
||||
return True
|
||||
|
||||
# The registry's floor describes its branch; a checkout
|
||||
# on another branch pulls another release, and the gate
|
||||
# after the pull (_gate_pulled_commit) still covers it.
|
||||
if (not remote_branch or remote_branch == local_branch) and \
|
||||
self._refuse_if_registry_incompatible(
|
||||
resolved_id, plugin_info_remote, "update", record_as=plugin_id):
|
||||
return False
|
||||
|
||||
# Update via git pull
|
||||
self.logger.info(f"Updating {plugin_id} via git pull (local branch: {local_branch})...")
|
||||
try:
|
||||
@@ -718,6 +736,12 @@ class _UpdateMixin:
|
||||
except Exception as e:
|
||||
self.logger.debug(f"Could not compare versions for {plugin_id}: {e}")
|
||||
|
||||
# A newer version this core cannot run: refuse now, while the
|
||||
# installed copy is untouched, rather than after a download.
|
||||
if self._refuse_if_registry_incompatible(
|
||||
registry_id, plugin_info_remote, "update", record_as=plugin_id):
|
||||
return False
|
||||
|
||||
# Plugin is not a git repo but is in registry and has a newer version - reinstall
|
||||
self.logger.info(f"Plugin {plugin_id} not installed via git; re-installing latest archive (registry id: {registry_id})")
|
||||
|
||||
|
||||
@@ -5,10 +5,12 @@ Main orchestrator for Vegas-style continuous scroll mode. Coordinates between
|
||||
StreamManager, RenderPipeline, and the display system to provide smooth
|
||||
continuous scrolling of all enabled plugin content.
|
||||
|
||||
Supports three display modes per plugin:
|
||||
- SCROLL: Content scrolls continuously within the stream
|
||||
- FIXED_SEGMENT: Fixed block that scrolls by with other content
|
||||
- STATIC: Scroll pauses, plugin displays for its duration, then resumes
|
||||
Each plugin takes part in one of three ways (its Vegas participation, see
|
||||
BasePlugin.get_vegas_participation):
|
||||
- 'scroll': its content scrolls by within the stream
|
||||
- 'pause': the scroll pauses, the plugin displays for its duration, then
|
||||
the scroll resumes
|
||||
- 'exclude': left out
|
||||
"""
|
||||
|
||||
import logging
|
||||
@@ -18,6 +20,7 @@ import time
|
||||
import threading
|
||||
from typing import Optional, Dict, Any, List, Callable, TYPE_CHECKING
|
||||
|
||||
from src import display_watchdog
|
||||
from src.common import render_gate
|
||||
from src.vegas_mode.config import VegasModeConfig
|
||||
from src.vegas_mode.plugin_adapter import PluginAdapter
|
||||
@@ -540,6 +543,9 @@ class VegasModeCoordinator:
|
||||
# the whole budget -- the render loop stalls for the size of the
|
||||
# correction. A forward jump inflates p99 and worst-frame instead.
|
||||
frame_started = time.monotonic()
|
||||
# An iteration runs for minutes (max_cycle_duration) without
|
||||
# returning to the display controller's loop.
|
||||
display_watchdog.beat()
|
||||
|
||||
# Check for STATIC mode plugin that should pause scroll
|
||||
static_plugin = self._check_static_plugin_trigger()
|
||||
@@ -921,6 +927,7 @@ class VegasModeCoordinator:
|
||||
|
||||
# Sleep in small increments to remain responsive
|
||||
time.sleep(0.1)
|
||||
display_watchdog.beat()
|
||||
|
||||
logger.info(
|
||||
"Static pause completed for %s after %.1fs",
|
||||
|
||||
@@ -1369,29 +1369,6 @@ class PluginAdapter:
|
||||
logger.debug("[%s] Cleared plugin scroll cache", plugin_id)
|
||||
return cleared
|
||||
|
||||
def get_content_type(self, plugin: 'BasePlugin', plugin_id: str) -> str:
|
||||
"""
|
||||
Get the type of content a plugin provides.
|
||||
|
||||
Args:
|
||||
plugin: Plugin instance
|
||||
plugin_id: Plugin identifier
|
||||
|
||||
Returns:
|
||||
'multi' for multiple items, 'static' for single frame, 'none' for excluded
|
||||
"""
|
||||
if hasattr(plugin, 'get_vegas_content_type'):
|
||||
try:
|
||||
return plugin.get_vegas_content_type()
|
||||
except (AttributeError, TypeError, ValueError):
|
||||
logger.exception(
|
||||
"Error calling get_vegas_content_type() on %s",
|
||||
plugin_id
|
||||
)
|
||||
|
||||
# Default to static for plugins without explicit type
|
||||
return 'static'
|
||||
|
||||
def cleanup(self) -> None:
|
||||
"""Clean up resources."""
|
||||
with self._cache_lock:
|
||||
|
||||
@@ -5,23 +5,26 @@ Manages plugin content streaming with look-ahead buffering. Maintains a queue
|
||||
of plugin content that's ready to be rendered, prefetching 1-2 plugins ahead
|
||||
of the current scroll position.
|
||||
|
||||
Supports three display modes:
|
||||
- SCROLL: Continuous scrolling content
|
||||
- FIXED_SEGMENT: Fixed block that scrolls by
|
||||
- STATIC: Pause scroll to display (marked for coordinator handling)
|
||||
Each plugin takes part in one of three ways (its Vegas participation, see
|
||||
BasePlugin.get_vegas_participation):
|
||||
- 'scroll': its content joins the strip
|
||||
- 'pause': the scroll pauses for its turn (a STATIC segment, marked for the
|
||||
coordinator)
|
||||
- 'exclude': left out of the rotation
|
||||
"""
|
||||
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING, cast
|
||||
from typing import Optional, List, Dict, Any, Deque, Tuple, TYPE_CHECKING
|
||||
from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
from PIL import Image
|
||||
|
||||
from src import display_watchdog
|
||||
from src.vegas_mode.config import VegasModeConfig
|
||||
from src.vegas_mode.plugin_adapter import PluginAdapter
|
||||
from src.plugin_system.base_plugin import VegasDisplayMode
|
||||
from src.plugin_system.base_plugin import VegasDisplayMode, resolve_vegas_participation
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from src.plugin_system.plugin_manager import PluginManager
|
||||
@@ -34,11 +37,12 @@ class ContentSegment:
|
||||
"""One plugin's content for a cycle.
|
||||
|
||||
A STATIC segment carries no images: it marks where the coordinator pauses
|
||||
the scroll to show the plugin full-screen.
|
||||
the scroll to show a plugin whose participation is ``'pause'``. Every
|
||||
other segment is SCROLL.
|
||||
"""
|
||||
plugin_id: str
|
||||
images: List[Image.Image]
|
||||
display_mode: VegasDisplayMode = field(default=VegasDisplayMode.FIXED_SEGMENT)
|
||||
display_mode: VegasDisplayMode = field(default=VegasDisplayMode.SCROLL)
|
||||
|
||||
|
||||
class StreamManager:
|
||||
@@ -335,25 +339,14 @@ class StreamManager:
|
||||
logger.debug("[%s] Vegas: skipped (not enabled)", plugin_id)
|
||||
continue
|
||||
|
||||
# Content type 'none' is left out, except for STATIC plugins,
|
||||
# which pause the scroll rather than contributing to it.
|
||||
content_type = self.plugin_adapter.get_content_type(plugin, plugin_id)
|
||||
display_mode = VegasDisplayMode.FIXED_SEGMENT
|
||||
try:
|
||||
display_mode = plugin.get_vegas_display_mode()
|
||||
except Exception:
|
||||
# Plugin error should not abort refresh; use default mode
|
||||
logger.exception(
|
||||
"[%s] (%s) get_vegas_display_mode() failed, using default",
|
||||
plugin_id, plugin.__class__.__name__
|
||||
)
|
||||
|
||||
included = (content_type != 'none'
|
||||
or display_mode == VegasDisplayMode.STATIC)
|
||||
# 'pause' plugins stay in the rotation: they pause the scroll
|
||||
# for their turn rather than contributing to it.
|
||||
participation = resolve_vegas_participation(plugin, plugin_id)
|
||||
included = participation != 'exclude'
|
||||
logger.debug(
|
||||
"[%s] Vegas: %s (content_type=%s, display_mode=%s)",
|
||||
"[%s] Vegas: %s (participation=%s)",
|
||||
plugin_id, "included" if included else "excluded",
|
||||
content_type, display_mode.value
|
||||
participation
|
||||
)
|
||||
if included:
|
||||
available_plugins.append(plugin_id)
|
||||
@@ -570,6 +563,9 @@ class StreamManager:
|
||||
Returns:
|
||||
ContentSegment or None if fetch failed
|
||||
"""
|
||||
# Composing a cycle fetches plugin after plugin on the render thread
|
||||
# (on the prefetch thread this is ignored), so check in between.
|
||||
display_watchdog.beat()
|
||||
try:
|
||||
if not hasattr(self.plugin_manager, 'plugins'):
|
||||
logger.warning("[%s] plugin_manager has no plugins attribute", plugin_id)
|
||||
@@ -580,23 +576,13 @@ class StreamManager:
|
||||
logger.warning("[%s] Plugin not found in plugin_manager.plugins", plugin_id)
|
||||
return None
|
||||
|
||||
display_mode = VegasDisplayMode.FIXED_SEGMENT
|
||||
try:
|
||||
display_mode = plugin.get_vegas_display_mode()
|
||||
except (AttributeError, TypeError) as e:
|
||||
logger.debug(
|
||||
"[%s] get_vegas_display_mode() not available: %s (using FIXED_SEGMENT)",
|
||||
plugin_id, e
|
||||
)
|
||||
|
||||
# For STATIC mode, we create a placeholder segment
|
||||
# The actual content will be displayed by coordinator during pause
|
||||
if display_mode == VegasDisplayMode.STATIC:
|
||||
# Create minimal placeholder - coordinator handles actual display
|
||||
# A 'pause' plugin gets a placeholder segment; the coordinator
|
||||
# draws it with display() when the scroll reaches its turn.
|
||||
if resolve_vegas_participation(plugin, plugin_id) == 'pause':
|
||||
segment = ContentSegment(
|
||||
plugin_id=plugin_id,
|
||||
images=[], # No images needed for static pause
|
||||
display_mode=display_mode
|
||||
display_mode=VegasDisplayMode.STATIC
|
||||
)
|
||||
self.stats['segments_fetched'] += 1
|
||||
logger.debug(
|
||||
@@ -605,7 +591,6 @@ class StreamManager:
|
||||
)
|
||||
return segment
|
||||
|
||||
# Get content via adapter for SCROLL/FIXED_SEGMENT modes
|
||||
images = self.plugin_adapter.get_content(plugin, plugin_id)
|
||||
if not images:
|
||||
# The adapter already warns when every content path failed;
|
||||
@@ -619,13 +604,13 @@ class StreamManager:
|
||||
segment = ContentSegment(
|
||||
plugin_id=plugin_id,
|
||||
images=images,
|
||||
display_mode=display_mode
|
||||
display_mode=VegasDisplayMode.SCROLL
|
||||
)
|
||||
|
||||
self.stats['segments_fetched'] += 1
|
||||
logger.debug(
|
||||
"[%s] Segment: %d image(s), %dpx, mode=%s",
|
||||
plugin_id, len(images), total_width, display_mode.value
|
||||
"[%s] Segment: %d image(s), %dpx",
|
||||
plugin_id, len(images), total_width
|
||||
)
|
||||
return segment
|
||||
|
||||
@@ -693,16 +678,16 @@ class StreamManager:
|
||||
return layout
|
||||
|
||||
def is_static_plugin(self, plugin_id: str) -> bool:
|
||||
"""Whether a loaded plugin asks Vegas to pause for it (STATIC mode)."""
|
||||
"""Whether a loaded plugin asks Vegas to pause for it (participation 'pause').
|
||||
|
||||
Only 'pause' is acted on here. A plugin whose participation has turned
|
||||
to 'exclude' since the rotation was built is still fetched this cycle,
|
||||
as it always was; the next refresh drops it.
|
||||
"""
|
||||
plugin = getattr(self.plugin_manager, 'plugins', {}).get(plugin_id)
|
||||
if plugin is None:
|
||||
return False
|
||||
try:
|
||||
return cast(bool, plugin.get_vegas_display_mode() == VegasDisplayMode.STATIC)
|
||||
except Exception:
|
||||
logger.debug("[%s] get_vegas_display_mode() failed; treating as not STATIC",
|
||||
plugin_id, exc_info=True)
|
||||
return False
|
||||
return resolve_vegas_participation(plugin, plugin_id) == 'pause'
|
||||
|
||||
def take_next_group(
|
||||
self, count: Optional[int] = None, offscreen_only: bool = False
|
||||
|
||||
@@ -15,7 +15,8 @@ from src.web_interface.errors import ErrorCode, WebInterfaceError
|
||||
def success_response(
|
||||
data: Any = None,
|
||||
message: Optional[str] = None,
|
||||
metadata: Optional[Dict] = None
|
||||
metadata: Optional[Dict] = None,
|
||||
extra: Optional[Dict[str, Any]] = None
|
||||
):
|
||||
"""
|
||||
Create a standardized success response.
|
||||
@@ -24,11 +25,15 @@ def success_response(
|
||||
data: Response data
|
||||
message: Optional success message
|
||||
metadata: Optional metadata (timing, version, etc.)
|
||||
extra: Optional top-level fields beside ``status``/``data``, such as
|
||||
``restart_required``; they cannot replace the standard keys
|
||||
|
||||
Returns:
|
||||
Flask jsonify response
|
||||
"""
|
||||
response_data = create_success_response(data, message, metadata)
|
||||
for key, value in (extra or {}).items():
|
||||
response_data.setdefault(key, value)
|
||||
|
||||
# Timing is merged into whatever the caller passed, without inventing a
|
||||
# metadata block for responses that have neither.
|
||||
|
||||
@@ -8,6 +8,10 @@ This directory contains systemd service unit files for LEDMatrix services.
|
||||
- Runs the display controller (`run.py`)
|
||||
- Starts automatically on boot
|
||||
- Runs as root for hardware access
|
||||
- Restarted by systemd's watchdog (`WatchdogSec=120`) when its render loop
|
||||
stops checking in, e.g. stuck inside a plugin; the loop writes a heartbeat
|
||||
to `/run/ledmatrix/display-heartbeat.json` (`RuntimeDirectory=`) that the
|
||||
web interface's health check reads. See `src/display_watchdog.py`
|
||||
|
||||
- **`ledmatrix-web.service`** - Web interface service
|
||||
- Runs the web interface conditionally based on config
|
||||
|
||||
@@ -27,6 +27,49 @@ ExecStart=/usr/bin/python3 __PROJECT_ROOT_DIR__/run.py
|
||||
# that a successful outcome and never bringing it back.
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
# Back off when it keeps failing: 10s after the first failure, growing to two
|
||||
# minutes by the fourth, so a plugin that crashes or hangs the display on
|
||||
# every start retries a couple of dozen times an hour instead of hundreds.
|
||||
# Deliberately not StartLimitBurst=: once that trips the unit stays failed --
|
||||
# the panel dark until someone reboots -- and every start is refused until the
|
||||
# interval passes, including the web UI's Start button and the automatic
|
||||
# update's rollback, neither of which may run "systemctl reset-failed".
|
||||
# systemd before 254 (Debian Bookworm has 252) ignores these two lines with an
|
||||
# "Unknown key name" warning and keeps the flat RestartSec.
|
||||
RestartSteps=4
|
||||
RestartMaxDelaySec=2min
|
||||
# Render-loop watchdog (src/display_watchdog.py). The render thread itself
|
||||
# pings systemd every few seconds, so a render loop stuck inside a plugin --
|
||||
# service still "active", panel frozen -- stops the pings, and systemd kills
|
||||
# the process (SIGABRT: faulthandler writes every thread's stack to the
|
||||
# journal) and restarts it.
|
||||
#
|
||||
# Type=simple, not Type=notify. Type=notify would hold "systemctl start" and
|
||||
# "restart" until READY=1, i.e. until plugins have loaded -- minutes on a slow
|
||||
# board -- and the web interface, the installer and the update health check all
|
||||
# call those with timeouts well short of that. The process still sends READY=1
|
||||
# (harmless here); NotifyAccess=main is what lets systemd hear it at all, and
|
||||
# only from run.py itself, not from pip or anything else it starts.
|
||||
#
|
||||
# 120s is the steady-state limit, four times the longest gap a healthy loop
|
||||
# has: a screen's first display() call runs under PluginExecutor's 30s
|
||||
# timeout. Everything else the loop blocks on checks in between steps (Vegas
|
||||
# frames, each plugin fetched for a Vegas cycle, each dwell second). Start-up
|
||||
# is longer than this and happens before the loop exists, so the process
|
||||
# widens the limit to 15 minutes as it starts and narrows it back to this
|
||||
# value after its first frame; loading a plugin enabled from the web UI, which
|
||||
# can run pip on the render thread, gets the same 15 minutes. Raise it with a
|
||||
# drop-in (systemctl edit ledmatrix) if a plugin legitimately needs longer;
|
||||
# WatchdogSec=0 turns it off.
|
||||
WatchdogSec=120
|
||||
NotifyAccess=main
|
||||
# /run/ledmatrix, for the render loop's heartbeat (display-heartbeat.json),
|
||||
# which /api/v3/health and the update health check read. /run is tmpfs, so a
|
||||
# write every few seconds never touches the SD card. 0755 and root-owned: the
|
||||
# web interface runs as another user and only needs to read it. Removed when
|
||||
# the service stops, so a stopped display leaves no stale heartbeat behind.
|
||||
RuntimeDirectory=ledmatrix
|
||||
RuntimeDirectoryMode=0755
|
||||
# Memory ceiling as a share of physical RAM, so one unit file suits a 512 MB
|
||||
# Pi Zero 2 W and an 8 GB Pi 5 alike. This is a backstop, not a tuning knob: it
|
||||
# turns "the board runs out of memory, stops being able to fork, and takes sshd
|
||||
|
||||
@@ -21,14 +21,32 @@ from flask import Flask
|
||||
# Every manager attribute the blueprint reads. Anything missing here keeps
|
||||
# whatever a previously-run test left on the singleton.
|
||||
API_V3_MANAGER_ATTRS = (
|
||||
'config_manager', 'plugin_manager', 'plugin_store_manager',
|
||||
'plugin_state_manager', 'saved_repositories_manager', 'schema_manager',
|
||||
'config_manager', 'plugin_catalog', 'plugin_store_manager',
|
||||
'saved_repositories_manager', 'schema_manager',
|
||||
'operation_queue', 'operation_history', 'cache_manager',
|
||||
'health_tracker', 'resource_monitor',
|
||||
)
|
||||
|
||||
_SENTINEL = object()
|
||||
|
||||
|
||||
def mock_plugin_catalog():
|
||||
"""A MagicMock shaped like PluginCatalog, and only like it.
|
||||
|
||||
``spec`` makes anything a PluginCatalog lacks raise AttributeError --
|
||||
``get_plugin``, ``load_plugin``, ``plugins`` -- so a route that reached
|
||||
for a plugin instance in the web process fails the test that drives it
|
||||
instead of quietly calling a mock. The instance attributes the spec
|
||||
cannot see are set explicitly.
|
||||
"""
|
||||
from src.plugin_system.plugin_catalog import PluginCatalog
|
||||
catalog = MagicMock(spec=PluginCatalog)
|
||||
for name in ('plugins_dir', 'config_manager', 'schema_manager',
|
||||
'plugin_manifests', 'plugin_directories'):
|
||||
setattr(catalog, name, MagicMock())
|
||||
return catalog
|
||||
|
||||
|
||||
def build_app(blueprint):
|
||||
app = Flask(__name__)
|
||||
app.config['TESTING'] = True
|
||||
@@ -52,7 +70,8 @@ def api_v3_module():
|
||||
for name in API_V3_MANAGER_ATTRS
|
||||
}
|
||||
for name in API_V3_MANAGER_ATTRS:
|
||||
setattr(module.api_v3, name, MagicMock())
|
||||
setattr(module.api_v3, name,
|
||||
mock_plugin_catalog() if name == 'plugin_catalog' else MagicMock())
|
||||
# Default to the direct path; queue tests opt in explicitly.
|
||||
module.api_v3.operation_queue = None
|
||||
|
||||
|
||||
+64
-1
@@ -16,7 +16,54 @@ if str(project_root) not in sys.path:
|
||||
sys.path.insert(0, str(project_root))
|
||||
|
||||
|
||||
class _DisarmStartupReconciliation:
|
||||
"""Import hook: every ``web_interface.app`` this process builds starts disarmed.
|
||||
|
||||
app.py wires itself to the checkout's real config/config.json and
|
||||
plugin-repos/ at import, and its before_request hook launches startup
|
||||
reconciliation on the first request any test sends. Reconciliation
|
||||
reinstalls every configured plugin missing on disk from the live store,
|
||||
so a full run downloaded basketball-scoreboard, calendar,
|
||||
football-scoreboard, leaderboard and ledmatrix-stocks into the real
|
||||
plugin-repos/ (not gitignored), minutes in, from a daemon thread no test
|
||||
waits on. Setting ``_reconciliation_started`` is the app's own run-once
|
||||
latch; doing it as the module finishes executing covers fixtures that
|
||||
import the app lazily and send a request at once, and ``importlib.reload``.
|
||||
StateReconciliation itself stays fully testable.
|
||||
"""
|
||||
|
||||
_MODULE = "web_interface.app"
|
||||
|
||||
def find_spec(self, fullname, path, target=None):
|
||||
if fullname != self._MODULE:
|
||||
return None
|
||||
import importlib.machinery
|
||||
spec = importlib.machinery.PathFinder.find_spec(fullname, path, target)
|
||||
if spec is None or spec.loader is None:
|
||||
return spec
|
||||
exec_module = spec.loader.exec_module
|
||||
|
||||
def exec_disarmed(module):
|
||||
exec_module(module)
|
||||
module._reconciliation_started = True
|
||||
|
||||
spec.loader.exec_module = exec_disarmed
|
||||
return spec
|
||||
|
||||
|
||||
_DISARM_HOOK = _DisarmStartupReconciliation()
|
||||
|
||||
|
||||
def pytest_configure(config):
|
||||
sys.meta_path.insert(0, _DISARM_HOOK)
|
||||
app_module = sys.modules.get(_DisarmStartupReconciliation._MODULE)
|
||||
if app_module is not None:
|
||||
app_module._reconciliation_started = True
|
||||
|
||||
_point_emulator_at_raw_adapter(config)
|
||||
|
||||
|
||||
def _point_emulator_at_raw_adapter(config):
|
||||
"""Point the emulator at a per-process config that binds no socket.
|
||||
|
||||
Six test modules set EMULATOR=true and build a real DisplayManager. The
|
||||
@@ -63,7 +110,9 @@ def pytest_configure(config):
|
||||
|
||||
|
||||
def pytest_unconfigure(config):
|
||||
"""Remove the throwaway emulator config written by pytest_configure."""
|
||||
"""Undo pytest_configure: the import hook and the throwaway emulator config."""
|
||||
if _DISARM_HOOK in sys.meta_path:
|
||||
sys.meta_path.remove(_DISARM_HOOK)
|
||||
tmp_dir = getattr(config, "_ledmatrix_emulator_tmp", None)
|
||||
if tmp_dir is not None:
|
||||
import shutil
|
||||
@@ -252,6 +301,20 @@ def emulator_mode(monkeypatch):
|
||||
return True
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _hermetic_display_watchdog(monkeypatch):
|
||||
"""Keep the render loop's watchdog off the host.
|
||||
|
||||
Every test that runs DisplayController.run() arms the process-wide
|
||||
watchdog, which would ping a real $NOTIFY_SOCKET and write a heartbeat
|
||||
into /run/ledmatrix -- the live display's, when the suite runs as root on
|
||||
a device. Each test gets a fresh instance that does neither.
|
||||
"""
|
||||
from src import display_watchdog
|
||||
monkeypatch.setattr(display_watchdog, 'watchdog',
|
||||
display_watchdog.RenderWatchdog(environ={}, heartbeat_dir=None))
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_logging():
|
||||
"""Reset logging configuration before each test."""
|
||||
|
||||
Vendored
+67
@@ -1,4 +1,54 @@
|
||||
[
|
||||
[
|
||||
"/api/v3/auth/disable",
|
||||
"api_v3.disable_web_login",
|
||||
[
|
||||
"OPTIONS",
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/auth/password",
|
||||
"api_v3.set_web_password",
|
||||
[
|
||||
"OPTIONS",
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/auth/status",
|
||||
"api_v3.get_web_auth_status",
|
||||
[
|
||||
"GET",
|
||||
"HEAD",
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/auth/tokens",
|
||||
"api_v3.create_api_token",
|
||||
[
|
||||
"OPTIONS",
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/auth/tokens",
|
||||
"api_v3.list_api_tokens",
|
||||
[
|
||||
"GET",
|
||||
"HEAD",
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/auth/tokens/<token_id>",
|
||||
"api_v3.revoke_api_token",
|
||||
[
|
||||
"DELETE",
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/backup/<path:filename>",
|
||||
"api_v3.backup_delete",
|
||||
@@ -853,6 +903,23 @@
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/system/update-channel",
|
||||
"api_v3.get_update_channel",
|
||||
[
|
||||
"GET",
|
||||
"HEAD",
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/system/update-channel",
|
||||
"api_v3.set_update_channel",
|
||||
[
|
||||
"OPTIONS",
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/system/version",
|
||||
"api_v3.get_system_version",
|
||||
|
||||
@@ -50,6 +50,7 @@ server has none.
|
||||
| `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
|
||||
| `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
|
||||
| `unit/test_inline_handler_escaping.js` | no | The store, saved-repository and custom-registry inline `onclick` handlers and the live `window.updateImageList` from `plugins_manager.js`: a registry id, URL or uploaded file name carrying `'`, `"` or entities adds no attributes and reaches the handler intact, and the store's View button opens only http(s) links |
|
||||
| `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
|
||||
| `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
|
||||
| `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
|
||||
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
|
||||
|
||||
+2
-1
@@ -21,7 +21,8 @@ const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
|
||||
'unit/test_style_editor_layout_leaf_columns.js',
|
||||
'unit/test_style_editor_layout_leaf_collision.js',
|
||||
'unit/test_update_all.js', 'unit/test_inline_handler_escaping.js',
|
||||
'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js'];
|
||||
'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js',
|
||||
'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js'];
|
||||
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
|
||||
'dom/test_tools_sections.js'];
|
||||
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
// The "restart the display" banner is raised by the server's answer, not by
|
||||
// which URL was called.
|
||||
//
|
||||
// app.js used to show it after any successful POST to /api/v3/config/main and
|
||||
// nothing else. A plugin update, install or uninstall that the running display
|
||||
// cannot pick up live now answers `restart_required: true` (with the banner's
|
||||
// wording in `restart_message`), and so does a main-config save. The htmx
|
||||
// after-request handler and window.noteRestartRequired must both follow the
|
||||
// flag. Runs the shipped app.js in a vm with a minimal fake DOM -- no jsdom and
|
||||
// no server needed, so it runs under test/test_js_unit_suites.py too.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
const V3 = path.resolve(__dirname, '../../../web_interface/static/v3');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
|
||||
|
||||
function load() {
|
||||
const handlers = {};
|
||||
const listen = (target) => (type, fn) => { (handlers[target + ':' + type] ||= []).push(fn); };
|
||||
const banner = { style: { display: 'none' } };
|
||||
const text = { dataset: {}, textContent: ' Configuration saved — restart the display to apply the changes ' };
|
||||
const store = {};
|
||||
const document = {
|
||||
body: { addEventListener: listen('body') },
|
||||
addEventListener: listen('document'),
|
||||
getElementById: (id) => ({ 'restart-pending-banner': banner, 'restart-pending-text': text })[id] || null,
|
||||
querySelector: () => null,
|
||||
querySelectorAll: () => [],
|
||||
};
|
||||
const window = {
|
||||
addEventListener: listen('window'),
|
||||
getApp: () => null,
|
||||
};
|
||||
const context = {
|
||||
window, document, console,
|
||||
sessionStorage: {
|
||||
setItem: (k, v) => { store[k] = String(v); },
|
||||
removeItem: (k) => { delete store[k]; },
|
||||
getItem: (k) => (k in store ? store[k] : null),
|
||||
},
|
||||
showNotification: () => {},
|
||||
setTimeout: () => 0,
|
||||
};
|
||||
vm.createContext(context);
|
||||
vm.runInContext(fs.readFileSync(path.join(V3, 'app.js'), 'utf8'), context);
|
||||
const afterRequest = (handlers['body:htmx:afterRequest'] || [])[0];
|
||||
const fire = ({ status = 200, body, path: reqPath = '/api/v3/anything', reportsItself = false }) => {
|
||||
const elt = { closest: () => (reportsItself ? {} : null) };
|
||||
afterRequest({
|
||||
target: { closest: () => null },
|
||||
detail: {
|
||||
xhr: { status, responseText: body === undefined ? '' : (typeof body === 'string' ? body : JSON.stringify(body)) },
|
||||
elt,
|
||||
requestConfig: { verb: 'post', path: reqPath },
|
||||
},
|
||||
});
|
||||
};
|
||||
return { window, banner, text, store, fire, afterRequest };
|
||||
}
|
||||
|
||||
console.log('\nwindow.noteRestartRequired');
|
||||
{
|
||||
const t = load();
|
||||
ok('app.js defines it', typeof t.window.noteRestartRequired === 'function');
|
||||
ok('no flag, no banner', t.window.noteRestartRequired({ status: 'success' }) === false
|
||||
&& t.banner.style.display === 'none');
|
||||
ok('a missing body is ignored', t.window.noteRestartRequired(null) === false);
|
||||
ok('restart_required: false is not a request',
|
||||
t.window.noteRestartRequired({ restart_required: false }) === false && t.banner.style.display === 'none');
|
||||
ok('restart_required: true shows the banner',
|
||||
t.window.noteRestartRequired({ restart_required: true }) === true && t.banner.style.display === 'block');
|
||||
ok('without a message the template wording stays',
|
||||
t.text.textContent === 'Configuration saved — restart the display to apply the changes', t.text.textContent);
|
||||
t.window.noteRestartRequired({ restart_required: true, restart_message: 'Plugin updated — restart the display to run the new version' });
|
||||
ok('restart_message becomes the wording',
|
||||
t.text.textContent === 'Plugin updated — restart the display to run the new version', t.text.textContent);
|
||||
ok('and survives a reload with the flag', t.store['ledmatrix-restart-pending'] === '1'
|
||||
&& t.store['ledmatrix-restart-pending-text'] === 'Plugin updated — restart the display to run the new version');
|
||||
}
|
||||
|
||||
console.log('\nhtmx after-request follows the flag, not the URL');
|
||||
{
|
||||
const t = load();
|
||||
ok('the handler is registered', typeof t.afterRequest === 'function');
|
||||
t.fire({ path: '/api/v3/config/main', body: { status: 'success', message: 'Configuration saved successfully' } });
|
||||
ok('a /config/main answer without the flag raises nothing', t.banner.style.display === 'none');
|
||||
t.fire({ path: '/api/v3/plugins/update', status: 500, body: { status: 'error', restart_required: true } });
|
||||
ok('an error answer never raises it', t.banner.style.display === 'none');
|
||||
t.fire({ path: '/api/v3/whatever', body: '<html>not json' });
|
||||
ok('a non-JSON answer is ignored', t.banner.style.display === 'none');
|
||||
// The main-config forms report their own result (hx-on after-request), so
|
||||
// the flag must be read even when the toast is not this handler's to show.
|
||||
t.fire({ path: '/api/v3/config/main', reportsItself: true,
|
||||
body: { status: 'success', message: 'Configuration saved successfully', restart_required: true } });
|
||||
ok('a flagged answer raises it, even from a form that reports itself', t.banner.style.display === 'block');
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
@@ -0,0 +1,105 @@
|
||||
// The store card shows the registry fields added after 3.7.0 -- the commit
|
||||
// that introduced the listed version and a warning when the plugin needs a
|
||||
// newer core -- and still renders a card from an older registry that has
|
||||
// neither. isStorePluginInstalled also answers to an entry's `aliases`.
|
||||
//
|
||||
// Rendered with the shipped functions (extracted from plugins_manager.js).
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const SRC = fs.readFileSync(
|
||||
path.resolve(__dirname, '../../../web_interface/static/v3/plugins_manager.js'), 'utf8');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra).slice(0, 400) : '')));
|
||||
|
||||
function extract(opener) {
|
||||
const start = SRC.indexOf(opener);
|
||||
if (start < 0) { console.error('FAIL: cannot find ' + JSON.stringify(opener)); process.exit(1); }
|
||||
let depth = 0;
|
||||
for (let j = SRC.indexOf('{', start); j < SRC.length; j++) {
|
||||
if (SRC[j] === '{') depth++;
|
||||
else if (SRC[j] === '}' && --depth === 0) return SRC.slice(start, j + 1);
|
||||
}
|
||||
console.error('FAIL: unbalanced braces after ' + opener); process.exit(1);
|
||||
}
|
||||
|
||||
class FakeEl {
|
||||
constructor() { this.innerHTML = ''; this.value = ''; this.textContent = ''; }
|
||||
}
|
||||
class TextEl {
|
||||
set textContent(v) { this._t = String(v == null ? '' : v); }
|
||||
get innerHTML() {
|
||||
return (this._t || '').replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>');
|
||||
}
|
||||
}
|
||||
const els = {};
|
||||
global.document = {
|
||||
getElementById: id => (els[id] ||= new FakeEl()),
|
||||
createElement: () => new TextEl(),
|
||||
};
|
||||
global.window = global;
|
||||
require('../led_escape').install(window);
|
||||
global.pluginLog = () => {};
|
||||
global.isNewPlugin = () => false;
|
||||
global.formatDate = () => '';
|
||||
global.setGridHtmlIfChanged = (container, html) => { container.innerHTML = html; };
|
||||
global.installedPlugins = [];
|
||||
|
||||
// eslint-disable-next-line no-eval
|
||||
eval([
|
||||
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
|
||||
'function isStorePluginInstalled(pluginIdOrPlugin) {', 'function renderPluginStore(plugins) {',
|
||||
].map(extract).join('\n') + '\nglobal.renderPluginStore = renderPluginStore;'
|
||||
+ '\nglobal.isStorePluginInstalled = isStorePluginInstalled;');
|
||||
|
||||
function render(plugin) {
|
||||
renderPluginStore([plugin]);
|
||||
return els['plugin-store-grid'].innerHTML;
|
||||
}
|
||||
|
||||
const SHA = '843588025a81197056f8d96779ccb2be19337ab8';
|
||||
const base = {
|
||||
id: 'weather', name: 'Weather', author: 'ChuckBuilds', category: 'weather',
|
||||
description: 'Forecasts', version: '2.1.0',
|
||||
repo: 'https://github.com/ChuckBuilds/ledmatrix-plugins', plugin_path: 'plugins/ledmatrix-weather',
|
||||
};
|
||||
|
||||
console.log('\n1. a registry with the new fields');
|
||||
let html = render({ ...base, commit: SHA, ledmatrix_min_version: '3.7.0', aliases: ['ledmatrix-weather'] });
|
||||
ok('shows the short commit', html.includes('>8435880<'), html.match(/v2\.1\.0[^\n]*/));
|
||||
ok('links it to the plugin at that commit',
|
||||
html.includes(`href="https://github.com/ChuckBuilds/ledmatrix-plugins/tree/${SHA}/plugins/ledmatrix-weather"`));
|
||||
ok('no compatibility warning when the core is new enough', !html.includes('Needs LEDMatrix'));
|
||||
|
||||
console.log('\n2. a plugin this core cannot run');
|
||||
html = render({ ...base, ledmatrix_min_version: '9.0.0',
|
||||
incompatible_reason: 'Weather requires LEDMatrix 9.0.0 or newer' });
|
||||
ok('warns with the floor', html.includes('Needs LEDMatrix 9.0.0+'));
|
||||
ok('and the reason as its title', html.includes('title="Weather requires LEDMatrix 9.0.0 or newer"'));
|
||||
|
||||
console.log('\n3. an older registry: no commit, floor or aliases');
|
||||
html = render({ ...base });
|
||||
ok('still renders the card and its version', html.includes('class="plugin-card"') && html.includes('v2.1.0'));
|
||||
ok('shows no commit and no warning', !html.includes('font-mono') && !html.includes('Needs LEDMatrix'));
|
||||
|
||||
console.log('\n4. a commit value that is not a SHA is not rendered');
|
||||
html = render({ ...base, commit: 'javascript:alert(1)' });
|
||||
ok('dropped', !html.includes('javascript:') && !html.includes('font-mono'));
|
||||
html = render({ ...base, commit: SHA, repo: 'javascript:alert(1)' });
|
||||
ok('without a web repo link it is plain text, not a link',
|
||||
html.includes('>8435880<') && !html.includes('/tree/'));
|
||||
|
||||
console.log('\n5. installed under an alias');
|
||||
global.installedPlugins = [{ id: 'ledmatrix-weather' }];
|
||||
ok('aliases count as installed',
|
||||
isStorePluginInstalled({ id: 'weather', plugin_path: '', aliases: ['ledmatrix-weather'] }));
|
||||
ok('plugin_path still does (older registry)',
|
||||
isStorePluginInstalled({ id: 'weather', plugin_path: 'plugins/ledmatrix-weather' }));
|
||||
ok('a different plugin is not installed', !isStorePluginInstalled({ id: 'stocks', plugin_path: '' }));
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
@@ -229,6 +229,78 @@ const noSleep = { sleep: async () => {} };
|
||||
allNoop.type === 'success' && allNoop.text === '2 already up to date', allNoop);
|
||||
}
|
||||
|
||||
console.log('\nrestart banner: driven by the server\'s restart_required');
|
||||
{
|
||||
const body = (restart_required, restart_message) => ({
|
||||
success: true,
|
||||
result: { status: 'success', data: { update_status: 'updated' }, restart_required, restart_message },
|
||||
});
|
||||
const needed = body(true, 'Plugin updated — restart the display to run the new version');
|
||||
ok('an update the display is running asks for the banner, with its wording',
|
||||
Manager.restartRequest([body(false), needed, body(true, 'second')]) === needed.result);
|
||||
ok('updates the display does not run need no restart',
|
||||
Manager.restartRequest([body(false), body(false)]) === null);
|
||||
ok('a failed request never raises the banner',
|
||||
Manager.restartRequest([{ success: false, error: { restart_required: true } }]) === null);
|
||||
ok('an older server that sends no flag raises nothing',
|
||||
Manager.restartRequest([{ success: true, result: { status: 'success' } }]) === null);
|
||||
ok('no results, no banner', Manager.restartRequest(undefined) === null);
|
||||
|
||||
// The first request's answer was lost; the re-sent one finds nothing to do.
|
||||
const lost = (enabled, update_status = 'up_to_date') => ({
|
||||
pluginId: 'clock', success: true, afterLostAnswer: true, enabled,
|
||||
result: { status: 'success', data: { update_status }, restart_required: false },
|
||||
});
|
||||
const maybe = Manager.restartRequest([body(false), lost(true)]);
|
||||
ok('an enabled plugin up to date after a lost answer may have been updated: banner',
|
||||
maybe && maybe.restart_required === true && /clock/.test(maybe.restart_message), maybe);
|
||||
ok('...but an explicit answer still wins, with its wording',
|
||||
Manager.restartRequest([lost(true), needed]) === needed.result);
|
||||
ok('a disabled one needs no restart (enabling it loads it)',
|
||||
Manager.restartRequest([lost(false)]) === null);
|
||||
ok('nor does one that was not retried',
|
||||
Manager.restartRequest([{ ...lost(true), afterLostAnswer: undefined }]) === null);
|
||||
}
|
||||
|
||||
console.log('\nupdateAll keeps what the banner needs');
|
||||
{
|
||||
// ledmatrix-flights (enabled) loses its first answer, then is up to date.
|
||||
const api = fakeApi({
|
||||
'ledmatrix-flights': (n) => {
|
||||
if (n === 1) throw netErr();
|
||||
return { status: 'success', data: { update_status: 'up_to_date' }, restart_required: false };
|
||||
},
|
||||
});
|
||||
setup(api, { windowList: INSTALLED });
|
||||
const results = await Manager.updateAll(null, noSleep);
|
||||
const flights = results.find(r => r.pluginId === 'ledmatrix-flights');
|
||||
ok('a retried entry is marked, with the plugin\'s enabled flag',
|
||||
flights.afterLostAnswer === true && flights.enabled === true, flights);
|
||||
ok('an entry answered first time is not marked',
|
||||
results.filter(r => r.afterLostAnswer).length === 1, results);
|
||||
ok('...so the run asks for a restart', Manager.restartRequest(results) !== null);
|
||||
}
|
||||
{
|
||||
const answer = { status: 'success', data: { update_status: 'updated' }, restart_required: true };
|
||||
const api = fakeApi({ 'ledmatrix-flights': () => answer });
|
||||
setup(api, { stateList: INSTALLED });
|
||||
window.PluginStateManager.loadInstalledPlugins = async () => { throw new Error('refresh failed'); };
|
||||
const warn = console.warn;
|
||||
console.warn = () => {};
|
||||
let results;
|
||||
try {
|
||||
results = await Manager.updateAll(null, noSleep);
|
||||
} catch (e) {
|
||||
results = e;
|
||||
} finally {
|
||||
console.warn = warn;
|
||||
}
|
||||
ok('a failed list refresh still returns the results',
|
||||
Array.isArray(results) && results.length === EXPECTED.length, String(results));
|
||||
ok('...with the restart flag intact',
|
||||
Array.isArray(results) && Manager.restartRequest(results) === answer);
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
|
||||
@@ -52,7 +52,7 @@ class TestPluginToggle:
|
||||
class TestOnDemandStart:
|
||||
@pytest.fixture
|
||||
def service(self, api_v3_module):
|
||||
api_v3_module.api_v3.plugin_manager = None
|
||||
api_v3_module.api_v3.plugin_catalog = None
|
||||
api_v3_module.api_v3.config_manager = None
|
||||
with patch("web_interface.blueprints.api_v3.display._get_display_service_status",
|
||||
return_value={"active": True}), \
|
||||
|
||||
@@ -45,7 +45,7 @@ VALID_CREDENTIALS = {
|
||||
def plugin_dir(tmp_path, api_v3_module):
|
||||
directory = tmp_path / "plugins" / "calendar"
|
||||
directory.mkdir(parents=True)
|
||||
api_v3_module.api_v3.plugin_manager.get_plugin_directory.return_value = str(directory)
|
||||
api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = str(directory)
|
||||
return directory
|
||||
|
||||
|
||||
@@ -98,7 +98,7 @@ class TestRequestValidation:
|
||||
assert not (plugin_dir / "credentials.json").exists()
|
||||
|
||||
def test_missing_plugin_directory_is_a_404(self, api_v3_client, api_v3_module, tmp_path):
|
||||
api_v3_module.api_v3.plugin_manager.get_plugin_directory.return_value = str(
|
||||
api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = str(
|
||||
tmp_path / "not-installed")
|
||||
assert upload(api_v3_client, VALID_CREDENTIALS).status_code == 404
|
||||
|
||||
|
||||
@@ -229,7 +229,7 @@ def display_page(monkeypatch):
|
||||
config_manager.get_config_path.return_value = 'config/config.json'
|
||||
config_manager.get_secrets_path.return_value = 'config/config_secrets.json'
|
||||
monkeypatch.setattr(pv.pages_v3, 'config_manager', config_manager, raising=False)
|
||||
monkeypatch.setattr(pv.pages_v3, 'plugin_manager', MagicMock(plugins={}), raising=False)
|
||||
monkeypatch.setattr(pv.pages_v3, 'plugin_catalog', MagicMock(), raising=False)
|
||||
app.register_blueprint(pv.pages_v3, url_prefix='/v3')
|
||||
response = app.test_client().get('/v3/partials/display')
|
||||
assert response.status_code == 200
|
||||
|
||||
@@ -34,7 +34,7 @@ CONFIG = {
|
||||
|
||||
@pytest.fixture
|
||||
def client(api_v3_module, api_v3_client):
|
||||
pm = api_v3_module.api_v3.plugin_manager
|
||||
pm = api_v3_module.api_v3.plugin_catalog
|
||||
pm.plugin_manifests = MANIFESTS
|
||||
pm.discover_plugins = MagicMock(return_value=list(MANIFESTS))
|
||||
pm.get_plugin_display_modes = MagicMock(
|
||||
@@ -85,17 +85,17 @@ class TestItWorksForACallerThatNeverOpensTheDashboard:
|
||||
"""Discovery is lazy and normally runs because a person loaded the
|
||||
dashboard; a bridge or script would otherwise get an empty list."""
|
||||
client.get('/api/v3/display/modes')
|
||||
api_v3_module.api_v3.plugin_manager.discover_plugins.assert_called_once()
|
||||
api_v3_module.api_v3.plugin_catalog.discover_plugins.assert_called_once()
|
||||
|
||||
def test_no_plugin_manager_is_a_clean_error(self, api_v3_module, api_v3_client):
|
||||
api_v3_module.api_v3.plugin_manager = None
|
||||
api_v3_module.api_v3.plugin_catalog = None
|
||||
response = api_v3_client.get('/api/v3/display/modes')
|
||||
assert response.status_code == 500
|
||||
assert response.get_json()['status'] == 'error'
|
||||
|
||||
def test_a_plugin_with_no_declared_modes_still_appears(self, client, api_v3_module):
|
||||
"""Its mode is its own id -- the same fallback the controller uses."""
|
||||
pm = api_v3_module.api_v3.plugin_manager
|
||||
pm = api_v3_module.api_v3.plugin_catalog
|
||||
pm.plugin_manifests = {'starlark-apps': {'name': 'Starlark Apps', 'display_modes': []}}
|
||||
pm.get_plugin_display_modes = MagicMock(return_value=[])
|
||||
api_v3_module.api_v3.config_manager.load_config = MagicMock(
|
||||
@@ -116,7 +116,7 @@ class TestOneBadConfigSectionDoesNotBlankTheList:
|
||||
|
||||
@pytest.fixture
|
||||
def client_with_bad_section(self, api_v3_module, api_v3_client):
|
||||
pm = api_v3_module.api_v3.plugin_manager
|
||||
pm = api_v3_module.api_v3.plugin_catalog
|
||||
pm.plugin_manifests = MANIFESTS
|
||||
pm.discover_plugins = MagicMock(return_value=list(MANIFESTS))
|
||||
pm.get_plugin_display_modes = MagicMock(
|
||||
@@ -143,7 +143,7 @@ class TestOneBadConfigSectionDoesNotBlankTheList:
|
||||
self, api_v3_module, api_v3_client):
|
||||
"""describe_exception, per test_web_error_detail's contract -- an
|
||||
opaque "see logs for details" is what that test exists to prevent."""
|
||||
api_v3_module.api_v3.plugin_manager.discover_plugins = MagicMock(
|
||||
api_v3_module.api_v3.plugin_catalog.discover_plugins = MagicMock(
|
||||
side_effect=RuntimeError("disk is gone"))
|
||||
resp = api_v3_client.get('/api/v3/display/modes')
|
||||
assert resp.status_code == 500
|
||||
@@ -151,7 +151,7 @@ class TestOneBadConfigSectionDoesNotBlankTheList:
|
||||
|
||||
def test_credentials_in_the_exception_are_redacted(self, api_v3_module, api_v3_client):
|
||||
"""describe_exception is what makes returning detail safe."""
|
||||
api_v3_module.api_v3.plugin_manager.discover_plugins = MagicMock(
|
||||
api_v3_module.api_v3.plugin_catalog.discover_plugins = MagicMock(
|
||||
side_effect=RuntimeError("GET https://x/y?api_key=SEC123 failed"))
|
||||
body = api_v3_client.get('/api/v3/display/modes').get_json()
|
||||
assert 'SEC123' not in json.dumps(body)
|
||||
|
||||
@@ -5,8 +5,10 @@ PluginManager does not have; a hasattr guard turned that into a permanent 0.
|
||||
Each check that fails answers "see logs for details", so it has to log.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
@@ -25,6 +27,28 @@ def _no_systemctl(monkeypatch):
|
||||
lambda: {"active": True})
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def heartbeat(tmp_path, monkeypatch):
|
||||
"""The display's heartbeat file, somewhere private; absent until written."""
|
||||
from src import display_watchdog
|
||||
path = tmp_path / "display-heartbeat.json"
|
||||
monkeypatch.setattr(display_watchdog, "HEARTBEAT_PATH", str(path))
|
||||
|
||||
def write(age):
|
||||
path.write_text(json.dumps({"pid": 1, "mono": time.monotonic() - age,
|
||||
"wall": time.time() - age}))
|
||||
return write
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fresh_preview(tmp_path, monkeypatch):
|
||||
"""A just-written preview frame, so only the heartbeat decides the verdict."""
|
||||
from web_interface import display_preview
|
||||
snapshot = tmp_path / "preview.png"
|
||||
snapshot.write_bytes(b"png")
|
||||
monkeypatch.setattr(display_preview, "SNAPSHOT_PATH", str(snapshot))
|
||||
|
||||
|
||||
def _checks(client):
|
||||
response = client.get(URL)
|
||||
assert response.status_code == 200, response.get_json()
|
||||
@@ -32,7 +56,7 @@ def _checks(client):
|
||||
|
||||
|
||||
def test_plugin_count_is_the_number_of_discovered_plugins(api_v3_client, api_v3_module):
|
||||
api_v3_module.api_v3.plugin_manager.plugin_manifests = {
|
||||
api_v3_module.api_v3.plugin_catalog.plugin_manifests = {
|
||||
"clock": {"id": "clock"}, "weather": {"id": "weather"}, "stocks": {"id": "stocks"},
|
||||
}
|
||||
|
||||
@@ -42,7 +66,7 @@ def test_plugin_count_is_the_number_of_discovered_plugins(api_v3_client, api_v3_
|
||||
|
||||
|
||||
def test_plugin_count_discovers_when_nothing_is_discovered_yet(api_v3_client, api_v3_module):
|
||||
pm = api_v3_module.api_v3.plugin_manager
|
||||
pm = api_v3_module.api_v3.plugin_catalog
|
||||
pm.plugin_manifests = {}
|
||||
|
||||
def discover():
|
||||
@@ -88,3 +112,59 @@ def test_a_failed_hardware_check_is_logged(api_v3_client, api_v3_module, caplog,
|
||||
assert check["status"] == "unknown"
|
||||
logged = [r for r in caplog.records if "snapshot" in r.getMessage()]
|
||||
assert logged and logged[0].exc_info
|
||||
|
||||
|
||||
# -- the display's render-loop heartbeat -----------------------------------------
|
||||
#
|
||||
# The preview frame's age said nothing about a panel frozen by a render thread
|
||||
# stuck in a plugin; the heartbeat is written by that thread itself.
|
||||
|
||||
def _health(client):
|
||||
response = client.get(URL)
|
||||
assert response.status_code == 200, response.get_json()
|
||||
return response.get_json()["data"]
|
||||
|
||||
|
||||
def test_a_fresh_heartbeat_is_a_running_display_loop(api_v3_client, heartbeat, fresh_preview):
|
||||
heartbeat(age=3)
|
||||
|
||||
data = _health(api_v3_client)
|
||||
|
||||
assert data["checks"]["display_loop"]["status"] == "running"
|
||||
assert 2 <= data["checks"]["display_loop"]["heartbeat_age_seconds"] < 10
|
||||
assert data["status"] == "healthy"
|
||||
|
||||
|
||||
def test_a_stale_heartbeat_is_a_stalled_display_loop(api_v3_client, heartbeat, fresh_preview):
|
||||
"""Service active, preview recent, and still the panel is frozen."""
|
||||
heartbeat(age=300)
|
||||
|
||||
data = _health(api_v3_client)
|
||||
|
||||
assert data["checks"]["display_loop"]["status"] == "stalled"
|
||||
assert data["checks"]["display_loop"]["heartbeat_age_seconds"] >= 299
|
||||
assert data["status"] == "degraded"
|
||||
|
||||
|
||||
def test_no_heartbeat_falls_back_to_the_older_checks(api_v3_client, fresh_preview):
|
||||
"""The dev server, the emulator, Windows, or a display without the
|
||||
feature: absence is not a failure, and the verdict is what it was."""
|
||||
data = _health(api_v3_client)
|
||||
|
||||
assert data["checks"]["display_loop"]["status"] == "not_reported"
|
||||
assert data["checks"]["hardware"]["status"] == "connected"
|
||||
assert data["status"] == "healthy"
|
||||
|
||||
|
||||
def test_an_unreadable_heartbeat_is_reported_not_raised(api_v3_client, monkeypatch, caplog):
|
||||
from src import display_watchdog
|
||||
|
||||
def boom(_path):
|
||||
raise RuntimeError("bad heartbeat")
|
||||
monkeypatch.setattr(display_watchdog, "read_heartbeat", boom)
|
||||
|
||||
with caplog.at_level(logging.WARNING):
|
||||
check = _checks(api_v3_client)["display_loop"]
|
||||
|
||||
assert check["status"] == "unknown"
|
||||
assert any("heartbeat" in r.getMessage() for r in caplog.records)
|
||||
|
||||
@@ -20,9 +20,8 @@ def installed(api_v3_module, api_v3_client, tmp_path):
|
||||
api = api_v3_module.api_v3
|
||||
info = {'id': 'demo', 'name': 'Demo', 'version': '1.0.0', 'loaded': False}
|
||||
info.update(manifest_extra)
|
||||
api.plugin_manager.plugins_dir = str(tmp_path) # no manifest on disk
|
||||
api.plugin_manager.get_all_plugin_info = MagicMock(return_value=[info])
|
||||
api.plugin_manager.get_plugin = MagicMock(return_value=None)
|
||||
api.plugin_catalog.plugins_dir = str(tmp_path) # no manifest on disk
|
||||
api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])
|
||||
api.plugin_store_manager.get_registry_info = MagicMock(return_value=None)
|
||||
api.config_manager.load_config = MagicMock(return_value={})
|
||||
response = api_v3_client.get('/api/v3/plugins/installed')
|
||||
|
||||
@@ -16,7 +16,7 @@ callers (the Home Assistant MQTT bridge, scripts) saw it after every restart.
|
||||
undiscovered plugin section skipped secret separation and wrote its API key
|
||||
into config.json in plain text.
|
||||
|
||||
A real PluginManager over a temporary plugins directory, so "empty until
|
||||
A real PluginCatalog over a temporary plugins directory, so "empty until
|
||||
discovered" is the real behaviour rather than a mock's.
|
||||
"""
|
||||
|
||||
@@ -30,7 +30,7 @@ import pytest
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from src.config_manager import ConfigManager # noqa: E402
|
||||
from src.plugin_system.plugin_manager import PluginManager # noqa: E402
|
||||
from src.plugin_system.plugin_catalog import PluginCatalog # noqa: E402
|
||||
from src.plugin_system.schema_manager import SchemaManager # noqa: E402
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
@@ -69,10 +69,10 @@ def plugins_dir(tmp_path):
|
||||
@pytest.fixture
|
||||
def fresh_web_process(api_v3_module, plugins_dir):
|
||||
"""The web process right after a restart: nothing discovered yet."""
|
||||
manager = PluginManager(plugins_dir=str(plugins_dir))
|
||||
assert not manager.plugin_manifests
|
||||
api_v3_module.api_v3.plugin_manager = manager
|
||||
return manager
|
||||
catalog = PluginCatalog(plugins_dir=plugins_dir)
|
||||
assert not catalog.plugin_manifests
|
||||
api_v3_module.api_v3.plugin_catalog = catalog
|
||||
return catalog
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
|
||||
@@ -1,25 +1,27 @@
|
||||
"""Regression test: POST /display/on-demand/start restarting a running
|
||||
service must not import a name that does not exist.
|
||||
"""POST /display/on-demand/start and /stop must not restart a running display.
|
||||
|
||||
display.py has `import web_interface.blueprints.api_v3 as _pkg` and reads
|
||||
mutable, test-patched attributes back through it (`_pkg.time.time()`,
|
||||
`_pkg._get_starlark_plugin()`, ...) rather than binding them by value, per
|
||||
the package's own docstring. One spot went further and wrote a genuine
|
||||
`import` *statement* against that alias --
|
||||
The start route used to treat ``start_service`` (default True, and what both
|
||||
the web UI and the MQTT bridge send) as "restart": with the service running it
|
||||
ran ``systemctl stop``, slept 1.5s and started it again. Every on-demand or
|
||||
"Preview on display" click therefore cold-restarted the display process --
|
||||
every plugin reloaded, the panel blank for seconds -- to deliver a request the
|
||||
running process polls for every ON_DEMAND_POLL_INTERVAL anyway (see
|
||||
test_on_demand_mailbox.py and test_display_pending_changes.py for the display
|
||||
side: the mailbox is read mid-dwell, mid-screen and mid-Vegas-iteration).
|
||||
|
||||
import _pkg.time as time_module
|
||||
The restart did not buy anything either: a freshly started display restores
|
||||
only the on-demand session it saved itself (``display_on_demand_config``), so
|
||||
the new request reached it through the same mailbox, one cold start later.
|
||||
|
||||
-- but `_pkg` is a local name bound by `import ... as _pkg` in this module,
|
||||
not a real top-level package, so `import _pkg.time` is not something Python
|
||||
can resolve; it raises ModuleNotFoundError. That line only runs when the
|
||||
display service is already running and the caller also asked to (re)start
|
||||
it, so this endpoint failed on exactly the restart path -- the one where a
|
||||
cache write recording the new on-demand request had already happened.
|
||||
This file previously pinned that restart path (it guarded a broken
|
||||
``import _pkg.time`` inside it). The path is gone; these tests pin its
|
||||
replacement: a running service is left alone, a stopped one is started (only
|
||||
when start_service is set), and the request lands in the mailbox either way.
|
||||
|
||||
The route wraps its body in `except Exception`, so the failure reached the
|
||||
caller as a handled 500 with a generic message, not an unhandled crash --
|
||||
but a 500 all the same on a request that should have restarted the service
|
||||
and reported success.
|
||||
The service helpers are patched where they run. display.py binds
|
||||
_get_display_service_status by value, while _ensure_display_service_running
|
||||
(in the package __init__) looks it up in its own module, so both are patched;
|
||||
_run_systemctl_command is the one place a systemctl command is issued.
|
||||
"""
|
||||
|
||||
import sys
|
||||
@@ -32,60 +34,137 @@ sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
URL = "/api/v3/display/on-demand/start"
|
||||
START_URL = "/api/v3/display/on-demand/start"
|
||||
STOP_URL = "/api/v3/display/on-demand/stop"
|
||||
MAILBOX = "display_on_demand_request"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def restart_path(api_v3_module):
|
||||
"""Force the `service_was_running and start_service` branch.
|
||||
def service(api_v3_module):
|
||||
"""A display service whose state the test sets; records systemctl calls.
|
||||
|
||||
plugin_manager and config_manager are set to None so the route takes
|
||||
the simplest path to that branch rather than tripping over unrelated
|
||||
MagicMock plumbing. The cache is the blueprint's cache_manager, which
|
||||
api_v3_module already set to a MagicMock. _get_display_service_status,
|
||||
_stop_display_service and _ensure_display_service_running are bound by
|
||||
value in display.py (see its own docstring), so they are patched on
|
||||
that submodule rather than on the package.
|
||||
plugin_manager and config_manager are None so the route skips plugin
|
||||
resolution (not what is under test here). The cache is the blueprint's
|
||||
MagicMock cache_manager, so mailbox writes are visible as set() calls.
|
||||
"""
|
||||
api_v3_module.api_v3.plugin_manager = None
|
||||
api_v3_module.api_v3.plugin_catalog = None
|
||||
api_v3_module.api_v3.config_manager = None
|
||||
state = {"active": True}
|
||||
|
||||
with patch("web_interface.blueprints.api_v3.display._get_display_service_status") as get_status, \
|
||||
patch("web_interface.blueprints.api_v3.display._stop_display_service") as stop_service, \
|
||||
patch("web_interface.blueprints.api_v3.display._ensure_display_service_running") as ensure_running:
|
||||
# Active before the request: service_was_running becomes True.
|
||||
get_status.return_value = {"active": True}
|
||||
ensure_running.return_value = {"active": True}
|
||||
def status():
|
||||
return {"active": state["active"]}
|
||||
|
||||
def systemctl(args):
|
||||
if args[-2:] == ["start", "ledmatrix.service"]:
|
||||
state["active"] = True
|
||||
elif args[-2:] == ["stop", "ledmatrix.service"]:
|
||||
state["active"] = False
|
||||
return {"returncode": 0, "stdout": "", "stderr": ""}
|
||||
|
||||
with patch("web_interface.blueprints.api_v3._get_display_service_status",
|
||||
side_effect=status), \
|
||||
patch("web_interface.blueprints.api_v3.display._get_display_service_status",
|
||||
side_effect=status), \
|
||||
patch("web_interface.blueprints.api_v3._run_systemctl_command",
|
||||
side_effect=systemctl) as run_systemctl, \
|
||||
patch("web_interface.blueprints.api_v3.display._stop_display_service") as stop_service:
|
||||
yield {
|
||||
"get_status": get_status,
|
||||
"state": state,
|
||||
"systemctl": run_systemctl,
|
||||
"stop_service": stop_service,
|
||||
"ensure_running": ensure_running,
|
||||
"cache": api_v3_module.api_v3.cache_manager,
|
||||
}
|
||||
|
||||
|
||||
class TestRestartingARunningService:
|
||||
def test_it_does_not_500(self, api_v3_client, restart_path):
|
||||
response = api_v3_client.post(
|
||||
URL, json={"plugin_id": "weather", "start_service": True})
|
||||
body = response.get_json()
|
||||
assert response.status_code == 200, body
|
||||
assert body["status"] == "success", body
|
||||
def _mailbox_writes(cache):
|
||||
return [c.args[1] for c in cache.set.call_args_list if c.args and c.args[0] == MAILBOX]
|
||||
|
||||
def test_the_service_is_actually_stopped_and_restarted(
|
||||
self, api_v3_client, restart_path):
|
||||
api_v3_client.post(
|
||||
URL, json={"plugin_id": "weather", "start_service": True})
|
||||
restart_path["stop_service"].assert_called_once()
|
||||
restart_path["ensure_running"].assert_called_once()
|
||||
|
||||
def test_a_service_that_was_not_running_is_not_stopped_first(
|
||||
self, api_v3_client, restart_path):
|
||||
# The buggy import sits inside `if service_was_running and
|
||||
# start_service`, so it only ever fired on the restart path --
|
||||
# this is the other side of that branch, unaffected either way,
|
||||
# kept here so the branch condition itself stays covered.
|
||||
restart_path["get_status"].return_value = {"active": False}
|
||||
response = api_v3_client.post(
|
||||
URL, json={"plugin_id": "weather", "start_service": True})
|
||||
def _systemctl_verbs(run_systemctl):
|
||||
return [c.args[0][-2] for c in run_systemctl.call_args_list]
|
||||
|
||||
|
||||
class TestStartWhileTheServiceIsRunning:
|
||||
@pytest.mark.parametrize("body", [
|
||||
{"plugin_id": "weather"}, # "Preview on display", MQTT
|
||||
{"plugin_id": "weather", "start_service": True}, # on-demand modal, box ticked
|
||||
{"plugin_id": "weather", "start_service": "true"},
|
||||
])
|
||||
def test_the_service_is_not_stopped_or_restarted(self, api_v3_client, service, body):
|
||||
response = api_v3_client.post(START_URL, json=body)
|
||||
assert response.status_code == 200, response.get_json()
|
||||
restart_path["stop_service"].assert_not_called()
|
||||
assert response.get_json()["status"] == "success"
|
||||
service["stop_service"].assert_not_called()
|
||||
assert _systemctl_verbs(service["systemctl"]) == [], (
|
||||
"a running display service was sent a systemctl command")
|
||||
|
||||
def test_the_request_is_posted_for_the_running_display(self, api_v3_client, service):
|
||||
response = api_v3_client.post(
|
||||
START_URL, json={"plugin_id": "weather", "mode": "weather_current",
|
||||
"duration": 60, "pinned": True})
|
||||
data = response.get_json()["data"]
|
||||
writes = _mailbox_writes(service["cache"])
|
||||
assert len(writes) == 1
|
||||
assert writes[0]["action"] == "start"
|
||||
assert writes[0]["request_id"] == data["request_id"]
|
||||
assert writes[0]["plugin_id"] == "weather"
|
||||
assert writes[0]["mode"] == "weather_current"
|
||||
assert writes[0]["duration"] == 60
|
||||
assert writes[0]["pinned"] is True
|
||||
|
||||
def test_the_response_reports_the_service_was_not_started(self, api_v3_client, service):
|
||||
data = api_v3_client.post(START_URL, json={"plugin_id": "weather"}).get_json()["data"]
|
||||
assert data["service"]["active"] is True
|
||||
assert data["service"]["started"] is False
|
||||
|
||||
def test_it_answers_without_the_old_restart_pause(self, api_v3_client, service):
|
||||
# The restart slept 1.5s; nothing here should sleep at all.
|
||||
with patch("time.sleep") as sleep:
|
||||
api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
sleep.assert_not_called()
|
||||
|
||||
|
||||
class TestStartWhileTheServiceIsStopped:
|
||||
def test_start_service_starts_it_once_and_never_stops_it(self, api_v3_client, service):
|
||||
service["state"]["active"] = False
|
||||
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert _systemctl_verbs(service["systemctl"]) == ["start"]
|
||||
service["stop_service"].assert_not_called()
|
||||
# Written before the start, so the new process finds it on its first poll.
|
||||
assert len(_mailbox_writes(service["cache"])) == 1
|
||||
|
||||
def test_without_start_service_it_is_left_stopped(self, api_v3_client, service):
|
||||
service["state"]["active"] = False
|
||||
response = api_v3_client.post(
|
||||
START_URL, json={"plugin_id": "weather", "start_service": "false"})
|
||||
assert response.status_code == 400
|
||||
assert _systemctl_verbs(service["systemctl"]) == []
|
||||
|
||||
def test_a_start_that_fails_is_reported(self, api_v3_client, service):
|
||||
service["state"]["active"] = False
|
||||
service["systemctl"].side_effect = lambda args: {
|
||||
"returncode": 1, "stdout": "", "stderr": "denied"}
|
||||
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 500
|
||||
assert response.get_json()["status"] == "error"
|
||||
|
||||
|
||||
class TestStop:
|
||||
def test_stop_posts_a_stop_request_and_leaves_the_service_running(
|
||||
self, api_v3_client, service):
|
||||
response = api_v3_client.post(STOP_URL, json={})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
writes = _mailbox_writes(service["cache"])
|
||||
assert [w["action"] for w in writes] == ["stop"]
|
||||
service["stop_service"].assert_not_called()
|
||||
assert _systemctl_verbs(service["systemctl"]) == []
|
||||
|
||||
def test_a_string_false_stop_service_does_not_stop_it(self, api_v3_client, service):
|
||||
# bool("false") is True: the flag was read raw and stopped the service.
|
||||
api_v3_client.post(STOP_URL, json={"stop_service": "false"})
|
||||
service["stop_service"].assert_not_called()
|
||||
|
||||
def test_stop_service_true_still_stops_it(self, api_v3_client, service):
|
||||
api_v3_client.post(STOP_URL, json={"stop_service": True})
|
||||
service["stop_service"].assert_called_once()
|
||||
|
||||
@@ -35,9 +35,8 @@ class SharedCache:
|
||||
@pytest.fixture
|
||||
def shared_cache(api_v3_module):
|
||||
cache = SharedCache()
|
||||
pm = api_v3_module.api_v3.plugin_manager
|
||||
pm.health_tracker = PluginHealthTracker(cache)
|
||||
pm.resource_monitor = PluginResourceMonitor(cache)
|
||||
api_v3_module.api_v3.health_tracker = PluginHealthTracker(cache)
|
||||
api_v3_module.api_v3.resource_monitor = PluginResourceMonitor(cache)
|
||||
return cache
|
||||
|
||||
|
||||
|
||||
@@ -5,6 +5,11 @@ Both were only ever tested at the PluginStoreManager layer, so the route
|
||||
logic — the queue-vs-direct branch, schema invalidation, plugin discovery,
|
||||
state and history recording — was unexercised.
|
||||
|
||||
Neither route loads the plugin: the web process only lists it (the catalog
|
||||
has no load_plugin, so calling one fails these tests). The display loads it
|
||||
when it is enabled; restart_required says when that won't happen by itself
|
||||
(test/web_interface/test_web_process_runs_no_plugin_code.py).
|
||||
|
||||
/plugins/install carries the same install logic twice: once inside the
|
||||
operation-queue callback and once in the direct fallback. The paired
|
||||
tests below assert both branches produce the same side effects, so the
|
||||
@@ -44,9 +49,7 @@ def side_effects(module):
|
||||
api = module.api_v3
|
||||
return {
|
||||
"schema_invalidated": api.schema_manager.invalidate_cache.call_args_list,
|
||||
"discovered": api.plugin_manager.discover_plugins.call_count,
|
||||
"loaded": api.plugin_manager.load_plugin.call_args_list,
|
||||
"state_set": api.plugin_state_manager.set_plugin_installed.call_args_list,
|
||||
"discovered": api.plugin_catalog.discover_plugins.call_count,
|
||||
"history": api.operation_history.record_operation.call_args_list,
|
||||
}
|
||||
|
||||
@@ -83,8 +86,6 @@ class TestInstallDirectPath:
|
||||
effects = side_effects(api_v3_module)
|
||||
assert effects["schema_invalidated"] == [(("clock",), {})]
|
||||
assert effects["discovered"] == 1
|
||||
assert effects["loaded"] == [(("clock",), {})]
|
||||
assert effects["state_set"] == [(("clock",), {})]
|
||||
assert effects["history"][0].kwargs["status"] == "success"
|
||||
|
||||
def test_branch_forwarded_to_the_manager(self, api_v3_client, api_v3_module):
|
||||
@@ -130,8 +131,7 @@ class TestInstallDirectPath:
|
||||
api_v3_client.post(INSTALL, json={"plugin_id": "clock"})
|
||||
effects = side_effects(api_v3_module)
|
||||
assert effects["schema_invalidated"] == []
|
||||
assert effects["loaded"] == []
|
||||
assert effects["state_set"] == []
|
||||
assert effects["discovered"] == 0
|
||||
|
||||
|
||||
class TestInstallQueuedPath:
|
||||
@@ -154,8 +154,6 @@ class TestInstallQueuedPath:
|
||||
effects = side_effects(api_v3_module)
|
||||
assert effects["schema_invalidated"] == [(("clock",), {})]
|
||||
assert effects["discovered"] == 1
|
||||
assert effects["loaded"] == [(("clock",), {})]
|
||||
assert effects["state_set"] == [(("clock",), {})]
|
||||
assert effects["history"][0].kwargs["status"] == "success"
|
||||
|
||||
def test_callback_reports_success(self, api_v3_client, api_v3_module, queued):
|
||||
@@ -196,8 +194,7 @@ class TestInstallPathsAgree:
|
||||
|
||||
# Reset and re-run through the queue.
|
||||
for mock in (api_v3_module.api_v3.schema_manager,
|
||||
api_v3_module.api_v3.plugin_manager,
|
||||
api_v3_module.api_v3.plugin_state_manager,
|
||||
api_v3_module.api_v3.plugin_catalog,
|
||||
api_v3_module.api_v3.operation_history):
|
||||
mock.reset_mock()
|
||||
queue = MagicMock()
|
||||
@@ -208,8 +205,6 @@ class TestInstallPathsAgree:
|
||||
|
||||
assert direct["schema_invalidated"] == queued["schema_invalidated"]
|
||||
assert direct["discovered"] == queued["discovered"]
|
||||
assert direct["loaded"] == queued["loaded"]
|
||||
assert direct["state_set"] == queued["state_set"]
|
||||
assert (direct["history"][0].kwargs["status"]
|
||||
== queued["history"][0].kwargs["status"])
|
||||
assert (direct["history"][0].kwargs["details"]
|
||||
@@ -260,21 +255,22 @@ class TestInstallFromUrl:
|
||||
branch="dev",
|
||||
)
|
||||
|
||||
def test_success_invalidates_schema_and_loads_plugin(self, api_v3_client, api_v3_module):
|
||||
def test_success_invalidates_schema_and_lists_plugin(self, api_v3_client, api_v3_module):
|
||||
api_v3_module.api_v3.plugin_store_manager.install_from_url.return_value = {
|
||||
"success": True, "plugin_id": "clock"}
|
||||
api_v3_client.post(FROM_URL, json={"repo_url": "http://x"})
|
||||
response = api_v3_client.post(FROM_URL, json={"repo_url": "http://x"})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
api_v3_module.api_v3.schema_manager.invalidate_cache.assert_called_once_with("clock")
|
||||
api_v3_module.api_v3.plugin_manager.load_plugin.assert_called_once_with("clock")
|
||||
api_v3_module.api_v3.plugin_catalog.discover_plugins.assert_called_once_with()
|
||||
|
||||
def test_success_without_plugin_id_skips_discovery(self, api_v3_client, api_v3_module):
|
||||
# install_from_url can succeed without naming the plugin; there is
|
||||
# then nothing to invalidate or load.
|
||||
# then nothing to invalidate or list.
|
||||
api_v3_module.api_v3.plugin_store_manager.install_from_url.return_value = {
|
||||
"success": True, "plugin_id": None}
|
||||
api_v3_client.post(FROM_URL, json={"repo_url": "http://x"})
|
||||
api_v3_module.api_v3.schema_manager.invalidate_cache.assert_not_called()
|
||||
api_v3_module.api_v3.plugin_manager.load_plugin.assert_not_called()
|
||||
api_v3_module.api_v3.plugin_catalog.discover_plugins.assert_not_called()
|
||||
|
||||
def test_branch_from_result_included(self, api_v3_client, api_v3_module):
|
||||
api_v3_module.api_v3.plugin_store_manager.install_from_url.return_value = {
|
||||
|
||||
@@ -110,7 +110,7 @@ class TestCalendarCredentials:
|
||||
def plugin_dir(self, tmp_path, api_v3_module):
|
||||
directory = tmp_path / "plugins" / "calendar"
|
||||
directory.mkdir(parents=True)
|
||||
api_v3_module.api_v3.plugin_manager.get_plugin_directory.return_value = str(directory)
|
||||
api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = str(directory)
|
||||
return directory
|
||||
|
||||
def _post(self, client):
|
||||
|
||||
@@ -793,7 +793,7 @@ def api_client(monkeypatch):
|
||||
cm.get_raw_file_content.return_value = {}
|
||||
cm.save_config_atomic.return_value = MagicMock(status=MagicMock(value='success'), message=None)
|
||||
api_v3.config_manager = cm
|
||||
api_v3.plugin_manager = MagicMock(plugins={})
|
||||
api_v3.plugin_catalog = MagicMock()
|
||||
# Never restart a real display from a test run.
|
||||
setup_calls = []
|
||||
monkeypatch.setattr(au, 'start_setup_if_needed',
|
||||
|
||||
@@ -59,10 +59,15 @@ class FakeHost:
|
||||
|
||||
Services run whatever commit was checked out when they were last
|
||||
restarted; ``failure`` says how they misbehave on the new commit
|
||||
("display_down", "web_down", "crash_loop") or on any commit ("always").
|
||||
("display_down", "web_down", "crash_loop", "frozen": active but the
|
||||
render loop stuck after its first frame) or on any commit ("always").
|
||||
|
||||
``heartbeat`` is whether the display writes one: never (``None``, code
|
||||
from before the heartbeat), or on every commit (``"always"``).
|
||||
"""
|
||||
|
||||
def __init__(self, repo, bad_head, failure=None, pip_ok=True, restart_failures=0, count_readable=True):
|
||||
def __init__(self, repo, bad_head, failure=None, pip_ok=True, restart_failures=0, count_readable=True,
|
||||
heartbeat=None):
|
||||
self.repo, self.bad_head, self.failure, self.pip_ok = repo, bad_head, failure, pip_ok
|
||||
self.restart_failures = restart_failures # how many restart commands fail, first to last
|
||||
self.count_readable = count_readable
|
||||
@@ -73,6 +78,8 @@ class FakeHost:
|
||||
self.pip = None # optional (args, host) -> result, or raises, instead of pip_ok
|
||||
self.now = 0.0
|
||||
self.nrestarts = 0
|
||||
self.heartbeat = heartbeat
|
||||
self.display_started_at = -1000.0 # the pre-update display, long running
|
||||
|
||||
def broken(self, kind):
|
||||
if self.running_head is None:
|
||||
@@ -102,28 +109,43 @@ class FakeHost:
|
||||
return done(args, rc=1) # the old process keeps running
|
||||
self.running_head = git(self.repo, 'rev-parse', 'HEAD')
|
||||
self.restarts.append((args[4], self.running_head))
|
||||
if args[4] == 'ledmatrix.service':
|
||||
self.display_started_at = self.now
|
||||
return done(args)
|
||||
raise AssertionError(f'unexpected command: {args}')
|
||||
|
||||
def web_responds(self):
|
||||
return not self.broken('web_down')
|
||||
|
||||
def read_heartbeat(self):
|
||||
if self.heartbeat is None:
|
||||
return None
|
||||
first_frame = self.display_started_at + 10 # plugins load, then it draws
|
||||
if self.now < first_frame:
|
||||
# Nothing from this process yet. A display whose unit predates
|
||||
# RuntimeDirectory= leaves its predecessor's file behind.
|
||||
return {'mono': self.display_started_at - 1}
|
||||
if self.broken('frozen'):
|
||||
return {'mono': first_frame} # drew once, then stuck
|
||||
return {'mono': self.now}
|
||||
|
||||
def sleep(self, seconds):
|
||||
self.now += seconds
|
||||
|
||||
def verifier(self):
|
||||
return av.Verifier(self.repo, run=self.run, sleep=self.sleep, clock=lambda: self.now,
|
||||
web_responds=self.web_responds, log=lambda msg: None)
|
||||
web_responds=self.web_responds, log=lambda msg: None,
|
||||
read_heartbeat=self.read_heartbeat)
|
||||
|
||||
|
||||
def check(tmp_path, failure=None, new_requirements=False, pip_ok=True, restart_failures=0,
|
||||
count_readable=True, **pending):
|
||||
count_readable=True, heartbeat=None, **pending):
|
||||
repo, old, new = updated_repo(tmp_path, new_requirements)
|
||||
fields = {'status': 'pending', 'old_head': old, 'new_head': new,
|
||||
'display_was_active': True, 'dependency_failures': []}
|
||||
fields.update(pending)
|
||||
av.write_pending(av.pending_path(repo), fields)
|
||||
host = FakeHost(repo, new, failure, pip_ok, restart_failures, count_readable)
|
||||
host = FakeHost(repo, new, failure, pip_ok, restart_failures, count_readable, heartbeat)
|
||||
code = host.verifier().verify()
|
||||
result = av.read_pending(av.pending_path(repo))
|
||||
return code, result, host, git(repo, 'rev-parse', 'HEAD'), old, new
|
||||
@@ -323,3 +345,63 @@ def test_units_installers_and_updater_agree():
|
||||
for sudoers in ('scripts/install/configure_web_sudo.sh', 'first_time_install.sh',
|
||||
'scripts/install/lib_sudoers.sh'):
|
||||
assert not re.search(r'NOPASSWD:.*update-verify', (ROOT / sudoers).read_text(encoding='utf-8')), sudoers
|
||||
|
||||
|
||||
# -- the display's heartbeat -------------------------------------------------------
|
||||
#
|
||||
# "Service active" plus one HTTP 200 passed a panel frozen by a render loop
|
||||
# stuck in a plugin. Where the display writes a heartbeat, the restarted
|
||||
# display has to keep it fresh too.
|
||||
|
||||
FROZEN_REASON = ('the display service is running but its panel is not being drawn '
|
||||
'(no fresh heartbeat)')
|
||||
|
||||
|
||||
def test_a_display_that_keeps_drawing_passes(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, heartbeat='always')
|
||||
assert result['status'] == 'success' and head == new
|
||||
|
||||
|
||||
def test_a_frozen_panel_is_rolled_back(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, 'frozen', heartbeat='always')
|
||||
assert result['status'] == 'rolled_back' and result['reason'] == FROZEN_REASON
|
||||
assert head == old
|
||||
|
||||
|
||||
def test_the_previous_processs_heartbeat_does_not_count(tmp_path):
|
||||
"""Under a unit without RuntimeDirectory= the old file outlives the old
|
||||
process; a restarted display that never draws must not pass on it."""
|
||||
repo, old, new = updated_repo(tmp_path)
|
||||
host = FakeHost(repo, new, heartbeat='always')
|
||||
verifier = host.verifier()
|
||||
verifier.expect_heartbeat = True
|
||||
host.display_started_at = host.now = 100.0
|
||||
verifier.display_restarted_at = 100.0
|
||||
host.now = 101.0 # the new process has not drawn yet
|
||||
assert verifier.display_drawing() is False
|
||||
host.now = 115.0
|
||||
assert verifier.display_drawing() is True
|
||||
|
||||
|
||||
def test_without_a_heartbeat_the_check_is_what_it_was(tmp_path):
|
||||
"""Code from before the heartbeat (or a display that cannot write one)
|
||||
never wrote one, so it cannot be asked for -- a frozen panel then passes,
|
||||
exactly as it did."""
|
||||
code, result, host, head, old, new = check(tmp_path, 'frozen', heartbeat=None)
|
||||
assert result['status'] == 'success'
|
||||
|
||||
|
||||
def test_a_stopped_display_is_not_asked_for_a_heartbeat(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, 'frozen', heartbeat='always',
|
||||
display_was_active=False)
|
||||
assert result['status'] == 'success'
|
||||
|
||||
|
||||
def test_the_heartbeat_location_and_freshness_match_the_display():
|
||||
"""A copy, not an import: the verifier must not depend on the code it checks."""
|
||||
from src import display_watchdog
|
||||
assert av.HEARTBEAT_PATH == display_watchdog.HEARTBEAT_PATH
|
||||
# A display frozen right after its first frame must go stale inside the
|
||||
# window it has to stay healthy for.
|
||||
assert av.HEARTBEAT_FRESH_SECONDS + av.POLL_SECONDS < av.STABLE_SECONDS
|
||||
assert av.HEARTBEAT_FRESH_SECONDS > display_watchdog.BEAT_INTERVAL_SECONDS * 2
|
||||
|
||||
@@ -72,7 +72,8 @@ def _make_project(root: Path) -> Path:
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
# plugin_state.json
|
||||
# A plugin_state.json left behind by an older release. Retired: the
|
||||
# listing must ignore it (see test_list_installed_plugins).
|
||||
(root / "data").mkdir()
|
||||
(root / "data" / "plugin_state.json").write_text(
|
||||
json.dumps(
|
||||
@@ -130,12 +131,82 @@ def test_bundled_fonts_matches_repo() -> None:
|
||||
|
||||
|
||||
def test_list_installed_plugins(project: Path) -> None:
|
||||
"""Installed = a manifest on disk; enabled = config.json. The retired
|
||||
plugin_state.json is not read: its "other-plugin" is not installed and
|
||||
not configured, so a restore must not install it."""
|
||||
plugins = list_installed_plugins(project)
|
||||
ids = [p["plugin_id"] for p in plugins]
|
||||
assert "my-plugin" in ids
|
||||
assert "other-plugin" in ids
|
||||
my = next(p for p in plugins if p["plugin_id"] == "my-plugin")
|
||||
assert my["version"] == "1.2.3"
|
||||
assert plugins == [{"plugin_id": "my-plugin", "version": "1.2.3", "enabled": True}]
|
||||
|
||||
|
||||
def test_list_installed_plugins_reads_enabled_from_config(project: Path) -> None:
|
||||
"""The display's rule: only "enabled": true is enabled; a plugin with no
|
||||
config section, or no flag, is disabled."""
|
||||
for pid in ("quiet-plugin", "unconfigured-plugin"):
|
||||
d = project / "plugin-repos" / pid
|
||||
d.mkdir()
|
||||
(d / "manifest.json").write_text(json.dumps({"id": pid, "version": "2.0.0"}),
|
||||
encoding="utf-8")
|
||||
config_path = project / "config" / "config.json"
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
config["quiet-plugin"] = {"favorites": []}
|
||||
config_path.write_text(json.dumps(config), encoding="utf-8")
|
||||
|
||||
by_id = {p["plugin_id"]: p for p in list_installed_plugins(project)}
|
||||
|
||||
assert by_id["my-plugin"]["enabled"] is True
|
||||
assert by_id["quiet-plugin"]["enabled"] is False
|
||||
assert by_id["unconfigured-plugin"]["enabled"] is False
|
||||
assert by_id["quiet-plugin"]["version"] == "2.0.0"
|
||||
|
||||
|
||||
def test_list_installed_plugins_without_a_state_file(project: Path) -> None:
|
||||
"""Nothing depends on plugin_state.json being there."""
|
||||
(project / "data" / "plugin_state.json").unlink()
|
||||
assert [p["plugin_id"] for p in list_installed_plugins(project)] == ["my-plugin"]
|
||||
|
||||
|
||||
def test_backup_restore_round_trip_ignores_the_retired_state_file(
|
||||
project: Path, empty_project: Path, tmp_path: Path) -> None:
|
||||
"""A backup made on a device that still has plugin_state.json restores
|
||||
the installed plugins and their enabled state (from config.json), and
|
||||
carries no state file of its own."""
|
||||
zip_path = create_backup(project, output_dir=tmp_path / "exports")
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
names = set(zf.namelist())
|
||||
listed = json.loads(zf.read("plugins.json"))
|
||||
assert not any("plugin_state" in n for n in names)
|
||||
assert listed == [{"plugin_id": "my-plugin", "version": "1.2.3", "enabled": True}]
|
||||
|
||||
result = restore_backup(zip_path, empty_project, RestoreOptions())
|
||||
|
||||
assert result.success, result.errors
|
||||
assert result.plugins_to_install == [{"plugin_id": "my-plugin", "version": "1.2.3"}]
|
||||
restored = json.loads((empty_project / "config" / "config.json").read_text())
|
||||
assert restored["my-plugin"]["enabled"] is True
|
||||
assert not (empty_project / "data" / "plugin_state.json").exists()
|
||||
|
||||
|
||||
def test_restore_of_a_backup_listing_a_state_file_only_plugin(
|
||||
project: Path, empty_project: Path, tmp_path: Path) -> None:
|
||||
"""A backup written by an older release could list a plugin known only
|
||||
to plugin_state.json. Restore reads plugins.json as written, so such a
|
||||
backup still restores everything it lists."""
|
||||
zip_path = tmp_path / "old.zip"
|
||||
with zipfile.ZipFile(zip_path, "w") as zf:
|
||||
zf.writestr("manifest.json", json.dumps({
|
||||
"schema_version": 1, "created_at": "2026-01-01T00:00:00Z",
|
||||
"ledmatrix_version": "3.6.0", "hostname": "old",
|
||||
"contents": ["config", "plugins"]}))
|
||||
zf.writestr("config/config.json", json.dumps({"my-plugin": {"enabled": True}}))
|
||||
zf.writestr("plugins.json", json.dumps([
|
||||
{"plugin_id": "my-plugin", "version": "1.2.3", "enabled": True},
|
||||
{"plugin_id": "other-plugin", "version": "0.1.0", "enabled": False},
|
||||
]))
|
||||
|
||||
result = restore_backup(zip_path, empty_project, RestoreOptions())
|
||||
|
||||
assert result.success, result.errors
|
||||
assert {p["plugin_id"] for p in result.plugins_to_install} == {"my-plugin", "other-plugin"}
|
||||
|
||||
|
||||
def test_preview_backup_contents(project: Path) -> None:
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
"""scripts/build_css.py: the pinned Tailwind CLI and what it builds.
|
||||
|
||||
The build itself needs the CLI download, so CI runs it in its own job
|
||||
(`build_css.py --check`); these check the parts that must hold without it.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"build_css", PROJECT_ROOT / "scripts" / "build_css.py"
|
||||
)
|
||||
build_css = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(build_css)
|
||||
|
||||
|
||||
def test_every_asset_is_pinned_to_a_sha256():
|
||||
assert re.fullmatch(r"3\.\d+\.\d+", build_css.TAILWIND_VERSION)
|
||||
assert build_css.TAILWIND_ASSETS
|
||||
for name, digest in build_css.TAILWIND_ASSETS.items():
|
||||
assert name.startswith("tailwindcss-"), name
|
||||
assert re.fullmatch(r"[0-9a-f]{64}", digest), name
|
||||
|
||||
|
||||
@pytest.mark.parametrize("system,machine,expected", [
|
||||
("Linux", "x86_64", "tailwindcss-linux-x64"),
|
||||
("Linux", "aarch64", "tailwindcss-linux-arm64"),
|
||||
("Linux", "armv7l", "tailwindcss-linux-armv7"),
|
||||
("Darwin", "arm64", "tailwindcss-macos-arm64"),
|
||||
("Darwin", "x86_64", "tailwindcss-macos-x64"),
|
||||
("Windows", "AMD64", "tailwindcss-windows-x64.exe"),
|
||||
("Windows", "ARM64", "tailwindcss-windows-arm64.exe"),
|
||||
])
|
||||
def test_asset_name_maps_each_platform(monkeypatch, system, machine, expected):
|
||||
monkeypatch.setattr(build_css.platform, "system", lambda: system)
|
||||
monkeypatch.setattr(build_css.platform, "machine", lambda: machine)
|
||||
assert build_css.asset_name() == expected
|
||||
assert expected in build_css.TAILWIND_ASSETS
|
||||
|
||||
|
||||
def test_unsupported_cpu_is_a_clear_error(monkeypatch):
|
||||
monkeypatch.setattr(build_css.platform, "system", lambda: "Linux")
|
||||
monkeypatch.setattr(build_css.platform, "machine", lambda: "armv6l")
|
||||
with pytest.raises(SystemExit, match="armv6l"):
|
||||
build_css.asset_name()
|
||||
|
||||
|
||||
def test_every_build_input_exists_and_its_output_is_committed():
|
||||
for input_css, config, output in build_css.BUILDS:
|
||||
assert (PROJECT_ROOT / input_css).is_file(), input_css
|
||||
assert (PROJECT_ROOT / config).is_file(), config
|
||||
assert (PROJECT_ROOT / output).is_file(), output
|
||||
|
||||
|
||||
def test_the_cli_is_fed_an_lf_copy_of_a_crlf_input(tmp_path, monkeypatch):
|
||||
"""Tailwind's minifier merges rules differently when the input CSS has
|
||||
CRLF line endings, so a Windows checkout built bytes CI's Linux build
|
||||
didn't, and --check failed. The CLI must always see LF."""
|
||||
monkeypatch.setattr(build_css, "PROJECT_ROOT", tmp_path)
|
||||
(tmp_path / "in.css").write_bytes(b"@tailwind base;\r\n@tailwind utilities;\r\n")
|
||||
seen = {}
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
seen["input"] = Path(cmd[cmd.index("--input") + 1]).read_bytes()
|
||||
Path(cmd[cmd.index("--output") + 1]).write_text(".a{b:c}", encoding="utf-8")
|
||||
|
||||
class Done:
|
||||
returncode = 0
|
||||
stdout = stderr = ""
|
||||
return Done()
|
||||
|
||||
monkeypatch.setattr(build_css.subprocess, "run", fake_run)
|
||||
work = tmp_path / "work"
|
||||
work.mkdir()
|
||||
out = tmp_path / "out.css"
|
||||
build_css.run_build(Path("cli"), "in.css", "cfg.js", out, work)
|
||||
|
||||
assert seen["input"] == b"@tailwind base;\n@tailwind utilities;\n"
|
||||
assert out.read_bytes() == b".a{b:c}\n"
|
||||
assert (tmp_path / "in.css").read_bytes().count(b"\r\n") == 2 # source untouched
|
||||
|
||||
|
||||
def test_a_corrupt_cached_cli_is_replaced(tmp_path, monkeypatch):
|
||||
"""A cached binary that fails its hash is deleted and fetched again,
|
||||
and the fresh download is hash-checked too."""
|
||||
monkeypatch.setenv("LEDMATRIX_TAILWIND_CACHE", str(tmp_path))
|
||||
name = build_css.asset_name()
|
||||
cached = tmp_path / f"v{build_css.TAILWIND_VERSION}" / name
|
||||
cached.parent.mkdir(parents=True)
|
||||
cached.write_bytes(b"not the cli")
|
||||
|
||||
class FakeResponse:
|
||||
def __init__(self, data):
|
||||
self.data = data
|
||||
|
||||
def read(self, n=-1):
|
||||
data, self.data = self.data, b""
|
||||
return data
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc):
|
||||
return False
|
||||
|
||||
monkeypatch.setattr(build_css.urllib.request, "urlopen",
|
||||
lambda url, timeout: FakeResponse(b"tampered"))
|
||||
with pytest.raises(SystemExit, match="SHA-256 mismatch"):
|
||||
build_css.ensure_cli()
|
||||
assert not cached.exists()
|
||||
assert not any(p.name.startswith(".download-") for p in cached.parent.iterdir())
|
||||
|
||||
|
||||
def test_the_cli_is_only_downloaded_over_https(tmp_path, monkeypatch):
|
||||
"""urlopen would also follow file:// and custom schemes; the download
|
||||
refuses anything but https before it opens the URL."""
|
||||
monkeypatch.setenv("LEDMATRIX_TAILWIND_CACHE", str(tmp_path))
|
||||
monkeypatch.setattr(build_css, "DOWNLOAD_URL", "file:///etc/{version}/{asset}")
|
||||
|
||||
def fail(*args, **kwargs):
|
||||
raise AssertionError("urlopen must not be called for a non-https URL")
|
||||
|
||||
monkeypatch.setattr(build_css.urllib.request, "urlopen", fail)
|
||||
with pytest.raises(SystemExit, match="non-https"):
|
||||
build_css.ensure_cli()
|
||||
@@ -97,14 +97,8 @@ def _reconcile(tmp_path, config, installed=(), secrets=None):
|
||||
_install(plugins_dir, pid)
|
||||
secrets_path = tmp_path / "config_secrets.json"
|
||||
secrets_path.write_text(json.dumps(secrets or {}), encoding="utf-8")
|
||||
state_manager = Mock()
|
||||
state_manager.get_all_states.return_value = {}
|
||||
plugin_manager = Mock()
|
||||
plugin_manager.plugin_manifests = {}
|
||||
reconciler = StateReconciliation(
|
||||
state_manager=state_manager,
|
||||
config_manager=_ConfigManager(config, str(secrets_path)),
|
||||
plugin_manager=plugin_manager,
|
||||
plugins_dir=plugins_dir,
|
||||
)
|
||||
return reconciler, reconciler.reconcile_state()
|
||||
@@ -241,7 +235,7 @@ class TestTheStatusEndpoint:
|
||||
pm = MagicMock()
|
||||
pm.plugins_dir = str(plugins_dir)
|
||||
monkeypatch.setattr(api_v3, "config_manager", cm, raising=False)
|
||||
monkeypatch.setattr(api_v3, "plugin_manager", pm, raising=False)
|
||||
monkeypatch.setattr(api_v3, "plugin_catalog", pm, raising=False)
|
||||
app = Flask(__name__)
|
||||
app.config["TESTING"] = True
|
||||
app.register_blueprint(api_v3, url_prefix="/api/v3")
|
||||
|
||||
+150
-6
@@ -1,19 +1,29 @@
|
||||
"""@deprecated: plugin-facing APIs nothing in core, the monorepo or the
|
||||
registry's third-party plugins calls, kept for one release with a warning."""
|
||||
|
||||
import ast
|
||||
import importlib.util
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import textwrap
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from packaging.version import Version
|
||||
|
||||
os.environ.setdefault("EMULATOR", "true")
|
||||
|
||||
from src import deprecation
|
||||
from src import __version__, deprecation
|
||||
from src.deprecation import deprecated
|
||||
|
||||
#: Everything deprecated for removal in 3.7.0. Removing one of these, or
|
||||
#: deprecating another, should be a deliberate edit here too.
|
||||
REPO = Path(__file__).resolve().parents[1]
|
||||
|
||||
#: Everything deprecated for removal in 3.8.0 (first announced for 3.7.0,
|
||||
#: which shipped with all of them still in place). docs/DEPRECATIONS_3.8.md
|
||||
#: says which are unused. Removing one of these, or deprecating another,
|
||||
#: should be a deliberate edit here too.
|
||||
DEPRECATED = {
|
||||
"src.cache_manager.CacheManager": [
|
||||
"has_data_changed", "update_cache", "setup_persistent_cache",
|
||||
@@ -35,6 +45,19 @@ DEPRECATED = {
|
||||
"src.plugin_system.plugin_manager.PluginManager": ["get_enabled_plugins"],
|
||||
}
|
||||
|
||||
#: Deprecated with Vegas participation, for removal in 3.9.0: core never read
|
||||
#: them (src.plugin_system.base_plugin.VEGAS_LEGACY_REMOVAL).
|
||||
DEPRECATED_3_9 = {
|
||||
"src.plugin_system.base_plugin.BasePlugin": [
|
||||
"get_supported_vegas_modes", "get_vegas_segment_width",
|
||||
],
|
||||
}
|
||||
|
||||
#: Every pinned marker: (class path, method) -> the release that removes it.
|
||||
PINNED = {(path, name): removal
|
||||
for removal, table in (("3.8.0", DEPRECATED), ("3.9.0", DEPRECATED_3_9))
|
||||
for path, names in table.items() for name in names}
|
||||
|
||||
|
||||
def _cls(path):
|
||||
import importlib
|
||||
@@ -42,14 +65,135 @@ def _cls(path):
|
||||
return getattr(importlib.import_module(module), name)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("path", sorted(DEPRECATED))
|
||||
@pytest.mark.parametrize("path", sorted({path for path, _ in PINNED}))
|
||||
def test_exactly_these_methods_are_deprecated(path):
|
||||
cls = _cls(path)
|
||||
marked = sorted(name for name, value in vars(cls).items()
|
||||
if hasattr(value, "__deprecated__"))
|
||||
assert marked == sorted(DEPRECATED[path])
|
||||
assert marked == sorted(name for owner, name in PINNED if owner == path)
|
||||
for name in marked:
|
||||
assert "3.7.0" in getattr(cls, name).__deprecated__
|
||||
assert f"LEDMatrix {PINNED[(path, name)]}" in getattr(cls, name).__deprecated__
|
||||
|
||||
|
||||
def _markers():
|
||||
"""(file:line, removal) for every ``@deprecated(...)`` under src/."""
|
||||
found = []
|
||||
for path in sorted((REPO / "src").rglob("*.py")):
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), str(path))
|
||||
for fn in ast.walk(tree):
|
||||
if not isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
continue
|
||||
for dec in fn.decorator_list:
|
||||
target = dec.func if isinstance(dec, ast.Call) else dec
|
||||
name = getattr(target, "id", None) or getattr(target, "attr", None)
|
||||
if name != "deprecated":
|
||||
continue
|
||||
where = f"{path.relative_to(REPO).as_posix()}:{fn.lineno} {fn.name}"
|
||||
arg = dec.args[0] if isinstance(dec, ast.Call) and dec.args else None
|
||||
found.append((where, arg.value if isinstance(arg, ast.Constant) else None))
|
||||
return found
|
||||
|
||||
|
||||
def test_markers_are_found():
|
||||
assert len(_markers()) == len(PINNED)
|
||||
|
||||
|
||||
def test_no_marker_names_a_release_already_shipped():
|
||||
"""3.7.0 shipped still warning that 35 methods are "removed in 3.7.0".
|
||||
|
||||
Once ``src.__version__`` reaches a marker's release, that release is here:
|
||||
remove the method (if scripts/plugin_api_usage.py reports it unused) or
|
||||
move the marker to a later release. Either way, never ship a warning that
|
||||
names a version the user is already running.
|
||||
"""
|
||||
current = Version(__version__)
|
||||
stale = [f"{where} -> {removal!r}" for where, removal in _markers()
|
||||
if not isinstance(removal, str) or Version(removal) <= current]
|
||||
assert not stale, (f"@deprecated markers at or below src.__version__ {__version__} "
|
||||
f"(or not a literal version): {stale}")
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def usage_script():
|
||||
path = REPO / "scripts" / "plugin_api_usage.py"
|
||||
spec = importlib.util.spec_from_file_location("plugin_api_usage_script", path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module # dataclasses resolve annotations through it
|
||||
try:
|
||||
spec.loader.exec_module(module)
|
||||
yield module
|
||||
finally:
|
||||
sys.modules.pop(spec.name, None)
|
||||
|
||||
|
||||
def test_usage_script_lists_exactly_the_pinned_markers(usage_script):
|
||||
found = {(f"{m.module}.{m.owner}", m.method, m.removal)
|
||||
for m in usage_script.find_markers(REPO)}
|
||||
assert found == {(path, name, removal) for (path, name), removal in PINNED.items()}
|
||||
|
||||
|
||||
def test_usage_script_tells_uses_from_name_collisions(usage_script, tmp_path):
|
||||
"""Calls on the owning object and overrides count; same-named methods of
|
||||
unrelated classes and hits in test files do not."""
|
||||
plugin = tmp_path / "demo"
|
||||
(plugin / "test").mkdir(parents=True)
|
||||
(plugin / "manager.py").write_text(textwrap.dedent("""\
|
||||
class Icons:
|
||||
@staticmethod
|
||||
def draw_sun(img):
|
||||
pass
|
||||
|
||||
class Plugin:
|
||||
def draw_cloud(self):
|
||||
return self.draw_cloud()
|
||||
|
||||
def display(self):
|
||||
Icons.draw_sun(None)
|
||||
self.cache_manager.update_cache('k', {})
|
||||
dm = self.display_manager
|
||||
dm.draw_rain(0, 0)
|
||||
thing.get_scrolling_stats()
|
||||
|
||||
class MyFonts(FontManager):
|
||||
def add_font(self, path, name):
|
||||
return super().add_font(path, name)
|
||||
"""), encoding="utf-8")
|
||||
(plugin / "test" / "test_manager.py").write_text(textwrap.dedent("""\
|
||||
def test_x(display_manager):
|
||||
display_manager.draw_snow.assert_not_called()
|
||||
"""), encoding="utf-8")
|
||||
|
||||
markers = usage_script.find_markers(REPO)
|
||||
source = usage_script.Source("demo", "monorepo", plugin)
|
||||
usage_script.scan_tree(source, [plugin], plugin, markers, core=False)
|
||||
kinds = {key: sorted(("test " if h.test else "") + h.kind for h in hits)
|
||||
for key, hits in source.hits.items()}
|
||||
|
||||
assert kinds == {
|
||||
"DisplayManager.draw_sun": ["unrelated", "unrelated"],
|
||||
"DisplayManager.draw_cloud": ["unrelated", "unrelated"],
|
||||
"CacheManager.update_cache": ["call"],
|
||||
"DisplayManager.draw_rain": ["call"],
|
||||
"DisplayManager.get_scrolling_stats": ["review"],
|
||||
"FontManager.add_font": ["call", "override"],
|
||||
"DisplayManager.draw_snow": ["test call"],
|
||||
}
|
||||
|
||||
status = usage_script.verdicts(markers, [source])
|
||||
assert status["CacheManager.update_cache"][0] == "used"
|
||||
assert status["DisplayManager.get_scrolling_stats"][0] == "review"
|
||||
assert status["DisplayManager.draw_sun"][0] == "unused" # a collision only
|
||||
assert status["DisplayManager.draw_snow"][0] == "unused" # a test mock only
|
||||
|
||||
|
||||
def test_usage_script_follows_calls_between_deprecated_core_methods(usage_script):
|
||||
"""draw_rain calls draw_cloud; with no outside callers both are unused."""
|
||||
markers = usage_script.find_markers(REPO)
|
||||
core = usage_script.Source("core", "core", REPO)
|
||||
usage_script.scan_tree(core, [REPO / "src" / "display_manager.py"], REPO, markers, core=True)
|
||||
kinds = {h.kind for h in core.hits["DisplayManager.draw_cloud"]}
|
||||
assert kinds == {"internal"}
|
||||
assert usage_script.verdicts(markers, [core])["DisplayManager.draw_cloud"][0] == "unused"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
|
||||
@@ -0,0 +1,660 @@
|
||||
"""The render loop's liveness signals: systemd watchdog pings and the heartbeat.
|
||||
|
||||
A panel can freeze while ledmatrix.service stays "active" -- a render thread
|
||||
stuck inside a plugin's display(). src/display_watchdog.py lets only the
|
||||
render thread vouch for itself, to systemd (sd_notify WATCHDOG=1) and to the
|
||||
web interface (a heartbeat file under /run/ledmatrix). These tests pin:
|
||||
|
||||
* the sd_notify wire format, including abstract-namespace sockets;
|
||||
* that nothing is armed until the first frame, so start-up keeps its allowance;
|
||||
* that beats from any other thread are ignored, so a stuck render thread
|
||||
goes quiet even while the update worker and Vegas's tick thread carry on;
|
||||
* the places the render loop checks in from (dwell sleeps, per-frame
|
||||
display, Vegas's own loop, a plugin load's longer allowance).
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
os.environ.setdefault("EMULATOR", "true") # display_controller imports without hardware
|
||||
|
||||
from src import display_watchdog # noqa: E402
|
||||
from src.display_watchdog import ( # noqa: E402
|
||||
RenderWatchdog, heartbeat_age, notify, read_heartbeat, watchdog_usec)
|
||||
|
||||
WATCHDOG_120 = {'NOTIFY_SOCKET': '/run/systemd/notify', 'WATCHDOG_USEC': '120000000'}
|
||||
|
||||
|
||||
class FakeSocket:
|
||||
"""Records what notify() does with the socket it creates."""
|
||||
|
||||
def __init__(self, record, fail_connect=False):
|
||||
self.record = record
|
||||
self.fail_connect = fail_connect
|
||||
self.closed = False
|
||||
|
||||
def connect(self, address):
|
||||
self.record['address'] = address
|
||||
if self.fail_connect:
|
||||
raise ConnectionRefusedError('nobody listening')
|
||||
|
||||
def sendall(self, data):
|
||||
self.record.setdefault('sent', []).append(data)
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
self.record['closed'] = True
|
||||
|
||||
|
||||
def fake_factory(record, fail_connect=False):
|
||||
def factory(family, kind):
|
||||
record['family'], record['type'] = family, kind
|
||||
return FakeSocket(record, fail_connect)
|
||||
return factory
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def af_unix(monkeypatch):
|
||||
"""AF_UNIX for the fake-socket tests, even on a Python built without it."""
|
||||
monkeypatch.setattr(socket, 'AF_UNIX', getattr(socket, 'AF_UNIX', 1), raising=False)
|
||||
return socket.AF_UNIX
|
||||
|
||||
|
||||
# -- sd_notify -------------------------------------------------------------
|
||||
|
||||
class TestNotify:
|
||||
def test_sends_one_datagram_to_the_socket_path(self, af_unix):
|
||||
record = {}
|
||||
assert notify('WATCHDOG=1', {'NOTIFY_SOCKET': '/run/systemd/notify'},
|
||||
socket_factory=fake_factory(record)) is True
|
||||
assert record['family'] == af_unix
|
||||
assert record['type'] & socket.SOCK_DGRAM == socket.SOCK_DGRAM
|
||||
assert record['address'] == '/run/systemd/notify'
|
||||
assert record['sent'] == [b'WATCHDOG=1']
|
||||
assert record['closed']
|
||||
|
||||
def test_an_at_sign_means_the_abstract_namespace(self, af_unix):
|
||||
record = {}
|
||||
notify('READY=1\nSTATUS=Rendering', {'NOTIFY_SOCKET': '@/org/freedesktop/systemd1/notify'},
|
||||
socket_factory=fake_factory(record))
|
||||
assert record['address'] == '\0/org/freedesktop/systemd1/notify'
|
||||
assert record['sent'] == [b'READY=1\nSTATUS=Rendering']
|
||||
|
||||
@pytest.mark.parametrize('address', [None, '', 'relative/path', 'vsock:2:1234'])
|
||||
def test_no_usable_socket_sends_nothing(self, af_unix, address):
|
||||
record = {}
|
||||
env = {} if address is None else {'NOTIFY_SOCKET': address}
|
||||
assert notify('WATCHDOG=1', env, socket_factory=fake_factory(record)) is False
|
||||
assert record == {}
|
||||
|
||||
def test_a_failed_send_is_false_not_an_exception(self, af_unix):
|
||||
record = {}
|
||||
assert notify('WATCHDOG=1', {'NOTIFY_SOCKET': '/nope'},
|
||||
socket_factory=fake_factory(record, fail_connect=True)) is False
|
||||
assert record['closed']
|
||||
|
||||
@pytest.mark.skipif(not hasattr(socket, 'AF_UNIX') or os.name != 'posix',
|
||||
reason='needs AF_UNIX datagram sockets')
|
||||
def test_a_real_socket_receives_the_message(self, tmp_path):
|
||||
path = str(tmp_path / 'notify')
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||
try:
|
||||
server.bind(path)
|
||||
server.settimeout(2)
|
||||
assert notify('WATCHDOG=1', {'NOTIFY_SOCKET': path}) is True
|
||||
assert server.recv(4096) == b'WATCHDOG=1'
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
@pytest.mark.skipif(not sys.platform.startswith('linux'),
|
||||
reason='abstract sockets are Linux-only')
|
||||
def test_a_real_abstract_socket_receives_the_message(self):
|
||||
name = f'ledmatrix-test-{os.getpid()}-{time.monotonic_ns()}'
|
||||
server = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||
try:
|
||||
server.bind('\0' + name)
|
||||
server.settimeout(2)
|
||||
assert notify('READY=1', {'NOTIFY_SOCKET': '@' + name}) is True
|
||||
assert server.recv(4096) == b'READY=1'
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
|
||||
class TestWatchdogUsec:
|
||||
def test_reads_the_units_value(self):
|
||||
assert watchdog_usec({'WATCHDOG_USEC': '120000000'}) == 120_000_000
|
||||
|
||||
def test_meant_for_another_process(self):
|
||||
assert watchdog_usec({'WATCHDOG_USEC': '120000000',
|
||||
'WATCHDOG_PID': str(os.getpid() + 1)}) is None
|
||||
|
||||
def test_meant_for_this_process(self):
|
||||
assert watchdog_usec({'WATCHDOG_USEC': '5000000',
|
||||
'WATCHDOG_PID': str(os.getpid())}) == 5_000_000
|
||||
|
||||
@pytest.mark.parametrize('value', [None, '', 'abc', '0', '-5'])
|
||||
def test_no_watchdog(self, value):
|
||||
env = {} if value is None else {'WATCHDOG_USEC': value}
|
||||
assert watchdog_usec(env) is None
|
||||
|
||||
|
||||
# -- the render loop's side ----------------------------------------------------
|
||||
|
||||
class Clock:
|
||||
def __init__(self, now=1000.0):
|
||||
self.now = now
|
||||
|
||||
def __call__(self):
|
||||
return self.now
|
||||
|
||||
|
||||
def make(environ=WATCHDOG_120, heartbeat_dir=None, clock=None):
|
||||
sent = []
|
||||
wd = RenderWatchdog(environ=environ, send=lambda m: sent.append(m) or True,
|
||||
clock=clock or Clock(), wall_clock=lambda: 1_700_000_000.0,
|
||||
heartbeat_dir=heartbeat_dir, enable_faulthandler=False)
|
||||
return wd, sent
|
||||
|
||||
|
||||
def pings(sent):
|
||||
return [m for m in sent if 'WATCHDOG=1' in m.split('\n')]
|
||||
|
||||
|
||||
def on_other_thread(fn):
|
||||
t = threading.Thread(target=fn)
|
||||
t.start()
|
||||
t.join()
|
||||
|
||||
|
||||
class TestStartup:
|
||||
def test_start_up_widens_the_limit(self):
|
||||
wd, sent = make()
|
||||
wd.begin_startup()
|
||||
assert sent == [f'WATCHDOG_USEC={int(display_watchdog.STARTUP_ALLOWANCE_SECONDS * 1e6)}'
|
||||
'\nSTATUS=Starting: loading plugins']
|
||||
|
||||
def test_start_up_never_shortens_a_longer_unit_limit(self):
|
||||
wd, sent = make({'NOTIFY_SOCKET': '/x', 'WATCHDOG_USEC': str(3600 * 10**6)})
|
||||
wd.begin_startup()
|
||||
assert sent[0].startswith(f'WATCHDOG_USEC={3600 * 10**6}\n')
|
||||
|
||||
def test_without_a_watchdog_start_up_sends_nothing(self):
|
||||
wd, sent = make({'NOTIFY_SOCKET': '/x'})
|
||||
wd.begin_startup()
|
||||
assert sent == []
|
||||
|
||||
|
||||
class TestArming:
|
||||
def test_nothing_before_the_render_loop_starts(self, tmp_path):
|
||||
wd, sent = make(heartbeat_dir=str(tmp_path))
|
||||
wd.note_frame() # a start-up screen
|
||||
wd.beat()
|
||||
wd.loop_pass()
|
||||
assert sent == [] and not wd.armed
|
||||
assert not (tmp_path / display_watchdog.HEARTBEAT_NAME).exists()
|
||||
|
||||
def test_nothing_before_the_first_frame(self, tmp_path):
|
||||
wd, sent = make(heartbeat_dir=str(tmp_path))
|
||||
wd.bind_render_thread()
|
||||
wd.loop_pass() # the first pass has begun...
|
||||
wd.beat() # ...and is, say, composing Vegas content
|
||||
assert sent == [] and not wd.armed
|
||||
assert not (tmp_path / display_watchdog.HEARTBEAT_NAME).exists()
|
||||
|
||||
def test_the_first_frame_arms_it(self, tmp_path):
|
||||
wd, sent = make(heartbeat_dir=str(tmp_path))
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
assert wd.armed
|
||||
assert sent == ['READY=1\nWATCHDOG_USEC=120000000\nWATCHDOG=1\nSTATUS=Rendering']
|
||||
heartbeat = json.loads((tmp_path / display_watchdog.HEARTBEAT_NAME).read_text())
|
||||
assert heartbeat == {'pid': os.getpid(), 'mono': 1000.0, 'wall': 1_700_000_000.0}
|
||||
|
||||
def test_a_first_frame_pushed_by_another_thread_arms_on_the_next_render_beat(self):
|
||||
"""A screen's first display() runs on PluginExecutor's thread."""
|
||||
wd, sent = make()
|
||||
wd.bind_render_thread()
|
||||
on_other_thread(wd.note_frame)
|
||||
assert not wd.armed and sent == []
|
||||
wd.beat()
|
||||
assert wd.armed and sent[0].startswith('READY=1\n')
|
||||
|
||||
def test_a_full_pass_with_nothing_drawn_arms_it(self):
|
||||
"""No plugins enabled, or every screen empty: the loop is still alive."""
|
||||
wd, sent = make()
|
||||
wd.bind_render_thread()
|
||||
wd.loop_pass()
|
||||
assert not wd.armed
|
||||
wd.loop_pass()
|
||||
assert wd.armed and sent[0].startswith('READY=1\n')
|
||||
|
||||
def test_without_a_watchdog_it_still_says_ready_and_writes_the_heartbeat(self, tmp_path):
|
||||
wd, sent = make({'NOTIFY_SOCKET': '/x'}, heartbeat_dir=str(tmp_path))
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
assert sent == ['READY=1\nSTATUS=Rendering']
|
||||
assert (tmp_path / display_watchdog.HEARTBEAT_NAME).exists()
|
||||
sent_before = len(sent)
|
||||
wd.beat()
|
||||
assert len(sent) == sent_before # no WATCHDOG=1 without a watchdog
|
||||
|
||||
|
||||
class TestBeats:
|
||||
def test_pings_are_rate_limited(self, tmp_path):
|
||||
clock = Clock()
|
||||
wd, sent = make(heartbeat_dir=str(tmp_path), clock=clock)
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
del sent[:]
|
||||
for _ in range(100): # a burst of frames
|
||||
wd.beat()
|
||||
assert sent == []
|
||||
clock.now += display_watchdog.BEAT_INTERVAL_SECONDS
|
||||
wd.beat()
|
||||
assert sent == ['WATCHDOG=1']
|
||||
heartbeat = read_heartbeat(str(tmp_path / display_watchdog.HEARTBEAT_NAME))
|
||||
assert heartbeat['mono'] == clock.now
|
||||
|
||||
def test_a_short_unit_limit_pings_more_often(self):
|
||||
clock = Clock()
|
||||
wd, sent = make({'NOTIFY_SOCKET': '/x', 'WATCHDOG_USEC': '3000000'}, clock=clock)
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
del sent[:]
|
||||
clock.now += 1.0 # a third of 3s
|
||||
wd.beat()
|
||||
assert sent == ['WATCHDOG=1']
|
||||
|
||||
def test_beats_from_other_threads_are_ignored(self, tmp_path):
|
||||
"""The update worker, Vegas's tick thread and the prefetcher keep
|
||||
running while the render thread is stuck; they must not keep the
|
||||
watchdog fed on its behalf."""
|
||||
clock = Clock()
|
||||
wd, sent = make(heartbeat_dir=str(tmp_path), clock=clock)
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
del sent[:]
|
||||
before = (tmp_path / display_watchdog.HEARTBEAT_NAME).read_text()
|
||||
clock.now += 60
|
||||
on_other_thread(wd.beat)
|
||||
on_other_thread(wd.note_frame)
|
||||
on_other_thread(wd.loop_pass)
|
||||
assert sent == []
|
||||
assert (tmp_path / display_watchdog.HEARTBEAT_NAME).read_text() == before
|
||||
|
||||
def test_the_module_shortcuts_reach_the_process_instance(self, monkeypatch):
|
||||
wd, sent = make()
|
||||
monkeypatch.setattr(display_watchdog, 'watchdog', wd)
|
||||
wd.bind_render_thread()
|
||||
display_watchdog.note_frame()
|
||||
assert wd.armed
|
||||
with display_watchdog.extended(600, 'x'):
|
||||
pass
|
||||
display_watchdog.beat()
|
||||
assert any('WATCHDOG_USEC=600000000' in m for m in sent)
|
||||
|
||||
|
||||
class TestExtended:
|
||||
def test_a_long_job_gets_the_longer_limit_then_the_units_back(self):
|
||||
wd, sent = make()
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
del sent[:]
|
||||
with wd.extended(900, 'loading plugin weather'):
|
||||
assert sent == ['WATCHDOG_USEC=900000000\nWATCHDOG=1\nSTATUS=Busy: loading plugin weather']
|
||||
assert sent[-1] == 'WATCHDOG_USEC=120000000\nWATCHDOG=1\nSTATUS=Rendering'
|
||||
|
||||
def test_nested_jobs_restore_once(self):
|
||||
wd, sent = make()
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
del sent[:]
|
||||
with wd.extended(900):
|
||||
with wd.extended(900):
|
||||
pass
|
||||
assert len(sent) == 1 # the inner one neither re-extends nor restores
|
||||
assert len(sent) == 2 and sent[-1].startswith('WATCHDOG_USEC=120000000\n')
|
||||
|
||||
def test_restored_even_when_the_job_fails(self):
|
||||
wd, sent = make()
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
with pytest.raises(RuntimeError):
|
||||
with wd.extended(900):
|
||||
raise RuntimeError('pip failed')
|
||||
assert sent[-1].startswith('WATCHDOG_USEC=120000000\n')
|
||||
|
||||
def test_off_the_render_thread_or_before_arming_it_does_nothing(self):
|
||||
"""Start-up loads run on a thread pool under the start-up allowance."""
|
||||
wd, sent = make()
|
||||
wd.bind_render_thread()
|
||||
with wd.extended(900):
|
||||
pass
|
||||
wd.note_frame()
|
||||
del sent[:]
|
||||
on_other_thread(lambda: wd.extended(900).__enter__())
|
||||
assert sent == []
|
||||
|
||||
|
||||
class TestStopping:
|
||||
def test_a_clean_stop_removes_the_heartbeat(self, tmp_path):
|
||||
wd, sent = make(heartbeat_dir=str(tmp_path))
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
wd.stopping()
|
||||
assert sent[-1] == 'STOPPING=1'
|
||||
assert not (tmp_path / display_watchdog.HEARTBEAT_NAME).exists()
|
||||
|
||||
def test_stopping_outside_systemd_is_harmless(self):
|
||||
wd, sent = make({})
|
||||
wd.stopping()
|
||||
assert sent == []
|
||||
|
||||
|
||||
class TestHeartbeatFile:
|
||||
def test_the_directory_is_created_when_missing(self, tmp_path):
|
||||
"""An install whose unit predates RuntimeDirectory=; the display is root."""
|
||||
target = tmp_path / 'run' / 'ledmatrix'
|
||||
wd, _ = make(heartbeat_dir=str(target))
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
assert (target / display_watchdog.HEARTBEAT_NAME).is_file()
|
||||
assert [p.name for p in target.iterdir()] == [display_watchdog.HEARTBEAT_NAME]
|
||||
|
||||
@pytest.mark.skipif(os.name != 'posix', reason='POSIX permissions')
|
||||
def test_other_users_can_read_it(self, tmp_path):
|
||||
wd, _ = make(heartbeat_dir=str(tmp_path))
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
mode = (tmp_path / display_watchdog.HEARTBEAT_NAME).stat().st_mode & 0o777
|
||||
assert mode == 0o644
|
||||
|
||||
def test_nowhere_to_write_is_not_an_error(self, tmp_path):
|
||||
blocker = tmp_path / 'not-a-dir'
|
||||
blocker.write_text('x')
|
||||
clock = Clock()
|
||||
wd, sent = make(heartbeat_dir=str(blocker / 'ledmatrix'), clock=clock)
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
clock.now += 10
|
||||
wd.beat()
|
||||
assert wd.armed and pings(sent) # the watchdog works regardless
|
||||
|
||||
def test_windows_gets_no_heartbeat_by_default(self, monkeypatch):
|
||||
monkeypatch.setattr(display_watchdog.os, 'name', 'nt')
|
||||
wd = RenderWatchdog(environ={}, send=lambda m: True)
|
||||
assert wd._heartbeat_path() is None
|
||||
|
||||
|
||||
class TestHeartbeatAge:
|
||||
def test_monotonic_is_preferred(self):
|
||||
assert heartbeat_age({'mono': 100.0, 'wall': 0.0}, now_mono=112.5, now_wall=9e9) == 12.5
|
||||
|
||||
def test_the_wall_clock_is_the_fallback(self):
|
||||
assert heartbeat_age({'wall': 50.0}, now_mono=1.0, now_wall=80.0) == 30.0
|
||||
|
||||
def test_a_monotonic_stamp_from_the_future_is_not_trusted(self):
|
||||
"""Not the same clock -- fall back rather than report a fresh heartbeat."""
|
||||
assert heartbeat_age({'mono': 500.0, 'wall': 50.0}, now_mono=100.0, now_wall=170.0) == 120.0
|
||||
|
||||
def test_no_time_at_all(self):
|
||||
assert heartbeat_age({'pid': 1}) is None
|
||||
assert heartbeat_age({'mono': True}) is None
|
||||
|
||||
def test_reading_a_missing_or_broken_file(self, tmp_path):
|
||||
assert read_heartbeat(str(tmp_path / 'absent.json')) is None
|
||||
(tmp_path / 'broken.json').write_text('{not json')
|
||||
assert read_heartbeat(str(tmp_path / 'broken.json')) is None
|
||||
(tmp_path / 'list.json').write_text('[1, 2]')
|
||||
assert read_heartbeat(str(tmp_path / 'list.json')) is None
|
||||
|
||||
|
||||
# -- where the render loop checks in -------------------------------------------
|
||||
|
||||
@pytest.fixture
|
||||
def armed(monkeypatch):
|
||||
"""A process watchdog bound to this thread and armed, with a clock to advance."""
|
||||
clock = Clock()
|
||||
wd, sent = make(clock=clock)
|
||||
monkeypatch.setattr(display_watchdog, 'watchdog', wd)
|
||||
wd.bind_render_thread()
|
||||
wd.note_frame()
|
||||
del sent[:]
|
||||
return SimpleNamespace(wd=wd, sent=sent, clock=clock)
|
||||
|
||||
|
||||
def _tick_clock(armed):
|
||||
"""Advance the watchdog's clock past the rate limit on every beat check."""
|
||||
original = armed.wd._clock
|
||||
|
||||
def advancing():
|
||||
armed.clock.now += display_watchdog.BEAT_INTERVAL_SECONDS
|
||||
return original()
|
||||
armed.wd._clock = advancing
|
||||
|
||||
|
||||
class TestCheckInPoints:
|
||||
def test_the_dwell_sleep_checks_in(self, armed):
|
||||
from src.display_controller import DisplayController
|
||||
dc = object.__new__(DisplayController)
|
||||
dc.current_display_mode = 'm'
|
||||
dc.is_display_active = True
|
||||
dc.on_demand_active = False
|
||||
dc._tick_plugin_updates = lambda: None
|
||||
dc._service_pending_changes = lambda: None
|
||||
_tick_clock(armed)
|
||||
dc._sleep_with_plugin_updates(0.05, tick_interval=0.01)
|
||||
assert len(pings(armed.sent)) >= 3
|
||||
|
||||
def test_every_frame_of_a_screen_checks_in(self, armed):
|
||||
from src.display_controller import DisplayController
|
||||
dc = object.__new__(DisplayController)
|
||||
dc.plugin_manager = None
|
||||
plugin = MagicMock(plugin_id='p')
|
||||
_tick_clock(armed)
|
||||
for _ in range(3):
|
||||
dc._display_once(plugin, 'm', accepts_display_mode=False)
|
||||
assert len(pings(armed.sent)) == 3
|
||||
|
||||
def test_vegas_checks_in_every_frame_of_its_own_loop(self, armed):
|
||||
"""An iteration runs for minutes without returning to run()."""
|
||||
import threading as _threading
|
||||
from src.vegas_mode.config import VegasModeConfig
|
||||
from src.vegas_mode.coordinator import VegasModeCoordinator
|
||||
coord = VegasModeCoordinator.__new__(VegasModeCoordinator)
|
||||
coord.vegas_config = VegasModeConfig.from_config({'display': {'vegas_scroll': {
|
||||
'enabled': True, 'max_cycle_duration': 60}}})
|
||||
coord.render_pipeline = MagicMock(frame_interval=0.0, target_fps=90)
|
||||
coord.display_manager = MagicMock()
|
||||
coord._state_lock = _threading.Lock()
|
||||
coord._is_active = True
|
||||
coord._is_paused = False
|
||||
coord._should_stop = False
|
||||
coord._live_priority_active = False
|
||||
coord._fps_last_health_log = 0.0
|
||||
coord._fps_was_degraded = False
|
||||
coord._interrupt_check = None
|
||||
coord._interrupt_check_interval = 10
|
||||
coord._update_callback = None
|
||||
coord._update_tick_running = False
|
||||
coord._check_static_plugin_trigger = lambda: None
|
||||
frames = []
|
||||
|
||||
def run_frame():
|
||||
frames.append(1)
|
||||
if len(frames) == 5:
|
||||
coord._should_stop = True
|
||||
return False
|
||||
return True
|
||||
coord.run_frame = run_frame
|
||||
_tick_clock(armed)
|
||||
coord.run_iteration()
|
||||
assert len(pings(armed.sent)) == 5
|
||||
|
||||
def test_each_plugin_fetched_for_a_vegas_cycle_checks_in(self, armed):
|
||||
from src.vegas_mode.stream_manager import StreamManager
|
||||
sm = StreamManager.__new__(StreamManager)
|
||||
sm.plugin_manager = SimpleNamespace(plugins={})
|
||||
_tick_clock(armed)
|
||||
for plugin_id in ('a', 'b'):
|
||||
sm._fetch_plugin_content(plugin_id)
|
||||
assert len(pings(armed.sent)) == 2
|
||||
|
||||
def test_loading_a_plugin_gets_the_longer_limit(self, armed):
|
||||
from src.plugin_system.plugin_manager import PluginManager
|
||||
pm = PluginManager.__new__(PluginManager)
|
||||
seen = []
|
||||
pm._load_plugin = lambda plugin_id, force_enabled=False: seen.append(list(armed.sent)) or True
|
||||
assert pm.load_plugin('weather') is True
|
||||
allowance = int(display_watchdog.PLUGIN_LOAD_ALLOWANCE_SECONDS * 1e6)
|
||||
assert seen[0] and seen[0][-1].startswith(f'WATCHDOG_USEC={allowance}\n')
|
||||
assert armed.sent[-1].startswith('WATCHDOG_USEC=120000000\n')
|
||||
|
||||
|
||||
class TestStuckRenderThread:
|
||||
def test_a_render_thread_stuck_in_display_stops_the_pings(self, test_display_controller,
|
||||
monkeypatch):
|
||||
"""End to end through DisplayController.run(): pings flow while frames
|
||||
do, stop while display() is stuck even though other threads keep
|
||||
calling beat(), and nothing else in the process keeps them alive."""
|
||||
sent = []
|
||||
lock = threading.Lock()
|
||||
|
||||
def record(message):
|
||||
with lock:
|
||||
sent.append(message)
|
||||
return True
|
||||
# 0.3s limit -> a ping at most every 0.1s.
|
||||
wd = RenderWatchdog(environ={'NOTIFY_SOCKET': '/x', 'WATCHDOG_USEC': '300000'},
|
||||
send=record, heartbeat_dir=None, enable_faulthandler=False)
|
||||
monkeypatch.setattr(display_watchdog, 'watchdog', wd)
|
||||
|
||||
stuck, release = threading.Event(), threading.Event()
|
||||
|
||||
class Plugin:
|
||||
plugin_id = 'stuck-plugin'
|
||||
needs_high_fps = True
|
||||
enabled = True
|
||||
calls = 0
|
||||
|
||||
def display(self, force_clear=False):
|
||||
Plugin.calls += 1
|
||||
display_watchdog.note_frame() # DisplayManager is mocked here
|
||||
if Plugin.calls >= 60: # about half a second of frames
|
||||
stuck.set()
|
||||
release.wait(10)
|
||||
raise KeyboardInterrupt # ends run() the way SIGTERM does
|
||||
|
||||
controller = test_display_controller
|
||||
controller.available_modes = ['stuck-mode']
|
||||
controller.plugin_modes = {'stuck-mode': Plugin()}
|
||||
controller.mode_to_plugin_id = {'stuck-mode': 'stuck-plugin'}
|
||||
controller.current_mode_index = 0
|
||||
|
||||
runner = threading.Thread(target=controller.run, daemon=True)
|
||||
runner.start()
|
||||
assert stuck.wait(10), 'the render loop never reached the plugin'
|
||||
with lock:
|
||||
before = len(pings(sent))
|
||||
assert wd.armed and any(m.startswith('READY=1') for m in sent)
|
||||
assert before >= 2, sent
|
||||
|
||||
# Other threads carry on while the render thread is stuck.
|
||||
stop_others = threading.Event()
|
||||
|
||||
def busy_other_thread():
|
||||
while not stop_others.is_set():
|
||||
display_watchdog.beat()
|
||||
display_watchdog.note_frame()
|
||||
time.sleep(0.01)
|
||||
other = threading.Thread(target=busy_other_thread, daemon=True)
|
||||
other.start()
|
||||
time.sleep(0.8) # well past the 0.3s limit
|
||||
with lock:
|
||||
after = len(pings(sent))
|
||||
stop_others.set()
|
||||
release.set()
|
||||
other.join(5)
|
||||
runner.join(10)
|
||||
assert after == before, 'something other than the render thread fed the watchdog'
|
||||
assert not runner.is_alive()
|
||||
assert sent[-1] == 'STOPPING=1'
|
||||
|
||||
|
||||
# -- the unit and the entry point ----------------------------------------------
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
|
||||
def _service_directives():
|
||||
with open(os.path.join(ROOT, 'systemd', 'ledmatrix.service'), encoding='utf-8') as f:
|
||||
lines = [line.strip() for line in f]
|
||||
return dict(line.split('=', 1) for line in lines
|
||||
if line and not line.startswith(('#', '[')) and '=' in line
|
||||
and not line.startswith('Environment='))
|
||||
|
||||
|
||||
def _seconds(value):
|
||||
value = value.strip()
|
||||
for suffix, factor in (('min', 60), ('s', 1)):
|
||||
if value.endswith(suffix):
|
||||
return float(value[:-len(suffix)]) * factor
|
||||
return float(value)
|
||||
|
||||
|
||||
class TestUnit:
|
||||
def test_the_display_unit_has_a_watchdog_the_process_can_feed(self):
|
||||
d = _service_directives()
|
||||
# Type=notify would block "systemctl start/restart" until READY=1 --
|
||||
# after plugins load -- and the web UI and the update check call those
|
||||
# with short timeouts.
|
||||
assert d['Type'] == 'simple'
|
||||
assert d['NotifyAccess'] == 'main'
|
||||
assert 'WatchdogSec' in d
|
||||
|
||||
def test_the_watchdog_outlasts_the_loops_longest_healthy_gap(self):
|
||||
from src.plugin_system.plugin_executor import PluginExecutor
|
||||
limit = _seconds(_service_directives()['WatchdogSec'])
|
||||
executor_timeout = PluginExecutor().default_timeout
|
||||
assert limit >= 3 * executor_timeout, (
|
||||
"a screen's first display() may legitimately take the executor's "
|
||||
f"{executor_timeout}s timeout; WatchdogSec={limit:.0f}s leaves too little margin")
|
||||
assert limit < display_watchdog.STARTUP_ALLOWANCE_SECONDS
|
||||
assert limit < display_watchdog.PLUGIN_LOAD_ALLOWANCE_SECONDS
|
||||
assert display_watchdog.BEAT_INTERVAL_SECONDS * 4 <= limit
|
||||
assert limit > display_watchdog.HEARTBEAT_STALE_SECONDS >= 2 * executor_timeout
|
||||
|
||||
def test_the_heartbeat_directory_is_created_readable_by_the_web_user(self):
|
||||
d = _service_directives()
|
||||
assert d['RuntimeDirectory'] == 'ledmatrix'
|
||||
assert display_watchdog.HEARTBEAT_DIR == '/run/' + d['RuntimeDirectory']
|
||||
assert d['RuntimeDirectoryMode'] == '0755'
|
||||
|
||||
def test_a_crash_loop_backs_off_instead_of_stopping_for_good(self):
|
||||
"""A tripped start limit leaves the panel dark and refuses the web UI's
|
||||
Start button and the update rollback's restart."""
|
||||
d = _service_directives()
|
||||
assert d['Restart'] == 'always'
|
||||
assert 'StartLimitBurst' not in d
|
||||
assert int(d['RestartSteps']) > 0
|
||||
assert _seconds(d['RestartMaxDelaySec']) > _seconds(d['RestartSec'])
|
||||
|
||||
def test_run_py_widens_the_watchdog_before_importing_anything_heavy(self):
|
||||
with open(os.path.join(ROOT, 'run.py'), encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
call = text.index('display_watchdog.watchdog.begin_startup()')
|
||||
assert call < text.index('from src.logging_config')
|
||||
assert call < text.index('from src.display_controller')
|
||||
|
||||
def test_the_module_imports_nothing_else_from_src(self):
|
||||
"""run.py loads it first thing; importing it must stay cheap."""
|
||||
with open(os.path.join(ROOT, 'src', 'display_watchdog.py'), encoding='utf-8') as f:
|
||||
imports = [line for line in f if line.startswith(('import ', 'from '))]
|
||||
assert not [line for line in imports if 'src' in line], imports
|
||||
@@ -439,3 +439,101 @@ class ScheduleNoteMatchdayTests(unittest.TestCase):
|
||||
# without games, so a future entry there is not a next fixture.
|
||||
note = self.note(self.payload([-9], [11], whitelist=False))
|
||||
self.assertIn("season has finished", note)
|
||||
|
||||
|
||||
class ScheduleNoteListCalendarTests(unittest.TestCase):
|
||||
"""A round still to start in a "list" calendar is not a finished season.
|
||||
|
||||
Shapes captured from ESPN on 2026-09-29, with dates kept relative to that
|
||||
day. The Europa League scoreboard still showed the 17 September matchday
|
||||
and its calendar is a ``"list"`` of rounds, not match days, so the check
|
||||
said the season had finished -- with the knockout rounds, and the next
|
||||
league-phase matchday, still to come. PLL, the World Cup and AFL really had
|
||||
finished and must still say so, although each has a season or round
|
||||
``endDate`` in the future.
|
||||
"""
|
||||
|
||||
note = ScheduleNoteTests.note
|
||||
|
||||
@staticmethod
|
||||
def iso(days):
|
||||
return (datetime.now(timezone.utc) + timedelta(days=days)).strftime(
|
||||
"%Y-%m-%dT%H:%MZ")
|
||||
|
||||
@classmethod
|
||||
def list_league(cls, event_days, rounds, league_type=14540,
|
||||
event_type=14540, phase_label="UEFA Europa League",
|
||||
extra_phases=()):
|
||||
"""``rounds`` is ``[(label, start_day, end_day), ...]`` for one phase."""
|
||||
return {
|
||||
"events": [{"date": cls.iso(d), "season": {"type": event_type}}
|
||||
for d in event_days],
|
||||
"leagues": [{
|
||||
"season": {"type": {"type": league_type}},
|
||||
"calendarType": "list",
|
||||
"calendarIsWhitelist": True,
|
||||
"calendar": [{
|
||||
"label": phase_label,
|
||||
"startDate": cls.iso(-90), "endDate": cls.iso(275),
|
||||
"entries": [{"label": label, "startDate": cls.iso(start),
|
||||
"endDate": cls.iso(end)}
|
||||
for label, start, end in rounds],
|
||||
}] + list(extra_phases),
|
||||
}],
|
||||
}
|
||||
|
||||
def test_europa_between_matchdays_is_not_finished(self):
|
||||
note = self.note(self.list_league([-12], [
|
||||
("League Phase", -31, 123),
|
||||
("Knockout Round Playoffs", 123, 151),
|
||||
("Rd of 16", 151, 172),
|
||||
("Quarterfinals", 172, 200),
|
||||
("Semifinals", 200, 221),
|
||||
("Final", 222, 275),
|
||||
]))
|
||||
self.assertIsNone(note)
|
||||
|
||||
def test_world_cup_after_the_final_is_still_finished(self):
|
||||
# The competition runs to 31 December, and the last round ended 12
|
||||
# days after the final; no round is still to start.
|
||||
note = self.note(self.list_league([-72], [
|
||||
("Group", -110, -93),
|
||||
("Semifinals", -77, -72),
|
||||
("Final", -72, -59),
|
||||
], league_type=13803, event_type=13803, phase_label="FIFA World Cup"))
|
||||
self.assertIn("season has finished", note)
|
||||
|
||||
def test_afl_after_the_grand_final_is_still_finished(self):
|
||||
# The Grand Final round had started but had not ended yet.
|
||||
note = self.note(self.list_league([-3], [
|
||||
("Preliminary Finals", -13, -6),
|
||||
("Grand Final", -6, 1),
|
||||
], league_type=3, event_type=3, phase_label="Postseason"))
|
||||
self.assertIn("season has finished", note)
|
||||
|
||||
def test_an_offseason_round_does_not_count(self):
|
||||
# College football's "Off Season" phase holds the All-Star week.
|
||||
offseason = {"label": "Off Season", "startDate": self.iso(2),
|
||||
"endDate": self.iso(6),
|
||||
"entries": [{"label": "All-Star", "startDate": self.iso(2),
|
||||
"endDate": self.iso(6)}]}
|
||||
note = self.note(self.list_league(
|
||||
[-3], [("CFP", -40, 1)], league_type=3, event_type=3,
|
||||
phase_label="Postseason", extra_phases=[offseason]))
|
||||
self.assertIn("season has finished", note)
|
||||
|
||||
def test_pll_with_a_season_end_date_in_the_future_is_still_finished(self):
|
||||
# A "day" whitelist whose last match day is past; the season's own
|
||||
# endDate (1 January) is ignored.
|
||||
note = self.note({
|
||||
"events": [{"date": self.iso(-9), "season": {"type": 2}}],
|
||||
"leagues": [{
|
||||
"season": {"type": {"type": 2}, "startDate": self.iso(-271),
|
||||
"endDate": self.iso(94)},
|
||||
"calendarType": "day",
|
||||
"calendarIsWhitelist": True,
|
||||
"calendarEndDate": self.iso(94),
|
||||
"calendar": [self.iso(-30), self.iso(-22), self.iso(-9)],
|
||||
}],
|
||||
})
|
||||
self.assertIn("season has finished", note)
|
||||
|
||||
@@ -0,0 +1,426 @@
|
||||
"""On-demand for a plugin that is installed but disabled in config.
|
||||
|
||||
The display process only loads enabled plugins, so a request for a disabled
|
||||
one -- "Preview on display" offers it on every plugin's config page, with a
|
||||
note that the plugin will be enabled for the preview -- failed with
|
||||
"invalid-mode". Nothing loaded it short of a restart, and the on-demand
|
||||
route no longer restarts the service.
|
||||
|
||||
The display now loads such a plugin live for the session (force_enabled, so
|
||||
config.json keeps saying disabled) and the main loop unloads it once
|
||||
on-demand moves off it: a stop, an expiry, or a request for another plugin.
|
||||
|
||||
Also here: a stop sent after a failed request clears the error instead of
|
||||
leaving status 'error' published until the state ages out.
|
||||
"""
|
||||
|
||||
import time
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from src.plugin_system.plugin_manager import PluginManager
|
||||
from src.plugin_system.plugin_state import PluginState
|
||||
|
||||
|
||||
def _make_plugin(modes):
|
||||
plugin = MagicMock()
|
||||
plugin.modes = list(modes)
|
||||
return plugin
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def controller(test_display_controller):
|
||||
"""An idle controller running 'clock', with 'preview-me' installed but disabled."""
|
||||
c = test_display_controller
|
||||
clock = _make_plugin(['clock'])
|
||||
preview = _make_plugin(['preview_a', 'preview_b'])
|
||||
instances = {'clock': clock}
|
||||
catalogue = {'clock': clock, 'preview-me': preview}
|
||||
|
||||
def load_plugin(plugin_id, force_enabled=False):
|
||||
instances[plugin_id] = catalogue[plugin_id]
|
||||
return True
|
||||
|
||||
def unload_plugin(plugin_id):
|
||||
return instances.pop(plugin_id, None) is not None
|
||||
|
||||
pm = c.plugin_manager
|
||||
pm.discovered_plugin_ids.return_value = set(catalogue)
|
||||
pm.discover_plugins.return_value = list(catalogue)
|
||||
pm.plugin_manifests = {}
|
||||
pm.load_plugin = MagicMock(side_effect=load_plugin)
|
||||
pm.unload_plugin = MagicMock(side_effect=unload_plugin)
|
||||
pm.get_plugin.side_effect = instances.get
|
||||
|
||||
config = {'clock': {'enabled': True}, 'preview-me': {'enabled': False}}
|
||||
c.config_service.get_config = lambda: config
|
||||
c.config_manager.save_config = MagicMock()
|
||||
c.cache_manager.set = MagicMock()
|
||||
c.cache_manager.clear_cache = MagicMock()
|
||||
|
||||
c._register_loaded_plugin('clock')
|
||||
c.current_mode_index = 0
|
||||
c.current_display_mode = 'clock'
|
||||
c.test_config = config
|
||||
c.test_instances = instances
|
||||
return c
|
||||
|
||||
|
||||
def _start(c, plugin_id='preview-me', mode=None, **extra):
|
||||
request = {'request_id': 'r-' + plugin_id, 'action': 'start',
|
||||
'plugin_id': plugin_id, 'mode': mode or plugin_id}
|
||||
request.update(extra)
|
||||
c._activate_on_demand(request)
|
||||
|
||||
|
||||
class TestLoadingForOnDemand:
|
||||
def test_a_disabled_plugin_is_loaded_and_shown(self, controller):
|
||||
_start(controller)
|
||||
|
||||
controller.plugin_manager.load_plugin.assert_called_once_with(
|
||||
'preview-me', force_enabled=True)
|
||||
assert controller.on_demand_active is True
|
||||
assert controller.on_demand_status == 'active'
|
||||
assert controller.on_demand_plugin_id == 'preview-me'
|
||||
assert controller.current_display_mode == 'preview_a'
|
||||
assert controller.plugin_display_modes['preview-me'] == ['preview_a', 'preview_b']
|
||||
|
||||
def test_config_json_is_not_written(self, controller):
|
||||
_start(controller)
|
||||
|
||||
controller.config_manager.save_config.assert_not_called()
|
||||
assert controller.test_config['preview-me'] == {'enabled': False}
|
||||
|
||||
def test_a_requested_mode_is_honoured(self, controller):
|
||||
_start(controller, mode='preview_b')
|
||||
assert controller.current_display_mode == 'preview_b'
|
||||
|
||||
def test_an_enabled_plugin_is_not_reloaded(self, controller):
|
||||
_start(controller, plugin_id='clock')
|
||||
|
||||
controller.plugin_manager.load_plugin.assert_not_called()
|
||||
assert controller.on_demand_active is True
|
||||
assert controller._on_demand_loaded_plugins == set()
|
||||
|
||||
def test_a_plugin_that_is_not_installed_is_not_loaded(self, controller):
|
||||
_start(controller, plugin_id='uninstalled')
|
||||
|
||||
controller.plugin_manager.load_plugin.assert_not_called()
|
||||
assert controller.on_demand_status == 'error'
|
||||
assert controller.on_demand_last_error == 'invalid-mode'
|
||||
|
||||
def test_a_plugin_installed_after_startup_is_found_by_rescanning(self, controller):
|
||||
controller.plugin_manager.discovered_plugin_ids.return_value = {'clock'}
|
||||
|
||||
_start(controller)
|
||||
|
||||
controller.plugin_manager.discover_plugins.assert_called()
|
||||
assert controller.on_demand_active is True
|
||||
|
||||
|
||||
class TestLoadFailures:
|
||||
def test_a_failed_load_reports_load_failed(self, controller):
|
||||
controller.plugin_manager.load_plugin = MagicMock(return_value=False)
|
||||
|
||||
_start(controller)
|
||||
|
||||
assert controller.on_demand_active is False
|
||||
assert controller.on_demand_status == 'error'
|
||||
assert controller.on_demand_last_error == 'load-failed'
|
||||
assert 'preview_a' not in controller.available_modes
|
||||
published = controller.cache_manager.set.call_args_list[-1]
|
||||
assert published.args[0] == 'display_on_demand_state'
|
||||
assert published.args[1]['status'] == 'error'
|
||||
assert published.args[1]['error'] == 'load-failed'
|
||||
|
||||
def test_a_load_that_raises_reports_load_failed(self, controller):
|
||||
controller.plugin_manager.load_plugin = MagicMock(side_effect=ImportError('no module'))
|
||||
|
||||
_start(controller)
|
||||
|
||||
assert controller.on_demand_status == 'error'
|
||||
assert controller.on_demand_last_error == 'load-failed'
|
||||
|
||||
def test_a_failed_load_leaves_the_rotation_alone(self, controller):
|
||||
controller.plugin_manager.load_plugin = MagicMock(return_value=False)
|
||||
|
||||
_start(controller)
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
assert controller.available_modes == ['clock']
|
||||
assert controller.current_display_mode == 'clock'
|
||||
assert controller._on_demand_loaded_plugins == set()
|
||||
controller.plugin_manager.unload_plugin.assert_not_called()
|
||||
|
||||
def test_a_plugin_that_loads_but_has_no_modes_is_unloaded_again(self, controller):
|
||||
"""Registered, then the activation fails: the release removes it."""
|
||||
controller._on_demand_modes_for_plugin = MagicMock(return_value=[])
|
||||
|
||||
_start(controller)
|
||||
assert controller.on_demand_last_error == 'no-modes'
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
controller.plugin_manager.unload_plugin.assert_called_once_with('preview-me')
|
||||
assert controller.available_modes == ['clock']
|
||||
|
||||
|
||||
class TestReleasingThePlugin:
|
||||
def test_it_stays_loaded_while_on_demand_shows_it(self, controller):
|
||||
_start(controller)
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
controller.plugin_manager.unload_plugin.assert_not_called()
|
||||
assert 'preview_a' in controller.plugin_modes
|
||||
|
||||
def test_a_stop_unloads_it_and_resumes_the_rotation(self, controller):
|
||||
_start(controller)
|
||||
controller._clear_on_demand(reason='requested-stop')
|
||||
# Deferred to the main loop: the stop may be read mid-display().
|
||||
controller.plugin_manager.unload_plugin.assert_not_called()
|
||||
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
controller.plugin_manager.unload_plugin.assert_called_once_with('preview-me')
|
||||
assert controller.available_modes == ['clock']
|
||||
assert 'preview-me' not in controller.plugin_display_modes
|
||||
assert 'preview_a' not in controller.plugin_modes
|
||||
assert controller.current_display_mode == 'clock'
|
||||
assert controller._on_demand_loaded_plugins == set()
|
||||
assert controller.test_config['preview-me'] == {'enabled': False}
|
||||
|
||||
def test_expiry_unloads_it(self, controller):
|
||||
_start(controller, duration=30)
|
||||
controller.on_demand_expires_at = time.time() - 1
|
||||
controller._check_on_demand_expiration()
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
assert controller.on_demand_last_event == 'expired'
|
||||
controller.plugin_manager.unload_plugin.assert_called_once_with('preview-me')
|
||||
|
||||
def test_a_request_for_another_plugin_unloads_it(self, controller):
|
||||
_start(controller)
|
||||
_start(controller, plugin_id='clock')
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
controller.plugin_manager.unload_plugin.assert_called_once_with('preview-me')
|
||||
assert controller.on_demand_active is True
|
||||
assert controller.on_demand_plugin_id == 'clock'
|
||||
assert controller.current_display_mode == 'clock'
|
||||
|
||||
def test_a_failed_request_that_ends_the_session_unloads_it(self, controller):
|
||||
_start(controller)
|
||||
_start(controller, plugin_id='uninstalled')
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
controller.plugin_manager.unload_plugin.assert_called_once_with('preview-me')
|
||||
assert controller.current_display_mode == 'clock'
|
||||
assert controller.force_change is True
|
||||
|
||||
def test_a_plugin_enabled_during_the_session_stays_loaded(self, controller):
|
||||
_start(controller)
|
||||
controller.test_config['preview-me'] = {'enabled': True}
|
||||
controller._clear_on_demand(reason='requested-stop')
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
controller.plugin_manager.unload_plugin.assert_not_called()
|
||||
assert 'preview_a' in controller.available_modes
|
||||
assert controller._on_demand_loaded_plugins == set()
|
||||
|
||||
def test_the_main_loop_releases_right_after_its_own_poll(self, controller):
|
||||
"""A stop read by the main loop unloads before the next screen, not
|
||||
one screen later. That poll runs with no display() on the stack."""
|
||||
import inspect
|
||||
source = inspect.getsource(type(controller).run)
|
||||
poll = source.index('self._check_on_demand_expiration()')
|
||||
release = source.index('self._release_on_demand_plugins()')
|
||||
render = source.index('self._tick_plugin_updates()')
|
||||
assert poll < release < render
|
||||
|
||||
def test_a_reconcile_that_runs_first_unloads_it_the_same_way(self, controller):
|
||||
"""A reconcile queued during the session runs at the top of the loop,
|
||||
before the release: it removes the plugin itself (not in the enabled
|
||||
set) and the release is then a no-op."""
|
||||
_start(controller)
|
||||
controller._clear_on_demand(reason='requested-stop')
|
||||
|
||||
controller._reconcile_enabled_plugins()
|
||||
controller._release_on_demand_plugins()
|
||||
|
||||
controller.plugin_manager.unload_plugin.assert_called_once_with('preview-me')
|
||||
assert controller.available_modes == ['clock']
|
||||
assert controller._on_demand_loaded_plugins == set()
|
||||
|
||||
def test_a_config_save_mid_session_keeps_the_instance_enabled(self, controller):
|
||||
"""on_config_change would otherwise read enabled: false and switch it off."""
|
||||
controller.config_service.subscribe = MagicMock()
|
||||
_start(controller)
|
||||
callback = controller._plugin_config_callbacks['preview-me']
|
||||
controller.plugin_manager.prepare_plugin_config = None
|
||||
|
||||
callback({}, {'enabled': False, 'color': 'red'})
|
||||
|
||||
# The change goes through the manager's locked apply_config_change
|
||||
# (which calls on_config_change under the plugin's lock).
|
||||
plugin = controller.plugin_modes['preview_a']
|
||||
controller.plugin_manager.apply_config_change.assert_called_once_with(
|
||||
'preview-me', {'enabled': True, 'color': 'red'}, plugin_instance=plugin)
|
||||
|
||||
|
||||
class TestRestoredSession:
|
||||
"""A restart during a session for a disabled plugin restores it the same way."""
|
||||
|
||||
def test_the_plugin_is_tracked_and_config_is_left_alone(self, test_display_controller):
|
||||
c = test_display_controller
|
||||
c.config.update({'clock': {'enabled': True}, 'disabled-one': {'enabled': False}})
|
||||
|
||||
selected = c._select_startup_plugins(
|
||||
['clock', 'disabled-one'], {'plugin_id': 'disabled-one', 'mode': 'x'})
|
||||
|
||||
assert 'disabled-one' in selected
|
||||
assert c._on_demand_loaded_plugins == {'disabled-one'}
|
||||
assert c.config['disabled-one']['enabled'] is False
|
||||
|
||||
|
||||
class TestResumingAfterTheSession:
|
||||
"""Ending a session never resumes the rotation onto the plugin that is
|
||||
about to be unloaded."""
|
||||
|
||||
def _restored_session(self, c, other_modes=('clock',)):
|
||||
"""As after a restart: no saved resume index, and the plugin's modes
|
||||
ordered in ahead of the rest (load order is not deterministic)."""
|
||||
c._on_demand_loaded_plugins.add('preview-me')
|
||||
c.plugin_manager.load_plugin('preview-me', force_enabled=True)
|
||||
c._register_loaded_plugin('preview-me')
|
||||
c.available_modes = ['preview_a', 'preview_b'] + list(other_modes)
|
||||
c.on_demand_active = True
|
||||
c.on_demand_status = 'active'
|
||||
c.on_demand_plugin_id = 'preview-me'
|
||||
c.on_demand_modes = ['preview_a', 'preview_b']
|
||||
c.rotation_resume_index = None
|
||||
c.current_mode_index = 0
|
||||
c.current_display_mode = 'preview_a'
|
||||
|
||||
def test_a_restored_session_resumes_on_an_enabled_mode(self, controller):
|
||||
self._restored_session(controller)
|
||||
|
||||
controller._clear_on_demand(reason='requested-stop')
|
||||
|
||||
assert controller.current_display_mode == 'clock'
|
||||
controller._release_on_demand_plugins()
|
||||
assert controller.available_modes == ['clock']
|
||||
assert controller.current_display_mode == 'clock'
|
||||
|
||||
def test_with_nothing_else_enabled_the_display_goes_idle(self, controller):
|
||||
controller._unregister_plugin('clock')
|
||||
self._restored_session(controller, other_modes=())
|
||||
|
||||
controller._clear_on_demand(reason='requested-stop')
|
||||
assert controller.current_display_mode is None
|
||||
|
||||
controller._release_on_demand_plugins()
|
||||
assert controller.available_modes == []
|
||||
assert controller.current_display_mode is None
|
||||
|
||||
def test_a_saved_resume_index_is_still_used(self, controller):
|
||||
c = controller
|
||||
c.available_modes = ['clock', 'other']
|
||||
c.plugin_modes['other'] = MagicMock()
|
||||
c.current_mode_index = 1
|
||||
c.current_display_mode = 'other'
|
||||
|
||||
_start(c)
|
||||
c._clear_on_demand(reason='requested-stop')
|
||||
|
||||
assert c.current_display_mode == 'other'
|
||||
|
||||
|
||||
class TestStopClearsAnError:
|
||||
def _post_stop(self, c):
|
||||
stop = {'request_id': 'S1', 'action': 'stop'}
|
||||
c._last_on_demand_poll = None
|
||||
c.cache_manager.get = MagicMock(
|
||||
side_effect=lambda key, *a, **kw:
|
||||
stop if key == 'display_on_demand_request' else None)
|
||||
c.cache_manager.delete = MagicMock()
|
||||
c._poll_on_demand_requests()
|
||||
|
||||
def test_a_stop_after_a_failed_request_clears_the_error(self, controller):
|
||||
_start(controller, plugin_id='uninstalled')
|
||||
assert controller.on_demand_status == 'error'
|
||||
|
||||
self._post_stop(controller)
|
||||
|
||||
assert controller.on_demand_status == 'idle'
|
||||
assert controller.on_demand_last_error is None
|
||||
state = controller.cache_manager.set.call_args_list[-1].args[1]
|
||||
assert state['status'] == 'idle'
|
||||
assert state['error'] is None
|
||||
|
||||
def test_clearing_the_error_leaves_the_rotation_alone(self, controller):
|
||||
_start(controller, plugin_id='uninstalled')
|
||||
controller.force_change = False
|
||||
|
||||
self._post_stop(controller)
|
||||
|
||||
assert controller.current_display_mode == 'clock'
|
||||
assert controller.force_change is False
|
||||
|
||||
def test_a_stop_while_idle_is_still_just_acknowledged(self, controller):
|
||||
controller._clear_on_demand = MagicMock()
|
||||
|
||||
self._post_stop(controller)
|
||||
|
||||
assert controller.on_demand_status == 'idle'
|
||||
assert controller.on_demand_request_id == 'S1'
|
||||
controller._clear_on_demand.assert_not_called()
|
||||
|
||||
|
||||
class TestForceEnabledLoad:
|
||||
"""PluginManager.load_plugin(force_enabled=True) runs the plugin enabled
|
||||
without touching the config it read."""
|
||||
|
||||
class _Plugin:
|
||||
def __init__(self, config):
|
||||
self.config = config
|
||||
self.enabled_calls = 0
|
||||
|
||||
def on_enable(self):
|
||||
self.enabled_calls += 1
|
||||
|
||||
@pytest.fixture
|
||||
def pm(self, tmp_path):
|
||||
plugins_dir = tmp_path / 'plugins'
|
||||
(plugins_dir / 'demo').mkdir(parents=True)
|
||||
manager = PluginManager(plugins_dir=str(plugins_dir))
|
||||
manager.plugin_manifests['demo'] = {'id': 'demo', 'name': 'Demo'}
|
||||
manager.schema_manager = MagicMock()
|
||||
manager.schema_manager.get_schema_path.return_value = None
|
||||
# Hand the section back as-is, as the fallback path can: the copy in
|
||||
# load_plugin is what keeps the cached config clean.
|
||||
manager.schema_manager.prepare_plugin_config.side_effect = (
|
||||
lambda pid, cfg, schema=None, changed_paths=None: cfg)
|
||||
manager.plugin_loader = MagicMock()
|
||||
manager.plugin_loader.find_plugin_directory.return_value = plugins_dir / 'demo'
|
||||
manager.plugin_loader.load_plugin.side_effect = (
|
||||
lambda **kw: (self._Plugin(kw['config']), None))
|
||||
manager.config_manager = MagicMock()
|
||||
manager.cached_config = {'demo': {'enabled': False, 'color': 'red'}}
|
||||
manager.config_manager.load_config.return_value = manager.cached_config
|
||||
return manager
|
||||
|
||||
def test_a_disabled_plugin_loads_disabled_by_default(self, pm):
|
||||
assert pm.load_plugin('demo') is True
|
||||
assert pm.plugins['demo'].enabled_calls == 0
|
||||
assert pm.state_manager.get_state('demo') == PluginState.DISABLED
|
||||
|
||||
def test_force_enabled_runs_it_enabled(self, pm):
|
||||
assert pm.load_plugin('demo', force_enabled=True) is True
|
||||
plugin = pm.plugins['demo']
|
||||
assert plugin.config == {'enabled': True, 'color': 'red'}
|
||||
assert plugin.enabled_calls == 1
|
||||
assert pm.state_manager.get_state('demo') == PluginState.ENABLED
|
||||
|
||||
def test_force_enabled_does_not_touch_the_cached_config(self, pm):
|
||||
pm.load_plugin('demo', force_enabled=True)
|
||||
assert pm.cached_config['demo'] == {'enabled': False, 'color': 'red'}
|
||||
@@ -164,12 +164,14 @@ class TestRestartDoesNotStarveTheOtherPlugins:
|
||||
assert controller.on_demand_mode == 'app_a'
|
||||
assert controller.on_demand_pinned is True
|
||||
|
||||
def test_a_disabled_on_demand_plugin_is_enabled_and_loaded(self, controller):
|
||||
"""Otherwise the mode being resumed has nothing behind it."""
|
||||
def test_a_disabled_on_demand_plugin_is_still_loaded(self, controller):
|
||||
"""Otherwise the mode being resumed has nothing behind it. It loads
|
||||
for on-demand only; its config section is left disabled."""
|
||||
selected = controller._select_startup_plugins(
|
||||
self.DISCOVERED, {'plugin_id': 'disabled-one', 'mode': 'x'})
|
||||
assert 'disabled-one' in selected
|
||||
assert controller.config['disabled-one']['enabled'] is True
|
||||
assert controller._on_demand_loaded_plugins == {'disabled-one'}
|
||||
assert controller.config['disabled-one']['enabled'] is False
|
||||
|
||||
def test_an_unknown_on_demand_plugin_falls_back_to_normal(self, controller):
|
||||
selected = controller._select_startup_plugins(
|
||||
|
||||
@@ -57,7 +57,7 @@ def render(config):
|
||||
# pages_v3 is a module-level singleton shared across the test process;
|
||||
# restore whatever the previous test left on it.
|
||||
original_cm = getattr(pv.pages_v3, "config_manager", None)
|
||||
original_pm = getattr(pv.pages_v3, "plugin_manager", None)
|
||||
original_pm = getattr(pv.pages_v3, "plugin_catalog", None)
|
||||
|
||||
mock_cm = MagicMock()
|
||||
mock_cm.load_config.return_value = config
|
||||
@@ -65,10 +65,9 @@ def render(config):
|
||||
pv.pages_v3.config_manager = mock_cm
|
||||
|
||||
mock_pm = MagicMock()
|
||||
mock_pm.plugins = {}
|
||||
mock_pm.get_all_plugin_info.return_value = []
|
||||
mock_pm.get_plugin_display_modes.side_effect = lambda pid: []
|
||||
pv.pages_v3.plugin_manager = mock_pm
|
||||
pv.pages_v3.plugin_catalog = mock_pm
|
||||
|
||||
app.register_blueprint(pv.pages_v3, url_prefix="")
|
||||
try:
|
||||
@@ -77,7 +76,7 @@ def render(config):
|
||||
return resp.get_data(as_text=True)
|
||||
finally:
|
||||
pv.pages_v3.config_manager = original_cm
|
||||
pv.pages_v3.plugin_manager = original_pm
|
||||
pv.pages_v3.plugin_catalog = original_pm
|
||||
|
||||
|
||||
def timezone_step(body):
|
||||
|
||||
@@ -17,7 +17,7 @@ from web_interface.blueprints import pages_v3 as module # noqa: E402
|
||||
def client(tmp_path, monkeypatch):
|
||||
plugin_manager = MagicMock()
|
||||
plugin_manager.plugins_dir = tmp_path
|
||||
monkeypatch.setattr(module.pages_v3, "plugin_manager", plugin_manager, raising=False)
|
||||
monkeypatch.setattr(module.pages_v3, "plugin_catalog", plugin_manager, raising=False)
|
||||
monkeypatch.setattr(module.pages_v3, "config_manager",
|
||||
MagicMock(load_config=lambda: {}), raising=False)
|
||||
app = Flask(__name__, template_folder=str(
|
||||
@@ -63,3 +63,17 @@ def test_web_ui_page_uses_the_ledmatrix_prefix_fallback(client, tmp_path):
|
||||
|
||||
assert response.status_code == 200
|
||||
assert "radar panel" in response.get_data(as_text=True)
|
||||
|
||||
|
||||
def test_web_ui_page_styles_come_from_the_pi_not_a_cdn(client, tmp_path):
|
||||
"""In AP mode there is no internet; a CDN stylesheet left fragments unstyled."""
|
||||
web_ui = tmp_path / "radar" / "web_ui"
|
||||
web_ui.mkdir(parents=True)
|
||||
(web_ui / "panel.html").write_text("<p>radar panel</p>", encoding="utf-8")
|
||||
|
||||
body = client.get("/plugin-ui/radar/web-ui/panel.html").get_data(as_text=True)
|
||||
|
||||
assert '<link rel="stylesheet" href="/static/v3/plugin-frame.css' in body
|
||||
assert "cdnjs" not in body and "https://" not in body
|
||||
assert (Path(module.__file__).resolve().parents[1]
|
||||
/ "static" / "v3" / "plugin-frame.css").is_file()
|
||||
|
||||
@@ -39,19 +39,18 @@ def pages(tmp_path):
|
||||
"<p>panel</p>", encoding="utf-8"
|
||||
)
|
||||
|
||||
original_pm = getattr(module.pages_v3, "plugin_manager", None)
|
||||
original_pm = getattr(module.pages_v3, "plugin_catalog", None)
|
||||
original_cm = getattr(module.pages_v3, "config_manager", None)
|
||||
|
||||
plugin_manager = MagicMock()
|
||||
plugin_manager.plugins_dir = plugins_dir
|
||||
plugin_manager.get_plugin_info.return_value = {"name": "Weather", "version": "1.0.0"}
|
||||
plugin_manager.get_plugin.return_value = None
|
||||
module.pages_v3.plugin_manager = plugin_manager
|
||||
module.pages_v3.plugin_catalog = plugin_manager
|
||||
module.pages_v3.config_manager = MagicMock(load_config=lambda: {})
|
||||
|
||||
yield module, plugins_dir
|
||||
|
||||
module.pages_v3.plugin_manager = original_pm
|
||||
module.pages_v3.plugin_catalog = original_pm
|
||||
module.pages_v3.config_manager = original_cm
|
||||
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user