mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 14:25:08 +00:00
Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 ChuckandClaude Opus 5.5
|
2236ff3081 | ||
|
ChuckandClaude Opus 5.5
|
5ad5e9aa59 | ||
|
ChuckandClaude Opus 5.5
|
8a0cce1aaf | ||
|
ChuckandClaude Opus 5.5
|
0b039c875f | ||
|
ChuckandClaude Opus 5.5
|
07abd87d5e | ||
|
ChuckandClaude Opus 5.5
|
e32d177cbd | ||
|
ChuckandClaude Opus 5.5
|
d18e4d3c9d | ||
|
ChuckandClaude Opus 5.5
|
04f0d8d134 | ||
|
ChuckandClaude Opus 5.5
|
064b9c9912 | ||
|
ChuckandClaude Opus 5.5
|
a6e9e3ef1c |
+290
-15
@@ -19,21 +19,6 @@ accepts both, but the store flags the old spelling as deprecated
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Fixed
|
||||
|
||||
- The web preview and `/api/v3/display/current` no longer stay black for a
|
||||
whole screen that draws its card once and then holds it. The snapshot is
|
||||
written from `update_display()` at most once per write interval, so a frame
|
||||
pushed inside that interval was skipped and left for the next
|
||||
`update_display()` -- which such a screen never makes. Soccer's
|
||||
recent/upcoming cards skip redundant redraws, and the first one after an
|
||||
on-demand start lands a few milliseconds after the start's clear wrote a
|
||||
black frame: on ledpi the preview showed 0 lit pixels for the whole 15 s
|
||||
while the panel showed the card. `DisplayManager` now remembers a skipped
|
||||
changed frame, and the render loop writes it (`write_owed_snapshot()`)
|
||||
once the interval has passed. The cadence is unchanged, and nothing extra
|
||||
runs when no frame is owed.
|
||||
|
||||
### Cheap per-frame and per-fetch savings
|
||||
|
||||
- `BaseOddsManager.get_odds()` no longer pretty-prints every odds response
|
||||
@@ -503,6 +488,23 @@ policies are unchanged.
|
||||
|
||||
### Fixes
|
||||
|
||||
- A cache key too long to be a filename is now cached. The calendar
|
||||
plugin's key joins every calendar id the user picked; on a real install
|
||||
it passed 300 bytes, ext4 refuses names over 255, and every write failed
|
||||
with `File name too long` — logged as "(permission denied)", so it read
|
||||
like a cache-directory ownership problem. `DiskCache.get_cache_path` now
|
||||
keeps a key of up to 200 UTF-8 bytes as its filename, as before, and
|
||||
turns a longer one into its first bytes plus a hash of the whole key. The
|
||||
web UI's cache list and delete keep working, because the shortened name
|
||||
maps back to the same file. A failed write now names the real error.
|
||||
- The cache's memory tier no longer serves data older than the reader asked
|
||||
for. A record loaded from disk was timed in memory from the load, not
|
||||
from when it was written, so `get(key, max_age=300)` could return data
|
||||
close to 600 s old (after a restart, after the hourly memory sweep, or in
|
||||
the other process, which only ever loads the record from disk), and a
|
||||
stored `ttl` was stretched the same way. A memory hit is now also checked against
|
||||
the record's own timestamp, and a stale one falls through to disk, which
|
||||
returns a newer write if there is one.
|
||||
- The garbage-collection timer (`GcMonitor`, above) no longer prints
|
||||
`Exception ignored while calling GC callback ... 'NoneType' object has no
|
||||
attribute 'perf_counter'` when the display service or a test run exits.
|
||||
@@ -513,6 +515,16 @@ policies are unchanged.
|
||||
`DisplayManager.cleanup()` (reached from SIGTERM through `run()`'s
|
||||
`finally`) unregisters it with the frame recorder. New
|
||||
`frame_timing.uninstall_gc_monitor()`.
|
||||
- The web interface's state subscription (`StateSubscription`,
|
||||
`src/ipc/client.py`) resubscribes about 1 s after a display restart, every
|
||||
time. Its reconnect wait went back to the minimum only when the
|
||||
subscription was stopped. A disconnect after a working connection kept
|
||||
doubling the wait, so successive display restarts were followed by waits
|
||||
of 1, 2, 4, 8, 16 and then 30 s for good.
|
||||
During each wait the web answered from one-shot `state.get` connections
|
||||
instead of its copy. The wait now resets once a connection has stored a
|
||||
snapshot. A display that does not offer the stream is still retried
|
||||
slowly.
|
||||
- A plugin reload after a store update (`plugin.reload`, #720) no longer
|
||||
freezes the panel during Vegas. On ledpi a football reload froze it for
|
||||
3.0 s (`Render stall over: no frame for 3043ms`). The reload ran on the
|
||||
@@ -529,6 +541,25 @@ policies are unchanged.
|
||||
for the plugin is refused (`plugin-reloading`), and a config reconcile
|
||||
neither loads it twice nor unloads it mid-load. A Vegas fetch that waited
|
||||
out a reload for the lock skips the old instance.
|
||||
- A plugin display duration that is not a number no longer stops the
|
||||
display. Several plugins (clock-simple, calendar, countdown) return their
|
||||
`display_duration` setting as it is in config.json, so a value saved as
|
||||
`"20"` or `null` (the raw config editor, a hand edit) reached the run loop
|
||||
as a string or None. Comparing it with 0 raised a TypeError that no
|
||||
handler in the loop caught: the display service exited when that plugin's
|
||||
screen came up, and systemd restarted it into the same crash. The
|
||||
controller now reads the plugin's answer as a number: a numeric string
|
||||
counts, and anything else (or a `get_display_duration()` that raises)
|
||||
shows the mode for 30 s, with one warning per plugin.
|
||||
- A scroll strip narrower than the panel scrolls instead of raising on every
|
||||
frame. When a frame ran off the end of the strip, `ScrollHelper` copied
|
||||
the strip's tail and then the rest of the frame from its head, which
|
||||
assumed the head was that wide; for a narrower strip that raised
|
||||
`ValueError: could not broadcast` at every position, so nothing was drawn
|
||||
and each frame logged a traceback. Vegas builds such a strip, with no
|
||||
lead-in, when its content is narrower than the chain. A frame that runs
|
||||
off the strip now continues from its head column by column, so a narrow
|
||||
strip repeats across the panel; a wide strip wraps exactly as before.
|
||||
- The schedule-off blank and the WiFi notice no longer start with a
|
||||
scroller's leftovers. Both are drawn by the display controller rather than
|
||||
dispatched to a plugin, so #716's handover never reached them: drawn while
|
||||
@@ -538,6 +569,46 @@ policies are unchanged.
|
||||
scroller or Vegas) were counted as 0.5-1 s freezes and logged as a
|
||||
`Render stall ... mid-scroll`. The controller now ends the scroll state
|
||||
before drawing either.
|
||||
- A plugin that keeps helpers in a package (elections' `providers/`,
|
||||
flights' `enrichment/`, olympics' `data/` and `renderers/`) now runs its
|
||||
updated helpers after a reload. Unloading dropped the package itself but
|
||||
left its modules (`providers.feed`) in `sys.modules`, so the reload after a
|
||||
store update imported the new `manager.py` and got the old helpers back from
|
||||
the cache until the display restarted. `PluginLoader` now drops a plugin's
|
||||
package modules when it unloads, and when a load fails part-way.
|
||||
- Uninstalling a dev plugin that `scripts/dev/dev_plugin_setup.sh` linked
|
||||
into the plugins directory now removes the link and leaves the checkout
|
||||
alone. The store's removal passed the link to `shutil.rmtree`, which
|
||||
refuses a symlink; its fallback then walked through the link and chmodded
|
||||
every directory and file of the linked checkout to 0700, and the sudo stage
|
||||
refused a path outside the plugins directory, so the uninstall failed with
|
||||
the link still in place. The same removal discards the set-aside copy after
|
||||
an install or update. A symlink, dangling or not, is now unlinked.
|
||||
- A dev plugin linked in under a name its checkout does not share now loads.
|
||||
`dev_plugin_setup.sh link-github foo <url>` clones `ledmatrix-foo` (the
|
||||
repository naming convention) and links it as `plugins/foo`. The loader's
|
||||
containment check for dependency installs resolved the link and looked for
|
||||
`ledmatrix-foo` among the plugins directory's entries, found none, and
|
||||
refused the plugin, so the load failed with "Dependency installation
|
||||
failed" even when it had no `requirements.txt`. The check now looks for the
|
||||
entry the path itself names in the plugins directory, the link, and still
|
||||
only ever answers with an entry it found there.
|
||||
- A plugin whose `update()` raises `asyncio.CancelledError` or `SystemExit`
|
||||
no longer goes dark until a restart. Both derive from `BaseException`, not
|
||||
`Exception`, and the update worker's bookkeeping caught only `Exception`:
|
||||
the plugin kept its lock and stayed RUNNING, so it was never updated again
|
||||
and every `display()` was skipped as busy. It is now recorded as that
|
||||
update's failure, the same as any other raise. The plugin executor
|
||||
reported such a call as a timeout; it now reports it as a failure.
|
||||
- Saving a config change no longer freezes the panel while a plugin is busy.
|
||||
`ConfigService` told its subscribers about a change while holding its lock,
|
||||
and the display's per-plugin subscriber waits up to 5 s for a plugin in the
|
||||
middle of an update. A save that enables or disables a plugin also queues a
|
||||
reconcile, which the render thread runs, and its `get_config()` and
|
||||
`unsubscribe()` waited behind every one of those callbacks. Subscribers now
|
||||
run after the lock is released. One reload's notifications still finish
|
||||
before the next one's start, and a callback `unsubscribe()` removed is not
|
||||
running, and will not run, once it returns.
|
||||
- A plugin whose `display()` raises now opens its circuit breaker. The first
|
||||
frame of each screen goes through the plugin executor, which caught the
|
||||
exception and returned False. The display read that as "no content" and
|
||||
@@ -547,6 +618,57 @@ policies are unchanged.
|
||||
the plugin leaves rotation until the cooldown ends, the same as a raising
|
||||
`update()`. The display still moves straight on to the next mode. A hung
|
||||
`display()` is still recorded once, as a hang.
|
||||
- A plugin settings save that failed validation no longer leaks into the next
|
||||
save. `ConfigManager.load_config()` returned its cached config itself (the
|
||||
fast path from #410), so the form save's edits went into the cache before
|
||||
validation ran, and a refused save left them there. The next save of any
|
||||
other setting (another plugin's, a plugin toggle, the schedule) wrote them
|
||||
to config.json: the refused value, and a nested secret typed into the same
|
||||
form (`mqtt.password`, `league.espn_s2`, `flightaware.api_key`) in plain
|
||||
text, because it had never reached config_secrets.json to be stripped.
|
||||
The form also reloaded showing the refused values. `load_config()` now
|
||||
returns a private copy, and the saves keep one, so nothing a caller edits
|
||||
reaches the cache unless it is saved. The copy duplicates only the dicts
|
||||
and lists (every other JSON value is immutable): 2.1 ms for a real 60 KiB
|
||||
config on a Pi 4, against 6.8 ms for `copy.deepcopy`.
|
||||
- `GET /api/v3/plugins/config` no longer returns secrets. It sent back the
|
||||
plugin's section with config_secrets.json merged in, API keys and tokens
|
||||
in plain text: the masking #276 added was dropped in #330. It also took
|
||||
any id, so `?plugin_id=web_auth` returned the login's cookie-signing key
|
||||
and password hash and `?plugin_id=github` the Plugin Store token. Secret
|
||||
fields now come back blank, as the settings page renders them, and a
|
||||
plugin with no schema has its credential-named fields blanked, as
|
||||
`GET /config/main` does. Blank rather than the `••••••••` of
|
||||
`GET /config/secrets`, because the save reads a blank secret as
|
||||
"unchanged", so a client can post the response back without erasing
|
||||
one. Core sections and malformed ids get a 400, as they already did from
|
||||
reset and uninstall.
|
||||
- Plugin settings with a table (a list of rows, such as geochron's cities
|
||||
or the countdowns) save again when a text cell is blank or holds only
|
||||
digits. A row posts its cells as `cities.0.timezone`, and the schema
|
||||
lookup stopped at the list, so each cell was parsed with no schema: a
|
||||
blank optional text cell became null, and a name like "2027" became a
|
||||
number. Either failed validation, and every save of the page failed for
|
||||
as long as the row existed. A plugin with a secret in its rows could not
|
||||
be saved from the page at all, since the secret cell is drawn blank. The
|
||||
lookup now steps from the index into the list's item schema.
|
||||
- A plugin whose API key is required and has no default (youtube-stats)
|
||||
can be saved from its settings page without typing the key in again. The
|
||||
page draws a stored secret blank and posts the blank back; for a required
|
||||
secret the save read that blank as null, failed validation, and refused
|
||||
every save of the page. A blank secret field now means "unchanged", as it
|
||||
already did for an optional one.
|
||||
- `POST /api/v3/plugins/config` refuses a core section or a malformed
|
||||
plugin id with a 400, as reset and uninstall already did.
|
||||
`{"plugin_id": "display", ...}` merged unvalidated values into the core
|
||||
display section (and added `"enabled": true` to it), and an id that was
|
||||
not a string answered with a 500.
|
||||
- A plugin text setting saves what was typed when that looks like a
|
||||
boolean or JSON. The form save tried `true`/`false` and `[...]`/`{...}`
|
||||
before it looked at the schema, so a text field holding "true", "False",
|
||||
"[1, 2]" or "{}" was stored as a boolean, list or object, and the save
|
||||
failed validation. Text fields, nullable ones included, are now taken as
|
||||
typed; other types convert as before.
|
||||
- A WiFi notice (such as "Connected to HomeNet" or "AP mode on") now shows
|
||||
within about a second of being posted. It was only checked between
|
||||
screens, so a 5 s notice posted during a 20 s screen expired before that
|
||||
@@ -555,6 +677,42 @@ policies are unchanged.
|
||||
notice is what shows next, and Vegas resumes after it; before, a rotation
|
||||
screen showed instead and the notice expired behind it. An active
|
||||
on-demand session still holds the panel until it ends.
|
||||
- The Config Editor tab no longer shows API keys and tokens in plain
|
||||
text. Its `config_secrets.json` editor (`/partials/raw-json`) was filled
|
||||
with the file as it is on disk, so while the web login is off (the
|
||||
default) anyone who could reach the port could read every credential,
|
||||
although `GET /api/v3/config/secrets` masks them. The editor now shows the
|
||||
same masked values. Saving it unchanged changes nothing, because the save
|
||||
drops the masks and merges onto the stored file; to change a secret,
|
||||
replace its mask. A list of secrets still needs every entry's real value
|
||||
to be changed. The `config.json` editor is unchanged: its save writes the
|
||||
file as given, so a mask there would be stored.
|
||||
- A disabled plugin keeps its place in the rotation order and its Vegas
|
||||
exclusion when the Display or Rotation & Durations tab is saved. The order
|
||||
lists show enabled plugins only and rewrite their hidden inputs from those
|
||||
rows as soon as they are drawn, so any save of either tab stored the lists
|
||||
without the disabled plugin. Once re-enabled, it came back at the end of
|
||||
the rotation and scrolling in Vegas again. A disabled plugin's saved id
|
||||
now stays in its saved place (`widgets/plugin-order-list.js`); the id of
|
||||
a plugin that is no longer installed is still dropped.
|
||||
- Restoring a backup with "Reinstall missing plugins" installs only the
|
||||
plugins that are missing. Every plugin the backup listed was sent to the
|
||||
store's install, which replaces an installed copy with a fresh download,
|
||||
so a restore onto the same device re-downloaded all of them in one
|
||||
request. A plugin installed from its own URL is not in the registry, so
|
||||
its "reinstall" failed and the restore answered "Restore failed" while
|
||||
the plugin sat there installed. An installed plugin, found by the store's
|
||||
own lookup (registry aliases included), is now listed under Skipped as
|
||||
`plugin:<id> (installed)`.
|
||||
- `POST /api/v3/config/main` answers a JSON body that does not parse with
|
||||
400 `Invalid JSON in request body`, as `/config/raw/main` does, and an
|
||||
empty JSON body with 400 `No data provided`. Both were a 500
|
||||
`CONFIG_SAVE_FAILED` suggesting file permissions and disk space, with a
|
||||
traceback logged at ERROR: `get_json()` raised inside the handler's
|
||||
catch-all.
|
||||
- Fonts restored from a backup show up in the Fonts tab and the font
|
||||
pickers straight away. The font catalog is cached for five minutes, and
|
||||
upload and delete cleared it but a restore did not.
|
||||
- A game that goes live now takes over the panel within about a second.
|
||||
Live priority was only checked between screens, so a game that went live
|
||||
during a 30 s screen waited for that screen to end. The frame loops and the
|
||||
@@ -564,6 +722,45 @@ policies are unchanged.
|
||||
screen showed first and the game came after it. Each check also asks each
|
||||
plugin `has_live_content()` once, where a plugin registered under several
|
||||
modes used to be asked once per mode.
|
||||
- A plugin action whose params hold `true`, `false` or `null` runs again.
|
||||
`/api/v3/plugins/action` wrote the params into the source of the wrapper
|
||||
that runs the plugin's script, and those JSON words are not Python, so the
|
||||
wrapper stopped with a NameError and the action answered "Action failed".
|
||||
The plugin file manager's category toggle sends `"enabled": true`, so
|
||||
turning a category on or off in of-the-day always failed. The params now
|
||||
reach the wrapper on its stdin; the script still receives them as JSON on
|
||||
its own stdin, as before.
|
||||
- An on-demand request that `/api/v3/display/on-demand/start` refuses no
|
||||
longer runs later. With the display stopped the request goes to the
|
||||
display's mailbox, and the display reads that mailbox for an hour without
|
||||
looking at a request's age. So with "Start display service" unticked, the
|
||||
answer was "Display service is not running", yet the next time the
|
||||
display was started it ran that plugin, pinned if the request said so.
|
||||
The same happened after "Failed to start display service". On either
|
||||
refusal the route now takes its request back out of the mailbox, unless a
|
||||
newer one has replaced it. A request the display acknowledges over the
|
||||
control socket is now a success whatever systemd reports: a display run
|
||||
by hand or in the emulator was told "not running" for a request it had
|
||||
already taken, and with "Start display service" ticked the route tried to
|
||||
start the service beside it.
|
||||
- `/api/v3/plugins/operation/<id>` reports a queued operation as `pending`
|
||||
instead of answering 500. The queue keeps an operation's callback among
|
||||
its parameters until it runs, and the status route tried to send that
|
||||
function as JSON. An install queued behind another plugin's install
|
||||
failed every status poll until the first one finished. Parameters whose
|
||||
name starts with `_` are internal and are no longer in the answer.
|
||||
- A second click on Install while that plugin is still installing, or an
|
||||
Uninstall during its install, now answers 409 "already has an install,
|
||||
update or uninstall in progress" instead of 500 "An error occurred". The
|
||||
first operation carried on either way. The uninstall route also stopped
|
||||
recording a failed uninstall in the operation history for an uninstall
|
||||
that never started.
|
||||
- `/api/v3/plugins/<plugin_id>/static/<path>` serves images and other
|
||||
binary files. It opened every file as UTF-8 text, so a plugin's icon or
|
||||
preview image answered 500 `UnicodeDecodeError`. Files are now sent as
|
||||
they are on disk, an image with its own content type; HTML, JavaScript,
|
||||
CSS, JSON and other text keep the types they had. The path checks are
|
||||
unchanged.
|
||||
- The display schedule turns the panel off at exactly the end time. A window
|
||||
now runs from its start time up to, but not including, its end time: with
|
||||
07:00-23:00 the panel is on at 07:00 and off at 23:00. Before, the end
|
||||
@@ -571,10 +768,77 @@ policies are unchanged.
|
||||
the panel went off at 23:00 or at 23:01 depending on when in the minute
|
||||
that check ran. Windows that cross midnight and per-day schedules follow
|
||||
the same rule, and so does the dim schedule.
|
||||
- The MQTT bridge settings on the Tools tab can save a broker password with
|
||||
TLS off. The server refuses that unless `allow_insecure_mqtt` is set, and
|
||||
the form had no way to set it, so a password-protected broker on a home
|
||||
network without TLS could not be saved from the web UI, and once such a
|
||||
password was stored every later save failed too. While "Use TLS" is
|
||||
unchecked the form now shows "Allow without TLS (trusted network)",
|
||||
prefilled from the saved settings. It is off until ticked, so the server
|
||||
still refuses a cleartext password by default.
|
||||
- The Overview's plugin-config warning check stops polling. It asked
|
||||
`/api/v3/plugins/reconciliation-status` every 2 s until startup
|
||||
reconciliation reported done, and the route reports not done whenever its
|
||||
status file is missing: reconciliation raised before writing it, or /tmp
|
||||
was cleaned under a long-running web service. The page then sent that
|
||||
request every 2 s for as long as it stayed open, whichever tab was showing.
|
||||
It now gives up after a minute and only polls while the Overview is on
|
||||
screen.
|
||||
- Moving the Brightness slider on the Display tab no longer throws an error
|
||||
in the browser console on every step. Its handler also updated a "LED
|
||||
brightness" line that was removed from the page in #387; the lookup is
|
||||
gone.
|
||||
- Creating an API token on the General tab no longer leaves the page asking
|
||||
"Leave site?" on reload. The unsaved-changes guard marks a form when you
|
||||
type in it and clears the mark only after an htmx save, and the token form
|
||||
saves with a plain request, so it stayed marked after the token was
|
||||
created. It is cleared once the token is saved.
|
||||
- An on-demand session that ends during scheduled-off hours, by expiring or
|
||||
being stopped, blanks the panel within about a second. It used to stay on
|
||||
until the next minute, because the once-a-minute schedule check had
|
||||
already run that minute and the session had overridden its answer.
|
||||
- Check & Update All updates what is installed now. A second run in the
|
||||
same page sent the plugins the first run had seen, so a plugin uninstalled
|
||||
since then failed with "plugin not found" and one installed since was
|
||||
skipped. After a run the installed cards and the Updates badge show the
|
||||
new versions; they kept offering "Update to vX" for what had just been
|
||||
updated until the page was reloaded.
|
||||
- The Run On-Demand dialog lists a plugin's display modes, so a mode other
|
||||
than the first can be started, and pinned. `/api/v3/plugins/installed`
|
||||
never sent `display_modes`, which the dialog reads, so every plugin
|
||||
offered only its own id under "This plugin exposes a single display
|
||||
mode", and the display started its first mode. Each entry now carries
|
||||
`display_modes`, the modes its manifest declares.
|
||||
- Installing Weather, Music, Stocks or Leaderboard from the Plugin Store
|
||||
enables it, as installing any other plugin does. Each installs under the
|
||||
id its manifest declares (`ledmatrix-weather` for the store's `weather`),
|
||||
but the store enabled the store id, which `/api/v3/plugins/toggle`
|
||||
answered with "Plugin not found": the plugin stayed disabled behind
|
||||
"installed, but enabling it failed". `POST /api/v3/plugins/install` now
|
||||
answers with the installed `plugin_id` (in the operation's result when it
|
||||
is queued), and the store enables that.
|
||||
- Reinstalling a plugin from the Plugin Store leaves it enabled or disabled
|
||||
as it was. Reinstall enabled it as a fresh install does, so a plugin the
|
||||
user had switched off came back on.
|
||||
- A Plugin Store install that takes more than a minute is no longer
|
||||
reported as failed. The store stopped waiting after 60 s and showed
|
||||
"Install operation timed out" while the server, which allows the
|
||||
plugin's dependency install 300 s on its own, carried on and usually
|
||||
succeeded; the plugin was then neither enabled nor listed until the page
|
||||
was reloaded. The store now waits up to 10 minutes, and if it still has
|
||||
no answer it reloads the installed list and says the install may still
|
||||
be running.
|
||||
- The Plugin Store's category filter lists every category its plugins
|
||||
have. It offered a fixed seven while the registry uses about twenty, so
|
||||
plugins filed under productivity, utility, transit and the rest could not
|
||||
be filtered to, and "Financial" missed the plugin filed under "finance".
|
||||
The choices are now built from the store's plugins, as the Starlark
|
||||
section's are.
|
||||
- The Install button under Install Single Plugin (Plugin Manager > Install
|
||||
from GitHub) runs one handler per click. It also had an inline `onclick`
|
||||
whose handler threw a `ReferenceError` on every click; only the other
|
||||
handler's request went out, and making the inline one work would have
|
||||
sent every install twice. The inline handler is gone.
|
||||
- `/api/v3/plugins/installed` no longer reports the display's plugins as
|
||||
`live` while `/api/v3/health` says `display_loop: stalled`. The runtime
|
||||
snapshot is written from its own thread, which kept going while the render
|
||||
@@ -586,6 +850,17 @@ policies are unchanged.
|
||||
heartbeat when the service stops), is `stale` at once instead of `live`
|
||||
for up to 180 s. No new files or writes: both checks are on the reading
|
||||
side.
|
||||
- A scoreboard's scroll and Vegas cards with `scroll_card.date_format:
|
||||
"weekday"` now show the printed date's own weekday. A Friday 8 PM ET game
|
||||
read "Sat Oct 2". The card took the weekday in the plugin's own
|
||||
`timezone` setting, which ships blank, so it fell back to UTC, while the
|
||||
"Oct 2" beside it came from the zone the plugin actually resolves (its
|
||||
setting, then the global one, then the system zone). Every zone is within a
|
||||
day of UTC, so the card now finds which day near the start's UTC date has
|
||||
the printed month and day and names that one. Games east of UTC (Auckland,
|
||||
Kiritimati) were off by a day the other way and are fixed the same way.
|
||||
The switch-mode scorebug, which already used the plugin's resolved zone,
|
||||
shares the same formatter and draws what it drew before.
|
||||
- `/api/v3/display/current-status` reflects a wake from scheduled-off, a
|
||||
schedule-off blank, or an on-demand session starting or ending at once,
|
||||
even when the mode name stays the same. The display republished its
|
||||
|
||||
@@ -213,8 +213,9 @@ def list_installed_plugins(project_root: Path) -> List[Dict[str, Any]]:
|
||||
The plugins are the ``manifest.json`` files in the configured plugin
|
||||
directory (see :func:`_plugins_directory`), with the manifest's version;
|
||||
``enabled`` is config.json's flag by the display's rule (a missing flag
|
||||
is disabled). A restore reinstalls every listed plugin and takes enabled
|
||||
state from the restored config.json, so ``enabled`` is informational.
|
||||
is disabled). A restore installs each listed plugin that is missing and
|
||||
takes enabled state from the restored config.json, so ``enabled`` is
|
||||
informational.
|
||||
|
||||
``data/plugin_state.json`` is not read: it only ever repeated config's
|
||||
enabled flags and the manifests' versions, and is retired (nothing
|
||||
|
||||
Vendored
+40
-5
@@ -4,6 +4,7 @@ Disk Cache
|
||||
Handles persistent disk-based caching with atomic writes and error recovery.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
@@ -31,6 +32,35 @@ except ImportError: # pragma: no cover - exercised on hosts without the wheel
|
||||
# useful, and a half-written file was never useful.
|
||||
_ORPHAN_TEMP_MAX_AGE_SECONDS = 3600
|
||||
|
||||
# Longest key, in UTF-8 bytes, used verbatim as a filename stem. ext4 caps a
|
||||
# name at 255 bytes and set()'s temp file is ".<stem>.json.<8 random>", 15
|
||||
# bytes longer than the stem, so anything near the cap could never be written:
|
||||
# the calendar plugin's key joins every calendar id and passed 300 bytes on a
|
||||
# real install, failing every write with ENAMETOOLONG. Longer keys keep this
|
||||
# many bytes as a readable prefix and end in a hash of the whole key.
|
||||
_MAX_KEY_FILENAME_BYTES = 200
|
||||
_KEY_HASH_CHARS = 16
|
||||
|
||||
|
||||
def _filename_stem(key: str) -> str:
|
||||
"""The filename stem for a key that is already a safe path component.
|
||||
|
||||
Short keys are used as they are, so every file already on disk keeps its
|
||||
name. A long one becomes its first bytes plus a hash of the full key: the
|
||||
prefix keeps the stem recognisable (and keeps the data-type words that
|
||||
cleanup's retention lookup reads from it), the hash keeps two keys that
|
||||
share a long prefix apart. The result is itself short, so a stem read back
|
||||
from a filename -- which is how the web UI names a key it deletes -- maps to
|
||||
the same file.
|
||||
"""
|
||||
encoded = key.encode('utf-8')
|
||||
if len(encoded) <= _MAX_KEY_FILENAME_BYTES:
|
||||
return key
|
||||
digest = hashlib.sha256(encoded).hexdigest()[:_KEY_HASH_CHARS]
|
||||
keep = _MAX_KEY_FILENAME_BYTES - _KEY_HASH_CHARS - 1
|
||||
prefix = encoded[:keep].decode('utf-8', errors='ignore')
|
||||
return f"{prefix}-{digest}"
|
||||
|
||||
|
||||
|
||||
class CacheStrategyProtocol(Protocol):
|
||||
@@ -343,6 +373,8 @@ class DiskCache:
|
||||
derives them), so rejecting anything with a path component turns
|
||||
away only inputs that could never have been written here.
|
||||
|
||||
A key too long to be a filename is shortened by _filename_stem.
|
||||
|
||||
Args:
|
||||
key: Cache key
|
||||
|
||||
@@ -356,7 +388,7 @@ class DiskCache:
|
||||
if safe_key is None:
|
||||
self.logger.warning("Rejected unsafe cache key %r", key)
|
||||
return None
|
||||
return os.path.join(self.cache_dir, f"{safe_key}.json")
|
||||
return os.path.join(self.cache_dir, f"{_filename_stem(safe_key)}.json")
|
||||
|
||||
def get(self, key: str, max_age: Optional[int] = 300) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
@@ -561,7 +593,7 @@ class DiskCache:
|
||||
# If direct write also fails, try fallback location
|
||||
self.logger.warning("Direct write failed for key '%s' to %s: %s", key, cache_path, write_error)
|
||||
raise # Re-raise to trigger fallback logic
|
||||
except (IOError, OSError, PermissionError):
|
||||
except (IOError, OSError, PermissionError) as primary_error:
|
||||
# Attempt one-time fallback write to user's home cache directory
|
||||
try:
|
||||
# Try user's home cache directory as fallback
|
||||
@@ -587,11 +619,14 @@ class DiskCache:
|
||||
self.logger.debug("Fallback cache write also failed for key '%s': %s", key, e2)
|
||||
|
||||
# If all write attempts failed, log warning but don't raise exception
|
||||
# Cache is a performance optimization, not critical for operation
|
||||
# Cache is a performance optimization, not critical for operation.
|
||||
# Name the real error: this used to say "permission denied"
|
||||
# whatever happened, which sent a too-long filename off to
|
||||
# be debugged as a directory-ownership problem.
|
||||
self.logger.warning(
|
||||
"Could not write cache for key '%s' to %s (permission denied). "
|
||||
"Could not write cache for key '%s' to %s (%s). "
|
||||
"Cache will be unavailable for this key, but application will continue.",
|
||||
key, cache_path
|
||||
key, cache_path, primary_error.strerror or primary_error
|
||||
)
|
||||
return # Exit gracefully without raising exception
|
||||
|
||||
|
||||
+34
-2
@@ -46,6 +46,32 @@ from src.cache.disk_cache import DateTimeEncoder # noqa: F401 - deliberate re-e
|
||||
# CacheManager.config_manager not built yet (None means "not available").
|
||||
_UNSET: Any = object()
|
||||
|
||||
|
||||
def _outlived(record: Any, max_age: Optional[float], now: float) -> bool:
|
||||
"""Whether a record's own timestamp puts it past max_age.
|
||||
|
||||
The memory tier times an entry from when it was put there, and a record
|
||||
loaded from disk is put there when it is read, not when it was written: a
|
||||
record 290 s old, read after a restart, could be served for another
|
||||
max_age from memory. This is the age check DiskCache.get makes, with the
|
||||
same rule that a stored ttl wins over the caller's max_age. A record that
|
||||
carries no timestamp is left to the memory tier's own clock.
|
||||
"""
|
||||
if not isinstance(record, dict):
|
||||
return False
|
||||
stored_ttl = record.get('ttl')
|
||||
if isinstance(stored_ttl, (int, float)) and not isinstance(stored_ttl, bool) \
|
||||
and stored_ttl >= 0:
|
||||
max_age = stored_ttl
|
||||
stamp = record.get('timestamp')
|
||||
if max_age is None or stamp is None or isinstance(stamp, bool):
|
||||
return False
|
||||
try:
|
||||
return now - float(stamp) > max_age
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
|
||||
|
||||
class CacheManager:
|
||||
"""Manages caching of API responses to reduce API calls."""
|
||||
|
||||
@@ -284,7 +310,11 @@ class CacheManager:
|
||||
# 1) Memory cache
|
||||
cached = self._memory_cache_component.get(key, max_age=in_memory_ttl)
|
||||
if cached is not None:
|
||||
return cached
|
||||
if not _outlived(cached, max_age, time.time()):
|
||||
return cached
|
||||
# Too old for this reader. Disk may hold a newer write (from the
|
||||
# other process), and if it does not, the miss is the right answer.
|
||||
self._memory_cache_component.clear(key)
|
||||
|
||||
# 2) Disk cache
|
||||
record = self._disk_cache_component.get(key, max_age=max_age)
|
||||
@@ -318,7 +348,9 @@ class CacheManager:
|
||||
# Check memory cache first (1 minute TTL)
|
||||
cached = self._memory_cache_component.get(key, max_age=60)
|
||||
if cached is not None:
|
||||
return cached
|
||||
if not _outlived(cached, 3600, time.time()):
|
||||
return cached
|
||||
self._memory_cache_component.clear(key)
|
||||
|
||||
# Check disk cache
|
||||
data = self._disk_cache_component.get(key, max_age=3600) # 1 hour for load_cache
|
||||
|
||||
+24
-24
@@ -561,7 +561,7 @@ class ScrollHelper:
|
||||
width = self.display_width
|
||||
strip_width = self.cached_array.shape[1]
|
||||
|
||||
if start_x + width + 1 <= strip_width:
|
||||
if 0 <= start_x and start_x + width + 1 <= strip_width:
|
||||
# Slice the backing array directly. Going via
|
||||
# _get_visible_portion_integer would build two PIL images only for
|
||||
# them to be converted straight back to arrays, which measured 15x
|
||||
@@ -569,9 +569,10 @@ class ScrollHelper:
|
||||
near = self.cached_array[:, start_x:start_x + width]
|
||||
far = self.cached_array[:, start_x + 1:start_x + 1 + width]
|
||||
else:
|
||||
# Close enough to the end that one of the slices wraps; let the
|
||||
# integer path handle that and pay the conversion. Continuous mode
|
||||
# extends the strip before reaching here, so this is the rare case.
|
||||
# One of the slices wraps (close to the end, or a strip narrower
|
||||
# than the panel); let the integer path handle that and pay the
|
||||
# conversion. Continuous mode extends the strip before reaching
|
||||
# here, so this is the rare case.
|
||||
near = np.asarray(
|
||||
self._get_visible_portion_integer(start_x, start_x + width))
|
||||
far = np.asarray(
|
||||
@@ -601,34 +602,33 @@ class ScrollHelper:
|
||||
_size = (self.display_width, self.display_height)
|
||||
img_w = self.cached_array.shape[1]
|
||||
|
||||
if end_x <= img_w:
|
||||
if 0 <= start_x and end_x <= img_w:
|
||||
# Normal case: single contiguous slice (fastest path). tobytes()
|
||||
# on the column-slice view already returns C-order bytes, so
|
||||
# ascontiguousarray() first only added a second full-frame copy.
|
||||
return Image.frombytes(
|
||||
'RGB', _size,
|
||||
self.cached_array[:, start_x:end_x].tobytes())
|
||||
|
||||
# Ensure frame buffer is allocated for all non-simple paths
|
||||
if self._frame_buffer is None or self._frame_buffer.shape != (self.display_height, self.display_width, 3):
|
||||
self._frame_buffer = np.zeros((self.display_height, self.display_width, 3), dtype=np.uint8)
|
||||
|
||||
if img_w == 0:
|
||||
self._frame_buffer[:] = 0
|
||||
else:
|
||||
# Ensure frame buffer is allocated for all non-simple paths
|
||||
if self._frame_buffer is None or self._frame_buffer.shape != (self.display_height, self.display_width, 3):
|
||||
self._frame_buffer = np.zeros((self.display_height, self.display_width, 3), dtype=np.uint8)
|
||||
# The frame runs off the strip, so it carries on from the head:
|
||||
# frame column j is strip column (start_x + j) modulo the strip's
|
||||
# width -- the tail and then the head, and a strip narrower than
|
||||
# the panel repeated across it. Copying the tail and then the rest
|
||||
# of the frame from the head assumed the head was that wide, and
|
||||
# raised at every position for a strip narrower than the panel
|
||||
# (Vegas composes one, with no lead-in, when its content is
|
||||
# narrower than the chain).
|
||||
np.take(self.cached_array, np.arange(start_x, end_x), axis=1,
|
||||
mode='wrap', out=self._frame_buffer)
|
||||
|
||||
width1 = img_w - start_x
|
||||
if width1 > 0:
|
||||
# Wrap-around: tail of image + head of image
|
||||
self._frame_buffer[:, :width1] = self.cached_array[:, start_x:]
|
||||
remaining_width = self.display_width - width1
|
||||
self._frame_buffer[:, width1:] = self.cached_array[:, :remaining_width]
|
||||
else:
|
||||
# Edge case: start_x at or past image end — show from beginning,
|
||||
# clamped to available width (scroll_position should wrap before
|
||||
# reaching this state in normal operation).
|
||||
available = min(self.display_width, img_w)
|
||||
self._frame_buffer[:, :available] = self.cached_array[:, :available]
|
||||
if available < self.display_width:
|
||||
self._frame_buffer[:, available:] = 0
|
||||
|
||||
return Image.frombytes('RGB', _size, self._frame_buffer.tobytes())
|
||||
return Image.frombytes('RGB', _size, self._frame_buffer.tobytes())
|
||||
|
||||
def calculate_dynamic_duration(self) -> int:
|
||||
"""
|
||||
|
||||
@@ -18,7 +18,7 @@ the extra guard only stops a None size raising TypeError.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
@@ -338,10 +338,46 @@ def format_game_date(config: Optional[Dict[str, Any]], logger, date_text: str,
|
||||
if not raw:
|
||||
return ""
|
||||
fmt = str(scroll_card_option(config, "date_format", "abbrev") or "abbrev")
|
||||
return _format_date_as(fmt, raw, lambda: weekday_for(config, logger, game))
|
||||
return _format_date_as(fmt, raw, lambda: weekday_for(config, logger, game),
|
||||
game=game)
|
||||
|
||||
|
||||
def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
|
||||
def _printed_weekday(game: Optional[Dict], month: int, day: int) -> str:
|
||||
"""The weekday of the date a card prints as month/day, or '' if unknown.
|
||||
|
||||
The extractor prints "M/D" in the plugin's resolved zone (its own setting,
|
||||
else the global one, else the system zone). The card cannot see that zone:
|
||||
it is handed the plugin's config, whose ``timezone`` ships as "", so
|
||||
card_tzinfo answers UTC and an evening kickoff in the Americas got the
|
||||
next day's weekday ("Sat Oct 2" for a Friday game). Every zone is within
|
||||
a day of UTC, so the printed date is the start's UTC date or a neighbour
|
||||
of it; the one with that month and day is the date on the card.
|
||||
"""
|
||||
if not isinstance(game, dict):
|
||||
return ""
|
||||
raw = game.get("start_time_utc") or game.get("start_time")
|
||||
if not raw:
|
||||
return ""
|
||||
try:
|
||||
start = raw if isinstance(raw, datetime) else datetime.fromisoformat(
|
||||
str(raw).replace("Z", "+00:00"))
|
||||
if start.utcoffset() is None:
|
||||
return "" # naive: no instant to place the date against
|
||||
utc_day = start.astimezone(timezone.utc).date()
|
||||
except (ValueError, TypeError, OverflowError):
|
||||
return ""
|
||||
for offset in (0, -1, 1):
|
||||
try:
|
||||
candidate = utc_day + timedelta(days=offset)
|
||||
except OverflowError:
|
||||
continue
|
||||
if (candidate.month, candidate.day) == (month, day):
|
||||
return WEEKDAY_ABBR[candidate.weekday()]
|
||||
return ""
|
||||
|
||||
|
||||
def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR,
|
||||
game: Optional[Dict] = None) -> str:
|
||||
"""Render a stripped, non-empty "M/D" *raw* in style *fmt*.
|
||||
|
||||
The body both date formatters share. They differ in which setting names the
|
||||
@@ -349,6 +385,9 @@ def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
|
||||
``SportsCoreSharedMixin._format_game_date``), so those arrive as arguments:
|
||||
*weekday* is a zero-argument callable, only called for the "weekday" style.
|
||||
*months* lets the mixin keep reading its (overridable) ``_MONTH_ABBR``.
|
||||
With *game*, the "weekday" style names the printed date's own weekday
|
||||
(:func:`_printed_weekday`), and *weekday* is only the fallback for a
|
||||
date its start time cannot place.
|
||||
"""
|
||||
if fmt == "numeric":
|
||||
return raw
|
||||
@@ -364,7 +403,7 @@ def _format_date_as(fmt: str, raw: str, weekday, months=MONTH_ABBR) -> str:
|
||||
if fmt == "day_first":
|
||||
return f"{day} {name}"
|
||||
if fmt == "weekday":
|
||||
day_name = weekday()
|
||||
day_name = _printed_weekday(game, month, day) or weekday()
|
||||
return f"{day_name} {name} {day}" if day_name else f"{name} {day}"
|
||||
return f"{name} {day}"
|
||||
|
||||
|
||||
@@ -360,14 +360,16 @@ class SportsCoreSharedMixin:
|
||||
The formatting is sports_card's. What differs from the card's
|
||||
``format_game_date`` is passed in: the setting (``switch_date_format``,
|
||||
see :meth:`_switch_date_format`) and the weekday, which comes from
|
||||
:meth:`_weekday_for` and so from this plugin's resolved timezone.
|
||||
:meth:`_weekday_for` and so from this plugin's resolved timezone
|
||||
when the game's start cannot place the printed date. The game goes
|
||||
in too, so both formatters name the printed date's own weekday.
|
||||
"""
|
||||
raw = str(date_text or "").strip()
|
||||
if not raw:
|
||||
return raw
|
||||
return _card._format_date_as(self._switch_date_format(), raw,
|
||||
lambda: self._weekday_for(game),
|
||||
self._MONTH_ABBR)
|
||||
self._MONTH_ABBR, game=game)
|
||||
|
||||
def _weekday_for(self, game: Optional[Dict]) -> str:
|
||||
"""Weekday abbreviation from the game's start time, or ''."""
|
||||
|
||||
+43
-10
@@ -46,6 +46,35 @@ from src.common.permission_utils import (
|
||||
get_config_dir_mode
|
||||
)
|
||||
|
||||
|
||||
def _private_copy(config: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""A deep copy of ``config`` that shares nothing with it.
|
||||
|
||||
load_config() hands one out per call, and the saves keep one, so the
|
||||
cached config is never an object a caller holds. A web handler edits what
|
||||
it loaded, validates, and may refuse the save; when the cache was that
|
||||
same object, the refused edit stayed in it, and the next save of any
|
||||
other setting wrote it to config.json -- a nested secret included, in
|
||||
plain text, since it had never reached config_secrets.json to be
|
||||
stripped.
|
||||
|
||||
The config is JSON data, so only its dicts and lists need copying; every
|
||||
other value in it is immutable. On a Pi 4 with a real 60 KiB config this
|
||||
takes 2.1 ms against copy.deepcopy's 6.8 ms, on a path ~30 handlers call
|
||||
(a pickle round trip is no faster, 1.9 ms, and brings pickle into the
|
||||
config path for nothing).
|
||||
"""
|
||||
return _copy_containers(config)
|
||||
|
||||
|
||||
def _copy_containers(value: Any) -> Any:
|
||||
if isinstance(value, dict):
|
||||
return {key: _copy_containers(item) for key, item in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_copy_containers(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
class ConfigManager:
|
||||
"""
|
||||
Reads and writes the main application configuration files.
|
||||
@@ -126,9 +155,10 @@ class ConfigManager:
|
||||
validate_after_write=validate_after_write
|
||||
)
|
||||
|
||||
# Update in-memory config if save was successful
|
||||
# Update in-memory config if save was successful. A copy: the caller
|
||||
# still holds new_config_data (see _private_copy).
|
||||
if result.status == SaveResultStatus.SUCCESS:
|
||||
self.config = new_config_data
|
||||
self.config = _private_copy(new_config_data)
|
||||
# In-memory config now matches what was just written, so the
|
||||
# load_config fast path may return it. It still carries the
|
||||
# merged secrets that were stripped on disk; that matches a full
|
||||
@@ -208,14 +238,16 @@ class ConfigManager:
|
||||
|
||||
Fast path: when config.json, config_secrets.json and the template
|
||||
are all unchanged since the last successful load (mtime_ns + size),
|
||||
the already-parsed self.config is returned without touching the
|
||||
files — same aliasing semantics as the full path, which also
|
||||
returns self.config.
|
||||
a copy of the already-parsed self.config is returned without
|
||||
touching the files.
|
||||
|
||||
Either way the caller gets its own copy (see _private_copy): editing
|
||||
it changes nothing here until it is saved.
|
||||
"""
|
||||
try:
|
||||
current_sig = self._files_signature()
|
||||
if self.config and self._loaded_sig == current_sig:
|
||||
return self.config
|
||||
return _private_copy(self.config)
|
||||
|
||||
# Check if config file exists, if not create from template
|
||||
if not os.path.exists(self.config_path):
|
||||
@@ -249,8 +281,8 @@ class ConfigManager:
|
||||
# Signature taken AFTER load + migration (migration may write the
|
||||
# config back), so it reflects exactly what was read/written.
|
||||
self._loaded_sig = self._files_signature()
|
||||
return self.config
|
||||
|
||||
return _private_copy(self.config)
|
||||
|
||||
except FileNotFoundError as e:
|
||||
# Only config.json can get here: a missing or unreadable secrets
|
||||
# file is handled where it is read.
|
||||
@@ -355,8 +387,9 @@ class ConfigManager:
|
||||
try:
|
||||
atomic_write_json(self.config_path, config_to_write)
|
||||
|
||||
# Update the in-memory config to the new state (which includes secrets for runtime)
|
||||
self.config = new_config_data
|
||||
# Update the in-memory config to the new state (which includes
|
||||
# secrets for runtime), as a copy -- see _private_copy
|
||||
self.config = _private_copy(new_config_data)
|
||||
self._loaded_sig = self._files_signature()
|
||||
self.logger.info(f"Configuration successfully saved to {os.path.abspath(self.config_path)}")
|
||||
if secrets_content:
|
||||
|
||||
+92
-42
@@ -14,7 +14,7 @@ import json
|
||||
import time
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional, List, Callable
|
||||
from typing import Dict, Any, Optional, List, Callable, Tuple
|
||||
from collections import defaultdict
|
||||
import logging
|
||||
import hashlib
|
||||
@@ -52,7 +52,18 @@ class ConfigService:
|
||||
|
||||
# Thread safety
|
||||
self._lock: threading.RLock = threading.RLock()
|
||||
|
||||
# Held across a whole reload -- read, swap, notify -- so one reload's
|
||||
# notifications finish before the next one's start. Subscribers run
|
||||
# under this lock and never under _lock: the display's per-plugin
|
||||
# subscriber can wait seconds for a busy plugin, and get_config(),
|
||||
# subscribe() and unsubscribe() -- called from the render thread --
|
||||
# must not wait behind it.
|
||||
self._notify_lock: threading.RLock = threading.RLock()
|
||||
# (key, callback, thread id) of the callback a notification is running,
|
||||
# so unsubscribe() can wait for that one call; signalled on its return.
|
||||
self._running_callback: Optional[Tuple[str, Callable[..., None], int]] = None
|
||||
self._callback_done = threading.Condition(self._lock)
|
||||
|
||||
# Current configuration
|
||||
self._current_config: Dict[str, Any] = {}
|
||||
self._current_checksum: Optional[str] = None
|
||||
@@ -87,32 +98,33 @@ class ConfigService:
|
||||
True if config changed, False otherwise
|
||||
"""
|
||||
try:
|
||||
new_config = self.config_manager.load_config()
|
||||
new_checksum = self._calculate_checksum(new_config)
|
||||
|
||||
with self._lock:
|
||||
# Check if config actually changed
|
||||
if new_checksum == self._current_checksum:
|
||||
self.logger.debug("Configuration unchanged, skipping reload")
|
||||
return False
|
||||
|
||||
# Store old config for change detection
|
||||
old_config = self._current_config.copy()
|
||||
|
||||
# Update current config
|
||||
self._current_config = new_config
|
||||
self._current_checksum = new_checksum
|
||||
|
||||
# Notify subscribers
|
||||
with self._notify_lock:
|
||||
new_config = self.config_manager.load_config()
|
||||
new_checksum = self._calculate_checksum(new_config)
|
||||
|
||||
with self._lock:
|
||||
# Check if config actually changed
|
||||
if new_checksum == self._current_checksum:
|
||||
self.logger.debug("Configuration unchanged, skipping reload")
|
||||
return False
|
||||
|
||||
# Store old config for change detection
|
||||
old_config = self._current_config.copy()
|
||||
|
||||
# Update current config
|
||||
self._current_config = new_config
|
||||
self._current_checksum = new_checksum
|
||||
|
||||
# Notify subscribers, outside _lock (see _notify_lock)
|
||||
self._notify_subscribers(old_config, new_config)
|
||||
|
||||
|
||||
self.logger.info(
|
||||
"Configuration reloaded (checksum: %s)",
|
||||
new_checksum[:8]
|
||||
)
|
||||
|
||||
|
||||
return True
|
||||
|
||||
|
||||
except ConfigError as e:
|
||||
self.logger.error("Error loading configuration: %s", e, exc_info=True)
|
||||
return False
|
||||
@@ -127,35 +139,64 @@ class ConfigService:
|
||||
Args:
|
||||
old_config: Previous configuration
|
||||
new_config: New configuration
|
||||
|
||||
Called without _lock held. The subscriber lists are copied under it,
|
||||
and each callback is checked against them again just before it runs.
|
||||
"""
|
||||
with self._lock:
|
||||
subscribers = {key: list(callbacks) for key, callbacks in self._subscribers.items()}
|
||||
|
||||
# Notify global subscribers (key: '*')
|
||||
for callback in self._subscribers.get('*', []):
|
||||
try:
|
||||
callback(old_config, new_config)
|
||||
except Exception as e:
|
||||
self.logger.error("Error in global config change callback: %s", e, exc_info=True)
|
||||
|
||||
for callback in subscribers.get('*', []):
|
||||
self._call_subscriber('*', callback, old_config, new_config)
|
||||
|
||||
# Notify plugin-specific subscribers
|
||||
for plugin_id in self._subscribers.keys():
|
||||
for plugin_id, callbacks in subscribers.items():
|
||||
if plugin_id == '*':
|
||||
continue
|
||||
|
||||
|
||||
old_plugin_config = old_config.get(plugin_id, {})
|
||||
new_plugin_config = new_config.get(plugin_id, {})
|
||||
|
||||
|
||||
# Only notify if plugin config actually changed
|
||||
if old_plugin_config != new_plugin_config:
|
||||
for callback in self._subscribers[plugin_id]:
|
||||
try:
|
||||
callback(old_plugin_config, new_plugin_config)
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
"Error in config change callback for %s: %s",
|
||||
plugin_id,
|
||||
e,
|
||||
exc_info=True
|
||||
)
|
||||
|
||||
for callback in callbacks:
|
||||
self._call_subscriber(plugin_id, callback,
|
||||
old_plugin_config, new_plugin_config)
|
||||
|
||||
def _call_subscriber(
|
||||
self,
|
||||
key: str,
|
||||
callback: Callable[[Dict[str, Any], Dict[str, Any]], None],
|
||||
old_config: Dict[str, Any],
|
||||
new_config: Dict[str, Any],
|
||||
) -> None:
|
||||
"""Run one callback, unless it was unsubscribed since the snapshot.
|
||||
|
||||
unsubscribe() promises that once it returns the callback is neither
|
||||
running nor will run: the display unloads the plugin straight after.
|
||||
"""
|
||||
with self._lock:
|
||||
if callback not in self._subscribers.get(key, ()):
|
||||
return
|
||||
self._running_callback = (key, callback, threading.get_ident())
|
||||
try:
|
||||
callback(old_config, new_config)
|
||||
except Exception as e:
|
||||
if key == '*':
|
||||
self.logger.error("Error in global config change callback: %s", e, exc_info=True)
|
||||
else:
|
||||
self.logger.error(
|
||||
"Error in config change callback for %s: %s",
|
||||
key,
|
||||
e,
|
||||
exc_info=True
|
||||
)
|
||||
finally:
|
||||
with self._lock:
|
||||
self._running_callback = None
|
||||
self._callback_done.notify_all()
|
||||
|
||||
def _check_file_changes(self) -> bool:
|
||||
"""
|
||||
Check if configuration files have been modified.
|
||||
@@ -276,6 +317,11 @@ class ConfigService:
|
||||
"""
|
||||
Unsubscribe from configuration changes.
|
||||
|
||||
Once this returns the callback is not running and will not be called
|
||||
again. A notification that is running this very callback is waited
|
||||
for (unless the callback is the caller); one running any other
|
||||
callback is not.
|
||||
|
||||
Args:
|
||||
callback: Callback function to remove
|
||||
plugin_id: Optional plugin ID (must match subscription)
|
||||
@@ -285,6 +331,10 @@ class ConfigService:
|
||||
if callback in self._subscribers[key]:
|
||||
self._subscribers[key].remove(callback)
|
||||
self.logger.debug("Unsubscribed from config changes for %s", key)
|
||||
while (self._running_callback is not None
|
||||
and self._running_callback[:2] == (key, callback)
|
||||
and self._running_callback[2] != threading.get_ident()):
|
||||
self._callback_done.wait()
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""Shutdown the configuration service."""
|
||||
|
||||
+48
-10
@@ -25,11 +25,12 @@ import os
|
||||
import inspect
|
||||
import signal
|
||||
import json
|
||||
import math
|
||||
import threading
|
||||
import types
|
||||
from collections import deque
|
||||
from contextlib import contextmanager
|
||||
from typing import Dict, Any, List, Optional, Callable, Set, Tuple
|
||||
from typing import Dict, Any, FrozenSet, List, Optional, Callable, Set, Tuple
|
||||
from datetime import datetime
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed # pylint: disable=no-name-in-module
|
||||
import pytz
|
||||
@@ -89,6 +90,19 @@ _MIN_INITIAL_UPDATE_TIMEOUT_SECONDS = 2.0
|
||||
DEFAULT_DYNAMIC_DURATION_CAP = 180.0
|
||||
|
||||
|
||||
def _finite_seconds(value: Any) -> Optional[float]:
|
||||
"""``value`` as seconds when it is a finite number or a numeric string,
|
||||
else None. A bool is not a number here, though it is an int: True would
|
||||
read as a one-second screen."""
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
try:
|
||||
seconds = float(value)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
return None
|
||||
return seconds if math.isfinite(seconds) else None
|
||||
|
||||
|
||||
class _PluginReloadJob:
|
||||
"""A ``plugin.reload`` whose slow half runs off the render thread.
|
||||
|
||||
@@ -1201,13 +1215,6 @@ class DisplayController:
|
||||
note = getattr(self.plugin_manager, 'note_display_duration', None)
|
||||
if note is not None and plugin_id:
|
||||
note(plugin_id, time.monotonic() - started)
|
||||
# A screen that drew once and holds makes no more
|
||||
# update_display() calls, so a frame the preview throttle
|
||||
# skipped would otherwise never reach the snapshot.
|
||||
write_owed = getattr(getattr(self, 'display_manager', None),
|
||||
'write_owed_snapshot', None)
|
||||
if write_owed is not None:
|
||||
write_owed()
|
||||
|
||||
def _health_tracker(self):
|
||||
"""The plugin circuit breaker, or None when it is not enabled."""
|
||||
@@ -1347,6 +1354,12 @@ class DisplayController:
|
||||
"until one does", self.EMPTY_ROTATION_PAUSE)
|
||||
self._sleep_with_plugin_updates(self.EMPTY_ROTATION_PAUSE)
|
||||
|
||||
#: Plugins already warned about a display duration that is not a number,
|
||||
#: so a bad setting logs once, not at every one of its screens. A
|
||||
#: frozenset, replaced rather than mutated; class-level default for
|
||||
#: controllers built without __init__ (tests).
|
||||
_duration_warned: FrozenSet[str] = frozenset()
|
||||
|
||||
def _get_display_duration(self, mode_key):
|
||||
"""Seconds to show a mode: the Rotation & Durations page's value for it
|
||||
(display.display_durations), else the plugin's own duration.
|
||||
@@ -1354,6 +1367,17 @@ class DisplayController:
|
||||
The saved value has to win. Every plugin inherits
|
||||
get_display_duration(), so checking the plugin first meant the page's
|
||||
values were never read.
|
||||
|
||||
The plugin's answer is checked here, not trusted. Several plugins
|
||||
return their display_duration setting straight from config.json, so
|
||||
one saved as "20" or null (the raw config editor, a hand edit) came
|
||||
back as a string or None; _resolve_durations compared it with 0, and
|
||||
the TypeError went past every handler in the loop and stopped the
|
||||
display service, which systemd restarted into the same screen. A
|
||||
numeric string counts, as in BasePlugin.get_display_duration; any
|
||||
other value that is not a finite number, or a raise, gets the 30 s a
|
||||
mode without a plugin gets. A number at or below zero is passed on:
|
||||
_resolve_durations has its own rule for that.
|
||||
"""
|
||||
display_durations = self.config.get('display', {}).get('display_durations', {}) or {}
|
||||
override = display_durations.get(mode_key)
|
||||
@@ -1361,8 +1385,22 @@ class DisplayController:
|
||||
return float(override)
|
||||
|
||||
plugin_instance = self.plugin_modes.get(mode_key)
|
||||
if plugin_instance is not None and hasattr(plugin_instance, 'get_display_duration'):
|
||||
return plugin_instance.get_display_duration()
|
||||
if plugin_instance is None or not hasattr(plugin_instance, 'get_display_duration'):
|
||||
return 30
|
||||
try:
|
||||
value = plugin_instance.get_display_duration()
|
||||
except Exception as err: # pylint: disable=broad-except
|
||||
problem = f"get_display_duration() raised {type(err).__name__}: {err}"
|
||||
else:
|
||||
seconds = _finite_seconds(value)
|
||||
if seconds is not None:
|
||||
return seconds
|
||||
problem = f"display duration {value!r} is not a number"
|
||||
plugin_id = getattr(plugin_instance, 'plugin_id', None) or mode_key
|
||||
if plugin_id not in self._duration_warned:
|
||||
self._duration_warned = self._duration_warned | {plugin_id}
|
||||
logger.warning("Plugin %s: %s; showing its modes for 30s (logged once)",
|
||||
plugin_id, problem)
|
||||
return 30
|
||||
|
||||
def _get_global_dynamic_cap(self) -> Optional[float]:
|
||||
|
||||
+2
-46
@@ -317,11 +317,6 @@ class DisplayManager:
|
||||
# is handed to the writer; this only once it has been saved, so an
|
||||
# mtime touch never vouches for a frame still waiting to be written.
|
||||
self._saved_snapshot_digest: Optional[int] = None
|
||||
# A changed frame reached _write_snapshot_if_due() inside the write
|
||||
# interval and was skipped. Nothing writes it unless update_display()
|
||||
# runs again, and a screen that draws once and holds never calls it
|
||||
# again -- see write_owed_snapshot().
|
||||
self._snapshot_owed = False
|
||||
self._snapshot_dir_prepared = False
|
||||
# Background writer used mid-scroll; see _write_snapshot_if_due.
|
||||
self._snapshot_cond = threading.Condition()
|
||||
@@ -1793,10 +1788,9 @@ class DisplayManager:
|
||||
|
||||
if frame_checksum is not None:
|
||||
digest = frame_checksum
|
||||
frame_changed = digest != self._last_snapshot_digest
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, frame_changed)
|
||||
viewer_fresh, digest != self._last_snapshot_digest)
|
||||
else:
|
||||
# Ask as if the frame had changed before paying to find out.
|
||||
# decide() is monotone in frame_changed -- a SKIP for a
|
||||
@@ -1808,24 +1802,15 @@ class DisplayManager:
|
||||
now, self._last_snapshot_ts, self._last_snapshot_touch_ts,
|
||||
viewer_fresh, True)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
# Not hashed, so not known to be unchanged: owed until a
|
||||
# later look finds it written or unchanged.
|
||||
self._snapshot_owed = True
|
||||
return
|
||||
digest = zlib.adler32(self.image.tobytes())
|
||||
frame_changed = digest != self._last_snapshot_digest
|
||||
if not frame_changed:
|
||||
if digest == self._last_snapshot_digest:
|
||||
# Unchanged after all: the decision an unchanged frame gets.
|
||||
action = snapshot_policy.decide(
|
||||
now, self._last_snapshot_ts,
|
||||
self._last_snapshot_touch_ts, viewer_fresh, False)
|
||||
if action is snapshot_policy.SnapshotAction.SKIP:
|
||||
# A changed frame inside the write interval stays owed: the
|
||||
# next update_display() would write it, but a static screen
|
||||
# may not make one -- write_owed_snapshot() covers that.
|
||||
self._snapshot_owed = frame_changed
|
||||
return
|
||||
self._snapshot_owed = False
|
||||
if (action is snapshot_policy.SnapshotAction.TOUCH
|
||||
and self._saved_snapshot_digest == digest):
|
||||
# mtime bump only: keeps the health check (snapshot age)
|
||||
@@ -1860,35 +1845,6 @@ class DisplayManager:
|
||||
except Exception as e:
|
||||
self._log_snapshot_failure(e)
|
||||
|
||||
def write_owed_snapshot(self) -> None:
|
||||
"""Write a frame the snapshot throttle skipped, once it is due.
|
||||
|
||||
The preview snapshot is only ever written from update_display(), and
|
||||
at most once per write interval (snapshot_policy). A frame pushed
|
||||
inside that interval is skipped, and is written by the next
|
||||
update_display() that comes after it -- but a screen that draws its
|
||||
card once and then holds it makes no further call. Its frame was on
|
||||
the panel and never in the preview: soccer's recent/upcoming cards
|
||||
skip redundant redraws, and the first one after an on-demand start
|
||||
(pushed a few milliseconds after the controller's clear) left
|
||||
/api/v3/display/current and the web preview black for the whole
|
||||
screen while the panel showed the card.
|
||||
|
||||
The render loop calls this after each frame. Cheap when nothing is
|
||||
owed (one attribute read); otherwise the usual policy decides, so
|
||||
the write still waits out the interval and an unchanged frame is
|
||||
never re-encoded.
|
||||
"""
|
||||
if not self._snapshot_owed:
|
||||
return
|
||||
try:
|
||||
if self._writes_suppressed():
|
||||
return
|
||||
with self._update_lock:
|
||||
self._write_snapshot_if_due()
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
self._log_snapshot_failure(e)
|
||||
|
||||
def _log_snapshot_failure(self, error: Exception) -> None:
|
||||
# Snapshot failures must never break display — but they must not
|
||||
# be silent either: the snapshot's mtime is the web UI's display
|
||||
|
||||
+6
-1
@@ -396,9 +396,9 @@ class StateSubscription:
|
||||
def _run(self) -> None:
|
||||
backoff = _RECONNECT_MIN_SECONDS
|
||||
while not self._stop.is_set():
|
||||
snapshots = self.snapshots
|
||||
try:
|
||||
self._follow()
|
||||
backoff = _RECONNECT_MIN_SECONDS
|
||||
except ControlError as e:
|
||||
self.last_error = e.reason
|
||||
if e.reason in _SLOW_RETRY_REASONS:
|
||||
@@ -414,6 +414,11 @@ class StateSubscription:
|
||||
sock.close()
|
||||
except OSError:
|
||||
pass
|
||||
if self.snapshots != snapshots:
|
||||
# This connection got as far as the display's state: whatever
|
||||
# ended it (a restart, most often), it was working, so the
|
||||
# next try starts from the shortest wait again.
|
||||
backoff = _RECONNECT_MIN_SECONDS
|
||||
if self._stop.wait(backoff):
|
||||
return
|
||||
backoff = min(backoff * 2, _RECONNECT_MAX_SECONDS)
|
||||
|
||||
@@ -48,12 +48,19 @@ class PluginOperation:
|
||||
completed_at: Optional[datetime] = None
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert operation to dictionary for serialization."""
|
||||
"""Convert operation to dictionary for serialization.
|
||||
|
||||
Parameters whose name starts with ``_`` are internal and left out:
|
||||
PluginOperationQueue keeps the operation's callback there as
|
||||
``_callback`` until its worker runs it, and a pending operation's
|
||||
status answered 500 because that function cannot be serialized.
|
||||
"""
|
||||
return {
|
||||
'operation_id': self.operation_id,
|
||||
'operation_type': self.operation_type.value,
|
||||
'plugin_id': self.plugin_id,
|
||||
'parameters': self.parameters,
|
||||
'parameters': {key: value for key, value in self.parameters.items()
|
||||
if not str(key).startswith('_')},
|
||||
'status': self.status.value,
|
||||
'progress': self.progress,
|
||||
'message': self.message,
|
||||
|
||||
@@ -92,7 +92,10 @@ class PluginExecutor:
|
||||
with plugin_scope(plugin_id):
|
||||
result_container['value'] = operation()
|
||||
result_container['completed'] = True
|
||||
except Exception as e:
|
||||
except BaseException as e: # pylint: disable=broad-except
|
||||
# asyncio.CancelledError and SystemExit too: uncaught, one
|
||||
# ended this thread with 'completed' unset, and an operation
|
||||
# that failed at once was reported as timing out.
|
||||
result_container['exception'] = e
|
||||
result_container['completed'] = True
|
||||
|
||||
|
||||
@@ -199,9 +199,22 @@ def contained_plugin_dir(plugin_dir: Path, plugins_dir: Path) -> Optional[str]:
|
||||
name that came out of ``os.scandir()`` on the trusted root carries no
|
||||
taint, which is a real containment guarantee (and one CodeQL's
|
||||
path-injection query can follow), not a string sanitiser.
|
||||
|
||||
The entry looked for is the one ``plugin_dir`` itself names when it sits
|
||||
directly in ``plugins_dir``: for a dev plugin symlinked in under its id,
|
||||
the link's name. Resolving the link first and looking for the target's
|
||||
folder name refused ``plugins/foo -> ~/.ledmatrix-dev-plugins/ledmatrix-foo``
|
||||
(what ``dev_plugin_setup.sh link-github foo <url>`` makes), so the plugin
|
||||
never loaded. Any other path is resolved and matched by its final name,
|
||||
as before.
|
||||
"""
|
||||
plugin_dir_real = os.path.realpath(str(plugin_dir))
|
||||
plugins_dir_real = os.path.realpath(str(plugins_dir))
|
||||
plugin_dir_abs = os.path.abspath(str(plugin_dir))
|
||||
if os.path.realpath(os.path.dirname(plugin_dir_abs)) == plugins_dir_real:
|
||||
matched_name = find_trusted_subdir(plugins_dir_real, os.path.basename(plugin_dir_abs))
|
||||
if matched_name is not None:
|
||||
return os.path.join(plugins_dir_real, matched_name)
|
||||
plugin_dir_real = os.path.realpath(str(plugin_dir))
|
||||
matched_name = find_trusted_subdir(plugins_dir_real, os.path.basename(plugin_dir_real))
|
||||
if matched_name is None:
|
||||
return None
|
||||
@@ -243,6 +256,10 @@ class PluginLoader:
|
||||
self.logger = logger or get_logger(__name__)
|
||||
self._loaded_modules: Dict[str, Any] = {}
|
||||
self._plugin_module_registry: Dict[str, set] = {} # Maps plugin_id to set of module names
|
||||
# plugin_id -> {dotted name: module} for the modules of the plugin's
|
||||
# own packages (``providers.feed``). They keep their names while the
|
||||
# plugin runs and are dropped with it; see _iter_plugin_submodules.
|
||||
self._plugin_submodules: Dict[str, Dict[str, Any]] = {}
|
||||
# Lock to serialize module loading when plugins share module names
|
||||
# (e.g., scroll_display.py, game_renderer.py across sport plugins).
|
||||
# During exec_module, bare-name sub-modules temporarily appear in
|
||||
@@ -449,6 +466,45 @@ class PluginLoader:
|
||||
continue
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _iter_plugin_submodules(
|
||||
plugin_dir: Path, before_keys: set
|
||||
) -> list:
|
||||
"""Return dotted-name modules from plugin_dir added after before_keys.
|
||||
|
||||
The modules of a package the plugin ships (``providers.feed`` from
|
||||
``providers/feed.py``). _iter_plugin_bare_modules skips them, so the
|
||||
bare ``providers`` was namespaced and dropped on unload while
|
||||
``providers.feed`` stayed in sys.modules: a reload after a store update
|
||||
imported a fresh ``providers`` and then got the old ``feed`` back from
|
||||
the cache, running the new manager.py against the old helpers until the
|
||||
display restarted.
|
||||
|
||||
A module counts when its ``__file__`` -- or, for a namespace package,
|
||||
which has none, every ``__path__`` entry -- is inside plugin_dir, so a
|
||||
library the plugin imports (``requests.adapters``) never does.
|
||||
|
||||
Returns a list of (mod_name, module) tuples.
|
||||
"""
|
||||
resolved_dir = plugin_dir.resolve()
|
||||
result = []
|
||||
for key in set(sys.modules.keys()) - before_keys:
|
||||
if "." not in key:
|
||||
continue
|
||||
mod = sys.modules.get(key)
|
||||
if mod is None:
|
||||
continue
|
||||
mod_file = getattr(mod, "__file__", None)
|
||||
locations = [mod_file] if mod_file else list(getattr(mod, "__path__", None) or [])
|
||||
if not locations:
|
||||
continue
|
||||
try:
|
||||
if all(Path(loc).resolve().is_relative_to(resolved_dir) for loc in locations):
|
||||
result.append((key, mod))
|
||||
except (ValueError, TypeError, OSError):
|
||||
continue
|
||||
return result
|
||||
|
||||
def _evict_stale_bare_modules(self, plugin_dir: Path) -> dict:
|
||||
"""Temporarily remove bare-name sys.modules entries from other plugins.
|
||||
|
||||
@@ -527,6 +583,13 @@ class PluginLoader:
|
||||
# Track for cleanup during unload
|
||||
self._plugin_module_registry[plugin_id] = namespaced_names
|
||||
|
||||
# The modules of the plugin's own packages keep their dotted names
|
||||
# while it runs -- as they always have, so the package and its
|
||||
# children stay a matching set in sys.modules -- and are dropped
|
||||
# with the plugin by unregister_plugin_modules().
|
||||
self._plugin_submodules[plugin_id] = dict(
|
||||
self._iter_plugin_submodules(plugin_dir, before_keys))
|
||||
|
||||
if namespaced_names:
|
||||
self.logger.info(
|
||||
"Namespace-isolated %d module(s) for plugin %s",
|
||||
@@ -537,10 +600,16 @@ class PluginLoader:
|
||||
"""Remove namespaced sub-modules and cached module for a plugin from sys.modules.
|
||||
|
||||
Called by PluginManager during unload to clean up all module entries
|
||||
that were created when the plugin was loaded.
|
||||
that were created when the plugin was loaded, including the dotted
|
||||
modules of its packages. A dotted name is dropped only while it still
|
||||
holds this plugin's module: the name is not namespaced, so another
|
||||
plugin may have put its own there since.
|
||||
"""
|
||||
for ns_name in self._plugin_module_registry.pop(plugin_id, set()):
|
||||
sys.modules.pop(ns_name, None)
|
||||
for name, mod in self._plugin_submodules.pop(plugin_id, {}).items():
|
||||
if sys.modules.get(name) is mod:
|
||||
sys.modules.pop(name, None)
|
||||
self._loaded_modules.pop(plugin_id, None)
|
||||
|
||||
def load_module(
|
||||
@@ -646,11 +715,13 @@ class PluginLoader:
|
||||
if evicted_name not in sys.modules:
|
||||
sys.modules[evicted_name] = evicted_mod
|
||||
# Clean up the partially-initialized main module and any
|
||||
# bare-name sub-modules that were added during exec_module
|
||||
# so they don't leak into subsequent plugin loads.
|
||||
# bare-name or package sub-modules that were added during
|
||||
# exec_module so they don't leak into subsequent plugin loads.
|
||||
sys.modules.pop(module_name, None)
|
||||
for key, _ in self._iter_plugin_bare_modules(plugin_dir, before_keys):
|
||||
sys.modules.pop(key, None)
|
||||
for key, _ in self._iter_plugin_submodules(plugin_dir, before_keys):
|
||||
sys.modules.pop(key, None)
|
||||
raise
|
||||
|
||||
self._loaded_modules[plugin_id] = module
|
||||
|
||||
@@ -1163,7 +1163,7 @@ class PluginManager:
|
||||
def _record_update_failure(
|
||||
self,
|
||||
plugin_id: str,
|
||||
exc: Optional[Exception] = None,
|
||||
exc: Optional[BaseException] = None,
|
||||
log: bool = True,
|
||||
count_failure: bool = True,
|
||||
) -> None:
|
||||
@@ -1187,7 +1187,7 @@ class PluginManager:
|
||||
"""
|
||||
failure_time = time.time()
|
||||
if exc is not None:
|
||||
err: Exception = exc
|
||||
err: BaseException = exc
|
||||
error_type = type(exc).__name__
|
||||
else:
|
||||
err = Exception(f"Plugin {plugin_id} execution failed (timeout or executor error)")
|
||||
@@ -1653,7 +1653,7 @@ class PluginManager:
|
||||
finish_guard = threading.Lock()
|
||||
finished = {'done': False}
|
||||
|
||||
def _finish(success: bool, exc: Optional[Exception] = None) -> None:
|
||||
def _finish(success: bool, exc: Optional[BaseException] = None) -> None:
|
||||
with finish_guard:
|
||||
if finished['done']:
|
||||
return
|
||||
@@ -1727,7 +1727,13 @@ class PluginManager:
|
||||
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
|
||||
else:
|
||||
plugin_instance.update()
|
||||
except Exception as exc:
|
||||
except BaseException as exc: # pylint: disable=broad-except
|
||||
# BaseException, not just Exception: asyncio.CancelledError
|
||||
# and SystemExit derive from it. Either one skipped _finish,
|
||||
# so the plugin kept its lock and stayed RUNNING for good --
|
||||
# never rescheduled, and every display() skipped as busy.
|
||||
# Re-raised for the executor, which reports it as this
|
||||
# update's failure.
|
||||
_finish(False, exc=exc)
|
||||
raise
|
||||
else:
|
||||
|
||||
@@ -344,12 +344,26 @@ class PluginStoreManager(_RegistryMixin, _InstallMixin, _UpdateMixin):
|
||||
2. Fix permissions via os.chmod() then retry (works for same-owner files)
|
||||
3. Use sudo rm -rf as last resort (works for root-owned __pycache__, etc.)
|
||||
|
||||
A symlink -- a dev plugin linked in by scripts/dev/dev_plugin_setup.sh
|
||||
-- is removed as a link, before any of that: rmtree refuses one, and
|
||||
stage 2 would walk through it and chmod the developer's checkout.
|
||||
|
||||
Args:
|
||||
path: Path to directory to remove
|
||||
|
||||
Returns:
|
||||
True if directory was removed successfully, False otherwise
|
||||
"""
|
||||
if path.is_symlink():
|
||||
# Checked before exists(), which follows the link: a dangling one
|
||||
# would read as already removed and be left behind.
|
||||
try:
|
||||
path.unlink()
|
||||
return True
|
||||
except OSError as e:
|
||||
self.logger.error(f"Could not remove the symlink {path}: {e}")
|
||||
return False
|
||||
|
||||
if not path.exists():
|
||||
return True # Already removed
|
||||
|
||||
|
||||
@@ -45,7 +45,12 @@ server has none.
|
||||
|---|---|---|
|
||||
| `unit/test_list_filter.js` | no | `ListFilter` search/filter/sort/count/sticky, and the installed-plugins config **extracted verbatim** from `plugins_manager.js` so the test can't drift from it |
|
||||
| `unit/test_update_all.js` | no | `PluginInstallManager.updateAll` from `plugins/install_manager.js`: Check & Update All sends only plugin ids (never `starlark:` app entries), re-sends a request that got no HTTP answer (web service restarting) instead of skipping that plugin, never re-sends one that got any HTTP answer (the real `api_client.js` classifies a proxy 502 or a JSON error without `error_code` as `API_ERROR`), and counts a no-op update as already up to date in the summary. Also run by `test/web_interface/test_update_all_plugins.py` so CI covers it |
|
||||
| `unit/test_store_install.js` | no | The store's Install button, with the whole of `plugins_manager.js` run by `plugins_manager_sandbox.js` (a vm context, fake DOM and API): a fresh install reloads the list, then enables the id the plugin was installed as -- the answer's `plugin_id`, else the installed entry the store entry matches (Weather installs as `ledmatrix-weather`); a Reinstall leaves the enabled state alone |
|
||||
| `unit/test_install_polling.js` | no | How long Install waits for a queued install (sandbox): at least the server's 300 s dependency-install timeout; when it stops waiting it reloads the installed list and warns, rather than reporting a failure or enabling anything |
|
||||
| `unit/test_store_categories.js` | no | The store's category filter (sandbox): the template ships only All Categories, the rest come from the store's plugins (one per category whatever its case), choosing one filters to it, and a swapped-in select is refilled from the cache keeping the choice |
|
||||
| `unit/test_github_url_install.js` | no | Install Single Plugin (sandbox, the button as `plugins.html` ships it): no inline `onclick`, so a click or Enter sends exactly one `install-from-url` request and raises no error |
|
||||
| `unit/test_render_cards.js` | no | `renderInstalledCards` markup, both empty states, and HTML-escaping of hostile plugin metadata |
|
||||
| `unit/test_plugin_order_list.js` | no | `widgets/plugin-order-list.js` (the Vegas and rotation order lists): a disabled plugin, which gets no row, keeps its slot in the saved order and its Vegas exclusion when the list rewrites its hidden inputs, around reordering and include/exclude; an uninstalled plugin's id is dropped, a failed plugin list leaves the inputs as saved, and only string ids are carried over, once each |
|
||||
| `unit/test_style_editor_element_keys.js` | no | `elementKeys()`/`styleRows()`/`positionRows()` from `widgets/style-editor.js`: every `customization.layout` entry gets exactly one row -- paired with its style element through core's `x-layout-key` (so `score` belongs to `score_text`, not a second row), or a position row of its own, leaves included -- since the widget claims the whole `layout` block from the generic fallback renderer |
|
||||
| `unit/test_style_editor_layout_leaf_columns.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only key whose own value is a leaf (no x/y sub-object, e.g. a `show_logo` toggle) gets a self-keyed column instead of a blank, uneditable row |
|
||||
| `unit/test_style_editor_layout_leaf_collision.js` | no | `columnsFor()` from `widgets/style-editor.js`: a layout-only leaf key still gets its own column even when its name collides with an unrelated element's style sub-field or another layout axis's sub-field |
|
||||
@@ -53,6 +58,9 @@ server has none.
|
||||
| `unit/test_store_registry_fields.js` | no | The store card's registry fields from `plugins_manager.js`: the commit that introduced the listed version (a hex SHA only, linked to that tree), the "Needs LEDMatrix X+" warning, a card from an older registry without either, and `isStorePluginInstalled` answering to `aliases` |
|
||||
| `unit/test_page_registry.js` | no | The page lifecycle in `js/core/registry.js` (a minimal DOM shim): one `init` per `data-page` root, `destroy` and an aborted `ctx.signal` when htmx swaps it away, a vetoed swap keeps it, lazy page modules, a root removed without htmx swept on the next swap |
|
||||
| `unit/test_core_modules.js` | no | `js/core/api.js` (JSON envelope, HTTP/`status: error`/network errors, abort passthrough, the #683 login redirect, same-server paths only) and `js/core/facade.js` (`window.LEDMatrix`, deprecated aliases) |
|
||||
| `unit/test_overview_reconciliation_poll.js` | no | The Overview's reconciliation-banner poll from `partials/overview.html`, run in a vm: it gives up after a bounded number of requests when the status never says done, runs only while the Overview is on screen (`LEDVisibility`, its own key), and stops once the banner is shown |
|
||||
| `unit/test_display_partial_ids.js` | no | `partials/display.html`: every literal `getElementById()` in its inline scripts names an id the partial renders, and moving the brightness slider (the shipped script, in a vm with a fake DOM) updates its label without throwing |
|
||||
| `unit/test_general_web_login_token.js` | no | `window.webLogin.createToken` from `partials/general.html`, run in a vm: a created API token clears the form's `data-dirty` mark (so a reload does not ask "Leave site?"), a refused one keeps it |
|
||||
| `unit/test_plugin_action_delegation.js` | no | The document-level card-action delegation and `handlePluginAction` from `plugins_manager.js`, run with the handler inside an IIFE as in the real file: each action is handled once, a Starlark app uninstall goes to `DELETE /starlark/apps/<id>`, and an uninstall is confirmed once |
|
||||
| `dom/test_installed_dom.js` | yes | The toolbar in a real DOM: pill/search/sort interaction, the HTMX partial re-swap, and a `getComputedStyle` check that `.filter-pill[data-active]` really matches the emitted markup |
|
||||
| `dom/test_store_dom.js` | yes | Store pagination, per-page, category, tri-state Installed button, and persistence across a re-boot, against the live registry |
|
||||
|
||||
@@ -98,7 +98,11 @@ const ok = (l, c, x) => c ? (pass++, console.log(' ok ' + l))
|
||||
|
||||
const lists = () => requests.filter(r => r.url === '/api/v3/plugins/installed').length;
|
||||
const $ = id => doc.getElementById(id);
|
||||
const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]');
|
||||
// The rows' ids, in order. The input also keeps saved ids that have no row
|
||||
// (a disabled plugin's place, see test/js/unit/test_plugin_order_list.js),
|
||||
// and the saved order comes from whatever config the server has.
|
||||
const SHOWN = plugins.filter(p => p.enabled).map(p => p.id);
|
||||
const order = () => JSON.parse($('rotation_plugin_order_value').value || '[]').filter(id => SHOWN.includes(id));
|
||||
async function swap() {
|
||||
panel.dispatchEvent(new window.CustomEvent('htmx:beforeSwap', { bubbles: true, detail: { target: panel, shouldSwap: true } }));
|
||||
panel.innerHTML = partial;
|
||||
|
||||
@@ -98,8 +98,50 @@ const get = p => new Promise((res, rej) =>
|
||||
window.saveMqttBridge();
|
||||
await tick(150);
|
||||
ok('save includes password once typed', sent && sent.mqtt_password === 'typed-secret');
|
||||
|
||||
// A password with TLS off is refused unless allow_insecure_mqtt is set
|
||||
// (CWE-319, api_v3/misc.py). The form has to be able to send it, or a
|
||||
// plain-LAN broker with a password can never be saved from here.
|
||||
const allowRow = () => $('mqtt-allow-insecure-row');
|
||||
const shown = el => !!el && !el.classList.contains('hidden');
|
||||
ok('allow-without-TLS control rendered', !!$('mqtt-allow-insecure'));
|
||||
ok('allow-without-TLS starts as saved',
|
||||
!!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === !!bridge.data.config.allow_insecure_mqtt);
|
||||
ok('allow-without-TLS shown only while TLS is off',
|
||||
shown(allowRow()) === !$('mqtt-tls').checked);
|
||||
$('mqtt-tls').checked = true;
|
||||
$('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
|
||||
ok('ticking TLS hides it', !shown(allowRow()));
|
||||
$('mqtt-tls').checked = false;
|
||||
$('mqtt-tls').dispatchEvent(new window.Event('change', { bubbles: true }));
|
||||
ok('unticking TLS shows it again', shown(allowRow()));
|
||||
|
||||
const setAllow = v => { if ($('mqtt-allow-insecure')) $('mqtt-allow-insecure').checked = v; };
|
||||
setAllow(false);
|
||||
window.saveMqttBridge();
|
||||
await tick(150);
|
||||
ok('save sends allow_insecure_mqtt false when unticked', !!sent && sent.allow_insecure_mqtt === false, sent);
|
||||
setAllow(true);
|
||||
window.saveMqttBridge();
|
||||
await tick(150);
|
||||
ok('save sends allow_insecure_mqtt true when ticked', !!sent && sent.allow_insecure_mqtt === true, sent);
|
||||
onPut = null;
|
||||
|
||||
// Prefilled from the saved settings, and hidden while TLS is saved on.
|
||||
bridgePayload = JSON.parse(JSON.stringify(bridge));
|
||||
bridgePayload.data.config.allow_insecure_mqtt = true;
|
||||
bridgePayload.data.config.mqtt_tls = false;
|
||||
window.loadMqttBridge();
|
||||
await tick(150);
|
||||
ok('a saved opt-in is prefilled', !!$('mqtt-allow-insecure') && $('mqtt-allow-insecure').checked === true);
|
||||
bridgePayload.data.config.mqtt_tls = true;
|
||||
window.loadMqttBridge();
|
||||
await tick(150);
|
||||
ok('hidden on load when TLS is saved on', !shown(allowRow()));
|
||||
bridgePayload = bridge;
|
||||
window.loadMqttBridge();
|
||||
await tick(150);
|
||||
|
||||
// ── Pixlet editor, idle ────────────────────────────────────────────────
|
||||
const appIds = (apps.data.apps || []).map(a => a.id);
|
||||
ok('editor lists the apps on disk',
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
// The whole of plugins_manager.js (and list_filter.js before it, as the page
|
||||
// loads them), evaluated in a node vm context against a small fake DOM.
|
||||
//
|
||||
// For suites that drive the plugin manager's real flows -- install, polling,
|
||||
// store filters, the GitHub-URL button -- rather than one function sliced
|
||||
// out of the file. Nothing is mocked inside the script: only what the page
|
||||
// gives it (document, fetch, timers, showNotification, LEDEscape).
|
||||
//
|
||||
// const sb = create({ route: (method, url, body) => ({ status, json }) });
|
||||
// sb.el('plugin-store-grid'); // make an element exist by id
|
||||
// sb.window.installPlugin('weather');
|
||||
// await sb.until(() => sb.requests.some(r => r.url.includes('/toggle')));
|
||||
//
|
||||
// Timers ignore their delays and run on the next turn, so a poll loop that
|
||||
// would take minutes in a browser finishes in milliseconds. The page is in
|
||||
// readyState "loading" with no #installed-plugins-grid, so the script's own
|
||||
// start-up does nothing until a suite asks for it (window.initPluginsPage()).
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
const ledEscape = require('./led_escape');
|
||||
|
||||
const V3 = path.resolve(__dirname, '../../web_interface/static/v3');
|
||||
const PLUGINS_HTML = path.resolve(__dirname, '../../web_interface/templates/v3/partials/plugins.html');
|
||||
|
||||
class FakeClassList {
|
||||
constructor() { this.set = new Set(); }
|
||||
add(...c) { c.forEach(x => this.set.add(x)); }
|
||||
remove(...c) { c.forEach(x => this.set.delete(x)); }
|
||||
contains(c) { return this.set.has(c); }
|
||||
toggle(c, force) {
|
||||
const on = force === undefined ? !this.set.has(c) : !!force;
|
||||
if (on) this.set.add(c); else this.set.delete(c);
|
||||
return on;
|
||||
}
|
||||
}
|
||||
|
||||
function create({ route } = {}) {
|
||||
const elements = new Map();
|
||||
const requests = [];
|
||||
const toasts = [];
|
||||
const errors = [];
|
||||
const restartNotes = [];
|
||||
|
||||
class FakeElement {
|
||||
constructor(id, tag = 'div', attributes = {}) {
|
||||
this.id = id;
|
||||
this.tagName = tag.toUpperCase();
|
||||
this.attributes = { ...attributes };
|
||||
this.listeners = {};
|
||||
this.children = [];
|
||||
this.classList = new FakeClassList();
|
||||
this.style = { removeProperty() {} };
|
||||
this.dataset = {};
|
||||
this.value = '';
|
||||
this.textContent = '';
|
||||
this.disabled = false;
|
||||
this.parentNode = null;
|
||||
this._html = '';
|
||||
}
|
||||
get innerHTML() { return this._html; }
|
||||
set innerHTML(v) { this._html = String(v); this.children = []; }
|
||||
getAttribute(n) { return n in this.attributes ? this.attributes[n] : null; }
|
||||
setAttribute(n, v) { this.attributes[n] = String(v); }
|
||||
hasAttribute(n) { return n in this.attributes; }
|
||||
removeAttribute(n) { delete this.attributes[n]; }
|
||||
addEventListener(type, fn) { (this.listeners[type] = this.listeners[type] || []).push(fn); }
|
||||
removeEventListener(type, fn) {
|
||||
this.listeners[type] = (this.listeners[type] || []).filter(f => f !== fn);
|
||||
}
|
||||
appendChild(child) { this.children.push(child); child.parentNode = this; return child; }
|
||||
querySelector() { return null; }
|
||||
querySelectorAll() { return []; }
|
||||
closest() { return null; }
|
||||
cloneNode() {
|
||||
const copy = new FakeElement(this.id, this.tagName, this.attributes);
|
||||
copy._html = this._html;
|
||||
copy.value = this.value;
|
||||
return copy;
|
||||
}
|
||||
replaceChild(next, prev) {
|
||||
next.parentNode = this;
|
||||
prev.parentNode = null;
|
||||
if (next.id) elements.set(next.id, next);
|
||||
return prev;
|
||||
}
|
||||
replaceWith(next) { if (this.parentNode) this.parentNode.replaceChild(next, this); }
|
||||
// A browser runs an inline on<type> attribute first (it was set before
|
||||
// any listener was added), then the listeners, and an exception in one
|
||||
// does not stop the next: it is reported, which is what `errors` holds.
|
||||
dispatch(type, init = {}) {
|
||||
const event = {
|
||||
type, target: this, currentTarget: this, key: init.key,
|
||||
defaultPrevented: false,
|
||||
preventDefault() { this.defaultPrevented = true; },
|
||||
stopPropagation() {}, stopImmediatePropagation() {},
|
||||
};
|
||||
const inline = this.getAttribute('on' + type);
|
||||
const handlers = [];
|
||||
if (inline !== null) {
|
||||
handlers.push(vm.runInContext(`(function(event) {\n${inline}\n})`, ctx));
|
||||
}
|
||||
handlers.push(...(this.listeners[type] || []));
|
||||
for (const h of handlers) {
|
||||
try { h.call(this, event); } catch (e) { errors.push(e); }
|
||||
}
|
||||
return event;
|
||||
}
|
||||
click() { return this.dispatch('click'); }
|
||||
}
|
||||
|
||||
function el(id, tag, attributes) {
|
||||
if (!elements.has(id)) {
|
||||
const parent = new FakeElement(null);
|
||||
parent.appendChild(new FakeElement(id, tag, attributes));
|
||||
elements.set(id, parent.children[0]);
|
||||
}
|
||||
return elements.get(id);
|
||||
}
|
||||
|
||||
const timers = [];
|
||||
const ctx = {
|
||||
// Warnings are the script noting elements this fake page doesn't have.
|
||||
console: { log: console.log.bind(console), error: console.error.bind(console),
|
||||
warn: () => {}, info: () => {}, debug: () => {} },
|
||||
debugLog: () => {},
|
||||
addEventListener() {},
|
||||
URL,
|
||||
document: {
|
||||
readyState: 'loading',
|
||||
body: { addEventListener() {} },
|
||||
getElementById: id => elements.get(id) || null,
|
||||
querySelector: () => null,
|
||||
querySelectorAll: () => [],
|
||||
addEventListener() {},
|
||||
dispatchEvent() { return true; },
|
||||
createElement: tag => new FakeElement(null, tag),
|
||||
},
|
||||
CustomEvent: class { constructor(type, init) { this.type = type; this.detail = init && init.detail; } },
|
||||
setTimeout: (fn, _ms, ...args) => { timers.push(setImmediate(() => fn(...args))); return timers.length; },
|
||||
clearTimeout: () => {},
|
||||
setInterval: () => 0,
|
||||
clearInterval: () => {},
|
||||
requestAnimationFrame: fn => setImmediate(fn),
|
||||
getComputedStyle: () => ({ display: 'block' }),
|
||||
scrollTo() {},
|
||||
sessionStorage: { getItem: () => null, setItem() {}, removeItem() {} },
|
||||
localStorage: { getItem: () => null, setItem() {}, removeItem() {} },
|
||||
confirm: () => true,
|
||||
alert: () => {},
|
||||
showNotification: (message, type) => {
|
||||
toasts.push({ message: String(message),
|
||||
type: type && typeof type === 'object' ? type.type : type });
|
||||
},
|
||||
noteRestartRequired: (body) => { restartNotes.push(body); },
|
||||
fetch: async (url, opts = {}) => {
|
||||
const method = (opts.method || 'GET').toUpperCase();
|
||||
let body = null;
|
||||
try { body = opts.body ? JSON.parse(opts.body) : null; } catch (e) { body = opts.body; }
|
||||
requests.push({ method, url: String(url), body });
|
||||
const answer = (route && route(method, String(url), body)) || { status: 200, json: { status: 'success' } };
|
||||
const status = answer.status || 200;
|
||||
return { ok: status < 400, status, json: async () => answer.json };
|
||||
},
|
||||
};
|
||||
ctx.window = ctx;
|
||||
vm.createContext(ctx);
|
||||
ledEscape.install(ctx);
|
||||
for (const file of ['js/plugins/list_filter.js', 'plugins_manager.js']) {
|
||||
vm.runInContext(fs.readFileSync(path.join(V3, file), 'utf8'), ctx, { filename: file });
|
||||
}
|
||||
|
||||
// Resolves once cond() is true, letting timers and promises run between
|
||||
// checks; rejects if it never is.
|
||||
async function until(cond, label = 'condition', turns = 20000) {
|
||||
for (let i = 0; i < turns; i++) {
|
||||
if (cond()) return;
|
||||
await new Promise(r => setImmediate(r));
|
||||
}
|
||||
throw new Error('timed out waiting for ' + label);
|
||||
}
|
||||
|
||||
// Lets every pending timer and promise run.
|
||||
async function settle(turns = 50) {
|
||||
for (let i = 0; i < turns; i++) await new Promise(r => setImmediate(r));
|
||||
}
|
||||
|
||||
return { window: ctx, el, FakeElement, requests, toasts, errors, restartNotes, until, settle };
|
||||
}
|
||||
|
||||
// The attributes of the element with this id in partials/plugins.html, as
|
||||
// the template ships them (no Jinja on the tags these suites read).
|
||||
function templateAttributes(id) {
|
||||
const html = fs.readFileSync(PLUGINS_HTML, 'utf8');
|
||||
const at = html.indexOf(`id="${id}"`);
|
||||
if (at < 0) throw new Error(`no element with id ${id} in plugins.html`);
|
||||
const start = html.lastIndexOf('<', at);
|
||||
let end = start, quote = null;
|
||||
for (; end < html.length; end++) {
|
||||
const ch = html[end];
|
||||
if (quote) { if (ch === quote) quote = null; } else if (ch === '"' || ch === "'") quote = ch;
|
||||
else if (ch === '>') break;
|
||||
}
|
||||
const tag = html.slice(start, end + 1);
|
||||
const attrs = {};
|
||||
const re = /([\w:-]+)\s*=\s*("([^"]*)"|'([^']*)')/g;
|
||||
let m;
|
||||
while ((m = re.exec(tag))) attrs[m[1]] = m[3] !== undefined ? m[3] : m[4];
|
||||
return { tag: tag.match(/^<(\w+)/)[1], attrs, source: tag };
|
||||
}
|
||||
|
||||
module.exports = { create, templateAttributes };
|
||||
+11
-2
@@ -17,13 +17,22 @@ const fs = require('fs');
|
||||
|
||||
const BASE = process.env.BASE || 'http://localhost:5000';
|
||||
const UNIT = ['unit/test_list_filter.js', 'unit/test_render_cards.js',
|
||||
'unit/test_plugin_order_list.js',
|
||||
'unit/test_html_escaping.js', 'unit/test_style_editor_element_keys.js',
|
||||
'unit/test_style_editor_layout_leaf_columns.js',
|
||||
'unit/test_style_editor_layout_leaf_collision.js',
|
||||
'unit/test_update_all.js', 'unit/test_inline_handler_escaping.js',
|
||||
'unit/test_update_all.js',
|
||||
'unit/test_store_install.js',
|
||||
'unit/test_install_polling.js',
|
||||
'unit/test_store_categories.js',
|
||||
'unit/test_github_url_install.js',
|
||||
'unit/test_inline_handler_escaping.js',
|
||||
'unit/test_plugin_action_delegation.js', 'unit/test_file_upload_widget.js',
|
||||
'unit/test_store_registry_fields.js', 'unit/test_restart_banner.js',
|
||||
'unit/test_page_registry.js', 'unit/test_core_modules.js'];
|
||||
'unit/test_page_registry.js', 'unit/test_core_modules.js',
|
||||
'unit/test_overview_reconciliation_poll.js',
|
||||
'unit/test_display_partial_ids.js',
|
||||
'unit/test_general_web_login_token.js'];
|
||||
const DOM = ['dom/test_installed_dom.js', 'dom/test_store_dom.js', 'dom/test_no_double_fetch.js',
|
||||
'dom/test_tools_sections.js', 'dom/test_cache_page.js',
|
||||
'dom/test_durations_page.js', 'dom/test_operation_history_page.js',
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
// The Display tab's inline script must only look up elements the partial
|
||||
// renders.
|
||||
//
|
||||
// Its brightness slider handler also wrote to #brightness-display, a "LED
|
||||
// brightness: N%" line that #387 removed from partials/display.html. The
|
||||
// lookup returned null, so every movement of the slider threw a TypeError.
|
||||
// This checks every literal getElementById() in the partial's inline scripts
|
||||
// against the ids its markup renders, and runs the shipped script in a vm
|
||||
// with a fake DOM (null for an id the markup lacks, as in a browser) to move
|
||||
// the slider.
|
||||
//
|
||||
// No jsdom and no server needed.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
|
||||
const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/display.html');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
|
||||
|
||||
const html = fs.readFileSync(PARTIAL, 'utf8');
|
||||
const blocks = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)];
|
||||
const scripts = blocks.map(m => m[1]);
|
||||
// The markup is what lies between the script blocks (sliced around them, not
|
||||
// a replace(), which CodeQL reads as an incomplete HTML sanitizer).
|
||||
let markup = '';
|
||||
let from = 0;
|
||||
for (const m of blocks) {
|
||||
markup += html.slice(from, m.index);
|
||||
from = m.index + m[0].length;
|
||||
}
|
||||
markup += html.slice(from);
|
||||
const rendered = new Set([...markup.matchAll(/\bid="([^"{}]+)"/g)].map(m => m[1]));
|
||||
|
||||
console.log('\n── Display partial: element lookups ──');
|
||||
|
||||
// 1. Static: every literal lookup names an id the partial renders.
|
||||
const lookups = scripts.flatMap(s => [...s.matchAll(/getElementById\('([^']+)'\)/g)].map(m => m[1]));
|
||||
const missing = [...new Set(lookups.filter(id => !rendered.has(id)))];
|
||||
ok('the inline scripts look elements up', lookups.length > 0, lookups.length);
|
||||
ok('every looked-up id is rendered by the partial', missing.length === 0, missing);
|
||||
|
||||
// 2. Behaviour: moving the brightness slider updates its label and throws nothing.
|
||||
function fakeElement(id) {
|
||||
const listeners = {};
|
||||
const classes = new Set();
|
||||
return {
|
||||
id, value: '', textContent: '', min: '', max: '', checked: false,
|
||||
style: {}, dataset: {}, className: '',
|
||||
classList: {
|
||||
add: c => classes.add(c), remove: c => classes.delete(c),
|
||||
toggle: (c, on) => (on === undefined ? (classes.has(c) ? classes.delete(c) : classes.add(c)) : (on ? classes.add(c) : classes.delete(c))),
|
||||
contains: c => classes.has(c),
|
||||
},
|
||||
addEventListener: (type, fn) => { (listeners[type] ||= []).push(fn); },
|
||||
dispatchEvent() { return true; },
|
||||
appendChild() {},
|
||||
listeners,
|
||||
};
|
||||
}
|
||||
|
||||
const main = scripts.find(s => s.includes("getElementById('brightness')"));
|
||||
ok('found the script that wires the brightness slider', !!main);
|
||||
if (main) {
|
||||
const elements = new Map();
|
||||
const document = {
|
||||
readyState: 'complete',
|
||||
hidden: false,
|
||||
getElementById: id => {
|
||||
if (!rendered.has(id)) return null;
|
||||
if (!elements.has(id)) elements.set(id, fakeElement(id));
|
||||
return elements.get(id);
|
||||
},
|
||||
createElement: () => fakeElement(''),
|
||||
createTextNode: () => ({}),
|
||||
addEventListener() {},
|
||||
};
|
||||
const window = {
|
||||
LEDEscape: { html: v => String(v), attr: v => String(v) },
|
||||
LEDVisibility: { onActive() {} },
|
||||
};
|
||||
const context = {
|
||||
window, document, console, URLSearchParams,
|
||||
fetch: () => new Promise(() => {}),
|
||||
setTimeout: () => 0, clearTimeout() {}, setInterval: () => 0, clearInterval() {},
|
||||
};
|
||||
vm.createContext(context);
|
||||
let loadError = null;
|
||||
try { vm.runInContext(main, context); } catch (e) { loadError = e; }
|
||||
ok('the script loads', !loadError, loadError && String(loadError));
|
||||
|
||||
const slider = elements.get('brightness');
|
||||
const handlers = (slider && slider.listeners.input) || [];
|
||||
ok('the slider has an input handler', handlers.length > 0);
|
||||
let thrown = null;
|
||||
slider.value = '42';
|
||||
try { handlers.forEach(fn => fn.call(slider, { target: slider })); } catch (e) { thrown = e; }
|
||||
ok('moving the slider throws nothing', !thrown, thrown && String(thrown));
|
||||
ok('...and shows the new value', elements.get('brightness-value').textContent === '42',
|
||||
elements.get('brightness-value').textContent);
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed\n`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
@@ -0,0 +1,107 @@
|
||||
// Creating an API token on the General tab must leave its form clean.
|
||||
//
|
||||
// app.js marks a form data-dirty on any input in it and clears the mark only
|
||||
// after a successful htmx request; its beforeunload handler then asks "Leave
|
||||
// site?" while any visible form is still dirty. The token form posts with
|
||||
// fetch (window.webLogin.createToken in partials/general.html), so after a
|
||||
// token was created the form stayed dirty and reloading the page while the
|
||||
// General tab was open prompted about changes that had been saved.
|
||||
//
|
||||
// Runs the shipped inline script in a vm with a fake fetch and DOM -- no jsdom
|
||||
// and no server needed.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
|
||||
const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/general.html');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
|
||||
|
||||
function webLoginScript() {
|
||||
const html = fs.readFileSync(PARTIAL, 'utf8');
|
||||
const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
|
||||
const found = scripts.find(s => s.includes('window.webLogin = {'));
|
||||
if (!found) throw new Error('webLogin script not found in general.html');
|
||||
return found;
|
||||
}
|
||||
|
||||
function el() {
|
||||
const classes = new Set(['hidden']);
|
||||
return {
|
||||
textContent: '', dataset: {}, style: {}, className: '',
|
||||
classList: { add: c => classes.add(c), remove: c => classes.delete(c), contains: c => classes.has(c) },
|
||||
appendChild() {}, addEventListener() {}, querySelector: () => null,
|
||||
};
|
||||
}
|
||||
|
||||
function load(answer) {
|
||||
const elements = {
|
||||
'web-login-tokens': el(),
|
||||
'web-login-new-token-value': el(),
|
||||
'web-login-new-token': el(),
|
||||
};
|
||||
const notes = [];
|
||||
const window = { showNotification: (m, t) => notes.push([m, t]), alert() {}, confirm: () => true };
|
||||
const context = {
|
||||
window, console,
|
||||
document: {
|
||||
getElementById: id => elements[id] || null,
|
||||
createElement: () => el(),
|
||||
querySelectorAll: () => [],
|
||||
},
|
||||
fetch: () => Promise.resolve({
|
||||
ok: answer.ok, status: answer.ok ? 200 : 400,
|
||||
json: () => Promise.resolve(answer.body),
|
||||
}),
|
||||
};
|
||||
vm.createContext(context);
|
||||
vm.runInContext(webLoginScript(), context);
|
||||
return { webLogin: context.window.webLogin, elements, notes };
|
||||
}
|
||||
|
||||
function dirtyForm() {
|
||||
const attrs = new Map([['data-dirty', '']]);
|
||||
return {
|
||||
querySelector: sel => (sel === '[name="name"]' ? { value: 'Home Assistant' } : null),
|
||||
reset() {},
|
||||
hasAttribute: name => attrs.has(name),
|
||||
setAttribute: (name, value) => attrs.set(name, String(value)),
|
||||
removeAttribute: name => attrs.delete(name),
|
||||
};
|
||||
}
|
||||
|
||||
const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
|
||||
|
||||
(async () => {
|
||||
console.log('\n── General tab: API token form ──');
|
||||
|
||||
{
|
||||
const t = load({ ok: true, body: {
|
||||
status: 'success', message: 'Token created',
|
||||
data: { token: 'lmx_secret', record: { id: 't1', name: 'Home Assistant', prefix: 'lmx_sec' } },
|
||||
} });
|
||||
const form = dirtyForm();
|
||||
t.webLogin.createToken(form);
|
||||
await flush();
|
||||
ok('the new token is shown', t.elements['web-login-new-token-value'].textContent === 'lmx_secret');
|
||||
ok('a created token leaves the form clean (no "Leave site?" on reload)',
|
||||
!form.hasAttribute('data-dirty'));
|
||||
}
|
||||
|
||||
{
|
||||
const t = load({ ok: false, body: { status: 'error', message: 'Name is required' } });
|
||||
const form = dirtyForm();
|
||||
t.webLogin.createToken(form);
|
||||
await flush();
|
||||
ok('a refused request reports the error', t.notes.some(([m, type]) => type === 'error' && /Name is required/.test(m)),
|
||||
t.notes);
|
||||
ok('...and keeps the form dirty: nothing was saved', form.hasAttribute('data-dirty'));
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed\n`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });
|
||||
@@ -0,0 +1,93 @@
|
||||
// Plugin Manager > Install from GitHub > Install Single Plugin: one click,
|
||||
// one request, no errors.
|
||||
//
|
||||
// The Install button carried an inline onclick calling
|
||||
// window.handleGitHubPluginInstall, and attachInstallButtonHandler also gave
|
||||
// it a click listener that installs. Both ran on every click. The inline one
|
||||
// threw a ReferenceError (it called isGithubUrl, which lives inside the
|
||||
// plugin-manager IIFE, from outside it), so only the listener's request went
|
||||
// out -- and fixing that scope alone would have sent every install twice.
|
||||
// The button now has the listener only.
|
||||
//
|
||||
// Runs the whole of plugins_manager.js in the sandbox, with the button as
|
||||
// partials/plugins.html ships it.
|
||||
|
||||
const { create, templateAttributes } = require('../plugins_manager_sandbox');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra).slice(0, 400) : '')));
|
||||
|
||||
const URL = 'https://github.com/someone/ledmatrix-demo';
|
||||
|
||||
function route(method, url) {
|
||||
if (method === 'POST' && url === '/api/v3/plugins/install-from-url') {
|
||||
return { json: { status: 'success', message: 'Plugin demo installed successfully', plugin_id: 'demo' } };
|
||||
}
|
||||
if (url.startsWith('/api/v3/plugins/installed')) return { json: { status: 'success', data: { plugins: [] } } };
|
||||
return { json: { status: 'success' } };
|
||||
}
|
||||
|
||||
function page() {
|
||||
const sb = create({ route });
|
||||
const button = templateAttributes('install-plugin-from-url');
|
||||
sb.el('install-plugin-from-url', button.tag, button.attrs);
|
||||
sb.el('github-plugin-url', 'input');
|
||||
sb.el('github-plugin-status');
|
||||
sb.el('plugin-branch-input', 'input');
|
||||
return sb;
|
||||
}
|
||||
|
||||
const installs = sb => sb.requests.filter(r => r.url === '/api/v3/plugins/install-from-url');
|
||||
|
||||
(async () => {
|
||||
console.log('\nthe template');
|
||||
{
|
||||
const { attrs } = templateAttributes('install-plugin-from-url');
|
||||
ok('the Install button has no inline onclick', !('onclick' in attrs), attrs.onclick);
|
||||
}
|
||||
|
||||
console.log('\na click');
|
||||
{
|
||||
const sb = page();
|
||||
sb.window.attachInstallButtonHandler();
|
||||
// htmx:afterSettle runs it again on every swap; that must not add a handler.
|
||||
sb.window.attachInstallButtonHandler();
|
||||
sb.el('github-plugin-url').value = URL;
|
||||
sb.window.document.getElementById('install-plugin-from-url').click();
|
||||
await sb.settle();
|
||||
ok('raises no error', sb.errors.length === 0, sb.errors.map(String));
|
||||
ok('sends exactly one install request', installs(sb).length === 1, installs(sb));
|
||||
ok('for the URL typed', installs(sb)[0] && installs(sb)[0].body.repo_url === URL, installs(sb));
|
||||
ok('and reports the result', /Successfully installed: demo/.test(sb.el('github-plugin-status').innerHTML),
|
||||
sb.el('github-plugin-status').innerHTML);
|
||||
}
|
||||
|
||||
console.log('\nEnter in the URL field');
|
||||
{
|
||||
const sb = page();
|
||||
sb.window.attachInstallButtonHandler();
|
||||
const input = sb.el('github-plugin-url');
|
||||
input.value = URL;
|
||||
input.dispatch('keypress', { key: 'Enter' });
|
||||
await sb.settle();
|
||||
ok('raises no error', sb.errors.length === 0, sb.errors.map(String));
|
||||
ok('sends exactly one install request', installs(sb).length === 1, installs(sb));
|
||||
}
|
||||
|
||||
console.log('\na URL that is not GitHub');
|
||||
{
|
||||
const sb = page();
|
||||
sb.window.attachInstallButtonHandler();
|
||||
sb.el('github-plugin-url').value = 'https://example.com/x';
|
||||
sb.window.document.getElementById('install-plugin-from-url').click();
|
||||
await sb.settle();
|
||||
ok('is refused without a request or an error',
|
||||
installs(sb).length === 0 && sb.errors.length === 0 && /valid GitHub URL/.test(sb.el('github-plugin-status').innerHTML),
|
||||
{ errors: sb.errors.map(String), status: sb.el('github-plugin-status').innerHTML });
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,89 @@
|
||||
// How long the store's Install waits for a queued install, and what it says
|
||||
// when it stops waiting.
|
||||
//
|
||||
// It polled the operation 60 times, a second apart, then reported "Install
|
||||
// operation timed out" as an error and did nothing else. The server is
|
||||
// allowed far longer: the plugin's dependency install alone may take 300 s
|
||||
// (install_requirements_file in src/plugin_system/store_install.py), after
|
||||
// a download that fetches the plugin one file at a time. So an install that
|
||||
// went on to succeed was reported as failed, never enabled, and missing from
|
||||
// the installed list until the page was reloaded.
|
||||
//
|
||||
// Runs the whole of plugins_manager.js in the sandbox; its timers ignore
|
||||
// their delays, so each poll here stands for one second on a real page.
|
||||
|
||||
const { create } = require('../plugins_manager_sandbox');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra).slice(0, 400) : '')));
|
||||
|
||||
// The server's dependency-install timeout, in polls (one a second).
|
||||
const DEPENDENCY_INSTALL_TIMEOUT_POLLS = 300;
|
||||
|
||||
function server(completesAfterPolls) {
|
||||
const state = { polls: 0, installed: [] };
|
||||
state.route = (method, url, body) => {
|
||||
if (url.startsWith('/api/v3/plugins/installed')) {
|
||||
return { json: { status: 'success', data: { plugins: state.installed.map(p => ({ ...p })) } } };
|
||||
}
|
||||
if (method === 'POST' && url === '/api/v3/plugins/install') {
|
||||
return { json: { status: 'success', message: 'queued', data: { operation_id: 'op-1' } } };
|
||||
}
|
||||
if (url === '/api/v3/plugins/operation/op-1') {
|
||||
state.polls++;
|
||||
if (completesAfterPolls === null || state.polls < completesAfterPolls) {
|
||||
return { json: { status: 'success', data: { status: 'running' } } };
|
||||
}
|
||||
state.installed = [{ id: 'clock-simple', name: 'Clock', enabled: false }];
|
||||
return { json: { status: 'success', data: { status: 'completed',
|
||||
result: { success: true, message: 'installed', plugin_id: 'clock-simple' } } } };
|
||||
}
|
||||
if (method === 'POST' && url === '/api/v3/plugins/toggle') {
|
||||
return { json: { status: 'success', message: 'enabled' } };
|
||||
}
|
||||
return { json: { status: 'success' } };
|
||||
};
|
||||
return state;
|
||||
}
|
||||
|
||||
(async () => {
|
||||
console.log('\nan install that takes longer than a minute');
|
||||
{
|
||||
// 200 s: well inside what the server allows.
|
||||
const srv = server(200);
|
||||
const sb = create({ route: srv.route });
|
||||
sb.window.installPlugin('clock-simple');
|
||||
await sb.until(() => sb.toasts.some(t => /installed and enabled|enabling it failed|timed out|still/i.test(t.message)),
|
||||
'the install to finish');
|
||||
await sb.settle();
|
||||
ok('is waited for until it completes', srv.polls === 200, srv.polls);
|
||||
ok('is not reported as an error', !sb.toasts.some(t => t.type === 'error'), sb.toasts);
|
||||
ok('and is enabled', sb.requests.some(r => r.url === '/api/v3/plugins/toggle' && r.body.plugin_id === 'clock-simple'),
|
||||
sb.requests.filter(r => r.method === 'POST'));
|
||||
}
|
||||
|
||||
console.log('\nan install that never reports back');
|
||||
{
|
||||
const srv = server(null);
|
||||
const sb = create({ route: srv.route });
|
||||
sb.window.installPlugin('clock-simple');
|
||||
await sb.until(() => sb.toasts.length >= 3, 'the poller to give up');
|
||||
await sb.settle();
|
||||
ok(`is polled for at least the ${DEPENDENCY_INSTALL_TIMEOUT_POLLS} s dependency-install timeout`,
|
||||
srv.polls >= DEPENDENCY_INSTALL_TIMEOUT_POLLS, srv.polls);
|
||||
ok('...but not forever', srv.polls <= 1200, srv.polls);
|
||||
const lastPoll = sb.requests.map(r => r.url).lastIndexOf('/api/v3/plugins/operation/op-1');
|
||||
ok('then the installed list is reloaded, to show what actually happened',
|
||||
sb.requests.slice(lastPoll + 1).some(r => r.url === '/api/v3/plugins/installed'),
|
||||
sb.requests.slice(lastPoll + 1).map(r => r.url));
|
||||
const last = sb.toasts[sb.toasts.length - 1];
|
||||
ok('it says the install may still be running, as a warning, not a failure',
|
||||
last && last.type === 'warning' && !/fail|timed out/i.test(last.message), sb.toasts);
|
||||
ok('nothing is enabled on a guess', !sb.requests.some(r => r.url === '/api/v3/plugins/toggle'));
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,137 @@
|
||||
// The Overview's "Plugin Config Warning" poll must end.
|
||||
//
|
||||
// The banner script in partials/overview.html asks
|
||||
// /api/v3/plugins/reconciliation-status every 2 s until startup reconciliation
|
||||
// says it is done. The route answers done: false whenever its status file is
|
||||
// missing -- reconciliation raised before writing it, or /tmp was cleaned
|
||||
// under a long-running web service -- so the poll used to run every 2 s for
|
||||
// as long as the page stayed open, on every tab. It now gives up after a
|
||||
// bounded number of tries and runs only while the Overview is on screen
|
||||
// (LEDVisibility, like the other partials' pollers).
|
||||
//
|
||||
// Runs the shipped inline script in a vm with fake timers, fetch and DOM --
|
||||
// no jsdom and no server needed.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
|
||||
const PARTIAL = path.resolve(__dirname, '../../../web_interface/templates/v3/partials/overview.html');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
|
||||
|
||||
function bannerScript() {
|
||||
const html = fs.readFileSync(PARTIAL, 'utf8');
|
||||
const scripts = [...html.matchAll(/<script\b[^>]*>([\s\S]*?)<\/script[^>]*>/gi)].map(m => m[1]);
|
||||
const found = scripts.find(s => s.includes('ledmatrix-recon-dismissed'));
|
||||
if (!found) throw new Error('reconciliation banner script not found in overview.html');
|
||||
return found;
|
||||
}
|
||||
|
||||
const flush = async () => { for (let i = 0; i < 10; i++) await new Promise(r => setImmediate(r)); };
|
||||
|
||||
function load({ payload, visibility = true }) {
|
||||
const timers = new Map();
|
||||
let nextId = 1;
|
||||
const calls = [];
|
||||
const banner = { style: { setProperty() {} }, dataset: {} };
|
||||
const text = { textContent: '' };
|
||||
const registrations = [];
|
||||
const window = {};
|
||||
if (visibility) {
|
||||
window.LEDVisibility = {
|
||||
onActive(tab, start, stop, key) { registrations.push({ tab, start, stop, key }); start(); },
|
||||
};
|
||||
}
|
||||
const context = {
|
||||
window,
|
||||
document: {
|
||||
getElementById: id => ({ 'reconciliation-banner': banner, 'reconciliation-banner-text': text })[id] || null,
|
||||
},
|
||||
sessionStorage: { getItem: () => null, setItem() {} },
|
||||
fetch: (url) => {
|
||||
calls.push(url);
|
||||
return Promise.resolve({ json: () => Promise.resolve(payload()) });
|
||||
},
|
||||
setTimeout: (fn) => { const id = nextId++; timers.set(id, fn); return id; },
|
||||
clearTimeout: (id) => { timers.delete(id); },
|
||||
};
|
||||
vm.createContext(context);
|
||||
vm.runInContext(bannerScript(), context);
|
||||
const fireTimers = async () => {
|
||||
const due = [...timers.entries()];
|
||||
timers.clear();
|
||||
due.forEach(([, fn]) => fn());
|
||||
await flush();
|
||||
};
|
||||
return { calls, timers, registrations, banner, text, window, fireTimers };
|
||||
}
|
||||
|
||||
(async () => {
|
||||
console.log('\n── Overview reconciliation poll ──');
|
||||
|
||||
// 1. A status file that never says done: the poll stops on its own.
|
||||
{
|
||||
const t = load({ payload: () => ({ status: 'success', data: { done: false, unresolved: [] } }) });
|
||||
await flush();
|
||||
for (let i = 0; i < 200; i++) await t.fireTimers();
|
||||
ok('a status that never turns done stops being polled', t.timers.size === 0,
|
||||
{ pending: t.timers.size, requests: t.calls.length });
|
||||
ok('...after a bounded number of requests (at most 30, a minute at 2 s)',
|
||||
t.calls.length > 1 && t.calls.length <= 30, t.calls.length);
|
||||
}
|
||||
|
||||
// 2. Runs only while the Overview is on screen.
|
||||
{
|
||||
const t = load({ payload: () => ({ status: 'success', data: { done: false, unresolved: [] } }) });
|
||||
await flush();
|
||||
const reg = t.registrations[0];
|
||||
ok('registers with LEDVisibility for the overview tab', !!reg && reg.tab === 'overview', reg && reg.tab);
|
||||
ok('under its own key, so it does not replace another overview poller',
|
||||
!!reg && !!reg.key && reg.key !== 'overview', reg && reg.key);
|
||||
ok('first request goes out at once', t.calls.length === 1, t.calls.length);
|
||||
if (reg) {
|
||||
reg.stop();
|
||||
ok('leaving the tab cancels the pending retry', t.timers.size === 0, t.timers.size);
|
||||
for (let i = 0; i < 5; i++) await t.fireTimers();
|
||||
ok('no requests while another tab is active', t.calls.length === 1, t.calls.length);
|
||||
reg.start();
|
||||
await flush();
|
||||
ok('coming back asks again at once', t.calls.length === 2, t.calls.length);
|
||||
ok('...and keeps polling', t.timers.size === 1, t.timers.size);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. A finished reconciliation with findings shows the banner and stops.
|
||||
{
|
||||
let done = false;
|
||||
const t = load({ payload: () => (done
|
||||
? { status: 'success', data: { done: true, unresolved: [{ plugin_id: 'clock', type: 'plugin_missing_on_disk' }] } }
|
||||
: { status: 'success', data: { done: false, unresolved: [] } }) });
|
||||
await flush();
|
||||
await t.fireTimers();
|
||||
done = true;
|
||||
await t.fireTimers();
|
||||
ok('the banner names the finding once reconciliation is done',
|
||||
t.text.textContent.includes('clock'), t.text.textContent);
|
||||
const before = t.calls.length;
|
||||
for (let i = 0; i < 5; i++) await t.fireTimers();
|
||||
ok('no more requests once it is done', t.calls.length === before && t.timers.size === 0,
|
||||
{ before, after: t.calls.length, pending: t.timers.size });
|
||||
}
|
||||
|
||||
// 4. Without LEDVisibility (base.html always has it) it still runs, bounded.
|
||||
{
|
||||
const t = load({ visibility: false, payload: () => ({ status: 'success', data: { done: false } }) });
|
||||
await flush();
|
||||
ok('runs without LEDVisibility', t.calls.length === 1, t.calls.length);
|
||||
for (let i = 0; i < 200; i++) await t.fireTimers();
|
||||
ok('...and is still bounded', t.timers.size === 0 && t.calls.length <= 30, t.calls.length);
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed\n`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.log('HARNESS ERROR: ' + e.stack); process.exit(1); });
|
||||
@@ -0,0 +1,189 @@
|
||||
// The shared plugin order list (widgets/plugin-order-list.js) keeps what it
|
||||
// does not show.
|
||||
//
|
||||
// It lists enabled plugins only, and rewrites its hidden inputs from those
|
||||
// rows as soon as it has drawn them. A disabled plugin's place in the order
|
||||
// and its Vegas exclusion used to vanish from the inputs on that rewrite, so
|
||||
// any later save of the Display or Rotation & Durations tab stored them
|
||||
// without it: re-enabled, the plugin came back at the end of the rotation and
|
||||
// scrolling in Vegas again. An uninstalled plugin's id is still dropped, as
|
||||
// before, so the lists don't collect ids nothing can show. Runs the shipped
|
||||
// widget in a vm with a minimal fake DOM -- no jsdom and no server needed, so
|
||||
// it runs under test/test_js_unit_suites.py too.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
const WIDGET = path.resolve(__dirname, '../../../web_interface/static/v3/js/widgets/plugin-order-list.js');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' ' + JSON.stringify(extra) : '')));
|
||||
const same = (a, b) => JSON.stringify(a) === JSON.stringify(b);
|
||||
|
||||
class FakeElement {
|
||||
constructor(tag) {
|
||||
this.tagName = tag.toUpperCase();
|
||||
this.children = [];
|
||||
this.parent = null;
|
||||
this.dataset = {};
|
||||
this.style = {};
|
||||
this.className = '';
|
||||
this.value = '';
|
||||
this.checked = false;
|
||||
this.listeners = {};
|
||||
this._text = '';
|
||||
}
|
||||
appendChild(child) {
|
||||
if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
|
||||
child.parent = this;
|
||||
this.children.push(child);
|
||||
return child;
|
||||
}
|
||||
insertBefore(child, ref) {
|
||||
if (!ref) return this.appendChild(child);
|
||||
if (child.parent) child.parent.children = child.parent.children.filter(c => c !== child);
|
||||
child.parent = this;
|
||||
this.children.splice(this.children.indexOf(ref), 0, child);
|
||||
return child;
|
||||
}
|
||||
get previousElementSibling() {
|
||||
const siblings = this.parent ? this.parent.children : [];
|
||||
return siblings[siblings.indexOf(this) - 1] || null;
|
||||
}
|
||||
get nextElementSibling() {
|
||||
const siblings = this.parent ? this.parent.children : [];
|
||||
const i = siblings.indexOf(this);
|
||||
return i < 0 ? null : siblings[i + 1] || null;
|
||||
}
|
||||
set textContent(value) { this._text = value; this.children = []; }
|
||||
get textContent() { return this._text; }
|
||||
setAttribute() {}
|
||||
focus() {}
|
||||
addEventListener(type, fn) { (this.listeners[type] ||= []).push(fn); }
|
||||
fire(type, event) { (this.listeners[type] || []).forEach(fn => fn.call(this, event || {})); }
|
||||
descendants() { return this.children.flatMap(c => [c, ...c.descendants()]); }
|
||||
querySelectorAll(selector) {
|
||||
const cls = selector.replace(/^\./, '');
|
||||
return this.descendants().filter(e => e.className.split(/\s+/).includes(cls));
|
||||
}
|
||||
querySelector(selector) { return this.querySelectorAll(selector)[0] || null; }
|
||||
}
|
||||
|
||||
/** Run the widget over `plugins` with the given saved inputs; resolves once it has drawn. */
|
||||
async function mount({ plugins, order, excluded, fetchFails }) {
|
||||
const els = {
|
||||
list: new FakeElement('div'),
|
||||
order: Object.assign(new FakeElement('input'), { value: JSON.stringify(order) }),
|
||||
};
|
||||
if (excluded !== undefined) {
|
||||
els.excluded = Object.assign(new FakeElement('input'), { value: JSON.stringify(excluded) });
|
||||
}
|
||||
const context = {
|
||||
// The widget logs a failed list; expected there, so kept off the output.
|
||||
console: fetchFails ? Object.assign({}, console, { error: () => {} }) : console,
|
||||
window: {},
|
||||
document: {
|
||||
getElementById: (id) => els[id] || null,
|
||||
createElement: (tag) => new FakeElement(tag),
|
||||
createTextNode: (text) => new FakeElement('#text'),
|
||||
},
|
||||
fetch: () => (fetchFails ? Promise.reject(new Error('service restarting')) : Promise.resolve({
|
||||
json: () => Promise.resolve({ status: 'success', data: { plugins } }),
|
||||
})),
|
||||
};
|
||||
vm.createContext(context);
|
||||
vm.runInContext(fs.readFileSync(WIDGET, 'utf8'), context);
|
||||
context.window.PluginOrderList.init({
|
||||
containerId: 'list', orderInputId: 'order',
|
||||
excludedInputId: excluded !== undefined ? 'excluded' : undefined,
|
||||
});
|
||||
await new Promise(resolve => setTimeout(resolve, 0));
|
||||
const rows = () => els.list.querySelectorAll('.plugin-order-item');
|
||||
return {
|
||||
rows,
|
||||
rowIds: () => rows().map(r => r.dataset.pluginId),
|
||||
order: () => JSON.parse(els.order.value),
|
||||
excluded: () => JSON.parse(els.excluded.value),
|
||||
row: (id) => rows().find(r => r.dataset.pluginId === id),
|
||||
};
|
||||
}
|
||||
|
||||
const PLUGINS = [
|
||||
{ id: 'weather', name: 'Weather', enabled: true },
|
||||
{ id: 'clock', name: 'Clock', enabled: false },
|
||||
{ id: 'stocks', name: 'Stocks', enabled: true },
|
||||
];
|
||||
|
||||
(async () => {
|
||||
console.log('\nVegas: a disabled plugin keeps its place and its exclusion');
|
||||
{
|
||||
const t = await mount({ plugins: PLUGINS, order: ['weather', 'clock', 'stocks'], excluded: ['clock'] });
|
||||
ok('only enabled plugins get a row', same(t.rowIds(), ['weather', 'stocks']), t.rowIds());
|
||||
ok('drawing the list keeps the disabled plugin in the order, in its place',
|
||||
same(t.order(), ['weather', 'clock', 'stocks']), t.order());
|
||||
ok('drawing the list keeps its exclusion', same(t.excluded(), ['clock']), t.excluded());
|
||||
|
||||
// Move Stocks up: the rows swap, and Clock stays in its saved slot.
|
||||
const up = t.row('stocks').querySelectorAll('.plugin-order-move')[0];
|
||||
up.fire('click');
|
||||
ok('reordering the rows fills the other slots in the new order',
|
||||
same(t.order(), ['stocks', 'clock', 'weather']), t.order());
|
||||
|
||||
const include = t.row('weather').querySelector('.plugin-order-include');
|
||||
include.checked = false;
|
||||
include.fire('change');
|
||||
ok('unchecking a row adds it, and the disabled exclusion stays',
|
||||
same([...t.excluded()].sort(), ['clock', 'weather']), t.excluded());
|
||||
include.checked = true;
|
||||
include.fire('change');
|
||||
ok('checking it again removes only that one', same(t.excluded(), ['clock']), t.excluded());
|
||||
}
|
||||
|
||||
console.log('\nRotation order: the same, without exclusions');
|
||||
{
|
||||
const plugins = [
|
||||
{ id: 'clock', enabled: true },
|
||||
{ id: 'off', enabled: false },
|
||||
{ id: 'weather', enabled: true },
|
||||
{ id: 'new', enabled: true },
|
||||
];
|
||||
const t = await mount({ plugins, order: ['clock', 'off', 'weather'] });
|
||||
ok('the disabled plugin keeps its slot; a plugin not in the saved order goes last',
|
||||
same(t.order(), ['clock', 'off', 'weather', 'new']), t.order());
|
||||
}
|
||||
|
||||
console.log('\nAn uninstalled plugin is dropped; a failed list keeps everything');
|
||||
{
|
||||
const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
|
||||
excluded: ['gone', 'clock'] });
|
||||
ok('the disabled plugin is kept and the uninstalled one dropped from the order',
|
||||
same(t.order(), ['weather', 'clock', 'stocks']), t.order());
|
||||
ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
|
||||
}
|
||||
{
|
||||
const t = await mount({ plugins: PLUGINS, order: ['weather', 'gone', 'clock', 'stocks'],
|
||||
excluded: ['gone', 'clock'], fetchFails: true });
|
||||
// No installed list, so nothing can be told apart: no rows, and the
|
||||
// inputs keep what was saved, uninstalled ids included.
|
||||
ok('a failed plugin list draws no rows', t.rowIds().length === 0, t.rowIds());
|
||||
ok('and leaves the saved order as it was',
|
||||
same(t.order(), ['weather', 'gone', 'clock', 'stocks']), t.order());
|
||||
ok('and the saved exclusions', same(t.excluded(), ['gone', 'clock']), t.excluded());
|
||||
}
|
||||
|
||||
console.log('\nOnly what the server would accept is carried over');
|
||||
{
|
||||
const t = await mount({ plugins: PLUGINS, order: ['weather', 7, 'clock', null, 'clock', 'stocks'],
|
||||
excluded: ['clock', 3, 'clock'] });
|
||||
// /config/main refuses a list holding anything but strings, which would
|
||||
// block every later Display save; a repeated id is kept once.
|
||||
ok('non-string and repeated saved ids are dropped from the order',
|
||||
same(t.order(), ['weather', 'clock', 'stocks']), t.order());
|
||||
ok('and from the exclusions', same(t.excluded(), ['clock']), t.excluded());
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,105 @@
|
||||
// The Plugin Store's category filter offers the categories its plugins have.
|
||||
//
|
||||
// The template listed seven fixed categories. The registry uses about
|
||||
// twenty (productivity, utility, transit, finance, ...), so roughly a third
|
||||
// of the store could not be filtered to at all, and "Financial" missed the
|
||||
// plugin filed under "finance". The options are now built from the store's
|
||||
// plugins, as the Starlark section builds its own; the template ships only
|
||||
// "All Categories".
|
||||
//
|
||||
// Runs the whole of plugins_manager.js in the sandbox.
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { create, templateAttributes } = require('../plugins_manager_sandbox');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra).slice(0, 400) : '')));
|
||||
|
||||
const STORE = [
|
||||
{ id: 'nfl', name: 'NFL', category: 'sports' },
|
||||
{ id: 'nba', name: 'NBA', category: 'Sports' },
|
||||
{ id: 'todo', name: 'Todo', category: 'productivity' },
|
||||
{ id: 'stocks', name: 'Stocks', category: 'finance' },
|
||||
{ id: 'crypto', name: 'Crypto', category: 'financial' },
|
||||
{ id: 'bus', name: 'Bus', category: 'transit' },
|
||||
{ id: 'mystery', name: 'Mystery' },
|
||||
];
|
||||
|
||||
function route(method, url) {
|
||||
if (url.startsWith('/api/v3/plugins/store/list')) return { json: { status: 'success', data: { plugins: STORE } } };
|
||||
if (url.startsWith('/api/v3/plugins/installed')) return { json: { status: 'success', data: { plugins: [] } } };
|
||||
if (url.startsWith('/api/v3/plugins/store/github-status')) {
|
||||
return { json: { status: 'success', data: { token_status: 'valid', authenticated: true, rate_limit: 5000 } } };
|
||||
}
|
||||
if (url.startsWith('/api/v3/plugins/saved-repositories')) {
|
||||
return { json: { status: 'success', data: { repositories: [] } } };
|
||||
}
|
||||
if (url.startsWith('/api/v3/display/on-demand/status')) {
|
||||
return { json: { status: 'success', data: { state: {}, service: {} } } };
|
||||
}
|
||||
return { json: { status: 'success' } };
|
||||
}
|
||||
|
||||
const options = sel => sel.children.map(o => o.value);
|
||||
const cardIds = sb => [...sb.el('plugin-store-grid').innerHTML.matchAll(/<h4[^>]*>([^<]*)<\/h4>/g)].map(m => m[1]);
|
||||
|
||||
(async () => {
|
||||
console.log('\nthe template');
|
||||
{
|
||||
const html = fs.readFileSync(path.resolve(__dirname,
|
||||
'../../../web_interface/templates/v3/partials/plugins.html'), 'utf8');
|
||||
const start = html.indexOf('<select id="plugin-category"');
|
||||
const block = html.slice(start, html.indexOf('</select>', start));
|
||||
const shipped = [...block.matchAll(/<option value="([^"]*)"/g)].map(m => m[1]);
|
||||
ok('ships only "All Categories"', JSON.stringify(shipped) === JSON.stringify(['']), shipped);
|
||||
}
|
||||
|
||||
const sb = create({ route });
|
||||
const attrs = templateAttributes('plugin-category').attrs;
|
||||
const select = sb.el('plugin-category', 'select', attrs);
|
||||
sb.el('plugin-store-grid');
|
||||
sb.el('installed-plugins-grid');
|
||||
sb.window.initPluginsPage();
|
||||
await sb.until(() => sb.requests.some(r => r.url.startsWith('/api/v3/plugins/store/list')), 'the store list');
|
||||
await sb.settle();
|
||||
|
||||
console.log('\noptions come from the store\'s plugins');
|
||||
ok('"All Categories" is still the first choice', /<option value="">All Categories<\/option>/.test(select.innerHTML),
|
||||
select.innerHTML);
|
||||
ok('every category a plugin has is offered once, whatever its case',
|
||||
JSON.stringify(options(select)) === JSON.stringify(['finance', 'financial', 'productivity', 'sports', 'transit']),
|
||||
options(select));
|
||||
ok('labels are capitalised',
|
||||
(select.children.find(o => o.value === 'productivity') || {}).textContent === 'Productivity');
|
||||
|
||||
console.log('\nchoosing one filters to it');
|
||||
select.value = 'productivity';
|
||||
select.dispatch('change');
|
||||
ok('productivity shows its plugin', JSON.stringify(cardIds(sb)) === JSON.stringify(['Todo']), cardIds(sb));
|
||||
select.value = 'finance';
|
||||
select.dispatch('change');
|
||||
ok('finance is not lost to "financial"', JSON.stringify(cardIds(sb)) === JSON.stringify(['Stocks']), cardIds(sb));
|
||||
select.value = 'sports';
|
||||
select.dispatch('change');
|
||||
ok('one option covers both spellings of sports',
|
||||
JSON.stringify(cardIds(sb).sort()) === JSON.stringify(['NBA', 'NFL']), cardIds(sb));
|
||||
|
||||
console.log('\nthe partial is swapped back in (tab switch)');
|
||||
{
|
||||
// A fresh <select> from the template, the store list still cached.
|
||||
const fresh = new sb.FakeElement('plugin-category', 'select', attrs);
|
||||
select.parentNode.replaceChild(fresh, select);
|
||||
sb.window.searchPluginStore(false);
|
||||
await sb.settle();
|
||||
ok('the new select is filled from the cache',
|
||||
JSON.stringify(options(fresh)) === JSON.stringify(['finance', 'financial', 'productivity', 'sports', 'transit']),
|
||||
options(fresh));
|
||||
ok('keeping the chosen category', fresh.value === 'sports', fresh.value);
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,138 @@
|
||||
// The store's Install button: which plugin it enables afterwards, and when.
|
||||
//
|
||||
// 1. Weather, Music, Stocks and Leaderboard are registry entries (`weather`)
|
||||
// whose manifests declare another id (`ledmatrix-weather`). The plugin
|
||||
// list, its config section and /plugins/toggle know them by that id, but
|
||||
// the button enabled the registry id: /plugins/toggle answered 404
|
||||
// "Plugin not found" and the plugin stayed disabled behind "installed,
|
||||
// but enabling it failed". It now enables the id the install answer
|
||||
// names (`plugin_id`), or, from an answer without one, the installed
|
||||
// entry the store entry matches (its id, plugin_path name or aliases).
|
||||
//
|
||||
// 2. Reinstall (the same button on an installed plugin) enabled it too, so
|
||||
// reinstalling a plugin the user had switched off switched it back on.
|
||||
// Only a fresh install enables.
|
||||
//
|
||||
// Runs the whole of plugins_manager.js in the sandbox against a fake API.
|
||||
|
||||
const { create } = require('../plugins_manager_sandbox');
|
||||
|
||||
let pass = 0, fail = 0;
|
||||
const ok = (label, cond, extra) => cond
|
||||
? (pass++, console.log(' ok ' + label))
|
||||
: (fail++, console.log(' FAIL ' + label + (extra !== undefined ? ' -> ' + JSON.stringify(extra).slice(0, 400) : '')));
|
||||
|
||||
const STORE = [
|
||||
{ id: 'weather', name: 'Weather', category: 'weather', plugin_path: 'plugins/ledmatrix-weather',
|
||||
aliases: ['ledmatrix-weather'] },
|
||||
{ id: 'clock-simple', name: 'Clock', category: 'time', plugin_path: 'plugins/clock-simple', aliases: [] },
|
||||
];
|
||||
|
||||
// A server with one install in flight. `queue` false answers the install
|
||||
// directly; `names` false leaves plugin_id out of the answer (an older
|
||||
// server); `installsAs` is the id the installed manifest declares.
|
||||
function server({ installed = [], queue = true, names = true, installsAs }) {
|
||||
const state = { installed: installed.map(p => ({ ...p })), polls: 0 };
|
||||
const done = (id) => {
|
||||
if (!state.installed.some(p => p.id === installsAs)) {
|
||||
state.installed.push({ id: installsAs, name: id, enabled: false });
|
||||
}
|
||||
const result = { success: true, message: `Plugin ${id} installed successfully`, restart_required: false };
|
||||
if (names) result.plugin_id = installsAs;
|
||||
return result;
|
||||
};
|
||||
state.route = (method, url, body) => {
|
||||
if (url.startsWith('/api/v3/plugins/store/list')) {
|
||||
return { json: { status: 'success', data: { plugins: STORE } } };
|
||||
}
|
||||
if (url.startsWith('/api/v3/plugins/installed')) {
|
||||
return { json: { status: 'success', data: { plugins: state.installed.map(p => ({ ...p })) } } };
|
||||
}
|
||||
if (method === 'POST' && url === '/api/v3/plugins/install') {
|
||||
if (queue) return { json: { status: 'success', message: 'queued', data: { operation_id: 'op-1' } } };
|
||||
return { json: { status: 'success', message: 'Plugin installed successfully', ...done(body.plugin_id) } };
|
||||
}
|
||||
if (url === '/api/v3/plugins/operation/op-1') {
|
||||
state.polls++;
|
||||
if (state.polls < 3) return { json: { status: 'success', data: { status: 'running' } } };
|
||||
return { json: { status: 'success', data: { status: 'completed', result: done('weather') } } };
|
||||
}
|
||||
if (method === 'POST' && url === '/api/v3/plugins/toggle') {
|
||||
const plugin = state.installed.find(p => p.id === body.plugin_id);
|
||||
if (!plugin) return { status: 404, json: { status: 'error', message: 'Plugin not found' } };
|
||||
plugin.enabled = body.enabled;
|
||||
return { json: { status: 'success', message: `Plugin ${body.plugin_id} enabled successfully` } };
|
||||
}
|
||||
return { json: { status: 'success' } };
|
||||
};
|
||||
return state;
|
||||
}
|
||||
|
||||
async function install(pluginId, opts) {
|
||||
const srv = server(opts);
|
||||
const sb = create({ route: srv.route });
|
||||
sb.window.searchPluginStore();
|
||||
await sb.until(() => sb.requests.some(r => r.url.startsWith('/api/v3/plugins/store/list')), 'store list');
|
||||
await sb.window.pluginManager.loadInstalledPlugins(true);
|
||||
await sb.settle();
|
||||
sb.requests.length = 0;
|
||||
sb.toasts.length = 0;
|
||||
sb.window.installPlugin(pluginId);
|
||||
await sb.until(() => sb.toasts.some(t => /installed and enabled|enabling it failed|reinstalled/.test(t.message)),
|
||||
'the install to finish');
|
||||
await sb.settle();
|
||||
const toggles = sb.requests.filter(r => r.url === '/api/v3/plugins/toggle').map(r => r.body);
|
||||
return { sb, srv, toggles };
|
||||
}
|
||||
|
||||
(async () => {
|
||||
console.log('\n1. a fresh install enables the id the plugin was installed as');
|
||||
{
|
||||
const { srv, toggles, sb } = await install('weather', { installsAs: 'ledmatrix-weather' });
|
||||
ok('enables ledmatrix-weather, not the registry id',
|
||||
JSON.stringify(toggles) === JSON.stringify([{ plugin_id: 'ledmatrix-weather', enabled: true }]), toggles);
|
||||
ok('...which the server enabled', srv.installed.find(p => p.id === 'ledmatrix-weather').enabled === true, srv.installed);
|
||||
ok('says so', sb.toasts.some(t => t.type === 'success' && /installed and enabled/.test(t.message)), sb.toasts);
|
||||
ok('no "Plugin not found"', !sb.toasts.some(t => /not found|failed/.test(t.message)), sb.toasts);
|
||||
const lastList = sb.requests.map(r => r.url).lastIndexOf('/api/v3/plugins/installed');
|
||||
const toggleAt = sb.requests.findIndex(r => r.url === '/api/v3/plugins/toggle');
|
||||
ok('the installed list is reloaded before enabling, so the new card is there to update',
|
||||
lastList >= 0 && lastList < toggleAt, sb.requests.map(r => r.method + ' ' + r.url));
|
||||
}
|
||||
{
|
||||
const { toggles } = await install('weather', { installsAs: 'ledmatrix-weather', names: false });
|
||||
ok('an answer without plugin_id: the installed entry the store entry matches (its alias)',
|
||||
JSON.stringify(toggles) === JSON.stringify([{ plugin_id: 'ledmatrix-weather', enabled: true }]), toggles);
|
||||
}
|
||||
{
|
||||
const { toggles } = await install('weather', { installsAs: 'ledmatrix-weather', queue: false });
|
||||
ok('without the operation queue, from the direct answer',
|
||||
JSON.stringify(toggles) === JSON.stringify([{ plugin_id: 'ledmatrix-weather', enabled: true }]), toggles);
|
||||
}
|
||||
{
|
||||
const { toggles } = await install('clock-simple', { installsAs: 'clock-simple', names: false });
|
||||
ok('a plugin installed under its registry id is enabled by that id',
|
||||
JSON.stringify(toggles) === JSON.stringify([{ plugin_id: 'clock-simple', enabled: true }]), toggles);
|
||||
}
|
||||
|
||||
console.log('\n2. a reinstall leaves the plugin as the user had it');
|
||||
{
|
||||
const { srv, toggles, sb } = await install('weather', {
|
||||
installsAs: 'ledmatrix-weather', installed: [{ id: 'ledmatrix-weather', name: 'Weather', enabled: false }],
|
||||
});
|
||||
ok('sends no toggle', toggles.length === 0, toggles);
|
||||
ok('the plugin stays disabled', srv.installed.find(p => p.id === 'ledmatrix-weather').enabled === false);
|
||||
ok('says it was reinstalled', sb.toasts.some(t => t.type === 'success' && /reinstalled/.test(t.message)), sb.toasts);
|
||||
ok('and reloads the list',
|
||||
sb.requests.some(r => r.url === '/api/v3/plugins/installed'), sb.requests.map(r => r.url));
|
||||
}
|
||||
{
|
||||
const { toggles } = await install('weather', {
|
||||
installsAs: 'ledmatrix-weather', installed: [{ id: 'ledmatrix-weather', name: 'Weather', enabled: true }],
|
||||
});
|
||||
ok('an enabled plugin is not toggled either', toggles.length === 0, toggles);
|
||||
}
|
||||
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(fail ? 1 : 0);
|
||||
})().catch(e => { console.error(e); process.exit(1); });
|
||||
@@ -52,7 +52,8 @@ global.installedPlugins = [];
|
||||
// eslint-disable-next-line no-eval
|
||||
eval([
|
||||
'function escapeHtml(text) {', 'function escapeAttribute(text) {', 'function jsStringAttr(value) {',
|
||||
'function isStorePluginInstalled(pluginIdOrPlugin) {', 'function renderPluginStore(plugins) {',
|
||||
'function isStorePluginInstalled(pluginIdOrPlugin) {',
|
||||
'function findInstalledStorePlugin(pluginIdOrPlugin) {', 'function renderPluginStore(plugins) {',
|
||||
].map(extract).join('\n') + '\nglobal.renderPluginStore = renderPluginStore;'
|
||||
+ '\nglobal.isStorePluginInstalled = isStorePluginInstalled;');
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ function fakeApi(behaviour = {}) {
|
||||
};
|
||||
}
|
||||
|
||||
function setup(api, { stateList, windowList } = {}) {
|
||||
function setup(api, { stateList, windowList, pluginManager } = {}) {
|
||||
global.window = {
|
||||
PluginAPI: api,
|
||||
installedPlugins: windowList,
|
||||
@@ -60,6 +60,7 @@ function setup(api, { stateList, windowList } = {}) {
|
||||
installedPlugins: stateList,
|
||||
loadInstalledPlugins: async () => stateList,
|
||||
},
|
||||
pluginManager,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -92,12 +93,68 @@ const noSleep = { sleep: async () => {} };
|
||||
ok('progress total counts only what is sent',
|
||||
progress.length === EXPECTED.length && progress.every(([, n]) => n === EXPECTED.length), progress);
|
||||
}
|
||||
{
|
||||
// A page without the plugin manager has no window.installedPlugins.
|
||||
const api = fakeApi();
|
||||
setup(api, { stateList: INSTALLED });
|
||||
await Manager.updateAll(null, noSleep);
|
||||
ok('the PluginStateManager list (no live list) is filtered the same way',
|
||||
JSON.stringify(api.calls) === JSON.stringify(EXPECTED), api.calls);
|
||||
}
|
||||
|
||||
console.log('\na second run sends the live list, not the first run\'s snapshot');
|
||||
{
|
||||
// Run 1 leaves PluginStateManager holding a, b, c. Then c is uninstalled
|
||||
// and d installed: plugins_manager.js publishes that only as
|
||||
// window.installedPlugins. Run 2 used to send a, b, c -- c failed as
|
||||
// "plugin not found" and d, which had an update waiting, was skipped.
|
||||
const api = fakeApi({
|
||||
c: () => { throw { error_code: 'PLUGIN_UPDATE_FAILED', message: 'Plugin update failed: plugin not found' }; },
|
||||
});
|
||||
const stale = [{ id: 'a' }, { id: 'b' }, { id: 'c' }];
|
||||
setup(api, { stateList: stale, windowList: [{ id: 'a' }, { id: 'b' }, { id: 'd' }] });
|
||||
const results = await Manager.updateAll(null, noSleep);
|
||||
ok('sends exactly what is installed now',
|
||||
JSON.stringify(api.calls) === JSON.stringify(['a', 'b', 'd']), api.calls);
|
||||
ok('...so nothing fails over an uninstalled plugin', results.every(r => r.success), results);
|
||||
}
|
||||
{
|
||||
const api = fakeApi();
|
||||
setup(api, { stateList: INSTALLED, windowList: [] });
|
||||
const results = await Manager.updateAll(null, noSleep);
|
||||
ok('an empty live list means nothing is installed: nothing is sent',
|
||||
api.calls.length === 0 && results.length === 0, api.calls);
|
||||
}
|
||||
|
||||
console.log('\nthe end-of-run refresh redraws the installed grid');
|
||||
{
|
||||
// PluginStateManager's refresh replaced window.installedPlugins and
|
||||
// nothing else: the cards kept "Update to vX" and the Updates badge
|
||||
// kept its count. The plugin manager's load renders the grid.
|
||||
const loads = [];
|
||||
let stateLoads = 0;
|
||||
const pluginManager = { loadInstalledPlugins: async (force) => { loads.push(force); } };
|
||||
setup(fakeApi(), { stateList: INSTALLED, windowList: INSTALLED, pluginManager });
|
||||
window.PluginStateManager.loadInstalledPlugins = async () => { stateLoads++; };
|
||||
await Manager.updateAll(null, noSleep);
|
||||
ok('the PluginStateManager list is filtered the same way',
|
||||
JSON.stringify(api.calls) === JSON.stringify(EXPECTED), api.calls);
|
||||
ok('reloads through the plugin manager once, forced past its caches',
|
||||
JSON.stringify(loads) === JSON.stringify([true]), loads);
|
||||
ok('...instead of PluginStateManager', stateLoads === 0, stateLoads);
|
||||
}
|
||||
{
|
||||
const pluginManager = { loadInstalledPlugins: async () => { throw new Error('offline'); } };
|
||||
const answer = { status: 'success', data: { update_status: 'updated' }, restart_required: true };
|
||||
setup(fakeApi({ 'ledmatrix-flights': () => answer }), { windowList: INSTALLED, pluginManager });
|
||||
const warn = console.warn;
|
||||
console.warn = () => {};
|
||||
let results;
|
||||
try {
|
||||
results = await Manager.updateAll(null, noSleep);
|
||||
} finally {
|
||||
console.warn = warn;
|
||||
}
|
||||
ok('a failed plugin-manager reload still returns the results with their restart flag',
|
||||
Array.isArray(results) && Manager.restartRequest(results) === answer);
|
||||
}
|
||||
{
|
||||
const api = fakeApi();
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
"""POST /plugins/install says which id the plugin was installed as.
|
||||
|
||||
A registry entry can install under another id: `weather` (aliases
|
||||
`ledmatrix-weather`) installs a directory whose manifest declares
|
||||
`ledmatrix-weather`, and that is the id the plugin list, the plugin's config
|
||||
section and /plugins/toggle know it by. The store's Install button enabled
|
||||
the new plugin by the registry id, which /plugins/toggle answered with 404
|
||||
"Plugin not found", so Weather, Music, Stocks and Leaderboard installed
|
||||
disabled behind an "enabling it failed" warning.
|
||||
|
||||
The answer -- the queued operation's result, or the direct response --
|
||||
carries `plugin_id`: the id the installed manifest declares, found the way
|
||||
the store's update and uninstall find an install.
|
||||
"""
|
||||
|
||||
import json
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401
|
||||
|
||||
INSTALL = "/api/v3/plugins/install"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def store(api_v3_module, tmp_path):
|
||||
manager = api_v3_module.api_v3.plugin_store_manager
|
||||
manager.install_plugin.return_value = True
|
||||
manager.get_registry_info.return_value = None
|
||||
manager._find_plugin_path.return_value = None
|
||||
|
||||
def installed_as(directory, manifest):
|
||||
path = tmp_path / directory
|
||||
path.mkdir()
|
||||
(path / "manifest.json").write_text(json.dumps(manifest), encoding="utf-8")
|
||||
manager._find_plugin_path.side_effect = (
|
||||
lambda pid: path if pid == "weather" else None)
|
||||
return path
|
||||
|
||||
manager.installed_as = installed_as
|
||||
return manager
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def queued(api_v3_module):
|
||||
queue = MagicMock()
|
||||
|
||||
def enqueue(operation_type, plugin_id, operation_callback=None):
|
||||
queue.callback_result = operation_callback(MagicMock())
|
||||
return "op-1"
|
||||
|
||||
queue.enqueue_operation.side_effect = enqueue
|
||||
api_v3_module.api_v3.operation_queue = queue
|
||||
return queue
|
||||
|
||||
|
||||
class TestDirectInstall:
|
||||
def test_an_aliased_entry_reports_the_manifest_id(self, api_v3_client, store):
|
||||
store.installed_as("ledmatrix-weather", {"id": "ledmatrix-weather"})
|
||||
body = api_v3_client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
|
||||
assert body["status"] == "success"
|
||||
assert body["plugin_id"] == "ledmatrix-weather"
|
||||
store._find_plugin_path.assert_called_with("weather")
|
||||
|
||||
def test_an_entry_installed_under_its_own_id_reports_that(self, api_v3_client, store):
|
||||
store.installed_as("weather", {"id": "weather"})
|
||||
body = api_v3_client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
|
||||
assert body["plugin_id"] == "weather"
|
||||
|
||||
def test_an_install_that_cannot_be_found_reports_the_requested_id(self, api_v3_client, store):
|
||||
body = api_v3_client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
|
||||
assert body["status"] == "success"
|
||||
assert body["plugin_id"] == "weather"
|
||||
|
||||
def test_a_manifest_id_that_is_not_a_plain_name_is_not_passed_on(self, api_v3_client, store):
|
||||
store.installed_as("ledmatrix-weather", {"id": "../elsewhere"})
|
||||
body = api_v3_client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
|
||||
assert body["plugin_id"] == "weather"
|
||||
|
||||
def test_an_unreadable_manifest_reports_the_requested_id(self, api_v3_client, store):
|
||||
path = store.installed_as("ledmatrix-weather", {})
|
||||
(path / "manifest.json").write_text("[not json", encoding="utf-8")
|
||||
body = api_v3_client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
|
||||
assert body["plugin_id"] == "weather"
|
||||
|
||||
def test_the_restart_fields_are_still_sent(self, api_v3_client, store):
|
||||
store.installed_as("ledmatrix-weather", {"id": "ledmatrix-weather"})
|
||||
body = api_v3_client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
|
||||
assert "restart_required" in body
|
||||
|
||||
|
||||
class TestQueuedInstall:
|
||||
def test_the_operation_result_names_the_manifest_id(self, api_v3_client, store, queued):
|
||||
store.installed_as("ledmatrix-weather", {"id": "ledmatrix-weather"})
|
||||
body = api_v3_client.post(INSTALL, json={"plugin_id": "weather"}).get_json()
|
||||
assert body["data"]["operation_id"] == "op-1"
|
||||
assert queued.callback_result["success"] is True
|
||||
assert queued.callback_result["plugin_id"] == "ledmatrix-weather"
|
||||
|
||||
def test_an_install_that_cannot_be_found_names_the_requested_id(
|
||||
self, api_v3_client, store, queued):
|
||||
api_v3_client.post(INSTALL, json={"plugin_id": "weather"})
|
||||
assert queued.callback_result["plugin_id"] == "weather"
|
||||
@@ -0,0 +1,59 @@
|
||||
"""GET /api/v3/plugins/installed carries each plugin's ``display_modes``.
|
||||
|
||||
The on-demand modal (plugins_manager.js) fills its Display Mode list from
|
||||
``plugin.display_modes``, but the route never included the field, so every
|
||||
plugin offered one option -- its own id -- under "This plugin exposes a
|
||||
single display mode". The display turns that id into the plugin's first
|
||||
mode, so a multi-mode plugin could only be started, and pinned, on that one.
|
||||
|
||||
The modes come from the plugin catalog (the manifests the web process
|
||||
discovered), the same source /display/modes and on-demand/start use.
|
||||
"""
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from test._api_v3_test_helpers import ( # noqa: F401 - fixtures
|
||||
api_v3_client, api_v3_module,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def installed(api_v3_module, api_v3_client, tmp_path):
|
||||
def _get(declared_modes):
|
||||
api = api_v3_module.api_v3
|
||||
# The listing's own metadata says nothing about modes: what the
|
||||
# route reports must come from the catalog.
|
||||
info = {'id': 'football-scoreboard', 'name': 'Football', 'version': '1.0.0'}
|
||||
api.plugin_catalog.plugins_dir = str(tmp_path) # no manifest on disk
|
||||
api.plugin_catalog.get_all_plugin_info = MagicMock(return_value=[info])
|
||||
api.plugin_catalog.get_plugin_display_modes = MagicMock(return_value=declared_modes)
|
||||
api.plugin_store_manager.get_registry_info = MagicMock(return_value=None)
|
||||
api.config_manager.load_config = MagicMock(return_value={})
|
||||
response = api_v3_client.get('/api/v3/plugins/installed')
|
||||
assert response.status_code == 200
|
||||
plugins = [p for p in response.get_json()['data']['plugins']
|
||||
if p['id'] == 'football-scoreboard']
|
||||
assert len(plugins) == 1
|
||||
api.plugin_catalog.get_plugin_display_modes.assert_any_call('football-scoreboard')
|
||||
return plugins[0]
|
||||
return _get
|
||||
|
||||
|
||||
def test_every_declared_mode_is_listed_in_order(installed):
|
||||
modes = ['nfl_live', 'nfl_recent', 'nfl_upcoming']
|
||||
assert installed(modes)['display_modes'] == modes
|
||||
|
||||
|
||||
def test_a_single_mode_plugin_lists_its_one_mode(installed):
|
||||
assert installed(['clock-simple'])['display_modes'] == ['clock-simple']
|
||||
|
||||
|
||||
def test_no_declared_modes_is_an_empty_list(installed):
|
||||
# The modal falls back to the plugin id for an empty list.
|
||||
assert installed([])['display_modes'] == []
|
||||
|
||||
|
||||
def test_a_hand_edited_manifest_cannot_put_non_strings_in_the_list(installed):
|
||||
assert installed(['nfl_live', 7, None, {'x': 1}])['display_modes'] == ['nfl_live']
|
||||
@@ -37,6 +37,7 @@ from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401
|
||||
START_URL = "/api/v3/display/on-demand/start"
|
||||
STOP_URL = "/api/v3/display/on-demand/stop"
|
||||
MAILBOX = "display_on_demand_request"
|
||||
DISPLAY = "web_interface.blueprints.api_v3.display"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
@@ -150,6 +151,102 @@ class TestStartWhileTheServiceIsStopped:
|
||||
assert response.get_json()["status"] == "error"
|
||||
|
||||
|
||||
class _Mailbox:
|
||||
"""The CacheManager calls the routes make, over a dict."""
|
||||
|
||||
def __init__(self):
|
||||
self.entries = {}
|
||||
|
||||
def set(self, key, value, ttl=None):
|
||||
self.entries[key] = value
|
||||
|
||||
def get(self, key, max_age=300, memory_ttl=None):
|
||||
return self.entries.get(key)
|
||||
|
||||
def delete(self, key):
|
||||
self.entries.pop(key, None)
|
||||
|
||||
|
||||
class TestARefusedStartLeavesNoRequestBehind:
|
||||
"""A start the route answers with an error must not run later.
|
||||
|
||||
The request was posted (to the mailbox, with the display stopped) before
|
||||
the route refused it, and the display reads the mailbox for an hour
|
||||
without looking at a request's age. So "Display service is not running"
|
||||
(start_service off) or "Failed to start display service" left the
|
||||
request waiting, and the next time the display started -- minutes later,
|
||||
by hand -- it ran that plugin, pinned if the request said so.
|
||||
|
||||
A socket acknowledgement is the other side of it: the display answered,
|
||||
so it is running and has the request, whatever systemd says (a display
|
||||
run by hand or in the emulator has no active unit). That is a success,
|
||||
not "not running", and no unit is started beside it.
|
||||
"""
|
||||
|
||||
@pytest.fixture
|
||||
def mailbox(self, api_v3_module, service):
|
||||
box = _Mailbox()
|
||||
api_v3_module.api_v3.cache_manager = box
|
||||
service["state"]["active"] = False
|
||||
return box
|
||||
|
||||
@pytest.mark.parametrize("body", [
|
||||
{"plugin_id": "weather", "start_service": False},
|
||||
{"plugin_id": "weather"}, # start_service defaults on
|
||||
])
|
||||
def test_a_socket_ack_is_a_success_whatever_systemd_says(
|
||||
self, api_v3_client, service, mailbox, body):
|
||||
with patch(f"{DISPLAY}.control_client.on_demand_start",
|
||||
side_effect=lambda request_id, *a: {"accepted": True}):
|
||||
response = api_v3_client.post(START_URL, json=body)
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert response.get_json()["data"]["transport"] == "socket"
|
||||
assert MAILBOX not in mailbox.entries
|
||||
assert _systemctl_verbs(service["systemctl"]) == [], (
|
||||
"a unit was started beside a display that answered the socket")
|
||||
|
||||
def test_without_start_service_the_request_is_taken_back(
|
||||
self, api_v3_client, service, mailbox):
|
||||
response = api_v3_client.post(START_URL, json={
|
||||
"plugin_id": "weather", "pinned": True, "start_service": False})
|
||||
assert response.status_code == 400
|
||||
assert response.get_json()["status"] == "error"
|
||||
assert MAILBOX not in mailbox.entries
|
||||
|
||||
def test_a_start_that_fails_takes_its_request_back(self, api_v3_client, service, mailbox):
|
||||
service["systemctl"].side_effect = lambda args: {
|
||||
"returncode": 1, "stdout": "", "stderr": "denied"}
|
||||
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 500
|
||||
assert MAILBOX not in mailbox.entries
|
||||
|
||||
def test_a_newer_request_is_left_alone_on_the_400(self, api_v3_client, service, mailbox):
|
||||
newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
|
||||
|
||||
def stopped_and_another_post_lands(*args):
|
||||
mailbox.entries[MAILBOX] = newer
|
||||
return {"active": False}
|
||||
|
||||
with patch(f"{DISPLAY}._get_display_service_status",
|
||||
side_effect=stopped_and_another_post_lands):
|
||||
response = api_v3_client.post(START_URL, json={
|
||||
"plugin_id": "weather", "start_service": False})
|
||||
assert response.status_code == 400
|
||||
assert mailbox.entries[MAILBOX] is newer
|
||||
|
||||
def test_a_newer_request_is_left_alone_on_the_500(self, api_v3_client, service, mailbox):
|
||||
newer = {"request_id": "someone-else", "action": "start", "plugin_id": "clock"}
|
||||
|
||||
def start_fails_after_another_post(args):
|
||||
mailbox.entries[MAILBOX] = newer
|
||||
return {"returncode": 1, "stdout": "", "stderr": "denied"}
|
||||
|
||||
service["systemctl"].side_effect = start_fails_after_another_post
|
||||
response = api_v3_client.post(START_URL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 500
|
||||
assert mailbox.entries[MAILBOX] is newer
|
||||
|
||||
|
||||
class TestStop:
|
||||
def test_stop_posts_a_stop_request_and_leaves_the_service_running(
|
||||
self, api_v3_client, service):
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
"""GET /api/v3/plugins/operation/<id> answers for an operation still waiting.
|
||||
|
||||
PluginOperationQueue keeps an operation's callback in its parameters, under
|
||||
``_callback``, until the worker takes it to run. PluginOperation.to_dict()
|
||||
returned the parameters as they were, so for a pending operation the route
|
||||
handed jsonify a function and answered 500 "A system error occurred". That
|
||||
is every poll of an install queued behind another plugin's: the second of
|
||||
two installs read as broken until the first one finished.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
from src.plugin_system.operation_queue import PluginOperationQueue # noqa: E402
|
||||
from src.plugin_system.operation_types import ( # noqa: E402
|
||||
OperationType, PluginOperation,
|
||||
)
|
||||
|
||||
|
||||
def _callback(op):
|
||||
return {"success": True, "message": "done"}
|
||||
|
||||
|
||||
class TestToDict:
|
||||
def test_private_parameters_are_left_out(self):
|
||||
op = PluginOperation(OperationType.INSTALL, "demo",
|
||||
parameters={"_callback": _callback, "branch": "main"})
|
||||
assert op.to_dict()["parameters"] == {"branch": "main"}
|
||||
json.dumps(op.to_dict()) # serializable
|
||||
|
||||
def test_the_operation_keeps_its_callback_for_the_worker(self):
|
||||
op = PluginOperation(OperationType.INSTALL, "demo",
|
||||
parameters={"_callback": _callback})
|
||||
op.to_dict()
|
||||
assert op.parameters["_callback"] is _callback
|
||||
|
||||
def test_the_other_fields_are_unchanged(self):
|
||||
op = PluginOperation(OperationType.UNINSTALL, "demo", operation_id="op-1")
|
||||
assert op.to_dict() == {
|
||||
"operation_id": "op-1", "operation_type": "uninstall", "plugin_id": "demo",
|
||||
"parameters": {}, "status": "pending", "progress": 0.0, "message": "",
|
||||
"error": None, "result": None,
|
||||
"created_at": op.created_at.isoformat(), "started_at": None,
|
||||
"completed_at": None,
|
||||
}
|
||||
|
||||
|
||||
class TestTheRoute:
|
||||
@pytest.fixture
|
||||
def busy_queue(self, api_v3_module):
|
||||
"""A real queue whose worker is held by another plugin's operation."""
|
||||
queue = PluginOperationQueue(max_history=10)
|
||||
api_v3_module.api_v3.operation_queue = queue
|
||||
started, release = threading.Event(), threading.Event()
|
||||
|
||||
def blocker(op):
|
||||
started.set()
|
||||
release.wait(10)
|
||||
return {"success": True, "message": "done"}
|
||||
|
||||
queue.enqueue_operation(OperationType.INSTALL, "busy", operation_callback=blocker)
|
||||
assert started.wait(5)
|
||||
yield queue
|
||||
release.set()
|
||||
queue.shutdown()
|
||||
|
||||
def test_a_pending_operation_reports_pending(self, api_v3_client, busy_queue):
|
||||
op_id = busy_queue.enqueue_operation(
|
||||
OperationType.INSTALL, "demo", operation_callback=_callback)
|
||||
response = api_v3_client.get(f"/api/v3/plugins/operation/{op_id}")
|
||||
assert response.status_code == 200, response.get_json()
|
||||
data = response.get_json()["data"]
|
||||
assert data["status"] == "pending"
|
||||
assert data["plugin_id"] == "demo"
|
||||
assert "_callback" not in data["parameters"]
|
||||
@@ -253,6 +253,32 @@ class TestVegasCycleDurations:
|
||||
assert saved['config']['display']['display_durations'] == {'clock': 45}
|
||||
|
||||
|
||||
class TestMalformedBody:
|
||||
"""A JSON body that does not parse is the caller's mistake: a 400.
|
||||
|
||||
get_json() raised Werkzeug's BadRequest inside the handler's try, whose
|
||||
catch-all answered 500 CONFIG_SAVE_FAILED with "check file permissions"
|
||||
advice and logged a traceback at ERROR.
|
||||
"""
|
||||
|
||||
def test_is_a_400_in_the_raw_routes_shape(self, api_v3_client, saved, api_v3_module):
|
||||
api_v3_module.api_v3.config_manager.get_raw_file_content.return_value = {}
|
||||
resp = api_v3_client.post('/api/v3/config/main', data='{not json',
|
||||
content_type='application/json')
|
||||
assert resp.status_code == 400
|
||||
assert resp.get_json() == {'status': 'error', 'message': 'Invalid JSON in request body'}
|
||||
assert 'config' not in saved
|
||||
raw = api_v3_client.post('/api/v3/config/raw/main', data='{not json',
|
||||
content_type='application/json')
|
||||
assert (raw.status_code, raw.get_json()) == (400, resp.get_json())
|
||||
|
||||
def test_an_empty_json_post_is_still_no_data(self, api_v3_client, saved):
|
||||
resp = api_v3_client.post('/api/v3/config/main', data='',
|
||||
content_type='application/json')
|
||||
assert resp.status_code == 400
|
||||
assert resp.get_json()['message'] == 'No data provided'
|
||||
|
||||
|
||||
class TestRawSaveStartsAutoUpdateSetup:
|
||||
@pytest.fixture
|
||||
def raw_env(self, api_v3_module, monkeypatch):
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
"""POST /api/v3/plugins/action hands ``params`` to the plugin's script intact.
|
||||
|
||||
The route runs the script through a generated wrapper, and the params went
|
||||
into that wrapper as Python source: ``params = {json.dumps(params)}``. JSON is
|
||||
not Python. ``true``, ``false`` and ``null`` are undefined names there, so any
|
||||
params holding a boolean or a null died with a NameError before the script
|
||||
ran. The plugin file manager's category toggle sends ``{"category_name": ...,
|
||||
"enabled": true}``, so of-the-day's category toggle failed every time with
|
||||
"Action failed".
|
||||
|
||||
The script's side of the contract is unchanged and pinned here too: the
|
||||
params arrive on stdin as one JSON document, LEDMATRIX_ROOT is set, and what
|
||||
the script prints to stdout is what the route parses.
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
ACTION_URL = "/api/v3/plugins/action"
|
||||
|
||||
# The action script: report what it was handed, as JSON on stdout.
|
||||
ECHO_SCRIPT = (
|
||||
"import json, os, sys\n"
|
||||
"raw = sys.stdin.read()\n"
|
||||
"print(json.dumps({'status': 'success', 'got': json.loads(raw),\n"
|
||||
" 'root': os.environ.get('LEDMATRIX_ROOT')}))\n"
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def echo_plugin(tmp_path, api_v3_module, monkeypatch):
|
||||
plugin_dir = tmp_path / "demo"
|
||||
plugin_dir.mkdir()
|
||||
(plugin_dir / "manifest.json").write_text(json.dumps({
|
||||
"id": "demo",
|
||||
"web_ui_actions": [{"id": "toggle", "type": "script", "script": "echo.py"}],
|
||||
}), encoding="utf-8")
|
||||
(plugin_dir / "echo.py").write_text(ECHO_SCRIPT, encoding="utf-8")
|
||||
api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = str(plugin_dir)
|
||||
|
||||
# The route runs `python3`; use this interpreter, so the test does not
|
||||
# depend on what that name resolves to here.
|
||||
real_run = subprocess.run
|
||||
|
||||
def run(cmd, *args, **kwargs):
|
||||
if isinstance(cmd, list) and cmd and cmd[0] == "python3":
|
||||
cmd = [sys.executable] + cmd[1:]
|
||||
return real_run(cmd, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", run)
|
||||
return plugin_dir
|
||||
|
||||
|
||||
@pytest.mark.parametrize("params", [
|
||||
{"category_name": "jokes", "enabled": True}, # the file manager's toggle
|
||||
{"category_name": "jokes", "enabled": False},
|
||||
{"filename": None},
|
||||
{"nested": {"list": [1, None, True, 2.5], "empty": {}}},
|
||||
{"text": "café ✓ \U0001F600"},
|
||||
{"text": "he said \"hi\" and 'bye' \\ ''' \"\"\" \n\t end"},
|
||||
], ids=["true", "false", "null", "nested", "unicode", "quotes"])
|
||||
def test_the_script_receives_the_params_it_was_sent(api_v3_client, echo_plugin, params):
|
||||
response = api_v3_client.post(ACTION_URL, json={
|
||||
"plugin_id": "demo", "action_id": "toggle", "params": params})
|
||||
body = response.get_json()
|
||||
assert response.status_code == 200, body
|
||||
assert body["got"] == params
|
||||
|
||||
|
||||
def test_a_param_cannot_run_code_in_the_wrapper(api_v3_client, echo_plugin, tmp_path):
|
||||
marker = tmp_path / "PWNED"
|
||||
hostile = "\"}\nopen(%r, 'w').write('ran')\n#" % str(marker)
|
||||
params = {"name": hostile, "flag": True}
|
||||
response = api_v3_client.post(ACTION_URL, json={
|
||||
"plugin_id": "demo", "action_id": "toggle", "params": params})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert response.get_json()["got"] == params
|
||||
assert not marker.exists(), "a param value ran as code"
|
||||
|
||||
|
||||
def test_the_script_still_gets_ledmatrix_root(api_v3_client, echo_plugin, api_v3_module):
|
||||
response = api_v3_client.post(ACTION_URL, json={
|
||||
"plugin_id": "demo", "action_id": "toggle", "params": {"enabled": True}})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert response.get_json()["root"] == str(api_v3_module.PROJECT_ROOT)
|
||||
@@ -0,0 +1,89 @@
|
||||
"""A second install or uninstall while one is in progress is a 409, not a 500.
|
||||
|
||||
PluginOperationQueue refuses a second operation for a plugin that already
|
||||
has one waiting or running (test_operation_queue_pending_and_trim.py), and
|
||||
says so by raising ValueError. /plugins/install let that escape to the
|
||||
blueprint's catch-all, so a double-clicked Install answered 500 "An error
|
||||
occurred; see logs for details" while the first install carried on.
|
||||
/plugins/uninstall caught it in its own catch-all: a 500 "Failed to
|
||||
uninstall plugin", and an "uninstall failed" entry in the operation
|
||||
history for an uninstall that never started.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
from src.plugin_system.operation_queue import PluginOperationQueue # noqa: E402
|
||||
|
||||
INSTALL = "/api/v3/plugins/install"
|
||||
UNINSTALL = "/api/v3/plugins/uninstall"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def installing(api_v3_module, tmp_path):
|
||||
"""A real queue with an install of "clock" running and held there."""
|
||||
queue = PluginOperationQueue(max_history=10)
|
||||
api_v3_module.api_v3.operation_queue = queue
|
||||
started, release = threading.Event(), threading.Event()
|
||||
|
||||
def slow_install(plugin_id, branch=None):
|
||||
started.set()
|
||||
release.wait(10)
|
||||
return True
|
||||
|
||||
store = api_v3_module.api_v3.plugin_store_manager
|
||||
store.install_plugin.side_effect = slow_install
|
||||
store.get_registry_info.return_value = None
|
||||
store.plugins_dir = str(tmp_path)
|
||||
api_v3_module.api_v3.plugin_catalog.get_plugin_directory.return_value = None
|
||||
yield {"queue": queue, "started": started, "store": store}
|
||||
release.set()
|
||||
queue.shutdown()
|
||||
|
||||
|
||||
def _start_first_install(client, installing):
|
||||
response = client.post(INSTALL, json={"plugin_id": "clock"})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert installing["started"].wait(5)
|
||||
|
||||
|
||||
def _failed_history(api_v3_module):
|
||||
return [c for c in api_v3_module.api_v3.operation_history.record_operation.call_args_list
|
||||
if c.kwargs.get("status") == "failed"]
|
||||
|
||||
|
||||
def test_a_second_install_click_is_a_conflict(api_v3_client, api_v3_module, installing):
|
||||
_start_first_install(api_v3_client, installing)
|
||||
response = api_v3_client.post(INSTALL, json={"plugin_id": "clock"})
|
||||
assert response.status_code == 409, response.get_json()
|
||||
body = response.get_json()
|
||||
assert body["status"] == "error"
|
||||
assert body["error_code"] == "PLUGIN_OPERATION_CONFLICT"
|
||||
assert "clock" in body["message"]
|
||||
assert installing["store"].install_plugin.call_count == 1
|
||||
assert _failed_history(api_v3_module) == []
|
||||
|
||||
|
||||
def test_an_uninstall_during_the_install_is_a_conflict(api_v3_client, api_v3_module,
|
||||
installing):
|
||||
_start_first_install(api_v3_client, installing)
|
||||
response = api_v3_client.post(UNINSTALL, json={"plugin_id": "clock"})
|
||||
assert response.status_code == 409, response.get_json()
|
||||
assert response.get_json()["error_code"] == "PLUGIN_OPERATION_CONFLICT"
|
||||
assert _failed_history(api_v3_module) == [], (
|
||||
"an uninstall that never started was recorded as failed")
|
||||
api_v3_module.api_v3.plugin_store_manager.uninstall_plugin.assert_not_called()
|
||||
|
||||
|
||||
def test_another_plugin_is_still_queued(api_v3_client, installing):
|
||||
_start_first_install(api_v3_client, installing)
|
||||
response = api_v3_client.post(INSTALL, json={"plugin_id": "weather"})
|
||||
assert response.status_code == 200, response.get_json()
|
||||
assert response.get_json()["data"]["operation_id"]
|
||||
@@ -0,0 +1,73 @@
|
||||
"""GET /api/v3/plugins/<plugin_id>/static/<path> serves binary files too.
|
||||
|
||||
The route opened every file as UTF-8 text, so an image -- what the API
|
||||
reference says it is for, plugin previews and icons -- failed to decode and
|
||||
answered 500 "UnicodeDecodeError". Files are now sent as bytes. The text
|
||||
types the route always set are unchanged, and the path checks are pinned in
|
||||
test_path_traversal_guards.py::TestServePluginStatic.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from test._api_v3_test_helpers import api_v3_client, api_v3_module # noqa: F401,E402
|
||||
|
||||
PNG = (b"\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\x01\x00\x00\x00\x01"
|
||||
b"\x08\x06\x00\x00\x00\x1f\x15\xc4\x89")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def plugin_dir(tmp_path, api_v3_module):
|
||||
d = tmp_path / "demo"
|
||||
(d / "web_ui").mkdir(parents=True)
|
||||
(d / "manifest.json").write_text(json.dumps({"id": "demo"}), encoding="utf-8")
|
||||
api_v3_module.api_v3.plugin_catalog.get_plugin_directory.side_effect = (
|
||||
lambda pid: str(d) if pid == "demo" else None)
|
||||
return d
|
||||
|
||||
|
||||
def _get(client, path):
|
||||
return client.get(f"/api/v3/plugins/demo/static/{path}")
|
||||
|
||||
|
||||
def test_an_image_is_served_as_its_bytes(api_v3_client, plugin_dir):
|
||||
(plugin_dir / "web_ui" / "icon.png").write_bytes(PNG)
|
||||
response = _get(api_v3_client, "web_ui/icon.png")
|
||||
assert response.status_code == 200, response.get_json(silent=True)
|
||||
assert response.mimetype == "image/png"
|
||||
assert response.data == PNG
|
||||
|
||||
|
||||
def test_an_unknown_binary_file_is_served_too(api_v3_client, plugin_dir):
|
||||
blob = bytes(range(256))
|
||||
(plugin_dir / "data.bin").write_bytes(blob)
|
||||
response = _get(api_v3_client, "data.bin")
|
||||
assert response.status_code == 200, response.get_json(silent=True)
|
||||
assert response.data == blob
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name,mimetype", [
|
||||
("page.html", "text/html"),
|
||||
("app.js", "application/javascript"),
|
||||
("style.css", "text/css"),
|
||||
("data.json", "application/json"),
|
||||
("notes.txt", "text/plain"),
|
||||
("README.md", "text/plain"),
|
||||
("helper.py", "text/plain"),
|
||||
])
|
||||
def test_text_files_keep_their_types(api_v3_client, plugin_dir, name, mimetype):
|
||||
content = "caf\u00e9 \u2713 <p>hi</p>\n"
|
||||
(plugin_dir / name).write_bytes(content.encode("utf-8"))
|
||||
response = _get(api_v3_client, name)
|
||||
assert response.status_code == 200
|
||||
assert response.mimetype == mimetype
|
||||
assert response.data == content.encode("utf-8")
|
||||
|
||||
|
||||
def test_a_missing_file_is_still_a_404(api_v3_client, plugin_dir):
|
||||
assert _get(api_v3_client, "nope.png").status_code == 404
|
||||
@@ -13,6 +13,7 @@ The invariants that keep this change safe:
|
||||
inline path exactly.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
@@ -209,6 +210,32 @@ class TestFailurePaths:
|
||||
assert pm.get_plugin_lock(plugin_id).acquire(blocking=False) is True
|
||||
pm.get_plugin_lock(plugin_id).release()
|
||||
|
||||
@pytest.mark.parametrize("raised", [asyncio.CancelledError, SystemExit])
|
||||
def test_update_raising_a_base_exception_still_releases_the_plugin(self, pm, raised):
|
||||
"""asyncio.CancelledError and SystemExit derive from BaseException,
|
||||
not Exception. Raised from update() on the worker, one skipped the
|
||||
bookkeeping entirely: the plugin kept its lock and stayed RUNNING for
|
||||
the life of the process -- never updated again, and every display()
|
||||
skipped as busy."""
|
||||
class CancellingPlugin(SlowPlugin):
|
||||
def update(self):
|
||||
self.update_calls += 1
|
||||
raise raised()
|
||||
|
||||
plugin_id = _install(pm, CancellingPlugin())
|
||||
pm.run_scheduled_updates()
|
||||
deadline = time.monotonic() + 3
|
||||
while pm.plugins[plugin_id].update_calls == 0 and time.monotonic() < deadline:
|
||||
time.sleep(0.05)
|
||||
time.sleep(0.2)
|
||||
|
||||
assert pm.get_plugin_lock(plugin_id).acquire(blocking=False) is True
|
||||
pm.get_plugin_lock(plugin_id).release()
|
||||
assert pm.state_manager.can_execute(plugin_id) is True
|
||||
assert pm.plugin_last_update.get(plugin_id, 0) > 0
|
||||
error = pm.state_manager.get_error_info(plugin_id)
|
||||
assert error is not None and error["error_type"] == raised.__name__
|
||||
|
||||
def test_unloaded_while_queued_is_harmless(self, pm):
|
||||
"""Exercise the public unload_plugin() lifecycle rather than
|
||||
deleting pm.plugins directly: queue the target's update behind a
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
"""A cache key too long to be a filename still gets a cache file.
|
||||
|
||||
The calendar plugin's key joins every calendar id the user picked; on a real
|
||||
install it passed 300 bytes, and since ext4 caps a filename at 255 every write
|
||||
failed with ENAMETOOLONG -- logged as "permission denied", every update.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
from unittest.mock import patch
|
||||
|
||||
from src.cache.disk_cache import DiskCache, _MAX_KEY_FILENAME_BYTES, _filename_stem
|
||||
from src.cache_manager import CacheManager
|
||||
|
||||
# The shape of the key that failed on hdpi, ids anonymised.
|
||||
CALENDAR_KEY = (
|
||||
"calendar_events_someone@example.com_en.usa#holiday@group.v.calendar.google.com_"
|
||||
"family13997378751670666433@group.calendar.google.com_ncaaf_-m-07kbp5_"
|
||||
"%47eorgia+%42ulldogs+football#sports@group.v.calendar.google.com_nfl_-m-07l24_"
|
||||
"%54ampa+%42ay+%42uccaneers#sports@group.v.calendar.google.com_primary"
|
||||
)
|
||||
|
||||
# ext4/xfs/btrfs NAME_MAX; set()'s temp file adds 15 bytes to the stem.
|
||||
NAME_MAX = 255
|
||||
TEMP_OVERHEAD = len(".") + len(".json") + len(".") + 8
|
||||
|
||||
|
||||
def test_the_real_key_was_too_long_to_write():
|
||||
assert len((CALENDAR_KEY + ".json").encode()) > NAME_MAX - 10
|
||||
|
||||
|
||||
def test_a_long_key_round_trips(tmp_path):
|
||||
cache = DiskCache(str(tmp_path))
|
||||
cache.set(CALENDAR_KEY, {"events": [1, 2, 3]})
|
||||
|
||||
assert cache.get(CALENDAR_KEY, max_age=None) == {"events": [1, 2, 3]}
|
||||
path = cache.get_cache_path(CALENDAR_KEY)
|
||||
assert os.path.isfile(path)
|
||||
stem = os.path.basename(path)[:-len(".json")]
|
||||
assert len(stem.encode()) + TEMP_OVERHEAD <= NAME_MAX
|
||||
|
||||
|
||||
def test_short_keys_keep_their_filename(tmp_path):
|
||||
cache = DiskCache(str(tmp_path))
|
||||
exactly = "k" * _MAX_KEY_FILENAME_BYTES
|
||||
assert cache.get_cache_path("weather_current") == str(tmp_path / "weather_current.json")
|
||||
assert cache.get_cache_path(exactly) == str(tmp_path / f"{exactly}.json")
|
||||
assert cache.get_cache_path(exactly + "k") != str(tmp_path / f"{exactly}k.json")
|
||||
|
||||
|
||||
def test_long_keys_sharing_a_prefix_stay_apart(tmp_path):
|
||||
cache = DiskCache(str(tmp_path))
|
||||
first, second = CALENDAR_KEY + "_a", CALENDAR_KEY + "_b"
|
||||
cache.set(first, {"which": "a"})
|
||||
cache.set(second, {"which": "b"})
|
||||
|
||||
assert cache.get_cache_path(first) != cache.get_cache_path(second)
|
||||
assert cache.get(first, max_age=None) == {"which": "a"}
|
||||
assert cache.get(second, max_age=None) == {"which": "b"}
|
||||
|
||||
|
||||
def test_the_prefix_never_splits_a_character():
|
||||
key = "news_" + "é" * 300 # two bytes each, so the cut lands mid-character
|
||||
stem = _filename_stem(key)
|
||||
|
||||
assert stem.startswith("news_é")
|
||||
assert len(stem.encode("utf-8")) <= _MAX_KEY_FILENAME_BYTES
|
||||
stem.encode("utf-8").decode("utf-8") # well-formed
|
||||
|
||||
|
||||
def test_a_stem_listed_by_the_web_ui_deletes_the_same_file(tmp_path):
|
||||
with patch('src.cache_manager.CacheManager._get_writable_cache_dir', return_value=str(tmp_path)):
|
||||
manager = CacheManager()
|
||||
try:
|
||||
manager.save_cache(CALENDAR_KEY, {"events": []})
|
||||
|
||||
listed = [entry["key"] for entry in manager.list_cache_files()]
|
||||
assert len(listed) == 1
|
||||
manager.clear_cache(listed[0])
|
||||
|
||||
assert [n for n in os.listdir(tmp_path) if n.endswith(".json")] == []
|
||||
finally:
|
||||
manager.stop_cleanup_thread()
|
||||
|
||||
|
||||
def test_a_failed_write_names_the_real_error(tmp_path, monkeypatch, caplog):
|
||||
blocker = tmp_path / "a-file"
|
||||
blocker.write_text("")
|
||||
# No writable fallback either, so set() gives up and says why.
|
||||
monkeypatch.setattr(os.path, "expanduser", lambda _p: str(blocker / "home"))
|
||||
cache = DiskCache(str(tmp_path / "missing"))
|
||||
|
||||
with caplog.at_level(logging.WARNING):
|
||||
cache.set("weather_current", {"t": 1})
|
||||
|
||||
gave_up = [r.getMessage() for r in caplog.records if "Could not write cache" in r.getMessage()]
|
||||
assert len(gave_up) == 1
|
||||
assert "permission denied" not in gave_up[0]
|
||||
assert os.strerror(2) in gave_up[0] # ENOENT: the directory does not exist
|
||||
@@ -0,0 +1,95 @@
|
||||
"""The memory tier never serves a record older than the reader asked for.
|
||||
|
||||
A record read from disk went into the memory tier timed from the read, not
|
||||
from when it was written, so get(max_age=300) could hand out data up to twice
|
||||
that old: after a restart, after the memory sweep, or in a second process that
|
||||
loaded a record once and kept serving it.
|
||||
"""
|
||||
|
||||
import time
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.cache_manager import CacheManager
|
||||
|
||||
|
||||
class Clock:
|
||||
def __init__(self, now):
|
||||
self.now = now
|
||||
|
||||
def __call__(self):
|
||||
return self.now
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clock(monkeypatch):
|
||||
fake = Clock(1_800_000_000.0)
|
||||
monkeypatch.setattr(time, "time", fake)
|
||||
return fake
|
||||
|
||||
|
||||
def _manager(path):
|
||||
# No disk sweep: it judges files by their real mtime against the fake
|
||||
# clock and would delete them as months old.
|
||||
with patch('src.cache_manager.CacheManager._get_writable_cache_dir',
|
||||
return_value=str(path)), \
|
||||
patch('src.cache_manager.CacheManager.start_cleanup_thread'):
|
||||
return CacheManager()
|
||||
|
||||
|
||||
def test_a_record_loaded_late_expires_on_its_own_timestamp(tmp_path, clock):
|
||||
writer = _manager(tmp_path)
|
||||
writer.set("weather_current", {"t": 1})
|
||||
|
||||
reader = _manager(tmp_path) # a restart, or the other process
|
||||
clock.now += 250
|
||||
assert reader.get("weather_current", max_age=300) == {"t": 1}
|
||||
|
||||
clock.now += 100 # the data is 350 s old; it sat in memory for 100 s
|
||||
assert reader.get("weather_current", max_age=300) is None
|
||||
|
||||
|
||||
def test_a_stored_ttl_bounds_the_memory_copy_too(tmp_path, clock):
|
||||
writer = _manager(tmp_path)
|
||||
writer.set("odds_espn_football_nfl_401", {"spread": 6.5}, ttl=60)
|
||||
|
||||
reader = _manager(tmp_path)
|
||||
clock.now += 55
|
||||
assert reader.get("odds_espn_football_nfl_401", max_age=3600) == {"spread": 6.5}
|
||||
|
||||
clock.now += 60
|
||||
assert reader.get("odds_espn_football_nfl_401", max_age=3600) is None
|
||||
|
||||
|
||||
def test_a_stale_memory_copy_gives_way_to_a_newer_write_on_disk(tmp_path, clock):
|
||||
writer = _manager(tmp_path)
|
||||
reader = _manager(tmp_path)
|
||||
writer.set("stocks_AAPL", {"price": 1})
|
||||
assert reader.get("stocks_AAPL", max_age=300) == {"price": 1}
|
||||
|
||||
clock.now += 280
|
||||
writer.set("stocks_AAPL", {"price": 2})
|
||||
clock.now += 40 # reader's copy: 40 s in memory, 320 s old
|
||||
|
||||
assert reader.get("stocks_AAPL", max_age=300) == {"price": 2}
|
||||
|
||||
|
||||
def test_fresh_records_are_still_served_from_memory(tmp_path, clock):
|
||||
manager = _manager(tmp_path)
|
||||
manager.set("news_NFL", {"items": []})
|
||||
clock.now += 100
|
||||
|
||||
with patch.object(manager._disk_cache_component, "get") as disk_get:
|
||||
assert manager.get("news_NFL", max_age=300) == {"items": []}
|
||||
disk_get.assert_not_called()
|
||||
|
||||
|
||||
def test_max_age_none_and_records_without_a_timestamp_never_expire(tmp_path, clock):
|
||||
manager = _manager(tmp_path)
|
||||
manager.set("plugin_health_x", {"ok": True})
|
||||
manager.save_cache("raw_record", {"no": "timestamp"})
|
||||
clock.now += 10 ** 6
|
||||
|
||||
assert manager.get("plugin_health_x", max_age=None) == {"ok": True}
|
||||
assert manager.get_cached_data("raw_record", max_age=None) == {"no": "timestamp"}
|
||||
@@ -58,7 +58,8 @@ class TestFastPath:
|
||||
for _ in range(10):
|
||||
again = m.load_config()
|
||||
assert counts["n"] == 0, "fast path must not re-open any config file"
|
||||
assert again is first # same aliasing semantics as the full path
|
||||
assert again == first
|
||||
assert again is not first # each caller gets its own copy, see below
|
||||
|
||||
def test_config_change_triggers_reload(self, mgr):
|
||||
m, config, secrets, template = mgr
|
||||
@@ -98,6 +99,33 @@ class TestFastPath:
|
||||
assert m.load_config()["timezone"] == "America/New_York"
|
||||
|
||||
|
||||
class TestCallersGetACopy:
|
||||
"""A web handler edits what load_config returned, then validates. When
|
||||
validation failed, the edit stayed in the cache the fast path serves, and
|
||||
the next unrelated save wrote it -- a nested secret included, in plain
|
||||
text, because it had never reached config_secrets.json to be stripped."""
|
||||
|
||||
def test_editing_a_loaded_config_does_not_change_the_next_load(self, mgr):
|
||||
m, config, secrets, template = mgr
|
||||
loaded = m.load_config()
|
||||
loaded["display"]["brightness"] = 1
|
||||
loaded["weather"]["api_key"] = "typed-but-never-saved"
|
||||
again = m.load_config()
|
||||
assert again["display"]["brightness"] == 90
|
||||
assert again["weather"]["api_key"] == "sek"
|
||||
|
||||
def test_the_full_path_also_returns_a_copy(self, mgr):
|
||||
m, config, secrets, template = mgr
|
||||
m.load_config()["display"]["brightness"] = 1 # first load: full path
|
||||
assert m.load_config()["display"]["brightness"] == 90
|
||||
|
||||
def test_an_edit_never_reaches_a_later_save(self, mgr):
|
||||
m, config, secrets, template = mgr
|
||||
m.load_config()["display"]["new_secret"] = "hunter2" # then bailed out
|
||||
m.save_config(m.load_config()) # some other handler saves
|
||||
assert "hunter2" not in config.read_text()
|
||||
|
||||
|
||||
class TestSaveCoherence:
|
||||
def test_save_config_then_load_returns_saved_data(self, mgr, monkeypatch):
|
||||
m, config, secrets, template = mgr
|
||||
@@ -111,6 +139,15 @@ class TestSaveCoherence:
|
||||
assert loaded["weather"]["api_key"] == "sek" # secrets survive in memory
|
||||
assert counts["n"] == 0 # signature refreshed by save; no re-read
|
||||
|
||||
def test_the_saved_dict_does_not_become_the_cache(self, mgr):
|
||||
m, config, secrets, template = mgr
|
||||
m.load_config()
|
||||
new = {"display": {"brightness": 42}, "timezone": "UTC",
|
||||
"weather": {"api_key": "sek"}}
|
||||
m.save_config(new)
|
||||
new["display"]["brightness"] = 7 # the caller keeps using its dict
|
||||
assert m.load_config()["display"]["brightness"] == 42
|
||||
|
||||
def test_cross_process_save_is_picked_up(self, mgr):
|
||||
"""Another process writing config.json (different mtime) must bust
|
||||
this process's fast path — the core cross-process guarantee."""
|
||||
|
||||
@@ -143,7 +143,10 @@ class TestLoadFastPath:
|
||||
manager = make_manager(tmp_path, config={"timezone": "UTC"})
|
||||
first = manager.load_config()
|
||||
second = manager.load_config()
|
||||
assert second is first # same aliased dict, no re-read
|
||||
# A copy of the cached dict, never the dict itself; that it is not
|
||||
# re-read is test_config_load_cache's test_unchanged_files_are_not_reread
|
||||
assert second == first
|
||||
assert second is not first
|
||||
|
||||
def test_touching_secrets_file_invalidates_cache(self, tmp_path):
|
||||
manager = make_manager(
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
"""ConfigService notifies subscribers outside its lock, in order.
|
||||
|
||||
Subscribers ran while _load_config held the service's lock. The display's
|
||||
per-plugin subscriber is PluginManager.apply_config_change, which waits up to
|
||||
PLUGIN_LOCK_TIMEOUT (5 s) for a busy plugin. The same save that toggles a
|
||||
plugin's ``enabled`` flags a reconcile, and the render thread runs it: its
|
||||
get_config() -- and the unsubscribe() of a plugin it disables -- waited behind
|
||||
every slow callback, freezing the panel for up to 5 s per busy plugin.
|
||||
|
||||
What callers could rely on before still holds: one reload's notifications
|
||||
finish before the next reload's start, and a callback unsubscribe() removed is
|
||||
not running, and will not run, once unsubscribe() returns.
|
||||
"""
|
||||
|
||||
import itertools
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from src.config_manager import ConfigManager
|
||||
from src.config_service import ConfigService
|
||||
|
||||
SLOW = 2.0 # how long a blocked callback waits before giving up
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def service(tmp_path):
|
||||
config_path = tmp_path / "config.json"
|
||||
config_path.write_text(json.dumps({"display": {"brightness": 50},
|
||||
"weather": {"enabled": True}}),
|
||||
encoding="utf-8")
|
||||
manager = ConfigManager(str(config_path), str(tmp_path / "config_secrets.json"))
|
||||
manager.template_path = str(tmp_path / "no-template.json")
|
||||
svc = ConfigService(manager, enable_hot_reload=False)
|
||||
yield svc, config_path
|
||||
svc.shutdown()
|
||||
|
||||
|
||||
_saves = itertools.count(1)
|
||||
|
||||
|
||||
def _save(config_path, **sections):
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
config.update(sections)
|
||||
config_path.write_text(json.dumps(config), encoding="utf-8")
|
||||
# ConfigManager re-reads only when (mtime, size) moves. Two quick saves of
|
||||
# the same size can share an mtime tick (about 16 ms on Windows), so step
|
||||
# it forward explicitly.
|
||||
st = config_path.stat()
|
||||
os.utime(config_path, ns=(st.st_atime_ns, st.st_mtime_ns + next(_saves) * 50_000_000))
|
||||
|
||||
|
||||
def _reload_in_background(svc):
|
||||
thread = threading.Thread(target=svc._load_config, daemon=True)
|
||||
thread.start()
|
||||
return thread
|
||||
|
||||
|
||||
def test_get_config_does_not_wait_for_a_slow_subscriber(service):
|
||||
svc, config_path = service
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
|
||||
def slow(_old, _new):
|
||||
entered.set()
|
||||
release.wait(SLOW)
|
||||
|
||||
svc.subscribe(slow, plugin_id="weather")
|
||||
_save(config_path, weather={"enabled": False})
|
||||
reload = _reload_in_background(svc)
|
||||
assert entered.wait(SLOW)
|
||||
|
||||
start = time.monotonic()
|
||||
config = svc.get_config()
|
||||
waited = time.monotonic() - start
|
||||
release.set()
|
||||
reload.join(SLOW)
|
||||
|
||||
assert waited < 0.5
|
||||
# Swapped before anyone was told: a subscriber that reads it sees the new one.
|
||||
assert config["weather"]["enabled"] is False
|
||||
|
||||
|
||||
def test_unsubscribing_another_callback_does_not_wait(service):
|
||||
svc, config_path = service
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
|
||||
def slow(_old, _new):
|
||||
entered.set()
|
||||
release.wait(SLOW)
|
||||
|
||||
def other(_old, _new):
|
||||
pass
|
||||
|
||||
svc.subscribe(slow, plugin_id="weather")
|
||||
svc.subscribe(other, plugin_id="clock")
|
||||
_save(config_path, weather={"enabled": False})
|
||||
reload = _reload_in_background(svc)
|
||||
assert entered.wait(SLOW)
|
||||
|
||||
start = time.monotonic()
|
||||
svc.unsubscribe(other, plugin_id="clock")
|
||||
waited = time.monotonic() - start
|
||||
release.set()
|
||||
reload.join(SLOW)
|
||||
|
||||
assert waited < 0.5
|
||||
|
||||
|
||||
def test_a_callback_unsubscribed_mid_notification_is_not_called(service):
|
||||
svc, config_path = service
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
called = []
|
||||
|
||||
def slow_global(_old, _new): # global subscribers are notified first
|
||||
entered.set()
|
||||
release.wait(SLOW)
|
||||
|
||||
def weather(_old, _new):
|
||||
called.append("weather")
|
||||
|
||||
svc.subscribe(slow_global)
|
||||
svc.subscribe(weather, plugin_id="weather")
|
||||
_save(config_path, weather={"enabled": False})
|
||||
reload = _reload_in_background(svc)
|
||||
assert entered.wait(SLOW)
|
||||
|
||||
svc.unsubscribe(weather, plugin_id="weather")
|
||||
release.set()
|
||||
reload.join(SLOW)
|
||||
|
||||
assert called == []
|
||||
|
||||
|
||||
def test_unsubscribe_waits_for_its_own_callback_to_return(service):
|
||||
svc, config_path = service
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
returned = threading.Event()
|
||||
|
||||
def slow(_old, _new):
|
||||
entered.set()
|
||||
release.wait(SLOW)
|
||||
returned.set()
|
||||
|
||||
svc.subscribe(slow, plugin_id="weather")
|
||||
_save(config_path, weather={"enabled": False})
|
||||
reload = _reload_in_background(svc)
|
||||
assert entered.wait(SLOW)
|
||||
|
||||
threading.Timer(0.2, release.set).start()
|
||||
svc.unsubscribe(slow, plugin_id="weather")
|
||||
|
||||
assert returned.is_set()
|
||||
reload.join(SLOW)
|
||||
|
||||
|
||||
def test_a_callback_may_read_config_and_unsubscribe_itself(service):
|
||||
svc, config_path = service
|
||||
seen = []
|
||||
|
||||
def once(_old, _new):
|
||||
seen.append(svc.get_config()["weather"]["enabled"])
|
||||
svc.unsubscribe(once, plugin_id="weather")
|
||||
|
||||
svc.subscribe(once, plugin_id="weather")
|
||||
_save(config_path, weather={"enabled": False})
|
||||
reload = _reload_in_background(svc)
|
||||
reload.join(SLOW)
|
||||
|
||||
assert not reload.is_alive()
|
||||
assert seen == [False]
|
||||
|
||||
|
||||
def test_two_reloads_notify_in_order(service):
|
||||
svc, config_path = service
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
seen = []
|
||||
|
||||
def record(old, new):
|
||||
seen.append((old["brightness"], new["brightness"]))
|
||||
if len(seen) == 1:
|
||||
entered.set()
|
||||
release.wait(SLOW)
|
||||
|
||||
svc.subscribe(record, plugin_id="display")
|
||||
_save(config_path, display={"brightness": 60})
|
||||
first = _reload_in_background(svc)
|
||||
assert entered.wait(SLOW)
|
||||
|
||||
_save(config_path, display={"brightness": 100})
|
||||
second = _reload_in_background(svc)
|
||||
time.sleep(0.2)
|
||||
release.set()
|
||||
first.join(SLOW)
|
||||
second.join(SLOW)
|
||||
|
||||
assert seen == [(50, 60), (60, 100)]
|
||||
@@ -0,0 +1,112 @@
|
||||
"""A plugin duration that is not a number must not stop the display.
|
||||
|
||||
Several plugins return their ``display_duration`` setting as it is in
|
||||
config.json (``return self.config.get('display_duration', 15.0)``), so a
|
||||
value saved as ``"20"`` or ``null`` -- from the raw config editor, or by
|
||||
hand -- reached run() as a string or None. _resolve_durations then compared
|
||||
it with 0, the TypeError went past every handler in the loop, and the
|
||||
display service exited; systemd restarted it into the same screen and the
|
||||
same crash.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
os.environ.setdefault("EMULATOR", "true")
|
||||
|
||||
import pytest
|
||||
|
||||
from src.display_controller import DisplayController
|
||||
from test._run_loop_harness import FakePlugin, RunLoopHarness
|
||||
|
||||
|
||||
def _controller(plugin_modes):
|
||||
dc = object.__new__(DisplayController)
|
||||
dc.config = {}
|
||||
dc.plugin_modes = plugin_modes
|
||||
return dc
|
||||
|
||||
|
||||
def _plugin(duration, plugin_id='clock-simple'):
|
||||
plugin = MagicMock()
|
||||
plugin.plugin_id = plugin_id
|
||||
plugin.get_display_duration.return_value = duration
|
||||
return plugin
|
||||
|
||||
|
||||
class TestPluginDurationIsCoerced:
|
||||
@pytest.mark.parametrize('value, expected', [
|
||||
('20', 20.0), (' 7.5 ', 7.5), (12, 12.0), (12.5, 12.5)])
|
||||
def test_numbers_and_numeric_strings_are_used(self, value, expected):
|
||||
dc = _controller({'clock': _plugin(value)})
|
||||
duration = dc._get_display_duration('clock')
|
||||
assert duration == expected and isinstance(duration, float)
|
||||
|
||||
@pytest.mark.parametrize('value', [
|
||||
None, '', 'twenty', True, False, float('nan'), float('inf'), 'inf',
|
||||
[20], {'seconds': 20}])
|
||||
def test_anything_but_a_finite_number_gets_the_default(self, value):
|
||||
dc = _controller({'clock': _plugin(value)})
|
||||
assert dc._get_display_duration('clock') == 30
|
||||
|
||||
@pytest.mark.parametrize('value', [0, -5, '-5', '0'])
|
||||
def test_a_number_not_above_zero_still_gets_the_15s_rule(self, value):
|
||||
"""Unchanged: _resolve_durations turns it into 15 s, with its warning."""
|
||||
plugin = _plugin(value)
|
||||
dc = _controller({'clock': plugin})
|
||||
base = dc._get_display_duration('clock')
|
||||
assert dc._resolve_durations(plugin, 'clock', base, False)[1] == 15.0
|
||||
|
||||
def test_a_raising_get_display_duration_gets_the_default(self):
|
||||
plugin = _plugin(None)
|
||||
plugin.get_display_duration.side_effect = KeyError('display_duration')
|
||||
assert _controller({'clock': plugin})._get_display_duration('clock') == 30
|
||||
|
||||
def test_the_result_feeds_resolve_durations(self):
|
||||
"""The two calls run() makes back to back, for one screen."""
|
||||
plugin = _plugin('bad')
|
||||
dc = _controller({'clock': plugin})
|
||||
base = dc._get_display_duration('clock')
|
||||
assert dc._resolve_durations(plugin, 'clock', base, False) == (30, 30)
|
||||
|
||||
def test_logged_once_per_plugin(self, caplog):
|
||||
dc = _controller({'clock': _plugin('twenty'),
|
||||
'clock_big': _plugin('twenty'),
|
||||
'calendar': _plugin(None, plugin_id='calendar')})
|
||||
# clock_big is a second mode of the same plugin.
|
||||
dc.plugin_modes['clock_big'].plugin_id = 'clock-simple'
|
||||
with caplog.at_level(logging.WARNING, logger='src.display_controller'):
|
||||
for _ in range(3):
|
||||
for mode in ('clock', 'clock_big', 'calendar'):
|
||||
dc._get_display_duration(mode)
|
||||
warnings = [r for r in caplog.records if 'display duration' in r.getMessage()]
|
||||
assert len(warnings) == 2
|
||||
assert {'clock-simple', 'calendar'} == {
|
||||
next(p for p in ('clock-simple', 'calendar') if p in r.getMessage())
|
||||
for r in warnings}
|
||||
|
||||
def test_a_good_value_after_a_bad_one_is_used(self):
|
||||
plugin = _plugin(None)
|
||||
dc = _controller({'clock': plugin})
|
||||
assert dc._get_display_duration('clock') == 30
|
||||
plugin.get_display_duration.return_value = 45
|
||||
assert dc._get_display_duration('clock') == 45.0
|
||||
|
||||
|
||||
class TestRunLoopSurvives:
|
||||
"""Through the real run() on the harness's fake clock."""
|
||||
|
||||
@pytest.mark.parametrize('duration, shown_for', [('20', 20.0), (None, 30.0),
|
||||
('twenty', 30.0)])
|
||||
def test_the_screen_runs_and_the_rotation_goes_on(self, tmp_path, duration, shown_for):
|
||||
harness = RunLoopHarness(tmp_path, horizon=120)
|
||||
harness.add_plugin(FakePlugin("weather", ["weather"], duration=30))
|
||||
harness.add_plugin(FakePlugin("clock-simple", ["clock"], duration=duration))
|
||||
# Before the fix run() returned at t=30, when the clock came up, and
|
||||
# the harness raised "run() returned ... before the horizon".
|
||||
rows = harness.run()["screens"]
|
||||
clock = next(row for row in rows if row[1] == "clock")
|
||||
assert math.isclose(clock[2], shown_for, abs_tol=1.0)
|
||||
assert [row[1] for row in rows][:3] == ["weather", "clock", "weather"]
|
||||
@@ -346,6 +346,52 @@ class TestSubscriptionStore:
|
||||
assert client.snapshot_loop_age(snap, now_mono=104.0) is None
|
||||
|
||||
|
||||
class TestReconnectBackoff:
|
||||
"""StateSubscription._run's waits between connections, without a socket."""
|
||||
|
||||
def test_a_connection_that_got_a_snapshot_starts_the_backoff_over(self, hub,
|
||||
monkeypatch):
|
||||
"""Three failed tries, then the display is back twice, restarting
|
||||
each time, then gone again. Each restart is retried after the
|
||||
shortest wait, not after whatever the waits had grown to."""
|
||||
sub = client.StateSubscription(paths=['/nowhere'])
|
||||
script = ['refused', 'refused', 'refused', 'snapshot', 'snapshot', 'refused']
|
||||
waits = []
|
||||
|
||||
def follow():
|
||||
step = script.pop(0)
|
||||
if step == 'snapshot': # subscribed, then the display restarted
|
||||
sub._store(hub.snapshot(), full=True)
|
||||
raise client.ControlError('closed', 'the display closed the connection')
|
||||
raise client.ControlError(step)
|
||||
|
||||
def wait(seconds):
|
||||
waits.append(seconds)
|
||||
return not script # True ends _run, as stop() would
|
||||
|
||||
monkeypatch.setattr(sub, '_follow', follow)
|
||||
monkeypatch.setattr(sub._stop, 'wait', wait)
|
||||
sub._run()
|
||||
first = client._RECONNECT_MIN_SECONDS
|
||||
assert waits == [first, 2 * first, 4 * first, first, first, 2 * first]
|
||||
|
||||
def test_a_display_without_the_stream_is_still_retried_slowly(self, monkeypatch):
|
||||
sub = client.StateSubscription(paths=['/nowhere'])
|
||||
waits = []
|
||||
|
||||
def follow():
|
||||
raise client.ControlError('unknown_command')
|
||||
|
||||
def wait(seconds):
|
||||
waits.append(seconds)
|
||||
return len(waits) == 2
|
||||
|
||||
monkeypatch.setattr(sub, '_follow', follow)
|
||||
monkeypatch.setattr(sub._stop, 'wait', wait)
|
||||
sub._run()
|
||||
assert waits == [client._RECONNECT_MAX_SECONDS] * 2
|
||||
|
||||
|
||||
# --- a real socket ------------------------------------------------------------------
|
||||
|
||||
def _wait_until(predicate, timeout=5.0):
|
||||
|
||||
@@ -136,3 +136,22 @@ class TestCleartextCredentialsNeedAnExplicitOptIn:
|
||||
"allow_insecure_mqtt": "false"})
|
||||
assert r.status_code == 400
|
||||
|
||||
def test_the_settings_read_reports_the_opt_in(self, client, monkeypatch):
|
||||
"""The Tools form prefills its "Allow without TLS" box from the GET.
|
||||
|
||||
Off until someone saves it on, so an untouched form sends false and
|
||||
the guard above still refuses a cleartext password.
|
||||
"""
|
||||
c, _ = client
|
||||
monkeypatch.setattr(misc, "_mqtt_bridge_service_state",
|
||||
lambda: {"installed": False, "active": False, "enabled": False})
|
||||
|
||||
def read():
|
||||
return c.get("/api/v3/integrations/mqtt-bridge").get_json()["data"]["config"]
|
||||
|
||||
assert read()["allow_insecure_mqtt"] is False
|
||||
r = c.put(URL, json={"mqtt_password": "hunter2", "mqtt_tls": False,
|
||||
"allow_insecure_mqtt": True})
|
||||
assert r.status_code == 200, r.get_json()
|
||||
assert read()["allow_insecure_mqtt"] is True
|
||||
|
||||
|
||||
@@ -58,3 +58,83 @@ def test_a_reloaded_plugin_still_gets_its_own_bare_module(plugins):
|
||||
assert reloaded.WHO == "alpha"
|
||||
assert sys.path.index(str(plugins["alpha"])) < sys.path.index(str(plugins["beta"]))
|
||||
assert sys.path.count(str(plugins["alpha"])) == 1
|
||||
|
||||
|
||||
# -- sub-packages ------------------------------------------------------------
|
||||
#
|
||||
# A plugin that keeps helpers in a package (``providers/feed.py``, imported as
|
||||
# ``from providers.feed import ...``) leaves dotted entries in sys.modules.
|
||||
# Only the bare ``providers`` used to be tracked, so ``providers.feed`` outlived
|
||||
# the plugin: a reload after a store update re-ran the new manager.py against
|
||||
# the old feed.py, until the display restarted. Elections (providers/),
|
||||
# flights (enrichment/) and olympics (data/, renderers/) ship packages.
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def package_plugin(tmp_path):
|
||||
before_path = list(sys.path)
|
||||
before_modules = set(sys.modules)
|
||||
plugin_dir = tmp_path / "pkgdemo"
|
||||
(plugin_dir / "providers").mkdir(parents=True)
|
||||
(plugin_dir / "providers" / "__init__.py").write_text("", encoding="utf-8")
|
||||
(plugin_dir / "providers" / "feed.py").write_text("VERSION = 'v1'\n", encoding="utf-8")
|
||||
(plugin_dir / "manager.py").write_text(
|
||||
"from providers.feed import VERSION\n", encoding="utf-8")
|
||||
yield plugin_dir
|
||||
sys.path[:] = before_path
|
||||
for key in set(sys.modules) - before_modules:
|
||||
sys.modules.pop(key, None)
|
||||
|
||||
|
||||
def test_a_reloaded_plugin_runs_its_updated_subpackage_module(package_plugin):
|
||||
loader = PluginLoader()
|
||||
assert loader.load_module("pkgdemo", package_plugin, "manager.py").VERSION == "v1"
|
||||
|
||||
_unload(loader, "pkgdemo")
|
||||
# The store update: a different size, so no cached bytecode can match.
|
||||
(package_plugin / "providers" / "feed.py").write_text(
|
||||
"VERSION = 'v2 from the update'\n", encoding="utf-8")
|
||||
reloaded = loader.load_module("pkgdemo", package_plugin, "manager.py")
|
||||
|
||||
assert reloaded.VERSION == "v2 from the update"
|
||||
|
||||
|
||||
def test_unload_drops_the_plugins_subpackage_modules(package_plugin):
|
||||
loader = PluginLoader()
|
||||
loader.load_module("pkgdemo", package_plugin, "manager.py")
|
||||
# Still importable while the plugin runs, as before.
|
||||
assert "providers.feed" in sys.modules
|
||||
|
||||
_unload(loader, "pkgdemo")
|
||||
|
||||
assert not [k for k in sys.modules if k.startswith("providers")]
|
||||
|
||||
|
||||
def test_a_failed_load_leaves_no_subpackage_module_behind(package_plugin):
|
||||
(package_plugin / "manager.py").write_text(
|
||||
"from providers.feed import VERSION\nraise RuntimeError('broken')\n",
|
||||
encoding="utf-8")
|
||||
loader = PluginLoader()
|
||||
|
||||
with pytest.raises(RuntimeError):
|
||||
loader.load_module("pkgdemo", package_plugin, "manager.py")
|
||||
|
||||
assert not [k for k in sys.modules if k.startswith("providers")]
|
||||
|
||||
|
||||
def test_unload_leaves_packages_from_outside_the_plugin_alone(package_plugin, tmp_path):
|
||||
# A library the plugin imports is not the plugin's to drop.
|
||||
lib_root = tmp_path / "site"
|
||||
(lib_root / "extlib").mkdir(parents=True)
|
||||
(lib_root / "extlib" / "__init__.py").write_text("", encoding="utf-8")
|
||||
(lib_root / "extlib" / "sub.py").write_text("X = 1\n", encoding="utf-8")
|
||||
sys.path.append(str(lib_root))
|
||||
(package_plugin / "manager.py").write_text(
|
||||
"import extlib.sub\nfrom providers.feed import VERSION\n", encoding="utf-8")
|
||||
loader = PluginLoader()
|
||||
loader.load_module("pkgdemo", package_plugin, "manager.py")
|
||||
|
||||
_unload(loader, "pkgdemo")
|
||||
|
||||
assert "extlib.sub" in sys.modules
|
||||
assert "extlib" in sys.modules
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
"""A dev plugin linked in under a name its checkout does not share still loads.
|
||||
|
||||
``scripts/dev/dev_plugin_setup.sh`` links a checkout into the plugins
|
||||
directory under the plugin's id: ``link-github foo <url>`` clones
|
||||
``ledmatrix-foo`` (the repository naming convention) and links it as
|
||||
``plugins/foo``. ``contained_plugin_dir`` resolved the link and looked for the
|
||||
*target's* folder name, ``ledmatrix-foo``, among the plugins directory's
|
||||
entries. There is none, so ``install_dependencies`` refused the plugin as
|
||||
outside the plugins directory and the load failed with "Dependency
|
||||
installation failed" -- even with no requirements.txt at all.
|
||||
|
||||
The containment it exists for still holds: the answer is always rebuilt from
|
||||
an entry enumerated under the plugins directory.
|
||||
|
||||
Skipped where this process cannot create a symlink (Windows without the
|
||||
privilege).
|
||||
"""
|
||||
|
||||
import os
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.plugin_system.plugin_loader import PluginLoader, contained_plugin_dir
|
||||
|
||||
|
||||
def _symlink_or_skip(target, link):
|
||||
try:
|
||||
os.symlink(target, link, target_is_directory=True)
|
||||
except (OSError, NotImplementedError) as e:
|
||||
pytest.skip(f"cannot create a symlink here: {e}")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def linked(tmp_path):
|
||||
checkout = tmp_path / "dev-plugins" / "ledmatrix-foo"
|
||||
checkout.mkdir(parents=True)
|
||||
plugins_dir = tmp_path / "plugins"
|
||||
plugins_dir.mkdir()
|
||||
link = plugins_dir / "foo"
|
||||
_symlink_or_skip(checkout, link)
|
||||
return plugins_dir, link, checkout
|
||||
|
||||
|
||||
def test_a_link_resolves_to_its_own_entry_in_the_plugins_dir(linked):
|
||||
plugins_dir, link, _checkout = linked
|
||||
|
||||
assert contained_plugin_dir(link, plugins_dir) == os.path.join(
|
||||
os.path.realpath(plugins_dir), "foo")
|
||||
|
||||
|
||||
def test_a_linked_plugin_without_requirements_needs_no_install(linked):
|
||||
plugins_dir, link, _checkout = linked
|
||||
|
||||
with patch("subprocess.run") as pip:
|
||||
assert PluginLoader().install_dependencies(link, "foo", plugins_dir=plugins_dir) is True
|
||||
pip.assert_not_called()
|
||||
|
||||
|
||||
@patch("src.plugin_system.plugin_loader.requirements_are_satisfied", return_value=False)
|
||||
def test_a_linked_plugins_requirements_are_installed_through_the_link(_satisfied, linked):
|
||||
plugins_dir, link, checkout = linked
|
||||
(checkout / "requirements.txt").write_text("package1==1.0.0\n", encoding="utf-8")
|
||||
|
||||
with patch("subprocess.run", return_value=MagicMock(returncode=0, stderr="")) as pip:
|
||||
assert PluginLoader().install_dependencies(link, "foo", plugins_dir=plugins_dir) is True
|
||||
|
||||
argv = pip.call_args[0][0]
|
||||
assert argv[argv.index("-r") + 1] == os.path.join(
|
||||
os.path.realpath(plugins_dir), "foo", "requirements.txt")
|
||||
|
||||
|
||||
def test_a_link_outside_the_plugins_dir_is_still_refused(linked, tmp_path):
|
||||
plugins_dir, _link, checkout = linked
|
||||
elsewhere = tmp_path / "elsewhere"
|
||||
elsewhere.mkdir()
|
||||
stray = elsewhere / "bar"
|
||||
_symlink_or_skip(checkout, stray)
|
||||
|
||||
assert contained_plugin_dir(stray, plugins_dir) is None
|
||||
assert contained_plugin_dir(plugins_dir / ".." / "elsewhere" / "bar", plugins_dir) is None
|
||||
@@ -189,6 +189,24 @@ class TestPluginExecutor:
|
||||
|
||||
assert result is False
|
||||
|
||||
def test_a_base_exception_is_a_failure_not_a_timeout(self):
|
||||
"""asyncio.CancelledError derives from BaseException. Uncaught on
|
||||
the executor's thread it ended the thread with the call never marked
|
||||
complete, so a call that failed at once was reported, and recorded,
|
||||
as timing out."""
|
||||
import asyncio
|
||||
import pytest
|
||||
from src.exceptions import PluginError
|
||||
from src.plugin_system.plugin_executor import PluginExecutor
|
||||
executor = PluginExecutor(default_timeout=5.0)
|
||||
|
||||
def cancelled():
|
||||
raise asyncio.CancelledError()
|
||||
|
||||
with pytest.raises(PluginError) as raised:
|
||||
executor.execute_with_timeout(cancelled, plugin_id="test_plugin")
|
||||
assert isinstance(raised.value.__cause__, asyncio.CancelledError)
|
||||
|
||||
|
||||
class TestPluginHealth:
|
||||
"""Test plugin health monitoring."""
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
"""
|
||||
ScrollHelper frames for a strip narrower than the panel, and other wraps.
|
||||
|
||||
A frame that runs past the end of the strip continues from its head: column
|
||||
j of the frame is strip column (position + j) modulo the strip's width. The
|
||||
wrap path sliced the strip's tail and then "the rest of the frame" from its
|
||||
head, which assumed the head was at least that wide. For a strip narrower
|
||||
than the panel it raised ValueError at every position, so a narrow strip
|
||||
(Vegas composes one when its content is narrower than the chain, with its
|
||||
lead-in of 0) logged a traceback every frame instead of drawing.
|
||||
"""
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from src.common.scroll_helper import ScrollHelper
|
||||
|
||||
W, H = 128, 32
|
||||
|
||||
|
||||
def _strip(width, height=H):
|
||||
"""A strip whose every column is distinct: R and B are the column number."""
|
||||
columns = np.arange(width)
|
||||
pixels = np.zeros((height, width, 3), dtype=np.uint8)
|
||||
pixels[:, :, 0] = columns % 256
|
||||
pixels[:, :, 1] = 255 - (columns % 256)
|
||||
pixels[:, :, 2] = columns // 256
|
||||
return Image.fromarray(pixels, 'RGB')
|
||||
|
||||
|
||||
def _helper(strip_width, sub_pixel=False):
|
||||
sh = ScrollHelper(W, H)
|
||||
sh.set_scrolling_image(_strip(strip_width))
|
||||
sh.sub_pixel_scrolling = sub_pixel
|
||||
return sh
|
||||
|
||||
|
||||
def _frame(sh, position):
|
||||
sh.scroll_position = position
|
||||
frame = sh.get_visible_portion()
|
||||
assert frame is not None and frame.size == (W, H) and frame.mode == 'RGB'
|
||||
return np.asarray(frame)
|
||||
|
||||
|
||||
def _wrapped(sh, start):
|
||||
"""What the panel should show from ``start``: the strip, wrapping."""
|
||||
return sh.cached_array[:, np.arange(start, start + W) % sh.cached_array.shape[1]]
|
||||
|
||||
|
||||
class TestNarrowStrip:
|
||||
@pytest.mark.parametrize('strip_width', [1, 40, 50, W - 1])
|
||||
@pytest.mark.parametrize('position', [0, 10, 39])
|
||||
def test_frame_repeats_the_strip_across_the_panel(self, strip_width, position):
|
||||
sh = _helper(strip_width)
|
||||
position %= strip_width
|
||||
assert np.array_equal(_frame(sh, position), _wrapped(sh, position))
|
||||
|
||||
def test_a_composed_strip_without_lead_in(self):
|
||||
# How Vegas builds its strip: lead_gap=0 (vegas_scroll.lead_in_width).
|
||||
sh = ScrollHelper(W, H)
|
||||
sh.create_scrolling_image([_strip(40)], item_gap=0, element_gap=0, lead_gap=0)
|
||||
assert sh.total_scroll_width == 40
|
||||
for position in range(40):
|
||||
assert np.array_equal(_frame(sh, position), _wrapped(sh, position))
|
||||
|
||||
def test_a_whole_pass_scrolls_without_raising(self):
|
||||
sh = _helper(50)
|
||||
sh.set_pixels_per_frame(3)
|
||||
for _ in range(60):
|
||||
sh.update_scroll_position()
|
||||
assert sh.get_visible_portion().size == (W, H)
|
||||
|
||||
@pytest.mark.parametrize('position', [0.5, 10.25, 49.5])
|
||||
def test_sub_pixel_blend_of_a_narrow_strip(self, position):
|
||||
sh = _helper(50, sub_pixel=True)
|
||||
frame = _frame(sh, position)
|
||||
start = int(position)
|
||||
near, far = _wrapped(sh, start), _wrapped(sh, start + 1)
|
||||
lo, hi = np.minimum(near, far), np.maximum(near, far)
|
||||
assert (frame >= lo).all() and (frame <= hi).all()
|
||||
|
||||
|
||||
class TestWrapOfAWideStrip:
|
||||
"""Unchanged: the tail, then the head."""
|
||||
|
||||
@pytest.mark.parametrize('position', [200 - W + 1, 150, 199])
|
||||
def test_tail_then_head(self, position):
|
||||
sh = _helper(200)
|
||||
frame = _frame(sh, position)
|
||||
tail = 200 - position
|
||||
assert np.array_equal(frame[:, :tail], sh.cached_array[:, position:])
|
||||
assert np.array_equal(frame[:, tail:], sh.cached_array[:, :W - tail])
|
||||
|
||||
def test_at_the_end_shows_the_head(self):
|
||||
sh = _helper(200)
|
||||
assert np.array_equal(_frame(sh, 200), sh.cached_array[:, :W])
|
||||
|
||||
def test_sub_pixel_at_the_last_column(self):
|
||||
sh = _helper(200, sub_pixel=True)
|
||||
assert _frame(sh, 199.5).shape == (H, W, 3)
|
||||
|
||||
def test_a_position_before_the_start_wraps_too(self):
|
||||
# Slicing [-10:118] of the array was an empty slice: frombytes raised.
|
||||
sh = _helper(200)
|
||||
assert np.array_equal(_frame(sh, -10), _wrapped(sh, -10))
|
||||
|
||||
|
||||
def test_a_zero_width_strip_is_a_black_frame():
|
||||
sh = ScrollHelper(W, H)
|
||||
sh.set_scrolling_image(Image.new('RGB', (0, H)))
|
||||
assert not _frame(sh, 0).any()
|
||||
@@ -1,162 +0,0 @@
|
||||
"""A frame the preview throttle skipped still reaches the snapshot.
|
||||
|
||||
The preview snapshot (/api/v3/display/current, the web UI's live preview) is
|
||||
only written from update_display(), at most once per write interval. A screen
|
||||
that draws its card once and then holds it -- soccer's recent/upcoming cards
|
||||
skip redundant redraws -- pushes exactly one frame. When that push lands inside
|
||||
the interval, e.g. a few milliseconds after the on-demand start's clear wrote a
|
||||
black frame, the throttle skips it and nothing ever writes it: on ledpi the
|
||||
preview stayed black for soccer's whole 15 s screen while the panel showed the
|
||||
card, and the next screen "rendered immediately".
|
||||
|
||||
Runs the real DisplayManager on the emulator, like test_display_dirty_tracking.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
|
||||
os.environ["EMULATOR"] = "true"
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def dm(tmp_path_factory):
|
||||
from src.display_manager import DisplayManager
|
||||
DisplayManager._instance = None
|
||||
manager = DisplayManager({
|
||||
"display": {
|
||||
"hardware": {"rows": 32, "cols": 64, "chain_length": 2,
|
||||
"parallel": 1, "brightness": 90},
|
||||
"runtime": {"gpio_slowdown": 0},
|
||||
},
|
||||
}, suppress_test_pattern=True)
|
||||
manager._snapshot_path = str(
|
||||
tmp_path_factory.mktemp("owed_snapshot") / "led_matrix_preview.png")
|
||||
yield manager
|
||||
DisplayManager._instance = None
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def viewer(dm, monkeypatch, tmp_path):
|
||||
"""A preview is open (1 s write interval); fresh snapshot bookkeeping."""
|
||||
monkeypatch.setattr(dm, "_viewer_is_fresh", lambda now: True)
|
||||
dm._viewer_was_fresh = True
|
||||
dm._snapshot_path = str(tmp_path / "snap.png")
|
||||
dm._last_snapshot_ts = 0.0
|
||||
dm._last_snapshot_touch_ts = 0.0
|
||||
dm._last_snapshot_digest = None
|
||||
dm._saved_snapshot_digest = None
|
||||
dm._snapshot_owed = False
|
||||
dm.set_scrolling_state(False)
|
||||
return dm
|
||||
|
||||
|
||||
def _lit(path):
|
||||
with Image.open(path) as img:
|
||||
return sum(1 for p in img.convert("RGB").getdata() if max(p) > 20)
|
||||
|
||||
|
||||
def _age_last_write(dm, seconds=2.0):
|
||||
"""As if `seconds` had passed since the last snapshot write."""
|
||||
dm._last_snapshot_ts -= seconds
|
||||
dm._last_snapshot_touch_ts -= seconds
|
||||
|
||||
|
||||
def _clear_then_draw_card(dm):
|
||||
"""The on-demand start's clear, then the card a few ms later."""
|
||||
dm.clear()
|
||||
dm.update_display() # black frame: written
|
||||
assert _lit(dm._snapshot_path) == 0
|
||||
dm.draw.rectangle([4, 4, 40, 20], fill=(255, 255, 0))
|
||||
dm.update_display() # the card: inside the interval
|
||||
|
||||
|
||||
def _controller(dm):
|
||||
from src import display_controller as dc_module
|
||||
controller = dc_module.DisplayController.__new__(dc_module.DisplayController)
|
||||
controller.plugin_manager = None
|
||||
controller.display_manager = dm
|
||||
return controller
|
||||
|
||||
|
||||
class _HoldingPlugin:
|
||||
"""Already showing its card: display() returns True and draws nothing."""
|
||||
|
||||
plugin_id = "holding"
|
||||
|
||||
def __init__(self):
|
||||
self.calls = 0
|
||||
|
||||
def display(self, display_mode=None, force_clear=False):
|
||||
self.calls += 1
|
||||
return True
|
||||
|
||||
|
||||
def test_a_held_card_reaches_the_preview_on_the_next_frame(viewer):
|
||||
dm = viewer
|
||||
_clear_then_draw_card(dm)
|
||||
assert _lit(dm._snapshot_path) == 0 # the throttle skipped the card
|
||||
|
||||
controller = _controller(dm)
|
||||
plugin = _HoldingPlugin()
|
||||
_age_last_write(dm)
|
||||
# The render loop's next frame: the plugin draws nothing and makes no
|
||||
# update_display() call, as soccer's switch cards do.
|
||||
assert controller._display_once(plugin, "soccer_eng.1_recent", True) is True
|
||||
assert plugin.calls == 1
|
||||
assert _lit(dm._snapshot_path) > 0
|
||||
|
||||
|
||||
def test_the_owed_write_still_waits_out_the_interval(viewer, monkeypatch):
|
||||
dm = viewer
|
||||
_clear_then_draw_card(dm)
|
||||
saves = []
|
||||
monkeypatch.setattr(dm, "_save_snapshot", lambda image: saves.append(image))
|
||||
dm.write_owed_snapshot() # still inside the interval
|
||||
assert saves == []
|
||||
_age_last_write(dm)
|
||||
dm.write_owed_snapshot()
|
||||
assert len(saves) == 1
|
||||
# Written: nothing is owed, so later frames do no work and the unchanged
|
||||
# frame is not encoded again.
|
||||
assert dm._snapshot_owed is False
|
||||
_age_last_write(dm)
|
||||
dm.write_owed_snapshot()
|
||||
assert len(saves) == 1
|
||||
|
||||
|
||||
def test_nothing_owed_after_a_frame_that_was_written(viewer, monkeypatch):
|
||||
dm = viewer
|
||||
dm.draw.rectangle([0, 0, 8, 8], fill=(0, 255, 0))
|
||||
dm.update_display() # due: written at once
|
||||
assert dm._snapshot_owed is False
|
||||
calls = []
|
||||
monkeypatch.setattr(dm, "_write_snapshot_if_due",
|
||||
lambda *a, **k: calls.append(a))
|
||||
dm.write_owed_snapshot()
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_an_unchanged_frame_inside_the_interval_is_not_owed(viewer):
|
||||
dm = viewer
|
||||
dm.draw.rectangle([0, 0, 8, 8], fill=(0, 0, 255))
|
||||
dm.update_display()
|
||||
dm.update_display() # same frame, inside the interval
|
||||
assert dm._snapshot_owed is False
|
||||
|
||||
|
||||
def test_a_controller_without_the_hook_still_draws():
|
||||
"""Controllers built without a display manager (tests) are unaffected."""
|
||||
from src import display_controller as dc_module
|
||||
controller = dc_module.DisplayController.__new__(dc_module.DisplayController)
|
||||
controller.plugin_manager = None
|
||||
plugin = _HoldingPlugin()
|
||||
assert controller._display_once(plugin, "x", True) is True
|
||||
controller.display_manager = types.SimpleNamespace()
|
||||
assert controller._display_once(plugin, "x", True) is True
|
||||
assert plugin.calls == 2
|
||||
@@ -13,6 +13,8 @@ body. That is what let all eight adopt this with byte-identical renders.
|
||||
import logging
|
||||
import json
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import pytest
|
||||
|
||||
@@ -169,6 +171,95 @@ class TestDateAndTime:
|
||||
assert C.card_tzinfo({"timezone": "Not/AZone"}, log) is timezone.utc
|
||||
|
||||
|
||||
class TestWeekdayMatchesThePrintedDate:
|
||||
"""The weekday is the printed date's, whichever zone printed it.
|
||||
|
||||
The extractor prints "M/D" in the plugin's resolved zone (its own
|
||||
setting, else the global one, else the system zone). The card is handed
|
||||
only the plugin's config, whose ``timezone`` ships as "" -- so a weekday
|
||||
taken in card_tzinfo's zone was UTC's, and an evening kickoff in the
|
||||
Americas read "Sat Oct 2" for a Friday game.
|
||||
"""
|
||||
|
||||
WEEKDAY = {"timezone": "", "scroll_card": {"date_format": "weekday"}}
|
||||
|
||||
@staticmethod
|
||||
def _as_printed(start_utc, zone):
|
||||
"""The game dict and the date text, as the extractor builds them."""
|
||||
local = datetime.fromisoformat(start_utc).astimezone(ZoneInfo(zone))
|
||||
game = {"start_time_utc": datetime.fromisoformat(start_utc),
|
||||
"game_date": f"{local.month}/{local.day}"}
|
||||
want = f"{C.WEEKDAY_ABBR[local.weekday()]} {C.MONTH_ABBR[local.month - 1]} {local.day}"
|
||||
return game, want
|
||||
|
||||
@pytest.mark.parametrize("start_utc, zone, want", [
|
||||
# Friday 8 PM EDT is Saturday in UTC.
|
||||
("2026-10-03T00:00:00+00:00", "America/New_York", "Fri Oct 2"),
|
||||
# The night US clocks go back: 8:30 PM EDT Saturday, then 11 PM EST
|
||||
# Sunday, each the next day in UTC.
|
||||
("2026-11-01T00:30:00+00:00", "America/New_York", "Sat Oct 31"),
|
||||
("2026-11-02T04:00:00+00:00", "America/New_York", "Sun Nov 1"),
|
||||
# New Year's Eve on the west coast is New Year's Day in UTC.
|
||||
("2027-01-01T04:00:00+00:00", "America/Los_Angeles", "Thu Dec 31"),
|
||||
# Just east of the date line: Pago Pago's Friday evening.
|
||||
("2026-10-03T05:00:00+00:00", "Pacific/Pago_Pago", "Fri Oct 2"),
|
||||
# Just west of it, the other way: Saturday morning in Auckland is
|
||||
# Friday in UTC -- and the 10 AM game on the day NZ clocks go forward.
|
||||
("2026-10-02T20:00:00+00:00", "Pacific/Auckland", "Sat Oct 3"),
|
||||
("2026-09-26T21:00:00+00:00", "Pacific/Auckland", "Sun Sep 27"),
|
||||
# UTC+14, the furthest any zone sits from UTC.
|
||||
("2026-10-02T11:00:00+00:00", "Pacific/Kiritimati", "Sat Oct 3"),
|
||||
# A zone on UTC's own date needs nothing.
|
||||
("2026-10-02T19:00:00+00:00", "Europe/London", "Fri Oct 2"),
|
||||
])
|
||||
def test_the_shipped_blank_timezone(self, log, start_utc, zone, want):
|
||||
game, printed = self._as_printed(start_utc, zone)
|
||||
assert printed == want # the case says what the extractor prints
|
||||
assert C.format_game_date(self.WEEKDAY, log, game["game_date"], game) == want
|
||||
|
||||
def test_an_iso_string_start_reads_the_same(self, log):
|
||||
game = {"start_time_utc": "2026-10-03T00:00:00Z"}
|
||||
assert C.format_game_date(self.WEEKDAY, log, "10/2", game) == "Fri Oct 2"
|
||||
assert C.format_game_date(self.WEEKDAY, log, "10/02", game) == "Fri Oct 2"
|
||||
|
||||
def test_a_plugin_level_zone_still_agrees(self, log):
|
||||
game, want = self._as_printed("2026-10-03T00:00:00+00:00", "America/Chicago")
|
||||
cfg = dict(self.WEEKDAY, timezone="America/Chicago")
|
||||
assert C.format_game_date(cfg, log, game["game_date"], game) == want == "Fri Oct 2"
|
||||
|
||||
def test_a_date_no_zone_could_print_keeps_the_zone_weekday(self, log):
|
||||
"""More than a day from the start: nothing to anchor to, so the
|
||||
weekday is card_tzinfo's, as it always was."""
|
||||
game = {"start_time_utc": datetime(2026, 10, 3, 0, 0, tzinfo=timezone.utc)}
|
||||
assert C.format_game_date(self.WEEKDAY, log, "10/9", game) == "Sat Oct 9"
|
||||
|
||||
def test_a_start_without_an_offset_keeps_the_zone_weekday(self, log):
|
||||
"""A naive time names no instant, so it cannot place the date."""
|
||||
game = {"start_time_utc": datetime(2026, 10, 2, 20, 0)}
|
||||
assert C.format_game_date(self.WEEKDAY, log, "10/3", game) == \
|
||||
f"{C.weekday_for(self.WEEKDAY, log, game)} Oct 3"
|
||||
|
||||
@pytest.mark.parametrize("game", [None, {}, {"start_time_utc": "garbage"}])
|
||||
def test_no_usable_start_draws_no_weekday(self, log, game):
|
||||
assert C.format_game_date(self.WEEKDAY, log, "10/2", game) == "Oct 2"
|
||||
|
||||
def test_the_scorebug_twin_formats_the_same(self, log):
|
||||
"""Switch mode (SportsCoreSharedMixin) shares the formatter body."""
|
||||
from src.common.sports_shared import SportsCoreSharedMixin
|
||||
|
||||
class Host(SportsCoreSharedMixin):
|
||||
config = {"scroll_card": {"date_format": "weekday",
|
||||
"switch_date_format": "inherit"}}
|
||||
logger = log
|
||||
|
||||
def _get_timezone(self):
|
||||
return ZoneInfo("America/New_York")
|
||||
|
||||
game, want = self._as_printed("2026-11-01T00:30:00+00:00", "America/New_York")
|
||||
assert Host()._format_game_date(game["game_date"], game) == want
|
||||
assert C.format_game_date(Host.config, log, game["game_date"], game) == want
|
||||
|
||||
|
||||
class TestFontSizing:
|
||||
def test_snaps_to_the_faces_pixel_grid(self):
|
||||
assert C.crisp_size("4x6-font.ttf", 6) == 7 # 7px grid
|
||||
|
||||
@@ -561,21 +561,24 @@ class TestPinnedDivergence:
|
||||
assert C.recent_score_color(on, LOG, game, (9, 9, 9)) == (9, 9, 9)
|
||||
|
||||
def test_weekday_zone_source(self):
|
||||
# DIVERGENCE, user-visible: the scorebug asks the plugin's
|
||||
# DIVERGENCE, not drawn: the scorebug asks the plugin's
|
||||
# _get_timezone() (plugin setting -> global setting -> system zone);
|
||||
# the card reads only config["timezone"] and falls back to UTC. The
|
||||
# scoreboards' schemas default that key to "", and the scroll display
|
||||
# hands the renderer the plugin config, so a board that sets only the
|
||||
# global zone gets UTC weekdays in scroll mode: an evening kickoff in
|
||||
# New York is labelled with the next day.
|
||||
# hands the renderer the plugin config, so the bare weekday helpers
|
||||
# still disagree for an evening kickoff in New York.
|
||||
game = {"start_time_utc": "2026-09-20T00:30:00+00:00"} # Sat 20:30 EDT
|
||||
host = _Host({}, tz=ZoneInfo("America/New_York"))
|
||||
assert host._weekday_for(game) == "Sat"
|
||||
assert C.weekday_for({}, LOG, game) == "Sun"
|
||||
# DECIDED: what a card draws is the printed date's own weekday, so
|
||||
# the scroll card no longer labels that kickoff with the next day
|
||||
# ("Sun Sep 19" before). Both formatters place the extractor's "M/D"
|
||||
# against the start time instead of re-deriving the day in a zone.
|
||||
cfg = {"scroll_card": {"date_format": "weekday", "switch_date_format": "inherit"}}
|
||||
host = _Host(cfg, tz=ZoneInfo("America/New_York"))
|
||||
assert host._format_game_date("9/19", game) == "Sat Sep 19"
|
||||
assert C.format_game_date(cfg, LOG, "9/19", game) == "Sun Sep 19"
|
||||
assert C.format_game_date(cfg, LOG, "9/19", game) == "Sat Sep 19"
|
||||
|
||||
def test_weekday_out_of_range_start(self):
|
||||
# DIVERGENCE: the mixin catches OverflowError from astimezone() and
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Removing a dev plugin linked into the plugins directory removes the link.
|
||||
|
||||
``scripts/dev/dev_plugin_setup.sh`` symlinks a checkout into the plugins
|
||||
directory. ``PluginStoreManager._safe_remove_directory`` -- behind uninstall,
|
||||
and behind discarding the set-aside copy after an install or update -- handed
|
||||
the link to ``shutil.rmtree``, which refuses a symlink. Its fallback then
|
||||
walked through the link and chmodded every directory and file of the linked
|
||||
checkout to 0700, and the sudo stage refused a path outside the plugins
|
||||
directory. So the uninstall failed, the link stayed, and the developer's
|
||||
checkout lost its group/other permissions and gained execute bits.
|
||||
|
||||
Skipped where this process cannot create a symlink (Windows without the
|
||||
privilege).
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from src.plugin_system.store_manager import PluginStoreManager
|
||||
|
||||
PLUGIN_ID = "linked-demo"
|
||||
|
||||
|
||||
def _symlink_or_skip(target, link):
|
||||
try:
|
||||
os.symlink(target, link, target_is_directory=True)
|
||||
except (OSError, NotImplementedError) as e:
|
||||
pytest.skip(f"cannot create a symlink here: {e}")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def linked(tmp_path):
|
||||
checkout = tmp_path / "dev-plugins" / PLUGIN_ID
|
||||
checkout.mkdir(parents=True)
|
||||
(checkout / "manifest.json").write_text(
|
||||
json.dumps({"id": PLUGIN_ID, "name": "Linked", "class_name": "P",
|
||||
"display_modes": ["linked"], "version": "1.0.0"}),
|
||||
encoding="utf-8")
|
||||
(checkout / "manager.py").write_text("X = 1\n", encoding="utf-8")
|
||||
plugins_dir = tmp_path / "plugins"
|
||||
plugins_dir.mkdir()
|
||||
link = plugins_dir / PLUGIN_ID
|
||||
_symlink_or_skip(checkout, link)
|
||||
store = PluginStoreManager(plugins_dir=str(plugins_dir))
|
||||
store.logger = MagicMock()
|
||||
return store, link, checkout
|
||||
|
||||
|
||||
def test_removing_a_linked_plugin_removes_only_the_link(linked):
|
||||
store, link, checkout = linked
|
||||
|
||||
assert store._safe_remove_directory(link) is True
|
||||
|
||||
assert not os.path.lexists(link)
|
||||
assert (checkout / "manager.py").read_text(encoding="utf-8") == "X = 1\n"
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != "posix", reason="POSIX permission bits")
|
||||
def test_removing_a_linked_plugin_leaves_the_checkouts_permissions(linked):
|
||||
store, link, checkout = linked
|
||||
os.chmod(checkout, 0o755)
|
||||
os.chmod(checkout / "manager.py", 0o644)
|
||||
|
||||
store._safe_remove_directory(link)
|
||||
|
||||
assert checkout.stat().st_mode & 0o777 == 0o755
|
||||
assert (checkout / "manager.py").stat().st_mode & 0o777 == 0o644
|
||||
|
||||
|
||||
def test_uninstalling_a_linked_plugin_removes_the_link(linked):
|
||||
store, link, checkout = linked
|
||||
|
||||
assert store.uninstall_plugin(PLUGIN_ID) is True
|
||||
|
||||
assert not os.path.lexists(link)
|
||||
assert (checkout / "manifest.json").exists()
|
||||
|
||||
|
||||
def test_a_dangling_link_is_removed_too(linked):
|
||||
store, link, checkout = linked
|
||||
for child in checkout.iterdir():
|
||||
child.unlink()
|
||||
checkout.rmdir()
|
||||
|
||||
assert store._safe_remove_directory(link) is True
|
||||
|
||||
assert not os.path.lexists(link)
|
||||
@@ -50,11 +50,13 @@ class FakeResult:
|
||||
self.plugins_to_install = plugins_to_install or []
|
||||
self.plugins_installed = []
|
||||
self.plugins_failed = []
|
||||
self.skipped = []
|
||||
|
||||
def to_dict(self):
|
||||
return {
|
||||
"success": self.success,
|
||||
"restored": self.restored,
|
||||
"skipped": self.skipped,
|
||||
"errors": self.errors,
|
||||
"plugins_installed": self.plugins_installed,
|
||||
"plugins_failed": self.plugins_failed,
|
||||
@@ -286,6 +288,109 @@ class TestPluginReinstall:
|
||||
assert body["data"]["plugins_failed"][0]["error"] == "Store manager unavailable"
|
||||
|
||||
|
||||
class TestInstalledPluginsAreNotReinstalled:
|
||||
""""Reinstall missing plugins" installs only what is missing.
|
||||
|
||||
Every plugin the backup listed went to install_plugin, which replaces an
|
||||
installed copy with a fresh download: restoring onto the same device
|
||||
re-downloaded all of them inside the request. One installed from its own
|
||||
URL is not in the registry, so its "reinstall" returned False and the
|
||||
whole restore answered 500 "Restore failed" with the plugin still there.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _installed(tmp_path, *names):
|
||||
found = {}
|
||||
for name in names:
|
||||
(tmp_path / name).mkdir()
|
||||
found[name] = tmp_path / name
|
||||
return lambda plugin_id: found.get(plugin_id)
|
||||
|
||||
def test_an_installed_plugin_is_skipped_and_a_missing_one_installed(
|
||||
self, client, restore, tmp_path):
|
||||
restore.return_value = FakeResult(
|
||||
plugins_to_install=[{"plugin_id": "clock"}, {"plugin_id": "weather"}])
|
||||
store = api_v3.plugin_store_manager
|
||||
store._existing_install.side_effect = self._installed(tmp_path, "clock")
|
||||
store.install_plugin.return_value = True
|
||||
response = post(client)
|
||||
assert response.status_code == 200
|
||||
store.install_plugin.assert_called_once_with("weather")
|
||||
data = response.get_json()["data"]
|
||||
assert data["plugins_installed"] == ["weather"]
|
||||
assert data["plugins_failed"] == []
|
||||
assert "plugin:clock (installed)" in data["skipped"]
|
||||
|
||||
def test_an_installed_plugin_the_store_cannot_install_is_not_a_failure(
|
||||
self, client, restore, tmp_path):
|
||||
restore.return_value = FakeResult(plugins_to_install=[{"plugin_id": "my-3p"}])
|
||||
store = api_v3.plugin_store_manager
|
||||
store._existing_install.side_effect = self._installed(tmp_path, "my-3p")
|
||||
store.install_plugin.return_value = False
|
||||
response = post(client)
|
||||
assert response.status_code == 200
|
||||
assert response.get_json()["data"]["plugins_failed"] == []
|
||||
store.install_plugin.assert_not_called()
|
||||
|
||||
@pytest.fixture
|
||||
def real_store(self, tmp_path):
|
||||
from src.plugin_system.store_manager import PluginStoreManager
|
||||
plugins_dir = tmp_path / "plugin-repos"
|
||||
for folder, manifest_id in (("ledmatrix-weather", "ledmatrix-weather"),
|
||||
("my-3p", "my-3p")):
|
||||
(plugins_dir / folder).mkdir(parents=True)
|
||||
(plugins_dir / folder / "manifest.json").write_text(
|
||||
json.dumps({"id": manifest_id, "version": "1.0.0"}))
|
||||
store = PluginStoreManager(plugins_dir=str(plugins_dir),
|
||||
uninstalled_registry_path=str(tmp_path / "uninstalled.json"))
|
||||
# The official weather plugin's registry id differs from the id it
|
||||
# installs under; my-3p was installed from its own URL.
|
||||
registry = {"plugins": [{
|
||||
"id": "weather", "repo": "https://github.com/ChuckBuilds/ledmatrix-plugins",
|
||||
"plugin_path": "plugins/ledmatrix-weather"}]}
|
||||
store.registry_cache = registry
|
||||
store.fetch_registry = lambda *a, **k: registry
|
||||
store.install_plugin = MagicMock(return_value=True)
|
||||
api_v3.plugin_store_manager = store
|
||||
return store
|
||||
|
||||
def test_with_the_real_store_aliases_and_third_party_installs_count(
|
||||
self, client, restore, real_store):
|
||||
restore.return_value = FakeResult(plugins_to_install=[
|
||||
{"plugin_id": "weather"}, {"plugin_id": "my-3p"}, {"plugin_id": "clock"}])
|
||||
response = post(client)
|
||||
assert response.status_code == 200
|
||||
real_store.install_plugin.assert_called_once_with("clock")
|
||||
skipped = response.get_json()["data"]["skipped"]
|
||||
assert "plugin:weather (installed)" in skipped
|
||||
assert "plugin:my-3p (installed)" in skipped
|
||||
|
||||
|
||||
class TestFontsCatalogCache:
|
||||
"""The Fonts tab's catalog is cached for 5 minutes (fonts.py).
|
||||
|
||||
Upload and delete clear it; a restore did not, so restored fonts were
|
||||
missing from the Fonts tab and every font picker until it expired.
|
||||
"""
|
||||
|
||||
@pytest.fixture
|
||||
def cached_catalog(self):
|
||||
from web_interface.cache import delete_cached, get_cached, set_cached
|
||||
set_cached('fonts_catalog', {'fonts': ['5x7.bdf']}, ttl_seconds=300)
|
||||
yield lambda: get_cached('fonts_catalog', ttl_seconds=300)
|
||||
delete_cached('fonts_catalog')
|
||||
|
||||
def test_a_restore_that_restored_fonts_clears_it(self, client, restore, cached_catalog):
|
||||
restore.return_value = FakeResult(restored=["config", "fonts (2)"])
|
||||
assert post(client).status_code == 200
|
||||
assert cached_catalog() is None
|
||||
|
||||
def test_a_restore_without_fonts_keeps_it(self, client, restore, cached_catalog):
|
||||
restore.return_value = FakeResult(restored=["config"])
|
||||
assert post(client).status_code == 200
|
||||
assert cached_catalog() == {'fonts': ['5x7.bdf']}
|
||||
|
||||
|
||||
class TestFailureReporting:
|
||||
def test_restore_errors_produce_a_500(self, client, restore):
|
||||
restore.return_value = FakeResult(
|
||||
|
||||
@@ -13,7 +13,9 @@ tmp_path so the assertions are against files on disk rather than mock
|
||||
calls.
|
||||
"""
|
||||
|
||||
import html
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
@@ -221,3 +223,66 @@ class TestRawEndpointsBypassSecretSeparation:
|
||||
env.client.post(MAIN, json={"weather": {"api_key": "PLAINTEXT-KEY"}})
|
||||
# Nothing was moved aside into the secrets file.
|
||||
assert not env.secrets_file.exists() or "PLAINTEXT-KEY" not in env.secrets_file.read_text()
|
||||
|
||||
|
||||
class TestConfigEditorRoundTrip:
|
||||
"""The Config Editor tab (/partials/raw-json) and the save it posts to.
|
||||
|
||||
The secrets editor is shown masked, like GET /config/secrets: the page is
|
||||
served to anyone who can reach the port while the optional web login is
|
||||
off. Its save strips the masks and merges onto the stored file, so a
|
||||
masked editor saved back as it is changes nothing.
|
||||
"""
|
||||
|
||||
STORED = {
|
||||
"github": {"api_token": "ghp_REAL_TOKEN_1234"},
|
||||
"ledmatrix-weather": {"api_key": "WEATHER_KEY_abcdef", "units_id": 42},
|
||||
"calendar": {"accounts": [{"name": "home", "token": "CAL_TOKEN_9"}]},
|
||||
"youtube": {"api_key": "YOUR_YOUTUBE_API_KEY", "channel_secret": ""},
|
||||
}
|
||||
REAL_VALUES = ("ghp_REAL_TOKEN_1234", "WEATHER_KEY_abcdef", "CAL_TOKEN_9")
|
||||
|
||||
@pytest.fixture
|
||||
def editor(self, env, monkeypatch):
|
||||
from web_interface.blueprints import pages_v3 as pages_module
|
||||
env.secrets_file.write_text(json.dumps(self.STORED))
|
||||
monkeypatch.setattr(pages_module.pages_v3, "config_manager",
|
||||
env.config_manager, raising=False)
|
||||
app = Flask(__name__, template_folder=str(project_root / "web_interface" / "templates"))
|
||||
app.config["TESTING"] = True
|
||||
app.register_blueprint(pages_module.pages_v3)
|
||||
app.register_blueprint(api_v3, url_prefix="/api/v3")
|
||||
return app.test_client()
|
||||
|
||||
@staticmethod
|
||||
def _secrets_textarea(client):
|
||||
page = client.get("/partials/raw-json")
|
||||
assert page.status_code == 200
|
||||
match = re.search(r'<textarea id="secrets-config-editor"[^>]*>(.*?)</textarea>',
|
||||
page.get_data(as_text=True), re.S)
|
||||
assert match, "the secrets editor is missing from the partial"
|
||||
return html.unescape(match.group(1))
|
||||
|
||||
def test_the_editor_shows_no_secret_value(self, editor):
|
||||
text = self._secrets_textarea(editor)
|
||||
for value in self.REAL_VALUES:
|
||||
assert value not in text
|
||||
shown = json.loads(text)
|
||||
assert shown["github"]["api_token"] == "\u2022" * 8
|
||||
# Same shape as the file, and "not set" still reads as not set.
|
||||
assert shown["calendar"]["accounts"][0]["name"] == "\u2022" * 8
|
||||
assert shown["youtube"] == {"api_key": "YOUR_YOUTUBE_API_KEY", "channel_secret": ""}
|
||||
|
||||
def test_saving_it_back_unchanged_keeps_every_secret(self, editor, env):
|
||||
shown = json.loads(self._secrets_textarea(editor))
|
||||
response = editor.post(SECRETS, json=shown)
|
||||
assert response.status_code == 200
|
||||
assert json.loads(env.secrets_file.read_text()) == self.STORED
|
||||
|
||||
def test_editing_one_secret_changes_only_that_one(self, editor, env):
|
||||
shown = json.loads(self._secrets_textarea(editor))
|
||||
shown["ledmatrix-weather"]["api_key"] = "NEW_WEATHER_KEY"
|
||||
assert editor.post(SECRETS, json=shown).status_code == 200
|
||||
expected = json.loads(json.dumps(self.STORED))
|
||||
expected["ledmatrix-weather"]["api_key"] = "NEW_WEATHER_KEY"
|
||||
assert json.loads(env.secrets_file.read_text()) == expected
|
||||
|
||||
@@ -169,6 +169,10 @@ class TestGetSchemaProperty:
|
||||
},
|
||||
"fifa.world": {"type": "object",
|
||||
"properties": {"enabled": {"type": "boolean"}}},
|
||||
"cities": {"type": "array",
|
||||
"items": {"type": "object",
|
||||
"properties": {"timezone": {"type": "string"}}}},
|
||||
"color": {"type": ["array", "null"], "items": {"type": "integer"}},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -185,6 +189,15 @@ class TestGetSchemaProperty:
|
||||
prop = _get_schema_property(self.SCHEMA, "fifa.world.enabled")
|
||||
assert prop == {"type": "boolean"}
|
||||
|
||||
def test_an_index_steps_into_the_array_items(self):
|
||||
# How a table row posts its cells
|
||||
assert _get_schema_property(self.SCHEMA, "cities.0.timezone") == {"type": "string"}
|
||||
assert _get_schema_property(self.SCHEMA, "color.2") == {"type": "integer"}
|
||||
|
||||
def test_a_non_index_under_an_array_is_not_found(self):
|
||||
assert _get_schema_property(self.SCHEMA, "cities.timezone") is None
|
||||
assert _get_schema_property(self.SCHEMA, "cities.0.nope") is None
|
||||
|
||||
def test_missing_path_returns_none(self):
|
||||
assert _get_schema_property(self.SCHEMA, "nope.nope") is None
|
||||
|
||||
|
||||
@@ -0,0 +1,393 @@
|
||||
"""GET and POST /plugins/config against a real ConfigManager and SchemaManager.
|
||||
|
||||
Each class is one bug, reproduced through the endpoint the settings form and
|
||||
API clients use, with assertions on config.json and config_secrets.json.
|
||||
"""
|
||||
|
||||
import json
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from flask import Flask
|
||||
|
||||
from src.config_manager import ConfigManager
|
||||
from src.plugin_system.schema_manager import SchemaManager
|
||||
from web_interface.blueprints.api_v3 import api_v3
|
||||
|
||||
PLUGIN_ID = "demo"
|
||||
OTHER_ID = "other"
|
||||
|
||||
SCHEMA = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"api_key": {"type": "string", "x-secret": True, "default": ""},
|
||||
"city": {"type": "string", "default": "Austin"},
|
||||
"mqtt": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"host": {"type": "string", "default": ""},
|
||||
"port": {"type": "integer", "default": 1883,
|
||||
"minimum": 1, "maximum": 65535},
|
||||
"password": {"type": "string", "x-secret": True, "default": ""},
|
||||
},
|
||||
},
|
||||
"accounts": {
|
||||
"type": "array",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"token": {"type": "string", "x-secret": True},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
OTHER_SCHEMA = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"label": {"type": "string", "default": "x"},
|
||||
},
|
||||
}
|
||||
|
||||
STORED = {
|
||||
PLUGIN_ID: {"enabled": True, "city": "Paris",
|
||||
"mqtt": {"host": "broker", "port": 1883},
|
||||
"accounts": [{"name": "a"}, {"name": "b"}]},
|
||||
OTHER_ID: {"enabled": True, "label": "hello"},
|
||||
}
|
||||
|
||||
STORED_SECRETS = {
|
||||
PLUGIN_ID: {"api_key": "TOPSECRET",
|
||||
"accounts": [{"token": "TOK-A"}, {"token": "TOK-B"}]},
|
||||
}
|
||||
|
||||
_ATTRS = ('config_manager', 'plugin_catalog', 'plugin_store_manager',
|
||||
'saved_repositories_manager', 'schema_manager',
|
||||
'operation_queue', 'operation_history', 'cache_manager')
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def env(tmp_path):
|
||||
config_file = tmp_path / "config.json"
|
||||
secrets_file = tmp_path / "config_secrets.json"
|
||||
plugins_dir = tmp_path / "plugins"
|
||||
for plugin_id, schema in ((PLUGIN_ID, SCHEMA), (OTHER_ID, OTHER_SCHEMA)):
|
||||
plugin_dir = plugins_dir / plugin_id
|
||||
plugin_dir.mkdir(parents=True)
|
||||
(plugin_dir / "config_schema.json").write_text(json.dumps(schema))
|
||||
(plugin_dir / "manifest.json").write_text(json.dumps({"id": plugin_id}))
|
||||
config_file.write_text(json.dumps(STORED))
|
||||
secrets_file.write_text(json.dumps(STORED_SECRETS))
|
||||
|
||||
sentinel = object()
|
||||
originals = {name: getattr(api_v3, name, sentinel) for name in _ATTRS}
|
||||
|
||||
config_manager = ConfigManager(config_path=str(config_file),
|
||||
secrets_path=str(secrets_file))
|
||||
config_manager.template_path = str(tmp_path / "no-template.json")
|
||||
plugin_manager = MagicMock()
|
||||
plugin_manager.plugin_manifests = {PLUGIN_ID: {"id": PLUGIN_ID},
|
||||
OTHER_ID: {"id": OTHER_ID}}
|
||||
plugin_manager.plugins_dir = plugins_dir
|
||||
|
||||
for name in _ATTRS:
|
||||
setattr(api_v3, name, MagicMock())
|
||||
api_v3.config_manager = config_manager
|
||||
api_v3.schema_manager = SchemaManager(plugins_dir=plugins_dir, project_root=tmp_path)
|
||||
api_v3.plugin_catalog = plugin_manager
|
||||
api_v3.operation_queue = None
|
||||
|
||||
app = Flask(__name__)
|
||||
app.config["TESTING"] = True
|
||||
app.register_blueprint(api_v3, url_prefix="/api/v3")
|
||||
|
||||
class Env:
|
||||
client = app.test_client()
|
||||
|
||||
@staticmethod
|
||||
def use_schema(schema, plugin_id=PLUGIN_ID):
|
||||
(plugins_dir / plugin_id / "config_schema.json").write_text(json.dumps(schema))
|
||||
|
||||
@staticmethod
|
||||
def store(section, plugin_id=PLUGIN_ID):
|
||||
main = json.loads(config_file.read_text())
|
||||
main[plugin_id] = section
|
||||
config_file.write_text(json.dumps(main))
|
||||
|
||||
@staticmethod
|
||||
def main():
|
||||
return json.loads(config_file.read_text())
|
||||
|
||||
@staticmethod
|
||||
def secrets():
|
||||
return json.loads(secrets_file.read_text())
|
||||
|
||||
@staticmethod
|
||||
def post_form(data, plugin_id=PLUGIN_ID):
|
||||
return Env.client.post(f"/api/v3/plugins/config?plugin_id={plugin_id}",
|
||||
data=data)
|
||||
|
||||
@staticmethod
|
||||
def post_json(config, plugin_id=PLUGIN_ID):
|
||||
return Env.client.post("/api/v3/plugins/config",
|
||||
json={"plugin_id": plugin_id, "config": config})
|
||||
|
||||
yield Env
|
||||
|
||||
for name, original in originals.items():
|
||||
if original is sentinel:
|
||||
if hasattr(api_v3, name):
|
||||
delattr(api_v3, name)
|
||||
else:
|
||||
setattr(api_v3, name, original)
|
||||
|
||||
|
||||
class TestARejectedSaveLeavesNothingBehind:
|
||||
"""The form save edited the cached config load_config hands out, then
|
||||
failed validation. The cache kept the edit, and the next save of any
|
||||
other setting wrote it to config.json -- the rejected value, and a
|
||||
nested secret typed into the same form in plain text."""
|
||||
|
||||
REJECTED = {"mqtt.host": "broker", "mqtt.port": "99999",
|
||||
"mqtt.password": "hunter2", "__rendered_section": ["mqtt"]}
|
||||
|
||||
def test_the_rejected_values_never_reach_config_json(self, env):
|
||||
assert env.post_form(self.REJECTED).status_code == 400
|
||||
|
||||
resp = env.post_json({"label": "bye"}, plugin_id=OTHER_ID)
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
|
||||
main = env.main()
|
||||
assert main[OTHER_ID]["label"] == "bye"
|
||||
assert main[PLUGIN_ID]["mqtt"] == {"host": "broker", "port": 1883}
|
||||
assert "hunter2" not in json.dumps(main)
|
||||
|
||||
def test_the_form_reloads_with_the_stored_values(self, env):
|
||||
assert env.post_form(self.REJECTED).status_code == 400
|
||||
assert api_v3.config_manager.load_config()[PLUGIN_ID]["mqtt"]["port"] == 1883
|
||||
|
||||
|
||||
class TestGetMasksSecrets:
|
||||
"""GET /plugins/config returned the section with config_secrets.json
|
||||
merged in, secrets and all: the masking #276 added was lost when the
|
||||
route was rewritten. The settings page and GET /config/secrets mask."""
|
||||
|
||||
def test_secrets_come_back_blank(self, env):
|
||||
data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
|
||||
assert data["api_key"] == ""
|
||||
assert data["accounts"] == [{"name": "a", "token": ""}, {"name": "b", "token": ""}]
|
||||
assert data["city"] == "Paris"
|
||||
|
||||
def test_posting_the_response_back_keeps_every_secret(self, env):
|
||||
data = env.client.get(f"/api/v3/plugins/config?plugin_id={PLUGIN_ID}").get_json()["data"]
|
||||
resp = env.post_json(data)
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
|
||||
assert "TOPSECRET" not in json.dumps(env.main())
|
||||
|
||||
def test_the_settings_form_posting_masked_fields_keeps_every_secret(self, env):
|
||||
# The page renders secrets blank (pages_v3 masks the same way)
|
||||
resp = env.post_form({
|
||||
"api_key": "", "city": "Lyon", "mqtt.host": "broker", "mqtt.port": "1883",
|
||||
"mqtt.password": "", "__rendered_section": ["api_key", "city", "mqtt"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
|
||||
assert env.main()[PLUGIN_ID]["city"] == "Lyon"
|
||||
|
||||
def test_a_plugin_without_a_schema_has_credential_named_fields_blanked(self, env, tmp_path):
|
||||
(tmp_path / "plugins" / "bare").mkdir()
|
||||
env.store({"enabled": True, "station": "KAUS"}, plugin_id="bare")
|
||||
secrets = env.secrets()
|
||||
secrets["bare"] = {"api_token": "BARE-TOKEN"}
|
||||
(tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
|
||||
data = env.client.get("/api/v3/plugins/config?plugin_id=bare").get_json()["data"]
|
||||
assert data["api_token"] == ""
|
||||
assert data["station"] == "KAUS"
|
||||
|
||||
@pytest.mark.parametrize("section", ["web_auth", "github", "display"])
|
||||
def test_a_core_section_is_refused(self, env, tmp_path, section):
|
||||
secrets = env.secrets()
|
||||
secrets["web_auth"] = {"cookie_secret": "COOKIE-KEY", "password_hash": "HASH"}
|
||||
secrets["github"] = {"api_token": "ghp_TOKEN"}
|
||||
(tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
|
||||
env.store({"hardware": {"rows": 32}}, plugin_id="display")
|
||||
resp = env.client.get(f"/api/v3/plugins/config?plugin_id={section}")
|
||||
assert resp.status_code == 400
|
||||
body = resp.get_data(as_text=True)
|
||||
assert "COOKIE-KEY" not in body and "ghp_TOKEN" not in body
|
||||
|
||||
|
||||
ROWS_SCHEMA = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"cities": {
|
||||
"type": "array",
|
||||
"x-widget": "array-table",
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"timezone": {"type": "string"},
|
||||
"lat": {"type": "number"},
|
||||
"show": {"type": "boolean", "default": True},
|
||||
},
|
||||
"required": ["name", "lat"],
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class TestArrayRowCellsFollowTheItemSchema:
|
||||
"""A table row posts its cells as ``cities.0.timezone``. The schema
|
||||
lookup stopped at the array, so each cell was parsed blind: a blank
|
||||
optional text cell became null and a text cell holding digits became a
|
||||
number, and either failed validation -- every save of the page, for as
|
||||
long as the row existed (geochron's city without a timezone, a countdown
|
||||
named "2027")."""
|
||||
|
||||
ROW = {"cities.0.name": "Tokyo", "cities.0.timezone": "Asia/Tokyo",
|
||||
"cities.0.lat": "35.68", "cities.0.show": "true",
|
||||
"__rendered_section": ["cities"]}
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _rows(self, env):
|
||||
env.use_schema(ROWS_SCHEMA)
|
||||
env.store({"enabled": True, "cities": [
|
||||
{"name": "Tokyo", "timezone": "Asia/Tokyo", "lat": 35.68, "show": True}]})
|
||||
|
||||
def test_a_blank_optional_text_cell_saves(self, env):
|
||||
resp = env.post_form({**self.ROW, "cities.0.timezone": ""})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.main()[PLUGIN_ID]["cities"][0]["timezone"] == ""
|
||||
|
||||
def test_a_text_cell_of_digits_stays_text(self, env):
|
||||
resp = env.post_form({**self.ROW, "cities.0.name": "2027"})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.main()[PLUGIN_ID]["cities"][0]["name"] == "2027"
|
||||
|
||||
def test_number_and_boolean_cells_still_convert(self, env):
|
||||
resp = env.post_form({**self.ROW, "cities.0.show": "false"})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.main()[PLUGIN_ID]["cities"] == [
|
||||
{"name": "Tokyo", "timezone": "Asia/Tokyo", "lat": 35.68, "show": False}]
|
||||
|
||||
|
||||
class TestMaskedSecretCellsInARow:
|
||||
"""The same lookup: a row's secret cell, rendered blank, came back as
|
||||
null and failed validation, so a plugin with secrets in a list could not
|
||||
be saved from its settings page at all."""
|
||||
|
||||
def test_the_stored_tokens_survive_a_save_of_the_form(self, env):
|
||||
resp = env.post_form({
|
||||
"city": "Lyon", "accounts.0.name": "a", "accounts.0.token": "",
|
||||
"accounts.1.name": "b", "accounts.1.token": "",
|
||||
"__rendered_section": ["city", "accounts"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.secrets()[PLUGIN_ID] == STORED_SECRETS[PLUGIN_ID]
|
||||
assert env.main()[PLUGIN_ID]["accounts"] == [{"name": "a"}, {"name": "b"}]
|
||||
|
||||
|
||||
class TestABlankSecretIsLeftAsStored:
|
||||
"""The form renders a secret blank and posts the blank back. For a
|
||||
required secret with no default (youtube-stats' api_key) the blank was
|
||||
read as null, failed validation, and blocked every save of the page
|
||||
until the key was typed in again."""
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _required_secret(self, env):
|
||||
schema = json.loads(json.dumps(SCHEMA))
|
||||
del schema["properties"]["api_key"]["default"]
|
||||
schema["required"] = ["api_key"]
|
||||
env.use_schema(schema)
|
||||
|
||||
def test_saving_other_settings_keeps_the_stored_secret(self, env):
|
||||
resp = env.post_form({"api_key": "", "city": "Lyon",
|
||||
"__rendered_section": ["api_key", "city"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.main()[PLUGIN_ID]["city"] == "Lyon"
|
||||
assert env.secrets()[PLUGIN_ID]["api_key"] == "TOPSECRET"
|
||||
|
||||
def test_a_new_secret_is_still_saved(self, env):
|
||||
resp = env.post_form({"api_key": "NEW-KEY", "city": "Lyon",
|
||||
"__rendered_section": ["api_key", "city"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.secrets()[PLUGIN_ID]["api_key"] == "NEW-KEY"
|
||||
|
||||
def test_a_changed_secret_then_left_blank_stays_changed(self, env):
|
||||
# The second save must not write back what the first one's load
|
||||
# had merged in (the old key)
|
||||
env.post_form({"api_key": "NEW-KEY", "__rendered_section": ["api_key"]})
|
||||
resp = env.post_form({"api_key": "", "city": "Nice",
|
||||
"__rendered_section": ["api_key", "city"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.secrets()[PLUGIN_ID]["api_key"] == "NEW-KEY"
|
||||
|
||||
def test_a_blank_list_secret_is_left_as_stored_too(self, env, tmp_path):
|
||||
schema = json.loads(json.dumps(SCHEMA))
|
||||
schema["properties"]["tokens"] = {"type": "array", "x-secret": True,
|
||||
"items": {"type": "string"}, "default": []}
|
||||
env.use_schema(schema)
|
||||
secrets = env.secrets()
|
||||
secrets[PLUGIN_ID]["tokens"] = ["t1", "t2"]
|
||||
(tmp_path / "config_secrets.json").write_text(json.dumps(secrets))
|
||||
resp = env.post_form({"tokens": "", "city": "Lyon",
|
||||
"__rendered_section": ["tokens", "city"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.secrets()[PLUGIN_ID]["tokens"] == ["t1", "t2"]
|
||||
|
||||
|
||||
class TestSaveRefusesWhatIsNotAPluginId:
|
||||
"""GET and reset refuse a core section or a malformed id; the save took
|
||||
any of them. ``{"plugin_id": "display"}`` merged unvalidated values into
|
||||
the core display section, and an id that was not a string raised a
|
||||
TypeError, answered as a 500."""
|
||||
|
||||
def test_a_core_section_is_refused_and_left_alone(self, env):
|
||||
env.store({"hardware": {"rows": 32}}, plugin_id="display")
|
||||
resp = env.post_json({"hardware": {"rows": "banana"}}, plugin_id="display")
|
||||
assert resp.status_code == 400
|
||||
assert env.main()["display"] == {"hardware": {"rows": 32}}
|
||||
|
||||
def test_the_form_save_refuses_one_too(self, env):
|
||||
resp = env.post_form({"password_hash": "x"}, plugin_id="web_auth")
|
||||
assert resp.status_code == 400
|
||||
assert "web_auth" not in env.main()
|
||||
|
||||
@pytest.mark.parametrize("plugin_id", [["demo"], {"id": "demo"}, 7, "", "../demo"])
|
||||
def test_a_malformed_id_is_a_400(self, env, plugin_id):
|
||||
resp = env.post_json({"city": "Lyon"}, plugin_id=plugin_id)
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
class TestTextFieldsKeepWhatWasTyped:
|
||||
"""A text field holding "true", "False", "[1, 2]" or "{}" was converted
|
||||
to a boolean, list or object before the schema's type was consulted, and
|
||||
the save then failed validation for a perfectly good string."""
|
||||
|
||||
@pytest.mark.parametrize("typed", ["true", "False", "[1, 2]", "{}", "42"])
|
||||
def test_a_text_field(self, env, typed):
|
||||
resp = env.post_form({"city": typed, "__rendered_section": ["city"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.main()[PLUGIN_ID]["city"] == typed
|
||||
|
||||
def test_a_nullable_text_field(self, env):
|
||||
schema = json.loads(json.dumps(SCHEMA))
|
||||
schema["properties"]["nickname"] = {"type": ["string", "null"], "default": None}
|
||||
env.use_schema(schema)
|
||||
resp = env.post_form({"nickname": "false", "__rendered_section": ["nickname"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.main()[PLUGIN_ID]["nickname"] == "false"
|
||||
|
||||
def test_other_types_still_convert(self, env):
|
||||
resp = env.post_form({"mqtt.host": "true", "mqtt.port": "8883",
|
||||
"__rendered_section": ["mqtt"]})
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert env.main()[PLUGIN_ID]["mqtt"] == {"host": "true", "port": 8883}
|
||||
@@ -957,6 +957,19 @@ def _get_schema_property(schema, key_path):
|
||||
i = j
|
||||
matched = True
|
||||
break
|
||||
# Through an array to its items: a table row posts its cells
|
||||
# as "cities.0.timezone", where the index names no property.
|
||||
# Stopping here left each cell parsed with no schema at all,
|
||||
# so a blank text cell became null and "2027" a number.
|
||||
items = prop.get('items') if _schema_type_is(prop, 'array') else None
|
||||
if isinstance(items, dict) and parts[j].isdigit():
|
||||
if j + 1 == len(parts):
|
||||
return items
|
||||
if 'properties' in items:
|
||||
current = items['properties']
|
||||
i = j + 1
|
||||
matched = True
|
||||
break
|
||||
# Matched a non-object before consuming the path — can't go deeper.
|
||||
return None
|
||||
if not matched:
|
||||
@@ -1040,6 +1053,16 @@ def _parse_form_value_with_schema(value, key_path, schema):
|
||||
|
||||
# Handle None/empty values
|
||||
if value is None or (isinstance(value, str) and value.strip() == ''):
|
||||
# The form draws a stored secret blank, so a blank secret means
|
||||
# "unchanged", and "" is what the save drops as unchanged
|
||||
# (remove_empty_secrets). A required one with no default fell
|
||||
# through to None below, failed validation, and blocked every save
|
||||
# of the page until the secret was typed in again. Not _SKIP_FIELD:
|
||||
# that keeps the merged value from load_config(), which the save
|
||||
# would then write back to config_secrets.json. Text secrets only:
|
||||
# a list or object one gets its empty value below, dropped the same.
|
||||
if prop and prop.get('x-secret') and prop.get('type', 'string') == 'string':
|
||||
return ""
|
||||
# A nullable field left blank means null, not an empty container.
|
||||
# This is the inherit sentinel for per-mode style overrides: an
|
||||
# empty list there would read as "the user chose no colour" rather
|
||||
@@ -1074,6 +1097,14 @@ def _parse_form_value_with_schema(value, key_path, schema):
|
||||
if isinstance(value, str):
|
||||
stripped = value.strip()
|
||||
|
||||
# A text field keeps what was typed. The guesses below ran first, so
|
||||
# "true", "False", "[1, 2]" or "{}" in a text field became a boolean,
|
||||
# list or object, and the save failed validation for a good string.
|
||||
declared = prop.get('type') if isinstance(prop, dict) else None
|
||||
if declared == 'string' or (isinstance(declared, list) and
|
||||
[t for t in declared if t != 'null'] == ['string']):
|
||||
return value
|
||||
|
||||
# Check for boolean strings
|
||||
if stripped.lower() == 'true':
|
||||
return True
|
||||
|
||||
@@ -16,6 +16,7 @@ import web_interface.blueprints.api_v3 as _pkg
|
||||
# as module attributes, and a value binding would not see the patch.
|
||||
# Several are also called from helpers that live in __init__, so the
|
||||
# package is the only patch point that covers every caller.
|
||||
from web_interface.cache import delete_cached
|
||||
|
||||
|
||||
@api_v3.route('/backup/preview', methods=['GET'])
|
||||
@@ -85,6 +86,17 @@ _RESTORE_OPTION_KEYS = frozenset((
|
||||
'restore_config', 'restore_secrets', 'restore_wifi', 'restore_fonts',
|
||||
'restore_plugin_uploads', 'reinstall_plugins',
|
||||
))
|
||||
def _installed_path(psm, plugin_id):
|
||||
"""Where the store finds ``plugin_id`` installed, or None.
|
||||
|
||||
The same lookup install_plugin makes to decide that a copy exists: the
|
||||
id, or an id the registry proves is the same plugin (``aliases``, the
|
||||
``plugin_path`` name), never a bare ``ledmatrix-<id>`` folder.
|
||||
"""
|
||||
found = psm._existing_install(plugin_id)
|
||||
return found if isinstance(found, Path) and found.exists() else None
|
||||
|
||||
|
||||
@api_v3.route('/backup/restore', methods=['POST'])
|
||||
def backup_restore():
|
||||
"""Restore a backup ZIP with optional RestoreOptions."""
|
||||
@@ -134,6 +146,10 @@ def backup_restore():
|
||||
os.unlink(tmp_path)
|
||||
except OSError:
|
||||
pass
|
||||
# Restored fonts reach the Fonts tab through a catalog cached for five
|
||||
# minutes (fonts.py); upload and delete clear it, and so must this.
|
||||
if any(str(item).startswith('fonts') for item in result.restored):
|
||||
delete_cached('fonts_catalog')
|
||||
|
||||
# Reinstall plugins if requested and store manager available
|
||||
if options.reinstall_plugins and result.plugins_to_install:
|
||||
@@ -143,6 +159,15 @@ def backup_restore():
|
||||
if not pid:
|
||||
continue
|
||||
try:
|
||||
# Only what is missing. install_plugin replaces an installed
|
||||
# copy with a fresh download, so restoring onto the same
|
||||
# device re-downloaded every plugin, and one installed from
|
||||
# its own URL (not in the registry) "failed" and failed the
|
||||
# whole restore while it sat there installed. The store's
|
||||
# own lookup, so registry aliases count as installed too.
|
||||
if psm and _installed_path(psm, pid) is not None:
|
||||
result.skipped.append(f'plugin:{pid} (installed)')
|
||||
continue
|
||||
if psm and hasattr(psm, 'install_plugin'):
|
||||
ok = psm.install_plugin(pid)
|
||||
if ok:
|
||||
|
||||
@@ -507,7 +507,11 @@ def save_main_config():
|
||||
# Try to get JSON data first, fallback to form data
|
||||
data = None
|
||||
if request.is_json:
|
||||
data = request.get_json()
|
||||
# silent=True, as in save_raw_main_config: get_json() raised
|
||||
# Werkzeug's BadRequest into the catch-all below, a 500.
|
||||
data = request.get_json(silent=True)
|
||||
if data is None and request.get_data():
|
||||
return jsonify({'status': 'error', 'message': 'Invalid JSON in request body'}), 400
|
||||
if data is not None and not isinstance(data, dict):
|
||||
return jsonify({'status': 'error', 'message': 'Request body must be a JSON object'}), 400
|
||||
else:
|
||||
|
||||
@@ -69,6 +69,26 @@ def _deliver_on_demand(payload):
|
||||
return 'mailbox', reason
|
||||
|
||||
|
||||
def _withdraw_on_demand(request_id):
|
||||
"""Take a start request the route has refused back out of the mailbox.
|
||||
|
||||
The display reads the mailbox for an hour without looking at a
|
||||
request's age, so one left there after an error answer ran whenever the
|
||||
display next started. Only this request is removed: the mailbox is
|
||||
re-read and cleared only while it still holds this request_id, as the
|
||||
display's _consume_on_demand_request does, so a newer request posted in
|
||||
the meantime stays for the display to take.
|
||||
"""
|
||||
cache = _cache_manager()
|
||||
try:
|
||||
current = cache.get('display_on_demand_request', max_age=3600, memory_ttl=0)
|
||||
if isinstance(current, dict) and current.get('request_id') == request_id:
|
||||
cache.delete('display_on_demand_request')
|
||||
except Exception: # the route is answering an error already
|
||||
logger.warning("Could not withdraw on-demand request %s from the mailbox",
|
||||
request_id, exc_info=True)
|
||||
|
||||
|
||||
@api_v3.route('/display/current', methods=['GET'])
|
||||
def get_display_current():
|
||||
"""The latest display preview, as the /stream/display SSE stream sends it.
|
||||
@@ -259,9 +279,26 @@ def start_on_demand_display():
|
||||
}
|
||||
transport, socket_error = _deliver_on_demand(request_payload)
|
||||
|
||||
# A socket acknowledgement is the display itself answering: it is
|
||||
# running and has the request queued, whatever systemd says (a display
|
||||
# run by hand or in the emulator has no active unit). So nothing is
|
||||
# checked or started for it -- that answered "not running" for a request
|
||||
# that had already taken effect. The service is still reported the way
|
||||
# _ensure_display_service_running reports a running one.
|
||||
if transport == 'socket':
|
||||
service_result = (dict(_get_display_service_status(), started=False)
|
||||
if start_service else None)
|
||||
return _on_demand_started(request_id, resolved_plugin, resolved_mode,
|
||||
duration, pinned, service_result, transport,
|
||||
socket_error)
|
||||
|
||||
service_status = _get_display_service_status()
|
||||
|
||||
if not service_status.get('active') and not start_service:
|
||||
# The request is in the mailbox, and the display reads it whenever
|
||||
# it next starts: taken back out, or a request answered with this
|
||||
# error ran later anyway.
|
||||
_withdraw_on_demand(request_id)
|
||||
return jsonify({
|
||||
'status': 'error',
|
||||
'message': 'Display service is not running. Please start the display service or enable "Start Service" option.',
|
||||
@@ -285,16 +322,25 @@ def start_on_demand_display():
|
||||
service_result = _ensure_display_service_running()
|
||||
# Check if service actually started
|
||||
if service_result and not service_result.get('active'):
|
||||
_withdraw_on_demand(request_id)
|
||||
return jsonify({
|
||||
'status': 'error',
|
||||
'message': 'Failed to start display service. Please check service logs or start it manually.',
|
||||
'service_result': service_result
|
||||
}), 500
|
||||
|
||||
return _on_demand_started(request_id, resolved_plugin, resolved_mode,
|
||||
duration, pinned, service_result, transport,
|
||||
socket_error)
|
||||
|
||||
|
||||
def _on_demand_started(request_id, plugin_id, mode, duration, pinned,
|
||||
service_result, transport, socket_error):
|
||||
"""The success answer of /display/on-demand/start."""
|
||||
response_data = {
|
||||
'request_id': request_id,
|
||||
'plugin_id': resolved_plugin,
|
||||
'mode': resolved_mode,
|
||||
'plugin_id': plugin_id,
|
||||
'mode': mode,
|
||||
'duration': duration,
|
||||
'pinned': pinned,
|
||||
'service': service_result,
|
||||
|
||||
@@ -3,8 +3,12 @@
|
||||
Routes decorate the shared `api_v3` Blueprint from the package `__init__`,
|
||||
so their endpoint names do not depend on which module they live in.
|
||||
"""
|
||||
import mimetypes
|
||||
|
||||
from flask import send_file
|
||||
|
||||
from web_interface.blueprints.api_v3 import (
|
||||
PROJECT_ROOT, Response, _plugin_directory, api_v3, datetime, hashlib,
|
||||
PROJECT_ROOT, _plugin_directory, api_v3, datetime, hashlib,
|
||||
json, jsonify, logger, os, request, uuid,
|
||||
)
|
||||
from src.common.path_safety import (
|
||||
@@ -231,8 +235,8 @@ def serve_plugin_static(plugin_id, file_path):
|
||||
if not requested_file.exists() or not requested_file.is_file():
|
||||
return jsonify({'status': 'error', 'message': 'File not found'}), 404
|
||||
|
||||
# Determine content type
|
||||
content_type = 'text/plain'
|
||||
# Determine content type. Text keeps the types this route always set;
|
||||
# anything else (an icon, a preview image) gets its own.
|
||||
name = requested_file.name
|
||||
if name.endswith('.html'):
|
||||
content_type = 'text/html'
|
||||
@@ -242,12 +246,14 @@ def serve_plugin_static(plugin_id, file_path):
|
||||
content_type = 'text/css'
|
||||
elif name.endswith('.json'):
|
||||
content_type = 'application/json'
|
||||
else:
|
||||
guessed = mimetypes.guess_type(name)[0]
|
||||
content_type = ('text/plain' if not guessed or guessed.startswith('text/')
|
||||
else guessed)
|
||||
|
||||
# Read and return file
|
||||
with open(requested_file, 'r', encoding='utf-8') as f:
|
||||
content = f.read()
|
||||
|
||||
return Response(content, mimetype=content_type)
|
||||
# Sent as bytes. Opening it as UTF-8 text failed to decode any binary
|
||||
# file, so an image answered 500 UnicodeDecodeError.
|
||||
return send_file(requested_file, mimetype=content_type)
|
||||
|
||||
|
||||
@api_v3.route('/plugins/assets/delete', methods=['POST'])
|
||||
|
||||
@@ -9,13 +9,15 @@ from web_interface.blueprints.api_v3 import (
|
||||
_enhance_schema_with_core_properties, _non_plugin_id_error,
|
||||
_filter_config_by_schema, _get_schema_property,
|
||||
_hidden_array_item_property, _plugin_directory,
|
||||
_parse_form_value_with_schema, _schema_allows_null, _schema_type_is,
|
||||
_set_missing_booleans_to_false, _set_nested_value, api_v3, datetime,
|
||||
deep_merge, error_response, exception_error_response, find_secret_fields,
|
||||
json, jsonify, logger, merge_secrets, os, remove_empty_secrets, request,
|
||||
separate_secrets, success_response, validate_request_json,
|
||||
_parse_form_value_with_schema, _redact_credentials, _schema_allows_null,
|
||||
_schema_type_is, _set_missing_booleans_to_false, _set_nested_value, api_v3,
|
||||
datetime, deep_merge, error_response, exception_error_response,
|
||||
find_secret_fields, json, jsonify, logger, merge_secrets, os,
|
||||
remove_empty_secrets, request, separate_secrets, success_response,
|
||||
validate_request_json,
|
||||
)
|
||||
from src.web_interface.config_arrays import coerce_array_shapes
|
||||
from src.web_interface.secret_helpers import mask_secret_fields
|
||||
from src.web_interface.validators import dedup_unique_arrays
|
||||
import web_interface.blueprints.api_v3 as _pkg
|
||||
# Read through the module rather than bound by value: tests patch these
|
||||
@@ -43,6 +45,12 @@ def get_plugin_config():
|
||||
context={'missing_params': ['plugin_id']},
|
||||
status_code=400
|
||||
)
|
||||
# load_config() merges config_secrets.json in, core sections
|
||||
# included: ?plugin_id=web_auth returned the login's cookie key and
|
||||
# password hash, and ?plugin_id=github the Plugin Store token.
|
||||
id_error = _non_plugin_id_error(plugin_id)
|
||||
if id_error:
|
||||
return id_error
|
||||
|
||||
# Get plugin configuration from config manager
|
||||
main_config = api_v3.config_manager.load_config()
|
||||
@@ -52,12 +60,13 @@ def get_plugin_config():
|
||||
# missing fields, reading legacy booleans as objects first: what the
|
||||
# plugin runs with, and what posts back through the JSON save
|
||||
schema_mgr = api_v3.schema_manager
|
||||
schema = None
|
||||
if schema_mgr:
|
||||
try:
|
||||
from src.plugin_system.schema_manager import prepare_plugin_config
|
||||
schema = schema_mgr.load_schema(plugin_id, use_cache=True)
|
||||
defaults = schema_mgr.generate_default_config(plugin_id, use_cache=True)
|
||||
plugin_config = prepare_plugin_config(
|
||||
plugin_config, schema_mgr.load_schema(plugin_id, use_cache=True), defaults)
|
||||
plugin_config = prepare_plugin_config(plugin_config, schema, defaults)
|
||||
except Exception as e:
|
||||
# Log but don't fail - defaults merge is best effort
|
||||
logger.warning("Could not merge defaults for %s: %s", plugin_id, e)
|
||||
@@ -158,6 +167,17 @@ def get_plugin_config():
|
||||
'display_duration': 30
|
||||
}
|
||||
|
||||
# Secrets go out blank, as the settings page renders them (#276 added
|
||||
# this; #330 dropped it). Blank, not the bullets GET /config/secrets
|
||||
# uses: the save reads a blank secret as "unchanged", so this
|
||||
# response posts back without erasing one.
|
||||
properties = schema.get('properties') if isinstance(schema, dict) else None
|
||||
if isinstance(properties, dict):
|
||||
plugin_config = mask_secret_fields(plugin_config, properties)
|
||||
else:
|
||||
# No schema to mark them: blank whatever is named like one
|
||||
plugin_config = _redact_credentials(plugin_config)
|
||||
|
||||
return success_response(data=plugin_config)
|
||||
except Exception as e:
|
||||
return exception_error_response(e, ErrorCode.CONFIG_LOAD_FAILED)
|
||||
@@ -183,6 +203,12 @@ def save_plugin_config():
|
||||
if error:
|
||||
return error
|
||||
plugin_id = data['plugin_id']
|
||||
# As reset and uninstall do: {"plugin_id": "display"} merged
|
||||
# unvalidated values into the core display section, and an id
|
||||
# that was not a string raised a TypeError, answered as a 500.
|
||||
id_error = _non_plugin_id_error(plugin_id)
|
||||
if id_error:
|
||||
return id_error
|
||||
submitted_config = data.get('config', {})
|
||||
if not isinstance(submitted_config, dict):
|
||||
return error_response(
|
||||
@@ -201,6 +227,9 @@ def save_plugin_config():
|
||||
'plugin_id required in query string',
|
||||
status_code=400
|
||||
)
|
||||
id_error = _non_plugin_id_error(plugin_id)
|
||||
if id_error:
|
||||
return id_error
|
||||
|
||||
# Load existing config as base (partial form updates should merge, not replace)
|
||||
existing_config = {}
|
||||
|
||||
@@ -42,6 +42,29 @@ def _store_incompatibility(plugin: dict) -> Optional[str]:
|
||||
return reason if isinstance(reason, str) and reason else None
|
||||
|
||||
|
||||
def _installed_plugin_id(plugin_id: str) -> str:
|
||||
"""The id the plugin installed for store entry ``plugin_id`` declares.
|
||||
|
||||
A registry entry can install under another id: ``weather`` installs a
|
||||
directory whose manifest says ``ledmatrix-weather``, and that is the id
|
||||
the plugin list, the config section and /plugins/toggle know it by. The
|
||||
install is found the way the store's update and uninstall find it (the
|
||||
entry's id, ``aliases`` and ``plugin_path`` name); ``plugin_id`` itself
|
||||
when its manifest can't be read.
|
||||
"""
|
||||
try:
|
||||
plugin_dir = api_v3.plugin_store_manager._find_plugin_path(plugin_id)
|
||||
manifest_path = (resolve_under(plugin_dir, 'manifest.json')
|
||||
if isinstance(plugin_dir, Path) else None)
|
||||
if manifest_path is None or not manifest_path.is_file():
|
||||
return plugin_id
|
||||
with open(manifest_path, 'r', encoding='utf-8') as f:
|
||||
manifest_id = json.load(f).get('id')
|
||||
except Exception: # noqa: BLE001 - only names the install for the client
|
||||
return plugin_id
|
||||
return manifest_id if isinstance(manifest_id, str) and safe_path_component(manifest_id) else plugin_id
|
||||
|
||||
|
||||
def _listed_plugin_dir(base: Path, name: str) -> Optional[Path]:
|
||||
"""The entry of ``base`` called ``name``, or None.
|
||||
|
||||
@@ -58,6 +81,27 @@ def _listed_plugin_dir(base: Path, name: str) -> Optional[Path]:
|
||||
return None
|
||||
|
||||
|
||||
def _enqueue_or_conflict(operation_type, plugin_id, callback):
|
||||
"""``(operation_id, None)``, or ``(None, a 409 response)``.
|
||||
|
||||
The queue raises ValueError when the plugin already has an operation
|
||||
waiting or running -- a double-clicked Install, an uninstall during an
|
||||
install. That is the caller's timing, not a server fault: it reached
|
||||
the client as a 500, and the uninstall route recorded a failed
|
||||
uninstall that had never started.
|
||||
"""
|
||||
try:
|
||||
return api_v3.operation_queue.enqueue_operation(
|
||||
operation_type, plugin_id, operation_callback=callback), None
|
||||
except ValueError:
|
||||
return None, error_response(
|
||||
ErrorCode.PLUGIN_OPERATION_CONFLICT,
|
||||
f'Plugin {plugin_id} already has an install, update or uninstall '
|
||||
'in progress; wait for it to finish, then try again',
|
||||
status_code=409
|
||||
)
|
||||
|
||||
|
||||
@api_v3.route('/plugins/update', methods=['POST'])
|
||||
def update_plugin():
|
||||
"""Update plugin"""
|
||||
@@ -379,11 +423,10 @@ def uninstall_plugin():
|
||||
preserve_config=preserve_config)}
|
||||
|
||||
# Enqueue operation
|
||||
operation_id = api_v3.operation_queue.enqueue_operation(
|
||||
OperationType.UNINSTALL,
|
||||
plugin_id,
|
||||
operation_callback=uninstall_callback
|
||||
)
|
||||
operation_id, conflict = _enqueue_or_conflict(
|
||||
OperationType.UNINSTALL, plugin_id, uninstall_callback)
|
||||
if conflict:
|
||||
return conflict
|
||||
|
||||
return success_response(
|
||||
data={'operation_id': operation_id},
|
||||
@@ -487,8 +530,10 @@ def install_plugin():
|
||||
)
|
||||
|
||||
branch_msg = f" (branch: {branch})" if branch else ""
|
||||
# plugin_id: the id to enable it by (see _installed_plugin_id).
|
||||
return {'success': True,
|
||||
'message': f'Plugin {plugin_id} installed successfully{branch_msg}',
|
||||
'plugin_id': _installed_plugin_id(plugin_id),
|
||||
**_store_restart_fields('install', _plugin_enabled_in_config(plugin_id))}
|
||||
else:
|
||||
error_msg = f'Failed to install plugin {plugin_id}'
|
||||
@@ -513,11 +558,10 @@ def install_plugin():
|
||||
raise Exception(error_msg)
|
||||
|
||||
# Enqueue operation
|
||||
operation_id = api_v3.operation_queue.enqueue_operation(
|
||||
OperationType.INSTALL,
|
||||
plugin_id,
|
||||
operation_callback=install_callback
|
||||
)
|
||||
operation_id, conflict = _enqueue_or_conflict(
|
||||
OperationType.INSTALL, plugin_id, install_callback)
|
||||
if conflict:
|
||||
return conflict
|
||||
|
||||
branch_msg = f" (branch: {branch})" if branch else ""
|
||||
return success_response(
|
||||
@@ -546,7 +590,8 @@ def install_plugin():
|
||||
branch_msg = f" (branch: {branch})" if branch else ""
|
||||
return success_response(
|
||||
message=f'Plugin installed successfully{branch_msg}',
|
||||
extra=_store_restart_fields('install', _plugin_enabled_in_config(plugin_id)))
|
||||
extra={'plugin_id': _installed_plugin_id(plugin_id),
|
||||
**_store_restart_fields('install', _plugin_enabled_in_config(plugin_id))})
|
||||
else:
|
||||
error_msg = f'Failed to install plugin {plugin_id}'
|
||||
if branch:
|
||||
|
||||
@@ -145,6 +145,14 @@ def get_installed_plugins():
|
||||
vegas_participation, vegas_participation_source = _vegas_participation(
|
||||
plugin_id, plugin_config, plugin_info)
|
||||
|
||||
# The modes the manifest declares, from the catalog as /display/modes
|
||||
# and on-demand/start read them. The on-demand modal offers these;
|
||||
# without them it offered only the plugin id, which the display
|
||||
# turns into the first mode. Strings only: a manifest is hand-edited.
|
||||
declared_modes = api_v3.plugin_catalog.get_plugin_display_modes(plugin_id)
|
||||
display_modes = ([m for m in declared_modes if isinstance(m, str)]
|
||||
if isinstance(declared_modes, list) else [])
|
||||
|
||||
return {
|
||||
'id': plugin_id,
|
||||
'name': plugin_info.get('name', plugin_id),
|
||||
@@ -158,6 +166,7 @@ def get_installed_plugins():
|
||||
# The tab nav uses this as the <i> element's Font Awesome class
|
||||
# (app-shell.js / app-early.js); only a string can be one.
|
||||
'icon': plugin_info.get('icon') if isinstance(plugin_info.get('icon'), str) else None,
|
||||
'display_modes': display_modes,
|
||||
'enabled': enabled,
|
||||
'verified': verified,
|
||||
# loaded, state, error_info, loaded_version, loaded_at: the
|
||||
@@ -430,6 +439,10 @@ sys.exit(proc.returncode)
|
||||
import tempfile
|
||||
import json as json_lib
|
||||
|
||||
# The params reach the wrapper on its stdin, never in
|
||||
# its source: written there as `params = <JSON>`, a
|
||||
# true, false or null was an undefined name and the
|
||||
# wrapper died with a NameError before the script ran.
|
||||
params_json = json_lib.dumps(action_params)
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as wrapper:
|
||||
wrapper.write(f'''import sys
|
||||
@@ -440,6 +453,9 @@ import json
|
||||
# Set LEDMATRIX_ROOT
|
||||
os.environ['LEDMATRIX_ROOT'] = r"{PROJECT_ROOT}"
|
||||
|
||||
# The params, as JSON on this wrapper's own stdin
|
||||
params = json.loads(sys.stdin.read())
|
||||
|
||||
# Run the script and provide params as JSON via stdin
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, r"{script_file}"],
|
||||
@@ -451,7 +467,6 @@ proc = subprocess.Popen(
|
||||
)
|
||||
|
||||
# Send params as JSON to stdin
|
||||
params = {params_json}
|
||||
stdout, _ = proc.communicate(input=json.dumps(params), timeout=120)
|
||||
print(stdout)
|
||||
sys.exit(proc.returncode)
|
||||
@@ -461,6 +476,7 @@ sys.exit(proc.returncode)
|
||||
try:
|
||||
result = subprocess.run(
|
||||
['python3', wrapper_path],
|
||||
input=params_json,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=120,
|
||||
|
||||
@@ -11,7 +11,7 @@ _SAFE_PLUGIN_ID_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
|
||||
_SAFE_WEB_UI_FILE_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}\.html$')
|
||||
_SAFE_WIDGET_NAME_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}$')
|
||||
_SAFE_WIDGET_SCRIPT_RE = re.compile(r'^[a-zA-Z0-9_-]{1,64}\.js$')
|
||||
from src.web_interface.secret_helpers import mask_secret_fields
|
||||
from src.web_interface.secret_helpers import mask_all_secret_values, mask_secret_fields
|
||||
from src.plugin_system.schema_manager import plugin_config_defaults, prepare_plugin_config
|
||||
from src.common.path_safety import resolve_under, safe_path_component
|
||||
from src.pi5_matrix_support import is_raspberry_pi_5
|
||||
@@ -623,9 +623,14 @@ def _load_raw_json_partial():
|
||||
main_config_data = pages_v3.config_manager.get_raw_file_content('main')
|
||||
# The web login section (password and token hashes) is managed in
|
||||
# General > Security, never in this editor; its save keeps it.
|
||||
# The rest is masked, as GET /api/v3/config/secrets masks it: this
|
||||
# page is served to anyone who can reach the port while the web
|
||||
# login is off, and it was handing them every credential in the
|
||||
# file. The save strips the masks and merges onto the stored file
|
||||
# (save_raw_secrets_config), so a value left masked stays as it is.
|
||||
from web_interface.auth import strip_auth_section
|
||||
secrets_config_data = strip_auth_section(
|
||||
pages_v3.config_manager.get_raw_file_content('secrets'))
|
||||
secrets_config_data = mask_all_secret_values(strip_auth_section(
|
||||
pages_v3.config_manager.get_raw_file_content('secrets')))
|
||||
main_config_json = json.dumps(main_config_data, indent=4)
|
||||
secrets_config_json = json.dumps(secrets_config_data, indent=4)
|
||||
|
||||
|
||||
@@ -48,12 +48,15 @@ const PluginInstallManager = {
|
||||
* @returns {Promise<Array>} Update results, one per plugin sent
|
||||
*/
|
||||
async updateAll(onProgress, options = {}) {
|
||||
// Prefer PluginStateManager if populated, fall back to window.installedPlugins
|
||||
// (plugins_manager.js populates window.installedPlugins independently)
|
||||
const stateManagerPlugins = window.PluginStateManager && window.PluginStateManager.installedPlugins;
|
||||
const listed = (stateManagerPlugins && stateManagerPlugins.length > 0)
|
||||
? stateManagerPlugins
|
||||
: (window.installedPlugins || []);
|
||||
// window.installedPlugins is the live list: plugins_manager.js
|
||||
// republishes it after every install, uninstall and refresh.
|
||||
// PluginStateManager's copy is written only by the refresh at the end
|
||||
// of a run, so preferring it sent a second run the first run's
|
||||
// plugins -- an uninstalled one failed, a new one was skipped. It is
|
||||
// the fallback for a page without the plugin manager.
|
||||
const listed = Array.isArray(window.installedPlugins)
|
||||
? window.installedPlugins
|
||||
: ((window.PluginStateManager && window.PluginStateManager.installedPlugins) || []);
|
||||
// Snapshot: the list can be replaced while this loop is awaiting.
|
||||
const plugins = this.updatablePlugins(listed);
|
||||
|
||||
@@ -102,10 +105,18 @@ const PluginInstallManager = {
|
||||
}
|
||||
|
||||
// Reload plugin list once at the end. A failed refresh must not
|
||||
// lose the results: they carry the restart flags.
|
||||
if (window.PluginStateManager) {
|
||||
// lose the results: they carry the restart flags. The plugin
|
||||
// manager's load, forced past its caches, also redraws the installed
|
||||
// grid and its Updates badge; PluginStateManager's only replaced
|
||||
// window.installedPlugins, so the cards kept offering "Update to vX"
|
||||
// for what had just been updated.
|
||||
const pluginManager = window.pluginManager;
|
||||
const refresh = (pluginManager && typeof pluginManager.loadInstalledPlugins === 'function')
|
||||
? () => pluginManager.loadInstalledPlugins(true)
|
||||
: (window.PluginStateManager ? () => window.PluginStateManager.loadInstalledPlugins() : null);
|
||||
if (refresh) {
|
||||
try {
|
||||
await window.PluginStateManager.loadInstalledPlugins();
|
||||
await refresh();
|
||||
} catch (error) {
|
||||
console.warn('Could not refresh the installed plugin list after updating:', error);
|
||||
}
|
||||
|
||||
@@ -18,7 +18,9 @@
|
||||
* });
|
||||
*
|
||||
* The container re-renders from /api/v3/plugins/installed each init; the
|
||||
* hidden input(s) must already hold the saved order/exclusions (JSON).
|
||||
* hidden input(s) must already hold the saved order/exclusions (JSON). Saved
|
||||
* ids of disabled plugins (installed, but without a row) stay in them, in
|
||||
* their saved places; ids of plugins no longer installed are dropped.
|
||||
*/
|
||||
(function() {
|
||||
'use strict';
|
||||
@@ -39,17 +41,60 @@
|
||||
const excludedInput = options.excludedInputId ? document.getElementById(options.excludedInputId) : null;
|
||||
if (!container || !orderInput) return;
|
||||
|
||||
// The saved lists as the inputs held them when the rows were drawn.
|
||||
// Only enabled plugins get a row, and the inputs are rewritten from
|
||||
// the rows, so a disabled plugin's place and exclusion have to be
|
||||
// carried over from these: dropped, the next Display or Durations
|
||||
// save stored the lists without it, and once re-enabled it came back
|
||||
// at the end of the rotation and scrolling in Vegas again.
|
||||
let savedOrder = [];
|
||||
let savedExcluded = [];
|
||||
// Every installed plugin's id, enabled or not, from the same
|
||||
// response. A saved id outside it belongs to an uninstalled plugin
|
||||
// and is dropped, as every save used to; without the list, nothing
|
||||
// is dropped.
|
||||
let installedIds = null;
|
||||
|
||||
// Saved ids of installed plugins with no row, once each. Only
|
||||
// strings: /config/main refuses a list holding anything else, which
|
||||
// would block every save.
|
||||
function unlisted(saved, rowIds) {
|
||||
const seen = new Set(rowIds);
|
||||
return saved.filter(id => {
|
||||
if (typeof id !== 'string' || seen.has(id)) return false;
|
||||
if (installedIds && !installedIds.has(id)) return false;
|
||||
seen.add(id);
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
function syncInputs() {
|
||||
const order = [];
|
||||
const rowIds = [];
|
||||
const excluded = [];
|
||||
container.querySelectorAll('.plugin-order-item').forEach(item => {
|
||||
const pluginId = item.dataset.pluginId;
|
||||
order.push(pluginId);
|
||||
rowIds.push(pluginId);
|
||||
const checkbox = item.querySelector('.plugin-order-include');
|
||||
if (checkbox && !checkbox.checked) excluded.push(pluginId);
|
||||
});
|
||||
orderInput.value = JSON.stringify(order);
|
||||
if (excludedInput) excludedInput.value = JSON.stringify(excluded);
|
||||
// An id without a row keeps its saved slot; the rows fill the
|
||||
// other slots in their current order, and any rows left over
|
||||
// (plugins not in the saved order) go last.
|
||||
const kept = new Set(unlisted(savedOrder, rowIds));
|
||||
const order = [];
|
||||
let next = 0;
|
||||
savedOrder.forEach(id => {
|
||||
if (kept.has(id)) {
|
||||
order.push(id);
|
||||
kept.delete(id);
|
||||
} else if (rowIds.includes(id) && next < rowIds.length) {
|
||||
order.push(rowIds[next++]);
|
||||
}
|
||||
});
|
||||
orderInput.value = JSON.stringify(order.concat(rowIds.slice(next)));
|
||||
if (excludedInput) {
|
||||
excludedInput.value = JSON.stringify(excluded.concat(unlisted(savedExcluded, rowIds)));
|
||||
}
|
||||
}
|
||||
|
||||
function setupDragAndDrop() {
|
||||
@@ -104,6 +149,7 @@
|
||||
.then(data => {
|
||||
const allPlugins = (data.data && data.data.plugins) || data.plugins || [];
|
||||
const plugins = allPlugins.filter(p => p.enabled);
|
||||
installedIds = new Set(allPlugins.map(p => p && p.id));
|
||||
if (plugins.length === 0) {
|
||||
const empty = document.createElement('p');
|
||||
empty.className = 'text-sm text-gray-500 italic';
|
||||
@@ -125,6 +171,8 @@
|
||||
// (e.g. a saved value of "null"); normalize to arrays.
|
||||
if (!Array.isArray(currentOrder)) currentOrder = [];
|
||||
if (!Array.isArray(excluded)) excluded = [];
|
||||
savedOrder = currentOrder;
|
||||
savedExcluded = excluded;
|
||||
|
||||
// Saved order first, then any newly enabled plugins.
|
||||
const orderedPlugins = [];
|
||||
|
||||
@@ -34,8 +34,8 @@
|
||||
*
|
||||
* Layout: a few handlers defined up front, outside any IIFE, because the
|
||||
* cards and other scripts call them through window (configurePlugin,
|
||||
* togglePlugin, the GitHub token helpers, handleGitHubPluginInstall,
|
||||
* checkGitHubAuthStatus); then the plugin-manager IIFE (private state:
|
||||
* togglePlugin, the GitHub token helpers, checkGitHubAuthStatus); then the
|
||||
* plugin-manager IIFE (private state:
|
||||
* installedPlugins, the store cache, the on-demand poller); then the
|
||||
* Starlark IIFE.
|
||||
*
|
||||
@@ -386,103 +386,6 @@ window.toggleGithubTokenContent = function(e) {
|
||||
}
|
||||
};
|
||||
|
||||
// Simple standalone handler for GitHub plugin installation
|
||||
// Defined early and globally to ensure it's always available
|
||||
debugLog('[DEFINE] Defining handleGitHubPluginInstall function...');
|
||||
window.handleGitHubPluginInstall = function() {
|
||||
debugLog('[handleGitHubPluginInstall] Function called!');
|
||||
|
||||
const urlInput = document.getElementById('github-plugin-url');
|
||||
const statusDiv = document.getElementById('github-plugin-status');
|
||||
const branchInput = document.getElementById('plugin-branch-input');
|
||||
const installBtn = document.getElementById('install-plugin-from-url');
|
||||
|
||||
if (!urlInput) {
|
||||
console.error('[handleGitHubPluginInstall] URL input not found');
|
||||
alert('Error: Could not find URL input field');
|
||||
return;
|
||||
}
|
||||
|
||||
const repoUrl = urlInput.value.trim();
|
||||
debugLog('[handleGitHubPluginInstall] Repo URL:', repoUrl);
|
||||
|
||||
if (!repoUrl) {
|
||||
if (statusDiv) {
|
||||
statusDiv.innerHTML = '<span class="text-red-600"><i class="fas fa-exclamation-circle mr-1"></i>Please enter a GitHub URL</span>';
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (!isGithubUrl(repoUrl)) {
|
||||
if (statusDiv) {
|
||||
statusDiv.innerHTML = '<span class="text-red-600"><i class="fas fa-exclamation-circle mr-1"></i>Please enter a valid GitHub URL</span>';
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Disable button and show loading
|
||||
if (installBtn) {
|
||||
installBtn.disabled = true;
|
||||
installBtn.innerHTML = '<i class="fas fa-spinner fa-spin mr-2"></i>Installing...';
|
||||
}
|
||||
if (statusDiv) {
|
||||
statusDiv.innerHTML = '<span class="text-blue-600"><i class="fas fa-spinner fa-spin mr-1"></i>Installing plugin...</span>';
|
||||
}
|
||||
|
||||
const branch = branchInput?.value?.trim() || null;
|
||||
const requestBody = { repo_url: repoUrl };
|
||||
if (branch) {
|
||||
requestBody.branch = branch;
|
||||
}
|
||||
|
||||
debugLog('[handleGitHubPluginInstall] Sending request:', requestBody);
|
||||
|
||||
fetch('/api/v3/plugins/install-from-url', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
body: JSON.stringify(requestBody)
|
||||
})
|
||||
.then(response => {
|
||||
debugLog('[handleGitHubPluginInstall] Response status:', response.status);
|
||||
return response.json();
|
||||
})
|
||||
.then(data => {
|
||||
debugLog('[handleGitHubPluginInstall] Response data:', data);
|
||||
if (data.status === 'success') {
|
||||
if (statusDiv) {
|
||||
statusDiv.innerHTML = `<span class="text-green-600"><i class="fas fa-check-circle mr-1"></i>Successfully installed: ${window.LEDEscape.html(data.plugin_id)}</span>`;
|
||||
}
|
||||
urlInput.value = '';
|
||||
|
||||
showNotification(`Plugin ${data.plugin_id} installed successfully`, 'success');
|
||||
window.noteRestartRequired(data);
|
||||
|
||||
setTimeout(() => window.pluginManager.loadInstalledPlugins(true).catch(() => {}), 1000);
|
||||
} else {
|
||||
if (statusDiv) {
|
||||
statusDiv.innerHTML = `<span class="text-red-600"><i class="fas fa-times-circle mr-1"></i>${window.LEDEscape.html(data.message || 'Installation failed')}</span>`;
|
||||
}
|
||||
showNotification(data.message || 'Installation failed', 'error');
|
||||
}
|
||||
})
|
||||
.catch(error => {
|
||||
console.error('[handleGitHubPluginInstall] Error:', error);
|
||||
if (statusDiv) {
|
||||
statusDiv.innerHTML = `<span class="text-red-600"><i class="fas fa-times-circle mr-1"></i>Error: ${window.LEDEscape.html(error.message)}</span>`;
|
||||
}
|
||||
showNotification('Error installing plugin: ' + error.message, 'error');
|
||||
})
|
||||
.finally(() => {
|
||||
if (installBtn) {
|
||||
installBtn.disabled = false;
|
||||
installBtn.innerHTML = '<i class="fas fa-download mr-2"></i>Install';
|
||||
}
|
||||
});
|
||||
};
|
||||
debugLog('[DEFINE] handleGitHubPluginInstall defined and ready');
|
||||
|
||||
// GitHub Authentication Status - Define early so it's available in IIFE
|
||||
// Shows warning banner only when token is missing or invalid
|
||||
// The token itself is never exposed to the frontend for security
|
||||
@@ -2083,6 +1986,13 @@ window.uninstallPlugin = function(pluginId) {
|
||||
});
|
||||
}
|
||||
|
||||
// How many times the store's Install polls a queued install, a second apart.
|
||||
// The server allows the plugin's dependency install 300 s on its own
|
||||
// (install_requirements_file in src/plugin_system/store_install.py), after a
|
||||
// download that fetches the plugin a file at a time; the 60 the poller
|
||||
// defaults to reported installs that then succeeded as timed out.
|
||||
const INSTALL_POLL_MAX_ATTEMPTS = 600;
|
||||
|
||||
function pollOperationStatus(operationId, pluginId, pluginName, options = {}) {
|
||||
const maxAttempts = options.maxAttempts || 60;
|
||||
const attempt = options.attempt || 0;
|
||||
@@ -2114,9 +2024,10 @@ function pollOperationStatus(operationId, pluginId, pluginName, options = {}) {
|
||||
|
||||
if (status === 'completed') {
|
||||
// The operation's result says whether the display picks
|
||||
// the change up by itself or needs a restart.
|
||||
// the change up by itself or needs a restart, and for an
|
||||
// install which id the plugin was installed as.
|
||||
window.noteRestartRequired(operation.result);
|
||||
onComplete();
|
||||
onComplete(operation.result);
|
||||
} else if (status === 'failed') {
|
||||
onFailed(operation.error || operation.message);
|
||||
} else {
|
||||
@@ -2270,10 +2181,18 @@ function showStoreLoading(show) {
|
||||
// ── Plugin Store: Client-Side Filter/Sort/Pagination ────────────────────────
|
||||
|
||||
function isStorePluginInstalled(pluginIdOrPlugin) {
|
||||
return Boolean(findInstalledStorePlugin(pluginIdOrPlugin));
|
||||
}
|
||||
|
||||
// The installed-list entry for a store plugin, or undefined. A registry entry
|
||||
// can be installed under another id -- `weather` is listed as the
|
||||
// `ledmatrix-weather` its manifest declares -- so its own id is tried first,
|
||||
// then its plugin_path name, then its aliases.
|
||||
function findInstalledStorePlugin(pluginIdOrPlugin) {
|
||||
const installed = window.installedPlugins || installedPlugins || [];
|
||||
// Accept either a plain ID string or a store plugin object (which may have plugin_path)
|
||||
if (typeof pluginIdOrPlugin === 'string') {
|
||||
return installed.some(p => p.id === pluginIdOrPlugin);
|
||||
return installed.find(p => p.id === pluginIdOrPlugin);
|
||||
}
|
||||
const storeId = pluginIdOrPlugin.id;
|
||||
// Derive the actual installed directory name from plugin_path (e.g. "plugins/ledmatrix-weather" → "ledmatrix-weather")
|
||||
@@ -2281,8 +2200,9 @@ function isStorePluginInstalled(pluginIdOrPlugin) {
|
||||
const pathDerivedId = pluginPath ? pluginPath.split('/').pop() : null;
|
||||
// Newer registries also list the other ids outright (the manifest id).
|
||||
const aliases = Array.isArray(pluginIdOrPlugin.aliases) ? pluginIdOrPlugin.aliases : [];
|
||||
return installed.some(p => p.id === storeId || (pathDerivedId && p.id === pathDerivedId)
|
||||
|| aliases.includes(p.id));
|
||||
return installed.find(p => p.id === storeId)
|
||||
|| (pathDerivedId ? installed.find(p => p.id === pathDerivedId) : undefined)
|
||||
|| installed.find(p => aliases.includes(p.id));
|
||||
}
|
||||
|
||||
// ── Plugin Store: search / filter / sort ────────────────────────────────
|
||||
@@ -2392,8 +2312,43 @@ function getStoreFilter() {
|
||||
return _storeFilter;
|
||||
}
|
||||
|
||||
// The category filter offers the categories the store's plugins have, as the
|
||||
// Starlark section does. The template ships only "All Categories": a fixed
|
||||
// list offered 7 of the registry's ~20 categories, so most plugins could not
|
||||
// be filtered to, and "Financial" missed the plugin filed under "finance".
|
||||
// One option per category whatever its case (the filter ignores case), and
|
||||
// rebuilt only when the set changes, or for a select freshly swapped in.
|
||||
function syncStoreCategoryOptions() {
|
||||
const select = document.getElementById('plugin-category');
|
||||
if (!select) return;
|
||||
const ctl = getStoreFilter();
|
||||
const selected = String((ctl ? ctl.state.filterCategory : select.value) || '');
|
||||
const byKey = new Map();
|
||||
(pluginStoreCache || []).forEach(plugin => {
|
||||
const category = plugin && typeof plugin.category === 'string' ? plugin.category : '';
|
||||
if (category.trim() && !byKey.has(category.toLowerCase())) {
|
||||
byKey.set(category.toLowerCase(), category);
|
||||
}
|
||||
});
|
||||
// The current choice stays selectable even if no plugin has it any more.
|
||||
if (selected && !byKey.has(selected.toLowerCase())) byKey.set(selected.toLowerCase(), selected);
|
||||
const categories = [...byKey.values()].sort((a, b) => a.localeCompare(b, undefined, { sensitivity: 'base' }));
|
||||
const key = categories.join('\n');
|
||||
if (select._storeCategories === key) return;
|
||||
select._storeCategories = key;
|
||||
select.innerHTML = '<option value="">All Categories</option>';
|
||||
categories.forEach(category => {
|
||||
const option = document.createElement('option');
|
||||
option.value = category;
|
||||
option.textContent = category.charAt(0).toUpperCase() + category.slice(1);
|
||||
select.appendChild(option);
|
||||
});
|
||||
select.value = selected;
|
||||
}
|
||||
|
||||
function applyStoreFiltersAndSort(skipPageReset) {
|
||||
if (!pluginStoreCache) return;
|
||||
syncStoreCategoryOptions();
|
||||
const ctl = getStoreFilter();
|
||||
if (ctl) {
|
||||
ctl.apply(skipPageReset);
|
||||
@@ -2510,11 +2465,45 @@ window.installPlugin = function(pluginId, branch = null) {
|
||||
requestBody.branch = branch;
|
||||
}
|
||||
|
||||
function enableAfterInstall() {
|
||||
const storeEntry = (pluginStoreCache || []).find(p => p && p.id === pluginId) || { id: pluginId };
|
||||
// Decided before the install changes the list, by the same match that
|
||||
// labelled the button Install or Reinstall. A reinstall keeps the plugin
|
||||
// as the user had it: enabling it here switched a deliberately disabled
|
||||
// plugin back on.
|
||||
const isReinstall = isStorePluginInstalled(storeEntry);
|
||||
|
||||
// The id the plugin was installed as, which can differ from the store's:
|
||||
// `weather` installs as the `ledmatrix-weather` its manifest declares,
|
||||
// and that is the id /plugins/toggle knows. The install answer names it
|
||||
// (plugin_id); from one that doesn't, the installed entry the store
|
||||
// entry matches, as for the Installed badge.
|
||||
function installedPluginId(result) {
|
||||
if (result && typeof result.plugin_id === 'string' && result.plugin_id) {
|
||||
return result.plugin_id;
|
||||
}
|
||||
const match = findInstalledStorePlugin(storeEntry);
|
||||
return match ? match.id : pluginId;
|
||||
}
|
||||
|
||||
function afterInstall(result) {
|
||||
// Reload first, so the new card exists (and, without plugin_id in the
|
||||
// answer, so the installed id can be found), then redraw the store's
|
||||
// badges from that list.
|
||||
loadInstalledPlugins(true).catch(() => {}).then(() => {
|
||||
applyStoreFiltersAndSort(true);
|
||||
if (isReinstall) {
|
||||
showNotification(`${pluginId} reinstalled`, 'success');
|
||||
return;
|
||||
}
|
||||
enableAfterInstall(installedPluginId(result));
|
||||
});
|
||||
}
|
||||
|
||||
function enableAfterInstall(installedId) {
|
||||
// Enable immediately so install -> enable is one step; only nudge
|
||||
// for a restart once enablement actually succeeded (persistent
|
||||
// toast; duration 0 = stays until dismissed).
|
||||
Promise.resolve(window.togglePlugin(pluginId, true)).then(toggleResult => {
|
||||
Promise.resolve(window.togglePlugin(installedId, true)).then(toggleResult => {
|
||||
if (toggleResult && toggleResult.status === 'success') {
|
||||
showNotification(
|
||||
`${pluginId} installed and enabled — restart the display to show it`,
|
||||
@@ -2532,9 +2521,6 @@ window.installPlugin = function(pluginId, branch = null) {
|
||||
);
|
||||
}
|
||||
});
|
||||
// Refresh installed plugins list, then re-render store to update badges
|
||||
loadInstalledPlugins().catch(() => {});
|
||||
setTimeout(() => applyStoreFiltersAndSort(true), 500);
|
||||
}
|
||||
|
||||
fetch('/api/v3/plugins/install', {
|
||||
@@ -2555,14 +2541,25 @@ window.installPlugin = function(pluginId, branch = null) {
|
||||
// live: "installation queued" followed immediately by a failed
|
||||
// enable). Wait for the operation to actually finish first.
|
||||
pollOperationStatus(data.data.operation_id, pluginId, pluginId, {
|
||||
onComplete: enableAfterInstall,
|
||||
onComplete: afterInstall,
|
||||
onFailed: (errorMsg) => showNotification(errorMsg || `Failed to install ${pluginId}`, 'error'),
|
||||
onTimeout: () => showNotification(`Install operation timed out for ${pluginId}`, 'error')
|
||||
maxAttempts: INSTALL_POLL_MAX_ATTEMPTS,
|
||||
// Out of patience is not a failure: the server may still be
|
||||
// installing. Show the list as it is now and say so; nothing
|
||||
// is enabled without the operation's answer.
|
||||
onTimeout: () => {
|
||||
showNotification(
|
||||
`${pluginId} is still installing — it will appear in the installed list when it finishes`,
|
||||
'warning'
|
||||
);
|
||||
loadInstalledPlugins(true).catch(() => {})
|
||||
.then(() => applyStoreFiltersAndSort(true));
|
||||
}
|
||||
});
|
||||
} else {
|
||||
// No operation queue configured - install already completed synchronously.
|
||||
window.noteRestartRequired(data);
|
||||
enableAfterInstall();
|
||||
afterInstall(data);
|
||||
}
|
||||
})
|
||||
.catch(error => {
|
||||
|
||||
@@ -884,7 +884,6 @@ With this off a live game takes over the whole display with the full-screen scor
|
||||
// Update brightness display
|
||||
document.getElementById('brightness').addEventListener('input', function() {
|
||||
document.getElementById('brightness-value').textContent = this.value;
|
||||
document.getElementById('brightness-display').textContent = this.value;
|
||||
});
|
||||
|
||||
|
||||
|
||||
@@ -404,6 +404,10 @@
|
||||
document.getElementById('web-login-new-token-value').textContent = res.d.data.token;
|
||||
document.getElementById('web-login-new-token').classList.remove('hidden');
|
||||
form.reset();
|
||||
// app.js marks a form dirty on input and clears the mark only
|
||||
// after an htmx save; this one posts with fetch, so clear it
|
||||
// here or a reload asks "Leave site?" about a saved token.
|
||||
form.removeAttribute('data-dirty');
|
||||
notify(res.d.message || 'Token created', 'success');
|
||||
}).catch(function(err) { notify('Request failed: ' + err.message, 'error'); });
|
||||
},
|
||||
|
||||
@@ -55,19 +55,40 @@
|
||||
<script>
|
||||
(function () {
|
||||
var DISMISS_KEY = 'ledmatrix-recon-dismissed';
|
||||
// Startup reconciliation is done within seconds of the web service
|
||||
// starting. The route also answers done: false when its status file is
|
||||
// missing (reconciliation raised before writing it, or /tmp was cleaned
|
||||
// under a long-running service), which used to keep this polling every
|
||||
// 2 s for as long as the page was open, on every tab. So: give up after
|
||||
// a minute, and poll only while the Overview is on screen.
|
||||
var POLL_MS = 2000;
|
||||
var MAX_POLLS = 30;
|
||||
var _recon_timer = null;
|
||||
var _polls = 0;
|
||||
var _finished = false; // done, given up, or dismissed
|
||||
var _active = false;
|
||||
var _inFlight = false;
|
||||
|
||||
function checkReconciliation() {
|
||||
_recon_timer = null;
|
||||
_inFlight = true;
|
||||
fetch('/api/v3/plugins/reconciliation-status')
|
||||
.then(function (r) { return r.json(); })
|
||||
.then(function (resp) {
|
||||
_inFlight = false;
|
||||
if (_finished) return;
|
||||
var d = resp.data || {};
|
||||
if (!d.done) {
|
||||
// Reconciliation still running — poll again shortly
|
||||
_recon_timer = setTimeout(checkReconciliation, 2000);
|
||||
// Reconciliation still running (or it never wrote its
|
||||
// status): ask again shortly, a bounded number of times.
|
||||
if (++_polls >= MAX_POLLS) {
|
||||
_finished = true;
|
||||
return;
|
||||
}
|
||||
if (_active) _recon_timer = setTimeout(checkReconciliation, POLL_MS);
|
||||
return;
|
||||
}
|
||||
_recon_timer = null;
|
||||
_finished = true;
|
||||
if (!d.unresolved || d.unresolved.length === 0) return;
|
||||
var key = d.unresolved.map(function (i) { return i.plugin_id; }).sort().join(',');
|
||||
if (sessionStorage.getItem(DISMISS_KEY) === key) return;
|
||||
@@ -102,13 +123,33 @@
|
||||
banner.dataset.dismissKey = key;
|
||||
banner.style.setProperty('display', 'flex', 'important');
|
||||
})
|
||||
.catch(function () {});
|
||||
.catch(function () { _inFlight = false; });
|
||||
}
|
||||
|
||||
function startReconciliationPoll() {
|
||||
_active = true;
|
||||
if (!_finished && _recon_timer === null && !_inFlight) checkReconciliation();
|
||||
}
|
||||
function stopReconciliationPoll() {
|
||||
_active = false;
|
||||
if (_recon_timer !== null) {
|
||||
clearTimeout(_recon_timer);
|
||||
_recon_timer = null;
|
||||
}
|
||||
}
|
||||
// Keyed apart from any other Overview registration, which a shared key
|
||||
// would replace.
|
||||
if (window.LEDVisibility) {
|
||||
window.LEDVisibility.onActive('overview', startReconciliationPoll,
|
||||
stopReconciliationPoll, 'overview-reconciliation');
|
||||
} else {
|
||||
startReconciliationPoll();
|
||||
}
|
||||
checkReconciliation();
|
||||
|
||||
window.dismissReconciliationBanner = function () {
|
||||
var banner = document.getElementById('reconciliation-banner');
|
||||
banner.style.setProperty('display', 'none', 'important');
|
||||
_finished = true;
|
||||
if (_recon_timer !== null) {
|
||||
clearTimeout(_recon_timer);
|
||||
_recon_timer = null;
|
||||
|
||||
@@ -250,13 +250,7 @@
|
||||
<input type="text" id="plugin-search" placeholder="Search plugins by name, description, or tags..." aria-label="Search the Plugin Store" class="form-control text-sm flex-[3] min-w-0 px-4 py-2.5 border border-gray-300 rounded-lg shadow-sm focus:shadow-md transition-shadow">
|
||||
<select id="plugin-category" aria-label="Filter the Plugin Store by category" class="form-control text-sm flex-1 px-3 py-2.5 border border-gray-300 rounded-lg shadow-sm focus:shadow-md transition-shadow">
|
||||
<option value="">All Categories</option>
|
||||
<option value="sports">Sports</option>
|
||||
<option value="content">Content</option>
|
||||
<option value="time">Time</option>
|
||||
<option value="weather">Weather</option>
|
||||
<option value="financial">Financial</option>
|
||||
<option value="media">Media</option>
|
||||
<option value="demo">Demo</option>
|
||||
<!-- The rest come from the store's plugins (plugins_manager.js, syncStoreCategoryOptions). -->
|
||||
</select>
|
||||
</div>
|
||||
|
||||
@@ -467,8 +461,9 @@
|
||||
<input type="text" id="github-plugin-url" aria-label="Plugin GitHub repository URL"
|
||||
placeholder="https://github.com/user/ledmatrix-plugin-name"
|
||||
class="flex-1 px-3 py-2 text-sm border border-gray-300 rounded-md focus:ring-blue-500 focus:border-blue-500">
|
||||
<button type="button" id="install-plugin-from-url"
|
||||
onclick="if(window.handleGitHubPluginInstall){window.handleGitHubPluginInstall()}else{alert('Function not loaded yet, please refresh the page')}"
|
||||
<!-- Wired by attachInstallButtonHandler (plugins_manager.js); an inline
|
||||
onclick here ran a second install handler on every click. -->
|
||||
<button type="button" id="install-plugin-from-url"
|
||||
class="px-4 py-2 bg-blue-600 hover:bg-blue-700 text-white text-sm rounded-md whitespace-nowrap">
|
||||
<i class="fas fa-download mr-2"></i>Install
|
||||
</button>
|
||||
|
||||
@@ -1093,6 +1093,12 @@
|
||||
Use TLS
|
||||
<span class="text-xs text-gray-500">(a password without TLS crosses the network in the clear)</span>
|
||||
</label>
|
||||
<label id="mqtt-allow-insecure-row" class="${c.mqtt_tls ? 'hidden' : 'flex'} items-center gap-2 text-sm text-gray-700">
|
||||
<input id="mqtt-allow-insecure" type="checkbox" ${c.allow_insecure_mqtt ? 'checked' : ''}
|
||||
class="rounded border-gray-300">
|
||||
Allow without TLS (trusted network)
|
||||
<span class="text-xs text-gray-500">(needed to save a password while TLS is off)</span>
|
||||
</label>
|
||||
|
||||
<div class="flex items-center justify-between gap-4 pt-2">
|
||||
<p class="text-xs text-gray-500">
|
||||
@@ -1132,6 +1138,15 @@
|
||||
if (clearBtn) clearBtn.addEventListener('click', () => clearMqttPassword());
|
||||
const clearTokenBtn = document.getElementById('mqtt-clear-api-token');
|
||||
if (clearTokenBtn) clearTokenBtn.addEventListener('click', () => clearMqttApiToken());
|
||||
// The cleartext opt-in only means something while TLS is off.
|
||||
const tlsBox = document.getElementById('mqtt-tls');
|
||||
const allowRow = document.getElementById('mqtt-allow-insecure-row');
|
||||
if (tlsBox && allowRow) {
|
||||
tlsBox.addEventListener('change', () => {
|
||||
allowRow.classList.toggle('hidden', tlsBox.checked);
|
||||
allowRow.classList.toggle('flex', !tlsBox.checked);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
window.loadMqttBridge = function() {
|
||||
@@ -1160,6 +1175,9 @@
|
||||
on_demand_duration: val('mqtt-duration') === '' ? null : val('mqtt-duration'),
|
||||
log_level: val('mqtt-log-level'),
|
||||
mqtt_tls: !!(document.getElementById('mqtt-tls') || {}).checked,
|
||||
// The server refuses a password with TLS off unless this is set
|
||||
// (CWE-319); it is off until the user ticks it.
|
||||
allow_insecure_mqtt: !!(document.getElementById('mqtt-allow-insecure') || {}).checked,
|
||||
};
|
||||
// Only send a password when one was typed; blank means "leave it alone".
|
||||
const pw = val('mqtt-password');
|
||||
|
||||
Reference in New Issue
Block a user