mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-13 10:36:35 +00:00
Compare commits
4
Commits
d01da3bd9f
...
fddb0e06db
| Author | SHA1 | Date | |
|---|---|---|---|
 ChuckandClaude Opus 5
|
fddb0e06db | ||
|
ChuckandClaude Opus 5
|
8360220809 | ||
|
ChuckandClaude Opus 5
|
9e3f184d81 | ||
|
ChuckandClaude Opus 5
|
869e36fb2f |
+167
-2
@@ -19,6 +19,35 @@ accepts both, but the store flags the old spelling as deprecated
|
||||
|
||||
## Unreleased
|
||||
|
||||
New module a plugin may import via `src.*` (floor on the release that ships
|
||||
this):
|
||||
|
||||
- `src/common/sports_helpers.py` — the helpers the scoreboards' `sports.py`
|
||||
carry byte-identical copies of: `clamp_window`, `clamp_seconds`,
|
||||
`logo_needs_refresh`, `spread_weighted_order` (+ `MIN_WINDOW_DAYS`,
|
||||
`MAX_WINDOW_DAYS`), and `SportsHelpersMixin` with `_mode_customization`,
|
||||
`_setting_int`, `_reset_dwell_on_reentry`, `_next_switch_index`,
|
||||
`_spread_weighted_order`, `_odds_color`, `_upcoming_date_and_time_text` under
|
||||
the plugins' names and signatures, plus the `_favorite_key` override point.
|
||||
Constructor-free; keeps lazy state on its host (see the module docstring,
|
||||
which also gives the host contract).
|
||||
A new module rather than more methods on `sports_shared`: a plugin that
|
||||
deletes a copy and leans on an older module having grown the method fails at
|
||||
runtime with `AttributeError`, which no load-time check sees, while a missing
|
||||
module fails at load. Nothing in core uses it yet.
|
||||
- `test/test_common_is_hardware_free.py` — `src/common` must import without
|
||||
`rgbmatrix` and never import `src.base_classes`, `src.display_manager` or
|
||||
`src.plugin_system` at module level.
|
||||
|
||||
Web interface:
|
||||
|
||||
- The plugin settings form honours `"x-display": "hidden"` in config schemas:
|
||||
the property gets no control at any depth (top level, nested objects, array
|
||||
rows, Advanced Settings), and saving the form never changes its stored value.
|
||||
JSON API saves are unaffected. Lets plugins keep deprecated or internal keys
|
||||
declared, e.g. countdown's row `id` and weather's `api_key` / `radar_zoom`.
|
||||
See `docs/widget-guide.md`.
|
||||
|
||||
## 3.4.0
|
||||
|
||||
Plugin-facing changes since 3.3.0 (tag `v3.3.1`) not covered further down:
|
||||
@@ -41,7 +70,143 @@ Plugin-facing changes since 3.3.0 (tag `v3.3.1`) not covered further down:
|
||||
- **Web preview size** now comes from `src/display_geometry.py`, the same
|
||||
computation `DisplayManager` uses: double-sided setups preview one screen,
|
||||
and a missing `chain_length` defaults to 2 everywhere (the Starlark magnify
|
||||
default and the sync handshake used 1).
|
||||
default and the sync handshake used 1). The module is core-internal: plugins
|
||||
keep reading `display_manager.width`/`height`.
|
||||
- `src.common.font_layout` (#539, #565) — `load_truetype()` is
|
||||
`ImageFont.truetype` with the layout engine pinned, so text lays out the same
|
||||
whether or not the host's Pillow was built with libraqm; `crisp_size()` and
|
||||
`FONT_PIXEL_GRID` give the size a bundled face renders on whole pixels at
|
||||
(`sports_card` still re-exports them); `resolve_asset_path()` resolves
|
||||
`assets/fonts/...` against the install root, not the working directory.
|
||||
Floor on 3.4.0 to import it. Relatedly, `DisplayManager` now draws text
|
||||
1-bit (#521), so golden images recorded against 3.3.x may need regenerating.
|
||||
|
||||
### Install and updates
|
||||
|
||||
**Weekly automatic updates (#581), off by default.** Switching on
|
||||
*Automatically check for and install updates once a week* on the General tab
|
||||
(or `first_time_install.sh --enable-auto-update` / `LEDMATRIX_AUTO_UPDATE=1`)
|
||||
updates the core and then every installed plugin once a week, preferably 2–5 AM
|
||||
local time. It follows the branch the checkout tracks — `main` on a standard
|
||||
install — so a device gets whatever has merged there, not only tagged releases.
|
||||
See `docs/WEB_INTERFACE_GUIDE.md`.
|
||||
|
||||
- The core step is skipped, with the reason shown, when the checkout has local
|
||||
edits or commits, a rebase or merge is in progress, the branch has no
|
||||
upstream, less than 300 MB is free, or that commit was already rolled back.
|
||||
- After pulling, `ledmatrix-update-verify.service` restarts the services and
|
||||
requires the web interface to answer and the display to stay up. If they
|
||||
don't, or the new requirements fail to install, it resets to the previous
|
||||
commit, reinstalls its requirements and restarts again. Anything but success
|
||||
shows under the toggle and as a banner on Overview.
|
||||
- Plugins update through the Plugin Store even when the core step is skipped,
|
||||
fails or is rolled back. A plugin version whose `ledmatrix_min_version` is
|
||||
above the device's core is held back, not installed. When the core did
|
||||
update, plugins wait for its health check, and are left alone if that check
|
||||
never reports or the rollback fails.
|
||||
- No SSH is needed: switching the toggle on restarts the display service, which
|
||||
installs the health-check units (`src/auto_update_setup.py`, core-internal
|
||||
and not a plugin API).
|
||||
|
||||
Installer and service fixes:
|
||||
|
||||
- rgbmatrix builds on ARMv6 boards (Pi Zero, Pi 1); an existing checkout is
|
||||
moved forward to the new pin and no longer left root-owned (#577).
|
||||
- `first_time_install.sh` grants the web user `safe_pip_install.sh`, as
|
||||
`configure_web_sudo.sh` already did, so plugin requirements install where
|
||||
the display service can see them (#579).
|
||||
- The web interface starts when `web_display_autostart` is missing or
|
||||
`config.json` is unreadable; only an explicit `false` keeps it down (#556).
|
||||
- Installers render every systemd unit from its `systemd/` template, so the
|
||||
boot-time unit-drift warning can clear, non-root installs included (#547).
|
||||
|
||||
### Scrolling
|
||||
|
||||
- **Frame pacing (#523).** The loop waits only for the rest of each panel
|
||||
refresh instead of a flat 8 ms: 44–46 fps → 100 fps, and slow frames 14% →
|
||||
0.02%, on a 2×128×64 chain. Sub-pixel blending is off by default again (it
|
||||
shimmered on pixel fonts; Vegas mode still opts in).
|
||||
- **Whole-pixel steps (#545).** At a speed `scroll_config` can render in whole
|
||||
pixels, every frame advances by exactly the same amount, removing about six
|
||||
hitches a second. A loop that can't keep up now scrolls slightly slow rather
|
||||
than jumping.
|
||||
- The eight sports scoreboards scroll through `scroll_config` too (#542): the
|
||||
default 50 px/s holds each frame for two refreshes instead of alternating
|
||||
0 px and 1 px steps.
|
||||
- **Frame stats ignore the pause between scrolls (#582).** The `Scroll frame
|
||||
stats` log line counted the idle wait before each scroll as one frame,
|
||||
inflating `max` and the stall rate. `docs/SCROLL_PERFORMANCE.md` now
|
||||
describes the line actually logged.
|
||||
|
||||
### Plugins
|
||||
|
||||
- `FontManager` registers the bundled `tom_thumb` font, so plugins no longer
|
||||
need a private loader (#534).
|
||||
- The test harness's `set_scrolling_state()` accepts `frame_hold`, as
|
||||
`DisplayManager`'s does (#534).
|
||||
- A `display()` with nothing to draw should return `False`, the only value the
|
||||
controller skips on; starlark-apps now does, rather than holding a black
|
||||
panel (#534).
|
||||
- Starlark apps may set `render_width`/`render_height` in their `config.json`
|
||||
to render at their own canvas size instead of Pixlet's 64×32 (#552).
|
||||
- `scripts/render_plugin.py --display-mode <mode>` renders one mode of a
|
||||
multi-mode plugin; scoreboards previously rendered blank (#522).
|
||||
- Scoreboards resolve their own directory under the real plugin loader
|
||||
(declare `_PLUGIN_DIR`), so 4x6 text snaps to its 7px grid instead of
|
||||
rendering a pixel narrow, and an unreadable schema is logged (#519, #520).
|
||||
`DisplayManager` loads 4x6 on that grid too (#565).
|
||||
- The 5x7 BDF face reports a real height, so rows stacked by
|
||||
`get_font_height()` no longer overlap (#539).
|
||||
- `LogoHelper` remembers a missing logo instead of warning every rotation
|
||||
(#548), and the decoded sports logo cache is bounded (#559).
|
||||
|
||||
### Web interface
|
||||
|
||||
- Installed Plugins has search, All / Enabled / Disabled / Updates filters and
|
||||
sort (#540).
|
||||
- Hardened and polished per the September 2026 audit (#568): utility classes
|
||||
such as `.hidden` actually exist, focus rings, labels and modal focus
|
||||
trapping, dark theme throughout, no overflow at phone width, and background
|
||||
streams pause when hidden, with first-load JS/CSS down from 1358 KB to 291 KB.
|
||||
- WiFi Connect works from the LEDMatrix-Setup hotspot: the page is answered
|
||||
before the hotspot drops, and reopening it shows why an attempt failed (#571).
|
||||
- Pixlet install, the Starlark app store and app toggles work again (#535,
|
||||
#537); the store uses the configured GitHub token and reports a rate limit
|
||||
instead of drawing a blank grid (#541).
|
||||
- Plugin config: geochron and news saves no longer always fail (#575), the page
|
||||
survives stored values the schema outgrew (#578), the form uses the full page
|
||||
height (#573), and file-manager widgets show the script's error (#574).
|
||||
- The live status stream reports real disk usage and available memory (#558);
|
||||
a system action refused for want of passwordless sudo says so and names
|
||||
`configure_web_sudo.sh` (#560).
|
||||
|
||||
### Tools and security
|
||||
|
||||
- **CodeQL triage (#561):** 129 of 134 alerts fixed. Three were exploitable
|
||||
path-handling flaws in the web interface and are closed; web UI escapers now
|
||||
escape quotes, and URL fields refuse script schemes. Path checks share
|
||||
`src/common/path_safety.py` (core-internal).
|
||||
- **Home Assistant MQTT bridge** (`integrations/mqtt_bridge`, #538): mode
|
||||
select, stop, power and brightness over MQTT Discovery.
|
||||
- **Tools tab** manages the MQTT bridge and the Pixlet editor (#554); the
|
||||
editor stays on loopback when `PIXLET_EDITOR_HOST` says so.
|
||||
|
||||
### Fixes
|
||||
|
||||
- Updating a plugin whose directory is named for its manifest id (leaderboard,
|
||||
music, stocks, weather) silently did nothing (#536).
|
||||
- Plugin reconciliation no longer reports working plugins as stale or replaces
|
||||
their config with a stub, and the Overview banner advises each case correctly
|
||||
(#557).
|
||||
- Two config saves in the same second no longer share one backup, so rollback
|
||||
restores the version asked for (#564).
|
||||
- On-demand: a second request is honoured without a restart (#534), a pinned
|
||||
request stays on its mode, and restarting mid-session loads every plugin
|
||||
again (#538).
|
||||
- `/health` and `/display/current` report real state, and the preview no longer
|
||||
freezes on a leftover snapshot temp file (#534).
|
||||
|
||||
### Per-element display customization
|
||||
|
||||
**Per-element display customization, and the last mile of it into the web UI.**
|
||||
A user can set the font, size, colour, position, visibility and alignment of
|
||||
@@ -133,7 +298,7 @@ Removed:
|
||||
|
||||
## 3.3.0
|
||||
|
||||
Historical note: tag `v3.3.0` reports `__version__` "3.2.0" and tag `v3.3.1` reports "3.3.0", so a "3.3.0" floor is effectively `v3.3.1`, the first release shipping `src/common/sports_shared.py`.
|
||||
Historical note: tags `v3.3.0` and `v3.3.1` both report `__version__` "3.3.0" and both ship `src/common/sports_shared.py`, so a "3.3.0" floor always means a core with `sports_shared`.
|
||||
|
||||
**The release the sports scoreboards floor on to delete their bundled copies.**
|
||||
3.2.0 shipped the unified sports library and made `ledmatrix_min_version`
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
{
|
||||
"web_display_autostart": true,
|
||||
"auto_update": {
|
||||
"enabled": false
|
||||
},
|
||||
"schedule": {
|
||||
"enabled": false,
|
||||
"mode": "per-day",
|
||||
|
||||
@@ -80,11 +80,30 @@ src/base_classes/sports/
|
||||
src/common/
|
||||
sports_scroll.py SportsScrollDisplay / …Manager — scroll orchestration
|
||||
(content building stays in the plugins)
|
||||
sports_helpers.py clamp/logo/rotation free functions + SportsHelpersMixin
|
||||
(unreleased) — the helpers byte-identical in the
|
||||
plugins' sports.py, and the _favorite_key seam
|
||||
```
|
||||
|
||||
`from src.base_classes.sports import SportsCore` keeps working — the package
|
||||
`__init__` re-exports, so the conversion is invisible to every existing importer.
|
||||
|
||||
### Converging on `src/common`
|
||||
|
||||
The scoreboards do not build on `src/base_classes`; their own `sports.py` copies
|
||||
have moved past it. So shared code now lands in hardware-free `src/common`
|
||||
modules taken from the plugin copies, each a **new module** rather than growth
|
||||
on an existing one: a plugin that deletes a method copy and relies on an older
|
||||
module having gained it fails at runtime with an `AttributeError`, while a
|
||||
missing module fails at load, where the version checks can see it.
|
||||
`sports_helpers.py` is the first (it holds `_favorite_key`, the override point
|
||||
listed below, for later phases); its parity test compares every body against
|
||||
the plugin copies when `LEDMATRIX_PLUGINS` points at a checkout, and
|
||||
`test/test_common_is_hardware_free.py` keeps `src/common` free of
|
||||
`rgbmatrix`, `src.base_classes` and `src.plugin_system`. How a plugin adopts a
|
||||
module and drops its copy is documented in the plugins repo's
|
||||
`docs/plugin-development/08-shared-sports-code.md`.
|
||||
|
||||
## Override points (the plugin-facing seam)
|
||||
|
||||
The base class calls these; plugins implement or override them. This table is the
|
||||
|
||||
@@ -96,6 +96,33 @@ Configure basic system settings:
|
||||
- **Plugin System Settings** — including the `plugins_directory` (default
|
||||
`plugin-repos/`) used by the plugin loader
|
||||
- **Autostart** options for the display service
|
||||
- **Automatic updates** — once a week, update LEDMatrix and every installed
|
||||
plugin with a newer version. Off by default. Runs 2–5 AM local time when
|
||||
possible, otherwise within a day of being due. The last result and next
|
||||
check are shown under the toggle, and anything other than success raises a
|
||||
banner on **Overview**.
|
||||
- *Checks first:* the code update is skipped, with the reason shown, if
|
||||
tracked files were edited locally, the checkout has local commits, a
|
||||
rebase/merge is in progress, the branch has no upstream, less than 300 MB
|
||||
is free, or the newest version already failed once. A failed fetch is
|
||||
retried the next day.
|
||||
- *Health check and rollback:* after pulling, `ledmatrix-update-verify.service`
|
||||
restarts the services and checks that the web interface responds and the
|
||||
display (if it was running) stays up. If not — or if the new dependencies
|
||||
failed to install — it resets to the previous commit, reinstalls the
|
||||
previous dependencies and restarts again. A running display is restarted;
|
||||
a stopped one stays stopped.
|
||||
- *Plugins* update through the Plugin Store, which refuses versions that need
|
||||
a newer LEDMatrix and restores the old copy when an install fails. When the
|
||||
code changed, plugins wait until it passes its health check. Plugins are
|
||||
not health-checked after updating.
|
||||
- *Setup needs no SSH.* Turning the toggle on restarts the display service,
|
||||
which installs the health check (`ledmatrix-update-verify.path` and
|
||||
`.service`); the General tab shows when it is ready, or why setup failed.
|
||||
Until then only plugins update. New installs set it up during
|
||||
installation and can switch updates on with
|
||||
`first_time_install.sh --enable-auto-update` (or `LEDMATRIX_AUTO_UPDATE=1`,
|
||||
which `one-shot-install.sh` passes through), or at the installer's prompt.
|
||||
|
||||
Click **Save** to write changes to `config/config.json`. Most changes
|
||||
require a display service restart from **Overview**.
|
||||
|
||||
@@ -281,6 +281,42 @@ Guidelines:
|
||||
their own collapsible sections) and is safely ignored by older cores, so
|
||||
adding it never breaks compatibility.
|
||||
|
||||
## Hiding Fields From the Form (`x-display: "hidden"`)
|
||||
|
||||
Add `"x-display": "hidden"` to a property that must stay in the schema but
|
||||
should not appear as a control: a deprecated key kept so existing configs keep
|
||||
validating, or an internal value such as an auto-generated row id.
|
||||
|
||||
```json
|
||||
{
|
||||
"properties": {
|
||||
"radar_zoom": {
|
||||
"type": "integer",
|
||||
"default": 6,
|
||||
"title": "Radar Zoom Level (deprecated)",
|
||||
"x-display": "hidden"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
What the core does with it:
|
||||
|
||||
- **Not rendered** at any depth: top-level fields, children of an object
|
||||
section, and properties of array-of-object items (never a table column, even
|
||||
if `x-columns` names it, and never in the row editor). A hidden field flagged
|
||||
`x-advanced` is not listed or counted in Advanced Settings, and an object
|
||||
whose children are all hidden draws no empty section. Hidden fields don't
|
||||
show up in the settings search either, since it indexes the rendered form.
|
||||
- **Stored value preserved on save.** Saving the form never changes a hidden
|
||||
value. The unchecked-checkbox rule ignores a hidden boolean. Array rows carry
|
||||
a hidden property's stored value through the form, so the value survives the
|
||||
row being posted back; a new row gets no value (the plugin fills it in).
|
||||
- **The API is unaffected.** A JSON save to `POST /api/v3/plugins/config` can
|
||||
still set a hidden field.
|
||||
|
||||
Older cores ignore the flag and render the field as a normal control.
|
||||
|
||||
## Creating Custom Widgets
|
||||
|
||||
### Step 1: Create Widget File
|
||||
|
||||
+54
-3
@@ -220,6 +220,8 @@ SKIP_PERF=${LEDMATRIX_SKIP_PERF:-0}
|
||||
SKIP_REBOOT_PROMPT=${LEDMATRIX_SKIP_REBOOT_PROMPT:-0}
|
||||
SKIP_SWAP=${LEDMATRIX_SKIP_SWAP:-0}
|
||||
BUILD_JOBS_OVERRIDE=${LEDMATRIX_BUILD_JOBS:-}
|
||||
# Weekly automatic updates: 1 on, 0 off, empty = ask (interactive) or leave as is.
|
||||
AUTO_UPDATE=${LEDMATRIX_AUTO_UPDATE:-}
|
||||
|
||||
usage() {
|
||||
cat <<USAGE
|
||||
@@ -234,12 +236,15 @@ Options:
|
||||
--skip-swap Never add temporary swap for the C++ build
|
||||
--build-jobs N Compile the C++ library with N parallel jobs
|
||||
(default: scaled to available RAM)
|
||||
--enable-auto-update Turn on weekly automatic updates (with health
|
||||
check and automatic rollback)
|
||||
--no-auto-update Leave weekly automatic updates off
|
||||
-h, --help Show this help message and exit
|
||||
|
||||
Environment variables (same effect as flags):
|
||||
LEDMATRIX_ASSUME_YES=1, RPI_RGB_FORCE_REBUILD=1, LEDMATRIX_SKIP_SOUND=1,
|
||||
LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1,
|
||||
LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N
|
||||
LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N, LEDMATRIX_AUTO_UPDATE=1|0
|
||||
|
||||
Low-memory devices:
|
||||
On a Pi with under 2GB of RAM the C++ build is limited to fewer parallel
|
||||
@@ -257,6 +262,8 @@ while [ $# -gt 0 ]; do
|
||||
--skip-perf) SKIP_PERF=1 ;;
|
||||
--no-reboot-prompt) SKIP_REBOOT_PROMPT=1 ;;
|
||||
--skip-swap) SKIP_SWAP=1 ;;
|
||||
--enable-auto-update) AUTO_UPDATE=1 ;;
|
||||
--no-auto-update) AUTO_UPDATE=0 ;;
|
||||
--build-jobs)
|
||||
shift
|
||||
if [ $# -eq 0 ]; then echo "--build-jobs requires a number"; usage; exit 1; fi
|
||||
@@ -863,6 +870,50 @@ else
|
||||
echo "✓ Main config file already exists"
|
||||
fi
|
||||
|
||||
# Weekly automatic updates (General tab -> Automatic Updates). Off unless asked
|
||||
# for: --enable-auto-update / LEDMATRIX_AUTO_UPDATE=1, or "y" at the prompt when
|
||||
# installing interactively. Only an explicit choice changes the setting, so
|
||||
# re-running the installer with -y never switches it silently.
|
||||
if [ -z "$AUTO_UPDATE" ] && [ "$ASSUME_YES" != "1" ] && [ -t 0 ]; then
|
||||
read -p "Automatically check for and install LEDMatrix updates once a week, with automatic rollback if an update breaks something? (y/N): " -n 1 -r
|
||||
echo
|
||||
if [[ $REPLY =~ ^[Yy]$ ]]; then AUTO_UPDATE=1; else AUTO_UPDATE=0; fi
|
||||
fi
|
||||
if [ "$AUTO_UPDATE" = "1" ] || [ "$AUTO_UPDATE" = "0" ]; then
|
||||
if python3 - "$PROJECT_ROOT_DIR/config/config.json" "$AUTO_UPDATE" <<'PY'
|
||||
import json, os, sys, tempfile
|
||||
path, enabled = sys.argv[1], sys.argv[2] == "1"
|
||||
with open(path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
if not isinstance(config.get("auto_update"), dict):
|
||||
config["auto_update"] = {}
|
||||
config["auto_update"]["enabled"] = enabled
|
||||
# Written beside the original and swapped in whole: the display service's
|
||||
# config watcher may be running and must never read a half-written file.
|
||||
original = os.stat(path)
|
||||
fd, tmp = tempfile.mkstemp(dir=os.path.dirname(os.path.abspath(path)), prefix=".config.")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
json.dump(config, f, indent=4)
|
||||
f.write("\n")
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.chmod(tmp, original.st_mode & 0o777)
|
||||
if hasattr(os, "chown"):
|
||||
os.chown(tmp, original.st_uid, original.st_gid)
|
||||
os.replace(tmp, path)
|
||||
except BaseException:
|
||||
if os.path.exists(tmp):
|
||||
os.unlink(tmp)
|
||||
raise
|
||||
PY
|
||||
then
|
||||
if [ "$AUTO_UPDATE" = "1" ]; then echo "✓ Weekly automatic updates enabled"; else echo "✓ Weekly automatic updates off"; fi
|
||||
else
|
||||
echo "⚠ Could not set auto_update in config/config.json; turn it on from the General tab instead"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Create config_secrets.json from template if missing
|
||||
if [ ! -f "$PROJECT_ROOT_DIR/config/config_secrets.json" ]; then
|
||||
if [ -f "$PROJECT_ROOT_DIR/config/config_secrets.template.json" ]; then
|
||||
@@ -1341,7 +1392,7 @@ if [ -f "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh" ]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -f "/etc/systemd/system/ledmatrix-web.service" ] || [ "$NEEDS_UPDATE" = true ]; then
|
||||
if [ ! -f "/etc/systemd/system/ledmatrix-web.service" ] || [ ! -f "/etc/systemd/system/ledmatrix-update-verify.path" ] || [ "$NEEDS_UPDATE" = true ]; then
|
||||
bash "$PROJECT_ROOT_DIR/scripts/install/install_web_service.sh"
|
||||
# Ensure systemd sees any new/changed unit files
|
||||
systemctl daemon-reload || true
|
||||
@@ -1357,7 +1408,7 @@ echo ""
|
||||
CURRENT_STEP="Harden systemd unit file permissions"
|
||||
echo "Step 8.1: Setting systemd unit file permissions..."
|
||||
echo "-----------------------------------------------"
|
||||
for unit in "/etc/systemd/system/ledmatrix.service" "/etc/systemd/system/ledmatrix-web.service" "/etc/systemd/system/ledmatrix-wifi-monitor.service"; do
|
||||
for unit in "/etc/systemd/system/ledmatrix.service" "/etc/systemd/system/ledmatrix-web.service" "/etc/systemd/system/ledmatrix-wifi-monitor.service" "/etc/systemd/system/ledmatrix-update-verify.service" "/etc/systemd/system/ledmatrix-update-verify.path"; do
|
||||
if [ -f "$unit" ]; then
|
||||
chown root:root "$unit" || true
|
||||
chmod 644 "$unit" || true
|
||||
|
||||
@@ -95,6 +95,19 @@ else
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Health check / rollback units for automatic updates; see install_web_service.sh.
|
||||
for VERIFY_UNIT in ledmatrix-update-verify.service ledmatrix-update-verify.path; do
|
||||
if [ -f "$PROJECT_ROOT_DIR/systemd/$VERIFY_UNIT" ]; then
|
||||
VERIFY_UNIT_TMP=$(mktemp)
|
||||
if sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|$ESCAPED_ACTUAL_USER|g" "$PROJECT_ROOT_DIR/systemd/$VERIFY_UNIT" > "$VERIFY_UNIT_TMP"; then
|
||||
sudo cp "$VERIFY_UNIT_TMP" "/etc/systemd/system/$VERIFY_UNIT"
|
||||
else
|
||||
echo "WARNING: failed to render $VERIFY_UNIT; automatic code updates will stay paused." >&2
|
||||
fi
|
||||
rm -f "$VERIFY_UNIT_TMP"
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Reloading systemd daemon for web service..."
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
@@ -102,6 +115,11 @@ if [ -f "/etc/systemd/system/ledmatrix-web.service" ]; then
|
||||
echo "Enabling ledmatrix-web.service to start on boot..."
|
||||
sudo systemctl enable ledmatrix-web.service
|
||||
|
||||
if [ -f /etc/systemd/system/ledmatrix-update-verify.path ]; then
|
||||
echo "Enabling ledmatrix-update-verify.path (automatic update health check)..."
|
||||
sudo systemctl enable --now ledmatrix-update-verify.path || echo "WARNING: could not enable ledmatrix-update-verify.path; automatic code updates will stay paused" >&2
|
||||
fi
|
||||
|
||||
echo "Starting ledmatrix-web.service..."
|
||||
sudo systemctl start ledmatrix-web.service
|
||||
|
||||
|
||||
@@ -46,6 +46,22 @@ ESCAPED_ACTUAL_USER=$(sed_escape_replacement "$ACTUAL_USER")
|
||||
sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|$ESCAPED_ACTUAL_USER|g" \
|
||||
"$TEMPLATE" > /etc/systemd/system/ledmatrix-web.service
|
||||
|
||||
# Health check and rollback for the web UI's automatic updates. Its own unit so
|
||||
# it survives the web service restart it performs; never enabled -- the web
|
||||
# interface starts it after an update. Without it, automatic code updates
|
||||
# stay paused rather than running with nothing to undo them.
|
||||
for VERIFY_UNIT in ledmatrix-update-verify.service ledmatrix-update-verify.path; do
|
||||
VERIFY_TEMPLATE="$PROJECT_ROOT_DIR/systemd/$VERIFY_UNIT"
|
||||
if [ -f "$VERIFY_TEMPLATE" ]; then
|
||||
echo "Writing unit file to /etc/systemd/system/$VERIFY_UNIT"
|
||||
sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g; s|__USER__|$ESCAPED_ACTUAL_USER|g" \
|
||||
"$VERIFY_TEMPLATE" > "/etc/systemd/system/$VERIFY_UNIT"
|
||||
chmod 644 "/etc/systemd/system/$VERIFY_UNIT"
|
||||
else
|
||||
echo "WARNING: $VERIFY_TEMPLATE not found; automatic code updates will stay paused"
|
||||
fi
|
||||
done
|
||||
|
||||
# Ensure cache directory exists with proper permissions
|
||||
# This is a fallback for older systemd versions that don't support CacheDirectory
|
||||
# Systemd 239+ will automatically create it via CacheDirectory directive
|
||||
@@ -80,6 +96,13 @@ systemctl daemon-reload
|
||||
echo "Enabling ledmatrix-web.service..."
|
||||
systemctl enable ledmatrix-web.service
|
||||
|
||||
# The path unit is what starts the health check after an automatic update.
|
||||
if [ -f /etc/systemd/system/ledmatrix-update-verify.path ]; then
|
||||
echo "Enabling ledmatrix-update-verify.path..."
|
||||
systemctl enable --now ledmatrix-update-verify.path || \
|
||||
echo "WARNING: could not enable ledmatrix-update-verify.path; automatic code updates will stay paused"
|
||||
fi
|
||||
|
||||
# Start the service
|
||||
echo "Starting ledmatrix-web.service..."
|
||||
systemctl start ledmatrix-web.service
|
||||
|
||||
@@ -408,6 +408,7 @@ main() {
|
||||
# which would silently reinstate the duplicate apt update.
|
||||
sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 \
|
||||
LEDMATRIX_APT_UPDATED="${LEDMATRIX_APT_UPDATED:-0}" \
|
||||
LEDMATRIX_AUTO_UPDATE="${LEDMATRIX_AUTO_UPDATE:-}" \
|
||||
bash ./first_time_install.sh -y </dev/null
|
||||
fi
|
||||
INSTALL_EXIT_CODE=$?
|
||||
|
||||
@@ -0,0 +1,277 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check that an automatic LEDMatrix update left the device working; roll it back if not.
|
||||
|
||||
Started by the web interface's weekly updater (web_interface/auto_update.py)
|
||||
through ledmatrix-update-verify.service, right after it pulls new code. It has
|
||||
to run outside the web service: checking the update means restarting that
|
||||
service, and a check running inside it would be killed by its own restart.
|
||||
|
||||
It must not be the code it is checking, either. The updater copies this file
|
||||
to data/auto_update_verifier.py *before* pulling and the unit runs that copy,
|
||||
so a broken update cannot break its own rollback. Standard library only for
|
||||
the same reason: the rollback cannot depend on packages the update changed.
|
||||
|
||||
The updater leaves data/auto_update_pending.json:
|
||||
|
||||
{"status": "pending", "old_head": ..., "new_head": ...,
|
||||
"display_was_active": bool, "dependency_failures": [...], "created_at": ...}
|
||||
|
||||
This moves its status to "verifying" and then to one of "success",
|
||||
"rolled_back" or "rollback_failed", with "reason" and "detail" saying why.
|
||||
The web interface reports that outcome and raises a banner for anything but
|
||||
success.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
import sys
|
||||
from collections import namedtuple
|
||||
import tempfile
|
||||
import time
|
||||
import traceback
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
PENDING_NAME = 'auto_update_pending.json'
|
||||
REQUIREMENT_FILES = ('requirements.txt', 'web_interface/requirements.txt')
|
||||
WEB_HEALTH_URL = 'http://127.0.0.1:5000/api/v3/system/version'
|
||||
#: How long the services get to come up after a restart...
|
||||
HEALTH_TIMEOUT_SECONDS = 180
|
||||
#: ...and how long they must then stay up. Restart=on-failure makes a crash
|
||||
#: loop look healthy between attempts, so a single "is-active" proves nothing.
|
||||
STABLE_SECONDS = 45
|
||||
POLL_SECONDS = 5
|
||||
PIP_TIMEOUT_SECONDS = 600
|
||||
#: sudoers matches the exact command line, so bash is named by path, the same
|
||||
#: candidates src/common/permission_utils.install_requirements_file tries.
|
||||
BASH_CANDIDATES = ('/usr/bin/bash', '/bin/bash')
|
||||
|
||||
#: What a command that could not run at all reports: its callers only read
|
||||
#: these three fields, the same ones a completed subprocess has.
|
||||
_Failed = namedtuple('_Failed', 'returncode stdout stderr')
|
||||
|
||||
|
||||
def pending_path(project_root):
|
||||
return Path(project_root) / 'data' / PENDING_NAME
|
||||
|
||||
|
||||
def read_pending(path):
|
||||
try:
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
return data if isinstance(data, dict) else None
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def write_pending(path, data):
|
||||
path = Path(path)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix='.auto_update_pending_')
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, indent=2)
|
||||
os.replace(tmp, path)
|
||||
except BaseException:
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
def _web_responds(url=WEB_HEALTH_URL):
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=5) as resp: # nosec B310 - fixed loopback URL
|
||||
return resp.status == 200
|
||||
except (urllib.error.URLError, OSError, ValueError):
|
||||
return False
|
||||
|
||||
|
||||
def _short(sha):
|
||||
return (sha or 'unknown')[:7]
|
||||
|
||||
|
||||
class Verifier:
|
||||
def __init__(self, project_root, run=subprocess.run, sleep=time.sleep,
|
||||
clock=time.monotonic, web_responds=_web_responds, log=None):
|
||||
self.project_root = Path(project_root)
|
||||
self.pending_file = pending_path(project_root)
|
||||
self.run = run
|
||||
self.sleep = sleep
|
||||
self.clock = clock
|
||||
self.web_responds = web_responds
|
||||
self.log = log or (lambda msg: print(f'[auto-update-verify] {msg}', flush=True))
|
||||
|
||||
def _run(self, args, timeout=60):
|
||||
try:
|
||||
return self.run(args, cwd=str(self.project_root), capture_output=True,
|
||||
text=True, timeout=timeout)
|
||||
except (subprocess.SubprocessError, OSError) as e:
|
||||
return _Failed(returncode=1, stdout='', stderr=str(e))
|
||||
|
||||
# -- services ---------------------------------------------------------
|
||||
|
||||
def service_active(self, unit):
|
||||
return self._run(['systemctl', 'is-active', unit], timeout=10).stdout.strip() == 'active'
|
||||
|
||||
def restart_count(self, unit):
|
||||
out = self._run(['systemctl', 'show', '-p', 'NRestarts', '--value', unit],
|
||||
timeout=10).stdout.strip()
|
||||
return int(out) if out.isdigit() else None
|
||||
|
||||
def restart(self, unit):
|
||||
result = self._run(['sudo', '-n', 'systemctl', 'restart', f'{unit}.service'], timeout=90)
|
||||
if result.returncode != 0:
|
||||
self.log(f'restarting {unit} failed: {(result.stderr or "").strip()}')
|
||||
return result.returncode == 0
|
||||
|
||||
def restart_services(self, display):
|
||||
"""Restart what should be running. False if any restart command failed."""
|
||||
ok = True
|
||||
# A display the user had stopped stays stopped.
|
||||
if display:
|
||||
ok = self.restart('ledmatrix') and ok
|
||||
return self.restart('ledmatrix-web') and ok
|
||||
|
||||
def wait_healthy(self, display):
|
||||
"""None once the services are up and stay up, else what went wrong."""
|
||||
deadline = self.clock() + HEALTH_TIMEOUT_SECONDS + STABLE_SECONDS
|
||||
healthy_since = baseline = None
|
||||
web = disp = False
|
||||
count_known = True
|
||||
while self.clock() < deadline:
|
||||
web = self.web_responds()
|
||||
disp = self.service_active('ledmatrix') if display else True
|
||||
restarts = self.restart_count('ledmatrix') if display else None
|
||||
# Without a restart count a crash loop looks healthy between
|
||||
# attempts, so an unreadable count never counts as stable.
|
||||
count_known = not display or restarts is not None
|
||||
if web and disp and count_known and (healthy_since is None or restarts == baseline):
|
||||
if healthy_since is None:
|
||||
healthy_since, baseline = self.clock(), restarts
|
||||
elif self.clock() - healthy_since >= STABLE_SECONDS:
|
||||
return None
|
||||
else:
|
||||
healthy_since = None
|
||||
self.sleep(POLL_SECONDS)
|
||||
problems = []
|
||||
if not web:
|
||||
problems.append('the web interface did not respond')
|
||||
if not disp:
|
||||
problems.append('the display service did not stay running')
|
||||
if web and disp and not count_known:
|
||||
problems.append("the display service's restart count could not be read")
|
||||
return '; '.join(problems) or 'the display service kept restarting'
|
||||
|
||||
# -- rollback ---------------------------------------------------------
|
||||
|
||||
def changed_requirements(self, old, new):
|
||||
result = self._run(['git', 'diff', '--name-only', old, new])
|
||||
# If the diff is unavailable, reinstall both rather than guess.
|
||||
changed = set(result.stdout.split()) if result.returncode == 0 else set(REQUIREMENT_FILES)
|
||||
return [rel for rel in REQUIREMENT_FILES if rel in changed]
|
||||
|
||||
def install_requirements(self, rel):
|
||||
wrapper = self.project_root / 'scripts' / 'fix_perms' / 'safe_pip_install.sh'
|
||||
req = self.project_root / rel
|
||||
if not req.exists():
|
||||
return True
|
||||
for bash in BASH_CANDIDATES:
|
||||
result = self._run(['sudo', '-n', bash, str(wrapper), str(req)],
|
||||
timeout=PIP_TIMEOUT_SECONDS)
|
||||
if result.returncode == 0:
|
||||
return True
|
||||
return False
|
||||
|
||||
def rollback(self, pending):
|
||||
"""Reset to the previous commit and its dependencies. Returns (ok, detail)."""
|
||||
old, new = pending.get('old_head'), pending.get('new_head')
|
||||
if not old:
|
||||
return False, 'the commit to roll back to is unknown'
|
||||
requirements = self.changed_requirements(old, new) if new else list(REQUIREMENT_FILES)
|
||||
# Safe to --hard: the updater refuses to run with local edits to
|
||||
# tracked files, so the only thing this discards is the update.
|
||||
result = self._run(['git', 'reset', '--hard', old], timeout=120)
|
||||
if result.returncode != 0:
|
||||
return False, (f'"git reset --hard {old}" failed: '
|
||||
f'{(result.stderr or result.stdout or "").strip()}')
|
||||
failed = [rel for rel in requirements if not self.install_requirements(rel)]
|
||||
if failed:
|
||||
return True, ('reinstalling the previous dependencies from ' + ', '.join(failed)
|
||||
+ ' failed; run Install Base Requirements from the Tools tab')
|
||||
return True, ''
|
||||
|
||||
# -- the check itself -------------------------------------------------
|
||||
|
||||
def _finish(self, pending, status, reason=None, detail=None):
|
||||
pending.update({'status': status, 'reason': reason, 'detail': detail or None,
|
||||
'finished_at': time.time()})
|
||||
write_pending(self.pending_file, pending)
|
||||
self.log(' '.join(p for p in (status, reason or '', detail or '') if p))
|
||||
|
||||
def verify(self):
|
||||
pending = read_pending(self.pending_file)
|
||||
if not pending or pending.get('status') != 'pending':
|
||||
self.log('no update is waiting to be verified')
|
||||
return 0
|
||||
pending['status'] = 'verifying'
|
||||
write_pending(self.pending_file, pending)
|
||||
|
||||
display = bool(pending.get('display_was_active'))
|
||||
dependency_failures = pending.get('dependency_failures') or []
|
||||
if dependency_failures:
|
||||
# Never restart onto code whose packages did not install.
|
||||
reason = 'installing its dependencies failed (' + ', '.join(dependency_failures) + ')'
|
||||
elif not self.restart_services(display):
|
||||
# The old process may still be answering; checking it would pass
|
||||
# an update that never started.
|
||||
reason = 'restarting the services failed'
|
||||
else:
|
||||
reason = self.wait_healthy(display)
|
||||
if reason is None:
|
||||
self._finish(pending, 'success')
|
||||
return 0
|
||||
|
||||
self.log(f'update to {_short(pending.get("new_head"))} is unhealthy ({reason}); '
|
||||
f'rolling back to {_short(pending.get("old_head"))}')
|
||||
ok, detail = self.rollback(pending)
|
||||
if not ok:
|
||||
self._finish(pending, 'rollback_failed', reason, detail)
|
||||
return 1
|
||||
still = (self.wait_healthy(display) if self.restart_services(display)
|
||||
else 'restarting the services failed')
|
||||
if still:
|
||||
self._finish(pending, 'rollback_failed', reason,
|
||||
f'still unhealthy after rolling back: {still}'
|
||||
+ (f'; {detail}' if detail else ''))
|
||||
return 1
|
||||
self._finish(pending, 'rolled_back', reason, detail)
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) != 2:
|
||||
print('usage: auto_update_verify.py PROJECT_ROOT', file=sys.stderr)
|
||||
return 2
|
||||
verifier = Verifier(Path(argv[1]))
|
||||
try:
|
||||
return verifier.verify()
|
||||
except Exception as e:
|
||||
traceback.print_exc()
|
||||
# Whatever happened, the web interface must not be left thinking the
|
||||
# check is still running.
|
||||
try:
|
||||
pending = read_pending(verifier.pending_file) or {}
|
||||
if pending.get('status') in ('pending', 'verifying'):
|
||||
pending.update({'status': 'rollback_failed', 'reason': 'the health check crashed',
|
||||
'detail': str(e), 'finished_at': time.time()})
|
||||
write_pending(verifier.pending_file, pending)
|
||||
except OSError:
|
||||
pass
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,221 @@
|
||||
"""Install the automatic-update health check, from the display service.
|
||||
|
||||
The weekly updater (web_interface/auto_update.py) will not update LEDMatrix
|
||||
code unless ledmatrix-update-verify.path and .service are installed: they
|
||||
restart the services after an update and roll it back if the device is
|
||||
unhealthy. Installing units takes root and the web interface is not root, and
|
||||
"SSH in and run an installer" means most people never get updates with a
|
||||
safety net.
|
||||
|
||||
The display service already runs this repository's code as root, so it
|
||||
installs them -- but only while the user has automatic updates turned on, only
|
||||
these two units, rendered from the repository's templates for the web
|
||||
interface's own user, and it reports what happened in
|
||||
data/auto_update_setup.json for the General tab. It grants nothing new: the
|
||||
units run as the web user, who can already change the code this process runs.
|
||||
|
||||
Called at display startup; the web interface restarts the display service
|
||||
when the toggle is switched on, so setup happens straight away. Refreshing a
|
||||
unit whose template changed happens the same way, which is why this compares
|
||||
content rather than only checking that the files exist.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
SYSTEMD_DIR = Path('/etc/systemd/system')
|
||||
SERVICE_UNIT = 'ledmatrix-update-verify.service'
|
||||
PATH_UNIT = 'ledmatrix-update-verify.path'
|
||||
UNITS = (SERVICE_UNIT, PATH_UNIT)
|
||||
WEB_UNIT = 'ledmatrix-web.service'
|
||||
RESULT_REL = Path('data') / 'auto_update_setup.json'
|
||||
_USER_RE = re.compile(r'^[a-z_][a-z0-9_-]{0,31}$')
|
||||
|
||||
|
||||
class SetupError(Exception):
|
||||
"""A reason setup cannot proceed, worded for the General tab."""
|
||||
|
||||
|
||||
def _is_root():
|
||||
return hasattr(os, 'geteuid') and os.geteuid() == 0
|
||||
|
||||
|
||||
def _lookup_ids(user):
|
||||
try:
|
||||
import pwd
|
||||
entry = pwd.getpwnam(user)
|
||||
return entry.pw_uid, entry.pw_gid
|
||||
except (ImportError, KeyError):
|
||||
return None
|
||||
|
||||
|
||||
def _directive(text, key):
|
||||
match = re.search(rf'^{key}=(.*)$', text or '', re.M)
|
||||
return match.group(1).strip() if match else None
|
||||
|
||||
|
||||
def _read(path):
|
||||
try:
|
||||
return Path(path).read_text(encoding='utf-8')
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
|
||||
def is_enabled(config):
|
||||
return bool((config.get('auto_update') or {}).get('enabled', False))
|
||||
|
||||
|
||||
class UpdateHelperSetup:
|
||||
def __init__(self, project_root=PROJECT_ROOT, systemd_dir=SYSTEMD_DIR, run=subprocess.run,
|
||||
is_root=_is_root, lookup_ids=_lookup_ids, clock=time.time):
|
||||
self.project_root = Path(project_root)
|
||||
self.systemd_dir = Path(systemd_dir)
|
||||
self.run = run
|
||||
self.is_root = is_root
|
||||
self.lookup_ids = lookup_ids
|
||||
self.clock = clock
|
||||
self.result_file = self.project_root / RESULT_REL
|
||||
self._web_ids = None
|
||||
|
||||
def _systemctl(self, *args):
|
||||
return self.run(['systemctl', *args], capture_output=True, text=True, timeout=60)
|
||||
|
||||
def _check(self, result, what):
|
||||
if result.returncode != 0:
|
||||
raise SetupError(f'"{what}" failed: {(result.stderr or result.stdout or "").strip()}')
|
||||
|
||||
def path_active(self):
|
||||
try:
|
||||
return self._systemctl('is-active', PATH_UNIT).stdout.strip() == 'active'
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return False
|
||||
|
||||
def ensure(self, config):
|
||||
"""Install or refresh the units while automatic updates are on.
|
||||
|
||||
Returns the result recorded for the General tab, or None when there
|
||||
was nothing to do (updates off, or not a systemd host at all).
|
||||
"""
|
||||
if not is_enabled(config) or not self.systemd_dir.is_dir():
|
||||
return None
|
||||
try:
|
||||
changed = self._install()
|
||||
except SetupError as e:
|
||||
return self._report('failed', str(e))
|
||||
except (OSError, subprocess.SubprocessError) as e:
|
||||
return self._report('failed', f'Could not install the update health check: {e}')
|
||||
if changed:
|
||||
return self._report('installed', 'Installed the update health check.')
|
||||
return self._report('installed', 'The update health check is installed.', quiet=True)
|
||||
|
||||
def _install(self):
|
||||
if not self.is_root():
|
||||
raise SetupError('The display service is not running as root, so it cannot install the '
|
||||
'update health check. Run "sudo ./scripts/install/install_web_service.sh" once.')
|
||||
|
||||
web_text = _read(self.systemd_dir / WEB_UNIT)
|
||||
if web_text is None:
|
||||
raise SetupError('The web interface service (ledmatrix-web.service) is not installed.')
|
||||
user = _directive(web_text, 'User') or 'root'
|
||||
ids = self.lookup_ids(user) if _USER_RE.match(user) else None
|
||||
if ids is None:
|
||||
raise SetupError(f'The web interface runs as "{user}", which is not a usable account.')
|
||||
self._web_ids = ids
|
||||
workdir = _directive(web_text, 'WorkingDirectory')
|
||||
if not workdir or Path(workdir).resolve() != self.project_root.resolve():
|
||||
raise SetupError(f'The web interface service runs from {workdir or "an unknown folder"}, '
|
||||
f'not {self.project_root}.')
|
||||
# Spaces are fine -- the templates quote every command-line path --
|
||||
# but systemd expands % specifiers, and a quote, backslash or line
|
||||
# break would be reinterpreted in a unit file. (On Windows, where the
|
||||
# tests also run, a backslash is the path separator, not a name.)
|
||||
root_text = str(self.project_root)
|
||||
unsafe = set('%"') | ({'\\'} if os.sep == '/' else set())
|
||||
if any(ch in unsafe or ord(ch) < 32 for ch in root_text):
|
||||
raise SetupError(f'LEDMatrix is installed in {root_text!r}, a folder name systemd cannot use '
|
||||
'in a unit file. Move it to a path without %, quotes, backslashes or '
|
||||
'control characters.')
|
||||
|
||||
rendered = {}
|
||||
for name in UNITS:
|
||||
template = _read(self.project_root / 'systemd' / name)
|
||||
if template is None:
|
||||
raise SetupError(f'The unit template systemd/{name} is missing.')
|
||||
rendered[name] = (template.replace('__PROJECT_ROOT_DIR__', str(self.project_root))
|
||||
.replace('__USER__', user))
|
||||
# The templates are ordinary repository files. Whatever they say,
|
||||
# this root process only installs a service that runs as the web user
|
||||
# and a path unit that starts exactly that service.
|
||||
if _directive(rendered[SERVICE_UNIT], 'User') != user:
|
||||
raise SetupError(f'systemd/{SERVICE_UNIT} does not run as the web interface user; '
|
||||
'refusing to install it.')
|
||||
if _directive(rendered[PATH_UNIT], 'Unit') != SERVICE_UNIT:
|
||||
raise SetupError(f'systemd/{PATH_UNIT} does not start {SERVICE_UNIT}; refusing to install it.')
|
||||
|
||||
changed = [name for name in UNITS if _read(self.systemd_dir / name) != rendered[name]]
|
||||
for name in changed:
|
||||
self._write_unit(self.systemd_dir / name, rendered[name])
|
||||
if changed:
|
||||
self._check(self._systemctl('daemon-reload'), 'systemctl daemon-reload')
|
||||
self._check(self._systemctl('enable', PATH_UNIT), f'systemctl enable {PATH_UNIT}')
|
||||
self._check(self._systemctl('restart', PATH_UNIT), f'systemctl restart {PATH_UNIT}')
|
||||
elif not self.path_active():
|
||||
self._check(self._systemctl('enable', '--now', PATH_UNIT), f'systemctl enable --now {PATH_UNIT}')
|
||||
changed = [PATH_UNIT]
|
||||
if not self.path_active():
|
||||
raise SetupError(f'{PATH_UNIT} did not start; see "journalctl -u {PATH_UNIT}".')
|
||||
return bool(changed)
|
||||
|
||||
def _write_unit(self, path, text):
|
||||
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix=f'.{path.name}.')
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
f.write(text)
|
||||
os.chmod(tmp, 0o644)
|
||||
os.replace(tmp, path)
|
||||
except BaseException:
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
def _report(self, status, message, quiet=False):
|
||||
previous = None
|
||||
try:
|
||||
previous = json.loads(_read(self.result_file) or 'null')
|
||||
except ValueError:
|
||||
pass
|
||||
if quiet and isinstance(previous, dict) and previous.get('status') == status:
|
||||
return previous # nothing new; don't rewrite it on every boot
|
||||
result = {'status': status, 'message': message, 'at': self.clock()}
|
||||
(logger.info if status == 'installed' else logger.warning)("Automatic update setup: %s", message)
|
||||
try:
|
||||
self.result_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=str(self.result_file.parent), prefix='.auto_update_setup_')
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
json.dump(result, f, indent=2)
|
||||
os.chmod(tmp, 0o644)
|
||||
if self._web_ids and hasattr(os, 'chown'):
|
||||
# Only root can give the file away; the result is readable
|
||||
# (0644) either way, so a failed chown must not lose it.
|
||||
try:
|
||||
os.chown(tmp, *self._web_ids)
|
||||
except OSError:
|
||||
pass
|
||||
os.replace(tmp, self.result_file)
|
||||
except OSError as e:
|
||||
logger.warning("Could not record automatic update setup result: %s", e)
|
||||
return result
|
||||
|
||||
|
||||
def ensure_update_helper(config):
|
||||
return UpdateHelperSetup().ensure(config)
|
||||
@@ -0,0 +1,408 @@
|
||||
"""Sports helpers every scoreboard's ``sports.py`` carries an identical copy of.
|
||||
|
||||
Ten helpers are byte-identical (executable AST, docstrings stripped) in the
|
||||
scoreboard plugins' ``sports.py`` and have no equivalent elsewhere in
|
||||
``src/common``. The bodies below were copied from those plugin copies -- not
|
||||
from ``src/base_classes`` -- at ledmatrix-plugins ``f09bff2`` (origin/main,
|
||||
2026-09-14):
|
||||
|
||||
- In all nine (afl, baseball, basketball, football, hockey, lacrosse, nrl,
|
||||
soccer, ufc): ``_clamp_window``, ``_clamp_seconds``, ``_logo_needs_refresh``
|
||||
(with ``_MIN_WINDOW_DAYS`` / ``_MAX_WINDOW_DAYS``), and the ``SportsCore``
|
||||
methods ``_mode_customization``, ``_setting_int``,
|
||||
``_reset_dwell_on_reentry`` (with ``_DWELL_REENTRY_GAP_SECONDS``),
|
||||
``_next_switch_index`` and ``_spread_weighted_order``.
|
||||
- In the eight that share a lineage (all but ufc, which has neither):
|
||||
``_odds_color`` and ``_upcoming_date_and_time_text``.
|
||||
|
||||
The module-level helpers become public free functions here (``clamp_window``,
|
||||
``clamp_seconds``, ``logo_needs_refresh``, ``spread_weighted_order``,
|
||||
``MIN_WINDOW_DAYS``, ``MAX_WINDOW_DAYS``); the methods keep the plugins' exact
|
||||
names and signatures on ``SportsHelpersMixin`` so adopting it is deleting the
|
||||
copies and adding one base. ``test/test_sports_helpers.py`` compares every body
|
||||
here against every plugin copy when a plugins checkout is available
|
||||
(``LEDMATRIX_PLUGINS``), so drift in either direction fails a test.
|
||||
|
||||
WHY A NEW MODULE, NOT MORE METHODS ON sports_shared
|
||||
---------------------------------------------------
|
||||
A plugin that deletes its copy of a method and relies on an *existing* module
|
||||
having grown it fails at runtime on an older core: ``AttributeError`` mid-render,
|
||||
usually swallowed into a blank board. The plugin loader cannot see that, and
|
||||
neither can the plugins' ``scripts/check_min_core_version.py``, which checks
|
||||
modules. A plugin importing a module that does not exist yet fails at load,
|
||||
where both can. Plugins import this and floor ``ledmatrix_min_version`` on the
|
||||
first core release that ships it (see ``CHANGELOG.md``).
|
||||
|
||||
``_favorite_key`` is the one method not taken from the plugins: it is the
|
||||
override point from ``src/base_classes/sports/core.py``, carried here so later
|
||||
phases (shared celebrations and game selection) have a hardware-free home for
|
||||
the seam. No plugin defines it today and nothing in this module calls it.
|
||||
|
||||
WHAT A HOST MUST PROVIDE
|
||||
------------------------
|
||||
Derived by walking every ``self.<attr>`` the mixin reads; the host-contract
|
||||
test in ``test/test_sports_helpers.py`` fails if a read is added without being
|
||||
listed here.
|
||||
|
||||
- ``config`` (dict) -- ``_mode_customization``.
|
||||
- ``mode_config`` (dict) and ``logger`` -- ``_setting_int``. ``league`` is
|
||||
read with ``getattr`` for the warning text only.
|
||||
- ``games_list`` and ``current_game_index`` -- ``_next_switch_index``; plus
|
||||
``_is_favorite_game`` (called with a game), which stays per-plugin and is
|
||||
only called when
|
||||
``favorite_rotation_boost`` is above 1. ``favorite_rotation_boost`` itself
|
||||
defaults to 1 on the mixin.
|
||||
- ``last_game_switch`` -- ``_reset_dwell_on_reentry``, read with ``getattr``
|
||||
and written back.
|
||||
- ``_card_option``, ``_format_game_date`` and ``_format_game_time`` --
|
||||
``_upcoming_date_and_time_text``. All three are on
|
||||
``SportsCoreSharedMixin`` (``src/common/sports_shared.py``).
|
||||
- ``_element_color`` -- ``_odds_color``, looked up with ``getattr`` and falling
|
||||
back to green, so a host without it still works. Also on
|
||||
``SportsCoreSharedMixin``.
|
||||
- ``SKIN_MODE`` -- optional, read with ``getattr`` by ``_mode_customization``;
|
||||
without it the mode overrides are simply not merged.
|
||||
|
||||
State the mixin keeps on the host, created on first use (all read with
|
||||
``getattr``, so nothing needs initialising): ``_last_display_call_monotonic``,
|
||||
``_switch_order``, ``_switch_order_key``, ``_switch_position``.
|
||||
|
||||
Class-level defaults: ``_DWELL_REENTRY_GAP_SECONDS = 5.0`` and
|
||||
``favorite_rotation_boost = 1``, matching the plugins. A host's own value
|
||||
shadows them.
|
||||
|
||||
MRO
|
||||
---
|
||||
No ``__init__`` and no bare ``super()``, so base order does not change what
|
||||
runs. By convention list it after ``SportsCoreSharedMixin``::
|
||||
|
||||
class SportsCore(SportsCoreSharedMixin, SportsHelpersMixin, ABC):
|
||||
|
||||
HARDWARE-FREE
|
||||
-------------
|
||||
Nothing here may import ``src.display_manager``, ``src.base_classes``,
|
||||
``src.plugin_system`` or anything else that reaches ``rgbmatrix`` at module
|
||||
level; ``test/test_common_is_hardware_free.py`` enforces that for all of
|
||||
``src/common``. ``logo_needs_refresh`` imports ``src.logo_downloader`` lazily,
|
||||
exactly as the plugin copy does.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from typing import Any, ClassVar, Dict, List, Optional, Tuple
|
||||
|
||||
__all__ = [
|
||||
"MIN_WINDOW_DAYS",
|
||||
"MAX_WINDOW_DAYS",
|
||||
"clamp_window",
|
||||
"clamp_seconds",
|
||||
"logo_needs_refresh",
|
||||
"spread_weighted_order",
|
||||
"SportsHelpersMixin",
|
||||
]
|
||||
|
||||
MIN_WINDOW_DAYS = 1
|
||||
MAX_WINDOW_DAYS = 60
|
||||
|
||||
|
||||
def clamp_window(value: Any, fallback: int) -> int:
|
||||
"""Days for one side of the schedule window, or the default if unusable."""
|
||||
try:
|
||||
days = int(value)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
# OverflowError: json parses a bare Infinity and int(inf) raises,
|
||||
# which crashed manager init from a hand-edited config.
|
||||
return fallback
|
||||
return max(MIN_WINDOW_DAYS, min(MAX_WINDOW_DAYS, days))
|
||||
|
||||
|
||||
def clamp_seconds(value: Any, fallback: int, low: int = 5,
|
||||
high: int = 86400) -> int:
|
||||
"""An interval in seconds, or the fallback when the value is unusable."""
|
||||
try:
|
||||
seconds = int(value)
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
# OverflowError: json parses bare Infinity by default and int(inf)
|
||||
# raises -- the same gap clamp_window above already covers.
|
||||
return fallback
|
||||
return max(low, min(high, seconds))
|
||||
|
||||
|
||||
def logo_needs_refresh(logo_file) -> bool:
|
||||
"""True if this file is a placeholder stale enough to retry the real logo.
|
||||
|
||||
A failed logo download is cached as a placeholder wearing the real logo's
|
||||
filename, so "the file exists" is not proof the logo was ever fetched.
|
||||
Without this check one transient failure leaves a team a grey box forever.
|
||||
|
||||
Returns False on a core that predates placeholder marking, which keeps the
|
||||
previous behaviour rather than breaking the load.
|
||||
"""
|
||||
# Imported from the core by its full path, never as a bare name: a
|
||||
# deferred bare-name import can bind another plugin's vendored
|
||||
# logo_downloader once the core isolates top-level plugin modules.
|
||||
try:
|
||||
from src.logo_downloader import (
|
||||
PLACEHOLDER_RETRY_SECONDS,
|
||||
is_placeholder_logo,
|
||||
placeholder_age_seconds,
|
||||
)
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
try:
|
||||
if not is_placeholder_logo(logo_file):
|
||||
return False
|
||||
age = placeholder_age_seconds(logo_file)
|
||||
return age is None or age >= PLACEHOLDER_RETRY_SECONDS
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def spread_weighted_order(weights: List[int]) -> List[int]:
|
||||
"""Indices into ``weights``, each repeated by its weight and spread out.
|
||||
|
||||
Each index keeps its own slot and places its extra turns at even
|
||||
fractions of the rotation after it, wrapping round. That keeps the
|
||||
list's schedule order for everything else and spaces a favourite's
|
||||
repeats evenly *around the loop* -- the live rotation's smooth
|
||||
weighted round-robin schedules a boosted game first and last, so a
|
||||
rotation that wraps shows it back to back. Equal weights come back in
|
||||
plain order, so a boost that applies to no card changes nothing.
|
||||
|
||||
Repeats are kept apart only where the ratio leaves room: once one
|
||||
weight exceeds all the others combined, no cyclic order can separate
|
||||
its turns ([3, 1, 1] gives [0, 1, 0, 2, 0]). Each index still gets
|
||||
exactly its weight in turns -- the configured ratio wins over spacing.
|
||||
"""
|
||||
count = len(weights)
|
||||
slots = []
|
||||
for index, weight in enumerate(weights):
|
||||
for turn in range(weight):
|
||||
slots.append(((index + turn * count / weight) % count, turn > 0, index))
|
||||
return [index for _, _, index in sorted(slots)]
|
||||
|
||||
|
||||
class SportsHelpersMixin:
|
||||
"""The ``SportsCore`` methods identical across the scoreboards' sports.py.
|
||||
|
||||
Constructor-free; keeps lazy state on its host (see the module docstring
|
||||
for that state, the host contract and base-order guidance).
|
||||
"""
|
||||
|
||||
#: Longest gap between two display() calls that still counts as one
|
||||
#: on-screen stint. Frames arrive many times a second while a mode is on
|
||||
#: the panel; between mode blocks the gap is the length of every other
|
||||
#: mode's block -- a minute or more. Anything past a few seconds can only
|
||||
#: be a block boundary, or the very first frame after startup.
|
||||
_DWELL_REENTRY_GAP_SECONDS: ClassVar[float] = 5.0
|
||||
|
||||
#: Turns a favourite's card gets per turn of any other card in switch
|
||||
#: mode. The plugins set it per instance from config; this default keeps
|
||||
#: a host that does not on the plain rotation.
|
||||
favorite_rotation_boost: int = 1
|
||||
|
||||
def _favorite_key(self, game: Dict, side: str) -> Optional[str]:
|
||||
"""Override point: which view-model field identifies a team when
|
||||
matching against ``favorite_teams``.
|
||||
|
||||
``side`` is ``"home"`` or ``"away"``. The default is the team
|
||||
abbreviation -- what eight of the nine scoreboards match on, and what
|
||||
users type into their favorites list.
|
||||
|
||||
NRL needs the team **id** instead, because NRL abbreviations are not
|
||||
unique: "NEW" is both Newcastle Knights and New Zealand Warriors,
|
||||
"CAN" both Canberra Raiders and Canterbury Bulldogs. It is a seam
|
||||
rather than a branch so core never has to learn the string "nrl"::
|
||||
|
||||
def _favorite_key(self, game, side):
|
||||
return str(game.get(f"{side}_id"))
|
||||
|
||||
An override that stringifies should note that a missing id becomes the
|
||||
literal ``"None"``, which would spuriously match a favorites list
|
||||
containing that string. The default returns ``None`` for a missing
|
||||
abbreviation, which never matches.
|
||||
|
||||
Carried from ``src/base_classes/sports/core.py`` for later phases;
|
||||
nothing in this module calls it yet.
|
||||
"""
|
||||
return game.get(f"{side}_abbr")
|
||||
|
||||
def _mode_customization(self) -> dict:
|
||||
"""``customization`` with this mode's overrides merged over it.
|
||||
|
||||
SportsUpcoming / SportsRecent / SportsLive are separate instances
|
||||
with their own SKIN_MODE, so merging once here makes every
|
||||
per-element lookup mode-aware without changing one of them.
|
||||
|
||||
``None`` in a mode block means "inherit", which is what lets a user
|
||||
restyle one element on live cards and leave everything else
|
||||
following the settings above. It has to stay distinct from 0: a mode
|
||||
y_offset of 0 means "sit at the base position", not "no preference".
|
||||
"""
|
||||
customization = self.config.get('customization', {})
|
||||
if not isinstance(customization, dict):
|
||||
return {}
|
||||
mode = getattr(self, 'SKIN_MODE', None)
|
||||
if not mode:
|
||||
return customization
|
||||
modes = customization.get('modes')
|
||||
block = modes.get(mode) if isinstance(modes, dict) else None
|
||||
if not isinstance(block, dict):
|
||||
return customization
|
||||
|
||||
merged = dict(customization)
|
||||
for element, override in block.items():
|
||||
if element == 'layout' or not isinstance(override, dict):
|
||||
continue
|
||||
base = merged.get(element)
|
||||
base = dict(base) if isinstance(base, dict) else {}
|
||||
base.update({k: v for k, v in override.items() if v is not None})
|
||||
merged[element] = base
|
||||
|
||||
mode_layout = block.get('layout')
|
||||
if isinstance(mode_layout, dict):
|
||||
base_layout = merged.get('layout')
|
||||
new_layout = dict(base_layout) if isinstance(base_layout, dict) else {}
|
||||
for element, axes in mode_layout.items():
|
||||
if not isinstance(axes, dict):
|
||||
continue
|
||||
current = new_layout.get(element)
|
||||
current = dict(current) if isinstance(current, dict) else {}
|
||||
current.update({k: v for k, v in axes.items() if v is not None})
|
||||
new_layout[element] = current
|
||||
merged['layout'] = new_layout
|
||||
return merged
|
||||
|
||||
def _setting_int(self, key: str, default: int, low: int, high: int) -> int:
|
||||
"""A count from config, clamped to the range its schema declares.
|
||||
|
||||
The schema constrains these, but config.json can be hand-edited or
|
||||
written by an older tool, and a string or a negative here does not
|
||||
raise where anyone would see it -- it raises inside update()'s own
|
||||
try/except, which shows up as a mode that silently renders nothing.
|
||||
Same shape as the favorite_live_boost clamp in the plugins.
|
||||
"""
|
||||
try:
|
||||
return max(low, min(high, int(self.mode_config.get(key, default))))
|
||||
except (TypeError, ValueError, OverflowError):
|
||||
# OverflowError: json parses a bare Infinity, and int(inf) raises
|
||||
# -- from __init__, outside any try/except, so the manager would
|
||||
# fail to construct instead of falling back.
|
||||
self.logger.warning(
|
||||
"%s: ignoring unusable %s=%r, using %s",
|
||||
getattr(self, "league", "?"), key,
|
||||
self.mode_config.get(key), default,
|
||||
)
|
||||
return default
|
||||
|
||||
def _reset_dwell_on_reentry(self) -> bool:
|
||||
"""Give the current card a full turn when this mode (re)takes the panel.
|
||||
|
||||
The dwell clock (last_game_switch) keeps running while the mode is off
|
||||
screen, so on re-entry it was always long expired and the first
|
||||
display() call advanced immediately: the card cut off by the end of
|
||||
the previous block was skipped instead of shown -- measured at one in
|
||||
five card transitions on a 30s block of 15s cards -- and after a
|
||||
service restart the clock started at manager construction, seconds
|
||||
before the first frame, shaving that much off the first card. Both are
|
||||
the same defect: the dwell clock counting time the viewer never saw.
|
||||
|
||||
Returns True when the dwell was reset, so the caller forces a redraw.
|
||||
The one-frame card at the end of a block (the advance that races the
|
||||
controller's mode switch) still renders -- this reset is what turns it
|
||||
into the card that opens the next block with a full turn, instead of
|
||||
one the rotation skipped.
|
||||
"""
|
||||
# getattr, and zero treated as "never displayed": the managers are
|
||||
# constructed in several places -- the plugin tests among them -- not
|
||||
# all of which set every attribute, and a freshly booted Pi can reach
|
||||
# the first frame while time.monotonic() itself is still under the
|
||||
# gap threshold, which would make `now - 0.0` look like one stint.
|
||||
last = getattr(self, "_last_display_call_monotonic", 0.0)
|
||||
now = time.monotonic()
|
||||
self._last_display_call_monotonic = now
|
||||
if last > 0.0 and now - last < self._DWELL_REENTRY_GAP_SECONDS:
|
||||
return False
|
||||
if getattr(self, "last_game_switch", 0) <= 0:
|
||||
# Zero is the live screen's "no game shown yet" sentinel with its
|
||||
# own handling; overwriting it here would hide the first game's
|
||||
# arrival from that logic.
|
||||
return False
|
||||
self.last_game_switch = time.time()
|
||||
return True
|
||||
|
||||
_spread_weighted_order = staticmethod(spread_weighted_order)
|
||||
|
||||
def _next_switch_index(self) -> int:
|
||||
"""The games_list index switch mode shows next.
|
||||
|
||||
favorite_rotation_boost gives a favourite's card that many turns for
|
||||
every one turn another card gets, spread through the rotation and kept
|
||||
apart wherever the other cards leave room (a boost above the number of
|
||||
other cards makes some repeats adjacent; the ratio is kept either way).
|
||||
games_list itself stays one entry per game -- the
|
||||
cycle-duration count, the scroll strip and the other-games re-cut all
|
||||
read it -- so the weighting is an order walked over it instead.
|
||||
|
||||
The order is rebuilt whenever the list's games change, and the walk
|
||||
resyncs from current_game_index whenever the two disagree: update()
|
||||
and the other-games rotation both set the index directly when they
|
||||
swap a list in, and the card on screen is where the walk resumes.
|
||||
|
||||
Called with _games_lock held and games_list non-empty.
|
||||
"""
|
||||
count = len(self.games_list)
|
||||
boost = getattr(self, "favorite_rotation_boost", 1)
|
||||
if boost <= 1 or count < 2:
|
||||
return (self.current_game_index + 1) % count
|
||||
key = (boost, tuple(g.get("id") for g in self.games_list))
|
||||
if getattr(self, "_switch_order_key", None) != key:
|
||||
self._switch_order = self._spread_weighted_order(
|
||||
[boost if self._is_favorite_game(g) else 1 for g in self.games_list]
|
||||
)
|
||||
self._switch_order_key = key
|
||||
self._switch_position = -1
|
||||
order = self._switch_order
|
||||
position = getattr(self, "_switch_position", -1)
|
||||
if not 0 <= position < len(order) or order[position] != self.current_game_index:
|
||||
position = (order.index(self.current_game_index)
|
||||
if self.current_game_index in order else -1)
|
||||
position = (position + 1) % len(order)
|
||||
self._switch_position = position
|
||||
return order[position]
|
||||
|
||||
def _odds_color(self) -> Tuple[int, int, int]:
|
||||
"""Colour for the odds text; the green it always drew unless configured.
|
||||
|
||||
Guarded with getattr because not every class that reaches
|
||||
_draw_dynamic_odds carries the element-colour helper -- the plugins'
|
||||
own test harnesses build minimal manager objects, and a bare
|
||||
AttributeError here is swallowed by the surrounding except, which
|
||||
drops the odds off the card instead of failing loudly.
|
||||
"""
|
||||
getter = getattr(self, "_element_color", None)
|
||||
if getter is None:
|
||||
return (0, 255, 0)
|
||||
try:
|
||||
return getter("odds_text", (0, 255, 0)) # pylint: disable=not-callable
|
||||
except Exception:
|
||||
return (0, 255, 0)
|
||||
|
||||
def _upcoming_date_and_time_text(self, game_date: str, game_time: str,
|
||||
game: Optional[Dict] = None) -> Tuple[str, str]:
|
||||
"""The formatted (date, time) pair, blanked by switch_show_date/_time.
|
||||
|
||||
Deliberately not the shared show_date/show_time: those governed only
|
||||
the scroll and Vegas cards before this display read the block, so a
|
||||
config that had turned them off there would silently blank a scorebug
|
||||
that has always drawn both lines. The switch keys default to True for
|
||||
the same reason switch_upcoming_center defaults to "date_time" -- an
|
||||
untouched panel keeps rendering exactly what it rendered before.
|
||||
"""
|
||||
date_text = (self._format_game_date(game_date, game)
|
||||
if self._card_option("switch_show_date", True) else "")
|
||||
time_text = (self._format_game_time(game_time)
|
||||
if self._card_option("switch_show_time", True) else "")
|
||||
return date_text, time_text
|
||||
@@ -119,6 +119,15 @@ class DisplayController:
|
||||
# validator.raise_on_errors() # Uncomment to fail fast on errors
|
||||
except Exception as e:
|
||||
logger.warning(f"Startup validation could not be completed: {e}")
|
||||
|
||||
# Automatic updates need their health-check units, and this is the
|
||||
# one root process running project code, so it installs them while
|
||||
# automatic updates are on. See src/auto_update_setup.py.
|
||||
try:
|
||||
from src.auto_update_setup import ensure_update_helper
|
||||
ensure_update_helper(self.config)
|
||||
except Exception as e:
|
||||
logger.warning("Automatic update setup could not be completed: %s", e)
|
||||
|
||||
config_time = time.time()
|
||||
self.display_manager = DisplayManager(self.config)
|
||||
|
||||
@@ -81,6 +81,8 @@ class StartupValidator:
|
||||
_UNITS = (
|
||||
("systemd/ledmatrix.service", "/etc/systemd/system/ledmatrix.service"),
|
||||
("systemd/ledmatrix-web.service", "/etc/systemd/system/ledmatrix-web.service"),
|
||||
("systemd/ledmatrix-update-verify.service", "/etc/systemd/system/ledmatrix-update-verify.service"),
|
||||
("systemd/ledmatrix-update-verify.path", "/etc/systemd/system/ledmatrix-update-verify.path"),
|
||||
)
|
||||
|
||||
def _validate_systemd_units(self) -> None:
|
||||
|
||||
+12
-1
@@ -14,6 +14,16 @@ This directory contains systemd service unit files for LEDMatrix services.
|
||||
- Starts automatically on boot if `web_display_autostart` is enabled
|
||||
- Uses `scripts/utils/start_web_conditionally.py`
|
||||
|
||||
- **`ledmatrix-update-verify.service`** / **`.path`** - Automatic update health check and rollback
|
||||
- The path unit starts the service when the web interface creates
|
||||
`data/auto_update_verify.request` after a weekly automatic update
|
||||
- Installed by the installers, or by the display service when automatic
|
||||
updates are turned on in the web UI (`src/auto_update_setup.py`)
|
||||
- Restarts the services, checks they stay up, and otherwise resets to the
|
||||
previous commit and its dependencies
|
||||
- Runs `data/auto_update_verifier.py`, a copy of
|
||||
`scripts/utils/auto_update_verify.py` taken before the update
|
||||
|
||||
- **`ledmatrix-wifi-monitor.service`** - WiFi monitor daemon service
|
||||
- Monitors WiFi/Ethernet connectivity
|
||||
- Automatically enables/disables access point mode
|
||||
@@ -43,7 +53,8 @@ This directory contains systemd service unit files for LEDMatrix services.
|
||||
|
||||
These service files are installed by the installation scripts in `scripts/install/`:
|
||||
- `install_service.sh` installs `ledmatrix.service`
|
||||
- `install_web_service.sh` installs `ledmatrix-web.service`
|
||||
- `install_web_service.sh` installs `ledmatrix-web.service` and the
|
||||
`ledmatrix-update-verify` service and path units
|
||||
- `install_wifi_monitor.sh` installs `ledmatrix-wifi-monitor.service`
|
||||
- `install_dns_fix.sh` installs `ledmatrix-dns-fix.service` (opt-in, not run
|
||||
by the normal installer)
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# This is a template. Do not install it directly -- it is installed with
|
||||
# ledmatrix-update-verify.service by scripts/install/install_web_service.sh,
|
||||
# or by the display service when automatic updates are turned on
|
||||
# (src/auto_update_setup.py). Substitutes:
|
||||
# __PROJECT_ROOT_DIR__ -> the directory the project is installed in
|
||||
#
|
||||
# Starts the automatic update health check when the web interface asks for it
|
||||
# by creating data/auto_update_verify.request. A path unit rather than a
|
||||
# sudoers rule: the web interface needs no extra privilege to trigger it, and
|
||||
# the service it starts runs as the web interface's own user.
|
||||
|
||||
[Unit]
|
||||
Description=Watch for LED Matrix automatic update health check requests
|
||||
|
||||
[Path]
|
||||
PathExists=__PROJECT_ROOT_DIR__/data/auto_update_verify.request
|
||||
Unit=ledmatrix-update-verify.service
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,31 @@
|
||||
# This is a template. Do not install it directly -- run
|
||||
# scripts/install/install_web_service.sh (or install_service.sh); the display
|
||||
# service also installs it when automatic updates are turned on
|
||||
# (src/auto_update_setup.py). Substitutes:
|
||||
# __PROJECT_ROOT_DIR__ -> the directory the project is installed in
|
||||
# __USER__ -> the user the web interface runs as
|
||||
#
|
||||
# Health check and rollback for the web interface's weekly automatic update
|
||||
# (web_interface/auto_update.py). A separate unit because the check restarts
|
||||
# ledmatrix-web.service, which would kill it if it ran there.
|
||||
#
|
||||
# Never enabled itself: ledmatrix-update-verify.path starts it when the web
|
||||
# interface creates data/auto_update_verify.request. It runs as the web
|
||||
# interface's own user, and ExecStart points at the copy the updater makes
|
||||
# before pulling, so the update being checked cannot replace its checker.
|
||||
# Command-line paths are quoted so an install folder with spaces still works.
|
||||
|
||||
[Unit]
|
||||
Description=LED Matrix automatic update health check and rollback
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=__USER__
|
||||
WorkingDirectory=__PROJECT_ROOT_DIR__
|
||||
# Consume the request first, so the path unit cannot start this again.
|
||||
ExecStartPre=/bin/rm -f "__PROJECT_ROOT_DIR__/data/auto_update_verify.request"
|
||||
ExecStart=/usr/bin/python3 "__PROJECT_ROOT_DIR__/data/auto_update_verifier.py" "__PROJECT_ROOT_DIR__"
|
||||
TimeoutStartSec=30min
|
||||
StandardOutput=syslog
|
||||
StandardError=syslog
|
||||
SyslogIdentifier=ledmatrix-update-verify
|
||||
Vendored
+17
@@ -841,6 +841,23 @@
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/system/auto-update",
|
||||
"api_v3.get_auto_update_status",
|
||||
[
|
||||
"GET",
|
||||
"HEAD",
|
||||
"OPTIONS"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/system/auto-update/dismiss",
|
||||
"api_v3.dismiss_auto_update_alert",
|
||||
[
|
||||
"OPTIONS",
|
||||
"POST"
|
||||
]
|
||||
],
|
||||
[
|
||||
"/api/v3/system/check-update",
|
||||
"api_v3.check_for_update",
|
||||
|
||||
@@ -0,0 +1,683 @@
|
||||
"""Weekly automatic updates (web_interface/auto_update.py).
|
||||
|
||||
The updater pulls code and restarts services unattended. These pin down that
|
||||
it runs only when asked and due, refuses to touch a checkout it could damage,
|
||||
never leaves new code running without a health check, and reports every
|
||||
failure instead of logging it and moving on.
|
||||
|
||||
Git runs for real against a throwaway origin/device clone pair; systemd, sudo
|
||||
and the health check service are faked.
|
||||
"""
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from web_interface import auto_update as au # noqa: E402
|
||||
|
||||
# Local time is pinned to UTC below, so 03:00 is inside the quiet hours.
|
||||
NIGHT = datetime(2026, 9, 14, 3, 0, tzinfo=timezone.utc).timestamp()
|
||||
NOON = datetime(2026, 9, 14, 12, 0, tzinfo=timezone.utc).timestamp()
|
||||
ON = {'auto_update': {'enabled': True}}
|
||||
OFF = {'auto_update': {'enabled': False}}
|
||||
|
||||
needs_git = pytest.mark.skipif(shutil.which('git') is None, reason='git not installed')
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def local_is_utc(monkeypatch):
|
||||
"""Pin 'local time' so quiet-hours checks don't depend on the host zone."""
|
||||
monkeypatch.setattr(au, '_zone', lambda name: timezone.utc)
|
||||
|
||||
|
||||
def git(cwd, *args):
|
||||
result = subprocess.run(['git', '-c', 'user.name=t', '-c', 'user.email=t@example.com', *args],
|
||||
cwd=str(cwd), capture_output=True, text=True)
|
||||
assert result.returncode == 0, result.stderr
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
class Repo:
|
||||
"""An origin, a checkout that publishes to it, and the device's clone."""
|
||||
|
||||
def __init__(self, tmp):
|
||||
self.seed = tmp / 'seed'
|
||||
self.seed.mkdir()
|
||||
git(self.seed, 'init', '-q', '-b', 'main')
|
||||
(self.seed / 'app.py').write_text('v = 1\n')
|
||||
git(self.seed, 'add', '.')
|
||||
git(self.seed, 'commit', '-qm', 'one')
|
||||
self.origin = tmp / 'origin.git'
|
||||
git(tmp, 'clone', '-q', '--bare', str(self.seed), str(self.origin))
|
||||
git(self.seed, 'remote', 'add', 'origin', str(self.origin))
|
||||
self.device = tmp / 'device'
|
||||
git(tmp, 'clone', '-q', str(self.origin), str(self.device))
|
||||
|
||||
def head(self):
|
||||
return git(self.device, 'rev-parse', 'HEAD')
|
||||
|
||||
def publish(self, text='v = 2\n'):
|
||||
(self.seed / 'app.py').write_text(text)
|
||||
git(self.seed, 'commit', '-qam', 'next')
|
||||
git(self.seed, 'push', '-q', 'origin', 'main')
|
||||
return git(self.seed, 'rev-parse', 'HEAD')
|
||||
|
||||
|
||||
def real_pull(device, **extra):
|
||||
def core_update():
|
||||
before = git(device, 'rev-parse', 'HEAD')
|
||||
r = subprocess.run(['git', 'pull', '-q', '--rebase', '--autostash'],
|
||||
cwd=str(device), capture_output=True, text=True)
|
||||
after = git(device, 'rev-parse', 'HEAD')
|
||||
result = {'status': 'success' if r.returncode == 0 else 'error',
|
||||
'message': 'Code updated successfully.' if r.returncode == 0 else r.stderr,
|
||||
'restart_required': before != after, 'dependency_failures': []}
|
||||
result.update(extra)
|
||||
return result
|
||||
return core_update
|
||||
|
||||
|
||||
class FakeStore:
|
||||
def __init__(self, plugins_dir, versions, bump=(), fail=()):
|
||||
self.plugins_dir = str(plugins_dir)
|
||||
self.bump, self.fail = set(bump), set(fail)
|
||||
for pid, version in versions.items():
|
||||
d = Path(plugins_dir) / pid
|
||||
d.mkdir(parents=True)
|
||||
(d / 'manifest.json').write_text(json.dumps({'id': pid, 'version': version}))
|
||||
self.updated_calls = []
|
||||
|
||||
def list_installed_plugins(self):
|
||||
return [p.name for p in Path(self.plugins_dir).iterdir()]
|
||||
|
||||
def _get_local_git_info(self, path):
|
||||
return None
|
||||
|
||||
def update_plugin(self, pid):
|
||||
self.updated_calls.append(pid)
|
||||
if pid in self.fail:
|
||||
return False
|
||||
if pid in self.bump:
|
||||
path = Path(self.plugins_dir) / pid / 'manifest.json'
|
||||
m = json.loads(path.read_text())
|
||||
m['version'] = '9.9.9'
|
||||
path.write_text(json.dumps(m))
|
||||
return True
|
||||
|
||||
|
||||
class Harness:
|
||||
"""An AutoUpdater on a real clone.
|
||||
|
||||
``helper`` is whether the health check's path unit is active; ``pickup``
|
||||
whether the health check actually starts when asked. Starting it is
|
||||
simulated the way systemd does it: the request file is consumed
|
||||
(ExecStartPre) and the verifier marks the pending update "verifying".
|
||||
"""
|
||||
|
||||
def __init__(self, tmp, repo, *, helper=True, pickup=True, core_update=None, store=None,
|
||||
disk_free=10 ** 12, display_active=True, enabled=True, clock=NIGHT):
|
||||
self.sudo, self.restarts, self.now = [], [], clock
|
||||
self.state_at_handoff = None
|
||||
|
||||
def run(args, **kwargs):
|
||||
if args[0] == 'sudo':
|
||||
self.sudo.append(list(args))
|
||||
return subprocess.CompletedProcess(args, 0, stdout='', stderr='')
|
||||
return subprocess.run(args, **kwargs)
|
||||
|
||||
def sleep(seconds):
|
||||
updater = self.updater
|
||||
if self.state_at_handoff is None:
|
||||
self.state_at_handoff = au.load_state(updater.state_file)
|
||||
if pickup and updater.request_file.exists():
|
||||
updater.request_file.unlink()
|
||||
pending = au._read_json(updater.pending_file)
|
||||
pending['status'] = 'verifying'
|
||||
au._write_json(updater.pending_file, pending)
|
||||
|
||||
self.config = {'auto_update': {'enabled': enabled}, 'timezone': 'UTC'}
|
||||
cm = MagicMock()
|
||||
cm.load_config.return_value = self.config
|
||||
self.updater = au.AutoUpdater(
|
||||
config_manager=cm, core_update=core_update or real_pull(repo.device),
|
||||
store_manager=store, project_root=repo.device, clock=lambda: self.now,
|
||||
restart=self.restarts.append, run=run,
|
||||
service_active=lambda unit: display_active, helper_ready=lambda: helper,
|
||||
disk_free=lambda path: disk_free, sleep=sleep)
|
||||
|
||||
@property
|
||||
def state(self):
|
||||
return au.load_state(self.updater.state_file)
|
||||
|
||||
@property
|
||||
def pending(self):
|
||||
return au._read_json(self.updater.pending_file)
|
||||
|
||||
def write_pending(self, **fields):
|
||||
au._write_json(self.updater.pending_file, fields)
|
||||
|
||||
|
||||
def stub_updater(tmp_path, outcome='up_to_date', message='ok', store=None, enabled=True, clock=NIGHT):
|
||||
"""For scheduling and plugin tests: the core update is stubbed out."""
|
||||
cm = MagicMock()
|
||||
cm.load_config.return_value = {'auto_update': {'enabled': enabled}, 'timezone': 'UTC'}
|
||||
restarts = []
|
||||
updater = au.AutoUpdater(config_manager=cm, core_update=MagicMock(), store_manager=store,
|
||||
project_root=tmp_path, clock=lambda: clock,
|
||||
restart=restarts.append)
|
||||
updater.update_core = MagicMock(return_value={'outcome': outcome, 'message': message})
|
||||
return updater, restarts
|
||||
|
||||
|
||||
# -- scheduling ---------------------------------------------------------------
|
||||
|
||||
class TestIsDue:
|
||||
def test_not_before_next_due(self):
|
||||
assert not au.is_due(100, 200, 3)
|
||||
|
||||
def test_in_quiet_hours_once_due(self):
|
||||
assert au.is_due(200, 200, 3)
|
||||
|
||||
def test_waits_for_quiet_hours(self):
|
||||
assert not au.is_due(200, 100, 12)
|
||||
|
||||
def test_gives_up_waiting_after_grace(self):
|
||||
assert au.is_due(100 + au.QUIET_HOURS_GRACE_SECONDS, 100, 12)
|
||||
|
||||
|
||||
class TestTick:
|
||||
def test_disabled_never_runs(self, tmp_path):
|
||||
updater, _ = stub_updater(tmp_path, enabled=False)
|
||||
assert updater.tick() is False
|
||||
updater.update_core.assert_not_called()
|
||||
|
||||
def test_enabling_at_noon_waits_for_the_night(self, tmp_path):
|
||||
updater, _ = stub_updater(tmp_path, clock=NOON)
|
||||
assert updater.tick() is False
|
||||
updater.update_core.assert_not_called()
|
||||
assert au.load_state(updater.state_file)['next_due'] == NOON
|
||||
|
||||
def test_runs_when_due_and_schedules_a_week_out(self, tmp_path):
|
||||
updater, _ = stub_updater(tmp_path)
|
||||
assert updater.tick() is True
|
||||
state = au.load_state(updater.state_file)
|
||||
assert state['next_due'] == NIGHT + au.UPDATE_INTERVAL_SECONDS
|
||||
assert updater.tick() is False, "ran twice in one night"
|
||||
|
||||
def test_transient_error_retries_next_day(self, tmp_path):
|
||||
updater, _ = stub_updater(tmp_path, outcome='error', message='no network')
|
||||
updater.tick()
|
||||
state = au.load_state(updater.state_file)
|
||||
assert state['next_due'] == NIGHT + au.RETRY_AFTER_FAILURE_SECONDS
|
||||
assert state['alert']['message'] == 'no network'
|
||||
|
||||
def test_blocked_keeps_weekly_cadence_but_alerts(self, tmp_path):
|
||||
updater, _ = stub_updater(tmp_path, outcome='blocked', message='local edits')
|
||||
updater.tick()
|
||||
state = au.load_state(updater.state_file)
|
||||
assert state['next_due'] == NIGHT + au.UPDATE_INTERVAL_SECONDS
|
||||
assert 'local edits' in state['alert']['message']
|
||||
|
||||
|
||||
# -- checks before touching the checkout ---------------------------------------
|
||||
|
||||
@needs_git
|
||||
class TestPreflightRefuses:
|
||||
def test_nothing_new_is_up_to_date(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
h = Harness(tmp_path, repo)
|
||||
result = h.updater.run()
|
||||
assert result['core_outcome'] == 'up_to_date'
|
||||
assert result['status'] == 'success'
|
||||
assert h.sudo == [] and 'alert' not in h.state
|
||||
|
||||
def test_without_the_health_check_code_is_not_touched(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
repo.publish()
|
||||
h = Harness(tmp_path, repo, helper=False)
|
||||
result = h.updater.run()
|
||||
assert result['core_outcome'] == 'blocked'
|
||||
assert repo.head() == old
|
||||
assert 'health check is set up' in result['core_message']
|
||||
assert h.state['alert']
|
||||
|
||||
def test_local_edits_are_never_stashed(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
repo.publish()
|
||||
(repo.device / 'app.py').write_text('mine\n')
|
||||
h = Harness(tmp_path, repo)
|
||||
result = h.updater.run()
|
||||
assert result['core_outcome'] == 'blocked'
|
||||
assert 'app.py' in result['core_message']
|
||||
assert repo.head() == old
|
||||
assert (repo.device / 'app.py').read_text() == 'mine\n'
|
||||
assert git(repo.device, 'stash', 'list') == ''
|
||||
|
||||
def test_local_commits_block(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
repo.publish()
|
||||
(repo.device / 'extra.txt').write_text('x\n')
|
||||
git(repo.device, 'add', 'extra.txt')
|
||||
git(repo.device, 'commit', '-qm', 'local')
|
||||
mine = repo.head()
|
||||
h = Harness(tmp_path, repo)
|
||||
result = h.updater.run()
|
||||
assert result['core_outcome'] == 'blocked'
|
||||
assert 'local commit' in result['core_message']
|
||||
assert repo.head() == mine
|
||||
|
||||
def test_a_rebase_really_in_progress_blocks(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
repo.publish('v = 2\n')
|
||||
(repo.device / 'app.py').write_text('v = mine\n')
|
||||
git(repo.device, 'commit', '-qam', 'local')
|
||||
git(repo.device, 'fetch', '-q')
|
||||
stopped = subprocess.run(['git', '-c', 'user.name=t', '-c', 'user.email=t@example.com',
|
||||
'rebase', 'origin/main'], cwd=str(repo.device), capture_output=True, text=True)
|
||||
assert stopped.returncode != 0, "the rebase should stop on the conflict"
|
||||
result = Harness(tmp_path, repo).updater.run()
|
||||
assert result['core_outcome'] == 'blocked'
|
||||
assert 'rebase is in progress' in result['core_message']
|
||||
assert (repo.device / '.git' / 'rebase-merge').exists(), "a live rebase must be left alone"
|
||||
|
||||
def test_an_abandoned_rebase_is_cleared_and_the_update_runs(self, tmp_path):
|
||||
"""Found on a real device: a pull stopped mid-rebase a week earlier,
|
||||
HEAD was later checked out to a branch, and the leftover rebase
|
||||
directory would have blocked every automatic update forever."""
|
||||
repo = Repo(tmp_path)
|
||||
leftover = repo.device / '.git' / 'rebase-merge'
|
||||
leftover.mkdir()
|
||||
(leftover / 'head-name').write_text('refs/heads/main\n')
|
||||
new = repo.publish()
|
||||
result = Harness(tmp_path, repo).updater.run()
|
||||
assert not leftover.exists()
|
||||
assert result['core_outcome'] == 'verifying'
|
||||
assert repo.head() == new
|
||||
|
||||
def test_a_merge_in_progress_blocks(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
repo.publish()
|
||||
(repo.device / '.git' / 'MERGE_HEAD').write_text(repo.head() + '\n')
|
||||
result = Harness(tmp_path, repo).updater.run()
|
||||
assert result['core_outcome'] == 'blocked'
|
||||
assert 'merge is in progress' in result['core_message']
|
||||
|
||||
def test_low_disk_blocks(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
repo.publish()
|
||||
result = Harness(tmp_path, repo, disk_free=10 * 1024 * 1024).updater.run()
|
||||
assert result['core_outcome'] == 'blocked'
|
||||
assert 'disk space' in result['core_message']
|
||||
|
||||
def test_unreachable_origin_retries_tomorrow(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
git(repo.device, 'remote', 'set-url', 'origin', str(tmp_path / 'nowhere'))
|
||||
h = Harness(tmp_path, repo)
|
||||
result = h.updater.run()
|
||||
assert result['core_outcome'] == 'error'
|
||||
assert h.state['next_due'] == NIGHT + au.RETRY_AFTER_FAILURE_SECONDS
|
||||
|
||||
def test_version_already_rolled_back_is_not_retried(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
bad = repo.publish()
|
||||
h = Harness(tmp_path, repo)
|
||||
au.save_state({'rolled_back_head': bad}, h.updater.state_file)
|
||||
result = h.updater.run()
|
||||
assert result['core_outcome'] == 'up_to_date'
|
||||
assert 'rolled back before' in result['core_message']
|
||||
assert repo.head() == old and h.sudo == []
|
||||
|
||||
|
||||
# -- the update and its hand-off to the health check ---------------------------
|
||||
|
||||
@needs_git
|
||||
class TestUpdateIsVerified:
|
||||
def test_pull_hands_off_to_the_health_check(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
new = repo.publish()
|
||||
store = FakeStore(tmp_path / 'plugins', {'weather': '1.0.0'}, bump={'weather'})
|
||||
h = Harness(tmp_path, repo, store=store)
|
||||
result = h.updater.run()
|
||||
|
||||
assert result['core_outcome'] == 'verifying' and result['status'] == 'pending'
|
||||
assert repo.head() == new
|
||||
assert h.pending == {**h.pending, 'status': 'verifying', 'old_head': old, 'new_head': new,
|
||||
'display_was_active': True, 'dependency_failures': []}
|
||||
assert not h.updater.request_file.exists(), "the request is consumed when the check starts"
|
||||
assert h.sudo == [], "triggering the health check needs no privilege"
|
||||
assert h.restarts == [], "restarts belong to the health check, not the web process"
|
||||
assert store.updated_calls == [], "plugins must wait until the new code is verified"
|
||||
assert h.state['plugins_pending'] is True
|
||||
assert h.updater.verifier_copy.read_text() == au.VERIFIER_SOURCE.read_text()
|
||||
|
||||
def test_state_is_saved_before_the_health_check_restarts_this_process(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
repo.publish()
|
||||
h = Harness(tmp_path, repo)
|
||||
h.updater.run()
|
||||
assert h.state_at_handoff['last_result']['core_outcome'] == 'verifying'
|
||||
assert h.state_at_handoff['plugins_pending'] is True
|
||||
|
||||
def test_dependency_failures_reach_the_health_check(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
repo.publish()
|
||||
h = Harness(tmp_path, repo,
|
||||
core_update=real_pull(repo.device, dependency_failures=['requirements.txt']))
|
||||
h.updater.run()
|
||||
assert h.pending['dependency_failures'] == ['requirements.txt']
|
||||
|
||||
def test_a_health_check_that_never_starts_means_the_update_is_undone(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
repo.publish()
|
||||
h = Harness(tmp_path, repo, pickup=False)
|
||||
result = h.updater.run()
|
||||
assert repo.head() == old
|
||||
assert result['core_outcome'] == 'blocked'
|
||||
assert 'did not start' in result['core_message']
|
||||
assert h.pending is None and not h.updater.request_file.exists()
|
||||
assert h.state['alert']
|
||||
|
||||
def test_failed_pull_retries_tomorrow(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
repo.publish()
|
||||
h = Harness(tmp_path, repo,
|
||||
core_update=lambda: {'status': 'error', 'message': 'Update failed: network'})
|
||||
result = h.updater.run()
|
||||
assert result['core_outcome'] == 'error'
|
||||
assert repo.head() == old
|
||||
assert h.state['next_due'] == NIGHT + au.RETRY_AFTER_FAILURE_SECONDS
|
||||
|
||||
def test_failed_pull_that_moved_head_is_rolled_back(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
repo.publish()
|
||||
pull = real_pull(repo.device)
|
||||
|
||||
def half_failed():
|
||||
pull()
|
||||
return {'status': 'error', 'message': 'Update failed: interrupted'}
|
||||
result = Harness(tmp_path, repo, core_update=half_failed).updater.run()
|
||||
assert repo.head() == old
|
||||
assert 'rolled back' in result['core_message']
|
||||
|
||||
|
||||
# -- reporting the health check's outcome --------------------------------------
|
||||
|
||||
@needs_git
|
||||
class TestHealthCheckOutcome:
|
||||
def _after_update(self, tmp_path, **pending):
|
||||
repo = Repo(tmp_path)
|
||||
store = FakeStore(tmp_path / 'plugins', {'weather': '1.0.0'}, bump={'weather'})
|
||||
h = Harness(tmp_path, repo, store=store)
|
||||
au.save_state({'plugins_pending': True, 'next_due': NIGHT + au.UPDATE_INTERVAL_SECONDS,
|
||||
'last_result': {'core_outcome': 'verifying', 'status': 'pending',
|
||||
'plugins_deferred': True}}, h.updater.state_file)
|
||||
h.write_pending(old_head='a' * 40, new_head='b' * 40, created_at=NIGHT, **pending)
|
||||
return h, store
|
||||
|
||||
def test_waits_while_the_check_runs(self, tmp_path):
|
||||
h, store = self._after_update(tmp_path, status='verifying')
|
||||
h.now = NIGHT + 60
|
||||
assert h.updater.tick() is False
|
||||
assert store.updated_calls == [] and h.pending
|
||||
|
||||
def test_success_then_runs_the_deferred_plugins(self, tmp_path):
|
||||
h, store = self._after_update(tmp_path, status='success')
|
||||
assert h.updater.tick() is True
|
||||
result = h.state['last_result']
|
||||
assert result['core_outcome'] == 'updated' and result['status'] == 'success'
|
||||
assert result['plugins_updated'] == ['weather']
|
||||
assert h.restarts == ['ledmatrix']
|
||||
assert 'alert' not in h.state and h.pending is None
|
||||
|
||||
def test_rollback_alerts_and_remembers_the_bad_commit(self, tmp_path):
|
||||
h, store = self._after_update(tmp_path, status='rolled_back',
|
||||
reason='the web interface did not respond')
|
||||
h.updater.tick()
|
||||
assert 'rolled back' in h.state['alert']['message']
|
||||
assert 'the web interface did not respond' in h.state['alert']['message']
|
||||
assert h.state['rolled_back_head'] == 'b' * 40
|
||||
assert store.updated_calls == ['weather'], "the old code is healthy; plugins may update"
|
||||
|
||||
def test_failed_rollback_alerts_and_skips_plugins(self, tmp_path):
|
||||
h, store = self._after_update(tmp_path, status='rollback_failed',
|
||||
reason='display down', detail='reset failed')
|
||||
h.updater.tick()
|
||||
assert f'git reset --hard {"a" * 40}' in h.state['alert']['message']
|
||||
assert store.updated_calls == []
|
||||
assert h.state['plugins_pending'] is False
|
||||
|
||||
def test_a_check_that_never_reports_back_alerts(self, tmp_path):
|
||||
h, _ = self._after_update(tmp_path, status='verifying')
|
||||
h.now = NIGHT + au.VERIFY_LOST_SECONDS + 1
|
||||
h.updater.tick()
|
||||
assert h.state['last_result']['core_outcome'] == 'lost'
|
||||
assert 'never reported back' in h.state['alert']['message']
|
||||
|
||||
def test_dismissed_alert_stays_hidden_until_a_new_one(self, tmp_path):
|
||||
h, _ = self._after_update(tmp_path, status='rolled_back', reason='x')
|
||||
h.updater.tick()
|
||||
alert_id = h.state['alert']['id']
|
||||
|
||||
def status():
|
||||
return au.describe_status(h.config, state_file=h.updater.state_file,
|
||||
pending_file=h.updater.pending_file,
|
||||
setup_file=tmp_path / 'no-setup.json', helper=lambda: True)
|
||||
assert status()['alert_id'] == alert_id
|
||||
au.dismiss_alert(alert_id, h.updater.state_file)
|
||||
assert status()['alert'] is None
|
||||
state = h.state
|
||||
state['alert'] = {'id': 'newer', 'message': 'again'}
|
||||
au.save_state(state, h.updater.state_file)
|
||||
assert status()['alert'] == 'again'
|
||||
|
||||
|
||||
def test_describe_status_reports_setup(tmp_path):
|
||||
setup = tmp_path / 'setup.json'
|
||||
setup.write_text(json.dumps({'status': 'failed', 'message': 'not root'}))
|
||||
kwargs = dict(state={}, state_file=tmp_path / 's.json', pending_file=tmp_path / 'p.json',
|
||||
setup_file=setup)
|
||||
on = au.describe_status(ON, helper=lambda: False, **kwargs)
|
||||
assert on['verifier_installed'] is False
|
||||
assert (on['setup_status'], on['setup_message']) == ('failed', 'not root')
|
||||
|
||||
def never(): raise AssertionError("systemctl asked while updates are off")
|
||||
assert au.describe_status(OFF, helper=never, **kwargs)['verifier_installed'] is None
|
||||
|
||||
|
||||
# -- setting it up from the web UI ----------------------------------------------
|
||||
|
||||
class TestSetupFromTheWebUI:
|
||||
def _host(self, monkeypatch, ready=False, active=True, restart_ok=True):
|
||||
restarts = []
|
||||
monkeypatch.setattr(au, 'helper_ready', lambda: ready)
|
||||
monkeypatch.setattr(au, '_service_active', lambda unit: active)
|
||||
monkeypatch.setattr(au, 'restart_service', lambda unit: restarts.append(unit) or restart_ok)
|
||||
return restarts
|
||||
|
||||
def test_switching_on_restarts_the_display_to_finish_setup(self, monkeypatch):
|
||||
restarts = self._host(monkeypatch)
|
||||
note = au.start_setup_if_needed(False, ON)
|
||||
assert restarts == ['ledmatrix']
|
||||
assert 'display is restarting' in note
|
||||
|
||||
@pytest.mark.parametrize('was_enabled, config, ready', [
|
||||
(True, ON, False), # already on: an unrelated save must not restart the display
|
||||
(False, OFF, False), # still off
|
||||
(False, ON, True), # already set up
|
||||
])
|
||||
def test_nothing_to_do(self, monkeypatch, was_enabled, config, ready):
|
||||
restarts = self._host(monkeypatch, ready=ready)
|
||||
assert au.start_setup_if_needed(was_enabled, config) is None
|
||||
assert restarts == []
|
||||
|
||||
def test_a_stopped_display_is_not_started(self, monkeypatch):
|
||||
restarts = self._host(monkeypatch, active=False)
|
||||
assert 'next time the display service starts' in au.start_setup_if_needed(False, ON)
|
||||
assert restarts == []
|
||||
|
||||
def test_a_failed_restart_says_so(self, monkeypatch):
|
||||
self._host(monkeypatch, restart_ok=False)
|
||||
assert 'Could not restart' in au.start_setup_if_needed(False, ON)
|
||||
|
||||
|
||||
# -- plugins --------------------------------------------------------------------
|
||||
|
||||
class TestPlugins:
|
||||
def test_nothing_changed_restarts_nothing(self, tmp_path):
|
||||
store = FakeStore(tmp_path / 'plugins', {'clock': '1.0.0'})
|
||||
updater, restarts = stub_updater(tmp_path, store=store)
|
||||
updater.run()
|
||||
assert store.updated_calls == ['clock']
|
||||
assert restarts == []
|
||||
|
||||
def test_plugin_update_restarts_display_only(self, tmp_path):
|
||||
store = FakeStore(tmp_path / 'plugins', {'clock': '1.0.0', 'weather': '1.0.0'},
|
||||
bump={'weather'})
|
||||
updater, restarts = stub_updater(tmp_path, store=store)
|
||||
result = updater.run()
|
||||
assert result['plugins_updated'] == ['weather']
|
||||
assert restarts == ['ledmatrix']
|
||||
|
||||
def test_plugin_failure_alerts_but_keeps_weekly_cadence(self, tmp_path):
|
||||
store = FakeStore(tmp_path / 'plugins', {'zip-only': '1.0.0'}, fail={'zip-only'})
|
||||
updater, _ = stub_updater(tmp_path, store=store)
|
||||
result = updater.run()
|
||||
assert result['plugins_failed'] == ['zip-only'] and result['status'] == 'error'
|
||||
state = au.load_state(updater.state_file)
|
||||
assert 'zip-only' in state['alert']['message']
|
||||
assert state['next_due'] == NIGHT + au.UPDATE_INTERVAL_SECONDS
|
||||
|
||||
def test_local_only_plugins_are_left_alone(self, tmp_path):
|
||||
store = FakeStore(tmp_path / 'plugins', {'mine': '1.0.0'})
|
||||
path = tmp_path / 'plugins' / 'mine' / 'manifest.json'
|
||||
path.write_text(json.dumps({'id': 'mine', 'version': '1.0.0', 'local_only': True}))
|
||||
updater, _ = stub_updater(tmp_path, store=store)
|
||||
updater.run()
|
||||
assert store.updated_calls == []
|
||||
|
||||
|
||||
def test_restart_service_skips_a_stopped_unit(monkeypatch):
|
||||
calls = []
|
||||
|
||||
def run(args, **kwargs):
|
||||
calls.append(args)
|
||||
return MagicMock(stdout='inactive\n', returncode=3, stderr='')
|
||||
monkeypatch.setattr(au.subprocess, 'run', run)
|
||||
assert au.restart_service('ledmatrix') is False
|
||||
assert calls == [['systemctl', 'is-active', 'ledmatrix']], (
|
||||
"a display the user stopped must not be started by an update")
|
||||
|
||||
|
||||
def test_core_update_names_the_requirement_files_that_failed():
|
||||
"""The health check refuses code whose dependencies did not install, so
|
||||
the failure has to arrive as data, not only as a sentence in the message."""
|
||||
from web_interface.blueprints import api_v3 as pkg
|
||||
from web_interface.blueprints.api_v3 import system
|
||||
|
||||
heads = iter(['aaa111\n', 'bbb222\n'])
|
||||
|
||||
def run(args, **kwargs):
|
||||
if args[:2] == ['git', 'rev-parse'] and args[-1] == 'HEAD':
|
||||
out = next(heads, 'bbb222\n')
|
||||
elif args[:2] == ['git', 'diff']:
|
||||
out = 'requirements.txt\n'
|
||||
elif '@{u}' in args:
|
||||
out = 'origin/main\n'
|
||||
else:
|
||||
out = ''
|
||||
return subprocess.CompletedProcess(args, 0, stdout=out, stderr='')
|
||||
|
||||
pkg.api_v3.plugin_store_manager = None
|
||||
failed_install = subprocess.CompletedProcess([], 1, stdout='', stderr='boom')
|
||||
with patch.object(pkg.subprocess, 'run', run), \
|
||||
patch.object(system, '_pip_install_requirements', return_value=failed_install):
|
||||
result = system.perform_core_update()
|
||||
assert result['status'] == 'success'
|
||||
assert result['dependency_failures'] == ['requirements.txt']
|
||||
|
||||
|
||||
# -- web routes -------------------------------------------------------------------
|
||||
|
||||
@pytest.fixture
|
||||
def api_client(monkeypatch):
|
||||
from flask import Flask
|
||||
from web_interface.blueprints.api_v3 import api_v3
|
||||
app = Flask(__name__)
|
||||
app.register_blueprint(api_v3, url_prefix='/api/v3')
|
||||
cm = MagicMock()
|
||||
cm.load_config.return_value = {'timezone': 'UTC', 'auto_update': {'enabled': True}}
|
||||
cm.get_raw_file_content.return_value = {}
|
||||
cm.save_config_atomic.return_value = MagicMock(status=MagicMock(value='success'), message=None)
|
||||
api_v3.config_manager = cm
|
||||
api_v3.plugin_manager = MagicMock(plugins={})
|
||||
# Never restart a real display from a test run.
|
||||
setup_calls = []
|
||||
monkeypatch.setattr(au, 'start_setup_if_needed',
|
||||
lambda was_enabled, config: setup_calls.append(was_enabled) or None)
|
||||
return app.test_client(), cm, setup_calls
|
||||
|
||||
|
||||
class TestStatusRoutes:
|
||||
def test_status_carries_the_alert(self, api_client, monkeypatch):
|
||||
client, _, _ = api_client
|
||||
monkeypatch.setattr(au, 'describe_status', lambda config: {'alert': 'rolled back', 'alert_id': '7'})
|
||||
data = client.get('/api/v3/system/auto-update').get_json()['data']
|
||||
assert data == {'alert': 'rolled back', 'alert_id': '7'}
|
||||
|
||||
@pytest.mark.parametrize('body', [{}, [1], 'x', 5])
|
||||
def test_dismiss_needs_an_id_in_a_json_object(self, api_client, body):
|
||||
client, _, _ = api_client
|
||||
assert client.post('/api/v3/system/auto-update/dismiss', json=body).status_code == 400
|
||||
|
||||
def test_dismiss(self, api_client, monkeypatch):
|
||||
client, _, _ = api_client
|
||||
calls = []
|
||||
monkeypatch.setattr(au, 'dismiss_alert', calls.append)
|
||||
assert client.post('/api/v3/system/auto-update/dismiss', json={'alert_id': '7'}).status_code == 200
|
||||
assert calls == ['7']
|
||||
|
||||
|
||||
class TestSettingsSave:
|
||||
def _saved(self, cm):
|
||||
return cm.save_config_atomic.call_args[0][0]
|
||||
|
||||
def test_checked_toggle_saves_enabled(self, api_client):
|
||||
client, cm, setup_calls = api_client
|
||||
cm.load_config.return_value = {'timezone': 'UTC'}
|
||||
resp = client.post('/api/v3/config/main', json={'timezone': 'UTC', 'auto_update_enabled': 'true'})
|
||||
assert resp.status_code == 200
|
||||
saved = self._saved(cm)
|
||||
assert saved['auto_update'] == {'enabled': True}
|
||||
assert 'auto_update_enabled' not in saved, "the form field must not leak into config.json"
|
||||
assert setup_calls == [False], "setup is told the toggle was previously off"
|
||||
|
||||
def test_unchecked_toggle_on_general_save_disables(self, api_client):
|
||||
client, cm, setup_calls = api_client
|
||||
resp = client.post('/api/v3/config/main', json={'timezone': 'UTC'})
|
||||
assert resp.status_code == 200
|
||||
assert self._saved(cm)['auto_update'] == {'enabled': False}
|
||||
assert setup_calls == [True]
|
||||
|
||||
def test_setup_note_reaches_the_user(self, api_client, monkeypatch):
|
||||
client, cm, _ = api_client
|
||||
cm.load_config.return_value = {'timezone': 'UTC'}
|
||||
monkeypatch.setattr(au, 'start_setup_if_needed',
|
||||
lambda was_enabled, config: 'Finishing automatic update setup: the display is restarting.')
|
||||
body = client.post('/api/v3/config/main',
|
||||
json={'timezone': 'UTC', 'auto_update_enabled': 'true'}).get_json()
|
||||
assert 'the display is restarting' in body['message']
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Installing the automatic-update health check from the display service.
|
||||
|
||||
src/auto_update_setup.py is how a user who never opens a terminal gets updates
|
||||
with a safety net: it runs as root inside the display service and writes
|
||||
systemd units. So it has to install exactly the right thing when asked, do
|
||||
nothing when not asked, refuse templates that would run as anyone but the web
|
||||
user, and say why whenever it could not finish.
|
||||
"""
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from src import auto_update_setup as aus # noqa: E402
|
||||
|
||||
ON = {'auto_update': {'enabled': True}}
|
||||
|
||||
|
||||
class FakeSystemctl:
|
||||
def __init__(self, fail=(), activates=True):
|
||||
self.calls = []
|
||||
self.fail = set(fail)
|
||||
self.activates = activates
|
||||
self.active = False
|
||||
|
||||
def __call__(self, args, **kwargs):
|
||||
assert args[0] == 'systemctl', args
|
||||
self.calls.append(args[1:])
|
||||
verb = args[1]
|
||||
if verb == 'is-active':
|
||||
return subprocess.CompletedProcess(args, 0 if self.active else 3,
|
||||
stdout='active\n' if self.active else 'inactive\n', stderr='')
|
||||
if verb in self.fail:
|
||||
return subprocess.CompletedProcess(args, 1, stdout='', stderr=f'{verb} refused')
|
||||
if verb in ('restart', 'enable') and self.activates and (verb == 'restart' or '--now' in args):
|
||||
self.active = True
|
||||
return subprocess.CompletedProcess(args, 0, stdout='', stderr='')
|
||||
|
||||
|
||||
def project(tmp_path, user='hdpi', workdir=None, name='LEDMatrix'):
|
||||
root = tmp_path / name
|
||||
(root / 'systemd').mkdir(parents=True)
|
||||
for name in aus.UNITS:
|
||||
shutil.copy(ROOT / 'systemd' / name, root / 'systemd' / name)
|
||||
etc = tmp_path / 'etc'
|
||||
etc.mkdir()
|
||||
(etc / aus.WEB_UNIT).write_text(
|
||||
f'[Service]\nUser={user}\nWorkingDirectory={workdir or root}\n', encoding='utf-8')
|
||||
return root, etc
|
||||
|
||||
|
||||
def setup(root, etc, systemctl, root_user=True):
|
||||
return aus.UpdateHelperSetup(project_root=root, systemd_dir=etc, run=systemctl,
|
||||
is_root=lambda: root_user,
|
||||
lookup_ids=lambda user: None if user == 'ghost' else (1000, 1000),
|
||||
clock=lambda: 1234.0)
|
||||
|
||||
|
||||
def result(root):
|
||||
return json.loads((root / aus.RESULT_REL).read_text())
|
||||
|
||||
|
||||
def test_does_nothing_while_updates_are_off(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
systemctl = FakeSystemctl()
|
||||
assert setup(root, etc, systemctl).ensure({'auto_update': {'enabled': False}}) is None
|
||||
assert systemctl.calls == []
|
||||
assert not (etc / aus.SERVICE_UNIT).exists()
|
||||
assert not (root / aus.RESULT_REL).exists()
|
||||
|
||||
|
||||
def test_installs_both_units_for_the_web_user(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
systemctl = FakeSystemctl()
|
||||
out = setup(root, etc, systemctl).ensure(ON)
|
||||
|
||||
assert out['status'] == 'installed' and result(root)['status'] == 'installed'
|
||||
service = (etc / aus.SERVICE_UNIT).read_text()
|
||||
assert 'User=hdpi' in service
|
||||
assert f'ExecStart=/usr/bin/python3 "{root}/data/auto_update_verifier.py" "{root}"' in service
|
||||
assert f'PathExists={root}/data/auto_update_verify.request' in (etc / aus.PATH_UNIT).read_text()
|
||||
assert '__' not in service
|
||||
assert ['daemon-reload'] in systemctl.calls
|
||||
assert ['enable', aus.PATH_UNIT] in systemctl.calls
|
||||
assert systemctl.active
|
||||
|
||||
|
||||
def test_second_start_changes_nothing(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
systemctl = FakeSystemctl()
|
||||
setup(root, etc, systemctl).ensure(ON)
|
||||
first = result(root)
|
||||
systemctl.calls.clear()
|
||||
setup(root, etc, systemctl).ensure(ON)
|
||||
assert systemctl.calls == [['is-active', aus.PATH_UNIT], ['is-active', aus.PATH_UNIT]]
|
||||
assert result(root) == first, "an unchanged setup must not rewrite its result every boot"
|
||||
|
||||
|
||||
def test_a_changed_template_is_reinstalled(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
systemctl = FakeSystemctl()
|
||||
setup(root, etc, systemctl).ensure(ON)
|
||||
template = root / 'systemd' / aus.SERVICE_UNIT
|
||||
template.write_text(template.read_text() + '\n# newer\n')
|
||||
systemctl.calls.clear()
|
||||
assert setup(root, etc, systemctl).ensure(ON)['message'] == 'Installed the update health check.'
|
||||
assert (etc / aus.SERVICE_UNIT).read_text().endswith('# newer\n')
|
||||
assert ['daemon-reload'] in systemctl.calls
|
||||
|
||||
|
||||
def test_a_template_that_would_not_run_as_the_web_user_is_refused(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
template = root / 'systemd' / aus.SERVICE_UNIT
|
||||
template.write_text(template.read_text().replace('User=__USER__', 'User=root'))
|
||||
systemctl = FakeSystemctl()
|
||||
out = setup(root, etc, systemctl).ensure(ON)
|
||||
assert out['status'] == 'failed' and 'refusing' in out['message']
|
||||
assert not (etc / aus.SERVICE_UNIT).exists()
|
||||
assert systemctl.calls == []
|
||||
|
||||
|
||||
def test_a_path_unit_that_starts_something_else_is_refused(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
template = root / 'systemd' / aus.PATH_UNIT
|
||||
template.write_text(template.read_text().replace('Unit=ledmatrix-update-verify.service', 'Unit=other.service'))
|
||||
out = setup(root, etc, FakeSystemctl()).ensure(ON)
|
||||
assert out['status'] == 'failed' and not (etc / aus.PATH_UNIT).exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize('case, expected', [
|
||||
('not_root', 'not running as root'),
|
||||
('no_web_unit', 'not installed'),
|
||||
('other_folder', 'runs from'),
|
||||
('bad_user', 'not a usable account'),
|
||||
])
|
||||
def test_reports_why_it_could_not_install(tmp_path, case, expected):
|
||||
root, etc = project(tmp_path, user='ghost' if case == 'bad_user' else 'hdpi',
|
||||
workdir=tmp_path / 'elsewhere' if case == 'other_folder' else None)
|
||||
if case == 'no_web_unit':
|
||||
(etc / aus.WEB_UNIT).unlink()
|
||||
out = setup(root, etc, FakeSystemctl(), root_user=case != 'not_root').ensure(ON)
|
||||
assert out['status'] == 'failed' and expected in out['message']
|
||||
assert result(root)['message'] == out['message']
|
||||
assert not (etc / aus.SERVICE_UNIT).exists()
|
||||
|
||||
|
||||
def test_a_path_unit_that_will_not_start_is_a_failure(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
out = setup(root, etc, FakeSystemctl(activates=False)).ensure(ON)
|
||||
assert out['status'] == 'failed' and 'did not start' in out['message']
|
||||
|
||||
|
||||
def test_systemctl_errors_are_reported(tmp_path):
|
||||
root, etc = project(tmp_path)
|
||||
out = setup(root, etc, FakeSystemctl(fail={'daemon-reload'})).ensure(ON)
|
||||
assert out['status'] == 'failed' and 'daemon-reload refused' in out['message']
|
||||
|
||||
|
||||
def test_not_a_systemd_host_is_left_alone(tmp_path):
|
||||
root, _ = project(tmp_path)
|
||||
systemctl = FakeSystemctl()
|
||||
assert setup(root, tmp_path / 'no-etc', systemctl).ensure(ON) is None
|
||||
assert systemctl.calls == []
|
||||
|
||||
|
||||
def test_a_folder_with_spaces_is_quoted_in_the_commands(tmp_path):
|
||||
root, etc = project(tmp_path, name='LED Matrix')
|
||||
assert setup(root, etc, FakeSystemctl()).ensure(ON)['status'] == 'installed'
|
||||
service = (etc / aus.SERVICE_UNIT).read_text()
|
||||
assert f'ExecStartPre=/bin/rm -f "{root}/data/auto_update_verify.request"' in service
|
||||
assert f'ExecStart=/usr/bin/python3 "{root}/data/auto_update_verifier.py" "{root}"' in service
|
||||
|
||||
|
||||
def test_a_folder_name_systemd_would_reinterpret_is_refused(tmp_path):
|
||||
root, etc = project(tmp_path, name='LED%Matrix')
|
||||
systemctl = FakeSystemctl()
|
||||
out = setup(root, etc, systemctl).ensure(ON)
|
||||
assert out['status'] == 'failed' and 'systemd cannot use' in out['message']
|
||||
assert not (etc / aus.SERVICE_UNIT).exists() and systemctl.calls == []
|
||||
|
||||
|
||||
def test_installer_sets_the_toggle_without_a_half_written_config(tmp_path):
|
||||
"""first_time_install.sh may run while the display service watches
|
||||
config.json; the file must be replaced whole, never truncated in place."""
|
||||
import re
|
||||
text = (ROOT / 'first_time_install.sh').read_text(encoding='utf-8').replace('\r\n', '\n')
|
||||
script = next(s for s in re.findall(r"<<'PY'\n(.*?)\nPY\n", text, re.S) if 'auto_update' in s)
|
||||
assert 'os.replace(' in script
|
||||
config = tmp_path / 'config.json'
|
||||
config.write_text(json.dumps({'timezone': 'UTC', 'auto_update': {'enabled': False}}))
|
||||
run = subprocess.run([sys.executable, '-', str(config), '1'], input=script, text=True,
|
||||
capture_output=True)
|
||||
assert run.returncode == 0, run.stderr
|
||||
assert json.loads(config.read_text()) == {'timezone': 'UTC', 'auto_update': {'enabled': True}}
|
||||
assert [p.name for p in tmp_path.iterdir()] == ['config.json'], "a temp file was left behind"
|
||||
|
||||
|
||||
def test_the_result_is_kept_when_it_cannot_be_given_to_the_web_user(tmp_path, monkeypatch):
|
||||
"""Caught by CI, which runs as a non-root Linux user: chown needs root, and
|
||||
a failed chown used to discard the result, so the General tab never
|
||||
learned whether setup worked."""
|
||||
def refuse(*args):
|
||||
raise PermissionError('Operation not permitted')
|
||||
monkeypatch.setattr(aus.os, 'chown', refuse, raising=False)
|
||||
root, etc = project(tmp_path)
|
||||
out = setup(root, etc, FakeSystemctl()).ensure(ON)
|
||||
assert out['status'] == 'installed'
|
||||
assert result(root) == out
|
||||
@@ -0,0 +1,248 @@
|
||||
"""The automatic update's health check and rollback (scripts/utils/auto_update_verify.py).
|
||||
|
||||
This is what stands between an unattended update and a device that no longer
|
||||
works, so each way an update can fail is exercised against a real git repo:
|
||||
the new commit is checked out, the services are "restarted", and whether they
|
||||
come up healthy depends on which commit they were restarted onto. Only
|
||||
systemd, sudo and the HTTP check are faked.
|
||||
"""
|
||||
import importlib.util
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
'auto_update_verify', ROOT / 'scripts' / 'utils' / 'auto_update_verify.py')
|
||||
av = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(av)
|
||||
|
||||
pytestmark = pytest.mark.skipif(shutil.which('git') is None, reason='git not installed')
|
||||
|
||||
|
||||
def git(cwd, *args):
|
||||
result = subprocess.run(['git', '-c', 'user.name=t', '-c', 'user.email=t@example.com', *args],
|
||||
cwd=str(cwd), capture_output=True, text=True)
|
||||
assert result.returncode == 0, result.stderr
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def done(args, stdout='', rc=0):
|
||||
return subprocess.CompletedProcess(args, rc, stdout=stdout, stderr='' if rc == 0 else 'failed')
|
||||
|
||||
|
||||
def updated_repo(tmp_path, new_requirements=False):
|
||||
"""A checkout on a new commit, with the commit it came from."""
|
||||
repo = tmp_path / 'LEDMatrix'
|
||||
repo.mkdir()
|
||||
git(repo, 'init', '-q', '-b', 'main')
|
||||
(repo / 'app.py').write_text('v = 1\n')
|
||||
(repo / 'requirements.txt').write_text('requests\n')
|
||||
git(repo, 'add', '.')
|
||||
git(repo, 'commit', '-qm', 'old')
|
||||
old = git(repo, 'rev-parse', 'HEAD')
|
||||
(repo / 'app.py').write_text('v = 2\n')
|
||||
if new_requirements:
|
||||
(repo / 'requirements.txt').write_text('requests\nnewthing\n')
|
||||
git(repo, 'commit', '-qam', 'new')
|
||||
return repo, old, git(repo, 'rev-parse', 'HEAD')
|
||||
|
||||
|
||||
class FakeHost:
|
||||
"""systemd, sudo and the web interface.
|
||||
|
||||
Services run whatever commit was checked out when they were last
|
||||
restarted; ``failure`` says how they misbehave on the new commit
|
||||
("display_down", "web_down", "crash_loop") or on any commit ("always").
|
||||
"""
|
||||
|
||||
def __init__(self, repo, bad_head, failure=None, pip_ok=True, restart_failures=0, count_readable=True):
|
||||
self.repo, self.bad_head, self.failure, self.pip_ok = repo, bad_head, failure, pip_ok
|
||||
self.restart_failures = restart_failures # how many restart commands fail, first to last
|
||||
self.count_readable = count_readable
|
||||
self.running_head = None # not restarted yet: still the old, healthy code
|
||||
self.restarts = [] # (unit, commit it was restarted onto)
|
||||
self.pip_installs = []
|
||||
self.now = 0.0
|
||||
self.nrestarts = 0
|
||||
|
||||
def broken(self, kind):
|
||||
if self.running_head is None:
|
||||
return False
|
||||
return self.failure == 'always' or (self.running_head == self.bad_head and self.failure == kind)
|
||||
|
||||
def run(self, args, **kwargs):
|
||||
if args[0] == 'git':
|
||||
return subprocess.run(args, **kwargs)
|
||||
if args[:2] == ['systemctl', 'is-active']:
|
||||
return done(args, 'failed\n' if self.broken('display_down') else 'active\n')
|
||||
if args[:2] == ['systemctl', 'show']:
|
||||
if not self.count_readable:
|
||||
return done(args, rc=1)
|
||||
if self.broken('crash_loop'):
|
||||
self.nrestarts += 1
|
||||
return done(args, f'{self.nrestarts}\n')
|
||||
if args[0] == 'sudo' and any(a.endswith('safe_pip_install.sh') for a in args):
|
||||
self.pip_installs.append(args[-1])
|
||||
return done(args, rc=0 if self.pip_ok else 1)
|
||||
if args[:4] == ['sudo', '-n', 'systemctl', 'restart']:
|
||||
if self.restart_failures:
|
||||
self.restart_failures -= 1
|
||||
return done(args, rc=1) # the old process keeps running
|
||||
self.running_head = git(self.repo, 'rev-parse', 'HEAD')
|
||||
self.restarts.append((args[4], self.running_head))
|
||||
return done(args)
|
||||
raise AssertionError(f'unexpected command: {args}')
|
||||
|
||||
def web_responds(self):
|
||||
return not self.broken('web_down')
|
||||
|
||||
def sleep(self, seconds):
|
||||
self.now += seconds
|
||||
|
||||
def verifier(self):
|
||||
return av.Verifier(self.repo, run=self.run, sleep=self.sleep, clock=lambda: self.now,
|
||||
web_responds=self.web_responds, log=lambda msg: None)
|
||||
|
||||
|
||||
def check(tmp_path, failure=None, new_requirements=False, pip_ok=True, restart_failures=0,
|
||||
count_readable=True, **pending):
|
||||
repo, old, new = updated_repo(tmp_path, new_requirements)
|
||||
fields = {'status': 'pending', 'old_head': old, 'new_head': new,
|
||||
'display_was_active': True, 'dependency_failures': []}
|
||||
fields.update(pending)
|
||||
av.write_pending(av.pending_path(repo), fields)
|
||||
host = FakeHost(repo, new, failure, pip_ok, restart_failures, count_readable)
|
||||
code = host.verifier().verify()
|
||||
result = av.read_pending(av.pending_path(repo))
|
||||
return code, result, host, git(repo, 'rev-parse', 'HEAD'), old, new
|
||||
|
||||
|
||||
def test_a_healthy_update_is_kept(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path)
|
||||
assert code == 0 and result['status'] == 'success'
|
||||
assert head == new
|
||||
assert host.restarts == [('ledmatrix.service', new), ('ledmatrix-web.service', new)]
|
||||
|
||||
|
||||
@pytest.mark.parametrize('failure, reason', [
|
||||
('display_down', 'the display service did not stay running'),
|
||||
('web_down', 'the web interface did not respond'),
|
||||
('crash_loop', 'the display service kept restarting'),
|
||||
])
|
||||
def test_an_unhealthy_update_is_rolled_back(tmp_path, failure, reason):
|
||||
code, result, host, head, old, new = check(tmp_path, failure)
|
||||
assert code == 0
|
||||
assert result['status'] == 'rolled_back' and result['reason'] == reason
|
||||
assert head == old
|
||||
assert host.restarts[-2:] == [('ledmatrix.service', old), ('ledmatrix-web.service', old)]
|
||||
|
||||
|
||||
def test_a_stopped_display_is_not_started(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, display_was_active=False)
|
||||
assert result['status'] == 'success'
|
||||
assert [unit for unit, _ in host.restarts] == ['ledmatrix-web.service']
|
||||
|
||||
|
||||
def test_failed_dependencies_roll_back_before_anything_restarts_onto_them(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, dependency_failures=['requirements.txt'])
|
||||
assert result['status'] == 'rolled_back'
|
||||
assert 'requirements.txt' in result['reason']
|
||||
assert head == old
|
||||
assert all(commit == old for _, commit in host.restarts), host.restarts
|
||||
|
||||
|
||||
def test_rollback_reinstalls_the_previous_dependencies(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, 'display_down', new_requirements=True)
|
||||
assert result['status'] == 'rolled_back' and result['detail'] is None
|
||||
assert host.pip_installs == [str(tmp_path / 'LEDMatrix' / 'requirements.txt')]
|
||||
|
||||
|
||||
def test_a_failed_dependency_reinstall_is_reported(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, 'display_down', new_requirements=True,
|
||||
pip_ok=False)
|
||||
assert result['status'] == 'rolled_back' and head == old
|
||||
assert 'Install Base Requirements' in result['detail']
|
||||
|
||||
|
||||
def test_still_broken_after_rolling_back_is_rollback_failed(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, 'always')
|
||||
assert code == 1 and result['status'] == 'rollback_failed'
|
||||
assert 'still unhealthy after rolling back' in result['detail']
|
||||
|
||||
|
||||
def test_a_failed_restart_is_not_mistaken_for_a_healthy_update(tmp_path):
|
||||
"""When the restart command itself fails the old process keeps answering,
|
||||
so checking it would pass an update that never started."""
|
||||
code, result, host, head, old, new = check(tmp_path, restart_failures=1)
|
||||
assert result['status'] == 'rolled_back'
|
||||
assert result['reason'] == 'restarting the services failed'
|
||||
assert head == old
|
||||
|
||||
|
||||
def test_restarts_that_keep_failing_after_rollback_are_rollback_failed(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, restart_failures=100)
|
||||
assert code == 1 and result['status'] == 'rollback_failed'
|
||||
assert 'restarting the services failed' in result['detail']
|
||||
|
||||
|
||||
def test_an_unreadable_restart_count_is_never_called_stable(tmp_path):
|
||||
"""With no restart count a crash loop looks healthy between attempts."""
|
||||
code, result, host, head, old, new = check(tmp_path, count_readable=False)
|
||||
assert result['status'] != 'success'
|
||||
assert 'restart count could not be read' in result['reason']
|
||||
|
||||
|
||||
def test_nothing_pending_does_nothing(tmp_path):
|
||||
repo, _, _ = updated_repo(tmp_path)
|
||||
host = FakeHost(repo, None)
|
||||
assert host.verifier().verify() == 0
|
||||
assert host.restarts == []
|
||||
|
||||
|
||||
def test_a_crash_is_recorded_not_left_as_running(tmp_path, monkeypatch):
|
||||
repo, old, new = updated_repo(tmp_path)
|
||||
av.write_pending(av.pending_path(repo), {'status': 'pending', 'old_head': old, 'new_head': new})
|
||||
|
||||
def boom(self):
|
||||
raise RuntimeError('boom')
|
||||
monkeypatch.setattr(av.Verifier, 'verify', boom)
|
||||
assert av.main(['auto_update_verify.py', str(repo)]) == 1
|
||||
result = av.read_pending(av.pending_path(repo))
|
||||
assert result['status'] == 'rollback_failed' and result['detail'] == 'boom'
|
||||
|
||||
|
||||
def test_units_installers_and_updater_agree():
|
||||
"""The request file, the copied verifier, the unit names and the places
|
||||
that install them must all line up, or the health check silently never
|
||||
starts on real devices -- and code updates stay paused."""
|
||||
from web_interface import auto_update as au
|
||||
from src import auto_update_setup as aus
|
||||
from src.startup_validator import StartupValidator
|
||||
|
||||
service = (ROOT / 'systemd' / aus.SERVICE_UNIT).read_text(encoding='utf-8')
|
||||
path = (ROOT / 'systemd' / aus.PATH_UNIT).read_text(encoding='utf-8')
|
||||
request = f'__PROJECT_ROOT_DIR__/{au.REQUEST_REL.as_posix()}'
|
||||
assert f'PathExists={request}' in path
|
||||
assert f'Unit={aus.SERVICE_UNIT}' in path
|
||||
assert f'ExecStartPre=/bin/rm -f "{request}"' in service, "the request must be consumed or the path unit re-fires"
|
||||
assert f'"__PROJECT_ROOT_DIR__/{au.VERIFIER_COPY_REL.as_posix()}" "__PROJECT_ROOT_DIR__"' in service
|
||||
assert au.PENDING_REL.name == av.PENDING_NAME
|
||||
assert au.PATH_UNIT == aus.PATH_UNIT and au.SETUP_RESULT_REL == aus.RESULT_REL
|
||||
|
||||
for installer in ('scripts/install/install_web_service.sh', 'scripts/install/install_service.sh'):
|
||||
text = (ROOT / installer).read_text(encoding='utf-8')
|
||||
assert 'ledmatrix-update-verify.service ledmatrix-update-verify.path' in text, installer
|
||||
assert 'enable --now ledmatrix-update-verify.path' in text, installer
|
||||
for unit in aus.UNITS:
|
||||
assert (f'systemd/{unit}', f'/etc/systemd/system/{unit}') in StartupValidator._UNITS
|
||||
|
||||
# Triggering takes no privilege any more; no sudoers rule should linger.
|
||||
for sudoers in ('scripts/install/configure_web_sudo.sh', 'first_time_install.sh'):
|
||||
assert not re.search(r'NOPASSWD:.*update-verify', (ROOT / sudoers).read_text(encoding='utf-8')), sudoers
|
||||
@@ -0,0 +1,163 @@
|
||||
"""src/common must stay importable without display hardware.
|
||||
|
||||
Plugins import src.common.sports_* in place of their bundled copies. If any of
|
||||
those modules reaches src.display_manager (and through it rgbmatrix) or the
|
||||
src.base_classes package (whose core.py imports DisplayManager), a scoreboard
|
||||
adopting it acquires a hardware dependency it never had, and the headless
|
||||
tooling -- the web preview, check_plugin.py, these tests on a laptop -- stops
|
||||
being able to load it.
|
||||
|
||||
Two checks, because each misses what the other catches:
|
||||
|
||||
1. A real import in a fresh interpreter with rgbmatrix made unimportable.
|
||||
Catches indirect imports (common -> X -> display_manager) that no text scan
|
||||
of src/common would see.
|
||||
2. An AST scan of src/common/*.py for module-level imports of the hardware and
|
||||
plugin-system packages. Catches a direct import even where the target
|
||||
happens to import cleanly on this machine.
|
||||
"""
|
||||
|
||||
import ast
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
COMMON = REPO_ROOT / "src" / "common"
|
||||
|
||||
FORBIDDEN = ("src.base_classes", "src.display_manager", "src.plugin_system")
|
||||
|
||||
#: Existing module-level violations, by file name, each with the reason it is
|
||||
#: tolerated. Empty when this test was added (core 3.4.0 + sports_helpers):
|
||||
#: no src/common module imported any of FORBIDDEN at module level. Add to this
|
||||
#: only with a comment saying why, never to get a new import past the test.
|
||||
ALLOWLIST = {}
|
||||
|
||||
|
||||
def test_sports_modules_import_without_hardware():
|
||||
modules = sorted(p.stem for p in COMMON.glob("sports_*.py"))
|
||||
assert "sports_helpers" in modules
|
||||
script = textwrap.dedent(f"""
|
||||
import importlib, json, sys
|
||||
sys.modules["rgbmatrix"] = None # any import of it now raises
|
||||
importlib.import_module("src.common")
|
||||
for name in {modules!r}:
|
||||
importlib.import_module("src.common." + name)
|
||||
loaded = sorted(m for m in sys.modules
|
||||
if m.startswith({FORBIDDEN!r}))
|
||||
print(json.dumps(loaded))
|
||||
""")
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-c", script], cwd=str(REPO_ROOT),
|
||||
capture_output=True, text=True, timeout=120)
|
||||
assert result.returncode == 0, (
|
||||
f"importing src.common without rgbmatrix failed:\n{result.stderr}")
|
||||
loaded = result.stdout.strip().splitlines()[-1]
|
||||
assert loaded == "[]", f"src.common pulled in hardware modules: {loaded}"
|
||||
|
||||
|
||||
def _module_level_imports(tree):
|
||||
"""Imports executed at import time: top level, including inside if/try
|
||||
blocks, but not inside functions or classes (those are deferred)."""
|
||||
stack = list(tree.body)
|
||||
while stack:
|
||||
node = stack.pop()
|
||||
if isinstance(node, (ast.Import, ast.ImportFrom)):
|
||||
yield node
|
||||
elif isinstance(node, (ast.If, ast.Try, ast.With)):
|
||||
stack.extend(node.body)
|
||||
stack.extend(getattr(node, "orelse", []))
|
||||
stack.extend(getattr(node, "finalbody", []))
|
||||
for handler in getattr(node, "handlers", []):
|
||||
stack.extend(handler.body)
|
||||
|
||||
|
||||
def _targets(node, package="src.common"):
|
||||
"""Absolute dotted names an import can load. Relative imports resolve
|
||||
against ``package``, the scanned module's package."""
|
||||
if isinstance(node, ast.Import):
|
||||
return [alias.name for alias in node.names]
|
||||
base = node.module
|
||||
if node.level:
|
||||
parts = package.split(".")
|
||||
if node.level > len(parts):
|
||||
return [] # beyond the top-level package; Python rejects it too
|
||||
anchor = ".".join(parts[:len(parts) - node.level + 1])
|
||||
base = f"{anchor}.{node.module}" if node.module else anchor
|
||||
if base is None:
|
||||
return []
|
||||
# `from pkg import name` may load the submodule pkg.name.
|
||||
return [base] + [f"{base}.{alias.name}" for alias in node.names
|
||||
if alias.name != "*"]
|
||||
|
||||
|
||||
def _violations():
|
||||
found = {}
|
||||
for path in sorted(COMMON.glob("*.py")):
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
|
||||
for node in _module_level_imports(tree):
|
||||
for target in _targets(node, "src.common"):
|
||||
if target == FORBIDDEN or any(
|
||||
target == f or target.startswith(f + ".") for f in FORBIDDEN):
|
||||
found.setdefault(path.name, []).append(f"line {node.lineno}: {target}")
|
||||
return found
|
||||
|
||||
|
||||
def test_no_module_level_hardware_or_plugin_system_imports():
|
||||
found = {name: hits for name, hits in _violations().items() if name not in ALLOWLIST}
|
||||
assert found == {}, (
|
||||
"src/common must not import these at module level (defer the import "
|
||||
f"into the function that needs it): {found}")
|
||||
|
||||
|
||||
def test_allowlist_has_no_stale_entries():
|
||||
stale = sorted(set(ALLOWLIST) - set(_violations()))
|
||||
assert stale == [], f"allowlisted but no longer violating, remove: {stale}"
|
||||
|
||||
|
||||
def test_the_scan_sees_a_direct_import(tmp_path):
|
||||
# Guard the guard: the walker must find imports nested in try/if.
|
||||
tree = ast.parse(textwrap.dedent("""
|
||||
try:
|
||||
from src.display_manager import DisplayManager
|
||||
except ImportError:
|
||||
pass
|
||||
if True:
|
||||
import src.base_classes.sports
|
||||
from src import plugin_system
|
||||
def later():
|
||||
from src.display_manager import DisplayManager
|
||||
"""))
|
||||
targets = {t for n in _module_level_imports(tree) for t in _targets(n)}
|
||||
assert {"src.base_classes.sports", "src.display_manager",
|
||||
"src.plugin_system"} <= targets
|
||||
|
||||
|
||||
def test_the_scan_resolves_relative_imports():
|
||||
# A module in src/common reaches src.plugin_system through `..`.
|
||||
tree = ast.parse(textwrap.dedent("""
|
||||
from .. import plugin_system
|
||||
from ..plugin_system import plugin_manager
|
||||
from . import sports_shared
|
||||
"""))
|
||||
targets = {t for n in _module_level_imports(tree)
|
||||
for t in _targets(n, "src.common")}
|
||||
assert {"src.plugin_system", "src.plugin_system.plugin_manager",
|
||||
"src.common.sports_shared"} <= targets
|
||||
# The package decides where `..` points.
|
||||
node = ast.parse("from .. import plugin_system").body[0]
|
||||
assert _targets(node, "src.common") == ["src", "src.plugin_system"]
|
||||
assert _targets(node, "a.b") == ["a", "a.plugin_system"]
|
||||
|
||||
|
||||
def test_the_scan_flags_relative_imports_in_src_common(monkeypatch, tmp_path):
|
||||
# End to end through _violations(): both relative forms are reported.
|
||||
(tmp_path / "bad.py").write_text(
|
||||
"from .. import plugin_system\nfrom ..plugin_system import x\n",
|
||||
encoding="utf-8")
|
||||
monkeypatch.setitem(globals(), "COMMON", tmp_path)
|
||||
found = {name: sorted(hits) for name, hits in _violations().items()}
|
||||
assert found == {"bad.py": ["line 1: src.plugin_system",
|
||||
"line 2: src.plugin_system",
|
||||
"line 2: src.plugin_system.x"]}
|
||||
@@ -0,0 +1,603 @@
|
||||
"""src.common.sports_helpers: behaviour, host contract, and parity with the plugins.
|
||||
|
||||
The bodies were copied from the scoreboard plugins' sports.py, so the plugins'
|
||||
own tests already cover them there. They are ported here for two reasons: core
|
||||
now owns a copy that can drift on its own, and a plugin that deletes its copy
|
||||
loses those tests with it.
|
||||
|
||||
The parity class is what keeps "byte-identical" true after this lands. Point
|
||||
LEDMATRIX_PLUGINS at a ledmatrix-plugins checkout and every promoted body is
|
||||
compared, as a docstring-stripped AST, against every plugin copy that carries
|
||||
it. Without the variable it skips rather than fails, since core CI has no
|
||||
plugins checkout; ledmatrix-plugins CI runs the same comparison against core
|
||||
(scripts/check_sports_helpers_parity.py, ledmatrix-plugins#495).
|
||||
"""
|
||||
|
||||
import ast
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
from PIL.PngImagePlugin import PngInfo
|
||||
|
||||
from src.common import sports_helpers
|
||||
from src.common.sports_helpers import (
|
||||
MAX_WINDOW_DAYS,
|
||||
MIN_WINDOW_DAYS,
|
||||
SportsHelpersMixin,
|
||||
clamp_seconds,
|
||||
clamp_window,
|
||||
logo_needs_refresh,
|
||||
spread_weighted_order,
|
||||
)
|
||||
from src.common.sports_shared import SportsCoreSharedMixin
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Free functions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestClampWindow:
|
||||
def test_within_range_is_kept(self):
|
||||
assert clamp_window(14, 7) == 14
|
||||
assert clamp_window("30", 7) == 30
|
||||
|
||||
def test_clamped_to_bounds(self):
|
||||
assert (MIN_WINDOW_DAYS, MAX_WINDOW_DAYS) == (1, 60)
|
||||
assert clamp_window(0, 7) == 1
|
||||
assert clamp_window(-5, 7) == 1
|
||||
assert clamp_window(365, 7) == 60
|
||||
|
||||
@pytest.mark.parametrize("bad", [None, "abc", [], {}, float("nan")])
|
||||
def test_unusable_falls_back(self, bad):
|
||||
assert clamp_window(bad, 7) == 7
|
||||
|
||||
@pytest.mark.parametrize("inf", [float("inf"), float("-inf")])
|
||||
def test_infinity_falls_back_instead_of_raising(self, inf):
|
||||
# json.loads accepts a bare Infinity; int(inf) raises OverflowError.
|
||||
with pytest.raises(OverflowError):
|
||||
int(inf)
|
||||
assert clamp_window(inf, 7) == 7
|
||||
|
||||
|
||||
class TestClampSeconds:
|
||||
def test_defaults_bound_five_seconds_to_a_day(self):
|
||||
assert clamp_seconds(1, 60) == 5
|
||||
assert clamp_seconds(10 ** 9, 60) == 86400
|
||||
assert clamp_seconds(300, 60) == 300
|
||||
|
||||
def test_custom_bounds(self):
|
||||
assert clamp_seconds(1, 60, low=30, high=900) == 30
|
||||
assert clamp_seconds(5000, 60, low=30, high=900) == 900
|
||||
|
||||
@pytest.mark.parametrize("bad", [None, "x", float("nan"), float("inf")])
|
||||
def test_unusable_falls_back(self, bad):
|
||||
assert clamp_seconds(bad, 60) == 60
|
||||
|
||||
|
||||
def _png(path, size=(64, 64), color=(100, 100, 100, 255), marker=None):
|
||||
img = Image.new("RGBA", size, color)
|
||||
info = None
|
||||
if marker is not None:
|
||||
info = PngInfo()
|
||||
info.add_text("ledmatrix_placeholder", marker)
|
||||
img.save(path, "PNG", pnginfo=info)
|
||||
return path
|
||||
|
||||
|
||||
class TestLogoNeedsRefresh:
|
||||
def test_stale_placeholder_is_retried(self, tmp_path):
|
||||
from src.logo_downloader import PLACEHOLDER_RETRY_SECONDS
|
||||
stamp = str(time.time() - PLACEHOLDER_RETRY_SECONDS - 60)
|
||||
assert logo_needs_refresh(_png(tmp_path / "A.png", marker=stamp)) is True
|
||||
|
||||
def test_fresh_placeholder_is_trusted(self, tmp_path):
|
||||
assert logo_needs_refresh(_png(tmp_path / "A.png", marker=str(time.time()))) is False
|
||||
|
||||
def test_real_logo_is_never_refreshed(self, tmp_path):
|
||||
assert logo_needs_refresh(_png(tmp_path / "A.png", size=(32, 32),
|
||||
color=(200, 0, 0, 255))) is False
|
||||
|
||||
def test_missing_file_is_not_a_placeholder(self, tmp_path):
|
||||
assert logo_needs_refresh(tmp_path / "nope.png") is False
|
||||
|
||||
def test_core_without_placeholder_marking_keeps_old_behaviour(self, monkeypatch, tmp_path):
|
||||
# A None entry in sys.modules makes the import raise ImportError, the
|
||||
# same thing an older core without these names does.
|
||||
monkeypatch.setitem(__import__("sys").modules, "src.logo_downloader", None)
|
||||
stamp = str(time.time() - 10 ** 7)
|
||||
assert logo_needs_refresh(_png(tmp_path / "A.png", marker=stamp)) is False
|
||||
|
||||
def test_the_import_is_deferred(self):
|
||||
tree = ast.parse(Path(sports_helpers.__file__).read_text(encoding="utf-8"))
|
||||
top = [n for n in tree.body if isinstance(n, (ast.Import, ast.ImportFrom))]
|
||||
assert not any(isinstance(n, ast.ImportFrom) and n.module == "src.logo_downloader"
|
||||
for n in top)
|
||||
|
||||
|
||||
def _back_to_back(cycle):
|
||||
return any(cycle[i] == cycle[(i + 1) % len(cycle)] for i in range(len(cycle)))
|
||||
|
||||
|
||||
class TestSpreadWeightedOrder:
|
||||
def test_equal_weights_are_plain_order(self):
|
||||
assert spread_weighted_order([1, 1, 1]) == [0, 1, 2]
|
||||
assert spread_weighted_order([]) == []
|
||||
|
||||
def test_a_favourite_at_the_end_wraps_its_extra_turn(self):
|
||||
order = spread_weighted_order([1, 1, 1, 2])
|
||||
assert sorted(order) == [0, 1, 2, 3, 3]
|
||||
assert not _back_to_back(order)
|
||||
|
||||
@pytest.mark.parametrize("weights,expected", [
|
||||
([3, 1, 1], [0, 1, 0, 2, 0]),
|
||||
([1, 3, 1], [0, 1, 1, 2, 1]),
|
||||
([5, 1], None),
|
||||
([2, 1], None),
|
||||
])
|
||||
def test_ratio_wins_over_spacing(self, weights, expected):
|
||||
order = spread_weighted_order(weights)
|
||||
assert [order.count(i) for i in range(len(weights))] == weights
|
||||
if expected is not None:
|
||||
assert order == expected
|
||||
|
||||
def test_mixin_exposes_it_as_a_staticmethod(self):
|
||||
assert SportsHelpersMixin._spread_weighted_order([1, 2]) == spread_weighted_order([1, 2])
|
||||
assert isinstance(SportsHelpersMixin.__dict__["_spread_weighted_order"], staticmethod)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Mixin methods, against stand-in hosts carrying only what each reads
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _Dwell(SportsHelpersMixin):
|
||||
def __init__(self, last_game_switch):
|
||||
self._last_display_call_monotonic = 0.0
|
||||
self.last_game_switch = last_game_switch
|
||||
|
||||
|
||||
class TestResetDwellOnReentry:
|
||||
def test_first_frame_resets(self):
|
||||
mgr = _Dwell(time.time() - 6.0)
|
||||
assert mgr._reset_dwell_on_reentry() is True
|
||||
assert time.time() - mgr.last_game_switch < 1.0
|
||||
|
||||
def test_frames_inside_one_stint_do_not(self):
|
||||
mgr = _Dwell(time.time() - 6.0)
|
||||
mgr._reset_dwell_on_reentry()
|
||||
stamp = mgr.last_game_switch
|
||||
assert [mgr._reset_dwell_on_reentry() for _ in range(3)] == [False] * 3
|
||||
assert mgr.last_game_switch == stamp
|
||||
|
||||
def test_returning_after_a_long_gap_resets(self):
|
||||
mgr = _Dwell(time.time() - 150.0)
|
||||
mgr._last_display_call_monotonic = time.monotonic() - 150.0
|
||||
assert mgr._reset_dwell_on_reentry() is True
|
||||
|
||||
def test_just_under_the_threshold_is_the_same_stint(self):
|
||||
mgr = _Dwell(time.time() - 150.0)
|
||||
mgr._last_display_call_monotonic = time.monotonic() - (
|
||||
SportsHelpersMixin._DWELL_REENTRY_GAP_SECONDS - 1.0)
|
||||
stamp = mgr.last_game_switch
|
||||
assert mgr._reset_dwell_on_reentry() is False
|
||||
assert mgr.last_game_switch == stamp
|
||||
|
||||
def test_live_zero_sentinel_survives(self):
|
||||
mgr = _Dwell(0)
|
||||
assert mgr._reset_dwell_on_reentry() is False
|
||||
assert mgr.last_game_switch == 0
|
||||
|
||||
def test_host_without_any_state(self):
|
||||
mgr = SportsHelpersMixin()
|
||||
assert mgr._reset_dwell_on_reentry() is False # no last_game_switch
|
||||
assert mgr._last_display_call_monotonic > 0
|
||||
|
||||
|
||||
def _game(gid, home="X", away="Y"):
|
||||
return {"id": gid, "home_abbr": home, "away_abbr": away}
|
||||
|
||||
|
||||
class _Switch(SportsHelpersMixin):
|
||||
def __init__(self, games, boost=None, favorites=("UF",)):
|
||||
self.games_list = games
|
||||
self.favorite_teams = list(favorites)
|
||||
if boost is not None:
|
||||
self.favorite_rotation_boost = boost
|
||||
self.current_game_index = 0
|
||||
|
||||
def _is_favorite_game(self, g):
|
||||
return g["home_abbr"] in self.favorite_teams or g["away_abbr"] in self.favorite_teams
|
||||
|
||||
def walk(self, steps):
|
||||
shown = []
|
||||
for _ in range(steps):
|
||||
self.current_game_index = self._next_switch_index()
|
||||
shown.append(self.games_list[self.current_game_index]["id"])
|
||||
return shown
|
||||
|
||||
|
||||
class TestNextSwitchIndex:
|
||||
def test_no_boost_attribute_is_the_plain_rotation(self):
|
||||
s = _Switch([_game("a"), _game("b", "UF"), _game("c")])
|
||||
assert s.walk(6) == ["b", "c", "a", "b", "c", "a"]
|
||||
|
||||
def test_boost_without_a_favourite_changes_nothing(self):
|
||||
assert _Switch([_game("a"), _game("b"), _game("c")], boost=3).walk(3) == ["b", "c", "a"]
|
||||
|
||||
def test_boost_two_spreads_the_favourite(self):
|
||||
s = _Switch([_game("a"), _game("uf", "UF"), _game("c"), _game("d")], boost=2)
|
||||
cycle = s.walk(5)
|
||||
assert sorted(cycle) == ["a", "c", "d", "uf", "uf"]
|
||||
assert not _back_to_back(cycle)
|
||||
assert s.walk(5) == cycle
|
||||
|
||||
def test_boost_three_two_favourites(self):
|
||||
s = _Switch([_game("a", "UF"), _game("b"), _game("c", "FSU"), _game("d"), _game("e")],
|
||||
boost=3, favorites=("UF", "FSU"))
|
||||
cycle = s.walk(9)
|
||||
assert cycle.count("a") == 3 and cycle.count("c") == 3
|
||||
|
||||
def test_index_reset_by_update_is_followed(self):
|
||||
s = _Switch([_game("a"), _game("uf", "UF"), _game("c"), _game("d")], boost=2)
|
||||
s.walk(3)
|
||||
s.current_game_index = 0
|
||||
assert s.walk(1)[0] == "uf"
|
||||
|
||||
def test_single_card_stays_put(self):
|
||||
assert _Switch([_game("uf", "UF")], boost=5).walk(3) == ["uf", "uf", "uf"]
|
||||
|
||||
|
||||
class _Settings(SportsHelpersMixin):
|
||||
def __init__(self, mode_config):
|
||||
self.mode_config = mode_config
|
||||
self.logger = logging.getLogger("test.sports_helpers")
|
||||
self.league = "nfl"
|
||||
|
||||
|
||||
class TestSettingInt:
|
||||
def test_reads_and_clamps(self):
|
||||
assert _Settings({"n": 3})._setting_int("n", 1, 1, 5) == 3
|
||||
assert _Settings({"n": "4"})._setting_int("n", 1, 1, 5) == 4
|
||||
assert _Settings({"n": 99})._setting_int("n", 1, 1, 5) == 5
|
||||
assert _Settings({"n": -2})._setting_int("n", 1, 1, 5) == 1
|
||||
assert _Settings({})._setting_int("n", 2, 1, 5) == 2
|
||||
|
||||
@pytest.mark.parametrize("bad", ["abc", None, [], float("inf"), float("nan")])
|
||||
def test_unusable_falls_back_with_a_warning(self, bad, caplog):
|
||||
with caplog.at_level(logging.WARNING, logger="test.sports_helpers"):
|
||||
assert _Settings({"n": bad})._setting_int("n", 2, 1, 5) == 2
|
||||
assert "ignoring unusable n" in caplog.text
|
||||
|
||||
|
||||
class _Custom(SportsHelpersMixin):
|
||||
def __init__(self, config, mode=None):
|
||||
self.config = config
|
||||
if mode is not None:
|
||||
self.SKIN_MODE = mode
|
||||
|
||||
|
||||
class TestModeCustomization:
|
||||
def test_no_mode_returns_block_as_is(self):
|
||||
cust = {"score": {"font_size": 10}}
|
||||
assert _Custom({"customization": cust})._mode_customization() is cust
|
||||
|
||||
def test_non_dict_customization(self):
|
||||
assert _Custom({"customization": "junk"}, "live")._mode_customization() == {}
|
||||
assert _Custom({}, "live")._mode_customization() == {}
|
||||
|
||||
def test_mode_overrides_merge_and_none_inherits(self):
|
||||
cfg = {"customization": {
|
||||
"score": {"font_size": 10, "text_color": [1, 1, 1]},
|
||||
"layout": {"score": {"x_offset": 2, "y_offset": 3}},
|
||||
"modes": {"live": {
|
||||
"score": {"font_size": 12, "text_color": None},
|
||||
"status": "not-a-dict",
|
||||
"layout": {"score": {"y_offset": 0, "x_offset": None}, "bad": 5},
|
||||
}},
|
||||
}}
|
||||
merged = _Custom(cfg, "live")._mode_customization()
|
||||
assert merged["score"] == {"font_size": 12, "text_color": [1, 1, 1]}
|
||||
assert merged["layout"]["score"] == {"x_offset": 2, "y_offset": 0}
|
||||
assert "bad" not in merged["layout"]
|
||||
# The stored config is not mutated.
|
||||
assert cfg["customization"]["score"]["font_size"] == 10
|
||||
assert cfg["customization"]["layout"]["score"]["y_offset"] == 3
|
||||
|
||||
def test_other_mode_is_untouched(self):
|
||||
cust = {"modes": {"live": {"score": {"font_size": 12}}}}
|
||||
assert _Custom({"customization": cust}, "recent")._mode_customization() is cust
|
||||
|
||||
|
||||
class TestOddsColor:
|
||||
def test_without_element_color_is_green(self):
|
||||
assert SportsHelpersMixin()._odds_color() == (0, 255, 0)
|
||||
|
||||
def test_raising_getter_is_green(self):
|
||||
class Host(SportsHelpersMixin):
|
||||
def _element_color(self, element, default):
|
||||
raise RuntimeError("boom")
|
||||
assert Host()._odds_color() == (0, 255, 0)
|
||||
|
||||
def test_asks_for_odds_text_with_green_default(self):
|
||||
seen = []
|
||||
|
||||
class Host(SportsHelpersMixin):
|
||||
def _element_color(self, element, default):
|
||||
seen.append((element, default))
|
||||
return (9, 9, 9)
|
||||
assert Host()._odds_color() == (9, 9, 9)
|
||||
assert seen == [("odds_text", (0, 255, 0))]
|
||||
|
||||
|
||||
class _Scorebug(SportsHelpersMixin):
|
||||
def __init__(self, scroll_card):
|
||||
self.config = {"scroll_card": scroll_card}
|
||||
|
||||
def _card_option(self, key, default=None):
|
||||
return SportsCoreSharedMixin._card_option(self, key, default)
|
||||
|
||||
def _format_game_date(self, game_date, game=None):
|
||||
return "DATE"
|
||||
|
||||
def _format_game_time(self, game_time):
|
||||
return "TIME"
|
||||
|
||||
|
||||
class TestUpcomingDateAndTimeText:
|
||||
def _text(self, block):
|
||||
return _Scorebug(block)._upcoming_date_and_time_text("2026-10-07", "19:00")
|
||||
|
||||
def test_defaults_draw_both(self):
|
||||
assert self._text({}) == ("DATE", "TIME")
|
||||
|
||||
def test_scroll_card_toggles_do_not_reach_the_scorebug(self):
|
||||
assert self._text({"show_date": False, "show_time": False}) == ("DATE", "TIME")
|
||||
|
||||
def test_switch_toggles(self):
|
||||
assert self._text({"switch_show_date": False}) == ("", "TIME")
|
||||
assert self._text({"switch_show_time": False}) == ("DATE", "")
|
||||
|
||||
|
||||
class TestFavoriteKey:
|
||||
def test_defaults_to_abbreviation(self):
|
||||
assert SportsHelpersMixin()._favorite_key({"home_abbr": "UF"}, "home") == "UF"
|
||||
|
||||
def test_missing_abbreviation_never_matches(self):
|
||||
assert SportsHelpersMixin()._favorite_key({}, "away") is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Host contract
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _self_reads():
|
||||
"""Every ``self.X`` / ``getattr(self, "X")`` the mixin reads, by parsing it."""
|
||||
tree = ast.parse(Path(sports_helpers.__file__).read_text(encoding="utf-8"))
|
||||
cls = next(n for n in tree.body
|
||||
if isinstance(n, ast.ClassDef) and n.name == "SportsHelpersMixin")
|
||||
names = set()
|
||||
for node in ast.walk(cls):
|
||||
if (isinstance(node, ast.Attribute) and isinstance(node.ctx, ast.Load)
|
||||
and isinstance(node.value, ast.Name) and node.value.id == "self"):
|
||||
names.add(node.attr)
|
||||
if (isinstance(node, ast.Call) and isinstance(node.func, ast.Name)
|
||||
and node.func.id == "getattr" and len(node.args) >= 2
|
||||
and isinstance(node.args[0], ast.Name) and node.args[0].id == "self"
|
||||
and isinstance(node.args[1], ast.Constant)):
|
||||
names.add(node.args[1].value)
|
||||
return names
|
||||
|
||||
|
||||
class _Host(SportsCoreSharedMixin, SportsHelpersMixin):
|
||||
"""The smallest thing a scoreboard's SportsCore has to be for these to run."""
|
||||
|
||||
def __init__(self, config, mode_config=None):
|
||||
self.config = config
|
||||
self.mode_config = mode_config or {}
|
||||
self.logger = logging.getLogger("test.sports_helpers.host")
|
||||
self.games_list = []
|
||||
self.current_game_index = 0
|
||||
self.last_game_switch = 0
|
||||
|
||||
def _is_favorite_game(self, game):
|
||||
return game.get("home_abbr") == "UF"
|
||||
|
||||
|
||||
class TestHostContract:
|
||||
def test_every_host_read_is_documented(self):
|
||||
defined = set(dir(SportsHelpersMixin))
|
||||
# Written by the mixin itself before (or instead of) being read.
|
||||
own_state = {"_last_display_call_monotonic", "_switch_order",
|
||||
"_switch_order_key", "_switch_position"}
|
||||
needed = _self_reads() - defined - own_state
|
||||
doc = sports_helpers.__doc__
|
||||
undocumented = sorted(n for n in needed | own_state if f"``{n}``" not in doc)
|
||||
assert undocumented == [], (
|
||||
f"self attributes read but not in the module docstring's host "
|
||||
f"contract: {undocumented}")
|
||||
|
||||
def test_every_constant_read_has_a_default(self):
|
||||
# SKIN_MODE is the one exception: the host's mode name, read with
|
||||
# getattr(self, 'SKIN_MODE', None) where absence means "no mode". A
|
||||
# default here would say nothing the getattr does not already.
|
||||
missing = sorted(n for n in _self_reads() - {"SKIN_MODE"}
|
||||
if n.upper() == n and not hasattr(SportsHelpersMixin, n))
|
||||
assert missing == []
|
||||
|
||||
def test_defaults_match_what_the_plugins_ship(self):
|
||||
assert SportsHelpersMixin._DWELL_REENTRY_GAP_SECONDS == 5.0
|
||||
assert SportsHelpersMixin.favorite_rotation_boost == 1
|
||||
|
||||
def test_stateless_and_constructor_free(self):
|
||||
assert "__init__" not in SportsHelpersMixin.__dict__
|
||||
assert SportsHelpersMixin().__dict__ == {}
|
||||
|
||||
def test_base_order_does_not_matter(self):
|
||||
class Reversed(SportsHelpersMixin, SportsCoreSharedMixin):
|
||||
__init__ = _Host.__init__
|
||||
_is_favorite_game = _Host._is_favorite_game
|
||||
|
||||
for cls in (_Host, Reversed):
|
||||
h = cls({"scroll_card": {"switch_show_time": False, "time_format": "24h"}})
|
||||
assert h._upcoming_date_and_time_text("10/7", "7:00 PM") == ("10/7", "")
|
||||
|
||||
def test_shared_mixin_supplies_the_card_helpers(self):
|
||||
h = _Host({"scroll_card": {"switch_date_format": "abbrev", "time_format": "24h"}})
|
||||
assert h._upcoming_date_and_time_text("10/7", "7:05 PM") == ("Oct 7", "19:05")
|
||||
|
||||
def test_odds_color_resolves_through_element_style(self):
|
||||
h = _Host({"customization": {"odds_text": {"text_color": [1, 2, 3]}}})
|
||||
assert tuple(h._odds_color()) == (1, 2, 3)
|
||||
assert tuple(_Host({})._odds_color()) == (0, 255, 0)
|
||||
|
||||
def test_rotation_settings_and_dwell_on_one_host(self):
|
||||
h = _Host({}, mode_config={"favorite_rotation_boost": "2"})
|
||||
h.favorite_rotation_boost = h._setting_int("favorite_rotation_boost", 1, 1, 5)
|
||||
h.games_list = [_game("a"), _game("uf", "UF"), _game("c")]
|
||||
seen = []
|
||||
for _ in range(4):
|
||||
h.current_game_index = h._next_switch_index()
|
||||
seen.append(h.games_list[h.current_game_index]["id"])
|
||||
assert sorted(seen) == ["a", "c", "uf", "uf"]
|
||||
h.last_game_switch = time.time() - 100
|
||||
assert h._reset_dwell_on_reentry() is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Parity with the plugin copies
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
SCOREBOARDS = ("afl", "baseball", "basketball", "football", "hockey",
|
||||
"lacrosse", "nrl", "soccer", "ufc")
|
||||
|
||||
#: core name -> (where it lives in the plugins' sports.py, plugin name,
|
||||
#: plugins expected to carry it).
|
||||
_ALL = SCOREBOARDS
|
||||
_NOT_UFC = tuple(p for p in SCOREBOARDS if p != "ufc")
|
||||
PROMOTED = {
|
||||
"clamp_window": ("module", "_clamp_window", _ALL),
|
||||
"clamp_seconds": ("module", "_clamp_seconds", _ALL),
|
||||
"logo_needs_refresh": ("module", "_logo_needs_refresh", _ALL),
|
||||
"spread_weighted_order": ("SportsCore", "_spread_weighted_order", _ALL),
|
||||
"_mode_customization": ("SportsCore", "_mode_customization", _ALL),
|
||||
"_setting_int": ("SportsCore", "_setting_int", _ALL),
|
||||
"_reset_dwell_on_reentry": ("SportsCore", "_reset_dwell_on_reentry", _ALL),
|
||||
"_next_switch_index": ("SportsCore", "_next_switch_index", _ALL),
|
||||
"_odds_color": ("SportsCore", "_odds_color", _NOT_UFC),
|
||||
"_upcoming_date_and_time_text": ("SportsCore", "_upcoming_date_and_time_text", _NOT_UFC),
|
||||
}
|
||||
|
||||
#: Public names here that are private in the plugins.
|
||||
_RENAMES = {
|
||||
"clamp_window": "_clamp_window",
|
||||
"clamp_seconds": "_clamp_seconds",
|
||||
"logo_needs_refresh": "_logo_needs_refresh",
|
||||
"spread_weighted_order": "_spread_weighted_order",
|
||||
"MIN_WINDOW_DAYS": "_MIN_WINDOW_DAYS",
|
||||
"MAX_WINDOW_DAYS": "_MAX_WINDOW_DAYS",
|
||||
}
|
||||
|
||||
|
||||
def _plugins_root():
|
||||
raw = os.environ.get("LEDMATRIX_PLUGINS")
|
||||
if not raw:
|
||||
pytest.skip("set LEDMATRIX_PLUGINS to a ledmatrix-plugins checkout to "
|
||||
"compare sports_helpers against the plugin copies; in CI "
|
||||
"this parity check runs in ledmatrix-plugins "
|
||||
"scripts/check_sports_helpers_parity.py (#495)")
|
||||
root = Path(raw)
|
||||
if (root / "plugins").is_dir():
|
||||
root = root / "plugins"
|
||||
if not (root / "football-scoreboard" / "sports.py").is_file():
|
||||
pytest.skip(f"LEDMATRIX_PLUGINS={raw} has no football-scoreboard/sports.py")
|
||||
return root
|
||||
|
||||
|
||||
class _Normalise(ast.NodeTransformer):
|
||||
def visit_Name(self, node):
|
||||
node.id = _RENAMES.get(node.id, node.id)
|
||||
return node
|
||||
|
||||
def _func(self, node):
|
||||
node.name = _RENAMES.get(node.name, node.name)
|
||||
node.decorator_list = []
|
||||
body = node.body
|
||||
if (body and isinstance(body[0], ast.Expr)
|
||||
and isinstance(body[0].value, ast.Constant)
|
||||
and isinstance(body[0].value.value, str)):
|
||||
node.body = body[1:] or [ast.Pass()]
|
||||
self.generic_visit(node)
|
||||
return node
|
||||
|
||||
visit_FunctionDef = _func
|
||||
|
||||
|
||||
def _dump(node):
|
||||
node = ast.parse(ast.unparse(node)).body[0] # detach and copy
|
||||
return ast.dump(_Normalise().visit(node))
|
||||
|
||||
|
||||
def _definitions(tree):
|
||||
module, core = {}, {}
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.FunctionDef):
|
||||
module[node.name] = node
|
||||
elif isinstance(node, ast.Assign) and isinstance(node.targets[0], ast.Name):
|
||||
module[node.targets[0].id] = node
|
||||
elif isinstance(node, ast.ClassDef) and node.name == "SportsCore":
|
||||
for item in node.body:
|
||||
if isinstance(item, ast.FunctionDef):
|
||||
core[item.name] = item
|
||||
elif isinstance(item, (ast.Assign, ast.AnnAssign)):
|
||||
target = item.targets[0] if isinstance(item, ast.Assign) else item.target
|
||||
if isinstance(target, ast.Name):
|
||||
core[target.id] = item
|
||||
return {"module": module, "SportsCore": core}
|
||||
|
||||
|
||||
def _core_definitions():
|
||||
tree = ast.parse(Path(sports_helpers.__file__).read_text(encoding="utf-8"))
|
||||
out = {}
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.FunctionDef):
|
||||
out[node.name] = node
|
||||
elif isinstance(node, ast.ClassDef) and node.name == "SportsHelpersMixin":
|
||||
for item in node.body:
|
||||
if isinstance(item, ast.FunctionDef):
|
||||
out[item.name] = item
|
||||
return out
|
||||
|
||||
|
||||
class TestParityWithPlugins:
|
||||
@pytest.mark.parametrize("name", sorted(PROMOTED))
|
||||
def test_body_matches_every_plugin_copy(self, name):
|
||||
root = _plugins_root()
|
||||
where, plugin_name, carriers = PROMOTED[name]
|
||||
ours = _dump(_core_definitions()[name])
|
||||
drifted, missing = [], []
|
||||
for sport in carriers:
|
||||
defs = _definitions(ast.parse(
|
||||
(root / f"{sport}-scoreboard" / "sports.py").read_text(encoding="utf-8")))
|
||||
theirs = defs[where].get(plugin_name)
|
||||
if theirs is None:
|
||||
missing.append(sport)
|
||||
elif _dump(theirs) != ours:
|
||||
drifted.append(sport)
|
||||
assert missing == [], f"{plugin_name} no longer in: {missing}"
|
||||
assert drifted == [], (
|
||||
f"{plugin_name} differs from src/common/sports_helpers.py in: {drifted}. "
|
||||
f"Port the change to both, or stop treating it as shared.")
|
||||
|
||||
@pytest.mark.parametrize("sport", SCOREBOARDS)
|
||||
def test_constants_match(self, sport):
|
||||
root = _plugins_root()
|
||||
defs = _definitions(ast.parse(
|
||||
(root / f"{sport}-scoreboard" / "sports.py").read_text(encoding="utf-8")))
|
||||
assert ast.literal_eval(defs["module"]["_MIN_WINDOW_DAYS"].value) == MIN_WINDOW_DAYS
|
||||
assert ast.literal_eval(defs["module"]["_MAX_WINDOW_DAYS"].value) == MAX_WINDOW_DAYS
|
||||
gap = defs["SportsCore"]["_DWELL_REENTRY_GAP_SECONDS"].value
|
||||
assert math.isclose(ast.literal_eval(gap), SportsHelpersMixin._DWELL_REENTRY_GAP_SECONDS)
|
||||
@@ -0,0 +1,577 @@
|
||||
"""``"x-display": "hidden"`` keeps a setting declared but off the form.
|
||||
|
||||
Plugins keep deprecated keys in their schema so configs that still carry them
|
||||
keep validating (weather's ``api_key`` and ``radar_zoom``, olympics'
|
||||
notification keys), and some keys are internal (countdown's auto-generated
|
||||
row ``id``). Drawn as live controls they invite edits that do nothing, or
|
||||
worse, change an internal id.
|
||||
|
||||
The contract pinned here:
|
||||
|
||||
* nothing hidden is drawn -- top level, nested, in Advanced Settings, or as an
|
||||
array-table column -- and visible siblings still are;
|
||||
* an object of nothing but hidden children draws no empty section;
|
||||
* a hidden top-level key is not reported in ``__rendered_section`` and does
|
||||
not count toward the Advanced Settings total;
|
||||
* saving the rendered form never changes a hidden value: plain and nested ones
|
||||
are simply not posted (the save deep-merges over the stored section), and a
|
||||
hidden array-row property is carried through JSON-encoded, because a posted
|
||||
row replaces the stored item wholesale;
|
||||
* an unchecked *visible* checkbox beside them is still cleared;
|
||||
* a JSON API save is unaffected.
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from html.parser import HTMLParser
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from flask import Flask
|
||||
from werkzeug.datastructures import MultiDict
|
||||
|
||||
PROJECT_ROOT = Path(__file__).parent.parent.parent
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
HIDDEN = {"x-display": "hidden"}
|
||||
|
||||
SCHEMA = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"city": {"type": "string", "title": "City", "default": "Dallas"},
|
||||
"legacy_key": {"type": "string", "title": "Legacy Key", **HIDDEN},
|
||||
"legacy_flag": {"type": "boolean", "title": "Legacy Flag", **HIDDEN},
|
||||
"timeout": {"type": "integer", "title": "Timeout", "default": 10,
|
||||
"x-advanced": True},
|
||||
"old_zoom": {"type": "integer", "title": "Old Zoom", "default": 6,
|
||||
"minimum": 4, "maximum": 8, "x-advanced": True, **HIDDEN},
|
||||
"display": {
|
||||
"type": "object",
|
||||
"title": "Display",
|
||||
"properties": {
|
||||
"show_clock": {"type": "boolean", "title": "Show Clock",
|
||||
"default": True},
|
||||
"old_flag": {"type": "boolean", "title": "Old Flag", **HIDDEN},
|
||||
"old_mode": {"type": "string", "title": "Old Mode", **HIDDEN},
|
||||
},
|
||||
},
|
||||
# Every child hidden: no empty section.
|
||||
"internal": {
|
||||
"type": "object",
|
||||
"title": "Internal Stuff",
|
||||
"properties": {
|
||||
"token_seen": {"type": "boolean", **HIDDEN},
|
||||
"revision": {"type": "integer", **HIDDEN},
|
||||
},
|
||||
},
|
||||
"entries": {
|
||||
"type": "array",
|
||||
"title": "Entries",
|
||||
"x-widget": "array-table",
|
||||
# x-columns naming a hidden property must not draw it either.
|
||||
"x-columns": ["enabled", "name", "id"],
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {"type": "string", "title": "Row ID", **HIDDEN},
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"name": {"type": "string"},
|
||||
"pinned": {"type": "boolean", "title": "Pinned", **HIDDEN},
|
||||
"weight": {"type": "number", "title": "Weight", **HIDDEN},
|
||||
"layout": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"x": {"type": "integer", "default": 0},
|
||||
"legacy_x": {"type": "integer", **HIDDEN},
|
||||
},
|
||||
},
|
||||
},
|
||||
"required": ["name"],
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
STORED = {
|
||||
"enabled": True,
|
||||
"city": "Dallas",
|
||||
"legacy_key": "keep-me",
|
||||
"legacy_flag": True,
|
||||
"timeout": 10,
|
||||
"old_zoom": 5,
|
||||
"display": {"show_clock": True, "old_flag": True, "old_mode": "classic"},
|
||||
"internal": {"token_seen": True, "revision": 7},
|
||||
"entries": [
|
||||
# An id that looks like a number must come back a string.
|
||||
{"id": "1", "enabled": True, "name": "First", "pinned": True,
|
||||
"weight": 2.5, "layout": {"x": 3, "legacy_x": 9}},
|
||||
{"id": "cd_ab12cd34ef56", "enabled": False, "name": "Second",
|
||||
"pinned": False},
|
||||
],
|
||||
}
|
||||
|
||||
HIDDEN_LABELS = ("Legacy Key", "Legacy Flag", "Old Zoom", "Old Flag",
|
||||
"Old Mode", "Internal Stuff", "Row ID", "Pinned", "Weight")
|
||||
|
||||
|
||||
class _FormFields(HTMLParser):
|
||||
"""Collect what a browser would submit from the rendered form."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.pairs = []
|
||||
self._select = None
|
||||
self._first = None
|
||||
self._chosen = None
|
||||
|
||||
def handle_starttag(self, tag, attrs):
|
||||
a = dict(attrs)
|
||||
name = a.get("name")
|
||||
if tag == "input" and name and "disabled" not in a:
|
||||
if a.get("type") in ("checkbox", "radio") and "checked" not in a:
|
||||
return
|
||||
if a.get("type") in ("button", "submit", "file"):
|
||||
return
|
||||
self.pairs.append((name, a.get("value", "")))
|
||||
elif tag == "select" and name:
|
||||
self._select, self._first, self._chosen = name, None, None
|
||||
elif tag == "option" and self._select:
|
||||
if self._first is None:
|
||||
self._first = a.get("value", "")
|
||||
if "selected" in a:
|
||||
self._chosen = a.get("value", "")
|
||||
|
||||
def handle_endtag(self, tag):
|
||||
if tag == "select" and self._select:
|
||||
chosen = self._chosen if self._chosen is not None else self._first
|
||||
self.pairs.append((self._select, chosen or ""))
|
||||
self._select = None
|
||||
|
||||
|
||||
def _render(schema, config, plugin_id="demo"):
|
||||
from jinja2 import Environment, FileSystemLoader, select_autoescape
|
||||
|
||||
env = Environment(
|
||||
loader=FileSystemLoader(str(PROJECT_ROOT / "web_interface" / "templates")),
|
||||
autoescape=select_autoescape(["html"]),
|
||||
)
|
||||
plugin = {"id": plugin_id, "name": "Demo", "description": "",
|
||||
"enabled": True, "author": "me", "version": "1.0.0"}
|
||||
return env.get_template("v3/partials/plugin_config.html").render(
|
||||
plugin=plugin, schema=schema, config=json.loads(json.dumps(config)))
|
||||
|
||||
|
||||
def _form_pairs(html):
|
||||
parser = _FormFields()
|
||||
parser.feed(html)
|
||||
return parser.pairs
|
||||
|
||||
|
||||
def _names(html):
|
||||
return {name for name, _ in _form_pairs(html)}
|
||||
|
||||
|
||||
def _visible_names(html):
|
||||
"""Names of controls a user can see (anything but type=hidden)."""
|
||||
names = set()
|
||||
for tag in re.findall(r"<(?:input|select|textarea)\b[^>]*>", html):
|
||||
m = re.search(r'\bname="([^"]+)"', tag)
|
||||
if m and 'type="hidden"' not in tag:
|
||||
names.add(m.group(1))
|
||||
return names
|
||||
|
||||
|
||||
def _make_post(tmp_path, schema, stored, plugin_id="demo"):
|
||||
from src.plugin_system.schema_manager import SchemaManager
|
||||
from web_interface.blueprints import api_v3 as a
|
||||
|
||||
originals = {k: getattr(a.api_v3, k, None)
|
||||
for k in ("config_manager", "schema_manager", "plugin_manager")}
|
||||
|
||||
pdir = tmp_path / "plugins" / plugin_id
|
||||
pdir.mkdir(parents=True)
|
||||
(pdir / "config_schema.json").write_text(json.dumps(schema), encoding="utf-8")
|
||||
|
||||
store = {plugin_id: json.loads(json.dumps(stored))}
|
||||
cm = MagicMock()
|
||||
cm.load_config.side_effect = lambda: json.loads(json.dumps(store))
|
||||
cm.get_raw_file_content.return_value = {}
|
||||
cm.get_config_path.return_value = str(tmp_path / "config.json")
|
||||
|
||||
def _save(cfg, **_kw):
|
||||
store.clear()
|
||||
store.update(cfg)
|
||||
return type("R", (), {"status": type("S", (), {"value": "success"})(),
|
||||
"message": None})()
|
||||
|
||||
cm.save_config_atomic.side_effect = _save
|
||||
|
||||
a.api_v3.config_manager = cm
|
||||
a.api_v3.schema_manager = SchemaManager(plugins_dir=tmp_path / "plugins",
|
||||
project_root=tmp_path)
|
||||
pm = MagicMock()
|
||||
pm.plugins = {}
|
||||
pm.get_plugin.return_value = None
|
||||
a.api_v3.plugin_manager = pm
|
||||
|
||||
app = Flask(__name__)
|
||||
app.config["TESTING"] = True
|
||||
app.register_blueprint(a.api_v3, url_prefix="/api/v3")
|
||||
client = app.test_client()
|
||||
|
||||
def _post(pairs):
|
||||
resp = client.post(f"/api/v3/plugins/config?plugin_id={plugin_id}",
|
||||
data=MultiDict(pairs))
|
||||
return resp, store.get(plugin_id, {})
|
||||
|
||||
def _post_json(config):
|
||||
resp = client.post("/api/v3/plugins/config",
|
||||
json={"plugin_id": plugin_id, "config": config})
|
||||
return resp, store.get(plugin_id, {})
|
||||
|
||||
def _restore():
|
||||
for k, v in originals.items():
|
||||
setattr(a.api_v3, k, v)
|
||||
|
||||
return _post, _post_json, _restore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def post(tmp_path):
|
||||
_post, _post_json, restore = _make_post(tmp_path, SCHEMA, STORED)
|
||||
_post.json = _post_json
|
||||
try:
|
||||
yield _post
|
||||
finally:
|
||||
restore()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Rendering
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
class TestRendering:
|
||||
html = None
|
||||
|
||||
@classmethod
|
||||
def setup_class(cls):
|
||||
cls.html = _render(SCHEMA, STORED)
|
||||
|
||||
def test_no_hidden_field_gets_a_visible_control(self):
|
||||
visible = _visible_names(self.html)
|
||||
for name in ("legacy_key", "legacy_flag", "old_zoom",
|
||||
"display.old_flag", "display.old_mode",
|
||||
"internal.token_seen", "internal.revision"):
|
||||
assert name not in visible, name
|
||||
assert not [n for n in visible if re.fullmatch(
|
||||
r"entries\.\d+\.(id|pinned|weight|layout\.legacy_x)", n)], visible
|
||||
|
||||
def test_plain_and_nested_hidden_fields_are_not_posted_at_all(self):
|
||||
names = _names(self.html)
|
||||
for name in ("legacy_key", "legacy_flag", "old_zoom",
|
||||
"display.old_flag", "display.old_mode",
|
||||
"internal.token_seen", "internal.revision"):
|
||||
assert name not in names, name
|
||||
|
||||
def test_no_hidden_label_is_drawn(self):
|
||||
for label in HIDDEN_LABELS:
|
||||
assert f">{label}<" not in self.html, label
|
||||
assert f"{label}\n" not in self.html, label
|
||||
|
||||
def test_visible_siblings_still_render(self):
|
||||
visible = _visible_names(self.html)
|
||||
assert {"city", "timeout", "display.show_clock",
|
||||
"entries.0.name", "entries.0.enabled"} <= visible
|
||||
|
||||
def test_rendered_sections_exclude_hidden_top_level_keys(self):
|
||||
sections = re.findall(r'name="__rendered_section" value="([^"]+)"',
|
||||
self.html)
|
||||
assert set(sections) == {"city", "timeout", "display", "entries"}
|
||||
|
||||
def test_an_object_of_only_hidden_children_draws_no_section(self):
|
||||
assert "Internal Stuff" not in self.html
|
||||
assert "section-internal" not in self.html
|
||||
|
||||
def test_the_advanced_count_excludes_hidden_fields(self):
|
||||
assert re.findall(r"Advanced Settings \((\d+)\)", self.html) == ["1"]
|
||||
|
||||
def test_advanced_settings_disappear_when_all_advanced_are_hidden(self):
|
||||
schema = json.loads(json.dumps(SCHEMA))
|
||||
del schema["properties"]["timeout"]
|
||||
html = _render(schema, STORED)
|
||||
assert "Advanced Settings" not in html
|
||||
|
||||
def test_hidden_array_properties_are_not_columns(self):
|
||||
headers = re.findall(r"<th[^>]*>([^<]+)</th>", self.html)
|
||||
assert "Row ID" not in headers and "Pinned" not in headers
|
||||
cols = re.search(r"data-display-columns='([^']*)'", self.html)
|
||||
assert json.loads(cols.group(1)) == ["enabled", "name"]
|
||||
|
||||
def test_hidden_array_properties_stay_out_of_the_row_editor(self):
|
||||
"""The editor draws a field for every data-nested-prop and every key
|
||||
of the advanced cell's data-prop-schema."""
|
||||
assert 'data-nested-prop="pinned"' not in self.html
|
||||
assert 'data-nested-prop="layout.legacy_x"' not in self.html
|
||||
assert 'data-nested-prop="layout.x"' in self.html
|
||||
for m in re.finditer(r"class=\"array-table-advanced-data\"\s+"
|
||||
r"data-prop-schema='([^']*)'", self.html):
|
||||
import html as _h
|
||||
adv = json.loads(_h.unescape(m.group(1)))
|
||||
assert "pinned" not in adv and "weight" not in adv and "id" not in adv
|
||||
|
||||
def test_stored_hidden_row_values_are_carried_json_encoded(self):
|
||||
pairs = dict(_form_pairs(self.html))
|
||||
assert pairs["entries.0.id"] == '"1"'
|
||||
assert pairs["entries.0.pinned"] == "true"
|
||||
assert pairs["entries.0.weight"] == "2.5"
|
||||
assert pairs["entries.0.layout.legacy_x"] == "9"
|
||||
assert pairs["entries.1.id"] == '"cd_ab12cd34ef56"'
|
||||
# Absent stored values are not invented.
|
||||
assert "entries.1.weight" not in pairs
|
||||
assert "entries.1.layout.legacy_x" not in pairs
|
||||
|
||||
|
||||
def test_a_schema_without_the_flag_renders_as_before():
|
||||
"""Regression guard: the flag's absence changes nothing."""
|
||||
schema = json.loads(json.dumps(SCHEMA))
|
||||
|
||||
def strip(node):
|
||||
if isinstance(node, dict):
|
||||
node.pop("x-display", None)
|
||||
for v in node.values():
|
||||
strip(v)
|
||||
strip(schema)
|
||||
html = _render(schema, STORED)
|
||||
visible = _visible_names(html)
|
||||
assert {"legacy_key", "legacy_flag", "old_zoom", "display.old_flag",
|
||||
"internal.token_seen", "entries.0.id"} <= visible
|
||||
# Not a column, so it lives in the row editor's advanced cell as before.
|
||||
assert 'data-nested-prop="pinned"' in html
|
||||
assert re.findall(r"Advanced Settings \((\d+)\)", html) == ["2"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Saving
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
class TestSavingTheRenderedForm:
|
||||
def test_the_untouched_form_changes_no_hidden_value(self, post):
|
||||
resp, cfg = post(_form_pairs(_render(SCHEMA, STORED)))
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert cfg["legacy_key"] == "keep-me"
|
||||
assert cfg["legacy_flag"] is True
|
||||
assert cfg["old_zoom"] == 5
|
||||
assert cfg["display"]["old_flag"] is True
|
||||
assert cfg["display"]["old_mode"] == "classic"
|
||||
assert cfg["internal"] == {"token_seen": True, "revision": 7}
|
||||
first, second = cfg["entries"]
|
||||
assert first["id"] == "1" and isinstance(first["id"], str)
|
||||
assert first["pinned"] is True
|
||||
assert first["weight"] == 2.5
|
||||
assert first["layout"]["legacy_x"] == 9
|
||||
assert second["id"] == "cd_ab12cd34ef56"
|
||||
assert second["pinned"] is False
|
||||
|
||||
def test_visible_edits_apply_and_an_unchecked_visible_box_is_cleared(self, post):
|
||||
pairs = [(k, v) for k, v in _form_pairs(_render(SCHEMA, STORED))
|
||||
if k != "display.show_clock"]
|
||||
pairs = [(k, "Austin" if k == "city" else v) for k, v in pairs]
|
||||
resp, cfg = post(pairs)
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert cfg["city"] == "Austin"
|
||||
# The existing unchecked-checkbox behaviour, in the same section as
|
||||
# hidden booleans that must not be touched.
|
||||
assert cfg["display"]["show_clock"] is False
|
||||
assert cfg["display"]["old_flag"] is True
|
||||
assert cfg["legacy_flag"] is True
|
||||
assert cfg["entries"][0]["pinned"] is True
|
||||
|
||||
def test_a_deleted_row_takes_its_hidden_values_with_it(self, post):
|
||||
"""Row 0 removed; the browser renumbers row 1 to index 0."""
|
||||
pairs = []
|
||||
for k, v in _form_pairs(_render(SCHEMA, STORED)):
|
||||
m = re.match(r"entries\.(\d+)\.(.*)", k)
|
||||
if m:
|
||||
if m.group(1) == "0":
|
||||
continue
|
||||
k = f"entries.0.{m.group(2)}"
|
||||
pairs.append((k, v))
|
||||
resp, cfg = post(pairs)
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert [e["id"] for e in cfg["entries"]] == ["cd_ab12cd34ef56"]
|
||||
|
||||
def test_a_new_row_gets_no_invented_hidden_values(self, post):
|
||||
pairs = _form_pairs(_render(SCHEMA, STORED)) + [
|
||||
("entries.2.enabled", "true"), ("entries.2.name", "Third")]
|
||||
resp, cfg = post(pairs)
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert cfg["entries"][2]["name"] == "Third"
|
||||
assert "id" not in cfg["entries"][2]
|
||||
assert "pinned" not in cfg["entries"][2]
|
||||
|
||||
def test_a_json_api_save_still_writes_hidden_fields(self, post):
|
||||
"""Hidden is a form concern: API callers can still set these."""
|
||||
config = json.loads(json.dumps(STORED))
|
||||
config["legacy_flag"] = False
|
||||
config["display"]["old_mode"] = "modern"
|
||||
config["entries"][0]["id"] = "renamed"
|
||||
resp, cfg = post.json(config)
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert cfg["legacy_flag"] is False
|
||||
assert cfg["display"]["old_mode"] == "modern"
|
||||
assert cfg["entries"][0]["id"] == "renamed"
|
||||
|
||||
|
||||
class TestMissingBooleans:
|
||||
"""_set_missing_booleans_to_false, directly, at every depth."""
|
||||
|
||||
def test_hidden_booleans_are_out_of_scope_everywhere(self):
|
||||
from web_interface.blueprints.api_v3 import _set_missing_booleans_to_false
|
||||
|
||||
cfg = json.loads(json.dumps(STORED))
|
||||
form_keys = {"city", "entries.0.name", "entries.1.name"}
|
||||
_set_missing_booleans_to_false(
|
||||
cfg, SCHEMA["properties"], form_keys,
|
||||
sections={"city", "display", "entries", "internal", "legacy_flag"})
|
||||
|
||||
assert cfg["legacy_flag"] is True
|
||||
assert cfg["display"]["old_flag"] is True
|
||||
assert cfg["internal"]["token_seen"] is True
|
||||
assert cfg["entries"][0]["pinned"] is True
|
||||
# Visible booleans in the same places are still cleared.
|
||||
assert cfg["display"]["show_clock"] is False
|
||||
assert cfg["entries"][0]["enabled"] is False
|
||||
|
||||
def test_hidden_array_item_property_lookup(self):
|
||||
from web_interface.blueprints.api_v3 import _hidden_array_item_property
|
||||
|
||||
assert _hidden_array_item_property(SCHEMA, "entries.0.id")
|
||||
assert _hidden_array_item_property(SCHEMA, "entries.12.layout.legacy_x")
|
||||
assert _hidden_array_item_property(SCHEMA, "entries.0.name") is None
|
||||
assert _hidden_array_item_property(SCHEMA, "entries.0.layout.x") is None
|
||||
# Only array rows qualify: plain hidden keys are never posted.
|
||||
assert _hidden_array_item_property(SCHEMA, "legacy_key") is None
|
||||
assert _hidden_array_item_property(SCHEMA, "display.old_flag") is None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Real plugin schemas (snippets copied from ledmatrix-plugins)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
# plugins/countdown/config_schema.json, trimmed to the countdowns table.
|
||||
COUNTDOWN_SCHEMA = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": True},
|
||||
"countdowns": {
|
||||
"type": "array",
|
||||
"x-widget": "array-table",
|
||||
"x-columns": ["enabled", "name", "target_date", "target_time", "mode"],
|
||||
"default": [],
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {"type": "string", "title": "ID",
|
||||
"description": "Unique identifier (auto-generated)",
|
||||
"x-display": "hidden"},
|
||||
"enabled": {"type": "boolean", "title": "Enabled", "default": True},
|
||||
"name": {"type": "string", "title": "Name",
|
||||
"minLength": 1, "maxLength": 30},
|
||||
"target_date": {"type": "string", "title": "Target Date",
|
||||
"x-widget": "date-picker",
|
||||
"pattern": "^\\d{4}-\\d{2}-\\d{2}$"},
|
||||
"target_time": {"type": "string", "title": "Target Time",
|
||||
"x-widget": "time-picker", "default": "00:00"},
|
||||
"mode": {"type": "string", "title": "Mode",
|
||||
"enum": ["until", "since"], "default": "until"},
|
||||
"display_order": {"x-advanced": True, "type": "integer",
|
||||
"title": "Order", "default": 0, "minimum": 0},
|
||||
},
|
||||
"required": ["name", "target_date"],
|
||||
},
|
||||
},
|
||||
"show_expired": {"x-advanced": True, "type": "boolean",
|
||||
"title": "Show Expired", "default": False},
|
||||
},
|
||||
"required": ["enabled"],
|
||||
}
|
||||
|
||||
COUNTDOWN_STORED = {
|
||||
"enabled": True,
|
||||
"show_expired": True,
|
||||
"countdowns": [
|
||||
{"id": "cd_0123456789ab", "enabled": True, "name": "Vacation",
|
||||
"target_date": "2026-12-20", "target_time": "00:00", "mode": "until",
|
||||
"display_order": 0},
|
||||
{"id": "42", "enabled": True, "name": "Anniversary",
|
||||
"target_date": "2020-06-01", "target_time": "00:00", "mode": "since",
|
||||
"display_order": 1},
|
||||
],
|
||||
}
|
||||
|
||||
# plugins/ledmatrix-weather/config_schema.json, trimmed.
|
||||
WEATHER_SCHEMA = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": {"type": "boolean", "default": False},
|
||||
"location_city": {"type": "string", "default": "Dallas", "title": "City"},
|
||||
"update_interval": {"x-advanced": True, "type": "integer", "default": 1800,
|
||||
"minimum": 300},
|
||||
"api_key": {"x-advanced": True, "x-display": "hidden", "type": "string",
|
||||
"title": "API Key (deprecated)"},
|
||||
"radar_range_miles": {"type": "integer", "default": 75, "minimum": 10,
|
||||
"maximum": 500, "title": "Radar Range (miles)"},
|
||||
"radar_zoom": {"x-advanced": True, "x-display": "hidden", "type": "integer",
|
||||
"default": 6, "minimum": 4, "maximum": 8,
|
||||
"title": "Radar Zoom Level (deprecated)"},
|
||||
"radar_show_nowcast": {"type": "boolean", "default": True,
|
||||
"title": "Show Forecast Frames"},
|
||||
},
|
||||
}
|
||||
|
||||
WEATHER_STORED = {"enabled": True, "location_city": "Tampa", "update_interval": 900,
|
||||
"api_key": "legacy-owm-key", "radar_range_miles": 75,
|
||||
"radar_zoom": 7, "radar_show_nowcast": True}
|
||||
|
||||
|
||||
def test_countdown_ids_survive_a_form_save(tmp_path):
|
||||
html = _render(COUNTDOWN_SCHEMA, COUNTDOWN_STORED, "countdown")
|
||||
assert ">ID<" not in html
|
||||
assert "countdowns.0.id" not in _visible_names(html)
|
||||
|
||||
_post, _json, restore = _make_post(tmp_path, COUNTDOWN_SCHEMA,
|
||||
COUNTDOWN_STORED, "countdown")
|
||||
try:
|
||||
# Edit a visible cell too.
|
||||
pairs = [(k, "Beach" if k == "countdowns.0.name" else v)
|
||||
for k, v in _form_pairs(html)]
|
||||
resp, cfg = _post(pairs)
|
||||
finally:
|
||||
restore()
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert [c["id"] for c in cfg["countdowns"]] == ["cd_0123456789ab", "42"]
|
||||
assert cfg["countdowns"][0]["name"] == "Beach"
|
||||
|
||||
|
||||
def test_weather_deprecated_keys_are_hidden_and_kept(tmp_path):
|
||||
html = _render(WEATHER_SCHEMA, WEATHER_STORED, "weather")
|
||||
assert "api_key" not in html and "radar_zoom" not in html
|
||||
assert "Radar Zoom Level" not in html and "API Key" not in html
|
||||
assert re.findall(r"Advanced Settings \((\d+)\)", html) == ["1"]
|
||||
assert "radar_range_miles" in _visible_names(html)
|
||||
|
||||
_post, _json, restore = _make_post(tmp_path, WEATHER_SCHEMA,
|
||||
WEATHER_STORED, "weather")
|
||||
try:
|
||||
pairs = [(k, v) for k, v in _form_pairs(html)
|
||||
if k != "radar_show_nowcast"]
|
||||
resp, cfg = _post(pairs)
|
||||
finally:
|
||||
restore()
|
||||
assert resp.status_code == 200, resp.get_json()
|
||||
assert cfg["api_key"] == "legacy-owm-key"
|
||||
assert cfg["radar_zoom"] == 7
|
||||
assert cfg["radar_show_nowcast"] is False
|
||||
@@ -1047,8 +1047,36 @@ def check_health_monitor():
|
||||
_reconciliation_started = True
|
||||
_threading.Thread(target=_run_startup_reconciliation, daemon=True).start()
|
||||
|
||||
_auto_updater = None
|
||||
|
||||
|
||||
def start_auto_update_scheduler():
|
||||
"""Start the weekly auto-update thread (web_interface/auto_update.py).
|
||||
|
||||
Called by the launchers, not at import: tests and dev tools import this
|
||||
module, and none of them should ever git pull. Idle unless
|
||||
auto_update.enabled is set, so it is safe to always start.
|
||||
"""
|
||||
global _auto_updater
|
||||
if _auto_updater is not None:
|
||||
return _auto_updater
|
||||
from web_interface.auto_update import AutoUpdater
|
||||
from web_interface.blueprints.api_v3.system import perform_core_update
|
||||
_auto_updater = AutoUpdater(
|
||||
config_manager=config_manager,
|
||||
core_update=perform_core_update,
|
||||
store_manager=plugin_store_manager,
|
||||
plugin_manager=plugin_manager,
|
||||
schema_manager=schema_manager,
|
||||
operation_history=operation_history,
|
||||
)
|
||||
_auto_updater.start()
|
||||
return _auto_updater
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
import os as _os
|
||||
start_auto_update_scheduler()
|
||||
# threaded=True is Flask's default since 1.0 but stated explicitly so that
|
||||
# long-lived /api/v3/stream/* SSE connections don't starve other requests.
|
||||
# Debug mode is off by default; opt in with FLASK_DEBUG=1 in the environment.
|
||||
|
||||
@@ -0,0 +1,700 @@
|
||||
"""Weekly automatic updates: LEDMatrix code, then installed plugins.
|
||||
|
||||
Turned on by ``auto_update.enabled`` in config.json (General tab, or the
|
||||
installer's --enable-auto-update). Off by default: an update pulls new code and
|
||||
restarts the display, which nobody should get without asking for it.
|
||||
|
||||
Nothing here is allowed to leave a device broken without saying so:
|
||||
|
||||
* **Checks first.** The code update is skipped, and the reason reported, when
|
||||
the checkout has local edits or commits, a rebase or merge is in progress,
|
||||
the branch has no upstream, disk is low, the newest commit was already
|
||||
rolled back once, or the health check is not set up.
|
||||
* **Verified, and rolled back.** The pull itself is the Overview "Update Code"
|
||||
path (``perform_core_update``). Restarting and checking the result is handed
|
||||
to ledmatrix-update-verify.service (scripts/utils/auto_update_verify.py),
|
||||
started through ledmatrix-update-verify.path by a request file, which
|
||||
survives the web service restart, confirms both services come up and stay
|
||||
up, and otherwise resets to the previous commit and its dependencies.
|
||||
* **Set up without SSH.** Those units are installed by the installer, or by the
|
||||
display service when the toggle is switched on (src/auto_update_setup.py).
|
||||
* **Plugins after the code.** Plugins use the Plugin Store's own update, which
|
||||
refuses versions this core cannot run and restores the old copy when an
|
||||
install fails. When the code changed they wait until it passed its check,
|
||||
so they are never gated against code that is about to be rolled back.
|
||||
* **Loud failures.** Anything other than success is written to the state file,
|
||||
shown under the toggle and raised as a banner on the Overview tab.
|
||||
|
||||
The schedule survives restarts through data/auto_update_state.json (gitignored).
|
||||
"""
|
||||
import importlib.util
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
STATE_REL = Path('data') / 'auto_update_state.json'
|
||||
PENDING_REL = Path('data') / 'auto_update_pending.json'
|
||||
REQUEST_REL = Path('data') / 'auto_update_verify.request'
|
||||
SETUP_RESULT_REL = Path('data') / 'auto_update_setup.json'
|
||||
VERIFIER_COPY_REL = Path('data') / 'auto_update_verifier.py'
|
||||
VERIFIER_SOURCE = PROJECT_ROOT / 'scripts' / 'utils' / 'auto_update_verify.py'
|
||||
STATE_FILE = PROJECT_ROOT / STATE_REL
|
||||
PENDING_FILE = PROJECT_ROOT / PENDING_REL
|
||||
SETUP_RESULT_FILE = PROJECT_ROOT / SETUP_RESULT_REL
|
||||
|
||||
VERIFY_UNIT = 'ledmatrix-update-verify'
|
||||
PATH_UNIT = f'{VERIFY_UNIT}.path'
|
||||
|
||||
UPDATE_INTERVAL_SECONDS = 7 * 24 * 3600
|
||||
#: A failure that is probably transient (usually no network) is retried the
|
||||
#: next day rather than a week later.
|
||||
RETRY_AFTER_FAILURE_SECONDS = 24 * 3600
|
||||
#: Local hours an update prefers, since it restarts the display...
|
||||
QUIET_HOURS = range(2, 5)
|
||||
#: ...but a device that is never powered at night must still get updates.
|
||||
QUIET_HOURS_GRACE_SECONDS = 24 * 3600
|
||||
CHECK_INTERVAL_SECONDS = 30 * 60
|
||||
#: Let the web service settle after boot (and never pull in a crash loop).
|
||||
STARTUP_DELAY_SECONDS = 10 * 60
|
||||
MIN_FREE_BYTES = 300 * 1024 * 1024
|
||||
#: How long the health check gets to pick up a request before the update is undone.
|
||||
HANDOFF_TIMEOUT_SECONDS = 90
|
||||
#: The health check takes a few minutes at most; far past that, it is lost.
|
||||
VERIFY_LOST_SECONDS = 45 * 60
|
||||
|
||||
#: Core outcomes that need the user's attention.
|
||||
ERROR_OUTCOMES = frozenset({'error', 'blocked', 'rolled_back', 'rollback_failed', 'lost'})
|
||||
|
||||
HELPER_MISSING_MESSAGE = (
|
||||
"LEDMatrix code updates are paused until the update health check is set up. "
|
||||
"The display service sets it up when it starts while automatic updates are on, "
|
||||
"so restart the display from the Overview tab. If this keeps happening, the "
|
||||
"setup result under Automatic Updates on the General tab says why. Plugin "
|
||||
"updates still run."
|
||||
)
|
||||
|
||||
_STATE_LOCK = threading.RLock()
|
||||
|
||||
|
||||
def is_enabled(config):
|
||||
return bool((config.get('auto_update') or {}).get('enabled', False))
|
||||
|
||||
|
||||
def _read_json(path):
|
||||
try:
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
return data if isinstance(data, dict) else None
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _write_json(path, data):
|
||||
path = Path(path)
|
||||
tmp = None
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=str(path.parent), prefix=f'.{path.stem}_')
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, indent=2)
|
||||
os.replace(tmp, path)
|
||||
tmp = None
|
||||
finally:
|
||||
if tmp and os.path.exists(tmp):
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _unlink(path):
|
||||
try:
|
||||
Path(path).unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def load_state(state_file=STATE_FILE):
|
||||
return _read_json(state_file) or {}
|
||||
|
||||
|
||||
def save_state(state, state_file=STATE_FILE):
|
||||
with _STATE_LOCK:
|
||||
try:
|
||||
_write_json(state_file, state)
|
||||
except OSError as e:
|
||||
logger.warning("Could not save auto-update state: %s", e)
|
||||
|
||||
|
||||
def dismiss_alert(alert_id, state_file=STATE_FILE):
|
||||
with _STATE_LOCK:
|
||||
state = load_state(state_file)
|
||||
state['alert_dismissed'] = str(alert_id)
|
||||
save_state(state, state_file)
|
||||
|
||||
|
||||
def _zone(tz_name):
|
||||
try:
|
||||
from zoneinfo import ZoneInfo
|
||||
return ZoneInfo(tz_name) if tz_name else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _local_datetime(ts, tz_name):
|
||||
return datetime.fromtimestamp(ts, _zone(tz_name))
|
||||
|
||||
|
||||
def _short(sha):
|
||||
return (sha or 'unknown')[:7]
|
||||
|
||||
|
||||
def is_due(now, next_due, local_hour):
|
||||
"""Due once ``next_due`` has passed, in quiet hours or after the grace period."""
|
||||
if next_due is None or now < next_due:
|
||||
return False
|
||||
return local_hour in QUIET_HOURS or now >= next_due + QUIET_HOURS_GRACE_SECONDS
|
||||
|
||||
|
||||
def _plugin_fingerprint(store_manager, plugin_dir):
|
||||
"""(manifest version, git sha): what changing tells us an update landed."""
|
||||
version = None
|
||||
try:
|
||||
with open(plugin_dir / 'manifest.json', 'r', encoding='utf-8') as f:
|
||||
manifest = json.load(f)
|
||||
if manifest.get('local_only'):
|
||||
return None
|
||||
version = manifest.get('version')
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
sha = None
|
||||
try:
|
||||
git_info = store_manager._get_local_git_info(plugin_dir)
|
||||
sha = git_info.get('sha') if git_info else None
|
||||
except Exception:
|
||||
logger.debug("git info unavailable for %s", plugin_dir, exc_info=True)
|
||||
return (version, sha)
|
||||
|
||||
|
||||
def update_plugins(store_manager, operation_history=None):
|
||||
"""Update every installed plugin that has an update. Returns (updated, failed)."""
|
||||
updated, failed = [], []
|
||||
plugins_dir = Path(store_manager.plugins_dir)
|
||||
for plugin_id in sorted(store_manager.list_installed_plugins()):
|
||||
plugin_dir = plugins_dir / plugin_id
|
||||
before = _plugin_fingerprint(store_manager, plugin_dir)
|
||||
if before is None:
|
||||
continue # local_only: managed by hand, never from the registry
|
||||
try:
|
||||
ok = store_manager.update_plugin(plugin_id)
|
||||
except Exception:
|
||||
logger.exception("Automatic update of plugin %s raised", plugin_id)
|
||||
ok = False
|
||||
if not ok:
|
||||
failed.append(plugin_id)
|
||||
status = 'failed'
|
||||
elif _plugin_fingerprint(store_manager, plugin_dir) != before:
|
||||
updated.append(plugin_id)
|
||||
status = 'success'
|
||||
else:
|
||||
continue # already current; not worth a history entry every week
|
||||
if operation_history:
|
||||
try:
|
||||
operation_history.record_operation(
|
||||
'update', plugin_id=plugin_id, status=status,
|
||||
details={'automatic': True})
|
||||
except Exception:
|
||||
logger.debug("Could not record auto-update history", exc_info=True)
|
||||
return updated, failed
|
||||
|
||||
|
||||
def _service_active(unit):
|
||||
try:
|
||||
result = subprocess.run(['systemctl', 'is-active', unit],
|
||||
capture_output=True, text=True, timeout=5)
|
||||
return result.stdout.strip() == 'active'
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
return False
|
||||
|
||||
|
||||
def helper_ready():
|
||||
"""True when the health check can be triggered (its path unit is watching)."""
|
||||
return _service_active(PATH_UNIT)
|
||||
|
||||
|
||||
def restart_service(unit):
|
||||
"""Restart a unit only if it is running, via the sudoers-allowed command."""
|
||||
if not _service_active(unit):
|
||||
return False
|
||||
try:
|
||||
result = subprocess.run(['sudo', '-n', 'systemctl', 'restart', f'{unit}.service'],
|
||||
capture_output=True, text=True, timeout=30)
|
||||
except (subprocess.SubprocessError, OSError) as e:
|
||||
logger.warning("Auto-update could not restart %s: %s", unit, e)
|
||||
return False
|
||||
if result.returncode != 0:
|
||||
logger.warning("Auto-update restart of %s failed: %s", unit, result.stderr.strip())
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def start_setup_if_needed(was_enabled, config):
|
||||
"""After settings are saved: switching updates on finishes setting them up.
|
||||
|
||||
The health check units are installed by the display service at startup
|
||||
(src/auto_update_setup.py), so switching the toggle on restarts it.
|
||||
Returns a note for the save message, or None.
|
||||
"""
|
||||
if was_enabled or not is_enabled(config) or helper_ready():
|
||||
return None
|
||||
if not _service_active('ledmatrix'):
|
||||
return 'Automatic update setup finishes the next time the display service starts.'
|
||||
if restart_service('ledmatrix'):
|
||||
return 'Finishing automatic update setup: the display is restarting.'
|
||||
return 'Could not restart the display to finish automatic update setup; restart it from the Overview tab.'
|
||||
|
||||
|
||||
def _load_verifier(path):
|
||||
spec = importlib.util.spec_from_file_location('ledmatrix_auto_update_verifier', str(path))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class AutoUpdater:
|
||||
"""Decides when an automatic update is due and runs it."""
|
||||
|
||||
def __init__(self, config_manager, core_update, store_manager=None,
|
||||
plugin_manager=None, schema_manager=None, operation_history=None,
|
||||
project_root=PROJECT_ROOT, state_file=None, clock=time.time,
|
||||
restart=restart_service, run=subprocess.run,
|
||||
service_active=_service_active, helper_ready=helper_ready,
|
||||
verifier_source=VERIFIER_SOURCE, disk_free=None, sleep=time.sleep):
|
||||
self.config_manager = config_manager
|
||||
self.core_update = core_update
|
||||
self.store_manager = store_manager
|
||||
self.plugin_manager = plugin_manager
|
||||
self.schema_manager = schema_manager
|
||||
self.operation_history = operation_history
|
||||
self.project_root = Path(project_root)
|
||||
self.state_file = Path(state_file) if state_file else self.project_root / STATE_REL
|
||||
self.pending_file = self.project_root / PENDING_REL
|
||||
self.request_file = self.project_root / REQUEST_REL
|
||||
self.verifier_copy = self.project_root / VERIFIER_COPY_REL
|
||||
self.verifier_source = Path(verifier_source)
|
||||
self.clock = clock
|
||||
self.restart = restart
|
||||
self.run_command = run
|
||||
self.service_active = service_active
|
||||
self.helper_ready = helper_ready
|
||||
self.disk_free = disk_free or (lambda path: shutil.disk_usage(path).free)
|
||||
self.sleep = sleep
|
||||
self._stop = threading.Event()
|
||||
self._thread = None
|
||||
|
||||
# -- scheduling ---------------------------------------------------------
|
||||
|
||||
def tick(self):
|
||||
"""Do whatever is due. Returns True if an update ran."""
|
||||
now = self.clock()
|
||||
state = load_state(self.state_file)
|
||||
# Before the enabled check: a verification started before the user
|
||||
# switched updates off still has to be reported.
|
||||
if self._finalize_verification(state, now) == 'waiting':
|
||||
return False
|
||||
|
||||
try:
|
||||
config = self.config_manager.load_config()
|
||||
except Exception:
|
||||
logger.debug("Auto-update could not load config", exc_info=True)
|
||||
return False
|
||||
if not is_enabled(config):
|
||||
return False
|
||||
|
||||
if state.get('plugins_pending'):
|
||||
self._run_deferred_plugins(state, now)
|
||||
return True
|
||||
|
||||
if state.get('next_due') is None:
|
||||
# Just switched on: due now, which in practice means the next
|
||||
# quiet-hours window rather than the moment Save was clicked.
|
||||
state['next_due'] = now
|
||||
save_state(state, self.state_file)
|
||||
|
||||
local_hour = _local_datetime(now, config.get('timezone')).hour
|
||||
if not is_due(now, state['next_due'], local_hour):
|
||||
return False
|
||||
self.run(state)
|
||||
return True
|
||||
|
||||
def run(self, state=None):
|
||||
state = load_state(self.state_file) if state is None else state
|
||||
logger.info("Automatic update starting")
|
||||
try:
|
||||
core = self.update_core(state)
|
||||
except Exception as e:
|
||||
logger.exception("Automatic core update raised")
|
||||
core = {'outcome': 'error', 'message': f'The LEDMatrix update failed unexpectedly: {e}'}
|
||||
|
||||
deferred = core['outcome'] == 'verifying'
|
||||
updated, failed = ([], []) if deferred else self._update_plugins()
|
||||
self._store_run(state, core, updated, failed)
|
||||
logger.info("Automatic update: core %s (%s); plugins updated=%s failed=%s%s",
|
||||
core['outcome'], core['message'], updated, failed,
|
||||
'; plugins wait for the health check' if deferred else '')
|
||||
|
||||
# Code restarts belong to the health check; this only covers plugins.
|
||||
if updated:
|
||||
self.restart('ledmatrix')
|
||||
return state['last_result']
|
||||
|
||||
def _store_run(self, state, core, updated, failed):
|
||||
now = self.clock()
|
||||
deferred = core['outcome'] == 'verifying'
|
||||
state.update({
|
||||
'last_run': now,
|
||||
'next_due': now + (RETRY_AFTER_FAILURE_SECONDS if core['outcome'] == 'error'
|
||||
else UPDATE_INTERVAL_SECONDS),
|
||||
'plugins_pending': deferred,
|
||||
'last_result': {
|
||||
'core_outcome': core['outcome'],
|
||||
'core_message': core['message'],
|
||||
'plugins_updated': updated,
|
||||
'plugins_failed': failed,
|
||||
'plugins_deferred': deferred,
|
||||
},
|
||||
})
|
||||
self._apply_result(state, now)
|
||||
save_state(state, self.state_file)
|
||||
|
||||
# -- core update --------------------------------------------------------
|
||||
|
||||
def _git(self, *args, timeout=60):
|
||||
return self.run_command(['git', *args], cwd=str(self.project_root),
|
||||
capture_output=True, text=True, timeout=timeout)
|
||||
|
||||
def _count(self, rev_range):
|
||||
out = self._git('rev-list', '--count', rev_range).stdout.strip()
|
||||
return int(out) if out.isdigit() else 0
|
||||
|
||||
def preflight(self, state):
|
||||
"""Decide whether the code update may run.
|
||||
|
||||
Returns ``(outcome, message, info)`` with outcome ``ready``,
|
||||
``up_to_date``, ``blocked`` (needs the user) or ``error`` (retry soon).
|
||||
"""
|
||||
if not self.helper_ready():
|
||||
return 'blocked', HELPER_MISSING_MESSAGE, {}
|
||||
|
||||
git_dir = self._git('rev-parse', '--git-dir')
|
||||
if git_dir.returncode != 0:
|
||||
return 'blocked', 'LEDMatrix is not installed as a git checkout, so it cannot update itself.', {}
|
||||
git_dir = Path(git_dir.stdout.strip())
|
||||
if not git_dir.is_absolute():
|
||||
git_dir = self.project_root / git_dir
|
||||
rebase_dirs = [git_dir / name for name in ('rebase-merge', 'rebase-apply') if (git_dir / name).exists()]
|
||||
if rebase_dirs:
|
||||
# A rebase really in progress leaves HEAD detached. With HEAD back
|
||||
# on a branch it was abandoned -- typically a pull that stopped on
|
||||
# a conflict, then a checkout -- and it makes every later pull
|
||||
# fail. It holds no work (local edits are checked separately
|
||||
# below), so clear it rather than strand a device nobody SSHes into.
|
||||
if self._git('symbolic-ref', '-q', 'HEAD').returncode != 0:
|
||||
return 'blocked', ('A git rebase is in progress in the LEDMatrix folder. '
|
||||
'Finish or abort it; automatic updates will not touch it.'), {}
|
||||
cleared = self._git('rebase', '--quit')
|
||||
if cleared.returncode != 0 or any(path.exists() for path in rebase_dirs):
|
||||
return 'blocked', ('An abandoned git rebase in the LEDMatrix folder could not be cleared: '
|
||||
f'{(cleared.stderr or "").strip() or "unknown error"}.'), {}
|
||||
logger.warning("Cleared an abandoned git rebase from %s", ', '.join(str(p) for p in rebase_dirs))
|
||||
if (git_dir / 'MERGE_HEAD').exists():
|
||||
return 'blocked', ('A git merge is in progress in the LEDMatrix folder. '
|
||||
'Finish or abort it; automatic updates will not touch it.'), {}
|
||||
|
||||
if self._git('rev-parse', '--abbrev-ref', '--symbolic-full-name', '@{u}').returncode != 0:
|
||||
return 'blocked', ('The current branch has no upstream to update from (or HEAD is detached). '
|
||||
'Use Update Code once, or Tools -> Switch branch.'), {}
|
||||
|
||||
# Mode-only changes are ignored: older installers chmod tracked
|
||||
# scripts, and --autostash in the pull carries those across. Plugin
|
||||
# folders are separate installs, as in perform_core_update.
|
||||
status = self._git('-c', 'core.fileMode=false', 'status', '--porcelain', '--untracked-files=no')
|
||||
changed = [line[3:] for line in status.stdout.splitlines()
|
||||
if line.strip() and 'plugins/' not in line and 'plugin-repos/' not in line]
|
||||
if status.returncode != 0 or changed:
|
||||
example = f' (for example {changed[0]})' if changed else ''
|
||||
return 'blocked', (f'{len(changed) or "Some"} tracked file(s) in the LEDMatrix folder were edited '
|
||||
f'locally{example}. Automatic updates will not stash your changes; '
|
||||
'commit or revert them, or update manually with Update Code.'), {}
|
||||
|
||||
free = self.disk_free(str(self.project_root))
|
||||
if free < MIN_FREE_BYTES:
|
||||
return 'blocked', (f'Only {free // (1024 * 1024)} MB of disk space is free; an update '
|
||||
f'needs at least {MIN_FREE_BYTES // (1024 * 1024)} MB.'), {}
|
||||
|
||||
fetch = self._git('fetch', '--quiet', timeout=120)
|
||||
if fetch.returncode != 0:
|
||||
detail = next((ln.strip() for ln in (fetch.stderr or '').splitlines() if ln.strip()), '')
|
||||
return 'error', f'Could not check for LEDMatrix updates: {detail or "git fetch failed"}.', {}
|
||||
|
||||
ahead = self._count('@{u}..HEAD')
|
||||
if ahead:
|
||||
return 'blocked', (f'This checkout has {ahead} local commit(s) that are not upstream. '
|
||||
'Automatic updates will not rebase them; update manually with Update Code.'), {}
|
||||
if not self._count('HEAD..@{u}'):
|
||||
return 'up_to_date', 'LEDMatrix is already up to date.', {}
|
||||
|
||||
upstream = self._git('rev-parse', '@{u}').stdout.strip()
|
||||
if upstream and upstream == state.get('rolled_back_head'):
|
||||
return 'up_to_date', (f'The newest LEDMatrix version ({_short(upstream)}) failed its health '
|
||||
'check and was rolled back before; waiting for a newer one.'), {}
|
||||
head = self._git('rev-parse', 'HEAD').stdout.strip()
|
||||
return 'ready', '', {'old_head': head, 'upstream_head': upstream}
|
||||
|
||||
def update_core(self, state):
|
||||
try:
|
||||
outcome, message, info = self.preflight(state)
|
||||
except (subprocess.SubprocessError, OSError) as e:
|
||||
return {'outcome': 'error', 'message': f'Pre-update checks failed: {e}.'}
|
||||
if outcome != 'ready':
|
||||
return {'outcome': outcome, 'message': message}
|
||||
old_head = info['old_head']
|
||||
|
||||
display_was_active = self.service_active('ledmatrix')
|
||||
try:
|
||||
# Copied before pulling, so the checker is the known-good version.
|
||||
self.verifier_copy.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(self.verifier_source, self.verifier_copy)
|
||||
verifier = _load_verifier(self.verifier_copy)
|
||||
except Exception as e:
|
||||
return {'outcome': 'error', 'message': f'Could not prepare the update health check: {e}.'}
|
||||
|
||||
try:
|
||||
core = self.core_update()
|
||||
except Exception as e:
|
||||
logger.exception("perform_core_update raised")
|
||||
core = {'status': 'error', 'message': f'Update failed: {e}'}
|
||||
new_head = self._git('rev-parse', 'HEAD').stdout.strip()
|
||||
pending = {
|
||||
'status': 'pending',
|
||||
'old_head': old_head,
|
||||
'new_head': new_head,
|
||||
'display_was_active': display_was_active,
|
||||
'dependency_failures': list(core.get('dependency_failures') or []),
|
||||
'created_at': self.clock(),
|
||||
}
|
||||
|
||||
def rollback():
|
||||
return verifier.Verifier(self.project_root, run=self.run_command).rollback(pending)
|
||||
|
||||
if core.get('status') != 'success':
|
||||
message = core.get('message') or 'The LEDMatrix update failed.'
|
||||
if new_head and new_head != old_head:
|
||||
ok, detail = rollback()
|
||||
message += (' The partial update was rolled back.' if ok
|
||||
else f' Rolling back the partial update also failed: {detail}.')
|
||||
if not ok:
|
||||
return {'outcome': 'rollback_failed', 'message': message}
|
||||
return {'outcome': 'error', 'message': message}
|
||||
if new_head == old_head:
|
||||
return {'outcome': 'up_to_date', 'message': core.get('message') or 'LEDMatrix is already up to date.'}
|
||||
|
||||
handoff = {'outcome': 'verifying',
|
||||
'message': (f'Updated LEDMatrix from {_short(old_head)} to {_short(new_head)}; '
|
||||
'restarting and checking the services.')}
|
||||
# Recorded before the handoff: the health check restarts this process.
|
||||
self._store_run(state, handoff, [], [])
|
||||
try:
|
||||
_write_json(self.pending_file, pending)
|
||||
self.request_file.write_text(new_head, encoding='utf-8')
|
||||
picked_up = self._wait_for_pickup()
|
||||
except OSError as e:
|
||||
logger.error("Could not request the update health check: %s", e)
|
||||
picked_up = False
|
||||
if picked_up:
|
||||
return handoff
|
||||
|
||||
# No health check means no update: undo it while the old code is
|
||||
# still the code that is running.
|
||||
logger.error("%s did not start the health check; rolling the update back", PATH_UNIT)
|
||||
_unlink(self.request_file)
|
||||
_unlink(self.pending_file)
|
||||
ok, detail = rollback()
|
||||
if not ok:
|
||||
return {'outcome': 'rollback_failed',
|
||||
'message': (f'The update health check did not start, and rolling back to '
|
||||
f'{_short(old_head)} failed: {detail}.')}
|
||||
return {'outcome': 'blocked',
|
||||
'message': ('The update health check did not start, so the update was undone. '
|
||||
f'Check "systemctl status {PATH_UNIT}".'
|
||||
+ (f' Note: {detail}.' if detail else ''))}
|
||||
|
||||
def _wait_for_pickup(self):
|
||||
for _ in range(HANDOFF_TIMEOUT_SECONDS):
|
||||
pending = _read_json(self.pending_file)
|
||||
if pending and pending.get('status') != 'pending':
|
||||
return True
|
||||
self.sleep(1)
|
||||
pending = _read_json(self.pending_file)
|
||||
return bool(pending and pending.get('status') != 'pending')
|
||||
|
||||
def _finalize_verification(self, state, now):
|
||||
"""Fold the health check's outcome into the state. None, 'waiting' or 'done'."""
|
||||
pending = _read_json(self.pending_file)
|
||||
if pending is None:
|
||||
return None
|
||||
status = pending.get('status')
|
||||
old, new = pending.get('old_head'), pending.get('new_head')
|
||||
reason, detail = pending.get('reason'), pending.get('detail')
|
||||
|
||||
if status in ('pending', 'verifying'):
|
||||
if now - float(pending.get('created_at') or now) < VERIFY_LOST_SECONDS:
|
||||
return 'waiting'
|
||||
outcome = 'lost'
|
||||
message = (f'LEDMatrix was updated to {_short(new)}, but its health check never reported back, '
|
||||
'so it is not known whether the device is healthy. '
|
||||
f'See "journalctl -u {VERIFY_UNIT}".')
|
||||
elif status == 'success':
|
||||
outcome = 'updated'
|
||||
message = (f'Updated LEDMatrix from {_short(old)} to {_short(new)}; '
|
||||
'the services restarted and stayed healthy.')
|
||||
elif status == 'rolled_back':
|
||||
outcome = 'rolled_back'
|
||||
message = (f'The LEDMatrix update to {_short(new)} was rolled back to {_short(old)} because '
|
||||
f'{reason or "it failed its health check"}.' + (f' Note: {detail}.' if detail else ''))
|
||||
state['rolled_back_head'] = new
|
||||
else:
|
||||
outcome = 'rollback_failed'
|
||||
message = (f'The LEDMatrix update to {_short(new)} failed ({reason or "unknown reason"}) and ' # nosec B608 - user-facing message, not SQL # nosemgrep
|
||||
f'could not be rolled back: {detail or "unknown error"}. The device may need '
|
||||
f'attention: run "git reset --hard {old}" in the LEDMatrix folder, then restart '
|
||||
'the display and web services.')
|
||||
|
||||
result = state.setdefault('last_result', {})
|
||||
result.update({'core_outcome': outcome, 'core_message': message})
|
||||
if outcome not in ('updated', 'rolled_back'):
|
||||
# The device is in an unknown state; don't pile plugin changes on it.
|
||||
state['plugins_pending'] = False
|
||||
result['plugins_deferred'] = False
|
||||
self._apply_result(state, now)
|
||||
save_state(state, self.state_file)
|
||||
_unlink(self.pending_file)
|
||||
(logger.info if outcome == 'updated' else logger.error)("Automatic update: %s", message)
|
||||
return 'done'
|
||||
|
||||
# -- plugins and reporting ----------------------------------------------
|
||||
|
||||
def _update_plugins(self):
|
||||
if not self.store_manager:
|
||||
return [], []
|
||||
updated, failed = update_plugins(self.store_manager, self.operation_history)
|
||||
for plugin_id in updated:
|
||||
if self.schema_manager:
|
||||
self.schema_manager.invalidate_cache(plugin_id)
|
||||
if updated and self.plugin_manager:
|
||||
try:
|
||||
self.plugin_manager.discover_plugins()
|
||||
except Exception:
|
||||
logger.debug("discover_plugins after auto-update failed", exc_info=True)
|
||||
return updated, failed
|
||||
|
||||
def _run_deferred_plugins(self, state, now):
|
||||
state['plugins_pending'] = False
|
||||
updated, failed = self._update_plugins()
|
||||
result = state.setdefault('last_result', {})
|
||||
result.update({'plugins_updated': updated, 'plugins_failed': failed, 'plugins_deferred': False})
|
||||
self._apply_result(state, now)
|
||||
save_state(state, self.state_file)
|
||||
if updated:
|
||||
self.restart('ledmatrix')
|
||||
|
||||
@staticmethod
|
||||
def _apply_result(state, now):
|
||||
result = state.get('last_result') or {}
|
||||
outcome = result.get('core_outcome')
|
||||
failed = result.get('plugins_failed') or []
|
||||
if outcome in ERROR_OUTCOMES or failed:
|
||||
result['status'] = 'error'
|
||||
elif outcome == 'verifying':
|
||||
result['status'] = 'pending'
|
||||
else:
|
||||
result['status'] = 'success'
|
||||
|
||||
if result['status'] == 'error':
|
||||
parts = []
|
||||
if outcome in ERROR_OUTCOMES:
|
||||
parts.append(result.get('core_message') or 'The LEDMatrix update failed.')
|
||||
if failed:
|
||||
parts.append('These plugins failed to update: ' + ', '.join(failed) + '.')
|
||||
state['alert'] = {'id': str(int(now * 1000)), 'message': ' '.join(parts)}
|
||||
elif result['status'] == 'success':
|
||||
state.pop('alert', None)
|
||||
|
||||
# -- thread -------------------------------------------------------------
|
||||
|
||||
def start(self):
|
||||
if self._thread and self._thread.is_alive():
|
||||
return
|
||||
self._thread = threading.Thread(target=self._loop, name='auto-update', daemon=True)
|
||||
self._thread.start()
|
||||
|
||||
def stop(self):
|
||||
self._stop.set()
|
||||
|
||||
def _loop(self):
|
||||
if self._stop.wait(STARTUP_DELAY_SECONDS):
|
||||
return
|
||||
while True:
|
||||
try:
|
||||
self.tick()
|
||||
except Exception:
|
||||
logger.exception("Automatic update check failed")
|
||||
if self._stop.wait(CHECK_INTERVAL_SECONDS):
|
||||
return
|
||||
|
||||
|
||||
def describe_status(config, state=None, state_file=STATE_FILE, pending_file=PENDING_FILE,
|
||||
setup_file=SETUP_RESULT_FILE, helper=None):
|
||||
"""What the General tab and the Overview banner show."""
|
||||
state = load_state(state_file) if state is None else state
|
||||
tz = config.get('timezone')
|
||||
fmt = '%b %d, %Y %I:%M %p'
|
||||
pending = _read_json(pending_file) or {}
|
||||
setup = _read_json(setup_file) or {}
|
||||
enabled = is_enabled(config)
|
||||
out = {
|
||||
'last_run': None, 'summary': None, 'status': None, 'next_due': None,
|
||||
'alert': None, 'alert_id': None,
|
||||
# Only asked while enabled: it runs systemctl, on every page load.
|
||||
'verifier_installed': (helper or helper_ready)() if enabled else None,
|
||||
'setup_status': setup.get('status'),
|
||||
'setup_message': setup.get('message'),
|
||||
'verifying': pending.get('status') in ('pending', 'verifying'),
|
||||
}
|
||||
if state.get('last_run'):
|
||||
out['last_run'] = _local_datetime(state['last_run'], tz).strftime(fmt)
|
||||
result = state.get('last_result') or {}
|
||||
out['status'] = result.get('status')
|
||||
parts = [result.get('core_message') or 'No result recorded.']
|
||||
if result.get('plugins_updated'):
|
||||
parts.append('Plugins updated: ' + ', '.join(result['plugins_updated']) + '.')
|
||||
if result.get('plugins_failed'):
|
||||
parts.append('Plugins that failed to update: ' + ', '.join(result['plugins_failed']) + '.')
|
||||
if result.get('plugins_deferred') and state.get('plugins_pending'):
|
||||
parts.append('Plugin updates run once the code update passes its health check.')
|
||||
out['summary'] = ' '.join(parts)
|
||||
if enabled and state.get('next_due'):
|
||||
out['next_due'] = _local_datetime(state['next_due'], tz).strftime(fmt)
|
||||
alert = state.get('alert')
|
||||
if alert and alert.get('id') != state.get('alert_dismissed'):
|
||||
out['alert'], out['alert_id'] = alert.get('message'), alert.get('id')
|
||||
return out
|
||||
@@ -1039,6 +1039,51 @@ def _set_nested_value(config, key_path, value):
|
||||
_RENDERED_SECTION_FIELD = '__rendered_section'
|
||||
|
||||
|
||||
def _is_hidden_prop(prop_schema):
|
||||
"""Whether a schema property is declared ``"x-display": "hidden"``.
|
||||
|
||||
Mirrors ``prop_is_hidden`` in ``plugin_config.html``: an object whose every
|
||||
child is hidden counts as hidden too. A hidden property stays declared (so
|
||||
stored configs carrying it keep validating) but the form draws no control
|
||||
for it, which puts it outside everything the save infers from the form.
|
||||
"""
|
||||
if not isinstance(prop_schema, dict):
|
||||
return False
|
||||
if prop_schema.get('x-display') == 'hidden':
|
||||
return True
|
||||
children = prop_schema.get('properties')
|
||||
if isinstance(children, dict) and children:
|
||||
return all(_is_hidden_prop(child) for child in children.values())
|
||||
return False
|
||||
|
||||
|
||||
def _hidden_array_item_property(schema, key_path):
|
||||
"""The hidden schema property a posted array-row key names, else None.
|
||||
|
||||
Only paths through an array index (``countdowns.0.id``) qualify: that is
|
||||
the one place the rendered form posts a hidden value, because a posted row
|
||||
replaces the stored item wholesale. The value is JSON-encoded there so its
|
||||
exact type survives the round trip.
|
||||
"""
|
||||
parts = key_path.split('.')
|
||||
digit_at = next((i for i, p in enumerate(parts) if p.isdigit()), None)
|
||||
if digit_at is None or digit_at == 0 or digit_at == len(parts) - 1:
|
||||
return None
|
||||
array_prop = _get_schema_property(schema, '.'.join(parts[:digit_at]))
|
||||
if not _schema_type_is(array_prop, 'array'):
|
||||
return None
|
||||
items = array_prop.get('items')
|
||||
if not isinstance(items, dict) or not isinstance(items.get('properties'), dict):
|
||||
return None
|
||||
rest = '.'.join(parts[digit_at + 1:])
|
||||
item_schema = {'properties': items['properties']}
|
||||
prop = _get_schema_property(item_schema, rest)
|
||||
if prop is None:
|
||||
# Deeper array inside the item (rare): keep walking.
|
||||
return _hidden_array_item_property(item_schema, rest)
|
||||
return prop if _is_hidden_prop(prop) else None
|
||||
|
||||
|
||||
def _boolean_is_in_scope(full_path, prefix, sections, submitted_parents):
|
||||
"""Whether a missing checkbox at ``full_path`` may be forced to False.
|
||||
|
||||
@@ -1112,6 +1157,12 @@ def _set_missing_booleans_to_false(plugin_config, schema_props, form_keys, prefi
|
||||
if not isinstance(prop_schema, dict):
|
||||
continue
|
||||
|
||||
# A hidden property (or an object of nothing but hidden ones) was never
|
||||
# drawn, so its absence from the form is not an unchecked box. Leave it
|
||||
# -- and everything under it -- exactly as stored.
|
||||
if _is_hidden_prop(prop_schema):
|
||||
continue
|
||||
|
||||
full_path = f"{prefix}.{prop_name}" if prefix else prop_name
|
||||
prop_type = prop_schema.get('type')
|
||||
|
||||
|
||||
@@ -454,16 +454,21 @@ def save_main_config():
|
||||
|
||||
# Merge with existing config (similar to original implementation)
|
||||
current_config = api_v3.config_manager.load_config()
|
||||
was_auto_update_enabled = bool((current_config.get('auto_update') or {}).get('enabled'))
|
||||
|
||||
# Handle general settings
|
||||
# Note: Checkboxes don't send data when unchecked, so we need to check if we're updating general settings
|
||||
# If any general setting is present, we're updating the general tab
|
||||
is_general_update = any(k in data for k in ['timezone', 'city', 'state', 'country', 'web_display_autostart',
|
||||
'auto_discover', 'auto_load_enabled', 'development_mode', 'plugins_directory'])
|
||||
'auto_discover', 'auto_load_enabled', 'development_mode', 'plugins_directory',
|
||||
'auto_update_enabled'])
|
||||
|
||||
if is_general_update:
|
||||
# For checkbox: if not present in data during general update, it means unchecked
|
||||
current_config['web_display_autostart'] = _coerce_to_bool(data.get('web_display_autostart'))
|
||||
if not isinstance(current_config.get('auto_update'), dict):
|
||||
current_config['auto_update'] = {}
|
||||
current_config['auto_update']['enabled'] = _coerce_to_bool(data.get('auto_update_enabled'))
|
||||
|
||||
if 'timezone' in data:
|
||||
current_config['timezone'] = data['timezone']
|
||||
@@ -1033,7 +1038,7 @@ def save_main_config():
|
||||
if key in ['timezone', 'city', 'state', 'country',
|
||||
'web_display_autostart', 'auto_discover',
|
||||
'auto_load_enabled', 'development_mode',
|
||||
'plugins_directory', 'target_fps']:
|
||||
'plugins_directory', 'target_fps', 'auto_update_enabled']:
|
||||
continue
|
||||
# Skip fields that are already handled above in their own named sections.
|
||||
# Without this, every form field name lands as a top-level config key too.
|
||||
@@ -1068,7 +1073,17 @@ def save_main_config():
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
return success_response(message='Configuration saved successfully')
|
||||
message = 'Configuration saved successfully'
|
||||
# Switching automatic updates on finishes their setup, which needs
|
||||
# the display service to restart (web_interface/auto_update.py).
|
||||
try:
|
||||
from web_interface import auto_update
|
||||
note = auto_update.start_setup_if_needed(was_auto_update_enabled, current_config)
|
||||
if note:
|
||||
message = f'{message}. {note}'
|
||||
except Exception:
|
||||
logger.warning("Automatic update setup could not be started", exc_info=True)
|
||||
return success_response(message=message)
|
||||
except Exception as e:
|
||||
logger.error("Error saving config", exc_info=True)
|
||||
return error_response(
|
||||
|
||||
@@ -8,7 +8,7 @@ from web_interface.blueprints.api_v3 import (
|
||||
_CALENDAR_LIST_MAX_PAGES, _RENDERED_SECTION_FIELD, _SKIP_FIELD, _coerce_to_bool,
|
||||
_do_transactional_uninstall, _enhance_schema_with_core_properties,
|
||||
_filter_config_by_schema, _get_plugin_version, _get_schema_property,
|
||||
_installed_plugin_ids, _is_plugin_update_available,
|
||||
_hidden_array_item_property, _installed_plugin_ids, _is_plugin_update_available,
|
||||
_parse_form_value_with_schema, _prune_credential_backups,
|
||||
_run_calendar_registration, _schema_allows_null, _schema_type_is,
|
||||
_set_missing_booleans_to_false,
|
||||
@@ -1848,7 +1848,21 @@ def save_plugin_config():
|
||||
# (to avoid overwriting the combined array with a single value)
|
||||
if key not in indexed_base_paths:
|
||||
# Parse value using schema to determine correct type
|
||||
parsed_value = _parse_form_value_with_schema(value, key, schema)
|
||||
parsed_value = _SKIP_FIELD
|
||||
decoded = False
|
||||
# A hidden property inside an array row is carried
|
||||
# through the form JSON-encoded (a posted row replaces
|
||||
# the stored item, so it must be posted at all). Decode
|
||||
# it exactly: the generic parse would turn an id "1"
|
||||
# into the integer 1 and fail validation.
|
||||
if _hidden_array_item_property(schema, key) is not None:
|
||||
try:
|
||||
parsed_value = json.loads(value)
|
||||
decoded = True
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
if not decoded:
|
||||
parsed_value = _parse_form_value_with_schema(value, key, schema)
|
||||
# Debug logging for array fields
|
||||
if schema:
|
||||
prop = _get_schema_property(schema, key)
|
||||
|
||||
@@ -12,6 +12,8 @@ from web_interface.blueprints.api_v3 import (
|
||||
get_git_version, jsonify, logger, os, request, resolve_pull_command,
|
||||
shutil, subprocess,
|
||||
)
|
||||
import threading
|
||||
|
||||
import web_interface.blueprints.api_v3 as _pkg
|
||||
# Read through the module rather than bound by value: tests patch these
|
||||
# as module attributes, and a value binding would not see the patch.
|
||||
@@ -120,6 +122,32 @@ def get_system_version():
|
||||
except Exception as e:
|
||||
logger.error("get_system_version failed: %s", e, exc_info=True)
|
||||
return jsonify({'status': 'error', 'message': 'Unable to retrieve version'}), 500
|
||||
@api_v3.route('/system/auto-update', methods=['GET'])
|
||||
def get_auto_update_status():
|
||||
"""Weekly automatic update status: last result, next check, and any alert.
|
||||
|
||||
No local except: a failure falls through to the app-wide handler in
|
||||
web_interface/app.py, which logs the traceback and returns the redacted
|
||||
detail -- the response every route gives, without a second copy here.
|
||||
"""
|
||||
from web_interface import auto_update
|
||||
config = api_v3.config_manager.load_config() if api_v3.config_manager else {}
|
||||
return jsonify({'status': 'success', 'data': auto_update.describe_status(config)})
|
||||
|
||||
|
||||
@api_v3.route('/system/auto-update/dismiss', methods=['POST'])
|
||||
def dismiss_auto_update_alert():
|
||||
"""Hide the current automatic-update banner until a new alert replaces it."""
|
||||
from web_interface import auto_update
|
||||
payload = request.get_json(silent=True)
|
||||
# A JSON array or scalar is a bad request, not a 500.
|
||||
alert_id = str(payload.get('alert_id') or '').strip() if isinstance(payload, dict) else ''
|
||||
if not alert_id:
|
||||
return jsonify({'status': 'error', 'message': 'alert_id required'}), 400
|
||||
auto_update.dismiss_alert(alert_id)
|
||||
return jsonify({'status': 'success'})
|
||||
|
||||
|
||||
@api_v3.route('/system/check-update', methods=['GET'])
|
||||
def check_for_update():
|
||||
"""Check whether a newer LEDMatrix commit is available on origin/main."""
|
||||
@@ -198,6 +226,219 @@ def _sudo_hint_for(text):
|
||||
return None
|
||||
|
||||
|
||||
_core_update_lock = threading.Lock()
|
||||
|
||||
|
||||
def perform_core_update():
|
||||
"""Pull the latest LEDMatrix code and sync its dependencies.
|
||||
|
||||
Shared by the Overview "Update Code" button and the weekly automatic
|
||||
updater (web_interface/auto_update.py), so both take exactly the same
|
||||
path. Returns the JSON-able payload the button has always received:
|
||||
``status``, ``message`` and ``restart_required``.
|
||||
"""
|
||||
# The button and the scheduler can fire together; two pulls racing
|
||||
# over one checkout (and one stash) is how local changes get lost.
|
||||
if not _core_update_lock.acquire(blocking=False):
|
||||
return {'status': 'error', 'restart_required': False,
|
||||
'message': 'An update is already in progress; try again shortly.'}
|
||||
try:
|
||||
return _perform_core_update_locked()
|
||||
finally:
|
||||
_core_update_lock.release()
|
||||
|
||||
|
||||
def _perform_core_update_locked():
|
||||
project_dir = str(PROJECT_ROOT)
|
||||
|
||||
# Decide how to pull BEFORE stashing. If this checkout cannot be
|
||||
# updated at all, stashing first would put the user's local changes
|
||||
# away for an update that was never going to run.
|
||||
pull_args, upstream_note, pull_error = resolve_pull_command(project_dir)
|
||||
if pull_error:
|
||||
logger.warning("git pull not attempted: %s", pull_error)
|
||||
return {'status': 'error', 'message': pull_error, 'restart_required': False}
|
||||
|
||||
# Check if there are local changes that need to be stashed
|
||||
# Exclude plugins directory - plugins are separate repos and shouldn't be stashed with base project
|
||||
# Use --untracked-files=no to skip untracked files check (much faster with symlinked plugins)
|
||||
try:
|
||||
status_result = subprocess.run(
|
||||
['git', 'status', '--porcelain', '--untracked-files=no'],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=project_dir
|
||||
)
|
||||
# Filter out any changes in plugins directory - plugins are separate repositories
|
||||
# Git status format: XY filename (where X is status of index, Y is status of work tree)
|
||||
status_lines = [line for line in status_result.stdout.strip().split('\n')
|
||||
if line.strip() and 'plugins/' not in line]
|
||||
has_changes = bool('\n'.join(status_lines).strip())
|
||||
except subprocess.TimeoutExpired:
|
||||
# If status check times out, assume there might be changes and proceed
|
||||
# This is safer than failing the update
|
||||
has_changes = True
|
||||
status_result = type('obj', (object,), {'stdout': '', 'stderr': 'Status check timed out'})()
|
||||
|
||||
stash_info = ""
|
||||
|
||||
# Stash local changes if they exist (excluding plugins)
|
||||
# Plugins are separate repositories and shouldn't be stashed with base project updates
|
||||
if has_changes:
|
||||
try:
|
||||
# Use pathspec to exclude plugins directory from stash
|
||||
stash_result = subprocess.run(
|
||||
['git', 'stash', 'push', '-m', 'LEDMatrix auto-stash before update', '--', ':!plugins'],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=project_dir
|
||||
)
|
||||
if stash_result.returncode == 0:
|
||||
logger.debug("git stash: stashed local changes before pull")
|
||||
stash_info = " Local changes were stashed."
|
||||
else:
|
||||
logger.warning("git stash failed before pull (returncode=%d)", stash_result.returncode)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("git stash timed out, proceeding with pull")
|
||||
|
||||
# Record HEAD before the pull so dependency changes can be detected
|
||||
old_head = None
|
||||
try:
|
||||
_pre = subprocess.run(['git', 'rev-parse', 'HEAD'],
|
||||
capture_output=True, text=True, timeout=10, cwd=project_dir)
|
||||
if _pre.returncode == 0:
|
||||
old_head = _pre.stdout.strip()
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("git rev-parse timed out before pull")
|
||||
|
||||
# Whether the pull actually brought new code in. "Already up to
|
||||
# date" is a success too, and prompting for a restart then would
|
||||
# train users to ignore the prompt.
|
||||
code_changed = False
|
||||
# Requirement files whose install failed. The automatic updater refuses
|
||||
# to restart onto code whose dependencies did not install.
|
||||
dependency_failures = []
|
||||
|
||||
# Perform the git pull. Branches without an upstream were given
|
||||
# an explicit "origin <branch>" above so the update still works.
|
||||
result = subprocess.run(
|
||||
pull_args,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60,
|
||||
cwd=project_dir
|
||||
)
|
||||
|
||||
# Give the branch tracking information so the next pull is a plain
|
||||
# `git pull` — otherwise every update repeats the fallback.
|
||||
if result.returncode == 0 and upstream_note:
|
||||
branch = _git_current_branch(project_dir)
|
||||
if branch:
|
||||
try:
|
||||
subprocess.run(
|
||||
['git', 'branch', f'--set-upstream-to=origin/{branch}', branch],
|
||||
capture_output=True, text=True, timeout=10, cwd=project_dir)
|
||||
except (subprocess.TimeoutExpired, OSError) as exc:
|
||||
logger.debug("could not set upstream for %s: %s", branch, exc)
|
||||
|
||||
# Return custom response for git_pull
|
||||
if result.returncode == 0:
|
||||
pull_message = "Code updated successfully."
|
||||
if has_changes:
|
||||
pull_message = f"Code updated successfully. Local changes were automatically stashed.{stash_info}"
|
||||
if result.stdout and "Already up to date" not in result.stdout:
|
||||
pull_message = f"Code updated successfully.{stash_info}"
|
||||
if upstream_note:
|
||||
pull_message = f"{pull_message} {upstream_note}"
|
||||
|
||||
# Keep Python dependencies in sync automatically: if the pull
|
||||
# changed a requirements file, install it now — users updating
|
||||
# from the web UI (most of them) never SSH in to pip install.
|
||||
# Installs go through the same root-visible path as the
|
||||
# Tools-tab buttons (_pip_install_requirements).
|
||||
dep_notes = []
|
||||
try:
|
||||
_post = subprocess.run(['git', 'rev-parse', 'HEAD'],
|
||||
capture_output=True, text=True, timeout=10, cwd=project_dir)
|
||||
new_head = _post.stdout.strip() if _post.returncode == 0 else None
|
||||
if old_head and new_head and old_head != new_head:
|
||||
code_changed = True
|
||||
diff = subprocess.run(
|
||||
['git', 'diff', '--name-only', f'{old_head}..{new_head}'],
|
||||
capture_output=True, text=True, timeout=15, cwd=project_dir)
|
||||
changed = set(diff.stdout.split()) if diff.returncode == 0 else set()
|
||||
for rel in ('requirements.txt', 'web_interface/requirements.txt'):
|
||||
req_path = PROJECT_ROOT / rel
|
||||
if rel not in changed or not req_path.exists():
|
||||
continue
|
||||
# Each file's install is isolated: a timeout or
|
||||
# OSError (e.g. the sudo wrapper/interpreter
|
||||
# missing) on one file must not abort the other.
|
||||
try:
|
||||
r = _pip_install_requirements(req_path, timeout=180)
|
||||
if r.returncode == 0:
|
||||
dep_notes.append(f"Dependencies from {rel} updated.")
|
||||
else:
|
||||
dependency_failures.append(rel)
|
||||
dep_notes.append(
|
||||
f"Dependency install from {rel} failed — "
|
||||
"run Install Base Requirements from the Tools tab.")
|
||||
logger.warning("post-update pip install failed for %s: %s",
|
||||
rel, _truncate_output(r.stdout, r.stderr))
|
||||
except subprocess.TimeoutExpired:
|
||||
dependency_failures.append(rel)
|
||||
dep_notes.append(
|
||||
f"Dependency install from {rel} timed out — "
|
||||
"run Install Base Requirements from the Tools tab.")
|
||||
logger.warning("post-update pip install timed out for %s", rel)
|
||||
except OSError as install_err:
|
||||
dependency_failures.append(rel)
|
||||
dep_notes.append(
|
||||
f"Dependency install from {rel} failed — "
|
||||
"run Install Base Requirements from the Tools tab.")
|
||||
logger.warning("post-update pip install errored for %s: %s",
|
||||
rel, install_err)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("post-update dependency sync timed out")
|
||||
if dep_notes:
|
||||
pull_message += " " + " ".join(dep_notes)
|
||||
# A `git pull` restores built-in plugins (committed under
|
||||
# plugin-repos/) even if the user uninstalled them. Re-remove
|
||||
# any the user previously uninstalled so the update doesn't
|
||||
# resurrect them.
|
||||
if api_v3.plugin_store_manager:
|
||||
try:
|
||||
purged = api_v3.plugin_store_manager.purge_uninstalled_plugins()
|
||||
if purged:
|
||||
logger.info(
|
||||
"Re-removed %d uninstalled plugin(s) restored by update: %s",
|
||||
len(purged), ", ".join(purged),
|
||||
)
|
||||
except (OSError, RuntimeError) as purge_err:
|
||||
logger.warning("Post-update plugin purge failed: %s", purge_err)
|
||||
else:
|
||||
logger.warning("git pull failed (returncode=%d): %s", result.returncode, result.stderr)
|
||||
# Show git's own first line: "check logs" leaves the user with
|
||||
# nothing to act on, and these failures are usually actionable
|
||||
# (conflicting local commits, no upstream, network).
|
||||
detail = next((ln.strip() for ln in (result.stderr or '').splitlines()
|
||||
if ln.strip()), '')
|
||||
pull_message = f"Update failed: {detail}" if detail else "Update failed; check logs for details"
|
||||
|
||||
# Nothing here restarts anything: the pull replaces files on
|
||||
# disk while the display and web services keep running the code
|
||||
# they loaded at boot. Without this the user is told the update
|
||||
# succeeded and sees no change until they happen to reboot.
|
||||
return {
|
||||
'status': 'success' if result.returncode == 0 else 'error',
|
||||
'message': pull_message,
|
||||
'restart_required': bool(result.returncode == 0 and code_changed),
|
||||
'dependency_failures': dependency_failures,
|
||||
}
|
||||
|
||||
|
||||
@api_v3.route('/system/action', methods=['POST'])
|
||||
def execute_system_action():
|
||||
"""Execute system actions (start/stop/reboot/etc)"""
|
||||
@@ -265,188 +506,7 @@ def execute_system_action():
|
||||
result = subprocess.run(['sudo', 'poweroff'],
|
||||
capture_output=True, text=True, timeout=10)
|
||||
elif action == 'git_pull':
|
||||
# Use PROJECT_ROOT instead of hardcoded path
|
||||
project_dir = str(PROJECT_ROOT)
|
||||
|
||||
# Decide how to pull BEFORE stashing. If this checkout cannot be
|
||||
# updated at all, stashing first would put the user's local changes
|
||||
# away for an update that was never going to run.
|
||||
pull_args, upstream_note, pull_error = resolve_pull_command(project_dir)
|
||||
if pull_error:
|
||||
logger.warning("git pull not attempted: %s", pull_error)
|
||||
return jsonify({'status': 'error', 'message': pull_error})
|
||||
|
||||
# Check if there are local changes that need to be stashed
|
||||
# Exclude plugins directory - plugins are separate repos and shouldn't be stashed with base project
|
||||
# Use --untracked-files=no to skip untracked files check (much faster with symlinked plugins)
|
||||
try:
|
||||
status_result = subprocess.run(
|
||||
['git', 'status', '--porcelain', '--untracked-files=no'],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=project_dir
|
||||
)
|
||||
# Filter out any changes in plugins directory - plugins are separate repositories
|
||||
# Git status format: XY filename (where X is status of index, Y is status of work tree)
|
||||
status_lines = [line for line in status_result.stdout.strip().split('\n')
|
||||
if line.strip() and 'plugins/' not in line]
|
||||
has_changes = bool('\n'.join(status_lines).strip())
|
||||
except subprocess.TimeoutExpired:
|
||||
# If status check times out, assume there might be changes and proceed
|
||||
# This is safer than failing the update
|
||||
has_changes = True
|
||||
status_result = type('obj', (object,), {'stdout': '', 'stderr': 'Status check timed out'})()
|
||||
|
||||
stash_info = ""
|
||||
|
||||
# Stash local changes if they exist (excluding plugins)
|
||||
# Plugins are separate repositories and shouldn't be stashed with base project updates
|
||||
if has_changes:
|
||||
try:
|
||||
# Use pathspec to exclude plugins directory from stash
|
||||
stash_result = subprocess.run(
|
||||
['git', 'stash', 'push', '-m', 'LEDMatrix auto-stash before update', '--', ':!plugins'],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=project_dir
|
||||
)
|
||||
if stash_result.returncode == 0:
|
||||
logger.debug("git stash: stashed local changes before pull")
|
||||
stash_info = " Local changes were stashed."
|
||||
else:
|
||||
logger.warning("git stash failed before pull (returncode=%d)", stash_result.returncode)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("git stash timed out, proceeding with pull")
|
||||
|
||||
# Record HEAD before the pull so dependency changes can be detected
|
||||
old_head = None
|
||||
try:
|
||||
_pre = subprocess.run(['git', 'rev-parse', 'HEAD'],
|
||||
capture_output=True, text=True, timeout=10, cwd=project_dir)
|
||||
if _pre.returncode == 0:
|
||||
old_head = _pre.stdout.strip()
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("git rev-parse timed out before pull")
|
||||
|
||||
# Whether the pull actually brought new code in. "Already up to
|
||||
# date" is a success too, and prompting for a restart then would
|
||||
# train users to ignore the prompt.
|
||||
code_changed = False
|
||||
|
||||
# Perform the git pull. Branches without an upstream were given
|
||||
# an explicit "origin <branch>" above so the update still works.
|
||||
result = subprocess.run(
|
||||
pull_args,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60,
|
||||
cwd=project_dir
|
||||
)
|
||||
|
||||
# Give the branch tracking information so the next pull is a plain
|
||||
# `git pull` — otherwise every update repeats the fallback.
|
||||
if result.returncode == 0 and upstream_note:
|
||||
branch = _git_current_branch(project_dir)
|
||||
if branch:
|
||||
try:
|
||||
subprocess.run(
|
||||
['git', 'branch', f'--set-upstream-to=origin/{branch}', branch],
|
||||
capture_output=True, text=True, timeout=10, cwd=project_dir)
|
||||
except (subprocess.TimeoutExpired, OSError) as exc:
|
||||
logger.debug("could not set upstream for %s: %s", branch, exc)
|
||||
|
||||
# Return custom response for git_pull
|
||||
if result.returncode == 0:
|
||||
pull_message = "Code updated successfully."
|
||||
if has_changes:
|
||||
pull_message = f"Code updated successfully. Local changes were automatically stashed.{stash_info}"
|
||||
if result.stdout and "Already up to date" not in result.stdout:
|
||||
pull_message = f"Code updated successfully.{stash_info}"
|
||||
if upstream_note:
|
||||
pull_message = f"{pull_message} {upstream_note}"
|
||||
|
||||
# Keep Python dependencies in sync automatically: if the pull
|
||||
# changed a requirements file, install it now — users updating
|
||||
# from the web UI (most of them) never SSH in to pip install.
|
||||
# Installs go through the same root-visible path as the
|
||||
# Tools-tab buttons (_pip_install_requirements).
|
||||
dep_notes = []
|
||||
try:
|
||||
_post = subprocess.run(['git', 'rev-parse', 'HEAD'],
|
||||
capture_output=True, text=True, timeout=10, cwd=project_dir)
|
||||
new_head = _post.stdout.strip() if _post.returncode == 0 else None
|
||||
if old_head and new_head and old_head != new_head:
|
||||
code_changed = True
|
||||
diff = subprocess.run(
|
||||
['git', 'diff', '--name-only', f'{old_head}..{new_head}'],
|
||||
capture_output=True, text=True, timeout=15, cwd=project_dir)
|
||||
changed = set(diff.stdout.split()) if diff.returncode == 0 else set()
|
||||
for rel in ('requirements.txt', 'web_interface/requirements.txt'):
|
||||
req_path = PROJECT_ROOT / rel
|
||||
if rel not in changed or not req_path.exists():
|
||||
continue
|
||||
# Each file's install is isolated: a timeout or
|
||||
# OSError (e.g. the sudo wrapper/interpreter
|
||||
# missing) on one file must not abort the other.
|
||||
try:
|
||||
r = _pip_install_requirements(req_path, timeout=180)
|
||||
if r.returncode == 0:
|
||||
dep_notes.append(f"Dependencies from {rel} updated.")
|
||||
else:
|
||||
dep_notes.append(
|
||||
f"Dependency install from {rel} failed — "
|
||||
"run Install Base Requirements from the Tools tab.")
|
||||
logger.warning("post-update pip install failed for %s: %s",
|
||||
rel, _truncate_output(r.stdout, r.stderr))
|
||||
except subprocess.TimeoutExpired:
|
||||
dep_notes.append(
|
||||
f"Dependency install from {rel} timed out — "
|
||||
"run Install Base Requirements from the Tools tab.")
|
||||
logger.warning("post-update pip install timed out for %s", rel)
|
||||
except OSError as install_err:
|
||||
dep_notes.append(
|
||||
f"Dependency install from {rel} failed — "
|
||||
"run Install Base Requirements from the Tools tab.")
|
||||
logger.warning("post-update pip install errored for %s: %s",
|
||||
rel, install_err)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("post-update dependency sync timed out")
|
||||
if dep_notes:
|
||||
pull_message += " " + " ".join(dep_notes)
|
||||
# A `git pull` restores built-in plugins (committed under
|
||||
# plugin-repos/) even if the user uninstalled them. Re-remove
|
||||
# any the user previously uninstalled so the update doesn't
|
||||
# resurrect them.
|
||||
if api_v3.plugin_store_manager:
|
||||
try:
|
||||
purged = api_v3.plugin_store_manager.purge_uninstalled_plugins()
|
||||
if purged:
|
||||
logger.info(
|
||||
"Re-removed %d uninstalled plugin(s) restored by update: %s",
|
||||
len(purged), ", ".join(purged),
|
||||
)
|
||||
except (OSError, RuntimeError) as purge_err:
|
||||
logger.warning("Post-update plugin purge failed: %s", purge_err)
|
||||
else:
|
||||
logger.warning("git pull failed (returncode=%d): %s", result.returncode, result.stderr)
|
||||
# Show git's own first line: "check logs" leaves the user with
|
||||
# nothing to act on, and these failures are usually actionable
|
||||
# (conflicting local commits, no upstream, network).
|
||||
detail = next((ln.strip() for ln in (result.stderr or '').splitlines()
|
||||
if ln.strip()), '')
|
||||
pull_message = f"Update failed: {detail}" if detail else "Update failed; check logs for details"
|
||||
|
||||
# Nothing here restarts anything: the pull replaces files on
|
||||
# disk while the display and web services keep running the code
|
||||
# they loaded at boot. Without this the user is told the update
|
||||
# succeeded and sees no change until they happen to reboot.
|
||||
return jsonify({
|
||||
'status': 'success' if result.returncode == 0 else 'error',
|
||||
'message': pull_message,
|
||||
'restart_required': bool(result.returncode == 0 and code_changed),
|
||||
})
|
||||
return jsonify(perform_core_update())
|
||||
elif action == 'checkout_branch':
|
||||
# Switch branches from the Tools tab. Needed because a checkout
|
||||
# that predates tracking (or a restored backup) can leave the pi
|
||||
|
||||
@@ -525,8 +525,15 @@ def _load_general_partial():
|
||||
try:
|
||||
if pages_v3.config_manager:
|
||||
main_config = pages_v3.config_manager.load_config()
|
||||
try:
|
||||
from web_interface.auto_update import describe_status
|
||||
auto_update_status = describe_status(main_config)
|
||||
except Exception:
|
||||
logger.debug("Could not read auto-update status", exc_info=True)
|
||||
auto_update_status = None
|
||||
return render_template('v3/partials/general.html',
|
||||
main_config=main_config)
|
||||
main_config=main_config,
|
||||
auto_update_status=auto_update_status)
|
||||
except Exception as e:
|
||||
logger.error("Error loading partial", exc_info=True)
|
||||
return "Error loading partial", 500
|
||||
|
||||
@@ -100,8 +100,9 @@ def main():
|
||||
werkzeug_logger.error = log_exception_filtered
|
||||
|
||||
# Import and run the Flask app
|
||||
from web_interface.app import app
|
||||
|
||||
from web_interface.app import app, start_auto_update_scheduler
|
||||
start_auto_update_scheduler()
|
||||
|
||||
print("Starting LED Matrix Web Interface V3...")
|
||||
print("Web server binding to: 0.0.0.0:5000")
|
||||
|
||||
|
||||
@@ -111,6 +111,37 @@
|
||||
|
||||
// ─── Helpers ────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* "x-display": "hidden" — the property stays in the schema but gets no
|
||||
* control. An object whose every child is hidden counts as hidden too.
|
||||
* Mirrors prop_is_hidden in plugin_config.html and _is_hidden_prop on the
|
||||
* server.
|
||||
*/
|
||||
function isHiddenProp(schema) {
|
||||
if (!schema || typeof schema !== 'object') return false;
|
||||
if (schema['x-display'] === 'hidden') return true;
|
||||
const children = schema.properties;
|
||||
if (children && typeof children === 'object') {
|
||||
const values = Object.values(children);
|
||||
return values.length > 0 && values.every(isHiddenProp);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* A hidden input carrying a hidden property's stored value, JSON-encoded so
|
||||
* the save restores its exact type. A posted row replaces the stored item,
|
||||
* so a value that isn't posted is lost.
|
||||
*/
|
||||
function hiddenValueInput(name, dataKey, value) {
|
||||
const input = document.createElement('input');
|
||||
input.type = 'hidden';
|
||||
input.name = name;
|
||||
input.value = JSON.stringify(value);
|
||||
input.dataset.hiddenProp = dataKey;
|
||||
return input;
|
||||
}
|
||||
|
||||
function safeSetHTML(target, html) {
|
||||
target.textContent = '';
|
||||
// createContextualFragment parses html relative to the document context
|
||||
@@ -343,6 +374,16 @@
|
||||
if (propType === 'object' && propSchema.properties) {
|
||||
const nestedVal = (item && item[propName]) || {};
|
||||
Object.entries(propSchema.properties).forEach(([subName, subSchema]) => {
|
||||
if (isHiddenProp(subSchema)) {
|
||||
// No data-nested-prop, so the row editor never shows it.
|
||||
const stored = Object.entries(nestedVal).find(([name]) => name === subName);
|
||||
if (stored && stored[1] !== undefined && stored[1] !== null) {
|
||||
cell.appendChild(hiddenValueInput(
|
||||
`${fullKey}.${index}.${propName}.${subName}`,
|
||||
`${propName}.${subName}`, stored[1]));
|
||||
}
|
||||
return;
|
||||
}
|
||||
const subType = Array.isArray(subSchema.type)
|
||||
? subSchema.type.find(t => t !== 'null') || 'string'
|
||||
: (subSchema.type || 'string');
|
||||
@@ -386,7 +427,16 @@
|
||||
row.className = 'array-table-row';
|
||||
row.setAttribute('data-index', index);
|
||||
|
||||
// "x-display": "hidden" item properties: never a column (even if
|
||||
// x-columns names one), never in the advanced cell or row editor.
|
||||
const hiddenKeys = new Set(
|
||||
Object.entries(fullItemProperties)
|
||||
.filter(([, propSchema]) => isHiddenProp(propSchema))
|
||||
.map(([propName]) => propName)
|
||||
);
|
||||
|
||||
// Visible column cells
|
||||
displayColumns = displayColumns.filter(colName => !hiddenKeys.has(colName));
|
||||
displayColumns.forEach(colName => {
|
||||
const colDef = itemProperties[colName] || {};
|
||||
const colType = Array.isArray(colDef.type) ? colDef.type.find(t => t !== 'null') || 'string' : (colDef.type || 'string');
|
||||
@@ -396,10 +446,11 @@
|
||||
row.appendChild(createCell(fullKey, index, colName, colDef, colValue, pluginId));
|
||||
});
|
||||
|
||||
// Determine non-displayed properties (these go into the advanced cell + edit modal)
|
||||
// Determine non-displayed properties (these go into the advanced cell + edit modal).
|
||||
// Hidden ones ("x-display": "hidden") get no column and no editor field.
|
||||
const nonDisplayed = {};
|
||||
Object.keys(fullItemProperties).forEach(k => {
|
||||
if (!displayColumns.includes(k) && k !== 'id') {
|
||||
if (!displayColumns.includes(k) && k !== 'id' && !hiddenKeys.has(k)) {
|
||||
nonDisplayed[k] = fullItemProperties[k];
|
||||
}
|
||||
});
|
||||
@@ -430,6 +481,14 @@
|
||||
actionsCell.appendChild(editBtn);
|
||||
}
|
||||
|
||||
// Hidden properties: carry stored values only. A new row has none, so
|
||||
// nothing is invented (countdown fills in its own id).
|
||||
Object.entries(item).forEach(([propName, stored]) => {
|
||||
if (hiddenKeys.has(propName) && stored !== undefined && stored !== null) {
|
||||
actionsCell.appendChild(hiddenValueInput(`${fullKey}.${index}.${propName}`, propName, stored));
|
||||
}
|
||||
});
|
||||
|
||||
row.appendChild(actionsCell);
|
||||
|
||||
// Hidden advanced data cell
|
||||
@@ -494,7 +553,8 @@
|
||||
grid.className = 'grid grid-cols-2 gap-3';
|
||||
|
||||
Object.entries(propSchema.properties).forEach(([subName, subSchema]) => {
|
||||
const subType = Array.isArray(subSchema.type) ? subSchema.type.find(t => t !== 'null') || 'string' : (subSchema.type || 'string');
|
||||
if (isHiddenProp(subSchema)) return;
|
||||
const subType =Array.isArray(subSchema.type) ? subSchema.type.find(t => t !== 'null') || 'string' : (subSchema.type || 'string');
|
||||
const subLabel = subSchema.title || subName.replace(/_/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
|
||||
const subDesc = subSchema.description || '';
|
||||
const nestedPath = `${propName}.${subName}`;
|
||||
|
||||
@@ -2409,6 +2409,19 @@ function getSchemaProperty(schema, path) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// "x-display": "hidden" (or an object of nothing but hidden children).
|
||||
// Mirrors prop_is_hidden in plugin_config.html.
|
||||
function isHiddenSchemaProp(schema) {
|
||||
if (!schema || typeof schema !== 'object') return false;
|
||||
if (schema['x-display'] === 'hidden') return true;
|
||||
const children = schema.properties;
|
||||
if (children && typeof children === 'object') {
|
||||
const values = Object.values(children);
|
||||
return values.length > 0 && values.every(isHiddenSchemaProp);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Helper function to render a single item in an array of objects
|
||||
function renderArrayObjectItem(fieldId, fullKey, itemProperties, itemValue, index, itemsSchema) {
|
||||
const item = itemValue || {};
|
||||
@@ -2422,6 +2435,9 @@ function renderArrayObjectItem(fieldId, fullKey, itemProperties, itemValue, inde
|
||||
const propertyOrder = itemsSchema['x-propertyOrder'] || Object.keys(itemProperties);
|
||||
propertyOrder.forEach(propKey => {
|
||||
if (!itemProperties[propKey]) return;
|
||||
// "x-display": "hidden": no control. The stored value survives through
|
||||
// data-item-data, which updateArrayObjectData overlays edits onto.
|
||||
if (isHiddenSchemaProp(itemProperties[propKey])) return;
|
||||
|
||||
const propSchema = itemProperties[propKey];
|
||||
const propValue = item[propKey] !== undefined ? item[propKey] : propSchema.default;
|
||||
@@ -5478,6 +5494,7 @@ if (typeof window !== 'undefined') {
|
||||
itemHtml = `<div class="border border-gray-300 rounded-lg p-4 bg-gray-50 array-object-item" data-index="${newIndex}">`;
|
||||
Object.keys(itemsSchema.properties || {}).forEach(propKey => {
|
||||
const propSchema = itemsSchema.properties[propKey];
|
||||
if (propSchema && propSchema['x-display'] === 'hidden') return;
|
||||
const propValue = newItem[propKey] !== undefined ? newItem[propKey] : propSchema.default;
|
||||
const propLabel = propSchema.title || propKey.replace(/_/g, ' ').replace(/\b\w/g, l => l.toUpperCase());
|
||||
itemHtml += `<div class="mb-3"><label class="block text-sm font-medium text-gray-700 mb-1">${escapeHtml(propLabel)}</label>`;
|
||||
|
||||
@@ -44,6 +44,44 @@
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<!-- Weekly Automatic Updates -->
|
||||
<div class="form-group" id="setting-general-auto_update" data-setting-key="auto_update.enabled">
|
||||
<label class="flex items-center">
|
||||
<input type="checkbox"
|
||||
name="auto_update_enabled"
|
||||
value="true"
|
||||
{% if (main_config.auto_update or {}).enabled %}checked{% endif %}
|
||||
class="form-control h-4 w-4 text-blue-600 focus:ring-blue-500 border-gray-300 rounded">
|
||||
<span class="ml-2 text-sm font-medium text-gray-900">Automatically check for and install updates once a week</span>
|
||||
{{ ui.help_tip('Once a week, update LEDMatrix and every installed plugin that has a newer version.\nSkipped (and reported) if you have local changes, low disk space, or the rollback service is missing. After updating, the services are restarted and checked; if they do not stay healthy, the update is rolled back automatically.\nRuns overnight (2-5 AM in your timezone) when possible. Problems show as a banner on Overview. A stopped display is not started. Default: off.', 'Automatic Updates') }}
|
||||
</label>
|
||||
{% if auto_update_status and (main_config.auto_update or {}).enabled and auto_update_status.verifier_installed is sameas false %}
|
||||
<p class="mt-1 ml-6 text-xs text-amber-700">
|
||||
{% if auto_update_status.setup_status == 'failed' %}
|
||||
Setting up the update health check failed: {{ auto_update_status.setup_message }}
|
||||
LEDMatrix code updates are paused until it is fixed; plugin updates still run.
|
||||
{% else %}
|
||||
Setting up the update health check. The display service does this when it starts, so it is
|
||||
ready shortly after saving (or once the display is started). LEDMatrix code updates begin after that.
|
||||
{% endif %}
|
||||
</p>
|
||||
{% endif %}
|
||||
{% if auto_update_status and auto_update_status.verifying %}
|
||||
<p class="mt-1 ml-6 text-xs text-blue-700">An update was just installed and its health check is running.</p>
|
||||
{% endif %}
|
||||
{% if auto_update_status and (auto_update_status.last_run or auto_update_status.next_due) %}
|
||||
<div class="mt-1 ml-6 text-xs text-gray-500 space-y-1">
|
||||
{% if auto_update_status.last_run %}
|
||||
<p>Last automatic update: {{ auto_update_status.last_run }}
|
||||
<span class="{{ 'text-red-600' if auto_update_status.status == 'error' else '' }}">— {{ auto_update_status.summary }}</span></p>
|
||||
{% endif %}
|
||||
{% if auto_update_status.next_due %}
|
||||
<p>Next check: {{ auto_update_status.next_due }} or the first overnight window after it</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Timezone -->
|
||||
<div class="form-group" id="setting-general-timezone" data-setting-key="timezone">
|
||||
<label for="timezone" class="block text-sm font-medium text-gray-700">Timezone{{ ui.help_tip('Time zone used for clocks, schedules, and time-based content.\nChoose the zone where the display physically lives so on/off schedules fire at the correct local time.', 'Timezone') }}</label>
|
||||
|
||||
@@ -1,3 +1,44 @@
|
||||
<!-- Automatic update banner: a weekly update that failed, was blocked or was rolled back -->
|
||||
<div id="auto-update-banner" class="bg-red-50 border border-red-300 rounded-lg p-4 mb-4 flex items-start" style="display:none !important" role="alert">
|
||||
<div class="flex-shrink-0 mr-3 mt-0.5">
|
||||
<i class="fas fa-exclamation-circle text-red-500"></i>
|
||||
</div>
|
||||
<div class="flex-1">
|
||||
<p class="text-sm font-medium text-red-800">Automatic Update Needs Attention</p>
|
||||
<p class="text-sm text-red-700 mt-1" id="auto-update-banner-text"></p>
|
||||
<p class="text-xs text-red-600 mt-1">Details are under Automatic Updates on the General tab.</p>
|
||||
</div>
|
||||
<button type="button" onclick="window.dismissAutoUpdateBanner()" class="ml-4 flex-shrink-0 text-red-500 hover:text-red-600" aria-label="Dismiss">
|
||||
<i class="fas fa-times"></i>
|
||||
</button>
|
||||
</div>
|
||||
<script>
|
||||
(function () {
|
||||
fetch('/api/v3/system/auto-update')
|
||||
.then(function (r) { return r.json(); })
|
||||
.then(function (resp) {
|
||||
var d = resp.data || {};
|
||||
if (!d.alert) return;
|
||||
document.getElementById('auto-update-banner-text').textContent = d.alert;
|
||||
var banner = document.getElementById('auto-update-banner');
|
||||
banner.dataset.alertId = d.alert_id || '';
|
||||
banner.style.setProperty('display', 'flex', 'important');
|
||||
})
|
||||
.catch(function () {});
|
||||
|
||||
// Dismissal is stored server-side, so the banner stays gone on every
|
||||
// device until a new failure raises a new alert.
|
||||
window.dismissAutoUpdateBanner = function () {
|
||||
var banner = document.getElementById('auto-update-banner');
|
||||
banner.style.setProperty('display', 'none', 'important');
|
||||
fetch('/api/v3/system/auto-update/dismiss', {
|
||||
method: 'POST',
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
body: JSON.stringify({alert_id: banner.dataset.alertId})
|
||||
}).catch(function () {});
|
||||
};
|
||||
}());
|
||||
</script>
|
||||
<!-- Reconciliation warning banner: shown when startup reconciliation found stale plugin config entries -->
|
||||
<div id="reconciliation-banner" class="bg-yellow-50 border border-yellow-300 rounded-lg p-4 mb-4 flex items-start" style="display:none !important" role="alert">
|
||||
<div class="flex-shrink-0 mr-3 mt-0.5">
|
||||
|
||||
@@ -6,7 +6,29 @@
|
||||
{# ===== MACROS FOR FORM FIELD GENERATION ===== #}
|
||||
|
||||
{# Render a single form field based on schema type #}
|
||||
{# "x-display": "hidden" keeps a property declared -- so stored configs that
|
||||
still carry it keep validating -- without drawing a control for it
|
||||
(deprecated keys, internal ids). An object whose every child is hidden
|
||||
counts as hidden too, so it doesn't render as an empty section. Outputs
|
||||
"1" when hidden, nothing otherwise; test it with `|trim`.
|
||||
Nothing hidden is posted for a plain or nested field: the save deep-merges
|
||||
the form over the stored section, so the stored value stays. Array rows are
|
||||
the exception (a posted list replaces the stored one), see array-table. #}
|
||||
{% macro prop_is_hidden(prop) -%}
|
||||
{%- if prop is mapping -%}
|
||||
{%- if prop.get('x-display') == 'hidden' -%}1
|
||||
{%- elif prop.get('properties') is mapping and prop.get('properties') -%}
|
||||
{%- set _hid = namespace(all=true) -%}
|
||||
{%- for _child in prop.get('properties').values() -%}
|
||||
{%- if not prop_is_hidden(_child)|trim -%}{%- set _hid.all = false -%}{%- endif -%}
|
||||
{%- endfor -%}
|
||||
{%- if _hid.all -%}1{%- endif -%}
|
||||
{%- endif -%}
|
||||
{%- endif -%}
|
||||
{%- endmacro %}
|
||||
|
||||
{% macro render_field(key, prop, value, prefix='', plugin_id='') %}
|
||||
{% if not prop_is_hidden(prop)|trim %}
|
||||
{% set full_key = (prefix ~ '.' ~ key) if prefix else key %}
|
||||
{% set field_id = (plugin_id ~ '-' ~ full_key)|replace('.', '-')|replace('_', '-') %}
|
||||
{% set label = prop.title if prop.title else key|replace('_', ' ')|title %}
|
||||
@@ -660,14 +682,20 @@
|
||||
{% set array_value = value if value is not none and value is iterable and value is not string else (prop.default if prop.default is defined and prop.default is iterable and prop.default is not string else []) %}
|
||||
|
||||
{# Use x-columns if specified, otherwise auto-detect first 4 simple properties #}
|
||||
{# Hidden item properties never become columns (see prop_is_hidden). #}
|
||||
{% set x_columns = prop.get('x-columns') %}
|
||||
{% if x_columns %}
|
||||
{% set display_columns = x_columns %}
|
||||
{% set display_columns = [] %}
|
||||
{% for col_name in x_columns %}
|
||||
{% if not prop_is_hidden(item_properties.get(col_name))|trim %}
|
||||
{% set _ = display_columns.append(col_name) %}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
{% set display_columns = [] %}
|
||||
{% for col_name in item_properties.keys() %}
|
||||
{% set col_def = item_properties[col_name] %}
|
||||
{% if col_def.get('type') not in ['object', 'array'] and display_columns|length < 4 %}
|
||||
{% if col_def.get('type') not in ['object', 'array'] and display_columns|length < 4 and not prop_is_hidden(col_def)|trim %}
|
||||
{% set _ = display_columns.append(col_name) %}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
@@ -814,8 +842,16 @@
|
||||
|
||||
{# Actions cell: delete + optional edit button for advanced props #}
|
||||
{% set has_advanced = namespace(value=false) %}
|
||||
{% for k in item_properties.keys() %}{% if k not in display_columns and k != 'id' %}{% set has_advanced.value = true %}{% endif %}{% endfor %}
|
||||
{% for k, v in item_properties.items() %}{% if k not in display_columns and k != 'id' and not prop_is_hidden(v)|trim %}{% set has_advanced.value = true %}{% endif %}{% endfor %}
|
||||
<td class="px-3 py-3 whitespace-nowrap text-center" style="min-width:90px;vertical-align:middle">
|
||||
{# Hidden item properties (e.g. countdown's auto-generated id) get
|
||||
no control, but a posted row replaces the stored item wholesale,
|
||||
so carry the stored value through. JSON-encoded so the save can
|
||||
restore its exact type (an id "1" must stay a string); absent
|
||||
values are not invented. #}
|
||||
{% for k, v in item_properties.items() %}{% if prop_is_hidden(v)|trim and k in item and item[k] is not none %}
|
||||
<input type="hidden" name="{{ full_key }}.{{ item_index }}.{{ k }}" data-hidden-prop="{{ k }}" value='{{ item[k]|tojson }}'>
|
||||
{% endif %}{% endfor %}
|
||||
<button type="button"
|
||||
onclick="removeArrayTableRow(this)"
|
||||
aria-label="Remove row {{ item_index + 1 }}"
|
||||
@@ -836,13 +872,23 @@
|
||||
{# Hidden cell: flat hidden inputs for non-displayed props (layout, style, etc.) #}
|
||||
{% if has_advanced.value %}
|
||||
{% set adv_schema = namespace(d={}) %}
|
||||
{% for k, v in item_properties.items() %}{% if k not in display_columns and k != 'id' %}{% set _ = adv_schema.d.update({k: v}) %}{% endif %}{% endfor %}
|
||||
{% for k, v in item_properties.items() %}{% if k not in display_columns and k != 'id' and not prop_is_hidden(v)|trim %}{% set _ = adv_schema.d.update({k: v}) %}{% endif %}{% endfor %}
|
||||
<td style="display:none" class="array-table-advanced-data"
|
||||
data-prop-schema='{{ adv_schema.d|tojson }}'>
|
||||
{% for prop_name, prop_schema in adv_schema.d.items() %}
|
||||
{% set prop_type = prop_schema.get('type', 'string') %}
|
||||
{% if prop_type == 'object' and prop_schema.get('properties') %}
|
||||
{% for sub_name, sub_schema in prop_schema.get('properties', {}).items() %}
|
||||
{% set _sub_obj = item.get(prop_name) if item.get(prop_name) is mapping else {} %}
|
||||
{% if prop_is_hidden(sub_schema)|trim %}
|
||||
{# Kept out of the row editor (no data-nested-prop); stored value only. #}
|
||||
{% if sub_name in _sub_obj and _sub_obj[sub_name] is not none %}
|
||||
<input type="hidden"
|
||||
name="{{ full_key }}.{{ item_index }}.{{ prop_name }}.{{ sub_name }}"
|
||||
data-hidden-prop="{{ prop_name }}.{{ sub_name }}"
|
||||
value='{{ _sub_obj[sub_name]|tojson }}'>
|
||||
{% endif %}
|
||||
{% else %}
|
||||
{% set sub_val = item.get(prop_name, {}).get(sub_name) %}
|
||||
{% set sub_default = sub_schema.get('default') %}
|
||||
{% set final_val = sub_val if sub_val is not none else sub_default %}
|
||||
@@ -852,6 +898,7 @@
|
||||
data-prop-type="{{ sub_schema.get('type', 'string') }}"
|
||||
data-prop-schema='{{ sub_schema|tojson }}'
|
||||
value="{{ final_val if final_val is not none else '' }}">
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
{% set prop_val = item.get(prop_name) %}
|
||||
@@ -1104,6 +1151,7 @@
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endif %}{# not prop_is_hidden #}
|
||||
{% endmacro %}
|
||||
|
||||
{# Render a nested/collapsible section for object types #}
|
||||
@@ -1142,7 +1190,7 @@
|
||||
<div id="{{ section_id }}" class="nested-content bg-gray-50 px-4 py-4 space-y-3 hidden" style="display: none;">
|
||||
{% set property_order = prop['x-propertyOrder'] if 'x-propertyOrder' in prop else prop.properties.keys()|list %}
|
||||
{% for nested_key in property_order %}
|
||||
{% if nested_key in prop.properties %}
|
||||
{% if nested_key in prop.properties and not prop_is_hidden(prop.properties[nested_key])|trim %}
|
||||
{% set nested_prop = prop.properties[nested_key] %}
|
||||
{% set nested_val = nested_value[nested_key] if nested_key in nested_value else none %}
|
||||
{# Tagged so a widget that takes over part of a block can
|
||||
@@ -1295,7 +1343,10 @@
|
||||
them. Schemas without the flag render exactly as before. #}
|
||||
{% set tiers = namespace(basic=[], advanced=[]) %}
|
||||
{% for key in property_order %}
|
||||
{% if key in schema.properties and key != 'enabled' %}
|
||||
{# A hidden key is neither drawn nor reported in
|
||||
__rendered_section, so the save never reads its
|
||||
absence as an unchecked box. #}
|
||||
{% if key in schema.properties and key != 'enabled' and not prop_is_hidden(schema.properties[key])|trim %}
|
||||
{% set prop = schema.properties[key] %}
|
||||
{% set is_object = prop.type is defined and 'object' in prop.type %}
|
||||
{% if prop.get('x-advanced') and not is_object %}
|
||||
|
||||
Reference in New Issue
Block a user