mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-04 06:15:09 +00:00
feat(install): updates refresh systemd units; new installs run the newest release (#729)
Updates that move HEAD now install changed systemd units through a root-owned helper (/usr/local/sbin/ledmatrix-refresh-units, two literal sudo lines), with a backup restored on rollback; a refresh that fails part-way puts the old units back. Devices without the new sudo rule keep updating and are told to re-run the installer once. The one-shot installer now checks out the newest vX.Y.Z release (LEDMATRIX_CHANNEL=beta keeps main) and never moves an existing checkout backwards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
@@ -10,3 +10,6 @@
|
||||
# Generated by scripts/build_css.py; collapsed in diffs, not hand-edited.
|
||||
web_interface/static/v3/tailwind.css linguist-generated=true
|
||||
web_interface/static/v3/plugin-frame.css linguist-generated=true
|
||||
|
||||
# Installed as an executable (its shebang runs it) by install_service.sh.
|
||||
scripts/install/ledmatrix_refresh_units.py text eol=lf
|
||||
|
||||
@@ -19,6 +19,49 @@ accepts both, but the store flags the old spelling as deprecated
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Updates refresh the systemd units; new installs run the newest release
|
||||
|
||||
- **Updates now install changed systemd units.** An update (Update Code, or
|
||||
the weekly automatic update) moved the checkout's `systemd/*.service`
|
||||
templates but never the units systemd runs, so settings added after a
|
||||
device was installed -- #687's render-loop watchdog, for one -- only ever
|
||||
arrived with a reinstall. After an update that moves HEAD, the web
|
||||
interface compares the installed `ledmatrix.service`,
|
||||
`ledmatrix-web.service` and `ledmatrix-update-verify.{service,path}` with
|
||||
the new templates (rendered exactly as `install_service.sh` does, comments
|
||||
ignored as the startup drift warning does) and, when they differ, runs the
|
||||
new root-owned helper `/usr/local/sbin/ledmatrix-refresh-units`
|
||||
(`scripts/install/ledmatrix_refresh_units.py`) through sudo: it installs
|
||||
the changed units and runs `systemctl daemon-reload`, so the restart that
|
||||
follows the update runs under them. Update Code's message says so.
|
||||
- **Rollback restores them.** The helper keeps the units it replaced
|
||||
(`/var/lib/ledmatrix/unit-backup`, root only); when the automatic update's
|
||||
health check rolls an update back, it runs `ledmatrix-refresh-units
|
||||
--restore` before restarting the services onto the old code.
|
||||
- **The sudo rule needs a reinstall.** `install_service.sh` installs the
|
||||
helper and `lib_sudoers.sh` grants it with exactly two command lines (no
|
||||
arguments, and `--restore`). A device installed before this has neither;
|
||||
its updates keep working, log that the new unit settings need a reinstall
|
||||
and say so in Update Code's message, the same remedy as the startup
|
||||
"unit drift" warning. Re-run `sudo ./first_time_install.sh` once (or
|
||||
`sudo ./scripts/install/install_service.sh` then
|
||||
`./scripts/install/configure_web_sudo.sh`).
|
||||
- `install_service.sh` now leaves the units it installs mode `0644`, as
|
||||
`first_time_install.sh` already did; run on its own it left them `0600`.
|
||||
- **New installs run the newest release.** The one-shot installer cloned
|
||||
`main`'s tip, so a new device ran unreleased code until the next release.
|
||||
It now checks out the newest `vX.Y.Z` tag after cloning (the same semver
|
||||
rules as `web_interface/update_channel.py`), and that release's own
|
||||
`first_time_install.sh` runs. `LEDMATRIX_CHANNEL=beta` installs `main`
|
||||
instead and records the beta channel; `first_time_install.sh --beta` (or
|
||||
`LEDMATRIX_CHANNEL=beta|stable`) records a channel for a manual install.
|
||||
- **Re-running the one-shot never moves backwards.** On an existing stable
|
||||
checkout it moves to the newest release only when that release contains
|
||||
the current commit; a checkout newer than every release keeps its
|
||||
fast-forward pull (on a branch) or stays put (detached), and beta keeps the
|
||||
pull it always had. It used to fast-forward a detached release checkout to
|
||||
`main`'s tip.
|
||||
|
||||
### Control socket stage 3: the display's state over the socket
|
||||
|
||||
- Two new commands, still protocol version 1. `state.get` returns a
|
||||
|
||||
@@ -39,6 +39,17 @@ Raspberry Pi OS Lite yourself:
|
||||
[README Installation Steps / Quick Install](../README.md#installation-steps)
|
||||
for full details
|
||||
|
||||
The one-shot installer installs the newest release (the **stable** update
|
||||
channel). To run the newest, unreleased code from `main` instead (the
|
||||
**beta** channel), put `LEDMATRIX_CHANNEL=beta` in front of `bash`:
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | LEDMATRIX_CHANNEL=beta bash
|
||||
```
|
||||
A manual clone starts on `main`; add `--beta` to `first_time_install.sh`
|
||||
to stay on it, or leave it off and the first update after the next
|
||||
release moves the device onto releases. You can switch channels later on
|
||||
the General tab.
|
||||
|
||||
**Expected Behavior after install:**
|
||||
- LED matrix will light up
|
||||
- A fresh install ships only the bundled `starlark-apps` and
|
||||
|
||||
+20
-1
@@ -33,6 +33,9 @@ in again (services pick them up on restart).
|
||||
| `/run/ledmatrix/control.sock` | `root` : cache directory's group (`ledmatrix`) | `660` | The display's control socket; only root and that group can connect. See [IPC_CONTROL_SOCKET.md](IPC_CONTROL_SOCKET.md#security-model) |
|
||||
| `scripts/fix_perms/safe_plugin_rm.sh`, `safe_pip_install.sh` | `root:root` | `755` | Run as root through sudo, so the web user must not be able to edit them |
|
||||
| `/etc/sudoers.d/ledmatrix_web`, `ledmatrix_wifi` | `root` | `440` | |
|
||||
| `/usr/local/sbin/ledmatrix-refresh-units` | `root:root` | `755` | Copy of `scripts/install/ledmatrix_refresh_units.py`, installed by `install_service.sh`. Outside the project so the web user cannot edit what sudo runs |
|
||||
| `/var/lib/ledmatrix/unit-backup/` | `root` | `700` | The units the last refresh replaced, for the automatic update's rollback |
|
||||
| `/etc/systemd/system/ledmatrix*.service`, `.path` | `root:root` | `644` | Readable so the web interface can compare them with the templates after an update |
|
||||
|
||||
What keeps it that way at runtime:
|
||||
|
||||
@@ -86,6 +89,20 @@ password:
|
||||
- `journalctl -u ledmatrix.service *`, `-u ledmatrix *`, `-t ledmatrix *`,
|
||||
tagged `NOEXEC`: journalctl opens a pager on a terminal, and a shell
|
||||
escape from that pager would be a root shell
|
||||
- `/usr/local/sbin/ledmatrix-refresh-units ""` and
|
||||
`/usr/local/sbin/ledmatrix-refresh-units --restore` — exactly these two
|
||||
command lines (`""` means "no arguments"). After an update the first
|
||||
installs the systemd units whose templates changed and runs
|
||||
`systemctl daemon-reload`; the automatic update's rollback runs the second
|
||||
to put the previous units back. The helper takes nothing from the caller:
|
||||
the project folder and the web user come from the installed, root-owned
|
||||
`ledmatrix.service` and `ledmatrix-web.service`. It only replaces the four
|
||||
units `install_service.sh` installs, only if they are already installed,
|
||||
and refuses a template that would change a unit's `User=` (root for the
|
||||
display, the web user for the rest) or `WorkingDirectory=`, or that is a
|
||||
symlink, not a regular file, or over 64 KB. It grants nothing new: the
|
||||
templates are files the web user can edit, but so is `run.py`, which the
|
||||
display service already runs as root.
|
||||
|
||||
### `/etc/sudoers.d/ledmatrix_wifi`
|
||||
|
||||
@@ -136,7 +153,9 @@ directory.
|
||||
| `safe_plugin_rm.sh`, `safe_pip_install.sh` | — | Called by the web interface through sudo | Not for manual use |
|
||||
|
||||
To reinstall the sudoers rules, run
|
||||
`./scripts/install/configure_web_sudo.sh` (web rules) or
|
||||
`./scripts/install/configure_web_sudo.sh` (web rules; the
|
||||
`ledmatrix-refresh-units` rules also need the helper itself, which
|
||||
`sudo ./scripts/install/install_service.sh` installs) or
|
||||
`./scripts/install/configure_wifi_permissions.sh` (WiFi rules and polkit) as
|
||||
the web user, not with `sudo`.
|
||||
|
||||
|
||||
+47
-3
@@ -328,6 +328,49 @@ commit, then switches to releases on its own.
|
||||
3. **Local changes after a channel switch:** edits that no longer fit the new
|
||||
version are kept in the git stash rather than lost; `git stash list`
|
||||
shows them as "LEDMatrix autostash before update".
|
||||
4. **A new install is on a release, not `main`.** The one-shot installer
|
||||
checks out the newest release. For the newest code instead, install with
|
||||
`LEDMATRIX_CHANNEL=beta`:
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | LEDMATRIX_CHANNEL=beta bash
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Issue: "service settings ... are not applied yet" after an update
|
||||
|
||||
**Symptoms:**
|
||||
- Update Code's message, or the web interface log, says an update changes
|
||||
service settings that are not applied yet, and to run the installer
|
||||
- The display logs `ledmatrix.service differs from systemd/ledmatrix.service`
|
||||
at startup
|
||||
|
||||
**Explanation:** updates install the systemd units a new version changes
|
||||
through the root helper `/usr/local/sbin/ledmatrix-refresh-units`, which the
|
||||
installer sets up and grants to the web user in
|
||||
`/etc/sudoers.d/ledmatrix_web`. A device installed before that has neither,
|
||||
so the new unit settings (for example the display's watchdog) wait for a
|
||||
reinstall. The update itself is fine.
|
||||
|
||||
**Solution:** re-run the installer once, as root:
|
||||
```bash
|
||||
cd ~/LEDMatrix
|
||||
sudo ./first_time_install.sh
|
||||
# or, lighter: install the units and helper, then the sudo rules
|
||||
sudo ./scripts/install/install_service.sh
|
||||
./scripts/install/configure_web_sudo.sh
|
||||
```
|
||||
Check it worked:
|
||||
```bash
|
||||
ls -l /usr/local/sbin/ledmatrix-refresh-units # root root, rwxr-xr-x
|
||||
sudo -l | grep ledmatrix-refresh-units # the two rules
|
||||
```
|
||||
A message that the helper **refused** a unit (`refusing to install it`)
|
||||
means a template in `systemd/` was edited so that it would run as another
|
||||
account or from another folder. The message names the template. Look at
|
||||
what changed with `git diff -- systemd/`, save any edit you want to keep,
|
||||
then restore only that file, for example
|
||||
`git checkout -- systemd/ledmatrix-web.service`.
|
||||
|
||||
---
|
||||
|
||||
@@ -590,9 +633,10 @@ stack into the log, so it says which plugin was stuck.
|
||||
apart, so a plugin that hangs on every start does not restart the display
|
||||
hundreds of times an hour.
|
||||
|
||||
4. **Is the watchdog installed?** Installs from before it keep their old unit
|
||||
until the installer is re-run (a startup warning says the unit differs
|
||||
from its template):
|
||||
4. **Is the watchdog installed?** Updates install new unit settings once the
|
||||
installer has set up `ledmatrix-refresh-units`; installs from before that
|
||||
keep their old unit until the installer is re-run (a startup warning says
|
||||
the unit differs from its template):
|
||||
```bash
|
||||
systemctl show -p WatchdogUSec ledmatrix # 2min once running; 0 = not installed
|
||||
sudo ./scripts/install/install_service.sh
|
||||
|
||||
+28
-7
@@ -223,6 +223,8 @@ SKIP_SWAP=${LEDMATRIX_SKIP_SWAP:-0}
|
||||
BUILD_JOBS_OVERRIDE=${LEDMATRIX_BUILD_JOBS:-}
|
||||
# Weekly automatic updates: 1 on, 0 off, empty = ask (interactive) or leave as is.
|
||||
AUTO_UPDATE=${LEDMATRIX_AUTO_UPDATE:-}
|
||||
# Update channel written to config.json: stable, beta, or empty = leave as is.
|
||||
UPDATE_CHANNEL=$(printf '%s' "${LEDMATRIX_CHANNEL:-}" | tr '[:upper:]' '[:lower:]')
|
||||
|
||||
usage() {
|
||||
cat <<USAGE
|
||||
@@ -240,12 +242,18 @@ Options:
|
||||
--enable-auto-update Turn on weekly automatic updates (with health
|
||||
check and automatic rollback)
|
||||
--no-auto-update Leave weekly automatic updates off
|
||||
--beta Follow main, the newest code (the beta update
|
||||
channel). Without it, updates follow releases
|
||||
(stable). It sets the channel; it does not move
|
||||
this checkout -- the one-shot installer picks the
|
||||
version, and so does the next update.
|
||||
-h, --help Show this help message and exit
|
||||
|
||||
Environment variables (same effect as flags):
|
||||
LEDMATRIX_ASSUME_YES=1, RPI_RGB_FORCE_REBUILD=1, LEDMATRIX_SKIP_SOUND=1,
|
||||
LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1,
|
||||
LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N, LEDMATRIX_AUTO_UPDATE=1|0
|
||||
LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N, LEDMATRIX_AUTO_UPDATE=1|0,
|
||||
LEDMATRIX_CHANNEL=stable|beta
|
||||
|
||||
Low-memory devices:
|
||||
On a Pi with under 2GB of RAM the C++ build is limited to fewer parallel
|
||||
@@ -265,6 +273,7 @@ while [ $# -gt 0 ]; do
|
||||
--skip-swap) SKIP_SWAP=1 ;;
|
||||
--enable-auto-update) AUTO_UPDATE=1 ;;
|
||||
--no-auto-update) AUTO_UPDATE=0 ;;
|
||||
--beta) UPDATE_CHANNEL=beta ;;
|
||||
--build-jobs)
|
||||
shift
|
||||
if [ $# -eq 0 ]; then echo "--build-jobs requires a number"; usage; exit 1; fi
|
||||
@@ -873,15 +882,25 @@ if [ -z "$AUTO_UPDATE" ] && [ "$ASSUME_YES" != "1" ] && [ -t 0 ]; then
|
||||
echo
|
||||
if [[ $REPLY =~ ^[Yy]$ ]]; then AUTO_UPDATE=1; else AUTO_UPDATE=0; fi
|
||||
fi
|
||||
if [ "$AUTO_UPDATE" = "1" ] || [ "$AUTO_UPDATE" = "0" ]; then
|
||||
if python3 - "$PROJECT_ROOT_DIR/config/config.json" "$AUTO_UPDATE" <<'PY'
|
||||
case "$UPDATE_CHANNEL" in
|
||||
stable|beta|"") ;;
|
||||
*) echo "⚠ LEDMATRIX_CHANNEL=$UPDATE_CHANNEL is not stable or beta; leaving the update channel as it is"
|
||||
UPDATE_CHANNEL="" ;;
|
||||
esac
|
||||
# The update channel, likewise only when asked for (--beta / LEDMATRIX_CHANNEL).
|
||||
if [ "$AUTO_UPDATE" = "1" ] || [ "$AUTO_UPDATE" = "0" ] || [ -n "$UPDATE_CHANNEL" ]; then
|
||||
if python3 - "$PROJECT_ROOT_DIR/config/config.json" "$AUTO_UPDATE" "$UPDATE_CHANNEL" <<'PY'
|
||||
import json, os, sys, tempfile
|
||||
path, enabled = sys.argv[1], sys.argv[2] == "1"
|
||||
path, enabled = sys.argv[1], sys.argv[2]
|
||||
channel = sys.argv[3] if len(sys.argv) > 3 else ""
|
||||
with open(path, encoding="utf-8") as f:
|
||||
config = json.load(f)
|
||||
if not isinstance(config.get("auto_update"), dict):
|
||||
config["auto_update"] = {}
|
||||
config["auto_update"]["enabled"] = enabled
|
||||
if enabled in ("0", "1"):
|
||||
config["auto_update"]["enabled"] = enabled == "1"
|
||||
if channel:
|
||||
config["auto_update"]["channel"] = channel
|
||||
# Written beside the original and swapped in whole: the display service's
|
||||
# config watcher may be running and must never read a half-written file.
|
||||
original = os.stat(path)
|
||||
@@ -902,9 +921,11 @@ except BaseException:
|
||||
raise
|
||||
PY
|
||||
then
|
||||
if [ "$AUTO_UPDATE" = "1" ]; then echo "✓ Weekly automatic updates enabled"; else echo "✓ Weekly automatic updates off"; fi
|
||||
if [ "$AUTO_UPDATE" = "1" ]; then echo "✓ Weekly automatic updates enabled"
|
||||
elif [ "$AUTO_UPDATE" = "0" ]; then echo "✓ Weekly automatic updates off"; fi
|
||||
if [ -n "$UPDATE_CHANNEL" ]; then echo "✓ Update channel: $UPDATE_CHANNEL"; fi
|
||||
else
|
||||
echo "⚠ Could not set auto_update in config/config.json; turn it on from the General tab instead"
|
||||
echo "⚠ Could not set auto_update in config/config.json; set it from the General tab instead"
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -5,11 +5,18 @@ This directory contains scripts for installing and configuring the LEDMatrix sys
|
||||
## Scripts
|
||||
|
||||
- **`one-shot-install.sh`** - Single-command installer; clones the
|
||||
repo, checks prerequisites, then runs `first_time_install.sh`.
|
||||
Invoked via `curl ... | bash` from the project root README.
|
||||
repo, checks out the newest release (or `main` with
|
||||
`LEDMATRIX_CHANNEL=beta`), checks prerequisites, then runs
|
||||
`first_time_install.sh`. Invoked via `curl ... | bash` from the project
|
||||
root README. Re-running it never moves a checkout to an older version.
|
||||
- **`install_service.sh`** - Installs, enables and starts the display
|
||||
service (`ledmatrix.service`), the web interface service
|
||||
(`ledmatrix-web.service`) and the update-verify units (systemd)
|
||||
(`ledmatrix-web.service`) and the update-verify units (systemd), and
|
||||
installs `/usr/local/sbin/ledmatrix-refresh-units`
|
||||
- **`ledmatrix_refresh_units.py`** - Not run from here: `install_service.sh`
|
||||
installs a root-owned copy as `/usr/local/sbin/ledmatrix-refresh-units`,
|
||||
which updates run through sudo to install changed units (and the
|
||||
automatic update's rollback, with `--restore`, to put them back)
|
||||
- **`install_web_service.sh`** - Installs only the web interface service
|
||||
and the update-verify units (systemd)
|
||||
- **`install_wifi_monitor.sh`** - Installs the WiFi monitor daemon service
|
||||
|
||||
@@ -138,6 +138,8 @@ echo "- View system logs via journalctl"
|
||||
echo "- Reboot and shutdown the system"
|
||||
echo "- Remove plugin directories (for update/uninstall when root-owned files block deletion)"
|
||||
echo "- Install plugin/base requirements.txt as root (so ledmatrix.service can see them)"
|
||||
echo "- Install the LEDMatrix systemd units an update changed, and restore them on rollback"
|
||||
echo " (/usr/local/sbin/ledmatrix-refresh-units, installed by install_service.sh)"
|
||||
echo ""
|
||||
|
||||
# Ask for confirmation
|
||||
|
||||
@@ -143,6 +143,30 @@ for VERIFY_UNIT in ledmatrix-update-verify.service ledmatrix-update-verify.path;
|
||||
fi
|
||||
done
|
||||
|
||||
# The helper updates run (through sudo, see lib_sudoers.sh) to install these
|
||||
# same units when a new version changes their templates, and to put the old
|
||||
# ones back if the automatic update rolls back. Root-owned and outside the
|
||||
# checkout, so the web user who owns the checkout cannot change what sudo runs.
|
||||
# Not fatal: without it, updates leave the units for the next reinstall.
|
||||
REFRESH_UNITS_SRC="$PROJECT_ROOT_DIR/scripts/install/ledmatrix_refresh_units.py"
|
||||
REFRESH_UNITS_DEST=/usr/local/sbin/ledmatrix-refresh-units
|
||||
if [ -f "$REFRESH_UNITS_SRC" ]; then
|
||||
if sudo install -D -o root -g root -m 0755 "$REFRESH_UNITS_SRC" "$REFRESH_UNITS_DEST"; then
|
||||
echo "Installed $REFRESH_UNITS_DEST (lets updates refresh these units)"
|
||||
else
|
||||
echo "WARNING: could not install $REFRESH_UNITS_DEST; updates will not refresh the systemd units" >&2
|
||||
fi
|
||||
fi
|
||||
# The units above are copied from mktemp files, which are 0600. 0644 is what
|
||||
# first_time_install.sh (Step 8.1) sets, and lets the web interface compare
|
||||
# them with the templates after an update without root.
|
||||
for INSTALLED_UNIT in ledmatrix.service ledmatrix-web.service \
|
||||
ledmatrix-update-verify.service ledmatrix-update-verify.path; do
|
||||
if [ -f "/etc/systemd/system/$INSTALLED_UNIT" ]; then
|
||||
sudo chmod 644 "/etc/systemd/system/$INSTALLED_UNIT" || true
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Reloading systemd daemon for web service..."
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
|
||||
Executable
+451
@@ -0,0 +1,451 @@
|
||||
#!/usr/bin/python3 -I
|
||||
"""Refresh the installed LEDMatrix systemd units from the checkout's templates.
|
||||
|
||||
Installed by scripts/install/install_service.sh as a root-owned copy,
|
||||
/usr/local/sbin/ledmatrix-refresh-units, and granted to the web interface's
|
||||
user by /etc/sudoers.d/ledmatrix_web (scripts/install/lib_sudoers.sh) with
|
||||
exactly two command lines:
|
||||
|
||||
ledmatrix-refresh-units (no arguments)
|
||||
ledmatrix-refresh-units --restore
|
||||
|
||||
An update (Update Code, or the weekly automatic update) pulls new unit
|
||||
templates into systemd/, but the units systemd runs are the copies in
|
||||
/etc/systemd/system, which only the installer used to write. So a setting
|
||||
added to a template -- the render-loop watchdog, a memory limit -- never
|
||||
reached a device that was already installed. After an update the web
|
||||
interface runs this, and the next restart picks the new units up.
|
||||
|
||||
* **No arguments:** render each installed unit from systemd/<unit> exactly as
|
||||
install_service.sh does (__PROJECT_ROOT_DIR__ and __USER__ replaced
|
||||
literally), and install the ones whose content differs (comments and blank
|
||||
lines aside, as src/startup_validator.py compares them), then
|
||||
``systemctl daemon-reload``. The units replaced are saved first, so the
|
||||
automatic update's rollback can put them back.
|
||||
* ``--restore``: put back the units the last refresh replaced, and
|
||||
daemon-reload. Nothing saved means nothing to do.
|
||||
* ``--check``: print the units that would change, one per line. Needs no
|
||||
root and changes nothing.
|
||||
|
||||
What it trusts, and why. It takes no other input: the project directory and
|
||||
the web interface's user come from the installed, root-owned
|
||||
ledmatrix.service and ledmatrix-web.service, not from the caller, and sudo
|
||||
strips the caller's environment (``-I`` ignores the PYTHON* variables too).
|
||||
It only replaces units that are already installed, only the four
|
||||
install_service.sh installs, and only with a rendering that keeps each unit's
|
||||
User= (root for the display, the web user for the others) and
|
||||
WorkingDirectory=. The templates are files the web user can edit -- but so is
|
||||
run.py, which ledmatrix.service already runs as root, so a template grants
|
||||
nothing that user did not have; the checks keep a damaged or hostile template
|
||||
from changing who a unit runs as, and keep this from reading anything but a
|
||||
regular file under the checkout's systemd/ folder.
|
||||
|
||||
Standard library only, and no imports from the checkout: the installed copy
|
||||
must not run code the web user can change.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import stat
|
||||
import subprocess # nosec B404 - fixed argv, no shell # nosemgrep
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
SYSTEMD_DIR = '/etc/systemd/system'
|
||||
#: Root-only: the units the last refresh replaced, for --restore.
|
||||
BACKUP_DIR = '/var/lib/ledmatrix/unit-backup'
|
||||
MANIFEST = 'manifest.json'
|
||||
INSTALLED_PATH = '/usr/local/sbin/ledmatrix-refresh-units'
|
||||
|
||||
DISPLAY_UNIT = 'ledmatrix.service'
|
||||
WEB_UNIT = 'ledmatrix-web.service'
|
||||
VERIFY_SERVICE = 'ledmatrix-update-verify.service'
|
||||
VERIFY_PATH = 'ledmatrix-update-verify.path'
|
||||
#: What install_service.sh installs, in its order. Nothing else is touched.
|
||||
UNITS = (DISPLAY_UNIT, WEB_UNIT, VERIFY_SERVICE, VERIFY_PATH)
|
||||
|
||||
MAX_TEMPLATE_BYTES = 64 * 1024
|
||||
_USER_RE = re.compile(r'^[a-z_][a-z0-9_-]{0,31}$')
|
||||
#: systemd expands % specifiers, and a quote, backslash or line break would
|
||||
#: be reinterpreted in a unit file (src/auto_update_setup.py refuses the same).
|
||||
#: (On Windows, where the tests also run, a backslash is the path separator.)
|
||||
_UNSAFE_PATH_CHARS = set('%"') | ({'\\'} if os.sep == '/' else set())
|
||||
|
||||
EXIT_OK = 0
|
||||
EXIT_FAILED = 1
|
||||
EXIT_USAGE = 2
|
||||
|
||||
|
||||
class RefreshError(Exception):
|
||||
"""Why the units were left alone, in words for the web interface's log."""
|
||||
|
||||
|
||||
class UnitsUnreadable(RefreshError):
|
||||
"""An installed unit is not readable by this (unprivileged) user.
|
||||
|
||||
install_service.sh used to leave units mode 0600 (first_time_install.sh's
|
||||
Step 8.1 makes them 0644), so ``--check`` as the web user cannot always
|
||||
tell; the root helper itself can.
|
||||
"""
|
||||
|
||||
|
||||
def directive_values(text, key):
|
||||
"""Every value of ``key=`` in a unit's text, in order (systemd allows spaces around ``=``)."""
|
||||
return [m.group(1).strip() for m in re.finditer(rf'^[ \t]*{key}[ \t]*=(.*)$', text or '', re.M)]
|
||||
|
||||
|
||||
def layout_problem(text, section, keys):
|
||||
"""What would make ``directive_values`` misread the unit as systemd reads it, or None.
|
||||
|
||||
A ``User=`` inside a backslash-continued line is part of the line before,
|
||||
and one under [Unit] is ignored, so either could pass a check that systemd
|
||||
then does not apply. Neither appears in the shipped templates.
|
||||
"""
|
||||
current = None
|
||||
for raw in (text or '').splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith(('#', ';')):
|
||||
continue
|
||||
if line.endswith('\\'):
|
||||
return 'continues a line with a backslash'
|
||||
if line.startswith('[') and line.endswith(']'):
|
||||
current = line[1:-1]
|
||||
continue
|
||||
key = line.split('=', 1)[0].strip()
|
||||
if key in keys and current != section:
|
||||
return f'sets {key}= outside [{section}]'
|
||||
return None
|
||||
|
||||
|
||||
def unit_body(text):
|
||||
"""A unit's meaningful lines in order: no comments, no blank lines.
|
||||
|
||||
The same comparison src/startup_validator.py uses for its drift warning,
|
||||
so what this refreshes is exactly what that warns about.
|
||||
"""
|
||||
lines = []
|
||||
for line in (text or '').splitlines():
|
||||
line = line.strip()
|
||||
if line and not line.startswith('#'):
|
||||
lines.append(line)
|
||||
return '\n'.join(lines)
|
||||
|
||||
|
||||
def render(template, project_root, user):
|
||||
"""install_service.sh's ``sed "s|__PROJECT_ROOT_DIR__|...|g; s|__USER__|...|g"``."""
|
||||
return template.replace('__PROJECT_ROOT_DIR__', project_root).replace('__USER__', user)
|
||||
|
||||
|
||||
def _read_regular(path, limit=MAX_TEMPLATE_BYTES, dir_fd=None):
|
||||
"""A regular file's text, never through a symlink, a FIFO or a device."""
|
||||
flags = os.O_RDONLY | getattr(os, 'O_NOFOLLOW', 0) | getattr(os, 'O_NONBLOCK', 0)
|
||||
kwargs = {'dir_fd': dir_fd} if dir_fd is not None else {}
|
||||
fd = os.open(path, flags, **kwargs)
|
||||
try:
|
||||
info = os.fstat(fd)
|
||||
if not stat.S_ISREG(info.st_mode):
|
||||
raise RefreshError(f'{path} is not a regular file')
|
||||
if info.st_size > limit:
|
||||
raise RefreshError(f'{path} is larger than {limit} bytes')
|
||||
data = b''
|
||||
while True:
|
||||
chunk = os.read(fd, limit + 1 - len(data))
|
||||
if not chunk:
|
||||
break
|
||||
data += chunk
|
||||
if len(data) > limit:
|
||||
raise RefreshError(f'{path} is larger than {limit} bytes')
|
||||
finally:
|
||||
os.close(fd)
|
||||
if b'\0' in data:
|
||||
raise RefreshError(f'{path} is not a text file')
|
||||
try:
|
||||
return data.decode('utf-8')
|
||||
except UnicodeDecodeError as e:
|
||||
raise RefreshError(f'{path} is not UTF-8') from e
|
||||
|
||||
|
||||
def _read_installed(systemd_dir, name):
|
||||
path = os.path.join(systemd_dir, name)
|
||||
try:
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
return f.read()
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except PermissionError as e:
|
||||
raise UnitsUnreadable(f'cannot read the installed {name}: {e}') from e
|
||||
except (OSError, UnicodeDecodeError) as e:
|
||||
raise RefreshError(f'cannot read the installed {name}: {e}') from e
|
||||
|
||||
|
||||
def _lookup_user(user):
|
||||
try:
|
||||
import pwd
|
||||
except ImportError: # not a POSIX host (the tests on Windows)
|
||||
return True
|
||||
try:
|
||||
pwd.getpwnam(user)
|
||||
return True
|
||||
except KeyError:
|
||||
return False
|
||||
|
||||
|
||||
class Refresher:
|
||||
def __init__(self, systemd_dir=SYSTEMD_DIR, backup_dir=BACKUP_DIR, run=subprocess.run,
|
||||
is_root=None, user_exists=_lookup_user, log=None):
|
||||
self.systemd_dir = systemd_dir
|
||||
self.backup_dir = backup_dir
|
||||
self.run = run
|
||||
self.is_root = is_root or (lambda: hasattr(os, 'geteuid') and os.geteuid() == 0)
|
||||
self.user_exists = user_exists
|
||||
self.log = log or (lambda msg: print(msg, flush=True))
|
||||
|
||||
# -- what the installed units say -------------------------------------
|
||||
|
||||
def context(self, installed):
|
||||
"""(project root, web user) from the installed, root-owned units."""
|
||||
display = installed.get(DISPLAY_UNIT)
|
||||
if display is None:
|
||||
raise RefreshError(f'{DISPLAY_UNIT} is not installed; run scripts/install/install_service.sh')
|
||||
roots = directive_values(display, 'WorkingDirectory')
|
||||
if len(roots) != 1:
|
||||
raise RefreshError(f'the installed {DISPLAY_UNIT} does not name one WorkingDirectory')
|
||||
root = roots[0]
|
||||
if (not os.path.isabs(root) or any(ch in _UNSAFE_PATH_CHARS or ord(ch) < 32 for ch in root)
|
||||
or os.path.normpath(root) != root):
|
||||
raise RefreshError(f'the installed {DISPLAY_UNIT} runs from {root!r}, which cannot be used')
|
||||
if not os.path.isdir(root):
|
||||
raise RefreshError(f'{root} (the installed {DISPLAY_UNIT} WorkingDirectory) does not exist')
|
||||
|
||||
user = None
|
||||
web = installed.get(WEB_UNIT)
|
||||
if web is not None:
|
||||
users = directive_values(web, 'User')
|
||||
user = users[0] if len(users) == 1 else ('root' if not users else None)
|
||||
if user is None or not _USER_RE.match(user) or not self.user_exists(user):
|
||||
raise RefreshError(f'the installed {WEB_UNIT} runs as an account that cannot be used')
|
||||
if directive_values(web, 'WorkingDirectory') != [root]:
|
||||
raise RefreshError(f'the installed {WEB_UNIT} and {DISPLAY_UNIT} run from different folders')
|
||||
return root, user
|
||||
|
||||
@staticmethod
|
||||
def expected_user(name, web_user):
|
||||
return 'root' if name == DISPLAY_UNIT else web_user
|
||||
|
||||
def _template(self, root, name):
|
||||
"""systemd/<name> under the checkout, as a regular file, never via a symlink."""
|
||||
dir_flags = os.O_RDONLY | getattr(os, 'O_DIRECTORY', 0) | getattr(os, 'O_NOFOLLOW', 0)
|
||||
if os.open in getattr(os, 'supports_dir_fd', set()):
|
||||
try:
|
||||
dfd = os.open(os.path.join(root, 'systemd'), dir_flags)
|
||||
except OSError as e:
|
||||
raise RefreshError(f'cannot open {root}/systemd: {e}') from e
|
||||
try:
|
||||
return _read_regular(name, dir_fd=dfd)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except OSError as e:
|
||||
raise RefreshError(f'cannot read systemd/{name}: {e}') from e
|
||||
finally:
|
||||
os.close(dfd)
|
||||
path = os.path.join(root, 'systemd', name)
|
||||
if os.path.islink(os.path.join(root, 'systemd')):
|
||||
raise RefreshError(f'{root}/systemd is a symlink')
|
||||
try:
|
||||
return _read_regular(path)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except OSError as e:
|
||||
raise RefreshError(f'cannot read systemd/{name}: {e}') from e
|
||||
|
||||
def _validate(self, name, rendered, root, user):
|
||||
problem = layout_problem(rendered, 'Service', ('User', 'WorkingDirectory'))
|
||||
if problem:
|
||||
raise RefreshError(f'systemd/{name} {problem}; refusing to install it')
|
||||
if directive_values(rendered, 'User') != [user]:
|
||||
raise RefreshError(f'systemd/{name} would not run as {user}; refusing to install it')
|
||||
if directive_values(rendered, 'WorkingDirectory') != [root]:
|
||||
raise RefreshError(f'systemd/{name} would not run from {root}; refusing to install it')
|
||||
|
||||
def plan(self):
|
||||
"""{unit: (installed text, new text)} for every installed unit that would change.
|
||||
|
||||
Raises RefreshError, and so changes nothing, if any unit cannot be
|
||||
rendered safely: four units refreshed as a set or not at all.
|
||||
"""
|
||||
installed = {name: _read_installed(self.systemd_dir, name) for name in UNITS}
|
||||
root, web_user = self.context(installed)
|
||||
changes = {}
|
||||
for name in UNITS:
|
||||
current = installed[name]
|
||||
if current is None:
|
||||
continue # never installed here: installing is the installer's job
|
||||
user = self.expected_user(name, web_user)
|
||||
if user is None:
|
||||
continue # the web unit is not installed, so neither is its user
|
||||
template = self._template(root, name)
|
||||
if template is None:
|
||||
continue # a version without this unit leaves the installed one alone
|
||||
rendered = render(template, root, user)
|
||||
# A path unit runs nothing itself; what matters is what it starts.
|
||||
if name.endswith('.service'):
|
||||
self._validate(name, rendered, root, user)
|
||||
else:
|
||||
self._validate_path(name, rendered)
|
||||
if unit_body(rendered) != unit_body(current):
|
||||
changes[name] = (current, rendered)
|
||||
return changes
|
||||
|
||||
def _validate_path(self, name, rendered):
|
||||
problem = layout_problem(rendered, 'Path', ('Unit',))
|
||||
if problem:
|
||||
raise RefreshError(f'systemd/{name} {problem}; refusing to install it')
|
||||
if directive_values(rendered, 'Unit') != [VERIFY_SERVICE]:
|
||||
raise RefreshError(f'systemd/{name} does not start {VERIFY_SERVICE}; refusing to install it')
|
||||
if directive_values(rendered, 'User'):
|
||||
raise RefreshError(f'systemd/{name} sets User=; refusing to install it')
|
||||
|
||||
# -- writing ------------------------------------------------------------
|
||||
|
||||
def _write_unit(self, name, text):
|
||||
fd, tmp = tempfile.mkstemp(dir=self.systemd_dir, prefix=f'.{name}.')
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8', newline='\n') as f:
|
||||
f.write(text)
|
||||
os.chmod(tmp, 0o644)
|
||||
os.replace(tmp, os.path.join(self.systemd_dir, name))
|
||||
except BaseException:
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
def _backup_dir(self):
|
||||
"""The backup folder, created root-only; refused if it is not a plain folder."""
|
||||
os.makedirs(os.path.dirname(self.backup_dir), mode=0o755, exist_ok=True)
|
||||
try:
|
||||
os.mkdir(self.backup_dir, 0o700)
|
||||
except FileExistsError:
|
||||
pass
|
||||
info = os.lstat(self.backup_dir)
|
||||
if not stat.S_ISDIR(info.st_mode):
|
||||
raise RefreshError(f'{self.backup_dir} is not a folder')
|
||||
if hasattr(os, 'geteuid') and info.st_uid != os.geteuid():
|
||||
raise RefreshError(f'{self.backup_dir} is not owned by root')
|
||||
return self.backup_dir
|
||||
|
||||
def _clear_backup(self, folder):
|
||||
for entry in os.listdir(folder):
|
||||
path = os.path.join(folder, entry)
|
||||
if os.path.isfile(path) or os.path.islink(path):
|
||||
os.unlink(path)
|
||||
|
||||
def _systemctl(self, *args):
|
||||
result = self.run(['systemctl', *args], capture_output=True, text=True, timeout=60)
|
||||
if result.returncode != 0:
|
||||
raise RefreshError(f'"systemctl {" ".join(args)}" failed: '
|
||||
f'{(result.stderr or result.stdout or "").strip()}')
|
||||
|
||||
def _restart_path_unit_if_active(self, names):
|
||||
"""A rewritten path unit watches the old path until it is restarted."""
|
||||
if VERIFY_PATH not in names:
|
||||
return
|
||||
state = self.run(['systemctl', 'is-active', VERIFY_PATH], capture_output=True, text=True, timeout=30)
|
||||
if (state.stdout or '').strip() == 'active':
|
||||
self._systemctl('restart', VERIFY_PATH)
|
||||
|
||||
def refresh(self):
|
||||
if not self.is_root():
|
||||
raise RefreshError('must run as root (sudo)')
|
||||
changes = self.plan()
|
||||
folder = self._backup_dir()
|
||||
# Always reset: the backup belongs to this refresh, so a --restore
|
||||
# after an update that changed nothing restores nothing.
|
||||
self._clear_backup(folder)
|
||||
if not changes:
|
||||
self.log('units: up to date')
|
||||
return []
|
||||
for name, (current, _) in changes.items():
|
||||
with open(os.path.join(folder, name), 'w', encoding='utf-8', newline='\n') as f:
|
||||
f.write(current)
|
||||
with open(os.path.join(folder, MANIFEST), 'w', encoding='utf-8') as f:
|
||||
json.dump({'units': sorted(changes)}, f)
|
||||
try:
|
||||
for name, (_, rendered) in changes.items():
|
||||
self._write_unit(name, rendered)
|
||||
self._systemctl('daemon-reload')
|
||||
except BaseException:
|
||||
# A failed refresh is reported as a failure, so the update records
|
||||
# no units_refreshed and a rollback would not --restore. Put the
|
||||
# replaced units back now, rather than leave a half-written set
|
||||
# under the old code.
|
||||
self._undo(changes, folder)
|
||||
raise
|
||||
self._restart_path_unit_if_active(changes)
|
||||
self.log('units refreshed: ' + ' '.join(sorted(changes)))
|
||||
return sorted(changes)
|
||||
|
||||
def _undo(self, changes, folder):
|
||||
"""Best effort: reinstall the units a failed refresh replaced."""
|
||||
undone = True
|
||||
for name, (current, _) in changes.items():
|
||||
try:
|
||||
self._write_unit(name, current)
|
||||
except OSError as e:
|
||||
undone = False
|
||||
self.log(f'units: could not put back {name}: {e}')
|
||||
try:
|
||||
self.run(['systemctl', 'daemon-reload'], capture_output=True, text=True, timeout=60)
|
||||
except (OSError, subprocess.SubprocessError) as e:
|
||||
self.log(f'units: daemon-reload after putting units back failed: {e}')
|
||||
if undone:
|
||||
# Nothing is left to restore; keep the backup only if a unit could
|
||||
# not be put back, so a manual --restore still can.
|
||||
self._clear_backup(folder)
|
||||
|
||||
def restore(self):
|
||||
if not self.is_root():
|
||||
raise RefreshError('must run as root (sudo)')
|
||||
folder = self._backup_dir()
|
||||
try:
|
||||
manifest = json.loads(_read_regular(os.path.join(folder, MANIFEST)))
|
||||
except FileNotFoundError:
|
||||
self.log('units: nothing to restore')
|
||||
return []
|
||||
names = [n for n in (manifest or {}).get('units', []) if n in UNITS]
|
||||
for name in names:
|
||||
self._write_unit(name, _read_regular(os.path.join(folder, name)))
|
||||
self._systemctl('daemon-reload')
|
||||
self._restart_path_unit_if_active(names)
|
||||
self._clear_backup(folder)
|
||||
self.log('units restored: ' + ' '.join(names))
|
||||
return names
|
||||
|
||||
|
||||
def main(argv, refresher=None):
|
||||
args = argv[1:]
|
||||
if args not in ([], ['--restore'], ['--check']):
|
||||
print('usage: ledmatrix-refresh-units [--restore | --check]', file=sys.stderr)
|
||||
return EXIT_USAGE
|
||||
refresher = refresher or Refresher()
|
||||
try:
|
||||
if args == ['--check']:
|
||||
for name in sorted(refresher.plan()):
|
||||
print(name)
|
||||
elif args == ['--restore']:
|
||||
refresher.restore()
|
||||
else:
|
||||
refresher.refresh()
|
||||
except (RefreshError, OSError, subprocess.SubprocessError, ValueError) as e:
|
||||
print(f'ledmatrix-refresh-units: {e}', file=sys.stderr)
|
||||
return EXIT_FAILED
|
||||
return EXIT_OK
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
# Only as the installed program: sudo already sets a secure PATH, and
|
||||
# this pins the one systemctl comes from. (Not in main(), which the
|
||||
# tests call in-process.)
|
||||
os.environ['PATH'] = '/usr/sbin:/usr/bin:/sbin:/bin'
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -10,6 +10,11 @@
|
||||
#
|
||||
# Add or remove a grant here and nowhere else.
|
||||
|
||||
# Root-owned copy of scripts/install/ledmatrix_refresh_units.py, installed by
|
||||
# install_service.sh. Outside the checkout on purpose: the web user owns the
|
||||
# checkout, so a granted file inside it could be rewritten and run as root.
|
||||
LEDMATRIX_REFRESH_UNITS_PATH=/usr/local/sbin/ledmatrix-refresh-units
|
||||
|
||||
# web_sudoers_rules WEB_USER PROJECT_ROOT SYSTEMCTL_PATH BASH_PATH REBOOT_PATH POWEROFF_PATH JOURNALCTL_PATH
|
||||
#
|
||||
# Print the ledmatrix_web sudoers rules to stdout.
|
||||
@@ -58,6 +63,10 @@ $WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_pl
|
||||
# Install a requirements.txt as root via vetted helper, so packages are visible
|
||||
# to root-run ledmatrix.service (not just the web interface's own user).
|
||||
$WEB_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh *
|
||||
# After an update, install the new systemd units (no arguments: "" allows none)
|
||||
# and, on the automatic update's rollback, put the previous ones back.
|
||||
$WEB_USER ALL=(ALL) NOPASSWD: $LEDMATRIX_REFRESH_UNITS_PATH ""
|
||||
$WEB_USER ALL=(ALL) NOPASSWD: $LEDMATRIX_REFRESH_UNITS_PATH --restore
|
||||
EOF
|
||||
if [ -n "$JOURNALCTL_PATH" ]; then
|
||||
cat << EOF
|
||||
|
||||
@@ -3,6 +3,10 @@
|
||||
# LED Matrix One-Shot Installation Script
|
||||
# This script provides a single-command installation experience
|
||||
# Usage: curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | bash
|
||||
#
|
||||
# A new install runs the newest release (the stable update channel). For the
|
||||
# newest code from main instead (the beta channel), set LEDMATRIX_CHANNEL=beta:
|
||||
# curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/install/one-shot-install.sh | LEDMATRIX_CHANNEL=beta bash
|
||||
|
||||
set -Eeuo pipefail
|
||||
|
||||
@@ -205,6 +209,114 @@ check_sudo() {
|
||||
print_success "Sudo access confirmed"
|
||||
}
|
||||
|
||||
# --- release checkout helpers ------------------------------------------------
|
||||
# Which version an install runs. The rules are web_interface/update_channel.py's,
|
||||
# so the installer and the web interface's updates agree:
|
||||
# stable (default) the newest vX.Y.Z tag by semantic version; pre-releases
|
||||
# (v3.8.0-rc1), leading zeros and other tags are ignored
|
||||
# beta main, the newest code
|
||||
# Never backwards: an existing checkout moves to a release only when that
|
||||
# release contains its current commit (git merge-base --is-ancestor).
|
||||
# Never fatal: whatever goes wrong, the install carries on with the checkout
|
||||
# as it is.
|
||||
|
||||
# Print "stable" or "beta": LEDMATRIX_CHANNEL when it is set, else the
|
||||
# existing install's auto_update.channel (CONFIG_FILE), else stable.
|
||||
_lm_channel() {
|
||||
local config_file="${1:-}" value
|
||||
value=$(printf '%s' "${LEDMATRIX_CHANNEL:-}" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]')
|
||||
case "$value" in
|
||||
stable|beta) printf '%s\n' "$value"; return 0 ;;
|
||||
"") ;;
|
||||
*) print_warning "LEDMATRIX_CHANNEL=${LEDMATRIX_CHANNEL} is not stable or beta; using stable" >&2
|
||||
printf 'stable\n'; return 0 ;;
|
||||
esac
|
||||
if [ -n "$config_file" ] && [ -f "$config_file" ] && command -v python3 >/dev/null 2>&1; then
|
||||
value=$(python3 - "$config_file" 2>/dev/null <<'PY' || true
|
||||
import json, sys
|
||||
try:
|
||||
with open(sys.argv[1], encoding="utf-8") as f:
|
||||
section = json.load(f).get("auto_update")
|
||||
value = section.get("channel") if isinstance(section, dict) else None
|
||||
print(value.strip().lower() if isinstance(value, str) else "")
|
||||
except Exception:
|
||||
print("")
|
||||
PY
|
||||
)
|
||||
if [ "$value" = "beta" ]; then
|
||||
printf 'beta\n'
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
printf 'stable\n'
|
||||
}
|
||||
|
||||
# Print the newest release tag of the repository in the current directory,
|
||||
# or nothing when it has none.
|
||||
_lm_newest_release_tag() {
|
||||
git tag --list 'v*' 2>/dev/null \
|
||||
| grep -E '^v(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)$' \
|
||||
| sort -t. -k1.2,1n -k2,2n -k3,3n \
|
||||
| tail -n 1 || true
|
||||
}
|
||||
|
||||
# A fresh clone (on main): move to the newest release unless beta was asked for.
|
||||
_lm_checkout_release_after_clone() {
|
||||
local channel tag
|
||||
channel=$(_lm_channel "")
|
||||
if [ "$channel" = "beta" ]; then
|
||||
print_success "Beta channel: installing the newest code from main"
|
||||
return 0
|
||||
fi
|
||||
tag=$(_lm_newest_release_tag)
|
||||
if [ -z "$tag" ]; then
|
||||
print_warning "No release found; installing the newest code from main"
|
||||
return 0
|
||||
fi
|
||||
if git -c advice.detachedHead=false checkout --quiet --detach "${tag}^{commit}"; then
|
||||
print_success "Installing release $tag (stable channel)"
|
||||
else
|
||||
print_warning "Could not check out release $tag; installing the newest code from main"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# An existing checkout: move it forward along its channel, never backwards.
|
||||
# Returns 1 when it should be updated the way it always was (a fast-forward
|
||||
# pull of its branch): beta, or stable on a branch newer than every release.
|
||||
_lm_update_existing_checkout() {
|
||||
local channel tag head tag_sha
|
||||
channel=$(_lm_channel "config/config.json")
|
||||
if [ "$channel" = "beta" ]; then
|
||||
return 1
|
||||
fi
|
||||
if ! git fetch --quiet --tags --force origin >/dev/null 2>&1; then
|
||||
print_warning "Could not fetch release tags; keeping the current version"
|
||||
return 0
|
||||
fi
|
||||
tag=$(_lm_newest_release_tag)
|
||||
head=$(git rev-parse --verify --quiet HEAD 2>/dev/null || true)
|
||||
if [ -n "$tag" ] && [ -n "$head" ] && git merge-base --is-ancestor "$head" "$tag" 2>/dev/null; then
|
||||
tag_sha=$(git rev-parse --verify --quiet "${tag}^{commit}" 2>/dev/null || true)
|
||||
if [ "$head" = "$tag_sha" ]; then
|
||||
print_success "Already on the newest release, $tag"
|
||||
elif git -c advice.detachedHead=false checkout --quiet --detach "${tag}^{commit}"; then
|
||||
print_success "Updated to release $tag (stable channel)"
|
||||
else
|
||||
print_warning "Could not move to release $tag (local changes?); keeping the current version"
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
if git symbolic-ref --quiet HEAD >/dev/null 2>&1; then
|
||||
# Newer than the newest release (or no release yet): follow the branch
|
||||
# until a release includes this version, as updates do.
|
||||
return 1
|
||||
fi
|
||||
print_success "This checkout is newer than the newest release${tag:+ ($tag)}; leaving it as it is"
|
||||
return 0
|
||||
}
|
||||
# --- end release checkout helpers --------------------------------------------
|
||||
|
||||
# Main installation function
|
||||
main() {
|
||||
print_step "LED Matrix One-Shot Installation"
|
||||
@@ -292,7 +404,10 @@ main() {
|
||||
|
||||
# Try to safely update current branch first (fast-forward only to avoid unintended merges)
|
||||
PULL_SUCCESS=false
|
||||
if git pull --ff-only origin "$CURRENT_BRANCH" >/dev/null 2>&1; then
|
||||
# Stable: the newest release, if it contains this version.
|
||||
if _lm_update_existing_checkout; then
|
||||
PULL_SUCCESS=true
|
||||
elif git pull --ff-only origin "$CURRENT_BRANCH" >/dev/null 2>&1; then
|
||||
print_success "Repository updated successfully (branch: $CURRENT_BRANCH)"
|
||||
PULL_SUCCESS=true
|
||||
else
|
||||
@@ -323,10 +438,12 @@ main() {
|
||||
rm -rf "$REPO_DIR"
|
||||
print_success "Cloning repository..."
|
||||
retry git clone "$REPO_URL" "$REPO_DIR"
|
||||
(cd "$REPO_DIR" && _lm_checkout_release_after_clone) || print_warning "Could not choose a release; installing the newest code from main"
|
||||
fi
|
||||
else
|
||||
print_success "Cloning repository to $REPO_DIR..."
|
||||
retry git clone "$REPO_URL" "$REPO_DIR"
|
||||
(cd "$REPO_DIR" && _lm_checkout_release_after_clone) || print_warning "Could not choose a release; installing the newest code from main"
|
||||
fi
|
||||
|
||||
# Verify repository is accessible
|
||||
@@ -397,6 +514,7 @@ main() {
|
||||
sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 \
|
||||
LEDMATRIX_APT_UPDATED="${LEDMATRIX_APT_UPDATED:-0}" \
|
||||
LEDMATRIX_AUTO_UPDATE="${LEDMATRIX_AUTO_UPDATE:-}" \
|
||||
LEDMATRIX_CHANNEL="${LEDMATRIX_CHANNEL:-}" \
|
||||
bash ./first_time_install.sh -y </dev/null
|
||||
fi
|
||||
INSTALL_EXIT_CODE=$?
|
||||
|
||||
@@ -15,7 +15,8 @@ The updater leaves data/auto_update_pending.json:
|
||||
|
||||
{"status": "pending", "old_head": ..., "new_head": ...,
|
||||
"old_ref": "main" | "" (detached) | absent (older updaters),
|
||||
"display_was_active": bool, "dependency_failures": [...], "created_at": ...}
|
||||
"display_was_active": bool, "dependency_failures": [...],
|
||||
"units_refreshed": bool (absent from older updaters), "created_at": ...}
|
||||
|
||||
This moves its status to "verifying" and then to one of "success",
|
||||
"rolled_back" or "rollback_failed", with "reason" and "detail" saying why.
|
||||
@@ -74,6 +75,10 @@ BASH_CANDIDATES = ('/usr/bin/bash', '/bin/bash')
|
||||
#: ...and, like it, moves to the next one only when sudo refused the command
|
||||
#: line (permission_utils.SUDO_REFUSAL_PHRASES), never after pip itself ran.
|
||||
SUDO_REFUSAL_PHRASES = ('a password is required', 'is not allowed to run', 'no tty present')
|
||||
#: The root-owned helper that installed the update's systemd units
|
||||
#: (web_interface/unit_refresh.py); ``--restore`` puts the previous ones back.
|
||||
REFRESH_UNITS_PATH = '/usr/local/sbin/ledmatrix-refresh-units'
|
||||
UNIT_RESTORE_TIMEOUT_SECONDS = 90
|
||||
|
||||
#: The longest one health check can take: restart and wait, roll back
|
||||
#: (diff, reset, reinstalls), restart and wait again. A wait's last poll can
|
||||
@@ -81,7 +86,8 @@ SUDO_REFUSAL_PHRASES = ('a password is required', 'is not allowed to run', 'no t
|
||||
_WAIT_WORST_SECONDS = (HEALTH_TIMEOUT_SECONDS + STABLE_SECONDS + WEB_CHECK_TIMEOUT_SECONDS
|
||||
+ 2 * SYSTEMCTL_QUERY_TIMEOUT_SECONDS + POLL_SECONDS)
|
||||
WORST_CASE_SECONDS = (2 * (2 * RESTART_TIMEOUT_SECONDS + _WAIT_WORST_SECONDS)
|
||||
+ GIT_TIMEOUT_SECONDS + GIT_RESET_TIMEOUT_SECONDS + PIP_BUDGET_SECONDS)
|
||||
+ GIT_TIMEOUT_SECONDS + GIT_RESET_TIMEOUT_SECONDS + UNIT_RESTORE_TIMEOUT_SECONDS
|
||||
+ PIP_BUDGET_SECONDS)
|
||||
|
||||
#: What a command that could not run at all reports: its callers only read
|
||||
#: these three fields, the same ones a completed subprocess has.
|
||||
@@ -300,12 +306,26 @@ class Verifier:
|
||||
if result.returncode != 0:
|
||||
return False, (f'"git reset --hard {old}" failed: '
|
||||
f'{(result.stderr or result.stdout or "").strip()}')
|
||||
notes = []
|
||||
# The update also installed its own systemd units: put the previous
|
||||
# ones back before anything restarts onto the rolled-back code.
|
||||
if pending.get('units_refreshed') and not self.restore_units():
|
||||
notes.append('restoring the previous service settings failed; run '
|
||||
'"sudo ./scripts/install/install_service.sh" in the LEDMatrix folder')
|
||||
deadline = self.clock() + PIP_BUDGET_SECONDS
|
||||
failed = [rel for rel in requirements if not self.install_requirements(rel, deadline)]
|
||||
if failed:
|
||||
return True, ('reinstalling the previous dependencies from ' + ', '.join(failed)
|
||||
notes.append('reinstalling the previous dependencies from ' + ', '.join(failed)
|
||||
+ ' failed; run Install Base Requirements from the Tools tab')
|
||||
return True, ''
|
||||
return True, '; '.join(notes)
|
||||
|
||||
def restore_units(self):
|
||||
"""Reinstall the systemd units the update replaced. True on success."""
|
||||
result = self._run(['sudo', '-n', REFRESH_UNITS_PATH, '--restore'],
|
||||
timeout=UNIT_RESTORE_TIMEOUT_SECONDS)
|
||||
if result.returncode != 0:
|
||||
self.log(f'restoring the previous systemd units failed: {(result.stderr or "").strip()}')
|
||||
return result.returncode == 0
|
||||
|
||||
# -- the check itself -------------------------------------------------
|
||||
|
||||
|
||||
@@ -95,11 +95,13 @@ class StartupValidator:
|
||||
def _validate_systemd_units(self) -> None:
|
||||
"""Warn when an installed unit has drifted from the repo's template.
|
||||
|
||||
Nothing re-applies these after the first install. `git pull` -- which is
|
||||
what the web UI's update button runs -- brings a new template into the
|
||||
checkout, but nothing copies it to /etc/systemd/system and nothing runs
|
||||
`systemctl daemon-reload`, so the unit that actually runs is whatever
|
||||
first_time_install.sh wrote on day one.
|
||||
Before updates refreshed units, nothing re-applied these after the
|
||||
first install: `git pull` brought a new template into the checkout,
|
||||
but nothing copied it to /etc/systemd/system, so the unit that
|
||||
actually ran was whatever first_time_install.sh wrote on day one.
|
||||
Updates now install changed units through the root helper
|
||||
ledmatrix-refresh-units (web_interface/unit_refresh.py) -- but only on
|
||||
a device whose installer granted it, so this still catches the rest.
|
||||
|
||||
That makes every hardening added to a unit inert on existing installs.
|
||||
Measured on one rig: the installed unit was thirteen days older than the
|
||||
@@ -141,10 +143,11 @@ class StartupValidator:
|
||||
|
||||
if self._unit_body(expected) != self._unit_body(actual):
|
||||
self.warnings.append(
|
||||
f"{installed.name} differs from {template_rel}; the "
|
||||
"installed unit is not refreshed by an update, so "
|
||||
f"{installed.name} differs from {template_rel}, so "
|
||||
"settings added to the template are not in effect. "
|
||||
"Re-run scripts/install/install_service.sh to apply them."
|
||||
"Updates apply them only once the installer has granted "
|
||||
"ledmatrix-refresh-units: re-run "
|
||||
"scripts/install/install_service.sh (or first_time_install.sh) to apply them."
|
||||
)
|
||||
except OSError as e:
|
||||
self.logger.debug("Could not compare systemd units: %s", e)
|
||||
|
||||
@@ -328,6 +328,21 @@ def _hermetic_control_socket(monkeypatch):
|
||||
monkeypatch.setenv(SOCKET_PATH_ENV, 'off')
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _hermetic_unit_refresh(monkeypatch, tmp_path_factory):
|
||||
"""Keep updates' systemd unit refresh off the host.
|
||||
|
||||
perform_core_update runs web_interface/unit_refresh.py after any update
|
||||
that moves HEAD, and several tests run the real one against a test clone.
|
||||
On a device -- or a machine where install_service.sh was tried out -- it
|
||||
would compare the clone's templates with the real /etc/systemd/system and
|
||||
run the real sudo helper. Point it at a folder that does not exist: no units
|
||||
installed, nothing to do. The unit refresh tests pass their own.
|
||||
"""
|
||||
from web_interface import unit_refresh
|
||||
monkeypatch.setattr(unit_refresh, 'SYSTEMD_DIR', str(tmp_path_factory.getbasetemp() / 'no-systemd'))
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def reset_logging():
|
||||
"""Reset logging configuration before each test."""
|
||||
|
||||
@@ -517,6 +517,32 @@ class TestUpdateIsVerified:
|
||||
h.updater.run()
|
||||
assert h.pending['dependency_failures'] == ['requirements.txt']
|
||||
|
||||
@pytest.mark.parametrize('unit_refresh, expected', [
|
||||
({'status': 'refreshed', 'message': '', 'units': ['ledmatrix.service']}, True),
|
||||
({'status': 'needs_reinstall', 'message': '', 'units': ['ledmatrix.service']}, False),
|
||||
(None, False),
|
||||
])
|
||||
def test_the_health_check_learns_whether_the_update_installed_units(self, tmp_path, unit_refresh,
|
||||
expected):
|
||||
"""Its rollback restores the previous units only when this update replaced them."""
|
||||
repo = Repo(tmp_path)
|
||||
repo.publish()
|
||||
h = Harness(tmp_path, repo, core_update=real_pull(repo.device, unit_refresh=unit_refresh))
|
||||
h.updater.run()
|
||||
assert h.pending['units_refreshed'] is expected
|
||||
|
||||
def test_a_health_check_that_never_starts_also_restores_the_units(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
repo.publish()
|
||||
refreshed = {'status': 'refreshed', 'message': '', 'units': ['ledmatrix.service']}
|
||||
h = Harness(tmp_path, repo, pickup=False,
|
||||
core_update=real_pull(repo.device, unit_refresh=refreshed))
|
||||
h.updater.run()
|
||||
assert repo.head() == old
|
||||
assert [a for a in h.sudo if a[-1] == '--restore'] == [
|
||||
['sudo', '-n', '/usr/local/sbin/ledmatrix-refresh-units', '--restore']]
|
||||
|
||||
def test_a_health_check_that_never_starts_means_the_update_is_undone(self, tmp_path):
|
||||
repo = Repo(tmp_path)
|
||||
old = repo.head()
|
||||
|
||||
@@ -80,6 +80,8 @@ class FakeHost:
|
||||
self.nrestarts = 0
|
||||
self.heartbeat = heartbeat
|
||||
self.display_started_at = -1000.0 # the pre-update display, long running
|
||||
self.unit_restores = [] # (argv, commit checked out, restarts so far)
|
||||
self.restore_ok = True
|
||||
|
||||
def broken(self, kind):
|
||||
if self.running_head is None:
|
||||
@@ -103,6 +105,10 @@ class FakeHost:
|
||||
if self.pip:
|
||||
return self.pip(args, self)
|
||||
return done(args, rc=0 if self.pip_ok else 1)
|
||||
if args[:3] == ['sudo', '-n', av.REFRESH_UNITS_PATH]:
|
||||
self.unit_restores.append((list(args), git(self.repo, 'rev-parse', 'HEAD'),
|
||||
len(self.restarts)))
|
||||
return done(args, rc=0 if self.restore_ok else 1)
|
||||
if args[:4] == ['sudo', '-n', 'systemctl', 'restart']:
|
||||
if self.restart_failures:
|
||||
self.restart_failures -= 1
|
||||
@@ -405,3 +411,45 @@ def test_the_heartbeat_location_and_freshness_match_the_display():
|
||||
# window it has to stay healthy for.
|
||||
assert av.HEARTBEAT_FRESH_SECONDS + av.POLL_SECONDS < av.STABLE_SECONDS
|
||||
assert av.HEARTBEAT_FRESH_SECONDS > display_watchdog.BEAT_INTERVAL_SECONDS * 2
|
||||
|
||||
|
||||
# -- systemd units the update installed ------------------------------------------
|
||||
|
||||
def test_a_rollback_restores_the_units_the_update_installed(tmp_path):
|
||||
"""The update installed new units (web_interface/unit_refresh.py); the
|
||||
rollback puts the old ones back before restarting onto the old code."""
|
||||
code, result, host, head, old, new = check(tmp_path, 'display_down', units_refreshed=True)
|
||||
assert result['status'] == 'rolled_back' and head == old
|
||||
assert len(host.unit_restores) == 1
|
||||
argv, commit, restarts_before = host.unit_restores[0]
|
||||
assert argv == ['sudo', '-n', av.REFRESH_UNITS_PATH, '--restore']
|
||||
assert commit == old, 'restored after the code was rolled back'
|
||||
assert restarts_before == 2, 'restored before the services restart onto the old code'
|
||||
assert host.restarts[-2:] == [('ledmatrix.service', old), ('ledmatrix-web.service', old)]
|
||||
assert result['detail'] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize('pending', [{}, {'units_refreshed': False}])
|
||||
def test_a_rollback_leaves_units_alone_when_the_update_did_not_change_them(tmp_path, pending):
|
||||
# {} is what an updater from before this change writes.
|
||||
code, result, host, head, old, new = check(tmp_path, 'display_down', **pending)
|
||||
assert result['status'] == 'rolled_back' and host.unit_restores == []
|
||||
|
||||
|
||||
def test_a_healthy_update_keeps_its_new_units(tmp_path):
|
||||
code, result, host, head, old, new = check(tmp_path, units_refreshed=True)
|
||||
assert result['status'] == 'success' and host.unit_restores == []
|
||||
|
||||
|
||||
def test_a_failed_unit_restore_is_reported_but_the_rollback_stands(tmp_path):
|
||||
repo, old, new = updated_repo(tmp_path)
|
||||
av.write_pending(av.pending_path(repo), {'status': 'pending', 'old_head': old, 'new_head': new,
|
||||
'display_was_active': True, 'dependency_failures': [],
|
||||
'units_refreshed': True})
|
||||
host = FakeHost(repo, new, 'display_down')
|
||||
host.restore_ok = False
|
||||
host.verifier().verify()
|
||||
result = av.read_pending(av.pending_path(repo))
|
||||
assert result['status'] == 'rolled_back'
|
||||
assert git(repo, 'rev-parse', 'HEAD') == old
|
||||
assert 'install_service.sh' in result['detail']
|
||||
|
||||
@@ -0,0 +1,370 @@
|
||||
"""New installs run the newest release; re-running the installer never moves backwards.
|
||||
|
||||
#684 made devices update along a channel -- stable follows the newest vX.Y.Z
|
||||
tag, beta follows main -- but a new install still cloned main's tip, so it ran
|
||||
unreleased code until the next release caught up with it. The one-shot
|
||||
installer (scripts/install/one-shot-install.sh, which is where the clone
|
||||
happens) now checks out the newest release after cloning, unless
|
||||
LEDMATRIX_CHANNEL=beta. Re-running it on an existing checkout moves a stable
|
||||
device forward to the newest release only when that release contains its
|
||||
commit, as update_channel.checkout_release() does, and leaves beta devices
|
||||
(and stable ones newer than every release) on the fast-forward pull they
|
||||
always had. first_time_install.sh writes an explicitly chosen channel
|
||||
(--beta / LEDMATRIX_CHANNEL) into config.json.
|
||||
|
||||
These run the installer's own bash, under its strict mode, against real git
|
||||
repositories.
|
||||
"""
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
ONE_SHOT = ROOT / "scripts" / "install" / "one-shot-install.sh"
|
||||
INSTALLER = ROOT / "first_time_install.sh"
|
||||
BEGIN = "# --- release checkout helpers"
|
||||
END = "# --- end release checkout helpers"
|
||||
|
||||
from web_interface import update_channel # noqa: E402
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
not sys.platform.startswith("linux"), reason="runs the installer's bash under Linux"
|
||||
)
|
||||
|
||||
|
||||
def helper_block() -> str:
|
||||
text = ONE_SHOT.read_text(encoding="utf-8")
|
||||
assert text.count(BEGIN) == 1 and text.count(END) == 1, "helper block markers missing or duplicated"
|
||||
return text[text.index(BEGIN): text.index(END)]
|
||||
|
||||
|
||||
def git(*args, cwd, env):
|
||||
result = subprocess.run(["git", *args], cwd=cwd, env=env, capture_output=True, text=True)
|
||||
assert result.returncode == 0, f"git {' '.join(args)} failed: {result.stderr}"
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def git_env(tmp_path):
|
||||
config = tmp_path / "gitconfig"
|
||||
config.write_text(
|
||||
"[user]\n\tname = t\n\temail = t@t\n"
|
||||
"[protocol \"file\"]\n\tallow = always\n"
|
||||
"[init]\n\tdefaultBranch = main\n"
|
||||
"[advice]\n\tdetachedHead = false\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
return {
|
||||
"PATH": "/usr/bin:/bin:/usr/sbin:/sbin",
|
||||
"HOME": str(tmp_path),
|
||||
"GIT_CONFIG_GLOBAL": str(config),
|
||||
"GIT_CONFIG_NOSYSTEM": "1",
|
||||
}
|
||||
|
||||
|
||||
#: Tags and the commit (index into the history) each points at. The newest
|
||||
#: release is v3.10.0: 10 > 8 numerically, the rc and the zero-padded tag are
|
||||
#: not releases, and v3.12 and nightly are not vX.Y.Z at all.
|
||||
TAGS = {
|
||||
"v3.7.0": 0, "v3.8.0": 1, "v3.10.0": 2,
|
||||
"v3.11.0-rc1": 3, "v03.12.0": 3, "v3.12": 3, "nightly": 3,
|
||||
}
|
||||
NEWEST = "v3.10.0"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def origin(tmp_path, git_env):
|
||||
"""A stand-in for GitHub: five commits on main (the last newer than any release)."""
|
||||
seed = tmp_path / "seed"
|
||||
seed.mkdir()
|
||||
git("init", "-q", ".", cwd=seed, env=git_env)
|
||||
commits = []
|
||||
for i in range(5):
|
||||
(seed / "version.txt").write_text(str(i), encoding="utf-8")
|
||||
git("add", ".", cwd=seed, env=git_env)
|
||||
git("commit", "-qm", f"c{i}", cwd=seed, env=git_env)
|
||||
commits.append(git("rev-parse", "HEAD", cwd=seed, env=git_env))
|
||||
for tag, index in TAGS.items():
|
||||
git("tag", tag, commits[index], cwd=seed, env=git_env)
|
||||
bare = tmp_path / "origin.git"
|
||||
git("clone", "-q", "--bare", str(seed), str(bare), cwd=tmp_path, env=git_env)
|
||||
return bare, commits, seed
|
||||
|
||||
|
||||
def run_block(snippet, cwd, env, channel=None):
|
||||
env = dict(env)
|
||||
if channel is not None:
|
||||
env["LEDMATRIX_CHANNEL"] = channel
|
||||
script = (
|
||||
"set -Eeuo pipefail\n"
|
||||
"trap 'echo ERR_TRAP_FIRED >&2; exit 99' ERR\n"
|
||||
'print_success() { echo "OK: $*"; }\n'
|
||||
'print_warning() { echo "W: $*"; }\n'
|
||||
f"{helper_block()}\n"
|
||||
f"{snippet}\n"
|
||||
)
|
||||
result = subprocess.run(["bash", "-c", script], cwd=cwd, capture_output=True, text=True, env=env)
|
||||
assert "ERR_TRAP_FIRED" not in result.stderr, result.stdout + result.stderr
|
||||
return result
|
||||
|
||||
|
||||
def clone(origin, tmp_path, env, name="LEDMatrix"):
|
||||
bare, _, _ = origin
|
||||
target = tmp_path / name
|
||||
git("clone", "-q", str(bare), str(target), cwd=tmp_path, env=env)
|
||||
return target
|
||||
|
||||
|
||||
def head(repo, env):
|
||||
return git("rev-parse", "HEAD", cwd=repo, env=env)
|
||||
|
||||
|
||||
def branch(repo, env):
|
||||
result = subprocess.run(["git", "symbolic-ref", "--quiet", "--short", "HEAD"], cwd=repo, env=env,
|
||||
capture_output=True, text=True)
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def set_channel(repo, channel):
|
||||
(repo / "config").mkdir(exist_ok=True)
|
||||
(repo / "config" / "config.json").write_text(
|
||||
json.dumps({"auto_update": {"enabled": False, "channel": channel}}), encoding="utf-8")
|
||||
|
||||
|
||||
# -- a fresh install -------------------------------------------------------------
|
||||
|
||||
def test_a_fresh_clone_checks_out_the_newest_release(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = clone(origin, tmp_path, git_env)
|
||||
out = run_block("_lm_checkout_release_after_clone", repo, git_env)
|
||||
assert out.returncode == 0
|
||||
assert head(repo, git_env) == commits[TAGS[NEWEST]]
|
||||
assert branch(repo, git_env) == "", "a release is checked out detached, as Update Code does"
|
||||
assert f"Installing release {NEWEST}" in out.stdout
|
||||
|
||||
|
||||
@pytest.mark.parametrize("channel", ["beta", "BETA", " beta "])
|
||||
def test_a_fresh_beta_install_stays_on_main(origin, tmp_path, git_env, channel):
|
||||
_, commits, _ = origin
|
||||
repo = clone(origin, tmp_path, git_env)
|
||||
run_block("_lm_checkout_release_after_clone", repo, git_env, channel=channel)
|
||||
assert head(repo, git_env) == commits[-1] and branch(repo, git_env) == "main"
|
||||
|
||||
|
||||
def test_an_unknown_channel_falls_back_to_stable_and_says_so(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = clone(origin, tmp_path, git_env)
|
||||
out = run_block("_lm_checkout_release_after_clone", repo, git_env, channel="nightly")
|
||||
assert head(repo, git_env) == commits[TAGS[NEWEST]]
|
||||
assert "not stable or beta" in out.stdout + out.stderr
|
||||
|
||||
|
||||
def test_a_repository_without_releases_installs_main(tmp_path, git_env):
|
||||
seed = tmp_path / "seed"
|
||||
seed.mkdir()
|
||||
git("init", "-q", ".", cwd=seed, env=git_env)
|
||||
(seed / "f").write_text("x", encoding="utf-8")
|
||||
git("add", ".", cwd=seed, env=git_env)
|
||||
git("commit", "-qm", "only", cwd=seed, env=git_env)
|
||||
git("tag", "v3.0", cwd=seed, env=git_env) # not a release tag
|
||||
repo = tmp_path / "LEDMatrix"
|
||||
git("clone", "-q", str(seed), str(repo), cwd=tmp_path, env=git_env)
|
||||
out = run_block("_lm_checkout_release_after_clone", repo, git_env)
|
||||
assert branch(repo, git_env) == "main" and "No release found" in out.stdout
|
||||
|
||||
|
||||
# -- re-running on an existing checkout -----------------------------------------------
|
||||
|
||||
def existing(origin, tmp_path, env, at, detached):
|
||||
"""An installed checkout, at commit index ``at``, on main or detached."""
|
||||
_, commits, _ = origin
|
||||
repo = clone(origin, tmp_path, env)
|
||||
if detached:
|
||||
git("checkout", "-q", "--detach", commits[at], cwd=repo, env=env)
|
||||
else:
|
||||
git("reset", "-q", "--hard", commits[at], cwd=repo, env=env)
|
||||
return repo
|
||||
|
||||
|
||||
def update(repo, env, channel=None):
|
||||
out = run_block("if _lm_update_existing_checkout; then echo RESULT=handled; "
|
||||
"else echo RESULT=pull; fi", repo, env, channel=channel)
|
||||
return re.search(r"RESULT=(\w+)", out.stdout).group(1), out
|
||||
|
||||
|
||||
def test_a_device_on_an_older_release_moves_to_the_newest(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS["v3.7.0"], detached=True)
|
||||
result, out = update(repo, git_env)
|
||||
assert result == "handled"
|
||||
assert head(repo, git_env) == commits[TAGS[NEWEST]]
|
||||
assert f"Updated to release {NEWEST}" in out.stdout
|
||||
|
||||
|
||||
def test_a_device_already_on_the_newest_release_stays(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS[NEWEST], detached=True)
|
||||
result, out = update(repo, git_env)
|
||||
assert result == "handled" and head(repo, git_env) == commits[TAGS[NEWEST]]
|
||||
assert "Already on the newest release" in out.stdout
|
||||
|
||||
|
||||
def test_a_device_on_main_behind_the_newest_release_moves_to_it(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS["v3.8.0"], detached=False)
|
||||
result, _ = update(repo, git_env)
|
||||
assert result == "handled" and head(repo, git_env) == commits[TAGS[NEWEST]]
|
||||
|
||||
|
||||
def test_a_device_on_main_newer_than_every_release_is_not_moved_back(origin, tmp_path, git_env):
|
||||
"""It keeps the fast-forward pull it always had, and waits for a release to contain it."""
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=4, detached=False)
|
||||
result, _ = update(repo, git_env)
|
||||
assert result == "pull"
|
||||
assert head(repo, git_env) == commits[4] and branch(repo, git_env) == "main"
|
||||
|
||||
|
||||
def test_a_detached_device_newer_than_every_release_is_left_alone(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=3, detached=True)
|
||||
result, out = update(repo, git_env)
|
||||
assert result == "handled" and head(repo, git_env) == commits[3]
|
||||
assert "newer than the newest release" in out.stdout
|
||||
|
||||
|
||||
def test_a_higher_version_on_an_older_commit_is_not_a_downgrade(origin, tmp_path, git_env):
|
||||
"""Newest by version is not newest by history: never move to a tag that does not contain HEAD."""
|
||||
bare, commits, seed = origin
|
||||
git("tag", "v9.0.0", commits[0], cwd=seed, env=git_env)
|
||||
git("push", "-q", str(bare), "v9.0.0", cwd=seed, env=git_env)
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS[NEWEST], detached=True)
|
||||
result, _ = update(repo, git_env)
|
||||
assert result == "handled" and head(repo, git_env) == commits[TAGS[NEWEST]]
|
||||
|
||||
|
||||
def test_a_new_release_published_since_the_clone_is_fetched(origin, tmp_path, git_env):
|
||||
bare, commits, seed = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS[NEWEST], detached=True)
|
||||
git("tag", "v3.11.0", commits[4], cwd=seed, env=git_env)
|
||||
git("push", "-q", str(bare), "v3.11.0", cwd=seed, env=git_env)
|
||||
result, _ = update(repo, git_env)
|
||||
assert result == "handled" and head(repo, git_env) == commits[4]
|
||||
|
||||
|
||||
def test_a_beta_device_keeps_its_pull(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS["v3.8.0"], detached=False)
|
||||
set_channel(repo, "beta")
|
||||
result, _ = update(repo, git_env)
|
||||
assert result == "pull" and head(repo, git_env) == commits[TAGS["v3.8.0"]]
|
||||
|
||||
|
||||
def test_the_environment_overrides_the_configured_channel(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS["v3.8.0"], detached=False)
|
||||
set_channel(repo, "stable")
|
||||
result, _ = update(repo, git_env, channel="beta")
|
||||
assert result == "pull" and head(repo, git_env) == commits[TAGS["v3.8.0"]]
|
||||
|
||||
|
||||
def test_local_edits_that_block_the_move_keep_the_checkout(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS["v3.7.0"], detached=True)
|
||||
(repo / "version.txt").write_text("my edit", encoding="utf-8")
|
||||
result, out = update(repo, git_env)
|
||||
assert result == "handled" and head(repo, git_env) == commits[TAGS["v3.7.0"]]
|
||||
assert (repo / "version.txt").read_text(encoding="utf-8") == "my edit"
|
||||
assert "Could not move to release" in out.stdout
|
||||
|
||||
|
||||
def test_an_unreachable_origin_keeps_the_checkout(origin, tmp_path, git_env):
|
||||
_, commits, _ = origin
|
||||
repo = existing(origin, tmp_path, git_env, at=TAGS["v3.7.0"], detached=True)
|
||||
git("remote", "set-url", "origin", str(tmp_path / "gone.git"), cwd=repo, env=git_env)
|
||||
result, out = update(repo, git_env)
|
||||
assert result == "handled" and head(repo, git_env) == commits[TAGS["v3.7.0"]]
|
||||
assert "Could not fetch" in out.stdout
|
||||
|
||||
|
||||
# -- same rules as the web interface -------------------------------------------------
|
||||
|
||||
NAMES = ["v1.2.3", "v1.10.0", "v1.9.9", "v2.0.0-rc1", "v02.0.0", "v2.0", "v10.0.0", "v9.99.99",
|
||||
"release-11", "v10.0.0+build", "v0.0.0", "v10.0.1", "v1.2.03", "V11.0.0"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("subset", [NAMES, NAMES[:4], ["v2.0", "nightly"], NAMES[::-1][:6]])
|
||||
def test_the_newest_tag_matches_update_channel(tmp_path, git_env, subset):
|
||||
repo = tmp_path / "tags"
|
||||
repo.mkdir()
|
||||
git("init", "-q", ".", cwd=repo, env=git_env)
|
||||
git("commit", "-q", "--allow-empty", "-m", "x", cwd=repo, env=git_env)
|
||||
for name in subset:
|
||||
git("tag", name, cwd=repo, env=git_env)
|
||||
out = run_block("_lm_newest_release_tag", repo, git_env).stdout.strip()
|
||||
assert out == (update_channel.newest_release_tag(subset) or "")
|
||||
|
||||
|
||||
# -- wiring --------------------------------------------------------------------------
|
||||
|
||||
def test_every_clone_is_followed_by_the_release_checkout():
|
||||
text = ONE_SHOT.read_text(encoding="utf-8")
|
||||
clones = [m.start() for m in re.finditer(r'retry git clone "\$REPO_URL" "\$REPO_DIR"\n', text)]
|
||||
assert clones
|
||||
for pos in clones:
|
||||
following = text[pos:].splitlines()[1]
|
||||
assert '_lm_checkout_release_after_clone' in following, following
|
||||
|
||||
|
||||
def test_the_existing_checkout_is_handled_before_the_old_pull():
|
||||
text = ONE_SHOT.read_text(encoding="utf-8")
|
||||
assert re.search(r'if _lm_update_existing_checkout; then\n\s+PULL_SUCCESS=true\n'
|
||||
r'\s+elif git pull --ff-only origin "\$CURRENT_BRANCH"', text)
|
||||
|
||||
|
||||
def test_the_one_shot_passes_the_channel_to_the_installer():
|
||||
text = ONE_SHOT.read_text(encoding="utf-8")
|
||||
assert 'LEDMATRIX_CHANNEL="${LEDMATRIX_CHANNEL:-}"' in text
|
||||
|
||||
|
||||
# -- first_time_install.sh records the chosen channel ----------------------------------
|
||||
|
||||
CHANNEL_BEGIN = 'case "$UPDATE_CHANNEL" in'
|
||||
CHANNEL_END = 'set it from the General tab instead"\n fi\nfi\n'
|
||||
|
||||
|
||||
def channel_block():
|
||||
text = INSTALLER.read_text(encoding="utf-8")
|
||||
start = text.index(CHANNEL_BEGIN)
|
||||
return text[start: text.index(CHANNEL_END, start) + len(CHANNEL_END)]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("auto_update, channel, expected", [
|
||||
("", "beta", {"enabled": False, "channel": "beta"}),
|
||||
("", "stable", {"enabled": False, "channel": "stable"}),
|
||||
("1", "", {"enabled": True, "channel": "stable"}),
|
||||
("", "", {"enabled": False, "channel": "stable"}), # nothing asked: untouched
|
||||
("", "nightly", {"enabled": False, "channel": "stable"}), # nonsense: untouched
|
||||
])
|
||||
def test_the_installer_writes_only_an_explicit_channel(tmp_path, auto_update, channel, expected):
|
||||
(tmp_path / "config").mkdir()
|
||||
config = tmp_path / "config" / "config.json"
|
||||
config.write_text(json.dumps({"auto_update": {"enabled": False, "channel": "stable"}, "x": 1}))
|
||||
script = (f'set -Eeuo pipefail\nPROJECT_ROOT_DIR="{tmp_path}"\nAUTO_UPDATE="{auto_update}"\n'
|
||||
f'UPDATE_CHANNEL="{channel}"\n{channel_block()}')
|
||||
result = subprocess.run(["bash", "-c", script], capture_output=True, text=True)
|
||||
assert result.returncode == 0, result.stdout + result.stderr
|
||||
data = json.loads(config.read_text())
|
||||
assert data["auto_update"] == expected and data["x"] == 1
|
||||
|
||||
|
||||
def test_the_installer_accepts_beta_as_a_flag_and_from_the_environment():
|
||||
text = INSTALLER.read_text(encoding="utf-8")
|
||||
assert re.search(r"^\s*--beta\) UPDATE_CHANNEL=beta ;;", text, re.M)
|
||||
assert 'UPDATE_CHANNEL=$(printf \'%s\' "${LEDMATRIX_CHANNEL:-}"' in text
|
||||
assert "LEDMATRIX_CHANNEL=stable|beta" in text, "documented in --help"
|
||||
@@ -0,0 +1,525 @@
|
||||
"""Updates refresh the installed systemd units: the root helper and its callers.
|
||||
|
||||
An update moved the checkout, and with it systemd/*.service, but systemd runs
|
||||
the copies in /etc/systemd/system, which only the installer wrote. So unit
|
||||
settings added after a device was installed (#687's render-loop watchdog)
|
||||
never reached it. scripts/install/ledmatrix_refresh_units.py, installed
|
||||
root-owned as /usr/local/sbin/ledmatrix-refresh-units and granted to the web
|
||||
user by exact command line, now installs changed units after an update, and
|
||||
puts the previous ones back when the automatic update rolls back.
|
||||
|
||||
The helper runs as root on input the web user can edit (the templates), so
|
||||
most of these are about what it refuses.
|
||||
"""
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
'ledmatrix_refresh_units', ROOT / 'scripts' / 'install' / 'ledmatrix_refresh_units.py')
|
||||
ru = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(ru)
|
||||
|
||||
from web_interface import unit_refresh # noqa: E402
|
||||
|
||||
try:
|
||||
import pwd
|
||||
# The web side checks the account exists, so use one that does.
|
||||
WEB_USER = pwd.getpwuid(os.getuid()).pw_name
|
||||
except ImportError: # Windows
|
||||
WEB_USER = 'ledpi'
|
||||
|
||||
# An account a hostile template switches the web unit to. Root, unless the
|
||||
# tests themselves run as root: then root *is* the web user and switching to
|
||||
# it changes nothing, so use another account.
|
||||
OTHER_USER = 'root' if WEB_USER != 'root' else 'nobody'
|
||||
|
||||
|
||||
class Host:
|
||||
"""A project checkout, an /etc/systemd/system, and a fake systemctl."""
|
||||
|
||||
def __init__(self, tmp_path, web_user=WEB_USER):
|
||||
self.project = tmp_path / 'LEDMatrix'
|
||||
shutil.copytree(ROOT / 'systemd', self.project / 'systemd')
|
||||
self.systemd = tmp_path / 'etc-systemd-system'
|
||||
self.systemd.mkdir()
|
||||
self.backup = tmp_path / 'var-lib-ledmatrix' / 'unit-backup'
|
||||
self.web_user = web_user
|
||||
self.calls = []
|
||||
self.path_active = True
|
||||
self.root = True
|
||||
for name in ru.UNITS:
|
||||
self.install(name)
|
||||
|
||||
def template(self, name):
|
||||
return (self.project / 'systemd' / name).read_text(encoding='utf-8')
|
||||
|
||||
def set_template(self, name, text):
|
||||
(self.project / 'systemd' / name).write_text(text, encoding='utf-8', newline='\n')
|
||||
|
||||
def rendered(self, name, text=None):
|
||||
user = 'root' if name == ru.DISPLAY_UNIT else self.web_user
|
||||
return ru.render(text if text is not None else self.template(name), str(self.project), user)
|
||||
|
||||
def install(self, name, text=None):
|
||||
(self.systemd / name).write_text(self.rendered(name, text), encoding='utf-8', newline='\n')
|
||||
|
||||
def installed(self, name):
|
||||
path = self.systemd / name
|
||||
return path.read_text(encoding='utf-8') if path.exists() else None
|
||||
|
||||
def run(self, args, **kwargs):
|
||||
self.calls.append(list(args))
|
||||
if args[:2] == ['systemctl', 'is-active']:
|
||||
out = 'active\n' if self.path_active else 'inactive\n'
|
||||
return subprocess.CompletedProcess(args, 0, stdout=out, stderr='')
|
||||
return subprocess.CompletedProcess(args, 0, stdout='', stderr='')
|
||||
|
||||
def refresher(self, log=None):
|
||||
return ru.Refresher(systemd_dir=str(self.systemd), backup_dir=str(self.backup), run=self.run,
|
||||
is_root=lambda: self.root, user_exists=lambda user: True,
|
||||
log=log or (lambda m: None))
|
||||
|
||||
@property
|
||||
def reloads(self):
|
||||
return self.calls.count(['systemctl', 'daemon-reload'])
|
||||
|
||||
|
||||
def watchdog_added(text):
|
||||
"""The kind of change #687 made: a new directive in [Service]."""
|
||||
return text.replace('[Service]\n', '[Service]\nWatchdogSec=60\n', 1)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host(tmp_path):
|
||||
return Host(tmp_path)
|
||||
|
||||
|
||||
# -- refresh ------------------------------------------------------------------
|
||||
|
||||
def test_units_that_match_are_left_alone(host):
|
||||
assert host.refresher().refresh() == []
|
||||
assert host.reloads == 0
|
||||
|
||||
|
||||
def test_a_changed_template_is_installed_and_systemd_reloaded(host):
|
||||
old = host.installed(ru.DISPLAY_UNIT)
|
||||
host.set_template(ru.DISPLAY_UNIT, watchdog_added(host.template(ru.DISPLAY_UNIT)))
|
||||
|
||||
assert host.refresher().refresh() == [ru.DISPLAY_UNIT]
|
||||
assert 'WatchdogSec=60' in host.installed(ru.DISPLAY_UNIT)
|
||||
assert host.installed(ru.DISPLAY_UNIT) == host.rendered(ru.DISPLAY_UNIT)
|
||||
assert host.reloads == 1
|
||||
# Only the unit that changed is replaced, and the one it replaced is kept.
|
||||
assert (host.backup / ru.DISPLAY_UNIT).read_text(encoding='utf-8') == old
|
||||
assert json.loads((host.backup / ru.MANIFEST).read_text()) == {'units': [ru.DISPLAY_UNIT]}
|
||||
|
||||
|
||||
def test_the_web_unit_keeps_the_web_users_account(host):
|
||||
host.set_template(ru.WEB_UNIT, watchdog_added(host.template(ru.WEB_UNIT)))
|
||||
host.refresher().refresh()
|
||||
assert ru.directive_values(host.installed(ru.WEB_UNIT), 'User') == [WEB_USER]
|
||||
assert ru.directive_values(host.installed(ru.DISPLAY_UNIT), 'User') == ['root']
|
||||
|
||||
|
||||
def test_comment_only_changes_are_not_a_refresh(host):
|
||||
host.set_template(ru.DISPLAY_UNIT, '# a new comment\n\n' + host.template(ru.DISPLAY_UNIT))
|
||||
assert host.refresher().refresh() == []
|
||||
assert host.reloads == 0
|
||||
|
||||
|
||||
def test_a_unit_that_was_never_installed_is_not_installed(host):
|
||||
(host.systemd / ru.VERIFY_SERVICE).unlink()
|
||||
(host.systemd / ru.VERIFY_PATH).unlink()
|
||||
host.set_template(ru.VERIFY_SERVICE, watchdog_added(host.template(ru.VERIFY_SERVICE)))
|
||||
host.refresher().refresh()
|
||||
assert host.installed(ru.VERIFY_SERVICE) is None
|
||||
|
||||
|
||||
def test_a_changed_path_unit_is_restarted_so_it_watches_the_new_path(host):
|
||||
host.set_template(ru.VERIFY_PATH, host.template(ru.VERIFY_PATH).replace(
|
||||
'[Path]\n', '[Path]\nMakeDirectory=yes\n'))
|
||||
host.refresher().refresh()
|
||||
assert ['systemctl', 'restart', ru.VERIFY_PATH] in host.calls
|
||||
|
||||
|
||||
def test_it_must_run_as_root(host):
|
||||
host.root = False
|
||||
host.set_template(ru.DISPLAY_UNIT, watchdog_added(host.template(ru.DISPLAY_UNIT)))
|
||||
with pytest.raises(ru.RefreshError, match='root'):
|
||||
host.refresher().refresh()
|
||||
assert 'WatchdogSec=60' not in host.installed(ru.DISPLAY_UNIT)
|
||||
|
||||
|
||||
def _failing_reload(host):
|
||||
real = host.run
|
||||
def run(args, **kwargs):
|
||||
if args == ['systemctl', 'daemon-reload'] and host.reloads == 0:
|
||||
host.calls.append(list(args))
|
||||
return subprocess.CompletedProcess(args, 1, stdout='', stderr='boom')
|
||||
return real(args, **kwargs)
|
||||
return run
|
||||
|
||||
|
||||
def test_a_failed_daemon_reload_puts_the_old_units_back(host):
|
||||
# The web side reports this as a failure and records no units_refreshed,
|
||||
# so a rollback would not --restore: the helper must undo it itself.
|
||||
old = host.installed(ru.DISPLAY_UNIT)
|
||||
host.set_template(ru.DISPLAY_UNIT, watchdog_added(host.template(ru.DISPLAY_UNIT)))
|
||||
host.run = _failing_reload(host)
|
||||
|
||||
with pytest.raises(ru.RefreshError):
|
||||
host.refresher().refresh()
|
||||
assert host.installed(ru.DISPLAY_UNIT) == old
|
||||
assert host.reloads == 2 # the failed one, then one after putting it back
|
||||
assert not (host.backup / ru.MANIFEST).exists()
|
||||
|
||||
|
||||
def test_a_failed_write_puts_back_the_units_already_written(host, monkeypatch):
|
||||
old = {name: host.installed(name) for name in (ru.DISPLAY_UNIT, ru.WEB_UNIT)}
|
||||
for name in old:
|
||||
host.set_template(name, watchdog_added(host.template(name)))
|
||||
refresher = host.refresher()
|
||||
real_write = refresher._write_unit
|
||||
writes = []
|
||||
|
||||
def write(name, text):
|
||||
writes.append(name)
|
||||
if len(writes) == 2:
|
||||
raise OSError('disk full')
|
||||
real_write(name, text)
|
||||
monkeypatch.setattr(refresher, '_write_unit', write)
|
||||
|
||||
with pytest.raises(OSError):
|
||||
refresher.refresh()
|
||||
first = writes[0]
|
||||
assert host.installed(first) == old[first]
|
||||
assert all(host.installed(name) == old[name] for name in old)
|
||||
|
||||
# -- what it refuses ------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize('unit, edit', [
|
||||
# The web interface's unit switched to root by a template edit.
|
||||
(ru.WEB_UNIT, lambda t: t.replace('User=__USER__', f'User={OTHER_USER}')),
|
||||
# The display's unit switched to another account.
|
||||
(ru.DISPLAY_UNIT, lambda t: t.replace('User=root', 'User=nobody')),
|
||||
# A second User= line.
|
||||
(ru.VERIFY_SERVICE, lambda t: t.replace('[Service]\n', '[Service]\nUser=root\n', 1)),
|
||||
# Run from somewhere else.
|
||||
(ru.DISPLAY_UNIT, lambda t: t.replace('WorkingDirectory=__PROJECT_ROOT_DIR__', 'WorkingDirectory=/tmp')),
|
||||
# A second User= written with spaces, which systemd accepts (last one wins).
|
||||
# Placed in [Service] (before [Install]), where it is not a layout problem.
|
||||
(ru.WEB_UNIT, lambda t: t.replace('\n[Install]', 'User = root\n\n[Install]')),
|
||||
# The web user's User= moved to [Unit], where systemd ignores it (so root).
|
||||
(ru.WEB_UNIT, lambda t: t.replace('User=__USER__\n', '').replace('[Unit]\n', '[Unit]\nUser=__USER__\n')),
|
||||
# The User= line hidden inside a continued line, where systemd does not see it.
|
||||
(ru.WEB_UNIT, lambda t: t.replace('User=__USER__\n', '').replace(
|
||||
'Description=LED Matrix Web Interface Service\n',
|
||||
'Description=LED Matrix Web Interface Service \\\\\nUser=__USER__\n')),
|
||||
# A path unit that starts something else.
|
||||
(ru.VERIFY_PATH, lambda t: t.replace('Unit=ledmatrix-update-verify.service', 'Unit=ledmatrix.service')),
|
||||
])
|
||||
def test_a_template_that_changes_who_or_where_is_refused_and_nothing_changes(host, unit, edit):
|
||||
before = {name: host.installed(name) for name in ru.UNITS}
|
||||
# A legitimate change alongside, which must not go in either.
|
||||
host.set_template(ru.DISPLAY_UNIT, watchdog_added(host.template(ru.DISPLAY_UNIT)))
|
||||
host.set_template(unit, edit(host.template(unit)))
|
||||
with pytest.raises(ru.RefreshError):
|
||||
host.refresher().refresh()
|
||||
assert {name: host.installed(name) for name in ru.UNITS} == before
|
||||
assert host.reloads == 0
|
||||
|
||||
|
||||
def test_the_project_folder_comes_from_the_installed_unit_not_the_caller(host, tmp_path):
|
||||
# The installed display unit names the project; a WorkingDirectory that
|
||||
# is not an existing absolute folder is refused before any template is read.
|
||||
host.install(ru.DISPLAY_UNIT, host.template(ru.DISPLAY_UNIT).replace(
|
||||
'WorkingDirectory=__PROJECT_ROOT_DIR__', 'WorkingDirectory=relative/path'))
|
||||
with pytest.raises(ru.RefreshError, match='cannot be used'):
|
||||
host.refresher().plan()
|
||||
|
||||
|
||||
def test_without_the_display_unit_installed_nothing_is_done(host):
|
||||
(host.systemd / ru.DISPLAY_UNIT).unlink()
|
||||
with pytest.raises(ru.RefreshError, match='not installed'):
|
||||
host.refresher().plan()
|
||||
|
||||
|
||||
@pytest.mark.skipif(not hasattr(os, 'O_NOFOLLOW'), reason='POSIX only')
|
||||
def test_a_template_symlink_is_not_followed(host, tmp_path):
|
||||
secret = tmp_path / 'secret'
|
||||
secret.write_text(host.template(ru.DISPLAY_UNIT) + 'Environment=SECRET=1\n', encoding='utf-8')
|
||||
target = host.project / 'systemd' / ru.DISPLAY_UNIT
|
||||
target.unlink()
|
||||
target.symlink_to(secret)
|
||||
with pytest.raises(ru.RefreshError):
|
||||
host.refresher().plan()
|
||||
|
||||
|
||||
@pytest.mark.skipif(not hasattr(os, 'O_NOFOLLOW'), reason='POSIX only')
|
||||
def test_a_symlinked_systemd_folder_is_not_followed(host, tmp_path):
|
||||
elsewhere = tmp_path / 'elsewhere'
|
||||
shutil.move(str(host.project / 'systemd'), str(elsewhere))
|
||||
(host.project / 'systemd').symlink_to(elsewhere, target_is_directory=True)
|
||||
with pytest.raises(ru.RefreshError):
|
||||
host.refresher().plan()
|
||||
|
||||
|
||||
def test_an_oversized_template_is_refused(host):
|
||||
host.set_template(ru.DISPLAY_UNIT, host.template(ru.DISPLAY_UNIT) + '#' * (ru.MAX_TEMPLATE_BYTES + 1))
|
||||
with pytest.raises(ru.RefreshError, match='larger'):
|
||||
host.refresher().plan()
|
||||
|
||||
|
||||
def test_main_leaves_the_callers_environment_alone(host):
|
||||
"""main() runs in-process in these tests; pinning PATH belongs to the installed program."""
|
||||
before = os.environ.get('PATH')
|
||||
ru.main(['ledmatrix-refresh-units', '--check'], refresher=host.refresher())
|
||||
assert os.environ.get('PATH') == before
|
||||
|
||||
|
||||
@pytest.mark.parametrize('argv', [
|
||||
['--restore', 'x'], ['--refresh'], ['/etc/passwd'], ['--check', '--restore'], ['']])
|
||||
def test_any_other_command_line_is_refused(argv):
|
||||
class Boom:
|
||||
def __getattr__(self, name):
|
||||
raise AssertionError('must not run')
|
||||
assert ru.main(['ledmatrix-refresh-units', *argv], refresher=Boom()) == ru.EXIT_USAGE
|
||||
|
||||
|
||||
def test_main_reports_a_refusal_as_a_failure(host, capsys):
|
||||
host.set_template(ru.WEB_UNIT, host.template(ru.WEB_UNIT).replace('User=__USER__', f'User={OTHER_USER}'))
|
||||
assert ru.main(['ledmatrix-refresh-units'], refresher=host.refresher()) == ru.EXIT_FAILED
|
||||
assert 'refusing' in capsys.readouterr().err
|
||||
|
||||
|
||||
# -- restore ------------------------------------------------------------------
|
||||
|
||||
def test_restore_puts_back_exactly_what_the_refresh_replaced(host):
|
||||
# A hand-edited installed unit: the rollback must give back this file,
|
||||
# not a rendering of the old template.
|
||||
hand_edited = host.installed(ru.DISPLAY_UNIT) + '# edited by hand\n'
|
||||
(host.systemd / ru.DISPLAY_UNIT).write_text(hand_edited, encoding='utf-8', newline='\n')
|
||||
untouched = host.installed(ru.WEB_UNIT)
|
||||
host.set_template(ru.DISPLAY_UNIT, watchdog_added(host.template(ru.DISPLAY_UNIT)))
|
||||
host.refresher().refresh()
|
||||
|
||||
assert host.refresher().restore() == [ru.DISPLAY_UNIT]
|
||||
assert host.installed(ru.DISPLAY_UNIT) == hand_edited
|
||||
assert host.installed(ru.WEB_UNIT) == untouched
|
||||
assert host.reloads == 2
|
||||
assert not (host.backup / ru.MANIFEST).exists(), 'a second restore must not repeat it'
|
||||
assert host.refresher().restore() == []
|
||||
|
||||
|
||||
def test_restore_after_an_update_that_changed_no_units_restores_nothing(host):
|
||||
host.set_template(ru.DISPLAY_UNIT, watchdog_added(host.template(ru.DISPLAY_UNIT)))
|
||||
host.refresher().refresh() # an earlier update...
|
||||
newer = host.installed(ru.DISPLAY_UNIT)
|
||||
host.refresher().refresh() # ...then one that changed no units
|
||||
assert host.refresher().restore() == []
|
||||
assert host.installed(ru.DISPLAY_UNIT) == newer
|
||||
|
||||
|
||||
def test_restore_must_run_as_root(host):
|
||||
host.root = False
|
||||
with pytest.raises(ru.RefreshError, match='root'):
|
||||
host.refresher().restore()
|
||||
|
||||
|
||||
# -- the real templates and install_service.sh ----------------------------------
|
||||
|
||||
def test_every_shipped_template_passes_the_helpers_checks(tmp_path):
|
||||
host = Host(tmp_path)
|
||||
for name in ru.UNITS:
|
||||
host.set_template(name, watchdog_added(host.template(name)) if name.endswith('.service')
|
||||
else host.template(name))
|
||||
assert host.refresher().refresh() == sorted(n for n in ru.UNITS if n.endswith('.service'))
|
||||
|
||||
|
||||
@pytest.mark.skipif(not sys.platform.startswith('linux'), reason='runs sed as install_service.sh does')
|
||||
def test_rendering_matches_install_service_sh(tmp_path):
|
||||
"""Same text as the installer's sed, including characters sed treats specially."""
|
||||
lib = ROOT / 'scripts' / 'install' / 'lib_systemd_render.sh'
|
||||
for project in ('/home/pi/LEDMatrix', '/opt/led matrix&co'):
|
||||
for name in ru.UNITS:
|
||||
user = 'root' if name == ru.DISPLAY_UNIT else 'pi'
|
||||
script = (f'source "{lib}"; R=$(sed_escape_replacement "$1"); U=$(sed_escape_replacement "$2"); '
|
||||
f'sed "s|__PROJECT_ROOT_DIR__|$R|g; s|__USER__|$U|g" "$3"')
|
||||
out = subprocess.run(['bash', '-c', script, 'x', project, user, str(ROOT / 'systemd' / name)],
|
||||
capture_output=True, text=True, check=True).stdout
|
||||
template = (ROOT / 'systemd' / name).read_text(encoding='utf-8')
|
||||
assert ru.render(template, project, user) == out, name
|
||||
|
||||
|
||||
def test_install_service_installs_the_helper_root_owned_at_the_granted_path():
|
||||
text = (ROOT / 'scripts' / 'install' / 'install_service.sh').read_text(encoding='utf-8')
|
||||
assert 'scripts/install/ledmatrix_refresh_units.py' in text
|
||||
m = re.search(r'install -D -o root -g root -m 0755 "\$REFRESH_UNITS_SRC" "\$REFRESH_UNITS_DEST"', text)
|
||||
assert m, 'install_service.sh must install the helper root:root 0755'
|
||||
assert f'REFRESH_UNITS_DEST={ru.INSTALLED_PATH}' in text
|
||||
|
||||
|
||||
def test_every_caller_names_the_same_helper_path():
|
||||
lib = (ROOT / 'scripts' / 'install' / 'lib_sudoers.sh').read_text(encoding='utf-8')
|
||||
verifier = (ROOT / 'scripts' / 'utils' / 'auto_update_verify.py').read_text(encoding='utf-8')
|
||||
assert f'LEDMATRIX_REFRESH_UNITS_PATH={ru.INSTALLED_PATH}' in lib
|
||||
assert unit_refresh.HELPER_PATH == ru.INSTALLED_PATH
|
||||
assert f"REFRESH_UNITS_PATH = '{ru.INSTALLED_PATH}'" in verifier
|
||||
assert ru.INSTALLED_PATH.startswith('/usr/local/sbin/'), 'must live outside the user-owned checkout'
|
||||
|
||||
|
||||
def test_the_helper_imports_nothing_from_the_checkout():
|
||||
source = (ROOT / 'scripts' / 'install' / 'ledmatrix_refresh_units.py').read_text(encoding='utf-8')
|
||||
imports = re.findall(r'^\s*(?:from|import)\s+([\w.]+)', source, re.M)
|
||||
assert not [m for m in imports if m.split('.')[0] in ('src', 'web_interface', 'scripts')]
|
||||
assert source.startswith('#!/usr/bin/python3 -I\n'), 'isolated mode: no PYTHON* env, no user site'
|
||||
|
||||
|
||||
# -- the web interface's side (web_interface/unit_refresh.py) ---------------------
|
||||
|
||||
class Sudo:
|
||||
"""sudo: refuses (``rc``/``stderr``), or runs the real helper as root against ``host``."""
|
||||
|
||||
def __init__(self, host=None, rc=0, stderr=''):
|
||||
self.host, self.rc, self.stderr, self.calls = host, rc, stderr, []
|
||||
self.as_root = False
|
||||
|
||||
def __call__(self, args, **kwargs):
|
||||
self.calls.append(list(args))
|
||||
if self.rc or self.host is None:
|
||||
return subprocess.CompletedProcess(args, self.rc, stdout='', stderr=self.stderr)
|
||||
lines = []
|
||||
self.as_root = True
|
||||
try:
|
||||
rc = ru.main(['ledmatrix-refresh-units', *args[3:]], refresher=self.host.refresher(lines.append))
|
||||
finally:
|
||||
self.as_root = False
|
||||
return subprocess.CompletedProcess(args, rc, stdout='\n'.join(lines) + '\n', stderr='')
|
||||
|
||||
|
||||
def _web(host, tmp_path, sudo, helper_installed=True):
|
||||
helper = tmp_path / 'usr-local-sbin' / 'ledmatrix-refresh-units'
|
||||
if helper_installed:
|
||||
helper.parent.mkdir(exist_ok=True)
|
||||
helper.write_text('#!/bin/true\n')
|
||||
return unit_refresh.refresh_after_update(run=sudo, systemd_dir=str(host.systemd),
|
||||
helper_path=str(helper))
|
||||
|
||||
|
||||
def _stale(host):
|
||||
# The web side compares the installed units with the checkout's templates.
|
||||
host.set_template(ru.DISPLAY_UNIT, watchdog_added(host.template(ru.DISPLAY_UNIT)))
|
||||
|
||||
|
||||
def test_web_side_does_nothing_when_the_units_match(host, tmp_path):
|
||||
sudo = Sudo(host)
|
||||
result = _web(host, tmp_path, sudo)
|
||||
assert result['status'] == unit_refresh.CURRENT and sudo.calls == []
|
||||
assert result['message'] == ''
|
||||
|
||||
|
||||
def test_web_side_runs_the_helper_through_sudo_with_no_arguments(host, tmp_path):
|
||||
_stale(host)
|
||||
sudo = Sudo(host)
|
||||
result = _web(host, tmp_path, sudo)
|
||||
assert result['status'] == unit_refresh.REFRESHED
|
||||
assert result['units'] == [ru.DISPLAY_UNIT]
|
||||
assert len(sudo.calls) == 1 and sudo.calls[0][:2] == ['sudo', '-n'] and len(sudo.calls[0]) == 3
|
||||
assert 'ledmatrix.service' in result['message']
|
||||
assert 'WatchdogSec=60' in host.installed(ru.DISPLAY_UNIT)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def unreadable(monkeypatch, host):
|
||||
"""Installed units only root can read (install_service.sh used to leave them 0600)."""
|
||||
real = ru._read_installed
|
||||
sudo = Sudo(host)
|
||||
|
||||
def read(systemd_dir, name):
|
||||
if not sudo.as_root:
|
||||
raise ru.UnitsUnreadable(f'cannot read the installed {name}: Permission denied')
|
||||
return real(systemd_dir, name)
|
||||
monkeypatch.setattr(ru, '_read_installed', read)
|
||||
monkeypatch.setattr(unit_refresh, '_load_helper', lambda path=None: ru)
|
||||
return sudo
|
||||
|
||||
|
||||
def test_web_side_lets_the_helper_decide_when_it_cannot_read_the_units(host, tmp_path, unreadable):
|
||||
_stale(host)
|
||||
result = _web(host, tmp_path, unreadable)
|
||||
assert len(unreadable.calls) == 1
|
||||
assert result['status'] == unit_refresh.REFRESHED and result['units'] == [ru.DISPLAY_UNIT]
|
||||
|
||||
|
||||
def test_web_side_unreadable_and_already_current_is_current(host, tmp_path, unreadable):
|
||||
result = _web(host, tmp_path, unreadable)
|
||||
assert result['status'] == unit_refresh.CURRENT and result['message'] == ''
|
||||
|
||||
|
||||
def test_web_side_unreadable_without_the_rule_asks_for_a_reinstall(host, tmp_path, unreadable, caplog):
|
||||
unreadable.rc, unreadable.stderr = 1, 'sudo: a password is required'
|
||||
result = _web(host, tmp_path, unreadable)
|
||||
assert result['status'] == unit_refresh.NEEDS_REINSTALL
|
||||
assert 'could not be checked' in result['message']
|
||||
|
||||
|
||||
def test_web_side_trusts_the_helper_about_what_changed(host, tmp_path):
|
||||
"""An older installed helper that renders differently changed nothing: nothing to roll back."""
|
||||
_stale(host)
|
||||
|
||||
def older_helper(args, **kwargs):
|
||||
return subprocess.CompletedProcess(args, 0, stdout='units: up to date\n', stderr='')
|
||||
result = _web(host, tmp_path, older_helper)
|
||||
assert result['status'] == unit_refresh.CURRENT
|
||||
|
||||
|
||||
def test_web_side_without_the_helper_asks_for_a_reinstall(host, tmp_path, caplog):
|
||||
_stale(host)
|
||||
sudo = Sudo()
|
||||
result = _web(host, tmp_path, sudo, helper_installed=False)
|
||||
assert result['status'] == unit_refresh.NEEDS_REINSTALL and sudo.calls == []
|
||||
assert 'first_time_install.sh' in result['message']
|
||||
assert 'reinstall' in caplog.text
|
||||
|
||||
|
||||
@pytest.mark.parametrize('stderr', [
|
||||
'sudo: a password is required',
|
||||
'Sorry, user ledpi is not allowed to run \'/usr/local/sbin/ledmatrix-refresh-units\' as root on ledpi.',
|
||||
])
|
||||
def test_web_side_without_the_sudo_rule_asks_for_a_reinstall(host, tmp_path, stderr, caplog):
|
||||
_stale(host)
|
||||
result = _web(host, tmp_path, Sudo(rc=1, stderr=stderr))
|
||||
assert result['status'] == unit_refresh.NEEDS_REINSTALL
|
||||
assert 'configure_web_sudo.sh' in result['message']
|
||||
assert 'no sudo rule' in caplog.text
|
||||
|
||||
|
||||
def test_web_side_reports_a_helper_refusal_as_a_failure(host, tmp_path):
|
||||
_stale(host)
|
||||
result = _web(host, tmp_path, Sudo(rc=1, stderr='ledmatrix-refresh-units: systemd/x refusing'))
|
||||
assert result['status'] == unit_refresh.FAILED
|
||||
assert 'refusing' in result['message']
|
||||
|
||||
|
||||
def test_web_side_on_a_machine_without_the_units_does_nothing(tmp_path):
|
||||
sudo = Sudo()
|
||||
result = unit_refresh.refresh_after_update(run=sudo, systemd_dir=str(tmp_path))
|
||||
assert result['status'] == unit_refresh.SKIPPED and sudo.calls == []
|
||||
|
||||
|
||||
def test_web_side_never_raises_on_a_broken_template(host, tmp_path):
|
||||
host.set_template(ru.WEB_UNIT, host.template(ru.WEB_UNIT).replace('User=__USER__', f'User={OTHER_USER}'))
|
||||
sudo = Sudo()
|
||||
result = _web(host, tmp_path, sudo)
|
||||
assert result['status'] == unit_refresh.FAILED and sudo.calls == []
|
||||
@@ -82,3 +82,37 @@ class TestRestartIsRequestedWhenCodeChanged:
|
||||
data = _pull(client)
|
||||
assert data['status'] == 'error'
|
||||
assert data['restart_required'] is False
|
||||
|
||||
|
||||
class TestUnitsAreRefreshedWithTheCode:
|
||||
"""New code may bring new systemd unit settings; installing them is part of the update."""
|
||||
|
||||
@pytest.fixture
|
||||
def refresh(self, monkeypatch):
|
||||
from web_interface import unit_refresh
|
||||
calls = []
|
||||
|
||||
def fake():
|
||||
calls.append(True)
|
||||
return {'status': 'refreshed', 'message': 'Service settings updated (ledmatrix.service).',
|
||||
'units': ['ledmatrix.service']}
|
||||
monkeypatch.setattr(unit_refresh, 'refresh_after_update', fake)
|
||||
return calls
|
||||
|
||||
def test_an_update_that_moved_head_refreshes_the_units(self, client, refresh):
|
||||
with patch.object(mod.subprocess, 'run', _git(['aaa111', 'bbb222'])):
|
||||
data = _pull(client)
|
||||
assert refresh == [True]
|
||||
assert data['unit_refresh']['status'] == 'refreshed'
|
||||
assert 'Service settings updated' in data['message']
|
||||
|
||||
def test_nothing_new_touches_no_units(self, client, refresh):
|
||||
with patch.object(mod.subprocess, 'run',
|
||||
_git(['aaa111', 'aaa111'], pull_out='Already up to date.\n')):
|
||||
data = _pull(client)
|
||||
assert refresh == [] and data['unit_refresh'] is None
|
||||
|
||||
def test_a_failed_pull_touches_no_units(self, client, refresh):
|
||||
with patch.object(mod.subprocess, 'run', _git(['aaa111'], pull_rc=1)):
|
||||
data = _pull(client)
|
||||
assert refresh == [] and data['unit_refresh'] is None
|
||||
|
||||
@@ -54,6 +54,10 @@ EXPECTED_GRANTS = frozenset({
|
||||
("NOPASSWD:", "$SYSTEMCTL_PATH restart ledmatrix-web.service"),
|
||||
("NOPASSWD:", "$BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_plugin_rm.sh *"),
|
||||
("NOPASSWD:", "$BASH_PATH $PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh *"),
|
||||
# The unit refresh helper (scripts/install/ledmatrix_refresh_units.py),
|
||||
# with no arguments (`""`; RULE below drops the closing quote) or --restore.
|
||||
("NOPASSWD:", '$LEDMATRIX_REFRESH_UNITS_PATH "'),
|
||||
("NOPASSWD:", "$LEDMATRIX_REFRESH_UNITS_PATH --restore"),
|
||||
("NOPASSWD:NOEXEC:", "$JOURNALCTL_PATH -u ledmatrix.service *"),
|
||||
("NOPASSWD:NOEXEC:", "$JOURNALCTL_PATH -u ledmatrix *"),
|
||||
("NOPASSWD:NOEXEC:", "$JOURNALCTL_PATH -t ledmatrix *"),
|
||||
@@ -145,7 +149,8 @@ def test_installer_call_renders_the_expected_rules(installer, tmp_path):
|
||||
rendered.add((m.group(2), m.group(3)))
|
||||
subst = {"$SYSTEMCTL_PATH": "/x/systemctl", "$BASH_PATH": "/x/bash",
|
||||
"$REBOOT_PATH": "/x/reboot", "$POWEROFF_PATH": "/x/poweroff",
|
||||
"$JOURNALCTL_PATH": "/x/journalctl", "$PROJECT_ROOT": "/srv/led root"}
|
||||
"$JOURNALCTL_PATH": "/x/journalctl", "$PROJECT_ROOT": "/srv/led root",
|
||||
"$LEDMATRIX_REFRESH_UNITS_PATH": "/usr/local/sbin/ledmatrix-refresh-units"}
|
||||
expected = set()
|
||||
for tags, command in EXPECTED_GRANTS:
|
||||
for var, value in subst.items():
|
||||
|
||||
@@ -603,6 +603,9 @@ class AutoUpdater:
|
||||
'release': info.get('release') if new_head == info.get('upstream_head') else None,
|
||||
'display_was_active': display_was_active,
|
||||
'dependency_failures': list(core.get('dependency_failures') or []),
|
||||
# The update installed new systemd units: a rollback puts the
|
||||
# previous ones back (ledmatrix-refresh-units --restore).
|
||||
'units_refreshed': (core.get('unit_refresh') or {}).get('status') == 'refreshed',
|
||||
'created_at': self.clock(),
|
||||
}
|
||||
|
||||
|
||||
@@ -415,6 +415,7 @@ def _perform_core_update_locked(stash_local_changes=True):
|
||||
# date" is a success too, and prompting for a restart then would
|
||||
# train users to ignore the prompt.
|
||||
code_changed = False
|
||||
unit_result = None
|
||||
# Requirement files whose install failed. The automatic updater refuses
|
||||
# to restart onto code whose dependencies did not install.
|
||||
dependency_failures = []
|
||||
@@ -537,6 +538,14 @@ def _perform_core_update_locked(stash_local_changes=True):
|
||||
)
|
||||
except (OSError, RuntimeError) as purge_err:
|
||||
logger.warning("Post-update plugin purge failed: %s", purge_err)
|
||||
# The new code may come with new systemd unit settings (systemd/*).
|
||||
# Install them now, so the restart that follows runs under them; the
|
||||
# automatic update's rollback puts the old ones back.
|
||||
if code_changed:
|
||||
from web_interface import unit_refresh
|
||||
unit_result = unit_refresh.refresh_after_update()
|
||||
if unit_result['message']:
|
||||
pull_message += " " + unit_result['message']
|
||||
else:
|
||||
logger.warning("git pull failed (returncode=%d): %s", result.returncode, result.stderr)
|
||||
# Show git's own first line: "check logs" leaves the user with
|
||||
@@ -556,6 +565,8 @@ def _perform_core_update_locked(stash_local_changes=True):
|
||||
'restart_required': bool(result.returncode == 0 and code_changed),
|
||||
'dependency_failures': dependency_failures,
|
||||
'channel': channel.channel,
|
||||
# web_interface/unit_refresh.py's result, or None when no code changed.
|
||||
'unit_refresh': unit_result,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
"""After an update, bring the installed systemd units in line with the new templates.
|
||||
|
||||
An update moves the checkout, and with it systemd/*.service, but systemd runs
|
||||
the copies in /etc/systemd/system, which used to be written only by the
|
||||
installer. Settings added to a template (the render-loop watchdog, a memory
|
||||
limit) therefore never reached a device that was already installed.
|
||||
|
||||
Updates now run the root-owned helper /usr/local/sbin/ledmatrix-refresh-units
|
||||
(scripts/install/ledmatrix_refresh_units.py) through sudo when the rendered
|
||||
units differ from the installed ones. The services pick the new units up at
|
||||
the restart that follows the update. The automatic update's rollback runs the
|
||||
same helper with ``--restore`` (scripts/utils/auto_update_verify.py).
|
||||
|
||||
The sudo rule is written by the installer, so a device installed before it
|
||||
existed cannot run the helper. That is reported, not fatal: the update
|
||||
itself stands, and the message says to re-run the installer once, the same
|
||||
remedy as the display's startup "unit drift" warning.
|
||||
"""
|
||||
import importlib.util
|
||||
import logging
|
||||
import os
|
||||
import subprocess # nosec B404 - list-form argv only, no shell # nosemgrep
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent
|
||||
HELPER_SOURCE = PROJECT_ROOT / 'scripts' / 'install' / 'ledmatrix_refresh_units.py'
|
||||
#: The installed, root-owned copy that sudo is allowed to run.
|
||||
HELPER_PATH = '/usr/local/sbin/ledmatrix-refresh-units'
|
||||
SYSTEMD_DIR = '/etc/systemd/system'
|
||||
TIMEOUT_SECONDS = 90
|
||||
|
||||
#: Outcomes (``result['status']``).
|
||||
CURRENT = 'current' # nothing differs
|
||||
REFRESHED = 'refreshed' # installed the new units
|
||||
NEEDS_REINSTALL = 'needs_reinstall' # the helper or its sudo rule is missing
|
||||
FAILED = 'failed' # the helper ran and refused or failed
|
||||
SKIPPED = 'skipped' # not a systemd install (dev machine, emulator)
|
||||
|
||||
REINSTALL_HINT = ('Run "sudo ./first_time_install.sh" in the LEDMatrix folder once '
|
||||
'(or "sudo ./scripts/install/install_service.sh" followed by '
|
||||
'"./scripts/install/configure_web_sudo.sh") so updates can apply them.')
|
||||
|
||||
#: sudo's words for "this command line is not allowed without a password".
|
||||
_SUDO_REFUSED = ('a password is required', 'is not allowed to run', 'no tty present',
|
||||
'a terminal is required', 'command not found')
|
||||
|
||||
|
||||
def _load_helper(path=HELPER_SOURCE):
|
||||
spec = importlib.util.spec_from_file_location('ledmatrix_refresh_units', str(path))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def stale_units(systemd_dir=None, helper_source=None):
|
||||
"""Installed units whose rendering from the checkout differs. Needs no root.
|
||||
|
||||
Returns a sorted list, or None when this is not a systemd install.
|
||||
Raises the helper's RefreshError when a unit cannot be rendered safely.
|
||||
"""
|
||||
systemd_dir = systemd_dir or SYSTEMD_DIR
|
||||
helper_source = helper_source or HELPER_SOURCE
|
||||
if not os.path.isfile(os.path.join(systemd_dir, 'ledmatrix.service')):
|
||||
return None
|
||||
helper = _load_helper(helper_source)
|
||||
return sorted(helper.Refresher(systemd_dir=systemd_dir).plan())
|
||||
|
||||
|
||||
def _result(status, message, units=()):
|
||||
return {'status': status, 'message': message, 'units': list(units)}
|
||||
|
||||
|
||||
def refresh_after_update(run=None, systemd_dir=None, helper_source=None, helper_path=None):
|
||||
"""Install the units an update changed. Never raises.
|
||||
|
||||
Returns ``{'status', 'message', 'units'}``; ``message`` is '' when there
|
||||
is nothing to tell the user. The defaults are the module's constants,
|
||||
read at call time (the test suite points SYSTEMD_DIR away from the host).
|
||||
"""
|
||||
run = run or subprocess.run
|
||||
helper_path = helper_path or HELPER_PATH
|
||||
try:
|
||||
stale = stale_units(systemd_dir, helper_source)
|
||||
except Exception as e: # a broken template must not fail the update itself
|
||||
if type(e).__name__ != 'UnitsUnreadable':
|
||||
logger.warning("Could not compare the installed systemd units with the new templates: %s", e)
|
||||
return _result(FAILED, f'The service settings could not be checked: {e}.')
|
||||
# Units installed mode 0600 (install_service.sh run on its own, before
|
||||
# it set 0644): only root can compare them, so let the helper decide.
|
||||
stale = []
|
||||
unknown = True
|
||||
else:
|
||||
unknown = False
|
||||
if stale is None:
|
||||
return _result(SKIPPED, '')
|
||||
if not stale:
|
||||
return _result(CURRENT, '')
|
||||
|
||||
names = ', '.join(stale) or 'the LEDMatrix units'
|
||||
changes = (f'This update changes service settings ({names}) that are not applied yet. ' if not unknown
|
||||
else 'Service settings this update may change could not be checked or applied. ')
|
||||
if not os.path.isfile(helper_path):
|
||||
logger.warning("Updates cannot install systemd unit changes (%s): %s is not installed; "
|
||||
"they take effect only after a reinstall. %s", names, helper_path, REINSTALL_HINT)
|
||||
return _result(NEEDS_REINSTALL, changes + REINSTALL_HINT, stale)
|
||||
try:
|
||||
result = run(['sudo', '-n', helper_path], capture_output=True, text=True,
|
||||
timeout=TIMEOUT_SECONDS)
|
||||
except (subprocess.SubprocessError, OSError) as e:
|
||||
logger.warning("Refreshing the systemd units failed: %s", e)
|
||||
return _result(FAILED, f'Updating the service settings ({names}) failed: {e}.', stale)
|
||||
if result.returncode == 0:
|
||||
# The helper says what it did: "units refreshed: a b" or "units: up to date".
|
||||
done = next((line.split(':', 1)[1].split() for line in (result.stdout or '').splitlines()
|
||||
if line.startswith('units refreshed:')), None)
|
||||
if not done:
|
||||
# Nothing replaced, so nothing for a rollback to restore.
|
||||
if stale:
|
||||
logger.warning("The unit helper found nothing to change in %s; the installed "
|
||||
"helper may be older than this version", names)
|
||||
return _result(CURRENT, '')
|
||||
logger.info("Refreshed systemd units after the update: %s", ', '.join(done))
|
||||
return _result(REFRESHED, f'Service settings updated ({", ".join(done)}).', done)
|
||||
detail = (result.stderr or result.stdout or '').strip()
|
||||
if any(phrase in detail.lower() for phrase in _SUDO_REFUSED):
|
||||
logger.warning("Updates cannot install systemd unit changes (%s): no sudo rule for %s; "
|
||||
"they take effect only after a reinstall. %s", names, helper_path, REINSTALL_HINT)
|
||||
return _result(NEEDS_REINSTALL, changes + REINSTALL_HINT, stale)
|
||||
logger.warning("Refreshing the systemd units failed (exit %s): %s", result.returncode, detail)
|
||||
return _result(FAILED, f'Updating the service settings ({names}) failed: {detail or "unknown error"}.',
|
||||
stale)
|
||||
Reference in New Issue
Block a user