mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-08-04 18:28:06 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2fab41924b | ||
|
|
183e23edb3 | ||
|
|
970ca2d04f | ||
|
|
f2b246ef03 | ||
|
|
963ab8292a | ||
|
|
16fbb7ebeb |
@@ -0,0 +1,40 @@
|
|||||||
|
name: Release version check
|
||||||
|
|
||||||
|
# A release tag, the CHANGELOG, and src.__version__ must agree. They have not
|
||||||
|
# always: v3.1.0 was tagged while src/__init__.py still said "1.0.0", which
|
||||||
|
# silently exempted every device installed from that release from plugin
|
||||||
|
# compatibility warnings. See docs/SPORTS_UNIFICATION.md (phase B4).
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
tags: ["v*"]
|
||||||
|
release:
|
||||||
|
types: [published]
|
||||||
|
# Pre-flight: run this against the tag you are about to create.
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
tag:
|
||||||
|
description: "Tag to check (e.g. v3.2.0)"
|
||||||
|
required: true
|
||||||
|
type: string
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
version-matches-tag:
|
||||||
|
name: Tag matches src.__version__
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
|
||||||
|
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
|
||||||
|
with:
|
||||||
|
python-version: "3.12"
|
||||||
|
|
||||||
|
# No dependencies: the script reads src/__init__.py and CHANGELOG.md only.
|
||||||
|
- name: Assert the tag, CHANGELOG and src.__version__ agree
|
||||||
|
run: python scripts/check_release_version.py "${TAG}"
|
||||||
|
env:
|
||||||
|
TAG: ${{ inputs.tag || github.ref_name }}
|
||||||
@@ -76,4 +76,8 @@ jobs:
|
|||||||
test/test_sports_core_promotions.py \
|
test/test_sports_core_promotions.py \
|
||||||
test/test_sports_modes_promotions.py \
|
test/test_sports_modes_promotions.py \
|
||||||
test/test_sports_capabilities.py \
|
test/test_sports_capabilities.py \
|
||||||
test/test_sports_scroll.py
|
test/test_sports_scroll.py \
|
||||||
|
test/test_version_consistency.py \
|
||||||
|
test/test_plugin_compatibility_gate.py \
|
||||||
|
test/test_install_preserves_existing.py \
|
||||||
|
test/test_core_owned_config_keys.py
|
||||||
|
|||||||
@@ -29,6 +29,21 @@ Adoption is deliberately staged: the modules below ship here, plugins adopt them
|
|||||||
behind guarded imports, and only then do the bundled copies go away. Nothing in
|
behind guarded imports, and only then do the bundled copies go away. Nothing in
|
||||||
this release changes what an existing plugin loads.
|
this release changes what an existing plugin loads.
|
||||||
|
|
||||||
|
**This is also the first release that *enforces* `ledmatrix_min_version`.**
|
||||||
|
Before it, the floor was advisory — the loader logged a warning and continued,
|
||||||
|
and the plugin store never compared the core version at all, so an update could
|
||||||
|
deliver a plugin that could not run. From 3.2.0 the store refuses such an
|
||||||
|
install. That matters for the sunset rule: a plugin may only delete its bundled
|
||||||
|
fallback once the cores in the field actually enforce the floor, which means
|
||||||
|
waiting for 3.2.0 to be widely installed rather than merely released. See
|
||||||
|
`docs/SPORTS_UNIFICATION.md`, phase B6.
|
||||||
|
|
||||||
|
One deliberate exception: a core reporting a version below `2.0.0` is treated as
|
||||||
|
*unknown* rather than old and is never blocked. The v3.1.0 release ships
|
||||||
|
`__version__ = "1.0.0"` (the tag was cut before the string was bumped), and
|
||||||
|
nearly every published manifest floors at `2.0.0` — so blocking on that number
|
||||||
|
would lock those users out of the plugin store entirely.
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- `src/element_style.py` — per-element style resolver backing the
|
- `src/element_style.py` — per-element style resolver backing the
|
||||||
`x-style-elements` config-schema extension. Already consumed (behind guarded
|
`x-style-elements` config-schema extension. Already consumed (behind guarded
|
||||||
@@ -71,8 +86,37 @@ this release changes what an existing plugin loads.
|
|||||||
override point — see `docs/SPORTS_UNIFICATION.md` for where the line falls
|
override point — see `docs/SPORTS_UNIFICATION.md` for where the line falls
|
||||||
and why.
|
and why.
|
||||||
|
|
||||||
|
- `src/plugin_system/compatibility.py` — the single place that answers "can this
|
||||||
|
plugin run on this core?", shared by the loader (advisory, at load time) and
|
||||||
|
the store (blocking, at install/update time) so the two cannot drift. Reads
|
||||||
|
every spelling published manifests use, including the deprecated
|
||||||
|
`versions[].ledmatrix_min`. It does **not** yet evaluate `compatible_versions`,
|
||||||
|
which is the schema-required field and can express upper bounds; closing that
|
||||||
|
is tracked in `docs/SPORTS_UNIFICATION.md` before B6.
|
||||||
|
- `scripts/check_release_version.py` and a `Release version check` workflow —
|
||||||
|
assert that a tag, the newest CHANGELOG heading and `src.__version__` agree,
|
||||||
|
on pushed `v*` tags and published releases. Runnable via `workflow_dispatch`
|
||||||
|
to check a tag *before* creating it. Added because `v3.1.0` was tagged six
|
||||||
|
weeks before `src/__init__.py` was bumped to match, which is why devices
|
||||||
|
installed from that release report `1.0.0`.
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
- `src/__init__.py` bumped to **3.2.0** — the number the sunset rule keys on.
|
- `src/__init__.py` bumped to **3.2.0** — the number the sunset rule keys on.
|
||||||
|
- **The plugin store refuses an incompatible install.**
|
||||||
|
`StoreManager.install_plugin` now checks the downloaded manifest's declared
|
||||||
|
floor against `src.__version__` and refuses when the plugin needs a newer
|
||||||
|
core. The check sits in `install_plugin` because `_reinstall_with_rollback`
|
||||||
|
calls it, so a refused *update* restores the version the user already had.
|
||||||
|
Refusal requires evidence: an undeclared floor, an unparseable version on
|
||||||
|
either side, or an untrustworthy core version all allow the install.
|
||||||
|
- **A failed install no longer destroys the plugin it replaced.**
|
||||||
|
`install_plugin` previously deleted the existing plugin directory before
|
||||||
|
downloading, so any later failure — a dropped connection, a malformed
|
||||||
|
manifest, or the new compatibility refusal — left the user with nothing. The
|
||||||
|
existing copy is now set aside and restored if the install fails, matching
|
||||||
|
the protection `_reinstall_with_rollback` already gave the update path.
|
||||||
|
- `web_interface.__version__` re-exports `src.__version__` instead of carrying
|
||||||
|
its own hardcoded `"3.0.0"`, which had drifted two majors from the core.
|
||||||
- **Live games are no longer dropped when the feed omits a game clock.**
|
- **Live games are no longer dropped when the feed omits a game clock.**
|
||||||
`SportsLive._is_game_really_over` previously (in the baseball and UFC
|
`SportsLive._is_game_really_over` previously (in the baseball and UFC
|
||||||
plugin lineages) coerced a missing or non-string clock to the literal
|
plugin lineages) coerced a missing or non-string clock to the literal
|
||||||
@@ -86,6 +130,12 @@ this release changes what an existing plugin loads.
|
|||||||
there means kickoff rather than expiry.
|
there means kickoff rather than expiry.
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
- **Plugin updates could hang the web request thread.** The per-plugin reinstall
|
||||||
|
locks were non-reentrant, and `_reinstall_with_rollback` holds one across its
|
||||||
|
call to `install_plugin` — which now takes the same lock to protect the
|
||||||
|
set-aside/restore above. That nesting deadlocked
|
||||||
|
`update_plugin → _reinstall_with_rollback → install_plugin`, the standard
|
||||||
|
path for every monorepo plugin update. The locks are now `RLock`s.
|
||||||
- `FontManager` resolves `assets/fonts` against the core install root instead
|
- `FontManager` resolves `assets/fonts` against the core install root instead
|
||||||
of the process working directory, so font loading works when the process
|
of the process working directory, so font loading works when the process
|
||||||
starts elsewhere (e.g. the plugin safety harness on CI).
|
starts elsewhere (e.g. the plugin safety harness on CI).
|
||||||
|
|||||||
@@ -141,6 +141,7 @@ The system supports live, recent, and upcoming game information for multiple spo
|
|||||||
sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh
|
sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh
|
||||||
```
|
```
|
||||||
- Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and set `gpio_slowdown` to `1` or `2`.
|
- Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and set `gpio_slowdown` to `1` or `2`.
|
||||||
|
- **1GB models (Pi 3B / 3B+) and other low-memory boards**: supported, but the `rpi-rgb-led-matrix` C++ build needs more memory than the Pi has. The installer detects this automatically, compiles with fewer parallel jobs, and adds a temporary swapfile for the build which it removes afterwards. Expect that step to take 15-25 minutes instead of 2-5, and leave at least **3GB free** on the SD card. If you manage swap yourself, opt out with `--skip-swap`. To pin the compiler down further, use `--build-jobs 1`.
|
||||||
|
|
||||||
|
|
||||||
### RGB Matrix Bonnet / HAT
|
### RGB Matrix Bonnet / HAT
|
||||||
@@ -314,12 +315,12 @@ curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/
|
|||||||
```
|
```
|
||||||
|
|
||||||
This one-shot installer will automatically:
|
This one-shot installer will automatically:
|
||||||
- Check system prerequisites (network, disk space, sudo access)
|
- Check system prerequisites (network, disk space, memory, sudo access)
|
||||||
- Install required system packages (git, python3, build tools, etc.)
|
- Install required system packages (git, python3, build tools, etc.)
|
||||||
- Clone or update the LEDMatrix repository
|
- Clone or update the LEDMatrix repository
|
||||||
- Run the complete first-time installation script
|
- Run the complete first-time installation script
|
||||||
|
|
||||||
The installation process typically takes 10-30 minutes depending on your internet connection and Pi model. All errors are reported explicitly with actionable fixes.
|
The installation process typically takes 10-30 minutes depending on your internet connection and Pi model. Pi 3B/3B+ and other 1GB boards land at the top of that range, because the C++ library is compiled serially to stay within available memory. All errors are reported explicitly with actionable fixes.
|
||||||
|
|
||||||
**Note:** The script is safe to run multiple times and will handle existing installations gracefully.
|
**Note:** The script is safe to run multiple times and will handle existing installations gracefully.
|
||||||
|
|
||||||
|
|||||||
@@ -82,6 +82,70 @@ python3 web_interface/start.py
|
|||||||
|
|
||||||
## Common Issues by Category
|
## Common Issues by Category
|
||||||
|
|
||||||
|
### Installation & Build Issues
|
||||||
|
|
||||||
|
#### Step 6 fails: "Failed building wheel for rgbmatrix"
|
||||||
|
|
||||||
|
**Symptoms:**
|
||||||
|
|
||||||
|
```
|
||||||
|
note: This error originates from a subprocess, and is likely not a problem with pip.
|
||||||
|
ERROR: Failed building wheel for rgbmatrix
|
||||||
|
Failed to build rgbmatrix
|
||||||
|
✗ Failed to install rpi-rgb-led-matrix Python package
|
||||||
|
```
|
||||||
|
|
||||||
|
**Cause:**
|
||||||
|
|
||||||
|
Almost always the kernel's out-of-memory killer, not missing build tools. The
|
||||||
|
`rpi-rgb-led-matrix` library compiles roughly 45 C++ translation units, two of
|
||||||
|
them Cython-generated — a single `cc1plus` on those can peak near 800MB. The
|
||||||
|
build system defaults to running several of those at once, which exceeds RAM on
|
||||||
|
512MB and 1GB boards. Because the OOM killer writes nothing to pip's output, the
|
||||||
|
failure looks like a toolchain problem, and `sudo apt install -y
|
||||||
|
python-dev-is-python3 cmake build-essential` will report everything is already
|
||||||
|
up to date.
|
||||||
|
|
||||||
|
**How to confirm:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dmesg -T | grep -i "out of memory" # look for "Killed process ... (cc1plus)"
|
||||||
|
free -h # total RAM and swap
|
||||||
|
```
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
Current versions of the installer handle this automatically: they cap build
|
||||||
|
parallelism based on available RAM and add a temporary swapfile for the build,
|
||||||
|
removing it when the build finishes. If you are on an older checkout, or the
|
||||||
|
temporary swapfile could not be created, either force a serial compile:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo ./first_time_install.sh --build-jobs 1
|
||||||
|
```
|
||||||
|
|
||||||
|
or add permanent swap and re-run the installer, which resumes at Step 6:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install -y dphys-swapfile
|
||||||
|
sudo sed -i 's/^#\?CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile
|
||||||
|
sudo sed -i 's/^#\?CONF_MAXSWAP=.*/CONF_MAXSWAP=2048/' /etc/dphys-swapfile
|
||||||
|
sudo dphys-swapfile swapoff && sudo dphys-swapfile setup && sudo dphys-swapfile swapon
|
||||||
|
sudo ./first_time_install.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
`CONF_MAXSWAP` matters: it defaults to 2048 and silently clamps `CONF_SWAPSIZE`,
|
||||||
|
so setting only `CONF_SWAPSIZE` to a larger value has no effect.
|
||||||
|
|
||||||
|
**Related:**
|
||||||
|
|
||||||
|
- The installer needs roughly 3GB free on the card to place the swapfile. If
|
||||||
|
disk is tight it will say so and skip the swapfile: `sudo apt clean` first.
|
||||||
|
- `sudo bash scripts/check_system_compatibility.sh` reports RAM and disk.
|
||||||
|
- `sudo bash scripts/diagnose_dependencies.sh` dumps build-dependency state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
### Web Interface & Service Issues
|
### Web Interface & Service Issues
|
||||||
|
|
||||||
#### Service Not Running/Starting
|
#### Service Not Running/Starting
|
||||||
|
|||||||
+220
-9
@@ -152,6 +152,8 @@ ASSUME_YES=${LEDMATRIX_ASSUME_YES:-0}
|
|||||||
SKIP_SOUND=${LEDMATRIX_SKIP_SOUND:-0}
|
SKIP_SOUND=${LEDMATRIX_SKIP_SOUND:-0}
|
||||||
SKIP_PERF=${LEDMATRIX_SKIP_PERF:-0}
|
SKIP_PERF=${LEDMATRIX_SKIP_PERF:-0}
|
||||||
SKIP_REBOOT_PROMPT=${LEDMATRIX_SKIP_REBOOT_PROMPT:-0}
|
SKIP_REBOOT_PROMPT=${LEDMATRIX_SKIP_REBOOT_PROMPT:-0}
|
||||||
|
SKIP_SWAP=${LEDMATRIX_SKIP_SWAP:-0}
|
||||||
|
BUILD_JOBS_OVERRIDE=${LEDMATRIX_BUILD_JOBS:-}
|
||||||
|
|
||||||
usage() {
|
usage() {
|
||||||
cat <<USAGE
|
cat <<USAGE
|
||||||
@@ -163,11 +165,21 @@ Options:
|
|||||||
--skip-sound Skip sound module configuration
|
--skip-sound Skip sound module configuration
|
||||||
--skip-perf Skip performance tweaks (isolcpus/audio)
|
--skip-perf Skip performance tweaks (isolcpus/audio)
|
||||||
--no-reboot-prompt Do not prompt for reboot at the end
|
--no-reboot-prompt Do not prompt for reboot at the end
|
||||||
|
--skip-swap Never add temporary swap for the C++ build
|
||||||
|
--build-jobs N Compile the C++ library with N parallel jobs
|
||||||
|
(default: scaled to available RAM)
|
||||||
-h, --help Show this help message and exit
|
-h, --help Show this help message and exit
|
||||||
|
|
||||||
Environment variables (same effect as flags):
|
Environment variables (same effect as flags):
|
||||||
LEDMATRIX_ASSUME_YES=1, RPI_RGB_FORCE_REBUILD=1, LEDMATRIX_SKIP_SOUND=1,
|
LEDMATRIX_ASSUME_YES=1, RPI_RGB_FORCE_REBUILD=1, LEDMATRIX_SKIP_SOUND=1,
|
||||||
LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1
|
LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1,
|
||||||
|
LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N
|
||||||
|
|
||||||
|
Low-memory devices:
|
||||||
|
On a Pi with under 2GB of RAM the C++ build is limited to fewer parallel
|
||||||
|
jobs and a temporary swapfile is added for the duration of the build, then
|
||||||
|
removed. Without this the compiler is killed by the kernel out-of-memory
|
||||||
|
killer on 512MB and 1GB models.
|
||||||
USAGE
|
USAGE
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -178,12 +190,38 @@ while [ $# -gt 0 ]; do
|
|||||||
--skip-sound) SKIP_SOUND=1 ;;
|
--skip-sound) SKIP_SOUND=1 ;;
|
||||||
--skip-perf) SKIP_PERF=1 ;;
|
--skip-perf) SKIP_PERF=1 ;;
|
||||||
--no-reboot-prompt) SKIP_REBOOT_PROMPT=1 ;;
|
--no-reboot-prompt) SKIP_REBOOT_PROMPT=1 ;;
|
||||||
|
--skip-swap) SKIP_SWAP=1 ;;
|
||||||
|
--build-jobs)
|
||||||
|
shift
|
||||||
|
if [ $# -eq 0 ]; then echo "--build-jobs requires a number"; usage; exit 1; fi
|
||||||
|
BUILD_JOBS_OVERRIDE="$1"
|
||||||
|
;;
|
||||||
-h|--help) usage; exit 0 ;;
|
-h|--help) usage; exit 0 ;;
|
||||||
*) echo "Unknown option: $1"; usage; exit 1 ;;
|
*) echo "Unknown option: $1"; usage; exit 1 ;;
|
||||||
esac
|
esac
|
||||||
shift
|
shift
|
||||||
done
|
done
|
||||||
|
|
||||||
|
# Low-memory build helpers (job sizing, temporary swap, OOM detection).
|
||||||
|
# Sourced rather than inlined so the sizing logic can be unit-tested; if the
|
||||||
|
# file is missing we fall back to the historical behaviour rather than failing
|
||||||
|
# the install.
|
||||||
|
LOWMEM_LIB="$PROJECT_ROOT_DIR/scripts/install/lib_lowmem.sh"
|
||||||
|
LOWMEM_AVAILABLE=0
|
||||||
|
if [ -f "$LOWMEM_LIB" ]; then
|
||||||
|
# shellcheck source=scripts/install/lib_lowmem.sh
|
||||||
|
. "$LOWMEM_LIB"
|
||||||
|
LOWMEM_AVAILABLE=1
|
||||||
|
else
|
||||||
|
echo "⚠ $LOWMEM_LIB not found; skipping low-memory build protections."
|
||||||
|
lm_remove_build_swap() { return 0; }
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Remove the temporary build swapfile no matter how the script ends. Step 6
|
||||||
|
# tears it down itself; this is the backstop for the error path, since
|
||||||
|
# on_error ends in `exit` and EXIT traps still run.
|
||||||
|
trap 'lm_remove_build_swap' EXIT
|
||||||
|
|
||||||
# Helpers
|
# Helpers
|
||||||
retry() {
|
retry() {
|
||||||
local attempt=1
|
local attempt=1
|
||||||
@@ -263,15 +301,144 @@ check_disk_space() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Decide how much memory Step 6's C++ build may use, and say so up front.
|
||||||
|
#
|
||||||
|
# Sets TOTAL_RAM_MB, TOTAL_SWAP_MB, BUILD_JOBS and LOW_RAM for later steps.
|
||||||
|
check_memory() {
|
||||||
|
command -v nproc >/dev/null 2>&1 && CPU_CORES=$(nproc) || CPU_CORES=1
|
||||||
|
|
||||||
|
# Validated up front rather than trusted: a non-numeric value would other-
|
||||||
|
# wise survive as far as an arithmetic test in Step 6 and fail there with a
|
||||||
|
# generic error. This must precede the fallback return below, which also
|
||||||
|
# honours the override.
|
||||||
|
if [ -n "$BUILD_JOBS_OVERRIDE" ]; then
|
||||||
|
if ! echo "$BUILD_JOBS_OVERRIDE" | grep -qE '^[1-9][0-9]*$'; then
|
||||||
|
echo "✗ Invalid build job count: '$BUILD_JOBS_OVERRIDE' (expected a positive integer)"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$LOWMEM_AVAILABLE" != "1" ]; then
|
||||||
|
TOTAL_RAM_MB=0
|
||||||
|
TOTAL_SWAP_MB=0
|
||||||
|
LOW_RAM=0
|
||||||
|
BUILD_JOBS=${BUILD_JOBS_OVERRIDE:-$CPU_CORES}
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
TOTAL_RAM_MB=$(lm_total_ram_mb)
|
||||||
|
TOTAL_SWAP_MB=$(lm_total_swap_mb)
|
||||||
|
|
||||||
|
# Test hook: exercise the low-memory path on a machine that has plenty.
|
||||||
|
if [ -n "${LEDMATRIX_FORCE_LOW_RAM:-}" ] && [ "${LEDMATRIX_FORCE_LOW_RAM}" != "0" ]; then
|
||||||
|
TOTAL_RAM_MB="${LEDMATRIX_FORCE_LOW_RAM}"
|
||||||
|
echo "⚠ LEDMATRIX_FORCE_LOW_RAM set: pretending this device has ${TOTAL_RAM_MB}MB of RAM"
|
||||||
|
fi
|
||||||
|
|
||||||
|
LOW_RAM=0
|
||||||
|
if [ "$TOTAL_RAM_MB" -gt 0 ] && [ "$TOTAL_RAM_MB" -lt 2048 ]; then
|
||||||
|
LOW_RAM=1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "$BUILD_JOBS_OVERRIDE" ]; then
|
||||||
|
BUILD_JOBS="$BUILD_JOBS_OVERRIDE"
|
||||||
|
else
|
||||||
|
BUILD_JOBS=$(lm_build_jobs "$TOTAL_RAM_MB" "$CPU_CORES")
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "System memory: ${TOTAL_RAM_MB}MB RAM, ${TOTAL_SWAP_MB}MB swap, ${CPU_CORES} core(s)"
|
||||||
|
if [ "$LOW_RAM" = "1" ]; then
|
||||||
|
echo "⚠ Low-memory device detected."
|
||||||
|
echo " The rpi-rgb-led-matrix C++ build in Step 6 will use ${BUILD_JOBS} parallel job(s)"
|
||||||
|
echo " instead of all cores, and a temporary swapfile will be added for the build"
|
||||||
|
echo " and removed afterwards. Without this the compiler is killed by the kernel"
|
||||||
|
echo " out-of-memory killer. Expect Step 6 to take 15-25 minutes."
|
||||||
|
if [ "$SKIP_SWAP" = "1" ]; then
|
||||||
|
echo " Temporary swap is disabled (--skip-swap); the build may still run out of memory."
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "✓ Memory sufficient for the rpi-rgb-led-matrix build (${BUILD_JOBS} parallel job(s))"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Compile and install the rgbmatrix Python package.
|
||||||
|
#
|
||||||
|
# CMAKE_BUILD_PARALLEL_LEVEL is the setting that actually caps the compile:
|
||||||
|
# upstream's pyproject.toml declares no [tool.scikit-build] options, so
|
||||||
|
# scikit-build-core drives Ninja through `cmake --build`, which reads this
|
||||||
|
# variable. Ninja's own default is nproc+2, i.e. six concurrent cc1plus
|
||||||
|
# processes on a 4-core Pi. MAKEFLAGS is ignored by Ninja and is set only to
|
||||||
|
# cover the Makefile-generator fallback if ninja-build is somehow absent.
|
||||||
|
#
|
||||||
|
# BUILD_TMPDIR redirects pip's build tree off tmpfs where applicable — see
|
||||||
|
# where it is computed in Step 6.
|
||||||
|
run_rgbmatrix_build() {
|
||||||
|
local jobs="$1" out="$2"
|
||||||
|
local pid elapsed=0
|
||||||
|
|
||||||
|
TMPDIR="${BUILD_TMPDIR:-${TMPDIR:-/tmp}}" \
|
||||||
|
CMAKE_BUILD_PARALLEL_LEVEL="$jobs" \
|
||||||
|
MAKEFLAGS="-j${jobs}" \
|
||||||
|
python3 -m pip install --break-system-packages . > "$out" 2>&1 &
|
||||||
|
pid=$!
|
||||||
|
|
||||||
|
# The build's output is captured to a file, so without a heartbeat a serial
|
||||||
|
# compile on a 1GB Pi looks like a 20-minute hang and invites a Ctrl-C.
|
||||||
|
#
|
||||||
|
# Polled at a short interval but reported every 30s: polling at the report
|
||||||
|
# interval instead would add most of that interval to the wall time of
|
||||||
|
# every build, including fast ones on a Pi 4/5.
|
||||||
|
while kill -0 "$pid" 2>/dev/null; do
|
||||||
|
sleep 2
|
||||||
|
elapsed=$((elapsed + 2))
|
||||||
|
if [ "$((elapsed % 30))" -eq 0 ] && kill -0 "$pid" 2>/dev/null; then
|
||||||
|
printf ' ... still compiling (%dm%02ds elapsed)\n' "$((elapsed / 60))" "$((elapsed % 60))"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
wait "$pid"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Explain a failed rgbmatrix build. The kernel OOM killer writes nothing to the
|
||||||
|
# build's own output, which is why this used to be reported as a missing
|
||||||
|
# build-tools problem and sent users chasing packages they already had.
|
||||||
|
print_rgbmatrix_build_failure() {
|
||||||
|
local out="$1"
|
||||||
|
|
||||||
|
if [ "$LOWMEM_AVAILABLE" = "1" ] && lm_build_failed_on_oom "$out"; then
|
||||||
|
echo "✗ The rpi-rgb-led-matrix build was killed: the system ran out of memory."
|
||||||
|
echo " This is NOT a missing build-tools problem — the C++ compiler ran out of RAM."
|
||||||
|
echo " RAM: ${TOTAL_RAM_MB}MB Swap: $(lm_total_swap_mb)MB Parallel jobs used: ${BUILD_JOBS}"
|
||||||
|
if [ -n "${LM_SWAP_SKIP_REASON:-}" ]; then
|
||||||
|
echo " No temporary swap was added: ${LM_SWAP_SKIP_REASON}"
|
||||||
|
fi
|
||||||
|
echo ""
|
||||||
|
echo " Try one of these, then re-run this script (it resumes at Step 6):"
|
||||||
|
echo " 1. Force a single compile job:"
|
||||||
|
echo " sudo ./first_time_install.sh --build-jobs 1"
|
||||||
|
echo " 2. Add permanent swap, if the temporary swapfile could not be created:"
|
||||||
|
echo " sudo apt install -y dphys-swapfile"
|
||||||
|
echo " sudo sed -i 's/^#\\?CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile"
|
||||||
|
echo " sudo sed -i 's/^#\\?CONF_MAXSWAP=.*/CONF_MAXSWAP=2048/' /etc/dphys-swapfile"
|
||||||
|
echo " sudo dphys-swapfile swapoff && sudo dphys-swapfile setup && sudo dphys-swapfile swapon"
|
||||||
|
echo " 3. Free up disk space so a larger swapfile fits: sudo apt clean"
|
||||||
|
else
|
||||||
|
echo "✗ Failed to install rpi-rgb-led-matrix Python package"
|
||||||
|
echo " Ensure build tools are installed:"
|
||||||
|
echo " sudo apt install -y python-dev-is-python3 cmake build-essential"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
echo "This script will perform the following steps:"
|
echo "This script will perform the following steps:"
|
||||||
echo "1. Install system dependencies"
|
echo "1. Check prerequisites (network, disk, memory) and install system dependencies"
|
||||||
echo "2. Fix cache permissions"
|
echo "2. Fix cache permissions"
|
||||||
echo "3. Fix assets directory permissions"
|
echo "3. Fix assets directory permissions"
|
||||||
echo "3.1. Fix plugin directory permissions"
|
echo "3.1. Fix plugin directory permissions"
|
||||||
echo "4. Ensure configuration files exist"
|
echo "4. Ensure configuration files exist"
|
||||||
echo "5. Install Python project dependencies (requirements.txt)"
|
echo "5. Install Python project dependencies (requirements.txt)"
|
||||||
echo "6. Build and install rpi-rgb-led-matrix and test import"
|
echo "6. Build and install rpi-rgb-led-matrix and test import"
|
||||||
|
echo " (compiles C++; low-memory Pis get temporary swap and a serial build)"
|
||||||
echo "7. Install web interface dependencies"
|
echo "7. Install web interface dependencies"
|
||||||
echo "7.5. Install main LED Matrix service"
|
echo "7.5. Install main LED Matrix service"
|
||||||
echo "8. Install web interface service"
|
echo "8. Install web interface service"
|
||||||
@@ -315,9 +482,16 @@ echo "----------------------------------------"
|
|||||||
# Pre-flight checks before APT operations
|
# Pre-flight checks before APT operations
|
||||||
check_network
|
check_network
|
||||||
check_disk_space
|
check_disk_space
|
||||||
|
check_memory
|
||||||
|
|
||||||
# Update package list
|
# Update package list. The one-shot installer refreshes the lists moments
|
||||||
apt_update
|
# before invoking this script and exports LEDMATRIX_APT_UPDATED=1, so skip the
|
||||||
|
# duplicate refresh on that path.
|
||||||
|
if [ "${LEDMATRIX_APT_UPDATED:-0}" = "1" ]; then
|
||||||
|
echo "Package lists already refreshed by the one-shot installer; skipping apt update."
|
||||||
|
else
|
||||||
|
apt_update
|
||||||
|
fi
|
||||||
|
|
||||||
# Install required system packages
|
# Install required system packages
|
||||||
echo "Installing Python packages and dependencies..."
|
echo "Installing Python packages and dependencies..."
|
||||||
@@ -902,29 +1076,66 @@ else
|
|||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Add temporary swap on low-memory devices so the compiler survives.
|
||||||
|
CURRENT_STEP="Prepare the low-memory build environment"
|
||||||
|
if [ "$LOWMEM_AVAILABLE" = "1" ] && [ "$SKIP_SWAP" != "1" ]; then
|
||||||
|
lm_ensure_build_swap "$(lm_swap_needed_mb "$TOTAL_RAM_MB" "$TOTAL_SWAP_MB")"
|
||||||
|
elif [ "$SKIP_SWAP" = "1" ]; then
|
||||||
|
LM_SWAP_SKIP_REASON="disabled with --skip-swap"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# pip builds in $TMPDIR. Debian 13 mounts /tmp as tmpfs, so the default
|
||||||
|
# would hold the entire C++ build tree in RAM — competing with the very
|
||||||
|
# compiler we are trying to keep under the memory limit.
|
||||||
|
BUILD_TMPDIR=""
|
||||||
|
if [ "$LOWMEM_AVAILABLE" = "1" ]; then
|
||||||
|
_disk_tmp=$(lm_disk_backed_tmpdir)
|
||||||
|
if [ -n "$_disk_tmp" ]; then
|
||||||
|
BUILD_TMPDIR="$_disk_tmp/ledmatrix-build"
|
||||||
|
# If this fails (a nearly-full disk being the likely cause on
|
||||||
|
# exactly the devices this targets), fall back to the default
|
||||||
|
# rather than pointing the build at a path that does not exist.
|
||||||
|
if mkdir -p "$BUILD_TMPDIR" 2>/dev/null; then
|
||||||
|
echo "Building in $BUILD_TMPDIR (TMPDIR is memory-backed; keeping the build tree on disk)"
|
||||||
|
else
|
||||||
|
echo "⚠ Could not create $BUILD_TMPDIR; falling back to the default TMPDIR"
|
||||||
|
BUILD_TMPDIR=""
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
CURRENT_STEP="Build and install rpi-rgb-led-matrix"
|
||||||
pushd "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" >/dev/null
|
pushd "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" >/dev/null
|
||||||
echo "Installing rpi-rgb-led-matrix Python package (scikit-build-core + cmake)..."
|
echo "Installing rpi-rgb-led-matrix Python package (scikit-build-core + cmake)..."
|
||||||
echo " Build deps required: python-dev-is-python3 cmake"
|
echo " Build deps required: python-dev-is-python3 cmake"
|
||||||
echo " This compiles C++ — may take 2-5 minutes on Pi 4/5..."
|
echo " Compiling C++ with ${BUILD_JOBS} parallel job(s)..."
|
||||||
|
if [ "$BUILD_JOBS" -le 1 ]; then
|
||||||
|
echo " Deliberately serial to stay within this device's memory — expect 15-25 minutes."
|
||||||
|
else
|
||||||
|
echo " This may take 2-5 minutes on a Pi 4/5..."
|
||||||
|
fi
|
||||||
BUILD_OUTPUT=$(mktemp)
|
BUILD_OUTPUT=$(mktemp)
|
||||||
BUILD_SUCCESS=false
|
BUILD_SUCCESS=false
|
||||||
if python3 -m pip install --break-system-packages . > "$BUILD_OUTPUT" 2>&1; then
|
if run_rgbmatrix_build "$BUILD_JOBS" "$BUILD_OUTPUT"; then
|
||||||
BUILD_SUCCESS=true
|
BUILD_SUCCESS=true
|
||||||
fi
|
fi
|
||||||
cat "$BUILD_OUTPUT" >> "$LOG_FILE"
|
cat "$BUILD_OUTPUT" >> "$LOG_FILE"
|
||||||
if [ "$BUILD_SUCCESS" != true ]; then
|
if [ "$BUILD_SUCCESS" != true ]; then
|
||||||
echo "✗ Failed to install rpi-rgb-led-matrix Python package"
|
print_rgbmatrix_build_failure "$BUILD_OUTPUT"
|
||||||
echo " Ensure build tools are installed:"
|
|
||||||
echo " sudo apt install -y python-dev-is-python3 cmake build-essential"
|
|
||||||
echo ""
|
echo ""
|
||||||
echo "-- Last 50 lines of build output --"
|
echo "-- Last 50 lines of build output --"
|
||||||
tail -n 50 "$BUILD_OUTPUT"
|
tail -n 50 "$BUILD_OUTPUT"
|
||||||
rm -f "$BUILD_OUTPUT"
|
rm -f "$BUILD_OUTPUT"
|
||||||
|
if [ -n "$BUILD_TMPDIR" ]; then rm -rf "$BUILD_TMPDIR"; fi
|
||||||
popd >/dev/null
|
popd >/dev/null
|
||||||
|
lm_remove_build_swap
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
rm -f "$BUILD_OUTPUT"
|
rm -f "$BUILD_OUTPUT"
|
||||||
|
if [ -n "$BUILD_TMPDIR" ]; then rm -rf "$BUILD_TMPDIR"; fi
|
||||||
popd >/dev/null
|
popd >/dev/null
|
||||||
|
# Hand the memory back well before Step 14's reboot.
|
||||||
|
lm_remove_build_swap
|
||||||
else
|
else
|
||||||
echo "✗ rpi-rgb-led-matrix-master directory not found at $PROJECT_ROOT_DIR"
|
echo "✗ rpi-rgb-led-matrix-master directory not found at $PROJECT_ROOT_DIR"
|
||||||
echo "Failed to initialize submodule or clone repository"
|
echo "Failed to initialize submodule or clone repository"
|
||||||
|
|||||||
@@ -0,0 +1,122 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Assert that a release tag, the CHANGELOG, and `src.__version__` all agree.
|
||||||
|
|
||||||
|
Run it *before* creating a tag to check yourself:
|
||||||
|
|
||||||
|
python scripts/check_release_version.py v3.2.0
|
||||||
|
|
||||||
|
Wiring it into CI (on pushed `v*` tags and published releases) is a follow-up
|
||||||
|
PR, so for now it is a manual pre-flight: run it before creating the tag and a
|
||||||
|
mismatch shows up here rather than as a silent wrong answer on user devices.
|
||||||
|
|
||||||
|
Why this exists: `v3.1.0` was tagged 2026-05-31 while `src/__init__.py` still
|
||||||
|
said `"1.0.0"`; the bump to `"3.1.0"` did not land until 2026-07-12. Devices
|
||||||
|
installed from that release report `1.0.0`, which is below the `(2, 0, 0)` floor
|
||||||
|
in `PluginLoader._warn_if_incompatible`, so they are silently exempt from every
|
||||||
|
plugin compatibility warning. Plugin `ledmatrix_min_version` floors are only as
|
||||||
|
trustworthy as this agreement. See `docs/SPORTS_UNIFICATION.md`, phase B4.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
sys.path.insert(0, str(REPO_ROOT))
|
||||||
|
|
||||||
|
# [0-9] rather than \d, and [ \t] rather than \s: \d also matches non-ASCII
|
||||||
|
# decimal digits (which int() parses), and \s matches newlines, so "##\n3.2.0"
|
||||||
|
# would otherwise read as a version heading. Keep these in step with
|
||||||
|
# test/test_version_consistency.py.
|
||||||
|
SEMVER = re.compile(r"^[0-9]+\.[0-9]+\.[0-9]+$")
|
||||||
|
HEADING = re.compile(
|
||||||
|
r"^##[ \t]+(?P<version>[0-9]+\.[0-9]+\.[0-9]+)[ \t]*$", re.MULTILINE)
|
||||||
|
|
||||||
|
|
||||||
|
def normalize(tag: str) -> str:
|
||||||
|
"""`v3.2.0` and `3.2.0` are the same release; tags here carry the `v`."""
|
||||||
|
return tag[1:] if tag.startswith("v") else tag
|
||||||
|
|
||||||
|
|
||||||
|
def newest_changelog_version(changelog: Path) -> str | None:
|
||||||
|
"""Newest version heading, or None when there is none.
|
||||||
|
|
||||||
|
Raises OSError if the file cannot be read; main() turns that into a clear
|
||||||
|
message rather than a traceback, because this runs as a release gate and a
|
||||||
|
traceback there reads as "the tooling is broken", not "your CHANGELOG is
|
||||||
|
missing".
|
||||||
|
"""
|
||||||
|
headings = HEADING.findall(changelog.read_text(encoding="utf-8"))
|
||||||
|
return headings[0] if headings else None
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument(
|
||||||
|
"tag",
|
||||||
|
help="Release tag to check, with or without the leading 'v' (e.g. v3.2.0)",
|
||||||
|
)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
from src import __version__ as core_version
|
||||||
|
|
||||||
|
tag_version = normalize(args.tag)
|
||||||
|
changelog_path = REPO_ROOT / "CHANGELOG.md"
|
||||||
|
|
||||||
|
problems: list[str] = []
|
||||||
|
|
||||||
|
try:
|
||||||
|
changelog_version = newest_changelog_version(changelog_path)
|
||||||
|
except OSError as e:
|
||||||
|
print(
|
||||||
|
f"Release version check FAILED for tag {args.tag}:\n"
|
||||||
|
f" - could not read {changelog_path}: {e}\n"
|
||||||
|
f" Restore the file (git checkout -- CHANGELOG.md) and re-run.",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
if not SEMVER.match(tag_version):
|
||||||
|
problems.append(
|
||||||
|
f"tag {args.tag!r} is not vX.Y.Z. Older tags (v2.5) predate this "
|
||||||
|
"check; new releases must be full semver so floors can parse them."
|
||||||
|
)
|
||||||
|
|
||||||
|
if not SEMVER.match(core_version):
|
||||||
|
problems.append(f"src.__version__ is {core_version!r}, which is not X.Y.Z")
|
||||||
|
|
||||||
|
if tag_version != core_version:
|
||||||
|
problems.append(
|
||||||
|
f"tag says {tag_version} but src.__version__ says {core_version}. "
|
||||||
|
"Bump src/__init__.py to match the tag before releasing — devices "
|
||||||
|
"report __version__, not the tag, and plugin floors compare "
|
||||||
|
"against it."
|
||||||
|
)
|
||||||
|
|
||||||
|
if changelog_version is None:
|
||||||
|
problems.append("CHANGELOG.md has no '## X.Y.Z' version heading")
|
||||||
|
elif changelog_version != core_version:
|
||||||
|
problems.append(
|
||||||
|
f"CHANGELOG.md's newest heading is {changelog_version} but "
|
||||||
|
f"src.__version__ is {core_version}. Plugin authors read the "
|
||||||
|
"CHANGELOG to pick a ledmatrix_min_version floor."
|
||||||
|
)
|
||||||
|
|
||||||
|
if problems:
|
||||||
|
print(f"Release version check FAILED for tag {args.tag}:", file=sys.stderr)
|
||||||
|
for problem in problems:
|
||||||
|
print(f" - {problem}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
print(
|
||||||
|
f"OK: tag {args.tag}, src.__version__ {core_version}, and the CHANGELOG "
|
||||||
|
"all agree."
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -156,9 +156,13 @@ echo ""
|
|||||||
echo "6. Check disk space - building packages requires temporary space"
|
echo "6. Check disk space - building packages requires temporary space"
|
||||||
echo " df -h"
|
echo " df -h"
|
||||||
echo ""
|
echo ""
|
||||||
echo "7. For slow builds, increase swap space:"
|
echo "7. For slow builds or out-of-memory kills, increase swap space."
|
||||||
|
echo " first_time_install.sh already adds temporary swap on low-memory devices;"
|
||||||
|
echo " this makes it permanent. Set CONF_MAXSWAP too - it defaults to 2048 and"
|
||||||
|
echo " silently clamps CONF_SWAPSIZE, so raising CONF_SWAPSIZE alone does nothing."
|
||||||
echo " sudo dphys-swapfile swapoff"
|
echo " sudo dphys-swapfile swapoff"
|
||||||
echo " sudo nano /etc/dphys-swapfile # Set CONF_SWAPSIZE=2048"
|
echo " sudo sed -i 's/^#\\?CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile"
|
||||||
|
echo " sudo sed -i 's/^#\\?CONF_MAXSWAP=.*/CONF_MAXSWAP=2048/' /etc/dphys-swapfile"
|
||||||
echo " sudo dphys-swapfile setup"
|
echo " sudo dphys-swapfile setup"
|
||||||
echo " sudo dphys-swapfile swapon"
|
echo " sudo dphys-swapfile swapon"
|
||||||
echo ""
|
echo ""
|
||||||
|
|||||||
@@ -0,0 +1,283 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
#
|
||||||
|
# Low-memory build helpers for the LED Matrix installer.
|
||||||
|
#
|
||||||
|
# Sourced by first_time_install.sh. These live in a separate, sourceable file
|
||||||
|
# so the pure sizing/detection functions can be unit-tested
|
||||||
|
# (test/test_install_lowmem.py); first_time_install.sh itself is not sourceable
|
||||||
|
# because it self-elevates and runs top to bottom.
|
||||||
|
#
|
||||||
|
# Why this exists: the rgbmatrix build compiles ~45 C++ translation units, two
|
||||||
|
# of them Cython-generated (a single cc1plus on those peaks around 400-800MB at
|
||||||
|
# -O3). Upstream's pyproject.toml sets no [tool.scikit-build] options, so
|
||||||
|
# scikit-build-core uses Ninja at its default of nproc+2 jobs -- six concurrent
|
||||||
|
# compiles on a 4-core Pi. On a 512MB-1GB Pi the OOM killer reaps cc1plus and
|
||||||
|
# pip reports only "Failed building wheel for rgbmatrix".
|
||||||
|
#
|
||||||
|
# The caller runs under `set -Eeuo pipefail` with an ERR trap, and these are
|
||||||
|
# invoked from the middle of numbered steps, so nothing here may call exit and
|
||||||
|
# the swap helpers must always return 0.
|
||||||
|
|
||||||
|
# Overridable so tests can point at fixture files instead of /proc.
|
||||||
|
LM_MEMINFO="${LM_MEMINFO:-/proc/meminfo}"
|
||||||
|
LM_SWAPS="${LM_SWAPS:-/proc/swaps}"
|
||||||
|
|
||||||
|
# Temporary swapfile created for the build and removed afterwards. Deliberately
|
||||||
|
# never added to /etc/fstab: a malformed fstab can leave a novice with an
|
||||||
|
# unbootable Pi, and this swap only needs to outlive the compile.
|
||||||
|
LM_SWAPFILE="${LM_SWAPFILE:-/var/swap.ledmatrix-install}"
|
||||||
|
|
||||||
|
# Bring RAM + real swap up to this much before compiling, capped per swapfile.
|
||||||
|
LM_SWAP_TARGET_MB="${LM_SWAP_TARGET_MB:-3072}"
|
||||||
|
LM_SWAP_MAX_MB="${LM_SWAP_MAX_MB:-2048}"
|
||||||
|
|
||||||
|
# Worst-case cc1plus footprint on the Cython translation unit, used to size
|
||||||
|
# build parallelism against available RAM.
|
||||||
|
LM_MB_PER_JOB="${LM_MB_PER_JOB:-768}"
|
||||||
|
|
||||||
|
# Set to 1 once swap is live, so lm_remove_build_swap (wired up as an EXIT
|
||||||
|
# trap) knows whether there is anything to undo.
|
||||||
|
LM_TEMP_SWAP_ACTIVE=0
|
||||||
|
|
||||||
|
# Human-readable reason no swapfile was created, quoted back in the failure
|
||||||
|
# message so a user who still OOMs is told why the safety net was absent.
|
||||||
|
LM_SWAP_SKIP_REASON=""
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Pure helpers (no side effects; unit-tested)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
# Total physical RAM in MB, or 0 if it cannot be determined.
|
||||||
|
lm_total_ram_mb() {
|
||||||
|
# Defaults are re-resolved here as well as at source time so the function
|
||||||
|
# stays safe under the installer's `set -u`.
|
||||||
|
awk '/^MemTotal:/ {printf "%d\n", $2 / 1024; found = 1; exit} END {if (!found) print 0}' \
|
||||||
|
"${LM_MEMINFO:-/proc/meminfo}" 2>/dev/null || echo 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# Total swap in MB, EXCLUDING zram devices.
|
||||||
|
#
|
||||||
|
# zram swap is compressed RAM: it consumes the very resource that is already
|
||||||
|
# exhausted and does nothing for a build OOM. Counting it would let a
|
||||||
|
# zram-enabled image decide it has enough swap and then fail exactly as before.
|
||||||
|
lm_total_swap_mb() {
|
||||||
|
awk 'NR > 1 && $1 !~ /^\/dev\/zram/ {total += $3} END {printf "%d\n", total / 1024}' \
|
||||||
|
"${LM_SWAPS:-/proc/swaps}" 2>/dev/null || echo 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# lm_build_jobs <ram_mb> <cores> -> max(1, min(cores, ram_mb / LM_MB_PER_JOB))
|
||||||
|
#
|
||||||
|
# Computed from RAM alone and never RAM+swap: handing out extra jobs because
|
||||||
|
# swap exists just guarantees SD-card thrash, which is far slower than
|
||||||
|
# compiling serially.
|
||||||
|
lm_build_jobs() {
|
||||||
|
local ram_mb="${1:-0}" cores="${2:-1}" jobs
|
||||||
|
local per_job="${LM_MB_PER_JOB:-768}"
|
||||||
|
if [ "$cores" -lt 1 ]; then
|
||||||
|
cores=1
|
||||||
|
fi
|
||||||
|
jobs=$(( ram_mb / per_job ))
|
||||||
|
if [ "$jobs" -lt 1 ]; then
|
||||||
|
jobs=1
|
||||||
|
fi
|
||||||
|
if [ "$jobs" -gt "$cores" ]; then
|
||||||
|
jobs="$cores"
|
||||||
|
fi
|
||||||
|
echo "$jobs"
|
||||||
|
}
|
||||||
|
|
||||||
|
# lm_swap_needed_mb <ram_mb> <existing_swap_mb> -> swapfile size in MB, or 0.
|
||||||
|
#
|
||||||
|
# Brings RAM + real swap up to LM_SWAP_TARGET_MB, capped at LM_SWAP_MAX_MB and
|
||||||
|
# rounded up to a 256MB multiple. Machines with enough memory get 0 and are
|
||||||
|
# left completely untouched.
|
||||||
|
lm_swap_needed_mb() {
|
||||||
|
local ram_mb="${1:-0}" swap_mb="${2:-0}" needed
|
||||||
|
local target="${LM_SWAP_TARGET_MB:-3072}" max="${LM_SWAP_MAX_MB:-2048}"
|
||||||
|
needed=$(( target - ram_mb - swap_mb ))
|
||||||
|
if [ "$needed" -le 0 ]; then
|
||||||
|
echo 0
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if [ "$needed" -gt "$max" ]; then
|
||||||
|
needed="$max"
|
||||||
|
fi
|
||||||
|
echo $(( ( (needed + 255) / 256 ) * 256 ))
|
||||||
|
}
|
||||||
|
|
||||||
|
# lm_build_failed_on_oom <build_output_file> -> 0 if the build was OOM-killed.
|
||||||
|
#
|
||||||
|
# Two independent evidence sources, because neither alone is reliable: the
|
||||||
|
# compiler sometimes reports its own allocation failure, but when the kernel
|
||||||
|
# OOM killer fires it writes nothing to the build's stdout. That silence is
|
||||||
|
# exactly why the old handler misdiagnosed this as missing build tools.
|
||||||
|
lm_build_failed_on_oom() {
|
||||||
|
local build_output="${1:-}" kernel_log=""
|
||||||
|
|
||||||
|
if [ -n "$build_output" ] && [ -f "$build_output" ]; then
|
||||||
|
if grep -qiE 'cc1plus: out of memory|virtual memory exhausted|Cannot allocate memory|MemoryError|fatal error: Killed signal terminated program|signal 9' \
|
||||||
|
"$build_output"; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# LM_KERNEL_LOG_FILE lets tests supply a fixture instead of the real kernel
|
||||||
|
# ring buffer, which on a shared CI machine may hold unrelated OOM events.
|
||||||
|
if [ -n "${LM_KERNEL_LOG_FILE:-}" ]; then
|
||||||
|
if [ -f "$LM_KERNEL_LOG_FILE" ]; then
|
||||||
|
kernel_log=$(cat "$LM_KERNEL_LOG_FILE" 2>/dev/null || true)
|
||||||
|
fi
|
||||||
|
elif command -v dmesg >/dev/null 2>&1; then
|
||||||
|
kernel_log=$(dmesg -T 2>/dev/null || dmesg 2>/dev/null || true)
|
||||||
|
fi
|
||||||
|
if [ -z "$kernel_log" ] && [ -z "${LM_KERNEL_LOG_FILE:-}" ] && command -v journalctl >/dev/null 2>&1; then
|
||||||
|
kernel_log=$(journalctl -k --since "30 min ago" --no-pager 2>/dev/null || true)
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "$kernel_log" ]; then
|
||||||
|
if printf '%s\n' "$kernel_log" | tail -n 300 | \
|
||||||
|
grep -qiE 'Out of memory: Kill|oom_kill|oom-kill|Killed process'; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# lm_disk_backed_tmpdir [candidate] -> a disk-backed temp dir, or nothing.
|
||||||
|
#
|
||||||
|
# pip builds in $TMPDIR. Debian 13 mounts /tmp as tmpfs, so the default puts the
|
||||||
|
# whole C++ build tree in RAM, competing with the compiler we are already trying
|
||||||
|
# to keep under the limit. Prints a replacement only when the current TMPDIR is
|
||||||
|
# memory-backed and the candidate is not; otherwise prints nothing and the
|
||||||
|
# caller keeps its default.
|
||||||
|
lm_disk_backed_tmpdir() {
|
||||||
|
local candidate="${1:-/var/tmp}"
|
||||||
|
local current="${TMPDIR:-/tmp}"
|
||||||
|
local current_fs="" candidate_fs=""
|
||||||
|
|
||||||
|
current_fs=$(lm_fstype_of "$current")
|
||||||
|
case "$current_fs" in
|
||||||
|
tmpfs|ramfs) ;;
|
||||||
|
*) return 0 ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
candidate_fs=$(lm_fstype_of "$candidate")
|
||||||
|
case "$candidate_fs" in
|
||||||
|
tmpfs|ramfs|"") return 0 ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
echo "$candidate"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Filesystem type backing a path, or empty if it cannot be determined.
|
||||||
|
lm_fstype_of() {
|
||||||
|
local path="${1:-/}"
|
||||||
|
if command -v findmnt >/dev/null 2>&1; then
|
||||||
|
findmnt -no FSTYPE --target "$path" 2>/dev/null | head -n 1
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if command -v stat >/dev/null 2>&1; then
|
||||||
|
stat -f -c %T "$path" 2>/dev/null | head -n 1
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Swap management (requires root; not unit-tested)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
# lm_ensure_build_swap <needed_mb>
|
||||||
|
#
|
||||||
|
# Always returns 0. On any refusal it sets LM_SWAP_SKIP_REASON and leaves the
|
||||||
|
# system untouched -- swap is a safety net for the build, never a precondition.
|
||||||
|
lm_ensure_build_swap() {
|
||||||
|
local needed_mb="${1:-0}"
|
||||||
|
local swap_dir free_mb budget
|
||||||
|
|
||||||
|
LM_SWAP_SKIP_REASON=""
|
||||||
|
|
||||||
|
if [ "$needed_mb" -le 0 ]; then
|
||||||
|
LM_SWAP_SKIP_REASON="not needed (RAM and existing swap are sufficient)"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! command -v mkswap >/dev/null 2>&1 || ! command -v swapon >/dev/null 2>&1; then
|
||||||
|
LM_SWAP_SKIP_REASON="mkswap/swapon are not available on this system"
|
||||||
|
echo "⚠ Cannot add build swap: $LM_SWAP_SKIP_REASON"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Clear a stale swapfile left by a run that was killed before its cleanup
|
||||||
|
# ran, so this is safe to call repeatedly.
|
||||||
|
if [ -e "$LM_SWAPFILE" ]; then
|
||||||
|
echo "Removing a leftover swapfile from a previous run: $LM_SWAPFILE"
|
||||||
|
swapoff "$LM_SWAPFILE" >/dev/null 2>&1 || true
|
||||||
|
rm -f "$LM_SWAPFILE" || true
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Keep a working margin for the build tree itself; never eat the last GB.
|
||||||
|
swap_dir=$(dirname "$LM_SWAPFILE")
|
||||||
|
free_mb=$(df -m "$swap_dir" 2>/dev/null | awk 'NR==2{print $4}')
|
||||||
|
free_mb=${free_mb:-0}
|
||||||
|
budget=$(( free_mb - 1024 ))
|
||||||
|
if [ "$budget" -lt 256 ]; then
|
||||||
|
LM_SWAP_SKIP_REASON="only ${free_mb}MB free on ${swap_dir}, need about $(( needed_mb + 1024 ))MB"
|
||||||
|
echo "⚠ Skipping the build swapfile: $LM_SWAP_SKIP_REASON"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if [ "$needed_mb" -gt "$budget" ]; then
|
||||||
|
echo "⚠ Trimming the build swapfile from ${needed_mb}MB to leave 1GB free on ${swap_dir}"
|
||||||
|
needed_mb=$(( ( budget / 256 ) * 256 ))
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Adding a temporary ${needed_mb}MB swapfile for the build: $LM_SWAPFILE"
|
||||||
|
echo " This is removed automatically once the build finishes."
|
||||||
|
|
||||||
|
# fallocate can produce a sparse file that mkswap rejects, and is not
|
||||||
|
# supported on every filesystem; dd always yields a usable file.
|
||||||
|
if ! fallocate -l "${needed_mb}M" "$LM_SWAPFILE" 2>/dev/null; then
|
||||||
|
if ! dd if=/dev/zero of="$LM_SWAPFILE" bs=1M count="$needed_mb" status=none 2>/dev/null; then
|
||||||
|
LM_SWAP_SKIP_REASON="could not allocate ${needed_mb}MB at $LM_SWAPFILE"
|
||||||
|
echo "⚠ $LM_SWAP_SKIP_REASON"
|
||||||
|
rm -f "$LM_SWAPFILE" || true
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
chmod 600 "$LM_SWAPFILE" || true
|
||||||
|
|
||||||
|
if ! mkswap "$LM_SWAPFILE" >/dev/null 2>&1; then
|
||||||
|
LM_SWAP_SKIP_REASON="mkswap failed on $LM_SWAPFILE"
|
||||||
|
echo "⚠ $LM_SWAP_SKIP_REASON"
|
||||||
|
rm -f "$LM_SWAPFILE" || true
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! swapon "$LM_SWAPFILE" >/dev/null 2>&1; then
|
||||||
|
LM_SWAP_SKIP_REASON="swapon failed on $LM_SWAPFILE"
|
||||||
|
echo "⚠ $LM_SWAP_SKIP_REASON"
|
||||||
|
rm -f "$LM_SWAPFILE" || true
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
LM_TEMP_SWAP_ACTIVE=1
|
||||||
|
echo "✓ Temporary build swap active (${needed_mb}MB; total swap is now $(lm_total_swap_mb)MB)"
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# Remove the temporary swapfile. Safe to call unconditionally and repeatedly.
|
||||||
|
#
|
||||||
|
# Wired up as an EXIT trap, so it must never return non-zero -- a failing trap
|
||||||
|
# would surface as a spurious installer error.
|
||||||
|
lm_remove_build_swap() {
|
||||||
|
if [ "${LM_TEMP_SWAP_ACTIVE:-0}" != "1" ]; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
LM_TEMP_SWAP_ACTIVE=0
|
||||||
|
echo "Removing the temporary build swapfile: $LM_SWAPFILE"
|
||||||
|
swapoff "$LM_SWAPFILE" >/dev/null 2>&1 || true
|
||||||
|
rm -f "$LM_SWAPFILE" || true
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -145,6 +145,34 @@ check_disk_space() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Report available memory so the user knows what to expect before the wait.
|
||||||
|
#
|
||||||
|
# Informational only — first_time_install.sh does the real work of capping
|
||||||
|
# build parallelism and adding temporary swap. Never fatal: a low-RAM Pi is
|
||||||
|
# supported, it is just slower.
|
||||||
|
check_memory() {
|
||||||
|
CURRENT_STEP="Memory check"
|
||||||
|
if [ ! -r /proc/meminfo ]; then
|
||||||
|
print_warning "Cannot read /proc/meminfo, skipping memory check"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
TOTAL_RAM_MB=$(awk '/^MemTotal:/ {printf "%d\n", $2 / 1024; exit}' /proc/meminfo 2>/dev/null || echo 0)
|
||||||
|
TOTAL_RAM_MB=${TOTAL_RAM_MB:-0}
|
||||||
|
|
||||||
|
if [ "$TOTAL_RAM_MB" -eq 0 ]; then
|
||||||
|
print_warning "Could not determine system memory, continuing"
|
||||||
|
elif [ "$TOTAL_RAM_MB" -lt 2048 ]; then
|
||||||
|
print_warning "Low memory: ${TOTAL_RAM_MB}MB RAM"
|
||||||
|
echo " The rpi-rgb-led-matrix C++ build needs more memory than this Pi has."
|
||||||
|
echo " The installer will compile with fewer parallel jobs and add a temporary"
|
||||||
|
echo " swapfile for the build, removing it afterwards. That step will take"
|
||||||
|
echo " 15-25 minutes rather than the usual 2-5."
|
||||||
|
else
|
||||||
|
print_success "Memory sufficient: ${TOTAL_RAM_MB}MB RAM"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
# Ensure sudo access
|
# Ensure sudo access
|
||||||
check_sudo() {
|
check_sudo() {
|
||||||
CURRENT_STEP="Sudo access check"
|
CURRENT_STEP="Sudo access check"
|
||||||
@@ -204,7 +232,7 @@ main() {
|
|||||||
print_step "LED Matrix One-Shot Installation"
|
print_step "LED Matrix One-Shot Installation"
|
||||||
|
|
||||||
echo "This script will:"
|
echo "This script will:"
|
||||||
echo " 1. Check prerequisites (network, disk space, sudo)"
|
echo " 1. Check prerequisites (network, disk space, memory, sudo)"
|
||||||
echo " 2. Install system dependencies (git, python3, build tools)"
|
echo " 2. Install system dependencies (git, python3, build tools)"
|
||||||
echo " 3. Clone the LEDMatrix repository"
|
echo " 3. Clone the LEDMatrix repository"
|
||||||
echo " 4. Run the first-time installation script"
|
echo " 4. Run the first-time installation script"
|
||||||
@@ -213,6 +241,7 @@ main() {
|
|||||||
# Check prerequisites
|
# Check prerequisites
|
||||||
check_network
|
check_network
|
||||||
check_disk_space
|
check_disk_space
|
||||||
|
check_memory
|
||||||
check_sudo
|
check_sudo
|
||||||
# Note: /tmp permissions are checked and fixed inline before running first_time_install.sh
|
# Note: /tmp permissions are checked and fixed inline before running first_time_install.sh
|
||||||
# (only if actually wrong, not preemptively)
|
# (only if actually wrong, not preemptively)
|
||||||
@@ -228,12 +257,14 @@ main() {
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Update package list first
|
# Update package list first. first_time_install.sh is told the lists are
|
||||||
|
# already fresh so it does not repeat this a minute later.
|
||||||
if [ "$EUID" -eq 0 ]; then
|
if [ "$EUID" -eq 0 ]; then
|
||||||
retry apt-get update -qq
|
retry apt-get update -qq
|
||||||
else
|
else
|
||||||
retry sudo apt-get update -qq
|
retry sudo apt-get update -qq
|
||||||
fi
|
fi
|
||||||
|
export LEDMATRIX_APT_UPDATED=1
|
||||||
|
|
||||||
# Install git and curl (needed for cloning and the script itself)
|
# Install git and curl (needed for cloning and the script itself)
|
||||||
if ! command -v git >/dev/null 2>&1 || ! command -v curl >/dev/null 2>&1; then
|
if ! command -v git >/dev/null 2>&1 || ! command -v curl >/dev/null 2>&1; then
|
||||||
@@ -372,7 +403,12 @@ main() {
|
|||||||
# Pass both -y flag AND environment variable for non-interactive mode
|
# Pass both -y flag AND environment variable for non-interactive mode
|
||||||
# This ensures it works even if the script re-executes itself with sudo
|
# This ensures it works even if the script re-executes itself with sudo
|
||||||
# Also ensure stdin is properly handled for non-interactive mode
|
# Also ensure stdin is properly handled for non-interactive mode
|
||||||
sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 bash ./first_time_install.sh -y </dev/null
|
# LEDMATRIX_APT_UPDATED is passed explicitly rather than relying on
|
||||||
|
# -E: a sudoers env_reset/env_keep policy can strip exported variables,
|
||||||
|
# which would silently reinstate the duplicate apt update.
|
||||||
|
sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 \
|
||||||
|
LEDMATRIX_APT_UPDATED="${LEDMATRIX_APT_UPDATED:-0}" \
|
||||||
|
bash ./first_time_install.sh -y </dev/null
|
||||||
fi
|
fi
|
||||||
INSTALL_EXIT_CODE=$?
|
INSTALL_EXIT_CODE=$?
|
||||||
trap 'on_error $LINENO' ERR # Re-enable ERR trap
|
trap 'on_error $LINENO' ERR # Re-enable ERR trap
|
||||||
|
|||||||
@@ -0,0 +1,102 @@
|
|||||||
|
"""One place that answers "can this plugin run on this core?".
|
||||||
|
|
||||||
|
Two callers ask that question and they must not drift apart:
|
||||||
|
|
||||||
|
- `PluginLoader._warn_if_incompatible` — at load time, **advisory**. A plugin
|
||||||
|
already on disk keeps loading regardless, because the guarded-import pattern
|
||||||
|
means most incompatibilities degrade rather than break.
|
||||||
|
- `PluginStoreManager.install_plugin` — at install/update time, **blocking**.
|
||||||
|
This is the point where refusing costs the user nothing (they keep the
|
||||||
|
version they already had) and allowing can cost them a plugin that fails to
|
||||||
|
load with only a log line to explain it.
|
||||||
|
|
||||||
|
## The trustworthiness problem
|
||||||
|
|
||||||
|
The core's own `__version__` has not always been right. `v3.1.0` was tagged
|
||||||
|
2026-05-31 while `src/__init__.py` still said `"1.0.0"`; the bump landed
|
||||||
|
2026-07-12. Devices installed from that release report `1.0.0` — below the
|
||||||
|
floor that essentially every published plugin declares.
|
||||||
|
|
||||||
|
So a core reporting a version below `TRUSTWORTHY_FLOOR` is treated as
|
||||||
|
**unknown, not old**: it neither warns nor blocks. Blocking on it would be far
|
||||||
|
worse than the problem being solved — nearly every manifest in the ecosystem
|
||||||
|
floors at `2.0.0`, so a strict gate would stop those users installing *any*
|
||||||
|
plugin. They are unprotected until they update the core, which is also what
|
||||||
|
fixes their version string. See `docs/SPORTS_UNIFICATION.md`, phase B4.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Dict, Optional, Tuple
|
||||||
|
|
||||||
|
# Below this, the core's self-reported version is not evidence of anything.
|
||||||
|
# See the module docstring.
|
||||||
|
TRUSTWORTHY_FLOOR: Tuple[int, int, int] = (2, 0, 0)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_semver(value: Any) -> Optional[Tuple[int, int, int]]:
|
||||||
|
"""Parse ``X.Y.Z`` (extra parts and suffixes ignored) into a comparable
|
||||||
|
3-tuple, or ``None`` when unparseable. A leading ``v`` is tolerated."""
|
||||||
|
if not isinstance(value, str):
|
||||||
|
return None
|
||||||
|
parts = value.strip().lstrip('v').split('.')
|
||||||
|
try:
|
||||||
|
nums = [int(''.join(ch for ch in p if ch.isdigit()) or 0) for p in parts[:3]]
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
while len(nums) < 3:
|
||||||
|
nums.append(0)
|
||||||
|
return tuple(nums) # type: ignore[return-value]
|
||||||
|
|
||||||
|
|
||||||
|
def declared_min_version(manifest: Dict[str, Any]) -> Optional[str]:
|
||||||
|
"""The core version this plugin says it needs, or ``None`` if it doesn't say.
|
||||||
|
|
||||||
|
Checked in order of specificity. `ledmatrix_min` is the deprecated spelling
|
||||||
|
of `ledmatrix_min_version` (`store_manager._validate_manifest_fields` flags
|
||||||
|
it); both are read because a large share of published manifests still carry
|
||||||
|
the old one.
|
||||||
|
"""
|
||||||
|
declared = (
|
||||||
|
manifest.get('min_ledmatrix_version')
|
||||||
|
or (manifest.get('requires') or {}).get('min_ledmatrix_version')
|
||||||
|
)
|
||||||
|
if declared:
|
||||||
|
return declared
|
||||||
|
|
||||||
|
versions = manifest.get('versions') or []
|
||||||
|
if versions and isinstance(versions[0], dict):
|
||||||
|
return (versions[0].get('ledmatrix_min_version')
|
||||||
|
or versions[0].get('ledmatrix_min'))
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def check(manifest: Dict[str, Any], core_version: str) -> Tuple[bool, Optional[str]]:
|
||||||
|
"""Return ``(compatible, reason)``.
|
||||||
|
|
||||||
|
``compatible`` is False **only** when the plugin declares a parseable floor,
|
||||||
|
the core reports a parseable and trustworthy version, and the floor is
|
||||||
|
genuinely above it. Every uncertain case resolves to compatible: an
|
||||||
|
undeclared floor, an unparseable version on either side, or a core whose
|
||||||
|
version is below `TRUSTWORTHY_FLOOR`. Refusing on a guess would break
|
||||||
|
working installs, which is the more expensive mistake here.
|
||||||
|
|
||||||
|
``reason`` is user-facing text, present only when incompatible.
|
||||||
|
"""
|
||||||
|
declared = declared_min_version(manifest)
|
||||||
|
needed = parse_semver(declared)
|
||||||
|
if needed is None:
|
||||||
|
return True, None
|
||||||
|
|
||||||
|
current = parse_semver(core_version)
|
||||||
|
if current is None or current < TRUSTWORTHY_FLOOR:
|
||||||
|
return True, None
|
||||||
|
|
||||||
|
if needed > current:
|
||||||
|
name = manifest.get('name') or manifest.get('id') or 'This plugin'
|
||||||
|
return False, (
|
||||||
|
f"{name} requires LEDMatrix {declared} or newer, but this system is "
|
||||||
|
f"running {core_version}. Update LEDMatrix first, then install it."
|
||||||
|
)
|
||||||
|
|
||||||
|
return True, None
|
||||||
@@ -702,30 +702,21 @@ class PluginLoader:
|
|||||||
newer than the running core. Advisory only — never raises — so a
|
newer than the running core. Advisory only — never raises — so a
|
||||||
plugin that guards optional features with try/except keeps working.
|
plugin that guards optional features with try/except keeps working.
|
||||||
"""
|
"""
|
||||||
declared = (
|
|
||||||
manifest.get('min_ledmatrix_version')
|
|
||||||
or manifest.get('requires', {}).get('min_ledmatrix_version')
|
|
||||||
)
|
|
||||||
if not declared:
|
|
||||||
versions = manifest.get('versions') or []
|
|
||||||
if versions and isinstance(versions[0], dict):
|
|
||||||
declared = (versions[0].get('ledmatrix_min_version')
|
|
||||||
or versions[0].get('ledmatrix_min'))
|
|
||||||
needed = self._parse_semver(declared)
|
|
||||||
if needed is None:
|
|
||||||
return
|
|
||||||
|
|
||||||
from src import __version__ as core_version
|
from src import __version__ as core_version
|
||||||
current = self._parse_semver(core_version)
|
from src.plugin_system import compatibility
|
||||||
# Anti-spam guard: if the core's own version number is stale (below
|
|
||||||
# the ecosystem floor every shipped plugin declares), comparing would
|
compatible, _reason = compatibility.check(manifest, core_version)
|
||||||
# warn on nearly everything — skip with a debug note instead.
|
if compatible:
|
||||||
if current is None or current < (2, 0, 0):
|
# Distinguish "fine" from "couldn't tell" for anyone reading logs:
|
||||||
|
# a core below the trustworthy floor is skipped, not cleared.
|
||||||
|
current = compatibility.parse_semver(core_version)
|
||||||
|
if current is None or current < compatibility.TRUSTWORTHY_FLOOR:
|
||||||
self.logger.debug(
|
self.logger.debug(
|
||||||
"Skipping version compatibility check for %s: core __version__ "
|
"Skipping version compatibility check for %s: core __version__ "
|
||||||
"(%s) is below the ecosystem floor", plugin_id, core_version)
|
"(%s) is below the ecosystem floor", plugin_id, core_version)
|
||||||
return
|
return
|
||||||
if needed > current:
|
|
||||||
|
declared = compatibility.declared_min_version(manifest)
|
||||||
self.logger.warning(
|
self.logger.warning(
|
||||||
"Plugin %s declares min LEDMatrix version %s but this core is %s — "
|
"Plugin %s declares min LEDMatrix version %s but this core is %s — "
|
||||||
"features it relies on may be missing; update the core or expect "
|
"features it relies on may be missing; update the core or expect "
|
||||||
|
|||||||
@@ -395,6 +395,37 @@ class PluginManager:
|
|||||||
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
|
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
#: Config keys the **core** reads out of a plugin's own config block. The
|
||||||
|
#: plugin never declares them, so a schema with
|
||||||
|
#: ``"additionalProperties": false`` — 37 of the 42 published ones — reports
|
||||||
|
#: them as violations and the plugin gets flagged degraded in the web UI for
|
||||||
|
#: using a documented core feature.
|
||||||
|
#:
|
||||||
|
#: Listed explicitly rather than matched on a ``vegas_`` prefix, because
|
||||||
|
#: ``vegas_mode`` is the opposite case: plugins *do* declare that one, and a
|
||||||
|
#: prefix rule would silently stop validating it.
|
||||||
|
#:
|
||||||
|
#: Read by: ``vegas_mode/plugin_adapter.py`` (``vegas_width_pct``,
|
||||||
|
#: ``vegas_overflow``) and ``base_plugin.py`` (``vegas_max_width_screens``).
|
||||||
|
CORE_OWNED_CONFIG_KEYS = frozenset({
|
||||||
|
'vegas_width_pct',
|
||||||
|
'vegas_overflow',
|
||||||
|
'vegas_max_width_screens',
|
||||||
|
})
|
||||||
|
|
||||||
|
def _strip_core_owned_keys(self, config: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""A shallow copy of ``config`` without the core's own tuning keys.
|
||||||
|
|
||||||
|
Only the top level is touched, and only when such a key is present, so
|
||||||
|
the common case allocates nothing extra.
|
||||||
|
"""
|
||||||
|
if not isinstance(config, dict):
|
||||||
|
return config
|
||||||
|
if not self.CORE_OWNED_CONFIG_KEYS.intersection(config):
|
||||||
|
return config
|
||||||
|
return {k: v for k, v in config.items()
|
||||||
|
if k not in self.CORE_OWNED_CONFIG_KEYS}
|
||||||
|
|
||||||
def _validate_config_schema_soft(self, plugin_id: str, config: Dict[str, Any]) -> None:
|
def _validate_config_schema_soft(self, plugin_id: str, config: Dict[str, Any]) -> None:
|
||||||
"""Validate a plugin's config against its JSON schema — warn/degrade only.
|
"""Validate a plugin's config against its JSON schema — warn/degrade only.
|
||||||
|
|
||||||
@@ -419,7 +450,7 @@ class PluginManager:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
is_valid, errors = self.schema_manager.validate_config_against_schema(
|
is_valid, errors = self.schema_manager.validate_config_against_schema(
|
||||||
config, schema, plugin_id
|
self._strip_core_owned_keys(config), schema, plugin_id
|
||||||
)
|
)
|
||||||
except Exception as e: # pragma: no cover - defensive
|
except Exception as e: # pragma: no cover - defensive
|
||||||
# Validation machinery itself failed — do not penalise the plugin.
|
# Validation machinery itself failed — do not penalise the plugin.
|
||||||
|
|||||||
@@ -149,18 +149,27 @@ class PluginStoreManager:
|
|||||||
# loser can end up renaming the winner's in-progress install aside
|
# loser can end up renaming the winner's in-progress install aside
|
||||||
# mid-download, stealing its own rollback safety net. Keyed by
|
# mid-download, stealing its own rollback safety net. Keyed by
|
||||||
# plugin_id so unrelated plugins still update concurrently.
|
# plugin_id so unrelated plugins still update concurrently.
|
||||||
self._reinstall_locks: Dict[str, threading.Lock] = {}
|
# Reentrant: install_plugin takes this lock, and _reinstall_with_rollback
|
||||||
|
# holds it across its call to install_plugin. A plain Lock would
|
||||||
|
# self-deadlock on that nesting.
|
||||||
|
self._reinstall_locks: Dict[str, "threading.RLock"] = {}
|
||||||
self._reinstall_locks_guard = threading.Lock()
|
self._reinstall_locks_guard = threading.Lock()
|
||||||
|
|
||||||
# Ensure plugins directory exists
|
# Ensure plugins directory exists
|
||||||
self.plugins_dir.mkdir(exist_ok=True)
|
self.plugins_dir.mkdir(exist_ok=True)
|
||||||
|
|
||||||
def _get_reinstall_lock(self, plugin_id: str) -> threading.Lock:
|
def _get_reinstall_lock(self, plugin_id: str):
|
||||||
"""Lazily create (or fetch) the per-plugin reinstall lock."""
|
"""Lazily create (or fetch) the per-plugin reinstall lock.
|
||||||
|
|
||||||
|
Reentrant by necessity: `install_plugin` acquires it to protect its
|
||||||
|
set-aside/restore, and `_reinstall_with_rollback` holds it across its
|
||||||
|
own call to `install_plugin`. With a plain `Lock` that nesting
|
||||||
|
deadlocks the request thread.
|
||||||
|
"""
|
||||||
with self._reinstall_locks_guard:
|
with self._reinstall_locks_guard:
|
||||||
lock = self._reinstall_locks.get(plugin_id)
|
lock = self._reinstall_locks.get(plugin_id)
|
||||||
if lock is None:
|
if lock is None:
|
||||||
lock = threading.Lock()
|
lock = threading.RLock()
|
||||||
self._reinstall_locks[plugin_id] = lock
|
self._reinstall_locks[plugin_id] = lock
|
||||||
return lock
|
return lock
|
||||||
|
|
||||||
@@ -1192,6 +1201,90 @@ class PluginStoreManager:
|
|||||||
return next((p for p in plugins if p.get('id') == plugin_id), None)
|
return next((p for p in plugins if p.get('id') == plugin_id), None)
|
||||||
|
|
||||||
def install_plugin(self, plugin_id: str, branch: Optional[str] = None) -> bool:
|
def install_plugin(self, plugin_id: str, branch: Optional[str] = None) -> bool:
|
||||||
|
"""Install a plugin, keeping any existing install until the new one is
|
||||||
|
known good.
|
||||||
|
|
||||||
|
`_install_plugin_impl` deletes the existing directory *before*
|
||||||
|
downloading, so every failure after that point — a dropped connection, a
|
||||||
|
malformed manifest, or the compatibility gate refusing the new version —
|
||||||
|
left the user with no plugin at all. `_reinstall_with_rollback` gives the
|
||||||
|
*update* path exactly this protection; a direct install had none, and the
|
||||||
|
compatibility gate added a new way to reach it.
|
||||||
|
|
||||||
|
Pass-through when nothing is installed, and when called from
|
||||||
|
`_reinstall_with_rollback`, which has already moved the old copy aside.
|
||||||
|
|
||||||
|
The aside name embeds '.standalone-backup-' so plugin discovery
|
||||||
|
(`plugin_manager._scan_directory_for_plugins`) skips it even though it
|
||||||
|
still holds a manifest.json.
|
||||||
|
|
||||||
|
Held under the per-plugin reinstall lock for the same reason
|
||||||
|
`_reinstall_with_rollback` is: the web UI runs Flask with
|
||||||
|
threaded=True, so a double-clicked Install button gives two threads the
|
||||||
|
same plugin_id. Interleaved, one thread's restore would delete the
|
||||||
|
other's freshly installed copy. The lock is reentrant because the
|
||||||
|
rollback path already holds it when it calls in here.
|
||||||
|
"""
|
||||||
|
with self._get_reinstall_lock(plugin_id):
|
||||||
|
plugin_path = self.plugins_dir / plugin_id
|
||||||
|
if not plugin_path.exists():
|
||||||
|
return self._install_plugin_impl(plugin_id, branch)
|
||||||
|
|
||||||
|
backup_path = plugin_path.with_name(
|
||||||
|
f"{plugin_path.name}.standalone-backup-preinstall")
|
||||||
|
if backup_path.exists() and not self._safe_remove_directory(backup_path):
|
||||||
|
# Can't stage a safety net. Better to attempt the install than
|
||||||
|
# to refuse outright, which is what callers got before this
|
||||||
|
# existed.
|
||||||
|
self.logger.warning(
|
||||||
|
"Could not clear stale pre-install backup for %s at %s; "
|
||||||
|
"installing without a rollback net", plugin_id, backup_path)
|
||||||
|
return self._install_plugin_impl(plugin_id, branch)
|
||||||
|
|
||||||
|
try:
|
||||||
|
plugin_path.rename(backup_path)
|
||||||
|
except OSError as e:
|
||||||
|
self.logger.warning(
|
||||||
|
"Could not set aside existing install of %s (%s); "
|
||||||
|
"installing without a rollback net", plugin_id, e)
|
||||||
|
return self._install_plugin_impl(plugin_id, branch)
|
||||||
|
|
||||||
|
try:
|
||||||
|
installed = self._install_plugin_impl(plugin_id, branch)
|
||||||
|
except Exception:
|
||||||
|
self._restore_preinstall_backup(plugin_id, plugin_path, backup_path)
|
||||||
|
raise
|
||||||
|
|
||||||
|
if installed:
|
||||||
|
if not self._safe_remove_directory(backup_path):
|
||||||
|
self.logger.warning(
|
||||||
|
"Install of %s succeeded but the previous copy at %s "
|
||||||
|
"could not be removed; it will be cleared on the next "
|
||||||
|
"install", plugin_id, backup_path)
|
||||||
|
return True
|
||||||
|
|
||||||
|
self._restore_preinstall_backup(plugin_id, plugin_path, backup_path)
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _restore_preinstall_backup(
|
||||||
|
self, plugin_id: str, plugin_path: Path, backup_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""Put the previous install back after a failed (re)install."""
|
||||||
|
self.logger.error(
|
||||||
|
"Install of %s failed; restoring the previous version", plugin_id)
|
||||||
|
try:
|
||||||
|
if plugin_path.exists():
|
||||||
|
# Partial download debris from the failed install.
|
||||||
|
self._safe_remove_directory(plugin_path)
|
||||||
|
backup_path.rename(plugin_path)
|
||||||
|
self.logger.info("Restored previous install of %s", plugin_id)
|
||||||
|
except OSError as e:
|
||||||
|
self.logger.error(
|
||||||
|
"CRITICAL: could not restore %s from %s: %s. The previous "
|
||||||
|
"install is preserved there — rename it back manually.",
|
||||||
|
plugin_id, backup_path, e)
|
||||||
|
|
||||||
|
def _install_plugin_impl(self, plugin_id: str, branch: Optional[str] = None) -> bool:
|
||||||
"""
|
"""
|
||||||
Install a plugin from the official registry. Always installs the latest commit
|
Install a plugin from the official registry. Always installs the latest commit
|
||||||
from the repository's default branch (or specified branch).
|
from the repository's default branch (or specified branch).
|
||||||
@@ -1333,6 +1426,26 @@ class PluginStoreManager:
|
|||||||
self._safe_remove_directory(plugin_path)
|
self._safe_remove_directory(plugin_path)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
# Refuse a plugin that needs a newer core than this one. The
|
||||||
|
# registry carries no compatibility field, so the floor is only
|
||||||
|
# knowable once the files are down — checking here, before
|
||||||
|
# dependency installation, is the earliest possible point.
|
||||||
|
#
|
||||||
|
# Refusing costs the user nothing: on an update this returns
|
||||||
|
# False and _reinstall_with_rollback restores the version they
|
||||||
|
# already had. Allowing it costs them a plugin that raises
|
||||||
|
# ModuleNotFoundError at load and is reported only as one line
|
||||||
|
# in the journal. See docs/SPORTS_UNIFICATION.md (phase B4/B6).
|
||||||
|
from src import __version__ as core_version
|
||||||
|
from src.plugin_system import compatibility
|
||||||
|
|
||||||
|
compatible, reason = compatibility.check(manifest, core_version)
|
||||||
|
if not compatible:
|
||||||
|
self.logger.error(
|
||||||
|
"Refusing to install %s: %s", plugin_id, reason)
|
||||||
|
self._safe_remove_directory(plugin_path)
|
||||||
|
return False
|
||||||
|
|
||||||
if 'entry_point' not in manifest:
|
if 'entry_point' not in manifest:
|
||||||
manifest['entry_point'] = 'manager.py'
|
manifest['entry_point'] = 'manager.py'
|
||||||
manifest_modified = True
|
manifest_modified = True
|
||||||
|
|||||||
@@ -0,0 +1,98 @@
|
|||||||
|
"""The core's own tuning keys must not make a plugin look broken.
|
||||||
|
|
||||||
|
`vegas_width_pct`, `vegas_overflow` and `vegas_max_width_screens` are read by
|
||||||
|
the *core* out of each plugin's config block — `vegas_mode/plugin_adapter.py`
|
||||||
|
and `base_plugin.py`. No plugin declares them, and 37 of the 42 published
|
||||||
|
config schemas set `"additionalProperties": false`, so schema validation
|
||||||
|
reported them as violations.
|
||||||
|
|
||||||
|
That is not just log noise: `_validate_config_schema_soft` sets `degraded` in
|
||||||
|
the health tracker, which the web UI surfaces. Measured on a real device, **9
|
||||||
|
of 27 installed plugins** were flagged degraded purely for using a documented
|
||||||
|
core feature — including `baseball-scoreboard` and `f1-scoreboard`.
|
||||||
|
|
||||||
|
The fix strips those keys before validating. It deliberately does *not* match
|
||||||
|
on a `vegas_` prefix: `vegas_mode` is plugin-owned and declared in schemas, and
|
||||||
|
a prefix rule would silently stop validating it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.plugin_system.plugin_manager import PluginManager
|
||||||
|
|
||||||
|
|
||||||
|
STRICT_SCHEMA = {
|
||||||
|
"type": "object",
|
||||||
|
"additionalProperties": False,
|
||||||
|
"properties": {
|
||||||
|
"enabled": {"type": "boolean"},
|
||||||
|
"vegas_mode": {"type": "string"}, # plugin-owned, must stay validated
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def manager():
|
||||||
|
mgr = PluginManager.__new__(PluginManager) # skip the heavy constructor
|
||||||
|
mgr.logger = MagicMock()
|
||||||
|
mgr.schema_manager = MagicMock()
|
||||||
|
mgr._set_degraded_safe = MagicMock()
|
||||||
|
return mgr
|
||||||
|
|
||||||
|
|
||||||
|
class TestStripCoreOwnedKeys:
|
||||||
|
def test_removes_every_core_owned_key(self, manager):
|
||||||
|
cfg = {"enabled": True, "vegas_width_pct": 50,
|
||||||
|
"vegas_overflow": "wrap", "vegas_max_width_screens": 2}
|
||||||
|
assert manager._strip_core_owned_keys(cfg) == {"enabled": True}
|
||||||
|
|
||||||
|
def test_leaves_plugin_owned_vegas_mode_alone(self, manager):
|
||||||
|
"""A prefix rule would have eaten this one."""
|
||||||
|
cfg = {"enabled": True, "vegas_mode": "scroll"}
|
||||||
|
assert manager._strip_core_owned_keys(cfg) == cfg
|
||||||
|
|
||||||
|
def test_returns_the_same_object_when_nothing_to_strip(self, manager):
|
||||||
|
cfg = {"enabled": True}
|
||||||
|
assert manager._strip_core_owned_keys(cfg) is cfg
|
||||||
|
|
||||||
|
def test_does_not_mutate_the_caller_config(self, manager):
|
||||||
|
cfg = {"enabled": True, "vegas_width_pct": 50}
|
||||||
|
manager._strip_core_owned_keys(cfg)
|
||||||
|
assert "vegas_width_pct" in cfg, "the live plugin config was mutated"
|
||||||
|
|
||||||
|
def test_tolerates_a_non_dict(self, manager):
|
||||||
|
assert manager._strip_core_owned_keys(None) is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestSoftValidation:
|
||||||
|
def _validate_with(self, manager, config, valid=True, errors=()):
|
||||||
|
manager.schema_manager.load_schema.return_value = STRICT_SCHEMA
|
||||||
|
manager.schema_manager.validate_config_against_schema.return_value = (
|
||||||
|
valid, list(errors))
|
||||||
|
manager._validate_config_schema_soft("baseball-scoreboard", config)
|
||||||
|
return manager.schema_manager.validate_config_against_schema.call_args
|
||||||
|
|
||||||
|
def test_core_keys_never_reach_the_validator(self, manager):
|
||||||
|
"""The regression: these keys reaching a strict schema is what flagged
|
||||||
|
9 of 27 plugins degraded."""
|
||||||
|
args = self._validate_with(
|
||||||
|
manager, {"enabled": True, "vegas_width_pct": 50})
|
||||||
|
validated = args[0][0]
|
||||||
|
assert "vegas_width_pct" not in validated
|
||||||
|
assert validated == {"enabled": True}
|
||||||
|
|
||||||
|
def test_plugin_owned_keys_still_reach_the_validator(self, manager):
|
||||||
|
args = self._validate_with(
|
||||||
|
manager, {"enabled": True, "vegas_mode": "scroll"})
|
||||||
|
assert args[0][0]["vegas_mode"] == "scroll"
|
||||||
|
|
||||||
|
def test_a_genuine_violation_is_still_reported(self, manager):
|
||||||
|
"""Stripping core keys must not turn the check into a no-op."""
|
||||||
|
self._validate_with(
|
||||||
|
manager, {"enabled": True, "typo_key": 1},
|
||||||
|
valid=False, errors=["Field root: 'typo_key' was unexpected"])
|
||||||
|
manager._set_degraded_safe.assert_called()
|
||||||
|
reason = manager._set_degraded_safe.call_args[0][1]
|
||||||
|
assert reason and "typo_key" in reason
|
||||||
@@ -0,0 +1,209 @@
|
|||||||
|
"""
|
||||||
|
Tests for scripts/install/lib_lowmem.sh, the installer's low-memory helpers.
|
||||||
|
|
||||||
|
Background: the rgbmatrix build compiles ~45 C++ translation units, two of them
|
||||||
|
Cython-generated. Upstream's pyproject.toml sets no [tool.scikit-build] options,
|
||||||
|
so scikit-build-core drives Ninja at its default of nproc+2 jobs -- six
|
||||||
|
concurrent cc1plus on a 4-core Pi. On 512MB and 1GB models the OOM killer reaps
|
||||||
|
the compiler and pip reports only "Failed building wheel for rgbmatrix", which
|
||||||
|
the installer used to misreport as a missing-build-tools problem.
|
||||||
|
|
||||||
|
These cover the pure sizing/detection functions. The swap-management functions
|
||||||
|
need root and mutate the system, so they are exercised manually instead.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
LIB = Path(__file__).resolve().parent.parent / "scripts" / "install" / "lib_lowmem.sh"
|
||||||
|
|
||||||
|
|
||||||
|
def run_lib(snippet: str, env: dict | None = None) -> subprocess.CompletedProcess:
|
||||||
|
"""Source the helper library and run a snippet against it."""
|
||||||
|
script = f". {LIB}\n{snippet}"
|
||||||
|
return subprocess.run(
|
||||||
|
["bash", "-c", script],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
env={"PATH": "/usr/bin:/bin:/usr/sbin:/sbin", **(env or {})},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def call(fn: str, *args: object, env: dict | None = None) -> str:
|
||||||
|
joined = " ".join(str(a) for a in args)
|
||||||
|
result = run_lib(f"{fn} {joined}", env=env)
|
||||||
|
assert result.returncode == 0, f"{fn} failed: {result.stderr}"
|
||||||
|
return result.stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
class TestLibraryLoads:
|
||||||
|
def test_library_exists_and_is_syntactically_valid(self):
|
||||||
|
assert LIB.is_file(), f"{LIB} is missing"
|
||||||
|
result = subprocess.run(["bash", "-n", str(LIB)], capture_output=True, text=True)
|
||||||
|
assert result.returncode == 0, result.stderr
|
||||||
|
|
||||||
|
def test_sourcing_is_safe_under_strict_mode(self):
|
||||||
|
# first_time_install.sh runs under `set -Eeuo pipefail` with an ERR
|
||||||
|
# trap, so sourcing must not trip either.
|
||||||
|
result = run_lib("set -Eeuo pipefail\ntrap 'exit 99' ERR\necho ok")
|
||||||
|
assert result.returncode == 0, result.stderr
|
||||||
|
assert "ok" in result.stdout
|
||||||
|
|
||||||
|
|
||||||
|
class TestBuildJobs:
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"ram_mb,cores,expected",
|
||||||
|
[
|
||||||
|
(512, 4, 1), # Pi Zero 2 W - must serialize
|
||||||
|
(1024, 4, 1), # Pi 3B/3B+ - the device from the bug report
|
||||||
|
(2048, 4, 2),
|
||||||
|
(4096, 4, 4), # core-capped
|
||||||
|
(8192, 4, 4), # core-capped
|
||||||
|
(2048, 1, 1), # single-core machine
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_jobs_scale_with_ram_and_cap_at_cores(self, ram_mb, cores, expected):
|
||||||
|
assert call("lm_build_jobs", ram_mb, cores) == str(expected)
|
||||||
|
|
||||||
|
def test_never_returns_zero_jobs(self):
|
||||||
|
assert call("lm_build_jobs", 0, 4) == "1"
|
||||||
|
|
||||||
|
def test_treats_zero_cores_as_one(self):
|
||||||
|
assert call("lm_build_jobs", 8192, 0) == "1"
|
||||||
|
|
||||||
|
|
||||||
|
class TestSwapSizing:
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"ram_mb,existing_swap_mb,expected",
|
||||||
|
[
|
||||||
|
(512, 0, 2048), # capped at LM_SWAP_MAX_MB
|
||||||
|
(1024, 0, 2048), # matches the workaround the reporter found
|
||||||
|
(2048, 0, 1024),
|
||||||
|
(2048, 1024, 0), # existing swap already covers it
|
||||||
|
(4096, 0, 0), # untouched on machines that already work
|
||||||
|
(8192, 0, 0),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_swap_target_scales_with_ram(self, ram_mb, existing_swap_mb, expected):
|
||||||
|
assert call("lm_swap_needed_mb", ram_mb, existing_swap_mb) == str(expected)
|
||||||
|
|
||||||
|
def test_result_is_a_multiple_of_256mb(self):
|
||||||
|
# 3072 - 900 = 2172, which must round up rather than produce an odd size.
|
||||||
|
assert int(call("lm_swap_needed_mb", 900, 0)) % 256 == 0
|
||||||
|
|
||||||
|
|
||||||
|
class TestSwapDetection:
|
||||||
|
def test_zram_swap_is_excluded(self, tmp_path):
|
||||||
|
# zram swap is compressed RAM: counting it would let a zram-enabled
|
||||||
|
# image skip provisioning and then OOM exactly as before.
|
||||||
|
swaps = tmp_path / "swaps"
|
||||||
|
swaps.write_text(
|
||||||
|
"Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority\n"
|
||||||
|
"/dev/zram0 partition\t1048572\t\t0\t\t100\n"
|
||||||
|
"/var/swap file\t\t524284\t\t0\t\t-2\n"
|
||||||
|
)
|
||||||
|
assert call("lm_total_swap_mb", env={"LM_SWAPS": str(swaps)}) == "511"
|
||||||
|
|
||||||
|
def test_zram_only_system_reports_no_usable_swap(self, tmp_path):
|
||||||
|
swaps = tmp_path / "swaps"
|
||||||
|
swaps.write_text(
|
||||||
|
"Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority\n"
|
||||||
|
"/dev/zram0 partition\t1048572\t\t0\t\t100\n"
|
||||||
|
)
|
||||||
|
assert call("lm_total_swap_mb", env={"LM_SWAPS": str(swaps)}) == "0"
|
||||||
|
|
||||||
|
def test_ram_is_read_from_meminfo(self, tmp_path):
|
||||||
|
meminfo = tmp_path / "meminfo"
|
||||||
|
# A real Pi 3B+ reports this; 948204/1024 truncates to 925.
|
||||||
|
meminfo.write_text("MemTotal: 948204 kB\nMemFree: 123456 kB\n")
|
||||||
|
assert call("lm_total_ram_mb", env={"LM_MEMINFO": str(meminfo)}) == "925"
|
||||||
|
|
||||||
|
def test_missing_files_report_zero_rather_than_failing(self, tmp_path):
|
||||||
|
missing = str(tmp_path / "nope")
|
||||||
|
assert call("lm_total_ram_mb", env={"LM_MEMINFO": missing}) == "0"
|
||||||
|
assert call("lm_total_swap_mb", env={"LM_SWAPS": missing}) == "0"
|
||||||
|
|
||||||
|
|
||||||
|
class TestOomDetection:
|
||||||
|
"""The regression tests for the misdiagnosis in the bug report."""
|
||||||
|
|
||||||
|
def _check(self, tmp_path, build_log: str, kernel_log: str = "") -> bool:
|
||||||
|
build_file = tmp_path / "build.log"
|
||||||
|
build_file.write_text(build_log)
|
||||||
|
kernel_file = tmp_path / "kernel.log"
|
||||||
|
kernel_file.write_text(kernel_log)
|
||||||
|
result = run_lib(
|
||||||
|
f"lm_build_failed_on_oom {build_file}",
|
||||||
|
env={"LM_KERNEL_LOG_FILE": str(kernel_file)},
|
||||||
|
)
|
||||||
|
return result.returncode == 0
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"line",
|
||||||
|
[
|
||||||
|
"c++: fatal error: Killed signal terminated program cc1plus",
|
||||||
|
"cc1plus: out of memory allocating 65536 bytes",
|
||||||
|
"virtual memory exhausted: Cannot allocate memory",
|
||||||
|
"error: command '/usr/bin/c++' died with signal 9",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_detects_compiler_reported_memory_failures(self, tmp_path, line):
|
||||||
|
log = f"[15/45] Building CXX object core.cpp.o\n{line}\nninja: build stopped.\n"
|
||||||
|
assert self._check(tmp_path, log) is True
|
||||||
|
|
||||||
|
def test_detects_oom_visible_only_in_the_kernel_log(self, tmp_path):
|
||||||
|
# The OOM killer writes nothing to the build's stdout. This silence is
|
||||||
|
# precisely why the old handler blamed missing build tools.
|
||||||
|
build_log = (
|
||||||
|
"[15/45] Building CXX object core.cpp.o\n"
|
||||||
|
"ninja: build stopped: subcommand failed.\n"
|
||||||
|
"ERROR: Failed building wheel for rgbmatrix\n"
|
||||||
|
)
|
||||||
|
kernel_log = (
|
||||||
|
"[12345.6] Out of memory: Killed process 4242 (cc1plus) "
|
||||||
|
"total-vm:812345kB, anon-rss:764000kB\n"
|
||||||
|
)
|
||||||
|
assert self._check(tmp_path, build_log, kernel_log) is True
|
||||||
|
|
||||||
|
def test_does_not_flag_a_genuine_missing_build_tool(self, tmp_path):
|
||||||
|
build_log = (
|
||||||
|
"CMake Error at CMakeLists.txt:12 (find_package):\n"
|
||||||
|
" Could NOT find Python (missing: Development.Module)\n"
|
||||||
|
"fatal error: Python.h: No such file or directory\n"
|
||||||
|
"ERROR: Failed building wheel for rgbmatrix\n"
|
||||||
|
)
|
||||||
|
assert self._check(tmp_path, build_log) is False
|
||||||
|
|
||||||
|
def test_does_not_flag_a_network_failure(self, tmp_path):
|
||||||
|
build_log = (
|
||||||
|
"WARNING: Retrying after connection broken by 'NewConnectionError'\n"
|
||||||
|
"ERROR: Could not install packages due to an OSError\n"
|
||||||
|
)
|
||||||
|
assert self._check(tmp_path, build_log) is False
|
||||||
|
|
||||||
|
def test_handles_a_missing_build_log(self, tmp_path):
|
||||||
|
kernel_file = tmp_path / "kernel.log"
|
||||||
|
kernel_file.write_text("")
|
||||||
|
result = run_lib(
|
||||||
|
f"lm_build_failed_on_oom {tmp_path / 'absent.log'}",
|
||||||
|
env={"LM_KERNEL_LOG_FILE": str(kernel_file)},
|
||||||
|
)
|
||||||
|
assert result.returncode == 1
|
||||||
|
|
||||||
|
|
||||||
|
class TestDiskBackedTmpdir:
|
||||||
|
def test_returns_nothing_when_tmpdir_is_already_disk_backed(self, tmp_path):
|
||||||
|
# tmp_path is on the regular filesystem, so the default must be kept.
|
||||||
|
assert call("lm_disk_backed_tmpdir", env={"TMPDIR": str(tmp_path)}) == ""
|
||||||
|
|
||||||
|
def test_redirects_away_from_a_memory_backed_tmpdir(self):
|
||||||
|
# Debian 13 mounts /tmp as tmpfs, which would otherwise hold the whole
|
||||||
|
# C++ build tree in RAM alongside the compiler.
|
||||||
|
shm = Path("/dev/shm")
|
||||||
|
if not shm.is_dir():
|
||||||
|
pytest.skip("/dev/shm not available")
|
||||||
|
result = run_lib("lm_disk_backed_tmpdir", env={"TMPDIR": str(shm)})
|
||||||
|
assert result.returncode == 0
|
||||||
|
assert result.stdout.strip() in ("", "/var/tmp")
|
||||||
@@ -0,0 +1,222 @@
|
|||||||
|
"""A failed (re)install must not destroy the working plugin it replaced.
|
||||||
|
|
||||||
|
`_install_plugin_impl` deletes the existing plugin directory *before* it
|
||||||
|
downloads anything, so any failure after that point used to leave the user with
|
||||||
|
nothing. The update path was protected — `_reinstall_with_rollback` renames the
|
||||||
|
old copy aside first — but a direct `install_plugin` was not, and the
|
||||||
|
compatibility gate added a new way to fail late: a plugin whose declared floor
|
||||||
|
exceeds the running core is now refused *after* the old copy is already gone.
|
||||||
|
|
||||||
|
Concretely, without the wrapper: a user on core 3.1.0 with a working
|
||||||
|
hockey-scoreboard clicks Install; the new manifest floors at 3.2.0; the gate
|
||||||
|
refuses; the plugin they had is deleted. Floors are hand-written and can be
|
||||||
|
over-declared, so this could remove a plugin that was working fine.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.plugin_system.store_manager import PluginStoreManager
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def store(tmp_path):
|
||||||
|
plugins_dir = tmp_path / "plugin-repos"
|
||||||
|
plugins_dir.mkdir()
|
||||||
|
mgr = PluginStoreManager(plugins_dir=str(plugins_dir))
|
||||||
|
mgr.logger = MagicMock()
|
||||||
|
return mgr, plugins_dir
|
||||||
|
|
||||||
|
|
||||||
|
def _existing_install(plugins_dir: Path, plugin_id: str, marker: str) -> Path:
|
||||||
|
path = plugins_dir / plugin_id
|
||||||
|
path.mkdir(parents=True)
|
||||||
|
(path / "manifest.json").write_text(
|
||||||
|
json.dumps({"id": plugin_id, "name": plugin_id, "class_name": "P",
|
||||||
|
"display_modes": ["a"], "version": "1.0.0"}),
|
||||||
|
encoding="utf-8")
|
||||||
|
(path / "marker.txt").write_text(marker, encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
class TestFailedInstallPreservesPrevious:
|
||||||
|
def test_failed_install_restores_the_old_copy(self, store, monkeypatch):
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
path = _existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
|
||||||
|
monkeypatch.setattr(mgr, "_install_plugin_impl", lambda *a, **k: False)
|
||||||
|
|
||||||
|
assert mgr.install_plugin("hockey-scoreboard") is False
|
||||||
|
assert path.exists(), "the previous install must be restored"
|
||||||
|
assert (path / "marker.txt").read_text() == "the-original"
|
||||||
|
|
||||||
|
def test_raising_install_restores_and_reraises(self, store, monkeypatch):
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
path = _existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
|
||||||
|
def boom(*a, **k):
|
||||||
|
raise RuntimeError("network died mid-install")
|
||||||
|
|
||||||
|
monkeypatch.setattr(mgr, "_install_plugin_impl", boom)
|
||||||
|
|
||||||
|
with pytest.raises(RuntimeError):
|
||||||
|
mgr.install_plugin("hockey-scoreboard")
|
||||||
|
assert path.exists()
|
||||||
|
assert (path / "marker.txt").read_text() == "the-original"
|
||||||
|
|
||||||
|
def test_successful_install_clears_the_backup(self, store, monkeypatch):
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
_existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
|
||||||
|
def succeed(plugin_id, branch=None):
|
||||||
|
_existing_install(plugins_dir, plugin_id, "the-new-one")
|
||||||
|
return True
|
||||||
|
|
||||||
|
monkeypatch.setattr(mgr, "_install_plugin_impl", succeed)
|
||||||
|
|
||||||
|
assert mgr.install_plugin("hockey-scoreboard") is True
|
||||||
|
assert (plugins_dir / "hockey-scoreboard" / "marker.txt").read_text() == "the-new-one"
|
||||||
|
leftovers = [p.name for p in plugins_dir.iterdir() if "backup" in p.name]
|
||||||
|
assert not leftovers, f"backup left behind: {leftovers}"
|
||||||
|
|
||||||
|
def test_backup_name_is_invisible_to_plugin_discovery(self, store, monkeypatch):
|
||||||
|
"""A backup that discovery can see becomes a duplicate plugin entry;
|
||||||
|
the marker '.standalone-backup-' is what makes it skip."""
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
_existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
|
||||||
|
seen = {}
|
||||||
|
|
||||||
|
def capture(plugin_id, branch=None):
|
||||||
|
seen["dirs"] = sorted(p.name for p in plugins_dir.iterdir())
|
||||||
|
return False
|
||||||
|
|
||||||
|
monkeypatch.setattr(mgr, "_install_plugin_impl", capture)
|
||||||
|
mgr.install_plugin("hockey-scoreboard")
|
||||||
|
|
||||||
|
backups = [d for d in seen["dirs"] if d != "hockey-scoreboard"]
|
||||||
|
assert backups, "expected the old copy to be set aside during install"
|
||||||
|
for name in backups:
|
||||||
|
assert ".standalone-backup-" in name, (
|
||||||
|
f"{name} would be picked up by "
|
||||||
|
"plugin_manager._scan_directory_for_plugins as a real plugin")
|
||||||
|
|
||||||
|
def test_fresh_install_is_a_pass_through(self, store, monkeypatch):
|
||||||
|
"""Nothing installed means nothing to protect; don't create stray dirs."""
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
calls = []
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mgr, "_install_plugin_impl",
|
||||||
|
lambda *a, **k: calls.append(a) or True)
|
||||||
|
|
||||||
|
assert mgr.install_plugin("brand-new") is True
|
||||||
|
assert calls, "the implementation must still be called"
|
||||||
|
assert list(plugins_dir.iterdir()) == []
|
||||||
|
|
||||||
|
def test_stale_backup_from_a_crash_does_not_block(self, store, monkeypatch):
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
_existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
stale = plugins_dir / "hockey-scoreboard.standalone-backup-preinstall"
|
||||||
|
stale.mkdir()
|
||||||
|
(stale / "junk.txt").write_text("from a previous crash", encoding="utf-8")
|
||||||
|
|
||||||
|
monkeypatch.setattr(mgr, "_install_plugin_impl", lambda *a, **k: False)
|
||||||
|
|
||||||
|
assert mgr.install_plugin("hockey-scoreboard") is False
|
||||||
|
assert (plugins_dir / "hockey-scoreboard" / "marker.txt").read_text() == "the-original"
|
||||||
|
|
||||||
|
|
||||||
|
class TestUpdatePathStillWorks:
|
||||||
|
def test_reinstall_with_rollback_is_not_double_wrapped(self, store, monkeypatch):
|
||||||
|
"""_reinstall_with_rollback moves the plugin aside itself, so by the
|
||||||
|
time install_plugin runs there is nothing at the original path and the
|
||||||
|
wrapper must be a pass-through rather than staging a second backup."""
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
path = _existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
|
||||||
|
observed = {}
|
||||||
|
|
||||||
|
def impl(plugin_id, branch=None):
|
||||||
|
observed["dirs"] = sorted(p.name for p in plugins_dir.iterdir())
|
||||||
|
return False
|
||||||
|
|
||||||
|
monkeypatch.setattr(mgr, "_install_plugin_impl", impl)
|
||||||
|
|
||||||
|
assert mgr._reinstall_with_rollback("hockey-scoreboard", path) is False
|
||||||
|
# Exactly one aside directory existed during the attempt — rollback's.
|
||||||
|
assert observed["dirs"] == ["hockey-scoreboard.standalone-backup-migrating"]
|
||||||
|
# And the user still has their plugin.
|
||||||
|
assert (plugins_dir / "hockey-scoreboard" / "marker.txt").read_text() == "the-original"
|
||||||
|
|
||||||
|
|
||||||
|
class TestConcurrency:
|
||||||
|
"""The web UI runs Flask threaded, so a double-clicked Install button puts
|
||||||
|
two threads on the same plugin_id. `_reinstall_with_rollback` already
|
||||||
|
guarded against this; the install wrapper has to as well, or one thread's
|
||||||
|
restore deletes the other's freshly installed copy."""
|
||||||
|
|
||||||
|
def test_rollback_calling_install_does_not_deadlock(self, store, monkeypatch):
|
||||||
|
"""The rollback path holds the per-plugin lock across its call to
|
||||||
|
install_plugin. A non-reentrant lock would hang the request thread
|
||||||
|
forever — this test would time out rather than fail."""
|
||||||
|
import threading
|
||||||
|
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
path = _existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mgr, "_install_plugin_impl",
|
||||||
|
lambda pid, branch=None: bool(_existing_install(plugins_dir, pid, "new")))
|
||||||
|
|
||||||
|
done = threading.Event()
|
||||||
|
result = {}
|
||||||
|
|
||||||
|
def run():
|
||||||
|
result["ok"] = mgr._reinstall_with_rollback("hockey-scoreboard", path)
|
||||||
|
done.set()
|
||||||
|
|
||||||
|
t = threading.Thread(target=run, daemon=True)
|
||||||
|
t.start()
|
||||||
|
assert done.wait(timeout=10), (
|
||||||
|
"install_plugin deadlocked when called from _reinstall_with_rollback "
|
||||||
|
"— the per-plugin lock must be reentrant"
|
||||||
|
)
|
||||||
|
assert result["ok"] is True
|
||||||
|
|
||||||
|
def test_concurrent_installs_serialize(self, store, monkeypatch):
|
||||||
|
"""Two threads installing the same plugin must not interleave their
|
||||||
|
set-aside/restore, and the survivor must be a complete install."""
|
||||||
|
import threading
|
||||||
|
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
_existing_install(plugins_dir, "hockey-scoreboard", "the-original")
|
||||||
|
|
||||||
|
in_flight = []
|
||||||
|
overlap = []
|
||||||
|
|
||||||
|
def slow_impl(plugin_id, branch=None):
|
||||||
|
in_flight.append(1)
|
||||||
|
if len(in_flight) > 1:
|
||||||
|
overlap.append(1)
|
||||||
|
threading.Event().wait(0.05)
|
||||||
|
_existing_install(plugins_dir, plugin_id, "installed")
|
||||||
|
in_flight.pop()
|
||||||
|
return True
|
||||||
|
|
||||||
|
monkeypatch.setattr(mgr, "_install_plugin_impl", slow_impl)
|
||||||
|
|
||||||
|
threads = [threading.Thread(target=mgr.install_plugin,
|
||||||
|
args=("hockey-scoreboard",), daemon=True)
|
||||||
|
for _ in range(2)]
|
||||||
|
for t in threads:
|
||||||
|
t.start()
|
||||||
|
for t in threads:
|
||||||
|
t.join(timeout=10)
|
||||||
|
assert not t.is_alive(), "concurrent install hung"
|
||||||
|
|
||||||
|
assert not overlap, "two installs of the same plugin ran concurrently"
|
||||||
|
assert (plugins_dir / "hockey-scoreboard" / "marker.txt").exists()
|
||||||
|
leftovers = [p.name for p in plugins_dir.iterdir() if "backup" in p.name]
|
||||||
|
assert not leftovers, f"backup left behind: {leftovers}"
|
||||||
@@ -0,0 +1,232 @@
|
|||||||
|
"""The install/update gate, and the shared compatibility rules behind it.
|
||||||
|
|
||||||
|
Before this existed, `ledmatrix_min_version` was decoration: the loader logged
|
||||||
|
an advisory warning and the store never looked at the core version at all, so a
|
||||||
|
routine store update happily delivered a plugin that could not run. Deleting a
|
||||||
|
plugin's bundled fallback under those conditions would have handed un-updated
|
||||||
|
users a scoreboard that fails to load with one line in the journal.
|
||||||
|
|
||||||
|
The rules being pinned here, in priority order:
|
||||||
|
|
||||||
|
1. Refuse only on **evidence**. Undeclared floor, unparseable version on either
|
||||||
|
side, or a core whose self-reported version is untrustworthy → allow. A
|
||||||
|
wrong refusal breaks a working install; a wrong allowance degrades to the
|
||||||
|
behavior we already had.
|
||||||
|
2. A core below `TRUSTWORTHY_FLOOR` is *unknown*, not old. The v3.1.0 release
|
||||||
|
reports `1.0.0` while nearly every manifest floors at `2.0.0`; blocking on
|
||||||
|
that number would stop those users installing anything at all.
|
||||||
|
3. The loader and the store must agree, because they read the same manifests.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from src.plugin_system import compatibility
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Floor resolution — every spelling published plugins actually use
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TestDeclaredMinVersion:
|
||||||
|
def test_top_level_min_ledmatrix_version(self):
|
||||||
|
assert compatibility.declared_min_version(
|
||||||
|
{"min_ledmatrix_version": "3.2.0"}) == "3.2.0"
|
||||||
|
|
||||||
|
def test_requires_block(self):
|
||||||
|
assert compatibility.declared_min_version(
|
||||||
|
{"requires": {"min_ledmatrix_version": "3.1.0"}}) == "3.1.0"
|
||||||
|
|
||||||
|
def test_versions_array_new_spelling(self):
|
||||||
|
assert compatibility.declared_min_version(
|
||||||
|
{"versions": [{"ledmatrix_min_version": "3.2.0"}]}) == "3.2.0"
|
||||||
|
|
||||||
|
def test_versions_array_deprecated_spelling(self):
|
||||||
|
"""Most published manifests still say `ledmatrix_min`; ignoring it
|
||||||
|
would silently exempt them from the gate."""
|
||||||
|
assert compatibility.declared_min_version(
|
||||||
|
{"versions": [{"ledmatrix_min": "2.0.0"}]}) == "2.0.0"
|
||||||
|
|
||||||
|
def test_absent(self):
|
||||||
|
assert compatibility.declared_min_version({"id": "x"}) is None
|
||||||
|
|
||||||
|
def test_requires_present_but_null(self):
|
||||||
|
assert compatibility.declared_min_version({"requires": None}) is None
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# The decision itself
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
class TestCheck:
|
||||||
|
def test_blocks_when_plugin_needs_a_newer_core(self):
|
||||||
|
ok, reason = compatibility.check(
|
||||||
|
{"name": "Hockey Scoreboard", "min_ledmatrix_version": "3.2.0"}, "3.1.0")
|
||||||
|
assert ok is False
|
||||||
|
assert "3.2.0" in reason and "3.1.0" in reason
|
||||||
|
assert "Hockey Scoreboard" in reason
|
||||||
|
|
||||||
|
def test_allows_equal_version(self):
|
||||||
|
ok, _ = compatibility.check({"min_ledmatrix_version": "3.2.0"}, "3.2.0")
|
||||||
|
assert ok is True
|
||||||
|
|
||||||
|
def test_allows_newer_core(self):
|
||||||
|
ok, _ = compatibility.check({"min_ledmatrix_version": "3.2.0"}, "4.0.0")
|
||||||
|
assert ok is True
|
||||||
|
|
||||||
|
def test_allows_when_no_floor_declared(self):
|
||||||
|
ok, reason = compatibility.check({"id": "x"}, "3.2.0")
|
||||||
|
assert ok is True and reason is None
|
||||||
|
|
||||||
|
def test_untrustworthy_core_version_allows_everything(self):
|
||||||
|
"""The v3.1.0 release reports 1.0.0. Nearly every manifest floors at
|
||||||
|
2.0.0, so blocking here would stop those users installing any plugin
|
||||||
|
at all — strictly worse than the problem being solved."""
|
||||||
|
ok, reason = compatibility.check(
|
||||||
|
{"min_ledmatrix_version": "3.2.0"}, "1.0.0")
|
||||||
|
assert ok is True and reason is None
|
||||||
|
|
||||||
|
def test_unparseable_core_version_allows(self):
|
||||||
|
ok, _ = compatibility.check({"min_ledmatrix_version": "3.2.0"}, "not-a-version")
|
||||||
|
assert ok is True
|
||||||
|
|
||||||
|
def test_unparseable_floor_allows(self):
|
||||||
|
ok, _ = compatibility.check({"min_ledmatrix_version": {"nope": 1}}, "3.2.0")
|
||||||
|
assert ok is True
|
||||||
|
|
||||||
|
def test_v_prefix_tolerated_on_both_sides(self):
|
||||||
|
ok, _ = compatibility.check({"min_ledmatrix_version": "v3.3.0"}, "v3.2.0")
|
||||||
|
assert ok is False
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("floor,core,expected_ok", [
|
||||||
|
("3.2.0", "3.2.1", True),
|
||||||
|
("3.2.1", "3.2.0", False),
|
||||||
|
("3.10.0", "3.9.0", False), # numeric compare, not lexical
|
||||||
|
("3.9.0", "3.10.0", True),
|
||||||
|
])
|
||||||
|
def test_ordering(self, floor, core, expected_ok):
|
||||||
|
ok, _ = compatibility.check({"min_ledmatrix_version": floor}, core)
|
||||||
|
assert ok is expected_ok
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# The gate in install_plugin
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _write_plugin(plugins_dir: Path, plugin_id: str, manifest: dict) -> Path:
|
||||||
|
path = plugins_dir / plugin_id
|
||||||
|
path.mkdir(parents=True)
|
||||||
|
(path / "manifest.json").write_text(json.dumps(manifest), encoding="utf-8")
|
||||||
|
(path / "manager.py").write_text("class P: pass\n", encoding="utf-8")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def store(tmp_path, monkeypatch):
|
||||||
|
"""A PluginStoreManager whose download step is stubbed to drop a plugin
|
||||||
|
directory in place, so the test exercises the post-download validation
|
||||||
|
path without touching the network."""
|
||||||
|
from src.plugin_system.store_manager import PluginStoreManager
|
||||||
|
|
||||||
|
plugins_dir = tmp_path / "plugin-repos"
|
||||||
|
plugins_dir.mkdir()
|
||||||
|
mgr = PluginStoreManager(plugins_dir=str(plugins_dir))
|
||||||
|
mgr.logger = MagicMock()
|
||||||
|
return mgr, plugins_dir
|
||||||
|
|
||||||
|
|
||||||
|
class TestInstallGate:
|
||||||
|
"""`install_plugin` is the chokepoint: `_reinstall_with_rollback` calls it,
|
||||||
|
so gating there covers updates too, and a refused update restores the
|
||||||
|
version the user already had."""
|
||||||
|
|
||||||
|
def _install_with_manifest(self, store, manifest, core_version, monkeypatch):
|
||||||
|
mgr, plugins_dir = store
|
||||||
|
plugin_id = manifest["id"]
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mgr, "get_plugin_info",
|
||||||
|
lambda *a, **k: {"repo": "https://example.invalid/r",
|
||||||
|
"plugin_path": f"plugins/{plugin_id}",
|
||||||
|
"branch": "main"})
|
||||||
|
# Stand in for the download: put the files where install_plugin expects.
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mgr, "_install_from_monorepo",
|
||||||
|
lambda *a, **k: bool(_write_plugin(plugins_dir, plugin_id, manifest)))
|
||||||
|
monkeypatch.setattr(mgr, "_install_from_monorepo_api", lambda *a, **k: False)
|
||||||
|
monkeypatch.setattr(mgr, "_install_dependencies", lambda *a, **k: True)
|
||||||
|
|
||||||
|
import src
|
||||||
|
monkeypatch.setattr(src, "__version__", core_version)
|
||||||
|
return mgr.install_plugin(plugin_id), plugins_dir / plugin_id
|
||||||
|
|
||||||
|
def test_refuses_and_leaves_nothing_behind(self, store, monkeypatch):
|
||||||
|
manifest = {
|
||||||
|
"id": "needs-newer", "name": "Needs Newer", "class_name": "P",
|
||||||
|
"display_modes": ["a"], "min_ledmatrix_version": "9.9.9",
|
||||||
|
}
|
||||||
|
ok, path = self._install_with_manifest(store, manifest, "3.2.0", monkeypatch)
|
||||||
|
|
||||||
|
assert ok is False, "install must refuse a plugin that needs a newer core"
|
||||||
|
assert not path.exists(), (
|
||||||
|
"a refused install must not leave a half-installed directory — "
|
||||||
|
"plugin discovery would pick it up and fail to load it")
|
||||||
|
|
||||||
|
def test_allows_a_compatible_plugin(self, store, monkeypatch):
|
||||||
|
manifest = {
|
||||||
|
"id": "fine", "name": "Fine", "class_name": "P",
|
||||||
|
"display_modes": ["a"], "min_ledmatrix_version": "3.0.0",
|
||||||
|
}
|
||||||
|
ok, path = self._install_with_manifest(store, manifest, "3.2.0", monkeypatch)
|
||||||
|
|
||||||
|
assert ok is True
|
||||||
|
assert (path / "manifest.json").exists()
|
||||||
|
|
||||||
|
def test_untrustworthy_core_does_not_block_installs(self, store, monkeypatch):
|
||||||
|
"""Regression guard for the worst possible outcome of this feature:
|
||||||
|
users on the v3.1.0 release (which reports 1.0.0) must not be locked
|
||||||
|
out of the plugin store entirely."""
|
||||||
|
manifest = {
|
||||||
|
"id": "floored", "name": "Floored", "class_name": "P",
|
||||||
|
"display_modes": ["a"], "versions": [{"ledmatrix_min": "2.0.0"}],
|
||||||
|
}
|
||||||
|
ok, path = self._install_with_manifest(store, manifest, "1.0.0", monkeypatch)
|
||||||
|
|
||||||
|
assert ok is True, (
|
||||||
|
"a core below the trustworthy floor must not block installs — "
|
||||||
|
"nearly every published manifest floors at 2.0.0")
|
||||||
|
assert (path / "manifest.json").exists()
|
||||||
|
|
||||||
|
|
||||||
|
class TestLoaderAndStoreAgree:
|
||||||
|
"""Both read the same manifests; a disagreement means one of them is
|
||||||
|
lying to the user."""
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("manifest,core,expected", [
|
||||||
|
({"min_ledmatrix_version": "3.2.0"}, "3.1.0", False),
|
||||||
|
({"versions": [{"ledmatrix_min": "2.0.0"}]}, "3.2.0", True),
|
||||||
|
({"versions": [{"ledmatrix_min_version": "9.0.0"}]}, "3.2.0", False),
|
||||||
|
({}, "3.2.0", True),
|
||||||
|
])
|
||||||
|
def test_same_verdict(self, manifest, core, expected):
|
||||||
|
from src.plugin_system.plugin_loader import PluginLoader
|
||||||
|
|
||||||
|
store_ok, _ = compatibility.check(manifest, core)
|
||||||
|
assert store_ok is expected
|
||||||
|
|
||||||
|
# The loader resolves the floor through the same helper, so a
|
||||||
|
# divergence in spelling handling would show up here.
|
||||||
|
loader_needed = compatibility.parse_semver(
|
||||||
|
compatibility.declared_min_version(manifest))
|
||||||
|
current = compatibility.parse_semver(core)
|
||||||
|
loader_would_warn = (
|
||||||
|
loader_needed is not None
|
||||||
|
and current is not None
|
||||||
|
and current >= compatibility.TRUSTWORTHY_FLOOR
|
||||||
|
and loader_needed > current
|
||||||
|
)
|
||||||
|
assert loader_would_warn is (not expected)
|
||||||
|
assert hasattr(PluginLoader, "_warn_if_incompatible")
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
"""Version reporting must have exactly one answer.
|
||||||
|
|
||||||
|
`src.__version__` is the canonical core version. The plugin loader compares
|
||||||
|
plugin `ledmatrix_min_version` floors against it, and the plugin ecosystem
|
||||||
|
floors on the number recorded in `CHANGELOG.md` — so if those two disagree, a
|
||||||
|
plugin can declare a floor that is satisfied by a core which does not actually
|
||||||
|
ship the module it needs.
|
||||||
|
|
||||||
|
This has already gone wrong once. The `v3.1.0` tag was cut 2026-05-31, but
|
||||||
|
`src/__init__.py` was not bumped from `"1.0.0"` to `"3.1.0"` until 2026-07-12,
|
||||||
|
six weeks later. Every device installed from that release reports `1.0.0`,
|
||||||
|
which is below the `(2, 0, 0)` floor in `PluginLoader._warn_if_incompatible` —
|
||||||
|
so those users get no compatibility warning at all. See
|
||||||
|
`docs/SPORTS_UNIFICATION.md` (phase B4).
|
||||||
|
|
||||||
|
A tag is not available here, so the tag half of the check lives in
|
||||||
|
`scripts/check_release_version.py`. Wiring that script into CI (on pushed `v*`
|
||||||
|
tags and published releases) is a follow-up PR; until it lands, run it by hand
|
||||||
|
before tagging:
|
||||||
|
|
||||||
|
python scripts/check_release_version.py v3.2.0
|
||||||
|
|
||||||
|
Note: `src.plugin_system.__version__` is deliberately NOT checked. That module
|
||||||
|
versions the *plugin API* (it sits beside `__api_version__` and is documented as
|
||||||
|
such), which moves independently of the core version.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
import src
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
CHANGELOG = REPO_ROOT / "CHANGELOG.md"
|
||||||
|
|
||||||
|
# [0-9] rather than \d: \d also matches non-ASCII decimal digits, which int()
|
||||||
|
# happily parses, so a heading in Arabic-Indic numerals would pass the pattern
|
||||||
|
# and then mismatch confusingly. [ \t] rather than \s for the same class of
|
||||||
|
# reason -- \s matches newlines, so "##\n3.2.0" would read as a heading.
|
||||||
|
SEMVER = re.compile(r"^([0-9]+)\.([0-9]+)\.([0-9]+)$")
|
||||||
|
# Version headings look like "## 3.2.0". A leading "## Unreleased" section is
|
||||||
|
# allowed and skipped -- it is where module additions are staged before a bump.
|
||||||
|
HEADING = re.compile(
|
||||||
|
r"^##[ \t]+(?P<version>[0-9]+\.[0-9]+\.[0-9]+)[ \t]*$", re.MULTILINE)
|
||||||
|
|
||||||
|
|
||||||
|
def test_core_version_is_semver():
|
||||||
|
"""A floor comparison parses this string; it has to be parseable."""
|
||||||
|
assert SEMVER.match(src.__version__), (
|
||||||
|
f"src.__version__ is {src.__version__!r}, which is not X.Y.Z. "
|
||||||
|
"The loader's floor comparison cannot parse it."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_changelog_documents_the_current_version():
|
||||||
|
"""The newest versioned CHANGELOG heading is the version we claim to be.
|
||||||
|
|
||||||
|
Plugins floor on the version recorded in the CHANGELOG as first shipping a
|
||||||
|
module. If the code says 3.2.0 and the CHANGELOG's newest entry is 3.1.0,
|
||||||
|
that record points at the wrong release.
|
||||||
|
"""
|
||||||
|
text = CHANGELOG.read_text(encoding="utf-8")
|
||||||
|
headings = HEADING.findall(text)
|
||||||
|
assert headings, "CHANGELOG.md has no '## X.Y.Z' version headings"
|
||||||
|
|
||||||
|
newest = headings[0]
|
||||||
|
assert newest == src.__version__, (
|
||||||
|
f"src.__version__ is {src.__version__!r} but the newest CHANGELOG "
|
||||||
|
f"heading is {newest!r}. Bump one to match the other: the CHANGELOG is "
|
||||||
|
"what plugin authors read to pick a ledmatrix_min_version floor."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_changelog_versions_are_ordered_and_unique():
|
||||||
|
"""A duplicated or out-of-order heading makes 'first release shipping X'
|
||||||
|
ambiguous, which is exactly the question the sunset rule asks."""
|
||||||
|
text = CHANGELOG.read_text(encoding="utf-8")
|
||||||
|
versions = [tuple(int(p) for p in v.split(".")) for v in HEADING.findall(text)]
|
||||||
|
|
||||||
|
duplicates = {v for v in versions if versions.count(v) > 1}
|
||||||
|
assert not duplicates, f"CHANGELOG.md has duplicate version headings: {duplicates}"
|
||||||
|
|
||||||
|
assert versions == sorted(versions, reverse=True), (
|
||||||
|
"CHANGELOG.md version headings are not in descending order; "
|
||||||
|
f"got {['.'.join(map(str, v)) for v in versions]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_web_interface_version_tracks_the_core():
|
||||||
|
"""web_interface used to carry its own hardcoded "3.0.0", a third answer to
|
||||||
|
'what version is this'. It now re-exports the canonical one."""
|
||||||
|
web_interface = pytest.importorskip(
|
||||||
|
"web_interface", reason="web_interface needs Flask, which is optional here"
|
||||||
|
)
|
||||||
|
assert getattr(web_interface, "__version__", None) == src.__version__, (
|
||||||
|
"web_interface.__version__ has drifted from src.__version__; it should "
|
||||||
|
"re-export the canonical value rather than hardcode its own."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_heading_pattern_is_strict_about_digits_and_whitespace():
|
||||||
|
"""`\\d` also matches non-ASCII decimal digits and `\\s` matches newlines,
|
||||||
|
either of which would let a malformed heading through and then fail the
|
||||||
|
comparison with a confusing message. Pin the tightened patterns."""
|
||||||
|
assert HEADING.findall("## 3.2.0\n") == ["3.2.0"]
|
||||||
|
assert HEADING.findall("##\t3.2.0 \n") == ["3.2.0"]
|
||||||
|
# A bare "##" whose version sits on the next line is not a heading.
|
||||||
|
assert HEADING.findall("##\n3.2.0\n") == []
|
||||||
|
# Arabic-Indic digits parse via int() but are not our version format.
|
||||||
|
assert HEADING.findall("## ٣.٢.٠\n") == []
|
||||||
|
assert SEMVER.match("٣.٢.٠") is None
|
||||||
@@ -2,5 +2,10 @@
|
|||||||
LED Matrix Web Interface V3
|
LED Matrix Web Interface V3
|
||||||
Modern web interface for controlling the LED Matrix display
|
Modern web interface for controlling the LED Matrix display
|
||||||
"""
|
"""
|
||||||
__version__ = "3.0.0"
|
|
||||||
|
# Re-exported, never hardcoded. This used to carry its own "3.0.0", a third
|
||||||
|
# answer to "what version is this" alongside the tag and src.__version__ —
|
||||||
|
# and disagreeing version numbers are what made plugin compatibility floors
|
||||||
|
# untrustworthy (see docs/SPORTS_UNIFICATION.md, phase B4).
|
||||||
|
from src import __version__ # noqa: F401
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user