Merge remote-tracking branch 'origin/main' into claude/scan-order-compensation

# Conflicts:
#	CHANGELOG.md

scripts/README.md

@@ -0,0 +1,59 @@
+# Scripts
+
+Helper scripts for installing, repairing, diagnosing and developing
+LEDMatrix. Most users only ever run the one-shot installer (see the project
+README); everything else here is for troubleshooting or development.
+
+Status key: **keep** — part of install/runtime or referenced by docs, CI,
+tests or code; **dev-only** — for plugin/core development, not needed on a
+display; **diagnostic** — run by hand on a Pi when something is wrong.
+
+## Directories
+
+| Directory | Status | What it holds |
+|---|---|---|
+| [`install/`](install/README.md) | keep | The installers: one-shot, services, sudoers/WiFi permissions, cache setup, and the shared `lib_*.sh` helpers `first_time_install.sh` sources |
+| [`fix_perms/`](fix_perms/README.md) | keep | Permission repair scripts, plus the two root helpers the web interface runs through sudo (`safe_plugin_rm.sh`, `safe_pip_install.sh`) |
+| [`utils/`](utils/README.md) | keep | Scripts run by systemd units or the web interface (conditional web start, WiFi monitor, update verify, DNS fix, Pixlet config editor, cache clearing) |
+| [`dev/`](dev/README.md) | dev-only | Plugin linking, emulator runner, Vegas density audit, Pillow smoke test |
+| `templates/` | dev-only | `dev_preview.html`, the page `dev_server.py` serves |
+
+## Top-level scripts
+
+| Script | Status | What it does |
+|---|---|---|
+| `build_rgbmatrix_nogil.sh` | keep | Rebuilds the rgbmatrix Python binding so `SwapOnVSync` releases the GIL (docs/SCROLL_PERFORMANCE.md) |
+| `check_plugin.py` | dev-only | Renders a plugin across every mode and matrix size and fails on crashes, overflow or golden-image drift |
+| `check_release_version.py` | keep | Checks a release tag, CHANGELOG and `src.__version__` agree (release-version-check workflow) |
+| `check_system_compatibility.sh` | diagnostic | Pre-install check of hardware, OS (Trixie only), kernel, Python, packages, disk and network |
+| `dev_server.py` | dev-only | Browser preview server for plugins without the display loop (http://localhost:5001) |
+| `diagnose_dependencies.sh` | diagnostic | Investigates pip installs stuck on "Preparing metadata" |
+| `diagnose_web_interface.sh` | diagnostic | Checks why the web interface is not reachable |
+| `download_pixlet.sh` | keep | Downloads the bundled Pixlet binaries for Starlark apps (also run from the web UI) |
+| `emergency_reconnect.sh` | diagnostic | Reconnects to your WiFi network if captive-portal testing leaves the Pi offline |
+| `install_dependencies_apt.py` | keep | Dependency installer that tries apt packages first, then pip (installer Step 7, plugin loader) |
+| `install_plugin_dependencies.sh` | diagnostic | Installs plugin requirements by hand when the automatic install fails |
+| `prove_security.py` | keep | Security property checks run by pre-commit |
+| `render_plugin.py` | dev-only | Runs a plugin's `update()` + `display()` and saves the frame as a PNG |
+| `run_plugin_tests.py` | dev-only | Discovers and runs plugin test suites |
+| `scroll_speeds.py` | keep | Shows and tries the scroll speeds your panel can display cleanly |
+| `troubleshoot_captive_portal.sh` | diagnostic | Troubleshoots captive-portal WiFi setup after you can SSH back in |
+| `update_plugin_repos.py` | dev-only | Pulls the latest `ledmatrix-plugins` monorepo |
+| `verify_installation.sh` | diagnostic | Checks that an installation completed correctly |
+| `verify_wifi_setup.sh` | diagnostic | Health check of the WiFi management setup |
+
+## Candidates for removal
+
+Nothing in the repo (docs, CI, tests, other scripts or code) refers to these.
+They are kept for now; each one needs an owner decision before it goes.
+
+| Script | What it does |
+|---|---|
+| `add_defaults_to_schemas.py` | One-off: adds missing `default` values to plugin config schemas |
+| `analyze_plugin_schemas.py` | One-off: reports duplicate/inconsistent fields across plugin schemas |
+| `audit_plugins.py` | AST security audit of plugin code; says it is "designed to run in CI" but no workflow runs it |
+| `audit_render_path.py` | Finds blocking calls reachable from a plugin's `display()` |
+| `sports_scroll_check.py` | Drives a sports scoreboard scroll on the panel and reports its pacing |
+| `test_captive_portal.sh` | Tests the captive portal from a device connected to the AP |
+| `verify_wifi_before_testing.sh` | Pre-flight check before unplugging Ethernet to test WiFi |
+| `dev/test_pillow_compat.py` | Pillow API smoke test to run after upgrading Pillow |

scripts/check_system_compatibility.sh

@@ -28,12 +28,12 @@ print_success() {
 
 print_warning() {
     echo -e "${YELLOW}⚠${NC} $1"
-    ((WARNINGS++))
+    WARNINGS=$((WARNINGS + 1))
 }
 
 print_error() {
     echo -e "${RED}✗${NC} $1"
-    ((COMPATIBILITY_ISSUES++))
+    COMPATIBILITY_ISSUES=$((COMPATIBILITY_ISSUES + 1))
 }
 
 # Check if running on Raspberry Pi
@@ -61,20 +61,18 @@ if [ -f /etc/os-release ]; then
     echo "OS: $PRETTY_NAME"
     echo "Version ID: ${VERSION_ID:-unknown}"
     
+    # first_time_install.sh refuses anything but Raspberry Pi OS / Debian 13
+    # (Trixie), so anything else is an error here too, not a warning.
     if [[ "$ID" == "raspbian" ]] || [[ "$ID" == "debian" ]]; then
-        if [ "${VERSION_ID:-0}" -ge "12" ]; then
-            print_success "Running compatible Debian/Raspbian version (${VERSION_ID})"
-            
-            if [ "${VERSION_ID:-0}" -eq "13" ]; then
-                print_success "Detected Debian 13 Trixie - full compatibility expected"
-            elif [ "${VERSION_ID:-0}" -eq "12" ]; then
-                print_success "Detected Debian 12 Bookworm - full compatibility confirmed"
-            fi
+        if [ "${VERSION_ID:-0}" = "13" ]; then
+            print_success "Detected Debian 13 Trixie - supported"
+        elif [ "${VERSION_ID:-0}" = "12" ]; then
+            print_error "Debian 12 Bookworm is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
         else
-            print_warning "Old Debian/Raspbian version (${VERSION_ID}) - upgrade recommended"
+            print_error "Debian/Raspbian ${VERSION_ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
         fi
     else
-        print_warning "Not running Debian/Raspbian - compatibility not guaranteed"
+        print_error "${ID:-unknown} is not supported - the installer requires Raspberry Pi OS Lite (Trixie), Debian 13"
     fi
 else
     print_error "Could not detect OS version"

scripts/dev/README.md

@@ -6,6 +6,8 @@ This directory contains scripts and utilities for development and testing.
 
 - **`dev_plugin_setup.sh`** - Sets up plugin development environment by linking plugin repositories
 - **`run_emulator.sh`** - Runs the LED Matrix display in emulator mode (for development without hardware)
+- **`vegas_audit.py`** - Measures how much of the Vegas ticker strip actually shows content (dead-frame ratio)
+- **`test_pillow_compat.py`** - Pillow API smoke test to run after upgrading Pillow (`python3 scripts/dev/test_pillow_compat.py`)
 
 ## Usage
 

scripts/fix_perms/README.md

@@ -18,21 +18,26 @@ system user.
   permissions on the `assets/` tree so plugins can download and cache
   team logos, fonts, and other static content.
 
-- **`fix_cache_permissions.sh`** — Creates (if missing) and fixes
-  permissions on `/var/cache/ledmatrix/` and `~/.ledmatrix_cache/` of the
-  user running `sudo`, and creates
-  `/var/cache/ledmatrix/placeholder_logos/` for the sports plugins. It does
-  not touch the cache manager's other fallbacks (`/opt/ledmatrix/cache`,
-  `$TMPDIR/ledmatrix_cache`).
+- **`fix_cache_permissions.sh`** — Restores `/var/cache/ledmatrix/` to the
+  shared `ledmatrix`-group setup by running
+  `scripts/install/setup_cache.sh` (the same script the installer uses),
+  and creates/fixes `~/.ledmatrix_cache/` of the user running `sudo`. It
+  does not touch the cache manager's other fallbacks
+  (`/opt/ledmatrix/cache`, `$TMPDIR/ledmatrix_cache`).
 
 - **`fix_plugin_permissions.sh`** — Fixes ownership on the plugins
   directory so both the root display service and the web service user
   can read and write plugin files (manifests, configs, requirements
   installs).
 
-- **`fix_web_permissions.sh`** — Fixes permissions on log files,
-  systemd journal access, and the sudoers entries the web interface
-  needs to control the display service.
+- **`fix_web_permissions.sh`** — Adds you to the `systemd-journal` and
+  `adm` groups so the web UI can read logs, and makes the project
+  directory yours again, keeping the root-owned sudo helpers
+  (`safe_plugin_rm.sh`, `safe_pip_install.sh`) and `config_secrets.json`
+  the way the installer leaves them. Run it as the web interface's user,
+  **without** `sudo` (it refuses to run as root and calls `sudo` itself).
+  It does not write sudoers rules; that is
+  `scripts/install/configure_web_sudo.sh`.
 
 - **`safe_pip_install.sh`** — Installs a `requirements.txt` as root
   after checking it is the project's own or one under `plugin-repos/` or
@@ -67,7 +72,9 @@ Run these scripts only when:
 sudo ./scripts/fix_perms/fix_cache_permissions.sh
 sudo ./scripts/fix_perms/fix_assets_permissions.sh
 sudo ./scripts/fix_perms/fix_plugin_permissions.sh
-sudo ./scripts/fix_perms/fix_web_permissions.sh
+
+# Run as the web interface's user, without sudo (it asks for sudo itself)
+./scripts/fix_perms/fix_web_permissions.sh
 ```
 
 If you're not sure which one you need, run `fix_cache_permissions.sh`

scripts/fix_perms/fix_assets_permissions.sh

@@ -36,11 +36,12 @@ else
     exit 1
 fi
 
-# Set permissions to allow read/write for owner, group, and others (for root service user)
-# Note: 777 allows root (service user) to write, which is necessary when service runs as root
+# 777: read/write for owner, group and every other account. Root (the display
+# service) does not need it -- root ignores mode bits -- so the "other" bits
+# only matter to accounts that are neither $REAL_USER nor root.
 echo "Setting permissions for assets directory..."
 if sudo chmod -R 777 "$ASSETS_DIR"; then
-    echo "✓ Set assets directory permissions to 777 (writable by root service user)"
+    echo "✓ Set assets directory permissions to 777"
 else
     echo "✗ Failed to set assets directory permissions"
     exit 1
@@ -70,8 +71,7 @@ for SPORTS_DIR in "${SPORTS_DIRS[@]}"; do
         echo "  - Current permissions:"
         ls -ld "$FULL_PATH"
         
-        # Ensure the directory is writable by both the real user and root (service user)
-        # Use 777 permissions to allow root (service) to write, or set group ownership
+        # Owned by the real user; 777 as above (root can write here regardless)
         sudo chmod 777 "$FULL_PATH"
         sudo chown "$REAL_USER:$REAL_GROUP" "$FULL_PATH"
         

scripts/fix_perms/fix_cache_permissions.sh

@@ -1,11 +1,23 @@
 #!/bin/bash
 
 # LEDMatrix Cache Permissions Fix Script
-# This script fixes permissions on all known cache directories so they're writable by the daemon or current user
-# Also sets up placeholder logo directories for sports managers
+#
+# /var/cache/ledmatrix is shared by the display service (root) and the web
+# interface (your user) through the ledmatrix group: root:ledmatrix, 2775,
+# files 660. scripts/install/setup_cache.sh is what sets that up (the
+# installer's Step 2 runs it, and install_web_service.sh keeps the group), so
+# this script runs it rather than applying a model of its own. It used to set
+# the directory 777 and re-group it to your own group, replacing the ledmatrix
+# group everything else relies on.
+#
+# It also repairs ~/.ledmatrix_cache, the cache manager's fallback when
+# /var/cache/ledmatrix is unusable.
 
 echo "Fixing LEDMatrix cache directory permissions..."
 
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SETUP_CACHE="$SCRIPT_DIR/../install/setup_cache.sh"
+
 # Get the real user (not root when running with sudo)
 REAL_USER=${SUDO_USER:-$USER}
 # Resolve the home directory of the real user robustly
@@ -16,72 +28,36 @@ else
 fi
 REAL_GROUP=$(id -gn "$REAL_USER")
 
-# Known cache directories for LEDMatrix. Use the actual user's home instead of a hard-coded path.
-CACHE_DIRS=(
-    "/var/cache/ledmatrix"
-    "$REAL_HOME/.ledmatrix_cache"
-)
-
-for CACHE_DIR in "${CACHE_DIRS[@]}"; do
-    echo ""
-    echo "Checking cache directory: $CACHE_DIR"
-    if [ ! -d "$CACHE_DIR" ]; then
-        echo "  - Directory does not exist. Creating it..."
-        sudo mkdir -p "$CACHE_DIR"
-    fi
-    echo "  - Current permissions:"
-    ls -ld "$CACHE_DIR"
-    echo "  - Fixing permissions..."
-    # Make directory writable by services regardless of user context
-    sudo chmod 777 "$CACHE_DIR"
-    sudo chown "$REAL_USER":"$REAL_GROUP" "$CACHE_DIR"
-    echo "  - Updated permissions:"
-    ls -ld "$CACHE_DIR"
-    echo "  - Testing write access as $REAL_USER..."
-    if sudo -u "$REAL_USER" test -w "$CACHE_DIR"; then
-        echo "    ✓ $CACHE_DIR is now writable by $REAL_USER"
-    else
-        echo "    ✗ $CACHE_DIR is still not writable by $REAL_USER"
-    fi
-    echo "  - Permissions fix complete for $CACHE_DIR."
-done
-
-# Set up placeholder logos directory for sports managers
 echo ""
-echo "Setting up placeholder logos directory for sports managers..."
-
-PLACEHOLDER_DIR="/var/cache/ledmatrix/placeholder_logos"
-if [ ! -d "$PLACEHOLDER_DIR" ]; then
-    echo "Creating placeholder logos directory: $PLACEHOLDER_DIR"
-    sudo mkdir -p "$PLACEHOLDER_DIR"
-    sudo chown "$REAL_USER":"$REAL_GROUP" "$PLACEHOLDER_DIR"
-    sudo chmod 777 "$PLACEHOLDER_DIR"
+echo "Checking cache directory: /var/cache/ledmatrix"
+if [ -f "$SETUP_CACHE" ]; then
+    bash "$SETUP_CACHE"
 else
-    echo "Placeholder logos directory already exists: $PLACEHOLDER_DIR"
-    sudo chmod 777 "$PLACEHOLDER_DIR"
-    sudo chown "$REAL_USER":"$REAL_GROUP" "$PLACEHOLDER_DIR"
+    echo "  ✗ $SETUP_CACHE not found; /var/cache/ledmatrix left unchanged."
 fi
 
+CACHE_DIR="$REAL_HOME/.ledmatrix_cache"
+echo ""
+echo "Checking cache directory: $CACHE_DIR"
+if [ ! -d "$CACHE_DIR" ]; then
+    echo "  - Directory does not exist. Creating it..."
+    sudo mkdir -p "$CACHE_DIR"
+fi
 echo "  - Current permissions:"
-ls -ld "$PLACEHOLDER_DIR"
+ls -ld "$CACHE_DIR"
+echo "  - Fixing permissions..."
+sudo chmod 777 "$CACHE_DIR"
+sudo chown "$REAL_USER":"$REAL_GROUP" "$CACHE_DIR"
+echo "  - Updated permissions:"
+ls -ld "$CACHE_DIR"
 echo "  - Testing write access as $REAL_USER..."
-if sudo -u "$REAL_USER" test -w "$PLACEHOLDER_DIR"; then
-    echo "    ✓ Placeholder logos directory is writable by $REAL_USER"
+if sudo -u "$REAL_USER" test -w "$CACHE_DIR"; then
+    echo "    ✓ $CACHE_DIR is now writable by $REAL_USER"
 else
-    echo "    ✗ Placeholder logos directory is not writable by $REAL_USER"
-fi
-
-# Test with daemon user (which the system might run as)
-if sudo -u daemon test -w "$PLACEHOLDER_DIR" 2>/dev/null; then
-    echo "    ✓ Placeholder logos directory is writable by daemon user"
-else
-    echo "    ✗ Placeholder logos directory is not writable by daemon user"
+    echo "    ✗ $CACHE_DIR is still not writable by $REAL_USER"
 fi
+echo "  - Permissions fix complete for $CACHE_DIR."
 
 echo ""
 echo "All cache directory permission fixes attempted."
 echo "If you still see errors, check which user is running the LEDMatrix service and ensure it matches the owner above."
-echo ""
-echo "The system will now create placeholder logos in:"
-echo "  $PLACEHOLDER_DIR"
-echo "This should eliminate the permission denied warnings for sports logos." 

scripts/fix_perms/fix_plugin_permissions.sh

@@ -51,9 +51,10 @@ fi
 echo "Setting ownership to root:$ACTUAL_USER..."
 sudo chown -R root:"$ACTUAL_USER" "$PLUGINS_DIR"
 
-# Set directory permissions (775: rwxrwxr-x)
-# Root: read/write/execute, Group (ACTUAL_USER): read/write/execute, Others: read/execute
-echo "Setting directory permissions to 2775 (rwxrwxr-x + sticky bit)..."
+# Set directory permissions (2775: rwxrwsr-x)
+# Owner (root) and group (ACTUAL_USER): read/write/execute, others: read/execute.
+# The setgid bit makes new entries inherit the ACTUAL_USER group.
+echo "Setting directory permissions to 2775 (rwxrwsr-x, setgid)..."
 find "$PLUGINS_DIR" -type d -exec sudo chmod 2775 {} \;
 
 # Set file permissions (664: rw-rw-r--)
@@ -71,7 +72,7 @@ fi
 echo "Setting ownership of plugin-repos to root:$ACTUAL_USER..."
 sudo chown -R root:"$ACTUAL_USER" "$PLUGIN_REPOS_DIR"
 
-echo "Setting plugin-repos directory permissions to 2775 (rwxrwxr-x + sticky bit)..."
+echo "Setting plugin-repos directory permissions to 2775 (rwxrwsr-x, setgid)..."
 find "$PLUGIN_REPOS_DIR" -type d -exec sudo chmod 2775 {} \;
 
 echo "Setting plugin-repos file permissions to 664..."
@@ -87,7 +88,7 @@ echo "plugin-repos/:"
 ls -la "$PLUGIN_REPOS_DIR" 2>/dev/null || echo "  (empty or not accessible)"
 echo ""
 echo "Permissions summary:"
-echo "- Root service: Can read/write plugins (for PWM hardware access)"
+echo "- Root service: Can read/write plugins (as root it needs no permission bits)"
 echo "- Web service ($ACTUAL_USER): Can read/write plugins (for installation)"
 echo "- Others: Can read plugins"
 

scripts/fix_perms/fix_web_permissions.sh

@@ -25,8 +25,9 @@ echo ""
 echo "This script will:"
 echo "1. Add the web user to the 'systemd-journal' group for log access"
 echo "2. Add the web user to the 'adm' group for additional system access"
-echo "3. Configure sudoers for passwordless access to system commands"
-echo "4. Set proper file permissions"
+echo "3. Make the project directory yours again, keeping the root-owned sudo"
+echo "   helpers and config_secrets.json as the installer leaves them"
+echo "   (sudoers rules are configure_web_sudo.sh's job, not this script's)"
 echo ""
 
 # Ask for confirmation
@@ -62,6 +63,51 @@ else
     echo "✗ Failed to set project ownership"
 fi
 
+# The chown above also takes back two kinds of file that first_time_install.sh
+# deliberately keeps from the web user. Put them back the way the installer
+# leaves them (its Steps 11 and 11.1), whether or not the chown succeeded.
+#
+# 1. The helpers /etc/sudoers.d/ledmatrix_web lets the web user run as root
+#    (scripts/install/lib_sudoers.sh). A copy the web user owns is a root shell
+#    for whoever can edit it, so they stay root-owned and writable by root only.
+#    Keep this list in step with the installer's Step 11.1 loop;
+#    test/test_web_sudoers_installers_agree.py checks both against the grants.
+for helper in safe_plugin_rm.sh safe_pip_install.sh; do
+    HELPER_PATH="$PROJECT_DIR/scripts/fix_perms/$helper"
+    if [ -f "$HELPER_PATH" ]; then
+        if sudo chown root:root "$HELPER_PATH" && sudo chmod 755 "$HELPER_PATH"; then
+            echo "✓ $helper is root-owned again (sudo runs it as root)"
+        else
+            echo "⚠ Could not make $HELPER_PATH root-owned, mode 755."
+            echo "  Fix it by hand: sudo chown root:root $HELPER_PATH && sudo chmod 755 $HELPER_PATH"
+        fi
+    fi
+done
+
+# 2. config_secrets.json: owned by the account ledmatrix-web.service runs as,
+#    group ledmatrix, mode 640 -- the same owner, group and mode as the
+#    installer's Step 11 gives it.
+SECRETS_FILE="$PROJECT_DIR/config/config_secrets.json"
+if [ -f "$SECRETS_FILE" ]; then
+    SECRETS_OWNER=""
+    if [ -f /etc/systemd/system/ledmatrix-web.service ]; then
+        SECRETS_OWNER=$(grep -m1 "^User=" /etc/systemd/system/ledmatrix-web.service | cut -d'=' -f2 || true)
+    fi
+    SECRETS_OWNER="${SECRETS_OWNER:-$WEB_USER}"
+    if getent group ledmatrix >/dev/null 2>&1; then
+        SECRETS_OWNERSHIP="$SECRETS_OWNER:ledmatrix"
+    else
+        # No ledmatrix group means the installer never ran; keep the chown's group.
+        SECRETS_OWNERSHIP="$SECRETS_OWNER"
+    fi
+    if sudo chown "$SECRETS_OWNERSHIP" "$SECRETS_FILE" && sudo chmod 640 "$SECRETS_FILE"; then
+        echo "✓ config_secrets.json restored to $SECRETS_OWNERSHIP, mode 640"
+    else
+        echo "⚠ Could not restore $SECRETS_FILE to $SECRETS_OWNERSHIP, mode 640."
+        echo "  Fix it by hand: sudo chown $SECRETS_OWNERSHIP $SECRETS_FILE && sudo chmod 640 $SECRETS_FILE"
+    fi
+fi
+
 # Set proper permissions for config files
 if sudo chmod 644 "$PROJECT_DIR/config/config.json" 2>/dev/null; then
     echo "✓ Set config file permissions"
@@ -86,7 +132,7 @@ echo "Step 5: Testing sudo access..."
 if sudo -n systemctl status ledmatrix.service > /dev/null 2>&1; then
     echo "✓ Sudo access test passed"
 else
-    echo "⚠ Sudo access test failed - you may need to run configure_web_sudo.sh"
+    echo "⚠ Sudo access test failed - you may need to run scripts/install/configure_web_sudo.sh"
 fi
 
 echo ""
@@ -101,5 +147,5 @@ echo ""
 echo "After logging back in, test journal access with:"
 echo "  journalctl --no-pager --lines=5"
 echo ""
-echo "If you still have sudo issues, run:"
-echo "  ./configure_web_sudo.sh"
+echo "If you still have sudo issues, run (as this user, without sudo):"
+echo "  $PROJECT_DIR/scripts/install/configure_web_sudo.sh"

scripts/install/README.md

@@ -19,6 +19,21 @@ This directory contains scripts for installing and configuring the LEDMatrix sys
   (the user who runs the script, i.e. the one you installed LEDMatrix as;
   there is no `ledmatrix` system user) the passwordless `nmcli` and related
   WiFi permissions the web interface needs
+- **`install_dns_fix.sh`** - Optional. Installs `ledmatrix-dns-fix.service`,
+  which adds `options single-request` to the resolver when API calls time
+  out (see `systemd/README.md`)
+- **`install_mqtt_bridge.sh`** - Optional. Installs the Home Assistant MQTT
+  bridge service (see `integrations/mqtt_bridge/README.md`)
+
+Libraries (sourced, not run):
+
+- **`lib_sudoers.sh`** - The web interface's sudo allow-list
+  (`/etc/sudoers.d/ledmatrix_web`), shared by `first_time_install.sh` and
+  `configure_web_sudo.sh`
+- **`lib_systemd_render.sh`** - `sed_escape_replacement`, used by every
+  script that renders a unit from `systemd/*.service`
+- **`lib_lowmem.sh`** - Build-job sizing and temporary swap for the C++
+  build on low-memory Pis (`first_time_install.sh` Step 6)
 
 ## Usage
 

scripts/install/configure_web_sudo.sh

@@ -26,7 +26,6 @@ fi
 # Get the full paths to commands and validate each one
 MISSING_CMDS=()
 
-PYTHON_PATH=$(command -v python3)   || true
 SYSTEMCTL_PATH=$(command -v systemctl) || true
 REBOOT_PATH=$(command -v reboot)    || true
 POWEROFF_PATH=$(command -v poweroff)  || true
@@ -35,8 +34,8 @@ JOURNALCTL_PATH=$(command -v journalctl) || true
 SAFE_RM_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_plugin_rm.sh"
 SAFE_PIP_INSTALL_PATH="$PROJECT_ROOT/scripts/fix_perms/safe_pip_install.sh"
 
-# Validate required commands (systemctl, bash, python3 are essential)
-for CMD_NAME in SYSTEMCTL_PATH BASH_PATH PYTHON_PATH; do
+# Validate required commands (systemctl and bash are essential)
+for CMD_NAME in SYSTEMCTL_PATH BASH_PATH; do
     CMD_VAL="${!CMD_NAME}"
     if [ -z "$CMD_VAL" ]; then
         MISSING_CMDS+=("$CMD_NAME")
@@ -70,7 +69,6 @@ fi
 . "$SUDOERS_LIB"
 
 echo "Command paths:"
-echo "  Python: $PYTHON_PATH"
 echo "  Systemctl: $SYSTEMCTL_PATH"
 echo "  Reboot: ${REBOOT_PATH:-(not found, skipping)}"
 echo "  Poweroff: ${POWEROFF_PATH:-(not found, skipping)}"
@@ -79,14 +77,24 @@ echo "  Journalctl: ${JOURNALCTL_PATH:-(not found, skipping)}"
 echo "  Safe plugin rm: $SAFE_RM_PATH"
 echo "  Safe pip install: $SAFE_PIP_INSTALL_PATH"
 
-# Create a temporary sudoers file
-TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
+# Create a temporary sudoers file. A predictable name in a world-writable
+# directory is a symlink target, and these rules end up in /etc/sudoers.d, so
+# let mktemp pick the name; the trap removes it however the script ends.
+TEMP_SUDOERS=$(mktemp "${TMPDIR:-/tmp}/ledmatrix_web_sudoers.XXXXXX") || {
+    echo "Error: could not create a temporary file" >&2
+    exit 1
+}
+trap 'rm -f "$TEMP_SUDOERS"' EXIT
 
 web_sudoers_rules "$WEB_USER" "$PROJECT_ROOT" "$SYSTEMCTL_PATH" "$BASH_PATH" \
     "$REBOOT_PATH" "$POWEROFF_PATH" "$JOURNALCTL_PATH" > "$TEMP_SUDOERS"
 
 # Never offer to install rules we have not parsed. A malformed drop-in in
 # /etc/sudoers.d makes sudo refuse every command for every user.
+# visudo lives in /usr/sbin, which is not on every user's PATH.
+if ! command -v visudo >/dev/null 2>&1 && [ -x /usr/sbin/visudo ]; then
+    PATH="$PATH:/usr/sbin"
+fi
 if command -v visudo >/dev/null 2>&1; then
     if ! visudo -c -f "$TEMP_SUDOERS" >/dev/null 2>&1; then
         echo ""
@@ -96,6 +104,8 @@ if command -v visudo >/dev/null 2>&1; then
         rm -f "$TEMP_SUDOERS"
         exit 1
     fi
+else
+    echo "⚠ visudo not found; the rules below have not been validated"
 fi
 
 echo ""
@@ -124,7 +134,7 @@ if [[ ! $REPLY =~ ^[Yy]$ ]]; then
     exit 0
 fi
 
-# Apply the configuration using visudo
+# Apply the configuration
 echo "Applying sudoers configuration..."
 # Harden the helper script: root-owned, not writable by web user
 echo "Hardening safe_plugin_rm.sh ownership..."
@@ -143,21 +153,29 @@ if ! sudo chmod 755 "$SAFE_PIP_INSTALL_PATH"; then
 fi
 
 if sudo cp "$TEMP_SUDOERS" /etc/sudoers.d/ledmatrix_web; then
+    # sudo reads /etc/sudoers.d files that are root-owned and not writable by
+    # group or other; 440 is the mode visudo and first_time_install.sh use.
+    if ! sudo chmod 440 /etc/sudoers.d/ledmatrix_web; then
+        echo "Warning: could not set mode 440 on /etc/sudoers.d/ledmatrix_web"
+    fi
     echo "Configuration applied successfully!"
     echo ""
     echo "Testing sudo access..."
     
-    # Test a few commands
-    if sudo -n systemctl status ledmatrix.service > /dev/null 2>&1; then
+    # Ask sudo whether two of the new rules let this user in without a
+    # password. `sudo -l CMD` answers from the rules without running CMD, so
+    # this does not depend on whether ledmatrix.service is running, and it
+    # tests commands the rules actually grant.
+    if sudo -n -l "$SYSTEMCTL_PATH" status ledmatrix.service > /dev/null 2>&1; then
         echo "✓ systemctl status ledmatrix.service - OK"
     else
-        echo "✗ systemctl status ledmatrix.service - Failed"
+        echo "✗ systemctl status ledmatrix.service - not allowed without a password"
     fi
-    
-    if sudo -n test -f "$PROJECT_ROOT/start_display.sh"; then
-        echo "✓ File access test - OK"
+
+    if sudo -n -l "$BASH_PATH" "$SAFE_RM_PATH" "$PROJECT_ROOT/plugin-repos/example" > /dev/null 2>&1; then
+        echo "✓ safe_plugin_rm.sh helper - OK"
     else
-        echo "✗ File access test - Failed"
+        echo "✗ safe_plugin_rm.sh helper - not allowed without a password"
     fi
     
     echo ""

scripts/install/configure_wifi_permissions.sh

@@ -144,6 +144,11 @@ $WEB_USER ALL=(ALL) NOPASSWD: $MKDIR_PATH -p /etc/NetworkManager/dnsmasq-shared.
 $WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/cp /tmp/hostapd.conf /etc/hostapd/hostapd.conf
 $WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/cp /tmp/dnsmasq.conf /etc/dnsmasq.d/ledmatrix-captive.conf
 $WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/rm -f /etc/dnsmasq.d/ledmatrix-captive.conf
+# The same captive-portal DNS drop-in for NetworkManager's shared-mode dnsmasq
+# (wifi_manager._write_nm_dnsmasq_captive_conf / _remove_nm_dnsmasq_captive_conf),
+# exact paths.
+$WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/cp /tmp/ledmatrix-nm-dnsmasq.conf /etc/NetworkManager/dnsmasq-shared.d/ledmatrix-captive.conf
+$WEB_USER ALL=(ALL) NOPASSWD: /usr/bin/rm -f /etc/NetworkManager/dnsmasq-shared.d/ledmatrix-captive.conf
 EOF
 
 echo "Generated sudoers configuration:"
@@ -151,6 +156,21 @@ echo "--------------------------------"
 cat "$TEMP_SUDOERS"
 echo "--------------------------------"
 
+# Never install rules we have not parsed. A malformed drop-in in
+# /etc/sudoers.d makes sudo refuse every command for every user, which on a
+# headless Pi leaves no way in at all. first_time_install.sh and
+# configure_web_sudo.sh check their rules the same way.
+if command -v visudo >/dev/null 2>&1; then
+    if ! visudo -c -f "$TEMP_SUDOERS" >/dev/null 2>&1; then
+        echo "✗ The generated sudoers rules did not parse:" >&2
+        visudo -c -f "$TEMP_SUDOERS" >&2 || true
+        echo "  Leaving $SUDOERS_FILE unchanged." >&2
+        exit 1
+    fi
+else
+    echo "⚠ visudo not found; installing the sudoers rules unvalidated"
+fi
+
 # Apply the sudoers configuration
 echo ""
 echo "Applying sudoers configuration..."
@@ -213,11 +233,14 @@ rm -f "$TEMP_POLKIT"
 echo ""
 echo "Step 3: Testing permissions..."
 
-# Test sudo access
-if sudo -n "$NMCLI_PATH" device status > /dev/null 2>&1; then
-    echo "✓ nmcli device status - OK"
+# Ask sudo whether one of the new rules lets this user in without a password.
+# `sudo -l CMD` answers from the rules without running CMD, so the radio is
+# left alone. (This used to run `nmcli device status`, which is not granted,
+# so it could only ever report a failure.)
+if sudo -n -l "$NMCLI_PATH" radio wifi on > /dev/null 2>&1; then
+    echo "✓ nmcli radio wifi on - OK"
 else
-    echo "✗ nmcli device status - Failed (this is expected if not connected)"
+    echo "✗ nmcli radio wifi on - not allowed without a password"
 fi
 
 echo ""

scripts/install/install_dns_fix.sh

@@ -10,6 +10,10 @@
 set -e
 
 PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
+
+# shellcheck source=scripts/install/lib_systemd_render.sh
+source "$PROJECT_ROOT_DIR/scripts/install/lib_systemd_render.sh"
+
 SERVICE_NAME="ledmatrix-dns-fix"
 UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
 UNIT_DEST="/etc/systemd/system/$SERVICE_NAME.service"
@@ -34,7 +38,8 @@ fi
 chmod +x "$PROJECT_ROOT_DIR/scripts/utils/apply_dns_single_request.sh"
 
 echo "Installing $UNIT_DEST..."
-sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
+ESCAPED_PROJECT_ROOT_DIR=$(sed_escape_replacement "$PROJECT_ROOT_DIR")
+sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
     | $SUDO tee "$UNIT_DEST" > /dev/null
 
 # Order ledmatrix.service after the fix. `Before=` in the unit itself only

scripts/install/install_mqtt_bridge.sh

@@ -9,6 +9,10 @@
 set -e
 
 PROJECT_ROOT_DIR=$(cd "$(dirname "$0")/../.." && pwd)
+
+# shellcheck source=scripts/install/lib_systemd_render.sh
+source "$PROJECT_ROOT_DIR/scripts/install/lib_systemd_render.sh"
+
 BRIDGE_DIR="$PROJECT_ROOT_DIR/integrations/mqtt_bridge"
 SERVICE_NAME="ledmatrix-mqtt-bridge"
 UNIT_SRC="$PROJECT_ROOT_DIR/systemd/$SERVICE_NAME.service"
@@ -40,7 +44,8 @@ python3 -m pip install -r "$BRIDGE_DIR/requirements.txt" 2>/dev/null \
     || python3 -m pip install --break-system-packages -r "$BRIDGE_DIR/requirements.txt"
 
 echo "Installing $UNIT_DEST..."
-sed "s|__PROJECT_ROOT_DIR__|$PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
+ESCAPED_PROJECT_ROOT_DIR=$(sed_escape_replacement "$PROJECT_ROOT_DIR")
+sed "s|__PROJECT_ROOT_DIR__|$ESCAPED_PROJECT_ROOT_DIR|g" "$UNIT_SRC" \
     | $SUDO tee "$UNIT_DEST" > /dev/null
 
 $SYSTEMCTL_CMD daemon-reload

scripts/install/install_wifi_monitor.sh

@@ -51,20 +51,25 @@ if [ ${#MISSING_PACKAGES[@]} -gt 0 ]; then
     
     # Install packages automatically (no prompt)
     # Use apt directly if running as root, otherwise use sudo
+    PACKAGES_OK=true
     if [ "$EUID" -eq 0 ]; then
         apt update || echo "⚠ apt update failed, continuing anyway..."
         apt install -y "${MISSING_PACKAGES[@]}" || {
+            PACKAGES_OK=false
             echo "⚠ Package installation failed, but continuing with WiFi monitor setup"
             echo "  You may need to install packages manually: apt install -y ${MISSING_PACKAGES[*]}"
         }
     else
         sudo apt update || echo "⚠ apt update failed, continuing anyway..."
         sudo apt install -y "${MISSING_PACKAGES[@]}" || {
+            PACKAGES_OK=false
             echo "⚠ Package installation failed, but continuing with WiFi monitor setup"
             echo "  You may need to install packages manually: sudo apt install -y ${MISSING_PACKAGES[*]}"
         }
     fi
-    echo "✓ Package installation completed"
+    if [ "$PACKAGES_OK" = true ]; then
+        echo "✓ Package installation completed"
+    fi
 fi
 
 # Render the unit from systemd/ledmatrix-wifi-monitor.service rather than

scripts/install/lib_systemd_render.sh

@@ -2,9 +2,10 @@
 #
 # Shared helper for rendering systemd unit templates via sed.
 #
-# Sourced by install_service.sh, install_web_service.sh and
-# install_wifi_monitor.sh so all three escape sed replacement text the same
-# way instead of carrying three copies of the same fix.
+# Sourced by install_service.sh, install_web_service.sh,
+# install_wifi_monitor.sh, install_dns_fix.sh and install_mqtt_bridge.sh so
+# every unit renderer escapes sed replacement text the same way instead of
+# carrying its own copy of the fix.
 
 # sed_escape_replacement VALUE
 #

scripts/install/one-shot-install.sh

@@ -205,27 +205,6 @@ check_sudo() {
     print_success "Sudo access confirmed"
 }
 
-# Fix /tmp permissions if needed (common issue when running via curl | bash)
-# Note: /tmp permission fixing is now done inline before running first_time_install.sh
-# This function is kept for backward compatibility but not actively used
-fix_tmp_permissions() {
-    CURRENT_STEP="TMP directory check"
-    # Only fix if /tmp is actually not writable (don't preemptively fix)
-    if [ ! -w /tmp ]; then
-        print_warning "/tmp is not writable, attempting to fix..."
-        if [ "$EUID" -eq 0 ]; then
-            chmod 1777 /tmp 2>/dev/null || true
-        else
-            sudo chmod 1777 /tmp 2>/dev/null || true
-        fi
-    fi
-    
-    # Ensure TMPDIR is set correctly
-    if [ -z "${TMPDIR:-}" ] || [ ! -w "${TMPDIR:-/tmp}" ]; then
-        export TMPDIR=/tmp
-    fi
-}
-
 # Main installation function
 main() {
     print_step "LED Matrix One-Shot Installation"
@@ -429,6 +408,13 @@ main() {
         print_step "Installation Complete!"
         print_success "LED Matrix has been successfully installed!"
         echo ""
+        # first_time_install.sh -y reboots as its last action, so by now the
+        # reboot is under way (unless LEDMATRIX_SKIP_REBOOT_PROMPT=1 was set).
+        if [ "${LEDMATRIX_SKIP_REBOOT_PROMPT:-0}" != "1" ]; then
+            echo "The installer has just started a reboot to finish setup, so this"
+            echo "session may disconnect now. Give the Pi a few minutes to come back, then:"
+            echo ""
+        fi
         echo "Next steps:"
         echo "  1. Configure your settings: sudo nano $REPO_DIR/config/config.json"
         if command -v hostname >/dev/null 2>&1; then
@@ -449,7 +435,7 @@ main() {
         else
             echo "  2. Or use the web interface: http://<your-pi-ip>:5000"
         fi
-        echo "  3. Start the service: sudo systemctl start ledmatrix.service"
+        echo "  3. The display service starts on boot; to start it by hand: sudo systemctl start ledmatrix.service"
         echo ""
     else
         print_error "Main installation script exited with code $INSTALL_EXIT_CODE"

scripts/utils/README.md

@@ -9,6 +9,7 @@ This directory contains utility scripts for maintenance and system operations.
 - **`wifi_monitor_daemon.py`** - Background daemon that monitors WiFi/Ethernet connection and manages access point mode
 - **`pixlet_config_editor.sh`** - Opens Pixlet's own config UI for one installed Starlark app
 - **`apply_dns_single_request.sh`** - Adds `options single-request` to the resolver (run by `ledmatrix-dns-fix.service`)
+- **`auto_update_verify.py`** - Health check after an automatic update, rolling back if it fails (the updater copies it to `data/` before pulling and `ledmatrix-update-verify.service` runs that copy)
 
 ## Usage