fix(install): survive the rgbmatrix build on low-memory Pis (#430)

* fix(install): survive the rgbmatrix build on low-memory Pis The one-shot installer failed at Step 6 on a 1GB Pi with "Failed building wheel for rgbmatrix", and told the user to install build tools they already had. The real cause was the kernel OOM killer. Upstream's pyproject.toml declares no [tool.scikit-build] options, so scikit-build-core drives Ninja at its default of nproc+2 jobs -- six concurrent compiles on a 4-core Pi. CMakeLists.txt compiles the same 14 sources three times (~45 translation units), two of them Cython-generated C++ where a single cc1plus peaks near 800MB. That does not fit in 512MB-1GB of RAM. Add scripts/install/lib_lowmem.sh and wire it into the installer: - Cap build parallelism at max(1, min(cores, RAM/768)) via CMAKE_BUILD_PARALLEL_LEVEL, which is what cmake --build actually reads. MAKEFLAGS is ignored by Ninja and is set only as a Makefile-generator fallback. A 4GB Pi 4 still gets 4 jobs; 512MB and 1GB boards get 1. - Add a temporary swapfile sized to bring RAM+swap to 3GB (capped at 2GB), removed once the build finishes. An EXIT trap is the backstop for the error path. Nothing is written to /etc/fstab or /etc/dphys-swapfile. Existing swap is measured excluding zram, which is compressed RAM and so does not help a build OOM. - Keep pip's build tree off tmpfs. Debian 13 mounts /tmp as tmpfs, so the default held the whole C++ build tree in RAM alongside the compiler. - Diagnose OOM failures from the build log and the kernel ring buffer, instead of always blaming missing build tools. The OOM killer writes nothing to pip's output, which is why this was misreported. - Report RAM and the chosen job count in the Step 1 preflight, and emit a heartbeat during the compile so a deliberately serial 15-25 minute build does not look like a hang. New flags --skip-swap and --build-jobs N, with LEDMATRIX_SKIP_SWAP and LEDMATRIX_BUILD_JOBS equivalents. Also skip the duplicate apt-get update that the one-shot installer and first_time_install.sh each ran a minute apart, and complete the dphys-swapfile advice in diagnose_dependencies.sh with the CONF_MAXSWAP line, without which raising CONF_SWAPSIZE above 2048 is silently clamped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q * fix(install): address review findings on the low-memory build path Three fixes from PR review: - Validate --build-jobs / LEDMATRIX_BUILD_JOBS before check_memory's fallback return. When lib_lowmem.sh is absent that return also honoured the override, so a non-numeric value skipped validation and instead blew up later in an arithmetic test in Step 6 with a generic error. - Fall back to the default TMPDIR when the disk-backed build directory cannot be created, rather than pointing the build at a path that does not exist. A nearly-full disk is the likely cause on exactly the devices this targets. - Pass LEDMATRIX_APT_UPDATED explicitly to the sudo child instead of relying on -E, which a sudoers env_reset/env_keep policy can strip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q * fix(install): don't add 30s to every rgbmatrix build The build progress heartbeat slept for the full 30s report interval before re-checking whether the compile had finished, so every build paid up to 30 seconds of dead wall time -- including fast ones on a Pi 4/5 and every --force-rebuild run. Poll every 2s and report every 30s instead. Measured: 30s of overhead on an instant build drops to 2s, with heartbeats still emitted on the same schedule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdsJs65WnUo8BtKHMAGA1q --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 18:28:06 +00:00 · 2026-08-03 15:21:18 -04:00
parent 21825cbfbc
commit 16fbb7ebeb
7 changed files with 824 additions and 16 deletions
@@ -141,6 +141,7 @@ The system supports live, recent, and upcoming game information for multiple spo
    sudo RPI_RGB_FORCE_REBUILD=1 ./first_time_install.sh
    ```
  - Pi 5 config: leave `rp1_rio` at `0` (PIO mode, default) and set `gpio_slowdown` to `1` or `2`.
  - **1GB models (Pi 3B / 3B+) and other low-memory boards**: supported, but the `rpi-rgb-led-matrix` C++ build needs more memory than the Pi has. The installer detects this automatically, compiles with fewer parallel jobs, and adds a temporary swapfile for the build which it removes afterwards. Expect that step to take 15-25 minutes instead of 2-5, and leave at least **3GB free** on the SD card. If you manage swap yourself, opt out with `--skip-swap`. To pin the compiler down further, use `--build-jobs 1`.
 ### RGB Matrix Bonnet / HAT
@@ -314,12 +315,12 @@ curl -fsSL https://raw.githubusercontent.com/ChuckBuilds/LEDMatrix/main/scripts/
 ```
 This one-shot installer will automatically:
- Check system prerequisites (network, disk space, sudo access)
+- Check system prerequisites (network, disk space, memory, sudo access)
 - Install required system packages (git, python3, build tools, etc.)
 - Clone or update the LEDMatrix repository
 - Run the complete first-time installation script
-The installation process typically takes 10-30 minutes depending on your internet connection and Pi model. All errors are reported explicitly with actionable fixes.
+The installation process typically takes 10-30 minutes depending on your internet connection and Pi model. Pi 3B/3B+ and other 1GB boards land at the top of that range, because the C++ library is compiled serially to stay within available memory. All errors are reported explicitly with actionable fixes.
 **Note:** The script is safe to run multiple times and will handle existing installations gracefully.
@@ -82,6 +82,70 @@ python3 web_interface/start.py
 ## Common Issues by Category
 ### Installation & Build Issues
 #### Step 6 fails: "Failed building wheel for rgbmatrix"
 **Symptoms:**
 ```
 note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for rgbmatrix
 Failed to build rgbmatrix
 ✗ Failed to install rpi-rgb-led-matrix Python package
 ```
 **Cause:**
 Almost always the kernel's out-of-memory killer, not missing build tools. The
 `rpi-rgb-led-matrix` library compiles roughly 45 C++ translation units, two of
 them Cython-generated — a single `cc1plus` on those can peak near 800MB. The
 build system defaults to running several of those at once, which exceeds RAM on
 512MB and 1GB boards. Because the OOM killer writes nothing to pip's output, the
 failure looks like a toolchain problem, and `sudo apt install -y
 python-dev-is-python3 cmake build-essential` will report everything is already
 up to date.
 **How to confirm:**
 ```bash
 dmesg -T | grep -i "out of memory"     # look for "Killed process ... (cc1plus)"
 free -h                                 # total RAM and swap
 ```
 **Fix:**
 Current versions of the installer handle this automatically: they cap build
 parallelism based on available RAM and add a temporary swapfile for the build,
 removing it when the build finishes. If you are on an older checkout, or the
 temporary swapfile could not be created, either force a serial compile:
 ```bash
 sudo ./first_time_install.sh --build-jobs 1
 ```
 or add permanent swap and re-run the installer, which resumes at Step 6:
 ```bash
 sudo apt install -y dphys-swapfile
 sudo sed -i 's/^#\?CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile
 sudo sed -i 's/^#\?CONF_MAXSWAP=.*/CONF_MAXSWAP=2048/' /etc/dphys-swapfile
 sudo dphys-swapfile swapoff && sudo dphys-swapfile setup && sudo dphys-swapfile swapon
 sudo ./first_time_install.sh
 ```
 `CONF_MAXSWAP` matters: it defaults to 2048 and silently clamps `CONF_SWAPSIZE`,
 so setting only `CONF_SWAPSIZE` to a larger value has no effect.
 **Related:**
 - The installer needs roughly 3GB free on the card to place the swapfile. If
  disk is tight it will say so and skip the swapfile: `sudo apt clean` first.
 - `sudo bash scripts/check_system_compatibility.sh` reports RAM and disk.
 - `sudo bash scripts/diagnose_dependencies.sh` dumps build-dependency state.
 ---
 ### Web Interface & Service Issues
 #### Service Not Running/Starting
@@ -152,6 +152,8 @@ ASSUME_YES=${LEDMATRIX_ASSUME_YES:-0}
 SKIP_SOUND=${LEDMATRIX_SKIP_SOUND:-0}
 SKIP_PERF=${LEDMATRIX_SKIP_PERF:-0}
 SKIP_REBOOT_PROMPT=${LEDMATRIX_SKIP_REBOOT_PROMPT:-0}
 SKIP_SWAP=${LEDMATRIX_SKIP_SWAP:-0}
 BUILD_JOBS_OVERRIDE=${LEDMATRIX_BUILD_JOBS:-}
 usage() {
    cat <<USAGE
@@ -163,11 +165,21 @@ Options:
      --skip-sound          Skip sound module configuration
      --skip-perf           Skip performance tweaks (isolcpus/audio)
      --no-reboot-prompt    Do not prompt for reboot at the end
      --skip-swap           Never add temporary swap for the C++ build
      --build-jobs N        Compile the C++ library with N parallel jobs
                            (default: scaled to available RAM)
  -h, --help                Show this help message and exit
 Environment variables (same effect as flags):
  LEDMATRIX_ASSUME_YES=1, RPI_RGB_FORCE_REBUILD=1, LEDMATRIX_SKIP_SOUND=1,
-  LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1
+  LEDMATRIX_SKIP_PERF=1, LEDMATRIX_SKIP_REBOOT_PROMPT=1,
  LEDMATRIX_SKIP_SWAP=1, LEDMATRIX_BUILD_JOBS=N
 Low-memory devices:
  On a Pi with under 2GB of RAM the C++ build is limited to fewer parallel
  jobs and a temporary swapfile is added for the duration of the build, then
  removed. Without this the compiler is killed by the kernel out-of-memory
  killer on 512MB and 1GB models.
 USAGE
 }
@@ -178,12 +190,38 @@ while [ $# -gt 0 ]; do
        --skip-sound) SKIP_SOUND=1 ;;
        --skip-perf) SKIP_PERF=1 ;;
        --no-reboot-prompt) SKIP_REBOOT_PROMPT=1 ;;
        --skip-swap) SKIP_SWAP=1 ;;
        --build-jobs)
            shift
            if [ $# -eq 0 ]; then echo "--build-jobs requires a number"; usage; exit 1; fi
            BUILD_JOBS_OVERRIDE="$1"
            ;;
        -h|--help) usage; exit 0 ;;
        *) echo "Unknown option: $1"; usage; exit 1 ;;
    esac
    shift
 done
 # Low-memory build helpers (job sizing, temporary swap, OOM detection).
 # Sourced rather than inlined so the sizing logic can be unit-tested; if the
 # file is missing we fall back to the historical behaviour rather than failing
 # the install.
 LOWMEM_LIB="$PROJECT_ROOT_DIR/scripts/install/lib_lowmem.sh"
 LOWMEM_AVAILABLE=0
 if [ -f "$LOWMEM_LIB" ]; then
    # shellcheck source=scripts/install/lib_lowmem.sh
    . "$LOWMEM_LIB"
    LOWMEM_AVAILABLE=1
 else
    echo "⚠ $LOWMEM_LIB not found; skipping low-memory build protections."
    lm_remove_build_swap() { return 0; }
 fi
 # Remove the temporary build swapfile no matter how the script ends. Step 6
 # tears it down itself; this is the backstop for the error path, since
 # on_error ends in `exit` and EXIT traps still run.
 trap 'lm_remove_build_swap' EXIT
 # Helpers
 retry() {
    local attempt=1
@@ -263,15 +301,144 @@ check_disk_space() {
    fi
 }
 # Decide how much memory Step 6's C++ build may use, and say so up front.
 #
 # Sets TOTAL_RAM_MB, TOTAL_SWAP_MB, BUILD_JOBS and LOW_RAM for later steps.
 check_memory() {
    command -v nproc >/dev/null 2>&1 && CPU_CORES=$(nproc) || CPU_CORES=1
    # Validated up front rather than trusted: a non-numeric value would other-
    # wise survive as far as an arithmetic test in Step 6 and fail there with a
    # generic error. This must precede the fallback return below, which also
    # honours the override.
    if [ -n "$BUILD_JOBS_OVERRIDE" ]; then
        if ! echo "$BUILD_JOBS_OVERRIDE" | grep -qE '^[1-9][0-9]*$'; then
            echo "✗ Invalid build job count: '$BUILD_JOBS_OVERRIDE' (expected a positive integer)"
            exit 1
        fi
    fi
    if [ "$LOWMEM_AVAILABLE" != "1" ]; then
        TOTAL_RAM_MB=0
        TOTAL_SWAP_MB=0
        LOW_RAM=0
        BUILD_JOBS=${BUILD_JOBS_OVERRIDE:-$CPU_CORES}
        return 0
    fi
    TOTAL_RAM_MB=$(lm_total_ram_mb)
    TOTAL_SWAP_MB=$(lm_total_swap_mb)
    # Test hook: exercise the low-memory path on a machine that has plenty.
    if [ -n "${LEDMATRIX_FORCE_LOW_RAM:-}" ] && [ "${LEDMATRIX_FORCE_LOW_RAM}" != "0" ]; then
        TOTAL_RAM_MB="${LEDMATRIX_FORCE_LOW_RAM}"
        echo "⚠ LEDMATRIX_FORCE_LOW_RAM set: pretending this device has ${TOTAL_RAM_MB}MB of RAM"
    fi
    LOW_RAM=0
    if [ "$TOTAL_RAM_MB" -gt 0 ] && [ "$TOTAL_RAM_MB" -lt 2048 ]; then
        LOW_RAM=1
    fi
    if [ -n "$BUILD_JOBS_OVERRIDE" ]; then
        BUILD_JOBS="$BUILD_JOBS_OVERRIDE"
    else
        BUILD_JOBS=$(lm_build_jobs "$TOTAL_RAM_MB" "$CPU_CORES")
    fi
    echo "System memory: ${TOTAL_RAM_MB}MB RAM, ${TOTAL_SWAP_MB}MB swap, ${CPU_CORES} core(s)"
    if [ "$LOW_RAM" = "1" ]; then
        echo "⚠ Low-memory device detected."
        echo "  The rpi-rgb-led-matrix C++ build in Step 6 will use ${BUILD_JOBS} parallel job(s)"
        echo "  instead of all cores, and a temporary swapfile will be added for the build"
        echo "  and removed afterwards. Without this the compiler is killed by the kernel"
        echo "  out-of-memory killer. Expect Step 6 to take 15-25 minutes."
        if [ "$SKIP_SWAP" = "1" ]; then
            echo "  Temporary swap is disabled (--skip-swap); the build may still run out of memory."
        fi
    else
        echo "✓ Memory sufficient for the rpi-rgb-led-matrix build (${BUILD_JOBS} parallel job(s))"
    fi
 }
 # Compile and install the rgbmatrix Python package.
 #
 # CMAKE_BUILD_PARALLEL_LEVEL is the setting that actually caps the compile:
 # upstream's pyproject.toml declares no [tool.scikit-build] options, so
 # scikit-build-core drives Ninja through `cmake --build`, which reads this
 # variable. Ninja's own default is nproc+2, i.e. six concurrent cc1plus
 # processes on a 4-core Pi. MAKEFLAGS is ignored by Ninja and is set only to
 # cover the Makefile-generator fallback if ninja-build is somehow absent.
 #
 # BUILD_TMPDIR redirects pip's build tree off tmpfs where applicable — see
 # where it is computed in Step 6.
 run_rgbmatrix_build() {
    local jobs="$1" out="$2"
    local pid elapsed=0
    TMPDIR="${BUILD_TMPDIR:-${TMPDIR:-/tmp}}" \
    CMAKE_BUILD_PARALLEL_LEVEL="$jobs" \
    MAKEFLAGS="-j${jobs}" \
        python3 -m pip install --break-system-packages . > "$out" 2>&1 &
    pid=$!
    # The build's output is captured to a file, so without a heartbeat a serial
    # compile on a 1GB Pi looks like a 20-minute hang and invites a Ctrl-C.
    #
    # Polled at a short interval but reported every 30s: polling at the report
    # interval instead would add most of that interval to the wall time of
    # every build, including fast ones on a Pi 4/5.
    while kill -0 "$pid" 2>/dev/null; do
        sleep 2
        elapsed=$((elapsed + 2))
        if [ "$((elapsed % 30))" -eq 0 ] && kill -0 "$pid" 2>/dev/null; then
            printf '  ... still compiling (%dm%02ds elapsed)\n' "$((elapsed / 60))" "$((elapsed % 60))"
        fi
    done
    wait "$pid"
 }
 # Explain a failed rgbmatrix build. The kernel OOM killer writes nothing to the
 # build's own output, which is why this used to be reported as a missing
 # build-tools problem and sent users chasing packages they already had.
 print_rgbmatrix_build_failure() {
    local out="$1"
    if [ "$LOWMEM_AVAILABLE" = "1" ] && lm_build_failed_on_oom "$out"; then
        echo "✗ The rpi-rgb-led-matrix build was killed: the system ran out of memory."
        echo "  This is NOT a missing build-tools problem — the C++ compiler ran out of RAM."
        echo "  RAM: ${TOTAL_RAM_MB}MB   Swap: $(lm_total_swap_mb)MB   Parallel jobs used: ${BUILD_JOBS}"
        if [ -n "${LM_SWAP_SKIP_REASON:-}" ]; then
            echo "  No temporary swap was added: ${LM_SWAP_SKIP_REASON}"
        fi
        echo ""
        echo "  Try one of these, then re-run this script (it resumes at Step 6):"
        echo "    1. Force a single compile job:"
        echo "       sudo ./first_time_install.sh --build-jobs 1"
        echo "    2. Add permanent swap, if the temporary swapfile could not be created:"
        echo "       sudo apt install -y dphys-swapfile"
        echo "       sudo sed -i 's/^#\\?CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile"
        echo "       sudo sed -i 's/^#\\?CONF_MAXSWAP=.*/CONF_MAXSWAP=2048/' /etc/dphys-swapfile"
        echo "       sudo dphys-swapfile swapoff && sudo dphys-swapfile setup && sudo dphys-swapfile swapon"
        echo "    3. Free up disk space so a larger swapfile fits: sudo apt clean"
    else
        echo "✗ Failed to install rpi-rgb-led-matrix Python package"
        echo "  Ensure build tools are installed:"
        echo "  sudo apt install -y python-dev-is-python3 cmake build-essential"
    fi
 }
 echo ""
 echo "This script will perform the following steps:"
-echo "1. Install system dependencies"
+echo "1. Check prerequisites (network, disk, memory) and install system dependencies"
 echo "2. Fix cache permissions"
 echo "3. Fix assets directory permissions"
 echo "3.1. Fix plugin directory permissions"
 echo "4. Ensure configuration files exist"
 echo "5. Install Python project dependencies (requirements.txt)"
 echo "6. Build and install rpi-rgb-led-matrix and test import"
 echo "   (compiles C++; low-memory Pis get temporary swap and a serial build)"
 echo "7. Install web interface dependencies"
 echo "7.5. Install main LED Matrix service"
 echo "8. Install web interface service"
@@ -315,9 +482,16 @@ echo "----------------------------------------"
 # Pre-flight checks before APT operations
 check_network
 check_disk_space
 check_memory
-# Update package list
+# Update package list. The one-shot installer refreshes the lists moments
-apt_update
+# before invoking this script and exports LEDMATRIX_APT_UPDATED=1, so skip the
 # duplicate refresh on that path.
 if [ "${LEDMATRIX_APT_UPDATED:-0}" = "1" ]; then
    echo "Package lists already refreshed by the one-shot installer; skipping apt update."
 else
    apt_update
 fi
 # Install required system packages
 echo "Installing Python packages and dependencies..."
@@ -902,29 +1076,66 @@ else
            fi
        fi
        # Add temporary swap on low-memory devices so the compiler survives.
        CURRENT_STEP="Prepare the low-memory build environment"
        if [ "$LOWMEM_AVAILABLE" = "1" ] && [ "$SKIP_SWAP" != "1" ]; then
            lm_ensure_build_swap "$(lm_swap_needed_mb "$TOTAL_RAM_MB" "$TOTAL_SWAP_MB")"
        elif [ "$SKIP_SWAP" = "1" ]; then
            LM_SWAP_SKIP_REASON="disabled with --skip-swap"
        fi
        # pip builds in $TMPDIR. Debian 13 mounts /tmp as tmpfs, so the default
        # would hold the entire C++ build tree in RAM — competing with the very
        # compiler we are trying to keep under the memory limit.
        BUILD_TMPDIR=""
        if [ "$LOWMEM_AVAILABLE" = "1" ]; then
            _disk_tmp=$(lm_disk_backed_tmpdir)
            if [ -n "$_disk_tmp" ]; then
                BUILD_TMPDIR="$_disk_tmp/ledmatrix-build"
                # If this fails (a nearly-full disk being the likely cause on
                # exactly the devices this targets), fall back to the default
                # rather than pointing the build at a path that does not exist.
                if mkdir -p "$BUILD_TMPDIR" 2>/dev/null; then
                    echo "Building in $BUILD_TMPDIR (TMPDIR is memory-backed; keeping the build tree on disk)"
                else
                    echo "⚠ Could not create $BUILD_TMPDIR; falling back to the default TMPDIR"
                    BUILD_TMPDIR=""
                fi
            fi
        fi
        CURRENT_STEP="Build and install rpi-rgb-led-matrix"
        pushd "$PROJECT_ROOT_DIR/rpi-rgb-led-matrix-master" >/dev/null
        echo "Installing rpi-rgb-led-matrix Python package (scikit-build-core + cmake)..."
        echo "  Build deps required: python-dev-is-python3 cmake"
-        echo "  This compiles C++ — may take 2-5 minutes on Pi 4/5..."
+        echo "  Compiling C++ with ${BUILD_JOBS} parallel job(s)..."
        if [ "$BUILD_JOBS" -le 1 ]; then
            echo "  Deliberately serial to stay within this device's memory — expect 15-25 minutes."
        else
            echo "  This may take 2-5 minutes on a Pi 4/5..."
        fi
        BUILD_OUTPUT=$(mktemp)
        BUILD_SUCCESS=false
-        if python3 -m pip install --break-system-packages . > "$BUILD_OUTPUT" 2>&1; then
+        if run_rgbmatrix_build "$BUILD_JOBS" "$BUILD_OUTPUT"; then
            BUILD_SUCCESS=true
        fi
        cat "$BUILD_OUTPUT" >> "$LOG_FILE"
        if [ "$BUILD_SUCCESS" != true ]; then
-            echo "✗ Failed to install rpi-rgb-led-matrix Python package"
+            print_rgbmatrix_build_failure "$BUILD_OUTPUT"
            echo "  Ensure build tools are installed:"
            echo "  sudo apt install -y python-dev-is-python3 cmake build-essential"
            echo ""
            echo "-- Last 50 lines of build output --"
            tail -n 50 "$BUILD_OUTPUT"
            rm -f "$BUILD_OUTPUT"
            if [ -n "$BUILD_TMPDIR" ]; then rm -rf "$BUILD_TMPDIR"; fi
            popd >/dev/null
            lm_remove_build_swap
            exit 1
        fi
        rm -f "$BUILD_OUTPUT"
        if [ -n "$BUILD_TMPDIR" ]; then rm -rf "$BUILD_TMPDIR"; fi
        popd >/dev/null
        # Hand the memory back well before Step 14's reboot.
        lm_remove_build_swap
    else
        echo "✗ rpi-rgb-led-matrix-master directory not found at $PROJECT_ROOT_DIR"
        echo "Failed to initialize submodule or clone repository"
@@ -156,9 +156,13 @@ echo ""
 echo "6. Check disk space - building packages requires temporary space"
 echo "   df -h"
 echo ""
-echo "7. For slow builds, increase swap space:"
+echo "7. For slow builds or out-of-memory kills, increase swap space."
 echo "   first_time_install.sh already adds temporary swap on low-memory devices;"
 echo "   this makes it permanent. Set CONF_MAXSWAP too - it defaults to 2048 and"
 echo "   silently clamps CONF_SWAPSIZE, so raising CONF_SWAPSIZE alone does nothing."
 echo "   sudo dphys-swapfile swapoff"
-echo "   sudo nano /etc/dphys-swapfile  # Set CONF_SWAPSIZE=2048"
+echo "   sudo sed -i 's/^#\\?CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile"
 echo "   sudo sed -i 's/^#\\?CONF_MAXSWAP=.*/CONF_MAXSWAP=2048/' /etc/dphys-swapfile"
 echo "   sudo dphys-swapfile setup"
 echo "   sudo dphys-swapfile swapon"
 echo ""
@@ -0,0 +1,283 @@
 #!/bin/bash
 #
 # Low-memory build helpers for the LED Matrix installer.
 #
 # Sourced by first_time_install.sh. These live in a separate, sourceable file
 # so the pure sizing/detection functions can be unit-tested
 # (test/test_install_lowmem.py); first_time_install.sh itself is not sourceable
 # because it self-elevates and runs top to bottom.
 #
 # Why this exists: the rgbmatrix build compiles ~45 C++ translation units, two
 # of them Cython-generated (a single cc1plus on those peaks around 400-800MB at
 # -O3). Upstream's pyproject.toml sets no [tool.scikit-build] options, so
 # scikit-build-core uses Ninja at its default of nproc+2 jobs -- six concurrent
 # compiles on a 4-core Pi. On a 512MB-1GB Pi the OOM killer reaps cc1plus and
 # pip reports only "Failed building wheel for rgbmatrix".
 #
 # The caller runs under `set -Eeuo pipefail` with an ERR trap, and these are
 # invoked from the middle of numbered steps, so nothing here may call exit and
 # the swap helpers must always return 0.
 # Overridable so tests can point at fixture files instead of /proc.
 LM_MEMINFO="${LM_MEMINFO:-/proc/meminfo}"
 LM_SWAPS="${LM_SWAPS:-/proc/swaps}"
 # Temporary swapfile created for the build and removed afterwards. Deliberately
 # never added to /etc/fstab: a malformed fstab can leave a novice with an
 # unbootable Pi, and this swap only needs to outlive the compile.
 LM_SWAPFILE="${LM_SWAPFILE:-/var/swap.ledmatrix-install}"
 # Bring RAM + real swap up to this much before compiling, capped per swapfile.
 LM_SWAP_TARGET_MB="${LM_SWAP_TARGET_MB:-3072}"
 LM_SWAP_MAX_MB="${LM_SWAP_MAX_MB:-2048}"
 # Worst-case cc1plus footprint on the Cython translation unit, used to size
 # build parallelism against available RAM.
 LM_MB_PER_JOB="${LM_MB_PER_JOB:-768}"
 # Set to 1 once swap is live, so lm_remove_build_swap (wired up as an EXIT
 # trap) knows whether there is anything to undo.
 LM_TEMP_SWAP_ACTIVE=0
 # Human-readable reason no swapfile was created, quoted back in the failure
 # message so a user who still OOMs is told why the safety net was absent.
 LM_SWAP_SKIP_REASON=""
 # ---------------------------------------------------------------------------
 # Pure helpers (no side effects; unit-tested)
 # ---------------------------------------------------------------------------
 # Total physical RAM in MB, or 0 if it cannot be determined.
 lm_total_ram_mb() {
    # Defaults are re-resolved here as well as at source time so the function
    # stays safe under the installer's `set -u`.
    awk '/^MemTotal:/ {printf "%d\n", $2 / 1024; found = 1; exit} END {if (!found) print 0}' \
        "${LM_MEMINFO:-/proc/meminfo}" 2>/dev/null || echo 0
 }
 # Total swap in MB, EXCLUDING zram devices.
 #
 # zram swap is compressed RAM: it consumes the very resource that is already
 # exhausted and does nothing for a build OOM. Counting it would let a
 # zram-enabled image decide it has enough swap and then fail exactly as before.
 lm_total_swap_mb() {
    awk 'NR > 1 && $1 !~ /^\/dev\/zram/ {total += $3} END {printf "%d\n", total / 1024}' \
        "${LM_SWAPS:-/proc/swaps}" 2>/dev/null || echo 0
 }
 # lm_build_jobs <ram_mb> <cores> -> max(1, min(cores, ram_mb / LM_MB_PER_JOB))
 #
 # Computed from RAM alone and never RAM+swap: handing out extra jobs because
 # swap exists just guarantees SD-card thrash, which is far slower than
 # compiling serially.
 lm_build_jobs() {
    local ram_mb="${1:-0}" cores="${2:-1}" jobs
    local per_job="${LM_MB_PER_JOB:-768}"
    if [ "$cores" -lt 1 ]; then
        cores=1
    fi
    jobs=$(( ram_mb / per_job ))
    if [ "$jobs" -lt 1 ]; then
        jobs=1
    fi
    if [ "$jobs" -gt "$cores" ]; then
        jobs="$cores"
    fi
    echo "$jobs"
 }
 # lm_swap_needed_mb <ram_mb> <existing_swap_mb> -> swapfile size in MB, or 0.
 #
 # Brings RAM + real swap up to LM_SWAP_TARGET_MB, capped at LM_SWAP_MAX_MB and
 # rounded up to a 256MB multiple. Machines with enough memory get 0 and are
 # left completely untouched.
 lm_swap_needed_mb() {
    local ram_mb="${1:-0}" swap_mb="${2:-0}" needed
    local target="${LM_SWAP_TARGET_MB:-3072}" max="${LM_SWAP_MAX_MB:-2048}"
    needed=$(( target - ram_mb - swap_mb ))
    if [ "$needed" -le 0 ]; then
        echo 0
        return 0
    fi
    if [ "$needed" -gt "$max" ]; then
        needed="$max"
    fi
    echo $(( ( (needed + 255) / 256 ) * 256 ))
 }
 # lm_build_failed_on_oom <build_output_file> -> 0 if the build was OOM-killed.
 #
 # Two independent evidence sources, because neither alone is reliable: the
 # compiler sometimes reports its own allocation failure, but when the kernel
 # OOM killer fires it writes nothing to the build's stdout. That silence is
 # exactly why the old handler misdiagnosed this as missing build tools.
 lm_build_failed_on_oom() {
    local build_output="${1:-}" kernel_log=""
    if [ -n "$build_output" ] && [ -f "$build_output" ]; then
        if grep -qiE 'cc1plus: out of memory|virtual memory exhausted|Cannot allocate memory|MemoryError|fatal error: Killed signal terminated program|signal 9' \
            "$build_output"; then
            return 0
        fi
    fi
    # LM_KERNEL_LOG_FILE lets tests supply a fixture instead of the real kernel
    # ring buffer, which on a shared CI machine may hold unrelated OOM events.
    if [ -n "${LM_KERNEL_LOG_FILE:-}" ]; then
        if [ -f "$LM_KERNEL_LOG_FILE" ]; then
            kernel_log=$(cat "$LM_KERNEL_LOG_FILE" 2>/dev/null || true)
        fi
    elif command -v dmesg >/dev/null 2>&1; then
        kernel_log=$(dmesg -T 2>/dev/null || dmesg 2>/dev/null || true)
    fi
    if [ -z "$kernel_log" ] && [ -z "${LM_KERNEL_LOG_FILE:-}" ] && command -v journalctl >/dev/null 2>&1; then
        kernel_log=$(journalctl -k --since "30 min ago" --no-pager 2>/dev/null || true)
    fi
    if [ -n "$kernel_log" ]; then
        if printf '%s\n' "$kernel_log" | tail -n 300 | \
            grep -qiE 'Out of memory: Kill|oom_kill|oom-kill|Killed process'; then
            return 0
        fi
    fi
    return 1
 }
 # lm_disk_backed_tmpdir [candidate] -> a disk-backed temp dir, or nothing.
 #
 # pip builds in $TMPDIR. Debian 13 mounts /tmp as tmpfs, so the default puts the
 # whole C++ build tree in RAM, competing with the compiler we are already trying
 # to keep under the limit. Prints a replacement only when the current TMPDIR is
 # memory-backed and the candidate is not; otherwise prints nothing and the
 # caller keeps its default.
 lm_disk_backed_tmpdir() {
    local candidate="${1:-/var/tmp}"
    local current="${TMPDIR:-/tmp}"
    local current_fs="" candidate_fs=""
    current_fs=$(lm_fstype_of "$current")
    case "$current_fs" in
        tmpfs|ramfs) ;;
        *) return 0 ;;
    esac
    candidate_fs=$(lm_fstype_of "$candidate")
    case "$candidate_fs" in
        tmpfs|ramfs|"") return 0 ;;
    esac
    echo "$candidate"
 }
 # Filesystem type backing a path, or empty if it cannot be determined.
 lm_fstype_of() {
    local path="${1:-/}"
    if command -v findmnt >/dev/null 2>&1; then
        findmnt -no FSTYPE --target "$path" 2>/dev/null | head -n 1
        return 0
    fi
    if command -v stat >/dev/null 2>&1; then
        stat -f -c %T "$path" 2>/dev/null | head -n 1
        return 0
    fi
    return 0
 }
 # ---------------------------------------------------------------------------
 # Swap management (requires root; not unit-tested)
 # ---------------------------------------------------------------------------
 # lm_ensure_build_swap <needed_mb>
 #
 # Always returns 0. On any refusal it sets LM_SWAP_SKIP_REASON and leaves the
 # system untouched -- swap is a safety net for the build, never a precondition.
 lm_ensure_build_swap() {
    local needed_mb="${1:-0}"
    local swap_dir free_mb budget
    LM_SWAP_SKIP_REASON=""
    if [ "$needed_mb" -le 0 ]; then
        LM_SWAP_SKIP_REASON="not needed (RAM and existing swap are sufficient)"
        return 0
    fi
    if ! command -v mkswap >/dev/null 2>&1 || ! command -v swapon >/dev/null 2>&1; then
        LM_SWAP_SKIP_REASON="mkswap/swapon are not available on this system"
        echo "⚠ Cannot add build swap: $LM_SWAP_SKIP_REASON"
        return 0
    fi
    # Clear a stale swapfile left by a run that was killed before its cleanup
    # ran, so this is safe to call repeatedly.
    if [ -e "$LM_SWAPFILE" ]; then
        echo "Removing a leftover swapfile from a previous run: $LM_SWAPFILE"
        swapoff "$LM_SWAPFILE" >/dev/null 2>&1 || true
        rm -f "$LM_SWAPFILE" || true
    fi
    # Keep a working margin for the build tree itself; never eat the last GB.
    swap_dir=$(dirname "$LM_SWAPFILE")
    free_mb=$(df -m "$swap_dir" 2>/dev/null | awk 'NR==2{print $4}')
    free_mb=${free_mb:-0}
    budget=$(( free_mb - 1024 ))
    if [ "$budget" -lt 256 ]; then
        LM_SWAP_SKIP_REASON="only ${free_mb}MB free on ${swap_dir}, need about $(( needed_mb + 1024 ))MB"
        echo "⚠ Skipping the build swapfile: $LM_SWAP_SKIP_REASON"
        return 0
    fi
    if [ "$needed_mb" -gt "$budget" ]; then
        echo "⚠ Trimming the build swapfile from ${needed_mb}MB to leave 1GB free on ${swap_dir}"
        needed_mb=$(( ( budget / 256 ) * 256 ))
    fi
    echo "Adding a temporary ${needed_mb}MB swapfile for the build: $LM_SWAPFILE"
    echo "  This is removed automatically once the build finishes."
    # fallocate can produce a sparse file that mkswap rejects, and is not
    # supported on every filesystem; dd always yields a usable file.
    if ! fallocate -l "${needed_mb}M" "$LM_SWAPFILE" 2>/dev/null; then
        if ! dd if=/dev/zero of="$LM_SWAPFILE" bs=1M count="$needed_mb" status=none 2>/dev/null; then
            LM_SWAP_SKIP_REASON="could not allocate ${needed_mb}MB at $LM_SWAPFILE"
            echo "⚠ $LM_SWAP_SKIP_REASON"
            rm -f "$LM_SWAPFILE" || true
            return 0
        fi
    fi
    chmod 600 "$LM_SWAPFILE" || true
    if ! mkswap "$LM_SWAPFILE" >/dev/null 2>&1; then
        LM_SWAP_SKIP_REASON="mkswap failed on $LM_SWAPFILE"
        echo "⚠ $LM_SWAP_SKIP_REASON"
        rm -f "$LM_SWAPFILE" || true
        return 0
    fi
    if ! swapon "$LM_SWAPFILE" >/dev/null 2>&1; then
        LM_SWAP_SKIP_REASON="swapon failed on $LM_SWAPFILE"
        echo "⚠ $LM_SWAP_SKIP_REASON"
        rm -f "$LM_SWAPFILE" || true
        return 0
    fi
    LM_TEMP_SWAP_ACTIVE=1
    echo "✓ Temporary build swap active (${needed_mb}MB; total swap is now $(lm_total_swap_mb)MB)"
    return 0
 }
 # Remove the temporary swapfile. Safe to call unconditionally and repeatedly.
 #
 # Wired up as an EXIT trap, so it must never return non-zero -- a failing trap
 # would surface as a spurious installer error.
 lm_remove_build_swap() {
    if [ "${LM_TEMP_SWAP_ACTIVE:-0}" != "1" ]; then
        return 0
    fi
    LM_TEMP_SWAP_ACTIVE=0
    echo "Removing the temporary build swapfile: $LM_SWAPFILE"
    swapoff "$LM_SWAPFILE" >/dev/null 2>&1 || true
    rm -f "$LM_SWAPFILE" || true
    return 0
 }
@@ -145,6 +145,34 @@ check_disk_space() {
    fi
 }
 # Report available memory so the user knows what to expect before the wait.
 #
 # Informational only — first_time_install.sh does the real work of capping
 # build parallelism and adding temporary swap. Never fatal: a low-RAM Pi is
 # supported, it is just slower.
 check_memory() {
    CURRENT_STEP="Memory check"
    if [ ! -r /proc/meminfo ]; then
        print_warning "Cannot read /proc/meminfo, skipping memory check"
        return 0
    fi
    TOTAL_RAM_MB=$(awk '/^MemTotal:/ {printf "%d\n", $2 / 1024; exit}' /proc/meminfo 2>/dev/null || echo 0)
    TOTAL_RAM_MB=${TOTAL_RAM_MB:-0}
    if [ "$TOTAL_RAM_MB" -eq 0 ]; then
        print_warning "Could not determine system memory, continuing"
    elif [ "$TOTAL_RAM_MB" -lt 2048 ]; then
        print_warning "Low memory: ${TOTAL_RAM_MB}MB RAM"
        echo "  The rpi-rgb-led-matrix C++ build needs more memory than this Pi has."
        echo "  The installer will compile with fewer parallel jobs and add a temporary"
        echo "  swapfile for the build, removing it afterwards. That step will take"
        echo "  15-25 minutes rather than the usual 2-5."
    else
        print_success "Memory sufficient: ${TOTAL_RAM_MB}MB RAM"
    fi
 }
 # Ensure sudo access
 check_sudo() {
    CURRENT_STEP="Sudo access check"
@@ -204,7 +232,7 @@ main() {
    print_step "LED Matrix One-Shot Installation"
    echo "This script will:"
-    echo "  1. Check prerequisites (network, disk space, sudo)"
+    echo "  1. Check prerequisites (network, disk space, memory, sudo)"
    echo "  2. Install system dependencies (git, python3, build tools)"
    echo "  3. Clone the LEDMatrix repository"
    echo "  4. Run the first-time installation script"
@@ -213,6 +241,7 @@ main() {
    # Check prerequisites
    check_network
    check_disk_space
    check_memory
    check_sudo
    # Note: /tmp permissions are checked and fixed inline before running first_time_install.sh
    # (only if actually wrong, not preemptively)
@@ -228,12 +257,14 @@ main() {
        exit 1
    fi
-    # Update package list first
+    # Update package list first. first_time_install.sh is told the lists are
    # already fresh so it does not repeat this a minute later.
    if [ "$EUID" -eq 0 ]; then
        retry apt-get update -qq
    else
        retry sudo apt-get update -qq
    fi
    export LEDMATRIX_APT_UPDATED=1
    # Install git and curl (needed for cloning and the script itself)
    if ! command -v git >/dev/null 2>&1 || ! command -v curl >/dev/null 2>&1; then
@@ -372,7 +403,12 @@ main() {
        # Pass both -y flag AND environment variable for non-interactive mode
        # This ensures it works even if the script re-executes itself with sudo
        # Also ensure stdin is properly handled for non-interactive mode
-        sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 bash ./first_time_install.sh -y </dev/null
+        # LEDMATRIX_APT_UPDATED is passed explicitly rather than relying on
        # -E: a sudoers env_reset/env_keep policy can strip exported variables,
        # which would silently reinstate the duplicate apt update.
        sudo -E env TMPDIR=/tmp LEDMATRIX_ASSUME_YES=1 \
            LEDMATRIX_APT_UPDATED="${LEDMATRIX_APT_UPDATED:-0}" \
            bash ./first_time_install.sh -y </dev/null
    fi
    INSTALL_EXIT_CODE=$?
    trap 'on_error $LINENO' ERR  # Re-enable ERR trap
@@ -0,0 +1,209 @@
 """
 Tests for scripts/install/lib_lowmem.sh, the installer's low-memory helpers.
 Background: the rgbmatrix build compiles ~45 C++ translation units, two of them
 Cython-generated. Upstream's pyproject.toml sets no [tool.scikit-build] options,
 so scikit-build-core drives Ninja at its default of nproc+2 jobs -- six
 concurrent cc1plus on a 4-core Pi. On 512MB and 1GB models the OOM killer reaps
 the compiler and pip reports only "Failed building wheel for rgbmatrix", which
 the installer used to misreport as a missing-build-tools problem.
 These cover the pure sizing/detection functions. The swap-management functions
 need root and mutate the system, so they are exercised manually instead.
 """
 import subprocess
 from pathlib import Path
 import pytest
 LIB = Path(__file__).resolve().parent.parent / "scripts" / "install" / "lib_lowmem.sh"
 def run_lib(snippet: str, env: dict | None = None) -> subprocess.CompletedProcess:
    """Source the helper library and run a snippet against it."""
    script = f". {LIB}\n{snippet}"
    return subprocess.run(
        ["bash", "-c", script],
        capture_output=True,
        text=True,
        env={"PATH": "/usr/bin:/bin:/usr/sbin:/sbin", **(env or {})},
    )
 def call(fn: str, *args: object, env: dict | None = None) -> str:
    joined = " ".join(str(a) for a in args)
    result = run_lib(f"{fn} {joined}", env=env)
    assert result.returncode == 0, f"{fn} failed: {result.stderr}"
    return result.stdout.strip()
 class TestLibraryLoads:
    def test_library_exists_and_is_syntactically_valid(self):
        assert LIB.is_file(), f"{LIB} is missing"
        result = subprocess.run(["bash", "-n", str(LIB)], capture_output=True, text=True)
        assert result.returncode == 0, result.stderr
    def test_sourcing_is_safe_under_strict_mode(self):
        # first_time_install.sh runs under `set -Eeuo pipefail` with an ERR
        # trap, so sourcing must not trip either.
        result = run_lib("set -Eeuo pipefail\ntrap 'exit 99' ERR\necho ok")
        assert result.returncode == 0, result.stderr
        assert "ok" in result.stdout
 class TestBuildJobs:
    @pytest.mark.parametrize(
        "ram_mb,cores,expected",
        [
            (512, 4, 1),  # Pi Zero 2 W - must serialize
            (1024, 4, 1),  # Pi 3B/3B+ - the device from the bug report
            (2048, 4, 2),
            (4096, 4, 4),  # core-capped
            (8192, 4, 4),  # core-capped
            (2048, 1, 1),  # single-core machine
        ],
    )
    def test_jobs_scale_with_ram_and_cap_at_cores(self, ram_mb, cores, expected):
        assert call("lm_build_jobs", ram_mb, cores) == str(expected)
    def test_never_returns_zero_jobs(self):
        assert call("lm_build_jobs", 0, 4) == "1"
    def test_treats_zero_cores_as_one(self):
        assert call("lm_build_jobs", 8192, 0) == "1"
 class TestSwapSizing:
    @pytest.mark.parametrize(
        "ram_mb,existing_swap_mb,expected",
        [
            (512, 0, 2048),  # capped at LM_SWAP_MAX_MB
            (1024, 0, 2048),  # matches the workaround the reporter found
            (2048, 0, 1024),
            (2048, 1024, 0),  # existing swap already covers it
            (4096, 0, 0),  # untouched on machines that already work
            (8192, 0, 0),
        ],
    )
    def test_swap_target_scales_with_ram(self, ram_mb, existing_swap_mb, expected):
        assert call("lm_swap_needed_mb", ram_mb, existing_swap_mb) == str(expected)
    def test_result_is_a_multiple_of_256mb(self):
        # 3072 - 900 = 2172, which must round up rather than produce an odd size.
        assert int(call("lm_swap_needed_mb", 900, 0)) % 256 == 0
 class TestSwapDetection:
    def test_zram_swap_is_excluded(self, tmp_path):
        # zram swap is compressed RAM: counting it would let a zram-enabled
        # image skip provisioning and then OOM exactly as before.
        swaps = tmp_path / "swaps"
        swaps.write_text(
            "Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority\n"
            "/dev/zram0                              partition\t1048572\t\t0\t\t100\n"
            "/var/swap                               file\t\t524284\t\t0\t\t-2\n"
        )
        assert call("lm_total_swap_mb", env={"LM_SWAPS": str(swaps)}) == "511"
    def test_zram_only_system_reports_no_usable_swap(self, tmp_path):
        swaps = tmp_path / "swaps"
        swaps.write_text(
            "Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority\n"
            "/dev/zram0                              partition\t1048572\t\t0\t\t100\n"
        )
        assert call("lm_total_swap_mb", env={"LM_SWAPS": str(swaps)}) == "0"
    def test_ram_is_read_from_meminfo(self, tmp_path):
        meminfo = tmp_path / "meminfo"
        # A real Pi 3B+ reports this; 948204/1024 truncates to 925.
        meminfo.write_text("MemTotal:         948204 kB\nMemFree:  123456 kB\n")
        assert call("lm_total_ram_mb", env={"LM_MEMINFO": str(meminfo)}) == "925"
    def test_missing_files_report_zero_rather_than_failing(self, tmp_path):
        missing = str(tmp_path / "nope")
        assert call("lm_total_ram_mb", env={"LM_MEMINFO": missing}) == "0"
        assert call("lm_total_swap_mb", env={"LM_SWAPS": missing}) == "0"
 class TestOomDetection:
    """The regression tests for the misdiagnosis in the bug report."""
    def _check(self, tmp_path, build_log: str, kernel_log: str = "") -> bool:
        build_file = tmp_path / "build.log"
        build_file.write_text(build_log)
        kernel_file = tmp_path / "kernel.log"
        kernel_file.write_text(kernel_log)
        result = run_lib(
            f"lm_build_failed_on_oom {build_file}",
            env={"LM_KERNEL_LOG_FILE": str(kernel_file)},
        )
        return result.returncode == 0
    @pytest.mark.parametrize(
        "line",
        [
            "c++: fatal error: Killed signal terminated program cc1plus",
            "cc1plus: out of memory allocating 65536 bytes",
            "virtual memory exhausted: Cannot allocate memory",
            "error: command '/usr/bin/c++' died with signal 9",
        ],
    )
    def test_detects_compiler_reported_memory_failures(self, tmp_path, line):
        log = f"[15/45] Building CXX object core.cpp.o\n{line}\nninja: build stopped.\n"
        assert self._check(tmp_path, log) is True
    def test_detects_oom_visible_only_in_the_kernel_log(self, tmp_path):
        # The OOM killer writes nothing to the build's stdout. This silence is
        # precisely why the old handler blamed missing build tools.
        build_log = (
            "[15/45] Building CXX object core.cpp.o\n"
            "ninja: build stopped: subcommand failed.\n"
            "ERROR: Failed building wheel for rgbmatrix\n"
        )
        kernel_log = (
            "[12345.6] Out of memory: Killed process 4242 (cc1plus) "
            "total-vm:812345kB, anon-rss:764000kB\n"
        )
        assert self._check(tmp_path, build_log, kernel_log) is True
    def test_does_not_flag_a_genuine_missing_build_tool(self, tmp_path):
        build_log = (
            "CMake Error at CMakeLists.txt:12 (find_package):\n"
            "  Could NOT find Python (missing: Development.Module)\n"
            "fatal error: Python.h: No such file or directory\n"
            "ERROR: Failed building wheel for rgbmatrix\n"
        )
        assert self._check(tmp_path, build_log) is False
    def test_does_not_flag_a_network_failure(self, tmp_path):
        build_log = (
            "WARNING: Retrying after connection broken by 'NewConnectionError'\n"
            "ERROR: Could not install packages due to an OSError\n"
        )
        assert self._check(tmp_path, build_log) is False
    def test_handles_a_missing_build_log(self, tmp_path):
        kernel_file = tmp_path / "kernel.log"
        kernel_file.write_text("")
        result = run_lib(
            f"lm_build_failed_on_oom {tmp_path / 'absent.log'}",
            env={"LM_KERNEL_LOG_FILE": str(kernel_file)},
        )
        assert result.returncode == 1
 class TestDiskBackedTmpdir:
    def test_returns_nothing_when_tmpdir_is_already_disk_backed(self, tmp_path):
        # tmp_path is on the regular filesystem, so the default must be kept.
        assert call("lm_disk_backed_tmpdir", env={"TMPDIR": str(tmp_path)}) == ""
    def test_redirects_away_from_a_memory_backed_tmpdir(self):
        # Debian 13 mounts /tmp as tmpfs, which would otherwise hold the whole
        # C++ build tree in RAM alongside the compiler.
        shm = Path("/dev/shm")
        if not shm.is_dir():
            pytest.skip("/dev/shm not available")
        result = run_lib("lm_disk_backed_tmpdir", env={"TMPDIR": str(shm)})
        assert result.returncode == 0
        assert result.stdout.strip() in ("", "/var/tmp")