fix(vegas): eliminate plugin re-appearance at scroll cycle boundaries (#327)

* fix(vegas): eliminate plugin re-appearance at scroll cycle boundaries

The Vegas scroll image is wider than the display. scroll_helper marks a
cycle complete only after total_distance_scrolled >= total_scroll_width +
display_width, meaning it keeps scrolling for an extra display_width of
pixels after all content has exited left. During that extra travel the
scroll_position wraps back to ~0 and the first plugin re-enters from the
right - visible for ~2-3 seconds as a plugin partially displaying before
the next one starts.

render_pipeline.render_frame(): end the cycle the moment
total_distance_scrolled >= total_scroll_width (the natural wrap point),
before any second-pass content becomes visible. Push a blank frame
immediately on detection so hardware never shows a frozen content
snapshot while start_new_cycle() recomposes (~100 ms).

display_manager.py: add capture_mode() context manager. When active,
update_display() and the canvas clear in clear() skip the hardware
write, preventing plugins that call update_display() internally from
flashing on the matrix during off-screen content capture inside
start_new_cycle().

plugin_adapter.py: wrap all plugin.display() calls in
_capture_display_content() and _trigger_scroll_content_generation()
with capture_mode() so the fallback capture path never produces
hardware output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(vegas): tighten exception handling in clear() and blank-frame push

display_manager.clear(): replace bare except/pass on the three hardware
Clear() calls with (RuntimeError, OSError) and a logger.error() so
failures are visible in logs rather than silently swallowed.  Still
non-fatal — the PIL image buffer is already black before these calls,
so the next update_display() will push clean content regardless.

render_pipeline.render_frame(): replace broad except/pass in the
blank-frame push with (ImportError, ValueError, TypeError, MemoryError)
and a logger.error() that includes display dimensions for context.
update_display() already handles its own hardware errors internally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(vegas): catch OSError and RuntimeError in blank-frame push

Image.new() can raise OSError in some PIL environments and hardware
libraries may surface RuntimeError on I/O failures.  Add both to the
exception tuple alongside the existing ImportError/ValueError/TypeError/
MemoryError so no boundary failure escapes the local handler.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

first_time_install.sh

@@ -1086,6 +1086,7 @@ SYSTEMCTL_PATH=$(which systemctl)
 REBOOT_PATH=$(which reboot)
 POWEROFF_PATH=$(which poweroff)
 BASH_PATH=$(which bash)
+JOURNALCTL_PATH=$(which journalctl 2>/dev/null || true)
 
 # Create sudoers content
 cat > /tmp/ledmatrix_web_sudoers << EOF
@@ -1101,10 +1102,23 @@ $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix.service
 $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH enable ledmatrix.service
 $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH disable ledmatrix.service
 $ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service
 $ACTUAL_USER ALL=(ALL) NOPASSWD: $PYTHON_PATH $PROJECT_ROOT_DIR/display_controller.py
 $ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/start_display.sh
 $ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/stop_display.sh
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $BASH_PATH $PROJECT_ROOT_DIR/scripts/fix_perms/safe_plugin_rm.sh *
 EOF
+if [ -n "$JOURNALCTL_PATH" ]; then
+    cat >> /tmp/ledmatrix_web_sudoers << EOF
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -u ledmatrix.service *
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -u ledmatrix *
+$ACTUAL_USER ALL=(ALL) NOPASSWD: $JOURNALCTL_PATH -t ledmatrix *
+EOF
+fi
 
 if [ -f "$SUDOERS_FILE" ] && cmp -s /tmp/ledmatrix_web_sudoers "$SUDOERS_FILE"; then
     echo "Sudoers configuration already up to date"

plugin-repos/march-madness/requirements.txt

@@ -1,4 +1,4 @@
 requests>=2.28.0
-Pillow>=9.1.0
+Pillow>=12.2.0
 pytz>=2022.1
 numpy>=1.24.0

scripts/download_pixlet.sh

@@ -118,7 +118,7 @@ total_count=${#ARCHITECTURES[@]}
 
 for arch in "${!ARCHITECTURES[@]}"; do
     if download_binary "$arch" "${ARCHITECTURES[$arch]}"; then
-        ((success_count++))
+        success_count=$((success_count + 1))
     fi
 done
 

scripts/install/configure_web_sudo.sh

@@ -89,9 +89,9 @@ TEMP_SUDOERS="/tmp/ledmatrix_web_sudoers_$$"
     echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH status ledmatrix.service"
     echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix"
     echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH is-active ledmatrix.service"
-    echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web"
+    echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH start ledmatrix-web.service"
-    echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web"
+    echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH stop ledmatrix-web.service"
-    echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web"
+    echo "$WEB_USER ALL=(ALL) NOPASSWD: $SYSTEMCTL_PATH restart ledmatrix-web.service"
 
     # Optional: journalctl (non-critical — skip if not found)
     if [ -n "$JOURNALCTL_PATH" ]; then

scripts/utils/wifi_monitor_daemon.py

@@ -10,6 +10,7 @@ import sys
 import time
 import logging
 import signal
+import subprocess
 from pathlib import Path
 
 # Add project root to path (parent of scripts/utils/)
@@ -146,12 +147,18 @@ class WiFiMonitorDaemon:
                                     capture_output=True, timeout=20, check=True
                                 )
                                 self._consecutive_internet_failures = 0
+                                # NM restart causes a brief WiFi drop; reset the AP-mode grace
+                                # counter so that transient disconnect doesn't count toward
+                                # triggering AP mode.
+                                self.wifi_manager._disconnected_checks = 0
                             except subprocess.CalledProcessError as e:
                                 logger.error(f"NetworkManager restart failed (rc={e.returncode}); "
-                                             "keeping failure counter unchanged")
+                                             "resetting failure counter to avoid tight retry loop")
-                            except Exception as e:
+                                self._consecutive_internet_failures = 0
+                            except (subprocess.SubprocessError, OSError) as e:
                                 logger.error(f"NetworkManager restart error: {e}; "
-                                             "keeping failure counter unchanged")
+                                             "resetting failure counter to avoid tight retry loop")
+                                self._consecutive_internet_failures = 0
                     else:
                         self._consecutive_internet_failures = 0
                 else:

src/display_manager.py

@@ -3,6 +3,7 @@ if os.getenv("EMULATOR", "false") == "true":
     from RGBMatrixEmulator import RGBMatrix, RGBMatrixOptions
 else:
     from rgbmatrix import RGBMatrix, RGBMatrixOptions
+from contextlib import contextmanager
 from PIL import Image, ImageDraw, ImageFont
 import time
 from typing import Dict, Any, List, Tuple
@@ -28,6 +29,8 @@ class DisplayManager:
         self.config = config or {}
         self._force_fallback = force_fallback
         self._suppress_test_pattern = suppress_test_pattern
+        # When True, update_display() and clear() skip hardware writes (used during off-screen content capture)
+        self._capture_mode_active = False
         # Snapshot settings for web preview integration (service writes, web reads)
         self._snapshot_path = "/tmp/led_matrix_preview.png"
         self._snapshot_min_interval_sec = 0.2  # max ~5 fps
@@ -255,6 +258,22 @@ class DisplayManager:
         except Exception as e:
             logger.error(f"Error drawing test pattern: {e}", exc_info=True)
 
+    @contextmanager
+    def capture_mode(self):
+        """Suppress hardware output during off-screen content capture.
+
+        Plugins call update_display() as part of their normal display() flow.
+        When fetching content for Vegas mode the render loop is still running,
+        so any incidental hardware write causes a visible flash on the matrix.
+        Entering this context prevents those writes without affecting the PIL
+        image buffer, which the adapter reads to extract content.
+        """
+        self._capture_mode_active = True
+        try:
+            yield
+        finally:
+            self._capture_mode_active = False
+
     def update_display(self):
         """Update the display using double buffering with proper sync."""
         try:
@@ -265,6 +284,9 @@ class DisplayManager:
                 self._write_snapshot_if_due()
                 return
 
+            if self._capture_mode_active:
+                return  # Skip hardware write — content is being captured off-screen
+
             # Copy the current image to the offscreen canvas
             self.offscreen_canvas.SetImage(self.image)
 
@@ -305,20 +327,22 @@ class DisplayManager:
             self.image = Image.new('RGB', (self.matrix.width, self.matrix.height))
             self.draw = ImageDraw.Draw(self.image)
 
-            # Clear both canvases and the underlying matrix to ensure no artifacts
+            if not self._capture_mode_active:
+                # Clear both canvases and the underlying matrix to ensure no artifacts.
+                # Failures are non-fatal — the image buffer is already black above, so
+                # the next update_display() call will push clean content regardless.
                 try:
                     self.offscreen_canvas.Clear()
-            except Exception:
+                except (RuntimeError, OSError) as e:
-                pass
+                    logger.error("Failed to clear offscreen canvas: %s", e)
                 try:
                     self.current_canvas.Clear()
-            except Exception:
+                except (RuntimeError, OSError) as e:
-                pass
+                    logger.error("Failed to clear current canvas: %s", e)
                 try:
-                # Extra safety: clear the matrix front buffer as well
                     self.matrix.Clear()
-            except Exception:
+                except (RuntimeError, OSError) as e:
-                pass
+                    logger.error("Failed to clear matrix front buffer: %s", e)
             
             # Note: We do NOT call update_display() here to avoid black flashes.
             # The caller should call update_display() after drawing new content.

src/vegas_mode/plugin_adapter.py

@@ -329,6 +329,7 @@ class PluginAdapter:
             # Save display state to restore after
             original_image = self.display_manager.image.copy()
 
+            with self.display_manager.capture_mode():
                 # Method 1: Try _create_scrolling_display (stocks pattern)
                 if hasattr(plugin, '_create_scrolling_display'):
                     logger.info(
@@ -408,10 +409,7 @@ class PluginAdapter:
             original_image = self.display_manager.image.copy()
             logger.info("[%s] Fallback: saved original display state", plugin_id)
 
-            # Lightweight in-memory data refresh before capturing.
+            # Ensure plugin has fresh data before capturing
-            # Full update() is intentionally skipped here — the background
-            # update tick in the Vegas coordinator handles periodic API
-            # refreshes so we don't block the content-fetch thread.
             has_update_data = hasattr(plugin, 'update_data')
             logger.info("[%s] Fallback: has update_data=%s", plugin_id, has_update_data)
             if has_update_data:
@@ -421,7 +419,9 @@ class PluginAdapter:
                 except (AttributeError, RuntimeError, OSError):
                     logger.exception("[%s] Fallback: update_data() failed", plugin_id)
 
-            # Clear and call plugin display
+            # Clear and call plugin display — use capture_mode to suppress hardware writes
+            # that plugins may trigger internally via update_display().
+            with self.display_manager.capture_mode():
                 self.display_manager.clear()
                 logger.info("[%s] Fallback: display cleared, calling display()", plugin_id)
 
@@ -436,6 +436,7 @@ class PluginAdapter:
 
                 # Capture the result
                 captured = self.display_manager.image.copy()
+
             logger.info(
                 "[%s] Fallback: captured frame %dx%d, mode=%s",
                 plugin_id, captured.width, captured.height, captured.mode
@@ -454,6 +455,7 @@ class PluginAdapter:
                     plugin_id
                 )
                 # Try once more with force_clear=True
+                with self.display_manager.capture_mode():
                     self.display_manager.clear()
                     plugin.display(force_clear=True)
                     captured = self.display_manager.image.copy()
@@ -585,28 +587,6 @@ class PluginAdapter:
             else:
                 self._content_cache.clear()
 
-    def invalidate_plugin_scroll_cache(self, plugin: 'BasePlugin', plugin_id: str) -> None:
-        """
-        Clear a plugin's scroll_helper cache so Vegas re-fetches fresh visuals.
-
-        Uses scroll_helper.clear_cache() to reset all cached state (cached_image,
-        cached_array, total_scroll_width, scroll_position, etc.) — not just the
-        image.  Without this, plugins that use scroll_helper (stocks, news,
-        odds-ticker, etc.) would keep serving stale scroll images even after
-        their data refreshes.
-
-        Args:
-            plugin: Plugin instance
-            plugin_id: Plugin identifier
-        """
-        scroll_helper = getattr(plugin, 'scroll_helper', None)
-        if scroll_helper is None:
-            return
-
-        if getattr(scroll_helper, 'cached_image', None) is not None:
-            scroll_helper.clear_cache()
-            logger.debug("[%s] Cleared scroll_helper cache", plugin_id)
-
     def get_content_type(self, plugin: 'BasePlugin', plugin_id: str) -> str:
         """
         Get the type of content a plugin provides.

src/vegas_mode/render_pipeline.py

@@ -202,8 +202,25 @@ class RenderPipeline:
             # Update scroll position
             self.scroll_helper.update_scroll_position()
 
-            # Check if cycle is complete
+            # Determine if the cycle is done.
-            if self.scroll_helper.is_scroll_complete():
+            #
+            # scroll_helper considers a cycle complete only after
+            # total_distance_scrolled >= total_scroll_width + display_width.
+            # That extra display_width of travel causes a "wrap-around" phase
+            # where scroll_position resets to ~0 and the first plugin's content
+            # re-enters from the right — the user sees this 2-3 s window as
+            # "a plugin partially displaying before the next one starts."
+            #
+            # We end the cycle as soon as total_distance_scrolled reaches
+            # total_scroll_width (the wrap-around point), before any second-pass
+            # content becomes visible.  scroll_helper.is_scroll_complete() is
+            # kept as a fallback for edge-cases where that threshold is skipped.
+            at_wrap_point = (
+                not self._cycle_complete and
+                self.scroll_helper.total_distance_scrolled >= self.scroll_helper.total_scroll_width
+            )
+
+            if at_wrap_point or self.scroll_helper.is_scroll_complete():
                 if not self._cycle_complete:
                     self._cycle_complete = True
                     self.stats['scroll_cycles'] += 1
@@ -211,6 +228,20 @@ class RenderPipeline:
                         "Scroll cycle complete after %.1fs",
                         time.time() - self._cycle_start_time
                     )
+                    # Push blank immediately so the hardware never shows
+                    # post-wrap content while the coordinator recomposes.
+                    try:
+                        from PIL import Image as _Image
+                        blank = _Image.new('RGB', (self.display_width, self.display_height))
+                        self.display_manager.image = blank
+                        self.display_manager.update_display()
+                    except (ImportError, OSError, RuntimeError, ValueError, TypeError, MemoryError) as exc:
+                        logger.error(
+                            "Failed to push blank frame at cycle end "
+                            "(display=%dx%d): %s",
+                            self.display_width, self.display_height, exc
+                        )
+                return True  # Cycle done; coordinator starts new cycle next frame
 
             # Get visible portion
             visible_frame = self.scroll_helper.get_visible_portion()

src/wifi_manager.py

@@ -144,6 +144,8 @@ class WiFiManager:
 
         # Timestamp set when AP mode is enabled; used for the idle-timeout check
         self._ap_enabled_at: Optional[float] = None
+        # Which redirect backend was used (iptables/nftables/None); set per-instance
+        self._redirect_backend: Optional[str] = None
 
         logger.info(f"WiFi Manager initialized - nmcli: {self.has_nmcli}, iwlist: {self.has_iwlist}, "
                    f"hostapd: {self.has_hostapd}, dnsmasq: {self.has_dnsmasq}, "
@@ -691,9 +693,8 @@ class WiFiManager:
 
     def _validate_ap_config(self) -> Tuple[str, int]:
         """Return a sanitized (ssid, channel) pair from config, falling back to defaults."""
-        import re as _re
         ssid = str(self.config.get("ap_ssid", DEFAULT_AP_SSID))
-        if not ssid or len(ssid) > 32 or not _re.match(r'^[\x20-\x7E]+$', ssid):
+        if not ssid or len(ssid) > 32 or not re.match(r'^[\x20-\x7E]+$', ssid):
             logger.warning(f"AP SSID '{ssid}' is invalid, falling back to default")
             ssid = DEFAULT_AP_SSID
         try:
@@ -705,10 +706,6 @@ class WiFiManager:
             channel = DEFAULT_AP_CHANNEL
         return ssid, channel
 
-    # Tracks which redirect backend was used so teardown uses the same one.
-    # Value is "iptables", "nftables", or None (not set up).
-    _redirect_backend: Optional[str] = None
-
     def _setup_iptables_redirect(self) -> bool:
         """
         Add port 80 → 5000 redirect rules for the captive portal.
@@ -936,14 +933,14 @@ class WiFiManager:
             if r.returncode == 0:
                 logger.debug("Internet connectivity confirmed via ping 8.8.8.8")
                 return True
-        except Exception:
+        except (subprocess.SubprocessError, OSError):
             pass
         try:
             import urllib.request as _ureq
             _ureq.urlopen("http://connectivity-check.ubuntu.com/", timeout=timeout)
             logger.debug("Internet connectivity confirmed via HTTP check")
             return True
-        except Exception:
+        except OSError:
             pass
         logger.debug("Internet connectivity check failed (both ping and HTTP)")
         return False
@@ -2074,6 +2071,7 @@ class WiFiManager:
             if up_result.returncode != 0:
                 error_msg = up_result.stderr.strip() or up_result.stdout.strip()
                 logger.error(f"Failed to bring up AP connection: {error_msg}")
+                self._remove_nm_dnsmasq_captive_conf()
                 subprocess.run(["nmcli", "connection", "delete", "LEDMatrix-Setup-AP"],
                                capture_output=True, timeout=10)
                 self._show_led_message("AP mode failed", duration=5)
@@ -2085,6 +2083,7 @@ class WiFiManager:
             # need to add the iptables port-redirect rules for the captive portal.
             if not self._setup_iptables_redirect():
                 logger.error("Captive-portal redirect setup failed; rolling back AP profile")
+                self._remove_nm_dnsmasq_captive_conf()
                 subprocess.run(["nmcli", "connection", "down", "LEDMatrix-Setup-AP"],
                                capture_output=True, timeout=10)
                 subprocess.run(["nmcli", "connection", "delete", "LEDMatrix-Setup-AP"],
@@ -2102,6 +2101,7 @@ class WiFiManager:
             else:
                 logger.error("AP mode started but not verified by status check — rolling back")
                 self._teardown_iptables_redirect()
+                self._remove_nm_dnsmasq_captive_conf()
                 subprocess.run(["nmcli", "connection", "down", "LEDMatrix-Setup-AP"],
                                capture_output=True, timeout=10)
                 subprocess.run(["nmcli", "connection", "delete", "LEDMatrix-Setup-AP"],
@@ -2111,6 +2111,7 @@ class WiFiManager:
 
         except Exception as e:
             logger.error(f"Error starting AP mode with nmcli: {e}")
+            self._remove_nm_dnsmasq_captive_conf()
             self._show_led_message("Setup mode error", duration=5)
             return False, str(e)
     
@@ -2498,7 +2499,10 @@ address=/detectportal.firefox.com/192.168.4.1
             # Idle-timeout check: disable AP if no client has connected within the window.
             # Only applies when AP is active and we haven't just decided to enable/disable it.
             if ap_active and self._ap_enabled_at is not None:
-                idle_timeout_min = self.config.get("ap_idle_timeout_minutes", 15)
+                try:
+                    idle_timeout_min = max(1, min(1440, int(self.config.get("ap_idle_timeout_minutes", 15))))
+                except (TypeError, ValueError):
+                    idle_timeout_min = 15
                 elapsed = time.time() - self._ap_enabled_at
                 if elapsed > idle_timeout_min * 60 and not self._has_ap_clients():
                     logger.info(

web_interface/app.py

@@ -226,13 +226,24 @@ def serve_plugin_asset(plugin_id, filename):
                 'message': 'Internal server error'
             }), 500
 
+# Prime psutil CPU measurement once at startup so interval=None returns a real value
+try:
+    import psutil as _psutil_prime
+    _psutil_prime.cpu_percent(interval=None)
+except ImportError:
+    pass
+
 # Cached AP mode check — avoids creating a WiFiManager per request
 _ap_mode_cache = {'value': False, 'timestamp': 0}
-_AP_MODE_CACHE_TTL = 5  # seconds
+_AP_MODE_CACHE_TTL = 30  # seconds — AP mode is user-initiated; 30s is fine
+
+# Cached ledmatrix service status for SSE stats stream
+_ledmatrix_service_cache = {'active': False, 'timestamp': 0}
+_LEDMATRIX_SERVICE_CACHE_TTL = 15  # seconds
 
 def is_ap_mode_active():
     """
-    Check if access point mode is currently active (cached, 5s TTL).
+    Check if access point mode is currently active (cached, 30s TTL).
     Uses a direct systemctl check instead of instantiating WiFiManager.
     """
     now = time.time()
@@ -444,7 +455,8 @@ def system_status_generator():
             # Try to import psutil for system stats
             try:
                 import psutil
-                cpu_percent = round(psutil.cpu_percent(interval=1), 1)
+                # interval=None is non-blocking; primed at module startup above
+                cpu_percent = round(psutil.cpu_percent(interval=None), 1)
                 memory = psutil.virtual_memory()
                 memory_used_percent = round(memory.percent, 1)
 
@@ -461,14 +473,17 @@ def system_status_generator():
                 memory_used_percent = 0
                 cpu_temp = 0
 
-            # Check if display service is running
+            # Check if display service is running (cached to avoid per-client subprocess forks)
-            service_active = False
+            now = time.time()
+            if (now - _ledmatrix_service_cache['timestamp']) >= _LEDMATRIX_SERVICE_CACHE_TTL:
                 try:
                     result = subprocess.run(['systemctl', 'is-active', 'ledmatrix'],
                                             capture_output=True, text=True, timeout=2)
-                service_active = result.stdout.strip() == 'active'
+                    _ledmatrix_service_cache['active'] = result.stdout.strip() == 'active'
                 except (subprocess.SubprocessError, OSError):
                     pass
+                _ledmatrix_service_cache['timestamp'] = now
+            service_active = _ledmatrix_service_cache['active']
             
             status = {
                 'timestamp': time.time(),
@@ -546,7 +561,7 @@ def display_preview_generator():
         except Exception as e:
             yield {'error': str(e)}
         
-        time.sleep(0.5)  # Check 2 times per second (reduced frequency for better performance)
+        time.sleep(1.0)  # Check once per second — halves PIL encode overhead vs 0.5s
 
 # Logs generator for SSE
 def logs_generator():

web_interface/blueprints/api_v3.py

@@ -2,6 +2,7 @@ from flask import Blueprint, request, jsonify, Response, send_from_directory
 import json
 import os
 import re
+import shutil
 import socket
 import sys
 import subprocess
@@ -16,6 +17,11 @@ from typing import Optional, Tuple, Dict, Any, Type
 
 logger = logging.getLogger(__name__)
 
+SUDO_BIN = shutil.which("sudo") or "/usr/bin/sudo"
+SYSTEMCTL_BIN = shutil.which("systemctl") or "/usr/bin/systemctl"
+REBOOT_BIN = shutil.which("reboot") or "/usr/sbin/reboot"
+POWEROFF_BIN = shutil.which("poweroff") or "/usr/sbin/poweroff"
+
 # Import new infrastructure
 from src.web_interface.api_helpers import success_response, error_response, validate_request_json
 from src.web_interface.errors import ErrorCode
@@ -218,7 +224,7 @@ def _ensure_display_service_running():
     if status.get('active'):
         status['started'] = False
         return status
-    result = _run_systemctl_command(['sudo', 'systemctl', 'start', 'ledmatrix'])
+    result = _run_systemctl_command([SUDO_BIN, SYSTEMCTL_BIN, 'start', 'ledmatrix.service'])
     service_status = _get_display_service_status()
     result['started'] = result.get('returncode') == 0
     result['active'] = service_status.get('active')
@@ -227,7 +233,7 @@ def _ensure_display_service_running():
 
 def _stop_display_service():
     """Stop the ledmatrix display service."""
-    result = _run_systemctl_command(['sudo', 'systemctl', 'stop', 'ledmatrix'])
+    result = _run_systemctl_command([SUDO_BIN, SYSTEMCTL_BIN, 'stop', 'ledmatrix.service'])
     status = _get_display_service_status()
     result['active'] = status.get('active')
     result['status'] = status
@@ -1716,33 +1722,34 @@ def execute_system_action():
             if mode:
                 # For on-demand modes, we would need to integrate with the display controller
                 # For now, just start the display service
-                result = subprocess.run(['sudo', 'systemctl', 'start', 'ledmatrix'],
+                result = subprocess.run([SUDO_BIN, SYSTEMCTL_BIN, 'start', 'ledmatrix.service'],
-                                     capture_output=True, text=True)
+                                     capture_output=True, text=True, timeout=15)
                 return jsonify({
                     'status': 'success' if result.returncode == 0 else 'error',
-                    'message': f'Started display in {mode} mode',
+                    'message': f'Started display in {mode} mode' if result.returncode == 0
+                               else f'Failed to start display in {mode} mode: {result.stderr.strip() or "check sudo systemctl status ledmatrix.service"}',
                     'returncode': result.returncode,
                     'stdout': result.stdout,
                     'stderr': result.stderr
                 })
             else:
-                result = subprocess.run(['sudo', 'systemctl', 'start', 'ledmatrix'],
+                result = subprocess.run([SUDO_BIN, SYSTEMCTL_BIN, 'start', 'ledmatrix.service'],
-                                     capture_output=True, text=True)
+                                     capture_output=True, text=True, timeout=15)
         elif action == 'stop_display':
-            result = subprocess.run(['sudo', 'systemctl', 'stop', 'ledmatrix'],
+            result = subprocess.run([SUDO_BIN, SYSTEMCTL_BIN, 'stop', 'ledmatrix.service'],
-                                 capture_output=True, text=True)
+                                 capture_output=True, text=True, timeout=15)
         elif action == 'enable_autostart':
-            result = subprocess.run(['sudo', 'systemctl', 'enable', 'ledmatrix'],
+            result = subprocess.run([SUDO_BIN, SYSTEMCTL_BIN, 'enable', 'ledmatrix.service'],
-                                 capture_output=True, text=True)
+                                 capture_output=True, text=True, timeout=15)
         elif action == 'disable_autostart':
-            result = subprocess.run(['sudo', 'systemctl', 'disable', 'ledmatrix'],
+            result = subprocess.run([SUDO_BIN, SYSTEMCTL_BIN, 'disable', 'ledmatrix.service'],
-                                 capture_output=True, text=True)
+                                 capture_output=True, text=True, timeout=15)
         elif action == 'reboot_system':
-            result = subprocess.run(['sudo', 'reboot'],
+            result = subprocess.run([SUDO_BIN, REBOOT_BIN],
-                                 capture_output=True, text=True)
+                                 capture_output=True, text=True, timeout=10)
         elif action == 'shutdown_system':
-            result = subprocess.run(['sudo', 'poweroff'],
+            result = subprocess.run([SUDO_BIN, POWEROFF_BIN],
-                                 capture_output=True, text=True)
+                                 capture_output=True, text=True, timeout=10)
         elif action == 'git_pull':
             # Use PROJECT_ROOT instead of hardcoded path
             project_dir = str(PROJECT_ROOT)
@@ -1823,12 +1830,11 @@ def execute_system_action():
                 'stderr': result.stderr
             })
         elif action == 'restart_display_service':
-            result = subprocess.run(['sudo', 'systemctl', 'restart', 'ledmatrix'],
+            result = subprocess.run([SUDO_BIN, SYSTEMCTL_BIN, 'restart', 'ledmatrix.service'],
-                                 capture_output=True, text=True)
+                                 capture_output=True, text=True, timeout=15)
         elif action == 'restart_web_service':
-            # Try to restart the web service (assuming it's ledmatrix-web.service)
+            result = subprocess.run([SUDO_BIN, SYSTEMCTL_BIN, 'restart', 'ledmatrix-web.service'],
-            result = subprocess.run(['sudo', 'systemctl', 'restart', 'ledmatrix-web'],
+                                 capture_output=True, text=True, timeout=15)
-                                 capture_output=True, text=True)
         else:
             return jsonify({'status': 'error', 'message': f'Unknown action: {action}'}), 400
 
@@ -1840,6 +1846,13 @@ def execute_system_action():
             'stderr': result.stderr
         })
 
+    except subprocess.TimeoutExpired:
+        if action == 'start_display' and mode:
+            msg = f'Failed to start display in {mode} mode: timed out'
+        else:
+            msg = f'Action {action} timed out'
+        logger.warning("[System] execute_system_action timed out: action=%s", action)
+        return jsonify({'status': 'error', 'message': msg, 'returncode': -1, 'stdout': '', 'stderr': 'timeout'}), 500
     except Exception as e:
         logger.exception("[System] execute_system_action failed")
         return jsonify({'status': 'error', 'message': 'Failed to execute system action'}), 500
@@ -4264,8 +4277,13 @@ def _set_missing_booleans_to_false(config, schema_props, form_keys, prefix='', c
         elif prop_type == 'object' and 'properties' in prop_schema:
             # Recurse into nested objects
             if config_node is not None:
-                # Inside an array item — ensure nested dict exists in item
+                # Inside an array item — only recurse if the sub-object already exists.
-                if prop_name not in node or not isinstance(node[prop_name], dict):
+                # Never create optional sub-objects that weren't submitted; doing so
+                # produces e.g. logo:{} on feed items with no logo, which then fails
+                # schema validation when the object has required fields (id, path).
+                if prop_name not in node:
+                    continue
+                if not isinstance(node[prop_name], dict):
                     node[prop_name] = {}
                 _set_missing_booleans_to_false(
                     config, prop_schema['properties'], form_keys, full_path,
@@ -4405,10 +4423,22 @@ def _filter_config_by_schema(config, schema, prefix=''):
         prop_schema = schema_props[key]
 
         # Handle nested objects recursively
+        item_prefix = f"{prefix}.{key}" if prefix else key
         if isinstance(value, dict) and prop_schema.get('type') == 'object' and 'properties' in prop_schema:
-            filtered[key] = _filter_config_by_schema(value, prop_schema, f"{prefix}.{key}" if prefix else key)
+            filtered[key] = _filter_config_by_schema(value, prop_schema, item_prefix)
+        elif isinstance(value, list) and prop_schema.get('type') == 'array':
+            items_schema = prop_schema.get('items', {})
+            if isinstance(items_schema, dict) and items_schema.get('type') == 'object' and 'properties' in items_schema:
+                # Filter each item in the array so extra fields are stripped before
+                # schema validation (important when items has additionalProperties: false)
+                filtered[key] = [
+                    _filter_config_by_schema(item, items_schema, item_prefix) if isinstance(item, dict) else item
+                    for item in value
+                ]
             else:
-            # Keep the value as-is for non-object types
+                filtered[key] = value
+        else:
+            # Keep the value as-is for non-object/non-array types
             filtered[key] = value
 
     return filtered
@@ -7136,7 +7166,7 @@ def connect_wifi():
             # Propagate structured error type so the captive portal UI can show
             # "Wrong password — try again" instead of a generic failure message.
             error_type = "wrong_password" if (message or "").startswith("wrong_password:") else "connection_failed"
-            clean_message = (message or "").removeprefix("wrong_password: ") or "Failed to connect to network"
+            clean_message = (message or "").removeprefix("wrong_password:").lstrip() or "Failed to connect to network"
             return jsonify({
                 'status': 'error',
                 'message': clean_message,
@@ -7564,6 +7594,126 @@ _STARLARK_APPS_DIR = PROJECT_ROOT / 'starlark-apps'
 _STARLARK_MANIFEST_FILE = _STARLARK_APPS_DIR / 'manifest.json'
 
 
+def _find_pixlet_binary(explicit_path: Optional[str] = None) -> Optional[str]:
+    """Find pixlet binary: explicit path → bundled binary → system PATH."""
+    import platform
+    if explicit_path and os.path.isfile(explicit_path) and os.access(explicit_path, os.X_OK):
+        return explicit_path
+    bin_dir = PROJECT_ROOT / "bin" / "pixlet"
+    system = platform.system().lower()
+    machine = platform.machine().lower()
+    if system == "linux":
+        if "aarch64" in machine or "arm64" in machine:
+            name = "pixlet-linux-arm64"
+        elif "x86_64" in machine or "amd64" in machine:
+            name = "pixlet-linux-amd64"
+        else:
+            name = None
+    elif system == "darwin":
+        name = "pixlet-darwin-arm64" if "arm64" in machine else "pixlet-darwin-amd64"
+    else:
+        name = None
+    if name:
+        bundled = bin_dir / name
+        if bundled.is_file():
+            if os.access(str(bundled), os.X_OK):
+                return str(bundled)
+            try:
+                bundled.chmod(0o755)
+            except OSError:
+                logger.warning("Could not make pixlet bundled binary executable (%s); falling back to PATH", bundled)
+            else:
+                if os.access(str(bundled), os.X_OK):
+                    return str(bundled)
+                logger.warning("Pixlet bundled binary still not executable after chmod (%s); falling back to PATH", bundled)
+    return shutil.which("pixlet")
+
+
+def _standalone_render_starlark_app(app_id: str) -> Tuple[bool, int, Optional[str]]:
+    """Render a Starlark app via pixlet directly (no plugin required).
+
+    Reads the .star file and config from starlark-apps/{app_id}/, runs pixlet,
+    and saves the output to cached_render.webp in the same directory.
+    This is the web-service fallback when starlark-apps plugin is not loaded.
+
+    Returns (success, http_status_code, error_message).
+    """
+    manifest = _read_starlark_manifest()
+    if not isinstance(manifest, dict):
+        return False, 400, "Invalid manifest shape: expected object with 'apps' mapping"
+    apps = manifest.get('apps', {})
+    if not isinstance(apps, dict):
+        return False, 400, "Invalid manifest shape: expected object with 'apps' mapping"
+    app_data = apps.get(app_id)
+    if not app_data:
+        return False, 404, f"App not found: {app_id}"
+
+    app_dir = _STARLARK_APPS_DIR / app_id
+    star_file = app_dir / app_data.get('star_file', f'{app_id}.star')
+    if not star_file.exists():
+        return False, 404, f"Star file not found: {star_file}"
+
+    full_config = api_v3.config_manager.load_config() if api_v3.config_manager else {}
+    plugin_config = full_config.get('starlark-apps', {})
+
+    pixlet_path = _find_pixlet_binary(plugin_config.get('pixlet_path'))
+    if not pixlet_path:
+        return False, 503, "Pixlet binary not found — install pixlet first"
+
+    magnify = plugin_config.get('magnify')
+    if magnify is None:
+        hw = full_config.get('display', {}).get('hardware', {})
+        cols = hw.get('cols', 64)
+        chain = hw.get('chain_length', 1)
+        rows = hw.get('rows', 32)
+        magnify = max(1, min(8, int(min((cols * chain) / 64, rows / 32))))
+    else:
+        try:
+            magnify = max(1, min(8, int(magnify)))
+        except (ValueError, TypeError):
+            magnify = 1
+
+    config_file = app_dir / 'config.json'
+    app_config: Dict[str, Any] = {}
+    if config_file.exists():
+        try:
+            with open(config_file) as f:
+                app_config = json.load(f)
+        except json.JSONDecodeError as e:
+            return False, 400, f"Invalid config.json for {app_id} ({config_file}): {e}"
+        except OSError as e:
+            return False, 400, f"Cannot read config.json for {app_id} ({config_file}): {e}"
+        if not isinstance(app_config, dict):
+            return False, 400, (
+                f"config.json for {app_id} must be a JSON object, "
+                f"got {type(app_config).__name__}"
+            )
+
+    INTERNAL_KEYS = {'render_interval', 'display_duration'}
+    pixlet_config = {k: v for k, v in app_config.items() if k not in INTERNAL_KEYS}
+
+    output_path = str(app_dir / 'cached_render.webp')
+    cmd = [pixlet_path, 'render', str(star_file)]
+    for key, value in pixlet_config.items():
+        if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_]*$', key):
+            continue
+        value_str = 'true' if value is True else 'false' if value is False else str(value)
+        if re.search(r'[`$|<>&;\x00]|\$\(', value_str):
+            continue
+        cmd.append(f'{key}={value_str}')
+    cmd.extend(['-o', output_path, '-m', str(magnify)])
+
+    try:
+        result = subprocess.run(cmd, capture_output=True, text=True, timeout=30, cwd=str(app_dir))
+        if result.returncode == 0 and os.path.isfile(output_path):
+            return True, 200, None
+        return False, 502, f"Pixlet failed (exit {result.returncode}): {result.stderr.strip()}"
+    except subprocess.TimeoutExpired:
+        return False, 504, "Render timed out after 30s"
+    except Exception as e:
+        return False, 500, f"Render error: {e}"
+
+
 def _read_starlark_manifest() -> Dict[str, Any]:
     """Read the starlark-apps manifest.json directly from disk."""
     try:
@@ -7683,24 +7833,11 @@ def get_starlark_status():
                 'display_info': magnify_info
             })
 
-        # Plugin not loaded - check Pixlet availability directly
+        # Plugin not loaded - check Pixlet availability via shared resolver
-        import shutil
+        # (respects user-configured pixlet_path, bundled binary, and system PATH)
-        import platform
+        full_config = api_v3.config_manager.load_config() if api_v3.config_manager else {}
-
+        pixlet_path = _find_pixlet_binary(full_config.get('starlark-apps', {}).get('pixlet_path'))
-        system = platform.system().lower()
+        pixlet_available = pixlet_path is not None
-        machine = platform.machine().lower()
-        bin_dir = PROJECT_ROOT / 'bin' / 'pixlet'
-
-        pixlet_binary = None
-        if system == "linux":
-            if "aarch64" in machine or "arm64" in machine:
-                pixlet_binary = bin_dir / "pixlet-linux-arm64"
-            elif "x86_64" in machine or "amd64" in machine:
-                pixlet_binary = bin_dir / "pixlet-linux-amd64"
-        elif system == "darwin":
-            pixlet_binary = bin_dir / ("pixlet-darwin-arm64" if "arm64" in machine else "pixlet-darwin-amd64")
-
-        pixlet_available = (pixlet_binary and pixlet_binary.exists()) or shutil.which('pixlet') is not None
 
         # Read app counts from manifest
         manifest = _read_starlark_manifest()
@@ -8153,20 +8290,27 @@ def toggle_starlark_app(app_id):
 def render_starlark_app(app_id):
     """Force render a Starlark app."""
     try:
-        starlark_plugin = _get_starlark_plugin()
+        is_valid, err = _validate_starlark_app_path(app_id)
-        if not starlark_plugin:
+        if not is_valid:
-            return jsonify({'status': 'error', 'message': 'Rendering requires the main LEDMatrix service (plugin not loaded in web service)'}), 503
+            return jsonify({'status': 'error', 'message': err}), 400
 
+        starlark_plugin = _get_starlark_plugin()
+        if starlark_plugin:
             app = starlark_plugin.apps.get(app_id)
             if not app:
                 return jsonify({'status': 'error', 'message': f'App not found: {app_id}'}), 404
-
             success = starlark_plugin._render_app(app, force=True)
             if success:
-            return jsonify({'status': 'success', 'message': 'App rendered', 'frame_count': len(app.frames) if app.frames else 0})
+                return jsonify({'status': 'success', 'message': 'App rendered',
-        else:
+                                'frame_count': len(app.frames) if app.frames else 0})
             return jsonify({'status': 'error', 'message': 'Failed to render app'}), 500
 
+        # Web-service context: plugin not loaded, call pixlet directly
+        success, status_code, error = _standalone_render_starlark_app(app_id)
+        if success:
+            return jsonify({'status': 'success', 'message': 'App rendered successfully', 'frame_count': 0}), status_code
+        return jsonify({'status': 'error', 'message': error or 'Render failed', 'frame_count': 0}), status_code
+
     except Exception as e:
         logger.exception("[Starlark] render_starlark_app failed")
         return jsonify({'status': 'error', 'message': 'Failed to render Starlark app'}), 500

web_interface/static/v3/plugins_manager.js

@@ -1225,7 +1225,9 @@ function initializePlugins() {
     window.pluginManager._reswap = false;
     // Await the installed-plugins fetch so window.installedPlugins is populated before
     // searchPluginStore renders Installed/Reinstall badges against it.
-    loadInstalledPlugins().then(() => {
+    loadInstalledPlugins().catch(err => {
+        console.error('[PluginStore] loadInstalledPlugins failed:', err);
+    }).finally(() => {
         searchPluginStore(!isReswapWarm);
     });