mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-10-13 10:36:35 +00:00
* feat(web): weekly automatic updates with health check and rollback A General-tab toggle (off by default) checks for and installs LEDMatrix and plugin updates once a week, overnight in the configured timezone. - Pre-update checks skip (and report) instead of forcing: local edits or commits, merge/live rebase, no upstream, low disk, missing health check, or a version that was already rolled back. An abandoned rebase (HEAD back on a branch) is cleared, since it would otherwise block every pull. - The pull reuses the Update Code path (now perform_core_update(), which reports dependency install failures as data). - ledmatrix-update-verify.service, started via a .path unit from a request file, restarts the services from its own cgroup, requires them to come up and stay up, and otherwise resets to the previous commit and reinstalls the previous requirements. It runs a copy of the checker taken before the pull. - No SSH needed: switching the toggle on restarts the display service, which (as root) installs the two units from the repo templates for the web user. first_time_install.sh installs them too and takes --enable-auto-update / LEDMATRIX_AUTO_UPDATE (passed through by one-shot-install.sh). - Plugins update after the code passes its check; failures, blocks and rollbacks raise an Overview banner and show under the toggle. Tested end to end on a Pi: web-UI setup, a good update, a broken web service and a broken display (both rolled back), a blocked local edit, and an abandoned rebase found on the device. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(auto-update): address static-analysis findings - Replace the subprocess.CompletedProcess the verifier fabricated for a command that could not start with a plain namedtuple; nothing is executed there, but the scanner flags any CompletedProcess built from variables. - Mark the subprocess imports with the repo's standard B404 annotation (all calls are list-form argv, no shell). - Mark the rollback-failed message as not SQL (B608 matched its wording). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(auto-update): CI failures on Linux - Keep the setup result when chown fails. CI runs as a non-root user, where chown to the web user raises; that discarded the result file, so the General tab would never learn whether setup worked. Regression test added. - Register the two new /api/v3/system/auto-update routes in the URL map snapshot. - Use utility classes app.css defines (space-y-1, hover:text-red-600). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(auto-update): address review feedback - Health check: a failed restart command no longer lets the check run against the still-running old process; it counts as a failure (and after a rollback, as a failed rollback). An unreadable restart count is never treated as stable, since a crash loop looks healthy between attempts. - Installer writes the auto_update setting to a temp file and swaps it in, keeping mode and owner, so a running config watcher never reads a truncated config.json. - Verify unit quotes its command-line paths (install folders with spaces); setup refuses folder names systemd would reinterpret (%, quotes, backslashes, control characters) and says so on the General tab. - The auto-update status route no longer returns exception text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(auto-update): keep error detail in the status route's 500 test_web_error_detail requires every 5xx handler to log the traceback and return describe_exception(e), which redacts credentials, so failures are diagnosable from the web UI. Dropping it for CodeQL broke that policy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(auto-update): dismiss route rejects non-object JSON with 400 A JSON array or scalar body made `.get('alert_id')` raise, returning 500. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(auto-update): let the app-wide handler answer status-route errors CodeQL (py/stack-trace-exposure, #709) flagged the route's own except, which returned describe_exception(e). web_interface/app.py's error handler already logs the traceback and returns the same redacted detail for any unhandled exception, so the local copy is removed: same response, no new exception-to-response flow, and test_web_error_detail's policy still holds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
334 lines
14 KiB
Python
334 lines
14 KiB
Python
"""
|
|
Startup Validator
|
|
|
|
Validates system configuration, plugins, and dependencies on startup.
|
|
Fails fast with clear error messages to prevent runtime issues.
|
|
"""
|
|
|
|
import os
|
|
from typing import Any, List, Optional, Tuple
|
|
from pathlib import Path
|
|
from src.exceptions import ConfigError, PluginError, CacheError
|
|
from src.logging_config import get_logger
|
|
|
|
|
|
class StartupValidator:
|
|
"""Validates system state on startup."""
|
|
|
|
def __init__(self, config_manager: Any, plugin_manager: Optional[Any] = None,
|
|
cache_manager: Optional[Any] = None) -> None:
|
|
"""
|
|
Initialize the startup validator.
|
|
|
|
Args:
|
|
config_manager: ConfigManager instance
|
|
plugin_manager: Optional PluginManager instance
|
|
cache_manager: The CacheManager the application will actually use.
|
|
Pass it. Without one this validator builds its own just to read
|
|
a directory path, which reports on a cache the app does not
|
|
use and leaves behind a cleanup thread that nothing stops --
|
|
validation runs twice per startup, so that was two of them.
|
|
"""
|
|
self.config_manager = config_manager
|
|
self.plugin_manager = plugin_manager
|
|
self.cache_manager = cache_manager
|
|
self.logger = get_logger(__name__)
|
|
self.errors: List[str] = []
|
|
self.warnings: List[str] = []
|
|
|
|
def validate_all(self) -> Tuple[bool, List[str], List[str]]:
|
|
"""
|
|
Run all validation checks.
|
|
|
|
Returns:
|
|
Tuple of (is_valid, errors, warnings)
|
|
"""
|
|
self.logger.info("Starting startup validation...")
|
|
|
|
# Fresh lists each run — without this, calling validate_all() twice
|
|
# duplicated every message.
|
|
self.errors = []
|
|
self.warnings = []
|
|
|
|
# Validate configuration
|
|
self._validate_config()
|
|
|
|
# Validate cache directory
|
|
self._validate_cache_directory()
|
|
|
|
# Validate display configuration
|
|
self._validate_display_config()
|
|
|
|
# Validate plugins if plugin manager is available
|
|
if self.plugin_manager:
|
|
self._validate_plugins()
|
|
|
|
# Warn when the running systemd unit no longer matches the repo's
|
|
self._validate_systemd_units()
|
|
|
|
is_valid = len(self.errors) == 0
|
|
|
|
if is_valid:
|
|
self.logger.info("Startup validation passed")
|
|
if self.warnings:
|
|
self.logger.warning(f"Startup validation completed with {len(self.warnings)} warning(s)")
|
|
else:
|
|
self.logger.error(f"Startup validation failed with {len(self.errors)} error(s)")
|
|
|
|
return (is_valid, self.errors.copy(), self.warnings.copy())
|
|
|
|
#: Units this project installs, and where each is installed to.
|
|
_UNITS = (
|
|
("systemd/ledmatrix.service", "/etc/systemd/system/ledmatrix.service"),
|
|
("systemd/ledmatrix-web.service", "/etc/systemd/system/ledmatrix-web.service"),
|
|
("systemd/ledmatrix-update-verify.service", "/etc/systemd/system/ledmatrix-update-verify.service"),
|
|
("systemd/ledmatrix-update-verify.path", "/etc/systemd/system/ledmatrix-update-verify.path"),
|
|
)
|
|
|
|
def _validate_systemd_units(self) -> None:
|
|
"""Warn when an installed unit has drifted from the repo's template.
|
|
|
|
Nothing re-applies these after the first install. `git pull` -- which is
|
|
what the web UI's update button runs -- brings a new template into the
|
|
checkout, but nothing copies it to /etc/systemd/system and nothing runs
|
|
`systemctl daemon-reload`, so the unit that actually runs is whatever
|
|
first_time_install.sh wrote on day one.
|
|
|
|
That makes every hardening added to a unit inert on existing installs.
|
|
Measured on one rig: the installed unit was thirteen days older than the
|
|
repo's and differed in content, so a MemoryMax the repo had specified
|
|
was not being enforced at all -- `systemctl show` reported
|
|
MemoryMax=infinity.
|
|
|
|
A warning rather than an error, and certainly not a silent rewrite:
|
|
editing files under /etc and restarting services is the installer's job,
|
|
not something a display process should do to a machine while it boots.
|
|
The remedy is to re-run scripts/install/install_service.sh.
|
|
"""
|
|
try:
|
|
project_root = Path(__file__).resolve().parent.parent
|
|
for template_rel, installed_path in self._UNITS:
|
|
template = project_root / template_rel
|
|
installed = Path(installed_path)
|
|
if not template.is_file() or not installed.is_file():
|
|
continue
|
|
|
|
try:
|
|
actual = installed.read_text(encoding="utf-8")
|
|
except PermissionError:
|
|
continue
|
|
|
|
# The template carries placeholders the installer substitutes,
|
|
# so compare the substituted form rather than the raw file.
|
|
expected = template.read_text(encoding="utf-8")
|
|
expected = expected.replace("__PROJECT_ROOT_DIR__", str(project_root))
|
|
# User= is an install-time decision, not something the template
|
|
# dictates: the installers write whoever ran them, which on a
|
|
# non-root install is never "root". Substituting a fixed "root"
|
|
# here reported drift on every such install, permanently -- and
|
|
# re-running the installer, which is what the warning tells you
|
|
# to do, could not clear it. Taking the installed unit's own
|
|
# value keeps the comparison on the directives the template
|
|
# actually controls.
|
|
expected = expected.replace("__USER__", self._installed_user(actual))
|
|
|
|
if self._unit_body(expected) != self._unit_body(actual):
|
|
self.warnings.append(
|
|
f"{installed.name} differs from {template_rel}; the "
|
|
"installed unit is not refreshed by an update, so "
|
|
"settings added to the template are not in effect. "
|
|
"Re-run scripts/install/install_service.sh to apply them."
|
|
)
|
|
except OSError as e:
|
|
self.logger.debug("Could not compare systemd units: %s", e)
|
|
|
|
@staticmethod
|
|
def _installed_user(unit_text: str) -> str:
|
|
"""The installed unit's ``User=``, or "root" when it does not set one.
|
|
|
|
systemd itself defaults to root for a system unit with no User=, so that
|
|
is the right fallback rather than an empty string.
|
|
"""
|
|
for line in unit_text.splitlines():
|
|
stripped = line.strip()
|
|
if stripped.startswith("User="):
|
|
return stripped.split("=", 1)[1].strip()
|
|
return "root"
|
|
|
|
@staticmethod
|
|
def _unit_body(text: str) -> str:
|
|
"""A unit's meaningful lines, in order: no comments, no blanks.
|
|
|
|
Order is preserved deliberately. This used to sort, which made the
|
|
comparison insensitive to two changes that matter in a systemd unit:
|
|
repeated directives such as ExecStartPre= and ExecStartPost= run in
|
|
the order they appear, and a directive that moves between [Unit],
|
|
[Service] and [Install] means something different -- or nothing --
|
|
where it lands. A drift check that normalises those away reports no
|
|
drift for a unit that has genuinely changed.
|
|
"""
|
|
lines = []
|
|
for line in text.splitlines():
|
|
line = line.strip()
|
|
if line and not line.startswith("#"):
|
|
lines.append(line)
|
|
return "\n".join(lines)
|
|
|
|
def _validate_config(self) -> None:
|
|
"""Validate configuration files."""
|
|
try:
|
|
config = self.config_manager.load_config()
|
|
|
|
# Check for required top-level keys
|
|
required_keys = ['display', 'timezone']
|
|
for key in required_keys:
|
|
if key not in config:
|
|
self.errors.append(f"Missing required configuration key: {key}")
|
|
|
|
# Validate display configuration
|
|
display_config = config.get('display', {})
|
|
if not display_config:
|
|
self.errors.append("Display configuration is missing or empty")
|
|
|
|
except ConfigError as e:
|
|
self.errors.append(f"Configuration error: {e}")
|
|
except Exception as e:
|
|
self.errors.append(f"Unexpected error validating configuration: {e}")
|
|
|
|
def _validate_cache_directory(self) -> None:
|
|
"""Validate cache directory permissions."""
|
|
try:
|
|
cache_manager = self.cache_manager
|
|
if cache_manager is None:
|
|
# No caller supplied one (older embedders, direct use in a
|
|
# script). Build one, but do not leave its cleanup thread
|
|
# running behind us -- this instance is discarded on the next
|
|
# line but the thread is a closure over it, so it would never
|
|
# be collected.
|
|
from src.cache_manager import CacheManager
|
|
cache_manager = CacheManager()
|
|
try:
|
|
cache_dir = cache_manager.get_cache_dir()
|
|
finally:
|
|
cache_manager.stop_cleanup_thread()
|
|
else:
|
|
cache_dir = cache_manager.get_cache_dir()
|
|
|
|
if not cache_dir:
|
|
self.warnings.append("Cache directory not available - caching will be disabled")
|
|
return
|
|
|
|
# Check if directory exists and is writable
|
|
if not os.path.exists(cache_dir):
|
|
self.errors.append(f"Cache directory does not exist: {cache_dir}")
|
|
return
|
|
|
|
if not os.access(cache_dir, os.W_OK):
|
|
self.errors.append(f"Cache directory is not writable: {cache_dir}")
|
|
return
|
|
|
|
# Test write access
|
|
test_file = os.path.join(cache_dir, '.startup_test')
|
|
try:
|
|
with open(test_file, 'w') as f:
|
|
f.write('test')
|
|
os.remove(test_file)
|
|
except (IOError, OSError) as e:
|
|
self.errors.append(f"Cannot write to cache directory {cache_dir}: {e}")
|
|
|
|
except Exception as e:
|
|
self.warnings.append(f"Could not validate cache directory: {e}")
|
|
|
|
def _validate_display_config(self) -> None:
|
|
"""Validate display configuration."""
|
|
try:
|
|
config = self.config_manager.get_config()
|
|
display_config = config.get('display', {})
|
|
|
|
if not display_config:
|
|
self.errors.append("Display configuration is missing")
|
|
return
|
|
|
|
hardware_config = display_config.get('hardware', {})
|
|
if not hardware_config:
|
|
self.errors.append("Display hardware configuration is missing")
|
|
return
|
|
|
|
# Check required hardware settings
|
|
required_hardware = ['rows', 'cols']
|
|
for key in required_hardware:
|
|
if key not in hardware_config:
|
|
self.warnings.append(f"Display hardware setting '{key}' not specified, using default")
|
|
|
|
except Exception as e:
|
|
self.warnings.append(f"Could not validate display configuration: {e}")
|
|
|
|
def _validate_plugins(self) -> None:
|
|
"""Validate plugin configurations and dependencies."""
|
|
if not self.plugin_manager:
|
|
return
|
|
|
|
try:
|
|
# Get enabled plugins from config
|
|
config = self.config_manager.get_config()
|
|
discovered_plugins = self.plugin_manager.discover_plugins()
|
|
|
|
# Check for enabled plugins that don't exist
|
|
for plugin_id, plugin_config in config.items():
|
|
# Skip non-plugin config sections
|
|
if plugin_id in ['display', 'schedule', 'timezone', 'plugin_system']:
|
|
continue
|
|
|
|
if not isinstance(plugin_config, dict):
|
|
continue
|
|
|
|
if plugin_config.get('enabled', False):
|
|
if plugin_id not in discovered_plugins:
|
|
self.warnings.append(f"Plugin '{plugin_id}' is enabled but not found in plugins directory")
|
|
|
|
# Validate plugin configurations
|
|
for plugin_id in discovered_plugins:
|
|
plugin_config = config.get(plugin_id, {})
|
|
if plugin_config.get('enabled', False):
|
|
# Check if plugin can be loaded (without actually loading it)
|
|
plugin_dir = self.plugin_manager.get_plugin_directory(plugin_id)
|
|
if plugin_dir:
|
|
manifest_path = Path(plugin_dir) / "manifest.json"
|
|
if not manifest_path.exists():
|
|
self.errors.append(f"Plugin '{plugin_id}' manifest.json not found")
|
|
|
|
except Exception as e:
|
|
self.warnings.append(f"Could not validate plugins: {e}")
|
|
|
|
def raise_on_errors(self) -> None:
|
|
"""
|
|
Raise exceptions if validation errors exist.
|
|
|
|
Raises:
|
|
ConfigError: If configuration validation fails
|
|
CacheError: If cache validation fails
|
|
PluginError: If plugin validation fails
|
|
"""
|
|
if not self.errors:
|
|
return
|
|
|
|
# Group errors by type
|
|
config_errors = [e for e in self.errors if 'configuration' in e.lower() or 'config' in e.lower()]
|
|
cache_errors = [e for e in self.errors if 'cache' in e.lower()]
|
|
plugin_errors = [e for e in self.errors if 'plugin' in e.lower()]
|
|
other_errors = [e for e in self.errors if e not in config_errors + cache_errors + plugin_errors]
|
|
|
|
# Raise appropriate exceptions
|
|
if config_errors:
|
|
raise ConfigError("Configuration validation failed", context={'errors': config_errors})
|
|
|
|
if cache_errors:
|
|
raise CacheError("Cache validation failed", context={'errors': cache_errors})
|
|
|
|
if plugin_errors:
|
|
raise PluginError("Plugin validation failed", context={'errors': plugin_errors})
|
|
|
|
if other_errors:
|
|
raise ConfigError("Startup validation failed", context={'errors': other_errors})
|
|
|