rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-18 04:49:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge main into the api_v3 split, carrying the three fixes it would have dropped

Browse Source 
main modified web_interface/blueprints/api_v3.py while this branch deletes it,
so git reports a modify/delete conflict. Resolving that by accepting the
deletion - the obvious move - would silently discard every api_v3.py change
merged since this branch forked. Three were:

  #560  the sudo diagnosis on /system/action  -> api_v3/system.py
  #557  the stale-reconciliation-findings filter -> api_v3/plugins.py
  #550  the deletion of dead _validate_time_range -> api_v3/__init__.py

All three were verified absent from the split beforehand: _sudo_hint_for,
_SUDO_NEEDS_PASSWORD and _drop_stale_reconciliation_findings appeared in zero
files, while _validate_time_range was still sitting at __init__.py:343 after
#550 removed it from main. Each is now ported to the module that owns the
affected route, and the dead function is gone.

Two further changes this surfaced:

- api_v3/__init__.py imported fcntl at module scope. fcntl is POSIX-only, the
  monolithic blueprint never imported it, and on this branch it made the whole
  api_v3 package unimportable off POSIX: the pristine branch cannot run its
  test suite on Windows at all - 14 collection errors interrupt the run. The
  import now happens inside _starlark_manifest_lock(), the only thing that uses
  it. Deliberately NOT degraded to a no-op lock off POSIX; that function's
  docstring describes a real lost-update race, and silently not locking would
  be worse than failing loudly on a platform that cannot drive the display.

- test/test_reconciliation_status_endpoint.py is new. #557's endpoint wiring had
  no test - only the pure still_unresolved() helper in src/ was covered, and
  that lives outside web_interface so it survived the split untouched. Losing
  the filter during this merge would therefore have been completely silent.
  Removing the filter call now fails three of its seven cases.

Suite on this merge: 115 failed / 4317 passed. main today: 105 / 4252. All 11
extra failures are in test_starlark_pixlet_routes.py and need POSIX file locks,
so they are Windows-only; on that same file the pristine branch shows 19 failed
plus 39 errors, so this is a large net improvement rather than a regression.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-11 09:07:35 -04:00
co-authored by Claude Opus 5
parent 7c3e4b8a38 0ab95586fb
commit 05cb9dc31f
33 changed files with 1856 additions and 234 deletions
Download Patch File Download Diff File Expand all files Collapse all files
web_interface/app.py
+10 -21
View File
@@ -32,6 +32,8 @@ _JOURNALCTL = shutil.which('journalctl')
_SYSTEMCTL = shutil.which('systemctl')
_VCGENCMD = shutil.which('vcgencmd')
from web_interface.system_metrics import collect_system_metrics
# Create Flask app
app = Flask(__name__)
app.secret_key = os.urandom(24)
@@ -606,26 +608,12 @@ def system_status_generator():
"""Generate system status updates"""
while True:
try:
# Try to import psutil for system stats
try:
import psutil
# interval=None is non-blocking; primed at module startup above
cpu_percent = round(psutil.cpu_percent(interval=None), 1)
memory = psutil.virtual_memory()
memory_used_percent = round(memory.percent, 1)
# Try to get CPU temperature (Raspberry Pi specific)
cpu_temp = 0
try:
with open('/sys/class/thermal/thermal_zone0/temp', 'r') as f:
cpu_temp = round(float(f.read()) / 1000.0, 1)
except (OSError, ValueError):
pass
except ImportError:
cpu_percent = 0
memory_used_percent = 0
cpu_temp = 0
metrics = collect_system_metrics()
cpu_percent = metrics['cpu_percent']
memory_used_percent = metrics['memory_used_percent']
memory_available_mb = metrics['memory_available_mb']
disk_used_percent = metrics['disk_used_percent']
cpu_temp = metrics['cpu_temp']
# Check if display service is running (cached to avoid per-client subprocess forks)
now = time.time()
@@ -646,8 +634,9 @@ def system_status_generator():
'service_active': service_active,
'cpu_percent': cpu_percent,
'memory_used_percent': memory_used_percent,
'memory_available_mb': memory_available_mb,
'cpu_temp': cpu_temp,
'disk_used_percent': 0,
'disk_used_percent': disk_used_percent,
'power': _get_power_status()
}
yield status
web_interface/blueprints/api_v3/__init__.py
+9 -14
View File
@@ -17,7 +17,6 @@ everything they need has to exist first.
"""
from flask import Blueprint, request, jsonify, Response
import contextlib
import fcntl
import json
import os
import re
@@ -340,19 +339,6 @@ def _validate_time_format(time_str):
return True, None
except (ValueError, TypeError):
return False, f"Invalid time format: {time_str}. Expected HH:MM format."
def _validate_time_range(start_time_str, end_time_str, allow_overnight=True):
"""Validate time range. Returns (is_valid, error_message)"""
try:
start_time = datetime.strptime(start_time_str, '%H:%M').time()
end_time = datetime.strptime(end_time_str, '%H:%M').time()
# Allow overnight schedules (start > end) or same-day schedules
if not allow_overnight and start_time >= end_time:
return False, f"Start time ({start_time_str}) must be before end time ({end_time_str}) for same-day schedules"
return True, None
except (ValueError, TypeError) as e:
return False, f"Invalid time format: {str(e)}"
def _git_current_branch(project_dir):
"""Current branch name, or '' when detached or git fails."""
try:
@@ -1380,6 +1366,15 @@ def _starlark_manifest_lock():
Callers should do their read, mutation and _write_starlark_manifest() call
entirely inside the `with` block, mirroring the plugin's lock scope.
"""
# Imported here, not at module scope: fcntl is POSIX-only, and a top-level
# import made the whole api_v3 package unimportable on Windows -- which the
# monolithic blueprint never was, so the split would have broken local dev
# and the test suite there. Deliberately NOT degraded to a no-op lock off
# POSIX: the docstring above describes a real lost-update race, and silently
# not locking would be worse than failing loudly on a platform that cannot
# run the display anyway.
import fcntl
_STARLARK_APPS_DIR.mkdir(parents=True, exist_ok=True)
lock_fd = os.open(str(_STARLARK_MANIFEST_LOCK_FILE), os.O_CREAT | os.O_RDWR, 0o644)
try:
web_interface/blueprints/api_v3/plugins.py
+31
View File
@@ -707,6 +707,35 @@ def reconcile_plugin_state():
context=error.context,
status_code=500
)
def _drop_stale_reconciliation_findings(unresolved):
"""Re-check a stored reconciliation verdict against current state.
The verdict is a snapshot written once per run, and a run that could not
apply a fix also refuses to retry -- so a resolved condition was reported
indefinitely. Best-effort: any failure here returns the list untouched,
because showing a stale warning beats failing the endpoint.
The disk set is enumerated directly rather than through
_installed_plugin_ids(): this mirrors StateReconciliation._get_disk_state(),
whose verdict we are re-checking, and avoids triggering a discovery scan on
an endpoint the overview page polls.
"""
try:
from src.plugin_system.state_reconciliation import still_unresolved
config_keys = set(api_v3.config_manager.load_config() or {})
installed = set()
plugins_dir = getattr(api_v3.plugin_manager, 'plugins_dir', None)
if plugins_dir:
for entry in Path(plugins_dir).iterdir():
if entry.is_dir() and (entry / 'manifest.json').exists():
installed.add(entry.name)
return still_unresolved(unresolved, config_keys, installed)
except Exception:
logger.debug("[Reconciliation] Could not re-check stored findings", exc_info=True)
return unresolved
@api_v3.route('/plugins/reconciliation-status', methods=['GET'])
def get_reconciliation_status():
"""Return the result of the last startup reconciliation from /tmp status file."""
@@ -721,6 +750,8 @@ def get_reconciliation_status():
try:
with open(_recon_path) as _f:
data = json.load(_f)
if data.get('unresolved'):
data['unresolved'] = _drop_stale_reconciliation_findings(data['unresolved'])
return jsonify({'status': 'success', 'data': data})
except json.JSONDecodeError:
logger.exception("[Reconciliation] Failed to parse status file: %s", _recon_path)
web_interface/blueprints/api_v3/system.py
+45 -2
View File
@@ -171,6 +171,33 @@ def check_for_update():
logger.warning("check-update failed: %s", e)
return jsonify(_update_check_failed(
"Could not check for updates; see logs for details."))
#: sudo's own wording when it needs a password it cannot ask for. The web
#: interface runs unprivileged, so its systemctl/reboot/journalctl calls only
#: work once scripts/install/configure_web_sudo.sh has granted NOPASSWD --
#: which first_time_install.sh does not do. That makes this the common case on
#: a fresh device, and "Action failed; see logs for details" named none of it,
#: while the log viewer was broken for the very same reason.
_SUDO_NEEDS_PASSWORD = (
'a password is required',
'no tty present',
'a terminal is required',
)
_SUDO_HINT = (
'Passwordless sudo is not configured for the web interface user, so this '
'action cannot run. Run scripts/install/configure_web_sudo.sh as that user, '
'then retry.'
)
def _sudo_hint_for(text):
"""An actionable hint when `text` is sudo refusing to prompt, else None."""
lowered = (text or '').lower()
if any(marker in lowered for marker in _SUDO_NEEDS_PASSWORD):
return _SUDO_HINT
return None
@api_v3.route('/system/action', methods=['POST'])
def execute_system_action():
"""Execute system actions (start/stop/reboot/etc)"""
@@ -206,7 +233,14 @@ def execute_system_action():
logger.error("start_display (%s) stderr: %s", mode, result.stderr.strip())
resp = {
'status': 'success' if result.returncode == 0 else 'error',
'message': 'Display started' if result.returncode == 0 else 'Failed to start display',
# This branch returns before the shared nonzero-result
# response below, so it needs the hint of its own or an
# on-demand start reports "Failed to start display" and
# says nothing about the sudo that actually refused it.
'message': (
'Display started' if result.returncode == 0
else _sudo_hint_for(result.stderr) or 'Failed to start display'
),
}
if result.returncode != 0:
resp['returncode'] = result.returncode
@@ -538,6 +572,9 @@ def execute_system_action():
if result.returncode != 0:
resp['returncode'] = result.returncode
resp['stderr'] = result.stderr.strip()
hint = _sudo_hint_for(result.stderr)
if hint:
resp['message'] = hint
return jsonify(resp)
except subprocess.TimeoutExpired as e:
@@ -545,7 +582,13 @@ def execute_system_action():
return jsonify({'status': 'error', 'message': 'Command timed out', 'returncode': -1, 'stderr': 'timeout'})
except Exception as e:
logger.error("execute_system_action failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'Action failed; see logs for details'}), 500
detail = describe_exception(e)
resp = {
'status': 'error',
'message': _sudo_hint_for(detail) or 'Action failed; see logs for details',
'details': detail,
}
return jsonify(resp), 500
@api_v3.route('/system/git-info', methods=['GET'])
def get_git_info():
"""Return branch, dirty state, recent commits and remote URL for the Tools tab."""
web_interface/system_metrics.py
+62
View File
@@ -0,0 +1,62 @@
"""System metrics for the status stream.
Deliberately free of Flask and app imports: importing web_interface.app
constructs the Flask application and a CacheManager, and the latter claims the
cache directory with a cleanup thread. Reading a CPU percentage should not do
that, and neither should testing it.
Every value is best-effort. A number that cannot be taken comes back as None so
the UI can render '--', because a confident wrong number is worse than a blank:
``disk_used_percent`` used to be hardcoded to 0, which read as "plenty of room"
on a card that was filling up.
"""
from typing import Any, Dict, Optional
_THERMAL_ZONE = '/sys/class/thermal/thermal_zone0/temp'
def _cpu_temp_c() -> Optional[float]:
"""CPU temperature in degrees C, or None off a Raspberry Pi."""
try:
with open(_THERMAL_ZONE, 'r') as f:
return round(float(f.read()) / 1000.0, 1)
except (OSError, ValueError):
return None
def collect_system_metrics() -> Dict[str, Any]:
"""Collect the numbers that predict trouble on a small board.
``memory_available_mb`` is MemAvailable rather than a used percentage: it
accounts for reclaimable page cache, so it is what separates a board at 70%
"used" that is fine from one at 70% that is about to fail fork(). A 1GB Pi
can sit at either and only this number tells them apart.
"""
try:
import psutil
except ImportError:
return {
'cpu_percent': 0,
'memory_used_percent': 0,
'memory_available_mb': None,
'disk_used_percent': None,
'cpu_temp': _cpu_temp_c() or 0,
}
# interval=None is non-blocking; app startup primes psutil's internal state.
cpu_percent = round(psutil.cpu_percent(interval=None), 1)
memory = psutil.virtual_memory()
try:
disk_used_percent: Optional[float] = round(psutil.disk_usage('/').percent, 1)
except OSError:
disk_used_percent = None
return {
'cpu_percent': cpu_percent,
'memory_used_percent': round(memory.percent, 1),
'memory_available_mb': round(memory.available / (1024 * 1024), 1),
'disk_used_percent': disk_used_percent,
'cpu_temp': _cpu_temp_c() or 0,
}
web_interface/templates/v3/partials/overview.html
+27 -4
View File
@@ -30,10 +30,33 @@
if (!d.unresolved || d.unresolved.length === 0) return;
var key = d.unresolved.map(function (i) { return i.plugin_id; }).sort().join(',');
if (sessionStorage.getItem(DISMISS_KEY) === key) return;
var ids = d.unresolved.map(function (i) { return i.plugin_id; }).join(', ');
document.getElementById('reconciliation-banner-text').textContent =
'Stale plugin config entries found: ' + ids +
'. Remove them from config.json or reinstall via the Plugin Store.';
// These are opposite problems and need opposite advice. Lumping
// them together told users to delete config.json entries for
// plugins that were installed and correctly configured.
function idsOfType(t) {
return d.unresolved.filter(function (i) { return i.type === t; })
.map(function (i) { return i.plugin_id; });
}
var notConfigured = idsOfType('plugin_missing_in_config');
var notInstalled = idsOfType('plugin_missing_on_disk');
var other = d.unresolved.filter(function (i) {
return i.type !== 'plugin_missing_in_config'
&& i.type !== 'plugin_missing_on_disk';
}).map(function (i) { return i.plugin_id; });
var parts = [];
if (notConfigured.length) {
parts.push('Installed but missing from config: ' + notConfigured.join(', ')
+ '. Configure them in the Plugin Store - do not remove anything from config.json.');
}
if (notInstalled.length) {
parts.push('In config but not installed: ' + notInstalled.join(', ')
+ '. Reinstall via the Plugin Store, or remove these entries from config.json.');
}
if (other.length) {
parts.push('Needs attention: ' + other.join(', ') + '.');
}
document.getElementById('reconciliation-banner-text').textContent = parts.join(' ');
var banner = document.getElementById('reconciliation-banner');
banner.dataset.dismissKey = key;
banner.style.setProperty('display', 'flex', 'important');