mirror of
https://github.com/ChuckBuilds/LEDMatrix.git
synced 2026-05-14 09:33:32 +00:00
fix: pixlet install false-failure, force render in web service, web UI perf (#328)
* fix: pixlet install false-failure, force render in web service, web UI perf Fixes three user-reported issues: 1. Pixlet install reported failure even on success — `set -e` in download_pixlet.sh caused the script to exit non-zero when `((success_count++))` evaluated the post-increment old value (0), which bash treats as a failed command. Fixed with shell arithmetic assignment instead. 2. Force render returned 503 "plugin not loaded in web service" — the web service runs its own PluginManager with display_manager=None, so the starlark-apps plugin can never be loaded there. Added a standalone render path (_standalone_render_starlark_app) that calls pixlet directly from the web service, writing to cached_render.webp. The main-service plugin path is preserved and tried first. 3. Web UI sluggishness — the SSE /stream/stats generator was blocking a Flask thread for 1s per tick via psutil.cpu_percent(interval=1). Switched to non-blocking interval=None (primed at startup). Also cached the per-client ledmatrix systemctl check (15s TTL), raised the AP mode check TTL from 5s to 30s, and halved the display preview poll rate from 0.5s to 1s. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(starlark): surface config errors, deduplicate pixlet lookup, uniform response shape - _standalone_render_starlark_app: split silent except into separate json.JSONDecodeError and OSError handlers that return (False, message) with the file path and parse error, so callers know when config.json is unreadable rather than silently rendering with empty config - get_starlark_status: replace 14-line inline platform/shutil pixlet lookup with _find_pixlet_binary(), which also checks the user- configured starlark-apps.pixlet_path — the old code never consulted that setting so the status endpoint could wrongly report pixlet unavailable even when a custom path was configured - render_starlark_app standalone branch: add frame_count: 0 to success and error responses so both branches return the same payload shape; 0 is accurate since standalone render writes cached_render.webp but does not extract frames into memory Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(starlark): safe chmod fallback in pixlet lookup, semantic HTTP codes in standalone render _find_pixlet_binary: bundled.chmod(0o755) could raise OSError (e.g. file owned by root) and abort the entire resolution chain, skipping the PATH fallback. Now checks os.access first; if chmod is needed, wraps it in try/except OSError, logs a warning, and falls through to shutil.which so PATH is always tried. _standalone_render_starlark_app: expanded return type from (bool, str) to (bool, int, str) so each failure mode carries a semantic HTTP status code: app/star file missing → 404 invalid/unreadable config.json → 400 pixlet binary missing → 503 pixlet non-zero exit → 502 subprocess timeout → 504 unexpected exception → 500 success → 200 Call site updated to unpack (success, status_code, error) and forward status_code directly in both success and error responses. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(starlark): validate config.json is a dict before iterating items json.load succeeds on valid JSON that isn't an object (e.g. an array or bare string), leaving app_config as a non-dict and causing an AttributeError on the subsequent .items() call. Added isinstance check immediately after json.load; non-dict values return (False, 400, ...) with the actual type name in the message. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(starlark): harden pixlet binary check and manifest shape validation _find_pixlet_binary: switched bundled.exists() to bundled.is_file() so a directory named pixlet-linux-arm64 (traversable, hence X_OK) is no longer returned as a binary; re-check os.access after chmod succeeds so we only return the path when executability is confirmed, with a separate warning if it still isn't executable after chmod. _standalone_render_starlark_app: added isinstance guards on the manifest and apps values returned by _read_starlark_manifest(); a manifest.json containing valid non-object JSON (e.g. an array) would previously raise AttributeError on .get(); invalid shape now returns (False, 400, ...) instead. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Chuck <chuck@example.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Chuck
@@ -118,7 +118,7 @@ total_count=${#ARCHITECTURES[@]}
|
||||
|
||||
for arch in "${!ARCHITECTURES[@]}"; do
|
||||
if download_binary "$arch" "${ARCHITECTURES[$arch]}"; then
|
||||
((success_count++))
|
||||
success_count=$((success_count + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
|
||||
@@ -226,13 +226,24 @@ def serve_plugin_asset(plugin_id, filename):
|
||||
'message': 'Internal server error'
|
||||
}), 500
|
||||
|
||||
# Prime psutil CPU measurement once at startup so interval=None returns a real value
|
||||
try:
|
||||
import psutil as _psutil_prime
|
||||
_psutil_prime.cpu_percent(interval=None)
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# Cached AP mode check — avoids creating a WiFiManager per request
|
||||
_ap_mode_cache = {'value': False, 'timestamp': 0}
|
||||
_AP_MODE_CACHE_TTL = 5 # seconds
|
||||
_AP_MODE_CACHE_TTL = 30 # seconds — AP mode is user-initiated; 30s is fine
|
||||
|
||||
# Cached ledmatrix service status for SSE stats stream
|
||||
_ledmatrix_service_cache = {'active': False, 'timestamp': 0}
|
||||
_LEDMATRIX_SERVICE_CACHE_TTL = 15 # seconds
|
||||
|
||||
def is_ap_mode_active():
|
||||
"""
|
||||
Check if access point mode is currently active (cached, 5s TTL).
|
||||
Check if access point mode is currently active (cached, 30s TTL).
|
||||
Uses a direct systemctl check instead of instantiating WiFiManager.
|
||||
"""
|
||||
now = time.time()
|
||||
@@ -444,10 +455,11 @@ def system_status_generator():
|
||||
# Try to import psutil for system stats
|
||||
try:
|
||||
import psutil
|
||||
cpu_percent = round(psutil.cpu_percent(interval=1), 1)
|
||||
# interval=None is non-blocking; primed at module startup above
|
||||
cpu_percent = round(psutil.cpu_percent(interval=None), 1)
|
||||
memory = psutil.virtual_memory()
|
||||
memory_used_percent = round(memory.percent, 1)
|
||||
|
||||
|
||||
# Try to get CPU temperature (Raspberry Pi specific)
|
||||
cpu_temp = 0
|
||||
try:
|
||||
@@ -455,20 +467,23 @@ def system_status_generator():
|
||||
cpu_temp = round(float(f.read()) / 1000.0, 1)
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
|
||||
|
||||
except ImportError:
|
||||
cpu_percent = 0
|
||||
memory_used_percent = 0
|
||||
cpu_temp = 0
|
||||
|
||||
# Check if display service is running
|
||||
service_active = False
|
||||
try:
|
||||
result = subprocess.run(['systemctl', 'is-active', 'ledmatrix'],
|
||||
capture_output=True, text=True, timeout=2)
|
||||
service_active = result.stdout.strip() == 'active'
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
pass
|
||||
|
||||
# Check if display service is running (cached to avoid per-client subprocess forks)
|
||||
now = time.time()
|
||||
if (now - _ledmatrix_service_cache['timestamp']) >= _LEDMATRIX_SERVICE_CACHE_TTL:
|
||||
try:
|
||||
result = subprocess.run(['systemctl', 'is-active', 'ledmatrix'],
|
||||
capture_output=True, text=True, timeout=2)
|
||||
_ledmatrix_service_cache['active'] = result.stdout.strip() == 'active'
|
||||
except (subprocess.SubprocessError, OSError):
|
||||
pass
|
||||
_ledmatrix_service_cache['timestamp'] = now
|
||||
service_active = _ledmatrix_service_cache['active']
|
||||
|
||||
status = {
|
||||
'timestamp': time.time(),
|
||||
@@ -546,7 +561,7 @@ def display_preview_generator():
|
||||
except Exception as e:
|
||||
yield {'error': str(e)}
|
||||
|
||||
time.sleep(0.5) # Check 2 times per second (reduced frequency for better performance)
|
||||
time.sleep(1.0) # Check once per second — halves PIL encode overhead vs 0.5s
|
||||
|
||||
# Logs generator for SSE
|
||||
def logs_generator():
|
||||
|
||||
@@ -7577,6 +7577,126 @@ _STARLARK_APPS_DIR = PROJECT_ROOT / 'starlark-apps'
|
||||
_STARLARK_MANIFEST_FILE = _STARLARK_APPS_DIR / 'manifest.json'
|
||||
|
||||
|
||||
def _find_pixlet_binary(explicit_path: Optional[str] = None) -> Optional[str]:
|
||||
"""Find pixlet binary: explicit path → bundled binary → system PATH."""
|
||||
import platform
|
||||
if explicit_path and os.path.isfile(explicit_path) and os.access(explicit_path, os.X_OK):
|
||||
return explicit_path
|
||||
bin_dir = PROJECT_ROOT / "bin" / "pixlet"
|
||||
system = platform.system().lower()
|
||||
machine = platform.machine().lower()
|
||||
if system == "linux":
|
||||
if "aarch64" in machine or "arm64" in machine:
|
||||
name = "pixlet-linux-arm64"
|
||||
elif "x86_64" in machine or "amd64" in machine:
|
||||
name = "pixlet-linux-amd64"
|
||||
else:
|
||||
name = None
|
||||
elif system == "darwin":
|
||||
name = "pixlet-darwin-arm64" if "arm64" in machine else "pixlet-darwin-amd64"
|
||||
else:
|
||||
name = None
|
||||
if name:
|
||||
bundled = bin_dir / name
|
||||
if bundled.is_file():
|
||||
if os.access(str(bundled), os.X_OK):
|
||||
return str(bundled)
|
||||
try:
|
||||
bundled.chmod(0o755)
|
||||
except OSError:
|
||||
logger.warning("Could not make pixlet bundled binary executable (%s); falling back to PATH", bundled)
|
||||
else:
|
||||
if os.access(str(bundled), os.X_OK):
|
||||
return str(bundled)
|
||||
logger.warning("Pixlet bundled binary still not executable after chmod (%s); falling back to PATH", bundled)
|
||||
return shutil.which("pixlet")
|
||||
|
||||
|
||||
def _standalone_render_starlark_app(app_id: str) -> Tuple[bool, int, Optional[str]]:
|
||||
"""Render a Starlark app via pixlet directly (no plugin required).
|
||||
|
||||
Reads the .star file and config from starlark-apps/{app_id}/, runs pixlet,
|
||||
and saves the output to cached_render.webp in the same directory.
|
||||
This is the web-service fallback when starlark-apps plugin is not loaded.
|
||||
|
||||
Returns (success, http_status_code, error_message).
|
||||
"""
|
||||
manifest = _read_starlark_manifest()
|
||||
if not isinstance(manifest, dict):
|
||||
return False, 400, "Invalid manifest shape: expected object with 'apps' mapping"
|
||||
apps = manifest.get('apps', {})
|
||||
if not isinstance(apps, dict):
|
||||
return False, 400, "Invalid manifest shape: expected object with 'apps' mapping"
|
||||
app_data = apps.get(app_id)
|
||||
if not app_data:
|
||||
return False, 404, f"App not found: {app_id}"
|
||||
|
||||
app_dir = _STARLARK_APPS_DIR / app_id
|
||||
star_file = app_dir / app_data.get('star_file', f'{app_id}.star')
|
||||
if not star_file.exists():
|
||||
return False, 404, f"Star file not found: {star_file}"
|
||||
|
||||
full_config = api_v3.config_manager.load_config() if api_v3.config_manager else {}
|
||||
plugin_config = full_config.get('starlark-apps', {})
|
||||
|
||||
pixlet_path = _find_pixlet_binary(plugin_config.get('pixlet_path'))
|
||||
if not pixlet_path:
|
||||
return False, 503, "Pixlet binary not found — install pixlet first"
|
||||
|
||||
magnify = plugin_config.get('magnify')
|
||||
if magnify is None:
|
||||
hw = full_config.get('display', {}).get('hardware', {})
|
||||
cols = hw.get('cols', 64)
|
||||
chain = hw.get('chain_length', 1)
|
||||
rows = hw.get('rows', 32)
|
||||
magnify = max(1, min(8, int(min((cols * chain) / 64, rows / 32))))
|
||||
else:
|
||||
try:
|
||||
magnify = max(1, min(8, int(magnify)))
|
||||
except (ValueError, TypeError):
|
||||
magnify = 1
|
||||
|
||||
config_file = app_dir / 'config.json'
|
||||
app_config: Dict[str, Any] = {}
|
||||
if config_file.exists():
|
||||
try:
|
||||
with open(config_file) as f:
|
||||
app_config = json.load(f)
|
||||
except json.JSONDecodeError as e:
|
||||
return False, 400, f"Invalid config.json for {app_id} ({config_file}): {e}"
|
||||
except OSError as e:
|
||||
return False, 400, f"Cannot read config.json for {app_id} ({config_file}): {e}"
|
||||
if not isinstance(app_config, dict):
|
||||
return False, 400, (
|
||||
f"config.json for {app_id} must be a JSON object, "
|
||||
f"got {type(app_config).__name__}"
|
||||
)
|
||||
|
||||
INTERNAL_KEYS = {'render_interval', 'display_duration'}
|
||||
pixlet_config = {k: v for k, v in app_config.items() if k not in INTERNAL_KEYS}
|
||||
|
||||
output_path = str(app_dir / 'cached_render.webp')
|
||||
cmd = [pixlet_path, 'render', str(star_file)]
|
||||
for key, value in pixlet_config.items():
|
||||
if not re.match(r'^[a-zA-Z_][a-zA-Z0-9_]*$', key):
|
||||
continue
|
||||
value_str = 'true' if value is True else 'false' if value is False else str(value)
|
||||
if re.search(r'[`$|<>&;\x00]|\$\(', value_str):
|
||||
continue
|
||||
cmd.append(f'{key}={value_str}')
|
||||
cmd.extend(['-o', output_path, '-m', str(magnify)])
|
||||
|
||||
try:
|
||||
result = subprocess.run(cmd, capture_output=True, text=True, timeout=30, cwd=str(app_dir))
|
||||
if result.returncode == 0 and os.path.isfile(output_path):
|
||||
return True, 200, None
|
||||
return False, 502, f"Pixlet failed (exit {result.returncode}): {result.stderr.strip()}"
|
||||
except subprocess.TimeoutExpired:
|
||||
return False, 504, "Render timed out after 30s"
|
||||
except Exception as e:
|
||||
return False, 500, f"Render error: {e}"
|
||||
|
||||
|
||||
def _read_starlark_manifest() -> Dict[str, Any]:
|
||||
"""Read the starlark-apps manifest.json directly from disk."""
|
||||
try:
|
||||
@@ -7696,24 +7816,11 @@ def get_starlark_status():
|
||||
'display_info': magnify_info
|
||||
})
|
||||
|
||||
# Plugin not loaded - check Pixlet availability directly
|
||||
import shutil
|
||||
import platform
|
||||
|
||||
system = platform.system().lower()
|
||||
machine = platform.machine().lower()
|
||||
bin_dir = PROJECT_ROOT / 'bin' / 'pixlet'
|
||||
|
||||
pixlet_binary = None
|
||||
if system == "linux":
|
||||
if "aarch64" in machine or "arm64" in machine:
|
||||
pixlet_binary = bin_dir / "pixlet-linux-arm64"
|
||||
elif "x86_64" in machine or "amd64" in machine:
|
||||
pixlet_binary = bin_dir / "pixlet-linux-amd64"
|
||||
elif system == "darwin":
|
||||
pixlet_binary = bin_dir / ("pixlet-darwin-arm64" if "arm64" in machine else "pixlet-darwin-amd64")
|
||||
|
||||
pixlet_available = (pixlet_binary and pixlet_binary.exists()) or shutil.which('pixlet') is not None
|
||||
# Plugin not loaded - check Pixlet availability via shared resolver
|
||||
# (respects user-configured pixlet_path, bundled binary, and system PATH)
|
||||
full_config = api_v3.config_manager.load_config() if api_v3.config_manager else {}
|
||||
pixlet_path = _find_pixlet_binary(full_config.get('starlark-apps', {}).get('pixlet_path'))
|
||||
pixlet_available = pixlet_path is not None
|
||||
|
||||
# Read app counts from manifest
|
||||
manifest = _read_starlark_manifest()
|
||||
@@ -8166,20 +8273,27 @@ def toggle_starlark_app(app_id):
|
||||
def render_starlark_app(app_id):
|
||||
"""Force render a Starlark app."""
|
||||
try:
|
||||
is_valid, err = _validate_starlark_app_path(app_id)
|
||||
if not is_valid:
|
||||
return jsonify({'status': 'error', 'message': err}), 400
|
||||
|
||||
starlark_plugin = _get_starlark_plugin()
|
||||
if not starlark_plugin:
|
||||
return jsonify({'status': 'error', 'message': 'Rendering requires the main LEDMatrix service (plugin not loaded in web service)'}), 503
|
||||
|
||||
app = starlark_plugin.apps.get(app_id)
|
||||
if not app:
|
||||
return jsonify({'status': 'error', 'message': f'App not found: {app_id}'}), 404
|
||||
|
||||
success = starlark_plugin._render_app(app, force=True)
|
||||
if success:
|
||||
return jsonify({'status': 'success', 'message': 'App rendered', 'frame_count': len(app.frames) if app.frames else 0})
|
||||
else:
|
||||
if starlark_plugin:
|
||||
app = starlark_plugin.apps.get(app_id)
|
||||
if not app:
|
||||
return jsonify({'status': 'error', 'message': f'App not found: {app_id}'}), 404
|
||||
success = starlark_plugin._render_app(app, force=True)
|
||||
if success:
|
||||
return jsonify({'status': 'success', 'message': 'App rendered',
|
||||
'frame_count': len(app.frames) if app.frames else 0})
|
||||
return jsonify({'status': 'error', 'message': 'Failed to render app'}), 500
|
||||
|
||||
# Web-service context: plugin not loaded, call pixlet directly
|
||||
success, status_code, error = _standalone_render_starlark_app(app_id)
|
||||
if success:
|
||||
return jsonify({'status': 'success', 'message': 'App rendered successfully', 'frame_count': 0}), status_code
|
||||
return jsonify({'status': 'error', 'message': error or 'Render failed', 'frame_count': 0}), status_code
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("[Starlark] render_starlark_app failed")
|
||||
return jsonify({'status': 'error', 'message': 'Failed to render Starlark app'}), 500
|
||||
|
||||
Reference in New Issue
Block a user