From 6096a22c3de10e03ab62a0bcbdd36405186dc270 Mon Sep 17 00:00:00 2001 From: Chuck <33324927+ChuckBuilds@users.noreply.github.com> Date: Mon, 29 Jun 2026 12:19:54 -0400 Subject: [PATCH] feat(web): add Tools tab and row address type display setting (#373) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(web): add Tools tab and row address type setting Adds a Tools/Utilities tab to the web interface with one-click maintenance buttons that previously required SSH: - Git status panel (branch, dirty state, recent commits) - Pull latest (rebase) and force reset to origin/main - Reinstall base requirements (pip, with output) - Reinstall per-plugin requirements (pass/fail per plugin) - Clear __pycache__ directories - Quick-access restart for display and web services Also exposes the hzeller row_address_type option (0–4) in the Display settings tab. The backend already read this value from config; the UI, API field list, and validation were missing. Co-Authored-By: Claude Sonnet 4.6 * fix(tools-tab): address code review findings - Add _GIT = shutil.which('git') alongside _SUDO/_JOURNALCTL; return 503 in force_git_reset and get_git_info if git is unavailable - Check git branch/status returncodes in get_git_info(); return a clear 500 error instead of silently treating a failed run as a clean repo - Cap pip stdout+stderr at 50 KB via _truncate_output() helper to avoid OOM on verbose dependency resolution or build failures - Scrub embedded HTTPS credentials from remote_url via _scrub_git_remote_url() using urllib.parse before returning to UI - Fix clear_pycache to track and report failed deletions separately instead of counting them as successes (removed ignore_errors=True, wrapped in try/except OSError) Skipped: plugin_manager-vs-api_v3.plugin_manager (api_v3 is the Blueprint object; accessing .plugin_manager on it would fail — module- level variable is the correct pattern used throughout this blueprint); pages_v3 broad-except (identical to every other _load_*_partial in the file); base.html HTMX fallback (loadTabContent handles all tabs generically; named fallbacks only exist for tabs needing JS re-init); tools.html auth (pre-existing architectural decision — reboot/shutdown on the same endpoint are also unauthenticated). Co-Authored-By: Claude Sonnet 4.6 * fix(tools-tab): resolve remaining PR review comments - api_v3: use getattr(api_v3, 'plugin_manager', None) instead of the module-level plugin_manager (always None); app.py sets the blueprint attribute, not the module global, so the fallback to plugin-repos was always taken - pages_v3: replace broad except Exception in _load_tools_partial with specific TemplateNotFound / OSError handlers and add [Pages V3][Tools] context prefix to log messages and error responses for easier Pi debugging - base.html: add Tools tab branch to the HTMX-unavailable fallback block in loadTabContent so the tab loads gracefully via direct fetch if HTMX never initialises Skipped: auth on execute_system_action — pre-existing app-wide design; reboot/shutdown and all other system actions share the same exposure. An app-level auth layer is the correct fix and is out of scope here. Co-Authored-By: Claude Sonnet 4.6 * fix(tools-tab): resolve second-pass review findings - Wrap per-plugin subprocess.run in try/except TimeoutExpired/OSError so one plugin's failure appends a result entry and continues the loop rather than collapsing the whole batch into a 500 - Validate double_sided_copies divisibility against chain_length (horizontal axis) or parallel (vertical axis) after the range check; reads effective axis from the current request or stored config - Exclude double_sided_fields from the generic key-merge loop so double_sided_enabled/copies/axis are never written as root-level keys - Fix tools.html copy: "then restores the stash" removed — git_pull stashes changes but never pops them - Check r.ok and d.status in loadGitInfo before building the panel; backend error messages now surface instead of silently showing a false-clean state Co-Authored-By: Claude Sonnet 4.6 * fix(tools-tab): don't expose filesystem paths in OSError messages CodeQL flagged str(exc) flowing into the JSON response for the install_plugin_requirements action. Use exc.strerror instead, which gives the OS error description ("No such file or directory", "Permission denied") without the internal filesystem path. Co-Authored-By: Claude Sonnet 4.6 --------- Co-authored-by: Chuck Co-authored-by: Claude Sonnet 4.6 --- web_interface/blueprints/api_v3.py | 162 ++++++++- web_interface/blueprints/pages_v3.py | 15 + web_interface/templates/v3/base.html | 34 +- .../templates/v3/partials/display.html | 12 + .../templates/v3/partials/tools.html | 314 ++++++++++++++++++ 5 files changed, 533 insertions(+), 4 deletions(-) create mode 100644 web_interface/templates/v3/partials/tools.html diff --git a/web_interface/blueprints/api_v3.py b/web_interface/blueprints/api_v3.py index cc459fe9..dba3d1c8 100644 --- a/web_interface/blueprints/api_v3.py +++ b/web_interface/blueprints/api_v3.py @@ -14,6 +14,7 @@ import logging from datetime import datetime from pathlib import Path from typing import Dict, Any +from urllib.parse import urlparse, urlunparse logger = logging.getLogger(__name__) @@ -28,6 +29,32 @@ from src.error_aggregator import get_error_aggregator _SUDO = shutil.which('sudo') _JOURNALCTL = shutil.which('journalctl') +_GIT = shutil.which('git') + +# Cap subprocess output returned to the browser — pip can produce MBs on build failures. +_MAX_OUTPUT_BYTES = 51_200 # 50 KB + + +def _truncate_output(stdout: str, stderr: str) -> str: + """Combine stdout+stderr and truncate to _MAX_OUTPUT_BYTES (keeping the tail).""" + combined = (stdout + stderr).strip() + if len(combined) > _MAX_OUTPUT_BYTES: + combined = '[...output truncated...]\n' + combined[-_MAX_OUTPUT_BYTES:] + return combined + + +def _scrub_git_remote_url(url: str) -> str: + """Strip embedded username/password from an HTTPS remote URL before returning it to the UI.""" + try: + p = urlparse(url) + if p.scheme in ('http', 'https') and (p.username or p.password): + netloc = p.hostname or '' + if p.port: + netloc += f':{p.port}' + return urlunparse(p._replace(netloc=netloc)) + except Exception: + pass + return url # Will be initialized when blueprint is registered config_manager = None @@ -705,7 +732,8 @@ def save_main_config(): display_fields = ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping', 'gpio_slowdown', 'rp1_rio', 'scan_mode', 'disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'use_short_date_format', - 'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type'] + 'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type', + 'row_address_type'] if any(k in data for k in display_fields): if 'display' not in current_config: @@ -736,14 +764,23 @@ def save_main_config(): except (ValueError, TypeError): return jsonify({'status': 'error', 'message': f"Invalid multiplexing value '{data['multiplexing']}'. Must be an integer from 0 to 22."}), 400 + # Validate row_address_type + if 'row_address_type' in data: + try: + rat_val = int(data['row_address_type']) + if rat_val < 0 or rat_val > 4: + return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400 + except (ValueError, TypeError): + return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400 + # Handle hardware settings for field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping', 'scan_mode', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', - 'led_rgb_sequence', 'multiplexing', 'panel_type']: + 'led_rgb_sequence', 'multiplexing', 'panel_type', 'row_address_type']: if field in data: if field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'scan_mode', 'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', - 'multiplexing']: + 'multiplexing', 'row_address_type']: current_config['display']['hardware'][field] = int(data[field]) else: current_config['display']['hardware'][field] = data[field] @@ -792,6 +829,19 @@ def save_main_config(): return jsonify({'status': 'error', 'message': "Double-sided copies must be an integer"}), 400 if not (2 <= copies <= 8): return jsonify({'status': 'error', 'message': "Double-sided copies must be between 2 and 8"}), 400 + # Validate divisibility against the relevant hardware dimension. + # Use axis from this request if provided, else from stored config. + hw = current_config.get('display', {}).get('hardware', {}) + effective_axis = (data.get('double_sided_axis') + or current_config.get('display', {}).get('double_sided', {}).get('axis', 'horizontal')) + if effective_axis == 'horizontal': + chain_length = int(hw.get('chain_length', 2) or 2) + if chain_length % copies != 0: + return jsonify({'status': 'error', 'message': f"Double-sided copies ({copies}) must divide chain length ({chain_length}) evenly"}), 400 + elif effective_axis == 'vertical': + parallel = int(hw.get('parallel', 1) or 1) + if parallel % copies != 0: + return jsonify({'status': 'error', 'message': f"Double-sided copies ({copies}) must divide parallel ({parallel}) evenly"}), 400 ds_config['copies'] = copies if 'double_sided_axis' in data: @@ -1045,6 +1095,8 @@ def save_main_config(): continue if key in vegas_fields: continue + if key in double_sided_fields: + continue # For any remaining keys (including plugin keys), use deep merge to preserve existing settings if key in current_config and isinstance(current_config[key], dict) and isinstance(data[key], dict): # Deep merge to preserve existing settings @@ -1615,6 +1667,81 @@ def execute_system_action(): # Try to restart the web service (assuming it's ledmatrix-web.service) result = subprocess.run(['sudo', 'systemctl', 'restart', 'ledmatrix-web.service'], capture_output=True, text=True, timeout=10) + elif action == 'install_base_requirements': + req_file = PROJECT_ROOT / 'requirements.txt' + if not req_file.exists(): + return jsonify({'status': 'error', 'message': 'No requirements.txt found at project root'}) + result = subprocess.run( + [sys.executable, '-m', 'pip', 'install', '--break-system-packages', '-r', str(req_file)], + capture_output=True, text=True, timeout=120, cwd=str(PROJECT_ROOT) + ) + return jsonify({ + 'status': 'success' if result.returncode == 0 else 'error', + 'message': 'Base requirements installed successfully' if result.returncode == 0 else 'pip install failed', + 'output': _truncate_output(result.stdout, result.stderr) + }) + elif action == 'install_plugin_requirements': + active_pm = getattr(api_v3, 'plugin_manager', None) + plugins_dir = Path(active_pm.plugins_dir) if active_pm else PROJECT_ROOT / 'plugin-repos' + results = [] + if plugins_dir.exists(): + for p in sorted(plugins_dir.iterdir()): + req = p / 'requirements.txt' + if p.is_dir() and req.exists(): + try: + r = subprocess.run( + [sys.executable, '-m', 'pip', 'install', '--break-system-packages', '-r', str(req)], + capture_output=True, text=True, timeout=60 + ) + results.append({ + 'plugin': p.name, + 'ok': r.returncode == 0, + 'output': _truncate_output(r.stdout, r.stderr) + }) + except subprocess.TimeoutExpired: + results.append({'plugin': p.name, 'ok': False, 'output': 'pip install timed out'}) + except OSError as exc: + results.append({'plugin': p.name, 'ok': False, 'output': exc.strerror or 'OS error'}) + ok_count = sum(1 for r in results if r['ok']) + all_ok = all(r['ok'] for r in results) if results else True + return jsonify({ + 'status': 'success' if all_ok else 'error', + 'message': f'Processed {len(results)} plugin(s) — {ok_count} succeeded' if results else 'No plugin requirements.txt files found', + 'details': results + }) + elif action == 'force_git_reset': + if not _GIT: + return jsonify({'status': 'error', 'message': 'git not found on this system'}), 503 + project_dir = str(PROJECT_ROOT) + fetch = subprocess.run( + [_GIT, 'fetch', 'origin'], + capture_output=True, text=True, timeout=30, cwd=project_dir + ) + if fetch.returncode != 0: + return jsonify({'status': 'error', 'message': 'git fetch failed', 'output': fetch.stderr.strip()}) + reset = subprocess.run( + [_GIT, 'reset', '--hard', 'origin/main'], + capture_output=True, text=True, timeout=30, cwd=project_dir + ) + return jsonify({ + 'status': 'success' if reset.returncode == 0 else 'error', + 'message': 'Reset to origin/main successfully' if reset.returncode == 0 else 'git reset failed', + 'output': (reset.stdout + reset.stderr).strip() + }) + elif action == 'clear_pycache': + cleared = 0 + failed = 0 + for d in PROJECT_ROOT.rglob('__pycache__'): + if d.is_dir(): + try: + shutil.rmtree(d) + cleared += 1 + except OSError: + failed += 1 + msg = f'Cleared {cleared} __pycache__ directories' + if failed: + msg += f' ({failed} could not be removed)' + return jsonify({'status': 'success', 'message': msg}) else: return jsonify({'status': 'error', 'message': 'Unknown action'}), 400 @@ -1637,6 +1764,35 @@ def execute_system_action(): logger.error("execute_system_action failed: %s", e, exc_info=True) return jsonify({'status': 'error', 'message': 'Action failed; see logs for details'}), 500 +@api_v3.route('/system/git-info', methods=['GET']) +def get_git_info(): + """Return branch, dirty state, recent commits and remote URL for the Tools tab.""" + if not _GIT: + return jsonify({'status': 'error', 'message': 'git not found on this system'}), 503 + d = str(PROJECT_ROOT) + try: + branch = subprocess.run([_GIT, 'branch', '--show-current'], capture_output=True, text=True, timeout=10, cwd=d) + if branch.returncode != 0: + return jsonify({'status': 'error', 'message': f'git branch failed: {branch.stderr.strip()}'}), 500 + + status = subprocess.run([_GIT, 'status', '--short', '--untracked-files=no'], capture_output=True, text=True, timeout=15, cwd=d) + if status.returncode != 0: + return jsonify({'status': 'error', 'message': f'git status failed: {status.stderr.strip()}'}), 500 + + log = subprocess.run([_GIT, 'log', '--oneline', '-5'], capture_output=True, text=True, timeout=10, cwd=d) + remote = subprocess.run([_GIT, 'remote', 'get-url', 'origin'], capture_output=True, text=True, timeout=10, cwd=d) + return jsonify({ + 'branch': branch.stdout.strip(), + 'dirty': bool(status.stdout.strip()), + 'status': status.stdout.strip(), + 'recent_commits': log.stdout.strip() if log.returncode == 0 else '', + 'remote_url': _scrub_git_remote_url(remote.stdout.strip()) if remote.returncode == 0 else '', + }) + except Exception as e: + logger.error("get_git_info failed: %s", e, exc_info=True) + return jsonify({'status': 'error', 'message': 'Failed to get git info'}), 500 + + @api_v3.route('/hardware/status', methods=['GET']) def get_hardware_status(): """Return LED matrix hardware initialization status written by display_manager at startup.""" diff --git a/web_interface/blueprints/pages_v3.py b/web_interface/blueprints/pages_v3.py index 73939f52..b011480a 100644 --- a/web_interface/blueprints/pages_v3.py +++ b/web_interface/blueprints/pages_v3.py @@ -1,4 +1,5 @@ from flask import Blueprint, render_template, flash +from jinja2 import TemplateNotFound from markupsafe import escape import json import logging @@ -90,6 +91,8 @@ def load_partial(partial_name): return _load_cache_partial() elif partial_name == 'operation-history': return _load_operation_history_partial() + elif partial_name == 'tools': + return _load_tools_partial() else: return "Partial not found", 404 @@ -448,6 +451,18 @@ def _load_operation_history_partial(): return "Error loading partial", 500 +def _load_tools_partial(): + """Load tools/utilities partial.""" + try: + return render_template('v3/partials/tools.html') + except TemplateNotFound: + logger.error("[Pages V3][Tools] Template not found: v3/partials/tools.html", exc_info=True) + return "[Pages V3][Tools] Template is missing.", 500 + except OSError as exc: + logger.error("[Pages V3][Tools] I/O error loading tools partial: %s", exc, exc_info=True) + return "[Pages V3][Tools] Failed to load due to a file system error. Check logs.", 500 + + def _load_plugin_config_partial(plugin_id): """ Load plugin configuration partial - server-side rendered form. diff --git a/web_interface/templates/v3/base.html b/web_interface/templates/v3/base.html index f8a08295..5dea81d5 100644 --- a/web_interface/templates/v3/base.html +++ b/web_interface/templates/v3/base.html @@ -1009,6 +1009,11 @@ class="nav-tab"> Operation History + @@ -1290,6 +1295,18 @@ + +
+
+
+
+
+
+
+
+
+
+ +
+
+

Git & Updates

+

Inspect the current git state and pull or reset to the latest remote code.

+
+ + +
+
Loading git info…
+
+ +
+ +
+
+

Pull latest (rebase)

+

Stashes any local changes, then runs git pull --rebase. The stash is preserved but not re-applied.

+
+ +
+ + + +
+
+

Force reset to origin/main

+

Runs git fetch origin then git reset --hard origin/main. Discards all local changes.

+
+
+ + +
+
+ +
+
+ + +
+
+

Python Dependencies

+

Re-run pip install to fix missing or broken packages.

+
+ +
+ +
+
+

Reinstall base requirements

+

Installs from requirements.txt in the project root.

+
+ +
+ + + +
+
+

Reinstall plugin requirements

+

Runs pip install for every installed plugin that has a requirements.txt.

+
+ +
+ +
+
+ + +
+
+

Maintenance

+

Housekeeping operations that don't affect config or plugins.

+
+ +
+
+
+

Clear Python cache

+

Deletes all __pycache__ directories in the project. Useful after switching branches or debugging import issues.

+
+ +
+ +
+
+ + +
+
+

Services

+

Quick access to service restarts.

+
+ +
+
+
+

Restart display service

+

Restarts ledmatrix.service.

+
+ +
+ + +
+
+

Restart web interface

+

Restarts ledmatrix-web.service. The page will go offline briefly.

+
+ +
+ +
+
+ + + +