7 Commits

Author SHA1 Message Date
Chuck
b2524e918d feat(web): add Tools tab and row address type setting
Adds a Tools/Utilities tab to the web interface with one-click
maintenance buttons that previously required SSH:
- Git status panel (branch, dirty state, recent commits)
- Pull latest (rebase) and force reset to origin/main
- Reinstall base requirements (pip, with output)
- Reinstall per-plugin requirements (pass/fail per plugin)
- Clear __pycache__ directories
- Quick-access restart for display and web services

Also exposes the hzeller row_address_type option (0–4) in the
Display settings tab. The backend already read this value from
config; the UI, API field list, and validation were missing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 09:41:33 -04:00
Chuck
eb6687ceca fix(install): correctly detect already-installed dateutil/websocket-client
Address remaining coderabbitai findings on PR #369:
- check_package_installed() did __import__(package_name) directly, but
  python-dateutil and websocket-client import as dateutil/websocket. Both
  always failed the "already installed" check and were reinstalled on
  every run. Add an IMPORT_NAME_MAP for the mismatched names.
- _run() still read the entire temp file into memory before slicing the
  tail. Stream it line-by-line into a deque(maxlen=ERROR_TAIL_LINES)
  instead so memory use stays bounded for very chatty commands.
2026-06-11 16:21:56 -04:00
Chuck
60b64144a5 fix(install): suppress remaining Codacy subprocess false-positive
Codacy's Semgrep-based check still flagged the cmd-built subprocess.run
call as "without a static string" even with the Bandit nosec applied.
Add a nosemgrep marker alongside it - cmd is always a hardcoded
apt/pip argument list, never user input.
2026-06-11 13:01:33 -04:00
Chuck
5a1a095e16 fix(install): bound subprocess output and dedupe apt update in dependency installer
Address coderabbitai review on PR #369:
- _run() now streams combined stdout/stderr to a temp file and returns
  only the last ERROR_TAIL_LINES lines, instead of buffering full
  output in memory (Codacy also flagged the previous capture_output
  call as a subprocess-without-static-string security issue; the new
  call is annotated as safe since cmd is built from hardcoded args).
- `apt update` now runs once in main() instead of once per package
  needing an apt fallback.
2026-06-11 12:57:30 -04:00
Chuck
6aec2d9b78 feat(install): harden first-time install against common Pi failure modes
- wait_for_apt_lock: apt_update/apt_install now wait (up to 3min) for
  unattended-upgrades to release the dpkg lock instead of failing
  outright with "Command failed after 3 attempts" right after first boot.
- check_disk_space: new pre-flight check (Step 1) so a full SD card fails
  fast with a clear message instead of a cryptic mid-build error.
- Step 6: wrap rpi-rgb-led-matrix git clone/submodule operations in retry
  for resilience to transient network issues.
- Step 6: capture `pip install .` build output and print the last 50
  lines on failure, so the actual cmake/compiler error is visible instead
  of just "Failed to install rpi-rgb-led-matrix Python package".
2026-06-11 12:57:30 -04:00
Chuck
6c9a3510ee feat(install): surface root cause of web dependency install failures
install_dependencies_apt.py previously reported only which packages
failed, not why - the actual apt/pip error was discarded (apt) or
could scroll out of the on_error log tail (pip), leaving "Step 7:
Install web interface dependencies (line 915)" as the only visible
detail.

Capture command output for each install attempt and print a compact
DEPENDENCY INSTALLATION FAILURES summary with the last lines of error
output per package. Also run the installer with `python3 -u` for
real-time, correctly-ordered logging, and widen the on_error tail from
50 to 100 lines so the summary isn't cut off.
2026-06-11 12:57:29 -04:00
Chuck
6ea9862c14 fix(install): don't let outer ERR trap mask first_time_install.sh failures
set +e alone doesn't suppress bash's ERR trap, so any non-zero exit from
first_time_install.sh inside the one-shot installer immediately triggered
the outer on_error handler with a generic "Main installation, line 370"
message — before the script could report the real exit code or point to
logs/. Suspend the trap for that block so the existing if/else handling
runs instead.
2026-06-11 12:57:29 -04:00
10 changed files with 442 additions and 327 deletions

1
.gitignore vendored
View File

@@ -8,7 +8,6 @@ config/config_secrets.json
config/config.json
config/config.json.backup
config/wifi_config.json
config/uninstalled_plugins.json
credentials.json
token.pickle

View File

@@ -322,19 +322,10 @@ class StateReconciliation:
and hasattr(self.store_manager, 'was_recently_uninstalled')
and self.store_manager.was_recently_uninstalled(plugin_id)
)
# Also refuse to resurrect a plugin the user has persistently
# uninstalled. Unlike the in-memory race guard above, this record
# survives restarts, so the user's removal sticks across updates.
persistently_uninstalled = (
self.store_manager is not None
and hasattr(self.store_manager, 'is_plugin_uninstalled')
and self.store_manager.is_plugin_uninstalled(plugin_id)
)
can_repair = (
self.store_manager is not None
and not previously_unrecoverable
and not recently_uninstalled
and not persistently_uninstalled
)
inconsistencies.append(Inconsistency(
plugin_id=plugin_id,

View File

@@ -7,7 +7,6 @@ from both the official registry and custom GitHub repositories.
import hashlib
import os
import re
import json
import stat
import subprocess
@@ -20,7 +19,7 @@ import time
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime
from pathlib import Path
from typing import List, Dict, Optional, Any, Tuple, Set
from typing import List, Dict, Optional, Any, Tuple
import logging
from urllib.parse import urlparse
@@ -44,24 +43,13 @@ class PluginStoreManager:
"""
REGISTRY_URL = "https://raw.githubusercontent.com/ChuckBuilds/ledmatrix-plugins/main/plugins.json"
# A valid plugin id is a single path component: starts alphanumeric, then
# alphanumerics / dot / dash / underscore. Used to keep the uninstall
# registry from ever turning a corrupt or hand-edited entry (e.g. "",
# "..", "../x") into a filesystem path that purge_uninstalled_plugins
# would delete — an empty id resolves to the plugins root itself.
_PLUGIN_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
def __init__(self, plugins_dir: str = "plugins",
uninstalled_registry_path: Optional[str] = None):
def __init__(self, plugins_dir: str = "plugins"):
"""
Initialize the plugin store manager.
Args:
plugins_dir: Directory where plugins are installed
uninstalled_registry_path: Path to the JSON file recording plugins
the user has uninstalled. Defaults to
``config/uninstalled_plugins.json`` under the project root.
"""
self.plugins_dir = Path(plugins_dir)
self.logger = logging.getLogger(__name__)
@@ -96,25 +84,6 @@ class PluginStoreManager:
self._uninstall_tombstones: Dict[str, float] = {}
self._uninstall_tombstone_ttl = 300 # 5 minutes
# Persistent record of plugins the user has uninstalled. Unlike the
# in-memory tombstones above (a short-lived race guard), this survives
# restarts so that a core ``git pull`` update cannot resurrect a
# built-in plugin the user removed. Built-in plugins (e.g.
# ``web-ui-info``, ``starlark-apps``) are committed into the repo under
# ``plugin-repos/``, so a plain ``git pull`` restores their files even
# after the user deleted them. ``purge_uninstalled_plugins`` re-removes
# any such resurrected directory; ``install_plugin`` clears the record
# when the user deliberately reinstalls. The file is gitignored.
if uninstalled_registry_path is not None:
self._uninstalled_registry_path = Path(uninstalled_registry_path)
else:
self._uninstalled_registry_path = (
Path(__file__).parent.parent.parent / "config" / "uninstalled_plugins.json"
)
# Serializes read-modify-write of the registry file so concurrent
# install/uninstall requests can't lose updates.
self._uninstalled_registry_lock = threading.Lock()
# Cache for _get_local_git_info: {plugin_path_str: (signature, data)}
# where ``signature`` is a tuple of (head_mtime, resolved_ref_mtime,
# head_contents) so a fast-forward update to the current branch
@@ -174,135 +143,6 @@ class PluginStoreManager:
return False
return True
def _is_valid_plugin_id(self, plugin_id: Any) -> bool:
"""Return True if ``plugin_id`` is a safe single-component plugin id.
Rejects empty strings, anything with a path separator, and traversal
sequences like ``..`` so a registry entry can never escape (or target
the root of) ``self.plugins_dir`` during a purge.
"""
return isinstance(plugin_id, str) and bool(self._PLUGIN_ID_RE.match(plugin_id))
def _read_uninstalled_registry(self) -> Set[str]:
"""Read the persistent set of uninstalled plugin IDs.
Returns an empty set if the file is missing, unreadable, or corrupt —
a broken registry must never block normal plugin operations. Invalid
ids are dropped here so callers never turn them into paths.
"""
try:
if not self._uninstalled_registry_path.exists():
return set()
with open(self._uninstalled_registry_path, 'r', encoding='utf-8') as f:
data = json.load(f)
if not isinstance(data, list):
self.logger.warning(
"Uninstalled-plugin registry at %s is not a list; ignoring it",
self._uninstalled_registry_path,
)
return set()
valid: Set[str] = set()
for pid in data:
if self._is_valid_plugin_id(pid):
valid.add(pid)
else:
self.logger.warning(
"Ignoring invalid plugin id in uninstall registry: %r", pid
)
return valid
except (OSError, ValueError) as e:
self.logger.warning(
"Could not read uninstalled-plugin registry at %s: %s",
self._uninstalled_registry_path, e,
)
return set()
def _write_uninstalled_registry(self, plugin_ids: Set[str]) -> None:
"""Persist the set of uninstalled plugin IDs (sorted, atomically)."""
path = self._uninstalled_registry_path
try:
path.parent.mkdir(parents=True, exist_ok=True)
tmp_path = path.with_suffix(path.suffix + ".tmp")
with open(tmp_path, 'w', encoding='utf-8') as f:
json.dump(sorted(plugin_ids), f, indent=2)
os.replace(tmp_path, path)
except OSError as e:
self.logger.error(
"Failed to write uninstalled-plugin registry at %s: %s", path, e
)
def record_uninstalled_plugin(self, plugin_id: str) -> None:
"""Persistently record that the user uninstalled ``plugin_id``.
Survives restarts so a core update cannot resurrect the plugin.
"""
if not self._is_valid_plugin_id(plugin_id):
self.logger.error("Refusing to record invalid plugin id: %r", plugin_id)
return
with self._uninstalled_registry_lock:
recorded = self._read_uninstalled_registry()
if plugin_id not in recorded:
recorded.add(plugin_id)
self._write_uninstalled_registry(recorded)
self.logger.info("Recorded %s as uninstalled (persistent)", plugin_id)
def forget_uninstalled_plugin(self, *plugin_ids: str) -> None:
"""Drop ``plugin_ids`` from the persistent uninstall registry.
Called when a plugin is deliberately (re)installed so future updates
keep it.
"""
with self._uninstalled_registry_lock:
recorded = self._read_uninstalled_registry()
to_remove = {pid for pid in plugin_ids if pid in recorded}
if to_remove:
self._write_uninstalled_registry(recorded - to_remove)
self.logger.info(
"Cleared uninstall record for %s", ", ".join(sorted(to_remove))
)
def get_uninstalled_plugins(self) -> Set[str]:
"""Return the persistent set of user-uninstalled plugin IDs."""
return self._read_uninstalled_registry()
def is_plugin_uninstalled(self, plugin_id: str) -> bool:
"""Return True if ``plugin_id`` is in the persistent uninstall registry."""
return plugin_id in self._read_uninstalled_registry()
def purge_uninstalled_plugins(self) -> List[str]:
"""Remove on-disk directories for plugins the user has uninstalled.
Built-in plugins committed into the repo are restored on disk by a
core ``git pull``; this re-removes any that the user previously
uninstalled. The registry entries are kept so the purge is idempotent
across every future update (until the user reinstalls). Returns the
list of plugin IDs whose directories were actually removed.
"""
removed: List[str] = []
plugins_root = self.plugins_dir.resolve()
for plugin_id in sorted(self._read_uninstalled_registry()):
plugin_path = self.plugins_dir / plugin_id
# Defense in depth: ids are already validated on read, but never
# remove anything that isn't a direct child of the plugins root.
resolved = plugin_path.resolve()
if resolved == plugins_root or resolved.parent != plugins_root:
self.logger.error(
"Refusing to purge unsafe plugin path for id %r", plugin_id
)
continue
if not plugin_path.exists():
continue
self.logger.info(
"Purging resurrected uninstalled plugin: %s", plugin_id
)
if self._safe_remove_directory(plugin_path):
removed.append(plugin_id)
else:
self.logger.error(
"Failed to purge resurrected plugin directory: %s", plugin_path
)
return removed
def _load_github_token(self) -> Optional[str]:
"""
Load GitHub API token from config_secrets.json if available.
@@ -1184,10 +1024,6 @@ class PluginStoreManager:
branch_info = f" (branch: {branch})" if branch else " (latest branch head)"
self.logger.info(f"Installing plugin: {plugin_id}{branch_info}")
# Remember the originally-requested id so we can clear its uninstall
# record on success even if the manifest renames the directory below.
requested_id = plugin_id
plugin_info = self.get_plugin_info(plugin_id, fetch_latest_from_github=True, force_refresh=True)
if not plugin_info:
self.logger.error(f"Plugin not found in registry: {plugin_id}")
@@ -1326,9 +1162,6 @@ class PluginStoreManager:
branch_display = branch_used or plugin_info.get('branch') or plugin_info.get('default_branch', 'unknown')
self.logger.info(f"Successfully installed plugin: {plugin_id} (branch {branch_display})")
# User deliberately (re)installed this plugin — clear any persistent
# uninstall record so future core updates keep it.
self.forget_uninstalled_plugin(requested_id, plugin_id)
return True
except Exception as e:

View File

@@ -43,115 +43,6 @@ class TestUninstallTombstone(unittest.TestCase):
self.assertNotIn("foo", self.sm._uninstall_tombstones)
class TestPersistentUninstallRegistry(unittest.TestCase):
"""Regression tests for the persistent uninstall registry that stops a
core `git pull` update from resurrecting built-in plugins the user
removed (plugins committed under plugin-repos/)."""
def setUp(self):
self._tmp = TemporaryDirectory()
self.addCleanup(self._tmp.cleanup)
self.plugins_dir = Path(self._tmp.name) / "plugin-repos"
self.plugins_dir.mkdir()
self.registry_path = Path(self._tmp.name) / "config" / "uninstalled_plugins.json"
self.sm = PluginStoreManager(
plugins_dir=str(self.plugins_dir),
uninstalled_registry_path=str(self.registry_path),
)
def _make_plugin_dir(self, plugin_id):
"""Simulate a built-in plugin restored on disk (e.g. by git pull)."""
d = self.plugins_dir / plugin_id
d.mkdir(parents=True)
(d / "manifest.json").write_text('{"id": "%s"}' % plugin_id)
return d
def test_unrecorded_plugin_is_not_uninstalled(self):
self.assertFalse(self.sm.is_plugin_uninstalled("web-ui-info"))
self.assertEqual(self.sm.get_uninstalled_plugins(), set())
def test_record_persists_across_instances(self):
self.sm.record_uninstalled_plugin("web-ui-info")
self.assertTrue(self.registry_path.exists())
# A fresh manager (simulating a service restart after update) still sees it.
fresh = PluginStoreManager(
plugins_dir=str(self.plugins_dir),
uninstalled_registry_path=str(self.registry_path),
)
self.assertTrue(fresh.is_plugin_uninstalled("web-ui-info"))
def test_forget_clears_record(self):
self.sm.record_uninstalled_plugin("web-ui-info")
self.sm.forget_uninstalled_plugin("web-ui-info")
self.assertFalse(self.sm.is_plugin_uninstalled("web-ui-info"))
def test_purge_removes_resurrected_plugin(self):
# The bug: user removed web-ui-info, then a git pull restored its
# committed files. Recorded uninstall + purge must re-remove it.
self._make_plugin_dir("web-ui-info")
self.sm.record_uninstalled_plugin("web-ui-info")
self.assertTrue((self.plugins_dir / "web-ui-info").exists())
removed = self.sm.purge_uninstalled_plugins()
self.assertEqual(removed, ["web-ui-info"])
self.assertFalse((self.plugins_dir / "web-ui-info").exists())
# Record is kept so the purge stays idempotent across future updates.
self.assertTrue(self.sm.is_plugin_uninstalled("web-ui-info"))
def test_purge_leaves_non_uninstalled_plugins_alone(self):
self._make_plugin_dir("baseball-scoreboard") # present, not recorded
self._make_plugin_dir("web-ui-info")
self.sm.record_uninstalled_plugin("web-ui-info")
self.sm.purge_uninstalled_plugins()
self.assertTrue((self.plugins_dir / "baseball-scoreboard").exists())
self.assertFalse((self.plugins_dir / "web-ui-info").exists())
def test_purge_noop_when_plugin_absent(self):
# Recorded but never restored on disk — nothing to remove.
self.sm.record_uninstalled_plugin("web-ui-info")
self.assertEqual(self.sm.purge_uninstalled_plugins(), [])
def test_corrupt_registry_is_ignored(self):
self.registry_path.parent.mkdir(parents=True, exist_ok=True)
self.registry_path.write_text("{ not valid json")
self.assertEqual(self.sm.get_uninstalled_plugins(), set())
self.assertFalse(self.sm.is_plugin_uninstalled("web-ui-info"))
def _write_raw_registry(self, value):
self.registry_path.parent.mkdir(parents=True, exist_ok=True)
import json as _json
self.registry_path.write_text(_json.dumps(value))
def test_empty_id_does_not_wipe_plugins_root(self):
# An empty id resolves to plugins_dir itself; purge must never delete it.
self._make_plugin_dir("baseball-scoreboard")
self._write_raw_registry([""])
removed = self.sm.purge_uninstalled_plugins()
self.assertEqual(removed, [])
self.assertTrue(self.plugins_dir.exists())
self.assertTrue((self.plugins_dir / "baseball-scoreboard").exists())
# Invalid id is filtered out entirely.
self.assertEqual(self.sm.get_uninstalled_plugins(), set())
def test_traversal_ids_are_ignored(self):
for bad in ["..", "../evil", "a/b", "."]:
with self.subTest(bad=bad):
self.assertFalse(self.sm._is_valid_plugin_id(bad))
self._write_raw_registry(["../evil", "..", "web-ui-info"])
# Only the safe id survives the read.
self.assertEqual(self.sm.get_uninstalled_plugins(), {"web-ui-info"})
def test_record_rejects_invalid_id(self):
self.sm.record_uninstalled_plugin("")
self.sm.record_uninstalled_plugin("../escape")
self.assertEqual(self.sm.get_uninstalled_plugins(), set())
class TestGitInfoCache(unittest.TestCase):
def setUp(self):
self._tmp = TemporaryDirectory()

View File

@@ -79,21 +79,6 @@ plugin_manager = PluginManager(
cache_manager=None # Not needed for web interface
)
plugin_store_manager = PluginStoreManager(plugins_dir=str(plugins_dir))
# A core `git pull` update (or any checkout) restores built-in plugins
# committed under plugin-repos/, even ones the user uninstalled. Re-remove any
# the user previously uninstalled at startup so a manual update on the Pi
# doesn't resurrect them.
try:
_purged = plugin_store_manager.purge_uninstalled_plugins()
if _purged:
logging.getLogger(__name__).info(
"Re-removed %d uninstalled plugin(s) restored since last run: %s",
len(_purged), ", ".join(_purged),
)
except (OSError, RuntimeError) as _purge_err:
logging.getLogger(__name__).warning(
"Startup plugin purge failed: %s", _purge_err
)
saved_repositories_manager = SavedRepositoriesManager()
# Initialize schema manager

View File

@@ -705,7 +705,8 @@ def save_main_config():
display_fields = ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping',
'gpio_slowdown', 'rp1_rio', 'scan_mode', 'disable_hardware_pulsing', 'inverse_colors', 'show_refresh_rate',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz', 'use_short_date_format',
'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type']
'max_dynamic_duration_seconds', 'led_rgb_sequence', 'multiplexing', 'panel_type',
'row_address_type']
if any(k in data for k in display_fields):
if 'display' not in current_config:
@@ -736,14 +737,23 @@ def save_main_config():
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': f"Invalid multiplexing value '{data['multiplexing']}'. Must be an integer from 0 to 22."}), 400
# Validate row_address_type
if 'row_address_type' in data:
try:
rat_val = int(data['row_address_type'])
if rat_val < 0 or rat_val > 4:
return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400
except (ValueError, TypeError):
return jsonify({'status': 'error', 'message': f"Invalid row_address_type '{data['row_address_type']}'. Must be an integer from 0 to 4."}), 400
# Handle hardware settings
for field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'hardware_mapping', 'scan_mode',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz',
'led_rgb_sequence', 'multiplexing', 'panel_type']:
'led_rgb_sequence', 'multiplexing', 'panel_type', 'row_address_type']:
if field in data:
if field in ['rows', 'cols', 'chain_length', 'parallel', 'brightness', 'scan_mode',
'pwm_bits', 'pwm_dither_bits', 'pwm_lsb_nanoseconds', 'limit_refresh_rate_hz',
'multiplexing']:
'multiplexing', 'row_address_type']:
current_config['display']['hardware'][field] = int(data[field])
else:
current_config['display']['hardware'][field] = data[field]
@@ -1559,20 +1569,6 @@ def execute_system_action():
pull_message = f"Code updated successfully. Local changes were automatically stashed.{stash_info}"
if result.stdout and "Already up to date" not in result.stdout:
pull_message = f"Code updated successfully.{stash_info}"
# A `git pull` restores built-in plugins (committed under
# plugin-repos/) even if the user uninstalled them. Re-remove
# any the user previously uninstalled so the update doesn't
# resurrect them.
if api_v3.plugin_store_manager:
try:
purged = api_v3.plugin_store_manager.purge_uninstalled_plugins()
if purged:
logger.info(
"Re-removed %d uninstalled plugin(s) restored by update: %s",
len(purged), ", ".join(purged),
)
except (OSError, RuntimeError) as purge_err:
logger.warning("Post-update plugin purge failed: %s", purge_err)
else:
logger.warning("git pull failed (returncode=%d): %s", result.returncode, result.stderr)
pull_message = "Update failed; check logs for details"
@@ -1588,6 +1584,66 @@ def execute_system_action():
# Try to restart the web service (assuming it's ledmatrix-web.service)
result = subprocess.run(['sudo', 'systemctl', 'restart', 'ledmatrix-web.service'],
capture_output=True, text=True, timeout=10)
elif action == 'install_base_requirements':
req_file = PROJECT_ROOT / 'requirements.txt'
if not req_file.exists():
return jsonify({'status': 'error', 'message': 'No requirements.txt found at project root'})
result = subprocess.run(
[sys.executable, '-m', 'pip', 'install', '--break-system-packages', '-r', str(req_file)],
capture_output=True, text=True, timeout=120, cwd=str(PROJECT_ROOT)
)
return jsonify({
'status': 'success' if result.returncode == 0 else 'error',
'message': 'Base requirements installed successfully' if result.returncode == 0 else 'pip install failed',
'output': (result.stdout + result.stderr).strip()
})
elif action == 'install_plugin_requirements':
plugins_dir = Path(plugin_manager.plugins_dir) if plugin_manager else PROJECT_ROOT / 'plugin-repos'
results = []
if plugins_dir.exists():
for p in sorted(plugins_dir.iterdir()):
req = p / 'requirements.txt'
if p.is_dir() and req.exists():
r = subprocess.run(
[sys.executable, '-m', 'pip', 'install', '--break-system-packages', '-r', str(req)],
capture_output=True, text=True, timeout=60
)
results.append({
'plugin': p.name,
'ok': r.returncode == 0,
'output': (r.stdout + r.stderr).strip()
})
ok_count = sum(1 for r in results if r['ok'])
all_ok = all(r['ok'] for r in results) if results else True
return jsonify({
'status': 'success' if all_ok else 'error',
'message': f'Processed {len(results)} plugin(s) — {ok_count} succeeded' if results else 'No plugin requirements.txt files found',
'details': results
})
elif action == 'force_git_reset':
project_dir = str(PROJECT_ROOT)
fetch = subprocess.run(
['git', 'fetch', 'origin'],
capture_output=True, text=True, timeout=30, cwd=project_dir
)
if fetch.returncode != 0:
return jsonify({'status': 'error', 'message': 'git fetch failed', 'output': fetch.stderr.strip()})
reset = subprocess.run(
['git', 'reset', '--hard', 'origin/main'],
capture_output=True, text=True, timeout=30, cwd=project_dir
)
return jsonify({
'status': 'success' if reset.returncode == 0 else 'error',
'message': 'Reset to origin/main successfully' if reset.returncode == 0 else 'git reset failed',
'output': (reset.stdout + reset.stderr).strip()
})
elif action == 'clear_pycache':
cleared = 0
for d in PROJECT_ROOT.rglob('__pycache__'):
if d.is_dir():
shutil.rmtree(d, ignore_errors=True)
cleared += 1
return jsonify({'status': 'success', 'message': f'Cleared {cleared} __pycache__ directories'})
else:
return jsonify({'status': 'error', 'message': 'Unknown action'}), 400
@@ -1610,6 +1666,27 @@ def execute_system_action():
logger.error("execute_system_action failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'Action failed; see logs for details'}), 500
@api_v3.route('/system/git-info', methods=['GET'])
def get_git_info():
"""Return branch, dirty state, recent commits and remote URL for the Tools tab."""
d = str(PROJECT_ROOT)
try:
branch = subprocess.run(['git', 'branch', '--show-current'], capture_output=True, text=True, timeout=10, cwd=d)
status = subprocess.run(['git', 'status', '--short', '--untracked-files=no'], capture_output=True, text=True, timeout=15, cwd=d)
log = subprocess.run(['git', 'log', '--oneline', '-5'], capture_output=True, text=True, timeout=10, cwd=d)
remote = subprocess.run(['git', 'remote', 'get-url', 'origin'], capture_output=True, text=True, timeout=10, cwd=d)
return jsonify({
'branch': branch.stdout.strip(),
'dirty': bool(status.stdout.strip()),
'status': status.stdout.strip(),
'recent_commits': log.stdout.strip(),
'remote_url': remote.stdout.strip(),
})
except Exception as e:
logger.error("get_git_info failed: %s", e, exc_info=True)
return jsonify({'status': 'error', 'message': 'Failed to get git info'}), 500
@api_v3.route('/hardware/status', methods=['GET'])
def get_hardware_status():
"""Return LED matrix hardware initialization status written by display_manager at startup."""
@@ -2947,13 +3024,6 @@ def _do_transactional_uninstall(plugin_id, preserve_config):
api_v3.schema_manager.invalidate_cache(plugin_id)
if api_v3.plugin_state_manager:
api_v3.plugin_state_manager.remove_plugin_state(plugin_id)
# Persistently record the uninstall so a later core `git pull` update
# cannot resurrect a built-in plugin (committed under plugin-repos/) that
# the user removed. Best-effort: never fail the uninstall over this.
try:
api_v3.plugin_store_manager.record_uninstalled_plugin(plugin_id)
except Exception as record_err:
logger.warning("Could not record uninstall for %s: %s", plugin_id, record_err)
return True, None

View File

@@ -90,6 +90,8 @@ def load_partial(partial_name):
return _load_cache_partial()
elif partial_name == 'operation-history':
return _load_operation_history_partial()
elif partial_name == 'tools':
return _load_tools_partial()
else:
return "Partial not found", 404
@@ -448,6 +450,15 @@ def _load_operation_history_partial():
return "Error loading partial", 500
def _load_tools_partial():
"""Load tools/utilities partial."""
try:
return render_template('v3/partials/tools.html')
except Exception:
logger.error("Error loading partial", exc_info=True)
return "Error loading partial", 500
def _load_plugin_config_partial(plugin_id):
"""
Load plugin configuration partial - server-side rendered form.

View File

@@ -1009,6 +1009,11 @@
class="nav-tab">
<i class="fas fa-history"></i>Operation History
</button>
<button @click="activeTab = 'tools'"
:class="activeTab === 'tools' ? 'nav-tab-active' : ''"
class="nav-tab">
<i class="fas fa-tools"></i>Tools
</button>
</nav>
</div>
@@ -1290,6 +1295,18 @@
</div>
</div>
<!-- Tools tab -->
<div x-show="activeTab === 'tools'" x-transition>
<div id="tools-content" hx-get="/v3/partials/tools" hx-trigger="loadtab" hx-swap="innerHTML">
<div class="animate-pulse">
<div class="bg-white rounded-lg shadow p-6">
<div class="h-4 bg-gray-200 rounded w-1/4 mb-4"></div>
<div class="h-32 bg-gray-200 rounded"></div>
</div>
</div>
</div>
</div>
<!-- Dynamic Plugin Tabs - HTMX Lazy Loading -->
<!--
Architecture: Server-side rendered plugin configuration forms

View File

@@ -166,6 +166,18 @@
</select>
<p class="mt-1 text-sm text-gray-600">Special panel chipset initialization (use Standard unless your panel requires it)</p>
</div>
<div class="form-group">
<label for="row_address_type" class="block text-sm font-medium text-gray-700">Row Address Type</label>
<select id="row_address_type" name="row_address_type" class="form-control">
<option value="0" {% if main_config.display.hardware.get('row_address_type', 0)|int == 0 %}selected{% endif %}>0 - Default</option>
<option value="1" {% if main_config.display.hardware.get('row_address_type', 0)|int == 1 %}selected{% endif %}>1 - AB-addressed panels</option>
<option value="2" {% if main_config.display.hardware.get('row_address_type', 0)|int == 2 %}selected{% endif %}>2 - Row direct</option>
<option value="3" {% if main_config.display.hardware.get('row_address_type', 0)|int == 3 %}selected{% endif %}>3 - ABC-addressed panels</option>
<option value="4" {% if main_config.display.hardware.get('row_address_type', 0)|int == 4 %}selected{% endif %}>4 - ABC Shift + DE direct</option>
</select>
<p class="mt-1 text-sm text-gray-600">Row addressing scheme — leave at Default (0) unless your panel requires a specific type</p>
</div>
</div>
<div class="grid grid-cols-1 md:grid-cols-3 gap-4">

View File

@@ -0,0 +1,306 @@
<div class="space-y-6" id="tools-root">
<!-- Git & Updates -->
<div class="bg-white rounded-lg shadow p-6">
<div class="border-b border-gray-200 pb-4 mb-6">
<h2 class="text-lg font-semibold text-gray-900">Git &amp; Updates</h2>
<p class="mt-1 text-sm text-gray-600">Inspect the current git state and pull or reset to the latest remote code.</p>
</div>
<!-- Git status info -->
<div id="git-info-panel" class="mb-6 bg-gray-50 border border-gray-200 rounded-lg p-4 text-sm">
<div class="animate-pulse text-gray-400">Loading git info…</div>
</div>
<div class="space-y-4">
<!-- Pull latest -->
<div class="flex items-start justify-between gap-4">
<div>
<p class="text-sm font-medium text-gray-900">Pull latest (rebase)</p>
<p class="text-xs text-gray-500 mt-0.5">Stashes local changes, runs <code class="bg-gray-100 px-1 rounded">git pull --rebase</code>, then restores the stash.</p>
</div>
<button id="btn-git-pull" onclick="toolsAction('git_pull', 'btn-git-pull', 'result-git-pull')"
class="shrink-0 inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-download mr-2"></i>Pull Latest
</button>
</div>
<div id="result-git-pull" class="hidden"></div>
<!-- Force reset -->
<div class="flex items-start justify-between gap-4 pt-4 border-t border-gray-100">
<div>
<p class="text-sm font-medium text-gray-900">Force reset to <code class="bg-gray-100 px-1 rounded">origin/main</code></p>
<p class="text-xs text-gray-500 mt-0.5">Runs <code class="bg-gray-100 px-1 rounded">git fetch origin</code> then <code class="bg-gray-100 px-1 rounded">git reset --hard origin/main</code>. Discards all local changes.</p>
</div>
<div class="shrink-0 flex flex-col items-end gap-2">
<button id="btn-force-reset-confirm" onclick="showForceResetConfirm()"
class="inline-flex items-center px-3 py-2 border border-red-300 text-sm font-medium rounded-md text-red-700 bg-white hover:bg-red-50">
<i class="fas fa-exclamation-triangle mr-2"></i>Force Reset…
</button>
<div id="force-reset-confirm-row" class="hidden flex items-center gap-2">
<span class="text-xs text-red-700 font-medium">This discards all local changes. Sure?</span>
<button onclick="toolsAction('force_git_reset', 'btn-force-reset-confirm', 'result-force-reset'); hideForceResetConfirm()"
class="inline-flex items-center px-3 py-2 border border-transparent text-sm font-medium rounded-md text-white bg-red-600 hover:bg-red-700">
Yes, reset
</button>
<button onclick="hideForceResetConfirm()"
class="inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
Cancel
</button>
</div>
</div>
</div>
<div id="result-force-reset" class="hidden"></div>
</div>
</div>
<!-- Python Dependencies -->
<div class="bg-white rounded-lg shadow p-6">
<div class="border-b border-gray-200 pb-4 mb-6">
<h2 class="text-lg font-semibold text-gray-900">Python Dependencies</h2>
<p class="mt-1 text-sm text-gray-600">Re-run <code class="bg-gray-100 px-1 rounded">pip install</code> to fix missing or broken packages.</p>
</div>
<div class="space-y-4">
<!-- Base requirements -->
<div class="flex items-start justify-between gap-4">
<div>
<p class="text-sm font-medium text-gray-900">Reinstall base requirements</p>
<p class="text-xs text-gray-500 mt-0.5">Installs from <code class="bg-gray-100 px-1 rounded">requirements.txt</code> in the project root.</p>
</div>
<button id="btn-base-reqs" onclick="toolsAction('install_base_requirements', 'btn-base-reqs', 'result-base-reqs', true)"
class="shrink-0 inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-box mr-2"></i>Reinstall Base
</button>
</div>
<div id="result-base-reqs" class="hidden"></div>
<!-- Plugin requirements -->
<div class="flex items-start justify-between gap-4 pt-4 border-t border-gray-100">
<div>
<p class="text-sm font-medium text-gray-900">Reinstall plugin requirements</p>
<p class="text-xs text-gray-500 mt-0.5">Runs <code class="bg-gray-100 px-1 rounded">pip install</code> for every installed plugin that has a <code class="bg-gray-100 px-1 rounded">requirements.txt</code>.</p>
</div>
<button id="btn-plugin-reqs" onclick="toolsAction('install_plugin_requirements', 'btn-plugin-reqs', 'result-plugin-reqs', false, true)"
class="shrink-0 inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-puzzle-piece mr-2"></i>Reinstall Plugin Deps
</button>
</div>
<div id="result-plugin-reqs" class="hidden"></div>
</div>
</div>
<!-- Maintenance -->
<div class="bg-white rounded-lg shadow p-6">
<div class="border-b border-gray-200 pb-4 mb-6">
<h2 class="text-lg font-semibold text-gray-900">Maintenance</h2>
<p class="mt-1 text-sm text-gray-600">Housekeeping operations that don't affect config or plugins.</p>
</div>
<div class="space-y-4">
<div class="flex items-start justify-between gap-4">
<div>
<p class="text-sm font-medium text-gray-900">Clear Python cache</p>
<p class="text-xs text-gray-500 mt-0.5">Deletes all <code class="bg-gray-100 px-1 rounded">__pycache__</code> directories in the project. Useful after switching branches or debugging import issues.</p>
</div>
<button id="btn-clear-pycache" onclick="toolsAction('clear_pycache', 'btn-clear-pycache', 'result-clear-pycache')"
class="shrink-0 inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-broom mr-2"></i>Clear Cache
</button>
</div>
<div id="result-clear-pycache" class="hidden"></div>
</div>
</div>
<!-- Services -->
<div class="bg-white rounded-lg shadow p-6">
<div class="border-b border-gray-200 pb-4 mb-6">
<h2 class="text-lg font-semibold text-gray-900">Services</h2>
<p class="mt-1 text-sm text-gray-600">Quick access to service restarts.</p>
</div>
<div class="space-y-4">
<div class="flex items-start justify-between gap-4">
<div>
<p class="text-sm font-medium text-gray-900">Restart display service</p>
<p class="text-xs text-gray-500 mt-0.5">Restarts <code class="bg-gray-100 px-1 rounded">ledmatrix.service</code>.</p>
</div>
<button id="btn-restart-display" onclick="toolsAction('restart_display_service', 'btn-restart-display', 'result-restart-display')"
class="shrink-0 inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-sync-alt mr-2"></i>Restart Display
</button>
</div>
<div id="result-restart-display" class="hidden"></div>
<div class="flex items-start justify-between gap-4 pt-4 border-t border-gray-100">
<div>
<p class="text-sm font-medium text-gray-900">Restart web interface</p>
<p class="text-xs text-gray-500 mt-0.5">Restarts <code class="bg-gray-100 px-1 rounded">ledmatrix-web.service</code>. The page will go offline briefly.</p>
</div>
<button id="btn-restart-web" onclick="toolsAction('restart_web_service', 'btn-restart-web', 'result-restart-web')"
class="shrink-0 inline-flex items-center px-3 py-2 border border-gray-300 text-sm font-medium rounded-md text-gray-700 bg-white hover:bg-gray-50">
<i class="fas fa-globe mr-2"></i>Restart Web
</button>
</div>
<div id="result-restart-web" class="hidden"></div>
</div>
</div>
</div>
<script>
(function () {
// ── helpers ──────────────────────────────────────────────────────────────
function setBusy(btnId, busy) {
const btn = document.getElementById(btnId);
if (!btn) return;
btn.disabled = busy;
btn.style.opacity = busy ? '0.6' : '';
btn.style.cursor = busy ? 'wait' : '';
const icon = btn.querySelector('i');
if (icon) {
if (busy) {
icon.dataset.origClass = icon.className;
icon.className = 'fas fa-spinner fa-spin mr-2';
} else if (icon.dataset.origClass) {
icon.className = icon.dataset.origClass;
}
}
}
function showResult(resultId, ok, message, output, pluginDetails) {
const el = document.getElementById(resultId);
if (!el) return;
el.classList.remove('hidden');
const color = ok ? 'green' : 'red';
const icon = ok ? 'fa-check-circle' : 'fa-times-circle';
let html = `
<div class="mt-3 rounded-md p-3 bg-${color}-50 border border-${color}-200">
<div class="flex items-start gap-2">
<i class="fas ${icon} text-${color}-600 mt-0.5"></i>
<span class="text-sm text-${color}-800">${escHtml(message)}</span>
</div>`;
if (output) {
html += `
<details class="mt-2">
<summary class="text-xs text-${color}-700 cursor-pointer hover:underline">Show output</summary>
<pre class="mt-2 text-xs bg-gray-900 text-gray-100 rounded p-3 overflow-x-auto whitespace-pre-wrap">${escHtml(output)}</pre>
</details>`;
}
if (pluginDetails && pluginDetails.length > 0) {
html += `<ul class="mt-3 space-y-1">`;
for (const d of pluginDetails) {
const dc = d.ok ? 'green' : 'red';
const di = d.ok ? 'fa-check' : 'fa-times';
html += `<li class="text-xs flex items-start gap-1">
<i class="fas ${di} text-${dc}-600 mt-0.5 w-3"></i>
<span class="text-gray-700">${escHtml(d.plugin)}</span>`;
if (d.output) {
html += ` <details class="inline"><summary class="cursor-pointer text-gray-400 hover:underline ml-1">output</summary>
<pre class="mt-1 text-xs bg-gray-900 text-gray-100 rounded p-2 overflow-x-auto whitespace-pre-wrap">${escHtml(d.output)}</pre></details>`;
}
html += `</li>`;
}
html += `</ul>`;
}
html += `</div>`;
el.innerHTML = html;
}
function escHtml(s) {
return String(s || '').replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;');
}
// ── main action dispatcher ────────────────────────────────────────────────
window.toolsAction = function(action, btnId, resultId, showOutput, showPluginDetails) {
setBusy(btnId, true);
const el = document.getElementById(resultId);
if (el) el.classList.add('hidden');
fetch('/api/v3/system/action', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({action})
})
.then(r => r.json())
.then(data => {
const ok = data.status === 'success';
showResult(
resultId, ok,
data.message || (ok ? 'Done' : 'Failed'),
showOutput ? (data.output || '') : '',
showPluginDetails ? (data.details || []) : null
);
})
.catch(err => {
showResult(resultId, false, 'Request failed: ' + err.message);
})
.finally(() => setBusy(btnId, false));
};
// ── force-reset confirm helpers ───────────────────────────────────────────
window.showForceResetConfirm = function() {
document.getElementById('force-reset-confirm-row').classList.remove('hidden');
document.getElementById('btn-force-reset-confirm').classList.add('hidden');
};
window.hideForceResetConfirm = function() {
document.getElementById('force-reset-confirm-row').classList.add('hidden');
document.getElementById('btn-force-reset-confirm').classList.remove('hidden');
};
// ── git info panel ────────────────────────────────────────────────────────
function loadGitInfo() {
const panel = document.getElementById('git-info-panel');
if (!panel) return;
fetch('/api/v3/system/git-info')
.then(r => r.json())
.then(d => {
const dirtyBadge = d.dirty
? '<span class="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-yellow-100 text-yellow-800">dirty</span>'
: '<span class="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-green-100 text-green-800">clean</span>';
let html = `<div class="space-y-2">
<div class="flex items-center gap-2">
<i class="fas fa-code-branch text-gray-400"></i>
<span class="font-mono text-gray-800">${escHtml(d.branch || 'unknown')}</span>
${dirtyBadge}
</div>`;
if (d.dirty && d.status) {
html += `<pre class="text-xs bg-yellow-50 border border-yellow-200 rounded p-2 overflow-x-auto whitespace-pre-wrap text-yellow-900">${escHtml(d.status)}</pre>`;
}
if (d.recent_commits) {
html += `<div class="mt-2">
<p class="text-xs text-gray-500 mb-1">Recent commits</p>
<pre class="text-xs bg-gray-50 border border-gray-200 rounded p-2 overflow-x-auto whitespace-pre-wrap text-gray-700">${escHtml(d.recent_commits)}</pre>
</div>`;
}
if (d.remote_url) {
html += `<p class="text-xs text-gray-400 mt-1"><i class="fas fa-cloud mr-1"></i>${escHtml(d.remote_url)}</p>`;
}
html += `</div>`;
panel.innerHTML = html;
})
.catch(() => {
panel.innerHTML = '<span class="text-sm text-red-600">Could not load git info.</span>';
});
}
// Load on first render; HTMX will have already swapped us in by this point.
loadGitInfo();
})();
</script>