rick/LEDMatrix
1
0
Fork 0
mirror of https://github.com/ChuckBuilds/LEDMatrix.git synced 2026-10-04 14:25:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: delete dead Python code in the core (and stop storing Wi-Fi passwords) (#608)

Browse Source 
* refactor(plugins): remove the no-op PluginHealthMonitor

Its monitor loop did nothing (`if callbacks: pass`), register_health_check
had no callers and api_v3.health_monitor was never read by any route. The
live health data comes from PluginHealthTracker, which is untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(store): drop the never-set uninstall tombstones

Nothing in production called mark_recently_uninstalled, so the
reconciler's was_recently_uninstalled check was always False. The
persistent uninstall registry is what actually stops resurrection; the
reconciler test now exercises that gate instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(common): delete unused config/display/game helpers, utils and error_handler

Nothing in core, the web UI, scripts or the plugin monorepo imports
config_helper, display_helper, game_helper, utils or error_handler; only
their own tests did. The error_handler re-exports leave src.common's
__all__; APIHelper, TextHelper, ScrollHelper, LogoHelper and the adaptive
layout exports are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(config): drop ConfigService's unused versioning and save API

ConfigVersion, get_version/get_version_history/get_version_config,
rollback, save_config, reload, get_plugin_config and the backward-compat
load_config/get_config_path/get_secrets_path had no callers. The display
controller only uses get_config, subscribe, unsubscribe and shutdown,
plus the file watcher. Change detection now compares against the
current checksum instead of the last history entry.

The subscriber tests asserted `callback.called or True`; they now
reload the way the watcher does and assert the notification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): drop unread plugin state history and callbacks

plugin_state.PluginStateManager kept a bounded per-plugin transition
history that only get_state_history (tests only) read; get_state_info
reports a separate lifetime count, which stays. set_error_info and
record_display had no callers, and set_state_with_error's `error`
argument only fed the history.

The web-side state_manager.PluginStateManager loses
subscribe_to_state_changes, _notify_callbacks, set_plugin_error and
get_state_version, none of which had callers; with no subscribers the
old-state copy in update_plugin_state went with them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): remove unused PluginManager methods and attribute guards

update_all_plugins was only called by a test (the display loop uses
run_scheduled_updates); get_plugin_health_metrics,
get_plugin_resource_metrics and get_plugin_state had no callers; and
plugin_modules was written but never read. plugin_directories is now
initialised in __init__, so the hasattr() guards around it go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): remove unused executor, loader, store and package helpers

- PluginExecutor.execute_safe: no callers.
- PluginLoader._parse_semver: only its own tests; compatibility.parse_semver
  is the live copy and test_compatibility.py already covers it.
- PluginStoreManager.get_installed_plugin_info: no callers.
- PluginResourceMonitor._local: never read.
- src.plugin_system.get_store_manager and __api_version__: no importers in
  core, scripts or the plugin monorepo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(wifi): stop storing Wi-Fi passwords in wifi_config.json

WiFiManager appended every joined network's SSID and password, in
plaintext, to saved_networks in config/wifi_config.json, and nothing
(web UI, backup restore, scripts) ever read them back: NetworkManager
keeps its own credentials. The writes are gone, and loading the config
now drops any saved_networks key and rewrites the file, so passwords
already on disk are scrubbed.

Also removes _check_dnsmasq_conflict (never called) and _detect_trixie,
whose result only reached one log line, along with the
NM_CONNECTIONS_PATHS constant only it used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(display): remove unreachable and unused DisplayController code

- _follower_rebuild_scroll_image: never called.
- mode_duration (never read) and last_mode_change (write-only).
- The `chosen_cap <= 0` branch: chosen_cap is either the minimum of
  caps already filtered to > 0 or DEFAULT_DYNAMIC_DURATION_CAP (180).
- The `max_duration < min_duration` branch directly after
  `max_duration = max(min_duration, max_duration)`.
- The circuit-breaker branch's `display_result = False` and
  `manager_to_display = None`: the first is overwritten a few lines
  later, the second is already None there.
- The bool-to-bool conversion of execute_display's result, which is
  always a bool.
- The `loaded_plugins` lookup in _update_modules: PluginManager has no
  such attribute, so it always fell through to `plugins`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(vegas): remove unused config update, boundary finder and refresh

VegasModeConfig.update had no callers outside its own tests (the
coordinator rebuilds the config with from_config on a change);
geometry.find_item_boundary and StreamManager._refresh_plugin_content
had no callers at all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(run): drop the debug block that pretended to import the plugin system

In debug mode run.py put src/plugin_system itself on sys.path and printed
"Plugin system import successful" without importing anything. Nothing
imports plugin_system modules by bare name, so the path entry did
nothing either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: delete tests that test nothing

- test/plugins/test_{basketball_scoreboard,calendar,clock_simple,
  odds_ticker,soccer_scoreboard,text_display}.py skip everywhere the named
  plugins are not installed, including CI (LEDMATRIX_PLUGINS_DIR holds only
  the fixture plugin); test_plugin_matrix.py already covers every
  discovered plugin. Their PluginTestBase and the fixtures only it used
  (plugins_dir, mock_display_manager, mock_cache_manager,
  mock_plugin_manager, base_plugin_config in test/plugins/conftest.py) go
  with them.
- test_plugin_system.py: test_discover_plugins (body was `pass`) and
  test_dependency_check (a comment), plus the test_plugin_manager fixture
  only the former requested.
- test_display_manager.py: test_draw_image asserted that an image it had
  just assigned was not None.
- test_display_controller.py: the rotation and schedule-override tests
  re-implemented the run-loop arithmetic inline and asserted on their own
  result without calling the controller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: expect one plugin_last_update success stamp after update_all_plugins

EveryStampRecordsACompletion required at least two success-path stamps;
the second was update_all_plugins, removed as test-only. The worker and
synchronous paths share the remaining stamp in _execute_update_now, and
the check that every stamp calls _note_update_completed is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Chuck
2026-09-23 12:36:26 -04:00
committed by GitHub
co-authored by Claude Opus 5.5
parent e1ce7189f1
commit 84afa9d64f
57 changed files with 273 additions and 6254 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/plugin_system/__init__.py
-13
View File
@@ -3,28 +3,15 @@ LEDMatrix Plugin System
This module provides the core plugin infrastructure for the LEDMatrix project.
It enables dynamic loading, management, and discovery of display plugins.
API Version: 1.0.0
"""
__version__ = "1.0.0"
__api_version__ = "1.0.0"
from .base_plugin import BasePlugin
from .plugin_manager import PluginManager
# Import store_manager only when needed to avoid dependency issues
def get_store_manager():
"""Get PluginStoreManager, importing only when needed."""
try:
from .store_manager import PluginStoreManager
return PluginStoreManager
except ImportError as e:
raise ImportError("PluginStoreManager requires additional dependencies. Install requests: pip install requests") from e
__all__ = [
'BasePlugin',
'PluginManager',
'get_store_manager',
]
src/plugin_system/health_monitor.py
-319
View File
@@ -1,319 +0,0 @@
"""
Enhanced plugin health monitoring with background checks and auto-recovery.
Builds on existing PluginHealthTracker to provide:
- Background health checks
- Health status determination (healthy/degraded/unhealthy)
- Auto-recovery suggestions
- Health metrics aggregation
"""
import threading
import time
from typing import Dict, Any, Optional, List, Callable
from datetime import datetime
from enum import Enum
from dataclasses import dataclass
from src.logging_config import get_logger
class HealthStatus(Enum):
"""Overall health status of a plugin."""
HEALTHY = "healthy"
DEGRADED = "degraded"
UNHEALTHY = "unhealthy"
UNKNOWN = "unknown"
@dataclass
class HealthMetrics:
"""Health metrics for a plugin."""
plugin_id: str
status: HealthStatus
last_successful_update: Optional[datetime]
error_rate: float # 0.0 to 1.0
average_response_time: Optional[float] # seconds
consecutive_failures: int
total_failures: int
total_successes: int
success_rate: float # 0.0 to 1.0
last_error: Optional[str]
circuit_breaker_state: str
recovery_suggestions: List[str]
class PluginHealthMonitor:
"""
Enhanced health monitoring for plugins.
Provides:
- Background health checks
- Health status determination
- Auto-recovery suggestions
- Health metrics aggregation
"""
def __init__(
self,
health_tracker,
check_interval: float = 60.0,
degraded_threshold: float = 0.5, # 50% error rate
unhealthy_threshold: float = 0.8, # 80% error rate
max_response_time: float = 5.0 # seconds
):
"""
Initialize health monitor.
Args:
health_tracker: PluginHealthTracker instance
check_interval: Interval between background health checks (seconds)
degraded_threshold: Error rate threshold for degraded status
unhealthy_threshold: Error rate threshold for unhealthy status
max_response_time: Maximum acceptable response time (seconds)
"""
self.health_tracker = health_tracker
self.check_interval = check_interval
self.degraded_threshold = degraded_threshold
self.unhealthy_threshold = unhealthy_threshold
self.max_response_time = max_response_time
self.logger = get_logger(__name__)
# Background check thread
self._monitor_thread: Optional[threading.Thread] = None
self._stop_event = threading.Event()
# Health check callbacks
self._health_check_callbacks: List[Callable[[str], Dict[str, Any]]] = []
def start_monitoring(self) -> None:
"""Start background health monitoring."""
if self._monitor_thread and self._monitor_thread.is_alive():
return
self._stop_event.clear()
self._monitor_thread = threading.Thread(
target=self._monitor_loop,
daemon=True,
name="PluginHealthMonitor"
)
self._monitor_thread.start()
self.logger.info("Started plugin health monitoring")
def stop_monitoring(self) -> None:
"""Stop background health monitoring."""
self._stop_event.set()
if self._monitor_thread and self._monitor_thread.is_alive():
self._monitor_thread.join(timeout=5.0)
self.logger.info("Stopped plugin health monitoring")
def register_health_check(self, callback: Callable[[str], Dict[str, Any]]) -> None:
"""
Register a callback for health checks.
Callback should accept plugin_id and return dict with health info.
"""
self._health_check_callbacks.append(callback)
def get_plugin_health_status(self, plugin_id: str) -> HealthStatus:
"""
Determine overall health status for a plugin.
Args:
plugin_id: Plugin identifier
Returns:
HealthStatus enum value
"""
if not self.health_tracker:
return HealthStatus.UNKNOWN
summary = self.health_tracker.get_health_summary(plugin_id)
if not summary:
return HealthStatus.UNKNOWN
# Check circuit breaker state
circuit_state = summary.get('circuit_state', 'closed')
if circuit_state == 'open':
return HealthStatus.UNHEALTHY
# Check error rate
success_rate = summary.get('success_rate', 100.0)
error_rate = 1.0 - (success_rate / 100.0)
if error_rate >= self.unhealthy_threshold:
return HealthStatus.UNHEALTHY
elif error_rate >= self.degraded_threshold:
return HealthStatus.DEGRADED
else:
return HealthStatus.HEALTHY
def get_plugin_health_metrics(self, plugin_id: str) -> HealthMetrics:
"""
Get comprehensive health metrics for a plugin.
Args:
plugin_id: Plugin identifier
Returns:
HealthMetrics object
"""
if not self.health_tracker:
return HealthMetrics(
plugin_id=plugin_id,
status=HealthStatus.UNKNOWN,
last_successful_update=None,
error_rate=0.0,
average_response_time=None,
consecutive_failures=0,
total_failures=0,
total_successes=0,
success_rate=0.0,
last_error=None,
circuit_breaker_state="unknown",
recovery_suggestions=[]
)
summary = self.health_tracker.get_health_summary(plugin_id)
if not summary:
return HealthMetrics(
plugin_id=plugin_id,
status=HealthStatus.UNKNOWN,
last_successful_update=None,
error_rate=0.0,
average_response_time=None,
consecutive_failures=0,
total_failures=0,
total_successes=0,
success_rate=0.0,
last_error=None,
circuit_breaker_state="unknown",
recovery_suggestions=[]
)
# Calculate metrics
success_rate = summary.get('success_rate', 100.0) / 100.0
error_rate = 1.0 - success_rate
# Parse last success time
last_success_time = None
if summary.get('last_success_time'):
try:
last_success_time = datetime.fromisoformat(summary['last_success_time'])
except (ValueError, TypeError):
pass
# Determine status
status = self.get_plugin_health_status(plugin_id)
# Get recovery suggestions
recovery_suggestions = self._get_recovery_suggestions(plugin_id, summary, status)
return HealthMetrics(
plugin_id=plugin_id,
status=status,
last_successful_update=last_success_time,
error_rate=error_rate,
average_response_time=None, # Would need resource monitor for this
consecutive_failures=summary.get('consecutive_failures', 0),
total_failures=summary.get('total_failures', 0),
total_successes=summary.get('total_successes', 0),
success_rate=success_rate,
last_error=summary.get('last_error'),
circuit_breaker_state=summary.get('circuit_state', 'closed'),
recovery_suggestions=recovery_suggestions
)
def get_all_plugin_health(self) -> Dict[str, HealthMetrics]:
"""
Get health metrics for all tracked plugins.
Returns:
Dictionary mapping plugin_id to HealthMetrics
"""
if not self.health_tracker:
return {}
summaries = self.health_tracker.get_all_health_summaries()
health_metrics = {}
for plugin_id in summaries.keys():
health_metrics[plugin_id] = self.get_plugin_health_metrics(plugin_id)
return health_metrics
def _get_recovery_suggestions(
self,
plugin_id: str,
summary: Dict[str, Any],
status: HealthStatus
) -> List[str]:
"""
Generate recovery suggestions based on health status.
Args:
plugin_id: Plugin identifier
summary: Health summary from tracker
status: Current health status
Returns:
List of suggested recovery actions
"""
suggestions = []
if status == HealthStatus.UNHEALTHY:
suggestions.append("Plugin is unhealthy - check plugin logs for errors")
suggestions.append("Verify plugin configuration is correct")
suggestions.append("Check if plugin dependencies are installed")
if summary.get('circuit_state') == 'open':
suggestions.append("Circuit breaker is open - plugin is being skipped")
suggestions.append("Wait for cooldown period or manually reset health")
if summary.get('consecutive_failures', 0) > 0:
suggestions.append(f"Plugin has {summary['consecutive_failures']} consecutive failures")
suggestions.append("Consider disabling plugin temporarily")
elif status == HealthStatus.DEGRADED:
suggestions.append("Plugin is degraded - experiencing intermittent failures")
suggestions.append("Monitor plugin performance")
suggestions.append("Check for resource constraints (CPU, memory)")
error_rate = (1.0 - (summary.get('success_rate', 100.0) / 100.0)) * 100
suggestions.append(f"Current error rate: {error_rate:.1f}%")
elif status == HealthStatus.HEALTHY:
suggestions.append("Plugin is healthy - no action needed")
# Add specific suggestions based on last error
last_error = summary.get('last_error')
if last_error:
if "timeout" in last_error.lower():
suggestions.append("Last error was a timeout - plugin may be slow or unresponsive")
elif "import" in last_error.lower() or "module" in last_error.lower():
suggestions.append("Last error suggests missing dependencies")
elif "permission" in last_error.lower() or "access" in last_error.lower():
suggestions.append("Last error suggests permission issues")
return suggestions
def _monitor_loop(self) -> None:
"""Background monitoring loop."""
while not self._stop_event.is_set():
try:
# Run health checks for all plugins
if self._health_check_callbacks:
# Get list of plugin IDs (would need plugin manager reference)
# For now, just wait
pass
# Sleep until next check
self._stop_event.wait(self.check_interval)
except Exception as e:
self.logger.error(f"Error in health monitor loop: {e}", exc_info=True)
# Continue monitoring even if there's an error
time.sleep(self.check_interval)
src/plugin_system/plugin_executor.py
-37
View File
@@ -238,40 +238,3 @@ class PluginExecutor:
)
record_error(e, plugin_id=plugin_id, operation="display")
return False
def execute_safe(
self,
operation: Callable[[], Any],
plugin_id: str,
operation_name: str = "operation",
timeout: Optional[float] = None,
default_return: Any = None
) -> Any:
"""
Execute an operation safely, returning default on error.
Args:
operation: Function to execute
plugin_id: Plugin identifier
operation_name: Name of operation for logging
timeout: Timeout in seconds (None = use default)
default_return: Value to return on error
Returns:
Result of operation or default_return on error
"""
try:
return self.execute_with_timeout(
operation,
timeout=timeout,
plugin_id=plugin_id
)
except Exception as e: # covers PluginTimeoutError, PluginError, and unexpected errors
self.logger.warning(
"Plugin %s %s failed, using default return: %s",
plugin_id,
operation_name,
e
)
return default_return
src/plugin_system/plugin_loader.py
-15
View File
@@ -752,21 +752,6 @@ class PluginLoader:
self.logger.error(error_msg, exc_info=True)
raise PluginError(error_msg, plugin_id=plugin_id) from e
@staticmethod
def _parse_semver(value: Any) -> Optional[Tuple[int, int, int]]:
"""Parse 'X.Y.Z' (extra parts/suffixes ignored) into a comparable
3-tuple, or None when unparseable."""
if not isinstance(value, str):
return None
parts = value.strip().lstrip('v').split('.')
try:
nums = [int(''.join(ch for ch in p if ch.isdigit()) or 0) for p in parts[:3]]
except ValueError:
return None
while len(nums) < 3:
nums.append(0)
return tuple(nums) # type: ignore[return-value]
def _warn_if_incompatible(self, plugin_id: str, manifest: Dict[str, Any]) -> None:
"""Log one warning when a plugin declares a minimum LEDMatrix version
newer than the running core. Advisory only — never raises — so a
src/plugin_system/plugin_manager.py
+8 -115
View File
@@ -86,15 +86,14 @@ class PluginManager:
self._skip_reported: set = set()
# Lock protecting plugin_last_update from concurrent mutation/iteration.
# It's written from run_scheduled_updates()/update_all_plugins() (main
# loop) and read/diffed by run_scheduled_updates_with_changes(), which
# It's written from run_scheduled_updates() (main loop) and read/diffed by run_scheduled_updates_with_changes(), which
# Vegas mode calls from its own background update-tick thread.
self._plugin_last_update_lock = threading.RLock()
# Active plugins
self.plugins: Dict[str, Any] = {}
self.plugin_manifests: Dict[str, Dict[str, Any]] = {}
self.plugin_modules: Dict[str, Any] = {}
self.plugin_directories: Dict[str, Path] = {}
self.plugin_last_update: Dict[str, float] = {}
# Cached data-fetch intervals per plugin_id.
@@ -263,10 +262,7 @@ class PluginManager:
with self._discovery_lock:
self.plugin_manifests.clear()
self.plugin_manifests.update(new_manifests)
if not hasattr(self, 'plugin_directories'):
self.plugin_directories = {}
else:
self.plugin_directories.clear()
self.plugin_directories.clear()
self.plugin_directories.update(new_directories)
return plugin_ids
@@ -327,11 +323,10 @@ class PluginManager:
self.state_manager.set_state(plugin_id, PluginState.LOADED)
# Find plugin directory using PluginLoader
plugin_directories = getattr(self, 'plugin_directories', None)
plugin_dir = self.plugin_loader.find_plugin_directory(
plugin_id,
self.plugins_dir,
plugin_directories
self.plugin_directories
)
if plugin_dir is None:
@@ -341,9 +336,7 @@ class PluginManager:
return False
# Update mapping if found via search
if plugin_directories is None or plugin_id not in plugin_directories:
if not hasattr(self, 'plugin_directories'):
self.plugin_directories = {}
if plugin_id not in self.plugin_directories:
self.plugin_directories[plugin_id] = plugin_dir
# Get plugin config
@@ -379,7 +372,7 @@ class PluginManager:
config = self.prepare_plugin_config(plugin_id, config, schema=schema)
# Use PluginLoader to load plugin
plugin_instance, module = self.plugin_loader.load_plugin(
plugin_instance, _module = self.plugin_loader.load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=plugin_dir,
@@ -391,9 +384,6 @@ class PluginManager:
plugins_dir=self.plugins_dir,
)
# Store module
self.plugin_modules[plugin_id] = module
# Register plugin-shipped fonts with the FontManager (if any).
# Plugin manifests can declare a "fonts" block that ships custom
# fonts with the plugin; FontManager.register_plugin_fonts handles
@@ -633,9 +623,6 @@ class PluginManager:
# Delegate sub-module and cached-module cleanup to the loader
self.plugin_loader.unregister_plugin_modules(plugin_id)
# Remove from plugin_modules
self.plugin_modules.pop(plugin_id, None)
# Update state
self.state_manager.set_state(plugin_id, PluginState.UNLOADED)
self.state_manager.clear_state(plugin_id)
@@ -778,7 +765,7 @@ class PluginManager:
resolved further: dev plugins are symlinks into ``plugins_dir``.
"""
with self._discovery_lock:
if hasattr(self, 'plugin_directories') and plugin_id in self.plugin_directories:
if plugin_id in self.plugin_directories:
return str(self.plugin_directories[plugin_id])
plugin_id = safe_path_component(plugin_id)
@@ -967,7 +954,7 @@ class PluginManager:
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = failure_time
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info, error=err)
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, err)
@@ -1299,97 +1286,3 @@ class PluginManager:
done = sorted(self._completed_updates)
self._completed_updates.clear()
return done
def update_all_plugins(self) -> None:
"""
Update all enabled plugins.
Calls update() on each enabled plugin using PluginExecutor.
"""
for plugin_id, plugin_instance in list(self.plugins.items()):
if not getattr(plugin_instance, "enabled", True):
continue
if not hasattr(plugin_instance, "update"):
continue
# Eligibility check and the RUNNING transition together, so a
# concurrent scheduler cannot claim the same plugin (see
# _reserve_for_update).
if not self._reserve_for_update(plugin_id):
continue
try:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
with self._plugin_last_update_lock:
self.plugin_last_update[plugin_id] = time.time()
self._note_update_completed(plugin_id)
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else:
self._record_update_failure(plugin_id)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
self._record_update_failure(plugin_id, exc=exc)
def get_plugin_health_metrics(self) -> Dict[str, Any]:
"""
Get health metrics for all plugins.
Returns:
Dictionary mapping plugin_id to health metrics
"""
metrics = {}
for plugin_id in self.plugins.keys():
plugin_metrics = {}
# Get state information
state_info = self.state_manager.get_state_info(plugin_id)
plugin_metrics.update(state_info)
# Get health tracker metrics if available
if self.health_tracker:
health_info = self.health_tracker.get_health_summary(plugin_id)
plugin_metrics['health'] = health_info
else:
plugin_metrics['health'] = {'status': 'unknown'}
metrics[plugin_id] = plugin_metrics
return metrics
def get_plugin_resource_metrics(self) -> Dict[str, Any]:
"""
Get resource usage metrics for all plugins.
Returns:
Dictionary mapping plugin_id to resource metrics
"""
metrics = {}
for plugin_id in self.plugins.keys():
plugin_metrics = {}
# Get state information
state_info = self.state_manager.get_state_info(plugin_id)
plugin_metrics.update(state_info)
# Get resource monitor metrics if available
if self.resource_monitor:
resource_info = self.resource_monitor.get_metrics_summary(plugin_id)
plugin_metrics['resources'] = resource_info
else:
plugin_metrics['resources'] = {'status': 'unknown'}
metrics[plugin_id] = plugin_metrics
return metrics
def get_plugin_state(self, plugin_id: str) -> Dict[str, Any]:
"""
Get comprehensive state information for a plugin.
Args:
plugin_id: Plugin identifier
Returns:
Dictionary with state information
"""
return self.state_manager.get_state_info(plugin_id)
src/plugin_system/plugin_state.py
+11 -121
View File
@@ -6,40 +6,14 @@ with state transitions and queries.
"""
import threading
import time
from collections import deque
from enum import Enum
from typing import Optional, Dict, Any, Deque, List, Tuple
from typing import Optional, Dict, Any
from datetime import datetime
import logging
from src.logging_config import get_logger
# The history is diagnostic only -- nothing reads the entries themselves, just
# their count -- but it is appended to on the hot scheduling path: every update
# cycle records RUNNING on reserve and ENABLED on finish. Unbounded, that is
# 2,880 entries per plugin per day at the default 60s interval, which on a 1 GB
# Pi exhausts memory in weeks.
#
# Two limits, because a single entry count answers the wrong question. What a
# reader wants is "the last couple of hours", and how many transitions that is
# depends entirely on the plugin's update interval -- which on a real board
# spans 2s to 3600s. A flat 200 entries is 4.2 days for the slowest plugin and
# 3.3 minutes for the fastest, so the plugin churning hardest, the one worth
# looking at, keeps the least history.
#
# So: trim by AGE first, which makes the retained window comparable across
# plugins whatever their cadence...
STATE_HISTORY_MAX_AGE_SECONDS = 2 * 60 * 60
# ...and cap by COUNT second, purely as a memory ceiling for the fast pollers
# whose age window would otherwise run to thousands of entries. At ~230 bytes
# an entry this is ~0.5 MB per plugin worst case, and only plugins updating
# faster than roughly every 4s can reach it.
MAX_STATE_HISTORY_PER_PLUGIN = 2000
class PluginState(Enum):
"""Plugin state enumeration."""
UNLOADED = "unloaded" # Plugin not loaded
@@ -63,39 +37,14 @@ class PluginStateManager:
self.logger = logger or get_logger(__name__)
self._lock = threading.RLock()
self._states: Dict[str, PluginState] = {}
# (monotonic timestamp, transition). The clock is monotonic so a DST
# shift or an NTP step cannot make entries look old and flush the
# history; the human-readable timestamp lives inside the transition.
self._state_history: Dict[str, Deque[Tuple[float, Dict[str, Any]]]] = {}
# Lifetime transition totals, kept separately so the count reported by
# get_state_info() stays truthful once the history above starts rolling.
# Lifetime transition totals, reported by get_state_info().
self._state_transition_counts: Dict[str, int] = {}
self._error_info: Dict[str, Dict[str, Any]] = {}
self._last_update: Dict[str, datetime] = {}
self._last_display: Dict[str, datetime] = {}
def _record_transition(
self,
plugin_id: str,
transition: Dict[str, Any]
) -> None:
"""Append a transition to the plugin's bounded history.
Callers must already hold ``_lock``. The deque discards its oldest
entry once it is full, so the history cannot grow without bound; the
lifetime total is tracked separately for get_state_info().
"""
history = self._state_history.get(plugin_id)
if history is None:
history = deque(maxlen=MAX_STATE_HISTORY_PER_PLUGIN)
self._state_history[plugin_id] = history
now = time.monotonic()
history.append((now, transition))
# Age out first; the deque's maxlen is the backstop for plugins that
# produce more than the ceiling within the window.
cutoff = now - STATE_HISTORY_MAX_AGE_SECONDS
while history and history[0][0] < cutoff:
history.popleft()
def _record_transition(self, plugin_id: str) -> None:
"""Count a state transition. Callers must already hold ``_lock``."""
self._state_transition_counts[plugin_id] = (
self._state_transition_counts.get(plugin_id, 0) + 1
)
@@ -117,14 +66,7 @@ class PluginStateManager:
with self._lock:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
transition = {
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None
}
self._record_transition(plugin_id, transition)
self._record_transition(plugin_id)
# Store error info if transitioning to ERROR state
if state == PluginState.ERROR and error:
@@ -181,56 +123,16 @@ class PluginStateManager:
state = self.get_state(plugin_id)
return state == PluginState.ENABLED
def get_state_history(self, plugin_id: str) -> List[Dict[str, Any]]:
"""
Get state transition history for a plugin.
Retention is by age first -- transitions older than
STATE_HISTORY_MAX_AGE_SECONDS are dropped -- and by count second, at
MAX_STATE_HISTORY_PER_PLUGIN, which only binds for plugins updating
fast enough to exceed it inside that window.
Args:
plugin_id: Plugin identifier
Returns:
List of recent state transitions, oldest first. Both the list and
the transition dicts are copies, so callers cannot mutate the
manager's own history. The values inside a transition are all
immutable, so a shallow copy per entry is enough.
"""
with self._lock:
return [
dict(transition)
for _stamp, transition in self._state_history.get(plugin_id, ())
]
def set_error_info(self, plugin_id: str, error_info: Dict[str, Any]) -> None:
"""
Persist structured error context without changing plugin state.
Used for recoverable failures (e.g. update timeout) where the plugin
stays ENABLED but the error details should remain queryable.
Args:
plugin_id: Plugin identifier
error_info: Arbitrary dict describing the error
"""
with self._lock:
self._error_info[plugin_id] = dict(error_info)
def set_state_with_error(
self,
plugin_id: str,
state: PluginState,
error_info: Dict[str, Any],
error: Optional[Exception] = None,
) -> None:
"""Set plugin state and persist error context atomically.
Unlike calling set_state() then set_error_info() separately, this
method holds ``_lock`` for both writes so no reader can observe the
new state without the accompanying error context.
Holds ``_lock`` for both writes so no reader can observe the new
state without the accompanying error context.
Intentionally does not clear ``_error_info`` the way set_state() does
for non-ERROR transitions — this is the recoverable-failure path where
@@ -240,19 +142,11 @@ class PluginStateManager:
plugin_id: Plugin identifier
state: New state
error_info: Structured error dict to persist alongside the state
error: Optional exception recorded in the transition history
"""
with self._lock:
old_state = self._states.get(plugin_id, PluginState.UNLOADED)
self._states[plugin_id] = state
self._record_transition(plugin_id, {
'timestamp': datetime.now(),
'from': old_state.value,
'to': state.value,
'error': str(error) if error else None,
})
self._record_transition(plugin_id)
self._error_info[plugin_id] = dict(error_info)
self.logger.debug(
@@ -284,10 +178,6 @@ class PluginStateManager:
"""Record that plugin update() was called."""
self._last_update[plugin_id] = datetime.now()
def record_display(self, plugin_id: str) -> None:
"""Record that plugin display() was called."""
self._last_display[plugin_id] = datetime.now()
def get_last_update(self, plugin_id: str) -> Optional[datetime]:
"""Get timestamp of last update() call."""
return self._last_update.get(plugin_id)
@@ -331,13 +221,13 @@ class PluginStateManager:
def clear_state(self, plugin_id: str) -> None:
"""Clear all state information for a plugin.
Held under ``_lock`` so the five dicts are dropped as one unit: every
Held under ``_lock`` so the dicts are dropped as one unit: every
other mutator takes the lock, and without it a concurrent set_state()
could interleave and leave a plugin with history but no state.
could interleave and leave a plugin with a transition count but no
state.
"""
with self._lock:
self._states.pop(plugin_id, None)
self._state_history.pop(plugin_id, None)
self._state_transition_counts.pop(plugin_id, None)
self._error_info.pop(plugin_id, None)
self._last_update.pop(plugin_id, None)
src/plugin_system/resource_monitor.py
-3
View File
@@ -94,9 +94,6 @@ class PluginResourceMonitor:
# they are rate-limited instead. See _METRICS_PERSIST_INTERVAL.
self._metrics_persisted_at: Dict[str, float] = {}
# Thread-local storage for execution tracking
self._local = threading.local()
# Lock for thread-safe access
self._lock = threading.Lock()
src/plugin_system/state_manager.py
+3 -104
View File
@@ -2,12 +2,12 @@
Centralized plugin state management.
Provides a single source of truth for plugin state (installed, enabled, version, etc.)
with state change events and persistence.
with persistence.
"""
import json
import threading
from typing import Dict, Any, Optional, List, Callable
from typing import Dict, Any, Optional
from pathlib import Path
from datetime import datetime
from dataclasses import dataclass, asdict
@@ -75,9 +75,7 @@ class PluginStateManager:
Provides:
- Single source of truth for plugin state
- State change events/notifications
- State persistence
- State versioning
"""
def __init__(
@@ -104,9 +102,6 @@ class PluginStateManager:
self._states: Dict[str, PluginState] = {}
self._state_version = 1
# State change callbacks
self._callbacks: Dict[str, List[Callable[[str, PluginState, PluginState], None]]] = {}
# Threading
self._lock = threading.RLock()
@@ -149,8 +144,7 @@ class PluginStateManager:
def update_plugin_state(
self,
plugin_id: str,
updates: Dict[str, Any],
notify: bool = True
updates: Dict[str, Any]
) -> bool:
"""
Update plugin state.
@@ -158,7 +152,6 @@ class PluginStateManager:
Args:
plugin_id: Plugin identifier
updates: Dictionary of state updates
notify: Whether to notify callbacks of changes
Returns:
True if update successful
@@ -174,18 +167,6 @@ class PluginStateManager:
enabled=False
)
# Create new state with updates
old_state = PluginState(
plugin_id=current_state.plugin_id,
status=current_state.status,
enabled=current_state.enabled,
version=current_state.version,
installed_at=current_state.installed_at,
last_updated=current_state.last_updated,
config_version=current_state.config_version,
metadata=current_state.metadata.copy() if current_state.metadata else {}
)
# Apply updates
if 'status' in updates:
if isinstance(updates['status'], str):
@@ -218,10 +199,6 @@ class PluginStateManager:
# Store updated state
self._states[plugin_id] = current_state
# Notify callbacks
if notify:
self._notify_callbacks(plugin_id, old_state, current_state)
# Auto-save if enabled
if self.auto_save:
self._save_state()
@@ -274,23 +251,6 @@ class PluginStateManager:
}
)
def set_plugin_error(self, plugin_id: str, error: Optional[str] = None) -> bool:
"""
Mark plugin as having an error.
Args:
plugin_id: Plugin identifier
error: Optional error message
Returns:
True if update successful
"""
updates = {'status': PluginStateStatus.ERROR}
if error:
updates['metadata'] = {'last_error': error}
return self.update_plugin_state(plugin_id, updates)
def remove_plugin_state(self, plugin_id: str) -> bool:
"""
Remove plugin state (e.g., after uninstall).
@@ -304,12 +264,8 @@ class PluginStateManager:
self._ensure_loaded()
with self._lock:
if plugin_id in self._states:
old_state = self._states[plugin_id]
del self._states[plugin_id]
# Notify callbacks
self._notify_callbacks(plugin_id, old_state, None)
# Auto-save if enabled
if self.auto_save:
self._save_state()
@@ -318,58 +274,6 @@ class PluginStateManager:
return False
def subscribe_to_state_changes(
self,
callback: Callable[[str, PluginState, Optional[PluginState]], None],
plugin_id: Optional[str] = None
) -> str:
"""
Subscribe to state changes.
Args:
callback: Callback function (plugin_id, old_state, new_state)
plugin_id: Optional plugin ID to filter on (None = all plugins)
Returns:
Subscription ID
"""
import uuid
subscription_id = str(uuid.uuid4())
with self._lock:
key = plugin_id or '*'
if key not in self._callbacks:
self._callbacks[key] = []
self._callbacks[key].append(callback)
return subscription_id
def _notify_callbacks(
self,
plugin_id: str,
old_state: PluginState,
new_state: Optional[PluginState]
) -> None:
"""Notify all relevant callbacks of state change."""
# Get callbacks for this plugin and all plugins
callbacks_to_notify = []
if plugin_id in self._callbacks:
callbacks_to_notify.extend(self._callbacks[plugin_id])
if '*' in self._callbacks:
callbacks_to_notify.extend(self._callbacks['*'])
# Call each callback
for callback in callbacks_to_notify:
try:
callback(plugin_id, old_state, new_state)
except Exception as e:
self.logger.error(
f"Error in state change callback: {e}",
exc_info=True
)
def _save_state(self) -> None:
"""Save state to file."""
if not self.state_file:
@@ -430,8 +334,3 @@ class PluginStateManager:
except Exception as e:
self.logger.error(f"Error loading plugin state: {e}", exc_info=True)
def get_state_version(self) -> int:
"""Get current state version (for detecting corruption)."""
return self._state_version
src/plugin_system/state_reconciliation.py
+2 -12
View File
@@ -425,18 +425,9 @@ class StateReconciliation:
# error. The entry is still surfaced as MANUAL_FIX_REQUIRED so the
# UI can show it, but no auto-repair will run.
previously_unrecoverable = plugin_id in self._unrecoverable_missing_on_disk
# Also refuse to re-install a plugin that the user just uninstalled
# through the UI — prevents a race where the reconciler fires
# between file removal and config cleanup and resurrects the
# plugin the user just deleted.
recently_uninstalled = (
self.store_manager is not None
and hasattr(self.store_manager, 'was_recently_uninstalled')
and self.store_manager.was_recently_uninstalled(plugin_id)
)
# Also refuse to resurrect a plugin the user has persistently
# uninstalled. Unlike the in-memory race guard above, this record
# survives restarts, so the user's removal sticks across updates.
# uninstalled. The record survives restarts, so the user's
# removal sticks across updates.
persistently_uninstalled = (
self.store_manager is not None
and hasattr(self.store_manager, 'is_plugin_uninstalled')
@@ -445,7 +436,6 @@ class StateReconciliation:
can_repair = (
self.store_manager is not None
and not previously_unrecoverable
and not recently_uninstalled
and not persistently_uninstalled
)
inconsistencies.append(Inconsistency(
src/plugin_system/store_manager.py
+1 -46
View File
@@ -91,15 +91,7 @@ class PluginStoreManager:
self._token_validation_cache = {} # Cache for token validation results: {token: (is_valid, timestamp, error_message)}
self._token_validation_cache_timeout = 300 # 5 minutes cache for token validation
# Per-plugin tombstone timestamps for plugins that were uninstalled
# recently via the UI. Used by the state reconciler to avoid
# resurrecting a plugin the user just deleted when reconciliation
# races against the uninstall operation. Cleared after ``_uninstall_tombstone_ttl``.
self._uninstall_tombstones: Dict[str, float] = {}
self._uninstall_tombstone_ttl = 300 # 5 minutes
# Persistent record of plugins the user has uninstalled. Unlike the
# in-memory tombstones above (a short-lived race guard), this survives
# Persistent record of plugins the user has uninstalled. It survives
# restarts so that a core ``git pull`` update cannot resurrect a
# built-in plugin the user removed. Built-in plugins (e.g.
# ``web-ui-info``, ``starlark-apps``) are committed into the repo under
@@ -189,21 +181,6 @@ class PluginStoreManager:
synthetic_ts = time.time() + self._failure_backoff_seconds - cache_timeout
cache_dict[cache_key] = (synthetic_ts, payload)
def mark_recently_uninstalled(self, plugin_id: str) -> None:
"""Record that ``plugin_id`` was just uninstalled by the user."""
self._uninstall_tombstones[plugin_id] = time.time()
def was_recently_uninstalled(self, plugin_id: str) -> bool:
"""Return True if ``plugin_id`` has an active uninstall tombstone."""
ts = self._uninstall_tombstones.get(plugin_id)
if ts is None:
return False
if time.time() - ts > self._uninstall_tombstone_ttl:
# Expired — clean up so the dict doesn't grow unbounded.
self._uninstall_tombstones.pop(plugin_id, None)
return False
return True
def _is_valid_plugin_id(self, plugin_id: Any) -> bool:
"""Return True if ``plugin_id`` is a safe single-component plugin id.
@@ -3269,25 +3246,3 @@ class PluginStoreManager:
installed.append(item.name)
return installed
def get_installed_plugin_info(self, plugin_id: str) -> Optional[Dict]:
"""
Get manifest information for an installed plugin.
Args:
plugin_id: Plugin identifier
Returns:
Manifest data or None if not found
"""
manifest_path = self.plugins_dir / plugin_id / "manifest.json"
if not manifest_path.exists():
return None
try:
with open(manifest_path, 'r') as f:
return json.load(f)
except Exception as e:
self.logger.error(f"Error reading manifest for {plugin_id}: {e}")
return None