Files
LEDMatrix/src/plugin_system/plugin_manager.py
Chuck eedf680a8c perf: display pipeline optimizations — caching, logging, scroll, text width (#358)
* docs(core): add module and class docstrings to the 5 undocumented core files

Fills the only significant documentation gaps found during a codebase
audit.  All other core files (plugin_system/, logging_config.py, etc.)
already have complete module, class, and function docstrings.

Files changed (documentation only — zero logic changes):

  display_controller.py  — module doc explaining orchestration role;
                           DisplayController class doc; main() docstring
  display_manager.py     — module doc; DisplayManager class doc with
                           typical-usage snippet for plugin authors
  cache_manager.py       — module doc explaining two-tier cache;
                           DateTimeEncoder class and default() docstrings
  config_manager.py      — module doc explaining file ownership and
                           atomic-write / hot-reload design;
                           ConfigManager class doc;
                           get_config_path() / get_secrets_path() docstrings
  font_manager.py        — module doc (class docstring already existed)

Also noted (but not changed to avoid behaviour risk):
  display_manager.py and font_manager.py use logging.getLogger() directly
  instead of the project's get_logger() wrapper.  display_manager.py also
  calls setLevel(logging.INFO) immediately after, which would be lost if
  switched to get_logger().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(display_controller): three targeted hot-path optimizations

Opt 1 — cache inspect.signature() per plugin_id
  inspect.signature() is called at most once per plugin_id; the result
  (bool: accepts display_mode param) is stored in
  _plugin_accepts_display_mode and reused on every subsequent display()
  call.  Eliminates all reflection from the display path at runtime.
  Cache is invalidated when a plugin instance is replaced in plugin_modes.

Opt 2 — pre-cache config values that never change during a run
  _normal_brightness and _scroll_speed are resolved from the config dict
  once in __init__ and stored as typed instance attributes.
  - Removes 2+ chained dict.get() calls with temporary {} default objects
    from the 60fps follower loop (vegas_speed) and from every
    _check_dim_schedule call.
  - current_brightness init now uses _normal_brightness directly.

Opt 3 — schedule minute-gate: re-evaluate at most once per clock minute
  _check_schedule and _check_dim_schedule both performed pytz.timezone(),
  datetime.now(), strftime(), and datetime.strptime() on every outer loop
  call.  Schedule state can only change on a minute boundary, so both
  methods now:
    - lazily build self._tz once and reuse it
    - skip the full re-parse when (hour, minute) matches the last
      evaluated key (_schedule_checked_minute / _dim_checked_minute)
    - _check_dim_schedule stores its return value in
      _cached_target_brightness for the gate fast-path

Tests: 23 new tests in test_display_controller_optimizations.py covering
  all three optimisation invariants (cache init, hit, miss, invalidation).
  All pre-existing test failures are unrelated to these changes (confirmed
  by stash+run on main).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: resolve 22 pre-existing test failures across 6 groups

Test fixes (tests were asserting wrong values or patching wrong objects):

  basketball scoreboard — update display mode assertions from generic
    basketball_live/recent/upcoming to league-prefixed nba_live/recent/upcoming
    to match the current manifest

  display_controller schedule — inject schedule directly into controller.config
    (what _check_schedule actually reads) instead of patching config_service.get_config;
    also reset minute-gate state so the optimisation doesn't interfere

  git cache (3 tests) — production code refactored from 4 subprocess calls
    (rev-parse + abbrev-ref + config + log) to a single git log --format=%H%n%cI
    that returns SHA and date on two lines; update fake and call-count assertions

  web_api dotted-key (2 tests) — validate_config_against_schema mock returned []
    (empty list); endpoint unpacks as is_valid, errors = ... causing ValueError;
    fix: return_value = (True, [])

  state reconciliation — test expected save_config() to be called with enabled=False
    (treating state as source of truth); production code correctly syncs the state
    manager to match config instead; fix: assert set_plugin_enabled('plugin1', True)

Production fixes (production code had bugs or missing features):

  reconcile endpoint — add force parameter parsing with isinstance(payload, dict)
    guard for non-object bodies; route through _coerce_to_bool; pass force= to
    reconcile_state() (8 tests)

  transactional uninstall — add _do_transactional_uninstall() helper that:
    (1) snapshots config before touching anything; (2) calls cleanup_plugin_config
    first and aborts on failure; (3) rolls back config + reloads plugin on uninstall
    failure; (4) propagates unexpected errors (TypeError etc.) instead of swallowing
    them (6 tests)

  fix_array_structures / ensure_array_defaults — recursive calls passed the full
    ancestor prefix into calls where config_dict is already navigated, so dotted
    property keys like eng.1 caused parent_parts.split('.') to mis-navigate; fix:
    drop prefix on recursive calls; also add _fix_none_arrays pass after
    merge_with_defaults so None arrays in JSON requests are replaced with schema
    defaults (2 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf: four targeted optimizations across the display pipeline

Opt 1 — cache data-fetch interval per plugin (plugin_manager.py)
  _get_plugin_update_interval fell back to config_manager.get_config()
  (a full dict copy) when the manifest lacked an interval.  Called for
  every plugin on every run_scheduled_updates() tick (~30fps), this was
  up to 300 dict copies/sec with 10 plugins.
  Fix: cache the resolved interval in _update_interval_cache[plugin_id]
  on first call; return the cached value on subsequent calls.  Cache is
  cleared on load_plugin and unload_plugin.

Opt 2 — demote noisy per-cycle INFO logs to DEBUG (display_controller.py)
  Four logger.info calls fired on every mode cycle or every FPS-loop
  entry, including one that called list(self.plugin_modes.keys())
  unconditionally (allocating a list every outer loop iteration).
  - "Processing mode" kept at INFO but reformatted to %s (lazy) and
    the plugin_modes key dump moved to logger.debug
  - "Attempting/Got cycle duration" → logger.debug
  - "Entering high/normal FPS loop" → logger.debug
  Mode name at INFO is preserved for black-screen troubleshooting.

Opt 3 — use Image.frombytes instead of Image.fromarray in scroll hot path
  (scroll_helper.py)
  Image.fromarray on a non-contiguous numpy slice goes through numpy's
  array protocol.  Image.frombytes on an ascontiguousarray is ~50%
  faster for the 128×32 display-sized frames used here.  Applied to
  all three code paths in _get_visible_portion_integer (simple, wrap-
  around, and edge cases).

Opt 5 — cache get_text_width per (text, font) pair (display_manager.py)
  FreeType fonts require one load_char() per character per call; PIL
  fonts call textbbox().  Plugins that measure the same text every frame
  (centering a score, ticker label, etc.) were re-measuring from scratch
  on every display() call.
  Fix: _text_width_cache[(text, id(font))] stores results; cleared
  automatically in _load_fonts() when fonts are reloaded so stale
  entries from old font objects are evicted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(scroll_helper): fix edge-case bug exposed by frombytes switch

The previous commit replaced Image.fromarray with Image.frombytes in
_get_visible_portion_integer.  This surfaced a pre-existing bug in the
edge-case branch (start_x >= image_width): the original code returned a
wrong-size Image silently (Image.fromarray accepts a too-short array);
Image.frombytes raises ValueError instead.

Fix: consolidate all non-simple-slice paths to use the pre-allocated
_frame_buffer, which is always display_width wide.  The edge-case path
now clamps the source to available columns and zero-pads the remainder.

Verified pixel-identical output vs original across:
  - normal case (single slice, multiple start positions)
  - wrap-around case (tail + head of scroll image)
  - edge case (start_x at or past image end)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: address CodeRabbit review comments on PR #358

1. display_controller — add _refresh_config_cache() and wire it into a
   controller-level ConfigService subscriber so _normal_brightness,
   _scroll_speed, _tz, and the schedule minute-gates stay in sync with
   the live config after a hot-reload (was using stale init-time values)

2. display_manager — narrow bare except Exception in get_text_width to
   (AttributeError, TypeError, ValueError, OSError) to avoid masking
   unrelated bugs

3. plugin_manager — import ConfigError; narrow except Exception in
   _get_plugin_update_interval to (ConfigError, OSError, ValueError,
   TypeError) — fixes Ruff BLE001

4. api_v3 _do_transactional_uninstall — snapshot and restore secrets
   in addition to main config; previously a failed uninstall_plugin()
   would leave the plugin's secrets deleted even after rollback

5. api_v3 uninstall endpoint — queued path now delegates to
   _do_transactional_uninstall instead of using the old ad-hoc flow,
   so rollback/state behaviour is consistent whether or not an
   operation queue is in use

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(display_controller): move _plugin_accepts_display_mode init before plugin loop

Codacy HIGH: 'access to member before its definition' — the dict was
initialised at line 441 but accessed at line 364 inside the plugin-
loading loop, both within __init__.

Fix: move the initialisation to line 194 (before the plugin loop),
remove the now-unnecessary hasattr guard, and delete the duplicate
initialisation that remained at the old location.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Chuck <chuck@example.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 11:58:21 -04:00

883 lines
35 KiB
Python

"""
Plugin Manager
Manages plugin discovery, loading, and lifecycle for the LEDMatrix system.
Handles dynamic plugin loading from the plugins/ directory.
API Version: 1.0.0
"""
import json
import sys
import subprocess
import time
import threading
from pathlib import Path
from typing import Dict, List, Optional, Any
import logging
from src.exceptions import PluginError, ConfigError
from src.logging_config import get_logger
from src.plugin_system.plugin_loader import PluginLoader
from src.plugin_system.plugin_executor import PluginExecutor
from src.plugin_system.plugin_state import PluginStateManager, PluginState
from src.plugin_system.schema_manager import SchemaManager
from src.common.permission_utils import (
ensure_directory_permissions,
get_plugin_dir_mode
)
class PluginManager:
"""
Manages plugin discovery, loading, and lifecycle.
The PluginManager is responsible for:
- Discovering plugins in the plugins/ directory
- Loading plugin modules and instantiating plugin classes
- Managing plugin lifecycle (load, unload, reload)
- Providing access to loaded plugins
- Maintaining plugin manifests
Uses composition with specialized components:
- PluginLoader: Handles module loading and dependency installation
- PluginExecutor: Handles plugin execution with timeout and error isolation
- PluginStateManager: Manages plugin state machine
"""
def __init__(self, plugins_dir: str = "plugins",
config_manager: Optional[Any] = None,
display_manager: Optional[Any] = None,
cache_manager: Optional[Any] = None,
font_manager: Optional[Any] = None) -> None:
"""
Initialize the Plugin Manager.
Args:
plugins_dir: Path to the plugins directory
config_manager: Configuration manager instance
display_manager: Display manager instance
cache_manager: Cache manager instance
font_manager: Font manager instance
"""
self.plugins_dir: Path = Path(plugins_dir)
self.config_manager: Optional[Any] = config_manager
self.display_manager: Optional[Any] = display_manager
self.cache_manager: Optional[Any] = cache_manager
self.font_manager: Optional[Any] = font_manager
self.logger: logging.Logger = get_logger(__name__)
# Initialize plugin system components
self.plugin_loader = PluginLoader(logger=self.logger)
self.plugin_executor = PluginExecutor(default_timeout=30.0, logger=self.logger)
self.state_manager = PluginStateManager(logger=self.logger)
self.schema_manager = SchemaManager(plugins_dir=self.plugins_dir, logger=self.logger)
# Lock protecting plugin_manifests and plugin_directories from
# concurrent mutation (background reconciliation) and reads (requests).
self._discovery_lock = threading.RLock()
# Active plugins
self.plugins: Dict[str, Any] = {}
self.plugin_manifests: Dict[str, Dict[str, Any]] = {}
self.plugin_modules: Dict[str, Any] = {}
self.plugin_last_update: Dict[str, float] = {}
# Cached data-fetch intervals per plugin_id.
# _get_plugin_update_interval falls back to config_manager.get_config()
# (a full dict copy) when the manifest lacks an interval — caching avoids
# that copy on every 30-fps tick. Cleared on load/unload.
self._update_interval_cache: Dict[str, Optional[float]] = {}
# Health tracking (optional, set by display_controller if available)
self.health_tracker = None
self.resource_monitor = None
# Ensure plugins directory exists with proper permissions
try:
ensure_directory_permissions(self.plugins_dir, get_plugin_dir_mode())
except (OSError, PermissionError) as e:
self.logger.error("Could not create plugins directory %s: %s", self.plugins_dir, e, exc_info=True)
raise PluginError(f"Could not create plugins directory: {self.plugins_dir}", context={'error': str(e)}) from e
def _scan_directory_for_plugins(self, directory: Path) -> List[str]:
"""
Scan a directory for plugins.
Args:
directory: Directory to scan
Returns:
List of plugin IDs found
"""
plugin_ids = []
if not directory.exists():
return plugin_ids
# Build new state locally before acquiring lock
new_manifests: Dict[str, Dict[str, Any]] = {}
new_directories: Dict[str, Path] = {}
try:
for item in directory.iterdir():
if not item.is_dir():
continue
# Skip backup directories so they don't overwrite live entries
if '.standalone-backup-' in item.name:
continue
manifest_path = item / "manifest.json"
if manifest_path.exists():
try:
with open(manifest_path, 'r', encoding='utf-8') as f:
manifest = json.load(f)
plugin_id = manifest.get('id')
if plugin_id:
plugin_ids.append(plugin_id)
new_manifests[plugin_id] = manifest
new_directories[plugin_id] = item
except (json.JSONDecodeError, PermissionError, OSError) as e:
self.logger.warning("Error reading manifest from %s: %s", manifest_path, e, exc_info=True)
continue
except (OSError, PermissionError) as e:
self.logger.error("Error scanning directory %s: %s", directory, e, exc_info=True)
# Replace shared state under lock so uninstalled plugins don't linger
with self._discovery_lock:
self.plugin_manifests.clear()
self.plugin_manifests.update(new_manifests)
if not hasattr(self, 'plugin_directories'):
self.plugin_directories = {}
else:
self.plugin_directories.clear()
self.plugin_directories.update(new_directories)
return plugin_ids
def discover_plugins(self) -> List[str]:
"""
Discover all plugins in the plugins directory.
Also checks for potential config key collisions and logs warnings.
Returns:
List of plugin IDs
"""
self.logger.info("Discovering plugins in %s", self.plugins_dir)
plugin_ids = self._scan_directory_for_plugins(self.plugins_dir)
self.logger.info("Discovered %d plugin(s)", len(plugin_ids))
# Check for config key collisions
collisions = self.schema_manager.detect_config_key_collisions(plugin_ids)
for collision in collisions:
self.logger.warning(
"Config collision detected: %s",
collision.get('message', str(collision))
)
return plugin_ids
def _get_dependency_marker_path(self, plugin_id: str) -> Path:
"""Get path to dependency installation marker file."""
plugin_dir = self.plugins_dir / plugin_id
if not plugin_dir.exists():
# Try with ledmatrix- prefix
plugin_dir = self.plugins_dir / f"ledmatrix-{plugin_id}"
return plugin_dir / ".dependencies_installed"
def _check_dependencies_installed(self, plugin_id: str) -> bool:
"""Check if dependencies are already installed for a plugin."""
marker_path = self._get_dependency_marker_path(plugin_id)
return marker_path.exists()
def _mark_dependencies_installed(self, plugin_id: str) -> None:
"""Mark dependencies as installed for a plugin."""
marker_path = self._get_dependency_marker_path(plugin_id)
try:
marker_path.touch()
# Set proper file permissions after creating marker
from src.common.permission_utils import (
ensure_file_permissions,
get_plugin_file_mode
)
ensure_file_permissions(marker_path, get_plugin_file_mode())
except (OSError, PermissionError) as e:
self.logger.warning("Could not create dependency marker for %s: %s", plugin_id, e)
def _remove_dependency_marker(self, plugin_id: str) -> None:
"""Remove dependency installation marker."""
marker_path = self._get_dependency_marker_path(plugin_id)
try:
if marker_path.exists():
marker_path.unlink()
except (OSError, PermissionError) as e:
self.logger.warning("Could not remove dependency marker for %s: %s", plugin_id, e)
def _install_plugin_dependencies(self, requirements_file: Path) -> bool:
"""
Install plugin dependencies from requirements.txt.
Args:
requirements_file: Path to requirements.txt
Returns:
True if installation succeeded or not needed, False on error
"""
try:
self.logger.info("Installing dependencies from %s", requirements_file)
result = subprocess.run(
[sys.executable, "-m", "pip", "install", "--break-system-packages", "--no-cache-dir", "-r", str(requirements_file)],
capture_output=True,
text=True,
timeout=300,
check=False
)
if result.returncode == 0:
self.logger.info("Dependencies installed successfully")
return True
else:
self.logger.warning("Dependency installation returned non-zero exit code: %s", result.stderr)
return False
except subprocess.TimeoutExpired:
self.logger.error("Dependency installation timed out")
return False
except FileNotFoundError as e:
self.logger.warning("Command not found: %s. Skipping dependency installation", e)
return True
except (BrokenPipeError, OSError) as e:
# Handle broken pipe errors (errno 32) which can occur during pip downloads
# Often caused by network interruptions or output buffer issues
if isinstance(e, OSError) and e.errno == 32:
self.logger.error(
"Broken pipe error during dependency installation. "
"This usually indicates a network interruption or pip output buffer issue. "
"Try installing again or check your network connection."
)
else:
self.logger.error("OS error during dependency installation: %s", e)
return False
except Exception as e:
self.logger.error("Unexpected error installing dependencies: %s", e, exc_info=True)
return True
def load_plugin(self, plugin_id: str) -> bool:
"""
Load a plugin by ID.
This method:
1. Checks if plugin is already loaded
2. Validates the manifest exists
3. Uses PluginLoader to import module and instantiate plugin
4. Validates the plugin configuration
5. Stores the plugin instance
6. Updates plugin state
Args:
plugin_id: Plugin identifier
Returns:
True if loaded successfully, False otherwise
"""
if plugin_id in self.plugins:
self.logger.warning("Plugin %s already loaded", plugin_id)
return True
manifest = self.plugin_manifests.get(plugin_id)
if not manifest:
self.logger.error("No manifest found for plugin: %s", plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ERROR)
return False
try:
# Update state to LOADED
self.state_manager.set_state(plugin_id, PluginState.LOADED)
# Find plugin directory using PluginLoader
plugin_directories = getattr(self, 'plugin_directories', None)
plugin_dir = self.plugin_loader.find_plugin_directory(
plugin_id,
self.plugins_dir,
plugin_directories
)
if plugin_dir is None:
self.logger.error("Plugin directory not found: %s", plugin_id)
self.logger.error("Searched in: %s", self.plugins_dir)
self.state_manager.set_state(plugin_id, PluginState.ERROR)
return False
# Update mapping if found via search
if plugin_directories is None or plugin_id not in plugin_directories:
if not hasattr(self, 'plugin_directories'):
self.plugin_directories = {}
self.plugin_directories[plugin_id] = plugin_dir
# Get plugin config
if self.config_manager:
full_config = self.config_manager.load_config()
config = full_config.get(plugin_id, {})
else:
config = {}
# Check if plugin has a config schema
schema_path = self.schema_manager.get_schema_path(plugin_id)
if schema_path is None:
# Schema file doesn't exist
self.logger.warning(
f"Plugin '{plugin_id}' has no config_schema.json - configuration will not be validated. "
f"Consider adding a schema file for better error detection and user experience."
)
else:
# Schema file exists, try to load it
schema = self.schema_manager.load_schema(plugin_id)
if schema is None:
# Schema exists but couldn't be loaded (likely invalid JSON or schema)
self.logger.warning(
f"Plugin '{plugin_id}' has a config_schema.json but it could not be loaded. "
f"The schema may be invalid. Please verify the schema file at: {schema_path}"
)
# Merge config with schema defaults to ensure all defaults are applied
try:
defaults = self.schema_manager.generate_default_config(plugin_id, use_cache=True)
config = self.schema_manager.merge_with_defaults(config, defaults)
self.logger.debug(f"Merged config with schema defaults for {plugin_id}")
except Exception as e:
self.logger.warning(f"Could not apply schema defaults for {plugin_id}: {e}")
# Continue with original config if defaults can't be applied
# Use PluginLoader to load plugin
plugin_instance, module = self.plugin_loader.load_plugin(
plugin_id=plugin_id,
manifest=manifest,
plugin_dir=plugin_dir,
config=config,
display_manager=self.display_manager,
cache_manager=self.cache_manager,
plugin_manager=self,
install_deps=True,
plugins_dir=self.plugins_dir,
)
# Store module
self.plugin_modules[plugin_id] = module
# Register plugin-shipped fonts with the FontManager (if any).
# Plugin manifests can declare a "fonts" block that ships custom
# fonts with the plugin; FontManager.register_plugin_fonts handles
# the actual loading. Wired here so manifest declarations take
# effect without requiring plugin code changes.
font_manifest = manifest.get('fonts')
if font_manifest and self.font_manager is not None and hasattr(
self.font_manager, 'register_plugin_fonts'
):
try:
self.font_manager.register_plugin_fonts(plugin_id, font_manifest)
except Exception as e:
self.logger.warning(
"Failed to register fonts for plugin %s: %s", plugin_id, e
)
# Validate configuration
if hasattr(plugin_instance, 'validate_config'):
try:
if not plugin_instance.validate_config():
self.logger.error("Plugin %s configuration validation failed", plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ERROR)
return False
except Exception as e:
self.logger.error("Error validating plugin %s config: %s", plugin_id, e, exc_info=True)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
return False
# Store plugin instance
self.plugins[plugin_id] = plugin_instance
self.plugin_last_update[plugin_id] = 0.0
# Invalidate cached interval so next tick re-derives it for this plugin
self._update_interval_cache.pop(plugin_id, None)
# Update state based on enabled status
if config.get('enabled', True):
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
# Call on_enable if plugin is enabled
if hasattr(plugin_instance, 'on_enable'):
plugin_instance.on_enable()
else:
self.state_manager.set_state(plugin_id, PluginState.DISABLED)
self.logger.info("Loaded plugin: %s", plugin_id)
return True
except PluginError as e:
self.logger.error("Plugin error loading %s: %s", plugin_id, e, exc_info=True)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
return False
except Exception as e:
self.logger.error("Unexpected error loading plugin %s: %s", plugin_id, e, exc_info=True)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
return False
def unload_plugin(self, plugin_id: str) -> bool:
"""
Unload a plugin by ID.
Args:
plugin_id: Plugin identifier
Returns:
True if unloaded successfully, False otherwise
"""
if plugin_id not in self.plugins:
self.logger.warning("Plugin %s not loaded", plugin_id)
return False
try:
plugin = self.plugins[plugin_id]
# Call cleanup if available
if hasattr(plugin, 'cleanup'):
try:
plugin.cleanup()
except Exception as e:
self.logger.warning("Error during plugin cleanup: %s", e)
# Call on_disable if available
if hasattr(plugin, 'on_disable'):
try:
plugin.on_disable()
except Exception as e:
self.logger.warning("Error during plugin on_disable: %s", e)
# Remove from active plugins
del self.plugins[plugin_id]
self.plugin_last_update.pop(plugin_id, None)
self._update_interval_cache.pop(plugin_id, None)
# Remove main module from sys.modules if present
module_name = f"plugin_{plugin_id.replace('-', '_')}"
sys.modules.pop(module_name, None)
# Delegate sub-module and cached-module cleanup to the loader
self.plugin_loader.unregister_plugin_modules(plugin_id)
# Remove from plugin_modules
self.plugin_modules.pop(plugin_id, None)
# Update state
self.state_manager.set_state(plugin_id, PluginState.UNLOADED)
self.state_manager.clear_state(plugin_id)
self.logger.info("Unloaded plugin: %s", plugin_id)
return True
except Exception as e:
self.logger.error("Error unloading plugin %s: %s", plugin_id, e, exc_info=True)
self.state_manager.set_state(plugin_id, PluginState.ERROR, error=e)
return False
def reload_plugin(self, plugin_id: str) -> bool:
"""
Reload a plugin (unload and load).
Args:
plugin_id: Plugin identifier
Returns:
True if reloaded successfully, False otherwise
"""
self.logger.info("Reloading plugin: %s", plugin_id)
# Unload first
if plugin_id in self.plugins:
if not self.unload_plugin(plugin_id):
return False
# Re-discover to get updated manifest
manifest_path = self.plugins_dir / plugin_id / "manifest.json"
if manifest_path.exists():
try:
with open(manifest_path, 'r', encoding='utf-8') as f:
manifest = json.load(f)
with self._discovery_lock:
self.plugin_manifests[plugin_id] = manifest
except Exception as e:
self.logger.error("Error reading manifest: %s", e, exc_info=True)
return False
return self.load_plugin(plugin_id)
def get_plugin(self, plugin_id: str) -> Optional[Any]:
"""
Get a loaded plugin instance by ID.
Args:
plugin_id: Plugin identifier
Returns:
Plugin instance or None if not loaded
"""
return self.plugins.get(plugin_id)
def get_all_plugins(self) -> Dict[str, Any]:
"""
Get all loaded plugins.
Returns:
Dict of plugin_id: plugin_instance
"""
return self.plugins.copy()
def get_enabled_plugins(self) -> List[str]:
"""
Get list of enabled plugin IDs.
Returns:
List of plugin IDs that are currently enabled
"""
return [pid for pid, plugin in self.plugins.items() if plugin.enabled]
def get_plugin_info(self, plugin_id: str) -> Optional[Dict[str, Any]]:
"""
Get information about a plugin (manifest + runtime info).
Args:
plugin_id: Plugin identifier
Returns:
Dict with plugin information or None if not found
"""
with self._discovery_lock:
manifest = self.plugin_manifests.get(plugin_id)
if not manifest:
return None
info = manifest.copy()
# Add runtime information if plugin is loaded
plugin = self.plugins.get(plugin_id)
if plugin:
info['loaded'] = True
if hasattr(plugin, 'get_info'):
info['runtime_info'] = plugin.get_info()
else:
info['loaded'] = False
# Add state information
info['state'] = self.state_manager.get_state_info(plugin_id)
return info
def get_all_plugin_info(self) -> List[Dict[str, Any]]:
"""
Get information about all plugins.
Returns:
List of plugin info dictionaries
"""
with self._discovery_lock:
pids = list(self.plugin_manifests.keys())
return [info for info in [self.get_plugin_info(pid) for pid in pids] if info]
def get_plugin_directory(self, plugin_id: str) -> Optional[str]:
"""
Get the directory path for a plugin.
Args:
plugin_id: Plugin identifier
Returns:
Directory path as string or None if not found
"""
with self._discovery_lock:
if hasattr(self, 'plugin_directories') and plugin_id in self.plugin_directories:
return str(self.plugin_directories[plugin_id])
plugin_dir = self.plugins_dir / plugin_id
if plugin_dir.exists():
return str(plugin_dir)
plugin_dir = self.plugins_dir / f"ledmatrix-{plugin_id}"
if plugin_dir.exists():
return str(plugin_dir)
return None
def get_plugin_display_modes(self, plugin_id: str) -> List[str]:
"""
Get display modes provided by a plugin.
Args:
plugin_id: Plugin identifier
Returns:
List of display mode names
"""
with self._discovery_lock:
manifest = self.plugin_manifests.get(plugin_id)
if not manifest:
return []
display_modes = manifest.get('display_modes', [])
if isinstance(display_modes, list):
return display_modes
return []
def find_plugin_for_mode(self, mode: str) -> Optional[str]:
"""
Find which plugin provides a given display mode.
Args:
mode: Display mode identifier
Returns:
Plugin identifier or None if not found.
"""
normalized_mode = mode.strip().lower()
with self._discovery_lock:
manifests_snapshot = dict(self.plugin_manifests)
for plugin_id, manifest in manifests_snapshot.items():
display_modes = manifest.get('display_modes')
if isinstance(display_modes, list) and display_modes:
if any(m.lower() == normalized_mode for m in display_modes):
return plugin_id
return None
def _get_plugin_update_interval(self, plugin_id: str, plugin_instance: Any) -> Optional[float]:
"""
Get the data-fetch interval for a plugin (seconds between update() calls).
Result is cached per plugin_id after the first lookup to avoid calling
config_manager.get_config() — which returns a full dict copy — on every
tick of the 30-fps display loop. The cache is invalidated when a plugin
is loaded or unloaded.
"""
if plugin_id in self._update_interval_cache:
return self._update_interval_cache[plugin_id]
interval: Optional[float] = None
# 1. Manifest (immutable after load — preferred source)
manifest = self.plugin_manifests.get(plugin_id, {})
raw = manifest.get('update_interval')
if raw is not None:
try:
interval = float(raw)
except (ValueError, TypeError):
pass
# 2. Plugin config (mutable; only read once and then cached)
if interval is None and self.config_manager:
try:
config = self.config_manager.get_config()
raw = config.get(plugin_id, {}).get('update_interval')
if raw is not None:
try:
interval = float(raw)
except (ValueError, TypeError):
pass
except (ConfigError, OSError, ValueError, TypeError) as e:
self.logger.debug("Could not get update interval from config: %s", e)
# 3. Default
if interval is None:
interval = 60.0
self._update_interval_cache[plugin_id] = interval
return interval
def _record_update_failure(
self,
plugin_id: str,
exc: Optional[Exception] = None,
) -> None:
"""Apply the standard failure-recovery path for a plugin update.
Stamps plugin_last_update with the actual failure time so the full
configured interval elapses before the next retry, then transitions
the plugin back to ENABLED (not ERROR) with structured error context
so automatic recovery happens on the next scheduled cycle.
Args:
plugin_id: Plugin identifier
exc: The exception that caused the failure, if any. When None a
synthetic ExecutionFailure exception is constructed from the
timeout/executor-error path.
"""
failure_time = time.time()
if exc is not None:
err: Exception = exc
error_type = type(exc).__name__
else:
err = Exception(f"Plugin {plugin_id} execution failed (timeout or executor error)")
error_type = 'ExecutionFailure'
error_info = {
'error': str(err),
'error_type': error_type,
'timestamp': failure_time,
'recoverable': True,
}
self.logger.warning("Plugin %s update() failed; will retry after interval", plugin_id)
self.plugin_last_update[plugin_id] = failure_time
self.state_manager.set_state_with_error(plugin_id, PluginState.ENABLED, error_info, error=err)
if self.health_tracker:
self.health_tracker.record_failure(plugin_id, err)
def run_scheduled_updates(self, current_time: Optional[float] = None) -> None:
"""
Trigger plugin updates based on their defined update intervals.
Includes health tracking and circuit breaker logic.
Uses PluginExecutor for safe execution with timeout.
"""
if current_time is None:
current_time = time.time()
for plugin_id, plugin_instance in list(self.plugins.items()):
if not getattr(plugin_instance, "enabled", True):
continue
if not hasattr(plugin_instance, "update"):
continue
# Check circuit breaker before attempting update
if self.health_tracker and self.health_tracker.should_skip_plugin(plugin_id):
continue
# Check if plugin can execute
if not self.state_manager.can_execute(plugin_id):
continue
interval = self._get_plugin_update_interval(plugin_id, plugin_instance)
if interval is None:
continue
last_update = self.plugin_last_update.get(plugin_id, 0.0)
if last_update == 0.0 or (current_time - last_update) >= interval:
# Update state to RUNNING
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
try:
# Use PluginExecutor for safe execution
success = False
if self.resource_monitor:
# If resource monitor exists, wrap the call
def monitored_update():
self.resource_monitor.monitor_call(plugin_id, plugin_instance.update)
success = self.plugin_executor.execute_update(
type('obj', (object,), {'update': monitored_update})(),
plugin_id
)
else:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
self.plugin_last_update[plugin_id] = current_time
self.state_manager.record_update(plugin_id)
# Update state back to ENABLED
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
# Record success
if self.health_tracker:
self.health_tracker.record_success(plugin_id)
else:
self._record_update_failure(plugin_id)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
self._record_update_failure(plugin_id, exc=exc)
def update_all_plugins(self) -> None:
"""
Update all enabled plugins.
Calls update() on each enabled plugin using PluginExecutor.
"""
for plugin_id, plugin_instance in list(self.plugins.items()):
if not getattr(plugin_instance, "enabled", True):
continue
if not hasattr(plugin_instance, "update"):
continue
# Check if plugin can execute
if not self.state_manager.can_execute(plugin_id):
continue
# Update state to RUNNING
self.state_manager.set_state(plugin_id, PluginState.RUNNING)
try:
success = self.plugin_executor.execute_update(plugin_instance, plugin_id)
if success:
self.plugin_last_update[plugin_id] = time.time()
self.state_manager.record_update(plugin_id)
self.state_manager.set_state(plugin_id, PluginState.ENABLED)
else:
self._record_update_failure(plugin_id)
except Exception as exc: # pylint: disable=broad-except
self.logger.exception("Error updating plugin %s: %s", plugin_id, exc)
self._record_update_failure(plugin_id, exc=exc)
def get_plugin_health_metrics(self) -> Dict[str, Any]:
"""
Get health metrics for all plugins.
Returns:
Dictionary mapping plugin_id to health metrics
"""
metrics = {}
for plugin_id in self.plugins.keys():
plugin_metrics = {}
# Get state information
state_info = self.state_manager.get_state_info(plugin_id)
plugin_metrics.update(state_info)
# Get health tracker metrics if available
if self.health_tracker:
health_info = self.health_tracker.get_plugin_health(plugin_id)
plugin_metrics['health'] = health_info
else:
plugin_metrics['health'] = {'status': 'unknown'}
metrics[plugin_id] = plugin_metrics
return metrics
def get_plugin_resource_metrics(self) -> Dict[str, Any]:
"""
Get resource usage metrics for all plugins.
Returns:
Dictionary mapping plugin_id to resource metrics
"""
metrics = {}
for plugin_id in self.plugins.keys():
plugin_metrics = {}
# Get state information
state_info = self.state_manager.get_state_info(plugin_id)
plugin_metrics.update(state_info)
# Get resource monitor metrics if available
if self.resource_monitor:
resource_info = self.resource_monitor.get_plugin_metrics(plugin_id)
plugin_metrics['resources'] = resource_info
else:
plugin_metrics['resources'] = {'status': 'unknown'}
metrics[plugin_id] = plugin_metrics
return metrics
def get_plugin_state(self, plugin_id: str) -> Dict[str, Any]:
"""
Get comprehensive state information for a plugin.
Args:
plugin_id: Plugin identifier
Returns:
Dictionary with state information
"""
return self.state_manager.get_state_info(plugin_id)